2024-04-25T10:13:41.696 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T10:13:41.697 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi --owner scheduled_pdonnell@teuthology --verbose --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673066 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/test_o_trunc}} -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673066/orig.config.yaml 2024-04-25T10:13:41.762 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:13:41.886 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673066 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/test_o_trunc}} email: pdonnell@redhat.com first_in_suite: false job_id: '7673066' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKWC3AZc1fu4gs+Wvkh9log/ylrgD7wYxP2i9dmBbU8koLvXqEZhZ+Pek8S0Ht9SUcKSnsP7qQMB22CkewLUJp0= smithi077.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ9QwBvE2vXAmUUf32ziCJrgsQFb4c1aQieXOgseoq92tqf/+FTNVRp6epL2GorNrVLOcBnbuMGiAPfXKp6+17E= smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOlhnnqI9uGkoy9tL6R+DXQiapE1GrPLmpxamA5feQrl5U/NVh1ma8p8sml8761xnoVolUGqlcVNA5xfO/qVzZY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T10:13:41.886 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T10:13:41.887 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T10:13:41.887 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T10:13:41.888 INFO:teuthology.task.internal:Checking packages... 2024-04-25T10:13:41.914 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T10:13:41.914 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T10:13:41.914 INFO:teuthology.packaging:ref: None 2024-04-25T10:13:41.914 INFO:teuthology.packaging:tag: None 2024-04-25T10:13:41.914 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T10:13:41.914 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:13:41.914 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T10:13:42.120 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291.g994e2f20 2024-04-25T10:13:42.121 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T10:13:42.127 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T10:13:42.127 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T10:13:42.178 INFO:teuthology.task.internal:Saving configuration 2024-04-25T10:13:42.207 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T10:13:42.214 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T10:13:42.238 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673066', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 10:08:20.024156', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKWC3AZc1fu4gs+Wvkh9log/ylrgD7wYxP2i9dmBbU8koLvXqEZhZ+Pek8S0Ht9SUcKSnsP7qQMB22CkewLUJp0='} 2024-04-25T10:13:42.272 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi077.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673066', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 10:08:20.021328', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ9QwBvE2vXAmUUf32ziCJrgsQFb4c1aQieXOgseoq92tqf/+FTNVRp6epL2GorNrVLOcBnbuMGiAPfXKp6+17E='} 2024-04-25T10:13:42.294 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673066', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 10:08:20.022895', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOlhnnqI9uGkoy9tL6R+DXQiapE1GrPLmpxamA5feQrl5U/NVh1ma8p8sml8761xnoVolUGqlcVNA5xfO/qVzZY='} 2024-04-25T10:13:42.294 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T10:13:42.300 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T10:13:42.301 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T10:13:42.301 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T10:13:42.301 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T10:13:42.434 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4c6effa0d0>, signals=[15]) 2024-04-25T10:13:42.434 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T10:13:42.441 INFO:teuthology.task.internal:Opening connections... 2024-04-25T10:13:42.441 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T10:13:42.443 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:13:42.521 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi077.front.sepia.ceph.com 2024-04-25T10:13:42.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:13:42.596 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-04-25T10:13:42.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:13:42.672 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T10:13:42.678 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2024-04-25T10:13:42.697 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2024-04-25T10:13:42.698 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2024-04-25T10:13:42.755 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Stream" 2024-04-25T10:13:42.755 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="9" 2024-04-25T10:13:42.755 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos" 2024-04-25T10:13:42.755 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora" 2024-04-25T10:13:42.755 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="9" 2024-04-25T10:13:42.756 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el9" 2024-04-25T10:13:42.756 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T10:13:42.756 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2024-04-25T10:13:42.756 INFO:teuthology.orchestra.run.smithi043.stdout:LOGO="fedora-logo-icon" 2024-04-25T10:13:42.756 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T10:13:42.756 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://centos.org/" 2024-04-25T10:13:42.756 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T10:13:42.756 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T10:13:42.756 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T10:13:42.757 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2024-04-25T10:13:42.778 DEBUG:teuthology.orchestra.run.smithi077:> uname -m 2024-04-25T10:13:42.796 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64 2024-04-25T10:13:42.797 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release 2024-04-25T10:13:42.854 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="CentOS Stream" 2024-04-25T10:13:42.854 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="9" 2024-04-25T10:13:42.854 INFO:teuthology.orchestra.run.smithi077.stdout:ID="centos" 2024-04-25T10:13:42.854 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="rhel fedora" 2024-04-25T10:13:42.854 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="9" 2024-04-25T10:13:42.855 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el9" 2024-04-25T10:13:42.855 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T10:13:42.855 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31" 2024-04-25T10:13:42.855 INFO:teuthology.orchestra.run.smithi077.stdout:LOGO="fedora-logo-icon" 2024-04-25T10:13:42.855 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T10:13:42.855 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://centos.org/" 2024-04-25T10:13:42.855 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T10:13:42.855 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T10:13:42.855 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T10:13:42.855 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2024-04-25T10:13:42.883 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-04-25T10:13:42.901 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-04-25T10:13:42.902 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-04-25T10:13:42.959 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2024-04-25T10:13:42.960 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9" 2024-04-25T10:13:42.960 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2024-04-25T10:13:42.960 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2024-04-25T10:13:42.960 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9" 2024-04-25T10:13:42.960 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9" 2024-04-25T10:13:42.960 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T10:13:42.960 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2024-04-25T10:13:42.960 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon" 2024-04-25T10:13:42.960 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T10:13:42.960 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2024-04-25T10:13:42.960 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T10:13:42.960 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T10:13:42.960 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T10:13:42.961 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-04-25T10:13:42.978 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T10:13:42.992 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T10:13:43.049 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T10:13:43.049 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2024-04-25T10:13:43.052 DEBUG:teuthology.orchestra.run.smithi077:> test '!' -e /home/ubuntu/cephtest 2024-04-25T10:13:43.054 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-04-25T10:13:43.070 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T10:13:43.084 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T10:13:43.084 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2024-04-25T10:13:43.109 DEBUG:teuthology.orchestra.run.smithi077:> test -z $(ls -A /var/lib/ceph) 2024-04-25T10:13:43.111 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-04-25T10:13:43.218 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T10:13:43.334 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T10:13:43.351 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T10:13:43.351 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-25T10:13:43.351 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-25T10:13:43.351 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T10:13:43.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:13:43.369 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-04-25T10:13:43.427 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-437.el9.x86_64 2024-04-25T10:13:43.427 INFO:teuthology.task.kernel:Running kernel on smithi043: 5.14.0-437.el9.x86_64 2024-04-25T10:13:43.427 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum install -y kernel 2024-04-25T10:13:45.645 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-25T10:13:48.104 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-25T10:13:52.913 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 5.6 MB/s | 6.2 MB 00:01 2024-04-25T10:13:55.289 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-25T10:13:57.018 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-25T10:14:02.357 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-04-25T10:14:04.313 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T10:14:04.313 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T10:14:04.410 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-04-25T10:14:04.412 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-04-25T10:14:04.412 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-04-25T10:14:04.533 DEBUG:teuthology.orchestra.run.smithi043:> echo no | sudo yum reinstall kernel || true 2024-04-25T10:14:05.055 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 10:14:02 AM UTC. 2024-04-25T10:14:05.180 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-04-25T10:14:05.194 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-25T10:14:05.194 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-04-25T10:14:05.194 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-25T10:14:05.194 INFO:teuthology.orchestra.run.smithi043.stdout:Reinstalling: 2024-04-25T10:14:05.194 INFO:teuthology.orchestra.run.smithi043.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T10:14:05.194 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:14:05.194 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-04-25T10:14:05.194 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-25T10:14:05.194 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:14:05.194 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 255 k 2024-04-25T10:14:05.194 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 0 2024-04-25T10:14:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:Operation aborted. 2024-04-25T10:14:05.252 INFO:teuthology.orchestra.run.smithi043.stdout:Is this ok [y/N]: 2024-04-25T10:14:05.253 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum reinstall -y kernel || true 2024-04-25T10:14:05.769 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 10:14:02 AM UTC. 2024-04-25T10:14:05.890 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-04-25T10:14:05.891 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-25T10:14:05.891 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-04-25T10:14:05.891 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-25T10:14:05.891 INFO:teuthology.orchestra.run.smithi043.stdout:Reinstalling: 2024-04-25T10:14:05.891 INFO:teuthology.orchestra.run.smithi043.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T10:14:05.891 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:14:05.891 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-04-25T10:14:05.891 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-25T10:14:05.891 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:14:05.891 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 255 k 2024-04-25T10:14:05.891 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 0 2024-04-25T10:14:05.891 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-04-25T10:14:06.408 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.1 MB/s | 255 kB 00:00 2024-04-25T10:14:06.409 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:14:06.409 INFO:teuthology.orchestra.run.smithi043.stdout:Total 492 kB/s | 255 kB 00:00 2024-04-25T10:14:06.427 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-04-25T10:14:06.601 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-04-25T10:14:06.601 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-04-25T10:14:06.606 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-04-25T10:14:06.606 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-04-25T10:14:06.745 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-04-25T10:14:06.826 INFO:teuthology.orchestra.run.smithi043.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T10:14:06.826 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:14:07.512 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:14:07.512 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T10:14:08.280 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:14:08.280 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:14:08.280 INFO:teuthology.orchestra.run.smithi043.stdout:Reinstalled: 2024-04-25T10:14:08.281 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T10:14:08.281 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:14:08.281 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-04-25T10:14:08.464 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T10:14:08.513 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T10:14:08.513 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T10:14:08.513 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T10:14:08.513 DEBUG:teuthology.parallel:result is None 2024-04-25T10:14:08.514 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T10:14:08.526 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T10:14:08.526 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T10:14:08.529 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T10:14:08.531 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T10:14:08.549 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T10:14:08.569 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T10:14:08.623 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T10:14:08.624 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T10:14:08.626 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T10:14:08.628 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T10:14:08.739 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T10:14:08.748 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T10:14:08.749 DEBUG:teuthology.orchestra.run.smithi077:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T10:14:08.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:08.767 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T10:14:08.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:08.784 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T10:14:08.787 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T10:14:08.810 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T10:14:08.829 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:14:08.845 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:14:08.846 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:14:08.864 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:14:08.864 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:14:08.911 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:14:08.913 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T10:14:08.922 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T10:14:08.923 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T10:14:08.925 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T10:14:08.929 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T10:14:08.980 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T10:14:08.992 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T10:14:08.992 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T10:14:08.994 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T10:14:08.997 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T10:14:09.038 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:14:09.089 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:14:09.139 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:14:09.219 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:14:09.260 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T10:14:09.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T10:14:09.324 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:14:09.365 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:14:09.413 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:14:09.493 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:14:09.540 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-25T10:14:09.540 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T10:14:09.608 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:14:09.656 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:14:09.706 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:14:09.793 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:14:09.832 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T10:14:09.832 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T10:14:09.898 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2024-04-25T10:14:09.900 DEBUG:teuthology.orchestra.run.smithi077:> sudo service rsyslog restart 2024-04-25T10:14:09.903 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-04-25T10:14:09.937 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:14:09.940 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:14:09.983 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:14:10.396 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T10:14:10.408 INFO:teuthology.task.internal:Starting timer... 2024-04-25T10:14:10.408 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T10:14:10.518 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T10:14:10.533 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T10:14:10.533 DEBUG:teuthology.orchestra.run.smithi043:> sudo service auditd rotate 2024-04-25T10:14:10.589 INFO:teuthology.orchestra.run.smithi043.stdout:Rotating logs: 2024-04-25T10:14:10.591 DEBUG:teuthology.orchestra.run.smithi077:> sudo service auditd rotate 2024-04-25T10:14:10.665 INFO:teuthology.orchestra.run.smithi077.stdout:Rotating logs: 2024-04-25T10:14:10.667 DEBUG:teuthology.orchestra.run.smithi164:> sudo service auditd rotate 2024-04-25T10:14:10.719 INFO:teuthology.orchestra.run.smithi164.stdout:Rotating logs: 2024-04-25T10:14:10.722 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T10:14:10.722 DEBUG:teuthology.orchestra.run.smithi043:> /usr/sbin/getenforce 2024-04-25T10:14:10.740 INFO:teuthology.orchestra.run.smithi043.stdout:Permissive 2024-04-25T10:14:10.740 DEBUG:teuthology.orchestra.run.smithi077:> /usr/sbin/getenforce 2024-04-25T10:14:10.772 INFO:teuthology.orchestra.run.smithi077.stdout:Permissive 2024-04-25T10:14:10.772 DEBUG:teuthology.orchestra.run.smithi164:> /usr/sbin/getenforce 2024-04-25T10:14:10.801 INFO:teuthology.orchestra.run.smithi164.stdout:Permissive 2024-04-25T10:14:10.801 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi043.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi077.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi164.front.sepia.ceph.com': 'permissive'} 2024-04-25T10:14:10.801 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T10:14:10.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:10.828 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T10:14:10.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:10.854 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T10:14:10.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:10.881 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T10:14:10.881 DEBUG:teuthology.orchestra.run.smithi043:> sudo /usr/sbin/setenforce permissive 2024-04-25T10:14:10.905 DEBUG:teuthology.orchestra.run.smithi077:> sudo /usr/sbin/setenforce permissive 2024-04-25T10:14:10.930 DEBUG:teuthology.orchestra.run.smithi164:> sudo /usr/sbin/setenforce permissive 2024-04-25T10:14:10.955 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T10:14:10.963 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T10:14:10.981 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T10:14:10.983 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi043.front.sepia.ceph.com,smithi077.front.sepia.ceph.com,smithi164.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T10:19:23.957 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi043.front.sepia.ceph.com'), Remote(name='ubuntu@smithi077.front.sepia.ceph.com'), Remote(name='ubuntu@smithi164.front.sepia.ceph.com')] 2024-04-25T10:19:23.958 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-04-25T10:19:23.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:19:24.036 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-04-25T10:19:24.117 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-04-25T10:19:24.118 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-04-25T10:19:24.118 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:19:24.190 DEBUG:teuthology.orchestra.run.smithi077:> true 2024-04-25T10:19:24.265 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-04-25T10:19:24.265 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-25T10:19:24.265 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:19:24.339 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-04-25T10:19:24.417 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-25T10:19:24.418 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T10:19:24.429 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T10:19:24.429 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:19:24.429 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:19:24.432 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:19:24.432 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:19:24.434 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:19:24.434 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:19:24.459 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T10:19:24.461 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T10:19:24.472 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T10:19:24.476 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T10:19:24.486 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T10:19:24.494 INFO:teuthology.orchestra.run.smithi077.stderr:sudo: ntpd: command not found 2024-04-25T10:19:24.500 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T10:19:24.501 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: ntpd: command not found 2024-04-25T10:19:24.505 INFO:teuthology.orchestra.run.smithi077.stdout:506 Cannot talk to daemon 2024-04-25T10:19:24.513 INFO:teuthology.orchestra.run.smithi043.stdout:506 Cannot talk to daemon 2024-04-25T10:19:24.518 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T10:19:24.523 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: ntpd: command not found 2024-04-25T10:19:24.527 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T10:19:24.530 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T10:19:24.533 INFO:teuthology.orchestra.run.smithi164.stdout:506 Cannot talk to daemon 2024-04-25T10:19:24.541 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T10:19:24.547 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T10:19:24.560 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T10:19:24.583 INFO:teuthology.orchestra.run.smithi077.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:19:24.586 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:19:24.586 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-04-25T10:19:24.586 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:19:24.586 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:19:24.586 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:19:24.586 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:19:24.596 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:19:24.599 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:19:24.600 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-04-25T10:19:24.600 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:19:24.600 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:19:24.600 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:19:24.600 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:19:24.622 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:19:24.625 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:19:24.625 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-04-25T10:19:24.626 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:19:24.626 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:19:24.626 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:19:24.626 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:19:24.627 INFO:teuthology.run_tasks:Running task install... 2024-04-25T10:19:24.635 DEBUG:teuthology.task.install:project ceph 2024-04-25T10:19:24.635 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T10:19:24.635 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T10:19:24.635 INFO:teuthology.task.install:Using flavor: default 2024-04-25T10:19:24.640 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T10:19:24.641 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T10:19:24.641 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T10:19:24.641 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:19:24.643 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T10:19:24.643 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:19:24.644 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T10:19:24.645 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:19:24.782 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T10:19:24.782 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T10:19:24.794 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T10:19:24.795 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T10:19:24.797 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T10:19:24.797 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T10:19:24.895 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T10:19:24.895 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T10:19:24.895 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T10:19:24.922 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T10:19:24.922 DEBUG:teuthology.orchestra.run.smithi043:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T10:19:24.988 DEBUG:teuthology.orchestra.run.smithi043:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T10:19:25.015 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T10:19:25.015 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-25T10:19:25.015 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T10:19:25.025 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T10:19:25.025 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T10:19:25.025 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T10:19:25.039 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T10:19:25.039 DEBUG:teuthology.orchestra.run.smithi077:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T10:19:25.050 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T10:19:25.050 DEBUG:teuthology.orchestra.run.smithi164:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T10:19:25.063 DEBUG:teuthology.orchestra.run.smithi043:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T10:19:25.103 DEBUG:teuthology.orchestra.run.smithi077:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T10:19:25.115 DEBUG:teuthology.orchestra.run.smithi164:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T10:19:25.127 INFO:teuthology.orchestra.run.smithi043.stdout:check_obsoletes = 1 2024-04-25T10:19:25.129 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum clean all 2024-04-25T10:19:25.176 DEBUG:teuthology.orchestra.run.smithi077:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T10:19:25.188 DEBUG:teuthology.orchestra.run.smithi164:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T10:19:25.239 INFO:teuthology.orchestra.run.smithi077.stdout:check_obsoletes = 1 2024-04-25T10:19:25.240 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum clean all 2024-04-25T10:19:25.251 INFO:teuthology.orchestra.run.smithi164.stdout:check_obsoletes = 1 2024-04-25T10:19:25.252 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum clean all 2024-04-25T10:19:25.363 INFO:teuthology.orchestra.run.smithi043.stdout:48 files removed 2024-04-25T10:19:25.390 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T10:19:25.466 INFO:teuthology.orchestra.run.smithi077.stdout:48 files removed 2024-04-25T10:19:25.476 INFO:teuthology.orchestra.run.smithi164.stdout:48 files removed 2024-04-25T10:19:25.493 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T10:19:25.503 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T10:19:26.134 INFO:teuthology.orchestra.run.smithi043.stdout:ceph packages for x86_64 178 kB/s | 84 kB 00:00 2024-04-25T10:19:26.331 INFO:teuthology.orchestra.run.smithi077.stdout:ceph packages for x86_64 144 kB/s | 84 kB 00:00 2024-04-25T10:19:26.355 INFO:teuthology.orchestra.run.smithi164.stdout:ceph packages for x86_64 141 kB/s | 84 kB 00:00 2024-04-25T10:19:26.743 INFO:teuthology.orchestra.run.smithi043.stdout:ceph noarch packages 21 kB/s | 12 kB 00:00 2024-04-25T10:19:26.839 INFO:teuthology.orchestra.run.smithi164.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-04-25T10:19:26.932 INFO:teuthology.orchestra.run.smithi077.stdout:ceph noarch packages 21 kB/s | 12 kB 00:00 2024-04-25T10:19:27.189 INFO:teuthology.orchestra.run.smithi164.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-04-25T10:19:27.210 INFO:teuthology.orchestra.run.smithi043.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-04-25T10:19:27.282 INFO:teuthology.orchestra.run.smithi077.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-04-25T10:19:27.965 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T10:19:28.111 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 9.3 MB/s | 8.0 MB 00:00 2024-04-25T10:19:28.466 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - BaseOS 7.0 MB/s | 8.0 MB 00:01 2024-04-25T10:19:30.120 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T10:19:30.292 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-25T10:19:31.499 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 7.7 MB/s | 19 MB 00:02 2024-04-25T10:19:34.971 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 5.5 MB/s | 6.2 MB 00:01 2024-04-25T10:19:35.443 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.2 MB 00:01 2024-04-25T10:19:36.358 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.2 MB 00:01 2024-04-25T10:19:36.613 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-25T10:19:36.853 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 16 kB 00:00 2024-04-25T10:19:37.707 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-25T10:19:38.196 INFO:teuthology.orchestra.run.smithi077.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-25T10:19:38.625 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-25T10:19:40.179 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-25T10:19:42.965 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-25T10:19:43.237 INFO:teuthology.orchestra.run.smithi077.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-25T10:19:45.194 INFO:teuthology.orchestra.run.smithi043.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:19:45.194 INFO:teuthology.orchestra.run.smithi043.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:19:45.204 INFO:teuthology.orchestra.run.smithi043.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T10:19:45.204 INFO:teuthology.orchestra.run.smithi043.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T10:19:45.205 INFO:teuthology.orchestra.run.smithi043.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T10:19:45.206 INFO:teuthology.orchestra.run.smithi043.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T10:19:45.208 INFO:teuthology.orchestra.run.smithi043.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T10:19:45.210 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T10:19:45.211 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T10:19:45.276 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-04-25T10:19:45.286 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-04-25T10:19:45.286 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-04-25T10:19:45.286 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-04-25T10:19:45.286 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-04-25T10:19:45.286 INFO:teuthology.orchestra.run.smithi043.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T10:19:45.286 INFO:teuthology.orchestra.run.smithi043.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T10:19:45.286 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T10:19:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T10:19:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T10:19:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T10:19:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T10:19:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T10:19:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T10:19:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T10:19:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T10:19:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T10:19:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T10:19:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T10:19:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T10:19:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T10:19:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T10:19:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T10:19:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T10:19:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T10:19:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T10:19:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T10:19:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T10:19:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T10:19:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T10:19:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T10:19:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T10:19:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T10:19:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T10:19:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T10:19:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T10:19:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T10:19:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T10:19:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T10:19:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T10:19:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T10:19:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T10:19:45.289 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2024-04-25T10:19:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T10:19:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T10:19:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T10:19:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T10:19:45.289 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-04-25T10:19:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T10:19:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T10:19:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T10:19:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T10:19:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T10:19:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T10:19:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T10:19:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T10:19:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T10:19:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T10:19:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T10:19:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T10:19:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T10:19:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T10:19:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T10:19:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T10:19:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T10:19:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T10:19:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T10:19:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T10:19:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T10:19:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T10:19:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T10:19:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T10:19:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T10:19:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T10:19:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T10:19:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T10:19:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T10:19:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T10:19:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T10:19:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T10:19:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T10:19:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T10:19:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T10:19:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T10:19:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T10:19:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T10:19:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T10:19:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T10:19:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T10:19:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T10:19:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T10:19:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T10:19:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T10:19:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T10:19:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T10:19:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T10:19:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T10:19:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T10:19:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T10:19:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T10:19:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T10:19:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T10:19:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T10:19:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T10:19:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T10:19:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T10:19:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T10:19:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T10:19:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T10:19:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T10:19:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T10:19:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T10:19:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T10:19:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T10:19:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T10:19:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T10:19:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T10:19:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T10:19:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T10:19:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T10:19:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T10:19:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T10:19:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T10:19:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T10:19:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T10:19:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T10:19:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T10:19:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T10:19:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T10:19:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T10:19:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T10:19:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T10:19:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T10:19:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T10:19:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T10:19:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T10:19:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T10:19:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T10:19:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T10:19:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T10:19:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T10:19:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T10:19:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T10:19:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T10:19:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T10:19:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T10:19:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T10:19:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T10:19:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T10:19:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T10:19:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T10:19:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T10:19:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T10:19:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T10:19:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T10:19:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T10:19:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T10:19:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T10:19:45.297 INFO:teuthology.orchestra.run.smithi043.stdout:Installing weak dependencies: 2024-04-25T10:19:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T10:19:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:19:45.298 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-04-25T10:19:45.298 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-04-25T10:19:45.298 INFO:teuthology.orchestra.run.smithi043.stdout:Install 148 Packages 2024-04-25T10:19:45.298 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 4 Packages 2024-04-25T10:19:45.298 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:19:45.298 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 214 M 2024-04-25T10:19:45.298 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-04-25T10:19:45.301 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-04-25T10:19:45.372 INFO:teuthology.orchestra.run.smithi077.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:19:45.372 INFO:teuthology.orchestra.run.smithi077.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:19:45.383 INFO:teuthology.orchestra.run.smithi077.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T10:19:45.384 INFO:teuthology.orchestra.run.smithi077.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T10:19:45.385 INFO:teuthology.orchestra.run.smithi077.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T10:19:45.386 INFO:teuthology.orchestra.run.smithi077.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T10:19:45.387 INFO:teuthology.orchestra.run.smithi077.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T10:19:45.389 INFO:teuthology.orchestra.run.smithi077.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T10:19:45.390 INFO:teuthology.orchestra.run.smithi077.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T10:19:45.451 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-25T10:19:45.477 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================= 2024-04-25T10:19:45.477 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2024-04-25T10:19:45.477 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================= 2024-04-25T10:19:45.477 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-04-25T10:19:45.477 INFO:teuthology.orchestra.run.smithi077.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T10:19:45.478 INFO:teuthology.orchestra.run.smithi077.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T10:19:45.478 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T10:19:45.478 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T10:19:45.478 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T10:19:45.478 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T10:19:45.478 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T10:19:45.478 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T10:19:45.478 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T10:19:45.478 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T10:19:45.478 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T10:19:45.478 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T10:19:45.478 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T10:19:45.478 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T10:19:45.479 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T10:19:45.479 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T10:19:45.479 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T10:19:45.479 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T10:19:45.479 INFO:teuthology.orchestra.run.smithi077.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T10:19:45.479 INFO:teuthology.orchestra.run.smithi077.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T10:19:45.479 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T10:19:45.479 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T10:19:45.479 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T10:19:45.479 INFO:teuthology.orchestra.run.smithi077.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T10:19:45.479 INFO:teuthology.orchestra.run.smithi077.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T10:19:45.479 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T10:19:45.479 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T10:19:45.479 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T10:19:45.480 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T10:19:45.480 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T10:19:45.480 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T10:19:45.480 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T10:19:45.480 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T10:19:45.480 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T10:19:45.480 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T10:19:45.480 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T10:19:45.480 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T10:19:45.480 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrading: 2024-04-25T10:19:45.480 INFO:teuthology.orchestra.run.smithi077.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T10:19:45.480 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T10:19:45.480 INFO:teuthology.orchestra.run.smithi077.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T10:19:45.480 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T10:19:45.480 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-04-25T10:19:45.481 INFO:teuthology.orchestra.run.smithi077.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T10:19:45.481 INFO:teuthology.orchestra.run.smithi077.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T10:19:45.481 INFO:teuthology.orchestra.run.smithi077.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T10:19:45.481 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T10:19:45.481 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T10:19:45.481 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T10:19:45.481 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T10:19:45.481 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T10:19:45.481 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T10:19:45.481 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T10:19:45.481 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T10:19:45.481 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T10:19:45.481 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T10:19:45.481 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T10:19:45.481 INFO:teuthology.orchestra.run.smithi077.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T10:19:45.482 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T10:19:45.482 INFO:teuthology.orchestra.run.smithi077.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T10:19:45.482 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T10:19:45.482 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T10:19:45.482 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T10:19:45.482 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T10:19:45.482 INFO:teuthology.orchestra.run.smithi077.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T10:19:45.482 INFO:teuthology.orchestra.run.smithi077.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T10:19:45.482 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T10:19:45.482 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T10:19:45.482 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T10:19:45.482 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T10:19:45.482 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T10:19:45.483 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T10:19:45.483 INFO:teuthology.orchestra.run.smithi077.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T10:19:45.483 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T10:19:45.483 INFO:teuthology.orchestra.run.smithi077.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T10:19:45.483 INFO:teuthology.orchestra.run.smithi077.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T10:19:45.483 INFO:teuthology.orchestra.run.smithi077.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T10:19:45.483 INFO:teuthology.orchestra.run.smithi077.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T10:19:45.483 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T10:19:45.483 INFO:teuthology.orchestra.run.smithi077.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T10:19:45.483 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T10:19:45.483 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T10:19:45.483 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T10:19:45.483 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T10:19:45.483 INFO:teuthology.orchestra.run.smithi077.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T10:19:45.484 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T10:19:45.484 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T10:19:45.484 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T10:19:45.484 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T10:19:45.484 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T10:19:45.484 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T10:19:45.484 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T10:19:45.484 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T10:19:45.484 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T10:19:45.484 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T10:19:45.484 INFO:teuthology.orchestra.run.smithi077.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T10:19:45.484 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T10:19:45.484 INFO:teuthology.orchestra.run.smithi077.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T10:19:45.485 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T10:19:45.485 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T10:19:45.485 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T10:19:45.485 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T10:19:45.485 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T10:19:45.485 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T10:19:45.485 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T10:19:45.485 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T10:19:45.485 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T10:19:45.485 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T10:19:45.485 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T10:19:45.485 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T10:19:45.485 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T10:19:45.485 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T10:19:45.485 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T10:19:45.486 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T10:19:45.486 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T10:19:45.486 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T10:19:45.486 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T10:19:45.486 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T10:19:45.486 INFO:teuthology.orchestra.run.smithi077.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T10:19:45.486 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T10:19:45.486 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T10:19:45.486 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T10:19:45.486 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T10:19:45.486 INFO:teuthology.orchestra.run.smithi077.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T10:19:45.486 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T10:19:45.486 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T10:19:45.487 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T10:19:45.487 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T10:19:45.487 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T10:19:45.487 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T10:19:45.487 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T10:19:45.487 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T10:19:45.487 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T10:19:45.487 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T10:19:45.487 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T10:19:45.487 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T10:19:45.487 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T10:19:45.487 INFO:teuthology.orchestra.run.smithi077.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T10:19:45.487 INFO:teuthology.orchestra.run.smithi077.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T10:19:45.487 INFO:teuthology.orchestra.run.smithi077.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T10:19:45.488 INFO:teuthology.orchestra.run.smithi077.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T10:19:45.488 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T10:19:45.488 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T10:19:45.488 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T10:19:45.488 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T10:19:45.488 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T10:19:45.488 INFO:teuthology.orchestra.run.smithi077.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T10:19:45.488 INFO:teuthology.orchestra.run.smithi077.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T10:19:45.488 INFO:teuthology.orchestra.run.smithi077.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T10:19:45.488 INFO:teuthology.orchestra.run.smithi077.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T10:19:45.488 INFO:teuthology.orchestra.run.smithi077.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T10:19:45.488 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T10:19:45.488 INFO:teuthology.orchestra.run.smithi077.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T10:19:45.488 INFO:teuthology.orchestra.run.smithi077.stdout:Installing weak dependencies: 2024-04-25T10:19:45.488 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T10:19:45.489 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-25T10:19:45.489 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-04-25T10:19:45.489 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================= 2024-04-25T10:19:45.489 INFO:teuthology.orchestra.run.smithi077.stdout:Install 148 Packages 2024-04-25T10:19:45.489 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrade 4 Packages 2024-04-25T10:19:45.489 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-25T10:19:45.489 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 214 M 2024-04-25T10:19:45.489 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-04-25T10:19:46.616 INFO:teuthology.orchestra.run.smithi043.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-04-25T10:19:46.814 INFO:teuthology.orchestra.run.smithi077.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-04-25T10:19:47.216 INFO:teuthology.orchestra.run.smithi043.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 1.4 MB/s | 863 kB 00:00 2024-04-25T10:19:47.400 INFO:teuthology.orchestra.run.smithi043.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 802 kB/s | 147 kB 00:00 2024-04-25T10:19:47.419 INFO:teuthology.orchestra.run.smithi164.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:19:47.420 INFO:teuthology.orchestra.run.smithi164.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:19:47.430 INFO:teuthology.orchestra.run.smithi164.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T10:19:47.431 INFO:teuthology.orchestra.run.smithi164.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T10:19:47.432 INFO:teuthology.orchestra.run.smithi164.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T10:19:47.432 INFO:teuthology.orchestra.run.smithi164.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T10:19:47.434 INFO:teuthology.orchestra.run.smithi164.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T10:19:47.436 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T10:19:47.437 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T10:19:47.494 INFO:teuthology.orchestra.run.smithi043.stdout:(4/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 4.8 MB/s | 5.4 MB 00:01 2024-04-25T10:19:47.497 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-25T10:19:47.516 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-04-25T10:19:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-04-25T10:19:47.516 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-04-25T10:19:47.516 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-25T10:19:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T10:19:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T10:19:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T10:19:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T10:19:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T10:19:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T10:19:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T10:19:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T10:19:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T10:19:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T10:19:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T10:19:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T10:19:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T10:19:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T10:19:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T10:19:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T10:19:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T10:19:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T10:19:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T10:19:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T10:19:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T10:19:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T10:19:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T10:19:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T10:19:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T10:19:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T10:19:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T10:19:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T10:19:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T10:19:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T10:19:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T10:19:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T10:19:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T10:19:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T10:19:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T10:19:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T10:19:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T10:19:47.519 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrading: 2024-04-25T10:19:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T10:19:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T10:19:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T10:19:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T10:19:47.519 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-04-25T10:19:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T10:19:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T10:19:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T10:19:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T10:19:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T10:19:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T10:19:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T10:19:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T10:19:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T10:19:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T10:19:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T10:19:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T10:19:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T10:19:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T10:19:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T10:19:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T10:19:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T10:19:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T10:19:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T10:19:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T10:19:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T10:19:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T10:19:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T10:19:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T10:19:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T10:19:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T10:19:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T10:19:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T10:19:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T10:19:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T10:19:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T10:19:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T10:19:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T10:19:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T10:19:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T10:19:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T10:19:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T10:19:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T10:19:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T10:19:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T10:19:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T10:19:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T10:19:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T10:19:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T10:19:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T10:19:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T10:19:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T10:19:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T10:19:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T10:19:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T10:19:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T10:19:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T10:19:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T10:19:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T10:19:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T10:19:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T10:19:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T10:19:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T10:19:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T10:19:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T10:19:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T10:19:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T10:19:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T10:19:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T10:19:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T10:19:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T10:19:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T10:19:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T10:19:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T10:19:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T10:19:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T10:19:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T10:19:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T10:19:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T10:19:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T10:19:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T10:19:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T10:19:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T10:19:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T10:19:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T10:19:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T10:19:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T10:19:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T10:19:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T10:19:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T10:19:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T10:19:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T10:19:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T10:19:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T10:19:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T10:19:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T10:19:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T10:19:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T10:19:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T10:19:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T10:19:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T10:19:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T10:19:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T10:19:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T10:19:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T10:19:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T10:19:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T10:19:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T10:19:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T10:19:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T10:19:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T10:19:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T10:19:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T10:19:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T10:19:47.527 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T10:19:47.527 INFO:teuthology.orchestra.run.smithi164.stdout:Installing weak dependencies: 2024-04-25T10:19:47.527 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T10:19:47.527 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T10:19:47.527 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-25T10:19:47.527 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-04-25T10:19:47.527 INFO:teuthology.orchestra.run.smithi164.stdout:Install 148 Packages 2024-04-25T10:19:47.527 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrade 4 Packages 2024-04-25T10:19:47.527 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T10:19:47.528 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 214 M 2024-04-25T10:19:47.528 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-25T10:19:47.572 INFO:teuthology.orchestra.run.smithi077.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 1.1 MB/s | 863 kB 00:00 2024-04-25T10:19:47.764 INFO:teuthology.orchestra.run.smithi077.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 767 kB/s | 147 kB 00:00 2024-04-25T10:19:47.784 INFO:teuthology.orchestra.run.smithi043.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 5.2 MB/s | 1.5 MB 00:00 2024-04-25T10:19:48.109 INFO:teuthology.orchestra.run.smithi043.stdout:(6/152): ceph-common-19.0.0-3291.g994e2f20.el9. 11 MB/s | 20 MB 00:01 2024-04-25T10:19:48.468 INFO:teuthology.orchestra.run.smithi043.stdout:(7/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 2.1 MB/s | 2.3 MB 00:01 2024-04-25T10:19:48.534 INFO:teuthology.orchestra.run.smithi043.stdout:(8/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 6.5 MB/s | 4.9 MB 00:00 2024-04-25T10:19:48.635 INFO:teuthology.orchestra.run.smithi043.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 252 kB/s | 25 kB 00:00 2024-04-25T10:19:48.698 INFO:teuthology.orchestra.run.smithi077.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 2.4 MB/s | 2.3 MB 00:00 2024-04-25T10:19:48.978 INFO:teuthology.orchestra.run.smithi164.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-04-25T10:19:49.515 INFO:teuthology.orchestra.run.smithi077.stdout:(5/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 1.8 MB/s | 5.4 MB 00:02 2024-04-25T10:19:49.556 INFO:teuthology.orchestra.run.smithi077.stdout:(6/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 1.8 MB/s | 1.5 MB 00:00 2024-04-25T10:19:50.020 INFO:teuthology.orchestra.run.smithi164.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 828 kB/s | 863 kB 00:01 2024-04-25T10:19:50.253 INFO:teuthology.orchestra.run.smithi164.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 630 kB/s | 147 kB 00:00 2024-04-25T10:19:52.235 INFO:teuthology.orchestra.run.smithi043.stdout:(10/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 4.3 MB/s | 18 MB 00:04 2024-04-25T10:19:52.362 INFO:teuthology.orchestra.run.smithi164.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 1.1 MB/s | 2.3 MB 00:02 2024-04-25T10:19:52.410 INFO:teuthology.orchestra.run.smithi043.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 1.3 MB/s | 224 kB 00:00 2024-04-25T10:19:52.510 INFO:teuthology.orchestra.run.smithi043.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 271 kB/s | 27 kB 00:00 2024-04-25T10:19:52.610 INFO:teuthology.orchestra.run.smithi043.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 317 kB/s | 32 kB 00:00 2024-04-25T10:19:52.882 INFO:teuthology.orchestra.run.smithi077.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 1.4 MB/s | 4.9 MB 00:03 2024-04-25T10:19:52.944 INFO:teuthology.orchestra.run.smithi043.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 2.2 MB/s | 756 kB 00:00 2024-04-25T10:19:53.044 INFO:teuthology.orchestra.run.smithi043.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.5 MB/s | 158 kB 00:00 2024-04-25T10:19:53.144 INFO:teuthology.orchestra.run.smithi043.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.2 MB/s | 127 kB 00:00 2024-04-25T10:19:53.470 INFO:teuthology.orchestra.run.smithi043.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 1.5 MB/s | 497 kB 00:00 2024-04-25T10:19:53.679 INFO:teuthology.orchestra.run.smithi043.stdout:(18/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 1.9 MB/s | 10 MB 00:05 2024-04-25T10:19:53.686 INFO:teuthology.orchestra.run.smithi164.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 1.1 MB/s | 1.5 MB 00:01 2024-04-25T10:19:53.763 INFO:teuthology.orchestra.run.smithi043.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 543 kB/s | 45 kB 00:00 2024-04-25T10:19:53.846 INFO:teuthology.orchestra.run.smithi043.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.6 MB/s | 135 kB 00:00 2024-04-25T10:19:53.980 INFO:teuthology.orchestra.run.smithi043.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.2 MB/s | 161 kB 00:00 2024-04-25T10:19:54.188 INFO:teuthology.orchestra.run.smithi043.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 1.5 MB/s | 321 kB 00:00 2024-04-25T10:19:54.364 INFO:teuthology.orchestra.run.smithi043.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 1.7 MB/s | 298 kB 00:00 2024-04-25T10:19:54.464 INFO:teuthology.orchestra.run.smithi043.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 998 kB/s | 100 kB 00:00 2024-04-25T10:19:54.564 INFO:teuthology.orchestra.run.smithi043.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 865 kB/s | 86 kB 00:00 2024-04-25T10:19:55.765 INFO:teuthology.orchestra.run.smithi043.stdout:(26/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 2.2 MB/s | 5.2 MB 00:02 2024-04-25T10:19:55.865 INFO:teuthology.orchestra.run.smithi043.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 1.7 MB/s | 169 kB 00:00 2024-04-25T10:19:55.965 INFO:teuthology.orchestra.run.smithi043.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 342 kB/s | 34 kB 00:00 2024-04-25T10:19:56.065 INFO:teuthology.orchestra.run.smithi043.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.4 MB/s | 142 kB 00:00 2024-04-25T10:19:56.207 INFO:teuthology.orchestra.run.smithi043.stdout:(30/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 1.9 MB/s | 3.0 MB 00:01 2024-04-25T10:19:56.386 INFO:teuthology.orchestra.run.smithi164.stdout:(6/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 699 kB/s | 5.4 MB 00:07 2024-04-25T10:19:57.173 INFO:teuthology.orchestra.run.smithi077.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 1.8 MB/s | 20 MB 00:10 2024-04-25T10:19:57.265 INFO:teuthology.orchestra.run.smithi077.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 275 kB/s | 25 kB 00:00 2024-04-25T10:19:57.524 INFO:teuthology.orchestra.run.smithi043.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 2.4 MB/s | 3.5 MB 00:01 2024-04-25T10:19:57.683 INFO:teuthology.orchestra.run.smithi043.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 1.5 MB/s | 249 kB 00:00 2024-04-25T10:19:57.783 INFO:teuthology.orchestra.run.smithi043.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 494 kB/s | 49 kB 00:00 2024-04-25T10:19:57.795 INFO:teuthology.orchestra.run.smithi164.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 1.2 MB/s | 4.9 MB 00:04 2024-04-25T10:19:57.875 INFO:teuthology.orchestra.run.smithi043.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 179 kB/s | 16 kB 00:00 2024-04-25T10:19:58.133 INFO:teuthology.orchestra.run.smithi043.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 1.0 MB/s | 273 kB 00:00 2024-04-25T10:19:58.515 INFO:teuthology.orchestra.run.smithi077.stdout:(10/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 1.8 MB/s | 10 MB 00:05 2024-04-25T10:19:58.600 INFO:teuthology.orchestra.run.smithi043.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 1.6 MB/s | 762 kB 00:00 2024-04-25T10:19:58.684 INFO:teuthology.orchestra.run.smithi043.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 219 kB/s | 18 kB 00:00 2024-04-25T10:19:58.732 INFO:teuthology.orchestra.run.smithi077.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 1.0 MB/s | 224 kB 00:00 2024-04-25T10:19:58.866 INFO:teuthology.orchestra.run.smithi077.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 203 kB/s | 27 kB 00:00 2024-04-25T10:19:58.983 INFO:teuthology.orchestra.run.smithi077.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 272 kB/s | 32 kB 00:00 2024-04-25T10:19:59.034 INFO:teuthology.orchestra.run.smithi043.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 230 kB/s | 80 kB 00:00 2024-04-25T10:19:59.442 INFO:teuthology.orchestra.run.smithi077.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 1.6 MB/s | 756 kB 00:00 2024-04-25T10:19:59.476 INFO:teuthology.orchestra.run.smithi043.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 803 kB 00:00 2024-04-25T10:19:59.558 INFO:teuthology.orchestra.run.smithi077.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.3 MB/s | 158 kB 00:00 2024-04-25T10:19:59.593 INFO:teuthology.orchestra.run.smithi043.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-04-25T10:19:59.675 INFO:teuthology.orchestra.run.smithi077.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.1 MB/s | 127 kB 00:00 2024-04-25T10:19:59.777 INFO:teuthology.orchestra.run.smithi043.stdout:(41/152): ceph-mgr-diskprediction-local-19.0.0- 2.1 MB/s | 7.4 MB 00:03 2024-04-25T10:19:59.802 INFO:teuthology.orchestra.run.smithi043.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-04-25T10:19:59.863 INFO:teuthology.orchestra.run.smithi043.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.7 MB/s | 106 kB 00:00 2024-04-25T10:19:59.892 INFO:teuthology.orchestra.run.smithi077.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 2.2 MB/s | 497 kB 00:00 2024-04-25T10:19:59.921 INFO:teuthology.orchestra.run.smithi043.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T10:19:59.980 INFO:teuthology.orchestra.run.smithi043.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-25T10:20:00.072 INFO:teuthology.orchestra.run.smithi043.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-25T10:20:00.147 INFO:teuthology.orchestra.run.smithi043.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 3.3 MB/s | 1.2 MB 00:00 2024-04-25T10:20:00.348 INFO:teuthology.orchestra.run.smithi043.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-04-25T10:20:00.373 INFO:teuthology.orchestra.run.smithi043.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-04-25T10:20:00.423 INFO:teuthology.orchestra.run.smithi043.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 431 kB/s | 21 kB 00:00 2024-04-25T10:20:00.448 INFO:teuthology.orchestra.run.smithi043.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-04-25T10:20:00.498 INFO:teuthology.orchestra.run.smithi043.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-04-25T10:20:00.532 INFO:teuthology.orchestra.run.smithi043.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-04-25T10:20:00.582 INFO:teuthology.orchestra.run.smithi043.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-04-25T10:20:00.624 INFO:teuthology.orchestra.run.smithi043.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-04-25T10:20:00.708 INFO:teuthology.orchestra.run.smithi043.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-04-25T10:20:00.742 INFO:teuthology.orchestra.run.smithi043.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-04-25T10:20:00.792 INFO:teuthology.orchestra.run.smithi043.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 541 kB/s | 45 kB 00:00 2024-04-25T10:20:00.859 INFO:teuthology.orchestra.run.smithi043.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-04-25T10:20:00.909 INFO:teuthology.orchestra.run.smithi043.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 582 kB 00:00 2024-04-25T10:20:00.934 INFO:teuthology.orchestra.run.smithi043.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 339 kB/s | 25 kB 00:00 2024-04-25T10:20:00.984 INFO:teuthology.orchestra.run.smithi043.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 632 kB/s | 47 kB 00:00 2024-04-25T10:20:01.035 INFO:teuthology.orchestra.run.smithi043.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-04-25T10:20:01.077 INFO:teuthology.orchestra.run.smithi043.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-25T10:20:01.102 INFO:teuthology.orchestra.run.smithi043.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 527 kB/s | 35 kB 00:00 2024-04-25T10:20:01.303 INFO:teuthology.orchestra.run.smithi043.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-04-25T10:20:01.370 INFO:teuthology.orchestra.run.smithi043.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-25T10:20:01.470 INFO:teuthology.orchestra.run.smithi043.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-04-25T10:20:01.503 INFO:teuthology.orchestra.run.smithi043.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-04-25T10:20:01.562 INFO:teuthology.orchestra.run.smithi043.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-25T10:20:01.712 INFO:teuthology.orchestra.run.smithi043.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-04-25T10:20:01.829 INFO:teuthology.orchestra.run.smithi043.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 18 MB/s | 5.8 MB 00:00 2024-04-25T10:20:01.855 INFO:teuthology.orchestra.run.smithi043.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 616 kB/s | 87 kB 00:00 2024-04-25T10:20:01.868 INFO:teuthology.orchestra.run.smithi077.stdout:(18/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 2.6 MB/s | 5.2 MB 00:01 2024-04-25T10:20:01.869 INFO:teuthology.orchestra.run.smithi164.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 1.5 MB/s | 20 MB 00:13 2024-04-25T10:20:01.930 INFO:teuthology.orchestra.run.smithi043.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-25T10:20:01.953 INFO:teuthology.orchestra.run.smithi164.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 303 kB/s | 25 kB 00:00 2024-04-25T10:20:01.960 INFO:teuthology.orchestra.run.smithi077.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 493 kB/s | 45 kB 00:00 2024-04-25T10:20:02.005 INFO:teuthology.orchestra.run.smithi043.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-04-25T10:20:02.055 INFO:teuthology.orchestra.run.smithi043.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-04-25T10:20:02.060 INFO:teuthology.orchestra.run.smithi077.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.3 MB/s | 135 kB 00:00 2024-04-25T10:20:02.139 INFO:teuthology.orchestra.run.smithi043.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 6.0 MB 00:00 2024-04-25T10:20:02.168 INFO:teuthology.orchestra.run.smithi077.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.5 MB/s | 161 kB 00:00 2024-04-25T10:20:02.189 INFO:teuthology.orchestra.run.smithi043.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 143 kB/s | 19 kB 00:00 2024-04-25T10:20:02.240 INFO:teuthology.orchestra.run.smithi043.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-04-25T10:20:02.256 INFO:teuthology.orchestra.run.smithi043.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 521 kB/s | 35 kB 00:00 2024-04-25T10:20:02.332 INFO:teuthology.orchestra.run.smithi043.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-25T10:20:02.336 INFO:teuthology.orchestra.run.smithi077.stdout:(22/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 1.4 MB/s | 18 MB 00:12 2024-04-25T10:20:02.369 INFO:teuthology.orchestra.run.smithi077.stdout:(23/152): python3-rados-19.0.0-3291.g994e2f20.e 1.6 MB/s | 321 kB 00:00 2024-04-25T10:20:02.407 INFO:teuthology.orchestra.run.smithi043.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-04-25T10:20:02.478 INFO:teuthology.orchestra.run.smithi077.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 920 kB/s | 100 kB 00:00 2024-04-25T10:20:02.491 INFO:teuthology.orchestra.run.smithi043.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-04-25T10:20:02.516 INFO:teuthology.orchestra.run.smithi043.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 711 kB/s | 77 kB 00:00 2024-04-25T10:20:02.519 INFO:teuthology.orchestra.run.smithi077.stdout:(25/152): python3-rbd-19.0.0-3291.g994e2f20.el9 1.6 MB/s | 298 kB 00:00 2024-04-25T10:20:02.558 INFO:teuthology.orchestra.run.smithi043.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 632 kB/s | 42 kB 00:00 2024-04-25T10:20:02.578 INFO:teuthology.orchestra.run.smithi077.stdout:(26/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 861 kB/s | 86 kB 00:00 2024-04-25T10:20:02.608 INFO:teuthology.orchestra.run.smithi043.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-04-25T10:20:02.642 INFO:teuthology.orchestra.run.smithi043.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-04-25T10:20:02.687 INFO:teuthology.orchestra.run.smithi077.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 1.5 MB/s | 169 kB 00:00 2024-04-25T10:20:02.701 INFO:teuthology.orchestra.run.smithi043.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 872 kB/s | 51 kB 00:00 2024-04-25T10:20:02.726 INFO:teuthology.orchestra.run.smithi043.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-04-25T10:20:02.776 INFO:teuthology.orchestra.run.smithi043.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 712 kB/s | 54 kB 00:00 2024-04-25T10:20:02.787 INFO:teuthology.orchestra.run.smithi077.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 342 kB/s | 34 kB 00:00 2024-04-25T10:20:02.835 INFO:teuthology.orchestra.run.smithi043.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 715 kB/s | 42 kB 00:00 2024-04-25T10:20:02.887 INFO:teuthology.orchestra.run.smithi077.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.4 MB/s | 142 kB 00:00 2024-04-25T10:20:02.893 INFO:teuthology.orchestra.run.smithi043.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 755 kB/s | 44 kB 00:00 2024-04-25T10:20:02.970 INFO:teuthology.orchestra.run.smithi043.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-25T10:20:03.027 INFO:teuthology.orchestra.run.smithi043.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 617 kB/s | 36 kB 00:00 2024-04-25T10:20:03.094 INFO:teuthology.orchestra.run.smithi043.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 710 kB/s | 47 kB 00:00 2024-04-25T10:20:03.653 INFO:teuthology.orchestra.run.smithi043.stdout:(96/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 2.9 MB/s | 43 MB 00:15 2024-04-25T10:20:03.713 INFO:teuthology.orchestra.run.smithi077.stdout:(30/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 2.6 MB/s | 3.0 MB 00:01 2024-04-25T10:20:03.820 INFO:teuthology.orchestra.run.smithi043.stdout:(97/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-04-25T10:20:03.845 INFO:teuthology.orchestra.run.smithi043.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 406 kB/s | 305 kB 00:00 2024-04-25T10:20:03.871 INFO:teuthology.orchestra.run.smithi043.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 266 kB/s | 58 kB 00:00 2024-04-25T10:20:03.904 INFO:teuthology.orchestra.run.smithi043.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-04-25T10:20:03.930 INFO:teuthology.orchestra.run.smithi043.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 758 kB/s | 64 kB 00:00 2024-04-25T10:20:03.954 INFO:teuthology.orchestra.run.smithi077.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 3.3 MB/s | 3.5 MB 00:01 2024-04-25T10:20:03.955 INFO:teuthology.orchestra.run.smithi043.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 536 kB/s | 45 kB 00:00 2024-04-25T10:20:04.014 INFO:teuthology.orchestra.run.smithi043.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-04-25T10:20:04.072 INFO:teuthology.orchestra.run.smithi077.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.1 MB/s | 249 kB 00:00 2024-04-25T10:20:04.189 INFO:teuthology.orchestra.run.smithi077.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 422 kB/s | 49 kB 00:00 2024-04-25T10:20:04.272 INFO:teuthology.orchestra.run.smithi043.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-04-25T10:20:04.297 INFO:teuthology.orchestra.run.smithi077.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 152 kB/s | 16 kB 00:00 2024-04-25T10:20:04.406 INFO:teuthology.orchestra.run.smithi077.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 2.5 MB/s | 273 kB 00:00 2024-04-25T10:20:04.681 INFO:teuthology.orchestra.run.smithi077.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 2.7 MB/s | 762 kB 00:00 2024-04-25T10:20:04.722 INFO:teuthology.orchestra.run.smithi043.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 43 kB/s | 19 kB 00:00 2024-04-25T10:20:04.773 INFO:teuthology.orchestra.run.smithi043.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 671 kB/s | 548 kB 00:00 2024-04-25T10:20:04.790 INFO:teuthology.orchestra.run.smithi077.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 168 kB/s | 18 kB 00:00 2024-04-25T10:20:04.923 INFO:teuthology.orchestra.run.smithi043.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 173 kB/s | 26 kB 00:00 2024-04-25T10:20:04.982 INFO:teuthology.orchestra.run.smithi077.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 419 kB/s | 80 kB 00:00 2024-04-25T10:20:05.245 INFO:teuthology.orchestra.run.smithi164.stdout:(10/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 1.4 MB/s | 10 MB 00:07 2024-04-25T10:20:05.403 INFO:teuthology.orchestra.run.smithi164.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 1.4 MB/s | 224 kB 00:00 2024-04-25T10:20:05.424 INFO:teuthology.orchestra.run.smithi077.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 803 kB 00:00 2024-04-25T10:20:05.453 INFO:teuthology.orchestra.run.smithi164.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 543 kB/s | 27 kB 00:00 2024-04-25T10:20:05.512 INFO:teuthology.orchestra.run.smithi164.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 545 kB/s | 32 kB 00:00 2024-04-25T10:20:05.524 INFO:teuthology.orchestra.run.smithi077.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-25T10:20:05.548 INFO:teuthology.orchestra.run.smithi043.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 78 kB/s | 49 kB 00:00 2024-04-25T10:20:05.658 INFO:teuthology.orchestra.run.smithi077.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-04-25T10:20:05.673 INFO:teuthology.orchestra.run.smithi043.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 539 kB/s | 67 kB 00:00 2024-04-25T10:20:05.854 INFO:teuthology.orchestra.run.smithi164.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 2.2 MB/s | 756 kB 00:00 2024-04-25T10:20:05.858 INFO:teuthology.orchestra.run.smithi077.stdout:(42/152): ceph-mgr-diskprediction-local-19.0.0- 3.4 MB/s | 7.4 MB 00:02 2024-04-25T10:20:05.954 INFO:teuthology.orchestra.run.smithi164.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.5 MB/s | 158 kB 00:00 2024-04-25T10:20:05.965 INFO:teuthology.orchestra.run.smithi043.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 518 kB/s | 151 kB 00:00 2024-04-25T10:20:05.992 INFO:teuthology.orchestra.run.smithi077.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 3.7 MB/s | 1.2 MB 00:00 2024-04-25T10:20:06.050 INFO:teuthology.orchestra.run.smithi077.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T10:20:06.075 INFO:teuthology.orchestra.run.smithi077.stdout:(45/152): python3-ply-3.11-14.el9.noarch.rpm 491 kB/s | 106 kB 00:00 2024-04-25T10:20:06.109 INFO:teuthology.orchestra.run.smithi077.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-25T10:20:06.155 INFO:teuthology.orchestra.run.smithi164.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 632 kB/s | 127 kB 00:00 2024-04-25T10:20:06.193 INFO:teuthology.orchestra.run.smithi077.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-04-25T10:20:06.218 INFO:teuthology.orchestra.run.smithi077.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 6.2 MB/s | 689 kB 00:00 2024-04-25T10:20:06.285 INFO:teuthology.orchestra.run.smithi077.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T10:20:06.305 INFO:teuthology.orchestra.run.smithi164.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 3.2 MB/s | 497 kB 00:00 2024-04-25T10:20:06.335 INFO:teuthology.orchestra.run.smithi077.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 432 kB/s | 21 kB 00:00 2024-04-25T10:20:06.411 INFO:teuthology.orchestra.run.smithi077.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-04-25T10:20:06.461 INFO:teuthology.orchestra.run.smithi077.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 655 kB/s | 33 kB 00:00 2024-04-25T10:20:06.670 INFO:teuthology.orchestra.run.smithi077.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-04-25T10:20:06.720 INFO:teuthology.orchestra.run.smithi077.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-04-25T10:20:06.762 INFO:teuthology.orchestra.run.smithi077.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-04-25T10:20:06.774 INFO:teuthology.orchestra.run.smithi043.stdout:(111/152): gperftools-libs-2.9.1-2.el9.x86_64.r 112 kB/s | 309 kB 00:02 2024-04-25T10:20:06.840 INFO:teuthology.orchestra.run.smithi077.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.0 MB/s | 160 kB 00:00 2024-04-25T10:20:06.891 INFO:teuthology.orchestra.run.smithi043.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 906 kB/s | 838 kB 00:00 2024-04-25T10:20:06.896 INFO:teuthology.orchestra.run.smithi077.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 823 kB/s | 45 kB 00:00 2024-04-25T10:20:06.980 INFO:teuthology.orchestra.run.smithi077.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-04-25T10:20:07.063 INFO:teuthology.orchestra.run.smithi077.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 582 kB 00:00 2024-04-25T10:20:07.099 INFO:teuthology.orchestra.run.smithi043.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 208 kB/s | 43 kB 00:00 2024-04-25T10:20:07.122 INFO:teuthology.orchestra.run.smithi077.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 434 kB/s | 25 kB 00:00 2024-04-25T10:20:07.181 INFO:teuthology.orchestra.run.smithi077.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-04-25T10:20:07.256 INFO:teuthology.orchestra.run.smithi077.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-25T10:20:07.323 INFO:teuthology.orchestra.run.smithi077.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-25T10:20:07.373 INFO:teuthology.orchestra.run.smithi077.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 708 kB/s | 35 kB 00:00 2024-04-25T10:20:07.441 INFO:teuthology.orchestra.run.smithi043.stdout:(114/152): libarrow-9.0.0-10.el9.x86_64.rpm 1.6 MB/s | 4.4 MB 00:02 2024-04-25T10:20:07.467 INFO:teuthology.orchestra.run.smithi043.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 88 kB/s | 32 kB 00:00 2024-04-25T10:20:07.558 INFO:teuthology.orchestra.run.smithi043.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-04-25T10:20:07.565 INFO:teuthology.orchestra.run.smithi077.stdout:(65/152): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-04-25T10:20:07.609 INFO:teuthology.orchestra.run.smithi043.stdout:(117/152): python3-asyncssh-2.13.2-5.el9.noarch 657 kB/s | 548 kB 00:00 2024-04-25T10:20:07.757 INFO:teuthology.orchestra.run.smithi077.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-04-25T10:20:07.883 INFO:teuthology.orchestra.run.smithi077.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-25T10:20:07.991 INFO:teuthology.orchestra.run.smithi077.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 8.2 MB/s | 911 kB 00:00 2024-04-25T10:20:08.058 INFO:teuthology.orchestra.run.smithi077.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 133 kB 00:00 2024-04-25T10:20:08.105 INFO:teuthology.orchestra.run.smithi164.stdout:(18/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 2.9 MB/s | 5.2 MB 00:01 2024-04-25T10:20:08.214 INFO:teuthology.orchestra.run.smithi164.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 418 kB/s | 45 kB 00:00 2024-04-25T10:20:08.292 INFO:teuthology.orchestra.run.smithi077.stdout:(70/152): postgresql-server-13.14-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-04-25T10:20:08.314 INFO:teuthology.orchestra.run.smithi164.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.3 MB/s | 135 kB 00:00 2024-04-25T10:20:08.384 INFO:teuthology.orchestra.run.smithi077.stdout:(71/152): bison-3.7.4-5.el9.x86_64.rpm 430 kB/s | 943 kB 00:02 2024-04-25T10:20:08.426 INFO:teuthology.orchestra.run.smithi077.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-04-25T10:20:08.464 INFO:teuthology.orchestra.run.smithi164.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.1 MB/s | 161 kB 00:00 2024-04-25T10:20:08.476 INFO:teuthology.orchestra.run.smithi077.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 952 kB/s | 87 kB 00:00 2024-04-25T10:20:08.551 INFO:teuthology.orchestra.run.smithi077.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-25T10:20:08.627 INFO:teuthology.orchestra.run.smithi077.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-04-25T10:20:08.677 INFO:teuthology.orchestra.run.smithi077.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 515 kB/s | 26 kB 00:00 2024-04-25T10:20:08.698 INFO:teuthology.orchestra.run.smithi164.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 1.3 MB/s | 321 kB 00:00 2024-04-25T10:20:08.744 INFO:teuthology.orchestra.run.smithi077.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 6.0 MB 00:00 2024-04-25T10:20:08.786 INFO:teuthology.orchestra.run.smithi077.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-04-25T10:20:08.807 INFO:teuthology.orchestra.run.smithi164.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 2.7 MB/s | 298 kB 00:00 2024-04-25T10:20:08.828 INFO:teuthology.orchestra.run.smithi077.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-04-25T10:20:08.842 INFO:teuthology.orchestra.run.smithi043.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 246 kB/s | 303 kB 00:01 2024-04-25T10:20:08.853 INFO:teuthology.orchestra.run.smithi077.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 519 kB/s | 35 kB 00:00 2024-04-25T10:20:08.867 INFO:teuthology.orchestra.run.smithi043.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 274 kB/s | 358 kB 00:01 2024-04-25T10:20:08.874 INFO:teuthology.orchestra.run.smithi164.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.5 MB/s | 100 kB 00:00 2024-04-25T10:20:08.893 INFO:teuthology.orchestra.run.smithi043.stdout:(120/152): python3-cheroot-8.6.0-4.el9.noarch.r 120 kB/s | 172 kB 00:01 2024-04-25T10:20:08.928 INFO:teuthology.orchestra.run.smithi077.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-25T10:20:08.965 INFO:teuthology.orchestra.run.smithi164.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 941 kB/s | 86 kB 00:00 2024-04-25T10:20:09.004 INFO:teuthology.orchestra.run.smithi077.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-04-25T10:20:09.121 INFO:teuthology.orchestra.run.smithi077.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-04-25T10:20:09.146 INFO:teuthology.orchestra.run.smithi077.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 544 kB/s | 77 kB 00:00 2024-04-25T10:20:09.160 INFO:teuthology.orchestra.run.smithi043.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 704 kB/s | 206 kB 00:00 2024-04-25T10:20:09.185 INFO:teuthology.orchestra.run.smithi043.stdout:(122/152): python3-colorama-0.4.4-12.el9.noarch 103 kB/s | 35 kB 00:00 2024-04-25T10:20:09.196 INFO:teuthology.orchestra.run.smithi077.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 562 kB/s | 42 kB 00:00 2024-04-25T10:20:09.247 INFO:teuthology.orchestra.run.smithi077.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-04-25T10:20:09.289 INFO:teuthology.orchestra.run.smithi077.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-04-25T10:20:09.339 INFO:teuthology.orchestra.run.smithi077.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-04-25T10:20:09.364 INFO:teuthology.orchestra.run.smithi077.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 677 kB/s | 51 kB 00:00 2024-04-25T10:20:09.415 INFO:teuthology.orchestra.run.smithi077.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 712 kB/s | 54 kB 00:00 2024-04-25T10:20:09.482 INFO:teuthology.orchestra.run.smithi077.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 624 kB/s | 42 kB 00:00 2024-04-25T10:20:09.549 INFO:teuthology.orchestra.run.smithi077.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 663 kB/s | 44 kB 00:00 2024-04-25T10:20:09.635 INFO:teuthology.orchestra.run.smithi043.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 24 kB/s | 11 kB 00:00 2024-04-25T10:20:09.640 INFO:teuthology.orchestra.run.smithi077.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-04-25T10:20:09.661 INFO:teuthology.orchestra.run.smithi043.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 289 kB/s | 144 kB 00:00 2024-04-25T10:20:09.699 INFO:teuthology.orchestra.run.smithi077.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-04-25T10:20:09.710 INFO:teuthology.orchestra.run.smithi043.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.5 MB/s | 2.0 MB 00:00 2024-04-25T10:20:09.736 INFO:teuthology.orchestra.run.smithi043.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-25T10:20:09.766 INFO:teuthology.orchestra.run.smithi077.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-04-25T10:20:09.983 INFO:teuthology.orchestra.run.smithi077.stdout:(96/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-04-25T10:20:10.008 INFO:teuthology.orchestra.run.smithi077.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 305 kB 00:00 2024-04-25T10:20:10.051 INFO:teuthology.orchestra.run.smithi077.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 865 kB/s | 58 kB 00:00 2024-04-25T10:20:10.084 INFO:teuthology.orchestra.run.smithi077.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-04-25T10:20:10.126 INFO:teuthology.orchestra.run.smithi077.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 845 kB/s | 64 kB 00:00 2024-04-25T10:20:10.151 INFO:teuthology.orchestra.run.smithi077.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 669 kB/s | 45 kB 00:00 2024-04-25T10:20:10.252 INFO:teuthology.orchestra.run.smithi077.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-04-25T10:20:10.335 INFO:teuthology.orchestra.run.smithi164.stdout:(26/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 1.3 MB/s | 18 MB 00:13 2024-04-25T10:20:10.510 INFO:teuthology.orchestra.run.smithi164.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 971 kB/s | 169 kB 00:00 2024-04-25T10:20:10.568 INFO:teuthology.orchestra.run.smithi164.stdout:(28/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 1.9 MB/s | 3.0 MB 00:01 2024-04-25T10:20:10.610 INFO:teuthology.orchestra.run.smithi164.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3291.g 341 kB/s | 34 kB 00:00 2024-04-25T10:20:10.635 INFO:teuthology.orchestra.run.smithi077.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-04-25T10:20:10.660 INFO:teuthology.orchestra.run.smithi164.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.5 MB/s | 142 kB 00:00 2024-04-25T10:20:10.769 INFO:teuthology.orchestra.run.smithi077.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-04-25T10:20:10.786 INFO:teuthology.orchestra.run.smithi043.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 21 kB/s | 23 kB 00:01 2024-04-25T10:20:10.803 INFO:teuthology.orchestra.run.smithi043.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 18 kB/s | 19 kB 00:01 2024-04-25T10:20:10.954 INFO:teuthology.orchestra.run.smithi043.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 308 kB/s | 46 kB 00:00 2024-04-25T10:20:11.470 INFO:teuthology.orchestra.run.smithi043.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 153 kB/s | 79 kB 00:00 2024-04-25T10:20:11.621 INFO:teuthology.orchestra.run.smithi043.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 1.2 MB/s | 1.0 MB 00:00 2024-04-25T10:20:11.646 INFO:teuthology.orchestra.run.smithi043.stdout:(132/152): python3-jaraco-text-3.2.0-6.el9.noar 10 kB/s | 20 kB 00:01 2024-04-25T10:20:11.663 INFO:teuthology.orchestra.run.smithi043.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 300 kB/s | 58 kB 00:00 2024-04-25T10:20:11.913 INFO:teuthology.orchestra.run.smithi043.stdout:(134/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 360 kB/s | 90 kB 00:00 2024-04-25T10:20:11.938 INFO:teuthology.orchestra.run.smithi043.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 859 kB/s | 272 kB 00:00 2024-04-25T10:20:12.147 INFO:teuthology.orchestra.run.smithi043.stdout:(136/152): python3-repoze-lru-0.7-16.el9.noarch 147 kB/s | 31 kB 00:00 2024-04-25T10:20:12.172 INFO:teuthology.orchestra.run.smithi043.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 109 kB/s | 28 kB 00:00 2024-04-25T10:20:12.297 INFO:teuthology.orchestra.run.smithi043.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 473 kB/s | 59 kB 00:00 2024-04-25T10:20:12.322 INFO:teuthology.orchestra.run.smithi043.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-04-25T10:20:12.469 INFO:teuthology.orchestra.run.smithi077.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 11 kB/s | 19 kB 00:01 2024-04-25T10:20:12.519 INFO:teuthology.orchestra.run.smithi077.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 164 kB/s | 309 kB 00:01 2024-04-25T10:20:12.598 INFO:teuthology.orchestra.run.smithi043.stdout:(140/152): python3-typing-extensions-4.4.0-2.el 185 kB/s | 51 kB 00:00 2024-04-25T10:20:12.623 INFO:teuthology.orchestra.run.smithi043.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 110 kB/s | 36 kB 00:00 2024-04-25T10:20:12.832 INFO:teuthology.orchestra.run.smithi043.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 984 kB/s | 230 kB 00:00 2024-04-25T10:20:13.028 INFO:teuthology.orchestra.run.smithi077.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 51 kB/s | 26 kB 00:00 2024-04-25T10:20:13.060 INFO:teuthology.orchestra.run.smithi164.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 1.4 MB/s | 3.5 MB 00:02 2024-04-25T10:20:13.177 INFO:teuthology.orchestra.run.smithi164.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.1 MB/s | 249 kB 00:00 2024-04-25T10:20:13.228 INFO:teuthology.orchestra.run.smithi164.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 990 kB/s | 49 kB 00:00 2024-04-25T10:20:13.278 INFO:teuthology.orchestra.run.smithi164.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 330 kB/s | 16 kB 00:00 2024-04-25T10:20:13.461 INFO:teuthology.orchestra.run.smithi077.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 112 kB/s | 49 kB 00:00 2024-04-25T10:20:13.695 INFO:teuthology.orchestra.run.smithi077.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-04-25T10:20:13.712 INFO:teuthology.orchestra.run.smithi164.stdout:(35/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 3.7 MB/s | 43 MB 00:11 2024-04-25T10:20:13.720 INFO:teuthology.orchestra.run.smithi077.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 260 kB/s | 67 kB 00:00 2024-04-25T10:20:13.737 INFO:teuthology.orchestra.run.smithi164.stdout:(36/152): ceph-volume-19.0.0-3291.g994e2f20.el9 595 kB/s | 273 kB 00:00 2024-04-25T10:20:13.795 INFO:teuthology.orchestra.run.smithi164.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 314 kB/s | 18 kB 00:00 2024-04-25T10:20:13.879 INFO:teuthology.orchestra.run.smithi164.stdout:(38/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 4.5 MB/s | 762 kB 00:00 2024-04-25T10:20:13.946 INFO:teuthology.orchestra.run.smithi164.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 534 kB/s | 80 kB 00:00 2024-04-25T10:20:14.046 INFO:teuthology.orchestra.run.smithi164.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-25T10:20:14.138 INFO:teuthology.orchestra.run.smithi164.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-25T10:20:14.171 INFO:teuthology.orchestra.run.smithi164.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-04-25T10:20:14.230 INFO:teuthology.orchestra.run.smithi164.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T10:20:14.271 INFO:teuthology.orchestra.run.smithi077.stdout:(111/152): ceph-test-19.0.0-3291.g994e2f20.el9. 2.5 MB/s | 43 MB 00:17 2024-04-25T10:20:14.364 INFO:teuthology.orchestra.run.smithi164.stdout:(44/152): ceph-mgr-diskprediction-local-19.0.0- 2.0 MB/s | 7.4 MB 00:03 2024-04-25T10:20:14.389 INFO:teuthology.orchestra.run.smithi164.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 852 kB/s | 135 kB 00:00 2024-04-25T10:20:14.448 INFO:teuthology.orchestra.run.smithi164.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-25T10:20:14.523 INFO:teuthology.orchestra.run.smithi164.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 3.2 MB/s | 1.2 MB 00:00 2024-04-25T10:20:14.548 INFO:teuthology.orchestra.run.smithi164.stdout:(48/152): python3-requests-2.25.1-8.el9.noarch. 679 kB/s | 125 kB 00:00 2024-04-25T10:20:14.574 INFO:teuthology.orchestra.run.smithi164.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 5.4 MB/s | 689 kB 00:00 2024-04-25T10:20:14.607 INFO:teuthology.orchestra.run.smithi164.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T10:20:14.632 INFO:teuthology.orchestra.run.smithi164.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 367 kB/s | 21 kB 00:00 2024-04-25T10:20:14.683 INFO:teuthology.orchestra.run.smithi164.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 655 kB/s | 33 kB 00:00 2024-04-25T10:20:14.716 INFO:teuthology.orchestra.run.smithi164.stdout:(53/152): bison-3.7.4-5.el9.x86_64.rpm 4.8 MB/s | 943 kB 00:00 2024-04-25T10:20:14.741 INFO:teuthology.orchestra.run.smithi164.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-04-25T10:20:14.775 INFO:teuthology.orchestra.run.smithi164.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 309 kB/s | 18 kB 00:00 2024-04-25T10:20:14.792 INFO:teuthology.orchestra.run.smithi164.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 220 kB/s | 11 kB 00:00 2024-04-25T10:20:14.834 INFO:teuthology.orchestra.run.smithi164.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T10:20:14.860 INFO:teuthology.orchestra.run.smithi164.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 675 kB/s | 45 kB 00:00 2024-04-25T10:20:14.977 INFO:teuthology.orchestra.run.smithi164.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-04-25T10:20:15.002 INFO:teuthology.orchestra.run.smithi164.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 4.0 MB/s | 582 kB 00:00 2024-04-25T10:20:15.027 INFO:teuthology.orchestra.run.smithi164.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 507 kB/s | 25 kB 00:00 2024-04-25T10:20:15.052 INFO:teuthology.orchestra.run.smithi164.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-04-25T10:20:15.103 INFO:teuthology.orchestra.run.smithi164.stdout:(63/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.1 MB/s | 3.0 MB 00:00 2024-04-25T10:20:15.128 INFO:teuthology.orchestra.run.smithi164.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-04-25T10:20:15.170 INFO:teuthology.orchestra.run.smithi164.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-04-25T10:20:15.187 INFO:teuthology.orchestra.run.smithi164.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 421 kB/s | 35 kB 00:00 2024-04-25T10:20:15.354 INFO:teuthology.orchestra.run.smithi164.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-25T10:20:15.462 INFO:teuthology.orchestra.run.smithi164.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 8.2 MB/s | 911 kB 00:00 2024-04-25T10:20:15.521 INFO:teuthology.orchestra.run.smithi164.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-25T10:20:15.648 INFO:teuthology.orchestra.run.smithi043.stdout:(143/152): python3-portend-3.1.0-2.el9.noarch.r 4.1 kB/s | 16 kB 00:04 2024-04-25T10:20:15.663 INFO:teuthology.orchestra.run.smithi164.stdout:(70/152): openblas-openmp-0.3.21-2.el9.x86_64.r 8.9 MB/s | 4.7 MB 00:00 2024-04-25T10:20:15.674 INFO:teuthology.orchestra.run.smithi043.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 29 kB/s | 90 kB 00:03 2024-04-25T10:20:15.707 INFO:teuthology.orchestra.run.smithi043.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 149 kB/s | 427 kB 00:02 2024-04-25T10:20:15.730 INFO:teuthology.orchestra.run.smithi164.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.9 MB/s | 4.4 MB 00:00 2024-04-25T10:20:15.788 INFO:teuthology.orchestra.run.smithi164.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T10:20:15.822 INFO:teuthology.orchestra.run.smithi164.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-04-25T10:20:15.898 INFO:teuthology.orchestra.run.smithi164.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-25T10:20:15.972 INFO:teuthology.orchestra.run.smithi164.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-04-25T10:20:15.998 INFO:teuthology.orchestra.run.smithi164.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-25T10:20:16.031 INFO:teuthology.orchestra.run.smithi164.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 439 kB/s | 26 kB 00:00 2024-04-25T10:20:16.033 INFO:teuthology.orchestra.run.smithi043.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 52 kB/s | 20 kB 00:00 2024-04-25T10:20:16.057 INFO:teuthology.orchestra.run.smithi164.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-04-25T10:20:16.090 INFO:teuthology.orchestra.run.smithi164.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-25T10:20:16.116 INFO:teuthology.orchestra.run.smithi164.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-04-25T10:20:16.174 INFO:teuthology.orchestra.run.smithi164.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-25T10:20:16.425 INFO:teuthology.orchestra.run.smithi164.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 889 kB/s | 222 kB 00:00 2024-04-25T10:20:16.492 INFO:teuthology.orchestra.run.smithi164.stdout:(83/152): python3-babel-2.9.1-2.el9.noarch.rpm 8.5 MB/s | 6.0 MB 00:00 2024-04-25T10:20:16.517 INFO:teuthology.orchestra.run.smithi164.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 836 kB/s | 77 kB 00:00 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi164.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 722 kB/s | 42 kB 00:00 2024-04-25T10:20:16.558 INFO:teuthology.orchestra.run.smithi043.stdout:(147/152): librados2-19.0.0-3291.g994e2f20.el9. 6.3 MB/s | 3.3 MB 00:00 2024-04-25T10:20:16.576 INFO:teuthology.orchestra.run.smithi164.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-25T10:20:16.643 INFO:teuthology.orchestra.run.smithi164.stdout:(87/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.2 MB/s | 5.1 MB 00:00 2024-04-25T10:20:16.668 INFO:teuthology.orchestra.run.smithi164.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-04-25T10:20:16.693 INFO:teuthology.orchestra.run.smithi164.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-04-25T10:20:16.719 INFO:teuthology.orchestra.run.smithi164.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 678 kB/s | 51 kB 00:00 2024-04-25T10:20:16.744 INFO:teuthology.orchestra.run.smithi164.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 710 kB/s | 54 kB 00:00 2024-04-25T10:20:16.769 INFO:teuthology.orchestra.run.smithi164.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-04-25T10:20:16.803 INFO:teuthology.orchestra.run.smithi164.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 754 kB/s | 44 kB 00:00 2024-04-25T10:20:16.828 INFO:teuthology.orchestra.run.smithi164.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-25T10:20:16.862 INFO:teuthology.orchestra.run.smithi164.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 613 kB/s | 36 kB 00:00 2024-04-25T10:20:16.887 INFO:teuthology.orchestra.run.smithi164.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-04-25T10:20:16.920 INFO:teuthology.orchestra.run.smithi164.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-25T10:20:16.929 INFO:teuthology.orchestra.run.smithi077.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 206 kB/s | 548 kB 00:02 2024-04-25T10:20:16.946 INFO:teuthology.orchestra.run.smithi164.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 985 kB/s | 58 kB 00:00 2024-04-25T10:20:16.950 INFO:teuthology.orchestra.run.smithi043.stdout:(148/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 7.9 MB/s | 3.1 MB 00:00 2024-04-25T10:20:16.980 INFO:teuthology.orchestra.run.smithi164.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T10:20:17.005 INFO:teuthology.orchestra.run.smithi164.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T10:20:17.030 INFO:teuthology.orchestra.run.smithi164.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 896 kB/s | 45 kB 00:00 2024-04-25T10:20:17.064 INFO:teuthology.orchestra.run.smithi164.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 382 kB/s | 22 kB 00:00 2024-04-25T10:20:17.139 INFO:teuthology.orchestra.run.smithi164.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 7.8 MB/s | 863 kB 00:00 2024-04-25T10:20:17.298 INFO:teuthology.orchestra.run.smithi077.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 118 kB/s | 43 kB 00:00 2024-04-25T10:20:17.331 INFO:teuthology.orchestra.run.smithi164.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.0 MB/s | 548 kB 00:00 2024-04-25T10:20:17.375 INFO:teuthology.orchestra.run.smithi043.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 3.2 MB/s | 1.3 MB 00:00 2024-04-25T10:20:17.389 INFO:teuthology.orchestra.run.smithi164.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-04-25T10:20:17.406 INFO:teuthology.orchestra.run.smithi164.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 258 kB/s | 19 kB 00:00 2024-04-25T10:20:17.456 INFO:teuthology.orchestra.run.smithi164.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 520 kB/s | 26 kB 00:00 2024-04-25T10:20:17.507 INFO:teuthology.orchestra.run.smithi164.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2024-04-25T10:20:17.566 INFO:teuthology.orchestra.run.smithi164.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-25T10:20:17.632 INFO:teuthology.orchestra.run.smithi164.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-25T10:20:17.658 INFO:teuthology.orchestra.run.smithi164.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-04-25T10:20:17.784 INFO:teuthology.orchestra.run.smithi043.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.9 MB/s | 2.4 MB 00:00 2024-04-25T10:20:17.790 INFO:teuthology.orchestra.run.smithi077.stdout:(114/152): luarocks-3.9.2-1.el9.noarch.rpm 37 kB/s | 151 kB 00:04 2024-04-25T10:20:17.816 INFO:teuthology.orchestra.run.smithi164.stdout:(112/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-04-25T10:20:17.842 INFO:teuthology.orchestra.run.smithi164.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-04-25T10:20:17.867 INFO:teuthology.orchestra.run.smithi164.stdout:(114/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.5 MB/s | 838 kB 00:00 2024-04-25T10:20:17.901 INFO:teuthology.orchestra.run.smithi164.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 551 kB/s | 32 kB 00:00 2024-04-25T10:20:17.926 INFO:teuthology.orchestra.run.smithi164.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 255 kB/s | 15 kB 00:00 2024-04-25T10:20:17.960 INFO:teuthology.orchestra.run.smithi164.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-25T10:20:17.985 INFO:teuthology.orchestra.run.smithi164.stdout:(118/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 258 kB/s | 43 kB 00:00 2024-04-25T10:20:18.019 INFO:teuthology.orchestra.run.smithi164.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-25T10:20:18.044 INFO:teuthology.orchestra.run.smithi164.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-04-25T10:20:18.065 INFO:teuthology.orchestra.run.smithi077.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 42 kB/s | 32 kB 00:00 2024-04-25T10:20:18.077 INFO:teuthology.orchestra.run.smithi164.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-04-25T10:20:18.103 INFO:teuthology.orchestra.run.smithi164.stdout:(122/152): python3-colorama-0.4.4-12.el9.noarch 300 kB/s | 35 kB 00:00 2024-04-25T10:20:18.136 INFO:teuthology.orchestra.run.smithi164.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-25T10:20:18.162 INFO:teuthology.orchestra.run.smithi164.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-04-25T10:20:18.212 INFO:teuthology.orchestra.run.smithi164.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-25T10:20:18.237 INFO:teuthology.orchestra.run.smithi164.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-25T10:20:18.262 INFO:teuthology.orchestra.run.smithi164.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-25T10:20:18.291 INFO:teuthology.orchestra.run.smithi164.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-04-25T10:20:18.321 INFO:teuthology.orchestra.run.smithi164.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 337 kB/s | 20 kB 00:00 2024-04-25T10:20:18.346 INFO:teuthology.orchestra.run.smithi164.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 789 kB/s | 46 kB 00:00 2024-04-25T10:20:18.380 INFO:teuthology.orchestra.run.smithi164.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-25T10:20:18.405 INFO:teuthology.orchestra.run.smithi164.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 987 kB/s | 58 kB 00:00 2024-04-25T10:20:18.439 INFO:teuthology.orchestra.run.smithi164.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-04-25T10:20:18.464 INFO:teuthology.orchestra.run.smithi164.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-04-25T10:20:18.498 INFO:teuthology.orchestra.run.smithi164.stdout:(135/152): python3-kubernetes-26.1.0-2.el9.noar 4.3 MB/s | 1.0 MB 00:00 2024-04-25T10:20:18.515 INFO:teuthology.orchestra.run.smithi164.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-04-25T10:20:18.540 INFO:teuthology.orchestra.run.smithi164.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 893 kB/s | 90 kB 00:00 2024-04-25T10:20:18.557 INFO:teuthology.orchestra.run.smithi164.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 524 kB/s | 31 kB 00:00 2024-04-25T10:20:18.582 INFO:teuthology.orchestra.run.smithi164.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-04-25T10:20:18.599 INFO:teuthology.orchestra.run.smithi164.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-25T10:20:18.616 INFO:teuthology.orchestra.run.smithi164.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 609 kB/s | 36 kB 00:00 2024-04-25T10:20:18.641 INFO:teuthology.orchestra.run.smithi164.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 869 kB/s | 51 kB 00:00 2024-04-25T10:20:18.667 INFO:teuthology.orchestra.run.smithi164.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-04-25T10:20:18.692 INFO:teuthology.orchestra.run.smithi164.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-25T10:20:18.726 INFO:teuthology.orchestra.run.smithi164.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-25T10:20:18.751 INFO:teuthology.orchestra.run.smithi164.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-04-25T10:20:18.777 INFO:teuthology.orchestra.run.smithi164.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-25T10:20:18.827 INFO:teuthology.orchestra.run.smithi164.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-25T10:20:19.085 INFO:teuthology.orchestra.run.smithi164.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 5.2 MB/s | 1.3 MB 00:00 2024-04-25T10:20:19.269 INFO:teuthology.orchestra.run.smithi164.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 13 MB/s | 2.4 MB 00:00 2024-04-25T10:20:19.290 INFO:teuthology.orchestra.run.smithi077.stdout:(116/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 150 kB/s | 838 kB 00:05 2024-04-25T10:20:19.424 INFO:teuthology.orchestra.run.smithi077.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 9.2 kB/s | 15 kB 00:01 2024-04-25T10:20:19.511 INFO:teuthology.orchestra.run.smithi164.stdout:(151/152): librados2-19.0.0-3291.g994e2f20.el9. 4.4 MB/s | 3.3 MB 00:00 2024-04-25T10:20:19.524 INFO:teuthology.orchestra.run.smithi077.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 1.5 MB/s | 358 kB 00:00 2024-04-25T10:20:19.644 INFO:teuthology.orchestra.run.smithi164.stdout:(152/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 3.6 MB/s | 3.1 MB 00:00 2024-04-25T10:20:19.647 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:20:19.647 INFO:teuthology.orchestra.run.smithi164.stdout:Total 6.7 MB/s | 214 MB 00:32 2024-04-25T10:20:19.675 INFO:teuthology.orchestra.run.smithi077.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 235 kB/s | 35 kB 00:00 2024-04-25T10:20:19.708 INFO:teuthology.orchestra.run.smithi077.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.0 MB/s | 303 kB 00:00 2024-04-25T10:20:19.808 INFO:teuthology.orchestra.run.smithi077.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-04-25T10:20:19.959 INFO:teuthology.orchestra.run.smithi077.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 963 kB/s | 144 kB 00:00 2024-04-25T10:20:20.017 INFO:teuthology.orchestra.run.smithi077.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.6 MB/s | 2.0 MB 00:00 2024-04-25T10:20:20.043 INFO:teuthology.orchestra.run.smithi077.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-04-25T10:20:20.068 INFO:teuthology.orchestra.run.smithi077.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-04-25T10:20:20.102 INFO:teuthology.orchestra.run.smithi077.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 394 kB/s | 23 kB 00:00 2024-04-25T10:20:20.127 INFO:teuthology.orchestra.run.smithi077.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-04-25T10:20:20.152 INFO:teuthology.orchestra.run.smithi077.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-04-25T10:20:20.386 INFO:teuthology.orchestra.run.smithi077.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 198 kB/s | 46 kB 00:00 2024-04-25T10:20:20.419 INFO:teuthology.orchestra.run.smithi077.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 3.5 MB/s | 1.0 MB 00:00 2024-04-25T10:20:20.520 INFO:teuthology.orchestra.run.smithi077.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 590 kB/s | 79 kB 00:00 2024-04-25T10:20:20.545 INFO:teuthology.orchestra.run.smithi077.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 460 kB/s | 58 kB 00:00 2024-04-25T10:20:20.724 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-25T10:20:20.869 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-25T10:20:20.869 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-25T10:20:21.933 INFO:teuthology.orchestra.run.smithi043.stdout:(151/152): re2-20211101-3.el9.x86_64.rpm 31 kB/s | 192 kB 00:06 2024-04-25T10:20:22.588 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-25T10:20:22.588 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-25T10:20:22.903 INFO:teuthology.orchestra.run.smithi077.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 114 kB/s | 272 kB 00:02 2024-04-25T10:20:23.004 INFO:teuthology.orchestra.run.smithi077.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 6.7 kB/s | 16 kB 00:02 2024-04-25T10:20:23.478 INFO:teuthology.orchestra.run.smithi077.stdout:(135/152): python3-pyperclip-1.8.2-2.el9.noarch 59 kB/s | 28 kB 00:00 2024-04-25T10:20:23.529 INFO:teuthology.orchestra.run.smithi077.stdout:(136/152): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-04-25T10:20:23.546 INFO:teuthology.orchestra.run.smithi077.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 140 kB/s | 90 kB 00:00 2024-04-25T10:20:23.646 INFO:teuthology.orchestra.run.smithi077.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-04-25T10:20:23.896 INFO:teuthology.orchestra.run.smithi077.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 143 kB/s | 36 kB 00:00 2024-04-25T10:20:23.922 INFO:teuthology.orchestra.run.smithi077.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 157 kB/s | 59 kB 00:00 2024-04-25T10:20:24.597 INFO:teuthology.orchestra.run.smithi077.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 73 kB/s | 51 kB 00:00 2024-04-25T10:20:25.014 INFO:teuthology.orchestra.run.smithi077.stdout:(142/152): python3-cheroot-8.6.0-4.el9.noarch.r 25 kB/s | 172 kB 00:06 2024-04-25T10:20:25.140 INFO:teuthology.orchestra.run.smithi077.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 165 kB/s | 90 kB 00:00 2024-04-25T10:20:25.198 INFO:teuthology.orchestra.run.smithi077.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 180 kB/s | 230 kB 00:01 2024-04-25T10:20:25.315 INFO:teuthology.orchestra.run.smithi077.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 114 kB/s | 20 kB 00:00 2024-04-25T10:20:28.503 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-25T10:20:28.589 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T10:20:28.636 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T10:20:28.689 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T10:20:28.723 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T10:20:28.749 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T10:20:28.791 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T10:20:29.118 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T10:20:29.250 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:20:29.367 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:20:29.401 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:20:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:20:29.514 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:20:29.649 INFO:teuthology.orchestra.run.smithi043.stdout:(152/152): thrift-0.15.0-2.el9.x86_64.rpm 116 kB/s | 1.6 MB 00:13 2024-04-25T10:20:29.656 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:20:29.656 INFO:teuthology.orchestra.run.smithi043.stdout:Total 4.8 MB/s | 214 MB 00:44 2024-04-25T10:20:29.687 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T10:20:29.689 INFO:teuthology.orchestra.run.smithi077.stdout:(146/152): thrift-0.15.0-2.el9.x86_64.rpm 370 kB/s | 1.6 MB 00:04 2024-04-25T10:20:29.715 INFO:teuthology.orchestra.run.smithi077.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 42 kB/s | 192 kB 00:04 2024-04-25T10:20:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T10:20:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:20:29.867 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:20:29.889 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:20:29.935 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:20:29.957 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:20:29.984 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:20:30.090 INFO:teuthology.orchestra.run.smithi077.stdout:(148/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 8.2 MB/s | 3.1 MB 00:00 2024-04-25T10:20:30.133 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:20:30.165 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:20:30.199 INFO:teuthology.orchestra.run.smithi077.stdout:(149/152): librados2-19.0.0-3291.g994e2f20.el9. 6.5 MB/s | 3.3 MB 00:00 2024-04-25T10:20:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T10:20:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T10:20:30.414 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T10:20:30.449 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T10:20:30.482 INFO:teuthology.orchestra.run.smithi077.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 3.4 MB/s | 1.3 MB 00:00 2024-04-25T10:20:30.513 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T10:20:30.548 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T10:20:30.608 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T10:20:30.640 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T10:20:30.668 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T10:20:30.716 INFO:teuthology.orchestra.run.smithi077.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 4.6 MB/s | 2.4 MB 00:00 2024-04-25T10:20:30.738 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T10:20:30.759 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-04-25T10:20:30.791 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T10:20:30.829 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T10:20:30.865 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T10:20:30.921 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-04-25T10:20:30.921 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-04-25T10:20:30.941 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T10:20:30.975 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T10:20:31.019 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T10:20:31.047 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T10:20:31.117 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T10:20:31.163 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T10:20:31.237 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T10:20:31.381 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T10:20:31.431 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T10:20:31.499 INFO:teuthology.orchestra.run.smithi077.stdout:(152/152): python3-werkzeug-2.0.3-3.el9.1.noarc 66 kB/s | 427 kB 00:06 2024-04-25T10:20:31.502 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:20:31.503 INFO:teuthology.orchestra.run.smithi077.stdout:Total 4.7 MB/s | 214 MB 00:46 2024-04-25T10:20:31.949 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T10:20:32.035 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T10:20:32.073 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T10:20:32.115 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T10:20:32.153 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T10:20:32.195 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T10:20:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T10:20:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T10:20:32.469 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T10:20:32.542 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T10:20:32.582 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T10:20:32.607 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-04-25T10:20:32.618 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T10:20:32.664 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T10:20:32.701 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T10:20:32.744 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T10:20:32.751 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-04-25T10:20:32.751 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-04-25T10:20:32.758 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-04-25T10:20:32.759 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-04-25T10:20:32.786 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T10:20:32.922 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T10:20:32.975 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T10:20:33.005 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T10:20:33.046 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T10:20:33.149 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T10:20:33.189 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T10:20:33.229 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T10:20:33.254 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T10:20:33.278 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T10:20:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T10:20:33.479 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T10:20:33.519 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T10:20:33.547 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T10:20:33.598 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T10:20:33.645 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T10:20:33.677 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T10:20:33.707 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T10:20:33.741 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T10:20:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T10:20:33.816 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T10:20:33.850 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T10:20:33.937 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T10:20:33.981 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T10:20:34.020 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T10:20:34.067 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T10:20:34.166 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T10:20:34.221 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T10:20:34.476 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-04-25T10:20:34.477 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-04-25T10:20:34.948 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T10:20:35.002 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T10:20:35.038 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T10:20:35.145 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T10:20:35.211 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T10:20:35.299 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T10:20:35.346 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T10:20:35.525 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T10:20:35.560 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T10:20:35.647 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T10:20:36.293 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T10:20:36.550 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T10:20:37.779 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T10:20:37.864 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T10:20:37.892 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T10:20:37.925 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T10:20:38.245 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T10:20:38.288 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:20:38.378 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:20:38.409 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T10:20:38.445 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T10:20:38.776 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-04-25T10:20:38.878 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T10:20:38.926 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T10:20:38.935 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T10:20:38.965 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:20:38.980 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T10:20:39.030 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T10:20:39.036 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:20:39.055 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T10:20:39.058 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T10:20:39.098 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T10:20:39.451 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T10:20:39.564 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:20:39.692 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:20:39.723 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:20:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:20:39.845 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:20:40.001 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T10:20:40.059 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T10:20:40.094 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:20:40.184 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:20:40.212 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:20:40.275 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:20:40.288 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:20:40.308 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:20:40.354 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-04-25T10:20:40.428 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T10:20:40.488 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T10:20:40.497 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:20:40.530 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:20:40.546 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T10:20:40.587 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T10:20:40.613 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T10:20:40.640 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T10:20:40.655 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T10:20:40.738 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T10:20:40.771 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T10:20:40.806 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T10:20:40.882 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T10:20:40.921 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T10:20:40.965 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T10:20:40.998 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T10:20:40.998 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T10:20:41.025 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T10:20:41.098 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T10:20:41.114 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:20:41.149 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T10:20:41.178 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T10:20:41.222 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T10:20:41.229 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:20:41.237 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:20:41.265 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:20:41.280 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:20:41.309 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T10:20:41.339 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:20:41.341 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T10:20:41.378 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:20:41.385 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T10:20:41.413 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T10:20:41.486 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T10:20:41.525 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T10:20:41.536 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T10:20:41.575 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T10:20:41.603 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:20:41.614 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T10:20:41.708 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:20:41.737 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:20:41.759 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T10:20:41.800 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:20:41.815 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T10:20:41.822 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:20:41.835 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:20:41.848 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:20:41.884 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:20:41.975 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:20:41.975 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T10:20:41.975 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T10:20:41.975 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T10:20:42.001 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:20:42.047 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:20:42.089 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:20:42.201 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T10:20:42.294 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T10:20:42.338 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T10:20:42.373 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T10:20:42.380 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T10:20:42.447 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T10:20:42.471 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T10:20:42.480 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T10:20:42.505 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T10:20:42.523 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T10:20:42.548 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T10:20:42.556 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T10:20:42.584 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T10:20:42.586 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T10:20:42.628 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T10:20:42.652 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T10:20:42.706 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T10:20:42.737 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T10:20:42.772 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T10:20:42.782 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T10:20:42.839 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T10:20:42.847 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T10:20:42.883 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T10:20:42.895 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T10:20:42.926 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T10:20:42.963 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T10:20:42.969 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T10:20:43.006 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T10:20:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T10:20:43.043 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T10:20:43.078 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T10:20:43.100 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T10:20:43.142 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T10:20:43.160 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T10:20:43.186 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T10:20:43.228 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T10:20:43.302 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T10:20:43.372 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T10:20:43.373 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T10:20:43.416 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T10:20:43.446 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T10:20:43.488 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T10:20:43.585 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T10:20:43.621 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T10:20:43.653 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T10:20:43.678 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T10:20:43.703 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T10:20:43.781 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T10:20:43.888 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T10:20:43.919 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T10:20:43.952 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T10:20:43.968 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T10:20:43.980 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T10:20:43.997 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T10:20:44.015 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T10:20:44.031 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T10:20:44.054 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T10:20:44.069 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T10:20:44.086 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T10:20:44.111 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T10:20:44.115 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T10:20:44.141 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T10:20:44.166 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T10:20:44.208 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T10:20:44.242 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T10:20:44.271 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T10:20:44.333 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T10:20:44.339 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T10:20:44.382 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T10:20:44.393 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T10:20:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T10:20:44.458 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T10:20:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T10:20:44.490 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T10:20:44.518 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T10:20:44.564 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T10:20:44.572 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T10:20:44.601 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T10:20:44.632 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T10:20:44.644 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T10:20:44.686 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T10:20:44.820 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T10:20:44.867 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T10:20:44.897 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T10:20:44.939 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T10:20:45.032 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T10:20:45.080 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T10:20:45.120 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T10:20:45.146 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T10:20:45.171 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T10:20:45.245 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T10:20:45.372 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T10:20:45.401 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T10:20:45.403 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T10:20:45.431 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T10:20:45.445 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T10:20:45.466 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T10:20:45.478 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T10:20:45.505 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T10:20:45.536 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T10:20:45.566 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T10:20:45.582 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T10:20:45.592 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T10:20:45.626 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T10:20:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T10:20:45.667 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T10:20:45.702 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T10:20:45.728 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T10:20:45.764 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T10:20:45.787 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T10:20:45.832 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T10:20:45.864 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T10:20:45.910 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T10:20:45.949 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T10:20:45.978 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T10:20:46.008 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T10:20:46.058 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T10:20:46.066 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T10:20:46.750 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T10:20:46.777 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T10:20:46.829 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T10:20:46.863 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T10:20:46.872 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T10:20:46.960 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T10:20:47.021 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T10:20:47.107 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T10:20:47.140 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T10:20:47.317 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T10:20:47.346 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T10:20:47.423 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T10:20:48.063 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T10:20:48.177 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T10:20:48.195 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T10:20:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T10:20:48.310 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T10:20:48.344 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T10:20:48.682 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T10:20:48.714 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:20:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:20:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T10:20:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T10:20:49.391 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T10:20:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T10:20:49.426 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:20:49.481 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:20:49.485 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T10:20:49.512 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T10:20:49.546 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T10:20:49.546 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T10:20:49.865 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T10:20:49.909 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:20:49.999 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:20:50.030 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T10:20:50.066 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T10:20:50.552 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T10:20:50.587 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:20:50.657 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:20:50.690 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T10:20:51.901 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:20:51.942 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:20:52.532 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:20:52.579 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:20:52.670 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:20:52.670 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T10:20:52.670 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T10:20:52.671 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:20:52.705 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:20:52.945 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:20:52.985 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:20:53.468 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:20:53.468 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /sys 2024-04-25T10:20:53.468 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /proc 2024-04-25T10:20:53.468 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /mnt 2024-04-25T10:20:53.468 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /var/tmp 2024-04-25T10:20:53.469 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /home 2024-04-25T10:20:53.469 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /root 2024-04-25T10:20:53.469 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /tmp 2024-04-25T10:20:53.469 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T10:20:53.549 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:20:53.589 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:20:53.638 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:20:53.653 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:20:53.653 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:20:53.653 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T10:20:53.653 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T10:20:53.682 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:20:53.682 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T10:20:53.682 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T10:20:53.682 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-25T10:20:53.715 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:20:55.240 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:20:55.266 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:20:55.266 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:20:55.266 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T10:20:55.266 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:20:55.266 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:20:55.266 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T10:20:55.500 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:20:55.522 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:20:55.522 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:20:55.522 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T10:20:55.522 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:20:55.522 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:20:55.523 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T10:20:55.983 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:20:56.006 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:20:56.006 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:20:56.006 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T10:20:56.006 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:20:56.006 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:20:56.006 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T10:20:56.122 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T10:20:56.203 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T10:20:56.268 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T10:20:56.453 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T10:20:56.500 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T10:20:56.577 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T10:20:56.676 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:20:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:20:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T10:20:56.891 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T10:20:56.918 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T10:20:57.371 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:20:57.400 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:20:57.432 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T10:20:57.493 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T10:20:57.532 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:20:58.571 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:20:58.608 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:20:59.578 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:20:59.609 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:20:59.711 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:20:59.883 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T10:20:59.911 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:20:59.934 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:20:59.935 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:20:59.935 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T10:20:59.935 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:20:59.935 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:20:59.935 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T10:20:59.978 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:20:59.989 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:21:00.088 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T10:21:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T10:21:01.074 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T10:21:01.106 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:21:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:21:01.129 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:01.129 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T10:21:01.129 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:21:01.129 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:21:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T10:21:01.173 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:21:01.195 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:21:01.195 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:01.195 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T10:21:01.195 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T10:21:01.240 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:21:01.263 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:21:01.263 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:01.263 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T10:21:01.263 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:21:01.263 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:21:01.264 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T10:21:01.591 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:21:01.614 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:21:01.614 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:01.614 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T10:21:01.614 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:21:01.614 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:21:01.614 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T10:21:04.551 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:21:04.551 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /sys 2024-04-25T10:21:04.551 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /proc 2024-04-25T10:21:04.551 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /mnt 2024-04-25T10:21:04.551 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /var/tmp 2024-04-25T10:21:04.551 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /home 2024-04-25T10:21:04.552 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /root 2024-04-25T10:21:04.552 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /tmp 2024-04-25T10:21:04.552 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:21:04.734 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:21:04.751 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:21:04.751 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:04.751 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T10:21:04.751 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:21:05.216 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:21:05.216 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /sys 2024-04-25T10:21:05.216 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /proc 2024-04-25T10:21:05.216 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /mnt 2024-04-25T10:21:05.217 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /var/tmp 2024-04-25T10:21:05.217 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /home 2024-04-25T10:21:05.217 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /root 2024-04-25T10:21:05.217 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /tmp 2024-04-25T10:21:05.217 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-25T10:21:05.388 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:21:05.403 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:21:05.403 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:05.404 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T10:21:05.404 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-25T10:21:05.905 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T10:21:05.945 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T10:21:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T10:21:06.017 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T10:21:06.044 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T10:21:06.225 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T10:21:06.283 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T10:21:06.312 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T10:21:06.396 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:21:06.423 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:21:06.423 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:06.423 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T10:21:06.423 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:21:06.423 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:21:06.423 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:21:06.663 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:21:06.687 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:21:06.687 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:06.687 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T10:21:06.687 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:21:06.687 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:21:06.687 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:21:06.925 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:21:06.968 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:21:06.974 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:21:06.987 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:21:07.000 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:21:07.000 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:07.000 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T10:21:07.000 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:21:07.000 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:21:07.000 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-25T10:21:07.163 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:21:07.187 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:21:07.187 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:07.187 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T10:21:07.187 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:21:07.187 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:21:07.187 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:21:07.225 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:21:07.247 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:21:07.247 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:07.248 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T10:21:07.248 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:21:07.248 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:21:07.248 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-25T10:21:07.294 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T10:21:07.352 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T10:21:07.417 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T10:21:07.611 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T10:21:07.665 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T10:21:07.709 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:21:07.731 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:21:07.732 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:07.732 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T10:21:07.732 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:21:07.732 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:21:07.732 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-25T10:21:07.745 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T10:21:07.836 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T10:21:07.839 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T10:21:07.843 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:21:07.896 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T10:21:07.931 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T10:21:07.951 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T10:21:07.972 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T10:21:07.986 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:21:08.023 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T10:21:08.055 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T10:21:08.062 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T10:21:08.083 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T10:21:08.102 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T10:21:08.102 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:21:08.155 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T10:21:08.194 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:21:08.202 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T10:21:08.228 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:21:08.228 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:21:08.279 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T10:21:08.320 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:21:08.367 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:21:08.384 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:21:08.506 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:21:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:21:08.543 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T10:21:08.565 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:21:08.575 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T10:21:08.597 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T10:21:08.603 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T10:21:08.661 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T10:21:08.696 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:21:09.081 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:21:09.110 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:21:09.142 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T10:21:09.203 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T10:21:09.242 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:21:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:21:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:21:10.270 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:21:10.302 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:21:10.856 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:21:10.881 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:21:10.989 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:21:11.156 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T10:21:11.184 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:21:11.208 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:21:11.208 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:11.208 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T10:21:11.208 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:21:11.208 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:21:11.208 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:21:11.259 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:21:11.261 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:21:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:21:11.286 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:21:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T10:21:11.388 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:21:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T10:21:11.559 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T10:21:11.588 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:21:11.610 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:21:11.610 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:11.610 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T10:21:11.611 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:21:11.611 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:21:11.611 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-25T10:21:11.655 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:21:11.666 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:21:11.767 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T10:21:11.811 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T10:21:12.424 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T10:21:12.462 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:21:12.486 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:21:12.486 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:12.486 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T10:21:12.486 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:21:12.486 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:21:12.486 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:21:12.538 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:21:12.561 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:21:12.561 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:12.561 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T10:21:12.561 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:21:12.613 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:21:12.637 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:21:12.637 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:12.637 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T10:21:12.637 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:21:12.637 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:21:12.637 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:21:12.751 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T10:21:12.783 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:21:12.806 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:21:12.806 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:12.806 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T10:21:12.806 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:21:12.806 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:21:12.806 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-25T10:21:12.850 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:21:12.872 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:21:12.872 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:12.872 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T10:21:12.872 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-25T10:21:12.938 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:21:12.961 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:21:12.961 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:12.961 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T10:21:12.961 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:21:12.961 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:21:12.961 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-25T10:21:12.964 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:21:12.987 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:21:12.987 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:12.987 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T10:21:12.988 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:21:12.988 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:21:12.988 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:21:13.268 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:21:13.291 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:21:13.291 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:21:13.291 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T10:21:13.291 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:21:13.291 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:21:13.291 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-25T10:21:14.256 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:21:14.256 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T10:21:14.256 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T10:21:14.256 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T10:21:14.256 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T10:21:14.256 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T10:21:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T10:21:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T10:21:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:21:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:21:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:21:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T10:21:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T10:21:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:21:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T10:21:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T10:21:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T10:21:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T10:21:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T10:21:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T10:21:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T10:21:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T10:21:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T10:21:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T10:21:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T10:21:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T10:21:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T10:21:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T10:21:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T10:21:14.270 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T10:21:14.270 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T10:21:14.271 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T10:21:14.271 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T10:21:14.271 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T10:21:14.271 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T10:21:14.271 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T10:21:14.271 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T10:21:14.271 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T10:21:14.271 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T10:21:14.271 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T10:21:14.271 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T10:21:14.271 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T10:21:14.271 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T10:21:14.271 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T10:21:14.272 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T10:21:14.272 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T10:21:14.272 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T10:21:14.272 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T10:21:14.272 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T10:21:14.272 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T10:21:14.272 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T10:21:14.272 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T10:21:14.272 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T10:21:14.272 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T10:21:14.272 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T10:21:14.272 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T10:21:14.273 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T10:21:14.273 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T10:21:14.273 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T10:21:14.273 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T10:21:14.273 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T10:21:14.273 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T10:21:14.273 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T10:21:14.273 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T10:21:14.273 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T10:21:14.273 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T10:21:14.273 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T10:21:14.274 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T10:21:14.274 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T10:21:14.274 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T10:21:14.274 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T10:21:14.274 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T10:21:14.274 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T10:21:14.274 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T10:21:14.274 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T10:21:14.274 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T10:21:14.274 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T10:21:14.274 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T10:21:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T10:21:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T10:21:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T10:21:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T10:21:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T10:21:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T10:21:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T10:21:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T10:21:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T10:21:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T10:21:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T10:21:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T10:21:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T10:21:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T10:21:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T10:21:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T10:21:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T10:21:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T10:21:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T10:21:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T10:21:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T10:21:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T10:21:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T10:21:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T10:21:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T10:21:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T10:21:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:21:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:21:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T10:21:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T10:21:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T10:21:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T10:21:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T10:21:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T10:21:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T10:21:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T10:21:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T10:21:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T10:21:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T10:21:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T10:21:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T10:21:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T10:21:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T10:21:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T10:21:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T10:21:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T10:21:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T10:21:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T10:21:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T10:21:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T10:21:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T10:21:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T10:21:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T10:21:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T10:21:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T10:21:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T10:21:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T10:21:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T10:21:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T10:21:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T10:21:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T10:21:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T10:21:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:21:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T10:21:16.390 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:21:16.390 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T10:21:16.390 INFO:teuthology.orchestra.run.smithi164.stdout:Upgraded: 2024-04-25T10:21:16.390 INFO:teuthology.orchestra.run.smithi164.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.390 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T10:21:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T10:21:16.391 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-25T10:21:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T10:21:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T10:21:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T10:21:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T10:21:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T10:21:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T10:21:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T10:21:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T10:21:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T10:21:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T10:21:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T10:21:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T10:21:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T10:21:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T10:21:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T10:21:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T10:21:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T10:21:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T10:21:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T10:21:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T10:21:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T10:21:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T10:21:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T10:21:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T10:21:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T10:21:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T10:21:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T10:21:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T10:21:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T10:21:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T10:21:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T10:21:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T10:21:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T10:21:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T10:21:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T10:21:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T10:21:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T10:21:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T10:21:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T10:21:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T10:21:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T10:21:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T10:21:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T10:21:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T10:21:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T10:21:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T10:21:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T10:21:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T10:21:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T10:21:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T10:21:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T10:21:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T10:21:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T10:21:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T10:21:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T10:21:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T10:21:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T10:21:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T10:21:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T10:21:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T10:21:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T10:21:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T10:21:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T10:21:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T10:21:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T10:21:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T10:21:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T10:21:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T10:21:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T10:21:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T10:21:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T10:21:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T10:21:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T10:21:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T10:21:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T10:21:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T10:21:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T10:21:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T10:21:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T10:21:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T10:21:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T10:21:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T10:21:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T10:21:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T10:21:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T10:21:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T10:21:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T10:21:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T10:21:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T10:21:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T10:21:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T10:21:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T10:21:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T10:21:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T10:21:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T10:21:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T10:21:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T10:21:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T10:21:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T10:21:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T10:21:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T10:21:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T10:21:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T10:21:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T10:21:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T10:21:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T10:21:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T10:21:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T10:21:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T10:21:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T10:21:16.402 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-25T10:21:16.802 DEBUG:teuthology.parallel:result is None 2024-04-25T10:21:17.518 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T10:21:17.547 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T10:21:17.560 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T10:21:17.589 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T10:21:17.601 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T10:21:17.630 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T10:21:17.632 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T10:21:17.661 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T10:21:17.668 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T10:21:17.688 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T10:21:17.864 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T10:21:17.870 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T10:21:17.905 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T10:21:17.910 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T10:21:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T10:21:17.949 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T10:21:18.494 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:21:18.530 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:21:18.549 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:21:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:21:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:21:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:21:19.369 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T10:21:19.469 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T10:21:19.509 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T10:21:19.534 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T10:21:19.589 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T10:21:19.623 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T10:21:19.623 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:21:19.629 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T10:21:19.672 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T10:21:19.732 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:21:19.756 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T10:21:19.765 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:21:19.765 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:21:19.792 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T10:21:19.792 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:21:19.856 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:21:19.868 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:21:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:21:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:21:19.922 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:21:19.989 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:21:20.061 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:21:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:21:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T10:21:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T10:21:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T10:21:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T10:21:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T10:21:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T10:21:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T10:21:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:21:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:21:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:21:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T10:21:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T10:21:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:21:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T10:21:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T10:21:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T10:21:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T10:21:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T10:21:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T10:21:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T10:21:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T10:21:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T10:21:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T10:21:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T10:21:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T10:21:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T10:21:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T10:21:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T10:21:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T10:21:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T10:21:25.839 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T10:21:25.839 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T10:21:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T10:21:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T10:21:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T10:21:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T10:21:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T10:21:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T10:21:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T10:21:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T10:21:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T10:21:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T10:21:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T10:21:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T10:21:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T10:21:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T10:21:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T10:21:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T10:21:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T10:21:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T10:21:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T10:21:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T10:21:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T10:21:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T10:21:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T10:21:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T10:21:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T10:21:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T10:21:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T10:21:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T10:21:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T10:21:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T10:21:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T10:21:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T10:21:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T10:21:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T10:21:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T10:21:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T10:21:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T10:21:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T10:21:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T10:21:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T10:21:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T10:21:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T10:21:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T10:21:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T10:21:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T10:21:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T10:21:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T10:21:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T10:21:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T10:21:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T10:21:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T10:21:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T10:21:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T10:21:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T10:21:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T10:21:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T10:21:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T10:21:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T10:21:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T10:21:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T10:21:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T10:21:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T10:21:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T10:21:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T10:21:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T10:21:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T10:21:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T10:21:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T10:21:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T10:21:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T10:21:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T10:21:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T10:21:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:21:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:21:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T10:21:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T10:21:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T10:21:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T10:21:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T10:21:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T10:21:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T10:21:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T10:21:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T10:21:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T10:21:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T10:21:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T10:21:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T10:21:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T10:21:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T10:21:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T10:21:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T10:21:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T10:21:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T10:21:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T10:21:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T10:21:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T10:21:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T10:21:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T10:21:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T10:21:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T10:21:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T10:21:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T10:21:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T10:21:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T10:21:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T10:21:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T10:21:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T10:21:25.849 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T10:21:25.849 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:21:25.849 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T10:21:25.857 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:21:25.857 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T10:21:25.858 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T10:21:25.858 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T10:21:25.858 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T10:21:25.858 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T10:21:25.858 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T10:21:25.858 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T10:21:25.858 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:21:25.858 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:21:25.858 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:21:25.858 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T10:21:25.858 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T10:21:25.858 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:21:25.859 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T10:21:25.859 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T10:21:25.859 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T10:21:25.859 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T10:21:25.859 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T10:21:25.859 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T10:21:25.859 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T10:21:25.859 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T10:21:25.859 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T10:21:25.859 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T10:21:25.859 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T10:21:25.859 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T10:21:25.859 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T10:21:25.860 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T10:21:25.860 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T10:21:25.860 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T10:21:25.860 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T10:21:25.860 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T10:21:25.860 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T10:21:25.860 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T10:21:25.860 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T10:21:25.860 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T10:21:25.860 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T10:21:25.860 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T10:21:25.860 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T10:21:25.860 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T10:21:25.860 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T10:21:25.861 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T10:21:25.861 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T10:21:25.861 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T10:21:25.861 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T10:21:25.861 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T10:21:25.861 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T10:21:25.861 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T10:21:25.861 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T10:21:25.861 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T10:21:25.861 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T10:21:25.861 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T10:21:25.861 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T10:21:25.861 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T10:21:25.861 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T10:21:25.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T10:21:25.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T10:21:25.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T10:21:25.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T10:21:25.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T10:21:25.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T10:21:25.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T10:21:25.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T10:21:25.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T10:21:25.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T10:21:25.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T10:21:25.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T10:21:25.863 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T10:21:25.863 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T10:21:25.863 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T10:21:25.863 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T10:21:25.863 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T10:21:25.863 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T10:21:25.863 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T10:21:25.863 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T10:21:25.863 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T10:21:25.863 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T10:21:25.863 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T10:21:25.863 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T10:21:25.863 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T10:21:25.863 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T10:21:25.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T10:21:25.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T10:21:25.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T10:21:25.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T10:21:25.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T10:21:25.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T10:21:25.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T10:21:25.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T10:21:25.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T10:21:25.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T10:21:25.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T10:21:25.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T10:21:25.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T10:21:25.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T10:21:25.865 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T10:21:25.865 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T10:21:25.865 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T10:21:25.865 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T10:21:25.865 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T10:21:25.865 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T10:21:25.865 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T10:21:25.865 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T10:21:25.865 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T10:21:25.865 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T10:21:25.865 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T10:21:25.865 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T10:21:25.865 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T10:21:25.865 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T10:21:25.866 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T10:21:25.866 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T10:21:25.866 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T10:21:25.866 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T10:21:25.866 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T10:21:25.866 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T10:21:25.866 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T10:21:25.866 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T10:21:25.866 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T10:21:25.866 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:21:25.866 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:21:25.866 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T10:21:25.866 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T10:21:25.867 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T10:21:25.867 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T10:21:25.867 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T10:21:25.867 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T10:21:25.867 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T10:21:25.867 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T10:21:25.867 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T10:21:25.867 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T10:21:25.867 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T10:21:25.867 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T10:21:25.867 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T10:21:25.867 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T10:21:25.867 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T10:21:25.867 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T10:21:25.868 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T10:21:25.868 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T10:21:25.868 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T10:21:25.868 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T10:21:25.868 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T10:21:25.868 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T10:21:25.868 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T10:21:25.868 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T10:21:25.868 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T10:21:25.868 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T10:21:25.868 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T10:21:25.868 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T10:21:25.868 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T10:21:25.868 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T10:21:25.869 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T10:21:25.869 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T10:21:25.869 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T10:21:25.869 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T10:21:25.869 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:21:25.869 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T10:21:27.925 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:21:27.925 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:21:27.925 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2024-04-25T10:21:27.925 INFO:teuthology.orchestra.run.smithi043.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.925 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.925 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T10:21:27.925 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T10:21:27.925 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-04-25T10:21:27.925 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T10:21:27.925 INFO:teuthology.orchestra.run.smithi043.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T10:21:27.925 INFO:teuthology.orchestra.run.smithi043.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T10:21:27.926 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T10:21:27.926 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.926 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.926 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.926 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.926 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:27.926 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.926 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.926 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.926 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:27.926 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:27.926 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:27.926 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:27.927 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:27.927 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.927 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.927 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:27.927 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.927 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.927 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.927 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:27.927 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:27.927 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.927 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.927 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:27.928 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T10:21:27.928 INFO:teuthology.orchestra.run.smithi043.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T10:21:27.928 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T10:21:27.928 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T10:21:27.928 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T10:21:27.928 INFO:teuthology.orchestra.run.smithi043.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T10:21:27.928 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T10:21:27.928 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T10:21:27.928 INFO:teuthology.orchestra.run.smithi043.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T10:21:27.928 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T10:21:27.928 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T10:21:27.928 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.928 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.928 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.929 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T10:21:27.929 INFO:teuthology.orchestra.run.smithi043.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T10:21:27.929 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T10:21:27.929 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T10:21:27.929 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T10:21:27.929 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.929 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.929 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T10:21:27.929 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.929 INFO:teuthology.orchestra.run.smithi043.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T10:21:27.929 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T10:21:27.929 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T10:21:27.929 INFO:teuthology.orchestra.run.smithi043.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T10:21:27.929 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T10:21:27.930 INFO:teuthology.orchestra.run.smithi043.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T10:21:27.930 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T10:21:27.930 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T10:21:27.930 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T10:21:27.930 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T10:21:27.930 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T10:21:27.930 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T10:21:27.930 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T10:21:27.930 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T10:21:27.930 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T10:21:27.930 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T10:21:27.930 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T10:21:27.931 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T10:21:27.931 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T10:21:27.931 INFO:teuthology.orchestra.run.smithi043.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T10:21:27.931 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T10:21:27.931 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T10:21:27.931 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T10:21:27.931 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.931 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.931 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.931 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T10:21:27.931 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T10:21:27.931 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T10:21:27.931 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T10:21:27.931 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T10:21:27.931 INFO:teuthology.orchestra.run.smithi043.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T10:21:27.932 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T10:21:27.932 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T10:21:27.932 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T10:21:27.932 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T10:21:27.932 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T10:21:27.932 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T10:21:27.932 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T10:21:27.932 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T10:21:27.932 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T10:21:27.932 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T10:21:27.932 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T10:21:27.932 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T10:21:27.932 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T10:21:27.932 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T10:21:27.933 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T10:21:27.933 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T10:21:27.933 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T10:21:27.933 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T10:21:27.933 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T10:21:27.933 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T10:21:27.933 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T10:21:27.933 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T10:21:27.933 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T10:21:27.933 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T10:21:27.933 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T10:21:27.933 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T10:21:27.933 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T10:21:27.933 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T10:21:27.934 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T10:21:27.934 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T10:21:27.934 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T10:21:27.934 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T10:21:27.934 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T10:21:27.934 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T10:21:27.934 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.934 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.934 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T10:21:27.934 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T10:21:27.934 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T10:21:27.934 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.934 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T10:21:27.935 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T10:21:27.935 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T10:21:27.935 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T10:21:27.935 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T10:21:27.935 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T10:21:27.935 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T10:21:27.935 INFO:teuthology.orchestra.run.smithi043.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T10:21:27.935 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T10:21:27.935 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T10:21:27.935 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T10:21:27.935 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T10:21:27.935 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T10:21:27.935 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T10:21:27.935 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T10:21:27.936 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.936 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.936 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.936 INFO:teuthology.orchestra.run.smithi043.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T10:21:27.936 INFO:teuthology.orchestra.run.smithi043.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T10:21:27.936 INFO:teuthology.orchestra.run.smithi043.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T10:21:27.936 INFO:teuthology.orchestra.run.smithi043.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T10:21:27.936 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T10:21:27.936 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T10:21:27.936 INFO:teuthology.orchestra.run.smithi043.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T10:21:27.937 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:21:27.937 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-04-25T10:21:27.973 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:21:27.974 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-25T10:21:27.974 INFO:teuthology.orchestra.run.smithi077.stdout:Upgraded: 2024-04-25T10:21:27.974 INFO:teuthology.orchestra.run.smithi077.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.974 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.974 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T10:21:27.974 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T10:21:27.974 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-04-25T10:21:27.974 INFO:teuthology.orchestra.run.smithi077.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T10:21:27.974 INFO:teuthology.orchestra.run.smithi077.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T10:21:27.974 INFO:teuthology.orchestra.run.smithi077.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T10:21:27.975 INFO:teuthology.orchestra.run.smithi077.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T10:21:27.975 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.975 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.975 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.975 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.975 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:27.975 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.975 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.975 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.975 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:27.975 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:27.975 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:27.976 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:27.976 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:27.976 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.976 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.976 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:27.976 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.976 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.976 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.976 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:27.976 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:27.976 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.976 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.976 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:21:27.976 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T10:21:27.976 INFO:teuthology.orchestra.run.smithi077.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T10:21:27.977 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T10:21:27.977 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T10:21:27.977 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T10:21:27.977 INFO:teuthology.orchestra.run.smithi077.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T10:21:27.977 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T10:21:27.977 INFO:teuthology.orchestra.run.smithi077.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T10:21:27.977 INFO:teuthology.orchestra.run.smithi077.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T10:21:27.977 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T10:21:27.977 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T10:21:27.977 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.977 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.977 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.977 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T10:21:27.977 INFO:teuthology.orchestra.run.smithi077.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T10:21:27.978 INFO:teuthology.orchestra.run.smithi077.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T10:21:27.978 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T10:21:27.978 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T10:21:27.978 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.978 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.978 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T10:21:27.978 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.978 INFO:teuthology.orchestra.run.smithi077.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T10:21:27.978 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T10:21:27.978 INFO:teuthology.orchestra.run.smithi077.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T10:21:27.978 INFO:teuthology.orchestra.run.smithi077.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T10:21:27.978 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T10:21:27.978 INFO:teuthology.orchestra.run.smithi077.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T10:21:27.979 INFO:teuthology.orchestra.run.smithi077.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T10:21:27.979 INFO:teuthology.orchestra.run.smithi077.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T10:21:27.979 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T10:21:27.979 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T10:21:27.979 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T10:21:27.979 INFO:teuthology.orchestra.run.smithi077.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T10:21:27.979 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T10:21:27.979 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T10:21:27.979 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T10:21:27.979 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T10:21:27.979 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T10:21:27.979 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T10:21:27.979 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T10:21:27.979 INFO:teuthology.orchestra.run.smithi077.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T10:21:27.980 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T10:21:27.980 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T10:21:27.980 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T10:21:27.980 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.980 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.980 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.980 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T10:21:27.980 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T10:21:27.980 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T10:21:27.980 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T10:21:27.980 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T10:21:27.980 INFO:teuthology.orchestra.run.smithi077.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T10:21:27.980 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T10:21:27.980 INFO:teuthology.orchestra.run.smithi077.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T10:21:27.980 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T10:21:27.981 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T10:21:27.981 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T10:21:27.981 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T10:21:27.981 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T10:21:27.981 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T10:21:27.981 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T10:21:27.981 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T10:21:27.981 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T10:21:27.981 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T10:21:27.981 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T10:21:27.981 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T10:21:27.981 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T10:21:27.981 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T10:21:27.982 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T10:21:27.982 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T10:21:27.982 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T10:21:27.982 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T10:21:27.982 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T10:21:27.982 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T10:21:27.982 INFO:teuthology.orchestra.run.smithi077.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T10:21:27.982 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T10:21:27.982 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T10:21:27.982 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T10:21:27.982 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T10:21:27.982 INFO:teuthology.orchestra.run.smithi077.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T10:21:27.982 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T10:21:27.983 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T10:21:27.983 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T10:21:27.983 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T10:21:27.983 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T10:21:27.983 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T10:21:27.983 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.983 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.983 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T10:21:27.983 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T10:21:27.983 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T10:21:27.983 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.983 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T10:21:27.983 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T10:21:27.983 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T10:21:27.984 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T10:21:27.984 INFO:teuthology.orchestra.run.smithi077.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T10:21:27.984 INFO:teuthology.orchestra.run.smithi077.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T10:21:27.984 INFO:teuthology.orchestra.run.smithi077.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T10:21:27.984 INFO:teuthology.orchestra.run.smithi077.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T10:21:27.984 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T10:21:27.984 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T10:21:27.984 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T10:21:27.984 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T10:21:27.984 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T10:21:27.984 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T10:21:27.984 INFO:teuthology.orchestra.run.smithi077.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T10:21:27.984 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.984 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.984 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:21:27.985 INFO:teuthology.orchestra.run.smithi077.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T10:21:27.985 INFO:teuthology.orchestra.run.smithi077.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T10:21:27.985 INFO:teuthology.orchestra.run.smithi077.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T10:21:27.985 INFO:teuthology.orchestra.run.smithi077.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T10:21:27.985 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T10:21:27.985 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T10:21:27.985 INFO:teuthology.orchestra.run.smithi077.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T10:21:27.985 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-25T10:21:27.985 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-25T10:21:28.309 DEBUG:teuthology.parallel:result is None 2024-04-25T10:21:28.361 DEBUG:teuthology.parallel:result is None 2024-04-25T10:21:28.362 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:21:28.511 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T10:21:28.533 INFO:teuthology.orchestra.run.smithi043.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T10:21:28.533 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T10:21:28.534 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T10:21:28.535 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:21:28.678 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T10:21:28.699 INFO:teuthology.orchestra.run.smithi077.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T10:21:28.699 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T10:21:28.699 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T10:21:28.701 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:21:28.877 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T10:21:28.903 INFO:teuthology.orchestra.run.smithi164.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T10:21:28.903 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T10:21:28.903 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T10:21:28.905 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T10:21:28.905 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T10:21:28.905 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:21:28.934 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-25T10:21:28.934 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:21:28.960 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T10:21:28.960 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:21:28.990 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T10:21:28.990 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T10:21:28.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:21:29.015 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:21:29.079 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-25T10:21:29.079 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:21:29.102 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:21:29.166 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T10:21:29.166 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:21:29.194 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:21:29.259 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T10:21:29.259 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T10:21:29.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:21:29.284 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:21:29.350 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-25T10:21:29.350 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:21:29.373 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:21:29.436 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T10:21:29.436 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:21:29.460 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:21:29.524 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T10:21:29.524 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T10:21:29.525 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:21:29.549 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:21:29.614 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-25T10:21:29.614 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:21:29.637 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:21:29.700 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T10:21:29.700 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:21:29.725 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:21:29.789 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T10:21:29.893 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-25T10:21:29.893 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:21:29.894 INFO:tasks.cephadm:Cluster fsid is 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:21:29.894 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T10:21:29.894 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.43', 'mon.b': '172.21.15.77', 'mon.c': '172.21.15.164'} 2024-04-25T10:21:29.894 INFO:tasks.cephadm:First mon is mon.a on smithi043 2024-04-25T10:21:29.894 INFO:tasks.cephadm:First mgr is x 2024-04-25T10:21:29.894 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T10:21:29.894 DEBUG:teuthology.orchestra.run.smithi043:> sudo hostname $(hostname -s) 2024-04-25T10:21:29.920 DEBUG:teuthology.orchestra.run.smithi077:> sudo hostname $(hostname -s) 2024-04-25T10:21:29.944 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname $(hostname -s) 2024-04-25T10:21:29.968 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T10:21:29.968 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:21:30.119 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78872/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3291-g994e2f20', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3291.g994e2f20'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-25 04:02:18.990013', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240425.015853-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-04-25T10:21:30.255 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240425.015853-debug, sha1 994e2f20102a20fa9a27966c14b625cc01884df8 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:21:30.257 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm 2024-04-25T10:21:30.257 DEBUG:teuthology.orchestra.run.smithi043:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T10:21:30.701 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 10:21 /home/ubuntu/cephtest/cephadm 2024-04-25T10:21:30.702 DEBUG:teuthology.orchestra.run.smithi077:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T10:21:31.127 INFO:teuthology.orchestra.run.smithi077.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 10:21 /home/ubuntu/cephtest/cephadm 2024-04-25T10:21:31.127 DEBUG:teuthology.orchestra.run.smithi164:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T10:21:31.514 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 10:21 /home/ubuntu/cephtest/cephadm 2024-04-25T10:21:31.515 DEBUG:teuthology.orchestra.run.smithi043:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T10:21:31.532 DEBUG:teuthology.orchestra.run.smithi077:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T10:21:31.548 DEBUG:teuthology.orchestra.run.smithi164:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T10:21:31.583 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 on all hosts... 2024-04-25T10:21:31.583 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T10:21:31.585 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T10:21:31.591 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T10:21:33.139 INFO:teuthology.orchestra.run.smithi077.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T10:21:33.347 INFO:teuthology.orchestra.run.smithi164.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T10:21:33.832 INFO:teuthology.orchestra.run.smithi043.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T10:22:06.822 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-25T10:22:06.823 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T10:22:06.823 INFO:teuthology.orchestra.run.smithi164.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T10:22:06.823 INFO:teuthology.orchestra.run.smithi164.stdout: "repo_digests": [ 2024-04-25T10:22:06.823 INFO:teuthology.orchestra.run.smithi164.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T10:22:06.823 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-04-25T10:22:06.823 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-25T10:22:14.002 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:22:14.002 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T10:22:14.002 INFO:teuthology.orchestra.run.smithi043.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T10:22:14.002 INFO:teuthology.orchestra.run.smithi043.stdout: "repo_digests": [ 2024-04-25T10:22:14.003 INFO:teuthology.orchestra.run.smithi043.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T10:22:14.003 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T10:22:14.003 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:22:19.283 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-25T10:22:19.283 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T10:22:19.284 INFO:teuthology.orchestra.run.smithi077.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T10:22:19.284 INFO:teuthology.orchestra.run.smithi077.stdout: "repo_digests": [ 2024-04-25T10:22:19.284 INFO:teuthology.orchestra.run.smithi077.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T10:22:19.284 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-04-25T10:22:19.284 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-25T10:22:19.308 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph 2024-04-25T10:22:19.344 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /etc/ceph 2024-04-25T10:22:19.371 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph 2024-04-25T10:22:19.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /etc/ceph 2024-04-25T10:22:19.434 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 777 /etc/ceph 2024-04-25T10:22:19.457 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /etc/ceph 2024-04-25T10:22:19.485 INFO:tasks.cephadm:Writing seed config... 2024-04-25T10:22:19.487 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T10:22:19.487 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T10:22:19.487 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T10:22:19.487 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T10:22:19.487 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T10:22:19.488 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T10:22:19.488 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T10:22:19.488 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T10:22:19.488 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T10:22:19.489 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T10:22:19.489 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T10:22:19.489 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T10:22:19.489 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T10:22:19.489 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T10:22:19.489 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T10:22:19.490 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T10:22:19.490 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T10:22:19.490 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-25T10:22:19.490 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-25T10:22:19.490 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T10:22:19.490 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T10:22:19.491 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T10:22:19.491 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T10:22:19.491 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T10:22:19.491 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T10:22:19.491 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T10:22:19.491 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T10:22:19.491 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T10:22:19.492 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T10:22:19.492 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T10:22:19.492 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T10:22:19.492 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T10:22:19.492 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-25T10:22:19.492 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-25T10:22:19.492 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-25T10:22:19.492 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T10:22:19.492 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T10:22:19.492 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-25T10:22:19.492 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-25T10:22:19.492 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T10:22:19.492 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T10:22:19.492 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-25T10:22:19.493 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T10:22:19.493 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T10:22:19.493 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T10:22:19.493 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T10:22:19.493 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T10:22:19.493 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T10:22:19.493 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T10:22:19.493 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T10:22:19.494 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T10:22:19.494 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T10:22:19.512 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 94d0035c-02ed-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T10:22:19.513 DEBUG:teuthology.orchestra.run.smithi043:mon.a> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T10:22:19.555 DEBUG:teuthology.orchestra.run.smithi043:mgr.x> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T10:22:19.598 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T10:22:19.598 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 -v bootstrap --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.43 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T10:22:19.802 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:22:19.802 INFO:teuthology.orchestra.run.smithi043.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8', '-v', 'bootstrap', '--fsid', '94d0035c-02ed-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.43', '--skip-admin-label'] 2024-04-25T10:22:19.808 INFO:teuthology.orchestra.run.smithi043.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T10:22:19.808 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying podman|docker is present... 2024-04-25T10:22:19.835 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T10:22:19.835 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying lvm2 is present... 2024-04-25T10:22:19.836 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying time synchronization is in place... 2024-04-25T10:22:19.843 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T10:22:19.843 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T10:22:19.849 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T10:22:19.849 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-04-25T10:22:19.856 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout enabled 2024-04-25T10:22:19.862 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout active 2024-04-25T10:22:19.862 INFO:teuthology.orchestra.run.smithi043.stdout:Unit chronyd.service is enabled and running 2024-04-25T10:22:19.862 INFO:teuthology.orchestra.run.smithi043.stdout:Repeating the final host check... 2024-04-25T10:22:19.961 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T10:22:19.961 INFO:teuthology.orchestra.run.smithi043.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T10:22:19.961 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl is present 2024-04-25T10:22:19.962 INFO:teuthology.orchestra.run.smithi043.stdout:lvcreate is present 2024-04-25T10:22:19.968 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T10:22:19.968 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T10:22:19.973 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T10:22:19.974 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-04-25T10:22:19.980 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout enabled 2024-04-25T10:22:19.986 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout active 2024-04-25T10:22:19.987 INFO:teuthology.orchestra.run.smithi043.stdout:Unit chronyd.service is enabled and running 2024-04-25T10:22:19.987 INFO:teuthology.orchestra.run.smithi043.stdout:Host looks OK 2024-04-25T10:22:19.987 INFO:teuthology.orchestra.run.smithi043.stdout:Cluster fsid: 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:22:19.987 INFO:teuthology.orchestra.run.smithi043.stdout:Acquiring lock 139939956173984 on /run/cephadm/94d0035c-02ed-11ef-bc93-c7b262605968.lock 2024-04-25T10:22:19.987 INFO:teuthology.orchestra.run.smithi043.stdout:Lock 139939956173984 acquired on /run/cephadm/94d0035c-02ed-11ef-bc93-c7b262605968.lock 2024-04-25T10:22:19.987 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying IP 172.21.15.43 port 3300 ... 2024-04-25T10:22:19.987 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying IP 172.21.15.43 port 6789 ... 2024-04-25T10:22:19.988 INFO:teuthology.orchestra.run.smithi043.stdout:Base mon IP(s) is [172.21.15.43:3300, 172.21.15.43:6789], mon addrv is [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-04-25T10:22:19.991 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.43 metric 100 2024-04-25T10:22:19.991 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.43 metric 100 2024-04-25T10:22:19.993 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T10:22:19.993 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-25T10:22:19.994 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-25T10:22:19.996 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T10:22:19.996 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T10:22:19.996 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T10:22:19.996 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-25T10:22:19.996 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-25T10:22:19.996 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T10:22:19.997 INFO:teuthology.orchestra.run.smithi043.stdout:Mon IP `172.21.15.43` is in CIDR network `172.21.0.0/20` 2024-04-25T10:22:19.997 INFO:teuthology.orchestra.run.smithi043.stdout:Mon IP `172.21.15.43` is in CIDR network `172.21.0.0/20` 2024-04-25T10:22:19.997 INFO:teuthology.orchestra.run.smithi043.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T10:22:19.997 INFO:teuthology.orchestra.run.smithi043.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T10:22:19.998 INFO:teuthology.orchestra.run.smithi043.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T10:22:23.513 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 2024-04-25T10:22:23.513 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T10:22:23.513 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T10:22:23.513 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-25T10:22:23.513 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying blob sha256:665339d96aad80d083ba9988a62422bcef0a42210fb2b28340e33ab98589f7b8 2024-04-25T10:22:23.513 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying config sha256:d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 2024-04-25T10:22:23.513 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T10:22:28.966 INFO:teuthology.orchestra.run.smithi043.stdout:ceph: stdout ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T10:22:28.966 INFO:teuthology.orchestra.run.smithi043.stdout:Ceph version: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T10:22:28.967 INFO:teuthology.orchestra.run.smithi043.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T10:22:30.013 INFO:teuthology.orchestra.run.smithi043.stdout:stat: stdout 167 167 2024-04-25T10:22:30.013 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial keys... 2024-04-25T10:22:31.780 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQDmLipmZwqNFRAANoz8MwPwbQK9qPs1BngR4Q== 2024-04-25T10:22:32.716 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQDoLipmAn3/CBAARzN53yalRmluhjO+GL0bew== 2024-04-25T10:22:33.538 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQDpLipmxD3RABAAp4GgZICPxtl764RTWwbsWA== 2024-04-25T10:22:33.538 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial monmap... 2024-04-25T10:22:35.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T10:22:35.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T10:22:35.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:22:35.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T10:22:35.402 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool for a [v2:172.21.15.43:3300,v1:172.21.15.43:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T10:22:35.402 INFO:teuthology.orchestra.run.smithi043.stdout:setting min_mon_release = quincy 2024-04-25T10:22:35.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: set fsid to 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:22:35.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T10:22:35.402 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:22:35.402 INFO:teuthology.orchestra.run.smithi043.stdout:Creating mon... 2024-04-25T10:22:36.364 INFO:teuthology.orchestra.run.smithi043.stdout:create mon.a on 2024-04-25T10:22:36.688 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T10:22:36.841 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T10:22:37.010 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-94d0035c-02ed-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-94d0035c-02ed-11ef-bc93-c7b262605968.target. 2024-04-25T10:22:37.010 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-94d0035c-02ed-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-94d0035c-02ed-11ef-bc93-c7b262605968.target. 2024-04-25T10:22:37.250 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mon.a 2024-04-25T10:22:37.250 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to reset failed state of unit ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T10:22:37.413 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-94d0035c-02ed-11ef-bc93-c7b262605968.target.wants/ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-94d0035c-02ed-11ef-bc93-c7b262605968@.service. 2024-04-25T10:22:37.881 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T10:22:37.881 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-04-25T10:22:37.888 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T10:22:37.888 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-04-25T10:22:37.888 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-04-25T10:22:37.888 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T10:22:37.889 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mon to start... 2024-04-25T10:22:37.889 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mon... 2024-04-25T10:22:38.010 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:37 smithi043 systemd[1]: Started Ceph mon.a for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:22:38.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T10:22:38.876 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout id: 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:22:38.876 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T10:22:38.876 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-25T10:22:38.876 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout services: 2024-04-25T10:22:38.876 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.510291s) 2024-04-25T10:22:38.877 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T10:22:38.877 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T10:22:38.877 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-25T10:22:38.877 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout data: 2024-04-25T10:22:38.877 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T10:22:38.877 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T10:22:38.877 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T10:22:38.877 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T10:22:38.877 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-25T10:22:38.877 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.289+0000 7f9c2e7d3700 1 Processor -- start 2024-04-25T10:22:38.877 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.289+0000 7f9c2e7d3700 1 -- start start 2024-04-25T10:22:38.877 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.289+0000 7f9c2e7d3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c28108ae0 0x7f9c28108ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:38.877 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.289+0000 7f9c2e7d3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281094e0 con 0x7f9c28108ae0 2024-04-25T10:22:38.877 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.290+0000 7f9c27fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c28108ae0 0x7f9c28108ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:38.878 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.290+0000 7f9c27fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c28108ae0 0x7f9c28108ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60400/0 (socket says 172.21.15.43:60400) 2024-04-25T10:22:38.878 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.290+0000 7f9c27fff700 1 -- 172.21.15.43:0/2969161702 learned_addr learned my addr 172.21.15.43:0/2969161702 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:38.878 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.290+0000 7f9c27fff700 1 -- 172.21.15.43:0/2969161702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c28109cb0 con 0x7f9c28108ae0 2024-04-25T10:22:38.878 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.290+0000 7f9c27fff700 1 --2- 172.21.15.43:0/2969161702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c28108ae0 0x7f9c28108ef0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800bb40 tx=0x7f9c1800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534d995d49ea8993 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:38.878 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.290+0000 7f9c26ffd700 1 -- 172.21.15.43:0/2969161702 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c1800dbe0 con 0x7f9c28108ae0 2024-04-25T10:22:38.878 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.291+0000 7f9c26ffd700 1 -- 172.21.15.43:0/2969161702 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c1800dd80 con 0x7f9c28108ae0 2024-04-25T10:22:38.878 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.291+0000 7f9c26ffd700 1 -- 172.21.15.43:0/2969161702 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c18004180 con 0x7f9c28108ae0 2024-04-25T10:22:38.878 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.291+0000 7f9c2e7d3700 1 -- 172.21.15.43:0/2969161702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c28108ae0 msgr2=0x7f9c28108ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:38.878 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.291+0000 7f9c2e7d3700 1 --2- 172.21.15.43:0/2969161702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c28108ae0 0x7f9c28108ef0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800bb40 tx=0x7f9c1800be50 comp rx=0 tx=0).stop 2024-04-25T10:22:38.878 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.291+0000 7f9c2e7d3700 1 -- 172.21.15.43:0/2969161702 shutdown_connections 2024-04-25T10:22:38.878 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.291+0000 7f9c2e7d3700 1 --2- 172.21.15.43:0/2969161702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c28108ae0 0x7f9c28108ef0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.878 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.291+0000 7f9c2e7d3700 1 -- 172.21.15.43:0/2969161702 >> 172.21.15.43:0/2969161702 conn(0x7f9c281041c0 msgr2=0x7f9c28106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:38.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.291+0000 7f9c2e7d3700 1 -- 172.21.15.43:0/2969161702 shutdown_connections 2024-04-25T10:22:38.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.291+0000 7f9c2e7d3700 1 -- 172.21.15.43:0/2969161702 wait complete. 2024-04-25T10:22:38.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.292+0000 7f9c2e7d3700 1 Processor -- start 2024-04-25T10:22:38.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.292+0000 7f9c2e7d3700 1 -- start start 2024-04-25T10:22:38.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.293+0000 7f9c2e7d3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c28108ae0 0x7f9c2819b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:38.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.293+0000 7f9c2e7d3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2810a180 con 0x7f9c28108ae0 2024-04-25T10:22:38.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.293+0000 7f9c27fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c28108ae0 0x7f9c2819b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:38.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.293+0000 7f9c27fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c28108ae0 0x7f9c2819b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60412/0 (socket says 172.21.15.43:60412) 2024-04-25T10:22:38.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.293+0000 7f9c27fff700 1 -- 172.21.15.43:0/1003031041 learned_addr learned my addr 172.21.15.43:0/1003031041 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:38.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.293+0000 7f9c27fff700 1 -- 172.21.15.43:0/1003031041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2819ba60 con 0x7f9c28108ae0 2024-04-25T10:22:38.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.293+0000 7f9c27fff700 1 --2- 172.21.15.43:0/1003031041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c28108ae0 0x7f9c2819b520 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800b340 tx=0x7f9c18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:38.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.293+0000 7f9c257fa700 1 -- 172.21.15.43:0/1003031041 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c18029070 con 0x7f9c28108ae0 2024-04-25T10:22:38.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.293+0000 7f9c257fa700 1 -- 172.21.15.43:0/1003031041 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c180044e0 con 0x7f9c28108ae0 2024-04-25T10:22:38.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.293+0000 7f9c2e7d3700 1 -- 172.21.15.43:0/1003031041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c2819bcf0 con 0x7f9c28108ae0 2024-04-25T10:22:38.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.293+0000 7f9c257fa700 1 -- 172.21.15.43:0/1003031041 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c180105a0 con 0x7f9c28108ae0 2024-04-25T10:22:38.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.293+0000 7f9c2e7d3700 1 -- 172.21.15.43:0/1003031041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c2819e900 con 0x7f9c28108ae0 2024-04-25T10:22:38.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.294+0000 7f9c257fa700 1 -- 172.21.15.43:0/1003031041 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9c1802c070 con 0x7f9c28108ae0 2024-04-25T10:22:38.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.294+0000 7f9c257fa700 1 -- 172.21.15.43:0/1003031041 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9c18021070 con 0x7f9c28108ae0 2024-04-25T10:22:38.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.294+0000 7f9c2e7d3700 1 -- 172.21.15.43:0/1003031041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c28064cf0 con 0x7f9c28108ae0 2024-04-25T10:22:38.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.297+0000 7f9c257fa700 1 -- 172.21.15.43:0/1003031041 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9c18015070 con 0x7f9c28108ae0 2024-04-25T10:22:38.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.370+0000 7f9c2e7d3700 1 -- 172.21.15.43:0/1003031041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f9c2802e510 con 0x7f9c28108ae0 2024-04-25T10:22:38.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.370+0000 7f9c257fa700 1 -- 172.21.15.43:0/1003031041 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f9c2802e510 con 0x7f9c28108ae0 2024-04-25T10:22:38.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.371+0000 7f9c2e7d3700 1 -- 172.21.15.43:0/1003031041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c28108ae0 msgr2=0x7f9c2819b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:38.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.371+0000 7f9c2e7d3700 1 --2- 172.21.15.43:0/1003031041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c28108ae0 0x7f9c2819b520 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800b340 tx=0x7f9c18004750 comp rx=0 tx=0).stop 2024-04-25T10:22:38.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.371+0000 7f9c2e7d3700 1 -- 172.21.15.43:0/1003031041 shutdown_connections 2024-04-25T10:22:38.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.371+0000 7f9c2e7d3700 1 --2- 172.21.15.43:0/1003031041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c28108ae0 0x7f9c2819b520 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.371+0000 7f9c2e7d3700 1 -- 172.21.15.43:0/1003031041 >> 172.21.15.43:0/1003031041 conn(0x7f9c281041c0 msgr2=0x7f9c28191c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:38.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.371+0000 7f9c2e7d3700 1 -- 172.21.15.43:0/1003031041 shutdown_connections 2024-04-25T10:22:38.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:38.371+0000 7f9c2e7d3700 1 -- 172.21.15.43:0/1003031041 wait complete. 2024-04-25T10:22:38.881 INFO:teuthology.orchestra.run.smithi043.stdout:mon is available 2024-04-25T10:22:38.881 INFO:teuthology.orchestra.run.smithi043.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T10:22:38.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:38 smithi043 ceph-mon[29105]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T10:22:38.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:38 smithi043 ceph-mon[29105]: monmap epoch 1 2024-04-25T10:22:38.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:38 smithi043 ceph-mon[29105]: fsid 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:22:38.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:38 smithi043 ceph-mon[29105]: last_changed 2024-04-25T10:22:33.876396+0000 2024-04-25T10:22:38.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:38 smithi043 ceph-mon[29105]: created 2024-04-25T10:22:33.876396+0000 2024-04-25T10:22:38.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:38 smithi043 ceph-mon[29105]: min_mon_release 19 (squid) 2024-04-25T10:22:38.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:38 smithi043 ceph-mon[29105]: election_strategy: 1 2024-04-25T10:22:38.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:38 smithi043 ceph-mon[29105]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-04-25T10:22:38.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:38 smithi043 ceph-mon[29105]: fsmap 2024-04-25T10:22:38.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:38 smithi043 ceph-mon[29105]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T10:22:38.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:38 smithi043 ceph-mon[29105]: mgrmap e1: no daemons active 2024-04-25T10:22:38.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:38 smithi043 ceph-mon[29105]: from='client.? 172.21.15.43:0/1003031041' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T10:22:39.935 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-25T10:22:39.935 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T10:22:39.935 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout fsid = 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:22:39.936 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T10:22:39.936 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-04-25T10:22:39.936 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T10:22:39.936 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T10:22:39.936 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T10:22:39.936 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T10:22:39.936 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-25T10:22:39.936 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T10:22:39.936 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T10:22:39.936 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-25T10:22:39.936 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T10:22:39.936 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T10:22:39.936 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T10:22:39.936 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T10:22:39.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T10:22:39.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T10:22:39.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T10:22:39.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.288+0000 7f5ae0dcc700 1 Processor -- start 2024-04-25T10:22:39.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.288+0000 7f5ae0dcc700 1 -- start start 2024-04-25T10:22:39.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.288+0000 7f5ae0dcc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5adc07f6e0 0x7f5adc07db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.288+0000 7f5ae0dcc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc07e0e0 con 0x7f5adc07f6e0 2024-04-25T10:22:39.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.289+0000 7f5ada59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5adc07f6e0 0x7f5adc07db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:39.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.289+0000 7f5ada59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5adc07f6e0 0x7f5adc07db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41930/0 (socket says 172.21.15.43:41930) 2024-04-25T10:22:39.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.289+0000 7f5ada59c700 1 -- 172.21.15.43:0/3064469997 learned_addr learned my addr 172.21.15.43:0/3064469997 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:39.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.289+0000 7f5ada59c700 1 -- 172.21.15.43:0/3064469997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5adc07e260 con 0x7f5adc07f6e0 2024-04-25T10:22:39.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.289+0000 7f5ada59c700 1 --2- 172.21.15.43:0/3064469997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5adc07f6e0 0x7f5adc07db10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5acc009e80 tx=0x7f5acc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667b7986cee63ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:39.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.289+0000 7f5ad959a700 1 -- 172.21.15.43:0/3064469997 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5acc017070 con 0x7f5adc07f6e0 2024-04-25T10:22:39.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.289+0000 7f5ad959a700 1 -- 172.21.15.43:0/3064469997 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5acc003a20 con 0x7f5adc07f6e0 2024-04-25T10:22:39.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.289+0000 7f5ad959a700 1 -- 172.21.15.43:0/3064469997 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5acc003d20 con 0x7f5adc07f6e0 2024-04-25T10:22:39.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.290+0000 7f5ae0dcc700 1 -- 172.21.15.43:0/3064469997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5adc07f6e0 msgr2=0x7f5adc07db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:39.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.290+0000 7f5ae0dcc700 1 --2- 172.21.15.43:0/3064469997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5adc07f6e0 0x7f5adc07db10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5acc009e80 tx=0x7f5acc00b330 comp rx=0 tx=0).stop 2024-04-25T10:22:39.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.290+0000 7f5ae0dcc700 1 -- 172.21.15.43:0/3064469997 shutdown_connections 2024-04-25T10:22:39.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.290+0000 7f5ae0dcc700 1 --2- 172.21.15.43:0/3064469997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5adc07f6e0 0x7f5adc07db10 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.290+0000 7f5ae0dcc700 1 -- 172.21.15.43:0/3064469997 >> 172.21.15.43:0/3064469997 conn(0x7f5adc105280 msgr2=0x7f5adc1076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:39.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.290+0000 7f5ae0dcc700 1 -- 172.21.15.43:0/3064469997 shutdown_connections 2024-04-25T10:22:39.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.290+0000 7f5ae0dcc700 1 -- 172.21.15.43:0/3064469997 wait complete. 2024-04-25T10:22:39.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.291+0000 7f5ae0dcc700 1 Processor -- start 2024-04-25T10:22:39.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.291+0000 7f5ae0dcc700 1 -- start start 2024-04-25T10:22:39.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.291+0000 7f5ae0dcc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5adc07f6e0 0x7f5adc19f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.291+0000 7f5ae0dcc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc07e790 con 0x7f5adc07f6e0 2024-04-25T10:22:39.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.291+0000 7f5ada59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5adc07f6e0 0x7f5adc19f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:39.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.291+0000 7f5ada59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5adc07f6e0 0x7f5adc19f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41944/0 (socket says 172.21.15.43:41944) 2024-04-25T10:22:39.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.291+0000 7f5ada59c700 1 -- 172.21.15.43:0/3379557054 learned_addr learned my addr 172.21.15.43:0/3379557054 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:39.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.292+0000 7f5ada59c700 1 -- 172.21.15.43:0/3379557054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5adc19fec0 con 0x7f5adc07f6e0 2024-04-25T10:22:39.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.292+0000 7f5ada59c700 1 --2- 172.21.15.43:0/3379557054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5adc07f6e0 0x7f5adc19f980 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5acc004000 tx=0x7f5acc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:39.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.292+0000 7f5ad37fe700 1 -- 172.21.15.43:0/3379557054 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5acc022020 con 0x7f5adc07f6e0 2024-04-25T10:22:39.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.292+0000 7f5ad37fe700 1 -- 172.21.15.43:0/3379557054 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5acc018040 con 0x7f5adc07f6e0 2024-04-25T10:22:39.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.292+0000 7f5ae0dcc700 1 -- 172.21.15.43:0/3379557054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5adc1a0150 con 0x7f5adc07f6e0 2024-04-25T10:22:39.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.292+0000 7f5ad37fe700 1 -- 172.21.15.43:0/3379557054 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5acc017070 con 0x7f5adc07f6e0 2024-04-25T10:22:39.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.292+0000 7f5ae0dcc700 1 -- 172.21.15.43:0/3379557054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5adc1a2d60 con 0x7f5adc07f6e0 2024-04-25T10:22:39.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.292+0000 7f5ad37fe700 1 -- 172.21.15.43:0/3379557054 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5acc025020 con 0x7f5adc07f6e0 2024-04-25T10:22:39.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.292+0000 7f5ad37fe700 1 -- 172.21.15.43:0/3379557054 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5acc00f050 con 0x7f5adc07f6e0 2024-04-25T10:22:39.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.293+0000 7f5ae0dcc700 1 -- 172.21.15.43:0/3379557054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5adc064cf0 con 0x7f5adc07f6e0 2024-04-25T10:22:39.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.295+0000 7f5ad37fe700 1 -- 172.21.15.43:0/3379557054 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5acc02c030 con 0x7f5adc07f6e0 2024-04-25T10:22:39.940 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.366+0000 7f5ae0dcc700 1 -- 172.21.15.43:0/3379557054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f5adc02e780 con 0x7f5adc07f6e0 2024-04-25T10:22:39.940 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.369+0000 7f5ad37fe700 1 -- 172.21.15.43:0/3379557054 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7f5acc01e480 con 0x7f5adc07f6e0 2024-04-25T10:22:39.940 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.370+0000 7f5ae0dcc700 1 -- 172.21.15.43:0/3379557054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5adc07f6e0 msgr2=0x7f5adc19f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:39.940 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.370+0000 7f5ae0dcc700 1 --2- 172.21.15.43:0/3379557054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5adc07f6e0 0x7f5adc19f980 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5acc004000 tx=0x7f5acc004ad0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.940 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.370+0000 7f5ae0dcc700 1 -- 172.21.15.43:0/3379557054 shutdown_connections 2024-04-25T10:22:39.940 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.370+0000 7f5ae0dcc700 1 --2- 172.21.15.43:0/3379557054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5adc07f6e0 0x7f5adc19f980 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.940 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.370+0000 7f5ae0dcc700 1 -- 172.21.15.43:0/3379557054 >> 172.21.15.43:0/3379557054 conn(0x7f5adc105280 msgr2=0x7f5adc107120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:39.940 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.370+0000 7f5ae0dcc700 1 -- 172.21.15.43:0/3379557054 shutdown_connections 2024-04-25T10:22:39.940 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:39.370+0000 7f5ae0dcc700 1 -- 172.21.15.43:0/3379557054 wait complete. 2024-04-25T10:22:39.940 INFO:teuthology.orchestra.run.smithi043.stdout:Generating new minimal ceph.conf... 2024-04-25T10:22:40.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:39 smithi043 ceph-mon[29105]: from='client.? 172.21.15.43:0/3379557054' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T10:22:40.051 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:39 smithi043 ceph-mon[29105]: from='client.? 172.21.15.43:0/3379557054' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T10:22:41.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:40 smithi043 ceph-mon[29105]: from='client.? 172.21.15.43:0/3106536018' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:22:41.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.376+0000 7fa2bdd41700 1 Processor -- start 2024-04-25T10:22:41.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.376+0000 7fa2bdd41700 1 -- start start 2024-04-25T10:22:41.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.376+0000 7fa2bdd41700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b810a6e0 0x7fa2b810cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.376+0000 7fa2bdd41700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b807d6f0 con 0x7fa2b810a6e0 2024-04-25T10:22:41.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.376+0000 7fa2b77fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b810a6e0 0x7fa2b810cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.376+0000 7fa2b77fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b810a6e0 0x7fa2b810cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41954/0 (socket says 172.21.15.43:41954) 2024-04-25T10:22:41.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.376+0000 7fa2b77fe700 1 -- 172.21.15.43:0/1108619551 learned_addr learned my addr 172.21.15.43:0/1108619551 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:41.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.376+0000 7fa2b77fe700 1 -- 172.21.15.43:0/1108619551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b810d040 con 0x7fa2b810a6e0 2024-04-25T10:22:41.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.377+0000 7fa2b77fe700 1 --2- 172.21.15.43:0/1108619551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b810a6e0 0x7fa2b810cb00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00098a0 tx=0x7fa2a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52c682778260f21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:41.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.377+0000 7fa2b67fc700 1 -- 172.21.15.43:0/1108619551 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2a0004160 con 0x7fa2b810a6e0 2024-04-25T10:22:41.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.377+0000 7fa2b67fc700 1 -- 172.21.15.43:0/1108619551 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa2a0004300 con 0x7fa2b810a6e0 2024-04-25T10:22:41.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.377+0000 7fa2b67fc700 1 -- 172.21.15.43:0/1108619551 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2a0011af0 con 0x7fa2b810a6e0 2024-04-25T10:22:41.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.378+0000 7fa2bdd41700 1 -- 172.21.15.43:0/1108619551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b810a6e0 msgr2=0x7fa2b810cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.378+0000 7fa2bdd41700 1 --2- 172.21.15.43:0/1108619551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b810a6e0 0x7fa2b810cb00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00098a0 tx=0x7fa2a0009bb0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.378+0000 7fa2bdd41700 1 -- 172.21.15.43:0/1108619551 shutdown_connections 2024-04-25T10:22:41.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.378+0000 7fa2bdd41700 1 --2- 172.21.15.43:0/1108619551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b810a6e0 0x7fa2b810cb00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.378+0000 7fa2bdd41700 1 -- 172.21.15.43:0/1108619551 >> 172.21.15.43:0/1108619551 conn(0x7fa2b8104120 msgr2=0x7fa2b81065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:41.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.378+0000 7fa2bdd41700 1 -- 172.21.15.43:0/1108619551 shutdown_connections 2024-04-25T10:22:41.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.378+0000 7fa2bdd41700 1 -- 172.21.15.43:0/1108619551 wait complete. 2024-04-25T10:22:41.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.378+0000 7fa2bdd41700 1 Processor -- start 2024-04-25T10:22:41.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.379+0000 7fa2bdd41700 1 -- start start 2024-04-25T10:22:41.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.379+0000 7fa2bdd41700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b810a6e0 0x7fa2b819e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.379+0000 7fa2bdd41700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b810dfa0 con 0x7fa2b810a6e0 2024-04-25T10:22:41.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.379+0000 7fa2b77fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b810a6e0 0x7fa2b819e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.379+0000 7fa2b77fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b810a6e0 0x7fa2b819e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41964/0 (socket says 172.21.15.43:41964) 2024-04-25T10:22:41.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.379+0000 7fa2b77fe700 1 -- 172.21.15.43:0/3106536018 learned_addr learned my addr 172.21.15.43:0/3106536018 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:41.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.379+0000 7fa2b77fe700 1 -- 172.21.15.43:0/3106536018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b819e5a0 con 0x7fa2b810a6e0 2024-04-25T10:22:41.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.379+0000 7fa2b77fe700 1 --2- 172.21.15.43:0/3106536018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b810a6e0 0x7fa2b819e060 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009fb0 tx=0x7fa2a0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:41.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.379+0000 7fa2b4ff9700 1 -- 172.21.15.43:0/3106536018 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2a0011bc0 con 0x7fa2b810a6e0 2024-04-25T10:22:41.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.379+0000 7fa2b4ff9700 1 -- 172.21.15.43:0/3106536018 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa2a0024440 con 0x7fa2b810a6e0 2024-04-25T10:22:41.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.379+0000 7fa2bdd41700 1 -- 172.21.15.43:0/3106536018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b819e830 con 0x7fa2b810a6e0 2024-04-25T10:22:41.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.380+0000 7fa2b4ff9700 1 -- 172.21.15.43:0/3106536018 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2a001a5c0 con 0x7fa2b810a6e0 2024-04-25T10:22:41.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.380+0000 7fa2bdd41700 1 -- 172.21.15.43:0/3106536018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b81a1440 con 0x7fa2b810a6e0 2024-04-25T10:22:41.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.380+0000 7fa2b4ff9700 1 -- 172.21.15.43:0/3106536018 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa2a001abc0 con 0x7fa2b810a6e0 2024-04-25T10:22:41.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.380+0000 7fa2b4ff9700 1 -- 172.21.15.43:0/3106536018 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa2a0019700 con 0x7fa2b810a6e0 2024-04-25T10:22:41.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.381+0000 7fa2bdd41700 1 -- 172.21.15.43:0/3106536018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b8064cf0 con 0x7fa2b810a6e0 2024-04-25T10:22:41.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.383+0000 7fa2b4ff9700 1 -- 172.21.15.43:0/3106536018 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa2a002dd90 con 0x7fa2b810a6e0 2024-04-25T10:22:41.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.454+0000 7fa2bdd41700 1 -- 172.21.15.43:0/3106536018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa2b81a1660 con 0x7fa2b810a6e0 2024-04-25T10:22:41.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.454+0000 7fa2b4ff9700 1 -- 172.21.15.43:0/3106536018 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fa2a00245f0 con 0x7fa2b810a6e0 2024-04-25T10:22:41.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.455+0000 7fa2bdd41700 1 -- 172.21.15.43:0/3106536018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b810a6e0 msgr2=0x7fa2b819e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.455+0000 7fa2bdd41700 1 --2- 172.21.15.43:0/3106536018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b810a6e0 0x7fa2b819e060 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009fb0 tx=0x7fa2a0011890 comp rx=0 tx=0).stop 2024-04-25T10:22:41.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.455+0000 7fa2bdd41700 1 -- 172.21.15.43:0/3106536018 shutdown_connections 2024-04-25T10:22:41.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.455+0000 7fa2bdd41700 1 --2- 172.21.15.43:0/3106536018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b810a6e0 0x7fa2b819e060 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.455+0000 7fa2bdd41700 1 -- 172.21.15.43:0/3106536018 >> 172.21.15.43:0/3106536018 conn(0x7fa2b8104120 msgr2=0x7fa2b8104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:41.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.455+0000 7fa2bdd41700 1 -- 172.21.15.43:0/3106536018 shutdown_connections 2024-04-25T10:22:41.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:40.455+0000 7fa2bdd41700 1 -- 172.21.15.43:0/3106536018 wait complete. 2024-04-25T10:22:41.012 INFO:teuthology.orchestra.run.smithi043.stdout:Restarting the monitor... 2024-04-25T10:22:41.277 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:41 smithi043 systemd[1]: Stopping Ceph mon.a for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:22:41.560 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:41 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-a[29101]: 2024-04-25T10:22:41.273+0000 7f68e8070700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:22:41.560 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:41 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-a[29101]: 2024-04-25T10:22:41.273+0000 7f68e8070700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T10:22:41.560 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:41 smithi043 podman[29349]: 2024-04-25 10:22:41.294938561 +0000 UTC m=+0.115884595 container died d7f88830382db77349da76428b812a2fe2fee0d0af6a218b5aba4e58efcd4d6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T10:22:42.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:41 smithi043 podman[29349]: 2024-04-25 10:22:41.874097357 +0000 UTC m=+0.695043395 container cleanup d7f88830382db77349da76428b812a2fe2fee0d0af6a218b5aba4e58efcd4d6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS) 2024-04-25T10:22:42.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:41 smithi043 bash[29349]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-a 2024-04-25T10:22:42.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 podman[29363]: 2024-04-25 10:22:42.116498651 +0000 UTC m=+0.812367139 container remove d7f88830382db77349da76428b812a2fe2fee0d0af6a218b5aba4e58efcd4d6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T10:22:42.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 systemd[1]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T10:22:42.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 systemd[1]: Stopped Ceph mon.a for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:22:42.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 systemd[1]: Starting Ceph mon.a for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:22:42.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 podman[29458]: 2024-04-25 10:22:42.467339116 +0000 UTC m=+0.099650313 container create 471bd00c8578c51d08f93c203a3cfdb4b6cf3e1922ec4fe4657ca7c01ff1bc70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:22:42.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 podman[29458]: 2024-04-25 10:22:42.386256816 +0000 UTC m=+0.018568024 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:22:42.726 INFO:teuthology.orchestra.run.smithi043.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T10:22:42.952 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 podman[29458]: 2024-04-25 10:22:42.645022334 +0000 UTC m=+0.277333533 container init 471bd00c8578c51d08f93c203a3cfdb4b6cf3e1922ec4fe4657ca7c01ff1bc70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T10:22:42.952 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 podman[29458]: 2024-04-25 10:22:42.648325702 +0000 UTC m=+0.280636903 container start 471bd00c8578c51d08f93c203a3cfdb4b6cf3e1922ec4fe4657ca7c01ff1bc70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True) 2024-04-25T10:22:42.952 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T10:22:42.952 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev), process ceph-mon, pid 2 2024-04-25T10:22:42.952 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: pidfile_write: ignore empty --pid-file 2024-04-25T10:22:42.952 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: load: jerasure load: lrc 2024-04-25T10:22:42.952 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: RocksDB version: 7.9.2 2024-04-25T10:22:42.952 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Git sha 0 2024-04-25T10:22:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Compile date 2024-04-25 02:57:34 2024-04-25T10:22:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: DB SUMMARY 2024-04-25T10:22:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: DB Session ID: WZ8LPRID3VMNZ819UWC4 2024-04-25T10:22:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: CURRENT file: CURRENT 2024-04-25T10:22:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T10:22:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T10:22:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T10:22:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106032 ; 2024-04-25T10:22:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.error_if_exists: 0 2024-04-25T10:22:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.create_if_missing: 0 2024-04-25T10:22:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.paranoid_checks: 1 2024-04-25T10:22:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T10:22:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T10:22:42.954 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T10:22:42.954 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.env: 0x5621c33dc500 2024-04-25T10:22:42.954 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T10:22:42.954 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.info_log: 0x5621c500dfc0 2024-04-25T10:22:42.954 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T10:22:42.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.statistics: (nil) 2024-04-25T10:22:42.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.use_fsync: 0 2024-04-25T10:22:42.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_log_file_size: 0 2024-04-25T10:22:42.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T10:22:42.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T10:22:42.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T10:22:42.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T10:22:42.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.allow_fallocate: 1 2024-04-25T10:22:42.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T10:22:42.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T10:22:42.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.use_direct_reads: 0 2024-04-25T10:22:42.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T10:22:42.956 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T10:22:42.956 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.db_log_dir: 2024-04-25T10:22:42.956 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.wal_dir: 2024-04-25T10:22:42.956 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T10:22:42.956 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T10:22:42.956 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T10:22:42.956 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T10:22:42.956 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T10:22:42.956 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T10:22:42.956 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T10:22:42.956 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T10:22:42.956 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.write_buffer_manager: 0x5621c4fde8c0 2024-04-25T10:22:42.956 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T10:22:42.956 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T10:22:42.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T10:22:42.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T10:22:42.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T10:22:42.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T10:22:42.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T10:22:42.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T10:22:42.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.unordered_write: 0 2024-04-25T10:22:42.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T10:22:42.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T10:22:42.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T10:22:42.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T10:22:42.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.row_cache: None 2024-04-25T10:22:42.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.wal_filter: None 2024-04-25T10:22:42.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T10:22:42.958 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T10:22:42.958 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.two_write_queues: 0 2024-04-25T10:22:42.958 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T10:22:42.958 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.wal_compression: 0 2024-04-25T10:22:42.958 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.atomic_flush: 0 2024-04-25T10:22:42.958 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T10:22:42.958 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T10:22:42.958 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T10:22:42.958 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.log_readahead_size: 0 2024-04-25T10:22:42.958 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T10:22:42.958 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T10:22:42.958 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T10:22:42.958 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T10:22:42.958 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T10:22:42.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T10:22:42.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T10:22:42.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_background_jobs: 2 2024-04-25T10:22:42.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_background_compactions: -1 2024-04-25T10:22:42.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_subcompactions: 1 2024-04-25T10:22:42.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T10:22:42.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T10:22:42.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T10:22:42.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T10:22:42.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T10:22:42.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T10:22:42.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T10:22:42.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T10:22:42.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_open_files: -1 2024-04-25T10:22:42.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T10:22:42.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T10:22:42.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T10:22:42.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T10:22:42.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_background_flushes: -1 2024-04-25T10:22:42.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Compression algorithms supported: 2024-04-25T10:22:42.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T10:22:42.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: kZSTD supported: 0 2024-04-25T10:22:42.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: kXpressCompression supported: 0 2024-04-25T10:22:42.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T10:22:42.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: kZlibCompression supported: 1 2024-04-25T10:22:42.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: kSnappyCompression supported: 1 2024-04-25T10:22:42.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: kLZ4Compression supported: 1 2024-04-25T10:22:42.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: kBZip2Compression supported: 0 2024-04-25T10:22:42.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T10:22:42.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T10:22:42.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T10:22:42.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T10:22:42.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T10:22:42.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.merge_operator: 2024-04-25T10:22:42.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compaction_filter: None 2024-04-25T10:22:42.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T10:22:42.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T10:22:42.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T10:22:42.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T10:22:42.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5621c51141a0) 2024-04-25T10:22:42.961 INFO:journalctl@ceph.mon.a.smithi043.stdout: cache_index_and_filter_blocks: 1 2024-04-25T10:22:42.961 INFO:journalctl@ceph.mon.a.smithi043.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T10:22:42.962 INFO:journalctl@ceph.mon.a.smithi043.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T10:22:42.962 INFO:journalctl@ceph.mon.a.smithi043.stdout: pin_top_level_index_and_filter: 1 2024-04-25T10:22:42.962 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_type: 0 2024-04-25T10:22:42.962 INFO:journalctl@ceph.mon.a.smithi043.stdout: data_block_index_type: 0 2024-04-25T10:22:42.962 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_shortening: 1 2024-04-25T10:22:42.962 INFO:journalctl@ceph.mon.a.smithi043.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T10:22:42.962 INFO:journalctl@ceph.mon.a.smithi043.stdout: checksum: 4 2024-04-25T10:22:42.962 INFO:journalctl@ceph.mon.a.smithi043.stdout: no_block_cache: 0 2024-04-25T10:22:42.962 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache: 0x5621c50111f0 2024-04-25T10:22:42.962 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_name: BinnedLRUCache 2024-04-25T10:22:42.962 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_options: 2024-04-25T10:22:42.962 INFO:journalctl@ceph.mon.a.smithi043.stdout: capacity : 536870912 2024-04-25T10:22:42.962 INFO:journalctl@ceph.mon.a.smithi043.stdout: num_shard_bits : 4 2024-04-25T10:22:42.963 INFO:journalctl@ceph.mon.a.smithi043.stdout: strict_capacity_limit : 0 2024-04-25T10:22:42.963 INFO:journalctl@ceph.mon.a.smithi043.stdout: high_pri_pool_ratio: 0.000 2024-04-25T10:22:42.963 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_compressed: (nil) 2024-04-25T10:22:42.963 INFO:journalctl@ceph.mon.a.smithi043.stdout: persistent_cache: (nil) 2024-04-25T10:22:42.963 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_size: 4096 2024-04-25T10:22:42.963 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_size_deviation: 10 2024-04-25T10:22:42.963 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_restart_interval: 16 2024-04-25T10:22:42.963 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_block_restart_interval: 1 2024-04-25T10:22:42.963 INFO:journalctl@ceph.mon.a.smithi043.stdout: metadata_block_size: 4096 2024-04-25T10:22:42.963 INFO:journalctl@ceph.mon.a.smithi043.stdout: partition_filters: 0 2024-04-25T10:22:42.963 INFO:journalctl@ceph.mon.a.smithi043.stdout: use_delta_encoding: 1 2024-04-25T10:22:42.963 INFO:journalctl@ceph.mon.a.smithi043.stdout: filter_policy: bloomfilter 2024-04-25T10:22:42.963 INFO:journalctl@ceph.mon.a.smithi043.stdout: whole_key_filtering: 1 2024-04-25T10:22:42.963 INFO:journalctl@ceph.mon.a.smithi043.stdout: verify_compression: 0 2024-04-25T10:22:42.964 INFO:journalctl@ceph.mon.a.smithi043.stdout: read_amp_bytes_per_bit: 0 2024-04-25T10:22:42.964 INFO:journalctl@ceph.mon.a.smithi043.stdout: format_version: 5 2024-04-25T10:22:42.964 INFO:journalctl@ceph.mon.a.smithi043.stdout: enable_index_compression: 1 2024-04-25T10:22:42.964 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_align: 0 2024-04-25T10:22:42.964 INFO:journalctl@ceph.mon.a.smithi043.stdout: max_auto_readahead_size: 262144 2024-04-25T10:22:42.964 INFO:journalctl@ceph.mon.a.smithi043.stdout: prepopulate_block_cache: 0 2024-04-25T10:22:42.964 INFO:journalctl@ceph.mon.a.smithi043.stdout: initial_auto_readahead_size: 8192 2024-04-25T10:22:42.964 INFO:journalctl@ceph.mon.a.smithi043.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T10:22:42.964 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T10:22:42.964 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T10:22:42.964 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compression: NoCompression 2024-04-25T10:22:42.964 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T10:22:42.964 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T10:22:42.964 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T10:22:42.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.num_levels: 7 2024-04-25T10:22:42.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T10:22:42.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T10:22:42.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T10:22:42.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T10:22:42.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T10:22:42.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T10:22:42.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T10:22:42.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T10:22:42.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T10:22:42.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T10:22:42.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T10:22:42.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T10:22:42.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T10:22:42.966 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T10:22:42.966 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T10:22:42.966 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T10:22:42.966 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T10:22:42.966 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T10:22:42.966 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T10:22:42.966 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T10:22:42.966 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T10:22:42.966 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T10:22:42.966 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T10:22:42.966 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T10:22:42.966 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T10:22:42.966 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T10:22:42.966 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T10:22:42.967 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T10:22:42.967 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T10:22:42.967 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T10:22:42.967 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T10:22:42.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T10:22:42.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T10:22:42.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T10:22:42.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T10:22:42.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T10:22:42.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T10:22:42.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T10:22:42.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T10:22:42.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T10:22:42.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T10:22:42.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T10:22:42.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T10:22:42.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T10:22:42.969 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T10:22:42.969 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T10:22:42.969 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T10:22:42.969 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T10:22:42.969 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T10:22:42.969 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T10:22:42.969 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T10:22:42.969 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T10:22:42.969 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T10:22:42.969 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T10:22:42.969 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.inplace_update_support: 0 2024-04-25T10:22:42.969 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T10:22:42.969 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T10:22:42.969 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T10:22:42.969 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T10:22:42.970 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.bloom_locality: 0 2024-04-25T10:22:42.970 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.max_successive_merges: 0 2024-04-25T10:22:42.970 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T10:22:42.970 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T10:22:42.970 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T10:22:42.970 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T10:22:42.970 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.ttl: 2592000 2024-04-25T10:22:42.970 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T10:22:42.970 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T10:22:42.970 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T10:22:42.970 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.enable_blob_files: false 2024-04-25T10:22:42.970 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.min_blob_size: 0 2024-04-25T10:22:42.970 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T10:22:42.971 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T10:22:42.971 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T10:22:42.971 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T10:22:42.971 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T10:22:42.971 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T10:22:42.971 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T10:22:42.971 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T10:22:42.971 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T10:22:42.971 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T10:22:42.971 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 217b7f21-f5ed-4da7-827c-acee229820e0 2024-04-25T10:22:42.971 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714040562688638, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T10:22:42.971 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T10:22:42.971 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714040562690621, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101084, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98973, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14405, "raw_average_key_size": 49, "raw_value_size": 91152, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714040562, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "217b7f21-f5ed-4da7-827c-acee229820e0", "db_session_id": "WZ8LPRID3VMNZ819UWC4", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T10:22:42.971 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714040562690737, "job": 1, "event": "recovery_finished"} 2024-04-25T10:22:42.972 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T10:22:42.972 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T10:22:42.972 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5621c4ff5c00 2024-04-25T10:22:42.972 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: DB pointer 0x5621c9de8000 2024-04-25T10:22:42.972 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T10:22:42.972 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T10:22:42.972 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** DB Stats ** 2024-04-25T10:22:42.972 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T10:22:42.972 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T10:22:42.972 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T10:22:42.972 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T10:22:42.972 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T10:22:42.972 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T10:22:42.972 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T10:22:42.973 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-04-25T10:22:42.973 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-04-25T10:22:42.973 INFO:journalctl@ceph.mon.a.smithi043.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T10:22:42.973 INFO:journalctl@ceph.mon.a.smithi043.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T10:22:42.973 INFO:journalctl@ceph.mon.a.smithi043.stdout: L0 2/0 100.58 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 85.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T10:22:42.973 INFO:journalctl@ceph.mon.a.smithi043.stdout: Sum 2/0 100.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 85.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T10:22:42.973 INFO:journalctl@ceph.mon.a.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 85.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T10:22:42.973 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-04-25T10:22:42.973 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-04-25T10:22:42.973 INFO:journalctl@ceph.mon.a.smithi043.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T10:22:42.973 INFO:journalctl@ceph.mon.a.smithi043.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T10:22:42.973 INFO:journalctl@ceph.mon.a.smithi043.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 85.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T10:22:42.973 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-04-25T10:22:42.973 INFO:journalctl@ceph.mon.a.smithi043.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T10:22:42.973 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-04-25T10:22:42.974 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T10:22:42.974 INFO:journalctl@ceph.mon.a.smithi043.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T10:22:42.974 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T10:22:42.974 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T10:22:42.974 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T10:22:42.974 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T10:22:42.974 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative compaction: 0.00 GB write, 19.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T10:22:42.974 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval compaction: 0.00 GB write, 19.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T10:22:42.974 INFO:journalctl@ceph.mon.a.smithi043.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T10:22:42.974 INFO:journalctl@ceph.mon.a.smithi043.stdout: Block cache BinnedLRUCache@0x5621c50111f0#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T10:22:42.974 INFO:journalctl@ceph.mon.a.smithi043.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T10:22:42.974 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-04-25T10:22:42.974 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T10:22:42.974 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: starting mon.a rank 0 at public addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] at bind addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:22:42.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: mon.a@-1(???) e1 preinit fsid 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:22:42.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: mon.a@-1(???).mds e1 new map 2024-04-25T10:22:42.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: mon.a@-1(???).mds e1 print_map 2024-04-25T10:22:42.975 INFO:journalctl@ceph.mon.a.smithi043.stdout: e1 2024-04-25T10:22:42.975 INFO:journalctl@ceph.mon.a.smithi043.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T10:22:42.975 INFO:journalctl@ceph.mon.a.smithi043.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T10:22:42.975 INFO:journalctl@ceph.mon.a.smithi043.stdout: legacy client fscid: -1 2024-04-25T10:22:42.975 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-04-25T10:22:42.975 INFO:journalctl@ceph.mon.a.smithi043.stdout: No filesystems configured 2024-04-25T10:22:42.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T10:22:42.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T10:22:42.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T10:22:42.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T10:22:42.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T10:22:42.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 bash[29458]: 471bd00c8578c51d08f93c203a3cfdb4b6cf3e1922ec4fe4657ca7c01ff1bc70 2024-04-25T10:22:42.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 systemd[1]: Started Ceph mon.a for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:22:42.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T10:22:42.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: monmap epoch 1 2024-04-25T10:22:42.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: fsid 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:22:42.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: last_changed 2024-04-25T10:22:33.876396+0000 2024-04-25T10:22:42.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: created 2024-04-25T10:22:33.876396+0000 2024-04-25T10:22:42.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: min_mon_release 19 (squid) 2024-04-25T10:22:42.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: election_strategy: 1 2024-04-25T10:22:42.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-04-25T10:22:42.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: fsmap 2024-04-25T10:22:42.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T10:22:42.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:42 smithi043 ceph-mon[29478]: mgrmap e1: no daemons active 2024-04-25T10:22:43.796 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.196+0000 7f25b101d700 1 Processor -- start 2024-04-25T10:22:43.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.196+0000 7f25b101d700 1 -- start start 2024-04-25T10:22:43.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.196+0000 7f25b101d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac108ab0 0x7f25ac108ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.196+0000 7f25b101d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac1094b0 con 0x7f25ac108ab0 2024-04-25T10:22:43.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.197+0000 7f25aad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac108ab0 0x7f25ac108ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.197+0000 7f25aad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac108ab0 0x7f25ac108ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41980/0 (socket says 172.21.15.43:41980) 2024-04-25T10:22:43.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.197+0000 7f25aad9d700 1 -- 172.21.15.43:0/4049653257 learned_addr learned my addr 172.21.15.43:0/4049653257 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:43.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.197+0000 7f25aad9d700 1 -- 172.21.15.43:0/4049653257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ac109c80 con 0x7f25ac108ab0 2024-04-25T10:22:43.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.197+0000 7f25aad9d700 1 --2- 172.21.15.43:0/4049653257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac108ab0 0x7f25ac108ec0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f259c00d180 tx=0x7f259c00d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3916e9f7ba5a478c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:43.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.197+0000 7f25a9d9b700 1 -- 172.21.15.43:0/4049653257 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f259c019070 con 0x7f25ac108ab0 2024-04-25T10:22:43.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.198+0000 7f25a9d9b700 1 -- 172.21.15.43:0/4049653257 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f259c00de20 con 0x7f25ac108ab0 2024-04-25T10:22:43.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.198+0000 7f25a9d9b700 1 -- 172.21.15.43:0/4049653257 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f259c009a70 con 0x7f25ac108ab0 2024-04-25T10:22:43.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.198+0000 7f25b101d700 1 -- 172.21.15.43:0/4049653257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac108ab0 msgr2=0x7f25ac108ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:43.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.198+0000 7f25b101d700 1 --2- 172.21.15.43:0/4049653257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac108ab0 0x7f25ac108ec0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f259c00d180 tx=0x7f259c00d490 comp rx=0 tx=0).stop 2024-04-25T10:22:43.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.198+0000 7f25b101d700 1 -- 172.21.15.43:0/4049653257 shutdown_connections 2024-04-25T10:22:43.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.198+0000 7f25b101d700 1 --2- 172.21.15.43:0/4049653257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac108ab0 0x7f25ac108ec0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.198+0000 7f25b101d700 1 -- 172.21.15.43:0/4049653257 >> 172.21.15.43:0/4049653257 conn(0x7f25ac104110 msgr2=0x7f25ac1065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:43.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.198+0000 7f25b101d700 1 -- 172.21.15.43:0/4049653257 shutdown_connections 2024-04-25T10:22:43.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.198+0000 7f25b101d700 1 -- 172.21.15.43:0/4049653257 wait complete. 2024-04-25T10:22:43.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.199+0000 7f25b101d700 1 Processor -- start 2024-04-25T10:22:43.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.199+0000 7f25b101d700 1 -- start start 2024-04-25T10:22:43.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.200+0000 7f25b101d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac108ab0 0x7f25ac1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.200+0000 7f25b101d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac07cba0 con 0x7f25ac108ab0 2024-04-25T10:22:43.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.200+0000 7f25aad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac108ab0 0x7f25ac1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.200+0000 7f25aad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac108ab0 0x7f25ac1a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41994/0 (socket says 172.21.15.43:41994) 2024-04-25T10:22:43.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.200+0000 7f25aad9d700 1 -- 172.21.15.43:0/3794248704 learned_addr learned my addr 172.21.15.43:0/3794248704 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:43.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.200+0000 7f25aad9d700 1 -- 172.21.15.43:0/3794248704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ac1a06d0 con 0x7f25ac108ab0 2024-04-25T10:22:43.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.200+0000 7f25aad9d700 1 --2- 172.21.15.43:0/3794248704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac108ab0 0x7f25ac1a0190 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f259c009780 tx=0x7f259c009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:43.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.201+0000 7f25a3fff700 1 -- 172.21.15.43:0/3794248704 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f259c019040 con 0x7f25ac108ab0 2024-04-25T10:22:43.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.201+0000 7f25b101d700 1 -- 172.21.15.43:0/3794248704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ac1a0960 con 0x7f25ac108ab0 2024-04-25T10:22:43.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.201+0000 7f25a3fff700 1 -- 172.21.15.43:0/3794248704 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f259c003b50 con 0x7f25ac108ab0 2024-04-25T10:22:43.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.201+0000 7f25a3fff700 1 -- 172.21.15.43:0/3794248704 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f259c01f750 con 0x7f25ac108ab0 2024-04-25T10:22:43.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.201+0000 7f25b101d700 1 -- 172.21.15.43:0/3794248704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ac1a3570 con 0x7f25ac108ab0 2024-04-25T10:22:43.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.201+0000 7f25a3fff700 1 -- 172.21.15.43:0/3794248704 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f259c01f8f0 con 0x7f25ac108ab0 2024-04-25T10:22:43.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.201+0000 7f25a3fff700 1 -- 172.21.15.43:0/3794248704 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f259c011050 con 0x7f25ac108ab0 2024-04-25T10:22:43.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.202+0000 7f25b101d700 1 -- 172.21.15.43:0/3794248704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25ac064cf0 con 0x7f25ac108ab0 2024-04-25T10:22:43.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.205+0000 7f25a3fff700 1 -- 172.21.15.43:0/3794248704 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f259c01fd70 con 0x7f25ac108ab0 2024-04-25T10:22:43.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.277+0000 7f25b101d700 1 -- 172.21.15.43:0/3794248704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f25ac194fe0 con 0x7f25ac108ab0 2024-04-25T10:22:43.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.279+0000 7f25a3fff700 1 -- 172.21.15.43:0/3794248704 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f259c003d00 con 0x7f25ac108ab0 2024-04-25T10:22:43.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.279+0000 7f25b101d700 1 -- 172.21.15.43:0/3794248704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac108ab0 msgr2=0x7f25ac1a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:43.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.279+0000 7f25b101d700 1 --2- 172.21.15.43:0/3794248704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac108ab0 0x7f25ac1a0190 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f259c009780 tx=0x7f259c009420 comp rx=0 tx=0).stop 2024-04-25T10:22:43.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.279+0000 7f25b101d700 1 -- 172.21.15.43:0/3794248704 shutdown_connections 2024-04-25T10:22:43.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.279+0000 7f25b101d700 1 --2- 172.21.15.43:0/3794248704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac108ab0 0x7f25ac1a0190 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.279+0000 7f25b101d700 1 -- 172.21.15.43:0/3794248704 >> 172.21.15.43:0/3794248704 conn(0x7f25ac104110 msgr2=0x7f25ac1920a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:43.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.280+0000 7f25b101d700 1 -- 172.21.15.43:0/3794248704 shutdown_connections 2024-04-25T10:22:43.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:43.280+0000 7f25b101d700 1 -- 172.21.15.43:0/3794248704 wait complete. 2024-04-25T10:22:43.804 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T10:22:43.827 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T10:22:43.827 INFO:teuthology.orchestra.run.smithi043.stdout:Creating mgr... 2024-04-25T10:22:43.827 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T10:22:43.828 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T10:22:44.071 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mgr.x 2024-04-25T10:22:44.071 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to reset failed state of unit ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T10:22:44.221 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-94d0035c-02ed-11ef-bc93-c7b262605968.target.wants/ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-94d0035c-02ed-11ef-bc93-c7b262605968@.service. 2024-04-25T10:22:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:44 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3794248704' entity='client.admin' 2024-04-25T10:22:44.915 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T10:22:44.915 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-04-25T10:22:44.924 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T10:22:44.924 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-04-25T10:22:44.924 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-04-25T10:22:44.924 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T10:22:44.934 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T10:22:44.935 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-04-25T10:22:44.943 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T10:22:44.943 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-04-25T10:22:44.943 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-04-25T10:22:44.943 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T10:22:44.943 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr to start... 2024-04-25T10:22:44.943 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr... 2024-04-25T10:22:45.030 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:44 smithi043 systemd[1]: Started Ceph mgr.x for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:22:45.030 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:45 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:45.027+0000 7fee016d2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:22:45.361 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:45 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:45.135+0000 7fee016d2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:22:45.726 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:45 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:45.461+0000 7fee016d2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:22:45.726 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:45 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:45.550+0000 7fee016d2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:22:46.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:45 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2699440147' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T10:22:46.335 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-25T10:22:46.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-04-25T10:22:46.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "94d0035c-02ed-11ef-bc93-c7b262605968", 2024-04-25T10:22:46.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T10:22:46.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T10:22:46.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T10:22:46.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T10:22:46.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:46.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T10:22:46.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T10:22:46.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-04-25T10:22:46.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:22:46.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T10:22:46.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T10:22:46.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:22:46.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-25T10:22:46.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T10:22:46.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:22:46.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T10:22:46.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T10:22:46.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:46.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T10:22:46.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:22:46.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T10:22:46.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T10:22:46.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T10:22:46.345 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T10:22:46.345 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T10:22:46.345 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T10:22:46.346 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:46.346 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T10:22:46.346 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T10:22:46.346 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T10:22:46.346 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T10:22:46.346 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T10:22:46.346 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T10:22:46.346 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T10:22:46.346 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T10:22:46.346 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T10:22:46.346 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:46.346 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T10:22:46.347 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:22:46.347 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T10:22:46.347 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T10:22:46.347 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:46.347 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T10:22:46.347 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T10:22:46.347 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T10:22:46.347 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T10:22:46.347 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T10:22:46.347 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T10:22:46.347 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T10:22:46.347 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:22:46.347 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T10:22:46.348 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:46.348 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T10:22:46.348 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:22:46.348 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T10:22:37.862485+0000", 2024-04-25T10:22:46.348 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T10:22:46.348 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:46.348 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T10:22:46.348 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-04-25T10:22:46.348 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.654+0000 7f08a89fa700 1 Processor -- start 2024-04-25T10:22:46.348 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.654+0000 7f08a89fa700 1 -- start start 2024-04-25T10:22:46.348 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.654+0000 7f08a89fa700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a410a710 0x7f08a410cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:46.348 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.654+0000 7f08a89fa700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f08a407ce60 con 0x7f08a410a710 2024-04-25T10:22:46.349 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.655+0000 7f08a259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a410a710 0x7f08a410cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:46.349 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.655+0000 7f08a259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a410a710 0x7f08a410cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42038/0 (socket says 172.21.15.43:42038) 2024-04-25T10:22:46.349 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.655+0000 7f08a259c700 1 -- 172.21.15.43:0/2846367976 learned_addr learned my addr 172.21.15.43:0/2846367976 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:46.349 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.655+0000 7f08a259c700 1 -- 172.21.15.43:0/2846367976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a410d070 con 0x7f08a410a710 2024-04-25T10:22:46.349 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.655+0000 7f08a259c700 1 --2- 172.21.15.43:0/2846367976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a410a710 0x7f08a410cb30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f08940098a0 tx=0x7f0894009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1675fb045d8e357e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:46.349 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.655+0000 7f08a159a700 1 -- 172.21.15.43:0/2846367976 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0894004160 con 0x7f08a410a710 2024-04-25T10:22:46.349 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.655+0000 7f08a159a700 1 -- 172.21.15.43:0/2846367976 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0894004300 con 0x7f08a410a710 2024-04-25T10:22:46.349 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.655+0000 7f08a159a700 1 -- 172.21.15.43:0/2846367976 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0894011af0 con 0x7f08a410a710 2024-04-25T10:22:46.349 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.656+0000 7f08a89fa700 1 -- 172.21.15.43:0/2846367976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a410a710 msgr2=0x7f08a410cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:46.349 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.656+0000 7f08a89fa700 1 --2- 172.21.15.43:0/2846367976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a410a710 0x7f08a410cb30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f08940098a0 tx=0x7f0894009bb0 comp rx=0 tx=0).stop 2024-04-25T10:22:46.349 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.656+0000 7f08a89fa700 1 -- 172.21.15.43:0/2846367976 shutdown_connections 2024-04-25T10:22:46.349 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.656+0000 7f08a89fa700 1 --2- 172.21.15.43:0/2846367976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a410a710 0x7f08a410cb30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:46.349 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.656+0000 7f08a89fa700 1 -- 172.21.15.43:0/2846367976 >> 172.21.15.43:0/2846367976 conn(0x7f08a4104170 msgr2=0x7f08a41065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:46.350 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.656+0000 7f08a89fa700 1 -- 172.21.15.43:0/2846367976 shutdown_connections 2024-04-25T10:22:46.350 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.656+0000 7f08a89fa700 1 -- 172.21.15.43:0/2846367976 wait complete. 2024-04-25T10:22:46.350 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.657+0000 7f08a89fa700 1 Processor -- start 2024-04-25T10:22:46.350 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.657+0000 7f08a89fa700 1 -- start start 2024-04-25T10:22:46.350 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.657+0000 7f08a89fa700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a410a710 0x7f08a419dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:46.350 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.657+0000 7f08a89fa700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f08a410dfd0 con 0x7f08a410a710 2024-04-25T10:22:46.350 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.657+0000 7f08a259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a410a710 0x7f08a419dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:46.350 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.657+0000 7f08a259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a410a710 0x7f08a419dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42048/0 (socket says 172.21.15.43:42048) 2024-04-25T10:22:46.350 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.657+0000 7f08a259c700 1 -- 172.21.15.43:0/2699440147 learned_addr learned my addr 172.21.15.43:0/2699440147 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:46.350 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.658+0000 7f08a259c700 1 -- 172.21.15.43:0/2699440147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a419e530 con 0x7f08a410a710 2024-04-25T10:22:46.350 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.658+0000 7f08a259c700 1 --2- 172.21.15.43:0/2699440147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a410a710 0x7f08a419dff0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0894009fb0 tx=0x7f0894011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:46.350 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.658+0000 7f08937fe700 1 -- 172.21.15.43:0/2699440147 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08940036a0 con 0x7f08a410a710 2024-04-25T10:22:46.350 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.658+0000 7f08a89fa700 1 -- 172.21.15.43:0/2699440147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a419e7c0 con 0x7f08a410a710 2024-04-25T10:22:46.351 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.658+0000 7f08937fe700 1 -- 172.21.15.43:0/2699440147 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f089401a430 con 0x7f08a410a710 2024-04-25T10:22:46.351 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.658+0000 7f08937fe700 1 -- 172.21.15.43:0/2699440147 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0894019550 con 0x7f08a410a710 2024-04-25T10:22:46.351 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.658+0000 7f08a89fa700 1 -- 172.21.15.43:0/2699440147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a41a13d0 con 0x7f08a410a710 2024-04-25T10:22:46.351 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.658+0000 7f08937fe700 1 -- 172.21.15.43:0/2699440147 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0894019be0 con 0x7f08a410a710 2024-04-25T10:22:46.351 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.658+0000 7f08937fe700 1 -- 172.21.15.43:0/2699440147 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0894024730 con 0x7f08a410a710 2024-04-25T10:22:46.351 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.659+0000 7f08a89fa700 1 -- 172.21.15.43:0/2699440147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08a4064cf0 con 0x7f08a410a710 2024-04-25T10:22:46.351 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.661+0000 7f08937fe700 1 -- 172.21.15.43:0/2699440147 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0894043e50 con 0x7f08a410a710 2024-04-25T10:22:46.351 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.736+0000 7f08a89fa700 1 -- 172.21.15.43:0/2699440147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f08a41a15a0 con 0x7f08a410a710 2024-04-25T10:22:46.351 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.736+0000 7f08937fe700 1 -- 172.21.15.43:0/2699440147 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0894032020 con 0x7f08a410a710 2024-04-25T10:22:46.351 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.738+0000 7f08a89fa700 1 -- 172.21.15.43:0/2699440147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a410a710 msgr2=0x7f08a419dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:46.351 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.738+0000 7f08a89fa700 1 --2- 172.21.15.43:0/2699440147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a410a710 0x7f08a419dff0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0894009fb0 tx=0x7f0894011940 comp rx=0 tx=0).stop 2024-04-25T10:22:46.351 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.738+0000 7f08a89fa700 1 -- 172.21.15.43:0/2699440147 shutdown_connections 2024-04-25T10:22:46.351 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.738+0000 7f08a89fa700 1 --2- 172.21.15.43:0/2699440147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a410a710 0x7f08a419dff0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:46.352 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.738+0000 7f08a89fa700 1 -- 172.21.15.43:0/2699440147 >> 172.21.15.43:0/2699440147 conn(0x7f08a4104170 msgr2=0x7f08a4104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:46.352 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.738+0000 7f08a89fa700 1 -- 172.21.15.43:0/2699440147 shutdown_connections 2024-04-25T10:22:46.352 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:45.738+0000 7f08a89fa700 1 -- 172.21.15.43:0/2699440147 wait complete. 2024-04-25T10:22:46.352 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (1/15)... 2024-04-25T10:22:47.111 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:46 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:46.831+0000 7fee016d2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:22:47.111 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:47 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:47.007+0000 7fee016d2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:22:47.485 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:47 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:47.184+0000 7fee016d2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:22:47.485 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:47 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:47.482+0000 7fee016d2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:22:47.790 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:47 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:47.578+0000 7fee016d2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:22:47.790 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:47 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:47.788+0000 7fee016d2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:22:48.111 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:47 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:47.986+0000 7fee016d2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:22:48.732 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:48 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:48.622+0000 7fee016d2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:22:49.107 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:48 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3992479750' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T10:22:49.361 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:49 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:49.105+0000 7fee016d2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:22:49.361 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:49 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:49.206+0000 7fee016d2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:22:49.482 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-25T10:22:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-04-25T10:22:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "94d0035c-02ed-11ef-bc93-c7b262605968", 2024-04-25T10:22:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T10:22:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T10:22:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T10:22:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T10:22:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T10:22:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T10:22:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-04-25T10:22:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:22:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T10:22:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T10:22:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:22:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-25T10:22:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T10:22:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:22:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T10:22:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T10:22:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T10:22:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:22:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T10:22:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T10:22:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T10:22:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T10:22:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T10:22:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T10:22:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T10:22:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T10:22:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T10:22:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T10:22:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T10:22:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T10:22:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T10:22:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T10:22:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T10:22:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T10:22:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:22:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T10:22:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T10:22:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T10:22:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T10:22:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T10:22:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T10:22:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T10:22:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T10:22:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T10:22:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:22:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T10:22:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:49.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T10:22:49.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:22:49.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T10:22:37.862485+0000", 2024-04-25T10:22:49.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T10:22:49.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:49.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T10:22:49.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-04-25T10:22:49.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.776+0000 7f7dfd1e9700 1 Processor -- start 2024-04-25T10:22:49.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.776+0000 7f7dfd1e9700 1 -- start start 2024-04-25T10:22:49.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.776+0000 7f7dfd1e9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8108530 0x7f7df8108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:49.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.776+0000 7f7dfd1e9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7df807ce00 con 0x7f7df8108530 2024-04-25T10:22:49.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.776+0000 7f7df6d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8108530 0x7f7df8108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:49.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.776+0000 7f7df6d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8108530 0x7f7df8108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32864/0 (socket says 172.21.15.43:32864) 2024-04-25T10:22:49.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.776+0000 7f7df6d9d700 1 -- 172.21.15.43:0/422182265 learned_addr learned my addr 172.21.15.43:0/422182265 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:49.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.776+0000 7f7df6d9d700 1 -- 172.21.15.43:0/422182265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df8108e80 con 0x7f7df8108530 2024-04-25T10:22:49.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.777+0000 7f7df6d9d700 1 --2- 172.21.15.43:0/422182265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8108530 0x7f7df8108940 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7de00098a0 tx=0x7f7de0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23241e8fa44b073 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:49.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.777+0000 7f7df5d9b700 1 -- 172.21.15.43:0/422182265 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de0004160 con 0x7f7df8108530 2024-04-25T10:22:49.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.777+0000 7f7df5d9b700 1 -- 172.21.15.43:0/422182265 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7de0004300 con 0x7f7df8108530 2024-04-25T10:22:49.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.777+0000 7f7df5d9b700 1 -- 172.21.15.43:0/422182265 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de0011af0 con 0x7f7df8108530 2024-04-25T10:22:49.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.778+0000 7f7dfd1e9700 1 -- 172.21.15.43:0/422182265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8108530 msgr2=0x7f7df8108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:49.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.778+0000 7f7dfd1e9700 1 --2- 172.21.15.43:0/422182265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8108530 0x7f7df8108940 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7de00098a0 tx=0x7f7de0009bb0 comp rx=0 tx=0).stop 2024-04-25T10:22:49.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.778+0000 7f7dfd1e9700 1 -- 172.21.15.43:0/422182265 shutdown_connections 2024-04-25T10:22:49.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.778+0000 7f7dfd1e9700 1 --2- 172.21.15.43:0/422182265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8108530 0x7f7df8108940 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:49.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.778+0000 7f7dfd1e9700 1 -- 172.21.15.43:0/422182265 >> 172.21.15.43:0/422182265 conn(0x7f7df8104120 msgr2=0x7f7df81065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:49.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.778+0000 7f7dfd1e9700 1 -- 172.21.15.43:0/422182265 shutdown_connections 2024-04-25T10:22:49.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.778+0000 7f7dfd1e9700 1 -- 172.21.15.43:0/422182265 wait complete. 2024-04-25T10:22:49.490 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.779+0000 7f7dfd1e9700 1 Processor -- start 2024-04-25T10:22:49.490 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.779+0000 7f7dfd1e9700 1 -- start start 2024-04-25T10:22:49.490 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.779+0000 7f7dfd1e9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8108530 0x7f7df81a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:49.490 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.779+0000 7f7dfd1e9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7df8109d80 con 0x7f7df8108530 2024-04-25T10:22:49.490 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.780+0000 7f7df6d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8108530 0x7f7df81a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:49.490 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.780+0000 7f7df6d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8108530 0x7f7df81a01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32874/0 (socket says 172.21.15.43:32874) 2024-04-25T10:22:49.490 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.780+0000 7f7df6d9d700 1 -- 172.21.15.43:0/3992479750 learned_addr learned my addr 172.21.15.43:0/3992479750 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:49.490 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.780+0000 7f7df6d9d700 1 -- 172.21.15.43:0/3992479750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df81a0720 con 0x7f7df8108530 2024-04-25T10:22:49.490 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.780+0000 7f7df6d9d700 1 --2- 172.21.15.43:0/3992479750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8108530 0x7f7df81a01e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009fb0 tx=0x7f7de0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:49.490 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.780+0000 7f7deffff700 1 -- 172.21.15.43:0/3992479750 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de00036a0 con 0x7f7df8108530 2024-04-25T10:22:49.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.780+0000 7f7deffff700 1 -- 172.21.15.43:0/3992479750 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7de001a430 con 0x7f7df8108530 2024-04-25T10:22:49.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.780+0000 7f7dfd1e9700 1 -- 172.21.15.43:0/3992479750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df81a09b0 con 0x7f7df8108530 2024-04-25T10:22:49.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.781+0000 7f7deffff700 1 -- 172.21.15.43:0/3992479750 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de0019550 con 0x7f7df8108530 2024-04-25T10:22:49.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.781+0000 7f7dfd1e9700 1 -- 172.21.15.43:0/3992479750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df81a35c0 con 0x7f7df8108530 2024-04-25T10:22:49.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.781+0000 7f7deffff700 1 -- 172.21.15.43:0/3992479750 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7de0019be0 con 0x7f7df8108530 2024-04-25T10:22:49.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.781+0000 7f7deffff700 1 -- 172.21.15.43:0/3992479750 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7de000ba40 con 0x7f7df8108530 2024-04-25T10:22:49.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.782+0000 7f7dfd1e9700 1 -- 172.21.15.43:0/3992479750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df8064cf0 con 0x7f7df8108530 2024-04-25T10:22:49.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.784+0000 7f7deffff700 1 -- 172.21.15.43:0/3992479750 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7de0032070 con 0x7f7df8108530 2024-04-25T10:22:49.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.899+0000 7f7dfd1e9700 1 -- 172.21.15.43:0/3992479750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7df802e8c0 con 0x7f7df8108530 2024-04-25T10:22:49.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.899+0000 7f7deffff700 1 -- 172.21.15.43:0/3992479750 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f7de001a5e0 con 0x7f7df8108530 2024-04-25T10:22:49.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.901+0000 7f7dfd1e9700 1 -- 172.21.15.43:0/3992479750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8108530 msgr2=0x7f7df81a01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:49.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.901+0000 7f7dfd1e9700 1 --2- 172.21.15.43:0/3992479750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8108530 0x7f7df81a01e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009fb0 tx=0x7f7de0011940 comp rx=0 tx=0).stop 2024-04-25T10:22:49.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.901+0000 7f7dfd1e9700 1 -- 172.21.15.43:0/3992479750 shutdown_connections 2024-04-25T10:22:49.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.901+0000 7f7dfd1e9700 1 --2- 172.21.15.43:0/3992479750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8108530 0x7f7df81a01e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:49.492 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.901+0000 7f7dfd1e9700 1 -- 172.21.15.43:0/3992479750 >> 172.21.15.43:0/3992479750 conn(0x7f7df8104120 msgr2=0x7f7df8194160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:49.492 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.901+0000 7f7dfd1e9700 1 -- 172.21.15.43:0/3992479750 shutdown_connections 2024-04-25T10:22:49.492 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:48.901+0000 7f7dfd1e9700 1 -- 172.21.15.43:0/3992479750 wait complete. 2024-04-25T10:22:49.492 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (2/15)... 2024-04-25T10:22:49.860 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:49 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:49.414+0000 7fee016d2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:22:50.313 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:49 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:49.962+0000 7fee016d2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:22:50.313 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:50 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:50.058+0000 7fee016d2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:22:50.611 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:50 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:50.310+0000 7fee016d2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:22:50.611 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:50 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:50.448+0000 7fee016d2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:22:50.611 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:50 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:50.547+0000 7fee016d2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:22:50.992 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:50 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:50.669+0000 7fee016d2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:22:50.992 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:50 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:50.989+0000 7fee016d2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:22:51.860 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:51 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:51.618+0000 7fee016d2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:22:51.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:51 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:51.847+0000 7fee016d2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:22:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:51 smithi043 ceph-mon[29478]: Activating manager daemon x 2024-04-25T10:22:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:51 smithi043 ceph-mon[29478]: mgrmap e2: x(active, starting, since 0.00402261s) 2024-04-25T10:22:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:51 smithi043 ceph-mon[29478]: from='mgr.14100 172.21.15.43:0/113454750' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T10:22:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:51 smithi043 ceph-mon[29478]: from='mgr.14100 172.21.15.43:0/113454750' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T10:22:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:51 smithi043 ceph-mon[29478]: from='mgr.14100 172.21.15.43:0/113454750' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T10:22:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:51 smithi043 ceph-mon[29478]: from='mgr.14100 172.21.15.43:0/113454750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:22:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:51 smithi043 ceph-mon[29478]: from='mgr.14100 172.21.15.43:0/113454750' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T10:22:52.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:51 smithi043 ceph-mon[29478]: Manager daemon x is now available 2024-04-25T10:22:52.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:51 smithi043 ceph-mon[29478]: from='mgr.14100 172.21.15.43:0/113454750' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T10:22:52.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:51 smithi043 ceph-mon[29478]: from='mgr.14100 172.21.15.43:0/113454750' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T10:22:52.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:51 smithi043 ceph-mon[29478]: from='mgr.14100 172.21.15.43:0/113454750' entity='mgr.x' 2024-04-25T10:22:52.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:51 smithi043 ceph-mon[29478]: from='mgr.14100 172.21.15.43:0/113454750' entity='mgr.x' 2024-04-25T10:22:52.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:51 smithi043 ceph-mon[29478]: from='mgr.14100 172.21.15.43:0/113454750' entity='mgr.x' 2024-04-25T10:22:52.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-25T10:22:52.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-04-25T10:22:52.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "94d0035c-02ed-11ef-bc93-c7b262605968", 2024-04-25T10:22:52.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T10:22:52.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T10:22:52.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T10:22:52.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T10:22:52.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:52.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T10:22:52.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T10:22:52.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-04-25T10:22:52.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:22:52.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T10:22:52.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T10:22:52.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:22:52.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-04-25T10:22:52.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T10:22:52.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:22:52.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T10:22:52.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T10:22:52.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:52.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T10:22:52.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:22:52.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T10:22:52.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T10:22:52.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T10:22:52.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T10:22:52.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T10:22:52.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T10:22:52.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:52.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T10:22:52.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T10:22:52.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T10:22:52.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T10:22:52.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T10:22:52.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T10:22:52.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T10:22:52.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T10:22:52.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T10:22:52.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:52.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T10:22:52.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:22:52.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T10:22:52.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T10:22:52.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:52.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T10:22:52.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T10:22:52.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T10:22:52.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T10:22:52.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T10:22:52.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T10:22:52.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T10:22:52.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:22:52.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T10:22:52.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:52.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T10:22:52.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:22:52.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T10:22:37.862485+0000", 2024-04-25T10:22:52.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T10:22:52.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:52.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T10:22:52.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-04-25T10:22:52.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.943+0000 7fa93fbdd700 1 Processor -- start 2024-04-25T10:22:52.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.944+0000 7fa93fbdd700 1 -- start start 2024-04-25T10:22:52.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.944+0000 7fa93fbdd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa93810a690 0x7fa93810cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:52.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.944+0000 7fa93fbdd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa93807d730 con 0x7fa93810a690 2024-04-25T10:22:52.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.944+0000 7fa93d979700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa93810a690 0x7fa93810cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:52.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.944+0000 7fa93d979700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa93810a690 0x7fa93810cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32960/0 (socket says 172.21.15.43:32960) 2024-04-25T10:22:52.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.944+0000 7fa93d979700 1 -- 172.21.15.43:0/4132061562 learned_addr learned my addr 172.21.15.43:0/4132061562 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:52.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.944+0000 7fa93d979700 1 -- 172.21.15.43:0/4132061562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93810cff0 con 0x7fa93810a690 2024-04-25T10:22:52.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.945+0000 7fa93d979700 1 --2- 172.21.15.43:0/4132061562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa93810a690 0x7fa93810cab0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa9340098a0 tx=0x7fa934009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e54f169dc17b977 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:52.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.945+0000 7fa93c977700 1 -- 172.21.15.43:0/4132061562 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa934004160 con 0x7fa93810a690 2024-04-25T10:22:52.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.945+0000 7fa93c977700 1 -- 172.21.15.43:0/4132061562 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa934004300 con 0x7fa93810a690 2024-04-25T10:22:52.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.945+0000 7fa93c977700 1 -- 172.21.15.43:0/4132061562 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa934011af0 con 0x7fa93810a690 2024-04-25T10:22:52.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.946+0000 7fa93fbdd700 1 -- 172.21.15.43:0/4132061562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa93810a690 msgr2=0x7fa93810cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:52.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.946+0000 7fa93fbdd700 1 --2- 172.21.15.43:0/4132061562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa93810a690 0x7fa93810cab0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa9340098a0 tx=0x7fa934009bb0 comp rx=0 tx=0).stop 2024-04-25T10:22:52.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.946+0000 7fa93fbdd700 1 -- 172.21.15.43:0/4132061562 shutdown_connections 2024-04-25T10:22:52.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.946+0000 7fa93fbdd700 1 --2- 172.21.15.43:0/4132061562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa93810a690 0x7fa93810cab0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:52.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.946+0000 7fa93fbdd700 1 -- 172.21.15.43:0/4132061562 >> 172.21.15.43:0/4132061562 conn(0x7fa938104170 msgr2=0x7fa9381065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:52.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.946+0000 7fa93fbdd700 1 -- 172.21.15.43:0/4132061562 shutdown_connections 2024-04-25T10:22:52.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.946+0000 7fa93fbdd700 1 -- 172.21.15.43:0/4132061562 wait complete. 2024-04-25T10:22:52.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.946+0000 7fa93fbdd700 1 Processor -- start 2024-04-25T10:22:52.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.946+0000 7fa93fbdd700 1 -- start start 2024-04-25T10:22:52.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.947+0000 7fa93fbdd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa93810a690 0x7fa93819dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:52.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.947+0000 7fa93fbdd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa93810dfb0 con 0x7fa93810a690 2024-04-25T10:22:52.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.947+0000 7fa93d979700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa93810a690 0x7fa93819dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:52.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.947+0000 7fa93d979700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa93810a690 0x7fa93819dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32962/0 (socket says 172.21.15.43:32962) 2024-04-25T10:22:52.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.947+0000 7fa93d979700 1 -- 172.21.15.43:0/1278794336 learned_addr learned my addr 172.21.15.43:0/1278794336 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:52.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.947+0000 7fa93d979700 1 -- 172.21.15.43:0/1278794336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93819e520 con 0x7fa93810a690 2024-04-25T10:22:52.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.947+0000 7fa93d979700 1 --2- 172.21.15.43:0/1278794336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa93810a690 0x7fa93819dfe0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa9340117b0 tx=0x7fa934011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:52.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.947+0000 7fa92effd700 1 -- 172.21.15.43:0/1278794336 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa934003af0 con 0x7fa93810a690 2024-04-25T10:22:52.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.947+0000 7fa92effd700 1 -- 172.21.15.43:0/1278794336 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa934020070 con 0x7fa93810a690 2024-04-25T10:22:52.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.947+0000 7fa93fbdd700 1 -- 172.21.15.43:0/1278794336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa93819e7b0 con 0x7fa93810a690 2024-04-25T10:22:52.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.947+0000 7fa92effd700 1 -- 172.21.15.43:0/1278794336 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa934019630 con 0x7fa93810a690 2024-04-25T10:22:52.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.947+0000 7fa93fbdd700 1 -- 172.21.15.43:0/1278794336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9381a13c0 con 0x7fa93810a690 2024-04-25T10:22:52.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.948+0000 7fa92effd700 1 -- 172.21.15.43:0/1278794336 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 2) ==== 49272+0+0 (secure 0 0 0) 0x7fa9340198a0 con 0x7fa93810a690 2024-04-25T10:22:52.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.948+0000 7fa93fbdd700 1 -- 172.21.15.43:0/1278794336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa938064cf0 con 0x7fa93810a690 2024-04-25T10:22:52.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.949+0000 7fa92effd700 1 -- 172.21.15.43:0/1278794336 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa934052520 con 0x7fa93810a690 2024-04-25T10:22:52.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:51.951+0000 7fa92effd700 1 -- 172.21.15.43:0/1278794336 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa93402ade0 con 0x7fa93810a690 2024-04-25T10:22:52.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:52.023+0000 7fa93fbdd700 1 -- 172.21.15.43:0/1278794336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa9381a16b0 con 0x7fa93810a690 2024-04-25T10:22:52.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:52.024+0000 7fa92effd700 1 -- 172.21.15.43:0/1278794336 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa93402bdc0 con 0x7fa93810a690 2024-04-25T10:22:52.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:52.024+0000 7fa93fbdd700 1 -- 172.21.15.43:0/1278794336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa93810a690 msgr2=0x7fa93819dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:52.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:52.024+0000 7fa93fbdd700 1 --2- 172.21.15.43:0/1278794336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa93810a690 0x7fa93819dfe0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa9340117b0 tx=0x7fa934011b70 comp rx=0 tx=0).stop 2024-04-25T10:22:52.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:52.024+0000 7fa93fbdd700 1 -- 172.21.15.43:0/1278794336 shutdown_connections 2024-04-25T10:22:52.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:52.024+0000 7fa93fbdd700 1 --2- 172.21.15.43:0/1278794336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa93810a690 0x7fa93819dfe0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:52.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:52.024+0000 7fa93fbdd700 1 -- 172.21.15.43:0/1278794336 >> 172.21.15.43:0/1278794336 conn(0x7fa938104170 msgr2=0x7fa938105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:52.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:52.025+0000 7fa93fbdd700 1 -- 172.21.15.43:0/1278794336 shutdown_connections 2024-04-25T10:22:52.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:52.025+0000 7fa93fbdd700 1 -- 172.21.15.43:0/1278794336 wait complete. 2024-04-25T10:22:52.622 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (3/15)... 2024-04-25T10:22:53.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:52 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1278794336' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T10:22:53.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:52 smithi043 ceph-mon[29478]: mgrmap e3: x(active, since 1.00879s) 2024-04-25T10:22:55.002 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:54 smithi043 ceph-mon[29478]: mgrmap e4: x(active, since 2s) 2024-04-25T10:22:55.909 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-25T10:22:55.921 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-04-25T10:22:55.921 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "94d0035c-02ed-11ef-bc93-c7b262605968", 2024-04-25T10:22:55.921 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T10:22:55.921 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T10:22:55.921 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T10:22:55.921 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T10:22:55.921 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:55.921 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T10:22:55.921 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T10:22:55.921 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-04-25T10:22:55.922 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:22:55.922 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T10:22:55.922 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T10:22:55.922 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:22:55.922 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-25T10:22:55.922 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T10:22:55.922 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:22:55.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T10:22:55.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T10:22:55.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:55.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T10:22:55.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:22:55.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T10:22:55.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T10:22:55.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T10:22:55.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T10:22:55.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T10:22:55.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T10:22:55.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:55.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T10:22:55.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T10:22:55.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T10:22:55.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T10:22:55.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T10:22:55.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T10:22:55.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T10:22:55.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T10:22:55.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T10:22:55.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:55.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T10:22:55.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:22:55.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T10:22:55.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T10:22:55.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:55.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T10:22:55.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T10:22:55.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T10:22:55.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T10:22:55.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T10:22:55.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T10:22:55.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T10:22:55.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-25T10:22:55.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T10:22:55.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:55.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T10:22:55.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T10:22:55.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T10:22:37.862485+0000", 2024-04-25T10:22:55.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T10:22:55.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-25T10:22:55.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T10:22:55.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-04-25T10:22:55.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.037+0000 7fce8b35c700 1 Processor -- start 2024-04-25T10:22:55.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.038+0000 7fce8b35c700 1 -- start start 2024-04-25T10:22:55.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.038+0000 7fce8b35c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce84108500 0x7fce84108910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:55.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.038+0000 7fce8b35c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fce8407ce60 con 0x7fce84108500 2024-04-25T10:22:55.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.038+0000 7fce890f8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce84108500 0x7fce84108910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:55.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.038+0000 7fce890f8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce84108500 0x7fce84108910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32976/0 (socket says 172.21.15.43:32976) 2024-04-25T10:22:55.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.038+0000 7fce890f8700 1 -- 172.21.15.43:0/821750730 learned_addr learned my addr 172.21.15.43:0/821750730 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:55.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.039+0000 7fce890f8700 1 -- 172.21.15.43:0/821750730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce84108e50 con 0x7fce84108500 2024-04-25T10:22:55.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.039+0000 7fce890f8700 1 --2- 172.21.15.43:0/821750730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce84108500 0x7fce84108910 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fce780098a0 tx=0x7fce78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f787a149f4c18f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:55.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.039+0000 7fce77fff700 1 -- 172.21.15.43:0/821750730 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce78004160 con 0x7fce84108500 2024-04-25T10:22:55.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.039+0000 7fce77fff700 1 -- 172.21.15.43:0/821750730 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fce78004300 con 0x7fce84108500 2024-04-25T10:22:55.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.039+0000 7fce77fff700 1 -- 172.21.15.43:0/821750730 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce78011af0 con 0x7fce84108500 2024-04-25T10:22:55.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.040+0000 7fce8b35c700 1 -- 172.21.15.43:0/821750730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce84108500 msgr2=0x7fce84108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:55.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.040+0000 7fce8b35c700 1 --2- 172.21.15.43:0/821750730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce84108500 0x7fce84108910 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fce780098a0 tx=0x7fce78009bb0 comp rx=0 tx=0).stop 2024-04-25T10:22:55.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.040+0000 7fce8b35c700 1 -- 172.21.15.43:0/821750730 shutdown_connections 2024-04-25T10:22:55.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.040+0000 7fce8b35c700 1 --2- 172.21.15.43:0/821750730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce84108500 0x7fce84108910 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:55.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.040+0000 7fce8b35c700 1 -- 172.21.15.43:0/821750730 >> 172.21.15.43:0/821750730 conn(0x7fce84104170 msgr2=0x7fce841065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:55.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.040+0000 7fce8b35c700 1 -- 172.21.15.43:0/821750730 shutdown_connections 2024-04-25T10:22:55.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.041+0000 7fce8b35c700 1 -- 172.21.15.43:0/821750730 wait complete. 2024-04-25T10:22:55.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.041+0000 7fce8b35c700 1 Processor -- start 2024-04-25T10:22:55.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.041+0000 7fce8b35c700 1 -- start start 2024-04-25T10:22:55.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.041+0000 7fce8b35c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce84108500 0x7fce841a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:55.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.041+0000 7fce8b35c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fce84109ed0 con 0x7fce84108500 2024-04-25T10:22:55.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.042+0000 7fce890f8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce84108500 0x7fce841a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:55.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.042+0000 7fce890f8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce84108500 0x7fce841a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32984/0 (socket says 172.21.15.43:32984) 2024-04-25T10:22:55.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.042+0000 7fce890f8700 1 -- 172.21.15.43:0/3948874096 learned_addr learned my addr 172.21.15.43:0/3948874096 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:55.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.042+0000 7fce890f8700 1 -- 172.21.15.43:0/3948874096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce841a08e0 con 0x7fce84108500 2024-04-25T10:22:55.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.042+0000 7fce890f8700 1 --2- 172.21.15.43:0/3948874096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce84108500 0x7fce841a03a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fce78009fb0 tx=0x7fce78011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:55.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.042+0000 7fce767fc700 1 -- 172.21.15.43:0/3948874096 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce780036a0 con 0x7fce84108500 2024-04-25T10:22:55.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.042+0000 7fce767fc700 1 -- 172.21.15.43:0/3948874096 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fce7801a430 con 0x7fce84108500 2024-04-25T10:22:55.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.042+0000 7fce8b35c700 1 -- 172.21.15.43:0/3948874096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce841a0b70 con 0x7fce84108500 2024-04-25T10:22:55.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.042+0000 7fce767fc700 1 -- 172.21.15.43:0/3948874096 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce78019550 con 0x7fce84108500 2024-04-25T10:22:55.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.042+0000 7fce8b35c700 1 -- 172.21.15.43:0/3948874096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce841a15b0 con 0x7fce84108500 2024-04-25T10:22:55.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.043+0000 7fce8b35c700 1 -- 172.21.15.43:0/3948874096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce84064cf0 con 0x7fce84108500 2024-04-25T10:22:55.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.044+0000 7fce767fc700 1 -- 172.21.15.43:0/3948874096 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7fce78019850 con 0x7fce84108500 2024-04-25T10:22:55.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.044+0000 7fce767fc700 1 --2- 172.21.15.43:0/3948874096 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7fce7003d4d0 0x7fce7003f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:55.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.044+0000 7fce767fc700 1 -- 172.21.15.43:0/3948874096 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fce78052680 con 0x7fce84108500 2024-04-25T10:22:55.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.044+0000 7fce888f7700 1 --2- 172.21.15.43:0/3948874096 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7fce7003d4d0 0x7fce7003f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:55.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.044+0000 7fce888f7700 1 --2- 172.21.15.43:0/3948874096 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7fce7003d4d0 0x7fce7003f9e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fce80006fd0 tx=0x7fce80006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:55.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.049+0000 7fce767fc700 1 -- 172.21.15.43:0/3948874096 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce78017e90 con 0x7fce84108500 2024-04-25T10:22:55.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.344+0000 7fce8b35c700 1 -- 172.21.15.43:0/3948874096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fce841a2250 con 0x7fce84108500 2024-04-25T10:22:55.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.345+0000 7fce767fc700 1 -- 172.21.15.43:0/3948874096 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fce78023390 con 0x7fce84108500 2024-04-25T10:22:55.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.347+0000 7fce8b35c700 1 -- 172.21.15.43:0/3948874096 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7fce7003d4d0 msgr2=0x7fce7003f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:55.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.347+0000 7fce8b35c700 1 --2- 172.21.15.43:0/3948874096 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7fce7003d4d0 0x7fce7003f9e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fce80006fd0 tx=0x7fce80006e40 comp rx=0 tx=0).stop 2024-04-25T10:22:55.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.347+0000 7fce8b35c700 1 -- 172.21.15.43:0/3948874096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce84108500 msgr2=0x7fce841a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:55.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.347+0000 7fce8b35c700 1 --2- 172.21.15.43:0/3948874096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce84108500 0x7fce841a03a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fce78009fb0 tx=0x7fce78011940 comp rx=0 tx=0).stop 2024-04-25T10:22:55.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.347+0000 7fce8b35c700 1 -- 172.21.15.43:0/3948874096 shutdown_connections 2024-04-25T10:22:55.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.347+0000 7fce8b35c700 1 --2- 172.21.15.43:0/3948874096 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7fce7003d4d0 0x7fce7003f9e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:55.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.347+0000 7fce8b35c700 1 --2- 172.21.15.43:0/3948874096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce84108500 0x7fce841a03a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:55.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.347+0000 7fce8b35c700 1 -- 172.21.15.43:0/3948874096 >> 172.21.15.43:0/3948874096 conn(0x7fce84104170 msgr2=0x7fce84194270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:55.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.347+0000 7fce8b35c700 1 -- 172.21.15.43:0/3948874096 shutdown_connections 2024-04-25T10:22:55.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:55.347+0000 7fce8b35c700 1 -- 172.21.15.43:0/3948874096 wait complete. 2024-04-25T10:22:55.934 INFO:teuthology.orchestra.run.smithi043.stdout:mgr is available 2024-04-25T10:22:56.023 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:55 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3948874096' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T10:22:57.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:56 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3921346091' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T10:22:57.121 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-25T10:22:57.121 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T10:22:57.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout fsid = 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:22:57.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T10:22:57.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T10:22:57.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T10:22:57.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T10:22:57.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T10:22:57.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-25T10:22:57.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T10:22:57.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T10:22:57.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-25T10:22:57.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T10:22:57.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T10:22:57.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T10:22:57.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T10:22:57.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T10:22:57.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T10:22:57.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T10:22:57.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.315+0000 7f1f7d13e700 1 Processor -- start 2024-04-25T10:22:57.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.315+0000 7f1f7d13e700 1 -- start start 2024-04-25T10:22:57.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.315+0000 7f1f7d13e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f78109bc0 0x7f1f78109fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:57.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.315+0000 7f1f7d13e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7810a5c0 con 0x7f1f78109bc0 2024-04-25T10:22:57.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.315+0000 7f1f76d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f78109bc0 0x7f1f78109fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:57.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.315+0000 7f1f76d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f78109bc0 0x7f1f78109fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32998/0 (socket says 172.21.15.43:32998) 2024-04-25T10:22:57.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.315+0000 7f1f76d9d700 1 -- 172.21.15.43:0/2783793517 learned_addr learned my addr 172.21.15.43:0/2783793517 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:57.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.315+0000 7f1f76d9d700 1 -- 172.21.15.43:0/2783793517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f7810ad90 con 0x7f1f78109bc0 2024-04-25T10:22:57.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.316+0000 7f1f76d9d700 1 --2- 172.21.15.43:0/2783793517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f78109bc0 0x7f1f78109fd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1f680098a0 tx=0x7f1f68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996a572ac35a9c57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:57.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.316+0000 7f1f75d9b700 1 -- 172.21.15.43:0/2783793517 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f68004160 con 0x7f1f78109bc0 2024-04-25T10:22:57.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.316+0000 7f1f75d9b700 1 -- 172.21.15.43:0/2783793517 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1f68004300 con 0x7f1f78109bc0 2024-04-25T10:22:57.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.316+0000 7f1f75d9b700 1 -- 172.21.15.43:0/2783793517 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f68011af0 con 0x7f1f78109bc0 2024-04-25T10:22:57.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.317+0000 7f1f7d13e700 1 -- 172.21.15.43:0/2783793517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f78109bc0 msgr2=0x7f1f78109fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:57.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.317+0000 7f1f7d13e700 1 --2- 172.21.15.43:0/2783793517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f78109bc0 0x7f1f78109fd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1f680098a0 tx=0x7f1f68009bb0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.317+0000 7f1f7d13e700 1 -- 172.21.15.43:0/2783793517 shutdown_connections 2024-04-25T10:22:57.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.317+0000 7f1f7d13e700 1 --2- 172.21.15.43:0/2783793517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f78109bc0 0x7f1f78109fd0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.317+0000 7f1f7d13e700 1 -- 172.21.15.43:0/2783793517 >> 172.21.15.43:0/2783793517 conn(0x7f1f78105280 msgr2=0x7f1f781076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:57.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.317+0000 7f1f7d13e700 1 -- 172.21.15.43:0/2783793517 shutdown_connections 2024-04-25T10:22:57.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.317+0000 7f1f7d13e700 1 -- 172.21.15.43:0/2783793517 wait complete. 2024-04-25T10:22:57.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.318+0000 7f1f7d13e700 1 Processor -- start 2024-04-25T10:22:57.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.318+0000 7f1f7d13e700 1 -- start start 2024-04-25T10:22:57.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.318+0000 7f1f7d13e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f78109bc0 0x7f1f7819e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:57.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.318+0000 7f1f7d13e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7810e140 con 0x7f1f78109bc0 2024-04-25T10:22:57.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.318+0000 7f1f76d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f78109bc0 0x7f1f7819e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:57.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.318+0000 7f1f76d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f78109bc0 0x7f1f7819e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33014/0 (socket says 172.21.15.43:33014) 2024-04-25T10:22:57.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.318+0000 7f1f76d9d700 1 -- 172.21.15.43:0/3921346091 learned_addr learned my addr 172.21.15.43:0/3921346091 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:57.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.318+0000 7f1f76d9d700 1 -- 172.21.15.43:0/3921346091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f7819e5a0 con 0x7f1f78109bc0 2024-04-25T10:22:57.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.319+0000 7f1f76d9d700 1 --2- 172.21.15.43:0/3921346091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f78109bc0 0x7f1f7819e060 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1f68011890 tx=0x7f1f68011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:57.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.319+0000 7f1f5ffff700 1 -- 172.21.15.43:0/3921346091 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f680039e0 con 0x7f1f78109bc0 2024-04-25T10:22:57.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.319+0000 7f1f5ffff700 1 -- 172.21.15.43:0/3921346091 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1f68003b80 con 0x7f1f78109bc0 2024-04-25T10:22:57.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.319+0000 7f1f7d13e700 1 -- 172.21.15.43:0/3921346091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f7819e830 con 0x7f1f78109bc0 2024-04-25T10:22:57.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.319+0000 7f1f5ffff700 1 -- 172.21.15.43:0/3921346091 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f680246e0 con 0x7f1f78109bc0 2024-04-25T10:22:57.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.319+0000 7f1f7d13e700 1 -- 172.21.15.43:0/3921346091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f781a1440 con 0x7f1f78109bc0 2024-04-25T10:22:57.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.320+0000 7f1f5ffff700 1 -- 172.21.15.43:0/3921346091 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7f1f68022020 con 0x7f1f78109bc0 2024-04-25T10:22:57.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.320+0000 7f1f7d13e700 1 -- 172.21.15.43:0/3921346091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f78064cf0 con 0x7f1f78109bc0 2024-04-25T10:22:57.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.320+0000 7f1f5ffff700 1 --2- 172.21.15.43:0/3921346091 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f1f6003d520 0x7f1f6003fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:57.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.320+0000 7f1f5ffff700 1 -- 172.21.15.43:0/3921346091 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1f680521f0 con 0x7f1f78109bc0 2024-04-25T10:22:57.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.320+0000 7f1f7659c700 1 --2- 172.21.15.43:0/3921346091 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f1f6003d520 0x7f1f6003fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:57.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.321+0000 7f1f7659c700 1 --2- 172.21.15.43:0/3921346091 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f1f6003d520 0x7f1f6003fa30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c006fd0 tx=0x7f1f6c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:57.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.325+0000 7f1f5ffff700 1 -- 172.21.15.43:0/3921346091 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f680172b0 con 0x7f1f78109bc0 2024-04-25T10:22:57.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.531+0000 7f1f7d13e700 1 -- 172.21.15.43:0/3921346091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f1f781a1610 con 0x7f1f78109bc0 2024-04-25T10:22:57.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.533+0000 7f1f5ffff700 1 -- 172.21.15.43:0/3921346091 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f1f68051f50 con 0x7f1f78109bc0 2024-04-25T10:22:57.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.536+0000 7f1f7d13e700 1 -- 172.21.15.43:0/3921346091 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f1f6003d520 msgr2=0x7f1f6003fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:57.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.536+0000 7f1f7d13e700 1 --2- 172.21.15.43:0/3921346091 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f1f6003d520 0x7f1f6003fa30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c006fd0 tx=0x7f1f6c006e40 comp rx=0 tx=0).stop 2024-04-25T10:22:57.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.536+0000 7f1f7d13e700 1 -- 172.21.15.43:0/3921346091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f78109bc0 msgr2=0x7f1f7819e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:57.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.536+0000 7f1f7d13e700 1 --2- 172.21.15.43:0/3921346091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f78109bc0 0x7f1f7819e060 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1f68011890 tx=0x7f1f68011970 comp rx=0 tx=0).stop 2024-04-25T10:22:57.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.536+0000 7f1f7d13e700 1 -- 172.21.15.43:0/3921346091 shutdown_connections 2024-04-25T10:22:57.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.536+0000 7f1f7d13e700 1 --2- 172.21.15.43:0/3921346091 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f1f6003d520 0x7f1f6003fa30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.536+0000 7f1f7d13e700 1 --2- 172.21.15.43:0/3921346091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f78109bc0 0x7f1f7819e060 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.536+0000 7f1f7d13e700 1 -- 172.21.15.43:0/3921346091 >> 172.21.15.43:0/3921346091 conn(0x7f1f78105280 msgr2=0x7f1f78106c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:57.127 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.536+0000 7f1f7d13e700 1 -- 172.21.15.43:0/3921346091 shutdown_connections 2024-04-25T10:22:57.127 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:56.536+0000 7f1f7d13e700 1 -- 172.21.15.43:0/3921346091 wait complete. 2024-04-25T10:22:57.127 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling cephadm module... 2024-04-25T10:22:58.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:57 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3896697146' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T10:22:58.111 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:57 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: ignoring --setuser ceph since I am not root 2024-04-25T10:22:58.111 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:57 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: ignoring --setgroup ceph since I am not root 2024-04-25T10:22:58.421 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:58 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:58.116+0000 7f0c07658200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:22:58.421 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:58 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:58.310+0000 7f0c07658200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:22:58.421 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:58 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:58.418+0000 7f0c07658200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:22:58.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.529+0000 7f8d8ae43700 1 Processor -- start 2024-04-25T10:22:58.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.529+0000 7f8d8ae43700 1 -- start start 2024-04-25T10:22:58.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.529+0000 7f8d8ae43700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d8407f6e0 0x7f8d8407db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:58.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.529+0000 7f8d8ae43700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8407e0e0 con 0x7f8d8407f6e0 2024-04-25T10:22:58.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.529+0000 7f8d88bdf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d8407f6e0 0x7f8d8407db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:58.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.529+0000 7f8d88bdf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d8407f6e0 0x7f8d8407db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33028/0 (socket says 172.21.15.43:33028) 2024-04-25T10:22:58.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.529+0000 7f8d88bdf700 1 -- 172.21.15.43:0/3232926654 learned_addr learned my addr 172.21.15.43:0/3232926654 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:58.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.530+0000 7f8d88bdf700 1 -- 172.21.15.43:0/3232926654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d8407e260 con 0x7f8d8407f6e0 2024-04-25T10:22:58.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.530+0000 7f8d88bdf700 1 --2- 172.21.15.43:0/3232926654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d8407f6e0 0x7f8d8407db10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8d780098a0 tx=0x7f8d78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1588fa2d519df165 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:58.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.530+0000 7f8d837fe700 1 -- 172.21.15.43:0/3232926654 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d78004160 con 0x7f8d8407f6e0 2024-04-25T10:22:58.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.530+0000 7f8d837fe700 1 -- 172.21.15.43:0/3232926654 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8d78004300 con 0x7f8d8407f6e0 2024-04-25T10:22:58.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.530+0000 7f8d837fe700 1 -- 172.21.15.43:0/3232926654 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d78011af0 con 0x7f8d8407f6e0 2024-04-25T10:22:58.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.531+0000 7f8d8ae43700 1 -- 172.21.15.43:0/3232926654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d8407f6e0 msgr2=0x7f8d8407db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:58.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.531+0000 7f8d8ae43700 1 --2- 172.21.15.43:0/3232926654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d8407f6e0 0x7f8d8407db10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8d780098a0 tx=0x7f8d78009bb0 comp rx=0 tx=0).stop 2024-04-25T10:22:58.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.531+0000 7f8d8ae43700 1 -- 172.21.15.43:0/3232926654 shutdown_connections 2024-04-25T10:22:58.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.531+0000 7f8d8ae43700 1 --2- 172.21.15.43:0/3232926654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d8407f6e0 0x7f8d8407db10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:58.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.531+0000 7f8d8ae43700 1 -- 172.21.15.43:0/3232926654 >> 172.21.15.43:0/3232926654 conn(0x7f8d841052c0 msgr2=0x7f8d84107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:58.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.531+0000 7f8d8ae43700 1 -- 172.21.15.43:0/3232926654 shutdown_connections 2024-04-25T10:22:58.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.531+0000 7f8d8ae43700 1 -- 172.21.15.43:0/3232926654 wait complete. 2024-04-25T10:22:58.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.532+0000 7f8d8ae43700 1 Processor -- start 2024-04-25T10:22:58.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.532+0000 7f8d8ae43700 1 -- start start 2024-04-25T10:22:58.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.532+0000 7f8d8ae43700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d8407f6e0 0x7f8d841a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:58.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.532+0000 7f8d8ae43700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8d841101f0 con 0x7f8d8407f6e0 2024-04-25T10:22:58.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.532+0000 7f8d88bdf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d8407f6e0 0x7f8d841a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:58.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.532+0000 7f8d88bdf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d8407f6e0 0x7f8d841a0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33038/0 (socket says 172.21.15.43:33038) 2024-04-25T10:22:58.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.532+0000 7f8d88bdf700 1 -- 172.21.15.43:0/3896697146 learned_addr learned my addr 172.21.15.43:0/3896697146 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:58.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.532+0000 7f8d88bdf700 1 -- 172.21.15.43:0/3896697146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d841a0750 con 0x7f8d8407f6e0 2024-04-25T10:22:58.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.532+0000 7f8d88bdf700 1 --2- 172.21.15.43:0/3896697146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d8407f6e0 0x7f8d841a0210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009fb0 tx=0x7f8d78011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:58.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.532+0000 7f8d81ffb700 1 -- 172.21.15.43:0/3896697146 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d780036a0 con 0x7f8d8407f6e0 2024-04-25T10:22:58.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.533+0000 7f8d81ffb700 1 -- 172.21.15.43:0/3896697146 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8d7801a430 con 0x7f8d8407f6e0 2024-04-25T10:22:58.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.533+0000 7f8d8ae43700 1 -- 172.21.15.43:0/3896697146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d841a09e0 con 0x7f8d8407f6e0 2024-04-25T10:22:58.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.533+0000 7f8d81ffb700 1 -- 172.21.15.43:0/3896697146 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d78019550 con 0x7f8d8407f6e0 2024-04-25T10:22:58.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.533+0000 7f8d8ae43700 1 -- 172.21.15.43:0/3896697146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d841a35f0 con 0x7f8d8407f6e0 2024-04-25T10:22:58.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.534+0000 7f8d81ffb700 1 -- 172.21.15.43:0/3896697146 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7f8d78019850 con 0x7f8d8407f6e0 2024-04-25T10:22:58.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.534+0000 7f8d8ae43700 1 -- 172.21.15.43:0/3896697146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d84064cf0 con 0x7f8d8407f6e0 2024-04-25T10:22:58.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.534+0000 7f8d81ffb700 1 --2- 172.21.15.43:0/3896697146 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f8d7403d4d0 0x7f8d7403f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:58.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.534+0000 7f8d81ffb700 1 -- 172.21.15.43:0/3896697146 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8d78052780 con 0x7f8d8407f6e0 2024-04-25T10:22:58.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.534+0000 7f8d83fff700 1 --2- 172.21.15.43:0/3896697146 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f8d7403d4d0 0x7f8d7403f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:58.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.535+0000 7f8d83fff700 1 --2- 172.21.15.43:0/3896697146 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f8d7403d4d0 0x7f8d7403f9e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8d70006fd0 tx=0x7f8d70006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:58.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.539+0000 7f8d81ffb700 1 -- 172.21.15.43:0/3896697146 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d78023390 con 0x7f8d8407f6e0 2024-04-25T10:22:58.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.808+0000 7f8d8ae43700 1 -- 172.21.15.43:0/3896697146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f8d841a3c70 con 0x7f8d8407f6e0 2024-04-25T10:22:58.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.906+0000 7f8d81ffb700 1 -- 172.21.15.43:0/3896697146 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f8d780521a0 con 0x7f8d8407f6e0 2024-04-25T10:22:58.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.908+0000 7f8d81ffb700 1 -- 172.21.15.43:0/3896697146 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7f8d7802d440 con 0x7f8d8407f6e0 2024-04-25T10:22:58.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.910+0000 7f8d8ae43700 1 -- 172.21.15.43:0/3896697146 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f8d7403d4d0 msgr2=0x7f8d7403f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:58.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.910+0000 7f8d8ae43700 1 --2- 172.21.15.43:0/3896697146 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f8d7403d4d0 0x7f8d7403f9e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8d70006fd0 tx=0x7f8d70006e40 comp rx=0 tx=0).stop 2024-04-25T10:22:58.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.910+0000 7f8d8ae43700 1 -- 172.21.15.43:0/3896697146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d8407f6e0 msgr2=0x7f8d841a0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:58.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.910+0000 7f8d8ae43700 1 --2- 172.21.15.43:0/3896697146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d8407f6e0 0x7f8d841a0210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009fb0 tx=0x7f8d78011940 comp rx=0 tx=0).stop 2024-04-25T10:22:58.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.910+0000 7f8d8ae43700 1 -- 172.21.15.43:0/3896697146 shutdown_connections 2024-04-25T10:22:58.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.910+0000 7f8d8ae43700 1 --2- 172.21.15.43:0/3896697146 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f8d7403d4d0 0x7f8d7403f9e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:58.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.910+0000 7f8d8ae43700 1 --2- 172.21.15.43:0/3896697146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d8407f6e0 0x7f8d841a0210 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:58.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.910+0000 7f8d8ae43700 1 -- 172.21.15.43:0/3896697146 >> 172.21.15.43:0/3896697146 conn(0x7f8d841052c0 msgr2=0x7f8d84106ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:58.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.910+0000 7f8d8ae43700 1 -- 172.21.15.43:0/3896697146 shutdown_connections 2024-04-25T10:22:58.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:57.910+0000 7f8d8ae43700 1 -- 172.21.15.43:0/3896697146 wait complete. 2024-04-25T10:22:59.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:58 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3896697146' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T10:22:59.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:58 smithi043 ceph-mon[29478]: mgrmap e5: x(active, since 6s) 2024-04-25T10:22:59.111 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:58 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:58.745+0000 7f0c07658200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:22:59.111 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:22:58 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:22:58.836+0000 7f0c07658200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:22:59.800 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-04-25T10:22:59.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T10:22:59.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T10:22:59.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T10:22:59.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T10:22:59.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-04-25T10:22:59.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.974+0000 7f412f5fc700 1 Processor -- start 2024-04-25T10:22:59.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.975+0000 7f412f5fc700 1 -- start start 2024-04-25T10:22:59.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.975+0000 7f412f5fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f412810a730 0x7f412810cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:59.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.975+0000 7f412f5fc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f412807ce40 con 0x7f412810a730 2024-04-25T10:22:59.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.975+0000 7f412d398700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f412810a730 0x7f412810cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:59.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.975+0000 7f412d398700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f412810a730 0x7f412810cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44712/0 (socket says 172.21.15.43:44712) 2024-04-25T10:22:59.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.975+0000 7f412d398700 1 -- 172.21.15.43:0/2577151006 learned_addr learned my addr 172.21.15.43:0/2577151006 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:59.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.975+0000 7f412d398700 1 -- 172.21.15.43:0/2577151006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412810d090 con 0x7f412810a730 2024-04-25T10:22:59.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.976+0000 7f412d398700 1 --2- 172.21.15.43:0/2577151006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f412810a730 0x7f412810cb50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f411c0098a0 tx=0x7f411c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15dc62d218465ec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:59.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.976+0000 7f411bfff700 1 -- 172.21.15.43:0/2577151006 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f411c004160 con 0x7f412810a730 2024-04-25T10:22:59.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.976+0000 7f411bfff700 1 -- 172.21.15.43:0/2577151006 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f411c004300 con 0x7f412810a730 2024-04-25T10:22:59.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.976+0000 7f411bfff700 1 -- 172.21.15.43:0/2577151006 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f411c011af0 con 0x7f412810a730 2024-04-25T10:22:59.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.977+0000 7f412f5fc700 1 -- 172.21.15.43:0/2577151006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f412810a730 msgr2=0x7f412810cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:59.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.977+0000 7f412f5fc700 1 --2- 172.21.15.43:0/2577151006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f412810a730 0x7f412810cb50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f411c0098a0 tx=0x7f411c009bb0 comp rx=0 tx=0).stop 2024-04-25T10:22:59.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.977+0000 7f412f5fc700 1 -- 172.21.15.43:0/2577151006 shutdown_connections 2024-04-25T10:22:59.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.977+0000 7f412f5fc700 1 --2- 172.21.15.43:0/2577151006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f412810a730 0x7f412810cb50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:59.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.977+0000 7f412f5fc700 1 -- 172.21.15.43:0/2577151006 >> 172.21.15.43:0/2577151006 conn(0x7f4128104170 msgr2=0x7f41281065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:59.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.977+0000 7f412f5fc700 1 -- 172.21.15.43:0/2577151006 shutdown_connections 2024-04-25T10:22:59.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.977+0000 7f412f5fc700 1 -- 172.21.15.43:0/2577151006 wait complete. 2024-04-25T10:22:59.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.978+0000 7f412f5fc700 1 Processor -- start 2024-04-25T10:22:59.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.978+0000 7f412f5fc700 1 -- start start 2024-04-25T10:22:59.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.978+0000 7f412f5fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f412810a730 0x7f4128109eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:59.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.978+0000 7f412f5fc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f412810dff0 con 0x7f412810a730 2024-04-25T10:22:59.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.978+0000 7f412d398700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f412810a730 0x7f4128109eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:59.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.978+0000 7f412d398700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f412810a730 0x7f4128109eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44718/0 (socket says 172.21.15.43:44718) 2024-04-25T10:22:59.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.978+0000 7f412d398700 1 -- 172.21.15.43:0/4013870681 learned_addr learned my addr 172.21.15.43:0/4013870681 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:22:59.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.978+0000 7f412d398700 1 -- 172.21.15.43:0/4013870681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412810a3f0 con 0x7f412810a730 2024-04-25T10:22:59.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.978+0000 7f412d398700 1 --2- 172.21.15.43:0/4013870681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f412810a730 0x7f4128109eb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f411c009fb0 tx=0x7f411c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:59.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.979+0000 7f411a7fc700 1 -- 172.21.15.43:0/4013870681 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f411c0036a0 con 0x7f412810a730 2024-04-25T10:22:59.805 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.979+0000 7f411a7fc700 1 -- 172.21.15.43:0/4013870681 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f411c01a430 con 0x7f412810a730 2024-04-25T10:22:59.805 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.979+0000 7f412f5fc700 1 -- 172.21.15.43:0/4013870681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4128108640 con 0x7f412810a730 2024-04-25T10:22:59.805 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.979+0000 7f411a7fc700 1 -- 172.21.15.43:0/4013870681 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f411c019550 con 0x7f412810a730 2024-04-25T10:22:59.805 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.979+0000 7f412f5fc700 1 -- 172.21.15.43:0/4013870681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4128108b20 con 0x7f412810a730 2024-04-25T10:22:59.805 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.980+0000 7f412f5fc700 1 -- 172.21.15.43:0/4013870681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4128064cf0 con 0x7f412810a730 2024-04-25T10:22:59.805 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.980+0000 7f411a7fc700 1 -- 172.21.15.43:0/4013870681 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7f411c01a8e0 con 0x7f412810a730 2024-04-25T10:22:59.805 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.981+0000 7f411a7fc700 1 --2- 172.21.15.43:0/4013870681 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f411403d520 0x7f411403fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:59.805 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.981+0000 7f411a7fc700 1 -- 172.21.15.43:0/4013870681 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f411c0529b0 con 0x7f412810a730 2024-04-25T10:22:59.805 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.981+0000 7f412cb97700 1 -- 172.21.15.43:0/4013870681 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f411403d520 msgr2=0x7f411403fa30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2568088261 2024-04-25T10:22:59.805 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.981+0000 7f412cb97700 1 --2- 172.21.15.43:0/4013870681 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f411403d520 0x7f411403fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:22:59.805 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:58.986+0000 7f411a7fc700 1 -- 172.21.15.43:0/4013870681 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f411c02c3a0 con 0x7f412810a730 2024-04-25T10:22:59.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:59.182+0000 7f412cb97700 1 -- 172.21.15.43:0/4013870681 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f411403d520 msgr2=0x7f411403fa30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2568088261 2024-04-25T10:22:59.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:59.182+0000 7f412cb97700 1 --2- 172.21.15.43:0/4013870681 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f411403d520 0x7f411403fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:22:59.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:59.286+0000 7f412f5fc700 1 -- 172.21.15.43:0/4013870681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f4128108dd0 con 0x7f412810a730 2024-04-25T10:22:59.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:59.286+0000 7f411a7fc700 1 -- 172.21.15.43:0/4013870681 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f411c00e260 con 0x7f412810a730 2024-04-25T10:22:59.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:59.290+0000 7f412f5fc700 1 -- 172.21.15.43:0/4013870681 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f411403d520 msgr2=0x7f411403fa30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:22:59.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:59.290+0000 7f412f5fc700 1 --2- 172.21.15.43:0/4013870681 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f411403d520 0x7f411403fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:59.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:59.290+0000 7f412f5fc700 1 -- 172.21.15.43:0/4013870681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f412810a730 msgr2=0x7f4128109eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:59.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:59.290+0000 7f412f5fc700 1 --2- 172.21.15.43:0/4013870681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f412810a730 0x7f4128109eb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f411c009fb0 tx=0x7f411c011940 comp rx=0 tx=0).stop 2024-04-25T10:22:59.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:59.291+0000 7f412f5fc700 1 -- 172.21.15.43:0/4013870681 shutdown_connections 2024-04-25T10:22:59.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:59.291+0000 7f412f5fc700 1 --2- 172.21.15.43:0/4013870681 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f411403d520 0x7f411403fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:59.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:59.291+0000 7f412f5fc700 1 --2- 172.21.15.43:0/4013870681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f412810a730 0x7f4128109eb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:59.807 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:59.291+0000 7f412f5fc700 1 -- 172.21.15.43:0/4013870681 >> 172.21.15.43:0/4013870681 conn(0x7f4128104170 msgr2=0x7f4128104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:59.807 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:59.291+0000 7f412f5fc700 1 -- 172.21.15.43:0/4013870681 shutdown_connections 2024-04-25T10:22:59.807 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:22:59.291+0000 7f412f5fc700 1 -- 172.21.15.43:0/4013870681 wait complete. 2024-04-25T10:22:59.807 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for the mgr to restart... 2024-04-25T10:22:59.807 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr epoch 5... 2024-04-25T10:23:00.220 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:22:59 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/4013870681' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T10:23:00.220 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:00 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:00.108+0000 7f0c07658200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:23:00.476 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:00 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:00.295+0000 7f0c07658200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:23:00.476 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:00 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:00.474+0000 7f0c07658200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:23:01.090 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:00 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:00.778+0000 7f0c07658200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:23:01.090 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:00 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:00.875+0000 7f0c07658200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:23:01.090 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:01 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:01.087+0000 7f0c07658200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:23:01.360 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:01 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:01.290+0000 7f0c07658200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:23:02.361 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:01 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:01.920+0000 7f0c07658200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:23:02.678 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:02 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:02.365+0000 7f0c07658200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:23:02.678 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:02 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:02.465+0000 7f0c07658200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:23:02.678 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:02 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:02.676+0000 7f0c07658200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:23:03.589 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:03 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:03.234+0000 7f0c07658200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:23:03.589 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:03 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:03.331+0000 7f0c07658200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:23:03.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:03 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:03.586+0000 7f0c07658200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:23:03.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:03 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:03.726+0000 7f0c07658200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:23:03.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:03 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:03.826+0000 7f0c07658200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:23:04.265 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:03 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:03.947+0000 7f0c07658200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:23:04.265 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:04 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:04.263+0000 7f0c07658200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:23:05.174 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:04 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:04.892+0000 7f0c07658200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:23:05.174 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:05 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:05.119+0000 7f0c07658200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:23:05.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:05 smithi043 ceph-mon[29478]: Active manager daemon x restarted 2024-04-25T10:23:05.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:05 smithi043 ceph-mon[29478]: Activating manager daemon x 2024-04-25T10:23:05.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:05 smithi043 ceph-mon[29478]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T10:23:05.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:05 smithi043 ceph-mon[29478]: mgrmap e6: x(active, starting, since 0.00367717s) 2024-04-25T10:23:05.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:05 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:23:05.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:05 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T10:23:05.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:05 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T10:23:05.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:05 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T10:23:05.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:05 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T10:23:05.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:05 smithi043 ceph-mon[29478]: Manager daemon x is now available 2024-04-25T10:23:05.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:05 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' 2024-04-25T10:23:05.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:05 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' 2024-04-25T10:23:05.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:05 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:23:05.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:05 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:23:06.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:06 smithi043 ceph-mon[29478]: Found migration_current of "None". Setting to last migration. 2024-04-25T10:23:06.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:06 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T10:23:06.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:06 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T10:23:06.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:06 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' 2024-04-25T10:23:06.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:06 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' 2024-04-25T10:23:06.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:06 smithi043 ceph-mon[29478]: mgrmap e7: x(active, since 1.0063s) 2024-04-25T10:23:06.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-04-25T10:23:06.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T10:23:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T10:23:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-04-25T10:23:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.247+0000 7f565e97f700 1 Processor -- start 2024-04-25T10:23:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.247+0000 7f565e97f700 1 -- start start 2024-04-25T10:23:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.248+0000 7f565e97f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f565810a6f0 0x7f565810cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.248+0000 7f565e97f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f565807d870 con 0x7f565810a6f0 2024-04-25T10:23:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.248+0000 7f5657fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f565810a6f0 0x7f565810cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.248+0000 7f5657fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f565810a6f0 0x7f565810cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44730/0 (socket says 172.21.15.43:44730) 2024-04-25T10:23:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.248+0000 7f5657fff700 1 -- 172.21.15.43:0/571172109 learned_addr learned my addr 172.21.15.43:0/571172109 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.248+0000 7f5657fff700 1 -- 172.21.15.43:0/571172109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565810d050 con 0x7f565810a6f0 2024-04-25T10:23:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.248+0000 7f5657fff700 1 --2- 172.21.15.43:0/571172109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f565810a6f0 0x7f565810cb10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f56480098a0 tx=0x7f5648009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beedf057f2c75279 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.249+0000 7f5656ffd700 1 -- 172.21.15.43:0/571172109 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5648004160 con 0x7f565810a6f0 2024-04-25T10:23:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.249+0000 7f5656ffd700 1 -- 172.21.15.43:0/571172109 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5648004300 con 0x7f565810a6f0 2024-04-25T10:23:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.249+0000 7f5656ffd700 1 -- 172.21.15.43:0/571172109 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5648011af0 con 0x7f565810a6f0 2024-04-25T10:23:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.249+0000 7f565e97f700 1 -- 172.21.15.43:0/571172109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f565810a6f0 msgr2=0x7f565810cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.249+0000 7f565e97f700 1 --2- 172.21.15.43:0/571172109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f565810a6f0 0x7f565810cb10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f56480098a0 tx=0x7f5648009bb0 comp rx=0 tx=0).stop 2024-04-25T10:23:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.249+0000 7f565e97f700 1 -- 172.21.15.43:0/571172109 shutdown_connections 2024-04-25T10:23:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.249+0000 7f565e97f700 1 --2- 172.21.15.43:0/571172109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f565810a6f0 0x7f565810cb10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.249+0000 7f565e97f700 1 -- 172.21.15.43:0/571172109 >> 172.21.15.43:0/571172109 conn(0x7f5658104110 msgr2=0x7f56581065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.250+0000 7f565e97f700 1 -- 172.21.15.43:0/571172109 shutdown_connections 2024-04-25T10:23:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.250+0000 7f565e97f700 1 -- 172.21.15.43:0/571172109 wait complete. 2024-04-25T10:23:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.250+0000 7f565e97f700 1 Processor -- start 2024-04-25T10:23:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.250+0000 7f565e97f700 1 -- start start 2024-04-25T10:23:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.251+0000 7f565e97f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f565810a6f0 0x7f5658109e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.251+0000 7f565e97f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f565810dfb0 con 0x7f565810a6f0 2024-04-25T10:23:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.251+0000 7f5657fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f565810a6f0 0x7f5658109e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.251+0000 7f5657fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f565810a6f0 0x7f5658109e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44744/0 (socket says 172.21.15.43:44744) 2024-04-25T10:23:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.251+0000 7f5657fff700 1 -- 172.21.15.43:0/1919224214 learned_addr learned my addr 172.21.15.43:0/1919224214 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.251+0000 7f5657fff700 1 -- 172.21.15.43:0/1919224214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565810a3b0 con 0x7f565810a6f0 2024-04-25T10:23:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.251+0000 7f5657fff700 1 --2- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f565810a6f0 0x7f5658109e70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f56480117b0 tx=0x7f5648011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.251+0000 7f56557fa700 1 -- 172.21.15.43:0/1919224214 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5648003af0 con 0x7f565810a6f0 2024-04-25T10:23:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.252+0000 7f56557fa700 1 -- 172.21.15.43:0/1919224214 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5648020070 con 0x7f565810a6f0 2024-04-25T10:23:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.252+0000 7f565e97f700 1 -- 172.21.15.43:0/1919224214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5658108600 con 0x7f565810a6f0 2024-04-25T10:23:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.252+0000 7f56557fa700 1 -- 172.21.15.43:0/1919224214 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5648019630 con 0x7f565810a6f0 2024-04-25T10:23:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.252+0000 7f565e97f700 1 -- 172.21.15.43:0/1919224214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5658108ae0 con 0x7f565810a6f0 2024-04-25T10:23:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.253+0000 7f56557fa700 1 -- 172.21.15.43:0/1919224214 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7f56480198f0 con 0x7f565810a6f0 2024-04-25T10:23:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.253+0000 7f56557fa700 1 --2- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f564003d570 0x7f564003fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.253+0000 7f56557fa700 1 -- 172.21.15.43:0/1919224214 --> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5640040190 con 0x7f564003d570 2024-04-25T10:23:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.253+0000 7f56557fa700 1 -- 172.21.15.43:0/1919224214 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5648052920 con 0x7f565810a6f0 2024-04-25T10:23:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.253+0000 7f56577fe700 1 -- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f564003d570 msgr2=0x7f564003fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2568088261 2024-04-25T10:23:06.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.253+0000 7f56577fe700 1 --2- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f564003d570 0x7f564003fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:23:06.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.454+0000 7f56577fe700 1 -- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f564003d570 msgr2=0x7f564003fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2568088261 2024-04-25T10:23:06.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.454+0000 7f56577fe700 1 --2- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f564003d570 0x7f564003fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:23:06.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.855+0000 7f56577fe700 1 -- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f564003d570 msgr2=0x7f564003fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2568088261 2024-04-25T10:23:06.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:00.855+0000 7f56577fe700 1 --2- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f564003d570 0x7f564003fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:23:06.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:01.656+0000 7f56577fe700 1 -- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f564003d570 msgr2=0x7f564003fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2568088261 2024-04-25T10:23:06.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:01.656+0000 7f56577fe700 1 --2- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f564003d570 0x7f564003fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:23:06.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:03.257+0000 7f56577fe700 1 -- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f564003d570 msgr2=0x7f564003fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2568088261 2024-04-25T10:23:06.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:03.257+0000 7f56577fe700 1 --2- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f564003d570 0x7f564003fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T10:23:06.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:05.122+0000 7f56557fa700 1 -- 172.21.15.43:0/1919224214 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mgrmap(e 6) ==== 49283+0+0 (secure 0 0 0) 0x7f5648019c30 con 0x7f565810a6f0 2024-04-25T10:23:06.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:05.123+0000 7f56557fa700 1 -- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f564003d570 msgr2=0x7f564003fa80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:23:06.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:05.123+0000 7f56557fa700 1 --2- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f564003d570 0x7f564003fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:06.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:06.124+0000 7f56557fa700 1 -- 172.21.15.43:0/1919224214 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 7) ==== 49410+0+0 (secure 0 0 0) 0x7f5648018760 con 0x7f565810a6f0 2024-04-25T10:23:06.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:06.124+0000 7f56557fa700 1 --2- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f5640040eb0 0x7f56400432d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:06.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:06.124+0000 7f56557fa700 1 -- 172.21.15.43:0/1919224214 --> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5640040190 con 0x7f5640040eb0 2024-04-25T10:23:06.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:06.124+0000 7f56577fe700 1 --2- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f5640040eb0 0x7f56400432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:06.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:06.125+0000 7f56577fe700 1 --2- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f5640040eb0 0x7f56400432d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f564c003a10 tx=0x7f564c0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:06.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:06.126+0000 7f56557fa700 1 -- 172.21.15.43:0/1919224214 <== mgr.14120 v2:172.21.15.43:6800/1017216348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f5640040190 con 0x7f5640040eb0 2024-04-25T10:23:06.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:06.134+0000 7f565e97f700 1 -- 172.21.15.43:0/1919224214 --> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f5658064cf0 con 0x7f5640040eb0 2024-04-25T10:23:06.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:06.134+0000 7f56557fa700 1 -- 172.21.15.43:0/1919224214 <== mgr.14120 v2:172.21.15.43:6800/1017216348 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f5658064cf0 con 0x7f5640040eb0 2024-04-25T10:23:06.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:06.135+0000 7f565e97f700 1 -- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f5640040eb0 msgr2=0x7f56400432d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:06.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:06.135+0000 7f565e97f700 1 --2- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f5640040eb0 0x7f56400432d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f564c003a10 tx=0x7f564c0092b0 comp rx=0 tx=0).stop 2024-04-25T10:23:06.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:06.135+0000 7f565e97f700 1 -- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f565810a6f0 msgr2=0x7f5658109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:06.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:06.135+0000 7f565e97f700 1 --2- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f565810a6f0 0x7f5658109e70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f56480117b0 tx=0x7f5648011b70 comp rx=0 tx=0).stop 2024-04-25T10:23:06.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:06.135+0000 7f565e97f700 1 -- 172.21.15.43:0/1919224214 shutdown_connections 2024-04-25T10:23:06.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:06.135+0000 7f565e97f700 1 --2- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f5640040eb0 0x7f56400432d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:06.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:06.135+0000 7f565e97f700 1 --2- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f565810a6f0 0x7f5658109e70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:06.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:06.135+0000 7f565e97f700 1 --2- 172.21.15.43:0/1919224214 >> [v2:172.21.15.43:6800/2568088261,v1:172.21.15.43:6801/2568088261] conn(0x7f564003d570 0x7f564003fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:06.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:06.135+0000 7f565e97f700 1 -- 172.21.15.43:0/1919224214 >> 172.21.15.43:0/1919224214 conn(0x7f5658104110 msgr2=0x7f5658104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:06.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:06.135+0000 7f565e97f700 1 -- 172.21.15.43:0/1919224214 shutdown_connections 2024-04-25T10:23:06.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:06.135+0000 7f565e97f700 1 -- 172.21.15.43:0/1919224214 wait complete. 2024-04-25T10:23:06.730 INFO:teuthology.orchestra.run.smithi043.stdout:mgr epoch 5 is available 2024-04-25T10:23:06.730 INFO:teuthology.orchestra.run.smithi043.stdout:Setting orchestrator backend to cephadm... 2024-04-25T10:23:07.334 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:07 smithi043 ceph-mon[29478]: [25/Apr/2024:10:23:06] ENGINE Bus STARTING 2024-04-25T10:23:07.335 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:07 smithi043 ceph-mon[29478]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T10:23:07.335 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:07 smithi043 ceph-mon[29478]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T10:23:07.335 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:07 smithi043 ceph-mon[29478]: [25/Apr/2024:10:23:06] ENGINE Serving on http://172.21.15.43:8765 2024-04-25T10:23:07.335 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:07 smithi043 ceph-mon[29478]: [25/Apr/2024:10:23:06] ENGINE Serving on https://172.21.15.43:7150 2024-04-25T10:23:07.335 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:07 smithi043 ceph-mon[29478]: [25/Apr/2024:10:23:06] ENGINE Bus STARTED 2024-04-25T10:23:07.335 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:07 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:23:07.903 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.095+0000 7f8f19c56700 1 Processor -- start 2024-04-25T10:23:07.903 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.095+0000 7f8f19c56700 1 -- start start 2024-04-25T10:23:07.903 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.095+0000 7f8f19c56700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f1410a710 0x7f8f1410cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:07.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.095+0000 7f8f19c56700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1407d6f0 con 0x7f8f1410a710 2024-04-25T10:23:07.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.095+0000 7f8f137fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f1410a710 0x7f8f1410cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:07.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.095+0000 7f8f137fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f1410a710 0x7f8f1410cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44832/0 (socket says 172.21.15.43:44832) 2024-04-25T10:23:07.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.095+0000 7f8f137fe700 1 -- 172.21.15.43:0/2632047053 learned_addr learned my addr 172.21.15.43:0/2632047053 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:07.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.096+0000 7f8f137fe700 1 -- 172.21.15.43:0/2632047053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1410d070 con 0x7f8f1410a710 2024-04-25T10:23:07.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.096+0000 7f8f137fe700 1 --2- 172.21.15.43:0/2632047053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f1410a710 0x7f8f1410cb30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8f040098a0 tx=0x7f8f04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb2fe56cde7d01b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:07.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.096+0000 7f8f127fc700 1 -- 172.21.15.43:0/2632047053 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f04004160 con 0x7f8f1410a710 2024-04-25T10:23:07.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.096+0000 7f8f127fc700 1 -- 172.21.15.43:0/2632047053 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8f04004300 con 0x7f8f1410a710 2024-04-25T10:23:07.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.096+0000 7f8f127fc700 1 -- 172.21.15.43:0/2632047053 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f04011af0 con 0x7f8f1410a710 2024-04-25T10:23:07.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.097+0000 7f8f19c56700 1 -- 172.21.15.43:0/2632047053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f1410a710 msgr2=0x7f8f1410cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:07.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.097+0000 7f8f19c56700 1 --2- 172.21.15.43:0/2632047053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f1410a710 0x7f8f1410cb30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8f040098a0 tx=0x7f8f04009bb0 comp rx=0 tx=0).stop 2024-04-25T10:23:07.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.097+0000 7f8f19c56700 1 -- 172.21.15.43:0/2632047053 shutdown_connections 2024-04-25T10:23:07.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.097+0000 7f8f19c56700 1 --2- 172.21.15.43:0/2632047053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f1410a710 0x7f8f1410cb30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:07.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.097+0000 7f8f19c56700 1 -- 172.21.15.43:0/2632047053 >> 172.21.15.43:0/2632047053 conn(0x7f8f14104130 msgr2=0x7f8f141065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:07.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.097+0000 7f8f19c56700 1 -- 172.21.15.43:0/2632047053 shutdown_connections 2024-04-25T10:23:07.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.097+0000 7f8f19c56700 1 -- 172.21.15.43:0/2632047053 wait complete. 2024-04-25T10:23:07.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.098+0000 7f8f19c56700 1 Processor -- start 2024-04-25T10:23:07.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.098+0000 7f8f19c56700 1 -- start start 2024-04-25T10:23:07.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.098+0000 7f8f19c56700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f1410a710 0x7f8f141a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:07.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.098+0000 7f8f19c56700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1410dfd0 con 0x7f8f1410a710 2024-04-25T10:23:07.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.098+0000 7f8f137fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f1410a710 0x7f8f141a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:07.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.098+0000 7f8f137fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f1410a710 0x7f8f141a02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44834/0 (socket says 172.21.15.43:44834) 2024-04-25T10:23:07.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.098+0000 7f8f137fe700 1 -- 172.21.15.43:0/3059879756 learned_addr learned my addr 172.21.15.43:0/3059879756 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:07.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.099+0000 7f8f137fe700 1 -- 172.21.15.43:0/3059879756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f141a0820 con 0x7f8f1410a710 2024-04-25T10:23:07.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.099+0000 7f8f137fe700 1 --2- 172.21.15.43:0/3059879756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f1410a710 0x7f8f141a02e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009fb0 tx=0x7f8f04011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:07.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.099+0000 7f8f10ff9700 1 -- 172.21.15.43:0/3059879756 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f04011bc0 con 0x7f8f1410a710 2024-04-25T10:23:07.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.099+0000 7f8f10ff9700 1 -- 172.21.15.43:0/3059879756 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8f04024440 con 0x7f8f1410a710 2024-04-25T10:23:07.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.099+0000 7f8f19c56700 1 -- 172.21.15.43:0/3059879756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f141a0ab0 con 0x7f8f1410a710 2024-04-25T10:23:07.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.099+0000 7f8f10ff9700 1 -- 172.21.15.43:0/3059879756 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f0401a5c0 con 0x7f8f1410a710 2024-04-25T10:23:07.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.099+0000 7f8f19c56700 1 -- 172.21.15.43:0/3059879756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f141a36c0 con 0x7f8f1410a710 2024-04-25T10:23:07.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.100+0000 7f8f10ff9700 1 -- 172.21.15.43:0/3059879756 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 7) ==== 49410+0+0 (secure 0 0 0) 0x7f8f0401a830 con 0x7f8f1410a710 2024-04-25T10:23:07.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.100+0000 7f8f10ff9700 1 --2- 172.21.15.43:0/3059879756 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f8efc03d460 0x7f8efc03f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:07.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.100+0000 7f8f10ff9700 1 -- 172.21.15.43:0/3059879756 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8f04053440 con 0x7f8f1410a710 2024-04-25T10:23:07.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.100+0000 7f8f12ffd700 1 --2- 172.21.15.43:0/3059879756 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f8efc03d460 0x7f8efc03f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:07.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.100+0000 7f8f19c56700 1 -- 172.21.15.43:0/3059879756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f14064cf0 con 0x7f8f1410a710 2024-04-25T10:23:07.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.101+0000 7f8f12ffd700 1 --2- 172.21.15.43:0/3059879756 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f8efc03d460 0x7f8efc03f970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8f08006fd0 tx=0x7f8f08006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:07.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.106+0000 7f8f10ff9700 1 -- 172.21.15.43:0/3059879756 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f0400b3e0 con 0x7f8f1410a710 2024-04-25T10:23:07.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.177+0000 7f8f10ff9700 1 -- 172.21.15.43:0/3059879756 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f8f040245f0 con 0x7f8f1410a710 2024-04-25T10:23:07.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.332+0000 7f8f19c56700 1 -- 172.21.15.43:0/3059879756 --> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f8f14063ad0 con 0x7f8efc03d460 2024-04-25T10:23:07.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.337+0000 7f8f10ff9700 1 -- 172.21.15.43:0/3059879756 <== mgr.14120 v2:172.21.15.43:6800/1017216348 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8f14063ad0 con 0x7f8efc03d460 2024-04-25T10:23:07.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.339+0000 7f8f19c56700 1 -- 172.21.15.43:0/3059879756 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f8efc03d460 msgr2=0x7f8efc03f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:07.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.339+0000 7f8f19c56700 1 --2- 172.21.15.43:0/3059879756 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f8efc03d460 0x7f8efc03f970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8f08006fd0 tx=0x7f8f08006e40 comp rx=0 tx=0).stop 2024-04-25T10:23:07.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.339+0000 7f8f19c56700 1 -- 172.21.15.43:0/3059879756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f1410a710 msgr2=0x7f8f141a02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:07.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.339+0000 7f8f19c56700 1 --2- 172.21.15.43:0/3059879756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f1410a710 0x7f8f141a02e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009fb0 tx=0x7f8f04011890 comp rx=0 tx=0).stop 2024-04-25T10:23:07.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.340+0000 7f8f19c56700 1 -- 172.21.15.43:0/3059879756 shutdown_connections 2024-04-25T10:23:07.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.340+0000 7f8f19c56700 1 --2- 172.21.15.43:0/3059879756 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f8efc03d460 0x7f8efc03f970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:07.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.340+0000 7f8f19c56700 1 --2- 172.21.15.43:0/3059879756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f1410a710 0x7f8f141a02e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:07.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.340+0000 7f8f19c56700 1 -- 172.21.15.43:0/3059879756 >> 172.21.15.43:0/3059879756 conn(0x7f8f14104130 msgr2=0x7f8f141086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:07.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.340+0000 7f8f19c56700 1 -- 172.21.15.43:0/3059879756 shutdown_connections 2024-04-25T10:23:07.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:07.340+0000 7f8f19c56700 1 -- 172.21.15.43:0/3059879756 wait complete. 2024-04-25T10:23:08.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:08 smithi043 ceph-mon[29478]: mgrmap e8: x(active, since 2s) 2024-04-25T10:23:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:08 smithi043 ceph-mon[29478]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:23:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:08 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' 2024-04-25T10:23:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:08 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:23:09.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T10:23:09.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.342+0000 7faf8d1be700 1 Processor -- start 2024-04-25T10:23:09.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.342+0000 7faf8d1be700 1 -- start start 2024-04-25T10:23:09.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.342+0000 7faf8d1be700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8810a6c0 0x7faf8810cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:09.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.342+0000 7faf8d1be700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faf8807d8d0 con 0x7faf8810a6c0 2024-04-25T10:23:09.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.342+0000 7faf86d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8810a6c0 0x7faf8810cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:09.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.342+0000 7faf86d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8810a6c0 0x7faf8810cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44844/0 (socket says 172.21.15.43:44844) 2024-04-25T10:23:09.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.342+0000 7faf86d9d700 1 -- 172.21.15.43:0/3091152251 learned_addr learned my addr 172.21.15.43:0/3091152251 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:09.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.343+0000 7faf86d9d700 1 -- 172.21.15.43:0/3091152251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf8810d020 con 0x7faf8810a6c0 2024-04-25T10:23:09.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.343+0000 7faf86d9d700 1 --2- 172.21.15.43:0/3091152251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8810a6c0 0x7faf8810cae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faf700098a0 tx=0x7faf70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d2fa0192eed218 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:09.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.343+0000 7faf85d9b700 1 -- 172.21.15.43:0/3091152251 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf70004160 con 0x7faf8810a6c0 2024-04-25T10:23:09.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.343+0000 7faf85d9b700 1 -- 172.21.15.43:0/3091152251 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faf70004300 con 0x7faf8810a6c0 2024-04-25T10:23:09.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.343+0000 7faf85d9b700 1 -- 172.21.15.43:0/3091152251 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf70011af0 con 0x7faf8810a6c0 2024-04-25T10:23:09.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.344+0000 7faf8d1be700 1 -- 172.21.15.43:0/3091152251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8810a6c0 msgr2=0x7faf8810cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:09.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.344+0000 7faf8d1be700 1 --2- 172.21.15.43:0/3091152251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8810a6c0 0x7faf8810cae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faf700098a0 tx=0x7faf70009bb0 comp rx=0 tx=0).stop 2024-04-25T10:23:09.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.344+0000 7faf8d1be700 1 -- 172.21.15.43:0/3091152251 shutdown_connections 2024-04-25T10:23:09.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.344+0000 7faf8d1be700 1 --2- 172.21.15.43:0/3091152251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8810a6c0 0x7faf8810cae0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:09.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.344+0000 7faf8d1be700 1 -- 172.21.15.43:0/3091152251 >> 172.21.15.43:0/3091152251 conn(0x7faf88104120 msgr2=0x7faf881065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:09.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.344+0000 7faf8d1be700 1 -- 172.21.15.43:0/3091152251 shutdown_connections 2024-04-25T10:23:09.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.344+0000 7faf8d1be700 1 -- 172.21.15.43:0/3091152251 wait complete. 2024-04-25T10:23:09.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.345+0000 7faf8d1be700 1 Processor -- start 2024-04-25T10:23:09.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.345+0000 7faf8d1be700 1 -- start start 2024-04-25T10:23:09.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.345+0000 7faf8d1be700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8810a6c0 0x7faf8819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:09.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.345+0000 7faf8d1be700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faf8810dfe0 con 0x7faf8810a6c0 2024-04-25T10:23:09.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.345+0000 7faf86d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8810a6c0 0x7faf8819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:09.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.345+0000 7faf86d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8810a6c0 0x7faf8819e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44850/0 (socket says 172.21.15.43:44850) 2024-04-25T10:23:09.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.345+0000 7faf86d9d700 1 -- 172.21.15.43:0/556410177 learned_addr learned my addr 172.21.15.43:0/556410177 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:09.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.346+0000 7faf86d9d700 1 -- 172.21.15.43:0/556410177 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf8819e550 con 0x7faf8810a6c0 2024-04-25T10:23:09.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.346+0000 7faf86d9d700 1 --2- 172.21.15.43:0/556410177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8810a6c0 0x7faf8819e010 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faf70009fb0 tx=0x7faf70011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:09.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.346+0000 7faf7ffff700 1 -- 172.21.15.43:0/556410177 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf700036a0 con 0x7faf8810a6c0 2024-04-25T10:23:09.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.346+0000 7faf8d1be700 1 -- 172.21.15.43:0/556410177 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf8819e7e0 con 0x7faf8810a6c0 2024-04-25T10:23:09.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.346+0000 7faf7ffff700 1 -- 172.21.15.43:0/556410177 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faf7001a430 con 0x7faf8810a6c0 2024-04-25T10:23:09.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.346+0000 7faf7ffff700 1 -- 172.21.15.43:0/556410177 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf70019550 con 0x7faf8810a6c0 2024-04-25T10:23:09.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.346+0000 7faf8d1be700 1 -- 172.21.15.43:0/556410177 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf881a13f0 con 0x7faf8810a6c0 2024-04-25T10:23:09.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.347+0000 7faf8d1be700 1 -- 172.21.15.43:0/556410177 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf88064cf0 con 0x7faf8810a6c0 2024-04-25T10:23:09.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.347+0000 7faf7ffff700 1 -- 172.21.15.43:0/556410177 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7faf70019850 con 0x7faf8810a6c0 2024-04-25T10:23:09.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.347+0000 7faf7ffff700 1 --2- 172.21.15.43:0/556410177 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7faf7403d520 0x7faf7403fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:09.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.347+0000 7faf7ffff700 1 -- 172.21.15.43:0/556410177 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faf70052700 con 0x7faf8810a6c0 2024-04-25T10:23:09.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.347+0000 7faf8659c700 1 --2- 172.21.15.43:0/556410177 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7faf7403d520 0x7faf7403fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:09.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.348+0000 7faf8659c700 1 --2- 172.21.15.43:0/556410177 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7faf7403d520 0x7faf7403fa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faf78006fd0 tx=0x7faf78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:09.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.353+0000 7faf7ffff700 1 -- 172.21.15.43:0/556410177 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf70017e90 con 0x7faf8810a6c0 2024-04-25T10:23:09.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.560+0000 7faf8d1be700 1 -- 172.21.15.43:0/556410177 --> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7faf88105250 con 0x7faf7403d520 2024-04-25T10:23:09.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.561+0000 7faf7ffff700 1 -- 172.21.15.43:0/556410177 <== mgr.14120 v2:172.21.15.43:6800/1017216348 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7faf88105250 con 0x7faf7403d520 2024-04-25T10:23:09.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.563+0000 7faf8d1be700 1 -- 172.21.15.43:0/556410177 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7faf7403d520 msgr2=0x7faf7403fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:09.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.563+0000 7faf8d1be700 1 --2- 172.21.15.43:0/556410177 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7faf7403d520 0x7faf7403fa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faf78006fd0 tx=0x7faf78006e40 comp rx=0 tx=0).stop 2024-04-25T10:23:09.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.563+0000 7faf8d1be700 1 -- 172.21.15.43:0/556410177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8810a6c0 msgr2=0x7faf8819e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:09.147 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.563+0000 7faf8d1be700 1 --2- 172.21.15.43:0/556410177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8810a6c0 0x7faf8819e010 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faf70009fb0 tx=0x7faf70011940 comp rx=0 tx=0).stop 2024-04-25T10:23:09.147 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.563+0000 7faf8d1be700 1 -- 172.21.15.43:0/556410177 shutdown_connections 2024-04-25T10:23:09.147 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.563+0000 7faf8d1be700 1 --2- 172.21.15.43:0/556410177 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7faf7403d520 0x7faf7403fa30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:09.147 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.563+0000 7faf8d1be700 1 --2- 172.21.15.43:0/556410177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8810a6c0 0x7faf8819e010 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:09.147 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.563+0000 7faf8d1be700 1 -- 172.21.15.43:0/556410177 >> 172.21.15.43:0/556410177 conn(0x7faf88104120 msgr2=0x7faf88104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:09.147 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.563+0000 7faf8d1be700 1 -- 172.21.15.43:0/556410177 shutdown_connections 2024-04-25T10:23:09.147 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:08.563+0000 7faf8d1be700 1 -- 172.21.15.43:0/556410177 wait complete. 2024-04-25T10:23:09.147 INFO:teuthology.orchestra.run.smithi043.stdout:Generating ssh key... 2024-04-25T10:23:09.389 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:09 smithi043 ceph-mon[29478]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:23:10.506 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:10 smithi043 ceph-mon[29478]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:23:10.506 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:10 smithi043 ceph-mon[29478]: Generating ssh key... 2024-04-25T10:23:10.507 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: Generating public/private rsa key pair. 2024-04-25T10:23:10.507 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: Your identification has been saved in /tmp/tmpw1a30t8d/key. 2024-04-25T10:23:10.507 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: Your public key has been saved in /tmp/tmpw1a30t8d/key.pub. 2024-04-25T10:23:10.507 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: The key fingerprint is: 2024-04-25T10:23:10.507 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: SHA256:cZFqqVWk7JsaFnrZL3/S8juDl4vIHYd10c0kYCgsXVc ceph-94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:23:10.507 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: The key's randomart image is: 2024-04-25T10:23:10.507 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: +---[RSA 3072]----+ 2024-04-25T10:23:10.507 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: | o o+o+oE .| 2024-04-25T10:23:10.507 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: | ..+.++ =.| 2024-04-25T10:23:10.507 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: | .+=. . +| 2024-04-25T10:23:10.507 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: | .=o . | 2024-04-25T10:23:10.507 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: | .+S . . | 2024-04-25T10:23:10.507 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: | ..+ o o . | 2024-04-25T10:23:10.507 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: | . = + oo.. | 2024-04-25T10:23:10.508 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: | o +.+++B | 2024-04-25T10:23:10.508 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: | . oo=*+= | 2024-04-25T10:23:10.508 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: +----[SHA256]-----+ 2024-04-25T10:23:10.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.609+0000 7f7352a93700 1 Processor -- start 2024-04-25T10:23:10.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.609+0000 7f7352a93700 1 -- start start 2024-04-25T10:23:10.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.609+0000 7f7352a93700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c10a690 0x7f734c10cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:10.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.609+0000 7f7352a93700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f734c07d8d0 con 0x7f734c10a690 2024-04-25T10:23:10.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.609+0000 7f735082f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c10a690 0x7f734c10cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:10.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.609+0000 7f735082f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c10a690 0x7f734c10cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60526/0 (socket says 172.21.15.43:60526) 2024-04-25T10:23:10.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.609+0000 7f735082f700 1 -- 172.21.15.43:0/351411154 learned_addr learned my addr 172.21.15.43:0/351411154 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:10.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.610+0000 7f735082f700 1 -- 172.21.15.43:0/351411154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734c10cff0 con 0x7f734c10a690 2024-04-25T10:23:10.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.610+0000 7f735082f700 1 --2- 172.21.15.43:0/351411154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c10a690 0x7f734c10cab0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f73400098a0 tx=0x7f7340009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92128f3e38acc4f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:10.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.610+0000 7f734b7fe700 1 -- 172.21.15.43:0/351411154 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7340004160 con 0x7f734c10a690 2024-04-25T10:23:10.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.610+0000 7f734b7fe700 1 -- 172.21.15.43:0/351411154 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7340004300 con 0x7f734c10a690 2024-04-25T10:23:10.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.610+0000 7f734b7fe700 1 -- 172.21.15.43:0/351411154 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7340011af0 con 0x7f734c10a690 2024-04-25T10:23:10.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.611+0000 7f7352a93700 1 -- 172.21.15.43:0/351411154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c10a690 msgr2=0x7f734c10cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:10.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.611+0000 7f7352a93700 1 --2- 172.21.15.43:0/351411154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c10a690 0x7f734c10cab0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f73400098a0 tx=0x7f7340009bb0 comp rx=0 tx=0).stop 2024-04-25T10:23:10.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.611+0000 7f7352a93700 1 -- 172.21.15.43:0/351411154 shutdown_connections 2024-04-25T10:23:10.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.611+0000 7f7352a93700 1 --2- 172.21.15.43:0/351411154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c10a690 0x7f734c10cab0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:10.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.611+0000 7f7352a93700 1 -- 172.21.15.43:0/351411154 >> 172.21.15.43:0/351411154 conn(0x7f734c104110 msgr2=0x7f734c1065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:10.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.611+0000 7f7352a93700 1 -- 172.21.15.43:0/351411154 shutdown_connections 2024-04-25T10:23:10.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.611+0000 7f7352a93700 1 -- 172.21.15.43:0/351411154 wait complete. 2024-04-25T10:23:10.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.612+0000 7f7352a93700 1 Processor -- start 2024-04-25T10:23:10.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.612+0000 7f7352a93700 1 -- start start 2024-04-25T10:23:10.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.612+0000 7f7352a93700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c10a690 0x7f734c109e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:10.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.612+0000 7f7352a93700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f734c10dfb0 con 0x7f734c10a690 2024-04-25T10:23:10.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.612+0000 7f735082f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c10a690 0x7f734c109e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:10.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.612+0000 7f735082f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c10a690 0x7f734c109e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60534/0 (socket says 172.21.15.43:60534) 2024-04-25T10:23:10.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.612+0000 7f735082f700 1 -- 172.21.15.43:0/994439181 learned_addr learned my addr 172.21.15.43:0/994439181 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:10.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.612+0000 7f735082f700 1 -- 172.21.15.43:0/994439181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734c10a350 con 0x7f734c10a690 2024-04-25T10:23:10.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.612+0000 7f735082f700 1 --2- 172.21.15.43:0/994439181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c10a690 0x7f734c109e10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7340009fb0 tx=0x7f7340011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:10.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.613+0000 7f7349ffb700 1 -- 172.21.15.43:0/994439181 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73400036a0 con 0x7f734c10a690 2024-04-25T10:23:10.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.613+0000 7f7349ffb700 1 -- 172.21.15.43:0/994439181 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f734001a430 con 0x7f734c10a690 2024-04-25T10:23:10.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.613+0000 7f7352a93700 1 -- 172.21.15.43:0/994439181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734c1085a0 con 0x7f734c10a690 2024-04-25T10:23:10.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.613+0000 7f7349ffb700 1 -- 172.21.15.43:0/994439181 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7340019550 con 0x7f734c10a690 2024-04-25T10:23:10.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.613+0000 7f7352a93700 1 -- 172.21.15.43:0/994439181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734c108a80 con 0x7f734c10a690 2024-04-25T10:23:10.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.614+0000 7f7349ffb700 1 -- 172.21.15.43:0/994439181 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f7340019800 con 0x7f734c10a690 2024-04-25T10:23:10.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.614+0000 7f7352a93700 1 -- 172.21.15.43:0/994439181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f734c064cf0 con 0x7f734c10a690 2024-04-25T10:23:10.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.614+0000 7f7349ffb700 1 --2- 172.21.15.43:0/994439181 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f733c03d570 0x7f733c03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:10.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.614+0000 7f7349ffb700 1 -- 172.21.15.43:0/994439181 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f73400526e0 con 0x7f734c10a690 2024-04-25T10:23:10.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.614+0000 7f734bfff700 1 --2- 172.21.15.43:0/994439181 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f733c03d570 0x7f733c03fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:10.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.615+0000 7f734bfff700 1 --2- 172.21.15.43:0/994439181 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f733c03d570 0x7f733c03fa80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7338006fd0 tx=0x7f7338006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:10.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.619+0000 7f7349ffb700 1 -- 172.21.15.43:0/994439181 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7340017e90 con 0x7f734c10a690 2024-04-25T10:23:10.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:09.828+0000 7f7352a93700 1 -- 172.21.15.43:0/994439181 --> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f734c109230 con 0x7f733c03d570 2024-04-25T10:23:10.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:10.209+0000 7f7349ffb700 1 -- 172.21.15.43:0/994439181 <== mgr.14120 v2:172.21.15.43:6800/1017216348 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f734c109230 con 0x7f733c03d570 2024-04-25T10:23:10.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:10.211+0000 7f7352a93700 1 -- 172.21.15.43:0/994439181 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f733c03d570 msgr2=0x7f733c03fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:10.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:10.211+0000 7f7352a93700 1 --2- 172.21.15.43:0/994439181 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f733c03d570 0x7f733c03fa80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7338006fd0 tx=0x7f7338006e40 comp rx=0 tx=0).stop 2024-04-25T10:23:10.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:10.212+0000 7f7352a93700 1 -- 172.21.15.43:0/994439181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c10a690 msgr2=0x7f734c109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:10.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:10.212+0000 7f7352a93700 1 --2- 172.21.15.43:0/994439181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c10a690 0x7f734c109e10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7340009fb0 tx=0x7f7340011940 comp rx=0 tx=0).stop 2024-04-25T10:23:10.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:10.212+0000 7f7352a93700 1 -- 172.21.15.43:0/994439181 shutdown_connections 2024-04-25T10:23:10.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:10.212+0000 7f7352a93700 1 --2- 172.21.15.43:0/994439181 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f733c03d570 0x7f733c03fa80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:10.740 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:10.212+0000 7f7352a93700 1 --2- 172.21.15.43:0/994439181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c10a690 0x7f734c109e10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:10.740 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:10.212+0000 7f7352a93700 1 -- 172.21.15.43:0/994439181 >> 172.21.15.43:0/994439181 conn(0x7f734c104110 msgr2=0x7f734c1049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:10.740 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:10.212+0000 7f7352a93700 1 -- 172.21.15.43:0/994439181 shutdown_connections 2024-04-25T10:23:10.740 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:10.212+0000 7f7352a93700 1 -- 172.21.15.43:0/994439181 wait complete. 2024-04-25T10:23:11.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:11 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' 2024-04-25T10:23:11.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:11 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' 2024-04-25T10:23:11.996 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp0FFj48MleeLmYi/P76pN3ALtbU1vznkMj8K1cW4pOuk79FXa9LX4ZUlKpHmncn6qhnprN14LC0VmxZn7sEagF1h3AB2LvjEedAOMe86nxdpn4D4Yos5VUD9qQSzcXBFdV4yxZ8R95F/6AEuzJOyCS8QBOEjMGQHb+LqEfEMDewK1puq2qAyaZ5mvNiNUn2OJD7FgYRmKv2WNxswFx0WIgUW4wud2uZ1lWvbCtzckPJhKHp5enH+UYBZUsOHnLrlOCawxn+4j9D40ohKOZSM3PiZgbgG44lPMqNttC/i/Q8HpSOEQJtEBy+cPWa6VOrN5WGVt+udqsUYqwD1GE8l1eJWW8TdRVMQyp9eJcPM2O+IofNuZJrJSdxxqqoQ2AWBmMRTFbpIyQTNwK3D0EUIWyHIHxeHvNhAe8ScCmt9by6zHTakOben6dRW3BUDaoDUDX/x+a209AGmDZ/xCSFX+pZ3eazXIGCOWVhKZrYrqNAAKVrvur9Ty2GRzeHz3ko8= ceph-94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:23:11.996 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.243+0000 7f07b28bc700 1 Processor -- start 2024-04-25T10:23:11.996 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.243+0000 7f07b28bc700 1 -- start start 2024-04-25T10:23:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.243+0000 7f07b28bc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ac108ac0 0x7f07ac108ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.243+0000 7f07b28bc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac1094c0 con 0x7f07ac108ac0 2024-04-25T10:23:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.244+0000 7f07abfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ac108ac0 0x7f07ac108ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.244+0000 7f07abfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ac108ac0 0x7f07ac108ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60550/0 (socket says 172.21.15.43:60550) 2024-04-25T10:23:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.244+0000 7f07abfff700 1 -- 172.21.15.43:0/3658799909 learned_addr learned my addr 172.21.15.43:0/3658799909 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.244+0000 7f07abfff700 1 -- 172.21.15.43:0/3658799909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ac109c90 con 0x7f07ac108ac0 2024-04-25T10:23:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.244+0000 7f07abfff700 1 --2- 172.21.15.43:0/3658799909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ac108ac0 0x7f07ac108ed0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f079c0098a0 tx=0x7f079c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0daf2c809893800 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.244+0000 7f07aaffd700 1 -- 172.21.15.43:0/3658799909 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f079c004160 con 0x7f07ac108ac0 2024-04-25T10:23:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.244+0000 7f07aaffd700 1 -- 172.21.15.43:0/3658799909 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f079c004300 con 0x7f07ac108ac0 2024-04-25T10:23:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.244+0000 7f07aaffd700 1 -- 172.21.15.43:0/3658799909 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f079c011af0 con 0x7f07ac108ac0 2024-04-25T10:23:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.245+0000 7f07b28bc700 1 -- 172.21.15.43:0/3658799909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ac108ac0 msgr2=0x7f07ac108ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.245+0000 7f07b28bc700 1 --2- 172.21.15.43:0/3658799909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ac108ac0 0x7f07ac108ed0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f079c0098a0 tx=0x7f079c009bb0 comp rx=0 tx=0).stop 2024-04-25T10:23:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.245+0000 7f07b28bc700 1 -- 172.21.15.43:0/3658799909 shutdown_connections 2024-04-25T10:23:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.245+0000 7f07b28bc700 1 --2- 172.21.15.43:0/3658799909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ac108ac0 0x7f07ac108ed0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.245+0000 7f07b28bc700 1 -- 172.21.15.43:0/3658799909 >> 172.21.15.43:0/3658799909 conn(0x7f07ac104120 msgr2=0x7f07ac1065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.245+0000 7f07b28bc700 1 -- 172.21.15.43:0/3658799909 shutdown_connections 2024-04-25T10:23:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.245+0000 7f07b28bc700 1 -- 172.21.15.43:0/3658799909 wait complete. 2024-04-25T10:23:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.246+0000 7f07b28bc700 1 Processor -- start 2024-04-25T10:23:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.246+0000 7f07b28bc700 1 -- start start 2024-04-25T10:23:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.246+0000 7f07b28bc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ac108ac0 0x7f07ac19e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.246+0000 7f07b28bc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac10d040 con 0x7f07ac108ac0 2024-04-25T10:23:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.246+0000 7f07abfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ac108ac0 0x7f07ac19e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.246+0000 7f07abfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ac108ac0 0x7f07ac19e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60556/0 (socket says 172.21.15.43:60556) 2024-04-25T10:23:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.247+0000 7f07abfff700 1 -- 172.21.15.43:0/3764589128 learned_addr learned my addr 172.21.15.43:0/3764589128 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.247+0000 7f07abfff700 1 -- 172.21.15.43:0/3764589128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ac19e5c0 con 0x7f07ac108ac0 2024-04-25T10:23:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.247+0000 7f07abfff700 1 --2- 172.21.15.43:0/3764589128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ac108ac0 0x7f07ac19e080 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f079c011890 tx=0x7f079c011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.247+0000 7f07a97fa700 1 -- 172.21.15.43:0/3764589128 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f079c0039e0 con 0x7f07ac108ac0 2024-04-25T10:23:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.247+0000 7f07b28bc700 1 -- 172.21.15.43:0/3764589128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07ac19e850 con 0x7f07ac108ac0 2024-04-25T10:23:11.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.247+0000 7f07a97fa700 1 -- 172.21.15.43:0/3764589128 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f079c003b80 con 0x7f07ac108ac0 2024-04-25T10:23:11.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.247+0000 7f07a97fa700 1 -- 172.21.15.43:0/3764589128 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f079c0246e0 con 0x7f07ac108ac0 2024-04-25T10:23:11.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.247+0000 7f07b28bc700 1 -- 172.21.15.43:0/3764589128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07ac1a1460 con 0x7f07ac108ac0 2024-04-25T10:23:11.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.248+0000 7f07a97fa700 1 -- 172.21.15.43:0/3764589128 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f079c022020 con 0x7f07ac108ac0 2024-04-25T10:23:11.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.248+0000 7f07b28bc700 1 -- 172.21.15.43:0/3764589128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07ac064cf0 con 0x7f07ac108ac0 2024-04-25T10:23:11.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.248+0000 7f07a97fa700 1 --2- 172.21.15.43:0/3764589128 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f079403d570 0x7f079403fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.248+0000 7f07a97fa700 1 -- 172.21.15.43:0/3764589128 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f079c052530 con 0x7f07ac108ac0 2024-04-25T10:23:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.249+0000 7f07ab7fe700 1 --2- 172.21.15.43:0/3764589128 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f079403d570 0x7f079403fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.249+0000 7f07ab7fe700 1 --2- 172.21.15.43:0/3764589128 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f079403d570 0x7f079403fa80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f07a0006fd0 tx=0x7f07a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.254+0000 7f07a97fa700 1 -- 172.21.15.43:0/3764589128 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f079c0094f0 con 0x7f07ac108ac0 2024-04-25T10:23:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.462+0000 7f07b28bc700 1 -- 172.21.15.43:0/3764589128 --> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f07ac02e880 con 0x7f079403d570 2024-04-25T10:23:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.462+0000 7f07a97fa700 1 -- 172.21.15.43:0/3764589128 <== mgr.14120 v2:172.21.15.43:6800/1017216348 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f07ac02e880 con 0x7f079403d570 2024-04-25T10:23:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.465+0000 7f07b28bc700 1 -- 172.21.15.43:0/3764589128 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f079403d570 msgr2=0x7f079403fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.465+0000 7f07b28bc700 1 --2- 172.21.15.43:0/3764589128 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f079403d570 0x7f079403fa80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f07a0006fd0 tx=0x7f07a0006e40 comp rx=0 tx=0).stop 2024-04-25T10:23:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.465+0000 7f07b28bc700 1 -- 172.21.15.43:0/3764589128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ac108ac0 msgr2=0x7f07ac19e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.465+0000 7f07b28bc700 1 --2- 172.21.15.43:0/3764589128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ac108ac0 0x7f07ac19e080 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f079c011890 tx=0x7f079c011970 comp rx=0 tx=0).stop 2024-04-25T10:23:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.465+0000 7f07b28bc700 1 -- 172.21.15.43:0/3764589128 shutdown_connections 2024-04-25T10:23:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.465+0000 7f07b28bc700 1 --2- 172.21.15.43:0/3764589128 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f079403d570 0x7f079403fa80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.465+0000 7f07b28bc700 1 --2- 172.21.15.43:0/3764589128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ac108ac0 0x7f07ac19e080 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.465+0000 7f07b28bc700 1 -- 172.21.15.43:0/3764589128 >> 172.21.15.43:0/3764589128 conn(0x7f07ac104120 msgr2=0x7f07ac194610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:12.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.465+0000 7f07b28bc700 1 -- 172.21.15.43:0/3764589128 shutdown_connections 2024-04-25T10:23:12.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:11.465+0000 7f07b28bc700 1 -- 172.21.15.43:0/3764589128 wait complete. 2024-04-25T10:23:12.001 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T10:23:12.001 INFO:teuthology.orchestra.run.smithi043.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T10:23:12.001 INFO:teuthology.orchestra.run.smithi043.stdout:Adding host smithi043... 2024-04-25T10:23:12.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:12 smithi043 ceph-mon[29478]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:23:13.334 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:13 smithi043 ceph-mon[29478]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi043", "addr": "172.21.15.43", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:23:14.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:14 smithi043 ceph-mon[29478]: Deploying cephadm binary to smithi043 2024-04-25T10:23:17.578 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Added host 'smithi043' with addr '172.21.15.43' 2024-04-25T10:23:17.578 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.485+0000 7f1ece7b6700 1 Processor -- start 2024-04-25T10:23:17.578 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.485+0000 7f1ece7b6700 1 -- start start 2024-04-25T10:23:17.578 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.486+0000 7f1ece7b6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ec8109bd0 0x7f1ec8109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:17.578 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.486+0000 7f1ece7b6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec810a5d0 con 0x7f1ec8109bd0 2024-04-25T10:23:17.578 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.486+0000 7f1ec7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ec8109bd0 0x7f1ec8109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:17.578 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.486+0000 7f1ec7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ec8109bd0 0x7f1ec8109fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60560/0 (socket says 172.21.15.43:60560) 2024-04-25T10:23:17.578 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.486+0000 7f1ec7fff700 1 -- 172.21.15.43:0/662021265 learned_addr learned my addr 172.21.15.43:0/662021265 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:17.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.486+0000 7f1ec7fff700 1 -- 172.21.15.43:0/662021265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec810ada0 con 0x7f1ec8109bd0 2024-04-25T10:23:17.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.486+0000 7f1ec7fff700 1 --2- 172.21.15.43:0/662021265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ec8109bd0 0x7f1ec8109fe0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1eb80098a0 tx=0x7f1eb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad8e48ebfaaaf4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:17.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.486+0000 7f1ec6ffd700 1 -- 172.21.15.43:0/662021265 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1eb8004160 con 0x7f1ec8109bd0 2024-04-25T10:23:17.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.486+0000 7f1ec6ffd700 1 -- 172.21.15.43:0/662021265 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1eb8004300 con 0x7f1ec8109bd0 2024-04-25T10:23:17.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.486+0000 7f1ec6ffd700 1 -- 172.21.15.43:0/662021265 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1eb8011af0 con 0x7f1ec8109bd0 2024-04-25T10:23:17.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.487+0000 7f1ece7b6700 1 -- 172.21.15.43:0/662021265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ec8109bd0 msgr2=0x7f1ec8109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:17.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.487+0000 7f1ece7b6700 1 --2- 172.21.15.43:0/662021265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ec8109bd0 0x7f1ec8109fe0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1eb80098a0 tx=0x7f1eb8009bb0 comp rx=0 tx=0).stop 2024-04-25T10:23:17.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.487+0000 7f1ece7b6700 1 -- 172.21.15.43:0/662021265 shutdown_connections 2024-04-25T10:23:17.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.487+0000 7f1ece7b6700 1 --2- 172.21.15.43:0/662021265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ec8109bd0 0x7f1ec8109fe0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:17.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.487+0000 7f1ece7b6700 1 -- 172.21.15.43:0/662021265 >> 172.21.15.43:0/662021265 conn(0x7f1ec8105290 msgr2=0x7f1ec81076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:17.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.487+0000 7f1ece7b6700 1 -- 172.21.15.43:0/662021265 shutdown_connections 2024-04-25T10:23:17.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.487+0000 7f1ece7b6700 1 -- 172.21.15.43:0/662021265 wait complete. 2024-04-25T10:23:17.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.488+0000 7f1ece7b6700 1 Processor -- start 2024-04-25T10:23:17.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.488+0000 7f1ece7b6700 1 -- start start 2024-04-25T10:23:17.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.488+0000 7f1ece7b6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ec8109bd0 0x7f1ec807f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:17.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.488+0000 7f1ece7b6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec810e150 con 0x7f1ec8109bd0 2024-04-25T10:23:17.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.488+0000 7f1ec7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ec8109bd0 0x7f1ec807f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:17.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.488+0000 7f1ec7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ec8109bd0 0x7f1ec807f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60564/0 (socket says 172.21.15.43:60564) 2024-04-25T10:23:17.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.489+0000 7f1ec7fff700 1 -- 172.21.15.43:0/4141701251 learned_addr learned my addr 172.21.15.43:0/4141701251 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:17.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.489+0000 7f1ec7fff700 1 -- 172.21.15.43:0/4141701251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec807fa00 con 0x7f1ec8109bd0 2024-04-25T10:23:17.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.489+0000 7f1ec7fff700 1 --2- 172.21.15.43:0/4141701251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ec8109bd0 0x7f1ec807f4c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009fb0 tx=0x7f1eb8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:17.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.489+0000 7f1ec57fa700 1 -- 172.21.15.43:0/4141701251 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1eb80036a0 con 0x7f1ec8109bd0 2024-04-25T10:23:17.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.489+0000 7f1ec57fa700 1 -- 172.21.15.43:0/4141701251 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1eb8003cd0 con 0x7f1ec8109bd0 2024-04-25T10:23:17.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.489+0000 7f1ece7b6700 1 -- 172.21.15.43:0/4141701251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec807dc50 con 0x7f1ec8109bd0 2024-04-25T10:23:17.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.489+0000 7f1ec57fa700 1 -- 172.21.15.43:0/4141701251 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1eb801a430 con 0x7f1ec8109bd0 2024-04-25T10:23:17.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.489+0000 7f1ece7b6700 1 -- 172.21.15.43:0/4141701251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec807e130 con 0x7f1ec8109bd0 2024-04-25T10:23:17.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.490+0000 7f1ece7b6700 1 -- 172.21.15.43:0/4141701251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec8064cf0 con 0x7f1ec8109bd0 2024-04-25T10:23:17.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.490+0000 7f1ec57fa700 1 -- 172.21.15.43:0/4141701251 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f1eb801a6a0 con 0x7f1ec8109bd0 2024-04-25T10:23:17.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.490+0000 7f1ec57fa700 1 --2- 172.21.15.43:0/4141701251 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f1eb003d570 0x7f1eb003fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:17.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.491+0000 7f1ec57fa700 1 -- 172.21.15.43:0/4141701251 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1eb8052490 con 0x7f1ec8109bd0 2024-04-25T10:23:17.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.491+0000 7f1ec77fe700 1 --2- 172.21.15.43:0/4141701251 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f1eb003d570 0x7f1eb003fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:17.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.491+0000 7f1ec77fe700 1 --2- 172.21.15.43:0/4141701251 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f1eb003d570 0x7f1eb003fa80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc006fd0 tx=0x7f1ebc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:17.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.496+0000 7f1ec57fa700 1 -- 172.21.15.43:0/4141701251 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1eb8017e90 con 0x7f1ec8109bd0 2024-04-25T10:23:17.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:12.712+0000 7f1ece7b6700 1 -- 172.21.15.43:0/4141701251 --> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi043", "addr": "172.21.15.43", "target": ["mon-mgr", ""]}) -- 0x7f1ec807e920 con 0x7f1eb003d570 2024-04-25T10:23:17.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:16.963+0000 7f1ec57fa700 1 -- 172.21.15.43:0/4141701251 <== mgr.14120 v2:172.21.15.43:6800/1017216348 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f1ec807e920 con 0x7f1eb003d570 2024-04-25T10:23:17.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:16.966+0000 7f1ece7b6700 1 -- 172.21.15.43:0/4141701251 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f1eb003d570 msgr2=0x7f1eb003fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:17.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:16.966+0000 7f1ece7b6700 1 --2- 172.21.15.43:0/4141701251 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f1eb003d570 0x7f1eb003fa80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc006fd0 tx=0x7f1ebc006e40 comp rx=0 tx=0).stop 2024-04-25T10:23:17.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:16.966+0000 7f1ece7b6700 1 -- 172.21.15.43:0/4141701251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ec8109bd0 msgr2=0x7f1ec807f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:17.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:16.966+0000 7f1ece7b6700 1 --2- 172.21.15.43:0/4141701251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ec8109bd0 0x7f1ec807f4c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009fb0 tx=0x7f1eb8011890 comp rx=0 tx=0).stop 2024-04-25T10:23:17.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:16.966+0000 7f1ece7b6700 1 -- 172.21.15.43:0/4141701251 shutdown_connections 2024-04-25T10:23:17.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:16.966+0000 7f1ece7b6700 1 --2- 172.21.15.43:0/4141701251 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f1eb003d570 0x7f1eb003fa80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:17.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:16.966+0000 7f1ece7b6700 1 --2- 172.21.15.43:0/4141701251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ec8109bd0 0x7f1ec807f4c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:17.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:16.966+0000 7f1ece7b6700 1 -- 172.21.15.43:0/4141701251 >> 172.21.15.43:0/4141701251 conn(0x7f1ec8105290 msgr2=0x7f1ec8106bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:17.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:16.966+0000 7f1ece7b6700 1 -- 172.21.15.43:0/4141701251 shutdown_connections 2024-04-25T10:23:17.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:16.966+0000 7f1ece7b6700 1 -- 172.21.15.43:0/4141701251 wait complete. 2024-04-25T10:23:17.582 INFO:teuthology.orchestra.run.smithi043.stdout:Deploying unmanaged mon service... 2024-04-25T10:23:18.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:17 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' 2024-04-25T10:23:18.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:17 smithi043 ceph-mon[29478]: Added host smithi043 2024-04-25T10:23:18.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:17 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:23:18.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T10:23:18.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.936+0000 7f158392e700 1 Processor -- start 2024-04-25T10:23:18.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.936+0000 7f158392e700 1 -- start start 2024-04-25T10:23:18.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.937+0000 7f158392e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f157c109bd0 0x7f157c109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:18.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.937+0000 7f158392e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f157c10a5d0 con 0x7f157c109bd0 2024-04-25T10:23:18.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.937+0000 7f15816ca700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f157c109bd0 0x7f157c109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:18.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.937+0000 7f15816ca700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f157c109bd0 0x7f157c109fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60570/0 (socket says 172.21.15.43:60570) 2024-04-25T10:23:18.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.937+0000 7f15816ca700 1 -- 172.21.15.43:0/2760419501 learned_addr learned my addr 172.21.15.43:0/2760419501 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:18.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.937+0000 7f15816ca700 1 -- 172.21.15.43:0/2760419501 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157c10ada0 con 0x7f157c109bd0 2024-04-25T10:23:18.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.937+0000 7f15816ca700 1 --2- 172.21.15.43:0/2760419501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f157c109bd0 0x7f157c109fe0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f156c0098a0 tx=0x7f156c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3cb05073c45c4d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:18.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.937+0000 7f1573fff700 1 -- 172.21.15.43:0/2760419501 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f156c004160 con 0x7f157c109bd0 2024-04-25T10:23:18.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.938+0000 7f1573fff700 1 -- 172.21.15.43:0/2760419501 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f156c004300 con 0x7f157c109bd0 2024-04-25T10:23:18.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.938+0000 7f1573fff700 1 -- 172.21.15.43:0/2760419501 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f156c011af0 con 0x7f157c109bd0 2024-04-25T10:23:18.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.938+0000 7f158392e700 1 -- 172.21.15.43:0/2760419501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f157c109bd0 msgr2=0x7f157c109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:18.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.938+0000 7f158392e700 1 --2- 172.21.15.43:0/2760419501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f157c109bd0 0x7f157c109fe0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f156c0098a0 tx=0x7f156c009bb0 comp rx=0 tx=0).stop 2024-04-25T10:23:18.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.938+0000 7f158392e700 1 -- 172.21.15.43:0/2760419501 shutdown_connections 2024-04-25T10:23:18.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.938+0000 7f158392e700 1 --2- 172.21.15.43:0/2760419501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f157c109bd0 0x7f157c109fe0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:18.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.938+0000 7f158392e700 1 -- 172.21.15.43:0/2760419501 >> 172.21.15.43:0/2760419501 conn(0x7f157c105290 msgr2=0x7f157c1076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:18.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.939+0000 7f158392e700 1 -- 172.21.15.43:0/2760419501 shutdown_connections 2024-04-25T10:23:18.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.939+0000 7f158392e700 1 -- 172.21.15.43:0/2760419501 wait complete. 2024-04-25T10:23:18.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.939+0000 7f158392e700 1 Processor -- start 2024-04-25T10:23:18.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.939+0000 7f158392e700 1 -- start start 2024-04-25T10:23:18.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.939+0000 7f158392e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f157c109bd0 0x7f157c19e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:18.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.939+0000 7f158392e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f157c10e150 con 0x7f157c109bd0 2024-04-25T10:23:18.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.940+0000 7f15816ca700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f157c109bd0 0x7f157c19e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:18.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.940+0000 7f15816ca700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f157c109bd0 0x7f157c19e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60578/0 (socket says 172.21.15.43:60578) 2024-04-25T10:23:18.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.940+0000 7f15816ca700 1 -- 172.21.15.43:0/1170554982 learned_addr learned my addr 172.21.15.43:0/1170554982 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:18.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.940+0000 7f15816ca700 1 -- 172.21.15.43:0/1170554982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157c19e5d0 con 0x7f157c109bd0 2024-04-25T10:23:18.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.940+0000 7f15816ca700 1 --2- 172.21.15.43:0/1170554982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f157c109bd0 0x7f157c19e090 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f156c009fb0 tx=0x7f156c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:18.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.940+0000 7f15727fc700 1 -- 172.21.15.43:0/1170554982 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f156c0036a0 con 0x7f157c109bd0 2024-04-25T10:23:18.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.940+0000 7f158392e700 1 -- 172.21.15.43:0/1170554982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f157c19e860 con 0x7f157c109bd0 2024-04-25T10:23:18.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.940+0000 7f15727fc700 1 -- 172.21.15.43:0/1170554982 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f156c003cd0 con 0x7f157c109bd0 2024-04-25T10:23:18.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.940+0000 7f15727fc700 1 -- 172.21.15.43:0/1170554982 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f156c01a430 con 0x7f157c109bd0 2024-04-25T10:23:18.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.940+0000 7f158392e700 1 -- 172.21.15.43:0/1170554982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f157c1a1470 con 0x7f157c109bd0 2024-04-25T10:23:18.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.941+0000 7f158392e700 1 -- 172.21.15.43:0/1170554982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f157c064cf0 con 0x7f157c109bd0 2024-04-25T10:23:18.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.942+0000 7f15727fc700 1 -- 172.21.15.43:0/1170554982 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f156c01a6f0 con 0x7f157c109bd0 2024-04-25T10:23:18.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.942+0000 7f15727fc700 1 --2- 172.21.15.43:0/1170554982 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f156803d520 0x7f156803fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:18.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.942+0000 7f15727fc700 1 -- 172.21.15.43:0/1170554982 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f156c052460 con 0x7f157c109bd0 2024-04-25T10:23:18.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.942+0000 7f1580ec9700 1 --2- 172.21.15.43:0/1170554982 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f156803d520 0x7f156803fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:18.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.943+0000 7f1580ec9700 1 --2- 172.21.15.43:0/1170554982 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f156803d520 0x7f156803fa30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1578006fd0 tx=0x7f1578006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:18.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:17.948+0000 7f15727fc700 1 -- 172.21.15.43:0/1170554982 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f156c017e90 con 0x7f157c109bd0 2024-04-25T10:23:18.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:18.190+0000 7f158392e700 1 -- 172.21.15.43:0/1170554982 --> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f157c107320 con 0x7f156803d520 2024-04-25T10:23:18.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:18.195+0000 7f15727fc700 1 -- 172.21.15.43:0/1170554982 <== mgr.14120 v2:172.21.15.43:6800/1017216348 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f157c107320 con 0x7f156803d520 2024-04-25T10:23:18.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:18.198+0000 7f158392e700 1 -- 172.21.15.43:0/1170554982 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f156803d520 msgr2=0x7f156803fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:18.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:18.198+0000 7f158392e700 1 --2- 172.21.15.43:0/1170554982 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f156803d520 0x7f156803fa30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1578006fd0 tx=0x7f1578006e40 comp rx=0 tx=0).stop 2024-04-25T10:23:18.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:18.198+0000 7f158392e700 1 -- 172.21.15.43:0/1170554982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f157c109bd0 msgr2=0x7f157c19e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:18.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:18.198+0000 7f158392e700 1 --2- 172.21.15.43:0/1170554982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f157c109bd0 0x7f157c19e090 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f156c009fb0 tx=0x7f156c011890 comp rx=0 tx=0).stop 2024-04-25T10:23:18.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:18.198+0000 7f158392e700 1 -- 172.21.15.43:0/1170554982 shutdown_connections 2024-04-25T10:23:18.883 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:18.198+0000 7f158392e700 1 --2- 172.21.15.43:0/1170554982 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f156803d520 0x7f156803fa30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:18.883 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:18.198+0000 7f158392e700 1 --2- 172.21.15.43:0/1170554982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f157c109bd0 0x7f157c19e090 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:18.883 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:18.198+0000 7f158392e700 1 -- 172.21.15.43:0/1170554982 >> 172.21.15.43:0/1170554982 conn(0x7f157c105290 msgr2=0x7f157c106bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:18.883 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:18.198+0000 7f158392e700 1 -- 172.21.15.43:0/1170554982 shutdown_connections 2024-04-25T10:23:18.883 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:18.198+0000 7f158392e700 1 -- 172.21.15.43:0/1170554982 wait complete. 2024-04-25T10:23:18.883 INFO:teuthology.orchestra.run.smithi043.stdout:Deploying unmanaged mgr service... 2024-04-25T10:23:19.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:19 smithi043 ceph-mon[29478]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:23:19.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:19 smithi043 ceph-mon[29478]: Saving service mon spec with placement count:5 2024-04-25T10:23:19.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:19 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' 2024-04-25T10:23:20.767 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:20 smithi043 ceph-mon[29478]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:23:20.767 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:20 smithi043 ceph-mon[29478]: Saving service mgr spec with placement count:2 2024-04-25T10:23:20.767 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:20 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' 2024-04-25T10:23:20.767 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:20 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' 2024-04-25T10:23:20.774 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T10:23:20.774 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.504+0000 7f5b8618b700 1 Processor -- start 2024-04-25T10:23:20.774 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.504+0000 7f5b8618b700 1 -- start start 2024-04-25T10:23:20.775 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.504+0000 7f5b8618b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b8010a6a0 0x7f5b8010cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:20.775 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.504+0000 7f5b8618b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8007cea0 con 0x7f5b8010a6a0 2024-04-25T10:23:20.775 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.504+0000 7f5b7f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b8010a6a0 0x7f5b8010cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:20.775 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.504+0000 7f5b7f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b8010a6a0 0x7f5b8010cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37050/0 (socket says 172.21.15.43:37050) 2024-04-25T10:23:20.775 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.504+0000 7f5b7f7fe700 1 -- 172.21.15.43:0/1217966290 learned_addr learned my addr 172.21.15.43:0/1217966290 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:20.775 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.505+0000 7f5b7f7fe700 1 -- 172.21.15.43:0/1217966290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8010d000 con 0x7f5b8010a6a0 2024-04-25T10:23:20.775 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.505+0000 7f5b7f7fe700 1 --2- 172.21.15.43:0/1217966290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b8010a6a0 0x7f5b8010cac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5b680098a0 tx=0x7f5b68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657e6ef309541b98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:20.775 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.505+0000 7f5b7e7fc700 1 -- 172.21.15.43:0/1217966290 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b68004160 con 0x7f5b8010a6a0 2024-04-25T10:23:20.775 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.505+0000 7f5b7e7fc700 1 -- 172.21.15.43:0/1217966290 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5b68004300 con 0x7f5b8010a6a0 2024-04-25T10:23:20.775 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.505+0000 7f5b7e7fc700 1 -- 172.21.15.43:0/1217966290 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b68011af0 con 0x7f5b8010a6a0 2024-04-25T10:23:20.775 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.506+0000 7f5b8618b700 1 -- 172.21.15.43:0/1217966290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b8010a6a0 msgr2=0x7f5b8010cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:20.775 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.506+0000 7f5b8618b700 1 --2- 172.21.15.43:0/1217966290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b8010a6a0 0x7f5b8010cac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5b680098a0 tx=0x7f5b68009bb0 comp rx=0 tx=0).stop 2024-04-25T10:23:20.776 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.506+0000 7f5b8618b700 1 -- 172.21.15.43:0/1217966290 shutdown_connections 2024-04-25T10:23:20.776 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.506+0000 7f5b8618b700 1 --2- 172.21.15.43:0/1217966290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b8010a6a0 0x7f5b8010cac0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:20.776 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.506+0000 7f5b8618b700 1 -- 172.21.15.43:0/1217966290 >> 172.21.15.43:0/1217966290 conn(0x7f5b80104120 msgr2=0x7f5b801065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:20.776 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.506+0000 7f5b8618b700 1 -- 172.21.15.43:0/1217966290 shutdown_connections 2024-04-25T10:23:20.776 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.506+0000 7f5b8618b700 1 -- 172.21.15.43:0/1217966290 wait complete. 2024-04-25T10:23:20.776 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.506+0000 7f5b8618b700 1 Processor -- start 2024-04-25T10:23:20.776 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.507+0000 7f5b8618b700 1 -- start start 2024-04-25T10:23:20.776 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.507+0000 7f5b8618b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b8010a6a0 0x7f5b8019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:20.776 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.507+0000 7f5b8618b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8010dfc0 con 0x7f5b8010a6a0 2024-04-25T10:23:20.776 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.507+0000 7f5b7f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b8010a6a0 0x7f5b8019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:20.776 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.507+0000 7f5b7f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b8010a6a0 0x7f5b8019dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37060/0 (socket says 172.21.15.43:37060) 2024-04-25T10:23:20.776 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.507+0000 7f5b7f7fe700 1 -- 172.21.15.43:0/333439975 learned_addr learned my addr 172.21.15.43:0/333439975 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:20.776 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.507+0000 7f5b7f7fe700 1 -- 172.21.15.43:0/333439975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8019e530 con 0x7f5b8010a6a0 2024-04-25T10:23:20.777 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.507+0000 7f5b7f7fe700 1 --2- 172.21.15.43:0/333439975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b8010a6a0 0x7f5b8019dff0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5b680117b0 tx=0x7f5b68011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:20.777 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.507+0000 7f5b7cff9700 1 -- 172.21.15.43:0/333439975 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b68003af0 con 0x7f5b8010a6a0 2024-04-25T10:23:20.777 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.508+0000 7f5b8618b700 1 -- 172.21.15.43:0/333439975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8019e7c0 con 0x7f5b8010a6a0 2024-04-25T10:23:20.777 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.508+0000 7f5b7cff9700 1 -- 172.21.15.43:0/333439975 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5b68020070 con 0x7f5b8010a6a0 2024-04-25T10:23:20.777 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.508+0000 7f5b7cff9700 1 -- 172.21.15.43:0/333439975 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b68019630 con 0x7f5b8010a6a0 2024-04-25T10:23:20.777 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.508+0000 7f5b8618b700 1 -- 172.21.15.43:0/333439975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b801a13d0 con 0x7f5b8010a6a0 2024-04-25T10:23:20.777 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.509+0000 7f5b7cff9700 1 -- 172.21.15.43:0/333439975 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f5b680198f0 con 0x7f5b8010a6a0 2024-04-25T10:23:20.777 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.509+0000 7f5b8618b700 1 -- 172.21.15.43:0/333439975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b80064cf0 con 0x7f5b8010a6a0 2024-04-25T10:23:20.777 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.509+0000 7f5b7cff9700 1 --2- 172.21.15.43:0/333439975 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f5b6c03d570 0x7f5b6c03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:20.777 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.509+0000 7f5b7cff9700 1 -- 172.21.15.43:0/333439975 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5b68052970 con 0x7f5b8010a6a0 2024-04-25T10:23:20.777 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.509+0000 7f5b7effd700 1 --2- 172.21.15.43:0/333439975 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f5b6c03d570 0x7f5b6c03fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:20.777 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.509+0000 7f5b7effd700 1 --2- 172.21.15.43:0/333439975 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f5b6c03d570 0x7f5b6c03fa80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5b70006fd0 tx=0x7f5b70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:20.777 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.514+0000 7f5b7cff9700 1 -- 172.21.15.43:0/333439975 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b68016260 con 0x7f5b8010a6a0 2024-04-25T10:23:20.777 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.739+0000 7f5b8618b700 1 -- 172.21.15.43:0/333439975 --> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f5b80105170 con 0x7f5b6c03d570 2024-04-25T10:23:20.778 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.744+0000 7f5b7cff9700 1 -- 172.21.15.43:0/333439975 <== mgr.14120 v2:172.21.15.43:6800/1017216348 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5b80105170 con 0x7f5b6c03d570 2024-04-25T10:23:20.778 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.746+0000 7f5b8618b700 1 -- 172.21.15.43:0/333439975 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f5b6c03d570 msgr2=0x7f5b6c03fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:20.778 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.746+0000 7f5b8618b700 1 --2- 172.21.15.43:0/333439975 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f5b6c03d570 0x7f5b6c03fa80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5b70006fd0 tx=0x7f5b70006e40 comp rx=0 tx=0).stop 2024-04-25T10:23:20.778 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.746+0000 7f5b8618b700 1 -- 172.21.15.43:0/333439975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b8010a6a0 msgr2=0x7f5b8019dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:20.778 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.746+0000 7f5b8618b700 1 --2- 172.21.15.43:0/333439975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b8010a6a0 0x7f5b8019dff0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5b680117b0 tx=0x7f5b68011b70 comp rx=0 tx=0).stop 2024-04-25T10:23:20.778 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.746+0000 7f5b8618b700 1 -- 172.21.15.43:0/333439975 shutdown_connections 2024-04-25T10:23:20.778 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.746+0000 7f5b8618b700 1 --2- 172.21.15.43:0/333439975 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f5b6c03d570 0x7f5b6c03fa80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:20.778 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.746+0000 7f5b8618b700 1 --2- 172.21.15.43:0/333439975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b8010a6a0 0x7f5b8019dff0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:20.778 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.746+0000 7f5b8618b700 1 -- 172.21.15.43:0/333439975 >> 172.21.15.43:0/333439975 conn(0x7f5b80104120 msgr2=0x7f5b80104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:20.778 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.747+0000 7f5b8618b700 1 -- 172.21.15.43:0/333439975 shutdown_connections 2024-04-25T10:23:20.778 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:19.747+0000 7f5b8618b700 1 -- 172.21.15.43:0/333439975 wait complete. 2024-04-25T10:23:22.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.199+0000 7f6f50f57700 1 Processor -- start 2024-04-25T10:23:22.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.199+0000 7f6f50f57700 1 -- start start 2024-04-25T10:23:22.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.200+0000 7f6f50f57700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c109bb0 0x7f6f4c109fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:22.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.200+0000 7f6f50f57700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c10a5b0 con 0x7f6f4c109bb0 2024-04-25T10:23:22.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.200+0000 7f6f4a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c109bb0 0x7f6f4c109fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:22.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.200+0000 7f6f4a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c109bb0 0x7f6f4c109fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37068/0 (socket says 172.21.15.43:37068) 2024-04-25T10:23:22.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.200+0000 7f6f4a59c700 1 -- 172.21.15.43:0/44859221 learned_addr learned my addr 172.21.15.43:0/44859221 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:22.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.200+0000 7f6f4a59c700 1 -- 172.21.15.43:0/44859221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c10ad80 con 0x7f6f4c109bb0 2024-04-25T10:23:22.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.200+0000 7f6f4a59c700 1 --2- 172.21.15.43:0/44859221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c109bb0 0x7f6f4c109fc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0098a0 tx=0x7f6f3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1b3e320eb54aaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:22.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.200+0000 7f6f4959a700 1 -- 172.21.15.43:0/44859221 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f3c004160 con 0x7f6f4c109bb0 2024-04-25T10:23:22.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.200+0000 7f6f4959a700 1 -- 172.21.15.43:0/44859221 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f3c004300 con 0x7f6f4c109bb0 2024-04-25T10:23:22.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.200+0000 7f6f4959a700 1 -- 172.21.15.43:0/44859221 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f3c011a20 con 0x7f6f4c109bb0 2024-04-25T10:23:22.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.201+0000 7f6f50f57700 1 -- 172.21.15.43:0/44859221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c109bb0 msgr2=0x7f6f4c109fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:22.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.201+0000 7f6f50f57700 1 --2- 172.21.15.43:0/44859221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c109bb0 0x7f6f4c109fc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0098a0 tx=0x7f6f3c009bb0 comp rx=0 tx=0).stop 2024-04-25T10:23:22.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.201+0000 7f6f50f57700 1 -- 172.21.15.43:0/44859221 shutdown_connections 2024-04-25T10:23:22.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.201+0000 7f6f50f57700 1 --2- 172.21.15.43:0/44859221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c109bb0 0x7f6f4c109fc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:22.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.201+0000 7f6f50f57700 1 -- 172.21.15.43:0/44859221 >> 172.21.15.43:0/44859221 conn(0x7f6f4c105230 msgr2=0x7f6f4c1076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:22.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.201+0000 7f6f50f57700 1 -- 172.21.15.43:0/44859221 shutdown_connections 2024-04-25T10:23:22.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.201+0000 7f6f50f57700 1 -- 172.21.15.43:0/44859221 wait complete. 2024-04-25T10:23:22.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.202+0000 7f6f50f57700 1 Processor -- start 2024-04-25T10:23:22.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.202+0000 7f6f50f57700 1 -- start start 2024-04-25T10:23:22.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.202+0000 7f6f50f57700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c109bb0 0x7f6f4c19e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:22.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.202+0000 7f6f50f57700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c10e130 con 0x7f6f4c109bb0 2024-04-25T10:23:22.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.202+0000 7f6f4a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c109bb0 0x7f6f4c19e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:22.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.203+0000 7f6f4a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c109bb0 0x7f6f4c19e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37074/0 (socket says 172.21.15.43:37074) 2024-04-25T10:23:22.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.203+0000 7f6f4a59c700 1 -- 172.21.15.43:0/1693465234 learned_addr learned my addr 172.21.15.43:0/1693465234 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:22.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.203+0000 7f6f4a59c700 1 -- 172.21.15.43:0/1693465234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c19e550 con 0x7f6f4c109bb0 2024-04-25T10:23:22.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.203+0000 7f6f4a59c700 1 --2- 172.21.15.43:0/1693465234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c109bb0 0x7f6f4c19e010 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c003e70 tx=0x7f6f3c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:22.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.203+0000 7f6f3b7fe700 1 -- 172.21.15.43:0/1693465234 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f3c003af0 con 0x7f6f4c109bb0 2024-04-25T10:23:22.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.203+0000 7f6f50f57700 1 -- 172.21.15.43:0/1693465234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4c19e7e0 con 0x7f6f4c109bb0 2024-04-25T10:23:22.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.203+0000 7f6f3b7fe700 1 -- 172.21.15.43:0/1693465234 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f3c01a8e0 con 0x7f6f4c109bb0 2024-04-25T10:23:22.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.203+0000 7f6f3b7fe700 1 -- 172.21.15.43:0/1693465234 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f3c024870 con 0x7f6f4c109bb0 2024-04-25T10:23:22.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.203+0000 7f6f50f57700 1 -- 172.21.15.43:0/1693465234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4c1a13f0 con 0x7f6f4c109bb0 2024-04-25T10:23:22.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.204+0000 7f6f3b7fe700 1 -- 172.21.15.43:0/1693465234 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f6f3c022020 con 0x7f6f4c109bb0 2024-04-25T10:23:22.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.204+0000 7f6f3b7fe700 1 --2- 172.21.15.43:0/1693465234 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f6f3403d520 0x7f6f3403fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:22.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.204+0000 7f6f3b7fe700 1 -- 172.21.15.43:0/1693465234 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6f3c053610 con 0x7f6f4c109bb0 2024-04-25T10:23:22.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.204+0000 7f6f49d9b700 1 --2- 172.21.15.43:0/1693465234 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f6f3403d520 0x7f6f3403fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:22.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.204+0000 7f6f50f57700 1 -- 172.21.15.43:0/1693465234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f4c064cf0 con 0x7f6f4c109bb0 2024-04-25T10:23:22.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.205+0000 7f6f49d9b700 1 --2- 172.21.15.43:0/1693465234 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f6f3403d520 0x7f6f3403fa30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6f40006fd0 tx=0x7f6f40006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:22.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.210+0000 7f6f3b7fe700 1 -- 172.21.15.43:0/1693465234 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f3c00e260 con 0x7f6f4c109bb0 2024-04-25T10:23:22.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.428+0000 7f6f50f57700 1 -- 172.21.15.43:0/1693465234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f6f4c1a19b0 con 0x7f6f4c109bb0 2024-04-25T10:23:22.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.432+0000 7f6f3b7fe700 1 -- 172.21.15.43:0/1693465234 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f6f3c01ad90 con 0x7f6f4c109bb0 2024-04-25T10:23:22.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.435+0000 7f6f50f57700 1 -- 172.21.15.43:0/1693465234 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f6f3403d520 msgr2=0x7f6f3403fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:22.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.435+0000 7f6f50f57700 1 --2- 172.21.15.43:0/1693465234 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f6f3403d520 0x7f6f3403fa30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6f40006fd0 tx=0x7f6f40006e40 comp rx=0 tx=0).stop 2024-04-25T10:23:22.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.435+0000 7f6f50f57700 1 -- 172.21.15.43:0/1693465234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c109bb0 msgr2=0x7f6f4c19e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:22.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.435+0000 7f6f50f57700 1 --2- 172.21.15.43:0/1693465234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c109bb0 0x7f6f4c19e010 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c003e70 tx=0x7f6f3c011aa0 comp rx=0 tx=0).stop 2024-04-25T10:23:22.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.435+0000 7f6f50f57700 1 -- 172.21.15.43:0/1693465234 shutdown_connections 2024-04-25T10:23:22.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.435+0000 7f6f50f57700 1 --2- 172.21.15.43:0/1693465234 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f6f3403d520 0x7f6f3403fa30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:22.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.435+0000 7f6f50f57700 1 --2- 172.21.15.43:0/1693465234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c109bb0 0x7f6f4c19e010 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:22.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.435+0000 7f6f50f57700 1 -- 172.21.15.43:0/1693465234 >> 172.21.15.43:0/1693465234 conn(0x7f6f4c105230 msgr2=0x7f6f4c105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:22.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.435+0000 7f6f50f57700 1 -- 172.21.15.43:0/1693465234 shutdown_connections 2024-04-25T10:23:22.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:21.435+0000 7f6f50f57700 1 -- 172.21.15.43:0/1693465234 wait complete. 2024-04-25T10:23:22.534 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:22 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' 2024-04-25T10:23:22.534 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:22 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1693465234' entity='client.admin' 2024-04-25T10:23:23.414 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.531+0000 7f7d068a5700 1 Processor -- start 2024-04-25T10:23:23.414 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.532+0000 7f7d068a5700 1 -- start start 2024-04-25T10:23:23.414 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.532+0000 7f7d068a5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d0007f6c0 0x7f7d0007daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:23.414 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.532+0000 7f7d068a5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0007e030 con 0x7f7d0007f6c0 2024-04-25T10:23:23.415 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.532+0000 7f7cfffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d0007f6c0 0x7f7d0007daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:23.415 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.532+0000 7f7cfffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d0007f6c0 0x7f7d0007daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37090/0 (socket says 172.21.15.43:37090) 2024-04-25T10:23:23.415 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.532+0000 7f7cfffff700 1 -- 172.21.15.43:0/3108814264 learned_addr learned my addr 172.21.15.43:0/3108814264 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:23.415 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.532+0000 7f7cfffff700 1 -- 172.21.15.43:0/3108814264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0007e1b0 con 0x7f7d0007f6c0 2024-04-25T10:23:23.415 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.533+0000 7f7cfffff700 1 --2- 172.21.15.43:0/3108814264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d0007f6c0 0x7f7d0007daf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00098a0 tx=0x7f7cf0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7deb42a1c945bef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:23.415 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.533+0000 7f7cfeffd700 1 -- 172.21.15.43:0/3108814264 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cf0004160 con 0x7f7d0007f6c0 2024-04-25T10:23:23.415 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.533+0000 7f7cfeffd700 1 -- 172.21.15.43:0/3108814264 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cf0004300 con 0x7f7d0007f6c0 2024-04-25T10:23:23.415 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.533+0000 7f7cfeffd700 1 -- 172.21.15.43:0/3108814264 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cf0011a20 con 0x7f7d0007f6c0 2024-04-25T10:23:23.415 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.533+0000 7f7d068a5700 1 -- 172.21.15.43:0/3108814264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d0007f6c0 msgr2=0x7f7d0007daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:23.415 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.533+0000 7f7d068a5700 1 --2- 172.21.15.43:0/3108814264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d0007f6c0 0x7f7d0007daf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00098a0 tx=0x7f7cf0009bb0 comp rx=0 tx=0).stop 2024-04-25T10:23:23.415 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.534+0000 7f7d068a5700 1 -- 172.21.15.43:0/3108814264 shutdown_connections 2024-04-25T10:23:23.415 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.534+0000 7f7d068a5700 1 --2- 172.21.15.43:0/3108814264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d0007f6c0 0x7f7d0007daf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:23.415 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.534+0000 7f7d068a5700 1 -- 172.21.15.43:0/3108814264 >> 172.21.15.43:0/3108814264 conn(0x7f7d00105280 msgr2=0x7f7d001076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:23.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.534+0000 7f7d068a5700 1 -- 172.21.15.43:0/3108814264 shutdown_connections 2024-04-25T10:23:23.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.534+0000 7f7d068a5700 1 -- 172.21.15.43:0/3108814264 wait complete. 2024-04-25T10:23:23.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.534+0000 7f7d068a5700 1 Processor -- start 2024-04-25T10:23:23.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.534+0000 7f7d068a5700 1 -- start start 2024-04-25T10:23:23.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.535+0000 7f7d068a5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d0007f6c0 0x7f7d001a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:23.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.535+0000 7f7d068a5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7d001101e0 con 0x7f7d0007f6c0 2024-04-25T10:23:23.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.535+0000 7f7cfffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d0007f6c0 0x7f7d001a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:23.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.535+0000 7f7cfffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d0007f6c0 0x7f7d001a0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37100/0 (socket says 172.21.15.43:37100) 2024-04-25T10:23:23.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.535+0000 7f7cfffff700 1 -- 172.21.15.43:0/703127211 learned_addr learned my addr 172.21.15.43:0/703127211 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:23.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.535+0000 7f7cfffff700 1 -- 172.21.15.43:0/703127211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d001a06c0 con 0x7f7d0007f6c0 2024-04-25T10:23:23.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.535+0000 7f7cfffff700 1 --2- 172.21.15.43:0/703127211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d0007f6c0 0x7f7d001a0180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00119f0 tx=0x7f7cf0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:23.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.535+0000 7f7cfd7fa700 1 -- 172.21.15.43:0/703127211 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cf00039e0 con 0x7f7d0007f6c0 2024-04-25T10:23:23.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.535+0000 7f7cfd7fa700 1 -- 172.21.15.43:0/703127211 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cf001b8f0 con 0x7f7d0007f6c0 2024-04-25T10:23:23.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.535+0000 7f7d068a5700 1 -- 172.21.15.43:0/703127211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d001a0950 con 0x7f7d0007f6c0 2024-04-25T10:23:23.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.536+0000 7f7cfd7fa700 1 -- 172.21.15.43:0/703127211 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cf00237a0 con 0x7f7d0007f6c0 2024-04-25T10:23:23.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.536+0000 7f7d068a5700 1 -- 172.21.15.43:0/703127211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d001a3560 con 0x7f7d0007f6c0 2024-04-25T10:23:23.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.537+0000 7f7cfd7fa700 1 -- 172.21.15.43:0/703127211 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f7cf002c3f0 con 0x7f7d0007f6c0 2024-04-25T10:23:23.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.537+0000 7f7cfd7fa700 1 --2- 172.21.15.43:0/703127211 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f7ce803d570 0x7f7ce803fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:23.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.537+0000 7f7d068a5700 1 -- 172.21.15.43:0/703127211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d00064cf0 con 0x7f7d0007f6c0 2024-04-25T10:23:23.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.537+0000 7f7cfd7fa700 1 -- 172.21.15.43:0/703127211 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7cf0052c00 con 0x7f7d0007f6c0 2024-04-25T10:23:23.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.537+0000 7f7cff7fe700 1 --2- 172.21.15.43:0/703127211 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f7ce803d570 0x7f7ce803fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:23.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.538+0000 7f7cff7fe700 1 --2- 172.21.15.43:0/703127211 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f7ce803d570 0x7f7ce803fa80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4006fd0 tx=0x7f7cf4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:23.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.543+0000 7f7cfd7fa700 1 -- 172.21.15.43:0/703127211 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7cf001f030 con 0x7f7d0007f6c0 2024-04-25T10:23:23.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.776+0000 7f7d068a5700 1 -- 172.21.15.43:0/703127211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f7d001a3850 con 0x7f7d0007f6c0 2024-04-25T10:23:23.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.779+0000 7f7cfd7fa700 1 -- 172.21.15.43:0/703127211 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f7cf001baa0 con 0x7f7d0007f6c0 2024-04-25T10:23:23.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.781+0000 7f7d068a5700 1 -- 172.21.15.43:0/703127211 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f7ce803d570 msgr2=0x7f7ce803fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:23.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.781+0000 7f7d068a5700 1 --2- 172.21.15.43:0/703127211 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f7ce803d570 0x7f7ce803fa80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4006fd0 tx=0x7f7cf4006e40 comp rx=0 tx=0).stop 2024-04-25T10:23:23.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.781+0000 7f7d068a5700 1 -- 172.21.15.43:0/703127211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d0007f6c0 msgr2=0x7f7d001a0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:23.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.781+0000 7f7d068a5700 1 --2- 172.21.15.43:0/703127211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d0007f6c0 0x7f7d001a0180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00119f0 tx=0x7f7cf0011ad0 comp rx=0 tx=0).stop 2024-04-25T10:23:23.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.781+0000 7f7d068a5700 1 -- 172.21.15.43:0/703127211 shutdown_connections 2024-04-25T10:23:23.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.781+0000 7f7d068a5700 1 --2- 172.21.15.43:0/703127211 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f7ce803d570 0x7f7ce803fa80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:23.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.781+0000 7f7d068a5700 1 --2- 172.21.15.43:0/703127211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d0007f6c0 0x7f7d001a0180 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:23.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.781+0000 7f7d068a5700 1 -- 172.21.15.43:0/703127211 >> 172.21.15.43:0/703127211 conn(0x7f7d00105280 msgr2=0x7f7d00106b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:23.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.781+0000 7f7d068a5700 1 -- 172.21.15.43:0/703127211 shutdown_connections 2024-04-25T10:23:23.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:22.781+0000 7f7d068a5700 1 -- 172.21.15.43:0/703127211 wait complete. 2024-04-25T10:23:23.419 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling the dashboard module... 2024-04-25T10:23:23.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:23 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/703127211' entity='client.admin' 2024-04-25T10:23:25.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:24 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/641589' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T10:23:25.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:24 smithi043 ceph-mon[29478]: from='mgr.14120 172.21.15.43:0/1907768388' entity='mgr.x' 2024-04-25T10:23:25.081 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:24 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: ignoring --setuser ceph since I am not root 2024-04-25T10:23:25.081 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:24 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: ignoring --setgroup ceph since I am not root 2024-04-25T10:23:25.081 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:24 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:24.995+0000 7ffb7d493200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:23:25.361 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:25 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:25.197+0000 7ffb7d493200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:23:25.361 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:25 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:25.305+0000 7ffb7d493200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:23:25.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.859+0000 7fd27ca10700 1 Processor -- start 2024-04-25T10:23:25.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.859+0000 7fd27ca10700 1 -- start start 2024-04-25T10:23:25.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.859+0000 7fd27ca10700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd278109bb0 0x7fd278109fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:25.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.859+0000 7fd27ca10700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd27810a5b0 con 0x7fd278109bb0 2024-04-25T10:23:25.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.859+0000 7fd27659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd278109bb0 0x7fd278109fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:25.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.859+0000 7fd27659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd278109bb0 0x7fd278109fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37112/0 (socket says 172.21.15.43:37112) 2024-04-25T10:23:25.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.859+0000 7fd27659c700 1 -- 172.21.15.43:0/3964062464 learned_addr learned my addr 172.21.15.43:0/3964062464 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:25.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.860+0000 7fd27659c700 1 -- 172.21.15.43:0/3964062464 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27810ad80 con 0x7fd278109bb0 2024-04-25T10:23:25.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.860+0000 7fd27659c700 1 --2- 172.21.15.43:0/3964062464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd278109bb0 0x7fd278109fc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd2680098a0 tx=0x7fd268009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86bae6e1162e35c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:25.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.860+0000 7fd27559a700 1 -- 172.21.15.43:0/3964062464 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd268004160 con 0x7fd278109bb0 2024-04-25T10:23:25.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.860+0000 7fd27559a700 1 -- 172.21.15.43:0/3964062464 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd268004300 con 0x7fd278109bb0 2024-04-25T10:23:25.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.860+0000 7fd27559a700 1 -- 172.21.15.43:0/3964062464 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd268011a20 con 0x7fd278109bb0 2024-04-25T10:23:25.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.861+0000 7fd27ca10700 1 -- 172.21.15.43:0/3964062464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd278109bb0 msgr2=0x7fd278109fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:25.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.861+0000 7fd27ca10700 1 --2- 172.21.15.43:0/3964062464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd278109bb0 0x7fd278109fc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd2680098a0 tx=0x7fd268009bb0 comp rx=0 tx=0).stop 2024-04-25T10:23:25.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.861+0000 7fd27ca10700 1 -- 172.21.15.43:0/3964062464 shutdown_connections 2024-04-25T10:23:25.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.861+0000 7fd27ca10700 1 --2- 172.21.15.43:0/3964062464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd278109bb0 0x7fd278109fc0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:25.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.861+0000 7fd27ca10700 1 -- 172.21.15.43:0/3964062464 >> 172.21.15.43:0/3964062464 conn(0x7fd278105230 msgr2=0x7fd2781076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:25.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.861+0000 7fd27ca10700 1 -- 172.21.15.43:0/3964062464 shutdown_connections 2024-04-25T10:23:25.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.861+0000 7fd27ca10700 1 -- 172.21.15.43:0/3964062464 wait complete. 2024-04-25T10:23:25.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.862+0000 7fd27ca10700 1 Processor -- start 2024-04-25T10:23:25.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.862+0000 7fd27ca10700 1 -- start start 2024-04-25T10:23:25.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.862+0000 7fd27ca10700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd278109bb0 0x7fd27807f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:25.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.862+0000 7fd27ca10700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd27810e130 con 0x7fd278109bb0 2024-04-25T10:23:25.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.862+0000 7fd27659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd278109bb0 0x7fd27807f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:25.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.862+0000 7fd27659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd278109bb0 0x7fd27807f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37114/0 (socket says 172.21.15.43:37114) 2024-04-25T10:23:25.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.862+0000 7fd27659c700 1 -- 172.21.15.43:0/641589 learned_addr learned my addr 172.21.15.43:0/641589 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:25.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.863+0000 7fd27659c700 1 -- 172.21.15.43:0/641589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27807fa00 con 0x7fd278109bb0 2024-04-25T10:23:25.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.863+0000 7fd27659c700 1 --2- 172.21.15.43:0/641589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd278109bb0 0x7fd27807f4c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd268003e70 tx=0x7fd268011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:25.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.863+0000 7fd2677fe700 1 -- 172.21.15.43:0/641589 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd268003aa0 con 0x7fd278109bb0 2024-04-25T10:23:25.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.863+0000 7fd2677fe700 1 -- 172.21.15.43:0/641589 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd26801a8e0 con 0x7fd278109bb0 2024-04-25T10:23:25.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.863+0000 7fd27ca10700 1 -- 172.21.15.43:0/641589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd27807dc50 con 0x7fd278109bb0 2024-04-25T10:23:25.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.863+0000 7fd2677fe700 1 -- 172.21.15.43:0/641589 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd268024870 con 0x7fd278109bb0 2024-04-25T10:23:25.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.863+0000 7fd27ca10700 1 -- 172.21.15.43:0/641589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd27807e130 con 0x7fd278109bb0 2024-04-25T10:23:25.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.864+0000 7fd27ca10700 1 -- 172.21.15.43:0/641589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd278064cf0 con 0x7fd278109bb0 2024-04-25T10:23:25.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.864+0000 7fd2677fe700 1 -- 172.21.15.43:0/641589 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fd268022020 con 0x7fd278109bb0 2024-04-25T10:23:25.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.864+0000 7fd2677fe700 1 --2- 172.21.15.43:0/641589 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7fd26003d570 0x7fd26003fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:25.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.864+0000 7fd2677fe700 1 -- 172.21.15.43:0/641589 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd2680526f0 con 0x7fd278109bb0 2024-04-25T10:23:25.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.864+0000 7fd275d9b700 1 --2- 172.21.15.43:0/641589 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7fd26003d570 0x7fd26003fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:25.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.865+0000 7fd275d9b700 1 --2- 172.21.15.43:0/641589 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7fd26003d570 0x7fd26003fa80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd26c006fd0 tx=0x7fd26c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:25.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:23.869+0000 7fd2677fe700 1 -- 172.21.15.43:0/641589 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd268029df0 con 0x7fd278109bb0 2024-04-25T10:23:25.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:24.140+0000 7fd27ca10700 1 -- 172.21.15.43:0/641589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fd27807e7f0 con 0x7fd278109bb0 2024-04-25T10:23:25.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:24.781+0000 7fd2677fe700 1 -- 172.21.15.43:0/641589 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fd26800b3e0 con 0x7fd278109bb0 2024-04-25T10:23:25.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:24.783+0000 7fd2677fe700 1 -- 172.21.15.43:0/641589 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7fd268054c70 con 0x7fd278109bb0 2024-04-25T10:23:25.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:24.784+0000 7fd27ca10700 1 -- 172.21.15.43:0/641589 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7fd26003d570 msgr2=0x7fd26003fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:25.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:24.784+0000 7fd27ca10700 1 --2- 172.21.15.43:0/641589 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7fd26003d570 0x7fd26003fa80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd26c006fd0 tx=0x7fd26c006e40 comp rx=0 tx=0).stop 2024-04-25T10:23:25.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:24.784+0000 7fd27ca10700 1 -- 172.21.15.43:0/641589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd278109bb0 msgr2=0x7fd27807f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:25.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:24.784+0000 7fd27ca10700 1 --2- 172.21.15.43:0/641589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd278109bb0 0x7fd27807f4c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd268003e70 tx=0x7fd268011aa0 comp rx=0 tx=0).stop 2024-04-25T10:23:25.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:24.784+0000 7fd27ca10700 1 -- 172.21.15.43:0/641589 shutdown_connections 2024-04-25T10:23:25.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:24.784+0000 7fd27ca10700 1 --2- 172.21.15.43:0/641589 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7fd26003d570 0x7fd26003fa80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:25.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:24.784+0000 7fd27ca10700 1 --2- 172.21.15.43:0/641589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd278109bb0 0x7fd27807f4c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:25.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:24.784+0000 7fd27ca10700 1 -- 172.21.15.43:0/641589 >> 172.21.15.43:0/641589 conn(0x7fd278105230 msgr2=0x7fd278105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:25.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:24.784+0000 7fd27ca10700 1 -- 172.21.15.43:0/641589 shutdown_connections 2024-04-25T10:23:25.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:24.784+0000 7fd27ca10700 1 -- 172.21.15.43:0/641589 wait complete. 2024-04-25T10:23:25.651 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:25 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:25.633+0000 7ffb7d493200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:23:26.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:25 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/641589' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T10:23:26.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:25 smithi043 ceph-mon[29478]: mgrmap e9: x(active, since 19s) 2024-04-25T10:23:26.069 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:25 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:25.727+0000 7ffb7d493200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:23:26.705 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-04-25T10:23:26.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T10:23:26.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T10:23:26.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T10:23:26.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T10:23:26.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-04-25T10:23:26.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.859+0000 7f9ca4589700 1 Processor -- start 2024-04-25T10:23:26.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.859+0000 7f9ca4589700 1 -- start start 2024-04-25T10:23:26.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.859+0000 7f9ca4589700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9c108510 0x7f9c9c108920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:26.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.859+0000 7f9ca4589700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c07ce20 con 0x7f9c9c108510 2024-04-25T10:23:26.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.860+0000 7f9ca2325700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9c108510 0x7f9c9c108920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:26.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.860+0000 7f9ca2325700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9c108510 0x7f9c9c108920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37136/0 (socket says 172.21.15.43:37136) 2024-04-25T10:23:26.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.860+0000 7f9ca2325700 1 -- 172.21.15.43:0/609911904 learned_addr learned my addr 172.21.15.43:0/609911904 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:26.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.860+0000 7f9ca2325700 1 -- 172.21.15.43:0/609911904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9c108e60 con 0x7f9c9c108510 2024-04-25T10:23:26.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.860+0000 7f9ca2325700 1 --2- 172.21.15.43:0/609911904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9c108510 0x7f9c9c108920 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9c980098a0 tx=0x7f9c98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383382ae1ac38d3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:26.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.860+0000 7f9ca1323700 1 -- 172.21.15.43:0/609911904 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c98004160 con 0x7f9c9c108510 2024-04-25T10:23:26.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.860+0000 7f9ca1323700 1 -- 172.21.15.43:0/609911904 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c98004300 con 0x7f9c9c108510 2024-04-25T10:23:26.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.860+0000 7f9ca1323700 1 -- 172.21.15.43:0/609911904 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c98011a20 con 0x7f9c9c108510 2024-04-25T10:23:26.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.861+0000 7f9ca4589700 1 -- 172.21.15.43:0/609911904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9c108510 msgr2=0x7f9c9c108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:26.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.861+0000 7f9ca4589700 1 --2- 172.21.15.43:0/609911904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9c108510 0x7f9c9c108920 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9c980098a0 tx=0x7f9c98009bb0 comp rx=0 tx=0).stop 2024-04-25T10:23:26.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.861+0000 7f9ca4589700 1 -- 172.21.15.43:0/609911904 shutdown_connections 2024-04-25T10:23:26.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.861+0000 7f9ca4589700 1 --2- 172.21.15.43:0/609911904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9c108510 0x7f9c9c108920 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:26.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.861+0000 7f9ca4589700 1 -- 172.21.15.43:0/609911904 >> 172.21.15.43:0/609911904 conn(0x7f9c9c104120 msgr2=0x7f9c9c1065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:26.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.861+0000 7f9ca4589700 1 -- 172.21.15.43:0/609911904 shutdown_connections 2024-04-25T10:23:26.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.862+0000 7f9ca4589700 1 -- 172.21.15.43:0/609911904 wait complete. 2024-04-25T10:23:26.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.862+0000 7f9ca4589700 1 Processor -- start 2024-04-25T10:23:26.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.862+0000 7f9ca4589700 1 -- start start 2024-04-25T10:23:26.708 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.862+0000 7f9ca4589700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9c108510 0x7f9c9c1a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:26.708 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.862+0000 7f9ca4589700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c109f20 con 0x7f9c9c108510 2024-04-25T10:23:26.708 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.863+0000 7f9ca2325700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9c108510 0x7f9c9c1a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:26.708 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.863+0000 7f9ca2325700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9c108510 0x7f9c9c1a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37150/0 (socket says 172.21.15.43:37150) 2024-04-25T10:23:26.708 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.863+0000 7f9ca2325700 1 -- 172.21.15.43:0/1503528130 learned_addr learned my addr 172.21.15.43:0/1503528130 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:26.708 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.863+0000 7f9ca2325700 1 -- 172.21.15.43:0/1503528130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9c1a06b0 con 0x7f9c9c108510 2024-04-25T10:23:26.708 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.863+0000 7f9ca2325700 1 --2- 172.21.15.43:0/1503528130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9c108510 0x7f9c9c1a0170 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9c980119f0 tx=0x7f9c98011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:26.708 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.863+0000 7f9c8f7fe700 1 -- 172.21.15.43:0/1503528130 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c980039e0 con 0x7f9c9c108510 2024-04-25T10:23:26.708 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.863+0000 7f9c8f7fe700 1 -- 172.21.15.43:0/1503528130 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c9801b8f0 con 0x7f9c9c108510 2024-04-25T10:23:26.708 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.863+0000 7f9ca4589700 1 -- 172.21.15.43:0/1503528130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9c1a0940 con 0x7f9c9c108510 2024-04-25T10:23:26.708 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.863+0000 7f9c8f7fe700 1 -- 172.21.15.43:0/1503528130 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c980237a0 con 0x7f9c9c108510 2024-04-25T10:23:26.708 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.863+0000 7f9ca4589700 1 -- 172.21.15.43:0/1503528130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c9c1a3550 con 0x7f9c9c108510 2024-04-25T10:23:26.708 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.864+0000 7f9c8f7fe700 1 -- 172.21.15.43:0/1503528130 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7f9c9802c3f0 con 0x7f9c9c108510 2024-04-25T10:23:26.708 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.864+0000 7f9ca4589700 1 -- 172.21.15.43:0/1503528130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c9c064cf0 con 0x7f9c9c108510 2024-04-25T10:23:26.709 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.865+0000 7f9c8f7fe700 1 --2- 172.21.15.43:0/1503528130 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f9c8803d5d0 0x7f9c8803fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:26.709 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.865+0000 7f9c8f7fe700 1 -- 172.21.15.43:0/1503528130 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9c98052c80 con 0x7f9c9c108510 2024-04-25T10:23:26.709 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.865+0000 7f9ca1b24700 1 -- 172.21.15.43:0/1503528130 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f9c8803d5d0 msgr2=0x7f9c8803fae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1017216348 2024-04-25T10:23:26.709 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.865+0000 7f9ca1b24700 1 --2- 172.21.15.43:0/1503528130 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f9c8803d5d0 0x7f9c8803fae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:23:26.709 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:25.874+0000 7f9c8f7fe700 1 -- 172.21.15.43:0/1503528130 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9c98059080 con 0x7f9c9c108510 2024-04-25T10:23:26.709 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:26.066+0000 7f9ca1b24700 1 -- 172.21.15.43:0/1503528130 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f9c8803d5d0 msgr2=0x7f9c8803fae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1017216348 2024-04-25T10:23:26.709 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:26.066+0000 7f9ca1b24700 1 --2- 172.21.15.43:0/1503528130 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f9c8803d5d0 0x7f9c8803fae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:23:26.709 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:26.160+0000 7f9ca4589700 1 -- 172.21.15.43:0/1503528130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f9c9c02e7d0 con 0x7f9c9c108510 2024-04-25T10:23:26.709 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:26.161+0000 7f9c8f7fe700 1 -- 172.21.15.43:0/1503528130 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f9c9801f030 con 0x7f9c9c108510 2024-04-25T10:23:26.709 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:26.164+0000 7f9ca4589700 1 -- 172.21.15.43:0/1503528130 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f9c8803d5d0 msgr2=0x7f9c8803fae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:23:26.710 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:26.164+0000 7f9ca4589700 1 --2- 172.21.15.43:0/1503528130 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f9c8803d5d0 0x7f9c8803fae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:26.710 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:26.164+0000 7f9ca4589700 1 -- 172.21.15.43:0/1503528130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9c108510 msgr2=0x7f9c9c1a0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:26.710 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:26.164+0000 7f9ca4589700 1 --2- 172.21.15.43:0/1503528130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9c108510 0x7f9c9c1a0170 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9c980119f0 tx=0x7f9c98011ad0 comp rx=0 tx=0).stop 2024-04-25T10:23:26.710 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:26.164+0000 7f9ca4589700 1 -- 172.21.15.43:0/1503528130 shutdown_connections 2024-04-25T10:23:26.710 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:26.164+0000 7f9ca4589700 1 --2- 172.21.15.43:0/1503528130 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f9c8803d5d0 0x7f9c8803fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:26.710 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:26.164+0000 7f9ca4589700 1 --2- 172.21.15.43:0/1503528130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9c108510 0x7f9c9c1a0170 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:26.710 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:26.165+0000 7f9ca4589700 1 -- 172.21.15.43:0/1503528130 >> 172.21.15.43:0/1503528130 conn(0x7f9c9c104120 msgr2=0x7f9c9c194140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:26.710 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:26.165+0000 7f9ca4589700 1 -- 172.21.15.43:0/1503528130 shutdown_connections 2024-04-25T10:23:26.710 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:26.165+0000 7f9ca4589700 1 -- 172.21.15.43:0/1503528130 wait complete. 2024-04-25T10:23:26.710 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for the mgr to restart... 2024-04-25T10:23:26.710 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr epoch 9... 2024-04-25T10:23:27.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:26 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1503528130' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T10:23:27.314 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:27 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:27.050+0000 7ffb7d493200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:23:27.314 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:27 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:27.236+0000 7ffb7d493200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:23:27.610 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:27 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:27.416+0000 7ffb7d493200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:23:28.034 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:27 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:27.723+0000 7ffb7d493200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:23:28.034 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:27 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:27.821+0000 7ffb7d493200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:23:28.034 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:28 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:28.032+0000 7ffb7d493200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:23:28.360 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:28 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:28.235+0000 7ffb7d493200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:23:29.319 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:28 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:28.865+0000 7ffb7d493200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:23:29.610 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:29 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:29.316+0000 7ffb7d493200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:23:29.611 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:29 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:29.418+0000 7ffb7d493200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:23:30.110 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:29 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:29.632+0000 7ffb7d493200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:23:30.536 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:30 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:30.179+0000 7ffb7d493200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:23:30.536 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:30 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:30.276+0000 7ffb7d493200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:23:30.860 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:30 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:30.533+0000 7ffb7d493200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:23:30.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:30 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:30.674+0000 7ffb7d493200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:23:30.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:30 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:30.774+0000 7ffb7d493200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:23:31.213 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:30 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:30.896+0000 7ffb7d493200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:23:31.610 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:31 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:31.211+0000 7ffb7d493200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:23:32.111 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:31 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:31.843+0000 7ffb7d493200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:23:32.111 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:23:32 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:23:32.069+0000 7ffb7d493200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:23:32.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:32 smithi043 ceph-mon[29478]: Active manager daemon x restarted 2024-04-25T10:23:32.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:32 smithi043 ceph-mon[29478]: Activating manager daemon x 2024-04-25T10:23:32.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:32 smithi043 ceph-mon[29478]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T10:23:32.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:32 smithi043 ceph-mon[29478]: mgrmap e10: x(active, starting, since 0.00362514s) 2024-04-25T10:23:32.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:23:32.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T10:23:32.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T10:23:32.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T10:23:32.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T10:23:32.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:32 smithi043 ceph-mon[29478]: Manager daemon x is now available 2024-04-25T10:23:32.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:23:33.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T10:23:33.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T10:23:33.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:33 smithi043 ceph-mon[29478]: mgrmap e11: x(active, since 1.0067s) 2024-04-25T10:23:33.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-04-25T10:23:33.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T10:23:33.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T10:23:33.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-04-25T10:23:33.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.104+0000 7f1ef48e3700 1 Processor -- start 2024-04-25T10:23:33.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.105+0000 7f1ef48e3700 1 -- start start 2024-04-25T10:23:33.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.105+0000 7f1ef48e3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ef0108510 0x7f1ef0108920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:33.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.105+0000 7f1ef48e3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef007ce20 con 0x7f1ef0108510 2024-04-25T10:23:33.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.105+0000 7f1eee59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ef0108510 0x7f1ef0108920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:33.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.105+0000 7f1eee59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ef0108510 0x7f1ef0108920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37164/0 (socket says 172.21.15.43:37164) 2024-04-25T10:23:33.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.105+0000 7f1eee59c700 1 -- 172.21.15.43:0/2639367975 learned_addr learned my addr 172.21.15.43:0/2639367975 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:33.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.106+0000 7f1eee59c700 1 -- 172.21.15.43:0/2639367975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef0108e60 con 0x7f1ef0108510 2024-04-25T10:23:33.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.106+0000 7f1eee59c700 1 --2- 172.21.15.43:0/2639367975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ef0108510 0x7f1ef0108920 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00098a0 tx=0x7f1ee0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7baf37a6ea225263 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:33.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.106+0000 7f1eed59a700 1 -- 172.21.15.43:0/2639367975 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ee0004160 con 0x7f1ef0108510 2024-04-25T10:23:33.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.106+0000 7f1eed59a700 1 -- 172.21.15.43:0/2639367975 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ee0004300 con 0x7f1ef0108510 2024-04-25T10:23:33.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.106+0000 7f1eed59a700 1 -- 172.21.15.43:0/2639367975 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ee0011a20 con 0x7f1ef0108510 2024-04-25T10:23:33.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.107+0000 7f1ef48e3700 1 -- 172.21.15.43:0/2639367975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ef0108510 msgr2=0x7f1ef0108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:33.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.107+0000 7f1ef48e3700 1 --2- 172.21.15.43:0/2639367975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ef0108510 0x7f1ef0108920 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00098a0 tx=0x7f1ee0009bb0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.107+0000 7f1ef48e3700 1 -- 172.21.15.43:0/2639367975 shutdown_connections 2024-04-25T10:23:34.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.107+0000 7f1ef48e3700 1 --2- 172.21.15.43:0/2639367975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ef0108510 0x7f1ef0108920 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.107+0000 7f1ef48e3700 1 -- 172.21.15.43:0/2639367975 >> 172.21.15.43:0/2639367975 conn(0x7f1ef0104120 msgr2=0x7f1ef01065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:34.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.107+0000 7f1ef48e3700 1 -- 172.21.15.43:0/2639367975 shutdown_connections 2024-04-25T10:23:34.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.107+0000 7f1ef48e3700 1 -- 172.21.15.43:0/2639367975 wait complete. 2024-04-25T10:23:34.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.108+0000 7f1ef48e3700 1 Processor -- start 2024-04-25T10:23:34.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.108+0000 7f1ef48e3700 1 -- start start 2024-04-25T10:23:34.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.108+0000 7f1ef48e3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ef0108510 0x7f1ef01a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.108+0000 7f1ef48e3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef0109f20 con 0x7f1ef0108510 2024-04-25T10:23:34.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.108+0000 7f1eee59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ef0108510 0x7f1ef01a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.109+0000 7f1eee59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ef0108510 0x7f1ef01a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37176/0 (socket says 172.21.15.43:37176) 2024-04-25T10:23:34.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.109+0000 7f1eee59c700 1 -- 172.21.15.43:0/1304586592 learned_addr learned my addr 172.21.15.43:0/1304586592 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:34.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.109+0000 7f1eee59c700 1 -- 172.21.15.43:0/1304586592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef01a06b0 con 0x7f1ef0108510 2024-04-25T10:23:34.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.109+0000 7f1eee59c700 1 --2- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ef0108510 0x7f1ef01a0170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0000c00 tx=0x7f1ee0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:34.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.109+0000 7f1edf7fe700 1 -- 172.21.15.43:0/1304586592 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ee0003aa0 con 0x7f1ef0108510 2024-04-25T10:23:34.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.109+0000 7f1ef48e3700 1 -- 172.21.15.43:0/1304586592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef01a0940 con 0x7f1ef0108510 2024-04-25T10:23:34.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.109+0000 7f1ef48e3700 1 -- 172.21.15.43:0/1304586592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef01a3550 con 0x7f1ef0108510 2024-04-25T10:23:34.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.109+0000 7f1edf7fe700 1 -- 172.21.15.43:0/1304586592 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ee001b8f0 con 0x7f1ef0108510 2024-04-25T10:23:34.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.109+0000 7f1edf7fe700 1 -- 172.21.15.43:0/1304586592 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ee0023890 con 0x7f1ef0108510 2024-04-25T10:23:34.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.110+0000 7f1edf7fe700 1 -- 172.21.15.43:0/1304586592 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7f1ee002c3f0 con 0x7f1ef0108510 2024-04-25T10:23:34.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.111+0000 7f1edf7fe700 1 --2- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f1ed803d620 0x7f1ed803fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.111+0000 7f1edf7fe700 1 -- 172.21.15.43:0/1304586592 --> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1ed8040240 con 0x7f1ed803d620 2024-04-25T10:23:34.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.111+0000 7f1edf7fe700 1 -- 172.21.15.43:0/1304586592 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1ee0053c00 con 0x7f1ef0108510 2024-04-25T10:23:34.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.111+0000 7f1eedd9b700 1 -- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f1ed803d620 msgr2=0x7f1ed803fb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1017216348 2024-04-25T10:23:34.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.111+0000 7f1eedd9b700 1 --2- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f1ed803d620 0x7f1ed803fb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:23:34.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.312+0000 7f1eedd9b700 1 -- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f1ed803d620 msgr2=0x7f1ed803fb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1017216348 2024-04-25T10:23:34.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.312+0000 7f1eedd9b700 1 --2- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f1ed803d620 0x7f1ed803fb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:23:34.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.713+0000 7f1eedd9b700 1 -- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f1ed803d620 msgr2=0x7f1ed803fb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1017216348 2024-04-25T10:23:34.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:27.713+0000 7f1eedd9b700 1 --2- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f1ed803d620 0x7f1ed803fb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:23:34.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:28.514+0000 7f1eedd9b700 1 -- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f1ed803d620 msgr2=0x7f1ed803fb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1017216348 2024-04-25T10:23:34.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:28.514+0000 7f1eedd9b700 1 --2- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f1ed803d620 0x7f1ed803fb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:23:34.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:30.115+0000 7f1eedd9b700 1 -- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f1ed803d620 msgr2=0x7f1ed803fb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1017216348 2024-04-25T10:23:34.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:30.115+0000 7f1eedd9b700 1 --2- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f1ed803d620 0x7f1ed803fb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T10:23:34.003 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:32.073+0000 7f1edf7fe700 1 -- 172.21.15.43:0/1304586592 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mgrmap(e 10) ==== 49296+0+0 (secure 0 0 0) 0x7f1ee0018350 con 0x7f1ef0108510 2024-04-25T10:23:34.003 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:32.073+0000 7f1edf7fe700 1 -- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f1ed803d620 msgr2=0x7f1ed803fb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:23:34.003 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:32.073+0000 7f1edf7fe700 1 --2- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f1ed803d620 0x7f1ed803fb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.003 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:33.075+0000 7f1edf7fe700 1 -- 172.21.15.43:0/1304586592 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 11) ==== 49423+0+0 (secure 0 0 0) 0x7f1ee0012d50 con 0x7f1ef0108510 2024-04-25T10:23:34.003 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:33.075+0000 7f1edf7fe700 1 --2- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1ee0053790 0x7f1ed8042fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.003 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:33.075+0000 7f1edf7fe700 1 -- 172.21.15.43:0/1304586592 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1ed8040240 con 0x7f1ee0053790 2024-04-25T10:23:34.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:33.076+0000 7f1eedd9b700 1 --2- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1ee0053790 0x7f1ed8042fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:33.077+0000 7f1eedd9b700 1 --2- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1ee0053790 0x7f1ed8042fc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4003a10 tx=0x7f1ee4009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:34.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:33.078+0000 7f1edf7fe700 1 -- 172.21.15.43:0/1304586592 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f1ed8040240 con 0x7f1ee0053790 2024-04-25T10:23:34.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:33.086+0000 7f1edd7fa700 1 -- 172.21.15.43:0/1304586592 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f1ef0064cf0 con 0x7f1ee0053790 2024-04-25T10:23:34.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:33.086+0000 7f1edf7fe700 1 -- 172.21.15.43:0/1304586592 <== mgr.14152 v2:172.21.15.43:6800/1953858940 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f1ef0064cf0 con 0x7f1ee0053790 2024-04-25T10:23:34.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:33.087+0000 7f1edd7fa700 1 -- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1ee0053790 msgr2=0x7f1ed8042fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:33.087+0000 7f1edd7fa700 1 --2- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1ee0053790 0x7f1ed8042fc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4003a10 tx=0x7f1ee4009270 comp rx=0 tx=0).stop 2024-04-25T10:23:34.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:33.087+0000 7f1edd7fa700 1 -- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ef0108510 msgr2=0x7f1ef01a0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:33.087+0000 7f1edd7fa700 1 --2- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ef0108510 0x7f1ef01a0170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0000c00 tx=0x7f1ee0011aa0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:33.087+0000 7f1edd7fa700 1 -- 172.21.15.43:0/1304586592 shutdown_connections 2024-04-25T10:23:34.005 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:33.087+0000 7f1edd7fa700 1 --2- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1ee0053790 0x7f1ed8042fc0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.005 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:33.087+0000 7f1edd7fa700 1 --2- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ef0108510 0x7f1ef01a0170 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.005 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:33.087+0000 7f1edd7fa700 1 --2- 172.21.15.43:0/1304586592 >> [v2:172.21.15.43:6800/1017216348,v1:172.21.15.43:6801/1017216348] conn(0x7f1ed803d620 0x7f1ed803fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.005 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:33.087+0000 7f1edd7fa700 1 -- 172.21.15.43:0/1304586592 >> 172.21.15.43:0/1304586592 conn(0x7f1ef0104120 msgr2=0x7f1ef0194140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:34.005 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:33.087+0000 7f1edd7fa700 1 -- 172.21.15.43:0/1304586592 shutdown_connections 2024-04-25T10:23:34.005 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:33.087+0000 7f1edd7fa700 1 -- 172.21.15.43:0/1304586592 wait complete. 2024-04-25T10:23:34.005 INFO:teuthology.orchestra.run.smithi043.stdout:mgr epoch 9 is available 2024-04-25T10:23:34.005 INFO:teuthology.orchestra.run.smithi043.stdout:Generating a dashboard self-signed certificate... 2024-04-25T10:23:34.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:34 smithi043 ceph-mon[29478]: [25/Apr/2024:10:23:33] ENGINE Bus STARTING 2024-04-25T10:23:34.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:34 smithi043 ceph-mon[29478]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T10:23:34.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:34 smithi043 ceph-mon[29478]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T10:23:34.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:34 smithi043 ceph-mon[29478]: [25/Apr/2024:10:23:33] ENGINE Serving on https://172.21.15.43:7150 2024-04-25T10:23:34.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:34 smithi043 ceph-mon[29478]: [25/Apr/2024:10:23:33] ENGINE Serving on http://172.21.15.43:8765 2024-04-25T10:23:34.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:34 smithi043 ceph-mon[29478]: [25/Apr/2024:10:23:33] ENGINE Bus STARTED 2024-04-25T10:23:34.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:23:35.577 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T10:23:35.578 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.473+0000 7f5fda089700 1 Processor -- start 2024-04-25T10:23:35.578 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.473+0000 7f5fda089700 1 -- start start 2024-04-25T10:23:35.578 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.474+0000 7f5fda089700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fd410a710 0x7f5fd410cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.578 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.474+0000 7f5fda089700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd407ce60 con 0x7f5fd410a710 2024-04-25T10:23:35.578 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.474+0000 7f5fd37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fd410a710 0x7f5fd410cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.578 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.474+0000 7f5fd37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fd410a710 0x7f5fd410cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38730/0 (socket says 172.21.15.43:38730) 2024-04-25T10:23:35.578 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.474+0000 7f5fd37fe700 1 -- 172.21.15.43:0/584524619 learned_addr learned my addr 172.21.15.43:0/584524619 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:35.578 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.474+0000 7f5fd37fe700 1 -- 172.21.15.43:0/584524619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd410d070 con 0x7f5fd410a710 2024-04-25T10:23:35.578 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.474+0000 7f5fd37fe700 1 --2- 172.21.15.43:0/584524619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fd410a710 0x7f5fd410cb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0098a0 tx=0x7f5fbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655917d3e2f3cdd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:35.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.474+0000 7f5fd27fc700 1 -- 172.21.15.43:0/584524619 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fbc004160 con 0x7f5fd410a710 2024-04-25T10:23:35.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.475+0000 7f5fd27fc700 1 -- 172.21.15.43:0/584524619 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fbc004300 con 0x7f5fd410a710 2024-04-25T10:23:35.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.475+0000 7f5fd27fc700 1 -- 172.21.15.43:0/584524619 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fbc011a20 con 0x7f5fd410a710 2024-04-25T10:23:35.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.475+0000 7f5fda089700 1 -- 172.21.15.43:0/584524619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fd410a710 msgr2=0x7f5fd410cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.475+0000 7f5fda089700 1 --2- 172.21.15.43:0/584524619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fd410a710 0x7f5fd410cb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0098a0 tx=0x7f5fbc009bb0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.475+0000 7f5fda089700 1 -- 172.21.15.43:0/584524619 shutdown_connections 2024-04-25T10:23:35.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.475+0000 7f5fda089700 1 --2- 172.21.15.43:0/584524619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fd410a710 0x7f5fd410cb30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.475+0000 7f5fda089700 1 -- 172.21.15.43:0/584524619 >> 172.21.15.43:0/584524619 conn(0x7f5fd4104170 msgr2=0x7f5fd41065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:35.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.476+0000 7f5fda089700 1 -- 172.21.15.43:0/584524619 shutdown_connections 2024-04-25T10:23:35.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.476+0000 7f5fda089700 1 -- 172.21.15.43:0/584524619 wait complete. 2024-04-25T10:23:35.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.476+0000 7f5fda089700 1 Processor -- start 2024-04-25T10:23:35.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.477+0000 7f5fda089700 1 -- start start 2024-04-25T10:23:35.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.477+0000 7f5fda089700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fd410a710 0x7f5fd419dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.477+0000 7f5fda089700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd410e010 con 0x7f5fd410a710 2024-04-25T10:23:35.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.477+0000 7f5fd37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fd410a710 0x7f5fd419dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.477+0000 7f5fd37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fd410a710 0x7f5fd419dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38734/0 (socket says 172.21.15.43:38734) 2024-04-25T10:23:35.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.477+0000 7f5fd37fe700 1 -- 172.21.15.43:0/3873218852 learned_addr learned my addr 172.21.15.43:0/3873218852 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:35.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.477+0000 7f5fd37fe700 1 -- 172.21.15.43:0/3873218852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd419e4f0 con 0x7f5fd410a710 2024-04-25T10:23:35.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.478+0000 7f5fd37fe700 1 --2- 172.21.15.43:0/3873218852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fd410a710 0x7f5fd419dfb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0119f0 tx=0x7f5fbc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:35.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.478+0000 7f5fd0ff9700 1 -- 172.21.15.43:0/3873218852 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fbc0039e0 con 0x7f5fd410a710 2024-04-25T10:23:35.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.478+0000 7f5fda089700 1 -- 172.21.15.43:0/3873218852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd419e780 con 0x7f5fd410a710 2024-04-25T10:23:35.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.478+0000 7f5fd0ff9700 1 -- 172.21.15.43:0/3873218852 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fbc01b8f0 con 0x7f5fd410a710 2024-04-25T10:23:35.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.478+0000 7f5fd0ff9700 1 -- 172.21.15.43:0/3873218852 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fbc0237a0 con 0x7f5fd410a710 2024-04-25T10:23:35.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.478+0000 7f5fda089700 1 -- 172.21.15.43:0/3873218852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd41a1390 con 0x7f5fd410a710 2024-04-25T10:23:35.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.479+0000 7f5fd0ff9700 1 -- 172.21.15.43:0/3873218852 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 11) ==== 49423+0+0 (secure 0 0 0) 0x7f5fbc02c3f0 con 0x7f5fd410a710 2024-04-25T10:23:35.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.479+0000 7f5fd0ff9700 1 --2- 172.21.15.43:0/3873218852 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5fc003d460 0x7f5fc003f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.479+0000 7f5fd0ff9700 1 -- 172.21.15.43:0/3873218852 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5fbc053b10 con 0x7f5fd410a710 2024-04-25T10:23:35.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.479+0000 7f5fd2ffd700 1 --2- 172.21.15.43:0/3873218852 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5fc003d460 0x7f5fc003f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.479+0000 7f5fda089700 1 -- 172.21.15.43:0/3873218852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fd4064cf0 con 0x7f5fd410a710 2024-04-25T10:23:35.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.480+0000 7f5fd2ffd700 1 --2- 172.21.15.43:0/3873218852 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5fc003d460 0x7f5fc003f970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4006fd0 tx=0x7f5fc4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:35.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.485+0000 7f5fd0ff9700 1 -- 172.21.15.43:0/3873218852 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5fbc01f030 con 0x7f5fd410a710 2024-04-25T10:23:35.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.689+0000 7f5fd0ff9700 1 -- 172.21.15.43:0/3873218852 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7f5fbc00bcc0 con 0x7f5fd410a710 2024-04-25T10:23:35.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:34.700+0000 7f5fda089700 1 -- 172.21.15.43:0/3873218852 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f5fd41a1620 con 0x7f5fc003d460 2024-04-25T10:23:35.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:35.019+0000 7f5fd0ff9700 1 -- 172.21.15.43:0/3873218852 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f5fd41a1620 con 0x7f5fc003d460 2024-04-25T10:23:35.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:35.021+0000 7f5fda089700 1 -- 172.21.15.43:0/3873218852 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5fc003d460 msgr2=0x7f5fc003f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:35.021+0000 7f5fda089700 1 --2- 172.21.15.43:0/3873218852 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5fc003d460 0x7f5fc003f970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4006fd0 tx=0x7f5fc4006e40 comp rx=0 tx=0).stop 2024-04-25T10:23:35.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:35.021+0000 7f5fda089700 1 -- 172.21.15.43:0/3873218852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fd410a710 msgr2=0x7f5fd419dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:35.021+0000 7f5fda089700 1 --2- 172.21.15.43:0/3873218852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fd410a710 0x7f5fd419dfb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0119f0 tx=0x7f5fbc011ad0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:35.021+0000 7f5fda089700 1 -- 172.21.15.43:0/3873218852 shutdown_connections 2024-04-25T10:23:35.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:35.021+0000 7f5fda089700 1 --2- 172.21.15.43:0/3873218852 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5fc003d460 0x7f5fc003f970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:35.021+0000 7f5fda089700 1 --2- 172.21.15.43:0/3873218852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fd410a710 0x7f5fd419dfb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:35.021+0000 7f5fda089700 1 -- 172.21.15.43:0/3873218852 >> 172.21.15.43:0/3873218852 conn(0x7f5fd4104170 msgr2=0x7f5fd4104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:35.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:35.022+0000 7f5fda089700 1 -- 172.21.15.43:0/3873218852 shutdown_connections 2024-04-25T10:23:35.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:35.022+0000 7f5fda089700 1 -- 172.21.15.43:0/3873218852 wait complete. 2024-04-25T10:23:35.582 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial admin user... 2024-04-25T10:23:35.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:35 smithi043 ceph-mon[29478]: mgrmap e12: x(active, since 2s) 2024-04-25T10:23:35.981 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:35 smithi043 ceph-mon[29478]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:23:35.981 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:35 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:23:35.981 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:35 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:23:35.981 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:35 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:23:37.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$wZWwHd77EmJgeY/hTlA9SeU3Zg/sIstFUN3G5ji36tcTtqt7GZDUm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714040616, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T10:23:37.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.013+0000 7ff2919b1700 1 Processor -- start 2024-04-25T10:23:37.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.013+0000 7ff2919b1700 1 -- start start 2024-04-25T10:23:37.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.014+0000 7ff2919b1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c109bd0 0x7ff28c109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:37.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.014+0000 7ff2919b1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c10a5d0 con 0x7ff28c109bd0 2024-04-25T10:23:37.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.014+0000 7ff28affd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c109bd0 0x7ff28c109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:37.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.014+0000 7ff28affd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c109bd0 0x7ff28c109fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38750/0 (socket says 172.21.15.43:38750) 2024-04-25T10:23:37.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.014+0000 7ff28affd700 1 -- 172.21.15.43:0/1445012285 learned_addr learned my addr 172.21.15.43:0/1445012285 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:37.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.014+0000 7ff28affd700 1 -- 172.21.15.43:0/1445012285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28c10ada0 con 0x7ff28c109bd0 2024-04-25T10:23:37.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.015+0000 7ff28affd700 1 --2- 172.21.15.43:0/1445012285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c109bd0 0x7ff28c109fe0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff27c0098a0 tx=0x7ff27c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9f7f1b70288ca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:37.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.015+0000 7ff289ffb700 1 -- 172.21.15.43:0/1445012285 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff27c004160 con 0x7ff28c109bd0 2024-04-25T10:23:37.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.015+0000 7ff289ffb700 1 -- 172.21.15.43:0/1445012285 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff27c004300 con 0x7ff28c109bd0 2024-04-25T10:23:37.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.015+0000 7ff289ffb700 1 -- 172.21.15.43:0/1445012285 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff27c011a20 con 0x7ff28c109bd0 2024-04-25T10:23:37.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.016+0000 7ff2919b1700 1 -- 172.21.15.43:0/1445012285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c109bd0 msgr2=0x7ff28c109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:37.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.016+0000 7ff2919b1700 1 --2- 172.21.15.43:0/1445012285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c109bd0 0x7ff28c109fe0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff27c0098a0 tx=0x7ff27c009bb0 comp rx=0 tx=0).stop 2024-04-25T10:23:37.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.016+0000 7ff2919b1700 1 -- 172.21.15.43:0/1445012285 shutdown_connections 2024-04-25T10:23:37.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.016+0000 7ff2919b1700 1 --2- 172.21.15.43:0/1445012285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c109bd0 0x7ff28c109fe0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:37.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.016+0000 7ff2919b1700 1 -- 172.21.15.43:0/1445012285 >> 172.21.15.43:0/1445012285 conn(0x7ff28c105290 msgr2=0x7ff28c1076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:37.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.016+0000 7ff2919b1700 1 -- 172.21.15.43:0/1445012285 shutdown_connections 2024-04-25T10:23:37.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.016+0000 7ff2919b1700 1 -- 172.21.15.43:0/1445012285 wait complete. 2024-04-25T10:23:37.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.017+0000 7ff2919b1700 1 Processor -- start 2024-04-25T10:23:37.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.017+0000 7ff2919b1700 1 -- start start 2024-04-25T10:23:37.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.017+0000 7ff2919b1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c109bd0 0x7ff28c19e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:37.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.017+0000 7ff2919b1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c10e150 con 0x7ff28c109bd0 2024-04-25T10:23:37.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.017+0000 7ff28affd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c109bd0 0x7ff28c19e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:37.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.017+0000 7ff28affd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c109bd0 0x7ff28c19e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38764/0 (socket says 172.21.15.43:38764) 2024-04-25T10:23:37.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.017+0000 7ff28affd700 1 -- 172.21.15.43:0/3065749497 learned_addr learned my addr 172.21.15.43:0/3065749497 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:37.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.018+0000 7ff28affd700 1 -- 172.21.15.43:0/3065749497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28c19e570 con 0x7ff28c109bd0 2024-04-25T10:23:37.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.018+0000 7ff28affd700 1 --2- 172.21.15.43:0/3065749497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c109bd0 0x7ff28c19e030 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff27c003e70 tx=0x7ff27c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:37.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.018+0000 7ff2909af700 1 -- 172.21.15.43:0/3065749497 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff27c003aa0 con 0x7ff28c109bd0 2024-04-25T10:23:37.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.018+0000 7ff2909af700 1 -- 172.21.15.43:0/3065749497 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff27c01a8e0 con 0x7ff28c109bd0 2024-04-25T10:23:37.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.018+0000 7ff2909af700 1 -- 172.21.15.43:0/3065749497 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff27c024870 con 0x7ff28c109bd0 2024-04-25T10:23:37.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.018+0000 7ff2919b1700 1 -- 172.21.15.43:0/3065749497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28c19e800 con 0x7ff28c109bd0 2024-04-25T10:23:37.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.018+0000 7ff2919b1700 1 -- 172.21.15.43:0/3065749497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff28c1a1410 con 0x7ff28c109bd0 2024-04-25T10:23:37.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.019+0000 7ff2909af700 1 -- 172.21.15.43:0/3065749497 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7ff27c022020 con 0x7ff28c109bd0 2024-04-25T10:23:37.091 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.020+0000 7ff2909af700 1 --2- 172.21.15.43:0/3065749497 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff27403d620 0x7ff27403fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:37.091 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.020+0000 7ff2919b1700 1 -- 172.21.15.43:0/3065749497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff28c064cf0 con 0x7ff28c109bd0 2024-04-25T10:23:37.091 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.020+0000 7ff2909af700 1 -- 172.21.15.43:0/3065749497 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff27c0527e0 con 0x7ff28c109bd0 2024-04-25T10:23:37.091 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.020+0000 7ff28a7fc700 1 --2- 172.21.15.43:0/3065749497 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff27403d620 0x7ff27403fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:37.091 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.020+0000 7ff28a7fc700 1 --2- 172.21.15.43:0/3065749497 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff27403d620 0x7ff27403fb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff280006fd0 tx=0x7ff280006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:37.091 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.025+0000 7ff2909af700 1 -- 172.21.15.43:0/3065749497 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff27c029df0 con 0x7ff28c109bd0 2024-04-25T10:23:37.091 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.257+0000 7ff2919b1700 1 -- 172.21.15.43:0/3065749497 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7ff28c1a16a0 con 0x7ff27403d620 2024-04-25T10:23:37.091 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.504+0000 7ff2909af700 1 -- 172.21.15.43:0/3065749497 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7ff28c1a16a0 con 0x7ff27403d620 2024-04-25T10:23:37.091 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.507+0000 7ff2919b1700 1 -- 172.21.15.43:0/3065749497 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff27403d620 msgr2=0x7ff27403fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:37.091 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.507+0000 7ff2919b1700 1 --2- 172.21.15.43:0/3065749497 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff27403d620 0x7ff27403fb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff280006fd0 tx=0x7ff280006e40 comp rx=0 tx=0).stop 2024-04-25T10:23:37.091 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.507+0000 7ff2919b1700 1 -- 172.21.15.43:0/3065749497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c109bd0 msgr2=0x7ff28c19e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:37.091 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.507+0000 7ff2919b1700 1 --2- 172.21.15.43:0/3065749497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c109bd0 0x7ff28c19e030 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff27c003e70 tx=0x7ff27c011aa0 comp rx=0 tx=0).stop 2024-04-25T10:23:37.091 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.507+0000 7ff2919b1700 1 -- 172.21.15.43:0/3065749497 shutdown_connections 2024-04-25T10:23:37.092 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.507+0000 7ff2919b1700 1 --2- 172.21.15.43:0/3065749497 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff27403d620 0x7ff27403fb30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:37.092 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.507+0000 7ff2919b1700 1 --2- 172.21.15.43:0/3065749497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c109bd0 0x7ff28c19e030 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:37.092 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.507+0000 7ff2919b1700 1 -- 172.21.15.43:0/3065749497 >> 172.21.15.43:0/3065749497 conn(0x7ff28c105290 msgr2=0x7ff28c106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:37.092 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.507+0000 7ff2919b1700 1 -- 172.21.15.43:0/3065749497 shutdown_connections 2024-04-25T10:23:37.092 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:36.507+0000 7ff2919b1700 1 -- 172.21.15.43:0/3065749497 wait complete. 2024-04-25T10:23:37.092 INFO:teuthology.orchestra.run.smithi043.stdout:Fetching dashboard port number... 2024-04-25T10:23:37.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:37 smithi043 ceph-mon[29478]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:23:37.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:37 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:23:38.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T10:23:38.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.648+0000 7f845759e700 1 Processor -- start 2024-04-25T10:23:38.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.649+0000 7f845759e700 1 -- start start 2024-04-25T10:23:38.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.649+0000 7f845759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8458108900 0x7f8458108d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:38.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.649+0000 7f845759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8458109300 con 0x7f8458108900 2024-04-25T10:23:38.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.649+0000 7f845659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8458108900 0x7f8458108d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:38.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.649+0000 7f845659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8458108900 0x7f8458108d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38768/0 (socket says 172.21.15.43:38768) 2024-04-25T10:23:38.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.649+0000 7f845659c700 1 -- 172.21.15.43:0/3458797890 learned_addr learned my addr 172.21.15.43:0/3458797890 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:38.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.649+0000 7f845659c700 1 -- 172.21.15.43:0/3458797890 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8458109ad0 con 0x7f8458108900 2024-04-25T10:23:38.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.650+0000 7f845659c700 1 --2- 172.21.15.43:0/3458797890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8458108900 0x7f8458108d10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f84480098a0 tx=0x7f8448009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9d6fabf3a81045 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:38.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.650+0000 7f845559a700 1 -- 172.21.15.43:0/3458797890 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8448004160 con 0x7f8458108900 2024-04-25T10:23:38.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.650+0000 7f845559a700 1 -- 172.21.15.43:0/3458797890 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8448004300 con 0x7f8458108900 2024-04-25T10:23:38.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.650+0000 7f845559a700 1 -- 172.21.15.43:0/3458797890 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8448011a20 con 0x7f8458108900 2024-04-25T10:23:38.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.650+0000 7f845759e700 1 -- 172.21.15.43:0/3458797890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8458108900 msgr2=0x7f8458108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:38.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.650+0000 7f845759e700 1 --2- 172.21.15.43:0/3458797890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8458108900 0x7f8458108d10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f84480098a0 tx=0x7f8448009bb0 comp rx=0 tx=0).stop 2024-04-25T10:23:38.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.651+0000 7f845759e700 1 -- 172.21.15.43:0/3458797890 shutdown_connections 2024-04-25T10:23:38.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.651+0000 7f845759e700 1 --2- 172.21.15.43:0/3458797890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8458108900 0x7f8458108d10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:38.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.651+0000 7f845759e700 1 -- 172.21.15.43:0/3458797890 >> 172.21.15.43:0/3458797890 conn(0x7f8458103f40 msgr2=0x7f84581063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:38.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.651+0000 7f845759e700 1 -- 172.21.15.43:0/3458797890 shutdown_connections 2024-04-25T10:23:38.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.651+0000 7f845759e700 1 -- 172.21.15.43:0/3458797890 wait complete. 2024-04-25T10:23:38.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.651+0000 7f845759e700 1 Processor -- start 2024-04-25T10:23:38.623 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.651+0000 7f845759e700 1 -- start start 2024-04-25T10:23:38.623 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.652+0000 7f845759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8458108900 0x7f845819de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:38.623 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.652+0000 7f845759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f845810ce80 con 0x7f8458108900 2024-04-25T10:23:38.623 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.652+0000 7f845659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8458108900 0x7f845819de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:38.623 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.652+0000 7f845659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8458108900 0x7f845819de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38770/0 (socket says 172.21.15.43:38770) 2024-04-25T10:23:38.624 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.652+0000 7f845659c700 1 -- 172.21.15.43:0/1700192525 learned_addr learned my addr 172.21.15.43:0/1700192525 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:38.624 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.652+0000 7f845659c700 1 -- 172.21.15.43:0/1700192525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845819e3c0 con 0x7f8458108900 2024-04-25T10:23:38.624 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.652+0000 7f845659c700 1 --2- 172.21.15.43:0/1700192525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8458108900 0x7f845819de80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8448011a20 tx=0x7f8448011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:38.624 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.652+0000 7f84477fe700 1 -- 172.21.15.43:0/1700192525 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84480039c0 con 0x7f8458108900 2024-04-25T10:23:38.624 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.652+0000 7f84477fe700 1 -- 172.21.15.43:0/1700192525 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8448003b60 con 0x7f8458108900 2024-04-25T10:23:38.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.653+0000 7f845759e700 1 -- 172.21.15.43:0/1700192525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f845819e650 con 0x7f8458108900 2024-04-25T10:23:38.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.653+0000 7f84477fe700 1 -- 172.21.15.43:0/1700192525 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8448019700 con 0x7f8458108900 2024-04-25T10:23:38.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.653+0000 7f845759e700 1 -- 172.21.15.43:0/1700192525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84581a1260 con 0x7f8458108900 2024-04-25T10:23:38.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.654+0000 7f845759e700 1 -- 172.21.15.43:0/1700192525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8458064cf0 con 0x7f8458108900 2024-04-25T10:23:38.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.654+0000 7f84477fe700 1 -- 172.21.15.43:0/1700192525 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7f84480199b0 con 0x7f8458108900 2024-04-25T10:23:38.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.654+0000 7f84477fe700 1 --2- 172.21.15.43:0/1700192525 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f844003d5d0 0x7f844003fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:38.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.654+0000 7f84477fe700 1 -- 172.21.15.43:0/1700192525 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8448052c20 con 0x7f8458108900 2024-04-25T10:23:38.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.654+0000 7f8455d9b700 1 --2- 172.21.15.43:0/1700192525 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f844003d5d0 0x7f844003fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:38.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.654+0000 7f8455d9b700 1 --2- 172.21.15.43:0/1700192525 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f844003d5d0 0x7f844003fae0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f844c006fd0 tx=0x7f844c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:38.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.659+0000 7f84477fe700 1 -- 172.21.15.43:0/1700192525 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8448020e20 con 0x7f8458108900 2024-04-25T10:23:38.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.866+0000 7f845759e700 1 -- 172.21.15.43:0/1700192525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f845802e7e0 con 0x7f8458108900 2024-04-25T10:23:38.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.866+0000 7f84477fe700 1 -- 172.21.15.43:0/1700192525 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f844800b3e0 con 0x7f8458108900 2024-04-25T10:23:38.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.868+0000 7f845759e700 1 -- 172.21.15.43:0/1700192525 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f844003d5d0 msgr2=0x7f844003fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:38.626 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.868+0000 7f845759e700 1 --2- 172.21.15.43:0/1700192525 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f844003d5d0 0x7f844003fae0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f844c006fd0 tx=0x7f844c006e40 comp rx=0 tx=0).stop 2024-04-25T10:23:38.626 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.868+0000 7f845759e700 1 -- 172.21.15.43:0/1700192525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8458108900 msgr2=0x7f845819de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:38.626 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.868+0000 7f845759e700 1 --2- 172.21.15.43:0/1700192525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8458108900 0x7f845819de80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8448011a20 tx=0x7f8448011b00 comp rx=0 tx=0).stop 2024-04-25T10:23:38.626 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.868+0000 7f845759e700 1 -- 172.21.15.43:0/1700192525 shutdown_connections 2024-04-25T10:23:38.626 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.868+0000 7f845759e700 1 --2- 172.21.15.43:0/1700192525 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f844003d5d0 0x7f844003fae0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:38.626 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.868+0000 7f845759e700 1 --2- 172.21.15.43:0/1700192525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8458108900 0x7f845819de80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:38.626 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.868+0000 7f845759e700 1 -- 172.21.15.43:0/1700192525 >> 172.21.15.43:0/1700192525 conn(0x7f8458103f40 msgr2=0x7f8458194430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:38.626 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.868+0000 7f845759e700 1 -- 172.21.15.43:0/1700192525 shutdown_connections 2024-04-25T10:23:38.626 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:37.868+0000 7f845759e700 1 -- 172.21.15.43:0/1700192525 wait complete. 2024-04-25T10:23:38.627 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T10:23:38.627 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-04-25T10:23:38.631 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T10:23:38.631 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-04-25T10:23:38.631 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-04-25T10:23:38.631 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T10:23:38.633 INFO:teuthology.orchestra.run.smithi043.stdout:Ceph Dashboard is now available at: 2024-04-25T10:23:38.633 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:23:38.633 INFO:teuthology.orchestra.run.smithi043.stdout: URL: https://smithi043:8443/ 2024-04-25T10:23:38.634 INFO:teuthology.orchestra.run.smithi043.stdout: User: admin 2024-04-25T10:23:38.634 INFO:teuthology.orchestra.run.smithi043.stdout: Password: jldlhhyhe3 2024-04-25T10:23:38.634 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:23:38.635 INFO:teuthology.orchestra.run.smithi043.stdout:Saving cluster configuration to /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config directory 2024-04-25T10:23:38.635 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling autotune for osd_memory_target 2024-04-25T10:23:38.761 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:38 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1700192525' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T10:23:39.602 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:39 smithi043 ceph-mon[29478]: mgrmap e13: x(active, since 6s) 2024-04-25T10:23:39.821 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.074+0000 7f63f545c700 1 Processor -- start 2024-04-25T10:23:39.912 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.074+0000 7f63f545c700 1 -- start start 2024-04-25T10:23:39.912 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.075+0000 7f63f545c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f0109bd0 0x7f63f0109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:39.912 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.075+0000 7f63f545c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f63f010a5d0 con 0x7f63f0109bd0 2024-04-25T10:23:39.912 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.075+0000 7f63eeffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f0109bd0 0x7f63f0109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:39.912 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.075+0000 7f63eeffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f0109bd0 0x7f63f0109fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35180/0 (socket says 172.21.15.43:35180) 2024-04-25T10:23:39.912 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.075+0000 7f63eeffd700 1 -- 172.21.15.43:0/3059291607 learned_addr learned my addr 172.21.15.43:0/3059291607 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:39.912 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.075+0000 7f63eeffd700 1 -- 172.21.15.43:0/3059291607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f010ada0 con 0x7f63f0109bd0 2024-04-25T10:23:39.912 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.075+0000 7f63eeffd700 1 --2- 172.21.15.43:0/3059291607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f0109bd0 0x7f63f0109fe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009e80 tx=0x7f63e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0279a80af86331e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:39.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.076+0000 7f63edffb700 1 -- 172.21.15.43:0/3059291607 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63e0017070 con 0x7f63f0109bd0 2024-04-25T10:23:39.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.076+0000 7f63edffb700 1 -- 172.21.15.43:0/3059291607 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63e000b8c0 con 0x7f63f0109bd0 2024-04-25T10:23:39.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.076+0000 7f63edffb700 1 -- 172.21.15.43:0/3059291607 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63e0004490 con 0x7f63f0109bd0 2024-04-25T10:23:39.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.076+0000 7f63f545c700 1 -- 172.21.15.43:0/3059291607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f0109bd0 msgr2=0x7f63f0109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:39.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.076+0000 7f63f545c700 1 --2- 172.21.15.43:0/3059291607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f0109bd0 0x7f63f0109fe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009e80 tx=0x7f63e000b330 comp rx=0 tx=0).stop 2024-04-25T10:23:39.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.076+0000 7f63f545c700 1 -- 172.21.15.43:0/3059291607 shutdown_connections 2024-04-25T10:23:39.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.076+0000 7f63f545c700 1 --2- 172.21.15.43:0/3059291607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f0109bd0 0x7f63f0109fe0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:39.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.077+0000 7f63f545c700 1 -- 172.21.15.43:0/3059291607 >> 172.21.15.43:0/3059291607 conn(0x7f63f0105290 msgr2=0x7f63f01076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:39.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.077+0000 7f63f545c700 1 -- 172.21.15.43:0/3059291607 shutdown_connections 2024-04-25T10:23:39.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.077+0000 7f63f545c700 1 -- 172.21.15.43:0/3059291607 wait complete. 2024-04-25T10:23:39.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.077+0000 7f63f545c700 1 Processor -- start 2024-04-25T10:23:39.914 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.078+0000 7f63f545c700 1 -- start start 2024-04-25T10:23:39.914 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.078+0000 7f63f545c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f0109bd0 0x7f63f019e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:39.914 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.078+0000 7f63f545c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f63f010e150 con 0x7f63f0109bd0 2024-04-25T10:23:39.914 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.078+0000 7f63eeffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f0109bd0 0x7f63f019e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:39.914 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.078+0000 7f63eeffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f0109bd0 0x7f63f019e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35182/0 (socket says 172.21.15.43:35182) 2024-04-25T10:23:39.914 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.078+0000 7f63eeffd700 1 -- 172.21.15.43:0/753666679 learned_addr learned my addr 172.21.15.43:0/753666679 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:39.914 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.078+0000 7f63eeffd700 1 -- 172.21.15.43:0/753666679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f019e590 con 0x7f63f0109bd0 2024-04-25T10:23:39.914 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.078+0000 7f63eeffd700 1 --2- 172.21.15.43:0/753666679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f0109bd0 0x7f63f019e050 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009fd0 tx=0x7f63e000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:39.914 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.079+0000 7f63e7fff700 1 -- 172.21.15.43:0/753666679 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63e0020020 con 0x7f63f0109bd0 2024-04-25T10:23:39.914 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.079+0000 7f63e7fff700 1 -- 172.21.15.43:0/753666679 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63e0003b50 con 0x7f63f0109bd0 2024-04-25T10:23:39.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.079+0000 7f63f545c700 1 -- 172.21.15.43:0/753666679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f019e820 con 0x7f63f0109bd0 2024-04-25T10:23:39.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.079+0000 7f63e7fff700 1 -- 172.21.15.43:0/753666679 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63e0017040 con 0x7f63f0109bd0 2024-04-25T10:23:39.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.079+0000 7f63f545c700 1 -- 172.21.15.43:0/753666679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f01a1430 con 0x7f63f0109bd0 2024-04-25T10:23:39.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.080+0000 7f63e7fff700 1 -- 172.21.15.43:0/753666679 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f63e0023020 con 0x7f63f0109bd0 2024-04-25T10:23:39.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.080+0000 7f63f545c700 1 -- 172.21.15.43:0/753666679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f0064cf0 con 0x7f63f0109bd0 2024-04-25T10:23:39.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.080+0000 7f63e7fff700 1 --2- 172.21.15.43:0/753666679 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f63dc03d5c0 0x7f63dc03fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:39.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.080+0000 7f63e7fff700 1 -- 172.21.15.43:0/753666679 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f63e000f050 con 0x7f63f0109bd0 2024-04-25T10:23:39.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.080+0000 7f63ee7fc700 1 --2- 172.21.15.43:0/753666679 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f63dc03d5c0 0x7f63dc03fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:39.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.080+0000 7f63ee7fc700 1 --2- 172.21.15.43:0/753666679 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f63dc03d5c0 0x7f63dc03fad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f63d8006fd0 tx=0x7f63d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:39.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.085+0000 7f63e7fff700 1 -- 172.21.15.43:0/753666679 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f63e002a020 con 0x7f63f0109bd0 2024-04-25T10:23:39.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.293+0000 7f63f545c700 1 -- 172.21.15.43:0/753666679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f63f01a1600 con 0x7f63f0109bd0 2024-04-25T10:23:39.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.293+0000 7f63e7fff700 1 -- 172.21.15.43:0/753666679 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f63e0011830 con 0x7f63f0109bd0 2024-04-25T10:23:39.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.296+0000 7f63f545c700 1 -- 172.21.15.43:0/753666679 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f63dc03d5c0 msgr2=0x7f63dc03fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:39.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.296+0000 7f63f545c700 1 --2- 172.21.15.43:0/753666679 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f63dc03d5c0 0x7f63dc03fad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f63d8006fd0 tx=0x7f63d8006e40 comp rx=0 tx=0).stop 2024-04-25T10:23:39.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.296+0000 7f63f545c700 1 -- 172.21.15.43:0/753666679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f0109bd0 msgr2=0x7f63f019e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:39.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.296+0000 7f63f545c700 1 --2- 172.21.15.43:0/753666679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f0109bd0 0x7f63f019e050 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009fd0 tx=0x7f63e000bd30 comp rx=0 tx=0).stop 2024-04-25T10:23:39.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.296+0000 7f63f545c700 1 -- 172.21.15.43:0/753666679 shutdown_connections 2024-04-25T10:23:39.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.296+0000 7f63f545c700 1 --2- 172.21.15.43:0/753666679 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f63dc03d5c0 0x7f63dc03fad0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:39.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.296+0000 7f63f545c700 1 --2- 172.21.15.43:0/753666679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f0109bd0 0x7f63f019e050 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:39.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.296+0000 7f63f545c700 1 -- 172.21.15.43:0/753666679 >> 172.21.15.43:0/753666679 conn(0x7f63f0105290 msgr2=0x7f63f0106c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:39.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.296+0000 7f63f545c700 1 -- 172.21.15.43:0/753666679 shutdown_connections 2024-04-25T10:23:39.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:39.296+0000 7f63f545c700 1 -- 172.21.15.43:0/753666679 wait complete. 2024-04-25T10:23:41.222 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.285+0000 7fa4fe6da700 1 Processor -- start 2024-04-25T10:23:41.222 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.285+0000 7fa4fe6da700 1 -- start start 2024-04-25T10:23:41.222 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.285+0000 7fa4fe6da700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f81084e0 0x7fa4f81088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:41.222 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.285+0000 7fa4fe6da700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f807d6b0 con 0x7fa4f81084e0 2024-04-25T10:23:41.222 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.286+0000 7fa4f7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f81084e0 0x7fa4f81088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:41.222 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.286+0000 7fa4f7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f81084e0 0x7fa4f81088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35194/0 (socket says 172.21.15.43:35194) 2024-04-25T10:23:41.223 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.286+0000 7fa4f7fff700 1 -- 172.21.15.43:0/2537482724 learned_addr learned my addr 172.21.15.43:0/2537482724 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:41.223 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.286+0000 7fa4f7fff700 1 -- 172.21.15.43:0/2537482724 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f8108e30 con 0x7fa4f81084e0 2024-04-25T10:23:41.223 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.286+0000 7fa4f7fff700 1 --2- 172.21.15.43:0/2537482724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f81084e0 0x7fa4f81088f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa4e80098a0 tx=0x7fa4e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c68fab04edd9bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:41.223 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.286+0000 7fa4f6ffd700 1 -- 172.21.15.43:0/2537482724 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4e8004160 con 0x7fa4f81084e0 2024-04-25T10:23:41.223 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.286+0000 7fa4f6ffd700 1 -- 172.21.15.43:0/2537482724 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4e8004300 con 0x7fa4f81084e0 2024-04-25T10:23:41.223 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.286+0000 7fa4f6ffd700 1 -- 172.21.15.43:0/2537482724 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4e8011a20 con 0x7fa4f81084e0 2024-04-25T10:23:41.223 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.287+0000 7fa4fe6da700 1 -- 172.21.15.43:0/2537482724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f81084e0 msgr2=0x7fa4f81088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:41.223 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.287+0000 7fa4fe6da700 1 --2- 172.21.15.43:0/2537482724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f81084e0 0x7fa4f81088f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa4e80098a0 tx=0x7fa4e8009bb0 comp rx=0 tx=0).stop 2024-04-25T10:23:41.224 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.287+0000 7fa4fe6da700 1 -- 172.21.15.43:0/2537482724 shutdown_connections 2024-04-25T10:23:41.224 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.287+0000 7fa4fe6da700 1 --2- 172.21.15.43:0/2537482724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f81084e0 0x7fa4f81088f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:41.224 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.287+0000 7fa4fe6da700 1 -- 172.21.15.43:0/2537482724 >> 172.21.15.43:0/2537482724 conn(0x7fa4f8104170 msgr2=0x7fa4f81065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:41.224 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.287+0000 7fa4fe6da700 1 -- 172.21.15.43:0/2537482724 shutdown_connections 2024-04-25T10:23:41.224 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.287+0000 7fa4fe6da700 1 -- 172.21.15.43:0/2537482724 wait complete. 2024-04-25T10:23:41.224 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.288+0000 7fa4fe6da700 1 Processor -- start 2024-04-25T10:23:41.224 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.288+0000 7fa4fe6da700 1 -- start start 2024-04-25T10:23:41.224 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.288+0000 7fa4fe6da700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f81084e0 0x7fa4f81a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:41.225 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.288+0000 7fa4fe6da700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f8109ef0 con 0x7fa4f81084e0 2024-04-25T10:23:41.225 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.288+0000 7fa4f7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f81084e0 0x7fa4f81a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:41.225 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.288+0000 7fa4f7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f81084e0 0x7fa4f81a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35204/0 (socket says 172.21.15.43:35204) 2024-04-25T10:23:41.225 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.288+0000 7fa4f7fff700 1 -- 172.21.15.43:0/2399201382 learned_addr learned my addr 172.21.15.43:0/2399201382 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:41.225 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.289+0000 7fa4f7fff700 1 -- 172.21.15.43:0/2399201382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f81a06d0 con 0x7fa4f81084e0 2024-04-25T10:23:41.225 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.289+0000 7fa4f7fff700 1 --2- 172.21.15.43:0/2399201382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f81084e0 0x7fa4f81a0190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8000c00 tx=0x7fa4e8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:41.225 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.289+0000 7fa4f57fa700 1 -- 172.21.15.43:0/2399201382 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4e8003aa0 con 0x7fa4f81084e0 2024-04-25T10:23:41.225 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.289+0000 7fa4fe6da700 1 -- 172.21.15.43:0/2399201382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f81a0960 con 0x7fa4f81084e0 2024-04-25T10:23:41.226 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.289+0000 7fa4f57fa700 1 -- 172.21.15.43:0/2399201382 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4e801b8f0 con 0x7fa4f81084e0 2024-04-25T10:23:41.226 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.289+0000 7fa4f57fa700 1 -- 172.21.15.43:0/2399201382 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4e8023890 con 0x7fa4f81084e0 2024-04-25T10:23:41.226 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.289+0000 7fa4fe6da700 1 -- 172.21.15.43:0/2399201382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f81a3570 con 0x7fa4f81084e0 2024-04-25T10:23:41.226 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.290+0000 7fa4fe6da700 1 -- 172.21.15.43:0/2399201382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f8064cf0 con 0x7fa4f81084e0 2024-04-25T10:23:41.226 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.290+0000 7fa4f57fa700 1 -- 172.21.15.43:0/2399201382 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fa4e802c3f0 con 0x7fa4f81084e0 2024-04-25T10:23:41.226 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.290+0000 7fa4f57fa700 1 --2- 172.21.15.43:0/2399201382 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fa4e003d660 0x7fa4e003fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:41.226 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.290+0000 7fa4f57fa700 1 -- 172.21.15.43:0/2399201382 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa4e8052ec0 con 0x7fa4f81084e0 2024-04-25T10:23:41.226 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.291+0000 7fa4f77fe700 1 --2- 172.21.15.43:0/2399201382 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fa4e003d660 0x7fa4e003fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:41.227 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.291+0000 7fa4f77fe700 1 --2- 172.21.15.43:0/2399201382 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fa4e003d660 0x7fa4e003fb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec006fd0 tx=0x7fa4ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:41.227 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.296+0000 7fa4f57fa700 1 -- 172.21.15.43:0/2399201382 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4e801f030 con 0x7fa4f81084e0 2024-04-25T10:23:41.227 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.619+0000 7fa4fe6da700 1 -- 172.21.15.43:0/2399201382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fa4f802e810 con 0x7fa4f81084e0 2024-04-25T10:23:41.227 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.622+0000 7fa4f57fa700 1 -- 172.21.15.43:0/2399201382 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7fa4e801bda0 con 0x7fa4f81084e0 2024-04-25T10:23:41.227 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.624+0000 7fa4fe6da700 1 -- 172.21.15.43:0/2399201382 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fa4e003d660 msgr2=0x7fa4e003fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:41.227 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.624+0000 7fa4fe6da700 1 --2- 172.21.15.43:0/2399201382 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fa4e003d660 0x7fa4e003fb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec006fd0 tx=0x7fa4ec006e40 comp rx=0 tx=0).stop 2024-04-25T10:23:41.227 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.624+0000 7fa4fe6da700 1 -- 172.21.15.43:0/2399201382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f81084e0 msgr2=0x7fa4f81a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:41.228 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.624+0000 7fa4fe6da700 1 --2- 172.21.15.43:0/2399201382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f81084e0 0x7fa4f81a0190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8000c00 tx=0x7fa4e8011aa0 comp rx=0 tx=0).stop 2024-04-25T10:23:41.228 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.624+0000 7fa4fe6da700 1 -- 172.21.15.43:0/2399201382 shutdown_connections 2024-04-25T10:23:41.228 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.624+0000 7fa4fe6da700 1 --2- 172.21.15.43:0/2399201382 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fa4e003d660 0x7fa4e003fb70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:41.228 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.624+0000 7fa4fe6da700 1 --2- 172.21.15.43:0/2399201382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f81084e0 0x7fa4f81a0190 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:41.228 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.624+0000 7fa4fe6da700 1 -- 172.21.15.43:0/2399201382 >> 172.21.15.43:0/2399201382 conn(0x7fa4f8104170 msgr2=0x7fa4f8194110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:41.228 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.624+0000 7fa4fe6da700 1 -- 172.21.15.43:0/2399201382 shutdown_connections 2024-04-25T10:23:41.228 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-25T10:23:40.624+0000 7fa4fe6da700 1 -- 172.21.15.43:0/2399201382 wait complete. 2024-04-25T10:23:41.228 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T10:23:41.229 INFO:teuthology.orchestra.run.smithi043.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T10:23:41.229 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:23:41.229 INFO:teuthology.orchestra.run.smithi043.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T10:23:41.229 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:23:41.229 INFO:teuthology.orchestra.run.smithi043.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T10:23:41.229 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:23:41.229 INFO:teuthology.orchestra.run.smithi043.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T10:23:41.229 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:23:41.229 INFO:teuthology.orchestra.run.smithi043.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T10:23:41.230 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:23:41.230 INFO:teuthology.orchestra.run.smithi043.stdout: ceph telemetry on 2024-04-25T10:23:41.230 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:23:41.230 INFO:teuthology.orchestra.run.smithi043.stdout:For more information see: 2024-04-25T10:23:41.230 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:23:41.230 INFO:teuthology.orchestra.run.smithi043.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T10:23:41.230 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:23:41.230 INFO:teuthology.orchestra.run.smithi043.stdout:Bootstrap complete. 2024-04-25T10:23:41.260 INFO:tasks.cephadm:Fetching config... 2024-04-25T10:23:41.261 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T10:23:41.261 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T10:23:41.277 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T10:23:41.278 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T10:23:41.278 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T10:23:41.335 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T10:23:41.336 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T10:23:41.336 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T10:23:41.401 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T10:23:41.401 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T10:23:41.401 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T10:23:41.457 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T10:23:41.457 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp0FFj48MleeLmYi/P76pN3ALtbU1vznkMj8K1cW4pOuk79FXa9LX4ZUlKpHmncn6qhnprN14LC0VmxZn7sEagF1h3AB2LvjEedAOMe86nxdpn4D4Yos5VUD9qQSzcXBFdV4yxZ8R95F/6AEuzJOyCS8QBOEjMGQHb+LqEfEMDewK1puq2qAyaZ5mvNiNUn2OJD7FgYRmKv2WNxswFx0WIgUW4wud2uZ1lWvbCtzckPJhKHp5enH+UYBZUsOHnLrlOCawxn+4j9D40ohKOZSM3PiZgbgG44lPMqNttC/i/Q8HpSOEQJtEBy+cPWa6VOrN5WGVt+udqsUYqwD1GE8l1eJWW8TdRVMQyp9eJcPM2O+IofNuZJrJSdxxqqoQ2AWBmMRTFbpIyQTNwK3D0EUIWyHIHxeHvNhAe8ScCmt9by6zHTakOben6dRW3BUDaoDUDX/x+a209AGmDZ/xCSFX+pZ3eazXIGCOWVhKZrYrqNAAKVrvur9Ty2GRzeHz3ko8= ceph-94d0035c-02ed-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T10:23:41.530 INFO:teuthology.orchestra.run.smithi043.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp0FFj48MleeLmYi/P76pN3ALtbU1vznkMj8K1cW4pOuk79FXa9LX4ZUlKpHmncn6qhnprN14LC0VmxZn7sEagF1h3AB2LvjEedAOMe86nxdpn4D4Yos5VUD9qQSzcXBFdV4yxZ8R95F/6AEuzJOyCS8QBOEjMGQHb+LqEfEMDewK1puq2qAyaZ5mvNiNUn2OJD7FgYRmKv2WNxswFx0WIgUW4wud2uZ1lWvbCtzckPJhKHp5enH+UYBZUsOHnLrlOCawxn+4j9D40ohKOZSM3PiZgbgG44lPMqNttC/i/Q8HpSOEQJtEBy+cPWa6VOrN5WGVt+udqsUYqwD1GE8l1eJWW8TdRVMQyp9eJcPM2O+IofNuZJrJSdxxqqoQ2AWBmMRTFbpIyQTNwK3D0EUIWyHIHxeHvNhAe8ScCmt9by6zHTakOben6dRW3BUDaoDUDX/x+a209AGmDZ/xCSFX+pZ3eazXIGCOWVhKZrYrqNAAKVrvur9Ty2GRzeHz3ko8= ceph-94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:23:41.541 DEBUG:teuthology.orchestra.run.smithi077:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp0FFj48MleeLmYi/P76pN3ALtbU1vznkMj8K1cW4pOuk79FXa9LX4ZUlKpHmncn6qhnprN14LC0VmxZn7sEagF1h3AB2LvjEedAOMe86nxdpn4D4Yos5VUD9qQSzcXBFdV4yxZ8R95F/6AEuzJOyCS8QBOEjMGQHb+LqEfEMDewK1puq2qAyaZ5mvNiNUn2OJD7FgYRmKv2WNxswFx0WIgUW4wud2uZ1lWvbCtzckPJhKHp5enH+UYBZUsOHnLrlOCawxn+4j9D40ohKOZSM3PiZgbgG44lPMqNttC/i/Q8HpSOEQJtEBy+cPWa6VOrN5WGVt+udqsUYqwD1GE8l1eJWW8TdRVMQyp9eJcPM2O+IofNuZJrJSdxxqqoQ2AWBmMRTFbpIyQTNwK3D0EUIWyHIHxeHvNhAe8ScCmt9by6zHTakOben6dRW3BUDaoDUDX/x+a209AGmDZ/xCSFX+pZ3eazXIGCOWVhKZrYrqNAAKVrvur9Ty2GRzeHz3ko8= ceph-94d0035c-02ed-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T10:23:41.581 INFO:teuthology.orchestra.run.smithi077.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp0FFj48MleeLmYi/P76pN3ALtbU1vznkMj8K1cW4pOuk79FXa9LX4ZUlKpHmncn6qhnprN14LC0VmxZn7sEagF1h3AB2LvjEedAOMe86nxdpn4D4Yos5VUD9qQSzcXBFdV4yxZ8R95F/6AEuzJOyCS8QBOEjMGQHb+LqEfEMDewK1puq2qAyaZ5mvNiNUn2OJD7FgYRmKv2WNxswFx0WIgUW4wud2uZ1lWvbCtzckPJhKHp5enH+UYBZUsOHnLrlOCawxn+4j9D40ohKOZSM3PiZgbgG44lPMqNttC/i/Q8HpSOEQJtEBy+cPWa6VOrN5WGVt+udqsUYqwD1GE8l1eJWW8TdRVMQyp9eJcPM2O+IofNuZJrJSdxxqqoQ2AWBmMRTFbpIyQTNwK3D0EUIWyHIHxeHvNhAe8ScCmt9by6zHTakOben6dRW3BUDaoDUDX/x+a209AGmDZ/xCSFX+pZ3eazXIGCOWVhKZrYrqNAAKVrvur9Ty2GRzeHz3ko8= ceph-94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:23:41.593 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp0FFj48MleeLmYi/P76pN3ALtbU1vznkMj8K1cW4pOuk79FXa9LX4ZUlKpHmncn6qhnprN14LC0VmxZn7sEagF1h3AB2LvjEedAOMe86nxdpn4D4Yos5VUD9qQSzcXBFdV4yxZ8R95F/6AEuzJOyCS8QBOEjMGQHb+LqEfEMDewK1puq2qAyaZ5mvNiNUn2OJD7FgYRmKv2WNxswFx0WIgUW4wud2uZ1lWvbCtzckPJhKHp5enH+UYBZUsOHnLrlOCawxn+4j9D40ohKOZSM3PiZgbgG44lPMqNttC/i/Q8HpSOEQJtEBy+cPWa6VOrN5WGVt+udqsUYqwD1GE8l1eJWW8TdRVMQyp9eJcPM2O+IofNuZJrJSdxxqqoQ2AWBmMRTFbpIyQTNwK3D0EUIWyHIHxeHvNhAe8ScCmt9by6zHTakOben6dRW3BUDaoDUDX/x+a209AGmDZ/xCSFX+pZ3eazXIGCOWVhKZrYrqNAAKVrvur9Ty2GRzeHz3ko8= ceph-94d0035c-02ed-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T10:23:41.633 INFO:teuthology.orchestra.run.smithi164.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp0FFj48MleeLmYi/P76pN3ALtbU1vznkMj8K1cW4pOuk79FXa9LX4ZUlKpHmncn6qhnprN14LC0VmxZn7sEagF1h3AB2LvjEedAOMe86nxdpn4D4Yos5VUD9qQSzcXBFdV4yxZ8R95F/6AEuzJOyCS8QBOEjMGQHb+LqEfEMDewK1puq2qAyaZ5mvNiNUn2OJD7FgYRmKv2WNxswFx0WIgUW4wud2uZ1lWvbCtzckPJhKHp5enH+UYBZUsOHnLrlOCawxn+4j9D40ohKOZSM3PiZgbgG44lPMqNttC/i/Q8HpSOEQJtEBy+cPWa6VOrN5WGVt+udqsUYqwD1GE8l1eJWW8TdRVMQyp9eJcPM2O+IofNuZJrJSdxxqqoQ2AWBmMRTFbpIyQTNwK3D0EUIWyHIHxeHvNhAe8ScCmt9by6zHTakOben6dRW3BUDaoDUDX/x+a209AGmDZ/xCSFX+pZ3eazXIGCOWVhKZrYrqNAAKVrvur9Ty2GRzeHz3ko8= ceph-94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:23:41.645 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T10:23:41.772 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:41 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2399201382' entity='client.admin' 2024-04-25T10:23:41.894 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:23:44.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.085+0000 7f484e622700 1 -- 172.21.15.43:0/3941126500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4848103af0 msgr2=0x7f4848103f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.085+0000 7f484e622700 1 --2- 172.21.15.43:0/3941126500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4848103af0 0x7f4848103f00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4838009910 tx=0x7f4838009c20 comp rx=0 tx=0).stop 2024-04-25T10:23:44.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.085+0000 7f484e622700 1 -- 172.21.15.43:0/3941126500 shutdown_connections 2024-04-25T10:23:44.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.085+0000 7f484e622700 1 --2- 172.21.15.43:0/3941126500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4848103af0 0x7f4848103f00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.085+0000 7f484e622700 1 -- 172.21.15.43:0/3941126500 >> 172.21.15.43:0/3941126500 conn(0x7f48480ff130 msgr2=0x7f48481015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:44.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.085+0000 7f484e622700 1 -- 172.21.15.43:0/3941126500 shutdown_connections 2024-04-25T10:23:44.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.085+0000 7f484e622700 1 -- 172.21.15.43:0/3941126500 wait complete. 2024-04-25T10:23:44.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.086+0000 7f484e622700 1 Processor -- start 2024-04-25T10:23:44.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.086+0000 7f484e622700 1 -- start start 2024-04-25T10:23:44.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.087+0000 7f484e622700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4848103af0 0x7f48481995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.087+0000 7f484e622700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4848109ab0 con 0x7f4848103af0 2024-04-25T10:23:44.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.087+0000 7f4847fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4848103af0 0x7f48481995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:44.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.087+0000 7f4847fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4848103af0 0x7f48481995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35230/0 (socket says 172.21.15.43:35230) 2024-04-25T10:23:44.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.087+0000 7f4847fff700 1 -- 172.21.15.43:0/1239243363 learned_addr learned my addr 172.21.15.43:0/1239243363 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:44.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.087+0000 7f4847fff700 1 -- 172.21.15.43:0/1239243363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4848199b20 con 0x7f4848103af0 2024-04-25T10:23:44.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.087+0000 7f4847fff700 1 --2- 172.21.15.43:0/1239243363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4848103af0 0x7f48481995e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f48380057d0 tx=0x7f4838005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:44.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.087+0000 7f48457fa700 1 -- 172.21.15.43:0/1239243363 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4838027070 con 0x7f4848103af0 2024-04-25T10:23:44.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.087+0000 7f48457fa700 1 -- 172.21.15.43:0/1239243363 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f483800be10 con 0x7f4848103af0 2024-04-25T10:23:44.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.087+0000 7f484e622700 1 -- 172.21.15.43:0/1239243363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4848199db0 con 0x7f4848103af0 2024-04-25T10:23:44.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.087+0000 7f48457fa700 1 -- 172.21.15.43:0/1239243363 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4838022dd0 con 0x7f4848103af0 2024-04-25T10:23:44.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.087+0000 7f484e622700 1 -- 172.21.15.43:0/1239243363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f484819c9c0 con 0x7f4848103af0 2024-04-25T10:23:44.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.088+0000 7f48457fa700 1 -- 172.21.15.43:0/1239243363 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f483802a070 con 0x7f4848103af0 2024-04-25T10:23:44.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.088+0000 7f484e622700 1 -- 172.21.15.43:0/1239243363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f484810a230 con 0x7f4848103af0 2024-04-25T10:23:44.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.089+0000 7f48457fa700 1 --2- 172.21.15.43:0/1239243363 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f483003d5c0 0x7f483003fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.089+0000 7f48457fa700 1 -- 172.21.15.43:0/1239243363 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f483801e030 con 0x7f4848103af0 2024-04-25T10:23:44.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.089+0000 7f48477fe700 1 --2- 172.21.15.43:0/1239243363 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f483003d5c0 0x7f483003fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:44.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.089+0000 7f48477fe700 1 --2- 172.21.15.43:0/1239243363 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f483003d5c0 0x7f483003fad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f483c006fd0 tx=0x7f483c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.094+0000 7f48457fa700 1 -- 172.21.15.43:0/1239243363 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4838013070 con 0x7f4848103af0 2024-04-25T10:23:44.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.301+0000 7f484e622700 1 -- 172.21.15.43:0/1239243363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f484802e620 con 0x7f4848103af0 2024-04-25T10:23:44.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.303+0000 7f48457fa700 1 -- 172.21.15.43:0/1239243363 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f48380310c0 con 0x7f4848103af0 2024-04-25T10:23:44.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.306+0000 7f484e622700 1 -- 172.21.15.43:0/1239243363 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f483003d5c0 msgr2=0x7f483003fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.306+0000 7f484e622700 1 --2- 172.21.15.43:0/1239243363 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f483003d5c0 0x7f483003fad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f483c006fd0 tx=0x7f483c006e40 comp rx=0 tx=0).stop 2024-04-25T10:23:44.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.306+0000 7f484e622700 1 -- 172.21.15.43:0/1239243363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4848103af0 msgr2=0x7f48481995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.306+0000 7f484e622700 1 --2- 172.21.15.43:0/1239243363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4848103af0 0x7f48481995e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f48380057d0 tx=0x7f4838005800 comp rx=0 tx=0).stop 2024-04-25T10:23:44.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.306+0000 7f484e622700 1 -- 172.21.15.43:0/1239243363 shutdown_connections 2024-04-25T10:23:44.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.306+0000 7f484e622700 1 --2- 172.21.15.43:0/1239243363 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f483003d5c0 0x7f483003fad0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.306+0000 7f484e622700 1 --2- 172.21.15.43:0/1239243363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4848103af0 0x7f48481995e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.306+0000 7f484e622700 1 -- 172.21.15.43:0/1239243363 >> 172.21.15.43:0/1239243363 conn(0x7f48480ff130 msgr2=0x7f48480ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:44.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.306+0000 7f484e622700 1 -- 172.21.15.43:0/1239243363 shutdown_connections 2024-04-25T10:23:44.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:44.306+0000 7f484e622700 1 -- 172.21.15.43:0/1239243363 wait complete. 2024-04-25T10:23:44.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:44 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:23:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:44 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:23:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:44 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi043", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:23:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:44 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:23:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:44 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:23:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:44 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:23:44.905 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T10:23:44.905 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T10:23:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:23:45.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:45 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1239243363' entity='client.admin' 2024-04-25T10:23:45.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:45 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:23:45.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:45 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:23:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.630+0000 7fed0eb93700 1 -- 172.21.15.43:0/413516825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed08103af0 msgr2=0x7fed08103f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.630+0000 7fed0eb93700 1 --2- 172.21.15.43:0/413516825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed08103af0 0x7fed08103f00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009910 tx=0x7fecfc009c20 comp rx=0 tx=0).stop 2024-04-25T10:23:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.630+0000 7fed0eb93700 1 -- 172.21.15.43:0/413516825 shutdown_connections 2024-04-25T10:23:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.630+0000 7fed0eb93700 1 --2- 172.21.15.43:0/413516825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed08103af0 0x7fed08103f00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.630+0000 7fed0eb93700 1 -- 172.21.15.43:0/413516825 >> 172.21.15.43:0/413516825 conn(0x7fed080ff130 msgr2=0x7fed081015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.630+0000 7fed0eb93700 1 -- 172.21.15.43:0/413516825 shutdown_connections 2024-04-25T10:23:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.630+0000 7fed0eb93700 1 -- 172.21.15.43:0/413516825 wait complete. 2024-04-25T10:23:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.631+0000 7fed0eb93700 1 Processor -- start 2024-04-25T10:23:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.631+0000 7fed0eb93700 1 -- start start 2024-04-25T10:23:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.631+0000 7fed0eb93700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed08103af0 0x7fed08199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.631+0000 7fed0eb93700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fed08109ab0 con 0x7fed08103af0 2024-04-25T10:23:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.631+0000 7fed0c92f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed08103af0 0x7fed08199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.631+0000 7fed0c92f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed08103af0 0x7fed08199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35244/0 (socket says 172.21.15.43:35244) 2024-04-25T10:23:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.631+0000 7fed0c92f700 1 -- 172.21.15.43:0/2740149465 learned_addr learned my addr 172.21.15.43:0/2740149465 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.631+0000 7fed0c92f700 1 -- 172.21.15.43:0/2740149465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed08199b70 con 0x7fed08103af0 2024-04-25T10:23:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.631+0000 7fed0c92f700 1 --2- 172.21.15.43:0/2740149465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed08103af0 0x7fed08199630 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0057d0 tx=0x7fecfc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.631+0000 7fed05ffb700 1 -- 172.21.15.43:0/2740149465 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecfc027070 con 0x7fed08103af0 2024-04-25T10:23:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.632+0000 7fed05ffb700 1 -- 172.21.15.43:0/2740149465 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecfc00be10 con 0x7fed08103af0 2024-04-25T10:23:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.632+0000 7fed0eb93700 1 -- 172.21.15.43:0/2740149465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed08199e00 con 0x7fed08103af0 2024-04-25T10:23:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.632+0000 7fed05ffb700 1 -- 172.21.15.43:0/2740149465 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecfc022dd0 con 0x7fed08103af0 2024-04-25T10:23:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.632+0000 7fed0eb93700 1 -- 172.21.15.43:0/2740149465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed0819ca10 con 0x7fed08103af0 2024-04-25T10:23:46.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.633+0000 7fed0eb93700 1 -- 172.21.15.43:0/2740149465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed0810a230 con 0x7fed08103af0 2024-04-25T10:23:46.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.633+0000 7fed05ffb700 1 -- 172.21.15.43:0/2740149465 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fecfc02a070 con 0x7fed08103af0 2024-04-25T10:23:46.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.633+0000 7fed05ffb700 1 --2- 172.21.15.43:0/2740149465 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fecf803d5c0 0x7fecf803fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:46.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.633+0000 7fed05ffb700 1 -- 172.21.15.43:0/2740149465 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fecfc01e030 con 0x7fed08103af0 2024-04-25T10:23:46.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.633+0000 7fed07fff700 1 --2- 172.21.15.43:0/2740149465 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fecf803d5c0 0x7fecf803fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:46.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.634+0000 7fed07fff700 1 --2- 172.21.15.43:0/2740149465 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fecf803d5c0 0x7fecf803fad0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fecf4006fd0 tx=0x7fecf4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:46.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.638+0000 7fed05ffb700 1 -- 172.21.15.43:0/2740149465 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fecfc013070 con 0x7fed08103af0 2024-04-25T10:23:46.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.849+0000 7fed0eb93700 1 -- 172.21.15.43:0/2740149465 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fed0802e620 con 0x7fecf803d5c0 2024-04-25T10:23:46.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.853+0000 7fed05ffb700 1 -- 172.21.15.43:0/2740149465 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fed0802e620 con 0x7fecf803d5c0 2024-04-25T10:23:46.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.855+0000 7fed0eb93700 1 -- 172.21.15.43:0/2740149465 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fecf803d5c0 msgr2=0x7fecf803fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:46.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.855+0000 7fed0eb93700 1 --2- 172.21.15.43:0/2740149465 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fecf803d5c0 0x7fecf803fad0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fecf4006fd0 tx=0x7fecf4006e40 comp rx=0 tx=0).stop 2024-04-25T10:23:46.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.855+0000 7fed0eb93700 1 -- 172.21.15.43:0/2740149465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed08103af0 msgr2=0x7fed08199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:46.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.855+0000 7fed0eb93700 1 --2- 172.21.15.43:0/2740149465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed08103af0 0x7fed08199630 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0057d0 tx=0x7fecfc005800 comp rx=0 tx=0).stop 2024-04-25T10:23:46.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.855+0000 7fed0eb93700 1 -- 172.21.15.43:0/2740149465 shutdown_connections 2024-04-25T10:23:46.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.855+0000 7fed0eb93700 1 --2- 172.21.15.43:0/2740149465 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fecf803d5c0 0x7fecf803fad0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:46.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.855+0000 7fed0eb93700 1 --2- 172.21.15.43:0/2740149465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed08103af0 0x7fed08199630 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:46.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.855+0000 7fed0eb93700 1 -- 172.21.15.43:0/2740149465 >> 172.21.15.43:0/2740149465 conn(0x7fed080ff130 msgr2=0x7fed080ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:46.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.855+0000 7fed0eb93700 1 -- 172.21.15.43:0/2740149465 shutdown_connections 2024-04-25T10:23:46.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:46.855+0000 7fed0eb93700 1 -- 172.21.15.43:0/2740149465 wait complete. 2024-04-25T10:23:47.584 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi077 2024-04-25T10:23:47.584 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-25T10:23:47.584 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/etc/ceph/ceph.conf 2024-04-25T10:23:47.604 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-25T10:23:47.604 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:23:47.662 INFO:tasks.cephadm:Adding host smithi077 to orchestrator... 2024-04-25T10:23:47.662 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph orch host add smithi077 2024-04-25T10:23:47.909 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:23:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:47 smithi043 ceph-mon[29478]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:23:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:47 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:23:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:47 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:23:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:47 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:23:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:47 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:23:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:47 smithi043 ceph-mon[29478]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-25T10:23:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:47 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:23:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:47 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:23:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:47 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:23:49.035 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:48 smithi043 ceph-mon[29478]: Updating smithi043:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:23:49.035 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:48 smithi043 ceph-mon[29478]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:23:49.036 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:48 smithi043 ceph-mon[29478]: Updating smithi043:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:23:49.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.567+0000 7f49c2f92700 1 -- 172.21.15.43:0/3327180310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49bc109a10 msgr2=0x7f49bc109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:49.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.567+0000 7f49c2f92700 1 --2- 172.21.15.43:0/3327180310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49bc109a10 0x7f49bc109e20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009910 tx=0x7f49b0009c20 comp rx=0 tx=0).stop 2024-04-25T10:23:49.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.567+0000 7f49c2f92700 1 -- 172.21.15.43:0/3327180310 shutdown_connections 2024-04-25T10:23:49.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.567+0000 7f49c2f92700 1 --2- 172.21.15.43:0/3327180310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49bc109a10 0x7f49bc109e20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:49.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.567+0000 7f49c2f92700 1 -- 172.21.15.43:0/3327180310 >> 172.21.15.43:0/3327180310 conn(0x7f49bc0ff560 msgr2=0x7f49bc1019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:49.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.567+0000 7f49c2f92700 1 -- 172.21.15.43:0/3327180310 shutdown_connections 2024-04-25T10:23:49.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.567+0000 7f49c2f92700 1 -- 172.21.15.43:0/3327180310 wait complete. 2024-04-25T10:23:49.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.568+0000 7f49c2f92700 1 Processor -- start 2024-04-25T10:23:49.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.568+0000 7f49c2f92700 1 -- start start 2024-04-25T10:23:49.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.568+0000 7f49c2f92700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49bc109a10 0x7f49bc075990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:49.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.568+0000 7f49c2f92700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc10df60 con 0x7f49bc109a10 2024-04-25T10:23:49.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.568+0000 7f49c0d2e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49bc109a10 0x7f49bc075990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:49.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.568+0000 7f49c0d2e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49bc109a10 0x7f49bc075990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36978/0 (socket says 172.21.15.43:36978) 2024-04-25T10:23:49.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.568+0000 7f49c0d2e700 1 -- 172.21.15.43:0/153022541 learned_addr learned my addr 172.21.15.43:0/153022541 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:49.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.569+0000 7f49c0d2e700 1 -- 172.21.15.43:0/153022541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49bc0777e0 con 0x7f49bc109a10 2024-04-25T10:23:49.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.569+0000 7f49c0d2e700 1 --2- 172.21.15.43:0/153022541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49bc109a10 0x7f49bc075990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f49b00056d0 tx=0x7f49b00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:49.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.569+0000 7f49b9ffb700 1 -- 172.21.15.43:0/153022541 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49b0027070 con 0x7f49bc109a10 2024-04-25T10:23:49.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.569+0000 7f49c2f92700 1 -- 172.21.15.43:0/153022541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49bc075ed0 con 0x7f49bc109a10 2024-04-25T10:23:49.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.569+0000 7f49b9ffb700 1 -- 172.21.15.43:0/153022541 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49b000be10 con 0x7f49bc109a10 2024-04-25T10:23:49.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.569+0000 7f49b9ffb700 1 -- 172.21.15.43:0/153022541 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49b0022d20 con 0x7f49bc109a10 2024-04-25T10:23:49.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.569+0000 7f49c2f92700 1 -- 172.21.15.43:0/153022541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49bc076390 con 0x7f49bc109a10 2024-04-25T10:23:49.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.570+0000 7f49b9ffb700 1 -- 172.21.15.43:0/153022541 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f49b002a070 con 0x7f49bc109a10 2024-04-25T10:23:49.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.570+0000 7f49c2f92700 1 -- 172.21.15.43:0/153022541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49bc105120 con 0x7f49bc109a10 2024-04-25T10:23:49.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.571+0000 7f49b9ffb700 1 --2- 172.21.15.43:0/153022541 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f49ac03d610 0x7f49ac03fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:49.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.571+0000 7f49b9ffb700 1 -- 172.21.15.43:0/153022541 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f49b001e030 con 0x7f49bc109a10 2024-04-25T10:23:49.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.571+0000 7f49bbfff700 1 --2- 172.21.15.43:0/153022541 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f49ac03d610 0x7f49ac03fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:49.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.571+0000 7f49bbfff700 1 --2- 172.21.15.43:0/153022541 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f49ac03d610 0x7f49ac03fb20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f49a8006fd0 tx=0x7f49a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:49.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.576+0000 7f49b9ffb700 1 -- 172.21.15.43:0/153022541 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49b0013070 con 0x7f49bc109a10 2024-04-25T10:23:49.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:49.788+0000 7f49c2f92700 1 -- 172.21.15.43:0/153022541 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi077", "target": ["mon-mgr", ""]}) -- 0x7f49bc076f60 con 0x7f49ac03d610 2024-04-25T10:23:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:49 smithi043 ceph-mon[29478]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi077", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:23:51.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:50 smithi043 ceph-mon[29478]: Deploying cephadm binary to smithi077 2024-04-25T10:23:53.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:53 smithi043 ceph-mon[29478]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:23:54.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:54.224+0000 7f49b9ffb700 1 -- 172.21.15.43:0/153022541 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f49bc076f60 con 0x7f49ac03d610 2024-04-25T10:23:54.227 INFO:teuthology.orchestra.run.smithi043.stdout:Added host 'smithi077' with addr '172.21.15.77' 2024-04-25T10:23:54.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:54.227+0000 7f49c2f92700 1 -- 172.21.15.43:0/153022541 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f49ac03d610 msgr2=0x7f49ac03fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:54.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:54.227+0000 7f49c2f92700 1 --2- 172.21.15.43:0/153022541 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f49ac03d610 0x7f49ac03fb20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f49a8006fd0 tx=0x7f49a8006e40 comp rx=0 tx=0).stop 2024-04-25T10:23:54.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:54.227+0000 7f49c2f92700 1 -- 172.21.15.43:0/153022541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49bc109a10 msgr2=0x7f49bc075990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:54.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:54.227+0000 7f49c2f92700 1 --2- 172.21.15.43:0/153022541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49bc109a10 0x7f49bc075990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f49b00056d0 tx=0x7f49b00057b0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:54.227+0000 7f49c2f92700 1 -- 172.21.15.43:0/153022541 shutdown_connections 2024-04-25T10:23:54.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:54.227+0000 7f49c2f92700 1 --2- 172.21.15.43:0/153022541 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f49ac03d610 0x7f49ac03fb20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:54.227+0000 7f49c2f92700 1 --2- 172.21.15.43:0/153022541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49bc109a10 0x7f49bc075990 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:54.227+0000 7f49c2f92700 1 -- 172.21.15.43:0/153022541 >> 172.21.15.43:0/153022541 conn(0x7f49bc0ff560 msgr2=0x7f49bc101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:54.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:54.227+0000 7f49c2f92700 1 -- 172.21.15.43:0/153022541 shutdown_connections 2024-04-25T10:23:54.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:54.228+0000 7f49c2f92700 1 -- 172.21.15.43:0/153022541 wait complete. 2024-04-25T10:23:54.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T10:23:55.068 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:23:55.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:55 smithi043 ceph-mon[29478]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:23:55.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:55 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:23:55.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:55 smithi043 ceph-mon[29478]: Added host smithi077 2024-04-25T10:23:55.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:55 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:23:55.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:55 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:23:56.436 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:56 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:23:56.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.641+0000 7f3a87c7e700 1 -- 172.21.15.43:0/1793888154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a80108940 msgr2=0x7f3a80108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:56.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.641+0000 7f3a87c7e700 1 --2- 172.21.15.43:0/1793888154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a80108940 0x7f3a80108d50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009910 tx=0x7f3a7c009c20 comp rx=0 tx=0).stop 2024-04-25T10:23:56.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.641+0000 7f3a87c7e700 1 -- 172.21.15.43:0/1793888154 shutdown_connections 2024-04-25T10:23:56.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.641+0000 7f3a87c7e700 1 --2- 172.21.15.43:0/1793888154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a80108940 0x7f3a80108d50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:56.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.641+0000 7f3a87c7e700 1 -- 172.21.15.43:0/1793888154 >> 172.21.15.43:0/1793888154 conn(0x7f3a80078460 msgr2=0x7f3a800788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:56.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.641+0000 7f3a87c7e700 1 -- 172.21.15.43:0/1793888154 shutdown_connections 2024-04-25T10:23:56.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.641+0000 7f3a87c7e700 1 -- 172.21.15.43:0/1793888154 wait complete. 2024-04-25T10:23:56.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.642+0000 7f3a87c7e700 1 Processor -- start 2024-04-25T10:23:56.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.642+0000 7f3a87c7e700 1 -- start start 2024-04-25T10:23:56.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.642+0000 7f3a87c7e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a80108940 0x7f3a8019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:56.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.642+0000 7f3a87c7e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8010ce90 con 0x7f3a80108940 2024-04-25T10:23:56.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.642+0000 7f3a85a1a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a80108940 0x7f3a8019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:56.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.642+0000 7f3a85a1a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a80108940 0x7f3a8019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37004/0 (socket says 172.21.15.43:37004) 2024-04-25T10:23:56.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.642+0000 7f3a85a1a700 1 -- 172.21.15.43:0/2161547247 learned_addr learned my addr 172.21.15.43:0/2161547247 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:56.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.643+0000 7f3a85a1a700 1 -- 172.21.15.43:0/2161547247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8006bbd0 con 0x7f3a80108940 2024-04-25T10:23:56.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.643+0000 7f3a85a1a700 1 --2- 172.21.15.43:0/2161547247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a80108940 0x7f3a8019a650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c0056d0 tx=0x7f3a7c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:56.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.643+0000 7f3a76ffd700 1 -- 172.21.15.43:0/2161547247 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a7c027070 con 0x7f3a80108940 2024-04-25T10:23:56.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.643+0000 7f3a76ffd700 1 -- 172.21.15.43:0/2161547247 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a7c00be10 con 0x7f3a80108940 2024-04-25T10:23:56.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.643+0000 7f3a87c7e700 1 -- 172.21.15.43:0/2161547247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8019ac50 con 0x7f3a80108940 2024-04-25T10:23:56.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.643+0000 7f3a76ffd700 1 -- 172.21.15.43:0/2161547247 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a7c022d20 con 0x7f3a80108940 2024-04-25T10:23:56.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.643+0000 7f3a87c7e700 1 -- 172.21.15.43:0/2161547247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a8019b110 con 0x7f3a80108940 2024-04-25T10:23:56.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.644+0000 7f3a87c7e700 1 -- 172.21.15.43:0/2161547247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a80104050 con 0x7f3a80108940 2024-04-25T10:23:56.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.644+0000 7f3a76ffd700 1 -- 172.21.15.43:0/2161547247 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f3a7c02a070 con 0x7f3a80108940 2024-04-25T10:23:56.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.644+0000 7f3a76ffd700 1 --2- 172.21.15.43:0/2161547247 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f3a6c03d670 0x7f3a6c03fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:56.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.644+0000 7f3a76ffd700 1 -- 172.21.15.43:0/2161547247 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3a7c01e030 con 0x7f3a80108940 2024-04-25T10:23:56.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.644+0000 7f3a85219700 1 --2- 172.21.15.43:0/2161547247 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f3a6c03d670 0x7f3a6c03fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:56.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.645+0000 7f3a85219700 1 --2- 172.21.15.43:0/2161547247 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f3a6c03d670 0x7f3a6c03fb80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3a70006fd0 tx=0x7f3a70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:56.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.654+0000 7f3a76ffd700 1 -- 172.21.15.43:0/2161547247 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a7c013070 con 0x7f3a80108940 2024-04-25T10:23:56.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.869+0000 7f3a87c7e700 1 -- 172.21.15.43:0/2161547247 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3a8006b320 con 0x7f3a6c03d670 2024-04-25T10:23:56.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.870+0000 7f3a76ffd700 1 -- 172.21.15.43:0/2161547247 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f3a8006b320 con 0x7f3a6c03d670 2024-04-25T10:23:56.873 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:23:56.873 INFO:teuthology.orchestra.run.smithi043.stdout:[{"addr": "172.21.15.43", "hostname": "smithi043", "labels": [], "status": ""}, {"addr": "172.21.15.77", "hostname": "smithi077", "labels": [], "status": ""}] 2024-04-25T10:23:56.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.873+0000 7f3a87c7e700 1 -- 172.21.15.43:0/2161547247 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f3a6c03d670 msgr2=0x7f3a6c03fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:56.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.873+0000 7f3a87c7e700 1 --2- 172.21.15.43:0/2161547247 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f3a6c03d670 0x7f3a6c03fb80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3a70006fd0 tx=0x7f3a70006e40 comp rx=0 tx=0).stop 2024-04-25T10:23:56.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.873+0000 7f3a87c7e700 1 -- 172.21.15.43:0/2161547247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a80108940 msgr2=0x7f3a8019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:56.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.873+0000 7f3a87c7e700 1 --2- 172.21.15.43:0/2161547247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a80108940 0x7f3a8019a650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c0056d0 tx=0x7f3a7c0057b0 comp rx=0 tx=0).stop 2024-04-25T10:23:56.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.874+0000 7f3a87c7e700 1 -- 172.21.15.43:0/2161547247 shutdown_connections 2024-04-25T10:23:56.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.874+0000 7f3a87c7e700 1 --2- 172.21.15.43:0/2161547247 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f3a6c03d670 0x7f3a6c03fb80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:56.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.874+0000 7f3a87c7e700 1 --2- 172.21.15.43:0/2161547247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a80108940 0x7f3a8019a650 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:56.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.874+0000 7f3a87c7e700 1 -- 172.21.15.43:0/2161547247 >> 172.21.15.43:0/2161547247 conn(0x7f3a80078460 msgr2=0x7f3a800ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:56.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.874+0000 7f3a87c7e700 1 -- 172.21.15.43:0/2161547247 shutdown_connections 2024-04-25T10:23:56.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:56.874+0000 7f3a87c7e700 1 -- 172.21.15.43:0/2161547247 wait complete. 2024-04-25T10:23:57.496 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi164 2024-04-25T10:23:57.497 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T10:23:57.497 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/etc/ceph/ceph.conf 2024-04-25T10:23:57.515 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T10:23:57.515 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:23:57.574 INFO:tasks.cephadm:Adding host smithi164 to orchestrator... 2024-04-25T10:23:57.574 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph orch host add smithi164 2024-04-25T10:23:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:23:58.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:57 smithi043 ceph-mon[29478]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:23:58.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:57 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:23:58.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:57 smithi043 ceph-mon[29478]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:23:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.255+0000 7fc89a620700 1 -- 172.21.15.43:0/2705176419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc894102320 msgr2=0x7fc894102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.255+0000 7fc89a620700 1 --2- 172.21.15.43:0/2705176419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc894102320 0x7fc894102730 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc884009910 tx=0x7fc884009c20 comp rx=0 tx=0).stop 2024-04-25T10:23:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.255+0000 7fc89a620700 1 -- 172.21.15.43:0/2705176419 shutdown_connections 2024-04-25T10:23:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.255+0000 7fc89a620700 1 --2- 172.21.15.43:0/2705176419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc894102320 0x7fc894102730 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.255+0000 7fc89a620700 1 -- 172.21.15.43:0/2705176419 >> 172.21.15.43:0/2705176419 conn(0x7fc8940fdd00 msgr2=0x7fc894100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.255+0000 7fc89a620700 1 -- 172.21.15.43:0/2705176419 shutdown_connections 2024-04-25T10:23:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.255+0000 7fc89a620700 1 -- 172.21.15.43:0/2705176419 wait complete. 2024-04-25T10:23:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.256+0000 7fc89a620700 1 Processor -- start 2024-04-25T10:23:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.256+0000 7fc89a620700 1 -- start start 2024-04-25T10:23:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.256+0000 7fc89a620700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc894102320 0x7fc89419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.256+0000 7fc89a620700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc89410c610 con 0x7fc894102320 2024-04-25T10:23:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.256+0000 7fc893fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc894102320 0x7fc89419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.256+0000 7fc893fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc894102320 0x7fc89419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56598/0 (socket says 172.21.15.43:56598) 2024-04-25T10:23:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.256+0000 7fc893fff700 1 -- 172.21.15.43:0/347227849 learned_addr learned my addr 172.21.15.43:0/347227849 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:23:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.257+0000 7fc893fff700 1 -- 172.21.15.43:0/347227849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc894108d80 con 0x7fc894102320 2024-04-25T10:23:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.257+0000 7fc893fff700 1 --2- 172.21.15.43:0/347227849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc894102320 0x7fc89419a620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc8840056d0 tx=0x7fc8840057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.257+0000 7fc8917fa700 1 -- 172.21.15.43:0/347227849 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc884026070 con 0x7fc894102320 2024-04-25T10:23:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.257+0000 7fc8917fa700 1 -- 172.21.15.43:0/347227849 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc88400bc00 con 0x7fc894102320 2024-04-25T10:23:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.257+0000 7fc89a620700 1 -- 172.21.15.43:0/347227849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89419ac20 con 0x7fc894102320 2024-04-25T10:23:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.257+0000 7fc8917fa700 1 -- 172.21.15.43:0/347227849 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc884003920 con 0x7fc894102320 2024-04-25T10:23:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.257+0000 7fc89a620700 1 -- 172.21.15.43:0/347227849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc89419b0e0 con 0x7fc894102320 2024-04-25T10:23:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.258+0000 7fc89a620700 1 -- 172.21.15.43:0/347227849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8941037d0 con 0x7fc894102320 2024-04-25T10:23:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.258+0000 7fc8917fa700 1 -- 172.21.15.43:0/347227849 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fc884029070 con 0x7fc894102320 2024-04-25T10:23:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.259+0000 7fc8917fa700 1 --2- 172.21.15.43:0/347227849 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc87c03d610 0x7fc87c03fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.259+0000 7fc8917fa700 1 -- 172.21.15.43:0/347227849 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc88401e070 con 0x7fc894102320 2024-04-25T10:23:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.259+0000 7fc8937fe700 1 --2- 172.21.15.43:0/347227849 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc87c03d610 0x7fc87c03fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.259+0000 7fc8937fe700 1 --2- 172.21.15.43:0/347227849 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc87c03d610 0x7fc87c03fb20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc888006fd0 tx=0x7fc888006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:59.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.264+0000 7fc8917fa700 1 -- 172.21.15.43:0/347227849 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc894108d80 con 0x7fc894102320 2024-04-25T10:23:59.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:23:59.476+0000 7fc89a620700 1 -- 172.21.15.43:0/347227849 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}) -- 0x7fc8941084d0 con 0x7fc87c03d610 2024-04-25T10:24:00.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:23:59 smithi043 ceph-mon[29478]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:01.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:00 smithi043 ceph-mon[29478]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:24:01.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:00 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:01.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:00 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:01.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:00 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:01.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:00 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:01.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:00 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi077", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:24:01.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:00 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:01.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:00 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:02.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:01 smithi043 ceph-mon[29478]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-25T10:24:02.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:01 smithi043 ceph-mon[29478]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:02.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:01 smithi043 ceph-mon[29478]: Deploying cephadm binary to smithi164 2024-04-25T10:24:02.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:01 smithi043 ceph-mon[29478]: Updating smithi077:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:24:02.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:01 smithi043 ceph-mon[29478]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:24:03.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:03.740+0000 7fc8917fa700 1 -- 172.21.15.43:0/347227849 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fc8941084d0 con 0x7fc87c03d610 2024-04-25T10:24:03.743 INFO:teuthology.orchestra.run.smithi043.stdout:Added host 'smithi164' with addr '172.21.15.164' 2024-04-25T10:24:03.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:03.743+0000 7fc89a620700 1 -- 172.21.15.43:0/347227849 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc87c03d610 msgr2=0x7fc87c03fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:03.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:03.743+0000 7fc89a620700 1 --2- 172.21.15.43:0/347227849 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc87c03d610 0x7fc87c03fb20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc888006fd0 tx=0x7fc888006e40 comp rx=0 tx=0).stop 2024-04-25T10:24:03.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:03.743+0000 7fc89a620700 1 -- 172.21.15.43:0/347227849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc894102320 msgr2=0x7fc89419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:03.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:03.743+0000 7fc89a620700 1 --2- 172.21.15.43:0/347227849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc894102320 0x7fc89419a620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc8840056d0 tx=0x7fc8840057b0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:03.743+0000 7fc89a620700 1 -- 172.21.15.43:0/347227849 shutdown_connections 2024-04-25T10:24:03.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:03.743+0000 7fc89a620700 1 --2- 172.21.15.43:0/347227849 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc87c03d610 0x7fc87c03fb20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:03.743+0000 7fc89a620700 1 --2- 172.21.15.43:0/347227849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc894102320 0x7fc89419a620 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:03.743+0000 7fc89a620700 1 -- 172.21.15.43:0/347227849 >> 172.21.15.43:0/347227849 conn(0x7fc8940fdd00 msgr2=0x7fc89410a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:03.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:03.743+0000 7fc89a620700 1 -- 172.21.15.43:0/347227849 shutdown_connections 2024-04-25T10:24:03.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:03.743+0000 7fc89a620700 1 -- 172.21.15.43:0/347227849 wait complete. 2024-04-25T10:24:04.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:03 smithi043 ceph-mon[29478]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:04.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:04.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:04.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:04.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:04.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:24:04.421 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T10:24:04.665 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:24:05.085 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:04 smithi043 ceph-mon[29478]: Updating smithi077:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:24:05.086 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:04 smithi043 ceph-mon[29478]: Added host smithi164 2024-04-25T10:24:05.086 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:04 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:06.058 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:05 smithi043 ceph-mon[29478]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:06.058 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:05 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:06.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.088+0000 7f2b125e0700 1 -- 172.21.15.43:0/2388853903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b0c109a40 msgr2=0x7f2b0c109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:06.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.088+0000 7f2b125e0700 1 --2- 172.21.15.43:0/2388853903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b0c109a40 0x7f2b0c109e50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009910 tx=0x7f2afc009c20 comp rx=0 tx=0).stop 2024-04-25T10:24:06.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.088+0000 7f2b125e0700 1 -- 172.21.15.43:0/2388853903 shutdown_connections 2024-04-25T10:24:06.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.088+0000 7f2b125e0700 1 --2- 172.21.15.43:0/2388853903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b0c109a40 0x7f2b0c109e50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:06.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.088+0000 7f2b125e0700 1 -- 172.21.15.43:0/2388853903 >> 172.21.15.43:0/2388853903 conn(0x7f2b0c0ff570 msgr2=0x7f2b0c1019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:06.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.088+0000 7f2b125e0700 1 -- 172.21.15.43:0/2388853903 shutdown_connections 2024-04-25T10:24:06.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.088+0000 7f2b125e0700 1 -- 172.21.15.43:0/2388853903 wait complete. 2024-04-25T10:24:06.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.089+0000 7f2b125e0700 1 Processor -- start 2024-04-25T10:24:06.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.089+0000 7f2b125e0700 1 -- start start 2024-04-25T10:24:06.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.089+0000 7f2b125e0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b0c109a40 0x7f2b0c075990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:06.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.089+0000 7f2b125e0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c10df90 con 0x7f2b0c109a40 2024-04-25T10:24:06.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.089+0000 7f2b0bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b0c109a40 0x7f2b0c075990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:06.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.089+0000 7f2b0bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b0c109a40 0x7f2b0c075990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56618/0 (socket says 172.21.15.43:56618) 2024-04-25T10:24:06.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.089+0000 7f2b0bfff700 1 -- 172.21.15.43:0/2198694138 learned_addr learned my addr 172.21.15.43:0/2198694138 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:24:06.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.090+0000 7f2b0bfff700 1 -- 172.21.15.43:0/2198694138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0c0777e0 con 0x7f2b0c109a40 2024-04-25T10:24:06.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.090+0000 7f2b0bfff700 1 --2- 172.21.15.43:0/2198694138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b0c109a40 0x7f2b0c075990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0092e0 tx=0x7f2afc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:06.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.090+0000 7f2b097fa700 1 -- 172.21.15.43:0/2198694138 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2afc026070 con 0x7f2b0c109a40 2024-04-25T10:24:06.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.090+0000 7f2b125e0700 1 -- 172.21.15.43:0/2198694138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0c075ed0 con 0x7f2b0c109a40 2024-04-25T10:24:06.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.090+0000 7f2b097fa700 1 -- 172.21.15.43:0/2198694138 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2afc00b6f0 con 0x7f2b0c109a40 2024-04-25T10:24:06.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.090+0000 7f2b125e0700 1 -- 172.21.15.43:0/2198694138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b0c076390 con 0x7f2b0c109a40 2024-04-25T10:24:06.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.090+0000 7f2b097fa700 1 -- 172.21.15.43:0/2198694138 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2afc0037f0 con 0x7f2b0c109a40 2024-04-25T10:24:06.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.091+0000 7f2b097fa700 1 -- 172.21.15.43:0/2198694138 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f2afc029070 con 0x7f2b0c109a40 2024-04-25T10:24:06.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.091+0000 7f2b097fa700 1 --2- 172.21.15.43:0/2198694138 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2af403d610 0x7f2af403fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:06.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.091+0000 7f2b097fa700 1 -- 172.21.15.43:0/2198694138 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2afc01e070 con 0x7f2b0c109a40 2024-04-25T10:24:06.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.091+0000 7f2b125e0700 1 -- 172.21.15.43:0/2198694138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b0c105150 con 0x7f2b0c109a40 2024-04-25T10:24:06.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.091+0000 7f2b0b7fe700 1 --2- 172.21.15.43:0/2198694138 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2af403d610 0x7f2af403fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:06.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.092+0000 7f2b0b7fe700 1 --2- 172.21.15.43:0/2198694138 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2af403d610 0x7f2af403fb20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2b00006fd0 tx=0x7f2b00006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:06.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.098+0000 7f2b097fa700 1 -- 172.21.15.43:0/2198694138 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2afc013070 con 0x7f2b0c109a40 2024-04-25T10:24:06.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.319+0000 7f2b125e0700 1 -- 172.21.15.43:0/2198694138 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2b0c076ec0 con 0x7f2af403d610 2024-04-25T10:24:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.320+0000 7f2b097fa700 1 -- 172.21.15.43:0/2198694138 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f2b0c076ec0 con 0x7f2af403d610 2024-04-25T10:24:06.323 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:24:06.323 INFO:teuthology.orchestra.run.smithi043.stdout:[{"addr": "172.21.15.43", "hostname": "smithi043", "labels": [], "status": ""}, {"addr": "172.21.15.77", "hostname": "smithi077", "labels": [], "status": ""}, {"addr": "172.21.15.164", "hostname": "smithi164", "labels": [], "status": ""}] 2024-04-25T10:24:06.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.323+0000 7f2b125e0700 1 -- 172.21.15.43:0/2198694138 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2af403d610 msgr2=0x7f2af403fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:06.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.323+0000 7f2b125e0700 1 --2- 172.21.15.43:0/2198694138 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2af403d610 0x7f2af403fb20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2b00006fd0 tx=0x7f2b00006e40 comp rx=0 tx=0).stop 2024-04-25T10:24:06.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.323+0000 7f2b125e0700 1 -- 172.21.15.43:0/2198694138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b0c109a40 msgr2=0x7f2b0c075990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:06.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.323+0000 7f2b125e0700 1 --2- 172.21.15.43:0/2198694138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b0c109a40 0x7f2b0c075990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0092e0 tx=0x7f2afc005780 comp rx=0 tx=0).stop 2024-04-25T10:24:06.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.323+0000 7f2b125e0700 1 -- 172.21.15.43:0/2198694138 shutdown_connections 2024-04-25T10:24:06.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.323+0000 7f2b125e0700 1 --2- 172.21.15.43:0/2198694138 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2af403d610 0x7f2af403fb20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:06.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.323+0000 7f2b125e0700 1 --2- 172.21.15.43:0/2198694138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b0c109a40 0x7f2b0c075990 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:06.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.323+0000 7f2b125e0700 1 -- 172.21.15.43:0/2198694138 >> 172.21.15.43:0/2198694138 conn(0x7f2b0c0ff570 msgr2=0x7f2b0c1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:06.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.323+0000 7f2b125e0700 1 -- 172.21.15.43:0/2198694138 shutdown_connections 2024-04-25T10:24:06.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:06.323+0000 7f2b125e0700 1 -- 172.21.15.43:0/2198694138 wait complete. 2024-04-25T10:24:06.942 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T10:24:06.942 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T10:24:07.198 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:24:07.772 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:07 smithi043 ceph-mon[29478]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:07.772 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:07 smithi043 ceph-mon[29478]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:24:07.773 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:07 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:08.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.661+0000 7f9c85375700 1 -- 172.21.15.43:0/2471349603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c80109a30 msgr2=0x7f9c80109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:08.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.661+0000 7f9c85375700 1 --2- 172.21.15.43:0/2471349603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c80109a30 0x7f9c80109e40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9c70009910 tx=0x7f9c70009c20 comp rx=0 tx=0).stop 2024-04-25T10:24:08.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.661+0000 7f9c85375700 1 -- 172.21.15.43:0/2471349603 shutdown_connections 2024-04-25T10:24:08.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.661+0000 7f9c85375700 1 --2- 172.21.15.43:0/2471349603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c80109a30 0x7f9c80109e40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:08.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.661+0000 7f9c85375700 1 -- 172.21.15.43:0/2471349603 >> 172.21.15.43:0/2471349603 conn(0x7f9c800ff540 msgr2=0x7f9c80101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:08.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.661+0000 7f9c85375700 1 -- 172.21.15.43:0/2471349603 shutdown_connections 2024-04-25T10:24:08.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.661+0000 7f9c85375700 1 -- 172.21.15.43:0/2471349603 wait complete. 2024-04-25T10:24:08.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.662+0000 7f9c85375700 1 Processor -- start 2024-04-25T10:24:08.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.662+0000 7f9c85375700 1 -- start start 2024-04-25T10:24:08.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.662+0000 7f9c85375700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c80109a30 0x7f9c8019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:08.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.662+0000 7f9c85375700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8010df80 con 0x7f9c80109a30 2024-04-25T10:24:08.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.663+0000 7f9c7effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c80109a30 0x7f9c8019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:08.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.663+0000 7f9c7effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c80109a30 0x7f9c8019f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56642/0 (socket says 172.21.15.43:56642) 2024-04-25T10:24:08.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.663+0000 7f9c7effd700 1 -- 172.21.15.43:0/1593744685 learned_addr learned my addr 172.21.15.43:0/1593744685 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:24:08.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.663+0000 7f9c7effd700 1 -- 172.21.15.43:0/1593744685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8019fa20 con 0x7f9c80109a30 2024-04-25T10:24:08.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.663+0000 7f9c7effd700 1 --2- 172.21.15.43:0/1593744685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c80109a30 0x7f9c8019f4e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9c700056d0 tx=0x7f9c700057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:08.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.663+0000 7f9c67fff700 1 -- 172.21.15.43:0/1593744685 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c70027070 con 0x7f9c80109a30 2024-04-25T10:24:08.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.663+0000 7f9c67fff700 1 -- 172.21.15.43:0/1593744685 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c7000be10 con 0x7f9c80109a30 2024-04-25T10:24:08.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.663+0000 7f9c85375700 1 -- 172.21.15.43:0/1593744685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8019fd10 con 0x7f9c80109a30 2024-04-25T10:24:08.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.663+0000 7f9c67fff700 1 -- 172.21.15.43:0/1593744685 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c70022d20 con 0x7f9c80109a30 2024-04-25T10:24:08.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.663+0000 7f9c85375700 1 -- 172.21.15.43:0/1593744685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c80199840 con 0x7f9c80109a30 2024-04-25T10:24:08.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.664+0000 7f9c85375700 1 -- 172.21.15.43:0/1593744685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c80105140 con 0x7f9c80109a30 2024-04-25T10:24:08.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.664+0000 7f9c67fff700 1 -- 172.21.15.43:0/1593744685 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f9c7002a070 con 0x7f9c80109a30 2024-04-25T10:24:08.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.665+0000 7f9c67fff700 1 --2- 172.21.15.43:0/1593744685 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9c6803d610 0x7f9c6803fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:08.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.665+0000 7f9c67fff700 1 -- 172.21.15.43:0/1593744685 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9c7001e030 con 0x7f9c80109a30 2024-04-25T10:24:08.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.665+0000 7f9c7e7fc700 1 --2- 172.21.15.43:0/1593744685 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9c6803d610 0x7f9c6803fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:08.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.665+0000 7f9c7e7fc700 1 --2- 172.21.15.43:0/1593744685 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9c6803d610 0x7f9c6803fb20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9c74006fd0 tx=0x7f9c74006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:08.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.670+0000 7f9c67fff700 1 -- 172.21.15.43:0/1593744685 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9c70013070 con 0x7f9c80109a30 2024-04-25T10:24:08.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:08.882+0000 7f9c85375700 1 -- 172.21.15.43:0/1593744685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f9c8019ab80 con 0x7f9c80109a30 2024-04-25T10:24:09.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:09.542+0000 7f9c67fff700 1 -- 172.21.15.43:0/1593744685 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f9c7000d340 con 0x7f9c80109a30 2024-04-25T10:24:09.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:09.545+0000 7f9c85375700 1 -- 172.21.15.43:0/1593744685 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9c6803d610 msgr2=0x7f9c6803fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:09.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:09.545+0000 7f9c85375700 1 --2- 172.21.15.43:0/1593744685 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9c6803d610 0x7f9c6803fb20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9c74006fd0 tx=0x7f9c74006e40 comp rx=0 tx=0).stop 2024-04-25T10:24:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:09.545+0000 7f9c85375700 1 -- 172.21.15.43:0/1593744685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c80109a30 msgr2=0x7f9c8019f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:09.545+0000 7f9c85375700 1 --2- 172.21.15.43:0/1593744685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c80109a30 0x7f9c8019f4e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9c700056d0 tx=0x7f9c700057b0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:09.545+0000 7f9c85375700 1 -- 172.21.15.43:0/1593744685 shutdown_connections 2024-04-25T10:24:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:09.545+0000 7f9c85375700 1 --2- 172.21.15.43:0/1593744685 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9c6803d610 0x7f9c6803fb20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:09.545+0000 7f9c85375700 1 --2- 172.21.15.43:0/1593744685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c80109a30 0x7f9c8019f4e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:09.545+0000 7f9c85375700 1 -- 172.21.15.43:0/1593744685 >> 172.21.15.43:0/1593744685 conn(0x7f9c800ff540 msgr2=0x7f9c801004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:09.545+0000 7f9c85375700 1 -- 172.21.15.43:0/1593744685 shutdown_connections 2024-04-25T10:24:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:09.545+0000 7f9c85375700 1 -- 172.21.15.43:0/1593744685 wait complete. 2024-04-25T10:24:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:adjusted tunables profile to default 2024-04-25T10:24:09.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:09 smithi043 ceph-mon[29478]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:09.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:09 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1593744685' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T10:24:09.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:09.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:09.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:09.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:09.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:24:09.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:09.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:10.189 INFO:tasks.cephadm:Adding mon.a on smithi043 2024-04-25T10:24:10.190 INFO:tasks.cephadm:Adding mon.b on smithi077 2024-04-25T10:24:10.190 INFO:tasks.cephadm:Adding mon.c on smithi164 2024-04-25T10:24:10.190 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi043:172.21.15.43=a;smithi077:172.21.15.77=b;smithi164:172.21.15.164=c' 2024-04-25T10:24:10.425 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:24:10.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:10 smithi043 ceph-mon[29478]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-25T10:24:10.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:10 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1593744685' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T10:24:10.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:10 smithi043 ceph-mon[29478]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T10:24:10.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:10 smithi043 ceph-mon[29478]: Updating smithi164:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:24:10.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:10 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:10.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:10 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:10.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:10 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:11.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.833+0000 7f137e347700 1 -- 172.21.15.164:0/3356398278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1378107960 msgr2=0x7f1378107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:11.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.833+0000 7f137e347700 1 --2- 172.21.15.164:0/3356398278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1378107960 0x7f1378107d70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1368009910 tx=0x7f1368009c20 comp rx=0 tx=0).stop 2024-04-25T10:24:11.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.834+0000 7f137e347700 1 -- 172.21.15.164:0/3356398278 shutdown_connections 2024-04-25T10:24:11.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.834+0000 7f137e347700 1 --2- 172.21.15.164:0/3356398278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1378107960 0x7f1378107d70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:11.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.834+0000 7f137e347700 1 -- 172.21.15.164:0/3356398278 >> 172.21.15.164:0/3356398278 conn(0x7f13780fdd00 msgr2=0x7f1378100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:11.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.834+0000 7f137e347700 1 -- 172.21.15.164:0/3356398278 shutdown_connections 2024-04-25T10:24:11.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.834+0000 7f137e347700 1 -- 172.21.15.164:0/3356398278 wait complete. 2024-04-25T10:24:11.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.834+0000 7f137e347700 1 Processor -- start 2024-04-25T10:24:11.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.834+0000 7f137e347700 1 -- start start 2024-04-25T10:24:11.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.835+0000 7f137e347700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1378107960 0x7f137819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:11.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.835+0000 7f137e347700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f137810cef0 con 0x7f1378107960 2024-04-25T10:24:11.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.835+0000 7f1377fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1378107960 0x7f137819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:11.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.835+0000 7f1377fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1378107960 0x7f137819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.164:54640/0 (socket says 172.21.15.164:54640) 2024-04-25T10:24:11.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.835+0000 7f1377fff700 1 -- 172.21.15.164:0/2567919796 learned_addr learned my addr 172.21.15.164:0/2567919796 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-25T10:24:11.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.835+0000 7f1377fff700 1 -- 172.21.15.164:0/2567919796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1378102e20 con 0x7f1378107960 2024-04-25T10:24:11.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.835+0000 7f1377fff700 1 --2- 172.21.15.164:0/2567919796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1378107960 0x7f137819a650 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f136800b5c0 tx=0x7f1368005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:11.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.836+0000 7f13757fa700 1 -- 172.21.15.164:0/2567919796 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1368026070 con 0x7f1378107960 2024-04-25T10:24:11.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.836+0000 7f137e347700 1 -- 172.21.15.164:0/2567919796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f137819abf0 con 0x7f1378107960 2024-04-25T10:24:11.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.836+0000 7f13757fa700 1 -- 172.21.15.164:0/2567919796 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f136800b750 con 0x7f1378107960 2024-04-25T10:24:11.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.836+0000 7f13757fa700 1 -- 172.21.15.164:0/2567919796 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13680037d0 con 0x7f1378107960 2024-04-25T10:24:11.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.836+0000 7f137e347700 1 -- 172.21.15.164:0/2567919796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f137819b0b0 con 0x7f1378107960 2024-04-25T10:24:11.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.837+0000 7f137e347700 1 -- 172.21.15.164:0/2567919796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1378104240 con 0x7f1378107960 2024-04-25T10:24:11.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.837+0000 7f13757fa700 1 -- 172.21.15.164:0/2567919796 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f1368029070 con 0x7f1378107960 2024-04-25T10:24:11.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.837+0000 7f13757fa700 1 --2- 172.21.15.164:0/2567919796 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f136003d610 0x7f136003fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:11.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.837+0000 7f13757fa700 1 -- 172.21.15.164:0/2567919796 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f136801e070 con 0x7f1378107960 2024-04-25T10:24:11.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.837+0000 7f13777fe700 1 --2- 172.21.15.164:0/2567919796 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f136003d610 0x7f136003fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:11.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.838+0000 7f13777fe700 1 --2- 172.21.15.164:0/2567919796 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f136003d610 0x7f136003fb20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f136c006fd0 tx=0x7f136c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:11.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:11.843+0000 7f13757fa700 1 -- 172.21.15.164:0/2567919796 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1368013070 con 0x7f1378107960 2024-04-25T10:24:11.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:11 smithi043 ceph-mon[29478]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:24:11.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:11 smithi043 ceph-mon[29478]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:11.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:11 smithi043 ceph-mon[29478]: Updating smithi164:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:24:12.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:12.056+0000 7f137e347700 1 -- 172.21.15.164:0/2567919796 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi077:172.21.15.77=b;smithi164:172.21.15.164=c", "target": ["mon-mgr", ""]}) -- 0x7f137819b330 con 0x7f136003d610 2024-04-25T10:24:12.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:12.064+0000 7f13757fa700 1 -- 172.21.15.164:0/2567919796 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f137819b330 con 0x7f136003d610 2024-04-25T10:24:12.067 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled mon update... 2024-04-25T10:24:12.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:12.065+0000 7f137e347700 1 -- 172.21.15.164:0/2567919796 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f136003d610 msgr2=0x7f136003fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:12.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:12.065+0000 7f137e347700 1 --2- 172.21.15.164:0/2567919796 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f136003d610 0x7f136003fb20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f136c006fd0 tx=0x7f136c006e40 comp rx=0 tx=0).stop 2024-04-25T10:24:12.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:12.065+0000 7f137e347700 1 -- 172.21.15.164:0/2567919796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1378107960 msgr2=0x7f137819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:12.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:12.065+0000 7f137e347700 1 --2- 172.21.15.164:0/2567919796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1378107960 0x7f137819a650 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f136800b5c0 tx=0x7f1368005780 comp rx=0 tx=0).stop 2024-04-25T10:24:12.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:12.065+0000 7f137e347700 1 -- 172.21.15.164:0/2567919796 shutdown_connections 2024-04-25T10:24:12.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:12.065+0000 7f137e347700 1 --2- 172.21.15.164:0/2567919796 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f136003d610 0x7f136003fb20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:12.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:12.065+0000 7f137e347700 1 --2- 172.21.15.164:0/2567919796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1378107960 0x7f137819a650 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:12.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:12.065+0000 7f137e347700 1 -- 172.21.15.164:0/2567919796 >> 172.21.15.164:0/2567919796 conn(0x7f13780fdd00 msgr2=0x7f1378104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:12.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:12.065+0000 7f137e347700 1 -- 172.21.15.164:0/2567919796 shutdown_connections 2024-04-25T10:24:12.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:12.065+0000 7f137e347700 1 -- 172.21.15.164:0/2567919796 wait complete. 2024-04-25T10:24:12.676 DEBUG:teuthology.orchestra.run.smithi077:mon.b> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T10:24:12.679 DEBUG:teuthology.orchestra.run.smithi164:mon.c> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T10:24:12.681 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T10:24:12.681 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T10:24:13.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:13 smithi043 ceph-mon[29478]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi077:172.21.15.77=b;smithi164:172.21.15.164=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:24:13.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:13 smithi043 ceph-mon[29478]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi077:172.21.15.77=b;smithi164:172.21.15.164=c;count:3 2024-04-25T10:24:13.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:13 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:13.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:13 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:24:13.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:13 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:13.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:13 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:13.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:13 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:13.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:13 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:24:13.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:13 smithi043 ceph-mon[29478]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:13.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:13 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:13.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:13 smithi043 ceph-mon[29478]: Deploying daemon mon.c on smithi164 2024-04-25T10:24:13.464 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:24:15.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:15 smithi043 ceph-mon[29478]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:16.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.546+0000 7f33794a0700 1 -- 172.21.15.164:0/1266669628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3374108930 msgr2=0x7f3374108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:16.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.546+0000 7f33794a0700 1 --2- 172.21.15.164:0/1266669628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3374108930 0x7f3374108d40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3364009910 tx=0x7f3364009c20 comp rx=0 tx=0).stop 2024-04-25T10:24:16.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.546+0000 7f33794a0700 1 -- 172.21.15.164:0/1266669628 shutdown_connections 2024-04-25T10:24:16.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.546+0000 7f33794a0700 1 --2- 172.21.15.164:0/1266669628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3374108930 0x7f3374108d40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:16.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.546+0000 7f33794a0700 1 -- 172.21.15.164:0/1266669628 >> 172.21.15.164:0/1266669628 conn(0x7f33740fdd00 msgr2=0x7f3374100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:16.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.546+0000 7f33794a0700 1 -- 172.21.15.164:0/1266669628 shutdown_connections 2024-04-25T10:24:16.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.546+0000 7f33794a0700 1 -- 172.21.15.164:0/1266669628 wait complete. 2024-04-25T10:24:16.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.546+0000 7f33794a0700 1 Processor -- start 2024-04-25T10:24:16.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.547+0000 7f33794a0700 1 -- start start 2024-04-25T10:24:16.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.547+0000 7f33794a0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3374108930 0x7f337419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:16.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.547+0000 7f33794a0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f337410ce80 con 0x7f3374108930 2024-04-25T10:24:16.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.547+0000 7f3372ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3374108930 0x7f337419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:16.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.547+0000 7f3372ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3374108930 0x7f337419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.164:55292/0 (socket says 172.21.15.164:55292) 2024-04-25T10:24:16.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.547+0000 7f3372ffd700 1 -- 172.21.15.164:0/2634654589 learned_addr learned my addr 172.21.15.164:0/2634654589 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-25T10:24:16.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.547+0000 7f3372ffd700 1 -- 172.21.15.164:0/2634654589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337419abb0 con 0x7f3374108930 2024-04-25T10:24:16.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.548+0000 7f3372ffd700 1 --2- 172.21.15.164:0/2634654589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3374108930 0x7f337419a670 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f33640092e0 tx=0x7f3364005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:16.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.548+0000 7f335bfff700 1 -- 172.21.15.164:0/2634654589 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3364026070 con 0x7f3374108930 2024-04-25T10:24:16.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.548+0000 7f335bfff700 1 -- 172.21.15.164:0/2634654589 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f336400b6f0 con 0x7f3374108930 2024-04-25T10:24:16.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.548+0000 7f33794a0700 1 -- 172.21.15.164:0/2634654589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f337419aea0 con 0x7f3374108930 2024-04-25T10:24:16.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.548+0000 7f335bfff700 1 -- 172.21.15.164:0/2634654589 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33640037f0 con 0x7f3374108930 2024-04-25T10:24:16.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.548+0000 7f33794a0700 1 -- 172.21.15.164:0/2634654589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f337419b360 con 0x7f3374108930 2024-04-25T10:24:16.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.549+0000 7f33794a0700 1 -- 172.21.15.164:0/2634654589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3374104180 con 0x7f3374108930 2024-04-25T10:24:16.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.549+0000 7f335bfff700 1 -- 172.21.15.164:0/2634654589 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f3364029070 con 0x7f3374108930 2024-04-25T10:24:16.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.549+0000 7f335bfff700 1 --2- 172.21.15.164:0/2634654589 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f335c03d610 0x7f335c03fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:16.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.549+0000 7f335bfff700 1 -- 172.21.15.164:0/2634654589 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f336401e070 con 0x7f3374108930 2024-04-25T10:24:16.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.549+0000 7f33727fc700 1 --2- 172.21.15.164:0/2634654589 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f335c03d610 0x7f335c03fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:16.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.550+0000 7f33727fc700 1 --2- 172.21.15.164:0/2634654589 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f335c03d610 0x7f335c03fb20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3368006fd0 tx=0x7f3368006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:16.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.556+0000 7f335bfff700 1 -- 172.21.15.164:0/2634654589 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3364013070 con 0x7f3374108930 2024-04-25T10:24:16.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.860+0000 7f33794a0700 1 -- 172.21.15.164:0/2634654589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f33741024a0 con 0x7f3374108930 2024-04-25T10:24:16.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.860+0000 7f335bfff700 1 -- 172.21.15.164:0/2634654589 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f3364030090 con 0x7f3374108930 2024-04-25T10:24:16.864 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T10:24:16.864 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":1,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","modified":"2024-04-25T10:22:33.876396Z","created":"2024-04-25T10:22:33.876396Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T10:24:16.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.863+0000 7f33794a0700 1 -- 172.21.15.164:0/2634654589 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f335c03d610 msgr2=0x7f335c03fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:16.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.863+0000 7f33794a0700 1 --2- 172.21.15.164:0/2634654589 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f335c03d610 0x7f335c03fb20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3368006fd0 tx=0x7f3368006e40 comp rx=0 tx=0).stop 2024-04-25T10:24:16.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.863+0000 7f33794a0700 1 -- 172.21.15.164:0/2634654589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3374108930 msgr2=0x7f337419a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:16.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.863+0000 7f33794a0700 1 --2- 172.21.15.164:0/2634654589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3374108930 0x7f337419a670 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f33640092e0 tx=0x7f3364005780 comp rx=0 tx=0).stop 2024-04-25T10:24:16.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.863+0000 7f33794a0700 1 -- 172.21.15.164:0/2634654589 shutdown_connections 2024-04-25T10:24:16.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.863+0000 7f33794a0700 1 --2- 172.21.15.164:0/2634654589 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f335c03d610 0x7f335c03fb20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:16.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.863+0000 7f33794a0700 1 --2- 172.21.15.164:0/2634654589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3374108930 0x7f337419a670 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:16.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.863+0000 7f33794a0700 1 -- 172.21.15.164:0/2634654589 >> 172.21.15.164:0/2634654589 conn(0x7f33740fdd00 msgr2=0x7f33740fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:16.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.863+0000 7f33794a0700 1 -- 172.21.15.164:0/2634654589 shutdown_connections 2024-04-25T10:24:16.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:16.863+0000 7f33794a0700 1 -- 172.21.15.164:0/2634654589 wait complete. 2024-04-25T10:24:16.867 INFO:teuthology.orchestra.run.smithi164.stderr:dumped monmap epoch 1 2024-04-25T10:24:17.165 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:16 smithi164 systemd[1]: Starting Ceph mon.c for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:24:17.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:17 smithi043 ceph-mon[29478]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:17.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:17 smithi043 ceph-mon[29478]: from='client.? 172.21.15.164:0/2634654589' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T10:24:17.735 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 podman[32694]: 2024-04-25 10:24:17.52282588 +0000 UTC m=+0.018090290 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:24:17.735 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 podman[32694]: 2024-04-25 10:24:17.624511475 +0000 UTC m=+0.119775899 container create 31632d504d56d6ecdb302e312f9f7a7a5aa6f6ca8821cee3cd792a3009f3dd7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-25T10:24:18.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 podman[32694]: 2024-04-25 10:24:17.835438136 +0000 UTC m=+0.330702525 container init 31632d504d56d6ecdb302e312f9f7a7a5aa6f6ca8821cee3cd792a3009f3dd7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424) 2024-04-25T10:24:18.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 podman[32694]: 2024-04-25 10:24:17.838394914 +0000 UTC m=+0.333659301 container start 31632d504d56d6ecdb302e312f9f7a7a5aa6f6ca8821cee3cd792a3009f3dd7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T10:24:18.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T10:24:18.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev), process ceph-mon, pid 2 2024-04-25T10:24:18.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: pidfile_write: ignore empty --pid-file 2024-04-25T10:24:18.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: load: jerasure load: lrc 2024-04-25T10:24:18.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: RocksDB version: 7.9.2 2024-04-25T10:24:18.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Git sha 0 2024-04-25T10:24:18.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Compile date 2024-04-25 02:57:34 2024-04-25T10:24:18.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: DB SUMMARY 2024-04-25T10:24:18.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: DB Session ID: Y472N4CR4UBUZHIGQ0OH 2024-04-25T10:24:18.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: CURRENT file: CURRENT 2024-04-25T10:24:18.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T10:24:18.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T10:24:18.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-25T10:24:18.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-25T10:24:18.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.error_if_exists: 0 2024-04-25T10:24:18.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.create_if_missing: 0 2024-04-25T10:24:18.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.paranoid_checks: 1 2024-04-25T10:24:18.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T10:24:18.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T10:24:18.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T10:24:18.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.env: 0x55ccf2b1c500 2024-04-25T10:24:18.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T10:24:18.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.info_log: 0x55ccf4857fc0 2024-04-25T10:24:18.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T10:24:18.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.statistics: (nil) 2024-04-25T10:24:18.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.use_fsync: 0 2024-04-25T10:24:18.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_log_file_size: 0 2024-04-25T10:24:18.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T10:24:18.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T10:24:18.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T10:24:18.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T10:24:18.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.allow_fallocate: 1 2024-04-25T10:24:18.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T10:24:18.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T10:24:18.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.use_direct_reads: 0 2024-04-25T10:24:18.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T10:24:18.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T10:24:18.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.db_log_dir: 2024-04-25T10:24:18.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.wal_dir: 2024-04-25T10:24:18.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T10:24:18.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T10:24:18.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T10:24:18.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T10:24:18.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T10:24:18.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T10:24:18.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T10:24:18.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T10:24:18.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.write_buffer_manager: 0x55ccf48288c0 2024-04-25T10:24:18.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T10:24:18.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T10:24:18.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T10:24:18.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T10:24:18.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T10:24:18.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T10:24:18.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T10:24:18.048 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T10:24:18.048 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.unordered_write: 0 2024-04-25T10:24:18.048 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T10:24:18.048 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T10:24:18.048 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T10:24:18.048 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T10:24:18.048 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.row_cache: None 2024-04-25T10:24:18.048 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.wal_filter: None 2024-04-25T10:24:18.048 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T10:24:18.048 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T10:24:18.048 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.two_write_queues: 0 2024-04-25T10:24:18.048 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T10:24:18.049 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.wal_compression: 0 2024-04-25T10:24:18.049 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.atomic_flush: 0 2024-04-25T10:24:18.049 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T10:24:18.049 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T10:24:18.049 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T10:24:18.049 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.log_readahead_size: 0 2024-04-25T10:24:18.049 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T10:24:18.049 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T10:24:18.049 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T10:24:18.049 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T10:24:18.049 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T10:24:18.049 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T10:24:18.050 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T10:24:18.050 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_background_jobs: 2 2024-04-25T10:24:18.050 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_background_compactions: -1 2024-04-25T10:24:18.050 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_subcompactions: 1 2024-04-25T10:24:18.050 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T10:24:18.050 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T10:24:18.050 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T10:24:18.050 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T10:24:18.050 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T10:24:18.050 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T10:24:18.050 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T10:24:18.050 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T10:24:18.050 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_open_files: -1 2024-04-25T10:24:18.050 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T10:24:18.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T10:24:18.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T10:24:18.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T10:24:18.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_background_flushes: -1 2024-04-25T10:24:18.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Compression algorithms supported: 2024-04-25T10:24:18.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T10:24:18.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: kZSTD supported: 0 2024-04-25T10:24:18.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: kXpressCompression supported: 0 2024-04-25T10:24:18.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T10:24:18.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: kZlibCompression supported: 1 2024-04-25T10:24:18.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: kSnappyCompression supported: 1 2024-04-25T10:24:18.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: kLZ4Compression supported: 1 2024-04-25T10:24:18.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: kBZip2Compression supported: 0 2024-04-25T10:24:18.052 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T10:24:18.052 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T10:24:18.052 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-25T10:24:18.052 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T10:24:18.052 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T10:24:18.052 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.merge_operator: 2024-04-25T10:24:18.052 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compaction_filter: None 2024-04-25T10:24:18.052 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T10:24:18.052 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T10:24:18.052 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T10:24:18.052 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T10:24:18.052 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ccf495e1a0) 2024-04-25T10:24:18.052 INFO:journalctl@ceph.mon.c.smithi164.stdout: cache_index_and_filter_blocks: 1 2024-04-25T10:24:18.052 INFO:journalctl@ceph.mon.c.smithi164.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T10:24:18.053 INFO:journalctl@ceph.mon.c.smithi164.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T10:24:18.053 INFO:journalctl@ceph.mon.c.smithi164.stdout: pin_top_level_index_and_filter: 1 2024-04-25T10:24:18.053 INFO:journalctl@ceph.mon.c.smithi164.stdout: index_type: 0 2024-04-25T10:24:18.053 INFO:journalctl@ceph.mon.c.smithi164.stdout: data_block_index_type: 0 2024-04-25T10:24:18.053 INFO:journalctl@ceph.mon.c.smithi164.stdout: index_shortening: 1 2024-04-25T10:24:18.053 INFO:journalctl@ceph.mon.c.smithi164.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T10:24:18.053 INFO:journalctl@ceph.mon.c.smithi164.stdout: checksum: 4 2024-04-25T10:24:18.053 INFO:journalctl@ceph.mon.c.smithi164.stdout: no_block_cache: 0 2024-04-25T10:24:18.053 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_cache: 0x55ccf485b1f0 2024-04-25T10:24:18.053 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_cache_name: BinnedLRUCache 2024-04-25T10:24:18.053 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_cache_options: 2024-04-25T10:24:18.053 INFO:journalctl@ceph.mon.c.smithi164.stdout: capacity : 536870912 2024-04-25T10:24:18.053 INFO:journalctl@ceph.mon.c.smithi164.stdout: num_shard_bits : 4 2024-04-25T10:24:18.054 INFO:journalctl@ceph.mon.c.smithi164.stdout: strict_capacity_limit : 0 2024-04-25T10:24:18.054 INFO:journalctl@ceph.mon.c.smithi164.stdout: high_pri_pool_ratio: 0.000 2024-04-25T10:24:18.054 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_cache_compressed: (nil) 2024-04-25T10:24:18.054 INFO:journalctl@ceph.mon.c.smithi164.stdout: persistent_cache: (nil) 2024-04-25T10:24:18.054 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_size: 4096 2024-04-25T10:24:18.054 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_size_deviation: 10 2024-04-25T10:24:18.054 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_restart_interval: 16 2024-04-25T10:24:18.054 INFO:journalctl@ceph.mon.c.smithi164.stdout: index_block_restart_interval: 1 2024-04-25T10:24:18.054 INFO:journalctl@ceph.mon.c.smithi164.stdout: metadata_block_size: 4096 2024-04-25T10:24:18.054 INFO:journalctl@ceph.mon.c.smithi164.stdout: partition_filters: 0 2024-04-25T10:24:18.054 INFO:journalctl@ceph.mon.c.smithi164.stdout: use_delta_encoding: 1 2024-04-25T10:24:18.054 INFO:journalctl@ceph.mon.c.smithi164.stdout: filter_policy: bloomfilter 2024-04-25T10:24:18.054 INFO:journalctl@ceph.mon.c.smithi164.stdout: whole_key_filtering: 1 2024-04-25T10:24:18.054 INFO:journalctl@ceph.mon.c.smithi164.stdout: verify_compression: 0 2024-04-25T10:24:18.055 INFO:journalctl@ceph.mon.c.smithi164.stdout: read_amp_bytes_per_bit: 0 2024-04-25T10:24:18.055 INFO:journalctl@ceph.mon.c.smithi164.stdout: format_version: 5 2024-04-25T10:24:18.055 INFO:journalctl@ceph.mon.c.smithi164.stdout: enable_index_compression: 1 2024-04-25T10:24:18.055 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_align: 0 2024-04-25T10:24:18.055 INFO:journalctl@ceph.mon.c.smithi164.stdout: max_auto_readahead_size: 262144 2024-04-25T10:24:18.055 INFO:journalctl@ceph.mon.c.smithi164.stdout: prepopulate_block_cache: 0 2024-04-25T10:24:18.055 INFO:journalctl@ceph.mon.c.smithi164.stdout: initial_auto_readahead_size: 8192 2024-04-25T10:24:18.055 INFO:journalctl@ceph.mon.c.smithi164.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T10:24:18.055 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T10:24:18.055 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T10:24:18.055 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compression: NoCompression 2024-04-25T10:24:18.055 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T10:24:18.055 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T10:24:18.055 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T10:24:18.056 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.num_levels: 7 2024-04-25T10:24:18.056 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T10:24:18.056 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T10:24:18.056 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T10:24:18.056 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T10:24:18.056 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T10:24:18.056 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T10:24:18.056 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T10:24:18.056 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T10:24:18.056 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T10:24:18.056 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T10:24:18.056 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T10:24:18.056 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T10:24:18.057 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T10:24:18.057 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T10:24:18.057 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T10:24:18.057 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T10:24:18.057 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T10:24:18.057 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T10:24:18.057 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T10:24:18.057 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T10:24:18.057 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T10:24:18.057 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T10:24:18.057 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T10:24:18.057 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T10:24:18.057 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T10:24:18.057 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T10:24:18.057 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T10:24:18.057 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T10:24:18.058 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T10:24:18.058 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T10:24:18.058 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T10:24:18.058 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T10:24:18.058 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T10:24:18.058 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T10:24:18.058 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T10:24:18.058 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T10:24:18.058 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T10:24:18.058 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T10:24:18.058 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T10:24:18.058 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T10:24:18.058 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T10:24:18.059 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T10:24:18.059 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T10:24:18.059 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T10:24:18.059 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T10:24:18.059 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T10:24:18.059 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T10:24:18.059 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T10:24:18.059 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T10:24:18.059 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T10:24:18.059 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T10:24:18.059 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T10:24:18.059 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T10:24:18.059 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T10:24:18.059 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.inplace_update_support: 0 2024-04-25T10:24:18.060 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T10:24:18.060 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T10:24:18.060 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T10:24:18.060 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T10:24:18.060 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.bloom_locality: 0 2024-04-25T10:24:18.060 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.max_successive_merges: 0 2024-04-25T10:24:18.060 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T10:24:18.060 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T10:24:18.060 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T10:24:18.060 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T10:24:18.060 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.ttl: 2592000 2024-04-25T10:24:18.060 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T10:24:18.060 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T10:24:18.060 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T10:24:18.060 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.enable_blob_files: false 2024-04-25T10:24:18.061 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.min_blob_size: 0 2024-04-25T10:24:18.061 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T10:24:18.061 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T10:24:18.061 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T10:24:18.061 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T10:24:18.061 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T10:24:18.061 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T10:24:18.061 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T10:24:18.061 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T10:24:18.061 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T10:24:18.061 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T10:24:18.061 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1a33382a-4f97-497e-b0e9-688dac84853c 2024-04-25T10:24:18.061 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714040657878879, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T10:24:18.061 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T10:24:18.061 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714040657879310, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714040657, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1a33382a-4f97-497e-b0e9-688dac84853c", "db_session_id": "Y472N4CR4UBUZHIGQ0OH", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T10:24:18.062 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714040657879390, "job": 1, "event": "recovery_finished"} 2024-04-25T10:24:18.062 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T10:24:18.062 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T10:24:18.062 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ccf483fc00 2024-04-25T10:24:18.062 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: DB pointer 0x55ccf9632000 2024-04-25T10:24:18.062 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T10:24:18.062 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T10:24:18.062 INFO:journalctl@ceph.mon.c.smithi164.stdout: ** DB Stats ** 2024-04-25T10:24:18.062 INFO:journalctl@ceph.mon.c.smithi164.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T10:24:18.062 INFO:journalctl@ceph.mon.c.smithi164.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T10:24:18.062 INFO:journalctl@ceph.mon.c.smithi164.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T10:24:18.062 INFO:journalctl@ceph.mon.c.smithi164.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T10:24:18.062 INFO:journalctl@ceph.mon.c.smithi164.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T10:24:18.063 INFO:journalctl@ceph.mon.c.smithi164.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T10:24:18.063 INFO:journalctl@ceph.mon.c.smithi164.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T10:24:18.063 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-04-25T10:24:18.063 INFO:journalctl@ceph.mon.c.smithi164.stdout: ** Compaction Stats [default] ** 2024-04-25T10:24:18.063 INFO:journalctl@ceph.mon.c.smithi164.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T10:24:18.063 INFO:journalctl@ceph.mon.c.smithi164.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T10:24:18.063 INFO:journalctl@ceph.mon.c.smithi164.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T10:24:18.063 INFO:journalctl@ceph.mon.c.smithi164.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T10:24:18.063 INFO:journalctl@ceph.mon.c.smithi164.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T10:24:18.063 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-04-25T10:24:18.063 INFO:journalctl@ceph.mon.c.smithi164.stdout: ** Compaction Stats [default] ** 2024-04-25T10:24:18.063 INFO:journalctl@ceph.mon.c.smithi164.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T10:24:18.063 INFO:journalctl@ceph.mon.c.smithi164.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T10:24:18.064 INFO:journalctl@ceph.mon.c.smithi164.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T10:24:18.064 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-04-25T10:24:18.064 INFO:journalctl@ceph.mon.c.smithi164.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T10:24:18.064 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-04-25T10:24:18.064 INFO:journalctl@ceph.mon.c.smithi164.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T10:24:18.064 INFO:journalctl@ceph.mon.c.smithi164.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T10:24:18.064 INFO:journalctl@ceph.mon.c.smithi164.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T10:24:18.064 INFO:journalctl@ceph.mon.c.smithi164.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T10:24:18.064 INFO:journalctl@ceph.mon.c.smithi164.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T10:24:18.064 INFO:journalctl@ceph.mon.c.smithi164.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T10:24:18.064 INFO:journalctl@ceph.mon.c.smithi164.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T10:24:18.064 INFO:journalctl@ceph.mon.c.smithi164.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T10:24:18.064 INFO:journalctl@ceph.mon.c.smithi164.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T10:24:18.064 INFO:journalctl@ceph.mon.c.smithi164.stdout: Block cache BinnedLRUCache@0x55ccf485b1f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-25T10:24:18.064 INFO:journalctl@ceph.mon.c.smithi164.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T10:24:18.064 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-04-25T10:24:18.065 INFO:journalctl@ceph.mon.c.smithi164.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T10:24:18.065 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-25T10:24:18.065 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: using public_addr v2:172.21.15.164:0/0 -> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-25T10:24:18.065 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: starting mon.c rank -1 at public addrs [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] at bind addrs [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:24:18.065 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: mon.c@-1(???) e0 preinit fsid 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:24:18.065 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: mon.c@-1(synchronizing).mds e1 new map 2024-04-25T10:24:18.065 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-25T10:24:18.065 INFO:journalctl@ceph.mon.c.smithi164.stdout: e1 2024-04-25T10:24:18.065 INFO:journalctl@ceph.mon.c.smithi164.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T10:24:18.065 INFO:journalctl@ceph.mon.c.smithi164.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T10:24:18.065 INFO:journalctl@ceph.mon.c.smithi164.stdout: legacy client fscid: -1 2024-04-25T10:24:18.065 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-04-25T10:24:18.065 INFO:journalctl@ceph.mon.c.smithi164.stdout: No filesystems configured 2024-04-25T10:24:18.065 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T10:24:18.066 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T10:24:18.066 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T10:24:18.066 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T10:24:18.066 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T10:24:18.066 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T10:24:18.066 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T10:24:18.066 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T10:24:18.066 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T10:24:18.066 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T10:24:18.066 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.066 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:18.066 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.066 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:24:18.067 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:18.067 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:24:18.067 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.067 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.067 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.067 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.067 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi077", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:24:18.067 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:18.067 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:18.067 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-25T10:24:18.067 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:18.067 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: Deploying cephadm binary to smithi164 2024-04-25T10:24:18.067 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: Updating smithi077:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:24:18.067 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:24:18.067 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:18.068 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.068 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.068 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.068 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.068 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:24:18.068 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: Updating smithi077:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:24:18.068 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: Added host smithi164 2024-04-25T10:24:18.068 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.068 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:18.068 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.068 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:18.068 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:24:18.068 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.068 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:18.069 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1593744685' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T10:24:18.069 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.069 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.069 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.069 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.069 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:24:18.069 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:18.069 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:18.069 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-25T10:24:18.069 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1593744685' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T10:24:18.069 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T10:24:18.069 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: Updating smithi164:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:24:18.069 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.069 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.069 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.069 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:24:18.070 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:18.070 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: Updating smithi164:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:24:18.070 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi077:172.21.15.77=b;smithi164:172.21.15.164=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:24:18.070 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi077:172.21.15.77=b;smithi164:172.21.15.164=c;count:3 2024-04-25T10:24:18.070 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.070 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:24:18.070 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:18.070 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:18.070 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:18.070 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:24:18.070 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:18.070 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:18.070 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: Deploying daemon mon.c on smithi164 2024-04-25T10:24:18.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:18.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:18.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: from='client.? 172.21.15.164:0/2634654589' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T10:24:18.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 ceph-mon[32713]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T10:24:18.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 bash[32694]: 31632d504d56d6ecdb302e312f9f7a7a5aa6f6ca8821cee3cd792a3009f3dd7c 2024-04-25T10:24:18.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:17 smithi164 systemd[1]: Started Ceph mon.c for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:24:18.459 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T10:24:18.460 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T10:24:18.695 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:24:23.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.134+0000 7f44d725b700 1 -- 172.21.15.164:0/3416477745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44c00035c0 msgr2=0x7f44c0005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:23.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.134+0000 7f44d725b700 1 --2- 172.21.15.164:0/3416477745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44c00035c0 0x7f44c0005aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f44d0103a30 tx=0x7f44c400feb0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.134+0000 7f44d725b700 1 -- 172.21.15.164:0/3416477745 shutdown_connections 2024-04-25T10:24:23.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.134+0000 7f44d725b700 1 --2- 172.21.15.164:0/3416477745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44c00035c0 0x7f44c0005aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.134+0000 7f44d725b700 1 -- 172.21.15.164:0/3416477745 >> 172.21.15.164:0/3416477745 conn(0x7f44d00ff600 msgr2=0x7f44d0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:23.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.134+0000 7f44d725b700 1 -- 172.21.15.164:0/3416477745 shutdown_connections 2024-04-25T10:24:23.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.134+0000 7f44d725b700 1 -- 172.21.15.164:0/3416477745 wait complete. 2024-04-25T10:24:23.138 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:24:23.138 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:23.138 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: mon.a calling monitor election 2024-04-25T10:24:23.138 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:23.138 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:23.138 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:23.138 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: mon.c calling monitor election 2024-04-25T10:24:23.139 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:23.139 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:23.139 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:23.139 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:23.139 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:23.139 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T10:24:23.139 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: monmap epoch 2 2024-04-25T10:24:23.139 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: fsid 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:24:23.139 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: last_changed 2024-04-25T10:24:17.905217+0000 2024-04-25T10:24:23.139 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: created 2024-04-25T10:22:33.876396+0000 2024-04-25T10:24:23.139 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: min_mon_release 19 (squid) 2024-04-25T10:24:23.139 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: election_strategy: 1 2024-04-25T10:24:23.139 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-04-25T10:24:23.139 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-04-25T10:24:23.140 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: fsmap 2024-04-25T10:24:23.140 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T10:24:23.140 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: mgrmap e13: x(active, since 50s) 2024-04-25T10:24:23.140 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: overall HEALTH_OK 2024-04-25T10:24:23.140 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:23.140 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:23.140 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:23.140 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:24:23.140 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:22 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:23.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.136+0000 7f44d725b700 1 Processor -- start 2024-04-25T10:24:23.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.136+0000 7f44d725b700 1 -- start start 2024-04-25T10:24:23.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.136+0000 7f44d725b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44c00035c0 0x7f44d0075990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.136+0000 7f44d725b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44d0075ed0 0x7f44d00762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.136+0000 7f44d725b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f44d010bae0 con 0x7f44d0075ed0 2024-04-25T10:24:23.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.136+0000 7f44d725b700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f44d010b960 con 0x7f44c00035c0 2024-04-25T10:24:23.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.136+0000 7f44d4ff7700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44c00035c0 0x7f44d0075990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.136+0000 7f44d4ff7700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44c00035c0 0x7f44d0075990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33120/0 (socket says 172.21.15.164:33120) 2024-04-25T10:24:23.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.136+0000 7f44d4ff7700 1 -- 172.21.15.164:0/1560860585 learned_addr learned my addr 172.21.15.164:0/1560860585 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-25T10:24:23.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.137+0000 7f44d4ff7700 1 -- 172.21.15.164:0/1560860585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44c00035c0 msgr2=0x7f44d0075990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T10:24:23.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.137+0000 7f44d4ff7700 1 -- 172.21.15.164:0/1560860585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44c00035c0 msgr2=0x7f44d0075990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:24:23.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.137+0000 7f44d4ff7700 1 --2- 172.21.15.164:0/1560860585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44c00035c0 0x7f44d0075990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:24:23.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.137+0000 7f44cffff700 1 --2- 172.21.15.164:0/1560860585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44d0075ed0 0x7f44d00762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.137+0000 7f44d4ff7700 1 --2- 172.21.15.164:0/1560860585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44c00035c0 0x7f44d0075990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:24:23.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.137+0000 7f44cffff700 1 -- 172.21.15.164:0/1560860585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44c00035c0 msgr2=0x7f44d0075990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:24:23.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.137+0000 7f44cffff700 1 --2- 172.21.15.164:0/1560860585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44c00035c0 0x7f44d0075990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.137+0000 7f44cffff700 1 -- 172.21.15.164:0/1560860585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d00778c0 con 0x7f44d0075ed0 2024-04-25T10:24:23.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.137+0000 7f44cffff700 1 --2- 172.21.15.164:0/1560860585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44d0075ed0 0x7f44d00762e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00b4d0 tx=0x7f44bc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:23.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.138+0000 7f44cdffb700 1 -- 172.21.15.164:0/1560860585 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f44bc011630 con 0x7f44d0075ed0 2024-04-25T10:24:23.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.138+0000 7f44cdffb700 1 -- 172.21.15.164:0/1560860585 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44bc011c80 con 0x7f44d0075ed0 2024-04-25T10:24:23.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.138+0000 7f44cdffb700 1 -- 172.21.15.164:0/1560860585 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f44bc010da0 con 0x7f44d0075ed0 2024-04-25T10:24:23.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.138+0000 7f44d725b700 1 -- 172.21.15.164:0/1560860585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d01a5be0 con 0x7f44d0075ed0 2024-04-25T10:24:23.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.138+0000 7f44d725b700 1 -- 172.21.15.164:0/1560860585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d01a6140 con 0x7f44d0075ed0 2024-04-25T10:24:23.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.139+0000 7f44cdffb700 1 -- 172.21.15.164:0/1560860585 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f44bc018020 con 0x7f44d0075ed0 2024-04-25T10:24:23.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.139+0000 7f44cdffb700 1 --2- 172.21.15.164:0/1560860585 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f44b403d400 0x7f44b403f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.140+0000 7f44cdffb700 1 -- 172.21.15.164:0/1560860585 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f44bc050f10 con 0x7f44d0075ed0 2024-04-25T10:24:23.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.140+0000 7f44d4ff7700 1 --2- 172.21.15.164:0/1560860585 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f44b403d400 0x7f44b403f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.140+0000 7f44d725b700 1 -- 172.21.15.164:0/1560860585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d0105200 con 0x7f44d0075ed0 2024-04-25T10:24:23.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.140+0000 7f44d4ff7700 1 --2- 172.21.15.164:0/1560860585 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f44b403d400 0x7f44b403f910 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f44c400fa60 tx=0x7f44c400f4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:23.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.146+0000 7f44cdffb700 1 -- 172.21.15.164:0/1560860585 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44bc01f020 con 0x7f44d0075ed0 2024-04-25T10:24:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:24:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: mon.a calling monitor election 2024-04-25T10:24:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: mon.c calling monitor election 2024-04-25T10:24:23.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:23.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:23.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:23.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:23.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:23.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T10:24:23.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: monmap epoch 2 2024-04-25T10:24:23.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: fsid 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:24:23.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: last_changed 2024-04-25T10:24:17.905217+0000 2024-04-25T10:24:23.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: created 2024-04-25T10:22:33.876396+0000 2024-04-25T10:24:23.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: min_mon_release 19 (squid) 2024-04-25T10:24:23.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: election_strategy: 1 2024-04-25T10:24:23.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-04-25T10:24:23.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-04-25T10:24:23.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: fsmap 2024-04-25T10:24:23.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T10:24:23.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: mgrmap e13: x(active, since 50s) 2024-04-25T10:24:23.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: overall HEALTH_OK 2024-04-25T10:24:23.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:23.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:23.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:23.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:24:23.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:22 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:23.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.432+0000 7f44d725b700 1 -- 172.21.15.164:0/1560860585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f44d0076ef0 con 0x7f44d0075ed0 2024-04-25T10:24:23.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.432+0000 7f44cdffb700 1 -- 172.21.15.164:0/1560860585 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f44bc010410 con 0x7f44d0075ed0 2024-04-25T10:24:23.435 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T10:24:23.435 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":2,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","modified":"2024-04-25T10:24:17.905217Z","created":"2024-04-25T10:22:33.876396Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T10:24:23.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.435+0000 7f44d725b700 1 -- 172.21.15.164:0/1560860585 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f44b403d400 msgr2=0x7f44b403f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:23.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.435+0000 7f44d725b700 1 --2- 172.21.15.164:0/1560860585 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f44b403d400 0x7f44b403f910 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f44c400fa60 tx=0x7f44c400f4a0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.435+0000 7f44d725b700 1 -- 172.21.15.164:0/1560860585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44d0075ed0 msgr2=0x7f44d00762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:23.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.435+0000 7f44d725b700 1 --2- 172.21.15.164:0/1560860585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44d0075ed0 0x7f44d00762e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00b4d0 tx=0x7f44bc00b890 comp rx=0 tx=0).stop 2024-04-25T10:24:23.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.435+0000 7f44d725b700 1 -- 172.21.15.164:0/1560860585 shutdown_connections 2024-04-25T10:24:23.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.435+0000 7f44d725b700 1 --2- 172.21.15.164:0/1560860585 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f44b403d400 0x7f44b403f910 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.435+0000 7f44d725b700 1 --2- 172.21.15.164:0/1560860585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44c00035c0 0x7f44d0075990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.435+0000 7f44d725b700 1 --2- 172.21.15.164:0/1560860585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44d0075ed0 0x7f44d00762e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.435+0000 7f44d725b700 1 -- 172.21.15.164:0/1560860585 >> 172.21.15.164:0/1560860585 conn(0x7f44d00ff600 msgr2=0x7f44d01017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:23.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.435+0000 7f44d725b700 1 -- 172.21.15.164:0/1560860585 shutdown_connections 2024-04-25T10:24:23.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:23.435+0000 7f44d725b700 1 -- 172.21.15.164:0/1560860585 wait complete. 2024-04-25T10:24:23.438 INFO:teuthology.orchestra.run.smithi164.stderr:dumped monmap epoch 2 2024-04-25T10:24:24.009 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:23 smithi164 ceph-mon[32713]: Deploying daemon mon.b on smithi077 2024-04-25T10:24:24.010 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:23 smithi164 ceph-mon[32713]: from='client.? 172.21.15.164:0/1560860585' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T10:24:24.010 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:24.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:23 smithi043 ceph-mon[29478]: Deploying daemon mon.b on smithi077 2024-04-25T10:24:24.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:23 smithi043 ceph-mon[29478]: from='client.? 172.21.15.164:0/1560860585' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T10:24:24.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:25.038 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T10:24:25.038 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T10:24:25.261 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:24 smithi043 ceph-mon[29478]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:25.283 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:24:25.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:24 smithi164 ceph-mon[32713]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:26.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.694+0000 7fb7ab288700 1 -- 172.21.15.164:0/3561373795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7a4108a40 msgr2=0x7fb7a4108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.694+0000 7fb7ab288700 1 --2- 172.21.15.164:0/3561373795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7a4108a40 0x7fb7a4108e50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0009910 tx=0x7fb7a0009c20 comp rx=0 tx=0).stop 2024-04-25T10:24:26.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.694+0000 7fb7ab288700 1 -- 172.21.15.164:0/3561373795 shutdown_connections 2024-04-25T10:24:26.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.694+0000 7fb7ab288700 1 --2- 172.21.15.164:0/3561373795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7a4108a40 0x7fb7a4108e50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.694+0000 7fb7ab288700 1 -- 172.21.15.164:0/3561373795 >> 172.21.15.164:0/3561373795 conn(0x7fb7a40796b0 msgr2=0x7fb7a4079af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:26.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.694+0000 7fb7ab288700 1 -- 172.21.15.164:0/3561373795 shutdown_connections 2024-04-25T10:24:26.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.694+0000 7fb7ab288700 1 -- 172.21.15.164:0/3561373795 wait complete. 2024-04-25T10:24:26.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.695+0000 7fb7ab288700 1 Processor -- start 2024-04-25T10:24:26.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.695+0000 7fb7ab288700 1 -- start start 2024-04-25T10:24:26.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.695+0000 7fb7ab288700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7a4108a40 0x7fb7a419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.695+0000 7fb7ab288700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7a419fbf0 0x7fb7a41998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.695+0000 7fb7ab288700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a410cf90 con 0x7fb7a4108a40 2024-04-25T10:24:26.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.695+0000 7fb7ab288700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a410ce10 con 0x7fb7a419fbf0 2024-04-25T10:24:26.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.695+0000 7fb7a9024700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7a4108a40 0x7fb7a419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.695+0000 7fb7a9024700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7a4108a40 0x7fb7a419f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.164:42728/0 (socket says 172.21.15.164:42728) 2024-04-25T10:24:26.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.695+0000 7fb7a9024700 1 -- 172.21.15.164:0/3352452296 learned_addr learned my addr 172.21.15.164:0/3352452296 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-25T10:24:26.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.696+0000 7fb7a8823700 1 --2- 172.21.15.164:0/3352452296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7a419fbf0 0x7fb7a41998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.696+0000 7fb7a8823700 1 -- 172.21.15.164:0/3352452296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7a419fbf0 msgr2=0x7fb7a41998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T10:24:26.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.696+0000 7fb7a8823700 1 -- 172.21.15.164:0/3352452296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7a419fbf0 msgr2=0x7fb7a41998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:24:26.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.696+0000 7fb7a8823700 1 --2- 172.21.15.164:0/3352452296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7a419fbf0 0x7fb7a41998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:24:26.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.696+0000 7fb7a8823700 1 --2- 172.21.15.164:0/3352452296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7a419fbf0 0x7fb7a41998d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:24:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.696+0000 7fb7a9024700 1 -- 172.21.15.164:0/3352452296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7a419fbf0 msgr2=0x7fb7a41998d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:24:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.696+0000 7fb7a9024700 1 --2- 172.21.15.164:0/3352452296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7a419fbf0 0x7fb7a41998d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.696+0000 7fb7a9024700 1 -- 172.21.15.164:0/3352452296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a41a01c0 con 0x7fb7a4108a40 2024-04-25T10:24:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.696+0000 7fb7a9024700 1 --2- 172.21.15.164:0/3352452296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7a4108a40 0x7fb7a419f6b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0006010 tx=0x7fb7a0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.696+0000 7fb79a7fc700 1 -- 172.21.15.164:0/3352452296 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb7a0005dc0 con 0x7fb7a4108a40 2024-04-25T10:24:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.696+0000 7fb7ab288700 1 -- 172.21.15.164:0/3352452296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a4199e70 con 0x7fb7a4108a40 2024-04-25T10:24:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.696+0000 7fb79a7fc700 1 -- 172.21.15.164:0/3352452296 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7a000be10 con 0x7fb7a4108a40 2024-04-25T10:24:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.696+0000 7fb79a7fc700 1 -- 172.21.15.164:0/3352452296 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb7a0022e00 con 0x7fb7a4108a40 2024-04-25T10:24:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.696+0000 7fb7ab288700 1 -- 172.21.15.164:0/3352452296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a419a400 con 0x7fb7a4108a40 2024-04-25T10:24:26.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.697+0000 7fb7ab288700 1 -- 172.21.15.164:0/3352452296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a419a710 con 0x7fb7a4108a40 2024-04-25T10:24:26.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.697+0000 7fb79a7fc700 1 -- 172.21.15.164:0/3352452296 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fb7a002a070 con 0x7fb7a4108a40 2024-04-25T10:24:26.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.697+0000 7fb79a7fc700 1 --2- 172.21.15.164:0/3352452296 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb79003d3b0 0x7fb79003f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.697+0000 7fb79a7fc700 1 -- 172.21.15.164:0/3352452296 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb7a001e030 con 0x7fb7a4108a40 2024-04-25T10:24:26.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.698+0000 7fb7a8823700 1 --2- 172.21.15.164:0/3352452296 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb79003d3b0 0x7fb79003f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.698+0000 7fb7a8823700 1 --2- 172.21.15.164:0/3352452296 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb79003d3b0 0x7fb79003f8c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb7a406c8b0 tx=0x7fb794008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:26.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.708+0000 7fb79a7fc700 1 -- 172.21.15.164:0/3352452296 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb7a0013070 con 0x7fb7a4108a40 2024-04-25T10:24:26.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 podman[32752]: 2024-04-25 10:24:26.688781988 +0000 UTC m=+0.129420186 container create 6cbd105360e0ff4dbfcde6f79266cd0ff93d10e228654ea9cf9bfe8873647170 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T10:24:26.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.973+0000 7fb79a7fc700 1 -- 172.21.15.164:0/3352452296 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a002be20 con 0x7fb7a4108a40 2024-04-25T10:24:26.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:26.991+0000 7fb7ab288700 1 -- 172.21.15.164:0/3352452296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb7a419aea0 con 0x7fb7a4108a40 2024-04-25T10:24:27.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 podman[32752]: 2024-04-25 10:24:26.899425942 +0000 UTC m=+0.340064141 container init 6cbd105360e0ff4dbfcde6f79266cd0ff93d10e228654ea9cf9bfe8873647170 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:24:27.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 podman[32752]: 2024-04-25 10:24:26.902746969 +0000 UTC m=+0.343385169 container start 6cbd105360e0ff4dbfcde6f79266cd0ff93d10e228654ea9cf9bfe8873647170 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T10:24:27.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T10:24:27.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev), process ceph-mon, pid 2 2024-04-25T10:24:27.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: pidfile_write: ignore empty --pid-file 2024-04-25T10:24:27.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: load: jerasure load: lrc 2024-04-25T10:24:27.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: RocksDB version: 7.9.2 2024-04-25T10:24:27.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Git sha 0 2024-04-25T10:24:27.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Compile date 2024-04-25 02:57:34 2024-04-25T10:24:27.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: DB SUMMARY 2024-04-25T10:24:27.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: DB Session ID: A4CXFM712UF0ZDKGJJQU 2024-04-25T10:24:27.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: CURRENT file: CURRENT 2024-04-25T10:24:27.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T10:24:27.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T10:24:27.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-25T10:24:27.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-25T10:24:27.294 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.error_if_exists: 0 2024-04-25T10:24:27.294 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.create_if_missing: 0 2024-04-25T10:24:27.294 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.paranoid_checks: 1 2024-04-25T10:24:27.294 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T10:24:27.294 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T10:24:27.295 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T10:24:27.295 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.env: 0x55622d882500 2024-04-25T10:24:27.295 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T10:24:27.295 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.info_log: 0x55622e847fc0 2024-04-25T10:24:27.295 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T10:24:27.295 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.statistics: (nil) 2024-04-25T10:24:27.295 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.use_fsync: 0 2024-04-25T10:24:27.296 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_log_file_size: 0 2024-04-25T10:24:27.296 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T10:24:27.296 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T10:24:27.296 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T10:24:27.296 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T10:24:27.297 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.allow_fallocate: 1 2024-04-25T10:24:27.297 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T10:24:27.297 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T10:24:27.297 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.use_direct_reads: 0 2024-04-25T10:24:27.297 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T10:24:27.297 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T10:24:27.298 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.db_log_dir: 2024-04-25T10:24:27.298 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.wal_dir: 2024-04-25T10:24:27.298 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T10:24:27.298 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T10:24:27.298 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T10:24:27.298 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T10:24:27.298 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T10:24:27.298 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T10:24:27.299 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T10:24:27.299 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T10:24:27.299 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.write_buffer_manager: 0x55622e8188c0 2024-04-25T10:24:27.299 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T10:24:27.299 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T10:24:27.299 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T10:24:27.299 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T10:24:27.299 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T10:24:27.299 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T10:24:27.299 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T10:24:27.299 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T10:24:27.299 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.unordered_write: 0 2024-04-25T10:24:27.300 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T10:24:27.300 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T10:24:27.300 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T10:24:27.300 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T10:24:27.300 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.row_cache: None 2024-04-25T10:24:27.300 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.wal_filter: None 2024-04-25T10:24:27.300 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T10:24:27.300 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T10:24:27.300 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.two_write_queues: 0 2024-04-25T10:24:27.300 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T10:24:27.300 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.wal_compression: 0 2024-04-25T10:24:27.300 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.atomic_flush: 0 2024-04-25T10:24:27.300 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T10:24:27.301 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T10:24:27.301 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T10:24:27.301 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.log_readahead_size: 0 2024-04-25T10:24:27.301 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T10:24:27.301 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T10:24:27.301 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T10:24:27.301 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T10:24:27.301 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T10:24:27.301 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T10:24:27.301 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T10:24:27.302 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_background_jobs: 2 2024-04-25T10:24:27.302 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_background_compactions: -1 2024-04-25T10:24:27.302 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_subcompactions: 1 2024-04-25T10:24:27.302 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T10:24:27.302 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T10:24:27.302 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T10:24:27.302 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T10:24:27.302 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T10:24:27.303 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T10:24:27.303 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T10:24:27.303 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T10:24:27.303 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_open_files: -1 2024-04-25T10:24:27.303 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T10:24:27.303 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T10:24:27.303 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T10:24:27.303 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T10:24:27.303 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_background_flushes: -1 2024-04-25T10:24:27.303 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Compression algorithms supported: 2024-04-25T10:24:27.303 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T10:24:27.303 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: kZSTD supported: 0 2024-04-25T10:24:27.303 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: kXpressCompression supported: 0 2024-04-25T10:24:27.303 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T10:24:27.303 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: kZlibCompression supported: 1 2024-04-25T10:24:27.304 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: kSnappyCompression supported: 1 2024-04-25T10:24:27.304 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: kLZ4Compression supported: 1 2024-04-25T10:24:27.304 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: kBZip2Compression supported: 0 2024-04-25T10:24:27.304 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T10:24:27.304 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T10:24:27.304 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-25T10:24:27.304 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T10:24:27.304 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T10:24:27.304 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.merge_operator: 2024-04-25T10:24:27.304 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compaction_filter: None 2024-04-25T10:24:27.304 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T10:24:27.304 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T10:24:27.304 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T10:24:27.304 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T10:24:27.304 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55622e94e1a0) 2024-04-25T10:24:27.304 INFO:journalctl@ceph.mon.b.smithi077.stdout: cache_index_and_filter_blocks: 1 2024-04-25T10:24:27.305 INFO:journalctl@ceph.mon.b.smithi077.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T10:24:27.305 INFO:journalctl@ceph.mon.b.smithi077.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T10:24:27.305 INFO:journalctl@ceph.mon.b.smithi077.stdout: pin_top_level_index_and_filter: 1 2024-04-25T10:24:27.305 INFO:journalctl@ceph.mon.b.smithi077.stdout: index_type: 0 2024-04-25T10:24:27.305 INFO:journalctl@ceph.mon.b.smithi077.stdout: data_block_index_type: 0 2024-04-25T10:24:27.305 INFO:journalctl@ceph.mon.b.smithi077.stdout: index_shortening: 1 2024-04-25T10:24:27.305 INFO:journalctl@ceph.mon.b.smithi077.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T10:24:27.305 INFO:journalctl@ceph.mon.b.smithi077.stdout: checksum: 4 2024-04-25T10:24:27.305 INFO:journalctl@ceph.mon.b.smithi077.stdout: no_block_cache: 0 2024-04-25T10:24:27.305 INFO:journalctl@ceph.mon.b.smithi077.stdout: block_cache: 0x55622e84b1f0 2024-04-25T10:24:27.305 INFO:journalctl@ceph.mon.b.smithi077.stdout: block_cache_name: BinnedLRUCache 2024-04-25T10:24:27.305 INFO:journalctl@ceph.mon.b.smithi077.stdout: block_cache_options: 2024-04-25T10:24:27.305 INFO:journalctl@ceph.mon.b.smithi077.stdout: capacity : 536870912 2024-04-25T10:24:27.305 INFO:journalctl@ceph.mon.b.smithi077.stdout: num_shard_bits : 4 2024-04-25T10:24:27.305 INFO:journalctl@ceph.mon.b.smithi077.stdout: strict_capacity_limit : 0 2024-04-25T10:24:27.306 INFO:journalctl@ceph.mon.b.smithi077.stdout: high_pri_pool_ratio: 0.000 2024-04-25T10:24:27.306 INFO:journalctl@ceph.mon.b.smithi077.stdout: block_cache_compressed: (nil) 2024-04-25T10:24:27.306 INFO:journalctl@ceph.mon.b.smithi077.stdout: persistent_cache: (nil) 2024-04-25T10:24:27.306 INFO:journalctl@ceph.mon.b.smithi077.stdout: block_size: 4096 2024-04-25T10:24:27.306 INFO:journalctl@ceph.mon.b.smithi077.stdout: block_size_deviation: 10 2024-04-25T10:24:27.306 INFO:journalctl@ceph.mon.b.smithi077.stdout: block_restart_interval: 16 2024-04-25T10:24:27.306 INFO:journalctl@ceph.mon.b.smithi077.stdout: index_block_restart_interval: 1 2024-04-25T10:24:27.306 INFO:journalctl@ceph.mon.b.smithi077.stdout: metadata_block_size: 4096 2024-04-25T10:24:27.306 INFO:journalctl@ceph.mon.b.smithi077.stdout: partition_filters: 0 2024-04-25T10:24:27.306 INFO:journalctl@ceph.mon.b.smithi077.stdout: use_delta_encoding: 1 2024-04-25T10:24:27.307 INFO:journalctl@ceph.mon.b.smithi077.stdout: filter_policy: bloomfilter 2024-04-25T10:24:27.307 INFO:journalctl@ceph.mon.b.smithi077.stdout: whole_key_filtering: 1 2024-04-25T10:24:27.307 INFO:journalctl@ceph.mon.b.smithi077.stdout: verify_compression: 0 2024-04-25T10:24:27.307 INFO:journalctl@ceph.mon.b.smithi077.stdout: read_amp_bytes_per_bit: 0 2024-04-25T10:24:27.307 INFO:journalctl@ceph.mon.b.smithi077.stdout: format_version: 5 2024-04-25T10:24:27.307 INFO:journalctl@ceph.mon.b.smithi077.stdout: enable_index_compression: 1 2024-04-25T10:24:27.307 INFO:journalctl@ceph.mon.b.smithi077.stdout: block_align: 0 2024-04-25T10:24:27.307 INFO:journalctl@ceph.mon.b.smithi077.stdout: max_auto_readahead_size: 262144 2024-04-25T10:24:27.307 INFO:journalctl@ceph.mon.b.smithi077.stdout: prepopulate_block_cache: 0 2024-04-25T10:24:27.307 INFO:journalctl@ceph.mon.b.smithi077.stdout: initial_auto_readahead_size: 8192 2024-04-25T10:24:27.310 INFO:journalctl@ceph.mon.b.smithi077.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T10:24:27.310 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T10:24:27.310 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T10:24:27.310 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compression: NoCompression 2024-04-25T10:24:27.310 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T10:24:27.310 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T10:24:27.310 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T10:24:27.311 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.num_levels: 7 2024-04-25T10:24:27.311 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T10:24:27.311 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T10:24:27.311 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T10:24:27.311 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T10:24:27.311 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T10:24:27.311 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T10:24:27.311 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T10:24:27.312 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T10:24:27.312 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T10:24:27.312 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T10:24:27.312 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T10:24:27.312 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T10:24:27.312 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T10:24:27.312 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T10:24:27.313 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T10:24:27.313 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T10:24:27.313 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T10:24:27.313 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T10:24:27.313 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T10:24:27.313 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T10:24:27.313 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T10:24:27.313 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T10:24:27.313 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T10:24:27.313 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T10:24:27.313 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T10:24:27.314 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T10:24:27.314 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T10:24:27.314 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T10:24:27.314 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T10:24:27.314 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T10:24:27.314 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T10:24:27.314 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T10:24:27.314 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T10:24:27.314 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T10:24:27.314 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T10:24:27.314 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T10:24:27.314 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T10:24:27.314 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T10:24:27.315 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T10:24:27.315 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T10:24:27.315 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T10:24:27.315 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T10:24:27.315 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T10:24:27.315 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T10:24:27.315 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T10:24:27.315 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T10:24:27.315 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T10:24:27.315 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T10:24:27.315 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T10:24:27.315 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T10:24:27.315 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T10:24:27.316 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T10:24:27.316 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T10:24:27.316 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T10:24:27.316 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.inplace_update_support: 0 2024-04-25T10:24:27.316 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T10:24:27.316 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T10:24:27.316 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T10:24:27.316 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T10:24:27.316 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.bloom_locality: 0 2024-04-25T10:24:27.316 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.max_successive_merges: 0 2024-04-25T10:24:27.316 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T10:24:27.316 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T10:24:27.316 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T10:24:27.316 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T10:24:27.317 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.ttl: 2592000 2024-04-25T10:24:27.317 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T10:24:27.317 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T10:24:27.317 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T10:24:27.317 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.enable_blob_files: false 2024-04-25T10:24:27.317 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.min_blob_size: 0 2024-04-25T10:24:27.317 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T10:24:27.317 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T10:24:27.317 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T10:24:27.317 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T10:24:27.317 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T10:24:27.317 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T10:24:27.317 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T10:24:27.318 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T10:24:27.318 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T10:24:27.318 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T10:24:27.318 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e3b55f29-6b55-4541-b277-3ff7c452551c 2024-04-25T10:24:27.318 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714040666943303, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T10:24:27.318 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T10:24:27.318 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714040666943726, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714040666, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e3b55f29-6b55-4541-b277-3ff7c452551c", "db_session_id": "A4CXFM712UF0ZDKGJJQU", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T10:24:27.318 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714040666943806, "job": 1, "event": "recovery_finished"} 2024-04-25T10:24:27.318 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T10:24:27.318 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T10:24:27.319 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55622e82fc00 2024-04-25T10:24:27.319 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: DB pointer 0x556233622000 2024-04-25T10:24:27.319 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T10:24:27.319 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T10:24:27.319 INFO:journalctl@ceph.mon.b.smithi077.stdout: ** DB Stats ** 2024-04-25T10:24:27.319 INFO:journalctl@ceph.mon.b.smithi077.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T10:24:27.319 INFO:journalctl@ceph.mon.b.smithi077.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T10:24:27.319 INFO:journalctl@ceph.mon.b.smithi077.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T10:24:27.319 INFO:journalctl@ceph.mon.b.smithi077.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T10:24:27.319 INFO:journalctl@ceph.mon.b.smithi077.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T10:24:27.319 INFO:journalctl@ceph.mon.b.smithi077.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T10:24:27.319 INFO:journalctl@ceph.mon.b.smithi077.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T10:24:27.319 INFO:journalctl@ceph.mon.b.smithi077.stdout: 2024-04-25T10:24:27.320 INFO:journalctl@ceph.mon.b.smithi077.stdout: ** Compaction Stats [default] ** 2024-04-25T10:24:27.320 INFO:journalctl@ceph.mon.b.smithi077.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T10:24:27.320 INFO:journalctl@ceph.mon.b.smithi077.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T10:24:27.320 INFO:journalctl@ceph.mon.b.smithi077.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T10:24:27.320 INFO:journalctl@ceph.mon.b.smithi077.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T10:24:27.320 INFO:journalctl@ceph.mon.b.smithi077.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T10:24:27.320 INFO:journalctl@ceph.mon.b.smithi077.stdout: 2024-04-25T10:24:27.320 INFO:journalctl@ceph.mon.b.smithi077.stdout: ** Compaction Stats [default] ** 2024-04-25T10:24:27.320 INFO:journalctl@ceph.mon.b.smithi077.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T10:24:27.320 INFO:journalctl@ceph.mon.b.smithi077.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T10:24:27.320 INFO:journalctl@ceph.mon.b.smithi077.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T10:24:27.320 INFO:journalctl@ceph.mon.b.smithi077.stdout: 2024-04-25T10:24:27.320 INFO:journalctl@ceph.mon.b.smithi077.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T10:24:27.321 INFO:journalctl@ceph.mon.b.smithi077.stdout: 2024-04-25T10:24:27.321 INFO:journalctl@ceph.mon.b.smithi077.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T10:24:27.321 INFO:journalctl@ceph.mon.b.smithi077.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T10:24:27.321 INFO:journalctl@ceph.mon.b.smithi077.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T10:24:27.321 INFO:journalctl@ceph.mon.b.smithi077.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T10:24:27.321 INFO:journalctl@ceph.mon.b.smithi077.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T10:24:27.321 INFO:journalctl@ceph.mon.b.smithi077.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T10:24:27.321 INFO:journalctl@ceph.mon.b.smithi077.stdout: Cumulative compaction: 0.00 GB write, 0.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T10:24:27.321 INFO:journalctl@ceph.mon.b.smithi077.stdout: Interval compaction: 0.00 GB write, 0.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T10:24:27.321 INFO:journalctl@ceph.mon.b.smithi077.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T10:24:27.321 INFO:journalctl@ceph.mon.b.smithi077.stdout: Block cache BinnedLRUCache@0x55622e84b1f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T10:24:27.321 INFO:journalctl@ceph.mon.b.smithi077.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T10:24:27.322 INFO:journalctl@ceph.mon.b.smithi077.stdout: 2024-04-25T10:24:27.322 INFO:journalctl@ceph.mon.b.smithi077.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T10:24:27.322 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-25T10:24:27.322 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: using public_addr v2:172.21.15.77:0/0 -> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] 2024-04-25T10:24:27.322 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: starting mon.b rank -1 at public addrs [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] at bind addrs [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:24:27.322 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: mon.b@-1(???) e0 preinit fsid 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:24:27.322 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: mon.b@-1(synchronizing).mds e1 new map 2024-04-25T10:24:27.322 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-25T10:24:27.322 INFO:journalctl@ceph.mon.b.smithi077.stdout: e1 2024-04-25T10:24:27.322 INFO:journalctl@ceph.mon.b.smithi077.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T10:24:27.323 INFO:journalctl@ceph.mon.b.smithi077.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T10:24:27.323 INFO:journalctl@ceph.mon.b.smithi077.stdout: legacy client fscid: -1 2024-04-25T10:24:27.323 INFO:journalctl@ceph.mon.b.smithi077.stdout: 2024-04-25T10:24:27.323 INFO:journalctl@ceph.mon.b.smithi077.stdout: No filesystems configured 2024-04-25T10:24:27.323 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T10:24:27.323 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T10:24:27.324 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T10:24:27.324 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T10:24:27.324 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T10:24:27.324 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T10:24:27.324 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T10:24:27.324 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T10:24:27.324 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T10:24:27.324 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T10:24:27.324 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.324 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:27.324 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.324 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:24:27.325 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:27.325 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:24:27.325 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.325 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.325 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.325 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.325 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi077", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:24:27.325 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:27.325 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:27.325 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-25T10:24:27.325 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:27.325 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: Deploying cephadm binary to smithi164 2024-04-25T10:24:27.325 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: Updating smithi077:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:24:27.326 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:24:27.326 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:27.326 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.326 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.326 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.326 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.326 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:24:27.326 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: Updating smithi077:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:24:27.326 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: Added host smithi164 2024-04-25T10:24:27.326 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.326 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:27.326 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.326 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:27.326 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:24:27.327 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.327 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:27.327 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1593744685' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T10:24:27.327 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.327 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.327 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.327 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.327 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:24:27.327 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:27.327 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:27.327 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-25T10:24:27.327 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1593744685' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T10:24:27.327 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T10:24:27.327 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: Updating smithi164:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:24:27.328 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.328 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.328 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.328 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:24:27.328 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:27.328 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: Updating smithi164:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:24:27.328 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi077:172.21.15.77=b;smithi164:172.21.15.164=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:24:27.328 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi077:172.21.15.77=b;smithi164:172.21.15.164=c;count:3 2024-04-25T10:24:27.328 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.328 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:24:27.328 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:27.328 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:27.328 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.329 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:24:27.329 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:27.329 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:27.329 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: Deploying daemon mon.c on smithi164 2024-04-25T10:24:27.329 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:27.329 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:27.329 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='client.? 172.21.15.164:0/2634654589' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T10:24:27.329 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:24:27.329 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:27.329 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: mon.a calling monitor election 2024-04-25T10:24:27.329 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:27.329 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:27.329 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:27.330 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: mon.c calling monitor election 2024-04-25T10:24:27.330 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:27.330 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:27.330 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:27.330 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:27.330 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:27.330 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T10:24:27.330 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: monmap epoch 2 2024-04-25T10:24:27.330 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: fsid 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:24:27.330 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: last_changed 2024-04-25T10:24:17.905217+0000 2024-04-25T10:24:27.330 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: created 2024-04-25T10:22:33.876396+0000 2024-04-25T10:24:27.330 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: min_mon_release 19 (squid) 2024-04-25T10:24:27.331 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: election_strategy: 1 2024-04-25T10:24:27.331 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-04-25T10:24:27.331 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-04-25T10:24:27.331 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: fsmap 2024-04-25T10:24:27.331 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T10:24:27.331 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: mgrmap e13: x(active, since 50s) 2024-04-25T10:24:27.331 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: overall HEALTH_OK 2024-04-25T10:24:27.331 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.331 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.331 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:27.331 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:24:27.331 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:27.331 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: Deploying daemon mon.b on smithi077 2024-04-25T10:24:27.332 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='client.? 172.21.15.164:0/1560860585' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T10:24:27.332 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:27.332 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:27.332 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 ceph-mon[32772]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T10:24:27.332 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 bash[32752]: 6cbd105360e0ff4dbfcde6f79266cd0ff93d10e228654ea9cf9bfe8873647170 2024-04-25T10:24:27.332 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:26 smithi077 systemd[1]: Started Ceph mon.b for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:24:31.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:31.984+0000 7fb79a7fc700 1 -- 172.21.15.164:0/3352452296 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7fb7a000b960 con 0x7fb7a4108a40 2024-04-25T10:24:31.986 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-25T10:24:31.987 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":3,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","modified":"2024-04-25T10:24:26.973463Z","created":"2024-04-25T10:22:33.876396Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-25T10:24:31.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:31.988+0000 7fb7ab288700 1 -- 172.21.15.164:0/3352452296 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb79003d3b0 msgr2=0x7fb79003f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:31.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:31.988+0000 7fb7ab288700 1 --2- 172.21.15.164:0/3352452296 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb79003d3b0 0x7fb79003f8c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb7a406c8b0 tx=0x7fb794008040 comp rx=0 tx=0).stop 2024-04-25T10:24:31.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:31.988+0000 7fb7ab288700 1 -- 172.21.15.164:0/3352452296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7a4108a40 msgr2=0x7fb7a419f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:31.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:31.988+0000 7fb7ab288700 1 --2- 172.21.15.164:0/3352452296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7a4108a40 0x7fb7a419f6b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0006010 tx=0x7fb7a0005030 comp rx=0 tx=0).stop 2024-04-25T10:24:31.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:31.988+0000 7fb7ab288700 1 -- 172.21.15.164:0/3352452296 shutdown_connections 2024-04-25T10:24:31.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:31.988+0000 7fb7ab288700 1 --2- 172.21.15.164:0/3352452296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7a4108a40 0x7fb7a419f6b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:31.988+0000 7fb7ab288700 1 --2- 172.21.15.164:0/3352452296 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb79003d3b0 0x7fb79003f8c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:31.988+0000 7fb7ab288700 1 --2- 172.21.15.164:0/3352452296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7a419fbf0 0x7fb7a41998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:31.988+0000 7fb7ab288700 1 -- 172.21.15.164:0/3352452296 >> 172.21.15.164:0/3352452296 conn(0x7fb7a40796b0 msgr2=0x7fb7a4106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:31.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:31.988+0000 7fb7ab288700 1 -- 172.21.15.164:0/3352452296 shutdown_connections 2024-04-25T10:24:31.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:31.988+0000 7fb7ab288700 1 -- 172.21.15.164:0/3352452296 wait complete. 2024-04-25T10:24:31.991 INFO:teuthology.orchestra.run.smithi164.stderr:dumped monmap epoch 3 2024-04-25T10:24:32.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:24:32.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: mon.a calling monitor election 2024-04-25T10:24:32.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:32.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:32.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: mon.c calling monitor election 2024-04-25T10:24:32.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: from='client.? 172.21.15.164:0/3352452296' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T10:24:32.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:32.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:32.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:32.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: mon.b calling monitor election 2024-04-25T10:24:32.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:32.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:32.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:32.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:32.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T10:24:32.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: monmap epoch 3 2024-04-25T10:24:32.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: fsid 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:24:32.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: last_changed 2024-04-25T10:24:26.973463+0000 2024-04-25T10:24:32.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: created 2024-04-25T10:22:33.876396+0000 2024-04-25T10:24:32.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: min_mon_release 19 (squid) 2024-04-25T10:24:32.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: election_strategy: 1 2024-04-25T10:24:32.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-04-25T10:24:32.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-04-25T10:24:32.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: 2: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.b 2024-04-25T10:24:32.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: fsmap 2024-04-25T10:24:32.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T10:24:32.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: mgrmap e13: x(active, since 59s) 2024-04-25T10:24:32.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: overall HEALTH_OK 2024-04-25T10:24:32.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:32.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:32.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:32.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:32.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:24:32.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: mon.a calling monitor election 2024-04-25T10:24:32.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:32.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:32.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: mon.c calling monitor election 2024-04-25T10:24:32.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: from='client.? 172.21.15.164:0/3352452296' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T10:24:32.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:32.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:32.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:32.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: mon.b calling monitor election 2024-04-25T10:24:32.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:32.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:32.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:32.295 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:32.295 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T10:24:32.295 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: monmap epoch 3 2024-04-25T10:24:32.295 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: fsid 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:24:32.295 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: last_changed 2024-04-25T10:24:26.973463+0000 2024-04-25T10:24:32.295 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: created 2024-04-25T10:22:33.876396+0000 2024-04-25T10:24:32.295 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: min_mon_release 19 (squid) 2024-04-25T10:24:32.295 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: election_strategy: 1 2024-04-25T10:24:32.295 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-04-25T10:24:32.295 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-04-25T10:24:32.295 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: 2: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.b 2024-04-25T10:24:32.295 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: fsmap 2024-04-25T10:24:32.295 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T10:24:32.295 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: mgrmap e13: x(active, since 59s) 2024-04-25T10:24:32.295 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: overall HEALTH_OK 2024-04-25T10:24:32.296 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:32.296 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:32.296 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:32.296 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:32.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:24:32.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: mon.a calling monitor election 2024-04-25T10:24:32.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:32.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:24:32.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: mon.c calling monitor election 2024-04-25T10:24:32.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: from='client.? 172.21.15.164:0/3352452296' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T10:24:32.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:32.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:32.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:32.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: mon.b calling monitor election 2024-04-25T10:24:32.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:32.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:32.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:32.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:32.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T10:24:32.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: monmap epoch 3 2024-04-25T10:24:32.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: fsid 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:24:32.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: last_changed 2024-04-25T10:24:26.973463+0000 2024-04-25T10:24:32.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: created 2024-04-25T10:22:33.876396+0000 2024-04-25T10:24:32.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: min_mon_release 19 (squid) 2024-04-25T10:24:32.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: election_strategy: 1 2024-04-25T10:24:32.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-04-25T10:24:32.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-04-25T10:24:32.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: 2: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.b 2024-04-25T10:24:32.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: fsmap 2024-04-25T10:24:32.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T10:24:32.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: mgrmap e13: x(active, since 59s) 2024-04-25T10:24:32.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: overall HEALTH_OK 2024-04-25T10:24:32.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:32.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:32.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:32.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:32.600 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T10:24:32.600 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T10:24:32.855 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:24:33.259 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:33 smithi043 ceph-mon[29478]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:33.259 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:33.259 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:24:33.259 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:33.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:33 smithi077 ceph-mon[32772]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:33.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:33.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:24:33.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:33.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:33 smithi164 ceph-mon[32713]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:33.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:33.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:24:33.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:24:34.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.345+0000 7f613642f700 1 -- 172.21.15.43:0/3098323817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6130104c00 msgr2=0x7f6130105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:34.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.345+0000 7f613642f700 1 --2- 172.21.15.43:0/3098323817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6130104c00 0x7f6130105010 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6120009910 tx=0x7f6120009c20 comp rx=0 tx=0).stop 2024-04-25T10:24:34.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.346+0000 7f613642f700 1 -- 172.21.15.43:0/3098323817 shutdown_connections 2024-04-25T10:24:34.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.346+0000 7f613642f700 1 --2- 172.21.15.43:0/3098323817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6130104c00 0x7f6130105010 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:34.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.346+0000 7f613642f700 1 -- 172.21.15.43:0/3098323817 >> 172.21.15.43:0/3098323817 conn(0x7f6130078460 msgr2=0x7f61300788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:34.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.346+0000 7f613642f700 1 -- 172.21.15.43:0/3098323817 shutdown_connections 2024-04-25T10:24:34.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.346+0000 7f613642f700 1 -- 172.21.15.43:0/3098323817 wait complete. 2024-04-25T10:24:34.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.347+0000 7f613642f700 1 Processor -- start 2024-04-25T10:24:34.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.347+0000 7f613642f700 1 -- start start 2024-04-25T10:24:34.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.347+0000 7f613642f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6130104c00 0x7f613019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:34.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.347+0000 7f613642f700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f613019fc30 0x7f6130199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:34.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.347+0000 7f613642f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6130199e50 0x7f613019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:34.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.347+0000 7f613642f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f613010dff0 con 0x7f6130104c00 2024-04-25T10:24:34.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.347+0000 7f613642f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f613010de70 con 0x7f6130199e50 2024-04-25T10:24:34.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.347+0000 7f613642f700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f613010e170 con 0x7f613019fc30 2024-04-25T10:24:34.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.347+0000 7f612ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6130104c00 0x7f613019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:34.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.347+0000 7f612ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6130104c00 0x7f613019f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60190/0 (socket says 172.21.15.43:60190) 2024-04-25T10:24:34.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.347+0000 7f612ffff700 1 -- 172.21.15.43:0/1288294311 learned_addr learned my addr 172.21.15.43:0/1288294311 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:24:34.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.347+0000 7f612f7fe700 1 --2- 172.21.15.43:0/1288294311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f613019fc30 0x7f6130199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.348+0000 7f61349cc700 1 --2- 172.21.15.43:0/1288294311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6130199e50 0x7f613019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.348+0000 7f612ffff700 1 -- 172.21.15.43:0/1288294311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f613019fc30 msgr2=0x7f6130199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.348+0000 7f612ffff700 1 --2- 172.21.15.43:0/1288294311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f613019fc30 0x7f6130199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.348+0000 7f612ffff700 1 -- 172.21.15.43:0/1288294311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6130199e50 msgr2=0x7f613019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.348+0000 7f612ffff700 1 --2- 172.21.15.43:0/1288294311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6130199e50 0x7f613019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.348+0000 7f612ffff700 1 -- 172.21.15.43:0/1288294311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61301a6580 con 0x7f6130104c00 2024-04-25T10:24:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.348+0000 7f61349cc700 1 --2- 172.21.15.43:0/1288294311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6130199e50 0x7f613019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.348+0000 7f612ffff700 1 --2- 172.21.15.43:0/1288294311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6130104c00 0x7f613019f6f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6120005b40 tx=0x7f6120004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.348+0000 7f612d7fa700 1 -- 172.21.15.43:0/1288294311 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120026070 con 0x7f6130104c00 2024-04-25T10:24:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.348+0000 7f613642f700 1 -- 172.21.15.43:0/1288294311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61301a67b0 con 0x7f6130104c00 2024-04-25T10:24:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.348+0000 7f612d7fa700 1 -- 172.21.15.43:0/1288294311 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f612000b750 con 0x7f6130104c00 2024-04-25T10:24:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.348+0000 7f612d7fa700 1 -- 172.21.15.43:0/1288294311 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61200038e0 con 0x7f6130104c00 2024-04-25T10:24:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.348+0000 7f613642f700 1 -- 172.21.15.43:0/1288294311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61301a6c60 con 0x7f6130104c00 2024-04-25T10:24:34.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.349+0000 7f613642f700 1 -- 172.21.15.43:0/1288294311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6130064cf0 con 0x7f6130104c00 2024-04-25T10:24:34.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.349+0000 7f612d7fa700 1 -- 172.21.15.43:0/1288294311 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f6120029070 con 0x7f6130104c00 2024-04-25T10:24:34.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.350+0000 7f612d7fa700 1 --2- 172.21.15.43:0/1288294311 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f611803daa0 0x7f611803ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:34.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.350+0000 7f612d7fa700 1 -- 172.21.15.43:0/1288294311 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6120052300 con 0x7f6130104c00 2024-04-25T10:24:34.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.350+0000 7f612f7fe700 1 --2- 172.21.15.43:0/1288294311 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f611803daa0 0x7f611803ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:34.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.350+0000 7f612f7fe700 1 --2- 172.21.15.43:0/1288294311 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f611803daa0 0x7f611803ffb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6130106e70 tx=0x7f6124009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:34.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.355+0000 7f612d7fa700 1 -- 172.21.15.43:0/1288294311 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6120013070 con 0x7f6130104c00 2024-04-25T10:24:34.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.565+0000 7f613642f700 1 -- 172.21.15.43:0/1288294311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f61301a6ef0 con 0x7f6130104c00 2024-04-25T10:24:34.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:34.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:34.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-25T10:24:34.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-25T10:24:34.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-25T10:24:34.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: Updating smithi043:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:24:34.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: Updating smithi077:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:24:34.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: Updating smithi164:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:24:34.569 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.569 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.569 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.569 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.569 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.569 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.569 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.569 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.569 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.569 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.569 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.569 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:24:34.569 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T10:24:34.569 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:34.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.566+0000 7f612d7fa700 1 -- 172.21.15.43:0/1288294311 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f612002acc0 con 0x7f6130104c00 2024-04-25T10:24:34.570 INFO:teuthology.orchestra.run.smithi043.stdout:# minimal ceph.conf for 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:24:34.570 INFO:teuthology.orchestra.run.smithi043.stdout:[global] 2024-04-25T10:24:34.570 INFO:teuthology.orchestra.run.smithi043.stdout: fsid = 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:24:34.570 INFO:teuthology.orchestra.run.smithi043.stdout: mon_host = [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-25T10:24:34.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.568+0000 7f613642f700 1 -- 172.21.15.43:0/1288294311 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f611803daa0 msgr2=0x7f611803ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:34.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.568+0000 7f613642f700 1 --2- 172.21.15.43:0/1288294311 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f611803daa0 0x7f611803ffb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6130106e70 tx=0x7f6124009210 comp rx=0 tx=0).stop 2024-04-25T10:24:34.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.568+0000 7f613642f700 1 -- 172.21.15.43:0/1288294311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6130104c00 msgr2=0x7f613019f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:34.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.568+0000 7f613642f700 1 --2- 172.21.15.43:0/1288294311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6130104c00 0x7f613019f6f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6120005b40 tx=0x7f6120004ed0 comp rx=0 tx=0).stop 2024-04-25T10:24:34.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.568+0000 7f613642f700 1 -- 172.21.15.43:0/1288294311 shutdown_connections 2024-04-25T10:24:34.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.568+0000 7f613642f700 1 --2- 172.21.15.43:0/1288294311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6130199e50 0x7f613019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:34.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.568+0000 7f613642f700 1 --2- 172.21.15.43:0/1288294311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6130104c00 0x7f613019f6f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:34.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.568+0000 7f613642f700 1 --2- 172.21.15.43:0/1288294311 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f611803daa0 0x7f611803ffb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:34.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.568+0000 7f613642f700 1 --2- 172.21.15.43:0/1288294311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f613019fc30 0x7f6130199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:34.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.568+0000 7f613642f700 1 -- 172.21.15.43:0/1288294311 >> 172.21.15.43:0/1288294311 conn(0x7f6130078460 msgr2=0x7f6130109fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:34.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.568+0000 7f613642f700 1 -- 172.21.15.43:0/1288294311 shutdown_connections 2024-04-25T10:24:34.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:34.568+0000 7f613642f700 1 -- 172.21.15.43:0/1288294311 wait complete. 2024-04-25T10:24:34.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:34.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:34.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-25T10:24:34.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-25T10:24:34.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-25T10:24:34.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: Updating smithi043:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:24:34.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: Updating smithi077:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:24:34.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: Updating smithi164:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:24:34.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:24:34.793 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T10:24:34.793 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:34.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:34.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:34.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-25T10:24:34.794 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-25T10:24:34.794 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-25T10:24:34.794 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: Updating smithi043:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:24:34.794 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: Updating smithi077:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:24:34.794 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: Updating smithi164:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:24:34.794 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.794 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.794 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.794 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.794 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.794 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.794 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.794 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.795 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.795 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.795 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:34.795 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:24:34.795 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T10:24:34.795 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:35.130 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T10:24:35.131 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T10:24:35.131 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T10:24:35.159 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T10:24:35.159 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:24:35.224 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-25T10:24:35.224 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T10:24:35.255 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-25T10:24:35.256 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:24:35.324 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T10:24:35.324 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T10:24:35.355 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T10:24:35.355 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:24:35.422 INFO:tasks.cephadm:Adding mgr.x on smithi043 2024-04-25T10:24:35.422 INFO:tasks.cephadm:Adding mgr.y on smithi077 2024-04-25T10:24:35.422 INFO:tasks.cephadm:Adding mgr.z on smithi164 2024-04-25T10:24:35.422 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi043=x;smithi077=y;smithi164=z' 2024-04-25T10:24:35.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:35 smithi077 ceph-mon[32772]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:35.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:35 smithi077 ceph-mon[32772]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T10:24:35.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:35 smithi077 ceph-mon[32772]: Reconfiguring daemon mon.a on smithi043 2024-04-25T10:24:35.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:35 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1288294311' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:35.557 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:35 smithi043 ceph-mon[29478]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:35.557 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:35 smithi043 ceph-mon[29478]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T10:24:35.557 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:35 smithi043 ceph-mon[29478]: Reconfiguring daemon mon.a on smithi043 2024-04-25T10:24:35.557 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:35 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1288294311' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:35.694 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:24:35.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:35 smithi164 ceph-mon[32713]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:35.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:35 smithi164 ceph-mon[32713]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T10:24:35.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:35 smithi164 ceph-mon[32713]: Reconfiguring daemon mon.a on smithi043 2024-04-25T10:24:35.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:35 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1288294311' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:37.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.178+0000 7f8910b9a700 1 -- 172.21.15.164:0/2934578981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f890c102220 msgr2=0x7f890c102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:37.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.178+0000 7f8910b9a700 1 --2- 172.21.15.164:0/2934578981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f890c102220 0x7f890c102630 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009910 tx=0x7f88fc009c20 comp rx=0 tx=0).stop 2024-04-25T10:24:37.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.179+0000 7f8910b9a700 1 -- 172.21.15.164:0/2934578981 shutdown_connections 2024-04-25T10:24:37.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.179+0000 7f8910b9a700 1 --2- 172.21.15.164:0/2934578981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f890c102220 0x7f890c102630 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.179+0000 7f8910b9a700 1 -- 172.21.15.164:0/2934578981 >> 172.21.15.164:0/2934578981 conn(0x7f890c0fd860 msgr2=0x7f890c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:37.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.179+0000 7f8910b9a700 1 -- 172.21.15.164:0/2934578981 shutdown_connections 2024-04-25T10:24:37.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.179+0000 7f8910b9a700 1 -- 172.21.15.164:0/2934578981 wait complete. 2024-04-25T10:24:37.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.179+0000 7f8910b9a700 1 Processor -- start 2024-04-25T10:24:37.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.180+0000 7f8910b9a700 1 -- start start 2024-04-25T10:24:37.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.180+0000 7f8910b9a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f890c102220 0x7f890c0ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:37.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.180+0000 7f8910b9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f890c19e9e0 0x7f890c1a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:37.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.180+0000 7f8910b9a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f890c1a1340 0x7f890c1a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:37.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.180+0000 7f8910b9a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1081e0 con 0x7f890c102220 2024-04-25T10:24:37.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.180+0000 7f8910b9a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f890c108060 con 0x7f890c1a1340 2024-04-25T10:24:37.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.180+0000 7f8910b9a700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f890c108360 con 0x7f890c19e9e0 2024-04-25T10:24:37.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.180+0000 7f8901d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f890c19e9e0 0x7f890c1a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:37.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.180+0000 7f8901d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f890c19e9e0 0x7f890c1a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53262/0 (socket says 172.21.15.164:53262) 2024-04-25T10:24:37.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.180+0000 7f8901d9b700 1 -- 172.21.15.164:0/1861444806 learned_addr learned my addr 172.21.15.164:0/1861444806 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-25T10:24:37.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.180+0000 7f890a59c700 1 --2- 172.21.15.164:0/1861444806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f890c102220 0x7f890c0ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:37.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.180+0000 7f890ad9d700 1 --2- 172.21.15.164:0/1861444806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f890c1a1340 0x7f890c1a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:37.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.180+0000 7f8901d9b700 1 -- 172.21.15.164:0/1861444806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f890c1a1340 msgr2=0x7f890c1a3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:37.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.180+0000 7f8901d9b700 1 --2- 172.21.15.164:0/1861444806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f890c1a1340 0x7f890c1a3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.180+0000 7f8901d9b700 1 -- 172.21.15.164:0/1861444806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f890c102220 msgr2=0x7f890c0ffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:37.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.180+0000 7f8901d9b700 1 --2- 172.21.15.164:0/1861444806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f890c102220 0x7f890c0ffa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.180+0000 7f8901d9b700 1 -- 172.21.15.164:0/1861444806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890c1a3dd0 con 0x7f890c19e9e0 2024-04-25T10:24:37.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.180+0000 7f8901d9b700 1 --2- 172.21.15.164:0/1861444806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f890c19e9e0 0x7f890c1a0e00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f88f400b780 tx=0x7f88f400ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:37.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.181+0000 7f8903fff700 1 -- 172.21.15.164:0/1861444806 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4016070 con 0x7f890c19e9e0 2024-04-25T10:24:37.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.181+0000 7f8903fff700 1 -- 172.21.15.164:0/1861444806 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88f40036a0 con 0x7f890c19e9e0 2024-04-25T10:24:37.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.181+0000 7f8910b9a700 1 -- 172.21.15.164:0/1861444806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890c1a40c0 con 0x7f890c19e9e0 2024-04-25T10:24:37.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.181+0000 7f8903fff700 1 -- 172.21.15.164:0/1861444806 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f40119c0 con 0x7f890c19e9e0 2024-04-25T10:24:37.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.181+0000 7f8910b9a700 1 -- 172.21.15.164:0/1861444806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f890c1a4680 con 0x7f890c19e9e0 2024-04-25T10:24:37.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.182+0000 7f8903fff700 1 -- 172.21.15.164:0/1861444806 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f88f4019070 con 0x7f890c19e9e0 2024-04-25T10:24:37.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.182+0000 7f8910b9a700 1 -- 172.21.15.164:0/1861444806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f890c064cf0 con 0x7f890c19e9e0 2024-04-25T10:24:37.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.182+0000 7f8903fff700 1 --2- 172.21.15.164:0/1861444806 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f88f803daa0 0x7f88f803ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:37.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.182+0000 7f8903fff700 1 -- 172.21.15.164:0/1861444806 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f88f4050100 con 0x7f890c19e9e0 2024-04-25T10:24:37.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.182+0000 7f890a59c700 1 --2- 172.21.15.164:0/1861444806 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f88f803daa0 0x7f88f803ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:37.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.183+0000 7f890a59c700 1 --2- 172.21.15.164:0/1861444806 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f88f803daa0 0x7f88f803ffb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f88fc006010 tx=0x7f88fc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:37.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.187+0000 7f8903fff700 1 -- 172.21.15.164:0/1861444806 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88f400f2e0 con 0x7f890c19e9e0 2024-04-25T10:24:37.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.398+0000 7f8910b9a700 1 -- 172.21.15.164:0/1861444806 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi043=x;smithi077=y;smithi164=z", "target": ["mon-mgr", ""]}) -- 0x7f890c063b60 con 0x7f88f803daa0 2024-04-25T10:24:37.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.405+0000 7f8903fff700 1 -- 172.21.15.164:0/1861444806 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f890c063b60 con 0x7f88f803daa0 2024-04-25T10:24:37.408 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled mgr update... 2024-04-25T10:24:37.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.408+0000 7f8910b9a700 1 -- 172.21.15.164:0/1861444806 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f88f803daa0 msgr2=0x7f88f803ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:37.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.408+0000 7f8910b9a700 1 --2- 172.21.15.164:0/1861444806 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f88f803daa0 0x7f88f803ffb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f88fc006010 tx=0x7f88fc01a040 comp rx=0 tx=0).stop 2024-04-25T10:24:37.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.408+0000 7f8910b9a700 1 -- 172.21.15.164:0/1861444806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f890c19e9e0 msgr2=0x7f890c1a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:37.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.408+0000 7f8910b9a700 1 --2- 172.21.15.164:0/1861444806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f890c19e9e0 0x7f890c1a0e00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f88f400b780 tx=0x7f88f400ba90 comp rx=0 tx=0).stop 2024-04-25T10:24:37.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.408+0000 7f8910b9a700 1 -- 172.21.15.164:0/1861444806 shutdown_connections 2024-04-25T10:24:37.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.408+0000 7f8910b9a700 1 --2- 172.21.15.164:0/1861444806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f890c1a1340 0x7f890c1a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.408+0000 7f8910b9a700 1 --2- 172.21.15.164:0/1861444806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f890c102220 0x7f890c0ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.408+0000 7f8910b9a700 1 --2- 172.21.15.164:0/1861444806 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f88f803daa0 0x7f88f803ffb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.408+0000 7f8910b9a700 1 --2- 172.21.15.164:0/1861444806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f890c19e9e0 0x7f890c1a0e00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.408+0000 7f8910b9a700 1 -- 172.21.15.164:0/1861444806 >> 172.21.15.164:0/1861444806 conn(0x7f890c0fd860 msgr2=0x7f890c0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:37.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.408+0000 7f8910b9a700 1 -- 172.21.15.164:0/1861444806 shutdown_connections 2024-04-25T10:24:37.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:24:37.408+0000 7f8910b9a700 1 -- 172.21.15.164:0/1861444806 wait complete. 2024-04-25T10:24:37.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:37 smithi077 ceph-mon[32772]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:37.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:37 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:37.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:37 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:37.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:37 smithi077 ceph-mon[32772]: Reconfiguring mon.b (monmap changed)... 2024-04-25T10:24:37.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:37 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:24:37.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:37 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T10:24:37.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:37 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:37.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:37 smithi077 ceph-mon[32772]: Reconfiguring daemon mon.b on smithi077 2024-04-25T10:24:37.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:37 smithi164 ceph-mon[32713]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:37.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:37 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:37.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:37 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:37.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:37 smithi164 ceph-mon[32713]: Reconfiguring mon.b (monmap changed)... 2024-04-25T10:24:37.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:37 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:24:37.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:37 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T10:24:37.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:37 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:37.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:37 smithi164 ceph-mon[32713]: Reconfiguring daemon mon.b on smithi077 2024-04-25T10:24:37.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:37 smithi043 ceph-mon[29478]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:37.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:37 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:37.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:37 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:37.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:37 smithi043 ceph-mon[29478]: Reconfiguring mon.b (monmap changed)... 2024-04-25T10:24:37.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:37 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:24:37.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:37 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T10:24:37.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:37 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:37.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:37 smithi043 ceph-mon[29478]: Reconfiguring daemon mon.b on smithi077 2024-04-25T10:24:38.093 DEBUG:teuthology.orchestra.run.smithi077:mgr.y> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T10:24:38.096 DEBUG:teuthology.orchestra.run.smithi164:mgr.z> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T10:24:38.098 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T10:24:38.098 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T10:24:38.098 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:24:38.120 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:24:38.120 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2024-04-25T10:24:38.179 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:24:38.179 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:24:38.179 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T10:24:38.179 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:24:38.179 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:24:38.179 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-25 10:23:42.197667064 +0000 2024-04-25T10:24:38.180 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-25 10:17:31.882186429 +0000 2024-04-25T10:24:38.180 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-25 10:17:31.882186429 +0000 2024-04-25T10:24:38.180 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-25 10:17:31.882186429 +0000 2024-04-25T10:24:38.180 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:24:38.249 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T10:24:38.250 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T10:24:38.250 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000121478 s, 4.2 MB/s 2024-04-25T10:24:38.251 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:24:38.309 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2024-04-25T10:24:38.367 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:24:38.367 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:24:38.367 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T10:24:38.367 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:24:38.367 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:24:38.367 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-25 10:23:42.228666695 +0000 2024-04-25T10:24:38.367 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-25 10:17:32.234182235 +0000 2024-04-25T10:24:38.367 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-25 10:17:32.234182235 +0000 2024-04-25T10:24:38.367 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-25 10:17:32.234182235 +0000 2024-04-25T10:24:38.368 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:24:38.432 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T10:24:38.432 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T10:24:38.432 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000112971 s, 4.5 MB/s 2024-04-25T10:24:38.433 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:24:38.491 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2024-04-25T10:24:38.547 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:24:38.547 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:24:38.547 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T10:24:38.547 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:24:38.547 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:24:38.548 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-25 10:23:42.255666373 +0000 2024-04-25T10:24:38.548 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-25 10:17:32.559178362 +0000 2024-04-25T10:24:38.548 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-25 10:17:32.559178362 +0000 2024-04-25T10:24:38.548 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-25 10:17:32.559178362 +0000 2024-04-25T10:24:38.548 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:24:38.609 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:38 smithi043 ceph-mon[29478]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi043=x;smithi077=y;smithi164=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:24:38.609 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:38 smithi043 ceph-mon[29478]: Saving service mgr spec with placement smithi043=x;smithi077=y;smithi164=z;count:3 2024-04-25T10:24:38.609 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:38 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:38.609 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:38 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:38.609 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:38 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:38.609 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:38 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:24:38.609 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:38 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T10:24:38.609 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:38 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:38.611 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T10:24:38.611 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T10:24:38.611 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 9.7026e-05 s, 5.3 MB/s 2024-04-25T10:24:38.612 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:24:38.670 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2024-04-25T10:24:38.675 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:38 smithi164 ceph-mon[32713]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi043=x;smithi077=y;smithi164=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:24:38.675 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:38 smithi164 ceph-mon[32713]: Saving service mgr spec with placement smithi043=x;smithi077=y;smithi164=z;count:3 2024-04-25T10:24:38.675 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:38 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:38.675 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:38 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:38.675 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:38 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:38.675 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:38 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:24:38.675 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:38 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T10:24:38.676 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:38 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:38.725 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:24:38.726 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:24:38.726 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T10:24:38.726 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:24:38.726 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:24:38.726 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-25 10:23:42.282666051 +0000 2024-04-25T10:24:38.726 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-25 10:17:32.895174359 +0000 2024-04-25T10:24:38.726 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-25 10:17:32.895174359 +0000 2024-04-25T10:24:38.727 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-25 10:17:32.895174359 +0000 2024-04-25T10:24:38.727 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:24:38.789 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T10:24:38.789 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T10:24:38.790 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.0001146 s, 4.5 MB/s 2024-04-25T10:24:38.791 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:24:38.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:38 smithi077 ceph-mon[32772]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi043=x;smithi077=y;smithi164=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:24:38.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:38 smithi077 ceph-mon[32772]: Saving service mgr spec with placement smithi043=x;smithi077=y;smithi164=z;count:3 2024-04-25T10:24:38.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:38 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:38.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:38 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:38.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:38 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:38.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:38 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T10:24:38.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:38 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T10:24:38.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:38 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:38.847 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-25T10:24:38.847 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:24:38.866 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:24:38.866 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_1 2024-04-25T10:24:38.923 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:24:38.923 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:24:38.923 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T10:24:38.923 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:24:38.923 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:24:38.924 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-04-25 10:24:26.363506742 +0000 2024-04-25T10:24:38.924 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-04-25 10:19:04.997832637 +0000 2024-04-25T10:24:38.924 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-04-25 10:19:04.997832637 +0000 2024-04-25T10:24:38.924 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-04-25 10:19:04.997832637 +0000 2024-04-25T10:24:38.924 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:24:38.989 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-04-25T10:24:38.989 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-04-25T10:24:38.989 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000285472 s, 1.8 MB/s 2024-04-25T10:24:38.991 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:24:39.048 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_2 2024-04-25T10:24:39.104 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:24:39.104 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:24:39.104 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T10:24:39.104 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:24:39.104 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:24:39.104 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-04-25 10:24:26.364506723 +0000 2024-04-25T10:24:39.104 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-04-25 10:19:05.333825939 +0000 2024-04-25T10:24:39.104 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-04-25 10:19:05.333825939 +0000 2024-04-25T10:24:39.104 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-04-25 10:19:05.333825939 +0000 2024-04-25T10:24:39.104 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:24:39.168 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-04-25T10:24:39.168 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-04-25T10:24:39.168 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000233289 s, 2.2 MB/s 2024-04-25T10:24:39.170 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:24:39.226 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_3 2024-04-25T10:24:39.283 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:24:39.283 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:24:39.283 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-25T10:24:39.283 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:24:39.283 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:24:39.283 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-04-25 10:24:26.364506723 +0000 2024-04-25T10:24:39.284 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-04-25 10:19:05.663819361 +0000 2024-04-25T10:24:39.284 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-04-25 10:19:05.663819361 +0000 2024-04-25T10:24:39.284 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-04-25 10:19:05.663819361 +0000 2024-04-25T10:24:39.284 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:24:39.346 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-04-25T10:24:39.346 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-04-25T10:24:39.346 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000204884 s, 2.5 MB/s 2024-04-25T10:24:39.347 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:24:39.404 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_4 2024-04-25T10:24:39.459 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:24:39.459 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:24:39.459 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T10:24:39.459 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:24:39.459 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:24:39.459 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-04-25 10:24:26.365506704 +0000 2024-04-25T10:24:39.459 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-04-25 10:19:06.010812445 +0000 2024-04-25T10:24:39.459 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-04-25 10:19:06.010812445 +0000 2024-04-25T10:24:39.459 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-04-25 10:19:06.010812445 +0000 2024-04-25T10:24:39.460 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:24:39.521 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-04-25T10:24:39.521 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-04-25T10:24:39.521 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000177851 s, 2.9 MB/s 2024-04-25T10:24:39.522 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:24:39.577 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-25T10:24:39.577 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:24:39.594 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:24:39.594 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2024-04-25T10:24:39.649 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:24:39.649 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:24:39.649 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T10:24:39.649 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:24:39.649 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:24:39.649 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-25 10:24:16.880917824 +0000 2024-04-25T10:24:39.649 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-25 10:19:05.594836679 +0000 2024-04-25T10:24:39.649 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-25 10:19:05.594836679 +0000 2024-04-25T10:24:39.649 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-25 10:19:05.594836679 +0000 2024-04-25T10:24:39.650 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:24:39.709 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:39 smithi164 ceph-mon[32713]: Reconfiguring mon.c (monmap changed)... 2024-04-25T10:24:39.709 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:39 smithi164 ceph-mon[32713]: Reconfiguring daemon mon.c on smithi164 2024-04-25T10:24:39.709 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:39 smithi164 ceph-mon[32713]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:39.712 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-25T10:24:39.712 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-25T10:24:39.712 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000265855 s, 1.9 MB/s 2024-04-25T10:24:39.713 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:24:39.770 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2024-04-25T10:24:39.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:39 smithi077 ceph-mon[32772]: Reconfiguring mon.c (monmap changed)... 2024-04-25T10:24:39.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:39 smithi077 ceph-mon[32772]: Reconfiguring daemon mon.c on smithi164 2024-04-25T10:24:39.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:39 smithi077 ceph-mon[32772]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:39.824 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:24:39.825 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:24:39.825 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-25T10:24:39.825 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:24:39.825 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:24:39.825 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-25 10:24:16.881917808 +0000 2024-04-25T10:24:39.825 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-25 10:19:05.930831141 +0000 2024-04-25T10:24:39.825 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-25 10:19:05.930831141 +0000 2024-04-25T10:24:39.825 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-25 10:19:05.930831141 +0000 2024-04-25T10:24:39.825 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:24:39.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:39 smithi043 ceph-mon[29478]: Reconfiguring mon.c (monmap changed)... 2024-04-25T10:24:39.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:39 smithi043 ceph-mon[29478]: Reconfiguring daemon mon.c on smithi164 2024-04-25T10:24:39.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:39 smithi043 ceph-mon[29478]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:39.887 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-25T10:24:39.887 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-25T10:24:39.887 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000238095 s, 2.2 MB/s 2024-04-25T10:24:39.888 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:24:39.944 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2024-04-25T10:24:39.999 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:24:39.999 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:24:39.999 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-25T10:24:39.999 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:24:39.999 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:24:39.999 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-25 10:24:16.881917808 +0000 2024-04-25T10:24:39.999 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-25 10:19:06.262825669 +0000 2024-04-25T10:24:39.999 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-25 10:19:06.262825669 +0000 2024-04-25T10:24:39.999 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-25 10:19:06.262825669 +0000 2024-04-25T10:24:39.999 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:24:40.061 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-25T10:24:40.061 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-25T10:24:40.061 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000225046 s, 2.3 MB/s 2024-04-25T10:24:40.062 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:24:40.118 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2024-04-25T10:24:40.173 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:24:40.173 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:24:40.173 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-25T10:24:40.173 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:24:40.173 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:24:40.174 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-25 10:24:16.882917792 +0000 2024-04-25T10:24:40.174 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-25 10:19:06.598820131 +0000 2024-04-25T10:24:40.174 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-25 10:19:06.598820131 +0000 2024-04-25T10:24:40.174 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-25 10:19:06.598820131 +0000 2024-04-25T10:24:40.174 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:24:40.237 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-25T10:24:40.237 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-25T10:24:40.237 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.00021697 s, 2.4 MB/s 2024-04-25T10:24:40.238 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:24:40.295 INFO:tasks.cephadm:Deploying osd.0 on smithi043 with /dev/vg_nvme/lv_4... 2024-04-25T10:24:40.296 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T10:24:40.534 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:24:40.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:40 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:40.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:40 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:40.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:40 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:24:40.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:40 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:40.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:40 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:40.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:40 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:40.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:40 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T10:24:40.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:40 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T10:24:40.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:40 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T10:24:40.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:40 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:40.688 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:40 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:40.688 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:40 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:40.689 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:40 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:24:40.689 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:40 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:40.689 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:40 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:40.689 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:40 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:40.689 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:40 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T10:24:40.689 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:40 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T10:24:40.689 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:40 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T10:24:40.689 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:40 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:40.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:40 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:40.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:40 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:40.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:40 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:24:40.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:40 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:40.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:40 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:40.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:40 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:40.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:40 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T10:24:40.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:40 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T10:24:40.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:40 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T10:24:40.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:40 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:41.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:41 smithi043 ceph-mon[29478]: Deploying daemon mgr.z on smithi164 2024-04-25T10:24:41.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:41 smithi043 ceph-mon[29478]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:41.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:41 smithi164 ceph-mon[32713]: Deploying daemon mgr.z on smithi164 2024-04-25T10:24:41.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:41 smithi164 ceph-mon[32713]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:41.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:41 smithi077 ceph-mon[32772]: Deploying daemon mgr.z on smithi164 2024-04-25T10:24:41.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:41 smithi077 ceph-mon[32772]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T10:24:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T10:24:42.539 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-04-25T10:24:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-04-25T10:24:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00819577 s, 1.3 GB/s 2024-04-25T10:24:42.540 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-04-25T10:24:42.644 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:42 smithi164 systemd[1]: Started Ceph mgr.z for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:24:42.942 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:42 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:42.642+0000 7fe1b57d4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:24:42.942 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:42 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:42.833+0000 7fe1b57d4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:24:42.942 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:42 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:42.938+0000 7fe1b57d4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:24:43.186 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_4 2024-04-25T10:24:43.429 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:24:43.541 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:43 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:43.260+0000 7fe1b57d4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:24:43.541 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:43 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:43.345+0000 7fe1b57d4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:24:43.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:43 smithi043 ceph-mon[29478]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:43.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:43 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:43.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:43 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:43.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:43 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:43.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:43 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T10:24:43.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:43 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T10:24:43.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:43 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T10:24:43.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:43 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:43.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:43 smithi043 ceph-mon[29478]: Deploying daemon mgr.y on smithi077 2024-04-25T10:24:44.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:43 smithi077 ceph-mon[32772]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:44.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:43 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:44.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:43 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:44.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:43 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:44.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:43 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T10:24:44.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:43 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T10:24:44.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:43 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T10:24:44.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:43 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:44.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:43 smithi077 ceph-mon[32772]: Deploying daemon mgr.y on smithi077 2024-04-25T10:24:44.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:43 smithi164 ceph-mon[32713]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:44.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:43 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:44.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:43 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:44.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:43 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:44.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:43 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T10:24:44.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:43 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T10:24:44.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:43 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T10:24:44.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:43 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:44.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:43 smithi164 ceph-mon[32713]: Deploying daemon mgr.y on smithi077 2024-04-25T10:24:44.739 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:44 smithi077 systemd[1]: Starting Ceph mgr.y for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:24:44.739 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:44 smithi077 podman[34136]: 2024-04-25 10:24:44.640417552 +0000 UTC m=+0.018249432 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:24:44.901 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:44 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:44.556+0000 7fe1b57d4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:24:44.902 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:44 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:44.727+0000 7fe1b57d4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:24:44.902 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:44 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:44.899+0000 7fe1b57d4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:24:45.011 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:44 smithi077 podman[34136]: 2024-04-25 10:24:44.770789022 +0000 UTC m=+0.148620893 container create 8e9cc76a10c1f8d0eae9506889f4e295104d5e4a2f285340447c35fd72191598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T10:24:45.011 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:44 smithi077 podman[34136]: 2024-04-25 10:24:44.972814787 +0000 UTC m=+0.350646659 container init 8e9cc76a10c1f8d0eae9506889f4e295104d5e4a2f285340447c35fd72191598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T10:24:45.011 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:44 smithi077 podman[34136]: 2024-04-25 10:24:44.975392826 +0000 UTC m=+0.353224706 container start 8e9cc76a10c1f8d0eae9506889f4e295104d5e4a2f285340447c35fd72191598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T10:24:45.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.046+0000 7f7752547700 1 -- 172.21.15.43:0/2851270793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f774c104210 msgr2=0x7f774c104620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:45.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.046+0000 7f7752547700 1 --2- 172.21.15.43:0/2851270793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f774c104210 0x7f774c104620 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f773c009990 tx=0x7f773c009ca0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.046+0000 7f7752547700 1 -- 172.21.15.43:0/2851270793 shutdown_connections 2024-04-25T10:24:45.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.046+0000 7f7752547700 1 --2- 172.21.15.43:0/2851270793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f774c105ce0 0x7f774c10c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.046+0000 7f7752547700 1 --2- 172.21.15.43:0/2851270793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f774c104210 0x7f774c104620 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.046+0000 7f7752547700 1 --2- 172.21.15.43:0/2851270793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f774c105340 0x7f774c105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.046+0000 7f7752547700 1 -- 172.21.15.43:0/2851270793 >> 172.21.15.43:0/2851270793 conn(0x7f774c0ff870 msgr2=0x7f774c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:45.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.046+0000 7f7752547700 1 -- 172.21.15.43:0/2851270793 shutdown_connections 2024-04-25T10:24:45.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.046+0000 7f7752547700 1 -- 172.21.15.43:0/2851270793 wait complete. 2024-04-25T10:24:45.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.047+0000 7f7752547700 1 Processor -- start 2024-04-25T10:24:45.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.047+0000 7f7752547700 1 -- start start 2024-04-25T10:24:45.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.047+0000 7f7752547700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f774c104210 0x7f774c19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:45.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.047+0000 7f7752547700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f774c105340 0x7f774c19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.047+0000 7f7752547700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f774c105ce0 0x7f774c1a3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.047+0000 7f7752547700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f774c10fca0 con 0x7f774c105340 2024-04-25T10:24:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.048+0000 7f7752547700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f774c10fb20 con 0x7f774c105ce0 2024-04-25T10:24:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.048+0000 7f7752547700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f774c10fe20 con 0x7f774c104210 2024-04-25T10:24:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.048+0000 7f774bfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f774c104210 0x7f774c19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.048+0000 7f774b7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f774c105340 0x7f774c19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.048+0000 7f774bfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f774c104210 0x7f774c19ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:37180/0 (socket says 172.21.15.43:37180) 2024-04-25T10:24:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.048+0000 7f774bfff700 1 -- 172.21.15.43:0/306637108 learned_addr learned my addr 172.21.15.43:0/306637108 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:24:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.048+0000 7f7750ae4700 1 --2- 172.21.15.43:0/306637108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f774c105ce0 0x7f774c1a3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.048+0000 7f774bfff700 1 -- 172.21.15.43:0/306637108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f774c105ce0 msgr2=0x7f774c1a3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.048+0000 7f774bfff700 1 --2- 172.21.15.43:0/306637108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f774c105ce0 0x7f774c1a3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.048+0000 7f774bfff700 1 -- 172.21.15.43:0/306637108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f774c105340 msgr2=0x7f774c19cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.048+0000 7f774bfff700 1 --2- 172.21.15.43:0/306637108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f774c105340 0x7f774c19cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.048+0000 7f774bfff700 1 -- 172.21.15.43:0/306637108 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774c1a4580 con 0x7f774c104210 2024-04-25T10:24:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.048+0000 7f774b7fe700 1 --2- 172.21.15.43:0/306637108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f774c105340 0x7f774c19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:24:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.048+0000 7f774bfff700 1 --2- 172.21.15.43:0/306637108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f774c104210 0x7f774c19ca20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f773c00b6c0 tx=0x7f773c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.048+0000 7f77497fa700 1 -- 172.21.15.43:0/306637108 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c026070 con 0x7f774c104210 2024-04-25T10:24:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.048+0000 7f77497fa700 1 -- 172.21.15.43:0/306637108 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f773c021410 con 0x7f774c104210 2024-04-25T10:24:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.048+0000 7f7752547700 1 -- 172.21.15.43:0/306637108 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774c1a4810 con 0x7f774c104210 2024-04-25T10:24:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.048+0000 7f77497fa700 1 -- 172.21.15.43:0/306637108 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c017650 con 0x7f774c104210 2024-04-25T10:24:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.048+0000 7f7752547700 1 -- 172.21.15.43:0/306637108 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f774c1a4cc0 con 0x7f774c104210 2024-04-25T10:24:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.049+0000 7f77497fa700 1 -- 172.21.15.43:0/306637108 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f773c029070 con 0x7f774c104210 2024-04-25T10:24:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.049+0000 7f7752547700 1 -- 172.21.15.43:0/306637108 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f774c051190 con 0x7f774c104210 2024-04-25T10:24:45.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.050+0000 7f77497fa700 1 --2- 172.21.15.43:0/306637108 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f773403d850 0x7f773403fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:45.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.050+0000 7f77497fa700 1 -- 172.21.15.43:0/306637108 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f773c052560 con 0x7f774c104210 2024-04-25T10:24:45.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.050+0000 7f774b7fe700 1 --2- 172.21.15.43:0/306637108 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f773403d850 0x7f773403fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:45.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.050+0000 7f774b7fe700 1 --2- 172.21.15.43:0/306637108 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f773403d850 0x7f773403fd60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f774c19dad0 tx=0x7f7740009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:45.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.055+0000 7f77497fa700 1 -- 172.21.15.43:0/306637108 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f773c013070 con 0x7f774c104210 2024-04-25T10:24:45.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:45.281+0000 7f7752547700 1 -- 172.21.15.43:0/306637108 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f774c0780d0 con 0x7f773403d850 2024-04-25T10:24:45.290 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:45 smithi077 ceph-mgr[34155]: -- 172.21.15.77:0/2879963379 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5569ec3225a0 con 0x5569f10ac000 2024-04-25T10:24:45.290 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:45 smithi077 bash[34136]: 8e9cc76a10c1f8d0eae9506889f4e295104d5e4a2f285340447c35fd72191598 2024-04-25T10:24:45.291 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:45 smithi077 systemd[1]: Started Ceph mgr.y for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:24:45.291 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:45 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:45.136+0000 7fd7715f4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:24:45.490 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:45 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:45.186+0000 7fe1b57d4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:24:45.491 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:45 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:45.278+0000 7fe1b57d4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:24:45.491 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:45 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:45.488+0000 7fe1b57d4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:24:45.631 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:45 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:45.329+0000 7fd7715f4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:24:45.631 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:45 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:45.440+0000 7fd7715f4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:24:45.791 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:45 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:45.687+0000 7fe1b57d4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:24:45.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:45 smithi164 ceph-mon[32713]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:45.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:45 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:45.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:45 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:45.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:45 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:45.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:45 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:45.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:45 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:24:45.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:45 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:24:45.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:45 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:45.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:45 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:24:45.916 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:45 smithi043 ceph-mon[29478]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:45.917 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:45 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:45.917 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:45 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:45.917 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:45 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:45.917 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:45 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:45.917 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:45 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:24:45.917 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:45 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:24:45.917 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:45 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:45.917 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:45 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:24:46.040 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:45 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:45.771+0000 7fd7715f4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:24:46.041 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:45 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:45.864+0000 7fd7715f4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:24:46.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:45 smithi077 ceph-mon[32772]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:46.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:45 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:46.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:45 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:46.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:45 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:46.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:45 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:46.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:45 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:24:46.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:45 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:24:46.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:45 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:46.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:45 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:24:46.541 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:46 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:46.325+0000 7fe1b57d4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:24:46.822 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:46 smithi043 ceph-mon[29478]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:24:47.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:46 smithi077 ceph-mon[32772]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:24:47.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:46 smithi164 ceph-mon[32713]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:24:47.041 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:46 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:46.782+0000 7fe1b57d4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:24:47.041 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:46 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:46.875+0000 7fe1b57d4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:24:47.506 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:47 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:47.166+0000 7fd7715f4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:24:47.506 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:47 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:47.334+0000 7fd7715f4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:24:47.507 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:47 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:47.504+0000 7fd7715f4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:24:47.541 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:47 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:47.076+0000 7fe1b57d4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:24:47.922 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:47 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:47.592+0000 7fe1b57d4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:24:47.922 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:47 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:47.682+0000 7fe1b57d4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:24:47.922 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:47 smithi164 ceph-mon[32713]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:47.922 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:47 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:47.922 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:47 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:47.923 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:47 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:47.923 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:47 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:47.923 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:47 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:47.923 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:47 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:47.923 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:47 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:47.923 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:47 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T10:24:47.923 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:47 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T10:24:47.923 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:47 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:47.923 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:47 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:48.040 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:47 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:47.791+0000 7fd7715f4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:24:48.041 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:47 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:47.882+0000 7fd7715f4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:24:48.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:47 smithi077 ceph-mon[32772]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:48.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:47 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:48.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:47 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:48.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:47 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:48.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:47 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:48.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:47 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:48.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:47 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:48.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:47 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:48.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:47 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T10:24:48.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:47 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T10:24:48.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:47 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:48.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:47 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:47 smithi043 ceph-mon[29478]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:47 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:47 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:47 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:47 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:47 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:47 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:47 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:47 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T10:24:48.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:47 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T10:24:48.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:47 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:48.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:47 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:48.257 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:47 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:47.920+0000 7fe1b57d4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:24:48.257 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:48 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:48.050+0000 7fe1b57d4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:24:48.257 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:48 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:48.142+0000 7fe1b57d4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:24:48.258 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:48 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:48.254+0000 7fe1b57d4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:24:48.540 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:48 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:48.084+0000 7fd7715f4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:24:48.541 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:48 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:48.275+0000 7fd7715f4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:24:49.040 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:48 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:48.876+0000 7fd7715f4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:24:49.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:48 smithi077 ceph-mon[32772]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T10:24:49.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:48 smithi077 ceph-mon[32772]: Reconfiguring daemon mgr.x on smithi043 2024-04-25T10:24:49.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:48 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3114502847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05115646-db4d-4cfc-a5df-867435ee79ac"}]: dispatch 2024-04-25T10:24:49.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:48 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3114502847' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05115646-db4d-4cfc-a5df-867435ee79ac"}]': finished 2024-04-25T10:24:49.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:48 smithi077 ceph-mon[32772]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T10:24:49.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:48 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:24:49.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:48 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3656802358' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:24:49.042 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:48 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:48.558+0000 7fe1b57d4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:24:49.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:48 smithi164 ceph-mon[32713]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T10:24:49.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:48 smithi164 ceph-mon[32713]: Reconfiguring daemon mgr.x on smithi043 2024-04-25T10:24:49.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:48 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3114502847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05115646-db4d-4cfc-a5df-867435ee79ac"}]: dispatch 2024-04-25T10:24:49.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:48 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3114502847' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05115646-db4d-4cfc-a5df-867435ee79ac"}]': finished 2024-04-25T10:24:49.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:48 smithi164 ceph-mon[32713]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T10:24:49.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:48 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:24:49.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:48 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3656802358' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:24:49.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:48 smithi043 ceph-mon[29478]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T10:24:49.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:48 smithi043 ceph-mon[29478]: Reconfiguring daemon mgr.x on smithi043 2024-04-25T10:24:49.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:48 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3114502847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05115646-db4d-4cfc-a5df-867435ee79ac"}]: dispatch 2024-04-25T10:24:49.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:48 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3114502847' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05115646-db4d-4cfc-a5df-867435ee79ac"}]': finished 2024-04-25T10:24:49.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:48 smithi043 ceph-mon[29478]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T10:24:49.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:48 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:24:49.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:48 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3656802358' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:24:49.541 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:49 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:49.145+0000 7fe1b57d4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:24:49.541 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:24:49 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:24:49.353+0000 7fe1b57d4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:24:49.590 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:49 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:49.293+0000 7fd7715f4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:24:49.590 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:49 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:49.388+0000 7fd7715f4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:24:49.590 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:49 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:49.587+0000 7fd7715f4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:24:50.040 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:49 smithi043 ceph-mon[29478]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:50.040 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:49 smithi043 ceph-mon[29478]: Standby manager daemon z started 2024-04-25T10:24:50.040 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:49 smithi043 ceph-mon[29478]: from='mgr.? 172.21.15.164:0/3620014542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T10:24:50.040 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:49 smithi043 ceph-mon[29478]: from='mgr.? 172.21.15.164:0/3620014542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:24:50.040 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:49 smithi043 ceph-mon[29478]: from='mgr.? 172.21.15.164:0/3620014542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T10:24:50.040 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:49 smithi043 ceph-mon[29478]: from='mgr.? 172.21.15.164:0/3620014542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:24:50.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:50.036+0000 7f77497fa700 1 -- 172.21.15.43:0/306637108 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mgrmap(e 14) ==== 98518+0+0 (secure 0 0 0) 0x7f773c009d10 con 0x7f774c104210 2024-04-25T10:24:50.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:49 smithi077 ceph-mon[32772]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:50.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:49 smithi077 ceph-mon[32772]: Standby manager daemon z started 2024-04-25T10:24:50.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:49 smithi077 ceph-mon[32772]: from='mgr.? 172.21.15.164:0/3620014542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T10:24:50.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:49 smithi077 ceph-mon[32772]: from='mgr.? 172.21.15.164:0/3620014542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:24:50.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:49 smithi077 ceph-mon[32772]: from='mgr.? 172.21.15.164:0/3620014542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T10:24:50.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:49 smithi077 ceph-mon[32772]: from='mgr.? 172.21.15.164:0/3620014542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:24:50.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:49 smithi164 ceph-mon[32713]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:50.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:49 smithi164 ceph-mon[32713]: Standby manager daemon z started 2024-04-25T10:24:50.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:49 smithi164 ceph-mon[32713]: from='mgr.? 172.21.15.164:0/3620014542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T10:24:50.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:49 smithi164 ceph-mon[32713]: from='mgr.? 172.21.15.164:0/3620014542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:24:50.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:49 smithi164 ceph-mon[32713]: from='mgr.? 172.21.15.164:0/3620014542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T10:24:50.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:49 smithi164 ceph-mon[32713]: from='mgr.? 172.21.15.164:0/3620014542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:24:50.458 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:50 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:50.120+0000 7fd7715f4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:24:50.459 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:50 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:50.212+0000 7fd7715f4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:24:50.790 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:50 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:50.456+0000 7fd7715f4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:24:50.791 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:50 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:50.589+0000 7fd7715f4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:24:50.791 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:50 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:50.685+0000 7fd7715f4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:24:51.109 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:51 smithi077 ceph-mon[32772]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T10:24:51.110 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:51 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T10:24:51.110 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:51 smithi077 ceph-mon[32772]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:51.110 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:50 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:50.802+0000 7fd7715f4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:24:51.110 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:51 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:51.107+0000 7fd7715f4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:24:51.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:51 smithi164 ceph-mon[32713]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T10:24:51.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:51 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T10:24:51.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:51 smithi164 ceph-mon[32713]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:51.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:51 smithi043 ceph-mon[29478]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T10:24:51.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:51 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T10:24:51.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:51 smithi043 ceph-mon[29478]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:52.041 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:51 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:51.699+0000 7fd7715f4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:24:52.041 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:24:51 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:24:51.914+0000 7fd7715f4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:24:52.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:24:52.052+0000 7f77497fa700 1 -- 172.21.15.43:0/306637108 <== mon.1 v2:172.21.15.164:3300/0 8 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7f773c0218c0 con 0x7f774c104210 2024-04-25T10:24:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:52 smithi043 ceph-mon[29478]: Standby manager daemon y started 2024-04-25T10:24:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:52 smithi043 ceph-mon[29478]: from='mgr.? 172.21.15.77:0/3457350621' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T10:24:52.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:52 smithi043 ceph-mon[29478]: from='mgr.? 172.21.15.77:0/3457350621' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:24:52.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:52 smithi043 ceph-mon[29478]: from='mgr.? 172.21.15.77:0/3457350621' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T10:24:52.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:52 smithi043 ceph-mon[29478]: from='mgr.? 172.21.15.77:0/3457350621' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:24:52.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:52 smithi077 ceph-mon[32772]: Standby manager daemon y started 2024-04-25T10:24:52.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:52 smithi077 ceph-mon[32772]: from='mgr.? 172.21.15.77:0/3457350621' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T10:24:52.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:52 smithi077 ceph-mon[32772]: from='mgr.? 172.21.15.77:0/3457350621' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:24:52.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:52 smithi077 ceph-mon[32772]: from='mgr.? 172.21.15.77:0/3457350621' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T10:24:52.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:52 smithi077 ceph-mon[32772]: from='mgr.? 172.21.15.77:0/3457350621' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:24:52.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:52 smithi164 ceph-mon[32713]: Standby manager daemon y started 2024-04-25T10:24:52.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:52 smithi164 ceph-mon[32713]: from='mgr.? 172.21.15.77:0/3457350621' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T10:24:52.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:52 smithi164 ceph-mon[32713]: from='mgr.? 172.21.15.77:0/3457350621' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:24:52.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:52 smithi164 ceph-mon[32713]: from='mgr.? 172.21.15.77:0/3457350621' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T10:24:52.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:52 smithi164 ceph-mon[32713]: from='mgr.? 172.21.15.77:0/3457350621' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:24:53.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:53 smithi043 ceph-mon[29478]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-25T10:24:53.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:53 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T10:24:53.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:53 smithi043 ceph-mon[29478]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:53.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:53 smithi077 ceph-mon[32772]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-25T10:24:53.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:53 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T10:24:53.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:53 smithi077 ceph-mon[32772]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:53.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:53 smithi164 ceph-mon[32713]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-25T10:24:53.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:53 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T10:24:53.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:53 smithi164 ceph-mon[32713]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:54.851 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:54 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:54.851 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:54 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:54.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:54 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:24:54.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:54 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:54.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:54 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:54.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:54 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:55.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:54 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:55.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:54 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:55.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:54 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:24:55.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:54 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:55.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:54 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:55.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:54 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:55.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:54 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:55.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:54 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:55.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:54 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:24:55.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:54 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:55.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:54 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:24:55.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:54 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:24:56.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:55 smithi077 ceph-mon[32772]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:56.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:55 smithi164 ceph-mon[32713]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:56.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:55 smithi043 ceph-mon[29478]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:56.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:56 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T10:24:56.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:56 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:57.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:56 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T10:24:57.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:56 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:57.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:56 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T10:24:57.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:56 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:24:57.970 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:57 smithi043 ceph-mon[29478]: Deploying daemon osd.0 on smithi043 2024-04-25T10:24:57.970 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:57 smithi043 ceph-mon[29478]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:58.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:57 smithi077 ceph-mon[32772]: Deploying daemon osd.0 on smithi043 2024-04-25T10:24:58.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:57 smithi077 ceph-mon[32772]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:58.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:57 smithi164 ceph-mon[32713]: Deploying daemon osd.0 on smithi043 2024-04-25T10:24:58.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:57 smithi164 ceph-mon[32713]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:24:59.716 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:24:59 smithi043 ceph-mon[29478]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:25:00.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:24:59 smithi077 ceph-mon[32772]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:25:00.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:24:59 smithi164 ceph-mon[32713]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:25:01.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:01 smithi043 ceph-mon[29478]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:25:01.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:01 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:01.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:01 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:01.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:01 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:25:02.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:01 smithi077 ceph-mon[32772]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:25:02.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:01 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:02.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:01 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:02.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:01 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:25:02.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:01 smithi164 ceph-mon[32713]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:25:02.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:01 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:02.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:01 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:02.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:01 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:25:04.022 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:03 smithi043 ceph-mon[29478]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:25:04.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:03 smithi077 ceph-mon[32772]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:25:04.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:03 smithi164 ceph-mon[32713]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:25:05.255 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:05 smithi043 ceph-mon[29478]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:25:05.255 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:05 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:05.256 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:05 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:05.256 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:05 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:05.256 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:05 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:25:05.256 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:05 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:05.347 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 0 on host 'smithi043' 2024-04-25T10:25:05.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:05.344+0000 7f77497fa700 1 -- 172.21.15.43:0/306637108 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f774c0780d0 con 0x7f773403d850 2024-04-25T10:25:05.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:05.347+0000 7f7752547700 1 -- 172.21.15.43:0/306637108 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f773403d850 msgr2=0x7f773403fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:05.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:05.347+0000 7f7752547700 1 --2- 172.21.15.43:0/306637108 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f773403d850 0x7f773403fd60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f774c19dad0 tx=0x7f7740009260 comp rx=0 tx=0).stop 2024-04-25T10:25:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:05.347+0000 7f7752547700 1 -- 172.21.15.43:0/306637108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f774c104210 msgr2=0x7f774c19ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:05.347+0000 7f7752547700 1 --2- 172.21.15.43:0/306637108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f774c104210 0x7f774c19ca20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f773c00b6c0 tx=0x7f773c009580 comp rx=0 tx=0).stop 2024-04-25T10:25:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:05.347+0000 7f7752547700 1 -- 172.21.15.43:0/306637108 shutdown_connections 2024-04-25T10:25:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:05.347+0000 7f7752547700 1 --2- 172.21.15.43:0/306637108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f774c105ce0 0x7f774c1a3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:05.347+0000 7f7752547700 1 --2- 172.21.15.43:0/306637108 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f773403d850 0x7f773403fd60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:05.347+0000 7f7752547700 1 --2- 172.21.15.43:0/306637108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f774c104210 0x7f774c19ca20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:05.347+0000 7f7752547700 1 --2- 172.21.15.43:0/306637108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f774c105340 0x7f774c19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:05.347+0000 7f7752547700 1 -- 172.21.15.43:0/306637108 >> 172.21.15.43:0/306637108 conn(0x7f774c0ff870 msgr2=0x7f774c0769b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:05.347+0000 7f7752547700 1 -- 172.21.15.43:0/306637108 shutdown_connections 2024-04-25T10:25:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:05.347+0000 7f7752547700 1 -- 172.21.15.43:0/306637108 wait complete. 2024-04-25T10:25:05.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:05 smithi077 ceph-mon[32772]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:25:05.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:05 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:05.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:05 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:05.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:05 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:05.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:05 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:25:05.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:05 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:05.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:05 smithi164 ceph-mon[32713]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:25:05.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:05 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:05.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:05 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:05.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:05 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:05.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:05 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:25:05.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:05 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:05.869 DEBUG:teuthology.orchestra.run.smithi043:osd.0> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T10:25:05.874 INFO:tasks.cephadm:Deploying osd.1 on smithi077 with /dev/vg_nvme/lv_4... 2024-04-25T10:25:05.874 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T10:25:06.118 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T10:25:06.196 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:06 smithi043 ceph-mon[29478]: from='osd.0 [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T10:25:06.196 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:06 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:06.196 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:06 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:06.196 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:06 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:25:06.240 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:06 smithi077 ceph-mon[32772]: from='osd.0 [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T10:25:06.240 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:06 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:06.241 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:06 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:06.241 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:06 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:25:06.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:06 smithi164 ceph-mon[32713]: from='osd.0 [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T10:25:06.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:06 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:06.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:06 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:06.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:06 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:25:07.367 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:07 smithi043 ceph-mon[29478]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:25:07.367 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:07 smithi043 ceph-mon[29478]: from='osd.0 [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T10:25:07.367 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:07 smithi043 ceph-mon[29478]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T10:25:07.367 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:07 smithi043 ceph-mon[29478]: from='osd.0 [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-25T10:25:07.367 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:07 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:25:07.367 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:07 smithi043 ceph-mon[29478]: from='osd.0 [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-25T10:25:07.367 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:07 smithi043 ceph-mon[29478]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T10:25:07.402 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:07 smithi077 ceph-mon[32772]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:25:07.403 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:07 smithi077 ceph-mon[32772]: from='osd.0 [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T10:25:07.403 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:07 smithi077 ceph-mon[32772]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T10:25:07.403 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:07 smithi077 ceph-mon[32772]: from='osd.0 [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-25T10:25:07.403 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:07 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:25:07.403 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:07 smithi077 ceph-mon[32772]: from='osd.0 [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-25T10:25:07.403 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:07 smithi077 ceph-mon[32772]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T10:25:07.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:07 smithi164 ceph-mon[32713]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:25:07.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:07 smithi164 ceph-mon[32713]: from='osd.0 [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T10:25:07.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:07 smithi164 ceph-mon[32713]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T10:25:07.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:07 smithi164 ceph-mon[32713]: from='osd.0 [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-25T10:25:07.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:07 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:25:07.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:07 smithi164 ceph-mon[32713]: from='osd.0 [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-25T10:25:07.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:07 smithi164 ceph-mon[32713]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T10:25:07.967 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T10:25:07.967 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T10:25:07.967 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-04-25T10:25:07.967 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-04-25T10:25:07.968 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115228 s, 910 MB/s 2024-04-25T10:25:07.968 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-04-25T10:25:08.220 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:08 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:25:08.221 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:08 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:25:08.221 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:08 smithi043 ceph-mon[29478]: from='osd.0 [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946]' entity='osd.0' 2024-04-25T10:25:08.221 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:25:07 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-0[41543]: 2024-04-25T10:25:07.916+0000 7feff4b39700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T10:25:08.221 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:25:07 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-0[41543]: 2024-04-25T10:25:07.919+0000 7fefec0e5700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:25:08.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:08 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:25:08.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:08 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:25:08.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:08 smithi077 ceph-mon[32772]: from='osd.0 [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946]' entity='osd.0' 2024-04-25T10:25:08.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:08 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:25:08.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:08 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:25:08.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:08 smithi164 ceph-mon[32713]: from='osd.0 [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946]' entity='osd.0' 2024-04-25T10:25:08.575 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_4 2024-04-25T10:25:08.817 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T10:25:09.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:09 smithi077 ceph-mon[32772]: purged_snaps scrub starts 2024-04-25T10:25:09.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:09 smithi077 ceph-mon[32772]: purged_snaps scrub ok 2024-04-25T10:25:09.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:09 smithi077 ceph-mon[32772]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:25:09.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:09 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:25:09.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:09 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:09.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:09 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:09.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:09 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:25:09.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:09 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:09.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:09 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:09.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:09 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:25:09.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:09 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:09.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:09 smithi077 ceph-mon[32772]: osd.0 [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946] boot 2024-04-25T10:25:09.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:09 smithi077 ceph-mon[32772]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T10:25:09.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:09 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:25:09.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:09 smithi164 ceph-mon[32713]: purged_snaps scrub starts 2024-04-25T10:25:09.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:09 smithi164 ceph-mon[32713]: purged_snaps scrub ok 2024-04-25T10:25:09.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:09 smithi164 ceph-mon[32713]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:25:09.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:09 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:25:09.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:09 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:09.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:09 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:09.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:09 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:25:09.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:09 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:09.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:09 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:09.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:09 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:25:09.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:09 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:09.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:09 smithi164 ceph-mon[32713]: osd.0 [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946] boot 2024-04-25T10:25:09.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:09 smithi164 ceph-mon[32713]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T10:25:09.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:09 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:25:09.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:09 smithi043 ceph-mon[29478]: purged_snaps scrub starts 2024-04-25T10:25:09.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:09 smithi043 ceph-mon[29478]: purged_snaps scrub ok 2024-04-25T10:25:09.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:09 smithi043 ceph-mon[29478]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T10:25:09.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:25:09.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:09.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:09.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:25:09.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:09.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:09.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:25:09.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:09.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:09 smithi043 ceph-mon[29478]: osd.0 [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946] boot 2024-04-25T10:25:09.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:09 smithi043 ceph-mon[29478]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T10:25:09.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:25:10.192 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:10 smithi077 ceph-mon[32772]: Detected new or changed devices on smithi043 2024-04-25T10:25:10.192 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:10 smithi077 ceph-mon[32772]: Adjusting osd_memory_target on smithi043 to 17093M 2024-04-25T10:25:10.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.428+0000 7fd7fcd38700 1 -- 172.21.15.77:0/2174531707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7f8105cd0 msgr2=0x7fd7f810c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:10.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.428+0000 7fd7fcd38700 1 --2- 172.21.15.77:0/2174531707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7f8105cd0 0x7fd7f810c580 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd7f000b0a0 tx=0x7fd7f000b3b0 comp rx=0 tx=0).stop 2024-04-25T10:25:10.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.428+0000 7fd7fcd38700 1 -- 172.21.15.77:0/2174531707 shutdown_connections 2024-04-25T10:25:10.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.428+0000 7fd7fcd38700 1 --2- 172.21.15.77:0/2174531707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7f8105cd0 0x7fd7f810c580 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:10.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.428+0000 7fd7fcd38700 1 --2- 172.21.15.77:0/2174531707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7f8104200 0x7fd7f8104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:10.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.428+0000 7fd7fcd38700 1 --2- 172.21.15.77:0/2174531707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7f8105330 0x7fd7f8105740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:10.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.428+0000 7fd7fcd38700 1 -- 172.21.15.77:0/2174531707 >> 172.21.15.77:0/2174531707 conn(0x7fd7f80ff840 msgr2=0x7fd7f8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:10.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.428+0000 7fd7fcd38700 1 -- 172.21.15.77:0/2174531707 shutdown_connections 2024-04-25T10:25:10.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.429+0000 7fd7fcd38700 1 -- 172.21.15.77:0/2174531707 wait complete. 2024-04-25T10:25:10.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.429+0000 7fd7fcd38700 1 Processor -- start 2024-04-25T10:25:10.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.429+0000 7fd7fcd38700 1 -- start start 2024-04-25T10:25:10.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7fcd38700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7f8104200 0x7fd7f8077330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:10.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7fcd38700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7f8105330 0x7fd7f80759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:10.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7fcd38700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7f8105cd0 0x7fd7f8075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:10.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7fcd38700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f810fd60 con 0x7fd7f8104200 2024-04-25T10:25:10.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7fcd38700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f810fbe0 con 0x7fd7f8105330 2024-04-25T10:25:10.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7fcd38700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f810fee0 con 0x7fd7f8105cd0 2024-04-25T10:25:10.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7f5d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7f8105330 0x7fd7f80759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:10.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7f5d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7f8105330 0x7fd7f80759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54288/0 (socket says 172.21.15.77:54288) 2024-04-25T10:25:10.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7f5d9b700 1 -- 172.21.15.77:0/3869364683 learned_addr learned my addr 172.21.15.77:0/3869364683 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-25T10:25:10.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7f6d9d700 1 --2- 172.21.15.77:0/3869364683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7f8105cd0 0x7fd7f8075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:10.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7f659c700 1 --2- 172.21.15.77:0/3869364683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7f8104200 0x7fd7f8077330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:10.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7f5d9b700 1 -- 172.21.15.77:0/3869364683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7f8105cd0 msgr2=0x7fd7f8075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:10.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7f5d9b700 1 --2- 172.21.15.77:0/3869364683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7f8105cd0 0x7fd7f8075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:10.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7f5d9b700 1 -- 172.21.15.77:0/3869364683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7f8104200 msgr2=0x7fd7f8077330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:10.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7f5d9b700 1 --2- 172.21.15.77:0/3869364683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7f8104200 0x7fd7f8077330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:10.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7f5d9b700 1 -- 172.21.15.77:0/3869364683 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f8076680 con 0x7fd7f8105330 2024-04-25T10:25:10.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7f659c700 1 --2- 172.21.15.77:0/3869364683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7f8104200 0x7fd7f8077330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:10.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7f6d9d700 1 --2- 172.21.15.77:0/3869364683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7f8105cd0 0x7fd7f8075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:10.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7f5d9b700 1 --2- 172.21.15.77:0/3869364683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7f8105330 0x7fd7f80759e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800b530 tx=0x7fd7e800b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:10.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7e77fe700 1 -- 172.21.15.77:0/3869364683 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8015020 con 0x7fd7f8105330 2024-04-25T10:25:10.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7e77fe700 1 -- 172.21.15.77:0/3869364683 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7e8011ce0 con 0x7fd7f8105330 2024-04-25T10:25:10.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7e77fe700 1 -- 172.21.15.77:0/3869364683 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8010d20 con 0x7fd7f8105330 2024-04-25T10:25:10.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7fcd38700 1 -- 172.21.15.77:0/3869364683 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f81a4910 con 0x7fd7f8105330 2024-04-25T10:25:10.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.430+0000 7fd7fcd38700 1 -- 172.21.15.77:0/3869364683 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f81a4e70 con 0x7fd7f8105330 2024-04-25T10:25:10.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.431+0000 7fd7fcd38700 1 -- 172.21.15.77:0/3869364683 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f8051190 con 0x7fd7f8105330 2024-04-25T10:25:10.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.433+0000 7fd7e77fe700 1 -- 172.21.15.77:0/3869364683 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7fd7e8018020 con 0x7fd7f8105330 2024-04-25T10:25:10.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.433+0000 7fd7e77fe700 1 --2- 172.21.15.77:0/3869364683 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd7e00afaf0 0x7fd7e00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:10.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.434+0000 7fd7e77fe700 1 -- 172.21.15.77:0/3869364683 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fd7e80dcf10 con 0x7fd7f8105330 2024-04-25T10:25:10.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.434+0000 7fd7f659c700 1 --2- 172.21.15.77:0/3869364683 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd7e00afaf0 0x7fd7e00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:10.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.434+0000 7fd7f659c700 1 --2- 172.21.15.77:0/3869364683 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd7e00afaf0 0x7fd7e00b2000 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8105230 tx=0x7fd7ec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:10.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.437+0000 7fd7e77fe700 1 -- 172.21.15.77:0/3869364683 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd7e801f020 con 0x7fd7f8105330 2024-04-25T10:25:10.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:10 smithi164 ceph-mon[32713]: Detected new or changed devices on smithi043 2024-04-25T10:25:10.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:10 smithi164 ceph-mon[32713]: Adjusting osd_memory_target on smithi043 to 17093M 2024-04-25T10:25:10.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:10 smithi043 ceph-mon[29478]: Detected new or changed devices on smithi043 2024-04-25T10:25:10.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:10 smithi043 ceph-mon[29478]: Adjusting osd_memory_target on smithi043 to 17093M 2024-04-25T10:25:10.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:10.642+0000 7fd7fcd38700 1 -- 172.21.15.77:0/3869364683 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd7f8063ad0 con 0x7fd7e00afaf0 2024-04-25T10:25:11.437 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:11 smithi077 ceph-mon[32772]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:11.438 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:11 smithi077 ceph-mon[32772]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T10:25:11.438 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:11 smithi077 ceph-mon[32772]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:25:11.438 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:11 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:25:11.438 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:11 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:25:11.438 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:11 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:11.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:11 smithi164 ceph-mon[32713]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:11.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:11 smithi164 ceph-mon[32713]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T10:25:11.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:11 smithi164 ceph-mon[32713]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:25:11.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:11 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:25:11.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:11 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:25:11.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:11 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:11.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:11 smithi043 ceph-mon[29478]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:11.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:11 smithi043 ceph-mon[29478]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T10:25:11.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:11 smithi043 ceph-mon[29478]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:25:11.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:11 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:25:11.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:11 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:25:11.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:11 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:13.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:13 smithi077 ceph-mon[32772]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:13.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:13 smithi164 ceph-mon[32713]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:13.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:13 smithi043 ceph-mon[29478]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:14.416 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:14 smithi043 ceph-mon[29478]: from='client.? 172.21.15.77:0/428082683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8e23410-bf31-4a5e-a7dc-d99673208270"}]: dispatch 2024-04-25T10:25:14.416 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:14 smithi043 ceph-mon[29478]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8e23410-bf31-4a5e-a7dc-d99673208270"}]: dispatch 2024-04-25T10:25:14.416 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:14 smithi043 ceph-mon[29478]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8e23410-bf31-4a5e-a7dc-d99673208270"}]': finished 2024-04-25T10:25:14.416 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:14 smithi043 ceph-mon[29478]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T10:25:14.416 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:14 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:14.416 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:14 smithi043 ceph-mon[29478]: from='client.? 172.21.15.77:0/1992872663' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:25:14.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:14 smithi077 ceph-mon[32772]: from='client.? 172.21.15.77:0/428082683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8e23410-bf31-4a5e-a7dc-d99673208270"}]: dispatch 2024-04-25T10:25:14.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:14 smithi077 ceph-mon[32772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8e23410-bf31-4a5e-a7dc-d99673208270"}]: dispatch 2024-04-25T10:25:14.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:14 smithi077 ceph-mon[32772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8e23410-bf31-4a5e-a7dc-d99673208270"}]': finished 2024-04-25T10:25:14.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:14 smithi077 ceph-mon[32772]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T10:25:14.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:14 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:14.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:14 smithi077 ceph-mon[32772]: from='client.? 172.21.15.77:0/1992872663' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:25:14.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:14 smithi164 ceph-mon[32713]: from='client.? 172.21.15.77:0/428082683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8e23410-bf31-4a5e-a7dc-d99673208270"}]: dispatch 2024-04-25T10:25:14.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:14 smithi164 ceph-mon[32713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8e23410-bf31-4a5e-a7dc-d99673208270"}]: dispatch 2024-04-25T10:25:14.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:14 smithi164 ceph-mon[32713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8e23410-bf31-4a5e-a7dc-d99673208270"}]': finished 2024-04-25T10:25:14.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:14 smithi164 ceph-mon[32713]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T10:25:14.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:14 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:14.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:14 smithi164 ceph-mon[32713]: from='client.? 172.21.15.77:0/1992872663' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:25:15.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:15 smithi077 ceph-mon[32772]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:15.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:15 smithi164 ceph-mon[32713]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:15.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:15 smithi043 ceph-mon[29478]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:17.523 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:17 smithi077 ceph-mon[32772]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:17.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:17 smithi164 ceph-mon[32713]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:17.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:17 smithi043 ceph-mon[29478]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:19.606 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:19 smithi077 ceph-mon[32772]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:19.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:19 smithi043 ceph-mon[29478]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:19.794 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:19 smithi164 ceph-mon[32713]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:21.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:21 smithi043 ceph-mon[29478]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:21.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:21 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T10:25:21.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:21 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:21.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:21 smithi043 ceph-mon[29478]: Deploying daemon osd.1 on smithi077 2024-04-25T10:25:21.621 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:21 smithi077 ceph-mon[32772]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:21.621 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:21 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T10:25:21.622 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:21 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:21.622 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:21 smithi077 ceph-mon[32772]: Deploying daemon osd.1 on smithi077 2024-04-25T10:25:21.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:21 smithi164 ceph-mon[32713]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:21.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:21 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T10:25:21.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:21 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:21.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:21 smithi164 ceph-mon[32713]: Deploying daemon osd.1 on smithi077 2024-04-25T10:25:23.414 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:23 smithi077 ceph-mon[32772]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:23.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:23 smithi043 ceph-mon[29478]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:23.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:23 smithi164 ceph-mon[32713]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:25.489 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:25 smithi077 ceph-mon[32772]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:25.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:25 smithi043 ceph-mon[29478]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:25.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:25 smithi164 ceph-mon[32713]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:27.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:27 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:27.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:27 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:27.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:27 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:25:27.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:27 smithi077 ceph-mon[32772]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:27.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:27 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:27.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:27 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:27.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:27 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:25:27.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:27 smithi164 ceph-mon[32713]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:27.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:27 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:27.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:27 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:27.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:27 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:25:27.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:27 smithi043 ceph-mon[29478]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:29.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:29 smithi077 ceph-mon[32772]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:29.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:29 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:29.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:29 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:29.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:29 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:29.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:29 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:25:29.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:29 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:29.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:29 smithi043 ceph-mon[29478]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:29.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:29 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:29.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:29 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:29.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:29 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:29.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:29 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:25:29.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:29 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:29.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:29 smithi164 ceph-mon[32713]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:29.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:29 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:29.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:29 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:29.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:29 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:29.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:29 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:25:29.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:29 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:29.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:29.909+0000 7fd7e77fe700 1 -- 172.21.15.77:0/3869364683 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd7f8063ad0 con 0x7fd7e00afaf0 2024-04-25T10:25:29.911 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 1 on host 'smithi077' 2024-04-25T10:25:29.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:29.912+0000 7fd7fcd38700 1 -- 172.21.15.77:0/3869364683 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd7e00afaf0 msgr2=0x7fd7e00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:29.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:29.912+0000 7fd7fcd38700 1 --2- 172.21.15.77:0/3869364683 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd7e00afaf0 0x7fd7e00b2000 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8105230 tx=0x7fd7ec008040 comp rx=0 tx=0).stop 2024-04-25T10:25:29.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:29.912+0000 7fd7fcd38700 1 -- 172.21.15.77:0/3869364683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7f8105330 msgr2=0x7fd7f80759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:29.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:29.912+0000 7fd7fcd38700 1 --2- 172.21.15.77:0/3869364683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7f8105330 0x7fd7f80759e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800b530 tx=0x7fd7e800b8f0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:29.912+0000 7fd7fcd38700 1 -- 172.21.15.77:0/3869364683 shutdown_connections 2024-04-25T10:25:29.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:29.912+0000 7fd7fcd38700 1 --2- 172.21.15.77:0/3869364683 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd7e00afaf0 0x7fd7e00b2000 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:29.912+0000 7fd7fcd38700 1 --2- 172.21.15.77:0/3869364683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7f8105cd0 0x7fd7f8075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:29.912+0000 7fd7fcd38700 1 --2- 172.21.15.77:0/3869364683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7f8104200 0x7fd7f8077330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:29.912+0000 7fd7fcd38700 1 --2- 172.21.15.77:0/3869364683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7f8105330 0x7fd7f80759e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:29.912+0000 7fd7fcd38700 1 -- 172.21.15.77:0/3869364683 >> 172.21.15.77:0/3869364683 conn(0x7fd7f80ff840 msgr2=0x7fd7f8108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:29.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:29.912+0000 7fd7fcd38700 1 -- 172.21.15.77:0/3869364683 shutdown_connections 2024-04-25T10:25:29.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:25:29.912+0000 7fd7fcd38700 1 -- 172.21.15.77:0/3869364683 wait complete. 2024-04-25T10:25:30.536 DEBUG:teuthology.orchestra.run.smithi077:osd.1> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T10:25:30.538 INFO:tasks.cephadm:Deploying osd.2 on smithi164 with /dev/vg_nvme/lv_4... 2024-04-25T10:25:30.538 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T10:25:30.783 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:25:31.185 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:30 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:31.185 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:30 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:31.185 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:30 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:25:31.185 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:30 smithi164 ceph-mon[32713]: from='osd.1 [v2:172.21.15.77:6800/3362129259,v1:172.21.15.77:6801/3362129259]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T10:25:31.186 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:30 smithi164 ceph-mon[32713]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T10:25:31.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:30 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:31.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:30 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:31.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:30 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:25:31.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:30 smithi077 ceph-mon[32772]: from='osd.1 [v2:172.21.15.77:6800/3362129259,v1:172.21.15.77:6801/3362129259]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T10:25:31.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:30 smithi077 ceph-mon[32772]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T10:25:31.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:30 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:31.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:30 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:31.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:30 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:25:31.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:30 smithi043 ceph-mon[29478]: from='osd.1 [v2:172.21.15.77:6800/3362129259,v1:172.21.15.77:6801/3362129259]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T10:25:31.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:30 smithi043 ceph-mon[29478]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T10:25:31.925 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:31 smithi077 ceph-mon[32772]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:31.925 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:31 smithi077 ceph-mon[32772]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T10:25:31.925 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:31 smithi077 ceph-mon[32772]: from='osd.1 [v2:172.21.15.77:6800/3362129259,v1:172.21.15.77:6801/3362129259]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:25:31.925 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:31 smithi077 ceph-mon[32772]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T10:25:31.926 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:31 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:31.926 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:31 smithi077 ceph-mon[32772]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:25:32.038 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:31 smithi164 ceph-mon[32713]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:32.038 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:31 smithi164 ceph-mon[32713]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T10:25:32.038 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:31 smithi164 ceph-mon[32713]: from='osd.1 [v2:172.21.15.77:6800/3362129259,v1:172.21.15.77:6801/3362129259]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:25:32.038 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:31 smithi164 ceph-mon[32713]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T10:25:32.039 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:31 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:32.039 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:31 smithi164 ceph-mon[32713]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:25:32.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:31 smithi043 ceph-mon[29478]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:32.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:31 smithi043 ceph-mon[29478]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T10:25:32.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:31 smithi043 ceph-mon[29478]: from='osd.1 [v2:172.21.15.77:6800/3362129259,v1:172.21.15.77:6801/3362129259]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:25:32.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:31 smithi043 ceph-mon[29478]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T10:25:32.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:31 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:32.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:31 smithi043 ceph-mon[29478]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:25:32.581 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T10:25:32.581 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T10:25:32.581 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-04-25T10:25:32.581 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-04-25T10:25:32.581 INFO:teuthology.orchestra.run.smithi164.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113599 s, 923 MB/s 2024-04-25T10:25:32.581 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-04-25T10:25:33.169 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:32 smithi164 ceph-mon[32713]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-25T10:25:33.169 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:32 smithi164 ceph-mon[32713]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T10:25:33.169 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:33.169 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:33.169 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:32 smithi164 ceph-mon[32713]: from='osd.1 ' entity='osd.1' 2024-04-25T10:25:33.195 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_4 2024-04-25T10:25:33.212 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:32 smithi077 ceph-mon[32772]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-25T10:25:33.212 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:32 smithi077 ceph-mon[32772]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T10:25:33.212 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:33.212 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:33.212 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:32 smithi077 ceph-mon[32772]: from='osd.1 ' entity='osd.1' 2024-04-25T10:25:33.213 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:25:32 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-1[37392]: 2024-04-25T10:25:32.792+0000 7f77e3501700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T10:25:33.213 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:25:32 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-1[37392]: 2024-04-25T10:25:32.953+0000 7f77db2ae700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:25:33.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:32 smithi043 ceph-mon[29478]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-25T10:25:33.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:32 smithi043 ceph-mon[29478]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T10:25:33.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:33.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:33.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:32 smithi043 ceph-mon[29478]: from='osd.1 ' entity='osd.1' 2024-04-25T10:25:33.451 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:25:34.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:33 smithi164 ceph-mon[32713]: purged_snaps scrub starts 2024-04-25T10:25:34.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:33 smithi164 ceph-mon[32713]: purged_snaps scrub ok 2024-04-25T10:25:34.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:33 smithi164 ceph-mon[32713]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:34.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:34.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:34.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:34.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:25:34.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:34.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:34.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:25:34.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:34.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:34.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:33 smithi077 ceph-mon[32772]: purged_snaps scrub starts 2024-04-25T10:25:34.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:33 smithi077 ceph-mon[32772]: purged_snaps scrub ok 2024-04-25T10:25:34.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:33 smithi077 ceph-mon[32772]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:34.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:34.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:34.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:34.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:25:34.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:34.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:34.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:25:34.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:34.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:34.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:33 smithi043 ceph-mon[29478]: purged_snaps scrub starts 2024-04-25T10:25:34.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:33 smithi043 ceph-mon[29478]: purged_snaps scrub ok 2024-04-25T10:25:34.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:33 smithi043 ceph-mon[29478]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:34.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:34.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:34.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:34.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:25:34.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:34.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:34.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:25:34.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:34.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:35.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.009+0000 7f33797c3700 1 -- 172.21.15.164:0/1751449094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3374103120 msgr2=0x7f3374103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.009+0000 7f33797c3700 1 --2- 172.21.15.164:0/1751449094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3374103120 0x7f3374103530 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3364009990 tx=0x7f3364009ca0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.009+0000 7f33797c3700 1 -- 172.21.15.164:0/1751449094 shutdown_connections 2024-04-25T10:25:35.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.009+0000 7f33797c3700 1 --2- 172.21.15.164:0/1751449094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3374104bf0 0x7f337410b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.009+0000 7f33797c3700 1 --2- 172.21.15.164:0/1751449094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3374103120 0x7f3374103530 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.009+0000 7f33797c3700 1 --2- 172.21.15.164:0/1751449094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3374104250 0x7f3374104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.009+0000 7f33797c3700 1 -- 172.21.15.164:0/1751449094 >> 172.21.15.164:0/1751449094 conn(0x7f33740fe760 msgr2=0x7f3374100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:35.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.009+0000 7f33797c3700 1 -- 172.21.15.164:0/1751449094 shutdown_connections 2024-04-25T10:25:35.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.009+0000 7f33797c3700 1 -- 172.21.15.164:0/1751449094 wait complete. 2024-04-25T10:25:35.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.010+0000 7f33797c3700 1 Processor -- start 2024-04-25T10:25:35.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.010+0000 7f33797c3700 1 -- start start 2024-04-25T10:25:35.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.010+0000 7f33797c3700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3374103120 0x7f3374076340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.010+0000 7f33797c3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3374104250 0x7f337419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.010+0000 7f33797c3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3374104bf0 0x7f33741a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.010+0000 7f33797c3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f337410ebe0 con 0x7f3374104250 2024-04-25T10:25:35.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.010+0000 7f33797c3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f337410ea60 con 0x7f3374104bf0 2024-04-25T10:25:35.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.010+0000 7f33797c3700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f337410ed60 con 0x7f3374103120 2024-04-25T10:25:35.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.010+0000 7f3372ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3374103120 0x7f3374076340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.010+0000 7f3372ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3374103120 0x7f3374076340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46812/0 (socket says 172.21.15.164:46812) 2024-04-25T10:25:35.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.010+0000 7f3372ffd700 1 -- 172.21.15.164:0/1531699310 learned_addr learned my addr 172.21.15.164:0/1531699310 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-25T10:25:35.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.010+0000 7f336bfff700 1 --2- 172.21.15.164:0/1531699310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3374104250 0x7f337419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.010+0000 7f33737fe700 1 --2- 172.21.15.164:0/1531699310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3374104bf0 0x7f33741a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.010+0000 7f3372ffd700 1 -- 172.21.15.164:0/1531699310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3374104bf0 msgr2=0x7f33741a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.010+0000 7f3372ffd700 1 --2- 172.21.15.164:0/1531699310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3374104bf0 0x7f33741a1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.010+0000 7f3372ffd700 1 -- 172.21.15.164:0/1531699310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3374104250 msgr2=0x7f337419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.010+0000 7f3372ffd700 1 --2- 172.21.15.164:0/1531699310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3374104250 0x7f337419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.011+0000 7f3372ffd700 1 -- 172.21.15.164:0/1531699310 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33741a2480 con 0x7f3374103120 2024-04-25T10:25:35.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.011+0000 7f3372ffd700 1 --2- 172.21.15.164:0/1531699310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3374103120 0x7f3374076340 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3364000c00 tx=0x7f33640047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:35.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.011+0000 7f3370ff9700 1 -- 172.21.15.164:0/1531699310 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364027070 con 0x7f3374103120 2024-04-25T10:25:35.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.011+0000 7f3370ff9700 1 -- 172.21.15.164:0/1531699310 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33640040c0 con 0x7f3374103120 2024-04-25T10:25:35.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.011+0000 7f33797c3700 1 -- 172.21.15.164:0/1531699310 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33741a2710 con 0x7f3374103120 2024-04-25T10:25:35.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.011+0000 7f3370ff9700 1 -- 172.21.15.164:0/1531699310 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364022de0 con 0x7f3374103120 2024-04-25T10:25:35.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.011+0000 7f33797c3700 1 -- 172.21.15.164:0/1531699310 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33741a2bc0 con 0x7f3374103120 2024-04-25T10:25:35.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.012+0000 7f33797c3700 1 -- 172.21.15.164:0/1531699310 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3374051190 con 0x7f3374103120 2024-04-25T10:25:35.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.013+0000 7f3370ff9700 1 -- 172.21.15.164:0/1531699310 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7f336402a070 con 0x7f3374103120 2024-04-25T10:25:35.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.014+0000 7f3370ff9700 1 --2- 172.21.15.164:0/1531699310 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f33600afb70 0x7f33600b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.014+0000 7f3370ff9700 1 -- 172.21.15.164:0/1531699310 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f33640de570 con 0x7f3374103120 2024-04-25T10:25:35.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.014+0000 7f336bfff700 1 --2- 172.21.15.164:0/1531699310 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f33600afb70 0x7f33600b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.015+0000 7f336bfff700 1 --2- 172.21.15.164:0/1531699310 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f33600afb70 0x7f33600b2080 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f335c006fd0 tx=0x7f335c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:35.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.018+0000 7f3370ff9700 1 -- 172.21.15.164:0/1531699310 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3364031080 con 0x7f3374103120 2024-04-25T10:25:35.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:35.225+0000 7f33797c3700 1 -- 172.21.15.164:0/1531699310 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3374063ad0 con 0x7f33600afb70 2024-04-25T10:25:35.227 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:34 smithi164 ceph-mon[32713]: Detected new or changed devices on smithi077 2024-04-25T10:25:35.227 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:34 smithi164 ceph-mon[32713]: Adjusting osd_memory_target on smithi077 to 17088M 2024-04-25T10:25:35.227 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:34 smithi164 ceph-mon[32713]: osd.1 [v2:172.21.15.77:6800/3362129259,v1:172.21.15.77:6801/3362129259] boot 2024-04-25T10:25:35.228 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:34 smithi164 ceph-mon[32713]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T10:25:35.228 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:35.228 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:34 smithi164 ceph-mon[32713]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:35.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:34 smithi077 ceph-mon[32772]: Detected new or changed devices on smithi077 2024-04-25T10:25:35.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:34 smithi077 ceph-mon[32772]: Adjusting osd_memory_target on smithi077 to 17088M 2024-04-25T10:25:35.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:34 smithi077 ceph-mon[32772]: osd.1 [v2:172.21.15.77:6800/3362129259,v1:172.21.15.77:6801/3362129259] boot 2024-04-25T10:25:35.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:34 smithi077 ceph-mon[32772]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T10:25:35.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:35.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:34 smithi077 ceph-mon[32772]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:35.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:34 smithi043 ceph-mon[29478]: Detected new or changed devices on smithi077 2024-04-25T10:25:35.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:34 smithi043 ceph-mon[29478]: Adjusting osd_memory_target on smithi077 to 17088M 2024-04-25T10:25:35.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:34 smithi043 ceph-mon[29478]: osd.1 [v2:172.21.15.77:6800/3362129259,v1:172.21.15.77:6801/3362129259] boot 2024-04-25T10:25:35.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:34 smithi043 ceph-mon[29478]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T10:25:35.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:25:35.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:34 smithi043 ceph-mon[29478]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T10:25:36.115 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:35 smithi164 ceph-mon[32713]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T10:25:36.115 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:35 smithi164 ceph-mon[32713]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:25:36.115 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:35 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:25:36.115 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:35 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:25:36.115 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:35 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:36.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:35 smithi077 ceph-mon[32772]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T10:25:36.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:35 smithi077 ceph-mon[32772]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:25:36.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:35 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:25:36.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:35 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:25:36.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:35 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:36.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:35 smithi043 ceph-mon[29478]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T10:25:36.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:35 smithi043 ceph-mon[29478]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:25:36.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:35 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:25:36.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:35 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:25:36.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:35 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:36.962 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:36 smithi164 ceph-mon[32713]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:37.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:36 smithi043 ceph-mon[29478]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:37.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:36 smithi077 ceph-mon[32772]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:38.176 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:37 smithi164 ceph-mon[32713]: from='client.? 172.21.15.164:0/217711294' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e85413f8-524b-41ed-8ab6-d811d2ed40ed"}]: dispatch 2024-04-25T10:25:38.176 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:37 smithi164 ceph-mon[32713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e85413f8-524b-41ed-8ab6-d811d2ed40ed"}]: dispatch 2024-04-25T10:25:38.176 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:37 smithi164 ceph-mon[32713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e85413f8-524b-41ed-8ab6-d811d2ed40ed"}]': finished 2024-04-25T10:25:38.176 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:37 smithi164 ceph-mon[32713]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T10:25:38.176 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:37 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:38.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:37 smithi077 ceph-mon[32772]: from='client.? 172.21.15.164:0/217711294' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e85413f8-524b-41ed-8ab6-d811d2ed40ed"}]: dispatch 2024-04-25T10:25:38.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:37 smithi077 ceph-mon[32772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e85413f8-524b-41ed-8ab6-d811d2ed40ed"}]: dispatch 2024-04-25T10:25:38.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:37 smithi077 ceph-mon[32772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e85413f8-524b-41ed-8ab6-d811d2ed40ed"}]': finished 2024-04-25T10:25:38.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:37 smithi077 ceph-mon[32772]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T10:25:38.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:37 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:38.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:37 smithi043 ceph-mon[29478]: from='client.? 172.21.15.164:0/217711294' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e85413f8-524b-41ed-8ab6-d811d2ed40ed"}]: dispatch 2024-04-25T10:25:38.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:37 smithi043 ceph-mon[29478]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e85413f8-524b-41ed-8ab6-d811d2ed40ed"}]: dispatch 2024-04-25T10:25:38.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:37 smithi043 ceph-mon[29478]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e85413f8-524b-41ed-8ab6-d811d2ed40ed"}]': finished 2024-04-25T10:25:38.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:37 smithi043 ceph-mon[29478]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T10:25:38.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:37 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:39.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:38 smithi077 ceph-mon[32772]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:39.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:38 smithi077 ceph-mon[32772]: from='client.? 172.21.15.164:0/3251349450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:25:39.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:38 smithi164 ceph-mon[32713]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:39.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:38 smithi164 ceph-mon[32713]: from='client.? 172.21.15.164:0/3251349450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:25:39.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:38 smithi043 ceph-mon[29478]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:39.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:38 smithi043 ceph-mon[29478]: from='client.? 172.21.15.164:0/3251349450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:25:41.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:41 smithi077 ceph-mon[32772]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:41.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:41 smithi164 ceph-mon[32713]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:41.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:41 smithi043 ceph-mon[29478]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:43.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:43 smithi077 ceph-mon[32772]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:43.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:43 smithi164 ceph-mon[32713]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:43.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:43 smithi043 ceph-mon[29478]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:45.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:45 smithi164 ceph-mon[32713]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:45.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:45 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T10:25:45.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:45 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:45.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:45 smithi077 ceph-mon[32772]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:45.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:45 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T10:25:45.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:45 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:45.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:45 smithi043 ceph-mon[29478]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:45.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:45 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T10:25:45.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:45 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:46.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:46 smithi164 ceph-mon[32713]: Deploying daemon osd.2 on smithi164 2024-04-25T10:25:46.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:46 smithi077 ceph-mon[32772]: Deploying daemon osd.2 on smithi164 2024-04-25T10:25:46.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:46 smithi043 ceph-mon[29478]: Deploying daemon osd.2 on smithi164 2024-04-25T10:25:47.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:47 smithi164 ceph-mon[32713]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:47.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:47 smithi077 ceph-mon[32772]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:47.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:47 smithi043 ceph-mon[29478]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:49.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:49 smithi077 ceph-mon[32772]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:49.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:49 smithi164 ceph-mon[32713]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:49.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:49 smithi043 ceph-mon[29478]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:51.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:51 smithi164 ceph-mon[32713]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:51.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:51 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:51.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:51 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:25:51.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:51 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:51.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:51 smithi077 ceph-mon[32772]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:51.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:51 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:51.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:51 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:25:51.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:51 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:51.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:51 smithi043 ceph-mon[29478]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:51.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:51 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:51.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:51 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:25:51.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:51 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:53.680 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:53 smithi164 ceph-mon[32713]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:53.680 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:53 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:53.681 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:53 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:53.681 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:53 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:53.681 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:53 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:25:53.681 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:53 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:53.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:53 smithi077 ceph-mon[32772]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:53.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:53 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:53.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:53 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:53.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:53 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:53.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:53 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:25:53.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:53 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:53 smithi043 ceph-mon[29478]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:53.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:53 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:53.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:53 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:53.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:53 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:53.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:53 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:25:53.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:53 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:54.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:53.997+0000 7f3370ff9700 1 -- 172.21.15.164:0/1531699310 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3374063ad0 con 0x7f33600afb70 2024-04-25T10:25:54.000 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 2 on host 'smithi164' 2024-04-25T10:25:54.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:54.000+0000 7f33797c3700 1 -- 172.21.15.164:0/1531699310 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f33600afb70 msgr2=0x7f33600b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:54.000+0000 7f33797c3700 1 --2- 172.21.15.164:0/1531699310 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f33600afb70 0x7f33600b2080 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f335c006fd0 tx=0x7f335c008040 comp rx=0 tx=0).stop 2024-04-25T10:25:54.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:54.000+0000 7f33797c3700 1 -- 172.21.15.164:0/1531699310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3374103120 msgr2=0x7f3374076340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:54.000+0000 7f33797c3700 1 --2- 172.21.15.164:0/1531699310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3374103120 0x7f3374076340 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3364000c00 tx=0x7f33640047b0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:54.000+0000 7f33797c3700 1 -- 172.21.15.164:0/1531699310 shutdown_connections 2024-04-25T10:25:54.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:54.000+0000 7f33797c3700 1 --2- 172.21.15.164:0/1531699310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3374104bf0 0x7f33741a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:54.000+0000 7f33797c3700 1 --2- 172.21.15.164:0/1531699310 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f33600afb70 0x7f33600b2080 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:54.000+0000 7f33797c3700 1 --2- 172.21.15.164:0/1531699310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3374103120 0x7f3374076340 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:54.000+0000 7f33797c3700 1 --2- 172.21.15.164:0/1531699310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3374104250 0x7f337419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:54.000+0000 7f33797c3700 1 -- 172.21.15.164:0/1531699310 >> 172.21.15.164:0/1531699310 conn(0x7f33740fe760 msgr2=0x7f3374107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:54.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:54.000+0000 7f33797c3700 1 -- 172.21.15.164:0/1531699310 shutdown_connections 2024-04-25T10:25:54.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:25:54.000+0000 7f33797c3700 1 -- 172.21.15.164:0/1531699310 wait complete. 2024-04-25T10:25:54.684 DEBUG:teuthology.orchestra.run.smithi164:osd.2> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T10:25:54.686 INFO:tasks.cephadm:Deploying osd.3 on smithi043 with /dev/vg_nvme/lv_3... 2024-04-25T10:25:54.687 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T10:25:54.943 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:25:55.285 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:54 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:55.285 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:54 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:55.286 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:54 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:25:55.286 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:54 smithi043 ceph-mon[29478]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:55.286 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:54 smithi043 ceph-mon[29478]: from='osd.2 [v2:172.21.15.164:6800/3408162352,v1:172.21.15.164:6801/3408162352]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T10:25:55.286 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:54 smithi043 ceph-mon[29478]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T10:25:55.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:54 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:55.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:54 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:55.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:54 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:25:55.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:54 smithi077 ceph-mon[32772]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:55.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:54 smithi077 ceph-mon[32772]: from='osd.2 [v2:172.21.15.164:6800/3408162352,v1:172.21.15.164:6801/3408162352]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T10:25:55.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:54 smithi077 ceph-mon[32772]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T10:25:55.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:54 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:55.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:54 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:55.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:54 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:25:55.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:54 smithi164 ceph-mon[32713]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:55.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:54 smithi164 ceph-mon[32713]: from='osd.2 [v2:172.21.15.164:6800/3408162352,v1:172.21.15.164:6801/3408162352]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T10:25:55.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:54 smithi164 ceph-mon[32713]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T10:25:56.090 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:55 smithi164 ceph-mon[32713]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T10:25:56.090 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:55 smithi164 ceph-mon[32713]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T10:25:56.090 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:55 smithi164 ceph-mon[32713]: from='osd.2 [v2:172.21.15.164:6800/3408162352,v1:172.21.15.164:6801/3408162352]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:25:56.090 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:55 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:56.090 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:55 smithi164 ceph-mon[32713]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:25:56.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:55 smithi077 ceph-mon[32772]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T10:25:56.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:55 smithi077 ceph-mon[32772]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T10:25:56.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:55 smithi077 ceph-mon[32772]: from='osd.2 [v2:172.21.15.164:6800/3408162352,v1:172.21.15.164:6801/3408162352]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:25:56.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:55 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:56.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:55 smithi077 ceph-mon[32772]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:25:56.323 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:55 smithi043 ceph-mon[29478]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T10:25:56.323 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:55 smithi043 ceph-mon[29478]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T10:25:56.323 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:55 smithi043 ceph-mon[29478]: from='osd.2 [v2:172.21.15.164:6800/3408162352,v1:172.21.15.164:6801/3408162352]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:25:56.323 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:55 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:56.323 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:55 smithi043 ceph-mon[29478]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:25:57.027 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T10:25:57.027 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T10:25:57.027 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-04-25T10:25:57.027 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-04-25T10:25:57.028 INFO:teuthology.orchestra.run.smithi043.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00851124 s, 1.2 GB/s 2024-04-25T10:25:57.028 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-04-25T10:25:57.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:57 smithi077 ceph-mon[32772]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T10:25:57.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:57 smithi077 ceph-mon[32772]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T10:25:57.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:57 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:57.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:57 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:57.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:57 smithi077 ceph-mon[32772]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:57.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:57 smithi077 ceph-mon[32772]: from='osd.2 ' entity='osd.2' 2024-04-25T10:25:57.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:57 smithi164 ceph-mon[32713]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T10:25:57.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:57 smithi164 ceph-mon[32713]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T10:25:57.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:57 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:57.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:57 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:57.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:57 smithi164 ceph-mon[32713]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:57.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:57 smithi164 ceph-mon[32713]: from='osd.2 ' entity='osd.2' 2024-04-25T10:25:57.292 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:25:56 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-2[37873]: 2024-04-25T10:25:56.952+0000 7f3f41e75700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T10:25:57.292 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:25:57 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-2[37873]: 2024-04-25T10:25:57.119+0000 7f3f39c22700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:25:57.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:57 smithi043 ceph-mon[29478]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T10:25:57.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:57 smithi043 ceph-mon[29478]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T10:25:57.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:57 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:57.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:57 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:57.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:57 smithi043 ceph-mon[29478]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:57.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:57 smithi043 ceph-mon[29478]: from='osd.2 ' entity='osd.2' 2024-04-25T10:25:57.634 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_3 2024-04-25T10:25:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:25:58.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:58 smithi077 ceph-mon[32772]: purged_snaps scrub starts 2024-04-25T10:25:58.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:58 smithi077 ceph-mon[32772]: purged_snaps scrub ok 2024-04-25T10:25:58.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:58 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:58.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:58 smithi077 ceph-mon[32772]: Detected new or changed devices on smithi164 2024-04-25T10:25:58.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:58 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:58.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:58 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:58.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:58 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:25:58.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:58 smithi077 ceph-mon[32772]: Adjusting osd_memory_target on smithi164 to 17088M 2024-04-25T10:25:58.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:58 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:58.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:58 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:58.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:58 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:25:58.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:58 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:58.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:58 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:58.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:58 smithi164 ceph-mon[32713]: purged_snaps scrub starts 2024-04-25T10:25:58.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:58 smithi164 ceph-mon[32713]: purged_snaps scrub ok 2024-04-25T10:25:58.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:58 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:58.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:58 smithi164 ceph-mon[32713]: Detected new or changed devices on smithi164 2024-04-25T10:25:58.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:58 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:58.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:58 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:58.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:58 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:25:58.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:58 smithi164 ceph-mon[32713]: Adjusting osd_memory_target on smithi164 to 17088M 2024-04-25T10:25:58.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:58 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:58.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:58 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:58.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:58 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:25:58.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:58 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:58.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:58 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:58.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:58 smithi043 ceph-mon[29478]: purged_snaps scrub starts 2024-04-25T10:25:58.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:58 smithi043 ceph-mon[29478]: purged_snaps scrub ok 2024-04-25T10:25:58.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:58 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:58.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:58 smithi043 ceph-mon[29478]: Detected new or changed devices on smithi164 2024-04-25T10:25:58.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:58 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:58.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:58 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:58.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:58 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:25:58.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:58 smithi043 ceph-mon[29478]: Adjusting osd_memory_target on smithi164 to 17088M 2024-04-25T10:25:58.342 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:58 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:58.342 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:58 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:25:58.342 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:58 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:25:58.342 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:58 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:25:58.342 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:58 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:59.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:59 smithi043 ceph-mon[29478]: osd.2 [v2:172.21.15.164:6800/3408162352,v1:172.21.15.164:6801/3408162352] boot 2024-04-25T10:25:59.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:59 smithi043 ceph-mon[29478]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T10:25:59.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:59 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:59.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:59 smithi043 ceph-mon[29478]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:59.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:25:59 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T10:25:59.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:59 smithi077 ceph-mon[32772]: osd.2 [v2:172.21.15.164:6800/3408162352,v1:172.21.15.164:6801/3408162352] boot 2024-04-25T10:25:59.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:59 smithi077 ceph-mon[32772]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T10:25:59.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:59 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:59.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:59 smithi077 ceph-mon[32772]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:59.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:25:59 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T10:25:59.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:59 smithi164 ceph-mon[32713]: osd.2 [v2:172.21.15.164:6800/3408162352,v1:172.21.15.164:6801/3408162352] boot 2024-04-25T10:25:59.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:59 smithi164 ceph-mon[32713]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T10:25:59.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:59 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:25:59.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:59 smithi164 ceph-mon[32713]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T10:25:59.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:25:59 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T10:25:59.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.479+0000 7faa53bbf700 1 -- 172.21.15.43:0/3668197425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa4c104bb0 msgr2=0x7faa4c10b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.479+0000 7faa53bbf700 1 --2- 172.21.15.43:0/3668197425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa4c104bb0 0x7faa4c10b460 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faa4400b0a0 tx=0x7faa4400b3b0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.479+0000 7faa53bbf700 1 -- 172.21.15.43:0/3668197425 shutdown_connections 2024-04-25T10:25:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.479+0000 7faa53bbf700 1 --2- 172.21.15.43:0/3668197425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa4c104bb0 0x7faa4c10b460 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.479+0000 7faa53bbf700 1 --2- 172.21.15.43:0/3668197425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa4c1030e0 0x7faa4c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.479+0000 7faa53bbf700 1 --2- 172.21.15.43:0/3668197425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa4c104210 0x7faa4c104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.479+0000 7faa53bbf700 1 -- 172.21.15.43:0/3668197425 >> 172.21.15.43:0/3668197425 conn(0x7faa4c0fe740 msgr2=0x7faa4c100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.479+0000 7faa53bbf700 1 -- 172.21.15.43:0/3668197425 shutdown_connections 2024-04-25T10:25:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.479+0000 7faa53bbf700 1 -- 172.21.15.43:0/3668197425 wait complete. 2024-04-25T10:25:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.480+0000 7faa53bbf700 1 Processor -- start 2024-04-25T10:25:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.480+0000 7faa53bbf700 1 -- start start 2024-04-25T10:25:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.480+0000 7faa53bbf700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa4c1030e0 0x7faa4c199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.480+0000 7faa53bbf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa4c104210 0x7faa4c199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.480+0000 7faa53bbf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa4c104bb0 0x7faa4c1a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.480+0000 7faa53bbf700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c10eb60 con 0x7faa4c104bb0 2024-04-25T10:25:59.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.480+0000 7faa53bbf700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c10e9e0 con 0x7faa4c104210 2024-04-25T10:25:59.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.480+0000 7faa53bbf700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c10ece0 con 0x7faa4c1030e0 2024-04-25T10:25:59.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.480+0000 7faa5215c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa4c104bb0 0x7faa4c1a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.480+0000 7faa5215c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa4c104bb0 0x7faa4c1a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42882/0 (socket says 172.21.15.43:42882) 2024-04-25T10:25:59.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.480+0000 7faa5215c700 1 -- 172.21.15.43:0/1317785572 learned_addr learned my addr 172.21.15.43:0/1317785572 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:25:59.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.480+0000 7faa5115a700 1 --2- 172.21.15.43:0/1317785572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa4c104210 0x7faa4c199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.480+0000 7faa5195b700 1 --2- 172.21.15.43:0/1317785572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa4c1030e0 0x7faa4c199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.480+0000 7faa5215c700 1 -- 172.21.15.43:0/1317785572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa4c1030e0 msgr2=0x7faa4c199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.481+0000 7faa5215c700 1 --2- 172.21.15.43:0/1317785572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa4c1030e0 0x7faa4c199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.481+0000 7faa5215c700 1 -- 172.21.15.43:0/1317785572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa4c104210 msgr2=0x7faa4c199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.481+0000 7faa5215c700 1 --2- 172.21.15.43:0/1317785572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa4c104210 0x7faa4c199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.481+0000 7faa5215c700 1 -- 172.21.15.43:0/1317785572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4c1a11a0 con 0x7faa4c104bb0 2024-04-25T10:25:59.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.481+0000 7faa5215c700 1 --2- 172.21.15.43:0/1317785572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa4c104bb0 0x7faa4c1a0b20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa4400b070 tx=0x7faa44004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:59.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.481+0000 7faa3effd700 1 -- 172.21.15.43:0/1317785572 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa44021070 con 0x7faa4c104bb0 2024-04-25T10:25:59.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.481+0000 7faa3effd700 1 -- 172.21.15.43:0/1317785572 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa44007b80 con 0x7faa4c104bb0 2024-04-25T10:25:59.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.481+0000 7faa53bbf700 1 -- 172.21.15.43:0/1317785572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa4c1a1430 con 0x7faa4c104bb0 2024-04-25T10:25:59.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.481+0000 7faa3effd700 1 -- 172.21.15.43:0/1317785572 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa440052a0 con 0x7faa4c104bb0 2024-04-25T10:25:59.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.481+0000 7faa53bbf700 1 -- 172.21.15.43:0/1317785572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa4c1a1970 con 0x7faa4c104bb0 2024-04-25T10:25:59.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.482+0000 7faa53bbf700 1 -- 172.21.15.43:0/1317785572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa4c051190 con 0x7faa4c104bb0 2024-04-25T10:25:59.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.484+0000 7faa3effd700 1 -- 172.21.15.43:0/1317785572 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7faa44024070 con 0x7faa4c104bb0 2024-04-25T10:25:59.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.484+0000 7faa3effd700 1 --2- 172.21.15.43:0/1317785572 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faa380afb70 0x7faa380b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.485+0000 7faa5195b700 1 --2- 172.21.15.43:0/1317785572 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faa380afb70 0x7faa380b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.485+0000 7faa3effd700 1 -- 172.21.15.43:0/1317785572 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7faa4400f020 con 0x7faa4c104bb0 2024-04-25T10:25:59.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.485+0000 7faa5195b700 1 --2- 172.21.15.43:0/1317785572 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faa380afb70 0x7faa380b2080 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faa4c104110 tx=0x7faa40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:59.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.488+0000 7faa3effd700 1 -- 172.21.15.43:0/1317785572 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa4402b020 con 0x7faa4c104bb0 2024-04-25T10:25:59.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:25:59.704+0000 7faa53bbf700 1 -- 172.21.15.43:0/1317785572 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7faa4c063ad0 con 0x7faa380afb70 2024-04-25T10:26:00.289 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:26:00 smithi043 sudo[46320]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T10:26:00.289 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:26:00 smithi043 sudo[46320]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:26:00.289 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:26:00 smithi043 sudo[46320]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:26:00.289 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:26:00 smithi043 sudo[46320]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:26:00.290 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:26:00 smithi043 sudo[46323]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T10:26:00.290 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:26:00 smithi043 sudo[46323]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:26:00.290 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:26:00 smithi043 sudo[46323]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:26:00.290 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:26:00 smithi043 sudo[46323]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:26:00.290 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:00 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T10:26:00.290 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:00 smithi043 ceph-mon[29478]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T10:26:00.290 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:00 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T10:26:00.290 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:00 smithi043 ceph-mon[29478]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:26:00.290 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:00 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:26:00.290 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:00 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:26:00.290 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:00 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:00.291 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:00 smithi043 sudo[46384]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T10:26:00.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:00 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T10:26:00.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:00 smithi077 ceph-mon[32772]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T10:26:00.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:00 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T10:26:00.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:00 smithi077 ceph-mon[32772]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:26:00.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:00 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:26:00.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:00 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:26:00.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:00 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:00.292 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:26:00 smithi077 sudo[41536]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T10:26:00.292 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:26:00 smithi077 sudo[41536]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:26:00.292 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:26:00 smithi077 sudo[41536]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:26:00.292 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:26:00 smithi077 sudo[41536]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:26:00.292 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:26:00 smithi077 sudo[41539]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T10:26:00.292 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:26:00 smithi077 sudo[41539]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:26:00.292 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:26:00 smithi077 sudo[41539]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:26:00.292 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:26:00 smithi077 sudo[41539]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:26:00.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:00 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T10:26:00.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:00 smithi164 ceph-mon[32713]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T10:26:00.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:00 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T10:26:00.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:00 smithi164 ceph-mon[32713]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:26:00.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:00 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:26:00.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:00 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:26:00.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:00 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:00.293 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:26:00 smithi164 sudo[41937]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T10:26:00.293 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:26:00 smithi164 sudo[41937]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:26:00.293 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:26:00 smithi164 sudo[41937]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:26:00.293 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:26:00 smithi164 sudo[41937]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:26:00.293 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:26:00 smithi164 sudo[41940]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T10:26:00.294 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:26:00 smithi164 sudo[41940]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:26:00.294 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:26:00 smithi164 sudo[41940]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:26:00.294 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:26:00 smithi164 sudo[41940]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:26:00.608 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:00 smithi043 sudo[46384]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:26:00.608 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:00 smithi043 sudo[46384]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:26:00.608 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:00 smithi043 sudo[46384]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:26:00.608 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:00 smithi043 sudo[46387]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T10:26:00.608 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:00 smithi043 sudo[46387]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:26:00.608 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:00 smithi043 sudo[46387]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:26:00.608 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:00 smithi043 sudo[46387]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:26:00.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:00 smithi077 sudo[41542]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T10:26:00.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:00 smithi077 sudo[41542]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:26:00.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:00 smithi077 sudo[41542]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:26:00.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:00 smithi077 sudo[41542]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:26:00.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:00 smithi077 sudo[41545]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T10:26:00.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:00 smithi077 sudo[41545]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:26:00.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:00 smithi077 sudo[41545]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:26:00.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:00 smithi077 sudo[41545]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:26:00.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:00 smithi164 sudo[41943]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T10:26:00.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:00 smithi164 sudo[41943]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:26:00.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:00 smithi164 sudo[41943]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:26:00.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:00 smithi164 sudo[41943]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:26:00.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:00 smithi164 sudo[41946]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T10:26:00.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:00 smithi164 sudo[41946]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T10:26:00.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:00 smithi164 sudo[41946]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T10:26:00.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:00 smithi164 sudo[41946]: pam_unix(sudo:session): session closed for user root 2024-04-25T10:26:01.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:26:01.048+0000 7faa3effd700 1 -- 172.21.15.43:0/1317785572 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7faa440a4eb0 con 0x7faa4c104bb0 2024-04-25T10:26:01.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T10:26:01.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T10:26:01.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:01.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:01.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:01.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:01.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T10:26:01.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T10:26:01.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:26:01.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:26:01.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:26:01.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T10:26:01.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:26:01.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:26:01.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:26:01.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T10:26:01.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T10:26:01.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:26:01.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:26:01.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:26:01.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:01 smithi043 ceph-mon[29478]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T10:26:01.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T10:26:01.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T10:26:01.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:01.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:01.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:01.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:01.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T10:26:01.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T10:26:01.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:26:01.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:26:01.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:26:01.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T10:26:01.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:26:01.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:26:01.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:26:01.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T10:26:01.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T10:26:01.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:26:01.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:26:01.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:26:01.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:01 smithi077 ceph-mon[32772]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T10:26:01.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T10:26:01.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T10:26:01.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:01.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:01.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:01.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:01.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T10:26:01.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T10:26:01.544 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:26:01.544 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:26:01.544 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:26:01.544 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T10:26:01.544 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:26:01.544 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:26:01.544 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:26:01.544 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T10:26:01.544 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T10:26:01.544 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:26:01.544 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:26:01.544 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:26:01.544 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:01 smithi164 ceph-mon[32713]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T10:26:02.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:02 smithi043 ceph-mon[29478]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T10:26:02.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:02 smithi043 ceph-mon[29478]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T10:26:02.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:02 smithi077 ceph-mon[32772]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T10:26:02.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:02 smithi077 ceph-mon[32772]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T10:26:02.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:02 smithi164 ceph-mon[32713]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T10:26:02.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:02 smithi164 ceph-mon[32713]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T10:26:03.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:03 smithi043 ceph-mon[29478]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:03.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:03 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/4271595733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "584d77ca-fb6f-4b35-9a11-2d2827de6dc8"}]: dispatch 2024-04-25T10:26:03.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:03 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/4271595733' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "584d77ca-fb6f-4b35-9a11-2d2827de6dc8"}]': finished 2024-04-25T10:26:03.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:03 smithi043 ceph-mon[29478]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T10:26:03.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:26:03.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:03 smithi077 ceph-mon[32772]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:03.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:03 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/4271595733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "584d77ca-fb6f-4b35-9a11-2d2827de6dc8"}]: dispatch 2024-04-25T10:26:03.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:03 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/4271595733' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "584d77ca-fb6f-4b35-9a11-2d2827de6dc8"}]': finished 2024-04-25T10:26:03.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:03 smithi077 ceph-mon[32772]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T10:26:03.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:26:03.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:03 smithi164 ceph-mon[32713]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:03.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:03 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/4271595733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "584d77ca-fb6f-4b35-9a11-2d2827de6dc8"}]: dispatch 2024-04-25T10:26:03.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:03 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/4271595733' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "584d77ca-fb6f-4b35-9a11-2d2827de6dc8"}]': finished 2024-04-25T10:26:03.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:03 smithi164 ceph-mon[32713]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T10:26:03.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:26:04.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:04 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1438041617' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:26:04.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:04 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1438041617' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:26:04.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:04 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1438041617' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:26:05.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:05 smithi043 ceph-mon[29478]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:05.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:05 smithi077 ceph-mon[32772]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:05.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:05 smithi164 ceph-mon[32713]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:07.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:07 smithi043 ceph-mon[29478]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:07.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:07 smithi077 ceph-mon[32772]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:07.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:07 smithi164 ceph-mon[32713]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:09.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:09 smithi043 ceph-mon[29478]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:09.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:09 smithi077 ceph-mon[32772]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:09.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:09 smithi164 ceph-mon[32713]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:10.498 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:10 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T10:26:10.498 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:10 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:10.498 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:10 smithi043 ceph-mon[29478]: Deploying daemon osd.3 on smithi043 2024-04-25T10:26:10.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:10 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T10:26:10.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:10 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:10.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:10 smithi077 ceph-mon[32772]: Deploying daemon osd.3 on smithi043 2024-04-25T10:26:10.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:10 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T10:26:10.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:10 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:10.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:10 smithi164 ceph-mon[32713]: Deploying daemon osd.3 on smithi043 2024-04-25T10:26:11.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:11 smithi164 ceph-mon[32713]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:11.519 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:11 smithi043 ceph-mon[29478]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:11.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:11 smithi077 ceph-mon[32772]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:13.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:13 smithi043 ceph-mon[29478]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:13.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:13 smithi077 ceph-mon[32772]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:13.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:13 smithi164 ceph-mon[32713]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:15.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:15 smithi077 ceph-mon[32772]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:15.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:15 smithi164 ceph-mon[32713]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:15.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:15 smithi043 ceph-mon[29478]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:16 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:16 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:16 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:26:16.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:16 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:16.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:16 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:16.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:16 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:26:16.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:16 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:16.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:16 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:16.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:16 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:26:17.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:17 smithi077 ceph-mon[32772]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:17.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:17 smithi164 ceph-mon[32713]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:17.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:17 smithi043 ceph-mon[29478]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:19.224 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 3 on host 'smithi043' 2024-04-25T10:26:19.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:26:19.221+0000 7faa3effd700 1 -- 172.21.15.43:0/1317785572 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7faa4c063ad0 con 0x7faa380afb70 2024-04-25T10:26:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:26:19.224+0000 7faa53bbf700 1 -- 172.21.15.43:0/1317785572 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faa380afb70 msgr2=0x7faa380b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:26:19.224+0000 7faa53bbf700 1 --2- 172.21.15.43:0/1317785572 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faa380afb70 0x7faa380b2080 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faa4c104110 tx=0x7faa40009260 comp rx=0 tx=0).stop 2024-04-25T10:26:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:26:19.224+0000 7faa53bbf700 1 -- 172.21.15.43:0/1317785572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa4c104bb0 msgr2=0x7faa4c1a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:26:19.224+0000 7faa53bbf700 1 --2- 172.21.15.43:0/1317785572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa4c104bb0 0x7faa4c1a0b20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa4400b070 tx=0x7faa44004ce0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:26:19.224+0000 7faa53bbf700 1 -- 172.21.15.43:0/1317785572 shutdown_connections 2024-04-25T10:26:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:26:19.224+0000 7faa53bbf700 1 --2- 172.21.15.43:0/1317785572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa4c104bb0 0x7faa4c1a0b20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:26:19.224+0000 7faa53bbf700 1 --2- 172.21.15.43:0/1317785572 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faa380afb70 0x7faa380b2080 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:26:19.224+0000 7faa53bbf700 1 --2- 172.21.15.43:0/1317785572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa4c1030e0 0x7faa4c199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:26:19.224+0000 7faa53bbf700 1 --2- 172.21.15.43:0/1317785572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa4c104210 0x7faa4c199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:26:19.224+0000 7faa53bbf700 1 -- 172.21.15.43:0/1317785572 >> 172.21.15.43:0/1317785572 conn(0x7faa4c0fe740 msgr2=0x7faa4c107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:26:19.224+0000 7faa53bbf700 1 -- 172.21.15.43:0/1317785572 shutdown_connections 2024-04-25T10:26:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:26:19.224+0000 7faa53bbf700 1 -- 172.21.15.43:0/1317785572 wait complete. 2024-04-25T10:26:19.516 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:19 smithi043 ceph-mon[29478]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:19.516 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:19 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:19.516 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:19 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:19.516 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:19 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:19.516 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:19 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:26:19.516 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:19 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:19.516 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:19 smithi043 ceph-mon[29478]: from='osd.3 [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T10:26:19.517 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:19 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:19.517 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:19 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:19.517 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:19 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:26:19.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:19 smithi164 ceph-mon[32713]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:19.695 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:19 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:19.695 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:19 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:19.695 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:19 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:19.695 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:19 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:26:19.695 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:19 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:19.695 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:19 smithi164 ceph-mon[32713]: from='osd.3 [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T10:26:19.695 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:19 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:19.695 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:19 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:19.695 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:19 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:26:19.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:19 smithi077 ceph-mon[32772]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:19.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:19 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:19.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:19 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:19.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:19 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:19.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:19 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:26:19.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:19 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:19.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:19 smithi077 ceph-mon[32772]: from='osd.3 [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T10:26:19.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:19 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:19.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:19 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:19.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:19 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:26:19.822 DEBUG:teuthology.orchestra.run.smithi043:osd.3> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T10:26:19.824 INFO:tasks.cephadm:Deploying osd.4 on smithi077 with /dev/vg_nvme/lv_3... 2024-04-25T10:26:19.824 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T10:26:20.069 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T10:26:20.515 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:20 smithi077 ceph-mon[32772]: from='osd.3 [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T10:26:20.515 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:20 smithi077 ceph-mon[32772]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T10:26:20.515 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:20 smithi077 ceph-mon[32772]: from='osd.3 [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-25T10:26:20.515 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:20 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:26:20.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:20 smithi043 ceph-mon[29478]: from='osd.3 [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T10:26:20.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:20 smithi043 ceph-mon[29478]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T10:26:20.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:20 smithi043 ceph-mon[29478]: from='osd.3 [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-25T10:26:20.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:20 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:26:20.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:20 smithi164 ceph-mon[32713]: from='osd.3 [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T10:26:20.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:20 smithi164 ceph-mon[32713]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T10:26:20.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:20 smithi164 ceph-mon[32713]: from='osd.3 [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-25T10:26:20.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:20 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:26:21.201 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 25 10:26:21 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-3[48519]: 2024-04-25T10:26:21.111+0000 7f83389c4700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T10:26:21.201 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 25 10:26:21 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-3[48519]: 2024-04-25T10:26:21.116+0000 7f8330771700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:26:21.498 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:21 smithi077 ceph-mon[32772]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:21.498 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:21 smithi077 ceph-mon[32772]: from='osd.3 [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-25T10:26:21.498 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:21 smithi077 ceph-mon[32772]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T10:26:21.498 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:21 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:26:21.498 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:21 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:26:21.499 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:21 smithi077 ceph-mon[32772]: from='osd.3 [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010]' entity='osd.3' 2024-04-25T10:26:21.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:21 smithi043 ceph-mon[29478]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:21.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:21 smithi043 ceph-mon[29478]: from='osd.3 [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-25T10:26:21.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:21 smithi043 ceph-mon[29478]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T10:26:21.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:21 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:26:21.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:21 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:26:21.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:21 smithi043 ceph-mon[29478]: from='osd.3 [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010]' entity='osd.3' 2024-04-25T10:26:21.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:21 smithi164 ceph-mon[32713]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T10:26:21.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:21 smithi164 ceph-mon[32713]: from='osd.3 [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-25T10:26:21.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:21 smithi164 ceph-mon[32713]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T10:26:21.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:21 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:26:21.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:21 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:26:21.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:21 smithi164 ceph-mon[32713]: from='osd.3 [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010]' entity='osd.3' 2024-04-25T10:26:22.075 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T10:26:22.076 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T10:26:22.076 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-04-25T10:26:22.076 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-04-25T10:26:22.076 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113837 s, 921 MB/s 2024-04-25T10:26:22.076 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-04-25T10:26:22.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:22 smithi043 ceph-mon[29478]: purged_snaps scrub starts 2024-04-25T10:26:22.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:22 smithi043 ceph-mon[29478]: purged_snaps scrub ok 2024-04-25T10:26:22.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:22 smithi043 ceph-mon[29478]: osd.3 [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010] boot 2024-04-25T10:26:22.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:22 smithi043 ceph-mon[29478]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T10:26:22.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:22 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:26:22.666 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:22 smithi077 ceph-mon[32772]: purged_snaps scrub starts 2024-04-25T10:26:22.666 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:22 smithi077 ceph-mon[32772]: purged_snaps scrub ok 2024-04-25T10:26:22.667 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:22 smithi077 ceph-mon[32772]: osd.3 [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010] boot 2024-04-25T10:26:22.667 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:22 smithi077 ceph-mon[32772]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T10:26:22.667 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:22 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:26:22.696 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_3 2024-04-25T10:26:22.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:22 smithi164 ceph-mon[32713]: purged_snaps scrub starts 2024-04-25T10:26:22.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:22 smithi164 ceph-mon[32713]: purged_snaps scrub ok 2024-04-25T10:26:22.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:22 smithi164 ceph-mon[32713]: osd.3 [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010] boot 2024-04-25T10:26:22.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:22 smithi164 ceph-mon[32713]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T10:26:22.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:22 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:26:22.955 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T10:26:23.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:23 smithi164 ceph-mon[32713]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:23.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:23 smithi164 ceph-mon[32713]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T10:26:23.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:23 smithi164 ceph-mon[32713]: Detected new or changed devices on smithi043 2024-04-25T10:26:23.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:23.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:23.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:26:23.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:26:23.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:23 smithi164 ceph-mon[32713]: Adjusting osd_memory_target on smithi043 to 8546M 2024-04-25T10:26:23.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:23.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:23.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:26:23.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:23.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:23 smithi043 ceph-mon[29478]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:23.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:23 smithi043 ceph-mon[29478]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T10:26:23.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:23 smithi043 ceph-mon[29478]: Detected new or changed devices on smithi043 2024-04-25T10:26:23.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:23.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:23.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:26:23.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:26:23.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:23 smithi043 ceph-mon[29478]: Adjusting osd_memory_target on smithi043 to 8546M 2024-04-25T10:26:23.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:23.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:23.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:26:23.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:23.739 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:23 smithi077 ceph-mon[32772]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:23.739 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:23 smithi077 ceph-mon[32772]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T10:26:23.740 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:23 smithi077 ceph-mon[32772]: Detected new or changed devices on smithi043 2024-04-25T10:26:23.740 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:23.740 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:23.740 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:26:23.740 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:26:23.740 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:23 smithi077 ceph-mon[32772]: Adjusting osd_memory_target on smithi043 to 8546M 2024-04-25T10:26:23.740 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:23.740 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:23.740 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:26:23.740 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:24.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.488+0000 7fb33ed46700 1 -- 172.21.15.77:0/86811568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3381030e0 msgr2=0x7fb3381034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.488+0000 7fb33ed46700 1 --2- 172.21.15.77:0/86811568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3381030e0 0x7fb3381034f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009990 tx=0x7fb32c009ca0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.488+0000 7fb33ed46700 1 -- 172.21.15.77:0/86811568 shutdown_connections 2024-04-25T10:26:24.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.488+0000 7fb33ed46700 1 --2- 172.21.15.77:0/86811568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb338104bb0 0x7fb33810b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.488+0000 7fb33ed46700 1 --2- 172.21.15.77:0/86811568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3381030e0 0x7fb3381034f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.488+0000 7fb33ed46700 1 --2- 172.21.15.77:0/86811568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb338104210 0x7fb338104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.488+0000 7fb33ed46700 1 -- 172.21.15.77:0/86811568 >> 172.21.15.77:0/86811568 conn(0x7fb3380fe740 msgr2=0x7fb338100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:24.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.488+0000 7fb33ed46700 1 -- 172.21.15.77:0/86811568 shutdown_connections 2024-04-25T10:26:24.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.488+0000 7fb33ed46700 1 -- 172.21.15.77:0/86811568 wait complete. 2024-04-25T10:26:24.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.488+0000 7fb33ed46700 1 Processor -- start 2024-04-25T10:26:24.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.488+0000 7fb33ed46700 1 -- start start 2024-04-25T10:26:24.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb33ed46700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3381030e0 0x7fb33819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb33ed46700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb338104210 0x7fb33819aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb33ed46700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb338104bb0 0x7fb3381a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb33ed46700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb33810ec20 con 0x7fb338104bb0 2024-04-25T10:26:24.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb33ed46700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb33810eaa0 con 0x7fb3381030e0 2024-04-25T10:26:24.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb33ed46700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb33810eda0 con 0x7fb338104210 2024-04-25T10:26:24.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb33cae2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3381030e0 0x7fb33819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb33cae2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3381030e0 0x7fb33819a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39710/0 (socket says 172.21.15.77:39710) 2024-04-25T10:26:24.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb33cae2700 1 -- 172.21.15.77:0/542975814 learned_addr learned my addr 172.21.15.77:0/542975814 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-25T10:26:24.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb33d2e3700 1 --2- 172.21.15.77:0/542975814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb338104bb0 0x7fb3381a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb337fff700 1 --2- 172.21.15.77:0/542975814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb338104210 0x7fb33819aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb33cae2700 1 -- 172.21.15.77:0/542975814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb338104210 msgr2=0x7fb33819aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb33cae2700 1 --2- 172.21.15.77:0/542975814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb338104210 0x7fb33819aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb33cae2700 1 -- 172.21.15.77:0/542975814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb338104bb0 msgr2=0x7fb3381a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb33cae2700 1 --2- 172.21.15.77:0/542975814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb338104bb0 0x7fb3381a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb33cae2700 1 -- 172.21.15.77:0/542975814 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3381a24c0 con 0x7fb3381030e0 2024-04-25T10:26:24.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb33cae2700 1 --2- 172.21.15.77:0/542975814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3381030e0 0x7fb33819a960 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb32c0057c0 tx=0x7fb32c0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:24.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb335ffb700 1 -- 172.21.15.77:0/542975814 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c027070 con 0x7fb3381030e0 2024-04-25T10:26:24.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb335ffb700 1 -- 172.21.15.77:0/542975814 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb32c004140 con 0x7fb3381030e0 2024-04-25T10:26:24.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb33ed46700 1 -- 172.21.15.77:0/542975814 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3381a2750 con 0x7fb3381030e0 2024-04-25T10:26:24.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.489+0000 7fb335ffb700 1 -- 172.21.15.77:0/542975814 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c022e10 con 0x7fb3381030e0 2024-04-25T10:26:24.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.490+0000 7fb33ed46700 1 -- 172.21.15.77:0/542975814 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3381a2c80 con 0x7fb3381030e0 2024-04-25T10:26:24.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.490+0000 7fb33ed46700 1 -- 172.21.15.77:0/542975814 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb338051190 con 0x7fb3381030e0 2024-04-25T10:26:24.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.493+0000 7fb335ffb700 1 -- 172.21.15.77:0/542975814 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fb32c02a070 con 0x7fb3381030e0 2024-04-25T10:26:24.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.493+0000 7fb335ffb700 1 --2- 172.21.15.77:0/542975814 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb3280afc00 0x7fb3280b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.493+0000 7fb335ffb700 1 -- 172.21.15.77:0/542975814 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7fb32c0decf0 con 0x7fb3381030e0 2024-04-25T10:26:24.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.494+0000 7fb337fff700 1 --2- 172.21.15.77:0/542975814 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb3280afc00 0x7fb3280b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.494+0000 7fb337fff700 1 --2- 172.21.15.77:0/542975814 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb3280afc00 0x7fb3280b2110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb324006fd0 tx=0x7fb324008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:24.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.497+0000 7fb335ffb700 1 -- 172.21.15.77:0/542975814 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb32c013070 con 0x7fb3381030e0 2024-04-25T10:26:24.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:24 smithi077 ceph-mon[32772]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T10:26:24.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:24.704+0000 7fb33ed46700 1 -- 172.21.15.77:0/542975814 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb338100830 con 0x7fb3280afc00 2024-04-25T10:26:24.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:24 smithi164 ceph-mon[32713]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T10:26:24.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:24 smithi043 ceph-mon[29478]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T10:26:25.612 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:25 smithi077 ceph-mon[32772]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:25.612 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:25 smithi077 ceph-mon[32772]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:26:25.612 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:25 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:26:25.612 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:25 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:26:25.612 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:25 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:25.762 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:25 smithi043 ceph-mon[29478]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:25.762 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:25 smithi043 ceph-mon[29478]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:26:25.762 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:25 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:26:25.762 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:25 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:26:25.762 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:25 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:25.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:25 smithi164 ceph-mon[32713]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:25.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:25 smithi164 ceph-mon[32713]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:26:25.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:25 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:26:25.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:25 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:26:25.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:25 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:27.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:27 smithi077 ceph-mon[32772]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-25T10:26:27.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:27 smithi164 ceph-mon[32713]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-25T10:26:27.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:27 smithi043 ceph-mon[29478]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-25T10:26:28.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:28 smithi077 ceph-mon[32772]: from='client.? 172.21.15.77:0/2139972765' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cf2e39e-255f-4410-aee3-938b0e19dff6"}]: dispatch 2024-04-25T10:26:28.948 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:28 smithi077 ceph-mon[32772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cf2e39e-255f-4410-aee3-938b0e19dff6"}]: dispatch 2024-04-25T10:26:28.948 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:28 smithi077 ceph-mon[32772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cf2e39e-255f-4410-aee3-938b0e19dff6"}]': finished 2024-04-25T10:26:28.949 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:28 smithi077 ceph-mon[32772]: osdmap e28: 5 total, 4 up, 5 in 2024-04-25T10:26:28.949 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:28 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:26:28.949 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:28 smithi077 ceph-mon[32772]: from='client.? 172.21.15.77:0/1556099807' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:26:28.949 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:28 smithi043 ceph-mon[29478]: from='client.? 172.21.15.77:0/2139972765' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cf2e39e-255f-4410-aee3-938b0e19dff6"}]: dispatch 2024-04-25T10:26:28.950 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:28 smithi043 ceph-mon[29478]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cf2e39e-255f-4410-aee3-938b0e19dff6"}]: dispatch 2024-04-25T10:26:28.950 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:28 smithi043 ceph-mon[29478]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cf2e39e-255f-4410-aee3-938b0e19dff6"}]': finished 2024-04-25T10:26:28.950 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:28 smithi043 ceph-mon[29478]: osdmap e28: 5 total, 4 up, 5 in 2024-04-25T10:26:28.950 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:28 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:26:28.950 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:28 smithi043 ceph-mon[29478]: from='client.? 172.21.15.77:0/1556099807' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:26:28.950 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:28 smithi164 ceph-mon[32713]: from='client.? 172.21.15.77:0/2139972765' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cf2e39e-255f-4410-aee3-938b0e19dff6"}]: dispatch 2024-04-25T10:26:28.950 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:28 smithi164 ceph-mon[32713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cf2e39e-255f-4410-aee3-938b0e19dff6"}]: dispatch 2024-04-25T10:26:28.950 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:28 smithi164 ceph-mon[32713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cf2e39e-255f-4410-aee3-938b0e19dff6"}]': finished 2024-04-25T10:26:28.950 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:28 smithi164 ceph-mon[32713]: osdmap e28: 5 total, 4 up, 5 in 2024-04-25T10:26:28.950 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:28 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:26:28.950 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:28 smithi164 ceph-mon[32713]: from='client.? 172.21.15.77:0/1556099807' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:26:29.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:29 smithi077 ceph-mon[32772]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T10:26:29.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:29 smithi164 ceph-mon[32713]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T10:26:29.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:29 smithi043 ceph-mon[29478]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T10:26:31.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:31 smithi077 ceph-mon[32772]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-25T10:26:31.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:31 smithi164 ceph-mon[32713]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-25T10:26:31.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:31 smithi043 ceph-mon[29478]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-25T10:26:33.738 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:33 smithi077 ceph-mon[32772]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-25T10:26:33.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:33 smithi164 ceph-mon[32713]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-25T10:26:33.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:33 smithi043 ceph-mon[29478]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-25T10:26:35.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:35 smithi077 ceph-mon[32772]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:26:35.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:35 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T10:26:35.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:35 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:35.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:35 smithi164 ceph-mon[32713]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:26:35.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:35 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T10:26:35.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:35 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:35.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:35 smithi043 ceph-mon[29478]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:26:35.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:35 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T10:26:35.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:35 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:36.650 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:36 smithi077 ceph-mon[32772]: Deploying daemon osd.4 on smithi077 2024-04-25T10:26:36.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:36 smithi164 ceph-mon[32713]: Deploying daemon osd.4 on smithi077 2024-04-25T10:26:36.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:36 smithi043 ceph-mon[29478]: Deploying daemon osd.4 on smithi077 2024-04-25T10:26:37.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:37 smithi077 ceph-mon[32772]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:37.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:37 smithi164 ceph-mon[32713]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:37.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:37 smithi043 ceph-mon[29478]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:39.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:39 smithi077 ceph-mon[32772]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:39.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:39 smithi164 ceph-mon[32713]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:39.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:39 smithi043 ceph-mon[29478]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:42.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:41 smithi077 ceph-mon[32772]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:42.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:41 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:42.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:41 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:42.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:41 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:26:42.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:41 smithi164 ceph-mon[32713]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:42.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:41 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:42.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:41 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:42.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:41 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:26:42.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:41 smithi043 ceph-mon[29478]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:42.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:41 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:42.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:41 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:42.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:41 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:26:43.908 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:43 smithi077 ceph-mon[32772]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:44.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:43 smithi164 ceph-mon[32713]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:44.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:43 smithi043 ceph-mon[29478]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:44.278 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 4 on host 'smithi077' 2024-04-25T10:26:44.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:44.275+0000 7fb335ffb700 1 -- 172.21.15.77:0/542975814 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb338100830 con 0x7fb3280afc00 2024-04-25T10:26:44.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:44.278+0000 7fb33ed46700 1 -- 172.21.15.77:0/542975814 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb3280afc00 msgr2=0x7fb3280b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:44.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:44.278+0000 7fb33ed46700 1 --2- 172.21.15.77:0/542975814 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb3280afc00 0x7fb3280b2110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb324006fd0 tx=0x7fb324008040 comp rx=0 tx=0).stop 2024-04-25T10:26:44.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:44.278+0000 7fb33ed46700 1 -- 172.21.15.77:0/542975814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3381030e0 msgr2=0x7fb33819a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:44.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:44.278+0000 7fb33ed46700 1 --2- 172.21.15.77:0/542975814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3381030e0 0x7fb33819a960 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb32c0057c0 tx=0x7fb32c0057f0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:44.278+0000 7fb33ed46700 1 -- 172.21.15.77:0/542975814 shutdown_connections 2024-04-25T10:26:44.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:44.278+0000 7fb33ed46700 1 --2- 172.21.15.77:0/542975814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb338104bb0 0x7fb3381a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:44.278+0000 7fb33ed46700 1 --2- 172.21.15.77:0/542975814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3381030e0 0x7fb33819a960 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:44.278+0000 7fb33ed46700 1 --2- 172.21.15.77:0/542975814 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb3280afc00 0x7fb3280b2110 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:44.278+0000 7fb33ed46700 1 --2- 172.21.15.77:0/542975814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb338104210 0x7fb33819aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:44.278+0000 7fb33ed46700 1 -- 172.21.15.77:0/542975814 >> 172.21.15.77:0/542975814 conn(0x7fb3380fe740 msgr2=0x7fb338100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:44.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:44.278+0000 7fb33ed46700 1 -- 172.21.15.77:0/542975814 shutdown_connections 2024-04-25T10:26:44.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:26:44.278+0000 7fb33ed46700 1 -- 172.21.15.77:0/542975814 wait complete. 2024-04-25T10:26:44.864 DEBUG:teuthology.orchestra.run.smithi077:osd.4> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T10:26:44.866 INFO:tasks.cephadm:Deploying osd.5 on smithi164 with /dev/vg_nvme/lv_3... 2024-04-25T10:26:44.867 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T10:26:45.113 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:26:45.538 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:45 smithi164 ceph-mon[32713]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:45.539 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:45 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:45.539 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:45 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:45.539 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:45 smithi164 ceph-mon[32713]: from='osd.4 [v2:172.21.15.77:6808/3648991803,v1:172.21.15.77:6809/3648991803]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T10:26:45.539 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:45 smithi164 ceph-mon[32713]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T10:26:45.539 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:45 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:45.539 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:45 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:45.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:45 smithi077 ceph-mon[32772]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:45.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:45 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:45.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:45 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:45.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:45 smithi077 ceph-mon[32772]: from='osd.4 [v2:172.21.15.77:6808/3648991803,v1:172.21.15.77:6809/3648991803]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T10:26:45.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:45 smithi077 ceph-mon[32772]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T10:26:45.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:45 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:45.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:45 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:45.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:45 smithi043 ceph-mon[29478]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:45.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:45 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:45.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:45 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:45.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:45 smithi043 ceph-mon[29478]: from='osd.4 [v2:172.21.15.77:6808/3648991803,v1:172.21.15.77:6809/3648991803]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T10:26:45.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:45 smithi043 ceph-mon[29478]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T10:26:45.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:45 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:45.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:45 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:46.447 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:46 smithi164 ceph-mon[32713]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T10:26:46.447 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:46 smithi164 ceph-mon[32713]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T10:26:46.447 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:46 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:26:46.448 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:46 smithi164 ceph-mon[32713]: from='osd.4 [v2:172.21.15.77:6808/3648991803,v1:172.21.15.77:6809/3648991803]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:26:46.448 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:46 smithi164 ceph-mon[32713]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:26:46.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:46 smithi077 ceph-mon[32772]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T10:26:46.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:46 smithi077 ceph-mon[32772]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T10:26:46.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:46 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:26:46.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:46 smithi077 ceph-mon[32772]: from='osd.4 [v2:172.21.15.77:6808/3648991803,v1:172.21.15.77:6809/3648991803]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:26:46.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:46 smithi077 ceph-mon[32772]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:26:46.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:46 smithi043 ceph-mon[29478]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T10:26:46.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:46 smithi043 ceph-mon[29478]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T10:26:46.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:46 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:26:46.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:46 smithi043 ceph-mon[29478]: from='osd.4 [v2:172.21.15.77:6808/3648991803,v1:172.21.15.77:6809/3648991803]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:26:46.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:46 smithi043 ceph-mon[29478]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:26:46.978 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T10:26:46.978 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T10:26:46.978 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-04-25T10:26:46.978 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-04-25T10:26:46.979 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114575 s, 915 MB/s 2024-04-25T10:26:46.979 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-04-25T10:26:47.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:47 smithi077 ceph-mon[32772]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:47.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:47 smithi077 ceph-mon[32772]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-25T10:26:47.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:47 smithi077 ceph-mon[32772]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T10:26:47.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:47 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:26:47.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:47 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:26:47.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:47 smithi077 ceph-mon[32772]: from='osd.4 ' entity='osd.4' 2024-04-25T10:26:47.542 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 25 10:26:47 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-4[44364]: 2024-04-25T10:26:47.158+0000 7eff2ae43700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T10:26:47.542 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 25 10:26:47 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-4[44364]: 2024-04-25T10:26:47.241+0000 7eff22bf0700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:26:47.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:47 smithi164 ceph-mon[32713]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:47.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:47 smithi164 ceph-mon[32713]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-25T10:26:47.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:47 smithi164 ceph-mon[32713]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T10:26:47.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:47 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:26:47.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:47 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:26:47.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:47 smithi164 ceph-mon[32713]: from='osd.4 ' entity='osd.4' 2024-04-25T10:26:47.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:47 smithi043 ceph-mon[29478]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:47.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:47 smithi043 ceph-mon[29478]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-25T10:26:47.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:47 smithi043 ceph-mon[29478]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T10:26:47.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:47 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:26:47.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:47 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:26:47.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:47 smithi043 ceph-mon[29478]: from='osd.4 ' entity='osd.4' 2024-04-25T10:26:47.661 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_3 2024-04-25T10:26:47.906 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:26:48.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:48 smithi164 ceph-mon[32713]: purged_snaps scrub starts 2024-04-25T10:26:48.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:48 smithi164 ceph-mon[32713]: purged_snaps scrub ok 2024-04-25T10:26:48.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:48 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:26:48.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:48 smithi164 ceph-mon[32713]: osd.4 [v2:172.21.15.77:6808/3648991803,v1:172.21.15.77:6809/3648991803] boot 2024-04-25T10:26:48.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:48 smithi164 ceph-mon[32713]: osdmap e31: 5 total, 5 up, 5 in 2024-04-25T10:26:48.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:48 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:26:48.357 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:48 smithi077 ceph-mon[32772]: purged_snaps scrub starts 2024-04-25T10:26:48.357 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:48 smithi077 ceph-mon[32772]: purged_snaps scrub ok 2024-04-25T10:26:48.358 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:48 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:26:48.358 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:48 smithi077 ceph-mon[32772]: osd.4 [v2:172.21.15.77:6808/3648991803,v1:172.21.15.77:6809/3648991803] boot 2024-04-25T10:26:48.358 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:48 smithi077 ceph-mon[32772]: osdmap e31: 5 total, 5 up, 5 in 2024-04-25T10:26:48.358 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:48 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:26:48.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:48 smithi043 ceph-mon[29478]: purged_snaps scrub starts 2024-04-25T10:26:48.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:48 smithi043 ceph-mon[29478]: purged_snaps scrub ok 2024-04-25T10:26:48.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:48 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:26:48.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:48 smithi043 ceph-mon[29478]: osd.4 [v2:172.21.15.77:6808/3648991803,v1:172.21.15.77:6809/3648991803] boot 2024-04-25T10:26:48.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:48 smithi043 ceph-mon[29478]: osdmap e31: 5 total, 5 up, 5 in 2024-04-25T10:26:48.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:48 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:26:49.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.516+0000 7f144da58700 1 -- 172.21.15.164:0/243107084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1448105370 msgr2=0x7f1448105780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:49.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.516+0000 7f144da58700 1 --2- 172.21.15.164:0/243107084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1448105370 0x7f1448105780 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1438009990 tx=0x7f1438009ca0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.516+0000 7f144da58700 1 -- 172.21.15.164:0/243107084 shutdown_connections 2024-04-25T10:26:49.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.516+0000 7f144da58700 1 --2- 172.21.15.164:0/243107084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1448105d10 0x7f144810c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.516+0000 7f144da58700 1 --2- 172.21.15.164:0/243107084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1448104240 0x7f1448104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.516+0000 7f144da58700 1 --2- 172.21.15.164:0/243107084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1448105370 0x7f1448105780 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.516+0000 7f144da58700 1 -- 172.21.15.164:0/243107084 >> 172.21.15.164:0/243107084 conn(0x7f14480ff8a0 msgr2=0x7f1448101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:49.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.516+0000 7f144da58700 1 -- 172.21.15.164:0/243107084 shutdown_connections 2024-04-25T10:26:49.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.516+0000 7f144da58700 1 -- 172.21.15.164:0/243107084 wait complete. 2024-04-25T10:26:49.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.517+0000 7f144da58700 1 Processor -- start 2024-04-25T10:26:49.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.517+0000 7f144da58700 1 -- start start 2024-04-25T10:26:49.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.517+0000 7f144da58700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1448104240 0x7f144819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:49.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.517+0000 7f144da58700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1448105370 0x7f144819acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:49.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.517+0000 7f144da58700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1448105d10 0x7f14481a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:49.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.517+0000 7f144da58700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f144810fca0 con 0x7f1448104240 2024-04-25T10:26:49.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.517+0000 7f144da58700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f144810fb20 con 0x7f1448105370 2024-04-25T10:26:49.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.517+0000 7f144da58700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f144810fe20 con 0x7f1448105d10 2024-04-25T10:26:49.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.517+0000 7f14477fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1448105d10 0x7f14481a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:49.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.517+0000 7f14477fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1448105d10 0x7f14481a1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60226/0 (socket says 172.21.15.164:60226) 2024-04-25T10:26:49.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.517+0000 7f14477fe700 1 -- 172.21.15.164:0/2195811404 learned_addr learned my addr 172.21.15.164:0/2195811404 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-25T10:26:49.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.517+0000 7f1446ffd700 1 --2- 172.21.15.164:0/2195811404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1448104240 0x7f144819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:49.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.518+0000 7f14467fc700 1 --2- 172.21.15.164:0/2195811404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1448105370 0x7f144819acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:49.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.518+0000 7f14477fe700 1 -- 172.21.15.164:0/2195811404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1448105370 msgr2=0x7f144819acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:49.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.518+0000 7f14477fe700 1 --2- 172.21.15.164:0/2195811404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1448105370 0x7f144819acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.518+0000 7f14477fe700 1 -- 172.21.15.164:0/2195811404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1448104240 msgr2=0x7f144819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:49.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.518+0000 7f14477fe700 1 --2- 172.21.15.164:0/2195811404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1448104240 0x7f144819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.518+0000 7f14477fe700 1 -- 172.21.15.164:0/2195811404 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14481a2310 con 0x7f1448105d10 2024-04-25T10:26:49.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.518+0000 7f14477fe700 1 --2- 172.21.15.164:0/2195811404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1448105d10 0x7f14481a1c90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1440007ee0 tx=0x7f144000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:49.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.518+0000 7f144ca56700 1 -- 172.21.15.164:0/2195811404 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144000dbd0 con 0x7f1448105d10 2024-04-25T10:26:49.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.518+0000 7f144da58700 1 -- 172.21.15.164:0/2195811404 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14481a2600 con 0x7f1448105d10 2024-04-25T10:26:49.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.518+0000 7f144ca56700 1 -- 172.21.15.164:0/2195811404 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f144000dd70 con 0x7f1448105d10 2024-04-25T10:26:49.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.518+0000 7f144ca56700 1 -- 172.21.15.164:0/2195811404 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440005990 con 0x7f1448105d10 2024-04-25T10:26:49.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.518+0000 7f144da58700 1 -- 172.21.15.164:0/2195811404 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14481a2af0 con 0x7f1448105d10 2024-04-25T10:26:49.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.519+0000 7f144da58700 1 -- 172.21.15.164:0/2195811404 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1448051190 con 0x7f1448105d10 2024-04-25T10:26:49.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.520+0000 7f144ca56700 1 -- 172.21.15.164:0/2195811404 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f144001a020 con 0x7f1448105d10 2024-04-25T10:26:49.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.521+0000 7f144ca56700 1 --2- 172.21.15.164:0/2195811404 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f14340afb80 0x7f14340b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:49.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.521+0000 7f144ca56700 1 -- 172.21.15.164:0/2195811404 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7f14400dd9e0 con 0x7f1448105d10 2024-04-25T10:26:49.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.521+0000 7f1446ffd700 1 --2- 172.21.15.164:0/2195811404 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f14340afb80 0x7f14340b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:49.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.522+0000 7f1446ffd700 1 --2- 172.21.15.164:0/2195811404 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f14340afb80 0x7f14340b2090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1448105270 tx=0x7f143000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:49.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.525+0000 7f144ca56700 1 -- 172.21.15.164:0/2195811404 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1440010070 con 0x7f1448105d10 2024-04-25T10:26:49.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:26:49.730+0000 7f144da58700 1 -- 172.21.15.164:0/2195811404 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1448063ad0 con 0x7f14340afb80 2024-04-25T10:26:49.732 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:49 smithi164 ceph-mon[32713]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:49.732 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:49 smithi164 ceph-mon[32713]: Detected new or changed devices on smithi077 2024-04-25T10:26:49.732 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:49.732 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:49.732 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:26:49.732 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:26:49.732 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:49 smithi164 ceph-mon[32713]: Adjusting osd_memory_target on smithi077 to 8544M 2024-04-25T10:26:49.732 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:49.733 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:49.733 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:26:49.733 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:49.733 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:26:49.733 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:49.733 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:26:49.733 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:49.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:49 smithi077 ceph-mon[32772]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:49.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:49 smithi077 ceph-mon[32772]: Detected new or changed devices on smithi077 2024-04-25T10:26:49.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:49.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:49.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:26:49.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:26:49.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:49 smithi077 ceph-mon[32772]: Adjusting osd_memory_target on smithi077 to 8544M 2024-04-25T10:26:49.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:49.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:49.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:26:49.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:49.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:26:49.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:49.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:26:49.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:49.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:49 smithi043 ceph-mon[29478]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T10:26:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:49 smithi043 ceph-mon[29478]: Detected new or changed devices on smithi077 2024-04-25T10:26:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:26:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:26:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:49 smithi043 ceph-mon[29478]: Adjusting osd_memory_target on smithi077 to 8544M 2024-04-25T10:26:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:26:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:26:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:26:49.862 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:26:50.597 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:50 smithi164 ceph-mon[32713]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T10:26:50.597 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:50 smithi164 ceph-mon[32713]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:26:50.597 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:50 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:26:50.597 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:50 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:26:50.597 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:50 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:50.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:50 smithi077 ceph-mon[32772]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T10:26:50.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:50 smithi077 ceph-mon[32772]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:26:50.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:50 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:26:50.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:50 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:26:50.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:50 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:50.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:50 smithi043 ceph-mon[29478]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T10:26:50.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:50 smithi043 ceph-mon[29478]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:26:50.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:50 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:26:50.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:50 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:26:50.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:50 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:26:51.594 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:51 smithi164 ceph-mon[32713]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:26:51.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:51 smithi077 ceph-mon[32772]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:26:51.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:51 smithi043 ceph-mon[29478]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:26:53.767 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:53 smithi077 ceph-mon[32772]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:26:53.768 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:53 smithi077 ceph-mon[32772]: from='client.? 172.21.15.164:0/1919863778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb7aa97e-5dde-4866-9c74-547e4ebdc619"}]: dispatch 2024-04-25T10:26:53.768 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:53 smithi077 ceph-mon[32772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb7aa97e-5dde-4866-9c74-547e4ebdc619"}]: dispatch 2024-04-25T10:26:53.768 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:53 smithi077 ceph-mon[32772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb7aa97e-5dde-4866-9c74-547e4ebdc619"}]': finished 2024-04-25T10:26:53.768 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:53 smithi077 ceph-mon[32772]: osdmap e33: 6 total, 5 up, 6 in 2024-04-25T10:26:53.768 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:53 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:26:53.768 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:53 smithi077 ceph-mon[32772]: from='client.? 172.21.15.164:0/3378023662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:26:53.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:53 smithi164 ceph-mon[32713]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:26:53.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:53 smithi164 ceph-mon[32713]: from='client.? 172.21.15.164:0/1919863778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb7aa97e-5dde-4866-9c74-547e4ebdc619"}]: dispatch 2024-04-25T10:26:53.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:53 smithi164 ceph-mon[32713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb7aa97e-5dde-4866-9c74-547e4ebdc619"}]: dispatch 2024-04-25T10:26:53.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:53 smithi164 ceph-mon[32713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb7aa97e-5dde-4866-9c74-547e4ebdc619"}]': finished 2024-04-25T10:26:53.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:53 smithi164 ceph-mon[32713]: osdmap e33: 6 total, 5 up, 6 in 2024-04-25T10:26:53.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:53 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:26:53.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:53 smithi164 ceph-mon[32713]: from='client.? 172.21.15.164:0/3378023662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:26:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:53 smithi043 ceph-mon[29478]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:26:53.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:53 smithi043 ceph-mon[29478]: from='client.? 172.21.15.164:0/1919863778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb7aa97e-5dde-4866-9c74-547e4ebdc619"}]: dispatch 2024-04-25T10:26:53.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:53 smithi043 ceph-mon[29478]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb7aa97e-5dde-4866-9c74-547e4ebdc619"}]: dispatch 2024-04-25T10:26:53.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:53 smithi043 ceph-mon[29478]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb7aa97e-5dde-4866-9c74-547e4ebdc619"}]': finished 2024-04-25T10:26:53.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:53 smithi043 ceph-mon[29478]: osdmap e33: 6 total, 5 up, 6 in 2024-04-25T10:26:53.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:53 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:26:53.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:53 smithi043 ceph-mon[29478]: from='client.? 172.21.15.164:0/3378023662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:26:55.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:55 smithi077 ceph-mon[32772]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:26:55.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:55 smithi164 ceph-mon[32713]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:26:55.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:55 smithi043 ceph-mon[29478]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:26:57.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:57 smithi077 ceph-mon[32772]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:26:57.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:57 smithi164 ceph-mon[32713]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:26:57.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:57 smithi043 ceph-mon[29478]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:26:59.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:26:59 smithi077 ceph-mon[32772]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:26:59.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:26:59 smithi164 ceph-mon[32713]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:26:59.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:26:59 smithi043 ceph-mon[29478]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:00.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:00 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T10:27:00.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:00 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:00.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:00 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T10:27:00.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:00 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:00.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:00 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T10:27:00.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:00 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:01.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:01 smithi164 ceph-mon[32713]: Deploying daemon osd.5 on smithi164 2024-04-25T10:27:01.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:01 smithi164 ceph-mon[32713]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:01.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:01 smithi077 ceph-mon[32772]: Deploying daemon osd.5 on smithi164 2024-04-25T10:27:01.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:01 smithi077 ceph-mon[32772]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:01.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:01 smithi043 ceph-mon[29478]: Deploying daemon osd.5 on smithi164 2024-04-25T10:27:01.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:01 smithi043 ceph-mon[29478]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:03.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:03 smithi077 ceph-mon[32772]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:03.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:03 smithi164 ceph-mon[32713]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:03.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:03 smithi043 ceph-mon[29478]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:05.767 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:05 smithi164 ceph-mon[32713]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:05.767 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:05 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:05.767 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:05 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:05.767 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:05 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:27:05.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:05 smithi077 ceph-mon[32772]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:05.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:05 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:05.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:05 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:05.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:05 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:27:05.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:05 smithi043 ceph-mon[29478]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:05.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:05 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:05.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:05 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:05.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:05 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:27:07.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:07 smithi077 ceph-mon[32772]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:07.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:07 smithi164 ceph-mon[32713]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:07 smithi043 ceph-mon[29478]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:08.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:27:08.898+0000 7f144ca56700 1 -- 172.21.15.164:0/2195811404 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1448063ad0 con 0x7f14340afb80 2024-04-25T10:27:08.901 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 5 on host 'smithi164' 2024-04-25T10:27:08.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:27:08.901+0000 7f144da58700 1 -- 172.21.15.164:0/2195811404 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f14340afb80 msgr2=0x7f14340b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:08.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:27:08.901+0000 7f144da58700 1 --2- 172.21.15.164:0/2195811404 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f14340afb80 0x7f14340b2090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1448105270 tx=0x7f143000a380 comp rx=0 tx=0).stop 2024-04-25T10:27:08.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:27:08.901+0000 7f144da58700 1 -- 172.21.15.164:0/2195811404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1448105d10 msgr2=0x7f14481a1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:08.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:27:08.901+0000 7f144da58700 1 --2- 172.21.15.164:0/2195811404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1448105d10 0x7f14481a1c90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1440007ee0 tx=0x7f144000d3b0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:27:08.901+0000 7f144da58700 1 -- 172.21.15.164:0/2195811404 shutdown_connections 2024-04-25T10:27:08.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:27:08.901+0000 7f144da58700 1 --2- 172.21.15.164:0/2195811404 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f14340afb80 0x7f14340b2090 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:27:08.901+0000 7f144da58700 1 --2- 172.21.15.164:0/2195811404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1448105d10 0x7f14481a1c90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:27:08.901+0000 7f144da58700 1 --2- 172.21.15.164:0/2195811404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1448104240 0x7f144819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:27:08.901+0000 7f144da58700 1 --2- 172.21.15.164:0/2195811404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1448105370 0x7f144819acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:27:08.901+0000 7f144da58700 1 -- 172.21.15.164:0/2195811404 >> 172.21.15.164:0/2195811404 conn(0x7f14480ff8a0 msgr2=0x7f14481085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:08.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:27:08.902+0000 7f144da58700 1 -- 172.21.15.164:0/2195811404 shutdown_connections 2024-04-25T10:27:08.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:27:08.902+0000 7f144da58700 1 -- 172.21.15.164:0/2195811404 wait complete. 2024-04-25T10:27:09.264 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:08 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:09.265 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:08 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:09.265 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:08 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:09.265 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:08 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:27:09.265 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:08 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:09.265 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:08 smithi164 ceph-mon[32713]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:09.265 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:08 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:09.265 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:08 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:09.265 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:08 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:27:09.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:08 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:09.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:08 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:09.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:08 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:09.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:08 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:27:09.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:08 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:09.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:08 smithi077 ceph-mon[32772]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:09.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:08 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:09.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:08 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:09.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:08 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:27:09.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:08 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:09.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:08 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:09.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:08 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:09.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:08 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:27:09.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:08 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:09.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:08 smithi043 ceph-mon[29478]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:09.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:08 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:09.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:08 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:09.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:08 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:27:09.583 DEBUG:teuthology.orchestra.run.smithi164:osd.5> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T10:27:09.585 INFO:tasks.cephadm:Deploying osd.6 on smithi043 with /dev/vg_nvme/lv_2... 2024-04-25T10:27:09.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T10:27:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:27:10.231 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:09 smithi043 ceph-mon[29478]: from='osd.5 [v2:172.21.15.164:6808/853209434,v1:172.21.15.164:6809/853209434]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T10:27:10.231 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:09 smithi043 ceph-mon[29478]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T10:27:10.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:09 smithi077 ceph-mon[32772]: from='osd.5 [v2:172.21.15.164:6808/853209434,v1:172.21.15.164:6809/853209434]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T10:27:10.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:09 smithi077 ceph-mon[32772]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T10:27:10.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:09 smithi164 ceph-mon[32713]: from='osd.5 [v2:172.21.15.164:6808/853209434,v1:172.21.15.164:6809/853209434]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T10:27:10.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:09 smithi164 ceph-mon[32713]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T10:27:11.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:10 smithi164 ceph-mon[32713]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T10:27:11.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:10 smithi164 ceph-mon[32713]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T10:27:11.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:10 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:27:11.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:10 smithi164 ceph-mon[32713]: from='osd.5 [v2:172.21.15.164:6808/853209434,v1:172.21.15.164:6809/853209434]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:27:11.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:10 smithi164 ceph-mon[32713]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:27:11.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:10 smithi164 ceph-mon[32713]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:11.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:10 smithi077 ceph-mon[32772]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T10:27:11.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:10 smithi077 ceph-mon[32772]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T10:27:11.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:10 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:27:11.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:10 smithi077 ceph-mon[32772]: from='osd.5 [v2:172.21.15.164:6808/853209434,v1:172.21.15.164:6809/853209434]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:27:11.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:10 smithi077 ceph-mon[32772]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:27:11.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:10 smithi077 ceph-mon[32772]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:11.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:10 smithi043 ceph-mon[29478]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T10:27:11.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:10 smithi043 ceph-mon[29478]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T10:27:11.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:10 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:27:11.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:10 smithi043 ceph-mon[29478]: from='osd.5 [v2:172.21.15.164:6808/853209434,v1:172.21.15.164:6809/853209434]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:27:11.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:10 smithi043 ceph-mon[29478]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:27:11.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:10 smithi043 ceph-mon[29478]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:11.894 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T10:27:11.894 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T10:27:11.894 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-04-25T10:27:11.894 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-04-25T10:27:11.894 INFO:teuthology.orchestra.run.smithi043.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00849264 s, 1.2 GB/s 2024-04-25T10:27:11.894 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-04-25T10:27:12.266 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:11 smithi043 ceph-mon[29478]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T10:27:12.266 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:11 smithi043 ceph-mon[29478]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T10:27:12.266 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:11 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:27:12.266 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:11 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:27:12.266 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:11 smithi043 ceph-mon[29478]: from='osd.5 ' entity='osd.5' 2024-04-25T10:27:12.285 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:11 smithi164 ceph-mon[32713]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T10:27:12.285 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:11 smithi164 ceph-mon[32713]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T10:27:12.285 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:11 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:27:12.285 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:11 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:27:12.286 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:11 smithi164 ceph-mon[32713]: from='osd.5 ' entity='osd.5' 2024-04-25T10:27:12.286 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:27:11 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:27:11.909+0000 7ff9a4517700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T10:27:12.286 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:27:11 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:27:11.949+0000 7ff99c2c4700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:27:12.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:11 smithi077 ceph-mon[32772]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T10:27:12.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:11 smithi077 ceph-mon[32772]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T10:27:12.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:11 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:27:12.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:11 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:27:12.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:11 smithi077 ceph-mon[32772]: from='osd.5 ' entity='osd.5' 2024-04-25T10:27:12.565 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_2 2024-04-25T10:27:12.839 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:27:13.126 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:12 smithi043 ceph-mon[29478]: purged_snaps scrub starts 2024-04-25T10:27:13.126 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:12 smithi043 ceph-mon[29478]: purged_snaps scrub ok 2024-04-25T10:27:13.126 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:12 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:27:13.126 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:12 smithi043 ceph-mon[29478]: osd.5 [v2:172.21.15.164:6808/853209434,v1:172.21.15.164:6809/853209434] boot 2024-04-25T10:27:13.126 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:12 smithi043 ceph-mon[29478]: osdmap e36: 6 total, 6 up, 6 in 2024-04-25T10:27:13.126 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:12 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:27:13.126 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:12 smithi043 ceph-mon[29478]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:13.127 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:12 smithi043 ceph-mon[29478]: Detected new or changed devices on smithi164 2024-04-25T10:27:13.127 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:12 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:13.127 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:12 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:13.127 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:12 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:27:13.127 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:12 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:27:13.127 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:12 smithi043 ceph-mon[29478]: Adjusting osd_memory_target on smithi164 to 8544M 2024-04-25T10:27:13.127 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:12 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:13.127 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:12 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:13.127 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:12 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:27:13.127 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:12 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:13.127 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:12 smithi043 ceph-mon[29478]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T10:27:13.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:12 smithi077 ceph-mon[32772]: purged_snaps scrub starts 2024-04-25T10:27:13.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:12 smithi077 ceph-mon[32772]: purged_snaps scrub ok 2024-04-25T10:27:13.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:12 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:27:13.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:12 smithi077 ceph-mon[32772]: osd.5 [v2:172.21.15.164:6808/853209434,v1:172.21.15.164:6809/853209434] boot 2024-04-25T10:27:13.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:12 smithi077 ceph-mon[32772]: osdmap e36: 6 total, 6 up, 6 in 2024-04-25T10:27:13.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:12 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:27:13.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:12 smithi077 ceph-mon[32772]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:13.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:12 smithi077 ceph-mon[32772]: Detected new or changed devices on smithi164 2024-04-25T10:27:13.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:12 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:13.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:12 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:13.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:12 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:27:13.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:12 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:27:13.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:12 smithi077 ceph-mon[32772]: Adjusting osd_memory_target on smithi164 to 8544M 2024-04-25T10:27:13.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:12 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:13.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:12 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:13.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:12 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:27:13.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:12 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:13.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:12 smithi077 ceph-mon[32772]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T10:27:13.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:12 smithi164 ceph-mon[32713]: purged_snaps scrub starts 2024-04-25T10:27:13.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:12 smithi164 ceph-mon[32713]: purged_snaps scrub ok 2024-04-25T10:27:13.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:12 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:27:13.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:12 smithi164 ceph-mon[32713]: osd.5 [v2:172.21.15.164:6808/853209434,v1:172.21.15.164:6809/853209434] boot 2024-04-25T10:27:13.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:12 smithi164 ceph-mon[32713]: osdmap e36: 6 total, 6 up, 6 in 2024-04-25T10:27:13.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:12 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:27:13.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:12 smithi164 ceph-mon[32713]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-25T10:27:13.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:12 smithi164 ceph-mon[32713]: Detected new or changed devices on smithi164 2024-04-25T10:27:13.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:12 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:13.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:12 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:13.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:12 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:27:13.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:12 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:27:13.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:12 smithi164 ceph-mon[32713]: Adjusting osd_memory_target on smithi164 to 8544M 2024-04-25T10:27:13.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:12 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:13.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:12 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:13.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:12 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:27:13.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:12 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:13.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:12 smithi164 ceph-mon[32713]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T10:27:14.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.584+0000 7f4805297700 1 -- 172.21.15.43:0/1757059298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4800105340 msgr2=0x7f4800105750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.584+0000 7f4805297700 1 --2- 172.21.15.43:0/1757059298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4800105340 0x7f4800105750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009990 tx=0x7f47f4009ca0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.584+0000 7f4805297700 1 -- 172.21.15.43:0/1757059298 shutdown_connections 2024-04-25T10:27:14.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.584+0000 7f4805297700 1 --2- 172.21.15.43:0/1757059298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4800105ce0 0x7f480010c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.584+0000 7f4805297700 1 --2- 172.21.15.43:0/1757059298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4800104210 0x7f4800104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.584+0000 7f4805297700 1 --2- 172.21.15.43:0/1757059298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4800105340 0x7f4800105750 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.584+0000 7f4805297700 1 -- 172.21.15.43:0/1757059298 >> 172.21.15.43:0/1757059298 conn(0x7f48000ff870 msgr2=0x7f4800101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:14.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.585+0000 7f4805297700 1 -- 172.21.15.43:0/1757059298 shutdown_connections 2024-04-25T10:27:14.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.585+0000 7f4805297700 1 -- 172.21.15.43:0/1757059298 wait complete. 2024-04-25T10:27:14.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.585+0000 7f4805297700 1 Processor -- start 2024-04-25T10:27:14.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.585+0000 7f4805297700 1 -- start start 2024-04-25T10:27:14.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.586+0000 7f4805297700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4800104210 0x7f480019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.586+0000 7f4805297700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4800105340 0x7f480019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.586+0000 7f4805297700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4800105ce0 0x7f48001a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.586+0000 7f4805297700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f480010fc70 con 0x7f4800105340 2024-04-25T10:27:14.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.586+0000 7f4805297700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f480010faf0 con 0x7f4800105ce0 2024-04-25T10:27:14.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.586+0000 7f4805297700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f480010fdf0 con 0x7f4800104210 2024-04-25T10:27:14.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.586+0000 7f47fe7fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4800105340 0x7f480019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.586+0000 7f47fe7fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4800105340 0x7f480019acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59776/0 (socket says 172.21.15.43:59776) 2024-04-25T10:27:14.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.586+0000 7f47fe7fc700 1 -- 172.21.15.43:0/3585031445 learned_addr learned my addr 172.21.15.43:0/3585031445 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:27:14.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.586+0000 7f47feffd700 1 --2- 172.21.15.43:0/3585031445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4800104210 0x7f480019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.586+0000 7f47ff7fe700 1 --2- 172.21.15.43:0/3585031445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4800105ce0 0x7f48001a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.586+0000 7f47fe7fc700 1 -- 172.21.15.43:0/3585031445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4800104210 msgr2=0x7f480019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.586+0000 7f47fe7fc700 1 --2- 172.21.15.43:0/3585031445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4800104210 0x7f480019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.586+0000 7f47fe7fc700 1 -- 172.21.15.43:0/3585031445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4800105ce0 msgr2=0x7f48001a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.586+0000 7f47fe7fc700 1 --2- 172.21.15.43:0/3585031445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4800105ce0 0x7f48001a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.586+0000 7f47fe7fc700 1 -- 172.21.15.43:0/3585031445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48001a22e0 con 0x7f4800105340 2024-04-25T10:27:14.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.586+0000 7f47fe7fc700 1 --2- 172.21.15.43:0/3585031445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4800105340 0x7f480019acc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f47f400b760 tx=0x7f47f4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:14.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.586+0000 7f47e7fff700 1 -- 172.21.15.43:0/3585031445 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4026070 con 0x7f4800105340 2024-04-25T10:27:14.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.587+0000 7f4805297700 1 -- 172.21.15.43:0/3585031445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48001a2570 con 0x7f4800105340 2024-04-25T10:27:14.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.587+0000 7f47e7fff700 1 -- 172.21.15.43:0/3585031445 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47f4021410 con 0x7f4800105340 2024-04-25T10:27:14.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.587+0000 7f47e7fff700 1 -- 172.21.15.43:0/3585031445 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4017650 con 0x7f4800105340 2024-04-25T10:27:14.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.587+0000 7f4805297700 1 -- 172.21.15.43:0/3585031445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48001a29c0 con 0x7f4800105340 2024-04-25T10:27:14.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.587+0000 7f4805297700 1 -- 172.21.15.43:0/3585031445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4800051190 con 0x7f4800105340 2024-04-25T10:27:14.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.589+0000 7f47e7fff700 1 -- 172.21.15.43:0/3585031445 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f47f4029070 con 0x7f4800105340 2024-04-25T10:27:14.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.590+0000 7f47e7fff700 1 --2- 172.21.15.43:0/3585031445 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f47e80afc00 0x7f47e80b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.590+0000 7f47feffd700 1 --2- 172.21.15.43:0/3585031445 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f47e80afc00 0x7f47e80b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.590+0000 7f47e7fff700 1 -- 172.21.15.43:0/3585031445 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7f47f40df2e0 con 0x7f4800105340 2024-04-25T10:27:14.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.590+0000 7f47feffd700 1 --2- 172.21.15.43:0/3585031445 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f47e80afc00 0x7f47e80b2110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4800105240 tx=0x7f47f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.594+0000 7f47e7fff700 1 -- 172.21.15.43:0/3585031445 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47f4030020 con 0x7f4800105340 2024-04-25T10:27:14.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:14.814+0000 7f4805297700 1 -- 172.21.15.43:0/3585031445 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4800063b90 con 0x7f47e80afc00 2024-04-25T10:27:15.010 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:14 smithi164 ceph-mon[32713]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T10:27:15.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:14 smithi077 ceph-mon[32772]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T10:27:15.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:14 smithi043 ceph-mon[29478]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T10:27:16.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:15 smithi043 ceph-mon[29478]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:16.035 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:15 smithi043 ceph-mon[29478]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:27:16.035 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:15 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:27:16.035 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:15 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:27:16.035 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:15 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:16.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:15 smithi077 ceph-mon[32772]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:16.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:15 smithi077 ceph-mon[32772]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:27:16.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:15 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:27:16.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:15 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:27:16.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:15 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:16.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:15 smithi164 ceph-mon[32713]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:16.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:15 smithi164 ceph-mon[32713]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:27:16.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:15 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:27:16.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:15 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:27:16.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:15 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:18.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:17 smithi077 ceph-mon[32772]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-25T10:27:18.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:17 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3924365935' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e8aba2c-984b-4570-8900-dd88f3e5774a"}]: dispatch 2024-04-25T10:27:18.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:17 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3924365935' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e8aba2c-984b-4570-8900-dd88f3e5774a"}]': finished 2024-04-25T10:27:18.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:17 smithi077 ceph-mon[32772]: osdmap e39: 7 total, 6 up, 7 in 2024-04-25T10:27:18.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:17 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:27:18.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:17 smithi164 ceph-mon[32713]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-25T10:27:18.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:17 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3924365935' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e8aba2c-984b-4570-8900-dd88f3e5774a"}]: dispatch 2024-04-25T10:27:18.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:17 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3924365935' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e8aba2c-984b-4570-8900-dd88f3e5774a"}]': finished 2024-04-25T10:27:18.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:17 smithi164 ceph-mon[32713]: osdmap e39: 7 total, 6 up, 7 in 2024-04-25T10:27:18.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:27:18.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:17 smithi043 ceph-mon[29478]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-25T10:27:18.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:17 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3924365935' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e8aba2c-984b-4570-8900-dd88f3e5774a"}]: dispatch 2024-04-25T10:27:18.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:17 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3924365935' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e8aba2c-984b-4570-8900-dd88f3e5774a"}]': finished 2024-04-25T10:27:18.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:17 smithi043 ceph-mon[29478]: osdmap e39: 7 total, 6 up, 7 in 2024-04-25T10:27:18.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:17 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:27:19.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:18 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2609652885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:27:19.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:18 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2609652885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:27:19.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:18 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2609652885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:27:20.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:19 smithi077 ceph-mon[32772]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T10:27:20.127 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:19 smithi043 ceph-mon[29478]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T10:27:20.127 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:19 smithi164 ceph-mon[32713]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T10:27:21.867 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:21 smithi043 ceph-mon[29478]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-25T10:27:22.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:21 smithi077 ceph-mon[32772]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-25T10:27:22.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:21 smithi164 ceph-mon[32713]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-25T10:27:23.882 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:23 smithi043 ceph-mon[29478]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-25T10:27:24.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:23 smithi077 ceph-mon[32772]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-25T10:27:24.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:23 smithi164 ceph-mon[32713]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-25T10:27:26.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:25 smithi077 ceph-mon[32772]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:27:26.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:25 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T10:27:26.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:25 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:26.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:25 smithi077 ceph-mon[32772]: Deploying daemon osd.6 on smithi043 2024-04-25T10:27:26.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:25 smithi164 ceph-mon[32713]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:27:26.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:25 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T10:27:26.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:25 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:26.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:25 smithi164 ceph-mon[32713]: Deploying daemon osd.6 on smithi043 2024-04-25T10:27:26.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:25 smithi043 ceph-mon[29478]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:27:26.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:25 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T10:27:26.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:25 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:26.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:25 smithi043 ceph-mon[29478]: Deploying daemon osd.6 on smithi043 2024-04-25T10:27:27.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:27 smithi043 ceph-mon[29478]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:28.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:27 smithi077 ceph-mon[32772]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:28.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:27 smithi164 ceph-mon[32713]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:30.005 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:29 smithi043 ceph-mon[29478]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:30.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:29 smithi164 ceph-mon[32713]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:30.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:29 smithi077 ceph-mon[32772]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:31.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:31 smithi043 ceph-mon[29478]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:31.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:31 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:31.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:31 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:31.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:31 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:27:32.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:31 smithi077 ceph-mon[32772]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:32.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:31 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:32.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:31 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:32.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:31 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:27:32.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:31 smithi164 ceph-mon[32713]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:32.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:31 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:32.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:31 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:32.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:31 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:27:33.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:33 smithi043 ceph-mon[29478]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:34.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:33 smithi077 ceph-mon[32772]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:34.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:33 smithi164 ceph-mon[32713]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:34.186 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 6 on host 'smithi043' 2024-04-25T10:27:34.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:34.184+0000 7f47e7fff700 1 -- 172.21.15.43:0/3585031445 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4800063b90 con 0x7f47e80afc00 2024-04-25T10:27:34.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:34.186+0000 7f4805297700 1 -- 172.21.15.43:0/3585031445 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f47e80afc00 msgr2=0x7f47e80b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:34.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:34.186+0000 7f4805297700 1 --2- 172.21.15.43:0/3585031445 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f47e80afc00 0x7f47e80b2110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4800105240 tx=0x7f47f0009260 comp rx=0 tx=0).stop 2024-04-25T10:27:34.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:34.186+0000 7f4805297700 1 -- 172.21.15.43:0/3585031445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4800105340 msgr2=0x7f480019acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:34.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:34.186+0000 7f4805297700 1 --2- 172.21.15.43:0/3585031445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4800105340 0x7f480019acc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f47f400b760 tx=0x7f47f4009580 comp rx=0 tx=0).stop 2024-04-25T10:27:34.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:34.186+0000 7f4805297700 1 -- 172.21.15.43:0/3585031445 shutdown_connections 2024-04-25T10:27:34.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:34.186+0000 7f4805297700 1 --2- 172.21.15.43:0/3585031445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4800105ce0 0x7f48001a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:34.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:34.186+0000 7f4805297700 1 --2- 172.21.15.43:0/3585031445 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f47e80afc00 0x7f47e80b2110 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:34.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:34.186+0000 7f4805297700 1 --2- 172.21.15.43:0/3585031445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4800104210 0x7f480019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:34.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:34.186+0000 7f4805297700 1 --2- 172.21.15.43:0/3585031445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4800105340 0x7f480019acc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:34.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:34.186+0000 7f4805297700 1 -- 172.21.15.43:0/3585031445 >> 172.21.15.43:0/3585031445 conn(0x7f48000ff870 msgr2=0x7f4800108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:34.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:34.186+0000 7f4805297700 1 -- 172.21.15.43:0/3585031445 shutdown_connections 2024-04-25T10:27:34.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:27:34.186+0000 7f4805297700 1 -- 172.21.15.43:0/3585031445 wait complete. 2024-04-25T10:27:34.825 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:34.825 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:34.826 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:34.826 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:27:34.826 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:34.826 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:34.826 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:34.826 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:27:34.826 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:34 smithi043 ceph-mon[29478]: from='osd.6 [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T10:27:34.853 DEBUG:teuthology.orchestra.run.smithi043:osd.6> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T10:27:34.855 INFO:tasks.cephadm:Deploying osd.7 on smithi077 with /dev/vg_nvme/lv_2... 2024-04-25T10:27:34.855 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T10:27:35.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:35.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:35.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:35.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:27:35.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:35.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:35.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:35.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:27:35.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:34 smithi077 ceph-mon[32772]: from='osd.6 [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T10:27:35.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:35.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:35.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:35.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:27:35.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:35.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:35.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:35.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:27:35.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:34 smithi164 ceph-mon[32713]: from='osd.6 [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T10:27:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T10:27:35.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:35 smithi043 ceph-mon[29478]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:35.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:35 smithi043 ceph-mon[29478]: from='osd.6 [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T10:27:35.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:35 smithi043 ceph-mon[29478]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T10:27:35.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:35 smithi043 ceph-mon[29478]: from='osd.6 [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-25T10:27:35.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:35 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:27:35.906 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:35 smithi077 ceph-mon[32772]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:35.906 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:35 smithi077 ceph-mon[32772]: from='osd.6 [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T10:27:35.906 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:35 smithi077 ceph-mon[32772]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T10:27:35.906 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:35 smithi077 ceph-mon[32772]: from='osd.6 [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-25T10:27:35.906 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:35 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:27:36.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:35 smithi164 ceph-mon[32713]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:36.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:35 smithi164 ceph-mon[32713]: from='osd.6 [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T10:27:36.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:35 smithi164 ceph-mon[32713]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T10:27:36.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:35 smithi164 ceph-mon[32713]: from='osd.6 [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-25T10:27:36.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:35 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:27:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T10:27:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T10:27:37.157 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-04-25T10:27:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-04-25T10:27:37.157 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011477 s, 914 MB/s 2024-04-25T10:27:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-04-25T10:27:37.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:37 smithi043 ceph-mon[29478]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:37.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:37 smithi043 ceph-mon[29478]: from='osd.6 [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-25T10:27:37.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:37 smithi043 ceph-mon[29478]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T10:27:37.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:37 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:27:37.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:37 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:27:37.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:37 smithi043 ceph-mon[29478]: from='osd.6 [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172]' entity='osd.6' 2024-04-25T10:27:37.361 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:27:36 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:27:36.971+0000 7f081d1b1700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T10:27:37.361 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:27:36 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:27:36.984+0000 7f0814f5e700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:27:37.451 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:37 smithi077 ceph-mon[32772]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:37.451 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:37 smithi077 ceph-mon[32772]: from='osd.6 [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-25T10:27:37.451 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:37 smithi077 ceph-mon[32772]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T10:27:37.451 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:37 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:27:37.451 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:37 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:27:37.451 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:37 smithi077 ceph-mon[32772]: from='osd.6 [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172]' entity='osd.6' 2024-04-25T10:27:37.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:37 smithi164 ceph-mon[32713]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:37.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:37 smithi164 ceph-mon[32713]: from='osd.6 [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-25T10:27:37.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:37 smithi164 ceph-mon[32713]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T10:27:37.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:37 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:27:37.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:37 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:27:37.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:37 smithi164 ceph-mon[32713]: from='osd.6 [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172]' entity='osd.6' 2024-04-25T10:27:37.830 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_2 2024-04-25T10:27:38.080 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T10:27:38.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:38 smithi043 ceph-mon[29478]: purged_snaps scrub starts 2024-04-25T10:27:38.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:38 smithi043 ceph-mon[29478]: purged_snaps scrub ok 2024-04-25T10:27:38.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:38 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:27:38.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:38 smithi043 ceph-mon[29478]: Detected new or changed devices on smithi043 2024-04-25T10:27:38.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:38 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:38.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:38 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:38.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:38 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:27:38.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:38 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:27:38.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:38 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:27:38.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:38 smithi043 ceph-mon[29478]: Adjusting osd_memory_target on smithi043 to 5697M 2024-04-25T10:27:38.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:38 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:38.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:38 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:38.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:38 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:27:38.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:38 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:38.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:38 smithi043 ceph-mon[29478]: osd.6 [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172] boot 2024-04-25T10:27:38.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:38 smithi043 ceph-mon[29478]: osdmap e42: 7 total, 7 up, 7 in 2024-04-25T10:27:38.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:38 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:27:38.421 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:38 smithi077 ceph-mon[32772]: purged_snaps scrub starts 2024-04-25T10:27:38.421 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:38 smithi077 ceph-mon[32772]: purged_snaps scrub ok 2024-04-25T10:27:38.421 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:38 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:27:38.422 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:38 smithi077 ceph-mon[32772]: Detected new or changed devices on smithi043 2024-04-25T10:27:38.422 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:38 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:38.422 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:38 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:38.422 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:38 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:27:38.422 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:38 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:27:38.422 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:38 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:27:38.422 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:38 smithi077 ceph-mon[32772]: Adjusting osd_memory_target on smithi043 to 5697M 2024-04-25T10:27:38.422 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:38 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:38.422 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:38 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:38.422 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:38 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:27:38.422 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:38 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:38.422 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:38 smithi077 ceph-mon[32772]: osd.6 [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172] boot 2024-04-25T10:27:38.422 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:38 smithi077 ceph-mon[32772]: osdmap e42: 7 total, 7 up, 7 in 2024-04-25T10:27:38.423 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:38 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:27:38.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:38 smithi164 ceph-mon[32713]: purged_snaps scrub starts 2024-04-25T10:27:38.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:38 smithi164 ceph-mon[32713]: purged_snaps scrub ok 2024-04-25T10:27:38.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:38 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:27:38.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:38 smithi164 ceph-mon[32713]: Detected new or changed devices on smithi043 2024-04-25T10:27:38.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:38 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:38.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:38 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:38.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:38 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:27:38.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:38 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:27:38.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:38 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:27:38.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:38 smithi164 ceph-mon[32713]: Adjusting osd_memory_target on smithi043 to 5697M 2024-04-25T10:27:38.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:38 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:38.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:38 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:38.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:38 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:27:38.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:38 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:38.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:38 smithi164 ceph-mon[32713]: osd.6 [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172] boot 2024-04-25T10:27:38.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:38 smithi164 ceph-mon[32713]: osdmap e42: 7 total, 7 up, 7 in 2024-04-25T10:27:38.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:38 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:27:39.456 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:39 smithi077 ceph-mon[32772]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:39.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:39 smithi164 ceph-mon[32713]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:39.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:39 smithi043 ceph-mon[29478]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-25T10:27:39.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.662+0000 7feaeb614700 1 -- 172.21.15.77:0/3079097844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae4104bb0 msgr2=0x7feae410b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:39.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.662+0000 7feaeb614700 1 --2- 172.21.15.77:0/3079097844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae4104bb0 0x7feae410b640 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feadc00b0a0 tx=0x7feadc00b3b0 comp rx=0 tx=0).stop 2024-04-25T10:27:39.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.663+0000 7feaeb614700 1 -- 172.21.15.77:0/3079097844 shutdown_connections 2024-04-25T10:27:39.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.663+0000 7feaeb614700 1 --2- 172.21.15.77:0/3079097844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae4104bb0 0x7feae410b640 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:39.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.663+0000 7feaeb614700 1 --2- 172.21.15.77:0/3079097844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae4103670 0x7feae4103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:39.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.663+0000 7feaeb614700 1 --2- 172.21.15.77:0/3079097844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feae4104050 0x7feae4104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:39.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.663+0000 7feaeb614700 1 -- 172.21.15.77:0/3079097844 >> 172.21.15.77:0/3079097844 conn(0x7feae40fdee0 msgr2=0x7feae4100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:39.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.663+0000 7feaeb614700 1 -- 172.21.15.77:0/3079097844 shutdown_connections 2024-04-25T10:27:39.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.663+0000 7feaeb614700 1 -- 172.21.15.77:0/3079097844 wait complete. 2024-04-25T10:27:39.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.663+0000 7feaeb614700 1 Processor -- start 2024-04-25T10:27:39.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.663+0000 7feaeb614700 1 -- start start 2024-04-25T10:27:39.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.664+0000 7feaeb614700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feae4103670 0x7feae4199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:39.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.664+0000 7feaeb614700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae4104050 0x7feae4199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:39.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.664+0000 7feaeb614700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae4104bb0 0x7feae41a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:39.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.664+0000 7feaeb614700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feae410ebe0 con 0x7feae4103670 2024-04-25T10:27:39.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.664+0000 7feaeb614700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feae410ea60 con 0x7feae4104bb0 2024-04-25T10:27:39.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.664+0000 7feaeb614700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feae410ed60 con 0x7feae4104050 2024-04-25T10:27:39.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.664+0000 7feae9bb1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae4104bb0 0x7feae41a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:39.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.664+0000 7feae9bb1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae4104bb0 0x7feae41a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47138/0 (socket says 172.21.15.77:47138) 2024-04-25T10:27:39.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.664+0000 7feae93b0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feae4103670 0x7feae4199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:39.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.664+0000 7feae9bb1700 1 -- 172.21.15.77:0/2617430799 learned_addr learned my addr 172.21.15.77:0/2617430799 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-25T10:27:39.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.664+0000 7feae8baf700 1 --2- 172.21.15.77:0/2617430799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae4104050 0x7feae4199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:39.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.664+0000 7feae9bb1700 1 -- 172.21.15.77:0/2617430799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae4104050 msgr2=0x7feae4199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:39.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.664+0000 7feae9bb1700 1 --2- 172.21.15.77:0/2617430799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae4104050 0x7feae4199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:39.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.664+0000 7feae9bb1700 1 -- 172.21.15.77:0/2617430799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feae4103670 msgr2=0x7feae4199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:39.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.664+0000 7feae9bb1700 1 --2- 172.21.15.77:0/2617430799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feae4103670 0x7feae4199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:39.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.664+0000 7feae9bb1700 1 -- 172.21.15.77:0/2617430799 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae41a11c0 con 0x7feae4104bb0 2024-04-25T10:27:39.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.664+0000 7feae9bb1700 1 --2- 172.21.15.77:0/2617430799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae4104bb0 0x7feae41a0b40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feadc0062a0 tx=0x7feadc004e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:39.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.665+0000 7feada7fc700 1 -- 172.21.15.77:0/2617430799 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc020070 con 0x7feae4104bb0 2024-04-25T10:27:39.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.665+0000 7feada7fc700 1 -- 172.21.15.77:0/2617430799 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feadc007b80 con 0x7feae4104bb0 2024-04-25T10:27:39.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.665+0000 7feaeb614700 1 -- 172.21.15.77:0/2617430799 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae41022d0 con 0x7feae4104bb0 2024-04-25T10:27:39.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.665+0000 7feada7fc700 1 -- 172.21.15.77:0/2617430799 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc004670 con 0x7feae4104bb0 2024-04-25T10:27:39.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.665+0000 7feaeb614700 1 -- 172.21.15.77:0/2617430799 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae41028e0 con 0x7feae4104bb0 2024-04-25T10:27:39.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.666+0000 7feaeb614700 1 -- 172.21.15.77:0/2617430799 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feae4051190 con 0x7feae4104bb0 2024-04-25T10:27:39.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.667+0000 7feada7fc700 1 -- 172.21.15.77:0/2617430799 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7feadc003a00 con 0x7feae4104bb0 2024-04-25T10:27:39.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.668+0000 7feada7fc700 1 --2- 172.21.15.77:0/2617430799 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fead00afc00 0x7fead00b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:39.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.668+0000 7feada7fc700 1 -- 172.21.15.77:0/2617430799 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7feadc00f030 con 0x7feae4104bb0 2024-04-25T10:27:39.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.668+0000 7feae93b0700 1 --2- 172.21.15.77:0/2617430799 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fead00afc00 0x7fead00b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:39.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.668+0000 7feae93b0700 1 --2- 172.21.15.77:0/2617430799 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fead00afc00 0x7fead00b2110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feae406b220 tx=0x7feae0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:39.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.671+0000 7feada7fc700 1 -- 172.21.15.77:0/2617430799 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feadc0aa6e0 con 0x7feae4104bb0 2024-04-25T10:27:39.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:39.877+0000 7feaeb614700 1 -- 172.21.15.77:0/2617430799 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7feae4102b40 con 0x7fead00afc00 2024-04-25T10:27:40.495 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:40 smithi077 ceph-mon[32772]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T10:27:40.495 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:40 smithi077 ceph-mon[32772]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:27:40.496 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:40 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:27:40.496 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:40 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:27:40.496 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:40 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:40.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:40 smithi164 ceph-mon[32713]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T10:27:40.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:40 smithi164 ceph-mon[32713]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:27:40.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:40 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:27:40.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:40 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:27:40.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:40 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:40.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:40 smithi043 ceph-mon[29478]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T10:27:40.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:40 smithi043 ceph-mon[29478]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:27:40.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:40 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:27:40.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:40 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:27:40.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:40 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:41.498 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:41 smithi077 ceph-mon[32772]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:41.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:41 smithi164 ceph-mon[32713]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:41.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:41 smithi043 ceph-mon[29478]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:43.201 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:43 smithi077 ceph-mon[32772]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:43.201 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:43 smithi077 ceph-mon[32772]: from='client.? 172.21.15.77:0/2746968956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0d47c22-d7bb-4566-802b-bca38d24b04e"}]: dispatch 2024-04-25T10:27:43.201 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:43 smithi077 ceph-mon[32772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0d47c22-d7bb-4566-802b-bca38d24b04e"}]: dispatch 2024-04-25T10:27:43.201 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:43 smithi077 ceph-mon[32772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0d47c22-d7bb-4566-802b-bca38d24b04e"}]': finished 2024-04-25T10:27:43.201 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:43 smithi077 ceph-mon[32772]: osdmap e44: 8 total, 7 up, 8 in 2024-04-25T10:27:43.201 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:43 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:27:43.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:43 smithi164 ceph-mon[32713]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:43.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:43 smithi164 ceph-mon[32713]: from='client.? 172.21.15.77:0/2746968956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0d47c22-d7bb-4566-802b-bca38d24b04e"}]: dispatch 2024-04-25T10:27:43.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:43 smithi164 ceph-mon[32713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0d47c22-d7bb-4566-802b-bca38d24b04e"}]: dispatch 2024-04-25T10:27:43.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:43 smithi164 ceph-mon[32713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0d47c22-d7bb-4566-802b-bca38d24b04e"}]': finished 2024-04-25T10:27:43.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:43 smithi164 ceph-mon[32713]: osdmap e44: 8 total, 7 up, 8 in 2024-04-25T10:27:43.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:43 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:27:43.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:43 smithi043 ceph-mon[29478]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:43.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:43 smithi043 ceph-mon[29478]: from='client.? 172.21.15.77:0/2746968956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0d47c22-d7bb-4566-802b-bca38d24b04e"}]: dispatch 2024-04-25T10:27:43.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:43 smithi043 ceph-mon[29478]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0d47c22-d7bb-4566-802b-bca38d24b04e"}]: dispatch 2024-04-25T10:27:43.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:43 smithi043 ceph-mon[29478]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0d47c22-d7bb-4566-802b-bca38d24b04e"}]': finished 2024-04-25T10:27:43.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:43 smithi043 ceph-mon[29478]: osdmap e44: 8 total, 7 up, 8 in 2024-04-25T10:27:43.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:43 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:27:44.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:44 smithi077 ceph-mon[32772]: from='client.? 172.21.15.77:0/815657599' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:27:44.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:44 smithi164 ceph-mon[32713]: from='client.? 172.21.15.77:0/815657599' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:27:44.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:44 smithi043 ceph-mon[29478]: from='client.? 172.21.15.77:0/815657599' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:27:45.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:45 smithi077 ceph-mon[32772]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:45.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:45 smithi164 ceph-mon[32713]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:45.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:45 smithi043 ceph-mon[29478]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:47.302 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:47 smithi077 ceph-mon[32772]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:47.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:47 smithi164 ceph-mon[32713]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:47.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:47 smithi043 ceph-mon[29478]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:49.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:49 smithi077 ceph-mon[32772]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:49.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:49 smithi164 ceph-mon[32713]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:49.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:49 smithi043 ceph-mon[29478]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:51.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:51 smithi077 ceph-mon[32772]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:51.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:51 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T10:27:51.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:51 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:51.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:51 smithi077 ceph-mon[32772]: Deploying daemon osd.7 on smithi077 2024-04-25T10:27:51.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:51 smithi164 ceph-mon[32713]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:51.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:51 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T10:27:51.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:51 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:51.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:51 smithi164 ceph-mon[32713]: Deploying daemon osd.7 on smithi077 2024-04-25T10:27:51.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:51 smithi043 ceph-mon[29478]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:51.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:51 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T10:27:51.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:51 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:27:51.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:51 smithi043 ceph-mon[29478]: Deploying daemon osd.7 on smithi077 2024-04-25T10:27:53.521 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:53 smithi077 ceph-mon[32772]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:53.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:53 smithi164 ceph-mon[32713]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:53.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:53 smithi043 ceph-mon[29478]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:55.448 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:55 smithi077 ceph-mon[32772]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:55.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:55 smithi164 ceph-mon[32713]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:55.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:55 smithi043 ceph-mon[29478]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:56.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:56 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:56.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:56 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:56.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:56 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:27:57.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:56 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:57.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:56 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:57.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:56 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:27:57.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:56 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:57.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:56 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:57.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:56 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:27:57.780 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:57 smithi077 ceph-mon[32772]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:58.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:57 smithi164 ceph-mon[32713]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:58.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:57 smithi043 ceph-mon[29478]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:59.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:59.150+0000 7feada7fc700 1 -- 172.21.15.77:0/2617430799 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7feae4102b40 con 0x7fead00afc00 2024-04-25T10:27:59.153 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 7 on host 'smithi077' 2024-04-25T10:27:59.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:59.153+0000 7feaeb614700 1 -- 172.21.15.77:0/2617430799 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fead00afc00 msgr2=0x7fead00b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:59.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:59.153+0000 7feaeb614700 1 --2- 172.21.15.77:0/2617430799 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fead00afc00 0x7fead00b2110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feae406b220 tx=0x7feae0009190 comp rx=0 tx=0).stop 2024-04-25T10:27:59.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:59.153+0000 7feaeb614700 1 -- 172.21.15.77:0/2617430799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae4104bb0 msgr2=0x7feae41a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:59.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:59.153+0000 7feaeb614700 1 --2- 172.21.15.77:0/2617430799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae4104bb0 0x7feae41a0b40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feadc0062a0 tx=0x7feadc004e40 comp rx=0 tx=0).stop 2024-04-25T10:27:59.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:59.153+0000 7feaeb614700 1 -- 172.21.15.77:0/2617430799 shutdown_connections 2024-04-25T10:27:59.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:59.153+0000 7feaeb614700 1 --2- 172.21.15.77:0/2617430799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae4104bb0 0x7feae41a0b40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:59.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:59.153+0000 7feaeb614700 1 --2- 172.21.15.77:0/2617430799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feae4103670 0x7feae4199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:59.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:59.153+0000 7feaeb614700 1 --2- 172.21.15.77:0/2617430799 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fead00afc00 0x7fead00b2110 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:59.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:59.153+0000 7feaeb614700 1 --2- 172.21.15.77:0/2617430799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae4104050 0x7feae4199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:59.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:59.153+0000 7feaeb614700 1 -- 172.21.15.77:0/2617430799 >> 172.21.15.77:0/2617430799 conn(0x7feae40fdee0 msgr2=0x7feae4107620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:59.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:59.153+0000 7feaeb614700 1 -- 172.21.15.77:0/2617430799 shutdown_connections 2024-04-25T10:27:59.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:27:59.153+0000 7feaeb614700 1 -- 172.21.15.77:0/2617430799 wait complete. 2024-04-25T10:27:59.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:59 smithi077 ceph-mon[32772]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:27:59.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:59 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:59.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:59 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:59.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:59 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:59.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:59 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:27:59.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:59 smithi077 ceph-mon[32772]: from='osd.7 [v2:172.21.15.77:6816/1331976864,v1:172.21.15.77:6817/1331976864]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T10:27:59.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:27:59 smithi077 ceph-mon[32772]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T10:27:59.834 DEBUG:teuthology.orchestra.run.smithi077:osd.7> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T10:27:59.836 INFO:tasks.cephadm:Deploying osd.8 on smithi164 with /dev/vg_nvme/lv_2... 2024-04-25T10:27:59.836 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T10:28:00.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:59 smithi164 ceph-mon[32713]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:28:00.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:59 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:00.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:59 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:00.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:59 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:00.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:59 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:00.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:59 smithi164 ceph-mon[32713]: from='osd.7 [v2:172.21.15.77:6816/1331976864,v1:172.21.15.77:6817/1331976864]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T10:28:00.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:27:59 smithi164 ceph-mon[32713]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T10:28:00.083 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:28:00.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:59 smithi043 ceph-mon[29478]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:28:00.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:59 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:00.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:59 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:00.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:59 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:00.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:59 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:00.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:59 smithi043 ceph-mon[29478]: from='osd.7 [v2:172.21.15.77:6816/1331976864,v1:172.21.15.77:6817/1331976864]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T10:28:00.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:27:59 smithi043 ceph-mon[29478]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T10:28:01.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:01 smithi077 ceph-mon[32772]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:28:01.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:01 smithi077 ceph-mon[32772]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T10:28:01.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:01 smithi077 ceph-mon[32772]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T10:28:01.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:01 smithi077 ceph-mon[32772]: from='osd.7 [v2:172.21.15.77:6816/1331976864,v1:172.21.15.77:6817/1331976864]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:28:01.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:01 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:01.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:01 smithi077 ceph-mon[32772]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:28:01.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:01 smithi164 ceph-mon[32713]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:28:01.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:01 smithi164 ceph-mon[32713]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T10:28:01.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:01 smithi164 ceph-mon[32713]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T10:28:01.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:01 smithi164 ceph-mon[32713]: from='osd.7 [v2:172.21.15.77:6816/1331976864,v1:172.21.15.77:6817/1331976864]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:28:01.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:01 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:01.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:01 smithi164 ceph-mon[32713]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:28:01.826 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T10:28:01.827 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T10:28:01.827 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-04-25T10:28:01.827 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-04-25T10:28:01.827 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0142007 s, 738 MB/s 2024-04-25T10:28:01.827 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-04-25T10:28:01.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:01 smithi043 ceph-mon[29478]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:28:01.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:01 smithi043 ceph-mon[29478]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T10:28:01.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:01 smithi043 ceph-mon[29478]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T10:28:01.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:01 smithi043 ceph-mon[29478]: from='osd.7 [v2:172.21.15.77:6816/1331976864,v1:172.21.15.77:6817/1331976864]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:28:01.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:01 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:01.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:01 smithi043 ceph-mon[29478]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:28:02.450 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_2 2024-04-25T10:28:02.693 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:28:02.705 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:02 smithi077 ceph-mon[32772]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-25T10:28:02.705 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:02 smithi077 ceph-mon[32772]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T10:28:02.705 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:02 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:02.705 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:02 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:02.705 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:02 smithi077 ceph-mon[32772]: from='osd.7 ' entity='osd.7' 2024-04-25T10:28:02.705 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:28:02 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:28:02.428+0000 7f8348cbe700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T10:28:02.705 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:28:02 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:28:02.534+0000 7f8340a6b700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:28:02.733 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:02 smithi164 ceph-mon[32713]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-25T10:28:02.733 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:02 smithi164 ceph-mon[32713]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T10:28:02.733 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:02 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:02.734 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:02 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:02.734 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:02 smithi164 ceph-mon[32713]: from='osd.7 ' entity='osd.7' 2024-04-25T10:28:02.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:02 smithi043 ceph-mon[29478]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-25T10:28:02.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:02 smithi043 ceph-mon[29478]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T10:28:02.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:02 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:02.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:02 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:02.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:02 smithi043 ceph-mon[29478]: from='osd.7 ' entity='osd.7' 2024-04-25T10:28:03.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:03 smithi077 ceph-mon[32772]: purged_snaps scrub starts 2024-04-25T10:28:03.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:03 smithi077 ceph-mon[32772]: purged_snaps scrub ok 2024-04-25T10:28:03.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:03 smithi077 ceph-mon[32772]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:28:03.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:03.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:03 smithi077 ceph-mon[32772]: osdmap e47: 8 total, 7 up, 8 in 2024-04-25T10:28:03.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:03.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:03.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:03.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:03.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:03.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:03.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:03.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:03.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:28:03.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:03.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:28:03.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:03.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:28:03.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:03.792 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:03.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:03 smithi164 ceph-mon[32713]: purged_snaps scrub starts 2024-04-25T10:28:03.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:03 smithi164 ceph-mon[32713]: purged_snaps scrub ok 2024-04-25T10:28:03.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:03 smithi164 ceph-mon[32713]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:28:03.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:03.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:03 smithi164 ceph-mon[32713]: osdmap e47: 8 total, 7 up, 8 in 2024-04-25T10:28:03.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:03.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:03.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:03.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:03.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:03.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:03.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:03.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:03.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:28:03.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:03.794 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:28:03.794 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:03.794 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:28:03.794 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:03.794 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:03.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:03 smithi043 ceph-mon[29478]: purged_snaps scrub starts 2024-04-25T10:28:03.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:03 smithi043 ceph-mon[29478]: purged_snaps scrub ok 2024-04-25T10:28:03.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:03 smithi043 ceph-mon[29478]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:28:03.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:03.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:03 smithi043 ceph-mon[29478]: osdmap e47: 8 total, 7 up, 8 in 2024-04-25T10:28:03.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:03.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:03.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:03.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:03.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:03.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:03.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:03.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:03.862 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:28:03.862 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:03.862 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:28:03.862 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:03.862 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:28:03.862 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:03.862 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:04.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.292+0000 7f8008a1b700 1 -- 172.21.15.164:0/3832069745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8004103e30 msgr2=0x7f8004104240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:04.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.292+0000 7f8008a1b700 1 --2- 172.21.15.164:0/3832069745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8004103e30 0x7f8004104240 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009990 tx=0x7f7ff4009ca0 comp rx=0 tx=0).stop 2024-04-25T10:28:04.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.292+0000 7f8008a1b700 1 -- 172.21.15.164:0/3832069745 shutdown_connections 2024-04-25T10:28:04.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.292+0000 7f8008a1b700 1 --2- 172.21.15.164:0/3832069745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80041048b0 0x7f800410c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:04.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.292+0000 7f8008a1b700 1 --2- 172.21.15.164:0/3832069745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8004105880 0x7f8004105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:04.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.292+0000 7f8008a1b700 1 --2- 172.21.15.164:0/3832069745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8004103e30 0x7f8004104240 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:04.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.292+0000 7f8008a1b700 1 -- 172.21.15.164:0/3832069745 >> 172.21.15.164:0/3832069745 conn(0x7f80040ff8a0 msgr2=0x7f8004101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:04.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.292+0000 7f8008a1b700 1 -- 172.21.15.164:0/3832069745 shutdown_connections 2024-04-25T10:28:04.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.292+0000 7f8008a1b700 1 -- 172.21.15.164:0/3832069745 wait complete. 2024-04-25T10:28:04.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.293+0000 7f8008a1b700 1 Processor -- start 2024-04-25T10:28:04.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.293+0000 7f8008a1b700 1 -- start start 2024-04-25T10:28:04.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.293+0000 7f8008a1b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8004103e30 0x7f800419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:04.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.293+0000 7f8008a1b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80041048b0 0x7f800419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.293+0000 7f8008a1b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8004105880 0x7f80041a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.293+0000 7f8008a1b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f800410fdc0 con 0x7f8004103e30 2024-04-25T10:28:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.293+0000 7f8008a1b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f800410fc40 con 0x7f8004105880 2024-04-25T10:28:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.293+0000 7f8008a1b700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f800410ff40 con 0x7f80041048b0 2024-04-25T10:28:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.293+0000 7f8001d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80041048b0 0x7f800419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.293+0000 7f8001d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80041048b0 0x7f800419ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55954/0 (socket says 172.21.15.164:55954) 2024-04-25T10:28:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.293+0000 7f8001d9b700 1 -- 172.21.15.164:0/2694831417 learned_addr learned my addr 172.21.15.164:0/2694831417 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-25T10:28:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.293+0000 7f800259c700 1 --2- 172.21.15.164:0/2694831417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8004103e30 0x7f800419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.293+0000 7f8002d9d700 1 --2- 172.21.15.164:0/2694831417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8004105880 0x7f80041a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.293+0000 7f8001d9b700 1 -- 172.21.15.164:0/2694831417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8004105880 msgr2=0x7f80041a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.293+0000 7f8001d9b700 1 --2- 172.21.15.164:0/2694831417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8004105880 0x7f80041a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.293+0000 7f8001d9b700 1 -- 172.21.15.164:0/2694831417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8004103e30 msgr2=0x7f800419a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.293+0000 7f8001d9b700 1 --2- 172.21.15.164:0/2694831417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8004103e30 0x7f800419a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.293+0000 7f8001d9b700 1 -- 172.21.15.164:0/2694831417 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80041a23d0 con 0x7f80041048b0 2024-04-25T10:28:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.293+0000 7f8002d9d700 1 --2- 172.21.15.164:0/2694831417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8004105880 0x7f80041a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.293+0000 7f800259c700 1 --2- 172.21.15.164:0/2694831417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8004103e30 0x7f800419a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.294+0000 7f8001d9b700 1 --2- 172.21.15.164:0/2694831417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80041048b0 0x7f800419ae40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4006010 tx=0x7f7ff4005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.294+0000 7f7ff37fe700 1 -- 172.21.15.164:0/2694831417 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4027070 con 0x7f80041048b0 2024-04-25T10:28:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.294+0000 7f7ff37fe700 1 -- 172.21.15.164:0/2694831417 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ff4004100 con 0x7f80041048b0 2024-04-25T10:28:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.294+0000 7f8008a1b700 1 -- 172.21.15.164:0/2694831417 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80041a2660 con 0x7f80041048b0 2024-04-25T10:28:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.294+0000 7f7ff37fe700 1 -- 172.21.15.164:0/2694831417 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4022de0 con 0x7f80041048b0 2024-04-25T10:28:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.294+0000 7f8008a1b700 1 -- 172.21.15.164:0/2694831417 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80041a2ab0 con 0x7f80041048b0 2024-04-25T10:28:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.295+0000 7f8008a1b700 1 -- 172.21.15.164:0/2694831417 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8004051190 con 0x7f80041048b0 2024-04-25T10:28:04.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.297+0000 7f7ff37fe700 1 -- 172.21.15.164:0/2694831417 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f7ff402a070 con 0x7f80041048b0 2024-04-25T10:28:04.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.297+0000 7f7ff37fe700 1 --2- 172.21.15.164:0/2694831417 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f7fec0afb80 0x7f7fec0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:04.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.297+0000 7f7ff37fe700 1 -- 172.21.15.164:0/2694831417 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5430+0+0 (secure 0 0 0) 0x7f7ff40df850 con 0x7f80041048b0 2024-04-25T10:28:04.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.298+0000 7f800259c700 1 --2- 172.21.15.164:0/2694831417 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f7fec0afb80 0x7f7fec0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:04.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.298+0000 7f800259c700 1 --2- 172.21.15.164:0/2694831417 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f7fec0afb80 0x7f7fec0b2090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8004105270 tx=0x7f7ff8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:04.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.301+0000 7f7ff37fe700 1 -- 172.21.15.164:0/2694831417 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ff4031080 con 0x7f80041048b0 2024-04-25T10:28:04.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:04.505+0000 7f8008a1b700 1 -- 172.21.15.164:0/2694831417 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f800402e8b0 con 0x7f7fec0afb80 2024-04-25T10:28:04.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:04 smithi077 ceph-mon[32772]: Detected new or changed devices on smithi077 2024-04-25T10:28:04.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:04 smithi077 ceph-mon[32772]: Adjusting osd_memory_target on smithi077 to 5696M 2024-04-25T10:28:04.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:04 smithi077 ceph-mon[32772]: osd.7 [v2:172.21.15.77:6816/1331976864,v1:172.21.15.77:6817/1331976864] boot 2024-04-25T10:28:04.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:04 smithi077 ceph-mon[32772]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T10:28:04.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:04 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:04.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:04 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:28:04.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:04 smithi164 ceph-mon[32713]: Detected new or changed devices on smithi077 2024-04-25T10:28:04.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:04 smithi164 ceph-mon[32713]: Adjusting osd_memory_target on smithi077 to 5696M 2024-04-25T10:28:04.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:04 smithi164 ceph-mon[32713]: osd.7 [v2:172.21.15.77:6816/1331976864,v1:172.21.15.77:6817/1331976864] boot 2024-04-25T10:28:04.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:04 smithi164 ceph-mon[32713]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T10:28:04.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:04 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:04.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:04 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:28:04.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:04 smithi043 ceph-mon[29478]: Detected new or changed devices on smithi077 2024-04-25T10:28:04.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:04 smithi043 ceph-mon[29478]: Adjusting osd_memory_target on smithi077 to 5696M 2024-04-25T10:28:04.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:04 smithi043 ceph-mon[29478]: osd.7 [v2:172.21.15.77:6816/1331976864,v1:172.21.15.77:6817/1331976864] boot 2024-04-25T10:28:04.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:04 smithi043 ceph-mon[29478]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T10:28:04.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:04 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:28:04.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:04 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:28:05.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:05 smithi077 ceph-mon[32772]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:28:05.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:05 smithi077 ceph-mon[32772]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:28:05.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:05 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:28:05.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:05 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:05.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:05 smithi077 ceph-mon[32772]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T10:28:05.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:05 smithi164 ceph-mon[32713]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:28:05.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:05 smithi164 ceph-mon[32713]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:28:05.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:05 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:28:05.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:05 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:05.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:05 smithi164 ceph-mon[32713]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T10:28:05.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:05 smithi043 ceph-mon[29478]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-25T10:28:05.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:05 smithi043 ceph-mon[29478]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:28:05.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:05 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:28:05.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:05 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:05.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:05 smithi043 ceph-mon[29478]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T10:28:06.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:06 smithi077 ceph-mon[32772]: osdmap e50: 8 total, 8 up, 8 in 2024-04-25T10:28:06.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:06 smithi164 ceph-mon[32713]: osdmap e50: 8 total, 8 up, 8 in 2024-04-25T10:28:06.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:06 smithi043 ceph-mon[29478]: osdmap e50: 8 total, 8 up, 8 in 2024-04-25T10:28:07.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:07 smithi077 ceph-mon[32772]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:28:07.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:07 smithi077 ceph-mon[32772]: from='client.? 172.21.15.164:0/3393434801' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29f691f2-fb63-4c00-9d78-3f326bd72b3a"}]: dispatch 2024-04-25T10:28:07.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:07 smithi077 ceph-mon[32772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29f691f2-fb63-4c00-9d78-3f326bd72b3a"}]: dispatch 2024-04-25T10:28:07.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:07 smithi077 ceph-mon[32772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29f691f2-fb63-4c00-9d78-3f326bd72b3a"}]': finished 2024-04-25T10:28:07.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:07 smithi077 ceph-mon[32772]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T10:28:07.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:07 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:28:07.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:07 smithi164 ceph-mon[32713]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:28:07.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:07 smithi164 ceph-mon[32713]: from='client.? 172.21.15.164:0/3393434801' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29f691f2-fb63-4c00-9d78-3f326bd72b3a"}]: dispatch 2024-04-25T10:28:07.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:07 smithi164 ceph-mon[32713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29f691f2-fb63-4c00-9d78-3f326bd72b3a"}]: dispatch 2024-04-25T10:28:07.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:07 smithi164 ceph-mon[32713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29f691f2-fb63-4c00-9d78-3f326bd72b3a"}]': finished 2024-04-25T10:28:07.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:07 smithi164 ceph-mon[32713]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T10:28:07.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:07 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:28:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:07 smithi043 ceph-mon[29478]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:28:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:07 smithi043 ceph-mon[29478]: from='client.? 172.21.15.164:0/3393434801' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29f691f2-fb63-4c00-9d78-3f326bd72b3a"}]: dispatch 2024-04-25T10:28:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:07 smithi043 ceph-mon[29478]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29f691f2-fb63-4c00-9d78-3f326bd72b3a"}]: dispatch 2024-04-25T10:28:07.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:07 smithi043 ceph-mon[29478]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29f691f2-fb63-4c00-9d78-3f326bd72b3a"}]': finished 2024-04-25T10:28:07.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:07 smithi043 ceph-mon[29478]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T10:28:07.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:07 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:28:08.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:08 smithi043 ceph-mon[29478]: from='client.? 172.21.15.164:0/3891169640' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:28:09.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:08 smithi077 ceph-mon[32772]: from='client.? 172.21.15.164:0/3891169640' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:28:09.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:08 smithi164 ceph-mon[32713]: from='client.? 172.21.15.164:0/3891169640' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:28:09.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:09 smithi043 ceph-mon[29478]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:28:10.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:09 smithi077 ceph-mon[32772]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:28:10.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:09 smithi164 ceph-mon[32713]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:28:11.661 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:11 smithi164 ceph-mon[32713]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-04-25T10:28:11.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:11 smithi043 ceph-mon[29478]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-04-25T10:28:12.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:11 smithi077 ceph-mon[32772]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-04-25T10:28:13.753 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:13 smithi164 ceph-mon[32713]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-04-25T10:28:13.843 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:13 smithi077 ceph-mon[32772]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-04-25T10:28:13.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:13 smithi043 ceph-mon[29478]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-04-25T10:28:15.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:15 smithi164 ceph-mon[32713]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-04-25T10:28:15.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:15 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T10:28:15.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:15 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:15.834 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:15 smithi164 ceph-mon[32713]: Deploying daemon osd.8 on smithi164 2024-04-25T10:28:15.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:15 smithi043 ceph-mon[29478]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-04-25T10:28:15.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:15 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T10:28:15.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:15 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:15.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:15 smithi043 ceph-mon[29478]: Deploying daemon osd.8 on smithi164 2024-04-25T10:28:16.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:15 smithi077 ceph-mon[32772]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-04-25T10:28:16.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:15 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T10:28:16.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:15 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:16.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:15 smithi077 ceph-mon[32772]: Deploying daemon osd.8 on smithi164 2024-04-25T10:28:17.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:17 smithi043 ceph-mon[29478]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T10:28:17.968 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:17 smithi164 ceph-mon[32713]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T10:28:18.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:17 smithi077 ceph-mon[32772]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T10:28:19.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:19 smithi164 ceph-mon[32713]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-25T10:28:19.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:19 smithi043 ceph-mon[29478]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-25T10:28:20.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:19 smithi077 ceph-mon[32772]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-25T10:28:21.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:21 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:21.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:21 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:21.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:21 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:28:21.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:21 smithi043 ceph-mon[29478]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T10:28:21.470 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:21 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:21.471 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:21 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:21.471 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:21 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:28:21.471 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:21 smithi164 ceph-mon[32713]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T10:28:21.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:21 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:21.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:21 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:21.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:21 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:28:21.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:21 smithi077 ceph-mon[32772]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T10:28:23.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:23 smithi077 ceph-mon[32772]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:28:23.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:23.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:23.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:23.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:28:23.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:23.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:23 smithi164 ceph-mon[32713]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:28:23.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:23.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:23.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:23.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:28:23.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:23.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:23 smithi043 ceph-mon[29478]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:28:23.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:23.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:23.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:23.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:28:23.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:23.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:23.973+0000 7f7ff37fe700 1 -- 172.21.15.164:0/2694831417 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f800402e8b0 con 0x7f7fec0afb80 2024-04-25T10:28:23.975 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 8 on host 'smithi164' 2024-04-25T10:28:23.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:23.974+0000 7f8008a1b700 1 -- 172.21.15.164:0/2694831417 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f7fec0afb80 msgr2=0x7f7fec0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:23.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:23.974+0000 7f8008a1b700 1 --2- 172.21.15.164:0/2694831417 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f7fec0afb80 0x7f7fec0b2090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8004105270 tx=0x7f7ff8005f90 comp rx=0 tx=0).stop 2024-04-25T10:28:23.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:23.974+0000 7f8008a1b700 1 -- 172.21.15.164:0/2694831417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80041048b0 msgr2=0x7f800419ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:23.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:23.974+0000 7f8008a1b700 1 --2- 172.21.15.164:0/2694831417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80041048b0 0x7f800419ae40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4006010 tx=0x7f7ff4005e80 comp rx=0 tx=0).stop 2024-04-25T10:28:23.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:23.974+0000 7f8008a1b700 1 -- 172.21.15.164:0/2694831417 shutdown_connections 2024-04-25T10:28:23.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:23.974+0000 7f8008a1b700 1 --2- 172.21.15.164:0/2694831417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8004105880 0x7f80041a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:23.974+0000 7f8008a1b700 1 --2- 172.21.15.164:0/2694831417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8004103e30 0x7f800419a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:23.974+0000 7f8008a1b700 1 --2- 172.21.15.164:0/2694831417 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f7fec0afb80 0x7f7fec0b2090 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:23.974+0000 7f8008a1b700 1 --2- 172.21.15.164:0/2694831417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80041048b0 0x7f800419ae40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:23.974+0000 7f8008a1b700 1 -- 172.21.15.164:0/2694831417 >> 172.21.15.164:0/2694831417 conn(0x7f80040ff8a0 msgr2=0x7f80041085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:23.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:23.974+0000 7f8008a1b700 1 -- 172.21.15.164:0/2694831417 shutdown_connections 2024-04-25T10:28:23.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:28:23.974+0000 7f8008a1b700 1 -- 172.21.15.164:0/2694831417 wait complete. 2024-04-25T10:28:24.618 DEBUG:teuthology.orchestra.run.smithi164:osd.8> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T10:28:24.621 INFO:tasks.cephadm:Deploying osd.9 on smithi043 with /dev/vg_nvme/lv_1... 2024-04-25T10:28:24.621 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T10:28:24.878 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:28:25.212 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:24 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:25.212 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:24 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:25.212 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:24 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:28:25.212 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:24 smithi043 ceph-mon[29478]: from='osd.8 [v2:172.21.15.164:6816/3396063471,v1:172.21.15.164:6817/3396063471]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T10:28:25.212 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:24 smithi043 ceph-mon[29478]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T10:28:25.212 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:24 smithi043 ceph-mon[29478]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:28:25.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:24 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:25.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:24 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:25.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:24 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:28:25.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:24 smithi077 ceph-mon[32772]: from='osd.8 [v2:172.21.15.164:6816/3396063471,v1:172.21.15.164:6817/3396063471]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T10:28:25.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:24 smithi077 ceph-mon[32772]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T10:28:25.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:24 smithi077 ceph-mon[32772]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:28:25.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:24 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:25.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:24 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:25.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:24 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:28:25.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:24 smithi164 ceph-mon[32713]: from='osd.8 [v2:172.21.15.164:6816/3396063471,v1:172.21.15.164:6817/3396063471]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T10:28:25.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:24 smithi164 ceph-mon[32713]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T10:28:25.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:24 smithi164 ceph-mon[32713]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:28:26.029 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:25 smithi164 ceph-mon[32713]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T10:28:26.029 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:25 smithi164 ceph-mon[32713]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T10:28:26.029 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:25 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:28:26.029 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:25 smithi164 ceph-mon[32713]: from='osd.8 [v2:172.21.15.164:6816/3396063471,v1:172.21.15.164:6817/3396063471]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:28:26.029 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:25 smithi164 ceph-mon[32713]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:28:26.277 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:25 smithi043 ceph-mon[29478]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T10:28:26.277 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:25 smithi043 ceph-mon[29478]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T10:28:26.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:25 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:28:26.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:25 smithi043 ceph-mon[29478]: from='osd.8 [v2:172.21.15.164:6816/3396063471,v1:172.21.15.164:6817/3396063471]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:28:26.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:25 smithi043 ceph-mon[29478]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:28:26.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:25 smithi077 ceph-mon[32772]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T10:28:26.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:25 smithi077 ceph-mon[32772]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T10:28:26.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:25 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:28:26.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:25 smithi077 ceph-mon[32772]: from='osd.8 [v2:172.21.15.164:6816/3396063471,v1:172.21.15.164:6817/3396063471]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:28:26.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:25 smithi077 ceph-mon[32772]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:28:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T10:28:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T10:28:26.869 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-04-25T10:28:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-04-25T10:28:26.869 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0087202 s, 1.2 GB/s 2024-04-25T10:28:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-04-25T10:28:27.180 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:26 smithi043 ceph-mon[29478]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T10:28:27.180 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:26 smithi043 ceph-mon[29478]: osdmap e53: 9 total, 8 up, 9 in 2024-04-25T10:28:27.180 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:26 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:28:27.180 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:26 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:28:27.180 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:26 smithi043 ceph-mon[29478]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:28:27.180 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:26 smithi043 ceph-mon[29478]: from='osd.8 ' entity='osd.8' 2024-04-25T10:28:27.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:26 smithi077 ceph-mon[32772]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T10:28:27.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:26 smithi077 ceph-mon[32772]: osdmap e53: 9 total, 8 up, 9 in 2024-04-25T10:28:27.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:28:27.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:28:27.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:26 smithi077 ceph-mon[32772]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:28:27.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:26 smithi077 ceph-mon[32772]: from='osd.8 ' entity='osd.8' 2024-04-25T10:28:27.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:26 smithi164 ceph-mon[32713]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T10:28:27.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:26 smithi164 ceph-mon[32713]: osdmap e53: 9 total, 8 up, 9 in 2024-04-25T10:28:27.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:26 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:28:27.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:26 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:28:27.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:26 smithi164 ceph-mon[32713]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-04-25T10:28:27.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:26 smithi164 ceph-mon[32713]: from='osd.8 ' entity='osd.8' 2024-04-25T10:28:27.292 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:28:26 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:28:26.864+0000 7f5dfa1cf700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T10:28:27.293 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:28:26 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:28:26.948+0000 7f5df1f7c700 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:28:27.556 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_1 2024-04-25T10:28:27.814 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:28:28.268 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:27 smithi043 ceph-mon[29478]: purged_snaps scrub starts 2024-04-25T10:28:28.268 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:27 smithi043 ceph-mon[29478]: purged_snaps scrub ok 2024-04-25T10:28:28.268 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:27 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:28:28.269 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:27 smithi043 ceph-mon[29478]: osd.8 [v2:172.21.15.164:6816/3396063471,v1:172.21.15.164:6817/3396063471] boot 2024-04-25T10:28:28.269 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:27 smithi043 ceph-mon[29478]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T10:28:28.269 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:27 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:28:28.269 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:27 smithi043 ceph-mon[29478]: Detected new or changed devices on smithi164 2024-04-25T10:28:28.269 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:27 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:28.269 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:27 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:28.269 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:27 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:28.269 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:27 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:28.269 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:27 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:28.269 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:27 smithi043 ceph-mon[29478]: Adjusting osd_memory_target on smithi164 to 5696M 2024-04-25T10:28:28.269 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:27 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:28.269 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:27 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:28.270 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:27 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:28:28.270 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:27 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:28.270 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:27 smithi043 ceph-mon[29478]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T10:28:28.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:27 smithi077 ceph-mon[32772]: purged_snaps scrub starts 2024-04-25T10:28:28.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:27 smithi077 ceph-mon[32772]: purged_snaps scrub ok 2024-04-25T10:28:28.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:27 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:28:28.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:27 smithi077 ceph-mon[32772]: osd.8 [v2:172.21.15.164:6816/3396063471,v1:172.21.15.164:6817/3396063471] boot 2024-04-25T10:28:28.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:27 smithi077 ceph-mon[32772]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T10:28:28.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:27 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:28:28.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:27 smithi077 ceph-mon[32772]: Detected new or changed devices on smithi164 2024-04-25T10:28:28.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:27 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:28.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:27 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:28.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:27 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:28.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:27 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:28.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:27 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:28.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:27 smithi077 ceph-mon[32772]: Adjusting osd_memory_target on smithi164 to 5696M 2024-04-25T10:28:28.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:27 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:28.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:27 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:28.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:27 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:28:28.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:27 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:28.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:27 smithi077 ceph-mon[32772]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T10:28:28.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:27 smithi164 ceph-mon[32713]: purged_snaps scrub starts 2024-04-25T10:28:28.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:27 smithi164 ceph-mon[32713]: purged_snaps scrub ok 2024-04-25T10:28:28.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:27 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:28:28.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:27 smithi164 ceph-mon[32713]: osd.8 [v2:172.21.15.164:6816/3396063471,v1:172.21.15.164:6817/3396063471] boot 2024-04-25T10:28:28.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:27 smithi164 ceph-mon[32713]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T10:28:28.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:27 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:28:28.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:27 smithi164 ceph-mon[32713]: Detected new or changed devices on smithi164 2024-04-25T10:28:28.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:27 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:28.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:27 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:28.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:27 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:28.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:27 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:28.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:27 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:28.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:27 smithi164 ceph-mon[32713]: Adjusting osd_memory_target on smithi164 to 5696M 2024-04-25T10:28:28.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:27 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:28.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:27 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:28.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:27 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:28:28.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:27 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:28.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:27 smithi164 ceph-mon[32713]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T10:28:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.367+0000 7f993cc1c700 1 -- 172.21.15.43:0/4123532598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938107640 msgr2=0x7f9938109a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.367+0000 7f993cc1c700 1 --2- 172.21.15.43:0/4123532598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938107640 0x7f9938109a60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f992c009990 tx=0x7f992c009ca0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.367+0000 7f993cc1c700 1 -- 172.21.15.43:0/4123532598 shutdown_connections 2024-04-25T10:28:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.367+0000 7f993cc1c700 1 --2- 172.21.15.43:0/4123532598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9938109fa0 0x7f993810c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.367+0000 7f993cc1c700 1 --2- 172.21.15.43:0/4123532598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9938104ce0 0x7f9938107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.367+0000 7f993cc1c700 1 --2- 172.21.15.43:0/4123532598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938107640 0x7f9938109a60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.367+0000 7f993cc1c700 1 -- 172.21.15.43:0/4123532598 >> 172.21.15.43:0/4123532598 conn(0x7f99380fe740 msgr2=0x7f9938100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.368+0000 7f993cc1c700 1 -- 172.21.15.43:0/4123532598 shutdown_connections 2024-04-25T10:28:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.368+0000 7f993cc1c700 1 -- 172.21.15.43:0/4123532598 wait complete. 2024-04-25T10:28:29.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.368+0000 7f993cc1c700 1 Processor -- start 2024-04-25T10:28:29.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.369+0000 7f993cc1c700 1 -- start start 2024-04-25T10:28:29.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.369+0000 7f993cc1c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9938104ce0 0x7f993819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.369+0000 7f993cc1c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9938107640 0x7f993819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.369+0000 7f993cc1c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938109fa0 0x7f99381a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.369+0000 7f993cc1c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f993810fc50 con 0x7f9938109fa0 2024-04-25T10:28:29.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.369+0000 7f993cc1c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f993810fad0 con 0x7f9938104ce0 2024-04-25T10:28:29.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.369+0000 7f993cc1c700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f993810fdd0 con 0x7f9938107640 2024-04-25T10:28:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.369+0000 7f9936d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938109fa0 0x7f99381a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.369+0000 7f9936d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938109fa0 0x7f99381a1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60440/0 (socket says 172.21.15.43:60440) 2024-04-25T10:28:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.369+0000 7f9936d9d700 1 -- 172.21.15.43:0/9180860 learned_addr learned my addr 172.21.15.43:0/9180860 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:28:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.369+0000 7f993659c700 1 --2- 172.21.15.43:0/9180860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9938104ce0 0x7f993819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.369+0000 7f9936d9d700 1 -- 172.21.15.43:0/9180860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9938107640 msgr2=0x7f993819ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.369+0000 7f9936d9d700 1 --2- 172.21.15.43:0/9180860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9938107640 0x7f993819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.369+0000 7f9936d9d700 1 -- 172.21.15.43:0/9180860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9938104ce0 msgr2=0x7f993819a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.369+0000 7f9936d9d700 1 --2- 172.21.15.43:0/9180860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9938104ce0 0x7f993819a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.369+0000 7f9936d9d700 1 -- 172.21.15.43:0/9180860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99381a2290 con 0x7f9938109fa0 2024-04-25T10:28:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.370+0000 7f9936d9d700 1 --2- 172.21.15.43:0/9180860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938109fa0 0x7f99381a1ca0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9930007d70 tx=0x7f993000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.370+0000 7f99277fe700 1 -- 172.21.15.43:0/9180860 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9930016020 con 0x7f9938109fa0 2024-04-25T10:28:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.370+0000 7f993cc1c700 1 -- 172.21.15.43:0/9180860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99381a2580 con 0x7f9938109fa0 2024-04-25T10:28:29.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.370+0000 7f99277fe700 1 -- 172.21.15.43:0/9180860 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f993000ad40 con 0x7f9938109fa0 2024-04-25T10:28:29.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.370+0000 7f99277fe700 1 -- 172.21.15.43:0/9180860 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99300115e0 con 0x7f9938109fa0 2024-04-25T10:28:29.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.370+0000 7f993cc1c700 1 -- 172.21.15.43:0/9180860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99381a2a70 con 0x7f9938109fa0 2024-04-25T10:28:29.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.371+0000 7f993cc1c700 1 -- 172.21.15.43:0/9180860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9938051190 con 0x7f9938109fa0 2024-04-25T10:28:29.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.372+0000 7f99277fe700 1 -- 172.21.15.43:0/9180860 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f9930019020 con 0x7f9938109fa0 2024-04-25T10:28:29.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.373+0000 7f99277fe700 1 --2- 172.21.15.43:0/9180860 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f99200afb80 0x7f99200b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.374+0000 7f993659c700 1 --2- 172.21.15.43:0/9180860 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f99200afb80 0x7f99200b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.374+0000 7f99277fe700 1 -- 172.21.15.43:0/9180860 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5829+0+0 (secure 0 0 0) 0x7f99300de720 con 0x7f9938109fa0 2024-04-25T10:28:29.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.374+0000 7f993659c700 1 --2- 172.21.15.43:0/9180860 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f99200afb80 0x7f99200b2090 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f993806bf00 tx=0x7f99280073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:29.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.377+0000 7f99277fe700 1 -- 172.21.15.43:0/9180860 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9930020020 con 0x7f9938109fa0 2024-04-25T10:28:29.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:29.598+0000 7f993cc1c700 1 -- 172.21.15.43:0/9180860 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9938063ad0 con 0x7f99200afb80 2024-04-25T10:28:29.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:29 smithi043 ceph-mon[29478]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:29.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:29 smithi043 ceph-mon[29478]: osdmap e56: 9 total, 9 up, 9 in 2024-04-25T10:28:29.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:29 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:28:29.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:29 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:28:29.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:29 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:30.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:29 smithi077 ceph-mon[32772]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:30.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:29 smithi077 ceph-mon[32772]: osdmap e56: 9 total, 9 up, 9 in 2024-04-25T10:28:30.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:29 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:28:30.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:29 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:28:30.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:29 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:30.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:29 smithi164 ceph-mon[32713]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:30.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:29 smithi164 ceph-mon[32713]: osdmap e56: 9 total, 9 up, 9 in 2024-04-25T10:28:30.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:29 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:28:30.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:29 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:28:30.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:29 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:30.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:30 smithi043 ceph-mon[29478]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:28:31.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:30 smithi077 ceph-mon[32772]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:28:31.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:30 smithi164 ceph-mon[32713]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:28:32.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:31 smithi077 ceph-mon[32772]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-25T10:28:32.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:31 smithi164 ceph-mon[32713]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-25T10:28:32.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:31 smithi043 ceph-mon[29478]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-25T10:28:33.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:32 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1432444976' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a77c9906-5352-4b5d-b18b-22d586109892"}]: dispatch 2024-04-25T10:28:33.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:32 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1432444976' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a77c9906-5352-4b5d-b18b-22d586109892"}]': finished 2024-04-25T10:28:33.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:32 smithi077 ceph-mon[32772]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T10:28:33.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:28:33.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:32 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1432444976' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a77c9906-5352-4b5d-b18b-22d586109892"}]: dispatch 2024-04-25T10:28:33.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:32 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1432444976' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a77c9906-5352-4b5d-b18b-22d586109892"}]': finished 2024-04-25T10:28:33.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:32 smithi164 ceph-mon[32713]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T10:28:33.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:28:33.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:32 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1432444976' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a77c9906-5352-4b5d-b18b-22d586109892"}]: dispatch 2024-04-25T10:28:33.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:32 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1432444976' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a77c9906-5352-4b5d-b18b-22d586109892"}]': finished 2024-04-25T10:28:33.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:32 smithi043 ceph-mon[29478]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T10:28:33.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:28:34.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:33 smithi077 ceph-mon[32772]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T10:28:34.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:33 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2069360980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:28:34.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:33 smithi164 ceph-mon[32713]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T10:28:34.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:33 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2069360980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:28:34.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:33 smithi043 ceph-mon[29478]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T10:28:34.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:33 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2069360980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:28:36.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:35 smithi043 ceph-mon[29478]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-25T10:28:36.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:35 smithi077 ceph-mon[32772]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-25T10:28:36.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:35 smithi164 ceph-mon[32713]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-25T10:28:38.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:37 smithi077 ceph-mon[32772]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T10:28:38.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:37 smithi164 ceph-mon[32713]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T10:28:38.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:37 smithi043 ceph-mon[29478]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T10:28:40.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:39 smithi077 ceph-mon[32772]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-25T10:28:40.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:39 smithi164 ceph-mon[32713]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-25T10:28:40.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:39 smithi043 ceph-mon[29478]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-25T10:28:41.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:40 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T10:28:41.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:40 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:41.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:40 smithi043 ceph-mon[29478]: Deploying daemon osd.9 on smithi043 2024-04-25T10:28:41.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:40 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T10:28:41.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:40 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:41.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:40 smithi077 ceph-mon[32772]: Deploying daemon osd.9 on smithi043 2024-04-25T10:28:41.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:40 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T10:28:41.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:40 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:41.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:40 smithi164 ceph-mon[32713]: Deploying daemon osd.9 on smithi043 2024-04-25T10:28:42.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:41 smithi043 ceph-mon[29478]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:42.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:41 smithi077 ceph-mon[32772]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:42.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:41 smithi164 ceph-mon[32713]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:44.092 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:43 smithi077 ceph-mon[32772]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:44.093 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:43 smithi164 ceph-mon[32713]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:44.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:43 smithi043 ceph-mon[29478]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:46.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:45 smithi043 ceph-mon[29478]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:46.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:45 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:46.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:45 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:46.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:45 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:28:46.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:45 smithi077 ceph-mon[32772]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:46.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:45 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:46.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:45 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:46.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:45 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:28:46.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:45 smithi164 ceph-mon[32713]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:46.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:45 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:46.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:45 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:46.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:45 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:28:47.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:47 smithi043 ceph-mon[29478]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:48.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:47 smithi077 ceph-mon[32772]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:48.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:47 smithi164 ceph-mon[32713]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:49.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:49.606+0000 7f99277fe700 1 -- 172.21.15.43:0/9180860 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9938063ad0 con 0x7f99200afb80 2024-04-25T10:28:49.609 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 9 on host 'smithi043' 2024-04-25T10:28:49.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:49.609+0000 7f993cc1c700 1 -- 172.21.15.43:0/9180860 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f99200afb80 msgr2=0x7f99200b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:49.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:49.609+0000 7f993cc1c700 1 --2- 172.21.15.43:0/9180860 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f99200afb80 0x7f99200b2090 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f993806bf00 tx=0x7f99280073d0 comp rx=0 tx=0).stop 2024-04-25T10:28:49.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:49.609+0000 7f993cc1c700 1 -- 172.21.15.43:0/9180860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938109fa0 msgr2=0x7f99381a1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:49.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:49.609+0000 7f993cc1c700 1 --2- 172.21.15.43:0/9180860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938109fa0 0x7f99381a1ca0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9930007d70 tx=0x7f993000a340 comp rx=0 tx=0).stop 2024-04-25T10:28:49.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:49.610+0000 7f993cc1c700 1 -- 172.21.15.43:0/9180860 shutdown_connections 2024-04-25T10:28:49.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:49.610+0000 7f993cc1c700 1 --2- 172.21.15.43:0/9180860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938109fa0 0x7f99381a1ca0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:49.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:49.610+0000 7f993cc1c700 1 --2- 172.21.15.43:0/9180860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9938104ce0 0x7f993819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:49.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:49.610+0000 7f993cc1c700 1 --2- 172.21.15.43:0/9180860 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f99200afb80 0x7f99200b2090 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:49.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:49.610+0000 7f993cc1c700 1 --2- 172.21.15.43:0/9180860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9938107640 0x7f993819ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:49.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:49.610+0000 7f993cc1c700 1 -- 172.21.15.43:0/9180860 >> 172.21.15.43:0/9180860 conn(0x7f99380fe740 msgr2=0x7f99381057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:49.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:49.610+0000 7f993cc1c700 1 -- 172.21.15.43:0/9180860 shutdown_connections 2024-04-25T10:28:49.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:28:49.610+0000 7f993cc1c700 1 -- 172.21.15.43:0/9180860 wait complete. 2024-04-25T10:28:49.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:49 smithi043 ceph-mon[29478]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:49.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:49.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:28:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:49 smithi043 ceph-mon[29478]: from='osd.9 [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T10:28:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:50.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:49 smithi077 ceph-mon[32772]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:50.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:50.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:50.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:50.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:28:50.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:50.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:49 smithi077 ceph-mon[32772]: from='osd.9 [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T10:28:50.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:50.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:50.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:49 smithi164 ceph-mon[32713]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:50.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:50.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:50.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:50.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:28:50.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:50.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:49 smithi164 ceph-mon[32713]: from='osd.9 [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T10:28:50.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:50.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:50.246 DEBUG:teuthology.orchestra.run.smithi043:osd.9> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T10:28:50.248 INFO:tasks.cephadm:Deploying osd.10 on smithi077 with /dev/vg_nvme/lv_1... 2024-04-25T10:28:50.248 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T10:28:50.493 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T10:28:50.837 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:50 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:28:50.838 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:50 smithi077 ceph-mon[32772]: from='osd.9 [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T10:28:50.838 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:50 smithi077 ceph-mon[32772]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T10:28:50.838 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:50 smithi077 ceph-mon[32772]: from='osd.9 [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-25T10:28:50.838 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:50 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:28:50.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:50 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:28:50.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:50 smithi043 ceph-mon[29478]: from='osd.9 [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T10:28:50.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:50 smithi043 ceph-mon[29478]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T10:28:50.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:50 smithi043 ceph-mon[29478]: from='osd.9 [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-25T10:28:50.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:50 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:28:51.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:50 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:28:51.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:50 smithi164 ceph-mon[32713]: from='osd.9 [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T10:28:51.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:50 smithi164 ceph-mon[32713]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T10:28:51.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:50 smithi164 ceph-mon[32713]: from='osd.9 [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-25T10:28:51.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:50 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:28:51.609 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:28:51 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:28:51.547+0000 7f84d84e3700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T10:28:51.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:28:51 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:28:51.566+0000 7f84d0290700 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:28:51.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:51 smithi043 ceph-mon[29478]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:51.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:51 smithi043 ceph-mon[29478]: from='osd.9 [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-25T10:28:51.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:51 smithi043 ceph-mon[29478]: osdmap e59: 10 total, 9 up, 10 in 2024-04-25T10:28:51.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:51 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:28:51.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:51 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:28:51.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:51 smithi043 ceph-mon[29478]: from='osd.9 [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413]' entity='osd.9' 2024-04-25T10:28:52.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:51 smithi077 ceph-mon[32772]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:52.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:51 smithi077 ceph-mon[32772]: from='osd.9 [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-25T10:28:52.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:51 smithi077 ceph-mon[32772]: osdmap e59: 10 total, 9 up, 10 in 2024-04-25T10:28:52.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:51 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:28:52.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:51 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:28:52.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:51 smithi077 ceph-mon[32772]: from='osd.9 [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413]' entity='osd.9' 2024-04-25T10:28:52.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:51 smithi164 ceph-mon[32713]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:52.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:51 smithi164 ceph-mon[32713]: from='osd.9 [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-25T10:28:52.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:51 smithi164 ceph-mon[32713]: osdmap e59: 10 total, 9 up, 10 in 2024-04-25T10:28:52.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:51 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:28:52.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:51 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:28:52.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:51 smithi164 ceph-mon[32713]: from='osd.9 [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413]' entity='osd.9' 2024-04-25T10:28:52.395 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T10:28:52.395 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T10:28:52.396 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-04-25T10:28:52.396 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-04-25T10:28:52.396 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115311 s, 909 MB/s 2024-04-25T10:28:52.396 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-04-25T10:28:53.022 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:52 smithi077 ceph-mon[32772]: purged_snaps scrub starts 2024-04-25T10:28:53.022 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:52 smithi077 ceph-mon[32772]: purged_snaps scrub ok 2024-04-25T10:28:53.022 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:52 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:28:53.022 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:52 smithi077 ceph-mon[32772]: osd.9 [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413] boot 2024-04-25T10:28:53.023 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:52 smithi077 ceph-mon[32772]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T10:28:53.023 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:52 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:28:53.051 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_1 2024-04-25T10:28:53.092 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:52 smithi043 ceph-mon[29478]: purged_snaps scrub starts 2024-04-25T10:28:53.092 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:52 smithi043 ceph-mon[29478]: purged_snaps scrub ok 2024-04-25T10:28:53.092 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:52 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:28:53.092 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:52 smithi043 ceph-mon[29478]: osd.9 [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413] boot 2024-04-25T10:28:53.093 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:52 smithi043 ceph-mon[29478]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T10:28:53.093 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:52 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:28:53.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:52 smithi164 ceph-mon[32713]: purged_snaps scrub starts 2024-04-25T10:28:53.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:52 smithi164 ceph-mon[32713]: purged_snaps scrub ok 2024-04-25T10:28:53.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:52 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:28:53.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:52 smithi164 ceph-mon[32713]: osd.9 [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413] boot 2024-04-25T10:28:53.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:52 smithi164 ceph-mon[32713]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T10:28:53.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:52 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:28:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T10:28:54.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:53 smithi077 ceph-mon[32772]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:54.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:53 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:54.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:53 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:54.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:53 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:54.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:53 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:54.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:53 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:54.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:53 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:54.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:53 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:54.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:53 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:54.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:53 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:28:54.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:53 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:54.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:53 smithi043 ceph-mon[29478]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:54.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:53 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:54.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:53 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:54.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:53 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:54.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:53 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:54.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:53 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:54.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:53 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:54.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:53 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:54.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:53 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:54.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:53 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:28:54.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:53 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:54.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:53 smithi164 ceph-mon[32713]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-04-25T10:28:54.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:53 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:54.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:53 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:54.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:53 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:54.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:53 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:54.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:53 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:54.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:53 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:28:54.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:53 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:54.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:53 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:54.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:53 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:28:54.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:53 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:28:54.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.914+0000 7f15f9ca8700 1 -- 172.21.15.77:0/4216864155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15f4107680 msgr2=0x7f15f4109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:54.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.914+0000 7f15f9ca8700 1 --2- 172.21.15.77:0/4216864155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15f4107680 0x7f15f4109aa0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009990 tx=0x7f15e8009ca0 comp rx=0 tx=0).stop 2024-04-25T10:28:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.914+0000 7f15f9ca8700 1 -- 172.21.15.77:0/4216864155 shutdown_connections 2024-04-25T10:28:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.914+0000 7f15f9ca8700 1 --2- 172.21.15.77:0/4216864155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15f4109fe0 0x7f15f410c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.914+0000 7f15f9ca8700 1 --2- 172.21.15.77:0/4216864155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f4104d20 0x7f15f4107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.914+0000 7f15f9ca8700 1 --2- 172.21.15.77:0/4216864155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15f4107680 0x7f15f4109aa0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.914+0000 7f15f9ca8700 1 -- 172.21.15.77:0/4216864155 >> 172.21.15.77:0/4216864155 conn(0x7f15f40fe760 msgr2=0x7f15f4100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.914+0000 7f15f9ca8700 1 -- 172.21.15.77:0/4216864155 shutdown_connections 2024-04-25T10:28:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.914+0000 7f15f9ca8700 1 -- 172.21.15.77:0/4216864155 wait complete. 2024-04-25T10:28:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.915+0000 7f15f9ca8700 1 Processor -- start 2024-04-25T10:28:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.915+0000 7f15f9ca8700 1 -- start start 2024-04-25T10:28:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.915+0000 7f15f9ca8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15f4104d20 0x7f15f4104580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.915+0000 7f15f9ca8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f4107680 0x7f15f4102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.915+0000 7f15f9ca8700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15f4109fe0 0x7f15f4103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.915+0000 7f15f9ca8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f15f410fc70 con 0x7f15f4107680 2024-04-25T10:28:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.915+0000 7f15f9ca8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f15f410faf0 con 0x7f15f4104d20 2024-04-25T10:28:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.915+0000 7f15f9ca8700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f15f410fdf0 con 0x7f15f4109fe0 2024-04-25T10:28:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.915+0000 7f15f37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15f4104d20 0x7f15f4104580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.915+0000 7f15f37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15f4104d20 0x7f15f4104580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50656/0 (socket says 172.21.15.77:50656) 2024-04-25T10:28:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.915+0000 7f15f37fe700 1 -- 172.21.15.77:0/581286023 learned_addr learned my addr 172.21.15.77:0/581286023 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-25T10:28:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.916+0000 7f15f2ffd700 1 --2- 172.21.15.77:0/581286023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f4107680 0x7f15f4102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.916+0000 7f15f3fff700 1 --2- 172.21.15.77:0/581286023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15f4109fe0 0x7f15f4103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.916+0000 7f15f37fe700 1 -- 172.21.15.77:0/581286023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15f4109fe0 msgr2=0x7f15f4103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.916+0000 7f15f37fe700 1 --2- 172.21.15.77:0/581286023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15f4109fe0 0x7f15f4103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.916+0000 7f15f37fe700 1 -- 172.21.15.77:0/581286023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f4107680 msgr2=0x7f15f4102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.916+0000 7f15f37fe700 1 --2- 172.21.15.77:0/581286023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f4107680 0x7f15f4102c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.916+0000 7f15f37fe700 1 -- 172.21.15.77:0/581286023 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f4103840 con 0x7f15f4104d20 2024-04-25T10:28:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.916+0000 7f15f3fff700 1 --2- 172.21.15.77:0/581286023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15f4109fe0 0x7f15f4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.916+0000 7f15f37fe700 1 --2- 172.21.15.77:0/581286023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15f4104d20 0x7f15f4104580 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f15e400c9e0 tx=0x7f15e400cda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.916+0000 7f15f0ff9700 1 -- 172.21.15.77:0/581286023 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4007aa0 con 0x7f15f4104d20 2024-04-25T10:28:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.916+0000 7f15f0ff9700 1 -- 172.21.15.77:0/581286023 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e4007c40 con 0x7f15f4104d20 2024-04-25T10:28:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.916+0000 7f15f0ff9700 1 -- 172.21.15.77:0/581286023 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4018660 con 0x7f15f4104d20 2024-04-25T10:28:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.916+0000 7f15f9ca8700 1 -- 172.21.15.77:0/581286023 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f41a48b0 con 0x7f15f4104d20 2024-04-25T10:28:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.916+0000 7f15f9ca8700 1 -- 172.21.15.77:0/581286023 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f41a4d00 con 0x7f15f4104d20 2024-04-25T10:28:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.917+0000 7f15f9ca8700 1 -- 172.21.15.77:0/581286023 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f4051190 con 0x7f15f4104d20 2024-04-25T10:28:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.918+0000 7f15f0ff9700 1 -- 172.21.15.77:0/581286023 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f15e4018800 con 0x7f15f4104d20 2024-04-25T10:28:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.919+0000 7f15f0ff9700 1 --2- 172.21.15.77:0/581286023 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f15dc0afb80 0x7f15dc0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.919+0000 7f15f0ff9700 1 -- 172.21.15.77:0/581286023 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6261+0+0 (secure 0 0 0) 0x7f15e40de5f0 con 0x7f15f4104d20 2024-04-25T10:28:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.919+0000 7f15f2ffd700 1 --2- 172.21.15.77:0/581286023 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f15dc0afb80 0x7f15dc0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.920+0000 7f15f2ffd700 1 --2- 172.21.15.77:0/581286023 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f15dc0afb80 0x7f15dc0b2090 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f15e800b2e0 tx=0x7f15e800b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:54.923+0000 7f15f0ff9700 1 -- 172.21.15.77:0/581286023 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15e4053ff0 con 0x7f15f4104d20 2024-04-25T10:28:55.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:54 smithi077 ceph-mon[32772]: Detected new or changed devices on smithi043 2024-04-25T10:28:55.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:54 smithi077 ceph-mon[32772]: Adjusting osd_memory_target on smithi043 to 4273M 2024-04-25T10:28:55.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:54 smithi077 ceph-mon[32772]: osdmap e61: 10 total, 10 up, 10 in 2024-04-25T10:28:55.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:54 smithi164 ceph-mon[32713]: Detected new or changed devices on smithi043 2024-04-25T10:28:55.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:54 smithi164 ceph-mon[32713]: Adjusting osd_memory_target on smithi043 to 4273M 2024-04-25T10:28:55.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:54 smithi164 ceph-mon[32713]: osdmap e61: 10 total, 10 up, 10 in 2024-04-25T10:28:55.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:54 smithi043 ceph-mon[29478]: Detected new or changed devices on smithi043 2024-04-25T10:28:55.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:54 smithi043 ceph-mon[29478]: Adjusting osd_memory_target on smithi043 to 4273M 2024-04-25T10:28:55.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:54 smithi043 ceph-mon[29478]: osdmap e61: 10 total, 10 up, 10 in 2024-04-25T10:28:55.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:28:55.130+0000 7f15f9ca8700 1 -- 172.21.15.77:0/581286023 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f15f4063ad0 con 0x7f15dc0afb80 2024-04-25T10:28:56.009 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:55 smithi077 ceph-mon[32772]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:28:56.009 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:55 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:28:56.009 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:55 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:28:56.009 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:55 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:56.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:55 smithi043 ceph-mon[29478]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:28:56.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:55 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:28:56.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:55 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:28:56.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:55 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:56.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:55 smithi164 ceph-mon[32713]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:28:56.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:55 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:28:56.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:55 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:28:56.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:55 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:28:57.033 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:56 smithi077 ceph-mon[32772]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:28:57.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:56 smithi043 ceph-mon[29478]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:28:57.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:56 smithi164 ceph-mon[32713]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:28:58.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:57 smithi043 ceph-mon[29478]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:28:58.258 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:57 smithi077 ceph-mon[32772]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:28:58.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:57 smithi164 ceph-mon[32713]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:28:59.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:58 smithi043 ceph-mon[29478]: from='client.? 172.21.15.77:0/906312884' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f728b6e7-cb77-4883-8b80-152cfa4a712c"}]: dispatch 2024-04-25T10:28:59.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:58 smithi043 ceph-mon[29478]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f728b6e7-cb77-4883-8b80-152cfa4a712c"}]: dispatch 2024-04-25T10:28:59.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:58 smithi043 ceph-mon[29478]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f728b6e7-cb77-4883-8b80-152cfa4a712c"}]': finished 2024-04-25T10:28:59.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:58 smithi043 ceph-mon[29478]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T10:28:59.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:58 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:28:59.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:58 smithi043 ceph-mon[29478]: from='client.? 172.21.15.77:0/3899482495' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:28:59.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:58 smithi077 ceph-mon[32772]: from='client.? 172.21.15.77:0/906312884' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f728b6e7-cb77-4883-8b80-152cfa4a712c"}]: dispatch 2024-04-25T10:28:59.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:58 smithi077 ceph-mon[32772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f728b6e7-cb77-4883-8b80-152cfa4a712c"}]: dispatch 2024-04-25T10:28:59.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:58 smithi077 ceph-mon[32772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f728b6e7-cb77-4883-8b80-152cfa4a712c"}]': finished 2024-04-25T10:28:59.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:58 smithi077 ceph-mon[32772]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T10:28:59.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:58 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:28:59.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:58 smithi077 ceph-mon[32772]: from='client.? 172.21.15.77:0/3899482495' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:28:59.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:58 smithi164 ceph-mon[32713]: from='client.? 172.21.15.77:0/906312884' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f728b6e7-cb77-4883-8b80-152cfa4a712c"}]: dispatch 2024-04-25T10:28:59.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:58 smithi164 ceph-mon[32713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f728b6e7-cb77-4883-8b80-152cfa4a712c"}]: dispatch 2024-04-25T10:28:59.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:58 smithi164 ceph-mon[32713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f728b6e7-cb77-4883-8b80-152cfa4a712c"}]': finished 2024-04-25T10:28:59.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:58 smithi164 ceph-mon[32713]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T10:28:59.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:58 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:28:59.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:58 smithi164 ceph-mon[32713]: from='client.? 172.21.15.77:0/3899482495' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:29:00.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:28:59 smithi043 ceph-mon[29478]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:00.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:28:59 smithi077 ceph-mon[32772]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:00.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:28:59 smithi164 ceph-mon[32713]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:02.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:01 smithi043 ceph-mon[29478]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:02.196 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:01 smithi077 ceph-mon[32772]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:02.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:01 smithi164 ceph-mon[32713]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:04.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:03 smithi077 ceph-mon[32772]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:04.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:03 smithi043 ceph-mon[29478]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:04.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:03 smithi164 ceph-mon[32713]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:06.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:05 smithi043 ceph-mon[29478]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:06.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:05 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T10:29:06.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:05 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:06.265 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:05 smithi077 ceph-mon[32772]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:06.266 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:05 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T10:29:06.266 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:05 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:06.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:05 smithi164 ceph-mon[32713]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:06.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:05 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T10:29:06.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:05 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:07.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:06 smithi043 ceph-mon[29478]: Deploying daemon osd.10 on smithi077 2024-04-25T10:29:07.215 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:06 smithi077 ceph-mon[32772]: Deploying daemon osd.10 on smithi077 2024-04-25T10:29:07.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:06 smithi164 ceph-mon[32713]: Deploying daemon osd.10 on smithi077 2024-04-25T10:29:08.238 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:07 smithi077 ceph-mon[32772]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:08.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:07 smithi164 ceph-mon[32713]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:08.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:07 smithi043 ceph-mon[29478]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:10.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:09 smithi077 ceph-mon[32772]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:10.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:09 smithi164 ceph-mon[32713]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:10.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:09 smithi043 ceph-mon[29478]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:12.017 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:11 smithi077 ceph-mon[32772]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:12.017 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:11 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:12.017 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:11 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:12.018 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:11 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:29:12.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:11 smithi164 ceph-mon[32713]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:12.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:11 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:12.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:11 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:12.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:11 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:29:12.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:11 smithi043 ceph-mon[29478]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:12.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:11 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:12.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:11 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:12.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:11 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:29:14.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:13 smithi077 ceph-mon[32772]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:14.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:13 smithi164 ceph-mon[32713]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:14.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:13 smithi043 ceph-mon[29478]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:15.153 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 10 on host 'smithi077' 2024-04-25T10:29:15.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:29:15.151+0000 7f15f0ff9700 1 -- 172.21.15.77:0/581286023 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f15f4063ad0 con 0x7f15dc0afb80 2024-04-25T10:29:15.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:29:15.153+0000 7f15f9ca8700 1 -- 172.21.15.77:0/581286023 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f15dc0afb80 msgr2=0x7f15dc0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:15.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:29:15.153+0000 7f15f9ca8700 1 --2- 172.21.15.77:0/581286023 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f15dc0afb80 0x7f15dc0b2090 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f15e800b2e0 tx=0x7f15e800b720 comp rx=0 tx=0).stop 2024-04-25T10:29:15.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:29:15.153+0000 7f15f9ca8700 1 -- 172.21.15.77:0/581286023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15f4104d20 msgr2=0x7f15f4104580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:15.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:29:15.153+0000 7f15f9ca8700 1 --2- 172.21.15.77:0/581286023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15f4104d20 0x7f15f4104580 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f15e400c9e0 tx=0x7f15e400cda0 comp rx=0 tx=0).stop 2024-04-25T10:29:15.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:29:15.153+0000 7f15f9ca8700 1 -- 172.21.15.77:0/581286023 shutdown_connections 2024-04-25T10:29:15.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:29:15.153+0000 7f15f9ca8700 1 --2- 172.21.15.77:0/581286023 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f15dc0afb80 0x7f15dc0b2090 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:15.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:29:15.153+0000 7f15f9ca8700 1 --2- 172.21.15.77:0/581286023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15f4109fe0 0x7f15f4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:15.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:29:15.153+0000 7f15f9ca8700 1 --2- 172.21.15.77:0/581286023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15f4104d20 0x7f15f4104580 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:15.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:29:15.153+0000 7f15f9ca8700 1 --2- 172.21.15.77:0/581286023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f4107680 0x7f15f4102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:15.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:29:15.153+0000 7f15f9ca8700 1 -- 172.21.15.77:0/581286023 >> 172.21.15.77:0/581286023 conn(0x7f15f40fe760 msgr2=0x7f15f41057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:15.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:29:15.154+0000 7f15f9ca8700 1 -- 172.21.15.77:0/581286023 shutdown_connections 2024-04-25T10:29:15.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-25T10:29:15.154+0000 7f15f9ca8700 1 -- 172.21.15.77:0/581286023 wait complete. 2024-04-25T10:29:15.435 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:15 smithi077 ceph-mon[32772]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:15.435 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:15 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:15.435 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:15 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:15.435 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:15 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:15.435 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:15 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:29:15.436 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:15 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:15.436 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:15 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:15.436 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:15 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:29:15.436 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:15 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:15.532 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:15 smithi164 ceph-mon[32713]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:15.532 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:15 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:15.533 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:15 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:15.533 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:15 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:15.533 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:15 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:29:15.533 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:15 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:15.533 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:15 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:15.533 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:15 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:29:15.533 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:15 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:15.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:15 smithi043 ceph-mon[29478]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:15.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:15 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:15.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:15 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:15.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:15 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:15.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:15 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:29:15.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:15 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:15.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:15 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:15.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:15 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:29:15.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:15 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:15.774 DEBUG:teuthology.orchestra.run.smithi077:osd.10> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T10:29:15.777 INFO:tasks.cephadm:Deploying osd.11 on smithi164 with /dev/vg_nvme/lv_1... 2024-04-25T10:29:15.777 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T10:29:16.013 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:29:16.430 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:16 smithi164 ceph-mon[32713]: from='osd.10 [v2:172.21.15.77:6824/1788538313,v1:172.21.15.77:6825/1788538313]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T10:29:16.430 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:16 smithi164 ceph-mon[32713]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T10:29:16.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:16 smithi077 ceph-mon[32772]: from='osd.10 [v2:172.21.15.77:6824/1788538313,v1:172.21.15.77:6825/1788538313]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T10:29:16.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:16 smithi077 ceph-mon[32772]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T10:29:16.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:16 smithi043 ceph-mon[29478]: from='osd.10 [v2:172.21.15.77:6824/1788538313,v1:172.21.15.77:6825/1788538313]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T10:29:16.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:16 smithi043 ceph-mon[29478]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T10:29:17.410 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:17 smithi164 ceph-mon[32713]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:17.410 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:17 smithi164 ceph-mon[32713]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T10:29:17.410 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:17 smithi164 ceph-mon[32713]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T10:29:17.410 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:29:17.410 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:17 smithi164 ceph-mon[32713]: from='osd.10 [v2:172.21.15.77:6824/1788538313,v1:172.21.15.77:6825/1788538313]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:29:17.410 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:17 smithi164 ceph-mon[32713]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:29:17.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:17 smithi077 ceph-mon[32772]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:17.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:17 smithi077 ceph-mon[32772]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T10:29:17.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:17 smithi077 ceph-mon[32772]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T10:29:17.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:17 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:29:17.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:17 smithi077 ceph-mon[32772]: from='osd.10 [v2:172.21.15.77:6824/1788538313,v1:172.21.15.77:6825/1788538313]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:29:17.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:17 smithi077 ceph-mon[32772]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:29:17.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:17 smithi043 ceph-mon[29478]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:17.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:17 smithi043 ceph-mon[29478]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T10:29:17.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:17 smithi043 ceph-mon[29478]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T10:29:17.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:17 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:29:17.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:17 smithi043 ceph-mon[29478]: from='osd.10 [v2:172.21.15.77:6824/1788538313,v1:172.21.15.77:6825/1788538313]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:29:17.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:17 smithi043 ceph-mon[29478]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-25T10:29:18.089 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T10:29:18.089 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T10:29:18.089 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-04-25T10:29:18.089 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-04-25T10:29:18.089 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011346 s, 924 MB/s 2024-04-25T10:29:18.089 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-04-25T10:29:18.399 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:18 smithi164 ceph-mon[32713]: purged_snaps scrub starts 2024-04-25T10:29:18.399 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:18 smithi164 ceph-mon[32713]: purged_snaps scrub ok 2024-04-25T10:29:18.399 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:18 smithi164 ceph-mon[32713]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-25T10:29:18.400 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:18 smithi164 ceph-mon[32713]: osdmap e64: 11 total, 10 up, 11 in 2024-04-25T10:29:18.400 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:18 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:29:18.400 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:18 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:29:18.400 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:18 smithi164 ceph-mon[32713]: osdmap e65: 11 total, 10 up, 11 in 2024-04-25T10:29:18.400 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:18 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:29:18.400 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:18 smithi164 ceph-mon[32713]: from='osd.10 ' entity='osd.10' 2024-04-25T10:29:18.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:18 smithi077 ceph-mon[32772]: purged_snaps scrub starts 2024-04-25T10:29:18.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:18 smithi077 ceph-mon[32772]: purged_snaps scrub ok 2024-04-25T10:29:18.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:18 smithi077 ceph-mon[32772]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-25T10:29:18.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:18 smithi077 ceph-mon[32772]: osdmap e64: 11 total, 10 up, 11 in 2024-04-25T10:29:18.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:18 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:29:18.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:18 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:29:18.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:18 smithi077 ceph-mon[32772]: osdmap e65: 11 total, 10 up, 11 in 2024-04-25T10:29:18.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:18 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:29:18.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:18 smithi077 ceph-mon[32772]: from='osd.10 ' entity='osd.10' 2024-04-25T10:29:18.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:29:18 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:29:18.111+0000 7f60041a5700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T10:29:18.542 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:29:18 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:29:18.177+0000 7f5ffbf52700 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:29:18.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:18 smithi043 ceph-mon[29478]: purged_snaps scrub starts 2024-04-25T10:29:18.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:18 smithi043 ceph-mon[29478]: purged_snaps scrub ok 2024-04-25T10:29:18.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:18 smithi043 ceph-mon[29478]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-25T10:29:18.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:18 smithi043 ceph-mon[29478]: osdmap e64: 11 total, 10 up, 11 in 2024-04-25T10:29:18.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:18 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:29:18.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:18 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:29:18.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:18 smithi043 ceph-mon[29478]: osdmap e65: 11 total, 10 up, 11 in 2024-04-25T10:29:18.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:18 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:29:18.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:18 smithi043 ceph-mon[29478]: from='osd.10 ' entity='osd.10' 2024-04-25T10:29:18.661 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_1 2024-04-25T10:29:18.898 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:29:19.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:19 smithi164 ceph-mon[32713]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:19.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:19 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:29:19.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:19 smithi164 ceph-mon[32713]: osd.10 [v2:172.21.15.77:6824/1788538313,v1:172.21.15.77:6825/1788538313] boot 2024-04-25T10:29:19.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:19 smithi164 ceph-mon[32713]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T10:29:19.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:19 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:29:19.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:19 smithi164 ceph-mon[32713]: Detected new or changed devices on smithi077 2024-04-25T10:29:19.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:19 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:19.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:19 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:19.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:19 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:19.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:19 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:19.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:19 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:19.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:19 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:19.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:19 smithi164 ceph-mon[32713]: Adjusting osd_memory_target on smithi077 to 4272M 2024-04-25T10:29:19.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:19 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:19.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:19 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:19.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:19 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:29:19.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:19 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:19.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:19 smithi077 ceph-mon[32772]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:19.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:19 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:29:19.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:19 smithi077 ceph-mon[32772]: osd.10 [v2:172.21.15.77:6824/1788538313,v1:172.21.15.77:6825/1788538313] boot 2024-04-25T10:29:19.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:19 smithi077 ceph-mon[32772]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T10:29:19.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:19 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:29:19.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:19 smithi077 ceph-mon[32772]: Detected new or changed devices on smithi077 2024-04-25T10:29:19.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:19 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:19.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:19 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:19.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:19 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:19.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:19 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:19.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:19 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:19.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:19 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:19.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:19 smithi077 ceph-mon[32772]: Adjusting osd_memory_target on smithi077 to 4272M 2024-04-25T10:29:19.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:19 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:19.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:19 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:19.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:19 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:29:19.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:19 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:19.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:19 smithi043 ceph-mon[29478]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-25T10:29:19.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:19 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:29:19.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:19 smithi043 ceph-mon[29478]: osd.10 [v2:172.21.15.77:6824/1788538313,v1:172.21.15.77:6825/1788538313] boot 2024-04-25T10:29:19.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:19 smithi043 ceph-mon[29478]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T10:29:19.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:19 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:29:19.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:19 smithi043 ceph-mon[29478]: Detected new or changed devices on smithi077 2024-04-25T10:29:19.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:19 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:19.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:19 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:19.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:19 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:19.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:19 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:19.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:19 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:19.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:19 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:19.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:19 smithi043 ceph-mon[29478]: Adjusting osd_memory_target on smithi077 to 4272M 2024-04-25T10:29:19.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:19 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:19.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:19 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:19.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:19 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:29:19.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:19 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:20.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.570+0000 7f474c996700 1 -- 172.21.15.164:0/118513164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47481039e0 msgr2=0x7f4748103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.570+0000 7f474c996700 1 --2- 172.21.15.164:0/118513164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47481039e0 0x7f4748103df0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4730009990 tx=0x7f4730009ca0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.570+0000 7f474c996700 1 -- 172.21.15.164:0/118513164 shutdown_connections 2024-04-25T10:29:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.570+0000 7f474c996700 1 --2- 172.21.15.164:0/118513164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4748104380 0x7f474810ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.570+0000 7f474c996700 1 --2- 172.21.15.164:0/118513164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47481028b0 0x7f4748102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.570+0000 7f474c996700 1 --2- 172.21.15.164:0/118513164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47481039e0 0x7f4748103df0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.570+0000 7f474c996700 1 -- 172.21.15.164:0/118513164 >> 172.21.15.164:0/118513164 conn(0x7f47480fdef0 msgr2=0x7f47481003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.570+0000 7f474c996700 1 -- 172.21.15.164:0/118513164 shutdown_connections 2024-04-25T10:29:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.570+0000 7f474c996700 1 -- 172.21.15.164:0/118513164 wait complete. 2024-04-25T10:29:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.571+0000 7f474c996700 1 Processor -- start 2024-04-25T10:29:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.571+0000 7f474c996700 1 -- start start 2024-04-25T10:29:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.571+0000 7f474c996700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47481028b0 0x7f474810c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.571+0000 7f474c996700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47481039e0 0x7f474819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.571+0000 7f474c996700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4748104380 0x7f47481a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.571+0000 7f474c996700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f474810ec10 con 0x7f47481039e0 2024-04-25T10:29:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.571+0000 7f474c996700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f474810ea90 con 0x7f4748104380 2024-04-25T10:29:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.571+0000 7f474c996700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f474810ed90 con 0x7f47481028b0 2024-04-25T10:29:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.571+0000 7f474659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47481028b0 0x7f474810c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.571+0000 7f474659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47481028b0 0x7f474810c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46400/0 (socket says 172.21.15.164:46400) 2024-04-25T10:29:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.571+0000 7f474659c700 1 -- 172.21.15.164:0/3534121625 learned_addr learned my addr 172.21.15.164:0/3534121625 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-25T10:29:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.572+0000 7f4746d9d700 1 --2- 172.21.15.164:0/3534121625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4748104380 0x7f47481a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.572+0000 7f473dd9b700 1 --2- 172.21.15.164:0/3534121625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47481039e0 0x7f474819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.572+0000 7f474659c700 1 -- 172.21.15.164:0/3534121625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4748104380 msgr2=0x7f47481a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.572+0000 7f474659c700 1 --2- 172.21.15.164:0/3534121625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4748104380 0x7f47481a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.572+0000 7f474659c700 1 -- 172.21.15.164:0/3534121625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47481039e0 msgr2=0x7f474819f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.572+0000 7f474659c700 1 --2- 172.21.15.164:0/3534121625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47481039e0 0x7f474819f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.572+0000 7f474659c700 1 -- 172.21.15.164:0/3534121625 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47481a2390 con 0x7f47481028b0 2024-04-25T10:29:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.572+0000 7f474659c700 1 --2- 172.21.15.164:0/3534121625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47481028b0 0x7f474810c200 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f473800b4d0 tx=0x7f473800b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:20.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.572+0000 7f473ffff700 1 -- 172.21.15.164:0/3534121625 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4738015020 con 0x7f47481028b0 2024-04-25T10:29:20.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.572+0000 7f474c996700 1 -- 172.21.15.164:0/3534121625 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47481a2680 con 0x7f47481028b0 2024-04-25T10:29:20.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.572+0000 7f474c996700 1 -- 172.21.15.164:0/3534121625 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47481a2c90 con 0x7f47481028b0 2024-04-25T10:29:20.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.572+0000 7f473ffff700 1 -- 172.21.15.164:0/3534121625 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4738011ce0 con 0x7f47481028b0 2024-04-25T10:29:20.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.572+0000 7f473ffff700 1 -- 172.21.15.164:0/3534121625 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4738010c40 con 0x7f47481028b0 2024-04-25T10:29:20.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.574+0000 7f474c996700 1 -- 172.21.15.164:0/3534121625 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4748051190 con 0x7f47481028b0 2024-04-25T10:29:20.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.575+0000 7f473ffff700 1 -- 172.21.15.164:0/3534121625 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f4738018020 con 0x7f47481028b0 2024-04-25T10:29:20.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.575+0000 7f473ffff700 1 --2- 172.21.15.164:0/3534121625 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f47340afc00 0x7f47340b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:20.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.575+0000 7f473ffff700 1 -- 172.21.15.164:0/3534121625 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6694+0+0 (secure 0 0 0) 0x7f47380ddbe0 con 0x7f47481028b0 2024-04-25T10:29:20.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.576+0000 7f473dd9b700 1 --2- 172.21.15.164:0/3534121625 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f47340afc00 0x7f47340b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:20.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.577+0000 7f473dd9b700 1 --2- 172.21.15.164:0/3534121625 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f47340afc00 0x7f47340b2110 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f473000b5c0 tx=0x7f473001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:20.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.580+0000 7f473ffff700 1 -- 172.21.15.164:0/3534121625 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f473801f020 con 0x7f47481028b0 2024-04-25T10:29:20.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:20.790+0000 7f474c996700 1 -- 172.21.15.164:0/3534121625 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f474802e970 con 0x7f47340afc00 2024-04-25T10:29:21.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:20 smithi077 ceph-mon[32772]: osdmap e67: 11 total, 11 up, 11 in 2024-04-25T10:29:21.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:20 smithi164 ceph-mon[32713]: osdmap e67: 11 total, 11 up, 11 in 2024-04-25T10:29:21.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:20 smithi043 ceph-mon[29478]: osdmap e67: 11 total, 11 up, 11 in 2024-04-25T10:29:22.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:21 smithi077 ceph-mon[32772]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 360 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:22.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:21 smithi077 ceph-mon[32772]: osdmap e68: 11 total, 11 up, 11 in 2024-04-25T10:29:22.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:21 smithi077 ceph-mon[32772]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:29:22.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:21 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:29:22.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:21 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:29:22.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:21 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:22.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:21 smithi164 ceph-mon[32713]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 360 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:22.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:21 smithi164 ceph-mon[32713]: osdmap e68: 11 total, 11 up, 11 in 2024-04-25T10:29:22.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:21 smithi164 ceph-mon[32713]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:29:22.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:21 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:29:22.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:21 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:29:22.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:21 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:22.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:21 smithi043 ceph-mon[29478]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 360 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:22.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:21 smithi043 ceph-mon[29478]: osdmap e68: 11 total, 11 up, 11 in 2024-04-25T10:29:22.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:21 smithi043 ceph-mon[29478]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:29:22.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:21 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T10:29:22.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:21 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T10:29:22.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:21 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:24.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:23 smithi077 ceph-mon[32772]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 360 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:24.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:23 smithi077 ceph-mon[32772]: from='client.? 172.21.15.164:0/2524533675' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6b51b3f-75f3-4864-bfd9-fe42814e8bfd"}]: dispatch 2024-04-25T10:29:24.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:23 smithi077 ceph-mon[32772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6b51b3f-75f3-4864-bfd9-fe42814e8bfd"}]: dispatch 2024-04-25T10:29:24.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:23 smithi077 ceph-mon[32772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6b51b3f-75f3-4864-bfd9-fe42814e8bfd"}]': finished 2024-04-25T10:29:24.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:23 smithi077 ceph-mon[32772]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T10:29:24.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:29:24.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:23 smithi164 ceph-mon[32713]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 360 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:24.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:23 smithi164 ceph-mon[32713]: from='client.? 172.21.15.164:0/2524533675' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6b51b3f-75f3-4864-bfd9-fe42814e8bfd"}]: dispatch 2024-04-25T10:29:24.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:23 smithi164 ceph-mon[32713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6b51b3f-75f3-4864-bfd9-fe42814e8bfd"}]: dispatch 2024-04-25T10:29:24.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:23 smithi164 ceph-mon[32713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6b51b3f-75f3-4864-bfd9-fe42814e8bfd"}]': finished 2024-04-25T10:29:24.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:23 smithi164 ceph-mon[32713]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T10:29:24.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:29:24.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:23 smithi043 ceph-mon[29478]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 360 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:24.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:23 smithi043 ceph-mon[29478]: from='client.? 172.21.15.164:0/2524533675' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6b51b3f-75f3-4864-bfd9-fe42814e8bfd"}]: dispatch 2024-04-25T10:29:24.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:23 smithi043 ceph-mon[29478]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6b51b3f-75f3-4864-bfd9-fe42814e8bfd"}]: dispatch 2024-04-25T10:29:24.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:23 smithi043 ceph-mon[29478]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6b51b3f-75f3-4864-bfd9-fe42814e8bfd"}]': finished 2024-04-25T10:29:24.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:23 smithi043 ceph-mon[29478]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T10:29:24.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:29:25.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:24 smithi043 ceph-mon[29478]: from='client.? 172.21.15.164:0/511801357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:29:25.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:24 smithi077 ceph-mon[32772]: from='client.? 172.21.15.164:0/511801357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:29:25.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:24 smithi164 ceph-mon[32713]: from='client.? 172.21.15.164:0/511801357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T10:29:26.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:25 smithi043 ceph-mon[29478]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 364 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:26.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:25 smithi077 ceph-mon[32772]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 364 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:26.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:25 smithi164 ceph-mon[32713]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 364 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:28.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:27 smithi164 ceph-mon[32713]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-25T10:29:28.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:27 smithi043 ceph-mon[29478]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-25T10:29:28.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:27 smithi077 ceph-mon[32772]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-25T10:29:30.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:29 smithi043 ceph-mon[29478]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T10:29:30.189 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:29 smithi164 ceph-mon[32713]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T10:29:30.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:29 smithi077 ceph-mon[32772]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T10:29:31.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:30 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T10:29:31.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:30 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:31.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:30 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T10:29:31.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:30 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:31.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:30 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T10:29:31.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:30 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:32.017 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:31 smithi164 ceph-mon[32713]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-25T10:29:32.018 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:31 smithi164 ceph-mon[32713]: Deploying daemon osd.11 on smithi164 2024-04-25T10:29:32.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:31 smithi043 ceph-mon[29478]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-25T10:29:32.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:31 smithi043 ceph-mon[29478]: Deploying daemon osd.11 on smithi164 2024-04-25T10:29:32.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:31 smithi077 ceph-mon[32772]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-25T10:29:32.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:31 smithi077 ceph-mon[32772]: Deploying daemon osd.11 on smithi164 2024-04-25T10:29:34.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:33 smithi164 ceph-mon[32713]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:29:34.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:33 smithi043 ceph-mon[29478]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:29:34.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:33 smithi077 ceph-mon[32772]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T10:29:36.007 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:35 smithi164 ceph-mon[32713]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-25T10:29:36.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:35 smithi043 ceph-mon[29478]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-25T10:29:36.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:35 smithi077 ceph-mon[32772]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-25T10:29:37.437 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:37 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:37.437 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:37 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:37.437 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:37 smithi164 ceph-mon[32713]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T10:29:37.437 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:37 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:29:37.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:37 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:37.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:37 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:37.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:37 smithi077 ceph-mon[32772]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T10:29:37.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:37 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:29:37.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:37 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:37.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:37 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:37.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:37 smithi043 ceph-mon[29478]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T10:29:37.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:37 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:29:39.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:39 smithi077 ceph-mon[32772]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:39.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:39 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:39.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:39 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:39.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:39 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:39.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:39 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:29:39.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:39 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:39.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:39 smithi164 ceph-mon[32713]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:39.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:39 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:39.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:39 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:39.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:39 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:39.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:39 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:29:39.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:39 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:39.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:39 smithi043 ceph-mon[29478]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:39.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:39 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:39.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:39 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:39.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:39 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:39.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:39 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:29:39.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:39 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:40.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:40.214+0000 7f473ffff700 1 -- 172.21.15.164:0/3534121625 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f47300039f0 con 0x7f47340afc00 2024-04-25T10:29:40.216 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 11 on host 'smithi164' 2024-04-25T10:29:40.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:40.216+0000 7f474c996700 1 -- 172.21.15.164:0/3534121625 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f47340afc00 msgr2=0x7f47340b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:40.216+0000 7f474c996700 1 --2- 172.21.15.164:0/3534121625 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f47340afc00 0x7f47340b2110 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f473000b5c0 tx=0x7f473001a040 comp rx=0 tx=0).stop 2024-04-25T10:29:40.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:40.216+0000 7f474c996700 1 -- 172.21.15.164:0/3534121625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47481028b0 msgr2=0x7f474810c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:40.216+0000 7f474c996700 1 --2- 172.21.15.164:0/3534121625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47481028b0 0x7f474810c200 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f473800b4d0 tx=0x7f473800b7e0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:40.216+0000 7f474c996700 1 -- 172.21.15.164:0/3534121625 shutdown_connections 2024-04-25T10:29:40.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:40.216+0000 7f474c996700 1 --2- 172.21.15.164:0/3534121625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4748104380 0x7f47481a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:40.216+0000 7f474c996700 1 --2- 172.21.15.164:0/3534121625 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f47340afc00 0x7f47340b2110 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:40.216+0000 7f474c996700 1 --2- 172.21.15.164:0/3534121625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47481028b0 0x7f474810c200 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:40.216+0000 7f474c996700 1 --2- 172.21.15.164:0/3534121625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47481039e0 0x7f474819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:40.216+0000 7f474c996700 1 -- 172.21.15.164:0/3534121625 >> 172.21.15.164:0/3534121625 conn(0x7f47480fdef0 msgr2=0x7f4748106c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:40.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:40.217+0000 7f474c996700 1 -- 172.21.15.164:0/3534121625 shutdown_connections 2024-04-25T10:29:40.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:40.217+0000 7f474c996700 1 -- 172.21.15.164:0/3534121625 wait complete. 2024-04-25T10:29:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:40 smithi164 ceph-mon[32713]: from='osd.11 [v2:172.21.15.164:6824/1085943743,v1:172.21.15.164:6825/1085943743]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T10:29:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:40 smithi164 ceph-mon[32713]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T10:29:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:40 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:40 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:40 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:29:40.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:40 smithi077 ceph-mon[32772]: from='osd.11 [v2:172.21.15.164:6824/1085943743,v1:172.21.15.164:6825/1085943743]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T10:29:40.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:40 smithi077 ceph-mon[32772]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T10:29:40.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:40 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:40.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:40 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:40.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:40 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:29:40.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:40 smithi043 ceph-mon[29478]: from='osd.11 [v2:172.21.15.164:6824/1085943743,v1:172.21.15.164:6825/1085943743]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T10:29:40.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:40 smithi043 ceph-mon[29478]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T10:29:40.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:40 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:40.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:40 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:40.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:40 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:29:40.767 DEBUG:teuthology.orchestra.run.smithi164:osd.11> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T10:29:40.770 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T10:29:40.770 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T10:29:41.029 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:29:41.434 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:41 smithi043 ceph-mon[29478]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:41.435 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:41 smithi043 ceph-mon[29478]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T10:29:41.435 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:41 smithi043 ceph-mon[29478]: osdmap e70: 12 total, 11 up, 12 in 2024-04-25T10:29:41.435 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:41 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:29:41.435 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:41 smithi043 ceph-mon[29478]: from='osd.11 [v2:172.21.15.164:6824/1085943743,v1:172.21.15.164:6825/1085943743]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:29:41.435 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:41 smithi043 ceph-mon[29478]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:29:41.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:41 smithi077 ceph-mon[32772]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:41.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:41 smithi077 ceph-mon[32772]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T10:29:41.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:41 smithi077 ceph-mon[32772]: osdmap e70: 12 total, 11 up, 12 in 2024-04-25T10:29:41.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:41 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:29:41.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:41 smithi077 ceph-mon[32772]: from='osd.11 [v2:172.21.15.164:6824/1085943743,v1:172.21.15.164:6825/1085943743]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:29:41.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:41 smithi077 ceph-mon[32772]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:29:41.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:41 smithi164 ceph-mon[32713]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:41.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:41 smithi164 ceph-mon[32713]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T10:29:41.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:41 smithi164 ceph-mon[32713]: osdmap e70: 12 total, 11 up, 12 in 2024-04-25T10:29:41.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:41 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:29:41.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:41 smithi164 ceph-mon[32713]: from='osd.11 [v2:172.21.15.164:6824/1085943743,v1:172.21.15.164:6825/1085943743]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:29:41.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:41 smithi164 ceph-mon[32713]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-25T10:29:42.152 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:29:42 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:29:42.123+0000 7fe7528a5700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T10:29:42.410 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:42 smithi043 ceph-mon[29478]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T10:29:42.410 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:42 smithi043 ceph-mon[29478]: osdmap e71: 12 total, 11 up, 12 in 2024-04-25T10:29:42.410 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:42 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:29:42.410 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:42 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:29:42.410 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:42 smithi043 ceph-mon[29478]: from='osd.11 ' entity='osd.11' 2024-04-25T10:29:42.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:42 smithi077 ceph-mon[32772]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T10:29:42.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:42 smithi077 ceph-mon[32772]: osdmap e71: 12 total, 11 up, 12 in 2024-04-25T10:29:42.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:42 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:29:42.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:42 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:29:42.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:42 smithi077 ceph-mon[32772]: from='osd.11 ' entity='osd.11' 2024-04-25T10:29:42.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:42 smithi164 ceph-mon[32713]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-25T10:29:42.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:42 smithi164 ceph-mon[32713]: osdmap e71: 12 total, 11 up, 12 in 2024-04-25T10:29:42.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:42 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:29:42.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:42 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:29:42.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:42 smithi164 ceph-mon[32713]: from='osd.11 ' entity='osd.11' 2024-04-25T10:29:42.542 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:29:42 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:29:42.258+0000 7fe749e51700 -1 osd.11 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:29:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.667+0000 7f69b64d8700 1 -- 172.21.15.43:0/1941511883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69b01036e0 msgr2=0x7f69b0103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.667+0000 7f69b64d8700 1 --2- 172.21.15.43:0/1941511883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69b01036e0 0x7f69b0103af0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f69a00099c0 tx=0x7f69a0009cd0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.667+0000 7f69b64d8700 1 -- 172.21.15.43:0/1941511883 shutdown_connections 2024-04-25T10:29:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.667+0000 7f69b64d8700 1 --2- 172.21.15.43:0/1941511883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69b0104f30 0x7f69b010b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.667+0000 7f69b64d8700 1 --2- 172.21.15.43:0/1941511883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69b01036e0 0x7f69b0103af0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.667+0000 7f69b64d8700 1 --2- 172.21.15.43:0/1941511883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69b01045e0 0x7f69b01049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.667+0000 7f69b64d8700 1 -- 172.21.15.43:0/1941511883 >> 172.21.15.43:0/1941511883 conn(0x7f69b00fded0 msgr2=0x7f69b0100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.667+0000 7f69b64d8700 1 -- 172.21.15.43:0/1941511883 shutdown_connections 2024-04-25T10:29:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.667+0000 7f69b64d8700 1 -- 172.21.15.43:0/1941511883 wait complete. 2024-04-25T10:29:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.668+0000 7f69b64d8700 1 Processor -- start 2024-04-25T10:29:42.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.668+0000 7f69b64d8700 1 -- start start 2024-04-25T10:29:42.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.668+0000 7f69b64d8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69b01036e0 0x7f69b00724f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:42.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.668+0000 7f69b64d8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69b01045e0 0x7f69b0072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:42.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.668+0000 7f69b64d8700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69b0104f30 0x7f69b00739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:42.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.668+0000 7f69b64d8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f69b010ed90 con 0x7f69b01036e0 2024-04-25T10:29:42.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.668+0000 7f69b64d8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f69b010ec10 con 0x7f69b01045e0 2024-04-25T10:29:42.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.668+0000 7f69b64d8700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f69b010ef10 con 0x7f69b0104f30 2024-04-25T10:29:42.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.669+0000 7f69affff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69b01036e0 0x7f69b00724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:42.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.669+0000 7f69affff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69b01036e0 0x7f69b00724f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48720/0 (socket says 172.21.15.43:48720) 2024-04-25T10:29:42.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.669+0000 7f69affff700 1 -- 172.21.15.43:0/2669911101 learned_addr learned my addr 172.21.15.43:0/2669911101 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:29:42.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.669+0000 7f69b4a75700 1 --2- 172.21.15.43:0/2669911101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69b0104f30 0x7f69b00739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:42.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.669+0000 7f69af7fe700 1 --2- 172.21.15.43:0/2669911101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69b01045e0 0x7f69b0072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:42.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.669+0000 7f69affff700 1 -- 172.21.15.43:0/2669911101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69b0104f30 msgr2=0x7f69b00739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.669+0000 7f69affff700 1 --2- 172.21.15.43:0/2669911101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69b0104f30 0x7f69b00739a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.669+0000 7f69affff700 1 -- 172.21.15.43:0/2669911101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69b01045e0 msgr2=0x7f69b0072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.669+0000 7f69affff700 1 --2- 172.21.15.43:0/2669911101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69b01045e0 0x7f69b0072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.669+0000 7f69affff700 1 -- 172.21.15.43:0/2669911101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b00740a0 con 0x7f69b01036e0 2024-04-25T10:29:42.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.669+0000 7f69af7fe700 1 --2- 172.21.15.43:0/2669911101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69b01045e0 0x7f69b0072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:42.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.669+0000 7f69affff700 1 --2- 172.21.15.43:0/2669911101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69b01036e0 0x7f69b00724f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f69a0006010 tx=0x7f69a00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:42.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.669+0000 7f69ad7fa700 1 -- 172.21.15.43:0/2669911101 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a0027070 con 0x7f69b01036e0 2024-04-25T10:29:42.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.670+0000 7f69b64d8700 1 -- 172.21.15.43:0/2669911101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b0074330 con 0x7f69b01036e0 2024-04-25T10:29:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.670+0000 7f69ad7fa700 1 -- 172.21.15.43:0/2669911101 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a00040e0 con 0x7f69b01036e0 2024-04-25T10:29:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.670+0000 7f69ad7fa700 1 -- 172.21.15.43:0/2669911101 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a0022d80 con 0x7f69b01036e0 2024-04-25T10:29:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.670+0000 7f69b64d8700 1 -- 172.21.15.43:0/2669911101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b0074810 con 0x7f69b01036e0 2024-04-25T10:29:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.671+0000 7f69b64d8700 1 -- 172.21.15.43:0/2669911101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69b0051190 con 0x7f69b01036e0 2024-04-25T10:29:42.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.673+0000 7f69ad7fa700 1 -- 172.21.15.43:0/2669911101 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f69a002a070 con 0x7f69b01036e0 2024-04-25T10:29:42.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.674+0000 7f69ad7fa700 1 --2- 172.21.15.43:0/2669911101 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f699c0afb80 0x7f699c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:42.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.674+0000 7f69af7fe700 1 --2- 172.21.15.43:0/2669911101 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f699c0afb80 0x7f699c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:42.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.674+0000 7f69ad7fa700 1 -- 172.21.15.43:0/2669911101 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6847+0+0 (secure 0 0 0) 0x7f69a00df530 con 0x7f69b01036e0 2024-04-25T10:29:42.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.674+0000 7f69af7fe700 1 --2- 172.21.15.43:0/2669911101 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f699c0afb80 0x7f699c0b2090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6998005950 tx=0x7f69980058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.679+0000 7f69ad7fa700 1 -- 172.21.15.43:0/2669911101 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f69a0013070 con 0x7f69b01036e0 2024-04-25T10:29:42.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.886+0000 7f69b64d8700 1 -- 172.21.15.43:0/2669911101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f69b0075680 con 0x7f69b01036e0 2024-04-25T10:29:42.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.887+0000 7f69ad7fa700 1 -- 172.21.15.43:0/2669911101 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f69a00a86c0 con 0x7f69b01036e0 2024-04-25T10:29:42.889 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:29:42.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.889+0000 7f69b64d8700 1 -- 172.21.15.43:0/2669911101 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f699c0afb80 msgr2=0x7f699c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.889+0000 7f69b64d8700 1 --2- 172.21.15.43:0/2669911101 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f699c0afb80 0x7f699c0b2090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6998005950 tx=0x7f69980058e0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.889+0000 7f69b64d8700 1 -- 172.21.15.43:0/2669911101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69b01036e0 msgr2=0x7f69b00724f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.889+0000 7f69b64d8700 1 --2- 172.21.15.43:0/2669911101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69b01036e0 0x7f69b00724f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f69a0006010 tx=0x7f69a00047b0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.889+0000 7f69b64d8700 1 -- 172.21.15.43:0/2669911101 shutdown_connections 2024-04-25T10:29:42.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.889+0000 7f69b64d8700 1 --2- 172.21.15.43:0/2669911101 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f699c0afb80 0x7f699c0b2090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.889+0000 7f69b64d8700 1 --2- 172.21.15.43:0/2669911101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69b0104f30 0x7f69b00739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.889+0000 7f69b64d8700 1 --2- 172.21.15.43:0/2669911101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69b01036e0 0x7f69b00724f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.889+0000 7f69b64d8700 1 --2- 172.21.15.43:0/2669911101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69b01045e0 0x7f69b0072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.889+0000 7f69b64d8700 1 -- 172.21.15.43:0/2669911101 >> 172.21.15.43:0/2669911101 conn(0x7f69b00fded0 msgr2=0x7f69b0107810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:42.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.889+0000 7f69b64d8700 1 -- 172.21.15.43:0/2669911101 shutdown_connections 2024-04-25T10:29:42.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:42.889+0000 7f69b64d8700 1 -- 172.21.15.43:0/2669911101 wait complete. 2024-04-25T10:29:43.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:43 smithi077 ceph-mon[32772]: purged_snaps scrub starts 2024-04-25T10:29:43.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:43 smithi077 ceph-mon[32772]: purged_snaps scrub ok 2024-04-25T10:29:43.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:43 smithi077 ceph-mon[32772]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:43.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:43 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:29:43.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:43 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2669911101' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T10:29:43.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:43 smithi164 ceph-mon[32713]: purged_snaps scrub starts 2024-04-25T10:29:43.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:43 smithi164 ceph-mon[32713]: purged_snaps scrub ok 2024-04-25T10:29:43.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:43 smithi164 ceph-mon[32713]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:43.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:43 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:29:43.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:43 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2669911101' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T10:29:43.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:43 smithi043 ceph-mon[29478]: purged_snaps scrub starts 2024-04-25T10:29:43.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:43 smithi043 ceph-mon[29478]: purged_snaps scrub ok 2024-04-25T10:29:43.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:43 smithi043 ceph-mon[29478]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:43.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:43 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:29:43.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:43 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2669911101' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T10:29:43.674 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1714040958,"num_in_osds":12,"osd_in_since":1714040963,"num_remapped_pgs":0} 2024-04-25T10:29:44.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:44 smithi077 ceph-mon[32772]: osd.11 [v2:172.21.15.164:6824/1085943743,v1:172.21.15.164:6825/1085943743] boot 2024-04-25T10:29:44.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:44 smithi077 ceph-mon[32772]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T10:29:44.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:44 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:29:44.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:44 smithi077 ceph-mon[32772]: Detected new or changed devices on smithi164 2024-04-25T10:29:44.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:44 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:44.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:44 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:44.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:44 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:44.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:44 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:44.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:44 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:44.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:44 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:44.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:44 smithi077 ceph-mon[32772]: Adjusting osd_memory_target on smithi164 to 4272M 2024-04-25T10:29:44.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:44 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:44.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:44 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:44.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:44 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:29:44.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:44 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:44.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:44 smithi164 ceph-mon[32713]: osd.11 [v2:172.21.15.164:6824/1085943743,v1:172.21.15.164:6825/1085943743] boot 2024-04-25T10:29:44.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:44 smithi164 ceph-mon[32713]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T10:29:44.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:44 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:29:44.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:44 smithi164 ceph-mon[32713]: Detected new or changed devices on smithi164 2024-04-25T10:29:44.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:44 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:44.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:44 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:44.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:44 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:44.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:44 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:44.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:44 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:44.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:44 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:44.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:44 smithi164 ceph-mon[32713]: Adjusting osd_memory_target on smithi164 to 4272M 2024-04-25T10:29:44.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:44 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:44.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:44 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:44.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:44 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:29:44.544 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:44 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:44.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:44 smithi043 ceph-mon[29478]: osd.11 [v2:172.21.15.164:6824/1085943743,v1:172.21.15.164:6825/1085943743] boot 2024-04-25T10:29:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:44 smithi043 ceph-mon[29478]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T10:29:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:44 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:29:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:44 smithi043 ceph-mon[29478]: Detected new or changed devices on smithi164 2024-04-25T10:29:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:44 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:44 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:44 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:44 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:44 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:44 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:29:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:44 smithi043 ceph-mon[29478]: Adjusting osd_memory_target on smithi164 to 4272M 2024-04-25T10:29:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:44 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:44 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:44 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:29:44.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:44 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:44.676 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T10:29:44.927 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:29:45.717 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:45 smithi077 ceph-mon[32772]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:45.718 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:45 smithi077 ceph-mon[32772]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T10:29:46.005 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:45 smithi043 ceph-mon[29478]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:46.005 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:45 smithi043 ceph-mon[29478]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T10:29:46.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:45 smithi164 ceph-mon[32713]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail 2024-04-25T10:29:46.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:45 smithi164 ceph-mon[32713]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T10:29:46.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.565+0000 7f1d0b4a1700 1 -- 172.21.15.43:0/2573081430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d04104bf0 msgr2=0x7f1d0410b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.565+0000 7f1d0b4a1700 1 --2- 172.21.15.43:0/2573081430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d04104bf0 0x7f1d0410b4a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00b0a0 tx=0x7f1cfc00b3b0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.565+0000 7f1d0b4a1700 1 -- 172.21.15.43:0/2573081430 shutdown_connections 2024-04-25T10:29:46.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.565+0000 7f1d0b4a1700 1 --2- 172.21.15.43:0/2573081430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d04104bf0 0x7f1d0410b4a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.565+0000 7f1d0b4a1700 1 --2- 172.21.15.43:0/2573081430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d04103120 0x7f1d04103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.565+0000 7f1d0b4a1700 1 --2- 172.21.15.43:0/2573081430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d04104250 0x7f1d04104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.565+0000 7f1d0b4a1700 1 -- 172.21.15.43:0/2573081430 >> 172.21.15.43:0/2573081430 conn(0x7f1d040fe760 msgr2=0x7f1d04100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.566+0000 7f1d0b4a1700 1 -- 172.21.15.43:0/2573081430 shutdown_connections 2024-04-25T10:29:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.566+0000 7f1d0b4a1700 1 -- 172.21.15.43:0/2573081430 wait complete. 2024-04-25T10:29:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.566+0000 7f1d0b4a1700 1 Processor -- start 2024-04-25T10:29:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.566+0000 7f1d0b4a1700 1 -- start start 2024-04-25T10:29:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.566+0000 7f1d0b4a1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d04103120 0x7f1d04076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.567+0000 7f1d0b4a1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d04104250 0x7f1d0419f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.567+0000 7f1d0b4a1700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d04104bf0 0x7f1d041a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.567+0000 7f1d0b4a1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0410eb90 con 0x7f1d04104250 2024-04-25T10:29:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.567+0000 7f1d0b4a1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0410ea10 con 0x7f1d04103120 2024-04-25T10:29:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.567+0000 7f1d0b4a1700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0410ed10 con 0x7f1d04104bf0 2024-04-25T10:29:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.567+0000 7f1d0923d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d04103120 0x7f1d04076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.567+0000 7f1d08a3c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d04104250 0x7f1d0419f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.567+0000 7f1d0923d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d04103120 0x7f1d04076320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:50316/0 (socket says 172.21.15.43:50316) 2024-04-25T10:29:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.567+0000 7f1d08a3c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d04104250 0x7f1d0419f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48748/0 (socket says 172.21.15.43:48748) 2024-04-25T10:29:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.567+0000 7f1d0923d700 1 -- 172.21.15.43:0/1512572642 learned_addr learned my addr 172.21.15.43:0/1512572642 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:29:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.567+0000 7f1d09a3e700 1 --2- 172.21.15.43:0/1512572642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d04104bf0 0x7f1d041a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.567+0000 7f1d0923d700 1 -- 172.21.15.43:0/1512572642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d04104bf0 msgr2=0x7f1d041a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.567+0000 7f1d0923d700 1 --2- 172.21.15.43:0/1512572642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d04104bf0 0x7f1d041a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.567+0000 7f1d0923d700 1 -- 172.21.15.43:0/1512572642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d04104250 msgr2=0x7f1d0419f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.567+0000 7f1d0923d700 1 --2- 172.21.15.43:0/1512572642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d04104250 0x7f1d0419f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.567+0000 7f1d0923d700 1 -- 172.21.15.43:0/1512572642 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d041a22d0 con 0x7f1d04103120 2024-04-25T10:29:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.567+0000 7f1d08a3c700 1 --2- 172.21.15.43:0/1512572642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d04104250 0x7f1d0419f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.567+0000 7f1d0923d700 1 --2- 172.21.15.43:0/1512572642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d04103120 0x7f1d04076320 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400d950 tx=0x7f1cf400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.567+0000 7f1cfa7fc700 1 -- 172.21.15.43:0/1512572642 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4017070 con 0x7f1d04103120 2024-04-25T10:29:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.568+0000 7f1d0b4a1700 1 -- 172.21.15.43:0/1512572642 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d041a25c0 con 0x7f1d04103120 2024-04-25T10:29:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.568+0000 7f1cfa7fc700 1 -- 172.21.15.43:0/1512572642 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cf400f690 con 0x7f1d04103120 2024-04-25T10:29:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.568+0000 7f1cfa7fc700 1 -- 172.21.15.43:0/1512572642 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4003aa0 con 0x7f1d04103120 2024-04-25T10:29:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.568+0000 7f1d0b4a1700 1 -- 172.21.15.43:0/1512572642 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d041a2b00 con 0x7f1d04103120 2024-04-25T10:29:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.568+0000 7f1d0b4a1700 1 -- 172.21.15.43:0/1512572642 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d04051190 con 0x7f1d04103120 2024-04-25T10:29:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.570+0000 7f1cfa7fc700 1 -- 172.21.15.43:0/1512572642 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f1cf400f840 con 0x7f1d04103120 2024-04-25T10:29:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.571+0000 7f1cfa7fc700 1 --2- 172.21.15.43:0/1512572642 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1cf00afb80 0x7f1cf00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.571+0000 7f1d08a3c700 1 --2- 172.21.15.43:0/1512572642 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1cf00afb80 0x7f1cf00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.571+0000 7f1cfa7fc700 1 -- 172.21.15.43:0/1512572642 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f1cf40de9d0 con 0x7f1d04103120 2024-04-25T10:29:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.571+0000 7f1d08a3c700 1 --2- 172.21.15.43:0/1512572642 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1cf00afb80 0x7f1cf00b2090 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1d04075cd0 tx=0x7f1d00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.574+0000 7f1cfa7fc700 1 -- 172.21.15.43:0/1512572642 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1cf4020030 con 0x7f1d04103120 2024-04-25T10:29:46.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.781+0000 7f1d0b4a1700 1 -- 172.21.15.43:0/1512572642 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f1d040642a0 con 0x7f1d04103120 2024-04-25T10:29:46.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.782+0000 7f1cfa7fc700 1 -- 172.21.15.43:0/1512572642 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f1cf40a7a30 con 0x7f1d04103120 2024-04-25T10:29:46.784 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:29:46.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.784+0000 7f1d0b4a1700 1 -- 172.21.15.43:0/1512572642 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1cf00afb80 msgr2=0x7f1cf00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.784+0000 7f1d0b4a1700 1 --2- 172.21.15.43:0/1512572642 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1cf00afb80 0x7f1cf00b2090 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1d04075cd0 tx=0x7f1d00009260 comp rx=0 tx=0).stop 2024-04-25T10:29:46.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.784+0000 7f1d0b4a1700 1 -- 172.21.15.43:0/1512572642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d04103120 msgr2=0x7f1d04076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.784+0000 7f1d0b4a1700 1 --2- 172.21.15.43:0/1512572642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d04103120 0x7f1d04076320 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400d950 tx=0x7f1cf400dc60 comp rx=0 tx=0).stop 2024-04-25T10:29:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.784+0000 7f1d0b4a1700 1 -- 172.21.15.43:0/1512572642 shutdown_connections 2024-04-25T10:29:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.784+0000 7f1d0b4a1700 1 --2- 172.21.15.43:0/1512572642 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1cf00afb80 0x7f1cf00b2090 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.784+0000 7f1d0b4a1700 1 --2- 172.21.15.43:0/1512572642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d04104bf0 0x7f1d041a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.784+0000 7f1d0b4a1700 1 --2- 172.21.15.43:0/1512572642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d04103120 0x7f1d04076320 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.784+0000 7f1d0b4a1700 1 --2- 172.21.15.43:0/1512572642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d04104250 0x7f1d0419f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.784+0000 7f1d0b4a1700 1 -- 172.21.15.43:0/1512572642 >> 172.21.15.43:0/1512572642 conn(0x7f1d040fe760 msgr2=0x7f1d04107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.784+0000 7f1d0b4a1700 1 -- 172.21.15.43:0/1512572642 shutdown_connections 2024-04-25T10:29:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:46.784+0000 7f1d0b4a1700 1 -- 172.21.15.43:0/1512572642 wait complete. 2024-04-25T10:29:47.419 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":73,"num_osds":12,"num_up_osds":12,"osd_up_since":1714040983,"num_in_osds":12,"osd_in_since":1714040963,"num_remapped_pgs":0} 2024-04-25T10:29:47.419 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T10:29:47.666 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:29:47.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:47 smithi043 ceph-mon[29478]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:29:47.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:47 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1512572642' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T10:29:48.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:47 smithi077 ceph-mon[32772]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:29:48.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:47 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1512572642' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T10:29:48.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:47 smithi164 ceph-mon[32713]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:29:48.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:47 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1512572642' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T10:29:49.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.298+0000 7faf7c54a700 1 -- 172.21.15.43:0/2294238719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf74108610 msgr2=0x7faf7410ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.298+0000 7faf7c54a700 1 --2- 172.21.15.43:0/2294238719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf74108610 0x7faf7410ff70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00b0d0 tx=0x7faf6c00b3e0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.298+0000 7faf7c54a700 1 -- 172.21.15.43:0/2294238719 shutdown_connections 2024-04-25T10:29:49.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.298+0000 7faf7c54a700 1 --2- 172.21.15.43:0/2294238719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf74108610 0x7faf7410ff70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.298+0000 7faf7c54a700 1 --2- 172.21.15.43:0/2294238719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf74103390 0x7faf741037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.298+0000 7faf7c54a700 1 --2- 172.21.15.43:0/2294238719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf74107cc0 0x7faf741080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.298+0000 7faf7c54a700 1 -- 172.21.15.43:0/2294238719 >> 172.21.15.43:0/2294238719 conn(0x7faf740fecc0 msgr2=0x7faf741010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:49.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.299+0000 7faf7c54a700 1 -- 172.21.15.43:0/2294238719 shutdown_connections 2024-04-25T10:29:49.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.299+0000 7faf7c54a700 1 -- 172.21.15.43:0/2294238719 wait complete. 2024-04-25T10:29:49.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.299+0000 7faf7c54a700 1 Processor -- start 2024-04-25T10:29:49.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.299+0000 7faf7c54a700 1 -- start start 2024-04-25T10:29:49.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.300+0000 7faf7c54a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf74103390 0x7faf7419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.300+0000 7faf7c54a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf74107cc0 0x7faf7419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.300+0000 7faf7c54a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf74108610 0x7faf741a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.300+0000 7faf7c54a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faf74113240 con 0x7faf74107cc0 2024-04-25T10:29:49.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.300+0000 7faf7c54a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faf741130c0 con 0x7faf74103390 2024-04-25T10:29:49.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.300+0000 7faf7c54a700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf741133c0 con 0x7faf74108610 2024-04-25T10:29:49.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.300+0000 7faf79ae5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf74107cc0 0x7faf7419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.300+0000 7faf79ae5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf74107cc0 0x7faf7419d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44542/0 (socket says 172.21.15.43:44542) 2024-04-25T10:29:49.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.300+0000 7faf79ae5700 1 -- 172.21.15.43:0/4045517299 learned_addr learned my addr 172.21.15.43:0/4045517299 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:29:49.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.300+0000 7faf7a2e6700 1 --2- 172.21.15.43:0/4045517299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf74103390 0x7faf7419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.300+0000 7faf7aae7700 1 --2- 172.21.15.43:0/4045517299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf74108610 0x7faf741a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.300+0000 7faf79ae5700 1 -- 172.21.15.43:0/4045517299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf74108610 msgr2=0x7faf741a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.300+0000 7faf79ae5700 1 --2- 172.21.15.43:0/4045517299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf74108610 0x7faf741a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.300+0000 7faf79ae5700 1 -- 172.21.15.43:0/4045517299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf74103390 msgr2=0x7faf7419d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.300+0000 7faf79ae5700 1 --2- 172.21.15.43:0/4045517299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf74103390 0x7faf7419d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.300+0000 7faf79ae5700 1 -- 172.21.15.43:0/4045517299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf741a1ed0 con 0x7faf74107cc0 2024-04-25T10:29:49.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.300+0000 7faf79ae5700 1 --2- 172.21.15.43:0/4045517299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf74107cc0 0x7faf7419d5a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faf6400c8f0 tx=0x7faf6400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:49.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.301+0000 7faf6b7fe700 1 -- 172.21.15.43:0/4045517299 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf640078d0 con 0x7faf74107cc0 2024-04-25T10:29:49.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.301+0000 7faf7c54a700 1 -- 172.21.15.43:0/4045517299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf741a21c0 con 0x7faf74107cc0 2024-04-25T10:29:49.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.301+0000 7faf6b7fe700 1 -- 172.21.15.43:0/4045517299 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf6400f450 con 0x7faf74107cc0 2024-04-25T10:29:49.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.301+0000 7faf6b7fe700 1 -- 172.21.15.43:0/4045517299 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf64018660 con 0x7faf74107cc0 2024-04-25T10:29:49.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.301+0000 7faf7c54a700 1 -- 172.21.15.43:0/4045517299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf741a9ae0 con 0x7faf74107cc0 2024-04-25T10:29:49.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.302+0000 7faf7c54a700 1 -- 172.21.15.43:0/4045517299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf74051190 con 0x7faf74107cc0 2024-04-25T10:29:49.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.304+0000 7faf6b7fe700 1 -- 172.21.15.43:0/4045517299 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7faf6400f900 con 0x7faf74107cc0 2024-04-25T10:29:49.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.304+0000 7faf6b7fe700 1 --2- 172.21.15.43:0/4045517299 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faf600afc00 0x7faf600b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.304+0000 7faf7a2e6700 1 --2- 172.21.15.43:0/4045517299 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faf600afc00 0x7faf600b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.304+0000 7faf6b7fe700 1 -- 172.21.15.43:0/4045517299 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7faf640de480 con 0x7faf74107cc0 2024-04-25T10:29:49.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.305+0000 7faf7a2e6700 1 --2- 172.21.15.43:0/4045517299 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faf600afc00 0x7faf600b2110 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faf70007be0 tx=0x7faf700073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:49.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.309+0000 7faf6b7fe700 1 -- 172.21.15.43:0/4045517299 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf64014030 con 0x7faf74107cc0 2024-04-25T10:29:49.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.519+0000 7faf7c54a700 1 -- 172.21.15.43:0/4045517299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf7419e360 con 0x7faf74107cc0 2024-04-25T10:29:49.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.520+0000 7faf6b7fe700 1 -- 172.21.15.43:0/4045517299 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+18422 (secure 0 0 0) 0x7faf640a6eb0 con 0x7faf74107cc0 2024-04-25T10:29:49.522 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:29:49.523 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":73,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:29:44.632016+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:49.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.522+0000 7faf7c54a700 1 -- 172.21.15.43:0/4045517299 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faf600afc00 msgr2=0x7faf600b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.522+0000 7faf7c54a700 1 --2- 172.21.15.43:0/4045517299 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faf600afc00 0x7faf600b2110 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faf70007be0 tx=0x7faf700073d0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.522+0000 7faf7c54a700 1 -- 172.21.15.43:0/4045517299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf74107cc0 msgr2=0x7faf7419d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.522+0000 7faf7c54a700 1 --2- 172.21.15.43:0/4045517299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf74107cc0 0x7faf7419d5a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faf6400c8f0 tx=0x7faf6400cc00 comp rx=0 tx=0).stop 2024-04-25T10:29:49.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.522+0000 7faf7c54a700 1 -- 172.21.15.43:0/4045517299 shutdown_connections 2024-04-25T10:29:49.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.522+0000 7faf7c54a700 1 --2- 172.21.15.43:0/4045517299 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faf600afc00 0x7faf600b2110 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.522+0000 7faf7c54a700 1 --2- 172.21.15.43:0/4045517299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf74108610 0x7faf741a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.522+0000 7faf7c54a700 1 --2- 172.21.15.43:0/4045517299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf74103390 0x7faf7419d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.522+0000 7faf7c54a700 1 --2- 172.21.15.43:0/4045517299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf74107cc0 0x7faf7419d5a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.522+0000 7faf7c54a700 1 -- 172.21.15.43:0/4045517299 >> 172.21.15.43:0/4045517299 conn(0x7faf740fecc0 msgr2=0x7faf7410ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:49.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.522+0000 7faf7c54a700 1 -- 172.21.15.43:0/4045517299 shutdown_connections 2024-04-25T10:29:49.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:49.522+0000 7faf7c54a700 1 -- 172.21.15.43:0/4045517299 wait complete. 2024-04-25T10:29:50.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:49 smithi077 ceph-mon[32772]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:29:50.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:49 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/4045517299' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:29:50.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:49 smithi164 ceph-mon[32713]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:29:50.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:49 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/4045517299' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:29:50.060 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:49 smithi043 ceph-mon[29478]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:29:50.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:49 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/4045517299' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:29:50.088 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:25:58.167476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:50.089 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T10:29:50.334 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:29:51.763 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:51 smithi043 ceph-mon[29478]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:29:51.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.987+0000 7f293ed1b700 1 -- 172.21.15.43:0/3936973678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2938103ab0 msgr2=0x7f2938103ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.987+0000 7f293ed1b700 1 --2- 172.21.15.43:0/3936973678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2938103ab0 0x7f2938103ec0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f292800b0d0 tx=0x7f292800b3e0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.987+0000 7f293ed1b700 1 -- 172.21.15.43:0/3936973678 shutdown_connections 2024-04-25T10:29:51.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.987+0000 7f293ed1b700 1 --2- 172.21.15.43:0/3936973678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2938108810 0x7f2938110170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.987+0000 7f293ed1b700 1 --2- 172.21.15.43:0/3936973678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2938103ab0 0x7f2938103ec0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.987+0000 7f293ed1b700 1 --2- 172.21.15.43:0/3936973678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2938107ec0 0x7f29381082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.987+0000 7f293ed1b700 1 -- 172.21.15.43:0/3936973678 >> 172.21.15.43:0/3936973678 conn(0x7f29380fe3e0 msgr2=0x7f2938100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:51.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.987+0000 7f293ed1b700 1 -- 172.21.15.43:0/3936973678 shutdown_connections 2024-04-25T10:29:51.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.987+0000 7f293ed1b700 1 -- 172.21.15.43:0/3936973678 wait complete. 2024-04-25T10:29:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.988+0000 7f293ed1b700 1 Processor -- start 2024-04-25T10:29:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.988+0000 7f293ed1b700 1 -- start start 2024-04-25T10:29:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.988+0000 7f293ed1b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2938103ab0 0x7f2938199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.988+0000 7f293ed1b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2938107ec0 0x7f293819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.988+0000 7f293ed1b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2938108810 0x7f293819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.988+0000 7f293ed1b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2938113440 con 0x7f2938103ab0 2024-04-25T10:29:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.988+0000 7f293ed1b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f29381132c0 con 0x7f2938108810 2024-04-25T10:29:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.988+0000 7f293ed1b700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f29381135c0 con 0x7f2938107ec0 2024-04-25T10:29:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.988+0000 7f293cab7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2938103ab0 0x7f2938199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.988+0000 7f293cab7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2938103ab0 0x7f2938199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44556/0 (socket says 172.21.15.43:44556) 2024-04-25T10:29:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.988+0000 7f293cab7700 1 -- 172.21.15.43:0/749192292 learned_addr learned my addr 172.21.15.43:0/749192292 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:29:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.988+0000 7f293d2b8700 1 --2- 172.21.15.43:0/749192292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2938108810 0x7f293819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.989+0000 7f2937fff700 1 --2- 172.21.15.43:0/749192292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2938107ec0 0x7f293819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.989+0000 7f293cab7700 1 -- 172.21.15.43:0/749192292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2938107ec0 msgr2=0x7f293819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.989+0000 7f293cab7700 1 --2- 172.21.15.43:0/749192292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2938107ec0 0x7f293819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.989+0000 7f293cab7700 1 -- 172.21.15.43:0/749192292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2938108810 msgr2=0x7f293819e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.989+0000 7f293cab7700 1 --2- 172.21.15.43:0/749192292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2938108810 0x7f293819e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.989+0000 7f293cab7700 1 -- 172.21.15.43:0/749192292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293819ec60 con 0x7f2938103ab0 2024-04-25T10:29:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.989+0000 7f2937fff700 1 --2- 172.21.15.43:0/749192292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2938107ec0 0x7f293819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.989+0000 7f293cab7700 1 --2- 172.21.15.43:0/749192292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2938103ab0 0x7f2938199d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f292800d010 tx=0x7f2928007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.989+0000 7f2935ffb700 1 -- 172.21.15.43:0/749192292 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292801f070 con 0x7f2938103ab0 2024-04-25T10:29:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.989+0000 7f293ed1b700 1 -- 172.21.15.43:0/749192292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f293819eef0 con 0x7f2938103ab0 2024-04-25T10:29:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.989+0000 7f2935ffb700 1 -- 172.21.15.43:0/749192292 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2928003c00 con 0x7f2938103ab0 2024-04-25T10:29:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.989+0000 7f2935ffb700 1 -- 172.21.15.43:0/749192292 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292801b6a0 con 0x7f2938103ab0 2024-04-25T10:29:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.989+0000 7f293ed1b700 1 -- 172.21.15.43:0/749192292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f293819f3d0 con 0x7f2938103ab0 2024-04-25T10:29:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.991+0000 7f293ed1b700 1 -- 172.21.15.43:0/749192292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2938051190 con 0x7f2938103ab0 2024-04-25T10:29:51.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.992+0000 7f2935ffb700 1 -- 172.21.15.43:0/749192292 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f2928022070 con 0x7f2938103ab0 2024-04-25T10:29:51.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.992+0000 7f2935ffb700 1 --2- 172.21.15.43:0/749192292 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f29240afb80 0x7f29240b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:51.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.993+0000 7f2937fff700 1 --2- 172.21.15.43:0/749192292 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f29240afb80 0x7f29240b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:51.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.993+0000 7f2935ffb700 1 -- 172.21.15.43:0/749192292 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f292800f020 con 0x7f2938103ab0 2024-04-25T10:29:51.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.993+0000 7f2937fff700 1 --2- 172.21.15.43:0/749192292 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f29240afb80 0x7f29240b2090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2920005fd0 tx=0x7f2920005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:51.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:51.997+0000 7f2935ffb700 1 -- 172.21.15.43:0/749192292 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2928029020 con 0x7f2938103ab0 2024-04-25T10:29:52.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:51 smithi077 ceph-mon[32772]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:29:52.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:51 smithi164 ceph-mon[32713]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:29:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:52.204+0000 7f293ed1b700 1 -- 172.21.15.43:0/749192292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f293806c030 con 0x7f2938103ab0 2024-04-25T10:29:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:52.205+0000 7f2935ffb700 1 -- 172.21.15.43:0/749192292 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7f29280a8af0 con 0x7f2938103ab0 2024-04-25T10:29:52.207 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T10:29:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:52.207+0000 7f293ed1b700 1 -- 172.21.15.43:0/749192292 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f29240afb80 msgr2=0x7f29240b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:52.207+0000 7f293ed1b700 1 --2- 172.21.15.43:0/749192292 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f29240afb80 0x7f29240b2090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2920005fd0 tx=0x7f2920005ee0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:52.207+0000 7f293ed1b700 1 -- 172.21.15.43:0/749192292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2938103ab0 msgr2=0x7f2938199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:52.207+0000 7f293ed1b700 1 --2- 172.21.15.43:0/749192292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2938103ab0 0x7f2938199d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f292800d010 tx=0x7f2928007a20 comp rx=0 tx=0).stop 2024-04-25T10:29:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:52.207+0000 7f293ed1b700 1 -- 172.21.15.43:0/749192292 shutdown_connections 2024-04-25T10:29:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:52.207+0000 7f293ed1b700 1 --2- 172.21.15.43:0/749192292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2938108810 0x7f293819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:52.207+0000 7f293ed1b700 1 --2- 172.21.15.43:0/749192292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2938103ab0 0x7f2938199d90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:52.207+0000 7f293ed1b700 1 --2- 172.21.15.43:0/749192292 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f29240afb80 0x7f29240b2090 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:52.207+0000 7f293ed1b700 1 --2- 172.21.15.43:0/749192292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2938107ec0 0x7f293819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:52.207+0000 7f293ed1b700 1 -- 172.21.15.43:0/749192292 >> 172.21.15.43:0/749192292 conn(0x7f29380fe3e0 msgr2=0x7f29381076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:52.207+0000 7f293ed1b700 1 -- 172.21.15.43:0/749192292 shutdown_connections 2024-04-25T10:29:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:52.207+0000 7f293ed1b700 1 -- 172.21.15.43:0/749192292 wait complete. 2024-04-25T10:29:52.817 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:52 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/749192292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:29:52.846 INFO:tasks.cephadm:Adding mds.a on smithi043 2024-04-25T10:29:52.847 INFO:tasks.cephadm:Adding mds.d on smithi043 2024-04-25T10:29:52.847 INFO:tasks.cephadm:Adding mds.g on smithi043 2024-04-25T10:29:52.847 INFO:tasks.cephadm:Adding mds.j on smithi043 2024-04-25T10:29:52.847 INFO:tasks.cephadm:Adding mds.b on smithi077 2024-04-25T10:29:52.847 INFO:tasks.cephadm:Adding mds.e on smithi077 2024-04-25T10:29:52.847 INFO:tasks.cephadm:Adding mds.h on smithi077 2024-04-25T10:29:52.847 INFO:tasks.cephadm:Adding mds.k on smithi077 2024-04-25T10:29:52.847 INFO:tasks.cephadm:Adding mds.c on smithi164 2024-04-25T10:29:52.847 INFO:tasks.cephadm:Adding mds.f on smithi164 2024-04-25T10:29:52.847 INFO:tasks.cephadm:Adding mds.i on smithi164 2024-04-25T10:29:52.847 INFO:tasks.cephadm:Adding mds.l on smithi164 2024-04-25T10:29:52.847 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l' 2024-04-25T10:29:53.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:52 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/749192292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:29:53.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:52 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/749192292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:29:53.099 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:29:53.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:53 smithi164 ceph-mon[32713]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:29:54.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:53 smithi077 ceph-mon[32772]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:29:54.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:53 smithi043 ceph-mon[29478]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:29:54.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.567+0000 7fd1758ff700 1 -- 172.21.15.164:0/1115209927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1701030e0 msgr2=0x7fd1701034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.567+0000 7fd1758ff700 1 --2- 172.21.15.164:0/1115209927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1701030e0 0x7fd1701034f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd160009990 tx=0x7fd160009ca0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.567+0000 7fd1758ff700 1 -- 172.21.15.164:0/1115209927 shutdown_connections 2024-04-25T10:29:54.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.567+0000 7fd1758ff700 1 --2- 172.21.15.164:0/1115209927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd170104bb0 0x7fd17010b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.567+0000 7fd1758ff700 1 --2- 172.21.15.164:0/1115209927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1701030e0 0x7fd1701034f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.567+0000 7fd1758ff700 1 --2- 172.21.15.164:0/1115209927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd170104210 0x7fd170104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.567+0000 7fd1758ff700 1 -- 172.21.15.164:0/1115209927 >> 172.21.15.164:0/1115209927 conn(0x7fd1700fe740 msgr2=0x7fd170100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:54.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.567+0000 7fd1758ff700 1 -- 172.21.15.164:0/1115209927 shutdown_connections 2024-04-25T10:29:54.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.567+0000 7fd1758ff700 1 -- 172.21.15.164:0/1115209927 wait complete. 2024-04-25T10:29:54.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.567+0000 7fd1758ff700 1 Processor -- start 2024-04-25T10:29:54.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.568+0000 7fd1758ff700 1 -- start start 2024-04-25T10:29:54.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.568+0000 7fd1758ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1701030e0 0x7fd17006bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.568+0000 7fd1758ff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd170104210 0x7fd17019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.568+0000 7fd1758ff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd170104bb0 0x7fd1701a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.568+0000 7fd1758ff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd17010eb30 con 0x7fd170104bb0 2024-04-25T10:29:54.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.568+0000 7fd1758ff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd17010e9b0 con 0x7fd170104210 2024-04-25T10:29:54.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.568+0000 7fd1758ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd17010ecb0 con 0x7fd1701030e0 2024-04-25T10:29:54.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.568+0000 7fd16effd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1701030e0 0x7fd17006bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.568+0000 7fd16effd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1701030e0 0x7fd17006bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55094/0 (socket says 172.21.15.164:55094) 2024-04-25T10:29:54.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.568+0000 7fd16effd700 1 -- 172.21.15.164:0/3494943689 learned_addr learned my addr 172.21.15.164:0/3494943689 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-25T10:29:54.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.568+0000 7fd167fff700 1 --2- 172.21.15.164:0/3494943689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd170104210 0x7fd17019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.568+0000 7fd16f7fe700 1 --2- 172.21.15.164:0/3494943689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd170104bb0 0x7fd1701a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.568+0000 7fd16effd700 1 -- 172.21.15.164:0/3494943689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd170104210 msgr2=0x7fd17019f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.568+0000 7fd16effd700 1 --2- 172.21.15.164:0/3494943689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd170104210 0x7fd17019f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.568+0000 7fd16effd700 1 -- 172.21.15.164:0/3494943689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd170104bb0 msgr2=0x7fd1701a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.568+0000 7fd16effd700 1 --2- 172.21.15.164:0/3494943689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd170104bb0 0x7fd1701a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.568+0000 7fd16effd700 1 -- 172.21.15.164:0/3494943689 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1701a22a0 con 0x7fd1701030e0 2024-04-25T10:29:54.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.568+0000 7fd16f7fe700 1 --2- 172.21.15.164:0/3494943689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd170104bb0 0x7fd1701a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:54.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.568+0000 7fd16effd700 1 --2- 172.21.15.164:0/3494943689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1701030e0 0x7fd17006bf40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd1600057c0 tx=0x7fd1600057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:54.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.569+0000 7fd1748fd700 1 -- 172.21.15.164:0/3494943689 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160027070 con 0x7fd1701030e0 2024-04-25T10:29:54.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.569+0000 7fd1748fd700 1 -- 172.21.15.164:0/3494943689 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1600040c0 con 0x7fd1701030e0 2024-04-25T10:29:54.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.569+0000 7fd1748fd700 1 -- 172.21.15.164:0/3494943689 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160022d10 con 0x7fd1701030e0 2024-04-25T10:29:54.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.569+0000 7fd1758ff700 1 -- 172.21.15.164:0/3494943689 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1701a2530 con 0x7fd1701030e0 2024-04-25T10:29:54.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.569+0000 7fd1758ff700 1 -- 172.21.15.164:0/3494943689 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1701a2930 con 0x7fd1701030e0 2024-04-25T10:29:54.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.570+0000 7fd1758ff700 1 -- 172.21.15.164:0/3494943689 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd170051190 con 0x7fd1701030e0 2024-04-25T10:29:54.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.571+0000 7fd1748fd700 1 -- 172.21.15.164:0/3494943689 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fd16002a070 con 0x7fd1701030e0 2024-04-25T10:29:54.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.572+0000 7fd1748fd700 1 --2- 172.21.15.164:0/3494943689 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd15c0afb80 0x7fd15c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.572+0000 7fd1748fd700 1 -- 172.21.15.164:0/3494943689 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fd1600df780 con 0x7fd1701030e0 2024-04-25T10:29:54.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.572+0000 7fd167fff700 1 --2- 172.21.15.164:0/3494943689 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd15c0afb80 0x7fd15c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.573+0000 7fd167fff700 1 --2- 172.21.15.164:0/3494943689 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd15c0afb80 0x7fd15c0b2090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd158005950 tx=0x7fd15800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:54.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.577+0000 7fd1748fd700 1 -- 172.21.15.164:0/3494943689 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd160013070 con 0x7fd1701030e0 2024-04-25T10:29:54.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.786+0000 7fd1758ff700 1 -- 172.21.15.164:0/3494943689 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l", "target": ["mon-mgr", ""]}) -- 0x7fd170063bd0 con 0x7fd15c0afb80 2024-04-25T10:29:54.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.795+0000 7fd1748fd700 1 -- 172.21.15.164:0/3494943689 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fd170063bd0 con 0x7fd15c0afb80 2024-04-25T10:29:54.797 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled mds.all update... 2024-04-25T10:29:54.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.798+0000 7fd1758ff700 1 -- 172.21.15.164:0/3494943689 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd15c0afb80 msgr2=0x7fd15c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.798+0000 7fd1758ff700 1 --2- 172.21.15.164:0/3494943689 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd15c0afb80 0x7fd15c0b2090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd158005950 tx=0x7fd15800a300 comp rx=0 tx=0).stop 2024-04-25T10:29:54.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.798+0000 7fd1758ff700 1 -- 172.21.15.164:0/3494943689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1701030e0 msgr2=0x7fd17006bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.798+0000 7fd1758ff700 1 --2- 172.21.15.164:0/3494943689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1701030e0 0x7fd17006bf40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd1600057c0 tx=0x7fd1600057f0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.798+0000 7fd1758ff700 1 -- 172.21.15.164:0/3494943689 shutdown_connections 2024-04-25T10:29:54.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.798+0000 7fd1758ff700 1 --2- 172.21.15.164:0/3494943689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd170104bb0 0x7fd1701a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.798+0000 7fd1758ff700 1 --2- 172.21.15.164:0/3494943689 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd15c0afb80 0x7fd15c0b2090 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.798+0000 7fd1758ff700 1 --2- 172.21.15.164:0/3494943689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1701030e0 0x7fd17006bf40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.798+0000 7fd1758ff700 1 --2- 172.21.15.164:0/3494943689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd170104210 0x7fd17019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.798+0000 7fd1758ff700 1 -- 172.21.15.164:0/3494943689 >> 172.21.15.164:0/3494943689 conn(0x7fd1700fe740 msgr2=0x7fd170107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:54.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.798+0000 7fd1758ff700 1 -- 172.21.15.164:0/3494943689 shutdown_connections 2024-04-25T10:29:54.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-25T10:29:54.798+0000 7fd1758ff700 1 -- 172.21.15.164:0/3494943689 wait complete. 2024-04-25T10:29:55.430 DEBUG:teuthology.orchestra.run.smithi043:mds.a> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T10:29:55.432 DEBUG:teuthology.orchestra.run.smithi043:mds.d> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T10:29:55.476 DEBUG:teuthology.orchestra.run.smithi043:mds.g> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T10:29:55.519 DEBUG:teuthology.orchestra.run.smithi043:mds.j> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T10:29:55.562 DEBUG:teuthology.orchestra.run.smithi077:mds.b> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T10:29:55.565 DEBUG:teuthology.orchestra.run.smithi077:mds.e> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T10:29:55.608 DEBUG:teuthology.orchestra.run.smithi077:mds.h> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T10:29:55.651 DEBUG:teuthology.orchestra.run.smithi077:mds.k> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T10:29:55.695 DEBUG:teuthology.orchestra.run.smithi164:mds.c> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T10:29:55.697 DEBUG:teuthology.orchestra.run.smithi164:mds.f> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T10:29:55.741 DEBUG:teuthology.orchestra.run.smithi164:mds.i> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T10:29:55.784 DEBUG:teuthology.orchestra.run.smithi164:mds.l> sudo journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T10:29:55.826 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T10:29:55.827 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:55.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.945+0000 7f7cdaa13640 1 -- 172.21.15.43:0/4045287755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cd41029c0 msgr2=0x7f7cd41036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.945+0000 7f7cdaa13640 1 --2- 172.21.15.43:0/4045287755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cd41029c0 0x7f7cd41036c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009900 tx=0x7f7cc402a410 comp rx=0 tx=0).stop 2024-04-25T10:29:55.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.945+0000 7f7cdaa13640 1 -- 172.21.15.43:0/4045287755 shutdown_connections 2024-04-25T10:29:55.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.945+0000 7f7cdaa13640 1 --2- 172.21.15.43:0/4045287755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cd4108900 0x7f7cd410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.945+0000 7f7cdaa13640 1 --2- 172.21.15.43:0/4045287755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7cd4103c00 0x7f7cd4104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.945+0000 7f7cdaa13640 1 --2- 172.21.15.43:0/4045287755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cd41029c0 0x7f7cd41036c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.945+0000 7f7cdaa13640 1 -- 172.21.15.43:0/4045287755 >> 172.21.15.43:0/4045287755 conn(0x7f7cd40fdbd0 msgr2=0x7f7cd4100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:55.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.945+0000 7f7cdaa13640 1 -- 172.21.15.43:0/4045287755 shutdown_connections 2024-04-25T10:29:55.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.945+0000 7f7cdaa13640 1 -- 172.21.15.43:0/4045287755 wait complete. 2024-04-25T10:29:55.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.946+0000 7f7cdaa13640 1 Processor -- start 2024-04-25T10:29:55.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.946+0000 7f7cdaa13640 1 -- start start 2024-04-25T10:29:55.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.946+0000 7f7cdaa13640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cd41029c0 0x7f7cd4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.946+0000 7f7cdaa13640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cd4103c00 0x7f7cd4198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.946+0000 7f7cdaa13640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7cd4108900 0x7f7cd419f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.946+0000 7f7cdaa13640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd410d3e0 con 0x7f7cd41029c0 2024-04-25T10:29:55.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.946+0000 7f7cdaa13640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd410d260 con 0x7f7cd4103c00 2024-04-25T10:29:55.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.946+0000 7f7cdaa13640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd410d560 con 0x7f7cd4108900 2024-04-25T10:29:55.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.947+0000 7f7cd3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cd41029c0 0x7f7cd4198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.947+0000 7f7cd3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cd41029c0 0x7f7cd4198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44580/0 (socket says 172.21.15.43:44580) 2024-04-25T10:29:55.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.947+0000 7f7cd3fff640 1 -- 172.21.15.43:0/2575177318 learned_addr learned my addr 172.21.15.43:0/2575177318 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:29:55.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.947+0000 7f7cd37fe640 1 --2- 172.21.15.43:0/2575177318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cd4103c00 0x7f7cd4198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.947+0000 7f7cd8fb0640 1 --2- 172.21.15.43:0/2575177318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7cd4108900 0x7f7cd419f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.947+0000 7f7cd3fff640 1 -- 172.21.15.43:0/2575177318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7cd4108900 msgr2=0x7f7cd419f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.947+0000 7f7cd3fff640 1 --2- 172.21.15.43:0/2575177318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7cd4108900 0x7f7cd419f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.947+0000 7f7cd3fff640 1 -- 172.21.15.43:0/2575177318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cd4103c00 msgr2=0x7f7cd4198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.947+0000 7f7cd3fff640 1 --2- 172.21.15.43:0/2575177318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cd4103c00 0x7f7cd4198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.947+0000 7f7cd3fff640 1 -- 172.21.15.43:0/2575177318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd419fb40 con 0x7f7cd41029c0 2024-04-25T10:29:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.947+0000 7f7cd37fe640 1 --2- 172.21.15.43:0/2575177318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cd4103c00 0x7f7cd4198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.947+0000 7f7cd3fff640 1 --2- 172.21.15.43:0/2575177318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cd41029c0 0x7f7cd4198000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4006010 tx=0x7f7cc4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.947+0000 7f7cd17fa640 1 -- 172.21.15.43:0/2575177318 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc403f030 con 0x7f7cd41029c0 2024-04-25T10:29:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.947+0000 7f7cdaa13640 1 -- 172.21.15.43:0/2575177318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd419fdd0 con 0x7f7cd41029c0 2024-04-25T10:29:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.947+0000 7f7cd17fa640 1 -- 172.21.15.43:0/2575177318 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cc4031910 con 0x7f7cd41029c0 2024-04-25T10:29:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.947+0000 7f7cd17fa640 1 -- 172.21.15.43:0/2575177318 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4035070 con 0x7f7cd41029c0 2024-04-25T10:29:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.947+0000 7f7cdaa13640 1 -- 172.21.15.43:0/2575177318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd41a0220 con 0x7f7cd41029c0 2024-04-25T10:29:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.949+0000 7f7cdaa13640 1 -- 172.21.15.43:0/2575177318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd4107710 con 0x7f7cd41029c0 2024-04-25T10:29:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.950+0000 7f7cd17fa640 1 -- 172.21.15.43:0/2575177318 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f7cc4042030 con 0x7f7cd41029c0 2024-04-25T10:29:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.950+0000 7f7cd17fa640 1 --2- 172.21.15.43:0/2575177318 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f7cbc0afaf0 0x7f7cbc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.951+0000 7f7cd37fe640 1 --2- 172.21.15.43:0/2575177318 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f7cbc0afaf0 0x7f7cbc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.951+0000 7f7cd17fa640 1 -- 172.21.15.43:0/2575177318 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f7cc40febc0 con 0x7f7cd41029c0 2024-04-25T10:29:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.951+0000 7f7cd37fe640 1 --2- 172.21.15.43:0/2575177318 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f7cbc0afaf0 0x7f7cbc0b2000 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7cd41990a0 tx=0x7f7cc8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:55.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:55.955+0000 7f7cd17fa640 1 -- 172.21.15.43:0/2575177318 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7cc40c7ba0 con 0x7f7cd41029c0 2024-04-25T10:29:56.032 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:55 smithi164 ceph-mon[32713]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:29:56.032 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:55 smithi164 ceph-mon[32713]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:29:56.032 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:55 smithi164 ceph-mon[32713]: Saving service mds.all spec with placement smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l;count:12 2024-04-25T10:29:56.032 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:55 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:56.032 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:55 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:29:56.032 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:55 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:56.032 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:55 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:29:56.032 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:55 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:56.032 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:55 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:29:56.032 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:55 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:29:56.032 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:55 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:56.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:55 smithi077 ceph-mon[32772]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:29:56.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:55 smithi077 ceph-mon[32772]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:29:56.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:55 smithi077 ceph-mon[32772]: Saving service mds.all spec with placement smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l;count:12 2024-04-25T10:29:56.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:55 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:56.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:55 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:29:56.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:55 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:56.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:55 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:29:56.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:55 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:56.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:55 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:29:56.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:55 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:29:56.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:55 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:56.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:55 smithi043 ceph-mon[29478]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:29:56.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:55 smithi043 ceph-mon[29478]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:29:56.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:55 smithi043 ceph-mon[29478]: Saving service mds.all spec with placement smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l;count:12 2024-04-25T10:29:56.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:55 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:56.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:55 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:29:56.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:55 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:56.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:55 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:29:56.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:55 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:56.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:55 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:29:56.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:55 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:29:56.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:55 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:56.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.145+0000 7f7cdaa13640 1 -- 172.21.15.43:0/2575177318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7cd40637a0 con 0x7f7cd41029c0 2024-04-25T10:29:56.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.146+0000 7f7cd17fa640 1 -- 172.21.15.43:0/2575177318 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+18422 (secure 0 0 0) 0x7f7cc40c74a0 con 0x7f7cd41029c0 2024-04-25T10:29:56.149 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:29:56.149 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":73,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:29:44.632016+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.148+0000 7f7cdaa13640 1 -- 172.21.15.43:0/2575177318 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f7cbc0afaf0 msgr2=0x7f7cbc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.148+0000 7f7cdaa13640 1 --2- 172.21.15.43:0/2575177318 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f7cbc0afaf0 0x7f7cbc0b2000 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7cd41990a0 tx=0x7f7cc8005f90 comp rx=0 tx=0).stop 2024-04-25T10:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.148+0000 7f7cdaa13640 1 -- 172.21.15.43:0/2575177318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cd41029c0 msgr2=0x7f7cd4198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.148+0000 7f7cdaa13640 1 --2- 172.21.15.43:0/2575177318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cd41029c0 0x7f7cd4198000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4006010 tx=0x7f7cc4004270 comp rx=0 tx=0).stop 2024-04-25T10:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.148+0000 7f7cdaa13640 1 -- 172.21.15.43:0/2575177318 shutdown_connections 2024-04-25T10:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.148+0000 7f7cdaa13640 1 --2- 172.21.15.43:0/2575177318 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f7cbc0afaf0 0x7f7cbc0b2000 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.148+0000 7f7cdaa13640 1 --2- 172.21.15.43:0/2575177318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cd4103c00 0x7f7cd4198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.148+0000 7f7cdaa13640 1 --2- 172.21.15.43:0/2575177318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7cd4108900 0x7f7cd419f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.148+0000 7f7cdaa13640 1 --2- 172.21.15.43:0/2575177318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cd41029c0 0x7f7cd4198000 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.148+0000 7f7cdaa13640 1 -- 172.21.15.43:0/2575177318 >> 172.21.15.43:0/2575177318 conn(0x7f7cd40fdbd0 msgr2=0x7f7cd41048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.148+0000 7f7cdaa13640 1 -- 172.21.15.43:0/2575177318 shutdown_connections 2024-04-25T10:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.149+0000 7f7cdaa13640 1 -- 172.21.15.43:0/2575177318 wait complete. 2024-04-25T10:29:56.162 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:25:58.167476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:56.162 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.275+0000 7f77ff425640 1 -- 172.21.15.43:0/3329274394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77f81045d0 msgr2=0x7f77f81069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.275+0000 7f77ff425640 1 --2- 172.21.15.43:0/3329274394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77f81045d0 0x7f77f81069f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009900 tx=0x7f77ec02a410 comp rx=0 tx=0).stop 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.275+0000 7f77ff425640 1 -- 172.21.15.43:0/3329274394 shutdown_connections 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.275+0000 7f77ff425640 1 --2- 172.21.15.43:0/3329274394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77f8109890 0x7f77f810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.275+0000 7f77ff425640 1 --2- 172.21.15.43:0/3329274394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77f8106f30 0x7f77f8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.275+0000 7f77ff425640 1 --2- 172.21.15.43:0/3329274394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77f81045d0 0x7f77f81069f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.275+0000 7f77ff425640 1 -- 172.21.15.43:0/3329274394 >> 172.21.15.43:0/3329274394 conn(0x7f77f80fdbd0 msgr2=0x7f77f8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.275+0000 7f77ff425640 1 -- 172.21.15.43:0/3329274394 shutdown_connections 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.275+0000 7f77ff425640 1 -- 172.21.15.43:0/3329274394 wait complete. 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.276+0000 7f77ff425640 1 Processor -- start 2024-04-25T10:29:56.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.276+0000 7f77ff425640 1 -- start start 2024-04-25T10:29:56.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.276+0000 7f77ff425640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77f81045d0 0x7f77f819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.276+0000 7f77ff425640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77f8106f30 0x7f77f819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.276+0000 7f77ff425640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77f8109890 0x7f77f81a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.276+0000 7f77ff425640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f77f810f5c0 con 0x7f77f8106f30 2024-04-25T10:29:56.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.276+0000 7f77ff425640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f77f810f440 con 0x7f77f81045d0 2024-04-25T10:29:56.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.276+0000 7f77ff425640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f77f810f740 con 0x7f77f8109890 2024-04-25T10:29:56.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.276+0000 7f77fc9c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77f8106f30 0x7f77f819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.276+0000 7f77fc9c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77f8106f30 0x7f77f819aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44594/0 (socket says 172.21.15.43:44594) 2024-04-25T10:29:56.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.276+0000 7f77fc9c0640 1 -- 172.21.15.43:0/2430141264 learned_addr learned my addr 172.21.15.43:0/2430141264 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:29:56.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.277+0000 7f77fd1c1640 1 --2- 172.21.15.43:0/2430141264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77f81045d0 0x7f77f819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.277+0000 7f77fd9c2640 1 --2- 172.21.15.43:0/2430141264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77f8109890 0x7f77f81a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.277+0000 7f77fc9c0640 1 -- 172.21.15.43:0/2430141264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77f8109890 msgr2=0x7f77f81a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.277+0000 7f77fc9c0640 1 --2- 172.21.15.43:0/2430141264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77f8109890 0x7f77f81a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.277+0000 7f77fc9c0640 1 -- 172.21.15.43:0/2430141264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77f81045d0 msgr2=0x7f77f819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.277+0000 7f77fc9c0640 1 --2- 172.21.15.43:0/2430141264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77f81045d0 0x7f77f819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.277+0000 7f77fc9c0640 1 -- 172.21.15.43:0/2430141264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f81a20e0 con 0x7f77f8106f30 2024-04-25T10:29:56.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.277+0000 7f77fc9c0640 1 --2- 172.21.15.43:0/2430141264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77f8106f30 0x7f77f819aae0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f77f400d950 tx=0x7f77f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:56.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.277+0000 7f77ea7fc640 1 -- 172.21.15.43:0/2430141264 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4017030 con 0x7f77f8106f30 2024-04-25T10:29:56.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.277+0000 7f77ea7fc640 1 -- 172.21.15.43:0/2430141264 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77f4004020 con 0x7f77f8106f30 2024-04-25T10:29:56.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.277+0000 7f77ff425640 1 -- 172.21.15.43:0/2430141264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f81a23d0 con 0x7f77f8106f30 2024-04-25T10:29:56.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.277+0000 7f77ea7fc640 1 -- 172.21.15.43:0/2430141264 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f40047a0 con 0x7f77f8106f30 2024-04-25T10:29:56.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.277+0000 7f77ff425640 1 -- 172.21.15.43:0/2430141264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f81a2920 con 0x7f77f8106f30 2024-04-25T10:29:56.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.278+0000 7f77ff425640 1 -- 172.21.15.43:0/2430141264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77f806c2a0 con 0x7f77f8106f30 2024-04-25T10:29:56.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.280+0000 7f77ea7fc640 1 -- 172.21.15.43:0/2430141264 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f77f4009a30 con 0x7f77f8106f30 2024-04-25T10:29:56.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.280+0000 7f77ea7fc640 1 --2- 172.21.15.43:0/2430141264 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f77e40afaf0 0x7f77e40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.281+0000 7f77fd1c1640 1 --2- 172.21.15.43:0/2430141264 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f77e40afaf0 0x7f77e40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.281+0000 7f77ea7fc640 1 -- 172.21.15.43:0/2430141264 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f77f40de8d0 con 0x7f77f8106f30 2024-04-25T10:29:56.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.281+0000 7f77fd1c1640 1 --2- 172.21.15.43:0/2430141264 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f77e40afaf0 0x7f77e40b2000 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009230 tx=0x7f77ec031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:56.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.284+0000 7f77ea7fc640 1 -- 172.21.15.43:0/2430141264 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77f40a7950 con 0x7f77f8106f30 2024-04-25T10:29:56.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.472+0000 7f77ff425640 1 -- 172.21.15.43:0/2430141264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f77f81a2be0 con 0x7f77f8106f30 2024-04-25T10:29:56.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.473+0000 7f77ea7fc640 1 -- 172.21.15.43:0/2430141264 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7f77f40a7250 con 0x7f77f8106f30 2024-04-25T10:29:56.475 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T10:29:56.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.475+0000 7f77ff425640 1 -- 172.21.15.43:0/2430141264 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f77e40afaf0 msgr2=0x7f77e40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.475+0000 7f77ff425640 1 --2- 172.21.15.43:0/2430141264 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f77e40afaf0 0x7f77e40b2000 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009230 tx=0x7f77ec031400 comp rx=0 tx=0).stop 2024-04-25T10:29:56.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.475+0000 7f77ff425640 1 -- 172.21.15.43:0/2430141264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77f8106f30 msgr2=0x7f77f819aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.475+0000 7f77ff425640 1 --2- 172.21.15.43:0/2430141264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77f8106f30 0x7f77f819aae0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f77f400d950 tx=0x7f77f400de10 comp rx=0 tx=0).stop 2024-04-25T10:29:56.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.475+0000 7f77ff425640 1 -- 172.21.15.43:0/2430141264 shutdown_connections 2024-04-25T10:29:56.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.475+0000 7f77ff425640 1 --2- 172.21.15.43:0/2430141264 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f77e40afaf0 0x7f77e40b2000 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.475+0000 7f77ff425640 1 --2- 172.21.15.43:0/2430141264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77f8109890 0x7f77f81a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.475+0000 7f77ff425640 1 --2- 172.21.15.43:0/2430141264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77f8106f30 0x7f77f819aae0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.475+0000 7f77ff425640 1 --2- 172.21.15.43:0/2430141264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77f81045d0 0x7f77f819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.475+0000 7f77ff425640 1 -- 172.21.15.43:0/2430141264 >> 172.21.15.43:0/2430141264 conn(0x7f77f80fdbd0 msgr2=0x7f77f8104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:56.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.475+0000 7f77ff425640 1 -- 172.21.15.43:0/2430141264 shutdown_connections 2024-04-25T10:29:56.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.475+0000 7f77ff425640 1 -- 172.21.15.43:0/2430141264 wait complete. 2024-04-25T10:29:56.488 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:29:56.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.598+0000 7f2143492640 1 -- 172.21.15.43:0/794249319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f213c1029c0 msgr2=0x7f213c1036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.598+0000 7f2143492640 1 --2- 172.21.15.43:0/794249319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f213c1029c0 0x7f213c1036c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2138009900 tx=0x7f213802a410 comp rx=0 tx=0).stop 2024-04-25T10:29:56.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.598+0000 7f2143492640 1 -- 172.21.15.43:0/794249319 shutdown_connections 2024-04-25T10:29:56.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.598+0000 7f2143492640 1 --2- 172.21.15.43:0/794249319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f213c108900 0x7f213c10ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.598+0000 7f2143492640 1 --2- 172.21.15.43:0/794249319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f213c103c00 0x7f213c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.598+0000 7f2143492640 1 --2- 172.21.15.43:0/794249319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f213c1029c0 0x7f213c1036c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.598+0000 7f2143492640 1 -- 172.21.15.43:0/794249319 >> 172.21.15.43:0/794249319 conn(0x7f213c0fdbd0 msgr2=0x7f213c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:56.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.599+0000 7f2143492640 1 -- 172.21.15.43:0/794249319 shutdown_connections 2024-04-25T10:29:56.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.599+0000 7f2143492640 1 -- 172.21.15.43:0/794249319 wait complete. 2024-04-25T10:29:56.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.599+0000 7f2143492640 1 Processor -- start 2024-04-25T10:29:56.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.599+0000 7f2143492640 1 -- start start 2024-04-25T10:29:56.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.599+0000 7f2143492640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f213c1029c0 0x7f213c19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.599+0000 7f2143492640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f213c103c00 0x7f213c19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.599+0000 7f2143492640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f213c108900 0x7f213c1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.599+0000 7f2143492640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f213c10e4c0 con 0x7f213c103c00 2024-04-25T10:29:56.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.599+0000 7f2143492640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f213c10e340 con 0x7f213c108900 2024-04-25T10:29:56.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.599+0000 7f2143492640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f213c10e640 con 0x7f213c1029c0 2024-04-25T10:29:56.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.600+0000 7f2140a2d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f213c103c00 0x7f213c19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.600+0000 7f2140a2d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f213c103c00 0x7f213c19a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44608/0 (socket says 172.21.15.43:44608) 2024-04-25T10:29:56.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.600+0000 7f2140a2d640 1 -- 172.21.15.43:0/2737015550 learned_addr learned my addr 172.21.15.43:0/2737015550 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:29:56.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.600+0000 7f2141a2f640 1 --2- 172.21.15.43:0/2737015550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f213c108900 0x7f213c1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.600+0000 7f214122e640 1 --2- 172.21.15.43:0/2737015550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f213c1029c0 0x7f213c19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.600+0000 7f2140a2d640 1 -- 172.21.15.43:0/2737015550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f213c1029c0 msgr2=0x7f213c19a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.600+0000 7f2140a2d640 1 --2- 172.21.15.43:0/2737015550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f213c1029c0 0x7f213c19a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.600+0000 7f2140a2d640 1 -- 172.21.15.43:0/2737015550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f213c108900 msgr2=0x7f213c1a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.600+0000 7f2140a2d640 1 --2- 172.21.15.43:0/2737015550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f213c108900 0x7f213c1a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.600+0000 7f2140a2d640 1 -- 172.21.15.43:0/2737015550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213c1a1da0 con 0x7f213c103c00 2024-04-25T10:29:56.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.600+0000 7f214122e640 1 --2- 172.21.15.43:0/2737015550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f213c1029c0 0x7f213c19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:56.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.600+0000 7f2140a2d640 1 --2- 172.21.15.43:0/2737015550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f213c103c00 0x7f213c19a810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f212c00e9f0 tx=0x7f212c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:56.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.600+0000 7f21327fc640 1 -- 172.21.15.43:0/2737015550 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c00cd80 con 0x7f213c103c00 2024-04-25T10:29:56.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.600+0000 7f21327fc640 1 -- 172.21.15.43:0/2737015550 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f212c0054d0 con 0x7f213c103c00 2024-04-25T10:29:56.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.600+0000 7f21327fc640 1 -- 172.21.15.43:0/2737015550 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c010690 con 0x7f213c103c00 2024-04-25T10:29:56.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.600+0000 7f2143492640 1 -- 172.21.15.43:0/2737015550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f213c1a2090 con 0x7f213c103c00 2024-04-25T10:29:56.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.600+0000 7f2143492640 1 -- 172.21.15.43:0/2737015550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f213c1a2590 con 0x7f213c103c00 2024-04-25T10:29:56.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.601+0000 7f2143492640 1 -- 172.21.15.43:0/2737015550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f213c1a2850 con 0x7f213c103c00 2024-04-25T10:29:56.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.603+0000 7f21327fc640 1 -- 172.21.15.43:0/2737015550 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f212c004e30 con 0x7f213c103c00 2024-04-25T10:29:56.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.604+0000 7f21327fc640 1 --2- 172.21.15.43:0/2737015550 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f21280afb70 0x7f21280b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.605+0000 7f214122e640 1 --2- 172.21.15.43:0/2737015550 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f21280afb70 0x7f21280b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.605+0000 7f21327fc640 1 -- 172.21.15.43:0/2737015550 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f212c0dedb0 con 0x7f213c103c00 2024-04-25T10:29:56.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.605+0000 7f214122e640 1 --2- 172.21.15.43:0/2737015550 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f21280afb70 0x7f21280b2080 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2138009230 tx=0x7f2138000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:56.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.608+0000 7f21327fc640 1 -- 172.21.15.43:0/2737015550 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f212c053ff0 con 0x7f213c103c00 2024-04-25T10:29:56.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.836+0000 7f2143492640 1 -- 172.21.15.43:0/2737015550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f213c1a2a20 con 0x7f213c103c00 2024-04-25T10:29:56.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.836+0000 7f21327fc640 1 -- 172.21.15.43:0/2737015550 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f212c0a7e30 con 0x7f213c103c00 2024-04-25T10:29:56.838 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:29:56.839 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T10:29:56.839 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 1 2024-04-25T10:29:56.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.838+0000 7f2143492640 1 -- 172.21.15.43:0/2737015550 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f21280afb70 msgr2=0x7f21280b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.838+0000 7f2143492640 1 --2- 172.21.15.43:0/2737015550 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f21280afb70 0x7f21280b2080 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2138009230 tx=0x7f2138000bc0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.838+0000 7f2143492640 1 -- 172.21.15.43:0/2737015550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f213c103c00 msgr2=0x7f213c19a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.838+0000 7f2143492640 1 --2- 172.21.15.43:0/2737015550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f213c103c00 0x7f213c19a810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f212c00e9f0 tx=0x7f212c00edf0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.838+0000 7f2143492640 1 -- 172.21.15.43:0/2737015550 shutdown_connections 2024-04-25T10:29:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.838+0000 7f2143492640 1 --2- 172.21.15.43:0/2737015550 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f21280afb70 0x7f21280b2080 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.838+0000 7f2143492640 1 --2- 172.21.15.43:0/2737015550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f213c108900 0x7f213c1a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.838+0000 7f2143492640 1 --2- 172.21.15.43:0/2737015550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f213c103c00 0x7f213c19a810 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.838+0000 7f2143492640 1 --2- 172.21.15.43:0/2737015550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f213c1029c0 0x7f213c19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.838+0000 7f2143492640 1 -- 172.21.15.43:0/2737015550 >> 172.21.15.43:0/2737015550 conn(0x7f213c0fdbd0 msgr2=0x7f213c109e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.838+0000 7f2143492640 1 -- 172.21.15.43:0/2737015550 shutdown_connections 2024-04-25T10:29:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:56.838+0000 7f2143492640 1 -- 172.21.15.43:0/2737015550 wait complete. 2024-04-25T10:29:57.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:56 smithi077 ceph-mon[32772]: Deploying daemon mds.l on smithi164 2024-04-25T10:29:57.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:56 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2575177318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:29:57.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:56 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2430141264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:29:57.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:56 smithi164 ceph-mon[32713]: Deploying daemon mds.l on smithi164 2024-04-25T10:29:57.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:56 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2575177318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:29:57.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:56 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2430141264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:29:57.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:56 smithi043 ceph-mon[29478]: Deploying daemon mds.l on smithi164 2024-04-25T10:29:57.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:56 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2575177318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:29:57.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:56 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2430141264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:29:57.541 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 25 10:29:57 smithi164 systemd[1]: Starting Ceph mds.l for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:29:57.801 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 25 10:29:57 smithi164 podman[63345]: 2024-04-25 10:29:57.547591842 +0000 UTC m=+0.105231144 container create a0a5434eda6a89ac170a570cf38cf651e54099aefa204f716941475d537c1828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-l, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2) 2024-04-25T10:29:57.801 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 25 10:29:57 smithi164 podman[63345]: 2024-04-25 10:29:57.460521156 +0000 UTC m=+0.018160478 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:29:57.801 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 25 10:29:57 smithi164 podman[63345]: 2024-04-25 10:29:57.733276878 +0000 UTC m=+0.290916180 container init a0a5434eda6a89ac170a570cf38cf651e54099aefa204f716941475d537c1828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-25T10:29:57.802 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 25 10:29:57 smithi164 podman[63345]: 2024-04-25 10:29:57.736011884 +0000 UTC m=+0.293651189 container start a0a5434eda6a89ac170a570cf38cf651e54099aefa204f716941475d537c1828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T10:29:57.802 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 25 10:29:57 smithi164 ceph-mds[63364]: -- 172.21.15.164:0/1966148178 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a9306fa40 con 0x559a930b7600 2024-04-25T10:29:57.802 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 25 10:29:57 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-l[63360]: starting mds.l at 2024-04-25T10:29:57.802 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 25 10:29:57 smithi164 bash[63345]: a0a5434eda6a89ac170a570cf38cf651e54099aefa204f716941475d537c1828 2024-04-25T10:29:58.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:57 smithi043 ceph-mon[29478]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:29:58.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:57 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2737015550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:29:58.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:57 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.164:6832/3601949896,v1:172.21.15.164:6833/3601949896] up:boot 2024-04-25T10:29:58.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:57 smithi043 ceph-mon[29478]: fsmap 1 up:standby 2024-04-25T10:29:58.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:57 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T10:29:58.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:57 smithi077 ceph-mon[32772]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:29:58.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:57 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2737015550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:29:58.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:57 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.164:6832/3601949896,v1:172.21.15.164:6833/3601949896] up:boot 2024-04-25T10:29:58.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:57 smithi077 ceph-mon[32772]: fsmap 1 up:standby 2024-04-25T10:29:58.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:57 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T10:29:58.291 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 25 10:29:57 smithi164 systemd[1]: Started Ceph mds.l for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:29:58.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:57 smithi164 ceph-mon[32713]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:29:58.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:57 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2737015550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:29:58.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:57 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.164:6832/3601949896,v1:172.21.15.164:6833/3601949896] up:boot 2024-04-25T10:29:58.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:57 smithi164 ceph-mon[32713]: fsmap 1 up:standby 2024-04-25T10:29:58.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:57 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T10:29:58.852 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:29:58.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.976+0000 7fc540e51640 1 -- 172.21.15.43:0/3935144633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c108900 msgr2=0x7fc53c10ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.976+0000 7fc540e51640 1 --2- 172.21.15.43:0/3935144633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c108900 0x7fc53c10ad20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009ca0 tx=0x7fc52c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.976+0000 7fc540e51640 1 -- 172.21.15.43:0/3935144633 shutdown_connections 2024-04-25T10:29:58.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.976+0000 7fc540e51640 1 --2- 172.21.15.43:0/3935144633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c108900 0x7fc53c10ad20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.976+0000 7fc540e51640 1 --2- 172.21.15.43:0/3935144633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc53c103c00 0x7fc53c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.976+0000 7fc540e51640 1 --2- 172.21.15.43:0/3935144633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc53c1029c0 0x7fc53c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.976+0000 7fc540e51640 1 -- 172.21.15.43:0/3935144633 >> 172.21.15.43:0/3935144633 conn(0x7fc53c0fdbd0 msgr2=0x7fc53c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:58.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.976+0000 7fc540e51640 1 -- 172.21.15.43:0/3935144633 shutdown_connections 2024-04-25T10:29:58.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.976+0000 7fc540e51640 1 -- 172.21.15.43:0/3935144633 wait complete. 2024-04-25T10:29:58.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.977+0000 7fc540e51640 1 Processor -- start 2024-04-25T10:29:58.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.977+0000 7fc540e51640 1 -- start start 2024-04-25T10:29:58.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.977+0000 7fc540e51640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc53c1029c0 0x7fc53c19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.977+0000 7fc540e51640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc53c103c00 0x7fc53c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.977+0000 7fc540e51640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c108900 0x7fc53c1a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.977+0000 7fc540e51640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c10f5f0 con 0x7fc53c108900 2024-04-25T10:29:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.977+0000 7fc540e51640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c10f470 con 0x7fc53c1029c0 2024-04-25T10:29:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.977+0000 7fc540e51640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c10f770 con 0x7fc53c103c00 2024-04-25T10:29:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.977+0000 7fc53ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c108900 0x7fc53c1a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.977+0000 7fc53ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c108900 0x7fc53c1a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38774/0 (socket says 172.21.15.43:38774) 2024-04-25T10:29:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.977+0000 7fc53ad9d640 1 -- 172.21.15.43:0/552286002 learned_addr learned my addr 172.21.15.43:0/552286002 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:29:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.977+0000 7fc539d9b640 1 --2- 172.21.15.43:0/552286002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc53c103c00 0x7fc53c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.977+0000 7fc53a59c640 1 --2- 172.21.15.43:0/552286002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc53c1029c0 0x7fc53c19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.978+0000 7fc53ad9d640 1 -- 172.21.15.43:0/552286002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc53c103c00 msgr2=0x7fc53c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.978+0000 7fc53ad9d640 1 --2- 172.21.15.43:0/552286002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc53c103c00 0x7fc53c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.978+0000 7fc53ad9d640 1 -- 172.21.15.43:0/552286002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc53c1029c0 msgr2=0x7fc53c19a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.978+0000 7fc53ad9d640 1 --2- 172.21.15.43:0/552286002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc53c1029c0 0x7fc53c19a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.978+0000 7fc53ad9d640 1 -- 172.21.15.43:0/552286002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53c1a1d00 con 0x7fc53c108900 2024-04-25T10:29:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.978+0000 7fc53a59c640 1 --2- 172.21.15.43:0/552286002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc53c1029c0 0x7fc53c19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:58.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.978+0000 7fc53ad9d640 1 --2- 172.21.15.43:0/552286002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c108900 0x7fc53c1a1690 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc52c0097b0 tx=0x7fc52c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:58.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.978+0000 7fc52b7fe640 1 -- 172.21.15.43:0/552286002 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c004240 con 0x7fc53c108900 2024-04-25T10:29:58.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.978+0000 7fc52b7fe640 1 -- 172.21.15.43:0/552286002 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc52c02f380 con 0x7fc53c108900 2024-04-25T10:29:58.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.978+0000 7fc540e51640 1 -- 172.21.15.43:0/552286002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53c1a1f90 con 0x7fc53c108900 2024-04-25T10:29:58.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.978+0000 7fc52b7fe640 1 -- 172.21.15.43:0/552286002 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c02fc30 con 0x7fc53c108900 2024-04-25T10:29:58.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.978+0000 7fc540e51640 1 -- 172.21.15.43:0/552286002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53c1a23f0 con 0x7fc53c108900 2024-04-25T10:29:58.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.979+0000 7fc540e51640 1 -- 172.21.15.43:0/552286002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc53c06afa0 con 0x7fc53c108900 2024-04-25T10:29:58.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.981+0000 7fc52b7fe640 1 -- 172.21.15.43:0/552286002 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fc52c0043e0 con 0x7fc53c108900 2024-04-25T10:29:58.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.981+0000 7fc52b7fe640 1 --2- 172.21.15.43:0/552286002 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc5240afaf0 0x7fc5240b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.981+0000 7fc52b7fe640 1 -- 172.21.15.43:0/552286002 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fc52c100080 con 0x7fc53c108900 2024-04-25T10:29:58.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.981+0000 7fc53a59c640 1 --2- 172.21.15.43:0/552286002 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc5240afaf0 0x7fc5240b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.982+0000 7fc53a59c640 1 --2- 172.21.15.43:0/552286002 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc5240afaf0 0x7fc5240b2000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc534005d00 tx=0x7fc534005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:58.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:58.985+0000 7fc52b7fe640 1 -- 172.21.15.43:0/552286002 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc52c0c9100 con 0x7fc53c108900 2024-04-25T10:29:59.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:58 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:59.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:58 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:59.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:58 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:59.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:58 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:29:59.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:58 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:29:59.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:58 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:59.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:58 smithi043 ceph-mon[29478]: Deploying daemon mds.k on smithi077 2024-04-25T10:29:59.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:59.211+0000 7fc540e51640 1 -- 172.21.15.43:0/552286002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc53c107710 con 0x7fc53c108900 2024-04-25T10:29:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:59.212+0000 7fc52b7fe640 1 -- 172.21.15.43:0/552286002 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fc53c107710 con 0x7fc53c108900 2024-04-25T10:29:59.214 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:29:59.214 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T10:29:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 2 2024-04-25T10:29:59.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:59.214+0000 7fc540e51640 1 -- 172.21.15.43:0/552286002 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc5240afaf0 msgr2=0x7fc5240b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:59.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:59.214+0000 7fc540e51640 1 --2- 172.21.15.43:0/552286002 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc5240afaf0 0x7fc5240b2000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc534005d00 tx=0x7fc534005c90 comp rx=0 tx=0).stop 2024-04-25T10:29:59.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:59.214+0000 7fc540e51640 1 -- 172.21.15.43:0/552286002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c108900 msgr2=0x7fc53c1a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:59.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:59.214+0000 7fc540e51640 1 --2- 172.21.15.43:0/552286002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c108900 0x7fc53c1a1690 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc52c0097b0 tx=0x7fc52c003c20 comp rx=0 tx=0).stop 2024-04-25T10:29:59.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:59.214+0000 7fc540e51640 1 -- 172.21.15.43:0/552286002 shutdown_connections 2024-04-25T10:29:59.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:59.214+0000 7fc540e51640 1 --2- 172.21.15.43:0/552286002 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc5240afaf0 0x7fc5240b2000 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:59.214+0000 7fc540e51640 1 --2- 172.21.15.43:0/552286002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c108900 0x7fc53c1a1690 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:59.214+0000 7fc540e51640 1 --2- 172.21.15.43:0/552286002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc53c103c00 0x7fc53c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:59.214+0000 7fc540e51640 1 --2- 172.21.15.43:0/552286002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc53c1029c0 0x7fc53c19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:59.214+0000 7fc540e51640 1 -- 172.21.15.43:0/552286002 >> 172.21.15.43:0/552286002 conn(0x7fc53c0fdbd0 msgr2=0x7fc53c1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:59.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:59.214+0000 7fc540e51640 1 -- 172.21.15.43:0/552286002 shutdown_connections 2024-04-25T10:29:59.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:29:59.214+0000 7fc540e51640 1 -- 172.21.15.43:0/552286002 wait complete. 2024-04-25T10:29:59.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:58 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:59.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:58 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:59.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:58 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:59.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:58 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:29:59.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:58 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:29:59.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:58 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:59.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:58 smithi077 ceph-mon[32772]: Deploying daemon mds.k on smithi077 2024-04-25T10:29:59.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:58 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:59.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:58 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:59.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:58 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:29:59.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:58 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:29:59.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:58 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:29:59.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:58 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:29:59.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:58 smithi164 ceph-mon[32713]: Deploying daemon mds.k on smithi077 2024-04-25T10:30:00.192 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 25 10:29:59 smithi077 systemd[1]: Starting Ceph mds.k for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:30:00.193 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:59 smithi077 ceph-mon[32772]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:00.193 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:29:59 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/552286002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:00.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:59 smithi164 ceph-mon[32713]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:00.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:29:59 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/552286002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:00.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:59 smithi043 ceph-mon[29478]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:00.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:29:59 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/552286002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:00.540 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 25 10:30:00 smithi077 podman[62719]: 2024-04-25 10:30:00.093497927 +0000 UTC m=+0.017659269 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:30:00.541 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 25 10:30:00 smithi077 podman[62719]: 2024-04-25 10:30:00.206035431 +0000 UTC m=+0.130196756 container create cdd9460375e8d24c455194ff6ea69e4ba2b0f1bdf65df6cc4ed7a65ecda4e4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T10:30:00.888 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 25 10:30:00 smithi077 podman[62719]: 2024-04-25 10:30:00.583684541 +0000 UTC m=+0.507845867 container init cdd9460375e8d24c455194ff6ea69e4ba2b0f1bdf65df6cc4ed7a65ecda4e4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T10:30:00.888 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 25 10:30:00 smithi077 podman[62719]: 2024-04-25 10:30:00.586470336 +0000 UTC m=+0.510631662 container start cdd9460375e8d24c455194ff6ea69e4ba2b0f1bdf65df6cc4ed7a65ecda4e4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T10:30:00.888 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 25 10:30:00 smithi077 ceph-mds[62739]: -- 172.21.15.77:0/151718817 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565124077a40 con 0x5651240bf600 2024-04-25T10:30:00.888 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 25 10:30:00 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-k[62735]: starting mds.k at 2024-04-25T10:30:00.888 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 25 10:30:00 smithi077 bash[62719]: cdd9460375e8d24c455194ff6ea69e4ba2b0f1bdf65df6cc4ed7a65ecda4e4f7 2024-04-25T10:30:00.888 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 25 10:30:00 smithi077 systemd[1]: Started Ceph mds.k for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:30:01.228 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:01.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:00 smithi043 ceph-mon[29478]: overall HEALTH_OK 2024-04-25T10:30:01.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:00 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:01.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:00 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:01.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:00 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:01.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:00 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:01.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:00 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:01.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:00 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:01.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:00 smithi077 ceph-mon[32772]: overall HEALTH_OK 2024-04-25T10:30:01.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:00 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:01.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:00 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:01.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:00 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:01.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:00 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:01.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:00 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:01.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:00 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:01.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:00 smithi164 ceph-mon[32713]: overall HEALTH_OK 2024-04-25T10:30:01.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:00 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:01.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:00 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:01.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:00 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:01.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:00 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:01.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:00 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:01.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:00 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:01.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.353+0000 7feea8384640 1 -- 172.21.15.43:0/2098733769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea0105090 msgr2=0x7feea01054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:01.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.353+0000 7feea8384640 1 --2- 172.21.15.43:0/2098733769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea0105090 0x7feea01054a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009900 tx=0x7fee9c02a410 comp rx=0 tx=0).stop 2024-04-25T10:30:01.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.353+0000 7feea8384640 1 -- 172.21.15.43:0/2098733769 shutdown_connections 2024-04-25T10:30:01.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.353+0000 7feea8384640 1 --2- 172.21.15.43:0/2098733769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feea0101740 0x7feea0101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.353+0000 7feea8384640 1 --2- 172.21.15.43:0/2098733769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feea0102240 0x7feea010bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.353+0000 7feea8384640 1 --2- 172.21.15.43:0/2098733769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea0105090 0x7feea01054a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.353+0000 7feea8384640 1 -- 172.21.15.43:0/2098733769 >> 172.21.15.43:0/2098733769 conn(0x7feea00fd340 msgr2=0x7feea00ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.353+0000 7feea8384640 1 -- 172.21.15.43:0/2098733769 shutdown_connections 2024-04-25T10:30:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.353+0000 7feea8384640 1 -- 172.21.15.43:0/2098733769 wait complete. 2024-04-25T10:30:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.353+0000 7feea8384640 1 Processor -- start 2024-04-25T10:30:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.353+0000 7feea8384640 1 -- start start 2024-04-25T10:30:01.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7feea8384640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea0101740 0x7feea0104240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7feea8384640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feea0102240 0x7feea01028e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7feea8384640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feea0105090 0x7feea0102e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7feea8384640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feea010f660 con 0x7feea0101740 2024-04-25T10:30:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7feea8384640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feea010f4e0 con 0x7feea0102240 2024-04-25T10:30:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7feea8384640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feea010f7e0 con 0x7feea0105090 2024-04-25T10:30:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7feea6120640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea0101740 0x7feea0104240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7feea6120640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea0101740 0x7feea0104240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38794/0 (socket says 172.21.15.43:38794) 2024-04-25T10:30:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7feea6120640 1 -- 172.21.15.43:0/2098427157 learned_addr learned my addr 172.21.15.43:0/2098427157 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7feea6921640 1 --2- 172.21.15.43:0/2098427157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feea0105090 0x7feea0102e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7feea6120640 1 -- 172.21.15.43:0/2098427157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feea0105090 msgr2=0x7feea0102e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7feea591f640 1 --2- 172.21.15.43:0/2098427157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feea0102240 0x7feea01028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7feea6120640 1 --2- 172.21.15.43:0/2098427157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feea0105090 0x7feea0102e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7feea6120640 1 -- 172.21.15.43:0/2098427157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feea0102240 msgr2=0x7feea01028e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7feea6120640 1 --2- 172.21.15.43:0/2098427157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feea0102240 0x7feea01028e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7feea6120640 1 -- 172.21.15.43:0/2098427157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea0103550 con 0x7feea0101740 2024-04-25T10:30:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7feea6120640 1 --2- 172.21.15.43:0/2098427157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea0101740 0x7feea0104240 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fee9c006010 tx=0x7fee9c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7fee977fe640 1 -- 172.21.15.43:0/2098427157 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c03f030 con 0x7feea0101740 2024-04-25T10:30:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7fee977fe640 1 -- 172.21.15.43:0/2098427157 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee9c003aa0 con 0x7feea0101740 2024-04-25T10:30:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7feea8384640 1 -- 172.21.15.43:0/2098427157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea01a4300 con 0x7feea0101740 2024-04-25T10:30:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7fee977fe640 1 -- 172.21.15.43:0/2098427157 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c035070 con 0x7feea0101740 2024-04-25T10:30:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.354+0000 7feea8384640 1 -- 172.21.15.43:0/2098427157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea01a4760 con 0x7feea0101740 2024-04-25T10:30:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.356+0000 7feea8384640 1 -- 172.21.15.43:0/2098427157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea0108a20 con 0x7feea0101740 2024-04-25T10:30:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.357+0000 7fee977fe640 1 -- 172.21.15.43:0/2098427157 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fee9c042030 con 0x7feea0101740 2024-04-25T10:30:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.358+0000 7fee977fe640 1 --2- 172.21.15.43:0/2098427157 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fee8c0afb70 0x7fee8c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.358+0000 7feea591f640 1 --2- 172.21.15.43:0/2098427157 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fee8c0afb70 0x7fee8c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.358+0000 7fee977fe640 1 -- 172.21.15.43:0/2098427157 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fee9c0ff260 con 0x7feea0101740 2024-04-25T10:30:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.358+0000 7feea591f640 1 --2- 172.21.15.43:0/2098427157 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fee8c0afb70 0x7fee8c0b2080 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fee90005fd0 tx=0x7fee90005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:01.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.362+0000 7fee977fe640 1 -- 172.21.15.43:0/2098427157 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fee9c0c8240 con 0x7feea0101740 2024-04-25T10:30:01.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.589+0000 7feea8384640 1 -- 172.21.15.43:0/2098427157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feea002db50 con 0x7feea0101740 2024-04-25T10:30:01.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.589+0000 7fee977fe640 1 -- 172.21.15.43:0/2098427157 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fee9c0d02c0 con 0x7feea0101740 2024-04-25T10:30:01.591 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:01.592 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T10:30:01.592 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 3 2024-04-25T10:30:01.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.591+0000 7feea8384640 1 -- 172.21.15.43:0/2098427157 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fee8c0afb70 msgr2=0x7fee8c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:01.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.591+0000 7feea8384640 1 --2- 172.21.15.43:0/2098427157 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fee8c0afb70 0x7fee8c0b2080 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fee90005fd0 tx=0x7fee90005c70 comp rx=0 tx=0).stop 2024-04-25T10:30:01.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.591+0000 7feea8384640 1 -- 172.21.15.43:0/2098427157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea0101740 msgr2=0x7feea0104240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:01.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.591+0000 7feea8384640 1 --2- 172.21.15.43:0/2098427157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea0101740 0x7feea0104240 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fee9c006010 tx=0x7fee9c031a50 comp rx=0 tx=0).stop 2024-04-25T10:30:01.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.591+0000 7feea8384640 1 -- 172.21.15.43:0/2098427157 shutdown_connections 2024-04-25T10:30:01.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.591+0000 7feea8384640 1 --2- 172.21.15.43:0/2098427157 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fee8c0afb70 0x7fee8c0b2080 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.591+0000 7feea8384640 1 --2- 172.21.15.43:0/2098427157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feea0102240 0x7feea01028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.591+0000 7feea8384640 1 --2- 172.21.15.43:0/2098427157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feea0105090 0x7feea0102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.591+0000 7feea8384640 1 --2- 172.21.15.43:0/2098427157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea0101740 0x7feea0104240 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.591+0000 7feea8384640 1 -- 172.21.15.43:0/2098427157 >> 172.21.15.43:0/2098427157 conn(0x7feea00fd340 msgr2=0x7feea0105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:01.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.592+0000 7feea8384640 1 -- 172.21.15.43:0/2098427157 shutdown_connections 2024-04-25T10:30:01.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:01.592+0000 7feea8384640 1 -- 172.21.15.43:0/2098427157 wait complete. 2024-04-25T10:30:02.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:01 smithi077 ceph-mon[32772]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:02.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:01 smithi077 ceph-mon[32772]: Deploying daemon mds.j on smithi043 2024-04-25T10:30:02.004 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:01 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.77:6832/2295020829,v1:172.21.15.77:6833/2295020829] up:boot 2024-04-25T10:30:02.004 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:01 smithi077 ceph-mon[32772]: fsmap 2 up:standby 2024-04-25T10:30:02.004 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:01 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T10:30:02.004 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:01 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2098427157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:02.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:01 smithi164 ceph-mon[32713]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:02.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:01 smithi164 ceph-mon[32713]: Deploying daemon mds.j on smithi043 2024-04-25T10:30:02.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:01 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.77:6832/2295020829,v1:172.21.15.77:6833/2295020829] up:boot 2024-04-25T10:30:02.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:01 smithi164 ceph-mon[32713]: fsmap 2 up:standby 2024-04-25T10:30:02.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:01 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T10:30:02.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:01 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2098427157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:02.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:01 smithi043 ceph-mon[29478]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:02.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:01 smithi043 ceph-mon[29478]: Deploying daemon mds.j on smithi043 2024-04-25T10:30:02.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:01 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.77:6832/2295020829,v1:172.21.15.77:6833/2295020829] up:boot 2024-04-25T10:30:02.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:01 smithi043 ceph-mon[29478]: fsmap 2 up:standby 2024-04-25T10:30:02.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:01 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T10:30:02.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:01 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2098427157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:03.110 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 25 10:30:02 smithi043 systemd[1]: Starting Ceph mds.j for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:30:03.432 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 25 10:30:03 smithi043 podman[67535]: 2024-04-25 10:30:03.049260069 +0000 UTC m=+0.018847438 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:30:03.432 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 25 10:30:03 smithi043 podman[67535]: 2024-04-25 10:30:03.151435487 +0000 UTC m=+0.121022898 container create 83b2057bdc494b83729a2d98aac706a32c369db1b8a0c9277db72283bc4f1982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T10:30:03.432 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 25 10:30:03 smithi043 podman[67535]: 2024-04-25 10:30:03.355805239 +0000 UTC m=+0.325392609 container init 83b2057bdc494b83729a2d98aac706a32c369db1b8a0c9277db72283bc4f1982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True) 2024-04-25T10:30:03.432 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 25 10:30:03 smithi043 podman[67535]: 2024-04-25 10:30:03.358502671 +0000 UTC m=+0.328090043 container start 83b2057bdc494b83729a2d98aac706a32c369db1b8a0c9277db72283bc4f1982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-j, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424) 2024-04-25T10:30:03.432 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 25 10:30:03 smithi043 ceph-mds[67554]: -- 172.21.15.43:0/4101614058 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b9764e7a40 con 0x55b97652fa80 2024-04-25T10:30:03.432 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 25 10:30:03 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-j[67550]: starting mds.j at 2024-04-25T10:30:03.433 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 25 10:30:03 smithi043 bash[67535]: 83b2057bdc494b83729a2d98aac706a32c369db1b8a0c9277db72283bc4f1982 2024-04-25T10:30:03.605 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:03.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.720+0000 7fc6fdd5e640 1 -- 172.21.15.43:0/1495064688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f8103ad0 msgr2=0x7fc6f81047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:03.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.720+0000 7fc6fdd5e640 1 --2- 172.21.15.43:0/1495064688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f8103ad0 0x7fc6f81047d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec009900 tx=0x7fc6ec02a3f0 comp rx=0 tx=0).stop 2024-04-25T10:30:03.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.720+0000 7fc6fdd5e640 1 -- 172.21.15.43:0/1495064688 shutdown_connections 2024-04-25T10:30:03.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.720+0000 7fc6fdd5e640 1 --2- 172.21.15.43:0/1495064688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6f8104d10 0x7fc6f8105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:03.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.720+0000 7fc6fdd5e640 1 --2- 172.21.15.43:0/1495064688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6f8109a10 0x7fc6f810be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:03.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.720+0000 7fc6fdd5e640 1 --2- 172.21.15.43:0/1495064688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f8103ad0 0x7fc6f81047d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:03.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.720+0000 7fc6fdd5e640 1 -- 172.21.15.43:0/1495064688 >> 172.21.15.43:0/1495064688 conn(0x7fc6f80fecc0 msgr2=0x7fc6f8101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:03.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.720+0000 7fc6fdd5e640 1 -- 172.21.15.43:0/1495064688 shutdown_connections 2024-04-25T10:30:03.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.720+0000 7fc6fdd5e640 1 -- 172.21.15.43:0/1495064688 wait complete. 2024-04-25T10:30:03.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.721+0000 7fc6fdd5e640 1 Processor -- start 2024-04-25T10:30:03.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.721+0000 7fc6fdd5e640 1 -- start start 2024-04-25T10:30:03.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.721+0000 7fc6fdd5e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6f8103ad0 0x7fc6f8076800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:03.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.721+0000 7fc6fdd5e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6f8104d10 0x7fc6f8074ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:03.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.721+0000 7fc6fdd5e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f8109a10 0x7fc6f8075430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:03.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.721+0000 7fc6fdd5e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f810f5b0 con 0x7fc6f8109a10 2024-04-25T10:30:03.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.721+0000 7fc6fdd5e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f810f430 con 0x7fc6f8104d10 2024-04-25T10:30:03.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.721+0000 7fc6fdd5e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f810f730 con 0x7fc6f8103ad0 2024-04-25T10:30:03.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.722+0000 7fc6f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f8109a10 0x7fc6f8075430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:03.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.722+0000 7fc6f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f8109a10 0x7fc6f8075430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38834/0 (socket says 172.21.15.43:38834) 2024-04-25T10:30:03.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.722+0000 7fc6f6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6f8104d10 0x7fc6f8074ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:03.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.722+0000 7fc6f7fff640 1 -- 172.21.15.43:0/2124762352 learned_addr learned my addr 172.21.15.43:0/2124762352 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:03.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.722+0000 7fc6f77fe640 1 --2- 172.21.15.43:0/2124762352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6f8103ad0 0x7fc6f8076800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:03.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.722+0000 7fc6f6ffd640 1 -- 172.21.15.43:0/2124762352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6f8103ad0 msgr2=0x7fc6f8076800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:03.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.722+0000 7fc6f6ffd640 1 --2- 172.21.15.43:0/2124762352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6f8103ad0 0x7fc6f8076800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:03.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.722+0000 7fc6f6ffd640 1 -- 172.21.15.43:0/2124762352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f8109a10 msgr2=0x7fc6f8075430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:03.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.722+0000 7fc6f6ffd640 1 --2- 172.21.15.43:0/2124762352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f8109a10 0x7fc6f8075430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:03.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.722+0000 7fc6f6ffd640 1 -- 172.21.15.43:0/2124762352 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f8075aa0 con 0x7fc6f8104d10 2024-04-25T10:30:03.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.722+0000 7fc6f7fff640 1 --2- 172.21.15.43:0/2124762352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f8109a10 0x7fc6f8075430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:03.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.722+0000 7fc6f6ffd640 1 --2- 172.21.15.43:0/2124762352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6f8104d10 0x7fc6f8074ef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800e9f0 tx=0x7fc6e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:03.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.722+0000 7fc6f4ff9640 1 -- 172.21.15.43:0/2124762352 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e800ccc0 con 0x7fc6f8104d10 2024-04-25T10:30:03.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.722+0000 7fc6f4ff9640 1 -- 172.21.15.43:0/2124762352 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e80108e0 con 0x7fc6f8104d10 2024-04-25T10:30:03.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.722+0000 7fc6fdd5e640 1 -- 172.21.15.43:0/2124762352 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f81a4330 con 0x7fc6f8104d10 2024-04-25T10:30:03.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.722+0000 7fc6f4ff9640 1 -- 172.21.15.43:0/2124762352 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8018880 con 0x7fc6f8104d10 2024-04-25T10:30:03.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.722+0000 7fc6fdd5e640 1 -- 172.21.15.43:0/2124762352 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f81a4730 con 0x7fc6f8104d10 2024-04-25T10:30:03.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.724+0000 7fc6fdd5e640 1 -- 172.21.15.43:0/2124762352 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6f8069310 con 0x7fc6f8104d10 2024-04-25T10:30:03.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.725+0000 7fc6f4ff9640 1 -- 172.21.15.43:0/2124762352 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fc6e8010430 con 0x7fc6f8104d10 2024-04-25T10:30:03.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.725+0000 7fc6f4ff9640 1 --2- 172.21.15.43:0/2124762352 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc6e00afaf0 0x7fc6e00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:03.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.725+0000 7fc6f77fe640 1 --2- 172.21.15.43:0/2124762352 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc6e00afaf0 0x7fc6e00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.725+0000 7fc6f4ff9640 1 -- 172.21.15.43:0/2124762352 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fc6e80deb30 con 0x7fc6f8104d10 2024-04-25T10:30:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.726+0000 7fc6f77fe640 1 --2- 172.21.15.43:0/2124762352 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc6e00afaf0 0x7fc6e00b2000 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec005ab0 tx=0x7fc6ec031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.730+0000 7fc6f4ff9640 1 -- 172.21.15.43:0/2124762352 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6e80a74c0 con 0x7fc6f8104d10 2024-04-25T10:30:03.860 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 25 10:30:03 smithi043 systemd[1]: Started Ceph mds.j for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:30:03.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.960+0000 7fc6fdd5e640 1 -- 172.21.15.43:0/2124762352 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6f802de90 con 0x7fc6f8104d10 2024-04-25T10:30:03.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.960+0000 7fc6f4ff9640 1 -- 172.21.15.43:0/2124762352 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7fc6e80ab7c0 con 0x7fc6f8104d10 2024-04-25T10:30:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:03.963 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T10:30:03.963 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 4 2024-04-25T10:30:03.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.963+0000 7fc6fdd5e640 1 -- 172.21.15.43:0/2124762352 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc6e00afaf0 msgr2=0x7fc6e00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:03.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.963+0000 7fc6fdd5e640 1 --2- 172.21.15.43:0/2124762352 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc6e00afaf0 0x7fc6e00b2000 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec005ab0 tx=0x7fc6ec031400 comp rx=0 tx=0).stop 2024-04-25T10:30:03.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.963+0000 7fc6fdd5e640 1 -- 172.21.15.43:0/2124762352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6f8104d10 msgr2=0x7fc6f8074ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:03.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.963+0000 7fc6fdd5e640 1 --2- 172.21.15.43:0/2124762352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6f8104d10 0x7fc6f8074ef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800e9f0 tx=0x7fc6e800eeb0 comp rx=0 tx=0).stop 2024-04-25T10:30:03.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.963+0000 7fc6fdd5e640 1 -- 172.21.15.43:0/2124762352 shutdown_connections 2024-04-25T10:30:03.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.963+0000 7fc6fdd5e640 1 --2- 172.21.15.43:0/2124762352 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc6e00afaf0 0x7fc6e00b2000 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:03.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.963+0000 7fc6fdd5e640 1 --2- 172.21.15.43:0/2124762352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6f8104d10 0x7fc6f8074ef0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:03.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.963+0000 7fc6fdd5e640 1 --2- 172.21.15.43:0/2124762352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f8109a10 0x7fc6f8075430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:03.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.963+0000 7fc6fdd5e640 1 --2- 172.21.15.43:0/2124762352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6f8103ad0 0x7fc6f8076800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:03.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.963+0000 7fc6fdd5e640 1 -- 172.21.15.43:0/2124762352 >> 172.21.15.43:0/2124762352 conn(0x7fc6f80fecc0 msgr2=0x7fc6f81059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:03.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.963+0000 7fc6fdd5e640 1 -- 172.21.15.43:0/2124762352 shutdown_connections 2024-04-25T10:30:03.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:03.963+0000 7fc6fdd5e640 1 -- 172.21.15.43:0/2124762352 wait complete. 2024-04-25T10:30:04.183 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:03 smithi043 ceph-mon[29478]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:04.184 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:04.184 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:04.184 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:04.184 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:04.184 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:04.184 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:03 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:04.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:03 smithi077 ceph-mon[32772]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:04.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:04.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:04.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:04.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:04.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:04.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:03 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:04.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:03 smithi164 ceph-mon[32713]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:04.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:04.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:04.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:04.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:04.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:04.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:03 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:05.187 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:04 smithi043 ceph-mon[29478]: Deploying daemon mds.d on smithi043 2024-04-25T10:30:05.187 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:04 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.43:6834/3551971560,v1:172.21.15.43:6835/3551971560] up:boot 2024-04-25T10:30:05.187 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:04 smithi043 ceph-mon[29478]: fsmap 3 up:standby 2024-04-25T10:30:05.187 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:04 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T10:30:05.187 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:04 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2124762352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:05.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:04 smithi077 ceph-mon[32772]: Deploying daemon mds.d on smithi043 2024-04-25T10:30:05.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:04 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.43:6834/3551971560,v1:172.21.15.43:6835/3551971560] up:boot 2024-04-25T10:30:05.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:04 smithi077 ceph-mon[32772]: fsmap 3 up:standby 2024-04-25T10:30:05.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:04 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T10:30:05.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:04 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2124762352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:04 smithi164 ceph-mon[32713]: Deploying daemon mds.d on smithi043 2024-04-25T10:30:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:04 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.43:6834/3551971560,v1:172.21.15.43:6835/3551971560] up:boot 2024-04-25T10:30:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:04 smithi164 ceph-mon[32713]: fsmap 3 up:standby 2024-04-25T10:30:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:04 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T10:30:05.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:04 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2124762352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:05.861 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 25 10:30:05 smithi043 systemd[1]: Starting Ceph mds.d for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:30:05.977 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:06.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.095+0000 7faf57842640 1 -- 172.21.15.43:0/2413671020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf50108940 msgr2=0x7faf5010ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.095+0000 7faf57842640 1 --2- 172.21.15.43:0/2413671020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf50108940 0x7faf5010ad60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faf48009ca0 tx=0x7faf4802a8e0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.095+0000 7faf57842640 1 -- 172.21.15.43:0/2413671020 shutdown_connections 2024-04-25T10:30:06.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.095+0000 7faf57842640 1 --2- 172.21.15.43:0/2413671020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf50103c40 0x7faf50104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.095+0000 7faf57842640 1 --2- 172.21.15.43:0/2413671020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf50108940 0x7faf5010ad60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.095+0000 7faf57842640 1 --2- 172.21.15.43:0/2413671020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf50102a00 0x7faf50103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.095+0000 7faf57842640 1 -- 172.21.15.43:0/2413671020 >> 172.21.15.43:0/2413671020 conn(0x7faf500fdbf0 msgr2=0x7faf501000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:06.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.095+0000 7faf57842640 1 -- 172.21.15.43:0/2413671020 shutdown_connections 2024-04-25T10:30:06.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.095+0000 7faf57842640 1 -- 172.21.15.43:0/2413671020 wait complete. 2024-04-25T10:30:06.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.096+0000 7faf57842640 1 Processor -- start 2024-04-25T10:30:06.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.096+0000 7faf57842640 1 -- start start 2024-04-25T10:30:06.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.096+0000 7faf57842640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf50102a00 0x7faf5006bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.096+0000 7faf57842640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf50103c40 0x7faf5019f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.096+0000 7faf57842640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf50108940 0x7faf501a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.096+0000 7faf57842640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faf5010e580 con 0x7faf50103c40 2024-04-25T10:30:06.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.096+0000 7faf57842640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faf5010e400 con 0x7faf50108940 2024-04-25T10:30:06.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.096+0000 7faf57842640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf5010e700 con 0x7faf50102a00 2024-04-25T10:30:06.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.096+0000 7faf54ddd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf50103c40 0x7faf5019f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.096+0000 7faf54ddd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf50103c40 0x7faf5019f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38858/0 (socket says 172.21.15.43:38858) 2024-04-25T10:30:06.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.097+0000 7faf54ddd640 1 -- 172.21.15.43:0/2752382918 learned_addr learned my addr 172.21.15.43:0/2752382918 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:06.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.097+0000 7faf555de640 1 --2- 172.21.15.43:0/2752382918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf50102a00 0x7faf5006bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.097+0000 7faf55ddf640 1 --2- 172.21.15.43:0/2752382918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf50108940 0x7faf501a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.097+0000 7faf54ddd640 1 -- 172.21.15.43:0/2752382918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf50102a00 msgr2=0x7faf5006bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.097+0000 7faf54ddd640 1 --2- 172.21.15.43:0/2752382918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf50102a00 0x7faf5006bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.097+0000 7faf54ddd640 1 -- 172.21.15.43:0/2752382918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf50108940 msgr2=0x7faf501a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.097+0000 7faf54ddd640 1 --2- 172.21.15.43:0/2752382918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf50108940 0x7faf501a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.097+0000 7faf54ddd640 1 -- 172.21.15.43:0/2752382918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf501a1fb0 con 0x7faf50103c40 2024-04-25T10:30:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.097+0000 7faf54ddd640 1 --2- 172.21.15.43:0/2752382918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf50103c40 0x7faf5019f2d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faf4000c920 tx=0x7faf4000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.097+0000 7faf467fc640 1 -- 172.21.15.43:0/2752382918 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40007bb0 con 0x7faf50103c40 2024-04-25T10:30:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.097+0000 7faf57842640 1 -- 172.21.15.43:0/2752382918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf501a22a0 con 0x7faf50103c40 2024-04-25T10:30:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.097+0000 7faf467fc640 1 -- 172.21.15.43:0/2752382918 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf40007d50 con 0x7faf50103c40 2024-04-25T10:30:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.097+0000 7faf467fc640 1 -- 172.21.15.43:0/2752382918 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40018730 con 0x7faf50103c40 2024-04-25T10:30:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.097+0000 7faf57842640 1 -- 172.21.15.43:0/2752382918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf501a28c0 con 0x7faf50103c40 2024-04-25T10:30:06.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.099+0000 7faf57842640 1 -- 172.21.15.43:0/2752382918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf50107750 con 0x7faf50103c40 2024-04-25T10:30:06.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.100+0000 7faf467fc640 1 -- 172.21.15.43:0/2752382918 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7faf4000f450 con 0x7faf50103c40 2024-04-25T10:30:06.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.101+0000 7faf467fc640 1 --2- 172.21.15.43:0/2752382918 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faf3c0afaf0 0x7faf3c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.101+0000 7faf555de640 1 --2- 172.21.15.43:0/2752382918 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faf3c0afaf0 0x7faf3c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.101+0000 7faf467fc640 1 -- 172.21.15.43:0/2752382918 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7faf400deb90 con 0x7faf50103c40 2024-04-25T10:30:06.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.101+0000 7faf555de640 1 --2- 172.21.15.43:0/2752382918 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faf3c0afaf0 0x7faf3c0b2000 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faf50103600 tx=0x7faf4c009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.108+0000 7faf467fc640 1 -- 172.21.15.43:0/2752382918 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf40014030 con 0x7faf50103c40 2024-04-25T10:30:06.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:05 smithi043 ceph-mon[29478]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:06.153 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 25 10:30:05 smithi043 podman[67878]: 2024-04-25 10:30:05.802549333 +0000 UTC m=+0.018939806 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:30:06.153 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 25 10:30:05 smithi043 podman[67878]: 2024-04-25 10:30:05.91606201 +0000 UTC m=+0.132452450 container create 37ca5ef7abd930639eb4328429753dfd9a223a4b703e213a77595c28b189af96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T10:30:06.153 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 25 10:30:06 smithi043 podman[67878]: 2024-04-25 10:30:06.151831527 +0000 UTC m=+0.368221968 container init 37ca5ef7abd930639eb4328429753dfd9a223a4b703e213a77595c28b189af96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T10:30:06.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:05 smithi077 ceph-mon[32772]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:06.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:05 smithi164 ceph-mon[32713]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:06.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.342+0000 7faf57842640 1 -- 172.21.15.43:0/2752382918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf50069b70 con 0x7faf50103c40 2024-04-25T10:30:06.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.343+0000 7faf467fc640 1 -- 172.21.15.43:0/2752382918 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7faf400a7c00 con 0x7faf50103c40 2024-04-25T10:30:06.345 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:06.345 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T10:30:06.345 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 4 2024-04-25T10:30:06.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.345+0000 7faf57842640 1 -- 172.21.15.43:0/2752382918 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faf3c0afaf0 msgr2=0x7faf3c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.345+0000 7faf57842640 1 --2- 172.21.15.43:0/2752382918 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faf3c0afaf0 0x7faf3c0b2000 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faf50103600 tx=0x7faf4c009110 comp rx=0 tx=0).stop 2024-04-25T10:30:06.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.345+0000 7faf57842640 1 -- 172.21.15.43:0/2752382918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf50103c40 msgr2=0x7faf5019f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.345+0000 7faf57842640 1 --2- 172.21.15.43:0/2752382918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf50103c40 0x7faf5019f2d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faf4000c920 tx=0x7faf4000cd20 comp rx=0 tx=0).stop 2024-04-25T10:30:06.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.345+0000 7faf57842640 1 -- 172.21.15.43:0/2752382918 shutdown_connections 2024-04-25T10:30:06.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.345+0000 7faf57842640 1 --2- 172.21.15.43:0/2752382918 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faf3c0afaf0 0x7faf3c0b2000 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.345+0000 7faf57842640 1 --2- 172.21.15.43:0/2752382918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf50108940 0x7faf501a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.345+0000 7faf57842640 1 --2- 172.21.15.43:0/2752382918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf50103c40 0x7faf5019f2d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.345+0000 7faf57842640 1 --2- 172.21.15.43:0/2752382918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf50102a00 0x7faf5006bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.345+0000 7faf57842640 1 -- 172.21.15.43:0/2752382918 >> 172.21.15.43:0/2752382918 conn(0x7faf500fdbf0 msgr2=0x7faf50104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:06.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.345+0000 7faf57842640 1 -- 172.21.15.43:0/2752382918 shutdown_connections 2024-04-25T10:30:06.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:06.345+0000 7faf57842640 1 -- 172.21.15.43:0/2752382918 wait complete. 2024-04-25T10:30:06.610 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 25 10:30:06 smithi043 podman[67878]: 2024-04-25 10:30:06.155176117 +0000 UTC m=+0.371566561 container start 37ca5ef7abd930639eb4328429753dfd9a223a4b703e213a77595c28b189af96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T10:30:06.611 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 25 10:30:06 smithi043 ceph-mds[67936]: -- 172.21.15.43:0/2263014605 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559b8446ba40 con 0x559b844b3180 2024-04-25T10:30:06.611 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 25 10:30:06 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-d[67913]: starting mds.d at 2024-04-25T10:30:06.611 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 25 10:30:06 smithi043 bash[67878]: 37ca5ef7abd930639eb4328429753dfd9a223a4b703e213a77595c28b189af96 2024-04-25T10:30:06.611 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 25 10:30:06 smithi043 systemd[1]: Started Ceph mds.d for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:30:07.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:07 smithi077 ceph-mon[32772]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:07.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:07 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:07.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:07 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:07.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:07 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:07.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:07 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:07.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:07 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:07.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:07 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:07.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:07 smithi077 ceph-mon[32772]: Deploying daemon mds.e on smithi077 2024-04-25T10:30:07.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:07 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2752382918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:07.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:07 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.43:6836/2500994561,v1:172.21.15.43:6837/2500994561] up:boot 2024-04-25T10:30:07.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:07 smithi077 ceph-mon[32772]: fsmap 4 up:standby 2024-04-25T10:30:07.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:07 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T10:30:07.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:07 smithi164 ceph-mon[32713]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:07.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:07 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:07.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:07 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:07.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:07 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:07.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:07 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:07.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:07 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:07.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:07 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:07.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:07 smithi164 ceph-mon[32713]: Deploying daemon mds.e on smithi077 2024-04-25T10:30:07.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:07 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2752382918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:07.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:07 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.43:6836/2500994561,v1:172.21.15.43:6837/2500994561] up:boot 2024-04-25T10:30:07.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:07 smithi164 ceph-mon[32713]: fsmap 4 up:standby 2024-04-25T10:30:07.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:07 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T10:30:07.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:07 smithi043 ceph-mon[29478]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:07.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:07 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:07.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:07 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:07.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:07 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:07.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:07 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:07.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:07 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:07.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:07 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:07.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:07 smithi043 ceph-mon[29478]: Deploying daemon mds.e on smithi077 2024-04-25T10:30:07.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:07 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2752382918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:07.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:07 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.43:6836/2500994561,v1:172.21.15.43:6837/2500994561] up:boot 2024-04-25T10:30:07.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:07 smithi043 ceph-mon[29478]: fsmap 4 up:standby 2024-04-25T10:30:07.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:07 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T10:30:08.360 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:08.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.486+0000 7fada1914640 1 -- 172.21.15.43:0/136265005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9c109a50 msgr2=0x7fad9c10be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.486+0000 7fada1914640 1 --2- 172.21.15.43:0/136265005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9c109a50 0x7fad9c10be70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fad8c009ca0 tx=0x7fad8c02a8e0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.486+0000 7fada1914640 1 -- 172.21.15.43:0/136265005 shutdown_connections 2024-04-25T10:30:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.486+0000 7fada1914640 1 --2- 172.21.15.43:0/136265005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9c109a50 0x7fad9c10be70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.486+0000 7fada1914640 1 --2- 172.21.15.43:0/136265005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad9c104d50 0x7fad9c105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.486+0000 7fada1914640 1 --2- 172.21.15.43:0/136265005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c103b10 0x7fad9c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.486+0000 7fada1914640 1 -- 172.21.15.43:0/136265005 >> 172.21.15.43:0/136265005 conn(0x7fad9c0fed40 msgr2=0x7fad9c1011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.486+0000 7fada1914640 1 -- 172.21.15.43:0/136265005 shutdown_connections 2024-04-25T10:30:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.486+0000 7fada1914640 1 -- 172.21.15.43:0/136265005 wait complete. 2024-04-25T10:30:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.486+0000 7fada1914640 1 Processor -- start 2024-04-25T10:30:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.487+0000 7fada1914640 1 -- start start 2024-04-25T10:30:08.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.487+0000 7fada1914640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad9c103b10 0x7fad9c0778f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.487+0000 7fada1914640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9c104d50 0x7fad9c076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.487+0000 7fada1914640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c109a50 0x7fad9c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.487+0000 7fada1914640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c10f6b0 con 0x7fad9c104d50 2024-04-25T10:30:08.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.487+0000 7fada1914640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c10f530 con 0x7fad9c109a50 2024-04-25T10:30:08.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.487+0000 7fada1914640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c10f830 con 0x7fad9c103b10 2024-04-25T10:30:08.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.487+0000 7fad9a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9c104d50 0x7fad9c076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.487+0000 7fad9a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9c104d50 0x7fad9c076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38904/0 (socket says 172.21.15.43:38904) 2024-04-25T10:30:08.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.487+0000 7fad9a7fc640 1 -- 172.21.15.43:0/1019840620 learned_addr learned my addr 172.21.15.43:0/1019840620 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:08.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.487+0000 7fad9b7fe640 1 --2- 172.21.15.43:0/1019840620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c109a50 0x7fad9c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.487+0000 7fad9a7fc640 1 -- 172.21.15.43:0/1019840620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad9c103b10 msgr2=0x7fad9c0778f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:30:08.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.487+0000 7fad9a7fc640 1 --2- 172.21.15.43:0/1019840620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad9c103b10 0x7fad9c0778f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.487+0000 7fad9a7fc640 1 -- 172.21.15.43:0/1019840620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c109a50 msgr2=0x7fad9c076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.487+0000 7fad9a7fc640 1 --2- 172.21.15.43:0/1019840620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c109a50 0x7fad9c076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.487+0000 7fad9a7fc640 1 -- 172.21.15.43:0/1019840620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9c076c70 con 0x7fad9c104d50 2024-04-25T10:30:08.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.488+0000 7fad9a7fc640 1 --2- 172.21.15.43:0/1019840620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9c104d50 0x7fad9c076000 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fad9000b450 tx=0x7fad9000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:08.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.488+0000 7fada0912640 1 -- 172.21.15.43:0/1019840620 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90015020 con 0x7fad9c104d50 2024-04-25T10:30:08.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.488+0000 7fada0912640 1 -- 172.21.15.43:0/1019840620 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad90011d40 con 0x7fad9c104d50 2024-04-25T10:30:08.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.488+0000 7fada1914640 1 -- 172.21.15.43:0/1019840620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad9c1a43a0 con 0x7fad9c104d50 2024-04-25T10:30:08.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.488+0000 7fada0912640 1 -- 172.21.15.43:0/1019840620 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9000f340 con 0x7fad9c104d50 2024-04-25T10:30:08.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.488+0000 7fada1914640 1 -- 172.21.15.43:0/1019840620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad9c1a48e0 con 0x7fad9c104d50 2024-04-25T10:30:08.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.489+0000 7fada1914640 1 -- 172.21.15.43:0/1019840620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad9c108860 con 0x7fad9c104d50 2024-04-25T10:30:08.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.491+0000 7fada0912640 1 -- 172.21.15.43:0/1019840620 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fad90018020 con 0x7fad9c104d50 2024-04-25T10:30:08.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.491+0000 7fada0912640 1 --2- 172.21.15.43:0/1019840620 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fad840afb70 0x7fad840b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.491+0000 7fad9affd640 1 --2- 172.21.15.43:0/1019840620 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fad840afb70 0x7fad840b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.492+0000 7fada0912640 1 -- 172.21.15.43:0/1019840620 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fad900de660 con 0x7fad9c104d50 2024-04-25T10:30:08.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.492+0000 7fad9affd640 1 --2- 172.21.15.43:0/1019840620 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fad840afb70 0x7fad840b2080 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fad940058c0 tx=0x7fad94005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:08.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.495+0000 7fada0912640 1 -- 172.21.15.43:0/1019840620 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad9001f020 con 0x7fad9c104d50 2024-04-25T10:30:08.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.726+0000 7fada1914640 1 -- 172.21.15.43:0/1019840620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad9c076e40 con 0x7fad9c104d50 2024-04-25T10:30:08.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.726+0000 7fada0912640 1 -- 172.21.15.43:0/1019840620 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7fad9000d040 con 0x7fad9c104d50 2024-04-25T10:30:08.728 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:08.729 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T10:30:08.729 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 5 2024-04-25T10:30:08.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.728+0000 7fada1914640 1 -- 172.21.15.43:0/1019840620 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fad840afb70 msgr2=0x7fad840b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.728+0000 7fada1914640 1 --2- 172.21.15.43:0/1019840620 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fad840afb70 0x7fad840b2080 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fad940058c0 tx=0x7fad94005850 comp rx=0 tx=0).stop 2024-04-25T10:30:08.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.728+0000 7fada1914640 1 -- 172.21.15.43:0/1019840620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9c104d50 msgr2=0x7fad9c076000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.728+0000 7fada1914640 1 --2- 172.21.15.43:0/1019840620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9c104d50 0x7fad9c076000 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fad9000b450 tx=0x7fad9000b910 comp rx=0 tx=0).stop 2024-04-25T10:30:08.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.728+0000 7fada1914640 1 -- 172.21.15.43:0/1019840620 shutdown_connections 2024-04-25T10:30:08.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.728+0000 7fada1914640 1 --2- 172.21.15.43:0/1019840620 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fad840afb70 0x7fad840b2080 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.728+0000 7fada1914640 1 --2- 172.21.15.43:0/1019840620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c109a50 0x7fad9c076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.728+0000 7fada1914640 1 --2- 172.21.15.43:0/1019840620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9c104d50 0x7fad9c076000 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.728+0000 7fada1914640 1 --2- 172.21.15.43:0/1019840620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad9c103b10 0x7fad9c0778f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.728+0000 7fada1914640 1 -- 172.21.15.43:0/1019840620 >> 172.21.15.43:0/1019840620 conn(0x7fad9c0fed40 msgr2=0x7fad9c105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:08.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.728+0000 7fada1914640 1 -- 172.21.15.43:0/1019840620 shutdown_connections 2024-04-25T10:30:08.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:08.728+0000 7fada1914640 1 -- 172.21.15.43:0/1019840620 wait complete. 2024-04-25T10:30:08.775 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 25 10:30:08 smithi077 systemd[1]: Starting Ceph mds.e for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:30:09.040 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 25 10:30:08 smithi077 podman[63040]: 2024-04-25 10:30:08.676523319 +0000 UTC m=+0.018863066 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:30:09.040 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 25 10:30:08 smithi077 podman[63040]: 2024-04-25 10:30:08.775068118 +0000 UTC m=+0.117407852 container create dd1c5cb7d7d036d03c0b4ec23512a4d823a22295c93f6d7dee935a94b6dd1b8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:30:09.040 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 25 10:30:09 smithi077 podman[63040]: 2024-04-25 10:30:09.002816206 +0000 UTC m=+0.345155940 container init dd1c5cb7d7d036d03c0b4ec23512a4d823a22295c93f6d7dee935a94b6dd1b8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T10:30:09.040 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 25 10:30:09 smithi077 podman[63040]: 2024-04-25 10:30:09.005342283 +0000 UTC m=+0.347682026 container start dd1c5cb7d7d036d03c0b4ec23512a4d823a22295c93f6d7dee935a94b6dd1b8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T10:30:09.041 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 25 10:30:09 smithi077 ceph-mds[63059]: -- 172.21.15.77:0/140247884 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b709eda40 con 0x563b70a35180 2024-04-25T10:30:09.290 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 25 10:30:09 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-e[63055]: starting mds.e at 2024-04-25T10:30:09.290 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 25 10:30:09 smithi077 bash[63040]: dd1c5cb7d7d036d03c0b4ec23512a4d823a22295c93f6d7dee935a94b6dd1b8c 2024-04-25T10:30:09.291 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 25 10:30:09 smithi077 systemd[1]: Started Ceph mds.e for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:30:09.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:09 smithi043 ceph-mon[29478]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:09.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:09 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1019840620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:09.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:09.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:09.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:09.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:09.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:09.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:09 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:09.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:09 smithi077 ceph-mon[32772]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:09.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:09 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1019840620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:09.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:09 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:09.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:09 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:09.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:09 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:09.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:09 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:09.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:09 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:09.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:09 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:09.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:09 smithi164 ceph-mon[32713]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:09.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:09 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1019840620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:09.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:09 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:09.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:09 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:09.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:09 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:09.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:09 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:09.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:09 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:09.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:09 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:10.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:10 smithi164 ceph-mon[32713]: Deploying daemon mds.f on smithi164 2024-04-25T10:30:10.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:10 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.77:6834/1783762538,v1:172.21.15.77:6835/1783762538] up:boot 2024-04-25T10:30:10.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:10 smithi164 ceph-mon[32713]: fsmap 5 up:standby 2024-04-25T10:30:10.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:10 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T10:30:10.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:10 smithi043 ceph-mon[29478]: Deploying daemon mds.f on smithi164 2024-04-25T10:30:10.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:10 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.77:6834/1783762538,v1:172.21.15.77:6835/1783762538] up:boot 2024-04-25T10:30:10.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:10 smithi043 ceph-mon[29478]: fsmap 5 up:standby 2024-04-25T10:30:10.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:10 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T10:30:10.743 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:10.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:10 smithi077 ceph-mon[32772]: Deploying daemon mds.f on smithi164 2024-04-25T10:30:10.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:10 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.77:6834/1783762538,v1:172.21.15.77:6835/1783762538] up:boot 2024-04-25T10:30:10.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:10 smithi077 ceph-mon[32772]: fsmap 5 up:standby 2024-04-25T10:30:10.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:10 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T10:30:10.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.868+0000 7f687210d640 1 -- 172.21.15.43:0/2495600760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f686c1029c0 msgr2=0x7f686c1036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.868+0000 7f687210d640 1 --2- 172.21.15.43:0/2495600760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f686c1029c0 0x7f686c1036c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f685c009900 tx=0x7f685c02a410 comp rx=0 tx=0).stop 2024-04-25T10:30:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.868+0000 7f687210d640 1 -- 172.21.15.43:0/2495600760 shutdown_connections 2024-04-25T10:30:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.868+0000 7f687210d640 1 --2- 172.21.15.43:0/2495600760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f686c103c00 0x7f686c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.868+0000 7f687210d640 1 --2- 172.21.15.43:0/2495600760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f686c108900 0x7f686c10ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.868+0000 7f687210d640 1 --2- 172.21.15.43:0/2495600760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f686c1029c0 0x7f686c1036c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.868+0000 7f687210d640 1 -- 172.21.15.43:0/2495600760 >> 172.21.15.43:0/2495600760 conn(0x7f686c0fdbd0 msgr2=0x7f686c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.868+0000 7f687210d640 1 -- 172.21.15.43:0/2495600760 shutdown_connections 2024-04-25T10:30:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.868+0000 7f687210d640 1 -- 172.21.15.43:0/2495600760 wait complete. 2024-04-25T10:30:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.869+0000 7f687210d640 1 Processor -- start 2024-04-25T10:30:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.869+0000 7f687210d640 1 -- start start 2024-04-25T10:30:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.869+0000 7f687210d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f686c1029c0 0x7f686c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.869+0000 7f687210d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f686c103c00 0x7f686c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.869+0000 7f687210d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f686c108900 0x7f686c1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.869+0000 7f687210d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f686c10e4c0 con 0x7f686c108900 2024-04-25T10:30:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.869+0000 7f687210d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f686c10e340 con 0x7f686c1029c0 2024-04-25T10:30:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.869+0000 7f687210d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f686c10e640 con 0x7f686c103c00 2024-04-25T10:30:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.869+0000 7f686bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f686c108900 0x7f686c1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.869+0000 7f686bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f686c108900 0x7f686c1a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50544/0 (socket says 172.21.15.43:50544) 2024-04-25T10:30:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.869+0000 7f686bfff640 1 -- 172.21.15.43:0/2186806751 learned_addr learned my addr 172.21.15.43:0/2186806751 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.869+0000 7f686b7fe640 1 --2- 172.21.15.43:0/2186806751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f686c1029c0 0x7f686c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.869+0000 7f686affd640 1 --2- 172.21.15.43:0/2186806751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f686c103c00 0x7f686c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.870+0000 7f686bfff640 1 -- 172.21.15.43:0/2186806751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f686c103c00 msgr2=0x7f686c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.870+0000 7f686bfff640 1 --2- 172.21.15.43:0/2186806751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f686c103c00 0x7f686c19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.870+0000 7f686bfff640 1 -- 172.21.15.43:0/2186806751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f686c1029c0 msgr2=0x7f686c19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.870+0000 7f686bfff640 1 --2- 172.21.15.43:0/2186806751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f686c1029c0 0x7f686c19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.870+0000 7f686bfff640 1 -- 172.21.15.43:0/2186806751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686c1a1d80 con 0x7f686c108900 2024-04-25T10:30:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.870+0000 7f686affd640 1 --2- 172.21.15.43:0/2186806751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f686c103c00 0x7f686c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.870+0000 7f686b7fe640 1 --2- 172.21.15.43:0/2186806751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f686c1029c0 0x7f686c19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.870+0000 7f686bfff640 1 --2- 172.21.15.43:0/2186806751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f686c108900 0x7f686c1a1710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f686400eb10 tx=0x7f686400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.870+0000 7f6868ff9640 1 -- 172.21.15.43:0/2186806751 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864004b20 con 0x7f686c108900 2024-04-25T10:30:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.870+0000 7f687210d640 1 -- 172.21.15.43:0/2186806751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f686c1a2070 con 0x7f686c108900 2024-04-25T10:30:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.870+0000 7f687210d640 1 -- 172.21.15.43:0/2186806751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f686c1a2610 con 0x7f686c108900 2024-04-25T10:30:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.870+0000 7f6868ff9640 1 -- 172.21.15.43:0/2186806751 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6864004cc0 con 0x7f686c108900 2024-04-25T10:30:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.870+0000 7f6868ff9640 1 -- 172.21.15.43:0/2186806751 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864010690 con 0x7f686c108900 2024-04-25T10:30:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.871+0000 7f687210d640 1 -- 172.21.15.43:0/2186806751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68580052c0 con 0x7f686c108900 2024-04-25T10:30:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.873+0000 7f6868ff9640 1 -- 172.21.15.43:0/2186806751 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6864005000 con 0x7f686c108900 2024-04-25T10:30:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.874+0000 7f6868ff9640 1 --2- 172.21.15.43:0/2186806751 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f68540afaf0 0x7f68540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.874+0000 7f686b7fe640 1 --2- 172.21.15.43:0/2186806751 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f68540afaf0 0x7f68540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.874+0000 7f6868ff9640 1 -- 172.21.15.43:0/2186806751 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f68640def00 con 0x7f686c108900 2024-04-25T10:30:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.874+0000 7f686b7fe640 1 --2- 172.21.15.43:0/2186806751 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f68540afaf0 0x7f68540b2000 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f685c009230 tx=0x7f685c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:10.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:10.878+0000 7f6868ff9640 1 -- 172.21.15.43:0/2186806751 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6864053ff0 con 0x7f686c108900 2024-04-25T10:30:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:11.105+0000 7f687210d640 1 -- 172.21.15.43:0/2186806751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68580055b0 con 0x7f686c108900 2024-04-25T10:30:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:11.106+0000 7f6868ff9640 1 -- 172.21.15.43:0/2186806751 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4696 (secure 0 0 0) 0x7f68640a7f60 con 0x7f686c108900 2024-04-25T10:30:11.108 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:11.108 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T10:30:11.109 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 6 2024-04-25T10:30:11.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:11.108+0000 7f687210d640 1 -- 172.21.15.43:0/2186806751 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f68540afaf0 msgr2=0x7f68540b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:11.108+0000 7f687210d640 1 --2- 172.21.15.43:0/2186806751 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f68540afaf0 0x7f68540b2000 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f685c009230 tx=0x7f685c000bc0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:11.108+0000 7f687210d640 1 -- 172.21.15.43:0/2186806751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f686c108900 msgr2=0x7f686c1a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:11.108+0000 7f687210d640 1 --2- 172.21.15.43:0/2186806751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f686c108900 0x7f686c1a1710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f686400eb10 tx=0x7f686400c2d0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:11.108+0000 7f687210d640 1 -- 172.21.15.43:0/2186806751 shutdown_connections 2024-04-25T10:30:11.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:11.108+0000 7f687210d640 1 --2- 172.21.15.43:0/2186806751 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f68540afaf0 0x7f68540b2000 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:11.108+0000 7f687210d640 1 --2- 172.21.15.43:0/2186806751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f686c108900 0x7f686c1a1710 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:11.108+0000 7f687210d640 1 --2- 172.21.15.43:0/2186806751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f686c103c00 0x7f686c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:11.108+0000 7f687210d640 1 --2- 172.21.15.43:0/2186806751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f686c1029c0 0x7f686c19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:11.108+0000 7f687210d640 1 -- 172.21.15.43:0/2186806751 >> 172.21.15.43:0/2186806751 conn(0x7f686c0fdbd0 msgr2=0x7f686c109e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:11.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:11.108+0000 7f687210d640 1 -- 172.21.15.43:0/2186806751 shutdown_connections 2024-04-25T10:30:11.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:11.108+0000 7f687210d640 1 -- 172.21.15.43:0/2186806751 wait complete. 2024-04-25T10:30:11.475 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:11 smithi164 ceph-mon[32713]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:11.476 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:11 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2186806751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:11.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:11 smithi043 ceph-mon[29478]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:11.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:11 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2186806751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:11.744 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 25 10:30:11 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-f[63673]: starting mds.f at 2024-04-25T10:30:11.745 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 25 10:30:11 smithi164 bash[63658]: b5692deb846c3776d4aea42f1077fe2332e17e66ac837e6f7222cf244cd662f9 2024-04-25T10:30:11.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:11 smithi077 ceph-mon[32772]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:11.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:11 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2186806751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:12.041 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 25 10:30:11 smithi164 systemd[1]: Started Ceph mds.f for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:30:13.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:12 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:13.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:12 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:13.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:12 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:13.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:12 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:13.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:12 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:13.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:12 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:13.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:12 smithi164 ceph-mon[32713]: Deploying daemon mds.c on smithi164 2024-04-25T10:30:13.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:12 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] up:boot 2024-04-25T10:30:13.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:12 smithi164 ceph-mon[32713]: fsmap 6 up:standby 2024-04-25T10:30:13.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:12 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T10:30:13.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:12 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:13.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:12 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:13.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:12 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:13.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:12 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:13.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:12 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:13.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:12 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:13.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:12 smithi043 ceph-mon[29478]: Deploying daemon mds.c on smithi164 2024-04-25T10:30:13.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:12 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] up:boot 2024-04-25T10:30:13.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:12 smithi043 ceph-mon[29478]: fsmap 6 up:standby 2024-04-25T10:30:13.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:12 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T10:30:13.123 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:13.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.247+0000 7f99afa82640 1 -- 172.21.15.43:0/1119336580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99a8109a40 msgr2=0x7f99a810be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.247+0000 7f99afa82640 1 --2- 172.21.15.43:0/1119336580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99a8109a40 0x7f99a810be60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f99a0009ca0 tx=0x7f99a002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.247+0000 7f99afa82640 1 -- 172.21.15.43:0/1119336580 shutdown_connections 2024-04-25T10:30:13.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.247+0000 7f99afa82640 1 --2- 172.21.15.43:0/1119336580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99a8104d40 0x7f99a8105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.247+0000 7f99afa82640 1 --2- 172.21.15.43:0/1119336580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99a8109a40 0x7f99a810be60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.247+0000 7f99afa82640 1 --2- 172.21.15.43:0/1119336580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99a8103b00 0x7f99a8104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.247+0000 7f99afa82640 1 -- 172.21.15.43:0/1119336580 >> 172.21.15.43:0/1119336580 conn(0x7f99a80fed10 msgr2=0x7f99a81011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:13.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.247+0000 7f99afa82640 1 -- 172.21.15.43:0/1119336580 shutdown_connections 2024-04-25T10:30:13.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.247+0000 7f99afa82640 1 -- 172.21.15.43:0/1119336580 wait complete. 2024-04-25T10:30:13.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.247+0000 7f99afa82640 1 Processor -- start 2024-04-25T10:30:13.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.247+0000 7f99afa82640 1 -- start start 2024-04-25T10:30:13.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.248+0000 7f99afa82640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99a8103b00 0x7f99a819a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.248+0000 7f99afa82640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99a8104d40 0x7f99a819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.248+0000 7f99afa82640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99a8109a40 0x7f99a81a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.248+0000 7f99afa82640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f99a810f630 con 0x7f99a8104d40 2024-04-25T10:30:13.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.248+0000 7f99afa82640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f99a810f4b0 con 0x7f99a8109a40 2024-04-25T10:30:13.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.248+0000 7f99afa82640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f99a810f7b0 con 0x7f99a8103b00 2024-04-25T10:30:13.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.248+0000 7f99ad01d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99a8104d40 0x7f99a819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.248+0000 7f99ad01d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99a8104d40 0x7f99a819a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50560/0 (socket says 172.21.15.43:50560) 2024-04-25T10:30:13.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.248+0000 7f99ad01d640 1 -- 172.21.15.43:0/1378587132 learned_addr learned my addr 172.21.15.43:0/1378587132 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:13.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.248+0000 7f99ad81e640 1 --2- 172.21.15.43:0/1378587132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99a8103b00 0x7f99a819a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.248+0000 7f99ae01f640 1 --2- 172.21.15.43:0/1378587132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99a8109a40 0x7f99a81a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.248+0000 7f99ad01d640 1 -- 172.21.15.43:0/1378587132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99a8103b00 msgr2=0x7f99a819a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.248+0000 7f99ad01d640 1 --2- 172.21.15.43:0/1378587132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99a8103b00 0x7f99a819a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.248+0000 7f99ad01d640 1 -- 172.21.15.43:0/1378587132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99a8109a40 msgr2=0x7f99a81a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.248+0000 7f99ad01d640 1 --2- 172.21.15.43:0/1378587132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99a8109a40 0x7f99a81a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.248+0000 7f99ad01d640 1 -- 172.21.15.43:0/1378587132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a81a1d10 con 0x7f99a8104d40 2024-04-25T10:30:13.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.248+0000 7f99ad01d640 1 --2- 172.21.15.43:0/1378587132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99a8104d40 0x7f99a819a710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f999800d950 tx=0x7f999800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:13.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.249+0000 7f999effd640 1 -- 172.21.15.43:0/1378587132 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9998017070 con 0x7f99a8104d40 2024-04-25T10:30:13.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.249+0000 7f999effd640 1 -- 172.21.15.43:0/1378587132 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9998009c30 con 0x7f99a8104d40 2024-04-25T10:30:13.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.249+0000 7f999effd640 1 -- 172.21.15.43:0/1378587132 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999800f620 con 0x7f99a8104d40 2024-04-25T10:30:13.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.249+0000 7f99afa82640 1 -- 172.21.15.43:0/1378587132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a81a1fa0 con 0x7f99a8104d40 2024-04-25T10:30:13.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.249+0000 7f99afa82640 1 -- 172.21.15.43:0/1378587132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a81a2490 con 0x7f99a8104d40 2024-04-25T10:30:13.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.250+0000 7f99afa82640 1 -- 172.21.15.43:0/1378587132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a8108850 con 0x7f99a8104d40 2024-04-25T10:30:13.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.251+0000 7f999effd640 1 -- 172.21.15.43:0/1378587132 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f9998010460 con 0x7f99a8104d40 2024-04-25T10:30:13.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.252+0000 7f999effd640 1 --2- 172.21.15.43:0/1378587132 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f99940afaf0 0x7f99940b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.252+0000 7f99ad81e640 1 --2- 172.21.15.43:0/1378587132 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f99940afaf0 0x7f99940b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.252+0000 7f999effd640 1 -- 172.21.15.43:0/1378587132 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f99980de350 con 0x7f99a8104d40 2024-04-25T10:30:13.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.252+0000 7f99ad81e640 1 --2- 172.21.15.43:0/1378587132 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f99940afaf0 0x7f99940b2000 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f99a4005fd0 tx=0x7f99a400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:13.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.255+0000 7f999effd640 1 -- 172.21.15.43:0/1378587132 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9998009de0 con 0x7f99a8104d40 2024-04-25T10:30:13.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:12 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:13.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:12 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:13.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:12 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:13.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:12 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:13.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:12 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:13.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:12 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:13.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:12 smithi077 ceph-mon[32772]: Deploying daemon mds.c on smithi164 2024-04-25T10:30:13.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:12 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] up:boot 2024-04-25T10:30:13.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:12 smithi077 ceph-mon[32772]: fsmap 6 up:standby 2024-04-25T10:30:13.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:12 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T10:30:13.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.483+0000 7f99afa82640 1 -- 172.21.15.43:0/1378587132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99a802db50 con 0x7f99a8104d40 2024-04-25T10:30:13.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.484+0000 7f999effd640 1 -- 172.21.15.43:0/1378587132 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7f99980a6ce0 con 0x7f99a8104d40 2024-04-25T10:30:13.486 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:13.486 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T10:30:13.486 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 7 2024-04-25T10:30:13.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.486+0000 7f99afa82640 1 -- 172.21.15.43:0/1378587132 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f99940afaf0 msgr2=0x7f99940b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.486+0000 7f99afa82640 1 --2- 172.21.15.43:0/1378587132 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f99940afaf0 0x7f99940b2000 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f99a4005fd0 tx=0x7f99a400b390 comp rx=0 tx=0).stop 2024-04-25T10:30:13.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.486+0000 7f99afa82640 1 -- 172.21.15.43:0/1378587132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99a8104d40 msgr2=0x7f99a819a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.486+0000 7f99afa82640 1 --2- 172.21.15.43:0/1378587132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99a8104d40 0x7f99a819a710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f999800d950 tx=0x7f999800de10 comp rx=0 tx=0).stop 2024-04-25T10:30:13.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.486+0000 7f99afa82640 1 -- 172.21.15.43:0/1378587132 shutdown_connections 2024-04-25T10:30:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.486+0000 7f99afa82640 1 --2- 172.21.15.43:0/1378587132 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f99940afaf0 0x7f99940b2000 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.486+0000 7f99afa82640 1 --2- 172.21.15.43:0/1378587132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99a8109a40 0x7f99a81a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.486+0000 7f99afa82640 1 --2- 172.21.15.43:0/1378587132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99a8104d40 0x7f99a819a710 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.486+0000 7f99afa82640 1 --2- 172.21.15.43:0/1378587132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99a8103b00 0x7f99a819a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.486+0000 7f99afa82640 1 -- 172.21.15.43:0/1378587132 >> 172.21.15.43:0/1378587132 conn(0x7f99a80fed10 msgr2=0x7f99a8105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.486+0000 7f99afa82640 1 -- 172.21.15.43:0/1378587132 shutdown_connections 2024-04-25T10:30:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:13.486+0000 7f99afa82640 1 -- 172.21.15.43:0/1378587132 wait complete. 2024-04-25T10:30:14.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:13 smithi164 ceph-mon[32713]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:14.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:13 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1378587132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:14.042 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 25 10:30:13 smithi164 systemd[1]: Starting Ceph mds.c for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:30:14.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:13 smithi043 ceph-mon[29478]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:14.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:13 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1378587132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:14.192 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:13 smithi077 ceph-mon[32772]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:14.193 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:13 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1378587132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:14.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 25 10:30:14 smithi164 podman[63964]: 2024-04-25 10:30:14.002124949 +0000 UTC m=+0.018660484 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:30:14.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 25 10:30:14 smithi164 podman[63964]: 2024-04-25 10:30:14.126595691 +0000 UTC m=+0.143131212 container create d7162b297d7c5f0a4530a1f2cc405d376c837f0123589e70fb1a1930280158f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:30:14.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 25 10:30:14 smithi164 podman[63964]: 2024-04-25 10:30:14.345649417 +0000 UTC m=+0.362184938 container init d7162b297d7c5f0a4530a1f2cc405d376c837f0123589e70fb1a1930280158f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T10:30:14.791 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 25 10:30:14 smithi164 podman[63964]: 2024-04-25 10:30:14.348572258 +0000 UTC m=+0.365107787 container start d7162b297d7c5f0a4530a1f2cc405d376c837f0123589e70fb1a1930280158f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T10:30:14.791 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 25 10:30:14 smithi164 ceph-mds[63984]: -- 172.21.15.164:0/835273181 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562351981a40 con 0x5623519c9180 2024-04-25T10:30:14.791 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 25 10:30:14 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-c[63980]: starting mds.c at 2024-04-25T10:30:14.791 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 25 10:30:14 smithi164 bash[63964]: d7162b297d7c5f0a4530a1f2cc405d376c837f0123589e70fb1a1930280158f6 2024-04-25T10:30:14.791 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 25 10:30:14 smithi164 systemd[1]: Started Ceph mds.c for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:30:15.500 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:15.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.623+0000 7f8f18946640 1 -- 172.21.15.43:0/1158776825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f141033c0 msgr2=0x7f8f141037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.623+0000 7f8f18946640 1 --2- 172.21.15.43:0/1158776825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f141033c0 0x7f8f141037d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8efc009900 tx=0x7f8efc02a410 comp rx=0 tx=0).stop 2024-04-25T10:30:15.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.623+0000 7f8f18946640 1 -- 172.21.15.43:0/1158776825 shutdown_connections 2024-04-25T10:30:15.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.623+0000 7f8f18946640 1 --2- 172.21.15.43:0/1158776825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f141080c0 0x7f8f1410a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.623+0000 7f8f18946640 1 --2- 172.21.15.43:0/1158776825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f141033c0 0x7f8f141037d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.623+0000 7f8f18946640 1 --2- 172.21.15.43:0/1158776825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f14102180 0x7f8f14102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.623+0000 7f8f18946640 1 -- 172.21.15.43:0/1158776825 >> 172.21.15.43:0/1158776825 conn(0x7f8f140fd370 msgr2=0x7f8f140ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:15.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.624+0000 7f8f18946640 1 -- 172.21.15.43:0/1158776825 shutdown_connections 2024-04-25T10:30:15.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.624+0000 7f8f18946640 1 -- 172.21.15.43:0/1158776825 wait complete. 2024-04-25T10:30:15.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.624+0000 7f8f18946640 1 Processor -- start 2024-04-25T10:30:15.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.624+0000 7f8f18946640 1 -- start start 2024-04-25T10:30:15.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.625+0000 7f8f18946640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f14102180 0x7f8f1419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.625+0000 7f8f18946640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f141033c0 0x7f8f1419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.625+0000 7f8f18946640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f141080c0 0x7f8f141a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.625+0000 7f8f18946640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1410e520 con 0x7f8f141080c0 2024-04-25T10:30:15.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.625+0000 7f8f18946640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1410e3a0 con 0x7f8f141033c0 2024-04-25T10:30:15.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.625+0000 7f8f18946640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1410e6a0 con 0x7f8f14102180 2024-04-25T10:30:15.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.625+0000 7f8f12d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f141080c0 0x7f8f141a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.625+0000 7f8f12d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f141080c0 0x7f8f141a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50582/0 (socket says 172.21.15.43:50582) 2024-04-25T10:30:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.625+0000 7f8f12d9d640 1 -- 172.21.15.43:0/1492612791 learned_addr learned my addr 172.21.15.43:0/1492612791 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.625+0000 7f8f1259c640 1 --2- 172.21.15.43:0/1492612791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f14102180 0x7f8f1419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.625+0000 7f8f11d9b640 1 --2- 172.21.15.43:0/1492612791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f141033c0 0x7f8f1419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.625+0000 7f8f12d9d640 1 -- 172.21.15.43:0/1492612791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f14102180 msgr2=0x7f8f1419a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.625+0000 7f8f12d9d640 1 --2- 172.21.15.43:0/1492612791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f14102180 0x7f8f1419a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.625+0000 7f8f12d9d640 1 -- 172.21.15.43:0/1492612791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f141033c0 msgr2=0x7f8f1419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.625+0000 7f8f12d9d640 1 --2- 172.21.15.43:0/1492612791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f141033c0 0x7f8f1419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.625+0000 7f8f12d9d640 1 -- 172.21.15.43:0/1492612791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f141a1db0 con 0x7f8f141080c0 2024-04-25T10:30:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.625+0000 7f8f1259c640 1 --2- 172.21.15.43:0/1492612791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f14102180 0x7f8f1419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.625+0000 7f8f11d9b640 1 --2- 172.21.15.43:0/1492612791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f141033c0 0x7f8f1419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.625+0000 7f8f12d9d640 1 --2- 172.21.15.43:0/1492612791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f141080c0 0x7f8f141a1740 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00eb10 tx=0x7f8f0c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:15.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.626+0000 7f8f0b7fe640 1 -- 172.21.15.43:0/1492612791 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c004b20 con 0x7f8f141080c0 2024-04-25T10:30:15.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.626+0000 7f8f0b7fe640 1 -- 172.21.15.43:0/1492612791 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f0c004cc0 con 0x7f8f141080c0 2024-04-25T10:30:15.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.626+0000 7f8f18946640 1 -- 172.21.15.43:0/1492612791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f141a20a0 con 0x7f8f141080c0 2024-04-25T10:30:15.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.626+0000 7f8f0b7fe640 1 -- 172.21.15.43:0/1492612791 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c010670 con 0x7f8f141080c0 2024-04-25T10:30:15.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.626+0000 7f8f18946640 1 -- 172.21.15.43:0/1492612791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f141a25a0 con 0x7f8f141080c0 2024-04-25T10:30:15.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.627+0000 7f8f18946640 1 -- 172.21.15.43:0/1492612791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f141a2860 con 0x7f8f141080c0 2024-04-25T10:30:15.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.628+0000 7f8f0b7fe640 1 -- 172.21.15.43:0/1492612791 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f8f0c005020 con 0x7f8f141080c0 2024-04-25T10:30:15.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.629+0000 7f8f0b7fe640 1 --2- 172.21.15.43:0/1492612791 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f8f000afb70 0x7f8f000b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.629+0000 7f8f1259c640 1 --2- 172.21.15.43:0/1492612791 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f8f000afb70 0x7f8f000b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.629+0000 7f8f0b7fe640 1 -- 172.21.15.43:0/1492612791 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f8f0c0ded80 con 0x7f8f141080c0 2024-04-25T10:30:15.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.629+0000 7f8f1259c640 1 --2- 172.21.15.43:0/1492612791 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f8f000afb70 0x7f8f000b2080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8f14102d80 tx=0x7f8f04005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:15.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.634+0000 7f8f0b7fe640 1 -- 172.21.15.43:0/1492612791 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f0c0a7d60 con 0x7f8f141080c0 2024-04-25T10:30:15.738 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:15 smithi164 ceph-mon[32713]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:15.738 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:15 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:15.738 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:15 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:15.738 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:15 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:15.738 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:15 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:15.739 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:15 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:15.739 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:15 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:15.739 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:15 smithi164 ceph-mon[32713]: Deploying daemon mds.h on smithi077 2024-04-25T10:30:15.739 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:15 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.164:6836/3305330947,v1:172.21.15.164:6837/3305330947] up:boot 2024-04-25T10:30:15.739 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:15 smithi164 ceph-mon[32713]: fsmap 7 up:standby 2024-04-25T10:30:15.739 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:15 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T10:30:15.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:15 smithi077 ceph-mon[32772]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:15.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:15 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:15.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:15 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:15.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:15 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:15.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:15 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:15.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:15 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:15.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:15 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:15.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:15 smithi077 ceph-mon[32772]: Deploying daemon mds.h on smithi077 2024-04-25T10:30:15.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:15 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.164:6836/3305330947,v1:172.21.15.164:6837/3305330947] up:boot 2024-04-25T10:30:15.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:15 smithi077 ceph-mon[32772]: fsmap 7 up:standby 2024-04-25T10:30:15.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:15 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T10:30:15.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:15 smithi043 ceph-mon[29478]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:15.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:15 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:15 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:15 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:15 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:15 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:15 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:15 smithi043 ceph-mon[29478]: Deploying daemon mds.h on smithi077 2024-04-25T10:30:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:15 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.164:6836/3305330947,v1:172.21.15.164:6837/3305330947] up:boot 2024-04-25T10:30:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:15 smithi043 ceph-mon[29478]: fsmap 7 up:standby 2024-04-25T10:30:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:15 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T10:30:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.861+0000 7f8f18946640 1 -- 172.21.15.43:0/1492612791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f141a2a30 con 0x7f8f141080c0 2024-04-25T10:30:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.861+0000 7f8f0b7fe640 1 -- 172.21.15.43:0/1492612791 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7f8f0c0a7660 con 0x7f8f141080c0 2024-04-25T10:30:15.864 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:15.865 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T10:30:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 8 2024-04-25T10:30:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.864+0000 7f8f18946640 1 -- 172.21.15.43:0/1492612791 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f8f000afb70 msgr2=0x7f8f000b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.864+0000 7f8f18946640 1 --2- 172.21.15.43:0/1492612791 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f8f000afb70 0x7f8f000b2080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8f14102d80 tx=0x7f8f04005f90 comp rx=0 tx=0).stop 2024-04-25T10:30:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.864+0000 7f8f18946640 1 -- 172.21.15.43:0/1492612791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f141080c0 msgr2=0x7f8f141a1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.864+0000 7f8f18946640 1 --2- 172.21.15.43:0/1492612791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f141080c0 0x7f8f141a1740 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00eb10 tx=0x7f8f0c00c2d0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.864+0000 7f8f18946640 1 -- 172.21.15.43:0/1492612791 shutdown_connections 2024-04-25T10:30:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.864+0000 7f8f18946640 1 --2- 172.21.15.43:0/1492612791 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f8f000afb70 0x7f8f000b2080 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.864+0000 7f8f18946640 1 --2- 172.21.15.43:0/1492612791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f141033c0 0x7f8f1419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.864+0000 7f8f18946640 1 --2- 172.21.15.43:0/1492612791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f141080c0 0x7f8f141a1740 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.864+0000 7f8f18946640 1 --2- 172.21.15.43:0/1492612791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f14102180 0x7f8f1419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.864+0000 7f8f18946640 1 -- 172.21.15.43:0/1492612791 >> 172.21.15.43:0/1492612791 conn(0x7f8f140fd370 msgr2=0x7f8f141095f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.864+0000 7f8f18946640 1 -- 172.21.15.43:0/1492612791 shutdown_connections 2024-04-25T10:30:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:15.864+0000 7f8f18946640 1 -- 172.21.15.43:0/1492612791 wait complete. 2024-04-25T10:30:16.527 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:16 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1492612791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:16.790 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:30:16 smithi077 systemd[1]: Starting Ceph mds.h for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:30:16.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:16 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1492612791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:16.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:16 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1492612791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:17.192 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:30:16 smithi077 podman[63355]: 2024-04-25 10:30:16.823093572 +0000 UTC m=+0.017936688 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:30:17.192 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:30:16 smithi077 podman[63355]: 2024-04-25 10:30:16.938239767 +0000 UTC m=+0.133082870 container create 26ed057570417f0a4730800b3c8d82b38ff400ecd946f78f0483626bcf0f7083 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:30:17.192 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:30:17 smithi077 podman[63355]: 2024-04-25 10:30:17.190774119 +0000 UTC m=+0.385617222 container init 26ed057570417f0a4730800b3c8d82b38ff400ecd946f78f0483626bcf0f7083 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T10:30:17.474 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:30:17 smithi077 podman[63355]: 2024-04-25 10:30:17.193405892 +0000 UTC m=+0.388248997 container start 26ed057570417f0a4730800b3c8d82b38ff400ecd946f78f0483626bcf0f7083 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T10:30:17.475 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:30:17 smithi077 ceph-mds[63374]: -- 172.21.15.77:0/1546680161 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557945155a40 con 0x55794519d180 2024-04-25T10:30:17.475 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:30:17 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: starting mds.h at 2024-04-25T10:30:17.475 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:30:17 smithi077 bash[63355]: 26ed057570417f0a4730800b3c8d82b38ff400ecd946f78f0483626bcf0f7083 2024-04-25T10:30:17.475 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:30:17 smithi077 systemd[1]: Started Ceph mds.h for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:30:17.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:17 smithi077 ceph-mon[32772]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:17.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:17 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:17.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:17 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:17.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:17 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:17.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:17 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:17.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:17 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:17.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:17 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:17.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:17 smithi164 ceph-mon[32713]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:17.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:17.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:17.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:17.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:17.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:17.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:17 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:17.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:17 smithi043 ceph-mon[29478]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:17.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:17 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:17.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:17 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:17.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:17 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:17.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:17 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:17.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:17 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:17.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:17 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:17.879 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:18.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.003+0000 7f1240d1f640 1 -- 172.21.15.43:0/2475532299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f123c109a40 msgr2=0x7f123c10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.003+0000 7f1240d1f640 1 --2- 172.21.15.43:0/2475532299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f123c109a40 0x7f123c10be60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1234009be0 tx=0x7f123402a6c0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.003+0000 7f1240d1f640 1 -- 172.21.15.43:0/2475532299 shutdown_connections 2024-04-25T10:30:18.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.003+0000 7f1240d1f640 1 --2- 172.21.15.43:0/2475532299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f123c109a40 0x7f123c10be60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.003+0000 7f1240d1f640 1 --2- 172.21.15.43:0/2475532299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f123c104d40 0x7f123c105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.003+0000 7f1240d1f640 1 --2- 172.21.15.43:0/2475532299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f123c103b00 0x7f123c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.003+0000 7f1240d1f640 1 -- 172.21.15.43:0/2475532299 >> 172.21.15.43:0/2475532299 conn(0x7f123c0fed10 msgr2=0x7f123c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:18.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.003+0000 7f1240d1f640 1 -- 172.21.15.43:0/2475532299 shutdown_connections 2024-04-25T10:30:18.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.003+0000 7f1240d1f640 1 -- 172.21.15.43:0/2475532299 wait complete. 2024-04-25T10:30:18.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.004+0000 7f1240d1f640 1 Processor -- start 2024-04-25T10:30:18.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.004+0000 7f1240d1f640 1 -- start start 2024-04-25T10:30:18.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.004+0000 7f1240d1f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f123c103b00 0x7f123c19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.004+0000 7f1240d1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f123c104d40 0x7f123c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.004+0000 7f1240d1f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f123c109a40 0x7f123c1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.004+0000 7f1240d1f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f123c10f630 con 0x7f123c104d40 2024-04-25T10:30:18.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.004+0000 7f1240d1f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f123c10f4b0 con 0x7f123c103b00 2024-04-25T10:30:18.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.004+0000 7f1240d1f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f123c10f7b0 con 0x7f123c109a40 2024-04-25T10:30:18.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.004+0000 7f1239d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f123c104d40 0x7f123c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.004+0000 7f123ad9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f123c109a40 0x7f123c1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.004+0000 7f1239d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f123c104d40 0x7f123c19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50596/0 (socket says 172.21.15.43:50596) 2024-04-25T10:30:18.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.004+0000 7f1239d9b640 1 -- 172.21.15.43:0/216614700 learned_addr learned my addr 172.21.15.43:0/216614700 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:18.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.004+0000 7f123a59c640 1 --2- 172.21.15.43:0/216614700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f123c103b00 0x7f123c19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.004+0000 7f1239d9b640 1 -- 172.21.15.43:0/216614700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f123c109a40 msgr2=0x7f123c1a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.004+0000 7f1239d9b640 1 --2- 172.21.15.43:0/216614700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f123c109a40 0x7f123c1a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.004+0000 7f1239d9b640 1 -- 172.21.15.43:0/216614700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f123c103b00 msgr2=0x7f123c19a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.004+0000 7f1239d9b640 1 --2- 172.21.15.43:0/216614700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f123c103b00 0x7f123c19a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.004+0000 7f1239d9b640 1 -- 172.21.15.43:0/216614700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123c1a1d50 con 0x7f123c104d40 2024-04-25T10:30:18.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.004+0000 7f123a59c640 1 --2- 172.21.15.43:0/216614700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f123c103b00 0x7f123c19a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:18.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.005+0000 7f1239d9b640 1 --2- 172.21.15.43:0/216614700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f123c104d40 0x7f123c19a750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f123000e9c0 tx=0x7f123000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.005+0000 7f122b7fe640 1 -- 172.21.15.43:0/216614700 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123000cde0 con 0x7f123c104d40 2024-04-25T10:30:18.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.005+0000 7f122b7fe640 1 -- 172.21.15.43:0/216614700 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12300035f0 con 0x7f123c104d40 2024-04-25T10:30:18.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.005+0000 7f122b7fe640 1 -- 172.21.15.43:0/216614700 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1230010690 con 0x7f123c104d40 2024-04-25T10:30:18.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.005+0000 7f1240d1f640 1 -- 172.21.15.43:0/216614700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f123c1a1fe0 con 0x7f123c104d40 2024-04-25T10:30:18.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.005+0000 7f1240d1f640 1 -- 172.21.15.43:0/216614700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f123c1a2480 con 0x7f123c104d40 2024-04-25T10:30:18.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.006+0000 7f1240d1f640 1 -- 172.21.15.43:0/216614700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f123c108850 con 0x7f123c104d40 2024-04-25T10:30:18.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.007+0000 7f122b7fe640 1 -- 172.21.15.43:0/216614700 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f1230004000 con 0x7f123c104d40 2024-04-25T10:30:18.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.008+0000 7f122b7fe640 1 --2- 172.21.15.43:0/216614700 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f12240afb70 0x7f12240b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.008+0000 7f123a59c640 1 --2- 172.21.15.43:0/216614700 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f12240afb70 0x7f12240b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.008+0000 7f122b7fe640 1 -- 172.21.15.43:0/216614700 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f12300deaf0 con 0x7f123c104d40 2024-04-25T10:30:18.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.009+0000 7f123a59c640 1 --2- 172.21.15.43:0/216614700 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f12240afb70 0x7f12240b2080 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f123c104700 tx=0x7f122c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.012+0000 7f122b7fe640 1 -- 172.21.15.43:0/216614700 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f12300a7c00 con 0x7f123c104d40 2024-04-25T10:30:18.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.239+0000 7f1240d1f640 1 -- 172.21.15.43:0/216614700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f123c02db50 con 0x7f123c104d40 2024-04-25T10:30:18.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.239+0000 7f122b7fe640 1 -- 172.21.15.43:0/216614700 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7177 (secure 0 0 0) 0x7f12300a7500 con 0x7f123c104d40 2024-04-25T10:30:18.242 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:18.243 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/2595248466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2595248466},{"type":"v1","addr":"172.21.15.77:6837","nonce":2595248466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T10:30:18.243 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-04-25T10:30:18.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.241+0000 7f1240d1f640 1 -- 172.21.15.43:0/216614700 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f12240afb70 msgr2=0x7f12240b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.241+0000 7f1240d1f640 1 --2- 172.21.15.43:0/216614700 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f12240afb70 0x7f12240b2080 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f123c104700 tx=0x7f122c005f90 comp rx=0 tx=0).stop 2024-04-25T10:30:18.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.241+0000 7f1240d1f640 1 -- 172.21.15.43:0/216614700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f123c104d40 msgr2=0x7f123c19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.241+0000 7f1240d1f640 1 --2- 172.21.15.43:0/216614700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f123c104d40 0x7f123c19a750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f123000e9c0 tx=0x7f123000ee80 comp rx=0 tx=0).stop 2024-04-25T10:30:18.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.241+0000 7f1240d1f640 1 -- 172.21.15.43:0/216614700 shutdown_connections 2024-04-25T10:30:18.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.241+0000 7f1240d1f640 1 --2- 172.21.15.43:0/216614700 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f12240afb70 0x7f12240b2080 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.241+0000 7f1240d1f640 1 --2- 172.21.15.43:0/216614700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f123c109a40 0x7f123c1a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.241+0000 7f1240d1f640 1 --2- 172.21.15.43:0/216614700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f123c104d40 0x7f123c19a750 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.241+0000 7f1240d1f640 1 --2- 172.21.15.43:0/216614700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f123c103b00 0x7f123c19a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.241+0000 7f1240d1f640 1 -- 172.21.15.43:0/216614700 >> 172.21.15.43:0/216614700 conn(0x7f123c0fed10 msgr2=0x7f123c105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:18.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.242+0000 7f1240d1f640 1 -- 172.21.15.43:0/216614700 shutdown_connections 2024-04-25T10:30:18.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:18.242+0000 7f1240d1f640 1 -- 172.21.15.43:0/216614700 wait complete. 2024-04-25T10:30:18.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:18 smithi043 ceph-mon[29478]: Deploying daemon mds.i on smithi164 2024-04-25T10:30:18.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:18 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.77:6836/2595248466,v1:172.21.15.77:6837/2595248466] up:boot 2024-04-25T10:30:18.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:18 smithi043 ceph-mon[29478]: fsmap 8 up:standby 2024-04-25T10:30:18.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:18 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T10:30:18.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:18 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/216614700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:18.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:18 smithi077 ceph-mon[32772]: Deploying daemon mds.i on smithi164 2024-04-25T10:30:18.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:18 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.77:6836/2595248466,v1:172.21.15.77:6837/2595248466] up:boot 2024-04-25T10:30:18.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:18 smithi077 ceph-mon[32772]: fsmap 8 up:standby 2024-04-25T10:30:18.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:18 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T10:30:18.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:18 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/216614700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:18.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:18 smithi164 ceph-mon[32713]: Deploying daemon mds.i on smithi164 2024-04-25T10:30:18.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:18 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.77:6836/2595248466,v1:172.21.15.77:6837/2595248466] up:boot 2024-04-25T10:30:18.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:18 smithi164 ceph-mon[32713]: fsmap 8 up:standby 2024-04-25T10:30:18.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:18 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T10:30:18.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:18 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/216614700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:19.488 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 25 10:30:19 smithi164 systemd[1]: Starting Ceph mds.i for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:30:19.740 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:19 smithi164 ceph-mon[32713]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:19.740 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 25 10:30:19 smithi164 podman[64285]: 2024-04-25 10:30:19.48941708 +0000 UTC m=+0.019139393 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:30:19.740 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 25 10:30:19 smithi164 podman[64285]: 2024-04-25 10:30:19.595870998 +0000 UTC m=+0.125593326 container create 23da590987536d447664426bd66b4766f49445ebf423d302792660e81d868476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T10:30:19.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:19 smithi077 ceph-mon[32772]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:19.862 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:19 smithi043 ceph-mon[29478]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:20.257 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:20.291 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 25 10:30:19 smithi164 podman[64285]: 2024-04-25 10:30:19.831641328 +0000 UTC m=+0.361363611 container init 23da590987536d447664426bd66b4766f49445ebf423d302792660e81d868476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T10:30:20.291 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 25 10:30:19 smithi164 podman[64285]: 2024-04-25 10:30:19.834288734 +0000 UTC m=+0.364011028 container start 23da590987536d447664426bd66b4766f49445ebf423d302792660e81d868476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T10:30:20.292 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 25 10:30:19 smithi164 ceph-mds[64304]: -- 172.21.15.164:0/1559604142 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559754ab1a40 con 0x559754af9180 2024-04-25T10:30:20.292 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 25 10:30:19 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-i[64300]: starting mds.i at 2024-04-25T10:30:20.292 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 25 10:30:19 smithi164 bash[64285]: 23da590987536d447664426bd66b4766f49445ebf423d302792660e81d868476 2024-04-25T10:30:20.292 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 25 10:30:19 smithi164 systemd[1]: Started Ceph mds.i for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:30:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.368+0000 7f3fc2451640 1 -- 172.21.15.43:0/2188411207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fbc106be0 msgr2=0x7f3fbc1090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.368+0000 7f3fc2451640 1 --2- 172.21.15.43:0/2188411207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fbc106be0 0x7f3fbc1090c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009900 tx=0x7f3fb002a410 comp rx=0 tx=0).stop 2024-04-25T10:30:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.368+0000 7f3fc2451640 1 -- 172.21.15.43:0/2188411207 shutdown_connections 2024-04-25T10:30:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.368+0000 7f3fc2451640 1 --2- 172.21.15.43:0/2188411207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fbc109750 0x7f3fbc10bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.368+0000 7f3fc2451640 1 --2- 172.21.15.43:0/2188411207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fbc106be0 0x7f3fbc1090c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.368+0000 7f3fc2451640 1 --2- 172.21.15.43:0/2188411207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fbc1041f0 0x7f3fbc106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.369+0000 7f3fc2451640 1 -- 172.21.15.43:0/2188411207 >> 172.21.15.43:0/2188411207 conn(0x7f3fbc0fdbd0 msgr2=0x7f3fbc100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.369+0000 7f3fc2451640 1 -- 172.21.15.43:0/2188411207 shutdown_connections 2024-04-25T10:30:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.369+0000 7f3fc2451640 1 -- 172.21.15.43:0/2188411207 wait complete. 2024-04-25T10:30:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.369+0000 7f3fc2451640 1 Processor -- start 2024-04-25T10:30:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.369+0000 7f3fc2451640 1 -- start start 2024-04-25T10:30:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.370+0000 7f3fc2451640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fbc1041f0 0x7f3fbc103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.370+0000 7f3fc2451640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fbc106be0 0x7f3fbc102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.370+0000 7f3fc2451640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fbc109750 0x7f3fbc102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.370+0000 7f3fc2451640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc10f610 con 0x7f3fbc109750 2024-04-25T10:30:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.370+0000 7f3fc2451640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc10f490 con 0x7f3fbc1041f0 2024-04-25T10:30:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.370+0000 7f3fc2451640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc10f790 con 0x7f3fbc106be0 2024-04-25T10:30:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.370+0000 7f3fc09ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fbc109750 0x7f3fbc102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.370+0000 7f3fc09ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fbc109750 0x7f3fbc102640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44160/0 (socket says 172.21.15.43:44160) 2024-04-25T10:30:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.370+0000 7f3fc09ee640 1 -- 172.21.15.43:0/3970263504 learned_addr learned my addr 172.21.15.43:0/3970263504 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.370+0000 7f3fbbfff640 1 --2- 172.21.15.43:0/3970263504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fbc1041f0 0x7f3fbc103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.370+0000 7f3fbb7fe640 1 --2- 172.21.15.43:0/3970263504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fbc106be0 0x7f3fbc102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.370+0000 7f3fc09ee640 1 -- 172.21.15.43:0/3970263504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fbc106be0 msgr2=0x7f3fbc102100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.370+0000 7f3fc09ee640 1 --2- 172.21.15.43:0/3970263504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fbc106be0 0x7f3fbc102100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.370+0000 7f3fc09ee640 1 -- 172.21.15.43:0/3970263504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fbc1041f0 msgr2=0x7f3fbc103a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.370+0000 7f3fc09ee640 1 --2- 172.21.15.43:0/3970263504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fbc1041f0 0x7f3fbc103a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.370+0000 7f3fc09ee640 1 -- 172.21.15.43:0/3970263504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fbc102d70 con 0x7f3fbc109750 2024-04-25T10:30:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.370+0000 7f3fbbfff640 1 --2- 172.21.15.43:0/3970263504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fbc1041f0 0x7f3fbc103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.370+0000 7f3fc09ee640 1 --2- 172.21.15.43:0/3970263504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fbc109750 0x7f3fbc102640 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400eaa0 tx=0x7f3fb400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:20.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.371+0000 7f3fb97fa640 1 -- 172.21.15.43:0/3970263504 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb400cdb0 con 0x7f3fbc109750 2024-04-25T10:30:20.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.371+0000 7f3fc2451640 1 -- 172.21.15.43:0/3970263504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fbc1a42b0 con 0x7f3fbc109750 2024-04-25T10:30:20.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.371+0000 7f3fb97fa640 1 -- 172.21.15.43:0/3970263504 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fb4005020 con 0x7f3fbc109750 2024-04-25T10:30:20.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.371+0000 7f3fb97fa640 1 -- 172.21.15.43:0/3970263504 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb40116d0 con 0x7f3fbc109750 2024-04-25T10:30:20.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.371+0000 7f3fc2451640 1 -- 172.21.15.43:0/3970263504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fbc1a4720 con 0x7f3fbc109750 2024-04-25T10:30:20.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.372+0000 7f3fc2451640 1 -- 172.21.15.43:0/3970263504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fbc0ff6f0 con 0x7f3fbc109750 2024-04-25T10:30:20.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.374+0000 7f3fb97fa640 1 -- 172.21.15.43:0/3970263504 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f3fb40051d0 con 0x7f3fbc109750 2024-04-25T10:30:20.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.374+0000 7f3fb97fa640 1 --2- 172.21.15.43:0/3970263504 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f3fa40afaf0 0x7f3fa40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.375+0000 7f3fbbfff640 1 --2- 172.21.15.43:0/3970263504 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f3fa40afaf0 0x7f3fa40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.375+0000 7f3fb97fa640 1 -- 172.21.15.43:0/3970263504 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f3fb40dec80 con 0x7f3fbc109750 2024-04-25T10:30:20.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.375+0000 7f3fbbfff640 1 --2- 172.21.15.43:0/3970263504 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f3fa40afaf0 0x7f3fa40b2000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc06a850 tx=0x7f3fac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:20.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.378+0000 7f3fb97fa640 1 -- 172.21.15.43:0/3970263504 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3fb4016030 con 0x7f3fbc109750 2024-04-25T10:30:20.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.620+0000 7f3fc2451640 1 -- 172.21.15.43:0/3970263504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fbc02db50 con 0x7f3fbc109750 2024-04-25T10:30:20.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.620+0000 7f3fb97fa640 1 -- 172.21.15.43:0/3970263504 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7f3fb40a1e20 con 0x7f3fbc109750 2024-04-25T10:30:20.623 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:20.623 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/2595248466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2595248466},{"type":"v1","addr":"172.21.15.77:6837","nonce":2595248466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T10:30:20.623 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 10 2024-04-25T10:30:20.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.623+0000 7f3fc2451640 1 -- 172.21.15.43:0/3970263504 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f3fa40afaf0 msgr2=0x7f3fa40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.623+0000 7f3fc2451640 1 --2- 172.21.15.43:0/3970263504 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f3fa40afaf0 0x7f3fa40b2000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc06a850 tx=0x7f3fac005f90 comp rx=0 tx=0).stop 2024-04-25T10:30:20.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.623+0000 7f3fc2451640 1 -- 172.21.15.43:0/3970263504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fbc109750 msgr2=0x7f3fbc102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.623+0000 7f3fc2451640 1 --2- 172.21.15.43:0/3970263504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fbc109750 0x7f3fbc102640 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400eaa0 tx=0x7f3fb400ef60 comp rx=0 tx=0).stop 2024-04-25T10:30:20.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.623+0000 7f3fc2451640 1 -- 172.21.15.43:0/3970263504 shutdown_connections 2024-04-25T10:30:20.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.623+0000 7f3fc2451640 1 --2- 172.21.15.43:0/3970263504 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f3fa40afaf0 0x7f3fa40b2000 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.623+0000 7f3fc2451640 1 --2- 172.21.15.43:0/3970263504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fbc109750 0x7f3fbc102640 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.623+0000 7f3fc2451640 1 --2- 172.21.15.43:0/3970263504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fbc106be0 0x7f3fbc102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.623+0000 7f3fc2451640 1 --2- 172.21.15.43:0/3970263504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fbc1041f0 0x7f3fbc103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.623+0000 7f3fc2451640 1 -- 172.21.15.43:0/3970263504 >> 172.21.15.43:0/3970263504 conn(0x7f3fbc0fdbd0 msgr2=0x7f3fbc104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:20.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.623+0000 7f3fc2451640 1 -- 172.21.15.43:0/3970263504 shutdown_connections 2024-04-25T10:30:20.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:20.623+0000 7f3fc2451640 1 -- 172.21.15.43:0/3970263504 wait complete. 2024-04-25T10:30:21.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:20 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:21.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:20 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:21.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:20 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:21.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:20 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:21.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:20 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:21.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:20 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:21.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:20 smithi077 ceph-mon[32772]: Deploying daemon mds.g on smithi043 2024-04-25T10:30:21.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:20 smithi077 ceph-mon[32772]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:21.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:20 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] up:boot 2024-04-25T10:30:21.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:20 smithi077 ceph-mon[32772]: fsmap 9 up:standby 2024-04-25T10:30:21.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:20 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T10:30:21.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:20 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3970263504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:21.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:20 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:21.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:20 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:21.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:20 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:21.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:20 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:21.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:20 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:21.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:20 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:21.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:20 smithi164 ceph-mon[32713]: Deploying daemon mds.g on smithi043 2024-04-25T10:30:21.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:20 smithi164 ceph-mon[32713]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:21.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:20 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] up:boot 2024-04-25T10:30:21.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:20 smithi164 ceph-mon[32713]: fsmap 9 up:standby 2024-04-25T10:30:21.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:20 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T10:30:21.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:20 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3970263504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:21.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:20 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:21.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:20 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:21.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:20 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:21.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:20 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:21.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:20 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:21.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:20 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:21.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:20 smithi043 ceph-mon[29478]: Deploying daemon mds.g on smithi043 2024-04-25T10:30:21.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:20 smithi043 ceph-mon[29478]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:21.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:20 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] up:boot 2024-04-25T10:30:21.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:20 smithi043 ceph-mon[29478]: fsmap 9 up:standby 2024-04-25T10:30:21.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:20 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T10:30:21.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:20 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3970263504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:22.361 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 25 10:30:22 smithi043 systemd[1]: Starting Ceph mds.g for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:30:22.638 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:22.725 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 25 10:30:22 smithi043 podman[68473]: 2024-04-25 10:30:22.357533333 +0000 UTC m=+0.018546653 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:30:22.725 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 25 10:30:22 smithi043 podman[68473]: 2024-04-25 10:30:22.479182163 +0000 UTC m=+0.140195467 container create e7c559510af1f8072990ffbc01cfc683465b42e6ce57170a45d2771091344265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T10:30:22.725 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 25 10:30:22 smithi043 podman[68473]: 2024-04-25 10:30:22.723437922 +0000 UTC m=+0.384451226 container init e7c559510af1f8072990ffbc01cfc683465b42e6ce57170a45d2771091344265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T10:30:22.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.755+0000 7f31ce492640 1 -- 172.21.15.43:0/3937916380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c81029c0 msgr2=0x7f31c81036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.755+0000 7f31ce492640 1 --2- 172.21.15.43:0/3937916380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c81029c0 0x7f31c81036c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009900 tx=0x7f31b802a410 comp rx=0 tx=0).stop 2024-04-25T10:30:22.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.755+0000 7f31ce492640 1 -- 172.21.15.43:0/3937916380 shutdown_connections 2024-04-25T10:30:22.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.755+0000 7f31ce492640 1 --2- 172.21.15.43:0/3937916380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31c8103c00 0x7f31c8104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.755+0000 7f31ce492640 1 --2- 172.21.15.43:0/3937916380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31c8108900 0x7f31c810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.755+0000 7f31ce492640 1 --2- 172.21.15.43:0/3937916380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c81029c0 0x7f31c81036c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.755+0000 7f31ce492640 1 -- 172.21.15.43:0/3937916380 >> 172.21.15.43:0/3937916380 conn(0x7f31c80fdbd0 msgr2=0x7f31c8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:22.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.755+0000 7f31ce492640 1 -- 172.21.15.43:0/3937916380 shutdown_connections 2024-04-25T10:30:22.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.755+0000 7f31ce492640 1 -- 172.21.15.43:0/3937916380 wait complete. 2024-04-25T10:30:22.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.756+0000 7f31ce492640 1 Processor -- start 2024-04-25T10:30:22.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.756+0000 7f31ce492640 1 -- start start 2024-04-25T10:30:22.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.756+0000 7f31ce492640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31c81029c0 0x7f31c819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.756+0000 7f31ce492640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31c8103c00 0x7f31c819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.756+0000 7f31ce492640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c8108900 0x7f31c81a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.756+0000 7f31ce492640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f31c810e4c0 con 0x7f31c8108900 2024-04-25T10:30:22.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.756+0000 7f31ce492640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f31c810e340 con 0x7f31c81029c0 2024-04-25T10:30:22.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.756+0000 7f31ce492640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f31c810e640 con 0x7f31c8103c00 2024-04-25T10:30:22.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.757+0000 7f31cca2f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c8108900 0x7f31c81a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.757+0000 7f31cca2f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c8108900 0x7f31c81a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44176/0 (socket says 172.21.15.43:44176) 2024-04-25T10:30:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.757+0000 7f31cca2f640 1 -- 172.21.15.43:0/3812514393 learned_addr learned my addr 172.21.15.43:0/3812514393 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.757+0000 7f31c7fff640 1 --2- 172.21.15.43:0/3812514393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31c81029c0 0x7f31c819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.757+0000 7f31c77fe640 1 --2- 172.21.15.43:0/3812514393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31c8103c00 0x7f31c819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.757+0000 7f31cca2f640 1 -- 172.21.15.43:0/3812514393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31c8103c00 msgr2=0x7f31c819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.757+0000 7f31cca2f640 1 --2- 172.21.15.43:0/3812514393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31c8103c00 0x7f31c819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.757+0000 7f31cca2f640 1 -- 172.21.15.43:0/3812514393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31c81029c0 msgr2=0x7f31c819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.757+0000 7f31cca2f640 1 --2- 172.21.15.43:0/3812514393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31c81029c0 0x7f31c819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.757+0000 7f31cca2f640 1 -- 172.21.15.43:0/3812514393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c81a1d80 con 0x7f31c8108900 2024-04-25T10:30:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.757+0000 7f31cca2f640 1 --2- 172.21.15.43:0/3812514393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c8108900 0x7f31c81a1710 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f31c000eaa0 tx=0x7f31c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.757+0000 7f31c57fa640 1 -- 172.21.15.43:0/3812514393 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c000cdb0 con 0x7f31c8108900 2024-04-25T10:30:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.757+0000 7f31c57fa640 1 -- 172.21.15.43:0/3812514393 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31c0005020 con 0x7f31c8108900 2024-04-25T10:30:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.757+0000 7f31ce492640 1 -- 172.21.15.43:0/3812514393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c81a2070 con 0x7f31c8108900 2024-04-25T10:30:22.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.757+0000 7f31c57fa640 1 -- 172.21.15.43:0/3812514393 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c00116d0 con 0x7f31c8108900 2024-04-25T10:30:22.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.757+0000 7f31ce492640 1 -- 172.21.15.43:0/3812514393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c81a2610 con 0x7f31c8108900 2024-04-25T10:30:22.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.759+0000 7f31ce492640 1 -- 172.21.15.43:0/3812514393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c81a28d0 con 0x7f31c8108900 2024-04-25T10:30:22.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.760+0000 7f31c57fa640 1 -- 172.21.15.43:0/3812514393 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f31c00051d0 con 0x7f31c8108900 2024-04-25T10:30:22.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.760+0000 7f31c57fa640 1 --2- 172.21.15.43:0/3812514393 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f31b40afb70 0x7f31b40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.761+0000 7f31c7fff640 1 --2- 172.21.15.43:0/3812514393 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f31b40afb70 0x7f31b40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.761+0000 7f31c57fa640 1 -- 172.21.15.43:0/3812514393 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f31c00de850 con 0x7f31c8108900 2024-04-25T10:30:22.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.761+0000 7f31c7fff640 1 --2- 172.21.15.43:0/3812514393 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f31b40afb70 0x7f31b40b2080 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f31b8005ab0 tx=0x7f31b8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:22.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.765+0000 7f31c57fa640 1 -- 172.21.15.43:0/3812514393 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31c0016030 con 0x7f31c8108900 2024-04-25T10:30:22.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.996+0000 7f31ce492640 1 -- 172.21.15.43:0/3812514393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31c81a2aa0 con 0x7f31c8108900 2024-04-25T10:30:22.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.997+0000 7f31c57fa640 1 -- 172.21.15.43:0/3812514393 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7f31c00a7280 con 0x7f31c8108900 2024-04-25T10:30:22.999 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:22.999 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/2595248466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2595248466},{"type":"v1","addr":"172.21.15.77:6837","nonce":2595248466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T10:30:22.999 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 10 2024-04-25T10:30:23.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.999+0000 7f31ce492640 1 -- 172.21.15.43:0/3812514393 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f31b40afb70 msgr2=0x7f31b40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.999+0000 7f31ce492640 1 --2- 172.21.15.43:0/3812514393 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f31b40afb70 0x7f31b40b2080 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f31b8005ab0 tx=0x7f31b8031400 comp rx=0 tx=0).stop 2024-04-25T10:30:23.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.999+0000 7f31ce492640 1 -- 172.21.15.43:0/3812514393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c8108900 msgr2=0x7f31c81a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.999+0000 7f31ce492640 1 --2- 172.21.15.43:0/3812514393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c8108900 0x7f31c81a1710 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f31c000eaa0 tx=0x7f31c000ef60 comp rx=0 tx=0).stop 2024-04-25T10:30:23.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.999+0000 7f31ce492640 1 -- 172.21.15.43:0/3812514393 shutdown_connections 2024-04-25T10:30:23.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.999+0000 7f31ce492640 1 --2- 172.21.15.43:0/3812514393 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f31b40afb70 0x7f31b40b2080 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.999+0000 7f31ce492640 1 --2- 172.21.15.43:0/3812514393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c8108900 0x7f31c81a1710 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.999+0000 7f31ce492640 1 --2- 172.21.15.43:0/3812514393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31c8103c00 0x7f31c819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.999+0000 7f31ce492640 1 --2- 172.21.15.43:0/3812514393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31c81029c0 0x7f31c819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.999+0000 7f31ce492640 1 -- 172.21.15.43:0/3812514393 >> 172.21.15.43:0/3812514393 conn(0x7f31c80fdbd0 msgr2=0x7f31c8109e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:23.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.999+0000 7f31ce492640 1 -- 172.21.15.43:0/3812514393 shutdown_connections 2024-04-25T10:30:23.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:22.999+0000 7f31ce492640 1 -- 172.21.15.43:0/3812514393 wait complete. 2024-04-25T10:30:23.012 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 25 10:30:22 smithi043 podman[68473]: 2024-04-25 10:30:22.728234844 +0000 UTC m=+0.389248156 container start e7c559510af1f8072990ffbc01cfc683465b42e6ce57170a45d2771091344265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:30:23.012 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 25 10:30:22 smithi043 ceph-mds[68512]: -- 172.21.15.43:0/1130648632 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bbabcb7a40 con 0x55bbabcff600 2024-04-25T10:30:23.012 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 25 10:30:22 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-g[68488]: starting mds.g at 2024-04-25T10:30:23.012 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 25 10:30:22 smithi043 bash[68473]: e7c559510af1f8072990ffbc01cfc683465b42e6ce57170a45d2771091344265 2024-04-25T10:30:23.012 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 25 10:30:22 smithi043 systemd[1]: Started Ceph mds.g for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:30:24.250 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:23 smithi043 ceph-mon[29478]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:24.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:24.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:24.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:24.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:24.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:24.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:24.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:23 smithi043 ceph-mon[29478]: Deploying daemon mds.a on smithi043 2024-04-25T10:30:24.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:23 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3812514393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:24.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:23 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.43:6838/3923057448,v1:172.21.15.43:6839/3923057448] up:boot 2024-04-25T10:30:24.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:23 smithi043 ceph-mon[29478]: fsmap 10 up:standby 2024-04-25T10:30:24.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:23 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T10:30:24.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:23 smithi077 ceph-mon[32772]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:24.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:24.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:24.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:24.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:24.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:24.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:24.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:23 smithi077 ceph-mon[32772]: Deploying daemon mds.a on smithi043 2024-04-25T10:30:24.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:23 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3812514393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:24.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:23 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.43:6838/3923057448,v1:172.21.15.43:6839/3923057448] up:boot 2024-04-25T10:30:24.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:23 smithi077 ceph-mon[32772]: fsmap 10 up:standby 2024-04-25T10:30:24.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:23 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T10:30:24.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:23 smithi164 ceph-mon[32713]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:24.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:24.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:24.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:24.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:24.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:24.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:24.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:23 smithi164 ceph-mon[32713]: Deploying daemon mds.a on smithi043 2024-04-25T10:30:24.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:23 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3812514393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:24.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:23 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.43:6838/3923057448,v1:172.21.15.43:6839/3923057448] up:boot 2024-04-25T10:30:24.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:23 smithi164 ceph-mon[32713]: fsmap 10 up:standby 2024-04-25T10:30:24.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:23 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T10:30:25.015 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:25.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.138+0000 7fc97d786640 1 -- 172.21.15.43:0/1740612413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9781045f0 msgr2=0x7fc978106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.138+0000 7fc97d786640 1 --2- 172.21.15.43:0/1740612413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9781045f0 0x7fc978106a10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc968009900 tx=0x7fc96802a410 comp rx=0 tx=0).stop 2024-04-25T10:30:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.138+0000 7fc97d786640 1 -- 172.21.15.43:0/1740612413 shutdown_connections 2024-04-25T10:30:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.138+0000 7fc97d786640 1 --2- 172.21.15.43:0/1740612413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc978106f50 0x7fc978109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.138+0000 7fc97d786640 1 --2- 172.21.15.43:0/1740612413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9781098b0 0x7fc97810bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.138+0000 7fc97d786640 1 --2- 172.21.15.43:0/1740612413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9781045f0 0x7fc978106a10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.138+0000 7fc97d786640 1 -- 172.21.15.43:0/1740612413 >> 172.21.15.43:0/1740612413 conn(0x7fc9780fdbf0 msgr2=0x7fc9781000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.138+0000 7fc97d786640 1 -- 172.21.15.43:0/1740612413 shutdown_connections 2024-04-25T10:30:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.138+0000 7fc97d786640 1 -- 172.21.15.43:0/1740612413 wait complete. 2024-04-25T10:30:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.139+0000 7fc97d786640 1 Processor -- start 2024-04-25T10:30:25.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.139+0000 7fc97d786640 1 -- start start 2024-04-25T10:30:25.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.140+0000 7fc97d786640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9781045f0 0x7fc97819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.140+0000 7fc97d786640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc978106f50 0x7fc97819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.140+0000 7fc97d786640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9781098b0 0x7fc9781a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.140+0000 7fc97d786640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc97810f600 con 0x7fc9781045f0 2024-04-25T10:30:25.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.140+0000 7fc97d786640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc97810f480 con 0x7fc978106f50 2024-04-25T10:30:25.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.140+0000 7fc97d786640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc97810f780 con 0x7fc9781098b0 2024-04-25T10:30:25.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.140+0000 7fc976ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9781045f0 0x7fc97819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.140+0000 7fc976ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9781045f0 0x7fc97819a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44216/0 (socket says 172.21.15.43:44216) 2024-04-25T10:30:25.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.140+0000 7fc976ffd640 1 -- 172.21.15.43:0/2321511495 learned_addr learned my addr 172.21.15.43:0/2321511495 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:25.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.140+0000 7fc9767fc640 1 --2- 172.21.15.43:0/2321511495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc978106f50 0x7fc97819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.140+0000 7fc9777fe640 1 --2- 172.21.15.43:0/2321511495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9781098b0 0x7fc9781a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.140+0000 7fc976ffd640 1 -- 172.21.15.43:0/2321511495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9781098b0 msgr2=0x7fc9781a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.140+0000 7fc976ffd640 1 --2- 172.21.15.43:0/2321511495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9781098b0 0x7fc9781a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.140+0000 7fc976ffd640 1 -- 172.21.15.43:0/2321511495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc978106f50 msgr2=0x7fc97819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.140+0000 7fc976ffd640 1 --2- 172.21.15.43:0/2321511495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc978106f50 0x7fc97819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.140+0000 7fc976ffd640 1 -- 172.21.15.43:0/2321511495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9781a2060 con 0x7fc9781045f0 2024-04-25T10:30:25.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.140+0000 7fc9767fc640 1 --2- 172.21.15.43:0/2321511495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc978106f50 0x7fc97819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:25.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.140+0000 7fc976ffd640 1 --2- 172.21.15.43:0/2321511495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9781045f0 0x7fc97819a520 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc968006010 tx=0x7fc968004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.141+0000 7fc95ffff640 1 -- 172.21.15.43:0/2321511495 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96803f030 con 0x7fc9781045f0 2024-04-25T10:30:25.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.141+0000 7fc95ffff640 1 -- 172.21.15.43:0/2321511495 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc968003aa0 con 0x7fc9781045f0 2024-04-25T10:30:25.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.141+0000 7fc97d786640 1 -- 172.21.15.43:0/2321511495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9781a22f0 con 0x7fc9781045f0 2024-04-25T10:30:25.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.141+0000 7fc95ffff640 1 -- 172.21.15.43:0/2321511495 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc968035070 con 0x7fc9781045f0 2024-04-25T10:30:25.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.141+0000 7fc97d786640 1 -- 172.21.15.43:0/2321511495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9781a2830 con 0x7fc9781045f0 2024-04-25T10:30:25.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.142+0000 7fc97d786640 1 -- 172.21.15.43:0/2321511495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc978069b70 con 0x7fc9781045f0 2024-04-25T10:30:25.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.144+0000 7fc95ffff640 1 -- 172.21.15.43:0/2321511495 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fc968042030 con 0x7fc9781045f0 2024-04-25T10:30:25.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.145+0000 7fc95ffff640 1 --2- 172.21.15.43:0/2321511495 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc9600afb70 0x7fc9600b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.145+0000 7fc9767fc640 1 --2- 172.21.15.43:0/2321511495 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc9600afb70 0x7fc9600b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.145+0000 7fc95ffff640 1 -- 172.21.15.43:0/2321511495 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fc9680ff140 con 0x7fc9781045f0 2024-04-25T10:30:25.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.145+0000 7fc9767fc640 1 --2- 172.21.15.43:0/2321511495 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc9600afb70 0x7fc9600b2080 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc97819b5c0 tx=0x7fc96c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.153+0000 7fc95ffff640 1 -- 172.21.15.43:0/2321511495 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc9680c7b70 con 0x7fc9781045f0 2024-04-25T10:30:25.361 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 25 10:30:25 smithi043 podman[68853]: 2024-04-25 10:30:25.278115114 +0000 UTC m=+0.130208060 container create bf76e4121c2819b808b5d241d42971a5ac60e1831b915fd22ecf8d84fb61899f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T10:30:25.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.395+0000 7fc97d786640 1 -- 172.21.15.43:0/2321511495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9781a2a90 con 0x7fc9781045f0 2024-04-25T10:30:25.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.396+0000 7fc95ffff640 1 -- 172.21.15.43:0/2321511495 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7fc9680cbe70 con 0x7fc9781045f0 2024-04-25T10:30:25.398 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:25.398 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/2595248466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2595248466},{"type":"v1","addr":"172.21.15.77:6837","nonce":2595248466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T10:30:25.399 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 11 2024-04-25T10:30:25.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.398+0000 7fc97d786640 1 -- 172.21.15.43:0/2321511495 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc9600afb70 msgr2=0x7fc9600b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.398+0000 7fc97d786640 1 --2- 172.21.15.43:0/2321511495 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc9600afb70 0x7fc9600b2080 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc97819b5c0 tx=0x7fc96c005f90 comp rx=0 tx=0).stop 2024-04-25T10:30:25.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.398+0000 7fc97d786640 1 -- 172.21.15.43:0/2321511495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9781045f0 msgr2=0x7fc97819a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.398+0000 7fc97d786640 1 --2- 172.21.15.43:0/2321511495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9781045f0 0x7fc97819a520 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc968006010 tx=0x7fc968004270 comp rx=0 tx=0).stop 2024-04-25T10:30:25.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.398+0000 7fc97d786640 1 -- 172.21.15.43:0/2321511495 shutdown_connections 2024-04-25T10:30:25.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.398+0000 7fc97d786640 1 --2- 172.21.15.43:0/2321511495 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc9600afb70 0x7fc9600b2080 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.398+0000 7fc97d786640 1 --2- 172.21.15.43:0/2321511495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc978106f50 0x7fc97819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.398+0000 7fc97d786640 1 --2- 172.21.15.43:0/2321511495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9781098b0 0x7fc9781a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.398+0000 7fc97d786640 1 --2- 172.21.15.43:0/2321511495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9781045f0 0x7fc97819a520 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.398+0000 7fc97d786640 1 -- 172.21.15.43:0/2321511495 >> 172.21.15.43:0/2321511495 conn(0x7fc9780fdbf0 msgr2=0x7fc978105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:25.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.398+0000 7fc97d786640 1 -- 172.21.15.43:0/2321511495 shutdown_connections 2024-04-25T10:30:25.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:25.399+0000 7fc97d786640 1 -- 172.21.15.43:0/2321511495 wait complete. 2024-04-25T10:30:25.661 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 25 10:30:25 smithi043 podman[68853]: 2024-04-25 10:30:25.622652893 +0000 UTC m=+0.474745852 container init bf76e4121c2819b808b5d241d42971a5ac60e1831b915fd22ecf8d84fb61899f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T10:30:25.661 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 25 10:30:25 smithi043 podman[68853]: 2024-04-25 10:30:25.626349559 +0000 UTC m=+0.478442506 container start bf76e4121c2819b808b5d241d42971a5ac60e1831b915fd22ecf8d84fb61899f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T10:30:26.086 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 25 10:30:25 smithi043 ceph-mds[68886]: -- 172.21.15.43:0/2618699426 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564dec19da40 con 0x564dec1e5a80 2024-04-25T10:30:26.086 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 25 10:30:25 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-a[68878]: starting mds.a at 2024-04-25T10:30:26.086 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 25 10:30:25 smithi043 bash[68853]: bf76e4121c2819b808b5d241d42971a5ac60e1831b915fd22ecf8d84fb61899f 2024-04-25T10:30:26.087 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 25 10:30:25 smithi043 systemd[1]: Started Ceph mds.a for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:30:26.087 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:25 smithi043 ceph-mon[29478]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:26.087 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:25 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2321511495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:26.087 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:25 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:26.087 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:25 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:26.087 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:25 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:26.087 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:25 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:26.087 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:25 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:26.087 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:25 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:26.171 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:25 smithi164 ceph-mon[32713]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:26.172 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:25 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2321511495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:26.172 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:25 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:26.172 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:25 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:26.172 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:25 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:26.172 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:25 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:26.172 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:25 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:26.172 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:25 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:26.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:25 smithi077 ceph-mon[32772]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:26.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:25 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2321511495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:26.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:25 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:26.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:25 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:26.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:25 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:26.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:25 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T10:30:26.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:25 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T10:30:26.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:25 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:27.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:26 smithi077 ceph-mon[32772]: Deploying daemon mds.b on smithi077 2024-04-25T10:30:27.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:26 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.43:6840/4254265277,v1:172.21.15.43:6841/4254265277] up:boot 2024-04-25T10:30:27.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:26 smithi077 ceph-mon[32772]: fsmap 11 up:standby 2024-04-25T10:30:27.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:26 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T10:30:27.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:26 smithi164 ceph-mon[32713]: Deploying daemon mds.b on smithi077 2024-04-25T10:30:27.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:26 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.43:6840/4254265277,v1:172.21.15.43:6841/4254265277] up:boot 2024-04-25T10:30:27.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:26 smithi164 ceph-mon[32713]: fsmap 11 up:standby 2024-04-25T10:30:27.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:26 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T10:30:27.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:26 smithi043 ceph-mon[29478]: Deploying daemon mds.b on smithi077 2024-04-25T10:30:27.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:26 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.43:6840/4254265277,v1:172.21.15.43:6841/4254265277] up:boot 2024-04-25T10:30:27.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:26 smithi043 ceph-mon[29478]: fsmap 11 up:standby 2024-04-25T10:30:27.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:26 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T10:30:27.413 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:27.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.539+0000 7f19e8127640 1 -- 172.21.15.43:0/3232169382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e0106f10 msgr2=0x7f19e0109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:27.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.539+0000 7f19e8127640 1 --2- 172.21.15.43:0/3232169382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e0106f10 0x7f19e0109330 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009900 tx=0x7f19d002a410 comp rx=0 tx=0).stop 2024-04-25T10:30:27.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.539+0000 7f19e8127640 1 -- 172.21.15.43:0/3232169382 shutdown_connections 2024-04-25T10:30:27.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.539+0000 7f19e8127640 1 --2- 172.21.15.43:0/3232169382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19e0109870 0x7f19e010bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.539+0000 7f19e8127640 1 --2- 172.21.15.43:0/3232169382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e0106f10 0x7f19e0109330 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.539+0000 7f19e8127640 1 --2- 172.21.15.43:0/3232169382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19e01045b0 0x7f19e01069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.539+0000 7f19e8127640 1 -- 172.21.15.43:0/3232169382 >> 172.21.15.43:0/3232169382 conn(0x7f19e00fdbd0 msgr2=0x7f19e0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:27.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.539+0000 7f19e8127640 1 -- 172.21.15.43:0/3232169382 shutdown_connections 2024-04-25T10:30:27.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.539+0000 7f19e8127640 1 -- 172.21.15.43:0/3232169382 wait complete. 2024-04-25T10:30:27.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.540+0000 7f19e8127640 1 Processor -- start 2024-04-25T10:30:27.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.540+0000 7f19e8127640 1 -- start start 2024-04-25T10:30:27.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.540+0000 7f19e8127640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e01045b0 0x7f19e019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:27.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.540+0000 7f19e8127640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19e0106f10 0x7f19e019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:27.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.540+0000 7f19e8127640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19e0109870 0x7f19e01a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:27.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.540+0000 7f19e8127640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f19e010f5f0 con 0x7f19e01045b0 2024-04-25T10:30:27.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.540+0000 7f19e8127640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f19e010f470 con 0x7f19e0109870 2024-04-25T10:30:27.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.540+0000 7f19e8127640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f19e010f770 con 0x7f19e0106f10 2024-04-25T10:30:27.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.540+0000 7f19e5ec3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e01045b0 0x7f19e019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:27.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.541+0000 7f19e5ec3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e01045b0 0x7f19e019a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44262/0 (socket says 172.21.15.43:44262) 2024-04-25T10:30:27.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.541+0000 7f19e5ec3640 1 -- 172.21.15.43:0/1928518829 learned_addr learned my addr 172.21.15.43:0/1928518829 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:27.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.541+0000 7f19e56c2640 1 --2- 172.21.15.43:0/1928518829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19e0106f10 0x7f19e019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:27.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.541+0000 7f19e66c4640 1 --2- 172.21.15.43:0/1928518829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19e0109870 0x7f19e01a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:27.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.541+0000 7f19e5ec3640 1 -- 172.21.15.43:0/1928518829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19e0106f10 msgr2=0x7f19e019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:27.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.541+0000 7f19e5ec3640 1 --2- 172.21.15.43:0/1928518829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19e0106f10 0x7f19e019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.541+0000 7f19e5ec3640 1 -- 172.21.15.43:0/1928518829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19e0109870 msgr2=0x7f19e01a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:27.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.541+0000 7f19e5ec3640 1 --2- 172.21.15.43:0/1928518829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19e0109870 0x7f19e01a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.541+0000 7f19e5ec3640 1 -- 172.21.15.43:0/1928518829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e01a1d70 con 0x7f19e01045b0 2024-04-25T10:30:27.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.541+0000 7f19e5ec3640 1 --2- 172.21.15.43:0/1928518829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e01045b0 0x7f19e019a230 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f19d800e9f0 tx=0x7f19d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:27.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.541+0000 7f19d6ffd640 1 -- 172.21.15.43:0/1928518829 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d800ccc0 con 0x7f19e01045b0 2024-04-25T10:30:27.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.541+0000 7f19d6ffd640 1 -- 172.21.15.43:0/1928518829 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19d80054d0 con 0x7f19e01045b0 2024-04-25T10:30:27.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.541+0000 7f19e8127640 1 -- 172.21.15.43:0/1928518829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e01a2060 con 0x7f19e01045b0 2024-04-25T10:30:27.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.541+0000 7f19d6ffd640 1 -- 172.21.15.43:0/1928518829 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8010430 con 0x7f19e01045b0 2024-04-25T10:30:27.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.541+0000 7f19e8127640 1 -- 172.21.15.43:0/1928518829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e01a2560 con 0x7f19e01045b0 2024-04-25T10:30:27.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.542+0000 7f19e8127640 1 -- 172.21.15.43:0/1928518829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e00ff6f0 con 0x7f19e01045b0 2024-04-25T10:30:27.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.544+0000 7f19d6ffd640 1 -- 172.21.15.43:0/1928518829 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f19d80046c0 con 0x7f19e01045b0 2024-04-25T10:30:27.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.544+0000 7f19d6ffd640 1 --2- 172.21.15.43:0/1928518829 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f19cc0afaf0 0x7f19cc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:27.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.545+0000 7f19e56c2640 1 --2- 172.21.15.43:0/1928518829 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f19cc0afaf0 0x7f19cc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:27.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.545+0000 7f19d6ffd640 1 -- 172.21.15.43:0/1928518829 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f19d80deae0 con 0x7f19e01045b0 2024-04-25T10:30:27.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.545+0000 7f19e56c2640 1 --2- 172.21.15.43:0/1928518829 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f19cc0afaf0 0x7f19cc0b2000 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009230 tx=0x7f19d0031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:27.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.549+0000 7f19d6ffd640 1 -- 172.21.15.43:0/1928518829 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f19d80a7b60 con 0x7f19e01045b0 2024-04-25T10:30:27.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.772+0000 7f19e8127640 1 -- 172.21.15.43:0/1928518829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19e002db50 con 0x7f19e01045b0 2024-04-25T10:30:27.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.773+0000 7f19d6ffd640 1 -- 172.21.15.43:0/1928518829 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7f19d80a7460 con 0x7f19e01045b0 2024-04-25T10:30:27.775 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:27.776 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/2595248466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2595248466},{"type":"v1","addr":"172.21.15.77:6837","nonce":2595248466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T10:30:27.776 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 12 2024-04-25T10:30:27.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.775+0000 7f19e8127640 1 -- 172.21.15.43:0/1928518829 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f19cc0afaf0 msgr2=0x7f19cc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:27.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.775+0000 7f19e8127640 1 --2- 172.21.15.43:0/1928518829 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f19cc0afaf0 0x7f19cc0b2000 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009230 tx=0x7f19d0031400 comp rx=0 tx=0).stop 2024-04-25T10:30:27.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.775+0000 7f19e8127640 1 -- 172.21.15.43:0/1928518829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e01045b0 msgr2=0x7f19e019a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:27.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.775+0000 7f19e8127640 1 --2- 172.21.15.43:0/1928518829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e01045b0 0x7f19e019a230 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f19d800e9f0 tx=0x7f19d800eeb0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.775+0000 7f19e8127640 1 -- 172.21.15.43:0/1928518829 shutdown_connections 2024-04-25T10:30:27.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.775+0000 7f19e8127640 1 --2- 172.21.15.43:0/1928518829 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f19cc0afaf0 0x7f19cc0b2000 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.775+0000 7f19e8127640 1 --2- 172.21.15.43:0/1928518829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19e0109870 0x7f19e01a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.775+0000 7f19e8127640 1 --2- 172.21.15.43:0/1928518829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19e0106f10 0x7f19e019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.775+0000 7f19e8127640 1 --2- 172.21.15.43:0/1928518829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e01045b0 0x7f19e019a230 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.775+0000 7f19e8127640 1 -- 172.21.15.43:0/1928518829 >> 172.21.15.43:0/1928518829 conn(0x7f19e00fdbd0 msgr2=0x7f19e01051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:27.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.775+0000 7f19e8127640 1 -- 172.21.15.43:0/1928518829 shutdown_connections 2024-04-25T10:30:27.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:27.775+0000 7f19e8127640 1 -- 172.21.15.43:0/1928518829 wait complete. 2024-04-25T10:30:28.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:27 smithi077 ceph-mon[32772]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:28.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:27 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1928518829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:28.041 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:30:27 smithi077 systemd[1]: Starting Ceph mds.b for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:30:28.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:27 smithi043 ceph-mon[29478]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:28.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:27 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1928518829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:28.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:27 smithi164 ceph-mon[32713]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:28.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:27 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1928518829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:28.397 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:30:28 smithi077 podman[63670]: 2024-04-25 10:30:28.040149608 +0000 UTC m=+0.018860757 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:30:28.397 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:30:28 smithi077 podman[63670]: 2024-04-25 10:30:28.142024507 +0000 UTC m=+0.120735627 container create 9922b8342e702b37525e72be1651333e8fb2c700cf26b8962306bba9bc2971e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-b, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T10:30:28.397 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:30:28 smithi077 podman[63670]: 2024-04-25 10:30:28.361108556 +0000 UTC m=+0.339819674 container init 9922b8342e702b37525e72be1651333e8fb2c700cf26b8962306bba9bc2971e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-04-25T10:30:28.397 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:30:28 smithi077 podman[63670]: 2024-04-25 10:30:28.363710086 +0000 UTC m=+0.342421200 container start 9922b8342e702b37525e72be1651333e8fb2c700cf26b8962306bba9bc2971e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T10:30:28.790 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:30:28 smithi077 ceph-mds[63689]: -- 172.21.15.77:0/119549356 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dce7fd7a40 con 0x55dce801f180 2024-04-25T10:30:28.790 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:30:28 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-b[63685]: starting mds.b at 2024-04-25T10:30:28.791 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:30:28 smithi077 bash[63670]: 9922b8342e702b37525e72be1651333e8fb2c700cf26b8962306bba9bc2971e4 2024-04-25T10:30:28.791 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:30:28 smithi077 systemd[1]: Started Ceph mds.b for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:30:29.790 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:29.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:29 smithi077 ceph-mon[32772]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:29.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:29 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:29.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:29 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:29.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:29 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:29.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:29 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:29.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:29 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:29.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:29 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:30:29.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:29 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] up:boot 2024-04-25T10:30:29.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:29 smithi077 ceph-mon[32772]: fsmap 12 up:standby 2024-04-25T10:30:29.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:29 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T10:30:29.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:29 smithi164 ceph-mon[32713]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:29.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:29 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:29.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:29 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:29.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:29 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:29.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:29 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:29.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:29 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:29.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:29 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:30:29.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:29 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] up:boot 2024-04-25T10:30:29.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:29 smithi164 ceph-mon[32713]: fsmap 12 up:standby 2024-04-25T10:30:29.793 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:29 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T10:30:29.793 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:29 smithi043 ceph-mon[29478]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:29.793 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:29 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:29.793 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:29 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:29.793 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:29 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:29.794 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:29 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:29.794 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:29 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:29.794 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:29 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:30:29.794 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:29 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] up:boot 2024-04-25T10:30:29.794 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:29 smithi043 ceph-mon[29478]: fsmap 12 up:standby 2024-04-25T10:30:29.794 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:29 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T10:30:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.904+0000 7f6586d8c640 1 -- 172.21.15.43:0/3782061879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6580103c00 msgr2=0x7f6580104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.904+0000 7f6586d8c640 1 --2- 172.21.15.43:0/3782061879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6580103c00 0x7f6580104010 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f656c009900 tx=0x7f656c02a410 comp rx=0 tx=0).stop 2024-04-25T10:30:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.904+0000 7f6586d8c640 1 -- 172.21.15.43:0/3782061879 shutdown_connections 2024-04-25T10:30:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.904+0000 7f6586d8c640 1 --2- 172.21.15.43:0/3782061879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6580108900 0x7f658010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.904+0000 7f6586d8c640 1 --2- 172.21.15.43:0/3782061879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6580103c00 0x7f6580104010 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.904+0000 7f6586d8c640 1 --2- 172.21.15.43:0/3782061879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65801029c0 0x7f65801036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.904+0000 7f6586d8c640 1 -- 172.21.15.43:0/3782061879 >> 172.21.15.43:0/3782061879 conn(0x7f65800fdbd0 msgr2=0x7f6580100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.904+0000 7f6586d8c640 1 -- 172.21.15.43:0/3782061879 shutdown_connections 2024-04-25T10:30:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.904+0000 7f6586d8c640 1 -- 172.21.15.43:0/3782061879 wait complete. 2024-04-25T10:30:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.905+0000 7f6586d8c640 1 Processor -- start 2024-04-25T10:30:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.905+0000 7f6586d8c640 1 -- start start 2024-04-25T10:30:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.905+0000 7f6586d8c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65801029c0 0x7f658006a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.905+0000 7f6586d8c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6580103c00 0x7f658019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.905+0000 7f6586d8c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6580108900 0x7f65801a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.905+0000 7f6586d8c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f658010e4e0 con 0x7f6580108900 2024-04-25T10:30:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.905+0000 7f6586d8c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f658010e360 con 0x7f65801029c0 2024-04-25T10:30:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.905+0000 7f6586d8c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f658010e660 con 0x7f6580103c00 2024-04-25T10:30:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.905+0000 7f6585329640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6580108900 0x7f65801a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.905+0000 7f6585329640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6580108900 0x7f65801a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38128/0 (socket says 172.21.15.43:38128) 2024-04-25T10:30:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.905+0000 7f6584b28640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65801029c0 0x7f658006a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.905+0000 7f6585329640 1 -- 172.21.15.43:0/3724960554 learned_addr learned my addr 172.21.15.43:0/3724960554 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.905+0000 7f657ffff640 1 --2- 172.21.15.43:0/3724960554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6580103c00 0x7f658019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.906+0000 7f6585329640 1 -- 172.21.15.43:0/3724960554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6580103c00 msgr2=0x7f658019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.906+0000 7f6585329640 1 --2- 172.21.15.43:0/3724960554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6580103c00 0x7f658019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.906+0000 7f6585329640 1 -- 172.21.15.43:0/3724960554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65801029c0 msgr2=0x7f658006a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.906+0000 7f6585329640 1 --2- 172.21.15.43:0/3724960554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65801029c0 0x7f658006a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.906+0000 7f6585329640 1 -- 172.21.15.43:0/3724960554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65801a1d40 con 0x7f6580108900 2024-04-25T10:30:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.906+0000 7f6584b28640 1 --2- 172.21.15.43:0/3724960554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65801029c0 0x7f658006a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.906+0000 7f6585329640 1 --2- 172.21.15.43:0/3724960554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6580108900 0x7f65801a1520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f657800eb10 tx=0x7f657800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.906+0000 7f657dffb640 1 -- 172.21.15.43:0/3724960554 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578004b20 con 0x7f6580108900 2024-04-25T10:30:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.906+0000 7f657dffb640 1 -- 172.21.15.43:0/3724960554 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6578004cc0 con 0x7f6580108900 2024-04-25T10:30:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.906+0000 7f6586d8c640 1 -- 172.21.15.43:0/3724960554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65801a1fd0 con 0x7f6580108900 2024-04-25T10:30:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.906+0000 7f657dffb640 1 -- 172.21.15.43:0/3724960554 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578010670 con 0x7f6580108900 2024-04-25T10:30:29.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.906+0000 7f6586d8c640 1 -- 172.21.15.43:0/3724960554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65801a2490 con 0x7f6580108900 2024-04-25T10:30:29.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.907+0000 7f6586d8c640 1 -- 172.21.15.43:0/3724960554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6580107710 con 0x7f6580108900 2024-04-25T10:30:29.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.909+0000 7f657dffb640 1 -- 172.21.15.43:0/3724960554 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6578005020 con 0x7f6580108900 2024-04-25T10:30:29.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.909+0000 7f657dffb640 1 --2- 172.21.15.43:0/3724960554 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f65700afb70 0x7f65700b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.909+0000 7f6584b28640 1 --2- 172.21.15.43:0/3724960554 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f65700afb70 0x7f65700b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.909+0000 7f657dffb640 1 -- 172.21.15.43:0/3724960554 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f65780dec80 con 0x7f6580108900 2024-04-25T10:30:29.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.910+0000 7f6584b28640 1 --2- 172.21.15.43:0/3724960554 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f65700afb70 0x7f65700b2080 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f65801035c0 tx=0x7f6574005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:29.914+0000 7f657dffb640 1 -- 172.21.15.43:0/3724960554 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65780a7d00 con 0x7f6580108900 2024-04-25T10:30:30.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.147+0000 7f6586d8c640 1 -- 172.21.15.43:0/3724960554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f658002dd70 con 0x7f6580108900 2024-04-25T10:30:30.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.147+0000 7f657dffb640 1 -- 172.21.15.43:0/3724960554 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10482 (secure 0 0 0) 0x7f65780a7600 con 0x7f6580108900 2024-04-25T10:30:30.149 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:30.150 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/2595248466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2595248466},{"type":"v1","addr":"172.21.15.77:6837","nonce":2595248466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T10:30:30.150 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 13 2024-04-25T10:30:30.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.150+0000 7f6586d8c640 1 -- 172.21.15.43:0/3724960554 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f65700afb70 msgr2=0x7f65700b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.150+0000 7f6586d8c640 1 --2- 172.21.15.43:0/3724960554 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f65700afb70 0x7f65700b2080 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f65801035c0 tx=0x7f6574005f90 comp rx=0 tx=0).stop 2024-04-25T10:30:30.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.150+0000 7f6586d8c640 1 -- 172.21.15.43:0/3724960554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6580108900 msgr2=0x7f65801a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.150+0000 7f6586d8c640 1 --2- 172.21.15.43:0/3724960554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6580108900 0x7f65801a1520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f657800eb10 tx=0x7f657800c2d0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.150+0000 7f6586d8c640 1 -- 172.21.15.43:0/3724960554 shutdown_connections 2024-04-25T10:30:30.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.150+0000 7f6586d8c640 1 --2- 172.21.15.43:0/3724960554 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f65700afb70 0x7f65700b2080 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.150+0000 7f6586d8c640 1 --2- 172.21.15.43:0/3724960554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6580108900 0x7f65801a1520 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.150+0000 7f6586d8c640 1 --2- 172.21.15.43:0/3724960554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6580103c00 0x7f658019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.150+0000 7f6586d8c640 1 --2- 172.21.15.43:0/3724960554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65801029c0 0x7f658006a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.150+0000 7f6586d8c640 1 -- 172.21.15.43:0/3724960554 >> 172.21.15.43:0/3724960554 conn(0x7f65800fdbd0 msgr2=0x7f65801048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:30.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.150+0000 7f6586d8c640 1 -- 172.21.15.43:0/3724960554 shutdown_connections 2024-04-25T10:30:30.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.150+0000 7f6586d8c640 1 -- 172.21.15.43:0/3724960554 wait complete. 2024-04-25T10:30:30.167 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:30.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.304+0000 7fe1251c5640 1 -- 172.21.15.43:0/3993484605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1201033a0 msgr2=0x7fe1201037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.304+0000 7fe1251c5640 1 --2- 172.21.15.43:0/3993484605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1201033a0 0x7fe1201037b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe11400aa10 tx=0x7fe11402b720 comp rx=0 tx=0).stop 2024-04-25T10:30:30.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.305+0000 7fe1251c5640 1 -- 172.21.15.43:0/3993484605 shutdown_connections 2024-04-25T10:30:30.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.305+0000 7fe1251c5640 1 --2- 172.21.15.43:0/3993484605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1201080a0 0x7fe12010a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.305+0000 7fe1251c5640 1 --2- 172.21.15.43:0/3993484605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1201033a0 0x7fe1201037b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.305+0000 7fe1251c5640 1 --2- 172.21.15.43:0/3993484605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe120102160 0x7fe120102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.305+0000 7fe1251c5640 1 -- 172.21.15.43:0/3993484605 >> 172.21.15.43:0/3993484605 conn(0x7fe1200fd330 msgr2=0x7fe1200ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:30.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.305+0000 7fe1251c5640 1 -- 172.21.15.43:0/3993484605 shutdown_connections 2024-04-25T10:30:30.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.305+0000 7fe1251c5640 1 -- 172.21.15.43:0/3993484605 wait complete. 2024-04-25T10:30:30.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.306+0000 7fe1251c5640 1 Processor -- start 2024-04-25T10:30:30.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.306+0000 7fe1251c5640 1 -- start start 2024-04-25T10:30:30.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.306+0000 7fe1251c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe120102160 0x7fe120198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.306+0000 7fe1251c5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1201033a0 0x7fe120198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.306+0000 7fe1251c5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1201080a0 0x7fe12019f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.306+0000 7fe1251c5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe12010d3f0 con 0x7fe120102160 2024-04-25T10:30:30.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.306+0000 7fe1251c5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe12010d270 con 0x7fe1201080a0 2024-04-25T10:30:30.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.306+0000 7fe1251c5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe12010d570 con 0x7fe1201033a0 2024-04-25T10:30:30.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.307+0000 7fe11ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe120102160 0x7fe120198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.307+0000 7fe11ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe120102160 0x7fe120198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38148/0 (socket says 172.21.15.43:38148) 2024-04-25T10:30:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.307+0000 7fe11ed9d640 1 -- 172.21.15.43:0/2904885556 learned_addr learned my addr 172.21.15.43:0/2904885556 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.307+0000 7fe11f59e640 1 --2- 172.21.15.43:0/2904885556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1201080a0 0x7fe12019f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.307+0000 7fe11e59c640 1 --2- 172.21.15.43:0/2904885556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1201033a0 0x7fe120198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.307+0000 7fe11f59e640 1 -- 172.21.15.43:0/2904885556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1201033a0 msgr2=0x7fe120198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.307+0000 7fe11f59e640 1 --2- 172.21.15.43:0/2904885556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1201033a0 0x7fe120198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.307+0000 7fe11f59e640 1 -- 172.21.15.43:0/2904885556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe120102160 msgr2=0x7fe120198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.307+0000 7fe11f59e640 1 --2- 172.21.15.43:0/2904885556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe120102160 0x7fe120198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.307+0000 7fe11f59e640 1 -- 172.21.15.43:0/2904885556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12019fb40 con 0x7fe1201080a0 2024-04-25T10:30:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.307+0000 7fe11e59c640 1 --2- 172.21.15.43:0/2904885556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1201033a0 0x7fe120198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.307+0000 7fe11f59e640 1 --2- 172.21.15.43:0/2904885556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1201080a0 0x7fe12019f4d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe118009fd0 tx=0x7fe11800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.307+0000 7fe107fff640 1 -- 172.21.15.43:0/2904885556 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11800ce20 con 0x7fe1201080a0 2024-04-25T10:30:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.307+0000 7fe107fff640 1 -- 172.21.15.43:0/2904885556 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe118005020 con 0x7fe1201080a0 2024-04-25T10:30:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.307+0000 7fe1251c5640 1 -- 172.21.15.43:0/2904885556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12019fe30 con 0x7fe1201080a0 2024-04-25T10:30:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.307+0000 7fe107fff640 1 -- 172.21.15.43:0/2904885556 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118010690 con 0x7fe1201080a0 2024-04-25T10:30:30.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.307+0000 7fe1251c5640 1 -- 172.21.15.43:0/2904885556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1201a0290 con 0x7fe1201080a0 2024-04-25T10:30:30.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.309+0000 7fe1251c5640 1 -- 172.21.15.43:0/2904885556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe120069310 con 0x7fe1201080a0 2024-04-25T10:30:30.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.310+0000 7fe107fff640 1 -- 172.21.15.43:0/2904885556 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fe118010830 con 0x7fe1201080a0 2024-04-25T10:30:30.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.311+0000 7fe107fff640 1 --2- 172.21.15.43:0/2904885556 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fe1080afb70 0x7fe1080b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.311+0000 7fe107fff640 1 -- 172.21.15.43:0/2904885556 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fe1180dedd0 con 0x7fe1201080a0 2024-04-25T10:30:30.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.311+0000 7fe11ed9d640 1 --2- 172.21.15.43:0/2904885556 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fe1080afb70 0x7fe1080b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.312+0000 7fe11ed9d640 1 --2- 172.21.15.43:0/2904885556 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fe1080afb70 0x7fe1080b2080 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe110009c30 tx=0x7fe110005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:30.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.316+0000 7fe107fff640 1 -- 172.21.15.43:0/2904885556 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe1180a7e50 con 0x7fe1201080a0 2024-04-25T10:30:30.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.514+0000 7fe1251c5640 1 -- 172.21.15.43:0/2904885556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe12002db50 con 0x7fe1201080a0 2024-04-25T10:30:30.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.515+0000 7fe107fff640 1 -- 172.21.15.43:0/2904885556 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+18422 (secure 0 0 0) 0x7fe1180a7750 con 0x7fe1201080a0 2024-04-25T10:30:30.517 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:30.518 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":73,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:29:44.632016+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:30.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.518+0000 7fe1251c5640 1 -- 172.21.15.43:0/2904885556 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fe1080afb70 msgr2=0x7fe1080b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.518+0000 7fe1251c5640 1 --2- 172.21.15.43:0/2904885556 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fe1080afb70 0x7fe1080b2080 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe110009c30 tx=0x7fe110005f90 comp rx=0 tx=0).stop 2024-04-25T10:30:30.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.518+0000 7fe1251c5640 1 -- 172.21.15.43:0/2904885556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1201080a0 msgr2=0x7fe12019f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.518+0000 7fe1251c5640 1 --2- 172.21.15.43:0/2904885556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1201080a0 0x7fe12019f4d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe118009fd0 tx=0x7fe11800ef10 comp rx=0 tx=0).stop 2024-04-25T10:30:30.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.518+0000 7fe1251c5640 1 -- 172.21.15.43:0/2904885556 shutdown_connections 2024-04-25T10:30:30.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.518+0000 7fe1251c5640 1 --2- 172.21.15.43:0/2904885556 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fe1080afb70 0x7fe1080b2080 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.518+0000 7fe1251c5640 1 --2- 172.21.15.43:0/2904885556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1201080a0 0x7fe12019f4d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.518+0000 7fe1251c5640 1 --2- 172.21.15.43:0/2904885556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1201033a0 0x7fe120198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.518+0000 7fe1251c5640 1 --2- 172.21.15.43:0/2904885556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe120102160 0x7fe120198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.518+0000 7fe1251c5640 1 -- 172.21.15.43:0/2904885556 >> 172.21.15.43:0/2904885556 conn(0x7fe1200fd330 msgr2=0x7fe120104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:30.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.518+0000 7fe1251c5640 1 -- 172.21.15.43:0/2904885556 shutdown_connections 2024-04-25T10:30:30.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.518+0000 7fe1251c5640 1 -- 172.21.15.43:0/2904885556 wait complete. 2024-04-25T10:30:30.531 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:25:58.167476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:30:30.532 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:30:30.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.644+0000 7f2a45805640 1 -- 172.21.15.43:0/1913961517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a40101760 msgr2=0x7f2a40101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.644+0000 7f2a45805640 1 --2- 172.21.15.43:0/1913961517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a40101760 0x7f2a40101bd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009900 tx=0x7f2a3402a410 comp rx=0 tx=0).stop 2024-04-25T10:30:30.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.644+0000 7f2a45805640 1 -- 172.21.15.43:0/1913961517 shutdown_connections 2024-04-25T10:30:30.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.644+0000 7f2a45805640 1 --2- 172.21.15.43:0/1913961517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a40102260 0x7f2a4010bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.644+0000 7f2a45805640 1 --2- 172.21.15.43:0/1913961517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a40101760 0x7f2a40101bd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.644+0000 7f2a45805640 1 --2- 172.21.15.43:0/1913961517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a401050b0 0x7f2a401054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.644+0000 7f2a45805640 1 -- 172.21.15.43:0/1913961517 >> 172.21.15.43:0/1913961517 conn(0x7f2a400fd380 msgr2=0x7f2a400ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:30.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.644+0000 7f2a45805640 1 -- 172.21.15.43:0/1913961517 shutdown_connections 2024-04-25T10:30:30.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.644+0000 7f2a45805640 1 -- 172.21.15.43:0/1913961517 wait complete. 2024-04-25T10:30:30.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.645+0000 7f2a45805640 1 Processor -- start 2024-04-25T10:30:30.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.645+0000 7f2a45805640 1 -- start start 2024-04-25T10:30:30.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.645+0000 7f2a45805640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a40101760 0x7f2a4019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.645+0000 7f2a45805640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a40102260 0x7f2a4019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.645+0000 7f2a45805640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a401050b0 0x7f2a401a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.645+0000 7f2a45805640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4010f640 con 0x7f2a40102260 2024-04-25T10:30:30.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.645+0000 7f2a45805640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4010f4c0 con 0x7f2a40101760 2024-04-25T10:30:30.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.645+0000 7f2a45805640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4010f7c0 con 0x7f2a401050b0 2024-04-25T10:30:30.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.645+0000 7f2a3e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a40102260 0x7f2a4019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.645+0000 7f2a3e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a40102260 0x7f2a4019a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38176/0 (socket says 172.21.15.43:38176) 2024-04-25T10:30:30.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.645+0000 7f2a3e7fc640 1 -- 172.21.15.43:0/355679870 learned_addr learned my addr 172.21.15.43:0/355679870 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:30.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.645+0000 7f2a3effd640 1 --2- 172.21.15.43:0/355679870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a40101760 0x7f2a4019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.646+0000 7f2a3f7fe640 1 --2- 172.21.15.43:0/355679870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a401050b0 0x7f2a401a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.646+0000 7f2a3e7fc640 1 -- 172.21.15.43:0/355679870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a401050b0 msgr2=0x7f2a401a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.646+0000 7f2a3e7fc640 1 --2- 172.21.15.43:0/355679870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a401050b0 0x7f2a401a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.646+0000 7f2a3e7fc640 1 -- 172.21.15.43:0/355679870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a40101760 msgr2=0x7f2a4019a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.646+0000 7f2a3e7fc640 1 --2- 172.21.15.43:0/355679870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a40101760 0x7f2a4019a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.646+0000 7f2a3e7fc640 1 -- 172.21.15.43:0/355679870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a401a1db0 con 0x7f2a40102260 2024-04-25T10:30:30.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.646+0000 7f2a3e7fc640 1 --2- 172.21.15.43:0/355679870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a40102260 0x7f2a4019a7b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2a34006010 tx=0x7f2a34031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:30.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.646+0000 7f2a44803640 1 -- 172.21.15.43:0/355679870 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3403f030 con 0x7f2a40102260 2024-04-25T10:30:30.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.646+0000 7f2a44803640 1 -- 172.21.15.43:0/355679870 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a34003aa0 con 0x7f2a40102260 2024-04-25T10:30:30.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.646+0000 7f2a45805640 1 -- 172.21.15.43:0/355679870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a401a2040 con 0x7f2a40102260 2024-04-25T10:30:30.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.646+0000 7f2a44803640 1 -- 172.21.15.43:0/355679870 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a34035070 con 0x7f2a40102260 2024-04-25T10:30:30.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.646+0000 7f2a45805640 1 -- 172.21.15.43:0/355679870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a401a2530 con 0x7f2a40102260 2024-04-25T10:30:30.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.647+0000 7f2a45805640 1 -- 172.21.15.43:0/355679870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a40108a40 con 0x7f2a40102260 2024-04-25T10:30:30.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.649+0000 7f2a44803640 1 -- 172.21.15.43:0/355679870 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f2a34042030 con 0x7f2a40102260 2024-04-25T10:30:30.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.650+0000 7f2a44803640 1 --2- 172.21.15.43:0/355679870 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2a280afaf0 0x7f2a280b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.650+0000 7f2a3effd640 1 --2- 172.21.15.43:0/355679870 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2a280afaf0 0x7f2a280b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.650+0000 7f2a44803640 1 -- 172.21.15.43:0/355679870 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f2a340ff360 con 0x7f2a40102260 2024-04-25T10:30:30.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.650+0000 7f2a3effd640 1 --2- 172.21.15.43:0/355679870 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2a280afaf0 0x7f2a280b2000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2a4006b2d0 tx=0x7f2a30005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:30.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.654+0000 7f2a44803640 1 -- 172.21.15.43:0/355679870 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a340c7d70 con 0x7f2a40102260 2024-04-25T10:30:30.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:30 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3724960554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:30.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:30 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:30.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:30 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:30.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:30 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:30.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:30 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:30.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:30 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:30.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:30 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3724960554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:30.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:30 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:30.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:30 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:30.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:30 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:30.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:30 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:30.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:30 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:30.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.841+0000 7f2a45805640 1 -- 172.21.15.43:0/355679870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2a4002dce0 con 0x7f2a40102260 2024-04-25T10:30:30.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.841+0000 7f2a44803640 1 -- 172.21.15.43:0/355679870 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7f2a340cc070 con 0x7f2a40102260 2024-04-25T10:30:30.844 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T10:30:30.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.843+0000 7f2a45805640 1 -- 172.21.15.43:0/355679870 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2a280afaf0 msgr2=0x7f2a280b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.843+0000 7f2a45805640 1 --2- 172.21.15.43:0/355679870 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2a280afaf0 0x7f2a280b2000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2a4006b2d0 tx=0x7f2a30005f90 comp rx=0 tx=0).stop 2024-04-25T10:30:30.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.843+0000 7f2a45805640 1 -- 172.21.15.43:0/355679870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a40102260 msgr2=0x7f2a4019a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.843+0000 7f2a45805640 1 --2- 172.21.15.43:0/355679870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a40102260 0x7f2a4019a7b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2a34006010 tx=0x7f2a34031a50 comp rx=0 tx=0).stop 2024-04-25T10:30:30.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.844+0000 7f2a45805640 1 -- 172.21.15.43:0/355679870 shutdown_connections 2024-04-25T10:30:30.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.844+0000 7f2a45805640 1 --2- 172.21.15.43:0/355679870 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2a280afaf0 0x7f2a280b2000 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.844+0000 7f2a45805640 1 --2- 172.21.15.43:0/355679870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a401050b0 0x7f2a401a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.844+0000 7f2a45805640 1 --2- 172.21.15.43:0/355679870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a40102260 0x7f2a4019a7b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.844+0000 7f2a45805640 1 --2- 172.21.15.43:0/355679870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a40101760 0x7f2a4019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.844+0000 7f2a45805640 1 -- 172.21.15.43:0/355679870 >> 172.21.15.43:0/355679870 conn(0x7f2a400fd380 msgr2=0x7f2a40105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:30.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.844+0000 7f2a45805640 1 -- 172.21.15.43:0/355679870 shutdown_connections 2024-04-25T10:30:30.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.844+0000 7f2a45805640 1 -- 172.21.15.43:0/355679870 wait complete. 2024-04-25T10:30:30.856 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:30 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3724960554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:30.856 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:30 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:30.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:30 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:30.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:30 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:30.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:30 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:30.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:30 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:30.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:30:30.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.969+0000 7f9621dc7640 1 -- 172.21.15.43:0/1227213132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f961c109870 msgr2=0x7f961c10bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.969+0000 7f9621dc7640 1 --2- 172.21.15.43:0/1227213132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f961c109870 0x7f961c10bc90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9614009ca0 tx=0x7f961402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.969+0000 7f9621dc7640 1 -- 172.21.15.43:0/1227213132 shutdown_connections 2024-04-25T10:30:30.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.969+0000 7f9621dc7640 1 --2- 172.21.15.43:0/1227213132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f961c109870 0x7f961c10bc90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.969+0000 7f9621dc7640 1 --2- 172.21.15.43:0/1227213132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f961c106f10 0x7f961c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.969+0000 7f9621dc7640 1 --2- 172.21.15.43:0/1227213132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f961c1045b0 0x7f961c1069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.969+0000 7f9621dc7640 1 -- 172.21.15.43:0/1227213132 >> 172.21.15.43:0/1227213132 conn(0x7f961c0fdbd0 msgr2=0x7f961c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:30.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.969+0000 7f9621dc7640 1 -- 172.21.15.43:0/1227213132 shutdown_connections 2024-04-25T10:30:30.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.969+0000 7f9621dc7640 1 -- 172.21.15.43:0/1227213132 wait complete. 2024-04-25T10:30:30.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.969+0000 7f9621dc7640 1 Processor -- start 2024-04-25T10:30:30.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.970+0000 7f9621dc7640 1 -- start start 2024-04-25T10:30:30.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.970+0000 7f9621dc7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f961c1045b0 0x7f961c19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.970+0000 7f9621dc7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f961c106f10 0x7f961c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.970+0000 7f9621dc7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f961c109870 0x7f961c1a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.970+0000 7f9621dc7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f961c10f5d0 con 0x7f961c106f10 2024-04-25T10:30:30.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.970+0000 7f9621dc7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f961c10f450 con 0x7f961c1045b0 2024-04-25T10:30:30.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.970+0000 7f9621dc7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f961c10f750 con 0x7f961c109870 2024-04-25T10:30:30.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.970+0000 7f961affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f961c106f10 0x7f961c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.970+0000 7f961affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f961c106f10 0x7f961c19a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38188/0 (socket says 172.21.15.43:38188) 2024-04-25T10:30:30.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.970+0000 7f961affd640 1 -- 172.21.15.43:0/3115531954 learned_addr learned my addr 172.21.15.43:0/3115531954 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:30.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.970+0000 7f961b7fe640 1 --2- 172.21.15.43:0/3115531954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f961c1045b0 0x7f961c19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.970+0000 7f961bfff640 1 --2- 172.21.15.43:0/3115531954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f961c109870 0x7f961c1a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.970+0000 7f961affd640 1 -- 172.21.15.43:0/3115531954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f961c109870 msgr2=0x7f961c1a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.970+0000 7f961affd640 1 --2- 172.21.15.43:0/3115531954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f961c109870 0x7f961c1a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.970+0000 7f961affd640 1 -- 172.21.15.43:0/3115531954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f961c1045b0 msgr2=0x7f961c19a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.970+0000 7f961affd640 1 --2- 172.21.15.43:0/3115531954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f961c1045b0 0x7f961c19a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.970+0000 7f961affd640 1 -- 172.21.15.43:0/3115531954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961c1a1c40 con 0x7f961c106f10 2024-04-25T10:30:30.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.970+0000 7f961bfff640 1 --2- 172.21.15.43:0/3115531954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f961c109870 0x7f961c1a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:30.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.970+0000 7f961affd640 1 --2- 172.21.15.43:0/3115531954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f961c106f10 0x7f961c19a6f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f960c00c920 tx=0x7f960c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.970+0000 7f9618ff9640 1 -- 172.21.15.43:0/3115531954 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c007c70 con 0x7f961c106f10 2024-04-25T10:30:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.971+0000 7f9618ff9640 1 -- 172.21.15.43:0/3115531954 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f960c007e10 con 0x7f961c106f10 2024-04-25T10:30:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.971+0000 7f9621dc7640 1 -- 172.21.15.43:0/3115531954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f961c1a1f30 con 0x7f961c106f10 2024-04-25T10:30:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.971+0000 7f9618ff9640 1 -- 172.21.15.43:0/3115531954 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c018820 con 0x7f961c106f10 2024-04-25T10:30:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.971+0000 7f9621dc7640 1 -- 172.21.15.43:0/3115531954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f961c1a23a0 con 0x7f961c106f10 2024-04-25T10:30:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.971+0000 7f9621dc7640 1 -- 172.21.15.43:0/3115531954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f961c0ff6f0 con 0x7f961c106f10 2024-04-25T10:30:30.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.973+0000 7f9618ff9640 1 -- 172.21.15.43:0/3115531954 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f960c00f450 con 0x7f961c106f10 2024-04-25T10:30:30.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.974+0000 7f9618ff9640 1 --2- 172.21.15.43:0/3115531954 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f96040afaf0 0x7f96040b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.974+0000 7f961b7fe640 1 --2- 172.21.15.43:0/3115531954 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f96040afaf0 0x7f96040b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.974+0000 7f9618ff9640 1 -- 172.21.15.43:0/3115531954 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f960c0deb60 con 0x7f961c106f10 2024-04-25T10:30:30.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.974+0000 7f961b7fe640 1 --2- 172.21.15.43:0/3115531954 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f96040afaf0 0x7f96040b2000 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f96100095f0 tx=0x7f9610005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:30.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:30.977+0000 7f9618ff9640 1 -- 172.21.15.43:0/3115531954 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f960c014030 con 0x7f961c106f10 2024-04-25T10:30:31.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.169+0000 7f9621dc7640 1 -- 172.21.15.43:0/3115531954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f961c02dda0 con 0x7f961c106f10 2024-04-25T10:30:31.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.170+0000 7f9618ff9640 1 -- 172.21.15.43:0/3115531954 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) ==== 84+0+65 (secure 0 0 0) 0x7f960c0a7c70 con 0x7f961c106f10 2024-04-25T10:30:31.172 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:31.172 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-04-25T10:30:31.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:30:31.172 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-04-25T10:30:31.172 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-04-25T10:30:31.172 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:30:31.172 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-04-25T10:30:31.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.172+0000 7f9621dc7640 1 -- 172.21.15.43:0/3115531954 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f96040afaf0 msgr2=0x7f96040b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.172+0000 7f9621dc7640 1 --2- 172.21.15.43:0/3115531954 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f96040afaf0 0x7f96040b2000 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f96100095f0 tx=0x7f9610005f90 comp rx=0 tx=0).stop 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.172+0000 7f9621dc7640 1 -- 172.21.15.43:0/3115531954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f961c106f10 msgr2=0x7f961c19a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.172+0000 7f9621dc7640 1 --2- 172.21.15.43:0/3115531954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f961c106f10 0x7f961c19a6f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f960c00c920 tx=0x7f960c00cde0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.172+0000 7f9621dc7640 1 -- 172.21.15.43:0/3115531954 shutdown_connections 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.172+0000 7f9621dc7640 1 --2- 172.21.15.43:0/3115531954 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f96040afaf0 0x7f96040b2000 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.172+0000 7f9621dc7640 1 --2- 172.21.15.43:0/3115531954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f961c109870 0x7f961c1a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.172+0000 7f9621dc7640 1 --2- 172.21.15.43:0/3115531954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f961c106f10 0x7f961c19a6f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.172+0000 7f9621dc7640 1 --2- 172.21.15.43:0/3115531954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f961c1045b0 0x7f961c19a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.172+0000 7f9621dc7640 1 -- 172.21.15.43:0/3115531954 >> 172.21.15.43:0/3115531954 conn(0x7f961c0fdbd0 msgr2=0x7f961c1054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.172+0000 7f9621dc7640 1 -- 172.21.15.43:0/3115531954 shutdown_connections 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.172+0000 7f9621dc7640 1 -- 172.21.15.43:0/3115531954 wait complete. 2024-04-25T10:30:31.186 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:30:31.186 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:30:31.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.302+0000 7f54690e0640 1 -- 172.21.15.43:0/1053672249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54641070b0 msgr2=0x7f54641094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.302+0000 7f54690e0640 1 --2- 172.21.15.43:0/1053672249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54641070b0 0x7f54641094d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5458009900 tx=0x7f545802a410 comp rx=0 tx=0).stop 2024-04-25T10:30:31.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.302+0000 7f54690e0640 1 -- 172.21.15.43:0/1053672249 shutdown_connections 2024-04-25T10:30:31.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.302+0000 7f54690e0640 1 --2- 172.21.15.43:0/1053672249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5464109a10 0x7f546410be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.302+0000 7f54690e0640 1 --2- 172.21.15.43:0/1053672249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54641070b0 0x7f54641094d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.302+0000 7f54690e0640 1 --2- 172.21.15.43:0/1053672249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5464101f00 0x7f5464106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.302+0000 7f54690e0640 1 -- 172.21.15.43:0/1053672249 >> 172.21.15.43:0/1053672249 conn(0x7f54640fd340 msgr2=0x7f54640ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:31.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.302+0000 7f54690e0640 1 -- 172.21.15.43:0/1053672249 shutdown_connections 2024-04-25T10:30:31.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.302+0000 7f54690e0640 1 -- 172.21.15.43:0/1053672249 wait complete. 2024-04-25T10:30:31.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.302+0000 7f54690e0640 1 Processor -- start 2024-04-25T10:30:31.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.303+0000 7f54690e0640 1 -- start start 2024-04-25T10:30:31.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.303+0000 7f54690e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5464101f00 0x7f546419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.303+0000 7f54690e0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54641070b0 0x7f546419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.303+0000 7f54690e0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5464109a10 0x7f54641a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.303+0000 7f54690e0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f546410f6b0 con 0x7f5464101f00 2024-04-25T10:30:31.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.303+0000 7f54690e0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f546410f530 con 0x7f5464109a10 2024-04-25T10:30:31.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.303+0000 7f54690e0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f546410f830 con 0x7f54641070b0 2024-04-25T10:30:31.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.303+0000 7f5462d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5464101f00 0x7f546419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.303+0000 7f5462d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5464101f00 0x7f546419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38202/0 (socket says 172.21.15.43:38202) 2024-04-25T10:30:31.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.303+0000 7f5462d9d640 1 -- 172.21.15.43:0/4101532524 learned_addr learned my addr 172.21.15.43:0/4101532524 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:31.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.303+0000 7f546259c640 1 --2- 172.21.15.43:0/4101532524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54641070b0 0x7f546419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.303+0000 7f546359e640 1 --2- 172.21.15.43:0/4101532524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5464109a10 0x7f54641a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.303+0000 7f5462d9d640 1 -- 172.21.15.43:0/4101532524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54641070b0 msgr2=0x7f546419a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.303+0000 7f5462d9d640 1 --2- 172.21.15.43:0/4101532524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54641070b0 0x7f546419a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.303+0000 7f5462d9d640 1 -- 172.21.15.43:0/4101532524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5464109a10 msgr2=0x7f54641a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.303+0000 7f5462d9d640 1 --2- 172.21.15.43:0/4101532524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5464109a10 0x7f54641a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.303+0000 7f5462d9d640 1 -- 172.21.15.43:0/4101532524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54641a1df0 con 0x7f5464101f00 2024-04-25T10:30:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.303+0000 7f546259c640 1 --2- 172.21.15.43:0/4101532524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54641070b0 0x7f546419a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.303+0000 7f546359e640 1 --2- 172.21.15.43:0/4101532524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5464109a10 0x7f54641a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.304+0000 7f5462d9d640 1 --2- 172.21.15.43:0/4101532524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5464101f00 0x7f546419a2b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f545400e9c0 tx=0x7f545400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.304+0000 7f544bfff640 1 -- 172.21.15.43:0/4101532524 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545400cd80 con 0x7f5464101f00 2024-04-25T10:30:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.304+0000 7f54690e0640 1 -- 172.21.15.43:0/4101532524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54641a20e0 con 0x7f5464101f00 2024-04-25T10:30:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.304+0000 7f544bfff640 1 -- 172.21.15.43:0/4101532524 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54540054d0 con 0x7f5464101f00 2024-04-25T10:30:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.304+0000 7f544bfff640 1 -- 172.21.15.43:0/4101532524 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5454010430 con 0x7f5464101f00 2024-04-25T10:30:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.304+0000 7f54690e0640 1 -- 172.21.15.43:0/4101532524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54641a2630 con 0x7f5464101f00 2024-04-25T10:30:31.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.305+0000 7f54690e0640 1 -- 172.21.15.43:0/4101532524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54640fee60 con 0x7f5464101f00 2024-04-25T10:30:31.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.307+0000 7f544bfff640 1 -- 172.21.15.43:0/4101532524 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f5454004fb0 con 0x7f5464101f00 2024-04-25T10:30:31.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.307+0000 7f544bfff640 1 --2- 172.21.15.43:0/4101532524 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f544c0afaf0 0x7f544c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.307+0000 7f546259c640 1 --2- 172.21.15.43:0/4101532524 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f544c0afaf0 0x7f544c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.307+0000 7f544bfff640 1 -- 172.21.15.43:0/4101532524 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f54540dede0 con 0x7f5464101f00 2024-04-25T10:30:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.308+0000 7f546259c640 1 --2- 172.21.15.43:0/4101532524 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f544c0afaf0 0x7f544c0b2000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5458009230 tx=0x7f5458031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:31.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.311+0000 7f544bfff640 1 -- 172.21.15.43:0/4101532524 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5454053ff0 con 0x7f5464101f00 2024-04-25T10:30:31.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.498+0000 7f54690e0640 1 -- 172.21.15.43:0/4101532524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f546402db50 con 0x7f5464101f00 2024-04-25T10:30:31.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:31 smithi077 ceph-mon[32772]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:31.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:31 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:31.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:31 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2904885556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:30:31.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:31 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/355679870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:30:31.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:31 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:31.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:31 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:30:31.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:31 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:31.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:31 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:30:31.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:31 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3115531954' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T10:30:31.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:31 smithi164 ceph-mon[32713]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:31.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:31 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:31.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:31 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2904885556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:30:31.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:31 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/355679870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:30:31.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:31 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:31.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:31 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:30:31.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:31 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:31.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:31 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:30:31.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:31 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3115531954' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T10:30:31.848 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:31 smithi043 ceph-mon[29478]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:31.848 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:31 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:31.848 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:31 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2904885556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:30:31.848 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:31 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/355679870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:30:31.848 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:31 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:31.848 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:31 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:30:31.849 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:31 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:31.849 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:31 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:30:31.849 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:31 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3115531954' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T10:30:31.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.879+0000 7f544bfff640 1 -- 172.21.15.43:0/4101532524 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) ==== 138+0+0 (secure 0 0 0) 0x7f54540a7dc0 con 0x7f5464101f00 2024-04-25T10:30:31.882 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-04-25T10:30:31.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.882+0000 7f54690e0640 1 -- 172.21.15.43:0/4101532524 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f544c0afaf0 msgr2=0x7f544c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.882+0000 7f54690e0640 1 --2- 172.21.15.43:0/4101532524 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f544c0afaf0 0x7f544c0b2000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5458009230 tx=0x7f5458031400 comp rx=0 tx=0).stop 2024-04-25T10:30:31.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.882+0000 7f54690e0640 1 -- 172.21.15.43:0/4101532524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5464101f00 msgr2=0x7f546419a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.882+0000 7f54690e0640 1 --2- 172.21.15.43:0/4101532524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5464101f00 0x7f546419a2b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f545400e9c0 tx=0x7f545400ee80 comp rx=0 tx=0).stop 2024-04-25T10:30:31.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.882+0000 7f54690e0640 1 -- 172.21.15.43:0/4101532524 shutdown_connections 2024-04-25T10:30:31.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.882+0000 7f54690e0640 1 --2- 172.21.15.43:0/4101532524 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f544c0afaf0 0x7f544c0b2000 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.882+0000 7f54690e0640 1 --2- 172.21.15.43:0/4101532524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5464109a10 0x7f54641a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.882+0000 7f54690e0640 1 --2- 172.21.15.43:0/4101532524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54641070b0 0x7f546419a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.882+0000 7f54690e0640 1 --2- 172.21.15.43:0/4101532524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5464101f00 0x7f546419a2b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.882+0000 7f54690e0640 1 -- 172.21.15.43:0/4101532524 >> 172.21.15.43:0/4101532524 conn(0x7f54640fd340 msgr2=0x7f5464105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:31.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.882+0000 7f54690e0640 1 -- 172.21.15.43:0/4101532524 shutdown_connections 2024-04-25T10:30:31.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:31.882+0000 7f54690e0640 1 -- 172.21.15.43:0/4101532524 wait complete. 2024-04-25T10:30:31.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:30:32.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.020+0000 7f331e851640 1 -- 172.21.15.43:0/1026228344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3318104670 msgr2=0x7f331810adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.020+0000 7f331e851640 1 --2- 172.21.15.43:0/1026228344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3318104670 0x7f331810adc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3310009ca0 tx=0x7f331002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.020+0000 7f331e851640 1 -- 172.21.15.43:0/1026228344 shutdown_connections 2024-04-25T10:30:32.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.020+0000 7f331e851640 1 --2- 172.21.15.43:0/1026228344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3318104670 0x7f331810adc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.020+0000 7f331e851640 1 --2- 172.21.15.43:0/1026228344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3318103b10 0x7f3318103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.020+0000 7f331e851640 1 --2- 172.21.15.43:0/1026228344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3318102fb0 0x7f33181033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.020+0000 7f331e851640 1 -- 172.21.15.43:0/1026228344 >> 172.21.15.43:0/1026228344 conn(0x7f33180fd330 msgr2=0x7f33180ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:32.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.020+0000 7f331e851640 1 -- 172.21.15.43:0/1026228344 shutdown_connections 2024-04-25T10:30:32.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.020+0000 7f331e851640 1 -- 172.21.15.43:0/1026228344 wait complete. 2024-04-25T10:30:32.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.021+0000 7f331e851640 1 Processor -- start 2024-04-25T10:30:32.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.021+0000 7f331e851640 1 -- start start 2024-04-25T10:30:32.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.021+0000 7f331e851640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3318102fb0 0x7f3318199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.021+0000 7f331e851640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3318103b10 0x7f331819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.021+0000 7f331e851640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f331819f480 0x7f33181a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.021+0000 7f331e851640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f331810f840 con 0x7f3318103b10 2024-04-25T10:30:32.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.021+0000 7f331e851640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f331810f6c0 con 0x7f3318102fb0 2024-04-25T10:30:32.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.021+0000 7f331e851640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f331810f9c0 con 0x7f331819f480 2024-04-25T10:30:32.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.021+0000 7f33177fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3318103b10 0x7f331819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.021+0000 7f33177fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3318103b10 0x7f331819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38218/0 (socket says 172.21.15.43:38218) 2024-04-25T10:30:32.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.021+0000 7f3317fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3318102fb0 0x7f3318199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.021+0000 7f33177fe640 1 -- 172.21.15.43:0/530384282 learned_addr learned my addr 172.21.15.43:0/530384282 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:32.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.021+0000 7f331cdee640 1 --2- 172.21.15.43:0/530384282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f331819f480 0x7f33181a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.022+0000 7f33177fe640 1 -- 172.21.15.43:0/530384282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f331819f480 msgr2=0x7f33181a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.022+0000 7f33177fe640 1 --2- 172.21.15.43:0/530384282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f331819f480 0x7f33181a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.022+0000 7f33177fe640 1 -- 172.21.15.43:0/530384282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3318102fb0 msgr2=0x7f3318199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.022+0000 7f33177fe640 1 --2- 172.21.15.43:0/530384282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3318102fb0 0x7f3318199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.022+0000 7f33177fe640 1 -- 172.21.15.43:0/530384282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33181a1f10 con 0x7f3318103b10 2024-04-25T10:30:32.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.022+0000 7f3317fff640 1 --2- 172.21.15.43:0/530384282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3318102fb0 0x7f3318199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:32.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.022+0000 7f33177fe640 1 --2- 172.21.15.43:0/530384282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3318103b10 0x7f331819a500 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f330800e9f0 tx=0x7f330800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:32.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.022+0000 7f33157fa640 1 -- 172.21.15.43:0/530384282 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330800cd80 con 0x7f3318103b10 2024-04-25T10:30:32.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.022+0000 7f33157fa640 1 -- 172.21.15.43:0/530384282 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33080054d0 con 0x7f3318103b10 2024-04-25T10:30:32.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.022+0000 7f331e851640 1 -- 172.21.15.43:0/530384282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33181a2200 con 0x7f3318103b10 2024-04-25T10:30:32.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.022+0000 7f33157fa640 1 -- 172.21.15.43:0/530384282 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3308010690 con 0x7f3318103b10 2024-04-25T10:30:32.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.022+0000 7f331e851640 1 -- 172.21.15.43:0/530384282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33181a2700 con 0x7f3318103b10 2024-04-25T10:30:32.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.023+0000 7f331e851640 1 -- 172.21.15.43:0/530384282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3318069c70 con 0x7f3318103b10 2024-04-25T10:30:32.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.026+0000 7f33157fa640 1 -- 172.21.15.43:0/530384282 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f3308004e30 con 0x7f3318103b10 2024-04-25T10:30:32.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.026+0000 7f33157fa640 1 --2- 172.21.15.43:0/530384282 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f33040afaf0 0x7f33040b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.026+0000 7f33157fa640 1 -- 172.21.15.43:0/530384282 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7497+0+0 (secure 0 0 0) 0x7f33080def90 con 0x7f3318103b10 2024-04-25T10:30:32.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.026+0000 7f3317fff640 1 --2- 172.21.15.43:0/530384282 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f33040afaf0 0x7f33040b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.027+0000 7f3317fff640 1 --2- 172.21.15.43:0/530384282 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f33040afaf0 0x7f33040b2000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3318103a10 tx=0x7f3300005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:32.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.030+0000 7f33157fa640 1 -- 172.21.15.43:0/530384282 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3308053ff0 con 0x7f3318103b10 2024-04-25T10:30:32.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.219+0000 7f331e851640 1 -- 172.21.15.43:0/530384282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3318109ae0 con 0x7f3318103b10 2024-04-25T10:30:32.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:32 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/4101532524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T10:30:32.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:32 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/4101532524' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T10:30:32.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:32 smithi077 ceph-mon[32772]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T10:30:32.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:32.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:32 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/530384282' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T10:30:32.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:32 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/4101532524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T10:30:32.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:32 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/4101532524' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T10:30:32.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:32 smithi164 ceph-mon[32713]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T10:30:32.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:32.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:32 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/530384282' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T10:30:32.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:32 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/4101532524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T10:30:32.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:32 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/4101532524' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T10:30:32.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:32 smithi043 ceph-mon[29478]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T10:30:32.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:32.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:32 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/530384282' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T10:30:32.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.881+0000 7f33157fa640 1 -- 172.21.15.43:0/530384282 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) ==== 170+0+0 (secure 0 0 0) 0x7f33080a7ea0 con 0x7f3318103b10 2024-04-25T10:30:32.883 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-04-25T10:30:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.884+0000 7f331e851640 1 -- 172.21.15.43:0/530384282 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f33040afaf0 msgr2=0x7f33040b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.884+0000 7f331e851640 1 --2- 172.21.15.43:0/530384282 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f33040afaf0 0x7f33040b2000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3318103a10 tx=0x7f3300005f90 comp rx=0 tx=0).stop 2024-04-25T10:30:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.884+0000 7f331e851640 1 -- 172.21.15.43:0/530384282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3318103b10 msgr2=0x7f331819a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.884+0000 7f331e851640 1 --2- 172.21.15.43:0/530384282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3318103b10 0x7f331819a500 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f330800e9f0 tx=0x7f330800edf0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.884+0000 7f331e851640 1 -- 172.21.15.43:0/530384282 shutdown_connections 2024-04-25T10:30:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.884+0000 7f331e851640 1 --2- 172.21.15.43:0/530384282 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f33040afaf0 0x7f33040b2000 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.884+0000 7f331e851640 1 --2- 172.21.15.43:0/530384282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f331819f480 0x7f33181a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.884+0000 7f331e851640 1 --2- 172.21.15.43:0/530384282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3318103b10 0x7f331819a500 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.884+0000 7f331e851640 1 --2- 172.21.15.43:0/530384282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3318102fb0 0x7f3318199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.884+0000 7f331e851640 1 -- 172.21.15.43:0/530384282 >> 172.21.15.43:0/530384282 conn(0x7f33180fd330 msgr2=0x7f3318106cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:32.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.884+0000 7f331e851640 1 -- 172.21.15.43:0/530384282 shutdown_connections 2024-04-25T10:30:32.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:32.884+0000 7f331e851640 1 -- 172.21.15.43:0/530384282 wait complete. 2024-04-25T10:30:32.907 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:30:33.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.022+0000 7f87489f3640 1 -- 172.21.15.43:0/734038108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87441029e0 msgr2=0x7f87441036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.022+0000 7f87489f3640 1 --2- 172.21.15.43:0/734038108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87441029e0 0x7f87441036e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8734009900 tx=0x7f873402a410 comp rx=0 tx=0).stop 2024-04-25T10:30:33.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.022+0000 7f87489f3640 1 -- 172.21.15.43:0/734038108 shutdown_connections 2024-04-25T10:30:33.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.022+0000 7f87489f3640 1 --2- 172.21.15.43:0/734038108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8744103c20 0x7f8744104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.022+0000 7f87489f3640 1 --2- 172.21.15.43:0/734038108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8744108920 0x7f874410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.022+0000 7f87489f3640 1 --2- 172.21.15.43:0/734038108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87441029e0 0x7f87441036e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.022+0000 7f87489f3640 1 -- 172.21.15.43:0/734038108 >> 172.21.15.43:0/734038108 conn(0x7f87440fdbd0 msgr2=0x7f8744100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.022+0000 7f87489f3640 1 -- 172.21.15.43:0/734038108 shutdown_connections 2024-04-25T10:30:33.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.022+0000 7f87489f3640 1 -- 172.21.15.43:0/734038108 wait complete. 2024-04-25T10:30:33.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.023+0000 7f87489f3640 1 Processor -- start 2024-04-25T10:30:33.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.023+0000 7f87489f3640 1 -- start start 2024-04-25T10:30:33.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.023+0000 7f87489f3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87441029e0 0x7f8744199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.023+0000 7f87489f3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8744103c20 0x7f8744199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.023+0000 7f87489f3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8744108920 0x7f87441a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.023+0000 7f87489f3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f874410e4e0 con 0x7f87441029e0 2024-04-25T10:30:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.023+0000 7f87489f3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f874410e360 con 0x7f8744103c20 2024-04-25T10:30:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.023+0000 7f87489f3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f874410e660 con 0x7f8744108920 2024-04-25T10:30:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.023+0000 7f874259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87441029e0 0x7f8744199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.023+0000 7f874259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87441029e0 0x7f8744199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38246/0 (socket says 172.21.15.43:38246) 2024-04-25T10:30:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.023+0000 7f874259c640 1 -- 172.21.15.43:0/640647423 learned_addr learned my addr 172.21.15.43:0/640647423 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.023+0000 7f8742d9d640 1 --2- 172.21.15.43:0/640647423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8744108920 0x7f87441a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.023+0000 7f8741d9b640 1 --2- 172.21.15.43:0/640647423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8744103c20 0x7f8744199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.023+0000 7f874259c640 1 -- 172.21.15.43:0/640647423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8744108920 msgr2=0x7f87441a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.023+0000 7f874259c640 1 --2- 172.21.15.43:0/640647423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8744108920 0x7f87441a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.023+0000 7f874259c640 1 -- 172.21.15.43:0/640647423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8744103c20 msgr2=0x7f8744199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.023+0000 7f874259c640 1 --2- 172.21.15.43:0/640647423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8744103c20 0x7f8744199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.023+0000 7f874259c640 1 -- 172.21.15.43:0/640647423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87441a0f50 con 0x7f87441029e0 2024-04-25T10:30:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.023+0000 7f8742d9d640 1 --2- 172.21.15.43:0/640647423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8744108920 0x7f87441a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.023+0000 7f8741d9b640 1 --2- 172.21.15.43:0/640647423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8744103c20 0x7f8744199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.024+0000 7f874259c640 1 --2- 172.21.15.43:0/640647423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87441029e0 0x7f8744199410 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8734006010 tx=0x7f8734004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.024+0000 7f87337fe640 1 -- 172.21.15.43:0/640647423 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873403f030 con 0x7f87441029e0 2024-04-25T10:30:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.024+0000 7f87337fe640 1 -- 172.21.15.43:0/640647423 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8734003aa0 con 0x7f87441029e0 2024-04-25T10:30:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.024+0000 7f87337fe640 1 -- 172.21.15.43:0/640647423 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734035070 con 0x7f87441029e0 2024-04-25T10:30:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.024+0000 7f87489f3640 1 -- 172.21.15.43:0/640647423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87441a11e0 con 0x7f87441029e0 2024-04-25T10:30:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.024+0000 7f87489f3640 1 -- 172.21.15.43:0/640647423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87441a16d0 con 0x7f87441029e0 2024-04-25T10:30:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.025+0000 7f87489f3640 1 -- 172.21.15.43:0/640647423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8744107730 con 0x7f87441029e0 2024-04-25T10:30:33.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.026+0000 7f87337fe640 1 -- 172.21.15.43:0/640647423 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f8734042030 con 0x7f87441029e0 2024-04-25T10:30:33.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.027+0000 7f87337fe640 1 --2- 172.21.15.43:0/640647423 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f872c0afaf0 0x7f872c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.027+0000 7f8741d9b640 1 --2- 172.21.15.43:0/640647423 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f872c0afaf0 0x7f872c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.027+0000 7f87337fe640 1 -- 172.21.15.43:0/640647423 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7879+0+0 (secure 0 0 0) 0x7f87340ff620 con 0x7f87441029e0 2024-04-25T10:30:33.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.027+0000 7f8741d9b640 1 --2- 172.21.15.43:0/640647423 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f872c0afaf0 0x7f872c0b2000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f874419a4b0 tx=0x7f8738005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.031+0000 7f87337fe640 1 -- 172.21.15.43:0/640647423 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f87340c83b0 con 0x7f87441029e0 2024-04-25T10:30:33.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:33.260+0000 7f87489f3640 1 -- 172.21.15.43:0/640647423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f874406c2a0 con 0x7f87441029e0 2024-04-25T10:30:34.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:33 smithi077 ceph-mon[32772]: pgmap v274: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:34.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:34.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:30:34.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:33 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/530384282' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T10:30:34.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:33 smithi077 ceph-mon[32772]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T10:30:34.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:33 smithi077 ceph-mon[32772]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T10:30:34.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:33 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/640647423' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T10:30:34.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:33 smithi077 ceph-mon[32772]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T10:30:34.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:33 smithi164 ceph-mon[32713]: pgmap v274: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:34.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:34.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:30:34.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:33 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/530384282' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T10:30:34.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:33 smithi164 ceph-mon[32713]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T10:30:34.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:33 smithi164 ceph-mon[32713]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T10:30:34.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:33 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/640647423' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T10:30:34.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:33 smithi164 ceph-mon[32713]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T10:30:34.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:33 smithi043 ceph-mon[29478]: pgmap v274: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:34.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:30:34.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:30:34.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:30:34.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:33 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/530384282' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T10:30:34.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:33 smithi043 ceph-mon[29478]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T10:30:34.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:33 smithi043 ceph-mon[29478]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T10:30:34.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:33 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/640647423' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T10:30:34.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:33 smithi043 ceph-mon[29478]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T10:30:34.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.152+0000 7f87337fe640 1 -- 172.21.15.43:0/640647423 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f87340c7cb0 con 0x7f87441029e0 2024-04-25T10:30:34.154 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T10:30:34.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.155+0000 7f87489f3640 1 -- 172.21.15.43:0/640647423 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f872c0afaf0 msgr2=0x7f872c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.155+0000 7f87489f3640 1 --2- 172.21.15.43:0/640647423 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f872c0afaf0 0x7f872c0b2000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f874419a4b0 tx=0x7f8738005f90 comp rx=0 tx=0).stop 2024-04-25T10:30:34.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.155+0000 7f87489f3640 1 -- 172.21.15.43:0/640647423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87441029e0 msgr2=0x7f8744199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.155+0000 7f87489f3640 1 --2- 172.21.15.43:0/640647423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87441029e0 0x7f8744199410 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8734006010 tx=0x7f8734004270 comp rx=0 tx=0).stop 2024-04-25T10:30:34.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.155+0000 7f87489f3640 1 -- 172.21.15.43:0/640647423 shutdown_connections 2024-04-25T10:30:34.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.155+0000 7f87489f3640 1 --2- 172.21.15.43:0/640647423 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f872c0afaf0 0x7f872c0b2000 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.155+0000 7f87489f3640 1 --2- 172.21.15.43:0/640647423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8744103c20 0x7f8744199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.155+0000 7f87489f3640 1 --2- 172.21.15.43:0/640647423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8744108920 0x7f87441a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.155+0000 7f87489f3640 1 --2- 172.21.15.43:0/640647423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87441029e0 0x7f8744199410 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.155+0000 7f87489f3640 1 -- 172.21.15.43:0/640647423 >> 172.21.15.43:0/640647423 conn(0x7f87440fdbd0 msgr2=0x7f8744104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:34.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.155+0000 7f87489f3640 1 -- 172.21.15.43:0/640647423 shutdown_connections 2024-04-25T10:30:34.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.155+0000 7f87489f3640 1 -- 172.21.15.43:0/640647423 wait complete. 2024-04-25T10:30:34.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:34.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.291+0000 7fef88613640 1 -- 172.21.15.43:0/1411721718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef80104690 msgr2=0x7fef8010ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.291+0000 7fef88613640 1 --2- 172.21.15.43:0/1411721718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef80104690 0x7fef8010ade0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fef78009ca0 tx=0x7fef7802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.291+0000 7fef88613640 1 -- 172.21.15.43:0/1411721718 shutdown_connections 2024-04-25T10:30:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.291+0000 7fef88613640 1 --2- 172.21.15.43:0/1411721718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef80104690 0x7fef8010ade0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.291+0000 7fef88613640 1 --2- 172.21.15.43:0/1411721718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef80103b30 0x7fef80104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.291+0000 7fef88613640 1 --2- 172.21.15.43:0/1411721718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef80102fd0 0x7fef801033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.291+0000 7fef88613640 1 -- 172.21.15.43:0/1411721718 >> 172.21.15.43:0/1411721718 conn(0x7fef800fd370 msgr2=0x7fef800ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.291+0000 7fef88613640 1 -- 172.21.15.43:0/1411721718 shutdown_connections 2024-04-25T10:30:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.291+0000 7fef88613640 1 -- 172.21.15.43:0/1411721718 wait complete. 2024-04-25T10:30:34.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.292+0000 7fef88613640 1 Processor -- start 2024-04-25T10:30:34.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.292+0000 7fef88613640 1 -- start start 2024-04-25T10:30:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.292+0000 7fef88613640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef80102fd0 0x7fef80102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.292+0000 7fef88613640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef80103b30 0x7fef8019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.292+0000 7fef88613640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef8019f280 0x7fef801a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.292+0000 7fef88613640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fef8010e740 con 0x7fef80102fd0 2024-04-25T10:30:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.292+0000 7fef88613640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fef8010e5c0 con 0x7fef8019f280 2024-04-25T10:30:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.292+0000 7fef88613640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fef8010e8c0 con 0x7fef80103b30 2024-04-25T10:30:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.292+0000 7fef863af640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef80102fd0 0x7fef80102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.292+0000 7fef863af640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef80102fd0 0x7fef80102270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38268/0 (socket says 172.21.15.43:38268) 2024-04-25T10:30:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.292+0000 7fef863af640 1 -- 172.21.15.43:0/1507775860 learned_addr learned my addr 172.21.15.43:0/1507775860 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.293+0000 7fef86bb0640 1 --2- 172.21.15.43:0/1507775860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef8019f280 0x7fef801a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.293+0000 7fef85bae640 1 --2- 172.21.15.43:0/1507775860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef80103b30 0x7fef8019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.293+0000 7fef863af640 1 -- 172.21.15.43:0/1507775860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef80103b30 msgr2=0x7fef8019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.293+0000 7fef863af640 1 --2- 172.21.15.43:0/1507775860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef80103b30 0x7fef8019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.293+0000 7fef863af640 1 -- 172.21.15.43:0/1507775860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef8019f280 msgr2=0x7fef801a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.293+0000 7fef863af640 1 --2- 172.21.15.43:0/1507775860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef8019f280 0x7fef801a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.293+0000 7fef863af640 1 -- 172.21.15.43:0/1507775860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef801a1e30 con 0x7fef80102fd0 2024-04-25T10:30:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.293+0000 7fef85bae640 1 --2- 172.21.15.43:0/1507775860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef80103b30 0x7fef8019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.293+0000 7fef863af640 1 --2- 172.21.15.43:0/1507775860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef80102fd0 0x7fef80102270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00c970 tx=0x7fef7c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.293+0000 7fef777fe640 1 -- 172.21.15.43:0/1507775860 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c007bb0 con 0x7fef80102fd0 2024-04-25T10:30:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.293+0000 7fef777fe640 1 -- 172.21.15.43:0/1507775860 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef7c007d50 con 0x7fef80102fd0 2024-04-25T10:30:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.293+0000 7fef88613640 1 -- 172.21.15.43:0/1507775860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef801a2120 con 0x7fef80102fd0 2024-04-25T10:30:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.293+0000 7fef777fe640 1 -- 172.21.15.43:0/1507775860 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c00f450 con 0x7fef80102fd0 2024-04-25T10:30:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.293+0000 7fef88613640 1 -- 172.21.15.43:0/1507775860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef801a2670 con 0x7fef80102fd0 2024-04-25T10:30:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.294+0000 7fef88613640 1 -- 172.21.15.43:0/1507775860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef8006b3a0 con 0x7fef80102fd0 2024-04-25T10:30:34.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.296+0000 7fef777fe640 1 -- 172.21.15.43:0/1507775860 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fef7c004f20 con 0x7fef80102fd0 2024-04-25T10:30:34.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.296+0000 7fef777fe640 1 --2- 172.21.15.43:0/1507775860 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fef6c0afaf0 0x7fef6c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.297+0000 7fef85bae640 1 --2- 172.21.15.43:0/1507775860 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fef6c0afaf0 0x7fef6c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.297+0000 7fef777fe640 1 -- 172.21.15.43:0/1507775860 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7fef7c0dec60 con 0x7fef80102fd0 2024-04-25T10:30:34.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.297+0000 7fef85bae640 1 --2- 172.21.15.43:0/1507775860 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fef6c0afaf0 0x7fef6c0b2000 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fef80101c10 tx=0x7fef70005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:34.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.300+0000 7fef777fe640 1 -- 172.21.15.43:0/1507775860 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef7c014030 con 0x7fef80102fd0 2024-04-25T10:30:34.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.487+0000 7fef88613640 1 -- 172.21.15.43:0/1507775860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef8002de70 con 0x7fef80102fd0 2024-04-25T10:30:34.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.488+0000 7fef777fe640 1 -- 172.21.15.43:0/1507775860 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22198 (secure 0 0 0) 0x7fef7c0a78f0 con 0x7fef80102fd0 2024-04-25T10:30:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:34.491 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":76,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:30:33.261612+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:31.503414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:32.224334+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:34.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.490+0000 7fef88613640 1 -- 172.21.15.43:0/1507775860 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fef6c0afaf0 msgr2=0x7fef6c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.490+0000 7fef88613640 1 --2- 172.21.15.43:0/1507775860 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fef6c0afaf0 0x7fef6c0b2000 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fef80101c10 tx=0x7fef70005db0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.490+0000 7fef88613640 1 -- 172.21.15.43:0/1507775860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef80102fd0 msgr2=0x7fef80102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.490+0000 7fef88613640 1 --2- 172.21.15.43:0/1507775860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef80102fd0 0x7fef80102270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00c970 tx=0x7fef7c00ce30 comp rx=0 tx=0).stop 2024-04-25T10:30:34.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.491+0000 7fef88613640 1 -- 172.21.15.43:0/1507775860 shutdown_connections 2024-04-25T10:30:34.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.491+0000 7fef88613640 1 --2- 172.21.15.43:0/1507775860 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fef6c0afaf0 0x7fef6c0b2000 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.491+0000 7fef88613640 1 --2- 172.21.15.43:0/1507775860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef8019f280 0x7fef801a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.491+0000 7fef88613640 1 --2- 172.21.15.43:0/1507775860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef80103b30 0x7fef8019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.491+0000 7fef88613640 1 --2- 172.21.15.43:0/1507775860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef80102fd0 0x7fef80102270 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.491+0000 7fef88613640 1 -- 172.21.15.43:0/1507775860 >> 172.21.15.43:0/1507775860 conn(0x7fef800fd370 msgr2=0x7fef80106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.491+0000 7fef88613640 1 -- 172.21.15.43:0/1507775860 shutdown_connections 2024-04-25T10:30:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.491+0000 7fef88613640 1 -- 172.21.15.43:0/1507775860 wait complete. 2024-04-25T10:30:34.504 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:34.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.613+0000 7f8acee42640 1 -- 172.21.15.43:0/265527479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ac8109a50 msgr2=0x7f8ac810be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.613+0000 7f8acee42640 1 --2- 172.21.15.43:0/265527479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ac8109a50 0x7f8ac810be70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009ca0 tx=0x7f8ac002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.613+0000 7f8acee42640 1 -- 172.21.15.43:0/265527479 shutdown_connections 2024-04-25T10:30:34.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.614+0000 7f8acee42640 1 --2- 172.21.15.43:0/265527479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ac8104d50 0x7f8ac8105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.614+0000 7f8acee42640 1 --2- 172.21.15.43:0/265527479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ac8109a50 0x7f8ac810be70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.614+0000 7f8acee42640 1 --2- 172.21.15.43:0/265527479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ac8103b10 0x7f8ac8104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.614+0000 7f8acee42640 1 -- 172.21.15.43:0/265527479 >> 172.21.15.43:0/265527479 conn(0x7f8ac80fed40 msgr2=0x7f8ac81011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:34.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.614+0000 7f8acee42640 1 -- 172.21.15.43:0/265527479 shutdown_connections 2024-04-25T10:30:34.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.614+0000 7f8acee42640 1 -- 172.21.15.43:0/265527479 wait complete. 2024-04-25T10:30:34.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.614+0000 7f8acee42640 1 Processor -- start 2024-04-25T10:30:34.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.614+0000 7f8acee42640 1 -- start start 2024-04-25T10:30:34.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.614+0000 7f8acee42640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ac8103b10 0x7f8ac8077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.614+0000 7f8acee42640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ac8104d50 0x7f8ac8076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.614+0000 7f8acee42640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ac8109a50 0x7f8ac8076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.614+0000 7f8acee42640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac810f640 con 0x7f8ac8109a50 2024-04-25T10:30:34.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.614+0000 7f8acee42640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac810f4c0 con 0x7f8ac8103b10 2024-04-25T10:30:34.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.614+0000 7f8acee42640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac810f7c0 con 0x7f8ac8104d50 2024-04-25T10:30:34.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.615+0000 7f8acd3df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ac8109a50 0x7f8ac8076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.615+0000 7f8acd3df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ac8109a50 0x7f8ac8076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38296/0 (socket says 172.21.15.43:38296) 2024-04-25T10:30:34.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.615+0000 7f8acd3df640 1 -- 172.21.15.43:0/1595484605 learned_addr learned my addr 172.21.15.43:0/1595484605 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:34.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.615+0000 7f8ac7fff640 1 --2- 172.21.15.43:0/1595484605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ac8104d50 0x7f8ac8076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.615+0000 7f8accbde640 1 --2- 172.21.15.43:0/1595484605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ac8103b10 0x7f8ac8077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.615+0000 7f8acd3df640 1 -- 172.21.15.43:0/1595484605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ac8104d50 msgr2=0x7f8ac8076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.615+0000 7f8acd3df640 1 --2- 172.21.15.43:0/1595484605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ac8104d50 0x7f8ac8076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.615+0000 7f8acd3df640 1 -- 172.21.15.43:0/1595484605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ac8103b10 msgr2=0x7f8ac8077960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.615+0000 7f8acd3df640 1 --2- 172.21.15.43:0/1595484605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ac8103b10 0x7f8ac8077960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.615+0000 7f8acd3df640 1 -- 172.21.15.43:0/1595484605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac8076c70 con 0x7f8ac8109a50 2024-04-25T10:30:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.615+0000 7f8acd3df640 1 --2- 172.21.15.43:0/1595484605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ac8109a50 0x7f8ac8076540 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0004af0 tx=0x7f8ac0031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.615+0000 7f8ac5ffb640 1 -- 172.21.15.43:0/1595484605 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0003b70 con 0x7f8ac8109a50 2024-04-25T10:30:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.615+0000 7f8acee42640 1 -- 172.21.15.43:0/1595484605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac81a42d0 con 0x7f8ac8109a50 2024-04-25T10:30:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.615+0000 7f8ac5ffb640 1 -- 172.21.15.43:0/1595484605 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ac0003d10 con 0x7f8ac8109a50 2024-04-25T10:30:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.615+0000 7f8ac5ffb640 1 -- 172.21.15.43:0/1595484605 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac002fb20 con 0x7f8ac8109a50 2024-04-25T10:30:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.615+0000 7f8acee42640 1 -- 172.21.15.43:0/1595484605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac81a4730 con 0x7f8ac8109a50 2024-04-25T10:30:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.616+0000 7f8acee42640 1 -- 172.21.15.43:0/1595484605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ac810fe00 con 0x7f8ac8109a50 2024-04-25T10:30:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.618+0000 7f8ac5ffb640 1 -- 172.21.15.43:0/1595484605 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f8ac0004580 con 0x7f8ac8109a50 2024-04-25T10:30:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.618+0000 7f8ac5ffb640 1 --2- 172.21.15.43:0/1595484605 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f8ab80afaf0 0x7f8ab80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.619+0000 7f8accbde640 1 --2- 172.21.15.43:0/1595484605 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f8ab80afaf0 0x7f8ab80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.619+0000 7f8ac5ffb640 1 -- 172.21.15.43:0/1595484605 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f8ac01003d0 con 0x7f8ac8109a50 2024-04-25T10:30:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.619+0000 7f8accbde640 1 --2- 172.21.15.43:0/1595484605 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f8ab80afaf0 0x7f8ab80b2000 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8104710 tx=0x7f8abc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:34.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.622+0000 7f8ac5ffb640 1 -- 172.21.15.43:0/1595484605 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ac00c9060 con 0x7f8ac8109a50 2024-04-25T10:30:34.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.816+0000 7f8acee42640 1 -- 172.21.15.43:0/1595484605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ac80637a0 con 0x7f8ac8109a50 2024-04-25T10:30:34.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.817+0000 7f8ac5ffb640 1 -- 172.21.15.43:0/1595484605 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22185 (secure 0 0 0) 0x7f8ac00c8960 con 0x7f8ac8109a50 2024-04-25T10:30:34.820 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:34.820 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":77,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:30:34.623807+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:31.503414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:32.224334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:34.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.819+0000 7f8acee42640 1 -- 172.21.15.43:0/1595484605 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f8ab80afaf0 msgr2=0x7f8ab80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.819+0000 7f8acee42640 1 --2- 172.21.15.43:0/1595484605 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f8ab80afaf0 0x7f8ab80b2000 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8104710 tx=0x7f8abc005f90 comp rx=0 tx=0).stop 2024-04-25T10:30:34.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.819+0000 7f8acee42640 1 -- 172.21.15.43:0/1595484605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ac8109a50 msgr2=0x7f8ac8076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.819+0000 7f8acee42640 1 --2- 172.21.15.43:0/1595484605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ac8109a50 0x7f8ac8076540 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0004af0 tx=0x7f8ac0031700 comp rx=0 tx=0).stop 2024-04-25T10:30:34.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.819+0000 7f8acee42640 1 -- 172.21.15.43:0/1595484605 shutdown_connections 2024-04-25T10:30:34.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.819+0000 7f8acee42640 1 --2- 172.21.15.43:0/1595484605 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f8ab80afaf0 0x7f8ab80b2000 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.819+0000 7f8acee42640 1 --2- 172.21.15.43:0/1595484605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ac8109a50 0x7f8ac8076540 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.819+0000 7f8acee42640 1 --2- 172.21.15.43:0/1595484605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ac8104d50 0x7f8ac8076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.819+0000 7f8acee42640 1 --2- 172.21.15.43:0/1595484605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ac8103b10 0x7f8ac8077960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.819+0000 7f8acee42640 1 -- 172.21.15.43:0/1595484605 >> 172.21.15.43:0/1595484605 conn(0x7f8ac80fed40 msgr2=0x7f8ac8105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:34.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.819+0000 7f8acee42640 1 -- 172.21.15.43:0/1595484605 shutdown_connections 2024-04-25T10:30:34.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.819+0000 7f8acee42640 1 -- 172.21.15.43:0/1595484605 wait complete. 2024-04-25T10:30:34.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:34 smithi043 ceph-mon[29478]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T10:30:34.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:34 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/640647423' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T10:30:34.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:34 smithi043 ceph-mon[29478]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T10:30:34.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:34 smithi043 ceph-mon[29478]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T10:30:34.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:34 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1507775860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:30:34.833 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:30:34.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.946+0000 7fc048e15640 1 -- 172.21.15.43:0/87535479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc044103c20 msgr2=0x7fc044104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.946+0000 7fc048e15640 1 --2- 172.21.15.43:0/87535479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc044103c20 0x7fc044104030 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc038009900 tx=0x7fc03802a3f0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.946+0000 7fc048e15640 1 -- 172.21.15.43:0/87535479 shutdown_connections 2024-04-25T10:30:34.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.946+0000 7fc048e15640 1 --2- 172.21.15.43:0/87535479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc044108920 0x7fc04410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.946+0000 7fc048e15640 1 --2- 172.21.15.43:0/87535479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc044103c20 0x7fc044104030 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.946+0000 7fc048e15640 1 --2- 172.21.15.43:0/87535479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0441029e0 0x7fc0441036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.946+0000 7fc048e15640 1 -- 172.21.15.43:0/87535479 >> 172.21.15.43:0/87535479 conn(0x7fc0440fdbd0 msgr2=0x7fc044100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:34.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.946+0000 7fc048e15640 1 -- 172.21.15.43:0/87535479 shutdown_connections 2024-04-25T10:30:34.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.946+0000 7fc048e15640 1 -- 172.21.15.43:0/87535479 wait complete. 2024-04-25T10:30:34.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.947+0000 7fc048e15640 1 Processor -- start 2024-04-25T10:30:34.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.947+0000 7fc048e15640 1 -- start start 2024-04-25T10:30:34.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.947+0000 7fc048e15640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0441029e0 0x7fc04419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.947+0000 7fc048e15640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc044103c20 0x7fc04419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.947+0000 7fc048e15640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc044108920 0x7fc0441a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.947+0000 7fc048e15640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc04410f610 con 0x7fc044108920 2024-04-25T10:30:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.947+0000 7fc048e15640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc04410f490 con 0x7fc044103c20 2024-04-25T10:30:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.947+0000 7fc048e15640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc04410f790 con 0x7fc0441029e0 2024-04-25T10:30:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.947+0000 7fc042d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc044108920 0x7fc0441a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.947+0000 7fc042d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc044108920 0x7fc0441a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38318/0 (socket says 172.21.15.43:38318) 2024-04-25T10:30:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.947+0000 7fc042d9d640 1 -- 172.21.15.43:0/27581567 learned_addr learned my addr 172.21.15.43:0/27581567 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.947+0000 7fc041d9b640 1 --2- 172.21.15.43:0/27581567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc044103c20 0x7fc04419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.947+0000 7fc04259c640 1 --2- 172.21.15.43:0/27581567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0441029e0 0x7fc04419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.947+0000 7fc042d9d640 1 -- 172.21.15.43:0/27581567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0441029e0 msgr2=0x7fc04419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.947+0000 7fc042d9d640 1 --2- 172.21.15.43:0/27581567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0441029e0 0x7fc04419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.947+0000 7fc042d9d640 1 -- 172.21.15.43:0/27581567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc044103c20 msgr2=0x7fc04419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.947+0000 7fc042d9d640 1 --2- 172.21.15.43:0/27581567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc044103c20 0x7fc04419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.947+0000 7fc042d9d640 1 -- 172.21.15.43:0/27581567 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0441a2080 con 0x7fc044108920 2024-04-25T10:30:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.948+0000 7fc042d9d640 1 --2- 172.21.15.43:0/27581567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc044108920 0x7fc0441a1a10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00eaa0 tx=0x7fc03c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.948+0000 7fc0337fe640 1 -- 172.21.15.43:0/27581567 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c00cdb0 con 0x7fc044108920 2024-04-25T10:30:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.948+0000 7fc0337fe640 1 -- 172.21.15.43:0/27581567 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc03c005020 con 0x7fc044108920 2024-04-25T10:30:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.948+0000 7fc048e15640 1 -- 172.21.15.43:0/27581567 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0441a2370 con 0x7fc044108920 2024-04-25T10:30:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.948+0000 7fc0337fe640 1 -- 172.21.15.43:0/27581567 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c0116d0 con 0x7fc044108920 2024-04-25T10:30:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.948+0000 7fc048e15640 1 -- 172.21.15.43:0/27581567 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0441a28c0 con 0x7fc044108920 2024-04-25T10:30:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.949+0000 7fc048e15640 1 -- 172.21.15.43:0/27581567 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc044107730 con 0x7fc044108920 2024-04-25T10:30:34.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.950+0000 7fc0337fe640 1 -- 172.21.15.43:0/27581567 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fc03c0051d0 con 0x7fc044108920 2024-04-25T10:30:34.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.951+0000 7fc0337fe640 1 --2- 172.21.15.43:0/27581567 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc02c0afaf0 0x7fc02c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.951+0000 7fc04259c640 1 --2- 172.21.15.43:0/27581567 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc02c0afaf0 0x7fc02c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.951+0000 7fc0337fe640 1 -- 172.21.15.43:0/27581567 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fc03c0df080 con 0x7fc044108920 2024-04-25T10:30:34.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.951+0000 7fc04259c640 1 --2- 172.21.15.43:0/27581567 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc02c0afaf0 0x7fc02c0b2000 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc034005fd0 tx=0x7fc034005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:34.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:34.954+0000 7fc0337fe640 1 -- 172.21.15.43:0/27581567 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc03c016030 con 0x7fc044108920 2024-04-25T10:30:35.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:34 smithi077 ceph-mon[32772]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T10:30:35.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:34 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/640647423' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T10:30:35.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:34 smithi077 ceph-mon[32772]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T10:30:35.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:34 smithi077 ceph-mon[32772]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T10:30:35.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:34 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1507775860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:30:35.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:34 smithi164 ceph-mon[32713]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T10:30:35.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:34 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/640647423' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T10:30:35.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:34 smithi164 ceph-mon[32713]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T10:30:35.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:34 smithi164 ceph-mon[32713]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T10:30:35.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:34 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1507775860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:30:35.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:35.183+0000 7fc048e15640 1 -- 172.21.15.43:0/27581567 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc04406c2a0 con 0x7fc044108920 2024-04-25T10:30:36.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:35 smithi077 ceph-mon[32772]: pgmap v277: 97 pgs: 20 creating+peering, 33 active+clean, 44 unknown; 577 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:36.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:35 smithi077 ceph-mon[32772]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T10:30:36.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:35 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1595484605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:30:36.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:35 smithi077 ceph-mon[32772]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T10:30:36.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:35 smithi077 ceph-mon[32772]: Cluster is now healthy 2024-04-25T10:30:36.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:35 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] up:active 2024-04-25T10:30:36.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:35 smithi077 ceph-mon[32772]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:30:36.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:35 smithi077 ceph-mon[32772]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:30:36.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:35 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/27581567' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T10:30:36.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:35 smithi164 ceph-mon[32713]: pgmap v277: 97 pgs: 20 creating+peering, 33 active+clean, 44 unknown; 577 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:36.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:35 smithi164 ceph-mon[32713]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T10:30:36.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:35 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1595484605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:30:36.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:35 smithi164 ceph-mon[32713]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T10:30:36.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:35 smithi164 ceph-mon[32713]: Cluster is now healthy 2024-04-25T10:30:36.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:35 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] up:active 2024-04-25T10:30:36.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:35 smithi164 ceph-mon[32713]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:30:36.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:35 smithi164 ceph-mon[32713]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:30:36.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:35 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/27581567' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T10:30:36.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:35 smithi043 ceph-mon[29478]: pgmap v277: 97 pgs: 20 creating+peering, 33 active+clean, 44 unknown; 577 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:30:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:35 smithi043 ceph-mon[29478]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T10:30:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:35 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1595484605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:30:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:35 smithi043 ceph-mon[29478]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T10:30:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:35 smithi043 ceph-mon[29478]: Cluster is now healthy 2024-04-25T10:30:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:35 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] up:active 2024-04-25T10:30:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:35 smithi043 ceph-mon[29478]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:30:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:35 smithi043 ceph-mon[29478]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:30:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:35 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/27581567' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T10:30:36.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.165+0000 7fc0337fe640 1 -- 172.21.15.43:0/27581567 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fc03c0a7d10 con 0x7fc044108920 2024-04-25T10:30:36.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.168+0000 7fc048e15640 1 -- 172.21.15.43:0/27581567 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc02c0afaf0 msgr2=0x7fc02c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.168+0000 7fc048e15640 1 --2- 172.21.15.43:0/27581567 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc02c0afaf0 0x7fc02c0b2000 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc034005fd0 tx=0x7fc034005c70 comp rx=0 tx=0).stop 2024-04-25T10:30:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.168+0000 7fc048e15640 1 -- 172.21.15.43:0/27581567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc044108920 msgr2=0x7fc0441a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.168+0000 7fc048e15640 1 --2- 172.21.15.43:0/27581567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc044108920 0x7fc0441a1a10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00eaa0 tx=0x7fc03c00ef60 comp rx=0 tx=0).stop 2024-04-25T10:30:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.168+0000 7fc048e15640 1 -- 172.21.15.43:0/27581567 shutdown_connections 2024-04-25T10:30:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.168+0000 7fc048e15640 1 --2- 172.21.15.43:0/27581567 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc02c0afaf0 0x7fc02c0b2000 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.168+0000 7fc048e15640 1 --2- 172.21.15.43:0/27581567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc044103c20 0x7fc04419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.168+0000 7fc048e15640 1 --2- 172.21.15.43:0/27581567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc044108920 0x7fc0441a1a10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.168+0000 7fc048e15640 1 --2- 172.21.15.43:0/27581567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0441029e0 0x7fc04419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.168+0000 7fc048e15640 1 -- 172.21.15.43:0/27581567 >> 172.21.15.43:0/27581567 conn(0x7fc0440fdbd0 msgr2=0x7fc044104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.168+0000 7fc048e15640 1 -- 172.21.15.43:0/27581567 shutdown_connections 2024-04-25T10:30:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.169+0000 7fc048e15640 1 -- 172.21.15.43:0/27581567 wait complete. 2024-04-25T10:30:36.184 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-25T10:30:36.185 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-25T10:30:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.306+0000 7f3dc430d640 1 -- 172.21.15.43:0/2968994425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dbc104d40 msgr2=0x7f3dbc105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.306+0000 7f3dc430d640 1 --2- 172.21.15.43:0/2968994425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dbc104d40 0x7f3dbc105150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009900 tx=0x7f3dac02a410 comp rx=0 tx=0).stop 2024-04-25T10:30:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.306+0000 7f3dc430d640 1 -- 172.21.15.43:0/2968994425 shutdown_connections 2024-04-25T10:30:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.306+0000 7f3dc430d640 1 --2- 172.21.15.43:0/2968994425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3dbc109a40 0x7f3dbc10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.306+0000 7f3dc430d640 1 --2- 172.21.15.43:0/2968994425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dbc104d40 0x7f3dbc105150 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.306+0000 7f3dc430d640 1 --2- 172.21.15.43:0/2968994425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dbc103b00 0x7f3dbc104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.306+0000 7f3dc430d640 1 -- 172.21.15.43:0/2968994425 >> 172.21.15.43:0/2968994425 conn(0x7f3dbc0fed10 msgr2=0x7f3dbc1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.306+0000 7f3dc430d640 1 -- 172.21.15.43:0/2968994425 shutdown_connections 2024-04-25T10:30:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.306+0000 7f3dc430d640 1 -- 172.21.15.43:0/2968994425 wait complete. 2024-04-25T10:30:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.306+0000 7f3dc430d640 1 Processor -- start 2024-04-25T10:30:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.307+0000 7f3dc430d640 1 -- start start 2024-04-25T10:30:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.307+0000 7f3dc430d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3dbc103b00 0x7f3dbc19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.307+0000 7f3dc430d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dbc104d40 0x7f3dbc19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.307+0000 7f3dc430d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dbc109a40 0x7f3dbc1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.307+0000 7f3dc430d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3dbc10f620 con 0x7f3dbc104d40 2024-04-25T10:30:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.307+0000 7f3dc430d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3dbc10f4a0 con 0x7f3dbc103b00 2024-04-25T10:30:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.307+0000 7f3dc430d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3dbc10f7a0 con 0x7f3dbc109a40 2024-04-25T10:30:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.307+0000 7f3dc18a8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dbc104d40 0x7f3dbc19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.307+0000 7f3dc18a8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dbc104d40 0x7f3dbc19a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38330/0 (socket says 172.21.15.43:38330) 2024-04-25T10:30:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.307+0000 7f3dc18a8640 1 -- 172.21.15.43:0/160977804 learned_addr learned my addr 172.21.15.43:0/160977804 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.307+0000 7f3dc20a9640 1 --2- 172.21.15.43:0/160977804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3dbc103b00 0x7f3dbc19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.307+0000 7f3dc28aa640 1 --2- 172.21.15.43:0/160977804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dbc109a40 0x7f3dbc1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.307+0000 7f3dc18a8640 1 -- 172.21.15.43:0/160977804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dbc109a40 msgr2=0x7f3dbc1a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.307+0000 7f3dc18a8640 1 --2- 172.21.15.43:0/160977804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dbc109a40 0x7f3dbc1a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.307+0000 7f3dc18a8640 1 -- 172.21.15.43:0/160977804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3dbc103b00 msgr2=0x7f3dbc19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.307+0000 7f3dc18a8640 1 --2- 172.21.15.43:0/160977804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3dbc103b00 0x7f3dbc19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.307+0000 7f3dc18a8640 1 -- 172.21.15.43:0/160977804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dbc1a1dc0 con 0x7f3dbc104d40 2024-04-25T10:30:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.307+0000 7f3dc18a8640 1 --2- 172.21.15.43:0/160977804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dbc104d40 0x7f3dbc19a830 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3dac006010 tx=0x7f3dac031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.308+0000 7f3db37fe640 1 -- 172.21.15.43:0/160977804 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac03f030 con 0x7f3dbc104d40 2024-04-25T10:30:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.308+0000 7f3db37fe640 1 -- 172.21.15.43:0/160977804 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dac003aa0 con 0x7f3dbc104d40 2024-04-25T10:30:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.308+0000 7f3dc430d640 1 -- 172.21.15.43:0/160977804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dbc1a2050 con 0x7f3dbc104d40 2024-04-25T10:30:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.308+0000 7f3db37fe640 1 -- 172.21.15.43:0/160977804 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac035070 con 0x7f3dbc104d40 2024-04-25T10:30:36.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.308+0000 7f3dc430d640 1 -- 172.21.15.43:0/160977804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dbc1a2590 con 0x7f3dbc104d40 2024-04-25T10:30:36.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.309+0000 7f3dc430d640 1 -- 172.21.15.43:0/160977804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dbc108850 con 0x7f3dbc104d40 2024-04-25T10:30:36.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.310+0000 7f3db37fe640 1 -- 172.21.15.43:0/160977804 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f3dac042030 con 0x7f3dbc104d40 2024-04-25T10:30:36.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.311+0000 7f3db37fe640 1 --2- 172.21.15.43:0/160977804 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f3da80afb70 0x7f3da80b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:36.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.311+0000 7f3dc20a9640 1 --2- 172.21.15.43:0/160977804 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f3da80afb70 0x7f3da80b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:36.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.311+0000 7f3db37fe640 1 -- 172.21.15.43:0/160977804 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f3dac0ff2a0 con 0x7f3dbc104d40 2024-04-25T10:30:36.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.311+0000 7f3dc20a9640 1 --2- 172.21.15.43:0/160977804 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f3da80afb70 0x7f3da80b2080 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc104700 tx=0x7f3db8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:36.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.314+0000 7f3db37fe640 1 -- 172.21.15.43:0/160977804 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3dac0c7fd0 con 0x7f3dbc104d40 2024-04-25T10:30:36.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:36.542+0000 7f3dc430d640 1 -- 172.21.15.43:0/160977804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f3dbc0637a0 con 0x7f3dbc104d40 2024-04-25T10:30:37.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.172+0000 7f3db37fe640 1 -- 172.21.15.43:0/160977804 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f3dac0c78d0 con 0x7f3dbc104d40 2024-04-25T10:30:37.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.174+0000 7f3dc430d640 1 -- 172.21.15.43:0/160977804 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f3da80afb70 msgr2=0x7f3da80b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.174+0000 7f3dc430d640 1 --2- 172.21.15.43:0/160977804 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f3da80afb70 0x7f3da80b2080 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc104700 tx=0x7f3db8009500 comp rx=0 tx=0).stop 2024-04-25T10:30:37.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.174+0000 7f3dc430d640 1 -- 172.21.15.43:0/160977804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dbc104d40 msgr2=0x7f3dbc19a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.174+0000 7f3dc430d640 1 --2- 172.21.15.43:0/160977804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dbc104d40 0x7f3dbc19a830 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3dac006010 tx=0x7f3dac031a50 comp rx=0 tx=0).stop 2024-04-25T10:30:37.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.174+0000 7f3dc430d640 1 -- 172.21.15.43:0/160977804 shutdown_connections 2024-04-25T10:30:37.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.174+0000 7f3dc430d640 1 --2- 172.21.15.43:0/160977804 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f3da80afb70 0x7f3da80b2080 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.174+0000 7f3dc430d640 1 --2- 172.21.15.43:0/160977804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dbc109a40 0x7f3dbc1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.174+0000 7f3dc430d640 1 --2- 172.21.15.43:0/160977804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dbc104d40 0x7f3dbc19a830 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.174+0000 7f3dc430d640 1 --2- 172.21.15.43:0/160977804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3dbc103b00 0x7f3dbc19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.174+0000 7f3dc430d640 1 -- 172.21.15.43:0/160977804 >> 172.21.15.43:0/160977804 conn(0x7f3dbc0fed10 msgr2=0x7f3dbc105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:37.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.174+0000 7f3dc430d640 1 -- 172.21.15.43:0/160977804 shutdown_connections 2024-04-25T10:30:37.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.174+0000 7f3dc430d640 1 -- 172.21.15.43:0/160977804 wait complete. 2024-04-25T10:30:37.190 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T10:30:37.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.304+0000 7f949618f640 1 -- 172.21.15.43:0/918403004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9490103af0 msgr2=0x7f94901047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.304+0000 7f949618f640 1 --2- 172.21.15.43:0/918403004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9490103af0 0x7f94901047f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9480009900 tx=0x7f948002a410 comp rx=0 tx=0).stop 2024-04-25T10:30:37.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.304+0000 7f949618f640 1 -- 172.21.15.43:0/918403004 shutdown_connections 2024-04-25T10:30:37.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.304+0000 7f949618f640 1 --2- 172.21.15.43:0/918403004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9490104d30 0x7f9490105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.304+0000 7f949618f640 1 --2- 172.21.15.43:0/918403004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9490109a30 0x7f949010be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.304+0000 7f949618f640 1 --2- 172.21.15.43:0/918403004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9490103af0 0x7f94901047f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.304+0000 7f949618f640 1 -- 172.21.15.43:0/918403004 >> 172.21.15.43:0/918403004 conn(0x7f94900fece0 msgr2=0x7f9490101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.304+0000 7f949618f640 1 -- 172.21.15.43:0/918403004 shutdown_connections 2024-04-25T10:30:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.304+0000 7f949618f640 1 -- 172.21.15.43:0/918403004 wait complete. 2024-04-25T10:30:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.305+0000 7f949618f640 1 Processor -- start 2024-04-25T10:30:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.305+0000 7f949618f640 1 -- start start 2024-04-25T10:30:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.305+0000 7f949618f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9490103af0 0x7f9490077910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.305+0000 7f949618f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9490104d30 0x7f9490076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.305+0000 7f949618f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9490109a30 0x7f9490076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.305+0000 7f949618f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f949010f5d0 con 0x7f9490103af0 2024-04-25T10:30:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.305+0000 7f949618f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f949010f450 con 0x7f9490109a30 2024-04-25T10:30:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.305+0000 7f949618f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f949010f750 con 0x7f9490104d30 2024-04-25T10:30:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.305+0000 7f948f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9490103af0 0x7f9490077910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.305+0000 7f948f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9490103af0 0x7f9490077910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38352/0 (socket says 172.21.15.43:38352) 2024-04-25T10:30:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.305+0000 7f948f7fe640 1 -- 172.21.15.43:0/2643757997 learned_addr learned my addr 172.21.15.43:0/2643757997 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:37.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.305+0000 7f948ffff640 1 --2- 172.21.15.43:0/2643757997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9490109a30 0x7f9490076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.305+0000 7f948effd640 1 --2- 172.21.15.43:0/2643757997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9490104d30 0x7f9490076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.306+0000 7f948f7fe640 1 -- 172.21.15.43:0/2643757997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9490104d30 msgr2=0x7f9490076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.306+0000 7f948f7fe640 1 --2- 172.21.15.43:0/2643757997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9490104d30 0x7f9490076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.306+0000 7f948f7fe640 1 -- 172.21.15.43:0/2643757997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9490109a30 msgr2=0x7f9490076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.306+0000 7f948f7fe640 1 --2- 172.21.15.43:0/2643757997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9490109a30 0x7f9490076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.306+0000 7f948f7fe640 1 -- 172.21.15.43:0/2643757997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9490076c20 con 0x7f9490103af0 2024-04-25T10:30:37.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.306+0000 7f948effd640 1 --2- 172.21.15.43:0/2643757997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9490104d30 0x7f9490076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:37.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.306+0000 7f948f7fe640 1 --2- 172.21.15.43:0/2643757997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9490103af0 0x7f9490077910 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9480006010 tx=0x7f9480031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:37.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.306+0000 7f948cff9640 1 -- 172.21.15.43:0/2643757997 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948003f030 con 0x7f9490103af0 2024-04-25T10:30:37.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.306+0000 7f948cff9640 1 -- 172.21.15.43:0/2643757997 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9480003aa0 con 0x7f9490103af0 2024-04-25T10:30:37.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.306+0000 7f949618f640 1 -- 172.21.15.43:0/2643757997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94901a4350 con 0x7f9490103af0 2024-04-25T10:30:37.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.306+0000 7f948cff9640 1 -- 172.21.15.43:0/2643757997 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480035070 con 0x7f9490103af0 2024-04-25T10:30:37.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.306+0000 7f949618f640 1 -- 172.21.15.43:0/2643757997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94901a4700 con 0x7f9490103af0 2024-04-25T10:30:37.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.307+0000 7f949618f640 1 -- 172.21.15.43:0/2643757997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9490108840 con 0x7f9490103af0 2024-04-25T10:30:37.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.309+0000 7f948cff9640 1 -- 172.21.15.43:0/2643757997 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f9480042030 con 0x7f9490103af0 2024-04-25T10:30:37.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.309+0000 7f948cff9640 1 --2- 172.21.15.43:0/2643757997 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f94780afaf0 0x7f94780b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.309+0000 7f948cff9640 1 -- 172.21.15.43:0/2643757997 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f94800ff4a0 con 0x7f9490103af0 2024-04-25T10:30:37.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.309+0000 7f948effd640 1 --2- 172.21.15.43:0/2643757997 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f94780afaf0 0x7f94780b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.310+0000 7f948effd640 1 --2- 172.21.15.43:0/2643757997 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f94780afaf0 0x7f94780b2000 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9484005fd0 tx=0x7f9484005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:37.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.313+0000 7f948cff9640 1 -- 172.21.15.43:0/2643757997 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f94800c81d0 con 0x7f9490103af0 2024-04-25T10:30:37.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:37 smithi077 ceph-mon[32772]: pgmap v279: 97 pgs: 20 creating+peering, 65 active+clean, 12 unknown; 578 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 718 B/s wr, 1 op/s 2024-04-25T10:30:37.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:37 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/27581567' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T10:30:37.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:37 smithi077 ceph-mon[32772]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:30:37.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:37 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/160977804' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T10:30:37.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:37 smithi164 ceph-mon[32713]: pgmap v279: 97 pgs: 20 creating+peering, 65 active+clean, 12 unknown; 578 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 718 B/s wr, 1 op/s 2024-04-25T10:30:37.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:37 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/27581567' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T10:30:37.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:37 smithi164 ceph-mon[32713]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:30:37.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:37 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/160977804' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T10:30:37.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.540+0000 7f949618f640 1 -- 172.21.15.43:0/2643757997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f949002db50 con 0x7f9490103af0 2024-04-25T10:30:37.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:37 smithi043 ceph-mon[29478]: pgmap v279: 97 pgs: 20 creating+peering, 65 active+clean, 12 unknown; 578 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 718 B/s wr, 1 op/s 2024-04-25T10:30:37.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:37 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/27581567' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T10:30:37.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:37 smithi043 ceph-mon[29478]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:30:37.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:37 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/160977804' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T10:30:37.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.793+0000 7f948cff9640 1 -- 172.21.15.43:0/2643757997 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f94800c7ad0 con 0x7f9490103af0 2024-04-25T10:30:37.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.795+0000 7f949618f640 1 -- 172.21.15.43:0/2643757997 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f94780afaf0 msgr2=0x7f94780b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.795+0000 7f949618f640 1 --2- 172.21.15.43:0/2643757997 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f94780afaf0 0x7f94780b2000 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9484005fd0 tx=0x7f9484005850 comp rx=0 tx=0).stop 2024-04-25T10:30:37.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.795+0000 7f949618f640 1 -- 172.21.15.43:0/2643757997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9490103af0 msgr2=0x7f9490077910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.795+0000 7f949618f640 1 --2- 172.21.15.43:0/2643757997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9490103af0 0x7f9490077910 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9480006010 tx=0x7f9480031a50 comp rx=0 tx=0).stop 2024-04-25T10:30:37.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.795+0000 7f949618f640 1 -- 172.21.15.43:0/2643757997 shutdown_connections 2024-04-25T10:30:37.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.795+0000 7f949618f640 1 --2- 172.21.15.43:0/2643757997 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f94780afaf0 0x7f94780b2000 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.795+0000 7f949618f640 1 --2- 172.21.15.43:0/2643757997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9490109a30 0x7f9490076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.795+0000 7f949618f640 1 --2- 172.21.15.43:0/2643757997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9490104d30 0x7f9490076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.795+0000 7f949618f640 1 --2- 172.21.15.43:0/2643757997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9490103af0 0x7f9490077910 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.795+0000 7f949618f640 1 -- 172.21.15.43:0/2643757997 >> 172.21.15.43:0/2643757997 conn(0x7f94900fece0 msgr2=0x7f9490105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:37.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.795+0000 7f949618f640 1 -- 172.21.15.43:0/2643757997 shutdown_connections 2024-04-25T10:30:37.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.795+0000 7f949618f640 1 -- 172.21.15.43:0/2643757997 wait complete. 2024-04-25T10:30:37.811 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T10:30:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.926+0000 7f15ad396640 1 -- 172.21.15.43:0/3706793911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a8103c30 msgr2=0x7f15a8104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.926+0000 7f15ad396640 1 --2- 172.21.15.43:0/3706793911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a8103c30 0x7f15a8104040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f159c009900 tx=0x7f159c02a410 comp rx=0 tx=0).stop 2024-04-25T10:30:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.926+0000 7f15ad396640 1 -- 172.21.15.43:0/3706793911 shutdown_connections 2024-04-25T10:30:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.926+0000 7f15ad396640 1 --2- 172.21.15.43:0/3706793911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15a8108930 0x7f15a810ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.926+0000 7f15ad396640 1 --2- 172.21.15.43:0/3706793911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a8103c30 0x7f15a8104040 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.926+0000 7f15ad396640 1 --2- 172.21.15.43:0/3706793911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15a81029f0 0x7f15a81036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.926+0000 7f15ad396640 1 -- 172.21.15.43:0/3706793911 >> 172.21.15.43:0/3706793911 conn(0x7f15a80fdbc0 msgr2=0x7f15a8100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.926+0000 7f15ad396640 1 -- 172.21.15.43:0/3706793911 shutdown_connections 2024-04-25T10:30:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.926+0000 7f15ad396640 1 -- 172.21.15.43:0/3706793911 wait complete. 2024-04-25T10:30:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.927+0000 7f15ad396640 1 Processor -- start 2024-04-25T10:30:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.927+0000 7f15ad396640 1 -- start start 2024-04-25T10:30:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.927+0000 7f15ad396640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15a81029f0 0x7f15a8075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.927+0000 7f15ad396640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15a8103c30 0x7f15a819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.927+0000 7f15ad396640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a8108930 0x7f15a81a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.927+0000 7f15ad396640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f15a810e520 con 0x7f15a8108930 2024-04-25T10:30:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.927+0000 7f15ad396640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f15a810e3a0 con 0x7f15a8103c30 2024-04-25T10:30:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.927+0000 7f15ad396640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f15a810e6a0 con 0x7f15a81029f0 2024-04-25T10:30:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.928+0000 7f15a77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a8108930 0x7f15a81a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.928+0000 7f15a77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a8108930 0x7f15a81a14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38360/0 (socket says 172.21.15.43:38360) 2024-04-25T10:30:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.928+0000 7f15a67fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15a8103c30 0x7f15a819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.928+0000 7f15a77fe640 1 -- 172.21.15.43:0/306027312 learned_addr learned my addr 172.21.15.43:0/306027312 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.928+0000 7f15a6ffd640 1 --2- 172.21.15.43:0/306027312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15a81029f0 0x7f15a8075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.928+0000 7f15a77fe640 1 -- 172.21.15.43:0/306027312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15a81029f0 msgr2=0x7f15a8075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.928+0000 7f15a77fe640 1 --2- 172.21.15.43:0/306027312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15a81029f0 0x7f15a8075860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.928+0000 7f15a77fe640 1 -- 172.21.15.43:0/306027312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15a8103c30 msgr2=0x7f15a819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.928+0000 7f15a77fe640 1 --2- 172.21.15.43:0/306027312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15a8103c30 0x7f15a819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.928+0000 7f15a77fe640 1 -- 172.21.15.43:0/306027312 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a81a1d10 con 0x7f15a8108930 2024-04-25T10:30:37.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.928+0000 7f15a67fc640 1 --2- 172.21.15.43:0/306027312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15a8103c30 0x7f15a819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:37.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.928+0000 7f15a77fe640 1 --2- 172.21.15.43:0/306027312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a8108930 0x7f15a81a14f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f15a000eaa0 tx=0x7f15a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:37.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.928+0000 7f158ffff640 1 -- 172.21.15.43:0/306027312 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a000cdb0 con 0x7f15a8108930 2024-04-25T10:30:37.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.928+0000 7f158ffff640 1 -- 172.21.15.43:0/306027312 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a0005020 con 0x7f15a8108930 2024-04-25T10:30:37.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.928+0000 7f15ad396640 1 -- 172.21.15.43:0/306027312 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a81a1fa0 con 0x7f15a8108930 2024-04-25T10:30:37.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.928+0000 7f158ffff640 1 -- 172.21.15.43:0/306027312 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a00116d0 con 0x7f15a8108930 2024-04-25T10:30:37.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.928+0000 7f15ad396640 1 -- 172.21.15.43:0/306027312 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a81a2460 con 0x7f15a8108930 2024-04-25T10:30:37.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.929+0000 7f15ad396640 1 -- 172.21.15.43:0/306027312 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a8107740 con 0x7f15a8108930 2024-04-25T10:30:37.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.931+0000 7f158ffff640 1 -- 172.21.15.43:0/306027312 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f15a00051d0 con 0x7f15a8108930 2024-04-25T10:30:37.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.932+0000 7f158ffff640 1 --2- 172.21.15.43:0/306027312 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f15900afaf0 0x7f15900b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.932+0000 7f15a6ffd640 1 --2- 172.21.15.43:0/306027312 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f15900afaf0 0x7f15900b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.932+0000 7f158ffff640 1 -- 172.21.15.43:0/306027312 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f15a00deff0 con 0x7f15a8108930 2024-04-25T10:30:37.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.932+0000 7f15a6ffd640 1 --2- 172.21.15.43:0/306027312 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f15900afaf0 0x7f15900b2000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f15a81035f0 tx=0x7f1598005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:37.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:37.935+0000 7f158ffff640 1 -- 172.21.15.43:0/306027312 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15a0016030 con 0x7f15a8108930 2024-04-25T10:30:38.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.161+0000 7f15ad396640 1 -- 172.21.15.43:0/306027312 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f15a80637a0 con 0x7f15a8108930 2024-04-25T10:30:38.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:38 smithi077 ceph-mon[32772]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T10:30:38.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:38 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/160977804' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T10:30:38.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:38 smithi077 ceph-mon[32772]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T10:30:38.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:38 smithi077 ceph-mon[32772]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:30:38.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:38 smithi077 ceph-mon[32772]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T10:30:38.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:38 smithi077 ceph-mon[32772]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T10:30:38.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:38 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2643757997' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T10:30:38.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:38 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2643757997' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T10:30:38.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:38 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] up:active 2024-04-25T10:30:38.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:38 smithi077 ceph-mon[32772]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T10:30:38.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:38 smithi077 ceph-mon[32772]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T10:30:38.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:38 smithi077 ceph-mon[32772]: Cluster is now healthy 2024-04-25T10:30:38.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:38 smithi077 ceph-mon[32772]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-04-25T10:30:38.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:38 smithi077 ceph-mon[32772]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-04-25T10:30:38.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:38 smithi077 ceph-mon[32772]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-04-25T10:30:38.542 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:38 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/306027312' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T10:30:38.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:38 smithi164 ceph-mon[32713]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T10:30:38.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:38 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/160977804' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T10:30:38.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:38 smithi164 ceph-mon[32713]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T10:30:38.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:38 smithi164 ceph-mon[32713]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:30:38.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:38 smithi164 ceph-mon[32713]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T10:30:38.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:38 smithi164 ceph-mon[32713]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T10:30:38.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:38 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2643757997' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T10:30:38.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:38 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2643757997' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T10:30:38.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:38 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] up:active 2024-04-25T10:30:38.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:38 smithi164 ceph-mon[32713]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T10:30:38.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:38 smithi164 ceph-mon[32713]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T10:30:38.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:38 smithi164 ceph-mon[32713]: Cluster is now healthy 2024-04-25T10:30:38.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:38 smithi164 ceph-mon[32713]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-04-25T10:30:38.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:38 smithi164 ceph-mon[32713]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-04-25T10:30:38.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:38 smithi164 ceph-mon[32713]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-04-25T10:30:38.543 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:38 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/306027312' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T10:30:38.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:38 smithi043 ceph-mon[29478]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T10:30:38.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:38 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/160977804' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T10:30:38.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:38 smithi043 ceph-mon[29478]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T10:30:38.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:38 smithi043 ceph-mon[29478]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T10:30:38.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:38 smithi043 ceph-mon[29478]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T10:30:38.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:38 smithi043 ceph-mon[29478]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T10:30:38.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:38 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2643757997' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T10:30:38.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:38 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2643757997' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T10:30:38.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:38 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] up:active 2024-04-25T10:30:38.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:38 smithi043 ceph-mon[29478]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T10:30:38.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:38 smithi043 ceph-mon[29478]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T10:30:38.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:38 smithi043 ceph-mon[29478]: Cluster is now healthy 2024-04-25T10:30:38.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:38 smithi043 ceph-mon[29478]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-04-25T10:30:38.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:38 smithi043 ceph-mon[29478]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-04-25T10:30:38.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:38 smithi043 ceph-mon[29478]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-04-25T10:30:38.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:38 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/306027312' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T10:30:38.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.799+0000 7f158ffff640 1 -- 172.21.15.43:0/306027312 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7f15a00a7630 con 0x7f15a8108930 2024-04-25T10:30:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.802+0000 7f15ad396640 1 -- 172.21.15.43:0/306027312 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f15900afaf0 msgr2=0x7f15900b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.802+0000 7f15ad396640 1 --2- 172.21.15.43:0/306027312 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f15900afaf0 0x7f15900b2000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f15a81035f0 tx=0x7f1598005f90 comp rx=0 tx=0).stop 2024-04-25T10:30:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.802+0000 7f15ad396640 1 -- 172.21.15.43:0/306027312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a8108930 msgr2=0x7f15a81a14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.802+0000 7f15ad396640 1 --2- 172.21.15.43:0/306027312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a8108930 0x7f15a81a14f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f15a000eaa0 tx=0x7f15a000ef60 comp rx=0 tx=0).stop 2024-04-25T10:30:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.802+0000 7f15ad396640 1 -- 172.21.15.43:0/306027312 shutdown_connections 2024-04-25T10:30:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.802+0000 7f15ad396640 1 --2- 172.21.15.43:0/306027312 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f15900afaf0 0x7f15900b2000 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.802+0000 7f15ad396640 1 --2- 172.21.15.43:0/306027312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15a8103c30 0x7f15a819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.802+0000 7f15ad396640 1 --2- 172.21.15.43:0/306027312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a8108930 0x7f15a81a14f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.802+0000 7f15ad396640 1 --2- 172.21.15.43:0/306027312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15a81029f0 0x7f15a8075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.802+0000 7f15ad396640 1 -- 172.21.15.43:0/306027312 >> 172.21.15.43:0/306027312 conn(0x7f15a80fdbc0 msgr2=0x7f15a8104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.802+0000 7f15ad396640 1 -- 172.21.15.43:0/306027312 shutdown_connections 2024-04-25T10:30:38.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.802+0000 7f15ad396640 1 -- 172.21.15.43:0/306027312 wait complete. 2024-04-25T10:30:38.818 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-04-25T10:30:38.818 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T10:30:38.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.935+0000 7f4dfe449640 1 -- 172.21.15.43:0/141468636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4df81045f0 msgr2=0x7f4df8106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.935+0000 7f4dfe449640 1 --2- 172.21.15.43:0/141468636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4df81045f0 0x7f4df8106a10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009900 tx=0x7f4de002a410 comp rx=0 tx=0).stop 2024-04-25T10:30:38.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.935+0000 7f4dfe449640 1 -- 172.21.15.43:0/141468636 shutdown_connections 2024-04-25T10:30:38.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.935+0000 7f4dfe449640 1 --2- 172.21.15.43:0/141468636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4df8106f50 0x7f4df8109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.935+0000 7f4dfe449640 1 --2- 172.21.15.43:0/141468636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4df81098b0 0x7f4df810bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.935+0000 7f4dfe449640 1 --2- 172.21.15.43:0/141468636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4df81045f0 0x7f4df8106a10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.935+0000 7f4dfe449640 1 -- 172.21.15.43:0/141468636 >> 172.21.15.43:0/141468636 conn(0x7f4df80fdbf0 msgr2=0x7f4df81000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.936+0000 7f4dfe449640 1 -- 172.21.15.43:0/141468636 shutdown_connections 2024-04-25T10:30:38.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.936+0000 7f4dfe449640 1 -- 172.21.15.43:0/141468636 wait complete. 2024-04-25T10:30:38.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.936+0000 7f4dfe449640 1 Processor -- start 2024-04-25T10:30:38.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.936+0000 7f4dfe449640 1 -- start start 2024-04-25T10:30:38.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.936+0000 7f4dfe449640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4df81045f0 0x7f4df819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4dfe449640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4df8106f50 0x7f4df819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4dfe449640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4df81098b0 0x7f4df81a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4dfe449640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4df810f680 con 0x7f4df8106f50 2024-04-25T10:30:38.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4dfe449640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4df810f500 con 0x7f4df81045f0 2024-04-25T10:30:38.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4dfe449640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4df810f800 con 0x7f4df81098b0 2024-04-25T10:30:38.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4df77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4df8106f50 0x7f4df819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4df7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4df81045f0 0x7f4df819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4df77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4df8106f50 0x7f4df819a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39304/0 (socket says 172.21.15.43:39304) 2024-04-25T10:30:38.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4df77fe640 1 -- 172.21.15.43:0/2827491392 learned_addr learned my addr 172.21.15.43:0/2827491392 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:38.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4dfc9e6640 1 --2- 172.21.15.43:0/2827491392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4df81098b0 0x7f4df81a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4df7fff640 1 -- 172.21.15.43:0/2827491392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4df81098b0 msgr2=0x7f4df81a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4df7fff640 1 --2- 172.21.15.43:0/2827491392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4df81098b0 0x7f4df81a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4df7fff640 1 -- 172.21.15.43:0/2827491392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4df8106f50 msgr2=0x7f4df819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4df7fff640 1 --2- 172.21.15.43:0/2827491392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4df8106f50 0x7f4df819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4df7fff640 1 -- 172.21.15.43:0/2827491392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df81a1e80 con 0x7f4df81045f0 2024-04-25T10:30:38.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4dfc9e6640 1 --2- 172.21.15.43:0/2827491392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4df81098b0 0x7f4df81a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:38.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4df7fff640 1 --2- 172.21.15.43:0/2827491392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4df81045f0 0x7f4df819a340 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4de0006010 tx=0x7f4de0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4df57fa640 1 -- 172.21.15.43:0/2827491392 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de003f070 con 0x7f4df81045f0 2024-04-25T10:30:38.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4dfe449640 1 -- 172.21.15.43:0/2827491392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df81a2110 con 0x7f4df81045f0 2024-04-25T10:30:38.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4df57fa640 1 -- 172.21.15.43:0/2827491392 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4de0003ca0 con 0x7f4df81045f0 2024-04-25T10:30:38.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4df57fa640 1 -- 172.21.15.43:0/2827491392 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de0035070 con 0x7f4df81045f0 2024-04-25T10:30:38.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.937+0000 7f4dfe449640 1 -- 172.21.15.43:0/2827491392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df81a2680 con 0x7f4df81045f0 2024-04-25T10:30:38.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.938+0000 7f4dfe449640 1 -- 172.21.15.43:0/2827491392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4df80ff710 con 0x7f4df81045f0 2024-04-25T10:30:38.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.940+0000 7f4df57fa640 1 -- 172.21.15.43:0/2827491392 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f4de003b460 con 0x7f4df81045f0 2024-04-25T10:30:38.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.941+0000 7f4df57fa640 1 --2- 172.21.15.43:0/2827491392 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f4de40afaf0 0x7f4de40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.941+0000 7f4df77fe640 1 --2- 172.21.15.43:0/2827491392 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f4de40afaf0 0x7f4de40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.941+0000 7f4df57fa640 1 -- 172.21.15.43:0/2827491392 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f4de00ff380 con 0x7f4df81045f0 2024-04-25T10:30:38.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.941+0000 7f4df77fe640 1 --2- 172.21.15.43:0/2827491392 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f4de40afaf0 0x7f4de40b2000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4df819b3e0 tx=0x7f4de8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:38.949+0000 7f4df57fa640 1 -- 172.21.15.43:0/2827491392 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4de00c8090 con 0x7f4df81045f0 2024-04-25T10:30:39.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.161+0000 7f4dfe449640 1 -- 172.21.15.43:0/2827491392 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4df8062fe0 con 0x7f4de40afaf0 2024-04-25T10:30:39.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.186+0000 7f4df57fa640 1 -- 172.21.15.43:0/2827491392 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4df8062fe0 con 0x7f4de40afaf0 2024-04-25T10:30:39.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.188+0000 7f4dfe449640 1 -- 172.21.15.43:0/2827491392 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f4de40afaf0 msgr2=0x7f4de40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.188+0000 7f4dfe449640 1 --2- 172.21.15.43:0/2827491392 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f4de40afaf0 0x7f4de40b2000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4df819b3e0 tx=0x7f4de8005f90 comp rx=0 tx=0).stop 2024-04-25T10:30:39.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.188+0000 7f4dfe449640 1 -- 172.21.15.43:0/2827491392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4df81045f0 msgr2=0x7f4df819a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.188+0000 7f4dfe449640 1 --2- 172.21.15.43:0/2827491392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4df81045f0 0x7f4df819a340 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4de0006010 tx=0x7f4de0004270 comp rx=0 tx=0).stop 2024-04-25T10:30:39.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.188+0000 7f4dfe449640 1 -- 172.21.15.43:0/2827491392 shutdown_connections 2024-04-25T10:30:39.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.188+0000 7f4dfe449640 1 --2- 172.21.15.43:0/2827491392 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f4de40afaf0 0x7f4de40b2000 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.188+0000 7f4dfe449640 1 --2- 172.21.15.43:0/2827491392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4df81098b0 0x7f4df81a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.188+0000 7f4dfe449640 1 --2- 172.21.15.43:0/2827491392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4df8106f50 0x7f4df819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.188+0000 7f4dfe449640 1 --2- 172.21.15.43:0/2827491392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4df81045f0 0x7f4df819a340 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.188+0000 7f4dfe449640 1 -- 172.21.15.43:0/2827491392 >> 172.21.15.43:0/2827491392 conn(0x7f4df80fdbf0 msgr2=0x7f4df8106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.188+0000 7f4dfe449640 1 -- 172.21.15.43:0/2827491392 shutdown_connections 2024-04-25T10:30:39.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.188+0000 7f4dfe449640 1 -- 172.21.15.43:0/2827491392 wait complete. 2024-04-25T10:30:39.201 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T10:30:39.290 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:30:38 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-b[63685]: 2024-04-25T10:30:38.911+0000 7f44b5f81700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:30:39.290 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:30:38 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-b[63685]: 2024-04-25T10:30:38.911+0000 7f44b5f81700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T10:30:39.290 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:30:38 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-b[63685]: 2024-04-25T10:30:38.911+0000 7f44b5f81700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-25T10:30:39.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.315+0000 7f37379c5640 1 -- 172.21.15.43:0/3027440623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3730108940 msgr2=0x7f373010ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.315+0000 7f37379c5640 1 --2- 172.21.15.43:0/3027440623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3730108940 0x7f373010ad60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3728009ca0 tx=0x7f372802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.315+0000 7f37379c5640 1 -- 172.21.15.43:0/3027440623 shutdown_connections 2024-04-25T10:30:39.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.315+0000 7f37379c5640 1 --2- 172.21.15.43:0/3027440623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3730108940 0x7f373010ad60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.315+0000 7f37379c5640 1 --2- 172.21.15.43:0/3027440623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3730103c40 0x7f3730104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.315+0000 7f37379c5640 1 --2- 172.21.15.43:0/3027440623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3730102a00 0x7f3730103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.315+0000 7f37379c5640 1 -- 172.21.15.43:0/3027440623 >> 172.21.15.43:0/3027440623 conn(0x7f37300fdbf0 msgr2=0x7f37301000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.315+0000 7f37379c5640 1 -- 172.21.15.43:0/3027440623 shutdown_connections 2024-04-25T10:30:39.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.316+0000 7f37379c5640 1 -- 172.21.15.43:0/3027440623 wait complete. 2024-04-25T10:30:39.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.316+0000 7f37379c5640 1 Processor -- start 2024-04-25T10:30:39.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.316+0000 7f37379c5640 1 -- start start 2024-04-25T10:30:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.316+0000 7f37379c5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3730102a00 0x7f3730199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.316+0000 7f37379c5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3730103c40 0x7f3730199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.316+0000 7f37379c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3730108940 0x7f37301a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.316+0000 7f37379c5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f373010e530 con 0x7f3730108940 2024-04-25T10:30:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.316+0000 7f37379c5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f373010e3b0 con 0x7f3730102a00 2024-04-25T10:30:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.316+0000 7f37379c5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f373010e6b0 con 0x7f3730103c40 2024-04-25T10:30:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.317+0000 7f3735f62640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3730108940 0x7f37301a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.317+0000 7f3735761640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3730102a00 0x7f3730199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.317+0000 7f3735f62640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3730108940 0x7f37301a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39338/0 (socket says 172.21.15.43:39338) 2024-04-25T10:30:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.317+0000 7f3735f62640 1 -- 172.21.15.43:0/3843507423 learned_addr learned my addr 172.21.15.43:0/3843507423 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.317+0000 7f3734f60640 1 --2- 172.21.15.43:0/3843507423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3730103c40 0x7f3730199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.317+0000 7f3735f62640 1 -- 172.21.15.43:0/3843507423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3730103c40 msgr2=0x7f3730199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.317+0000 7f3735f62640 1 --2- 172.21.15.43:0/3843507423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3730103c40 0x7f3730199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.317+0000 7f3735f62640 1 -- 172.21.15.43:0/3843507423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3730102a00 msgr2=0x7f3730199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.317+0000 7f3735f62640 1 --2- 172.21.15.43:0/3843507423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3730102a00 0x7f3730199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.317+0000 7f3735f62640 1 -- 172.21.15.43:0/3843507423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3730074dd0 con 0x7f3730108940 2024-04-25T10:30:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.317+0000 7f3735761640 1 --2- 172.21.15.43:0/3843507423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3730102a00 0x7f3730199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.317+0000 7f3735f62640 1 --2- 172.21.15.43:0/3843507423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3730108940 0x7f37301a05e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3728005e30 tx=0x7f3728031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.317+0000 7f37267fc640 1 -- 172.21.15.43:0/3843507423 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728003b70 con 0x7f3730108940 2024-04-25T10:30:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.317+0000 7f37267fc640 1 -- 172.21.15.43:0/3843507423 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3728003d10 con 0x7f3730108940 2024-04-25T10:30:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.317+0000 7f37267fc640 1 -- 172.21.15.43:0/3843507423 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372802fac0 con 0x7f3730108940 2024-04-25T10:30:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.317+0000 7f37379c5640 1 -- 172.21.15.43:0/3843507423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3730075060 con 0x7f3730108940 2024-04-25T10:30:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.317+0000 7f37379c5640 1 -- 172.21.15.43:0/3843507423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37300755b0 con 0x7f3730108940 2024-04-25T10:30:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.318+0000 7f37379c5640 1 -- 172.21.15.43:0/3843507423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3730107750 con 0x7f3730108940 2024-04-25T10:30:39.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.320+0000 7f37267fc640 1 -- 172.21.15.43:0/3843507423 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f372802fc60 con 0x7f3730108940 2024-04-25T10:30:39.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.321+0000 7f37267fc640 1 --2- 172.21.15.43:0/3843507423 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f371c0afaf0 0x7f371c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.321+0000 7f3735761640 1 --2- 172.21.15.43:0/3843507423 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f371c0afaf0 0x7f371c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.321+0000 7f37267fc640 1 -- 172.21.15.43:0/3843507423 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f3728100140 con 0x7f3730108940 2024-04-25T10:30:39.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.321+0000 7f3735761640 1 --2- 172.21.15.43:0/3843507423 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f371c0afaf0 0x7f371c0b2000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3730103600 tx=0x7f372c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.324+0000 7f37267fc640 1 -- 172.21.15.43:0/3843507423 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f37280c8dd0 con 0x7f3730108940 2024-04-25T10:30:39.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.541+0000 7f37379c5640 1 -- 172.21.15.43:0/3843507423 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3730075880 con 0x7f371c0afaf0 2024-04-25T10:30:39.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.566+0000 7f37267fc640 1 -- 172.21.15.43:0/3843507423 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3730075880 con 0x7f371c0afaf0 2024-04-25T10:30:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.568+0000 7f37379c5640 1 -- 172.21.15.43:0/3843507423 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f371c0afaf0 msgr2=0x7f371c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.568+0000 7f37379c5640 1 --2- 172.21.15.43:0/3843507423 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f371c0afaf0 0x7f371c0b2000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3730103600 tx=0x7f372c005f90 comp rx=0 tx=0).stop 2024-04-25T10:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.568+0000 7f37379c5640 1 -- 172.21.15.43:0/3843507423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3730108940 msgr2=0x7f37301a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.568+0000 7f37379c5640 1 --2- 172.21.15.43:0/3843507423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3730108940 0x7f37301a05e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3728005e30 tx=0x7f3728031840 comp rx=0 tx=0).stop 2024-04-25T10:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.568+0000 7f37379c5640 1 -- 172.21.15.43:0/3843507423 shutdown_connections 2024-04-25T10:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.568+0000 7f37379c5640 1 --2- 172.21.15.43:0/3843507423 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f371c0afaf0 0x7f371c0b2000 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.568+0000 7f37379c5640 1 --2- 172.21.15.43:0/3843507423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3730108940 0x7f37301a05e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.568+0000 7f37379c5640 1 --2- 172.21.15.43:0/3843507423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3730103c40 0x7f3730199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.568+0000 7f37379c5640 1 --2- 172.21.15.43:0/3843507423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3730102a00 0x7f3730199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.568+0000 7f37379c5640 1 -- 172.21.15.43:0/3843507423 >> 172.21.15.43:0/3843507423 conn(0x7f37300fdbf0 msgr2=0x7f3730104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.568+0000 7f37379c5640 1 -- 172.21.15.43:0/3843507423 shutdown_connections 2024-04-25T10:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.568+0000 7f37379c5640 1 -- 172.21.15.43:0/3843507423 wait complete. 2024-04-25T10:30:39.581 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:30:39 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:30:39.166+0000 7ffadeec9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:30:39.581 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:30:39 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:30:39.166+0000 7ffadeec9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:30:39.581 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:30:39 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:30:39.166+0000 7ffadeec9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:30:39.581 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:30:39 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:30:39.166+0000 7ffadeec9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:30:39.581 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:30:39 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:30:39.166+0000 7ffadeec9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:30:39.582 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T10:30:39.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.697+0000 7fb4acc4a640 1 -- 172.21.15.43:0/1954029902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a8108920 msgr2=0x7fb4a810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.697+0000 7fb4acc4a640 1 --2- 172.21.15.43:0/1954029902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a8108920 0x7fb4a810ad40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009ca0 tx=0x7fb4a002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.697+0000 7fb4acc4a640 1 -- 172.21.15.43:0/1954029902 shutdown_connections 2024-04-25T10:30:39.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.697+0000 7fb4acc4a640 1 --2- 172.21.15.43:0/1954029902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4a8103c20 0x7fb4a8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.697+0000 7fb4acc4a640 1 --2- 172.21.15.43:0/1954029902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a8108920 0x7fb4a810ad40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.697+0000 7fb4acc4a640 1 --2- 172.21.15.43:0/1954029902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4a81029e0 0x7fb4a81036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.697+0000 7fb4acc4a640 1 -- 172.21.15.43:0/1954029902 >> 172.21.15.43:0/1954029902 conn(0x7fb4a80fdbd0 msgr2=0x7fb4a8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.697+0000 7fb4acc4a640 1 -- 172.21.15.43:0/1954029902 shutdown_connections 2024-04-25T10:30:39.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.697+0000 7fb4acc4a640 1 -- 172.21.15.43:0/1954029902 wait complete. 2024-04-25T10:30:39.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.698+0000 7fb4acc4a640 1 Processor -- start 2024-04-25T10:30:39.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.698+0000 7fb4acc4a640 1 -- start start 2024-04-25T10:30:39.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.698+0000 7fb4acc4a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4a81029e0 0x7fb4a8199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.698+0000 7fb4acc4a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a8103c20 0x7fb4a8199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.698+0000 7fb4acc4a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4a8108920 0x7fb4a81a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.698+0000 7fb4acc4a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a810e510 con 0x7fb4a8103c20 2024-04-25T10:30:39.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.698+0000 7fb4acc4a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a810e390 con 0x7fb4a81029e0 2024-04-25T10:30:39.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.698+0000 7fb4acc4a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a810e690 con 0x7fb4a8108920 2024-04-25T10:30:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.698+0000 7fb4a5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a8103c20 0x7fb4a8199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.698+0000 7fb4a5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a8103c20 0x7fb4a8199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39344/0 (socket says 172.21.15.43:39344) 2024-04-25T10:30:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.698+0000 7fb4a5d9b640 1 -- 172.21.15.43:0/2460602386 learned_addr learned my addr 172.21.15.43:0/2460602386 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.698+0000 7fb4a659c640 1 --2- 172.21.15.43:0/2460602386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4a81029e0 0x7fb4a8199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.698+0000 7fb4a5d9b640 1 -- 172.21.15.43:0/2460602386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4a8108920 msgr2=0x7fb4a81a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.698+0000 7fb4a5d9b640 1 --2- 172.21.15.43:0/2460602386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4a8108920 0x7fb4a81a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.698+0000 7fb4a5d9b640 1 -- 172.21.15.43:0/2460602386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4a81029e0 msgr2=0x7fb4a8199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.698+0000 7fb4a5d9b640 1 --2- 172.21.15.43:0/2460602386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4a81029e0 0x7fb4a8199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.698+0000 7fb4a5d9b640 1 -- 172.21.15.43:0/2460602386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a81a0f50 con 0x7fb4a8103c20 2024-04-25T10:30:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.699+0000 7fb4a5d9b640 1 --2- 172.21.15.43:0/2460602386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a8103c20 0x7fb4a8199950 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb49800ea60 tx=0x7fb49800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.699+0000 7fb4977fe640 1 -- 172.21.15.43:0/2460602386 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49800cde0 con 0x7fb4a8103c20 2024-04-25T10:30:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.699+0000 7fb4977fe640 1 -- 172.21.15.43:0/2460602386 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4980035f0 con 0x7fb4a8103c20 2024-04-25T10:30:39.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.699+0000 7fb4acc4a640 1 -- 172.21.15.43:0/2460602386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a81a1240 con 0x7fb4a8103c20 2024-04-25T10:30:39.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.699+0000 7fb4977fe640 1 -- 172.21.15.43:0/2460602386 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498010690 con 0x7fb4a8103c20 2024-04-25T10:30:39.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.699+0000 7fb4acc4a640 1 -- 172.21.15.43:0/2460602386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a81a1790 con 0x7fb4a8103c20 2024-04-25T10:30:39.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.700+0000 7fb4acc4a640 1 -- 172.21.15.43:0/2460602386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a8107730 con 0x7fb4a8103c20 2024-04-25T10:30:39.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.701+0000 7fb4977fe640 1 -- 172.21.15.43:0/2460602386 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fb4980037a0 con 0x7fb4a8103c20 2024-04-25T10:30:39.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.702+0000 7fb4977fe640 1 --2- 172.21.15.43:0/2460602386 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb4900afaf0 0x7fb4900b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.702+0000 7fb4a659c640 1 --2- 172.21.15.43:0/2460602386 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb4900afaf0 0x7fb4900b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.702+0000 7fb4977fe640 1 -- 172.21.15.43:0/2460602386 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fb4980df0f0 con 0x7fb4a8103c20 2024-04-25T10:30:39.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.703+0000 7fb4a659c640 1 --2- 172.21.15.43:0/2460602386 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb4900afaf0 0x7fb4900b2000 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb49c0058c0 tx=0x7fb49c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.706+0000 7fb4977fe640 1 -- 172.21.15.43:0/2460602386 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb498053ff0 con 0x7fb4a8103c20 2024-04-25T10:30:39.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:39 smithi043 ceph-mon[29478]: pgmap v280: 97 pgs: 8 creating+peering, 89 active+clean; 580 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-04-25T10:30:39.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:39 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/306027312' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T10:30:39.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:39 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] up:active 2024-04-25T10:30:39.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:39 smithi043 ceph-mon[29478]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T10:30:39.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:39 smithi043 ceph-mon[29478]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T10:30:39.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.925+0000 7fb4acc4a640 1 -- 172.21.15.43:0/2460602386 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fb4a802dda0 con 0x7fb4900afaf0 2024-04-25T10:30:39.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.931+0000 7fb4977fe640 1 -- 172.21.15.43:0/2460602386 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb4a802dda0 con 0x7fb4900afaf0 2024-04-25T10:30:39.933 INFO:teuthology.orchestra.run.smithi043.stdout:/volumes/qa/sv_0/560ae1f3-284b-4299-9239-9b42c9e2293a 2024-04-25T10:30:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.933+0000 7fb4acc4a640 1 -- 172.21.15.43:0/2460602386 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb4900afaf0 msgr2=0x7fb4900b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.933+0000 7fb4acc4a640 1 --2- 172.21.15.43:0/2460602386 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb4900afaf0 0x7fb4900b2000 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb49c0058c0 tx=0x7fb49c00a2d0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.933+0000 7fb4acc4a640 1 -- 172.21.15.43:0/2460602386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a8103c20 msgr2=0x7fb4a8199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.933+0000 7fb4acc4a640 1 --2- 172.21.15.43:0/2460602386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a8103c20 0x7fb4a8199950 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb49800ea60 tx=0x7fb49800ef20 comp rx=0 tx=0).stop 2024-04-25T10:30:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.933+0000 7fb4acc4a640 1 -- 172.21.15.43:0/2460602386 shutdown_connections 2024-04-25T10:30:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.933+0000 7fb4acc4a640 1 --2- 172.21.15.43:0/2460602386 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb4900afaf0 0x7fb4900b2000 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.933+0000 7fb4acc4a640 1 --2- 172.21.15.43:0/2460602386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4a8108920 0x7fb4a81a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.933+0000 7fb4acc4a640 1 --2- 172.21.15.43:0/2460602386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a8103c20 0x7fb4a8199950 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.933+0000 7fb4acc4a640 1 --2- 172.21.15.43:0/2460602386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4a81029e0 0x7fb4a8199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.933+0000 7fb4acc4a640 1 -- 172.21.15.43:0/2460602386 >> 172.21.15.43:0/2460602386 conn(0x7fb4a80fdbd0 msgr2=0x7fb4a8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.933+0000 7fb4acc4a640 1 -- 172.21.15.43:0/2460602386 shutdown_connections 2024-04-25T10:30:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:39.933+0000 7fb4acc4a640 1 -- 172.21.15.43:0/2460602386 wait complete. 2024-04-25T10:30:39.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T10:30:40.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.060+0000 7f5daa292640 1 -- 172.21.15.43:0/1775806348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5da41045d0 msgr2=0x7f5da41069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.060+0000 7f5daa292640 1 --2- 172.21.15.43:0/1775806348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5da41045d0 0x7f5da41069f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009900 tx=0x7f5d9402a410 comp rx=0 tx=0).stop 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.060+0000 7f5daa292640 1 -- 172.21.15.43:0/1775806348 shutdown_connections 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.060+0000 7f5daa292640 1 --2- 172.21.15.43:0/1775806348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5da4109890 0x7f5da410bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.060+0000 7f5daa292640 1 --2- 172.21.15.43:0/1775806348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da4106f30 0x7f5da4109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.060+0000 7f5daa292640 1 --2- 172.21.15.43:0/1775806348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5da41045d0 0x7f5da41069f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.060+0000 7f5daa292640 1 -- 172.21.15.43:0/1775806348 >> 172.21.15.43:0/1775806348 conn(0x7f5da40fdbd0 msgr2=0x7f5da4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.060+0000 7f5daa292640 1 -- 172.21.15.43:0/1775806348 shutdown_connections 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.060+0000 7f5daa292640 1 -- 172.21.15.43:0/1775806348 wait complete. 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.061+0000 7f5daa292640 1 Processor -- start 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.061+0000 7f5daa292640 1 -- start start 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.061+0000 7f5daa292640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5da41045d0 0x7f5da419a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.061+0000 7f5daa292640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5da4106f30 0x7f5da419a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.061+0000 7f5daa292640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da4109890 0x7f5da41a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:40.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.061+0000 7f5daa292640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5da410f630 con 0x7f5da4106f30 2024-04-25T10:30:40.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.061+0000 7f5daa292640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5da410f4b0 con 0x7f5da41045d0 2024-04-25T10:30:40.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.061+0000 7f5daa292640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5da410f7b0 con 0x7f5da4109890 2024-04-25T10:30:40.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.062+0000 7f5da37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5da4106f30 0x7f5da419a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:40.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.062+0000 7f5da3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5da41045d0 0x7f5da419a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:40.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.062+0000 7f5da37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5da4106f30 0x7f5da419a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39372/0 (socket says 172.21.15.43:39372) 2024-04-25T10:30:40.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.062+0000 7f5da37fe640 1 -- 172.21.15.43:0/3059878066 learned_addr learned my addr 172.21.15.43:0/3059878066 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:40.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.062+0000 7f5da3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5da41045d0 0x7f5da419a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:34224/0 (socket says 172.21.15.43:34224) 2024-04-25T10:30:40.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.062+0000 7f5da882f640 1 --2- 172.21.15.43:0/3059878066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da4109890 0x7f5da41a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:40.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.062+0000 7f5da37fe640 1 -- 172.21.15.43:0/3059878066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da4109890 msgr2=0x7f5da41a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:40.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.062+0000 7f5da37fe640 1 --2- 172.21.15.43:0/3059878066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da4109890 0x7f5da41a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.062+0000 7f5da37fe640 1 -- 172.21.15.43:0/3059878066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5da41045d0 msgr2=0x7f5da419a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:40.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.062+0000 7f5da37fe640 1 --2- 172.21.15.43:0/3059878066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5da41045d0 0x7f5da419a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.062+0000 7f5da37fe640 1 -- 172.21.15.43:0/3059878066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da41a1e10 con 0x7f5da4106f30 2024-04-25T10:30:40.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.062+0000 7f5da3fff640 1 --2- 172.21.15.43:0/3059878066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5da41045d0 0x7f5da419a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:40.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.062+0000 7f5da37fe640 1 --2- 172.21.15.43:0/3059878066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5da4106f30 0x7f5da419a810 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800c970 tx=0x7f5d9800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:40.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.062+0000 7f5da17fa640 1 -- 172.21.15.43:0/3059878066 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98007bb0 con 0x7f5da4106f30 2024-04-25T10:30:40.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.062+0000 7f5daa292640 1 -- 172.21.15.43:0/3059878066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da41a2100 con 0x7f5da4106f30 2024-04-25T10:30:40.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.062+0000 7f5da17fa640 1 -- 172.21.15.43:0/3059878066 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d98007d50 con 0x7f5da4106f30 2024-04-25T10:30:40.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.062+0000 7f5da17fa640 1 -- 172.21.15.43:0/3059878066 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9800f450 con 0x7f5da4106f30 2024-04-25T10:30:40.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.062+0000 7f5daa292640 1 -- 172.21.15.43:0/3059878066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da41a2600 con 0x7f5da4106f30 2024-04-25T10:30:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.063+0000 7f5daa292640 1 -- 172.21.15.43:0/3059878066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da40ff6f0 con 0x7f5da4106f30 2024-04-25T10:30:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.066+0000 7f5da17fa640 1 -- 172.21.15.43:0/3059878066 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f5d98004f00 con 0x7f5da4106f30 2024-04-25T10:30:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.066+0000 7f5da17fa640 1 --2- 172.21.15.43:0/3059878066 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5d8c0afaf0 0x7f5d8c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.067+0000 7f5da3fff640 1 --2- 172.21.15.43:0/3059878066 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5d8c0afaf0 0x7f5d8c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.067+0000 7f5da17fa640 1 -- 172.21.15.43:0/3059878066 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f5d980deb60 con 0x7f5da4106f30 2024-04-25T10:30:40.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.067+0000 7f5da3fff640 1 --2- 172.21.15.43:0/3059878066 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5d8c0afaf0 0x7f5d8c0b2000 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5d94031440 tx=0x7f5d940314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:40.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.070+0000 7f5da17fa640 1 -- 172.21.15.43:0/3059878066 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d98014030 con 0x7f5da4106f30 2024-04-25T10:30:40.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.284+0000 7f5daa292640 1 -- 172.21.15.43:0/3059878066 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5da4062fe0 con 0x7f5d8c0afaf0 2024-04-25T10:30:40.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:39 smithi077 ceph-mon[32772]: pgmap v280: 97 pgs: 8 creating+peering, 89 active+clean; 580 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-04-25T10:30:40.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:39 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/306027312' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T10:30:40.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:39 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] up:active 2024-04-25T10:30:40.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:39 smithi077 ceph-mon[32772]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T10:30:40.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:39 smithi077 ceph-mon[32772]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T10:30:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:39 smithi164 ceph-mon[32713]: pgmap v280: 97 pgs: 8 creating+peering, 89 active+clean; 580 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-04-25T10:30:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:39 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/306027312' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T10:30:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:39 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] up:active 2024-04-25T10:30:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:39 smithi164 ceph-mon[32713]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T10:30:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:39 smithi164 ceph-mon[32713]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T10:30:40.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.309+0000 7f5da17fa640 1 -- 172.21.15.43:0/3059878066 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5da4062fe0 con 0x7f5d8c0afaf0 2024-04-25T10:30:40.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.311+0000 7f5daa292640 1 -- 172.21.15.43:0/3059878066 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5d8c0afaf0 msgr2=0x7f5d8c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:40.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.311+0000 7f5daa292640 1 --2- 172.21.15.43:0/3059878066 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5d8c0afaf0 0x7f5d8c0b2000 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5d94031440 tx=0x7f5d940314a0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.311+0000 7f5daa292640 1 -- 172.21.15.43:0/3059878066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5da4106f30 msgr2=0x7f5da419a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:40.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.311+0000 7f5daa292640 1 --2- 172.21.15.43:0/3059878066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5da4106f30 0x7f5da419a810 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800c970 tx=0x7f5d9800ce30 comp rx=0 tx=0).stop 2024-04-25T10:30:40.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.311+0000 7f5daa292640 1 -- 172.21.15.43:0/3059878066 shutdown_connections 2024-04-25T10:30:40.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.311+0000 7f5daa292640 1 --2- 172.21.15.43:0/3059878066 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5d8c0afaf0 0x7f5d8c0b2000 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.311+0000 7f5daa292640 1 --2- 172.21.15.43:0/3059878066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da4109890 0x7f5da41a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.311+0000 7f5daa292640 1 --2- 172.21.15.43:0/3059878066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5da4106f30 0x7f5da419a810 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.311+0000 7f5daa292640 1 --2- 172.21.15.43:0/3059878066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5da41045d0 0x7f5da419a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.311+0000 7f5daa292640 1 -- 172.21.15.43:0/3059878066 >> 172.21.15.43:0/3059878066 conn(0x7f5da40fdbd0 msgr2=0x7f5da4106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:40.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.311+0000 7f5daa292640 1 -- 172.21.15.43:0/3059878066 shutdown_connections 2024-04-25T10:30:40.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.311+0000 7f5daa292640 1 -- 172.21.15.43:0/3059878066 wait complete. 2024-04-25T10:30:40.325 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T10:30:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.439+0000 7f2ee29e3640 1 -- 172.21.15.43:0/919939446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2edc1033d0 msgr2=0x7f2edc1037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.439+0000 7f2ee29e3640 1 --2- 172.21.15.43:0/919939446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2edc1033d0 0x7f2edc1037e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009900 tx=0x7f2ed002a410 comp rx=0 tx=0).stop 2024-04-25T10:30:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.439+0000 7f2ee29e3640 1 -- 172.21.15.43:0/919939446 shutdown_connections 2024-04-25T10:30:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.439+0000 7f2ee29e3640 1 --2- 172.21.15.43:0/919939446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2edc1080d0 0x7f2edc10a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.439+0000 7f2ee29e3640 1 --2- 172.21.15.43:0/919939446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2edc1033d0 0x7f2edc1037e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.439+0000 7f2ee29e3640 1 --2- 172.21.15.43:0/919939446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2edc102190 0x7f2edc102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.439+0000 7f2ee29e3640 1 -- 172.21.15.43:0/919939446 >> 172.21.15.43:0/919939446 conn(0x7f2edc0fd380 msgr2=0x7f2edc0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.439+0000 7f2ee29e3640 1 -- 172.21.15.43:0/919939446 shutdown_connections 2024-04-25T10:30:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.439+0000 7f2ee29e3640 1 -- 172.21.15.43:0/919939446 wait complete. 2024-04-25T10:30:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.440+0000 7f2ee29e3640 1 Processor -- start 2024-04-25T10:30:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.440+0000 7f2ee29e3640 1 -- start start 2024-04-25T10:30:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.440+0000 7f2ee29e3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2edc102190 0x7f2edc19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.440+0000 7f2ee29e3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2edc1033d0 0x7f2edc19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.440+0000 7f2ee29e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2edc1080d0 0x7f2edc1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.440+0000 7f2ee29e3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc10f630 con 0x7f2edc1080d0 2024-04-25T10:30:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.440+0000 7f2ee29e3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc10f4b0 con 0x7f2edc1033d0 2024-04-25T10:30:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.440+0000 7f2ee29e3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc10f7b0 con 0x7f2edc102190 2024-04-25T10:30:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.440+0000 7f2ee0f80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2edc1080d0 0x7f2edc1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.440+0000 7f2ee0f80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2edc1080d0 0x7f2edc1a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39402/0 (socket says 172.21.15.43:39402) 2024-04-25T10:30:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.440+0000 7f2ee0f80640 1 -- 172.21.15.43:0/345037354 learned_addr learned my addr 172.21.15.43:0/345037354 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.440+0000 7f2edbfff640 1 --2- 172.21.15.43:0/345037354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2edc102190 0x7f2edc19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.441+0000 7f2edb7fe640 1 --2- 172.21.15.43:0/345037354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2edc1033d0 0x7f2edc19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.441+0000 7f2ee0f80640 1 -- 172.21.15.43:0/345037354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2edc102190 msgr2=0x7f2edc19a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.441+0000 7f2ee0f80640 1 --2- 172.21.15.43:0/345037354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2edc102190 0x7f2edc19a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.441+0000 7f2ee0f80640 1 -- 172.21.15.43:0/345037354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2edc1033d0 msgr2=0x7f2edc19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:40.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.441+0000 7f2ee0f80640 1 --2- 172.21.15.43:0/345037354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2edc1033d0 0x7f2edc19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.441+0000 7f2ee0f80640 1 -- 172.21.15.43:0/345037354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2edc1a1cf0 con 0x7f2edc1080d0 2024-04-25T10:30:40.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.441+0000 7f2ee0f80640 1 --2- 172.21.15.43:0/345037354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2edc1080d0 0x7f2edc1a1730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400cce0 tx=0x7f2ed400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:40.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.441+0000 7f2ed97fa640 1 -- 172.21.15.43:0/345037354 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed400f450 con 0x7f2edc1080d0 2024-04-25T10:30:40.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.441+0000 7f2ed97fa640 1 -- 172.21.15.43:0/345037354 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ed400faa0 con 0x7f2edc1080d0 2024-04-25T10:30:40.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.441+0000 7f2ee29e3640 1 -- 172.21.15.43:0/345037354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2edc1a1fe0 con 0x7f2edc1080d0 2024-04-25T10:30:40.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.441+0000 7f2ed97fa640 1 -- 172.21.15.43:0/345037354 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4018b70 con 0x7f2edc1080d0 2024-04-25T10:30:40.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.441+0000 7f2ee29e3640 1 -- 172.21.15.43:0/345037354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2edc1a23f0 con 0x7f2edc1080d0 2024-04-25T10:30:40.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.442+0000 7f2ee29e3640 1 -- 172.21.15.43:0/345037354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2edc06a190 con 0x7f2edc1080d0 2024-04-25T10:30:40.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.444+0000 7f2ed97fa640 1 -- 172.21.15.43:0/345037354 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f2ed4018d10 con 0x7f2edc1080d0 2024-04-25T10:30:40.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.444+0000 7f2ed97fa640 1 --2- 172.21.15.43:0/345037354 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2ec40afaf0 0x7f2ec40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:40.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.444+0000 7f2edbfff640 1 --2- 172.21.15.43:0/345037354 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2ec40afaf0 0x7f2ec40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:40.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.444+0000 7f2ed97fa640 1 -- 172.21.15.43:0/345037354 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f2ed40def30 con 0x7f2edc1080d0 2024-04-25T10:30:40.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.445+0000 7f2edbfff640 1 --2- 172.21.15.43:0/345037354 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2ec40afaf0 0x7f2ec40b2000 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc005fd0 tx=0x7f2ecc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:40.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.449+0000 7f2ed97fa640 1 -- 172.21.15.43:0/345037354 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2edc06a190 con 0x7f2edc1080d0 2024-04-25T10:30:40.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.664+0000 7f2ee29e3640 1 -- 172.21.15.43:0/345037354 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f2edc062fe0 con 0x7f2ec40afaf0 2024-04-25T10:30:40.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.671+0000 7f2ed97fa640 1 -- 172.21.15.43:0/345037354 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2edc062fe0 con 0x7f2ec40afaf0 2024-04-25T10:30:40.672 INFO:teuthology.orchestra.run.smithi043.stdout:/volumes/qa/sv_1/71d48da6-eb39-4602-a283-dfdb3fceec5f 2024-04-25T10:30:40.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.672+0000 7f2ee29e3640 1 -- 172.21.15.43:0/345037354 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2ec40afaf0 msgr2=0x7f2ec40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:40.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.672+0000 7f2ee29e3640 1 --2- 172.21.15.43:0/345037354 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2ec40afaf0 0x7f2ec40b2000 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc005fd0 tx=0x7f2ecc00b390 comp rx=0 tx=0).stop 2024-04-25T10:30:40.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.672+0000 7f2ee29e3640 1 -- 172.21.15.43:0/345037354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2edc1080d0 msgr2=0x7f2edc1a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:40.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.672+0000 7f2ee29e3640 1 --2- 172.21.15.43:0/345037354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2edc1080d0 0x7f2edc1a1730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400cce0 tx=0x7f2ed400a620 comp rx=0 tx=0).stop 2024-04-25T10:30:40.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.673+0000 7f2ee29e3640 1 -- 172.21.15.43:0/345037354 shutdown_connections 2024-04-25T10:30:40.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.673+0000 7f2ee29e3640 1 --2- 172.21.15.43:0/345037354 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2ec40afaf0 0x7f2ec40b2000 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.673+0000 7f2ee29e3640 1 --2- 172.21.15.43:0/345037354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2edc1033d0 0x7f2edc19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.673+0000 7f2ee29e3640 1 --2- 172.21.15.43:0/345037354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2edc1080d0 0x7f2edc1a1730 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.673+0000 7f2ee29e3640 1 --2- 172.21.15.43:0/345037354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2edc102190 0x7f2edc19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.673+0000 7f2ee29e3640 1 -- 172.21.15.43:0/345037354 >> 172.21.15.43:0/345037354 conn(0x7f2edc0fd380 msgr2=0x7f2edc1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:40.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.673+0000 7f2ee29e3640 1 -- 172.21.15.43:0/345037354 shutdown_connections 2024-04-25T10:30:40.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.673+0000 7f2ee29e3640 1 -- 172.21.15.43:0/345037354 wait complete. 2024-04-25T10:30:40.686 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:40.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.796+0000 7fbe98842640 1 -- 172.21.15.43:0/1666921317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe94103b00 msgr2=0x7fbe94104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:40.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.796+0000 7fbe98842640 1 --2- 172.21.15.43:0/1666921317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe94103b00 0x7fbe94104800 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009900 tx=0x7fbe8402a410 comp rx=0 tx=0).stop 2024-04-25T10:30:40.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.796+0000 7fbe98842640 1 -- 172.21.15.43:0/1666921317 shutdown_connections 2024-04-25T10:30:40.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.796+0000 7fbe98842640 1 --2- 172.21.15.43:0/1666921317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe94109a40 0x7fbe9410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.796+0000 7fbe98842640 1 --2- 172.21.15.43:0/1666921317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe94104d40 0x7fbe94105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.796+0000 7fbe98842640 1 --2- 172.21.15.43:0/1666921317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe94103b00 0x7fbe94104800 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.796+0000 7fbe98842640 1 -- 172.21.15.43:0/1666921317 >> 172.21.15.43:0/1666921317 conn(0x7fbe940fed10 msgr2=0x7fbe941011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:40.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.796+0000 7fbe98842640 1 -- 172.21.15.43:0/1666921317 shutdown_connections 2024-04-25T10:30:40.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.796+0000 7fbe98842640 1 -- 172.21.15.43:0/1666921317 wait complete. 2024-04-25T10:30:40.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.797+0000 7fbe98842640 1 Processor -- start 2024-04-25T10:30:40.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.797+0000 7fbe98842640 1 -- start start 2024-04-25T10:30:40.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.797+0000 7fbe98842640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe94103b00 0x7fbe9419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:40.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.797+0000 7fbe98842640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe94104d40 0x7fbe9419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:40.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.797+0000 7fbe98842640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe94109a40 0x7fbe941a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:40.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.797+0000 7fbe98842640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9410f600 con 0x7fbe94109a40 2024-04-25T10:30:40.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.797+0000 7fbe98842640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9410f480 con 0x7fbe94104d40 2024-04-25T10:30:40.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.797+0000 7fbe98842640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9410f780 con 0x7fbe94103b00 2024-04-25T10:30:40.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.797+0000 7fbe92d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe94109a40 0x7fbe941a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:40.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.797+0000 7fbe92d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe94109a40 0x7fbe941a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39414/0 (socket says 172.21.15.43:39414) 2024-04-25T10:30:40.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.797+0000 7fbe92d9d640 1 -- 172.21.15.43:0/632097809 learned_addr learned my addr 172.21.15.43:0/632097809 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:40.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.797+0000 7fbe9259c640 1 --2- 172.21.15.43:0/632097809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe94103b00 0x7fbe9419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:40.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.797+0000 7fbe91d9b640 1 --2- 172.21.15.43:0/632097809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe94104d40 0x7fbe9419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:40.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.798+0000 7fbe92d9d640 1 -- 172.21.15.43:0/632097809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe94103b00 msgr2=0x7fbe9419a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:40.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.798+0000 7fbe92d9d640 1 --2- 172.21.15.43:0/632097809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe94103b00 0x7fbe9419a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.798+0000 7fbe92d9d640 1 -- 172.21.15.43:0/632097809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe94104d40 msgr2=0x7fbe9419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:40.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.798+0000 7fbe92d9d640 1 --2- 172.21.15.43:0/632097809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe94104d40 0x7fbe9419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.798+0000 7fbe92d9d640 1 -- 172.21.15.43:0/632097809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe941a1da0 con 0x7fbe94109a40 2024-04-25T10:30:40.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.798+0000 7fbe92d9d640 1 --2- 172.21.15.43:0/632097809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe94109a40 0x7fbe941a1730 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00eb10 tx=0x7fbe8c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:40.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.798+0000 7fbe837fe640 1 -- 172.21.15.43:0/632097809 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c004b20 con 0x7fbe94109a40 2024-04-25T10:30:40.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.798+0000 7fbe837fe640 1 -- 172.21.15.43:0/632097809 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe8c004cc0 con 0x7fbe94109a40 2024-04-25T10:30:40.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.798+0000 7fbe98842640 1 -- 172.21.15.43:0/632097809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe941a2090 con 0x7fbe94109a40 2024-04-25T10:30:40.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.798+0000 7fbe837fe640 1 -- 172.21.15.43:0/632097809 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c010670 con 0x7fbe94109a40 2024-04-25T10:30:40.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.798+0000 7fbe98842640 1 -- 172.21.15.43:0/632097809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe941a2590 con 0x7fbe94109a40 2024-04-25T10:30:40.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.799+0000 7fbe98842640 1 -- 172.21.15.43:0/632097809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe94108850 con 0x7fbe94109a40 2024-04-25T10:30:40.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.801+0000 7fbe837fe640 1 -- 172.21.15.43:0/632097809 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fbe8c005000 con 0x7fbe94109a40 2024-04-25T10:30:40.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.802+0000 7fbe837fe640 1 --2- 172.21.15.43:0/632097809 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fbe7c0afaf0 0x7fbe7c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:40.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.802+0000 7fbe9259c640 1 --2- 172.21.15.43:0/632097809 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fbe7c0afaf0 0x7fbe7c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:40.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.802+0000 7fbe837fe640 1 -- 172.21.15.43:0/632097809 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fbe8c0df270 con 0x7fbe94109a40 2024-04-25T10:30:40.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.802+0000 7fbe9259c640 1 --2- 172.21.15.43:0/632097809 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fbe7c0afaf0 0x7fbe7c0b2000 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbe84005ab0 tx=0x7fbe84031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:40.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.806+0000 7fbe837fe640 1 -- 172.21.15.43:0/632097809 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe8c053ff0 con 0x7fbe94109a40 2024-04-25T10:30:40.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:40.826+0000 7fbe837fe640 1 -- 172.21.15.43:0/632097809 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fbe8c0a3b10 con 0x7fbe94109a40 2024-04-25T10:30:41.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.041+0000 7fbe98842640 1 -- 172.21.15.43:0/632097809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe9402db50 con 0x7fbe94109a40 2024-04-25T10:30:41.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.042+0000 7fbe837fe640 1 -- 172.21.15.43:0/632097809 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11869 (secure 0 0 0) 0x7fbe8c026020 con 0x7fbe94109a40 2024-04-25T10:30:41.044 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:41.044 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:30:38.804706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2595248466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2595248466},{"type":"v1","addr":"172.21.15.77:6837","nonce":2595248466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:30:41.044 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-04-25T10:30:41.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.044+0000 7fbe98842640 1 -- 172.21.15.43:0/632097809 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fbe7c0afaf0 msgr2=0x7fbe7c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.044+0000 7fbe98842640 1 --2- 172.21.15.43:0/632097809 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fbe7c0afaf0 0x7fbe7c0b2000 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbe84005ab0 tx=0x7fbe84031400 comp rx=0 tx=0).stop 2024-04-25T10:30:41.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.044+0000 7fbe98842640 1 -- 172.21.15.43:0/632097809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe94109a40 msgr2=0x7fbe941a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.044+0000 7fbe98842640 1 --2- 172.21.15.43:0/632097809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe94109a40 0x7fbe941a1730 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00eb10 tx=0x7fbe8c00c2d0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.044+0000 7fbe98842640 1 -- 172.21.15.43:0/632097809 shutdown_connections 2024-04-25T10:30:41.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.044+0000 7fbe98842640 1 --2- 172.21.15.43:0/632097809 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fbe7c0afaf0 0x7fbe7c0b2000 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.044+0000 7fbe98842640 1 --2- 172.21.15.43:0/632097809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe94104d40 0x7fbe9419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.044+0000 7fbe98842640 1 --2- 172.21.15.43:0/632097809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe94109a40 0x7fbe941a1730 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.044+0000 7fbe98842640 1 --2- 172.21.15.43:0/632097809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe94103b00 0x7fbe9419a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.044+0000 7fbe98842640 1 -- 172.21.15.43:0/632097809 >> 172.21.15.43:0/632097809 conn(0x7fbe940fed10 msgr2=0x7fbe94105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.044+0000 7fbe98842640 1 -- 172.21.15.43:0/632097809 shutdown_connections 2024-04-25T10:30:41.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.044+0000 7fbe98842640 1 -- 172.21.15.43:0/632097809 wait complete. 2024-04-25T10:30:41.056 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:40 smithi043 ceph-mon[29478]: from='client.24632 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:30:41.056 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:40 smithi043 ceph-mon[29478]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:30:41.056 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:40 smithi043 ceph-mon[29478]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:30:41.058 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.167+0000 7f97d11e3640 1 -- 172.21.15.43:0/222175570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97cc101750 msgr2=0x7f97cc101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.167+0000 7f97d11e3640 1 --2- 172.21.15.43:0/222175570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97cc101750 0x7f97cc101bc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009900 tx=0x7f97c002a410 comp rx=0 tx=0).stop 2024-04-25T10:30:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.167+0000 7f97d11e3640 1 -- 172.21.15.43:0/222175570 shutdown_connections 2024-04-25T10:30:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.167+0000 7f97d11e3640 1 --2- 172.21.15.43:0/222175570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97cc102250 0x7f97cc10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.167+0000 7f97d11e3640 1 --2- 172.21.15.43:0/222175570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97cc101750 0x7f97cc101bc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.167+0000 7f97d11e3640 1 --2- 172.21.15.43:0/222175570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97cc1050a0 0x7f97cc1054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.167+0000 7f97d11e3640 1 -- 172.21.15.43:0/222175570 >> 172.21.15.43:0/222175570 conn(0x7f97cc0fd370 msgr2=0x7f97cc0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.167+0000 7f97d11e3640 1 -- 172.21.15.43:0/222175570 shutdown_connections 2024-04-25T10:30:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.167+0000 7f97d11e3640 1 -- 172.21.15.43:0/222175570 wait complete. 2024-04-25T10:30:41.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.168+0000 7f97d11e3640 1 Processor -- start 2024-04-25T10:30:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.168+0000 7f97d11e3640 1 -- start start 2024-04-25T10:30:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.169+0000 7f97d11e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97cc101750 0x7f97cc104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.169+0000 7f97d11e3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97cc102250 0x7f97cc1028f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.169+0000 7f97d11e3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97cc1050a0 0x7f97cc102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.169+0000 7f97d11e3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97cc10f6d0 con 0x7f97cc101750 2024-04-25T10:30:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.169+0000 7f97d11e3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f97cc10f550 con 0x7f97cc102250 2024-04-25T10:30:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.169+0000 7f97d11e3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f97cc10f850 con 0x7f97cc1050a0 2024-04-25T10:30:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.169+0000 7f97cad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97cc101750 0x7f97cc104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.169+0000 7f97cad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97cc101750 0x7f97cc104250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39434/0 (socket says 172.21.15.43:39434) 2024-04-25T10:30:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.169+0000 7f97cad9d640 1 -- 172.21.15.43:0/2906119318 learned_addr learned my addr 172.21.15.43:0/2906119318 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.169+0000 7f97ca59c640 1 --2- 172.21.15.43:0/2906119318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97cc102250 0x7f97cc1028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.169+0000 7f97cb59e640 1 --2- 172.21.15.43:0/2906119318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97cc1050a0 0x7f97cc102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.169+0000 7f97cad9d640 1 -- 172.21.15.43:0/2906119318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97cc1050a0 msgr2=0x7f97cc102e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.169+0000 7f97cad9d640 1 --2- 172.21.15.43:0/2906119318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97cc1050a0 0x7f97cc102e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.169+0000 7f97cad9d640 1 -- 172.21.15.43:0/2906119318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97cc102250 msgr2=0x7f97cc1028f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.169+0000 7f97cad9d640 1 --2- 172.21.15.43:0/2906119318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97cc102250 0x7f97cc1028f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.169+0000 7f97cad9d640 1 -- 172.21.15.43:0/2906119318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97cc103560 con 0x7f97cc101750 2024-04-25T10:30:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.169+0000 7f97cad9d640 1 --2- 172.21.15.43:0/2906119318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97cc101750 0x7f97cc104250 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00c970 tx=0x7f97bc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.169+0000 7f97b3fff640 1 -- 172.21.15.43:0/2906119318 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc004220 con 0x7f97cc101750 2024-04-25T10:30:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.170+0000 7f97b3fff640 1 -- 172.21.15.43:0/2906119318 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97bc0043c0 con 0x7f97cc101750 2024-04-25T10:30:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.170+0000 7f97d11e3640 1 -- 172.21.15.43:0/2906119318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97cc1a4360 con 0x7f97cc101750 2024-04-25T10:30:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.170+0000 7f97b3fff640 1 -- 172.21.15.43:0/2906119318 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc00f450 con 0x7f97cc101750 2024-04-25T10:30:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.170+0000 7f97d11e3640 1 -- 172.21.15.43:0/2906119318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97cc1a4820 con 0x7f97cc101750 2024-04-25T10:30:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.171+0000 7f97d11e3640 1 -- 172.21.15.43:0/2906119318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97cc108320 con 0x7f97cc101750 2024-04-25T10:30:41.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.172+0000 7f97b3fff640 1 -- 172.21.15.43:0/2906119318 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f97bc0079b0 con 0x7f97cc101750 2024-04-25T10:30:41.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.173+0000 7f97b3fff640 1 --2- 172.21.15.43:0/2906119318 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f97b40afb80 0x7f97b40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.173+0000 7f97ca59c640 1 --2- 172.21.15.43:0/2906119318 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f97b40afb80 0x7f97b40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.173+0000 7f97b3fff640 1 -- 172.21.15.43:0/2906119318 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f97bc0dedd0 con 0x7f97cc101750 2024-04-25T10:30:41.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.173+0000 7f97ca59c640 1 --2- 172.21.15.43:0/2906119318 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f97b40afb80 0x7f97b40b2090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f97cc103bf0 tx=0x7f97c00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.177+0000 7f97b3fff640 1 -- 172.21.15.43:0/2906119318 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f97bc014030 con 0x7f97cc101750 2024-04-25T10:30:41.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:40 smithi077 ceph-mon[32772]: from='client.24632 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:30:41.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:40 smithi077 ceph-mon[32772]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:30:41.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:40 smithi077 ceph-mon[32772]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:30:41.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:40 smithi164 ceph-mon[32713]: from='client.24632 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:30:41.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:40 smithi164 ceph-mon[32713]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:30:41.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:40 smithi164 ceph-mon[32713]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:30:41.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.368+0000 7f97d11e3640 1 -- 172.21.15.43:0/2906119318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97cc0637a0 con 0x7f97cc101750 2024-04-25T10:30:41.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.370+0000 7f97b3fff640 1 -- 172.21.15.43:0/2906119318 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22185 (secure 0 0 0) 0x7f97bc0a72b0 con 0x7f97cc101750 2024-04-25T10:30:41.372 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:41.372 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":77,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:30:34.623807+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:31.503414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:32.224334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:41.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.372+0000 7f97d11e3640 1 -- 172.21.15.43:0/2906119318 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f97b40afb80 msgr2=0x7f97b40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.372+0000 7f97d11e3640 1 --2- 172.21.15.43:0/2906119318 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f97b40afb80 0x7f97b40b2090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f97cc103bf0 tx=0x7f97c00314a0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.372+0000 7f97d11e3640 1 -- 172.21.15.43:0/2906119318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97cc101750 msgr2=0x7f97cc104250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.372+0000 7f97d11e3640 1 --2- 172.21.15.43:0/2906119318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97cc101750 0x7f97cc104250 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00c970 tx=0x7f97bc00ce30 comp rx=0 tx=0).stop 2024-04-25T10:30:41.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.372+0000 7f97d11e3640 1 -- 172.21.15.43:0/2906119318 shutdown_connections 2024-04-25T10:30:41.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.372+0000 7f97d11e3640 1 --2- 172.21.15.43:0/2906119318 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f97b40afb80 0x7f97b40b2090 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.372+0000 7f97d11e3640 1 --2- 172.21.15.43:0/2906119318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97cc102250 0x7f97cc1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.372+0000 7f97d11e3640 1 --2- 172.21.15.43:0/2906119318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97cc1050a0 0x7f97cc102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.372+0000 7f97d11e3640 1 --2- 172.21.15.43:0/2906119318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97cc101750 0x7f97cc104250 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.372+0000 7f97d11e3640 1 -- 172.21.15.43:0/2906119318 >> 172.21.15.43:0/2906119318 conn(0x7f97cc0fd370 msgr2=0x7f97cc1007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.372+0000 7f97d11e3640 1 -- 172.21.15.43:0/2906119318 shutdown_connections 2024-04-25T10:30:41.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.372+0000 7f97d11e3640 1 -- 172.21.15.43:0/2906119318 wait complete. 2024-04-25T10:30:41.387 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:30:41.387 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:30:41.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.497+0000 7f62e0d0d640 1 -- 172.21.15.43:0/3666171568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62dc103c20 msgr2=0x7f62dc104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.497+0000 7f62e0d0d640 1 --2- 172.21.15.43:0/3666171568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62dc103c20 0x7f62dc104030 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009900 tx=0x7f62c402a410 comp rx=0 tx=0).stop 2024-04-25T10:30:41.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.497+0000 7f62e0d0d640 1 -- 172.21.15.43:0/3666171568 shutdown_connections 2024-04-25T10:30:41.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.497+0000 7f62e0d0d640 1 --2- 172.21.15.43:0/3666171568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62dc108920 0x7f62dc10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.497+0000 7f62e0d0d640 1 --2- 172.21.15.43:0/3666171568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62dc103c20 0x7f62dc104030 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.497+0000 7f62e0d0d640 1 --2- 172.21.15.43:0/3666171568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62dc1029e0 0x7f62dc1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.497+0000 7f62e0d0d640 1 -- 172.21.15.43:0/3666171568 >> 172.21.15.43:0/3666171568 conn(0x7f62dc0fdbd0 msgr2=0x7f62dc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.497+0000 7f62e0d0d640 1 -- 172.21.15.43:0/3666171568 shutdown_connections 2024-04-25T10:30:41.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.497+0000 7f62e0d0d640 1 -- 172.21.15.43:0/3666171568 wait complete. 2024-04-25T10:30:41.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.498+0000 7f62e0d0d640 1 Processor -- start 2024-04-25T10:30:41.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.498+0000 7f62e0d0d640 1 -- start start 2024-04-25T10:30:41.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.498+0000 7f62e0d0d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62dc1029e0 0x7f62dc19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.498+0000 7f62e0d0d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62dc103c20 0x7f62dc19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.498+0000 7f62e0d0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62dc108920 0x7f62dc1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.498+0000 7f62e0d0d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f62dc10e560 con 0x7f62dc108920 2024-04-25T10:30:41.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.498+0000 7f62e0d0d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f62dc10e3e0 con 0x7f62dc1029e0 2024-04-25T10:30:41.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.498+0000 7f62e0d0d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f62dc10e6e0 con 0x7f62dc103c20 2024-04-25T10:30:41.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.498+0000 7f62dad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62dc108920 0x7f62dc1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.498+0000 7f62dad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62dc108920 0x7f62dc1a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39452/0 (socket says 172.21.15.43:39452) 2024-04-25T10:30:41.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.498+0000 7f62dad9d640 1 -- 172.21.15.43:0/1800641172 learned_addr learned my addr 172.21.15.43:0/1800641172 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:41.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.498+0000 7f62da59c640 1 --2- 172.21.15.43:0/1800641172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62dc1029e0 0x7f62dc19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.499+0000 7f62d3fff640 1 --2- 172.21.15.43:0/1800641172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62dc103c20 0x7f62dc19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.499+0000 7f62dad9d640 1 -- 172.21.15.43:0/1800641172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62dc103c20 msgr2=0x7f62dc19a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.499+0000 7f62dad9d640 1 --2- 172.21.15.43:0/1800641172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62dc103c20 0x7f62dc19a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.499+0000 7f62dad9d640 1 -- 172.21.15.43:0/1800641172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62dc1029e0 msgr2=0x7f62dc19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.499+0000 7f62dad9d640 1 --2- 172.21.15.43:0/1800641172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62dc1029e0 0x7f62dc19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.499+0000 7f62dad9d640 1 -- 172.21.15.43:0/1800641172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62dc1a1e50 con 0x7f62dc108920 2024-04-25T10:30:41.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.499+0000 7f62dad9d640 1 --2- 172.21.15.43:0/1800641172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62dc108920 0x7f62dc1a17e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f62d400cad0 tx=0x7f62d400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.499+0000 7f62d37fe640 1 -- 172.21.15.43:0/1800641172 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d400ad60 con 0x7f62dc108920 2024-04-25T10:30:41.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.499+0000 7f62e0d0d640 1 -- 172.21.15.43:0/1800641172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62dc1a2140 con 0x7f62dc108920 2024-04-25T10:30:41.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.499+0000 7f62d37fe640 1 -- 172.21.15.43:0/1800641172 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62d4005020 con 0x7f62dc108920 2024-04-25T10:30:41.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.499+0000 7f62d37fe640 1 -- 172.21.15.43:0/1800641172 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d400f6b0 con 0x7f62dc108920 2024-04-25T10:30:41.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.499+0000 7f62e0d0d640 1 -- 172.21.15.43:0/1800641172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62dc1a26c0 con 0x7f62dc108920 2024-04-25T10:30:41.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.500+0000 7f62e0d0d640 1 -- 172.21.15.43:0/1800641172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62dc1a2980 con 0x7f62dc108920 2024-04-25T10:30:41.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.501+0000 7f62d37fe640 1 -- 172.21.15.43:0/1800641172 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f62d40051d0 con 0x7f62dc108920 2024-04-25T10:30:41.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.502+0000 7f62d37fe640 1 --2- 172.21.15.43:0/1800641172 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f62c80afb80 0x7f62c80b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.502+0000 7f62da59c640 1 --2- 172.21.15.43:0/1800641172 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f62c80afb80 0x7f62c80b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.502+0000 7f62d37fe640 1 -- 172.21.15.43:0/1800641172 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f62d40dec80 con 0x7f62dc108920 2024-04-25T10:30:41.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.503+0000 7f62da59c640 1 --2- 172.21.15.43:0/1800641172 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f62c80afb80 0x7f62c80b2090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f62dc1035e0 tx=0x7f62cc009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.507+0000 7f62d37fe640 1 -- 172.21.15.43:0/1800641172 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62d4014030 con 0x7f62dc108920 2024-04-25T10:30:41.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.693+0000 7f62e0d0d640 1 -- 172.21.15.43:0/1800641172 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f62dc10a610 con 0x7f62c80afb80 2024-04-25T10:30:41.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.701+0000 7f62d37fe640 1 -- 172.21.15.43:0/1800641172 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258721 (secure 0 0 0) 0x7f62dc10a610 con 0x7f62c80afb80 2024-04-25T10:30:41.706 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:41.708 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T10:30:41.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.703+0000 7f62e0d0d640 1 -- 172.21.15.43:0/1800641172 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f62c80afb80 msgr2=0x7f62c80b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.703+0000 7f62e0d0d640 1 --2- 172.21.15.43:0/1800641172 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f62c80afb80 0x7f62c80b2090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f62dc1035e0 tx=0x7f62cc009110 comp rx=0 tx=0).stop 2024-04-25T10:30:41.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.703+0000 7f62e0d0d640 1 -- 172.21.15.43:0/1800641172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62dc108920 msgr2=0x7f62dc1a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.703+0000 7f62e0d0d640 1 --2- 172.21.15.43:0/1800641172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62dc108920 0x7f62dc1a17e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f62d400cad0 tx=0x7f62d400ced0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.703+0000 7f62e0d0d640 1 -- 172.21.15.43:0/1800641172 shutdown_connections 2024-04-25T10:30:41.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.703+0000 7f62e0d0d640 1 --2- 172.21.15.43:0/1800641172 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f62c80afb80 0x7f62c80b2090 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.703+0000 7f62e0d0d640 1 --2- 172.21.15.43:0/1800641172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62dc108920 0x7f62dc1a17e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.703+0000 7f62e0d0d640 1 --2- 172.21.15.43:0/1800641172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62dc103c20 0x7f62dc19a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.703+0000 7f62e0d0d640 1 --2- 172.21.15.43:0/1800641172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62dc1029e0 0x7f62dc19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.703+0000 7f62e0d0d640 1 -- 172.21.15.43:0/1800641172 >> 172.21.15.43:0/1800641172 conn(0x7f62dc0fdbd0 msgr2=0x7f62dc109f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.704+0000 7f62e0d0d640 1 -- 172.21.15.43:0/1800641172 shutdown_connections 2024-04-25T10:30:41.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.704+0000 7f62e0d0d640 1 -- 172.21.15.43:0/1800641172 wait complete. 2024-04-25T10:30:41.717 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-04-25T10:30:40.160391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":602100,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":103,"num_read_kb":88,"num_write":246,"num_write_kb":2195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":414844,"kb_used_data":11604,"kb_used_omap":18,"kb_used_meta":403117,"kb_avail":1124526980,"statfs":{"total":1151940427776,"available":1151515627520,"internally_reserved":0,"allocated":11882496,"data_stored":6541722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":412792121},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11732,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":22,"num_read_kb":19,"num_write":123,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.277487"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545347+0000","last_change":"2024-04-25T10:30:33.645032+0000","last_active":"2024-04-25T10:30:35.545347+0000","last_peered":"2024-04-25T10:30:35.545347+0000","last_clean":"2024-04-25T10:30:35.545347+0000","last_became_active":"2024-04-25T10:30:33.644687+0000","last_became_peered":"2024-04-25T10:30:33.644687+0000","last_unstale":"2024-04-25T10:30:35.545347+0000","last_undegraded":"2024-04-25T10:30:35.545347+0000","last_fullsized":"2024-04-25T10:30:35.545347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:39.893228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.996533+0000","last_change":"2024-04-25T10:30:33.650466+0000","last_active":"2024-04-25T10:30:34.996533+0000","last_peered":"2024-04-25T10:30:34.996533+0000","last_clean":"2024-04-25T10:30:34.996533+0000","last_became_active":"2024-04-25T10:30:33.649889+0000","last_became_peered":"2024-04-25T10:30:33.649889+0000","last_unstale":"2024-04-25T10:30:34.996533+0000","last_undegraded":"2024-04-25T10:30:34.996533+0000","last_fullsized":"2024-04-25T10:30:34.996533+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:53.832494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.996492+0000","last_change":"2024-04-25T10:30:33.648798+0000","last_active":"2024-04-25T10:30:34.996492+0000","last_peered":"2024-04-25T10:30:34.996492+0000","last_clean":"2024-04-25T10:30:34.996492+0000","last_became_active":"2024-04-25T10:30:33.648423+0000","last_became_peered":"2024-04-25T10:30:33.648423+0000","last_unstale":"2024-04-25T10:30:34.996492+0000","last_undegraded":"2024-04-25T10:30:34.996492+0000","last_fullsized":"2024-04-25T10:30:34.996492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:19.119189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.638405+0000","last_change":"2024-04-25T10:30:33.642061+0000","last_active":"2024-04-25T10:30:34.638405+0000","last_peered":"2024-04-25T10:30:34.638405+0000","last_clean":"2024-04-25T10:30:34.638405+0000","last_became_active":"2024-04-25T10:30:33.641299+0000","last_became_peered":"2024-04-25T10:30:33.641299+0000","last_unstale":"2024-04-25T10:30:34.638405+0000","last_undegraded":"2024-04-25T10:30:34.638405+0000","last_fullsized":"2024-04-25T10:30:34.638405+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:17.903107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.633470+0000","last_change":"2024-04-25T10:30:33.646786+0000","last_active":"2024-04-25T10:30:34.633470+0000","last_peered":"2024-04-25T10:30:34.633470+0000","last_clean":"2024-04-25T10:30:34.633470+0000","last_became_active":"2024-04-25T10:30:33.646456+0000","last_became_peered":"2024-04-25T10:30:33.646456+0000","last_unstale":"2024-04-25T10:30:34.633470+0000","last_undegraded":"2024-04-25T10:30:34.633470+0000","last_fullsized":"2024-04-25T10:30:34.633470+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:55.006723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545019+0000","last_change":"2024-04-25T10:30:33.639952+0000","last_active":"2024-04-25T10:30:35.545019+0000","last_peered":"2024-04-25T10:30:35.545019+0000","last_clean":"2024-04-25T10:30:35.545019+0000","last_became_active":"2024-04-25T10:30:33.639558+0000","last_became_peered":"2024-04-25T10:30:33.639558+0000","last_unstale":"2024-04-25T10:30:35.545019+0000","last_undegraded":"2024-04-25T10:30:35.545019+0000","last_fullsized":"2024-04-25T10:30:35.545019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:17.162902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.505976+0000","last_change":"2024-04-25T10:30:33.651530+0000","last_active":"2024-04-25T10:30:35.505976+0000","last_peered":"2024-04-25T10:30:35.505976+0000","last_clean":"2024-04-25T10:30:35.505976+0000","last_became_active":"2024-04-25T10:30:33.651151+0000","last_became_peered":"2024-04-25T10:30:33.651151+0000","last_unstale":"2024-04-25T10:30:35.505976+0000","last_undegraded":"2024-04-25T10:30:35.505976+0000","last_fullsized":"2024-04-25T10:30:35.505976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:31.835367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544378+0000","last_change":"2024-04-25T10:30:33.645146+0000","last_active":"2024-04-25T10:30:35.544378+0000","last_peered":"2024-04-25T10:30:35.544378+0000","last_clean":"2024-04-25T10:30:35.544378+0000","last_became_active":"2024-04-25T10:30:33.644310+0000","last_became_peered":"2024-04-25T10:30:33.644310+0000","last_unstale":"2024-04-25T10:30:35.544378+0000","last_undegraded":"2024-04-25T10:30:35.544378+0000","last_fullsized":"2024-04-25T10:30:35.544378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:45.034934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545469+0000","last_change":"2024-04-25T10:30:33.649385+0000","last_active":"2024-04-25T10:30:35.545469+0000","last_peered":"2024-04-25T10:30:35.545469+0000","last_clean":"2024-04-25T10:30:35.545469+0000","last_became_active":"2024-04-25T10:30:33.649002+0000","last_became_peered":"2024-04-25T10:30:33.649002+0000","last_unstale":"2024-04-25T10:30:35.545469+0000","last_undegraded":"2024-04-25T10:30:35.545469+0000","last_fullsized":"2024-04-25T10:30:35.545469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:42.259290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545253+0000","last_change":"2024-04-25T10:30:33.641918+0000","last_active":"2024-04-25T10:30:35.545253+0000","last_peered":"2024-04-25T10:30:35.545253+0000","last_clean":"2024-04-25T10:30:35.545253+0000","last_became_active":"2024-04-25T10:30:33.641553+0000","last_became_peered":"2024-04-25T10:30:33.641553+0000","last_unstale":"2024-04-25T10:30:35.545253+0000","last_undegraded":"2024-04-25T10:30:35.545253+0000","last_fullsized":"2024-04-25T10:30:35.545253+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:04.755694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.506398+0000","last_change":"2024-04-25T10:30:33.638054+0000","last_active":"2024-04-25T10:30:35.506398+0000","last_peered":"2024-04-25T10:30:35.506398+0000","last_clean":"2024-04-25T10:30:35.506398+0000","last_became_active":"2024-04-25T10:30:33.637720+0000","last_became_peered":"2024-04-25T10:30:33.637720+0000","last_unstale":"2024-04-25T10:30:35.506398+0000","last_undegraded":"2024-04-25T10:30:35.506398+0000","last_fullsized":"2024-04-25T10:30:35.506398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:41.557430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.638495+0000","last_change":"2024-04-25T10:30:33.642705+0000","last_active":"2024-04-25T10:30:34.638495+0000","last_peered":"2024-04-25T10:30:34.638495+0000","last_clean":"2024-04-25T10:30:34.638495+0000","last_became_active":"2024-04-25T10:30:33.641936+0000","last_became_peered":"2024-04-25T10:30:33.641936+0000","last_unstale":"2024-04-25T10:30:34.638495+0000","last_undegraded":"2024-04-25T10:30:34.638495+0000","last_fullsized":"2024-04-25T10:30:34.638495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:36.450389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.749723+0000","last_change":"2024-04-25T10:30:33.648987+0000","last_active":"2024-04-25T10:30:34.749723+0000","last_peered":"2024-04-25T10:30:34.749723+0000","last_clean":"2024-04-25T10:30:34.749723+0000","last_became_active":"2024-04-25T10:30:33.648621+0000","last_became_peered":"2024-04-25T10:30:33.648621+0000","last_unstale":"2024-04-25T10:30:34.749723+0000","last_undegraded":"2024-04-25T10:30:34.749723+0000","last_fullsized":"2024-04-25T10:30:34.749723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:19.999002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.996297+0000","last_change":"2024-04-25T10:30:33.647360+0000","last_active":"2024-04-25T10:30:34.996297+0000","last_peered":"2024-04-25T10:30:34.996297+0000","last_clean":"2024-04-25T10:30:34.996297+0000","last_became_active":"2024-04-25T10:30:33.646719+0000","last_became_peered":"2024-04-25T10:30:33.646719+0000","last_unstale":"2024-04-25T10:30:34.996297+0000","last_undegraded":"2024-04-25T10:30:34.996297+0000","last_fullsized":"2024-04-25T10:30:34.996297+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:19.016333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.996198+0000","last_change":"2024-04-25T10:30:33.646503+0000","last_active":"2024-04-25T10:30:34.996198+0000","last_peered":"2024-04-25T10:30:34.996198+0000","last_clean":"2024-04-25T10:30:34.996198+0000","last_became_active":"2024-04-25T10:30:33.646123+0000","last_became_peered":"2024-04-25T10:30:33.646123+0000","last_unstale":"2024-04-25T10:30:34.996198+0000","last_undegraded":"2024-04-25T10:30:34.996198+0000","last_fullsized":"2024-04-25T10:30:34.996198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:57.712216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544515+0000","last_change":"2024-04-25T10:30:33.640463+0000","last_active":"2024-04-25T10:30:35.544515+0000","last_peered":"2024-04-25T10:30:35.544515+0000","last_clean":"2024-04-25T10:30:35.544515+0000","last_became_active":"2024-04-25T10:30:33.639931+0000","last_became_peered":"2024-04-25T10:30:33.639931+0000","last_unstale":"2024-04-25T10:30:35.544515+0000","last_undegraded":"2024-04-25T10:30:35.544515+0000","last_fullsized":"2024-04-25T10:30:35.544515+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:18.863405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545484+0000","last_change":"2024-04-25T10:30:33.645695+0000","last_active":"2024-04-25T10:30:35.545484+0000","last_peered":"2024-04-25T10:30:35.545484+0000","last_clean":"2024-04-25T10:30:35.545484+0000","last_became_active":"2024-04-25T10:30:33.645387+0000","last_became_peered":"2024-04-25T10:30:33.645387+0000","last_unstale":"2024-04-25T10:30:35.545484+0000","last_undegraded":"2024-04-25T10:30:35.545484+0000","last_fullsized":"2024-04-25T10:30:35.545484+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:20.044456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.632775+0000","last_change":"2024-04-25T10:30:33.639087+0000","last_active":"2024-04-25T10:30:34.632775+0000","last_peered":"2024-04-25T10:30:34.632775+0000","last_clean":"2024-04-25T10:30:34.632775+0000","last_became_active":"2024-04-25T10:30:33.638726+0000","last_became_peered":"2024-04-25T10:30:33.638726+0000","last_unstale":"2024-04-25T10:30:34.632775+0000","last_undegraded":"2024-04-25T10:30:34.632775+0000","last_fullsized":"2024-04-25T10:30:34.632775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:36.887289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.852731+0000","last_change":"2024-04-25T10:30:33.638646+0000","last_active":"2024-04-25T10:30:34.852731+0000","last_peered":"2024-04-25T10:30:34.852731+0000","last_clean":"2024-04-25T10:30:34.852731+0000","last_became_active":"2024-04-25T10:30:33.638261+0000","last_became_peered":"2024-04-25T10:30:33.638261+0000","last_unstale":"2024-04-25T10:30:34.852731+0000","last_undegraded":"2024-04-25T10:30:34.852731+0000","last_fullsized":"2024-04-25T10:30:34.852731+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:08.212310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.814050+0000","last_change":"2024-04-25T10:30:32.899992+0000","last_active":"2024-04-25T10:30:37.814050+0000","last_peered":"2024-04-25T10:30:37.814050+0000","last_clean":"2024-04-25T10:30:37.814050+0000","last_became_active":"2024-04-25T10:30:32.899643+0000","last_became_peered":"2024-04-25T10:30:32.899643+0000","last_unstale":"2024-04-25T10:30:37.814050+0000","last_undegraded":"2024-04-25T10:30:37.814050+0000","last_fullsized":"2024-04-25T10:30:37.814050+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:34.245121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544806+0000","last_change":"2024-04-25T10:30:33.644884+0000","last_active":"2024-04-25T10:30:35.544806+0000","last_peered":"2024-04-25T10:30:35.544806+0000","last_clean":"2024-04-25T10:30:35.544806+0000","last_became_active":"2024-04-25T10:30:33.644510+0000","last_became_peered":"2024-04-25T10:30:33.644510+0000","last_unstale":"2024-04-25T10:30:35.544806+0000","last_undegraded":"2024-04-25T10:30:35.544806+0000","last_fullsized":"2024-04-25T10:30:35.544806+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:09.282865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"76'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544712+0000","last_change":"2024-04-25T10:30:32.909175+0000","last_active":"2024-04-25T10:30:35.544712+0000","last_peered":"2024-04-25T10:30:35.544712+0000","last_clean":"2024-04-25T10:30:35.544712+0000","last_became_active":"2024-04-25T10:30:32.908813+0000","last_became_peered":"2024-04-25T10:30:32.908813+0000","last_unstale":"2024-04-25T10:30:35.544712+0000","last_undegraded":"2024-04-25T10:30:35.544712+0000","last_fullsized":"2024-04-25T10:30:35.544712+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:04.256452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.996498+0000","last_change":"2024-04-25T10:30:33.642095+0000","last_active":"2024-04-25T10:30:34.996498+0000","last_peered":"2024-04-25T10:30:34.996498+0000","last_clean":"2024-04-25T10:30:34.996498+0000","last_became_active":"2024-04-25T10:30:33.641713+0000","last_became_peered":"2024-04-25T10:30:33.641713+0000","last_unstale":"2024-04-25T10:30:34.996498+0000","last_undegraded":"2024-04-25T10:30:34.996498+0000","last_fullsized":"2024-04-25T10:30:34.996498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:42.425197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.813937+0000","last_change":"2024-04-25T10:30:32.904376+0000","last_active":"2024-04-25T10:30:37.813937+0000","last_peered":"2024-04-25T10:30:37.813937+0000","last_clean":"2024-04-25T10:30:37.813937+0000","last_became_active":"2024-04-25T10:30:32.903994+0000","last_became_peered":"2024-04-25T10:30:32.903994+0000","last_unstale":"2024-04-25T10:30:37.813937+0000","last_undegraded":"2024-04-25T10:30:37.813937+0000","last_fullsized":"2024-04-25T10:30:37.813937+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:52.727800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544730+0000","last_change":"2024-04-25T10:30:33.637171+0000","last_active":"2024-04-25T10:30:35.544730+0000","last_peered":"2024-04-25T10:30:35.544730+0000","last_clean":"2024-04-25T10:30:35.544730+0000","last_became_active":"2024-04-25T10:30:33.636822+0000","last_became_peered":"2024-04-25T10:30:33.636822+0000","last_unstale":"2024-04-25T10:30:35.544730+0000","last_undegraded":"2024-04-25T10:30:35.544730+0000","last_fullsized":"2024-04-25T10:30:35.544730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:54.220784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.633679+0000","last_change":"2024-04-25T10:30:32.900856+0000","last_active":"2024-04-25T10:30:34.633679+0000","last_peered":"2024-04-25T10:30:34.633679+0000","last_clean":"2024-04-25T10:30:34.633679+0000","last_became_active":"2024-04-25T10:30:32.900519+0000","last_became_peered":"2024-04-25T10:30:32.900519+0000","last_unstale":"2024-04-25T10:30:34.633679+0000","last_undegraded":"2024-04-25T10:30:34.633679+0000","last_fullsized":"2024-04-25T10:30:34.633679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:49.738005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545639+0000","last_change":"2024-04-25T10:30:33.651736+0000","last_active":"2024-04-25T10:30:35.545639+0000","last_peered":"2024-04-25T10:30:35.545639+0000","last_clean":"2024-04-25T10:30:35.545639+0000","last_became_active":"2024-04-25T10:30:33.651395+0000","last_became_peered":"2024-04-25T10:30:33.651395+0000","last_unstale":"2024-04-25T10:30:35.545639+0000","last_undegraded":"2024-04-25T10:30:35.545639+0000","last_fullsized":"2024-04-25T10:30:35.545639+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:58.230092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.802980+0000","last_change":"2024-04-25T10:30:32.901348+0000","last_active":"2024-04-25T10:30:37.802980+0000","last_peered":"2024-04-25T10:30:37.802980+0000","last_clean":"2024-04-25T10:30:37.802980+0000","last_became_active":"2024-04-25T10:30:32.901025+0000","last_became_peered":"2024-04-25T10:30:32.901025+0000","last_unstale":"2024-04-25T10:30:37.802980+0000","last_undegraded":"2024-04-25T10:30:37.802980+0000","last_fullsized":"2024-04-25T10:30:37.802980+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:00.521934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.633827+0000","last_change":"2024-04-25T10:30:33.649206+0000","last_active":"2024-04-25T10:30:34.633827+0000","last_peered":"2024-04-25T10:30:34.633827+0000","last_clean":"2024-04-25T10:30:34.633827+0000","last_became_active":"2024-04-25T10:30:33.648883+0000","last_became_peered":"2024-04-25T10:30:33.648883+0000","last_unstale":"2024-04-25T10:30:34.633827+0000","last_undegraded":"2024-04-25T10:30:34.633827+0000","last_fullsized":"2024-04-25T10:30:34.633827+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:43.105562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.812920+0000","last_change":"2024-04-25T10:30:32.901201+0000","last_active":"2024-04-25T10:30:37.812920+0000","last_peered":"2024-04-25T10:30:37.812920+0000","last_clean":"2024-04-25T10:30:37.812920+0000","last_became_active":"2024-04-25T10:30:32.900876+0000","last_became_peered":"2024-04-25T10:30:32.900876+0000","last_unstale":"2024-04-25T10:30:37.812920+0000","last_undegraded":"2024-04-25T10:30:37.812920+0000","last_fullsized":"2024-04-25T10:30:37.812920+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:58.077045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545552+0000","last_change":"2024-04-25T10:30:33.651935+0000","last_active":"2024-04-25T10:30:35.545552+0000","last_peered":"2024-04-25T10:30:35.545552+0000","last_clean":"2024-04-25T10:30:35.545552+0000","last_became_active":"2024-04-25T10:30:33.651568+0000","last_became_peered":"2024-04-25T10:30:33.651568+0000","last_unstale":"2024-04-25T10:30:35.545552+0000","last_undegraded":"2024-04-25T10:30:35.545552+0000","last_fullsized":"2024-04-25T10:30:35.545552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:58.849535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.813717+0000","last_change":"2024-04-25T10:30:32.896500+0000","last_active":"2024-04-25T10:30:37.813717+0000","last_peered":"2024-04-25T10:30:37.813717+0000","last_clean":"2024-04-25T10:30:37.813717+0000","last_became_active":"2024-04-25T10:30:32.896071+0000","last_became_peered":"2024-04-25T10:30:32.896071+0000","last_unstale":"2024-04-25T10:30:37.813717+0000","last_undegraded":"2024-04-25T10:30:37.813717+0000","last_fullsized":"2024-04-25T10:30:37.813717+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:10.385443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544569+0000","last_change":"2024-04-25T10:30:33.631730+0000","last_active":"2024-04-25T10:30:35.544569+0000","last_peered":"2024-04-25T10:30:35.544569+0000","last_clean":"2024-04-25T10:30:35.544569+0000","last_became_active":"2024-04-25T10:30:33.631374+0000","last_became_peered":"2024-04-25T10:30:33.631374+0000","last_unstale":"2024-04-25T10:30:35.544569+0000","last_undegraded":"2024-04-25T10:30:35.544569+0000","last_fullsized":"2024-04-25T10:30:35.544569+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:14.957866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544462+0000","last_change":"2024-04-25T10:30:32.900762+0000","last_active":"2024-04-25T10:30:35.544462+0000","last_peered":"2024-04-25T10:30:35.544462+0000","last_clean":"2024-04-25T10:30:35.544462+0000","last_became_active":"2024-04-25T10:30:32.900459+0000","last_became_peered":"2024-04-25T10:30:32.900459+0000","last_unstale":"2024-04-25T10:30:35.544462+0000","last_undegraded":"2024-04-25T10:30:35.544462+0000","last_fullsized":"2024-04-25T10:30:35.544462+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:22.286261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.632589+0000","last_change":"2024-04-25T10:30:33.637727+0000","last_active":"2024-04-25T10:30:34.632589+0000","last_peered":"2024-04-25T10:30:34.632589+0000","last_clean":"2024-04-25T10:30:34.632589+0000","last_became_active":"2024-04-25T10:30:33.637359+0000","last_became_peered":"2024-04-25T10:30:33.637359+0000","last_unstale":"2024-04-25T10:30:34.632589+0000","last_undegraded":"2024-04-25T10:30:34.632589+0000","last_fullsized":"2024-04-25T10:30:34.632589+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:21.877562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'3","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.802361+0000","last_change":"2024-04-25T10:30:32.894210+0000","last_active":"2024-04-25T10:30:37.802361+0000","last_peered":"2024-04-25T10:30:37.802361+0000","last_clean":"2024-04-25T10:30:37.802361+0000","last_became_active":"2024-04-25T10:30:32.893241+0000","last_became_peered":"2024-04-25T10:30:32.893241+0000","last_unstale":"2024-04-25T10:30:37.802361+0000","last_undegraded":"2024-04-25T10:30:37.802361+0000","last_fullsized":"2024-04-25T10:30:37.802361+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:06.340392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545029+0000","last_change":"2024-04-25T10:30:33.632349+0000","last_active":"2024-04-25T10:30:35.545029+0000","last_peered":"2024-04-25T10:30:35.545029+0000","last_clean":"2024-04-25T10:30:35.545029+0000","last_became_active":"2024-04-25T10:30:33.631968+0000","last_became_peered":"2024-04-25T10:30:33.631968+0000","last_unstale":"2024-04-25T10:30:35.545029+0000","last_undegraded":"2024-04-25T10:30:35.545029+0000","last_fullsized":"2024-04-25T10:30:35.545029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:29.090932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.189939+0000","last_change":"2024-04-25T10:30:32.900105+0000","last_active":"2024-04-25T10:30:37.189939+0000","last_peered":"2024-04-25T10:30:37.189939+0000","last_clean":"2024-04-25T10:30:37.189939+0000","last_became_active":"2024-04-25T10:30:32.899740+0000","last_became_peered":"2024-04-25T10:30:32.899740+0000","last_unstale":"2024-04-25T10:30:37.189939+0000","last_undegraded":"2024-04-25T10:30:37.189939+0000","last_fullsized":"2024-04-25T10:30:37.189939+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:49.390799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.996650+0000","last_change":"2024-04-25T10:30:33.649293+0000","last_active":"2024-04-25T10:30:34.996650+0000","last_peered":"2024-04-25T10:30:34.996650+0000","last_clean":"2024-04-25T10:30:34.996650+0000","last_became_active":"2024-04-25T10:30:33.648899+0000","last_became_peered":"2024-04-25T10:30:33.648899+0000","last_unstale":"2024-04-25T10:30:34.996650+0000","last_undegraded":"2024-04-25T10:30:34.996650+0000","last_fullsized":"2024-04-25T10:30:34.996650+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:13.485155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.189424+0000","last_change":"2024-04-25T10:30:32.893453+0000","last_active":"2024-04-25T10:30:37.189424+0000","last_peered":"2024-04-25T10:30:37.189424+0000","last_clean":"2024-04-25T10:30:37.189424+0000","last_became_active":"2024-04-25T10:30:32.893106+0000","last_became_peered":"2024-04-25T10:30:32.893106+0000","last_unstale":"2024-04-25T10:30:37.189424+0000","last_undegraded":"2024-04-25T10:30:37.189424+0000","last_fullsized":"2024-04-25T10:30:37.189424+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:57.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545058+0000","last_change":"2024-04-25T10:30:33.645693+0000","last_active":"2024-04-25T10:30:35.545058+0000","last_peered":"2024-04-25T10:30:35.545058+0000","last_clean":"2024-04-25T10:30:35.545058+0000","last_became_active":"2024-04-25T10:30:33.645342+0000","last_became_peered":"2024-04-25T10:30:33.645342+0000","last_unstale":"2024-04-25T10:30:35.545058+0000","last_undegraded":"2024-04-25T10:30:35.545058+0000","last_fullsized":"2024-04-25T10:30:35.545058+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:15.729860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'11","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:39.179891+0000","last_change":"2024-04-25T10:30:32.897973+0000","last_active":"2024-04-25T10:30:39.179891+0000","last_peered":"2024-04-25T10:30:39.179891+0000","last_clean":"2024-04-25T10:30:39.179891+0000","last_became_active":"2024-04-25T10:30:32.897409+0000","last_became_peered":"2024-04-25T10:30:32.897409+0000","last_unstale":"2024-04-25T10:30:39.179891+0000","last_undegraded":"2024-04-25T10:30:39.179891+0000","last_fullsized":"2024-04-25T10:30:39.179891+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:47.024554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8212,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":12,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.633484+0000","last_change":"2024-04-25T10:30:33.641768+0000","last_active":"2024-04-25T10:30:34.633484+0000","last_peered":"2024-04-25T10:30:34.633484+0000","last_clean":"2024-04-25T10:30:34.633484+0000","last_became_active":"2024-04-25T10:30:33.641415+0000","last_became_peered":"2024-04-25T10:30:33.641415+0000","last_unstale":"2024-04-25T10:30:34.633484+0000","last_undegraded":"2024-04-25T10:30:34.633484+0000","last_fullsized":"2024-04-25T10:30:34.633484+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:38.569474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.813688+0000","last_change":"2024-04-25T10:30:32.898703+0000","last_active":"2024-04-25T10:30:37.813688+0000","last_peered":"2024-04-25T10:30:37.813688+0000","last_clean":"2024-04-25T10:30:37.813688+0000","last_became_active":"2024-04-25T10:30:32.898340+0000","last_became_peered":"2024-04-25T10:30:32.898340+0000","last_unstale":"2024-04-25T10:30:37.813688+0000","last_undegraded":"2024-04-25T10:30:37.813688+0000","last_fullsized":"2024-04-25T10:30:37.813688+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:27.024144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.749575+0000","last_change":"2024-04-25T10:30:33.640393+0000","last_active":"2024-04-25T10:30:34.749575+0000","last_peered":"2024-04-25T10:30:34.749575+0000","last_clean":"2024-04-25T10:30:34.749575+0000","last_became_active":"2024-04-25T10:30:33.639822+0000","last_became_peered":"2024-04-25T10:30:33.639822+0000","last_unstale":"2024-04-25T10:30:34.749575+0000","last_undegraded":"2024-04-25T10:30:34.749575+0000","last_fullsized":"2024-04-25T10:30:34.749575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:08.618406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.189214+0000","last_change":"2024-04-25T10:30:32.900880+0000","last_active":"2024-04-25T10:30:37.189214+0000","last_peered":"2024-04-25T10:30:37.189214+0000","last_clean":"2024-04-25T10:30:37.189214+0000","last_became_active":"2024-04-25T10:30:32.899987+0000","last_became_peered":"2024-04-25T10:30:32.899987+0000","last_unstale":"2024-04-25T10:30:37.189214+0000","last_undegraded":"2024-04-25T10:30:37.189214+0000","last_fullsized":"2024-04-25T10:30:37.189214+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:35.687773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.506389+0000","last_change":"2024-04-25T10:30:33.641933+0000","last_active":"2024-04-25T10:30:35.506389+0000","last_peered":"2024-04-25T10:30:35.506389+0000","last_clean":"2024-04-25T10:30:35.506389+0000","last_became_active":"2024-04-25T10:30:33.641549+0000","last_became_peered":"2024-04-25T10:30:33.641549+0000","last_unstale":"2024-04-25T10:30:35.506389+0000","last_undegraded":"2024-04-25T10:30:35.506389+0000","last_fullsized":"2024-04-25T10:30:35.506389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:36.696523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'3","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.813062+0000","last_change":"2024-04-25T10:30:32.901178+0000","last_active":"2024-04-25T10:30:37.813062+0000","last_peered":"2024-04-25T10:30:37.813062+0000","last_clean":"2024-04-25T10:30:37.813062+0000","last_became_active":"2024-04-25T10:30:32.900817+0000","last_became_peered":"2024-04-25T10:30:32.900817+0000","last_unstale":"2024-04-25T10:30:37.813062+0000","last_undegraded":"2024-04-25T10:30:37.813062+0000","last_fullsized":"2024-04-25T10:30:37.813062+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:53.999253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.505920+0000","last_change":"2024-04-25T10:30:33.653245+0000","last_active":"2024-04-25T10:30:35.505920+0000","last_peered":"2024-04-25T10:30:35.505920+0000","last_clean":"2024-04-25T10:30:35.505920+0000","last_became_active":"2024-04-25T10:30:33.652885+0000","last_became_peered":"2024-04-25T10:30:33.652885+0000","last_unstale":"2024-04-25T10:30:35.505920+0000","last_undegraded":"2024-04-25T10:30:35.505920+0000","last_fullsized":"2024-04-25T10:30:35.505920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:59.888048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.814486+0000","last_change":"2024-04-25T10:30:32.902862+0000","last_active":"2024-04-25T10:30:37.814486+0000","last_peered":"2024-04-25T10:30:37.814486+0000","last_clean":"2024-04-25T10:30:37.814486+0000","last_became_active":"2024-04-25T10:30:32.902537+0000","last_became_peered":"2024-04-25T10:30:32.902537+0000","last_unstale":"2024-04-25T10:30:37.814486+0000","last_undegraded":"2024-04-25T10:30:37.814486+0000","last_fullsized":"2024-04-25T10:30:37.814486+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:55.974128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545098+0000","last_change":"2024-04-25T10:30:33.651409+0000","last_active":"2024-04-25T10:30:35.545098+0000","last_peered":"2024-04-25T10:30:35.545098+0000","last_clean":"2024-04-25T10:30:35.545098+0000","last_became_active":"2024-04-25T10:30:33.650673+0000","last_became_peered":"2024-04-25T10:30:33.650673+0000","last_unstale":"2024-04-25T10:30:35.545098+0000","last_undegraded":"2024-04-25T10:30:35.545098+0000","last_fullsized":"2024-04-25T10:30:35.545098+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:48.989198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'1","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.633868+0000","last_change":"2024-04-25T10:30:32.900524+0000","last_active":"2024-04-25T10:30:34.633868+0000","last_peered":"2024-04-25T10:30:34.633868+0000","last_clean":"2024-04-25T10:30:34.633868+0000","last_became_active":"2024-04-25T10:30:32.899719+0000","last_became_peered":"2024-04-25T10:30:32.899719+0000","last_unstale":"2024-04-25T10:30:34.633868+0000","last_undegraded":"2024-04-25T10:30:34.633868+0000","last_fullsized":"2024-04-25T10:30:34.633868+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:45.749965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545021+0000","last_change":"2024-04-25T10:30:33.643929+0000","last_active":"2024-04-25T10:30:35.545021+0000","last_peered":"2024-04-25T10:30:35.545021+0000","last_clean":"2024-04-25T10:30:35.545021+0000","last_became_active":"2024-04-25T10:30:33.643593+0000","last_became_peered":"2024-04-25T10:30:33.643593+0000","last_unstale":"2024-04-25T10:30:35.545021+0000","last_undegraded":"2024-04-25T10:30:35.545021+0000","last_fullsized":"2024-04-25T10:30:35.545021+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:39.591022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.632535+0000","last_change":"2024-04-25T10:30:32.898937+0000","last_active":"2024-04-25T10:30:34.632535+0000","last_peered":"2024-04-25T10:30:34.632535+0000","last_clean":"2024-04-25T10:30:34.632535+0000","last_became_active":"2024-04-25T10:30:32.898570+0000","last_became_peered":"2024-04-25T10:30:32.898570+0000","last_unstale":"2024-04-25T10:30:34.632535+0000","last_undegraded":"2024-04-25T10:30:34.632535+0000","last_fullsized":"2024-04-25T10:30:34.632535+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:40.123035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545555+0000","last_change":"2024-04-25T10:30:33.632857+0000","last_active":"2024-04-25T10:30:35.545555+0000","last_peered":"2024-04-25T10:30:35.545555+0000","last_clean":"2024-04-25T10:30:35.545555+0000","last_became_active":"2024-04-25T10:30:33.632043+0000","last_became_peered":"2024-04-25T10:30:33.632043+0000","last_unstale":"2024-04-25T10:30:35.545555+0000","last_undegraded":"2024-04-25T10:30:35.545555+0000","last_fullsized":"2024-04-25T10:30:35.545555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:12.683035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.506121+0000","last_change":"2024-04-25T10:30:32.901349+0000","last_active":"2024-04-25T10:30:35.506121+0000","last_peered":"2024-04-25T10:30:35.506121+0000","last_clean":"2024-04-25T10:30:35.506121+0000","last_became_active":"2024-04-25T10:30:32.900881+0000","last_became_peered":"2024-04-25T10:30:32.900881+0000","last_unstale":"2024-04-25T10:30:35.506121+0000","last_undegraded":"2024-04-25T10:30:35.506121+0000","last_fullsized":"2024-04-25T10:30:35.506121+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:34.963564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545185+0000","last_change":"2024-04-25T10:30:33.644376+0000","last_active":"2024-04-25T10:30:35.545185+0000","last_peered":"2024-04-25T10:30:35.545185+0000","last_clean":"2024-04-25T10:30:35.545185+0000","last_became_active":"2024-04-25T10:30:33.644055+0000","last_became_peered":"2024-04-25T10:30:33.644055+0000","last_unstale":"2024-04-25T10:30:35.545185+0000","last_undegraded":"2024-04-25T10:30:35.545185+0000","last_fullsized":"2024-04-25T10:30:35.545185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:48.134775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544608+0000","last_change":"2024-04-25T10:30:33.646312+0000","last_active":"2024-04-25T10:30:35.544608+0000","last_peered":"2024-04-25T10:30:35.544608+0000","last_clean":"2024-04-25T10:30:35.544608+0000","last_became_active":"2024-04-25T10:30:33.642338+0000","last_became_peered":"2024-04-25T10:30:33.642338+0000","last_unstale":"2024-04-25T10:30:35.544608+0000","last_undegraded":"2024-04-25T10:30:35.544608+0000","last_fullsized":"2024-04-25T10:30:35.544608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:42.588360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"77'4","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.814265+0000","last_change":"2024-04-25T10:30:32.909680+0000","last_active":"2024-04-25T10:30:37.814265+0000","last_peered":"2024-04-25T10:30:37.814265+0000","last_clean":"2024-04-25T10:30:37.814265+0000","last_became_active":"2024-04-25T10:30:32.909169+0000","last_became_peered":"2024-04-25T10:30:32.909169+0000","last_unstale":"2024-04-25T10:30:37.814265+0000","last_undegraded":"2024-04-25T10:30:37.814265+0000","last_fullsized":"2024-04-25T10:30:37.814265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:54.876475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544800+0000","last_change":"2024-04-25T10:30:33.649524+0000","last_active":"2024-04-25T10:30:35.544800+0000","last_peered":"2024-04-25T10:30:35.544800+0000","last_clean":"2024-04-25T10:30:35.544800+0000","last_became_active":"2024-04-25T10:30:33.648958+0000","last_became_peered":"2024-04-25T10:30:33.648958+0000","last_unstale":"2024-04-25T10:30:35.544800+0000","last_undegraded":"2024-04-25T10:30:35.544800+0000","last_fullsized":"2024-04-25T10:30:35.544800+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:13.407742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.800832+0000","last_change":"2024-04-25T10:30:32.895270+0000","last_active":"2024-04-25T10:30:37.800832+0000","last_peered":"2024-04-25T10:30:37.800832+0000","last_clean":"2024-04-25T10:30:37.800832+0000","last_became_active":"2024-04-25T10:30:32.894899+0000","last_became_peered":"2024-04-25T10:30:32.894899+0000","last_unstale":"2024-04-25T10:30:37.800832+0000","last_undegraded":"2024-04-25T10:30:37.800832+0000","last_fullsized":"2024-04-25T10:30:37.800832+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:21.233217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":91,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.505963+0000","last_change":"2024-04-25T10:29:21.105073+0000","last_active":"2024-04-25T10:30:35.505963+0000","last_peered":"2024-04-25T10:30:35.505963+0000","last_clean":"2024-04-25T10:30:35.505963+0000","last_became_active":"2024-04-25T10:29:21.098771+0000","last_became_peered":"2024-04-25T10:29:21.098771+0000","last_unstale":"2024-04-25T10:30:35.505963+0000","last_undegraded":"2024-04-25T10:30:35.505963+0000","last_fullsized":"2024-04-25T10:30:35.505963+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:59.015440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:59.015440+0000","last_clean_scrub_stamp":"2024-04-25T10:25:59.015440+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:41.728977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.749397+0000","last_change":"2024-04-25T10:30:33.639159+0000","last_active":"2024-04-25T10:30:34.749397+0000","last_peered":"2024-04-25T10:30:34.749397+0000","last_clean":"2024-04-25T10:30:34.749397+0000","last_became_active":"2024-04-25T10:30:33.638761+0000","last_became_peered":"2024-04-25T10:30:33.638761+0000","last_unstale":"2024-04-25T10:30:34.749397+0000","last_undegraded":"2024-04-25T10:30:34.749397+0000","last_fullsized":"2024-04-25T10:30:34.749397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:43.262103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.813530+0000","last_change":"2024-04-25T10:30:32.898587+0000","last_active":"2024-04-25T10:30:37.813530+0000","last_peered":"2024-04-25T10:30:37.813530+0000","last_clean":"2024-04-25T10:30:37.813530+0000","last_became_active":"2024-04-25T10:30:32.898113+0000","last_became_peered":"2024-04-25T10:30:32.898113+0000","last_unstale":"2024-04-25T10:30:37.813530+0000","last_undegraded":"2024-04-25T10:30:37.813530+0000","last_fullsized":"2024-04-25T10:30:37.813530+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:24.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.187770+0000","last_change":"2024-04-25T10:30:32.907188+0000","last_active":"2024-04-25T10:30:37.187770+0000","last_peered":"2024-04-25T10:30:37.187770+0000","last_clean":"2024-04-25T10:30:37.187770+0000","last_became_active":"2024-04-25T10:30:32.906837+0000","last_became_peered":"2024-04-25T10:30:32.906837+0000","last_unstale":"2024-04-25T10:30:37.187770+0000","last_undegraded":"2024-04-25T10:30:37.187770+0000","last_fullsized":"2024-04-25T10:30:37.187770+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:11.835053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.749783+0000","last_change":"2024-04-25T10:30:33.636159+0000","last_active":"2024-04-25T10:30:34.749783+0000","last_peered":"2024-04-25T10:30:34.749783+0000","last_clean":"2024-04-25T10:30:34.749783+0000","last_became_active":"2024-04-25T10:30:33.635098+0000","last_became_peered":"2024-04-25T10:30:33.635098+0000","last_unstale":"2024-04-25T10:30:34.749783+0000","last_undegraded":"2024-04-25T10:30:34.749783+0000","last_fullsized":"2024-04-25T10:30:34.749783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:13.092368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'3","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.812860+0000","last_change":"2024-04-25T10:30:32.905469+0000","last_active":"2024-04-25T10:30:37.812860+0000","last_peered":"2024-04-25T10:30:37.812860+0000","last_clean":"2024-04-25T10:30:37.812860+0000","last_became_active":"2024-04-25T10:30:32.905123+0000","last_became_peered":"2024-04-25T10:30:32.905123+0000","last_unstale":"2024-04-25T10:30:37.812860+0000","last_undegraded":"2024-04-25T10:30:37.812860+0000","last_fullsized":"2024-04-25T10:30:37.812860+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:12.884809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544691+0000","last_change":"2024-04-25T10:30:33.640690+0000","last_active":"2024-04-25T10:30:35.544691+0000","last_peered":"2024-04-25T10:30:35.544691+0000","last_clean":"2024-04-25T10:30:35.544691+0000","last_became_active":"2024-04-25T10:30:33.640310+0000","last_became_peered":"2024-04-25T10:30:33.640310+0000","last_unstale":"2024-04-25T10:30:35.544691+0000","last_undegraded":"2024-04-25T10:30:35.544691+0000","last_fullsized":"2024-04-25T10:30:35.544691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:03.671495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.187977+0000","last_change":"2024-04-25T10:30:32.893743+0000","last_active":"2024-04-25T10:30:37.187977+0000","last_peered":"2024-04-25T10:30:37.187977+0000","last_clean":"2024-04-25T10:30:37.187977+0000","last_became_active":"2024-04-25T10:30:32.893360+0000","last_became_peered":"2024-04-25T10:30:32.893360+0000","last_unstale":"2024-04-25T10:30:37.187977+0000","last_undegraded":"2024-04-25T10:30:37.187977+0000","last_fullsized":"2024-04-25T10:30:37.187977+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:39.830376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.996500+0000","last_change":"2024-04-25T10:30:33.645127+0000","last_active":"2024-04-25T10:30:34.996500+0000","last_peered":"2024-04-25T10:30:34.996500+0000","last_clean":"2024-04-25T10:30:34.996500+0000","last_became_active":"2024-04-25T10:30:33.644259+0000","last_became_peered":"2024-04-25T10:30:33.644259+0000","last_unstale":"2024-04-25T10:30:34.996500+0000","last_undegraded":"2024-04-25T10:30:34.996500+0000","last_fullsized":"2024-04-25T10:30:34.996500+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:56.849861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.189254+0000","last_change":"2024-04-25T10:30:32.896885+0000","last_active":"2024-04-25T10:30:37.189254+0000","last_peered":"2024-04-25T10:30:37.189254+0000","last_clean":"2024-04-25T10:30:37.189254+0000","last_became_active":"2024-04-25T10:30:32.896447+0000","last_became_peered":"2024-04-25T10:30:32.896447+0000","last_unstale":"2024-04-25T10:30:37.189254+0000","last_undegraded":"2024-04-25T10:30:37.189254+0000","last_fullsized":"2024-04-25T10:30:37.189254+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:13.133549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.633322+0000","last_change":"2024-04-25T10:30:33.645016+0000","last_active":"2024-04-25T10:30:34.633322+0000","last_peered":"2024-04-25T10:30:34.633322+0000","last_clean":"2024-04-25T10:30:34.633322+0000","last_became_active":"2024-04-25T10:30:33.644661+0000","last_became_peered":"2024-04-25T10:30:33.644661+0000","last_unstale":"2024-04-25T10:30:34.633322+0000","last_undegraded":"2024-04-25T10:30:34.633322+0000","last_fullsized":"2024-04-25T10:30:34.633322+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:20.935124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544370+0000","last_change":"2024-04-25T10:30:32.907827+0000","last_active":"2024-04-25T10:30:35.544370+0000","last_peered":"2024-04-25T10:30:35.544370+0000","last_clean":"2024-04-25T10:30:35.544370+0000","last_became_active":"2024-04-25T10:30:32.907402+0000","last_became_peered":"2024-04-25T10:30:32.907402+0000","last_unstale":"2024-04-25T10:30:35.544370+0000","last_undegraded":"2024-04-25T10:30:35.544370+0000","last_fullsized":"2024-04-25T10:30:35.544370+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:38.501617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544613+0000","last_change":"2024-04-25T10:30:33.636935+0000","last_active":"2024-04-25T10:30:35.544613+0000","last_peered":"2024-04-25T10:30:35.544613+0000","last_clean":"2024-04-25T10:30:35.544613+0000","last_became_active":"2024-04-25T10:30:33.636566+0000","last_became_peered":"2024-04-25T10:30:33.636566+0000","last_unstale":"2024-04-25T10:30:35.544613+0000","last_undegraded":"2024-04-25T10:30:35.544613+0000","last_fullsized":"2024-04-25T10:30:35.544613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:24.556275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.852370+0000","last_change":"2024-04-25T10:30:32.902457+0000","last_active":"2024-04-25T10:30:34.852370+0000","last_peered":"2024-04-25T10:30:34.852370+0000","last_clean":"2024-04-25T10:30:34.852370+0000","last_became_active":"2024-04-25T10:30:32.901173+0000","last_became_peered":"2024-04-25T10:30:32.901173+0000","last_unstale":"2024-04-25T10:30:34.852370+0000","last_undegraded":"2024-04-25T10:30:34.852370+0000","last_fullsized":"2024-04-25T10:30:34.852370+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:59.305176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544622+0000","last_change":"2024-04-25T10:30:33.642986+0000","last_active":"2024-04-25T10:30:35.544622+0000","last_peered":"2024-04-25T10:30:35.544622+0000","last_clean":"2024-04-25T10:30:35.544622+0000","last_became_active":"2024-04-25T10:30:33.642545+0000","last_became_peered":"2024-04-25T10:30:33.642545+0000","last_unstale":"2024-04-25T10:30:35.544622+0000","last_undegraded":"2024-04-25T10:30:35.544622+0000","last_fullsized":"2024-04-25T10:30:35.544622+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:49.369648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.188806+0000","last_change":"2024-04-25T10:30:32.902239+0000","last_active":"2024-04-25T10:30:37.188806+0000","last_peered":"2024-04-25T10:30:37.188806+0000","last_clean":"2024-04-25T10:30:37.188806+0000","last_became_active":"2024-04-25T10:30:32.901653+0000","last_became_peered":"2024-04-25T10:30:32.901653+0000","last_unstale":"2024-04-25T10:30:37.188806+0000","last_undegraded":"2024-04-25T10:30:37.188806+0000","last_fullsized":"2024-04-25T10:30:37.188806+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:34.732428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.506300+0000","last_change":"2024-04-25T10:30:33.639763+0000","last_active":"2024-04-25T10:30:35.506300+0000","last_peered":"2024-04-25T10:30:35.506300+0000","last_clean":"2024-04-25T10:30:35.506300+0000","last_became_active":"2024-04-25T10:30:33.638004+0000","last_became_peered":"2024-04-25T10:30:33.638004+0000","last_unstale":"2024-04-25T10:30:35.506300+0000","last_undegraded":"2024-04-25T10:30:35.506300+0000","last_fullsized":"2024-04-25T10:30:35.506300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:09.722333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.814590+0000","last_change":"2024-04-25T10:30:32.894823+0000","last_active":"2024-04-25T10:30:37.814590+0000","last_peered":"2024-04-25T10:30:37.814590+0000","last_clean":"2024-04-25T10:30:37.814590+0000","last_became_active":"2024-04-25T10:30:32.893819+0000","last_became_peered":"2024-04-25T10:30:32.893819+0000","last_unstale":"2024-04-25T10:30:37.814590+0000","last_undegraded":"2024-04-25T10:30:37.814590+0000","last_fullsized":"2024-04-25T10:30:37.814590+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:59.809410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.506475+0000","last_change":"2024-04-25T10:30:33.637046+0000","last_active":"2024-04-25T10:30:35.506475+0000","last_peered":"2024-04-25T10:30:35.506475+0000","last_clean":"2024-04-25T10:30:35.506475+0000","last_became_active":"2024-04-25T10:30:33.634903+0000","last_became_peered":"2024-04-25T10:30:33.634903+0000","last_unstale":"2024-04-25T10:30:35.506475+0000","last_undegraded":"2024-04-25T10:30:35.506475+0000","last_fullsized":"2024-04-25T10:30:35.506475+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:52.252812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'1","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545506+0000","last_change":"2024-04-25T10:30:32.898979+0000","last_active":"2024-04-25T10:30:35.545506+0000","last_peered":"2024-04-25T10:30:35.545506+0000","last_clean":"2024-04-25T10:30:35.545506+0000","last_became_active":"2024-04-25T10:30:32.898611+0000","last_became_peered":"2024-04-25T10:30:32.898611+0000","last_unstale":"2024-04-25T10:30:35.545506+0000","last_undegraded":"2024-04-25T10:30:35.545506+0000","last_fullsized":"2024-04-25T10:30:35.545506+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.549769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.506274+0000","last_change":"2024-04-25T10:30:33.651596+0000","last_active":"2024-04-25T10:30:35.506274+0000","last_peered":"2024-04-25T10:30:35.506274+0000","last_clean":"2024-04-25T10:30:35.506274+0000","last_became_active":"2024-04-25T10:30:33.651140+0000","last_became_peered":"2024-04-25T10:30:33.651140+0000","last_unstale":"2024-04-25T10:30:35.506274+0000","last_undegraded":"2024-04-25T10:30:35.506274+0000","last_fullsized":"2024-04-25T10:30:35.506274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:16.762135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.749776+0000","last_change":"2024-04-25T10:30:32.909982+0000","last_active":"2024-04-25T10:30:34.749776+0000","last_peered":"2024-04-25T10:30:34.749776+0000","last_clean":"2024-04-25T10:30:34.749776+0000","last_became_active":"2024-04-25T10:30:32.909506+0000","last_became_peered":"2024-04-25T10:30:32.909506+0000","last_unstale":"2024-04-25T10:30:34.749776+0000","last_undegraded":"2024-04-25T10:30:34.749776+0000","last_fullsized":"2024-04-25T10:30:34.749776+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:47.396963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.506605+0000","last_change":"2024-04-25T10:30:33.642913+0000","last_active":"2024-04-25T10:30:35.506605+0000","last_peered":"2024-04-25T10:30:35.506605+0000","last_clean":"2024-04-25T10:30:35.506605+0000","last_became_active":"2024-04-25T10:30:33.642614+0000","last_became_peered":"2024-04-25T10:30:33.642614+0000","last_unstale":"2024-04-25T10:30:35.506605+0000","last_undegraded":"2024-04-25T10:30:35.506605+0000","last_fullsized":"2024-04-25T10:30:35.506605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:56.651230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544675+0000","last_change":"2024-04-25T10:30:33.644525+0000","last_active":"2024-04-25T10:30:35.544675+0000","last_peered":"2024-04-25T10:30:35.544675+0000","last_clean":"2024-04-25T10:30:35.544675+0000","last_became_active":"2024-04-25T10:30:33.644188+0000","last_became_peered":"2024-04-25T10:30:33.644188+0000","last_unstale":"2024-04-25T10:30:35.544675+0000","last_undegraded":"2024-04-25T10:30:35.544675+0000","last_fullsized":"2024-04-25T10:30:35.544675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:28.625695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.852405+0000","last_change":"2024-04-25T10:30:33.648912+0000","last_active":"2024-04-25T10:30:34.852405+0000","last_peered":"2024-04-25T10:30:34.852405+0000","last_clean":"2024-04-25T10:30:34.852405+0000","last_became_active":"2024-04-25T10:30:33.648541+0000","last_became_peered":"2024-04-25T10:30:33.648541+0000","last_unstale":"2024-04-25T10:30:34.852405+0000","last_undegraded":"2024-04-25T10:30:34.852405+0000","last_fullsized":"2024-04-25T10:30:34.852405+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:06.303096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.852304+0000","last_change":"2024-04-25T10:30:33.644721+0000","last_active":"2024-04-25T10:30:34.852304+0000","last_peered":"2024-04-25T10:30:34.852304+0000","last_clean":"2024-04-25T10:30:34.852304+0000","last_became_active":"2024-04-25T10:30:33.644392+0000","last_became_peered":"2024-04-25T10:30:33.644392+0000","last_unstale":"2024-04-25T10:30:34.852304+0000","last_undegraded":"2024-04-25T10:30:34.852304+0000","last_fullsized":"2024-04-25T10:30:34.852304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:50.571318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.996470+0000","last_change":"2024-04-25T10:30:33.647970+0000","last_active":"2024-04-25T10:30:34.996470+0000","last_peered":"2024-04-25T10:30:34.996470+0000","last_clean":"2024-04-25T10:30:34.996470+0000","last_became_active":"2024-04-25T10:30:33.647547+0000","last_became_peered":"2024-04-25T10:30:33.647547+0000","last_unstale":"2024-04-25T10:30:34.996470+0000","last_undegraded":"2024-04-25T10:30:34.996470+0000","last_fullsized":"2024-04-25T10:30:34.996470+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:25.609085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.749746+0000","last_change":"2024-04-25T10:30:33.641618+0000","last_active":"2024-04-25T10:30:34.749746+0000","last_peered":"2024-04-25T10:30:34.749746+0000","last_clean":"2024-04-25T10:30:34.749746+0000","last_became_active":"2024-04-25T10:30:33.640830+0000","last_became_peered":"2024-04-25T10:30:33.640830+0000","last_unstale":"2024-04-25T10:30:34.749746+0000","last_undegraded":"2024-04-25T10:30:34.749746+0000","last_fullsized":"2024-04-25T10:30:34.749746+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:36.870127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545059+0000","last_change":"2024-04-25T10:30:33.644010+0000","last_active":"2024-04-25T10:30:35.545059+0000","last_peered":"2024-04-25T10:30:35.545059+0000","last_clean":"2024-04-25T10:30:35.545059+0000","last_became_active":"2024-04-25T10:30:33.643639+0000","last_became_peered":"2024-04-25T10:30:33.643639+0000","last_unstale":"2024-04-25T10:30:35.545059+0000","last_undegraded":"2024-04-25T10:30:35.545059+0000","last_fullsized":"2024-04-25T10:30:35.545059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:32.820552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.852833+0000","last_change":"2024-04-25T10:30:33.639727+0000","last_active":"2024-04-25T10:30:34.852833+0000","last_peered":"2024-04-25T10:30:34.852833+0000","last_clean":"2024-04-25T10:30:34.852833+0000","last_became_active":"2024-04-25T10:30:33.639185+0000","last_became_peered":"2024-04-25T10:30:33.639185+0000","last_unstale":"2024-04-25T10:30:34.852833+0000","last_undegraded":"2024-04-25T10:30:34.852833+0000","last_fullsized":"2024-04-25T10:30:34.852833+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:17.860864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544873+0000","last_change":"2024-04-25T10:30:33.644650+0000","last_active":"2024-04-25T10:30:35.544873+0000","last_peered":"2024-04-25T10:30:35.544873+0000","last_clean":"2024-04-25T10:30:35.544873+0000","last_became_active":"2024-04-25T10:30:33.644264+0000","last_became_peered":"2024-04-25T10:30:33.644264+0000","last_unstale":"2024-04-25T10:30:35.544873+0000","last_undegraded":"2024-04-25T10:30:35.544873+0000","last_fullsized":"2024-04-25T10:30:35.544873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:41.122318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544554+0000","last_change":"2024-04-25T10:30:33.651875+0000","last_active":"2024-04-25T10:30:35.544554+0000","last_peered":"2024-04-25T10:30:35.544554+0000","last_clean":"2024-04-25T10:30:35.544554+0000","last_became_active":"2024-04-25T10:30:33.651486+0000","last_became_peered":"2024-04-25T10:30:33.651486+0000","last_unstale":"2024-04-25T10:30:35.544554+0000","last_undegraded":"2024-04-25T10:30:35.544554+0000","last_fullsized":"2024-04-25T10:30:35.544554+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:32.598209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544464+0000","last_change":"2024-04-25T10:30:33.638728+0000","last_active":"2024-04-25T10:30:35.544464+0000","last_peered":"2024-04-25T10:30:35.544464+0000","last_clean":"2024-04-25T10:30:35.544464+0000","last_became_active":"2024-04-25T10:30:33.638200+0000","last_became_peered":"2024-04-25T10:30:33.638200+0000","last_unstale":"2024-04-25T10:30:35.544464+0000","last_undegraded":"2024-04-25T10:30:35.544464+0000","last_fullsized":"2024-04-25T10:30:35.544464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:19.178542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544846+0000","last_change":"2024-04-25T10:30:33.639915+0000","last_active":"2024-04-25T10:30:35.544846+0000","last_peered":"2024-04-25T10:30:35.544846+0000","last_clean":"2024-04-25T10:30:35.544846+0000","last_became_active":"2024-04-25T10:30:33.639137+0000","last_became_peered":"2024-04-25T10:30:33.639137+0000","last_unstale":"2024-04-25T10:30:35.544846+0000","last_undegraded":"2024-04-25T10:30:35.544846+0000","last_fullsized":"2024-04-25T10:30:35.544846+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:13.143405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.749634+0000","last_change":"2024-04-25T10:30:33.643547+0000","last_active":"2024-04-25T10:30:34.749634+0000","last_peered":"2024-04-25T10:30:34.749634+0000","last_clean":"2024-04-25T10:30:34.749634+0000","last_became_active":"2024-04-25T10:30:33.643158+0000","last_became_peered":"2024-04-25T10:30:33.643158+0000","last_unstale":"2024-04-25T10:30:34.749634+0000","last_undegraded":"2024-04-25T10:30:34.749634+0000","last_fullsized":"2024-04-25T10:30:34.749634+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:07.517132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544990+0000","last_change":"2024-04-25T10:30:33.645746+0000","last_active":"2024-04-25T10:30:35.544990+0000","last_peered":"2024-04-25T10:30:35.544990+0000","last_clean":"2024-04-25T10:30:35.544990+0000","last_became_active":"2024-04-25T10:30:33.644646+0000","last_became_peered":"2024-04-25T10:30:33.644646+0000","last_unstale":"2024-04-25T10:30:35.544990+0000","last_undegraded":"2024-04-25T10:30:35.544990+0000","last_fullsized":"2024-04-25T10:30:35.544990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:21.591264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":11732,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":22,"num_read_kb":19,"num_write":123,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":42151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645327,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":827392,"data_stored":396357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841555,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36764,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708388,"statfs":{"total":95995035648,"available":95957389312,"internally_reserved":0,"allocated":1462272,"data_stored":989333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037784,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36156,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708996,"statfs":{"total":95995035648,"available":95958011904,"internally_reserved":0,"allocated":839680,"data_stored":396559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234013,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":835584,"data_stored":396401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430243,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32580,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712572,"statfs":{"total":95995035648,"available":95961673728,"internally_reserved":0,"allocated":1433600,"data_stored":987935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626470,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36144,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709008,"statfs":{"total":95995035648,"available":95958024192,"internally_reserved":0,"allocated":827392,"data_stored":395865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":839680,"data_stored":403987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40328,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704824,"statfs":{"total":95995035648,"available":95953739776,"internally_reserved":0,"allocated":851968,"data_stored":397253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":860160,"data_stored":398737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36152,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709000,"statfs":{"total":95995035648,"available":95958016000,"internally_reserved":0,"allocated":831488,"data_stored":395879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986225,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":827392,"data_stored":395797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822699,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36748,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708404,"statfs":{"total":95995035648,"available":95957405696,"internally_reserved":0,"allocated":1445888,"data_stored":987619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":20895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":10181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:30:41.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:30:41.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.867+0000 7f638caca640 1 -- 172.21.15.43:0/15554995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6388103b10 msgr2=0x7f6388104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.867+0000 7f638caca640 1 --2- 172.21.15.43:0/15554995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6388103b10 0x7f6388104810 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6378009900 tx=0x7f637802a410 comp rx=0 tx=0).stop 2024-04-25T10:30:41.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.867+0000 7f638caca640 1 -- 172.21.15.43:0/15554995 shutdown_connections 2024-04-25T10:30:41.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.867+0000 7f638caca640 1 --2- 172.21.15.43:0/15554995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6388109a50 0x7f638810be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.867+0000 7f638caca640 1 --2- 172.21.15.43:0/15554995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6388104d50 0x7f6388105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.867+0000 7f638caca640 1 --2- 172.21.15.43:0/15554995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6388103b10 0x7f6388104810 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.867+0000 7f638caca640 1 -- 172.21.15.43:0/15554995 >> 172.21.15.43:0/15554995 conn(0x7f63880fed40 msgr2=0x7f63881011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.867+0000 7f638caca640 1 -- 172.21.15.43:0/15554995 shutdown_connections 2024-04-25T10:30:41.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.867+0000 7f638caca640 1 -- 172.21.15.43:0/15554995 wait complete. 2024-04-25T10:30:41.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.868+0000 7f638caca640 1 Processor -- start 2024-04-25T10:30:41.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.868+0000 7f638caca640 1 -- start start 2024-04-25T10:30:41.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.868+0000 7f638caca640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6388103b10 0x7f638819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.868+0000 7f638caca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6388104d50 0x7f638819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.868+0000 7f638caca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6388109a50 0x7f63881a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.868+0000 7f638caca640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f638810f660 con 0x7f6388109a50 2024-04-25T10:30:41.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.868+0000 7f638caca640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f638810f4e0 con 0x7f6388104d50 2024-04-25T10:30:41.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.868+0000 7f638caca640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f638810f7e0 con 0x7f6388103b10 2024-04-25T10:30:41.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.869+0000 7f6386d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6388109a50 0x7f63881a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.869+0000 7f6386d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6388109a50 0x7f63881a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39474/0 (socket says 172.21.15.43:39474) 2024-04-25T10:30:41.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.869+0000 7f6386d9d640 1 -- 172.21.15.43:0/447254773 learned_addr learned my addr 172.21.15.43:0/447254773 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:41.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.869+0000 7f637dd9b640 1 --2- 172.21.15.43:0/447254773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6388104d50 0x7f638819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.869+0000 7f638659c640 1 --2- 172.21.15.43:0/447254773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6388103b10 0x7f638819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.869+0000 7f6386d9d640 1 -- 172.21.15.43:0/447254773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6388103b10 msgr2=0x7f638819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.869+0000 7f6386d9d640 1 --2- 172.21.15.43:0/447254773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6388103b10 0x7f638819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.869+0000 7f6386d9d640 1 -- 172.21.15.43:0/447254773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6388104d50 msgr2=0x7f638819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.869+0000 7f6386d9d640 1 --2- 172.21.15.43:0/447254773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6388104d50 0x7f638819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.869+0000 7f6386d9d640 1 -- 172.21.15.43:0/447254773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63881a1d80 con 0x7f6388109a50 2024-04-25T10:30:41.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.869+0000 7f6386d9d640 1 --2- 172.21.15.43:0/447254773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6388109a50 0x7f63881a1710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6380009fd0 tx=0x7f638000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.869+0000 7f637ffff640 1 -- 172.21.15.43:0/447254773 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380017070 con 0x7f6388109a50 2024-04-25T10:30:41.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.869+0000 7f637ffff640 1 -- 172.21.15.43:0/447254773 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6380003e20 con 0x7f6388109a50 2024-04-25T10:30:41.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.869+0000 7f638caca640 1 -- 172.21.15.43:0/447254773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63881a2070 con 0x7f6388109a50 2024-04-25T10:30:41.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.869+0000 7f637ffff640 1 -- 172.21.15.43:0/447254773 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380010460 con 0x7f6388109a50 2024-04-25T10:30:41.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.869+0000 7f638caca640 1 -- 172.21.15.43:0/447254773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63881a2570 con 0x7f6388109a50 2024-04-25T10:30:41.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.870+0000 7f638caca640 1 -- 172.21.15.43:0/447254773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6388108860 con 0x7f6388109a50 2024-04-25T10:30:41.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.872+0000 7f637ffff640 1 -- 172.21.15.43:0/447254773 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f6380003970 con 0x7f6388109a50 2024-04-25T10:30:41.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.872+0000 7f637ffff640 1 --2- 172.21.15.43:0/447254773 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f63740afb80 0x7f63740b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.872+0000 7f638659c640 1 --2- 172.21.15.43:0/447254773 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f63740afb80 0x7f63740b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.872+0000 7f637ffff640 1 -- 172.21.15.43:0/447254773 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f63800de930 con 0x7f6388109a50 2024-04-25T10:30:41.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.873+0000 7f638659c640 1 --2- 172.21.15.43:0/447254773 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f63740afb80 0x7f63740b2090 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f637802aa00 tx=0x7f63780314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:41.877+0000 7f637ffff640 1 -- 172.21.15.43:0/447254773 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f63800a6f10 con 0x7f6388109a50 2024-04-25T10:30:42.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:42.061+0000 7f638caca640 1 -- 172.21.15.43:0/447254773 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6388062fe0 con 0x7f63740afb80 2024-04-25T10:30:42.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:42.068+0000 7f637ffff640 1 -- 172.21.15.43:0/447254773 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258721 (secure 0 0 0) 0x7f6388062fe0 con 0x7f63740afb80 2024-04-25T10:30:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:42.074 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T10:30:42.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:42.072+0000 7f638caca640 1 -- 172.21.15.43:0/447254773 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f63740afb80 msgr2=0x7f63740b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:42.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:42.072+0000 7f638caca640 1 --2- 172.21.15.43:0/447254773 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f63740afb80 0x7f63740b2090 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f637802aa00 tx=0x7f63780314a0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:42.072+0000 7f638caca640 1 -- 172.21.15.43:0/447254773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6388109a50 msgr2=0x7f63881a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:42.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:42.072+0000 7f638caca640 1 --2- 172.21.15.43:0/447254773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6388109a50 0x7f63881a1710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6380009fd0 tx=0x7f638000b260 comp rx=0 tx=0).stop 2024-04-25T10:30:42.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:42.072+0000 7f638caca640 1 -- 172.21.15.43:0/447254773 shutdown_connections 2024-04-25T10:30:42.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:42.072+0000 7f638caca640 1 --2- 172.21.15.43:0/447254773 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f63740afb80 0x7f63740b2090 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:42.072+0000 7f638caca640 1 --2- 172.21.15.43:0/447254773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6388104d50 0x7f638819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:42.072+0000 7f638caca640 1 --2- 172.21.15.43:0/447254773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6388109a50 0x7f63881a1710 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:42.072+0000 7f638caca640 1 --2- 172.21.15.43:0/447254773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6388103b10 0x7f638819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:42.072+0000 7f638caca640 1 -- 172.21.15.43:0/447254773 >> 172.21.15.43:0/447254773 conn(0x7f63880fed40 msgr2=0x7f6388105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:42.072+0000 7f638caca640 1 -- 172.21.15.43:0/447254773 shutdown_connections 2024-04-25T10:30:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:42.072+0000 7f638caca640 1 -- 172.21.15.43:0/447254773 wait complete. 2024-04-25T10:30:42.084 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:41 smithi043 ceph-mon[29478]: pgmap v281: 97 pgs: 97 active+clean; 588 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6.2 KiB/s wr, 19 op/s 2024-04-25T10:30:42.084 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:41 smithi043 ceph-mon[29478]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:30:42.084 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:41 smithi043 ceph-mon[29478]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:30:42.084 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:41 smithi043 ceph-mon[29478]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T10:30:42.085 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:41 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/632097809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:42.085 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:41 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2906119318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:30:42.085 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-04-25T10:30:40.160391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":602100,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":103,"num_read_kb":88,"num_write":246,"num_write_kb":2195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":414844,"kb_used_data":11604,"kb_used_omap":18,"kb_used_meta":403117,"kb_avail":1124526980,"statfs":{"total":1151940427776,"available":1151515627520,"internally_reserved":0,"allocated":11882496,"data_stored":6541722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":412792121},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11732,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":22,"num_read_kb":19,"num_write":123,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.277487"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545347+0000","last_change":"2024-04-25T10:30:33.645032+0000","last_active":"2024-04-25T10:30:35.545347+0000","last_peered":"2024-04-25T10:30:35.545347+0000","last_clean":"2024-04-25T10:30:35.545347+0000","last_became_active":"2024-04-25T10:30:33.644687+0000","last_became_peered":"2024-04-25T10:30:33.644687+0000","last_unstale":"2024-04-25T10:30:35.545347+0000","last_undegraded":"2024-04-25T10:30:35.545347+0000","last_fullsized":"2024-04-25T10:30:35.545347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:39.893228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.996533+0000","last_change":"2024-04-25T10:30:33.650466+0000","last_active":"2024-04-25T10:30:34.996533+0000","last_peered":"2024-04-25T10:30:34.996533+0000","last_clean":"2024-04-25T10:30:34.996533+0000","last_became_active":"2024-04-25T10:30:33.649889+0000","last_became_peered":"2024-04-25T10:30:33.649889+0000","last_unstale":"2024-04-25T10:30:34.996533+0000","last_undegraded":"2024-04-25T10:30:34.996533+0000","last_fullsized":"2024-04-25T10:30:34.996533+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:53.832494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.996492+0000","last_change":"2024-04-25T10:30:33.648798+0000","last_active":"2024-04-25T10:30:34.996492+0000","last_peered":"2024-04-25T10:30:34.996492+0000","last_clean":"2024-04-25T10:30:34.996492+0000","last_became_active":"2024-04-25T10:30:33.648423+0000","last_became_peered":"2024-04-25T10:30:33.648423+0000","last_unstale":"2024-04-25T10:30:34.996492+0000","last_undegraded":"2024-04-25T10:30:34.996492+0000","last_fullsized":"2024-04-25T10:30:34.996492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:19.119189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.638405+0000","last_change":"2024-04-25T10:30:33.642061+0000","last_active":"2024-04-25T10:30:34.638405+0000","last_peered":"2024-04-25T10:30:34.638405+0000","last_clean":"2024-04-25T10:30:34.638405+0000","last_became_active":"2024-04-25T10:30:33.641299+0000","last_became_peered":"2024-04-25T10:30:33.641299+0000","last_unstale":"2024-04-25T10:30:34.638405+0000","last_undegraded":"2024-04-25T10:30:34.638405+0000","last_fullsized":"2024-04-25T10:30:34.638405+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:17.903107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.633470+0000","last_change":"2024-04-25T10:30:33.646786+0000","last_active":"2024-04-25T10:30:34.633470+0000","last_peered":"2024-04-25T10:30:34.633470+0000","last_clean":"2024-04-25T10:30:34.633470+0000","last_became_active":"2024-04-25T10:30:33.646456+0000","last_became_peered":"2024-04-25T10:30:33.646456+0000","last_unstale":"2024-04-25T10:30:34.633470+0000","last_undegraded":"2024-04-25T10:30:34.633470+0000","last_fullsized":"2024-04-25T10:30:34.633470+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:55.006723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545019+0000","last_change":"2024-04-25T10:30:33.639952+0000","last_active":"2024-04-25T10:30:35.545019+0000","last_peered":"2024-04-25T10:30:35.545019+0000","last_clean":"2024-04-25T10:30:35.545019+0000","last_became_active":"2024-04-25T10:30:33.639558+0000","last_became_peered":"2024-04-25T10:30:33.639558+0000","last_unstale":"2024-04-25T10:30:35.545019+0000","last_undegraded":"2024-04-25T10:30:35.545019+0000","last_fullsized":"2024-04-25T10:30:35.545019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:17.162902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.505976+0000","last_change":"2024-04-25T10:30:33.651530+0000","last_active":"2024-04-25T10:30:35.505976+0000","last_peered":"2024-04-25T10:30:35.505976+0000","last_clean":"2024-04-25T10:30:35.505976+0000","last_became_active":"2024-04-25T10:30:33.651151+0000","last_became_peered":"2024-04-25T10:30:33.651151+0000","last_unstale":"2024-04-25T10:30:35.505976+0000","last_undegraded":"2024-04-25T10:30:35.505976+0000","last_fullsized":"2024-04-25T10:30:35.505976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:31.835367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544378+0000","last_change":"2024-04-25T10:30:33.645146+0000","last_active":"2024-04-25T10:30:35.544378+0000","last_peered":"2024-04-25T10:30:35.544378+0000","last_clean":"2024-04-25T10:30:35.544378+0000","last_became_active":"2024-04-25T10:30:33.644310+0000","last_became_peered":"2024-04-25T10:30:33.644310+0000","last_unstale":"2024-04-25T10:30:35.544378+0000","last_undegraded":"2024-04-25T10:30:35.544378+0000","last_fullsized":"2024-04-25T10:30:35.544378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:45.034934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545469+0000","last_change":"2024-04-25T10:30:33.649385+0000","last_active":"2024-04-25T10:30:35.545469+0000","last_peered":"2024-04-25T10:30:35.545469+0000","last_clean":"2024-04-25T10:30:35.545469+0000","last_became_active":"2024-04-25T10:30:33.649002+0000","last_became_peered":"2024-04-25T10:30:33.649002+0000","last_unstale":"2024-04-25T10:30:35.545469+0000","last_undegraded":"2024-04-25T10:30:35.545469+0000","last_fullsized":"2024-04-25T10:30:35.545469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:42.259290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545253+0000","last_change":"2024-04-25T10:30:33.641918+0000","last_active":"2024-04-25T10:30:35.545253+0000","last_peered":"2024-04-25T10:30:35.545253+0000","last_clean":"2024-04-25T10:30:35.545253+0000","last_became_active":"2024-04-25T10:30:33.641553+0000","last_became_peered":"2024-04-25T10:30:33.641553+0000","last_unstale":"2024-04-25T10:30:35.545253+0000","last_undegraded":"2024-04-25T10:30:35.545253+0000","last_fullsized":"2024-04-25T10:30:35.545253+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:04.755694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.506398+0000","last_change":"2024-04-25T10:30:33.638054+0000","last_active":"2024-04-25T10:30:35.506398+0000","last_peered":"2024-04-25T10:30:35.506398+0000","last_clean":"2024-04-25T10:30:35.506398+0000","last_became_active":"2024-04-25T10:30:33.637720+0000","last_became_peered":"2024-04-25T10:30:33.637720+0000","last_unstale":"2024-04-25T10:30:35.506398+0000","last_undegraded":"2024-04-25T10:30:35.506398+0000","last_fullsized":"2024-04-25T10:30:35.506398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:41.557430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.638495+0000","last_change":"2024-04-25T10:30:33.642705+0000","last_active":"2024-04-25T10:30:34.638495+0000","last_peered":"2024-04-25T10:30:34.638495+0000","last_clean":"2024-04-25T10:30:34.638495+0000","last_became_active":"2024-04-25T10:30:33.641936+0000","last_became_peered":"2024-04-25T10:30:33.641936+0000","last_unstale":"2024-04-25T10:30:34.638495+0000","last_undegraded":"2024-04-25T10:30:34.638495+0000","last_fullsized":"2024-04-25T10:30:34.638495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:36.450389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.749723+0000","last_change":"2024-04-25T10:30:33.648987+0000","last_active":"2024-04-25T10:30:34.749723+0000","last_peered":"2024-04-25T10:30:34.749723+0000","last_clean":"2024-04-25T10:30:34.749723+0000","last_became_active":"2024-04-25T10:30:33.648621+0000","last_became_peered":"2024-04-25T10:30:33.648621+0000","last_unstale":"2024-04-25T10:30:34.749723+0000","last_undegraded":"2024-04-25T10:30:34.749723+0000","last_fullsized":"2024-04-25T10:30:34.749723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:19.999002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.996297+0000","last_change":"2024-04-25T10:30:33.647360+0000","last_active":"2024-04-25T10:30:34.996297+0000","last_peered":"2024-04-25T10:30:34.996297+0000","last_clean":"2024-04-25T10:30:34.996297+0000","last_became_active":"2024-04-25T10:30:33.646719+0000","last_became_peered":"2024-04-25T10:30:33.646719+0000","last_unstale":"2024-04-25T10:30:34.996297+0000","last_undegraded":"2024-04-25T10:30:34.996297+0000","last_fullsized":"2024-04-25T10:30:34.996297+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:19.016333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.996198+0000","last_change":"2024-04-25T10:30:33.646503+0000","last_active":"2024-04-25T10:30:34.996198+0000","last_peered":"2024-04-25T10:30:34.996198+0000","last_clean":"2024-04-25T10:30:34.996198+0000","last_became_active":"2024-04-25T10:30:33.646123+0000","last_became_peered":"2024-04-25T10:30:33.646123+0000","last_unstale":"2024-04-25T10:30:34.996198+0000","last_undegraded":"2024-04-25T10:30:34.996198+0000","last_fullsized":"2024-04-25T10:30:34.996198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:57.712216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544515+0000","last_change":"2024-04-25T10:30:33.640463+0000","last_active":"2024-04-25T10:30:35.544515+0000","last_peered":"2024-04-25T10:30:35.544515+0000","last_clean":"2024-04-25T10:30:35.544515+0000","last_became_active":"2024-04-25T10:30:33.639931+0000","last_became_peered":"2024-04-25T10:30:33.639931+0000","last_unstale":"2024-04-25T10:30:35.544515+0000","last_undegraded":"2024-04-25T10:30:35.544515+0000","last_fullsized":"2024-04-25T10:30:35.544515+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:18.863405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545484+0000","last_change":"2024-04-25T10:30:33.645695+0000","last_active":"2024-04-25T10:30:35.545484+0000","last_peered":"2024-04-25T10:30:35.545484+0000","last_clean":"2024-04-25T10:30:35.545484+0000","last_became_active":"2024-04-25T10:30:33.645387+0000","last_became_peered":"2024-04-25T10:30:33.645387+0000","last_unstale":"2024-04-25T10:30:35.545484+0000","last_undegraded":"2024-04-25T10:30:35.545484+0000","last_fullsized":"2024-04-25T10:30:35.545484+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:20.044456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.632775+0000","last_change":"2024-04-25T10:30:33.639087+0000","last_active":"2024-04-25T10:30:34.632775+0000","last_peered":"2024-04-25T10:30:34.632775+0000","last_clean":"2024-04-25T10:30:34.632775+0000","last_became_active":"2024-04-25T10:30:33.638726+0000","last_became_peered":"2024-04-25T10:30:33.638726+0000","last_unstale":"2024-04-25T10:30:34.632775+0000","last_undegraded":"2024-04-25T10:30:34.632775+0000","last_fullsized":"2024-04-25T10:30:34.632775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:36.887289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.852731+0000","last_change":"2024-04-25T10:30:33.638646+0000","last_active":"2024-04-25T10:30:34.852731+0000","last_peered":"2024-04-25T10:30:34.852731+0000","last_clean":"2024-04-25T10:30:34.852731+0000","last_became_active":"2024-04-25T10:30:33.638261+0000","last_became_peered":"2024-04-25T10:30:33.638261+0000","last_unstale":"2024-04-25T10:30:34.852731+0000","last_undegraded":"2024-04-25T10:30:34.852731+0000","last_fullsized":"2024-04-25T10:30:34.852731+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:08.212310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.814050+0000","last_change":"2024-04-25T10:30:32.899992+0000","last_active":"2024-04-25T10:30:37.814050+0000","last_peered":"2024-04-25T10:30:37.814050+0000","last_clean":"2024-04-25T10:30:37.814050+0000","last_became_active":"2024-04-25T10:30:32.899643+0000","last_became_peered":"2024-04-25T10:30:32.899643+0000","last_unstale":"2024-04-25T10:30:37.814050+0000","last_undegraded":"2024-04-25T10:30:37.814050+0000","last_fullsized":"2024-04-25T10:30:37.814050+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:34.245121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544806+0000","last_change":"2024-04-25T10:30:33.644884+0000","last_active":"2024-04-25T10:30:35.544806+0000","last_peered":"2024-04-25T10:30:35.544806+0000","last_clean":"2024-04-25T10:30:35.544806+0000","last_became_active":"2024-04-25T10:30:33.644510+0000","last_became_peered":"2024-04-25T10:30:33.644510+0000","last_unstale":"2024-04-25T10:30:35.544806+0000","last_undegraded":"2024-04-25T10:30:35.544806+0000","last_fullsized":"2024-04-25T10:30:35.544806+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:09.282865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"76'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544712+0000","last_change":"2024-04-25T10:30:32.909175+0000","last_active":"2024-04-25T10:30:35.544712+0000","last_peered":"2024-04-25T10:30:35.544712+0000","last_clean":"2024-04-25T10:30:35.544712+0000","last_became_active":"2024-04-25T10:30:32.908813+0000","last_became_peered":"2024-04-25T10:30:32.908813+0000","last_unstale":"2024-04-25T10:30:35.544712+0000","last_undegraded":"2024-04-25T10:30:35.544712+0000","last_fullsized":"2024-04-25T10:30:35.544712+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:04.256452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.996498+0000","last_change":"2024-04-25T10:30:33.642095+0000","last_active":"2024-04-25T10:30:34.996498+0000","last_peered":"2024-04-25T10:30:34.996498+0000","last_clean":"2024-04-25T10:30:34.996498+0000","last_became_active":"2024-04-25T10:30:33.641713+0000","last_became_peered":"2024-04-25T10:30:33.641713+0000","last_unstale":"2024-04-25T10:30:34.996498+0000","last_undegraded":"2024-04-25T10:30:34.996498+0000","last_fullsized":"2024-04-25T10:30:34.996498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:42.425197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.813937+0000","last_change":"2024-04-25T10:30:32.904376+0000","last_active":"2024-04-25T10:30:37.813937+0000","last_peered":"2024-04-25T10:30:37.813937+0000","last_clean":"2024-04-25T10:30:37.813937+0000","last_became_active":"2024-04-25T10:30:32.903994+0000","last_became_peered":"2024-04-25T10:30:32.903994+0000","last_unstale":"2024-04-25T10:30:37.813937+0000","last_undegraded":"2024-04-25T10:30:37.813937+0000","last_fullsized":"2024-04-25T10:30:37.813937+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:52.727800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544730+0000","last_change":"2024-04-25T10:30:33.637171+0000","last_active":"2024-04-25T10:30:35.544730+0000","last_peered":"2024-04-25T10:30:35.544730+0000","last_clean":"2024-04-25T10:30:35.544730+0000","last_became_active":"2024-04-25T10:30:33.636822+0000","last_became_peered":"2024-04-25T10:30:33.636822+0000","last_unstale":"2024-04-25T10:30:35.544730+0000","last_undegraded":"2024-04-25T10:30:35.544730+0000","last_fullsized":"2024-04-25T10:30:35.544730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:54.220784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.633679+0000","last_change":"2024-04-25T10:30:32.900856+0000","last_active":"2024-04-25T10:30:34.633679+0000","last_peered":"2024-04-25T10:30:34.633679+0000","last_clean":"2024-04-25T10:30:34.633679+0000","last_became_active":"2024-04-25T10:30:32.900519+0000","last_became_peered":"2024-04-25T10:30:32.900519+0000","last_unstale":"2024-04-25T10:30:34.633679+0000","last_undegraded":"2024-04-25T10:30:34.633679+0000","last_fullsized":"2024-04-25T10:30:34.633679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:49.738005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545639+0000","last_change":"2024-04-25T10:30:33.651736+0000","last_active":"2024-04-25T10:30:35.545639+0000","last_peered":"2024-04-25T10:30:35.545639+0000","last_clean":"2024-04-25T10:30:35.545639+0000","last_became_active":"2024-04-25T10:30:33.651395+0000","last_became_peered":"2024-04-25T10:30:33.651395+0000","last_unstale":"2024-04-25T10:30:35.545639+0000","last_undegraded":"2024-04-25T10:30:35.545639+0000","last_fullsized":"2024-04-25T10:30:35.545639+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:58.230092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.802980+0000","last_change":"2024-04-25T10:30:32.901348+0000","last_active":"2024-04-25T10:30:37.802980+0000","last_peered":"2024-04-25T10:30:37.802980+0000","last_clean":"2024-04-25T10:30:37.802980+0000","last_became_active":"2024-04-25T10:30:32.901025+0000","last_became_peered":"2024-04-25T10:30:32.901025+0000","last_unstale":"2024-04-25T10:30:37.802980+0000","last_undegraded":"2024-04-25T10:30:37.802980+0000","last_fullsized":"2024-04-25T10:30:37.802980+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:00.521934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.633827+0000","last_change":"2024-04-25T10:30:33.649206+0000","last_active":"2024-04-25T10:30:34.633827+0000","last_peered":"2024-04-25T10:30:34.633827+0000","last_clean":"2024-04-25T10:30:34.633827+0000","last_became_active":"2024-04-25T10:30:33.648883+0000","last_became_peered":"2024-04-25T10:30:33.648883+0000","last_unstale":"2024-04-25T10:30:34.633827+0000","last_undegraded":"2024-04-25T10:30:34.633827+0000","last_fullsized":"2024-04-25T10:30:34.633827+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:43.105562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.812920+0000","last_change":"2024-04-25T10:30:32.901201+0000","last_active":"2024-04-25T10:30:37.812920+0000","last_peered":"2024-04-25T10:30:37.812920+0000","last_clean":"2024-04-25T10:30:37.812920+0000","last_became_active":"2024-04-25T10:30:32.900876+0000","last_became_peered":"2024-04-25T10:30:32.900876+0000","last_unstale":"2024-04-25T10:30:37.812920+0000","last_undegraded":"2024-04-25T10:30:37.812920+0000","last_fullsized":"2024-04-25T10:30:37.812920+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:58.077045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545552+0000","last_change":"2024-04-25T10:30:33.651935+0000","last_active":"2024-04-25T10:30:35.545552+0000","last_peered":"2024-04-25T10:30:35.545552+0000","last_clean":"2024-04-25T10:30:35.545552+0000","last_became_active":"2024-04-25T10:30:33.651568+0000","last_became_peered":"2024-04-25T10:30:33.651568+0000","last_unstale":"2024-04-25T10:30:35.545552+0000","last_undegraded":"2024-04-25T10:30:35.545552+0000","last_fullsized":"2024-04-25T10:30:35.545552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:58.849535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.813717+0000","last_change":"2024-04-25T10:30:32.896500+0000","last_active":"2024-04-25T10:30:37.813717+0000","last_peered":"2024-04-25T10:30:37.813717+0000","last_clean":"2024-04-25T10:30:37.813717+0000","last_became_active":"2024-04-25T10:30:32.896071+0000","last_became_peered":"2024-04-25T10:30:32.896071+0000","last_unstale":"2024-04-25T10:30:37.813717+0000","last_undegraded":"2024-04-25T10:30:37.813717+0000","last_fullsized":"2024-04-25T10:30:37.813717+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:10.385443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544569+0000","last_change":"2024-04-25T10:30:33.631730+0000","last_active":"2024-04-25T10:30:35.544569+0000","last_peered":"2024-04-25T10:30:35.544569+0000","last_clean":"2024-04-25T10:30:35.544569+0000","last_became_active":"2024-04-25T10:30:33.631374+0000","last_became_peered":"2024-04-25T10:30:33.631374+0000","last_unstale":"2024-04-25T10:30:35.544569+0000","last_undegraded":"2024-04-25T10:30:35.544569+0000","last_fullsized":"2024-04-25T10:30:35.544569+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:14.957866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544462+0000","last_change":"2024-04-25T10:30:32.900762+0000","last_active":"2024-04-25T10:30:35.544462+0000","last_peered":"2024-04-25T10:30:35.544462+0000","last_clean":"2024-04-25T10:30:35.544462+0000","last_became_active":"2024-04-25T10:30:32.900459+0000","last_became_peered":"2024-04-25T10:30:32.900459+0000","last_unstale":"2024-04-25T10:30:35.544462+0000","last_undegraded":"2024-04-25T10:30:35.544462+0000","last_fullsized":"2024-04-25T10:30:35.544462+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:22.286261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.632589+0000","last_change":"2024-04-25T10:30:33.637727+0000","last_active":"2024-04-25T10:30:34.632589+0000","last_peered":"2024-04-25T10:30:34.632589+0000","last_clean":"2024-04-25T10:30:34.632589+0000","last_became_active":"2024-04-25T10:30:33.637359+0000","last_became_peered":"2024-04-25T10:30:33.637359+0000","last_unstale":"2024-04-25T10:30:34.632589+0000","last_undegraded":"2024-04-25T10:30:34.632589+0000","last_fullsized":"2024-04-25T10:30:34.632589+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:21.877562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'3","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.802361+0000","last_change":"2024-04-25T10:30:32.894210+0000","last_active":"2024-04-25T10:30:37.802361+0000","last_peered":"2024-04-25T10:30:37.802361+0000","last_clean":"2024-04-25T10:30:37.802361+0000","last_became_active":"2024-04-25T10:30:32.893241+0000","last_became_peered":"2024-04-25T10:30:32.893241+0000","last_unstale":"2024-04-25T10:30:37.802361+0000","last_undegraded":"2024-04-25T10:30:37.802361+0000","last_fullsized":"2024-04-25T10:30:37.802361+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:06.340392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545029+0000","last_change":"2024-04-25T10:30:33.632349+0000","last_active":"2024-04-25T10:30:35.545029+0000","last_peered":"2024-04-25T10:30:35.545029+0000","last_clean":"2024-04-25T10:30:35.545029+0000","last_became_active":"2024-04-25T10:30:33.631968+0000","last_became_peered":"2024-04-25T10:30:33.631968+0000","last_unstale":"2024-04-25T10:30:35.545029+0000","last_undegraded":"2024-04-25T10:30:35.545029+0000","last_fullsized":"2024-04-25T10:30:35.545029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:29.090932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.189939+0000","last_change":"2024-04-25T10:30:32.900105+0000","last_active":"2024-04-25T10:30:37.189939+0000","last_peered":"2024-04-25T10:30:37.189939+0000","last_clean":"2024-04-25T10:30:37.189939+0000","last_became_active":"2024-04-25T10:30:32.899740+0000","last_became_peered":"2024-04-25T10:30:32.899740+0000","last_unstale":"2024-04-25T10:30:37.189939+0000","last_undegraded":"2024-04-25T10:30:37.189939+0000","last_fullsized":"2024-04-25T10:30:37.189939+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:49.390799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.996650+0000","last_change":"2024-04-25T10:30:33.649293+0000","last_active":"2024-04-25T10:30:34.996650+0000","last_peered":"2024-04-25T10:30:34.996650+0000","last_clean":"2024-04-25T10:30:34.996650+0000","last_became_active":"2024-04-25T10:30:33.648899+0000","last_became_peered":"2024-04-25T10:30:33.648899+0000","last_unstale":"2024-04-25T10:30:34.996650+0000","last_undegraded":"2024-04-25T10:30:34.996650+0000","last_fullsized":"2024-04-25T10:30:34.996650+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:13.485155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.189424+0000","last_change":"2024-04-25T10:30:32.893453+0000","last_active":"2024-04-25T10:30:37.189424+0000","last_peered":"2024-04-25T10:30:37.189424+0000","last_clean":"2024-04-25T10:30:37.189424+0000","last_became_active":"2024-04-25T10:30:32.893106+0000","last_became_peered":"2024-04-25T10:30:32.893106+0000","last_unstale":"2024-04-25T10:30:37.189424+0000","last_undegraded":"2024-04-25T10:30:37.189424+0000","last_fullsized":"2024-04-25T10:30:37.189424+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:57.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545058+0000","last_change":"2024-04-25T10:30:33.645693+0000","last_active":"2024-04-25T10:30:35.545058+0000","last_peered":"2024-04-25T10:30:35.545058+0000","last_clean":"2024-04-25T10:30:35.545058+0000","last_became_active":"2024-04-25T10:30:33.645342+0000","last_became_peered":"2024-04-25T10:30:33.645342+0000","last_unstale":"2024-04-25T10:30:35.545058+0000","last_undegraded":"2024-04-25T10:30:35.545058+0000","last_fullsized":"2024-04-25T10:30:35.545058+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:15.729860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'11","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:39.179891+0000","last_change":"2024-04-25T10:30:32.897973+0000","last_active":"2024-04-25T10:30:39.179891+0000","last_peered":"2024-04-25T10:30:39.179891+0000","last_clean":"2024-04-25T10:30:39.179891+0000","last_became_active":"2024-04-25T10:30:32.897409+0000","last_became_peered":"2024-04-25T10:30:32.897409+0000","last_unstale":"2024-04-25T10:30:39.179891+0000","last_undegraded":"2024-04-25T10:30:39.179891+0000","last_fullsized":"2024-04-25T10:30:39.179891+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:47.024554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8212,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":12,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.633484+0000","last_change":"2024-04-25T10:30:33.641768+0000","last_active":"2024-04-25T10:30:34.633484+0000","last_peered":"2024-04-25T10:30:34.633484+0000","last_clean":"2024-04-25T10:30:34.633484+0000","last_became_active":"2024-04-25T10:30:33.641415+0000","last_became_peered":"2024-04-25T10:30:33.641415+0000","last_unstale":"2024-04-25T10:30:34.633484+0000","last_undegraded":"2024-04-25T10:30:34.633484+0000","last_fullsized":"2024-04-25T10:30:34.633484+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:38.569474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.813688+0000","last_change":"2024-04-25T10:30:32.898703+0000","last_active":"2024-04-25T10:30:37.813688+0000","last_peered":"2024-04-25T10:30:37.813688+0000","last_clean":"2024-04-25T10:30:37.813688+0000","last_became_active":"2024-04-25T10:30:32.898340+0000","last_became_peered":"2024-04-25T10:30:32.898340+0000","last_unstale":"2024-04-25T10:30:37.813688+0000","last_undegraded":"2024-04-25T10:30:37.813688+0000","last_fullsized":"2024-04-25T10:30:37.813688+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:27.024144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.749575+0000","last_change":"2024-04-25T10:30:33.640393+0000","last_active":"2024-04-25T10:30:34.749575+0000","last_peered":"2024-04-25T10:30:34.749575+0000","last_clean":"2024-04-25T10:30:34.749575+0000","last_became_active":"2024-04-25T10:30:33.639822+0000","last_became_peered":"2024-04-25T10:30:33.639822+0000","last_unstale":"2024-04-25T10:30:34.749575+0000","last_undegraded":"2024-04-25T10:30:34.749575+0000","last_fullsized":"2024-04-25T10:30:34.749575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:08.618406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.189214+0000","last_change":"2024-04-25T10:30:32.900880+0000","last_active":"2024-04-25T10:30:37.189214+0000","last_peered":"2024-04-25T10:30:37.189214+0000","last_clean":"2024-04-25T10:30:37.189214+0000","last_became_active":"2024-04-25T10:30:32.899987+0000","last_became_peered":"2024-04-25T10:30:32.899987+0000","last_unstale":"2024-04-25T10:30:37.189214+0000","last_undegraded":"2024-04-25T10:30:37.189214+0000","last_fullsized":"2024-04-25T10:30:37.189214+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:35.687773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.506389+0000","last_change":"2024-04-25T10:30:33.641933+0000","last_active":"2024-04-25T10:30:35.506389+0000","last_peered":"2024-04-25T10:30:35.506389+0000","last_clean":"2024-04-25T10:30:35.506389+0000","last_became_active":"2024-04-25T10:30:33.641549+0000","last_became_peered":"2024-04-25T10:30:33.641549+0000","last_unstale":"2024-04-25T10:30:35.506389+0000","last_undegraded":"2024-04-25T10:30:35.506389+0000","last_fullsized":"2024-04-25T10:30:35.506389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:36.696523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'3","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.813062+0000","last_change":"2024-04-25T10:30:32.901178+0000","last_active":"2024-04-25T10:30:37.813062+0000","last_peered":"2024-04-25T10:30:37.813062+0000","last_clean":"2024-04-25T10:30:37.813062+0000","last_became_active":"2024-04-25T10:30:32.900817+0000","last_became_peered":"2024-04-25T10:30:32.900817+0000","last_unstale":"2024-04-25T10:30:37.813062+0000","last_undegraded":"2024-04-25T10:30:37.813062+0000","last_fullsized":"2024-04-25T10:30:37.813062+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:53.999253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.505920+0000","last_change":"2024-04-25T10:30:33.653245+0000","last_active":"2024-04-25T10:30:35.505920+0000","last_peered":"2024-04-25T10:30:35.505920+0000","last_clean":"2024-04-25T10:30:35.505920+0000","last_became_active":"2024-04-25T10:30:33.652885+0000","last_became_peered":"2024-04-25T10:30:33.652885+0000","last_unstale":"2024-04-25T10:30:35.505920+0000","last_undegraded":"2024-04-25T10:30:35.505920+0000","last_fullsized":"2024-04-25T10:30:35.505920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:59.888048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.814486+0000","last_change":"2024-04-25T10:30:32.902862+0000","last_active":"2024-04-25T10:30:37.814486+0000","last_peered":"2024-04-25T10:30:37.814486+0000","last_clean":"2024-04-25T10:30:37.814486+0000","last_became_active":"2024-04-25T10:30:32.902537+0000","last_became_peered":"2024-04-25T10:30:32.902537+0000","last_unstale":"2024-04-25T10:30:37.814486+0000","last_undegraded":"2024-04-25T10:30:37.814486+0000","last_fullsized":"2024-04-25T10:30:37.814486+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:55.974128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545098+0000","last_change":"2024-04-25T10:30:33.651409+0000","last_active":"2024-04-25T10:30:35.545098+0000","last_peered":"2024-04-25T10:30:35.545098+0000","last_clean":"2024-04-25T10:30:35.545098+0000","last_became_active":"2024-04-25T10:30:33.650673+0000","last_became_peered":"2024-04-25T10:30:33.650673+0000","last_unstale":"2024-04-25T10:30:35.545098+0000","last_undegraded":"2024-04-25T10:30:35.545098+0000","last_fullsized":"2024-04-25T10:30:35.545098+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:48.989198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'1","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.633868+0000","last_change":"2024-04-25T10:30:32.900524+0000","last_active":"2024-04-25T10:30:34.633868+0000","last_peered":"2024-04-25T10:30:34.633868+0000","last_clean":"2024-04-25T10:30:34.633868+0000","last_became_active":"2024-04-25T10:30:32.899719+0000","last_became_peered":"2024-04-25T10:30:32.899719+0000","last_unstale":"2024-04-25T10:30:34.633868+0000","last_undegraded":"2024-04-25T10:30:34.633868+0000","last_fullsized":"2024-04-25T10:30:34.633868+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:45.749965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545021+0000","last_change":"2024-04-25T10:30:33.643929+0000","last_active":"2024-04-25T10:30:35.545021+0000","last_peered":"2024-04-25T10:30:35.545021+0000","last_clean":"2024-04-25T10:30:35.545021+0000","last_became_active":"2024-04-25T10:30:33.643593+0000","last_became_peered":"2024-04-25T10:30:33.643593+0000","last_unstale":"2024-04-25T10:30:35.545021+0000","last_undegraded":"2024-04-25T10:30:35.545021+0000","last_fullsized":"2024-04-25T10:30:35.545021+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:39.591022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.632535+0000","last_change":"2024-04-25T10:30:32.898937+0000","last_active":"2024-04-25T10:30:34.632535+0000","last_peered":"2024-04-25T10:30:34.632535+0000","last_clean":"2024-04-25T10:30:34.632535+0000","last_became_active":"2024-04-25T10:30:32.898570+0000","last_became_peered":"2024-04-25T10:30:32.898570+0000","last_unstale":"2024-04-25T10:30:34.632535+0000","last_undegraded":"2024-04-25T10:30:34.632535+0000","last_fullsized":"2024-04-25T10:30:34.632535+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:40.123035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545555+0000","last_change":"2024-04-25T10:30:33.632857+0000","last_active":"2024-04-25T10:30:35.545555+0000","last_peered":"2024-04-25T10:30:35.545555+0000","last_clean":"2024-04-25T10:30:35.545555+0000","last_became_active":"2024-04-25T10:30:33.632043+0000","last_became_peered":"2024-04-25T10:30:33.632043+0000","last_unstale":"2024-04-25T10:30:35.545555+0000","last_undegraded":"2024-04-25T10:30:35.545555+0000","last_fullsized":"2024-04-25T10:30:35.545555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:12.683035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.506121+0000","last_change":"2024-04-25T10:30:32.901349+0000","last_active":"2024-04-25T10:30:35.506121+0000","last_peered":"2024-04-25T10:30:35.506121+0000","last_clean":"2024-04-25T10:30:35.506121+0000","last_became_active":"2024-04-25T10:30:32.900881+0000","last_became_peered":"2024-04-25T10:30:32.900881+0000","last_unstale":"2024-04-25T10:30:35.506121+0000","last_undegraded":"2024-04-25T10:30:35.506121+0000","last_fullsized":"2024-04-25T10:30:35.506121+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:34.963564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545185+0000","last_change":"2024-04-25T10:30:33.644376+0000","last_active":"2024-04-25T10:30:35.545185+0000","last_peered":"2024-04-25T10:30:35.545185+0000","last_clean":"2024-04-25T10:30:35.545185+0000","last_became_active":"2024-04-25T10:30:33.644055+0000","last_became_peered":"2024-04-25T10:30:33.644055+0000","last_unstale":"2024-04-25T10:30:35.545185+0000","last_undegraded":"2024-04-25T10:30:35.545185+0000","last_fullsized":"2024-04-25T10:30:35.545185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:48.134775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544608+0000","last_change":"2024-04-25T10:30:33.646312+0000","last_active":"2024-04-25T10:30:35.544608+0000","last_peered":"2024-04-25T10:30:35.544608+0000","last_clean":"2024-04-25T10:30:35.544608+0000","last_became_active":"2024-04-25T10:30:33.642338+0000","last_became_peered":"2024-04-25T10:30:33.642338+0000","last_unstale":"2024-04-25T10:30:35.544608+0000","last_undegraded":"2024-04-25T10:30:35.544608+0000","last_fullsized":"2024-04-25T10:30:35.544608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:42.588360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"77'4","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.814265+0000","last_change":"2024-04-25T10:30:32.909680+0000","last_active":"2024-04-25T10:30:37.814265+0000","last_peered":"2024-04-25T10:30:37.814265+0000","last_clean":"2024-04-25T10:30:37.814265+0000","last_became_active":"2024-04-25T10:30:32.909169+0000","last_became_peered":"2024-04-25T10:30:32.909169+0000","last_unstale":"2024-04-25T10:30:37.814265+0000","last_undegraded":"2024-04-25T10:30:37.814265+0000","last_fullsized":"2024-04-25T10:30:37.814265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:54.876475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544800+0000","last_change":"2024-04-25T10:30:33.649524+0000","last_active":"2024-04-25T10:30:35.544800+0000","last_peered":"2024-04-25T10:30:35.544800+0000","last_clean":"2024-04-25T10:30:35.544800+0000","last_became_active":"2024-04-25T10:30:33.648958+0000","last_became_peered":"2024-04-25T10:30:33.648958+0000","last_unstale":"2024-04-25T10:30:35.544800+0000","last_undegraded":"2024-04-25T10:30:35.544800+0000","last_fullsized":"2024-04-25T10:30:35.544800+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:13.407742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.800832+0000","last_change":"2024-04-25T10:30:32.895270+0000","last_active":"2024-04-25T10:30:37.800832+0000","last_peered":"2024-04-25T10:30:37.800832+0000","last_clean":"2024-04-25T10:30:37.800832+0000","last_became_active":"2024-04-25T10:30:32.894899+0000","last_became_peered":"2024-04-25T10:30:32.894899+0000","last_unstale":"2024-04-25T10:30:37.800832+0000","last_undegraded":"2024-04-25T10:30:37.800832+0000","last_fullsized":"2024-04-25T10:30:37.800832+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:21.233217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":91,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.505963+0000","last_change":"2024-04-25T10:29:21.105073+0000","last_active":"2024-04-25T10:30:35.505963+0000","last_peered":"2024-04-25T10:30:35.505963+0000","last_clean":"2024-04-25T10:30:35.505963+0000","last_became_active":"2024-04-25T10:29:21.098771+0000","last_became_peered":"2024-04-25T10:29:21.098771+0000","last_unstale":"2024-04-25T10:30:35.505963+0000","last_undegraded":"2024-04-25T10:30:35.505963+0000","last_fullsized":"2024-04-25T10:30:35.505963+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:59.015440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:59.015440+0000","last_clean_scrub_stamp":"2024-04-25T10:25:59.015440+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:41.728977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.749397+0000","last_change":"2024-04-25T10:30:33.639159+0000","last_active":"2024-04-25T10:30:34.749397+0000","last_peered":"2024-04-25T10:30:34.749397+0000","last_clean":"2024-04-25T10:30:34.749397+0000","last_became_active":"2024-04-25T10:30:33.638761+0000","last_became_peered":"2024-04-25T10:30:33.638761+0000","last_unstale":"2024-04-25T10:30:34.749397+0000","last_undegraded":"2024-04-25T10:30:34.749397+0000","last_fullsized":"2024-04-25T10:30:34.749397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:43.262103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.813530+0000","last_change":"2024-04-25T10:30:32.898587+0000","last_active":"2024-04-25T10:30:37.813530+0000","last_peered":"2024-04-25T10:30:37.813530+0000","last_clean":"2024-04-25T10:30:37.813530+0000","last_became_active":"2024-04-25T10:30:32.898113+0000","last_became_peered":"2024-04-25T10:30:32.898113+0000","last_unstale":"2024-04-25T10:30:37.813530+0000","last_undegraded":"2024-04-25T10:30:37.813530+0000","last_fullsized":"2024-04-25T10:30:37.813530+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:24.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.187770+0000","last_change":"2024-04-25T10:30:32.907188+0000","last_active":"2024-04-25T10:30:37.187770+0000","last_peered":"2024-04-25T10:30:37.187770+0000","last_clean":"2024-04-25T10:30:37.187770+0000","last_became_active":"2024-04-25T10:30:32.906837+0000","last_became_peered":"2024-04-25T10:30:32.906837+0000","last_unstale":"2024-04-25T10:30:37.187770+0000","last_undegraded":"2024-04-25T10:30:37.187770+0000","last_fullsized":"2024-04-25T10:30:37.187770+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:11.835053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.749783+0000","last_change":"2024-04-25T10:30:33.636159+0000","last_active":"2024-04-25T10:30:34.749783+0000","last_peered":"2024-04-25T10:30:34.749783+0000","last_clean":"2024-04-25T10:30:34.749783+0000","last_became_active":"2024-04-25T10:30:33.635098+0000","last_became_peered":"2024-04-25T10:30:33.635098+0000","last_unstale":"2024-04-25T10:30:34.749783+0000","last_undegraded":"2024-04-25T10:30:34.749783+0000","last_fullsized":"2024-04-25T10:30:34.749783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:13.092368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'3","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.812860+0000","last_change":"2024-04-25T10:30:32.905469+0000","last_active":"2024-04-25T10:30:37.812860+0000","last_peered":"2024-04-25T10:30:37.812860+0000","last_clean":"2024-04-25T10:30:37.812860+0000","last_became_active":"2024-04-25T10:30:32.905123+0000","last_became_peered":"2024-04-25T10:30:32.905123+0000","last_unstale":"2024-04-25T10:30:37.812860+0000","last_undegraded":"2024-04-25T10:30:37.812860+0000","last_fullsized":"2024-04-25T10:30:37.812860+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:12.884809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544691+0000","last_change":"2024-04-25T10:30:33.640690+0000","last_active":"2024-04-25T10:30:35.544691+0000","last_peered":"2024-04-25T10:30:35.544691+0000","last_clean":"2024-04-25T10:30:35.544691+0000","last_became_active":"2024-04-25T10:30:33.640310+0000","last_became_peered":"2024-04-25T10:30:33.640310+0000","last_unstale":"2024-04-25T10:30:35.544691+0000","last_undegraded":"2024-04-25T10:30:35.544691+0000","last_fullsized":"2024-04-25T10:30:35.544691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:03.671495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.187977+0000","last_change":"2024-04-25T10:30:32.893743+0000","last_active":"2024-04-25T10:30:37.187977+0000","last_peered":"2024-04-25T10:30:37.187977+0000","last_clean":"2024-04-25T10:30:37.187977+0000","last_became_active":"2024-04-25T10:30:32.893360+0000","last_became_peered":"2024-04-25T10:30:32.893360+0000","last_unstale":"2024-04-25T10:30:37.187977+0000","last_undegraded":"2024-04-25T10:30:37.187977+0000","last_fullsized":"2024-04-25T10:30:37.187977+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:39.830376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.996500+0000","last_change":"2024-04-25T10:30:33.645127+0000","last_active":"2024-04-25T10:30:34.996500+0000","last_peered":"2024-04-25T10:30:34.996500+0000","last_clean":"2024-04-25T10:30:34.996500+0000","last_became_active":"2024-04-25T10:30:33.644259+0000","last_became_peered":"2024-04-25T10:30:33.644259+0000","last_unstale":"2024-04-25T10:30:34.996500+0000","last_undegraded":"2024-04-25T10:30:34.996500+0000","last_fullsized":"2024-04-25T10:30:34.996500+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:56.849861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.189254+0000","last_change":"2024-04-25T10:30:32.896885+0000","last_active":"2024-04-25T10:30:37.189254+0000","last_peered":"2024-04-25T10:30:37.189254+0000","last_clean":"2024-04-25T10:30:37.189254+0000","last_became_active":"2024-04-25T10:30:32.896447+0000","last_became_peered":"2024-04-25T10:30:32.896447+0000","last_unstale":"2024-04-25T10:30:37.189254+0000","last_undegraded":"2024-04-25T10:30:37.189254+0000","last_fullsized":"2024-04-25T10:30:37.189254+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:13.133549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.633322+0000","last_change":"2024-04-25T10:30:33.645016+0000","last_active":"2024-04-25T10:30:34.633322+0000","last_peered":"2024-04-25T10:30:34.633322+0000","last_clean":"2024-04-25T10:30:34.633322+0000","last_became_active":"2024-04-25T10:30:33.644661+0000","last_became_peered":"2024-04-25T10:30:33.644661+0000","last_unstale":"2024-04-25T10:30:34.633322+0000","last_undegraded":"2024-04-25T10:30:34.633322+0000","last_fullsized":"2024-04-25T10:30:34.633322+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:20.935124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544370+0000","last_change":"2024-04-25T10:30:32.907827+0000","last_active":"2024-04-25T10:30:35.544370+0000","last_peered":"2024-04-25T10:30:35.544370+0000","last_clean":"2024-04-25T10:30:35.544370+0000","last_became_active":"2024-04-25T10:30:32.907402+0000","last_became_peered":"2024-04-25T10:30:32.907402+0000","last_unstale":"2024-04-25T10:30:35.544370+0000","last_undegraded":"2024-04-25T10:30:35.544370+0000","last_fullsized":"2024-04-25T10:30:35.544370+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:38.501617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544613+0000","last_change":"2024-04-25T10:30:33.636935+0000","last_active":"2024-04-25T10:30:35.544613+0000","last_peered":"2024-04-25T10:30:35.544613+0000","last_clean":"2024-04-25T10:30:35.544613+0000","last_became_active":"2024-04-25T10:30:33.636566+0000","last_became_peered":"2024-04-25T10:30:33.636566+0000","last_unstale":"2024-04-25T10:30:35.544613+0000","last_undegraded":"2024-04-25T10:30:35.544613+0000","last_fullsized":"2024-04-25T10:30:35.544613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:24.556275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.852370+0000","last_change":"2024-04-25T10:30:32.902457+0000","last_active":"2024-04-25T10:30:34.852370+0000","last_peered":"2024-04-25T10:30:34.852370+0000","last_clean":"2024-04-25T10:30:34.852370+0000","last_became_active":"2024-04-25T10:30:32.901173+0000","last_became_peered":"2024-04-25T10:30:32.901173+0000","last_unstale":"2024-04-25T10:30:34.852370+0000","last_undegraded":"2024-04-25T10:30:34.852370+0000","last_fullsized":"2024-04-25T10:30:34.852370+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:59.305176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544622+0000","last_change":"2024-04-25T10:30:33.642986+0000","last_active":"2024-04-25T10:30:35.544622+0000","last_peered":"2024-04-25T10:30:35.544622+0000","last_clean":"2024-04-25T10:30:35.544622+0000","last_became_active":"2024-04-25T10:30:33.642545+0000","last_became_peered":"2024-04-25T10:30:33.642545+0000","last_unstale":"2024-04-25T10:30:35.544622+0000","last_undegraded":"2024-04-25T10:30:35.544622+0000","last_fullsized":"2024-04-25T10:30:35.544622+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:49.369648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.188806+0000","last_change":"2024-04-25T10:30:32.902239+0000","last_active":"2024-04-25T10:30:37.188806+0000","last_peered":"2024-04-25T10:30:37.188806+0000","last_clean":"2024-04-25T10:30:37.188806+0000","last_became_active":"2024-04-25T10:30:32.901653+0000","last_became_peered":"2024-04-25T10:30:32.901653+0000","last_unstale":"2024-04-25T10:30:37.188806+0000","last_undegraded":"2024-04-25T10:30:37.188806+0000","last_fullsized":"2024-04-25T10:30:37.188806+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:34.732428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.506300+0000","last_change":"2024-04-25T10:30:33.639763+0000","last_active":"2024-04-25T10:30:35.506300+0000","last_peered":"2024-04-25T10:30:35.506300+0000","last_clean":"2024-04-25T10:30:35.506300+0000","last_became_active":"2024-04-25T10:30:33.638004+0000","last_became_peered":"2024-04-25T10:30:33.638004+0000","last_unstale":"2024-04-25T10:30:35.506300+0000","last_undegraded":"2024-04-25T10:30:35.506300+0000","last_fullsized":"2024-04-25T10:30:35.506300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:09.722333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.814590+0000","last_change":"2024-04-25T10:30:32.894823+0000","last_active":"2024-04-25T10:30:37.814590+0000","last_peered":"2024-04-25T10:30:37.814590+0000","last_clean":"2024-04-25T10:30:37.814590+0000","last_became_active":"2024-04-25T10:30:32.893819+0000","last_became_peered":"2024-04-25T10:30:32.893819+0000","last_unstale":"2024-04-25T10:30:37.814590+0000","last_undegraded":"2024-04-25T10:30:37.814590+0000","last_fullsized":"2024-04-25T10:30:37.814590+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:59.809410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.506475+0000","last_change":"2024-04-25T10:30:33.637046+0000","last_active":"2024-04-25T10:30:35.506475+0000","last_peered":"2024-04-25T10:30:35.506475+0000","last_clean":"2024-04-25T10:30:35.506475+0000","last_became_active":"2024-04-25T10:30:33.634903+0000","last_became_peered":"2024-04-25T10:30:33.634903+0000","last_unstale":"2024-04-25T10:30:35.506475+0000","last_undegraded":"2024-04-25T10:30:35.506475+0000","last_fullsized":"2024-04-25T10:30:35.506475+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:52.252812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'1","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545506+0000","last_change":"2024-04-25T10:30:32.898979+0000","last_active":"2024-04-25T10:30:35.545506+0000","last_peered":"2024-04-25T10:30:35.545506+0000","last_clean":"2024-04-25T10:30:35.545506+0000","last_became_active":"2024-04-25T10:30:32.898611+0000","last_became_peered":"2024-04-25T10:30:32.898611+0000","last_unstale":"2024-04-25T10:30:35.545506+0000","last_undegraded":"2024-04-25T10:30:35.545506+0000","last_fullsized":"2024-04-25T10:30:35.545506+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.549769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.506274+0000","last_change":"2024-04-25T10:30:33.651596+0000","last_active":"2024-04-25T10:30:35.506274+0000","last_peered":"2024-04-25T10:30:35.506274+0000","last_clean":"2024-04-25T10:30:35.506274+0000","last_became_active":"2024-04-25T10:30:33.651140+0000","last_became_peered":"2024-04-25T10:30:33.651140+0000","last_unstale":"2024-04-25T10:30:35.506274+0000","last_undegraded":"2024-04-25T10:30:35.506274+0000","last_fullsized":"2024-04-25T10:30:35.506274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:16.762135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.749776+0000","last_change":"2024-04-25T10:30:32.909982+0000","last_active":"2024-04-25T10:30:34.749776+0000","last_peered":"2024-04-25T10:30:34.749776+0000","last_clean":"2024-04-25T10:30:34.749776+0000","last_became_active":"2024-04-25T10:30:32.909506+0000","last_became_peered":"2024-04-25T10:30:32.909506+0000","last_unstale":"2024-04-25T10:30:34.749776+0000","last_undegraded":"2024-04-25T10:30:34.749776+0000","last_fullsized":"2024-04-25T10:30:34.749776+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:47.396963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.506605+0000","last_change":"2024-04-25T10:30:33.642913+0000","last_active":"2024-04-25T10:30:35.506605+0000","last_peered":"2024-04-25T10:30:35.506605+0000","last_clean":"2024-04-25T10:30:35.506605+0000","last_became_active":"2024-04-25T10:30:33.642614+0000","last_became_peered":"2024-04-25T10:30:33.642614+0000","last_unstale":"2024-04-25T10:30:35.506605+0000","last_undegraded":"2024-04-25T10:30:35.506605+0000","last_fullsized":"2024-04-25T10:30:35.506605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:56.651230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544675+0000","last_change":"2024-04-25T10:30:33.644525+0000","last_active":"2024-04-25T10:30:35.544675+0000","last_peered":"2024-04-25T10:30:35.544675+0000","last_clean":"2024-04-25T10:30:35.544675+0000","last_became_active":"2024-04-25T10:30:33.644188+0000","last_became_peered":"2024-04-25T10:30:33.644188+0000","last_unstale":"2024-04-25T10:30:35.544675+0000","last_undegraded":"2024-04-25T10:30:35.544675+0000","last_fullsized":"2024-04-25T10:30:35.544675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:28.625695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.852405+0000","last_change":"2024-04-25T10:30:33.648912+0000","last_active":"2024-04-25T10:30:34.852405+0000","last_peered":"2024-04-25T10:30:34.852405+0000","last_clean":"2024-04-25T10:30:34.852405+0000","last_became_active":"2024-04-25T10:30:33.648541+0000","last_became_peered":"2024-04-25T10:30:33.648541+0000","last_unstale":"2024-04-25T10:30:34.852405+0000","last_undegraded":"2024-04-25T10:30:34.852405+0000","last_fullsized":"2024-04-25T10:30:34.852405+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:06.303096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.852304+0000","last_change":"2024-04-25T10:30:33.644721+0000","last_active":"2024-04-25T10:30:34.852304+0000","last_peered":"2024-04-25T10:30:34.852304+0000","last_clean":"2024-04-25T10:30:34.852304+0000","last_became_active":"2024-04-25T10:30:33.644392+0000","last_became_peered":"2024-04-25T10:30:33.644392+0000","last_unstale":"2024-04-25T10:30:34.852304+0000","last_undegraded":"2024-04-25T10:30:34.852304+0000","last_fullsized":"2024-04-25T10:30:34.852304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:50.571318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.996470+0000","last_change":"2024-04-25T10:30:33.647970+0000","last_active":"2024-04-25T10:30:34.996470+0000","last_peered":"2024-04-25T10:30:34.996470+0000","last_clean":"2024-04-25T10:30:34.996470+0000","last_became_active":"2024-04-25T10:30:33.647547+0000","last_became_peered":"2024-04-25T10:30:33.647547+0000","last_unstale":"2024-04-25T10:30:34.996470+0000","last_undegraded":"2024-04-25T10:30:34.996470+0000","last_fullsized":"2024-04-25T10:30:34.996470+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:25.609085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.749746+0000","last_change":"2024-04-25T10:30:33.641618+0000","last_active":"2024-04-25T10:30:34.749746+0000","last_peered":"2024-04-25T10:30:34.749746+0000","last_clean":"2024-04-25T10:30:34.749746+0000","last_became_active":"2024-04-25T10:30:33.640830+0000","last_became_peered":"2024-04-25T10:30:33.640830+0000","last_unstale":"2024-04-25T10:30:34.749746+0000","last_undegraded":"2024-04-25T10:30:34.749746+0000","last_fullsized":"2024-04-25T10:30:34.749746+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:36.870127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.545059+0000","last_change":"2024-04-25T10:30:33.644010+0000","last_active":"2024-04-25T10:30:35.545059+0000","last_peered":"2024-04-25T10:30:35.545059+0000","last_clean":"2024-04-25T10:30:35.545059+0000","last_became_active":"2024-04-25T10:30:33.643639+0000","last_became_peered":"2024-04-25T10:30:33.643639+0000","last_unstale":"2024-04-25T10:30:35.545059+0000","last_undegraded":"2024-04-25T10:30:35.545059+0000","last_fullsized":"2024-04-25T10:30:35.545059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:32.820552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.852833+0000","last_change":"2024-04-25T10:30:33.639727+0000","last_active":"2024-04-25T10:30:34.852833+0000","last_peered":"2024-04-25T10:30:34.852833+0000","last_clean":"2024-04-25T10:30:34.852833+0000","last_became_active":"2024-04-25T10:30:33.639185+0000","last_became_peered":"2024-04-25T10:30:33.639185+0000","last_unstale":"2024-04-25T10:30:34.852833+0000","last_undegraded":"2024-04-25T10:30:34.852833+0000","last_fullsized":"2024-04-25T10:30:34.852833+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:17.860864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544873+0000","last_change":"2024-04-25T10:30:33.644650+0000","last_active":"2024-04-25T10:30:35.544873+0000","last_peered":"2024-04-25T10:30:35.544873+0000","last_clean":"2024-04-25T10:30:35.544873+0000","last_became_active":"2024-04-25T10:30:33.644264+0000","last_became_peered":"2024-04-25T10:30:33.644264+0000","last_unstale":"2024-04-25T10:30:35.544873+0000","last_undegraded":"2024-04-25T10:30:35.544873+0000","last_fullsized":"2024-04-25T10:30:35.544873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:41.122318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544554+0000","last_change":"2024-04-25T10:30:33.651875+0000","last_active":"2024-04-25T10:30:35.544554+0000","last_peered":"2024-04-25T10:30:35.544554+0000","last_clean":"2024-04-25T10:30:35.544554+0000","last_became_active":"2024-04-25T10:30:33.651486+0000","last_became_peered":"2024-04-25T10:30:33.651486+0000","last_unstale":"2024-04-25T10:30:35.544554+0000","last_undegraded":"2024-04-25T10:30:35.544554+0000","last_fullsized":"2024-04-25T10:30:35.544554+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:32.598209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544464+0000","last_change":"2024-04-25T10:30:33.638728+0000","last_active":"2024-04-25T10:30:35.544464+0000","last_peered":"2024-04-25T10:30:35.544464+0000","last_clean":"2024-04-25T10:30:35.544464+0000","last_became_active":"2024-04-25T10:30:33.638200+0000","last_became_peered":"2024-04-25T10:30:33.638200+0000","last_unstale":"2024-04-25T10:30:35.544464+0000","last_undegraded":"2024-04-25T10:30:35.544464+0000","last_fullsized":"2024-04-25T10:30:35.544464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:19.178542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544846+0000","last_change":"2024-04-25T10:30:33.639915+0000","last_active":"2024-04-25T10:30:35.544846+0000","last_peered":"2024-04-25T10:30:35.544846+0000","last_clean":"2024-04-25T10:30:35.544846+0000","last_became_active":"2024-04-25T10:30:33.639137+0000","last_became_peered":"2024-04-25T10:30:33.639137+0000","last_unstale":"2024-04-25T10:30:35.544846+0000","last_undegraded":"2024-04-25T10:30:35.544846+0000","last_fullsized":"2024-04-25T10:30:35.544846+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:13.143405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.749634+0000","last_change":"2024-04-25T10:30:33.643547+0000","last_active":"2024-04-25T10:30:34.749634+0000","last_peered":"2024-04-25T10:30:34.749634+0000","last_clean":"2024-04-25T10:30:34.749634+0000","last_became_active":"2024-04-25T10:30:33.643158+0000","last_became_peered":"2024-04-25T10:30:33.643158+0000","last_unstale":"2024-04-25T10:30:34.749634+0000","last_undegraded":"2024-04-25T10:30:34.749634+0000","last_fullsized":"2024-04-25T10:30:34.749634+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:07.517132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.544990+0000","last_change":"2024-04-25T10:30:33.645746+0000","last_active":"2024-04-25T10:30:35.544990+0000","last_peered":"2024-04-25T10:30:35.544990+0000","last_clean":"2024-04-25T10:30:35.544990+0000","last_became_active":"2024-04-25T10:30:33.644646+0000","last_became_peered":"2024-04-25T10:30:33.644646+0000","last_unstale":"2024-04-25T10:30:35.544990+0000","last_undegraded":"2024-04-25T10:30:35.544990+0000","last_fullsized":"2024-04-25T10:30:35.544990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:21.591264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":11732,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":22,"num_read_kb":19,"num_write":123,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":42151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645327,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":827392,"data_stored":396357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841555,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36764,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708388,"statfs":{"total":95995035648,"available":95957389312,"internally_reserved":0,"allocated":1462272,"data_stored":989333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037784,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36156,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708996,"statfs":{"total":95995035648,"available":95958011904,"internally_reserved":0,"allocated":839680,"data_stored":396559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234013,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":835584,"data_stored":396401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430243,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32580,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712572,"statfs":{"total":95995035648,"available":95961673728,"internally_reserved":0,"allocated":1433600,"data_stored":987935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626470,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36144,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709008,"statfs":{"total":95995035648,"available":95958024192,"internally_reserved":0,"allocated":827392,"data_stored":395865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":839680,"data_stored":403987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40328,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704824,"statfs":{"total":95995035648,"available":95953739776,"internally_reserved":0,"allocated":851968,"data_stored":397253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":860160,"data_stored":398737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36152,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709000,"statfs":{"total":95995035648,"available":95958016000,"internally_reserved":0,"allocated":831488,"data_stored":395879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986225,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":827392,"data_stored":395797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822699,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36748,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708404,"statfs":{"total":95995035648,"available":95957405696,"internally_reserved":0,"allocated":1445888,"data_stored":987619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":20895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":10181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:30:42.089 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:30:42.089 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T10:30:42.090 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T10:30:42.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:41 smithi077 ceph-mon[32772]: pgmap v281: 97 pgs: 97 active+clean; 588 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6.2 KiB/s wr, 19 op/s 2024-04-25T10:30:42.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:41 smithi077 ceph-mon[32772]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:30:42.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:41 smithi077 ceph-mon[32772]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:30:42.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:41 smithi077 ceph-mon[32772]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T10:30:42.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:41 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/632097809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:42.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:41 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2906119318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:30:42.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:41 smithi164 ceph-mon[32713]: pgmap v281: 97 pgs: 97 active+clean; 588 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6.2 KiB/s wr, 19 op/s 2024-04-25T10:30:42.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:41 smithi164 ceph-mon[32713]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:30:42.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:41 smithi164 ceph-mon[32713]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:30:42.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:41 smithi164 ceph-mon[32713]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T10:30:42.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:41 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/632097809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:30:42.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:41 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2906119318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:30:42.370 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:30:43.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:42 smithi043 ceph-mon[29478]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:30:43.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:42 smithi077 ceph-mon[32772]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:30:43.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:42 smithi164 ceph-mon[32713]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:30:43.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.983+0000 7f9186150700 1 -- 172.21.15.43:0/95112389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9180076540 msgr2=0x7f9180106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:43.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.983+0000 7f9186150700 1 --2- 172.21.15.43:0/95112389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9180076540 0x7f9180106d90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9170009990 tx=0x7f9170009ca0 comp rx=0 tx=0).stop 2024-04-25T10:30:43.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.983+0000 7f9186150700 1 -- 172.21.15.43:0/95112389 shutdown_connections 2024-04-25T10:30:43.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.983+0000 7f9186150700 1 --2- 172.21.15.43:0/95112389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9180109e90 0x7f918010c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:43.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.983+0000 7f9186150700 1 --2- 172.21.15.43:0/95112389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9180076540 0x7f9180106d90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:43.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.983+0000 7f9186150700 1 --2- 172.21.15.43:0/95112389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180107360 0x7f9180109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:43.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.983+0000 7f9186150700 1 -- 172.21.15.43:0/95112389 >> 172.21.15.43:0/95112389 conn(0x7f91800fe740 msgr2=0x7f9180100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:43.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.983+0000 7f9186150700 1 -- 172.21.15.43:0/95112389 shutdown_connections 2024-04-25T10:30:43.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.983+0000 7f9186150700 1 -- 172.21.15.43:0/95112389 wait complete. 2024-04-25T10:30:43.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:43 smithi043 ceph-mon[29478]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:30:43.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:43 smithi043 ceph-mon[29478]: pgmap v282: 97 pgs: 97 active+clean; 588 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5.4 KiB/s wr, 17 op/s 2024-04-25T10:30:43.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.984+0000 7f9186150700 1 Processor -- start 2024-04-25T10:30:43.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.984+0000 7f9186150700 1 -- start start 2024-04-25T10:30:43.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.985+0000 7f9186150700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180076540 0x7f918019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:43.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.985+0000 7f9186150700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9180107360 0x7f918019acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:43.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.985+0000 7f9186150700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9180109e90 0x7f91801a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:43.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.985+0000 7f9186150700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f918010fc20 con 0x7f9180109e90 2024-04-25T10:30:43.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.985+0000 7f917effd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9180107360 0x7f918019acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:43.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.985+0000 7f9186150700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f918010faa0 con 0x7f9180076540 2024-04-25T10:30:43.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.985+0000 7f9186150700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f918010fda0 con 0x7f9180107360 2024-04-25T10:30:43.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.985+0000 7f917effd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9180107360 0x7f918019acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:56658/0 (socket says 172.21.15.43:56658) 2024-04-25T10:30:43.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.985+0000 7f917effd700 1 -- 172.21.15.43:0/1148123009 learned_addr learned my addr 172.21.15.43:0/1148123009 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:43.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.985+0000 7f917ffff700 1 --2- 172.21.15.43:0/1148123009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9180109e90 0x7f91801a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:43.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.985+0000 7f917f7fe700 1 --2- 172.21.15.43:0/1148123009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180076540 0x7f918019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:43.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.985+0000 7f917ffff700 1 -- 172.21.15.43:0/1148123009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9180107360 msgr2=0x7f918019acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:43.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.985+0000 7f917ffff700 1 --2- 172.21.15.43:0/1148123009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9180107360 0x7f918019acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:43.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.985+0000 7f917ffff700 1 -- 172.21.15.43:0/1148123009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180076540 msgr2=0x7f918019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:43.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.985+0000 7f917ffff700 1 --2- 172.21.15.43:0/1148123009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180076540 0x7f918019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:43.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.985+0000 7f917ffff700 1 -- 172.21.15.43:0/1148123009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91801a2260 con 0x7f9180109e90 2024-04-25T10:30:43.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.985+0000 7f917effd700 1 --2- 172.21.15.43:0/1148123009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9180107360 0x7f918019acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:43.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.986+0000 7f917ffff700 1 --2- 172.21.15.43:0/1148123009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9180109e90 0x7f91801a1c70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9178007e90 tx=0x7f917800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:43.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.986+0000 7f917cff9700 1 -- 172.21.15.43:0/1148123009 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178009e20 con 0x7f9180109e90 2024-04-25T10:30:43.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.986+0000 7f917cff9700 1 -- 172.21.15.43:0/1148123009 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f917800fc40 con 0x7f9180109e90 2024-04-25T10:30:43.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.986+0000 7f9186150700 1 -- 172.21.15.43:0/1148123009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9180102bd0 con 0x7f9180109e90 2024-04-25T10:30:43.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.986+0000 7f917cff9700 1 -- 172.21.15.43:0/1148123009 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178005020 con 0x7f9180109e90 2024-04-25T10:30:43.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.986+0000 7f9186150700 1 -- 172.21.15.43:0/1148123009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91801030c0 con 0x7f9180109e90 2024-04-25T10:30:43.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.987+0000 7f9186150700 1 -- 172.21.15.43:0/1148123009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9180051190 con 0x7f9180109e90 2024-04-25T10:30:43.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.988+0000 7f917cff9700 1 -- 172.21.15.43:0/1148123009 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f917801a420 con 0x7f9180109e90 2024-04-25T10:30:43.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.989+0000 7f917cff9700 1 --2- 172.21.15.43:0/1148123009 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f91680afc10 0x7f91680b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:43.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.989+0000 7f917f7fe700 1 --2- 172.21.15.43:0/1148123009 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f91680afc10 0x7f91680b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:43.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.989+0000 7f917cff9700 1 -- 172.21.15.43:0/1148123009 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f91780e2450 con 0x7f9180109e90 2024-04-25T10:30:43.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.989+0000 7f917f7fe700 1 --2- 172.21.15.43:0/1148123009 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f91680afc10 0x7f91680b2120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f917000b5c0 tx=0x7f917001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:43.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:43.994+0000 7f917cff9700 1 -- 172.21.15.43:0/1148123009 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f91780ab080 con 0x7f9180109e90 2024-04-25T10:30:44.192 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:43 smithi077 ceph-mon[32772]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:30:44.192 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:43 smithi077 ceph-mon[32772]: pgmap v282: 97 pgs: 97 active+clean; 588 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5.4 KiB/s wr, 17 op/s 2024-04-25T10:30:44.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:43 smithi164 ceph-mon[32713]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:30:44.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:43 smithi164 ceph-mon[32713]: pgmap v282: 97 pgs: 97 active+clean; 588 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5.4 KiB/s wr, 17 op/s 2024-04-25T10:30:44.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:44.300+0000 7f9186150700 1 -- 172.21.15.43:0/1148123009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f91801044a0 con 0x7f9180109e90 2024-04-25T10:30:44.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:44.303+0000 7f917cff9700 1 -- 172.21.15.43:0/1148123009 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f9178091e20 con 0x7f9180109e90 2024-04-25T10:30:44.305 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-04-25T10:30:44.305 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQDUMCpm9+r5ERAAEXllawEdjePetApFUNBvdA== 2024-04-25T10:30:44.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:44.306+0000 7f9186150700 1 -- 172.21.15.43:0/1148123009 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f91680afc10 msgr2=0x7f91680b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:44.306+0000 7f9186150700 1 --2- 172.21.15.43:0/1148123009 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f91680afc10 0x7f91680b2120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f917000b5c0 tx=0x7f917001a040 comp rx=0 tx=0).stop 2024-04-25T10:30:44.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:44.306+0000 7f9186150700 1 -- 172.21.15.43:0/1148123009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9180109e90 msgr2=0x7f91801a1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:44.306+0000 7f9186150700 1 --2- 172.21.15.43:0/1148123009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9180109e90 0x7f91801a1c70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9178007e90 tx=0x7f917800f740 comp rx=0 tx=0).stop 2024-04-25T10:30:44.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:44.306+0000 7f9186150700 1 -- 172.21.15.43:0/1148123009 shutdown_connections 2024-04-25T10:30:44.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:44.306+0000 7f9186150700 1 --2- 172.21.15.43:0/1148123009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9180109e90 0x7f91801a1c70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:44.306+0000 7f9186150700 1 --2- 172.21.15.43:0/1148123009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180076540 0x7f918019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:44.306+0000 7f9186150700 1 --2- 172.21.15.43:0/1148123009 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f91680afc10 0x7f91680b2120 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:44.306+0000 7f9186150700 1 --2- 172.21.15.43:0/1148123009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9180107360 0x7f918019acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:44.306+0000 7f9186150700 1 -- 172.21.15.43:0/1148123009 >> 172.21.15.43:0/1148123009 conn(0x7f91800fe740 msgr2=0x7f9180105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:44.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:44.306+0000 7f9186150700 1 -- 172.21.15.43:0/1148123009 shutdown_connections 2024-04-25T10:30:44.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:44.306+0000 7f9186150700 1 -- 172.21.15.43:0/1148123009 wait complete. 2024-04-25T10:30:44.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:44 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1148123009' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T10:30:44.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:44 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1148123009' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T10:30:44.907 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T10:30:44.907 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T10:30:44.907 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T10:30:44.945 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T10:30:44.945 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T10:30:44.945 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T10:30:45.205 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:44 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1148123009' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T10:30:45.205 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:44 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1148123009' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T10:30:45.231 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:30:45.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:44 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1148123009' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T10:30:45.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:44 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1148123009' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T10:30:45.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:45 smithi043 ceph-mon[29478]: pgmap v283: 97 pgs: 97 active+clean; 588 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 4.9 KiB/s wr, 17 op/s 2024-04-25T10:30:46.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:45 smithi077 ceph-mon[32772]: pgmap v283: 97 pgs: 97 active+clean; 588 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 4.9 KiB/s wr, 17 op/s 2024-04-25T10:30:46.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:45 smithi164 ceph-mon[32713]: pgmap v283: 97 pgs: 97 active+clean; 588 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 4.9 KiB/s wr, 17 op/s 2024-04-25T10:30:46.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.834+0000 7f9a67e8a700 1 -- 172.21.15.43:0/3302923038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a601085e0 msgr2=0x7f9a6010ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:46.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.834+0000 7f9a67e8a700 1 --2- 172.21.15.43:0/3302923038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a601085e0 0x7f9a6010ff40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9a5800b0d0 tx=0x7f9a5800b3e0 comp rx=0 tx=0).stop 2024-04-25T10:30:46.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.834+0000 7f9a67e8a700 1 -- 172.21.15.43:0/3302923038 shutdown_connections 2024-04-25T10:30:46.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.834+0000 7f9a67e8a700 1 --2- 172.21.15.43:0/3302923038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a601085e0 0x7f9a6010ff40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:46.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.834+0000 7f9a67e8a700 1 --2- 172.21.15.43:0/3302923038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a60103360 0x7f9a60103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:46.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.834+0000 7f9a67e8a700 1 --2- 172.21.15.43:0/3302923038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a60107c90 0x7f9a601080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:46.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.834+0000 7f9a67e8a700 1 -- 172.21.15.43:0/3302923038 >> 172.21.15.43:0/3302923038 conn(0x7f9a600fecb0 msgr2=0x7f9a601010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.834+0000 7f9a67e8a700 1 -- 172.21.15.43:0/3302923038 shutdown_connections 2024-04-25T10:30:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.834+0000 7f9a67e8a700 1 -- 172.21.15.43:0/3302923038 wait complete. 2024-04-25T10:30:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.835+0000 7f9a67e8a700 1 Processor -- start 2024-04-25T10:30:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.835+0000 7f9a67e8a700 1 -- start start 2024-04-25T10:30:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.835+0000 7f9a67e8a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a60103360 0x7f9a6019ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.835+0000 7f9a67e8a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a60107c90 0x7f9a6019b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.835+0000 7f9a67e8a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a601085e0 0x7f9a6019f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.835+0000 7f9a67e8a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a60113260 con 0x7f9a60103360 2024-04-25T10:30:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.835+0000 7f9a67e8a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601130e0 con 0x7f9a601085e0 2024-04-25T10:30:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.835+0000 7f9a67e8a700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601133e0 con 0x7f9a60107c90 2024-04-25T10:30:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.835+0000 7f9a65c26700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a60103360 0x7f9a6019ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.835+0000 7f9a65c26700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a60103360 0x7f9a6019ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39502/0 (socket says 172.21.15.43:39502) 2024-04-25T10:30:46.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.835+0000 7f9a65c26700 1 -- 172.21.15.43:0/3459378323 learned_addr learned my addr 172.21.15.43:0/3459378323 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:46.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.835+0000 7f9a66427700 1 --2- 172.21.15.43:0/3459378323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a601085e0 0x7f9a6019f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:46.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.835+0000 7f9a65425700 1 --2- 172.21.15.43:0/3459378323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a60107c90 0x7f9a6019b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:46.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.835+0000 7f9a65c26700 1 -- 172.21.15.43:0/3459378323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a60107c90 msgr2=0x7f9a6019b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:46.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.835+0000 7f9a65c26700 1 --2- 172.21.15.43:0/3459378323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a60107c90 0x7f9a6019b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:46.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.835+0000 7f9a65c26700 1 -- 172.21.15.43:0/3459378323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a601085e0 msgr2=0x7f9a6019f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:46.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.835+0000 7f9a65c26700 1 --2- 172.21.15.43:0/3459378323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a601085e0 0x7f9a6019f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:46.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.836+0000 7f9a65c26700 1 -- 172.21.15.43:0/3459378323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6019fbf0 con 0x7f9a60103360 2024-04-25T10:30:46.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.836+0000 7f9a65c26700 1 --2- 172.21.15.43:0/3459378323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a60103360 0x7f9a6019ad20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00d980 tx=0x7f9a5c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:46.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.836+0000 7f9a56ffd700 1 -- 172.21.15.43:0/3459378323 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c016070 con 0x7f9a60103360 2024-04-25T10:30:46.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.836+0000 7f9a56ffd700 1 -- 172.21.15.43:0/3459378323 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a5c0044d0 con 0x7f9a60103360 2024-04-25T10:30:46.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.836+0000 7f9a67e8a700 1 -- 172.21.15.43:0/3459378323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a6019fee0 con 0x7f9a60103360 2024-04-25T10:30:46.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.836+0000 7f9a56ffd700 1 -- 172.21.15.43:0/3459378323 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c004d00 con 0x7f9a60103360 2024-04-25T10:30:46.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.836+0000 7f9a67e8a700 1 -- 172.21.15.43:0/3459378323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a601a7800 con 0x7f9a60103360 2024-04-25T10:30:46.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.837+0000 7f9a67e8a700 1 -- 172.21.15.43:0/3459378323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a60051190 con 0x7f9a60103360 2024-04-25T10:30:46.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.838+0000 7f9a56ffd700 1 -- 172.21.15.43:0/3459378323 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f9a5c004020 con 0x7f9a60103360 2024-04-25T10:30:46.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.839+0000 7f9a56ffd700 1 --2- 172.21.15.43:0/3459378323 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9a4c0afc90 0x7f9a4c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:46.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.839+0000 7f9a65425700 1 --2- 172.21.15.43:0/3459378323 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9a4c0afc90 0x7f9a4c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:46.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.839+0000 7f9a56ffd700 1 -- 172.21.15.43:0/3459378323 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f9a5c0defe0 con 0x7f9a60103360 2024-04-25T10:30:46.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.840+0000 7f9a65425700 1 --2- 172.21.15.43:0/3459378323 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9a4c0afc90 0x7f9a4c0b21a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9a50005fd0 tx=0x7f9a50005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:46.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:46.843+0000 7f9a56ffd700 1 -- 172.21.15.43:0/3459378323 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a5c01f020 con 0x7f9a60103360 2024-04-25T10:30:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:47.113+0000 7f9a67e8a700 1 -- 172.21.15.43:0/3459378323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f9a6019c080 con 0x7f9a60103360 2024-04-25T10:30:47.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:47.118+0000 7f9a56ffd700 1 -- 172.21.15.43:0/3459378323 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+284479 (secure 0 0 0) 0x7f9a5c0a7c10 con 0x7f9a60103360 2024-04-25T10:30:47.124 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:47.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:47.121+0000 7f9a67e8a700 1 -- 172.21.15.43:0/3459378323 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9a4c0afc90 msgr2=0x7f9a4c0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:47.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:47.121+0000 7f9a67e8a700 1 --2- 172.21.15.43:0/3459378323 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9a4c0afc90 0x7f9a4c0b21a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9a50005fd0 tx=0x7f9a50005ee0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:47.121+0000 7f9a67e8a700 1 -- 172.21.15.43:0/3459378323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a60103360 msgr2=0x7f9a6019ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:47.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:47.121+0000 7f9a67e8a700 1 --2- 172.21.15.43:0/3459378323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a60103360 0x7f9a6019ad20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00d980 tx=0x7f9a5c00dd40 comp rx=0 tx=0).stop 2024-04-25T10:30:47.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:47.122+0000 7f9a67e8a700 1 -- 172.21.15.43:0/3459378323 shutdown_connections 2024-04-25T10:30:47.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:47.122+0000 7f9a67e8a700 1 --2- 172.21.15.43:0/3459378323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a601085e0 0x7f9a6019f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:47.122+0000 7f9a67e8a700 1 --2- 172.21.15.43:0/3459378323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a60103360 0x7f9a6019ad20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:47.122+0000 7f9a67e8a700 1 --2- 172.21.15.43:0/3459378323 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9a4c0afc90 0x7f9a4c0b21a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:47.122+0000 7f9a67e8a700 1 --2- 172.21.15.43:0/3459378323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a60107c90 0x7f9a6019b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:47.122+0000 7f9a67e8a700 1 -- 172.21.15.43:0/3459378323 >> 172.21.15.43:0/3459378323 conn(0x7f9a600fecb0 msgr2=0x7f9a60075bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:47.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:47.122+0000 7f9a67e8a700 1 -- 172.21.15.43:0/3459378323 shutdown_connections 2024-04-25T10:30:47.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:47.122+0000 7f9a67e8a700 1 -- 172.21.15.43:0/3459378323 wait complete. 2024-04-25T10:30:47.755 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1953858940},{"type":"v1","addr":"172.21.15.43:6801","nonce":1953858940}]},"active_addr":"172.21.15.43:6801/1953858940","active_change":"2024-04-25T10:23:32.072218+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.43:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":188939647}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":1287690800}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":1599315071}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":3026776968}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":2641311961}]}]} 2024-04-25T10:30:47.762 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T10:30:47.762 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T10:30:47.762 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T10:30:48.013 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:30:48.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:47 smithi043 ceph-mon[29478]: pgmap v284: 97 pgs: 97 active+clean; 623 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.3 KiB/s wr, 17 op/s 2024-04-25T10:30:48.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:47 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3459378323' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T10:30:48.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:47 smithi077 ceph-mon[32772]: pgmap v284: 97 pgs: 97 active+clean; 623 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.3 KiB/s wr, 17 op/s 2024-04-25T10:30:48.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:47 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3459378323' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T10:30:48.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:47 smithi164 ceph-mon[32713]: pgmap v284: 97 pgs: 97 active+clean; 623 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.3 KiB/s wr, 17 op/s 2024-04-25T10:30:48.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:47 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3459378323' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T10:30:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.634+0000 7f71e5905700 1 -- 172.21.15.43:0/1289766459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71e0103360 msgr2=0x7f71e0103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.634+0000 7f71e5905700 1 --2- 172.21.15.43:0/1289766459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71e0103360 0x7f71e0103770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f71d00099c0 tx=0x7f71d0009cd0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.634+0000 7f71e5905700 1 -- 172.21.15.43:0/1289766459 shutdown_connections 2024-04-25T10:30:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.634+0000 7f71e5905700 1 --2- 172.21.15.43:0/1289766459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71e01085e0 0x7f71e010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.634+0000 7f71e5905700 1 --2- 172.21.15.43:0/1289766459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71e0103360 0x7f71e0103770 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.634+0000 7f71e5905700 1 --2- 172.21.15.43:0/1289766459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71e0107c90 0x7f71e01080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.634+0000 7f71e5905700 1 -- 172.21.15.43:0/1289766459 >> 172.21.15.43:0/1289766459 conn(0x7f71e00fecb0 msgr2=0x7f71e01010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.634+0000 7f71e5905700 1 -- 172.21.15.43:0/1289766459 shutdown_connections 2024-04-25T10:30:49.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.635+0000 7f71e5905700 1 -- 172.21.15.43:0/1289766459 wait complete. 2024-04-25T10:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.635+0000 7f71e5905700 1 Processor -- start 2024-04-25T10:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.636+0000 7f71e5905700 1 -- start start 2024-04-25T10:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.636+0000 7f71e5905700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71e0103360 0x7f71e019baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.636+0000 7f71e5905700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71e0107c90 0x7f71e019c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.636+0000 7f71e5905700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71e01085e0 0x7f71e019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.636+0000 7f71e5905700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f71e0113210 con 0x7f71e01085e0 2024-04-25T10:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.636+0000 7f71e5905700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f71e0113090 con 0x7f71e0103360 2024-04-25T10:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.636+0000 7f71e5905700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f71e0113390 con 0x7f71e0107c90 2024-04-25T10:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.636+0000 7f71df7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71e01085e0 0x7f71e019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.636+0000 7f71deffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71e0103360 0x7f71e019baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.636+0000 7f71df7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71e01085e0 0x7f71e019c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35220/0 (socket says 172.21.15.43:35220) 2024-04-25T10:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.636+0000 7f71deffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71e0103360 0x7f71e019baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:41216/0 (socket says 172.21.15.43:41216) 2024-04-25T10:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.636+0000 7f71df7fe700 1 -- 172.21.15.43:0/495140164 learned_addr learned my addr 172.21.15.43:0/495140164 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.636+0000 7f71de7fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71e0107c90 0x7f71e019c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.637+0000 7f71df7fe700 1 -- 172.21.15.43:0/495140164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71e0107c90 msgr2=0x7f71e019c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.637+0000 7f71df7fe700 1 --2- 172.21.15.43:0/495140164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71e0107c90 0x7f71e019c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.637+0000 7f71df7fe700 1 -- 172.21.15.43:0/495140164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71e0103360 msgr2=0x7f71e019baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.637+0000 7f71df7fe700 1 --2- 172.21.15.43:0/495140164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71e0103360 0x7f71e019baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.637+0000 7f71df7fe700 1 -- 172.21.15.43:0/495140164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e01a6e40 con 0x7f71e01085e0 2024-04-25T10:30:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.637+0000 7f71deffd700 1 --2- 172.21.15.43:0/495140164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71e0103360 0x7f71e019baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.637+0000 7f71df7fe700 1 --2- 172.21.15.43:0/495140164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71e01085e0 0x7f71e019c590 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f71d8007ee0 tx=0x7f71d800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.637+0000 7f71e4903700 1 -- 172.21.15.43:0/495140164 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8004bb0 con 0x7f71e01085e0 2024-04-25T10:30:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.637+0000 7f71e5905700 1 -- 172.21.15.43:0/495140164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e01a7130 con 0x7f71e01085e0 2024-04-25T10:30:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.637+0000 7f71e4903700 1 -- 172.21.15.43:0/495140164 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d8004d50 con 0x7f71e01085e0 2024-04-25T10:30:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.637+0000 7f71e4903700 1 -- 172.21.15.43:0/495140164 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8005920 con 0x7f71e01085e0 2024-04-25T10:30:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.637+0000 7f71e5905700 1 -- 172.21.15.43:0/495140164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e01a7620 con 0x7f71e01085e0 2024-04-25T10:30:49.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.639+0000 7f71e5905700 1 -- 172.21.15.43:0/495140164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e0051190 con 0x7f71e01085e0 2024-04-25T10:30:49.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.640+0000 7f71e4903700 1 -- 172.21.15.43:0/495140164 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f71d801a020 con 0x7f71e01085e0 2024-04-25T10:30:49.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.641+0000 7f71e4903700 1 --2- 172.21.15.43:0/495140164 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f71c80afc10 0x7f71c80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.641+0000 7f71deffd700 1 --2- 172.21.15.43:0/495140164 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f71c80afc10 0x7f71c80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.641+0000 7f71e4903700 1 -- 172.21.15.43:0/495140164 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f71d80de280 con 0x7f71e01085e0 2024-04-25T10:30:49.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.642+0000 7f71deffd700 1 --2- 172.21.15.43:0/495140164 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f71c80afc10 0x7f71c80b2120 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f71d0000c00 tx=0x7f71d001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:49.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.647+0000 7f71e4903700 1 -- 172.21.15.43:0/495140164 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f71d8010070 con 0x7f71e01085e0 2024-04-25T10:30:49.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.858+0000 7f71e5905700 1 -- 172.21.15.43:0/495140164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71e002e510 con 0x7f71e01085e0 2024-04-25T10:30:49.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.859+0000 7f71e4903700 1 -- 172.21.15.43:0/495140164 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22185 (secure 0 0 0) 0x7f71d80a6eb0 con 0x7f71e01085e0 2024-04-25T10:30:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:49.862 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":77,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:30:34.623807+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:31.503414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:32.224334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:49.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.862+0000 7f71e5905700 1 -- 172.21.15.43:0/495140164 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f71c80afc10 msgr2=0x7f71c80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.862+0000 7f71e5905700 1 --2- 172.21.15.43:0/495140164 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f71c80afc10 0x7f71c80b2120 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f71d0000c00 tx=0x7f71d001a040 comp rx=0 tx=0).stop 2024-04-25T10:30:49.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.862+0000 7f71e5905700 1 -- 172.21.15.43:0/495140164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71e01085e0 msgr2=0x7f71e019c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.862+0000 7f71e5905700 1 --2- 172.21.15.43:0/495140164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71e01085e0 0x7f71e019c590 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f71d8007ee0 tx=0x7f71d800d3b0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.862+0000 7f71e5905700 1 -- 172.21.15.43:0/495140164 shutdown_connections 2024-04-25T10:30:49.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.862+0000 7f71e5905700 1 --2- 172.21.15.43:0/495140164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71e01085e0 0x7f71e019c590 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.862+0000 7f71e5905700 1 --2- 172.21.15.43:0/495140164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71e0103360 0x7f71e019baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.862+0000 7f71e5905700 1 --2- 172.21.15.43:0/495140164 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f71c80afc10 0x7f71c80b2120 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.862+0000 7f71e5905700 1 --2- 172.21.15.43:0/495140164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71e0107c90 0x7f71e019c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.862+0000 7f71e5905700 1 -- 172.21.15.43:0/495140164 >> 172.21.15.43:0/495140164 conn(0x7f71e00fecb0 msgr2=0x7f71e0107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:49.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.862+0000 7f71e5905700 1 -- 172.21.15.43:0/495140164 shutdown_connections 2024-04-25T10:30:49.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:49.862+0000 7f71e5905700 1 -- 172.21.15.43:0/495140164 wait complete. 2024-04-25T10:30:50.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:49 smithi043 ceph-mon[29478]: pgmap v285: 97 pgs: 97 active+clean; 623 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.1 KiB/s wr, 17 op/s 2024-04-25T10:30:50.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:49 smithi077 ceph-mon[32772]: pgmap v285: 97 pgs: 97 active+clean; 623 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.1 KiB/s wr, 17 op/s 2024-04-25T10:30:50.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:49 smithi164 ceph-mon[32713]: pgmap v285: 97 pgs: 97 active+clean; 623 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.1 KiB/s wr, 17 op/s 2024-04-25T10:30:50.509 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T10:30:50.509 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T10:30:50.757 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:30:51.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:50 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/495140164' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:30:51.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:50 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/495140164' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:30:51.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:50 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/495140164' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:30:52.104 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:51 smithi043 ceph-mon[29478]: pgmap v286: 97 pgs: 97 active+clean; 635 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-04-25T10:30:52.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:51 smithi077 ceph-mon[32772]: pgmap v286: 97 pgs: 97 active+clean; 635 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-04-25T10:30:52.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:51 smithi164 ceph-mon[32713]: pgmap v286: 97 pgs: 97 active+clean; 635 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-04-25T10:30:52.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.331+0000 7fd68730d700 1 -- 172.21.15.43:0/487768147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd680103360 msgr2=0x7fd680103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.331+0000 7fd68730d700 1 --2- 172.21.15.43:0/487768147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd680103360 0x7fd680103770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0099c0 tx=0x7fd67c009cd0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.331+0000 7fd68730d700 1 -- 172.21.15.43:0/487768147 shutdown_connections 2024-04-25T10:30:52.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.331+0000 7fd68730d700 1 --2- 172.21.15.43:0/487768147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6801085e0 0x7fd68010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.331+0000 7fd68730d700 1 --2- 172.21.15.43:0/487768147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd680103360 0x7fd680103770 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.331+0000 7fd68730d700 1 --2- 172.21.15.43:0/487768147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd680107c90 0x7fd6801080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.331+0000 7fd68730d700 1 -- 172.21.15.43:0/487768147 >> 172.21.15.43:0/487768147 conn(0x7fd6800fecb0 msgr2=0x7fd6801010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:52.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.331+0000 7fd68730d700 1 -- 172.21.15.43:0/487768147 shutdown_connections 2024-04-25T10:30:52.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.331+0000 7fd68730d700 1 -- 172.21.15.43:0/487768147 wait complete. 2024-04-25T10:30:52.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.332+0000 7fd68730d700 1 Processor -- start 2024-04-25T10:30:52.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.332+0000 7fd68730d700 1 -- start start 2024-04-25T10:30:52.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.332+0000 7fd68730d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd680103360 0x7fd68019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.332+0000 7fd68730d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd680107c90 0x7fd68019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.332+0000 7fd68730d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6801085e0 0x7fd68019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.332+0000 7fd68730d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd680114380 con 0x7fd6801085e0 2024-04-25T10:30:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.332+0000 7fd68730d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd680114200 con 0x7fd680107c90 2024-04-25T10:30:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.332+0000 7fd68730d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd680114500 con 0x7fd680103360 2024-04-25T10:30:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.332+0000 7fd6858aa700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6801085e0 0x7fd68019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.332+0000 7fd6858aa700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6801085e0 0x7fd68019f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35236/0 (socket says 172.21.15.43:35236) 2024-04-25T10:30:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.332+0000 7fd6858aa700 1 -- 172.21.15.43:0/2870707605 learned_addr learned my addr 172.21.15.43:0/2870707605 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:30:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.332+0000 7fd6850a9700 1 --2- 172.21.15.43:0/2870707605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd680103360 0x7fd68019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.332+0000 7fd6848a8700 1 --2- 172.21.15.43:0/2870707605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd680107c90 0x7fd68019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.333+0000 7fd6858aa700 1 -- 172.21.15.43:0/2870707605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd680103360 msgr2=0x7fd68019ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.333+0000 7fd6858aa700 1 --2- 172.21.15.43:0/2870707605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd680103360 0x7fd68019ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.333+0000 7fd6858aa700 1 -- 172.21.15.43:0/2870707605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd680107c90 msgr2=0x7fd68019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.333+0000 7fd6858aa700 1 --2- 172.21.15.43:0/2870707605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd680107c90 0x7fd68019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.333+0000 7fd6858aa700 1 -- 172.21.15.43:0/2870707605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68019fbe0 con 0x7fd6801085e0 2024-04-25T10:30:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.333+0000 7fd6850a9700 1 --2- 172.21.15.43:0/2870707605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd680103360 0x7fd68019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.333+0000 7fd6848a8700 1 --2- 172.21.15.43:0/2870707605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd680107c90 0x7fd68019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.333+0000 7fd6858aa700 1 --2- 172.21.15.43:0/2870707605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6801085e0 0x7fd68019f540 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd67800b970 tx=0x7fd67800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.333+0000 7fd6767fc700 1 -- 172.21.15.43:0/2870707605 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678019070 con 0x7fd6801085e0 2024-04-25T10:30:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.333+0000 7fd6767fc700 1 -- 172.21.15.43:0/2870707605 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd678014440 con 0x7fd6801085e0 2024-04-25T10:30:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.333+0000 7fd68730d700 1 -- 172.21.15.43:0/2870707605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68019fed0 con 0x7fd6801085e0 2024-04-25T10:30:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.333+0000 7fd6767fc700 1 -- 172.21.15.43:0/2870707605 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67800ce60 con 0x7fd6801085e0 2024-04-25T10:30:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.333+0000 7fd68730d700 1 -- 172.21.15.43:0/2870707605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6801a77f0 con 0x7fd6801085e0 2024-04-25T10:30:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.334+0000 7fd68730d700 1 -- 172.21.15.43:0/2870707605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd680051190 con 0x7fd6801085e0 2024-04-25T10:30:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.336+0000 7fd6767fc700 1 -- 172.21.15.43:0/2870707605 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fd67801c070 con 0x7fd6801085e0 2024-04-25T10:30:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.336+0000 7fd6767fc700 1 --2- 172.21.15.43:0/2870707605 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd66c0afc10 0x7fd66c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.336+0000 7fd6850a9700 1 --2- 172.21.15.43:0/2870707605 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd66c0afc10 0x7fd66c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.336+0000 7fd6767fc700 1 -- 172.21.15.43:0/2870707605 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fd6780de3d0 con 0x7fd6801085e0 2024-04-25T10:30:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.337+0000 7fd6850a9700 1 --2- 172.21.15.43:0/2870707605 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd66c0afc10 0x7fd66c0b2120 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd67c000c00 tx=0x7fd67c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:52.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.340+0000 7fd6767fc700 1 -- 172.21.15.43:0/2870707605 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6780a7110 con 0x7fd6801085e0 2024-04-25T10:30:52.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.549+0000 7fd68730d700 1 -- 172.21.15.43:0/2870707605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd68019c070 con 0x7fd6801085e0 2024-04-25T10:30:52.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.551+0000 7fd6767fc700 1 -- 172.21.15.43:0/2870707605 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22185 (secure 0 0 0) 0x7fd6780a6a10 con 0x7fd6801085e0 2024-04-25T10:30:52.553 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:30:52.554 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":77,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:30:34.623807+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:31.503414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:32.224334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:52.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.553+0000 7fd68730d700 1 -- 172.21.15.43:0/2870707605 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd66c0afc10 msgr2=0x7fd66c0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.553+0000 7fd68730d700 1 --2- 172.21.15.43:0/2870707605 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd66c0afc10 0x7fd66c0b2120 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd67c000c00 tx=0x7fd67c01a040 comp rx=0 tx=0).stop 2024-04-25T10:30:52.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.553+0000 7fd68730d700 1 -- 172.21.15.43:0/2870707605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6801085e0 msgr2=0x7fd68019f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.553+0000 7fd68730d700 1 --2- 172.21.15.43:0/2870707605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6801085e0 0x7fd68019f540 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd67800b970 tx=0x7fd67800bd30 comp rx=0 tx=0).stop 2024-04-25T10:30:52.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.553+0000 7fd68730d700 1 -- 172.21.15.43:0/2870707605 shutdown_connections 2024-04-25T10:30:52.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.553+0000 7fd68730d700 1 --2- 172.21.15.43:0/2870707605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6801085e0 0x7fd68019f540 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.553+0000 7fd68730d700 1 --2- 172.21.15.43:0/2870707605 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd66c0afc10 0x7fd66c0b2120 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.553+0000 7fd68730d700 1 --2- 172.21.15.43:0/2870707605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd680103360 0x7fd68019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.553+0000 7fd68730d700 1 --2- 172.21.15.43:0/2870707605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd680107c90 0x7fd68019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.553+0000 7fd68730d700 1 -- 172.21.15.43:0/2870707605 >> 172.21.15.43:0/2870707605 conn(0x7fd6800fecb0 msgr2=0x7fd680107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:52.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.553+0000 7fd68730d700 1 -- 172.21.15.43:0/2870707605 shutdown_connections 2024-04-25T10:30:52.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:30:52.553+0000 7fd68730d700 1 -- 172.21.15.43:0/2870707605 wait complete. 2024-04-25T10:30:53.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:52 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2870707605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:30:53.201 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T10:30:53.202 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T10:30:53.202 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T10:30:53.202 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T10:30:53.203 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T10:30:53.203 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T10:30:53.204 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T10:30:53.204 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T10:30:53.204 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T10:30:53.205 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T10:30:53.205 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T10:30:53.206 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T10:30:53.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:52 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2870707605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:30:53.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:52 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2870707605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:30:53.956 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:53 smithi043 ceph-mon[29478]: pgmap v287: 97 pgs: 97 active+clean; 635 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-04-25T10:30:53.980 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:30:53.981 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:30:53.985 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:30:53.989 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:30:54.030 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:30:54.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:53 smithi077 ceph-mon[32772]: pgmap v287: 97 pgs: 97 active+clean; 635 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-04-25T10:30:54.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:53 smithi164 ceph-mon[32713]: pgmap v287: 97 pgs: 97 active+clean; 635 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-04-25T10:30:56.114 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:55 smithi043 ceph-mon[29478]: pgmap v288: 97 pgs: 97 active+clean; 635 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-04-25T10:30:56.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:55 smithi077 ceph-mon[32772]: pgmap v288: 97 pgs: 97 active+clean; 635 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-04-25T10:30:56.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:55 smithi164 ceph-mon[32713]: pgmap v288: 97 pgs: 97 active+clean; 635 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-04-25T10:30:56.903 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:30:56.903 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:30:56.903 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:30:57.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:57 smithi043 ceph-mon[29478]: pgmap v289: 97 pgs: 97 active+clean; 635 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-04-25T10:30:58.032 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:30:58.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:57 smithi077 ceph-mon[32772]: pgmap v289: 97 pgs: 97 active+clean; 635 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-04-25T10:30:58.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:57 smithi164 ceph-mon[32713]: pgmap v289: 97 pgs: 97 active+clean; 635 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-04-25T10:30:59.006 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:30:59.006 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:30:59.007 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:31:00.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:30:59 smithi043 ceph-mon[29478]: pgmap v290: 97 pgs: 97 active+clean; 635 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-25T10:31:00.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:30:59 smithi077 ceph-mon[32772]: pgmap v290: 97 pgs: 97 active+clean; 635 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-25T10:31:00.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:30:59 smithi164 ceph-mon[32713]: pgmap v290: 97 pgs: 97 active+clean; 635 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-25T10:31:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.689+0000 7f5d42382700 1 -- 172.21.15.43:0/1491193363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d3c105cb0 msgr2=0x7f5d3c10e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.689+0000 7f5d42382700 1 --2- 172.21.15.43:0/1491193363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d3c105cb0 0x7f5d3c10e200 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009990 tx=0x7f5d2c009ca0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.689+0000 7f5d42382700 1 -- 172.21.15.43:0/1491193363 shutdown_connections 2024-04-25T10:31:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.689+0000 7f5d42382700 1 --2- 172.21.15.43:0/1491193363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d3c10e790 0x7f5d3c110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.689+0000 7f5d42382700 1 --2- 172.21.15.43:0/1491193363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d3c105360 0x7f5d3c105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.689+0000 7f5d42382700 1 --2- 172.21.15.43:0/1491193363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d3c105cb0 0x7f5d3c10e200 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.689+0000 7f5d42382700 1 -- 172.21.15.43:0/1491193363 >> 172.21.15.43:0/1491193363 conn(0x7f5d3c0fecb0 msgr2=0x7f5d3c1010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.689+0000 7f5d42382700 1 -- 172.21.15.43:0/1491193363 shutdown_connections 2024-04-25T10:31:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.689+0000 7f5d42382700 1 -- 172.21.15.43:0/1491193363 wait complete. 2024-04-25T10:31:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.690+0000 7f5d42382700 1 Processor -- start 2024-04-25T10:31:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.690+0000 7f5d42382700 1 -- start start 2024-04-25T10:31:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.690+0000 7f5d42382700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d3c105360 0x7f5d3c1a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.690+0000 7f5d42382700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d3c105cb0 0x7f5d3c1a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.690+0000 7f5d42382700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d3c10e790 0x7f5d3c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.690+0000 7f5d42382700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1142f0 con 0x7f5d3c105360 2024-04-25T10:31:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.690+0000 7f5d42382700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c114170 con 0x7f5d3c105cb0 2024-04-25T10:31:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.690+0000 7f5d42382700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c114470 con 0x7f5d3c10e790 2024-04-25T10:31:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.690+0000 7f5d3bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d3c105360 0x7f5d3c1a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.690+0000 7f5d3bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d3c105360 0x7f5d3c1a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37672/0 (socket says 172.21.15.43:37672) 2024-04-25T10:31:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.690+0000 7f5d3bfff700 1 -- 172.21.15.43:0/4184708873 learned_addr learned my addr 172.21.15.43:0/4184708873 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.690+0000 7f5d3b7fe700 1 --2- 172.21.15.43:0/4184708873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d3c105cb0 0x7f5d3c1a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.691+0000 7f5d4091f700 1 --2- 172.21.15.43:0/4184708873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d3c10e790 0x7f5d3c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.691+0000 7f5d3bfff700 1 -- 172.21.15.43:0/4184708873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d3c10e790 msgr2=0x7f5d3c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.691+0000 7f5d3bfff700 1 --2- 172.21.15.43:0/4184708873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d3c10e790 0x7f5d3c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.691+0000 7f5d3bfff700 1 -- 172.21.15.43:0/4184708873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d3c105cb0 msgr2=0x7f5d3c1a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.691+0000 7f5d3bfff700 1 --2- 172.21.15.43:0/4184708873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d3c105cb0 0x7f5d3c1a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.691+0000 7f5d3bfff700 1 -- 172.21.15.43:0/4184708873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c19b140 con 0x7f5d3c105360 2024-04-25T10:31:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.691+0000 7f5d4091f700 1 --2- 172.21.15.43:0/4184708873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d3c10e790 0x7f5d3c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.691+0000 7f5d3bfff700 1 --2- 172.21.15.43:0/4184708873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d3c105360 0x7f5d3c1a0890 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400c9b0 tx=0x7f5d2400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.691+0000 7f5d397fa700 1 -- 172.21.15.43:0/4184708873 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24007b00 con 0x7f5d3c105360 2024-04-25T10:31:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.691+0000 7f5d42382700 1 -- 172.21.15.43:0/4184708873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3c19b430 con 0x7f5d3c105360 2024-04-25T10:31:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.691+0000 7f5d397fa700 1 -- 172.21.15.43:0/4184708873 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d24007ca0 con 0x7f5d3c105360 2024-04-25T10:31:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.691+0000 7f5d42382700 1 -- 172.21.15.43:0/4184708873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3c1a7740 con 0x7f5d3c105360 2024-04-25T10:31:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.691+0000 7f5d397fa700 1 -- 172.21.15.43:0/4184708873 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24018660 con 0x7f5d3c105360 2024-04-25T10:31:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.692+0000 7f5d42382700 1 -- 172.21.15.43:0/4184708873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5d3c063e20 con 0x7f5d3c105360 2024-04-25T10:31:01.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.694+0000 7f5d397fa700 1 -- 172.21.15.43:0/4184708873 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f5d2400f450 con 0x7f5d3c105360 2024-04-25T10:31:01.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.694+0000 7f5d397fa700 1 --2- 172.21.15.43:0/4184708873 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5d280afc10 0x7f5d280b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.694+0000 7f5d3b7fe700 1 --2- 172.21.15.43:0/4184708873 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5d280afc10 0x7f5d280b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.694+0000 7f5d397fa700 1 -- 172.21.15.43:0/4184708873 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f5d240df120 con 0x7f5d3c105360 2024-04-25T10:31:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.695+0000 7f5d397fa700 1 --2- 172.21.15.43:0/4184708873 >> [v2:172.21.15.164:6808/853209434,v1:172.21.15.164:6809/853209434] conn(0x7f5d280b9440 0x7f5d280bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.695+0000 7f5d397fa700 1 -- 172.21.15.43:0/4184708873 --> [v2:172.21.15.164:6808/853209434,v1:172.21.15.164:6809/853209434] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d24015e70 con 0x7f5d280b9440 2024-04-25T10:31:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.695+0000 7f5d397fa700 1 -- 172.21.15.43:0/4184708873 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f5d240df4d0 con 0x7f5d3c105360 2024-04-25T10:31:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.695+0000 7f5d3b7fe700 1 --2- 172.21.15.43:0/4184708873 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5d280afc10 0x7f5d280b2120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c000c00 tx=0x7f5d2c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.695+0000 7f5d4091f700 1 --2- 172.21.15.43:0/4184708873 >> [v2:172.21.15.164:6808/853209434,v1:172.21.15.164:6809/853209434] conn(0x7f5d280b9440 0x7f5d280bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.695+0000 7f5d4091f700 1 --2- 172.21.15.43:0/4184708873 >> [v2:172.21.15.164:6808/853209434,v1:172.21.15.164:6809/853209434] conn(0x7f5d280b9440 0x7f5d280bb890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.696+0000 7f5d397fa700 1 -- 172.21.15.43:0/4184708873 <== osd.5 v2:172.21.15.164:6808/853209434 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d24015e70 con 0x7f5d280b9440 2024-04-25T10:31:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.713+0000 7f5d42382700 1 -- 172.21.15.43:0/4184708873 --> [v2:172.21.15.164:6808/853209434,v1:172.21.15.164:6809/853209434] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5d3c0642a0 con 0x7f5d280b9440 2024-04-25T10:31:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.713+0000 7f5d397fa700 1 -- 172.21.15.43:0/4184708873 <== osd.5 v2:172.21.15.164:6808/853209434 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5d3c0642a0 con 0x7f5d280b9440 2024-04-25T10:31:01.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.714+0000 7f5d42382700 1 -- 172.21.15.43:0/4184708873 >> [v2:172.21.15.164:6808/853209434,v1:172.21.15.164:6809/853209434] conn(0x7f5d280b9440 msgr2=0x7f5d280bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.714+0000 7f5d42382700 1 --2- 172.21.15.43:0/4184708873 >> [v2:172.21.15.164:6808/853209434,v1:172.21.15.164:6809/853209434] conn(0x7f5d280b9440 0x7f5d280bb890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.714+0000 7f5d42382700 1 -- 172.21.15.43:0/4184708873 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5d280afc10 msgr2=0x7f5d280b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.714+0000 7f5d42382700 1 --2- 172.21.15.43:0/4184708873 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5d280afc10 0x7f5d280b2120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c000c00 tx=0x7f5d2c005960 comp rx=0 tx=0).stop 2024-04-25T10:31:01.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.714+0000 7f5d42382700 1 -- 172.21.15.43:0/4184708873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d3c105360 msgr2=0x7f5d3c1a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.714+0000 7f5d42382700 1 --2- 172.21.15.43:0/4184708873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d3c105360 0x7f5d3c1a0890 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400c9b0 tx=0x7f5d2400ccc0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.714+0000 7f5d4091f700 1 -- 172.21.15.43:0/4184708873 reap_dead start 2024-04-25T10:31:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.714+0000 7f5d42382700 1 -- 172.21.15.43:0/4184708873 shutdown_connections 2024-04-25T10:31:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.714+0000 7f5d42382700 1 -- 172.21.15.43:0/4184708873 >> 172.21.15.43:0/4184708873 conn(0x7f5d3c0fecb0 msgr2=0x7f5d3c0ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.714+0000 7f5d42382700 1 -- 172.21.15.43:0/4184708873 shutdown_connections 2024-04-25T10:31:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.714+0000 7f5d42382700 1 -- 172.21.15.43:0/4184708873 wait complete. 2024-04-25T10:31:01.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.857+0000 7f0ca099f700 1 -- 172.21.15.43:0/1952997601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c9c109730 msgr2=0x7f0c9c111090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.857+0000 7f0ca099f700 1 --2- 172.21.15.43:0/1952997601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c9c109730 0x7f0c9c111090 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400b0d0 tx=0x7f0c9400b3e0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.858+0000 7f0ca099f700 1 -- 172.21.15.43:0/1952997601 shutdown_connections 2024-04-25T10:31:01.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.858+0000 7f0ca099f700 1 --2- 172.21.15.43:0/1952997601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c9c109730 0x7f0c9c111090 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.858+0000 7f0ca099f700 1 --2- 172.21.15.43:0/1952997601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c9c1029c0 0x7f0c9c102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.858+0000 7f0ca099f700 1 --2- 172.21.15.43:0/1952997601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c9c108de0 0x7f0c9c1091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.858+0000 7f0ca099f700 1 -- 172.21.15.43:0/1952997601 >> 172.21.15.43:0/1952997601 conn(0x7f0c9c0fe3e0 msgr2=0x7f0c9c100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:01.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.858+0000 7f0ca099f700 1 -- 172.21.15.43:0/1952997601 shutdown_connections 2024-04-25T10:31:01.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.858+0000 7f0ca099f700 1 -- 172.21.15.43:0/1952997601 wait complete. 2024-04-25T10:31:01.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.859+0000 7f0ca099f700 1 Processor -- start 2024-04-25T10:31:01.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.859+0000 7f0ca099f700 1 -- start start 2024-04-25T10:31:01.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.859+0000 7f0ca099f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c9c1029c0 0x7f0c9c1a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.859+0000 7f0ca099f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c9c108de0 0x7f0c9c1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.859+0000 7f0ca099f700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c9c109730 0x7f0c9c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.859+0000 7f0ca099f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9c114370 con 0x7f0c9c108de0 2024-04-25T10:31:01.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.859+0000 7f0ca099f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9c1141f0 con 0x7f0c9c1029c0 2024-04-25T10:31:01.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.859+0000 7f0ca099f700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9c1144f0 con 0x7f0c9c109730 2024-04-25T10:31:01.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.859+0000 7f0c99d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c9c108de0 0x7f0c9c1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.859+0000 7f0c99d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c9c108de0 0x7f0c9c1a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37692/0 (socket says 172.21.15.43:37692) 2024-04-25T10:31:01.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.859+0000 7f0c9a59c700 1 --2- 172.21.15.43:0/2359274104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c9c1029c0 0x7f0c9c1a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.859+0000 7f0c99d9b700 1 -- 172.21.15.43:0/2359274104 learned_addr learned my addr 172.21.15.43:0/2359274104 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:01.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.859+0000 7f0c9ad9d700 1 --2- 172.21.15.43:0/2359274104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c9c109730 0x7f0c9c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.860+0000 7f0c9a59c700 1 -- 172.21.15.43:0/2359274104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c9c109730 msgr2=0x7f0c9c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.860+0000 7f0c9a59c700 1 --2- 172.21.15.43:0/2359274104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c9c109730 0x7f0c9c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.860+0000 7f0c9a59c700 1 -- 172.21.15.43:0/2359274104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c9c108de0 msgr2=0x7f0c9c1a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.860+0000 7f0c9a59c700 1 --2- 172.21.15.43:0/2359274104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c9c108de0 0x7f0c9c1a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.860+0000 7f0c9a59c700 1 -- 172.21.15.43:0/2359274104 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c9c19b070 con 0x7f0c9c1029c0 2024-04-25T10:31:01.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.860+0000 7f0c99d9b700 1 --2- 172.21.15.43:0/2359274104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c9c108de0 0x7f0c9c1a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:01.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.860+0000 7f0c9a59c700 1 --2- 172.21.15.43:0/2359274104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c9c1029c0 0x7f0c9c1a0810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00d740 tx=0x7f0c8c00da50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:01.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.867+0000 7f0c8b7fe700 1 -- 172.21.15.43:0/2359274104 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c016070 con 0x7f0c9c1029c0 2024-04-25T10:31:01.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.867+0000 7f0c8b7fe700 1 -- 172.21.15.43:0/2359274104 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c8c00f690 con 0x7f0c9c1029c0 2024-04-25T10:31:01.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.867+0000 7f0c8b7fe700 1 -- 172.21.15.43:0/2359274104 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c003b50 con 0x7f0c9c1029c0 2024-04-25T10:31:01.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.867+0000 7f0ca099f700 1 -- 172.21.15.43:0/2359274104 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c9c19b360 con 0x7f0c9c1029c0 2024-04-25T10:31:01.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.868+0000 7f0ca099f700 1 -- 172.21.15.43:0/2359274104 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c9c19b870 con 0x7f0c9c1029c0 2024-04-25T10:31:01.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.868+0000 7f0ca099f700 1 -- 172.21.15.43:0/2359274104 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0c9c063e20 con 0x7f0c9c1029c0 2024-04-25T10:31:01.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.870+0000 7f0c8b7fe700 1 -- 172.21.15.43:0/2359274104 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f0c8c003cf0 con 0x7f0c9c1029c0 2024-04-25T10:31:01.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.871+0000 7f0c8b7fe700 1 --2- 172.21.15.43:0/2359274104 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f0c840afb60 0x7f0c840b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.871+0000 7f0c99d9b700 1 --2- 172.21.15.43:0/2359274104 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f0c840afb60 0x7f0c840b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.871+0000 7f0c8b7fe700 1 -- 172.21.15.43:0/2359274104 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f0c8c0deb90 con 0x7f0c9c1029c0 2024-04-25T10:31:01.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.871+0000 7f0c8b7fe700 1 --2- 172.21.15.43:0/2359274104 >> [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172] conn(0x7f0c840b9390 0x7f0c840bb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.871+0000 7f0c8b7fe700 1 -- 172.21.15.43:0/2359274104 --> [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c8c008e50 con 0x7f0c840b9390 2024-04-25T10:31:01.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.871+0000 7f0c8b7fe700 1 -- 172.21.15.43:0/2359274104 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f0c8c0def40 con 0x7f0c9c1029c0 2024-04-25T10:31:01.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.871+0000 7f0c9ad9d700 1 --2- 172.21.15.43:0/2359274104 >> [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172] conn(0x7f0c840b9390 0x7f0c840bb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.872+0000 7f0c99d9b700 1 --2- 172.21.15.43:0/2359274104 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f0c840afb60 0x7f0c840b2070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c101e40 tx=0x7f0c9000a400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:01.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.872+0000 7f0c9ad9d700 1 --2- 172.21.15.43:0/2359274104 >> [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172] conn(0x7f0c840b9390 0x7f0c840bb7e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:01.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.873+0000 7f0c8b7fe700 1 -- 172.21.15.43:0/2359274104 <== osd.6 v2:172.21.15.43:6818/1494696172 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0c8c008e50 con 0x7f0c840b9390 2024-04-25T10:31:01.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.889+0000 7f0ca099f700 1 -- 172.21.15.43:0/2359274104 --> [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0c9c0642a0 con 0x7f0c840b9390 2024-04-25T10:31:01.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.889+0000 7f0c8b7fe700 1 -- 172.21.15.43:0/2359274104 <== osd.6 v2:172.21.15.43:6818/1494696172 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0c9c0642a0 con 0x7f0c840b9390 2024-04-25T10:31:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.890+0000 7f0ca099f700 1 -- 172.21.15.43:0/2359274104 >> [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172] conn(0x7f0c840b9390 msgr2=0x7f0c840bb7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.890+0000 7f0ca099f700 1 --2- 172.21.15.43:0/2359274104 >> [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172] conn(0x7f0c840b9390 0x7f0c840bb7e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.890+0000 7f0ca099f700 1 -- 172.21.15.43:0/2359274104 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f0c840afb60 msgr2=0x7f0c840b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.890+0000 7f0ca099f700 1 --2- 172.21.15.43:0/2359274104 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f0c840afb60 0x7f0c840b2070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c101e40 tx=0x7f0c9000a400 comp rx=0 tx=0).stop 2024-04-25T10:31:01.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.890+0000 7f0ca099f700 1 -- 172.21.15.43:0/2359274104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c9c1029c0 msgr2=0x7f0c9c1a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.890+0000 7f0ca099f700 1 --2- 172.21.15.43:0/2359274104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c9c1029c0 0x7f0c9c1a0810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00d740 tx=0x7f0c8c00da50 comp rx=0 tx=0).stop 2024-04-25T10:31:01.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.890+0000 7f0c9ad9d700 1 -- 172.21.15.43:0/2359274104 reap_dead start 2024-04-25T10:31:01.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.890+0000 7f0ca099f700 1 -- 172.21.15.43:0/2359274104 shutdown_connections 2024-04-25T10:31:01.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.890+0000 7f0ca099f700 1 -- 172.21.15.43:0/2359274104 >> 172.21.15.43:0/2359274104 conn(0x7f0c9c0fe3e0 msgr2=0x7f0c9c108550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:01.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.890+0000 7f0ca099f700 1 -- 172.21.15.43:0/2359274104 shutdown_connections 2024-04-25T10:31:01.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:01.890+0000 7f0ca099f700 1 -- 172.21.15.43:0/2359274104 wait complete. 2024-04-25T10:31:01.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.216+0000 7fa5ce2da700 1 -- 172.21.15.43:0/3460265849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c8108630 msgr2=0x7fa5c810ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.216+0000 7fa5ce2da700 1 --2- 172.21.15.43:0/3460265849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c8108630 0x7fa5c810ff90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000b0d0 tx=0x7fa5c000b3e0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.216+0000 7fa5ce2da700 1 -- 172.21.15.43:0/3460265849 shutdown_connections 2024-04-25T10:31:01.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.216+0000 7fa5ce2da700 1 --2- 172.21.15.43:0/3460265849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c8108630 0x7fa5c810ff90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.216+0000 7fa5ce2da700 1 --2- 172.21.15.43:0/3460265849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5c81033b0 0x7fa5c81037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.216+0000 7fa5ce2da700 1 --2- 172.21.15.43:0/3460265849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5c8107ce0 0x7fa5c81080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.216+0000 7fa5ce2da700 1 -- 172.21.15.43:0/3460265849 >> 172.21.15.43:0/3460265849 conn(0x7fa5c80fee00 msgr2=0x7fa5c8101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:01.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.216+0000 7fa5ce2da700 1 -- 172.21.15.43:0/3460265849 shutdown_connections 2024-04-25T10:31:01.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.216+0000 7fa5ce2da700 1 -- 172.21.15.43:0/3460265849 wait complete. 2024-04-25T10:31:01.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.217+0000 7fa5ce2da700 1 Processor -- start 2024-04-25T10:31:01.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.217+0000 7fa5ce2da700 1 -- start start 2024-04-25T10:31:01.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.217+0000 7fa5ce2da700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c81033b0 0x7fa5c8110770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.217+0000 7fa5ce2da700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5c8107ce0 0x7fa5c8110cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.217+0000 7fa5ce2da700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5c8108630 0x7fa5c81111f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.217+0000 7fa5ce2da700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81143b0 con 0x7fa5c81033b0 2024-04-25T10:31:01.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.217+0000 7fa5ce2da700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c8114230 con 0x7fa5c8108630 2024-04-25T10:31:01.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.217+0000 7fa5ce2da700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c8114530 con 0x7fa5c8107ce0 2024-04-25T10:31:01.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.217+0000 7fa5c7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c81033b0 0x7fa5c8110770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.217+0000 7fa5c7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c81033b0 0x7fa5c8110770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37664/0 (socket says 172.21.15.43:37664) 2024-04-25T10:31:01.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.217+0000 7fa5c7fff700 1 -- 172.21.15.43:0/3536586297 learned_addr learned my addr 172.21.15.43:0/3536586297 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:01.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.218+0000 7fa5c77fe700 1 --2- 172.21.15.43:0/3536586297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5c8107ce0 0x7fa5c8110cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.218+0000 7fa5cc877700 1 --2- 172.21.15.43:0/3536586297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5c8108630 0x7fa5c81111f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.218+0000 7fa5c7fff700 1 -- 172.21.15.43:0/3536586297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5c8107ce0 msgr2=0x7fa5c8110cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.218+0000 7fa5c7fff700 1 --2- 172.21.15.43:0/3536586297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5c8107ce0 0x7fa5c8110cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.218+0000 7fa5c7fff700 1 -- 172.21.15.43:0/3536586297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5c8108630 msgr2=0x7fa5c81111f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.218+0000 7fa5c7fff700 1 --2- 172.21.15.43:0/3536586297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5c8108630 0x7fa5c81111f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.218+0000 7fa5c7fff700 1 -- 172.21.15.43:0/3536586297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c81a9060 con 0x7fa5c81033b0 2024-04-25T10:31:01.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.218+0000 7fa5cc877700 1 --2- 172.21.15.43:0/3536586297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5c8108630 0x7fa5c81111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:01.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.218+0000 7fa5c7fff700 1 --2- 172.21.15.43:0/3536586297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c81033b0 0x7fa5c8110770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800e9a0 tx=0x7fa5b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:01.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.218+0000 7fa5c57fa700 1 -- 172.21.15.43:0/3536586297 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b800cbd0 con 0x7fa5c81033b0 2024-04-25T10:31:01.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.218+0000 7fa5ce2da700 1 -- 172.21.15.43:0/3536586297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c81a9350 con 0x7fa5c81033b0 2024-04-25T10:31:01.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.218+0000 7fa5c57fa700 1 -- 172.21.15.43:0/3536586297 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5b800cd70 con 0x7fa5c81033b0 2024-04-25T10:31:01.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.218+0000 7fa5c57fa700 1 -- 172.21.15.43:0/3536586297 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8018680 con 0x7fa5c81033b0 2024-04-25T10:31:01.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.218+0000 7fa5ce2da700 1 -- 172.21.15.43:0/3536586297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c81a98e0 con 0x7fa5c81033b0 2024-04-25T10:31:01.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.219+0000 7fa5ce2da700 1 -- 172.21.15.43:0/3536586297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa5c8063da0 con 0x7fa5c81033b0 2024-04-25T10:31:01.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.221+0000 7fa5c57fa700 1 -- 172.21.15.43:0/3536586297 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fa5b8010430 con 0x7fa5c81033b0 2024-04-25T10:31:01.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.221+0000 7fa5c57fa700 1 --2- 172.21.15.43:0/3536586297 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fa5b00afc10 0x7fa5b00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.221+0000 7fa5c77fe700 1 --2- 172.21.15.43:0/3536586297 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fa5b00afc10 0x7fa5b00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.221+0000 7fa5c57fa700 1 -- 172.21.15.43:0/3536586297 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fa5b80df9f0 con 0x7fa5c81033b0 2024-04-25T10:31:01.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.222+0000 7fa5c57fa700 1 --2- 172.21.15.43:0/3536586297 >> [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413] conn(0x7fa5b00b9440 0x7fa5b00bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.222+0000 7fa5c57fa700 1 -- 172.21.15.43:0/3536586297 --> [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa5b8019ec0 con 0x7fa5b00b9440 2024-04-25T10:31:01.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.222+0000 7fa5c57fa700 1 -- 172.21.15.43:0/3536586297 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fa5b80dfed0 con 0x7fa5c81033b0 2024-04-25T10:31:01.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.222+0000 7fa5cc877700 1 --2- 172.21.15.43:0/3536586297 >> [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413] conn(0x7fa5b00b9440 0x7fa5b00bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.222+0000 7fa5c77fe700 1 --2- 172.21.15.43:0/3536586297 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fa5b00afc10 0x7fa5b00b2120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8106640 tx=0x7fa5bc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:01.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.222+0000 7fa5cc877700 1 --2- 172.21.15.43:0/3536586297 >> [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413] conn(0x7fa5b00b9440 0x7fa5b00bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:01.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.223+0000 7fa5c57fa700 1 -- 172.21.15.43:0/3536586297 <== osd.9 v2:172.21.15.43:6826/537180413 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa5b8019ec0 con 0x7fa5b00b9440 2024-04-25T10:31:01.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.240+0000 7fa5ce2da700 1 -- 172.21.15.43:0/3536586297 --> [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa5c80641d0 con 0x7fa5b00b9440 2024-04-25T10:31:01.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.240+0000 7fa5c57fa700 1 -- 172.21.15.43:0/3536586297 <== osd.9 v2:172.21.15.43:6826/537180413 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa5c80641d0 con 0x7fa5b00b9440 2024-04-25T10:31:01.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.240+0000 7fa5ce2da700 1 -- 172.21.15.43:0/3536586297 >> [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413] conn(0x7fa5b00b9440 msgr2=0x7fa5b00bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.240+0000 7fa5ce2da700 1 --2- 172.21.15.43:0/3536586297 >> [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413] conn(0x7fa5b00b9440 0x7fa5b00bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.240+0000 7fa5ce2da700 1 -- 172.21.15.43:0/3536586297 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fa5b00afc10 msgr2=0x7fa5b00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.240+0000 7fa5ce2da700 1 --2- 172.21.15.43:0/3536586297 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fa5b00afc10 0x7fa5b00b2120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8106640 tx=0x7fa5bc00b410 comp rx=0 tx=0).stop 2024-04-25T10:31:01.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.240+0000 7fa5ce2da700 1 -- 172.21.15.43:0/3536586297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c81033b0 msgr2=0x7fa5c8110770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.240+0000 7fa5ce2da700 1 --2- 172.21.15.43:0/3536586297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c81033b0 0x7fa5c8110770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800e9a0 tx=0x7fa5b800ecb0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.240+0000 7fa5cc877700 1 -- 172.21.15.43:0/3536586297 reap_dead start 2024-04-25T10:31:01.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.241+0000 7fa5ce2da700 1 -- 172.21.15.43:0/3536586297 shutdown_connections 2024-04-25T10:31:01.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.241+0000 7fa5ce2da700 1 -- 172.21.15.43:0/3536586297 >> 172.21.15.43:0/3536586297 conn(0x7fa5c80fee00 msgr2=0x7fa5c8107450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:01.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.241+0000 7fa5ce2da700 1 -- 172.21.15.43:0/3536586297 shutdown_connections 2024-04-25T10:31:01.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:00.241+0000 7fa5ce2da700 1 -- 172.21.15.43:0/3536586297 wait complete. 2024-04-25T10:31:02.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:01 smithi043 ceph-mon[29478]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T10:31:02.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:01 smithi077 ceph-mon[32772]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T10:31:02.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:01 smithi164 ceph-mon[32713]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T10:31:03.718 INFO:teuthology.orchestra.run.smithi043.stdout:154618822704 2024-04-25T10:31:03.719 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T10:31:03.963 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:03 smithi043 ceph-mon[29478]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T10:31:04.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:03 smithi077 ceph-mon[32772]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T10:31:04.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:03 smithi164 ceph-mon[32713]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T10:31:04.380 INFO:teuthology.orchestra.run.smithi043.stdout:257698037789 2024-04-25T10:31:04.381 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T10:31:05.996 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:05 smithi043 ceph-mon[29478]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T10:31:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.018+0000 7f4590ef9700 1 -- 172.21.15.43:0/3270216241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f458c105cd0 msgr2=0x7f458c10e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.018+0000 7f4590ef9700 1 --2- 172.21.15.43:0/3270216241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f458c105cd0 0x7f458c10e220 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4580009990 tx=0x7f4580009ca0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.018+0000 7f4590ef9700 1 -- 172.21.15.43:0/3270216241 shutdown_connections 2024-04-25T10:31:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.018+0000 7f4590ef9700 1 --2- 172.21.15.43:0/3270216241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f458c10e7b0 0x7f458c110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.018+0000 7f4590ef9700 1 --2- 172.21.15.43:0/3270216241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f458c105380 0x7f458c105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.018+0000 7f4590ef9700 1 --2- 172.21.15.43:0/3270216241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f458c105cd0 0x7f458c10e220 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.018+0000 7f4590ef9700 1 -- 172.21.15.43:0/3270216241 >> 172.21.15.43:0/3270216241 conn(0x7f458c0fec90 msgr2=0x7f458c1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.018+0000 7f4590ef9700 1 -- 172.21.15.43:0/3270216241 shutdown_connections 2024-04-25T10:31:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.018+0000 7f4590ef9700 1 -- 172.21.15.43:0/3270216241 wait complete. 2024-04-25T10:31:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.019+0000 7f4590ef9700 1 Processor -- start 2024-04-25T10:31:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.019+0000 7f4590ef9700 1 -- start start 2024-04-25T10:31:06.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.019+0000 7f4590ef9700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f458c105380 0x7f458c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.019+0000 7f4590ef9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f458c105cd0 0x7f458c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.019+0000 7f4590ef9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f458c10e7b0 0x7f458c19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.019+0000 7f4590ef9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f458c114330 con 0x7f458c105cd0 2024-04-25T10:31:06.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.019+0000 7f4590ef9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f458c1141b0 con 0x7f458c10e7b0 2024-04-25T10:31:06.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.019+0000 7f4590ef9700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f458c1144b0 con 0x7f458c105380 2024-04-25T10:31:06.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.019+0000 7f4589d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f458c105cd0 0x7f458c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.019+0000 7f4589d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f458c105cd0 0x7f458c19b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37698/0 (socket says 172.21.15.43:37698) 2024-04-25T10:31:06.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.019+0000 7f4589d9b700 1 -- 172.21.15.43:0/1921146392 learned_addr learned my addr 172.21.15.43:0/1921146392 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:06.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.019+0000 7f458a59c700 1 --2- 172.21.15.43:0/1921146392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f458c105380 0x7f458c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.020+0000 7f458ad9d700 1 --2- 172.21.15.43:0/1921146392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f458c10e7b0 0x7f458c19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.020+0000 7f4589d9b700 1 -- 172.21.15.43:0/1921146392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f458c105380 msgr2=0x7f458c19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.020+0000 7f4589d9b700 1 --2- 172.21.15.43:0/1921146392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f458c105380 0x7f458c19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.020+0000 7f4589d9b700 1 -- 172.21.15.43:0/1921146392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f458c10e7b0 msgr2=0x7f458c19f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.020+0000 7f4589d9b700 1 --2- 172.21.15.43:0/1921146392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f458c10e7b0 0x7f458c19f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.020+0000 7f4589d9b700 1 -- 172.21.15.43:0/1921146392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c19fc10 con 0x7f458c105cd0 2024-04-25T10:31:06.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.020+0000 7f4589d9b700 1 --2- 172.21.15.43:0/1921146392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f458c105cd0 0x7f458c19b2e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4580005910 tx=0x7f45800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.020+0000 7f457b7fe700 1 -- 172.21.15.43:0/1921146392 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4580027070 con 0x7f458c105cd0 2024-04-25T10:31:06.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.020+0000 7f4590ef9700 1 -- 172.21.15.43:0/1921146392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458c19fea0 con 0x7f458c105cd0 2024-04-25T10:31:06.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.020+0000 7f457b7fe700 1 -- 172.21.15.43:0/1921146392 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4580003f20 con 0x7f458c105cd0 2024-04-25T10:31:06.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.020+0000 7f457b7fe700 1 -- 172.21.15.43:0/1921146392 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4580022d00 con 0x7f458c105cd0 2024-04-25T10:31:06.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.020+0000 7f4590ef9700 1 -- 172.21.15.43:0/1921146392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f458c1a7820 con 0x7f458c105cd0 2024-04-25T10:31:06.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.021+0000 7f4590ef9700 1 -- 172.21.15.43:0/1921146392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f458c1a03d0 con 0x7f458c105cd0 2024-04-25T10:31:06.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.023+0000 7f457b7fe700 1 -- 172.21.15.43:0/1921146392 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f458002a070 con 0x7f458c105cd0 2024-04-25T10:31:06.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.023+0000 7f457b7fe700 1 --2- 172.21.15.43:0/1921146392 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f45740afc10 0x7f45740b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.024+0000 7f458a59c700 1 --2- 172.21.15.43:0/1921146392 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f45740afc10 0x7f45740b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.024+0000 7f457b7fe700 1 -- 172.21.15.43:0/1921146392 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f45800e0250 con 0x7f458c105cd0 2024-04-25T10:31:06.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.024+0000 7f457b7fe700 1 --2- 172.21.15.43:0/1921146392 >> [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946] conn(0x7f45740b9440 0x7f45740bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.024+0000 7f457b7fe700 1 -- 172.21.15.43:0/1921146392 --> [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4580001d30 con 0x7f45740b9440 2024-04-25T10:31:06.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.024+0000 7f457b7fe700 1 -- 172.21.15.43:0/1921146392 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f45800e0600 con 0x7f458c105cd0 2024-04-25T10:31:06.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.024+0000 7f458ad9d700 1 --2- 172.21.15.43:0/1921146392 >> [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946] conn(0x7f45740b9440 0x7f45740bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.024+0000 7f458a59c700 1 --2- 172.21.15.43:0/1921146392 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f45740afc10 0x7f45740b2120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f457c005950 tx=0x7f457c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.024+0000 7f458ad9d700 1 --2- 172.21.15.43:0/1921146392 >> [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946] conn(0x7f45740b9440 0x7f45740bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.025+0000 7f457b7fe700 1 -- 172.21.15.43:0/1921146392 <== osd.0 v2:172.21.15.43:6802/3598858946 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4580001d30 con 0x7f45740b9440 2024-04-25T10:31:06.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.041+0000 7f4590ef9700 1 -- 172.21.15.43:0/1921146392 --> [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f458c063fb0 con 0x7f45740b9440 2024-04-25T10:31:06.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.042+0000 7f457b7fe700 1 -- 172.21.15.43:0/1921146392 <== osd.0 v2:172.21.15.43:6802/3598858946 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f458c063fb0 con 0x7f45740b9440 2024-04-25T10:31:06.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.042+0000 7f4590ef9700 1 -- 172.21.15.43:0/1921146392 >> [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946] conn(0x7f45740b9440 msgr2=0x7f45740bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.042+0000 7f4590ef9700 1 --2- 172.21.15.43:0/1921146392 >> [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946] conn(0x7f45740b9440 0x7f45740bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.042+0000 7f4590ef9700 1 -- 172.21.15.43:0/1921146392 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f45740afc10 msgr2=0x7f45740b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.042+0000 7f4590ef9700 1 --2- 172.21.15.43:0/1921146392 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f45740afc10 0x7f45740b2120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f457c005950 tx=0x7f457c00a300 comp rx=0 tx=0).stop 2024-04-25T10:31:06.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.042+0000 7f4590ef9700 1 -- 172.21.15.43:0/1921146392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f458c105cd0 msgr2=0x7f458c19b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.042+0000 7f4590ef9700 1 --2- 172.21.15.43:0/1921146392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f458c105cd0 0x7f458c19b2e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4580005910 tx=0x7f45800047b0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.042+0000 7f458ad9d700 1 -- 172.21.15.43:0/1921146392 reap_dead start 2024-04-25T10:31:06.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.042+0000 7f4590ef9700 1 -- 172.21.15.43:0/1921146392 shutdown_connections 2024-04-25T10:31:06.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.042+0000 7f4590ef9700 1 -- 172.21.15.43:0/1921146392 >> 172.21.15.43:0/1921146392 conn(0x7f458c0fec90 msgr2=0x7f458c100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.042+0000 7f4590ef9700 1 -- 172.21.15.43:0/1921146392 shutdown_connections 2024-04-25T10:31:06.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:06.042+0000 7f4590ef9700 1 -- 172.21.15.43:0/1921146392 wait complete. 2024-04-25T10:31:06.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:05 smithi077 ceph-mon[32772]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T10:31:06.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:05 smithi164 ceph-mon[32713]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T10:31:07.056 INFO:teuthology.orchestra.run.smithi043.stdout:180388626475 2024-04-25T10:31:07.057 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T10:31:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.277+0000 7f696d207700 1 -- 172.21.15.43:0/3285294684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968108610 msgr2=0x7f696810ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.277+0000 7f696d207700 1 --2- 172.21.15.43:0/3285294684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968108610 0x7f696810ff70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f696000b0d0 tx=0x7f696000b3e0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.278+0000 7f696d207700 1 -- 172.21.15.43:0/3285294684 shutdown_connections 2024-04-25T10:31:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.278+0000 7f696d207700 1 --2- 172.21.15.43:0/3285294684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968108610 0x7f696810ff70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.278+0000 7f696d207700 1 --2- 172.21.15.43:0/3285294684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6968103390 0x7f69681037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.278+0000 7f696d207700 1 --2- 172.21.15.43:0/3285294684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6968107cc0 0x7f69681080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.278+0000 7f696d207700 1 -- 172.21.15.43:0/3285294684 >> 172.21.15.43:0/3285294684 conn(0x7f69680fecc0 msgr2=0x7f69681010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.278+0000 7f696d207700 1 -- 172.21.15.43:0/3285294684 shutdown_connections 2024-04-25T10:31:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.278+0000 7f696d207700 1 -- 172.21.15.43:0/3285294684 wait complete. 2024-04-25T10:31:07.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.278+0000 7f696d207700 1 Processor -- start 2024-04-25T10:31:07.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.279+0000 7f696d207700 1 -- start start 2024-04-25T10:31:07.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.279+0000 7f696d207700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968103390 0x7f69681a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:07.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.279+0000 7f696d207700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6968107cc0 0x7f69681a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:07.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.279+0000 7f696d207700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6968108610 0x7f69681a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:07.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.279+0000 7f696d207700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6968113240 con 0x7f6968103390 2024-04-25T10:31:07.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.279+0000 7f696d207700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f69681130c0 con 0x7f6968108610 2024-04-25T10:31:07.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.279+0000 7f696d207700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f69681133c0 con 0x7f6968107cc0 2024-04-25T10:31:07.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.279+0000 7f6966d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968103390 0x7f69681a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:07.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.279+0000 7f6966d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968103390 0x7f69681a12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37726/0 (socket says 172.21.15.43:37726) 2024-04-25T10:31:07.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.279+0000 7f6966d9d700 1 -- 172.21.15.43:0/1976141549 learned_addr learned my addr 172.21.15.43:0/1976141549 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:07.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.279+0000 7f696759e700 1 --2- 172.21.15.43:0/1976141549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6968108610 0x7f69681a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:07.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.279+0000 7f696659c700 1 --2- 172.21.15.43:0/1976141549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6968107cc0 0x7f69681a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:07.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.280+0000 7f6966d9d700 1 -- 172.21.15.43:0/1976141549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6968107cc0 msgr2=0x7f69681a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.280+0000 7f6966d9d700 1 --2- 172.21.15.43:0/1976141549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6968107cc0 0x7f69681a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.280+0000 7f6966d9d700 1 -- 172.21.15.43:0/1976141549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6968108610 msgr2=0x7f69681a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.280+0000 7f6966d9d700 1 --2- 172.21.15.43:0/1976141549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6968108610 0x7f69681a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.280+0000 7f6966d9d700 1 -- 172.21.15.43:0/1976141549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69681a6190 con 0x7f6968103390 2024-04-25T10:31:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.280+0000 7f6966d9d700 1 --2- 172.21.15.43:0/1976141549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968103390 0x7f69681a12c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f695800c920 tx=0x7f695800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.280+0000 7f694ffff700 1 -- 172.21.15.43:0/1976141549 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958004a30 con 0x7f6968103390 2024-04-25T10:31:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.280+0000 7f696d207700 1 -- 172.21.15.43:0/1976141549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69681a6480 con 0x7f6968103390 2024-04-25T10:31:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.280+0000 7f694ffff700 1 -- 172.21.15.43:0/1976141549 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69580054d0 con 0x7f6968103390 2024-04-25T10:31:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.280+0000 7f694ffff700 1 -- 172.21.15.43:0/1976141549 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958007bd0 con 0x7f6968103390 2024-04-25T10:31:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.280+0000 7f696d207700 1 -- 172.21.15.43:0/1976141549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69681adda0 con 0x7f6968103390 2024-04-25T10:31:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.281+0000 7f696d207700 1 -- 172.21.15.43:0/1976141549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6968101430 con 0x7f6968103390 2024-04-25T10:31:07.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.283+0000 7f694ffff700 1 -- 172.21.15.43:0/1976141549 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f6958004bd0 con 0x7f6968103390 2024-04-25T10:31:07.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.284+0000 7f694ffff700 1 --2- 172.21.15.43:0/1976141549 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f69500afc10 0x7f69500b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:07.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.284+0000 7f696659c700 1 --2- 172.21.15.43:0/1976141549 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f69500afc10 0x7f69500b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:07.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.284+0000 7f694ffff700 1 -- 172.21.15.43:0/1976141549 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f69580dec70 con 0x7f6968103390 2024-04-25T10:31:07.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.284+0000 7f694ffff700 1 --2- 172.21.15.43:0/1976141549 >> [v2:172.21.15.77:6800/3362129259,v1:172.21.15.77:6801/3362129259] conn(0x7f69500b9440 0x7f69500bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.284+0000 7f694ffff700 1 -- 172.21.15.43:0/1976141549 --> [v2:172.21.15.77:6800/3362129259,v1:172.21.15.77:6801/3362129259] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f695800eec0 con 0x7f69500b9440 2024-04-25T10:31:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.284+0000 7f694ffff700 1 -- 172.21.15.43:0/1976141549 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f69580df020 con 0x7f6968103390 2024-04-25T10:31:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.284+0000 7f696659c700 1 --2- 172.21.15.43:0/1976141549 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f69500afc10 0x7f69500b2120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6968102710 tx=0x7f695c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.284+0000 7f696759e700 1 --2- 172.21.15.43:0/1976141549 >> [v2:172.21.15.77:6800/3362129259,v1:172.21.15.77:6801/3362129259] conn(0x7f69500b9440 0x7f69500bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.285+0000 7f696759e700 1 --2- 172.21.15.43:0/1976141549 >> [v2:172.21.15.77:6800/3362129259,v1:172.21.15.77:6801/3362129259] conn(0x7f69500b9440 0x7f69500bb890 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:07.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.286+0000 7f694ffff700 1 -- 172.21.15.43:0/1976141549 <== osd.1 v2:172.21.15.77:6800/3362129259 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f695800eec0 con 0x7f69500b9440 2024-04-25T10:31:07.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.302+0000 7f696d207700 1 -- 172.21.15.43:0/1976141549 --> [v2:172.21.15.77:6800/3362129259,v1:172.21.15.77:6801/3362129259] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6968104840 con 0x7f69500b9440 2024-04-25T10:31:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.303+0000 7f694ffff700 1 -- 172.21.15.43:0/1976141549 <== osd.1 v2:172.21.15.77:6800/3362129259 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6968104840 con 0x7f69500b9440 2024-04-25T10:31:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.303+0000 7f696d207700 1 -- 172.21.15.43:0/1976141549 >> [v2:172.21.15.77:6800/3362129259,v1:172.21.15.77:6801/3362129259] conn(0x7f69500b9440 msgr2=0x7f69500bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.303+0000 7f696d207700 1 --2- 172.21.15.43:0/1976141549 >> [v2:172.21.15.77:6800/3362129259,v1:172.21.15.77:6801/3362129259] conn(0x7f69500b9440 0x7f69500bb890 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.303+0000 7f696d207700 1 -- 172.21.15.43:0/1976141549 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f69500afc10 msgr2=0x7f69500b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.303+0000 7f696d207700 1 --2- 172.21.15.43:0/1976141549 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f69500afc10 0x7f69500b2120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6968102710 tx=0x7f695c00a300 comp rx=0 tx=0).stop 2024-04-25T10:31:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.303+0000 7f696d207700 1 -- 172.21.15.43:0/1976141549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968103390 msgr2=0x7f69681a12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.303+0000 7f696d207700 1 --2- 172.21.15.43:0/1976141549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968103390 0x7f69681a12c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f695800c920 tx=0x7f695800cc30 comp rx=0 tx=0).stop 2024-04-25T10:31:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.303+0000 7f696759e700 1 -- 172.21.15.43:0/1976141549 reap_dead start 2024-04-25T10:31:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.303+0000 7f696d207700 1 -- 172.21.15.43:0/1976141549 shutdown_connections 2024-04-25T10:31:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.303+0000 7f696d207700 1 -- 172.21.15.43:0/1976141549 >> 172.21.15.43:0/1976141549 conn(0x7f69680fecc0 msgr2=0x7f6968107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.304+0000 7f696d207700 1 -- 172.21.15.43:0/1976141549 shutdown_connections 2024-04-25T10:31:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:07.304+0000 7f696d207700 1 -- 172.21.15.43:0/1976141549 wait complete. 2024-04-25T10:31:08.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:07 smithi043 ceph-mon[29478]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T10:31:08.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:07 smithi077 ceph-mon[32772]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T10:31:08.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:07 smithi164 ceph-mon[32713]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T10:31:10.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:09 smithi043 ceph-mon[29478]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T10:31:10.110 INFO:teuthology.orchestra.run.smithi043.stdout:34359738443 2024-04-25T10:31:10.110 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T10:31:10.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:09 smithi077 ceph-mon[32772]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T10:31:10.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:09 smithi164 ceph-mon[32713]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T10:31:10.512 INFO:teuthology.orchestra.run.smithi043.stdout:55834574919 2024-04-25T10:31:10.512 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T10:31:11.886 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:11 smithi043 ceph-mon[29478]: pgmap v296: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T10:31:11.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.916+0000 7ff710e90700 1 -- 172.21.15.43:0/3746318380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70c109710 msgr2=0x7ff70c111070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.916+0000 7ff710e90700 1 --2- 172.21.15.43:0/3746318380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70c109710 0x7ff70c111070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff70400b0d0 tx=0x7ff70400b3e0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.916+0000 7ff710e90700 1 -- 172.21.15.43:0/3746318380 shutdown_connections 2024-04-25T10:31:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.916+0000 7ff710e90700 1 --2- 172.21.15.43:0/3746318380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70c109710 0x7ff70c111070 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.916+0000 7ff710e90700 1 --2- 172.21.15.43:0/3746318380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff70c104490 0x7ff70c1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.916+0000 7ff710e90700 1 --2- 172.21.15.43:0/3746318380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff70c108dc0 0x7ff70c1091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.916+0000 7ff710e90700 1 -- 172.21.15.43:0/3746318380 >> 172.21.15.43:0/3746318380 conn(0x7ff70c0ffd60 msgr2=0x7ff70c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.916+0000 7ff710e90700 1 -- 172.21.15.43:0/3746318380 shutdown_connections 2024-04-25T10:31:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.916+0000 7ff710e90700 1 -- 172.21.15.43:0/3746318380 wait complete. 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.917+0000 7ff710e90700 1 Processor -- start 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.917+0000 7ff710e90700 1 -- start start 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.917+0000 7ff710e90700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff70c104490 0x7ff70c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.917+0000 7ff710e90700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff70c108dc0 0x7ff70c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.917+0000 7ff710e90700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70c109710 0x7ff70c19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.917+0000 7ff710e90700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c114480 con 0x7ff70c109710 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.917+0000 7ff710e90700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c114300 con 0x7ff70c108dc0 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.917+0000 7ff710e90700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c114600 con 0x7ff70c104490 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.918+0000 7ff70ad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70c109710 0x7ff70c19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.918+0000 7ff70ad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70c109710 0x7ff70c19f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40570/0 (socket says 172.21.15.43:40570) 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.918+0000 7ff70ad9d700 1 -- 172.21.15.43:0/2254133122 learned_addr learned my addr 172.21.15.43:0/2254133122 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.918+0000 7ff70a59c700 1 --2- 172.21.15.43:0/2254133122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff70c104490 0x7ff70c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.918+0000 7ff709d9b700 1 --2- 172.21.15.43:0/2254133122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff70c108dc0 0x7ff70c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.918+0000 7ff70ad9d700 1 -- 172.21.15.43:0/2254133122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff70c104490 msgr2=0x7ff70c19adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.918+0000 7ff70ad9d700 1 --2- 172.21.15.43:0/2254133122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff70c104490 0x7ff70c19adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.918+0000 7ff70ad9d700 1 -- 172.21.15.43:0/2254133122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff70c108dc0 msgr2=0x7ff70c19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.918+0000 7ff70ad9d700 1 --2- 172.21.15.43:0/2254133122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff70c108dc0 0x7ff70c19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.918+0000 7ff70ad9d700 1 -- 172.21.15.43:0/2254133122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70c19fc80 con 0x7ff70c109710 2024-04-25T10:31:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.918+0000 7ff70ad9d700 1 --2- 172.21.15.43:0/2254133122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70c109710 0x7ff70c19f5e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff7040079a0 tx=0x7ff7040079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.918+0000 7ff6fb7fe700 1 -- 172.21.15.43:0/2254133122 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70401f070 con 0x7ff70c109710 2024-04-25T10:31:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.918+0000 7ff710e90700 1 -- 172.21.15.43:0/2254133122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff70c19ff10 con 0x7ff70c109710 2024-04-25T10:31:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.918+0000 7ff6fb7fe700 1 -- 172.21.15.43:0/2254133122 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff704005070 con 0x7ff70c109710 2024-04-25T10:31:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.918+0000 7ff6fb7fe700 1 -- 172.21.15.43:0/2254133122 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70401b680 con 0x7ff70c109710 2024-04-25T10:31:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.918+0000 7ff710e90700 1 -- 172.21.15.43:0/2254133122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff70c1a03f0 con 0x7ff70c109710 2024-04-25T10:31:11.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.919+0000 7ff710e90700 1 -- 172.21.15.43:0/2254133122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff70c063e20 con 0x7ff70c109710 2024-04-25T10:31:11.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.921+0000 7ff6fb7fe700 1 -- 172.21.15.43:0/2254133122 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7ff704022070 con 0x7ff70c109710 2024-04-25T10:31:11.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.922+0000 7ff6fb7fe700 1 --2- 172.21.15.43:0/2254133122 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff6f40afc10 0x7ff6f40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.922+0000 7ff70a59c700 1 --2- 172.21.15.43:0/2254133122 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff6f40afc10 0x7ff6f40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.922+0000 7ff6fb7fe700 1 -- 172.21.15.43:0/2254133122 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7ff70400f020 con 0x7ff70c109710 2024-04-25T10:31:11.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.922+0000 7ff6fb7fe700 1 --2- 172.21.15.43:0/2254133122 >> [v2:172.21.15.164:6816/3396063471,v1:172.21.15.164:6817/3396063471] conn(0x7ff6f40b9440 0x7ff6f40bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.922+0000 7ff6fb7fe700 1 -- 172.21.15.43:0/2254133122 --> [v2:172.21.15.164:6816/3396063471,v1:172.21.15.164:6817/3396063471] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff7040075c0 con 0x7ff6f40b9440 2024-04-25T10:31:11.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.922+0000 7ff6fb7fe700 1 -- 172.21.15.43:0/2254133122 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7ff7040e0580 con 0x7ff70c109710 2024-04-25T10:31:11.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.922+0000 7ff70a59c700 1 --2- 172.21.15.43:0/2254133122 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff6f40afc10 0x7ff6f40b2120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009a10 tx=0x7ff6fc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.922+0000 7ff709d9b700 1 --2- 172.21.15.43:0/2254133122 >> [v2:172.21.15.164:6816/3396063471,v1:172.21.15.164:6817/3396063471] conn(0x7ff6f40b9440 0x7ff6f40bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.923+0000 7ff709d9b700 1 --2- 172.21.15.43:0/2254133122 >> [v2:172.21.15.164:6816/3396063471,v1:172.21.15.164:6817/3396063471] conn(0x7ff6f40b9440 0x7ff6f40bb890 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.924+0000 7ff6fb7fe700 1 -- 172.21.15.43:0/2254133122 <== osd.8 v2:172.21.15.164:6816/3396063471 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff7040075c0 con 0x7ff6f40b9440 2024-04-25T10:31:11.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.941+0000 7ff710e90700 1 -- 172.21.15.43:0/2254133122 --> [v2:172.21.15.164:6816/3396063471,v1:172.21.15.164:6817/3396063471] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff70c0642a0 con 0x7ff6f40b9440 2024-04-25T10:31:11.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.941+0000 7ff6fb7fe700 1 -- 172.21.15.43:0/2254133122 <== osd.8 v2:172.21.15.164:6816/3396063471 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff70c0642a0 con 0x7ff6f40b9440 2024-04-25T10:31:11.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.941+0000 7ff710e90700 1 -- 172.21.15.43:0/2254133122 >> [v2:172.21.15.164:6816/3396063471,v1:172.21.15.164:6817/3396063471] conn(0x7ff6f40b9440 msgr2=0x7ff6f40bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.941+0000 7ff710e90700 1 --2- 172.21.15.43:0/2254133122 >> [v2:172.21.15.164:6816/3396063471,v1:172.21.15.164:6817/3396063471] conn(0x7ff6f40b9440 0x7ff6f40bb890 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.941+0000 7ff710e90700 1 -- 172.21.15.43:0/2254133122 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff6f40afc10 msgr2=0x7ff6f40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.941+0000 7ff710e90700 1 --2- 172.21.15.43:0/2254133122 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff6f40afc10 0x7ff6f40b2120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009a10 tx=0x7ff6fc009190 comp rx=0 tx=0).stop 2024-04-25T10:31:11.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.941+0000 7ff710e90700 1 -- 172.21.15.43:0/2254133122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70c109710 msgr2=0x7ff70c19f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.941+0000 7ff710e90700 1 --2- 172.21.15.43:0/2254133122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70c109710 0x7ff70c19f5e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff7040079a0 tx=0x7ff7040079d0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.941+0000 7ff70ad9d700 1 -- 172.21.15.43:0/2254133122 reap_dead start 2024-04-25T10:31:11.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.942+0000 7ff710e90700 1 -- 172.21.15.43:0/2254133122 shutdown_connections 2024-04-25T10:31:11.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.942+0000 7ff710e90700 1 -- 172.21.15.43:0/2254133122 >> 172.21.15.43:0/2254133122 conn(0x7ff70c0ffd60 msgr2=0x7ff70c108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:11.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.942+0000 7ff710e90700 1 -- 172.21.15.43:0/2254133122 shutdown_connections 2024-04-25T10:31:11.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:11.942+0000 7ff710e90700 1 -- 172.21.15.43:0/2254133122 wait complete. 2024-04-25T10:31:12.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:11 smithi077 ceph-mon[32772]: pgmap v296: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T10:31:12.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:11 smithi164 ceph-mon[32713]: pgmap v296: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T10:31:12.787 INFO:teuthology.orchestra.run.smithi043.stdout:231928234020 2024-04-25T10:31:12.787 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T10:31:13.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.111+0000 7f6689024700 1 -- 172.21.15.43:0/3879580636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6684103af0 msgr2=0x7f6684103f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.111+0000 7f6689024700 1 --2- 172.21.15.43:0/3879580636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6684103af0 0x7f6684103f00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f66780099c0 tx=0x7f6678009cd0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.111+0000 7f6689024700 1 -- 172.21.15.43:0/3879580636 shutdown_connections 2024-04-25T10:31:13.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.111+0000 7f6689024700 1 --2- 172.21.15.43:0/3879580636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6684108640 0x7f668410ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.111+0000 7f6689024700 1 --2- 172.21.15.43:0/3879580636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6684103af0 0x7f6684103f00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.111+0000 7f6689024700 1 --2- 172.21.15.43:0/3879580636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6684107cf0 0x7f6684108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.111+0000 7f6689024700 1 -- 172.21.15.43:0/3879580636 >> 172.21.15.43:0/3879580636 conn(0x7f66840fe5c0 msgr2=0x7f66841009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:13.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.111+0000 7f6689024700 1 -- 172.21.15.43:0/3879580636 shutdown_connections 2024-04-25T10:31:13.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.111+0000 7f6689024700 1 -- 172.21.15.43:0/3879580636 wait complete. 2024-04-25T10:31:13.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.112+0000 7f6689024700 1 Processor -- start 2024-04-25T10:31:13.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.112+0000 7f6689024700 1 -- start start 2024-04-25T10:31:13.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.112+0000 7f6689024700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6684103af0 0x7f6684110780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.112+0000 7f6689024700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6684107cf0 0x7f6684110cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.112+0000 7f6689024700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6684108640 0x7f6684111200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.113+0000 7f668259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6684107cf0 0x7f6684110cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.113+0000 7f668259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6684107cf0 0x7f6684110cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40598/0 (socket says 172.21.15.43:40598) 2024-04-25T10:31:13.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.113+0000 7f668259c700 1 -- 172.21.15.43:0/385636433 learned_addr learned my addr 172.21.15.43:0/385636433 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:13.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.113+0000 7f6689024700 1 -- 172.21.15.43:0/385636433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6684114440 con 0x7f6684107cf0 2024-04-25T10:31:13.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.113+0000 7f6689024700 1 -- 172.21.15.43:0/385636433 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f66841142c0 con 0x7f6684103af0 2024-04-25T10:31:13.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.113+0000 7f6689024700 1 -- 172.21.15.43:0/385636433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f66841145c0 con 0x7f6684108640 2024-04-25T10:31:13.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.113+0000 7f6682d9d700 1 --2- 172.21.15.43:0/385636433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6684103af0 0x7f6684110780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.113+0000 7f668359e700 1 --2- 172.21.15.43:0/385636433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6684108640 0x7f6684111200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.113+0000 7f668259c700 1 -- 172.21.15.43:0/385636433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6684108640 msgr2=0x7f6684111200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.113+0000 7f668259c700 1 --2- 172.21.15.43:0/385636433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6684108640 0x7f6684111200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.113+0000 7f668259c700 1 -- 172.21.15.43:0/385636433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6684103af0 msgr2=0x7f6684110780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.113+0000 7f668259c700 1 --2- 172.21.15.43:0/385636433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6684103af0 0x7f6684110780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.113+0000 7f668259c700 1 -- 172.21.15.43:0/385636433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66841a9190 con 0x7f6684107cf0 2024-04-25T10:31:13.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.113+0000 7f668259c700 1 --2- 172.21.15.43:0/385636433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6684107cf0 0x7f6684110cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f667400e970 tx=0x7f667400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:13.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.113+0000 7f666bfff700 1 -- 172.21.15.43:0/385636433 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6674004a30 con 0x7f6684107cf0 2024-04-25T10:31:13.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.113+0000 7f6689024700 1 -- 172.21.15.43:0/385636433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66841a9480 con 0x7f6684107cf0 2024-04-25T10:31:13.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.113+0000 7f666bfff700 1 -- 172.21.15.43:0/385636433 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f667400cc90 con 0x7f6684107cf0 2024-04-25T10:31:13.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.114+0000 7f666bfff700 1 -- 172.21.15.43:0/385636433 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6674005610 con 0x7f6684107cf0 2024-04-25T10:31:13.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.114+0000 7f6689024700 1 -- 172.21.15.43:0/385636433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66841a9a10 con 0x7f6684107cf0 2024-04-25T10:31:13.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.114+0000 7f6689024700 1 -- 172.21.15.43:0/385636433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6684063d60 con 0x7f6684107cf0 2024-04-25T10:31:13.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.116+0000 7f666bfff700 1 -- 172.21.15.43:0/385636433 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f667400c7e0 con 0x7f6684107cf0 2024-04-25T10:31:13.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.117+0000 7f666bfff700 1 --2- 172.21.15.43:0/385636433 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f666c0afc90 0x7f666c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.117+0000 7f6682d9d700 1 --2- 172.21.15.43:0/385636433 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f666c0afc90 0x7f666c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.117+0000 7f666bfff700 1 -- 172.21.15.43:0/385636433 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f66740df460 con 0x7f6684107cf0 2024-04-25T10:31:13.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.117+0000 7f666bfff700 1 --2- 172.21.15.43:0/385636433 >> [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010] conn(0x7f666c0b94c0 0x7f666c0bb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.117+0000 7f666bfff700 1 -- 172.21.15.43:0/385636433 --> [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f666c0bbfe0 con 0x7f666c0b94c0 2024-04-25T10:31:13.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.117+0000 7f666bfff700 1 -- 172.21.15.43:0/385636433 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f66740df810 con 0x7f6684107cf0 2024-04-25T10:31:13.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.117+0000 7f668359e700 1 --2- 172.21.15.43:0/385636433 >> [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010] conn(0x7f666c0b94c0 0x7f666c0bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.117+0000 7f6682d9d700 1 --2- 172.21.15.43:0/385636433 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f666c0afc90 0x7f666c0b21a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6678000c00 tx=0x7f667801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:13.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.118+0000 7f668359e700 1 --2- 172.21.15.43:0/385636433 >> [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010] conn(0x7f666c0b94c0 0x7f666c0bb910 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:13.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.118+0000 7f666bfff700 1 -- 172.21.15.43:0/385636433 <== osd.3 v2:172.21.15.43:6810/2368600010 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f666c0bbfe0 con 0x7f666c0b94c0 2024-04-25T10:31:13.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.135+0000 7f6689024700 1 -- 172.21.15.43:0/385636433 --> [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6684063fa0 con 0x7f666c0b94c0 2024-04-25T10:31:13.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.135+0000 7f666bfff700 1 -- 172.21.15.43:0/385636433 <== osd.3 v2:172.21.15.43:6810/2368600010 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6684063fa0 con 0x7f666c0b94c0 2024-04-25T10:31:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.135+0000 7f6689024700 1 -- 172.21.15.43:0/385636433 >> [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010] conn(0x7f666c0b94c0 msgr2=0x7f666c0bb910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.135+0000 7f6689024700 1 --2- 172.21.15.43:0/385636433 >> [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010] conn(0x7f666c0b94c0 0x7f666c0bb910 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.135+0000 7f6689024700 1 -- 172.21.15.43:0/385636433 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f666c0afc90 msgr2=0x7f666c0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.135+0000 7f6689024700 1 --2- 172.21.15.43:0/385636433 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f666c0afc90 0x7f666c0b21a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6678000c00 tx=0x7f667801a040 comp rx=0 tx=0).stop 2024-04-25T10:31:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.135+0000 7f6689024700 1 -- 172.21.15.43:0/385636433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6684107cf0 msgr2=0x7f6684110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.135+0000 7f6689024700 1 --2- 172.21.15.43:0/385636433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6684107cf0 0x7f6684110cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f667400e970 tx=0x7f667400ec80 comp rx=0 tx=0).stop 2024-04-25T10:31:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.135+0000 7f668359e700 1 -- 172.21.15.43:0/385636433 reap_dead start 2024-04-25T10:31:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.135+0000 7f6689024700 1 -- 172.21.15.43:0/385636433 shutdown_connections 2024-04-25T10:31:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.135+0000 7f6689024700 1 -- 172.21.15.43:0/385636433 >> 172.21.15.43:0/385636433 conn(0x7f66840fe5c0 msgr2=0x7f66841074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.136+0000 7f6689024700 1 -- 172.21.15.43:0/385636433 shutdown_connections 2024-04-25T10:31:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.136+0000 7f6689024700 1 -- 172.21.15.43:0/385636433 wait complete. 2024-04-25T10:31:13.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.988+0000 7f43f52af700 1 -- 172.21.15.43:0/2899559096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43f0108dd0 msgr2=0x7f43f01091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.988+0000 7f43f52af700 1 --2- 172.21.15.43:0/2899559096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43f0108dd0 0x7f43f01091e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f43d80099c0 tx=0x7f43d8009cd0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.988+0000 7f43f52af700 1 -- 172.21.15.43:0/2899559096 shutdown_connections 2024-04-25T10:31:13.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.988+0000 7f43f52af700 1 --2- 172.21.15.43:0/2899559096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43f0109720 0x7f43f0111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.988+0000 7f43f52af700 1 --2- 172.21.15.43:0/2899559096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43f01044a0 0x7f43f01048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.988+0000 7f43f52af700 1 --2- 172.21.15.43:0/2899559096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43f0108dd0 0x7f43f01091e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.988+0000 7f43f52af700 1 -- 172.21.15.43:0/2899559096 >> 172.21.15.43:0/2899559096 conn(0x7f43f00ffd70 msgr2=0x7f43f01021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:13.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.988+0000 7f43f52af700 1 -- 172.21.15.43:0/2899559096 shutdown_connections 2024-04-25T10:31:13.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.988+0000 7f43f52af700 1 -- 172.21.15.43:0/2899559096 wait complete. 2024-04-25T10:31:13.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.988+0000 7f43f52af700 1 Processor -- start 2024-04-25T10:31:13.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.989+0000 7f43f52af700 1 -- start start 2024-04-25T10:31:13.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.989+0000 7f43f52af700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43f01044a0 0x7f43f019ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.989+0000 7f43f52af700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43f0108dd0 0x7f43f019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.989+0000 7f43f52af700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43f0109720 0x7f43f019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.989+0000 7f43f52af700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43f0114330 con 0x7f43f0108dd0 2024-04-25T10:31:13.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.989+0000 7f43f52af700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f43f01141b0 con 0x7f43f01044a0 2024-04-25T10:31:13.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.989+0000 7f43f52af700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f43f01144b0 con 0x7f43f0109720 2024-04-25T10:31:13.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.989+0000 7f43ee7fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43f0108dd0 0x7f43f019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.989+0000 7f43ee7fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43f0108dd0 0x7f43f019b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40584/0 (socket says 172.21.15.43:40584) 2024-04-25T10:31:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.989+0000 7f43ee7fc700 1 -- 172.21.15.43:0/3628824986 learned_addr learned my addr 172.21.15.43:0/3628824986 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.989+0000 7f43ef7fe700 1 --2- 172.21.15.43:0/3628824986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43f0109720 0x7f43f019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.989+0000 7f43eeffd700 1 --2- 172.21.15.43:0/3628824986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43f01044a0 0x7f43f019ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.989+0000 7f43ee7fc700 1 -- 172.21.15.43:0/3628824986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43f0109720 msgr2=0x7f43f019f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.989+0000 7f43ee7fc700 1 --2- 172.21.15.43:0/3628824986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43f0109720 0x7f43f019f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.989+0000 7f43ee7fc700 1 -- 172.21.15.43:0/3628824986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43f01044a0 msgr2=0x7f43f019ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.989+0000 7f43ee7fc700 1 --2- 172.21.15.43:0/3628824986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43f01044a0 0x7f43f019ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.989+0000 7f43ee7fc700 1 -- 172.21.15.43:0/3628824986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f019fc50 con 0x7f43f0108dd0 2024-04-25T10:31:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.989+0000 7f43eeffd700 1 --2- 172.21.15.43:0/3628824986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43f01044a0 0x7f43f019ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.990+0000 7f43ee7fc700 1 --2- 172.21.15.43:0/3628824986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43f0108dd0 0x7f43f019b320 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f43d80045e0 tx=0x7f43d8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.990+0000 7f43e7fff700 1 -- 172.21.15.43:0/3628824986 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d8026070 con 0x7f43f0108dd0 2024-04-25T10:31:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.990+0000 7f43f52af700 1 -- 172.21.15.43:0/3628824986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f019fee0 con 0x7f43f0108dd0 2024-04-25T10:31:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.990+0000 7f43e7fff700 1 -- 172.21.15.43:0/3628824986 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43d800f460 con 0x7f43f0108dd0 2024-04-25T10:31:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.990+0000 7f43e7fff700 1 -- 172.21.15.43:0/3628824986 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d8009e60 con 0x7f43f0108dd0 2024-04-25T10:31:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.991+0000 7f43f52af700 1 -- 172.21.15.43:0/3628824986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f01a7860 con 0x7f43f0108dd0 2024-04-25T10:31:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.991+0000 7f43e5ffb700 1 -- 172.21.15.43:0/3628824986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f43f01a0410 con 0x7f43f0108dd0 2024-04-25T10:31:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.993+0000 7f43e7fff700 1 -- 172.21.15.43:0/3628824986 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f43d8029070 con 0x7f43f0108dd0 2024-04-25T10:31:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.993+0000 7f43e7fff700 1 --2- 172.21.15.43:0/3628824986 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f43dc0afc10 0x7f43dc0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.993+0000 7f43eeffd700 1 --2- 172.21.15.43:0/3628824986 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f43dc0afc10 0x7f43dc0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.993+0000 7f43e7fff700 1 -- 172.21.15.43:0/3628824986 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f43d80df460 con 0x7f43f0108dd0 2024-04-25T10:31:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.994+0000 7f43e7fff700 1 --2- 172.21.15.43:0/3628824986 >> [v2:172.21.15.164:6824/1085943743,v1:172.21.15.164:6825/1085943743] conn(0x7f43dc0b9440 0x7f43dc0bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.994+0000 7f43e7fff700 1 -- 172.21.15.43:0/3628824986 --> [v2:172.21.15.164:6824/1085943743,v1:172.21.15.164:6825/1085943743] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f43dc0bbf60 con 0x7f43dc0b9440 2024-04-25T10:31:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.994+0000 7f43e7fff700 1 -- 172.21.15.43:0/3628824986 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f43d80e6050 con 0x7f43f0108dd0 2024-04-25T10:31:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.994+0000 7f43eeffd700 1 --2- 172.21.15.43:0/3628824986 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f43dc0afc10 0x7f43dc0b2120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f43e0005fd0 tx=0x7f43e0005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.994+0000 7f43ef7fe700 1 --2- 172.21.15.43:0/3628824986 >> [v2:172.21.15.164:6824/1085943743,v1:172.21.15.164:6825/1085943743] conn(0x7f43dc0b9440 0x7f43dc0bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.994+0000 7f43ef7fe700 1 --2- 172.21.15.43:0/3628824986 >> [v2:172.21.15.164:6824/1085943743,v1:172.21.15.164:6825/1085943743] conn(0x7f43dc0b9440 0x7f43dc0bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:12.995+0000 7f43e7fff700 1 -- 172.21.15.43:0/3628824986 <== osd.11 v2:172.21.15.164:6824/1085943743 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f43dc0bbf60 con 0x7f43dc0b9440 2024-04-25T10:31:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.030+0000 7f43e5ffb700 1 -- 172.21.15.43:0/3628824986 --> [v2:172.21.15.164:6824/1085943743,v1:172.21.15.164:6825/1085943743] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f43f0063ff0 con 0x7f43dc0b9440 2024-04-25T10:31:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.031+0000 7f43e7fff700 1 -- 172.21.15.43:0/3628824986 <== osd.11 v2:172.21.15.164:6824/1085943743 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f43f0063ff0 con 0x7f43dc0b9440 2024-04-25T10:31:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.031+0000 7f43e5ffb700 1 -- 172.21.15.43:0/3628824986 >> [v2:172.21.15.164:6824/1085943743,v1:172.21.15.164:6825/1085943743] conn(0x7f43dc0b9440 msgr2=0x7f43dc0bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.031+0000 7f43e5ffb700 1 --2- 172.21.15.43:0/3628824986 >> [v2:172.21.15.164:6824/1085943743,v1:172.21.15.164:6825/1085943743] conn(0x7f43dc0b9440 0x7f43dc0bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.031+0000 7f43e5ffb700 1 -- 172.21.15.43:0/3628824986 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f43dc0afc10 msgr2=0x7f43dc0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.031+0000 7f43e5ffb700 1 --2- 172.21.15.43:0/3628824986 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f43dc0afc10 0x7f43dc0b2120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f43e0005fd0 tx=0x7f43e0005ee0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.031+0000 7f43e5ffb700 1 -- 172.21.15.43:0/3628824986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43f0108dd0 msgr2=0x7f43f019b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.031+0000 7f43e5ffb700 1 --2- 172.21.15.43:0/3628824986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43f0108dd0 0x7f43f019b320 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f43d80045e0 tx=0x7f43d8004610 comp rx=0 tx=0).stop 2024-04-25T10:31:13.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.031+0000 7f43ef7fe700 1 -- 172.21.15.43:0/3628824986 reap_dead start 2024-04-25T10:31:13.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.031+0000 7f43e5ffb700 1 -- 172.21.15.43:0/3628824986 shutdown_connections 2024-04-25T10:31:13.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.031+0000 7f43e5ffb700 1 -- 172.21.15.43:0/3628824986 >> 172.21.15.43:0/3628824986 conn(0x7f43f00ffd70 msgr2=0x7f43f010a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:13.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.031+0000 7f43e5ffb700 1 -- 172.21.15.43:0/3628824986 shutdown_connections 2024-04-25T10:31:13.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:13.031+0000 7f43e5ffb700 1 -- 172.21.15.43:0/3628824986 wait complete. 2024-04-25T10:31:13.827 INFO:teuthology.orchestra.run.smithi043.stdout:309237645334 2024-04-25T10:31:13.827 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T10:31:13.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:13 smithi043 ceph-mon[29478]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T10:31:14.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:13 smithi077 ceph-mon[32772]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T10:31:14.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:13 smithi164 ceph-mon[32713]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T10:31:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.832+0000 7fca74275700 1 -- 172.21.15.43:0/3708721037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c109710 msgr2=0x7fca6c111070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.832+0000 7fca74275700 1 --2- 172.21.15.43:0/3708721037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c109710 0x7fca6c111070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fca5c0099c0 tx=0x7fca5c009cd0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.832+0000 7fca74275700 1 -- 172.21.15.43:0/3708721037 shutdown_connections 2024-04-25T10:31:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.832+0000 7fca74275700 1 --2- 172.21.15.43:0/3708721037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c109710 0x7fca6c111070 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.832+0000 7fca74275700 1 --2- 172.21.15.43:0/3708721037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca6c104490 0x7fca6c1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.832+0000 7fca74275700 1 --2- 172.21.15.43:0/3708721037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca6c108dc0 0x7fca6c1091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.832+0000 7fca74275700 1 -- 172.21.15.43:0/3708721037 >> 172.21.15.43:0/3708721037 conn(0x7fca6c0ffda0 msgr2=0x7fca6c1021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.832+0000 7fca74275700 1 -- 172.21.15.43:0/3708721037 shutdown_connections 2024-04-25T10:31:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.832+0000 7fca74275700 1 -- 172.21.15.43:0/3708721037 wait complete. 2024-04-25T10:31:14.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.833+0000 7fca74275700 1 Processor -- start 2024-04-25T10:31:14.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.833+0000 7fca74275700 1 -- start start 2024-04-25T10:31:14.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.833+0000 7fca74275700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca6c104490 0x7fca6c1a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.833+0000 7fca74275700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c108dc0 0x7fca6c1a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.833+0000 7fca74275700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca6c109710 0x7fca6c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.833+0000 7fca74275700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c114340 con 0x7fca6c108dc0 2024-04-25T10:31:14.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.833+0000 7fca74275700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1141c0 con 0x7fca6c109710 2024-04-25T10:31:14.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.833+0000 7fca74275700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1144c0 con 0x7fca6c104490 2024-04-25T10:31:14.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.833+0000 7fca71810700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c108dc0 0x7fca6c1a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.834+0000 7fca71810700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c108dc0 0x7fca6c1a0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40608/0 (socket says 172.21.15.43:40608) 2024-04-25T10:31:14.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.834+0000 7fca71810700 1 -- 172.21.15.43:0/3827297334 learned_addr learned my addr 172.21.15.43:0/3827297334 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:14.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.834+0000 7fca72812700 1 --2- 172.21.15.43:0/3827297334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca6c109710 0x7fca6c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.834+0000 7fca72011700 1 --2- 172.21.15.43:0/3827297334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca6c104490 0x7fca6c1a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.834+0000 7fca71810700 1 -- 172.21.15.43:0/3827297334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca6c104490 msgr2=0x7fca6c1a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.834+0000 7fca71810700 1 --2- 172.21.15.43:0/3827297334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca6c104490 0x7fca6c1a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.834+0000 7fca71810700 1 -- 172.21.15.43:0/3827297334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca6c109710 msgr2=0x7fca6c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.834+0000 7fca71810700 1 --2- 172.21.15.43:0/3827297334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca6c109710 0x7fca6c19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.834+0000 7fca71810700 1 -- 172.21.15.43:0/3827297334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c19b110 con 0x7fca6c108dc0 2024-04-25T10:31:14.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.834+0000 7fca71810700 1 --2- 172.21.15.43:0/3827297334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c108dc0 0x7fca6c1a0d80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fca6000d980 tx=0x7fca6000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.834+0000 7fca6b7fe700 1 -- 172.21.15.43:0/3827297334 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca60017070 con 0x7fca6c108dc0 2024-04-25T10:31:14.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.834+0000 7fca74275700 1 -- 172.21.15.43:0/3827297334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6c19b400 con 0x7fca6c108dc0 2024-04-25T10:31:14.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.834+0000 7fca6b7fe700 1 -- 172.21.15.43:0/3827297334 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca6000fd40 con 0x7fca6c108dc0 2024-04-25T10:31:14.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.834+0000 7fca74275700 1 -- 172.21.15.43:0/3827297334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca6c0769d0 con 0x7fca6c108dc0 2024-04-25T10:31:14.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.834+0000 7fca6b7fe700 1 -- 172.21.15.43:0/3827297334 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6000e3b0 con 0x7fca6c108dc0 2024-04-25T10:31:14.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.835+0000 7fca74275700 1 -- 172.21.15.43:0/3827297334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fca6c063e20 con 0x7fca6c108dc0 2024-04-25T10:31:14.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.837+0000 7fca6b7fe700 1 -- 172.21.15.43:0/3827297334 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fca60010610 con 0x7fca6c108dc0 2024-04-25T10:31:14.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.837+0000 7fca6b7fe700 1 --2- 172.21.15.43:0/3827297334 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fca540afc90 0x7fca540b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.838+0000 7fca72011700 1 --2- 172.21.15.43:0/3827297334 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fca540afc90 0x7fca540b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.838+0000 7fca6b7fe700 1 -- 172.21.15.43:0/3827297334 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fca600dea70 con 0x7fca6c108dc0 2024-04-25T10:31:14.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.838+0000 7fca6b7fe700 1 --2- 172.21.15.43:0/3827297334 >> [v2:172.21.15.77:6808/3648991803,v1:172.21.15.77:6809/3648991803] conn(0x7fca540b94c0 0x7fca540bb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.838+0000 7fca6b7fe700 1 -- 172.21.15.43:0/3827297334 --> [v2:172.21.15.77:6808/3648991803,v1:172.21.15.77:6809/3648991803] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fca60016ec0 con 0x7fca540b94c0 2024-04-25T10:31:14.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.838+0000 7fca6b7fe700 1 -- 172.21.15.43:0/3827297334 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fca600dee20 con 0x7fca6c108dc0 2024-04-25T10:31:14.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.838+0000 7fca72011700 1 --2- 172.21.15.43:0/3827297334 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fca540afc90 0x7fca540b21a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fca64009510 tx=0x7fca64009230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.838+0000 7fca72812700 1 --2- 172.21.15.43:0/3827297334 >> [v2:172.21.15.77:6808/3648991803,v1:172.21.15.77:6809/3648991803] conn(0x7fca540b94c0 0x7fca540bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.839+0000 7fca72812700 1 --2- 172.21.15.43:0/3827297334 >> [v2:172.21.15.77:6808/3648991803,v1:172.21.15.77:6809/3648991803] conn(0x7fca540b94c0 0x7fca540bb910 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.840+0000 7fca6b7fe700 1 -- 172.21.15.43:0/3827297334 <== osd.4 v2:172.21.15.77:6808/3648991803 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fca60016ec0 con 0x7fca540b94c0 2024-04-25T10:31:14.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.856+0000 7fca74275700 1 -- 172.21.15.43:0/3827297334 --> [v2:172.21.15.77:6808/3648991803,v1:172.21.15.77:6809/3648991803] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fca6c064220 con 0x7fca540b94c0 2024-04-25T10:31:14.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.856+0000 7fca6b7fe700 1 -- 172.21.15.43:0/3827297334 <== osd.4 v2:172.21.15.77:6808/3648991803 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fca6c064220 con 0x7fca540b94c0 2024-04-25T10:31:14.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.857+0000 7fca74275700 1 -- 172.21.15.43:0/3827297334 >> [v2:172.21.15.77:6808/3648991803,v1:172.21.15.77:6809/3648991803] conn(0x7fca540b94c0 msgr2=0x7fca540bb910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.857+0000 7fca74275700 1 --2- 172.21.15.43:0/3827297334 >> [v2:172.21.15.77:6808/3648991803,v1:172.21.15.77:6809/3648991803] conn(0x7fca540b94c0 0x7fca540bb910 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.857+0000 7fca74275700 1 -- 172.21.15.43:0/3827297334 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fca540afc90 msgr2=0x7fca540b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.857+0000 7fca74275700 1 --2- 172.21.15.43:0/3827297334 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fca540afc90 0x7fca540b21a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fca64009510 tx=0x7fca64009230 comp rx=0 tx=0).stop 2024-04-25T10:31:14.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.857+0000 7fca74275700 1 -- 172.21.15.43:0/3827297334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c108dc0 msgr2=0x7fca6c1a0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.857+0000 7fca74275700 1 --2- 172.21.15.43:0/3827297334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c108dc0 0x7fca6c1a0d80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fca6000d980 tx=0x7fca6000dc90 comp rx=0 tx=0).stop 2024-04-25T10:31:14.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.857+0000 7fca72812700 1 -- 172.21.15.43:0/3827297334 reap_dead start 2024-04-25T10:31:14.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.857+0000 7fca74275700 1 -- 172.21.15.43:0/3827297334 shutdown_connections 2024-04-25T10:31:14.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.857+0000 7fca74275700 1 -- 172.21.15.43:0/3827297334 >> 172.21.15.43:0/3827297334 conn(0x7fca6c0ffda0 msgr2=0x7fca6c108530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:14.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.857+0000 7fca74275700 1 -- 172.21.15.43:0/3827297334 shutdown_connections 2024-04-25T10:31:14.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:14.857+0000 7fca74275700 1 -- 172.21.15.43:0/3827297334 wait complete. 2024-04-25T10:31:16.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:15 smithi043 ceph-mon[29478]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T10:31:16.109 INFO:teuthology.orchestra.run.smithi043.stdout:133143986233 2024-04-25T10:31:16.110 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T10:31:16.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:15 smithi077 ceph-mon[32772]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T10:31:16.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:15 smithi164 ceph-mon[32713]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T10:31:16.810 INFO:teuthology.orchestra.run.smithi043.stdout:107374182461 2024-04-25T10:31:16.811 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T10:31:17.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:17 smithi043 ceph-mon[29478]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T10:31:18.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.049+0000 7f438ab22700 1 -- 172.21.15.43:0/2303029276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4384103390 msgr2=0x7f43841037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:18.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.049+0000 7f438ab22700 1 --2- 172.21.15.43:0/2303029276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4384103390 0x7f43841037a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f43780099c0 tx=0x7f4378009cd0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.049+0000 7f438ab22700 1 -- 172.21.15.43:0/2303029276 shutdown_connections 2024-04-25T10:31:18.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.049+0000 7f438ab22700 1 --2- 172.21.15.43:0/2303029276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4384108610 0x7f438410ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.049+0000 7f438ab22700 1 --2- 172.21.15.43:0/2303029276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4384103390 0x7f43841037a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.049+0000 7f438ab22700 1 --2- 172.21.15.43:0/2303029276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4384107cc0 0x7f43841080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.049+0000 7f438ab22700 1 -- 172.21.15.43:0/2303029276 >> 172.21.15.43:0/2303029276 conn(0x7f43840fec90 msgr2=0x7f43841010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:18.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.049+0000 7f438ab22700 1 -- 172.21.15.43:0/2303029276 shutdown_connections 2024-04-25T10:31:18.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.049+0000 7f438ab22700 1 -- 172.21.15.43:0/2303029276 wait complete. 2024-04-25T10:31:18.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.050+0000 7f438ab22700 1 Processor -- start 2024-04-25T10:31:18.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.050+0000 7f438ab22700 1 -- start start 2024-04-25T10:31:18.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.050+0000 7f438ab22700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4384103390 0x7f43841a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:18.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.050+0000 7f438ab22700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4384107cc0 0x7f43841a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:18.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.050+0000 7f438ab22700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4384108610 0x7f438419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:18.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.050+0000 7f438ab22700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4384114390 con 0x7f4384107cc0 2024-04-25T10:31:18.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.050+0000 7f438ab22700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4384114210 con 0x7f4384108610 2024-04-25T10:31:18.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.050+0000 7f438ab22700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4384114510 con 0x7f4384103390 2024-04-25T10:31:18.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.051+0000 7f4383fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4384107cc0 0x7f43841a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:18.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.051+0000 7f4383fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4384107cc0 0x7f43841a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40626/0 (socket says 172.21.15.43:40626) 2024-04-25T10:31:18.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.051+0000 7f4383fff700 1 -- 172.21.15.43:0/2686576463 learned_addr learned my addr 172.21.15.43:0/2686576463 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:18.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.051+0000 7f43888be700 1 --2- 172.21.15.43:0/2686576463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4384103390 0x7f43841a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:18.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.051+0000 7f43890bf700 1 --2- 172.21.15.43:0/2686576463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4384108610 0x7f438419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:18.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.051+0000 7f4383fff700 1 -- 172.21.15.43:0/2686576463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4384103390 msgr2=0x7f43841a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:18.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.051+0000 7f4383fff700 1 --2- 172.21.15.43:0/2686576463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4384103390 0x7f43841a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.051+0000 7f4383fff700 1 -- 172.21.15.43:0/2686576463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4384108610 msgr2=0x7f438419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:18.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.051+0000 7f4383fff700 1 --2- 172.21.15.43:0/2686576463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4384108610 0x7f438419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.051+0000 7f4383fff700 1 -- 172.21.15.43:0/2686576463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f438419b140 con 0x7f4384107cc0 2024-04-25T10:31:18.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.051+0000 7f43888be700 1 --2- 172.21.15.43:0/2686576463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4384103390 0x7f43841a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:18.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.051+0000 7f4383fff700 1 --2- 172.21.15.43:0/2686576463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4384107cc0 0x7f43841a0df0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f437000c9e0 tx=0x7f437000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:18.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.051+0000 7f4381ffb700 1 -- 172.21.15.43:0/2686576463 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370007aa0 con 0x7f4384107cc0 2024-04-25T10:31:18.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.051+0000 7f4381ffb700 1 -- 172.21.15.43:0/2686576463 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4370007c40 con 0x7f4384107cc0 2024-04-25T10:31:18.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.051+0000 7f438ab22700 1 -- 172.21.15.43:0/2686576463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f438419b430 con 0x7f4384107cc0 2024-04-25T10:31:18.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.051+0000 7f4381ffb700 1 -- 172.21.15.43:0/2686576463 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370018660 con 0x7f4384107cc0 2024-04-25T10:31:18.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.051+0000 7f438ab22700 1 -- 172.21.15.43:0/2686576463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43841a7740 con 0x7f4384107cc0 2024-04-25T10:31:18.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.052+0000 7f438ab22700 1 -- 172.21.15.43:0/2686576463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4384063e20 con 0x7f4384107cc0 2024-04-25T10:31:18.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.054+0000 7f4381ffb700 1 -- 172.21.15.43:0/2686576463 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f4370018800 con 0x7f4384107cc0 2024-04-25T10:31:18.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.055+0000 7f4381ffb700 1 --2- 172.21.15.43:0/2686576463 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f43740afc90 0x7f43740b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:18.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.055+0000 7f43888be700 1 --2- 172.21.15.43:0/2686576463 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f43740afc90 0x7f43740b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:18.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.055+0000 7f4381ffb700 1 -- 172.21.15.43:0/2686576463 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f43700defe0 con 0x7f4384107cc0 2024-04-25T10:31:18.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.055+0000 7f4381ffb700 1 --2- 172.21.15.43:0/2686576463 >> [v2:172.21.15.164:6800/3408162352,v1:172.21.15.164:6801/3408162352] conn(0x7f43740b94c0 0x7f43740bb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:18.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.055+0000 7f4381ffb700 1 -- 172.21.15.43:0/2686576463 --> [v2:172.21.15.164:6800/3408162352,v1:172.21.15.164:6801/3408162352] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4370015e70 con 0x7f43740b94c0 2024-04-25T10:31:18.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.055+0000 7f4381ffb700 1 -- 172.21.15.43:0/2686576463 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f43700df390 con 0x7f4384107cc0 2024-04-25T10:31:18.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.055+0000 7f43888be700 1 --2- 172.21.15.43:0/2686576463 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f43740afc90 0x7f43740b21a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4378000c00 tx=0x7f437801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:18.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.055+0000 7f43890bf700 1 --2- 172.21.15.43:0/2686576463 >> [v2:172.21.15.164:6800/3408162352,v1:172.21.15.164:6801/3408162352] conn(0x7f43740b94c0 0x7f43740bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:18.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.056+0000 7f43890bf700 1 --2- 172.21.15.43:0/2686576463 >> [v2:172.21.15.164:6800/3408162352,v1:172.21.15.164:6801/3408162352] conn(0x7f43740b94c0 0x7f43740bb910 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:18.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.057+0000 7f4381ffb700 1 -- 172.21.15.43:0/2686576463 <== osd.2 v2:172.21.15.164:6800/3408162352 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4370015e70 con 0x7f43740b94c0 2024-04-25T10:31:18.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.073+0000 7f438ab22700 1 -- 172.21.15.43:0/2686576463 --> [v2:172.21.15.164:6800/3408162352,v1:172.21.15.164:6801/3408162352] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4384064220 con 0x7f43740b94c0 2024-04-25T10:31:18.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.073+0000 7f4381ffb700 1 -- 172.21.15.43:0/2686576463 <== osd.2 v2:172.21.15.164:6800/3408162352 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4384064220 con 0x7f43740b94c0 2024-04-25T10:31:18.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.074+0000 7f438ab22700 1 -- 172.21.15.43:0/2686576463 >> [v2:172.21.15.164:6800/3408162352,v1:172.21.15.164:6801/3408162352] conn(0x7f43740b94c0 msgr2=0x7f43740bb910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:18.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.074+0000 7f438ab22700 1 --2- 172.21.15.43:0/2686576463 >> [v2:172.21.15.164:6800/3408162352,v1:172.21.15.164:6801/3408162352] conn(0x7f43740b94c0 0x7f43740bb910 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.074+0000 7f438ab22700 1 -- 172.21.15.43:0/2686576463 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f43740afc90 msgr2=0x7f43740b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:18.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.074+0000 7f438ab22700 1 --2- 172.21.15.43:0/2686576463 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f43740afc90 0x7f43740b21a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4378000c00 tx=0x7f437801a040 comp rx=0 tx=0).stop 2024-04-25T10:31:18.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.074+0000 7f438ab22700 1 -- 172.21.15.43:0/2686576463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4384107cc0 msgr2=0x7f43841a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:18.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.074+0000 7f438ab22700 1 --2- 172.21.15.43:0/2686576463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4384107cc0 0x7f43841a0df0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f437000c9e0 tx=0x7f437000cda0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.074+0000 7f43890bf700 1 -- 172.21.15.43:0/2686576463 reap_dead start 2024-04-25T10:31:18.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.074+0000 7f438ab22700 1 -- 172.21.15.43:0/2686576463 shutdown_connections 2024-04-25T10:31:18.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.074+0000 7f438ab22700 1 -- 172.21.15.43:0/2686576463 >> 172.21.15.43:0/2686576463 conn(0x7f43840fec90 msgr2=0x7f43841074a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:18.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.074+0000 7f438ab22700 1 -- 172.21.15.43:0/2686576463 shutdown_connections 2024-04-25T10:31:18.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:17.074+0000 7f438ab22700 1 -- 172.21.15.43:0/2686576463 wait complete. 2024-04-25T10:31:18.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:17 smithi077 ceph-mon[32772]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T10:31:18.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:17 smithi164 ceph-mon[32713]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T10:31:18.486 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:31:18.493 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:31:18.493 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:31:18.502 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:31:20.173 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:19 smithi043 ceph-mon[29478]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T10:31:20.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:19 smithi077 ceph-mon[32772]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T10:31:20.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:19 smithi164 ceph-mon[32713]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T10:31:21.498 INFO:teuthology.orchestra.run.smithi043.stdout:77309411396 2024-04-25T10:31:21.498 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T10:31:22.005 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:21 smithi043 ceph-mon[29478]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T10:31:22.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:21 smithi077 ceph-mon[32772]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T10:31:22.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:21 smithi164 ceph-mon[32713]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T10:31:23.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.285+0000 7f56f7501700 1 -- 172.21.15.43:0/478953685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f0103350 msgr2=0x7f56f0103760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.285+0000 7f56f7501700 1 --2- 172.21.15.43:0/478953685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f0103350 0x7f56f0103760 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f56ec0099c0 tx=0x7f56ec009cd0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.285+0000 7f56f7501700 1 -- 172.21.15.43:0/478953685 shutdown_connections 2024-04-25T10:31:23.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.285+0000 7f56f7501700 1 --2- 172.21.15.43:0/478953685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56f01085d0 0x7f56f010ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.285+0000 7f56f7501700 1 --2- 172.21.15.43:0/478953685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f0103350 0x7f56f0103760 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.285+0000 7f56f7501700 1 --2- 172.21.15.43:0/478953685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56f0107c80 0x7f56f0108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.285+0000 7f56f7501700 1 -- 172.21.15.43:0/478953685 >> 172.21.15.43:0/478953685 conn(0x7f56f00fec30 msgr2=0x7f56f0101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:23.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.285+0000 7f56f7501700 1 -- 172.21.15.43:0/478953685 shutdown_connections 2024-04-25T10:31:23.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.285+0000 7f56f7501700 1 -- 172.21.15.43:0/478953685 wait complete. 2024-04-25T10:31:23.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.286+0000 7f56f7501700 1 Processor -- start 2024-04-25T10:31:23.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.286+0000 7f56f7501700 1 -- start start 2024-04-25T10:31:23.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.286+0000 7f56f7501700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56f0103350 0x7f56f019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.286+0000 7f56f7501700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f0107c80 0x7f56f019c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.286+0000 7f56f7501700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56f01085d0 0x7f56f01a6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.286+0000 7f56f7501700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f56f0113200 con 0x7f56f0107c80 2024-04-25T10:31:23.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.286+0000 7f56f7501700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f56f0113080 con 0x7f56f0103350 2024-04-25T10:31:23.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.286+0000 7f56f7501700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f56f0113380 con 0x7f56f01085d0 2024-04-25T10:31:23.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.286+0000 7f56f4a9c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f0107c80 0x7f56f019c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.286+0000 7f56f4a9c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f0107c80 0x7f56f019c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36716/0 (socket says 172.21.15.43:36716) 2024-04-25T10:31:23.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.286+0000 7f56f4a9c700 1 -- 172.21.15.43:0/3587318940 learned_addr learned my addr 172.21.15.43:0/3587318940 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:23.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.286+0000 7f56f5a9e700 1 --2- 172.21.15.43:0/3587318940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56f01085d0 0x7f56f01a6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.287+0000 7f56f529d700 1 --2- 172.21.15.43:0/3587318940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56f0103350 0x7f56f019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.287+0000 7f56f4a9c700 1 -- 172.21.15.43:0/3587318940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56f01085d0 msgr2=0x7f56f01a6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.287+0000 7f56f4a9c700 1 --2- 172.21.15.43:0/3587318940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56f01085d0 0x7f56f01a6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.287+0000 7f56f4a9c700 1 -- 172.21.15.43:0/3587318940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56f0103350 msgr2=0x7f56f019bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.287+0000 7f56f4a9c700 1 --2- 172.21.15.43:0/3587318940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56f0103350 0x7f56f019bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.287+0000 7f56f4a9c700 1 -- 172.21.15.43:0/3587318940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f019c8b0 con 0x7f56f0107c80 2024-04-25T10:31:23.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.287+0000 7f56f4a9c700 1 --2- 172.21.15.43:0/3587318940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f0107c80 0x7f56f019c000 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f56e400d980 tx=0x7f56e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:23.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.287+0000 7f56e27fc700 1 -- 172.21.15.43:0/3587318940 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e4017070 con 0x7f56f0107c80 2024-04-25T10:31:23.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.287+0000 7f56f7501700 1 -- 172.21.15.43:0/3587318940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f01a71a0 con 0x7f56f0107c80 2024-04-25T10:31:23.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.287+0000 7f56e27fc700 1 -- 172.21.15.43:0/3587318940 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56e400fd40 con 0x7f56f0107c80 2024-04-25T10:31:23.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.287+0000 7f56e27fc700 1 -- 172.21.15.43:0/3587318940 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e400e3b0 con 0x7f56f0107c80 2024-04-25T10:31:23.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.287+0000 7f56f7501700 1 -- 172.21.15.43:0/3587318940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f01a76e0 con 0x7f56f0107c80 2024-04-25T10:31:23.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.288+0000 7f56f7501700 1 -- 172.21.15.43:0/3587318940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f56f0063da0 con 0x7f56f0107c80 2024-04-25T10:31:23.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.290+0000 7f56e27fc700 1 -- 172.21.15.43:0/3587318940 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f56e400f890 con 0x7f56f0107c80 2024-04-25T10:31:23.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.290+0000 7f56e27fc700 1 --2- 172.21.15.43:0/3587318940 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f56dc0afc10 0x7f56dc0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.291+0000 7f56f529d700 1 --2- 172.21.15.43:0/3587318940 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f56dc0afc10 0x7f56dc0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.291+0000 7f56e27fc700 1 -- 172.21.15.43:0/3587318940 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f56e40decc0 con 0x7f56f0107c80 2024-04-25T10:31:23.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.291+0000 7f56e27fc700 1 --2- 172.21.15.43:0/3587318940 >> [v2:172.21.15.77:6824/1788538313,v1:172.21.15.77:6825/1788538313] conn(0x7f56dc0b9440 0x7f56dc0bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.291+0000 7f56e27fc700 1 -- 172.21.15.43:0/3587318940 --> [v2:172.21.15.77:6824/1788538313,v1:172.21.15.77:6825/1788538313] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f56dc0bbf60 con 0x7f56dc0b9440 2024-04-25T10:31:23.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.291+0000 7f56e27fc700 1 -- 172.21.15.43:0/3587318940 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f56e40df1a0 con 0x7f56f0107c80 2024-04-25T10:31:23.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.291+0000 7f56f529d700 1 --2- 172.21.15.43:0/3587318940 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f56dc0afc10 0x7f56dc0b2120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f56ec000c00 tx=0x7f56ec01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:23.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.291+0000 7f56f5a9e700 1 --2- 172.21.15.43:0/3587318940 >> [v2:172.21.15.77:6824/1788538313,v1:172.21.15.77:6825/1788538313] conn(0x7f56dc0b9440 0x7f56dc0bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.292+0000 7f56f5a9e700 1 --2- 172.21.15.43:0/3587318940 >> [v2:172.21.15.77:6824/1788538313,v1:172.21.15.77:6825/1788538313] conn(0x7f56dc0b9440 0x7f56dc0bb890 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:23.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.293+0000 7f56e27fc700 1 -- 172.21.15.43:0/3587318940 <== osd.10 v2:172.21.15.77:6824/1788538313 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f56dc0bbf60 con 0x7f56dc0b9440 2024-04-25T10:31:23.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.309+0000 7f56f7501700 1 -- 172.21.15.43:0/3587318940 --> [v2:172.21.15.77:6824/1788538313,v1:172.21.15.77:6825/1788538313] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f56f00641a0 con 0x7f56dc0b9440 2024-04-25T10:31:23.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.310+0000 7f56e27fc700 1 -- 172.21.15.43:0/3587318940 <== osd.10 v2:172.21.15.77:6824/1788538313 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f56f00641a0 con 0x7f56dc0b9440 2024-04-25T10:31:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.310+0000 7f56f7501700 1 -- 172.21.15.43:0/3587318940 >> [v2:172.21.15.77:6824/1788538313,v1:172.21.15.77:6825/1788538313] conn(0x7f56dc0b9440 msgr2=0x7f56dc0bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.310+0000 7f56f7501700 1 --2- 172.21.15.43:0/3587318940 >> [v2:172.21.15.77:6824/1788538313,v1:172.21.15.77:6825/1788538313] conn(0x7f56dc0b9440 0x7f56dc0bb890 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.310+0000 7f56f7501700 1 -- 172.21.15.43:0/3587318940 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f56dc0afc10 msgr2=0x7f56dc0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.310+0000 7f56f7501700 1 --2- 172.21.15.43:0/3587318940 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f56dc0afc10 0x7f56dc0b2120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f56ec000c00 tx=0x7f56ec01a040 comp rx=0 tx=0).stop 2024-04-25T10:31:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.310+0000 7f56f7501700 1 -- 172.21.15.43:0/3587318940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f0107c80 msgr2=0x7f56f019c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.310+0000 7f56f7501700 1 --2- 172.21.15.43:0/3587318940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f0107c80 0x7f56f019c000 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f56e400d980 tx=0x7f56e400dc90 comp rx=0 tx=0).stop 2024-04-25T10:31:23.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.310+0000 7f56f5a9e700 1 -- 172.21.15.43:0/3587318940 reap_dead start 2024-04-25T10:31:23.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.310+0000 7f56f7501700 1 -- 172.21.15.43:0/3587318940 shutdown_connections 2024-04-25T10:31:23.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.310+0000 7f56f7501700 1 -- 172.21.15.43:0/3587318940 >> 172.21.15.43:0/3587318940 conn(0x7f56f00fec30 msgr2=0x7f56f0101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:23.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.310+0000 7f56f7501700 1 -- 172.21.15.43:0/3587318940 shutdown_connections 2024-04-25T10:31:23.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.310+0000 7f56f7501700 1 -- 172.21.15.43:0/3587318940 wait complete. 2024-04-25T10:31:23.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.510+0000 7f121d07c700 1 -- 172.21.15.43:0/2760856220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1218108e10 msgr2=0x7f1218109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.510+0000 7f121d07c700 1 --2- 172.21.15.43:0/2760856220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1218108e10 0x7f1218109220 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f120c009990 tx=0x7f120c009ca0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.510+0000 7f121d07c700 1 -- 172.21.15.43:0/2760856220 shutdown_connections 2024-04-25T10:31:23.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.510+0000 7f121d07c700 1 --2- 172.21.15.43:0/2760856220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1218109760 0x7f12181110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.510+0000 7f121d07c700 1 --2- 172.21.15.43:0/2760856220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12181029f0 0x7f1218102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.510+0000 7f121d07c700 1 --2- 172.21.15.43:0/2760856220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1218108e10 0x7f1218109220 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.510+0000 7f121d07c700 1 -- 172.21.15.43:0/2760856220 >> 172.21.15.43:0/2760856220 conn(0x7f12180fe370 msgr2=0x7f12181007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:23.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.510+0000 7f121d07c700 1 -- 172.21.15.43:0/2760856220 shutdown_connections 2024-04-25T10:31:23.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.510+0000 7f121d07c700 1 -- 172.21.15.43:0/2760856220 wait complete. 2024-04-25T10:31:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.511+0000 7f121d07c700 1 Processor -- start 2024-04-25T10:31:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.511+0000 7f121d07c700 1 -- start start 2024-04-25T10:31:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.512+0000 7f121d07c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12181029f0 0x7f12181a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.512+0000 7f121d07c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1218108e10 0x7f12181a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.512+0000 7f121d07c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1218109760 0x7f121819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.512+0000 7f121d07c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1218114320 con 0x7f1218109760 2024-04-25T10:31:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.512+0000 7f121d07c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f12181141a0 con 0x7f1218108e10 2024-04-25T10:31:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.512+0000 7f121d07c700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12181144a0 con 0x7f12181029f0 2024-04-25T10:31:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.512+0000 7f121759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1218109760 0x7f121819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.512+0000 7f121759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1218109760 0x7f121819ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36732/0 (socket says 172.21.15.43:36732) 2024-04-25T10:31:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.512+0000 7f121759e700 1 -- 172.21.15.43:0/2575235855 learned_addr learned my addr 172.21.15.43:0/2575235855 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.512+0000 7f1216d9d700 1 --2- 172.21.15.43:0/2575235855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12181029f0 0x7f12181a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.512+0000 7f121659c700 1 --2- 172.21.15.43:0/2575235855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1218108e10 0x7f12181a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.512+0000 7f1216d9d700 1 -- 172.21.15.43:0/2575235855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1218108e10 msgr2=0x7f12181a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.512+0000 7f1216d9d700 1 --2- 172.21.15.43:0/2575235855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1218108e10 0x7f12181a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.512+0000 7f1216d9d700 1 -- 172.21.15.43:0/2575235855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1218109760 msgr2=0x7f121819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.512+0000 7f1216d9d700 1 --2- 172.21.15.43:0/2575235855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1218109760 0x7f121819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.512+0000 7f1216d9d700 1 -- 172.21.15.43:0/2575235855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121819b270 con 0x7f12181029f0 2024-04-25T10:31:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.513+0000 7f1216d9d700 1 --2- 172.21.15.43:0/2575235855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12181029f0 0x7f12181a09b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f121806bae0 tx=0x7f120800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.513+0000 7f11fffff700 1 -- 172.21.15.43:0/2575235855 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208016070 con 0x7f12181029f0 2024-04-25T10:31:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.513+0000 7f121d07c700 1 -- 172.21.15.43:0/2575235855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121819b560 con 0x7f12181029f0 2024-04-25T10:31:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.513+0000 7f11fffff700 1 -- 172.21.15.43:0/2575235855 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f120800fd40 con 0x7f12181029f0 2024-04-25T10:31:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.513+0000 7f11fffff700 1 -- 172.21.15.43:0/2575235855 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120800e3b0 con 0x7f12181029f0 2024-04-25T10:31:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.513+0000 7f121d07c700 1 -- 172.21.15.43:0/2575235855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12181a7760 con 0x7f12181029f0 2024-04-25T10:31:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.514+0000 7f121d07c700 1 -- 172.21.15.43:0/2575235855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1204005330 con 0x7f12181029f0 2024-04-25T10:31:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.516+0000 7f11fffff700 1 -- 172.21.15.43:0/2575235855 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f1208019070 con 0x7f12181029f0 2024-04-25T10:31:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.516+0000 7f11fffff700 1 --2- 172.21.15.43:0/2575235855 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f12000afc10 0x7f12000b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.517+0000 7f121659c700 1 --2- 172.21.15.43:0/2575235855 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f12000afc10 0x7f12000b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.517+0000 7f11fffff700 1 -- 172.21.15.43:0/2575235855 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f12080de6c0 con 0x7f12181029f0 2024-04-25T10:31:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.517+0000 7f121659c700 1 --2- 172.21.15.43:0/2575235855 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f12000afc10 0x7f12000b2120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f120c0059d0 tx=0x7f120c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:23.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.520+0000 7f11fffff700 1 -- 172.21.15.43:0/2575235855 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f12080a71f0 con 0x7f12181029f0 2024-04-25T10:31:23.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.735+0000 7f121d07c700 1 -- 172.21.15.43:0/2575235855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f12040058f0 con 0x7f12181029f0 2024-04-25T10:31:23.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.736+0000 7f11fffff700 1 -- 172.21.15.43:0/2575235855 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f12080a6af0 con 0x7f12181029f0 2024-04-25T10:31:23.738 INFO:teuthology.orchestra.run.smithi043.stdout:309237645336 2024-04-25T10:31:23.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.738+0000 7f121d07c700 1 -- 172.21.15.43:0/2575235855 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f12000afc10 msgr2=0x7f12000b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.738+0000 7f121d07c700 1 --2- 172.21.15.43:0/2575235855 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f12000afc10 0x7f12000b2120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f120c0059d0 tx=0x7f120c005960 comp rx=0 tx=0).stop 2024-04-25T10:31:23.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.738+0000 7f121d07c700 1 -- 172.21.15.43:0/2575235855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12181029f0 msgr2=0x7f12181a09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.738+0000 7f121d07c700 1 --2- 172.21.15.43:0/2575235855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12181029f0 0x7f12181a09b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f121806bae0 tx=0x7f120800dc30 comp rx=0 tx=0).stop 2024-04-25T10:31:23.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.738+0000 7f121d07c700 1 -- 172.21.15.43:0/2575235855 shutdown_connections 2024-04-25T10:31:23.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.738+0000 7f121d07c700 1 --2- 172.21.15.43:0/2575235855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1218109760 0x7f121819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.738+0000 7f121d07c700 1 --2- 172.21.15.43:0/2575235855 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f12000afc10 0x7f12000b2120 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.738+0000 7f121d07c700 1 --2- 172.21.15.43:0/2575235855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12181029f0 0x7f12181a09b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.738+0000 7f121d07c700 1 --2- 172.21.15.43:0/2575235855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1218108e10 0x7f12181a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.738+0000 7f121d07c700 1 -- 172.21.15.43:0/2575235855 >> 172.21.15.43:0/2575235855 conn(0x7f12180fe370 msgr2=0x7f12180feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:23.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.738+0000 7f121d07c700 1 -- 172.21.15.43:0/2575235855 shutdown_connections 2024-04-25T10:31:23.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:23.738+0000 7f121d07c700 1 -- 172.21.15.43:0/2575235855 wait complete. 2024-04-25T10:31:24.025 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:23 smithi043 ceph-mon[29478]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:31:24.026 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:23 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2575235855' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T10:31:24.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.056+0000 7f0e4febc700 1 -- 172.21.15.43:0/3929099150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e48105ca0 msgr2=0x7f0e4810e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:24.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.056+0000 7f0e4febc700 1 --2- 172.21.15.43:0/3929099150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e48105ca0 0x7f0e4810e1f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009990 tx=0x7f0e38009ca0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.056+0000 7f0e4febc700 1 -- 172.21.15.43:0/3929099150 shutdown_connections 2024-04-25T10:31:24.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.056+0000 7f0e4febc700 1 --2- 172.21.15.43:0/3929099150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e4810e7c0 0x7f0e48110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.056+0000 7f0e4febc700 1 --2- 172.21.15.43:0/3929099150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e48105350 0x7f0e48105760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.056+0000 7f0e4febc700 1 --2- 172.21.15.43:0/3929099150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e48105ca0 0x7f0e4810e1f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.056+0000 7f0e4febc700 1 -- 172.21.15.43:0/3929099150 >> 172.21.15.43:0/3929099150 conn(0x7f0e480fee00 msgr2=0x7f0e48101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:24.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.056+0000 7f0e4febc700 1 -- 172.21.15.43:0/3929099150 shutdown_connections 2024-04-25T10:31:24.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.056+0000 7f0e4febc700 1 -- 172.21.15.43:0/3929099150 wait complete. 2024-04-25T10:31:24.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.057+0000 7f0e4febc700 1 Processor -- start 2024-04-25T10:31:24.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.057+0000 7f0e4febc700 1 -- start start 2024-04-25T10:31:24.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.058+0000 7f0e4febc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e48105350 0x7f0e481a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:24.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.058+0000 7f0e4febc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e48105ca0 0x7f0e481a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:24.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.058+0000 7f0e4febc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e4810e7c0 0x7f0e4819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:24.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.058+0000 7f0e4febc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0e48114320 con 0x7f0e48105350 2024-04-25T10:31:24.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.058+0000 7f0e4febc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0e481141a0 con 0x7f0e4810e7c0 2024-04-25T10:31:24.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.058+0000 7f0e4febc700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0e481144a0 con 0x7f0e48105ca0 2024-04-25T10:31:24.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.058+0000 7f0e4dc58700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e48105350 0x7f0e481a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:24.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.058+0000 7f0e4dc58700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e48105350 0x7f0e481a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36760/0 (socket says 172.21.15.43:36760) 2024-04-25T10:31:24.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.058+0000 7f0e4dc58700 1 -- 172.21.15.43:0/2989802306 learned_addr learned my addr 172.21.15.43:0/2989802306 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:24.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.058+0000 7f0e4e459700 1 --2- 172.21.15.43:0/2989802306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e4810e7c0 0x7f0e4819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:24.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.058+0000 7f0e4d457700 1 --2- 172.21.15.43:0/2989802306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e48105ca0 0x7f0e481a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:24.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.058+0000 7f0e4dc58700 1 -- 172.21.15.43:0/2989802306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e48105ca0 msgr2=0x7f0e481a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:24.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.058+0000 7f0e4dc58700 1 --2- 172.21.15.43:0/2989802306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e48105ca0 0x7f0e481a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.058+0000 7f0e4dc58700 1 -- 172.21.15.43:0/2989802306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e4810e7c0 msgr2=0x7f0e4819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:24.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.058+0000 7f0e4dc58700 1 --2- 172.21.15.43:0/2989802306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e4810e7c0 0x7f0e4819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.058+0000 7f0e4dc58700 1 -- 172.21.15.43:0/2989802306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4819b170 con 0x7f0e48105350 2024-04-25T10:31:24.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.058+0000 7f0e4d457700 1 --2- 172.21.15.43:0/2989802306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e48105ca0 0x7f0e481a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:24.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.058+0000 7f0e4dc58700 1 --2- 172.21.15.43:0/2989802306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e48105350 0x7f0e481a08c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0e4400c9b0 tx=0x7f0e4400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:24.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.059+0000 7f0e3effd700 1 -- 172.21.15.43:0/2989802306 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44007b00 con 0x7f0e48105350 2024-04-25T10:31:24.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.059+0000 7f0e3effd700 1 -- 172.21.15.43:0/2989802306 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e44007ca0 con 0x7f0e48105350 2024-04-25T10:31:24.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.059+0000 7f0e4febc700 1 -- 172.21.15.43:0/2989802306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4819b460 con 0x7f0e48105350 2024-04-25T10:31:24.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.059+0000 7f0e3effd700 1 -- 172.21.15.43:0/2989802306 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44018660 con 0x7f0e48105350 2024-04-25T10:31:24.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.059+0000 7f0e4febc700 1 -- 172.21.15.43:0/2989802306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e481a7770 con 0x7f0e48105350 2024-04-25T10:31:24.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.060+0000 7f0e4febc700 1 -- 172.21.15.43:0/2989802306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e48051190 con 0x7f0e48105350 2024-04-25T10:31:24.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.061+0000 7f0e3effd700 1 -- 172.21.15.43:0/2989802306 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f0e4400f450 con 0x7f0e48105350 2024-04-25T10:31:24.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.062+0000 7f0e3effd700 1 --2- 172.21.15.43:0/2989802306 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f0e340afc90 0x7f0e340b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:24.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.062+0000 7f0e4d457700 1 --2- 172.21.15.43:0/2989802306 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f0e340afc90 0x7f0e340b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:24.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.062+0000 7f0e3effd700 1 -- 172.21.15.43:0/2989802306 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f0e440def20 con 0x7f0e48105350 2024-04-25T10:31:24.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.062+0000 7f0e4d457700 1 --2- 172.21.15.43:0/2989802306 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f0e340afc90 0x7f0e340b21a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800b2e0 tx=0x7f0e38005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:24.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.066+0000 7f0e3effd700 1 -- 172.21.15.43:0/2989802306 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e44014030 con 0x7f0e48105350 2024-04-25T10:31:24.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.275+0000 7f0e4febc700 1 -- 172.21.15.43:0/2989802306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f0e4819bff0 con 0x7f0e48105350 2024-04-25T10:31:24.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.275+0000 7f0e3effd700 1 -- 172.21.15.43:0/2989802306 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0e440a7b60 con 0x7f0e48105350 2024-04-25T10:31:24.278 INFO:teuthology.orchestra.run.smithi043.stdout:180388626479 2024-04-25T10:31:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.277+0000 7f0e4febc700 1 -- 172.21.15.43:0/2989802306 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f0e340afc90 msgr2=0x7f0e340b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.277+0000 7f0e4febc700 1 --2- 172.21.15.43:0/2989802306 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f0e340afc90 0x7f0e340b21a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800b2e0 tx=0x7f0e38005fb0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.277+0000 7f0e4febc700 1 -- 172.21.15.43:0/2989802306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e48105350 msgr2=0x7f0e481a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.277+0000 7f0e4febc700 1 --2- 172.21.15.43:0/2989802306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e48105350 0x7f0e481a08c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0e4400c9b0 tx=0x7f0e4400ccc0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.277+0000 7f0e4febc700 1 -- 172.21.15.43:0/2989802306 shutdown_connections 2024-04-25T10:31:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.277+0000 7f0e4febc700 1 --2- 172.21.15.43:0/2989802306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e4810e7c0 0x7f0e4819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.277+0000 7f0e4febc700 1 --2- 172.21.15.43:0/2989802306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e48105350 0x7f0e481a08c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.277+0000 7f0e4febc700 1 --2- 172.21.15.43:0/2989802306 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f0e340afc90 0x7f0e340b21a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.277+0000 7f0e4febc700 1 --2- 172.21.15.43:0/2989802306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e48105ca0 0x7f0e481a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.277+0000 7f0e4febc700 1 -- 172.21.15.43:0/2989802306 >> 172.21.15.43:0/2989802306 conn(0x7f0e480fee00 msgr2=0x7f0e48100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.278+0000 7f0e4febc700 1 -- 172.21.15.43:0/2989802306 shutdown_connections 2024-04-25T10:31:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:24.278+0000 7f0e4febc700 1 -- 172.21.15.43:0/2989802306 wait complete. 2024-04-25T10:31:24.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:23 smithi077 ceph-mon[32772]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:31:24.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:23 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2575235855' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T10:31:24.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:23 smithi164 ceph-mon[32713]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:31:24.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:23 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2575235855' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T10:31:24.636 INFO:teuthology.orchestra.run.smithi043.stdout:283467841564 2024-04-25T10:31:24.637 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T10:31:25.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:24 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2989802306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T10:31:25.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:24 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2989802306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T10:31:25.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:24 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2989802306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T10:31:25.431 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626475 got 180388626479 for osd.6 2024-04-25T10:31:25.431 DEBUG:teuthology.parallel:result is None 2024-04-25T10:31:25.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.866+0000 7fa8af43b700 1 -- 172.21.15.43:0/3735008759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8a8110b00 msgr2=0x7fa8a8112f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:25.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.866+0000 7fa8af43b700 1 --2- 172.21.15.43:0/3735008759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8a8110b00 0x7fa8a8112f20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000b0a0 tx=0x7fa8a000b3b0 comp rx=0 tx=0).stop 2024-04-25T10:31:25.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.866+0000 7fa8af43b700 1 -- 172.21.15.43:0/3735008759 shutdown_connections 2024-04-25T10:31:25.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.866+0000 7fa8af43b700 1 --2- 172.21.15.43:0/3735008759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8a8110b00 0x7fa8a8112f20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:25.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.866+0000 7fa8af43b700 1 --2- 172.21.15.43:0/3735008759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8a8102f50 0x7fa8a8103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:25.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.866+0000 7fa8af43b700 1 --2- 172.21.15.43:0/3735008759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8a81038a0 0x7fa8a81105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:25.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.866+0000 7fa8af43b700 1 -- 172.21.15.43:0/3735008759 >> 172.21.15.43:0/3735008759 conn(0x7fa8a80fecc0 msgr2=0x7fa8a81010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:25.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.866+0000 7fa8af43b700 1 -- 172.21.15.43:0/3735008759 shutdown_connections 2024-04-25T10:31:25.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.866+0000 7fa8af43b700 1 -- 172.21.15.43:0/3735008759 wait complete. 2024-04-25T10:31:25.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.867+0000 7fa8af43b700 1 Processor -- start 2024-04-25T10:31:25.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.867+0000 7fa8af43b700 1 -- start start 2024-04-25T10:31:25.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.867+0000 7fa8af43b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8a8102f50 0x7fa8a819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:25.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.867+0000 7fa8af43b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8a81038a0 0x7fa8a819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:25.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.867+0000 7fa8af43b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8a8110b00 0x7fa8a81a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:25.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.867+0000 7fa8af43b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a81165d0 con 0x7fa8a8102f50 2024-04-25T10:31:25.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.867+0000 7fa8af43b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a8116450 con 0x7fa8a8110b00 2024-04-25T10:31:25.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.867+0000 7fa8af43b700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a8116750 con 0x7fa8a81038a0 2024-04-25T10:31:25.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.868+0000 7fa8ad1d7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8a8102f50 0x7fa8a819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:25.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.868+0000 7fa8ad1d7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8a8102f50 0x7fa8a819cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36790/0 (socket says 172.21.15.43:36790) 2024-04-25T10:31:25.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.868+0000 7fa8ad1d7700 1 -- 172.21.15.43:0/3175377228 learned_addr learned my addr 172.21.15.43:0/3175377228 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:25.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.868+0000 7fa8ad9d8700 1 --2- 172.21.15.43:0/3175377228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8a8110b00 0x7fa8a81a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:25.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.868+0000 7fa8ac9d6700 1 --2- 172.21.15.43:0/3175377228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8a81038a0 0x7fa8a819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:25.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.868+0000 7fa8ad1d7700 1 -- 172.21.15.43:0/3175377228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8a81038a0 msgr2=0x7fa8a819d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:25.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.868+0000 7fa8ad1d7700 1 --2- 172.21.15.43:0/3175377228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8a81038a0 0x7fa8a819d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:25.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.868+0000 7fa8ad1d7700 1 -- 172.21.15.43:0/3175377228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8a8110b00 msgr2=0x7fa8a81a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:25.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.868+0000 7fa8ad1d7700 1 --2- 172.21.15.43:0/3175377228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8a8110b00 0x7fa8a81a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:25.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.868+0000 7fa8ad1d7700 1 -- 172.21.15.43:0/3175377228 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a81a1cc0 con 0x7fa8a8102f50 2024-04-25T10:31:25.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.868+0000 7fa8ad1d7700 1 --2- 172.21.15.43:0/3175377228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8a8102f50 0x7fa8a819cea0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400c9b0 tx=0x7fa8a400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:25.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.868+0000 7fa89e7fc700 1 -- 172.21.15.43:0/3175377228 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4007b00 con 0x7fa8a8102f50 2024-04-25T10:31:25.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.868+0000 7fa89e7fc700 1 -- 172.21.15.43:0/3175377228 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8a400ce50 con 0x7fa8a8102f50 2024-04-25T10:31:25.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.868+0000 7fa8af43b700 1 -- 172.21.15.43:0/3175377228 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a81a1fb0 con 0x7fa8a8102f50 2024-04-25T10:31:25.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.868+0000 7fa89e7fc700 1 -- 172.21.15.43:0/3175377228 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4018660 con 0x7fa8a8102f50 2024-04-25T10:31:25.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.868+0000 7fa8af43b700 1 -- 172.21.15.43:0/3175377228 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a81a24f0 con 0x7fa8a8102f50 2024-04-25T10:31:25.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.869+0000 7fa8af43b700 1 -- 172.21.15.43:0/3175377228 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8a8051190 con 0x7fa8a8102f50 2024-04-25T10:31:25.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.871+0000 7fa89e7fc700 1 -- 172.21.15.43:0/3175377228 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fa8a400f450 con 0x7fa8a8102f50 2024-04-25T10:31:25.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.872+0000 7fa89e7fc700 1 --2- 172.21.15.43:0/3175377228 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fa8940afc10 0x7fa8940b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:25.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.872+0000 7fa8ac9d6700 1 --2- 172.21.15.43:0/3175377228 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fa8940afc10 0x7fa8940b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:25.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.872+0000 7fa89e7fc700 1 -- 172.21.15.43:0/3175377228 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fa8a40def30 con 0x7fa8a8102f50 2024-04-25T10:31:25.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.872+0000 7fa8ac9d6700 1 --2- 172.21.15.43:0/3175377228 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fa8940afc10 0x7fa8940b2120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8113030 tx=0x7fa898009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:25.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:25.875+0000 7fa89e7fc700 1 -- 172.21.15.43:0/3175377228 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa8a4014030 con 0x7fa8a8102f50 2024-04-25T10:31:26.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:26.088+0000 7fa8af43b700 1 -- 172.21.15.43:0/3175377228 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fa8a819e200 con 0x7fa8a8102f50 2024-04-25T10:31:26.090 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:25 smithi043 ceph-mon[29478]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-25T10:31:26.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:26.088+0000 7fa89e7fc700 1 -- 172.21.15.43:0/3175377228 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa8a40a7af0 con 0x7fa8a8102f50 2024-04-25T10:31:26.091 INFO:teuthology.orchestra.run.smithi043.stdout:231928234023 2024-04-25T10:31:26.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:26.090+0000 7fa8af43b700 1 -- 172.21.15.43:0/3175377228 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fa8940afc10 msgr2=0x7fa8940b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:26.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:26.090+0000 7fa8af43b700 1 --2- 172.21.15.43:0/3175377228 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fa8940afc10 0x7fa8940b2120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8113030 tx=0x7fa898009190 comp rx=0 tx=0).stop 2024-04-25T10:31:26.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:26.090+0000 7fa8af43b700 1 -- 172.21.15.43:0/3175377228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8a8102f50 msgr2=0x7fa8a819cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:26.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:26.090+0000 7fa8af43b700 1 --2- 172.21.15.43:0/3175377228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8a8102f50 0x7fa8a819cea0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400c9b0 tx=0x7fa8a400ccc0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:26.090+0000 7fa8af43b700 1 -- 172.21.15.43:0/3175377228 shutdown_connections 2024-04-25T10:31:26.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:26.090+0000 7fa8af43b700 1 --2- 172.21.15.43:0/3175377228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8a8110b00 0x7fa8a81a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:26.090+0000 7fa8af43b700 1 --2- 172.21.15.43:0/3175377228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8a8102f50 0x7fa8a819cea0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:26.090+0000 7fa8af43b700 1 --2- 172.21.15.43:0/3175377228 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fa8940afc10 0x7fa8940b2120 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:26.090+0000 7fa8af43b700 1 --2- 172.21.15.43:0/3175377228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8a81038a0 0x7fa8a819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:26.090+0000 7fa8af43b700 1 -- 172.21.15.43:0/3175377228 >> 172.21.15.43:0/3175377228 conn(0x7fa8a80fecc0 msgr2=0x7fa8a810a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:26.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:26.090+0000 7fa8af43b700 1 -- 172.21.15.43:0/3175377228 shutdown_connections 2024-04-25T10:31:26.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:26.090+0000 7fa8af43b700 1 -- 172.21.15.43:0/3175377228 wait complete. 2024-04-25T10:31:26.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:25 smithi077 ceph-mon[32772]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-25T10:31:26.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:25 smithi164 ceph-mon[32713]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-25T10:31:26.999 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234020 got 231928234023 for osd.8 2024-04-25T10:31:26.999 DEBUG:teuthology.parallel:result is None 2024-04-25T10:31:27.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:26 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3175377228' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T10:31:27.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:26 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3175377228' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T10:31:27.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:26 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3175377228' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T10:31:27.421 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645334 got 309237645336 for osd.11 2024-04-25T10:31:27.421 DEBUG:teuthology.parallel:result is None 2024-04-25T10:31:27.586 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:31:27.587 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:31:27.587 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:31:27.587 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:31:27.592 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:31:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.657+0000 7f1766fe0700 1 -- 172.21.15.43:0/4128092923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f176011d800 msgr2=0x7f176011fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.657+0000 7f1766fe0700 1 --2- 172.21.15.43:0/4128092923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f176011d800 0x7f176011fc20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1758009fc0 tx=0x7f175800b330 comp rx=0 tx=0).stop 2024-04-25T10:31:27.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.658+0000 7f1766fe0700 1 -- 172.21.15.43:0/4128092923 shutdown_connections 2024-04-25T10:31:27.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.658+0000 7f1766fe0700 1 --2- 172.21.15.43:0/4128092923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f176011d800 0x7f176011fc20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.658+0000 7f1766fe0700 1 --2- 172.21.15.43:0/4128092923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f176010a7e0 0x7f176010abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.658+0000 7f1766fe0700 1 --2- 172.21.15.43:0/4128092923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f176010b130 0x7f176011d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.658+0000 7f1766fe0700 1 -- 172.21.15.43:0/4128092923 >> 172.21.15.43:0/4128092923 conn(0x7f176006f370 msgr2=0x7f176006f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:27.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.658+0000 7f1766fe0700 1 -- 172.21.15.43:0/4128092923 shutdown_connections 2024-04-25T10:31:27.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.658+0000 7f1766fe0700 1 -- 172.21.15.43:0/4128092923 wait complete. 2024-04-25T10:31:27.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.659+0000 7f1766fe0700 1 Processor -- start 2024-04-25T10:31:27.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.659+0000 7f1766fe0700 1 -- start start 2024-04-25T10:31:27.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.659+0000 7f1766fe0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f176010a7e0 0x7f176011d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.659+0000 7f1766fe0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f176010b130 0x7f1760119180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.659+0000 7f1766fe0700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f176011d800 0x7f17601196c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.659+0000 7f1766fe0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1760115270 con 0x7f176010a7e0 2024-04-25T10:31:27.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.659+0000 7f1766fe0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17601150f0 con 0x7f176010b130 2024-04-25T10:31:27.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.659+0000 7f1766fe0700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f17601153f0 con 0x7f176011d800 2024-04-25T10:31:27.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.660+0000 7f1764d7c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f176010a7e0 0x7f176011d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:27.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.660+0000 7f1764d7c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f176010a7e0 0x7f176011d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36812/0 (socket says 172.21.15.43:36812) 2024-04-25T10:31:27.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.660+0000 7f175ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f176010b130 0x7f1760119180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:27.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.660+0000 7f1764d7c700 1 -- 172.21.15.43:0/3914418052 learned_addr learned my addr 172.21.15.43:0/3914418052 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:27.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.660+0000 7f176557d700 1 --2- 172.21.15.43:0/3914418052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f176011d800 0x7f17601196c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:27.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.660+0000 7f175ffff700 1 -- 172.21.15.43:0/3914418052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f176011d800 msgr2=0x7f17601196c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.660+0000 7f175ffff700 1 --2- 172.21.15.43:0/3914418052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f176011d800 0x7f17601196c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.660+0000 7f175ffff700 1 -- 172.21.15.43:0/3914418052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f176010a7e0 msgr2=0x7f176011d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.660+0000 7f175ffff700 1 --2- 172.21.15.43:0/3914418052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f176010a7e0 0x7f176011d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.660+0000 7f175ffff700 1 -- 172.21.15.43:0/3914418052 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1760119dd0 con 0x7f176010b130 2024-04-25T10:31:27.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.660+0000 7f176557d700 1 --2- 172.21.15.43:0/3914418052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f176011d800 0x7f17601196c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:27.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.660+0000 7f1764d7c700 1 --2- 172.21.15.43:0/3914418052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f176010a7e0 0x7f176011d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:27.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.660+0000 7f175ffff700 1 --2- 172.21.15.43:0/3914418052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f176010b130 0x7f1760119180 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f175401afd0 tx=0x7f175401d5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:27.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.661+0000 7f175dffb700 1 -- 172.21.15.43:0/3914418052 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175401dde0 con 0x7f176010b130 2024-04-25T10:31:27.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.661+0000 7f175dffb700 1 -- 172.21.15.43:0/3914418052 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17540044d0 con 0x7f176010b130 2024-04-25T10:31:27.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.661+0000 7f1766fe0700 1 -- 172.21.15.43:0/3914418052 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17601ad760 con 0x7f176010b130 2024-04-25T10:31:27.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.661+0000 7f175dffb700 1 -- 172.21.15.43:0/3914418052 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1754021440 con 0x7f176010b130 2024-04-25T10:31:27.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.661+0000 7f1766fe0700 1 -- 172.21.15.43:0/3914418052 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17601adc10 con 0x7f176010b130 2024-04-25T10:31:27.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.662+0000 7f1766fe0700 1 -- 172.21.15.43:0/3914418052 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1760063e20 con 0x7f176010b130 2024-04-25T10:31:27.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.664+0000 7f175dffb700 1 -- 172.21.15.43:0/3914418052 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f1754004000 con 0x7f176010b130 2024-04-25T10:31:27.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.665+0000 7f175dffb700 1 --2- 172.21.15.43:0/3914418052 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f17480afc10 0x7f17480b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.665+0000 7f1764d7c700 1 --2- 172.21.15.43:0/3914418052 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f17480afc10 0x7f17480b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:27.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.665+0000 7f175dffb700 1 -- 172.21.15.43:0/3914418052 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f17540f0550 con 0x7f176010b130 2024-04-25T10:31:27.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.665+0000 7f175dffb700 1 --2- 172.21.15.43:0/3914418052 >> [v2:172.21.15.77:6816/1331976864,v1:172.21.15.77:6817/1331976864] conn(0x7f17480b9440 0x7f17480bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.665+0000 7f175dffb700 1 -- 172.21.15.43:0/3914418052 --> [v2:172.21.15.77:6816/1331976864,v1:172.21.15.77:6817/1331976864] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f17480bbf60 con 0x7f17480b9440 2024-04-25T10:31:27.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.665+0000 7f175dffb700 1 -- 172.21.15.43:0/3914418052 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f17540f0900 con 0x7f176010b130 2024-04-25T10:31:27.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.665+0000 7f176557d700 1 --2- 172.21.15.43:0/3914418052 >> [v2:172.21.15.77:6816/1331976864,v1:172.21.15.77:6817/1331976864] conn(0x7f17480b9440 0x7f17480bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:27.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.665+0000 7f1764d7c700 1 --2- 172.21.15.43:0/3914418052 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f17480afc10 0x7f17480b2120 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1750011550 tx=0x7f1750009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:27.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.666+0000 7f176557d700 1 --2- 172.21.15.43:0/3914418052 >> [v2:172.21.15.77:6816/1331976864,v1:172.21.15.77:6817/1331976864] conn(0x7f17480b9440 0x7f17480bb890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:27.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.667+0000 7f175dffb700 1 -- 172.21.15.43:0/3914418052 <== osd.7 v2:172.21.15.77:6816/1331976864 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f17480bbf60 con 0x7f17480b9440 2024-04-25T10:31:27.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.683+0000 7f1766fe0700 1 -- 172.21.15.43:0/3914418052 --> [v2:172.21.15.77:6816/1331976864,v1:172.21.15.77:6817/1331976864] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f17600640f0 con 0x7f17480b9440 2024-04-25T10:31:27.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.684+0000 7f175dffb700 1 -- 172.21.15.43:0/3914418052 <== osd.7 v2:172.21.15.77:6816/1331976864 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f17600640f0 con 0x7f17480b9440 2024-04-25T10:31:27.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.685+0000 7f1766fe0700 1 -- 172.21.15.43:0/3914418052 >> [v2:172.21.15.77:6816/1331976864,v1:172.21.15.77:6817/1331976864] conn(0x7f17480b9440 msgr2=0x7f17480bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.685+0000 7f1766fe0700 1 --2- 172.21.15.43:0/3914418052 >> [v2:172.21.15.77:6816/1331976864,v1:172.21.15.77:6817/1331976864] conn(0x7f17480b9440 0x7f17480bb890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.685+0000 7f1766fe0700 1 -- 172.21.15.43:0/3914418052 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f17480afc10 msgr2=0x7f17480b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.685+0000 7f1766fe0700 1 --2- 172.21.15.43:0/3914418052 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f17480afc10 0x7f17480b2120 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1750011550 tx=0x7f1750009260 comp rx=0 tx=0).stop 2024-04-25T10:31:27.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.685+0000 7f1766fe0700 1 -- 172.21.15.43:0/3914418052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f176010b130 msgr2=0x7f1760119180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.685+0000 7f1766fe0700 1 --2- 172.21.15.43:0/3914418052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f176010b130 0x7f1760119180 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f175401afd0 tx=0x7f175401d5c0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.685+0000 7f176557d700 1 -- 172.21.15.43:0/3914418052 reap_dead start 2024-04-25T10:31:27.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.685+0000 7f1766fe0700 1 -- 172.21.15.43:0/3914418052 shutdown_connections 2024-04-25T10:31:27.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.685+0000 7f1766fe0700 1 -- 172.21.15.43:0/3914418052 >> 172.21.15.43:0/3914418052 conn(0x7f176006f370 msgr2=0x7f176011def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:27.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.685+0000 7f1766fe0700 1 -- 172.21.15.43:0/3914418052 shutdown_connections 2024-04-25T10:31:27.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:27.685+0000 7f1766fe0700 1 -- 172.21.15.43:0/3914418052 wait complete. 2024-04-25T10:31:28.196 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:27 smithi043 ceph-mon[29478]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T10:31:28.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.226+0000 7faf7807f700 1 -- 172.21.15.43:0/3584801227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf701044c0 msgr2=0x7faf701048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.226+0000 7faf7807f700 1 --2- 172.21.15.43:0/3584801227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf701044c0 0x7faf701048d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0099c0 tx=0x7faf6c009cd0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.226+0000 7faf7807f700 1 -- 172.21.15.43:0/3584801227 shutdown_connections 2024-04-25T10:31:28.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.226+0000 7faf7807f700 1 --2- 172.21.15.43:0/3584801227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf70109740 0x7faf701110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.226+0000 7faf7807f700 1 --2- 172.21.15.43:0/3584801227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf701044c0 0x7faf701048d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.226+0000 7faf7807f700 1 --2- 172.21.15.43:0/3584801227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf70108df0 0x7faf70109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.226+0000 7faf7807f700 1 -- 172.21.15.43:0/3584801227 >> 172.21.15.43:0/3584801227 conn(0x7faf700ffeb0 msgr2=0x7faf701022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:28.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.226+0000 7faf7807f700 1 -- 172.21.15.43:0/3584801227 shutdown_connections 2024-04-25T10:31:28.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.226+0000 7faf7807f700 1 -- 172.21.15.43:0/3584801227 wait complete. 2024-04-25T10:31:28.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.227+0000 7faf7807f700 1 Processor -- start 2024-04-25T10:31:28.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.227+0000 7faf7807f700 1 -- start start 2024-04-25T10:31:28.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.227+0000 7faf7807f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf701044c0 0x7faf701a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.227+0000 7faf7807f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf70108df0 0x7faf701a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.227+0000 7faf7807f700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf70109740 0x7faf7019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.227+0000 7faf7807f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faf70114370 con 0x7faf701044c0 2024-04-25T10:31:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.227+0000 7faf7807f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faf701141f0 con 0x7faf70108df0 2024-04-25T10:31:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.227+0000 7faf7807f700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf701144f0 con 0x7faf70109740 2024-04-25T10:31:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.227+0000 7faf75e1b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf701044c0 0x7faf701a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.228+0000 7faf75e1b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf701044c0 0x7faf701a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36826/0 (socket says 172.21.15.43:36826) 2024-04-25T10:31:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.228+0000 7faf75e1b700 1 -- 172.21.15.43:0/1334177491 learned_addr learned my addr 172.21.15.43:0/1334177491 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.228+0000 7faf7661c700 1 --2- 172.21.15.43:0/1334177491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf70109740 0x7faf7019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.228+0000 7faf7561a700 1 --2- 172.21.15.43:0/1334177491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf70108df0 0x7faf701a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.228+0000 7faf75e1b700 1 -- 172.21.15.43:0/1334177491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf70109740 msgr2=0x7faf7019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.228+0000 7faf75e1b700 1 --2- 172.21.15.43:0/1334177491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf70109740 0x7faf7019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.228+0000 7faf75e1b700 1 -- 172.21.15.43:0/1334177491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf70108df0 msgr2=0x7faf701a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.228+0000 7faf75e1b700 1 --2- 172.21.15.43:0/1334177491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf70108df0 0x7faf701a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.228+0000 7faf75e1b700 1 -- 172.21.15.43:0/1334177491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7019b1d0 con 0x7faf701044c0 2024-04-25T10:31:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.228+0000 7faf75e1b700 1 --2- 172.21.15.43:0/1334177491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf701044c0 0x7faf701a09a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00b580 tx=0x7faf6c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.228+0000 7faf66ffd700 1 -- 172.21.15.43:0/1334177491 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c027070 con 0x7faf701044c0 2024-04-25T10:31:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.228+0000 7faf66ffd700 1 -- 172.21.15.43:0/1334177491 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf6c003f20 con 0x7faf701044c0 2024-04-25T10:31:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.228+0000 7faf66ffd700 1 -- 172.21.15.43:0/1334177491 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c022d10 con 0x7faf701044c0 2024-04-25T10:31:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.228+0000 7faf7807f700 1 -- 172.21.15.43:0/1334177491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7019b460 con 0x7faf701044c0 2024-04-25T10:31:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.228+0000 7faf7807f700 1 -- 172.21.15.43:0/1334177491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf7019b990 con 0x7faf701044c0 2024-04-25T10:31:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.229+0000 7faf7807f700 1 -- 172.21.15.43:0/1334177491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf70105970 con 0x7faf701044c0 2024-04-25T10:31:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.232+0000 7faf66ffd700 1 -- 172.21.15.43:0/1334177491 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7faf6c022470 con 0x7faf701044c0 2024-04-25T10:31:28.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.232+0000 7faf66ffd700 1 --2- 172.21.15.43:0/1334177491 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faf5c0afc90 0x7faf5c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.232+0000 7faf7561a700 1 --2- 172.21.15.43:0/1334177491 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faf5c0afc90 0x7faf5c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.232+0000 7faf66ffd700 1 -- 172.21.15.43:0/1334177491 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7faf6c0dfb90 con 0x7faf701044c0 2024-04-25T10:31:28.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.233+0000 7faf7561a700 1 --2- 172.21.15.43:0/1334177491 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faf5c0afc90 0x7faf5c0b21a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faf7019c260 tx=0x7faf600073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.236+0000 7faf66ffd700 1 -- 172.21.15.43:0/1334177491 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf7019b990 con 0x7faf701044c0 2024-04-25T10:31:28.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:27 smithi077 ceph-mon[32772]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T10:31:28.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:27 smithi164 ceph-mon[32713]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T10:31:28.397 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:31:28.398 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:31:28.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.458+0000 7faf7807f700 1 -- 172.21.15.43:0/1334177491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7faf7002e510 con 0x7faf701044c0 2024-04-25T10:31:28.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.458+0000 7faf66ffd700 1 -- 172.21.15.43:0/1334177491 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faf6c013070 con 0x7faf701044c0 2024-04-25T10:31:28.460 INFO:teuthology.orchestra.run.smithi043.stdout:55834574923 2024-04-25T10:31:28.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.460+0000 7faf7807f700 1 -- 172.21.15.43:0/1334177491 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faf5c0afc90 msgr2=0x7faf5c0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.460+0000 7faf7807f700 1 --2- 172.21.15.43:0/1334177491 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faf5c0afc90 0x7faf5c0b21a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faf7019c260 tx=0x7faf600073d0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.460+0000 7faf7807f700 1 -- 172.21.15.43:0/1334177491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf701044c0 msgr2=0x7faf701a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.460+0000 7faf7807f700 1 --2- 172.21.15.43:0/1334177491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf701044c0 0x7faf701a09a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00b580 tx=0x7faf6c0047b0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.460+0000 7faf7807f700 1 -- 172.21.15.43:0/1334177491 shutdown_connections 2024-04-25T10:31:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.460+0000 7faf7807f700 1 --2- 172.21.15.43:0/1334177491 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7faf5c0afc90 0x7faf5c0b21a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.460+0000 7faf7807f700 1 --2- 172.21.15.43:0/1334177491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf70109740 0x7faf7019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.460+0000 7faf7807f700 1 --2- 172.21.15.43:0/1334177491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf701044c0 0x7faf701a09a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.460+0000 7faf7807f700 1 --2- 172.21.15.43:0/1334177491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf70108df0 0x7faf701a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.460+0000 7faf7807f700 1 -- 172.21.15.43:0/1334177491 >> 172.21.15.43:0/1334177491 conn(0x7faf700ffeb0 msgr2=0x7faf70108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.461+0000 7faf7807f700 1 -- 172.21.15.43:0/1334177491 shutdown_connections 2024-04-25T10:31:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:28.461+0000 7faf7807f700 1 -- 172.21.15.43:0/1334177491 wait complete. 2024-04-25T10:31:29.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:28 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1334177491' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T10:31:29.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:28 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1334177491' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T10:31:29.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:28 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1334177491' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T10:31:30.277 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:29 smithi043 ceph-mon[29478]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T10:31:30.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:29 smithi077 ceph-mon[32772]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T10:31:30.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:29 smithi164 ceph-mon[32713]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T10:31:31.671 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574919 got 55834574923 for osd.1 2024-04-25T10:31:31.671 DEBUG:teuthology.parallel:result is None 2024-04-25T10:31:32.056 INFO:teuthology.orchestra.run.smithi043.stdout:206158430253 2024-04-25T10:31:32.057 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T10:31:32.232 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:31 smithi043 ceph-mon[29478]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T10:31:32.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:31 smithi077 ceph-mon[32772]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T10:31:32.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:31 smithi164 ceph-mon[32713]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T10:31:33.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T10:31:33.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-25T10:31:33.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-25T10:31:33.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-04-25T10:31:33.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-25T10:31:33.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-25T10:31:33.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-04-25T10:31:33.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:32 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T10:31:33.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T10:31:33.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-25T10:31:33.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-25T10:31:33.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-04-25T10:31:33.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-25T10:31:33.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-25T10:31:33.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-04-25T10:31:33.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:32 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T10:31:33.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T10:31:33.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-25T10:31:33.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-25T10:31:33.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-04-25T10:31:33.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-25T10:31:33.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-25T10:31:33.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-04-25T10:31:33.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:32 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T10:31:34.104 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:33 smithi077 ceph-mon[32772]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:31:34.104 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T10:31:34.104 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-25T10:31:34.104 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-25T10:31:34.104 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-04-25T10:31:34.104 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-25T10:31:34.104 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-25T10:31:34.104 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-04-25T10:31:34.104 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T10:31:34.104 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:33 smithi077 ceph-mon[32772]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T10:31:34.104 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:33 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:31:34.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:33 smithi164 ceph-mon[32713]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:31:34.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T10:31:34.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-25T10:31:34.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-25T10:31:34.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-04-25T10:31:34.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-25T10:31:34.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-25T10:31:34.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-04-25T10:31:34.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T10:31:34.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:33 smithi164 ceph-mon[32713]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T10:31:34.106 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:33 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:31:34.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:33 smithi043 ceph-mon[29478]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:31:34.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T10:31:34.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-25T10:31:34.239 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-25T10:31:34.239 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-04-25T10:31:34.239 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-25T10:31:34.239 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-25T10:31:34.239 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-04-25T10:31:34.239 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T10:31:34.239 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:33 smithi043 ceph-mon[29478]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T10:31:34.239 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:33 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:31:35.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:34 smithi043 ceph-mon[29478]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T10:31:35.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:35.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:35.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:35.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:34 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:35.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:34 smithi077 ceph-mon[32772]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T10:31:35.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:35.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:35.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:35.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:34 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:35.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:34 smithi164 ceph-mon[32713]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T10:31:35.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:35.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:35.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:35.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:34 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:36.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:35 smithi043 ceph-mon[29478]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T10:31:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.075+0000 7fc7ae241700 1 -- 172.21.15.43:0/2388017828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7a8109730 msgr2=0x7fc7a8111090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.075+0000 7fc7ae241700 1 --2- 172.21.15.43:0/2388017828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7a8109730 0x7fc7a8111090 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000b0d0 tx=0x7fc7a000b3e0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.075+0000 7fc7ae241700 1 -- 172.21.15.43:0/2388017828 shutdown_connections 2024-04-25T10:31:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.075+0000 7fc7ae241700 1 --2- 172.21.15.43:0/2388017828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7a8109730 0x7fc7a8111090 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.075+0000 7fc7ae241700 1 --2- 172.21.15.43:0/2388017828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7a81044b0 0x7fc7a81048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.075+0000 7fc7ae241700 1 --2- 172.21.15.43:0/2388017828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7a8108de0 0x7fc7a81091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.075+0000 7fc7ae241700 1 -- 172.21.15.43:0/2388017828 >> 172.21.15.43:0/2388017828 conn(0x7fc7a80ffd40 msgr2=0x7fc7a8102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.075+0000 7fc7ae241700 1 -- 172.21.15.43:0/2388017828 shutdown_connections 2024-04-25T10:31:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.075+0000 7fc7ae241700 1 -- 172.21.15.43:0/2388017828 wait complete. 2024-04-25T10:31:36.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.076+0000 7fc7ae241700 1 Processor -- start 2024-04-25T10:31:36.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.076+0000 7fc7ae241700 1 -- start start 2024-04-25T10:31:36.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.076+0000 7fc7ae241700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7a81044b0 0x7fc7a819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.076+0000 7fc7ae241700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7a8108de0 0x7fc7a819d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.076+0000 7fc7ae241700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7a8109730 0x7fc7a81a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.076+0000 7fc7ae241700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a81143d0 con 0x7fc7a8108de0 2024-04-25T10:31:36.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.076+0000 7fc7ae241700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a8114250 con 0x7fc7a81044b0 2024-04-25T10:31:36.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.076+0000 7fc7ae241700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a8114550 con 0x7fc7a8109730 2024-04-25T10:31:36.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.076+0000 7fc7a6ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7a8108de0 0x7fc7a819d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.076+0000 7fc7a6ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7a8108de0 0x7fc7a819d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36958/0 (socket says 172.21.15.43:36958) 2024-04-25T10:31:36.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.076+0000 7fc7a6ffd700 1 -- 172.21.15.43:0/2194223408 learned_addr learned my addr 172.21.15.43:0/2194223408 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:36.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.077+0000 7fc7a77fe700 1 --2- 172.21.15.43:0/2194223408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7a81044b0 0x7fc7a819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.077+0000 7fc7a7fff700 1 --2- 172.21.15.43:0/2194223408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7a8109730 0x7fc7a81a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.077+0000 7fc7a6ffd700 1 -- 172.21.15.43:0/2194223408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7a8109730 msgr2=0x7fc7a81a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.077+0000 7fc7a6ffd700 1 --2- 172.21.15.43:0/2194223408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7a8109730 0x7fc7a81a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.077+0000 7fc7a6ffd700 1 -- 172.21.15.43:0/2194223408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7a81044b0 msgr2=0x7fc7a819cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.077+0000 7fc7a6ffd700 1 --2- 172.21.15.43:0/2194223408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7a81044b0 0x7fc7a819cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.077+0000 7fc7a6ffd700 1 -- 172.21.15.43:0/2194223408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a81a1ce0 con 0x7fc7a8108de0 2024-04-25T10:31:36.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.077+0000 7fc7a6ffd700 1 --2- 172.21.15.43:0/2194223408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7a8108de0 0x7fc7a819d460 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc79800b500 tx=0x7fc79800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:36.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.077+0000 7fc7a4ff9700 1 -- 172.21.15.43:0/2194223408 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798015020 con 0x7fc7a8108de0 2024-04-25T10:31:36.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.077+0000 7fc7ae241700 1 -- 172.21.15.43:0/2194223408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a81a1fd0 con 0x7fc7a8108de0 2024-04-25T10:31:36.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.077+0000 7fc7a4ff9700 1 -- 172.21.15.43:0/2194223408 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc798011ce0 con 0x7fc7a8108de0 2024-04-25T10:31:36.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.077+0000 7fc7a4ff9700 1 -- 172.21.15.43:0/2194223408 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798010d40 con 0x7fc7a8108de0 2024-04-25T10:31:36.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.077+0000 7fc7ae241700 1 -- 172.21.15.43:0/2194223408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a81a2510 con 0x7fc7a8108de0 2024-04-25T10:31:36.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.078+0000 7fc7ae241700 1 -- 172.21.15.43:0/2194223408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a8051190 con 0x7fc7a8108de0 2024-04-25T10:31:36.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.080+0000 7fc7a4ff9700 1 -- 172.21.15.43:0/2194223408 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fc798018020 con 0x7fc7a8108de0 2024-04-25T10:31:36.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.080+0000 7fc7a4ff9700 1 --2- 172.21.15.43:0/2194223408 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc7900afc10 0x7fc7900b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.080+0000 7fc7a77fe700 1 --2- 172.21.15.43:0/2194223408 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc7900afc10 0x7fc7900b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.081+0000 7fc7a4ff9700 1 -- 172.21.15.43:0/2194223408 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fc7980dea50 con 0x7fc7a8108de0 2024-04-25T10:31:36.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.081+0000 7fc7a77fe700 1 --2- 172.21.15.43:0/2194223408 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc7900afc10 0x7fc7900b2120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc79c005ea0 tx=0x7fc79c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:36.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.084+0000 7fc7a4ff9700 1 -- 172.21.15.43:0/2194223408 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc79801f020 con 0x7fc7a8108de0 2024-04-25T10:31:36.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:35 smithi077 ceph-mon[32772]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T10:31:36.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:35 smithi164 ceph-mon[32713]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T10:31:36.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.320+0000 7fc7ae241700 1 -- 172.21.15.43:0/2194223408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fc7a81a9a90 con 0x7fc7a8108de0 2024-04-25T10:31:36.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.320+0000 7fc7a4ff9700 1 -- 172.21.15.43:0/2194223408 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fc7980a75f0 con 0x7fc7a8108de0 2024-04-25T10:31:36.322 INFO:teuthology.orchestra.run.smithi043.stdout:283467841566 2024-04-25T10:31:36.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.322+0000 7fc7ae241700 1 -- 172.21.15.43:0/2194223408 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc7900afc10 msgr2=0x7fc7900b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.322+0000 7fc7ae241700 1 --2- 172.21.15.43:0/2194223408 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc7900afc10 0x7fc7900b2120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc79c005ea0 tx=0x7fc79c005e30 comp rx=0 tx=0).stop 2024-04-25T10:31:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.322+0000 7fc7ae241700 1 -- 172.21.15.43:0/2194223408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7a8108de0 msgr2=0x7fc7a819d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.322+0000 7fc7ae241700 1 --2- 172.21.15.43:0/2194223408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7a8108de0 0x7fc7a819d460 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc79800b500 tx=0x7fc79800b8c0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.322+0000 7fc7ae241700 1 -- 172.21.15.43:0/2194223408 shutdown_connections 2024-04-25T10:31:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.322+0000 7fc7ae241700 1 --2- 172.21.15.43:0/2194223408 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc7900afc10 0x7fc7900b2120 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.322+0000 7fc7ae241700 1 --2- 172.21.15.43:0/2194223408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7a8109730 0x7fc7a81a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.322+0000 7fc7ae241700 1 --2- 172.21.15.43:0/2194223408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7a81044b0 0x7fc7a819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.322+0000 7fc7ae241700 1 --2- 172.21.15.43:0/2194223408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7a8108de0 0x7fc7a819d460 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.322+0000 7fc7ae241700 1 -- 172.21.15.43:0/2194223408 >> 172.21.15.43:0/2194223408 conn(0x7fc7a80ffd40 msgr2=0x7fc7a8110cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.323+0000 7fc7ae241700 1 -- 172.21.15.43:0/2194223408 shutdown_connections 2024-04-25T10:31:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.323+0000 7fc7ae241700 1 -- 172.21.15.43:0/2194223408 wait complete. 2024-04-25T10:31:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.343+0000 7f5832b62700 1 -- 172.21.15.43:0/3926379482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582c109720 msgr2=0x7f582c111080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.343+0000 7f5832b62700 1 --2- 172.21.15.43:0/3926379482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582c109720 0x7f582c111080 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f582400b0d0 tx=0x7f582400b3e0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.343+0000 7f5832b62700 1 -- 172.21.15.43:0/3926379482 shutdown_connections 2024-04-25T10:31:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.343+0000 7f5832b62700 1 --2- 172.21.15.43:0/3926379482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582c109720 0x7f582c111080 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.343+0000 7f5832b62700 1 --2- 172.21.15.43:0/3926379482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f582c1044a0 0x7f582c1048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.343+0000 7f5832b62700 1 --2- 172.21.15.43:0/3926379482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f582c108dd0 0x7f582c1091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.343+0000 7f5832b62700 1 -- 172.21.15.43:0/3926379482 >> 172.21.15.43:0/3926379482 conn(0x7f582c0ffeb0 msgr2=0x7f582c1022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.343+0000 7f5832b62700 1 -- 172.21.15.43:0/3926379482 shutdown_connections 2024-04-25T10:31:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.343+0000 7f5832b62700 1 -- 172.21.15.43:0/3926379482 wait complete. 2024-04-25T10:31:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.344+0000 7f5832b62700 1 Processor -- start 2024-04-25T10:31:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.344+0000 7f5832b62700 1 -- start start 2024-04-25T10:31:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.344+0000 7f5832b62700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f582c1044a0 0x7f582c1a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.344+0000 7f5832b62700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582c108dd0 0x7f582c1a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.344+0000 7f5832b62700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f582c109720 0x7f582c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.344+0000 7f5832b62700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f582c114350 con 0x7f582c108dd0 2024-04-25T10:31:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.344+0000 7f5832b62700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1141d0 con 0x7f582c109720 2024-04-25T10:31:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.344+0000 7f5832b62700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1144d0 con 0x7f582c1044a0 2024-04-25T10:31:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.344+0000 7f582bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582c108dd0 0x7f582c1a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.344+0000 7f582bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582c108dd0 0x7f582c1a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36988/0 (socket says 172.21.15.43:36988) 2024-04-25T10:31:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.344+0000 7f58310ff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f582c109720 0x7f582c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.344+0000 7f582bfff700 1 -- 172.21.15.43:0/2721592817 learned_addr learned my addr 172.21.15.43:0/2721592817 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.344+0000 7f58308fe700 1 --2- 172.21.15.43:0/2721592817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f582c1044a0 0x7f582c1a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.345+0000 7f58310ff700 1 -- 172.21.15.43:0/2721592817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f582c1044a0 msgr2=0x7f582c1a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.345+0000 7f58310ff700 1 --2- 172.21.15.43:0/2721592817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f582c1044a0 0x7f582c1a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.345+0000 7f58310ff700 1 -- 172.21.15.43:0/2721592817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582c108dd0 msgr2=0x7f582c1a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.345+0000 7f58310ff700 1 --2- 172.21.15.43:0/2721592817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582c108dd0 0x7f582c1a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.345+0000 7f58310ff700 1 -- 172.21.15.43:0/2721592817 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f582c19b100 con 0x7f582c109720 2024-04-25T10:31:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.345+0000 7f582bfff700 1 --2- 172.21.15.43:0/2721592817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582c108dd0 0x7f582c1a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.345+0000 7f58308fe700 1 --2- 172.21.15.43:0/2721592817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f582c1044a0 0x7f582c1a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.345+0000 7f58310ff700 1 --2- 172.21.15.43:0/2721592817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f582c109720 0x7f582c19aa50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f582400b0d0 tx=0x7f5824007a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.345+0000 7f5829ffb700 1 -- 172.21.15.43:0/2721592817 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582401f070 con 0x7f582c109720 2024-04-25T10:31:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.345+0000 7f5832b62700 1 -- 172.21.15.43:0/2721592817 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f582c19b390 con 0x7f582c109720 2024-04-25T10:31:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.346+0000 7f5832b62700 1 -- 172.21.15.43:0/2721592817 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f582c1a76c0 con 0x7f582c109720 2024-04-25T10:31:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.346+0000 7f5829ffb700 1 -- 172.21.15.43:0/2721592817 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5824004c50 con 0x7f582c109720 2024-04-25T10:31:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.346+0000 7f5829ffb700 1 -- 172.21.15.43:0/2721592817 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582401b430 con 0x7f582c109720 2024-04-25T10:31:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.347+0000 7f5832b62700 1 -- 172.21.15.43:0/2721592817 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5810005360 con 0x7f582c109720 2024-04-25T10:31:36.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.348+0000 7f5829ffb700 1 -- 172.21.15.43:0/2721592817 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f5824022070 con 0x7f582c109720 2024-04-25T10:31:36.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.349+0000 7f5829ffb700 1 --2- 172.21.15.43:0/2721592817 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f581c0afc90 0x7f581c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.350+0000 7f58308fe700 1 --2- 172.21.15.43:0/2721592817 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f581c0afc90 0x7f581c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.350+0000 7f5829ffb700 1 -- 172.21.15.43:0/2721592817 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f58240df8c0 con 0x7f582c109720 2024-04-25T10:31:36.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.350+0000 7f58308fe700 1 --2- 172.21.15.43:0/2721592817 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f581c0afc90 0x7f581c0b21a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5820005950 tx=0x7f5820005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:36.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.353+0000 7f5829ffb700 1 -- 172.21.15.43:0/2721592817 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5824029030 con 0x7f582c109720 2024-04-25T10:31:36.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.563+0000 7f5832b62700 1 -- 172.21.15.43:0/2721592817 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5810005920 con 0x7f582c109720 2024-04-25T10:31:36.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.564+0000 7f5829ffb700 1 -- 172.21.15.43:0/2721592817 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f58240a8380 con 0x7f582c109720 2024-04-25T10:31:36.566 INFO:teuthology.orchestra.run.smithi043.stdout:77309411400 2024-04-25T10:31:36.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.566+0000 7f5832b62700 1 -- 172.21.15.43:0/2721592817 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f581c0afc90 msgr2=0x7f581c0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.566+0000 7f5832b62700 1 --2- 172.21.15.43:0/2721592817 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f581c0afc90 0x7f581c0b21a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5820005950 tx=0x7f5820005f50 comp rx=0 tx=0).stop 2024-04-25T10:31:36.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.566+0000 7f5832b62700 1 -- 172.21.15.43:0/2721592817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f582c109720 msgr2=0x7f582c19aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.566+0000 7f5832b62700 1 --2- 172.21.15.43:0/2721592817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f582c109720 0x7f582c19aa50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f582400b0d0 tx=0x7f5824007a00 comp rx=0 tx=0).stop 2024-04-25T10:31:36.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.566+0000 7f5832b62700 1 -- 172.21.15.43:0/2721592817 shutdown_connections 2024-04-25T10:31:36.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.566+0000 7f5832b62700 1 --2- 172.21.15.43:0/2721592817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f582c109720 0x7f582c19aa50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.566+0000 7f5832b62700 1 --2- 172.21.15.43:0/2721592817 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f581c0afc90 0x7f581c0b21a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.566+0000 7f5832b62700 1 --2- 172.21.15.43:0/2721592817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f582c1044a0 0x7f582c1a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.566+0000 7f5832b62700 1 --2- 172.21.15.43:0/2721592817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582c108dd0 0x7f582c1a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.566+0000 7f5832b62700 1 -- 172.21.15.43:0/2721592817 >> 172.21.15.43:0/2721592817 conn(0x7f582c0ffeb0 msgr2=0x7f582c108540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:36.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.566+0000 7f5832b62700 1 -- 172.21.15.43:0/2721592817 shutdown_connections 2024-04-25T10:31:36.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:36.566+0000 7f5832b62700 1 -- 172.21.15.43:0/2721592817 wait complete. 2024-04-25T10:31:37.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:36 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2194223408' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T10:31:37.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:36 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2721592817' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T10:31:37.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:36 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2194223408' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T10:31:37.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:36 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2721592817' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T10:31:37.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:36 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2194223408' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T10:31:37.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:36 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2721592817' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T10:31:37.419 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841564 got 283467841566 for osd.10 2024-04-25T10:31:37.419 DEBUG:teuthology.parallel:result is None 2024-04-25T10:31:37.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.606+0000 7faccd46d700 1 -- 172.21.15.43:0/4222622344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc8103420 msgr2=0x7facc8103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.606+0000 7faccd46d700 1 --2- 172.21.15.43:0/4222622344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc8103420 0x7facc8103830 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7facb80099c0 tx=0x7facb8009cd0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.606+0000 7faccd46d700 1 -- 172.21.15.43:0/4222622344 shutdown_connections 2024-04-25T10:31:37.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.606+0000 7faccd46d700 1 --2- 172.21.15.43:0/4222622344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facc8108650 0x7facc810ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.606+0000 7faccd46d700 1 --2- 172.21.15.43:0/4222622344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc8103420 0x7facc8103830 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.606+0000 7faccd46d700 1 --2- 172.21.15.43:0/4222622344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facc8107d00 0x7facc8108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.606+0000 7faccd46d700 1 -- 172.21.15.43:0/4222622344 >> 172.21.15.43:0/4222622344 conn(0x7facc8078cd0 msgr2=0x7facc8079110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:37.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.606+0000 7faccd46d700 1 -- 172.21.15.43:0/4222622344 shutdown_connections 2024-04-25T10:31:37.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.606+0000 7faccd46d700 1 -- 172.21.15.43:0/4222622344 wait complete. 2024-04-25T10:31:37.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.607+0000 7faccd46d700 1 Processor -- start 2024-04-25T10:31:37.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.607+0000 7faccd46d700 1 -- start start 2024-04-25T10:31:37.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.608+0000 7faccd46d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc8103420 0x7facc819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:37.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.608+0000 7faccd46d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facc8107d00 0x7facc819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:37.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.608+0000 7faccd46d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facc8108650 0x7facc819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:37.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.608+0000 7faccd46d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7facc8113280 con 0x7facc8103420 2024-04-25T10:31:37.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.608+0000 7faccd46d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7facc8113100 con 0x7facc8107d00 2024-04-25T10:31:37.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.608+0000 7faccd46d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7facc8113400 con 0x7facc8108650 2024-04-25T10:31:37.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.608+0000 7facc6ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc8103420 0x7facc819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:37.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.608+0000 7facc6ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc8103420 0x7facc819ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37012/0 (socket says 172.21.15.43:37012) 2024-04-25T10:31:37.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.608+0000 7facc6ffd700 1 -- 172.21.15.43:0/1457717410 learned_addr learned my addr 172.21.15.43:0/1457717410 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:37.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.608+0000 7facc67fc700 1 --2- 172.21.15.43:0/1457717410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facc8107d00 0x7facc819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:37.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.608+0000 7facc77fe700 1 --2- 172.21.15.43:0/1457717410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facc8108650 0x7facc819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:37.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.608+0000 7facc6ffd700 1 -- 172.21.15.43:0/1457717410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facc8108650 msgr2=0x7facc819f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.608+0000 7facc6ffd700 1 --2- 172.21.15.43:0/1457717410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facc8108650 0x7facc819f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.608+0000 7facc6ffd700 1 -- 172.21.15.43:0/1457717410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facc8107d00 msgr2=0x7facc819b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.608+0000 7facc6ffd700 1 --2- 172.21.15.43:0/1457717410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facc8107d00 0x7facc819b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.608+0000 7facc6ffd700 1 -- 172.21.15.43:0/1457717410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc819fc50 con 0x7facc8103420 2024-04-25T10:31:37.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.608+0000 7facc77fe700 1 --2- 172.21.15.43:0/1457717410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facc8108650 0x7facc819f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:37.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.608+0000 7facc67fc700 1 --2- 172.21.15.43:0/1457717410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facc8107d00 0x7facc819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:37.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.608+0000 7facc6ffd700 1 --2- 172.21.15.43:0/1457717410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc8103420 0x7facc819ad80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7facb8006010 tx=0x7facb8004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:37.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.608+0000 7facaffff700 1 -- 172.21.15.43:0/1457717410 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8026070 con 0x7facc8103420 2024-04-25T10:31:37.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.609+0000 7faccd46d700 1 -- 172.21.15.43:0/1457717410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc819fee0 con 0x7facc8103420 2024-04-25T10:31:37.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.609+0000 7facaffff700 1 -- 172.21.15.43:0/1457717410 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facb800f460 con 0x7facc8103420 2024-04-25T10:31:37.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.609+0000 7facaffff700 1 -- 172.21.15.43:0/1457717410 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8009e60 con 0x7facc8103420 2024-04-25T10:31:37.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.609+0000 7faccd46d700 1 -- 172.21.15.43:0/1457717410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc81a7860 con 0x7facc8103420 2024-04-25T10:31:37.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.610+0000 7faccd46d700 1 -- 172.21.15.43:0/1457717410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc8104880 con 0x7facc8103420 2024-04-25T10:31:37.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.611+0000 7facaffff700 1 -- 172.21.15.43:0/1457717410 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7facb8029070 con 0x7facc8103420 2024-04-25T10:31:37.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.612+0000 7facaffff700 1 --2- 172.21.15.43:0/1457717410 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7facb00afc10 0x7facb00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:37.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.612+0000 7facc67fc700 1 --2- 172.21.15.43:0/1457717410 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7facb00afc10 0x7facb00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:37.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.612+0000 7facaffff700 1 -- 172.21.15.43:0/1457717410 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7facb80df960 con 0x7facc8103420 2024-04-25T10:31:37.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.612+0000 7facc67fc700 1 --2- 172.21.15.43:0/1457717410 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7facb00afc10 0x7facb00b2120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7facc8079270 tx=0x7facbc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:37.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.615+0000 7facaffff700 1 -- 172.21.15.43:0/1457717410 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7facb8013070 con 0x7facc8103420 2024-04-25T10:31:37.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.827+0000 7faccd46d700 1 -- 172.21.15.43:0/1457717410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7facc819c0e0 con 0x7facc8103420 2024-04-25T10:31:37.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.828+0000 7facaffff700 1 -- 172.21.15.43:0/1457717410 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7facb80a8480 con 0x7facc8103420 2024-04-25T10:31:37.830 INFO:teuthology.orchestra.run.smithi043.stdout:133143986237 2024-04-25T10:31:37.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.830+0000 7faccd46d700 1 -- 172.21.15.43:0/1457717410 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7facb00afc10 msgr2=0x7facb00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.830+0000 7faccd46d700 1 --2- 172.21.15.43:0/1457717410 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7facb00afc10 0x7facb00b2120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7facc8079270 tx=0x7facbc009260 comp rx=0 tx=0).stop 2024-04-25T10:31:37.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.830+0000 7faccd46d700 1 -- 172.21.15.43:0/1457717410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc8103420 msgr2=0x7facc819ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.830+0000 7faccd46d700 1 --2- 172.21.15.43:0/1457717410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc8103420 0x7facc819ad80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7facb8006010 tx=0x7facb8004690 comp rx=0 tx=0).stop 2024-04-25T10:31:37.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.830+0000 7faccd46d700 1 -- 172.21.15.43:0/1457717410 shutdown_connections 2024-04-25T10:31:37.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.830+0000 7faccd46d700 1 --2- 172.21.15.43:0/1457717410 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7facb00afc10 0x7facb00b2120 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.830+0000 7faccd46d700 1 --2- 172.21.15.43:0/1457717410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facc8108650 0x7facc819f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.830+0000 7faccd46d700 1 --2- 172.21.15.43:0/1457717410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc8103420 0x7facc819ad80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.830+0000 7faccd46d700 1 --2- 172.21.15.43:0/1457717410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facc8107d00 0x7facc819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.830+0000 7faccd46d700 1 -- 172.21.15.43:0/1457717410 >> 172.21.15.43:0/1457717410 conn(0x7facc8078cd0 msgr2=0x7facc8075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:37.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.830+0000 7faccd46d700 1 -- 172.21.15.43:0/1457717410 shutdown_connections 2024-04-25T10:31:37.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:37.830+0000 7faccd46d700 1 -- 172.21.15.43:0/1457717410 wait complete. 2024-04-25T10:31:38.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:37 smithi043 ceph-mon[29478]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 7.5 KiB/s, 0 objects/s recovering 2024-04-25T10:31:38.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:37 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1457717410' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T10:31:38.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:37 smithi077 ceph-mon[32772]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 7.5 KiB/s, 0 objects/s recovering 2024-04-25T10:31:38.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:37 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1457717410' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T10:31:38.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:37 smithi164 ceph-mon[32713]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 7.5 KiB/s, 0 objects/s recovering 2024-04-25T10:31:38.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:37 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1457717410' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T10:31:38.388 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411396 got 77309411400 for osd.2 2024-04-25T10:31:38.389 DEBUG:teuthology.parallel:result is None 2024-04-25T10:31:39.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.510+0000 7f05aecb4700 1 -- 172.21.15.43:0/3121688207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05a8102fd0 msgr2=0x7f05a81033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:39.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.510+0000 7f05aecb4700 1 --2- 172.21.15.43:0/3121688207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05a8102fd0 0x7f05a81033e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0598009990 tx=0x7f0598009ca0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.510+0000 7f05aecb4700 1 -- 172.21.15.43:0/3121688207 shutdown_connections 2024-04-25T10:31:39.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.510+0000 7f05aecb4700 1 --2- 172.21.15.43:0/3121688207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a8110b80 0x7f05a8112fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.510+0000 7f05aecb4700 1 --2- 172.21.15.43:0/3121688207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05a8102fd0 0x7f05a81033e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.510+0000 7f05aecb4700 1 --2- 172.21.15.43:0/3121688207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05a8103920 0x7f05a8110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.510+0000 7f05aecb4700 1 -- 172.21.15.43:0/3121688207 >> 172.21.15.43:0/3121688207 conn(0x7f05a80fec90 msgr2=0x7f05a81010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:39.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.510+0000 7f05aecb4700 1 -- 172.21.15.43:0/3121688207 shutdown_connections 2024-04-25T10:31:39.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.510+0000 7f05aecb4700 1 -- 172.21.15.43:0/3121688207 wait complete. 2024-04-25T10:31:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.511+0000 7f05aecb4700 1 Processor -- start 2024-04-25T10:31:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.511+0000 7f05aecb4700 1 -- start start 2024-04-25T10:31:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.512+0000 7f05aecb4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05a8102fd0 0x7f05a819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.512+0000 7f05aecb4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a8103920 0x7f05a819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.512+0000 7f05aecb4700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05a8110b80 0x7f05a81a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.512+0000 7f05aecb4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f05a8116530 con 0x7f05a8102fd0 2024-04-25T10:31:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.512+0000 7f05aecb4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f05a81163b0 con 0x7f05a8103920 2024-04-25T10:31:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.512+0000 7f05aecb4700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f05a81166b0 con 0x7f05a8110b80 2024-04-25T10:31:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.512+0000 7f05aca50700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05a8102fd0 0x7f05a819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.512+0000 7f05aca50700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05a8102fd0 0x7f05a819cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57510/0 (socket says 172.21.15.43:57510) 2024-04-25T10:31:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.512+0000 7f05aca50700 1 -- 172.21.15.43:0/4051479935 learned_addr learned my addr 172.21.15.43:0/4051479935 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.512+0000 7f05a7fff700 1 --2- 172.21.15.43:0/4051479935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a8103920 0x7f05a819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.512+0000 7f05ad251700 1 --2- 172.21.15.43:0/4051479935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05a8110b80 0x7f05a81a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:39.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.512+0000 7f05aca50700 1 -- 172.21.15.43:0/4051479935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05a8110b80 msgr2=0x7f05a81a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:39.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.512+0000 7f05aca50700 1 --2- 172.21.15.43:0/4051479935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05a8110b80 0x7f05a81a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.512+0000 7f05aca50700 1 -- 172.21.15.43:0/4051479935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a8103920 msgr2=0x7f05a819d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:39.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.512+0000 7f05aca50700 1 --2- 172.21.15.43:0/4051479935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a8103920 0x7f05a819d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.512+0000 7f05aca50700 1 -- 172.21.15.43:0/4051479935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a81a1d20 con 0x7f05a8102fd0 2024-04-25T10:31:39.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.512+0000 7f05aca50700 1 --2- 172.21.15.43:0/4051479935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05a8102fd0 0x7f05a819cf00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0598000c00 tx=0x7f05980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:39.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.513+0000 7f05a5ffb700 1 -- 172.21.15.43:0/4051479935 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598027070 con 0x7f05a8102fd0 2024-04-25T10:31:39.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.513+0000 7f05a5ffb700 1 -- 172.21.15.43:0/4051479935 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0598003f20 con 0x7f05a8102fd0 2024-04-25T10:31:39.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.513+0000 7f05aecb4700 1 -- 172.21.15.43:0/4051479935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a81a1fb0 con 0x7f05a8102fd0 2024-04-25T10:31:39.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.513+0000 7f05a5ffb700 1 -- 172.21.15.43:0/4051479935 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598022cf0 con 0x7f05a8102fd0 2024-04-25T10:31:39.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.513+0000 7f05aecb4700 1 -- 172.21.15.43:0/4051479935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a81a2490 con 0x7f05a8102fd0 2024-04-25T10:31:39.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.514+0000 7f05aecb4700 1 -- 172.21.15.43:0/4051479935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a8051190 con 0x7f05a8102fd0 2024-04-25T10:31:39.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.515+0000 7f05a5ffb700 1 -- 172.21.15.43:0/4051479935 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f059802a070 con 0x7f05a8102fd0 2024-04-25T10:31:39.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.516+0000 7f05a5ffb700 1 --2- 172.21.15.43:0/4051479935 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f05940afc90 0x7f05940b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:39.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.516+0000 7f05a7fff700 1 --2- 172.21.15.43:0/4051479935 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f05940afc90 0x7f05940b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:39.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.516+0000 7f05a5ffb700 1 -- 172.21.15.43:0/4051479935 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f05980dfb80 con 0x7f05a8102fd0 2024-04-25T10:31:39.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.516+0000 7f05a7fff700 1 --2- 172.21.15.43:0/4051479935 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f05940afc90 0x7f05940b21a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f05a8104900 tx=0x7f059000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.521+0000 7f05a5ffb700 1 -- 172.21.15.43:0/4051479935 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0598013070 con 0x7f05a8102fd0 2024-04-25T10:31:39.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.734+0000 7f05aecb4700 1 -- 172.21.15.43:0/4051479935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f05a802e510 con 0x7f05a8102fd0 2024-04-25T10:31:39.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.735+0000 7f05a5ffb700 1 -- 172.21.15.43:0/4051479935 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f05980a8640 con 0x7f05a8102fd0 2024-04-25T10:31:39.738 INFO:teuthology.orchestra.run.smithi043.stdout:107374182466 2024-04-25T10:31:39.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.738+0000 7f05aecb4700 1 -- 172.21.15.43:0/4051479935 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f05940afc90 msgr2=0x7f05940b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:39.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.738+0000 7f05aecb4700 1 --2- 172.21.15.43:0/4051479935 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f05940afc90 0x7f05940b21a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f05a8104900 tx=0x7f059000b410 comp rx=0 tx=0).stop 2024-04-25T10:31:39.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.738+0000 7f05aecb4700 1 -- 172.21.15.43:0/4051479935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05a8102fd0 msgr2=0x7f05a819cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:39.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.738+0000 7f05aecb4700 1 --2- 172.21.15.43:0/4051479935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05a8102fd0 0x7f05a819cf00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0598000c00 tx=0x7f05980047b0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.738+0000 7f05aecb4700 1 -- 172.21.15.43:0/4051479935 shutdown_connections 2024-04-25T10:31:39.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.738+0000 7f05aecb4700 1 --2- 172.21.15.43:0/4051479935 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f05940afc90 0x7f05940b21a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.738+0000 7f05aecb4700 1 --2- 172.21.15.43:0/4051479935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05a8110b80 0x7f05a81a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.738+0000 7f05aecb4700 1 --2- 172.21.15.43:0/4051479935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05a8102fd0 0x7f05a819cf00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.738+0000 7f05aecb4700 1 --2- 172.21.15.43:0/4051479935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a8103920 0x7f05a819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.738+0000 7f05aecb4700 1 -- 172.21.15.43:0/4051479935 >> 172.21.15.43:0/4051479935 conn(0x7f05a80fec90 msgr2=0x7f05a81126b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:39.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.738+0000 7f05aecb4700 1 -- 172.21.15.43:0/4051479935 shutdown_connections 2024-04-25T10:31:39.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:39.738+0000 7f05aecb4700 1 -- 172.21.15.43:0/4051479935 wait complete. 2024-04-25T10:31:40.289 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:39 smithi043 ceph-mon[29478]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3 op/s; 7.5 KiB/s, 0 objects/s recovering 2024-04-25T10:31:40.289 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:39 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/4051479935' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T10:31:40.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:39 smithi077 ceph-mon[32772]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3 op/s; 7.5 KiB/s, 0 objects/s recovering 2024-04-25T10:31:40.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:39 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/4051479935' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T10:31:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:39 smithi164 ceph-mon[32713]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3 op/s; 7.5 KiB/s, 0 objects/s recovering 2024-04-25T10:31:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:39 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/4051479935' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T10:31:40.700 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986233 got 133143986237 for osd.4 2024-04-25T10:31:40.700 DEBUG:teuthology.parallel:result is None 2024-04-25T10:31:41.516 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182461 got 107374182466 for osd.3 2024-04-25T10:31:41.516 DEBUG:teuthology.parallel:result is None 2024-04-25T10:31:41.712 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:31:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.741+0000 7f23823e7700 1 -- 172.21.15.43:0/2011960679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f237c108d90 msgr2=0x7f237c1091a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.741+0000 7f23823e7700 1 --2- 172.21.15.43:0/2011960679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f237c108d90 0x7f237c1091a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f23700099c0 tx=0x7f2370009cd0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.741+0000 7f23823e7700 1 -- 172.21.15.43:0/2011960679 shutdown_connections 2024-04-25T10:31:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.741+0000 7f23823e7700 1 --2- 172.21.15.43:0/2011960679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f237c1096e0 0x7f237c111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.741+0000 7f23823e7700 1 --2- 172.21.15.43:0/2011960679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f237c104460 0x7f237c104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.741+0000 7f23823e7700 1 --2- 172.21.15.43:0/2011960679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f237c108d90 0x7f237c1091a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.741+0000 7f23823e7700 1 -- 172.21.15.43:0/2011960679 >> 172.21.15.43:0/2011960679 conn(0x7f237c0ffd40 msgr2=0x7f237c102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:41.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.741+0000 7f23823e7700 1 -- 172.21.15.43:0/2011960679 shutdown_connections 2024-04-25T10:31:41.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.741+0000 7f23823e7700 1 -- 172.21.15.43:0/2011960679 wait complete. 2024-04-25T10:31:41.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.742+0000 7f23823e7700 1 Processor -- start 2024-04-25T10:31:41.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.742+0000 7f23823e7700 1 -- start start 2024-04-25T10:31:41.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.742+0000 7f23823e7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f237c104460 0x7f237c1a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.742+0000 7f23823e7700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f237c108d90 0x7f237c1a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.742+0000 7f23823e7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f237c1096e0 0x7f237c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.742+0000 7f23823e7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f237c1142f0 con 0x7f237c1096e0 2024-04-25T10:31:41.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.742+0000 7f23823e7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f237c114170 con 0x7f237c104460 2024-04-25T10:31:41.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.742+0000 7f23823e7700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f237c114470 con 0x7f237c108d90 2024-04-25T10:31:41.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.743+0000 7f2380984700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f237c1096e0 0x7f237c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.743+0000 7f2380984700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f237c1096e0 0x7f237c19aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57514/0 (socket says 172.21.15.43:57514) 2024-04-25T10:31:41.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.743+0000 7f2380984700 1 -- 172.21.15.43:0/1167113564 learned_addr learned my addr 172.21.15.43:0/1167113564 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:41.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.743+0000 7f237bfff700 1 --2- 172.21.15.43:0/1167113564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f237c104460 0x7f237c1a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.743+0000 7f237b7fe700 1 --2- 172.21.15.43:0/1167113564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f237c108d90 0x7f237c1a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.743+0000 7f2380984700 1 -- 172.21.15.43:0/1167113564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f237c108d90 msgr2=0x7f237c1a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:41.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.743+0000 7f2380984700 1 --2- 172.21.15.43:0/1167113564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f237c108d90 0x7f237c1a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.743+0000 7f2380984700 1 -- 172.21.15.43:0/1167113564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f237c104460 msgr2=0x7f237c1a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:41.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.743+0000 7f2380984700 1 --2- 172.21.15.43:0/1167113564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f237c104460 0x7f237c1a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.743+0000 7f2380984700 1 -- 172.21.15.43:0/1167113564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237c19b0d0 con 0x7f237c1096e0 2024-04-25T10:31:41.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.743+0000 7f2380984700 1 --2- 172.21.15.43:0/1167113564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f237c1096e0 0x7f237c19aa90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2374007d70 tx=0x7f237400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:41.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.743+0000 7f23797fa700 1 -- 172.21.15.43:0/1167113564 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2374016020 con 0x7f237c1096e0 2024-04-25T10:31:41.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.743+0000 7f23823e7700 1 -- 172.21.15.43:0/1167113564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f237c19b3c0 con 0x7f237c1096e0 2024-04-25T10:31:41.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.743+0000 7f23797fa700 1 -- 172.21.15.43:0/1167113564 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f237400ac90 con 0x7f237c1096e0 2024-04-25T10:31:41.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.743+0000 7f23797fa700 1 -- 172.21.15.43:0/1167113564 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23740115e0 con 0x7f237c1096e0 2024-04-25T10:31:41.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.743+0000 7f23823e7700 1 -- 172.21.15.43:0/1167113564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f237c19b8d0 con 0x7f237c1096e0 2024-04-25T10:31:41.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.745+0000 7f23823e7700 1 -- 172.21.15.43:0/1167113564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f237c051190 con 0x7f237c1096e0 2024-04-25T10:31:41.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.746+0000 7f23797fa700 1 -- 172.21.15.43:0/1167113564 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f2374019020 con 0x7f237c1096e0 2024-04-25T10:31:41.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.747+0000 7f23797fa700 1 --2- 172.21.15.43:0/1167113564 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f23640afc90 0x7f23640b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.747+0000 7f237bfff700 1 --2- 172.21.15.43:0/1167113564 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f23640afc90 0x7f23640b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.747+0000 7f23797fa700 1 -- 172.21.15.43:0/1167113564 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f23740def80 con 0x7f237c1096e0 2024-04-25T10:31:41.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.748+0000 7f237bfff700 1 --2- 172.21.15.43:0/1167113564 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f23640afc90 0x7f23640b21a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f237c19bbd0 tx=0x7f236c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:41.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.751+0000 7f23797fa700 1 -- 172.21.15.43:0/1167113564 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2374020020 con 0x7f237c1096e0 2024-04-25T10:31:41.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.961+0000 7f23823e7700 1 -- 172.21.15.43:0/1167113564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f237c19bfc0 con 0x7f237c1096e0 2024-04-25T10:31:41.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.961+0000 7f23797fa700 1 -- 172.21.15.43:0/1167113564 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f23740a7b20 con 0x7f237c1096e0 2024-04-25T10:31:41.964 INFO:teuthology.orchestra.run.smithi043.stdout:34359738450 2024-04-25T10:31:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.966+0000 7f23823e7700 1 -- 172.21.15.43:0/1167113564 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f23640afc90 msgr2=0x7f23640b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.966+0000 7f23823e7700 1 --2- 172.21.15.43:0/1167113564 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f23640afc90 0x7f23640b21a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f237c19bbd0 tx=0x7f236c00a380 comp rx=0 tx=0).stop 2024-04-25T10:31:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.966+0000 7f23823e7700 1 -- 172.21.15.43:0/1167113564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f237c1096e0 msgr2=0x7f237c19aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.966+0000 7f23823e7700 1 --2- 172.21.15.43:0/1167113564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f237c1096e0 0x7f237c19aa90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2374007d70 tx=0x7f237400a340 comp rx=0 tx=0).stop 2024-04-25T10:31:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.966+0000 7f23823e7700 1 -- 172.21.15.43:0/1167113564 shutdown_connections 2024-04-25T10:31:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.966+0000 7f23823e7700 1 --2- 172.21.15.43:0/1167113564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f237c1096e0 0x7f237c19aa90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.966+0000 7f23823e7700 1 --2- 172.21.15.43:0/1167113564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f237c104460 0x7f237c1a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.966+0000 7f23823e7700 1 --2- 172.21.15.43:0/1167113564 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f23640afc90 0x7f23640b21a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.966+0000 7f23823e7700 1 --2- 172.21.15.43:0/1167113564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f237c108d90 0x7f237c1a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.966+0000 7f23823e7700 1 -- 172.21.15.43:0/1167113564 >> 172.21.15.43:0/1167113564 conn(0x7f237c0ffd40 msgr2=0x7f237c10a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.966+0000 7f23823e7700 1 -- 172.21.15.43:0/1167113564 shutdown_connections 2024-04-25T10:31:41.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:41.966+0000 7f23823e7700 1 -- 172.21.15.43:0/1167113564 wait complete. 2024-04-25T10:31:42.048 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:41 smithi043 ceph-mon[29478]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 7.5 KiB/s, 0 objects/s recovering 2024-04-25T10:31:42.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.070+0000 7f9b0c4a2700 1 -- 172.21.15.43:0/3564223797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b041044e0 msgr2=0x7f9b041048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.070+0000 7f9b0c4a2700 1 --2- 172.21.15.43:0/3564223797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b041044e0 0x7f9b041048f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9b000099c0 tx=0x7f9b00009cd0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.071+0000 7f9b0c4a2700 1 -- 172.21.15.43:0/3564223797 shutdown_connections 2024-04-25T10:31:42.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.071+0000 7f9b0c4a2700 1 --2- 172.21.15.43:0/3564223797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b04109760 0x7f9b041110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.071+0000 7f9b0c4a2700 1 --2- 172.21.15.43:0/3564223797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b041044e0 0x7f9b041048f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.071+0000 7f9b0c4a2700 1 --2- 172.21.15.43:0/3564223797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b04108e10 0x7f9b04109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.071+0000 7f9b0c4a2700 1 -- 172.21.15.43:0/3564223797 >> 172.21.15.43:0/3564223797 conn(0x7f9b0407ae40 msgr2=0x7f9b0407b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:42.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.071+0000 7f9b0c4a2700 1 -- 172.21.15.43:0/3564223797 shutdown_connections 2024-04-25T10:31:42.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.071+0000 7f9b0c4a2700 1 -- 172.21.15.43:0/3564223797 wait complete. 2024-04-25T10:31:42.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.071+0000 7f9b0c4a2700 1 Processor -- start 2024-04-25T10:31:42.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.071+0000 7f9b0c4a2700 1 -- start start 2024-04-25T10:31:42.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.072+0000 7f9b0c4a2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b041044e0 0x7f9b0419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.072+0000 7f9b0c4a2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b04108e10 0x7f9b0419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.072+0000 7f9b0c4a2700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b04109760 0x7f9b0419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.072+0000 7f9b0c4a2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9b04114460 con 0x7f9b04108e10 2024-04-25T10:31:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.072+0000 7f9b0c4a2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9b041142e0 con 0x7f9b041044e0 2024-04-25T10:31:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.072+0000 7f9b0c4a2700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9b041145e0 con 0x7f9b04109760 2024-04-25T10:31:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.072+0000 7f9b09a3d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b04108e10 0x7f9b0419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.072+0000 7f9b09a3d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b04108e10 0x7f9b0419b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57538/0 (socket says 172.21.15.43:57538) 2024-04-25T10:31:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.072+0000 7f9b09a3d700 1 -- 172.21.15.43:0/3983353830 learned_addr learned my addr 172.21.15.43:0/3983353830 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.072+0000 7f9b0a23e700 1 --2- 172.21.15.43:0/3983353830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b041044e0 0x7f9b0419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.072+0000 7f9b0aa3f700 1 --2- 172.21.15.43:0/3983353830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b04109760 0x7f9b0419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.072+0000 7f9b09a3d700 1 -- 172.21.15.43:0/3983353830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b04109760 msgr2=0x7f9b0419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.072+0000 7f9b09a3d700 1 --2- 172.21.15.43:0/3983353830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b04109760 0x7f9b0419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.072+0000 7f9b09a3d700 1 -- 172.21.15.43:0/3983353830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b041044e0 msgr2=0x7f9b0419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.072+0000 7f9b09a3d700 1 --2- 172.21.15.43:0/3983353830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b041044e0 0x7f9b0419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.072+0000 7f9b09a3d700 1 -- 172.21.15.43:0/3983353830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0419fcc0 con 0x7f9b04108e10 2024-04-25T10:31:42.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.072+0000 7f9b09a3d700 1 --2- 172.21.15.43:0/3983353830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b04108e10 0x7f9b0419b390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9af400d980 tx=0x7f9af400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:42.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.072+0000 7f9afb7fe700 1 -- 172.21.15.43:0/3983353830 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af4017070 con 0x7f9b04108e10 2024-04-25T10:31:42.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.073+0000 7f9b0c4a2700 1 -- 172.21.15.43:0/3983353830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0419ffb0 con 0x7f9b04108e10 2024-04-25T10:31:42.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.073+0000 7f9afb7fe700 1 -- 172.21.15.43:0/3983353830 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9af4009de0 con 0x7f9b04108e10 2024-04-25T10:31:42.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.073+0000 7f9afb7fe700 1 -- 172.21.15.43:0/3983353830 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af400f3c0 con 0x7f9b04108e10 2024-04-25T10:31:42.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.073+0000 7f9b0c4a2700 1 -- 172.21.15.43:0/3983353830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b041a78d0 con 0x7f9b04108e10 2024-04-25T10:31:42.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.073+0000 7f9b0c4a2700 1 -- 172.21.15.43:0/3983353830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b04051190 con 0x7f9b04108e10 2024-04-25T10:31:42.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.075+0000 7f9afb7fe700 1 -- 172.21.15.43:0/3983353830 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f9af4009930 con 0x7f9b04108e10 2024-04-25T10:31:42.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.076+0000 7f9afb7fe700 1 --2- 172.21.15.43:0/3983353830 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9af00afc10 0x7f9af00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.076+0000 7f9b0a23e700 1 --2- 172.21.15.43:0/3983353830 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9af00afc10 0x7f9af00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.076+0000 7f9afb7fe700 1 -- 172.21.15.43:0/3983353830 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f9af40de8e0 con 0x7f9b04108e10 2024-04-25T10:31:42.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.077+0000 7f9b0a23e700 1 --2- 172.21.15.43:0/3983353830 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9af00afc10 0x7f9af00b2120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000b580 tx=0x7f9b000036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:42.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.081+0000 7f9afb7fe700 1 -- 172.21.15.43:0/3983353830 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9af40a7370 con 0x7f9b04108e10 2024-04-25T10:31:42.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:41 smithi077 ceph-mon[32772]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 7.5 KiB/s, 0 objects/s recovering 2024-04-25T10:31:42.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:41 smithi164 ceph-mon[32713]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 7.5 KiB/s, 0 objects/s recovering 2024-04-25T10:31:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.296+0000 7f9b0c4a2700 1 -- 172.21.15.43:0/3983353830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9b0402e510 con 0x7f9b04108e10 2024-04-25T10:31:42.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.297+0000 7f9afb7fe700 1 -- 172.21.15.43:0/3983353830 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9af40a6c70 con 0x7f9b04108e10 2024-04-25T10:31:42.300 INFO:teuthology.orchestra.run.smithi043.stdout:154618822712 2024-04-25T10:31:42.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.299+0000 7f9b0c4a2700 1 -- 172.21.15.43:0/3983353830 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9af00afc10 msgr2=0x7f9af00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.299+0000 7f9b0c4a2700 1 --2- 172.21.15.43:0/3983353830 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9af00afc10 0x7f9af00b2120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000b580 tx=0x7f9b000036e0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.299+0000 7f9b0c4a2700 1 -- 172.21.15.43:0/3983353830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b04108e10 msgr2=0x7f9b0419b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.299+0000 7f9b0c4a2700 1 --2- 172.21.15.43:0/3983353830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b04108e10 0x7f9b0419b390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9af400d980 tx=0x7f9af400dc90 comp rx=0 tx=0).stop 2024-04-25T10:31:42.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.299+0000 7f9b0c4a2700 1 -- 172.21.15.43:0/3983353830 shutdown_connections 2024-04-25T10:31:42.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.299+0000 7f9b0c4a2700 1 --2- 172.21.15.43:0/3983353830 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9af00afc10 0x7f9af00b2120 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.299+0000 7f9b0c4a2700 1 --2- 172.21.15.43:0/3983353830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b04109760 0x7f9b0419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.299+0000 7f9b0c4a2700 1 --2- 172.21.15.43:0/3983353830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b041044e0 0x7f9b0419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.299+0000 7f9b0c4a2700 1 --2- 172.21.15.43:0/3983353830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b04108e10 0x7f9b0419b390 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.299+0000 7f9b0c4a2700 1 -- 172.21.15.43:0/3983353830 >> 172.21.15.43:0/3983353830 conn(0x7f9b0407ae40 msgr2=0x7f9b041018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:42.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.299+0000 7f9b0c4a2700 1 -- 172.21.15.43:0/3983353830 shutdown_connections 2024-04-25T10:31:42.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:42.299+0000 7f9b0c4a2700 1 -- 172.21.15.43:0/3983353830 wait complete. 2024-04-25T10:31:42.640 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738443 got 34359738450 for osd.0 2024-04-25T10:31:42.640 DEBUG:teuthology.parallel:result is None 2024-04-25T10:31:43.231 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:42 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1167113564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T10:31:43.232 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:42 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3983353830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T10:31:43.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:42 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1167113564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T10:31:43.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:42 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3983353830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T10:31:43.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:42 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1167113564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T10:31:43.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:42 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3983353830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T10:31:43.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.683+0000 7f5b05f66700 1 -- 172.21.15.43:0/3575647111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b00108db0 msgr2=0x7f5b001091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:43.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.683+0000 7f5b05f66700 1 --2- 172.21.15.43:0/3575647111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b00108db0 0x7f5b001091c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5af40099c0 tx=0x7f5af4009cd0 comp rx=0 tx=0).stop 2024-04-25T10:31:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.683+0000 7f5b05f66700 1 -- 172.21.15.43:0/3575647111 shutdown_connections 2024-04-25T10:31:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.683+0000 7f5b05f66700 1 --2- 172.21.15.43:0/3575647111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b00109700 0x7f5b00111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.683+0000 7f5b05f66700 1 --2- 172.21.15.43:0/3575647111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b00104480 0x7f5b00104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.683+0000 7f5b05f66700 1 --2- 172.21.15.43:0/3575647111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b00108db0 0x7f5b001091c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.683+0000 7f5b05f66700 1 -- 172.21.15.43:0/3575647111 >> 172.21.15.43:0/3575647111 conn(0x7f5b000ffd70 msgr2=0x7f5b001021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.683+0000 7f5b05f66700 1 -- 172.21.15.43:0/3575647111 shutdown_connections 2024-04-25T10:31:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.683+0000 7f5b05f66700 1 -- 172.21.15.43:0/3575647111 wait complete. 2024-04-25T10:31:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.684+0000 7f5b05f66700 1 Processor -- start 2024-04-25T10:31:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.684+0000 7f5b05f66700 1 -- start start 2024-04-25T10:31:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.684+0000 7f5b05f66700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b00104480 0x7f5b0019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.684+0000 7f5b05f66700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b00108db0 0x7f5b0019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.684+0000 7f5b05f66700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b00109700 0x7f5b0019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.684+0000 7f5b05f66700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b00114360 con 0x7f5b00108db0 2024-04-25T10:31:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.684+0000 7f5b05f66700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5b001141e0 con 0x7f5b00104480 2024-04-25T10:31:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.684+0000 7f5b05f66700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5b001144e0 con 0x7f5b00109700 2024-04-25T10:31:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.685+0000 7f5afeffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b00108db0 0x7f5b0019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.685+0000 7f5afeffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b00108db0 0x7f5b0019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57564/0 (socket says 172.21.15.43:57564) 2024-04-25T10:31:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.685+0000 7f5afeffd700 1 -- 172.21.15.43:0/634862475 learned_addr learned my addr 172.21.15.43:0/634862475 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.685+0000 7f5aff7fe700 1 --2- 172.21.15.43:0/634862475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b00104480 0x7f5b0019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.685+0000 7f5afffff700 1 --2- 172.21.15.43:0/634862475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b00109700 0x7f5b0019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.685+0000 7f5afeffd700 1 -- 172.21.15.43:0/634862475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b00109700 msgr2=0x7f5b0019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.685+0000 7f5afeffd700 1 --2- 172.21.15.43:0/634862475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b00109700 0x7f5b0019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.685+0000 7f5afeffd700 1 -- 172.21.15.43:0/634862475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b00104480 msgr2=0x7f5b0019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.685+0000 7f5afeffd700 1 --2- 172.21.15.43:0/634862475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b00104480 0x7f5b0019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.685+0000 7f5afeffd700 1 -- 172.21.15.43:0/634862475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0019fca0 con 0x7f5b00108db0 2024-04-25T10:31:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.685+0000 7f5afeffd700 1 --2- 172.21.15.43:0/634862475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b00108db0 0x7f5b0019b370 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5af4006010 tx=0x7f5af40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.685+0000 7f5afcff9700 1 -- 172.21.15.43:0/634862475 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af4027070 con 0x7f5b00108db0 2024-04-25T10:31:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.685+0000 7f5b05f66700 1 -- 172.21.15.43:0/634862475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b0019ff30 con 0x7f5b00108db0 2024-04-25T10:31:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.685+0000 7f5afcff9700 1 -- 172.21.15.43:0/634862475 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5af4003f20 con 0x7f5b00108db0 2024-04-25T10:31:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.685+0000 7f5afcff9700 1 -- 172.21.15.43:0/634862475 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af4022c60 con 0x7f5b00108db0 2024-04-25T10:31:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.685+0000 7f5b05f66700 1 -- 172.21.15.43:0/634862475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b001a0410 con 0x7f5b00108db0 2024-04-25T10:31:43.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.686+0000 7f5ae67fc700 1 -- 172.21.15.43:0/634862475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b00051190 con 0x7f5b00108db0 2024-04-25T10:31:43.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.688+0000 7f5afcff9700 1 -- 172.21.15.43:0/634862475 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f5af402a070 con 0x7f5b00108db0 2024-04-25T10:31:43.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.689+0000 7f5afcff9700 1 --2- 172.21.15.43:0/634862475 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5ae80afc10 0x7f5ae80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:43.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.689+0000 7f5aff7fe700 1 --2- 172.21.15.43:0/634862475 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5ae80afc10 0x7f5ae80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:43.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.689+0000 7f5afcff9700 1 -- 172.21.15.43:0/634862475 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f5af4022e00 con 0x7f5b00108db0 2024-04-25T10:31:43.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.689+0000 7f5aff7fe700 1 --2- 172.21.15.43:0/634862475 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5ae80afc10 0x7f5ae80b2120 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5af0005950 tx=0x7f5af000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:43.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.692+0000 7f5afcff9700 1 -- 172.21.15.43:0/634862475 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5af4031080 con 0x7f5b00108db0 2024-04-25T10:31:43.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.900+0000 7f5ae67fc700 1 -- 172.21.15.43:0/634862475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f5b000642a0 con 0x7f5b00108db0 2024-04-25T10:31:43.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.900+0000 7f5afcff9700 1 -- 172.21.15.43:0/634862475 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5af4013070 con 0x7f5b00108db0 2024-04-25T10:31:43.902 INFO:teuthology.orchestra.run.smithi043.stdout:257698037797 2024-04-25T10:31:43.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.902+0000 7f5ae67fc700 1 -- 172.21.15.43:0/634862475 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5ae80afc10 msgr2=0x7f5ae80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:43.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.902+0000 7f5ae67fc700 1 --2- 172.21.15.43:0/634862475 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5ae80afc10 0x7f5ae80b2120 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5af0005950 tx=0x7f5af000b410 comp rx=0 tx=0).stop 2024-04-25T10:31:43.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.902+0000 7f5ae67fc700 1 -- 172.21.15.43:0/634862475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b00108db0 msgr2=0x7f5b0019b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:43.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.902+0000 7f5ae67fc700 1 --2- 172.21.15.43:0/634862475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b00108db0 0x7f5b0019b370 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5af4006010 tx=0x7f5af40047b0 comp rx=0 tx=0).stop 2024-04-25T10:31:43.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.902+0000 7f5ae67fc700 1 -- 172.21.15.43:0/634862475 shutdown_connections 2024-04-25T10:31:43.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.902+0000 7f5ae67fc700 1 --2- 172.21.15.43:0/634862475 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f5ae80afc10 0x7f5ae80b2120 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:43.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.902+0000 7f5ae67fc700 1 --2- 172.21.15.43:0/634862475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b00109700 0x7f5b0019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:43.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.902+0000 7f5ae67fc700 1 --2- 172.21.15.43:0/634862475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b00104480 0x7f5b0019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:43.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.902+0000 7f5ae67fc700 1 --2- 172.21.15.43:0/634862475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b00108db0 0x7f5b0019b370 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:43.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.902+0000 7f5ae67fc700 1 -- 172.21.15.43:0/634862475 >> 172.21.15.43:0/634862475 conn(0x7f5b000ffd70 msgr2=0x7f5b00108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:43.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.902+0000 7f5ae67fc700 1 -- 172.21.15.43:0/634862475 shutdown_connections 2024-04-25T10:31:43.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:43.902+0000 7f5ae67fc700 1 -- 172.21.15.43:0/634862475 wait complete. 2024-04-25T10:31:44.099 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822704 got 154618822712 for osd.5 2024-04-25T10:31:44.099 DEBUG:teuthology.parallel:result is None 2024-04-25T10:31:44.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:43 smithi077 ceph-mon[32772]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 6.4 KiB/s, 0 objects/s recovering 2024-04-25T10:31:44.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:43 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/634862475' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T10:31:44.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:43 smithi164 ceph-mon[32713]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 6.4 KiB/s, 0 objects/s recovering 2024-04-25T10:31:44.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:43 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/634862475' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T10:31:44.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:43 smithi043 ceph-mon[29478]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 6.4 KiB/s, 0 objects/s recovering 2024-04-25T10:31:44.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:43 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/634862475' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T10:31:45.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.217+0000 7f49ba9fa700 1 -- 172.21.15.43:0/415800677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b4109720 msgr2=0x7f49b4111080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:45.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.217+0000 7f49ba9fa700 1 --2- 172.21.15.43:0/415800677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b4109720 0x7f49b4111080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00b0d0 tx=0x7f49ac00b3e0 comp rx=0 tx=0).stop 2024-04-25T10:31:45.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.217+0000 7f49ba9fa700 1 -- 172.21.15.43:0/415800677 shutdown_connections 2024-04-25T10:31:45.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.217+0000 7f49ba9fa700 1 --2- 172.21.15.43:0/415800677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b4109720 0x7f49b4111080 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:45.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.217+0000 7f49ba9fa700 1 --2- 172.21.15.43:0/415800677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49b41044a0 0x7f49b41048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:45.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.217+0000 7f49ba9fa700 1 --2- 172.21.15.43:0/415800677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49b4108dd0 0x7f49b41091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:45.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.217+0000 7f49ba9fa700 1 -- 172.21.15.43:0/415800677 >> 172.21.15.43:0/415800677 conn(0x7f49b40ffd70 msgr2=0x7f49b41021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:45.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.217+0000 7f49ba9fa700 1 -- 172.21.15.43:0/415800677 shutdown_connections 2024-04-25T10:31:45.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.217+0000 7f49ba9fa700 1 -- 172.21.15.43:0/415800677 wait complete. 2024-04-25T10:31:45.220 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:44 smithi043 ceph-mon[29478]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 5.8 KiB/s, 0 objects/s recovering 2024-04-25T10:31:45.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.218+0000 7f49ba9fa700 1 Processor -- start 2024-04-25T10:31:45.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.219+0000 7f49ba9fa700 1 -- start start 2024-04-25T10:31:45.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.219+0000 7f49ba9fa700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b41044a0 0x7f49b41a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:45.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.219+0000 7f49ba9fa700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49b4108dd0 0x7f49b41a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:45.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.219+0000 7f49ba9fa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49b4109720 0x7f49b419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:45.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.219+0000 7f49ba9fa700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49b4114350 con 0x7f49b41044a0 2024-04-25T10:31:45.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.219+0000 7f49ba9fa700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f49b41141d0 con 0x7f49b4109720 2024-04-25T10:31:45.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.219+0000 7f49ba9fa700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f49b41144d0 con 0x7f49b4108dd0 2024-04-25T10:31:45.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.219+0000 7f49b3fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b41044a0 0x7f49b41a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:45.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.219+0000 7f49b3fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b41044a0 0x7f49b41a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57568/0 (socket says 172.21.15.43:57568) 2024-04-25T10:31:45.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.219+0000 7f49b3fff700 1 -- 172.21.15.43:0/4203580529 learned_addr learned my addr 172.21.15.43:0/4203580529 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:45.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.219+0000 7f49b37fe700 1 --2- 172.21.15.43:0/4203580529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49b4108dd0 0x7f49b41a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:45.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.219+0000 7f49b8f97700 1 --2- 172.21.15.43:0/4203580529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49b4109720 0x7f49b419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:45.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.219+0000 7f49b3fff700 1 -- 172.21.15.43:0/4203580529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49b4108dd0 msgr2=0x7f49b41a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:45.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.219+0000 7f49b3fff700 1 --2- 172.21.15.43:0/4203580529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49b4108dd0 0x7f49b41a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:45.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.219+0000 7f49b3fff700 1 -- 172.21.15.43:0/4203580529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49b4109720 msgr2=0x7f49b419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:45.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.219+0000 7f49b3fff700 1 --2- 172.21.15.43:0/4203580529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49b4109720 0x7f49b419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:45.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.219+0000 7f49b3fff700 1 -- 172.21.15.43:0/4203580529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b419b160 con 0x7f49b41044a0 2024-04-25T10:31:45.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.220+0000 7f49b8f97700 1 --2- 172.21.15.43:0/4203580529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49b4109720 0x7f49b419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:45.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.220+0000 7f49b3fff700 1 --2- 172.21.15.43:0/4203580529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b41044a0 0x7f49b41a0890 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f49a400d980 tx=0x7f49a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:45.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.220+0000 7f49b17fa700 1 -- 172.21.15.43:0/4203580529 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a4016070 con 0x7f49b41044a0 2024-04-25T10:31:45.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.220+0000 7f49ba9fa700 1 -- 172.21.15.43:0/4203580529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b419b450 con 0x7f49b41044a0 2024-04-25T10:31:45.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.220+0000 7f49b17fa700 1 -- 172.21.15.43:0/4203580529 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49a4004830 con 0x7f49b41044a0 2024-04-25T10:31:45.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.220+0000 7f49b17fa700 1 -- 172.21.15.43:0/4203580529 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a40054c0 con 0x7f49b41044a0 2024-04-25T10:31:45.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.220+0000 7f49ba9fa700 1 -- 172.21.15.43:0/4203580529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b41a7720 con 0x7f49b41044a0 2024-04-25T10:31:45.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.221+0000 7f49ba9fa700 1 -- 172.21.15.43:0/4203580529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b4051190 con 0x7f49b41044a0 2024-04-25T10:31:45.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.223+0000 7f49b17fa700 1 -- 172.21.15.43:0/4203580529 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f49a4004220 con 0x7f49b41044a0 2024-04-25T10:31:45.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.223+0000 7f49b17fa700 1 --2- 172.21.15.43:0/4203580529 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f499c0afc10 0x7f499c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:45.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.224+0000 7f49b37fe700 1 --2- 172.21.15.43:0/4203580529 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f499c0afc10 0x7f499c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:45.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.224+0000 7f49b17fa700 1 -- 172.21.15.43:0/4203580529 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f49a40df150 con 0x7f49b41044a0 2024-04-25T10:31:45.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.224+0000 7f49b37fe700 1 --2- 172.21.15.43:0/4203580529 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f499c0afc10 0x7f499c0b2120 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009680 tx=0x7f49a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:45.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.227+0000 7f49b17fa700 1 -- 172.21.15.43:0/4203580529 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49a401f020 con 0x7f49b41044a0 2024-04-25T10:31:45.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:44 smithi077 ceph-mon[32772]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 5.8 KiB/s, 0 objects/s recovering 2024-04-25T10:31:45.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:44 smithi164 ceph-mon[32713]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 5.8 KiB/s, 0 objects/s recovering 2024-04-25T10:31:45.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.438+0000 7f49ba9fa700 1 -- 172.21.15.43:0/4203580529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f49b402e510 con 0x7f49b41044a0 2024-04-25T10:31:45.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.438+0000 7f49b17fa700 1 -- 172.21.15.43:0/4203580529 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f49a40a7b80 con 0x7f49b41044a0 2024-04-25T10:31:45.440 INFO:teuthology.orchestra.run.smithi043.stdout:206158430256 2024-04-25T10:31:45.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.440+0000 7f49ba9fa700 1 -- 172.21.15.43:0/4203580529 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f499c0afc10 msgr2=0x7f499c0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.440+0000 7f49ba9fa700 1 --2- 172.21.15.43:0/4203580529 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f499c0afc10 0x7f499c0b2120 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009680 tx=0x7f49a8009260 comp rx=0 tx=0).stop 2024-04-25T10:31:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.440+0000 7f49ba9fa700 1 -- 172.21.15.43:0/4203580529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b41044a0 msgr2=0x7f49b41a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.440+0000 7f49ba9fa700 1 --2- 172.21.15.43:0/4203580529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b41044a0 0x7f49b41a0890 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f49a400d980 tx=0x7f49a400dc90 comp rx=0 tx=0).stop 2024-04-25T10:31:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.441+0000 7f49ba9fa700 1 -- 172.21.15.43:0/4203580529 shutdown_connections 2024-04-25T10:31:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.441+0000 7f49ba9fa700 1 --2- 172.21.15.43:0/4203580529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49b4109720 0x7f49b419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.441+0000 7f49ba9fa700 1 --2- 172.21.15.43:0/4203580529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b41044a0 0x7f49b41a0890 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.441+0000 7f49ba9fa700 1 --2- 172.21.15.43:0/4203580529 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f499c0afc10 0x7f499c0b2120 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.441+0000 7f49ba9fa700 1 --2- 172.21.15.43:0/4203580529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49b4108dd0 0x7f49b41a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.441+0000 7f49ba9fa700 1 -- 172.21.15.43:0/4203580529 >> 172.21.15.43:0/4203580529 conn(0x7f49b40ffd70 msgr2=0x7f49b4108510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.441+0000 7f49ba9fa700 1 -- 172.21.15.43:0/4203580529 shutdown_connections 2024-04-25T10:31:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:45.441+0000 7f49ba9fa700 1 -- 172.21.15.43:0/4203580529 wait complete. 2024-04-25T10:31:45.660 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037789 got 257698037797 for osd.9 2024-04-25T10:31:45.660 DEBUG:teuthology.parallel:result is None 2024-04-25T10:31:46.287 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:45 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/4203580529' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T10:31:46.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:45 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/4203580529' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T10:31:46.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:45 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/4203580529' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T10:31:46.322 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430253 got 206158430256 for osd.7 2024-04-25T10:31:46.322 DEBUG:teuthology.parallel:result is None 2024-04-25T10:31:46.322 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T10:31:46.323 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T10:31:47.114 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:31:47.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:46 smithi043 ceph-mon[29478]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 5.0 KiB/s, 0 objects/s recovering 2024-04-25T10:31:47.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:46 smithi077 ceph-mon[32772]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 5.0 KiB/s, 0 objects/s recovering 2024-04-25T10:31:47.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:46 smithi164 ceph-mon[32713]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 5.0 KiB/s, 0 objects/s recovering 2024-04-25T10:31:48.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.771+0000 7f0f276a1700 1 -- 172.21.15.43:0/3170937766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f20107c90 msgr2=0x7f0f201080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:48.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.771+0000 7f0f276a1700 1 --2- 172.21.15.43:0/3170937766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f20107c90 0x7f0f201080a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0f100099c0 tx=0x7f0f10009cd0 comp rx=0 tx=0).stop 2024-04-25T10:31:48.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.771+0000 7f0f276a1700 1 -- 172.21.15.43:0/3170937766 shutdown_connections 2024-04-25T10:31:48.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.771+0000 7f0f276a1700 1 --2- 172.21.15.43:0/3170937766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f201085e0 0x7f0f2010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:48.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.771+0000 7f0f276a1700 1 --2- 172.21.15.43:0/3170937766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f20103360 0x7f0f20103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:48.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.771+0000 7f0f276a1700 1 --2- 172.21.15.43:0/3170937766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f20107c90 0x7f0f201080a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:48.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.771+0000 7f0f276a1700 1 -- 172.21.15.43:0/3170937766 >> 172.21.15.43:0/3170937766 conn(0x7f0f200fec90 msgr2=0x7f0f201010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:48.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.771+0000 7f0f276a1700 1 -- 172.21.15.43:0/3170937766 shutdown_connections 2024-04-25T10:31:48.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.771+0000 7f0f276a1700 1 -- 172.21.15.43:0/3170937766 wait complete. 2024-04-25T10:31:48.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.772+0000 7f0f276a1700 1 Processor -- start 2024-04-25T10:31:48.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.772+0000 7f0f276a1700 1 -- start start 2024-04-25T10:31:48.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.773+0000 7f0f276a1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f20103360 0x7f0f2019bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:48.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.773+0000 7f0f276a1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f20107c90 0x7f0f2019c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:48.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.773+0000 7f0f276a1700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f201085e0 0x7f0f2019c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:48.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.773+0000 7f0f276a1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0f201131f0 con 0x7f0f20103360 2024-04-25T10:31:48.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.773+0000 7f0f276a1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0f20113070 con 0x7f0f20107c90 2024-04-25T10:31:48.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.773+0000 7f0f276a1700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0f20113370 con 0x7f0f201085e0 2024-04-25T10:31:48.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.773+0000 7f0f2543d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f20103360 0x7f0f2019bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:48.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.773+0000 7f0f2543d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f20103360 0x7f0f2019bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50890/0 (socket says 172.21.15.43:50890) 2024-04-25T10:31:48.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.773+0000 7f0f2543d700 1 -- 172.21.15.43:0/2826963786 learned_addr learned my addr 172.21.15.43:0/2826963786 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:48.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.773+0000 7f0f24c3c700 1 --2- 172.21.15.43:0/2826963786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f20107c90 0x7f0f2019c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:48.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.773+0000 7f0f25c3e700 1 --2- 172.21.15.43:0/2826963786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f201085e0 0x7f0f2019c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:48.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.773+0000 7f0f2543d700 1 -- 172.21.15.43:0/2826963786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f201085e0 msgr2=0x7f0f2019c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:48.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.773+0000 7f0f2543d700 1 --2- 172.21.15.43:0/2826963786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f201085e0 0x7f0f2019c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:48.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.773+0000 7f0f2543d700 1 -- 172.21.15.43:0/2826963786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f20107c90 msgr2=0x7f0f2019c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:48.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.773+0000 7f0f2543d700 1 --2- 172.21.15.43:0/2826963786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f20107c90 0x7f0f2019c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:48.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.773+0000 7f0f2543d700 1 -- 172.21.15.43:0/2826963786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f201a6e20 con 0x7f0f20103360 2024-04-25T10:31:48.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.773+0000 7f0f25c3e700 1 --2- 172.21.15.43:0/2826963786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f201085e0 0x7f0f2019c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:48.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.774+0000 7f0f2543d700 1 --2- 172.21.15.43:0/2826963786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f20103360 0x7f0f2019bad0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c00d980 tx=0x7f0f1c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:48.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.774+0000 7f0f167fc700 1 -- 172.21.15.43:0/2826963786 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c016070 con 0x7f0f20103360 2024-04-25T10:31:48.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.774+0000 7f0f167fc700 1 -- 172.21.15.43:0/2826963786 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f1c0044d0 con 0x7f0f20103360 2024-04-25T10:31:48.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.774+0000 7f0f276a1700 1 -- 172.21.15.43:0/2826963786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f201a7110 con 0x7f0f20103360 2024-04-25T10:31:48.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.774+0000 7f0f167fc700 1 -- 172.21.15.43:0/2826963786 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c004d00 con 0x7f0f20103360 2024-04-25T10:31:48.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.774+0000 7f0f276a1700 1 -- 172.21.15.43:0/2826963786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f201a7670 con 0x7f0f20103360 2024-04-25T10:31:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.775+0000 7f0f276a1700 1 -- 172.21.15.43:0/2826963786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f20051190 con 0x7f0f20103360 2024-04-25T10:31:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.777+0000 7f0f167fc700 1 -- 172.21.15.43:0/2826963786 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f0f1c004020 con 0x7f0f20103360 2024-04-25T10:31:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.777+0000 7f0f167fc700 1 --2- 172.21.15.43:0/2826963786 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f0f0c0afc90 0x7f0f0c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.778+0000 7f0f24c3c700 1 --2- 172.21.15.43:0/2826963786 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f0f0c0afc90 0x7f0f0c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:48.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.778+0000 7f0f167fc700 1 -- 172.21.15.43:0/2826963786 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f0f1c0df160 con 0x7f0f20103360 2024-04-25T10:31:48.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.778+0000 7f0f24c3c700 1 --2- 172.21.15.43:0/2826963786 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f0f0c0afc90 0x7f0f0c0b21a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0f10000c00 tx=0x7f0f1001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:48.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.781+0000 7f0f167fc700 1 -- 172.21.15.43:0/2826963786 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f1c01f020 con 0x7f0f20103360 2024-04-25T10:31:48.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:48.994+0000 7f0f276a1700 1 -- 172.21.15.43:0/2826963786 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0f20063ad0 con 0x7f0f0c0afc90 2024-04-25T10:31:49.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:49.002+0000 7f0f167fc700 1 -- 172.21.15.43:0/2826963786 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258710 (secure 0 0 0) 0x7f0f20063ad0 con 0x7f0f0c0afc90 2024-04-25T10:31:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:31:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:49.005+0000 7f0f276a1700 1 -- 172.21.15.43:0/2826963786 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f0f0c0afc90 msgr2=0x7f0f0c0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:49.005+0000 7f0f276a1700 1 --2- 172.21.15.43:0/2826963786 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f0f0c0afc90 0x7f0f0c0b21a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0f10000c00 tx=0x7f0f1001a040 comp rx=0 tx=0).stop 2024-04-25T10:31:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:49.005+0000 7f0f276a1700 1 -- 172.21.15.43:0/2826963786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f20103360 msgr2=0x7f0f2019bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:49.005+0000 7f0f276a1700 1 --2- 172.21.15.43:0/2826963786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f20103360 0x7f0f2019bad0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c00d980 tx=0x7f0f1c00dd40 comp rx=0 tx=0).stop 2024-04-25T10:31:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:49.005+0000 7f0f276a1700 1 -- 172.21.15.43:0/2826963786 shutdown_connections 2024-04-25T10:31:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:49.005+0000 7f0f276a1700 1 --2- 172.21.15.43:0/2826963786 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f0f0c0afc90 0x7f0f0c0b21a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:49.005+0000 7f0f276a1700 1 --2- 172.21.15.43:0/2826963786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f201085e0 0x7f0f2019c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:49.005+0000 7f0f276a1700 1 --2- 172.21.15.43:0/2826963786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f20103360 0x7f0f2019bad0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:49.005+0000 7f0f276a1700 1 --2- 172.21.15.43:0/2826963786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f20107c90 0x7f0f2019c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:49.005+0000 7f0f276a1700 1 -- 172.21.15.43:0/2826963786 >> 172.21.15.43:0/2826963786 conn(0x7f0f200fec90 msgr2=0x7f0f20109540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:49.005+0000 7f0f276a1700 1 -- 172.21.15.43:0/2826963786 shutdown_connections 2024-04-25T10:31:49.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:49.005+0000 7f0f276a1700 1 -- 172.21.15.43:0/2826963786 wait complete. 2024-04-25T10:31:49.012 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T10:31:49.328 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:49 smithi043 ceph-mon[29478]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T10:31:49.328 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:49.329 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:49 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:49.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:49 smithi077 ceph-mon[32772]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T10:31:49.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:49.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:49 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:49.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:49 smithi164 ceph-mon[32713]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T10:31:49.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:49.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:49 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:49.660 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-04-25T10:31:48.178270+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655649,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":501,"num_read_kb":345,"num_write":281,"num_write_kb":2260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":3012908,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":419580,"kb_used_data":12108,"kb_used_omap":18,"kb_used_meta":407277,"kb_avail":1124522244,"statfs":{"total":1151940427776,"available":1151510777856,"internally_reserved":0,"allocated":12398592,"data_stored":6921471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":417051961},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003235"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.938407+0000","last_change":"2024-04-25T10:30:33.645032+0000","last_active":"2024-04-25T10:31:33.938407+0000","last_peered":"2024-04-25T10:31:33.938407+0000","last_clean":"2024-04-25T10:31:33.938407+0000","last_became_active":"2024-04-25T10:30:33.644687+0000","last_became_peered":"2024-04-25T10:30:33.644687+0000","last_unstale":"2024-04-25T10:31:33.938407+0000","last_undegraded":"2024-04-25T10:31:33.938407+0000","last_fullsized":"2024-04-25T10:31:33.938407+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:39.893228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.939115+0000","last_change":"2024-04-25T10:30:33.650466+0000","last_active":"2024-04-25T10:31:33.939115+0000","last_peered":"2024-04-25T10:31:33.939115+0000","last_clean":"2024-04-25T10:31:33.939115+0000","last_became_active":"2024-04-25T10:30:33.649889+0000","last_became_peered":"2024-04-25T10:30:33.649889+0000","last_unstale":"2024-04-25T10:31:33.939115+0000","last_undegraded":"2024-04-25T10:31:33.939115+0000","last_fullsized":"2024-04-25T10:31:33.939115+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:53.832494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.939051+0000","last_change":"2024-04-25T10:30:33.648798+0000","last_active":"2024-04-25T10:31:33.939051+0000","last_peered":"2024-04-25T10:31:33.939051+0000","last_clean":"2024-04-25T10:31:33.939051+0000","last_became_active":"2024-04-25T10:30:33.648423+0000","last_became_peered":"2024-04-25T10:30:33.648423+0000","last_unstale":"2024-04-25T10:31:33.939051+0000","last_undegraded":"2024-04-25T10:31:33.939051+0000","last_fullsized":"2024-04-25T10:31:33.939051+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:19.119189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.942219+0000","last_change":"2024-04-25T10:31:33.941691+0000","last_active":"2024-04-25T10:31:33.942219+0000","last_peered":"2024-04-25T10:31:33.942219+0000","last_clean":"2024-04-25T10:31:33.942219+0000","last_became_active":"2024-04-25T10:31:33.941304+0000","last_became_peered":"2024-04-25T10:31:33.941304+0000","last_unstale":"2024-04-25T10:31:33.942219+0000","last_undegraded":"2024-04-25T10:31:33.942219+0000","last_fullsized":"2024-04-25T10:31:33.942219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:18.573452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.936121+0000","last_change":"2024-04-25T10:30:33.646786+0000","last_active":"2024-04-25T10:31:33.936121+0000","last_peered":"2024-04-25T10:31:33.936121+0000","last_clean":"2024-04-25T10:31:33.936121+0000","last_became_active":"2024-04-25T10:30:33.646456+0000","last_became_peered":"2024-04-25T10:30:33.646456+0000","last_unstale":"2024-04-25T10:31:33.936121+0000","last_undegraded":"2024-04-25T10:31:33.936121+0000","last_fullsized":"2024-04-25T10:31:33.936121+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:55.006723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.938868+0000","last_change":"2024-04-25T10:30:33.639952+0000","last_active":"2024-04-25T10:31:33.938868+0000","last_peered":"2024-04-25T10:31:33.938868+0000","last_clean":"2024-04-25T10:31:33.938868+0000","last_became_active":"2024-04-25T10:30:33.639558+0000","last_became_peered":"2024-04-25T10:30:33.639558+0000","last_unstale":"2024-04-25T10:31:33.938868+0000","last_undegraded":"2024-04-25T10:31:33.938868+0000","last_fullsized":"2024-04-25T10:31:33.938868+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:17.162902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940596+0000","last_change":"2024-04-25T10:30:33.651530+0000","last_active":"2024-04-25T10:31:33.940596+0000","last_peered":"2024-04-25T10:31:33.940596+0000","last_clean":"2024-04-25T10:31:33.940596+0000","last_became_active":"2024-04-25T10:30:33.651151+0000","last_became_peered":"2024-04-25T10:30:33.651151+0000","last_unstale":"2024-04-25T10:31:33.940596+0000","last_undegraded":"2024-04-25T10:31:33.940596+0000","last_fullsized":"2024-04-25T10:31:33.940596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:31.835367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940102+0000","last_change":"2024-04-25T10:30:33.645146+0000","last_active":"2024-04-25T10:31:33.940102+0000","last_peered":"2024-04-25T10:31:33.940102+0000","last_clean":"2024-04-25T10:31:33.940102+0000","last_became_active":"2024-04-25T10:30:33.644310+0000","last_became_peered":"2024-04-25T10:30:33.644310+0000","last_unstale":"2024-04-25T10:31:33.940102+0000","last_undegraded":"2024-04-25T10:31:33.940102+0000","last_fullsized":"2024-04-25T10:31:33.940102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:45.034934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937726+0000","last_change":"2024-04-25T10:30:33.649385+0000","last_active":"2024-04-25T10:31:33.937726+0000","last_peered":"2024-04-25T10:31:33.937726+0000","last_clean":"2024-04-25T10:31:33.937726+0000","last_became_active":"2024-04-25T10:30:33.649002+0000","last_became_peered":"2024-04-25T10:30:33.649002+0000","last_unstale":"2024-04-25T10:31:33.937726+0000","last_undegraded":"2024-04-25T10:31:33.937726+0000","last_fullsized":"2024-04-25T10:31:33.937726+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:42.259290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.939076+0000","last_change":"2024-04-25T10:30:33.641918+0000","last_active":"2024-04-25T10:31:33.939076+0000","last_peered":"2024-04-25T10:31:33.939076+0000","last_clean":"2024-04-25T10:31:33.939076+0000","last_became_active":"2024-04-25T10:30:33.641553+0000","last_became_peered":"2024-04-25T10:30:33.641553+0000","last_unstale":"2024-04-25T10:31:33.939076+0000","last_undegraded":"2024-04-25T10:31:33.939076+0000","last_fullsized":"2024-04-25T10:31:33.939076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:04.755694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941019+0000","last_change":"2024-04-25T10:30:33.638054+0000","last_active":"2024-04-25T10:31:33.941019+0000","last_peered":"2024-04-25T10:31:33.941019+0000","last_clean":"2024-04-25T10:31:33.941019+0000","last_became_active":"2024-04-25T10:30:33.637720+0000","last_became_peered":"2024-04-25T10:30:33.637720+0000","last_unstale":"2024-04-25T10:31:33.941019+0000","last_undegraded":"2024-04-25T10:31:33.941019+0000","last_fullsized":"2024-04-25T10:31:33.941019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:41.557430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941989+0000","last_change":"2024-04-25T10:30:33.642705+0000","last_active":"2024-04-25T10:31:33.941989+0000","last_peered":"2024-04-25T10:31:33.941989+0000","last_clean":"2024-04-25T10:31:33.941989+0000","last_became_active":"2024-04-25T10:30:33.641936+0000","last_became_peered":"2024-04-25T10:30:33.641936+0000","last_unstale":"2024-04-25T10:31:33.941989+0000","last_undegraded":"2024-04-25T10:31:33.941989+0000","last_fullsized":"2024-04-25T10:31:33.941989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:36.450389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.935792+0000","last_change":"2024-04-25T10:30:33.648987+0000","last_active":"2024-04-25T10:31:33.935792+0000","last_peered":"2024-04-25T10:31:33.935792+0000","last_clean":"2024-04-25T10:31:33.935792+0000","last_became_active":"2024-04-25T10:30:33.648621+0000","last_became_peered":"2024-04-25T10:30:33.648621+0000","last_unstale":"2024-04-25T10:31:33.935792+0000","last_undegraded":"2024-04-25T10:31:33.935792+0000","last_fullsized":"2024-04-25T10:31:33.935792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:19.999002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.938628+0000","last_change":"2024-04-25T10:30:33.647360+0000","last_active":"2024-04-25T10:31:33.938628+0000","last_peered":"2024-04-25T10:31:33.938628+0000","last_clean":"2024-04-25T10:31:33.938628+0000","last_became_active":"2024-04-25T10:30:33.646719+0000","last_became_peered":"2024-04-25T10:30:33.646719+0000","last_unstale":"2024-04-25T10:31:33.938628+0000","last_undegraded":"2024-04-25T10:31:33.938628+0000","last_fullsized":"2024-04-25T10:31:33.938628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:19.016333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.938530+0000","last_change":"2024-04-25T10:30:33.646503+0000","last_active":"2024-04-25T10:31:33.938530+0000","last_peered":"2024-04-25T10:31:33.938530+0000","last_clean":"2024-04-25T10:31:33.938530+0000","last_became_active":"2024-04-25T10:30:33.646123+0000","last_became_peered":"2024-04-25T10:30:33.646123+0000","last_unstale":"2024-04-25T10:31:33.938530+0000","last_undegraded":"2024-04-25T10:31:33.938530+0000","last_fullsized":"2024-04-25T10:31:33.938530+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:57.712216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.936931+0000","last_change":"2024-04-25T10:30:33.640463+0000","last_active":"2024-04-25T10:31:33.936931+0000","last_peered":"2024-04-25T10:31:33.936931+0000","last_clean":"2024-04-25T10:31:33.936931+0000","last_became_active":"2024-04-25T10:30:33.639931+0000","last_became_peered":"2024-04-25T10:30:33.639931+0000","last_unstale":"2024-04-25T10:31:33.936931+0000","last_undegraded":"2024-04-25T10:31:33.936931+0000","last_fullsized":"2024-04-25T10:31:33.936931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:18.863405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.938295+0000","last_change":"2024-04-25T10:30:33.645695+0000","last_active":"2024-04-25T10:31:33.938295+0000","last_peered":"2024-04-25T10:31:33.938295+0000","last_clean":"2024-04-25T10:31:33.938295+0000","last_became_active":"2024-04-25T10:30:33.645387+0000","last_became_peered":"2024-04-25T10:30:33.645387+0000","last_unstale":"2024-04-25T10:31:33.938295+0000","last_undegraded":"2024-04-25T10:31:33.938295+0000","last_fullsized":"2024-04-25T10:31:33.938295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:20.044456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937794+0000","last_change":"2024-04-25T10:30:33.639087+0000","last_active":"2024-04-25T10:31:33.937794+0000","last_peered":"2024-04-25T10:31:33.937794+0000","last_clean":"2024-04-25T10:31:33.937794+0000","last_became_active":"2024-04-25T10:30:33.638726+0000","last_became_peered":"2024-04-25T10:30:33.638726+0000","last_unstale":"2024-04-25T10:31:33.937794+0000","last_undegraded":"2024-04-25T10:31:33.937794+0000","last_fullsized":"2024-04-25T10:31:33.937794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:36.887289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.943067+0000","last_change":"2024-04-25T10:30:33.638646+0000","last_active":"2024-04-25T10:31:33.943067+0000","last_peered":"2024-04-25T10:31:33.943067+0000","last_clean":"2024-04-25T10:31:33.943067+0000","last_became_active":"2024-04-25T10:30:33.638261+0000","last_became_peered":"2024-04-25T10:30:33.638261+0000","last_unstale":"2024-04-25T10:31:33.943067+0000","last_undegraded":"2024-04-25T10:31:33.943067+0000","last_fullsized":"2024-04-25T10:31:33.943067+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:08.212310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937730+0000","last_change":"2024-04-25T10:30:32.899992+0000","last_active":"2024-04-25T10:31:33.937730+0000","last_peered":"2024-04-25T10:31:33.937730+0000","last_clean":"2024-04-25T10:31:33.937730+0000","last_became_active":"2024-04-25T10:30:32.899643+0000","last_became_peered":"2024-04-25T10:30:32.899643+0000","last_unstale":"2024-04-25T10:31:33.937730+0000","last_undegraded":"2024-04-25T10:31:33.937730+0000","last_fullsized":"2024-04-25T10:31:33.937730+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:34.245121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940447+0000","last_change":"2024-04-25T10:30:33.644884+0000","last_active":"2024-04-25T10:31:33.940447+0000","last_peered":"2024-04-25T10:31:33.940447+0000","last_clean":"2024-04-25T10:31:33.940447+0000","last_became_active":"2024-04-25T10:30:33.644510+0000","last_became_peered":"2024-04-25T10:30:33.644510+0000","last_unstale":"2024-04-25T10:31:33.940447+0000","last_undegraded":"2024-04-25T10:31:33.940447+0000","last_fullsized":"2024-04-25T10:31:33.940447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:09.282865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940526+0000","last_change":"2024-04-25T10:30:32.909175+0000","last_active":"2024-04-25T10:31:33.940526+0000","last_peered":"2024-04-25T10:31:33.940526+0000","last_clean":"2024-04-25T10:31:33.940526+0000","last_became_active":"2024-04-25T10:30:32.908813+0000","last_became_peered":"2024-04-25T10:30:32.908813+0000","last_unstale":"2024-04-25T10:31:33.940526+0000","last_undegraded":"2024-04-25T10:31:33.940526+0000","last_fullsized":"2024-04-25T10:31:33.940526+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:04.256452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.939102+0000","last_change":"2024-04-25T10:30:33.642095+0000","last_active":"2024-04-25T10:31:33.939102+0000","last_peered":"2024-04-25T10:31:33.939102+0000","last_clean":"2024-04-25T10:31:33.939102+0000","last_became_active":"2024-04-25T10:30:33.641713+0000","last_became_peered":"2024-04-25T10:30:33.641713+0000","last_unstale":"2024-04-25T10:31:33.939102+0000","last_undegraded":"2024-04-25T10:31:33.939102+0000","last_fullsized":"2024-04-25T10:31:33.939102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:42.425197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.936797+0000","last_change":"2024-04-25T10:30:32.904376+0000","last_active":"2024-04-25T10:31:33.936797+0000","last_peered":"2024-04-25T10:31:33.936797+0000","last_clean":"2024-04-25T10:31:33.936797+0000","last_became_active":"2024-04-25T10:30:32.903994+0000","last_became_peered":"2024-04-25T10:30:32.903994+0000","last_unstale":"2024-04-25T10:31:33.936797+0000","last_undegraded":"2024-04-25T10:31:33.936797+0000","last_fullsized":"2024-04-25T10:31:33.936797+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:52.727800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937192+0000","last_change":"2024-04-25T10:30:33.637171+0000","last_active":"2024-04-25T10:31:33.937192+0000","last_peered":"2024-04-25T10:31:33.937192+0000","last_clean":"2024-04-25T10:31:33.937192+0000","last_became_active":"2024-04-25T10:30:33.636822+0000","last_became_peered":"2024-04-25T10:30:33.636822+0000","last_unstale":"2024-04-25T10:31:33.937192+0000","last_undegraded":"2024-04-25T10:31:33.937192+0000","last_fullsized":"2024-04-25T10:31:33.937192+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:54.220784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.936429+0000","last_change":"2024-04-25T10:30:32.900856+0000","last_active":"2024-04-25T10:31:33.936429+0000","last_peered":"2024-04-25T10:31:33.936429+0000","last_clean":"2024-04-25T10:31:33.936429+0000","last_became_active":"2024-04-25T10:30:32.900519+0000","last_became_peered":"2024-04-25T10:30:32.900519+0000","last_unstale":"2024-04-25T10:31:33.936429+0000","last_undegraded":"2024-04-25T10:31:33.936429+0000","last_fullsized":"2024-04-25T10:31:33.936429+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:49.738005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941925+0000","last_change":"2024-04-25T10:30:33.651736+0000","last_active":"2024-04-25T10:31:33.941925+0000","last_peered":"2024-04-25T10:31:33.941925+0000","last_clean":"2024-04-25T10:31:33.941925+0000","last_became_active":"2024-04-25T10:30:33.651395+0000","last_became_peered":"2024-04-25T10:30:33.651395+0000","last_unstale":"2024-04-25T10:31:33.941925+0000","last_undegraded":"2024-04-25T10:31:33.941925+0000","last_fullsized":"2024-04-25T10:31:33.941925+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:58.230092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"77'5","reported_seq":97,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:46.906644+0000","last_change":"2024-04-25T10:30:32.901348+0000","last_active":"2024-04-25T10:31:46.906644+0000","last_peered":"2024-04-25T10:31:46.906644+0000","last_clean":"2024-04-25T10:31:46.906644+0000","last_became_active":"2024-04-25T10:30:32.901025+0000","last_became_peered":"2024-04-25T10:30:32.901025+0000","last_unstale":"2024-04-25T10:31:46.906644+0000","last_undegraded":"2024-04-25T10:31:46.906644+0000","last_fullsized":"2024-04-25T10:31:46.906644+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:00.521934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.936115+0000","last_change":"2024-04-25T10:30:33.649206+0000","last_active":"2024-04-25T10:31:33.936115+0000","last_peered":"2024-04-25T10:31:33.936115+0000","last_clean":"2024-04-25T10:31:33.936115+0000","last_became_active":"2024-04-25T10:30:33.648883+0000","last_became_peered":"2024-04-25T10:30:33.648883+0000","last_unstale":"2024-04-25T10:31:33.936115+0000","last_undegraded":"2024-04-25T10:31:33.936115+0000","last_fullsized":"2024-04-25T10:31:33.936115+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:43.105562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.938755+0000","last_change":"2024-04-25T10:30:32.901201+0000","last_active":"2024-04-25T10:31:33.938755+0000","last_peered":"2024-04-25T10:31:33.938755+0000","last_clean":"2024-04-25T10:31:33.938755+0000","last_became_active":"2024-04-25T10:30:32.900876+0000","last_became_peered":"2024-04-25T10:30:32.900876+0000","last_unstale":"2024-04-25T10:31:33.938755+0000","last_undegraded":"2024-04-25T10:31:33.938755+0000","last_fullsized":"2024-04-25T10:31:33.938755+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:58.077045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941479+0000","last_change":"2024-04-25T10:30:33.651935+0000","last_active":"2024-04-25T10:31:33.941479+0000","last_peered":"2024-04-25T10:31:33.941479+0000","last_clean":"2024-04-25T10:31:33.941479+0000","last_became_active":"2024-04-25T10:30:33.651568+0000","last_became_peered":"2024-04-25T10:30:33.651568+0000","last_unstale":"2024-04-25T10:31:33.941479+0000","last_undegraded":"2024-04-25T10:31:33.941479+0000","last_fullsized":"2024-04-25T10:31:33.941479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:58.849535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.935662+0000","last_change":"2024-04-25T10:30:32.896500+0000","last_active":"2024-04-25T10:31:33.935662+0000","last_peered":"2024-04-25T10:31:33.935662+0000","last_clean":"2024-04-25T10:31:33.935662+0000","last_became_active":"2024-04-25T10:30:32.896071+0000","last_became_peered":"2024-04-25T10:30:32.896071+0000","last_unstale":"2024-04-25T10:31:33.935662+0000","last_undegraded":"2024-04-25T10:31:33.935662+0000","last_fullsized":"2024-04-25T10:31:33.935662+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:10.385443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940171+0000","last_change":"2024-04-25T10:30:33.631730+0000","last_active":"2024-04-25T10:31:33.940171+0000","last_peered":"2024-04-25T10:31:33.940171+0000","last_clean":"2024-04-25T10:31:33.940171+0000","last_became_active":"2024-04-25T10:30:33.631374+0000","last_became_peered":"2024-04-25T10:30:33.631374+0000","last_unstale":"2024-04-25T10:31:33.940171+0000","last_undegraded":"2024-04-25T10:31:33.940171+0000","last_fullsized":"2024-04-25T10:31:33.940171+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:14.957866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941082+0000","last_change":"2024-04-25T10:30:32.900762+0000","last_active":"2024-04-25T10:31:33.941082+0000","last_peered":"2024-04-25T10:31:33.941082+0000","last_clean":"2024-04-25T10:31:33.941082+0000","last_became_active":"2024-04-25T10:30:32.900459+0000","last_became_peered":"2024-04-25T10:30:32.900459+0000","last_unstale":"2024-04-25T10:31:33.941082+0000","last_undegraded":"2024-04-25T10:31:33.941082+0000","last_fullsized":"2024-04-25T10:31:33.941082+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:22.286261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937757+0000","last_change":"2024-04-25T10:30:33.637727+0000","last_active":"2024-04-25T10:31:33.937757+0000","last_peered":"2024-04-25T10:31:33.937757+0000","last_clean":"2024-04-25T10:31:33.937757+0000","last_became_active":"2024-04-25T10:30:33.637359+0000","last_became_peered":"2024-04-25T10:30:33.637359+0000","last_unstale":"2024-04-25T10:31:33.937757+0000","last_undegraded":"2024-04-25T10:31:33.937757+0000","last_fullsized":"2024-04-25T10:31:33.937757+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:21.877562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'3","reported_seq":89,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:42.899972+0000","last_change":"2024-04-25T10:30:32.894210+0000","last_active":"2024-04-25T10:31:42.899972+0000","last_peered":"2024-04-25T10:31:42.899972+0000","last_clean":"2024-04-25T10:31:42.899972+0000","last_became_active":"2024-04-25T10:30:32.893241+0000","last_became_peered":"2024-04-25T10:30:32.893241+0000","last_unstale":"2024-04-25T10:31:42.899972+0000","last_undegraded":"2024-04-25T10:31:42.899972+0000","last_fullsized":"2024-04-25T10:31:42.899972+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:06.340392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937853+0000","last_change":"2024-04-25T10:30:33.632349+0000","last_active":"2024-04-25T10:31:33.937853+0000","last_peered":"2024-04-25T10:31:33.937853+0000","last_clean":"2024-04-25T10:31:33.937853+0000","last_became_active":"2024-04-25T10:30:33.631968+0000","last_became_peered":"2024-04-25T10:30:33.631968+0000","last_unstale":"2024-04-25T10:31:33.937853+0000","last_undegraded":"2024-04-25T10:31:33.937853+0000","last_fullsized":"2024-04-25T10:31:33.937853+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:29.090932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940786+0000","last_change":"2024-04-25T10:30:32.900105+0000","last_active":"2024-04-25T10:31:33.940786+0000","last_peered":"2024-04-25T10:31:33.940786+0000","last_clean":"2024-04-25T10:31:33.940786+0000","last_became_active":"2024-04-25T10:30:32.899740+0000","last_became_peered":"2024-04-25T10:30:32.899740+0000","last_unstale":"2024-04-25T10:31:33.940786+0000","last_undegraded":"2024-04-25T10:31:33.940786+0000","last_fullsized":"2024-04-25T10:31:33.940786+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:49.390799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.943769+0000","last_change":"2024-04-25T10:31:33.943769+0000","last_active":"2024-04-25T10:31:33.943769+0000","last_peered":"2024-04-25T10:31:33.943769+0000","last_clean":"2024-04-25T10:31:33.943769+0000","last_became_active":"2024-04-25T10:31:33.943331+0000","last_became_peered":"2024-04-25T10:31:33.943331+0000","last_unstale":"2024-04-25T10:31:33.943769+0000","last_undegraded":"2024-04-25T10:31:33.943769+0000","last_fullsized":"2024-04-25T10:31:33.943769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:07.183059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.942017+0000","last_change":"2024-04-25T10:30:32.893453+0000","last_active":"2024-04-25T10:31:33.942017+0000","last_peered":"2024-04-25T10:31:33.942017+0000","last_clean":"2024-04-25T10:31:33.942017+0000","last_became_active":"2024-04-25T10:30:32.893106+0000","last_became_peered":"2024-04-25T10:30:32.893106+0000","last_unstale":"2024-04-25T10:31:33.942017+0000","last_undegraded":"2024-04-25T10:31:33.942017+0000","last_fullsized":"2024-04-25T10:31:33.942017+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:57.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937345+0000","last_change":"2024-04-25T10:30:33.645693+0000","last_active":"2024-04-25T10:31:33.937345+0000","last_peered":"2024-04-25T10:31:33.937345+0000","last_clean":"2024-04-25T10:31:33.937345+0000","last_became_active":"2024-04-25T10:30:33.645342+0000","last_became_peered":"2024-04-25T10:30:33.645342+0000","last_unstale":"2024-04-25T10:31:33.937345+0000","last_undegraded":"2024-04-25T10:31:33.937345+0000","last_fullsized":"2024-04-25T10:31:33.937345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:15.729860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'28","reported_seq":207,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:43.958190+0000","last_change":"2024-04-25T10:31:33.949796+0000","last_active":"2024-04-25T10:31:43.958190+0000","last_peered":"2024-04-25T10:31:43.958190+0000","last_clean":"2024-04-25T10:31:43.958190+0000","last_became_active":"2024-04-25T10:31:33.942989+0000","last_became_peered":"2024-04-25T10:31:33.942989+0000","last_unstale":"2024-04-25T10:31:43.958190+0000","last_undegraded":"2024-04-25T10:31:43.958190+0000","last_fullsized":"2024-04-25T10:31:43.958190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:04.194643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":134,"num_read_kb":126,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61046,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.936604+0000","last_change":"2024-04-25T10:30:33.641768+0000","last_active":"2024-04-25T10:31:33.936604+0000","last_peered":"2024-04-25T10:31:33.936604+0000","last_clean":"2024-04-25T10:31:33.936604+0000","last_became_active":"2024-04-25T10:30:33.641415+0000","last_became_peered":"2024-04-25T10:30:33.641415+0000","last_unstale":"2024-04-25T10:31:33.936604+0000","last_undegraded":"2024-04-25T10:31:33.936604+0000","last_fullsized":"2024-04-25T10:31:33.936604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:38.569474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940233+0000","last_change":"2024-04-25T10:30:32.898703+0000","last_active":"2024-04-25T10:31:33.940233+0000","last_peered":"2024-04-25T10:31:33.940233+0000","last_clean":"2024-04-25T10:31:33.940233+0000","last_became_active":"2024-04-25T10:30:32.898340+0000","last_became_peered":"2024-04-25T10:30:32.898340+0000","last_unstale":"2024-04-25T10:31:33.940233+0000","last_undegraded":"2024-04-25T10:31:33.940233+0000","last_fullsized":"2024-04-25T10:31:33.940233+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:27.024144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.935624+0000","last_change":"2024-04-25T10:30:33.640393+0000","last_active":"2024-04-25T10:31:33.935624+0000","last_peered":"2024-04-25T10:31:33.935624+0000","last_clean":"2024-04-25T10:31:33.935624+0000","last_became_active":"2024-04-25T10:30:33.639822+0000","last_became_peered":"2024-04-25T10:30:33.639822+0000","last_unstale":"2024-04-25T10:31:33.935624+0000","last_undegraded":"2024-04-25T10:31:33.935624+0000","last_fullsized":"2024-04-25T10:31:33.935624+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:08.618406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941265+0000","last_change":"2024-04-25T10:30:32.900880+0000","last_active":"2024-04-25T10:31:33.941265+0000","last_peered":"2024-04-25T10:31:33.941265+0000","last_clean":"2024-04-25T10:31:33.941265+0000","last_became_active":"2024-04-25T10:30:32.899987+0000","last_became_peered":"2024-04-25T10:30:32.899987+0000","last_unstale":"2024-04-25T10:31:33.941265+0000","last_undegraded":"2024-04-25T10:31:33.941265+0000","last_fullsized":"2024-04-25T10:31:33.941265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:35.687773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941600+0000","last_change":"2024-04-25T10:30:33.641933+0000","last_active":"2024-04-25T10:31:33.941600+0000","last_peered":"2024-04-25T10:31:33.941600+0000","last_clean":"2024-04-25T10:31:33.941600+0000","last_became_active":"2024-04-25T10:30:33.641549+0000","last_became_peered":"2024-04-25T10:30:33.641549+0000","last_unstale":"2024-04-25T10:31:33.941600+0000","last_undegraded":"2024-04-25T10:31:33.941600+0000","last_fullsized":"2024-04-25T10:31:33.941600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:36.696523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'4","reported_seq":91,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:41.908641+0000","last_change":"2024-04-25T10:30:32.901178+0000","last_active":"2024-04-25T10:31:41.908641+0000","last_peered":"2024-04-25T10:31:41.908641+0000","last_clean":"2024-04-25T10:31:41.908641+0000","last_became_active":"2024-04-25T10:30:32.900817+0000","last_became_peered":"2024-04-25T10:30:32.900817+0000","last_unstale":"2024-04-25T10:31:41.908641+0000","last_undegraded":"2024-04-25T10:31:41.908641+0000","last_fullsized":"2024-04-25T10:31:41.908641+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:53.999253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.943470+0000","last_change":"2024-04-25T10:31:33.943470+0000","last_active":"2024-04-25T10:31:33.943470+0000","last_peered":"2024-04-25T10:31:33.943470+0000","last_clean":"2024-04-25T10:31:33.943470+0000","last_became_active":"2024-04-25T10:31:33.943098+0000","last_became_peered":"2024-04-25T10:31:33.943098+0000","last_unstale":"2024-04-25T10:31:33.943470+0000","last_undegraded":"2024-04-25T10:31:33.943470+0000","last_fullsized":"2024-04-25T10:31:33.943470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:27.707195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.935960+0000","last_change":"2024-04-25T10:30:32.902862+0000","last_active":"2024-04-25T10:31:33.935960+0000","last_peered":"2024-04-25T10:31:33.935960+0000","last_clean":"2024-04-25T10:31:33.935960+0000","last_became_active":"2024-04-25T10:30:32.902537+0000","last_became_peered":"2024-04-25T10:30:32.902537+0000","last_unstale":"2024-04-25T10:31:33.935960+0000","last_undegraded":"2024-04-25T10:31:33.935960+0000","last_fullsized":"2024-04-25T10:31:33.935960+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:55.974128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941592+0000","last_change":"2024-04-25T10:30:33.651409+0000","last_active":"2024-04-25T10:31:33.941592+0000","last_peered":"2024-04-25T10:31:33.941592+0000","last_clean":"2024-04-25T10:31:33.941592+0000","last_became_active":"2024-04-25T10:30:33.650673+0000","last_became_peered":"2024-04-25T10:30:33.650673+0000","last_unstale":"2024-04-25T10:31:33.941592+0000","last_undegraded":"2024-04-25T10:31:33.941592+0000","last_fullsized":"2024-04-25T10:31:33.941592+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:48.989198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.953944+0000","last_change":"2024-04-25T10:31:33.953408+0000","last_active":"2024-04-25T10:31:33.953944+0000","last_peered":"2024-04-25T10:31:33.953944+0000","last_clean":"2024-04-25T10:31:33.953944+0000","last_became_active":"2024-04-25T10:31:33.943495+0000","last_became_peered":"2024-04-25T10:31:33.943495+0000","last_unstale":"2024-04-25T10:31:33.953944+0000","last_undegraded":"2024-04-25T10:31:33.953944+0000","last_fullsized":"2024-04-25T10:31:33.953944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:03.947967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.938718+0000","last_change":"2024-04-25T10:30:33.643929+0000","last_active":"2024-04-25T10:31:33.938718+0000","last_peered":"2024-04-25T10:31:33.938718+0000","last_clean":"2024-04-25T10:31:33.938718+0000","last_became_active":"2024-04-25T10:30:33.643593+0000","last_became_peered":"2024-04-25T10:30:33.643593+0000","last_unstale":"2024-04-25T10:31:33.938718+0000","last_undegraded":"2024-04-25T10:31:33.938718+0000","last_fullsized":"2024-04-25T10:31:33.938718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:39.591022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937940+0000","last_change":"2024-04-25T10:30:32.898937+0000","last_active":"2024-04-25T10:31:33.937940+0000","last_peered":"2024-04-25T10:31:33.937940+0000","last_clean":"2024-04-25T10:31:33.937940+0000","last_became_active":"2024-04-25T10:30:32.898570+0000","last_became_peered":"2024-04-25T10:30:32.898570+0000","last_unstale":"2024-04-25T10:31:33.937940+0000","last_undegraded":"2024-04-25T10:31:33.937940+0000","last_fullsized":"2024-04-25T10:31:33.937940+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:40.123035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'1","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.939346+0000","last_change":"2024-04-25T10:30:33.632857+0000","last_active":"2024-04-25T10:31:33.939346+0000","last_peered":"2024-04-25T10:31:33.939346+0000","last_clean":"2024-04-25T10:31:33.939346+0000","last_became_active":"2024-04-25T10:30:33.632043+0000","last_became_peered":"2024-04-25T10:30:33.632043+0000","last_unstale":"2024-04-25T10:31:33.939346+0000","last_undegraded":"2024-04-25T10:31:33.939346+0000","last_fullsized":"2024-04-25T10:31:33.939346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:12.683035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940901+0000","last_change":"2024-04-25T10:30:32.901349+0000","last_active":"2024-04-25T10:31:33.940901+0000","last_peered":"2024-04-25T10:31:33.940901+0000","last_clean":"2024-04-25T10:31:33.940901+0000","last_became_active":"2024-04-25T10:30:32.900881+0000","last_became_peered":"2024-04-25T10:30:32.900881+0000","last_unstale":"2024-04-25T10:31:33.940901+0000","last_undegraded":"2024-04-25T10:31:33.940901+0000","last_fullsized":"2024-04-25T10:31:33.940901+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:34.963564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.939214+0000","last_change":"2024-04-25T10:30:33.644376+0000","last_active":"2024-04-25T10:31:33.939214+0000","last_peered":"2024-04-25T10:31:33.939214+0000","last_clean":"2024-04-25T10:31:33.939214+0000","last_became_active":"2024-04-25T10:30:33.644055+0000","last_became_peered":"2024-04-25T10:30:33.644055+0000","last_unstale":"2024-04-25T10:31:33.939214+0000","last_undegraded":"2024-04-25T10:31:33.939214+0000","last_fullsized":"2024-04-25T10:31:33.939214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:48.134775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941290+0000","last_change":"2024-04-25T10:30:33.646312+0000","last_active":"2024-04-25T10:31:33.941290+0000","last_peered":"2024-04-25T10:31:33.941290+0000","last_clean":"2024-04-25T10:31:33.941290+0000","last_became_active":"2024-04-25T10:30:33.642338+0000","last_became_peered":"2024-04-25T10:30:33.642338+0000","last_unstale":"2024-04-25T10:31:33.941290+0000","last_undegraded":"2024-04-25T10:31:33.941290+0000","last_fullsized":"2024-04-25T10:31:33.941290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:42.588360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"77'4","reported_seq":88,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:41.907931+0000","last_change":"2024-04-25T10:30:32.909680+0000","last_active":"2024-04-25T10:31:41.907931+0000","last_peered":"2024-04-25T10:31:41.907931+0000","last_clean":"2024-04-25T10:31:41.907931+0000","last_became_active":"2024-04-25T10:30:32.909169+0000","last_became_peered":"2024-04-25T10:30:32.909169+0000","last_unstale":"2024-04-25T10:31:41.907931+0000","last_undegraded":"2024-04-25T10:31:41.907931+0000","last_fullsized":"2024-04-25T10:31:41.907931+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:54.876475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940403+0000","last_change":"2024-04-25T10:30:33.649524+0000","last_active":"2024-04-25T10:31:33.940403+0000","last_peered":"2024-04-25T10:31:33.940403+0000","last_clean":"2024-04-25T10:31:33.940403+0000","last_became_active":"2024-04-25T10:30:33.648958+0000","last_became_peered":"2024-04-25T10:30:33.648958+0000","last_unstale":"2024-04-25T10:31:33.940403+0000","last_undegraded":"2024-04-25T10:31:33.940403+0000","last_fullsized":"2024-04-25T10:31:33.940403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:13.407742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941491+0000","last_change":"2024-04-25T10:30:32.895270+0000","last_active":"2024-04-25T10:31:33.941491+0000","last_peered":"2024-04-25T10:31:33.941491+0000","last_clean":"2024-04-25T10:31:33.941491+0000","last_became_active":"2024-04-25T10:30:32.894899+0000","last_became_peered":"2024-04-25T10:30:32.894899+0000","last_unstale":"2024-04-25T10:31:33.941491+0000","last_undegraded":"2024-04-25T10:31:33.941491+0000","last_fullsized":"2024-04-25T10:31:33.941491+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:21.233217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":95,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941397+0000","last_change":"2024-04-25T10:29:21.105073+0000","last_active":"2024-04-25T10:31:33.941397+0000","last_peered":"2024-04-25T10:31:33.941397+0000","last_clean":"2024-04-25T10:31:33.941397+0000","last_became_active":"2024-04-25T10:29:21.098771+0000","last_became_peered":"2024-04-25T10:29:21.098771+0000","last_unstale":"2024-04-25T10:31:33.941397+0000","last_undegraded":"2024-04-25T10:31:33.941397+0000","last_fullsized":"2024-04-25T10:31:33.941397+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:59.015440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:59.015440+0000","last_clean_scrub_stamp":"2024-04-25T10:25:59.015440+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:41.728977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.935487+0000","last_change":"2024-04-25T10:30:33.639159+0000","last_active":"2024-04-25T10:31:33.935487+0000","last_peered":"2024-04-25T10:31:33.935487+0000","last_clean":"2024-04-25T10:31:33.935487+0000","last_became_active":"2024-04-25T10:30:33.638761+0000","last_became_peered":"2024-04-25T10:30:33.638761+0000","last_unstale":"2024-04-25T10:31:33.935487+0000","last_undegraded":"2024-04-25T10:31:33.935487+0000","last_fullsized":"2024-04-25T10:31:33.935487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:43.262103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941588+0000","last_change":"2024-04-25T10:30:32.898587+0000","last_active":"2024-04-25T10:31:33.941588+0000","last_peered":"2024-04-25T10:31:33.941588+0000","last_clean":"2024-04-25T10:31:33.941588+0000","last_became_active":"2024-04-25T10:30:32.898113+0000","last_became_peered":"2024-04-25T10:30:32.898113+0000","last_unstale":"2024-04-25T10:31:33.941588+0000","last_undegraded":"2024-04-25T10:31:33.941588+0000","last_fullsized":"2024-04-25T10:31:33.941588+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:24.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.936615+0000","last_change":"2024-04-25T10:30:32.907188+0000","last_active":"2024-04-25T10:31:33.936615+0000","last_peered":"2024-04-25T10:31:33.936615+0000","last_clean":"2024-04-25T10:31:33.936615+0000","last_became_active":"2024-04-25T10:30:32.906837+0000","last_became_peered":"2024-04-25T10:30:32.906837+0000","last_unstale":"2024-04-25T10:31:33.936615+0000","last_undegraded":"2024-04-25T10:31:33.936615+0000","last_fullsized":"2024-04-25T10:31:33.936615+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:11.835053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.935910+0000","last_change":"2024-04-25T10:30:33.636159+0000","last_active":"2024-04-25T10:31:33.935910+0000","last_peered":"2024-04-25T10:31:33.935910+0000","last_clean":"2024-04-25T10:31:33.935910+0000","last_became_active":"2024-04-25T10:30:33.635098+0000","last_became_peered":"2024-04-25T10:30:33.635098+0000","last_unstale":"2024-04-25T10:31:33.935910+0000","last_undegraded":"2024-04-25T10:31:33.935910+0000","last_fullsized":"2024-04-25T10:31:33.935910+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:13.092368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'8","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937759+0000","last_change":"2024-04-25T10:30:32.905469+0000","last_active":"2024-04-25T10:31:33.937759+0000","last_peered":"2024-04-25T10:31:33.937759+0000","last_clean":"2024-04-25T10:31:33.937759+0000","last_became_active":"2024-04-25T10:30:32.905123+0000","last_became_peered":"2024-04-25T10:30:32.905123+0000","last_unstale":"2024-04-25T10:31:33.937759+0000","last_undegraded":"2024-04-25T10:31:33.937759+0000","last_fullsized":"2024-04-25T10:31:33.937759+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:12.884809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940634+0000","last_change":"2024-04-25T10:30:33.640690+0000","last_active":"2024-04-25T10:31:33.940634+0000","last_peered":"2024-04-25T10:31:33.940634+0000","last_clean":"2024-04-25T10:31:33.940634+0000","last_became_active":"2024-04-25T10:30:33.640310+0000","last_became_peered":"2024-04-25T10:30:33.640310+0000","last_unstale":"2024-04-25T10:31:33.940634+0000","last_undegraded":"2024-04-25T10:31:33.940634+0000","last_fullsized":"2024-04-25T10:31:33.940634+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:03.671495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941790+0000","last_change":"2024-04-25T10:30:32.893743+0000","last_active":"2024-04-25T10:31:33.941790+0000","last_peered":"2024-04-25T10:31:33.941790+0000","last_clean":"2024-04-25T10:31:33.941790+0000","last_became_active":"2024-04-25T10:30:32.893360+0000","last_became_peered":"2024-04-25T10:30:32.893360+0000","last_unstale":"2024-04-25T10:31:33.941790+0000","last_undegraded":"2024-04-25T10:31:33.941790+0000","last_fullsized":"2024-04-25T10:31:33.941790+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:39.830376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.939052+0000","last_change":"2024-04-25T10:30:33.645127+0000","last_active":"2024-04-25T10:31:33.939052+0000","last_peered":"2024-04-25T10:31:33.939052+0000","last_clean":"2024-04-25T10:31:33.939052+0000","last_became_active":"2024-04-25T10:30:33.644259+0000","last_became_peered":"2024-04-25T10:30:33.644259+0000","last_unstale":"2024-04-25T10:31:33.939052+0000","last_undegraded":"2024-04-25T10:31:33.939052+0000","last_fullsized":"2024-04-25T10:31:33.939052+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:56.849861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.936945+0000","last_change":"2024-04-25T10:30:32.896885+0000","last_active":"2024-04-25T10:31:33.936945+0000","last_peered":"2024-04-25T10:31:33.936945+0000","last_clean":"2024-04-25T10:31:33.936945+0000","last_became_active":"2024-04-25T10:30:32.896447+0000","last_became_peered":"2024-04-25T10:30:32.896447+0000","last_unstale":"2024-04-25T10:31:33.936945+0000","last_undegraded":"2024-04-25T10:31:33.936945+0000","last_fullsized":"2024-04-25T10:31:33.936945+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:13.133549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.935959+0000","last_change":"2024-04-25T10:30:33.645016+0000","last_active":"2024-04-25T10:31:33.935959+0000","last_peered":"2024-04-25T10:31:33.935959+0000","last_clean":"2024-04-25T10:31:33.935959+0000","last_became_active":"2024-04-25T10:30:33.644661+0000","last_became_peered":"2024-04-25T10:30:33.644661+0000","last_unstale":"2024-04-25T10:31:33.935959+0000","last_undegraded":"2024-04-25T10:31:33.935959+0000","last_fullsized":"2024-04-25T10:31:33.935959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:20.935124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940182+0000","last_change":"2024-04-25T10:30:32.907827+0000","last_active":"2024-04-25T10:31:33.940182+0000","last_peered":"2024-04-25T10:31:33.940182+0000","last_clean":"2024-04-25T10:31:33.940182+0000","last_became_active":"2024-04-25T10:30:32.907402+0000","last_became_peered":"2024-04-25T10:30:32.907402+0000","last_unstale":"2024-04-25T10:31:33.940182+0000","last_undegraded":"2024-04-25T10:31:33.940182+0000","last_fullsized":"2024-04-25T10:31:33.940182+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:38.501617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937725+0000","last_change":"2024-04-25T10:30:33.636935+0000","last_active":"2024-04-25T10:31:33.937725+0000","last_peered":"2024-04-25T10:31:33.937725+0000","last_clean":"2024-04-25T10:31:33.937725+0000","last_became_active":"2024-04-25T10:30:33.636566+0000","last_became_peered":"2024-04-25T10:30:33.636566+0000","last_unstale":"2024-04-25T10:31:33.937725+0000","last_undegraded":"2024-04-25T10:31:33.937725+0000","last_fullsized":"2024-04-25T10:31:33.937725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:24.556275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.947259+0000","last_change":"2024-04-25T10:31:33.946786+0000","last_active":"2024-04-25T10:31:33.947259+0000","last_peered":"2024-04-25T10:31:33.947259+0000","last_clean":"2024-04-25T10:31:33.947259+0000","last_became_active":"2024-04-25T10:31:33.941298+0000","last_became_peered":"2024-04-25T10:31:33.941298+0000","last_unstale":"2024-04-25T10:31:33.947259+0000","last_undegraded":"2024-04-25T10:31:33.947259+0000","last_fullsized":"2024-04-25T10:31:33.947259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:55.009673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940260+0000","last_change":"2024-04-25T10:30:33.642986+0000","last_active":"2024-04-25T10:31:33.940260+0000","last_peered":"2024-04-25T10:31:33.940260+0000","last_clean":"2024-04-25T10:31:33.940260+0000","last_became_active":"2024-04-25T10:30:33.642545+0000","last_became_peered":"2024-04-25T10:30:33.642545+0000","last_unstale":"2024-04-25T10:31:33.940260+0000","last_undegraded":"2024-04-25T10:31:33.940260+0000","last_fullsized":"2024-04-25T10:31:33.940260+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:49.369648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.936250+0000","last_change":"2024-04-25T10:30:32.902239+0000","last_active":"2024-04-25T10:31:33.936250+0000","last_peered":"2024-04-25T10:31:33.936250+0000","last_clean":"2024-04-25T10:31:33.936250+0000","last_became_active":"2024-04-25T10:30:32.901653+0000","last_became_peered":"2024-04-25T10:30:32.901653+0000","last_unstale":"2024-04-25T10:31:33.936250+0000","last_undegraded":"2024-04-25T10:31:33.936250+0000","last_fullsized":"2024-04-25T10:31:33.936250+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:34.732428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941029+0000","last_change":"2024-04-25T10:30:33.639763+0000","last_active":"2024-04-25T10:31:33.941029+0000","last_peered":"2024-04-25T10:31:33.941029+0000","last_clean":"2024-04-25T10:31:33.941029+0000","last_became_active":"2024-04-25T10:30:33.638004+0000","last_became_peered":"2024-04-25T10:30:33.638004+0000","last_unstale":"2024-04-25T10:31:33.941029+0000","last_undegraded":"2024-04-25T10:31:33.941029+0000","last_fullsized":"2024-04-25T10:31:33.941029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:09.722333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940617+0000","last_change":"2024-04-25T10:30:32.894823+0000","last_active":"2024-04-25T10:31:33.940617+0000","last_peered":"2024-04-25T10:31:33.940617+0000","last_clean":"2024-04-25T10:31:33.940617+0000","last_became_active":"2024-04-25T10:30:32.893819+0000","last_became_peered":"2024-04-25T10:30:32.893819+0000","last_unstale":"2024-04-25T10:31:33.940617+0000","last_undegraded":"2024-04-25T10:31:33.940617+0000","last_fullsized":"2024-04-25T10:31:33.940617+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:59.809410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941124+0000","last_change":"2024-04-25T10:30:33.637046+0000","last_active":"2024-04-25T10:31:33.941124+0000","last_peered":"2024-04-25T10:31:33.941124+0000","last_clean":"2024-04-25T10:31:33.941124+0000","last_became_active":"2024-04-25T10:30:33.634903+0000","last_became_peered":"2024-04-25T10:30:33.634903+0000","last_unstale":"2024-04-25T10:31:33.941124+0000","last_undegraded":"2024-04-25T10:31:33.941124+0000","last_fullsized":"2024-04-25T10:31:33.941124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:52.252812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941907+0000","last_change":"2024-04-25T10:30:32.898979+0000","last_active":"2024-04-25T10:31:33.941907+0000","last_peered":"2024-04-25T10:31:33.941907+0000","last_clean":"2024-04-25T10:31:33.941907+0000","last_became_active":"2024-04-25T10:30:32.898611+0000","last_became_peered":"2024-04-25T10:30:32.898611+0000","last_unstale":"2024-04-25T10:31:33.941907+0000","last_undegraded":"2024-04-25T10:31:33.941907+0000","last_fullsized":"2024-04-25T10:31:33.941907+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.549769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'1","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940827+0000","last_change":"2024-04-25T10:30:33.651596+0000","last_active":"2024-04-25T10:31:33.940827+0000","last_peered":"2024-04-25T10:31:33.940827+0000","last_clean":"2024-04-25T10:31:33.940827+0000","last_became_active":"2024-04-25T10:30:33.651140+0000","last_became_peered":"2024-04-25T10:30:33.651140+0000","last_unstale":"2024-04-25T10:31:33.940827+0000","last_undegraded":"2024-04-25T10:31:33.940827+0000","last_fullsized":"2024-04-25T10:31:33.940827+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:16.762135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.935870+0000","last_change":"2024-04-25T10:30:32.909982+0000","last_active":"2024-04-25T10:31:33.935870+0000","last_peered":"2024-04-25T10:31:33.935870+0000","last_clean":"2024-04-25T10:31:33.935870+0000","last_became_active":"2024-04-25T10:30:32.909506+0000","last_became_peered":"2024-04-25T10:30:32.909506+0000","last_unstale":"2024-04-25T10:31:33.935870+0000","last_undegraded":"2024-04-25T10:31:33.935870+0000","last_fullsized":"2024-04-25T10:31:33.935870+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:47.396963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941351+0000","last_change":"2024-04-25T10:30:33.642913+0000","last_active":"2024-04-25T10:31:33.941351+0000","last_peered":"2024-04-25T10:31:33.941351+0000","last_clean":"2024-04-25T10:31:33.941351+0000","last_became_active":"2024-04-25T10:30:33.642614+0000","last_became_peered":"2024-04-25T10:30:33.642614+0000","last_unstale":"2024-04-25T10:31:33.941351+0000","last_undegraded":"2024-04-25T10:31:33.941351+0000","last_fullsized":"2024-04-25T10:31:33.941351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:56.651230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937774+0000","last_change":"2024-04-25T10:30:33.644525+0000","last_active":"2024-04-25T10:31:33.937774+0000","last_peered":"2024-04-25T10:31:33.937774+0000","last_clean":"2024-04-25T10:31:33.937774+0000","last_became_active":"2024-04-25T10:30:33.644188+0000","last_became_peered":"2024-04-25T10:30:33.644188+0000","last_unstale":"2024-04-25T10:31:33.937774+0000","last_undegraded":"2024-04-25T10:31:33.937774+0000","last_fullsized":"2024-04-25T10:31:33.937774+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:28.625695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.946016+0000","last_change":"2024-04-25T10:31:33.946016+0000","last_active":"2024-04-25T10:31:33.946016+0000","last_peered":"2024-04-25T10:31:33.946016+0000","last_clean":"2024-04-25T10:31:33.946016+0000","last_became_active":"2024-04-25T10:31:33.945611+0000","last_became_peered":"2024-04-25T10:31:33.945611+0000","last_unstale":"2024-04-25T10:31:33.946016+0000","last_undegraded":"2024-04-25T10:31:33.946016+0000","last_fullsized":"2024-04-25T10:31:33.946016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:02.352885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940260+0000","last_change":"2024-04-25T10:30:33.644721+0000","last_active":"2024-04-25T10:31:33.940260+0000","last_peered":"2024-04-25T10:31:33.940260+0000","last_clean":"2024-04-25T10:31:33.940260+0000","last_became_active":"2024-04-25T10:30:33.644392+0000","last_became_peered":"2024-04-25T10:30:33.644392+0000","last_unstale":"2024-04-25T10:31:33.940260+0000","last_undegraded":"2024-04-25T10:31:33.940260+0000","last_fullsized":"2024-04-25T10:31:33.940260+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:50.571318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.938762+0000","last_change":"2024-04-25T10:30:33.647970+0000","last_active":"2024-04-25T10:31:33.938762+0000","last_peered":"2024-04-25T10:31:33.938762+0000","last_clean":"2024-04-25T10:31:33.938762+0000","last_became_active":"2024-04-25T10:30:33.647547+0000","last_became_peered":"2024-04-25T10:30:33.647547+0000","last_unstale":"2024-04-25T10:31:33.938762+0000","last_undegraded":"2024-04-25T10:31:33.938762+0000","last_fullsized":"2024-04-25T10:31:33.938762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:25.609085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.935784+0000","last_change":"2024-04-25T10:30:33.641618+0000","last_active":"2024-04-25T10:31:33.935784+0000","last_peered":"2024-04-25T10:31:33.935784+0000","last_clean":"2024-04-25T10:31:33.935784+0000","last_became_active":"2024-04-25T10:30:33.640830+0000","last_became_peered":"2024-04-25T10:30:33.640830+0000","last_unstale":"2024-04-25T10:31:33.935784+0000","last_undegraded":"2024-04-25T10:31:33.935784+0000","last_fullsized":"2024-04-25T10:31:33.935784+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:36.870127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.938525+0000","last_change":"2024-04-25T10:30:33.644010+0000","last_active":"2024-04-25T10:31:33.938525+0000","last_peered":"2024-04-25T10:31:33.938525+0000","last_clean":"2024-04-25T10:31:33.938525+0000","last_became_active":"2024-04-25T10:30:33.643639+0000","last_became_peered":"2024-04-25T10:30:33.643639+0000","last_unstale":"2024-04-25T10:31:33.938525+0000","last_undegraded":"2024-04-25T10:31:33.938525+0000","last_fullsized":"2024-04-25T10:31:33.938525+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:32.820552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.943425+0000","last_change":"2024-04-25T10:30:33.639727+0000","last_active":"2024-04-25T10:31:33.943425+0000","last_peered":"2024-04-25T10:31:33.943425+0000","last_clean":"2024-04-25T10:31:33.943425+0000","last_became_active":"2024-04-25T10:30:33.639185+0000","last_became_peered":"2024-04-25T10:30:33.639185+0000","last_unstale":"2024-04-25T10:31:33.943425+0000","last_undegraded":"2024-04-25T10:31:33.943425+0000","last_fullsized":"2024-04-25T10:31:33.943425+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:17.860864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940540+0000","last_change":"2024-04-25T10:30:33.644650+0000","last_active":"2024-04-25T10:31:33.940540+0000","last_peered":"2024-04-25T10:31:33.940540+0000","last_clean":"2024-04-25T10:31:33.940540+0000","last_became_active":"2024-04-25T10:30:33.644264+0000","last_became_peered":"2024-04-25T10:30:33.644264+0000","last_unstale":"2024-04-25T10:31:33.940540+0000","last_undegraded":"2024-04-25T10:31:33.940540+0000","last_fullsized":"2024-04-25T10:31:33.940540+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:41.122318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941199+0000","last_change":"2024-04-25T10:30:33.651875+0000","last_active":"2024-04-25T10:31:33.941199+0000","last_peered":"2024-04-25T10:31:33.941199+0000","last_clean":"2024-04-25T10:31:33.941199+0000","last_became_active":"2024-04-25T10:30:33.651486+0000","last_became_peered":"2024-04-25T10:30:33.651486+0000","last_unstale":"2024-04-25T10:31:33.941199+0000","last_undegraded":"2024-04-25T10:31:33.941199+0000","last_fullsized":"2024-04-25T10:31:33.941199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:32.598209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.942690+0000","last_change":"2024-04-25T10:31:33.942171+0000","last_active":"2024-04-25T10:31:33.942690+0000","last_peered":"2024-04-25T10:31:33.942690+0000","last_clean":"2024-04-25T10:31:33.942690+0000","last_became_active":"2024-04-25T10:31:33.941684+0000","last_became_peered":"2024-04-25T10:31:33.941684+0000","last_unstale":"2024-04-25T10:31:33.942690+0000","last_undegraded":"2024-04-25T10:31:33.942690+0000","last_fullsized":"2024-04-25T10:31:33.942690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:45.291952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937157+0000","last_change":"2024-04-25T10:30:33.639915+0000","last_active":"2024-04-25T10:31:33.937157+0000","last_peered":"2024-04-25T10:31:33.937157+0000","last_clean":"2024-04-25T10:31:33.937157+0000","last_became_active":"2024-04-25T10:30:33.639137+0000","last_became_peered":"2024-04-25T10:30:33.639137+0000","last_unstale":"2024-04-25T10:31:33.937157+0000","last_undegraded":"2024-04-25T10:31:33.937157+0000","last_fullsized":"2024-04-25T10:31:33.937157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:13.143405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.935743+0000","last_change":"2024-04-25T10:30:33.643547+0000","last_active":"2024-04-25T10:31:33.935743+0000","last_peered":"2024-04-25T10:31:33.935743+0000","last_clean":"2024-04-25T10:31:33.935743+0000","last_became_active":"2024-04-25T10:30:33.643158+0000","last_became_peered":"2024-04-25T10:30:33.643158+0000","last_unstale":"2024-04-25T10:31:33.935743+0000","last_undegraded":"2024-04-25T10:31:33.935743+0000","last_fullsized":"2024-04-25T10:31:33.935743+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:07.517132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.938270+0000","last_change":"2024-04-25T10:30:33.645746+0000","last_active":"2024-04-25T10:31:33.938270+0000","last_peered":"2024-04-25T10:31:33.938270+0000","last_clean":"2024-04-25T10:31:33.938270+0000","last_became_active":"2024-04-25T10:30:33.644646+0000","last_became_peered":"2024-04-25T10:30:33.644646+0000","last_unstale":"2024-04-25T10:31:33.938270+0000","last_undegraded":"2024-04-25T10:31:33.938270+0000","last_fullsized":"2024-04-25T10:31:33.938270+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:21.591264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65047,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":420,"num_read_kb":276,"num_write":156,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":61068,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":195141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645341,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":856064,"data_stored":413481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841569,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36852,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708300,"statfs":{"total":95995035648,"available":95957299200,"internally_reserved":0,"allocated":1548288,"data_stored":1066377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037799,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":868352,"data_stored":413695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234028,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":864256,"data_stored":413593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430257,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32612,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712540,"statfs":{"total":95995035648,"available":95961640960,"internally_reserved":0,"allocated":1462272,"data_stored":1005545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626484,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":856064,"data_stored":413084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":473923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40360,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704792,"statfs":{"total":95995035648,"available":95953707008,"internally_reserved":0,"allocated":876544,"data_stored":414355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":872448,"data_stored":414227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":872448,"data_stored":413395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986239,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":474958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822714,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36780,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708372,"statfs":{"total":95995035648,"available":95957372928,"internally_reserved":0,"allocated":1474560,"data_stored":1004838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:31:49.666 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T10:31:49.922 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:31:50.318 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:50 smithi043 ceph-mon[29478]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:31:50.318 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:50 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:31:50.318 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:50 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:31:50.318 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:50 smithi043 ceph-mon[29478]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:50.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:50 smithi077 ceph-mon[32772]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:31:50.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:50 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:31:50.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:50 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:31:50.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:50 smithi077 ceph-mon[32772]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:50.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:50 smithi164 ceph-mon[32713]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:31:50.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:50 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:31:50.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:50 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:31:50.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:50 smithi164 ceph-mon[32713]: from='mgr.14152 172.21.15.43:0/2649468352' entity='mgr.x' 2024-04-25T10:31:51.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.606+0000 7fdb9547a700 1 -- 172.21.15.43:0/2634760578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb901033c0 msgr2=0x7fdb901037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:51.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.606+0000 7fdb9547a700 1 --2- 172.21.15.43:0/2634760578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb901033c0 0x7fdb901037d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdb800099c0 tx=0x7fdb80009cd0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.606+0000 7fdb9547a700 1 -- 172.21.15.43:0/2634760578 shutdown_connections 2024-04-25T10:31:51.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.606+0000 7fdb9547a700 1 --2- 172.21.15.43:0/2634760578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb90108640 0x7fdb9010ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.606+0000 7fdb9547a700 1 --2- 172.21.15.43:0/2634760578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb901033c0 0x7fdb901037d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.606+0000 7fdb9547a700 1 --2- 172.21.15.43:0/2634760578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb90107cf0 0x7fdb90108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.606+0000 7fdb9547a700 1 -- 172.21.15.43:0/2634760578 >> 172.21.15.43:0/2634760578 conn(0x7fdb900fedb0 msgr2=0x7fdb901011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:51.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.606+0000 7fdb9547a700 1 -- 172.21.15.43:0/2634760578 shutdown_connections 2024-04-25T10:31:51.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.606+0000 7fdb9547a700 1 -- 172.21.15.43:0/2634760578 wait complete. 2024-04-25T10:31:51.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.607+0000 7fdb9547a700 1 Processor -- start 2024-04-25T10:31:51.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.607+0000 7fdb9547a700 1 -- start start 2024-04-25T10:31:51.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.608+0000 7fdb9547a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb901033c0 0x7fdb901a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:51.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.608+0000 7fdb9547a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb90107cf0 0x7fdb901a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:51.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.608+0000 7fdb9547a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb90108640 0x7fdb9019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:51.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.608+0000 7fdb9547a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdb90113270 con 0x7fdb90108640 2024-04-25T10:31:51.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.608+0000 7fdb9547a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901130f0 con 0x7fdb901033c0 2024-04-25T10:31:51.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.608+0000 7fdb9547a700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901133f0 con 0x7fdb90107cf0 2024-04-25T10:31:51.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.608+0000 7fdb8f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb90108640 0x7fdb9019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.608+0000 7fdb8f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb90108640 0x7fdb9019ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50902/0 (socket says 172.21.15.43:50902) 2024-04-25T10:31:51.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.608+0000 7fdb8f7fe700 1 -- 172.21.15.43:0/4059621894 learned_addr learned my addr 172.21.15.43:0/4059621894 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:51.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.608+0000 7fdb8effd700 1 --2- 172.21.15.43:0/4059621894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb901033c0 0x7fdb901a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.608+0000 7fdb8e7fc700 1 --2- 172.21.15.43:0/4059621894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb90107cf0 0x7fdb901a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.608+0000 7fdb8f7fe700 1 -- 172.21.15.43:0/4059621894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb90107cf0 msgr2=0x7fdb901a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:51.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.608+0000 7fdb8f7fe700 1 --2- 172.21.15.43:0/4059621894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb90107cf0 0x7fdb901a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.608+0000 7fdb8f7fe700 1 -- 172.21.15.43:0/4059621894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb901033c0 msgr2=0x7fdb901a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:51.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.608+0000 7fdb8f7fe700 1 --2- 172.21.15.43:0/4059621894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb901033c0 0x7fdb901a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.608+0000 7fdb8f7fe700 1 -- 172.21.15.43:0/4059621894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9019b1e0 con 0x7fdb90108640 2024-04-25T10:31:51.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.609+0000 7fdb8f7fe700 1 --2- 172.21.15.43:0/4059621894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb90108640 0x7fdb9019ab00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800ada0 tx=0x7fdb8800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:51.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.609+0000 7fdb77fff700 1 -- 172.21.15.43:0/4059621894 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88017020 con 0x7fdb90108640 2024-04-25T10:31:51.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.609+0000 7fdb9547a700 1 -- 172.21.15.43:0/4059621894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9019b4d0 con 0x7fdb90108640 2024-04-25T10:31:51.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.609+0000 7fdb77fff700 1 -- 172.21.15.43:0/4059621894 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb8800ce30 con 0x7fdb90108640 2024-04-25T10:31:51.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.609+0000 7fdb77fff700 1 -- 172.21.15.43:0/4059621894 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88005350 con 0x7fdb90108640 2024-04-25T10:31:51.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.609+0000 7fdb9547a700 1 -- 172.21.15.43:0/4059621894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb901a7770 con 0x7fdb90108640 2024-04-25T10:31:51.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.610+0000 7fdb9547a700 1 -- 172.21.15.43:0/4059621894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb90051190 con 0x7fdb90108640 2024-04-25T10:31:51.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.611+0000 7fdb77fff700 1 -- 172.21.15.43:0/4059621894 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fdb8800c980 con 0x7fdb90108640 2024-04-25T10:31:51.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.612+0000 7fdb77fff700 1 --2- 172.21.15.43:0/4059621894 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fdb780afc10 0x7fdb780b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:51.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.612+0000 7fdb8effd700 1 --2- 172.21.15.43:0/4059621894 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fdb780afc10 0x7fdb780b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.612+0000 7fdb77fff700 1 -- 172.21.15.43:0/4059621894 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fdb880dea80 con 0x7fdb90108640 2024-04-25T10:31:51.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.613+0000 7fdb8effd700 1 --2- 172.21.15.43:0/4059621894 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fdb780afc10 0x7fdb780b2120 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdb9019bc40 tx=0x7fdb8001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:51.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.616+0000 7fdb77fff700 1 -- 172.21.15.43:0/4059621894 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb88021020 con 0x7fdb90108640 2024-04-25T10:31:51.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.818+0000 7fdb9547a700 1 -- 172.21.15.43:0/4059621894 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdb9019bff0 con 0x7fdb780afc10 2024-04-25T10:31:51.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.825+0000 7fdb77fff700 1 -- 172.21.15.43:0/4059621894 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258710 (secure 0 0 0) 0x7fdb9019bff0 con 0x7fdb780afc10 2024-04-25T10:31:51.830 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:31:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.828+0000 7fdb9547a700 1 -- 172.21.15.43:0/4059621894 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fdb780afc10 msgr2=0x7fdb780b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.828+0000 7fdb9547a700 1 --2- 172.21.15.43:0/4059621894 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fdb780afc10 0x7fdb780b2120 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdb9019bc40 tx=0x7fdb8001a040 comp rx=0 tx=0).stop 2024-04-25T10:31:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.828+0000 7fdb9547a700 1 -- 172.21.15.43:0/4059621894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb90108640 msgr2=0x7fdb9019ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.828+0000 7fdb9547a700 1 --2- 172.21.15.43:0/4059621894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb90108640 0x7fdb9019ab00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800ada0 tx=0x7fdb8800c330 comp rx=0 tx=0).stop 2024-04-25T10:31:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.828+0000 7fdb9547a700 1 -- 172.21.15.43:0/4059621894 shutdown_connections 2024-04-25T10:31:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.828+0000 7fdb9547a700 1 --2- 172.21.15.43:0/4059621894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb90108640 0x7fdb9019ab00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.828+0000 7fdb9547a700 1 --2- 172.21.15.43:0/4059621894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb901033c0 0x7fdb901a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.828+0000 7fdb9547a700 1 --2- 172.21.15.43:0/4059621894 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fdb780afc10 0x7fdb780b2120 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.828+0000 7fdb9547a700 1 --2- 172.21.15.43:0/4059621894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb90107cf0 0x7fdb901a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.828+0000 7fdb9547a700 1 -- 172.21.15.43:0/4059621894 >> 172.21.15.43:0/4059621894 conn(0x7fdb900fedb0 msgr2=0x7fdb901073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.828+0000 7fdb9547a700 1 -- 172.21.15.43:0/4059621894 shutdown_connections 2024-04-25T10:31:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:51.828+0000 7fdb9547a700 1 -- 172.21.15.43:0/4059621894 wait complete. 2024-04-25T10:31:51.834 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T10:31:51.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:51 smithi043 ceph-mon[29478]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T10:31:52.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:51 smithi077 ceph-mon[32772]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T10:31:52.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:51 smithi164 ceph-mon[32713]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T10:31:52.472 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-04-25T10:31:50.178994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655649,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":526,"num_read_kb":360,"num_write":281,"num_write_kb":2260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":3012908,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":419580,"kb_used_data":12108,"kb_used_omap":18,"kb_used_meta":407277,"kb_avail":1124522244,"statfs":{"total":1151940427776,"available":1151510777856,"internally_reserved":0,"allocated":12398592,"data_stored":6921471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":417051961},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003422"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.938407+0000","last_change":"2024-04-25T10:30:33.645032+0000","last_active":"2024-04-25T10:31:33.938407+0000","last_peered":"2024-04-25T10:31:33.938407+0000","last_clean":"2024-04-25T10:31:33.938407+0000","last_became_active":"2024-04-25T10:30:33.644687+0000","last_became_peered":"2024-04-25T10:30:33.644687+0000","last_unstale":"2024-04-25T10:31:33.938407+0000","last_undegraded":"2024-04-25T10:31:33.938407+0000","last_fullsized":"2024-04-25T10:31:33.938407+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:39.893228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.939115+0000","last_change":"2024-04-25T10:30:33.650466+0000","last_active":"2024-04-25T10:31:33.939115+0000","last_peered":"2024-04-25T10:31:33.939115+0000","last_clean":"2024-04-25T10:31:33.939115+0000","last_became_active":"2024-04-25T10:30:33.649889+0000","last_became_peered":"2024-04-25T10:30:33.649889+0000","last_unstale":"2024-04-25T10:31:33.939115+0000","last_undegraded":"2024-04-25T10:31:33.939115+0000","last_fullsized":"2024-04-25T10:31:33.939115+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:53.832494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.939051+0000","last_change":"2024-04-25T10:30:33.648798+0000","last_active":"2024-04-25T10:31:33.939051+0000","last_peered":"2024-04-25T10:31:33.939051+0000","last_clean":"2024-04-25T10:31:33.939051+0000","last_became_active":"2024-04-25T10:30:33.648423+0000","last_became_peered":"2024-04-25T10:30:33.648423+0000","last_unstale":"2024-04-25T10:31:33.939051+0000","last_undegraded":"2024-04-25T10:31:33.939051+0000","last_fullsized":"2024-04-25T10:31:33.939051+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:19.119189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.942219+0000","last_change":"2024-04-25T10:31:33.941691+0000","last_active":"2024-04-25T10:31:33.942219+0000","last_peered":"2024-04-25T10:31:33.942219+0000","last_clean":"2024-04-25T10:31:33.942219+0000","last_became_active":"2024-04-25T10:31:33.941304+0000","last_became_peered":"2024-04-25T10:31:33.941304+0000","last_unstale":"2024-04-25T10:31:33.942219+0000","last_undegraded":"2024-04-25T10:31:33.942219+0000","last_fullsized":"2024-04-25T10:31:33.942219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:18.573452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.936121+0000","last_change":"2024-04-25T10:30:33.646786+0000","last_active":"2024-04-25T10:31:33.936121+0000","last_peered":"2024-04-25T10:31:33.936121+0000","last_clean":"2024-04-25T10:31:33.936121+0000","last_became_active":"2024-04-25T10:30:33.646456+0000","last_became_peered":"2024-04-25T10:30:33.646456+0000","last_unstale":"2024-04-25T10:31:33.936121+0000","last_undegraded":"2024-04-25T10:31:33.936121+0000","last_fullsized":"2024-04-25T10:31:33.936121+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:55.006723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.938868+0000","last_change":"2024-04-25T10:30:33.639952+0000","last_active":"2024-04-25T10:31:33.938868+0000","last_peered":"2024-04-25T10:31:33.938868+0000","last_clean":"2024-04-25T10:31:33.938868+0000","last_became_active":"2024-04-25T10:30:33.639558+0000","last_became_peered":"2024-04-25T10:30:33.639558+0000","last_unstale":"2024-04-25T10:31:33.938868+0000","last_undegraded":"2024-04-25T10:31:33.938868+0000","last_fullsized":"2024-04-25T10:31:33.938868+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:17.162902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940596+0000","last_change":"2024-04-25T10:30:33.651530+0000","last_active":"2024-04-25T10:31:33.940596+0000","last_peered":"2024-04-25T10:31:33.940596+0000","last_clean":"2024-04-25T10:31:33.940596+0000","last_became_active":"2024-04-25T10:30:33.651151+0000","last_became_peered":"2024-04-25T10:30:33.651151+0000","last_unstale":"2024-04-25T10:31:33.940596+0000","last_undegraded":"2024-04-25T10:31:33.940596+0000","last_fullsized":"2024-04-25T10:31:33.940596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:31.835367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940102+0000","last_change":"2024-04-25T10:30:33.645146+0000","last_active":"2024-04-25T10:31:33.940102+0000","last_peered":"2024-04-25T10:31:33.940102+0000","last_clean":"2024-04-25T10:31:33.940102+0000","last_became_active":"2024-04-25T10:30:33.644310+0000","last_became_peered":"2024-04-25T10:30:33.644310+0000","last_unstale":"2024-04-25T10:31:33.940102+0000","last_undegraded":"2024-04-25T10:31:33.940102+0000","last_fullsized":"2024-04-25T10:31:33.940102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:45.034934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937726+0000","last_change":"2024-04-25T10:30:33.649385+0000","last_active":"2024-04-25T10:31:33.937726+0000","last_peered":"2024-04-25T10:31:33.937726+0000","last_clean":"2024-04-25T10:31:33.937726+0000","last_became_active":"2024-04-25T10:30:33.649002+0000","last_became_peered":"2024-04-25T10:30:33.649002+0000","last_unstale":"2024-04-25T10:31:33.937726+0000","last_undegraded":"2024-04-25T10:31:33.937726+0000","last_fullsized":"2024-04-25T10:31:33.937726+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:42.259290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.939076+0000","last_change":"2024-04-25T10:30:33.641918+0000","last_active":"2024-04-25T10:31:33.939076+0000","last_peered":"2024-04-25T10:31:33.939076+0000","last_clean":"2024-04-25T10:31:33.939076+0000","last_became_active":"2024-04-25T10:30:33.641553+0000","last_became_peered":"2024-04-25T10:30:33.641553+0000","last_unstale":"2024-04-25T10:31:33.939076+0000","last_undegraded":"2024-04-25T10:31:33.939076+0000","last_fullsized":"2024-04-25T10:31:33.939076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:04.755694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941019+0000","last_change":"2024-04-25T10:30:33.638054+0000","last_active":"2024-04-25T10:31:33.941019+0000","last_peered":"2024-04-25T10:31:33.941019+0000","last_clean":"2024-04-25T10:31:33.941019+0000","last_became_active":"2024-04-25T10:30:33.637720+0000","last_became_peered":"2024-04-25T10:30:33.637720+0000","last_unstale":"2024-04-25T10:31:33.941019+0000","last_undegraded":"2024-04-25T10:31:33.941019+0000","last_fullsized":"2024-04-25T10:31:33.941019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:41.557430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941989+0000","last_change":"2024-04-25T10:30:33.642705+0000","last_active":"2024-04-25T10:31:33.941989+0000","last_peered":"2024-04-25T10:31:33.941989+0000","last_clean":"2024-04-25T10:31:33.941989+0000","last_became_active":"2024-04-25T10:30:33.641936+0000","last_became_peered":"2024-04-25T10:30:33.641936+0000","last_unstale":"2024-04-25T10:31:33.941989+0000","last_undegraded":"2024-04-25T10:31:33.941989+0000","last_fullsized":"2024-04-25T10:31:33.941989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:36.450389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.935792+0000","last_change":"2024-04-25T10:30:33.648987+0000","last_active":"2024-04-25T10:31:33.935792+0000","last_peered":"2024-04-25T10:31:33.935792+0000","last_clean":"2024-04-25T10:31:33.935792+0000","last_became_active":"2024-04-25T10:30:33.648621+0000","last_became_peered":"2024-04-25T10:30:33.648621+0000","last_unstale":"2024-04-25T10:31:33.935792+0000","last_undegraded":"2024-04-25T10:31:33.935792+0000","last_fullsized":"2024-04-25T10:31:33.935792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:19.999002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.938628+0000","last_change":"2024-04-25T10:30:33.647360+0000","last_active":"2024-04-25T10:31:33.938628+0000","last_peered":"2024-04-25T10:31:33.938628+0000","last_clean":"2024-04-25T10:31:33.938628+0000","last_became_active":"2024-04-25T10:30:33.646719+0000","last_became_peered":"2024-04-25T10:30:33.646719+0000","last_unstale":"2024-04-25T10:31:33.938628+0000","last_undegraded":"2024-04-25T10:31:33.938628+0000","last_fullsized":"2024-04-25T10:31:33.938628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:19.016333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.938530+0000","last_change":"2024-04-25T10:30:33.646503+0000","last_active":"2024-04-25T10:31:33.938530+0000","last_peered":"2024-04-25T10:31:33.938530+0000","last_clean":"2024-04-25T10:31:33.938530+0000","last_became_active":"2024-04-25T10:30:33.646123+0000","last_became_peered":"2024-04-25T10:30:33.646123+0000","last_unstale":"2024-04-25T10:31:33.938530+0000","last_undegraded":"2024-04-25T10:31:33.938530+0000","last_fullsized":"2024-04-25T10:31:33.938530+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:57.712216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.936931+0000","last_change":"2024-04-25T10:30:33.640463+0000","last_active":"2024-04-25T10:31:33.936931+0000","last_peered":"2024-04-25T10:31:33.936931+0000","last_clean":"2024-04-25T10:31:33.936931+0000","last_became_active":"2024-04-25T10:30:33.639931+0000","last_became_peered":"2024-04-25T10:30:33.639931+0000","last_unstale":"2024-04-25T10:31:33.936931+0000","last_undegraded":"2024-04-25T10:31:33.936931+0000","last_fullsized":"2024-04-25T10:31:33.936931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:18.863405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.938295+0000","last_change":"2024-04-25T10:30:33.645695+0000","last_active":"2024-04-25T10:31:33.938295+0000","last_peered":"2024-04-25T10:31:33.938295+0000","last_clean":"2024-04-25T10:31:33.938295+0000","last_became_active":"2024-04-25T10:30:33.645387+0000","last_became_peered":"2024-04-25T10:30:33.645387+0000","last_unstale":"2024-04-25T10:31:33.938295+0000","last_undegraded":"2024-04-25T10:31:33.938295+0000","last_fullsized":"2024-04-25T10:31:33.938295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:20.044456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937794+0000","last_change":"2024-04-25T10:30:33.639087+0000","last_active":"2024-04-25T10:31:33.937794+0000","last_peered":"2024-04-25T10:31:33.937794+0000","last_clean":"2024-04-25T10:31:33.937794+0000","last_became_active":"2024-04-25T10:30:33.638726+0000","last_became_peered":"2024-04-25T10:30:33.638726+0000","last_unstale":"2024-04-25T10:31:33.937794+0000","last_undegraded":"2024-04-25T10:31:33.937794+0000","last_fullsized":"2024-04-25T10:31:33.937794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:36.887289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.943067+0000","last_change":"2024-04-25T10:30:33.638646+0000","last_active":"2024-04-25T10:31:33.943067+0000","last_peered":"2024-04-25T10:31:33.943067+0000","last_clean":"2024-04-25T10:31:33.943067+0000","last_became_active":"2024-04-25T10:30:33.638261+0000","last_became_peered":"2024-04-25T10:30:33.638261+0000","last_unstale":"2024-04-25T10:31:33.943067+0000","last_undegraded":"2024-04-25T10:31:33.943067+0000","last_fullsized":"2024-04-25T10:31:33.943067+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:08.212310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937730+0000","last_change":"2024-04-25T10:30:32.899992+0000","last_active":"2024-04-25T10:31:33.937730+0000","last_peered":"2024-04-25T10:31:33.937730+0000","last_clean":"2024-04-25T10:31:33.937730+0000","last_became_active":"2024-04-25T10:30:32.899643+0000","last_became_peered":"2024-04-25T10:30:32.899643+0000","last_unstale":"2024-04-25T10:31:33.937730+0000","last_undegraded":"2024-04-25T10:31:33.937730+0000","last_fullsized":"2024-04-25T10:31:33.937730+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:34.245121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940447+0000","last_change":"2024-04-25T10:30:33.644884+0000","last_active":"2024-04-25T10:31:33.940447+0000","last_peered":"2024-04-25T10:31:33.940447+0000","last_clean":"2024-04-25T10:31:33.940447+0000","last_became_active":"2024-04-25T10:30:33.644510+0000","last_became_peered":"2024-04-25T10:30:33.644510+0000","last_unstale":"2024-04-25T10:31:33.940447+0000","last_undegraded":"2024-04-25T10:31:33.940447+0000","last_fullsized":"2024-04-25T10:31:33.940447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:09.282865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940526+0000","last_change":"2024-04-25T10:30:32.909175+0000","last_active":"2024-04-25T10:31:33.940526+0000","last_peered":"2024-04-25T10:31:33.940526+0000","last_clean":"2024-04-25T10:31:33.940526+0000","last_became_active":"2024-04-25T10:30:32.908813+0000","last_became_peered":"2024-04-25T10:30:32.908813+0000","last_unstale":"2024-04-25T10:31:33.940526+0000","last_undegraded":"2024-04-25T10:31:33.940526+0000","last_fullsized":"2024-04-25T10:31:33.940526+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:04.256452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.939102+0000","last_change":"2024-04-25T10:30:33.642095+0000","last_active":"2024-04-25T10:31:33.939102+0000","last_peered":"2024-04-25T10:31:33.939102+0000","last_clean":"2024-04-25T10:31:33.939102+0000","last_became_active":"2024-04-25T10:30:33.641713+0000","last_became_peered":"2024-04-25T10:30:33.641713+0000","last_unstale":"2024-04-25T10:31:33.939102+0000","last_undegraded":"2024-04-25T10:31:33.939102+0000","last_fullsized":"2024-04-25T10:31:33.939102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:42.425197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.936797+0000","last_change":"2024-04-25T10:30:32.904376+0000","last_active":"2024-04-25T10:31:33.936797+0000","last_peered":"2024-04-25T10:31:33.936797+0000","last_clean":"2024-04-25T10:31:33.936797+0000","last_became_active":"2024-04-25T10:30:32.903994+0000","last_became_peered":"2024-04-25T10:30:32.903994+0000","last_unstale":"2024-04-25T10:31:33.936797+0000","last_undegraded":"2024-04-25T10:31:33.936797+0000","last_fullsized":"2024-04-25T10:31:33.936797+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:52.727800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937192+0000","last_change":"2024-04-25T10:30:33.637171+0000","last_active":"2024-04-25T10:31:33.937192+0000","last_peered":"2024-04-25T10:31:33.937192+0000","last_clean":"2024-04-25T10:31:33.937192+0000","last_became_active":"2024-04-25T10:30:33.636822+0000","last_became_peered":"2024-04-25T10:30:33.636822+0000","last_unstale":"2024-04-25T10:31:33.937192+0000","last_undegraded":"2024-04-25T10:31:33.937192+0000","last_fullsized":"2024-04-25T10:31:33.937192+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:54.220784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.936429+0000","last_change":"2024-04-25T10:30:32.900856+0000","last_active":"2024-04-25T10:31:33.936429+0000","last_peered":"2024-04-25T10:31:33.936429+0000","last_clean":"2024-04-25T10:31:33.936429+0000","last_became_active":"2024-04-25T10:30:32.900519+0000","last_became_peered":"2024-04-25T10:30:32.900519+0000","last_unstale":"2024-04-25T10:31:33.936429+0000","last_undegraded":"2024-04-25T10:31:33.936429+0000","last_fullsized":"2024-04-25T10:31:33.936429+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:49.738005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941925+0000","last_change":"2024-04-25T10:30:33.651736+0000","last_active":"2024-04-25T10:31:33.941925+0000","last_peered":"2024-04-25T10:31:33.941925+0000","last_clean":"2024-04-25T10:31:33.941925+0000","last_became_active":"2024-04-25T10:30:33.651395+0000","last_became_peered":"2024-04-25T10:30:33.651395+0000","last_unstale":"2024-04-25T10:31:33.941925+0000","last_undegraded":"2024-04-25T10:31:33.941925+0000","last_fullsized":"2024-04-25T10:31:33.941925+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:58.230092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"77'5","reported_seq":97,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:46.906644+0000","last_change":"2024-04-25T10:30:32.901348+0000","last_active":"2024-04-25T10:31:46.906644+0000","last_peered":"2024-04-25T10:31:46.906644+0000","last_clean":"2024-04-25T10:31:46.906644+0000","last_became_active":"2024-04-25T10:30:32.901025+0000","last_became_peered":"2024-04-25T10:30:32.901025+0000","last_unstale":"2024-04-25T10:31:46.906644+0000","last_undegraded":"2024-04-25T10:31:46.906644+0000","last_fullsized":"2024-04-25T10:31:46.906644+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:00.521934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.936115+0000","last_change":"2024-04-25T10:30:33.649206+0000","last_active":"2024-04-25T10:31:33.936115+0000","last_peered":"2024-04-25T10:31:33.936115+0000","last_clean":"2024-04-25T10:31:33.936115+0000","last_became_active":"2024-04-25T10:30:33.648883+0000","last_became_peered":"2024-04-25T10:30:33.648883+0000","last_unstale":"2024-04-25T10:31:33.936115+0000","last_undegraded":"2024-04-25T10:31:33.936115+0000","last_fullsized":"2024-04-25T10:31:33.936115+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:43.105562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.938755+0000","last_change":"2024-04-25T10:30:32.901201+0000","last_active":"2024-04-25T10:31:33.938755+0000","last_peered":"2024-04-25T10:31:33.938755+0000","last_clean":"2024-04-25T10:31:33.938755+0000","last_became_active":"2024-04-25T10:30:32.900876+0000","last_became_peered":"2024-04-25T10:30:32.900876+0000","last_unstale":"2024-04-25T10:31:33.938755+0000","last_undegraded":"2024-04-25T10:31:33.938755+0000","last_fullsized":"2024-04-25T10:31:33.938755+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:58.077045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941479+0000","last_change":"2024-04-25T10:30:33.651935+0000","last_active":"2024-04-25T10:31:33.941479+0000","last_peered":"2024-04-25T10:31:33.941479+0000","last_clean":"2024-04-25T10:31:33.941479+0000","last_became_active":"2024-04-25T10:30:33.651568+0000","last_became_peered":"2024-04-25T10:30:33.651568+0000","last_unstale":"2024-04-25T10:31:33.941479+0000","last_undegraded":"2024-04-25T10:31:33.941479+0000","last_fullsized":"2024-04-25T10:31:33.941479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:58.849535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.935662+0000","last_change":"2024-04-25T10:30:32.896500+0000","last_active":"2024-04-25T10:31:33.935662+0000","last_peered":"2024-04-25T10:31:33.935662+0000","last_clean":"2024-04-25T10:31:33.935662+0000","last_became_active":"2024-04-25T10:30:32.896071+0000","last_became_peered":"2024-04-25T10:30:32.896071+0000","last_unstale":"2024-04-25T10:31:33.935662+0000","last_undegraded":"2024-04-25T10:31:33.935662+0000","last_fullsized":"2024-04-25T10:31:33.935662+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:10.385443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940171+0000","last_change":"2024-04-25T10:30:33.631730+0000","last_active":"2024-04-25T10:31:33.940171+0000","last_peered":"2024-04-25T10:31:33.940171+0000","last_clean":"2024-04-25T10:31:33.940171+0000","last_became_active":"2024-04-25T10:30:33.631374+0000","last_became_peered":"2024-04-25T10:30:33.631374+0000","last_unstale":"2024-04-25T10:31:33.940171+0000","last_undegraded":"2024-04-25T10:31:33.940171+0000","last_fullsized":"2024-04-25T10:31:33.940171+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:14.957866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941082+0000","last_change":"2024-04-25T10:30:32.900762+0000","last_active":"2024-04-25T10:31:33.941082+0000","last_peered":"2024-04-25T10:31:33.941082+0000","last_clean":"2024-04-25T10:31:33.941082+0000","last_became_active":"2024-04-25T10:30:32.900459+0000","last_became_peered":"2024-04-25T10:30:32.900459+0000","last_unstale":"2024-04-25T10:31:33.941082+0000","last_undegraded":"2024-04-25T10:31:33.941082+0000","last_fullsized":"2024-04-25T10:31:33.941082+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:22.286261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937757+0000","last_change":"2024-04-25T10:30:33.637727+0000","last_active":"2024-04-25T10:31:33.937757+0000","last_peered":"2024-04-25T10:31:33.937757+0000","last_clean":"2024-04-25T10:31:33.937757+0000","last_became_active":"2024-04-25T10:30:33.637359+0000","last_became_peered":"2024-04-25T10:30:33.637359+0000","last_unstale":"2024-04-25T10:31:33.937757+0000","last_undegraded":"2024-04-25T10:31:33.937757+0000","last_fullsized":"2024-04-25T10:31:33.937757+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:21.877562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'3","reported_seq":94,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:47.907519+0000","last_change":"2024-04-25T10:30:32.894210+0000","last_active":"2024-04-25T10:31:47.907519+0000","last_peered":"2024-04-25T10:31:47.907519+0000","last_clean":"2024-04-25T10:31:47.907519+0000","last_became_active":"2024-04-25T10:30:32.893241+0000","last_became_peered":"2024-04-25T10:30:32.893241+0000","last_unstale":"2024-04-25T10:31:47.907519+0000","last_undegraded":"2024-04-25T10:31:47.907519+0000","last_fullsized":"2024-04-25T10:31:47.907519+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:06.340392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937853+0000","last_change":"2024-04-25T10:30:33.632349+0000","last_active":"2024-04-25T10:31:33.937853+0000","last_peered":"2024-04-25T10:31:33.937853+0000","last_clean":"2024-04-25T10:31:33.937853+0000","last_became_active":"2024-04-25T10:30:33.631968+0000","last_became_peered":"2024-04-25T10:30:33.631968+0000","last_unstale":"2024-04-25T10:31:33.937853+0000","last_undegraded":"2024-04-25T10:31:33.937853+0000","last_fullsized":"2024-04-25T10:31:33.937853+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:29.090932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940786+0000","last_change":"2024-04-25T10:30:32.900105+0000","last_active":"2024-04-25T10:31:33.940786+0000","last_peered":"2024-04-25T10:31:33.940786+0000","last_clean":"2024-04-25T10:31:33.940786+0000","last_became_active":"2024-04-25T10:30:32.899740+0000","last_became_peered":"2024-04-25T10:30:32.899740+0000","last_unstale":"2024-04-25T10:31:33.940786+0000","last_undegraded":"2024-04-25T10:31:33.940786+0000","last_fullsized":"2024-04-25T10:31:33.940786+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:49.390799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.943769+0000","last_change":"2024-04-25T10:31:33.943769+0000","last_active":"2024-04-25T10:31:33.943769+0000","last_peered":"2024-04-25T10:31:33.943769+0000","last_clean":"2024-04-25T10:31:33.943769+0000","last_became_active":"2024-04-25T10:31:33.943331+0000","last_became_peered":"2024-04-25T10:31:33.943331+0000","last_unstale":"2024-04-25T10:31:33.943769+0000","last_undegraded":"2024-04-25T10:31:33.943769+0000","last_fullsized":"2024-04-25T10:31:33.943769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:07.183059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.942017+0000","last_change":"2024-04-25T10:30:32.893453+0000","last_active":"2024-04-25T10:31:33.942017+0000","last_peered":"2024-04-25T10:31:33.942017+0000","last_clean":"2024-04-25T10:31:33.942017+0000","last_became_active":"2024-04-25T10:30:32.893106+0000","last_became_peered":"2024-04-25T10:30:32.893106+0000","last_unstale":"2024-04-25T10:31:33.942017+0000","last_undegraded":"2024-04-25T10:31:33.942017+0000","last_fullsized":"2024-04-25T10:31:33.942017+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:57.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937345+0000","last_change":"2024-04-25T10:30:33.645693+0000","last_active":"2024-04-25T10:31:33.937345+0000","last_peered":"2024-04-25T10:31:33.937345+0000","last_clean":"2024-04-25T10:31:33.937345+0000","last_became_active":"2024-04-25T10:30:33.645342+0000","last_became_peered":"2024-04-25T10:30:33.645342+0000","last_unstale":"2024-04-25T10:31:33.937345+0000","last_undegraded":"2024-04-25T10:31:33.937345+0000","last_fullsized":"2024-04-25T10:31:33.937345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:15.729860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'28","reported_seq":217,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:48.964705+0000","last_change":"2024-04-25T10:31:33.949796+0000","last_active":"2024-04-25T10:31:48.964705+0000","last_peered":"2024-04-25T10:31:48.964705+0000","last_clean":"2024-04-25T10:31:48.964705+0000","last_became_active":"2024-04-25T10:31:33.942989+0000","last_became_peered":"2024-04-25T10:31:33.942989+0000","last_unstale":"2024-04-25T10:31:48.964705+0000","last_undegraded":"2024-04-25T10:31:48.964705+0000","last_fullsized":"2024-04-25T10:31:48.964705+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:04.194643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":144,"num_read_kb":131,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61046,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.936604+0000","last_change":"2024-04-25T10:30:33.641768+0000","last_active":"2024-04-25T10:31:33.936604+0000","last_peered":"2024-04-25T10:31:33.936604+0000","last_clean":"2024-04-25T10:31:33.936604+0000","last_became_active":"2024-04-25T10:30:33.641415+0000","last_became_peered":"2024-04-25T10:30:33.641415+0000","last_unstale":"2024-04-25T10:31:33.936604+0000","last_undegraded":"2024-04-25T10:31:33.936604+0000","last_fullsized":"2024-04-25T10:31:33.936604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:38.569474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940233+0000","last_change":"2024-04-25T10:30:32.898703+0000","last_active":"2024-04-25T10:31:33.940233+0000","last_peered":"2024-04-25T10:31:33.940233+0000","last_clean":"2024-04-25T10:31:33.940233+0000","last_became_active":"2024-04-25T10:30:32.898340+0000","last_became_peered":"2024-04-25T10:30:32.898340+0000","last_unstale":"2024-04-25T10:31:33.940233+0000","last_undegraded":"2024-04-25T10:31:33.940233+0000","last_fullsized":"2024-04-25T10:31:33.940233+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:27.024144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.935624+0000","last_change":"2024-04-25T10:30:33.640393+0000","last_active":"2024-04-25T10:31:33.935624+0000","last_peered":"2024-04-25T10:31:33.935624+0000","last_clean":"2024-04-25T10:31:33.935624+0000","last_became_active":"2024-04-25T10:30:33.639822+0000","last_became_peered":"2024-04-25T10:30:33.639822+0000","last_unstale":"2024-04-25T10:31:33.935624+0000","last_undegraded":"2024-04-25T10:31:33.935624+0000","last_fullsized":"2024-04-25T10:31:33.935624+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:08.618406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941265+0000","last_change":"2024-04-25T10:30:32.900880+0000","last_active":"2024-04-25T10:31:33.941265+0000","last_peered":"2024-04-25T10:31:33.941265+0000","last_clean":"2024-04-25T10:31:33.941265+0000","last_became_active":"2024-04-25T10:30:32.899987+0000","last_became_peered":"2024-04-25T10:30:32.899987+0000","last_unstale":"2024-04-25T10:31:33.941265+0000","last_undegraded":"2024-04-25T10:31:33.941265+0000","last_fullsized":"2024-04-25T10:31:33.941265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:35.687773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941600+0000","last_change":"2024-04-25T10:30:33.641933+0000","last_active":"2024-04-25T10:31:33.941600+0000","last_peered":"2024-04-25T10:31:33.941600+0000","last_clean":"2024-04-25T10:31:33.941600+0000","last_became_active":"2024-04-25T10:30:33.641549+0000","last_became_peered":"2024-04-25T10:30:33.641549+0000","last_unstale":"2024-04-25T10:31:33.941600+0000","last_undegraded":"2024-04-25T10:31:33.941600+0000","last_fullsized":"2024-04-25T10:31:33.941600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:36.696523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'4","reported_seq":96,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:46.916556+0000","last_change":"2024-04-25T10:30:32.901178+0000","last_active":"2024-04-25T10:31:46.916556+0000","last_peered":"2024-04-25T10:31:46.916556+0000","last_clean":"2024-04-25T10:31:46.916556+0000","last_became_active":"2024-04-25T10:30:32.900817+0000","last_became_peered":"2024-04-25T10:30:32.900817+0000","last_unstale":"2024-04-25T10:31:46.916556+0000","last_undegraded":"2024-04-25T10:31:46.916556+0000","last_fullsized":"2024-04-25T10:31:46.916556+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:53.999253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.943470+0000","last_change":"2024-04-25T10:31:33.943470+0000","last_active":"2024-04-25T10:31:33.943470+0000","last_peered":"2024-04-25T10:31:33.943470+0000","last_clean":"2024-04-25T10:31:33.943470+0000","last_became_active":"2024-04-25T10:31:33.943098+0000","last_became_peered":"2024-04-25T10:31:33.943098+0000","last_unstale":"2024-04-25T10:31:33.943470+0000","last_undegraded":"2024-04-25T10:31:33.943470+0000","last_fullsized":"2024-04-25T10:31:33.943470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:27.707195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.935960+0000","last_change":"2024-04-25T10:30:32.902862+0000","last_active":"2024-04-25T10:31:33.935960+0000","last_peered":"2024-04-25T10:31:33.935960+0000","last_clean":"2024-04-25T10:31:33.935960+0000","last_became_active":"2024-04-25T10:30:32.902537+0000","last_became_peered":"2024-04-25T10:30:32.902537+0000","last_unstale":"2024-04-25T10:31:33.935960+0000","last_undegraded":"2024-04-25T10:31:33.935960+0000","last_fullsized":"2024-04-25T10:31:33.935960+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:55.974128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941592+0000","last_change":"2024-04-25T10:30:33.651409+0000","last_active":"2024-04-25T10:31:33.941592+0000","last_peered":"2024-04-25T10:31:33.941592+0000","last_clean":"2024-04-25T10:31:33.941592+0000","last_became_active":"2024-04-25T10:30:33.650673+0000","last_became_peered":"2024-04-25T10:30:33.650673+0000","last_unstale":"2024-04-25T10:31:33.941592+0000","last_undegraded":"2024-04-25T10:31:33.941592+0000","last_fullsized":"2024-04-25T10:31:33.941592+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:48.989198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.953944+0000","last_change":"2024-04-25T10:31:33.953408+0000","last_active":"2024-04-25T10:31:33.953944+0000","last_peered":"2024-04-25T10:31:33.953944+0000","last_clean":"2024-04-25T10:31:33.953944+0000","last_became_active":"2024-04-25T10:31:33.943495+0000","last_became_peered":"2024-04-25T10:31:33.943495+0000","last_unstale":"2024-04-25T10:31:33.953944+0000","last_undegraded":"2024-04-25T10:31:33.953944+0000","last_fullsized":"2024-04-25T10:31:33.953944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:03.947967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.938718+0000","last_change":"2024-04-25T10:30:33.643929+0000","last_active":"2024-04-25T10:31:33.938718+0000","last_peered":"2024-04-25T10:31:33.938718+0000","last_clean":"2024-04-25T10:31:33.938718+0000","last_became_active":"2024-04-25T10:30:33.643593+0000","last_became_peered":"2024-04-25T10:30:33.643593+0000","last_unstale":"2024-04-25T10:31:33.938718+0000","last_undegraded":"2024-04-25T10:31:33.938718+0000","last_fullsized":"2024-04-25T10:31:33.938718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:39.591022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937940+0000","last_change":"2024-04-25T10:30:32.898937+0000","last_active":"2024-04-25T10:31:33.937940+0000","last_peered":"2024-04-25T10:31:33.937940+0000","last_clean":"2024-04-25T10:31:33.937940+0000","last_became_active":"2024-04-25T10:30:32.898570+0000","last_became_peered":"2024-04-25T10:30:32.898570+0000","last_unstale":"2024-04-25T10:31:33.937940+0000","last_undegraded":"2024-04-25T10:31:33.937940+0000","last_fullsized":"2024-04-25T10:31:33.937940+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:40.123035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'1","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.939346+0000","last_change":"2024-04-25T10:30:33.632857+0000","last_active":"2024-04-25T10:31:33.939346+0000","last_peered":"2024-04-25T10:31:33.939346+0000","last_clean":"2024-04-25T10:31:33.939346+0000","last_became_active":"2024-04-25T10:30:33.632043+0000","last_became_peered":"2024-04-25T10:30:33.632043+0000","last_unstale":"2024-04-25T10:31:33.939346+0000","last_undegraded":"2024-04-25T10:31:33.939346+0000","last_fullsized":"2024-04-25T10:31:33.939346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:12.683035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940901+0000","last_change":"2024-04-25T10:30:32.901349+0000","last_active":"2024-04-25T10:31:33.940901+0000","last_peered":"2024-04-25T10:31:33.940901+0000","last_clean":"2024-04-25T10:31:33.940901+0000","last_became_active":"2024-04-25T10:30:32.900881+0000","last_became_peered":"2024-04-25T10:30:32.900881+0000","last_unstale":"2024-04-25T10:31:33.940901+0000","last_undegraded":"2024-04-25T10:31:33.940901+0000","last_fullsized":"2024-04-25T10:31:33.940901+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:34.963564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.939214+0000","last_change":"2024-04-25T10:30:33.644376+0000","last_active":"2024-04-25T10:31:33.939214+0000","last_peered":"2024-04-25T10:31:33.939214+0000","last_clean":"2024-04-25T10:31:33.939214+0000","last_became_active":"2024-04-25T10:30:33.644055+0000","last_became_peered":"2024-04-25T10:30:33.644055+0000","last_unstale":"2024-04-25T10:31:33.939214+0000","last_undegraded":"2024-04-25T10:31:33.939214+0000","last_fullsized":"2024-04-25T10:31:33.939214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:48.134775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941290+0000","last_change":"2024-04-25T10:30:33.646312+0000","last_active":"2024-04-25T10:31:33.941290+0000","last_peered":"2024-04-25T10:31:33.941290+0000","last_clean":"2024-04-25T10:31:33.941290+0000","last_became_active":"2024-04-25T10:30:33.642338+0000","last_became_peered":"2024-04-25T10:30:33.642338+0000","last_unstale":"2024-04-25T10:31:33.941290+0000","last_undegraded":"2024-04-25T10:31:33.941290+0000","last_fullsized":"2024-04-25T10:31:33.941290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:42.588360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"77'4","reported_seq":93,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:46.915825+0000","last_change":"2024-04-25T10:30:32.909680+0000","last_active":"2024-04-25T10:31:46.915825+0000","last_peered":"2024-04-25T10:31:46.915825+0000","last_clean":"2024-04-25T10:31:46.915825+0000","last_became_active":"2024-04-25T10:30:32.909169+0000","last_became_peered":"2024-04-25T10:30:32.909169+0000","last_unstale":"2024-04-25T10:31:46.915825+0000","last_undegraded":"2024-04-25T10:31:46.915825+0000","last_fullsized":"2024-04-25T10:31:46.915825+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:54.876475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940403+0000","last_change":"2024-04-25T10:30:33.649524+0000","last_active":"2024-04-25T10:31:33.940403+0000","last_peered":"2024-04-25T10:31:33.940403+0000","last_clean":"2024-04-25T10:31:33.940403+0000","last_became_active":"2024-04-25T10:30:33.648958+0000","last_became_peered":"2024-04-25T10:30:33.648958+0000","last_unstale":"2024-04-25T10:31:33.940403+0000","last_undegraded":"2024-04-25T10:31:33.940403+0000","last_fullsized":"2024-04-25T10:31:33.940403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:13.407742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941491+0000","last_change":"2024-04-25T10:30:32.895270+0000","last_active":"2024-04-25T10:31:33.941491+0000","last_peered":"2024-04-25T10:31:33.941491+0000","last_clean":"2024-04-25T10:31:33.941491+0000","last_became_active":"2024-04-25T10:30:32.894899+0000","last_became_peered":"2024-04-25T10:30:32.894899+0000","last_unstale":"2024-04-25T10:31:33.941491+0000","last_undegraded":"2024-04-25T10:31:33.941491+0000","last_fullsized":"2024-04-25T10:31:33.941491+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:21.233217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":95,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941397+0000","last_change":"2024-04-25T10:29:21.105073+0000","last_active":"2024-04-25T10:31:33.941397+0000","last_peered":"2024-04-25T10:31:33.941397+0000","last_clean":"2024-04-25T10:31:33.941397+0000","last_became_active":"2024-04-25T10:29:21.098771+0000","last_became_peered":"2024-04-25T10:29:21.098771+0000","last_unstale":"2024-04-25T10:31:33.941397+0000","last_undegraded":"2024-04-25T10:31:33.941397+0000","last_fullsized":"2024-04-25T10:31:33.941397+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:59.015440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:59.015440+0000","last_clean_scrub_stamp":"2024-04-25T10:25:59.015440+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:41.728977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.935487+0000","last_change":"2024-04-25T10:30:33.639159+0000","last_active":"2024-04-25T10:31:33.935487+0000","last_peered":"2024-04-25T10:31:33.935487+0000","last_clean":"2024-04-25T10:31:33.935487+0000","last_became_active":"2024-04-25T10:30:33.638761+0000","last_became_peered":"2024-04-25T10:30:33.638761+0000","last_unstale":"2024-04-25T10:31:33.935487+0000","last_undegraded":"2024-04-25T10:31:33.935487+0000","last_fullsized":"2024-04-25T10:31:33.935487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:43.262103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941588+0000","last_change":"2024-04-25T10:30:32.898587+0000","last_active":"2024-04-25T10:31:33.941588+0000","last_peered":"2024-04-25T10:31:33.941588+0000","last_clean":"2024-04-25T10:31:33.941588+0000","last_became_active":"2024-04-25T10:30:32.898113+0000","last_became_peered":"2024-04-25T10:30:32.898113+0000","last_unstale":"2024-04-25T10:31:33.941588+0000","last_undegraded":"2024-04-25T10:31:33.941588+0000","last_fullsized":"2024-04-25T10:31:33.941588+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:24.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.936615+0000","last_change":"2024-04-25T10:30:32.907188+0000","last_active":"2024-04-25T10:31:33.936615+0000","last_peered":"2024-04-25T10:31:33.936615+0000","last_clean":"2024-04-25T10:31:33.936615+0000","last_became_active":"2024-04-25T10:30:32.906837+0000","last_became_peered":"2024-04-25T10:30:32.906837+0000","last_unstale":"2024-04-25T10:31:33.936615+0000","last_undegraded":"2024-04-25T10:31:33.936615+0000","last_fullsized":"2024-04-25T10:31:33.936615+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:11.835053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.935910+0000","last_change":"2024-04-25T10:30:33.636159+0000","last_active":"2024-04-25T10:31:33.935910+0000","last_peered":"2024-04-25T10:31:33.935910+0000","last_clean":"2024-04-25T10:31:33.935910+0000","last_became_active":"2024-04-25T10:30:33.635098+0000","last_became_peered":"2024-04-25T10:30:33.635098+0000","last_unstale":"2024-04-25T10:31:33.935910+0000","last_undegraded":"2024-04-25T10:31:33.935910+0000","last_fullsized":"2024-04-25T10:31:33.935910+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:13.092368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'8","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937759+0000","last_change":"2024-04-25T10:30:32.905469+0000","last_active":"2024-04-25T10:31:33.937759+0000","last_peered":"2024-04-25T10:31:33.937759+0000","last_clean":"2024-04-25T10:31:33.937759+0000","last_became_active":"2024-04-25T10:30:32.905123+0000","last_became_peered":"2024-04-25T10:30:32.905123+0000","last_unstale":"2024-04-25T10:31:33.937759+0000","last_undegraded":"2024-04-25T10:31:33.937759+0000","last_fullsized":"2024-04-25T10:31:33.937759+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:12.884809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940634+0000","last_change":"2024-04-25T10:30:33.640690+0000","last_active":"2024-04-25T10:31:33.940634+0000","last_peered":"2024-04-25T10:31:33.940634+0000","last_clean":"2024-04-25T10:31:33.940634+0000","last_became_active":"2024-04-25T10:30:33.640310+0000","last_became_peered":"2024-04-25T10:30:33.640310+0000","last_unstale":"2024-04-25T10:31:33.940634+0000","last_undegraded":"2024-04-25T10:31:33.940634+0000","last_fullsized":"2024-04-25T10:31:33.940634+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:03.671495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941790+0000","last_change":"2024-04-25T10:30:32.893743+0000","last_active":"2024-04-25T10:31:33.941790+0000","last_peered":"2024-04-25T10:31:33.941790+0000","last_clean":"2024-04-25T10:31:33.941790+0000","last_became_active":"2024-04-25T10:30:32.893360+0000","last_became_peered":"2024-04-25T10:30:32.893360+0000","last_unstale":"2024-04-25T10:31:33.941790+0000","last_undegraded":"2024-04-25T10:31:33.941790+0000","last_fullsized":"2024-04-25T10:31:33.941790+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:39.830376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.939052+0000","last_change":"2024-04-25T10:30:33.645127+0000","last_active":"2024-04-25T10:31:33.939052+0000","last_peered":"2024-04-25T10:31:33.939052+0000","last_clean":"2024-04-25T10:31:33.939052+0000","last_became_active":"2024-04-25T10:30:33.644259+0000","last_became_peered":"2024-04-25T10:30:33.644259+0000","last_unstale":"2024-04-25T10:31:33.939052+0000","last_undegraded":"2024-04-25T10:31:33.939052+0000","last_fullsized":"2024-04-25T10:31:33.939052+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:56.849861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.936945+0000","last_change":"2024-04-25T10:30:32.896885+0000","last_active":"2024-04-25T10:31:33.936945+0000","last_peered":"2024-04-25T10:31:33.936945+0000","last_clean":"2024-04-25T10:31:33.936945+0000","last_became_active":"2024-04-25T10:30:32.896447+0000","last_became_peered":"2024-04-25T10:30:32.896447+0000","last_unstale":"2024-04-25T10:31:33.936945+0000","last_undegraded":"2024-04-25T10:31:33.936945+0000","last_fullsized":"2024-04-25T10:31:33.936945+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:13.133549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.935959+0000","last_change":"2024-04-25T10:30:33.645016+0000","last_active":"2024-04-25T10:31:33.935959+0000","last_peered":"2024-04-25T10:31:33.935959+0000","last_clean":"2024-04-25T10:31:33.935959+0000","last_became_active":"2024-04-25T10:30:33.644661+0000","last_became_peered":"2024-04-25T10:30:33.644661+0000","last_unstale":"2024-04-25T10:31:33.935959+0000","last_undegraded":"2024-04-25T10:31:33.935959+0000","last_fullsized":"2024-04-25T10:31:33.935959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:20.935124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940182+0000","last_change":"2024-04-25T10:30:32.907827+0000","last_active":"2024-04-25T10:31:33.940182+0000","last_peered":"2024-04-25T10:31:33.940182+0000","last_clean":"2024-04-25T10:31:33.940182+0000","last_became_active":"2024-04-25T10:30:32.907402+0000","last_became_peered":"2024-04-25T10:30:32.907402+0000","last_unstale":"2024-04-25T10:31:33.940182+0000","last_undegraded":"2024-04-25T10:31:33.940182+0000","last_fullsized":"2024-04-25T10:31:33.940182+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:38.501617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937725+0000","last_change":"2024-04-25T10:30:33.636935+0000","last_active":"2024-04-25T10:31:33.937725+0000","last_peered":"2024-04-25T10:31:33.937725+0000","last_clean":"2024-04-25T10:31:33.937725+0000","last_became_active":"2024-04-25T10:30:33.636566+0000","last_became_peered":"2024-04-25T10:30:33.636566+0000","last_unstale":"2024-04-25T10:31:33.937725+0000","last_undegraded":"2024-04-25T10:31:33.937725+0000","last_fullsized":"2024-04-25T10:31:33.937725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:24.556275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.947259+0000","last_change":"2024-04-25T10:31:33.946786+0000","last_active":"2024-04-25T10:31:33.947259+0000","last_peered":"2024-04-25T10:31:33.947259+0000","last_clean":"2024-04-25T10:31:33.947259+0000","last_became_active":"2024-04-25T10:31:33.941298+0000","last_became_peered":"2024-04-25T10:31:33.941298+0000","last_unstale":"2024-04-25T10:31:33.947259+0000","last_undegraded":"2024-04-25T10:31:33.947259+0000","last_fullsized":"2024-04-25T10:31:33.947259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:55.009673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940260+0000","last_change":"2024-04-25T10:30:33.642986+0000","last_active":"2024-04-25T10:31:33.940260+0000","last_peered":"2024-04-25T10:31:33.940260+0000","last_clean":"2024-04-25T10:31:33.940260+0000","last_became_active":"2024-04-25T10:30:33.642545+0000","last_became_peered":"2024-04-25T10:30:33.642545+0000","last_unstale":"2024-04-25T10:31:33.940260+0000","last_undegraded":"2024-04-25T10:31:33.940260+0000","last_fullsized":"2024-04-25T10:31:33.940260+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:49.369648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.936250+0000","last_change":"2024-04-25T10:30:32.902239+0000","last_active":"2024-04-25T10:31:33.936250+0000","last_peered":"2024-04-25T10:31:33.936250+0000","last_clean":"2024-04-25T10:31:33.936250+0000","last_became_active":"2024-04-25T10:30:32.901653+0000","last_became_peered":"2024-04-25T10:30:32.901653+0000","last_unstale":"2024-04-25T10:31:33.936250+0000","last_undegraded":"2024-04-25T10:31:33.936250+0000","last_fullsized":"2024-04-25T10:31:33.936250+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:34.732428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941029+0000","last_change":"2024-04-25T10:30:33.639763+0000","last_active":"2024-04-25T10:31:33.941029+0000","last_peered":"2024-04-25T10:31:33.941029+0000","last_clean":"2024-04-25T10:31:33.941029+0000","last_became_active":"2024-04-25T10:30:33.638004+0000","last_became_peered":"2024-04-25T10:30:33.638004+0000","last_unstale":"2024-04-25T10:31:33.941029+0000","last_undegraded":"2024-04-25T10:31:33.941029+0000","last_fullsized":"2024-04-25T10:31:33.941029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:09.722333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940617+0000","last_change":"2024-04-25T10:30:32.894823+0000","last_active":"2024-04-25T10:31:33.940617+0000","last_peered":"2024-04-25T10:31:33.940617+0000","last_clean":"2024-04-25T10:31:33.940617+0000","last_became_active":"2024-04-25T10:30:32.893819+0000","last_became_peered":"2024-04-25T10:30:32.893819+0000","last_unstale":"2024-04-25T10:31:33.940617+0000","last_undegraded":"2024-04-25T10:31:33.940617+0000","last_fullsized":"2024-04-25T10:31:33.940617+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:59.809410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941124+0000","last_change":"2024-04-25T10:30:33.637046+0000","last_active":"2024-04-25T10:31:33.941124+0000","last_peered":"2024-04-25T10:31:33.941124+0000","last_clean":"2024-04-25T10:31:33.941124+0000","last_became_active":"2024-04-25T10:30:33.634903+0000","last_became_peered":"2024-04-25T10:30:33.634903+0000","last_unstale":"2024-04-25T10:31:33.941124+0000","last_undegraded":"2024-04-25T10:31:33.941124+0000","last_fullsized":"2024-04-25T10:31:33.941124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:52.252812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941907+0000","last_change":"2024-04-25T10:30:32.898979+0000","last_active":"2024-04-25T10:31:33.941907+0000","last_peered":"2024-04-25T10:31:33.941907+0000","last_clean":"2024-04-25T10:31:33.941907+0000","last_became_active":"2024-04-25T10:30:32.898611+0000","last_became_peered":"2024-04-25T10:30:32.898611+0000","last_unstale":"2024-04-25T10:31:33.941907+0000","last_undegraded":"2024-04-25T10:31:33.941907+0000","last_fullsized":"2024-04-25T10:31:33.941907+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.549769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'1","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940827+0000","last_change":"2024-04-25T10:30:33.651596+0000","last_active":"2024-04-25T10:31:33.940827+0000","last_peered":"2024-04-25T10:31:33.940827+0000","last_clean":"2024-04-25T10:31:33.940827+0000","last_became_active":"2024-04-25T10:30:33.651140+0000","last_became_peered":"2024-04-25T10:30:33.651140+0000","last_unstale":"2024-04-25T10:31:33.940827+0000","last_undegraded":"2024-04-25T10:31:33.940827+0000","last_fullsized":"2024-04-25T10:31:33.940827+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:16.762135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.935870+0000","last_change":"2024-04-25T10:30:32.909982+0000","last_active":"2024-04-25T10:31:33.935870+0000","last_peered":"2024-04-25T10:31:33.935870+0000","last_clean":"2024-04-25T10:31:33.935870+0000","last_became_active":"2024-04-25T10:30:32.909506+0000","last_became_peered":"2024-04-25T10:30:32.909506+0000","last_unstale":"2024-04-25T10:31:33.935870+0000","last_undegraded":"2024-04-25T10:31:33.935870+0000","last_fullsized":"2024-04-25T10:31:33.935870+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:47.396963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941351+0000","last_change":"2024-04-25T10:30:33.642913+0000","last_active":"2024-04-25T10:31:33.941351+0000","last_peered":"2024-04-25T10:31:33.941351+0000","last_clean":"2024-04-25T10:31:33.941351+0000","last_became_active":"2024-04-25T10:30:33.642614+0000","last_became_peered":"2024-04-25T10:30:33.642614+0000","last_unstale":"2024-04-25T10:31:33.941351+0000","last_undegraded":"2024-04-25T10:31:33.941351+0000","last_fullsized":"2024-04-25T10:31:33.941351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:56.651230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937774+0000","last_change":"2024-04-25T10:30:33.644525+0000","last_active":"2024-04-25T10:31:33.937774+0000","last_peered":"2024-04-25T10:31:33.937774+0000","last_clean":"2024-04-25T10:31:33.937774+0000","last_became_active":"2024-04-25T10:30:33.644188+0000","last_became_peered":"2024-04-25T10:30:33.644188+0000","last_unstale":"2024-04-25T10:31:33.937774+0000","last_undegraded":"2024-04-25T10:31:33.937774+0000","last_fullsized":"2024-04-25T10:31:33.937774+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:28.625695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.946016+0000","last_change":"2024-04-25T10:31:33.946016+0000","last_active":"2024-04-25T10:31:33.946016+0000","last_peered":"2024-04-25T10:31:33.946016+0000","last_clean":"2024-04-25T10:31:33.946016+0000","last_became_active":"2024-04-25T10:31:33.945611+0000","last_became_peered":"2024-04-25T10:31:33.945611+0000","last_unstale":"2024-04-25T10:31:33.946016+0000","last_undegraded":"2024-04-25T10:31:33.946016+0000","last_fullsized":"2024-04-25T10:31:33.946016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:02.352885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940260+0000","last_change":"2024-04-25T10:30:33.644721+0000","last_active":"2024-04-25T10:31:33.940260+0000","last_peered":"2024-04-25T10:31:33.940260+0000","last_clean":"2024-04-25T10:31:33.940260+0000","last_became_active":"2024-04-25T10:30:33.644392+0000","last_became_peered":"2024-04-25T10:30:33.644392+0000","last_unstale":"2024-04-25T10:31:33.940260+0000","last_undegraded":"2024-04-25T10:31:33.940260+0000","last_fullsized":"2024-04-25T10:31:33.940260+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:50.571318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.938762+0000","last_change":"2024-04-25T10:30:33.647970+0000","last_active":"2024-04-25T10:31:33.938762+0000","last_peered":"2024-04-25T10:31:33.938762+0000","last_clean":"2024-04-25T10:31:33.938762+0000","last_became_active":"2024-04-25T10:30:33.647547+0000","last_became_peered":"2024-04-25T10:30:33.647547+0000","last_unstale":"2024-04-25T10:31:33.938762+0000","last_undegraded":"2024-04-25T10:31:33.938762+0000","last_fullsized":"2024-04-25T10:31:33.938762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:25.609085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.935784+0000","last_change":"2024-04-25T10:30:33.641618+0000","last_active":"2024-04-25T10:31:33.935784+0000","last_peered":"2024-04-25T10:31:33.935784+0000","last_clean":"2024-04-25T10:31:33.935784+0000","last_became_active":"2024-04-25T10:30:33.640830+0000","last_became_peered":"2024-04-25T10:30:33.640830+0000","last_unstale":"2024-04-25T10:31:33.935784+0000","last_undegraded":"2024-04-25T10:31:33.935784+0000","last_fullsized":"2024-04-25T10:31:33.935784+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:36.870127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.938525+0000","last_change":"2024-04-25T10:30:33.644010+0000","last_active":"2024-04-25T10:31:33.938525+0000","last_peered":"2024-04-25T10:31:33.938525+0000","last_clean":"2024-04-25T10:31:33.938525+0000","last_became_active":"2024-04-25T10:30:33.643639+0000","last_became_peered":"2024-04-25T10:30:33.643639+0000","last_unstale":"2024-04-25T10:31:33.938525+0000","last_undegraded":"2024-04-25T10:31:33.938525+0000","last_fullsized":"2024-04-25T10:31:33.938525+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:32.820552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.943425+0000","last_change":"2024-04-25T10:30:33.639727+0000","last_active":"2024-04-25T10:31:33.943425+0000","last_peered":"2024-04-25T10:31:33.943425+0000","last_clean":"2024-04-25T10:31:33.943425+0000","last_became_active":"2024-04-25T10:30:33.639185+0000","last_became_peered":"2024-04-25T10:30:33.639185+0000","last_unstale":"2024-04-25T10:31:33.943425+0000","last_undegraded":"2024-04-25T10:31:33.943425+0000","last_fullsized":"2024-04-25T10:31:33.943425+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:17.860864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.940540+0000","last_change":"2024-04-25T10:30:33.644650+0000","last_active":"2024-04-25T10:31:33.940540+0000","last_peered":"2024-04-25T10:31:33.940540+0000","last_clean":"2024-04-25T10:31:33.940540+0000","last_became_active":"2024-04-25T10:30:33.644264+0000","last_became_peered":"2024-04-25T10:30:33.644264+0000","last_unstale":"2024-04-25T10:31:33.940540+0000","last_undegraded":"2024-04-25T10:31:33.940540+0000","last_fullsized":"2024-04-25T10:31:33.940540+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:41.122318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.941199+0000","last_change":"2024-04-25T10:30:33.651875+0000","last_active":"2024-04-25T10:31:33.941199+0000","last_peered":"2024-04-25T10:31:33.941199+0000","last_clean":"2024-04-25T10:31:33.941199+0000","last_became_active":"2024-04-25T10:30:33.651486+0000","last_became_peered":"2024-04-25T10:30:33.651486+0000","last_unstale":"2024-04-25T10:31:33.941199+0000","last_undegraded":"2024-04-25T10:31:33.941199+0000","last_fullsized":"2024-04-25T10:31:33.941199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:32.598209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.942690+0000","last_change":"2024-04-25T10:31:33.942171+0000","last_active":"2024-04-25T10:31:33.942690+0000","last_peered":"2024-04-25T10:31:33.942690+0000","last_clean":"2024-04-25T10:31:33.942690+0000","last_became_active":"2024-04-25T10:31:33.941684+0000","last_became_peered":"2024-04-25T10:31:33.941684+0000","last_unstale":"2024-04-25T10:31:33.942690+0000","last_undegraded":"2024-04-25T10:31:33.942690+0000","last_fullsized":"2024-04-25T10:31:33.942690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:45.291952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.937157+0000","last_change":"2024-04-25T10:30:33.639915+0000","last_active":"2024-04-25T10:31:33.937157+0000","last_peered":"2024-04-25T10:31:33.937157+0000","last_clean":"2024-04-25T10:31:33.937157+0000","last_became_active":"2024-04-25T10:30:33.639137+0000","last_became_peered":"2024-04-25T10:30:33.639137+0000","last_unstale":"2024-04-25T10:31:33.937157+0000","last_undegraded":"2024-04-25T10:31:33.937157+0000","last_fullsized":"2024-04-25T10:31:33.937157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:13.143405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.935743+0000","last_change":"2024-04-25T10:30:33.643547+0000","last_active":"2024-04-25T10:31:33.935743+0000","last_peered":"2024-04-25T10:31:33.935743+0000","last_clean":"2024-04-25T10:31:33.935743+0000","last_became_active":"2024-04-25T10:30:33.643158+0000","last_became_peered":"2024-04-25T10:30:33.643158+0000","last_unstale":"2024-04-25T10:31:33.935743+0000","last_undegraded":"2024-04-25T10:31:33.935743+0000","last_fullsized":"2024-04-25T10:31:33.935743+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:07.517132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:31:33.938270+0000","last_change":"2024-04-25T10:30:33.645746+0000","last_active":"2024-04-25T10:31:33.938270+0000","last_peered":"2024-04-25T10:31:33.938270+0000","last_clean":"2024-04-25T10:31:33.938270+0000","last_became_active":"2024-04-25T10:30:33.644646+0000","last_became_peered":"2024-04-25T10:30:33.644646+0000","last_unstale":"2024-04-25T10:31:33.938270+0000","last_undegraded":"2024-04-25T10:31:33.938270+0000","last_fullsized":"2024-04-25T10:31:33.938270+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:21.591264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65047,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":445,"num_read_kb":291,"num_write":156,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":61068,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":195141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645342,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":856064,"data_stored":413481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841570,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36852,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708300,"statfs":{"total":95995035648,"available":95957299200,"internally_reserved":0,"allocated":1548288,"data_stored":1066377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037799,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":868352,"data_stored":413695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234028,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":864256,"data_stored":413593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430258,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32612,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712540,"statfs":{"total":95995035648,"available":95961640960,"internally_reserved":0,"allocated":1462272,"data_stored":1005545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626485,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":856064,"data_stored":413084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":473923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40360,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704792,"statfs":{"total":95995035648,"available":95953707008,"internally_reserved":0,"allocated":876544,"data_stored":414355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":872448,"data_stored":414227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":872448,"data_stored":413395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986240,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":474958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822714,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36780,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708372,"statfs":{"total":95995035648,"available":95957372928,"internally_reserved":0,"allocated":1474560,"data_stored":1004838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:31:52.477 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T10:31:52.477 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T10:31:52.478 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T10:31:52.478 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T10:31:52.765 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:31:53.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:52 smithi043 ceph-mon[29478]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:31:53.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:52 smithi077 ceph-mon[32772]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:31:53.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:52 smithi164 ceph-mon[32713]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:31:54.098 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:53 smithi043 ceph-mon[29478]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:31:54.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:53 smithi077 ceph-mon[32772]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:31:54.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:53 smithi164 ceph-mon[32713]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:31:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.434+0000 7f861852a700 1 -- 172.21.15.43:0/1110536757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8610109700 msgr2=0x7f8610111060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.434+0000 7f861852a700 1 --2- 172.21.15.43:0/1110536757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8610109700 0x7f8610111060 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f860800b0d0 tx=0x7f860800b3e0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.434+0000 7f861852a700 1 -- 172.21.15.43:0/1110536757 shutdown_connections 2024-04-25T10:31:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.434+0000 7f861852a700 1 --2- 172.21.15.43:0/1110536757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8610109700 0x7f8610111060 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.434+0000 7f861852a700 1 --2- 172.21.15.43:0/1110536757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8610104480 0x7f8610104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.434+0000 7f861852a700 1 --2- 172.21.15.43:0/1110536757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8610108db0 0x7f86101091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.434+0000 7f861852a700 1 -- 172.21.15.43:0/1110536757 >> 172.21.15.43:0/1110536757 conn(0x7f86100ffd70 msgr2=0x7f86101021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.434+0000 7f861852a700 1 -- 172.21.15.43:0/1110536757 shutdown_connections 2024-04-25T10:31:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.434+0000 7f861852a700 1 -- 172.21.15.43:0/1110536757 wait complete. 2024-04-25T10:31:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.434+0000 7f861852a700 1 Processor -- start 2024-04-25T10:31:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.435+0000 7f861852a700 1 -- start start 2024-04-25T10:31:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.435+0000 7f861852a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8610104480 0x7f8610076aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.435+0000 7f861852a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8610108db0 0x7f8610076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.435+0000 7f861852a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8610109700 0x7f8610077520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.435+0000 7f861852a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8610114330 con 0x7f8610109700 2024-04-25T10:31:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.435+0000 7f861852a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86101141b0 con 0x7f8610108db0 2024-04-25T10:31:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.435+0000 7f861852a700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f86101144b0 con 0x7f8610104480 2024-04-25T10:31:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.435+0000 7f8616ac7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8610109700 0x7f8610077520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.435+0000 7f8616ac7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8610109700 0x7f8610077520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50922/0 (socket says 172.21.15.43:50922) 2024-04-25T10:31:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.435+0000 7f8616ac7700 1 -- 172.21.15.43:0/653663346 learned_addr learned my addr 172.21.15.43:0/653663346 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.435+0000 7f86162c6700 1 --2- 172.21.15.43:0/653663346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8610104480 0x7f8610076aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.435+0000 7f8615ac5700 1 --2- 172.21.15.43:0/653663346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8610108db0 0x7f8610076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.435+0000 7f8616ac7700 1 -- 172.21.15.43:0/653663346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8610104480 msgr2=0x7f8610076aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.435+0000 7f8616ac7700 1 --2- 172.21.15.43:0/653663346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8610104480 0x7f8610076aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.435+0000 7f8616ac7700 1 -- 172.21.15.43:0/653663346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8610108db0 msgr2=0x7f8610076fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.435+0000 7f8616ac7700 1 --2- 172.21.15.43:0/653663346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8610108db0 0x7f8610076fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.435+0000 7f8616ac7700 1 -- 172.21.15.43:0/653663346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86101a8fc0 con 0x7f8610109700 2024-04-25T10:31:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.436+0000 7f8616ac7700 1 --2- 172.21.15.43:0/653663346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8610109700 0x7f8610077520 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8608009850 tx=0x7f8608007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.436+0000 7f86077fe700 1 -- 172.21.15.43:0/653663346 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860801f070 con 0x7f8610109700 2024-04-25T10:31:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.436+0000 7f86077fe700 1 -- 172.21.15.43:0/653663346 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8608004c50 con 0x7f8610109700 2024-04-25T10:31:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.436+0000 7f861852a700 1 -- 172.21.15.43:0/653663346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86101a9250 con 0x7f8610109700 2024-04-25T10:31:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.436+0000 7f86077fe700 1 -- 172.21.15.43:0/653663346 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860801b7b0 con 0x7f8610109700 2024-04-25T10:31:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.436+0000 7f861852a700 1 -- 172.21.15.43:0/653663346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86101a96a0 con 0x7f8610109700 2024-04-25T10:31:54.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.437+0000 7f861852a700 1 -- 172.21.15.43:0/653663346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8610051190 con 0x7f8610109700 2024-04-25T10:31:54.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.439+0000 7f86077fe700 1 -- 172.21.15.43:0/653663346 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f8608022070 con 0x7f8610109700 2024-04-25T10:31:54.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.439+0000 7f86077fe700 1 --2- 172.21.15.43:0/653663346 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f85fc0afc10 0x7f85fc0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.439+0000 7f86162c6700 1 --2- 172.21.15.43:0/653663346 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f85fc0afc10 0x7f85fc0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.439+0000 7f86077fe700 1 -- 172.21.15.43:0/653663346 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f860800f020 con 0x7f8610109700 2024-04-25T10:31:54.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.440+0000 7f86162c6700 1 --2- 172.21.15.43:0/653663346 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f85fc0afc10 0x7f85fc0b2120 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f860c007be0 tx=0x7f860c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.443+0000 7f86077fe700 1 -- 172.21.15.43:0/653663346 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8608029020 con 0x7f8610109700 2024-04-25T10:31:54.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.697+0000 7f861852a700 1 -- 172.21.15.43:0/653663346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f861002e5b0 con 0x7f8610109700 2024-04-25T10:31:54.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.697+0000 7f86077fe700 1 -- 172.21.15.43:0/653663346 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f86080a8e00 con 0x7f8610109700 2024-04-25T10:31:54.699 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:31:54.700 INFO:teuthology.orchestra.run.smithi043.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:31:54.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.700+0000 7f861852a700 1 -- 172.21.15.43:0/653663346 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f85fc0afc10 msgr2=0x7f85fc0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:54.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.700+0000 7f861852a700 1 --2- 172.21.15.43:0/653663346 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f85fc0afc10 0x7f85fc0b2120 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f860c007be0 tx=0x7f860c0073d0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.700+0000 7f861852a700 1 -- 172.21.15.43:0/653663346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8610109700 msgr2=0x7f8610077520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:54.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.700+0000 7f861852a700 1 --2- 172.21.15.43:0/653663346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8610109700 0x7f8610077520 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8608009850 tx=0x7f8608007a20 comp rx=0 tx=0).stop 2024-04-25T10:31:54.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.700+0000 7f861852a700 1 -- 172.21.15.43:0/653663346 shutdown_connections 2024-04-25T10:31:54.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.700+0000 7f861852a700 1 --2- 172.21.15.43:0/653663346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8610109700 0x7f8610077520 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.700+0000 7f861852a700 1 --2- 172.21.15.43:0/653663346 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f85fc0afc10 0x7f85fc0b2120 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.700+0000 7f861852a700 1 --2- 172.21.15.43:0/653663346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8610104480 0x7f8610076aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.700+0000 7f861852a700 1 --2- 172.21.15.43:0/653663346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8610108db0 0x7f8610076fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.700+0000 7f861852a700 1 -- 172.21.15.43:0/653663346 >> 172.21.15.43:0/653663346 conn(0x7f86100ffd70 msgr2=0x7f8610108520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:54.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.700+0000 7f861852a700 1 -- 172.21.15.43:0/653663346 shutdown_connections 2024-04-25T10:31:54.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:54.700+0000 7f861852a700 1 -- 172.21.15.43:0/653663346 wait complete. 2024-04-25T10:31:55.015 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:54 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/653663346' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T10:31:55.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:54 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/653663346' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T10:31:55.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:54 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/653663346' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T10:31:55.337 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T10:31:55.338 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:55.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.454+0000 7fd6d3842640 1 -- 172.21.15.43:0/2529767644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6cc1029c0 msgr2=0x7fd6cc1036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:55.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.454+0000 7fd6d3842640 1 --2- 172.21.15.43:0/2529767644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6cc1029c0 0x7fd6cc1036c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009900 tx=0x7fd6c802a410 comp rx=0 tx=0).stop 2024-04-25T10:31:55.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.454+0000 7fd6d3842640 1 -- 172.21.15.43:0/2529767644 shutdown_connections 2024-04-25T10:31:55.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.454+0000 7fd6d3842640 1 --2- 172.21.15.43:0/2529767644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6cc108900 0x7fd6cc10ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.454+0000 7fd6d3842640 1 --2- 172.21.15.43:0/2529767644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6cc103c00 0x7fd6cc104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.454+0000 7fd6d3842640 1 --2- 172.21.15.43:0/2529767644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6cc1029c0 0x7fd6cc1036c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.454+0000 7fd6d3842640 1 -- 172.21.15.43:0/2529767644 >> 172.21.15.43:0/2529767644 conn(0x7fd6cc0fdbd0 msgr2=0x7fd6cc100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:55.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.454+0000 7fd6d3842640 1 -- 172.21.15.43:0/2529767644 shutdown_connections 2024-04-25T10:31:55.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.454+0000 7fd6d3842640 1 -- 172.21.15.43:0/2529767644 wait complete. 2024-04-25T10:31:55.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.455+0000 7fd6d3842640 1 Processor -- start 2024-04-25T10:31:55.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.455+0000 7fd6d3842640 1 -- start start 2024-04-25T10:31:55.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.455+0000 7fd6d3842640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6cc1029c0 0x7fd6cc06a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:55.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.455+0000 7fd6d3842640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6cc103c00 0x7fd6cc19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:55.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.455+0000 7fd6d3842640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6cc108900 0x7fd6cc1a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:55.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.455+0000 7fd6d3842640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc10e4c0 con 0x7fd6cc108900 2024-04-25T10:31:55.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.455+0000 7fd6d3842640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc10e340 con 0x7fd6cc103c00 2024-04-25T10:31:55.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.455+0000 7fd6d3842640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc10e640 con 0x7fd6cc1029c0 2024-04-25T10:31:55.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.455+0000 7fd6d1ddf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6cc108900 0x7fd6cc1a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:55.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.455+0000 7fd6d1ddf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6cc108900 0x7fd6cc1a1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50934/0 (socket says 172.21.15.43:50934) 2024-04-25T10:31:55.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.455+0000 7fd6d1ddf640 1 -- 172.21.15.43:0/931906673 learned_addr learned my addr 172.21.15.43:0/931906673 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:55.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.456+0000 7fd6d15de640 1 --2- 172.21.15.43:0/931906673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6cc1029c0 0x7fd6cc06a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:55.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.456+0000 7fd6d0ddd640 1 --2- 172.21.15.43:0/931906673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6cc103c00 0x7fd6cc19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:55.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.456+0000 7fd6d1ddf640 1 -- 172.21.15.43:0/931906673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6cc1029c0 msgr2=0x7fd6cc06a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:55.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.456+0000 7fd6d1ddf640 1 --2- 172.21.15.43:0/931906673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6cc1029c0 0x7fd6cc06a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.456+0000 7fd6d1ddf640 1 -- 172.21.15.43:0/931906673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6cc103c00 msgr2=0x7fd6cc19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:55.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.456+0000 7fd6d1ddf640 1 --2- 172.21.15.43:0/931906673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6cc103c00 0x7fd6cc19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.456+0000 7fd6d1ddf640 1 -- 172.21.15.43:0/931906673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6cc1a1d50 con 0x7fd6cc108900 2024-04-25T10:31:55.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.456+0000 7fd6d0ddd640 1 --2- 172.21.15.43:0/931906673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6cc103c00 0x7fd6cc19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:55.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.456+0000 7fd6d1ddf640 1 --2- 172.21.15.43:0/931906673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6cc108900 0x7fd6cc1a1530 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400eaa0 tx=0x7fd6c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:55.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.456+0000 7fd6c27fc640 1 -- 172.21.15.43:0/931906673 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c400cdb0 con 0x7fd6cc108900 2024-04-25T10:31:55.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.456+0000 7fd6c27fc640 1 -- 172.21.15.43:0/931906673 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6c4005020 con 0x7fd6cc108900 2024-04-25T10:31:55.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.456+0000 7fd6d3842640 1 -- 172.21.15.43:0/931906673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6cc1a1fe0 con 0x7fd6cc108900 2024-04-25T10:31:55.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.456+0000 7fd6c27fc640 1 -- 172.21.15.43:0/931906673 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c40116d0 con 0x7fd6cc108900 2024-04-25T10:31:55.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.456+0000 7fd6d3842640 1 -- 172.21.15.43:0/931906673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6cc1a24a0 con 0x7fd6cc108900 2024-04-25T10:31:55.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.457+0000 7fd6d3842640 1 -- 172.21.15.43:0/931906673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6cc107710 con 0x7fd6cc108900 2024-04-25T10:31:55.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.459+0000 7fd6c27fc640 1 -- 172.21.15.43:0/931906673 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fd6c40051d0 con 0x7fd6cc108900 2024-04-25T10:31:55.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.460+0000 7fd6c27fc640 1 --2- 172.21.15.43:0/931906673 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd6b80afc00 0x7fd6b80b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:55.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.460+0000 7fd6d15de640 1 --2- 172.21.15.43:0/931906673 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd6b80afc00 0x7fd6b80b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:55.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.460+0000 7fd6c27fc640 1 -- 172.21.15.43:0/931906673 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fd6c40df2c0 con 0x7fd6cc108900 2024-04-25T10:31:55.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.460+0000 7fd6d15de640 1 --2- 172.21.15.43:0/931906673 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd6b80afc00 0x7fd6b80b2110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8005ab0 tx=0x7fd6c8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:55.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.463+0000 7fd6c27fc640 1 -- 172.21.15.43:0/931906673 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6c4016030 con 0x7fd6cc108900 2024-04-25T10:31:55.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.652+0000 7fd6d3842640 1 -- 172.21.15.43:0/931906673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6cc02dd20 con 0x7fd6cc108900 2024-04-25T10:31:55.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.653+0000 7fd6c27fc640 1 -- 172.21.15.43:0/931906673 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22619 (secure 0 0 0) 0x7fd6c40a7d70 con 0x7fd6cc108900 2024-04-25T10:31:55.655 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:31:55.656 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":79,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:31:33.928341+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:31.503414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:32.224334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:55.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.655+0000 7fd6d3842640 1 -- 172.21.15.43:0/931906673 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd6b80afc00 msgr2=0x7fd6b80b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:55.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.655+0000 7fd6d3842640 1 --2- 172.21.15.43:0/931906673 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd6b80afc00 0x7fd6b80b2110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8005ab0 tx=0x7fd6c8031400 comp rx=0 tx=0).stop 2024-04-25T10:31:55.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.655+0000 7fd6d3842640 1 -- 172.21.15.43:0/931906673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6cc108900 msgr2=0x7fd6cc1a1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:55.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.655+0000 7fd6d3842640 1 --2- 172.21.15.43:0/931906673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6cc108900 0x7fd6cc1a1530 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400eaa0 tx=0x7fd6c400ef60 comp rx=0 tx=0).stop 2024-04-25T10:31:55.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.655+0000 7fd6d3842640 1 -- 172.21.15.43:0/931906673 shutdown_connections 2024-04-25T10:31:55.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.655+0000 7fd6d3842640 1 --2- 172.21.15.43:0/931906673 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd6b80afc00 0x7fd6b80b2110 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.655+0000 7fd6d3842640 1 --2- 172.21.15.43:0/931906673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6cc103c00 0x7fd6cc19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.655+0000 7fd6d3842640 1 --2- 172.21.15.43:0/931906673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6cc108900 0x7fd6cc1a1530 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.655+0000 7fd6d3842640 1 --2- 172.21.15.43:0/931906673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6cc1029c0 0x7fd6cc06a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.656+0000 7fd6d3842640 1 -- 172.21.15.43:0/931906673 >> 172.21.15.43:0/931906673 conn(0x7fd6cc0fdbd0 msgr2=0x7fd6cc1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:55.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.656+0000 7fd6d3842640 1 -- 172.21.15.43:0/931906673 shutdown_connections 2024-04-25T10:31:55.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.656+0000 7fd6d3842640 1 -- 172.21.15.43:0/931906673 wait complete. 2024-04-25T10:31:55.669 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:25:58.167476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:31.503414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:32.224334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:31:55.670 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:31:55.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.785+0000 7f1bbf384640 1 -- 172.21.15.43:0/918225568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bb8103c30 msgr2=0x7f1bb8104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:55.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.785+0000 7f1bbf384640 1 --2- 172.21.15.43:0/918225568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bb8103c30 0x7f1bb8104040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8009900 tx=0x7f1ba802a410 comp rx=0 tx=0).stop 2024-04-25T10:31:55.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.785+0000 7f1bbf384640 1 -- 172.21.15.43:0/918225568 shutdown_connections 2024-04-25T10:31:55.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.785+0000 7f1bbf384640 1 --2- 172.21.15.43:0/918225568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bb8108930 0x7f1bb810ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.785+0000 7f1bbf384640 1 --2- 172.21.15.43:0/918225568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bb8103c30 0x7f1bb8104040 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.785+0000 7f1bbf384640 1 --2- 172.21.15.43:0/918225568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bb81029f0 0x7f1bb81036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.785+0000 7f1bbf384640 1 -- 172.21.15.43:0/918225568 >> 172.21.15.43:0/918225568 conn(0x7f1bb80fdbc0 msgr2=0x7f1bb8100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:55.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.785+0000 7f1bbf384640 1 -- 172.21.15.43:0/918225568 shutdown_connections 2024-04-25T10:31:55.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.785+0000 7f1bbf384640 1 -- 172.21.15.43:0/918225568 wait complete. 2024-04-25T10:31:55.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.786+0000 7f1bbf384640 1 Processor -- start 2024-04-25T10:31:55.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.786+0000 7f1bbf384640 1 -- start start 2024-04-25T10:31:55.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.786+0000 7f1bbf384640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bb81029f0 0x7f1bb81991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:55.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.786+0000 7f1bbf384640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bb8103c30 0x7f1bb81996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:55.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.786+0000 7f1bbf384640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bb8108930 0x7f1bb81a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:55.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.786+0000 7f1bbf384640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb810e510 con 0x7f1bb8108930 2024-04-25T10:31:55.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.786+0000 7f1bbf384640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb810e390 con 0x7f1bb8103c30 2024-04-25T10:31:55.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.786+0000 7f1bbf384640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb810e690 con 0x7f1bb81029f0 2024-04-25T10:31:55.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.786+0000 7f1bbd921640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bb8108930 0x7f1bb81a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:55.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.786+0000 7f1bbd120640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bb81029f0 0x7f1bb81991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:55.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.786+0000 7f1bbd921640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bb8108930 0x7f1bb81a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50962/0 (socket says 172.21.15.43:50962) 2024-04-25T10:31:55.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.786+0000 7f1bbd921640 1 -- 172.21.15.43:0/716160476 learned_addr learned my addr 172.21.15.43:0/716160476 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:55.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.787+0000 7f1bbc91f640 1 --2- 172.21.15.43:0/716160476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bb8103c30 0x7f1bb81996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:55.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.787+0000 7f1bbd921640 1 -- 172.21.15.43:0/716160476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bb81029f0 msgr2=0x7f1bb81991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:55.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.787+0000 7f1bbd921640 1 --2- 172.21.15.43:0/716160476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bb81029f0 0x7f1bb81991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.787+0000 7f1bbd921640 1 -- 172.21.15.43:0/716160476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bb8103c30 msgr2=0x7f1bb81996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:55.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.787+0000 7f1bbd921640 1 --2- 172.21.15.43:0/716160476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bb8103c30 0x7f1bb81996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.787+0000 7f1bbd921640 1 -- 172.21.15.43:0/716160476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb81a0c80 con 0x7f1bb8108930 2024-04-25T10:31:55.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.787+0000 7f1bbd921640 1 --2- 172.21.15.43:0/716160476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bb8108930 0x7f1bb81a0610 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000cad0 tx=0x7f1bb000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:55.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.787+0000 7f1bae7fc640 1 -- 172.21.15.43:0/716160476 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb000ae20 con 0x7f1bb8108930 2024-04-25T10:31:55.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.787+0000 7f1bae7fc640 1 -- 172.21.15.43:0/716160476 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bb0005020 con 0x7f1bb8108930 2024-04-25T10:31:55.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.787+0000 7f1bae7fc640 1 -- 172.21.15.43:0/716160476 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb000f6b0 con 0x7f1bb8108930 2024-04-25T10:31:55.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.787+0000 7f1bbf384640 1 -- 172.21.15.43:0/716160476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb81a0f70 con 0x7f1bb8108930 2024-04-25T10:31:55.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.787+0000 7f1bbf384640 1 -- 172.21.15.43:0/716160476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb81a1380 con 0x7f1bb8108930 2024-04-25T10:31:55.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.788+0000 7f1bbf384640 1 -- 172.21.15.43:0/716160476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb806a190 con 0x7f1bb8108930 2024-04-25T10:31:55.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.790+0000 7f1bae7fc640 1 -- 172.21.15.43:0/716160476 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f1bb00051d0 con 0x7f1bb8108930 2024-04-25T10:31:55.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.791+0000 7f1bae7fc640 1 --2- 172.21.15.43:0/716160476 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1ba40afb80 0x7f1ba40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:55.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.791+0000 7f1bbd120640 1 --2- 172.21.15.43:0/716160476 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1ba40afb80 0x7f1ba40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:55.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.791+0000 7f1bae7fc640 1 -- 172.21.15.43:0/716160476 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f1bb00df190 con 0x7f1bb8108930 2024-04-25T10:31:55.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.791+0000 7f1bbd120640 1 --2- 172.21.15.43:0/716160476 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1ba40afb80 0x7f1ba40b2090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4005fd0 tx=0x7f1bb400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:55.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.795+0000 7f1bae7fc640 1 -- 172.21.15.43:0/716160476 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1bb806a190 con 0x7f1bb8108930 2024-04-25T10:31:55.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.983+0000 7f1bbf384640 1 -- 172.21.15.43:0/716160476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1bb802dbd0 con 0x7f1bb8108930 2024-04-25T10:31:55.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.983+0000 7f1bae7fc640 1 -- 172.21.15.43:0/716160476 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f1bb0014030 con 0x7f1bb8108930 2024-04-25T10:31:55.985 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T10:31:55.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.986+0000 7f1bbf384640 1 -- 172.21.15.43:0/716160476 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1ba40afb80 msgr2=0x7f1ba40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:55.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.986+0000 7f1bbf384640 1 --2- 172.21.15.43:0/716160476 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1ba40afb80 0x7f1ba40b2090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4005fd0 tx=0x7f1bb400b390 comp rx=0 tx=0).stop 2024-04-25T10:31:55.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.986+0000 7f1bbf384640 1 -- 172.21.15.43:0/716160476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bb8108930 msgr2=0x7f1bb81a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:55.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.986+0000 7f1bbf384640 1 --2- 172.21.15.43:0/716160476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bb8108930 0x7f1bb81a0610 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000cad0 tx=0x7f1bb000cf90 comp rx=0 tx=0).stop 2024-04-25T10:31:55.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.986+0000 7f1bbf384640 1 -- 172.21.15.43:0/716160476 shutdown_connections 2024-04-25T10:31:55.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.986+0000 7f1bbf384640 1 --2- 172.21.15.43:0/716160476 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1ba40afb80 0x7f1ba40b2090 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.986+0000 7f1bbf384640 1 --2- 172.21.15.43:0/716160476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bb8103c30 0x7f1bb81996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.986+0000 7f1bbf384640 1 --2- 172.21.15.43:0/716160476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bb8108930 0x7f1bb81a0610 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.986+0000 7f1bbf384640 1 --2- 172.21.15.43:0/716160476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bb81029f0 0x7f1bb81991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.986+0000 7f1bbf384640 1 -- 172.21.15.43:0/716160476 >> 172.21.15.43:0/716160476 conn(0x7f1bb80fdbc0 msgr2=0x7f1bb8104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:55.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.986+0000 7f1bbf384640 1 -- 172.21.15.43:0/716160476 shutdown_connections 2024-04-25T10:31:55.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:55.986+0000 7f1bbf384640 1 -- 172.21.15.43:0/716160476 wait complete. 2024-04-25T10:31:55.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:55 smithi043 ceph-mon[29478]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T10:31:55.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:55 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/931906673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:31:55.999 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:31:56.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.114+0000 7effd0a20640 1 -- 172.21.15.43:0/3705402835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effcc102240 msgr2=0x7effcc10bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.114+0000 7effd0a20640 1 --2- 172.21.15.43:0/3705402835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effcc102240 0x7effcc10bca0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7effc4009ca0 tx=0x7effc402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.114+0000 7effd0a20640 1 -- 172.21.15.43:0/3705402835 shutdown_connections 2024-04-25T10:31:56.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.114+0000 7effd0a20640 1 --2- 172.21.15.43:0/3705402835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effcc102240 0x7effcc10bca0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.114+0000 7effd0a20640 1 --2- 172.21.15.43:0/3705402835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effcc101740 0x7effcc101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.114+0000 7effd0a20640 1 --2- 172.21.15.43:0/3705402835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effcc105090 0x7effcc1054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.114+0000 7effd0a20640 1 -- 172.21.15.43:0/3705402835 >> 172.21.15.43:0/3705402835 conn(0x7effcc0fd340 msgr2=0x7effcc0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:56.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.114+0000 7effd0a20640 1 -- 172.21.15.43:0/3705402835 shutdown_connections 2024-04-25T10:31:56.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.114+0000 7effd0a20640 1 -- 172.21.15.43:0/3705402835 wait complete. 2024-04-25T10:31:56.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.115+0000 7effd0a20640 1 Processor -- start 2024-04-25T10:31:56.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.115+0000 7effd0a20640 1 -- start start 2024-04-25T10:31:56.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.115+0000 7effd0a20640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effcc101740 0x7effcc19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.115+0000 7effd0a20640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effcc102240 0x7effcc19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.115+0000 7effd0a20640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effcc105090 0x7effcc1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.115+0000 7effd0a20640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7effcc10f620 con 0x7effcc101740 2024-04-25T10:31:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.115+0000 7effd0a20640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7effcc10f4a0 con 0x7effcc105090 2024-04-25T10:31:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.115+0000 7effd0a20640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7effcc10f7a0 con 0x7effcc102240 2024-04-25T10:31:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.115+0000 7effca59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effcc101740 0x7effcc19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.116+0000 7effca59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effcc101740 0x7effcc19a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50978/0 (socket says 172.21.15.43:50978) 2024-04-25T10:31:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.116+0000 7effca59c640 1 -- 172.21.15.43:0/3679342347 learned_addr learned my addr 172.21.15.43:0/3679342347 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.116+0000 7effc9d9b640 1 --2- 172.21.15.43:0/3679342347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effcc102240 0x7effcc19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.116+0000 7effcad9d640 1 --2- 172.21.15.43:0/3679342347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effcc105090 0x7effcc1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.116+0000 7effca59c640 1 -- 172.21.15.43:0/3679342347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effcc102240 msgr2=0x7effcc19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.116+0000 7effca59c640 1 --2- 172.21.15.43:0/3679342347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effcc102240 0x7effcc19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.116+0000 7effca59c640 1 -- 172.21.15.43:0/3679342347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effcc105090 msgr2=0x7effcc1a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.116+0000 7effca59c640 1 --2- 172.21.15.43:0/3679342347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effcc105090 0x7effcc1a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.116+0000 7effca59c640 1 -- 172.21.15.43:0/3679342347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effcc1a1ce0 con 0x7effcc101740 2024-04-25T10:31:56.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.116+0000 7effc9d9b640 1 --2- 172.21.15.43:0/3679342347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effcc102240 0x7effcc19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:56.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.116+0000 7effcad9d640 1 --2- 172.21.15.43:0/3679342347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effcc105090 0x7effcc1a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:56.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.116+0000 7effca59c640 1 --2- 172.21.15.43:0/3679342347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effcc101740 0x7effcc19a1a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7effbc005fd0 tx=0x7effbc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:56.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.116+0000 7effbb7fe640 1 -- 172.21.15.43:0/3679342347 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc017070 con 0x7effcc101740 2024-04-25T10:31:56.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.116+0000 7effd0a20640 1 -- 172.21.15.43:0/3679342347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effcc1a1fd0 con 0x7effcc101740 2024-04-25T10:31:56.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.116+0000 7effbb7fe640 1 -- 172.21.15.43:0/3679342347 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effbc009c30 con 0x7effcc101740 2024-04-25T10:31:56.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.116+0000 7effbb7fe640 1 -- 172.21.15.43:0/3679342347 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc003d10 con 0x7effcc101740 2024-04-25T10:31:56.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.116+0000 7effd0a20640 1 -- 172.21.15.43:0/3679342347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effcc1a2430 con 0x7effcc101740 2024-04-25T10:31:56.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.118+0000 7effd0a20640 1 -- 172.21.15.43:0/3679342347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effcc0692e0 con 0x7effcc101740 2024-04-25T10:31:56.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.119+0000 7effbb7fe640 1 -- 172.21.15.43:0/3679342347 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7effbc003eb0 con 0x7effcc101740 2024-04-25T10:31:56.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.119+0000 7effbb7fe640 1 --2- 172.21.15.43:0/3679342347 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7effb40afb80 0x7effb40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.120+0000 7effc9d9b640 1 --2- 172.21.15.43:0/3679342347 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7effb40afb80 0x7effb40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.120+0000 7effbb7fe640 1 -- 172.21.15.43:0/3679342347 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7effbc0deb40 con 0x7effcc101740 2024-04-25T10:31:56.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.120+0000 7effc9d9b640 1 --2- 172.21.15.43:0/3679342347 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7effb40afb80 0x7effb40b2090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7effc0005fd0 tx=0x7effc0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:56.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.124+0000 7effbb7fe640 1 -- 172.21.15.43:0/3679342347 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7effbc020030 con 0x7effcc101740 2024-04-25T10:31:56.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:55 smithi077 ceph-mon[32772]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T10:31:56.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:55 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/931906673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:31:56.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:55 smithi164 ceph-mon[32713]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T10:31:56.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:55 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/931906673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:31:56.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.309+0000 7effd0a20640 1 -- 172.21.15.43:0/3679342347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7effcc02db50 con 0x7effcc101740 2024-04-25T10:31:56.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.310+0000 7effbb7fe640 1 -- 172.21.15.43:0/3679342347 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7effbc009de0 con 0x7effcc101740 2024-04-25T10:31:56.312 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T10:31:56.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.311+0000 7effd0a20640 1 -- 172.21.15.43:0/3679342347 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7effb40afb80 msgr2=0x7effb40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.311+0000 7effd0a20640 1 --2- 172.21.15.43:0/3679342347 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7effb40afb80 0x7effb40b2090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7effc0005fd0 tx=0x7effc0005850 comp rx=0 tx=0).stop 2024-04-25T10:31:56.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.312+0000 7effd0a20640 1 -- 172.21.15.43:0/3679342347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effcc101740 msgr2=0x7effcc19a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.312+0000 7effd0a20640 1 --2- 172.21.15.43:0/3679342347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effcc101740 0x7effcc19a1a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7effbc005fd0 tx=0x7effbc00dde0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.312+0000 7effd0a20640 1 -- 172.21.15.43:0/3679342347 shutdown_connections 2024-04-25T10:31:56.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.312+0000 7effd0a20640 1 --2- 172.21.15.43:0/3679342347 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7effb40afb80 0x7effb40b2090 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.312+0000 7effd0a20640 1 --2- 172.21.15.43:0/3679342347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effcc105090 0x7effcc1a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.312+0000 7effd0a20640 1 --2- 172.21.15.43:0/3679342347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effcc102240 0x7effcc19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.312+0000 7effd0a20640 1 --2- 172.21.15.43:0/3679342347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effcc101740 0x7effcc19a1a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.312+0000 7effd0a20640 1 -- 172.21.15.43:0/3679342347 >> 172.21.15.43:0/3679342347 conn(0x7effcc0fd340 msgr2=0x7effcc107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:56.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.312+0000 7effd0a20640 1 -- 172.21.15.43:0/3679342347 shutdown_connections 2024-04-25T10:31:56.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.312+0000 7effd0a20640 1 -- 172.21.15.43:0/3679342347 wait complete. 2024-04-25T10:31:56.326 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:31:56.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.440+0000 7f6f35630640 1 -- 172.21.15.43:0/1363121622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f301098b0 msgr2=0x7f6f3010bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.440+0000 7f6f35630640 1 --2- 172.21.15.43:0/1363121622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f301098b0 0x7f6f3010bcd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009ca0 tx=0x7f6f2802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.440+0000 7f6f35630640 1 -- 172.21.15.43:0/1363121622 shutdown_connections 2024-04-25T10:31:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.440+0000 7f6f35630640 1 --2- 172.21.15.43:0/1363121622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f30106f50 0x7f6f30109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.440+0000 7f6f35630640 1 --2- 172.21.15.43:0/1363121622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f301098b0 0x7f6f3010bcd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.440+0000 7f6f35630640 1 --2- 172.21.15.43:0/1363121622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f301045f0 0x7f6f30106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.440+0000 7f6f35630640 1 -- 172.21.15.43:0/1363121622 >> 172.21.15.43:0/1363121622 conn(0x7f6f300fdbf0 msgr2=0x7f6f301000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.440+0000 7f6f35630640 1 -- 172.21.15.43:0/1363121622 shutdown_connections 2024-04-25T10:31:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.440+0000 7f6f35630640 1 -- 172.21.15.43:0/1363121622 wait complete. 2024-04-25T10:31:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.441+0000 7f6f35630640 1 Processor -- start 2024-04-25T10:31:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.441+0000 7f6f35630640 1 -- start start 2024-04-25T10:31:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.441+0000 7f6f35630640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f301045f0 0x7f6f3019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.441+0000 7f6f35630640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f30106f50 0x7f6f3019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.441+0000 7f6f35630640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f301098b0 0x7f6f301a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.441+0000 7f6f35630640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3010f630 con 0x7f6f301098b0 2024-04-25T10:31:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.441+0000 7f6f35630640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3010f4b0 con 0x7f6f30106f50 2024-04-25T10:31:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.441+0000 7f6f35630640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3010f7b0 con 0x7f6f301045f0 2024-04-25T10:31:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.441+0000 7f6f2f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f301098b0 0x7f6f301a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.441+0000 7f6f2f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f301098b0 0x7f6f301a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50986/0 (socket says 172.21.15.43:50986) 2024-04-25T10:31:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.441+0000 7f6f2f7fe640 1 -- 172.21.15.43:0/1035222016 learned_addr learned my addr 172.21.15.43:0/1035222016 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.442+0000 7f6f2effd640 1 --2- 172.21.15.43:0/1035222016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f301045f0 0x7f6f3019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.442+0000 7f6f2e7fc640 1 --2- 172.21.15.43:0/1035222016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f30106f50 0x7f6f3019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.442+0000 7f6f2f7fe640 1 -- 172.21.15.43:0/1035222016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f301045f0 msgr2=0x7f6f3019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.442+0000 7f6f2f7fe640 1 --2- 172.21.15.43:0/1035222016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f301045f0 0x7f6f3019a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.442+0000 7f6f2f7fe640 1 -- 172.21.15.43:0/1035222016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f30106f50 msgr2=0x7f6f3019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.442+0000 7f6f2f7fe640 1 --2- 172.21.15.43:0/1035222016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f30106f50 0x7f6f3019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.442+0000 7f6f2f7fe640 1 -- 172.21.15.43:0/1035222016 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f301a1d70 con 0x7f6f301098b0 2024-04-25T10:31:56.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.442+0000 7f6f2f7fe640 1 --2- 172.21.15.43:0/1035222016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f301098b0 0x7f6f301a1700 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009430 tx=0x7f6f280319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:56.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.442+0000 7f6f17fff640 1 -- 172.21.15.43:0/1035222016 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28003b70 con 0x7f6f301098b0 2024-04-25T10:31:56.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.442+0000 7f6f35630640 1 -- 172.21.15.43:0/1035222016 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f301a2000 con 0x7f6f301098b0 2024-04-25T10:31:56.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.442+0000 7f6f17fff640 1 -- 172.21.15.43:0/1035222016 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f28003d10 con 0x7f6f301098b0 2024-04-25T10:31:56.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.442+0000 7f6f17fff640 1 -- 172.21.15.43:0/1035222016 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2802fb30 con 0x7f6f301098b0 2024-04-25T10:31:56.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.442+0000 7f6f35630640 1 -- 172.21.15.43:0/1035222016 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f301a2550 con 0x7f6f301098b0 2024-04-25T10:31:56.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.443+0000 7f6f35630640 1 -- 172.21.15.43:0/1035222016 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f300ff710 con 0x7f6f301098b0 2024-04-25T10:31:56.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.446+0000 7f6f17fff640 1 -- 172.21.15.43:0/1035222016 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f6f2802fcd0 con 0x7f6f301098b0 2024-04-25T10:31:56.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.447+0000 7f6f17fff640 1 --2- 172.21.15.43:0/1035222016 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f6f180afb80 0x7f6f180b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.447+0000 7f6f2effd640 1 --2- 172.21.15.43:0/1035222016 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f6f180afb80 0x7f6f180b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.447+0000 7f6f17fff640 1 -- 172.21.15.43:0/1035222016 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f6f281002c0 con 0x7f6f301098b0 2024-04-25T10:31:56.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.447+0000 7f6f2effd640 1 --2- 172.21.15.43:0/1035222016 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f6f180afb80 0x7f6f180b2090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6f300757f0 tx=0x7f6f24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:56.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.450+0000 7f6f17fff640 1 -- 172.21.15.43:0/1035222016 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f280c8ef0 con 0x7f6f301098b0 2024-04-25T10:31:56.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.638+0000 7f6f35630640 1 -- 172.21.15.43:0/1035222016 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6f3002dce0 con 0x7f6f301098b0 2024-04-25T10:31:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.638+0000 7f6f17fff640 1 -- 172.21.15.43:0/1035222016 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7f6f280c87f0 con 0x7f6f301098b0 2024-04-25T10:31:56.640 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T10:31:56.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.640+0000 7f6f35630640 1 -- 172.21.15.43:0/1035222016 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f6f180afb80 msgr2=0x7f6f180b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.640+0000 7f6f35630640 1 --2- 172.21.15.43:0/1035222016 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f6f180afb80 0x7f6f180b2090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6f300757f0 tx=0x7f6f24005f90 comp rx=0 tx=0).stop 2024-04-25T10:31:56.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.640+0000 7f6f35630640 1 -- 172.21.15.43:0/1035222016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f301098b0 msgr2=0x7f6f301a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.640+0000 7f6f35630640 1 --2- 172.21.15.43:0/1035222016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f301098b0 0x7f6f301a1700 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009430 tx=0x7f6f280319f0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.640+0000 7f6f35630640 1 -- 172.21.15.43:0/1035222016 shutdown_connections 2024-04-25T10:31:56.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.640+0000 7f6f35630640 1 --2- 172.21.15.43:0/1035222016 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f6f180afb80 0x7f6f180b2090 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.640+0000 7f6f35630640 1 --2- 172.21.15.43:0/1035222016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f30106f50 0x7f6f3019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.640+0000 7f6f35630640 1 --2- 172.21.15.43:0/1035222016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f301098b0 0x7f6f301a1700 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.640+0000 7f6f35630640 1 --2- 172.21.15.43:0/1035222016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f301045f0 0x7f6f3019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.640+0000 7f6f35630640 1 -- 172.21.15.43:0/1035222016 >> 172.21.15.43:0/1035222016 conn(0x7f6f300fdbf0 msgr2=0x7f6f30105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:56.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.640+0000 7f6f35630640 1 -- 172.21.15.43:0/1035222016 shutdown_connections 2024-04-25T10:31:56.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.640+0000 7f6f35630640 1 -- 172.21.15.43:0/1035222016 wait complete. 2024-04-25T10:31:56.654 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:31:56.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.763+0000 7fefc5096640 1 -- 172.21.15.43:0/1212252846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefc0108900 msgr2=0x7fefc010ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.763+0000 7fefc5096640 1 --2- 172.21.15.43:0/1212252846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefc0108900 0x7fefc010ad20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009ca0 tx=0x7fefb802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.764+0000 7fefc5096640 1 -- 172.21.15.43:0/1212252846 shutdown_connections 2024-04-25T10:31:56.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.764+0000 7fefc5096640 1 --2- 172.21.15.43:0/1212252846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc0103c00 0x7fefc0104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.764+0000 7fefc5096640 1 --2- 172.21.15.43:0/1212252846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefc0108900 0x7fefc010ad20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.764+0000 7fefc5096640 1 --2- 172.21.15.43:0/1212252846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefc01029c0 0x7fefc01036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.764+0000 7fefc5096640 1 -- 172.21.15.43:0/1212252846 >> 172.21.15.43:0/1212252846 conn(0x7fefc00fdbd0 msgr2=0x7fefc0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:56.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.764+0000 7fefc5096640 1 -- 172.21.15.43:0/1212252846 shutdown_connections 2024-04-25T10:31:56.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.764+0000 7fefc5096640 1 -- 172.21.15.43:0/1212252846 wait complete. 2024-04-25T10:31:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.764+0000 7fefc5096640 1 Processor -- start 2024-04-25T10:31:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.765+0000 7fefc5096640 1 -- start start 2024-04-25T10:31:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.765+0000 7fefc5096640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefc01029c0 0x7fefc019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.765+0000 7fefc5096640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefc0103c00 0x7fefc019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.765+0000 7fefc5096640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc0108900 0x7fefc01a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.765+0000 7fefc5096640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fefc010e4f0 con 0x7fefc0103c00 2024-04-25T10:31:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.765+0000 7fefc5096640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fefc010e370 con 0x7fefc0108900 2024-04-25T10:31:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.765+0000 7fefc5096640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fefc010e670 con 0x7fefc01029c0 2024-04-25T10:31:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.765+0000 7fefbe59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefc0103c00 0x7fefc019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.765+0000 7fefbe59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefc0103c00 0x7fefc019a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51004/0 (socket says 172.21.15.43:51004) 2024-04-25T10:31:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.765+0000 7fefbe59c640 1 -- 172.21.15.43:0/3185208098 learned_addr learned my addr 172.21.15.43:0/3185208098 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.765+0000 7fefbed9d640 1 --2- 172.21.15.43:0/3185208098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefc01029c0 0x7fefc019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.765+0000 7fefbf59e640 1 --2- 172.21.15.43:0/3185208098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc0108900 0x7fefc01a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.765+0000 7fefbe59c640 1 -- 172.21.15.43:0/3185208098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefc01029c0 msgr2=0x7fefc019a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.765+0000 7fefbe59c640 1 --2- 172.21.15.43:0/3185208098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefc01029c0 0x7fefc019a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.765+0000 7fefbe59c640 1 -- 172.21.15.43:0/3185208098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc0108900 msgr2=0x7fefc01a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.765+0000 7fefbe59c640 1 --2- 172.21.15.43:0/3185208098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc0108900 0x7fefc01a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.765+0000 7fefbe59c640 1 -- 172.21.15.43:0/3185208098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc01a1d00 con 0x7fefc0103c00 2024-04-25T10:31:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.766+0000 7fefbe59c640 1 --2- 172.21.15.43:0/3185208098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefc0103c00 0x7fefc019a700 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fefb000e9f0 tx=0x7fefb000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.766+0000 7fefa7fff640 1 -- 172.21.15.43:0/3185208098 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb000cd80 con 0x7fefc0103c00 2024-04-25T10:31:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.766+0000 7fefc5096640 1 -- 172.21.15.43:0/3185208098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc01a1ff0 con 0x7fefc0103c00 2024-04-25T10:31:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.766+0000 7fefa7fff640 1 -- 172.21.15.43:0/3185208098 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefb00108e0 con 0x7fefc0103c00 2024-04-25T10:31:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.766+0000 7fefa7fff640 1 -- 172.21.15.43:0/3185208098 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb0018980 con 0x7fefc0103c00 2024-04-25T10:31:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.766+0000 7fefc5096640 1 -- 172.21.15.43:0/3185208098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc01a24b0 con 0x7fefc0103c00 2024-04-25T10:31:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.767+0000 7fefc5096640 1 -- 172.21.15.43:0/3185208098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc01a2710 con 0x7fefc0103c00 2024-04-25T10:31:56.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.768+0000 7fefa7fff640 1 -- 172.21.15.43:0/3185208098 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fefb0010430 con 0x7fefc0103c00 2024-04-25T10:31:56.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.769+0000 7fefa7fff640 1 --2- 172.21.15.43:0/3185208098 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fefa80afb80 0x7fefa80b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.769+0000 7fefbed9d640 1 --2- 172.21.15.43:0/3185208098 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fefa80afb80 0x7fefa80b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.769+0000 7fefa7fff640 1 -- 172.21.15.43:0/3185208098 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fefb00df580 con 0x7fefc0103c00 2024-04-25T10:31:56.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.770+0000 7fefbed9d640 1 --2- 172.21.15.43:0/3185208098 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fefa80afb80 0x7fefa80b2090 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fefc01035c0 tx=0x7fefb4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:56.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.773+0000 7fefa7fff640 1 -- 172.21.15.43:0/3185208098 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fefb00a79c0 con 0x7fefc0103c00 2024-04-25T10:31:56.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.998+0000 7fefc5096640 1 -- 172.21.15.43:0/3185208098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefc01a28e0 con 0x7fefc0103c00 2024-04-25T10:31:57.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:56.998+0000 7fefa7fff640 1 -- 172.21.15.43:0/3185208098 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11869 (secure 0 0 0) 0x7fefb00b0090 con 0x7fefc0103c00 2024-04-25T10:31:57.000 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:31:57.000 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:30:38.804706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2595248466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2595248466},{"type":"v1","addr":"172.21.15.77:6837","nonce":2595248466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:31:57.001 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-04-25T10:31:57.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.000+0000 7fefc5096640 1 -- 172.21.15.43:0/3185208098 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fefa80afb80 msgr2=0x7fefa80b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:57.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.000+0000 7fefc5096640 1 --2- 172.21.15.43:0/3185208098 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fefa80afb80 0x7fefa80b2090 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fefc01035c0 tx=0x7fefb4005f90 comp rx=0 tx=0).stop 2024-04-25T10:31:57.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.000+0000 7fefc5096640 1 -- 172.21.15.43:0/3185208098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefc0103c00 msgr2=0x7fefc019a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:57.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.000+0000 7fefc5096640 1 --2- 172.21.15.43:0/3185208098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefc0103c00 0x7fefc019a700 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fefb000e9f0 tx=0x7fefb000eeb0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.000+0000 7fefc5096640 1 -- 172.21.15.43:0/3185208098 shutdown_connections 2024-04-25T10:31:57.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.000+0000 7fefc5096640 1 --2- 172.21.15.43:0/3185208098 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fefa80afb80 0x7fefa80b2090 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.000+0000 7fefc5096640 1 --2- 172.21.15.43:0/3185208098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc0108900 0x7fefc01a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.000+0000 7fefc5096640 1 --2- 172.21.15.43:0/3185208098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefc0103c00 0x7fefc019a700 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.000+0000 7fefc5096640 1 --2- 172.21.15.43:0/3185208098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefc01029c0 0x7fefc019a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.000+0000 7fefc5096640 1 -- 172.21.15.43:0/3185208098 >> 172.21.15.43:0/3185208098 conn(0x7fefc00fdbd0 msgr2=0x7fefc0069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:57.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.001+0000 7fefc5096640 1 -- 172.21.15.43:0/3185208098 shutdown_connections 2024-04-25T10:31:57.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.001+0000 7fefc5096640 1 -- 172.21.15.43:0/3185208098 wait complete. 2024-04-25T10:31:57.014 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:57.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:56 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/716160476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:31:57.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:56 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3679342347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:31:57.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:56 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1035222016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:31:57.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.124+0000 7f42d560c640 1 -- 172.21.15.43:0/820154565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42d0103c00 msgr2=0x7f42d0104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:57.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.124+0000 7f42d560c640 1 --2- 172.21.15.43:0/820154565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42d0103c00 0x7f42d0104010 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009900 tx=0x7f42b802a410 comp rx=0 tx=0).stop 2024-04-25T10:31:57.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.124+0000 7f42d560c640 1 -- 172.21.15.43:0/820154565 shutdown_connections 2024-04-25T10:31:57.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.124+0000 7f42d560c640 1 --2- 172.21.15.43:0/820154565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d0108900 0x7f42d010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.124+0000 7f42d560c640 1 --2- 172.21.15.43:0/820154565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42d0103c00 0x7f42d0104010 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.124+0000 7f42d560c640 1 --2- 172.21.15.43:0/820154565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d01029c0 0x7f42d01036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.124+0000 7f42d560c640 1 -- 172.21.15.43:0/820154565 >> 172.21.15.43:0/820154565 conn(0x7f42d00fdbd0 msgr2=0x7f42d0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:57.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.124+0000 7f42d560c640 1 -- 172.21.15.43:0/820154565 shutdown_connections 2024-04-25T10:31:57.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.124+0000 7f42d560c640 1 -- 172.21.15.43:0/820154565 wait complete. 2024-04-25T10:31:57.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.124+0000 7f42d560c640 1 Processor -- start 2024-04-25T10:31:57.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.124+0000 7f42d560c640 1 -- start start 2024-04-25T10:31:57.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.125+0000 7f42d560c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42d01029c0 0x7f42d006a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:57.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.125+0000 7f42d560c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d0103c00 0x7f42d019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:57.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.125+0000 7f42d560c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d0108900 0x7f42d01a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:57.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.125+0000 7f42d560c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f42d010e540 con 0x7f42d01029c0 2024-04-25T10:31:57.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.125+0000 7f42d560c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f42d010e3c0 con 0x7f42d0103c00 2024-04-25T10:31:57.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.125+0000 7f42d560c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f42d010e6c0 con 0x7f42d0108900 2024-04-25T10:31:57.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.125+0000 7f42ceffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42d01029c0 0x7f42d006a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:57.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.125+0000 7f42ceffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42d01029c0 0x7f42d006a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51028/0 (socket says 172.21.15.43:51028) 2024-04-25T10:31:57.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.125+0000 7f42ceffd640 1 -- 172.21.15.43:0/831694763 learned_addr learned my addr 172.21.15.43:0/831694763 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:57.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.125+0000 7f42c7fff640 1 --2- 172.21.15.43:0/831694763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d0103c00 0x7f42d019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:57.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.125+0000 7f42cf7fe640 1 --2- 172.21.15.43:0/831694763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d0108900 0x7f42d01a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:57.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.125+0000 7f42ceffd640 1 -- 172.21.15.43:0/831694763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d0108900 msgr2=0x7f42d01a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:57.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.125+0000 7f42ceffd640 1 --2- 172.21.15.43:0/831694763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d0108900 0x7f42d01a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.125+0000 7f42ceffd640 1 -- 172.21.15.43:0/831694763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d0103c00 msgr2=0x7f42d019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:57.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.125+0000 7f42ceffd640 1 --2- 172.21.15.43:0/831694763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d0103c00 0x7f42d019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.125+0000 7f42ceffd640 1 -- 172.21.15.43:0/831694763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d01a1d60 con 0x7f42d01029c0 2024-04-25T10:31:57.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.125+0000 7f42ceffd640 1 --2- 172.21.15.43:0/831694763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42d01029c0 0x7f42d006a920 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f42c0006fd0 tx=0x7f42c000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:57.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.125+0000 7f42ccff9640 1 -- 172.21.15.43:0/831694763 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0011830 con 0x7f42d01029c0 2024-04-25T10:31:57.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.125+0000 7f42ccff9640 1 -- 172.21.15.43:0/831694763 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42c0016070 con 0x7f42d01029c0 2024-04-25T10:31:57.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.126+0000 7f42d560c640 1 -- 172.21.15.43:0/831694763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d01a1ff0 con 0x7f42d01029c0 2024-04-25T10:31:57.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.126+0000 7f42ccff9640 1 -- 172.21.15.43:0/831694763 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0010e00 con 0x7f42d01029c0 2024-04-25T10:31:57.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.126+0000 7f42d560c640 1 -- 172.21.15.43:0/831694763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d01a2530 con 0x7f42d01029c0 2024-04-25T10:31:57.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.127+0000 7f42d560c640 1 -- 172.21.15.43:0/831694763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d0107710 con 0x7f42d01029c0 2024-04-25T10:31:57.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.128+0000 7f42ccff9640 1 -- 172.21.15.43:0/831694763 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f42c001b460 con 0x7f42d01029c0 2024-04-25T10:31:57.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.129+0000 7f42ccff9640 1 --2- 172.21.15.43:0/831694763 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f42bc0afb80 0x7f42bc0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:57.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.129+0000 7f42c7fff640 1 --2- 172.21.15.43:0/831694763 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f42bc0afb80 0x7f42bc0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:57.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.129+0000 7f42ccff9640 1 -- 172.21.15.43:0/831694763 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f42c00de370 con 0x7f42d01029c0 2024-04-25T10:31:57.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.129+0000 7f42c7fff640 1 --2- 172.21.15.43:0/831694763 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f42bc0afb80 0x7f42bc0b2090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009230 tx=0x7f42b8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:57.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.133+0000 7f42ccff9640 1 -- 172.21.15.43:0/831694763 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f42c00a6eb0 con 0x7f42d01029c0 2024-04-25T10:31:57.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:56 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/716160476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:31:57.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:56 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3679342347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:31:57.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:56 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1035222016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:31:57.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:56 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/716160476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:31:57.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:56 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3679342347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:31:57.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:56 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1035222016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:31:57.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.319+0000 7f42d560c640 1 -- 172.21.15.43:0/831694763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42d002dba0 con 0x7f42d01029c0 2024-04-25T10:31:57.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.320+0000 7f42ccff9640 1 -- 172.21.15.43:0/831694763 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22619 (secure 0 0 0) 0x7f42c00a67b0 con 0x7f42d01029c0 2024-04-25T10:31:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:31:57.323 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":79,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:31:33.928341+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:31.503414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:32.224334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:57.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.322+0000 7f42d560c640 1 -- 172.21.15.43:0/831694763 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f42bc0afb80 msgr2=0x7f42bc0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:57.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.322+0000 7f42d560c640 1 --2- 172.21.15.43:0/831694763 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f42bc0afb80 0x7f42bc0b2090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009230 tx=0x7f42b8031400 comp rx=0 tx=0).stop 2024-04-25T10:31:57.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.322+0000 7f42d560c640 1 -- 172.21.15.43:0/831694763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42d01029c0 msgr2=0x7f42d006a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:57.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.322+0000 7f42d560c640 1 --2- 172.21.15.43:0/831694763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42d01029c0 0x7f42d006a920 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f42c0006fd0 tx=0x7f42c000b8e0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.322+0000 7f42d560c640 1 -- 172.21.15.43:0/831694763 shutdown_connections 2024-04-25T10:31:57.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.322+0000 7f42d560c640 1 --2- 172.21.15.43:0/831694763 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f42bc0afb80 0x7f42bc0b2090 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.322+0000 7f42d560c640 1 --2- 172.21.15.43:0/831694763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d0103c00 0x7f42d019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.322+0000 7f42d560c640 1 --2- 172.21.15.43:0/831694763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d0108900 0x7f42d01a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.322+0000 7f42d560c640 1 --2- 172.21.15.43:0/831694763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42d01029c0 0x7f42d006a920 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.323+0000 7f42d560c640 1 -- 172.21.15.43:0/831694763 >> 172.21.15.43:0/831694763 conn(0x7f42d00fdbd0 msgr2=0x7f42d01048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:57.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.323+0000 7f42d560c640 1 -- 172.21.15.43:0/831694763 shutdown_connections 2024-04-25T10:31:57.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.323+0000 7f42d560c640 1 -- 172.21.15.43:0/831694763 wait complete. 2024-04-25T10:31:57.337 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:25:58.167476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:31.503414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:32.224334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:31:57.337 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:31:57.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.449+0000 7f625af84640 1 -- 172.21.15.43:0/3148504380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6254101760 msgr2=0x7f6254101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:57.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.449+0000 7f625af84640 1 --2- 172.21.15.43:0/3148504380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6254101760 0x7f6254101bd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6240009900 tx=0x7f624002a410 comp rx=0 tx=0).stop 2024-04-25T10:31:57.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.449+0000 7f625af84640 1 -- 172.21.15.43:0/3148504380 shutdown_connections 2024-04-25T10:31:57.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.449+0000 7f625af84640 1 --2- 172.21.15.43:0/3148504380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6254102260 0x7f625410bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.449+0000 7f625af84640 1 --2- 172.21.15.43:0/3148504380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6254101760 0x7f6254101bd0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.449+0000 7f625af84640 1 --2- 172.21.15.43:0/3148504380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62541050b0 0x7f62541054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.449+0000 7f625af84640 1 -- 172.21.15.43:0/3148504380 >> 172.21.15.43:0/3148504380 conn(0x7f62540fd380 msgr2=0x7f62540ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:57.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.450+0000 7f625af84640 1 -- 172.21.15.43:0/3148504380 shutdown_connections 2024-04-25T10:31:57.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.450+0000 7f625af84640 1 -- 172.21.15.43:0/3148504380 wait complete. 2024-04-25T10:31:57.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.450+0000 7f625af84640 1 Processor -- start 2024-04-25T10:31:57.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.450+0000 7f625af84640 1 -- start start 2024-04-25T10:31:57.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.450+0000 7f625af84640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6254101760 0x7f62541041f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:57.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.450+0000 7f625af84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6254102260 0x7f6254102900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:57.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.450+0000 7f625af84640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62541050b0 0x7f6254102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:57.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.450+0000 7f625af84640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f625410f640 con 0x7f6254102260 2024-04-25T10:31:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.450+0000 7f625af84640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f625410f4c0 con 0x7f62541050b0 2024-04-25T10:31:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.450+0000 7f625af84640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f625410f7c0 con 0x7f6254101760 2024-04-25T10:31:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.450+0000 7f6253fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6254102260 0x7f6254102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.450+0000 7f6253fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6254102260 0x7f6254102900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51048/0 (socket says 172.21.15.43:51048) 2024-04-25T10:31:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.450+0000 7f6253fff640 1 -- 172.21.15.43:0/939590147 learned_addr learned my addr 172.21.15.43:0/939590147 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.451+0000 7f6259521640 1 --2- 172.21.15.43:0/939590147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62541050b0 0x7f6254102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.451+0000 7f6258d20640 1 --2- 172.21.15.43:0/939590147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6254101760 0x7f62541041f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.451+0000 7f6253fff640 1 -- 172.21.15.43:0/939590147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6254101760 msgr2=0x7f62541041f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.451+0000 7f6253fff640 1 --2- 172.21.15.43:0/939590147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6254101760 0x7f62541041f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.451+0000 7f6253fff640 1 -- 172.21.15.43:0/939590147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62541050b0 msgr2=0x7f6254102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.451+0000 7f6253fff640 1 --2- 172.21.15.43:0/939590147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62541050b0 0x7f6254102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.451+0000 7f6253fff640 1 -- 172.21.15.43:0/939590147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6254103570 con 0x7f6254102260 2024-04-25T10:31:57.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.451+0000 7f6258d20640 1 --2- 172.21.15.43:0/939590147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6254101760 0x7f62541041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:57.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.451+0000 7f6253fff640 1 --2- 172.21.15.43:0/939590147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6254102260 0x7f6254102900 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6240006010 tx=0x7f6240031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:57.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.451+0000 7f6251ffb640 1 -- 172.21.15.43:0/939590147 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624003f030 con 0x7f6254102260 2024-04-25T10:31:57.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.451+0000 7f6251ffb640 1 -- 172.21.15.43:0/939590147 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6240003aa0 con 0x7f6254102260 2024-04-25T10:31:57.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.451+0000 7f625af84640 1 -- 172.21.15.43:0/939590147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62541a4370 con 0x7f6254102260 2024-04-25T10:31:57.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.451+0000 7f6251ffb640 1 -- 172.21.15.43:0/939590147 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240035070 con 0x7f6254102260 2024-04-25T10:31:57.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.451+0000 7f625af84640 1 -- 172.21.15.43:0/939590147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62541a4770 con 0x7f6254102260 2024-04-25T10:31:57.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.452+0000 7f625af84640 1 -- 172.21.15.43:0/939590147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f625406a160 con 0x7f6254102260 2024-04-25T10:31:57.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.454+0000 7f6251ffb640 1 -- 172.21.15.43:0/939590147 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f6240042030 con 0x7f6254102260 2024-04-25T10:31:57.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.455+0000 7f6251ffb640 1 --2- 172.21.15.43:0/939590147 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f62440afc80 0x7f62440b2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:57.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.455+0000 7f6258d20640 1 --2- 172.21.15.43:0/939590147 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f62440afc80 0x7f62440b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:57.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.455+0000 7f6251ffb640 1 -- 172.21.15.43:0/939590147 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f62400ffa50 con 0x7f6254102260 2024-04-25T10:31:57.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.455+0000 7f6258d20640 1 --2- 172.21.15.43:0/939590147 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f62440afc80 0x7f62440b2190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f625406b2d0 tx=0x7f6248005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:57.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.458+0000 7f6251ffb640 1 -- 172.21.15.43:0/939590147 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62400c8480 con 0x7f6254102260 2024-04-25T10:31:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.644+0000 7f625af84640 1 -- 172.21.15.43:0/939590147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f62540637a0 con 0x7f6254102260 2024-04-25T10:31:57.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.644+0000 7f6251ffb640 1 -- 172.21.15.43:0/939590147 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f62400c7d80 con 0x7f6254102260 2024-04-25T10:31:57.646 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T10:31:57.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.646+0000 7f625af84640 1 -- 172.21.15.43:0/939590147 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f62440afc80 msgr2=0x7f62440b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:57.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.646+0000 7f625af84640 1 --2- 172.21.15.43:0/939590147 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f62440afc80 0x7f62440b2190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f625406b2d0 tx=0x7f6248005f90 comp rx=0 tx=0).stop 2024-04-25T10:31:57.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.646+0000 7f625af84640 1 -- 172.21.15.43:0/939590147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6254102260 msgr2=0x7f6254102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:57.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.646+0000 7f625af84640 1 --2- 172.21.15.43:0/939590147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6254102260 0x7f6254102900 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6240006010 tx=0x7f6240031a50 comp rx=0 tx=0).stop 2024-04-25T10:31:57.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.646+0000 7f625af84640 1 -- 172.21.15.43:0/939590147 shutdown_connections 2024-04-25T10:31:57.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.646+0000 7f625af84640 1 --2- 172.21.15.43:0/939590147 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f62440afc80 0x7f62440b2190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.646+0000 7f625af84640 1 --2- 172.21.15.43:0/939590147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62541050b0 0x7f6254102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.646+0000 7f625af84640 1 --2- 172.21.15.43:0/939590147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6254102260 0x7f6254102900 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.646+0000 7f625af84640 1 --2- 172.21.15.43:0/939590147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6254101760 0x7f62541041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.646+0000 7f625af84640 1 -- 172.21.15.43:0/939590147 >> 172.21.15.43:0/939590147 conn(0x7f62540fd380 msgr2=0x7f6254105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:57.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.646+0000 7f625af84640 1 -- 172.21.15.43:0/939590147 shutdown_connections 2024-04-25T10:31:57.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.646+0000 7f625af84640 1 -- 172.21.15.43:0/939590147 wait complete. 2024-04-25T10:31:57.660 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:31:57.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.772+0000 7fd9be3c7640 1 -- 172.21.15.43:0/1132827022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b81050b0 msgr2=0x7fd9b81054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:57.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.772+0000 7fd9be3c7640 1 --2- 172.21.15.43:0/1132827022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b81050b0 0x7fd9b81054c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8009900 tx=0x7fd9a802a410 comp rx=0 tx=0).stop 2024-04-25T10:31:57.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.772+0000 7fd9be3c7640 1 -- 172.21.15.43:0/1132827022 shutdown_connections 2024-04-25T10:31:57.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.772+0000 7fd9be3c7640 1 --2- 172.21.15.43:0/1132827022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9b8102260 0x7fd9b810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.772+0000 7fd9be3c7640 1 --2- 172.21.15.43:0/1132827022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9b8101760 0x7fd9b8101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.772+0000 7fd9be3c7640 1 --2- 172.21.15.43:0/1132827022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b81050b0 0x7fd9b81054c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.772+0000 7fd9be3c7640 1 -- 172.21.15.43:0/1132827022 >> 172.21.15.43:0/1132827022 conn(0x7fd9b80fd380 msgr2=0x7fd9b80ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:57.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.772+0000 7fd9be3c7640 1 -- 172.21.15.43:0/1132827022 shutdown_connections 2024-04-25T10:31:57.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.772+0000 7fd9be3c7640 1 -- 172.21.15.43:0/1132827022 wait complete. 2024-04-25T10:31:57.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.773+0000 7fd9be3c7640 1 Processor -- start 2024-04-25T10:31:57.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.773+0000 7fd9be3c7640 1 -- start start 2024-04-25T10:31:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.773+0000 7fd9be3c7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b8101760 0x7fd9b8104260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.773+0000 7fd9be3c7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9b8102260 0x7fd9b8102900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.773+0000 7fd9be3c7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9b81050b0 0x7fd9b8102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.773+0000 7fd9be3c7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b810f600 con 0x7fd9b8101760 2024-04-25T10:31:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.773+0000 7fd9be3c7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b810f480 con 0x7fd9b8102260 2024-04-25T10:31:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.773+0000 7fd9be3c7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b810f780 con 0x7fd9b81050b0 2024-04-25T10:31:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.773+0000 7fd9b7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b8101760 0x7fd9b8104260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.774+0000 7fd9b7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b8101760 0x7fd9b8104260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51062/0 (socket says 172.21.15.43:51062) 2024-04-25T10:31:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.774+0000 7fd9b7fff640 1 -- 172.21.15.43:0/796878543 learned_addr learned my addr 172.21.15.43:0/796878543 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.774+0000 7fd9b77fe640 1 --2- 172.21.15.43:0/796878543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9b8102260 0x7fd9b8102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.774+0000 7fd9bc964640 1 --2- 172.21.15.43:0/796878543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9b81050b0 0x7fd9b8102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.774+0000 7fd9b7fff640 1 -- 172.21.15.43:0/796878543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9b81050b0 msgr2=0x7fd9b8102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.774+0000 7fd9b7fff640 1 --2- 172.21.15.43:0/796878543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9b81050b0 0x7fd9b8102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.774+0000 7fd9b7fff640 1 -- 172.21.15.43:0/796878543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9b8102260 msgr2=0x7fd9b8102900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.774+0000 7fd9b7fff640 1 --2- 172.21.15.43:0/796878543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9b8102260 0x7fd9b8102900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.774+0000 7fd9b7fff640 1 -- 172.21.15.43:0/796878543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b81034e0 con 0x7fd9b8101760 2024-04-25T10:31:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.774+0000 7fd9b7fff640 1 --2- 172.21.15.43:0/796878543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b8101760 0x7fd9b8104260 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8006010 tx=0x7fd9a8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.774+0000 7fd9b57fa640 1 -- 172.21.15.43:0/796878543 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a803f030 con 0x7fd9b8101760 2024-04-25T10:31:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.774+0000 7fd9b57fa640 1 -- 172.21.15.43:0/796878543 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a8003aa0 con 0x7fd9b8101760 2024-04-25T10:31:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.774+0000 7fd9be3c7640 1 -- 172.21.15.43:0/796878543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b81a4320 con 0x7fd9b8101760 2024-04-25T10:31:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.774+0000 7fd9b57fa640 1 -- 172.21.15.43:0/796878543 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8035070 con 0x7fd9b8101760 2024-04-25T10:31:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.774+0000 7fd9be3c7640 1 -- 172.21.15.43:0/796878543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b81a46d0 con 0x7fd9b8101760 2024-04-25T10:31:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.775+0000 7fd9be3c7640 1 -- 172.21.15.43:0/796878543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9b8108a40 con 0x7fd9b8101760 2024-04-25T10:31:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.777+0000 7fd9b57fa640 1 -- 172.21.15.43:0/796878543 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fd9a8042030 con 0x7fd9b8101760 2024-04-25T10:31:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.777+0000 7fd9b57fa640 1 --2- 172.21.15.43:0/796878543 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd9a00afb80 0x7fd9a00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.778+0000 7fd9b77fe640 1 --2- 172.21.15.43:0/796878543 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd9a00afb80 0x7fd9a00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.778+0000 7fd9b57fa640 1 -- 172.21.15.43:0/796878543 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fd9a80ff700 con 0x7fd9b8101760 2024-04-25T10:31:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.778+0000 7fd9b77fe640 1 --2- 172.21.15.43:0/796878543 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd9a00afb80 0x7fd9a00b2090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8103c00 tx=0x7fd9ac00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:57.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.781+0000 7fd9b57fa640 1 -- 172.21.15.43:0/796878543 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd9a80c7b40 con 0x7fd9b8101760 2024-04-25T10:31:57.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.969+0000 7fd9be3c7640 1 -- 172.21.15.43:0/796878543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd9b80637a0 con 0x7fd9b8101760 2024-04-25T10:31:57.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.969+0000 7fd9b57fa640 1 -- 172.21.15.43:0/796878543 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7fd9a80d0210 con 0x7fd9b8101760 2024-04-25T10:31:57.972 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T10:31:57.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.971+0000 7fd9be3c7640 1 -- 172.21.15.43:0/796878543 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd9a00afb80 msgr2=0x7fd9a00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.971+0000 7fd9be3c7640 1 --2- 172.21.15.43:0/796878543 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd9a00afb80 0x7fd9a00b2090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8103c00 tx=0x7fd9ac00b390 comp rx=0 tx=0).stop 2024-04-25T10:31:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.971+0000 7fd9be3c7640 1 -- 172.21.15.43:0/796878543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b8101760 msgr2=0x7fd9b8104260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.971+0000 7fd9be3c7640 1 --2- 172.21.15.43:0/796878543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b8101760 0x7fd9b8104260 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8006010 tx=0x7fd9a8031a50 comp rx=0 tx=0).stop 2024-04-25T10:31:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.971+0000 7fd9be3c7640 1 -- 172.21.15.43:0/796878543 shutdown_connections 2024-04-25T10:31:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.971+0000 7fd9be3c7640 1 --2- 172.21.15.43:0/796878543 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd9a00afb80 0x7fd9a00b2090 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.971+0000 7fd9be3c7640 1 --2- 172.21.15.43:0/796878543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9b8102260 0x7fd9b8102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.971+0000 7fd9be3c7640 1 --2- 172.21.15.43:0/796878543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9b81050b0 0x7fd9b8102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.971+0000 7fd9be3c7640 1 --2- 172.21.15.43:0/796878543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b8101760 0x7fd9b8104260 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.971+0000 7fd9be3c7640 1 -- 172.21.15.43:0/796878543 >> 172.21.15.43:0/796878543 conn(0x7fd9b80fd380 msgr2=0x7fd9b8105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.971+0000 7fd9be3c7640 1 -- 172.21.15.43:0/796878543 shutdown_connections 2024-04-25T10:31:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:57.972+0000 7fd9be3c7640 1 -- 172.21.15.43:0/796878543 wait complete. 2024-04-25T10:31:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:57 smithi043 ceph-mon[29478]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T10:31:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:57 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3185208098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:31:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:57 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/831694763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:31:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:57 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/939590147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:31:57.985 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:31:58.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.097+0000 7f58db14d640 1 -- 172.21.15.43:0/906676490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58d4106f50 msgr2=0x7f58d4109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.097+0000 7f58db14d640 1 --2- 172.21.15.43:0/906676490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58d4106f50 0x7f58d4109370 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009900 tx=0x7f58c802a410 comp rx=0 tx=0).stop 2024-04-25T10:31:58.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.097+0000 7f58db14d640 1 -- 172.21.15.43:0/906676490 shutdown_connections 2024-04-25T10:31:58.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.097+0000 7f58db14d640 1 --2- 172.21.15.43:0/906676490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58d41098b0 0x7f58d410bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.097+0000 7f58db14d640 1 --2- 172.21.15.43:0/906676490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58d4106f50 0x7f58d4109370 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.097+0000 7f58db14d640 1 --2- 172.21.15.43:0/906676490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58d41045f0 0x7f58d4106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.097+0000 7f58db14d640 1 -- 172.21.15.43:0/906676490 >> 172.21.15.43:0/906676490 conn(0x7f58d40fdbf0 msgr2=0x7f58d41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:58.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.097+0000 7f58db14d640 1 -- 172.21.15.43:0/906676490 shutdown_connections 2024-04-25T10:31:58.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.097+0000 7f58db14d640 1 -- 172.21.15.43:0/906676490 wait complete. 2024-04-25T10:31:58.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.098+0000 7f58db14d640 1 Processor -- start 2024-04-25T10:31:58.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.098+0000 7f58db14d640 1 -- start start 2024-04-25T10:31:58.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.098+0000 7f58db14d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58d41045f0 0x7f58d419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:58.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.098+0000 7f58db14d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58d4106f50 0x7f58d419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:58.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.098+0000 7f58db14d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58d41098b0 0x7f58d41a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:58.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.098+0000 7f58db14d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f58d410f6a0 con 0x7f58d41098b0 2024-04-25T10:31:58.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.098+0000 7f58db14d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f58d410f520 con 0x7f58d4106f50 2024-04-25T10:31:58.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.098+0000 7f58db14d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f58d410f820 con 0x7f58d41045f0 2024-04-25T10:31:58.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.098+0000 7f58d96ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58d41098b0 0x7f58d41a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:58.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.098+0000 7f58d96ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58d41098b0 0x7f58d41a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51078/0 (socket says 172.21.15.43:51078) 2024-04-25T10:31:58.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.098+0000 7f58d96ea640 1 -- 172.21.15.43:0/1807969013 learned_addr learned my addr 172.21.15.43:0/1807969013 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:58.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.098+0000 7f58d3fff640 1 --2- 172.21.15.43:0/1807969013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58d4106f50 0x7f58d419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:58.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.098+0000 7f58d8ee9640 1 --2- 172.21.15.43:0/1807969013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58d41045f0 0x7f58d419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:58.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.099+0000 7f58d96ea640 1 -- 172.21.15.43:0/1807969013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58d41045f0 msgr2=0x7f58d419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.099+0000 7f58d96ea640 1 --2- 172.21.15.43:0/1807969013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58d41045f0 0x7f58d419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.099+0000 7f58d96ea640 1 -- 172.21.15.43:0/1807969013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58d4106f50 msgr2=0x7f58d419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.099+0000 7f58d96ea640 1 --2- 172.21.15.43:0/1807969013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58d4106f50 0x7f58d419a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.099+0000 7f58d96ea640 1 -- 172.21.15.43:0/1807969013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d41a1dc0 con 0x7f58d41098b0 2024-04-25T10:31:58.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.099+0000 7f58d96ea640 1 --2- 172.21.15.43:0/1807969013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58d41098b0 0x7f58d41a1750 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00b9e0 tx=0x7f58cc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:58.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.099+0000 7f58d1ffb640 1 -- 172.21.15.43:0/1807969013 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc019070 con 0x7f58d41098b0 2024-04-25T10:31:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.099+0000 7f58d1ffb640 1 -- 172.21.15.43:0/1807969013 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58cc014440 con 0x7f58d41098b0 2024-04-25T10:31:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.099+0000 7f58d1ffb640 1 -- 172.21.15.43:0/1807969013 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc0125a0 con 0x7f58d41098b0 2024-04-25T10:31:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.099+0000 7f58db14d640 1 -- 172.21.15.43:0/1807969013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d41a20b0 con 0x7f58d41098b0 2024-04-25T10:31:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.099+0000 7f58db14d640 1 -- 172.21.15.43:0/1807969013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d41a26d0 con 0x7f58d41098b0 2024-04-25T10:31:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.100+0000 7f58db14d640 1 -- 172.21.15.43:0/1807969013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d40ff710 con 0x7f58d41098b0 2024-04-25T10:31:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.101+0000 7f58d1ffb640 1 -- 172.21.15.43:0/1807969013 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f58cc01c070 con 0x7f58d41098b0 2024-04-25T10:31:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.102+0000 7f58d1ffb640 1 --2- 172.21.15.43:0/1807969013 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f58bc0afc00 0x7f58bc0b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.102+0000 7f58d8ee9640 1 --2- 172.21.15.43:0/1807969013 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f58bc0afc00 0x7f58bc0b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.102+0000 7f58d1ffb640 1 -- 172.21.15.43:0/1807969013 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f58cc0de6f0 con 0x7f58d41098b0 2024-04-25T10:31:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.103+0000 7f58d8ee9640 1 --2- 172.21.15.43:0/1807969013 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f58bc0afc00 0x7f58bc0b2110 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f58c4005fd0 tx=0x7f58c4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:58.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.106+0000 7f58d1ffb640 1 -- 172.21.15.43:0/1807969013 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f58cc0a6c40 con 0x7f58d41098b0 2024-04-25T10:31:58.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:57 smithi077 ceph-mon[32772]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T10:31:58.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:57 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3185208098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:31:58.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:57 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/831694763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:31:58.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:57 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/939590147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:31:58.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:57 smithi164 ceph-mon[32713]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T10:31:58.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:57 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3185208098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:31:58.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:57 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/831694763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:31:58.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:57 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/939590147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:31:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.292+0000 7f58db14d640 1 -- 172.21.15.43:0/1807969013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f58d40637a0 con 0x7f58d41098b0 2024-04-25T10:31:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.293+0000 7f58d1ffb640 1 -- 172.21.15.43:0/1807969013 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7f58cc0aaf40 con 0x7f58d41098b0 2024-04-25T10:31:58.295 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T10:31:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.295+0000 7f58db14d640 1 -- 172.21.15.43:0/1807969013 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f58bc0afc00 msgr2=0x7f58bc0b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.295+0000 7f58db14d640 1 --2- 172.21.15.43:0/1807969013 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f58bc0afc00 0x7f58bc0b2110 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f58c4005fd0 tx=0x7f58c4005850 comp rx=0 tx=0).stop 2024-04-25T10:31:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.295+0000 7f58db14d640 1 -- 172.21.15.43:0/1807969013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58d41098b0 msgr2=0x7f58d41a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.295+0000 7f58db14d640 1 --2- 172.21.15.43:0/1807969013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58d41098b0 0x7f58d41a1750 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00b9e0 tx=0x7f58cc00bea0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.295+0000 7f58db14d640 1 -- 172.21.15.43:0/1807969013 shutdown_connections 2024-04-25T10:31:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.295+0000 7f58db14d640 1 --2- 172.21.15.43:0/1807969013 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f58bc0afc00 0x7f58bc0b2110 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.295+0000 7f58db14d640 1 --2- 172.21.15.43:0/1807969013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58d4106f50 0x7f58d419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.295+0000 7f58db14d640 1 --2- 172.21.15.43:0/1807969013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58d41098b0 0x7f58d41a1750 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.295+0000 7f58db14d640 1 --2- 172.21.15.43:0/1807969013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58d41045f0 0x7f58d419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.295+0000 7f58db14d640 1 -- 172.21.15.43:0/1807969013 >> 172.21.15.43:0/1807969013 conn(0x7f58d40fdbf0 msgr2=0x7f58d41051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.295+0000 7f58db14d640 1 -- 172.21.15.43:0/1807969013 shutdown_connections 2024-04-25T10:31:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.295+0000 7f58db14d640 1 -- 172.21.15.43:0/1807969013 wait complete. 2024-04-25T10:31:58.308 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:31:58.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.417+0000 7fc74b7b2640 1 -- 172.21.15.43:0/4130896554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc744102f30 msgr2=0x7fc744103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.417+0000 7fc74b7b2640 1 --2- 172.21.15.43:0/4130896554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc744102f30 0x7fc744103340 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc740009900 tx=0x7fc74002a410 comp rx=0 tx=0).stop 2024-04-25T10:31:58.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.417+0000 7fc74b7b2640 1 -- 172.21.15.43:0/4130896554 shutdown_connections 2024-04-25T10:31:58.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.417+0000 7fc74b7b2640 1 --2- 172.21.15.43:0/4130896554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc744104450 0x7fc74410ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.417+0000 7fc74b7b2640 1 --2- 172.21.15.43:0/4130896554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc744103910 0x7fc744103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.417+0000 7fc74b7b2640 1 --2- 172.21.15.43:0/4130896554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc744102f30 0x7fc744103340 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.417+0000 7fc74b7b2640 1 -- 172.21.15.43:0/4130896554 >> 172.21.15.43:0/4130896554 conn(0x7fc7440fd330 msgr2=0x7fc7440ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:58.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.417+0000 7fc74b7b2640 1 -- 172.21.15.43:0/4130896554 shutdown_connections 2024-04-25T10:31:58.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.417+0000 7fc74b7b2640 1 -- 172.21.15.43:0/4130896554 wait complete. 2024-04-25T10:31:58.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.418+0000 7fc74b7b2640 1 Processor -- start 2024-04-25T10:31:58.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.418+0000 7fc74b7b2640 1 -- start start 2024-04-25T10:31:58.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.418+0000 7fc74b7b2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc744102f30 0x7fc74419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:58.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.418+0000 7fc74b7b2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc744103910 0x7fc74419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:58.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.418+0000 7fc74b7b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc744104450 0x7fc7441a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:58.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.418+0000 7fc74b7b2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc74410e4e0 con 0x7fc744104450 2024-04-25T10:31:58.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.418+0000 7fc74b7b2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc74410e360 con 0x7fc744103910 2024-04-25T10:31:58.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.418+0000 7fc74b7b2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc74410e660 con 0x7fc744102f30 2024-04-25T10:31:58.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.418+0000 7fc749d4f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc744104450 0x7fc7441a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:58.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.418+0000 7fc749d4f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc744104450 0x7fc7441a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51090/0 (socket says 172.21.15.43:51090) 2024-04-25T10:31:58.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.418+0000 7fc749d4f640 1 -- 172.21.15.43:0/3328250304 learned_addr learned my addr 172.21.15.43:0/3328250304 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:58.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.418+0000 7fc748d4d640 1 --2- 172.21.15.43:0/3328250304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc744103910 0x7fc74419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:58.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.419+0000 7fc74954e640 1 --2- 172.21.15.43:0/3328250304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc744102f30 0x7fc74419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:58.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.419+0000 7fc749d4f640 1 -- 172.21.15.43:0/3328250304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc744102f30 msgr2=0x7fc74419a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.419+0000 7fc749d4f640 1 --2- 172.21.15.43:0/3328250304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc744102f30 0x7fc74419a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.419+0000 7fc749d4f640 1 -- 172.21.15.43:0/3328250304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc744103910 msgr2=0x7fc74419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.419+0000 7fc749d4f640 1 --2- 172.21.15.43:0/3328250304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc744103910 0x7fc74419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.419+0000 7fc749d4f640 1 -- 172.21.15.43:0/3328250304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7441a1db0 con 0x7fc744104450 2024-04-25T10:31:58.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.419+0000 7fc74954e640 1 --2- 172.21.15.43:0/3328250304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc744102f30 0x7fc74419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:58.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.419+0000 7fc749d4f640 1 --2- 172.21.15.43:0/3328250304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc744104450 0x7fc7441a1740 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc73c00eaa0 tx=0x7fc73c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:58.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.419+0000 7fc73a7fc640 1 -- 172.21.15.43:0/3328250304 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c00cdb0 con 0x7fc744104450 2024-04-25T10:31:58.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.419+0000 7fc74b7b2640 1 -- 172.21.15.43:0/3328250304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7441a20a0 con 0x7fc744104450 2024-04-25T10:31:58.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.419+0000 7fc73a7fc640 1 -- 172.21.15.43:0/3328250304 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc73c005020 con 0x7fc744104450 2024-04-25T10:31:58.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.419+0000 7fc73a7fc640 1 -- 172.21.15.43:0/3328250304 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c0116d0 con 0x7fc744104450 2024-04-25T10:31:58.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.419+0000 7fc74b7b2640 1 -- 172.21.15.43:0/3328250304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7441a25f0 con 0x7fc744104450 2024-04-25T10:31:58.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.420+0000 7fc74b7b2640 1 -- 172.21.15.43:0/3328250304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7441a28b0 con 0x7fc744104450 2024-04-25T10:31:58.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.422+0000 7fc73a7fc640 1 -- 172.21.15.43:0/3328250304 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fc73c0051d0 con 0x7fc744104450 2024-04-25T10:31:58.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.422+0000 7fc73a7fc640 1 --2- 172.21.15.43:0/3328250304 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc7300afb80 0x7fc7300b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:58.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.422+0000 7fc74954e640 1 --2- 172.21.15.43:0/3328250304 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc7300afb80 0x7fc7300b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:58.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.422+0000 7fc73a7fc640 1 -- 172.21.15.43:0/3328250304 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fc73c0df360 con 0x7fc744104450 2024-04-25T10:31:58.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.423+0000 7fc74954e640 1 --2- 172.21.15.43:0/3328250304 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc7300afb80 0x7fc7300b2090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc740005f70 tx=0x7fc740000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:58.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.426+0000 7fc73a7fc640 1 -- 172.21.15.43:0/3328250304 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc73c016030 con 0x7fc744104450 2024-04-25T10:31:58.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.653+0000 7fc74b7b2640 1 -- 172.21.15.43:0/3328250304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7441a2a80 con 0x7fc744104450 2024-04-25T10:31:58.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.654+0000 7fc73a7fc640 1 -- 172.21.15.43:0/3328250304 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11869 (secure 0 0 0) 0x7fc73c0aee20 con 0x7fc744104450 2024-04-25T10:31:58.656 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:31:58.657 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:30:38.804706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2595248466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2595248466},{"type":"v1","addr":"172.21.15.77:6837","nonce":2595248466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:31:58.657 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-04-25T10:31:58.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.656+0000 7fc74b7b2640 1 -- 172.21.15.43:0/3328250304 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc7300afb80 msgr2=0x7fc7300b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.656+0000 7fc74b7b2640 1 --2- 172.21.15.43:0/3328250304 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc7300afb80 0x7fc7300b2090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc740005f70 tx=0x7fc740000bc0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.656+0000 7fc74b7b2640 1 -- 172.21.15.43:0/3328250304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc744104450 msgr2=0x7fc7441a1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.656+0000 7fc74b7b2640 1 --2- 172.21.15.43:0/3328250304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc744104450 0x7fc7441a1740 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc73c00eaa0 tx=0x7fc73c00ef60 comp rx=0 tx=0).stop 2024-04-25T10:31:58.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.656+0000 7fc74b7b2640 1 -- 172.21.15.43:0/3328250304 shutdown_connections 2024-04-25T10:31:58.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.656+0000 7fc74b7b2640 1 --2- 172.21.15.43:0/3328250304 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fc7300afb80 0x7fc7300b2090 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.656+0000 7fc74b7b2640 1 --2- 172.21.15.43:0/3328250304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc744103910 0x7fc74419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.656+0000 7fc74b7b2640 1 --2- 172.21.15.43:0/3328250304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc744104450 0x7fc7441a1740 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.656+0000 7fc74b7b2640 1 --2- 172.21.15.43:0/3328250304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc744102f30 0x7fc74419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.656+0000 7fc74b7b2640 1 -- 172.21.15.43:0/3328250304 >> 172.21.15.43:0/3328250304 conn(0x7fc7440fd330 msgr2=0x7fc744109e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:58.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.656+0000 7fc74b7b2640 1 -- 172.21.15.43:0/3328250304 shutdown_connections 2024-04-25T10:31:58.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.656+0000 7fc74b7b2640 1 -- 172.21.15.43:0/3328250304 wait complete. 2024-04-25T10:31:58.670 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:58.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.779+0000 7fb480d4d640 1 -- 172.21.15.43:0/1817767731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c108900 msgr2=0x7fb47c10ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.779+0000 7fb480d4d640 1 --2- 172.21.15.43:0/1817767731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c108900 0x7fb47c10ad20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb474009ca0 tx=0x7fb47402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.779+0000 7fb480d4d640 1 -- 172.21.15.43:0/1817767731 shutdown_connections 2024-04-25T10:31:58.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.779+0000 7fb480d4d640 1 --2- 172.21.15.43:0/1817767731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c108900 0x7fb47c10ad20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.779+0000 7fb480d4d640 1 --2- 172.21.15.43:0/1817767731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb47c103c00 0x7fb47c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.779+0000 7fb480d4d640 1 --2- 172.21.15.43:0/1817767731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb47c1029c0 0x7fb47c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.779+0000 7fb480d4d640 1 -- 172.21.15.43:0/1817767731 >> 172.21.15.43:0/1817767731 conn(0x7fb47c0fdbd0 msgr2=0x7fb47c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:58.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.779+0000 7fb480d4d640 1 -- 172.21.15.43:0/1817767731 shutdown_connections 2024-04-25T10:31:58.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.779+0000 7fb480d4d640 1 -- 172.21.15.43:0/1817767731 wait complete. 2024-04-25T10:31:58.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.780+0000 7fb480d4d640 1 Processor -- start 2024-04-25T10:31:58.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.780+0000 7fb480d4d640 1 -- start start 2024-04-25T10:31:58.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.780+0000 7fb480d4d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb47c1029c0 0x7fb47c1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:58.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.780+0000 7fb480d4d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb47c103c00 0x7fb47c199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:58.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.780+0000 7fb480d4d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c108900 0x7fb47c1a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:58.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.780+0000 7fb480d4d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c10e4f0 con 0x7fb47c108900 2024-04-25T10:31:58.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.780+0000 7fb480d4d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c10e370 con 0x7fb47c103c00 2024-04-25T10:31:58.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.780+0000 7fb480d4d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c10e670 con 0x7fb47c1029c0 2024-04-25T10:31:58.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.780+0000 7fb47ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c108900 0x7fb47c1a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:58.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.780+0000 7fb47ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c108900 0x7fb47c1a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52866/0 (socket says 172.21.15.43:52866) 2024-04-25T10:31:58.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.780+0000 7fb47ad9d640 1 -- 172.21.15.43:0/3846515863 learned_addr learned my addr 172.21.15.43:0/3846515863 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:58.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.780+0000 7fb479d9b640 1 --2- 172.21.15.43:0/3846515863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb47c103c00 0x7fb47c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:58.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.780+0000 7fb47a59c640 1 --2- 172.21.15.43:0/3846515863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb47c1029c0 0x7fb47c1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:58.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.780+0000 7fb47ad9d640 1 -- 172.21.15.43:0/3846515863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb47c1029c0 msgr2=0x7fb47c1990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.780+0000 7fb47ad9d640 1 --2- 172.21.15.43:0/3846515863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb47c1029c0 0x7fb47c1990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.780+0000 7fb47ad9d640 1 -- 172.21.15.43:0/3846515863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb47c103c00 msgr2=0x7fb47c199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.780+0000 7fb47ad9d640 1 --2- 172.21.15.43:0/3846515863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb47c103c00 0x7fb47c199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.780+0000 7fb47ad9d640 1 -- 172.21.15.43:0/3846515863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47c069e90 con 0x7fb47c108900 2024-04-25T10:31:58.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.780+0000 7fb479d9b640 1 --2- 172.21.15.43:0/3846515863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb47c103c00 0x7fb47c199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:58.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.780+0000 7fb47a59c640 1 --2- 172.21.15.43:0/3846515863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb47c1029c0 0x7fb47c1990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:58.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.781+0000 7fb47ad9d640 1 --2- 172.21.15.43:0/3846515863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c108900 0x7fb47c1a05a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb4740097b0 tx=0x7fb474031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:58.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.781+0000 7fb46b7fe640 1 -- 172.21.15.43:0/3846515863 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474003bf0 con 0x7fb47c108900 2024-04-25T10:31:58.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.781+0000 7fb480d4d640 1 -- 172.21.15.43:0/3846515863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47c06a120 con 0x7fb47c108900 2024-04-25T10:31:58.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.781+0000 7fb46b7fe640 1 -- 172.21.15.43:0/3846515863 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb474003d90 con 0x7fb47c108900 2024-04-25T10:31:58.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.781+0000 7fb46b7fe640 1 -- 172.21.15.43:0/3846515863 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47402fc30 con 0x7fb47c108900 2024-04-25T10:31:58.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.781+0000 7fb480d4d640 1 -- 172.21.15.43:0/3846515863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb47c06a6c0 con 0x7fb47c108900 2024-04-25T10:31:58.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.782+0000 7fb480d4d640 1 -- 172.21.15.43:0/3846515863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb47c107710 con 0x7fb47c108900 2024-04-25T10:31:58.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.783+0000 7fb46b7fe640 1 -- 172.21.15.43:0/3846515863 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fb474004040 con 0x7fb47c108900 2024-04-25T10:31:58.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.784+0000 7fb46b7fe640 1 --2- 172.21.15.43:0/3846515863 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb4640afb80 0x7fb4640b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:58.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.784+0000 7fb47a59c640 1 --2- 172.21.15.43:0/3846515863 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb4640afb80 0x7fb4640b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:58.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.784+0000 7fb46b7fe640 1 -- 172.21.15.43:0/3846515863 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fb4741005f0 con 0x7fb47c108900 2024-04-25T10:31:58.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.785+0000 7fb47a59c640 1 --2- 172.21.15.43:0/3846515863 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb4640afb80 0x7fb4640b2090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb47c1035c0 tx=0x7fb46c005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:58.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.788+0000 7fb46b7fe640 1 -- 172.21.15.43:0/3846515863 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb4740046a0 con 0x7fb47c108900 2024-04-25T10:31:58.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.976+0000 7fb480d4d640 1 -- 172.21.15.43:0/3846515863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb47c1a2e20 con 0x7fb47c108900 2024-04-25T10:31:58.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.977+0000 7fb46b7fe640 1 -- 172.21.15.43:0/3846515863 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22619 (secure 0 0 0) 0x7fb4740c9190 con 0x7fb47c108900 2024-04-25T10:31:58.979 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:31:58.979 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":79,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:31:33.928341+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:31.503414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:32.224334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:58.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.979+0000 7fb480d4d640 1 -- 172.21.15.43:0/3846515863 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb4640afb80 msgr2=0x7fb4640b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.979+0000 7fb480d4d640 1 --2- 172.21.15.43:0/3846515863 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb4640afb80 0x7fb4640b2090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb47c1035c0 tx=0x7fb46c005e70 comp rx=0 tx=0).stop 2024-04-25T10:31:58.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.979+0000 7fb480d4d640 1 -- 172.21.15.43:0/3846515863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c108900 msgr2=0x7fb47c1a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.979+0000 7fb480d4d640 1 --2- 172.21.15.43:0/3846515863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c108900 0x7fb47c1a05a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb4740097b0 tx=0x7fb474031840 comp rx=0 tx=0).stop 2024-04-25T10:31:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.979+0000 7fb480d4d640 1 -- 172.21.15.43:0/3846515863 shutdown_connections 2024-04-25T10:31:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.979+0000 7fb480d4d640 1 --2- 172.21.15.43:0/3846515863 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb4640afb80 0x7fb4640b2090 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.979+0000 7fb480d4d640 1 --2- 172.21.15.43:0/3846515863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb47c103c00 0x7fb47c199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.979+0000 7fb480d4d640 1 --2- 172.21.15.43:0/3846515863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c108900 0x7fb47c1a05a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.979+0000 7fb480d4d640 1 --2- 172.21.15.43:0/3846515863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb47c1029c0 0x7fb47c1990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.979+0000 7fb480d4d640 1 -- 172.21.15.43:0/3846515863 >> 172.21.15.43:0/3846515863 conn(0x7fb47c0fdbd0 msgr2=0x7fb47c1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.979+0000 7fb480d4d640 1 -- 172.21.15.43:0/3846515863 shutdown_connections 2024-04-25T10:31:58.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:58.979+0000 7fb480d4d640 1 -- 172.21.15.43:0/3846515863 wait complete. 2024-04-25T10:31:58.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:58 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/796878543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:31:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:58 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1807969013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:31:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:58 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3328250304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:31:58.993 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:31:59.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.102+0000 7f8e3cd06640 1 -- 172.21.15.43:0/3140311922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e38104d40 msgr2=0x7f8e38105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.102+0000 7f8e3cd06640 1 --2- 172.21.15.43:0/3140311922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e38104d40 0x7f8e38105150 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009900 tx=0x7f8e2c02a410 comp rx=0 tx=0).stop 2024-04-25T10:31:59.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.102+0000 7f8e3cd06640 1 -- 172.21.15.43:0/3140311922 shutdown_connections 2024-04-25T10:31:59.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.102+0000 7f8e3cd06640 1 --2- 172.21.15.43:0/3140311922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e38109a40 0x7f8e3810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.102+0000 7f8e3cd06640 1 --2- 172.21.15.43:0/3140311922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e38104d40 0x7f8e38105150 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.102+0000 7f8e3cd06640 1 --2- 172.21.15.43:0/3140311922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e38103b00 0x7f8e38104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.102+0000 7f8e3cd06640 1 -- 172.21.15.43:0/3140311922 >> 172.21.15.43:0/3140311922 conn(0x7f8e380fed10 msgr2=0x7f8e381011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.102+0000 7f8e3cd06640 1 -- 172.21.15.43:0/3140311922 shutdown_connections 2024-04-25T10:31:59.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.102+0000 7f8e3cd06640 1 -- 172.21.15.43:0/3140311922 wait complete. 2024-04-25T10:31:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.103+0000 7f8e3cd06640 1 Processor -- start 2024-04-25T10:31:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.103+0000 7f8e3cd06640 1 -- start start 2024-04-25T10:31:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.103+0000 7f8e3cd06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e38103b00 0x7f8e3819c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.103+0000 7f8e3cd06640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e38104d40 0x7f8e3819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.103+0000 7f8e3cd06640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e38109a40 0x7f8e381a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.103+0000 7f8e3cd06640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3810f620 con 0x7f8e38103b00 2024-04-25T10:31:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.103+0000 7f8e3cd06640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3810f4a0 con 0x7f8e38104d40 2024-04-25T10:31:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.103+0000 7f8e3cd06640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3810f7a0 con 0x7f8e38109a40 2024-04-25T10:31:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.103+0000 7f8e3659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e38103b00 0x7f8e3819c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.103+0000 7f8e3659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e38103b00 0x7f8e3819c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52886/0 (socket says 172.21.15.43:52886) 2024-04-25T10:31:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.103+0000 7f8e3659c640 1 -- 172.21.15.43:0/1225937164 learned_addr learned my addr 172.21.15.43:0/1225937164 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:59.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.104+0000 7f8e36d9d640 1 --2- 172.21.15.43:0/1225937164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e38109a40 0x7f8e381a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.104+0000 7f8e35d9b640 1 --2- 172.21.15.43:0/1225937164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e38104d40 0x7f8e3819c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.104+0000 7f8e3659c640 1 -- 172.21.15.43:0/1225937164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e38109a40 msgr2=0x7f8e381a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.104+0000 7f8e3659c640 1 --2- 172.21.15.43:0/1225937164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e38109a40 0x7f8e381a3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.104+0000 7f8e3659c640 1 -- 172.21.15.43:0/1225937164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e38104d40 msgr2=0x7f8e3819c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.104+0000 7f8e3659c640 1 --2- 172.21.15.43:0/1225937164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e38104d40 0x7f8e3819c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.104+0000 7f8e3659c640 1 -- 172.21.15.43:0/1225937164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e381a3fb0 con 0x7f8e38103b00 2024-04-25T10:31:59.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.104+0000 7f8e35d9b640 1 --2- 172.21.15.43:0/1225937164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e38104d40 0x7f8e3819c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:59.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.104+0000 7f8e3659c640 1 --2- 172.21.15.43:0/1225937164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e38103b00 0x7f8e3819c470 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8e28005d00 tx=0x7f8e2800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.104+0000 7f8e277fe640 1 -- 172.21.15.43:0/1225937164 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e28017070 con 0x7f8e38103b00 2024-04-25T10:31:59.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.104+0000 7f8e277fe640 1 -- 172.21.15.43:0/1225937164 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e28009c30 con 0x7f8e38103b00 2024-04-25T10:31:59.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.104+0000 7f8e3cd06640 1 -- 172.21.15.43:0/1225937164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e381a42a0 con 0x7f8e38103b00 2024-04-25T10:31:59.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.104+0000 7f8e277fe640 1 -- 172.21.15.43:0/1225937164 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2800f620 con 0x7f8e38103b00 2024-04-25T10:31:59.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.104+0000 7f8e3cd06640 1 -- 172.21.15.43:0/1225937164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e381a47a0 con 0x7f8e38103b00 2024-04-25T10:31:59.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.105+0000 7f8e3cd06640 1 -- 172.21.15.43:0/1225937164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e381a4a60 con 0x7f8e38103b00 2024-04-25T10:31:59.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.107+0000 7f8e277fe640 1 -- 172.21.15.43:0/1225937164 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f8e2801a070 con 0x7f8e38103b00 2024-04-25T10:31:59.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.107+0000 7f8e277fe640 1 --2- 172.21.15.43:0/1225937164 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f8e200afb80 0x7f8e200b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.107+0000 7f8e35d9b640 1 --2- 172.21.15.43:0/1225937164 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f8e200afb80 0x7f8e200b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.107+0000 7f8e277fe640 1 -- 172.21.15.43:0/1225937164 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f8e280de860 con 0x7f8e38103b00 2024-04-25T10:31:59.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.108+0000 7f8e35d9b640 1 --2- 172.21.15.43:0/1225937164 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f8e200afb80 0x7f8e200b2090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c005f70 tx=0x7f8e2c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.111+0000 7f8e277fe640 1 -- 172.21.15.43:0/1225937164 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e28009de0 con 0x7f8e38103b00 2024-04-25T10:31:59.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:58 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/796878543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:31:59.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:58 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1807969013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:31:59.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:58 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3328250304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:31:59.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:58 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/796878543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:31:59.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:58 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1807969013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:31:59.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:58 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3328250304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:31:59.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.336+0000 7f8e3cd06640 1 -- 172.21.15.43:0/1225937164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e381a4c30 con 0x7f8e38103b00 2024-04-25T10:31:59.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.337+0000 7f8e277fe640 1 -- 172.21.15.43:0/1225937164 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11869 (secure 0 0 0) 0x7f8e280a7290 con 0x7f8e38103b00 2024-04-25T10:31:59.339 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:31:59.339 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:30:38.804706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2595248466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2595248466},{"type":"v1","addr":"172.21.15.77:6837","nonce":2595248466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:31:59.339 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-04-25T10:31:59.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.339+0000 7f8e3cd06640 1 -- 172.21.15.43:0/1225937164 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f8e200afb80 msgr2=0x7f8e200b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.339+0000 7f8e3cd06640 1 --2- 172.21.15.43:0/1225937164 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f8e200afb80 0x7f8e200b2090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c005f70 tx=0x7f8e2c031400 comp rx=0 tx=0).stop 2024-04-25T10:31:59.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.339+0000 7f8e3cd06640 1 -- 172.21.15.43:0/1225937164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e38103b00 msgr2=0x7f8e3819c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.339+0000 7f8e3cd06640 1 --2- 172.21.15.43:0/1225937164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e38103b00 0x7f8e3819c470 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8e28005d00 tx=0x7f8e2800dde0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.339+0000 7f8e3cd06640 1 -- 172.21.15.43:0/1225937164 shutdown_connections 2024-04-25T10:31:59.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.339+0000 7f8e3cd06640 1 --2- 172.21.15.43:0/1225937164 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f8e200afb80 0x7f8e200b2090 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.339+0000 7f8e3cd06640 1 --2- 172.21.15.43:0/1225937164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e38104d40 0x7f8e3819c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.339+0000 7f8e3cd06640 1 --2- 172.21.15.43:0/1225937164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e38109a40 0x7f8e381a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.339+0000 7f8e3cd06640 1 --2- 172.21.15.43:0/1225937164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e38103b00 0x7f8e3819c470 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.339+0000 7f8e3cd06640 1 -- 172.21.15.43:0/1225937164 >> 172.21.15.43:0/1225937164 conn(0x7f8e380fed10 msgr2=0x7f8e380ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.339+0000 7f8e3cd06640 1 -- 172.21.15.43:0/1225937164 shutdown_connections 2024-04-25T10:31:59.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.339+0000 7f8e3cd06640 1 -- 172.21.15.43:0/1225937164 wait complete. 2024-04-25T10:31:59.353 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T10:30:33.261560+0000', 'modified': '2024-04-25T10:30:38.804706+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24554, 'mds_1': 24508, 'mds_2': 24472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24470': {'gid': 24470, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6835/1783762538', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 1783762538}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 1783762538}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.164:6835/1437008220', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 1437008220}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 1437008220}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6837/3305330947', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6836', 'nonce': 3305330947}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 3305330947}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6837/2595248466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6836', 'nonce': 2595248466}, {'type': 'v1', 'addr': '172.21.15.77:6837', 'nonce': 2595248466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24508': {'gid': 24508, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.164:6839/4082280107', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 4082280107}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 4082280107}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24554': {'gid': 24554, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.77:6839/172043982', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 172043982}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 172043982}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24554, 'qdb_cluster': [24472, 24508, 24554]} 2024-04-25T10:31:59.353 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T10:31:59.353 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-25T10:31:59.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.464+0000 7f2a4a4af640 1 -- 172.21.15.43:0/3755659451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a44103c20 msgr2=0x7f2a44104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.464+0000 7f2a4a4af640 1 --2- 172.21.15.43:0/3755659451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a44103c20 0x7f2a44104030 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009900 tx=0x7f2a3802a410 comp rx=0 tx=0).stop 2024-04-25T10:31:59.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.464+0000 7f2a4a4af640 1 -- 172.21.15.43:0/3755659451 shutdown_connections 2024-04-25T10:31:59.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.464+0000 7f2a4a4af640 1 --2- 172.21.15.43:0/3755659451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a44108920 0x7f2a4410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.464+0000 7f2a4a4af640 1 --2- 172.21.15.43:0/3755659451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a44103c20 0x7f2a44104030 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.464+0000 7f2a4a4af640 1 --2- 172.21.15.43:0/3755659451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a441029e0 0x7f2a441036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.464+0000 7f2a4a4af640 1 -- 172.21.15.43:0/3755659451 >> 172.21.15.43:0/3755659451 conn(0x7f2a440fdbd0 msgr2=0x7f2a44100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.464+0000 7f2a4a4af640 1 -- 172.21.15.43:0/3755659451 shutdown_connections 2024-04-25T10:31:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.464+0000 7f2a4a4af640 1 -- 172.21.15.43:0/3755659451 wait complete. 2024-04-25T10:31:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.464+0000 7f2a4a4af640 1 Processor -- start 2024-04-25T10:31:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.464+0000 7f2a4a4af640 1 -- start start 2024-04-25T10:31:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.465+0000 7f2a4a4af640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a441029e0 0x7f2a4419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.465+0000 7f2a4a4af640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a44103c20 0x7f2a4419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.465+0000 7f2a4a4af640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a44108920 0x7f2a441a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.465+0000 7f2a4a4af640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4410e510 con 0x7f2a44103c20 2024-04-25T10:31:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.465+0000 7f2a4a4af640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4410e390 con 0x7f2a44108920 2024-04-25T10:31:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.465+0000 7f2a4a4af640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4410e690 con 0x7f2a441029e0 2024-04-25T10:31:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.465+0000 7f2a437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a44103c20 0x7f2a4419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.465+0000 7f2a437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a44103c20 0x7f2a4419aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52910/0 (socket says 172.21.15.43:52910) 2024-04-25T10:31:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.465+0000 7f2a437fe640 1 -- 172.21.15.43:0/2563229556 learned_addr learned my addr 172.21.15.43:0/2563229556 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.465+0000 7f2a43fff640 1 --2- 172.21.15.43:0/2563229556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a441029e0 0x7f2a4419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.465+0000 7f2a48a4c640 1 --2- 172.21.15.43:0/2563229556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a44108920 0x7f2a441a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.465+0000 7f2a437fe640 1 -- 172.21.15.43:0/2563229556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a441029e0 msgr2=0x7f2a4419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.465+0000 7f2a437fe640 1 --2- 172.21.15.43:0/2563229556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a441029e0 0x7f2a4419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.465+0000 7f2a437fe640 1 -- 172.21.15.43:0/2563229556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a44108920 msgr2=0x7f2a441a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.465+0000 7f2a437fe640 1 --2- 172.21.15.43:0/2563229556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a44108920 0x7f2a441a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.465+0000 7f2a437fe640 1 -- 172.21.15.43:0/2563229556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a441a2070 con 0x7f2a44103c20 2024-04-25T10:31:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.465+0000 7f2a437fe640 1 --2- 172.21.15.43:0/2563229556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a44103c20 0x7f2a4419aa70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2a38006010 tx=0x7f2a380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.465+0000 7f2a417fa640 1 -- 172.21.15.43:0/2563229556 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3803f030 con 0x7f2a44103c20 2024-04-25T10:31:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.466+0000 7f2a4a4af640 1 -- 172.21.15.43:0/2563229556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a441a2300 con 0x7f2a44103c20 2024-04-25T10:31:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.466+0000 7f2a417fa640 1 -- 172.21.15.43:0/2563229556 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a38031910 con 0x7f2a44103c20 2024-04-25T10:31:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.466+0000 7f2a417fa640 1 -- 172.21.15.43:0/2563229556 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38035070 con 0x7f2a44103c20 2024-04-25T10:31:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.466+0000 7f2a4a4af640 1 -- 172.21.15.43:0/2563229556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a441a27f0 con 0x7f2a44103c20 2024-04-25T10:31:59.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.468+0000 7f2a417fa640 1 -- 172.21.15.43:0/2563229556 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f2a38042030 con 0x7f2a44103c20 2024-04-25T10:31:59.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.468+0000 7f2a51364740 1 Processor -- start 2024-04-25T10:31:59.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.468+0000 7f2a51364740 1 -- start start 2024-04-25T10:31:59.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.468+0000 7f2a51364740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee78270 0x55887f219370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.468+0000 7f2a51364740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55887ee4f3b0 0x55887f2198f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.468+0000 7f2a51364740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee57590 0x55887f2207f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.469+0000 7f2a51364740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55887f17e740 con 0x55887ee57590 2024-04-25T10:31:59.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.469+0000 7f2a51364740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55887f11a3a0 con 0x55887ee78270 2024-04-25T10:31:59.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.469+0000 7f2a51364740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55887f17a700 con 0x55887ee4f3b0 2024-04-25T10:31:59.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.469+0000 7f2a437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee57590 0x55887f2207f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.469+0000 7f2a437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee57590 0x55887f2207f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52924/0 (socket says 172.21.15.43:52924) 2024-04-25T10:31:59.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.469+0000 7f2a437fe640 1 -- 172.21.15.43:0/1582843077 learned_addr learned my addr 172.21.15.43:0/1582843077 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:59.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.469+0000 7f2a48a4c640 1 --2- 172.21.15.43:0/1582843077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee78270 0x55887f219370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.469+0000 7f2a43fff640 1 --2- 172.21.15.43:0/1582843077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55887ee4f3b0 0x55887f2198f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.469+0000 7f2a437fe640 1 -- 172.21.15.43:0/1582843077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55887ee4f3b0 msgr2=0x55887f2198f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.469+0000 7f2a437fe640 1 --2- 172.21.15.43:0/1582843077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55887ee4f3b0 0x55887f2198f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.469+0000 7f2a417fa640 1 --2- 172.21.15.43:0/2563229556 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2a2c0afb80 0x7f2a2c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.469+0000 7f2a437fe640 1 -- 172.21.15.43:0/1582843077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee78270 msgr2=0x55887f219370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.469+0000 7f2a437fe640 1 --2- 172.21.15.43:0/1582843077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee78270 0x55887f219370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.469+0000 7f2a437fe640 1 -- 172.21.15.43:0/1582843077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55887f136750 con 0x55887ee57590 2024-04-25T10:31:59.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.469+0000 7f2a48a4c640 1 --2- 172.21.15.43:0/1582843077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee78270 0x55887f219370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:59.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.469+0000 7f2a48a4c640 1 --2- 172.21.15.43:0/2563229556 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2a2c0afb80 0x7f2a2c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.469+0000 7f2a437fe640 1 --2- 172.21.15.43:0/1582843077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee57590 0x55887f2207f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2a380fcc40 tx=0x7f2a380c3de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cad53341bd0f8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.469+0000 7f2a417fa640 1 -- 172.21.15.43:0/2563229556 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f2a380ff340 con 0x7f2a44103c20 2024-04-25T10:31:59.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.469+0000 7f2a2a7fc640 1 -- 172.21.15.43:0/1582843077 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a380c74a0 con 0x55887ee57590 2024-04-25T10:31:59.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.470+0000 7f2a2a7fc640 1 -- 172.21.15.43:0/1582843077 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a380c71e0 con 0x55887ee57590 2024-04-25T10:31:59.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.470+0000 7f2a2a7fc640 1 -- 172.21.15.43:0/1582843077 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a380044e0 con 0x55887ee57590 2024-04-25T10:31:59.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.470+0000 7f2a48a4c640 1 --2- 172.21.15.43:0/2563229556 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2a2c0afb80 0x7f2a2c0b2090 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55887f21a540 tx=0x7f2a3c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.470+0000 7f2a51364740 1 -- 172.21.15.43:0/1582843077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee57590 msgr2=0x55887f2207f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.470+0000 7f2a51364740 1 --2- 172.21.15.43:0/1582843077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee57590 0x55887f2207f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2a380fcc40 tx=0x7f2a380c3de0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.470+0000 7f2a51364740 1 -- 172.21.15.43:0/1582843077 shutdown_connections 2024-04-25T10:31:59.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.470+0000 7f2a51364740 1 --2- 172.21.15.43:0/1582843077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee57590 0x55887f2207f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.470+0000 7f2a51364740 1 --2- 172.21.15.43:0/1582843077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55887ee4f3b0 0x55887f2198f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.470+0000 7f2a51364740 1 --2- 172.21.15.43:0/1582843077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee78270 0x55887f219370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.470+0000 7f2a51364740 1 -- 172.21.15.43:0/1582843077 >> 172.21.15.43:0/1582843077 conn(0x55887ee6c3c0 msgr2=0x55887f214d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.470+0000 7f2a51364740 1 -- 172.21.15.43:0/1582843077 shutdown_connections 2024-04-25T10:31:59.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.470+0000 7f2a51364740 1 -- 172.21.15.43:0/1582843077 wait complete. 2024-04-25T10:31:59.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.471+0000 7f2a51364740 1 -- start start 2024-04-25T10:31:59.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.471+0000 7f2a51364740 1 Processor -- start 2024-04-25T10:31:59.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.471+0000 7f2a51364740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee4f3b0 0x55887f222f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.471+0000 7f2a51364740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55887ee57590 0x55887f2234d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.471+0000 7f2a51364740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee78270 0x55887f22a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.471+0000 7f2a51364740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55887f221000 con 0x55887ee4f3b0 2024-04-25T10:31:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.471+0000 7f2a51364740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55887f105fe0 con 0x55887ee78270 2024-04-25T10:31:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.471+0000 7f2a51364740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55887f221180 con 0x55887ee57590 2024-04-25T10:31:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.471+0000 7f2a51364740 10 client.0 _ll_register_callbacks cb 0x55887f11c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.471+0000 7f2a43fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee4f3b0 0x55887f222f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.471+0000 7f2a43fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee4f3b0 0x55887f222f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52926/0 (socket says 172.21.15.43:52926) 2024-04-25T10:31:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.471+0000 7f2a43fff640 1 -- 172.21.15.43:0/4013870809 learned_addr learned my addr 172.21.15.43:0/4013870809 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.472+0000 7f2a437fe640 1 --2- 172.21.15.43:0/4013870809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55887ee57590 0x55887f2234d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.472+0000 7f2a48a4c640 1 --2- 172.21.15.43:0/4013870809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee78270 0x55887f22a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.472+0000 7f2a43fff640 1 -- 172.21.15.43:0/4013870809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55887ee57590 msgr2=0x55887f2234d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.472+0000 7f2a43fff640 1 --2- 172.21.15.43:0/4013870809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55887ee57590 0x55887f2234d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.472+0000 7f2a43fff640 1 -- 172.21.15.43:0/4013870809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee78270 msgr2=0x55887f22a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.472+0000 7f2a43fff640 1 --2- 172.21.15.43:0/4013870809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee78270 0x55887f22a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.472+0000 7f2a43fff640 1 -- 172.21.15.43:0/4013870809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55887ee902d0 con 0x55887ee4f3b0 2024-04-25T10:31:59.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.472+0000 7f2a43fff640 1 --2- 172.21.15.43:0/4013870809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee4f3b0 0x55887f222f90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2a34012700 tx=0x7f2a34012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225b9eddd0c7eb11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.472+0000 7f2a237fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:31:59.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.472+0000 7f2a237fe640 1 -- 172.21.15.43:0/4013870809 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a34004050 con 0x55887ee4f3b0 2024-04-25T10:31:59.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.472+0000 7f2a237fe640 1 -- 172.21.15.43:0/4013870809 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f2a340041f0 con 0x55887ee4f3b0 2024-04-25T10:31:59.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.472+0000 7f2a51364740 1 -- 172.21.15.43:0/4013870809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a380c71e0 con 0x55887ee4f3b0 2024-04-25T10:31:59.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.472+0000 7f2a237fe640 1 -- 172.21.15.43:0/4013870809 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2a340043f0 con 0x55887ee4f3b0 2024-04-25T10:31:59.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.472+0000 7f2a51364740 10 client.15063 fetch_fsmap learned FSMap version 23 2024-04-25T10:31:59.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.472+0000 7f2a51364740 1 -- 172.21.15.43:0/4013870809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55887ef5a6b0 con 0x55887ee4f3b0 2024-04-25T10:31:59.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.473+0000 7f2a237fe640 1 -- 172.21.15.43:0/4013870809 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a380c71e0 con 0x55887ee4f3b0 2024-04-25T10:31:59.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.473+0000 7f2a237fe640 1 -- 172.21.15.43:0/4013870809 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f2a34003ad0 con 0x55887ee4f3b0 2024-04-25T10:31:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.473+0000 7f2a51364740 10 client.15063 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:31:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.473+0000 7f2a51364740 10 client.15063 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T10:31:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.473+0000 7f2a51364740 20 client.15063 populate_metadata read hostname 'smithi043' 2024-04-25T10:31:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.473+0000 7f2a51364740 1 --2- 172.21.15.43:0/4013870809 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55887ee36e40 0x55887f230d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.473+0000 7f2a51364740 4 client.15063 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:31:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.473+0000 7f2a51364740 1 -- 172.21.15.43:0/4013870809 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55887f18c350 con 0x55887ee36e40 2024-04-25T10:31:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.473+0000 7f2a437fe640 1 --2- 172.21.15.43:0/4013870809 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55887ee36e40 0x55887f230d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.474+0000 7f2a237fe640 10 client.15063 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:31:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.474+0000 7f2a437fe640 1 --2- 172.21.15.43:0/4013870809 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55887ee36e40 0x55887f230d50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ad1921d7e59b230 server_cookie=7beac17a72df6443 in_seq=0 out_seq=0 2024-04-25T10:31:59.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.474+0000 7f2a237fe640 1 -- 172.21.15.43:0/4013870809 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55887f18c350 con 0x55887ee36e40 2024-04-25T10:31:59.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.474+0000 7f2a237fe640 10 client.15063 handle_command_reply: tid=0 2024-04-25T10:31:59.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.474+0000 7f2a51364740 1 client.15063 shutdown 2024-04-25T10:31:59.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.475+0000 7f2a51364740 1 -- 172.21.15.43:0/4013870809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee4f3b0 msgr2=0x55887f222f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.475+0000 7f2a51364740 1 --2- 172.21.15.43:0/4013870809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee4f3b0 0x55887f222f90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2a34012700 tx=0x7f2a34012bc0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.475+0000 7f2a51364740 1 -- 172.21.15.43:0/4013870809 shutdown_connections 2024-04-25T10:31:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.475+0000 7f2a51364740 1 --2- 172.21.15.43:0/4013870809 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55887ee36e40 0x55887f230d50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.475+0000 7f2a51364740 1 --2- 172.21.15.43:0/4013870809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee78270 0x55887f22a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.475+0000 7f2a51364740 1 --2- 172.21.15.43:0/4013870809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55887ee57590 0x55887f2234d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.475+0000 7f2a51364740 1 --2- 172.21.15.43:0/4013870809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee4f3b0 0x55887f222f90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.475+0000 7f2a237fe640 0 client.15063 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:31:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.475+0000 7f2a51364740 1 -- 172.21.15.43:0/4013870809 >> 172.21.15.43:0/4013870809 conn(0x55887ee6c3c0 msgr2=0x55887f213d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.475+0000 7f2a51364740 1 -- 172.21.15.43:0/4013870809 shutdown_connections 2024-04-25T10:31:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.475+0000 7f2a51364740 1 -- 172.21.15.43:0/4013870809 wait complete. 2024-04-25T10:31:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.475+0000 7f2a51364740 20 client.15063 trim_cache size 0 max 16384 2024-04-25T10:31:59.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.493+0000 7f2a51364740 1 Processor -- start 2024-04-25T10:31:59.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.493+0000 7f2a51364740 1 -- start start 2024-04-25T10:31:59.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a51364740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55887ee4f3b0 0x55887f20fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a51364740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee57590 0x55887f22a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a51364740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee78270 0x55887f2146b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a51364740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55887f22f090 con 0x55887ee57590 2024-04-25T10:31:59.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a51364740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55887f22f760 con 0x55887ee78270 2024-04-25T10:31:59.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a51364740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55887f22fe30 con 0x55887ee4f3b0 2024-04-25T10:31:59.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee57590 0x55887f22a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee57590 0x55887f22a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52942/0 (socket says 172.21.15.43:52942) 2024-04-25T10:31:59.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a437fe640 1 -- 172.21.15.43:0/1753248202 learned_addr learned my addr 172.21.15.43:0/1753248202 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:59.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a43fff640 1 --2- 172.21.15.43:0/1753248202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55887ee4f3b0 0x55887f20fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a48a4c640 1 --2- 172.21.15.43:0/1753248202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee78270 0x55887f2146b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a437fe640 1 -- 172.21.15.43:0/1753248202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55887ee4f3b0 msgr2=0x55887f20fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a437fe640 1 --2- 172.21.15.43:0/1753248202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55887ee4f3b0 0x55887f20fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a437fe640 1 -- 172.21.15.43:0/1753248202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee78270 msgr2=0x55887f2146b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a437fe640 1 --2- 172.21.15.43:0/1753248202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee78270 0x55887f2146b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a437fe640 1 -- 172.21.15.43:0/1753248202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55887f21e0f0 con 0x55887ee57590 2024-04-25T10:31:59.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a48a4c640 1 --2- 172.21.15.43:0/1753248202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee78270 0x55887f2146b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:59.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a437fe640 1 --2- 172.21.15.43:0/1753248202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee57590 0x55887f22a350 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2a380fcc40 tx=0x7f2a3803b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bafb094c85e85b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a29ffb640 1 -- 172.21.15.43:0/1753248202 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38049040 con 0x55887ee57590 2024-04-25T10:31:59.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a29ffb640 1 -- 172.21.15.43:0/1753248202 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a38102c60 con 0x55887ee57590 2024-04-25T10:31:59.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.494+0000 7f2a29ffb640 1 -- 172.21.15.43:0/1753248202 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a380c9a10 con 0x55887ee57590 2024-04-25T10:31:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.495+0000 7f2a51364740 1 -- 172.21.15.43:0/1753248202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee57590 msgr2=0x55887f22a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.495+0000 7f2a51364740 1 --2- 172.21.15.43:0/1753248202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee57590 0x55887f22a350 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2a380fcc40 tx=0x7f2a3803b460 comp rx=0 tx=0).stop 2024-04-25T10:31:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.495+0000 7f2a51364740 1 -- 172.21.15.43:0/1753248202 shutdown_connections 2024-04-25T10:31:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.495+0000 7f2a51364740 1 --2- 172.21.15.43:0/1753248202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee78270 0x55887f2146b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.495+0000 7f2a51364740 1 --2- 172.21.15.43:0/1753248202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee57590 0x55887f22a350 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.495+0000 7f2a51364740 1 --2- 172.21.15.43:0/1753248202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55887ee4f3b0 0x55887f20fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.495+0000 7f2a51364740 1 -- 172.21.15.43:0/1753248202 >> 172.21.15.43:0/1753248202 conn(0x55887ee6c3c0 msgr2=0x55887f2215d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.495+0000 7f2a51364740 1 -- 172.21.15.43:0/1753248202 shutdown_connections 2024-04-25T10:31:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.495+0000 7f2a51364740 1 -- 172.21.15.43:0/1753248202 wait complete. 2024-04-25T10:31:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.495+0000 7f2a51364740 1 -- start start 2024-04-25T10:31:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.496+0000 7f2a51364740 1 Processor -- start 2024-04-25T10:31:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.496+0000 7f2a51364740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee4f3b0 0x55887f2256e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.496+0000 7f2a51364740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee57590 0x55887f225c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.496+0000 7f2a51364740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55887ee78270 0x55887f228b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.496+0000 7f2a51364740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55887f230bd0 con 0x55887ee4f3b0 2024-04-25T10:31:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.496+0000 7f2a51364740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55887f230500 con 0x55887ee57590 2024-04-25T10:31:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.496+0000 7f2a51364740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55887f2312a0 con 0x55887ee78270 2024-04-25T10:31:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.496+0000 7f2a51364740 10 client.0 _ll_register_callbacks cb 0x7f2a340097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.496+0000 7f2a43fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee4f3b0 0x55887f2256e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.496+0000 7f2a43fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee4f3b0 0x55887f2256e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52954/0 (socket says 172.21.15.43:52954) 2024-04-25T10:31:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.496+0000 7f2a43fff640 1 -- 172.21.15.43:0/1583132489 learned_addr learned my addr 172.21.15.43:0/1583132489 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.496+0000 7f2a48a4c640 1 --2- 172.21.15.43:0/1583132489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55887ee78270 0x55887f228b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.496+0000 7f2a437fe640 1 --2- 172.21.15.43:0/1583132489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee57590 0x55887f225c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.496+0000 7f2a43fff640 1 -- 172.21.15.43:0/1583132489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55887ee78270 msgr2=0x55887f228b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.496+0000 7f2a43fff640 1 --2- 172.21.15.43:0/1583132489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55887ee78270 0x55887f228b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.496+0000 7f2a43fff640 1 -- 172.21.15.43:0/1583132489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee57590 msgr2=0x55887f225c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.496+0000 7f2a43fff640 1 --2- 172.21.15.43:0/1583132489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee57590 0x55887f225c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.496+0000 7f2a43fff640 1 -- 172.21.15.43:0/1583132489 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55887f229120 con 0x55887ee4f3b0 2024-04-25T10:31:59.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.497+0000 7f2a43fff640 1 --2- 172.21.15.43:0/1583132489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee4f3b0 0x55887f2256e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2a34003ef0 tx=0x7f2a34009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339a7898765d9dd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.497+0000 7f2a237fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:31:59.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.497+0000 7f2a237fe640 1 -- 172.21.15.43:0/1583132489 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3400f920 con 0x55887ee4f3b0 2024-04-25T10:31:59.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.497+0000 7f2a237fe640 1 -- 172.21.15.43:0/1583132489 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f2a34018040 con 0x55887ee4f3b0 2024-04-25T10:31:59.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.497+0000 7f2a51364740 1 -- 172.21.15.43:0/1583132489 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a38102c60 con 0x55887ee4f3b0 2024-04-25T10:31:59.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.497+0000 7f2a237fe640 1 -- 172.21.15.43:0/1583132489 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2a3400fac0 con 0x55887ee4f3b0 2024-04-25T10:31:59.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.497+0000 7f2a51364740 10 client.15069 fetch_fsmap learned FSMap version 23 2024-04-25T10:31:59.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.497+0000 7f2a51364740 1 -- 172.21.15.43:0/1583132489 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55887f22a380 con 0x55887ee4f3b0 2024-04-25T10:31:59.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.497+0000 7f2a237fe640 1 -- 172.21.15.43:0/1583132489 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a38102c60 con 0x55887ee4f3b0 2024-04-25T10:31:59.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.497+0000 7f2a237fe640 1 -- 172.21.15.43:0/1583132489 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f2a34005040 con 0x55887ee4f3b0 2024-04-25T10:31:59.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.497+0000 7f2a51364740 10 client.15069 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:31:59.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.497+0000 7f2a51364740 10 client.15069 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T10:31:59.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.497+0000 7f2a51364740 20 client.15069 populate_metadata read hostname 'smithi043' 2024-04-25T10:31:59.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.497+0000 7f2a51364740 1 --2- 172.21.15.43:0/1583132489 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55887ee0c280 0x55887f29cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.497+0000 7f2a51364740 4 client.15069 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:31:59.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.497+0000 7f2a51364740 1 -- 172.21.15.43:0/1583132489 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "status"}) -- 0x55887f1861c0 con 0x55887ee0c280 2024-04-25T10:31:59.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.498+0000 7f2a437fe640 1 --2- 172.21.15.43:0/1583132489 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55887ee0c280 0x55887f29cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.498+0000 7f2a237fe640 10 client.15069 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:31:59.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.498+0000 7f2a437fe640 1 --2- 172.21.15.43:0/1583132489 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55887ee0c280 0x55887f29cf00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=575e81c61dbb3573 server_cookie=fc7874c29e955e7a in_seq=0 out_seq=0 2024-04-25T10:31:59.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.498+0000 7f2a237fe640 1 -- 172.21.15.43:0/1583132489 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55887f1861c0 con 0x55887ee0c280 2024-04-25T10:31:59.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.498+0000 7f2a237fe640 10 client.15069 handle_command_reply: tid=0 2024-04-25T10:31:59.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.498+0000 7f2a51364740 1 client.15069 shutdown 2024-04-25T10:31:59.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.498+0000 7f2a51364740 1 -- 172.21.15.43:0/1583132489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee4f3b0 msgr2=0x55887f2256e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.498+0000 7f2a51364740 1 --2- 172.21.15.43:0/1583132489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee4f3b0 0x55887f2256e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2a34003ef0 tx=0x7f2a34009bf0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.498+0000 7f2a51364740 1 -- 172.21.15.43:0/1583132489 shutdown_connections 2024-04-25T10:31:59.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.498+0000 7f2a51364740 1 --2- 172.21.15.43:0/1583132489 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55887ee0c280 0x55887f29cf00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.498+0000 7f2a51364740 1 --2- 172.21.15.43:0/1583132489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55887ee57590 0x55887f225c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.498+0000 7f2a51364740 1 --2- 172.21.15.43:0/1583132489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55887ee78270 0x55887f228b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.498+0000 7f2a51364740 1 --2- 172.21.15.43:0/1583132489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55887ee4f3b0 0x55887f2256e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.498+0000 7f2a237fe640 0 client.15069 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:31:59.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.498+0000 7f2a51364740 1 -- 172.21.15.43:0/1583132489 >> 172.21.15.43:0/1583132489 conn(0x55887ee6c3c0 msgr2=0x55887f2146b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.499+0000 7f2a51364740 1 -- 172.21.15.43:0/1583132489 shutdown_connections 2024-04-25T10:31:59.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.499+0000 7f2a51364740 1 -- 172.21.15.43:0/1583132489 wait complete. 2024-04-25T10:31:59.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.499+0000 7f2a51364740 20 client.15069 trim_cache size 0 max 16384 2024-04-25T10:31:59.512 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:31:59.512 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "94d0035c-02ed-11ef-bc93-c7b262605968", 2024-04-25T10:31:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 2, 2024-04-25T10:31:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24472, 2024-04-25T10:31:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-25T10:31:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-25T10:31:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-25T10:31:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 81.696717556999999, 2024-04-25T10:31:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 23, 2024-04-25T10:31:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 79, 2024-04-25T10:31:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 77, 2024-04-25T10:31:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 107.792053428 2024-04-25T10:31:59.513 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:31:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.499+0000 7f2a4a4af640 1 -- 172.21.15.43:0/2563229556 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2a2c0afb80 msgr2=0x7f2a2c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.499+0000 7f2a4a4af640 1 --2- 172.21.15.43:0/2563229556 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2a2c0afb80 0x7f2a2c0b2090 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55887f21a540 tx=0x7f2a3c004a40 comp rx=0 tx=0).stop 2024-04-25T10:31:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.499+0000 7f2a4a4af640 1 -- 172.21.15.43:0/2563229556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a44103c20 msgr2=0x7f2a4419aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.499+0000 7f2a4a4af640 1 --2- 172.21.15.43:0/2563229556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a44103c20 0x7f2a4419aa70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2a38006010 tx=0x7f2a380042f0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.499+0000 7f2a4a4af640 1 -- 172.21.15.43:0/2563229556 shutdown_connections 2024-04-25T10:31:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.499+0000 7f2a4a4af640 1 --2- 172.21.15.43:0/2563229556 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f2a2c0afb80 0x7f2a2c0b2090 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.499+0000 7f2a4a4af640 1 --2- 172.21.15.43:0/2563229556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a44108920 0x7f2a441a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.499+0000 7f2a4a4af640 1 --2- 172.21.15.43:0/2563229556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a44103c20 0x7f2a4419aa70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.499+0000 7f2a4a4af640 1 --2- 172.21.15.43:0/2563229556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a441029e0 0x7f2a4419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.499+0000 7f2a4a4af640 1 -- 172.21.15.43:0/2563229556 >> 172.21.15.43:0/2563229556 conn(0x7f2a440fdbd0 msgr2=0x7f2a44069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.499+0000 7f2a4a4af640 1 -- 172.21.15.43:0/2563229556 shutdown_connections 2024-04-25T10:31:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.499+0000 7f2a4a4af640 1 -- 172.21.15.43:0/2563229556 wait complete. 2024-04-25T10:31:59.516 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T10:31:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.626+0000 7fe1e54b6640 1 -- 172.21.15.43:0/3671490645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e0106f30 msgr2=0x7fe1e0109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.626+0000 7fe1e54b6640 1 --2- 172.21.15.43:0/3671490645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e0106f30 0x7fe1e0109350 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4009900 tx=0x7fe1d402a410 comp rx=0 tx=0).stop 2024-04-25T10:31:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.626+0000 7fe1e54b6640 1 -- 172.21.15.43:0/3671490645 shutdown_connections 2024-04-25T10:31:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.626+0000 7fe1e54b6640 1 --2- 172.21.15.43:0/3671490645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1e0109890 0x7fe1e010bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.627+0000 7fe1e54b6640 1 --2- 172.21.15.43:0/3671490645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e0106f30 0x7fe1e0109350 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.627+0000 7fe1e54b6640 1 --2- 172.21.15.43:0/3671490645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1e01045d0 0x7fe1e01069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.627+0000 7fe1e54b6640 1 -- 172.21.15.43:0/3671490645 >> 172.21.15.43:0/3671490645 conn(0x7fe1e00fdbd0 msgr2=0x7fe1e0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.627+0000 7fe1e54b6640 1 -- 172.21.15.43:0/3671490645 shutdown_connections 2024-04-25T10:31:59.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.627+0000 7fe1e54b6640 1 -- 172.21.15.43:0/3671490645 wait complete. 2024-04-25T10:31:59.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.627+0000 7fe1e54b6640 1 Processor -- start 2024-04-25T10:31:59.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.627+0000 7fe1e54b6640 1 -- start start 2024-04-25T10:31:59.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1e54b6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e01045d0 0x7fe1e019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1e54b6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1e0106f30 0x7fe1e019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1e54b6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1e0109890 0x7fe1e01a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1e54b6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e010f5f0 con 0x7fe1e01045d0 2024-04-25T10:31:59.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1e54b6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e010f470 con 0x7fe1e0106f30 2024-04-25T10:31:59.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1e54b6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e010f770 con 0x7fe1e0109890 2024-04-25T10:31:59.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1deffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e01045d0 0x7fe1e019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1deffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e01045d0 0x7fe1e019a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52980/0 (socket says 172.21.15.43:52980) 2024-04-25T10:31:59.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1deffd640 1 -- 172.21.15.43:0/3043404228 learned_addr learned my addr 172.21.15.43:0/3043404228 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:59.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1de7fc640 1 --2- 172.21.15.43:0/3043404228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1e0106f30 0x7fe1e019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1df7fe640 1 --2- 172.21.15.43:0/3043404228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1e0109890 0x7fe1e01a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1deffd640 1 -- 172.21.15.43:0/3043404228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1e0109890 msgr2=0x7fe1e01a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1deffd640 1 --2- 172.21.15.43:0/3043404228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1e0109890 0x7fe1e01a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1deffd640 1 -- 172.21.15.43:0/3043404228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1e0106f30 msgr2=0x7fe1e019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1deffd640 1 --2- 172.21.15.43:0/3043404228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1e0106f30 0x7fe1e019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1deffd640 1 -- 172.21.15.43:0/3043404228 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e01a20f0 con 0x7fe1e01045d0 2024-04-25T10:31:59.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1df7fe640 1 --2- 172.21.15.43:0/3043404228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1e0109890 0x7fe1e01a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:59.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1deffd640 1 --2- 172.21.15.43:0/3043404228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e01045d0 0x7fe1e019a5b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0005fd0 tx=0x7fe1d000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1c7fff640 1 -- 172.21.15.43:0/3043404228 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0017070 con 0x7fe1e01045d0 2024-04-25T10:31:59.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1e54b6640 1 -- 172.21.15.43:0/3043404228 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e01a23e0 con 0x7fe1e01045d0 2024-04-25T10:31:59.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1c7fff640 1 -- 172.21.15.43:0/3043404228 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1d0009c30 con 0x7fe1e01045d0 2024-04-25T10:31:59.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.628+0000 7fe1c7fff640 1 -- 172.21.15.43:0/3043404228 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0003d30 con 0x7fe1e01045d0 2024-04-25T10:31:59.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.629+0000 7fe1e54b6640 1 -- 172.21.15.43:0/3043404228 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e01a2980 con 0x7fe1e01045d0 2024-04-25T10:31:59.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.631+0000 7fe1ec36e740 1 Processor -- start 2024-04-25T10:31:59.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.631+0000 7fe1ec36e740 1 -- start start 2024-04-25T10:31:59.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.631+0000 7fe1c7fff640 1 -- 172.21.15.43:0/3043404228 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fe1d0003ed0 con 0x7fe1e01045d0 2024-04-25T10:31:59.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.631+0000 7fe1ec36e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652269c6270 0x565226d67370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.631+0000 7fe1ec36e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56522699d3b0 0x565226d678f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.631+0000 7fe1ec36e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652269a5590 0x565226d6e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.631+0000 7fe1ec36e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565226ccc740 con 0x56522699d3b0 2024-04-25T10:31:59.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.631+0000 7fe1ec36e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x565226c683a0 con 0x5652269c6270 2024-04-25T10:31:59.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.631+0000 7fe1ec36e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565226cc8700 con 0x5652269a5590 2024-04-25T10:31:59.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.631+0000 7fe1deffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56522699d3b0 0x565226d678f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.631+0000 7fe1deffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56522699d3b0 0x565226d678f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52988/0 (socket says 172.21.15.43:52988) 2024-04-25T10:31:59.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.631+0000 7fe1deffd640 1 -- 172.21.15.43:0/1910021268 learned_addr learned my addr 172.21.15.43:0/1910021268 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:59.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.631+0000 7fe1df7fe640 1 --2- 172.21.15.43:0/1910021268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652269c6270 0x565226d67370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.631+0000 7fe1de7fc640 1 --2- 172.21.15.43:0/1910021268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652269a5590 0x565226d6e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.632+0000 7fe1deffd640 1 -- 172.21.15.43:0/1910021268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652269a5590 msgr2=0x565226d6e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.632+0000 7fe1deffd640 1 --2- 172.21.15.43:0/1910021268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652269a5590 0x565226d6e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.632+0000 7fe1deffd640 1 -- 172.21.15.43:0/1910021268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652269c6270 msgr2=0x565226d67370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.632+0000 7fe1deffd640 1 --2- 172.21.15.43:0/1910021268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652269c6270 0x565226d67370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.632+0000 7fe1deffd640 1 -- 172.21.15.43:0/1910021268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565226c84750 con 0x56522699d3b0 2024-04-25T10:31:59.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.632+0000 7fe1c7fff640 1 --2- 172.21.15.43:0/3043404228 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fe1c80afb80 0x7fe1c80b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.632+0000 7fe1deffd640 1 --2- 172.21.15.43:0/1910021268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56522699d3b0 0x565226d678f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00dc3a0 tx=0x7fe1d00e7600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7da0ae038b36e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.632+0000 7fe1df7fe640 1 --2- 172.21.15.43:0/3043404228 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fe1c80afb80 0x7fe1c80b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.632+0000 7fe1c7fff640 1 -- 172.21.15.43:0/3043404228 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fe1d00deaa0 con 0x7fe1e01045d0 2024-04-25T10:31:59.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.632+0000 7fe1c57fa640 1 -- 172.21.15.43:0/1910021268 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d00a2cf0 con 0x56522699d3b0 2024-04-25T10:31:59.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.632+0000 7fe1c57fa640 1 -- 172.21.15.43:0/1910021268 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1d00a6940 con 0x56522699d3b0 2024-04-25T10:31:59.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.632+0000 7fe1c57fa640 1 -- 172.21.15.43:0/1910021268 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d00a65a0 con 0x56522699d3b0 2024-04-25T10:31:59.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.633+0000 7fe1ec36e740 1 -- 172.21.15.43:0/1910021268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56522699d3b0 msgr2=0x565226d678f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.633+0000 7fe1df7fe640 1 --2- 172.21.15.43:0/3043404228 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fe1c80afb80 0x7fe1c80b2090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x565226d68540 tx=0x7fe1d8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.633+0000 7fe1ec36e740 1 --2- 172.21.15.43:0/1910021268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56522699d3b0 0x565226d678f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00dc3a0 tx=0x7fe1d00e7600 comp rx=0 tx=0).stop 2024-04-25T10:31:59.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.633+0000 7fe1ec36e740 1 -- 172.21.15.43:0/1910021268 shutdown_connections 2024-04-25T10:31:59.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.633+0000 7fe1ec36e740 1 --2- 172.21.15.43:0/1910021268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652269a5590 0x565226d6e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.633+0000 7fe1ec36e740 1 --2- 172.21.15.43:0/1910021268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56522699d3b0 0x565226d678f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.633+0000 7fe1ec36e740 1 --2- 172.21.15.43:0/1910021268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652269c6270 0x565226d67370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.633+0000 7fe1ec36e740 1 -- 172.21.15.43:0/1910021268 >> 172.21.15.43:0/1910021268 conn(0x5652269ba3c0 msgr2=0x565226d62d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.633+0000 7fe1ec36e740 1 -- 172.21.15.43:0/1910021268 shutdown_connections 2024-04-25T10:31:59.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.633+0000 7fe1ec36e740 1 -- 172.21.15.43:0/1910021268 wait complete. 2024-04-25T10:31:59.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.633+0000 7fe1ec36e740 1 -- start start 2024-04-25T10:31:59.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.633+0000 7fe1ec36e740 1 Processor -- start 2024-04-25T10:31:59.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1ec36e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56522699d3b0 0x565226d71250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1ec36e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269a5590 0x565226d71790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1ec36e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652269c6270 0x565226d786f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1ec36e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565226d6f240 con 0x5652269a5590 2024-04-25T10:31:59.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1ec36e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x565226c53fe0 con 0x5652269c6270 2024-04-25T10:31:59.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1ec36e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565226d6f3c0 con 0x56522699d3b0 2024-04-25T10:31:59.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1ec36e740 10 client.0 _ll_register_callbacks cb 0x565226c6a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:59.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1de7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269a5590 0x565226d71790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1de7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269a5590 0x565226d71790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52990/0 (socket says 172.21.15.43:52990) 2024-04-25T10:31:59.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1de7fc640 1 -- 172.21.15.43:0/60196968 learned_addr learned my addr 172.21.15.43:0/60196968 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:59.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1deffd640 1 --2- 172.21.15.43:0/60196968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56522699d3b0 0x565226d71250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1df7fe640 1 --2- 172.21.15.43:0/60196968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652269c6270 0x565226d786f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1de7fc640 1 -- 172.21.15.43:0/60196968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56522699d3b0 msgr2=0x565226d71250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1de7fc640 1 --2- 172.21.15.43:0/60196968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56522699d3b0 0x565226d71250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1de7fc640 1 -- 172.21.15.43:0/60196968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652269c6270 msgr2=0x565226d786f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1de7fc640 1 --2- 172.21.15.43:0/60196968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652269c6270 0x565226d786f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1de7fc640 1 -- 172.21.15.43:0/60196968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652269de2d0 con 0x5652269a5590 2024-04-25T10:31:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1df7fe640 1 --2- 172.21.15.43:0/60196968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652269c6270 0x565226d786f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1de7fc640 1 --2- 172.21.15.43:0/60196968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269a5590 0x565226d71790 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe1d402a930 tx=0x7fe1d4031ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f9df81c39d9bea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:31:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.634+0000 7fe1c2ff5640 1 -- 172.21.15.43:0/60196968 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4035040 con 0x5652269a5590 2024-04-25T10:31:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.635+0000 7fe1c2ff5640 1 -- 172.21.15.43:0/60196968 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fe1d4004240 con 0x5652269a5590 2024-04-25T10:31:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.635+0000 7fe1ec36e740 1 -- 172.21.15.43:0/60196968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1d00a6940 con 0x5652269a5590 2024-04-25T10:31:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.635+0000 7fe1c2ff5640 1 -- 172.21.15.43:0/60196968 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe1d40318f0 con 0x5652269a5590 2024-04-25T10:31:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.635+0000 7fe1ec36e740 10 client.15081 fetch_fsmap learned FSMap version 23 2024-04-25T10:31:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.635+0000 7fe1ec36e740 1 -- 172.21.15.43:0/60196968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x565226aa86b0 con 0x5652269a5590 2024-04-25T10:31:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.635+0000 7fe1c2ff5640 1 -- 172.21.15.43:0/60196968 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1d00a6940 con 0x5652269a5590 2024-04-25T10:31:59.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.635+0000 7fe1c2ff5640 1 -- 172.21.15.43:0/60196968 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe1d4044020 con 0x5652269a5590 2024-04-25T10:31:59.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.635+0000 7fe1ec36e740 10 client.15081 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:31:59.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.635+0000 7fe1ec36e740 10 client.15081 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T10:31:59.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.635+0000 7fe1ec36e740 20 client.15081 populate_metadata read hostname 'smithi043' 2024-04-25T10:31:59.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.635+0000 7fe1ec36e740 1 --2- 172.21.15.43:0/60196968 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x565226984e40 0x565226d7efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.635+0000 7fe1ec36e740 4 client.15081 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:31:59.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.635+0000 7fe1ec36e740 1 -- 172.21.15.43:0/60196968 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565226cda350 con 0x565226984e40 2024-04-25T10:31:59.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.635+0000 7fe1deffd640 1 --2- 172.21.15.43:0/60196968 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x565226984e40 0x565226d7efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.636+0000 7fe1deffd640 1 --2- 172.21.15.43:0/60196968 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x565226984e40 0x565226d7efa0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f884a7c9b84439a server_cookie=bc16edea1a629369 in_seq=0 out_seq=0 2024-04-25T10:31:59.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.636+0000 7fe1c2ff5640 10 client.15081 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:31:59.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.637+0000 7fe1c2ff5640 1 -- 172.21.15.43:0/60196968 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565226cda350 con 0x565226984e40 2024-04-25T10:31:59.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.637+0000 7fe1c2ff5640 10 client.15081 handle_command_reply: tid=0 2024-04-25T10:31:59.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.637+0000 7fe1ec36e740 1 client.15081 shutdown 2024-04-25T10:31:59.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.637+0000 7fe1ec36e740 1 -- 172.21.15.43:0/60196968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269a5590 msgr2=0x565226d71790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.637+0000 7fe1ec36e740 1 --2- 172.21.15.43:0/60196968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269a5590 0x565226d71790 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe1d402a930 tx=0x7fe1d4031ef0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.637+0000 7fe1ec36e740 1 -- 172.21.15.43:0/60196968 shutdown_connections 2024-04-25T10:31:59.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.637+0000 7fe1ec36e740 1 --2- 172.21.15.43:0/60196968 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x565226984e40 0x565226d7efa0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.637+0000 7fe1ec36e740 1 --2- 172.21.15.43:0/60196968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652269c6270 0x565226d786f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.637+0000 7fe1ec36e740 1 --2- 172.21.15.43:0/60196968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269a5590 0x565226d71790 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.637+0000 7fe1c2ff5640 0 client.15081 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:31:59.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.637+0000 7fe1ec36e740 1 --2- 172.21.15.43:0/60196968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56522699d3b0 0x565226d71250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.637+0000 7fe1ec36e740 1 -- 172.21.15.43:0/60196968 >> 172.21.15.43:0/60196968 conn(0x5652269ba3c0 msgr2=0x565226d61d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.637+0000 7fe1ec36e740 1 -- 172.21.15.43:0/60196968 shutdown_connections 2024-04-25T10:31:59.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.637+0000 7fe1ec36e740 1 -- 172.21.15.43:0/60196968 wait complete. 2024-04-25T10:31:59.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.637+0000 7fe1ec36e740 20 client.15081 trim_cache size 0 max 16384 2024-04-25T10:31:59.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1ec36e740 1 Processor -- start 2024-04-25T10:31:59.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1ec36e740 1 -- start start 2024-04-25T10:31:59.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1ec36e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56522699d3b0 0x565226d6bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1ec36e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652269a5590 0x565226d78610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1ec36e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269c6270 0x565226d5da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1ec36e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565226d7d2e0 con 0x5652269c6270 2024-04-25T10:31:59.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1ec36e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x565226d7d9b0 con 0x5652269a5590 2024-04-25T10:31:59.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1ec36e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565226d7e080 con 0x56522699d3b0 2024-04-25T10:31:59.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1df7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269c6270 0x565226d5da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1df7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269c6270 0x565226d5da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52992/0 (socket says 172.21.15.43:52992) 2024-04-25T10:31:59.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1df7fe640 1 -- 172.21.15.43:0/1385553846 learned_addr learned my addr 172.21.15.43:0/1385553846 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:59.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1de7fc640 1 --2- 172.21.15.43:0/1385553846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652269a5590 0x565226d78610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1df7fe640 1 -- 172.21.15.43:0/1385553846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56522699d3b0 msgr2=0x565226d6bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1deffd640 1 --2- 172.21.15.43:0/1385553846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56522699d3b0 0x565226d6bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1df7fe640 1 --2- 172.21.15.43:0/1385553846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56522699d3b0 0x565226d6bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1df7fe640 1 -- 172.21.15.43:0/1385553846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652269a5590 msgr2=0x565226d78610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1df7fe640 1 --2- 172.21.15.43:0/1385553846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652269a5590 0x565226d78610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1df7fe640 1 -- 172.21.15.43:0/1385553846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565226d6ea50 con 0x5652269c6270 2024-04-25T10:31:59.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1de7fc640 1 --2- 172.21.15.43:0/1385553846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652269a5590 0x565226d78610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:59.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1deffd640 1 --2- 172.21.15.43:0/1385553846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56522699d3b0 0x565226d6bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:59.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.656+0000 7fe1df7fe640 1 --2- 172.21.15.43:0/1385553846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269c6270 0x565226d5da20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8010970 tx=0x7fe1d8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa28d05d32a4b9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.657+0000 7fe1c4ff9640 1 -- 172.21.15.43:0/1385553846 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d802d070 con 0x5652269c6270 2024-04-25T10:31:59.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.657+0000 7fe1c4ff9640 1 -- 172.21.15.43:0/1385553846 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1d8028410 con 0x5652269c6270 2024-04-25T10:31:59.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.657+0000 7fe1c4ff9640 1 -- 172.21.15.43:0/1385553846 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d801fe60 con 0x5652269c6270 2024-04-25T10:31:59.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.657+0000 7fe1ec36e740 1 -- 172.21.15.43:0/1385553846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269c6270 msgr2=0x565226d5da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.657+0000 7fe1ec36e740 1 --2- 172.21.15.43:0/1385553846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269c6270 0x565226d5da20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8010970 tx=0x7fe1d8016000 comp rx=0 tx=0).stop 2024-04-25T10:31:59.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.657+0000 7fe1ec36e740 1 -- 172.21.15.43:0/1385553846 shutdown_connections 2024-04-25T10:31:59.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.657+0000 7fe1ec36e740 1 --2- 172.21.15.43:0/1385553846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652269a5590 0x565226d78610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.657+0000 7fe1ec36e740 1 --2- 172.21.15.43:0/1385553846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269c6270 0x565226d5da20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.657+0000 7fe1ec36e740 1 --2- 172.21.15.43:0/1385553846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56522699d3b0 0x565226d6bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.657+0000 7fe1ec36e740 1 -- 172.21.15.43:0/1385553846 >> 172.21.15.43:0/1385553846 conn(0x5652269ba3c0 msgr2=0x565226d626b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.657+0000 7fe1ec36e740 1 -- 172.21.15.43:0/1385553846 shutdown_connections 2024-04-25T10:31:59.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.657+0000 7fe1ec36e740 1 -- 172.21.15.43:0/1385553846 wait complete. 2024-04-25T10:31:59.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.658+0000 7fe1ec36e740 1 -- start start 2024-04-25T10:31:59.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.658+0000 7fe1ec36e740 1 Processor -- start 2024-04-25T10:31:59.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.658+0000 7fe1ec36e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56522699d3b0 0x565226d73c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.658+0000 7fe1ec36e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269a5590 0x565226d74140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.658+0000 7fe1ec36e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652269c6270 0x565226d77080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.658+0000 7fe1ec36e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565226d7ee20 con 0x5652269a5590 2024-04-25T10:31:59.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.658+0000 7fe1ec36e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x565226d7e750 con 0x56522699d3b0 2024-04-25T10:31:59.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.658+0000 7fe1ec36e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565226d7f4f0 con 0x5652269c6270 2024-04-25T10:31:59.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.658+0000 7fe1ec36e740 10 client.0 _ll_register_callbacks cb 0x7fe1d4005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:59.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.658+0000 7fe1de7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269a5590 0x565226d74140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.658+0000 7fe1de7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269a5590 0x565226d74140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53000/0 (socket says 172.21.15.43:53000) 2024-04-25T10:31:59.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.658+0000 7fe1de7fc640 1 -- 172.21.15.43:0/4109723017 learned_addr learned my addr 172.21.15.43:0/4109723017 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:59.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.658+0000 7fe1df7fe640 1 --2- 172.21.15.43:0/4109723017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652269c6270 0x565226d77080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.658+0000 7fe1deffd640 1 --2- 172.21.15.43:0/4109723017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56522699d3b0 0x565226d73c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.658+0000 7fe1de7fc640 1 -- 172.21.15.43:0/4109723017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652269c6270 msgr2=0x565226d77080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.658+0000 7fe1de7fc640 1 --2- 172.21.15.43:0/4109723017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652269c6270 0x565226d77080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.658+0000 7fe1de7fc640 1 -- 172.21.15.43:0/4109723017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56522699d3b0 msgr2=0x565226d73c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.658+0000 7fe1de7fc640 1 --2- 172.21.15.43:0/4109723017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56522699d3b0 0x565226d73c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.658+0000 7fe1de7fc640 1 -- 172.21.15.43:0/4109723017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565226d77640 con 0x5652269a5590 2024-04-25T10:31:59.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.658+0000 7fe1deffd640 1 --2- 172.21.15.43:0/4109723017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56522699d3b0 0x565226d73c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:59.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.659+0000 7fe1de7fc640 1 --2- 172.21.15.43:0/4109723017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269a5590 0x565226d74140 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4031440 tx=0x7fe1d403b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a475bd33ca5a28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.659+0000 7fe1c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:31:59.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.659+0000 7fe1c2ff5640 1 -- 172.21.15.43:0/4109723017 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4035040 con 0x5652269a5590 2024-04-25T10:31:59.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.659+0000 7fe1c2ff5640 1 -- 172.21.15.43:0/4109723017 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fe1d403baf0 con 0x5652269a5590 2024-04-25T10:31:59.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.659+0000 7fe1ec36e740 1 -- 172.21.15.43:0/4109723017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1d8028410 con 0x5652269a5590 2024-04-25T10:31:59.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.659+0000 7fe1c2ff5640 1 -- 172.21.15.43:0/4109723017 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe1d4046440 con 0x5652269a5590 2024-04-25T10:31:59.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.659+0000 7fe1ec36e740 10 client.15087 fetch_fsmap learned FSMap version 23 2024-04-25T10:31:59.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.659+0000 7fe1ec36e740 1 -- 172.21.15.43:0/4109723017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x565226d788a0 con 0x5652269a5590 2024-04-25T10:31:59.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.659+0000 7fe1c2ff5640 1 -- 172.21.15.43:0/4109723017 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1d8028410 con 0x5652269a5590 2024-04-25T10:31:59.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.659+0000 7fe1c2ff5640 1 -- 172.21.15.43:0/4109723017 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe1d4044050 con 0x5652269a5590 2024-04-25T10:31:59.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.659+0000 7fe1ec36e740 10 client.15087 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:31:59.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.659+0000 7fe1ec36e740 10 client.15087 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T10:31:59.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.659+0000 7fe1ec36e740 20 client.15087 populate_metadata read hostname 'smithi043' 2024-04-25T10:31:59.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.659+0000 7fe1ec36e740 1 --2- 172.21.15.43:0/4109723017 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x56522695a280 0x565226deb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.659+0000 7fe1ec36e740 4 client.15087 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:31:59.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.659+0000 7fe1ec36e740 1 -- 172.21.15.43:0/4109723017 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "status"}) -- 0x565226cd41c0 con 0x56522695a280 2024-04-25T10:31:59.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.660+0000 7fe1deffd640 1 --2- 172.21.15.43:0/4109723017 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x56522695a280 0x565226deb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.660+0000 7fe1c2ff5640 10 client.15087 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:31:59.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.660+0000 7fe1deffd640 1 --2- 172.21.15.43:0/4109723017 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x56522695a280 0x565226deb3d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ca72f4d558ac122 server_cookie=eae8865ce3fc30fa in_seq=0 out_seq=0 2024-04-25T10:31:59.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.660+0000 7fe1c2ff5640 1 -- 172.21.15.43:0/4109723017 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x565226cd41c0 con 0x56522695a280 2024-04-25T10:31:59.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.660+0000 7fe1c2ff5640 10 client.15087 handle_command_reply: tid=0 2024-04-25T10:31:59.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.660+0000 7fe1ec36e740 1 client.15087 shutdown 2024-04-25T10:31:59.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.660+0000 7fe1ec36e740 1 -- 172.21.15.43:0/4109723017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269a5590 msgr2=0x565226d74140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.660+0000 7fe1ec36e740 1 --2- 172.21.15.43:0/4109723017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269a5590 0x565226d74140 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4031440 tx=0x7fe1d403b670 comp rx=0 tx=0).stop 2024-04-25T10:31:59.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.660+0000 7fe1ec36e740 1 -- 172.21.15.43:0/4109723017 shutdown_connections 2024-04-25T10:31:59.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.660+0000 7fe1ec36e740 1 --2- 172.21.15.43:0/4109723017 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x56522695a280 0x565226deb3d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.660+0000 7fe1ec36e740 1 --2- 172.21.15.43:0/4109723017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652269c6270 0x565226d77080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.660+0000 7fe1ec36e740 1 --2- 172.21.15.43:0/4109723017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652269a5590 0x565226d74140 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.660+0000 7fe1ec36e740 1 --2- 172.21.15.43:0/4109723017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56522699d3b0 0x565226d73c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.660+0000 7fe1ec36e740 1 -- 172.21.15.43:0/4109723017 >> 172.21.15.43:0/4109723017 conn(0x5652269ba3c0 msgr2=0x565226d65ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.660+0000 7fe1c2ff5640 0 client.15087 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:31:59.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.661+0000 7fe1ec36e740 1 -- 172.21.15.43:0/4109723017 shutdown_connections 2024-04-25T10:31:59.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.661+0000 7fe1ec36e740 1 -- 172.21.15.43:0/4109723017 wait complete. 2024-04-25T10:31:59.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.661+0000 7fe1ec36e740 20 client.15087 trim_cache size 0 max 16384 2024-04-25T10:31:59.670 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:31:59.670 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "94d0035c-02ed-11ef-bc93-c7b262605968", 2024-04-25T10:31:59.670 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 1, 2024-04-25T10:31:59.670 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24508, 2024-04-25T10:31:59.671 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-25T10:31:59.671 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-25T10:31:59.671 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-25T10:31:59.671 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 82.482227769000005, 2024-04-25T10:31:59.671 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 23, 2024-04-25T10:31:59.671 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 79, 2024-04-25T10:31:59.671 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 77, 2024-04-25T10:31:59.671 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 99.791080569000002 2024-04-25T10:31:59.671 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:31:59.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.661+0000 7fe1e54b6640 1 -- 172.21.15.43:0/3043404228 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fe1c80afb80 msgr2=0x7fe1c80b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.661+0000 7fe1e54b6640 1 --2- 172.21.15.43:0/3043404228 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fe1c80afb80 0x7fe1c80b2090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x565226d68540 tx=0x7fe1d8009500 comp rx=0 tx=0).stop 2024-04-25T10:31:59.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.661+0000 7fe1e54b6640 1 -- 172.21.15.43:0/3043404228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e01045d0 msgr2=0x7fe1e019a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.661+0000 7fe1e54b6640 1 --2- 172.21.15.43:0/3043404228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e01045d0 0x7fe1e019a5b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0005fd0 tx=0x7fe1d000dde0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.661+0000 7fe1e54b6640 1 -- 172.21.15.43:0/3043404228 shutdown_connections 2024-04-25T10:31:59.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.661+0000 7fe1e54b6640 1 --2- 172.21.15.43:0/3043404228 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fe1c80afb80 0x7fe1c80b2090 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.662+0000 7fe1e54b6640 1 --2- 172.21.15.43:0/3043404228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1e0106f30 0x7fe1e019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.662+0000 7fe1e54b6640 1 --2- 172.21.15.43:0/3043404228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1e0109890 0x7fe1e01a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.662+0000 7fe1e54b6640 1 --2- 172.21.15.43:0/3043404228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e01045d0 0x7fe1e019a5b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.662+0000 7fe1e54b6640 1 -- 172.21.15.43:0/3043404228 >> 172.21.15.43:0/3043404228 conn(0x7fe1e00fdbd0 msgr2=0x7fe1e0104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.662+0000 7fe1e54b6640 1 -- 172.21.15.43:0/3043404228 shutdown_connections 2024-04-25T10:31:59.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.662+0000 7fe1e54b6640 1 -- 172.21.15.43:0/3043404228 wait complete. 2024-04-25T10:31:59.675 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:31:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.785+0000 7efc4157a640 1 -- 172.21.15.43:0/2314357956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc3c1080d0 msgr2=0x7efc3c10a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.785+0000 7efc4157a640 1 --2- 172.21.15.43:0/2314357956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc3c1080d0 0x7efc3c10a4f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efc34009ca0 tx=0x7efc3402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.785+0000 7efc4157a640 1 -- 172.21.15.43:0/2314357956 shutdown_connections 2024-04-25T10:31:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.785+0000 7efc4157a640 1 --2- 172.21.15.43:0/2314357956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc3c1080d0 0x7efc3c10a4f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.785+0000 7efc4157a640 1 --2- 172.21.15.43:0/2314357956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc3c1033d0 0x7efc3c1037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.785+0000 7efc4157a640 1 --2- 172.21.15.43:0/2314357956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc3c102190 0x7efc3c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.785+0000 7efc4157a640 1 -- 172.21.15.43:0/2314357956 >> 172.21.15.43:0/2314357956 conn(0x7efc3c0fd380 msgr2=0x7efc3c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.786+0000 7efc4157a640 1 -- 172.21.15.43:0/2314357956 shutdown_connections 2024-04-25T10:31:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.786+0000 7efc4157a640 1 -- 172.21.15.43:0/2314357956 wait complete. 2024-04-25T10:31:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.786+0000 7efc4157a640 1 Processor -- start 2024-04-25T10:31:59.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.786+0000 7efc4157a640 1 -- start start 2024-04-25T10:31:59.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.787+0000 7efc4157a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc3c102190 0x7efc3c1990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.787+0000 7efc4157a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc3c1033d0 0x7efc3c1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.787+0000 7efc4157a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc3c1080d0 0x7efc3c1a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.787+0000 7efc4157a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c10e530 con 0x7efc3c1080d0 2024-04-25T10:31:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.787+0000 7efc4157a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c10e3b0 con 0x7efc3c102190 2024-04-25T10:31:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.787+0000 7efc4157a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c10e6b0 con 0x7efc3c1033d0 2024-04-25T10:31:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.787+0000 7efc3b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc3c1080d0 0x7efc3c1a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.787+0000 7efc3b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc3c1080d0 0x7efc3c1a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53016/0 (socket says 172.21.15.43:53016) 2024-04-25T10:31:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.787+0000 7efc3b7fe640 1 -- 172.21.15.43:0/1516896527 learned_addr learned my addr 172.21.15.43:0/1516896527 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.787+0000 7efc3a7fc640 1 --2- 172.21.15.43:0/1516896527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc3c1033d0 0x7efc3c1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.787+0000 7efc3affd640 1 --2- 172.21.15.43:0/1516896527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc3c102190 0x7efc3c1990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.787+0000 7efc3b7fe640 1 -- 172.21.15.43:0/1516896527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc3c1033d0 msgr2=0x7efc3c1995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.787+0000 7efc3b7fe640 1 --2- 172.21.15.43:0/1516896527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc3c1033d0 0x7efc3c1995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.787+0000 7efc3b7fe640 1 -- 172.21.15.43:0/1516896527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc3c102190 msgr2=0x7efc3c1990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.787+0000 7efc3b7fe640 1 --2- 172.21.15.43:0/1516896527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc3c102190 0x7efc3c1990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.787+0000 7efc3b7fe640 1 -- 172.21.15.43:0/1516896527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3c1a0bf0 con 0x7efc3c1080d0 2024-04-25T10:31:59.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.787+0000 7efc3b7fe640 1 --2- 172.21.15.43:0/1516896527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc3c1080d0 0x7efc3c1a0580 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efc340097b0 tx=0x7efc34003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.787+0000 7efc23fff640 1 -- 172.21.15.43:0/1516896527 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc34004240 con 0x7efc3c1080d0 2024-04-25T10:31:59.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.788+0000 7efc23fff640 1 -- 172.21.15.43:0/1516896527 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc3402f380 con 0x7efc3c1080d0 2024-04-25T10:31:59.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.788+0000 7efc4157a640 1 -- 172.21.15.43:0/1516896527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3c1a0e80 con 0x7efc3c1080d0 2024-04-25T10:31:59.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.788+0000 7efc23fff640 1 -- 172.21.15.43:0/1516896527 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3402fc30 con 0x7efc3c1080d0 2024-04-25T10:31:59.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.788+0000 7efc4157a640 1 -- 172.21.15.43:0/1516896527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc3c1a12e0 con 0x7efc3c1080d0 2024-04-25T10:31:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.790+0000 7efc48420740 1 Processor -- start 2024-04-25T10:31:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.790+0000 7efc23fff640 1 -- 172.21.15.43:0/1516896527 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7efc340043e0 con 0x7efc3c1080d0 2024-04-25T10:31:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.790+0000 7efc48420740 1 -- start start 2024-04-25T10:31:59.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.790+0000 7efc48420740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311ea5270 0x5633122479f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.790+0000 7efc48420740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311e7c3b0 0x5633122460c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.790+0000 7efc48420740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e84590 0x563312246660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.790+0000 7efc48420740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5633121ab740 con 0x563311ea5270 2024-04-25T10:31:59.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.790+0000 7efc48420740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5633121473a0 con 0x563311e7c3b0 2024-04-25T10:31:59.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.790+0000 7efc48420740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5633121a7700 con 0x563311e84590 2024-04-25T10:31:59.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.790+0000 7efc3a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311ea5270 0x5633122479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.790+0000 7efc3a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311ea5270 0x5633122479f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53018/0 (socket says 172.21.15.43:53018) 2024-04-25T10:31:59.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.790+0000 7efc3a7fc640 1 -- 172.21.15.43:0/1363182233 learned_addr learned my addr 172.21.15.43:0/1363182233 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:59.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.791+0000 7efc3affd640 1 --2- 172.21.15.43:0/1363182233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311e7c3b0 0x5633122460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.791+0000 7efc23fff640 1 --2- 172.21.15.43:0/1516896527 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7efc240afb80 0x7efc240b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.791+0000 7efc3a7fc640 1 -- 172.21.15.43:0/1363182233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e84590 msgr2=0x563312246660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.791+0000 7efc3a7fc640 1 --2- 172.21.15.43:0/1363182233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e84590 0x563312246660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.791+0000 7efc3a7fc640 1 -- 172.21.15.43:0/1363182233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311e7c3b0 msgr2=0x5633122460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.791+0000 7efc3a7fc640 1 --2- 172.21.15.43:0/1363182233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311e7c3b0 0x5633122460c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.791+0000 7efc3a7fc640 1 -- 172.21.15.43:0/1363182233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563312163750 con 0x563311ea5270 2024-04-25T10:31:59.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.791+0000 7efc3affd640 1 --2- 172.21.15.43:0/1363182233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311e7c3b0 0x5633122460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:59.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.791+0000 7efc3b7fe640 1 --2- 172.21.15.43:0/1516896527 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7efc240afb80 0x7efc240b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.791+0000 7efc3a7fc640 1 --2- 172.21.15.43:0/1363182233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311ea5270 0x5633122479f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efc3000bd40 tx=0x7efc300127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9689b0dbf98197c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.791+0000 7efc23fff640 1 -- 172.21.15.43:0/1516896527 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7efc34100370 con 0x7efc3c1080d0 2024-04-25T10:31:59.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.791+0000 7efc217fa640 1 -- 172.21.15.43:0/1363182233 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3001b470 con 0x563311ea5270 2024-04-25T10:31:59.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.791+0000 7efc217fa640 1 -- 172.21.15.43:0/1363182233 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc30016070 con 0x563311ea5270 2024-04-25T10:31:59.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.791+0000 7efc217fa640 1 -- 172.21.15.43:0/1363182233 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3000f620 con 0x563311ea5270 2024-04-25T10:31:59.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.792+0000 7efc48420740 1 -- 172.21.15.43:0/1363182233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311ea5270 msgr2=0x5633122479f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.792+0000 7efc3b7fe640 1 --2- 172.21.15.43:0/1516896527 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7efc240afb80 0x7efc240b2090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efc340c4800 tx=0x7efc34096030 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.792+0000 7efc48420740 1 --2- 172.21.15.43:0/1363182233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311ea5270 0x5633122479f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efc3000bd40 tx=0x7efc300127c0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.792+0000 7efc48420740 1 -- 172.21.15.43:0/1363182233 shutdown_connections 2024-04-25T10:31:59.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.792+0000 7efc48420740 1 --2- 172.21.15.43:0/1363182233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311e7c3b0 0x5633122460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.792+0000 7efc48420740 1 --2- 172.21.15.43:0/1363182233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e84590 0x563312246660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.792+0000 7efc48420740 1 --2- 172.21.15.43:0/1363182233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311ea5270 0x5633122479f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.792+0000 7efc48420740 1 -- 172.21.15.43:0/1363182233 >> 172.21.15.43:0/1363182233 conn(0x563311e993c0 msgr2=0x563312241d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.792+0000 7efc48420740 1 -- 172.21.15.43:0/1363182233 shutdown_connections 2024-04-25T10:31:59.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.792+0000 7efc48420740 1 -- 172.21.15.43:0/1363182233 wait complete. 2024-04-25T10:31:59.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.792+0000 7efc48420740 1 -- start start 2024-04-25T10:31:59.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc48420740 1 Processor -- start 2024-04-25T10:31:59.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc48420740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311e7c3b0 0x5633122546e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc48420740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e84590 0x563312254c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc48420740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311ea5270 0x56331225bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc48420740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5633122526d0 con 0x563311ea5270 2024-04-25T10:31:59.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc48420740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563312132fe0 con 0x563311e7c3b0 2024-04-25T10:31:59.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc48420740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563312252850 con 0x563311e84590 2024-04-25T10:31:59.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc48420740 10 client.0 _ll_register_callbacks cb 0x563312149150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:59.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc3a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311ea5270 0x56331225bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc3a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311ea5270 0x56331225bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53026/0 (socket says 172.21.15.43:53026) 2024-04-25T10:31:59.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc3a7fc640 1 -- 172.21.15.43:0/1713767868 learned_addr learned my addr 172.21.15.43:0/1713767868 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:59.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc3a7fc640 1 --2- 172.21.15.43:0/1713767868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311e7c3b0 0x5633122546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc3affd640 1 --2- 172.21.15.43:0/1713767868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e84590 0x563312254c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc3a7fc640 1 -- 172.21.15.43:0/1713767868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e84590 msgr2=0x563312254c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc3a7fc640 1 --2- 172.21.15.43:0/1713767868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e84590 0x563312254c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc3a7fc640 1 -- 172.21.15.43:0/1713767868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311e7c3b0 msgr2=0x5633122546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc3a7fc640 1 --2- 172.21.15.43:0/1713767868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311e7c3b0 0x5633122546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc3a7fc640 1 -- 172.21.15.43:0/1713767868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563311ebd2d0 con 0x563311ea5270 2024-04-25T10:31:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc3affd640 1 --2- 172.21.15.43:0/1713767868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e84590 0x563312254c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc3a7fc640 1 --2- 172.21.15.43:0/1713767868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311ea5270 0x56331225bb80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efc3000b470 tx=0x7efc30014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424f928662fc3759 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.793+0000 7efc1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:31:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.794+0000 7efc1eff5640 1 -- 172.21.15.43:0/1713767868 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3001f030 con 0x563311ea5270 2024-04-25T10:31:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.794+0000 7efc1eff5640 1 -- 172.21.15.43:0/1713767868 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7efc30020070 con 0x563311ea5270 2024-04-25T10:31:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.794+0000 7efc48420740 1 -- 172.21.15.43:0/1713767868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc30016070 con 0x563311ea5270 2024-04-25T10:31:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.794+0000 7efc1eff5640 1 -- 172.21.15.43:0/1713767868 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7efc30031050 con 0x563311ea5270 2024-04-25T10:31:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.794+0000 7efc48420740 10 client.15099 fetch_fsmap learned FSMap version 23 2024-04-25T10:31:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.794+0000 7efc48420740 1 -- 172.21.15.43:0/1713767868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x563311f876b0 con 0x563311ea5270 2024-04-25T10:31:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.794+0000 7efc1eff5640 1 -- 172.21.15.43:0/1713767868 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc30016070 con 0x563311ea5270 2024-04-25T10:31:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.794+0000 7efc1eff5640 1 -- 172.21.15.43:0/1713767868 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7efc300048f0 con 0x563311ea5270 2024-04-25T10:31:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.794+0000 7efc48420740 10 client.15099 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:31:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.794+0000 7efc48420740 10 client.15099 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:31:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.794+0000 7efc48420740 20 client.15099 populate_metadata read hostname 'smithi043' 2024-04-25T10:31:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.794+0000 7efc48420740 1 --2- 172.21.15.43:0/1713767868 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563311e63e40 0x563312261fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.794+0000 7efc48420740 4 client.15099 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:31:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.794+0000 7efc48420740 1 -- 172.21.15.43:0/1713767868 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633121b9350 con 0x563311e63e40 2024-04-25T10:31:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.795+0000 7efc3affd640 1 --2- 172.21.15.43:0/1713767868 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563311e63e40 0x563312261fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.795+0000 7efc3affd640 1 --2- 172.21.15.43:0/1713767868 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563311e63e40 0x563312261fc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=590dc14ec7e6afc8 server_cookie=67a3f46c5b799dc5 in_seq=0 out_seq=0 2024-04-25T10:31:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.795+0000 7efc1eff5640 10 client.15099 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:31:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.796+0000 7efc1eff5640 1 -- 172.21.15.43:0/1713767868 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5633121b9350 con 0x563311e63e40 2024-04-25T10:31:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.796+0000 7efc1eff5640 10 client.15099 handle_command_reply: tid=0 2024-04-25T10:31:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.796+0000 7efc48420740 1 client.15099 shutdown 2024-04-25T10:31:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.796+0000 7efc48420740 1 -- 172.21.15.43:0/1713767868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311ea5270 msgr2=0x56331225bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.796+0000 7efc48420740 1 --2- 172.21.15.43:0/1713767868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311ea5270 0x56331225bb80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efc3000b470 tx=0x7efc30014040 comp rx=0 tx=0).stop 2024-04-25T10:31:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.796+0000 7efc48420740 1 -- 172.21.15.43:0/1713767868 shutdown_connections 2024-04-25T10:31:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.796+0000 7efc48420740 1 --2- 172.21.15.43:0/1713767868 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563311e63e40 0x563312261fc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.796+0000 7efc48420740 1 --2- 172.21.15.43:0/1713767868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311ea5270 0x56331225bb80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.796+0000 7efc48420740 1 --2- 172.21.15.43:0/1713767868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e84590 0x563312254c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.796+0000 7efc48420740 1 --2- 172.21.15.43:0/1713767868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311e7c3b0 0x5633122546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.796+0000 7efc48420740 1 -- 172.21.15.43:0/1713767868 >> 172.21.15.43:0/1713767868 conn(0x563311e993c0 msgr2=0x563312240d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.796+0000 7efc1eff5640 0 client.15099 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:31:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.796+0000 7efc48420740 1 -- 172.21.15.43:0/1713767868 shutdown_connections 2024-04-25T10:31:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.796+0000 7efc48420740 1 -- 172.21.15.43:0/1713767868 wait complete. 2024-04-25T10:31:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.796+0000 7efc48420740 20 client.15099 trim_cache size 0 max 16384 2024-04-25T10:31:59.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.815+0000 7efc48420740 1 Processor -- start 2024-04-25T10:31:59.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.815+0000 7efc48420740 1 -- start start 2024-04-25T10:31:59.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.815+0000 7efc48420740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e7c3b0 0x56331225fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.815+0000 7efc48420740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311e84590 0x56331224ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.815+0000 7efc48420740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311ea5270 0x56331224f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.815+0000 7efc48420740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563312260300 con 0x563311e84590 2024-04-25T10:31:59.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.815+0000 7efc48420740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5633122609d0 con 0x563311ea5270 2024-04-25T10:31:59.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.815+0000 7efc48420740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5633122610a0 con 0x563311e7c3b0 2024-04-25T10:31:59.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.815+0000 7efc3affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311e84590 0x56331224ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.815+0000 7efc3affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311e84590 0x56331224ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53038/0 (socket says 172.21.15.43:53038) 2024-04-25T10:31:59.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.815+0000 7efc3affd640 1 -- 172.21.15.43:0/2451055723 learned_addr learned my addr 172.21.15.43:0/2451055723 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:59.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.816+0000 7efc3a7fc640 1 --2- 172.21.15.43:0/2451055723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e7c3b0 0x56331225fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.816+0000 7efc3affd640 1 -- 172.21.15.43:0/2451055723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e7c3b0 msgr2=0x56331225fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.816+0000 7efc3a7fc640 1 --2- 172.21.15.43:0/2451055723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311ea5270 0x56331224f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.816+0000 7efc3affd640 1 --2- 172.21.15.43:0/2451055723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e7c3b0 0x56331225fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.816+0000 7efc3affd640 1 -- 172.21.15.43:0/2451055723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311ea5270 msgr2=0x56331224f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.816+0000 7efc3affd640 1 --2- 172.21.15.43:0/2451055723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311ea5270 0x56331224f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.816+0000 7efc3affd640 1 -- 172.21.15.43:0/2451055723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563312252980 con 0x563311e84590 2024-04-25T10:31:59.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.816+0000 7efc3a7fc640 1 --2- 172.21.15.43:0/2451055723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311ea5270 0x56331224f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:59.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.816+0000 7efc3affd640 1 --2- 172.21.15.43:0/2451055723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311e84590 0x56331224ef10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x563312247710 tx=0x7efc2c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c86d6827f8f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.816+0000 7efc20ff9640 1 -- 172.21.15.43:0/2451055723 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c004e60 con 0x563311e84590 2024-04-25T10:31:59.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.816+0000 7efc20ff9640 1 -- 172.21.15.43:0/2451055723 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc2c01a2c0 con 0x563311e84590 2024-04-25T10:31:59.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.816+0000 7efc20ff9640 1 -- 172.21.15.43:0/2451055723 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c003850 con 0x563311e84590 2024-04-25T10:31:59.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc48420740 1 -- 172.21.15.43:0/2451055723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311e84590 msgr2=0x56331224ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc48420740 1 --2- 172.21.15.43:0/2451055723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311e84590 0x56331224ef10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x563312247710 tx=0x7efc2c00b390 comp rx=0 tx=0).stop 2024-04-25T10:31:59.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc48420740 1 -- 172.21.15.43:0/2451055723 shutdown_connections 2024-04-25T10:31:59.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc48420740 1 --2- 172.21.15.43:0/2451055723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311ea5270 0x56331224f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc48420740 1 --2- 172.21.15.43:0/2451055723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311e84590 0x56331224ef10 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc48420740 1 --2- 172.21.15.43:0/2451055723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e7c3b0 0x56331225fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc48420740 1 -- 172.21.15.43:0/2451055723 >> 172.21.15.43:0/2451055723 conn(0x563311e993c0 msgr2=0x56331224e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc48420740 1 -- 172.21.15.43:0/2451055723 shutdown_connections 2024-04-25T10:31:59.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc48420740 1 -- 172.21.15.43:0/2451055723 wait complete. 2024-04-25T10:31:59.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc48420740 1 -- start start 2024-04-25T10:31:59.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc48420740 1 Processor -- start 2024-04-25T10:31:59.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc48420740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e7c3b0 0x56331224cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc48420740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311e84590 0x56331224d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc48420740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311ea5270 0x56331224da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc48420740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563312261e40 con 0x563311e84590 2024-04-25T10:31:59.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc48420740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563312261770 con 0x563311ea5270 2024-04-25T10:31:59.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc48420740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563312262510 con 0x563311e7c3b0 2024-04-25T10:31:59.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc48420740 10 client.0 _ll_register_callbacks cb 0x7efc3000bc40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:59.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc3affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311e84590 0x56331224d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc3affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311e84590 0x56331224d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53050/0 (socket says 172.21.15.43:53050) 2024-04-25T10:31:59.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.817+0000 7efc3affd640 1 -- 172.21.15.43:0/4260041846 learned_addr learned my addr 172.21.15.43:0/4260041846 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:31:59.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.818+0000 7efc3a7fc640 1 --2- 172.21.15.43:0/4260041846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311ea5270 0x56331224da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.818+0000 7efc3a7fc640 1 --2- 172.21.15.43:0/4260041846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e7c3b0 0x56331224cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.818+0000 7efc3affd640 1 -- 172.21.15.43:0/4260041846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e7c3b0 msgr2=0x56331224cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.818+0000 7efc3affd640 1 --2- 172.21.15.43:0/4260041846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e7c3b0 0x56331224cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.818+0000 7efc3affd640 1 -- 172.21.15.43:0/4260041846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311ea5270 msgr2=0x56331224da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.818+0000 7efc3affd640 1 --2- 172.21.15.43:0/4260041846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311ea5270 0x56331224da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.818+0000 7efc3affd640 1 -- 172.21.15.43:0/4260041846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563312259ad0 con 0x563311e84590 2024-04-25T10:31:59.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.818+0000 7efc3a7fc640 1 --2- 172.21.15.43:0/4260041846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e7c3b0 0x56331224cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:59.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.818+0000 7efc3affd640 1 --2- 172.21.15.43:0/4260041846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311e84590 0x56331224d4e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efc2c01c040 tx=0x7efc2c004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d761fe1de7603c17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.818+0000 7efc1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:31:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.818+0000 7efc1eff5640 1 -- 172.21.15.43:0/4260041846 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c004350 con 0x563311e84590 2024-04-25T10:31:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.818+0000 7efc1eff5640 1 -- 172.21.15.43:0/4260041846 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7efc2c01a470 con 0x563311e84590 2024-04-25T10:31:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.818+0000 7efc48420740 1 -- 172.21.15.43:0/4260041846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc2c01a2c0 con 0x563311e84590 2024-04-25T10:31:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.818+0000 7efc1eff5640 1 -- 172.21.15.43:0/4260041846 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7efc2c01a670 con 0x563311e84590 2024-04-25T10:31:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.818+0000 7efc48420740 10 client.15105 fetch_fsmap learned FSMap version 23 2024-04-25T10:31:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.818+0000 7efc48420740 1 -- 172.21.15.43:0/4260041846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56331225acd0 con 0x563311e84590 2024-04-25T10:31:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.818+0000 7efc1eff5640 1 -- 172.21.15.43:0/4260041846 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc2c01a2c0 con 0x563311e84590 2024-04-25T10:31:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.819+0000 7efc1eff5640 1 -- 172.21.15.43:0/4260041846 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7efc2c035020 con 0x563311e84590 2024-04-25T10:31:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.819+0000 7efc48420740 10 client.15105 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:31:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.819+0000 7efc48420740 10 client.15105 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:31:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.819+0000 7efc48420740 20 client.15105 populate_metadata read hostname 'smithi043' 2024-04-25T10:31:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.819+0000 7efc48420740 1 --2- 172.21.15.43:0/4260041846 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563311e39280 0x5633122cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.819+0000 7efc48420740 4 client.15105 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:31:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.819+0000 7efc48420740 1 -- 172.21.15.43:0/4260041846 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "status"}) -- 0x5633121b31c0 con 0x563311e39280 2024-04-25T10:31:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.819+0000 7efc3a7fc640 1 --2- 172.21.15.43:0/4260041846 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563311e39280 0x5633122cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.819+0000 7efc1eff5640 10 client.15105 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:31:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.819+0000 7efc3a7fc640 1 --2- 172.21.15.43:0/4260041846 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563311e39280 0x5633122cdb80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87c648e8b6a26385 server_cookie=3b819f580802d7c6 in_seq=0 out_seq=0 2024-04-25T10:31:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc1eff5640 1 -- 172.21.15.43:0/4260041846 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5633121b31c0 con 0x563311e39280 2024-04-25T10:31:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc1eff5640 10 client.15105 handle_command_reply: tid=0 2024-04-25T10:31:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc48420740 1 client.15105 shutdown 2024-04-25T10:31:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc48420740 1 -- 172.21.15.43:0/4260041846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311e84590 msgr2=0x56331224d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc48420740 1 --2- 172.21.15.43:0/4260041846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311e84590 0x56331224d4e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efc2c01c040 tx=0x7efc2c004f00 comp rx=0 tx=0).stop 2024-04-25T10:31:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc48420740 1 -- 172.21.15.43:0/4260041846 shutdown_connections 2024-04-25T10:31:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc48420740 1 --2- 172.21.15.43:0/4260041846 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563311e39280 0x5633122cdb80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc48420740 1 --2- 172.21.15.43:0/4260041846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563311ea5270 0x56331224da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc48420740 1 --2- 172.21.15.43:0/4260041846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563311e84590 0x56331224d4e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc48420740 1 --2- 172.21.15.43:0/4260041846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563311e7c3b0 0x56331224cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc1eff5640 0 client.15105 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:31:59.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc48420740 1 -- 172.21.15.43:0/4260041846 >> 172.21.15.43:0/4260041846 conn(0x563311e993c0 msgr2=0x5633122416b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc48420740 1 -- 172.21.15.43:0/4260041846 shutdown_connections 2024-04-25T10:31:59.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc48420740 1 -- 172.21.15.43:0/4260041846 wait complete. 2024-04-25T10:31:59.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc48420740 20 client.15105 trim_cache size 0 max 16384 2024-04-25T10:31:59.829 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:31:59.829 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "94d0035c-02ed-11ef-bc93-c7b262605968", 2024-04-25T10:31:59.829 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-25T10:31:59.829 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24554, 2024-04-25T10:31:59.829 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-25T10:31:59.830 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-25T10:31:59.830 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-25T10:31:59.830 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 85.666193948, 2024-04-25T10:31:59.830 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 23, 2024-04-25T10:31:59.830 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 79, 2024-04-25T10:31:59.830 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 77, 2024-04-25T10:31:59.830 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 91.421684326999994 2024-04-25T10:31:59.830 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:31:59.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc4157a640 1 -- 172.21.15.43:0/1516896527 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7efc240afb80 msgr2=0x7efc240b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc4157a640 1 --2- 172.21.15.43:0/1516896527 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7efc240afb80 0x7efc240b2090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efc340c4800 tx=0x7efc34096030 comp rx=0 tx=0).stop 2024-04-25T10:31:59.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc4157a640 1 -- 172.21.15.43:0/1516896527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc3c1080d0 msgr2=0x7efc3c1a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc4157a640 1 --2- 172.21.15.43:0/1516896527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc3c1080d0 0x7efc3c1a0580 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efc340097b0 tx=0x7efc34003c20 comp rx=0 tx=0).stop 2024-04-25T10:31:59.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc4157a640 1 -- 172.21.15.43:0/1516896527 shutdown_connections 2024-04-25T10:31:59.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc4157a640 1 --2- 172.21.15.43:0/1516896527 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7efc240afb80 0x7efc240b2090 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc4157a640 1 --2- 172.21.15.43:0/1516896527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc3c1080d0 0x7efc3c1a0580 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc4157a640 1 --2- 172.21.15.43:0/1516896527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc3c1033d0 0x7efc3c1995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.820+0000 7efc4157a640 1 --2- 172.21.15.43:0/1516896527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc3c102190 0x7efc3c1990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.821+0000 7efc4157a640 1 -- 172.21.15.43:0/1516896527 >> 172.21.15.43:0/1516896527 conn(0x7efc3c0fd380 msgr2=0x7efc3c1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.821+0000 7efc4157a640 1 -- 172.21.15.43:0/1516896527 shutdown_connections 2024-04-25T10:31:59.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:31:59.821+0000 7efc4157a640 1 -- 172.21.15.43:0/1516896527 wait complete. 2024-04-25T10:31:59.834 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T10:31:59.834 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T10:31:59.843 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T10:31:59.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T10:32:00.091 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:32:00.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:59 smithi043 ceph-mon[29478]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T10:32:00.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:59 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3846515863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:00.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:31:59 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1225937164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:32:00.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:59 smithi077 ceph-mon[32772]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T10:32:00.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:59 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3846515863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:00.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:31:59 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1225937164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:32:00.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:59 smithi164 ceph-mon[32713]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T10:32:00.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:59 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3846515863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:00.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:31:59 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1225937164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:32:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.673+0000 7f4cc80c1700 1 -- 172.21.15.43:0/3883671176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cc0107640 msgr2=0x7f4cc0109a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.673+0000 7f4cc80c1700 1 --2- 172.21.15.43:0/3883671176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cc0107640 0x7f4cc0109a60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009990 tx=0x7f4cbc009ca0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.673+0000 7f4cc80c1700 1 -- 172.21.15.43:0/3883671176 shutdown_connections 2024-04-25T10:32:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.673+0000 7f4cc80c1700 1 --2- 172.21.15.43:0/3883671176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cc0109fa0 0x7f4cc010c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.673+0000 7f4cc80c1700 1 --2- 172.21.15.43:0/3883671176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc0102550 0x7f4cc01029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.673+0000 7f4cc80c1700 1 --2- 172.21.15.43:0/3883671176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cc0107640 0x7f4cc0109a60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.673+0000 7f4cc80c1700 1 -- 172.21.15.43:0/3883671176 >> 172.21.15.43:0/3883671176 conn(0x7f4cc00fded0 msgr2=0x7f4cc0100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.673+0000 7f4cc80c1700 1 -- 172.21.15.43:0/3883671176 shutdown_connections 2024-04-25T10:32:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.673+0000 7f4cc80c1700 1 -- 172.21.15.43:0/3883671176 wait complete. 2024-04-25T10:32:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.673+0000 7f4cc80c1700 1 Processor -- start 2024-04-25T10:32:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.674+0000 7f4cc80c1700 1 -- start start 2024-04-25T10:32:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.674+0000 7f4cc80c1700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cc0102550 0x7f4cc019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.674+0000 7f4cc80c1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc0107640 0x7f4cc019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.674+0000 7f4cc80c1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cc0109fa0 0x7f4cc01a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.674+0000 7f4cc80c1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc010fc30 con 0x7f4cc0109fa0 2024-04-25T10:32:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.674+0000 7f4cc80c1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc010fab0 con 0x7f4cc0107640 2024-04-25T10:32:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.674+0000 7f4cc80c1700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc010fdb0 con 0x7f4cc0102550 2024-04-25T10:32:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.674+0000 7f4cc665e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cc0109fa0 0x7f4cc01a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.674+0000 7f4cc665e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cc0109fa0 0x7f4cc01a1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53068/0 (socket says 172.21.15.43:53068) 2024-04-25T10:32:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.674+0000 7f4cc665e700 1 -- 172.21.15.43:0/2092137914 learned_addr learned my addr 172.21.15.43:0/2092137914 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.674+0000 7f4cc5e5d700 1 --2- 172.21.15.43:0/2092137914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cc0102550 0x7f4cc019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.674+0000 7f4cc565c700 1 --2- 172.21.15.43:0/2092137914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc0107640 0x7f4cc019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.674+0000 7f4cc665e700 1 -- 172.21.15.43:0/2092137914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cc0102550 msgr2=0x7f4cc019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.674+0000 7f4cc665e700 1 --2- 172.21.15.43:0/2092137914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cc0102550 0x7f4cc019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.674+0000 7f4cc665e700 1 -- 172.21.15.43:0/2092137914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc0107640 msgr2=0x7f4cc019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.674+0000 7f4cc665e700 1 --2- 172.21.15.43:0/2092137914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc0107640 0x7f4cc019ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.674+0000 7f4cc665e700 1 -- 172.21.15.43:0/2092137914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc01a2220 con 0x7f4cc0109fa0 2024-04-25T10:32:01.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.674+0000 7f4cc665e700 1 --2- 172.21.15.43:0/2092137914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cc0109fa0 0x7f4cc01a1c30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8007d70 tx=0x7f4cb800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:01.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.675+0000 7f4cb6ffd700 1 -- 172.21.15.43:0/2092137914 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8016020 con 0x7f4cc0109fa0 2024-04-25T10:32:01.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.675+0000 7f4cb6ffd700 1 -- 172.21.15.43:0/2092137914 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cb800ac90 con 0x7f4cc0109fa0 2024-04-25T10:32:01.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.675+0000 7f4cb6ffd700 1 -- 172.21.15.43:0/2092137914 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb80115e0 con 0x7f4cc0109fa0 2024-04-25T10:32:01.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.675+0000 7f4cc80c1700 1 -- 172.21.15.43:0/2092137914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc01a2510 con 0x7f4cc0109fa0 2024-04-25T10:32:01.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.675+0000 7f4cc80c1700 1 -- 172.21.15.43:0/2092137914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc01a2a00 con 0x7f4cc0109fa0 2024-04-25T10:32:01.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.676+0000 7f4cc80c1700 1 -- 172.21.15.43:0/2092137914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc0051190 con 0x7f4cc0109fa0 2024-04-25T10:32:01.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.677+0000 7f4cb6ffd700 1 -- 172.21.15.43:0/2092137914 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f4cb8019020 con 0x7f4cc0109fa0 2024-04-25T10:32:01.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.678+0000 7f4cb6ffd700 1 --2- 172.21.15.43:0/2092137914 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f4cac0afc10 0x7f4cac0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.678+0000 7f4cb6ffd700 1 -- 172.21.15.43:0/2092137914 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f4cb80dec00 con 0x7f4cc0109fa0 2024-04-25T10:32:01.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.678+0000 7f4cc5e5d700 1 --2- 172.21.15.43:0/2092137914 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f4cac0afc10 0x7f4cac0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.678+0000 7f4cc5e5d700 1 --2- 172.21.15.43:0/2092137914 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f4cac0afc10 0x7f4cac0b2120 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0005950 tx=0x7f4cb000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:01.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.683+0000 7f4cb6ffd700 1 -- 172.21.15.43:0/2092137914 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4cb8020020 con 0x7f4cc0109fa0 2024-04-25T10:32:01.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.915+0000 7f4cc80c1700 1 -- 172.21.15.43:0/2092137914 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f4cc0063ad0 con 0x7f4cac0afc10 2024-04-25T10:32:01.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.916+0000 7f4cb6ffd700 1 -- 172.21.15.43:0/2092137914 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f4cc0063ad0 con 0x7f4cac0afc10 2024-04-25T10:32:01.918 INFO:teuthology.orchestra.run.smithi043.stdout:Backend: cephadm 2024-04-25T10:32:01.918 INFO:teuthology.orchestra.run.smithi043.stdout:Available: Yes 2024-04-25T10:32:01.918 INFO:teuthology.orchestra.run.smithi043.stdout:Paused: No 2024-04-25T10:32:01.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.918+0000 7f4cc80c1700 1 -- 172.21.15.43:0/2092137914 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f4cac0afc10 msgr2=0x7f4cac0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.918+0000 7f4cc80c1700 1 --2- 172.21.15.43:0/2092137914 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f4cac0afc10 0x7f4cac0b2120 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0005950 tx=0x7f4cb000b410 comp rx=0 tx=0).stop 2024-04-25T10:32:01.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.918+0000 7f4cc80c1700 1 -- 172.21.15.43:0/2092137914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cc0109fa0 msgr2=0x7f4cc01a1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.918+0000 7f4cc80c1700 1 --2- 172.21.15.43:0/2092137914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cc0109fa0 0x7f4cc01a1c30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8007d70 tx=0x7f4cb800a340 comp rx=0 tx=0).stop 2024-04-25T10:32:01.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.918+0000 7f4cc80c1700 1 -- 172.21.15.43:0/2092137914 shutdown_connections 2024-04-25T10:32:01.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.918+0000 7f4cc80c1700 1 --2- 172.21.15.43:0/2092137914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cc0109fa0 0x7f4cc01a1c30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.918+0000 7f4cc80c1700 1 --2- 172.21.15.43:0/2092137914 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f4cac0afc10 0x7f4cac0b2120 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.918+0000 7f4cc80c1700 1 --2- 172.21.15.43:0/2092137914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cc0102550 0x7f4cc019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.918+0000 7f4cc80c1700 1 --2- 172.21.15.43:0/2092137914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc0107640 0x7f4cc019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.918+0000 7f4cc80c1700 1 -- 172.21.15.43:0/2092137914 >> 172.21.15.43:0/2092137914 conn(0x7f4cc00fded0 msgr2=0x7f4cc0108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:01.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.919+0000 7f4cc80c1700 1 -- 172.21.15.43:0/2092137914 shutdown_connections 2024-04-25T10:32:01.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:01.919+0000 7f4cc80c1700 1 -- 172.21.15.43:0/2092137914 wait complete. 2024-04-25T10:32:02.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:01 smithi043 ceph-mon[29478]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T10:32:02.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:01 smithi077 ceph-mon[32772]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T10:32:02.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:01 smithi164 ceph-mon[32713]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T10:32:02.550 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T10:32:02.797 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:32:03.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:02 smithi043 ceph-mon[29478]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:03.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:02 smithi077 ceph-mon[32772]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:03.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:02 smithi164 ceph-mon[32713]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:04.159 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:03 smithi043 ceph-mon[29478]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:32:04.159 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:03 smithi043 ceph-mon[29478]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T10:32:04.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:03 smithi077 ceph-mon[32772]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:32:04.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:03 smithi077 ceph-mon[32772]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T10:32:04.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:03 smithi164 ceph-mon[32713]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:32:04.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:03 smithi164 ceph-mon[32713]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T10:32:04.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.389+0000 7fbfa6425700 1 -- 172.21.15.43:0/1003019255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfa0104bb0 msgr2=0x7fbfa010b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.389+0000 7fbfa6425700 1 --2- 172.21.15.43:0/1003019255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfa0104bb0 0x7fbfa010b460 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800b0a0 tx=0x7fbf9800b3b0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.389+0000 7fbfa6425700 1 -- 172.21.15.43:0/1003019255 shutdown_connections 2024-04-25T10:32:04.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.389+0000 7fbfa6425700 1 --2- 172.21.15.43:0/1003019255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfa0104bb0 0x7fbfa010b460 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.389+0000 7fbfa6425700 1 --2- 172.21.15.43:0/1003019255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbfa01030e0 0x7fbfa01034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.389+0000 7fbfa6425700 1 --2- 172.21.15.43:0/1003019255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfa0104210 0x7fbfa0104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.389+0000 7fbfa6425700 1 -- 172.21.15.43:0/1003019255 >> 172.21.15.43:0/1003019255 conn(0x7fbfa00fe740 msgr2=0x7fbfa0100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:04.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.389+0000 7fbfa6425700 1 -- 172.21.15.43:0/1003019255 shutdown_connections 2024-04-25T10:32:04.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.389+0000 7fbfa6425700 1 -- 172.21.15.43:0/1003019255 wait complete. 2024-04-25T10:32:04.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.390+0000 7fbfa6425700 1 Processor -- start 2024-04-25T10:32:04.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.390+0000 7fbfa6425700 1 -- start start 2024-04-25T10:32:04.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.390+0000 7fbfa6425700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfa01030e0 0x7fbfa019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:04.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.390+0000 7fbfa6425700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfa0104210 0x7fbfa019ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:04.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.390+0000 7fbfa6425700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbfa0104bb0 0x7fbfa01a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:04.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.390+0000 7fbfa6425700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa010eb60 con 0x7fbfa0104210 2024-04-25T10:32:04.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.390+0000 7fbfa6425700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa010e9e0 con 0x7fbfa0104bb0 2024-04-25T10:32:04.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.390+0000 7fbfa6425700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa010ece0 con 0x7fbfa01030e0 2024-04-25T10:32:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.390+0000 7fbf9f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfa0104210 0x7fbfa019ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.390+0000 7fbf9f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfa0104210 0x7fbfa019ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53082/0 (socket says 172.21.15.43:53082) 2024-04-25T10:32:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.391+0000 7fbf9f7fe700 1 -- 172.21.15.43:0/1688016574 learned_addr learned my addr 172.21.15.43:0/1688016574 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.391+0000 7fbf9ffff700 1 --2- 172.21.15.43:0/1688016574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfa01030e0 0x7fbfa019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.391+0000 7fbfa49c2700 1 --2- 172.21.15.43:0/1688016574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbfa0104bb0 0x7fbfa01a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.391+0000 7fbf9f7fe700 1 -- 172.21.15.43:0/1688016574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfa01030e0 msgr2=0x7fbfa019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.391+0000 7fbf9f7fe700 1 --2- 172.21.15.43:0/1688016574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfa01030e0 0x7fbfa019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.391+0000 7fbf9f7fe700 1 -- 172.21.15.43:0/1688016574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbfa0104bb0 msgr2=0x7fbfa01a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.391+0000 7fbf9f7fe700 1 --2- 172.21.15.43:0/1688016574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbfa0104bb0 0x7fbfa01a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.391+0000 7fbf9f7fe700 1 -- 172.21.15.43:0/1688016574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa01a2300 con 0x7fbfa0104210 2024-04-25T10:32:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.391+0000 7fbfa49c2700 1 --2- 172.21.15.43:0/1688016574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbfa0104bb0 0x7fbfa01a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.391+0000 7fbf9f7fe700 1 --2- 172.21.15.43:0/1688016574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfa0104210 0x7fbfa019ace0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800d950 tx=0x7fbf8800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.391+0000 7fbf9d7fa700 1 -- 172.21.15.43:0/1688016574 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf88016070 con 0x7fbfa0104210 2024-04-25T10:32:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.391+0000 7fbfa6425700 1 -- 172.21.15.43:0/1688016574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa01a25f0 con 0x7fbfa0104210 2024-04-25T10:32:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.391+0000 7fbf9d7fa700 1 -- 172.21.15.43:0/1688016574 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf88009df0 con 0x7fbfa0104210 2024-04-25T10:32:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.391+0000 7fbf9d7fa700 1 -- 172.21.15.43:0/1688016574 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8800f3c0 con 0x7fbfa0104210 2024-04-25T10:32:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.391+0000 7fbfa6425700 1 -- 172.21.15.43:0/1688016574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa01a2b80 con 0x7fbfa0104210 2024-04-25T10:32:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.392+0000 7fbfa6425700 1 -- 172.21.15.43:0/1688016574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa0051190 con 0x7fbfa0104210 2024-04-25T10:32:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.394+0000 7fbf9d7fa700 1 -- 172.21.15.43:0/1688016574 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fbf88019070 con 0x7fbfa0104210 2024-04-25T10:32:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.394+0000 7fbf9d7fa700 1 --2- 172.21.15.43:0/1688016574 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fbf8c0afb80 0x7fbf8c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.395+0000 7fbf9ffff700 1 --2- 172.21.15.43:0/1688016574 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fbf8c0afb80 0x7fbf8c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.395+0000 7fbf9d7fa700 1 -- 172.21.15.43:0/1688016574 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fbf880de830 con 0x7fbfa0104210 2024-04-25T10:32:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.395+0000 7fbf9ffff700 1 --2- 172.21.15.43:0/1688016574 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fbf8c0afb80 0x7fbf8c0b2090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0104110 tx=0x7fbf9000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.398+0000 7fbf9d7fa700 1 -- 172.21.15.43:0/1688016574 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf880a7420 con 0x7fbfa0104210 2024-04-25T10:32:04.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.618+0000 7fbfa6425700 1 -- 172.21.15.43:0/1688016574 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fbfa01a2de0 con 0x7fbf8c0afb80 2024-04-25T10:32:04.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.632+0000 7fbf9d7fa700 1 -- 172.21.15.43:0/1688016574 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fbfa01a2de0 con 0x7fbf8c0afb80 2024-04-25T10:32:04.635 INFO:teuthology.orchestra.run.smithi043.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T10:32:04.635 INFO:teuthology.orchestra.run.smithi043.stdout:mds.a smithi043 running (99s) 15s ago 98s 81.5M - 19.0.0-3291-g994e2f20 d53c96779a29 bf76e4121c28 2024-04-25T10:32:04.635 INFO:teuthology.orchestra.run.smithi043.stdout:mds.b smithi077 running (96s) 29s ago 96s 96.3M - 19.0.0-3291-g994e2f20 d53c96779a29 9922b8342e70 2024-04-25T10:32:04.635 INFO:teuthology.orchestra.run.smithi043.stdout:mds.c smithi164 running (110s) 29s ago 110s 84.5M - 19.0.0-3291-g994e2f20 d53c96779a29 d7162b297d7c 2024-04-25T10:32:04.635 INFO:teuthology.orchestra.run.smithi043.stdout:mds.d smithi043 running (118s) 15s ago 118s 81.1M - 19.0.0-3291-g994e2f20 d53c96779a29 37ca5ef7abd9 2024-04-25T10:32:04.635 INFO:teuthology.orchestra.run.smithi043.stdout:mds.e smithi077 running (115s) 29s ago 115s 86.5M - 19.0.0-3291-g994e2f20 d53c96779a29 dd1c5cb7d7d0 2024-04-25T10:32:04.635 INFO:teuthology.orchestra.run.smithi043.stdout:mds.f smithi164 running (113s) 29s ago 112s 87.9M - 19.0.0-3291-g994e2f20 d53c96779a29 b5692deb846c 2024-04-25T10:32:04.635 INFO:teuthology.orchestra.run.smithi043.stdout:mds.g smithi043 running (102s) 15s ago 101s 80.7M - 19.0.0-3291-g994e2f20 d53c96779a29 e7c559510af1 2024-04-25T10:32:04.635 INFO:teuthology.orchestra.run.smithi043.stdout:mds.h smithi077 running (107s) 29s ago 107s 85.5M - 19.0.0-3291-g994e2f20 d53c96779a29 26ed05757041 2024-04-25T10:32:04.635 INFO:teuthology.orchestra.run.smithi043.stdout:mds.i smithi164 running (105s) 29s ago 104s 84.8M - 19.0.0-3291-g994e2f20 d53c96779a29 23da59098753 2024-04-25T10:32:04.635 INFO:teuthology.orchestra.run.smithi043.stdout:mds.j smithi043 running (2m) 15s ago 2m 80.7M - 19.0.0-3291-g994e2f20 d53c96779a29 83b2057bdc49 2024-04-25T10:32:04.635 INFO:teuthology.orchestra.run.smithi043.stdout:mds.k smithi077 running (2m) 29s ago 2m 81.2M - 19.0.0-3291-g994e2f20 d53c96779a29 cdd9460375e8 2024-04-25T10:32:04.635 INFO:teuthology.orchestra.run.smithi043.stdout:mds.l smithi164 running (2m) 29s ago 2m 81.1M - 19.0.0-3291-g994e2f20 d53c96779a29 a0a5434eda6a 2024-04-25T10:32:04.636 INFO:teuthology.orchestra.run.smithi043.stdout:mgr.x smithi043 *:9283,8765 running (9m) 15s ago 9m 583M - 19.0.0-3291-g994e2f20 d53c96779a29 788b64a21bab 2024-04-25T10:32:04.636 INFO:teuthology.orchestra.run.smithi043.stdout:mgr.y smithi077 *:8443,8765 running (7m) 29s ago 7m 519M - 19.0.0-3291-g994e2f20 d53c96779a29 8e9cc76a10c1 2024-04-25T10:32:04.636 INFO:teuthology.orchestra.run.smithi043.stdout:mgr.z smithi164 *:8443,8765 running (7m) 29s ago 7m 521M - 19.0.0-3291-g994e2f20 d53c96779a29 45598f416ec8 2024-04-25T10:32:04.636 INFO:teuthology.orchestra.run.smithi043.stdout:mon.a smithi043 running (9m) 15s ago 9m 120M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 471bd00c8578 2024-04-25T10:32:04.636 INFO:teuthology.orchestra.run.smithi043.stdout:mon.b smithi077 running (7m) 29s ago 7m 109M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 6cbd105360e0 2024-04-25T10:32:04.636 INFO:teuthology.orchestra.run.smithi043.stdout:mon.c smithi164 running (7m) 29s ago 7m 108M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 31632d504d56 2024-04-25T10:32:04.636 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0 smithi043 running (7m) 15s ago 7m 122M 4273M 19.0.0-3291-g994e2f20 d53c96779a29 8f4ebdcaed14 2024-04-25T10:32:04.636 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1 smithi077 running (6m) 29s ago 6m 124M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 ef8b3fdac05b 2024-04-25T10:32:04.636 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2 smithi164 running (6m) 29s ago 6m 126M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 8fb0727b6cfa 2024-04-25T10:32:04.636 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3 smithi043 running (5m) 15s ago 5m 121M 4273M 19.0.0-3291-g994e2f20 d53c96779a29 d8c1a287a887 2024-04-25T10:32:04.636 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4 smithi077 running (5m) 29s ago 5m 123M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 22404b7aa16a 2024-04-25T10:32:04.636 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5 smithi164 running (4m) 29s ago 4m 124M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 5f3815c34c77 2024-04-25T10:32:04.636 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6 smithi043 running (4m) 15s ago 4m 116M 4273M 19.0.0-3291-g994e2f20 d53c96779a29 b0e7d1a71407 2024-04-25T10:32:04.636 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7 smithi077 running (4m) 29s ago 4m 125M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 2a820433855b 2024-04-25T10:32:04.636 INFO:teuthology.orchestra.run.smithi043.stdout:osd.8 smithi164 running (3m) 29s ago 3m 124M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 2b8f1683cd08 2024-04-25T10:32:04.637 INFO:teuthology.orchestra.run.smithi043.stdout:osd.9 smithi043 running (3m) 15s ago 3m 116M 4273M 19.0.0-3291-g994e2f20 d53c96779a29 3391eb3e13c6 2024-04-25T10:32:04.637 INFO:teuthology.orchestra.run.smithi043.stdout:osd.10 smithi077 running (2m) 29s ago 2m 123M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 14bf19662a4e 2024-04-25T10:32:04.637 INFO:teuthology.orchestra.run.smithi043.stdout:osd.11 smithi164 running (2m) 29s ago 2m 120M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 b15f56f11b11 2024-04-25T10:32:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.635+0000 7fbfa6425700 1 -- 172.21.15.43:0/1688016574 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fbf8c0afb80 msgr2=0x7fbf8c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.635+0000 7fbfa6425700 1 --2- 172.21.15.43:0/1688016574 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fbf8c0afb80 0x7fbf8c0b2090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0104110 tx=0x7fbf9000b410 comp rx=0 tx=0).stop 2024-04-25T10:32:04.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.635+0000 7fbfa6425700 1 -- 172.21.15.43:0/1688016574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfa0104210 msgr2=0x7fbfa019ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.635+0000 7fbfa6425700 1 --2- 172.21.15.43:0/1688016574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfa0104210 0x7fbfa019ace0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800d950 tx=0x7fbf8800dc60 comp rx=0 tx=0).stop 2024-04-25T10:32:04.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.635+0000 7fbfa6425700 1 -- 172.21.15.43:0/1688016574 shutdown_connections 2024-04-25T10:32:04.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.635+0000 7fbfa6425700 1 --2- 172.21.15.43:0/1688016574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbfa0104bb0 0x7fbfa01a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.635+0000 7fbfa6425700 1 --2- 172.21.15.43:0/1688016574 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fbf8c0afb80 0x7fbf8c0b2090 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.635+0000 7fbfa6425700 1 --2- 172.21.15.43:0/1688016574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfa01030e0 0x7fbfa019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.635+0000 7fbfa6425700 1 --2- 172.21.15.43:0/1688016574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfa0104210 0x7fbfa019ace0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.635+0000 7fbfa6425700 1 -- 172.21.15.43:0/1688016574 >> 172.21.15.43:0/1688016574 conn(0x7fbfa00fe740 msgr2=0x7fbfa0100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:04.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.635+0000 7fbfa6425700 1 -- 172.21.15.43:0/1688016574 shutdown_connections 2024-04-25T10:32:04.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:04.635+0000 7fbfa6425700 1 -- 172.21.15.43:0/1688016574 wait complete. 2024-04-25T10:32:05.281 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T10:32:05.528 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:32:05.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:05 smithi043 ceph-mon[29478]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T10:32:05.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:05 smithi043 ceph-mon[29478]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:06.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:05 smithi077 ceph-mon[32772]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T10:32:06.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:05 smithi077 ceph-mon[32772]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:06.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:05 smithi164 ceph-mon[32713]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T10:32:06.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:05 smithi164 ceph-mon[32713]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:07.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.234+0000 7f75158c7700 1 -- 172.21.15.43:0/846516773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7510104bb0 msgr2=0x7f751010b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.234+0000 7f75158c7700 1 --2- 172.21.15.43:0/846516773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7510104bb0 0x7f751010b460 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f750800b0a0 tx=0x7f750800b3b0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.234+0000 7f75158c7700 1 -- 172.21.15.43:0/846516773 shutdown_connections 2024-04-25T10:32:07.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.234+0000 7f75158c7700 1 --2- 172.21.15.43:0/846516773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7510104bb0 0x7f751010b460 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.234+0000 7f75158c7700 1 --2- 172.21.15.43:0/846516773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75101030e0 0x7f75101034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.234+0000 7f75158c7700 1 --2- 172.21.15.43:0/846516773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7510104210 0x7f7510104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.234+0000 7f75158c7700 1 -- 172.21.15.43:0/846516773 >> 172.21.15.43:0/846516773 conn(0x7f75100fe740 msgr2=0x7f7510100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:07.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.234+0000 7f75158c7700 1 -- 172.21.15.43:0/846516773 shutdown_connections 2024-04-25T10:32:07.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.234+0000 7f75158c7700 1 -- 172.21.15.43:0/846516773 wait complete. 2024-04-25T10:32:07.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.235+0000 7f75158c7700 1 Processor -- start 2024-04-25T10:32:07.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.235+0000 7f75158c7700 1 -- start start 2024-04-25T10:32:07.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.236+0000 7f75158c7700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75101030e0 0x7f7510076340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.236+0000 7f75158c7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7510104210 0x7f751019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.236+0000 7f75158c7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7510104bb0 0x7f75101a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.236+0000 7f75158c7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f751010eb50 con 0x7f7510104bb0 2024-04-25T10:32:07.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.236+0000 7f75158c7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f751010e9d0 con 0x7f7510104210 2024-04-25T10:32:07.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.236+0000 7f75158c7700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f751010ecd0 con 0x7f75101030e0 2024-04-25T10:32:07.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.236+0000 7f750f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7510104bb0 0x7f75101a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.236+0000 7f750f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7510104bb0 0x7f75101a1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53106/0 (socket says 172.21.15.43:53106) 2024-04-25T10:32:07.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.236+0000 7f750f7fe700 1 -- 172.21.15.43:0/3114090925 learned_addr learned my addr 172.21.15.43:0/3114090925 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:07.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.236+0000 7f750effd700 1 --2- 172.21.15.43:0/3114090925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75101030e0 0x7f7510076340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.236+0000 7f750e7fc700 1 --2- 172.21.15.43:0/3114090925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7510104210 0x7f751019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.236+0000 7f750f7fe700 1 -- 172.21.15.43:0/3114090925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75101030e0 msgr2=0x7f7510076340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.236+0000 7f750f7fe700 1 --2- 172.21.15.43:0/3114090925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75101030e0 0x7f7510076340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.236+0000 7f750f7fe700 1 -- 172.21.15.43:0/3114090925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7510104210 msgr2=0x7f751019f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.236+0000 7f750f7fe700 1 --2- 172.21.15.43:0/3114090925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7510104210 0x7f751019f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.236+0000 7f750f7fe700 1 -- 172.21.15.43:0/3114090925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75101a22a0 con 0x7f7510104bb0 2024-04-25T10:32:07.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.236+0000 7f750f7fe700 1 --2- 172.21.15.43:0/3114090925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7510104bb0 0x7f75101a1a70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f75080062a0 tx=0x7f7508004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:07.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.237+0000 7f75148c5700 1 -- 172.21.15.43:0/3114090925 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7508021070 con 0x7f7510104bb0 2024-04-25T10:32:07.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.237+0000 7f75148c5700 1 -- 172.21.15.43:0/3114090925 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7508007b80 con 0x7f7510104bb0 2024-04-25T10:32:07.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.237+0000 7f75158c7700 1 -- 172.21.15.43:0/3114090925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75101a2530 con 0x7f7510104bb0 2024-04-25T10:32:07.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.237+0000 7f75148c5700 1 -- 172.21.15.43:0/3114090925 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7508004670 con 0x7f7510104bb0 2024-04-25T10:32:07.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.237+0000 7f75158c7700 1 -- 172.21.15.43:0/3114090925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75101a2a70 con 0x7f7510104bb0 2024-04-25T10:32:07.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.238+0000 7f75158c7700 1 -- 172.21.15.43:0/3114090925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7510051190 con 0x7f7510104bb0 2024-04-25T10:32:07.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.239+0000 7f75148c5700 1 -- 172.21.15.43:0/3114090925 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7508003a00 con 0x7f7510104bb0 2024-04-25T10:32:07.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.240+0000 7f75148c5700 1 --2- 172.21.15.43:0/3114090925 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f74f80afb80 0x7f74f80b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.240+0000 7f75148c5700 1 -- 172.21.15.43:0/3114090925 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f750800f020 con 0x7f7510104bb0 2024-04-25T10:32:07.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.240+0000 7f750effd700 1 --2- 172.21.15.43:0/3114090925 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f74f80afb80 0x7f74f80b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.241+0000 7f750effd700 1 --2- 172.21.15.43:0/3114090925 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f74f80afb80 0x7f74f80b2090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7510104110 tx=0x7f750400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:07.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.246+0000 7f75148c5700 1 -- 172.21.15.43:0/3114090925 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f750802a020 con 0x7f7510104bb0 2024-04-25T10:32:07.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.461+0000 7f75158c7700 1 -- 172.21.15.43:0/3114090925 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f7510063ad0 con 0x7f74f80afb80 2024-04-25T10:32:07.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.467+0000 7f75148c5700 1 -- 172.21.15.43:0/3114090925 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f7510063ad0 con 0x7f74f80afb80 2024-04-25T10:32:07.470 INFO:teuthology.orchestra.run.smithi043.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T10:32:07.470 INFO:teuthology.orchestra.run.smithi043.stdout:mds.all 12/12 32s ago 2m smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l;count:12 2024-04-25T10:32:07.470 INFO:teuthology.orchestra.run.smithi043.stdout:mgr 3/3 32s ago 7m smithi043=x;smithi077=y;smithi164=z;count:3 2024-04-25T10:32:07.470 INFO:teuthology.orchestra.run.smithi043.stdout:mon 3/3 32s ago 7m smithi043:172.21.15.43=a;smithi077:172.21.15.77=b;smithi164:172.21.15.164=c;count:3 2024-04-25T10:32:07.470 INFO:teuthology.orchestra.run.smithi043.stdout:osd 12 32s ago - 2024-04-25T10:32:07.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.469+0000 7f75158c7700 1 -- 172.21.15.43:0/3114090925 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f74f80afb80 msgr2=0x7f74f80b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.469+0000 7f75158c7700 1 --2- 172.21.15.43:0/3114090925 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f74f80afb80 0x7f74f80b2090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7510104110 tx=0x7f750400b410 comp rx=0 tx=0).stop 2024-04-25T10:32:07.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.469+0000 7f75158c7700 1 -- 172.21.15.43:0/3114090925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7510104bb0 msgr2=0x7f75101a1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.469+0000 7f75158c7700 1 --2- 172.21.15.43:0/3114090925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7510104bb0 0x7f75101a1a70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f75080062a0 tx=0x7f7508004e40 comp rx=0 tx=0).stop 2024-04-25T10:32:07.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.469+0000 7f75158c7700 1 -- 172.21.15.43:0/3114090925 shutdown_connections 2024-04-25T10:32:07.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.469+0000 7f75158c7700 1 --2- 172.21.15.43:0/3114090925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7510104bb0 0x7f75101a1a70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.469+0000 7f75158c7700 1 --2- 172.21.15.43:0/3114090925 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f74f80afb80 0x7f74f80b2090 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.469+0000 7f75158c7700 1 --2- 172.21.15.43:0/3114090925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75101030e0 0x7f7510076340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.469+0000 7f75158c7700 1 --2- 172.21.15.43:0/3114090925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7510104210 0x7f751019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.469+0000 7f75158c7700 1 -- 172.21.15.43:0/3114090925 >> 172.21.15.43:0/3114090925 conn(0x7f75100fe740 msgr2=0x7f7510107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:07.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.470+0000 7f75158c7700 1 -- 172.21.15.43:0/3114090925 shutdown_connections 2024-04-25T10:32:07.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:07.470+0000 7f75158c7700 1 -- 172.21.15.43:0/3114090925 wait complete. 2024-04-25T10:32:08.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:07 smithi043 ceph-mon[29478]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T10:32:08.165 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T10:32:08.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:07 smithi077 ceph-mon[32772]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T10:32:08.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:07 smithi164 ceph-mon[32713]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T10:32:08.416 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:32:09.090 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:08 smithi043 ceph-mon[29478]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:09.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:08 smithi077 ceph-mon[32772]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:09.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:08 smithi164 ceph-mon[32713]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.959+0000 7f3470abb700 1 -- 172.21.15.43:0/627304666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f346c1030e0 msgr2=0x7f346c1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.959+0000 7f3470abb700 1 --2- 172.21.15.43:0/627304666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f346c1030e0 0x7f346c1034f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f345c009990 tx=0x7f345c009ca0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.959+0000 7f3470abb700 1 -- 172.21.15.43:0/627304666 shutdown_connections 2024-04-25T10:32:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.959+0000 7f3470abb700 1 --2- 172.21.15.43:0/627304666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f346c104bb0 0x7f346c10b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.959+0000 7f3470abb700 1 --2- 172.21.15.43:0/627304666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f346c1030e0 0x7f346c1034f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.959+0000 7f3470abb700 1 --2- 172.21.15.43:0/627304666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f346c104210 0x7f346c104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.959+0000 7f3470abb700 1 -- 172.21.15.43:0/627304666 >> 172.21.15.43:0/627304666 conn(0x7f346c0fe740 msgr2=0x7f346c100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.959+0000 7f3470abb700 1 -- 172.21.15.43:0/627304666 shutdown_connections 2024-04-25T10:32:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.959+0000 7f3470abb700 1 -- 172.21.15.43:0/627304666 wait complete. 2024-04-25T10:32:09.962 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:09 smithi043 ceph-mon[29478]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T10:32:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.960+0000 7f3470abb700 1 Processor -- start 2024-04-25T10:32:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.960+0000 7f3470abb700 1 -- start start 2024-04-25T10:32:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.960+0000 7f3470abb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f346c1030e0 0x7f346c106ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.960+0000 7f3470abb700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f346c104210 0x7f346c105550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.960+0000 7f3470abb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f346c104bb0 0x7f346c105a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.960+0000 7f3470abb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f346c10eb30 con 0x7f346c1030e0 2024-04-25T10:32:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.960+0000 7f3470abb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f346c10e9b0 con 0x7f346c104bb0 2024-04-25T10:32:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.960+0000 7f3470abb700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f346c10ecb0 con 0x7f346c104210 2024-04-25T10:32:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.960+0000 7f346a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f346c1030e0 0x7f346c106ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.960+0000 7f346a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f346c1030e0 0x7f346c106ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55074/0 (socket says 172.21.15.43:55074) 2024-04-25T10:32:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.960+0000 7f346a59c700 1 -- 172.21.15.43:0/2100540623 learned_addr learned my addr 172.21.15.43:0/2100540623 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.960+0000 7f3469d9b700 1 --2- 172.21.15.43:0/2100540623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f346c104210 0x7f346c105550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.960+0000 7f346ad9d700 1 --2- 172.21.15.43:0/2100540623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f346c104bb0 0x7f346c105a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.961+0000 7f346a59c700 1 -- 172.21.15.43:0/2100540623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f346c104210 msgr2=0x7f346c105550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.961+0000 7f346a59c700 1 --2- 172.21.15.43:0/2100540623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f346c104210 0x7f346c105550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.961+0000 7f346a59c700 1 -- 172.21.15.43:0/2100540623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f346c104bb0 msgr2=0x7f346c105a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.961+0000 7f346a59c700 1 --2- 172.21.15.43:0/2100540623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f346c104bb0 0x7f346c105a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.961+0000 7f346a59c700 1 -- 172.21.15.43:0/2100540623 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346c106160 con 0x7f346c1030e0 2024-04-25T10:32:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.961+0000 7f346a59c700 1 --2- 172.21.15.43:0/2100540623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f346c1030e0 0x7f346c106ea0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f345c0057c0 tx=0x7f345c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.961+0000 7f345b7fe700 1 -- 172.21.15.43:0/2100540623 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c027070 con 0x7f346c1030e0 2024-04-25T10:32:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.961+0000 7f3470abb700 1 -- 172.21.15.43:0/2100540623 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f346c1a48e0 con 0x7f346c1030e0 2024-04-25T10:32:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.961+0000 7f345b7fe700 1 -- 172.21.15.43:0/2100540623 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f345c004000 con 0x7f346c1030e0 2024-04-25T10:32:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.961+0000 7f3470abb700 1 -- 172.21.15.43:0/2100540623 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f346c1a4c80 con 0x7f346c1030e0 2024-04-25T10:32:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.961+0000 7f345b7fe700 1 -- 172.21.15.43:0/2100540623 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c022c60 con 0x7f346c1030e0 2024-04-25T10:32:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.962+0000 7f3470abb700 1 -- 172.21.15.43:0/2100540623 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f346c051190 con 0x7f346c1030e0 2024-04-25T10:32:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.964+0000 7f345b7fe700 1 -- 172.21.15.43:0/2100540623 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f345c02a070 con 0x7f346c1030e0 2024-04-25T10:32:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.965+0000 7f345b7fe700 1 --2- 172.21.15.43:0/2100540623 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f34540afc00 0x7f34540b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.965+0000 7f3469d9b700 1 --2- 172.21.15.43:0/2100540623 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f34540afc00 0x7f34540b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.965+0000 7f345b7fe700 1 -- 172.21.15.43:0/2100540623 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f345c0e01c0 con 0x7f346c1030e0 2024-04-25T10:32:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.965+0000 7f3469d9b700 1 --2- 172.21.15.43:0/2100540623 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f34540afc00 0x7f34540b2110 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3460005950 tx=0x7f346000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:09.968+0000 7f345b7fe700 1 -- 172.21.15.43:0/2100540623 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f345c031080 con 0x7f346c1030e0 2024-04-25T10:32:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:10.184+0000 7f3470abb700 1 -- 172.21.15.43:0/2100540623 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f346c100820 con 0x7f34540afc00 2024-04-25T10:32:10.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:10.186+0000 7f345b7fe700 1 -- 172.21.15.43:0/2100540623 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f346c100820 con 0x7f34540afc00 2024-04-25T10:32:10.188 INFO:teuthology.orchestra.run.smithi043.stdout:HOST ADDR LABELS STATUS 2024-04-25T10:32:10.188 INFO:teuthology.orchestra.run.smithi043.stdout:smithi043 172.21.15.43 2024-04-25T10:32:10.188 INFO:teuthology.orchestra.run.smithi043.stdout:smithi077 172.21.15.77 2024-04-25T10:32:10.188 INFO:teuthology.orchestra.run.smithi043.stdout:smithi164 172.21.15.164 2024-04-25T10:32:10.188 INFO:teuthology.orchestra.run.smithi043.stdout:3 hosts in cluster 2024-04-25T10:32:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:10.188+0000 7f3470abb700 1 -- 172.21.15.43:0/2100540623 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f34540afc00 msgr2=0x7f34540b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:10.188+0000 7f3470abb700 1 --2- 172.21.15.43:0/2100540623 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f34540afc00 0x7f34540b2110 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3460005950 tx=0x7f346000a300 comp rx=0 tx=0).stop 2024-04-25T10:32:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:10.188+0000 7f3470abb700 1 -- 172.21.15.43:0/2100540623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f346c1030e0 msgr2=0x7f346c106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:10.188+0000 7f3470abb700 1 --2- 172.21.15.43:0/2100540623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f346c1030e0 0x7f346c106ea0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f345c0057c0 tx=0x7f345c0057f0 comp rx=0 tx=0).stop 2024-04-25T10:32:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:10.188+0000 7f3470abb700 1 -- 172.21.15.43:0/2100540623 shutdown_connections 2024-04-25T10:32:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:10.188+0000 7f3470abb700 1 --2- 172.21.15.43:0/2100540623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f346c104bb0 0x7f346c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:10.188+0000 7f3470abb700 1 --2- 172.21.15.43:0/2100540623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f346c1030e0 0x7f346c106ea0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:10.188+0000 7f3470abb700 1 --2- 172.21.15.43:0/2100540623 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f34540afc00 0x7f34540b2110 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:10.188+0000 7f3470abb700 1 --2- 172.21.15.43:0/2100540623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f346c104210 0x7f346c105550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:10.188+0000 7f3470abb700 1 -- 172.21.15.43:0/2100540623 >> 172.21.15.43:0/2100540623 conn(0x7f346c0fe740 msgr2=0x7f346c100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:10.188+0000 7f3470abb700 1 -- 172.21.15.43:0/2100540623 shutdown_connections 2024-04-25T10:32:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:10.188+0000 7f3470abb700 1 -- 172.21.15.43:0/2100540623 wait complete. 2024-04-25T10:32:10.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:09 smithi077 ceph-mon[32772]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T10:32:10.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:09 smithi164 ceph-mon[32713]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T10:32:10.882 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T10:32:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:32:12.269 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:11 smithi043 ceph-mon[29478]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T10:32:12.269 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:11 smithi043 ceph-mon[29478]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:12.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:11 smithi077 ceph-mon[32772]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T10:32:12.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:11 smithi077 ceph-mon[32772]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:12.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:11 smithi164 ceph-mon[32713]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T10:32:12.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:11 smithi164 ceph-mon[32713]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:13.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.032+0000 7ff2ce239700 1 -- 172.21.15.43:0/31998010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2c8103120 msgr2=0x7ff2c8103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:13.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.032+0000 7ff2ce239700 1 --2- 172.21.15.43:0/31998010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2c8103120 0x7ff2c8103530 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8009990 tx=0x7ff2b8009ca0 comp rx=0 tx=0).stop 2024-04-25T10:32:13.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.032+0000 7ff2ce239700 1 -- 172.21.15.43:0/31998010 shutdown_connections 2024-04-25T10:32:13.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.032+0000 7ff2ce239700 1 --2- 172.21.15.43:0/31998010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2c8104bf0 0x7ff2c810b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:13.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.032+0000 7ff2ce239700 1 --2- 172.21.15.43:0/31998010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2c8103120 0x7ff2c8103530 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:13.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.032+0000 7ff2ce239700 1 --2- 172.21.15.43:0/31998010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2c8104250 0x7ff2c8104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:13.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.032+0000 7ff2ce239700 1 -- 172.21.15.43:0/31998010 >> 172.21.15.43:0/31998010 conn(0x7ff2c80fe760 msgr2=0x7ff2c8100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:13.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.032+0000 7ff2ce239700 1 -- 172.21.15.43:0/31998010 shutdown_connections 2024-04-25T10:32:13.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.032+0000 7ff2ce239700 1 -- 172.21.15.43:0/31998010 wait complete. 2024-04-25T10:32:13.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.033+0000 7ff2ce239700 1 Processor -- start 2024-04-25T10:32:13.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.033+0000 7ff2ce239700 1 -- start start 2024-04-25T10:32:13.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.033+0000 7ff2ce239700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2c8103120 0x7ff2c81996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:13.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.033+0000 7ff2ce239700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2c8104250 0x7ff2c8199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:13.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.033+0000 7ff2ce239700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2c8104bf0 0x7ff2c81a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:13.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.033+0000 7ff2ce239700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c810eb60 con 0x7ff2c8103120 2024-04-25T10:32:13.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.033+0000 7ff2ce239700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c810e9e0 con 0x7ff2c8104bf0 2024-04-25T10:32:13.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.033+0000 7ff2ce239700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c810ece0 con 0x7ff2c8104250 2024-04-25T10:32:13.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.033+0000 7ff2c77fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2c8103120 0x7ff2c81996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:13.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.033+0000 7ff2c77fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2c8103120 0x7ff2c81996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55082/0 (socket says 172.21.15.43:55082) 2024-04-25T10:32:13.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.033+0000 7ff2c77fe700 1 -- 172.21.15.43:0/1932831017 learned_addr learned my addr 172.21.15.43:0/1932831017 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:13.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.033+0000 7ff2c6ffd700 1 --2- 172.21.15.43:0/1932831017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2c8104250 0x7ff2c8199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:13.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.034+0000 7ff2c7fff700 1 --2- 172.21.15.43:0/1932831017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2c8104bf0 0x7ff2c81a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:13.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.034+0000 7ff2c77fe700 1 -- 172.21.15.43:0/1932831017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2c8104250 msgr2=0x7ff2c8199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:13.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.034+0000 7ff2c77fe700 1 --2- 172.21.15.43:0/1932831017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2c8104250 0x7ff2c8199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:13.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.034+0000 7ff2c77fe700 1 -- 172.21.15.43:0/1932831017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2c8104bf0 msgr2=0x7ff2c81a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:13.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.034+0000 7ff2c77fe700 1 --2- 172.21.15.43:0/1932831017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2c8104bf0 0x7ff2c81a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:13.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.034+0000 7ff2c77fe700 1 -- 172.21.15.43:0/1932831017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c81a1230 con 0x7ff2c8103120 2024-04-25T10:32:13.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.034+0000 7ff2c7fff700 1 --2- 172.21.15.43:0/1932831017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2c8104bf0 0x7ff2c81a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:13.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.034+0000 7ff2c77fe700 1 --2- 172.21.15.43:0/1932831017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2c8103120 0x7ff2c81996d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8000c00 tx=0x7ff2b80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:13.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.034+0000 7ff2c4ff9700 1 -- 172.21.15.43:0/1932831017 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8027070 con 0x7ff2c8103120 2024-04-25T10:32:13.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.034+0000 7ff2ce239700 1 -- 172.21.15.43:0/1932831017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c81a14c0 con 0x7ff2c8103120 2024-04-25T10:32:13.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.034+0000 7ff2c4ff9700 1 -- 172.21.15.43:0/1932831017 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b8004000 con 0x7ff2c8103120 2024-04-25T10:32:13.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.034+0000 7ff2c4ff9700 1 -- 172.21.15.43:0/1932831017 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8022d80 con 0x7ff2c8103120 2024-04-25T10:32:13.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.034+0000 7ff2ce239700 1 -- 172.21.15.43:0/1932831017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c81a1910 con 0x7ff2c8103120 2024-04-25T10:32:13.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.035+0000 7ff2ce239700 1 -- 172.21.15.43:0/1932831017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c8051190 con 0x7ff2c8103120 2024-04-25T10:32:13.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.037+0000 7ff2c4ff9700 1 -- 172.21.15.43:0/1932831017 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff2b802a070 con 0x7ff2c8103120 2024-04-25T10:32:13.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.038+0000 7ff2c4ff9700 1 --2- 172.21.15.43:0/1932831017 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff2b00afb80 0x7ff2b00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:13.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.038+0000 7ff2c6ffd700 1 --2- 172.21.15.43:0/1932831017 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff2b00afb80 0x7ff2b00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:13.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.038+0000 7ff2c4ff9700 1 -- 172.21.15.43:0/1932831017 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7ff2b80dffb0 con 0x7ff2c8103120 2024-04-25T10:32:13.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.038+0000 7ff2c6ffd700 1 --2- 172.21.15.43:0/1932831017 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff2b00afb80 0x7ff2b00b2090 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc005950 tx=0x7ff2bc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:13.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.041+0000 7ff2c4ff9700 1 -- 172.21.15.43:0/1932831017 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff2b8013070 con 0x7ff2c8103120 2024-04-25T10:32:13.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.260+0000 7ff2ce239700 1 -- 172.21.15.43:0/1932831017 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7ff2c8063ad0 con 0x7ff2b00afb80 2024-04-25T10:32:13.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.262+0000 7ff2c4ff9700 1 -- 172.21.15.43:0/1932831017 <== mgr.14152 v2:172.21.15.43:6800/1953858940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7ff2c8063ad0 con 0x7ff2b00afb80 2024-04-25T10:32:13.264 INFO:teuthology.orchestra.run.smithi043.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T10:32:13.264 INFO:teuthology.orchestra.run.smithi043.stdout:smithi043 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300GH480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-25T10:32:13.264 INFO:teuthology.orchestra.run.smithi043.stdout:smithi077 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400ZJ400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T10:32:13.264 INFO:teuthology.orchestra.run.smithi043.stdout:smithi164 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300H0400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T10:32:13.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.264+0000 7ff2ce239700 1 -- 172.21.15.43:0/1932831017 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff2b00afb80 msgr2=0x7ff2b00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:13.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.264+0000 7ff2ce239700 1 --2- 172.21.15.43:0/1932831017 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff2b00afb80 0x7ff2b00b2090 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc005950 tx=0x7ff2bc00a300 comp rx=0 tx=0).stop 2024-04-25T10:32:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.264+0000 7ff2ce239700 1 -- 172.21.15.43:0/1932831017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2c8103120 msgr2=0x7ff2c81996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.264+0000 7ff2ce239700 1 --2- 172.21.15.43:0/1932831017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2c8103120 0x7ff2c81996d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8000c00 tx=0x7ff2b80047b0 comp rx=0 tx=0).stop 2024-04-25T10:32:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.265+0000 7ff2ce239700 1 -- 172.21.15.43:0/1932831017 shutdown_connections 2024-04-25T10:32:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.265+0000 7ff2ce239700 1 --2- 172.21.15.43:0/1932831017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2c8104bf0 0x7ff2c81a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.265+0000 7ff2ce239700 1 --2- 172.21.15.43:0/1932831017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2c8103120 0x7ff2c81996d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.265+0000 7ff2ce239700 1 --2- 172.21.15.43:0/1932831017 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff2b00afb80 0x7ff2b00b2090 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.265+0000 7ff2ce239700 1 --2- 172.21.15.43:0/1932831017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2c8104250 0x7ff2c8199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.265+0000 7ff2ce239700 1 -- 172.21.15.43:0/1932831017 >> 172.21.15.43:0/1932831017 conn(0x7ff2c80fe760 msgr2=0x7ff2c8107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.265+0000 7ff2ce239700 1 -- 172.21.15.43:0/1932831017 shutdown_connections 2024-04-25T10:32:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:13.265+0000 7ff2ce239700 1 -- 172.21.15.43:0/1932831017 wait complete. 2024-04-25T10:32:13.945 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T10:32:13.954 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T10:32:13.954 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T10:32:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:32:14.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:13 smithi077 ceph-mon[32772]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T10:32:14.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:13 smithi164 ceph-mon[32713]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T10:32:14.329 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:13 smithi043 ceph-mon[29478]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T10:32:15.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:14 smithi043 ceph-mon[29478]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:15.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:14 smithi077 ceph-mon[32772]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:15.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:14 smithi164 ceph-mon[32713]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:16.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.009+0000 7ff79a725700 1 -- 172.21.15.43:0/2014179691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff794104240 msgr2=0x7ff794104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:16.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.009+0000 7ff79a725700 1 --2- 172.21.15.43:0/2014179691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff794104240 0x7ff794104650 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff788009990 tx=0x7ff788009ca0 comp rx=0 tx=0).stop 2024-04-25T10:32:16.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.009+0000 7ff79a725700 1 -- 172.21.15.43:0/2014179691 shutdown_connections 2024-04-25T10:32:16.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.009+0000 7ff79a725700 1 --2- 172.21.15.43:0/2014179691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff794104be0 0x7ff79410b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:16.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.010+0000 7ff79a725700 1 --2- 172.21.15.43:0/2014179691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff794103110 0x7ff794103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:16.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.010+0000 7ff79a725700 1 --2- 172.21.15.43:0/2014179691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff794104240 0x7ff794104650 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:16.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.010+0000 7ff79a725700 1 -- 172.21.15.43:0/2014179691 >> 172.21.15.43:0/2014179691 conn(0x7ff7940fe750 msgr2=0x7ff794100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:16.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.010+0000 7ff79a725700 1 -- 172.21.15.43:0/2014179691 shutdown_connections 2024-04-25T10:32:16.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.010+0000 7ff79a725700 1 -- 172.21.15.43:0/2014179691 wait complete. 2024-04-25T10:32:16.013 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:15 smithi043 ceph-mon[29478]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T10:32:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.010+0000 7ff79a725700 1 Processor -- start 2024-04-25T10:32:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.010+0000 7ff79a725700 1 -- start start 2024-04-25T10:32:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.011+0000 7ff79a725700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff794103110 0x7ff79410da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.011+0000 7ff79a725700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff794104240 0x7ff79410c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.011+0000 7ff79a725700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff794104be0 0x7ff79410c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.011+0000 7ff79a725700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff79410fd40 con 0x7ff794104240 2024-04-25T10:32:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.011+0000 7ff79a725700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff79410fbc0 con 0x7ff794103110 2024-04-25T10:32:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.011+0000 7ff79a725700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff79410fec0 con 0x7ff794104be0 2024-04-25T10:32:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.011+0000 7ff7937fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff794104240 0x7ff79410c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.011+0000 7ff7937fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff794104240 0x7ff79410c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55096/0 (socket says 172.21.15.43:55096) 2024-04-25T10:32:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.011+0000 7ff7937fe700 1 -- 172.21.15.43:0/2287261047 learned_addr learned my addr 172.21.15.43:0/2287261047 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.011+0000 7ff793fff700 1 --2- 172.21.15.43:0/2287261047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff794103110 0x7ff79410da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.011+0000 7ff798cc2700 1 --2- 172.21.15.43:0/2287261047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff794104be0 0x7ff79410c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.011+0000 7ff7937fe700 1 -- 172.21.15.43:0/2287261047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff794104be0 msgr2=0x7ff79410c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.011+0000 7ff7937fe700 1 --2- 172.21.15.43:0/2287261047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff794104be0 0x7ff79410c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.011+0000 7ff7937fe700 1 -- 172.21.15.43:0/2287261047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff794103110 msgr2=0x7ff79410da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.011+0000 7ff7937fe700 1 --2- 172.21.15.43:0/2287261047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff794103110 0x7ff79410da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.011+0000 7ff7937fe700 1 -- 172.21.15.43:0/2287261047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79410cd10 con 0x7ff794104240 2024-04-25T10:32:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.011+0000 7ff793fff700 1 --2- 172.21.15.43:0/2287261047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff794103110 0x7ff79410da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.012+0000 7ff7937fe700 1 --2- 172.21.15.43:0/2287261047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff794104240 0x7ff79410c100 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff788006010 tx=0x7ff788009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.012+0000 7ff7917fa700 1 -- 172.21.15.43:0/2287261047 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788027070 con 0x7ff794104240 2024-04-25T10:32:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.012+0000 7ff79a725700 1 -- 172.21.15.43:0/2287261047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7941a4950 con 0x7ff794104240 2024-04-25T10:32:16.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.012+0000 7ff7917fa700 1 -- 172.21.15.43:0/2287261047 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff788022470 con 0x7ff794104240 2024-04-25T10:32:16.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.012+0000 7ff7917fa700 1 -- 172.21.15.43:0/2287261047 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78800f6c0 con 0x7ff794104240 2024-04-25T10:32:16.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.012+0000 7ff79a725700 1 -- 172.21.15.43:0/2287261047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7941a4d70 con 0x7ff794104240 2024-04-25T10:32:16.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.013+0000 7ff79a725700 1 -- 172.21.15.43:0/2287261047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff794051190 con 0x7ff794104240 2024-04-25T10:32:16.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.015+0000 7ff7917fa700 1 -- 172.21.15.43:0/2287261047 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff78802a070 con 0x7ff794104240 2024-04-25T10:32:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.016+0000 7ff7917fa700 1 --2- 172.21.15.43:0/2287261047 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff77c0afc00 0x7ff77c0b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.016+0000 7ff793fff700 1 --2- 172.21.15.43:0/2287261047 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff77c0afc00 0x7ff77c0b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.016+0000 7ff7917fa700 1 -- 172.21.15.43:0/2287261047 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7ff7880dffd0 con 0x7ff794104240 2024-04-25T10:32:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.016+0000 7ff793fff700 1 --2- 172.21.15.43:0/2287261047 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff77c0afc00 0x7ff77c0b2110 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff794104140 tx=0x7ff78400a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:16.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.020+0000 7ff7917fa700 1 -- 172.21.15.43:0/2287261047 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff788031080 con 0x7ff794104240 2024-04-25T10:32:16.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.275+0000 7ff79a725700 1 -- 172.21.15.43:0/2287261047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7ff79402e510 con 0x7ff794104240 2024-04-25T10:32:16.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.275+0000 7ff7917fa700 1 -- 172.21.15.43:0/2287261047 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) ==== 76+0+3048 (secure 0 0 0) 0x7ff788013070 con 0x7ff794104240 2024-04-25T10:32:16.277 INFO:teuthology.orchestra.run.smithi043.stdout:e23 2024-04-25T10:32:16.277 INFO:teuthology.orchestra.run.smithi043.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T10:32:16.277 INFO:teuthology.orchestra.run.smithi043.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T10:32:16.278 INFO:teuthology.orchestra.run.smithi043.stdout:legacy client fscid: 1 2024-04-25T10:32:16.278 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:32:16.278 INFO:teuthology.orchestra.run.smithi043.stdout:Filesystem 'cephfs' (1) 2024-04-25T10:32:16.278 INFO:teuthology.orchestra.run.smithi043.stdout:fs_name cephfs 2024-04-25T10:32:16.278 INFO:teuthology.orchestra.run.smithi043.stdout:epoch 23 2024-04-25T10:32:16.278 INFO:teuthology.orchestra.run.smithi043.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T10:32:16.278 INFO:teuthology.orchestra.run.smithi043.stdout:created 2024-04-25T10:30:33.261560+0000 2024-04-25T10:32:16.278 INFO:teuthology.orchestra.run.smithi043.stdout:modified 2024-04-25T10:30:38.804706+0000 2024-04-25T10:32:16.278 INFO:teuthology.orchestra.run.smithi043.stdout:tableserver 0 2024-04-25T10:32:16.278 INFO:teuthology.orchestra.run.smithi043.stdout:root 0 2024-04-25T10:32:16.278 INFO:teuthology.orchestra.run.smithi043.stdout:session_timeout 300 2024-04-25T10:32:16.278 INFO:teuthology.orchestra.run.smithi043.stdout:session_autoclose 300 2024-04-25T10:32:16.278 INFO:teuthology.orchestra.run.smithi043.stdout:max_file_size 1099511627776 2024-04-25T10:32:16.278 INFO:teuthology.orchestra.run.smithi043.stdout:max_xattr_size 65536 2024-04-25T10:32:16.279 INFO:teuthology.orchestra.run.smithi043.stdout:required_client_features {} 2024-04-25T10:32:16.279 INFO:teuthology.orchestra.run.smithi043.stdout:last_failure 0 2024-04-25T10:32:16.279 INFO:teuthology.orchestra.run.smithi043.stdout:last_failure_osd_epoch 0 2024-04-25T10:32:16.279 INFO:teuthology.orchestra.run.smithi043.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T10:32:16.279 INFO:teuthology.orchestra.run.smithi043.stdout:max_mds 3 2024-04-25T10:32:16.279 INFO:teuthology.orchestra.run.smithi043.stdout:in 0,1,2 2024-04-25T10:32:16.279 INFO:teuthology.orchestra.run.smithi043.stdout:up {0=24554,1=24508,2=24472} 2024-04-25T10:32:16.279 INFO:teuthology.orchestra.run.smithi043.stdout:failed 2024-04-25T10:32:16.279 INFO:teuthology.orchestra.run.smithi043.stdout:damaged 2024-04-25T10:32:16.279 INFO:teuthology.orchestra.run.smithi043.stdout:stopped 2024-04-25T10:32:16.279 INFO:teuthology.orchestra.run.smithi043.stdout:data_pools [3] 2024-04-25T10:32:16.279 INFO:teuthology.orchestra.run.smithi043.stdout:metadata_pool 2 2024-04-25T10:32:16.279 INFO:teuthology.orchestra.run.smithi043.stdout:inline_data disabled 2024-04-25T10:32:16.279 INFO:teuthology.orchestra.run.smithi043.stdout:balancer 2024-04-25T10:32:16.280 INFO:teuthology.orchestra.run.smithi043.stdout:bal_rank_mask -1 2024-04-25T10:32:16.280 INFO:teuthology.orchestra.run.smithi043.stdout:standby_count_wanted 0 2024-04-25T10:32:16.280 INFO:teuthology.orchestra.run.smithi043.stdout:qdb_cluster leader: 24554 members: 24472,24508,24554 2024-04-25T10:32:16.280 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.b{0:24554} state up:active seq 3 addr [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:32:16.280 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.h{0:24503} state up:standby-replay seq 1 addr [v2:172.21.15.77:6836/2595248466,v1:172.21.15.77:6837/2595248466] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:32:16.280 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.i{1:24508} state up:active seq 6 addr [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:32:16.280 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.c{1:24484} state up:standby-replay seq 1 addr [v2:172.21.15.164:6836/3305330947,v1:172.21.15.164:6837/3305330947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:32:16.280 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.f{2:24472} state up:active seq 8 addr [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:32:16.280 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.e{2:24470} state up:standby-replay seq 1 addr [v2:172.21.15.77:6834/1783762538,v1:172.21.15.77:6835/1783762538] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:32:16.280 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:32:16.280 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:32:16.280 INFO:teuthology.orchestra.run.smithi043.stdout:Standby daemons: 2024-04-25T10:32:16.280 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:32:16.280 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.j{-1:14547} state up:standby seq 1 addr [v2:172.21.15.43:6834/3551971560,v1:172.21.15.43:6835/3551971560] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:32:16.281 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.d{-1:14565} state up:standby seq 1 addr [v2:172.21.15.43:6836/2500994561,v1:172.21.15.43:6837/2500994561] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:32:16.281 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.g{-1:14637} state up:standby seq 1 addr [v2:172.21.15.43:6838/3923057448,v1:172.21.15.43:6839/3923057448] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:32:16.281 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.a{-1:14649} state up:standby seq 1 addr [v2:172.21.15.43:6840/4254265277,v1:172.21.15.43:6841/4254265277] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:32:16.281 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.k{-1:24434} state up:standby seq 1 addr [v2:172.21.15.77:6832/2295020829,v1:172.21.15.77:6833/2295020829] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:32:16.281 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.l{-1:24436} state up:standby seq 1 addr [v2:172.21.15.164:6832/3601949896,v1:172.21.15.164:6833/3601949896] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T10:32:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.277+0000 7ff79a725700 1 -- 172.21.15.43:0/2287261047 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff77c0afc00 msgr2=0x7ff77c0b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.277+0000 7ff79a725700 1 --2- 172.21.15.43:0/2287261047 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff77c0afc00 0x7ff77c0b2110 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff794104140 tx=0x7ff78400a680 comp rx=0 tx=0).stop 2024-04-25T10:32:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.277+0000 7ff79a725700 1 -- 172.21.15.43:0/2287261047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff794104240 msgr2=0x7ff79410c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.277+0000 7ff79a725700 1 --2- 172.21.15.43:0/2287261047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff794104240 0x7ff79410c100 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff788006010 tx=0x7ff788009f90 comp rx=0 tx=0).stop 2024-04-25T10:32:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.277+0000 7ff79a725700 1 -- 172.21.15.43:0/2287261047 shutdown_connections 2024-04-25T10:32:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.277+0000 7ff79a725700 1 --2- 172.21.15.43:0/2287261047 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7ff77c0afc00 0x7ff77c0b2110 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.277+0000 7ff79a725700 1 --2- 172.21.15.43:0/2287261047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff794104be0 0x7ff79410c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.277+0000 7ff79a725700 1 --2- 172.21.15.43:0/2287261047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff794103110 0x7ff79410da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.278+0000 7ff79a725700 1 --2- 172.21.15.43:0/2287261047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff794104240 0x7ff79410c100 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.278+0000 7ff79a725700 1 -- 172.21.15.43:0/2287261047 >> 172.21.15.43:0/2287261047 conn(0x7ff7940fe750 msgr2=0x7ff794107470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.278+0000 7ff79a725700 1 -- 172.21.15.43:0/2287261047 shutdown_connections 2024-04-25T10:32:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:16.278+0000 7ff79a725700 1 -- 172.21.15.43:0/2287261047 wait complete. 2024-04-25T10:32:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-04-25T10:32:16.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:15 smithi077 ceph-mon[32772]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T10:32:16.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:15 smithi164 ceph-mon[32713]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T10:32:16.935 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T10:32:17.186 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:32:17.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:16 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2287261047' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T10:32:17.290 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:16 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2287261047' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T10:32:17.309 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:16 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2287261047' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T10:32:18.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:17 smithi077 ceph-mon[32772]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T10:32:18.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:17 smithi164 ceph-mon[32713]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T10:32:18.309 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:17 smithi043 ceph-mon[29478]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T10:32:18.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.844+0000 7fd0c8118700 1 -- 172.21.15.43:0/2924717752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0c01030e0 msgr2=0x7fd0c01034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:18.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.844+0000 7fd0c8118700 1 --2- 172.21.15.43:0/2924717752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0c01030e0 0x7fd0c01034f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd0b000b0a0 tx=0x7fd0b000b3b0 comp rx=0 tx=0).stop 2024-04-25T10:32:18.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.844+0000 7fd0c8118700 1 -- 172.21.15.43:0/2924717752 shutdown_connections 2024-04-25T10:32:18.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.844+0000 7fd0c8118700 1 --2- 172.21.15.43:0/2924717752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0c0104bb0 0x7fd0c010b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:18.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.844+0000 7fd0c8118700 1 --2- 172.21.15.43:0/2924717752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0c01030e0 0x7fd0c01034f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:18.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.844+0000 7fd0c8118700 1 --2- 172.21.15.43:0/2924717752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0c0104210 0x7fd0c0104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:18.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.844+0000 7fd0c8118700 1 -- 172.21.15.43:0/2924717752 >> 172.21.15.43:0/2924717752 conn(0x7fd0c00fe740 msgr2=0x7fd0c0100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:18.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.844+0000 7fd0c8118700 1 -- 172.21.15.43:0/2924717752 shutdown_connections 2024-04-25T10:32:18.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.844+0000 7fd0c8118700 1 -- 172.21.15.43:0/2924717752 wait complete. 2024-04-25T10:32:18.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.845+0000 7fd0c8118700 1 Processor -- start 2024-04-25T10:32:18.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.845+0000 7fd0c8118700 1 -- start start 2024-04-25T10:32:18.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.845+0000 7fd0c8118700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0c01030e0 0x7fd0c019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:18.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.845+0000 7fd0c8118700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0c0104210 0x7fd0c019ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:18.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.845+0000 7fd0c8118700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0c0104bb0 0x7fd0c01a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:18.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.845+0000 7fd0c8118700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c010fcc0 con 0x7fd0c0104bb0 2024-04-25T10:32:18.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.845+0000 7fd0c8118700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c010fb40 con 0x7fd0c0104210 2024-04-25T10:32:18.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.845+0000 7fd0c8118700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c010fe40 con 0x7fd0c01030e0 2024-04-25T10:32:18.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.845+0000 7fd0c66b5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0c0104bb0 0x7fd0c01a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:18.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.845+0000 7fd0c66b5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0c0104bb0 0x7fd0c01a1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39704/0 (socket says 172.21.15.43:39704) 2024-04-25T10:32:18.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.845+0000 7fd0c66b5700 1 -- 172.21.15.43:0/3466036940 learned_addr learned my addr 172.21.15.43:0/3466036940 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:18.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.845+0000 7fd0c5eb4700 1 --2- 172.21.15.43:0/3466036940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0c01030e0 0x7fd0c019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:18.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.845+0000 7fd0c56b3700 1 --2- 172.21.15.43:0/3466036940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0c0104210 0x7fd0c019ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:18.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.846+0000 7fd0c66b5700 1 -- 172.21.15.43:0/3466036940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0c01030e0 msgr2=0x7fd0c019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:18.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.846+0000 7fd0c66b5700 1 --2- 172.21.15.43:0/3466036940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0c01030e0 0x7fd0c019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:18.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.846+0000 7fd0c66b5700 1 -- 172.21.15.43:0/3466036940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0c0104210 msgr2=0x7fd0c019ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:18.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.846+0000 7fd0c66b5700 1 --2- 172.21.15.43:0/3466036940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0c0104210 0x7fd0c019ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:18.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.846+0000 7fd0c66b5700 1 -- 172.21.15.43:0/3466036940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c01a2320 con 0x7fd0c0104bb0 2024-04-25T10:32:18.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.846+0000 7fd0c66b5700 1 --2- 172.21.15.43:0/3466036940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0c0104bb0 0x7fd0c01a1ca0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800c970 tx=0x7fd0b800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:18.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.846+0000 7fd0beffd700 1 -- 172.21.15.43:0/3466036940 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8004a30 con 0x7fd0c0104bb0 2024-04-25T10:32:18.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.846+0000 7fd0c8118700 1 -- 172.21.15.43:0/3466036940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c01a2610 con 0x7fd0c0104bb0 2024-04-25T10:32:18.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.846+0000 7fd0beffd700 1 -- 172.21.15.43:0/3466036940 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0b80054d0 con 0x7fd0c0104bb0 2024-04-25T10:32:18.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.846+0000 7fd0beffd700 1 -- 172.21.15.43:0/3466036940 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b800ec20 con 0x7fd0c0104bb0 2024-04-25T10:32:18.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.846+0000 7fd0c8118700 1 -- 172.21.15.43:0/3466036940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c01a2b00 con 0x7fd0c0104bb0 2024-04-25T10:32:18.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.847+0000 7fd0c8118700 1 -- 172.21.15.43:0/3466036940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0c0051190 con 0x7fd0c0104bb0 2024-04-25T10:32:18.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.849+0000 7fd0beffd700 1 -- 172.21.15.43:0/3466036940 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd0b8004fc0 con 0x7fd0c0104bb0 2024-04-25T10:32:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.849+0000 7fd0beffd700 1 --2- 172.21.15.43:0/3466036940 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd0a80afb80 0x7fd0a80b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.850+0000 7fd0c5eb4700 1 --2- 172.21.15.43:0/3466036940 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd0a80afb80 0x7fd0a80b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.850+0000 7fd0beffd700 1 -- 172.21.15.43:0/3466036940 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fd0b80deae0 con 0x7fd0c0104bb0 2024-04-25T10:32:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.850+0000 7fd0c5eb4700 1 --2- 172.21.15.43:0/3466036940 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd0a80afb80 0x7fd0a80b2090 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009fd0 tx=0x7fd0b000b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:18.853+0000 7fd0beffd700 1 -- 172.21.15.43:0/3466036940 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd0b8014030 con 0x7fd0c0104bb0 2024-04-25T10:32:19.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.067+0000 7fd0c8118700 1 -- 172.21.15.43:0/3466036940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fd0c00642a0 con 0x7fd0c0104bb0 2024-04-25T10:32:19.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.068+0000 7fd0beffd700 1 -- 172.21.15.43:0/3466036940 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v79) ==== 56+0+5274 (secure 0 0 0) 0x7fd0b80a76d0 con 0x7fd0c0104bb0 2024-04-25T10:32:19.070 INFO:teuthology.orchestra.run.smithi043.stdout:epoch 79 2024-04-25T10:32:19.070 INFO:teuthology.orchestra.run.smithi043.stdout:fsid 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:32:19.070 INFO:teuthology.orchestra.run.smithi043.stdout:created 2024-04-25T10:22:37.861853+0000 2024-04-25T10:32:19.070 INFO:teuthology.orchestra.run.smithi043.stdout:modified 2024-04-25T10:31:33.928341+0000 2024-04-25T10:32:19.070 INFO:teuthology.orchestra.run.smithi043.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T10:32:19.070 INFO:teuthology.orchestra.run.smithi043.stdout:crush_version 26 2024-04-25T10:32:19.070 INFO:teuthology.orchestra.run.smithi043.stdout:full_ratio 0.95 2024-04-25T10:32:19.070 INFO:teuthology.orchestra.run.smithi043.stdout:backfillfull_ratio 0.9 2024-04-25T10:32:19.070 INFO:teuthology.orchestra.run.smithi043.stdout:nearfull_ratio 0.85 2024-04-25T10:32:19.070 INFO:teuthology.orchestra.run.smithi043.stdout:require_min_compat_client luminous 2024-04-25T10:32:19.071 INFO:teuthology.orchestra.run.smithi043.stdout:min_compat_client luminous 2024-04-25T10:32:19.071 INFO:teuthology.orchestra.run.smithi043.stdout:require_osd_release squid 2024-04-25T10:32:19.071 INFO:teuthology.orchestra.run.smithi043.stdout:stretch_mode_enabled false 2024-04-25T10:32:19.071 INFO:teuthology.orchestra.run.smithi043.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T10:32:19.071 INFO:teuthology.orchestra.run.smithi043.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 76 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T10:32:19.071 INFO:teuthology.orchestra.run.smithi043.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 77 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T10:32:19.071 INFO:teuthology.orchestra.run.smithi043.stdout:max_osd 12 2024-04-25T10:32:19.071 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0 up in weight 1 up_from 8 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6802/3598858946,v1:172.21.15.43:6803/3598858946] [v2:172.21.15.43:6804/3598858946,v1:172.21.15.43:6805/3598858946] exists,up 05115646-db4d-4cfc-a5df-867435ee79ac 2024-04-25T10:32:19.071 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1 up in weight 1 up_from 13 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6800/3362129259,v1:172.21.15.77:6801/3362129259] [v2:172.21.15.77:6802/3362129259,v1:172.21.15.77:6803/3362129259] exists,up a8e23410-bf31-4a5e-a7dc-d99673208270 2024-04-25T10:32:19.071 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2 up in weight 1 up_from 18 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6800/3408162352,v1:172.21.15.164:6801/3408162352] [v2:172.21.15.164:6802/3408162352,v1:172.21.15.164:6803/3408162352] exists,up e85413f8-524b-41ed-8ab6-d811d2ed40ed 2024-04-25T10:32:19.071 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3 up in weight 1 up_from 25 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6810/2368600010,v1:172.21.15.43:6811/2368600010] [v2:172.21.15.43:6812/2368600010,v1:172.21.15.43:6813/2368600010] exists,up 584d77ca-fb6f-4b35-9a11-2d2827de6dc8 2024-04-25T10:32:19.071 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4 up in weight 1 up_from 31 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6808/3648991803,v1:172.21.15.77:6809/3648991803] [v2:172.21.15.77:6810/3648991803,v1:172.21.15.77:6811/3648991803] exists,up 8cf2e39e-255f-4410-aee3-938b0e19dff6 2024-04-25T10:32:19.071 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5 up in weight 1 up_from 36 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6808/853209434,v1:172.21.15.164:6809/853209434] [v2:172.21.15.164:6810/853209434,v1:172.21.15.164:6811/853209434] exists,up cb7aa97e-5dde-4866-9c74-547e4ebdc619 2024-04-25T10:32:19.071 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6 up in weight 1 up_from 42 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6818/1494696172,v1:172.21.15.43:6819/1494696172] [v2:172.21.15.43:6820/1494696172,v1:172.21.15.43:6821/1494696172] exists,up 1e8aba2c-984b-4570-8900-dd88f3e5774a 2024-04-25T10:32:19.072 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7 up in weight 1 up_from 48 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6816/1331976864,v1:172.21.15.77:6817/1331976864] [v2:172.21.15.77:6818/1331976864,v1:172.21.15.77:6819/1331976864] exists,up e0d47c22-d7bb-4566-802b-bca38d24b04e 2024-04-25T10:32:19.072 INFO:teuthology.orchestra.run.smithi043.stdout:osd.8 up in weight 1 up_from 54 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6816/3396063471,v1:172.21.15.164:6817/3396063471] [v2:172.21.15.164:6818/3396063471,v1:172.21.15.164:6819/3396063471] exists,up 29f691f2-fb63-4c00-9d78-3f326bd72b3a 2024-04-25T10:32:19.072 INFO:teuthology.orchestra.run.smithi043.stdout:osd.9 up in weight 1 up_from 60 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6826/537180413,v1:172.21.15.43:6827/537180413] [v2:172.21.15.43:6828/537180413,v1:172.21.15.43:6829/537180413] exists,up a77c9906-5352-4b5d-b18b-22d586109892 2024-04-25T10:32:19.072 INFO:teuthology.orchestra.run.smithi043.stdout:osd.10 up in weight 1 up_from 66 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6824/1788538313,v1:172.21.15.77:6825/1788538313] [v2:172.21.15.77:6826/1788538313,v1:172.21.15.77:6827/1788538313] exists,up f728b6e7-cb77-4883-8b80-152cfa4a712c 2024-04-25T10:32:19.072 INFO:teuthology.orchestra.run.smithi043.stdout:osd.11 up in weight 1 up_from 72 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6824/1085943743,v1:172.21.15.164:6825/1085943743] [v2:172.21.15.164:6826/1085943743,v1:172.21.15.164:6827/1085943743] exists,up f6b51b3f-75f3-4864-bfd9-fe42814e8bfd 2024-04-25T10:32:19.072 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 2.a [2,11] 2024-04-25T10:32:19.072 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 2.f [8,11] 2024-04-25T10:32:19.072 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 2.14 [2,4] 2024-04-25T10:32:19.072 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.10 [5,9] 2024-04-25T10:32:19.072 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.17 [4,8] 2024-04-25T10:32:19.072 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.21 [4,6] 2024-04-25T10:32:19.072 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.29 [3,9] 2024-04-25T10:32:19.072 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.3c [4,9] 2024-04-25T10:32:19.072 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/4063490531 expires 2024-04-26T10:23:32.072095+0000 2024-04-25T10:32:19.073 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/1477725871 expires 2024-04-26T10:23:32.072095+0000 2024-04-25T10:32:19.073 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6800/2568088261 expires 2024-04-26T10:23:05.121271+0000 2024-04-25T10:32:19.073 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/3381426507 expires 2024-04-26T10:23:05.121271+0000 2024-04-25T10:32:19.073 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6800/1017216348 expires 2024-04-26T10:23:32.072095+0000 2024-04-25T10:32:19.073 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/1545240575 expires 2024-04-26T10:23:05.121271+0000 2024-04-25T10:32:19.073 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6801/2568088261 expires 2024-04-26T10:23:05.121271+0000 2024-04-25T10:32:19.073 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/4255994574 expires 2024-04-26T10:23:05.121271+0000 2024-04-25T10:32:19.073 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6801/1017216348 expires 2024-04-26T10:23:32.072095+0000 2024-04-25T10:32:19.073 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/3837436213 expires 2024-04-26T10:23:32.072095+0000 2024-04-25T10:32:19.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.070+0000 7fd0c8118700 1 -- 172.21.15.43:0/3466036940 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd0a80afb80 msgr2=0x7fd0a80b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:19.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.070+0000 7fd0c8118700 1 --2- 172.21.15.43:0/3466036940 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd0a80afb80 0x7fd0a80b2090 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009fd0 tx=0x7fd0b000b7b0 comp rx=0 tx=0).stop 2024-04-25T10:32:19.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.070+0000 7fd0c8118700 1 -- 172.21.15.43:0/3466036940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0c0104bb0 msgr2=0x7fd0c01a1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:19.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.070+0000 7fd0c8118700 1 --2- 172.21.15.43:0/3466036940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0c0104bb0 0x7fd0c01a1ca0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800c970 tx=0x7fd0b800cc80 comp rx=0 tx=0).stop 2024-04-25T10:32:19.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.070+0000 7fd0c8118700 1 -- 172.21.15.43:0/3466036940 shutdown_connections 2024-04-25T10:32:19.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.070+0000 7fd0c8118700 1 --2- 172.21.15.43:0/3466036940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0c0104bb0 0x7fd0c01a1ca0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:19.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.070+0000 7fd0c8118700 1 --2- 172.21.15.43:0/3466036940 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd0a80afb80 0x7fd0a80b2090 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:19.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.070+0000 7fd0c8118700 1 --2- 172.21.15.43:0/3466036940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0c01030e0 0x7fd0c019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:19.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.070+0000 7fd0c8118700 1 --2- 172.21.15.43:0/3466036940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0c0104210 0x7fd0c019ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:19.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.070+0000 7fd0c8118700 1 -- 172.21.15.43:0/3466036940 >> 172.21.15.43:0/3466036940 conn(0x7fd0c00fe740 msgr2=0x7fd0c0107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:19.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.070+0000 7fd0c8118700 1 -- 172.21.15.43:0/3466036940 shutdown_connections 2024-04-25T10:32:19.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.070+0000 7fd0c8118700 1 -- 172.21.15.43:0/3466036940 wait complete. 2024-04-25T10:32:19.775 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T10:32:19.787 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:32:19.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.908+0000 7fdd97c67640 1 -- 172.21.15.43:0/3361291044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9010b9c0 msgr2=0x7fdd9010ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:19.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.908+0000 7fdd97c67640 1 --2- 172.21.15.43:0/3361291044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9010b9c0 0x7fdd9010ded0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009ca0 tx=0x7fdd8002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:32:19.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.908+0000 7fdd97c67640 1 -- 172.21.15.43:0/3361291044 shutdown_connections 2024-04-25T10:32:19.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.908+0000 7fdd97c67640 1 --2- 172.21.15.43:0/3361291044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd90108e90 0x7fdd9010b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:19.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.908+0000 7fdd97c67640 1 --2- 172.21.15.43:0/3361291044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9010b9c0 0x7fdd9010ded0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:19.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.908+0000 7fdd97c67640 1 --2- 172.21.15.43:0/3361291044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd90103fe0 0x7fdd901043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:19.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.908+0000 7fdd97c67640 1 -- 172.21.15.43:0/3361291044 >> 172.21.15.43:0/3361291044 conn(0x7fdd900fd360 msgr2=0x7fdd900ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:19.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.908+0000 7fdd97c67640 1 -- 172.21.15.43:0/3361291044 shutdown_connections 2024-04-25T10:32:19.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.908+0000 7fdd97c67640 1 -- 172.21.15.43:0/3361291044 wait complete. 2024-04-25T10:32:19.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.909+0000 7fdd97c67640 1 Processor -- start 2024-04-25T10:32:19.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.909+0000 7fdd97c67640 1 -- start start 2024-04-25T10:32:19.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.909+0000 7fdd97c67640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd90103fe0 0x7fdd9019c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:19.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.909+0000 7fdd97c67640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90108e90 0x7fdd9019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:19.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.909+0000 7fdd97c67640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd9010b9c0 0x7fdd901a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:19.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.909+0000 7fdd97c67640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901118b0 con 0x7fdd90108e90 2024-04-25T10:32:19.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.909+0000 7fdd97c67640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdd90111730 con 0x7fdd90103fe0 2024-04-25T10:32:19.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.909+0000 7fdd97c67640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdd90111a30 con 0x7fdd9010b9c0 2024-04-25T10:32:19.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.909+0000 7fdd95202640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90108e90 0x7fdd9019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:19.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.910+0000 7fdd95202640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90108e90 0x7fdd9019c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39714/0 (socket says 172.21.15.43:39714) 2024-04-25T10:32:19.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.910+0000 7fdd95202640 1 -- 172.21.15.43:0/1977350287 learned_addr learned my addr 172.21.15.43:0/1977350287 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:19.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.910+0000 7fdd95a03640 1 --2- 172.21.15.43:0/1977350287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd90103fe0 0x7fdd9019c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:19.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.910+0000 7fdd96204640 1 --2- 172.21.15.43:0/1977350287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd9010b9c0 0x7fdd901a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:19.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.910+0000 7fdd95202640 1 -- 172.21.15.43:0/1977350287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd9010b9c0 msgr2=0x7fdd901a3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:19.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.910+0000 7fdd95202640 1 --2- 172.21.15.43:0/1977350287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd9010b9c0 0x7fdd901a3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:19.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.910+0000 7fdd95202640 1 -- 172.21.15.43:0/1977350287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd90103fe0 msgr2=0x7fdd9019c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:19.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.910+0000 7fdd95202640 1 --2- 172.21.15.43:0/1977350287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd90103fe0 0x7fdd9019c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:19.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.910+0000 7fdd95202640 1 -- 172.21.15.43:0/1977350287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd901a3e90 con 0x7fdd90108e90 2024-04-25T10:32:19.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.910+0000 7fdd95202640 1 --2- 172.21.15.43:0/1977350287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90108e90 0x7fdd9019c890 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400e9f0 tx=0x7fdd8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:19.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.910+0000 7fdd8effd640 1 -- 172.21.15.43:0/1977350287 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8400cd80 con 0x7fdd90108e90 2024-04-25T10:32:19.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.910+0000 7fdd97c67640 1 -- 172.21.15.43:0/1977350287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd901a4180 con 0x7fdd90108e90 2024-04-25T10:32:19.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.910+0000 7fdd8effd640 1 -- 172.21.15.43:0/1977350287 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd840108e0 con 0x7fdd90108e90 2024-04-25T10:32:19.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.910+0000 7fdd8effd640 1 -- 172.21.15.43:0/1977350287 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84018980 con 0x7fdd90108e90 2024-04-25T10:32:19.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.910+0000 7fdd97c67640 1 -- 172.21.15.43:0/1977350287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd901a4720 con 0x7fdd90108e90 2024-04-25T10:32:19.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.911+0000 7fdd97c67640 1 -- 172.21.15.43:0/1977350287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd900fee80 con 0x7fdd90108e90 2024-04-25T10:32:19.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.913+0000 7fdd8effd640 1 -- 172.21.15.43:0/1977350287 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fdd84010430 con 0x7fdd90108e90 2024-04-25T10:32:19.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.914+0000 7fdd8effd640 1 --2- 172.21.15.43:0/1977350287 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fdd780afaf0 0x7fdd780b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:19.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.914+0000 7fdd95a03640 1 --2- 172.21.15.43:0/1977350287 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fdd780afaf0 0x7fdd780b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:19.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.914+0000 7fdd8effd640 1 -- 172.21.15.43:0/1977350287 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fdd840df500 con 0x7fdd90108e90 2024-04-25T10:32:19.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.914+0000 7fdd95a03640 1 --2- 172.21.15.43:0/1977350287 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fdd780afaf0 0x7fdd780b2000 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdd9006b4e0 tx=0x7fdd88005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:19.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:19.918+0000 7fdd8effd640 1 -- 172.21.15.43:0/1977350287 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd840a80f0 con 0x7fdd90108e90 2024-04-25T10:32:20.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.106+0000 7fdd97c67640 1 -- 172.21.15.43:0/1977350287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd9002dd20 con 0x7fdd90108e90 2024-04-25T10:32:20.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.108+0000 7fdd8effd640 1 -- 172.21.15.43:0/1977350287 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22619 (secure 0 0 0) 0x7fdd840a79f0 con 0x7fdd90108e90 2024-04-25T10:32:20.110 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:32:20.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:19 smithi043 ceph-mon[29478]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T10:32:20.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:19 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3466036940' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T10:32:20.111 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":79,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:31:33.928341+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:31.503414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:32.224334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:32:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.110+0000 7fdd97c67640 1 -- 172.21.15.43:0/1977350287 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fdd780afaf0 msgr2=0x7fdd780b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.110+0000 7fdd97c67640 1 --2- 172.21.15.43:0/1977350287 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fdd780afaf0 0x7fdd780b2000 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdd9006b4e0 tx=0x7fdd88005f90 comp rx=0 tx=0).stop 2024-04-25T10:32:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.110+0000 7fdd97c67640 1 -- 172.21.15.43:0/1977350287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90108e90 msgr2=0x7fdd9019c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.110+0000 7fdd97c67640 1 --2- 172.21.15.43:0/1977350287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90108e90 0x7fdd9019c890 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400e9f0 tx=0x7fdd8400eeb0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.110+0000 7fdd97c67640 1 -- 172.21.15.43:0/1977350287 shutdown_connections 2024-04-25T10:32:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.110+0000 7fdd97c67640 1 --2- 172.21.15.43:0/1977350287 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fdd780afaf0 0x7fdd780b2000 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.110+0000 7fdd97c67640 1 --2- 172.21.15.43:0/1977350287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd9010b9c0 0x7fdd901a3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.110+0000 7fdd97c67640 1 --2- 172.21.15.43:0/1977350287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90108e90 0x7fdd9019c890 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.110+0000 7fdd97c67640 1 --2- 172.21.15.43:0/1977350287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd90103fe0 0x7fdd9019c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.110+0000 7fdd97c67640 1 -- 172.21.15.43:0/1977350287 >> 172.21.15.43:0/1977350287 conn(0x7fdd900fd360 msgr2=0x7fdd90109990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.110+0000 7fdd97c67640 1 -- 172.21.15.43:0/1977350287 shutdown_connections 2024-04-25T10:32:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.110+0000 7fdd97c67640 1 -- 172.21.15.43:0/1977350287 wait complete. 2024-04-25T10:32:20.124 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:25:58.167476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:31.503414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:32.224334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:32:20.125 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:32:20.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.237+0000 7f2892ebd640 1 -- 172.21.15.43:0/3043155538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f288c1050a0 msgr2=0x7f288c1054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.237+0000 7f2892ebd640 1 --2- 172.21.15.43:0/3043155538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f288c1050a0 0x7f288c1054b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2880009900 tx=0x7f288002a410 comp rx=0 tx=0).stop 2024-04-25T10:32:20.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.237+0000 7f2892ebd640 1 -- 172.21.15.43:0/3043155538 shutdown_connections 2024-04-25T10:32:20.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.237+0000 7f2892ebd640 1 --2- 172.21.15.43:0/3043155538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f288c101750 0x7f288c101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.237+0000 7f2892ebd640 1 --2- 172.21.15.43:0/3043155538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f288c102250 0x7f288c10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.237+0000 7f2892ebd640 1 --2- 172.21.15.43:0/3043155538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f288c1050a0 0x7f288c1054b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.237+0000 7f2892ebd640 1 -- 172.21.15.43:0/3043155538 >> 172.21.15.43:0/3043155538 conn(0x7f288c0fd350 msgr2=0x7f288c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:20.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.237+0000 7f2892ebd640 1 -- 172.21.15.43:0/3043155538 shutdown_connections 2024-04-25T10:32:20.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.237+0000 7f2892ebd640 1 -- 172.21.15.43:0/3043155538 wait complete. 2024-04-25T10:32:20.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.238+0000 7f2892ebd640 1 Processor -- start 2024-04-25T10:32:20.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.238+0000 7f2892ebd640 1 -- start start 2024-04-25T10:32:20.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.238+0000 7f2892ebd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f288c101750 0x7f288c19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.238+0000 7f2892ebd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f288c102250 0x7f288c19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.238+0000 7f2892ebd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f288c1050a0 0x7f288c1a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.238+0000 7f2892ebd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f288c10f670 con 0x7f288c102250 2024-04-25T10:32:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.238+0000 7f2892ebd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f288c10f4f0 con 0x7f288c1050a0 2024-04-25T10:32:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.238+0000 7f2892ebd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f288c10f7f0 con 0x7f288c101750 2024-04-25T10:32:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.238+0000 7f288bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f288c102250 0x7f288c19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.238+0000 7f288bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f288c102250 0x7f288c19a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39728/0 (socket says 172.21.15.43:39728) 2024-04-25T10:32:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.238+0000 7f288bfff640 1 -- 172.21.15.43:0/2676747647 learned_addr learned my addr 172.21.15.43:0/2676747647 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.239+0000 7f2890c59640 1 --2- 172.21.15.43:0/2676747647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f288c101750 0x7f288c19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.239+0000 7f289145a640 1 --2- 172.21.15.43:0/2676747647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f288c1050a0 0x7f288c1a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.239+0000 7f288bfff640 1 -- 172.21.15.43:0/2676747647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f288c101750 msgr2=0x7f288c19a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.239+0000 7f288bfff640 1 --2- 172.21.15.43:0/2676747647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f288c101750 0x7f288c19a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.239+0000 7f288bfff640 1 -- 172.21.15.43:0/2676747647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f288c1050a0 msgr2=0x7f288c1a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.239+0000 7f288bfff640 1 --2- 172.21.15.43:0/2676747647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f288c1050a0 0x7f288c1a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.239+0000 7f288bfff640 1 -- 172.21.15.43:0/2676747647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288c1a1d00 con 0x7f288c102250 2024-04-25T10:32:20.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.239+0000 7f289145a640 1 --2- 172.21.15.43:0/2676747647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f288c1050a0 0x7f288c1a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:32:20.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.239+0000 7f288bfff640 1 --2- 172.21.15.43:0/2676747647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f288c102250 0x7f288c19a820 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f287800c920 tx=0x7f287800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:20.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.239+0000 7f2889ffb640 1 -- 172.21.15.43:0/2676747647 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2878007c70 con 0x7f288c102250 2024-04-25T10:32:20.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.239+0000 7f2889ffb640 1 -- 172.21.15.43:0/2676747647 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2878007e10 con 0x7f288c102250 2024-04-25T10:32:20.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.239+0000 7f2892ebd640 1 -- 172.21.15.43:0/2676747647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f288c1a1ff0 con 0x7f288c102250 2024-04-25T10:32:20.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.239+0000 7f2889ffb640 1 -- 172.21.15.43:0/2676747647 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2878018820 con 0x7f288c102250 2024-04-25T10:32:20.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.239+0000 7f2892ebd640 1 -- 172.21.15.43:0/2676747647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f288c1a2530 con 0x7f288c102250 2024-04-25T10:32:20.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.240+0000 7f2892ebd640 1 -- 172.21.15.43:0/2676747647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f288c108a30 con 0x7f288c102250 2024-04-25T10:32:20.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.242+0000 7f2889ffb640 1 -- 172.21.15.43:0/2676747647 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f287800f450 con 0x7f288c102250 2024-04-25T10:32:20.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.243+0000 7f2889ffb640 1 --2- 172.21.15.43:0/2676747647 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f287c0afb70 0x7f287c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.243+0000 7f2890c59640 1 --2- 172.21.15.43:0/2676747647 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f287c0afb70 0x7f287c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.243+0000 7f2889ffb640 1 -- 172.21.15.43:0/2676747647 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f28780ded50 con 0x7f288c102250 2024-04-25T10:32:20.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.243+0000 7f2890c59640 1 --2- 172.21.15.43:0/2676747647 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f287c0afb70 0x7f287c0b2080 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2880009230 tx=0x7f2880031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:20.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.247+0000 7f2889ffb640 1 -- 172.21.15.43:0/2676747647 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f28780e2030 con 0x7f288c102250 2024-04-25T10:32:20.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:19 smithi077 ceph-mon[32772]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T10:32:20.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:19 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3466036940' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T10:32:20.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:19 smithi164 ceph-mon[32713]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T10:32:20.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:19 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3466036940' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T10:32:20.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.432+0000 7f2892ebd640 1 -- 172.21.15.43:0/2676747647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f288c0637d0 con 0x7f288c102250 2024-04-25T10:32:20.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.433+0000 7f2889ffb640 1 -- 172.21.15.43:0/2676747647 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f2878014030 con 0x7f288c102250 2024-04-25T10:32:20.435 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T10:32:20.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.435+0000 7f2892ebd640 1 -- 172.21.15.43:0/2676747647 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f287c0afb70 msgr2=0x7f287c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.435+0000 7f2892ebd640 1 --2- 172.21.15.43:0/2676747647 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f287c0afb70 0x7f287c0b2080 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2880009230 tx=0x7f2880031400 comp rx=0 tx=0).stop 2024-04-25T10:32:20.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.435+0000 7f2892ebd640 1 -- 172.21.15.43:0/2676747647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f288c102250 msgr2=0x7f288c19a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.435+0000 7f2892ebd640 1 --2- 172.21.15.43:0/2676747647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f288c102250 0x7f288c19a820 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f287800c920 tx=0x7f287800cde0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.435+0000 7f2892ebd640 1 -- 172.21.15.43:0/2676747647 shutdown_connections 2024-04-25T10:32:20.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.435+0000 7f2892ebd640 1 --2- 172.21.15.43:0/2676747647 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f287c0afb70 0x7f287c0b2080 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.435+0000 7f2892ebd640 1 --2- 172.21.15.43:0/2676747647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f288c1050a0 0x7f288c1a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.435+0000 7f2892ebd640 1 --2- 172.21.15.43:0/2676747647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f288c102250 0x7f288c19a820 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.435+0000 7f2892ebd640 1 --2- 172.21.15.43:0/2676747647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f288c101750 0x7f288c19a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.435+0000 7f2892ebd640 1 -- 172.21.15.43:0/2676747647 >> 172.21.15.43:0/2676747647 conn(0x7f288c0fd350 msgr2=0x7f288c105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:20.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.435+0000 7f2892ebd640 1 -- 172.21.15.43:0/2676747647 shutdown_connections 2024-04-25T10:32:20.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.435+0000 7f2892ebd640 1 -- 172.21.15.43:0/2676747647 wait complete. 2024-04-25T10:32:20.448 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:32:20.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.561+0000 7fe34c1e3640 1 -- 172.21.15.43:0/1439396902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3441045d0 msgr2=0x7fe3441069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.561+0000 7fe34c1e3640 1 --2- 172.21.15.43:0/1439396902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3441045d0 0x7fe3441069f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe340009900 tx=0x7fe34002a410 comp rx=0 tx=0).stop 2024-04-25T10:32:20.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.561+0000 7fe34c1e3640 1 -- 172.21.15.43:0/1439396902 shutdown_connections 2024-04-25T10:32:20.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.561+0000 7fe34c1e3640 1 --2- 172.21.15.43:0/1439396902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe344106f30 0x7fe344109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.561+0000 7fe34c1e3640 1 --2- 172.21.15.43:0/1439396902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe344109890 0x7fe34410bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.561+0000 7fe34c1e3640 1 --2- 172.21.15.43:0/1439396902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3441045d0 0x7fe3441069f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.561+0000 7fe34c1e3640 1 -- 172.21.15.43:0/1439396902 >> 172.21.15.43:0/1439396902 conn(0x7fe3440fdbd0 msgr2=0x7fe344100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:20.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.561+0000 7fe34c1e3640 1 -- 172.21.15.43:0/1439396902 shutdown_connections 2024-04-25T10:32:20.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.561+0000 7fe34c1e3640 1 -- 172.21.15.43:0/1439396902 wait complete. 2024-04-25T10:32:20.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.561+0000 7fe34c1e3640 1 Processor -- start 2024-04-25T10:32:20.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.561+0000 7fe34c1e3640 1 -- start start 2024-04-25T10:32:20.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.562+0000 7fe34c1e3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3441045d0 0x7fe344103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.562+0000 7fe34c1e3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe344106f30 0x7fe3441020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.562+0000 7fe34c1e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe344109890 0x7fe344102600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.562+0000 7fe34c1e3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe34410f5c0 con 0x7fe344109890 2024-04-25T10:32:20.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.562+0000 7fe34c1e3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe34410f440 con 0x7fe344106f30 2024-04-25T10:32:20.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.562+0000 7fe34c1e3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe34410f740 con 0x7fe3441045d0 2024-04-25T10:32:20.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.562+0000 7fe34a780640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe344109890 0x7fe344102600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.562+0000 7fe34a780640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe344109890 0x7fe344102600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39748/0 (socket says 172.21.15.43:39748) 2024-04-25T10:32:20.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.562+0000 7fe34a780640 1 -- 172.21.15.43:0/791585529 learned_addr learned my addr 172.21.15.43:0/791585529 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:20.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.562+0000 7fe349f7f640 1 --2- 172.21.15.43:0/791585529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3441045d0 0x7fe344103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.562+0000 7fe34977e640 1 --2- 172.21.15.43:0/791585529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe344106f30 0x7fe3441020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.562+0000 7fe34a780640 1 -- 172.21.15.43:0/791585529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3441045d0 msgr2=0x7fe344103a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.562+0000 7fe34a780640 1 --2- 172.21.15.43:0/791585529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3441045d0 0x7fe344103a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.562+0000 7fe34a780640 1 -- 172.21.15.43:0/791585529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe344106f30 msgr2=0x7fe3441020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.562+0000 7fe34a780640 1 --2- 172.21.15.43:0/791585529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe344106f30 0x7fe3441020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.562+0000 7fe34a780640 1 -- 172.21.15.43:0/791585529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe344102d30 con 0x7fe344109890 2024-04-25T10:32:20.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.562+0000 7fe34977e640 1 --2- 172.21.15.43:0/791585529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe344106f30 0x7fe3441020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:20.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.562+0000 7fe34a780640 1 --2- 172.21.15.43:0/791585529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe344109890 0x7fe344102600 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00dac0 tx=0x7fe33c00df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:20.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.563+0000 7fe33affd640 1 -- 172.21.15.43:0/791585529 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c017070 con 0x7fe344109890 2024-04-25T10:32:20.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.563+0000 7fe34c1e3640 1 -- 172.21.15.43:0/791585529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3441a4660 con 0x7fe344109890 2024-04-25T10:32:20.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.563+0000 7fe33affd640 1 -- 172.21.15.43:0/791585529 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe33c00be30 con 0x7fe344109890 2024-04-25T10:32:20.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.563+0000 7fe33affd640 1 -- 172.21.15.43:0/791585529 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c00f820 con 0x7fe344109890 2024-04-25T10:32:20.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.563+0000 7fe34c1e3640 1 -- 172.21.15.43:0/791585529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3441a4b20 con 0x7fe344109890 2024-04-25T10:32:20.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.564+0000 7fe34c1e3640 1 -- 172.21.15.43:0/791585529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe34406c2a0 con 0x7fe344109890 2024-04-25T10:32:20.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.565+0000 7fe33affd640 1 -- 172.21.15.43:0/791585529 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fe33c010ac0 con 0x7fe344109890 2024-04-25T10:32:20.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.566+0000 7fe33affd640 1 --2- 172.21.15.43:0/791585529 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fe3300afaf0 0x7fe3300b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.566+0000 7fe349f7f640 1 --2- 172.21.15.43:0/791585529 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fe3300afaf0 0x7fe3300b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.566+0000 7fe33affd640 1 -- 172.21.15.43:0/791585529 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fe33c0deb90 con 0x7fe344109890 2024-04-25T10:32:20.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.566+0000 7fe349f7f640 1 --2- 172.21.15.43:0/791585529 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fe3300afaf0 0x7fe3300b2000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe340000c00 tx=0x7fe340031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:20.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.569+0000 7fe33affd640 1 -- 172.21.15.43:0/791585529 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe33c0a7780 con 0x7fe344109890 2024-04-25T10:32:20.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.755+0000 7fe34c1e3640 1 -- 172.21.15.43:0/791585529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe3441a4d80 con 0x7fe344109890 2024-04-25T10:32:20.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.756+0000 7fe33affd640 1 -- 172.21.15.43:0/791585529 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7fe33c0a7080 con 0x7fe344109890 2024-04-25T10:32:20.758 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T10:32:20.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.758+0000 7fe34c1e3640 1 -- 172.21.15.43:0/791585529 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fe3300afaf0 msgr2=0x7fe3300b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.758+0000 7fe34c1e3640 1 --2- 172.21.15.43:0/791585529 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fe3300afaf0 0x7fe3300b2000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe340000c00 tx=0x7fe340031400 comp rx=0 tx=0).stop 2024-04-25T10:32:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.758+0000 7fe34c1e3640 1 -- 172.21.15.43:0/791585529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe344109890 msgr2=0x7fe344102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.758+0000 7fe34c1e3640 1 --2- 172.21.15.43:0/791585529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe344109890 0x7fe344102600 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00dac0 tx=0x7fe33c00df80 comp rx=0 tx=0).stop 2024-04-25T10:32:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.758+0000 7fe34c1e3640 1 -- 172.21.15.43:0/791585529 shutdown_connections 2024-04-25T10:32:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.758+0000 7fe34c1e3640 1 --2- 172.21.15.43:0/791585529 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fe3300afaf0 0x7fe3300b2000 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.758+0000 7fe34c1e3640 1 --2- 172.21.15.43:0/791585529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe344106f30 0x7fe3441020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.758+0000 7fe34c1e3640 1 --2- 172.21.15.43:0/791585529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe344109890 0x7fe344102600 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.758+0000 7fe34c1e3640 1 --2- 172.21.15.43:0/791585529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3441045d0 0x7fe344103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.758+0000 7fe34c1e3640 1 -- 172.21.15.43:0/791585529 >> 172.21.15.43:0/791585529 conn(0x7fe3440fdbd0 msgr2=0x7fe344104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.758+0000 7fe34c1e3640 1 -- 172.21.15.43:0/791585529 shutdown_connections 2024-04-25T10:32:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.758+0000 7fe34c1e3640 1 -- 172.21.15.43:0/791585529 wait complete. 2024-04-25T10:32:20.772 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:32:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.884+0000 7f67fb5f3640 1 -- 172.21.15.43:0/1233129334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67f4103af0 msgr2=0x7f67f41047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.884+0000 7f67fb5f3640 1 --2- 172.21.15.43:0/1233129334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67f4103af0 0x7f67f41047f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f67f0009900 tx=0x7f67f002a410 comp rx=0 tx=0).stop 2024-04-25T10:32:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.884+0000 7f67fb5f3640 1 -- 172.21.15.43:0/1233129334 shutdown_connections 2024-04-25T10:32:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.884+0000 7f67fb5f3640 1 --2- 172.21.15.43:0/1233129334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67f4109a30 0x7f67f410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.884+0000 7f67fb5f3640 1 --2- 172.21.15.43:0/1233129334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67f4104d30 0x7f67f4105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.884+0000 7f67fb5f3640 1 --2- 172.21.15.43:0/1233129334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67f4103af0 0x7f67f41047f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.884+0000 7f67fb5f3640 1 -- 172.21.15.43:0/1233129334 >> 172.21.15.43:0/1233129334 conn(0x7f67f40fece0 msgr2=0x7f67f4101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.884+0000 7f67fb5f3640 1 -- 172.21.15.43:0/1233129334 shutdown_connections 2024-04-25T10:32:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.884+0000 7f67fb5f3640 1 -- 172.21.15.43:0/1233129334 wait complete. 2024-04-25T10:32:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.885+0000 7f67fb5f3640 1 Processor -- start 2024-04-25T10:32:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.885+0000 7f67fb5f3640 1 -- start start 2024-04-25T10:32:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.885+0000 7f67fb5f3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67f4103af0 0x7f67f419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.885+0000 7f67fb5f3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67f4104d30 0x7f67f419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.885+0000 7f67fb5f3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67f4109a30 0x7f67f41a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.885+0000 7f67fb5f3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f67f410f5f0 con 0x7f67f4109a30 2024-04-25T10:32:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.885+0000 7f67fb5f3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f67f410f470 con 0x7f67f4104d30 2024-04-25T10:32:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.885+0000 7f67fb5f3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f67f410f770 con 0x7f67f4103af0 2024-04-25T10:32:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.885+0000 7f67f9b90640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67f4109a30 0x7f67f41a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.885+0000 7f67f9b90640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67f4109a30 0x7f67f41a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39770/0 (socket says 172.21.15.43:39770) 2024-04-25T10:32:20.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.885+0000 7f67f9b90640 1 -- 172.21.15.43:0/315126555 learned_addr learned my addr 172.21.15.43:0/315126555 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:20.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.886+0000 7f67f8b8e640 1 --2- 172.21.15.43:0/315126555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67f4104d30 0x7f67f419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.886+0000 7f67f938f640 1 --2- 172.21.15.43:0/315126555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67f4103af0 0x7f67f419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.886+0000 7f67f9b90640 1 -- 172.21.15.43:0/315126555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67f4103af0 msgr2=0x7f67f419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.886+0000 7f67f9b90640 1 --2- 172.21.15.43:0/315126555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67f4103af0 0x7f67f419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.886+0000 7f67f9b90640 1 -- 172.21.15.43:0/315126555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67f4104d30 msgr2=0x7f67f419a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.886+0000 7f67f9b90640 1 --2- 172.21.15.43:0/315126555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67f4104d30 0x7f67f419a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.886+0000 7f67f9b90640 1 -- 172.21.15.43:0/315126555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f41a1d90 con 0x7f67f4109a30 2024-04-25T10:32:20.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.886+0000 7f67f9b90640 1 --2- 172.21.15.43:0/315126555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67f4109a30 0x7f67f41a1720 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00eaa0 tx=0x7f67ec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:20.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.886+0000 7f67e67fc640 1 -- 172.21.15.43:0/315126555 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec00cdb0 con 0x7f67f4109a30 2024-04-25T10:32:20.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.886+0000 7f67e67fc640 1 -- 172.21.15.43:0/315126555 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67ec005020 con 0x7f67f4109a30 2024-04-25T10:32:20.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.886+0000 7f67e67fc640 1 -- 172.21.15.43:0/315126555 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec0116d0 con 0x7f67f4109a30 2024-04-25T10:32:20.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.886+0000 7f67fb5f3640 1 -- 172.21.15.43:0/315126555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f41a2080 con 0x7f67f4109a30 2024-04-25T10:32:20.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.886+0000 7f67fb5f3640 1 -- 172.21.15.43:0/315126555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f41a25d0 con 0x7f67f4109a30 2024-04-25T10:32:20.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.887+0000 7f67fb5f3640 1 -- 172.21.15.43:0/315126555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f4108840 con 0x7f67f4109a30 2024-04-25T10:32:20.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.888+0000 7f67e67fc640 1 -- 172.21.15.43:0/315126555 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f67ec0051d0 con 0x7f67f4109a30 2024-04-25T10:32:20.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.889+0000 7f67e67fc640 1 --2- 172.21.15.43:0/315126555 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f67e00afb70 0x7f67e00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.890+0000 7f67f938f640 1 --2- 172.21.15.43:0/315126555 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f67e00afb70 0x7f67e00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.890+0000 7f67e67fc640 1 -- 172.21.15.43:0/315126555 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f67ec0df130 con 0x7f67f4109a30 2024-04-25T10:32:20.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.890+0000 7f67f938f640 1 --2- 172.21.15.43:0/315126555 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f67e00afb70 0x7f67e00b2080 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f67f0009230 tx=0x7f67f0031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:20.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:20.893+0000 7f67e67fc640 1 -- 172.21.15.43:0/315126555 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f67ec016030 con 0x7f67f4109a30 2024-04-25T10:32:21.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.082+0000 7f67fb5f3640 1 -- 172.21.15.43:0/315126555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f67f40637a0 con 0x7f67f4109a30 2024-04-25T10:32:21.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.082+0000 7f67e67fc640 1 -- 172.21.15.43:0/315126555 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7f67ec0a7da0 con 0x7f67f4109a30 2024-04-25T10:32:21.085 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T10:32:21.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.084+0000 7f67fb5f3640 1 -- 172.21.15.43:0/315126555 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f67e00afb70 msgr2=0x7f67e00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:21.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.084+0000 7f67fb5f3640 1 --2- 172.21.15.43:0/315126555 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f67e00afb70 0x7f67e00b2080 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f67f0009230 tx=0x7f67f0031400 comp rx=0 tx=0).stop 2024-04-25T10:32:21.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.084+0000 7f67fb5f3640 1 -- 172.21.15.43:0/315126555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67f4109a30 msgr2=0x7f67f41a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:21.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.084+0000 7f67fb5f3640 1 --2- 172.21.15.43:0/315126555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67f4109a30 0x7f67f41a1720 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00eaa0 tx=0x7f67ec00ef60 comp rx=0 tx=0).stop 2024-04-25T10:32:21.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.084+0000 7f67fb5f3640 1 -- 172.21.15.43:0/315126555 shutdown_connections 2024-04-25T10:32:21.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.084+0000 7f67fb5f3640 1 --2- 172.21.15.43:0/315126555 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f67e00afb70 0x7f67e00b2080 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.084+0000 7f67fb5f3640 1 --2- 172.21.15.43:0/315126555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67f4104d30 0x7f67f419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.084+0000 7f67fb5f3640 1 --2- 172.21.15.43:0/315126555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67f4109a30 0x7f67f41a1720 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.084+0000 7f67fb5f3640 1 --2- 172.21.15.43:0/315126555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67f4103af0 0x7f67f419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.084+0000 7f67fb5f3640 1 -- 172.21.15.43:0/315126555 >> 172.21.15.43:0/315126555 conn(0x7f67f40fece0 msgr2=0x7f67f4105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:21.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.084+0000 7f67fb5f3640 1 -- 172.21.15.43:0/315126555 shutdown_connections 2024-04-25T10:32:21.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.085+0000 7f67fb5f3640 1 -- 172.21.15.43:0/315126555 wait complete. 2024-04-25T10:32:21.097 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:20 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1977350287' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:21.097 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:20 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2676747647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:32:21.097 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:20 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/791585529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:32:21.098 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:32:21.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.207+0000 7f6c93d27640 1 -- 172.21.15.43:0/3912449908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8c103b00 msgr2=0x7f6c8c104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:21.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.207+0000 7f6c93d27640 1 --2- 172.21.15.43:0/3912449908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8c103b00 0x7f6c8c104800 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6c84009ca0 tx=0x7f6c8402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.208+0000 7f6c93d27640 1 -- 172.21.15.43:0/3912449908 shutdown_connections 2024-04-25T10:32:21.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.208+0000 7f6c93d27640 1 --2- 172.21.15.43:0/3912449908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c8c109a40 0x7f6c8c10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.208+0000 7f6c93d27640 1 --2- 172.21.15.43:0/3912449908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c8c104d40 0x7f6c8c105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.208+0000 7f6c93d27640 1 --2- 172.21.15.43:0/3912449908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8c103b00 0x7f6c8c104800 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.208+0000 7f6c93d27640 1 -- 172.21.15.43:0/3912449908 >> 172.21.15.43:0/3912449908 conn(0x7f6c8c0fed10 msgr2=0x7f6c8c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:21.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.208+0000 7f6c93d27640 1 -- 172.21.15.43:0/3912449908 shutdown_connections 2024-04-25T10:32:21.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.208+0000 7f6c93d27640 1 -- 172.21.15.43:0/3912449908 wait complete. 2024-04-25T10:32:21.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.208+0000 7f6c93d27640 1 Processor -- start 2024-04-25T10:32:21.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.208+0000 7f6c93d27640 1 -- start start 2024-04-25T10:32:21.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c93d27640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c8c103b00 0x7f6c8c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:21.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c93d27640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8c104d40 0x7f6c8c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:21.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c93d27640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c8c109a40 0x7f6c8c1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:21.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c93d27640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c10f600 con 0x7f6c8c104d40 2024-04-25T10:32:21.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c93d27640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c10f480 con 0x7f6c8c109a40 2024-04-25T10:32:21.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c93d27640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c10f780 con 0x7f6c8c103b00 2024-04-25T10:32:21.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c912c2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8c104d40 0x7f6c8c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:21.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c912c2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8c104d40 0x7f6c8c19a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39784/0 (socket says 172.21.15.43:39784) 2024-04-25T10:32:21.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c912c2640 1 -- 172.21.15.43:0/4235242081 learned_addr learned my addr 172.21.15.43:0/4235242081 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:21.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c91ac3640 1 --2- 172.21.15.43:0/4235242081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c8c103b00 0x7f6c8c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:21.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c922c4640 1 --2- 172.21.15.43:0/4235242081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c8c109a40 0x7f6c8c1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:21.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c912c2640 1 -- 172.21.15.43:0/4235242081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c8c103b00 msgr2=0x7f6c8c19a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:21.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c912c2640 1 --2- 172.21.15.43:0/4235242081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c8c103b00 0x7f6c8c19a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c912c2640 1 -- 172.21.15.43:0/4235242081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c8c109a40 msgr2=0x7f6c8c1a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:21.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c912c2640 1 --2- 172.21.15.43:0/4235242081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c8c109a40 0x7f6c8c1a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c912c2640 1 -- 172.21.15.43:0/4235242081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8c1a1da0 con 0x7f6c8c104d40 2024-04-25T10:32:21.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c922c4640 1 --2- 172.21.15.43:0/4235242081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c8c109a40 0x7f6c8c1a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:21.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c912c2640 1 --2- 172.21.15.43:0/4235242081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8c104d40 0x7f6c8c19a7a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000c970 tx=0x7f6c8000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:21.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c8affd640 1 -- 172.21.15.43:0/4235242081 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c80007bb0 con 0x7f6c8c104d40 2024-04-25T10:32:21.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c8affd640 1 -- 172.21.15.43:0/4235242081 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c80007d50 con 0x7f6c8c104d40 2024-04-25T10:32:21.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c93d27640 1 -- 172.21.15.43:0/4235242081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8c1a2090 con 0x7f6c8c104d40 2024-04-25T10:32:21.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c8affd640 1 -- 172.21.15.43:0/4235242081 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8000f450 con 0x7f6c8c104d40 2024-04-25T10:32:21.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.209+0000 7f6c93d27640 1 -- 172.21.15.43:0/4235242081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c8c1a2590 con 0x7f6c8c104d40 2024-04-25T10:32:21.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.211+0000 7f6c93d27640 1 -- 172.21.15.43:0/4235242081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c8c108850 con 0x7f6c8c104d40 2024-04-25T10:32:21.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.212+0000 7f6c8affd640 1 -- 172.21.15.43:0/4235242081 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6c80004ee0 con 0x7f6c8c104d40 2024-04-25T10:32:21.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.213+0000 7f6c8affd640 1 --2- 172.21.15.43:0/4235242081 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f6c740afaf0 0x7f6c740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:21.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.213+0000 7f6c91ac3640 1 --2- 172.21.15.43:0/4235242081 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f6c740afaf0 0x7f6c740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:21.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.213+0000 7f6c8affd640 1 -- 172.21.15.43:0/4235242081 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f6c800ded20 con 0x7f6c8c104d40 2024-04-25T10:32:21.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.214+0000 7f6c91ac3640 1 --2- 172.21.15.43:0/4235242081 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f6c740afaf0 0x7f6c740b2000 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6c84000f80 tx=0x7f6c84004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:21.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.217+0000 7f6c8affd640 1 -- 172.21.15.43:0/4235242081 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c80014030 con 0x7f6c8c104d40 2024-04-25T10:32:21.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:20 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1977350287' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:21.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:20 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2676747647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:32:21.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:20 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/791585529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:32:21.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:20 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1977350287' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:21.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:20 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2676747647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:32:21.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:20 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/791585529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:32:21.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.444+0000 7f6c93d27640 1 -- 172.21.15.43:0/4235242081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c8c02db50 con 0x7f6c8c104d40 2024-04-25T10:32:21.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.445+0000 7f6c8affd640 1 -- 172.21.15.43:0/4235242081 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11869 (secure 0 0 0) 0x7f6c800a7910 con 0x7f6c8c104d40 2024-04-25T10:32:21.447 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:32:21.448 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:30:38.804706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2595248466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2595248466},{"type":"v1","addr":"172.21.15.77:6837","nonce":2595248466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:32:21.448 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-04-25T10:32:21.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.447+0000 7f6c93d27640 1 -- 172.21.15.43:0/4235242081 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f6c740afaf0 msgr2=0x7f6c740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:21.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.447+0000 7f6c93d27640 1 --2- 172.21.15.43:0/4235242081 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f6c740afaf0 0x7f6c740b2000 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6c84000f80 tx=0x7f6c84004a40 comp rx=0 tx=0).stop 2024-04-25T10:32:21.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.447+0000 7f6c93d27640 1 -- 172.21.15.43:0/4235242081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8c104d40 msgr2=0x7f6c8c19a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:21.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.447+0000 7f6c93d27640 1 --2- 172.21.15.43:0/4235242081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8c104d40 0x7f6c8c19a7a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000c970 tx=0x7f6c8000ce30 comp rx=0 tx=0).stop 2024-04-25T10:32:21.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.447+0000 7f6c93d27640 1 -- 172.21.15.43:0/4235242081 shutdown_connections 2024-04-25T10:32:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.447+0000 7f6c93d27640 1 --2- 172.21.15.43:0/4235242081 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f6c740afaf0 0x7f6c740b2000 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.447+0000 7f6c93d27640 1 --2- 172.21.15.43:0/4235242081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c8c109a40 0x7f6c8c1a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.447+0000 7f6c93d27640 1 --2- 172.21.15.43:0/4235242081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8c104d40 0x7f6c8c19a7a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.447+0000 7f6c93d27640 1 --2- 172.21.15.43:0/4235242081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c8c103b00 0x7f6c8c19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.447+0000 7f6c93d27640 1 -- 172.21.15.43:0/4235242081 >> 172.21.15.43:0/4235242081 conn(0x7f6c8c0fed10 msgr2=0x7f6c8c105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.447+0000 7f6c93d27640 1 -- 172.21.15.43:0/4235242081 shutdown_connections 2024-04-25T10:32:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.447+0000 7f6c93d27640 1 -- 172.21.15.43:0/4235242081 wait complete. 2024-04-25T10:32:21.461 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:32:21.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.571+0000 7f6b12492640 1 -- 172.21.15.43:0/3887118956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b0c104d20 msgr2=0x7f6b0c105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:21.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.571+0000 7f6b12492640 1 --2- 172.21.15.43:0/3887118956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b0c104d20 0x7f6b0c105130 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009900 tx=0x7f6b0002a410 comp rx=0 tx=0).stop 2024-04-25T10:32:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.571+0000 7f6b12492640 1 -- 172.21.15.43:0/3887118956 shutdown_connections 2024-04-25T10:32:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.571+0000 7f6b12492640 1 --2- 172.21.15.43:0/3887118956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b0c109a20 0x7f6b0c10be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.571+0000 7f6b12492640 1 --2- 172.21.15.43:0/3887118956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b0c104d20 0x7f6b0c105130 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.571+0000 7f6b12492640 1 --2- 172.21.15.43:0/3887118956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b0c103ae0 0x7f6b0c1047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.571+0000 7f6b12492640 1 -- 172.21.15.43:0/3887118956 >> 172.21.15.43:0/3887118956 conn(0x7f6b0c0fecd0 msgr2=0x7f6b0c101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.571+0000 7f6b12492640 1 -- 172.21.15.43:0/3887118956 shutdown_connections 2024-04-25T10:32:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.571+0000 7f6b12492640 1 -- 172.21.15.43:0/3887118956 wait complete. 2024-04-25T10:32:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.572+0000 7f6b12492640 1 Processor -- start 2024-04-25T10:32:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.572+0000 7f6b12492640 1 -- start start 2024-04-25T10:32:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.572+0000 7f6b12492640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b0c103ae0 0x7f6b0c076850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.572+0000 7f6b12492640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b0c104d20 0x7f6b0c074ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.572+0000 7f6b12492640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b0c109a20 0x7f6b0c075430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.572+0000 7f6b12492640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c10f600 con 0x7f6b0c109a20 2024-04-25T10:32:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.572+0000 7f6b12492640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c10f480 con 0x7f6b0c104d20 2024-04-25T10:32:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.572+0000 7f6b12492640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c10f780 con 0x7f6b0c103ae0 2024-04-25T10:32:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.572+0000 7f6b10a2f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b0c109a20 0x7f6b0c075430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.572+0000 7f6b10a2f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b0c109a20 0x7f6b0c075430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39800/0 (socket says 172.21.15.43:39800) 2024-04-25T10:32:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.572+0000 7f6b10a2f640 1 -- 172.21.15.43:0/244724254 learned_addr learned my addr 172.21.15.43:0/244724254 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.572+0000 7f6b0bfff640 1 --2- 172.21.15.43:0/244724254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b0c103ae0 0x7f6b0c076850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.572+0000 7f6b0b7fe640 1 --2- 172.21.15.43:0/244724254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b0c104d20 0x7f6b0c074ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.572+0000 7f6b10a2f640 1 -- 172.21.15.43:0/244724254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b0c103ae0 msgr2=0x7f6b0c076850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.572+0000 7f6b10a2f640 1 --2- 172.21.15.43:0/244724254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b0c103ae0 0x7f6b0c076850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.572+0000 7f6b10a2f640 1 -- 172.21.15.43:0/244724254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b0c104d20 msgr2=0x7f6b0c074ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.572+0000 7f6b10a2f640 1 --2- 172.21.15.43:0/244724254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b0c104d20 0x7f6b0c074ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.572+0000 7f6b10a2f640 1 -- 172.21.15.43:0/244724254 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b0c075b60 con 0x7f6b0c109a20 2024-04-25T10:32:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.573+0000 7f6b10a2f640 1 --2- 172.21.15.43:0/244724254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b0c109a20 0x7f6b0c075430 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009fd0 tx=0x7f6b0400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.573+0000 7f6b097fa640 1 -- 172.21.15.43:0/244724254 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b04017070 con 0x7f6b0c109a20 2024-04-25T10:32:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.573+0000 7f6b12492640 1 -- 172.21.15.43:0/244724254 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b0c1a42c0 con 0x7f6b0c109a20 2024-04-25T10:32:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.573+0000 7f6b097fa640 1 -- 172.21.15.43:0/244724254 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b04003e20 con 0x7f6b0c109a20 2024-04-25T10:32:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.573+0000 7f6b097fa640 1 -- 172.21.15.43:0/244724254 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b04010460 con 0x7f6b0c109a20 2024-04-25T10:32:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.573+0000 7f6b12492640 1 -- 172.21.15.43:0/244724254 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b0c1a4730 con 0x7f6b0c109a20 2024-04-25T10:32:21.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.574+0000 7f6b12492640 1 -- 172.21.15.43:0/244724254 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b0c108830 con 0x7f6b0c109a20 2024-04-25T10:32:21.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.575+0000 7f6b097fa640 1 -- 172.21.15.43:0/244724254 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6b04003970 con 0x7f6b0c109a20 2024-04-25T10:32:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.576+0000 7f6b097fa640 1 --2- 172.21.15.43:0/244724254 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f6af40afaf0 0x7f6af40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.576+0000 7f6b0bfff640 1 --2- 172.21.15.43:0/244724254 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f6af40afaf0 0x7f6af40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.576+0000 7f6b097fa640 1 -- 172.21.15.43:0/244724254 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f6b040de9b0 con 0x7f6b0c109a20 2024-04-25T10:32:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.576+0000 7f6b0bfff640 1 --2- 172.21.15.43:0/244724254 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f6af40afaf0 0x7f6af40b2000 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6afc005fd0 tx=0x7f6afc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.581+0000 7f6b097fa640 1 -- 172.21.15.43:0/244724254 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b040a75a0 con 0x7f6b0c109a20 2024-04-25T10:32:21.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.767+0000 7f6b12492640 1 -- 172.21.15.43:0/244724254 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b0c0637a0 con 0x7f6b0c109a20 2024-04-25T10:32:21.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.769+0000 7f6b097fa640 1 -- 172.21.15.43:0/244724254 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22619 (secure 0 0 0) 0x7f6b040a6ea0 con 0x7f6b0c109a20 2024-04-25T10:32:21.771 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:32:21.771 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":79,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:31:33.928341+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:31.503414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:32.224334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:32:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.771+0000 7f6b12492640 1 -- 172.21.15.43:0/244724254 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f6af40afaf0 msgr2=0x7f6af40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.771+0000 7f6b12492640 1 --2- 172.21.15.43:0/244724254 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f6af40afaf0 0x7f6af40b2000 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6afc005fd0 tx=0x7f6afc00b390 comp rx=0 tx=0).stop 2024-04-25T10:32:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.771+0000 7f6b12492640 1 -- 172.21.15.43:0/244724254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b0c109a20 msgr2=0x7f6b0c075430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.771+0000 7f6b12492640 1 --2- 172.21.15.43:0/244724254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b0c109a20 0x7f6b0c075430 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009fd0 tx=0x7f6b0400b260 comp rx=0 tx=0).stop 2024-04-25T10:32:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.771+0000 7f6b12492640 1 -- 172.21.15.43:0/244724254 shutdown_connections 2024-04-25T10:32:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.771+0000 7f6b12492640 1 --2- 172.21.15.43:0/244724254 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f6af40afaf0 0x7f6af40b2000 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.771+0000 7f6b12492640 1 --2- 172.21.15.43:0/244724254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b0c104d20 0x7f6b0c074ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.771+0000 7f6b12492640 1 --2- 172.21.15.43:0/244724254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b0c109a20 0x7f6b0c075430 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.771+0000 7f6b12492640 1 --2- 172.21.15.43:0/244724254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b0c103ae0 0x7f6b0c076850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.771+0000 7f6b12492640 1 -- 172.21.15.43:0/244724254 >> 172.21.15.43:0/244724254 conn(0x7f6b0c0fecd0 msgr2=0x7f6b0c105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:21.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.771+0000 7f6b12492640 1 -- 172.21.15.43:0/244724254 shutdown_connections 2024-04-25T10:32:21.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.771+0000 7f6b12492640 1 -- 172.21.15.43:0/244724254 wait complete. 2024-04-25T10:32:21.785 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:25:58.167476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:31.503414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:32.224334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:32:21.785 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:32:21.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.898+0000 7fb476049640 1 -- 172.21.15.43:0/3425760405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4701045f0 msgr2=0x7fb470106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:21.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.898+0000 7fb476049640 1 --2- 172.21.15.43:0/3425760405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4701045f0 0x7fb470106a10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb460009900 tx=0x7fb46002a410 comp rx=0 tx=0).stop 2024-04-25T10:32:21.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.898+0000 7fb476049640 1 -- 172.21.15.43:0/3425760405 shutdown_connections 2024-04-25T10:32:21.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.898+0000 7fb476049640 1 --2- 172.21.15.43:0/3425760405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb470106f50 0x7fb470109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.898+0000 7fb476049640 1 --2- 172.21.15.43:0/3425760405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4701098b0 0x7fb47010bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.898+0000 7fb476049640 1 --2- 172.21.15.43:0/3425760405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4701045f0 0x7fb470106a10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.898+0000 7fb476049640 1 -- 172.21.15.43:0/3425760405 >> 172.21.15.43:0/3425760405 conn(0x7fb4700fdbf0 msgr2=0x7fb4701000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.898+0000 7fb476049640 1 -- 172.21.15.43:0/3425760405 shutdown_connections 2024-04-25T10:32:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.898+0000 7fb476049640 1 -- 172.21.15.43:0/3425760405 wait complete. 2024-04-25T10:32:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.899+0000 7fb476049640 1 Processor -- start 2024-04-25T10:32:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.899+0000 7fb476049640 1 -- start start 2024-04-25T10:32:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.899+0000 7fb476049640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4701045f0 0x7fb47019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.899+0000 7fb476049640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb470106f50 0x7fb47019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.899+0000 7fb476049640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4701098b0 0x7fb4701a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.899+0000 7fb476049640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb47010f600 con 0x7fb4701098b0 2024-04-25T10:32:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.899+0000 7fb476049640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb47010f480 con 0x7fb4701045f0 2024-04-25T10:32:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.899+0000 7fb476049640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb47010f780 con 0x7fb470106f50 2024-04-25T10:32:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.899+0000 7fb46ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4701098b0 0x7fb4701a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.899+0000 7fb46ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4701098b0 0x7fb4701a1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39814/0 (socket says 172.21.15.43:39814) 2024-04-25T10:32:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.899+0000 7fb46ffff640 1 -- 172.21.15.43:0/2205378295 learned_addr learned my addr 172.21.15.43:0/2205378295 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.899+0000 7fb46f7fe640 1 --2- 172.21.15.43:0/2205378295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4701045f0 0x7fb47019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.899+0000 7fb46effd640 1 --2- 172.21.15.43:0/2205378295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb470106f50 0x7fb47019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.900+0000 7fb46ffff640 1 -- 172.21.15.43:0/2205378295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb470106f50 msgr2=0x7fb47019a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:21.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.900+0000 7fb46ffff640 1 --2- 172.21.15.43:0/2205378295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb470106f50 0x7fb47019a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.900+0000 7fb46ffff640 1 -- 172.21.15.43:0/2205378295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4701045f0 msgr2=0x7fb47019a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:21.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.900+0000 7fb46ffff640 1 --2- 172.21.15.43:0/2205378295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4701045f0 0x7fb47019a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:21.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.900+0000 7fb46ffff640 1 -- 172.21.15.43:0/2205378295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4701a1d50 con 0x7fb4701098b0 2024-04-25T10:32:21.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.900+0000 7fb46f7fe640 1 --2- 172.21.15.43:0/2205378295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4701045f0 0x7fb47019a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:21.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.900+0000 7fb46ffff640 1 --2- 172.21.15.43:0/2205378295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4701098b0 0x7fb4701a1790 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb46800e9d0 tx=0x7fb46800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:21.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.900+0000 7fb46cff9640 1 -- 172.21.15.43:0/2205378295 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46800cd80 con 0x7fb4701098b0 2024-04-25T10:32:21.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.900+0000 7fb46cff9640 1 -- 172.21.15.43:0/2205378295 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb468005020 con 0x7fb4701098b0 2024-04-25T10:32:21.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.900+0000 7fb476049640 1 -- 172.21.15.43:0/2205378295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4701a1fe0 con 0x7fb4701098b0 2024-04-25T10:32:21.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.900+0000 7fb46cff9640 1 -- 172.21.15.43:0/2205378295 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4680116d0 con 0x7fb4701098b0 2024-04-25T10:32:21.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.900+0000 7fb476049640 1 -- 172.21.15.43:0/2205378295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4701a24a0 con 0x7fb4701098b0 2024-04-25T10:32:21.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.901+0000 7fb476049640 1 -- 172.21.15.43:0/2205378295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4700ff710 con 0x7fb4701098b0 2024-04-25T10:32:21.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.902+0000 7fb46cff9640 1 -- 172.21.15.43:0/2205378295 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb4680051d0 con 0x7fb4701098b0 2024-04-25T10:32:21.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.903+0000 7fb46cff9640 1 --2- 172.21.15.43:0/2205378295 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb4580afaf0 0x7fb4580b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:21.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.903+0000 7fb46cff9640 1 -- 172.21.15.43:0/2205378295 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fb4680df1d0 con 0x7fb4701098b0 2024-04-25T10:32:21.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.903+0000 7fb46f7fe640 1 --2- 172.21.15.43:0/2205378295 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb4580afaf0 0x7fb4580b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:21.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.903+0000 7fb46f7fe640 1 --2- 172.21.15.43:0/2205378295 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb4580afaf0 0x7fb4580b2000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb460000c00 tx=0x7fb4600314c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:21.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:21.908+0000 7fb46cff9640 1 -- 172.21.15.43:0/2205378295 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb468016030 con 0x7fb4701098b0 2024-04-25T10:32:22.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.096+0000 7fb476049640 1 -- 172.21.15.43:0/2205378295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb4700637a0 con 0x7fb4701098b0 2024-04-25T10:32:22.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.097+0000 7fb46cff9640 1 -- 172.21.15.43:0/2205378295 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7fb4680a7dc0 con 0x7fb4701098b0 2024-04-25T10:32:22.099 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T10:32:22.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.099+0000 7fb476049640 1 -- 172.21.15.43:0/2205378295 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb4580afaf0 msgr2=0x7fb4580b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:22.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.099+0000 7fb476049640 1 --2- 172.21.15.43:0/2205378295 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb4580afaf0 0x7fb4580b2000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb460000c00 tx=0x7fb4600314c0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.099+0000 7fb476049640 1 -- 172.21.15.43:0/2205378295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4701098b0 msgr2=0x7fb4701a1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:22.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.099+0000 7fb476049640 1 --2- 172.21.15.43:0/2205378295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4701098b0 0x7fb4701a1790 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb46800e9d0 tx=0x7fb46800ee90 comp rx=0 tx=0).stop 2024-04-25T10:32:22.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.099+0000 7fb476049640 1 -- 172.21.15.43:0/2205378295 shutdown_connections 2024-04-25T10:32:22.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.099+0000 7fb476049640 1 --2- 172.21.15.43:0/2205378295 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fb4580afaf0 0x7fb4580b2000 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.099+0000 7fb476049640 1 --2- 172.21.15.43:0/2205378295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4701098b0 0x7fb4701a1790 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.099+0000 7fb476049640 1 --2- 172.21.15.43:0/2205378295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb470106f50 0x7fb47019a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.099+0000 7fb476049640 1 --2- 172.21.15.43:0/2205378295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4701045f0 0x7fb47019a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.099+0000 7fb476049640 1 -- 172.21.15.43:0/2205378295 >> 172.21.15.43:0/2205378295 conn(0x7fb4700fdbf0 msgr2=0x7fb470106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:22.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.099+0000 7fb476049640 1 -- 172.21.15.43:0/2205378295 shutdown_connections 2024-04-25T10:32:22.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.099+0000 7fb476049640 1 -- 172.21.15.43:0/2205378295 wait complete. 2024-04-25T10:32:22.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:21 smithi043 ceph-mon[29478]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T10:32:22.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:21 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/315126555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:32:22.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:21 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/4235242081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:32:22.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:21 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/244724254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:22.113 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:32:22.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.226+0000 7f49c6b3f640 1 -- 172.21.15.43:0/1121904427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c0103c20 msgr2=0x7f49c0104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:22.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.226+0000 7f49c6b3f640 1 --2- 172.21.15.43:0/1121904427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c0103c20 0x7f49c0104030 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009900 tx=0x7f49b002a410 comp rx=0 tx=0).stop 2024-04-25T10:32:22.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.226+0000 7f49c6b3f640 1 -- 172.21.15.43:0/1121904427 shutdown_connections 2024-04-25T10:32:22.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.226+0000 7f49c6b3f640 1 --2- 172.21.15.43:0/1121904427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49c0108920 0x7f49c010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.226+0000 7f49c6b3f640 1 --2- 172.21.15.43:0/1121904427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c0103c20 0x7f49c0104030 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.226+0000 7f49c6b3f640 1 --2- 172.21.15.43:0/1121904427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49c01029e0 0x7f49c01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.226+0000 7f49c6b3f640 1 -- 172.21.15.43:0/1121904427 >> 172.21.15.43:0/1121904427 conn(0x7f49c00fdbd0 msgr2=0x7f49c0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:22.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.226+0000 7f49c6b3f640 1 -- 172.21.15.43:0/1121904427 shutdown_connections 2024-04-25T10:32:22.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.227+0000 7f49c6b3f640 1 -- 172.21.15.43:0/1121904427 wait complete. 2024-04-25T10:32:22.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.227+0000 7f49c6b3f640 1 Processor -- start 2024-04-25T10:32:22.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.227+0000 7f49c6b3f640 1 -- start start 2024-04-25T10:32:22.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.227+0000 7f49c6b3f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49c01029e0 0x7f49c006a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:22.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.227+0000 7f49c6b3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c0103c20 0x7f49c019f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:22.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.227+0000 7f49c6b3f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49c0108920 0x7f49c01a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:22.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.227+0000 7f49c6b3f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49c010e510 con 0x7f49c0103c20 2024-04-25T10:32:22.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.227+0000 7f49c6b3f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f49c010e390 con 0x7f49c01029e0 2024-04-25T10:32:22.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.227+0000 7f49c6b3f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f49c010e690 con 0x7f49c0108920 2024-04-25T10:32:22.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.227+0000 7f49b7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c0103c20 0x7f49c019f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:22.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.227+0000 7f49b7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c0103c20 0x7f49c019f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39832/0 (socket says 172.21.15.43:39832) 2024-04-25T10:32:22.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.227+0000 7f49b7fff640 1 -- 172.21.15.43:0/3797939015 learned_addr learned my addr 172.21.15.43:0/3797939015 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:22.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.228+0000 7f49c50dc640 1 --2- 172.21.15.43:0/3797939015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49c0108920 0x7f49c01a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:22.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.228+0000 7f49c48db640 1 --2- 172.21.15.43:0/3797939015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49c01029e0 0x7f49c006a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:22.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.228+0000 7f49b7fff640 1 -- 172.21.15.43:0/3797939015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49c0108920 msgr2=0x7f49c01a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:22.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.228+0000 7f49b7fff640 1 --2- 172.21.15.43:0/3797939015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49c0108920 0x7f49c01a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.228+0000 7f49b7fff640 1 -- 172.21.15.43:0/3797939015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49c01029e0 msgr2=0x7f49c006a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.228+0000 7f49b7fff640 1 --2- 172.21.15.43:0/3797939015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49c01029e0 0x7f49c006a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.228+0000 7f49b7fff640 1 -- 172.21.15.43:0/3797939015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c01a2060 con 0x7f49c0103c20 2024-04-25T10:32:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.228+0000 7f49b7fff640 1 --2- 172.21.15.43:0/3797939015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c0103c20 0x7f49c019f2d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f49b0006010 tx=0x7f49b0004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.228+0000 7f49b5ffb640 1 -- 172.21.15.43:0/3797939015 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b003f030 con 0x7f49c0103c20 2024-04-25T10:32:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.228+0000 7f49b5ffb640 1 -- 172.21.15.43:0/3797939015 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49b0003cb0 con 0x7f49c0103c20 2024-04-25T10:32:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.228+0000 7f49c6b3f640 1 -- 172.21.15.43:0/3797939015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c01a22f0 con 0x7f49c0103c20 2024-04-25T10:32:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.228+0000 7f49b5ffb640 1 -- 172.21.15.43:0/3797939015 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b0035070 con 0x7f49c0103c20 2024-04-25T10:32:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.228+0000 7f49c6b3f640 1 -- 172.21.15.43:0/3797939015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c01a27e0 con 0x7f49c0103c20 2024-04-25T10:32:22.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.229+0000 7f49c6b3f640 1 -- 172.21.15.43:0/3797939015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c0107730 con 0x7f49c0103c20 2024-04-25T10:32:22.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.231+0000 7f49b5ffb640 1 -- 172.21.15.43:0/3797939015 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f49b0042030 con 0x7f49c0103c20 2024-04-25T10:32:22.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.231+0000 7f49b5ffb640 1 --2- 172.21.15.43:0/3797939015 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f49a80afaf0 0x7f49a80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:22.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.232+0000 7f49c48db640 1 --2- 172.21.15.43:0/3797939015 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f49a80afaf0 0x7f49a80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:22.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.232+0000 7f49b5ffb640 1 -- 172.21.15.43:0/3797939015 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f49b00ff900 con 0x7f49c0103c20 2024-04-25T10:32:22.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.232+0000 7f49c48db640 1 --2- 172.21.15.43:0/3797939015 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f49a80afaf0 0x7f49a80b2000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f49bc005d00 tx=0x7f49bc005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:22.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.235+0000 7f49b5ffb640 1 -- 172.21.15.43:0/3797939015 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49b00c84f0 con 0x7f49c0103c20 2024-04-25T10:32:22.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:21 smithi077 ceph-mon[32772]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T10:32:22.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:21 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/315126555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:32:22.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:21 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/4235242081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:32:22.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:21 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/244724254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:22.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:21 smithi164 ceph-mon[32713]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T10:32:22.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:21 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/315126555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:32:22.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:21 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/4235242081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:32:22.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:21 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/244724254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:22.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.423+0000 7f49c6b3f640 1 -- 172.21.15.43:0/3797939015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f49c006c2a0 con 0x7f49c0103c20 2024-04-25T10:32:22.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.423+0000 7f49b5ffb640 1 -- 172.21.15.43:0/3797939015 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7f49c006c2a0 con 0x7f49c0103c20 2024-04-25T10:32:22.425 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T10:32:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.425+0000 7f49c6b3f640 1 -- 172.21.15.43:0/3797939015 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f49a80afaf0 msgr2=0x7f49a80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.425+0000 7f49c6b3f640 1 --2- 172.21.15.43:0/3797939015 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f49a80afaf0 0x7f49a80b2000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f49bc005d00 tx=0x7f49bc005c50 comp rx=0 tx=0).stop 2024-04-25T10:32:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.425+0000 7f49c6b3f640 1 -- 172.21.15.43:0/3797939015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c0103c20 msgr2=0x7f49c019f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.425+0000 7f49c6b3f640 1 --2- 172.21.15.43:0/3797939015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c0103c20 0x7f49c019f2d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f49b0006010 tx=0x7f49b0004340 comp rx=0 tx=0).stop 2024-04-25T10:32:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.425+0000 7f49c6b3f640 1 -- 172.21.15.43:0/3797939015 shutdown_connections 2024-04-25T10:32:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.425+0000 7f49c6b3f640 1 --2- 172.21.15.43:0/3797939015 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f49a80afaf0 0x7f49a80b2000 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.425+0000 7f49c6b3f640 1 --2- 172.21.15.43:0/3797939015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49c0108920 0x7f49c01a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.425+0000 7f49c6b3f640 1 --2- 172.21.15.43:0/3797939015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c0103c20 0x7f49c019f2d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.425+0000 7f49c6b3f640 1 --2- 172.21.15.43:0/3797939015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49c01029e0 0x7f49c006a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.425+0000 7f49c6b3f640 1 -- 172.21.15.43:0/3797939015 >> 172.21.15.43:0/3797939015 conn(0x7f49c00fdbd0 msgr2=0x7f49c0104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.425+0000 7f49c6b3f640 1 -- 172.21.15.43:0/3797939015 shutdown_connections 2024-04-25T10:32:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.425+0000 7f49c6b3f640 1 -- 172.21.15.43:0/3797939015 wait complete. 2024-04-25T10:32:22.439 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:32:22.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.551+0000 7f78b4ba8640 1 -- 172.21.15.43:0/3613249061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78b0103b00 msgr2=0x7f78b0104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:22.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.551+0000 7f78b4ba8640 1 --2- 172.21.15.43:0/3613249061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78b0103b00 0x7f78b0104800 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009ca0 tx=0x7f78a802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.552+0000 7f78b4ba8640 1 -- 172.21.15.43:0/3613249061 shutdown_connections 2024-04-25T10:32:22.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.552+0000 7f78b4ba8640 1 --2- 172.21.15.43:0/3613249061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78b0104d40 0x7f78b0105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.552+0000 7f78b4ba8640 1 --2- 172.21.15.43:0/3613249061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78b0109a40 0x7f78b010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.552+0000 7f78b4ba8640 1 --2- 172.21.15.43:0/3613249061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78b0103b00 0x7f78b0104800 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.552+0000 7f78b4ba8640 1 -- 172.21.15.43:0/3613249061 >> 172.21.15.43:0/3613249061 conn(0x7f78b00fed10 msgr2=0x7f78b01011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:22.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.552+0000 7f78b4ba8640 1 -- 172.21.15.43:0/3613249061 shutdown_connections 2024-04-25T10:32:22.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.552+0000 7f78b4ba8640 1 -- 172.21.15.43:0/3613249061 wait complete. 2024-04-25T10:32:22.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.552+0000 7f78b4ba8640 1 Processor -- start 2024-04-25T10:32:22.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.552+0000 7f78b4ba8640 1 -- start start 2024-04-25T10:32:22.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.553+0000 7f78b4ba8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78b0103b00 0x7f78b0077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.553+0000 7f78b4ba8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78b0104d40 0x7f78b0076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.553+0000 7f78b4ba8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78b0109a40 0x7f78b0076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.553+0000 7f78b4ba8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78b010f600 con 0x7f78b0103b00 2024-04-25T10:32:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.553+0000 7f78b4ba8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78b010f480 con 0x7f78b0104d40 2024-04-25T10:32:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.553+0000 7f78b4ba8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f78b010f780 con 0x7f78b0109a40 2024-04-25T10:32:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.553+0000 7f78ae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78b0103b00 0x7f78b0077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.553+0000 7f78ae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78b0103b00 0x7f78b0077960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39860/0 (socket says 172.21.15.43:39860) 2024-04-25T10:32:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.553+0000 7f78ae59c640 1 -- 172.21.15.43:0/2854718215 learned_addr learned my addr 172.21.15.43:0/2854718215 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.553+0000 7f78add9b640 1 --2- 172.21.15.43:0/2854718215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78b0104d40 0x7f78b0076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.553+0000 7f78aed9d640 1 --2- 172.21.15.43:0/2854718215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78b0109a40 0x7f78b0076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.553+0000 7f78ae59c640 1 -- 172.21.15.43:0/2854718215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78b0109a40 msgr2=0x7f78b0076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.553+0000 7f78ae59c640 1 --2- 172.21.15.43:0/2854718215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78b0109a40 0x7f78b0076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.553+0000 7f78ae59c640 1 -- 172.21.15.43:0/2854718215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78b0104d40 msgr2=0x7f78b0076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.553+0000 7f78ae59c640 1 --2- 172.21.15.43:0/2854718215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78b0104d40 0x7f78b0076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.553+0000 7f78ae59c640 1 -- 172.21.15.43:0/2854718215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b0076c70 con 0x7f78b0103b00 2024-04-25T10:32:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.553+0000 7f78ae59c640 1 --2- 172.21.15.43:0/2854718215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78b0103b00 0x7f78b0077960 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f78a8004af0 tx=0x7f78a8031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.553+0000 7f789f7fe640 1 -- 172.21.15.43:0/2854718215 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a8003c30 con 0x7f78b0103b00 2024-04-25T10:32:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.554+0000 7f78b4ba8640 1 -- 172.21.15.43:0/2854718215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b01a4360 con 0x7f78b0103b00 2024-04-25T10:32:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.554+0000 7f789f7fe640 1 -- 172.21.15.43:0/2854718215 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a8003dd0 con 0x7f78b0103b00 2024-04-25T10:32:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.554+0000 7f789f7fe640 1 -- 172.21.15.43:0/2854718215 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a802fb60 con 0x7f78b0103b00 2024-04-25T10:32:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.554+0000 7f78b4ba8640 1 -- 172.21.15.43:0/2854718215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b01a4820 con 0x7f78b0103b00 2024-04-25T10:32:22.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.555+0000 7f78b4ba8640 1 -- 172.21.15.43:0/2854718215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78b0108850 con 0x7f78b0103b00 2024-04-25T10:32:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.556+0000 7f789f7fe640 1 -- 172.21.15.43:0/2854718215 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f78a802fd00 con 0x7f78b0103b00 2024-04-25T10:32:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.557+0000 7f789f7fe640 1 --2- 172.21.15.43:0/2854718215 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f78980afaf0 0x7f78980b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.557+0000 7f78add9b640 1 --2- 172.21.15.43:0/2854718215 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f78980afaf0 0x7f78980b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.557+0000 7f789f7fe640 1 -- 172.21.15.43:0/2854718215 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f78a8100230 con 0x7f78b0103b00 2024-04-25T10:32:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.557+0000 7f78add9b640 1 --2- 172.21.15.43:0/2854718215 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f78980afaf0 0x7f78980b2000 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f78b0077300 tx=0x7f78a400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:22.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.560+0000 7f789f7fe640 1 -- 172.21.15.43:0/2854718215 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78a80c8e20 con 0x7f78b0103b00 2024-04-25T10:32:22.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.746+0000 7f78b4ba8640 1 -- 172.21.15.43:0/2854718215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f78b00637a0 con 0x7f78b0103b00 2024-04-25T10:32:22.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.747+0000 7f789f7fe640 1 -- 172.21.15.43:0/2854718215 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7f78a80c8720 con 0x7f78b0103b00 2024-04-25T10:32:22.749 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T10:32:22.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.748+0000 7f78b4ba8640 1 -- 172.21.15.43:0/2854718215 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f78980afaf0 msgr2=0x7f78980b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:22.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.748+0000 7f78b4ba8640 1 --2- 172.21.15.43:0/2854718215 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f78980afaf0 0x7f78980b2000 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f78b0077300 tx=0x7f78a400b390 comp rx=0 tx=0).stop 2024-04-25T10:32:22.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.748+0000 7f78b4ba8640 1 -- 172.21.15.43:0/2854718215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78b0103b00 msgr2=0x7f78b0077960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:22.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.748+0000 7f78b4ba8640 1 --2- 172.21.15.43:0/2854718215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78b0103b00 0x7f78b0077960 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f78a8004af0 tx=0x7f78a8031840 comp rx=0 tx=0).stop 2024-04-25T10:32:22.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.749+0000 7f78b4ba8640 1 -- 172.21.15.43:0/2854718215 shutdown_connections 2024-04-25T10:32:22.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.749+0000 7f78b4ba8640 1 --2- 172.21.15.43:0/2854718215 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f78980afaf0 0x7f78980b2000 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.749+0000 7f78b4ba8640 1 --2- 172.21.15.43:0/2854718215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78b0104d40 0x7f78b0076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.749+0000 7f78b4ba8640 1 --2- 172.21.15.43:0/2854718215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78b0109a40 0x7f78b0076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.749+0000 7f78b4ba8640 1 --2- 172.21.15.43:0/2854718215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78b0103b00 0x7f78b0077960 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.749+0000 7f78b4ba8640 1 -- 172.21.15.43:0/2854718215 >> 172.21.15.43:0/2854718215 conn(0x7f78b00fed10 msgr2=0x7f78b0105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:22.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.749+0000 7f78b4ba8640 1 -- 172.21.15.43:0/2854718215 shutdown_connections 2024-04-25T10:32:22.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.749+0000 7f78b4ba8640 1 -- 172.21.15.43:0/2854718215 wait complete. 2024-04-25T10:32:22.762 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:32:22.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.872+0000 7fd3dfc70640 1 -- 172.21.15.43:0/616759426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3d81045f0 msgr2=0x7fd3d8106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:22.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.872+0000 7fd3dfc70640 1 --2- 172.21.15.43:0/616759426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3d81045f0 0x7fd3d8106a10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009900 tx=0x7fd3d402a410 comp rx=0 tx=0).stop 2024-04-25T10:32:22.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.872+0000 7fd3dfc70640 1 -- 172.21.15.43:0/616759426 shutdown_connections 2024-04-25T10:32:22.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.872+0000 7fd3dfc70640 1 --2- 172.21.15.43:0/616759426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d8106f50 0x7fd3d8109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.872+0000 7fd3dfc70640 1 --2- 172.21.15.43:0/616759426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3d81098b0 0x7fd3d810bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.872+0000 7fd3dfc70640 1 --2- 172.21.15.43:0/616759426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3d81045f0 0x7fd3d8106a10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.872+0000 7fd3dfc70640 1 -- 172.21.15.43:0/616759426 >> 172.21.15.43:0/616759426 conn(0x7fd3d80fdbf0 msgr2=0x7fd3d81000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:22.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.873+0000 7fd3dfc70640 1 -- 172.21.15.43:0/616759426 shutdown_connections 2024-04-25T10:32:22.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.873+0000 7fd3dfc70640 1 -- 172.21.15.43:0/616759426 wait complete. 2024-04-25T10:32:22.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.873+0000 7fd3dfc70640 1 Processor -- start 2024-04-25T10:32:22.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.873+0000 7fd3dfc70640 1 -- start start 2024-04-25T10:32:22.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.873+0000 7fd3dfc70640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d81045f0 0x7fd3d819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:22.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.873+0000 7fd3dfc70640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3d8106f50 0x7fd3d819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:22.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.873+0000 7fd3dfc70640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3d81098b0 0x7fd3d81a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:22.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.873+0000 7fd3dfc70640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d810f5e0 con 0x7fd3d8106f50 2024-04-25T10:32:22.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.873+0000 7fd3dfc70640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d810f460 con 0x7fd3d81045f0 2024-04-25T10:32:22.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.873+0000 7fd3dfc70640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d810f760 con 0x7fd3d81098b0 2024-04-25T10:32:22.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.874+0000 7fd3dd20b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3d8106f50 0x7fd3d819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:22.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.874+0000 7fd3dd20b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3d8106f50 0x7fd3d819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39890/0 (socket says 172.21.15.43:39890) 2024-04-25T10:32:22.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.874+0000 7fd3dd20b640 1 -- 172.21.15.43:0/2158581795 learned_addr learned my addr 172.21.15.43:0/2158581795 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:22.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.874+0000 7fd3de20d640 1 --2- 172.21.15.43:0/2158581795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3d81098b0 0x7fd3d81a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.874+0000 7fd3dda0c640 1 --2- 172.21.15.43:0/2158581795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d81045f0 0x7fd3d819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.874+0000 7fd3dd20b640 1 -- 172.21.15.43:0/2158581795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3d81098b0 msgr2=0x7fd3d81a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.874+0000 7fd3dd20b640 1 --2- 172.21.15.43:0/2158581795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3d81098b0 0x7fd3d81a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.874+0000 7fd3dd20b640 1 -- 172.21.15.43:0/2158581795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d81045f0 msgr2=0x7fd3d819a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.874+0000 7fd3dd20b640 1 --2- 172.21.15.43:0/2158581795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d81045f0 0x7fd3d819a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.874+0000 7fd3dd20b640 1 -- 172.21.15.43:0/2158581795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d81a1c80 con 0x7fd3d8106f50 2024-04-25T10:32:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.874+0000 7fd3dd20b640 1 --2- 172.21.15.43:0/2158581795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3d8106f50 0x7fd3d819a730 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd3c800d950 tx=0x7fd3c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.874+0000 7fd3ceffd640 1 -- 172.21.15.43:0/2158581795 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8017070 con 0x7fd3d8106f50 2024-04-25T10:32:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.874+0000 7fd3ceffd640 1 -- 172.21.15.43:0/2158581795 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3c8009c30 con 0x7fd3d8106f50 2024-04-25T10:32:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.874+0000 7fd3dfc70640 1 -- 172.21.15.43:0/2158581795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d81a1f70 con 0x7fd3d8106f50 2024-04-25T10:32:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.874+0000 7fd3ceffd640 1 -- 172.21.15.43:0/2158581795 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c800f620 con 0x7fd3d8106f50 2024-04-25T10:32:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.874+0000 7fd3dfc70640 1 -- 172.21.15.43:0/2158581795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d81a23e0 con 0x7fd3d8106f50 2024-04-25T10:32:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.875+0000 7fd3dfc70640 1 -- 172.21.15.43:0/2158581795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d80ff710 con 0x7fd3d8106f50 2024-04-25T10:32:22.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.877+0000 7fd3ceffd640 1 -- 172.21.15.43:0/2158581795 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd3c8010460 con 0x7fd3d8106f50 2024-04-25T10:32:22.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.877+0000 7fd3ceffd640 1 --2- 172.21.15.43:0/2158581795 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd3c40afaf0 0x7fd3c40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:22.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.877+0000 7fd3dda0c640 1 --2- 172.21.15.43:0/2158581795 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd3c40afaf0 0x7fd3c40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:22.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.877+0000 7fd3ceffd640 1 -- 172.21.15.43:0/2158581795 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fd3c80de740 con 0x7fd3d8106f50 2024-04-25T10:32:22.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.878+0000 7fd3dda0c640 1 --2- 172.21.15.43:0/2158581795 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd3c40afaf0 0x7fd3c40b2000 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009230 tx=0x7fd3d4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:22.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:22.881+0000 7fd3ceffd640 1 -- 172.21.15.43:0/2158581795 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd3c8009de0 con 0x7fd3d8106f50 2024-04-25T10:32:23.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:22 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2205378295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:32:23.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:22 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3797939015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:32:23.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:22 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2854718215' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:32:23.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.110+0000 7fd3dfc70640 1 -- 172.21.15.43:0/2158581795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3d802dd10 con 0x7fd3d8106f50 2024-04-25T10:32:23.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.111+0000 7fd3ceffd640 1 -- 172.21.15.43:0/2158581795 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11869 (secure 0 0 0) 0x7fd3c80a6ce0 con 0x7fd3d8106f50 2024-04-25T10:32:23.113 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:32:23.113 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:30:38.804706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2595248466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2595248466},{"type":"v1","addr":"172.21.15.77:6837","nonce":2595248466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:32:23.113 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-04-25T10:32:23.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.113+0000 7fd3dfc70640 1 -- 172.21.15.43:0/2158581795 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd3c40afaf0 msgr2=0x7fd3c40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.113+0000 7fd3dfc70640 1 --2- 172.21.15.43:0/2158581795 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd3c40afaf0 0x7fd3c40b2000 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009230 tx=0x7fd3d4031400 comp rx=0 tx=0).stop 2024-04-25T10:32:23.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.113+0000 7fd3dfc70640 1 -- 172.21.15.43:0/2158581795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3d8106f50 msgr2=0x7fd3d819a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.113+0000 7fd3dfc70640 1 --2- 172.21.15.43:0/2158581795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3d8106f50 0x7fd3d819a730 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd3c800d950 tx=0x7fd3c800de10 comp rx=0 tx=0).stop 2024-04-25T10:32:23.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.113+0000 7fd3dfc70640 1 -- 172.21.15.43:0/2158581795 shutdown_connections 2024-04-25T10:32:23.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.113+0000 7fd3dfc70640 1 --2- 172.21.15.43:0/2158581795 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd3c40afaf0 0x7fd3c40b2000 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.113+0000 7fd3dfc70640 1 --2- 172.21.15.43:0/2158581795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3d81098b0 0x7fd3d81a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.113+0000 7fd3dfc70640 1 --2- 172.21.15.43:0/2158581795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3d8106f50 0x7fd3d819a730 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.113+0000 7fd3dfc70640 1 --2- 172.21.15.43:0/2158581795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d81045f0 0x7fd3d819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.113+0000 7fd3dfc70640 1 -- 172.21.15.43:0/2158581795 >> 172.21.15.43:0/2158581795 conn(0x7fd3d80fdbf0 msgr2=0x7fd3d8105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.113+0000 7fd3dfc70640 1 -- 172.21.15.43:0/2158581795 shutdown_connections 2024-04-25T10:32:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.113+0000 7fd3dfc70640 1 -- 172.21.15.43:0/2158581795 wait complete. 2024-04-25T10:32:23.128 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:32:23.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.251+0000 7f2017492640 1 -- 172.21.15.43:0/587262710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2010103b00 msgr2=0x7f2010104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.251+0000 7f2017492640 1 --2- 172.21.15.43:0/587262710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2010103b00 0x7f2010104800 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f200c009900 tx=0x7f200c02a410 comp rx=0 tx=0).stop 2024-04-25T10:32:23.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.251+0000 7f2017492640 1 -- 172.21.15.43:0/587262710 shutdown_connections 2024-04-25T10:32:23.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.251+0000 7f2017492640 1 --2- 172.21.15.43:0/587262710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2010109a40 0x7f201010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.251+0000 7f2017492640 1 --2- 172.21.15.43:0/587262710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2010104d40 0x7f2010105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.251+0000 7f2017492640 1 --2- 172.21.15.43:0/587262710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2010103b00 0x7f2010104800 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.251+0000 7f2017492640 1 -- 172.21.15.43:0/587262710 >> 172.21.15.43:0/587262710 conn(0x7f20100fed10 msgr2=0x7f20101011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.251+0000 7f2017492640 1 -- 172.21.15.43:0/587262710 shutdown_connections 2024-04-25T10:32:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.251+0000 7f2017492640 1 -- 172.21.15.43:0/587262710 wait complete. 2024-04-25T10:32:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.251+0000 7f2017492640 1 Processor -- start 2024-04-25T10:32:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.251+0000 7f2017492640 1 -- start start 2024-04-25T10:32:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f2017492640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2010103b00 0x7f201019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f2017492640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2010104d40 0x7f201019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f2017492640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2010109a40 0x7f20101a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f2017492640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f201010f5e0 con 0x7f2010103b00 2024-04-25T10:32:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f2017492640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f201010f460 con 0x7f2010109a40 2024-04-25T10:32:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f2017492640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f201010f760 con 0x7f2010104d40 2024-04-25T10:32:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f201522e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2010103b00 0x7f201019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f201522e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2010103b00 0x7f201019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39904/0 (socket says 172.21.15.43:39904) 2024-04-25T10:32:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f201522e640 1 -- 172.21.15.43:0/956232509 learned_addr learned my addr 172.21.15.43:0/956232509 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f2015a2f640 1 --2- 172.21.15.43:0/956232509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2010109a40 0x7f20101a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f2014a2d640 1 --2- 172.21.15.43:0/956232509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2010104d40 0x7f201019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f201522e640 1 -- 172.21.15.43:0/956232509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2010104d40 msgr2=0x7f201019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f201522e640 1 --2- 172.21.15.43:0/956232509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2010104d40 0x7f201019a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f201522e640 1 -- 172.21.15.43:0/956232509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2010109a40 msgr2=0x7f20101a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f201522e640 1 --2- 172.21.15.43:0/956232509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2010109a40 0x7f20101a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f201522e640 1 -- 172.21.15.43:0/956232509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20101a1cd0 con 0x7f2010103b00 2024-04-25T10:32:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f201522e640 1 --2- 172.21.15.43:0/956232509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2010103b00 0x7f201019a240 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f200c006010 tx=0x7f200c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f20067fc640 1 -- 172.21.15.43:0/956232509 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c03f030 con 0x7f2010103b00 2024-04-25T10:32:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f20067fc640 1 -- 172.21.15.43:0/956232509 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f200c003aa0 con 0x7f2010103b00 2024-04-25T10:32:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f20067fc640 1 -- 172.21.15.43:0/956232509 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c035070 con 0x7f2010103b00 2024-04-25T10:32:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f2017492640 1 -- 172.21.15.43:0/956232509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20101a1f60 con 0x7f2010103b00 2024-04-25T10:32:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.252+0000 7f2017492640 1 -- 172.21.15.43:0/956232509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20101a23c0 con 0x7f2010103b00 2024-04-25T10:32:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.253+0000 7f2017492640 1 -- 172.21.15.43:0/956232509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2010108850 con 0x7f2010103b00 2024-04-25T10:32:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.255+0000 7f20067fc640 1 -- 172.21.15.43:0/956232509 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f200c042030 con 0x7f2010103b00 2024-04-25T10:32:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.255+0000 7f20067fc640 1 --2- 172.21.15.43:0/956232509 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1ffc0afb70 0x7f1ffc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.255+0000 7f2014a2d640 1 --2- 172.21.15.43:0/956232509 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1ffc0afb70 0x7f1ffc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.255+0000 7f20067fc640 1 -- 172.21.15.43:0/956232509 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f200c0ff600 con 0x7f2010103b00 2024-04-25T10:32:23.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.256+0000 7f2014a2d640 1 --2- 172.21.15.43:0/956232509 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1ffc0afb70 0x7f1ffc0b2080 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2000005fd0 tx=0x7f2000005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.259+0000 7f20067fc640 1 -- 172.21.15.43:0/956232509 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f200c0c81f0 con 0x7f2010103b00 2024-04-25T10:32:23.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:22 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2205378295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:32:23.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:22 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3797939015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:32:23.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:22 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2854718215' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:32:23.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:22 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2205378295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:32:23.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:22 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3797939015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:32:23.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:22 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2854718215' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:32:23.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.446+0000 7f2017492640 1 -- 172.21.15.43:0/956232509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f201002dd20 con 0x7f2010103b00 2024-04-25T10:32:23.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.447+0000 7f20067fc640 1 -- 172.21.15.43:0/956232509 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22619 (secure 0 0 0) 0x7f200c0d0270 con 0x7f2010103b00 2024-04-25T10:32:23.450 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:32:23.450 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":79,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:31:33.928341+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:31.503414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:32.224334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:32:23.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.449+0000 7f2017492640 1 -- 172.21.15.43:0/956232509 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1ffc0afb70 msgr2=0x7f1ffc0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.449+0000 7f2017492640 1 --2- 172.21.15.43:0/956232509 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1ffc0afb70 0x7f1ffc0b2080 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2000005fd0 tx=0x7f2000005ce0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.449+0000 7f2017492640 1 -- 172.21.15.43:0/956232509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2010103b00 msgr2=0x7f201019a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.449+0000 7f2017492640 1 --2- 172.21.15.43:0/956232509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2010103b00 0x7f201019a240 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f200c006010 tx=0x7f200c031a50 comp rx=0 tx=0).stop 2024-04-25T10:32:23.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.449+0000 7f2017492640 1 -- 172.21.15.43:0/956232509 shutdown_connections 2024-04-25T10:32:23.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.449+0000 7f2017492640 1 --2- 172.21.15.43:0/956232509 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f1ffc0afb70 0x7f1ffc0b2080 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.449+0000 7f2017492640 1 --2- 172.21.15.43:0/956232509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2010109a40 0x7f20101a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.449+0000 7f2017492640 1 --2- 172.21.15.43:0/956232509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2010104d40 0x7f201019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.449+0000 7f2017492640 1 --2- 172.21.15.43:0/956232509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2010103b00 0x7f201019a240 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.449+0000 7f2017492640 1 -- 172.21.15.43:0/956232509 >> 172.21.15.43:0/956232509 conn(0x7f20100fed10 msgr2=0x7f2010105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.450+0000 7f2017492640 1 -- 172.21.15.43:0/956232509 shutdown_connections 2024-04-25T10:32:23.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.450+0000 7f2017492640 1 -- 172.21.15.43:0/956232509 wait complete. 2024-04-25T10:32:23.464 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T10:30:33.261560+0000', 'modified': '2024-04-25T10:30:38.804706+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24554, 'mds_1': 24508, 'mds_2': 24472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24470': {'gid': 24470, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6835/1783762538', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 1783762538}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 1783762538}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.164:6835/1437008220', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 1437008220}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 1437008220}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6837/3305330947', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6836', 'nonce': 3305330947}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 3305330947}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6837/2595248466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6836', 'nonce': 2595248466}, {'type': 'v1', 'addr': '172.21.15.77:6837', 'nonce': 2595248466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24508': {'gid': 24508, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.164:6839/4082280107', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 4082280107}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 4082280107}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24554': {'gid': 24554, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.77:6839/172043982', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 172043982}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 172043982}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24554, 'qdb_cluster': [24472, 24508, 24554]} 2024-04-25T10:32:23.464 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T10:32:23.465 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-25T10:32:23.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.577+0000 7fdab6df3640 1 -- 172.21.15.43:0/1023647312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdab0108930 msgr2=0x7fdab010ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.577+0000 7fdab6df3640 1 --2- 172.21.15.43:0/1023647312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdab0108930 0x7fdab010ad50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009ca0 tx=0x7fdaa802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.577+0000 7fdab6df3640 1 -- 172.21.15.43:0/1023647312 shutdown_connections 2024-04-25T10:32:23.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.577+0000 7fdab6df3640 1 --2- 172.21.15.43:0/1023647312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdab0108930 0x7fdab010ad50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.577+0000 7fdab6df3640 1 --2- 172.21.15.43:0/1023647312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab0103c30 0x7fdab0104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.577+0000 7fdab6df3640 1 --2- 172.21.15.43:0/1023647312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdab01029f0 0x7fdab01036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.577+0000 7fdab6df3640 1 -- 172.21.15.43:0/1023647312 >> 172.21.15.43:0/1023647312 conn(0x7fdab00fdbc0 msgr2=0x7fdab0100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.577+0000 7fdab6df3640 1 -- 172.21.15.43:0/1023647312 shutdown_connections 2024-04-25T10:32:23.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.577+0000 7fdab6df3640 1 -- 172.21.15.43:0/1023647312 wait complete. 2024-04-25T10:32:23.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdab6df3640 1 Processor -- start 2024-04-25T10:32:23.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdab6df3640 1 -- start start 2024-04-25T10:32:23.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdab6df3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdab01029f0 0x7fdab0075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdab6df3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab0103c30 0x7fdab01a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdab6df3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdab0108930 0x7fdab01a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdab6df3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdab010e520 con 0x7fdab01029f0 2024-04-25T10:32:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdab6df3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdab010e3a0 con 0x7fdab0108930 2024-04-25T10:32:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdab6df3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdab010e6a0 con 0x7fdab0103c30 2024-04-25T10:32:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdab4b8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdab01029f0 0x7fdab0075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdab4b8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdab01029f0 0x7fdab0075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39924/0 (socket says 172.21.15.43:39924) 2024-04-25T10:32:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdab4b8f640 1 -- 172.21.15.43:0/930071703 learned_addr learned my addr 172.21.15.43:0/930071703 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdab5390640 1 --2- 172.21.15.43:0/930071703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdab0108930 0x7fdab01a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdaaffff640 1 --2- 172.21.15.43:0/930071703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab0103c30 0x7fdab01a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdab4b8f640 1 -- 172.21.15.43:0/930071703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab0103c30 msgr2=0x7fdab01a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdab4b8f640 1 --2- 172.21.15.43:0/930071703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab0103c30 0x7fdab01a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdab4b8f640 1 -- 172.21.15.43:0/930071703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdab0108930 msgr2=0x7fdab01a38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdab4b8f640 1 --2- 172.21.15.43:0/930071703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdab0108930 0x7fdab01a38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdab4b8f640 1 -- 172.21.15.43:0/930071703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab01a3f00 con 0x7fdab01029f0 2024-04-25T10:32:23.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdab5390640 1 --2- 172.21.15.43:0/930071703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdab0108930 0x7fdab01a38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:32:23.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdaaffff640 1 --2- 172.21.15.43:0/930071703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab0103c30 0x7fdab01a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:32:23.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.578+0000 7fdab4b8f640 1 --2- 172.21.15.43:0/930071703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdab01029f0 0x7fdab0075840 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4005fd0 tx=0x7fdaa400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:23.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.579+0000 7fdaadffb640 1 -- 172.21.15.43:0/930071703 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4017070 con 0x7fdab01029f0 2024-04-25T10:32:23.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.579+0000 7fdaadffb640 1 -- 172.21.15.43:0/930071703 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaa4009c30 con 0x7fdab01029f0 2024-04-25T10:32:23.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.579+0000 7fdaadffb640 1 -- 172.21.15.43:0/930071703 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4003d10 con 0x7fdab01029f0 2024-04-25T10:32:23.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.579+0000 7fdab6df3640 1 -- 172.21.15.43:0/930071703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab01a41f0 con 0x7fdab01029f0 2024-04-25T10:32:23.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.579+0000 7fdab6df3640 1 -- 172.21.15.43:0/930071703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab01a46f0 con 0x7fdab01029f0 2024-04-25T10:32:23.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.581+0000 7fdaadffb640 1 -- 172.21.15.43:0/930071703 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fdaa4003eb0 con 0x7fdab01029f0 2024-04-25T10:32:23.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.581+0000 7fdabdd7f740 1 Processor -- start 2024-04-25T10:32:23.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.581+0000 7fdabdd7f740 1 -- start start 2024-04-25T10:32:23.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.582+0000 7fdabdd7f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf97e270 0x55f7cfd1f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.582+0000 7fdabdd7f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7cf9553b0 0x55f7cfd1f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.582+0000 7fdabdd7f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7cf95d590 0x55f7cfd267f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.582+0000 7fdabdd7f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f7cfc84740 con 0x55f7cf97e270 2024-04-25T10:32:23.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.582+0000 7fdabdd7f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f7cfc203a0 con 0x55f7cf95d590 2024-04-25T10:32:23.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.582+0000 7fdabdd7f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f7cfc80700 con 0x55f7cf9553b0 2024-04-25T10:32:23.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.582+0000 7fdab5390640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf97e270 0x55f7cfd1f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.582+0000 7fdab5390640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf97e270 0x55f7cfd1f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39928/0 (socket says 172.21.15.43:39928) 2024-04-25T10:32:23.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.582+0000 7fdab5390640 1 -- 172.21.15.43:0/2041387815 learned_addr learned my addr 172.21.15.43:0/2041387815 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:23.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.582+0000 7fdaadffb640 1 --2- 172.21.15.43:0/930071703 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fdaa00afb70 0x7fdaa00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.582+0000 7fdaaffff640 1 --2- 172.21.15.43:0/2041387815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7cf95d590 0x55f7cfd267f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.582+0000 7fdab4b8f640 1 --2- 172.21.15.43:0/2041387815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7cf9553b0 0x55f7cfd1f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.582+0000 7fdab5390640 1 --2- 172.21.15.43:0/930071703 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fdaa00afb70 0x7fdaa00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.582+0000 7fdab5390640 1 -- 172.21.15.43:0/2041387815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7cf9553b0 msgr2=0x55f7cfd1f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.582+0000 7fdab5390640 1 --2- 172.21.15.43:0/2041387815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7cf9553b0 0x55f7cfd1f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.582+0000 7fdaadffb640 1 -- 172.21.15.43:0/930071703 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fdaa40de8f0 con 0x7fdab01029f0 2024-04-25T10:32:23.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.582+0000 7fdab5390640 1 -- 172.21.15.43:0/2041387815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7cf95d590 msgr2=0x55f7cfd267f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.582+0000 7fdab5390640 1 --2- 172.21.15.43:0/2041387815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7cf95d590 0x55f7cfd267f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.582+0000 7fdab5390640 1 -- 172.21.15.43:0/2041387815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7cfc3c750 con 0x55f7cf97e270 2024-04-25T10:32:23.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.583+0000 7fdab5390640 1 --2- 172.21.15.43:0/2041387815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf97e270 0x55f7cfd1f370 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8031a90 tx=0x7fdaa80316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40a05090b7ccba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.583+0000 7fda9affd640 1 -- 172.21.15.43:0/2041387815 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8004d40 con 0x55f7cf97e270 2024-04-25T10:32:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.583+0000 7fdab5390640 1 --2- 172.21.15.43:0/930071703 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fdaa00afb70 0x7fdaa00b2080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdaa804c000 tx=0x7fdaa804a000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.583+0000 7fda9affd640 1 -- 172.21.15.43:0/2041387815 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaa804d070 con 0x55f7cf97e270 2024-04-25T10:32:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.583+0000 7fda9affd640 1 -- 172.21.15.43:0/2041387815 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa802f9a0 con 0x55f7cf97e270 2024-04-25T10:32:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.584+0000 7fdabdd7f740 1 -- 172.21.15.43:0/2041387815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf97e270 msgr2=0x55f7cfd1f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.584+0000 7fdabdd7f740 1 --2- 172.21.15.43:0/2041387815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf97e270 0x55f7cfd1f370 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8031a90 tx=0x7fdaa80316e0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.584+0000 7fdabdd7f740 1 -- 172.21.15.43:0/2041387815 shutdown_connections 2024-04-25T10:32:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.584+0000 7fdabdd7f740 1 --2- 172.21.15.43:0/2041387815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7cf95d590 0x55f7cfd267f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.584+0000 7fdabdd7f740 1 --2- 172.21.15.43:0/2041387815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7cf9553b0 0x55f7cfd1f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.584+0000 7fdabdd7f740 1 --2- 172.21.15.43:0/2041387815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf97e270 0x55f7cfd1f370 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.584+0000 7fdabdd7f740 1 -- 172.21.15.43:0/2041387815 >> 172.21.15.43:0/2041387815 conn(0x55f7cf9723c0 msgr2=0x55f7cfd1ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.584+0000 7fdabdd7f740 1 -- 172.21.15.43:0/2041387815 shutdown_connections 2024-04-25T10:32:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.584+0000 7fdabdd7f740 1 -- 172.21.15.43:0/2041387815 wait complete. 2024-04-25T10:32:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.584+0000 7fdabdd7f740 1 -- start start 2024-04-25T10:32:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.584+0000 7fdabdd7f740 1 Processor -- start 2024-04-25T10:32:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.584+0000 7fdabdd7f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7cf9553b0 0x55f7cfd292d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.584+0000 7fdabdd7f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7cf95d590 0x55f7cfd29810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.584+0000 7fdabdd7f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf97e270 0x55f7cfd30770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.585+0000 7fdabdd7f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f7cfd27240 con 0x55f7cf97e270 2024-04-25T10:32:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.585+0000 7fdabdd7f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f7cfc0bfe0 con 0x55f7cf95d590 2024-04-25T10:32:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.585+0000 7fdabdd7f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f7cfd273c0 con 0x55f7cf9553b0 2024-04-25T10:32:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.585+0000 7fdabdd7f740 10 client.0 _ll_register_callbacks cb 0x55f7cfc22150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.585+0000 7fdab5390640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf97e270 0x55f7cfd30770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.585+0000 7fdab5390640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf97e270 0x55f7cfd30770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39940/0 (socket says 172.21.15.43:39940) 2024-04-25T10:32:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.585+0000 7fdab5390640 1 -- 172.21.15.43:0/1932871173 learned_addr learned my addr 172.21.15.43:0/1932871173 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.585+0000 7fdaaffff640 1 --2- 172.21.15.43:0/1932871173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7cf95d590 0x55f7cfd29810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.585+0000 7fdab4b8f640 1 --2- 172.21.15.43:0/1932871173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7cf9553b0 0x55f7cfd292d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.585+0000 7fdab5390640 1 -- 172.21.15.43:0/1932871173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7cf9553b0 msgr2=0x55f7cfd292d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.585+0000 7fdab5390640 1 --2- 172.21.15.43:0/1932871173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7cf9553b0 0x55f7cfd292d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.585+0000 7fdab5390640 1 -- 172.21.15.43:0/1932871173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7cf95d590 msgr2=0x55f7cfd29810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.585+0000 7fdab5390640 1 --2- 172.21.15.43:0/1932871173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7cf95d590 0x55f7cfd29810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.585+0000 7fdab5390640 1 -- 172.21.15.43:0/1932871173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7cf9962d0 con 0x55f7cf97e270 2024-04-25T10:32:23.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.585+0000 7fdab5390640 1 --2- 172.21.15.43:0/1932871173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf97e270 0x55f7cfd30770 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdaa802a900 tx=0x7fdaa8031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1667b3ac8aeb8dff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:23.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.585+0000 7fda8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:32:23.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.585+0000 7fda8ffff640 1 -- 172.21.15.43:0/1932871173 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8059020 con 0x55f7cf97e270 2024-04-25T10:32:23.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.585+0000 7fda8ffff640 1 -- 172.21.15.43:0/1932871173 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fdaa8004b40 con 0x55f7cf97e270 2024-04-25T10:32:23.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.585+0000 7fdabdd7f740 1 -- 172.21.15.43:0/1932871173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaa804d070 con 0x55f7cf97e270 2024-04-25T10:32:23.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.586+0000 7fda8ffff640 1 -- 172.21.15.43:0/1932871173 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fdaa8004d40 con 0x55f7cf97e270 2024-04-25T10:32:23.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.586+0000 7fdabdd7f740 10 client.15225 fetch_fsmap learned FSMap version 23 2024-04-25T10:32:23.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.586+0000 7fdabdd7f740 1 -- 172.21.15.43:0/1932871173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f7cfa606b0 con 0x55f7cf97e270 2024-04-25T10:32:23.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.586+0000 7fda8ffff640 1 -- 172.21.15.43:0/1932871173 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaa804d070 con 0x55f7cf97e270 2024-04-25T10:32:23.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.586+0000 7fda8ffff640 1 -- 172.21.15.43:0/1932871173 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fdaa8030820 con 0x55f7cf97e270 2024-04-25T10:32:23.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.586+0000 7fdabdd7f740 10 client.15225 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:32:23.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.586+0000 7fdabdd7f740 10 client.15225 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T10:32:23.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.586+0000 7fdabdd7f740 20 client.15225 populate_metadata read hostname 'smithi043' 2024-04-25T10:32:23.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.586+0000 7fdabdd7f740 1 --2- 172.21.15.43:0/1932871173 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55f7cf93ce40 0x55f7cfd36bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.586+0000 7fdabdd7f740 4 client.15225 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:32:23.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.586+0000 7fdabdd7f740 1 -- 172.21.15.43:0/1932871173 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7cfc92350 con 0x55f7cf93ce40 2024-04-25T10:32:23.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.586+0000 7fdab4b8f640 1 --2- 172.21.15.43:0/1932871173 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55f7cf93ce40 0x55f7cfd36bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.587+0000 7fdab4b8f640 1 --2- 172.21.15.43:0/1932871173 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55f7cf93ce40 0x55f7cfd36bb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=530743cfb7ce65e4 server_cookie=36cd29d2d3b7d8c in_seq=0 out_seq=0 2024-04-25T10:32:23.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.587+0000 7fda8ffff640 10 client.15225 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:32:23.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.588+0000 7fda8ffff640 1 -- 172.21.15.43:0/1932871173 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f7cfc92350 con 0x55f7cf93ce40 2024-04-25T10:32:23.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.588+0000 7fda8ffff640 10 client.15225 handle_command_reply: tid=0 2024-04-25T10:32:23.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.588+0000 7fdabdd7f740 1 client.15225 shutdown 2024-04-25T10:32:23.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.588+0000 7fdabdd7f740 1 -- 172.21.15.43:0/1932871173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf97e270 msgr2=0x55f7cfd30770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.588+0000 7fdabdd7f740 1 --2- 172.21.15.43:0/1932871173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf97e270 0x55f7cfd30770 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdaa802a900 tx=0x7fdaa8031840 comp rx=0 tx=0).stop 2024-04-25T10:32:23.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.588+0000 7fdabdd7f740 1 -- 172.21.15.43:0/1932871173 shutdown_connections 2024-04-25T10:32:23.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.588+0000 7fdabdd7f740 1 --2- 172.21.15.43:0/1932871173 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55f7cf93ce40 0x55f7cfd36bb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.588+0000 7fdabdd7f740 1 --2- 172.21.15.43:0/1932871173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7cf95d590 0x55f7cfd29810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.588+0000 7fdabdd7f740 1 --2- 172.21.15.43:0/1932871173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf97e270 0x55f7cfd30770 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.588+0000 7fdabdd7f740 1 --2- 172.21.15.43:0/1932871173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7cf9553b0 0x55f7cfd292d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.588+0000 7fda8ffff640 0 client.15225 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:32:23.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.588+0000 7fdabdd7f740 1 -- 172.21.15.43:0/1932871173 >> 172.21.15.43:0/1932871173 conn(0x55f7cf9723c0 msgr2=0x55f7cfd19d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.588+0000 7fdabdd7f740 1 -- 172.21.15.43:0/1932871173 shutdown_connections 2024-04-25T10:32:23.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.588+0000 7fdabdd7f740 1 -- 172.21.15.43:0/1932871173 wait complete. 2024-04-25T10:32:23.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.588+0000 7fdabdd7f740 20 client.15225 trim_cache size 0 max 16384 2024-04-25T10:32:23.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.607+0000 7fdabdd7f740 1 Processor -- start 2024-04-25T10:32:23.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.607+0000 7fdabdd7f740 1 -- start start 2024-04-25T10:32:23.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.607+0000 7fdabdd7f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf9553b0 0x55f7cfd229c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.607+0000 7fdabdd7f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7cf95d590 0x55f7cfd28580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.607+0000 7fdabdd7f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7cf97e270 0x55f7cfd28ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.607+0000 7fdabdd7f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f7cfd34ef0 con 0x55f7cf9553b0 2024-04-25T10:32:23.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.607+0000 7fdabdd7f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f7cfd355c0 con 0x55f7cf95d590 2024-04-25T10:32:23.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.607+0000 7fdabdd7f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f7cfd35c90 con 0x55f7cf97e270 2024-04-25T10:32:23.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.607+0000 7fdab4b8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf9553b0 0x55f7cfd229c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.607+0000 7fdab4b8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf9553b0 0x55f7cfd229c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39944/0 (socket says 172.21.15.43:39944) 2024-04-25T10:32:23.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.607+0000 7fdab4b8f640 1 -- 172.21.15.43:0/3989342364 learned_addr learned my addr 172.21.15.43:0/3989342364 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:23.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.607+0000 7fdab4b8f640 1 -- 172.21.15.43:0/3989342364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7cf97e270 msgr2=0x55f7cfd28ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.607+0000 7fdab4b8f640 1 --2- 172.21.15.43:0/3989342364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7cf97e270 0x55f7cfd28ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.607+0000 7fdab4b8f640 1 -- 172.21.15.43:0/3989342364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7cf95d590 msgr2=0x55f7cfd28580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.607+0000 7fdab4b8f640 1 --2- 172.21.15.43:0/3989342364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7cf95d590 0x55f7cfd28580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.607+0000 7fdab4b8f640 1 -- 172.21.15.43:0/3989342364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7cfd240f0 con 0x55f7cf9553b0 2024-04-25T10:32:23.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.608+0000 7fdab4b8f640 1 --2- 172.21.15.43:0/3989342364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf9553b0 0x55f7cfd229c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40a4050 tx=0x7fdaa4075930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e87d925f763ea03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:23.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.608+0000 7fda9a7fc640 1 -- 172.21.15.43:0/3989342364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa40ab190 con 0x55f7cf9553b0 2024-04-25T10:32:23.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.608+0000 7fda9a7fc640 1 -- 172.21.15.43:0/3989342364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaa40afc40 con 0x55f7cf9553b0 2024-04-25T10:32:23.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.608+0000 7fda9a7fc640 1 -- 172.21.15.43:0/3989342364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4004400 con 0x55f7cf9553b0 2024-04-25T10:32:23.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.608+0000 7fdabdd7f740 1 -- 172.21.15.43:0/3989342364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf9553b0 msgr2=0x55f7cfd229c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.608+0000 7fdabdd7f740 1 --2- 172.21.15.43:0/3989342364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf9553b0 0x55f7cfd229c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40a4050 tx=0x7fdaa4075930 comp rx=0 tx=0).stop 2024-04-25T10:32:23.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.609+0000 7fdabdd7f740 1 -- 172.21.15.43:0/3989342364 shutdown_connections 2024-04-25T10:32:23.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.609+0000 7fdabdd7f740 1 --2- 172.21.15.43:0/3989342364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7cf95d590 0x55f7cfd28580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.609+0000 7fdabdd7f740 1 --2- 172.21.15.43:0/3989342364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7cf97e270 0x55f7cfd28ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.609+0000 7fdabdd7f740 1 --2- 172.21.15.43:0/3989342364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf9553b0 0x55f7cfd229c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.609+0000 7fdabdd7f740 1 -- 172.21.15.43:0/3989342364 >> 172.21.15.43:0/3989342364 conn(0x55f7cf9723c0 msgr2=0x55f7cfd27910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.609+0000 7fdabdd7f740 1 -- 172.21.15.43:0/3989342364 shutdown_connections 2024-04-25T10:32:23.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.609+0000 7fdabdd7f740 1 -- 172.21.15.43:0/3989342364 wait complete. 2024-04-25T10:32:23.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.609+0000 7fdabdd7f740 1 -- start start 2024-04-25T10:32:23.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.609+0000 7fdabdd7f740 1 Processor -- start 2024-04-25T10:32:23.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.609+0000 7fdabdd7f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf9553b0 0x55f7cfd2dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.609+0000 7fdabdd7f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7cf95d590 0x55f7cfd2e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.609+0000 7fdabdd7f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7cf97e270 0x55f7cfd2f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.609+0000 7fdabdd7f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f7cfd36a30 con 0x55f7cf9553b0 2024-04-25T10:32:23.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.609+0000 7fdabdd7f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f7cfd36360 con 0x55f7cf97e270 2024-04-25T10:32:23.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.609+0000 7fdabdd7f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f7cfd37100 con 0x55f7cf95d590 2024-04-25T10:32:23.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.609+0000 7fdabdd7f740 10 client.0 _ll_register_callbacks cb 0x7fda9c009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:23.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.609+0000 7fdab4b8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf9553b0 0x55f7cfd2dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.609+0000 7fdab4b8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf9553b0 0x55f7cfd2dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39952/0 (socket says 172.21.15.43:39952) 2024-04-25T10:32:23.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.609+0000 7fdab4b8f640 1 -- 172.21.15.43:0/2687947189 learned_addr learned my addr 172.21.15.43:0/2687947189 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:23.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.610+0000 7fdab5390640 1 --2- 172.21.15.43:0/2687947189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7cf97e270 0x55f7cfd2f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.610+0000 7fdaaffff640 1 --2- 172.21.15.43:0/2687947189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7cf95d590 0x55f7cfd2e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.610+0000 7fdab4b8f640 1 -- 172.21.15.43:0/2687947189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7cf95d590 msgr2=0x55f7cfd2e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.610+0000 7fdab4b8f640 1 --2- 172.21.15.43:0/2687947189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7cf95d590 0x55f7cfd2e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.610+0000 7fdab4b8f640 1 -- 172.21.15.43:0/2687947189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7cf97e270 msgr2=0x55f7cfd2f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.610+0000 7fdab4b8f640 1 --2- 172.21.15.43:0/2687947189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7cf97e270 0x55f7cfd2f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.610+0000 7fdab4b8f640 1 -- 172.21.15.43:0/2687947189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7cfd2f730 con 0x55f7cf9553b0 2024-04-25T10:32:23.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.610+0000 7fdaaffff640 1 --2- 172.21.15.43:0/2687947189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7cf95d590 0x55f7cfd2e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:23.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.610+0000 7fdab4b8f640 1 --2- 172.21.15.43:0/2687947189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf9553b0 0x55f7cfd2dd00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40a7690 tx=0x7fdaa4005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2711dd5570352b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:23.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.610+0000 7fda8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:32:23.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.610+0000 7fda8ffff640 1 -- 172.21.15.43:0/2687947189 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa40dc510 con 0x55f7cf9553b0 2024-04-25T10:32:23.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.610+0000 7fda8ffff640 1 -- 172.21.15.43:0/2687947189 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fdaa40041a0 con 0x55f7cf9553b0 2024-04-25T10:32:23.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.610+0000 7fdabdd7f740 1 -- 172.21.15.43:0/2687947189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaa40afc40 con 0x55f7cf9553b0 2024-04-25T10:32:23.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.610+0000 7fda8ffff640 1 -- 172.21.15.43:0/2687947189 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fdaa40ab190 con 0x55f7cf9553b0 2024-04-25T10:32:23.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.610+0000 7fdabdd7f740 10 client.15231 fetch_fsmap learned FSMap version 23 2024-04-25T10:32:23.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.610+0000 7fdabdd7f740 1 -- 172.21.15.43:0/2687947189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f7cfda00f0 con 0x55f7cf9553b0 2024-04-25T10:32:23.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.610+0000 7fda8ffff640 1 -- 172.21.15.43:0/2687947189 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaa40afc40 con 0x55f7cf9553b0 2024-04-25T10:32:23.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.611+0000 7fda8ffff640 1 -- 172.21.15.43:0/2687947189 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fdaa40a2950 con 0x55f7cf9553b0 2024-04-25T10:32:23.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.611+0000 7fdabdd7f740 10 client.15231 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:32:23.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.611+0000 7fdabdd7f740 10 client.15231 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T10:32:23.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.611+0000 7fdabdd7f740 20 client.15231 populate_metadata read hostname 'smithi043' 2024-04-25T10:32:23.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.611+0000 7fdabdd7f740 1 --2- 172.21.15.43:0/2687947189 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55f7cf912280 0x55f7cfda34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.611+0000 7fdabdd7f740 4 client.15231 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:32:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.611+0000 7fdabdd7f740 1 -- 172.21.15.43:0/2687947189 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "status"}) -- 0x55f7cfc8c1c0 con 0x55f7cf912280 2024-04-25T10:32:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.611+0000 7fdaaffff640 1 --2- 172.21.15.43:0/2687947189 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55f7cf912280 0x55f7cfda34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.611+0000 7fdaaffff640 1 --2- 172.21.15.43:0/2687947189 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55f7cf912280 0x55f7cfda34e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1007e81cae38845 server_cookie=25ed61ab6f99a847 in_seq=0 out_seq=0 2024-04-25T10:32:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.611+0000 7fda8ffff640 10 client.15231 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:32:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.612+0000 7fda8ffff640 1 -- 172.21.15.43:0/2687947189 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f7cfc8c1c0 con 0x55f7cf912280 2024-04-25T10:32:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.612+0000 7fda8ffff640 10 client.15231 handle_command_reply: tid=0 2024-04-25T10:32:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.612+0000 7fdabdd7f740 1 client.15231 shutdown 2024-04-25T10:32:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.612+0000 7fdabdd7f740 1 -- 172.21.15.43:0/2687947189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf9553b0 msgr2=0x55f7cfd2dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.612+0000 7fdabdd7f740 1 --2- 172.21.15.43:0/2687947189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf9553b0 0x55f7cfd2dd00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40a7690 tx=0x7fdaa4005af0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.612+0000 7fdabdd7f740 1 -- 172.21.15.43:0/2687947189 shutdown_connections 2024-04-25T10:32:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.612+0000 7fdabdd7f740 1 --2- 172.21.15.43:0/2687947189 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55f7cf912280 0x55f7cfda34e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.612+0000 7fdabdd7f740 1 --2- 172.21.15.43:0/2687947189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7cf97e270 0x55f7cfd2f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.612+0000 7fda8ffff640 0 client.15231 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:32:23.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.612+0000 7fdabdd7f740 1 --2- 172.21.15.43:0/2687947189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7cf95d590 0x55f7cfd2e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.612+0000 7fdabdd7f740 1 --2- 172.21.15.43:0/2687947189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7cf9553b0 0x55f7cfd2dd00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.612+0000 7fdabdd7f740 1 -- 172.21.15.43:0/2687947189 >> 172.21.15.43:0/2687947189 conn(0x55f7cf9723c0 msgr2=0x55f7cfd27850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.612+0000 7fdabdd7f740 1 -- 172.21.15.43:0/2687947189 shutdown_connections 2024-04-25T10:32:23.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.612+0000 7fdabdd7f740 1 -- 172.21.15.43:0/2687947189 wait complete. 2024-04-25T10:32:23.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.612+0000 7fdabdd7f740 20 client.15231 trim_cache size 0 max 16384 2024-04-25T10:32:23.621 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:32:23.621 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "94d0035c-02ed-11ef-bc93-c7b262605968", 2024-04-25T10:32:23.622 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 2, 2024-04-25T10:32:23.622 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24472, 2024-04-25T10:32:23.622 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-25T10:32:23.622 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-25T10:32:23.622 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-25T10:32:23.622 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 105.81005356599999, 2024-04-25T10:32:23.622 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 23, 2024-04-25T10:32:23.622 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 79, 2024-04-25T10:32:23.622 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 77, 2024-04-25T10:32:23.622 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 131.90538876900001 2024-04-25T10:32:23.622 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:32:23.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.613+0000 7fdab6df3640 1 -- 172.21.15.43:0/930071703 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fdaa00afb70 msgr2=0x7fdaa00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.613+0000 7fdab6df3640 1 --2- 172.21.15.43:0/930071703 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fdaa00afb70 0x7fdaa00b2080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdaa804c000 tx=0x7fdaa804a000 comp rx=0 tx=0).stop 2024-04-25T10:32:23.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.613+0000 7fdab6df3640 1 -- 172.21.15.43:0/930071703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdab01029f0 msgr2=0x7fdab0075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.613+0000 7fdab6df3640 1 --2- 172.21.15.43:0/930071703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdab01029f0 0x7fdab0075840 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4005fd0 tx=0x7fdaa400dde0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.613+0000 7fdab6df3640 1 -- 172.21.15.43:0/930071703 shutdown_connections 2024-04-25T10:32:23.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.613+0000 7fdab6df3640 1 --2- 172.21.15.43:0/930071703 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fdaa00afb70 0x7fdaa00b2080 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.613+0000 7fdab6df3640 1 --2- 172.21.15.43:0/930071703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdab0108930 0x7fdab01a38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.613+0000 7fdab6df3640 1 --2- 172.21.15.43:0/930071703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab0103c30 0x7fdab01a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.613+0000 7fdab6df3640 1 --2- 172.21.15.43:0/930071703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdab01029f0 0x7fdab0075840 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.613+0000 7fdab6df3640 1 -- 172.21.15.43:0/930071703 >> 172.21.15.43:0/930071703 conn(0x7fdab00fdbc0 msgr2=0x7fdab0104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.613+0000 7fdab6df3640 1 -- 172.21.15.43:0/930071703 shutdown_connections 2024-04-25T10:32:23.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.613+0000 7fdab6df3640 1 -- 172.21.15.43:0/930071703 wait complete. 2024-04-25T10:32:23.627 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T10:32:23.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.738+0000 7f13a74f0640 1 -- 172.21.15.43:0/1163479167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a0103c30 msgr2=0x7f13a0104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.738+0000 7f13a74f0640 1 --2- 172.21.15.43:0/1163479167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a0103c30 0x7f13a0104040 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f139c009900 tx=0x7f139c02a410 comp rx=0 tx=0).stop 2024-04-25T10:32:23.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.738+0000 7f13a74f0640 1 -- 172.21.15.43:0/1163479167 shutdown_connections 2024-04-25T10:32:23.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.738+0000 7f13a74f0640 1 --2- 172.21.15.43:0/1163479167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a0108930 0x7f13a010ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.738+0000 7f13a74f0640 1 --2- 172.21.15.43:0/1163479167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a0103c30 0x7f13a0104040 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.738+0000 7f13a74f0640 1 --2- 172.21.15.43:0/1163479167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a01029f0 0x7f13a01036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.738+0000 7f13a74f0640 1 -- 172.21.15.43:0/1163479167 >> 172.21.15.43:0/1163479167 conn(0x7f13a00fdbc0 msgr2=0x7f13a0100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.738+0000 7f13a74f0640 1 -- 172.21.15.43:0/1163479167 shutdown_connections 2024-04-25T10:32:23.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.738+0000 7f13a74f0640 1 -- 172.21.15.43:0/1163479167 wait complete. 2024-04-25T10:32:23.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.739+0000 7f13a74f0640 1 Processor -- start 2024-04-25T10:32:23.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.739+0000 7f13a74f0640 1 -- start start 2024-04-25T10:32:23.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.739+0000 7f13a74f0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a01029f0 0x7f13a00758a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.739+0000 7f13a74f0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a0103c30 0x7f13a019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.739+0000 7f13a74f0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a0108930 0x7f13a01a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.739+0000 7f13a74f0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f13a010e510 con 0x7f13a01029f0 2024-04-25T10:32:23.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.739+0000 7f13a74f0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f13a010e390 con 0x7f13a0103c30 2024-04-25T10:32:23.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.739+0000 7f13a74f0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f13a010e690 con 0x7f13a0108930 2024-04-25T10:32:23.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.739+0000 7f13a528c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a01029f0 0x7f13a00758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.739+0000 7f13a528c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a01029f0 0x7f13a00758a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39970/0 (socket says 172.21.15.43:39970) 2024-04-25T10:32:23.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.739+0000 7f13a528c640 1 -- 172.21.15.43:0/2823107653 learned_addr learned my addr 172.21.15.43:0/2823107653 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:23.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.739+0000 7f13a5a8d640 1 --2- 172.21.15.43:0/2823107653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a0108930 0x7f13a01a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.740+0000 7f13a4a8b640 1 --2- 172.21.15.43:0/2823107653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a0103c30 0x7f13a019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.740+0000 7f13a528c640 1 -- 172.21.15.43:0/2823107653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a0108930 msgr2=0x7f13a01a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.740+0000 7f13a528c640 1 --2- 172.21.15.43:0/2823107653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a0108930 0x7f13a01a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.740+0000 7f13a528c640 1 -- 172.21.15.43:0/2823107653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a0103c30 msgr2=0x7f13a019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.740+0000 7f13a528c640 1 --2- 172.21.15.43:0/2823107653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a0103c30 0x7f13a019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.740+0000 7f13a528c640 1 -- 172.21.15.43:0/2823107653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a01a1ca0 con 0x7f13a01029f0 2024-04-25T10:32:23.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.740+0000 7f13a4a8b640 1 --2- 172.21.15.43:0/2823107653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a0103c30 0x7f13a019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:23.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.740+0000 7f13a528c640 1 --2- 172.21.15.43:0/2823107653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a01029f0 0x7f13a00758a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f139400e9f0 tx=0x7f139400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.740+0000 7f13927fc640 1 -- 172.21.15.43:0/2823107653 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139400ccc0 con 0x7f13a01029f0 2024-04-25T10:32:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.740+0000 7f13927fc640 1 -- 172.21.15.43:0/2823107653 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13940035f0 con 0x7f13a01029f0 2024-04-25T10:32:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.740+0000 7f13a74f0640 1 -- 172.21.15.43:0/2823107653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a01a1f90 con 0x7f13a01029f0 2024-04-25T10:32:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.740+0000 7f13927fc640 1 -- 172.21.15.43:0/2823107653 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139400ce60 con 0x7f13a01029f0 2024-04-25T10:32:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.740+0000 7f13a74f0640 1 -- 172.21.15.43:0/2823107653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a01a23f0 con 0x7f13a01029f0 2024-04-25T10:32:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.743+0000 7f13ae39c740 1 Processor -- start 2024-04-25T10:32:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.743+0000 7f13927fc640 1 -- 172.21.15.43:0/2823107653 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1394003aa0 con 0x7f13a01029f0 2024-04-25T10:32:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.743+0000 7f13ae39c740 1 -- start start 2024-04-25T10:32:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.743+0000 7f13ae39c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c80270 0x5614b2021370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.743+0000 7f13ae39c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614b1c573b0 0x5614b20218f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.743+0000 7f13ae39c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614b1c5f590 0x5614b20287f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.743+0000 7f13ae39c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5614b1f86740 con 0x5614b1c80270 2024-04-25T10:32:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.743+0000 7f13ae39c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5614b1f223a0 con 0x5614b1c573b0 2024-04-25T10:32:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.743+0000 7f13ae39c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5614b1f82700 con 0x5614b1c5f590 2024-04-25T10:32:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.743+0000 7f13a5a8d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c80270 0x5614b2021370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.743+0000 7f13a5a8d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c80270 0x5614b2021370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39980/0 (socket says 172.21.15.43:39980) 2024-04-25T10:32:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.743+0000 7f13a5a8d640 1 -- 172.21.15.43:0/2219943327 learned_addr learned my addr 172.21.15.43:0/2219943327 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.743+0000 7f13a4a8b640 1 --2- 172.21.15.43:0/2219943327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614b1c5f590 0x5614b20287f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.743+0000 7f13a528c640 1 --2- 172.21.15.43:0/2219943327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614b1c573b0 0x5614b20218f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.743+0000 7f13a5a8d640 1 -- 172.21.15.43:0/2219943327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614b1c5f590 msgr2=0x5614b20287f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.743+0000 7f13a5a8d640 1 --2- 172.21.15.43:0/2219943327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614b1c5f590 0x5614b20287f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.743+0000 7f13a5a8d640 1 -- 172.21.15.43:0/2219943327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614b1c573b0 msgr2=0x5614b20218f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.743+0000 7f13a5a8d640 1 --2- 172.21.15.43:0/2219943327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614b1c573b0 0x5614b20218f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.743+0000 7f13a5a8d640 1 -- 172.21.15.43:0/2219943327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614b1f3e750 con 0x5614b1c80270 2024-04-25T10:32:23.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.743+0000 7f13a4a8b640 1 --2- 172.21.15.43:0/2219943327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614b1c5f590 0x5614b20287f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:23.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.744+0000 7f13927fc640 1 --2- 172.21.15.43:0/2823107653 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f138c0afb70 0x7f138c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.744+0000 7f13a5a8d640 1 --2- 172.21.15.43:0/2219943327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c80270 0x5614b2021370 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1398007f40 tx=0x7f139800f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e318fd006eb3dd14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:23.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.744+0000 7f13a528c640 1 --2- 172.21.15.43:0/2823107653 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f138c0afb70 0x7f138c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.744+0000 7f138b7fe640 1 -- 172.21.15.43:0/2219943327 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398010900 con 0x5614b1c80270 2024-04-25T10:32:23.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.744+0000 7f13927fc640 1 -- 172.21.15.43:0/2823107653 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f13940def90 con 0x7f13a01029f0 2024-04-25T10:32:23.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.744+0000 7f138b7fe640 1 -- 172.21.15.43:0/2219943327 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13980183f0 con 0x5614b1c80270 2024-04-25T10:32:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.744+0000 7f138b7fe640 1 -- 172.21.15.43:0/2219943327 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398017600 con 0x5614b1c80270 2024-04-25T10:32:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.745+0000 7f13ae39c740 1 -- 172.21.15.43:0/2219943327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c80270 msgr2=0x5614b2021370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.745+0000 7f13ae39c740 1 --2- 172.21.15.43:0/2219943327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c80270 0x5614b2021370 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1398007f40 tx=0x7f139800f850 comp rx=0 tx=0).stop 2024-04-25T10:32:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.745+0000 7f13a528c640 1 --2- 172.21.15.43:0/2823107653 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f138c0afb70 0x7f138c0b2080 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x5614b2022750 tx=0x7f1394090040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.745+0000 7f13ae39c740 1 -- 172.21.15.43:0/2219943327 shutdown_connections 2024-04-25T10:32:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.745+0000 7f13ae39c740 1 --2- 172.21.15.43:0/2219943327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614b1c573b0 0x5614b20218f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.745+0000 7f13ae39c740 1 --2- 172.21.15.43:0/2219943327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614b1c5f590 0x5614b20287f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.745+0000 7f13ae39c740 1 --2- 172.21.15.43:0/2219943327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c80270 0x5614b2021370 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.745+0000 7f13ae39c740 1 -- 172.21.15.43:0/2219943327 >> 172.21.15.43:0/2219943327 conn(0x5614b1c743c0 msgr2=0x5614b201cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.745+0000 7f13ae39c740 1 -- 172.21.15.43:0/2219943327 shutdown_connections 2024-04-25T10:32:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.745+0000 7f13ae39c740 1 -- 172.21.15.43:0/2219943327 wait complete. 2024-04-25T10:32:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.745+0000 7f13ae39c740 1 -- start start 2024-04-25T10:32:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.745+0000 7f13ae39c740 1 Processor -- start 2024-04-25T10:32:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f13ae39c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c573b0 0x5614b202b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f13ae39c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614b1c5f590 0x5614b202b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f13ae39c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614b1c80270 0x5614b20326a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f13ae39c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5614b2029240 con 0x5614b1c573b0 2024-04-25T10:32:23.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f13ae39c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5614b1f0dfe0 con 0x5614b1c80270 2024-04-25T10:32:23.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f13ae39c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5614b20293c0 con 0x5614b1c5f590 2024-04-25T10:32:23.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f13ae39c740 10 client.0 _ll_register_callbacks cb 0x5614b1f24150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:23.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f13a5a8d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c573b0 0x5614b202b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f13a5a8d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c573b0 0x5614b202b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39988/0 (socket says 172.21.15.43:39988) 2024-04-25T10:32:23.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f13a5a8d640 1 -- 172.21.15.43:0/2832352847 learned_addr learned my addr 172.21.15.43:0/2832352847 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:23.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f13a4a8b640 1 --2- 172.21.15.43:0/2832352847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614b1c5f590 0x5614b202b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f13a5a8d640 1 --2- 172.21.15.43:0/2832352847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614b1c80270 0x5614b20326a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f13a5a8d640 1 -- 172.21.15.43:0/2832352847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614b1c5f590 msgr2=0x5614b202b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f13a5a8d640 1 --2- 172.21.15.43:0/2832352847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614b1c5f590 0x5614b202b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f13a5a8d640 1 -- 172.21.15.43:0/2832352847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614b1c80270 msgr2=0x5614b20326a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f13a5a8d640 1 --2- 172.21.15.43:0/2832352847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614b1c80270 0x5614b20326a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f13a5a8d640 1 -- 172.21.15.43:0/2832352847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614b1c982d0 con 0x5614b1c573b0 2024-04-25T10:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f13a5a8d640 1 --2- 172.21.15.43:0/2832352847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c573b0 0x5614b202b200 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1398000f80 tx=0x7f13980072a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd6fbf9791633ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f1388ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f1388ff9640 1 -- 172.21.15.43:0/2832352847 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139802b060 con 0x5614b1c573b0 2024-04-25T10:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f1388ff9640 1 -- 172.21.15.43:0/2832352847 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f1398004c40 con 0x5614b1c573b0 2024-04-25T10:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.746+0000 7f13ae39c740 1 -- 172.21.15.43:0/2832352847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13980183f0 con 0x5614b1c573b0 2024-04-25T10:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.747+0000 7f1388ff9640 1 -- 172.21.15.43:0/2832352847 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1398004e40 con 0x5614b1c573b0 2024-04-25T10:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.747+0000 7f13ae39c740 10 client.15243 fetch_fsmap learned FSMap version 23 2024-04-25T10:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.747+0000 7f13ae39c740 1 -- 172.21.15.43:0/2832352847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5614b1d626b0 con 0x5614b1c573b0 2024-04-25T10:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.747+0000 7f1388ff9640 1 -- 172.21.15.43:0/2832352847 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13980183f0 con 0x5614b1c573b0 2024-04-25T10:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.747+0000 7f1388ff9640 1 -- 172.21.15.43:0/2832352847 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f1398004310 con 0x5614b1c573b0 2024-04-25T10:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.747+0000 7f13ae39c740 10 client.15243 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.747+0000 7f13ae39c740 10 client.15243 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T10:32:23.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.747+0000 7f13ae39c740 20 client.15243 populate_metadata read hostname 'smithi043' 2024-04-25T10:32:23.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.747+0000 7f13ae39c740 1 --2- 172.21.15.43:0/2832352847 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5614b1c3ee40 0x5614b2038b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.747+0000 7f13ae39c740 4 client.15243 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:32:23.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.747+0000 7f13ae39c740 1 -- 172.21.15.43:0/2832352847 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614b1f94350 con 0x5614b1c3ee40 2024-04-25T10:32:23.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.748+0000 7f13a4a8b640 1 --2- 172.21.15.43:0/2832352847 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5614b1c3ee40 0x5614b2038b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.748+0000 7f13a4a8b640 1 --2- 172.21.15.43:0/2832352847 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5614b1c3ee40 0x5614b2038b40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b68d26bbffa22ec9 server_cookie=cb4d11ea34754caa in_seq=0 out_seq=0 2024-04-25T10:32:23.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.748+0000 7f1388ff9640 10 client.15243 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:32:23.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.749+0000 7f1388ff9640 1 -- 172.21.15.43:0/2832352847 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5614b1f94350 con 0x5614b1c3ee40 2024-04-25T10:32:23.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.749+0000 7f1388ff9640 10 client.15243 handle_command_reply: tid=0 2024-04-25T10:32:23.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.749+0000 7f13ae39c740 1 client.15243 shutdown 2024-04-25T10:32:23.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.749+0000 7f13ae39c740 1 -- 172.21.15.43:0/2832352847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c573b0 msgr2=0x5614b202b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.749+0000 7f13ae39c740 1 --2- 172.21.15.43:0/2832352847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c573b0 0x5614b202b200 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1398000f80 tx=0x7f13980072a0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.749+0000 7f13ae39c740 1 -- 172.21.15.43:0/2832352847 shutdown_connections 2024-04-25T10:32:23.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.749+0000 7f13ae39c740 1 --2- 172.21.15.43:0/2832352847 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5614b1c3ee40 0x5614b2038b40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.749+0000 7f13ae39c740 1 --2- 172.21.15.43:0/2832352847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614b1c80270 0x5614b20326a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.749+0000 7f13ae39c740 1 --2- 172.21.15.43:0/2832352847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614b1c5f590 0x5614b202b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.749+0000 7f1388ff9640 0 client.15243 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:32:23.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.749+0000 7f13ae39c740 1 --2- 172.21.15.43:0/2832352847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c573b0 0x5614b202b200 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.749+0000 7f13ae39c740 1 -- 172.21.15.43:0/2832352847 >> 172.21.15.43:0/2832352847 conn(0x5614b1c743c0 msgr2=0x5614b201bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.749+0000 7f13ae39c740 1 -- 172.21.15.43:0/2832352847 shutdown_connections 2024-04-25T10:32:23.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.749+0000 7f13ae39c740 1 -- 172.21.15.43:0/2832352847 wait complete. 2024-04-25T10:32:23.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.749+0000 7f13ae39c740 20 client.15243 trim_cache size 0 max 16384 2024-04-25T10:32:23.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.768+0000 7f13ae39c740 1 Processor -- start 2024-04-25T10:32:23.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.768+0000 7f13ae39c740 1 -- start start 2024-04-25T10:32:23.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.768+0000 7f13ae39c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614b1c573b0 0x5614b20248e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.768+0000 7f13ae39c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614b1c5f590 0x5614b20297c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.768+0000 7f13ae39c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c80270 0x5614b2029d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.768+0000 7f13ae39c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5614b2036e80 con 0x5614b1c80270 2024-04-25T10:32:23.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.768+0000 7f13ae39c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5614b2037550 con 0x5614b1c573b0 2024-04-25T10:32:23.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.768+0000 7f13ae39c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5614b2037c20 con 0x5614b1c5f590 2024-04-25T10:32:23.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.768+0000 7f13a5a8d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c80270 0x5614b2029d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.768+0000 7f13a5a8d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c80270 0x5614b2029d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39994/0 (socket says 172.21.15.43:39994) 2024-04-25T10:32:23.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.768+0000 7f13a5a8d640 1 -- 172.21.15.43:0/2707200966 learned_addr learned my addr 172.21.15.43:0/2707200966 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:23.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.769+0000 7f13a5a8d640 1 -- 172.21.15.43:0/2707200966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614b1c5f590 msgr2=0x5614b20297c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.769+0000 7f13a5a8d640 1 --2- 172.21.15.43:0/2707200966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614b1c5f590 0x5614b20297c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.769+0000 7f13a5a8d640 1 -- 172.21.15.43:0/2707200966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614b1c573b0 msgr2=0x5614b20248e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.769+0000 7f13a5a8d640 1 --2- 172.21.15.43:0/2707200966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614b1c573b0 0x5614b20248e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.769+0000 7f13a5a8d640 1 -- 172.21.15.43:0/2707200966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614b2026050 con 0x5614b1c80270 2024-04-25T10:32:23.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.769+0000 7f13a5a8d640 1 --2- 172.21.15.43:0/2707200966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c80270 0x5614b2029d00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1398007390 tx=0x7f1398003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8aefe70eecc2a4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:23.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.769+0000 7f138affd640 1 -- 172.21.15.43:0/2707200966 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398036040 con 0x5614b1c80270 2024-04-25T10:32:23.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.769+0000 7f138affd640 1 -- 172.21.15.43:0/2707200966 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13980183f0 con 0x5614b1c80270 2024-04-25T10:32:23.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.769+0000 7f138affd640 1 -- 172.21.15.43:0/2707200966 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139802b040 con 0x5614b1c80270 2024-04-25T10:32:23.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.770+0000 7f13ae39c740 1 -- 172.21.15.43:0/2707200966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c80270 msgr2=0x5614b2029d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.770+0000 7f13ae39c740 1 --2- 172.21.15.43:0/2707200966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c80270 0x5614b2029d00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1398007390 tx=0x7f1398003ed0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.770+0000 7f13ae39c740 1 -- 172.21.15.43:0/2707200966 shutdown_connections 2024-04-25T10:32:23.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.770+0000 7f13ae39c740 1 --2- 172.21.15.43:0/2707200966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c80270 0x5614b2029d00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.770+0000 7f13ae39c740 1 --2- 172.21.15.43:0/2707200966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614b1c5f590 0x5614b20297c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.770+0000 7f13ae39c740 1 --2- 172.21.15.43:0/2707200966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614b1c573b0 0x5614b20248e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.770+0000 7f13ae39c740 1 -- 172.21.15.43:0/2707200966 >> 172.21.15.43:0/2707200966 conn(0x5614b1c743c0 msgr2=0x5614b2021fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.770+0000 7f13ae39c740 1 -- 172.21.15.43:0/2707200966 shutdown_connections 2024-04-25T10:32:23.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.770+0000 7f13ae39c740 1 -- 172.21.15.43:0/2707200966 wait complete. 2024-04-25T10:32:23.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.770+0000 7f13ae39c740 1 -- start start 2024-04-25T10:32:23.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.770+0000 7f13ae39c740 1 Processor -- start 2024-04-25T10:32:23.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.771+0000 7f13ae39c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614b1c573b0 0x5614b202d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.771+0000 7f13ae39c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c5f590 0x5614b202dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.771+0000 7f13ae39c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614b1c80270 0x5614b2030b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.771+0000 7f13ae39c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5614b20389c0 con 0x5614b1c5f590 2024-04-25T10:32:23.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.771+0000 7f13ae39c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5614b20382f0 con 0x5614b1c80270 2024-04-25T10:32:23.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.771+0000 7f13ae39c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5614b2039090 con 0x5614b1c573b0 2024-04-25T10:32:23.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.771+0000 7f13ae39c740 10 client.0 _ll_register_callbacks cb 0x7f1398007d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:23.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.771+0000 7f13a4a8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c5f590 0x5614b202dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.771+0000 7f13a4a8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c5f590 0x5614b202dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39998/0 (socket says 172.21.15.43:39998) 2024-04-25T10:32:23.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.771+0000 7f13a4a8b640 1 -- 172.21.15.43:0/2397220139 learned_addr learned my addr 172.21.15.43:0/2397220139 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:23.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.771+0000 7f13a5a8d640 1 --2- 172.21.15.43:0/2397220139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614b1c573b0 0x5614b202d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.771+0000 7f13a4a8b640 1 -- 172.21.15.43:0/2397220139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614b1c573b0 msgr2=0x5614b202d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.771+0000 7f13a4a8b640 1 --2- 172.21.15.43:0/2397220139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614b1c573b0 0x5614b202d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.771+0000 7f13a4a8b640 1 -- 172.21.15.43:0/2397220139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614b1c80270 msgr2=0x5614b2030b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.771+0000 7f13a5a8d640 1 --2- 172.21.15.43:0/2397220139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614b1c80270 0x5614b2030b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.771+0000 7f13a4a8b640 1 --2- 172.21.15.43:0/2397220139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614b1c80270 0x5614b2030b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.771+0000 7f13a4a8b640 1 -- 172.21.15.43:0/2397220139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614b20310e0 con 0x5614b1c5f590 2024-04-25T10:32:23.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.771+0000 7f13a4a8b640 1 --2- 172.21.15.43:0/2397220139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c5f590 0x5614b202dc50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f139c02a930 tx=0x7f139c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1be6d18b2e1c5ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:23.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.771+0000 7f1388ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:32:23.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.771+0000 7f1388ff9640 1 -- 172.21.15.43:0/2397220139 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c035070 con 0x5614b1c5f590 2024-04-25T10:32:23.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.772+0000 7f1388ff9640 1 -- 172.21.15.43:0/2397220139 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f139c004240 con 0x5614b1c5f590 2024-04-25T10:32:23.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.772+0000 7f13ae39c740 1 -- 172.21.15.43:0/2397220139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13980183f0 con 0x5614b1c5f590 2024-04-25T10:32:23.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.772+0000 7f1388ff9640 1 -- 172.21.15.43:0/2397220139 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f139c032980 con 0x5614b1c5f590 2024-04-25T10:32:23.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.772+0000 7f13ae39c740 10 client.15249 fetch_fsmap learned FSMap version 23 2024-04-25T10:32:23.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.772+0000 7f13ae39c740 1 -- 172.21.15.43:0/2397220139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5614b20a1b20 con 0x5614b1c5f590 2024-04-25T10:32:23.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.772+0000 7f1388ff9640 1 -- 172.21.15.43:0/2397220139 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13980183f0 con 0x5614b1c5f590 2024-04-25T10:32:23.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.772+0000 7f1388ff9640 1 -- 172.21.15.43:0/2397220139 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f139c004b40 con 0x5614b1c5f590 2024-04-25T10:32:23.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.772+0000 7f13ae39c740 10 client.15249 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:32:23.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.772+0000 7f13ae39c740 10 client.15249 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T10:32:23.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.772+0000 7f13ae39c740 20 client.15249 populate_metadata read hostname 'smithi043' 2024-04-25T10:32:23.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.772+0000 7f13ae39c740 1 --2- 172.21.15.43:0/2397220139 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5614b1c14280 0x5614b20a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.772+0000 7f13ae39c740 4 client.15249 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:32:23.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.772+0000 7f13ae39c740 1 -- 172.21.15.43:0/2397220139 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "status"}) -- 0x5614b1f8e1c0 con 0x5614b1c14280 2024-04-25T10:32:23.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.772+0000 7f13a5a8d640 1 --2- 172.21.15.43:0/2397220139 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5614b1c14280 0x5614b20a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.773+0000 7f1388ff9640 10 client.15249 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:32:23.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.773+0000 7f13a5a8d640 1 --2- 172.21.15.43:0/2397220139 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5614b1c14280 0x5614b20a54b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b07b8f943e3340a5 server_cookie=a7d74fbe320e0046 in_seq=0 out_seq=0 2024-04-25T10:32:23.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.773+0000 7f1388ff9640 1 -- 172.21.15.43:0/2397220139 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5614b1f8e1c0 con 0x5614b1c14280 2024-04-25T10:32:23.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.773+0000 7f1388ff9640 10 client.15249 handle_command_reply: tid=0 2024-04-25T10:32:23.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.773+0000 7f13ae39c740 1 client.15249 shutdown 2024-04-25T10:32:23.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.773+0000 7f13ae39c740 1 -- 172.21.15.43:0/2397220139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c5f590 msgr2=0x5614b202dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.773+0000 7f13ae39c740 1 --2- 172.21.15.43:0/2397220139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c5f590 0x5614b202dc50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f139c02a930 tx=0x7f139c005a20 comp rx=0 tx=0).stop 2024-04-25T10:32:23.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.773+0000 7f13ae39c740 1 -- 172.21.15.43:0/2397220139 shutdown_connections 2024-04-25T10:32:23.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.773+0000 7f13ae39c740 1 --2- 172.21.15.43:0/2397220139 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5614b1c14280 0x5614b20a54b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.773+0000 7f13ae39c740 1 --2- 172.21.15.43:0/2397220139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614b1c80270 0x5614b2030b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.773+0000 7f13ae39c740 1 --2- 172.21.15.43:0/2397220139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614b1c5f590 0x5614b202dc50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.773+0000 7f13ae39c740 1 --2- 172.21.15.43:0/2397220139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614b1c573b0 0x5614b202d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.773+0000 7f1388ff9640 0 client.15249 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:32:23.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.773+0000 7f13ae39c740 1 -- 172.21.15.43:0/2397220139 >> 172.21.15.43:0/2397220139 conn(0x5614b1c743c0 msgr2=0x5614b2024790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.773+0000 7f13ae39c740 1 -- 172.21.15.43:0/2397220139 shutdown_connections 2024-04-25T10:32:23.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.773+0000 7f13ae39c740 1 -- 172.21.15.43:0/2397220139 wait complete. 2024-04-25T10:32:23.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.774+0000 7f13ae39c740 20 client.15249 trim_cache size 0 max 16384 2024-04-25T10:32:23.786 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:32:23.786 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "94d0035c-02ed-11ef-bc93-c7b262605968", 2024-04-25T10:32:23.786 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 1, 2024-04-25T10:32:23.786 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24508, 2024-04-25T10:32:23.786 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-25T10:32:23.786 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-25T10:32:23.787 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-25T10:32:23.787 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 106.59509830499999, 2024-04-25T10:32:23.787 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 23, 2024-04-25T10:32:23.787 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 79, 2024-04-25T10:32:23.787 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 77, 2024-04-25T10:32:23.787 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 123.903950714 2024-04-25T10:32:23.787 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:32:23.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.774+0000 7f13a74f0640 1 -- 172.21.15.43:0/2823107653 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f138c0afb70 msgr2=0x7f138c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.774+0000 7f13a74f0640 1 --2- 172.21.15.43:0/2823107653 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f138c0afb70 0x7f138c0b2080 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x5614b2022750 tx=0x7f1394090040 comp rx=0 tx=0).stop 2024-04-25T10:32:23.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.774+0000 7f13a74f0640 1 -- 172.21.15.43:0/2823107653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a01029f0 msgr2=0x7f13a00758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.774+0000 7f13a74f0640 1 --2- 172.21.15.43:0/2823107653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a01029f0 0x7f13a00758a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f139400e9f0 tx=0x7f139400eeb0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.774+0000 7f13a74f0640 1 -- 172.21.15.43:0/2823107653 shutdown_connections 2024-04-25T10:32:23.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.774+0000 7f13a74f0640 1 --2- 172.21.15.43:0/2823107653 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f138c0afb70 0x7f138c0b2080 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.774+0000 7f13a74f0640 1 --2- 172.21.15.43:0/2823107653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a0103c30 0x7f13a019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.774+0000 7f13a74f0640 1 --2- 172.21.15.43:0/2823107653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a0108930 0x7f13a01a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.774+0000 7f13a74f0640 1 --2- 172.21.15.43:0/2823107653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a01029f0 0x7f13a00758a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.774+0000 7f13a74f0640 1 -- 172.21.15.43:0/2823107653 >> 172.21.15.43:0/2823107653 conn(0x7f13a00fdbc0 msgr2=0x7f13a0104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.774+0000 7f13a74f0640 1 -- 172.21.15.43:0/2823107653 shutdown_connections 2024-04-25T10:32:23.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.774+0000 7f13a74f0640 1 -- 172.21.15.43:0/2823107653 wait complete. 2024-04-25T10:32:23.790 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:32:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.900+0000 7f9ba7b0d640 1 -- 172.21.15.43:0/1788681984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ba0109a30 msgr2=0x7f9ba010be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.900+0000 7f9ba7b0d640 1 --2- 172.21.15.43:0/1788681984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ba0109a30 0x7f9ba010be50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9b98009ca0 tx=0x7f9b9802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.900+0000 7f9ba7b0d640 1 -- 172.21.15.43:0/1788681984 shutdown_connections 2024-04-25T10:32:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.900+0000 7f9ba7b0d640 1 --2- 172.21.15.43:0/1788681984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba0104d30 0x7f9ba0105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.900+0000 7f9ba7b0d640 1 --2- 172.21.15.43:0/1788681984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ba0109a30 0x7f9ba010be50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.900+0000 7f9ba7b0d640 1 --2- 172.21.15.43:0/1788681984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ba0103af0 0x7f9ba01047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.900+0000 7f9ba7b0d640 1 -- 172.21.15.43:0/1788681984 >> 172.21.15.43:0/1788681984 conn(0x7f9ba00fece0 msgr2=0x7f9ba0101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.900+0000 7f9ba7b0d640 1 -- 172.21.15.43:0/1788681984 shutdown_connections 2024-04-25T10:32:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.900+0000 7f9ba7b0d640 1 -- 172.21.15.43:0/1788681984 wait complete. 2024-04-25T10:32:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.901+0000 7f9ba7b0d640 1 Processor -- start 2024-04-25T10:32:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.901+0000 7f9ba7b0d640 1 -- start start 2024-04-25T10:32:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.901+0000 7f9ba7b0d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba0103af0 0x7f9ba019a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.901+0000 7f9ba7b0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ba0104d30 0x7f9ba019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.901+0000 7f9ba7b0d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ba0109a30 0x7f9ba01a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.901+0000 7f9ba7b0d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba010f670 con 0x7f9ba0104d30 2024-04-25T10:32:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.901+0000 7f9ba7b0d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba010f4f0 con 0x7f9ba0103af0 2024-04-25T10:32:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.901+0000 7f9ba7b0d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba010f7f0 con 0x7f9ba0109a30 2024-04-25T10:32:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.901+0000 7f9ba50a8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ba0104d30 0x7f9ba019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.901+0000 7f9ba50a8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ba0104d30 0x7f9ba019a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40008/0 (socket says 172.21.15.43:40008) 2024-04-25T10:32:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.901+0000 7f9ba50a8640 1 -- 172.21.15.43:0/528918267 learned_addr learned my addr 172.21.15.43:0/528918267 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.901+0000 7f9ba58a9640 1 --2- 172.21.15.43:0/528918267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba0103af0 0x7f9ba019a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.901+0000 7f9ba60aa640 1 --2- 172.21.15.43:0/528918267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ba0109a30 0x7f9ba01a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.901+0000 7f9ba50a8640 1 -- 172.21.15.43:0/528918267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ba0109a30 msgr2=0x7f9ba01a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.901+0000 7f9ba50a8640 1 --2- 172.21.15.43:0/528918267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ba0109a30 0x7f9ba01a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.901+0000 7f9ba50a8640 1 -- 172.21.15.43:0/528918267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba0103af0 msgr2=0x7f9ba019a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.901+0000 7f9ba50a8640 1 --2- 172.21.15.43:0/528918267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba0103af0 0x7f9ba019a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.901+0000 7f9ba50a8640 1 -- 172.21.15.43:0/528918267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba01a1d20 con 0x7f9ba0104d30 2024-04-25T10:32:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.901+0000 7f9ba58a9640 1 --2- 172.21.15.43:0/528918267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba0103af0 0x7f9ba019a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.902+0000 7f9ba50a8640 1 --2- 172.21.15.43:0/528918267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ba0104d30 0x7f9ba019a720 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000e9f0 tx=0x7f9b9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.902+0000 7f9b96ffd640 1 -- 172.21.15.43:0/528918267 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9000cd80 con 0x7f9ba0104d30 2024-04-25T10:32:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.902+0000 7f9b96ffd640 1 -- 172.21.15.43:0/528918267 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b900108e0 con 0x7f9ba0104d30 2024-04-25T10:32:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.902+0000 7f9ba7b0d640 1 -- 172.21.15.43:0/528918267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba01a1fb0 con 0x7f9ba0104d30 2024-04-25T10:32:23.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.902+0000 7f9b96ffd640 1 -- 172.21.15.43:0/528918267 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b90018980 con 0x7f9ba0104d30 2024-04-25T10:32:23.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.902+0000 7f9ba7b0d640 1 -- 172.21.15.43:0/528918267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba01a24f0 con 0x7f9ba0104d30 2024-04-25T10:32:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.905+0000 7f9bae9c3740 1 Processor -- start 2024-04-25T10:32:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.905+0000 7f9bae9c3740 1 -- start start 2024-04-25T10:32:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.905+0000 7f9bae9c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd823270 0x5606cdbc4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.905+0000 7f9bae9c3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd7fa3b0 0x5606cdbc48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.905+0000 7f9bae9c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd802590 0x5606cdbcb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.905+0000 7f9bae9c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5606cdb29740 con 0x5606cd823270 2024-04-25T10:32:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.905+0000 7f9bae9c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5606cdac53a0 con 0x5606cd802590 2024-04-25T10:32:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.905+0000 7f9bae9c3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5606cdb25700 con 0x5606cd7fa3b0 2024-04-25T10:32:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.906+0000 7f9ba60aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd823270 0x5606cdbc4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.906+0000 7f9ba60aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd823270 0x5606cdbc4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40014/0 (socket says 172.21.15.43:40014) 2024-04-25T10:32:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.906+0000 7f9ba60aa640 1 -- 172.21.15.43:0/1984984951 learned_addr learned my addr 172.21.15.43:0/1984984951 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.906+0000 7f9b96ffd640 1 -- 172.21.15.43:0/528918267 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9b90018b20 con 0x7f9ba0104d30 2024-04-25T10:32:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.906+0000 7f9ba58a9640 1 --2- 172.21.15.43:0/1984984951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd7fa3b0 0x5606cdbc48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.906+0000 7f9ba60aa640 1 -- 172.21.15.43:0/1984984951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd7fa3b0 msgr2=0x5606cdbc48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.906+0000 7f9ba60aa640 1 --2- 172.21.15.43:0/1984984951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd7fa3b0 0x5606cdbc48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.906+0000 7f9ba60aa640 1 -- 172.21.15.43:0/1984984951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd802590 msgr2=0x5606cdbcb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.906+0000 7f9ba50a8640 1 --2- 172.21.15.43:0/1984984951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd802590 0x5606cdbcb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.906+0000 7f9ba60aa640 1 --2- 172.21.15.43:0/1984984951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd802590 0x5606cdbcb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.906+0000 7f9ba60aa640 1 -- 172.21.15.43:0/1984984951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606cdae1750 con 0x5606cd823270 2024-04-25T10:32:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.906+0000 7f9ba50a8640 1 --2- 172.21.15.43:0/1984984951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd802590 0x5606cdbcb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:32:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.906+0000 7f9ba60aa640 1 --2- 172.21.15.43:0/1984984951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd823270 0x5606cdbc4370 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9b98006010 tx=0x7f9b9802f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675e89a041c25a86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.906+0000 7f9b8bfff640 1 -- 172.21.15.43:0/1984984951 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b98031450 con 0x5606cd823270 2024-04-25T10:32:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.906+0000 7f9b96ffd640 1 --2- 172.21.15.43:0/528918267 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9b8c0afb70 0x7f9b8c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.906+0000 7f9b8bfff640 1 -- 172.21.15.43:0/1984984951 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b98036070 con 0x5606cd823270 2024-04-25T10:32:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.906+0000 7f9b8bfff640 1 -- 172.21.15.43:0/1984984951 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b98004530 con 0x5606cd823270 2024-04-25T10:32:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.907+0000 7f9ba58a9640 1 --2- 172.21.15.43:0/528918267 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9b8c0afb70 0x7f9b8c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.907+0000 7f9b96ffd640 1 -- 172.21.15.43:0/528918267 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f9b900def90 con 0x7f9ba0104d30 2024-04-25T10:32:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.907+0000 7f9bae9c3740 1 -- 172.21.15.43:0/1984984951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd823270 msgr2=0x5606cdbc4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.907+0000 7f9ba58a9640 1 --2- 172.21.15.43:0/528918267 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9b8c0afb70 0x7f9b8c0b2080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x5606cdbc5750 tx=0x7f9b9c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.907+0000 7f9bae9c3740 1 --2- 172.21.15.43:0/1984984951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd823270 0x5606cdbc4370 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9b98006010 tx=0x7f9b9802f3a0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.907+0000 7f9bae9c3740 1 -- 172.21.15.43:0/1984984951 shutdown_connections 2024-04-25T10:32:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.907+0000 7f9bae9c3740 1 --2- 172.21.15.43:0/1984984951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd802590 0x5606cdbcb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.907+0000 7f9bae9c3740 1 --2- 172.21.15.43:0/1984984951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd7fa3b0 0x5606cdbc48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.907+0000 7f9bae9c3740 1 --2- 172.21.15.43:0/1984984951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd823270 0x5606cdbc4370 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.907+0000 7f9bae9c3740 1 -- 172.21.15.43:0/1984984951 >> 172.21.15.43:0/1984984951 conn(0x5606cd8173c0 msgr2=0x5606cdbbfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.908+0000 7f9bae9c3740 1 -- 172.21.15.43:0/1984984951 shutdown_connections 2024-04-25T10:32:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.908+0000 7f9bae9c3740 1 -- 172.21.15.43:0/1984984951 wait complete. 2024-04-25T10:32:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.908+0000 7f9bae9c3740 1 -- start start 2024-04-25T10:32:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.908+0000 7f9bae9c3740 1 Processor -- start 2024-04-25T10:32:23.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.908+0000 7f9bae9c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd7fa3b0 0x5606cdbce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.908+0000 7f9bae9c3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd802590 0x5606cdbce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.908+0000 7f9bae9c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd823270 0x5606cdbd5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.908+0000 7f9bae9c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5606cdbcc1e0 con 0x5606cd823270 2024-04-25T10:32:23.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.908+0000 7f9bae9c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5606cdab0fe0 con 0x5606cd7fa3b0 2024-04-25T10:32:23.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.908+0000 7f9bae9c3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5606cdbcc360 con 0x5606cd802590 2024-04-25T10:32:23.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.908+0000 7f9bae9c3740 10 client.0 _ll_register_callbacks cb 0x5606cdac7150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:23.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.909+0000 7f9ba60aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd823270 0x5606cdbd5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.909+0000 7f9ba60aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd823270 0x5606cdbd5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40018/0 (socket says 172.21.15.43:40018) 2024-04-25T10:32:23.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.909+0000 7f9ba60aa640 1 -- 172.21.15.43:0/4121054004 learned_addr learned my addr 172.21.15.43:0/4121054004 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:23.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.909+0000 7f9ba58a9640 1 --2- 172.21.15.43:0/4121054004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd7fa3b0 0x5606cdbce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.909+0000 7f9ba50a8640 1 --2- 172.21.15.43:0/4121054004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd802590 0x5606cdbce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.909+0000 7f9ba60aa640 1 -- 172.21.15.43:0/4121054004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd802590 msgr2=0x5606cdbce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.909+0000 7f9ba60aa640 1 --2- 172.21.15.43:0/4121054004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd802590 0x5606cdbce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.909+0000 7f9ba60aa640 1 -- 172.21.15.43:0/4121054004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd7fa3b0 msgr2=0x5606cdbce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.909+0000 7f9ba60aa640 1 --2- 172.21.15.43:0/4121054004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd7fa3b0 0x5606cdbce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.909+0000 7f9ba60aa640 1 -- 172.21.15.43:0/4121054004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606cd83b2d0 con 0x5606cd823270 2024-04-25T10:32:23.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.909+0000 7f9ba50a8640 1 --2- 172.21.15.43:0/4121054004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd802590 0x5606cdbce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:23.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.909+0000 7f9ba58a9640 1 --2- 172.21.15.43:0/4121054004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd7fa3b0 0x5606cdbce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:23.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.909+0000 7f9ba60aa640 1 --2- 172.21.15.43:0/4121054004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd823270 0x5606cdbd5760 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9b98031a60 tx=0x7f9b98009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ce486890b2279e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:23.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.909+0000 7f9b897fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:32:23.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.909+0000 7f9b897fa640 1 -- 172.21.15.43:0/4121054004 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b98051020 con 0x5606cd823270 2024-04-25T10:32:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.909+0000 7f9b897fa640 1 -- 172.21.15.43:0/4121054004 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f9b98031750 con 0x5606cd823270 2024-04-25T10:32:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.909+0000 7f9bae9c3740 1 -- 172.21.15.43:0/4121054004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b98036070 con 0x5606cd823270 2024-04-25T10:32:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.909+0000 7f9b897fa640 1 -- 172.21.15.43:0/4121054004 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9b98049040 con 0x5606cd823270 2024-04-25T10:32:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.909+0000 7f9bae9c3740 10 client.15261 fetch_fsmap learned FSMap version 23 2024-04-25T10:32:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.909+0000 7f9bae9c3740 1 -- 172.21.15.43:0/4121054004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5606cd9056b0 con 0x5606cd823270 2024-04-25T10:32:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.910+0000 7f9b897fa640 1 -- 172.21.15.43:0/4121054004 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b98036070 con 0x5606cd823270 2024-04-25T10:32:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.910+0000 7f9b897fa640 1 -- 172.21.15.43:0/4121054004 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f9b9804c030 con 0x5606cd823270 2024-04-25T10:32:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.910+0000 7f9bae9c3740 10 client.15261 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:32:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.910+0000 7f9bae9c3740 10 client.15261 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:32:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.910+0000 7f9bae9c3740 20 client.15261 populate_metadata read hostname 'smithi043' 2024-04-25T10:32:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.910+0000 7f9bae9c3740 1 --2- 172.21.15.43:0/4121054004 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5606cd7e1e40 0x5606cdbdbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.910+0000 7f9bae9c3740 4 client.15261 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:32:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.910+0000 7f9bae9c3740 1 -- 172.21.15.43:0/4121054004 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606cdb37350 con 0x5606cd7e1e40 2024-04-25T10:32:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.910+0000 7f9ba58a9640 1 --2- 172.21.15.43:0/4121054004 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5606cd7e1e40 0x5606cdbdbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.911+0000 7f9ba58a9640 1 --2- 172.21.15.43:0/4121054004 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5606cd7e1e40 0x5606cdbdbac0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d28f2e18a1fd450d server_cookie=fd85799915d2890f in_seq=0 out_seq=0 2024-04-25T10:32:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.911+0000 7f9b897fa640 10 client.15261 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:32:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.911+0000 7f9b897fa640 1 -- 172.21.15.43:0/4121054004 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5606cdb37350 con 0x5606cd7e1e40 2024-04-25T10:32:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.911+0000 7f9b897fa640 10 client.15261 handle_command_reply: tid=0 2024-04-25T10:32:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.911+0000 7f9bae9c3740 1 client.15261 shutdown 2024-04-25T10:32:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.912+0000 7f9bae9c3740 1 -- 172.21.15.43:0/4121054004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd823270 msgr2=0x5606cdbd5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.912+0000 7f9bae9c3740 1 --2- 172.21.15.43:0/4121054004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd823270 0x5606cdbd5760 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9b98031a60 tx=0x7f9b98009970 comp rx=0 tx=0).stop 2024-04-25T10:32:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.912+0000 7f9bae9c3740 1 -- 172.21.15.43:0/4121054004 shutdown_connections 2024-04-25T10:32:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.912+0000 7f9bae9c3740 1 --2- 172.21.15.43:0/4121054004 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5606cd7e1e40 0x5606cdbdbac0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.912+0000 7f9bae9c3740 1 --2- 172.21.15.43:0/4121054004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd823270 0x5606cdbd5760 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.912+0000 7f9bae9c3740 1 --2- 172.21.15.43:0/4121054004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd802590 0x5606cdbce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.912+0000 7f9b897fa640 0 client.15261 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:32:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.912+0000 7f9bae9c3740 1 --2- 172.21.15.43:0/4121054004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd7fa3b0 0x5606cdbce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.912+0000 7f9bae9c3740 1 -- 172.21.15.43:0/4121054004 >> 172.21.15.43:0/4121054004 conn(0x5606cd8173c0 msgr2=0x5606cdbbed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.912+0000 7f9bae9c3740 1 -- 172.21.15.43:0/4121054004 shutdown_connections 2024-04-25T10:32:23.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.912+0000 7f9bae9c3740 1 -- 172.21.15.43:0/4121054004 wait complete. 2024-04-25T10:32:23.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.912+0000 7f9bae9c3740 20 client.15261 trim_cache size 0 max 16384 2024-04-25T10:32:23.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.930+0000 7f9bae9c3740 1 Processor -- start 2024-04-25T10:32:23.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.930+0000 7f9bae9c3740 1 -- start start 2024-04-25T10:32:23.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9bae9c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd7e1e40 0x5606cdbcc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9bae9c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd7fa3b0 0x5606cdbcce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9bae9c3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd802590 0x5606cdbce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9bae9c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5606cdbd9e00 con 0x5606cd7fa3b0 2024-04-25T10:32:23.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9bae9c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5606cdbda4d0 con 0x5606cd7e1e40 2024-04-25T10:32:23.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9bae9c3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5606cdbdaba0 con 0x5606cd802590 2024-04-25T10:32:23.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9ba50a8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd7fa3b0 0x5606cdbcce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9ba50a8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd7fa3b0 0x5606cdbcce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40022/0 (socket says 172.21.15.43:40022) 2024-04-25T10:32:23.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9ba50a8640 1 -- 172.21.15.43:0/3072424557 learned_addr learned my addr 172.21.15.43:0/3072424557 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:23.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9ba60aa640 1 --2- 172.21.15.43:0/3072424557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd802590 0x5606cdbce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9ba58a9640 1 --2- 172.21.15.43:0/3072424557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd7e1e40 0x5606cdbcc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9ba50a8640 1 -- 172.21.15.43:0/3072424557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd802590 msgr2=0x5606cdbce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9ba50a8640 1 --2- 172.21.15.43:0/3072424557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd802590 0x5606cdbce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9ba50a8640 1 -- 172.21.15.43:0/3072424557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd7e1e40 msgr2=0x5606cdbcc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9ba50a8640 1 --2- 172.21.15.43:0/3072424557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd7e1e40 0x5606cdbcc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9ba50a8640 1 -- 172.21.15.43:0/3072424557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606cdbc7790 con 0x5606cd7fa3b0 2024-04-25T10:32:23.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9ba60aa640 1 --2- 172.21.15.43:0/3072424557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd802590 0x5606cdbce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:32:23.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9ba58a9640 1 --2- 172.21.15.43:0/3072424557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd7e1e40 0x5606cdbcc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:32:23.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9ba50a8640 1 --2- 172.21.15.43:0/3072424557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd7fa3b0 0x5606cdbcce20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9b900a35a0 tx=0x7f9b900a7400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8649e88ca5b15e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:23.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9b8b7fe640 1 -- 172.21.15.43:0/3072424557 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b900ab8d0 con 0x5606cd7fa3b0 2024-04-25T10:32:23.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9b8b7fe640 1 -- 172.21.15.43:0/3072424557 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b900b0380 con 0x5606cd7fa3b0 2024-04-25T10:32:23.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.931+0000 7f9b8b7fe640 1 -- 172.21.15.43:0/3072424557 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b900a6780 con 0x5606cd7fa3b0 2024-04-25T10:32:23.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.932+0000 7f9bae9c3740 1 -- 172.21.15.43:0/3072424557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd7fa3b0 msgr2=0x5606cdbcce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.932+0000 7f9bae9c3740 1 --2- 172.21.15.43:0/3072424557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd7fa3b0 0x5606cdbcce20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9b900a35a0 tx=0x7f9b900a7400 comp rx=0 tx=0).stop 2024-04-25T10:32:23.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.932+0000 7f9bae9c3740 1 -- 172.21.15.43:0/3072424557 shutdown_connections 2024-04-25T10:32:23.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.932+0000 7f9bae9c3740 1 --2- 172.21.15.43:0/3072424557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd802590 0x5606cdbce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.932+0000 7f9bae9c3740 1 --2- 172.21.15.43:0/3072424557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd7fa3b0 0x5606cdbcce20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.932+0000 7f9bae9c3740 1 --2- 172.21.15.43:0/3072424557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd7e1e40 0x5606cdbcc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.932+0000 7f9bae9c3740 1 -- 172.21.15.43:0/3072424557 >> 172.21.15.43:0/3072424557 conn(0x5606cd8173c0 msgr2=0x5606cdbc79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.932+0000 7f9bae9c3740 1 -- 172.21.15.43:0/3072424557 shutdown_connections 2024-04-25T10:32:23.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.932+0000 7f9bae9c3740 1 -- 172.21.15.43:0/3072424557 wait complete. 2024-04-25T10:32:23.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.933+0000 7f9bae9c3740 1 -- start start 2024-04-25T10:32:23.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.933+0000 7f9bae9c3740 1 Processor -- start 2024-04-25T10:32:23.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.933+0000 7f9bae9c3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd7e1e40 0x5606cdbd24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.933+0000 7f9bae9c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd7fa3b0 0x5606cdbd2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.933+0000 7f9bae9c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd802590 0x5606cdbd38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.933+0000 7f9bae9c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5606cdbdb940 con 0x5606cd7fa3b0 2024-04-25T10:32:23.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.933+0000 7f9bae9c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5606cdbdb270 con 0x5606cd802590 2024-04-25T10:32:23.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.933+0000 7f9bae9c3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5606cdbdc010 con 0x5606cd7e1e40 2024-04-25T10:32:23.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.933+0000 7f9bae9c3740 10 client.0 _ll_register_callbacks cb 0x7f9b9c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:23.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.933+0000 7f9ba50a8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd7fa3b0 0x5606cdbd2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.933+0000 7f9ba50a8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd7fa3b0 0x5606cdbd2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40038/0 (socket says 172.21.15.43:40038) 2024-04-25T10:32:23.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.933+0000 7f9ba50a8640 1 -- 172.21.15.43:0/1671354960 learned_addr learned my addr 172.21.15.43:0/1671354960 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:23.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.933+0000 7f9ba60aa640 1 --2- 172.21.15.43:0/1671354960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd802590 0x5606cdbd38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.933+0000 7f9ba58a9640 1 --2- 172.21.15.43:0/1671354960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd7e1e40 0x5606cdbd24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.934+0000 7f9ba50a8640 1 -- 172.21.15.43:0/1671354960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd7e1e40 msgr2=0x5606cdbd24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.934+0000 7f9ba50a8640 1 --2- 172.21.15.43:0/1671354960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd7e1e40 0x5606cdbd24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.934+0000 7f9ba50a8640 1 -- 172.21.15.43:0/1671354960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd802590 msgr2=0x5606cdbd38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:23.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.934+0000 7f9ba50a8640 1 --2- 172.21.15.43:0/1671354960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd802590 0x5606cdbd38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.934+0000 7f9ba50a8640 1 -- 172.21.15.43:0/1671354960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606cdbc9140 con 0x5606cd7fa3b0 2024-04-25T10:32:23.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.934+0000 7f9ba50a8640 1 --2- 172.21.15.43:0/1671354960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd7fa3b0 0x5606cdbd2a00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000e9c0 tx=0x7f9b90010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb285519bd546d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:23.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.934+0000 7f9b897fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:32:23.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.936+0000 7f9b897fa640 1 -- 172.21.15.43:0/1671354960 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b900c1060 con 0x5606cd7fa3b0 2024-04-25T10:32:23.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.936+0000 7f9b897fa640 1 -- 172.21.15.43:0/1671354960 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f9b90010640 con 0x5606cd7fa3b0 2024-04-25T10:32:23.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.936+0000 7f9bae9c3740 1 -- 172.21.15.43:0/1671354960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b900b0380 con 0x5606cd7fa3b0 2024-04-25T10:32:23.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.936+0000 7f9b897fa640 1 -- 172.21.15.43:0/1671354960 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9b900c8050 con 0x5606cd7fa3b0 2024-04-25T10:32:23.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.936+0000 7f9bae9c3740 10 client.15267 fetch_fsmap learned FSMap version 23 2024-04-25T10:32:23.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.936+0000 7f9bae9c3740 1 -- 172.21.15.43:0/1671354960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5606cdbd4e30 con 0x5606cd7fa3b0 2024-04-25T10:32:23.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.936+0000 7f9b897fa640 1 -- 172.21.15.43:0/1671354960 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b900b0380 con 0x5606cd7fa3b0 2024-04-25T10:32:23.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.936+0000 7f9b897fa640 1 -- 172.21.15.43:0/1671354960 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f9b900a6380 con 0x5606cd7fa3b0 2024-04-25T10:32:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.936+0000 7f9bae9c3740 10 client.15267 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:32:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.937+0000 7f9bae9c3740 10 client.15267 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:32:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.937+0000 7f9bae9c3740 20 client.15267 populate_metadata read hostname 'smithi043' 2024-04-25T10:32:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.937+0000 7f9bae9c3740 1 --2- 172.21.15.43:0/1671354960 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5606cd823270 0x5606cdc477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.937+0000 7f9bae9c3740 4 client.15267 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:32:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.937+0000 7f9bae9c3740 1 -- 172.21.15.43:0/1671354960 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "status"}) -- 0x5606cdb311c0 con 0x5606cd823270 2024-04-25T10:32:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.937+0000 7f9ba58a9640 1 --2- 172.21.15.43:0/1671354960 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5606cd823270 0x5606cdc477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.937+0000 7f9ba58a9640 1 --2- 172.21.15.43:0/1671354960 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5606cd823270 0x5606cdc477b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9ecbeb8c40a4652 server_cookie=a413d2279247ab69 in_seq=0 out_seq=0 2024-04-25T10:32:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.937+0000 7f9b897fa640 10 client.15267 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:32:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.937+0000 7f9b897fa640 1 -- 172.21.15.43:0/1671354960 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5606cdb311c0 con 0x5606cd823270 2024-04-25T10:32:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.937+0000 7f9b897fa640 10 client.15267 handle_command_reply: tid=0 2024-04-25T10:32:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.937+0000 7f9bae9c3740 1 client.15267 shutdown 2024-04-25T10:32:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9bae9c3740 1 -- 172.21.15.43:0/1671354960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd7fa3b0 msgr2=0x5606cdbd2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9bae9c3740 1 --2- 172.21.15.43:0/1671354960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd7fa3b0 0x5606cdbd2a00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000e9c0 tx=0x7f9b90010cc0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9bae9c3740 1 -- 172.21.15.43:0/1671354960 shutdown_connections 2024-04-25T10:32:23.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9bae9c3740 1 --2- 172.21.15.43:0/1671354960 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5606cd823270 0x5606cdc477b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9bae9c3740 1 --2- 172.21.15.43:0/1671354960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5606cd802590 0x5606cdbd38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9bae9c3740 1 --2- 172.21.15.43:0/1671354960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606cd7fa3b0 0x5606cdbd2a00 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9bae9c3740 1 --2- 172.21.15.43:0/1671354960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd7e1e40 0x5606cdbd24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9b897fa640 0 client.15267 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:32:23.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9bae9c3740 1 -- 172.21.15.43:0/1671354960 >> 172.21.15.43:0/1671354960 conn(0x5606cd8173c0 msgr2=0x5606cdbce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9bae9c3740 1 -- 172.21.15.43:0/1671354960 shutdown_connections 2024-04-25T10:32:23.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9bae9c3740 1 -- 172.21.15.43:0/1671354960 wait complete. 2024-04-25T10:32:23.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9bae9c3740 20 client.15267 trim_cache size 0 max 16384 2024-04-25T10:32:23.947 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:32:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "94d0035c-02ed-11ef-bc93-c7b262605968", 2024-04-25T10:32:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-25T10:32:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24554, 2024-04-25T10:32:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-25T10:32:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-25T10:32:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-25T10:32:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 109.78403631499999, 2024-04-25T10:32:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 23, 2024-04-25T10:32:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 79, 2024-04-25T10:32:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 77, 2024-04-25T10:32:23.948 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 115.539528069 2024-04-25T10:32:23.948 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:32:23.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9ba7b0d640 1 -- 172.21.15.43:0/528918267 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9b8c0afb70 msgr2=0x7f9b8c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9ba7b0d640 1 --2- 172.21.15.43:0/528918267 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9b8c0afb70 0x7f9b8c0b2080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x5606cdbc5750 tx=0x7f9b9c005f90 comp rx=0 tx=0).stop 2024-04-25T10:32:23.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9ba7b0d640 1 -- 172.21.15.43:0/528918267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ba0104d30 msgr2=0x7f9ba019a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:23.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9ba7b0d640 1 --2- 172.21.15.43:0/528918267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ba0104d30 0x7f9ba019a720 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000e9f0 tx=0x7f9b9000eeb0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9ba7b0d640 1 -- 172.21.15.43:0/528918267 shutdown_connections 2024-04-25T10:32:23.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9ba7b0d640 1 --2- 172.21.15.43:0/528918267 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f9b8c0afb70 0x7f9b8c0b2080 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9ba7b0d640 1 --2- 172.21.15.43:0/528918267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ba0109a30 0x7f9ba01a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9ba7b0d640 1 --2- 172.21.15.43:0/528918267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ba0104d30 0x7f9ba019a720 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9ba7b0d640 1 --2- 172.21.15.43:0/528918267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba0103af0 0x7f9ba019a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:23.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.938+0000 7f9ba7b0d640 1 -- 172.21.15.43:0/528918267 >> 172.21.15.43:0/528918267 conn(0x7f9ba00fece0 msgr2=0x7f9ba0105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:23.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.939+0000 7f9ba7b0d640 1 -- 172.21.15.43:0/528918267 shutdown_connections 2024-04-25T10:32:23.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:23.939+0000 7f9ba7b0d640 1 -- 172.21.15.43:0/528918267 wait complete. 2024-04-25T10:32:23.952 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T10:32:23.966 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T10:32:23.966 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T10:32:23.966 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T10:32:23.967 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T10:32:23.967 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T10:32:23.967 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T10:32:23.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T10:32:24.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.077+0000 7fd31291f640 1 -- 172.21.15.43:0/3890048634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd30c074f90 msgr2=0x7fd30c0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:24.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.077+0000 7fd31291f640 1 --2- 172.21.15.43:0/3890048634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd30c074f90 0x7fd30c0753a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009900 tx=0x7fd2fc02a410 comp rx=0 tx=0).stop 2024-04-25T10:32:24.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.078+0000 7fd31291f640 1 -- 172.21.15.43:0/3890048634 shutdown_connections 2024-04-25T10:32:24.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.078+0000 7fd31291f640 1 --2- 172.21.15.43:0/3890048634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd30c0758e0 0x7fd30c10fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:24.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.078+0000 7fd31291f640 1 --2- 172.21.15.43:0/3890048634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd30c1102b0 0x7fd30c1126d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:24.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.078+0000 7fd31291f640 1 --2- 172.21.15.43:0/3890048634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd30c074f90 0x7fd30c0753a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:24.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.078+0000 7fd31291f640 1 -- 172.21.15.43:0/3890048634 >> 172.21.15.43:0/3890048634 conn(0x7fd30c0fed20 msgr2=0x7fd30c1011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:24.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.078+0000 7fd31291f640 1 -- 172.21.15.43:0/3890048634 shutdown_connections 2024-04-25T10:32:24.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.078+0000 7fd31291f640 1 -- 172.21.15.43:0/3890048634 wait complete. 2024-04-25T10:32:24.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.078+0000 7fd31291f640 1 Processor -- start 2024-04-25T10:32:24.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.078+0000 7fd31291f640 1 -- start start 2024-04-25T10:32:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.079+0000 7fd31291f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd30c074f90 0x7fd30c1a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.079+0000 7fd31291f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd30c0758e0 0x7fd30c1a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.079+0000 7fd31291f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd30c1102b0 0x7fd30c1a7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.079+0000 7fd31291f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c115ce0 con 0x7fd30c074f90 2024-04-25T10:32:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.079+0000 7fd31291f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c115b60 con 0x7fd30c0758e0 2024-04-25T10:32:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.079+0000 7fd31291f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c115e60 con 0x7fd30c1102b0 2024-04-25T10:32:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.079+0000 7fd30bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd30c074f90 0x7fd30c1a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.079+0000 7fd30bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd30c074f90 0x7fd30c1a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40058/0 (socket says 172.21.15.43:40058) 2024-04-25T10:32:24.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.079+0000 7fd30bfff640 1 -- 172.21.15.43:0/340766210 learned_addr learned my addr 172.21.15.43:0/340766210 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:24.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.079+0000 7fd310ebc640 1 --2- 172.21.15.43:0/340766210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd30c1102b0 0x7fd30c1a7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:24.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.079+0000 7fd30bfff640 1 -- 172.21.15.43:0/340766210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd30c1102b0 msgr2=0x7fd30c1a7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:24.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.079+0000 7fd30b7fe640 1 --2- 172.21.15.43:0/340766210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd30c0758e0 0x7fd30c1a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:24.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.079+0000 7fd30bfff640 1 --2- 172.21.15.43:0/340766210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd30c1102b0 0x7fd30c1a7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:24.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.079+0000 7fd30bfff640 1 -- 172.21.15.43:0/340766210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd30c0758e0 msgr2=0x7fd30c1a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:24.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.079+0000 7fd30bfff640 1 --2- 172.21.15.43:0/340766210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd30c0758e0 0x7fd30c1a0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:24.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.079+0000 7fd30bfff640 1 -- 172.21.15.43:0/340766210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd30c1a8380 con 0x7fd30c074f90 2024-04-25T10:32:24.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.079+0000 7fd310ebc640 1 --2- 172.21.15.43:0/340766210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd30c1102b0 0x7fd30c1a7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:32:24.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.079+0000 7fd30bfff640 1 --2- 172.21.15.43:0/340766210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd30c074f90 0x7fd30c1a0840 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc031440 tx=0x7fd2fc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:24.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.079+0000 7fd3097fa640 1 -- 172.21.15.43:0/340766210 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc03f070 con 0x7fd30c074f90 2024-04-25T10:32:24.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.080+0000 7fd3097fa640 1 -- 172.21.15.43:0/340766210 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2fc031c50 con 0x7fd30c074f90 2024-04-25T10:32:24.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.080+0000 7fd3097fa640 1 -- 172.21.15.43:0/340766210 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc035030 con 0x7fd30c074f90 2024-04-25T10:32:24.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.080+0000 7fd31291f640 1 -- 172.21.15.43:0/340766210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd30c1a8610 con 0x7fd30c074f90 2024-04-25T10:32:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.080+0000 7fd31291f640 1 -- 172.21.15.43:0/340766210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd30c1a8b50 con 0x7fd30c074f90 2024-04-25T10:32:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.081+0000 7fd31291f640 1 -- 172.21.15.43:0/340766210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd30c105f40 con 0x7fd30c074f90 2024-04-25T10:32:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.082+0000 7fd3097fa640 1 -- 172.21.15.43:0/340766210 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd2fc042070 con 0x7fd30c074f90 2024-04-25T10:32:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.083+0000 7fd3097fa640 1 --2- 172.21.15.43:0/340766210 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd2f40afb60 0x7fd2f40b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.083+0000 7fd30b7fe640 1 --2- 172.21.15.43:0/340766210 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd2f40afb60 0x7fd2f40b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.083+0000 7fd3097fa640 1 -- 172.21.15.43:0/340766210 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fd2fc0ffc90 con 0x7fd30c074f90 2024-04-25T10:32:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.083+0000 7fd30b7fe640 1 --2- 172.21.15.43:0/340766210 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd2f40afb60 0x7fd2f40b2070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd30c1a18e0 tx=0x7fd300005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.086+0000 7fd3097fa640 1 -- 172.21.15.43:0/340766210 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd2fc0c8820 con 0x7fd30c074f90 2024-04-25T10:32:24.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:23 smithi043 ceph-mon[29478]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:32:24.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:23 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2158581795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:32:24.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:23 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/956232509' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:24.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:23 smithi077 ceph-mon[32772]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:32:24.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:23 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2158581795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:32:24.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:23 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/956232509' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:24.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:23 smithi164 ceph-mon[32713]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:32:24.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:23 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2158581795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:32:24.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:23 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/956232509' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:24.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.326+0000 7fd31291f640 1 -- 172.21.15.43:0/340766210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fd30c0637a0 con 0x7fd30c074f90 2024-04-25T10:32:24.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.941+0000 7fd3097fa640 1 -- 172.21.15.43:0/340766210 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fd2fc0c8120 con 0x7fd30c074f90 2024-04-25T10:32:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.944+0000 7fd31291f640 1 -- 172.21.15.43:0/340766210 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd2f40afb60 msgr2=0x7fd2f40b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.944+0000 7fd31291f640 1 --2- 172.21.15.43:0/340766210 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd2f40afb60 0x7fd2f40b2070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd30c1a18e0 tx=0x7fd300005f90 comp rx=0 tx=0).stop 2024-04-25T10:32:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.945+0000 7fd31291f640 1 -- 172.21.15.43:0/340766210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd30c074f90 msgr2=0x7fd30c1a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.945+0000 7fd31291f640 1 --2- 172.21.15.43:0/340766210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd30c074f90 0x7fd30c1a0840 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc031440 tx=0x7fd2fc003c10 comp rx=0 tx=0).stop 2024-04-25T10:32:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.945+0000 7fd31291f640 1 -- 172.21.15.43:0/340766210 shutdown_connections 2024-04-25T10:32:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.945+0000 7fd31291f640 1 --2- 172.21.15.43:0/340766210 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fd2f40afb60 0x7fd2f40b2070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.945+0000 7fd31291f640 1 --2- 172.21.15.43:0/340766210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd30c0758e0 0x7fd30c1a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.945+0000 7fd31291f640 1 --2- 172.21.15.43:0/340766210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd30c1102b0 0x7fd30c1a7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.945+0000 7fd31291f640 1 --2- 172.21.15.43:0/340766210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd30c074f90 0x7fd30c1a0840 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.945+0000 7fd31291f640 1 -- 172.21.15.43:0/340766210 >> 172.21.15.43:0/340766210 conn(0x7fd30c0fed20 msgr2=0x7fd30c103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.945+0000 7fd31291f640 1 -- 172.21.15.43:0/340766210 shutdown_connections 2024-04-25T10:32:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:24.945+0000 7fd31291f640 1 -- 172.21.15.43:0/340766210 wait complete. 2024-04-25T10:32:24.968 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T10:32:24.992 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T10:32:24.992 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T10:32:24.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:24.992 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-25T10:32:25.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:25.010 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-04-25T10:32:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:Cannot find device "ceph-brx" 2024-04-25T10:32:25.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:32:25.081 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-25T10:32:25.081 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:32:25.081 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:32:25.081 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T10:32:25.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:24 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/340766210' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T10:32:25.291 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:25 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: ignoring --setuser ceph since I am not root 2024-04-25T10:32:25.291 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:25 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: ignoring --setgroup ceph since I am not root 2024-04-25T10:32:25.291 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:25 smithi077 ceph-mgr[34155]: -- 172.21.15.77:0/1646551785 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bd110185a0 con 0x55bd15da2000 2024-04-25T10:32:25.291 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:25 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:25.146+0000 7ff4d55d7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:32:25.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:24 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/340766210' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T10:32:25.292 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:25 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: ignoring --setuser ceph since I am not root 2024-04-25T10:32:25.292 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:25 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: ignoring --setgroup ceph since I am not root 2024-04-25T10:32:25.292 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:25 smithi164 ceph-mgr[34581]: -- 172.21.15.164:0/747404095 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c64d29c5a0 con 0x55c652026000 2024-04-25T10:32:25.292 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:25 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:25.145+0000 7f7aa6f63200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:32:25.297 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:25 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: ignoring --setuser ceph since I am not root 2024-04-25T10:32:25.297 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:25 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: ignoring --setgroup ceph since I am not root 2024-04-25T10:32:25.298 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:25 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:25.174+0000 7fa2298e2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:32:25.298 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:24 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/340766210' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T10:32:25.383 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:32:25.610 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:25 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:25.385+0000 7fa2298e2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:32:25.611 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:25 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:25.504+0000 7fa2298e2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:32:25.745 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:25 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:25.331+0000 7ff4d55d7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:32:25.745 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:25 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:25.432+0000 7ff4d55d7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:32:25.745 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:25 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:25.743+0000 7ff4d55d7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:32:25.756 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:25 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:25.334+0000 7f7aa6f63200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:32:25.756 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:25 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:25.438+0000 7f7aa6f63200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:32:25.944 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:25 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:25.853+0000 7fa2298e2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:32:26.040 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:25 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:25.828+0000 7ff4d55d7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:32:26.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:25 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/340766210' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T10:32:26.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:25 smithi077 ceph-mon[32772]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-25T10:32:26.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:25 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/340766210' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T10:32:26.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:25 smithi164 ceph-mon[32713]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-25T10:32:26.041 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:25 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:25.753+0000 7f7aa6f63200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:32:26.041 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:25 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:25.840+0000 7f7aa6f63200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:32:26.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:25 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/340766210' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T10:32:26.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:25 smithi043 ceph-mon[29478]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-25T10:32:26.312 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:25 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:25.947+0000 7fa2298e2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:32:27.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.160+0000 7fbe705ef700 1 -- 172.21.15.43:0/4233548513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe68108610 msgr2=0x7fbe6810ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:27.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.160+0000 7fbe705ef700 1 --2- 172.21.15.43:0/4233548513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe68108610 0x7fbe6810ff70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000b0d0 tx=0x7fbe6000b3e0 comp rx=0 tx=0).stop 2024-04-25T10:32:27.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.160+0000 7fbe705ef700 1 -- 172.21.15.43:0/4233548513 shutdown_connections 2024-04-25T10:32:27.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.160+0000 7fbe705ef700 1 --2- 172.21.15.43:0/4233548513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe68108610 0x7fbe6810ff70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:27.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.160+0000 7fbe705ef700 1 --2- 172.21.15.43:0/4233548513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe68103390 0x7fbe681037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:27.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.160+0000 7fbe705ef700 1 --2- 172.21.15.43:0/4233548513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe68107cc0 0x7fbe681080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:27.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.160+0000 7fbe705ef700 1 -- 172.21.15.43:0/4233548513 >> 172.21.15.43:0/4233548513 conn(0x7fbe680fec90 msgr2=0x7fbe681010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:27.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.160+0000 7fbe705ef700 1 -- 172.21.15.43:0/4233548513 shutdown_connections 2024-04-25T10:32:27.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.160+0000 7fbe705ef700 1 -- 172.21.15.43:0/4233548513 wait complete. 2024-04-25T10:32:27.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.161+0000 7fbe705ef700 1 Processor -- start 2024-04-25T10:32:27.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.161+0000 7fbe705ef700 1 -- start start 2024-04-25T10:32:27.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.161+0000 7fbe705ef700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe68103390 0x7fbe681a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:27.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.161+0000 7fbe705ef700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe68107cc0 0x7fbe681a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:27.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.161+0000 7fbe705ef700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe68108610 0x7fbe6819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:27.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.162+0000 7fbe705ef700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbe68113240 con 0x7fbe68103390 2024-04-25T10:32:27.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.162+0000 7fbe705ef700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbe681130c0 con 0x7fbe68108610 2024-04-25T10:32:27.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.162+0000 7fbe705ef700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbe681133c0 con 0x7fbe68107cc0 2024-04-25T10:32:27.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.162+0000 7fbe6eb8c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe68108610 0x7fbe6819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:27.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.162+0000 7fbe6db8a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe68107cc0 0x7fbe681a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:27.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.162+0000 7fbe6eb8c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe68108610 0x7fbe6819aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:51372/0 (socket says 172.21.15.43:51372) 2024-04-25T10:32:27.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.162+0000 7fbe6e38b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe68103390 0x7fbe681a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:27.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.162+0000 7fbe6eb8c700 1 -- 172.21.15.43:0/2467085564 learned_addr learned my addr 172.21.15.43:0/2467085564 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:27.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.162+0000 7fbe6db8a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe68107cc0 0x7fbe681a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:50858/0 (socket says 172.21.15.43:50858) 2024-04-25T10:32:27.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.162+0000 7fbe6eb8c700 1 -- 172.21.15.43:0/2467085564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe68107cc0 msgr2=0x7fbe681a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:27.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.162+0000 7fbe6eb8c700 1 --2- 172.21.15.43:0/2467085564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe68107cc0 0x7fbe681a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:27.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.162+0000 7fbe6eb8c700 1 -- 172.21.15.43:0/2467085564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe68103390 msgr2=0x7fbe681a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:27.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.162+0000 7fbe6eb8c700 1 --2- 172.21.15.43:0/2467085564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe68103390 0x7fbe681a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:27.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.162+0000 7fbe6eb8c700 1 -- 172.21.15.43:0/2467085564 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6819b120 con 0x7fbe68108610 2024-04-25T10:32:27.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.162+0000 7fbe6e38b700 1 --2- 172.21.15.43:0/2467085564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe68103390 0x7fbe681a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:27.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.162+0000 7fbe6eb8c700 1 --2- 172.21.15.43:0/2467085564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe68108610 0x7fbe6819aa70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009850 tx=0x7fbe60007a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:27.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.162+0000 7fbe5b7fe700 1 -- 172.21.15.43:0/2467085564 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6001f070 con 0x7fbe68108610 2024-04-25T10:32:27.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.163+0000 7fbe5b7fe700 1 -- 172.21.15.43:0/2467085564 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe60004c50 con 0x7fbe68108610 2024-04-25T10:32:27.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.163+0000 7fbe705ef700 1 -- 172.21.15.43:0/2467085564 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6819b3b0 con 0x7fbe68108610 2024-04-25T10:32:27.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.163+0000 7fbe5b7fe700 1 -- 172.21.15.43:0/2467085564 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6001b690 con 0x7fbe68108610 2024-04-25T10:32:27.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.163+0000 7fbe705ef700 1 -- 172.21.15.43:0/2467085564 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe681a76e0 con 0x7fbe68108610 2024-04-25T10:32:27.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.164+0000 7fbe705ef700 1 -- 172.21.15.43:0/2467085564 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe68104840 con 0x7fbe68108610 2024-04-25T10:32:27.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.165+0000 7fbe5b7fe700 1 -- 172.21.15.43:0/2467085564 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) ==== 147537+0+0 (secure 0 0 0) 0x7fbe60022070 con 0x7fbe68108610 2024-04-25T10:32:27.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.166+0000 7fbe5b7fe700 1 --2- 172.21.15.43:0/2467085564 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fbe540afc50 0x7fbe540b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:27.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.166+0000 7fbe6e38b700 1 -- 172.21.15.43:0/2467085564 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fbe540afc50 msgr2=0x7fbe540b2160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1953858940 2024-04-25T10:32:27.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.166+0000 7fbe5b7fe700 1 -- 172.21.15.43:0/2467085564 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7fbe6000f020 con 0x7fbe68108610 2024-04-25T10:32:27.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.166+0000 7fbe6e38b700 1 --2- 172.21.15.43:0/2467085564 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fbe540afc50 0x7fbe540b2160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:27.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.171+0000 7fbe5b7fe700 1 -- 172.21.15.43:0/2467085564 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe60029020 con 0x7fbe68108610 2024-04-25T10:32:27.290 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:27 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:27.006+0000 7ff4d55d7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:32:27.291 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:27 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:27.177+0000 7ff4d55d7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:32:27.291 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:27 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:27.034+0000 7f7aa6f63200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:32:27.291 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:27 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:27.206+0000 7f7aa6f63200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:32:27.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.367+0000 7fbe6e38b700 1 -- 172.21.15.43:0/2467085564 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fbe540afc50 msgr2=0x7fbe540b2160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1953858940 2024-04-25T10:32:27.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.367+0000 7fbe6e38b700 1 --2- 172.21.15.43:0/2467085564 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fbe540afc50 0x7fbe540b2160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:32:27.429 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:27 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:27.242+0000 7fa2298e2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:32:27.430 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:27 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:27.427+0000 7fa2298e2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:32:27.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.429+0000 7fbe705ef700 1 -- 172.21.15.43:0/2467085564 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbe68051190 con 0x7fbe68108610 2024-04-25T10:32:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.430+0000 7fbe5b7fe700 1 -- 172.21.15.43:0/2467085564 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7fbe600a8940 con 0x7fbe68108610 2024-04-25T10:32:27.432 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:32:27.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.432+0000 7fbe705ef700 1 -- 172.21.15.43:0/2467085564 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fbe540afc50 msgr2=0x7fbe540b2160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:32:27.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.432+0000 7fbe705ef700 1 --2- 172.21.15.43:0/2467085564 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fbe540afc50 0x7fbe540b2160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:27.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.432+0000 7fbe705ef700 1 -- 172.21.15.43:0/2467085564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe68108610 msgr2=0x7fbe6819aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:27.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.432+0000 7fbe705ef700 1 --2- 172.21.15.43:0/2467085564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe68108610 0x7fbe6819aa70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009850 tx=0x7fbe60007a20 comp rx=0 tx=0).stop 2024-04-25T10:32:27.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.432+0000 7fbe705ef700 1 -- 172.21.15.43:0/2467085564 shutdown_connections 2024-04-25T10:32:27.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.432+0000 7fbe705ef700 1 --2- 172.21.15.43:0/2467085564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe68108610 0x7fbe6819aa70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:27.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.432+0000 7fbe705ef700 1 --2- 172.21.15.43:0/2467085564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe68103390 0x7fbe681a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:27.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.432+0000 7fbe705ef700 1 --2- 172.21.15.43:0/2467085564 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7fbe540afc50 0x7fbe540b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:27.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.432+0000 7fbe705ef700 1 --2- 172.21.15.43:0/2467085564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe68107cc0 0x7fbe681a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:27.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.432+0000 7fbe705ef700 1 -- 172.21.15.43:0/2467085564 >> 172.21.15.43:0/2467085564 conn(0x7fbe680fec90 msgr2=0x7fbe6806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:27.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.432+0000 7fbe705ef700 1 -- 172.21.15.43:0/2467085564 shutdown_connections 2024-04-25T10:32:27.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:27.432+0000 7fbe705ef700 1 -- 172.21.15.43:0/2467085564 wait complete. 2024-04-25T10:32:27.642 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:27 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2467085564' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T10:32:27.643 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:27 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:27.349+0000 7ff4d55d7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:32:27.675 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:27 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2467085564' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T10:32:27.675 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:27 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:27.380+0000 7f7aa6f63200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:32:27.731 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:27 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2467085564' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T10:32:27.731 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:27 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:27.613+0000 7fa2298e2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:32:27.939 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:27 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:27.640+0000 7ff4d55d7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:32:27.939 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:27 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:27.733+0000 7ff4d55d7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:32:27.939 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:27 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:27.937+0000 7ff4d55d7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:32:27.974 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:27 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:27.672+0000 7f7aa6f63200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:32:27.975 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:27 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:27.766+0000 7f7aa6f63200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:32:28.029 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:27 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:27.927+0000 7fa2298e2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:32:28.164 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:32:28.165 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:32:28.165 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-04-25T10:32:28.165 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:32:28.165 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:32:28.165 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:32:28.165 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:32:28.165 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:32:28.165 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:32:28.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:28.166 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-04-25T10:32:28.182 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:32:28.182 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:32:28.182 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:32:28.182 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:32:28.182 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-25T10:32:28.182 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:32:28.182 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:32:28.182 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-04-25T10:32:28.183 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-04-25T10:32:28.183 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:32:28.183 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-04-25T10:32:28.183 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-04-25T10:32:28.183 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:32:28.183 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:32:28.183 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-04-25T10:32:28.183 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:32:28.183 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42009sec preferred_lft 42009sec 2024-04-25T10:32:28.183 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-25T10:32:28.183 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:32:28.183 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:32:28.183 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:32:28.183 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-04-25T10:32:28.184 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:32:28.184 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:32:28.184 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T10:32:28.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:32:28.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr flush dev ceph-brx 2024-04-25T10:32:28.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set ceph-brx up 2024-04-25T10:32:28.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:32:28.185 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T10:32:28.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:32:28.281 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:28 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:28.026+0000 7fa2298e2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:32:28.281 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:28 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:28.243+0000 7fa2298e2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:32:28.290 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:28 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:28.133+0000 7ff4d55d7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:32:28.291 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:27 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:27.971+0000 7f7aa6f63200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:32:28.291 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:28 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:28.167+0000 7f7aa6f63200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:32:28.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:32:28.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:28.375 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:32:28.402 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-04-25T10:32:28.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:28.404 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-04-25T10:32:28.469 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-04-25T10:32:28.470 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:32:28.470 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:32:28.470 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:32:28.470 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:32:28.470 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:32:28.470 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T10:32:28.470 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:32:28.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:32:28.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:32:28.471 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T10:32:28.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:32:28.568 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:28 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:28.464+0000 7fa2298e2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:32:28.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:32:28.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:28.748 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-25T10:32:28.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:28.767 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2024-04-25T10:32:28.823 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:32:28.823 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T10:32:28.823 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:32:28.823 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T10:32:28.823 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T10:32:28.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:32:28.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:32:28.943 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T10:32:28.944 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:32:28.944 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T10:32:28.944 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T10:32:28.944 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:32:28.944 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T10:32:28.944 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T10:32:28.944 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T10:32:28.944 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T10:32:28.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:32:29.040 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:28 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:28.730+0000 7ff4d55d7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:32:29.040 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:28 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:28.770+0000 7f7aa6f63200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:32:29.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:32:29.077 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:32:29.077 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T10:32:29.077 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-04-25T10:32:29.077 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:32:29.077 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T10:32:29.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:32:29.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:32:29.157 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:32:29.158 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:32:29.174 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:32:29.175 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:32:29.175 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:32:29.230 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T10:32:29.231 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T10:32:29.231 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/71d48da6-eb39-4602-a283-dfdb3fceec5f 2024-04-25T10:32:29.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:29.231 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/71d48da6-eb39-4602-a283-dfdb3fceec5f /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T10:32:29.292 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:29 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:29.125+0000 7fa2298e2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:32:29.346 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T10:32:29.346 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-25T10:32:29.456 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T10:32:29.456 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-25T10:32:29.456 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.77:3300,172.21.15.164:3300" 2024-04-25T10:32:29.456 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with new device syntax: 0@94d0035c-02ed-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/71d48da6-eb39-4602-a283-dfdb3fceec5f 2024-04-25T10:32:29.456 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.43:3300/172.21.15.77:3300/172.21.15.164:3300" will pass to kernel 2024-04-25T10:32:29.457 INFO:teuthology.orchestra.run.smithi043.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:32:29.457 INFO:teuthology.orchestra.run.smithi043.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:32:29.457 INFO:teuthology.orchestra.run.smithi043.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:32:29.457 INFO:teuthology.orchestra.run.smithi043.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:32:29.457 INFO:teuthology.orchestra.run.smithi043.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:32:29.459 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:32:29.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:29.459 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:32:29.460 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:29 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:29.154+0000 7ff4d55d7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:32:29.460 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:29 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:29.251+0000 7ff4d55d7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:32:29.460 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:29 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:29.457+0000 7ff4d55d7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:32:29.498 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:29 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:29.196+0000 7f7aa6f63200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:32:29.498 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:29 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:29.293+0000 7f7aa6f63200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:32:29.498 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:29 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:29.495+0000 7f7aa6f63200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:32:29.526 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:32:29.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T10:32:29.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:32:29.626 INFO:teuthology.orchestra.run.smithi043.stdout:client15300 2024-04-25T10:32:29.626 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T10:32:29.626 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:32:29.643 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:32:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T10:32:29.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:32:29.741 INFO:teuthology.orchestra.run.smithi043.stdout:client15300 2024-04-25T10:32:29.742 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:32:29.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T10:32:29.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:32:29.799 INFO:teuthology.orchestra.run.smithi043.stdout:client15300 2024-04-25T10:32:29.800 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/94d0035c-02ed-11ef-bc93-c7b262605968.client15300/status') 2024-04-25T10:32:29.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:32:29.857 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:29 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:29.598+0000 7fa2298e2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:32:29.858 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:29 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:29.707+0000 7fa2298e2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:32:29.860 INFO:teuthology.orchestra.run.smithi043.stdout:instance: client.15300 (3)192.168.0.1:0/1998962038 2024-04-25T10:32:29.860 INFO:teuthology.orchestra.run.smithi043.stdout:blocklisted: false 2024-04-25T10:32:29.860 INFO:teuthology.orchestra.run.smithi043.stderr:0+1 records in 2024-04-25T10:32:29.860 INFO:teuthology.orchestra.run.smithi043.stderr:0+1 records out 2024-04-25T10:32:29.860 INFO:teuthology.orchestra.run.smithi043.stderr:70 bytes copied, 2.7383e-05 s, 2.6 MB/s 2024-04-25T10:32:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:32:29.868 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T10:32:29.880 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T10:32:29.881 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T10:32:29.881 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-25T10:32:29.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.993+0000 7f7b57251640 1 -- 172.21.15.43:0/1706313770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b50104d80 msgr2=0x7f7b50105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:29.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.993+0000 7f7b57251640 1 --2- 172.21.15.43:0/1706313770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b50104d80 0x7f7b50105190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009900 tx=0x7f7b3c02a410 comp rx=0 tx=0).stop 2024-04-25T10:32:29.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.993+0000 7f7b57251640 1 -- 172.21.15.43:0/1706313770 shutdown_connections 2024-04-25T10:32:29.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.993+0000 7f7b57251640 1 --2- 172.21.15.43:0/1706313770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b50109a80 0x7f7b5010bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:29.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.993+0000 7f7b57251640 1 --2- 172.21.15.43:0/1706313770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b50104d80 0x7f7b50105190 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:29.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.993+0000 7f7b57251640 1 --2- 172.21.15.43:0/1706313770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b50103b40 0x7f7b50104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:29.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.993+0000 7f7b57251640 1 -- 172.21.15.43:0/1706313770 >> 172.21.15.43:0/1706313770 conn(0x7f7b500fed30 msgr2=0x7f7b501011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:29.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.993+0000 7f7b57251640 1 -- 172.21.15.43:0/1706313770 shutdown_connections 2024-04-25T10:32:29.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.993+0000 7f7b57251640 1 -- 172.21.15.43:0/1706313770 wait complete. 2024-04-25T10:32:29.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.994+0000 7f7b57251640 1 Processor -- start 2024-04-25T10:32:29.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.994+0000 7f7b57251640 1 -- start start 2024-04-25T10:32:29.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.994+0000 7f7b57251640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b50103b40 0x7f7b5019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:29.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.994+0000 7f7b57251640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b50104d80 0x7f7b5019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:29.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.994+0000 7f7b57251640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b50109a80 0x7f7b501a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:29.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.994+0000 7f7b57251640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5010f670 con 0x7f7b50104d80 2024-04-25T10:32:30.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.994+0000 7f7b57251640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5010f4f0 con 0x7f7b50109a80 2024-04-25T10:32:30.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.994+0000 7f7b57251640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5010f7f0 con 0x7f7b50103b40 2024-04-25T10:32:30.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.994+0000 7f7b4ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b50104d80 0x7f7b5019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:30.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.994+0000 7f7b54fed640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b50103b40 0x7f7b5019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:30.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.994+0000 7f7b4ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b50104d80 0x7f7b5019a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48768/0 (socket says 172.21.15.43:48768) 2024-04-25T10:32:30.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.994+0000 7f7b4ffff640 1 -- 172.21.15.43:0/362656845 learned_addr learned my addr 172.21.15.43:0/362656845 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:30.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.995+0000 7f7b557ee640 1 --2- 172.21.15.43:0/362656845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b50109a80 0x7f7b501a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:30.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.995+0000 7f7b4ffff640 1 -- 172.21.15.43:0/362656845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b50103b40 msgr2=0x7f7b5019a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:30.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.995+0000 7f7b4ffff640 1 --2- 172.21.15.43:0/362656845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b50103b40 0x7f7b5019a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:30.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.995+0000 7f7b4ffff640 1 -- 172.21.15.43:0/362656845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b50109a80 msgr2=0x7f7b501a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:30.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.995+0000 7f7b4ffff640 1 --2- 172.21.15.43:0/362656845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b50109a80 0x7f7b501a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:30.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.995+0000 7f7b4ffff640 1 -- 172.21.15.43:0/362656845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b501a1d50 con 0x7f7b50104d80 2024-04-25T10:32:30.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.995+0000 7f7b54fed640 1 --2- 172.21.15.43:0/362656845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b50103b40 0x7f7b5019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:30.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.995+0000 7f7b4ffff640 1 --2- 172.21.15.43:0/362656845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b50104d80 0x7f7b5019a750 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c006010 tx=0x7f7b3c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:30.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.995+0000 7f7b4dffb640 1 -- 172.21.15.43:0/362656845 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c03f030 con 0x7f7b50104d80 2024-04-25T10:32:30.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.995+0000 7f7b57251640 1 -- 172.21.15.43:0/362656845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b501a1fe0 con 0x7f7b50104d80 2024-04-25T10:32:30.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.995+0000 7f7b4dffb640 1 -- 172.21.15.43:0/362656845 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b3c031910 con 0x7f7b50104d80 2024-04-25T10:32:30.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.995+0000 7f7b4dffb640 1 -- 172.21.15.43:0/362656845 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c035070 con 0x7f7b50104d80 2024-04-25T10:32:30.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.995+0000 7f7b57251640 1 -- 172.21.15.43:0/362656845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b501a2440 con 0x7f7b50104d80 2024-04-25T10:32:30.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.996+0000 7f7b57251640 1 -- 172.21.15.43:0/362656845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b50108890 con 0x7f7b50104d80 2024-04-25T10:32:30.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.998+0000 7f7b4dffb640 1 -- 172.21.15.43:0/362656845 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 19) ==== 147537+0+0 (secure 0 0 0) 0x7f7b3c042030 con 0x7f7b50104d80 2024-04-25T10:32:30.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.999+0000 7f7b4dffb640 1 --2- 172.21.15.43:0/362656845 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f7b400afb30 0x7f7b400b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:30.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:29.999+0000 7f7b54fed640 1 -- 172.21.15.43:0/362656845 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f7b400afb30 msgr2=0x7f7b400b2040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1953858940 2024-04-25T10:32:30.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.000+0000 7f7b4dffb640 1 -- 172.21.15.43:0/362656845 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7f7b3c0ff7d0 con 0x7f7b50104d80 2024-04-25T10:32:30.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.000+0000 7f7b54fed640 1 --2- 172.21.15.43:0/362656845 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f7b400afb30 0x7f7b400b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:30.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.003+0000 7f7b4dffb640 1 -- 172.21.15.43:0/362656845 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b3c0c7d70 con 0x7f7b50104d80 2024-04-25T10:32:30.110 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:29 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:29.933+0000 7fa2298e2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:32:30.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.201+0000 7f7b54fed640 1 -- 172.21.15.43:0/362656845 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f7b400afb30 msgr2=0x7f7b400b2040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1953858940 2024-04-25T10:32:30.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.201+0000 7f7b54fed640 1 --2- 172.21.15.43:0/362656845 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f7b400afb30 0x7f7b400b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:32:30.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.233+0000 7f7b57251640 1 -- 172.21.15.43:0/362656845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f7b5002db50 con 0x7f7b50104d80 2024-04-25T10:32:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.285+0000 7f7b4dffb640 1 -- 172.21.15.43:0/362656845 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v24) ==== 118+0+0 (secure 0 0 0) 0x7f7b3c0cc070 con 0x7f7b50104d80 2024-04-25T10:32:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.287+0000 7f7b57251640 1 -- 172.21.15.43:0/362656845 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f7b400afb30 msgr2=0x7f7b400b2040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:32:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.287+0000 7f7b57251640 1 --2- 172.21.15.43:0/362656845 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f7b400afb30 0x7f7b400b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.287+0000 7f7b57251640 1 -- 172.21.15.43:0/362656845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b50104d80 msgr2=0x7f7b5019a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.287+0000 7f7b57251640 1 --2- 172.21.15.43:0/362656845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b50104d80 0x7f7b5019a750 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c006010 tx=0x7f7b3c0042f0 comp rx=0 tx=0).stop 2024-04-25T10:32:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.287+0000 7f7b57251640 1 -- 172.21.15.43:0/362656845 shutdown_connections 2024-04-25T10:32:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.288+0000 7f7b57251640 1 --2- 172.21.15.43:0/362656845 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7f7b400afb30 0x7f7b400b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.288+0000 7f7b57251640 1 --2- 172.21.15.43:0/362656845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b50109a80 0x7f7b501a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.288+0000 7f7b57251640 1 --2- 172.21.15.43:0/362656845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b50104d80 0x7f7b5019a750 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.288+0000 7f7b57251640 1 --2- 172.21.15.43:0/362656845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b50103b40 0x7f7b5019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.288+0000 7f7b57251640 1 -- 172.21.15.43:0/362656845 >> 172.21.15.43:0/362656845 conn(0x7f7b500fed30 msgr2=0x7f7b50105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.288+0000 7f7b57251640 1 -- 172.21.15.43:0/362656845 shutdown_connections 2024-04-25T10:32:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.288+0000 7f7b57251640 1 -- 172.21.15.43:0/362656845 wait complete. 2024-04-25T10:32:30.291 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:30 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:30.031+0000 7f7aa6f63200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:32:30.291 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:30 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:30.125+0000 7f7aa6f63200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:32:30.291 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:29 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:29.981+0000 7ff4d55d7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:32:30.291 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:30 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:30.076+0000 7ff4d55d7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:32:30.301 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-25T10:32:30.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.414+0000 7efd5617d640 1 -- 172.21.15.43:0/1297402499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd50103b50 msgr2=0x7efd50104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:30.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.414+0000 7efd5617d640 1 --2- 172.21.15.43:0/1297402499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd50103b50 0x7efd50104850 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efd40009900 tx=0x7efd4002a410 comp rx=0 tx=0).stop 2024-04-25T10:32:30.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.414+0000 7efd5617d640 1 -- 172.21.15.43:0/1297402499 shutdown_connections 2024-04-25T10:32:30.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.414+0000 7efd5617d640 1 --2- 172.21.15.43:0/1297402499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd50109a90 0x7efd5010beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:30.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.414+0000 7efd5617d640 1 --2- 172.21.15.43:0/1297402499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd50104d90 0x7efd501051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:30.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.414+0000 7efd5617d640 1 --2- 172.21.15.43:0/1297402499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd50103b50 0x7efd50104850 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:30.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.414+0000 7efd5617d640 1 -- 172.21.15.43:0/1297402499 >> 172.21.15.43:0/1297402499 conn(0x7efd500fed20 msgr2=0x7efd501011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:30.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.414+0000 7efd5617d640 1 -- 172.21.15.43:0/1297402499 shutdown_connections 2024-04-25T10:32:30.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.414+0000 7efd5617d640 1 -- 172.21.15.43:0/1297402499 wait complete. 2024-04-25T10:32:30.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.415+0000 7efd5617d640 1 Processor -- start 2024-04-25T10:32:30.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.415+0000 7efd5617d640 1 -- start start 2024-04-25T10:32:30.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.415+0000 7efd5617d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd50103b50 0x7efd5019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:30.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.415+0000 7efd5617d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd50104d90 0x7efd5019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:30.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.415+0000 7efd5617d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd50109a90 0x7efd501a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:30.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.415+0000 7efd5617d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efd5010f650 con 0x7efd50109a90 2024-04-25T10:32:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.415+0000 7efd5617d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efd5010f4d0 con 0x7efd50103b50 2024-04-25T10:32:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.415+0000 7efd5617d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efd5010f7d0 con 0x7efd50104d90 2024-04-25T10:32:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.415+0000 7efd4ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd50109a90 0x7efd501a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.415+0000 7efd4ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd50109a90 0x7efd501a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48782/0 (socket says 172.21.15.43:48782) 2024-04-25T10:32:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.415+0000 7efd4f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd50103b50 0x7efd5019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.415+0000 7efd4ffff640 1 -- 172.21.15.43:0/3688266776 learned_addr learned my addr 172.21.15.43:0/3688266776 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.415+0000 7efd4effd640 1 --2- 172.21.15.43:0/3688266776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd50104d90 0x7efd5019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.416+0000 7efd4f7fe640 1 -- 172.21.15.43:0/3688266776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd50104d90 msgr2=0x7efd5019a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.416+0000 7efd4f7fe640 1 --2- 172.21.15.43:0/3688266776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd50104d90 0x7efd5019a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.416+0000 7efd4f7fe640 1 -- 172.21.15.43:0/3688266776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd50109a90 msgr2=0x7efd501a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.416+0000 7efd4f7fe640 1 --2- 172.21.15.43:0/3688266776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd50109a90 0x7efd501a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.416+0000 7efd4f7fe640 1 -- 172.21.15.43:0/3688266776 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd501a1d90 con 0x7efd50103b50 2024-04-25T10:32:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.416+0000 7efd4effd640 1 --2- 172.21.15.43:0/3688266776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd50104d90 0x7efd5019a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.416+0000 7efd4f7fe640 1 --2- 172.21.15.43:0/3688266776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd50103b50 0x7efd5019a2c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efd4002a930 tx=0x7efd40004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:30.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.416+0000 7efd4cff9640 1 -- 172.21.15.43:0/3688266776 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4003f070 con 0x7efd50103b50 2024-04-25T10:32:30.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.416+0000 7efd4cff9640 1 -- 172.21.15.43:0/3688266776 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd40031910 con 0x7efd50103b50 2024-04-25T10:32:30.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.416+0000 7efd5617d640 1 -- 172.21.15.43:0/3688266776 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd501a2020 con 0x7efd50103b50 2024-04-25T10:32:30.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.416+0000 7efd4cff9640 1 -- 172.21.15.43:0/3688266776 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40035070 con 0x7efd50103b50 2024-04-25T10:32:30.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.416+0000 7efd5617d640 1 -- 172.21.15.43:0/3688266776 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd501a24e0 con 0x7efd50103b50 2024-04-25T10:32:30.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.417+0000 7efd5617d640 1 -- 172.21.15.43:0/3688266776 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd501088a0 con 0x7efd50103b50 2024-04-25T10:32:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.419+0000 7efd4cff9640 1 -- 172.21.15.43:0/3688266776 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) ==== 147537+0+0 (secure 0 0 0) 0x7efd4003bbb0 con 0x7efd50103b50 2024-04-25T10:32:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.419+0000 7efd4cff9640 1 --2- 172.21.15.43:0/3688266776 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7efd380afb30 0x7efd380b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.419+0000 7efd4effd640 1 -- 172.21.15.43:0/3688266776 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7efd380afb30 msgr2=0x7efd380b2040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1953858940 2024-04-25T10:32:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.420+0000 7efd4cff9640 1 -- 172.21.15.43:0/3688266776 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8211+0+0 (secure 0 0 0) 0x7efd400ff4b0 con 0x7efd50103b50 2024-04-25T10:32:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.420+0000 7efd4effd640 1 --2- 172.21.15.43:0/3688266776 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7efd380afb30 0x7efd380b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:30.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.423+0000 7efd4cff9640 1 -- 172.21.15.43:0/3688266776 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd400c7a50 con 0x7efd50103b50 2024-04-25T10:32:30.555 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:30 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/362656845' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T10:32:30.556 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:30 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:30.323+0000 7ff4d55d7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:32:30.556 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:30 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:30.458+0000 7ff4d55d7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:32:30.556 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:30 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:30.553+0000 7ff4d55d7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:32:30.609 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:30 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/362656845' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T10:32:30.609 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:30 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:30.374+0000 7f7aa6f63200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:32:30.609 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:30 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:30.510+0000 7f7aa6f63200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:32:30.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:30 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/362656845' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T10:32:30.610 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:30 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:30.508+0000 7fa2298e2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:32:30.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.621+0000 7efd4effd640 1 -- 172.21.15.43:0/3688266776 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7efd380afb30 msgr2=0x7efd380b2040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1953858940 2024-04-25T10:32:30.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.621+0000 7efd4effd640 1 --2- 172.21.15.43:0/3688266776 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7efd380afb30 0x7efd380b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:32:30.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:30.638+0000 7efd5617d640 1 -- 172.21.15.43:0/3688266776 --> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7efd50063080 con 0x7efd380afb30 2024-04-25T10:32:30.878 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:30 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:30.610+0000 7fa2298e2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:32:30.975 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:30 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:30.670+0000 7ff4d55d7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:32:31.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:31.022+0000 7efd4effd640 1 -- 172.21.15.43:0/3688266776 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7efd380afb30 msgr2=0x7efd380b2040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1953858940 2024-04-25T10:32:31.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:31.022+0000 7efd4effd640 1 --2- 172.21.15.43:0/3688266776 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7efd380afb30 0x7efd380b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:32:31.032 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:30 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:30.606+0000 7f7aa6f63200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:32:31.032 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:30 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:30.723+0000 7f7aa6f63200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:32:31.250 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:30 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:30.876+0000 7fa2298e2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:32:31.251 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:31 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:31.020+0000 7fa2298e2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:32:31.251 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:31 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:31.123+0000 7fa2298e2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:32:31.290 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:30 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:30.973+0000 7ff4d55d7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:32:31.567 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:31 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:31.248+0000 7fa2298e2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:32:31.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:31 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/362656845' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T10:32:31.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:31 smithi043 ceph-mon[29478]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T10:32:31.581 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:31 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/362656845' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T10:32:31.581 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:31 smithi077 ceph-mon[32772]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T10:32:31.640 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:31 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/362656845' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T10:32:31.640 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:31 smithi164 ceph-mon[32713]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T10:32:31.640 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:31 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:31.030+0000 7f7aa6f63200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:32:31.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:31.823+0000 7efd4effd640 1 -- 172.21.15.43:0/3688266776 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7efd380afb30 msgr2=0x7efd380b2040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1953858940 2024-04-25T10:32:31.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:31.823+0000 7efd4effd640 1 --2- 172.21.15.43:0/3688266776 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7efd380afb30 0x7efd380b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:32:31.860 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:31 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:31.565+0000 7fa2298e2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:32:32.040 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:31 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:31.579+0000 7ff4d55d7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:32:32.040 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:32:31 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y[34151]: 2024-04-25T10:32:31.796+0000 7ff4d55d7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:32:32.041 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:31 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:31.638+0000 7f7aa6f63200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:32:32.041 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:32:31 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z[34569]: 2024-04-25T10:32:31.856+0000 7f7aa6f63200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:32:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:32.309+0000 7efd4cff9640 1 -- 172.21.15.43:0/3688266776 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 20) ==== 147537+0+0 (secure 0 0 0) 0x7efd400c3ce0 con 0x7efd50103b50 2024-04-25T10:32:32.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:32.467+0000 7efd4cff9640 1 -- 172.21.15.43:0/3688266776 <== mon.2 v2:172.21.15.77:3300/0 8 ==== mgrmap(e 21) ==== 147205+0+0 (secure 0 0 0) 0x7efd4003ece0 con 0x7efd50103b50 2024-04-25T10:32:32.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:32.468+0000 7efd4cff9640 1 -- 172.21.15.43:0/3688266776 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7efd380afb30 msgr2=0x7efd380b2040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:32:32.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:32.468+0000 7efd4cff9640 1 --2- 172.21.15.43:0/3688266776 >> [v2:172.21.15.43:6800/1953858940,v1:172.21.15.43:6801/1953858940] conn(0x7efd380afb30 0x7efd380b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:32.610 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:32.214+0000 7fa2298e2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:32:32.610 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:32.454+0000 7fa2298e2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:32:32.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-mon[29478]: Standby manager daemon y restarted 2024-04-25T10:32:32.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-mon[29478]: Standby manager daemon y started 2024-04-25T10:32:32.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-mon[29478]: from='mgr.? 172.21.15.77:0/3669003398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T10:32:32.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-mon[29478]: from='mgr.? 172.21.15.77:0/3669003398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:32:32.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-mon[29478]: from='mgr.? 172.21.15.77:0/3669003398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T10:32:32.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-mon[29478]: from='mgr.? 172.21.15.77:0/3669003398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:32:32.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-mon[29478]: Standby manager daemon z restarted 2024-04-25T10:32:32.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-mon[29478]: Standby manager daemon z started 2024-04-25T10:32:32.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-mon[29478]: from='mgr.? 172.21.15.164:0/1076214821' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T10:32:32.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-mon[29478]: from='mgr.? 172.21.15.164:0/1076214821' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:32:32.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-mon[29478]: from='mgr.? 172.21.15.164:0/1076214821' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T10:32:32.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-mon[29478]: from='mgr.? 172.21.15.164:0/1076214821' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:32:32.621 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:32 smithi077 ceph-mon[32772]: Standby manager daemon y restarted 2024-04-25T10:32:32.621 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:32 smithi077 ceph-mon[32772]: Standby manager daemon y started 2024-04-25T10:32:32.621 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:32 smithi077 ceph-mon[32772]: from='mgr.? 172.21.15.77:0/3669003398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T10:32:32.621 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:32 smithi077 ceph-mon[32772]: from='mgr.? 172.21.15.77:0/3669003398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:32:32.621 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:32 smithi077 ceph-mon[32772]: from='mgr.? 172.21.15.77:0/3669003398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T10:32:32.621 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:32 smithi077 ceph-mon[32772]: from='mgr.? 172.21.15.77:0/3669003398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:32:32.621 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:32 smithi077 ceph-mon[32772]: Standby manager daemon z restarted 2024-04-25T10:32:32.621 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:32 smithi077 ceph-mon[32772]: Standby manager daemon z started 2024-04-25T10:32:32.622 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:32 smithi077 ceph-mon[32772]: from='mgr.? 172.21.15.164:0/1076214821' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T10:32:32.622 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:32 smithi077 ceph-mon[32772]: from='mgr.? 172.21.15.164:0/1076214821' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:32:32.622 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:32 smithi077 ceph-mon[32772]: from='mgr.? 172.21.15.164:0/1076214821' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T10:32:32.622 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:32 smithi077 ceph-mon[32772]: from='mgr.? 172.21.15.164:0/1076214821' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:32:32.622 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:32 smithi164 ceph-mon[32713]: Standby manager daemon y restarted 2024-04-25T10:32:32.622 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:32 smithi164 ceph-mon[32713]: Standby manager daemon y started 2024-04-25T10:32:32.622 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:32 smithi164 ceph-mon[32713]: from='mgr.? 172.21.15.77:0/3669003398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T10:32:32.622 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:32 smithi164 ceph-mon[32713]: from='mgr.? 172.21.15.77:0/3669003398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:32:32.622 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:32 smithi164 ceph-mon[32713]: from='mgr.? 172.21.15.77:0/3669003398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T10:32:32.622 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:32 smithi164 ceph-mon[32713]: from='mgr.? 172.21.15.77:0/3669003398' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:32:32.623 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:32 smithi164 ceph-mon[32713]: Standby manager daemon z restarted 2024-04-25T10:32:32.623 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:32 smithi164 ceph-mon[32713]: Standby manager daemon z started 2024-04-25T10:32:32.623 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:32 smithi164 ceph-mon[32713]: from='mgr.? 172.21.15.164:0/1076214821' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T10:32:32.623 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:32 smithi164 ceph-mon[32713]: from='mgr.? 172.21.15.164:0/1076214821' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T10:32:32.623 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:32 smithi164 ceph-mon[32713]: from='mgr.? 172.21.15.164:0/1076214821' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T10:32:32.623 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:32 smithi164 ceph-mon[32713]: from='mgr.? 172.21.15.164:0/1076214821' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T10:32:33.111 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:32.726+0000 7fa187fd2700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:32:33.111 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:32.726+0000 7fa187fd2700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:32:33.111 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:32.726+0000 7fa187fd2700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:32:33.111 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:32.726+0000 7fa187fd2700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:32:33.111 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:32.726+0000 7fa187fd2700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:32:33.111 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:32.726+0000 7fa189fd6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:32:33.111 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:32.726+0000 7fa189fd6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:32:33.111 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:32.726+0000 7fa189fd6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:32:33.111 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:32.726+0000 7fa189fd6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:32:33.112 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:32 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:32.726+0000 7fa189fd6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:32:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.476+0000 7efd4cff9640 1 -- 172.21.15.43:0/3688266776 <== mon.2 v2:172.21.15.77:3300/0 9 ==== mgrmap(e 22) ==== 147332+0+0 (secure 0 0 0) 0x7efd400fd010 con 0x7efd50103b50 2024-04-25T10:32:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.477+0000 7efd4cff9640 1 --2- 172.21.15.43:0/3688266776 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efd38038b80 0x7efd38038f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.477+0000 7efd4cff9640 1 -- 172.21.15.43:0/3688266776 --> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7efd3801ef20 con 0x7efd38038b80 2024-04-25T10:32:33.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.481+0000 7efd4effd640 1 --2- 172.21.15.43:0/3688266776 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efd38038b80 0x7efd38038f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.484+0000 7efd4effd640 1 --2- 172.21.15.43:0/3688266776 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efd38038b80 0x7efd38038f90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efd5019b360 tx=0x7efd44005f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:33.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.507+0000 7efd4cff9640 1 -- 172.21.15.43:0/3688266776 <== mgr.15291 v2:172.21.15.43:6800/3658924116 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7efd3801ef20 con 0x7efd38038b80 2024-04-25T10:32:33.509 INFO:teuthology.orchestra.run.smithi043.stdout:{} 2024-04-25T10:32:33.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.509+0000 7efd5617d640 1 -- 172.21.15.43:0/3688266776 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efd38038b80 msgr2=0x7efd38038f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:33.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.509+0000 7efd5617d640 1 --2- 172.21.15.43:0/3688266776 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efd38038b80 0x7efd38038f90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efd5019b360 tx=0x7efd44005f90 comp rx=0 tx=0).stop 2024-04-25T10:32:33.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.509+0000 7efd5617d640 1 -- 172.21.15.43:0/3688266776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd50103b50 msgr2=0x7efd5019a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:33.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.509+0000 7efd5617d640 1 --2- 172.21.15.43:0/3688266776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd50103b50 0x7efd5019a2c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efd4002a930 tx=0x7efd40004270 comp rx=0 tx=0).stop 2024-04-25T10:32:33.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.509+0000 7efd4ffff640 1 -- 172.21.15.43:0/3688266776 reap_dead start 2024-04-25T10:32:33.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.509+0000 7efd5617d640 1 -- 172.21.15.43:0/3688266776 shutdown_connections 2024-04-25T10:32:33.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.509+0000 7efd5617d640 1 -- 172.21.15.43:0/3688266776 >> 172.21.15.43:0/3688266776 conn(0x7efd500fed20 msgr2=0x7efd50105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:33.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.509+0000 7efd5617d640 1 -- 172.21.15.43:0/3688266776 shutdown_connections 2024-04-25T10:32:33.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.509+0000 7efd5617d640 1 -- 172.21.15.43:0/3688266776 wait complete. 2024-04-25T10:32:33.525 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T10:32:33.539 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-25T10:32:33.549 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:32:33.571 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:33.495+0000 7fa1857cd700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:32:33.571 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:33.495+0000 7fa1857cd700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:32:33.571 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:33.495+0000 7fa1857cd700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:32:33.572 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:33.495+0000 7fa1857cd700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:32:33.572 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:32:33.495+0000 7fa1857cd700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:32:33.572 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-25T10:32:33.572 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: Active manager daemon x restarted 2024-04-25T10:32:33.572 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: Activating manager daemon x 2024-04-25T10:32:33.572 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T10:32:33.573 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: mgrmap e21: x(active, starting, since 0.0133389s), standbys: z, y 2024-04-25T10:32:33.573 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:32:33.573 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:32:33.573 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:32:33.573 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T10:32:33.573 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T10:32:33.573 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T10:32:33.573 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T10:32:33.573 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T10:32:33.573 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T10:32:33.573 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T10:32:33.573 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T10:32:33.573 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T10:32:33.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T10:32:33.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T10:32:33.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T10:32:33.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T10:32:33.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T10:32:33.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T10:32:33.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:32:33.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:32:33.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:32:33.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:32:33.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:32:33.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:32:33.575 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:32:33.575 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:32:33.575 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:32:33.575 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:32:33.575 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:32:33.575 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:32:33.575 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T10:32:33.575 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T10:32:33.575 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T10:32:33.575 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: Manager daemon x is now available 2024-04-25T10:32:33.575 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:32:33.575 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T10:32:33.575 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T10:32:33.575 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T10:32:33.576 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:33 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T10:32:33.654 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-25T10:32:33.654 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: Active manager daemon x restarted 2024-04-25T10:32:33.654 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: Activating manager daemon x 2024-04-25T10:32:33.654 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T10:32:33.654 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: mgrmap e21: x(active, starting, since 0.0133389s), standbys: z, y 2024-04-25T10:32:33.654 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:32:33.654 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:32:33.655 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:32:33.655 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T10:32:33.655 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T10:32:33.655 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T10:32:33.655 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T10:32:33.655 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T10:32:33.655 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T10:32:33.655 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T10:32:33.655 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T10:32:33.655 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T10:32:33.655 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T10:32:33.655 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T10:32:33.655 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T10:32:33.655 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T10:32:33.656 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T10:32:33.656 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T10:32:33.656 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:32:33.656 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:32:33.656 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:32:33.656 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:32:33.656 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:32:33.656 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:32:33.656 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:32:33.656 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:32:33.656 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:32:33.656 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:32:33.656 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:32:33.656 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:32:33.657 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T10:32:33.657 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T10:32:33.657 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T10:32:33.657 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: Manager daemon x is now available 2024-04-25T10:32:33.657 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:32:33.657 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T10:32:33.657 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T10:32:33.657 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T10:32:33.657 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:33 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T10:32:33.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.665+0000 7f582a292640 1 -- 172.21.15.43:0/2647591597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5824102d10 msgr2=0x7f5824103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:33.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.665+0000 7f582a292640 1 --2- 172.21.15.43:0/2647591597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5824102d10 0x7f5824103120 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5818009930 tx=0x7f581802a420 comp rx=0 tx=0).stop 2024-04-25T10:32:33.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.665+0000 7f582a292640 1 -- 172.21.15.43:0/2647591597 shutdown_connections 2024-04-25T10:32:33.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.665+0000 7f582a292640 1 --2- 172.21.15.43:0/2647591597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5824103660 0x7f5824103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:33.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.665+0000 7f582a292640 1 --2- 172.21.15.43:0/2647591597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5824102d10 0x7f5824103120 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:33.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.665+0000 7f582a292640 1 --2- 172.21.15.43:0/2647591597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58241023c0 0x7f58241027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:33.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.665+0000 7f582a292640 1 -- 172.21.15.43:0/2647591597 >> 172.21.15.43:0/2647591597 conn(0x7f58240fdbf0 msgr2=0x7f5824100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:33.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.665+0000 7f582a292640 1 -- 172.21.15.43:0/2647591597 shutdown_connections 2024-04-25T10:32:33.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.665+0000 7f582a292640 1 -- 172.21.15.43:0/2647591597 wait complete. 2024-04-25T10:32:33.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.665+0000 7f582a292640 1 Processor -- start 2024-04-25T10:32:33.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.665+0000 7f582a292640 1 -- start start 2024-04-25T10:32:33.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.666+0000 7f582a292640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58241023c0 0x7f582419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:33.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.666+0000 7f582a292640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5824102d10 0x7f582419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:33.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.666+0000 7f582a292640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5824103660 0x7f58241a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:33.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.666+0000 7f582a292640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5824111820 con 0x7f58241023c0 2024-04-25T10:32:33.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.666+0000 7f582a292640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f58241116a0 con 0x7f5824102d10 2024-04-25T10:32:33.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.666+0000 7f582a292640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f58241119a0 con 0x7f5824103660 2024-04-25T10:32:33.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.666+0000 7f5823fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58241023c0 0x7f582419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:33.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.666+0000 7f5823fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58241023c0 0x7f582419c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48962/0 (socket says 172.21.15.43:48962) 2024-04-25T10:32:33.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.666+0000 7f5823fff640 1 -- 172.21.15.43:0/3012512221 learned_addr learned my addr 172.21.15.43:0/3012512221 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:33.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.666+0000 7f582882f640 1 --2- 172.21.15.43:0/3012512221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5824103660 0x7f58241a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:33.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.666+0000 7f58237fe640 1 --2- 172.21.15.43:0/3012512221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5824102d10 0x7f582419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:33.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.666+0000 7f5823fff640 1 -- 172.21.15.43:0/3012512221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5824103660 msgr2=0x7f58241a39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:33.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.666+0000 7f5823fff640 1 --2- 172.21.15.43:0/3012512221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5824103660 0x7f58241a39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:33.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.666+0000 7f5823fff640 1 -- 172.21.15.43:0/3012512221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5824102d10 msgr2=0x7f582419ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:33.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.666+0000 7f5823fff640 1 --2- 172.21.15.43:0/3012512221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5824102d10 0x7f582419ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:33.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.666+0000 7f5823fff640 1 -- 172.21.15.43:0/3012512221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58241a4020 con 0x7f58241023c0 2024-04-25T10:32:33.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.666+0000 7f5823fff640 1 --2- 172.21.15.43:0/3012512221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58241023c0 0x7f582419c4e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5814005fd0 tx=0x7f581400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:33.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.667+0000 7f58217fa640 1 -- 172.21.15.43:0/3012512221 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814017070 con 0x7f58241023c0 2024-04-25T10:32:33.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.667+0000 7f58217fa640 1 -- 172.21.15.43:0/3012512221 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5814009c30 con 0x7f58241023c0 2024-04-25T10:32:33.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.667+0000 7f58217fa640 1 -- 172.21.15.43:0/3012512221 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814003d30 con 0x7f58241023c0 2024-04-25T10:32:33.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.667+0000 7f582a292640 1 -- 172.21.15.43:0/3012512221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58241a4310 con 0x7f58241023c0 2024-04-25T10:32:33.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.667+0000 7f582a292640 1 -- 172.21.15.43:0/3012512221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58241a4860 con 0x7f58241023c0 2024-04-25T10:32:33.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.668+0000 7f582a292640 1 -- 172.21.15.43:0/3012512221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f582410d400 con 0x7f58241023c0 2024-04-25T10:32:33.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.669+0000 7f58217fa640 1 -- 172.21.15.43:0/3012512221 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) ==== 147332+0+0 (secure 0 0 0) 0x7f5814003ed0 con 0x7f58241023c0 2024-04-25T10:32:33.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.670+0000 7f58217fa640 1 --2- 172.21.15.43:0/3012512221 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f580c0af8e0 0x7f580c0b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:33.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.670+0000 7f58237fe640 1 --2- 172.21.15.43:0/3012512221 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f580c0af8e0 0x7f580c0b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:33.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.670+0000 7f58217fa640 1 -- 172.21.15.43:0/3012512221 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f58140ded80 con 0x7f58241023c0 2024-04-25T10:32:33.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.670+0000 7f58237fe640 1 --2- 172.21.15.43:0/3012512221 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f580c0af8e0 0x7f580c0b1df0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f582419d580 tx=0x7f5818000bc0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:33.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.674+0000 7f58217fa640 1 -- 172.21.15.43:0/3012512221 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5814020030 con 0x7f58241023c0 2024-04-25T10:32:33.695 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-25T10:32:33.696 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: Active manager daemon x restarted 2024-04-25T10:32:33.696 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: Activating manager daemon x 2024-04-25T10:32:33.696 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T10:32:33.696 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: mgrmap e21: x(active, starting, since 0.0133389s), standbys: z, y 2024-04-25T10:32:33.696 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T10:32:33.696 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T10:32:33.696 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T10:32:33.696 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T10:32:33.696 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T10:32:33.696 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T10:32:33.696 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T10:32:33.696 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T10:32:33.696 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T10:32:33.696 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T10:32:33.697 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T10:32:33.697 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T10:32:33.697 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T10:32:33.697 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T10:32:33.697 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T10:32:33.697 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T10:32:33.697 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T10:32:33.697 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T10:32:33.697 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T10:32:33.697 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T10:32:33.697 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T10:32:33.697 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T10:32:33.697 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T10:32:33.697 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T10:32:33.698 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T10:32:33.698 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T10:32:33.698 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T10:32:33.698 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T10:32:33.698 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T10:32:33.698 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T10:32:33.698 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T10:32:33.698 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T10:32:33.698 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T10:32:33.698 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: Manager daemon x is now available 2024-04-25T10:32:33.698 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:32:33.698 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T10:32:33.698 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T10:32:33.698 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T10:32:33.699 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:33 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T10:32:33.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.866+0000 7f582a292640 1 -- 172.21.15.43:0/3012512221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58240637a0 con 0x7f58241023c0 2024-04-25T10:32:33.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.868+0000 7f58217fa640 1 -- 172.21.15.43:0/3012512221 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22996 (secure 0 0 0) 0x7f5814009de0 con 0x7f58241023c0 2024-04-25T10:32:33.870 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:32:33.870 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":80,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:32:32.456657+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:31.503414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:32.224334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1599315071":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6801/1953858940":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6800/1953858940":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3026776968":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/188939647":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/1287690800":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:32:33.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.870+0000 7f582a292640 1 -- 172.21.15.43:0/3012512221 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f580c0af8e0 msgr2=0x7f580c0b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:33.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.870+0000 7f582a292640 1 --2- 172.21.15.43:0/3012512221 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f580c0af8e0 0x7f580c0b1df0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f582419d580 tx=0x7f5818000bc0 comp rx=0 tx=0).stop 2024-04-25T10:32:33.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.870+0000 7f582a292640 1 -- 172.21.15.43:0/3012512221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58241023c0 msgr2=0x7f582419c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:33.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.870+0000 7f582a292640 1 --2- 172.21.15.43:0/3012512221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58241023c0 0x7f582419c4e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5814005fd0 tx=0x7f581400dde0 comp rx=0 tx=0).stop 2024-04-25T10:32:33.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.870+0000 7f582a292640 1 -- 172.21.15.43:0/3012512221 shutdown_connections 2024-04-25T10:32:33.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.870+0000 7f582a292640 1 --2- 172.21.15.43:0/3012512221 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f580c0af8e0 0x7f580c0b1df0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:33.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.870+0000 7f582a292640 1 --2- 172.21.15.43:0/3012512221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5824102d10 0x7f582419ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:33.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.870+0000 7f582a292640 1 --2- 172.21.15.43:0/3012512221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5824103660 0x7f58241a39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:33.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.870+0000 7f582a292640 1 --2- 172.21.15.43:0/3012512221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58241023c0 0x7f582419c4e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:33.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.870+0000 7f582a292640 1 -- 172.21.15.43:0/3012512221 >> 172.21.15.43:0/3012512221 conn(0x7f58240fdbf0 msgr2=0x7f582410a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:33.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.870+0000 7f582a292640 1 -- 172.21.15.43:0/3012512221 shutdown_connections 2024-04-25T10:32:33.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:33.870+0000 7f582a292640 1 -- 172.21.15.43:0/3012512221 wait complete. 2024-04-25T10:32:33.885 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:25:58.167476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:31.503414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:32.224334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:32:33.886 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:32:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.041+0000 7facb896c640 1 -- 172.21.15.43:0/3163212616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facb41029f0 msgr2=0x7facb41036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.041+0000 7facb896c640 1 --2- 172.21.15.43:0/3163212616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facb41029f0 0x7facb41036f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7faca4009900 tx=0x7faca402a410 comp rx=0 tx=0).stop 2024-04-25T10:32:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.041+0000 7facb896c640 1 -- 172.21.15.43:0/3163212616 shutdown_connections 2024-04-25T10:32:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.041+0000 7facb896c640 1 --2- 172.21.15.43:0/3163212616 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facb4108930 0x7facb410ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.041+0000 7facb896c640 1 --2- 172.21.15.43:0/3163212616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facb4103c30 0x7facb4104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.041+0000 7facb896c640 1 --2- 172.21.15.43:0/3163212616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facb41029f0 0x7facb41036f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.041+0000 7facb896c640 1 -- 172.21.15.43:0/3163212616 >> 172.21.15.43:0/3163212616 conn(0x7facb40fdbc0 msgr2=0x7facb4100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:34.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.041+0000 7facb896c640 1 -- 172.21.15.43:0/3163212616 shutdown_connections 2024-04-25T10:32:34.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.041+0000 7facb896c640 1 -- 172.21.15.43:0/3163212616 wait complete. 2024-04-25T10:32:34.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.042+0000 7facb896c640 1 Processor -- start 2024-04-25T10:32:34.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.042+0000 7facb896c640 1 -- start start 2024-04-25T10:32:34.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.042+0000 7facb896c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facb41029f0 0x7facb40757f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:34.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.042+0000 7facb896c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facb4103c30 0x7facb419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:34.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.042+0000 7facb896c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facb4108930 0x7facb41a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:34.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.042+0000 7facb896c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7facb410e4d0 con 0x7facb4103c30 2024-04-25T10:32:34.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.042+0000 7facb896c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7facb410e350 con 0x7facb41029f0 2024-04-25T10:32:34.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.042+0000 7facb896c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7facb410e650 con 0x7facb4108930 2024-04-25T10:32:34.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.042+0000 7faca9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facb4103c30 0x7facb419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:34.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.042+0000 7facb2d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facb4108930 0x7facb41a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:34.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.042+0000 7faca9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facb4103c30 0x7facb419f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48976/0 (socket says 172.21.15.43:48976) 2024-04-25T10:32:34.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.042+0000 7faca9d9b640 1 -- 172.21.15.43:0/3328586956 learned_addr learned my addr 172.21.15.43:0/3328586956 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:34.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.042+0000 7facb259c640 1 --2- 172.21.15.43:0/3328586956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facb41029f0 0x7facb40757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:34.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.043+0000 7faca9d9b640 1 -- 172.21.15.43:0/3328586956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facb4108930 msgr2=0x7facb41a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:34.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.043+0000 7faca9d9b640 1 --2- 172.21.15.43:0/3328586956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facb4108930 0x7facb41a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.043+0000 7faca9d9b640 1 -- 172.21.15.43:0/3328586956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facb41029f0 msgr2=0x7facb40757f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:34.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.043+0000 7faca9d9b640 1 --2- 172.21.15.43:0/3328586956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facb41029f0 0x7facb40757f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.043+0000 7faca9d9b640 1 -- 172.21.15.43:0/3328586956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb41a1d50 con 0x7facb4103c30 2024-04-25T10:32:34.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.043+0000 7faca9d9b640 1 --2- 172.21.15.43:0/3328586956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facb4103c30 0x7facb419f030 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00e9f0 tx=0x7fac9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:34.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.043+0000 7facabfff640 1 -- 172.21.15.43:0/3328586956 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c00cd80 con 0x7facb4103c30 2024-04-25T10:32:34.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.043+0000 7facb896c640 1 -- 172.21.15.43:0/3328586956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb41a1fe0 con 0x7facb4103c30 2024-04-25T10:32:34.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.043+0000 7facabfff640 1 -- 172.21.15.43:0/3328586956 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac9c0108e0 con 0x7facb4103c30 2024-04-25T10:32:34.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.043+0000 7facabfff640 1 -- 172.21.15.43:0/3328586956 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c018980 con 0x7facb4103c30 2024-04-25T10:32:34.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.043+0000 7facb896c640 1 -- 172.21.15.43:0/3328586956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb41a24a0 con 0x7facb4103c30 2024-04-25T10:32:34.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.044+0000 7facb896c640 1 -- 172.21.15.43:0/3328586956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb4107740 con 0x7facb4103c30 2024-04-25T10:32:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.046+0000 7facabfff640 1 -- 172.21.15.43:0/3328586956 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) ==== 147332+0+0 (secure 0 0 0) 0x7fac9c010430 con 0x7facb4103c30 2024-04-25T10:32:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.046+0000 7facabfff640 1 --2- 172.21.15.43:0/3328586956 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7faca00af8e0 0x7faca00b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.046+0000 7facabfff640 1 -- 172.21.15.43:0/3328586956 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fac9c0df100 con 0x7facb4103c30 2024-04-25T10:32:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.046+0000 7facb259c640 1 --2- 172.21.15.43:0/3328586956 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7faca00af8e0 0x7faca00b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.047+0000 7facb259c640 1 --2- 172.21.15.43:0/3328586956 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7faca00af8e0 0x7faca00b1df0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faca4005ab0 tx=0x7faca4031400 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:34.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.050+0000 7facabfff640 1 -- 172.21.15.43:0/3328586956 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fac9c0a7890 con 0x7facb4103c30 2024-04-25T10:32:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.254+0000 7facb896c640 1 -- 172.21.15.43:0/3328586956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7facb40637a0 con 0x7facb4103c30 2024-04-25T10:32:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.255+0000 7facabfff640 1 -- 172.21.15.43:0/3328586956 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7fac9c0abb90 con 0x7facb4103c30 2024-04-25T10:32:34.257 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T10:32:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.257+0000 7facb896c640 1 -- 172.21.15.43:0/3328586956 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7faca00af8e0 msgr2=0x7faca00b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.257+0000 7facb896c640 1 --2- 172.21.15.43:0/3328586956 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7faca00af8e0 0x7faca00b1df0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faca4005ab0 tx=0x7faca4031400 comp rx=0 tx=0).stop 2024-04-25T10:32:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.257+0000 7facb896c640 1 -- 172.21.15.43:0/3328586956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facb4103c30 msgr2=0x7facb419f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.257+0000 7facb896c640 1 --2- 172.21.15.43:0/3328586956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facb4103c30 0x7facb419f030 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00e9f0 tx=0x7fac9c00eeb0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.257+0000 7facb896c640 1 -- 172.21.15.43:0/3328586956 shutdown_connections 2024-04-25T10:32:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.257+0000 7facb896c640 1 --2- 172.21.15.43:0/3328586956 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7faca00af8e0 0x7faca00b1df0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.257+0000 7facb896c640 1 --2- 172.21.15.43:0/3328586956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facb4108930 0x7facb41a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.257+0000 7facb896c640 1 --2- 172.21.15.43:0/3328586956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facb4103c30 0x7facb419f030 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.257+0000 7facb896c640 1 --2- 172.21.15.43:0/3328586956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facb41029f0 0x7facb40757f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.257+0000 7facb896c640 1 -- 172.21.15.43:0/3328586956 >> 172.21.15.43:0/3328586956 conn(0x7facb40fdbc0 msgr2=0x7facb4104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.257+0000 7facb896c640 1 -- 172.21.15.43:0/3328586956 shutdown_connections 2024-04-25T10:32:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.257+0000 7facb896c640 1 -- 172.21.15.43:0/3328586956 wait complete. 2024-04-25T10:32:34.271 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:32:34.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.390+0000 7f451c851640 1 -- 172.21.15.43:0/1684753336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45181098a0 msgr2=0x7f451810bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:34.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.390+0000 7f451c851640 1 --2- 172.21.15.43:0/1684753336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45181098a0 0x7f451810bcc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4510009ca0 tx=0x7f451002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.390+0000 7f451c851640 1 -- 172.21.15.43:0/1684753336 shutdown_connections 2024-04-25T10:32:34.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.390+0000 7f451c851640 1 --2- 172.21.15.43:0/1684753336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45181098a0 0x7f451810bcc0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.390+0000 7f451c851640 1 --2- 172.21.15.43:0/1684753336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4518106f40 0x7f4518109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.390+0000 7f451c851640 1 --2- 172.21.15.43:0/1684753336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45181045e0 0x7f4518106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.390+0000 7f451c851640 1 -- 172.21.15.43:0/1684753336 >> 172.21.15.43:0/1684753336 conn(0x7f45180fdc00 msgr2=0x7f4518100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:34.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.390+0000 7f451c851640 1 -- 172.21.15.43:0/1684753336 shutdown_connections 2024-04-25T10:32:34.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.390+0000 7f451c851640 1 -- 172.21.15.43:0/1684753336 wait complete. 2024-04-25T10:32:34.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.391+0000 7f451c851640 1 Processor -- start 2024-04-25T10:32:34.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.391+0000 7f451c851640 1 -- start start 2024-04-25T10:32:34.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.391+0000 7f451c851640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45181045e0 0x7f451819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:34.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.391+0000 7f451c851640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4518106f40 0x7f451819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:34.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.391+0000 7f451c851640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45181098a0 0x7f45181a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:34.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.391+0000 7f451c851640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f451810f620 con 0x7f4518106f40 2024-04-25T10:32:34.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.391+0000 7f451c851640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f451810f4a0 con 0x7f45181045e0 2024-04-25T10:32:34.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.391+0000 7f451c851640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f451810f7a0 con 0x7f45181098a0 2024-04-25T10:32:34.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.391+0000 7f4515d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4518106f40 0x7f451819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:34.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.392+0000 7f4515d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4518106f40 0x7f451819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48980/0 (socket says 172.21.15.43:48980) 2024-04-25T10:32:34.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.392+0000 7f451659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45181045e0 0x7f451819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:34.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.392+0000 7f4515d9b640 1 -- 172.21.15.43:0/3328916500 learned_addr learned my addr 172.21.15.43:0/3328916500 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:34.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.392+0000 7f4516d9d640 1 --2- 172.21.15.43:0/3328916500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45181098a0 0x7f45181a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:34.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.392+0000 7f4515d9b640 1 -- 172.21.15.43:0/3328916500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45181098a0 msgr2=0x7f45181a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:34.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.392+0000 7f4515d9b640 1 --2- 172.21.15.43:0/3328916500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45181098a0 0x7f45181a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.392+0000 7f4515d9b640 1 -- 172.21.15.43:0/3328916500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45181045e0 msgr2=0x7f451819a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:34.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.392+0000 7f4515d9b640 1 --2- 172.21.15.43:0/3328916500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45181045e0 0x7f451819a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.392+0000 7f4515d9b640 1 -- 172.21.15.43:0/3328916500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45181a1d30 con 0x7f4518106f40 2024-04-25T10:32:34.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.392+0000 7f451659c640 1 --2- 172.21.15.43:0/3328916500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45181045e0 0x7f451819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:34.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.392+0000 7f4515d9b640 1 --2- 172.21.15.43:0/3328916500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4518106f40 0x7f451819a730 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f450c00e9f0 tx=0x7f450c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:34.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.392+0000 7f45077fe640 1 -- 172.21.15.43:0/3328916500 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c00cd80 con 0x7f4518106f40 2024-04-25T10:32:34.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.392+0000 7f45077fe640 1 -- 172.21.15.43:0/3328916500 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f450c0054d0 con 0x7f4518106f40 2024-04-25T10:32:34.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.392+0000 7f451c851640 1 -- 172.21.15.43:0/3328916500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45181a1fc0 con 0x7f4518106f40 2024-04-25T10:32:34.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.392+0000 7f45077fe640 1 -- 172.21.15.43:0/3328916500 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c010690 con 0x7f4518106f40 2024-04-25T10:32:34.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.392+0000 7f451c851640 1 -- 172.21.15.43:0/3328916500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45181a24b0 con 0x7f4518106f40 2024-04-25T10:32:34.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.394+0000 7f451c851640 1 -- 172.21.15.43:0/3328916500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45180ff720 con 0x7f4518106f40 2024-04-25T10:32:34.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.395+0000 7f45077fe640 1 -- 172.21.15.43:0/3328916500 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) ==== 147332+0+0 (secure 0 0 0) 0x7f450c004e90 con 0x7f4518106f40 2024-04-25T10:32:34.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.396+0000 7f45077fe640 1 --2- 172.21.15.43:0/3328916500 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f45000af960 0x7f45000b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:34.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.396+0000 7f451659c640 1 --2- 172.21.15.43:0/3328916500 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f45000af960 0x7f45000b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:34.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.396+0000 7f45077fe640 1 -- 172.21.15.43:0/3328916500 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f450c0decb0 con 0x7f4518106f40 2024-04-25T10:32:34.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.397+0000 7f451659c640 1 --2- 172.21.15.43:0/3328916500 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f45000af960 0x7f45000b1e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f45080095f0 tx=0x7f4508005f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:34.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.401+0000 7f45077fe640 1 -- 172.21.15.43:0/3328916500 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f450c0a7a90 con 0x7f4518106f40 2024-04-25T10:32:34.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.483+0000 7f45077fe640 1 -- 172.21.15.43:0/3328916500 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 23) ==== 147691+0+0 (secure 0 0 0) 0x7f450c0a3620 con 0x7f4518106f40 2024-04-25T10:32:34.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:34 smithi043 ceph-mon[29478]: mgrmap e22: x(active, since 1.02444s), standbys: z, y 2024-04-25T10:32:34.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:34 smithi043 ceph-mon[29478]: [25/Apr/2024:10:32:33] ENGINE Bus STARTING 2024-04-25T10:32:34.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:34 smithi043 ceph-mon[29478]: [25/Apr/2024:10:32:33] ENGINE Serving on https://172.21.15.43:7150 2024-04-25T10:32:34.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:34 smithi043 ceph-mon[29478]: [25/Apr/2024:10:32:33] ENGINE Serving on http://172.21.15.43:8765 2024-04-25T10:32:34.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:34 smithi043 ceph-mon[29478]: [25/Apr/2024:10:32:33] ENGINE Bus STARTED 2024-04-25T10:32:34.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:34 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3012512221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:34.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:34 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3328586956' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:32:34.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.622+0000 7f451c851640 1 -- 172.21.15.43:0/3328916500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f45180637a0 con 0x7f4518106f40 2024-04-25T10:32:34.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.623+0000 7f45077fe640 1 -- 172.21.15.43:0/3328916500 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7f450c055aa0 con 0x7f4518106f40 2024-04-25T10:32:34.624 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T10:32:34.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.625+0000 7f451c851640 1 -- 172.21.15.43:0/3328916500 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f45000af960 msgr2=0x7f45000b1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:34.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.625+0000 7f451c851640 1 --2- 172.21.15.43:0/3328916500 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f45000af960 0x7f45000b1e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f45080095f0 tx=0x7f4508005f90 comp rx=0 tx=0).stop 2024-04-25T10:32:34.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.625+0000 7f451c851640 1 -- 172.21.15.43:0/3328916500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4518106f40 msgr2=0x7f451819a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:34.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.625+0000 7f451c851640 1 --2- 172.21.15.43:0/3328916500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4518106f40 0x7f451819a730 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f450c00e9f0 tx=0x7f450c00edf0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.625+0000 7f451c851640 1 -- 172.21.15.43:0/3328916500 shutdown_connections 2024-04-25T10:32:34.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.625+0000 7f451c851640 1 --2- 172.21.15.43:0/3328916500 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f45000af960 0x7f45000b1e70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.625+0000 7f451c851640 1 --2- 172.21.15.43:0/3328916500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45181098a0 0x7f45181a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.625+0000 7f451c851640 1 --2- 172.21.15.43:0/3328916500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4518106f40 0x7f451819a730 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.625+0000 7f451c851640 1 --2- 172.21.15.43:0/3328916500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45181045e0 0x7f451819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.625+0000 7f451c851640 1 -- 172.21.15.43:0/3328916500 >> 172.21.15.43:0/3328916500 conn(0x7f45180fdc00 msgr2=0x7f4518105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:34.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.625+0000 7f451c851640 1 -- 172.21.15.43:0/3328916500 shutdown_connections 2024-04-25T10:32:34.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.625+0000 7f451c851640 1 -- 172.21.15.43:0/3328916500 wait complete. 2024-04-25T10:32:34.640 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:32:34.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.765+0000 7fdc17c49640 1 -- 172.21.15.43:0/271277014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc101029f0 msgr2=0x7fdc101036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.765+0000 7fdc17c49640 1 --2- 172.21.15.43:0/271277014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc101029f0 0x7fdc101036f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009900 tx=0x7fdc0c02e820 comp rx=0 tx=0).stop 2024-04-25T10:32:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.765+0000 7fdc17c49640 1 -- 172.21.15.43:0/271277014 shutdown_connections 2024-04-25T10:32:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.765+0000 7fdc17c49640 1 --2- 172.21.15.43:0/271277014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc10103c30 0x7fdc10104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.765+0000 7fdc17c49640 1 --2- 172.21.15.43:0/271277014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc10108930 0x7fdc1010ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.765+0000 7fdc17c49640 1 --2- 172.21.15.43:0/271277014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc101029f0 0x7fdc101036f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.765+0000 7fdc17c49640 1 -- 172.21.15.43:0/271277014 >> 172.21.15.43:0/271277014 conn(0x7fdc100fdc00 msgr2=0x7fdc10100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.765+0000 7fdc17c49640 1 -- 172.21.15.43:0/271277014 shutdown_connections 2024-04-25T10:32:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.765+0000 7fdc17c49640 1 -- 172.21.15.43:0/271277014 wait complete. 2024-04-25T10:32:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.766+0000 7fdc17c49640 1 Processor -- start 2024-04-25T10:32:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.766+0000 7fdc17c49640 1 -- start start 2024-04-25T10:32:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.766+0000 7fdc17c49640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc101029f0 0x7fdc101980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.766+0000 7fdc17c49640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc10103c30 0x7fdc101985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.766+0000 7fdc17c49640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc10108930 0x7fdc1019f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.766+0000 7fdc17c49640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1010d410 con 0x7fdc10108930 2024-04-25T10:32:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.766+0000 7fdc17c49640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1010d290 con 0x7fdc10103c30 2024-04-25T10:32:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.766+0000 7fdc17c49640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1010d590 con 0x7fdc101029f0 2024-04-25T10:32:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.766+0000 7fdc161e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc10108930 0x7fdc1019f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.766+0000 7fdc161e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc10108930 0x7fdc1019f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49012/0 (socket says 172.21.15.43:49012) 2024-04-25T10:32:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.766+0000 7fdc161e6640 1 -- 172.21.15.43:0/4271564083 learned_addr learned my addr 172.21.15.43:0/4271564083 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.766+0000 7fdc151e4640 1 --2- 172.21.15.43:0/4271564083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc10103c30 0x7fdc101985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.766+0000 7fdc159e5640 1 --2- 172.21.15.43:0/4271564083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc101029f0 0x7fdc101980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.767+0000 7fdc161e6640 1 -- 172.21.15.43:0/4271564083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc101029f0 msgr2=0x7fdc101980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.767+0000 7fdc161e6640 1 --2- 172.21.15.43:0/4271564083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc101029f0 0x7fdc101980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.767+0000 7fdc161e6640 1 -- 172.21.15.43:0/4271564083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc10103c30 msgr2=0x7fdc101985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:34.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.767+0000 7fdc161e6640 1 --2- 172.21.15.43:0/4271564083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc10103c30 0x7fdc101985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.767+0000 7fdc161e6640 1 -- 172.21.15.43:0/4271564083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1019fb70 con 0x7fdc10108930 2024-04-25T10:32:34.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.767+0000 7fdc151e4640 1 --2- 172.21.15.43:0/4271564083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc10103c30 0x7fdc101985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:34.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.767+0000 7fdc161e6640 1 --2- 172.21.15.43:0/4271564083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc10108930 0x7fdc1019f500 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800eb10 tx=0x7fdc0800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:34.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.767+0000 7fdc02ffd640 1 -- 172.21.15.43:0/4271564083 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08004b20 con 0x7fdc10108930 2024-04-25T10:32:34.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.767+0000 7fdc17c49640 1 -- 172.21.15.43:0/4271564083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1019fe60 con 0x7fdc10108930 2024-04-25T10:32:34.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.767+0000 7fdc02ffd640 1 -- 172.21.15.43:0/4271564083 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc08004cc0 con 0x7fdc10108930 2024-04-25T10:32:34.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.767+0000 7fdc02ffd640 1 -- 172.21.15.43:0/4271564083 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08010670 con 0x7fdc10108930 2024-04-25T10:32:34.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.767+0000 7fdc17c49640 1 -- 172.21.15.43:0/4271564083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc101a02c0 con 0x7fdc10108930 2024-04-25T10:32:34.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.768+0000 7fdc17c49640 1 -- 172.21.15.43:0/4271564083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc10107740 con 0x7fdc10108930 2024-04-25T10:32:34.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.770+0000 7fdc02ffd640 1 -- 172.21.15.43:0/4271564083 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 147691+0+0 (secure 0 0 0) 0x7fdc08005020 con 0x7fdc10108930 2024-04-25T10:32:34.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.770+0000 7fdc02ffd640 1 --2- 172.21.15.43:0/4271564083 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdbfc0afcf0 0x7fdbfc0b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:34.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.771+0000 7fdc159e5640 1 --2- 172.21.15.43:0/4271564083 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdbfc0afcf0 0x7fdbfc0b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:34.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.771+0000 7fdc02ffd640 1 -- 172.21.15.43:0/4271564083 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fdc080df7c0 con 0x7fdc10108930 2024-04-25T10:32:34.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.771+0000 7fdc159e5640 1 --2- 172.21.15.43:0/4271564083 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdbfc0afcf0 0x7fdbfc0b2200 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c038040 tx=0x7fdc0c036410 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:34.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.775+0000 7fdc02ffd640 1 -- 172.21.15.43:0/4271564083 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc080a7ec0 con 0x7fdc10108930 2024-04-25T10:32:34.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:34 smithi077 ceph-mon[32772]: mgrmap e22: x(active, since 1.02444s), standbys: z, y 2024-04-25T10:32:34.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:34 smithi077 ceph-mon[32772]: [25/Apr/2024:10:32:33] ENGINE Bus STARTING 2024-04-25T10:32:34.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:34 smithi077 ceph-mon[32772]: [25/Apr/2024:10:32:33] ENGINE Serving on https://172.21.15.43:7150 2024-04-25T10:32:34.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:34 smithi077 ceph-mon[32772]: [25/Apr/2024:10:32:33] ENGINE Serving on http://172.21.15.43:8765 2024-04-25T10:32:34.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:34 smithi077 ceph-mon[32772]: [25/Apr/2024:10:32:33] ENGINE Bus STARTED 2024-04-25T10:32:34.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:34 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3012512221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:34.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:34 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3328586956' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:32:34.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:34 smithi164 ceph-mon[32713]: mgrmap e22: x(active, since 1.02444s), standbys: z, y 2024-04-25T10:32:34.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:34 smithi164 ceph-mon[32713]: [25/Apr/2024:10:32:33] ENGINE Bus STARTING 2024-04-25T10:32:34.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:34 smithi164 ceph-mon[32713]: [25/Apr/2024:10:32:33] ENGINE Serving on https://172.21.15.43:7150 2024-04-25T10:32:34.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:34 smithi164 ceph-mon[32713]: [25/Apr/2024:10:32:33] ENGINE Serving on http://172.21.15.43:8765 2024-04-25T10:32:34.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:34 smithi164 ceph-mon[32713]: [25/Apr/2024:10:32:33] ENGINE Bus STARTED 2024-04-25T10:32:34.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:34 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3012512221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:34.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:34 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3328586956' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:32:34.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.963+0000 7fdc17c49640 1 -- 172.21.15.43:0/4271564083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdc1002dda0 con 0x7fdc10108930 2024-04-25T10:32:34.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.964+0000 7fdc02ffd640 1 -- 172.21.15.43:0/4271564083 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7fdc080a77c0 con 0x7fdc10108930 2024-04-25T10:32:34.966 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T10:32:34.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.966+0000 7fdc17c49640 1 -- 172.21.15.43:0/4271564083 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdbfc0afcf0 msgr2=0x7fdbfc0b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:34.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.966+0000 7fdc17c49640 1 --2- 172.21.15.43:0/4271564083 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdbfc0afcf0 0x7fdbfc0b2200 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c038040 tx=0x7fdc0c036410 comp rx=0 tx=0).stop 2024-04-25T10:32:34.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.966+0000 7fdc17c49640 1 -- 172.21.15.43:0/4271564083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc10108930 msgr2=0x7fdc1019f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:34.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.966+0000 7fdc17c49640 1 --2- 172.21.15.43:0/4271564083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc10108930 0x7fdc1019f500 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800eb10 tx=0x7fdc0800c2d0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.966+0000 7fdc17c49640 1 -- 172.21.15.43:0/4271564083 shutdown_connections 2024-04-25T10:32:34.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.966+0000 7fdc17c49640 1 --2- 172.21.15.43:0/4271564083 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdbfc0afcf0 0x7fdbfc0b2200 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.966+0000 7fdc17c49640 1 --2- 172.21.15.43:0/4271564083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc10103c30 0x7fdc101985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.966+0000 7fdc17c49640 1 --2- 172.21.15.43:0/4271564083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc10108930 0x7fdc1019f500 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.966+0000 7fdc17c49640 1 --2- 172.21.15.43:0/4271564083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc101029f0 0x7fdc101980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:34.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.966+0000 7fdc17c49640 1 -- 172.21.15.43:0/4271564083 >> 172.21.15.43:0/4271564083 conn(0x7fdc100fdc00 msgr2=0x7fdc10104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:34.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.966+0000 7fdc17c49640 1 -- 172.21.15.43:0/4271564083 shutdown_connections 2024-04-25T10:32:34.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:34.966+0000 7fdc17c49640 1 -- 172.21.15.43:0/4271564083 wait complete. 2024-04-25T10:32:34.980 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:32:35.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.093+0000 7fa60f04c640 1 -- 172.21.15.43:0/270923796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6081029c0 msgr2=0x7fa6081036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:35.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.093+0000 7fa60f04c640 1 --2- 172.21.15.43:0/270923796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6081029c0 0x7fa6081036c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009900 tx=0x7fa5f802a410 comp rx=0 tx=0).stop 2024-04-25T10:32:35.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.093+0000 7fa60f04c640 1 -- 172.21.15.43:0/270923796 shutdown_connections 2024-04-25T10:32:35.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.093+0000 7fa60f04c640 1 --2- 172.21.15.43:0/270923796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa608103c00 0x7fa608104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.093+0000 7fa60f04c640 1 --2- 172.21.15.43:0/270923796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa608108900 0x7fa60810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.093+0000 7fa60f04c640 1 --2- 172.21.15.43:0/270923796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6081029c0 0x7fa6081036c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.093+0000 7fa60f04c640 1 -- 172.21.15.43:0/270923796 >> 172.21.15.43:0/270923796 conn(0x7fa6080fdbd0 msgr2=0x7fa608100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:35.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.093+0000 7fa60f04c640 1 -- 172.21.15.43:0/270923796 shutdown_connections 2024-04-25T10:32:35.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.093+0000 7fa60f04c640 1 -- 172.21.15.43:0/270923796 wait complete. 2024-04-25T10:32:35.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.093+0000 7fa60f04c640 1 Processor -- start 2024-04-25T10:32:35.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.093+0000 7fa60f04c640 1 -- start start 2024-04-25T10:32:35.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.094+0000 7fa60f04c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6081029c0 0x7fa60806a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:35.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.094+0000 7fa60f04c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa608103c00 0x7fa60819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:35.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.094+0000 7fa60f04c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa608108900 0x7fa6081a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:35.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.094+0000 7fa60f04c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa60810e4c0 con 0x7fa608103c00 2024-04-25T10:32:35.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.094+0000 7fa60f04c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa60810e340 con 0x7fa608108900 2024-04-25T10:32:35.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.094+0000 7fa60f04c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa60810e640 con 0x7fa6081029c0 2024-04-25T10:32:35.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.094+0000 7fa607fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa608103c00 0x7fa60819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:35.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.094+0000 7fa607fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa608103c00 0x7fa60819efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49038/0 (socket says 172.21.15.43:49038) 2024-04-25T10:32:35.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.094+0000 7fa607fff640 1 -- 172.21.15.43:0/2982543071 learned_addr learned my addr 172.21.15.43:0/2982543071 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:35.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.094+0000 7fa60cde8640 1 --2- 172.21.15.43:0/2982543071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6081029c0 0x7fa60806a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:35.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.094+0000 7fa60d5e9640 1 --2- 172.21.15.43:0/2982543071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa608108900 0x7fa6081a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:35.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.094+0000 7fa60d5e9640 1 -- 172.21.15.43:0/2982543071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6081029c0 msgr2=0x7fa60806a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:35.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.094+0000 7fa60d5e9640 1 --2- 172.21.15.43:0/2982543071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6081029c0 0x7fa60806a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.094+0000 7fa60d5e9640 1 -- 172.21.15.43:0/2982543071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa608103c00 msgr2=0x7fa60819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:35.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.094+0000 7fa60d5e9640 1 --2- 172.21.15.43:0/2982543071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa608103c00 0x7fa60819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.094+0000 7fa60d5e9640 1 -- 172.21.15.43:0/2982543071 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6081a1d40 con 0x7fa608108900 2024-04-25T10:32:35.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.094+0000 7fa60cde8640 1 --2- 172.21.15.43:0/2982543071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6081029c0 0x7fa60806a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:35.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.094+0000 7fa607fff640 1 --2- 172.21.15.43:0/2982543071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa608103c00 0x7fa60819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:35.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.094+0000 7fa60d5e9640 1 --2- 172.21.15.43:0/2982543071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa608108900 0x7fa6081a1520 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa600009fd0 tx=0x7fa60000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:35.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.095+0000 7fa605ffb640 1 -- 172.21.15.43:0/2982543071 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60000ce20 con 0x7fa608108900 2024-04-25T10:32:35.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.095+0000 7fa605ffb640 1 -- 172.21.15.43:0/2982543071 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa600005020 con 0x7fa608108900 2024-04-25T10:32:35.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.095+0000 7fa60f04c640 1 -- 172.21.15.43:0/2982543071 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6081a1fd0 con 0x7fa608108900 2024-04-25T10:32:35.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.095+0000 7fa605ffb640 1 -- 172.21.15.43:0/2982543071 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa600010690 con 0x7fa608108900 2024-04-25T10:32:35.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.095+0000 7fa60f04c640 1 -- 172.21.15.43:0/2982543071 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6081a2490 con 0x7fa608108900 2024-04-25T10:32:35.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.096+0000 7fa60f04c640 1 -- 172.21.15.43:0/2982543071 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa608107710 con 0x7fa608108900 2024-04-25T10:32:35.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.097+0000 7fa605ffb640 1 -- 172.21.15.43:0/2982543071 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 23) ==== 147691+0+0 (secure 0 0 0) 0x7fa600004b20 con 0x7fa608108900 2024-04-25T10:32:35.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.098+0000 7fa605ffb640 1 --2- 172.21.15.43:0/2982543071 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa5f40afcf0 0x7fa5f40b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:35.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.098+0000 7fa605ffb640 1 -- 172.21.15.43:0/2982543071 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fa6000df880 con 0x7fa608108900 2024-04-25T10:32:35.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.098+0000 7fa60cde8640 1 --2- 172.21.15.43:0/2982543071 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa5f40afcf0 0x7fa5f40b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:35.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.098+0000 7fa60cde8640 1 --2- 172.21.15.43:0/2982543071 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa5f40afcf0 0x7fa5f40b2200 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8005ab0 tx=0x7fa5f8031400 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:35.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.103+0000 7fa605ffb640 1 -- 172.21.15.43:0/2982543071 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa6000a7f80 con 0x7fa608108900 2024-04-25T10:32:35.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.294+0000 7fa60f04c640 1 -- 172.21.15.43:0/2982543071 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6080637a0 con 0x7fa608108900 2024-04-25T10:32:35.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.296+0000 7fa605ffb640 1 -- 172.21.15.43:0/2982543071 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22996 (secure 0 0 0) 0x7fa6000a7880 con 0x7fa608108900 2024-04-25T10:32:35.299 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:32:35.299 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":80,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:32:32.456657+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:31.503414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:32.224334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1599315071":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6801/1953858940":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6800/1953858940":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3026776968":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/188939647":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/1287690800":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:32:35.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.298+0000 7fa60f04c640 1 -- 172.21.15.43:0/2982543071 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa5f40afcf0 msgr2=0x7fa5f40b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:35.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.298+0000 7fa60f04c640 1 --2- 172.21.15.43:0/2982543071 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa5f40afcf0 0x7fa5f40b2200 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8005ab0 tx=0x7fa5f8031400 comp rx=0 tx=0).stop 2024-04-25T10:32:35.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.298+0000 7fa60f04c640 1 -- 172.21.15.43:0/2982543071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa608108900 msgr2=0x7fa6081a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:35.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.298+0000 7fa60f04c640 1 --2- 172.21.15.43:0/2982543071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa608108900 0x7fa6081a1520 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa600009fd0 tx=0x7fa60000ef10 comp rx=0 tx=0).stop 2024-04-25T10:32:35.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.298+0000 7fa60f04c640 1 -- 172.21.15.43:0/2982543071 shutdown_connections 2024-04-25T10:32:35.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.298+0000 7fa60f04c640 1 --2- 172.21.15.43:0/2982543071 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa5f40afcf0 0x7fa5f40b2200 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.298+0000 7fa60f04c640 1 --2- 172.21.15.43:0/2982543071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa608108900 0x7fa6081a1520 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.298+0000 7fa60f04c640 1 --2- 172.21.15.43:0/2982543071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa608103c00 0x7fa60819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.298+0000 7fa60f04c640 1 --2- 172.21.15.43:0/2982543071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6081029c0 0x7fa60806a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.298+0000 7fa60f04c640 1 -- 172.21.15.43:0/2982543071 >> 172.21.15.43:0/2982543071 conn(0x7fa6080fdbd0 msgr2=0x7fa6081048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:35.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.298+0000 7fa60f04c640 1 -- 172.21.15.43:0/2982543071 shutdown_connections 2024-04-25T10:32:35.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.298+0000 7fa60f04c640 1 -- 172.21.15.43:0/2982543071 wait complete. 2024-04-25T10:32:35.313 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:25:58.167476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:31.503414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:32.224334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:32:35.313 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:32:35.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.427+0000 7f9d60042640 1 -- 172.21.15.43:0/1057868736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d58109a40 msgr2=0x7f9d5810be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:35.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.427+0000 7f9d60042640 1 --2- 172.21.15.43:0/1057868736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d58109a40 0x7f9d5810be60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009ca0 tx=0x7f9d5002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.427+0000 7f9d60042640 1 -- 172.21.15.43:0/1057868736 shutdown_connections 2024-04-25T10:32:35.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.428+0000 7f9d60042640 1 --2- 172.21.15.43:0/1057868736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d58109a40 0x7f9d5810be60 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.428+0000 7f9d60042640 1 --2- 172.21.15.43:0/1057868736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d58104d40 0x7f9d58105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.428+0000 7f9d60042640 1 --2- 172.21.15.43:0/1057868736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d58103b00 0x7f9d58104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.428+0000 7f9d60042640 1 -- 172.21.15.43:0/1057868736 >> 172.21.15.43:0/1057868736 conn(0x7f9d580fecf0 msgr2=0x7f9d581011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:35.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.428+0000 7f9d60042640 1 -- 172.21.15.43:0/1057868736 shutdown_connections 2024-04-25T10:32:35.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.428+0000 7f9d60042640 1 -- 172.21.15.43:0/1057868736 wait complete. 2024-04-25T10:32:35.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.428+0000 7f9d60042640 1 Processor -- start 2024-04-25T10:32:35.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.429+0000 7f9d60042640 1 -- start start 2024-04-25T10:32:35.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.429+0000 7f9d60042640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d58103b00 0x7f9d5819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:35.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.429+0000 7f9d60042640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d58104d40 0x7f9d5819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.429+0000 7f9d60042640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d58109a40 0x7f9d581a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.429+0000 7f9d60042640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5810f680 con 0x7f9d58109a40 2024-04-25T10:32:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.429+0000 7f9d60042640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5810f500 con 0x7f9d58103b00 2024-04-25T10:32:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.429+0000 7f9d60042640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5810f800 con 0x7f9d58104d40 2024-04-25T10:32:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.429+0000 7f9d5e5df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d58109a40 0x7f9d581a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.429+0000 7f9d5e5df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d58109a40 0x7f9d581a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49052/0 (socket says 172.21.15.43:49052) 2024-04-25T10:32:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.429+0000 7f9d5e5df640 1 -- 172.21.15.43:0/3327052020 learned_addr learned my addr 172.21.15.43:0/3327052020 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.429+0000 7f9d5ddde640 1 --2- 172.21.15.43:0/3327052020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d58103b00 0x7f9d5819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.429+0000 7f9d5d5dd640 1 --2- 172.21.15.43:0/3327052020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d58104d40 0x7f9d5819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.429+0000 7f9d5e5df640 1 -- 172.21.15.43:0/3327052020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d58104d40 msgr2=0x7f9d5819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.429+0000 7f9d5e5df640 1 --2- 172.21.15.43:0/3327052020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d58104d40 0x7f9d5819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.429+0000 7f9d5e5df640 1 -- 172.21.15.43:0/3327052020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d58103b00 msgr2=0x7f9d5819a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.429+0000 7f9d5e5df640 1 --2- 172.21.15.43:0/3327052020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d58103b00 0x7f9d5819a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.429+0000 7f9d5e5df640 1 -- 172.21.15.43:0/3327052020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d581a1d30 con 0x7f9d58109a40 2024-04-25T10:32:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.429+0000 7f9d5ddde640 1 --2- 172.21.15.43:0/3327052020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d58103b00 0x7f9d5819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.429+0000 7f9d5e5df640 1 --2- 172.21.15.43:0/3327052020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d58109a40 0x7f9d581a16c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009430 tx=0x7f9d500316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.430+0000 7f9d4effd640 1 -- 172.21.15.43:0/3327052020 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50003b70 con 0x7f9d58109a40 2024-04-25T10:32:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.430+0000 7f9d4effd640 1 -- 172.21.15.43:0/3327052020 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d50003d10 con 0x7f9d58109a40 2024-04-25T10:32:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.430+0000 7f9d60042640 1 -- 172.21.15.43:0/3327052020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d581a1fc0 con 0x7f9d58109a40 2024-04-25T10:32:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.430+0000 7f9d4effd640 1 -- 172.21.15.43:0/3327052020 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5002fb40 con 0x7f9d58109a40 2024-04-25T10:32:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.430+0000 7f9d60042640 1 -- 172.21.15.43:0/3327052020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d581a2420 con 0x7f9d58109a40 2024-04-25T10:32:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.431+0000 7f9d60042640 1 -- 172.21.15.43:0/3327052020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d5806afa0 con 0x7f9d58109a40 2024-04-25T10:32:35.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.432+0000 7f9d4effd640 1 -- 172.21.15.43:0/3327052020 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 147691+0+0 (secure 0 0 0) 0x7f9d50004610 con 0x7f9d58109a40 2024-04-25T10:32:35.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.433+0000 7f9d4effd640 1 --2- 172.21.15.43:0/3327052020 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9d440afcf0 0x7f9d440b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:35.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.433+0000 7f9d5ddde640 1 --2- 172.21.15.43:0/3327052020 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9d440afcf0 0x7f9d440b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:35.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.433+0000 7f9d4effd640 1 -- 172.21.15.43:0/3327052020 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f9d50100630 con 0x7f9d58109a40 2024-04-25T10:32:35.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.434+0000 7f9d5ddde640 1 --2- 172.21.15.43:0/3327052020 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9d440afcf0 0x7f9d440b2200 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9d58104700 tx=0x7f9d54005f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:35.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.437+0000 7f9d4effd640 1 -- 172.21.15.43:0/3327052020 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9d500c8ea0 con 0x7f9d58109a40 2024-04-25T10:32:35.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:35 smithi043 ceph-mon[29478]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:32:35.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:35 smithi043 ceph-mon[29478]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T10:32:35.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:35 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:35.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:35 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:35.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:35 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3328916500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:32:35.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:35 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:35.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:35 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:35.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:35 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:35.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:35 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:35.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:35 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/4271564083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:32:35.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:35 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2982543071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:35.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.636+0000 7f9d60042640 1 -- 172.21.15.43:0/3327052020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9d5802dbd0 con 0x7f9d58109a40 2024-04-25T10:32:35.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.636+0000 7f9d4effd640 1 -- 172.21.15.43:0/3327052020 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7f9d500c87a0 con 0x7f9d58109a40 2024-04-25T10:32:35.639 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T10:32:35.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.639+0000 7f9d60042640 1 -- 172.21.15.43:0/3327052020 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9d440afcf0 msgr2=0x7f9d440b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:35.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.639+0000 7f9d60042640 1 --2- 172.21.15.43:0/3327052020 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9d440afcf0 0x7f9d440b2200 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9d58104700 tx=0x7f9d54005f90 comp rx=0 tx=0).stop 2024-04-25T10:32:35.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.639+0000 7f9d60042640 1 -- 172.21.15.43:0/3327052020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d58109a40 msgr2=0x7f9d581a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:35.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.639+0000 7f9d60042640 1 --2- 172.21.15.43:0/3327052020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d58109a40 0x7f9d581a16c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009430 tx=0x7f9d500316f0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.639+0000 7f9d60042640 1 -- 172.21.15.43:0/3327052020 shutdown_connections 2024-04-25T10:32:35.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.639+0000 7f9d60042640 1 --2- 172.21.15.43:0/3327052020 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9d440afcf0 0x7f9d440b2200 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.639+0000 7f9d60042640 1 --2- 172.21.15.43:0/3327052020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d58109a40 0x7f9d581a16c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.639+0000 7f9d60042640 1 --2- 172.21.15.43:0/3327052020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d58104d40 0x7f9d5819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.639+0000 7f9d60042640 1 --2- 172.21.15.43:0/3327052020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d58103b00 0x7f9d5819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.639+0000 7f9d60042640 1 -- 172.21.15.43:0/3327052020 >> 172.21.15.43:0/3327052020 conn(0x7f9d580fecf0 msgr2=0x7f9d58105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:35.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.640+0000 7f9d60042640 1 -- 172.21.15.43:0/3327052020 shutdown_connections 2024-04-25T10:32:35.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.640+0000 7f9d60042640 1 -- 172.21.15.43:0/3327052020 wait complete. 2024-04-25T10:32:35.657 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:32:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.772+0000 7fe33d684640 1 -- 172.21.15.43:0/1302527601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3381045f0 msgr2=0x7fe338106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.772+0000 7fe33d684640 1 --2- 172.21.15.43:0/1302527601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3381045f0 0x7fe338106a10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe328009900 tx=0x7fe32802a410 comp rx=0 tx=0).stop 2024-04-25T10:32:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.772+0000 7fe33d684640 1 -- 172.21.15.43:0/1302527601 shutdown_connections 2024-04-25T10:32:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.772+0000 7fe33d684640 1 --2- 172.21.15.43:0/1302527601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe338106f50 0x7fe338109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.772+0000 7fe33d684640 1 --2- 172.21.15.43:0/1302527601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3381098b0 0x7fe33810bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.772+0000 7fe33d684640 1 --2- 172.21.15.43:0/1302527601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3381045f0 0x7fe338106a10 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.772+0000 7fe33d684640 1 -- 172.21.15.43:0/1302527601 >> 172.21.15.43:0/1302527601 conn(0x7fe3380fdbf0 msgr2=0x7fe3381000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.773+0000 7fe33d684640 1 -- 172.21.15.43:0/1302527601 shutdown_connections 2024-04-25T10:32:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.773+0000 7fe33d684640 1 -- 172.21.15.43:0/1302527601 wait complete. 2024-04-25T10:32:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.773+0000 7fe33d684640 1 Processor -- start 2024-04-25T10:32:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.773+0000 7fe33d684640 1 -- start start 2024-04-25T10:32:35.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.774+0000 7fe33d684640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3381045f0 0x7fe33819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:35.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.774+0000 7fe33d684640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe338106f50 0x7fe33819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:35.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.774+0000 7fe33d684640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3381098b0 0x7fe3381a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:35.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.774+0000 7fe33d684640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe33810f5e0 con 0x7fe3381045f0 2024-04-25T10:32:35.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.774+0000 7fe33d684640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe33810f460 con 0x7fe338106f50 2024-04-25T10:32:35.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.774+0000 7fe33d684640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe33810f760 con 0x7fe3381098b0 2024-04-25T10:32:35.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.774+0000 7fe336ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3381045f0 0x7fe33819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:35.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.774+0000 7fe336ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3381045f0 0x7fe33819a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49064/0 (socket says 172.21.15.43:49064) 2024-04-25T10:32:35.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.774+0000 7fe336ffd640 1 -- 172.21.15.43:0/236906191 learned_addr learned my addr 172.21.15.43:0/236906191 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:35.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.774+0000 7fe3377fe640 1 --2- 172.21.15.43:0/236906191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3381098b0 0x7fe3381a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:35.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.774+0000 7fe3367fc640 1 --2- 172.21.15.43:0/236906191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe338106f50 0x7fe33819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:35.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.774+0000 7fe336ffd640 1 -- 172.21.15.43:0/236906191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3381098b0 msgr2=0x7fe3381a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:35.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.774+0000 7fe336ffd640 1 --2- 172.21.15.43:0/236906191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3381098b0 0x7fe3381a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.774+0000 7fe336ffd640 1 -- 172.21.15.43:0/236906191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe338106f50 msgr2=0x7fe33819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:35.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.774+0000 7fe336ffd640 1 --2- 172.21.15.43:0/236906191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe338106f50 0x7fe33819a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:35.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.774+0000 7fe336ffd640 1 -- 172.21.15.43:0/236906191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3381a1d90 con 0x7fe3381045f0 2024-04-25T10:32:35.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.774+0000 7fe336ffd640 1 --2- 172.21.15.43:0/236906191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3381045f0 0x7fe33819a250 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe328006010 tx=0x7fe328004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:35.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.774+0000 7fe31ffff640 1 -- 172.21.15.43:0/236906191 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32803f030 con 0x7fe3381045f0 2024-04-25T10:32:35.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.775+0000 7fe31ffff640 1 -- 172.21.15.43:0/236906191 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe328003aa0 con 0x7fe3381045f0 2024-04-25T10:32:35.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.775+0000 7fe31ffff640 1 -- 172.21.15.43:0/236906191 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328035070 con 0x7fe3381045f0 2024-04-25T10:32:35.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.775+0000 7fe33d684640 1 -- 172.21.15.43:0/236906191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3381a2020 con 0x7fe3381045f0 2024-04-25T10:32:35.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.775+0000 7fe33d684640 1 -- 172.21.15.43:0/236906191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3381a2510 con 0x7fe3381045f0 2024-04-25T10:32:35.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.776+0000 7fe33d684640 1 -- 172.21.15.43:0/236906191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3380ff710 con 0x7fe3381045f0 2024-04-25T10:32:35.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.777+0000 7fe31ffff640 1 -- 172.21.15.43:0/236906191 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 147691+0+0 (secure 0 0 0) 0x7fe328042030 con 0x7fe3381045f0 2024-04-25T10:32:35.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.778+0000 7fe31ffff640 1 --2- 172.21.15.43:0/236906191 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe3200afcf0 0x7fe3200b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:35.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.778+0000 7fe31ffff640 1 -- 172.21.15.43:0/236906191 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fe3280ffc90 con 0x7fe3381045f0 2024-04-25T10:32:35.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.778+0000 7fe3367fc640 1 --2- 172.21.15.43:0/236906191 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe3200afcf0 0x7fe3200b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:35.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.779+0000 7fe3367fc640 1 --2- 172.21.15.43:0/236906191 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe3200afcf0 0x7fe3200b2200 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe33819b2f0 tx=0x7fe32c005f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:35.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:35.782+0000 7fe31ffff640 1 -- 172.21.15.43:0/236906191 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe3280c7d40 con 0x7fe3381045f0 2024-04-25T10:32:35.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:35 smithi077 ceph-mon[32772]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:32:35.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:35 smithi077 ceph-mon[32772]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T10:32:35.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:35 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:35.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:35 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:35.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:35 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3328916500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:32:35.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:35 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:35.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:35 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:35.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:35 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:35.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:35 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:35.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:35 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/4271564083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:32:35.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:35 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2982543071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:35.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:35 smithi164 ceph-mon[32713]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:32:35.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:35 smithi164 ceph-mon[32713]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T10:32:35.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:35 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:35.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:35 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:35.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:35 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3328916500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:32:35.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:35 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:35.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:35 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:35.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:35 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:35.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:35 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:35.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:35 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/4271564083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:32:35.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:35 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2982543071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:36.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.000+0000 7fe33d684640 1 -- 172.21.15.43:0/236906191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe3380637a0 con 0x7fe3381045f0 2024-04-25T10:32:36.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.001+0000 7fe31ffff640 1 -- 172.21.15.43:0/236906191 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7fe3280d0410 con 0x7fe3381045f0 2024-04-25T10:32:36.003 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T10:32:36.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.003+0000 7fe33d684640 1 -- 172.21.15.43:0/236906191 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe3200afcf0 msgr2=0x7fe3200b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:36.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.003+0000 7fe33d684640 1 --2- 172.21.15.43:0/236906191 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe3200afcf0 0x7fe3200b2200 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe33819b2f0 tx=0x7fe32c005f90 comp rx=0 tx=0).stop 2024-04-25T10:32:36.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.003+0000 7fe33d684640 1 -- 172.21.15.43:0/236906191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3381045f0 msgr2=0x7fe33819a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:36.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.003+0000 7fe33d684640 1 --2- 172.21.15.43:0/236906191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3381045f0 0x7fe33819a250 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe328006010 tx=0x7fe328004270 comp rx=0 tx=0).stop 2024-04-25T10:32:36.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.003+0000 7fe33d684640 1 -- 172.21.15.43:0/236906191 shutdown_connections 2024-04-25T10:32:36.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.003+0000 7fe33d684640 1 --2- 172.21.15.43:0/236906191 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe3200afcf0 0x7fe3200b2200 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.003+0000 7fe33d684640 1 --2- 172.21.15.43:0/236906191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe338106f50 0x7fe33819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.003+0000 7fe33d684640 1 --2- 172.21.15.43:0/236906191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3381098b0 0x7fe3381a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.003+0000 7fe33d684640 1 --2- 172.21.15.43:0/236906191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3381045f0 0x7fe33819a250 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.003+0000 7fe33d684640 1 -- 172.21.15.43:0/236906191 >> 172.21.15.43:0/236906191 conn(0x7fe3380fdbf0 msgr2=0x7fe338105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:36.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.003+0000 7fe33d684640 1 -- 172.21.15.43:0/236906191 shutdown_connections 2024-04-25T10:32:36.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.003+0000 7fe33d684640 1 -- 172.21.15.43:0/236906191 wait complete. 2024-04-25T10:32:36.018 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:32:36.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.133+0000 7fcd7d1d7640 1 -- 172.21.15.43:0/2573276786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd781029f0 msgr2=0x7fcd781036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:36.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.133+0000 7fcd7d1d7640 1 --2- 172.21.15.43:0/2573276786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd781029f0 0x7fcd781036f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009860 tx=0x7fcd6802a340 comp rx=0 tx=0).stop 2024-04-25T10:32:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.133+0000 7fcd7d1d7640 1 -- 172.21.15.43:0/2573276786 shutdown_connections 2024-04-25T10:32:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.133+0000 7fcd7d1d7640 1 --2- 172.21.15.43:0/2573276786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd78108930 0x7fcd7810ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.133+0000 7fcd7d1d7640 1 --2- 172.21.15.43:0/2573276786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd78103c30 0x7fcd78104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.133+0000 7fcd7d1d7640 1 --2- 172.21.15.43:0/2573276786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd781029f0 0x7fcd781036f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.133+0000 7fcd7d1d7640 1 -- 172.21.15.43:0/2573276786 >> 172.21.15.43:0/2573276786 conn(0x7fcd780fdbc0 msgr2=0x7fcd78100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.133+0000 7fcd7d1d7640 1 -- 172.21.15.43:0/2573276786 shutdown_connections 2024-04-25T10:32:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.133+0000 7fcd7d1d7640 1 -- 172.21.15.43:0/2573276786 wait complete. 2024-04-25T10:32:36.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.134+0000 7fcd7d1d7640 1 Processor -- start 2024-04-25T10:32:36.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.134+0000 7fcd7d1d7640 1 -- start start 2024-04-25T10:32:36.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.134+0000 7fcd7d1d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd781029f0 0x7fcd78075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:36.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.134+0000 7fcd7d1d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd78103c30 0x7fcd7819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:36.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.134+0000 7fcd7d1d7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd78108930 0x7fcd781a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:36.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.134+0000 7fcd7d1d7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7810e4f0 con 0x7fcd781029f0 2024-04-25T10:32:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.134+0000 7fcd7d1d7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7810e370 con 0x7fcd78103c30 2024-04-25T10:32:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.134+0000 7fcd7d1d7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7810e670 con 0x7fcd78108930 2024-04-25T10:32:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.134+0000 7fcd76d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd781029f0 0x7fcd78075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.134+0000 7fcd7659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd78103c30 0x7fcd7819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.134+0000 7fcd76d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd781029f0 0x7fcd78075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49084/0 (socket says 172.21.15.43:49084) 2024-04-25T10:32:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.134+0000 7fcd76d9d640 1 -- 172.21.15.43:0/867701629 learned_addr learned my addr 172.21.15.43:0/867701629 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.135+0000 7fcd7759e640 1 --2- 172.21.15.43:0/867701629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd78108930 0x7fcd781a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.135+0000 7fcd76d9d640 1 -- 172.21.15.43:0/867701629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd78108930 msgr2=0x7fcd781a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.135+0000 7fcd76d9d640 1 --2- 172.21.15.43:0/867701629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd78108930 0x7fcd781a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.135+0000 7fcd76d9d640 1 -- 172.21.15.43:0/867701629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd78103c30 msgr2=0x7fcd7819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.135+0000 7fcd76d9d640 1 --2- 172.21.15.43:0/867701629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd78103c30 0x7fcd7819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.135+0000 7fcd76d9d640 1 -- 172.21.15.43:0/867701629 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd781a1d70 con 0x7fcd781029f0 2024-04-25T10:32:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.135+0000 7fcd7659c640 1 --2- 172.21.15.43:0/867701629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd78103c30 0x7fcd7819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.135+0000 7fcd76d9d640 1 --2- 172.21.15.43:0/867701629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd781029f0 0x7fcd78075840 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcd6802a860 tx=0x7fcd68032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.135+0000 7fcd5ffff640 1 -- 172.21.15.43:0/867701629 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68037040 con 0x7fcd781029f0 2024-04-25T10:32:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.135+0000 7fcd7d1d7640 1 -- 172.21.15.43:0/867701629 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd781a2000 con 0x7fcd781029f0 2024-04-25T10:32:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.135+0000 7fcd5ffff640 1 -- 172.21.15.43:0/867701629 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd68030c10 con 0x7fcd781029f0 2024-04-25T10:32:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.135+0000 7fcd5ffff640 1 -- 172.21.15.43:0/867701629 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd680052b0 con 0x7fcd781029f0 2024-04-25T10:32:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.135+0000 7fcd7d1d7640 1 -- 172.21.15.43:0/867701629 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd781a24e0 con 0x7fcd781029f0 2024-04-25T10:32:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.136+0000 7fcd7d1d7640 1 -- 172.21.15.43:0/867701629 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd78107740 con 0x7fcd781029f0 2024-04-25T10:32:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.139+0000 7fcd5ffff640 1 -- 172.21.15.43:0/867701629 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 147691+0+0 (secure 0 0 0) 0x7fcd68043020 con 0x7fcd781029f0 2024-04-25T10:32:36.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.140+0000 7fcd5ffff640 1 --2- 172.21.15.43:0/867701629 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcd600afcf0 0x7fcd600b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:36.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.140+0000 7fcd5ffff640 1 -- 172.21.15.43:0/867701629 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fcd680fe820 con 0x7fcd781029f0 2024-04-25T10:32:36.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.140+0000 7fcd7659c640 1 --2- 172.21.15.43:0/867701629 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcd600afcf0 0x7fcd600b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:36.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.140+0000 7fcd7659c640 1 --2- 172.21.15.43:0/867701629 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcd600afcf0 0x7fcd600b2200 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcd780751e0 tx=0x7fcd6c005f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:36.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.143+0000 7fcd5ffff640 1 -- 172.21.15.43:0/867701629 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd6804a020 con 0x7fcd781029f0 2024-04-25T10:32:36.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.350+0000 7fcd7d1d7640 1 -- 172.21.15.43:0/867701629 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcd7802dce0 con 0x7fcd781029f0 2024-04-25T10:32:36.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.351+0000 7fcd5ffff640 1 -- 172.21.15.43:0/867701629 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7fcd680c2020 con 0x7fcd781029f0 2024-04-25T10:32:36.353 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T10:32:36.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.353+0000 7fcd7d1d7640 1 -- 172.21.15.43:0/867701629 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcd600afcf0 msgr2=0x7fcd600b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:36.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.353+0000 7fcd7d1d7640 1 --2- 172.21.15.43:0/867701629 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcd600afcf0 0x7fcd600b2200 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcd780751e0 tx=0x7fcd6c005f90 comp rx=0 tx=0).stop 2024-04-25T10:32:36.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.353+0000 7fcd7d1d7640 1 -- 172.21.15.43:0/867701629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd781029f0 msgr2=0x7fcd78075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:36.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.353+0000 7fcd7d1d7640 1 --2- 172.21.15.43:0/867701629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd781029f0 0x7fcd78075840 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcd6802a860 tx=0x7fcd68032d50 comp rx=0 tx=0).stop 2024-04-25T10:32:36.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.353+0000 7fcd7d1d7640 1 -- 172.21.15.43:0/867701629 shutdown_connections 2024-04-25T10:32:36.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.353+0000 7fcd7d1d7640 1 --2- 172.21.15.43:0/867701629 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcd600afcf0 0x7fcd600b2200 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.353+0000 7fcd7d1d7640 1 --2- 172.21.15.43:0/867701629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd78103c30 0x7fcd7819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.353+0000 7fcd7d1d7640 1 --2- 172.21.15.43:0/867701629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd78108930 0x7fcd781a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.353+0000 7fcd7d1d7640 1 --2- 172.21.15.43:0/867701629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd781029f0 0x7fcd78075840 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.353+0000 7fcd7d1d7640 1 -- 172.21.15.43:0/867701629 >> 172.21.15.43:0/867701629 conn(0x7fcd780fdbc0 msgr2=0x7fcd78104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:36.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.353+0000 7fcd7d1d7640 1 -- 172.21.15.43:0/867701629 shutdown_connections 2024-04-25T10:32:36.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.353+0000 7fcd7d1d7640 1 -- 172.21.15.43:0/867701629 wait complete. 2024-04-25T10:32:36.367 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-25T10:32:36.367 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:32:36.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.490+0000 7fc53740c640 1 -- 172.21.15.43:0/1622860179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5301045f0 msgr2=0x7fc530106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:36.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.490+0000 7fc53740c640 1 --2- 172.21.15.43:0/1622860179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5301045f0 0x7fc530106a10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc520009ca0 tx=0x7fc52002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.490+0000 7fc53740c640 1 -- 172.21.15.43:0/1622860179 shutdown_connections 2024-04-25T10:32:36.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.490+0000 7fc53740c640 1 --2- 172.21.15.43:0/1622860179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5301098b0 0x7fc53010bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.490+0000 7fc53740c640 1 --2- 172.21.15.43:0/1622860179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc530106f50 0x7fc530109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.490+0000 7fc53740c640 1 --2- 172.21.15.43:0/1622860179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5301045f0 0x7fc530106a10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.490+0000 7fc53740c640 1 -- 172.21.15.43:0/1622860179 >> 172.21.15.43:0/1622860179 conn(0x7fc5300fdbf0 msgr2=0x7fc5301000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:36.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.490+0000 7fc53740c640 1 -- 172.21.15.43:0/1622860179 shutdown_connections 2024-04-25T10:32:36.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.490+0000 7fc53740c640 1 -- 172.21.15.43:0/1622860179 wait complete. 2024-04-25T10:32:36.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.491+0000 7fc53740c640 1 Processor -- start 2024-04-25T10:32:36.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.491+0000 7fc53740c640 1 -- start start 2024-04-25T10:32:36.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.491+0000 7fc53740c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5301045f0 0x7fc53019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:36.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.491+0000 7fc53740c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc530106f50 0x7fc53019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:36.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.491+0000 7fc53740c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5301098b0 0x7fc5301a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:36.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.491+0000 7fc53740c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc53010f5e0 con 0x7fc530106f50 2024-04-25T10:32:36.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.491+0000 7fc53740c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc53010f460 con 0x7fc5301045f0 2024-04-25T10:32:36.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.491+0000 7fc53740c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc53010f760 con 0x7fc5301098b0 2024-04-25T10:32:36.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.491+0000 7fc5349a7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc530106f50 0x7fc53019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:36.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.491+0000 7fc5349a7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc530106f50 0x7fc53019a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49106/0 (socket says 172.21.15.43:49106) 2024-04-25T10:32:36.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.491+0000 7fc5349a7640 1 -- 172.21.15.43:0/1446860079 learned_addr learned my addr 172.21.15.43:0/1446860079 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:36.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.491+0000 7fc5351a8640 1 --2- 172.21.15.43:0/1446860079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5301045f0 0x7fc53019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:36.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.491+0000 7fc5359a9640 1 --2- 172.21.15.43:0/1446860079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5301098b0 0x7fc5301a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:36.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.492+0000 7fc5349a7640 1 -- 172.21.15.43:0/1446860079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5301098b0 msgr2=0x7fc5301a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:36.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.492+0000 7fc5349a7640 1 --2- 172.21.15.43:0/1446860079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5301098b0 0x7fc5301a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.492+0000 7fc5349a7640 1 -- 172.21.15.43:0/1446860079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5301045f0 msgr2=0x7fc53019a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:36.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.492+0000 7fc5349a7640 1 --2- 172.21.15.43:0/1446860079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5301045f0 0x7fc53019a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.492+0000 7fc5349a7640 1 -- 172.21.15.43:0/1446860079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5301a1d50 con 0x7fc530106f50 2024-04-25T10:32:36.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.492+0000 7fc5351a8640 1 --2- 172.21.15.43:0/1446860079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5301045f0 0x7fc53019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:36.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.492+0000 7fc5349a7640 1 --2- 172.21.15.43:0/1446860079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc530106f50 0x7fc53019a750 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc52400e9f0 tx=0x7fc52400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:36.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.492+0000 7fc52e7fc640 1 -- 172.21.15.43:0/1446860079 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52400cd80 con 0x7fc530106f50 2024-04-25T10:32:36.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.492+0000 7fc52e7fc640 1 -- 172.21.15.43:0/1446860079 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5240108e0 con 0x7fc530106f50 2024-04-25T10:32:36.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.492+0000 7fc53740c640 1 -- 172.21.15.43:0/1446860079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5301a1fe0 con 0x7fc530106f50 2024-04-25T10:32:36.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.492+0000 7fc52e7fc640 1 -- 172.21.15.43:0/1446860079 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524018980 con 0x7fc530106f50 2024-04-25T10:32:36.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.492+0000 7fc53740c640 1 -- 172.21.15.43:0/1446860079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5301a24d0 con 0x7fc530106f50 2024-04-25T10:32:36.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.493+0000 7fc53740c640 1 -- 172.21.15.43:0/1446860079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5300ff710 con 0x7fc530106f50 2024-04-25T10:32:36.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.495+0000 7fc52e7fc640 1 -- 172.21.15.43:0/1446860079 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 147691+0+0 (secure 0 0 0) 0x7fc524010430 con 0x7fc530106f50 2024-04-25T10:32:36.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.495+0000 7fc52e7fc640 1 --2- 172.21.15.43:0/1446860079 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc5180afcf0 0x7fc5180b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:36.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.496+0000 7fc5351a8640 1 --2- 172.21.15.43:0/1446860079 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc5180afcf0 0x7fc5180b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:36.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.496+0000 7fc52e7fc640 1 -- 172.21.15.43:0/1446860079 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fc5240df960 con 0x7fc530106f50 2024-04-25T10:32:36.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.496+0000 7fc5351a8640 1 --2- 172.21.15.43:0/1446860079 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc5180afcf0 0x7fc5180b2200 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc5200094e0 tx=0x7fc52002f370 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:36.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.499+0000 7fc52e7fc640 1 -- 172.21.15.43:0/1446860079 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc5240a7a70 con 0x7fc530106f50 2024-04-25T10:32:36.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.731+0000 7fc53740c640 1 -- 172.21.15.43:0/1446860079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc53002db50 con 0x7fc530106f50 2024-04-25T10:32:36.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.732+0000 7fc52e7fc640 1 -- 172.21.15.43:0/1446860079 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11869 (secure 0 0 0) 0x7fc5240abd70 con 0x7fc530106f50 2024-04-25T10:32:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:32:36.734 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:32:30.234841+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2595248466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2595248466},{"type":"v1","addr":"172.21.15.77:6837","nonce":2595248466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24472,24508]},"id":1}]} 2024-04-25T10:32:36.734 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 24 2024-04-25T10:32:36.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.734+0000 7fc53740c640 1 -- 172.21.15.43:0/1446860079 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc5180afcf0 msgr2=0x7fc5180b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.734+0000 7fc53740c640 1 --2- 172.21.15.43:0/1446860079 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc5180afcf0 0x7fc5180b2200 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc5200094e0 tx=0x7fc52002f370 comp rx=0 tx=0).stop 2024-04-25T10:32:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.734+0000 7fc53740c640 1 -- 172.21.15.43:0/1446860079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc530106f50 msgr2=0x7fc53019a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.734+0000 7fc53740c640 1 --2- 172.21.15.43:0/1446860079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc530106f50 0x7fc53019a750 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc52400e9f0 tx=0x7fc52400eeb0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.734+0000 7fc53740c640 1 -- 172.21.15.43:0/1446860079 shutdown_connections 2024-04-25T10:32:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.734+0000 7fc53740c640 1 --2- 172.21.15.43:0/1446860079 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc5180afcf0 0x7fc5180b2200 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.734+0000 7fc53740c640 1 --2- 172.21.15.43:0/1446860079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5301098b0 0x7fc5301a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.734+0000 7fc53740c640 1 --2- 172.21.15.43:0/1446860079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc530106f50 0x7fc53019a750 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.734+0000 7fc53740c640 1 --2- 172.21.15.43:0/1446860079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5301045f0 0x7fc53019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.734+0000 7fc53740c640 1 -- 172.21.15.43:0/1446860079 >> 172.21.15.43:0/1446860079 conn(0x7fc5300fdbf0 msgr2=0x7fc5301052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.734+0000 7fc53740c640 1 -- 172.21.15.43:0/1446860079 shutdown_connections 2024-04-25T10:32:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.734+0000 7fc53740c640 1 -- 172.21.15.43:0/1446860079 wait complete. 2024-04-25T10:32:36.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:36.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:36.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: Adjusting osd_memory_target on smithi164 to 176.2M 2024-04-25T10:32:36.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: Unable to set osd_memory_target on smithi164 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T10:32:36.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3327052020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:32:36.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:36.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:36.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.751 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.751 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.751 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-04-25T10:32:36.751 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T10:32:36.751 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:36.751 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:36.751 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.751 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.751 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.751 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.751 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.751 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.751 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.751 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:36.752 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: Adjusting osd_memory_target on smithi043 to 177.2M 2024-04-25T10:32:36.752 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: Unable to set osd_memory_target on smithi043 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T10:32:36.752 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:32:36.752 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:32:36.752 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-25T10:32:36.752 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-25T10:32:36.752 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-25T10:32:36.752 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/236906191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:32:36.752 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:36 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/867701629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:32:36.753 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-25T10:32:36.753 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-25T10:32:36.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:32:36.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.865+0000 7fcd84184640 1 -- 172.21.15.43:0/3762626208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd7c102af0 msgr2=0x7fcd7c102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:36.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.865+0000 7fcd84184640 1 --2- 172.21.15.43:0/3762626208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd7c102af0 0x7fcd7c102f00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009900 tx=0x7fcd6c02a410 comp rx=0 tx=0).stop 2024-04-25T10:32:36.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.865+0000 7fcd84184640 1 -- 172.21.15.43:0/3762626208 shutdown_connections 2024-04-25T10:32:36.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.865+0000 7fcd84184640 1 --2- 172.21.15.43:0/3762626208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd7c10d380 0x7fcd7c10f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.865+0000 7fcd84184640 1 --2- 172.21.15.43:0/3762626208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd7c102af0 0x7fcd7c102f00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.865+0000 7fcd84184640 1 --2- 172.21.15.43:0/3762626208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd7c108a20 0x7fcd7c1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.865+0000 7fcd84184640 1 -- 172.21.15.43:0/3762626208 >> 172.21.15.43:0/3762626208 conn(0x7fcd7c078110 msgr2=0x7fcd7c078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:36.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.865+0000 7fcd84184640 1 -- 172.21.15.43:0/3762626208 shutdown_connections 2024-04-25T10:32:36.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.865+0000 7fcd84184640 1 -- 172.21.15.43:0/3762626208 wait complete. 2024-04-25T10:32:36.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.866+0000 7fcd84184640 1 Processor -- start 2024-04-25T10:32:36.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.866+0000 7fcd84184640 1 -- start start 2024-04-25T10:32:36.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.866+0000 7fcd84184640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd7c102af0 0x7fcd7c19b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:36.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.866+0000 7fcd84184640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd7c108a20 0x7fcd7c19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:36.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.866+0000 7fcd84184640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd7c10d380 0x7fcd7c1a65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:36.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.866+0000 7fcd84184640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c112be0 con 0x7fcd7c108a20 2024-04-25T10:32:36.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.866+0000 7fcd84184640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c112a60 con 0x7fcd7c10d380 2024-04-25T10:32:36.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.866+0000 7fcd84184640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c112d60 con 0x7fcd7c102af0 2024-04-25T10:32:36.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.866+0000 7fcd8171f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd7c108a20 0x7fcd7c19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:36.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.866+0000 7fcd8171f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd7c108a20 0x7fcd7c19ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49116/0 (socket says 172.21.15.43:49116) 2024-04-25T10:32:36.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.866+0000 7fcd8171f640 1 -- 172.21.15.43:0/1849490823 learned_addr learned my addr 172.21.15.43:0/1849490823 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:36.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.866+0000 7fcd82721640 1 --2- 172.21.15.43:0/1849490823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd7c10d380 0x7fcd7c1a65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:36.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.867+0000 7fcd81f20640 1 --2- 172.21.15.43:0/1849490823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd7c102af0 0x7fcd7c19b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:36.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.867+0000 7fcd8171f640 1 -- 172.21.15.43:0/1849490823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd7c102af0 msgr2=0x7fcd7c19b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:36.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.867+0000 7fcd8171f640 1 --2- 172.21.15.43:0/1849490823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd7c102af0 0x7fcd7c19b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.867+0000 7fcd8171f640 1 -- 172.21.15.43:0/1849490823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd7c10d380 msgr2=0x7fcd7c1a65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:36.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.867+0000 7fcd8171f640 1 --2- 172.21.15.43:0/1849490823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd7c10d380 0x7fcd7c1a65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:36.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.867+0000 7fcd8171f640 1 -- 172.21.15.43:0/1849490823 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7c19c2e0 con 0x7fcd7c108a20 2024-04-25T10:32:36.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.867+0000 7fcd82721640 1 --2- 172.21.15.43:0/1849490823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd7c10d380 0x7fcd7c1a65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:36.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.867+0000 7fcd8171f640 1 --2- 172.21.15.43:0/1849490823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd7c108a20 0x7fcd7c19ba60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c02a930 tx=0x7fcd6c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:36.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.867+0000 7fcd72ffd640 1 -- 172.21.15.43:0/1849490823 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c03f030 con 0x7fcd7c108a20 2024-04-25T10:32:36.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.867+0000 7fcd72ffd640 1 -- 172.21.15.43:0/1849490823 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd6c003aa0 con 0x7fcd7c108a20 2024-04-25T10:32:36.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.867+0000 7fcd84184640 1 -- 172.21.15.43:0/1849490823 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7c1a6b40 con 0x7fcd7c108a20 2024-04-25T10:32:36.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.867+0000 7fcd72ffd640 1 -- 172.21.15.43:0/1849490823 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c035070 con 0x7fcd7c108a20 2024-04-25T10:32:36.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.867+0000 7fcd84184640 1 -- 172.21.15.43:0/1849490823 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd7c1a7080 con 0x7fcd7c108a20 2024-04-25T10:32:36.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.868+0000 7fcd84184640 1 -- 172.21.15.43:0/1849490823 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd7c103cd0 con 0x7fcd7c108a20 2024-04-25T10:32:36.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.870+0000 7fcd72ffd640 1 -- 172.21.15.43:0/1849490823 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 147691+0+0 (secure 0 0 0) 0x7fcd6c042030 con 0x7fcd7c108a20 2024-04-25T10:32:36.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.870+0000 7fcd72ffd640 1 --2- 172.21.15.43:0/1849490823 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcd680afd70 0x7fcd680b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:36.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.871+0000 7fcd72ffd640 1 -- 172.21.15.43:0/1849490823 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fcd6c0ff3f0 con 0x7fcd7c108a20 2024-04-25T10:32:36.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.871+0000 7fcd81f20640 1 --2- 172.21.15.43:0/1849490823 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcd680afd70 0x7fcd680b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:36.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.871+0000 7fcd81f20640 1 --2- 172.21.15.43:0/1849490823 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcd680afd70 0x7fcd680b2280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcd78005fd0 tx=0x7fcd7800b390 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:36.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.876+0000 7fcd72ffd640 1 -- 172.21.15.43:0/1849490823 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd6c0c7500 con 0x7fcd7c108a20 2024-04-25T10:32:36.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:36.922+0000 7fcd72ffd640 1 -- 172.21.15.43:0/1849490823 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 24) ==== 147747+0+0 (secure 0 0 0) 0x7fcd6c0c3790 con 0x7fcd7c108a20 2024-04-25T10:32:37.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:37.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:37.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: Adjusting osd_memory_target on smithi164 to 176.2M 2024-04-25T10:32:37.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: Unable to set osd_memory_target on smithi164 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T10:32:37.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3327052020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:32:37.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:37.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:37.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-04-25T10:32:37.043 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T10:32:37.043 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:37.043 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:37.043 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.043 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.043 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.043 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.043 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.043 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.043 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.043 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.043 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: Adjusting osd_memory_target on smithi043 to 177.2M 2024-04-25T10:32:37.043 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: Unable to set osd_memory_target on smithi043 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T10:32:37.043 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:32:37.044 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:32:37.044 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-25T10:32:37.044 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-25T10:32:37.044 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-25T10:32:37.044 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/236906191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:32:37.044 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:36 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/867701629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:32:37.044 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:37.044 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:37.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: Adjusting osd_memory_target on smithi164 to 176.2M 2024-04-25T10:32:37.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: Unable to set osd_memory_target on smithi164 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T10:32:37.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3327052020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:32:37.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:37.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:37.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-04-25T10:32:37.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T10:32:37.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:37.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:37.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.046 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:32:37.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: Adjusting osd_memory_target on smithi043 to 177.2M 2024-04-25T10:32:37.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: Unable to set osd_memory_target on smithi043 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T10:32:37.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:32:37.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:32:37.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-25T10:32:37.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-25T10:32:37.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-25T10:32:37.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/236906191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:32:37.047 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:36 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/867701629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:32:37.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.086+0000 7fcd84184640 1 -- 172.21.15.43:0/1849490823 --> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcd7c062fe0 con 0x7fcd680afd70 2024-04-25T10:32:37.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.095+0000 7fcd72ffd640 1 -- 172.21.15.43:0/1849490823 <== mgr.15291 v2:172.21.15.43:6800/3658924116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257726 (secure 0 0 0) 0x7fcd7c062fe0 con 0x7fcd680afd70 2024-04-25T10:32:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:32:37.102 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T10:32:37.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.098+0000 7fcd84184640 1 -- 172.21.15.43:0/1849490823 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcd680afd70 msgr2=0x7fcd680b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:37.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.098+0000 7fcd84184640 1 --2- 172.21.15.43:0/1849490823 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcd680afd70 0x7fcd680b2280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcd78005fd0 tx=0x7fcd7800b390 comp rx=0 tx=0).stop 2024-04-25T10:32:37.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.098+0000 7fcd84184640 1 -- 172.21.15.43:0/1849490823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd7c108a20 msgr2=0x7fcd7c19ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:37.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.098+0000 7fcd84184640 1 --2- 172.21.15.43:0/1849490823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd7c108a20 0x7fcd7c19ba60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c02a930 tx=0x7fcd6c004270 comp rx=0 tx=0).stop 2024-04-25T10:32:37.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.098+0000 7fcd84184640 1 -- 172.21.15.43:0/1849490823 shutdown_connections 2024-04-25T10:32:37.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.098+0000 7fcd84184640 1 --2- 172.21.15.43:0/1849490823 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcd680afd70 0x7fcd680b2280 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.098+0000 7fcd84184640 1 --2- 172.21.15.43:0/1849490823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd7c10d380 0x7fcd7c1a65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.098+0000 7fcd84184640 1 --2- 172.21.15.43:0/1849490823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd7c108a20 0x7fcd7c19ba60 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.098+0000 7fcd84184640 1 --2- 172.21.15.43:0/1849490823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd7c102af0 0x7fcd7c19b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.098+0000 7fcd84184640 1 -- 172.21.15.43:0/1849490823 >> 172.21.15.43:0/1849490823 conn(0x7fcd7c078110 msgr2=0x7fcd7c1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:37.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.099+0000 7fcd84184640 1 -- 172.21.15.43:0/1849490823 shutdown_connections 2024-04-25T10:32:37.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.099+0000 7fcd84184640 1 -- 172.21.15.43:0/1849490823 wait complete. 2024-04-25T10:32:37.113 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-25T10:32:36.476276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187709,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":890,"num_read_kb":583,"num_write":408,"num_write_kb":3585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":3012908,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":816,"ondisk_log_size":816,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":429720,"kb_used_data":13808,"kb_used_omap":18,"kb_used_meta":415597,"kb_avail":1124512104,"statfs":{"total":1151940427776,"available":1151500394496,"internally_reserved":0,"allocated":14139392,"data_stored":8588285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":425571641},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.983241"},"pg_stats":[{"pgid":"3.3f","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567443+0000","last_change":"2024-04-25T10:30:33.645032+0000","last_active":"2024-04-25T10:32:32.567443+0000","last_peered":"2024-04-25T10:32:32.567443+0000","last_clean":"2024-04-25T10:32:32.567443+0000","last_became_active":"2024-04-25T10:30:33.644687+0000","last_became_peered":"2024-04-25T10:30:33.644687+0000","last_unstale":"2024-04-25T10:32:32.567443+0000","last_undegraded":"2024-04-25T10:32:32.567443+0000","last_fullsized":"2024-04-25T10:32:32.567443+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:39.893228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869077+0000","last_change":"2024-04-25T10:30:33.650466+0000","last_active":"2024-04-25T10:32:32.869077+0000","last_peered":"2024-04-25T10:32:32.869077+0000","last_clean":"2024-04-25T10:32:32.869077+0000","last_became_active":"2024-04-25T10:30:33.649889+0000","last_became_peered":"2024-04-25T10:30:33.649889+0000","last_unstale":"2024-04-25T10:32:32.869077+0000","last_undegraded":"2024-04-25T10:32:32.869077+0000","last_fullsized":"2024-04-25T10:32:32.869077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:53.832494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869024+0000","last_change":"2024-04-25T10:30:33.648798+0000","last_active":"2024-04-25T10:32:32.869024+0000","last_peered":"2024-04-25T10:32:32.869024+0000","last_clean":"2024-04-25T10:32:32.869024+0000","last_became_active":"2024-04-25T10:30:33.648423+0000","last_became_peered":"2024-04-25T10:30:33.648423+0000","last_unstale":"2024-04-25T10:32:32.869024+0000","last_undegraded":"2024-04-25T10:32:32.869024+0000","last_fullsized":"2024-04-25T10:32:32.869024+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:19.119189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"79'8","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.562870+0000","last_change":"2024-04-25T10:31:33.941691+0000","last_active":"2024-04-25T10:32:32.562870+0000","last_peered":"2024-04-25T10:32:32.562870+0000","last_clean":"2024-04-25T10:32:32.562870+0000","last_became_active":"2024-04-25T10:31:33.941304+0000","last_became_peered":"2024-04-25T10:31:33.941304+0000","last_unstale":"2024-04-25T10:32:32.562870+0000","last_undegraded":"2024-04-25T10:32:32.562870+0000","last_fullsized":"2024-04-25T10:32:32.562870+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:18.573452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.564974+0000","last_change":"2024-04-25T10:30:33.646786+0000","last_active":"2024-04-25T10:32:32.564974+0000","last_peered":"2024-04-25T10:32:32.564974+0000","last_clean":"2024-04-25T10:32:32.564974+0000","last_became_active":"2024-04-25T10:30:33.646456+0000","last_became_peered":"2024-04-25T10:30:33.646456+0000","last_unstale":"2024-04-25T10:32:32.564974+0000","last_undegraded":"2024-04-25T10:32:32.564974+0000","last_fullsized":"2024-04-25T10:32:32.564974+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:55.006723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.572822+0000","last_change":"2024-04-25T10:30:33.639952+0000","last_active":"2024-04-25T10:32:32.572822+0000","last_peered":"2024-04-25T10:32:32.572822+0000","last_clean":"2024-04-25T10:32:32.572822+0000","last_became_active":"2024-04-25T10:30:33.639558+0000","last_became_peered":"2024-04-25T10:30:33.639558+0000","last_unstale":"2024-04-25T10:32:32.572822+0000","last_undegraded":"2024-04-25T10:32:32.572822+0000","last_fullsized":"2024-04-25T10:32:32.572822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:17.162902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.733860+0000","last_change":"2024-04-25T10:30:33.651530+0000","last_active":"2024-04-25T10:32:32.733860+0000","last_peered":"2024-04-25T10:32:32.733860+0000","last_clean":"2024-04-25T10:32:32.733860+0000","last_became_active":"2024-04-25T10:30:33.651151+0000","last_became_peered":"2024-04-25T10:30:33.651151+0000","last_unstale":"2024-04-25T10:32:32.733860+0000","last_undegraded":"2024-04-25T10:32:32.733860+0000","last_fullsized":"2024-04-25T10:32:32.733860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:31.835367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.461672+0000","last_change":"2024-04-25T10:30:33.645146+0000","last_active":"2024-04-25T10:32:32.461672+0000","last_peered":"2024-04-25T10:32:32.461672+0000","last_clean":"2024-04-25T10:32:32.461672+0000","last_became_active":"2024-04-25T10:30:33.644310+0000","last_became_peered":"2024-04-25T10:30:33.644310+0000","last_unstale":"2024-04-25T10:32:32.461672+0000","last_undegraded":"2024-04-25T10:32:32.461672+0000","last_fullsized":"2024-04-25T10:32:32.461672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:45.034934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462982+0000","last_change":"2024-04-25T10:30:33.649385+0000","last_active":"2024-04-25T10:32:32.462982+0000","last_peered":"2024-04-25T10:32:32.462982+0000","last_clean":"2024-04-25T10:32:32.462982+0000","last_became_active":"2024-04-25T10:30:33.649002+0000","last_became_peered":"2024-04-25T10:30:33.649002+0000","last_unstale":"2024-04-25T10:32:32.462982+0000","last_undegraded":"2024-04-25T10:32:32.462982+0000","last_fullsized":"2024-04-25T10:32:32.462982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:42.259290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"79'16","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.573098+0000","last_change":"2024-04-25T10:30:33.641918+0000","last_active":"2024-04-25T10:32:32.573098+0000","last_peered":"2024-04-25T10:32:32.573098+0000","last_clean":"2024-04-25T10:32:32.573098+0000","last_became_active":"2024-04-25T10:30:33.641553+0000","last_became_peered":"2024-04-25T10:30:33.641553+0000","last_unstale":"2024-04-25T10:32:32.573098+0000","last_undegraded":"2024-04-25T10:32:32.573098+0000","last_fullsized":"2024-04-25T10:32:32.573098+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:04.755694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734376+0000","last_change":"2024-04-25T10:30:33.638054+0000","last_active":"2024-04-25T10:32:32.734376+0000","last_peered":"2024-04-25T10:32:32.734376+0000","last_clean":"2024-04-25T10:32:32.734376+0000","last_became_active":"2024-04-25T10:30:33.637720+0000","last_became_peered":"2024-04-25T10:30:33.637720+0000","last_unstale":"2024-04-25T10:32:32.734376+0000","last_undegraded":"2024-04-25T10:32:32.734376+0000","last_fullsized":"2024-04-25T10:32:32.734376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:41.557430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734895+0000","last_change":"2024-04-25T10:30:33.642705+0000","last_active":"2024-04-25T10:32:32.734895+0000","last_peered":"2024-04-25T10:32:32.734895+0000","last_clean":"2024-04-25T10:32:32.734895+0000","last_became_active":"2024-04-25T10:30:33.641936+0000","last_became_peered":"2024-04-25T10:30:33.641936+0000","last_unstale":"2024-04-25T10:32:32.734895+0000","last_undegraded":"2024-04-25T10:32:32.734895+0000","last_fullsized":"2024-04-25T10:32:32.734895+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:36.450389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869566+0000","last_change":"2024-04-25T10:30:33.648987+0000","last_active":"2024-04-25T10:32:32.869566+0000","last_peered":"2024-04-25T10:32:32.869566+0000","last_clean":"2024-04-25T10:32:32.869566+0000","last_became_active":"2024-04-25T10:30:33.648621+0000","last_became_peered":"2024-04-25T10:30:33.648621+0000","last_unstale":"2024-04-25T10:32:32.869566+0000","last_undegraded":"2024-04-25T10:32:32.869566+0000","last_fullsized":"2024-04-25T10:32:32.869566+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:19.999002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.868610+0000","last_change":"2024-04-25T10:30:33.647360+0000","last_active":"2024-04-25T10:32:32.868610+0000","last_peered":"2024-04-25T10:32:32.868610+0000","last_clean":"2024-04-25T10:32:32.868610+0000","last_became_active":"2024-04-25T10:30:33.646719+0000","last_became_peered":"2024-04-25T10:30:33.646719+0000","last_unstale":"2024-04-25T10:32:32.868610+0000","last_undegraded":"2024-04-25T10:32:32.868610+0000","last_fullsized":"2024-04-25T10:32:32.868610+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:19.016333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.868373+0000","last_change":"2024-04-25T10:30:33.646503+0000","last_active":"2024-04-25T10:32:32.868373+0000","last_peered":"2024-04-25T10:32:32.868373+0000","last_clean":"2024-04-25T10:32:32.868373+0000","last_became_active":"2024-04-25T10:30:33.646123+0000","last_became_peered":"2024-04-25T10:30:33.646123+0000","last_unstale":"2024-04-25T10:32:32.868373+0000","last_undegraded":"2024-04-25T10:32:32.868373+0000","last_fullsized":"2024-04-25T10:32:32.868373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:57.712216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462498+0000","last_change":"2024-04-25T10:30:33.640463+0000","last_active":"2024-04-25T10:32:32.462498+0000","last_peered":"2024-04-25T10:32:32.462498+0000","last_clean":"2024-04-25T10:32:32.462498+0000","last_became_active":"2024-04-25T10:30:33.639931+0000","last_became_peered":"2024-04-25T10:30:33.639931+0000","last_unstale":"2024-04-25T10:32:32.462498+0000","last_undegraded":"2024-04-25T10:32:32.462498+0000","last_fullsized":"2024-04-25T10:32:32.462498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:18.863405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567342+0000","last_change":"2024-04-25T10:30:33.645695+0000","last_active":"2024-04-25T10:32:32.567342+0000","last_peered":"2024-04-25T10:32:32.567342+0000","last_clean":"2024-04-25T10:32:32.567342+0000","last_became_active":"2024-04-25T10:30:33.645387+0000","last_became_peered":"2024-04-25T10:30:33.645387+0000","last_unstale":"2024-04-25T10:32:32.567342+0000","last_undegraded":"2024-04-25T10:32:32.567342+0000","last_fullsized":"2024-04-25T10:32:32.567342+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:20.044456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.562870+0000","last_change":"2024-04-25T10:30:33.639087+0000","last_active":"2024-04-25T10:32:32.562870+0000","last_peered":"2024-04-25T10:32:32.562870+0000","last_clean":"2024-04-25T10:32:32.562870+0000","last_became_active":"2024-04-25T10:30:33.638726+0000","last_became_peered":"2024-04-25T10:30:33.638726+0000","last_unstale":"2024-04-25T10:32:32.562870+0000","last_undegraded":"2024-04-25T10:32:32.562870+0000","last_fullsized":"2024-04-25T10:32:32.562870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:36.887289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734245+0000","last_change":"2024-04-25T10:30:33.638646+0000","last_active":"2024-04-25T10:32:32.734245+0000","last_peered":"2024-04-25T10:32:32.734245+0000","last_clean":"2024-04-25T10:32:32.734245+0000","last_became_active":"2024-04-25T10:30:33.638261+0000","last_became_peered":"2024-04-25T10:30:33.638261+0000","last_unstale":"2024-04-25T10:32:32.734245+0000","last_undegraded":"2024-04-25T10:32:32.734245+0000","last_fullsized":"2024-04-25T10:32:32.734245+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:08.212310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.563028+0000","last_change":"2024-04-25T10:30:32.899992+0000","last_active":"2024-04-25T10:32:32.563028+0000","last_peered":"2024-04-25T10:32:32.563028+0000","last_clean":"2024-04-25T10:32:32.563028+0000","last_became_active":"2024-04-25T10:30:32.899643+0000","last_became_peered":"2024-04-25T10:30:32.899643+0000","last_unstale":"2024-04-25T10:32:32.563028+0000","last_undegraded":"2024-04-25T10:32:32.563028+0000","last_fullsized":"2024-04-25T10:32:32.563028+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:34.245121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462175+0000","last_change":"2024-04-25T10:30:33.644884+0000","last_active":"2024-04-25T10:32:32.462175+0000","last_peered":"2024-04-25T10:32:32.462175+0000","last_clean":"2024-04-25T10:32:32.462175+0000","last_became_active":"2024-04-25T10:30:33.644510+0000","last_became_peered":"2024-04-25T10:30:33.644510+0000","last_unstale":"2024-04-25T10:32:32.462175+0000","last_undegraded":"2024-04-25T10:32:32.462175+0000","last_fullsized":"2024-04-25T10:32:32.462175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:09.282865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462247+0000","last_change":"2024-04-25T10:30:32.909175+0000","last_active":"2024-04-25T10:32:32.462247+0000","last_peered":"2024-04-25T10:32:32.462247+0000","last_clean":"2024-04-25T10:32:32.462247+0000","last_became_active":"2024-04-25T10:30:32.908813+0000","last_became_peered":"2024-04-25T10:30:32.908813+0000","last_unstale":"2024-04-25T10:32:32.462247+0000","last_undegraded":"2024-04-25T10:32:32.462247+0000","last_fullsized":"2024-04-25T10:32:32.462247+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:04.256452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869245+0000","last_change":"2024-04-25T10:30:33.642095+0000","last_active":"2024-04-25T10:32:32.869245+0000","last_peered":"2024-04-25T10:32:32.869245+0000","last_clean":"2024-04-25T10:32:32.869245+0000","last_became_active":"2024-04-25T10:30:33.641713+0000","last_became_peered":"2024-04-25T10:30:33.641713+0000","last_unstale":"2024-04-25T10:32:32.869245+0000","last_undegraded":"2024-04-25T10:32:32.869245+0000","last_fullsized":"2024-04-25T10:32:32.869245+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:42.425197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.565390+0000","last_change":"2024-04-25T10:30:32.904376+0000","last_active":"2024-04-25T10:32:32.565390+0000","last_peered":"2024-04-25T10:32:32.565390+0000","last_clean":"2024-04-25T10:32:32.565390+0000","last_became_active":"2024-04-25T10:30:32.903994+0000","last_became_peered":"2024-04-25T10:30:32.903994+0000","last_unstale":"2024-04-25T10:32:32.565390+0000","last_undegraded":"2024-04-25T10:32:32.565390+0000","last_fullsized":"2024-04-25T10:32:32.565390+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:52.727800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462581+0000","last_change":"2024-04-25T10:30:33.637171+0000","last_active":"2024-04-25T10:32:32.462581+0000","last_peered":"2024-04-25T10:32:32.462581+0000","last_clean":"2024-04-25T10:32:32.462581+0000","last_became_active":"2024-04-25T10:30:33.636822+0000","last_became_peered":"2024-04-25T10:30:33.636822+0000","last_unstale":"2024-04-25T10:32:32.462581+0000","last_undegraded":"2024-04-25T10:32:32.462581+0000","last_fullsized":"2024-04-25T10:32:32.462581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:54.220784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.565240+0000","last_change":"2024-04-25T10:30:32.900856+0000","last_active":"2024-04-25T10:32:32.565240+0000","last_peered":"2024-04-25T10:32:32.565240+0000","last_clean":"2024-04-25T10:32:32.565240+0000","last_became_active":"2024-04-25T10:30:32.900519+0000","last_became_peered":"2024-04-25T10:30:32.900519+0000","last_unstale":"2024-04-25T10:32:32.565240+0000","last_undegraded":"2024-04-25T10:32:32.565240+0000","last_fullsized":"2024-04-25T10:32:32.565240+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:49.738005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463759+0000","last_change":"2024-04-25T10:30:33.651736+0000","last_active":"2024-04-25T10:32:32.463759+0000","last_peered":"2024-04-25T10:32:32.463759+0000","last_clean":"2024-04-25T10:32:32.463759+0000","last_became_active":"2024-04-25T10:30:33.651395+0000","last_became_peered":"2024-04-25T10:30:33.651395+0000","last_unstale":"2024-04-25T10:32:32.463759+0000","last_undegraded":"2024-04-25T10:32:32.463759+0000","last_fullsized":"2024-04-25T10:32:32.463759+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:58.230092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":146,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.982457+0000","last_change":"2024-04-25T10:30:32.901348+0000","last_active":"2024-04-25T10:32:32.982457+0000","last_peered":"2024-04-25T10:32:32.982457+0000","last_clean":"2024-04-25T10:32:32.982457+0000","last_became_active":"2024-04-25T10:30:32.901025+0000","last_became_peered":"2024-04-25T10:30:32.901025+0000","last_unstale":"2024-04-25T10:32:32.982457+0000","last_undegraded":"2024-04-25T10:32:32.982457+0000","last_fullsized":"2024-04-25T10:32:32.982457+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:00.521934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.565033+0000","last_change":"2024-04-25T10:30:33.649206+0000","last_active":"2024-04-25T10:32:32.565033+0000","last_peered":"2024-04-25T10:32:32.565033+0000","last_clean":"2024-04-25T10:32:32.565033+0000","last_became_active":"2024-04-25T10:30:33.648883+0000","last_became_peered":"2024-04-25T10:30:33.648883+0000","last_unstale":"2024-04-25T10:32:32.565033+0000","last_undegraded":"2024-04-25T10:32:32.565033+0000","last_fullsized":"2024-04-25T10:32:32.565033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:43.105562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.572768+0000","last_change":"2024-04-25T10:30:32.901201+0000","last_active":"2024-04-25T10:32:32.572768+0000","last_peered":"2024-04-25T10:32:32.572768+0000","last_clean":"2024-04-25T10:32:32.572768+0000","last_became_active":"2024-04-25T10:30:32.900876+0000","last_became_peered":"2024-04-25T10:30:32.900876+0000","last_unstale":"2024-04-25T10:32:32.572768+0000","last_undegraded":"2024-04-25T10:32:32.572768+0000","last_fullsized":"2024-04-25T10:32:32.572768+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:58.077045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463571+0000","last_change":"2024-04-25T10:30:33.651935+0000","last_active":"2024-04-25T10:32:32.463571+0000","last_peered":"2024-04-25T10:32:32.463571+0000","last_clean":"2024-04-25T10:32:32.463571+0000","last_became_active":"2024-04-25T10:30:33.651568+0000","last_became_peered":"2024-04-25T10:30:33.651568+0000","last_unstale":"2024-04-25T10:32:32.463571+0000","last_undegraded":"2024-04-25T10:32:32.463571+0000","last_fullsized":"2024-04-25T10:32:32.463571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:58.849535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869003+0000","last_change":"2024-04-25T10:30:32.896500+0000","last_active":"2024-04-25T10:32:32.869003+0000","last_peered":"2024-04-25T10:32:32.869003+0000","last_clean":"2024-04-25T10:32:32.869003+0000","last_became_active":"2024-04-25T10:30:32.896071+0000","last_became_peered":"2024-04-25T10:30:32.896071+0000","last_unstale":"2024-04-25T10:32:32.869003+0000","last_undegraded":"2024-04-25T10:32:32.869003+0000","last_fullsized":"2024-04-25T10:32:32.869003+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:10.385443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.461754+0000","last_change":"2024-04-25T10:30:33.631730+0000","last_active":"2024-04-25T10:32:32.461754+0000","last_peered":"2024-04-25T10:32:32.461754+0000","last_clean":"2024-04-25T10:32:32.461754+0000","last_became_active":"2024-04-25T10:30:33.631374+0000","last_became_peered":"2024-04-25T10:30:33.631374+0000","last_unstale":"2024-04-25T10:32:32.461754+0000","last_undegraded":"2024-04-25T10:32:32.461754+0000","last_fullsized":"2024-04-25T10:32:32.461754+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:14.957866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463314+0000","last_change":"2024-04-25T10:30:32.900762+0000","last_active":"2024-04-25T10:32:32.463314+0000","last_peered":"2024-04-25T10:32:32.463314+0000","last_clean":"2024-04-25T10:32:32.463314+0000","last_became_active":"2024-04-25T10:30:32.900459+0000","last_became_peered":"2024-04-25T10:30:32.900459+0000","last_unstale":"2024-04-25T10:32:32.463314+0000","last_undegraded":"2024-04-25T10:32:32.463314+0000","last_fullsized":"2024-04-25T10:32:32.463314+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:22.286261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.562715+0000","last_change":"2024-04-25T10:30:33.637727+0000","last_active":"2024-04-25T10:32:32.562715+0000","last_peered":"2024-04-25T10:32:32.562715+0000","last_clean":"2024-04-25T10:32:32.562715+0000","last_became_active":"2024-04-25T10:30:33.637359+0000","last_became_peered":"2024-04-25T10:30:33.637359+0000","last_unstale":"2024-04-25T10:32:32.562715+0000","last_undegraded":"2024-04-25T10:32:32.562715+0000","last_fullsized":"2024-04-25T10:32:32.562715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:21.877562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":141,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.733608+0000","last_change":"2024-04-25T10:30:32.894210+0000","last_active":"2024-04-25T10:32:32.733608+0000","last_peered":"2024-04-25T10:32:32.733608+0000","last_clean":"2024-04-25T10:32:32.733608+0000","last_became_active":"2024-04-25T10:30:32.893241+0000","last_became_peered":"2024-04-25T10:30:32.893241+0000","last_unstale":"2024-04-25T10:32:32.733608+0000","last_undegraded":"2024-04-25T10:32:32.733608+0000","last_fullsized":"2024-04-25T10:32:32.733608+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:06.340392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463016+0000","last_change":"2024-04-25T10:30:33.632349+0000","last_active":"2024-04-25T10:32:32.463016+0000","last_peered":"2024-04-25T10:32:32.463016+0000","last_clean":"2024-04-25T10:32:32.463016+0000","last_became_active":"2024-04-25T10:30:33.631968+0000","last_became_peered":"2024-04-25T10:30:33.631968+0000","last_unstale":"2024-04-25T10:32:32.463016+0000","last_undegraded":"2024-04-25T10:32:32.463016+0000","last_fullsized":"2024-04-25T10:32:32.463016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:29.090932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734093+0000","last_change":"2024-04-25T10:30:32.900105+0000","last_active":"2024-04-25T10:32:32.734093+0000","last_peered":"2024-04-25T10:32:32.734093+0000","last_clean":"2024-04-25T10:32:32.734093+0000","last_became_active":"2024-04-25T10:30:32.899740+0000","last_became_peered":"2024-04-25T10:30:32.899740+0000","last_unstale":"2024-04-25T10:32:32.734093+0000","last_undegraded":"2024-04-25T10:32:32.734093+0000","last_fullsized":"2024-04-25T10:32:32.734093+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:49.390799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"79'9","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869399+0000","last_change":"2024-04-25T10:31:33.943769+0000","last_active":"2024-04-25T10:32:32.869399+0000","last_peered":"2024-04-25T10:32:32.869399+0000","last_clean":"2024-04-25T10:32:32.869399+0000","last_became_active":"2024-04-25T10:31:33.943331+0000","last_became_peered":"2024-04-25T10:31:33.943331+0000","last_unstale":"2024-04-25T10:32:32.869399+0000","last_undegraded":"2024-04-25T10:32:32.869399+0000","last_fullsized":"2024-04-25T10:32:32.869399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:07.183059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463831+0000","last_change":"2024-04-25T10:30:32.893453+0000","last_active":"2024-04-25T10:32:32.463831+0000","last_peered":"2024-04-25T10:32:32.463831+0000","last_clean":"2024-04-25T10:32:32.463831+0000","last_became_active":"2024-04-25T10:30:32.893106+0000","last_became_peered":"2024-04-25T10:30:32.893106+0000","last_unstale":"2024-04-25T10:32:32.463831+0000","last_undegraded":"2024-04-25T10:32:32.463831+0000","last_fullsized":"2024-04-25T10:32:32.463831+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:57.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462772+0000","last_change":"2024-04-25T10:30:33.645693+0000","last_active":"2024-04-25T10:32:32.462772+0000","last_peered":"2024-04-25T10:32:32.462772+0000","last_clean":"2024-04-25T10:32:32.462772+0000","last_became_active":"2024-04-25T10:30:33.645342+0000","last_became_peered":"2024-04-25T10:30:33.645342+0000","last_unstale":"2024-04-25T10:32:32.462772+0000","last_undegraded":"2024-04-25T10:32:32.462772+0000","last_fullsized":"2024-04-25T10:32:32.462772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:15.729860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'35","reported_seq":317,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:33.028479+0000","last_change":"2024-04-25T10:31:33.949796+0000","last_active":"2024-04-25T10:32:33.028479+0000","last_peered":"2024-04-25T10:32:33.028479+0000","last_clean":"2024-04-25T10:32:33.028479+0000","last_became_active":"2024-04-25T10:31:33.942989+0000","last_became_peered":"2024-04-25T10:31:33.942989+0000","last_unstale":"2024-04-25T10:32:33.028479+0000","last_undegraded":"2024-04-25T10:32:33.028479+0000","last_fullsized":"2024-04-25T10:32:33.028479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:04.194643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67758,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":235,"num_read_kb":184,"num_write":36,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61046,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.565318+0000","last_change":"2024-04-25T10:30:33.641768+0000","last_active":"2024-04-25T10:32:32.565318+0000","last_peered":"2024-04-25T10:32:32.565318+0000","last_clean":"2024-04-25T10:32:32.565318+0000","last_became_active":"2024-04-25T10:30:33.641415+0000","last_became_peered":"2024-04-25T10:30:33.641415+0000","last_unstale":"2024-04-25T10:32:32.565318+0000","last_undegraded":"2024-04-25T10:32:32.565318+0000","last_fullsized":"2024-04-25T10:32:32.565318+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:38.569474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734173+0000","last_change":"2024-04-25T10:30:32.898703+0000","last_active":"2024-04-25T10:32:32.734173+0000","last_peered":"2024-04-25T10:32:32.734173+0000","last_clean":"2024-04-25T10:32:32.734173+0000","last_became_active":"2024-04-25T10:30:32.898340+0000","last_became_peered":"2024-04-25T10:30:32.898340+0000","last_unstale":"2024-04-25T10:32:32.734173+0000","last_undegraded":"2024-04-25T10:32:32.734173+0000","last_fullsized":"2024-04-25T10:32:32.734173+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:27.024144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869241+0000","last_change":"2024-04-25T10:30:33.640393+0000","last_active":"2024-04-25T10:32:32.869241+0000","last_peered":"2024-04-25T10:32:32.869241+0000","last_clean":"2024-04-25T10:32:32.869241+0000","last_became_active":"2024-04-25T10:30:33.639822+0000","last_became_peered":"2024-04-25T10:30:33.639822+0000","last_unstale":"2024-04-25T10:32:32.869241+0000","last_undegraded":"2024-04-25T10:32:32.869241+0000","last_fullsized":"2024-04-25T10:32:32.869241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:08.618406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463750+0000","last_change":"2024-04-25T10:30:32.900880+0000","last_active":"2024-04-25T10:32:32.463750+0000","last_peered":"2024-04-25T10:32:32.463750+0000","last_clean":"2024-04-25T10:32:32.463750+0000","last_became_active":"2024-04-25T10:30:32.899987+0000","last_became_peered":"2024-04-25T10:30:32.899987+0000","last_unstale":"2024-04-25T10:32:32.463750+0000","last_undegraded":"2024-04-25T10:32:32.463750+0000","last_fullsized":"2024-04-25T10:32:32.463750+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:35.687773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734321+0000","last_change":"2024-04-25T10:30:33.641933+0000","last_active":"2024-04-25T10:32:32.734321+0000","last_peered":"2024-04-25T10:32:32.734321+0000","last_clean":"2024-04-25T10:32:32.734321+0000","last_became_active":"2024-04-25T10:30:33.641549+0000","last_became_peered":"2024-04-25T10:30:33.641549+0000","last_unstale":"2024-04-25T10:32:32.734321+0000","last_undegraded":"2024-04-25T10:32:32.734321+0000","last_fullsized":"2024-04-25T10:32:32.734321+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:36.696523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'5","reported_seq":145,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734534+0000","last_change":"2024-04-25T10:30:32.901178+0000","last_active":"2024-04-25T10:32:32.734534+0000","last_peered":"2024-04-25T10:32:32.734534+0000","last_clean":"2024-04-25T10:32:32.734534+0000","last_became_active":"2024-04-25T10:30:32.900817+0000","last_became_peered":"2024-04-25T10:30:32.900817+0000","last_unstale":"2024-04-25T10:32:32.734534+0000","last_undegraded":"2024-04-25T10:32:32.734534+0000","last_fullsized":"2024-04-25T10:32:32.734534+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:53.999253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"79'6","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.733811+0000","last_change":"2024-04-25T10:31:33.943470+0000","last_active":"2024-04-25T10:32:32.733811+0000","last_peered":"2024-04-25T10:32:32.733811+0000","last_clean":"2024-04-25T10:32:32.733811+0000","last_became_active":"2024-04-25T10:31:33.943098+0000","last_became_peered":"2024-04-25T10:31:33.943098+0000","last_unstale":"2024-04-25T10:32:32.733811+0000","last_undegraded":"2024-04-25T10:32:32.733811+0000","last_fullsized":"2024-04-25T10:32:32.733811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:27.707195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.564919+0000","last_change":"2024-04-25T10:30:32.902862+0000","last_active":"2024-04-25T10:32:32.564919+0000","last_peered":"2024-04-25T10:32:32.564919+0000","last_clean":"2024-04-25T10:32:32.564919+0000","last_became_active":"2024-04-25T10:30:32.902537+0000","last_became_peered":"2024-04-25T10:30:32.902537+0000","last_unstale":"2024-04-25T10:32:32.564919+0000","last_undegraded":"2024-04-25T10:32:32.564919+0000","last_fullsized":"2024-04-25T10:32:32.564919+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:55.974128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463465+0000","last_change":"2024-04-25T10:30:33.651409+0000","last_active":"2024-04-25T10:32:32.463465+0000","last_peered":"2024-04-25T10:32:32.463465+0000","last_clean":"2024-04-25T10:32:32.463465+0000","last_became_active":"2024-04-25T10:30:33.650673+0000","last_became_peered":"2024-04-25T10:30:33.650673+0000","last_unstale":"2024-04-25T10:32:32.463465+0000","last_undegraded":"2024-04-25T10:32:32.463465+0000","last_fullsized":"2024-04-25T10:32:32.463465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:48.989198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567343+0000","last_change":"2024-04-25T10:31:33.953408+0000","last_active":"2024-04-25T10:32:32.567343+0000","last_peered":"2024-04-25T10:32:32.567343+0000","last_clean":"2024-04-25T10:32:32.567343+0000","last_became_active":"2024-04-25T10:31:33.943495+0000","last_became_peered":"2024-04-25T10:31:33.943495+0000","last_unstale":"2024-04-25T10:32:32.567343+0000","last_undegraded":"2024-04-25T10:32:32.567343+0000","last_fullsized":"2024-04-25T10:32:32.567343+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:03.947967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567280+0000","last_change":"2024-04-25T10:30:33.643929+0000","last_active":"2024-04-25T10:32:32.567280+0000","last_peered":"2024-04-25T10:32:32.567280+0000","last_clean":"2024-04-25T10:32:32.567280+0000","last_became_active":"2024-04-25T10:30:33.643593+0000","last_became_peered":"2024-04-25T10:30:33.643593+0000","last_unstale":"2024-04-25T10:32:32.567280+0000","last_undegraded":"2024-04-25T10:32:32.567280+0000","last_fullsized":"2024-04-25T10:32:32.567280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:39.591022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.562806+0000","last_change":"2024-04-25T10:30:32.898937+0000","last_active":"2024-04-25T10:32:32.562806+0000","last_peered":"2024-04-25T10:32:32.562806+0000","last_clean":"2024-04-25T10:32:32.562806+0000","last_became_active":"2024-04-25T10:30:32.898570+0000","last_became_peered":"2024-04-25T10:30:32.898570+0000","last_unstale":"2024-04-25T10:32:32.562806+0000","last_undegraded":"2024-04-25T10:32:32.562806+0000","last_fullsized":"2024-04-25T10:32:32.562806+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:40.123035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.573011+0000","last_change":"2024-04-25T10:30:33.632857+0000","last_active":"2024-04-25T10:32:32.573011+0000","last_peered":"2024-04-25T10:32:32.573011+0000","last_clean":"2024-04-25T10:32:32.573011+0000","last_became_active":"2024-04-25T10:30:33.632043+0000","last_became_peered":"2024-04-25T10:30:33.632043+0000","last_unstale":"2024-04-25T10:32:32.573011+0000","last_undegraded":"2024-04-25T10:32:32.573011+0000","last_fullsized":"2024-04-25T10:32:32.573011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:12.683035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734027+0000","last_change":"2024-04-25T10:30:32.901349+0000","last_active":"2024-04-25T10:32:32.734027+0000","last_peered":"2024-04-25T10:32:32.734027+0000","last_clean":"2024-04-25T10:32:32.734027+0000","last_became_active":"2024-04-25T10:30:32.900881+0000","last_became_peered":"2024-04-25T10:30:32.900881+0000","last_unstale":"2024-04-25T10:32:32.734027+0000","last_undegraded":"2024-04-25T10:32:32.734027+0000","last_fullsized":"2024-04-25T10:32:32.734027+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:34.963564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.572921+0000","last_change":"2024-04-25T10:30:33.644376+0000","last_active":"2024-04-25T10:32:32.572921+0000","last_peered":"2024-04-25T10:32:32.572921+0000","last_clean":"2024-04-25T10:32:32.572921+0000","last_became_active":"2024-04-25T10:30:33.644055+0000","last_became_peered":"2024-04-25T10:30:33.644055+0000","last_unstale":"2024-04-25T10:32:32.572921+0000","last_undegraded":"2024-04-25T10:32:32.572921+0000","last_fullsized":"2024-04-25T10:32:32.572921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:48.134775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"79'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463526+0000","last_change":"2024-04-25T10:30:33.646312+0000","last_active":"2024-04-25T10:32:32.463526+0000","last_peered":"2024-04-25T10:32:32.463526+0000","last_clean":"2024-04-25T10:32:32.463526+0000","last_became_active":"2024-04-25T10:30:33.642338+0000","last_became_peered":"2024-04-25T10:30:33.642338+0000","last_unstale":"2024-04-25T10:32:32.463526+0000","last_undegraded":"2024-04-25T10:32:32.463526+0000","last_fullsized":"2024-04-25T10:32:32.463526+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:42.588360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":141,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.868833+0000","last_change":"2024-04-25T10:30:32.909680+0000","last_active":"2024-04-25T10:32:32.868833+0000","last_peered":"2024-04-25T10:32:32.868833+0000","last_clean":"2024-04-25T10:32:32.868833+0000","last_became_active":"2024-04-25T10:30:32.909169+0000","last_became_peered":"2024-04-25T10:30:32.909169+0000","last_unstale":"2024-04-25T10:32:32.868833+0000","last_undegraded":"2024-04-25T10:32:32.868833+0000","last_fullsized":"2024-04-25T10:32:32.868833+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:54.876475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.461954+0000","last_change":"2024-04-25T10:30:33.649524+0000","last_active":"2024-04-25T10:32:32.461954+0000","last_peered":"2024-04-25T10:32:32.461954+0000","last_clean":"2024-04-25T10:32:32.461954+0000","last_became_active":"2024-04-25T10:30:33.648958+0000","last_became_peered":"2024-04-25T10:30:33.648958+0000","last_unstale":"2024-04-25T10:32:32.461954+0000","last_undegraded":"2024-04-25T10:32:32.461954+0000","last_fullsized":"2024-04-25T10:32:32.461954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:13.407742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734705+0000","last_change":"2024-04-25T10:30:32.895270+0000","last_active":"2024-04-25T10:32:32.734705+0000","last_peered":"2024-04-25T10:32:32.734705+0000","last_clean":"2024-04-25T10:32:32.734705+0000","last_became_active":"2024-04-25T10:30:32.894899+0000","last_became_peered":"2024-04-25T10:30:32.894899+0000","last_unstale":"2024-04-25T10:32:32.734705+0000","last_undegraded":"2024-04-25T10:32:32.734705+0000","last_fullsized":"2024-04-25T10:32:32.734705+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:21.233217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.733901+0000","last_change":"2024-04-25T10:29:21.105073+0000","last_active":"2024-04-25T10:32:32.733901+0000","last_peered":"2024-04-25T10:32:32.733901+0000","last_clean":"2024-04-25T10:32:32.733901+0000","last_became_active":"2024-04-25T10:29:21.098771+0000","last_became_peered":"2024-04-25T10:29:21.098771+0000","last_unstale":"2024-04-25T10:32:32.733901+0000","last_undegraded":"2024-04-25T10:32:32.733901+0000","last_fullsized":"2024-04-25T10:32:32.733901+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:59.015440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:59.015440+0000","last_clean_scrub_stamp":"2024-04-25T10:25:59.015440+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:41.728977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.868619+0000","last_change":"2024-04-25T10:30:33.639159+0000","last_active":"2024-04-25T10:32:32.868619+0000","last_peered":"2024-04-25T10:32:32.868619+0000","last_clean":"2024-04-25T10:32:32.868619+0000","last_became_active":"2024-04-25T10:30:33.638761+0000","last_became_peered":"2024-04-25T10:30:33.638761+0000","last_unstale":"2024-04-25T10:32:32.868619+0000","last_undegraded":"2024-04-25T10:32:32.868619+0000","last_fullsized":"2024-04-25T10:32:32.868619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:43.262103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734611+0000","last_change":"2024-04-25T10:30:32.898587+0000","last_active":"2024-04-25T10:32:32.734611+0000","last_peered":"2024-04-25T10:32:32.734611+0000","last_clean":"2024-04-25T10:32:32.734611+0000","last_became_active":"2024-04-25T10:30:32.898113+0000","last_became_peered":"2024-04-25T10:30:32.898113+0000","last_unstale":"2024-04-25T10:32:32.734611+0000","last_undegraded":"2024-04-25T10:32:32.734611+0000","last_fullsized":"2024-04-25T10:32:32.734611+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:24.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.565148+0000","last_change":"2024-04-25T10:30:32.907188+0000","last_active":"2024-04-25T10:32:32.565148+0000","last_peered":"2024-04-25T10:32:32.565148+0000","last_clean":"2024-04-25T10:32:32.565148+0000","last_became_active":"2024-04-25T10:30:32.906837+0000","last_became_peered":"2024-04-25T10:30:32.906837+0000","last_unstale":"2024-04-25T10:32:32.565148+0000","last_undegraded":"2024-04-25T10:32:32.565148+0000","last_fullsized":"2024-04-25T10:32:32.565148+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:11.835053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869498+0000","last_change":"2024-04-25T10:30:33.636159+0000","last_active":"2024-04-25T10:32:32.869498+0000","last_peered":"2024-04-25T10:32:32.869498+0000","last_clean":"2024-04-25T10:32:32.869498+0000","last_became_active":"2024-04-25T10:30:33.635098+0000","last_became_peered":"2024-04-25T10:30:33.635098+0000","last_unstale":"2024-04-25T10:32:32.869498+0000","last_undegraded":"2024-04-25T10:32:32.869498+0000","last_fullsized":"2024-04-25T10:32:32.869498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:13.092368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'45","reported_seq":89,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.689001+0000","last_change":"2024-04-25T10:30:32.905469+0000","last_active":"2024-04-25T10:32:32.689001+0000","last_peered":"2024-04-25T10:32:32.689001+0000","last_clean":"2024-04-25T10:32:32.689001+0000","last_became_active":"2024-04-25T10:30:32.905123+0000","last_became_peered":"2024-04-25T10:30:32.905123+0000","last_unstale":"2024-04-25T10:32:32.689001+0000","last_undegraded":"2024-04-25T10:32:32.689001+0000","last_fullsized":"2024-04-25T10:32:32.689001+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:12.884809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462239+0000","last_change":"2024-04-25T10:30:33.640690+0000","last_active":"2024-04-25T10:32:32.462239+0000","last_peered":"2024-04-25T10:32:32.462239+0000","last_clean":"2024-04-25T10:32:32.462239+0000","last_became_active":"2024-04-25T10:30:33.640310+0000","last_became_peered":"2024-04-25T10:30:33.640310+0000","last_unstale":"2024-04-25T10:32:32.462239+0000","last_undegraded":"2024-04-25T10:32:32.462239+0000","last_fullsized":"2024-04-25T10:32:32.462239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:03.671495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734684+0000","last_change":"2024-04-25T10:30:32.893743+0000","last_active":"2024-04-25T10:32:32.734684+0000","last_peered":"2024-04-25T10:32:32.734684+0000","last_clean":"2024-04-25T10:32:32.734684+0000","last_became_active":"2024-04-25T10:30:32.893360+0000","last_became_peered":"2024-04-25T10:30:32.893360+0000","last_unstale":"2024-04-25T10:32:32.734684+0000","last_undegraded":"2024-04-25T10:32:32.734684+0000","last_fullsized":"2024-04-25T10:32:32.734684+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:39.830376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869026+0000","last_change":"2024-04-25T10:30:33.645127+0000","last_active":"2024-04-25T10:32:32.869026+0000","last_peered":"2024-04-25T10:32:32.869026+0000","last_clean":"2024-04-25T10:32:32.869026+0000","last_became_active":"2024-04-25T10:30:33.644259+0000","last_became_peered":"2024-04-25T10:30:33.644259+0000","last_unstale":"2024-04-25T10:32:32.869026+0000","last_undegraded":"2024-04-25T10:32:32.869026+0000","last_fullsized":"2024-04-25T10:32:32.869026+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:56.849861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462504+0000","last_change":"2024-04-25T10:30:32.896885+0000","last_active":"2024-04-25T10:32:32.462504+0000","last_peered":"2024-04-25T10:32:32.462504+0000","last_clean":"2024-04-25T10:32:32.462504+0000","last_became_active":"2024-04-25T10:30:32.896447+0000","last_became_peered":"2024-04-25T10:30:32.896447+0000","last_unstale":"2024-04-25T10:32:32.462504+0000","last_undegraded":"2024-04-25T10:32:32.462504+0000","last_fullsized":"2024-04-25T10:32:32.462504+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:13.133549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"79'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.564914+0000","last_change":"2024-04-25T10:30:33.645016+0000","last_active":"2024-04-25T10:32:32.564914+0000","last_peered":"2024-04-25T10:32:32.564914+0000","last_clean":"2024-04-25T10:32:32.564914+0000","last_became_active":"2024-04-25T10:30:33.644661+0000","last_became_peered":"2024-04-25T10:30:33.644661+0000","last_unstale":"2024-04-25T10:32:32.564914+0000","last_undegraded":"2024-04-25T10:32:32.564914+0000","last_fullsized":"2024-04-25T10:32:32.564914+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:20.935124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.461752+0000","last_change":"2024-04-25T10:30:32.907827+0000","last_active":"2024-04-25T10:32:32.461752+0000","last_peered":"2024-04-25T10:32:32.461752+0000","last_clean":"2024-04-25T10:32:32.461752+0000","last_became_active":"2024-04-25T10:30:32.907402+0000","last_became_peered":"2024-04-25T10:30:32.907402+0000","last_unstale":"2024-04-25T10:32:32.461752+0000","last_undegraded":"2024-04-25T10:32:32.461752+0000","last_fullsized":"2024-04-25T10:32:32.461752+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:38.501617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567040+0000","last_change":"2024-04-25T10:30:33.636935+0000","last_active":"2024-04-25T10:32:32.567040+0000","last_peered":"2024-04-25T10:32:32.567040+0000","last_clean":"2024-04-25T10:32:32.567040+0000","last_became_active":"2024-04-25T10:30:33.636566+0000","last_became_peered":"2024-04-25T10:30:33.636566+0000","last_unstale":"2024-04-25T10:32:32.567040+0000","last_undegraded":"2024-04-25T10:32:32.567040+0000","last_fullsized":"2024-04-25T10:32:32.567040+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:24.556275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'2","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567152+0000","last_change":"2024-04-25T10:31:33.946786+0000","last_active":"2024-04-25T10:32:32.567152+0000","last_peered":"2024-04-25T10:32:32.567152+0000","last_clean":"2024-04-25T10:32:32.567152+0000","last_became_active":"2024-04-25T10:31:33.941298+0000","last_became_peered":"2024-04-25T10:31:33.941298+0000","last_unstale":"2024-04-25T10:32:32.567152+0000","last_undegraded":"2024-04-25T10:32:32.567152+0000","last_fullsized":"2024-04-25T10:32:32.567152+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:55.009673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.461836+0000","last_change":"2024-04-25T10:30:33.642986+0000","last_active":"2024-04-25T10:32:32.461836+0000","last_peered":"2024-04-25T10:32:32.461836+0000","last_clean":"2024-04-25T10:32:32.461836+0000","last_became_active":"2024-04-25T10:30:33.642545+0000","last_became_peered":"2024-04-25T10:30:33.642545+0000","last_unstale":"2024-04-25T10:32:32.461836+0000","last_undegraded":"2024-04-25T10:32:32.461836+0000","last_fullsized":"2024-04-25T10:32:32.461836+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:49.369648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.565008+0000","last_change":"2024-04-25T10:30:32.902239+0000","last_active":"2024-04-25T10:32:32.565008+0000","last_peered":"2024-04-25T10:32:32.565008+0000","last_clean":"2024-04-25T10:32:32.565008+0000","last_became_active":"2024-04-25T10:30:32.901653+0000","last_became_peered":"2024-04-25T10:30:32.901653+0000","last_unstale":"2024-04-25T10:32:32.565008+0000","last_undegraded":"2024-04-25T10:32:32.565008+0000","last_fullsized":"2024-04-25T10:32:32.565008+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:34.732428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734130+0000","last_change":"2024-04-25T10:30:33.639763+0000","last_active":"2024-04-25T10:32:32.734130+0000","last_peered":"2024-04-25T10:32:32.734130+0000","last_clean":"2024-04-25T10:32:32.734130+0000","last_became_active":"2024-04-25T10:30:33.638004+0000","last_became_peered":"2024-04-25T10:30:33.638004+0000","last_unstale":"2024-04-25T10:32:32.734130+0000","last_undegraded":"2024-04-25T10:32:32.734130+0000","last_fullsized":"2024-04-25T10:32:32.734130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:09.722333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734268+0000","last_change":"2024-04-25T10:30:32.894823+0000","last_active":"2024-04-25T10:32:32.734268+0000","last_peered":"2024-04-25T10:32:32.734268+0000","last_clean":"2024-04-25T10:32:32.734268+0000","last_became_active":"2024-04-25T10:30:32.893819+0000","last_became_peered":"2024-04-25T10:30:32.893819+0000","last_unstale":"2024-04-25T10:32:32.734268+0000","last_undegraded":"2024-04-25T10:32:32.734268+0000","last_fullsized":"2024-04-25T10:32:32.734268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:59.809410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734513+0000","last_change":"2024-04-25T10:30:33.637046+0000","last_active":"2024-04-25T10:32:32.734513+0000","last_peered":"2024-04-25T10:32:32.734513+0000","last_clean":"2024-04-25T10:32:32.734513+0000","last_became_active":"2024-04-25T10:30:33.634903+0000","last_became_peered":"2024-04-25T10:30:33.634903+0000","last_unstale":"2024-04-25T10:32:32.734513+0000","last_undegraded":"2024-04-25T10:32:32.734513+0000","last_fullsized":"2024-04-25T10:32:32.734513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:52.252812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'39","reported_seq":68,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.683867+0000","last_change":"2024-04-25T10:30:32.898979+0000","last_active":"2024-04-25T10:32:32.683867+0000","last_peered":"2024-04-25T10:32:32.683867+0000","last_clean":"2024-04-25T10:32:32.683867+0000","last_became_active":"2024-04-25T10:30:32.898611+0000","last_became_peered":"2024-04-25T10:30:32.898611+0000","last_unstale":"2024-04-25T10:32:32.683867+0000","last_undegraded":"2024-04-25T10:32:32.683867+0000","last_fullsized":"2024-04-25T10:32:32.683867+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.549769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734130+0000","last_change":"2024-04-25T10:30:33.651596+0000","last_active":"2024-04-25T10:32:32.734130+0000","last_peered":"2024-04-25T10:32:32.734130+0000","last_clean":"2024-04-25T10:32:32.734130+0000","last_became_active":"2024-04-25T10:30:33.651140+0000","last_became_peered":"2024-04-25T10:30:33.651140+0000","last_unstale":"2024-04-25T10:32:32.734130+0000","last_undegraded":"2024-04-25T10:32:32.734130+0000","last_fullsized":"2024-04-25T10:32:32.734130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:16.762135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869451+0000","last_change":"2024-04-25T10:30:32.909982+0000","last_active":"2024-04-25T10:32:32.869451+0000","last_peered":"2024-04-25T10:32:32.869451+0000","last_clean":"2024-04-25T10:32:32.869451+0000","last_became_active":"2024-04-25T10:30:32.909506+0000","last_became_peered":"2024-04-25T10:30:32.909506+0000","last_unstale":"2024-04-25T10:32:32.869451+0000","last_undegraded":"2024-04-25T10:32:32.869451+0000","last_fullsized":"2024-04-25T10:32:32.869451+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:47.396963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734492+0000","last_change":"2024-04-25T10:30:33.642913+0000","last_active":"2024-04-25T10:32:32.734492+0000","last_peered":"2024-04-25T10:32:32.734492+0000","last_clean":"2024-04-25T10:32:32.734492+0000","last_became_active":"2024-04-25T10:30:33.642614+0000","last_became_peered":"2024-04-25T10:30:33.642614+0000","last_unstale":"2024-04-25T10:32:32.734492+0000","last_undegraded":"2024-04-25T10:32:32.734492+0000","last_fullsized":"2024-04-25T10:32:32.734492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:56.651230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567112+0000","last_change":"2024-04-25T10:30:33.644525+0000","last_active":"2024-04-25T10:32:32.567112+0000","last_peered":"2024-04-25T10:32:32.567112+0000","last_clean":"2024-04-25T10:32:32.567112+0000","last_became_active":"2024-04-25T10:30:33.644188+0000","last_became_peered":"2024-04-25T10:30:33.644188+0000","last_unstale":"2024-04-25T10:32:32.567112+0000","last_undegraded":"2024-04-25T10:32:32.567112+0000","last_fullsized":"2024-04-25T10:32:32.567112+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:28.625695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"79'12","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734197+0000","last_change":"2024-04-25T10:31:33.946016+0000","last_active":"2024-04-25T10:32:32.734197+0000","last_peered":"2024-04-25T10:32:32.734197+0000","last_clean":"2024-04-25T10:32:32.734197+0000","last_became_active":"2024-04-25T10:31:33.945611+0000","last_became_peered":"2024-04-25T10:31:33.945611+0000","last_unstale":"2024-04-25T10:32:32.734197+0000","last_undegraded":"2024-04-25T10:32:32.734197+0000","last_fullsized":"2024-04-25T10:32:32.734197+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:02.352885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734068+0000","last_change":"2024-04-25T10:30:33.644721+0000","last_active":"2024-04-25T10:32:32.734068+0000","last_peered":"2024-04-25T10:32:32.734068+0000","last_clean":"2024-04-25T10:32:32.734068+0000","last_became_active":"2024-04-25T10:30:33.644392+0000","last_became_peered":"2024-04-25T10:30:33.644392+0000","last_unstale":"2024-04-25T10:32:32.734068+0000","last_undegraded":"2024-04-25T10:32:32.734068+0000","last_fullsized":"2024-04-25T10:32:32.734068+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:50.571318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.868724+0000","last_change":"2024-04-25T10:30:33.647970+0000","last_active":"2024-04-25T10:32:32.868724+0000","last_peered":"2024-04-25T10:32:32.868724+0000","last_clean":"2024-04-25T10:32:32.868724+0000","last_became_active":"2024-04-25T10:30:33.647547+0000","last_became_peered":"2024-04-25T10:30:33.647547+0000","last_unstale":"2024-04-25T10:32:32.868724+0000","last_undegraded":"2024-04-25T10:32:32.868724+0000","last_fullsized":"2024-04-25T10:32:32.868724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:25.609085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869336+0000","last_change":"2024-04-25T10:30:33.641618+0000","last_active":"2024-04-25T10:32:32.869336+0000","last_peered":"2024-04-25T10:32:32.869336+0000","last_clean":"2024-04-25T10:32:32.869336+0000","last_became_active":"2024-04-25T10:30:33.640830+0000","last_became_peered":"2024-04-25T10:30:33.640830+0000","last_unstale":"2024-04-25T10:32:32.869336+0000","last_undegraded":"2024-04-25T10:32:32.869336+0000","last_fullsized":"2024-04-25T10:32:32.869336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:36.870127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567255+0000","last_change":"2024-04-25T10:30:33.644010+0000","last_active":"2024-04-25T10:32:32.567255+0000","last_peered":"2024-04-25T10:32:32.567255+0000","last_clean":"2024-04-25T10:32:32.567255+0000","last_became_active":"2024-04-25T10:30:33.643639+0000","last_became_peered":"2024-04-25T10:30:33.643639+0000","last_unstale":"2024-04-25T10:32:32.567255+0000","last_undegraded":"2024-04-25T10:32:32.567255+0000","last_fullsized":"2024-04-25T10:32:32.567255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:32.820552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734555+0000","last_change":"2024-04-25T10:30:33.639727+0000","last_active":"2024-04-25T10:32:32.734555+0000","last_peered":"2024-04-25T10:32:32.734555+0000","last_clean":"2024-04-25T10:32:32.734555+0000","last_became_active":"2024-04-25T10:30:33.639185+0000","last_became_peered":"2024-04-25T10:30:33.639185+0000","last_unstale":"2024-04-25T10:32:32.734555+0000","last_undegraded":"2024-04-25T10:32:32.734555+0000","last_fullsized":"2024-04-25T10:32:32.734555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:17.860864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"79'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462081+0000","last_change":"2024-04-25T10:30:33.644650+0000","last_active":"2024-04-25T10:32:32.462081+0000","last_peered":"2024-04-25T10:32:32.462081+0000","last_clean":"2024-04-25T10:32:32.462081+0000","last_became_active":"2024-04-25T10:30:33.644264+0000","last_became_peered":"2024-04-25T10:30:33.644264+0000","last_unstale":"2024-04-25T10:32:32.462081+0000","last_undegraded":"2024-04-25T10:32:32.462081+0000","last_fullsized":"2024-04-25T10:32:32.462081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:41.122318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463411+0000","last_change":"2024-04-25T10:30:33.651875+0000","last_active":"2024-04-25T10:32:32.463411+0000","last_peered":"2024-04-25T10:32:32.463411+0000","last_clean":"2024-04-25T10:32:32.463411+0000","last_became_active":"2024-04-25T10:30:33.651486+0000","last_became_peered":"2024-04-25T10:30:33.651486+0000","last_unstale":"2024-04-25T10:32:32.463411+0000","last_undegraded":"2024-04-25T10:32:32.463411+0000","last_fullsized":"2024-04-25T10:32:32.463411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:32.598209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"79'8","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.562510+0000","last_change":"2024-04-25T10:31:33.942171+0000","last_active":"2024-04-25T10:32:32.562510+0000","last_peered":"2024-04-25T10:32:32.562510+0000","last_clean":"2024-04-25T10:32:32.562510+0000","last_became_active":"2024-04-25T10:31:33.941684+0000","last_became_peered":"2024-04-25T10:31:33.941684+0000","last_unstale":"2024-04-25T10:32:32.562510+0000","last_undegraded":"2024-04-25T10:32:32.562510+0000","last_fullsized":"2024-04-25T10:32:32.562510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:45.291952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462705+0000","last_change":"2024-04-25T10:30:33.639915+0000","last_active":"2024-04-25T10:32:32.462705+0000","last_peered":"2024-04-25T10:32:32.462705+0000","last_clean":"2024-04-25T10:32:32.462705+0000","last_became_active":"2024-04-25T10:30:33.639137+0000","last_became_peered":"2024-04-25T10:30:33.639137+0000","last_unstale":"2024-04-25T10:32:32.462705+0000","last_undegraded":"2024-04-25T10:32:32.462705+0000","last_fullsized":"2024-04-25T10:32:32.462705+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:13.143405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869541+0000","last_change":"2024-04-25T10:30:33.643547+0000","last_active":"2024-04-25T10:32:32.869541+0000","last_peered":"2024-04-25T10:32:32.869541+0000","last_clean":"2024-04-25T10:32:32.869541+0000","last_became_active":"2024-04-25T10:30:33.643158+0000","last_became_peered":"2024-04-25T10:30:33.643158+0000","last_unstale":"2024-04-25T10:32:32.869541+0000","last_undegraded":"2024-04-25T10:32:32.869541+0000","last_fullsized":"2024-04-25T10:32:32.869541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:07.517132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567168+0000","last_change":"2024-04-25T10:30:33.645746+0000","last_active":"2024-04-25T10:32:32.567168+0000","last_peered":"2024-04-25T10:32:32.567168+0000","last_clean":"2024-04-25T10:32:32.567168+0000","last_became_active":"2024-04-25T10:30:33.644646+0000","last_became_peered":"2024-04-25T10:30:33.644646+0000","last_unstale":"2024-04-25T10:32:32.567168+0000","last_undegraded":"2024-04-25T10:32:32.567168+0000","last_fullsized":"2024-04-25T10:32:32.567168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:21.591264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597107,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":809,"num_read_kb":514,"num_write":283,"num_write_kb":1433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":61068,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1791321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645350,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":856064,"data_stored":413481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841578,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37004,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708148,"statfs":{"total":95995035648,"available":95957143552,"internally_reserved":0,"allocated":1708032,"data_stored":1213757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037807,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36588,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708564,"statfs":{"total":95995035648,"available":95957569536,"internally_reserved":0,"allocated":1277952,"data_stored":815838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234036,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36584,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708568,"statfs":{"total":95995035648,"available":95957573632,"internally_reserved":0,"allocated":1273856,"data_stored":815736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430266,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712528,"statfs":{"total":95995035648,"available":95961628672,"internally_reserved":0,"allocated":1478656,"data_stored":1015004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626493,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":856064,"data_stored":413084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":942080,"data_stored":489562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40352,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704800,"statfs":{"total":95995035648,"available":95953715200,"internally_reserved":0,"allocated":876544,"data_stored":414355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":872448,"data_stored":414227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36720,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708432,"statfs":{"total":95995035648,"available":95957434368,"internally_reserved":0,"allocated":1417216,"data_stored":947138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986248,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":954368,"data_stored":490738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37104,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708048,"statfs":{"total":95995035648,"available":95957041152,"internally_reserved":0,"allocated":1626112,"data_stored":1145365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":201468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:32:37.117 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:32:37.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.270+0000 7f62feae2640 1 -- 172.21.15.43:0/956903297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62f8108a20 msgr2=0x7f62f81025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:37.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.270+0000 7f62feae2640 1 --2- 172.21.15.43:0/956903297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62f8108a20 0x7f62f81025b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009900 tx=0x7f62e002a410 comp rx=0 tx=0).stop 2024-04-25T10:32:37.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.270+0000 7f62feae2640 1 -- 172.21.15.43:0/956903297 shutdown_connections 2024-04-25T10:32:37.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.270+0000 7f62feae2640 1 --2- 172.21.15.43:0/956903297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62f810d380 0x7f62f810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.270+0000 7f62feae2640 1 --2- 172.21.15.43:0/956903297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62f8102af0 0x7f62f8102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.270+0000 7f62feae2640 1 --2- 172.21.15.43:0/956903297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62f8108a20 0x7f62f81025b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.270+0000 7f62feae2640 1 -- 172.21.15.43:0/956903297 >> 172.21.15.43:0/956903297 conn(0x7f62f8078110 msgr2=0x7f62f8078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:37.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.270+0000 7f62feae2640 1 -- 172.21.15.43:0/956903297 shutdown_connections 2024-04-25T10:32:37.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.270+0000 7f62feae2640 1 -- 172.21.15.43:0/956903297 wait complete. 2024-04-25T10:32:37.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.270+0000 7f62feae2640 1 Processor -- start 2024-04-25T10:32:37.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.271+0000 7f62feae2640 1 -- start start 2024-04-25T10:32:37.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.271+0000 7f62feae2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62f8102af0 0x7f62f81995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:37.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.271+0000 7f62feae2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62f8108a20 0x7f62f8199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:37.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.271+0000 7f62feae2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62f810d380 0x7f62f819dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.271+0000 7f62feae2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f62f8112be0 con 0x7f62f810d380 2024-04-25T10:32:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.271+0000 7f62feae2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f62f8112a60 con 0x7f62f8102af0 2024-04-25T10:32:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.271+0000 7f62feae2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f62f8112d60 con 0x7f62f8108a20 2024-04-25T10:32:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.271+0000 7f62fd07f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62f810d380 0x7f62f819dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.271+0000 7f62fd07f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62f810d380 0x7f62f819dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49132/0 (socket says 172.21.15.43:49132) 2024-04-25T10:32:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.271+0000 7f62fd07f640 1 -- 172.21.15.43:0/1562328776 learned_addr learned my addr 172.21.15.43:0/1562328776 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.271+0000 7f62fc87e640 1 --2- 172.21.15.43:0/1562328776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62f8102af0 0x7f62f81995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.271+0000 7f62f7fff640 1 --2- 172.21.15.43:0/1562328776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62f8108a20 0x7f62f8199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.271+0000 7f62fd07f640 1 -- 172.21.15.43:0/1562328776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62f8108a20 msgr2=0x7f62f8199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.271+0000 7f62fd07f640 1 --2- 172.21.15.43:0/1562328776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62f8108a20 0x7f62f8199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.271+0000 7f62fd07f640 1 -- 172.21.15.43:0/1562328776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62f8102af0 msgr2=0x7f62f81995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.271+0000 7f62fd07f640 1 --2- 172.21.15.43:0/1562328776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62f8102af0 0x7f62f81995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.271+0000 7f62fd07f640 1 -- 172.21.15.43:0/1562328776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f819e420 con 0x7f62f810d380 2024-04-25T10:32:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.271+0000 7f62fd07f640 1 --2- 172.21.15.43:0/1562328776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62f810d380 0x7f62f819dde0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009860 tx=0x7f62f0009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:37.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.272+0000 7f62f5ffb640 1 -- 172.21.15.43:0/1562328776 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f0005020 con 0x7f62f810d380 2024-04-25T10:32:37.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.272+0000 7f62f5ffb640 1 -- 172.21.15.43:0/1562328776 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62f00047a0 con 0x7f62f810d380 2024-04-25T10:32:37.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.272+0000 7f62feae2640 1 -- 172.21.15.43:0/1562328776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f819e710 con 0x7f62f810d380 2024-04-25T10:32:37.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.272+0000 7f62f5ffb640 1 -- 172.21.15.43:0/1562328776 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f0007a80 con 0x7f62f810d380 2024-04-25T10:32:37.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.272+0000 7f62feae2640 1 -- 172.21.15.43:0/1562328776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f819ec10 con 0x7f62f810d380 2024-04-25T10:32:37.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.273+0000 7f62feae2640 1 -- 172.21.15.43:0/1562328776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62f819a910 con 0x7f62f810d380 2024-04-25T10:32:37.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.274+0000 7f62f5ffb640 1 -- 172.21.15.43:0/1562328776 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 147747+0+0 (secure 0 0 0) 0x7f62f0004950 con 0x7f62f810d380 2024-04-25T10:32:37.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.275+0000 7f62f5ffb640 1 --2- 172.21.15.43:0/1562328776 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f62e40afd80 0x7f62e40b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:37.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.276+0000 7f62fc87e640 1 --2- 172.21.15.43:0/1562328776 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f62e40afd80 0x7f62e40b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:37.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.276+0000 7f62f5ffb640 1 -- 172.21.15.43:0/1562328776 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f62f00e0f30 con 0x7f62f810d380 2024-04-25T10:32:37.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.276+0000 7f62fc87e640 1 --2- 172.21.15.43:0/1562328776 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f62e40afd80 0x7f62e40b2290 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f62e0000c00 tx=0x7f62e0031400 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:37.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.279+0000 7f62f5ffb640 1 -- 172.21.15.43:0/1562328776 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62f819ec10 con 0x7f62f810d380 2024-04-25T10:32:37.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.476+0000 7f62feae2640 1 -- 172.21.15.43:0/1562328776 --> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f62f8062fe0 con 0x7f62e40afd80 2024-04-25T10:32:37.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.484+0000 7f62f5ffb640 1 -- 172.21.15.43:0/1562328776 <== mgr.15291 v2:172.21.15.43:6800/3658924116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257726 (secure 0 0 0) 0x7f62f8062fe0 con 0x7f62e40afd80 2024-04-25T10:32:37.489 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:32:37.490 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T10:32:37.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.487+0000 7f62feae2640 1 -- 172.21.15.43:0/1562328776 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f62e40afd80 msgr2=0x7f62e40b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:37.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.487+0000 7f62feae2640 1 --2- 172.21.15.43:0/1562328776 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f62e40afd80 0x7f62e40b2290 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f62e0000c00 tx=0x7f62e0031400 comp rx=0 tx=0).stop 2024-04-25T10:32:37.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.487+0000 7f62feae2640 1 -- 172.21.15.43:0/1562328776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62f810d380 msgr2=0x7f62f819dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:37.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.487+0000 7f62feae2640 1 --2- 172.21.15.43:0/1562328776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62f810d380 0x7f62f819dde0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009860 tx=0x7f62f0009d20 comp rx=0 tx=0).stop 2024-04-25T10:32:37.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.487+0000 7f62feae2640 1 -- 172.21.15.43:0/1562328776 shutdown_connections 2024-04-25T10:32:37.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.487+0000 7f62feae2640 1 --2- 172.21.15.43:0/1562328776 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f62e40afd80 0x7f62e40b2290 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.487+0000 7f62feae2640 1 --2- 172.21.15.43:0/1562328776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62f810d380 0x7f62f819dde0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.487+0000 7f62feae2640 1 --2- 172.21.15.43:0/1562328776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62f8108a20 0x7f62f8199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.487+0000 7f62feae2640 1 --2- 172.21.15.43:0/1562328776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62f8102af0 0x7f62f81995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.487+0000 7f62feae2640 1 -- 172.21.15.43:0/1562328776 >> 172.21.15.43:0/1562328776 conn(0x7f62f8078110 msgr2=0x7f62f81040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:37.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.487+0000 7f62feae2640 1 -- 172.21.15.43:0/1562328776 shutdown_connections 2024-04-25T10:32:37.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.487+0000 7f62feae2640 1 -- 172.21.15.43:0/1562328776 wait complete. 2024-04-25T10:32:37.503 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-25T10:32:36.476276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187709,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":890,"num_read_kb":583,"num_write":408,"num_write_kb":3585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":3012908,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":816,"ondisk_log_size":816,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":429720,"kb_used_data":13808,"kb_used_omap":18,"kb_used_meta":415597,"kb_avail":1124512104,"statfs":{"total":1151940427776,"available":1151500394496,"internally_reserved":0,"allocated":14139392,"data_stored":8588285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":425571641},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.983241"},"pg_stats":[{"pgid":"3.3f","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567443+0000","last_change":"2024-04-25T10:30:33.645032+0000","last_active":"2024-04-25T10:32:32.567443+0000","last_peered":"2024-04-25T10:32:32.567443+0000","last_clean":"2024-04-25T10:32:32.567443+0000","last_became_active":"2024-04-25T10:30:33.644687+0000","last_became_peered":"2024-04-25T10:30:33.644687+0000","last_unstale":"2024-04-25T10:32:32.567443+0000","last_undegraded":"2024-04-25T10:32:32.567443+0000","last_fullsized":"2024-04-25T10:32:32.567443+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:39.893228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869077+0000","last_change":"2024-04-25T10:30:33.650466+0000","last_active":"2024-04-25T10:32:32.869077+0000","last_peered":"2024-04-25T10:32:32.869077+0000","last_clean":"2024-04-25T10:32:32.869077+0000","last_became_active":"2024-04-25T10:30:33.649889+0000","last_became_peered":"2024-04-25T10:30:33.649889+0000","last_unstale":"2024-04-25T10:32:32.869077+0000","last_undegraded":"2024-04-25T10:32:32.869077+0000","last_fullsized":"2024-04-25T10:32:32.869077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:53.832494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869024+0000","last_change":"2024-04-25T10:30:33.648798+0000","last_active":"2024-04-25T10:32:32.869024+0000","last_peered":"2024-04-25T10:32:32.869024+0000","last_clean":"2024-04-25T10:32:32.869024+0000","last_became_active":"2024-04-25T10:30:33.648423+0000","last_became_peered":"2024-04-25T10:30:33.648423+0000","last_unstale":"2024-04-25T10:32:32.869024+0000","last_undegraded":"2024-04-25T10:32:32.869024+0000","last_fullsized":"2024-04-25T10:32:32.869024+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:19.119189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"79'8","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.562870+0000","last_change":"2024-04-25T10:31:33.941691+0000","last_active":"2024-04-25T10:32:32.562870+0000","last_peered":"2024-04-25T10:32:32.562870+0000","last_clean":"2024-04-25T10:32:32.562870+0000","last_became_active":"2024-04-25T10:31:33.941304+0000","last_became_peered":"2024-04-25T10:31:33.941304+0000","last_unstale":"2024-04-25T10:32:32.562870+0000","last_undegraded":"2024-04-25T10:32:32.562870+0000","last_fullsized":"2024-04-25T10:32:32.562870+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:18.573452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.564974+0000","last_change":"2024-04-25T10:30:33.646786+0000","last_active":"2024-04-25T10:32:32.564974+0000","last_peered":"2024-04-25T10:32:32.564974+0000","last_clean":"2024-04-25T10:32:32.564974+0000","last_became_active":"2024-04-25T10:30:33.646456+0000","last_became_peered":"2024-04-25T10:30:33.646456+0000","last_unstale":"2024-04-25T10:32:32.564974+0000","last_undegraded":"2024-04-25T10:32:32.564974+0000","last_fullsized":"2024-04-25T10:32:32.564974+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:55.006723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.572822+0000","last_change":"2024-04-25T10:30:33.639952+0000","last_active":"2024-04-25T10:32:32.572822+0000","last_peered":"2024-04-25T10:32:32.572822+0000","last_clean":"2024-04-25T10:32:32.572822+0000","last_became_active":"2024-04-25T10:30:33.639558+0000","last_became_peered":"2024-04-25T10:30:33.639558+0000","last_unstale":"2024-04-25T10:32:32.572822+0000","last_undegraded":"2024-04-25T10:32:32.572822+0000","last_fullsized":"2024-04-25T10:32:32.572822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:17.162902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.733860+0000","last_change":"2024-04-25T10:30:33.651530+0000","last_active":"2024-04-25T10:32:32.733860+0000","last_peered":"2024-04-25T10:32:32.733860+0000","last_clean":"2024-04-25T10:32:32.733860+0000","last_became_active":"2024-04-25T10:30:33.651151+0000","last_became_peered":"2024-04-25T10:30:33.651151+0000","last_unstale":"2024-04-25T10:32:32.733860+0000","last_undegraded":"2024-04-25T10:32:32.733860+0000","last_fullsized":"2024-04-25T10:32:32.733860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:31.835367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.461672+0000","last_change":"2024-04-25T10:30:33.645146+0000","last_active":"2024-04-25T10:32:32.461672+0000","last_peered":"2024-04-25T10:32:32.461672+0000","last_clean":"2024-04-25T10:32:32.461672+0000","last_became_active":"2024-04-25T10:30:33.644310+0000","last_became_peered":"2024-04-25T10:30:33.644310+0000","last_unstale":"2024-04-25T10:32:32.461672+0000","last_undegraded":"2024-04-25T10:32:32.461672+0000","last_fullsized":"2024-04-25T10:32:32.461672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:45.034934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462982+0000","last_change":"2024-04-25T10:30:33.649385+0000","last_active":"2024-04-25T10:32:32.462982+0000","last_peered":"2024-04-25T10:32:32.462982+0000","last_clean":"2024-04-25T10:32:32.462982+0000","last_became_active":"2024-04-25T10:30:33.649002+0000","last_became_peered":"2024-04-25T10:30:33.649002+0000","last_unstale":"2024-04-25T10:32:32.462982+0000","last_undegraded":"2024-04-25T10:32:32.462982+0000","last_fullsized":"2024-04-25T10:32:32.462982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:42.259290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"79'16","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.573098+0000","last_change":"2024-04-25T10:30:33.641918+0000","last_active":"2024-04-25T10:32:32.573098+0000","last_peered":"2024-04-25T10:32:32.573098+0000","last_clean":"2024-04-25T10:32:32.573098+0000","last_became_active":"2024-04-25T10:30:33.641553+0000","last_became_peered":"2024-04-25T10:30:33.641553+0000","last_unstale":"2024-04-25T10:32:32.573098+0000","last_undegraded":"2024-04-25T10:32:32.573098+0000","last_fullsized":"2024-04-25T10:32:32.573098+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:04.755694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734376+0000","last_change":"2024-04-25T10:30:33.638054+0000","last_active":"2024-04-25T10:32:32.734376+0000","last_peered":"2024-04-25T10:32:32.734376+0000","last_clean":"2024-04-25T10:32:32.734376+0000","last_became_active":"2024-04-25T10:30:33.637720+0000","last_became_peered":"2024-04-25T10:30:33.637720+0000","last_unstale":"2024-04-25T10:32:32.734376+0000","last_undegraded":"2024-04-25T10:32:32.734376+0000","last_fullsized":"2024-04-25T10:32:32.734376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:41.557430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734895+0000","last_change":"2024-04-25T10:30:33.642705+0000","last_active":"2024-04-25T10:32:32.734895+0000","last_peered":"2024-04-25T10:32:32.734895+0000","last_clean":"2024-04-25T10:32:32.734895+0000","last_became_active":"2024-04-25T10:30:33.641936+0000","last_became_peered":"2024-04-25T10:30:33.641936+0000","last_unstale":"2024-04-25T10:32:32.734895+0000","last_undegraded":"2024-04-25T10:32:32.734895+0000","last_fullsized":"2024-04-25T10:32:32.734895+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:36.450389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869566+0000","last_change":"2024-04-25T10:30:33.648987+0000","last_active":"2024-04-25T10:32:32.869566+0000","last_peered":"2024-04-25T10:32:32.869566+0000","last_clean":"2024-04-25T10:32:32.869566+0000","last_became_active":"2024-04-25T10:30:33.648621+0000","last_became_peered":"2024-04-25T10:30:33.648621+0000","last_unstale":"2024-04-25T10:32:32.869566+0000","last_undegraded":"2024-04-25T10:32:32.869566+0000","last_fullsized":"2024-04-25T10:32:32.869566+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:19.999002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.868610+0000","last_change":"2024-04-25T10:30:33.647360+0000","last_active":"2024-04-25T10:32:32.868610+0000","last_peered":"2024-04-25T10:32:32.868610+0000","last_clean":"2024-04-25T10:32:32.868610+0000","last_became_active":"2024-04-25T10:30:33.646719+0000","last_became_peered":"2024-04-25T10:30:33.646719+0000","last_unstale":"2024-04-25T10:32:32.868610+0000","last_undegraded":"2024-04-25T10:32:32.868610+0000","last_fullsized":"2024-04-25T10:32:32.868610+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:19.016333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.868373+0000","last_change":"2024-04-25T10:30:33.646503+0000","last_active":"2024-04-25T10:32:32.868373+0000","last_peered":"2024-04-25T10:32:32.868373+0000","last_clean":"2024-04-25T10:32:32.868373+0000","last_became_active":"2024-04-25T10:30:33.646123+0000","last_became_peered":"2024-04-25T10:30:33.646123+0000","last_unstale":"2024-04-25T10:32:32.868373+0000","last_undegraded":"2024-04-25T10:32:32.868373+0000","last_fullsized":"2024-04-25T10:32:32.868373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:57.712216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462498+0000","last_change":"2024-04-25T10:30:33.640463+0000","last_active":"2024-04-25T10:32:32.462498+0000","last_peered":"2024-04-25T10:32:32.462498+0000","last_clean":"2024-04-25T10:32:32.462498+0000","last_became_active":"2024-04-25T10:30:33.639931+0000","last_became_peered":"2024-04-25T10:30:33.639931+0000","last_unstale":"2024-04-25T10:32:32.462498+0000","last_undegraded":"2024-04-25T10:32:32.462498+0000","last_fullsized":"2024-04-25T10:32:32.462498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:18.863405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567342+0000","last_change":"2024-04-25T10:30:33.645695+0000","last_active":"2024-04-25T10:32:32.567342+0000","last_peered":"2024-04-25T10:32:32.567342+0000","last_clean":"2024-04-25T10:32:32.567342+0000","last_became_active":"2024-04-25T10:30:33.645387+0000","last_became_peered":"2024-04-25T10:30:33.645387+0000","last_unstale":"2024-04-25T10:32:32.567342+0000","last_undegraded":"2024-04-25T10:32:32.567342+0000","last_fullsized":"2024-04-25T10:32:32.567342+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:20.044456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.562870+0000","last_change":"2024-04-25T10:30:33.639087+0000","last_active":"2024-04-25T10:32:32.562870+0000","last_peered":"2024-04-25T10:32:32.562870+0000","last_clean":"2024-04-25T10:32:32.562870+0000","last_became_active":"2024-04-25T10:30:33.638726+0000","last_became_peered":"2024-04-25T10:30:33.638726+0000","last_unstale":"2024-04-25T10:32:32.562870+0000","last_undegraded":"2024-04-25T10:32:32.562870+0000","last_fullsized":"2024-04-25T10:32:32.562870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:36.887289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734245+0000","last_change":"2024-04-25T10:30:33.638646+0000","last_active":"2024-04-25T10:32:32.734245+0000","last_peered":"2024-04-25T10:32:32.734245+0000","last_clean":"2024-04-25T10:32:32.734245+0000","last_became_active":"2024-04-25T10:30:33.638261+0000","last_became_peered":"2024-04-25T10:30:33.638261+0000","last_unstale":"2024-04-25T10:32:32.734245+0000","last_undegraded":"2024-04-25T10:32:32.734245+0000","last_fullsized":"2024-04-25T10:32:32.734245+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:08.212310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.563028+0000","last_change":"2024-04-25T10:30:32.899992+0000","last_active":"2024-04-25T10:32:32.563028+0000","last_peered":"2024-04-25T10:32:32.563028+0000","last_clean":"2024-04-25T10:32:32.563028+0000","last_became_active":"2024-04-25T10:30:32.899643+0000","last_became_peered":"2024-04-25T10:30:32.899643+0000","last_unstale":"2024-04-25T10:32:32.563028+0000","last_undegraded":"2024-04-25T10:32:32.563028+0000","last_fullsized":"2024-04-25T10:32:32.563028+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:34.245121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462175+0000","last_change":"2024-04-25T10:30:33.644884+0000","last_active":"2024-04-25T10:32:32.462175+0000","last_peered":"2024-04-25T10:32:32.462175+0000","last_clean":"2024-04-25T10:32:32.462175+0000","last_became_active":"2024-04-25T10:30:33.644510+0000","last_became_peered":"2024-04-25T10:30:33.644510+0000","last_unstale":"2024-04-25T10:32:32.462175+0000","last_undegraded":"2024-04-25T10:32:32.462175+0000","last_fullsized":"2024-04-25T10:32:32.462175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:09.282865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462247+0000","last_change":"2024-04-25T10:30:32.909175+0000","last_active":"2024-04-25T10:32:32.462247+0000","last_peered":"2024-04-25T10:32:32.462247+0000","last_clean":"2024-04-25T10:32:32.462247+0000","last_became_active":"2024-04-25T10:30:32.908813+0000","last_became_peered":"2024-04-25T10:30:32.908813+0000","last_unstale":"2024-04-25T10:32:32.462247+0000","last_undegraded":"2024-04-25T10:32:32.462247+0000","last_fullsized":"2024-04-25T10:32:32.462247+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:04.256452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869245+0000","last_change":"2024-04-25T10:30:33.642095+0000","last_active":"2024-04-25T10:32:32.869245+0000","last_peered":"2024-04-25T10:32:32.869245+0000","last_clean":"2024-04-25T10:32:32.869245+0000","last_became_active":"2024-04-25T10:30:33.641713+0000","last_became_peered":"2024-04-25T10:30:33.641713+0000","last_unstale":"2024-04-25T10:32:32.869245+0000","last_undegraded":"2024-04-25T10:32:32.869245+0000","last_fullsized":"2024-04-25T10:32:32.869245+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:42.425197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.565390+0000","last_change":"2024-04-25T10:30:32.904376+0000","last_active":"2024-04-25T10:32:32.565390+0000","last_peered":"2024-04-25T10:32:32.565390+0000","last_clean":"2024-04-25T10:32:32.565390+0000","last_became_active":"2024-04-25T10:30:32.903994+0000","last_became_peered":"2024-04-25T10:30:32.903994+0000","last_unstale":"2024-04-25T10:32:32.565390+0000","last_undegraded":"2024-04-25T10:32:32.565390+0000","last_fullsized":"2024-04-25T10:32:32.565390+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:52.727800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462581+0000","last_change":"2024-04-25T10:30:33.637171+0000","last_active":"2024-04-25T10:32:32.462581+0000","last_peered":"2024-04-25T10:32:32.462581+0000","last_clean":"2024-04-25T10:32:32.462581+0000","last_became_active":"2024-04-25T10:30:33.636822+0000","last_became_peered":"2024-04-25T10:30:33.636822+0000","last_unstale":"2024-04-25T10:32:32.462581+0000","last_undegraded":"2024-04-25T10:32:32.462581+0000","last_fullsized":"2024-04-25T10:32:32.462581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:54.220784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.565240+0000","last_change":"2024-04-25T10:30:32.900856+0000","last_active":"2024-04-25T10:32:32.565240+0000","last_peered":"2024-04-25T10:32:32.565240+0000","last_clean":"2024-04-25T10:32:32.565240+0000","last_became_active":"2024-04-25T10:30:32.900519+0000","last_became_peered":"2024-04-25T10:30:32.900519+0000","last_unstale":"2024-04-25T10:32:32.565240+0000","last_undegraded":"2024-04-25T10:32:32.565240+0000","last_fullsized":"2024-04-25T10:32:32.565240+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:49.738005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463759+0000","last_change":"2024-04-25T10:30:33.651736+0000","last_active":"2024-04-25T10:32:32.463759+0000","last_peered":"2024-04-25T10:32:32.463759+0000","last_clean":"2024-04-25T10:32:32.463759+0000","last_became_active":"2024-04-25T10:30:33.651395+0000","last_became_peered":"2024-04-25T10:30:33.651395+0000","last_unstale":"2024-04-25T10:32:32.463759+0000","last_undegraded":"2024-04-25T10:32:32.463759+0000","last_fullsized":"2024-04-25T10:32:32.463759+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:58.230092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":146,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.982457+0000","last_change":"2024-04-25T10:30:32.901348+0000","last_active":"2024-04-25T10:32:32.982457+0000","last_peered":"2024-04-25T10:32:32.982457+0000","last_clean":"2024-04-25T10:32:32.982457+0000","last_became_active":"2024-04-25T10:30:32.901025+0000","last_became_peered":"2024-04-25T10:30:32.901025+0000","last_unstale":"2024-04-25T10:32:32.982457+0000","last_undegraded":"2024-04-25T10:32:32.982457+0000","last_fullsized":"2024-04-25T10:32:32.982457+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:00.521934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.565033+0000","last_change":"2024-04-25T10:30:33.649206+0000","last_active":"2024-04-25T10:32:32.565033+0000","last_peered":"2024-04-25T10:32:32.565033+0000","last_clean":"2024-04-25T10:32:32.565033+0000","last_became_active":"2024-04-25T10:30:33.648883+0000","last_became_peered":"2024-04-25T10:30:33.648883+0000","last_unstale":"2024-04-25T10:32:32.565033+0000","last_undegraded":"2024-04-25T10:32:32.565033+0000","last_fullsized":"2024-04-25T10:32:32.565033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:43.105562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.572768+0000","last_change":"2024-04-25T10:30:32.901201+0000","last_active":"2024-04-25T10:32:32.572768+0000","last_peered":"2024-04-25T10:32:32.572768+0000","last_clean":"2024-04-25T10:32:32.572768+0000","last_became_active":"2024-04-25T10:30:32.900876+0000","last_became_peered":"2024-04-25T10:30:32.900876+0000","last_unstale":"2024-04-25T10:32:32.572768+0000","last_undegraded":"2024-04-25T10:32:32.572768+0000","last_fullsized":"2024-04-25T10:32:32.572768+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:58.077045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463571+0000","last_change":"2024-04-25T10:30:33.651935+0000","last_active":"2024-04-25T10:32:32.463571+0000","last_peered":"2024-04-25T10:32:32.463571+0000","last_clean":"2024-04-25T10:32:32.463571+0000","last_became_active":"2024-04-25T10:30:33.651568+0000","last_became_peered":"2024-04-25T10:30:33.651568+0000","last_unstale":"2024-04-25T10:32:32.463571+0000","last_undegraded":"2024-04-25T10:32:32.463571+0000","last_fullsized":"2024-04-25T10:32:32.463571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:58.849535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869003+0000","last_change":"2024-04-25T10:30:32.896500+0000","last_active":"2024-04-25T10:32:32.869003+0000","last_peered":"2024-04-25T10:32:32.869003+0000","last_clean":"2024-04-25T10:32:32.869003+0000","last_became_active":"2024-04-25T10:30:32.896071+0000","last_became_peered":"2024-04-25T10:30:32.896071+0000","last_unstale":"2024-04-25T10:32:32.869003+0000","last_undegraded":"2024-04-25T10:32:32.869003+0000","last_fullsized":"2024-04-25T10:32:32.869003+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:10.385443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.461754+0000","last_change":"2024-04-25T10:30:33.631730+0000","last_active":"2024-04-25T10:32:32.461754+0000","last_peered":"2024-04-25T10:32:32.461754+0000","last_clean":"2024-04-25T10:32:32.461754+0000","last_became_active":"2024-04-25T10:30:33.631374+0000","last_became_peered":"2024-04-25T10:30:33.631374+0000","last_unstale":"2024-04-25T10:32:32.461754+0000","last_undegraded":"2024-04-25T10:32:32.461754+0000","last_fullsized":"2024-04-25T10:32:32.461754+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:14.957866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463314+0000","last_change":"2024-04-25T10:30:32.900762+0000","last_active":"2024-04-25T10:32:32.463314+0000","last_peered":"2024-04-25T10:32:32.463314+0000","last_clean":"2024-04-25T10:32:32.463314+0000","last_became_active":"2024-04-25T10:30:32.900459+0000","last_became_peered":"2024-04-25T10:30:32.900459+0000","last_unstale":"2024-04-25T10:32:32.463314+0000","last_undegraded":"2024-04-25T10:32:32.463314+0000","last_fullsized":"2024-04-25T10:32:32.463314+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:22.286261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.562715+0000","last_change":"2024-04-25T10:30:33.637727+0000","last_active":"2024-04-25T10:32:32.562715+0000","last_peered":"2024-04-25T10:32:32.562715+0000","last_clean":"2024-04-25T10:32:32.562715+0000","last_became_active":"2024-04-25T10:30:33.637359+0000","last_became_peered":"2024-04-25T10:30:33.637359+0000","last_unstale":"2024-04-25T10:32:32.562715+0000","last_undegraded":"2024-04-25T10:32:32.562715+0000","last_fullsized":"2024-04-25T10:32:32.562715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:21.877562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":141,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.733608+0000","last_change":"2024-04-25T10:30:32.894210+0000","last_active":"2024-04-25T10:32:32.733608+0000","last_peered":"2024-04-25T10:32:32.733608+0000","last_clean":"2024-04-25T10:32:32.733608+0000","last_became_active":"2024-04-25T10:30:32.893241+0000","last_became_peered":"2024-04-25T10:30:32.893241+0000","last_unstale":"2024-04-25T10:32:32.733608+0000","last_undegraded":"2024-04-25T10:32:32.733608+0000","last_fullsized":"2024-04-25T10:32:32.733608+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:06.340392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463016+0000","last_change":"2024-04-25T10:30:33.632349+0000","last_active":"2024-04-25T10:32:32.463016+0000","last_peered":"2024-04-25T10:32:32.463016+0000","last_clean":"2024-04-25T10:32:32.463016+0000","last_became_active":"2024-04-25T10:30:33.631968+0000","last_became_peered":"2024-04-25T10:30:33.631968+0000","last_unstale":"2024-04-25T10:32:32.463016+0000","last_undegraded":"2024-04-25T10:32:32.463016+0000","last_fullsized":"2024-04-25T10:32:32.463016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:29.090932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734093+0000","last_change":"2024-04-25T10:30:32.900105+0000","last_active":"2024-04-25T10:32:32.734093+0000","last_peered":"2024-04-25T10:32:32.734093+0000","last_clean":"2024-04-25T10:32:32.734093+0000","last_became_active":"2024-04-25T10:30:32.899740+0000","last_became_peered":"2024-04-25T10:30:32.899740+0000","last_unstale":"2024-04-25T10:32:32.734093+0000","last_undegraded":"2024-04-25T10:32:32.734093+0000","last_fullsized":"2024-04-25T10:32:32.734093+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:49.390799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"79'9","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869399+0000","last_change":"2024-04-25T10:31:33.943769+0000","last_active":"2024-04-25T10:32:32.869399+0000","last_peered":"2024-04-25T10:32:32.869399+0000","last_clean":"2024-04-25T10:32:32.869399+0000","last_became_active":"2024-04-25T10:31:33.943331+0000","last_became_peered":"2024-04-25T10:31:33.943331+0000","last_unstale":"2024-04-25T10:32:32.869399+0000","last_undegraded":"2024-04-25T10:32:32.869399+0000","last_fullsized":"2024-04-25T10:32:32.869399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:07.183059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463831+0000","last_change":"2024-04-25T10:30:32.893453+0000","last_active":"2024-04-25T10:32:32.463831+0000","last_peered":"2024-04-25T10:32:32.463831+0000","last_clean":"2024-04-25T10:32:32.463831+0000","last_became_active":"2024-04-25T10:30:32.893106+0000","last_became_peered":"2024-04-25T10:30:32.893106+0000","last_unstale":"2024-04-25T10:32:32.463831+0000","last_undegraded":"2024-04-25T10:32:32.463831+0000","last_fullsized":"2024-04-25T10:32:32.463831+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:57.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462772+0000","last_change":"2024-04-25T10:30:33.645693+0000","last_active":"2024-04-25T10:32:32.462772+0000","last_peered":"2024-04-25T10:32:32.462772+0000","last_clean":"2024-04-25T10:32:32.462772+0000","last_became_active":"2024-04-25T10:30:33.645342+0000","last_became_peered":"2024-04-25T10:30:33.645342+0000","last_unstale":"2024-04-25T10:32:32.462772+0000","last_undegraded":"2024-04-25T10:32:32.462772+0000","last_fullsized":"2024-04-25T10:32:32.462772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:15.729860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'35","reported_seq":317,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:33.028479+0000","last_change":"2024-04-25T10:31:33.949796+0000","last_active":"2024-04-25T10:32:33.028479+0000","last_peered":"2024-04-25T10:32:33.028479+0000","last_clean":"2024-04-25T10:32:33.028479+0000","last_became_active":"2024-04-25T10:31:33.942989+0000","last_became_peered":"2024-04-25T10:31:33.942989+0000","last_unstale":"2024-04-25T10:32:33.028479+0000","last_undegraded":"2024-04-25T10:32:33.028479+0000","last_fullsized":"2024-04-25T10:32:33.028479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:04.194643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67758,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":235,"num_read_kb":184,"num_write":36,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61046,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.565318+0000","last_change":"2024-04-25T10:30:33.641768+0000","last_active":"2024-04-25T10:32:32.565318+0000","last_peered":"2024-04-25T10:32:32.565318+0000","last_clean":"2024-04-25T10:32:32.565318+0000","last_became_active":"2024-04-25T10:30:33.641415+0000","last_became_peered":"2024-04-25T10:30:33.641415+0000","last_unstale":"2024-04-25T10:32:32.565318+0000","last_undegraded":"2024-04-25T10:32:32.565318+0000","last_fullsized":"2024-04-25T10:32:32.565318+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:38.569474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734173+0000","last_change":"2024-04-25T10:30:32.898703+0000","last_active":"2024-04-25T10:32:32.734173+0000","last_peered":"2024-04-25T10:32:32.734173+0000","last_clean":"2024-04-25T10:32:32.734173+0000","last_became_active":"2024-04-25T10:30:32.898340+0000","last_became_peered":"2024-04-25T10:30:32.898340+0000","last_unstale":"2024-04-25T10:32:32.734173+0000","last_undegraded":"2024-04-25T10:32:32.734173+0000","last_fullsized":"2024-04-25T10:32:32.734173+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:27.024144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869241+0000","last_change":"2024-04-25T10:30:33.640393+0000","last_active":"2024-04-25T10:32:32.869241+0000","last_peered":"2024-04-25T10:32:32.869241+0000","last_clean":"2024-04-25T10:32:32.869241+0000","last_became_active":"2024-04-25T10:30:33.639822+0000","last_became_peered":"2024-04-25T10:30:33.639822+0000","last_unstale":"2024-04-25T10:32:32.869241+0000","last_undegraded":"2024-04-25T10:32:32.869241+0000","last_fullsized":"2024-04-25T10:32:32.869241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:08.618406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463750+0000","last_change":"2024-04-25T10:30:32.900880+0000","last_active":"2024-04-25T10:32:32.463750+0000","last_peered":"2024-04-25T10:32:32.463750+0000","last_clean":"2024-04-25T10:32:32.463750+0000","last_became_active":"2024-04-25T10:30:32.899987+0000","last_became_peered":"2024-04-25T10:30:32.899987+0000","last_unstale":"2024-04-25T10:32:32.463750+0000","last_undegraded":"2024-04-25T10:32:32.463750+0000","last_fullsized":"2024-04-25T10:32:32.463750+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:35.687773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734321+0000","last_change":"2024-04-25T10:30:33.641933+0000","last_active":"2024-04-25T10:32:32.734321+0000","last_peered":"2024-04-25T10:32:32.734321+0000","last_clean":"2024-04-25T10:32:32.734321+0000","last_became_active":"2024-04-25T10:30:33.641549+0000","last_became_peered":"2024-04-25T10:30:33.641549+0000","last_unstale":"2024-04-25T10:32:32.734321+0000","last_undegraded":"2024-04-25T10:32:32.734321+0000","last_fullsized":"2024-04-25T10:32:32.734321+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:36.696523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'5","reported_seq":145,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734534+0000","last_change":"2024-04-25T10:30:32.901178+0000","last_active":"2024-04-25T10:32:32.734534+0000","last_peered":"2024-04-25T10:32:32.734534+0000","last_clean":"2024-04-25T10:32:32.734534+0000","last_became_active":"2024-04-25T10:30:32.900817+0000","last_became_peered":"2024-04-25T10:30:32.900817+0000","last_unstale":"2024-04-25T10:32:32.734534+0000","last_undegraded":"2024-04-25T10:32:32.734534+0000","last_fullsized":"2024-04-25T10:32:32.734534+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:53.999253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"79'6","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.733811+0000","last_change":"2024-04-25T10:31:33.943470+0000","last_active":"2024-04-25T10:32:32.733811+0000","last_peered":"2024-04-25T10:32:32.733811+0000","last_clean":"2024-04-25T10:32:32.733811+0000","last_became_active":"2024-04-25T10:31:33.943098+0000","last_became_peered":"2024-04-25T10:31:33.943098+0000","last_unstale":"2024-04-25T10:32:32.733811+0000","last_undegraded":"2024-04-25T10:32:32.733811+0000","last_fullsized":"2024-04-25T10:32:32.733811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:27.707195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.564919+0000","last_change":"2024-04-25T10:30:32.902862+0000","last_active":"2024-04-25T10:32:32.564919+0000","last_peered":"2024-04-25T10:32:32.564919+0000","last_clean":"2024-04-25T10:32:32.564919+0000","last_became_active":"2024-04-25T10:30:32.902537+0000","last_became_peered":"2024-04-25T10:30:32.902537+0000","last_unstale":"2024-04-25T10:32:32.564919+0000","last_undegraded":"2024-04-25T10:32:32.564919+0000","last_fullsized":"2024-04-25T10:32:32.564919+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:55.974128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463465+0000","last_change":"2024-04-25T10:30:33.651409+0000","last_active":"2024-04-25T10:32:32.463465+0000","last_peered":"2024-04-25T10:32:32.463465+0000","last_clean":"2024-04-25T10:32:32.463465+0000","last_became_active":"2024-04-25T10:30:33.650673+0000","last_became_peered":"2024-04-25T10:30:33.650673+0000","last_unstale":"2024-04-25T10:32:32.463465+0000","last_undegraded":"2024-04-25T10:32:32.463465+0000","last_fullsized":"2024-04-25T10:32:32.463465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:48.989198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567343+0000","last_change":"2024-04-25T10:31:33.953408+0000","last_active":"2024-04-25T10:32:32.567343+0000","last_peered":"2024-04-25T10:32:32.567343+0000","last_clean":"2024-04-25T10:32:32.567343+0000","last_became_active":"2024-04-25T10:31:33.943495+0000","last_became_peered":"2024-04-25T10:31:33.943495+0000","last_unstale":"2024-04-25T10:32:32.567343+0000","last_undegraded":"2024-04-25T10:32:32.567343+0000","last_fullsized":"2024-04-25T10:32:32.567343+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:03.947967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567280+0000","last_change":"2024-04-25T10:30:33.643929+0000","last_active":"2024-04-25T10:32:32.567280+0000","last_peered":"2024-04-25T10:32:32.567280+0000","last_clean":"2024-04-25T10:32:32.567280+0000","last_became_active":"2024-04-25T10:30:33.643593+0000","last_became_peered":"2024-04-25T10:30:33.643593+0000","last_unstale":"2024-04-25T10:32:32.567280+0000","last_undegraded":"2024-04-25T10:32:32.567280+0000","last_fullsized":"2024-04-25T10:32:32.567280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:39.591022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.562806+0000","last_change":"2024-04-25T10:30:32.898937+0000","last_active":"2024-04-25T10:32:32.562806+0000","last_peered":"2024-04-25T10:32:32.562806+0000","last_clean":"2024-04-25T10:32:32.562806+0000","last_became_active":"2024-04-25T10:30:32.898570+0000","last_became_peered":"2024-04-25T10:30:32.898570+0000","last_unstale":"2024-04-25T10:32:32.562806+0000","last_undegraded":"2024-04-25T10:32:32.562806+0000","last_fullsized":"2024-04-25T10:32:32.562806+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:40.123035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.573011+0000","last_change":"2024-04-25T10:30:33.632857+0000","last_active":"2024-04-25T10:32:32.573011+0000","last_peered":"2024-04-25T10:32:32.573011+0000","last_clean":"2024-04-25T10:32:32.573011+0000","last_became_active":"2024-04-25T10:30:33.632043+0000","last_became_peered":"2024-04-25T10:30:33.632043+0000","last_unstale":"2024-04-25T10:32:32.573011+0000","last_undegraded":"2024-04-25T10:32:32.573011+0000","last_fullsized":"2024-04-25T10:32:32.573011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:12.683035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734027+0000","last_change":"2024-04-25T10:30:32.901349+0000","last_active":"2024-04-25T10:32:32.734027+0000","last_peered":"2024-04-25T10:32:32.734027+0000","last_clean":"2024-04-25T10:32:32.734027+0000","last_became_active":"2024-04-25T10:30:32.900881+0000","last_became_peered":"2024-04-25T10:30:32.900881+0000","last_unstale":"2024-04-25T10:32:32.734027+0000","last_undegraded":"2024-04-25T10:32:32.734027+0000","last_fullsized":"2024-04-25T10:32:32.734027+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:34.963564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.572921+0000","last_change":"2024-04-25T10:30:33.644376+0000","last_active":"2024-04-25T10:32:32.572921+0000","last_peered":"2024-04-25T10:32:32.572921+0000","last_clean":"2024-04-25T10:32:32.572921+0000","last_became_active":"2024-04-25T10:30:33.644055+0000","last_became_peered":"2024-04-25T10:30:33.644055+0000","last_unstale":"2024-04-25T10:32:32.572921+0000","last_undegraded":"2024-04-25T10:32:32.572921+0000","last_fullsized":"2024-04-25T10:32:32.572921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:48.134775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"79'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463526+0000","last_change":"2024-04-25T10:30:33.646312+0000","last_active":"2024-04-25T10:32:32.463526+0000","last_peered":"2024-04-25T10:32:32.463526+0000","last_clean":"2024-04-25T10:32:32.463526+0000","last_became_active":"2024-04-25T10:30:33.642338+0000","last_became_peered":"2024-04-25T10:30:33.642338+0000","last_unstale":"2024-04-25T10:32:32.463526+0000","last_undegraded":"2024-04-25T10:32:32.463526+0000","last_fullsized":"2024-04-25T10:32:32.463526+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:42.588360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":141,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.868833+0000","last_change":"2024-04-25T10:30:32.909680+0000","last_active":"2024-04-25T10:32:32.868833+0000","last_peered":"2024-04-25T10:32:32.868833+0000","last_clean":"2024-04-25T10:32:32.868833+0000","last_became_active":"2024-04-25T10:30:32.909169+0000","last_became_peered":"2024-04-25T10:30:32.909169+0000","last_unstale":"2024-04-25T10:32:32.868833+0000","last_undegraded":"2024-04-25T10:32:32.868833+0000","last_fullsized":"2024-04-25T10:32:32.868833+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:54.876475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.461954+0000","last_change":"2024-04-25T10:30:33.649524+0000","last_active":"2024-04-25T10:32:32.461954+0000","last_peered":"2024-04-25T10:32:32.461954+0000","last_clean":"2024-04-25T10:32:32.461954+0000","last_became_active":"2024-04-25T10:30:33.648958+0000","last_became_peered":"2024-04-25T10:30:33.648958+0000","last_unstale":"2024-04-25T10:32:32.461954+0000","last_undegraded":"2024-04-25T10:32:32.461954+0000","last_fullsized":"2024-04-25T10:32:32.461954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:13.407742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734705+0000","last_change":"2024-04-25T10:30:32.895270+0000","last_active":"2024-04-25T10:32:32.734705+0000","last_peered":"2024-04-25T10:32:32.734705+0000","last_clean":"2024-04-25T10:32:32.734705+0000","last_became_active":"2024-04-25T10:30:32.894899+0000","last_became_peered":"2024-04-25T10:30:32.894899+0000","last_unstale":"2024-04-25T10:32:32.734705+0000","last_undegraded":"2024-04-25T10:32:32.734705+0000","last_fullsized":"2024-04-25T10:32:32.734705+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:21.233217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.733901+0000","last_change":"2024-04-25T10:29:21.105073+0000","last_active":"2024-04-25T10:32:32.733901+0000","last_peered":"2024-04-25T10:32:32.733901+0000","last_clean":"2024-04-25T10:32:32.733901+0000","last_became_active":"2024-04-25T10:29:21.098771+0000","last_became_peered":"2024-04-25T10:29:21.098771+0000","last_unstale":"2024-04-25T10:32:32.733901+0000","last_undegraded":"2024-04-25T10:32:32.733901+0000","last_fullsized":"2024-04-25T10:32:32.733901+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:59.015440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:59.015440+0000","last_clean_scrub_stamp":"2024-04-25T10:25:59.015440+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:41.728977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.868619+0000","last_change":"2024-04-25T10:30:33.639159+0000","last_active":"2024-04-25T10:32:32.868619+0000","last_peered":"2024-04-25T10:32:32.868619+0000","last_clean":"2024-04-25T10:32:32.868619+0000","last_became_active":"2024-04-25T10:30:33.638761+0000","last_became_peered":"2024-04-25T10:30:33.638761+0000","last_unstale":"2024-04-25T10:32:32.868619+0000","last_undegraded":"2024-04-25T10:32:32.868619+0000","last_fullsized":"2024-04-25T10:32:32.868619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:43.262103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734611+0000","last_change":"2024-04-25T10:30:32.898587+0000","last_active":"2024-04-25T10:32:32.734611+0000","last_peered":"2024-04-25T10:32:32.734611+0000","last_clean":"2024-04-25T10:32:32.734611+0000","last_became_active":"2024-04-25T10:30:32.898113+0000","last_became_peered":"2024-04-25T10:30:32.898113+0000","last_unstale":"2024-04-25T10:32:32.734611+0000","last_undegraded":"2024-04-25T10:32:32.734611+0000","last_fullsized":"2024-04-25T10:32:32.734611+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:24.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.565148+0000","last_change":"2024-04-25T10:30:32.907188+0000","last_active":"2024-04-25T10:32:32.565148+0000","last_peered":"2024-04-25T10:32:32.565148+0000","last_clean":"2024-04-25T10:32:32.565148+0000","last_became_active":"2024-04-25T10:30:32.906837+0000","last_became_peered":"2024-04-25T10:30:32.906837+0000","last_unstale":"2024-04-25T10:32:32.565148+0000","last_undegraded":"2024-04-25T10:32:32.565148+0000","last_fullsized":"2024-04-25T10:32:32.565148+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:11.835053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869498+0000","last_change":"2024-04-25T10:30:33.636159+0000","last_active":"2024-04-25T10:32:32.869498+0000","last_peered":"2024-04-25T10:32:32.869498+0000","last_clean":"2024-04-25T10:32:32.869498+0000","last_became_active":"2024-04-25T10:30:33.635098+0000","last_became_peered":"2024-04-25T10:30:33.635098+0000","last_unstale":"2024-04-25T10:32:32.869498+0000","last_undegraded":"2024-04-25T10:32:32.869498+0000","last_fullsized":"2024-04-25T10:32:32.869498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:13.092368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'45","reported_seq":89,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.689001+0000","last_change":"2024-04-25T10:30:32.905469+0000","last_active":"2024-04-25T10:32:32.689001+0000","last_peered":"2024-04-25T10:32:32.689001+0000","last_clean":"2024-04-25T10:32:32.689001+0000","last_became_active":"2024-04-25T10:30:32.905123+0000","last_became_peered":"2024-04-25T10:30:32.905123+0000","last_unstale":"2024-04-25T10:32:32.689001+0000","last_undegraded":"2024-04-25T10:32:32.689001+0000","last_fullsized":"2024-04-25T10:32:32.689001+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:12.884809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462239+0000","last_change":"2024-04-25T10:30:33.640690+0000","last_active":"2024-04-25T10:32:32.462239+0000","last_peered":"2024-04-25T10:32:32.462239+0000","last_clean":"2024-04-25T10:32:32.462239+0000","last_became_active":"2024-04-25T10:30:33.640310+0000","last_became_peered":"2024-04-25T10:30:33.640310+0000","last_unstale":"2024-04-25T10:32:32.462239+0000","last_undegraded":"2024-04-25T10:32:32.462239+0000","last_fullsized":"2024-04-25T10:32:32.462239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:03.671495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734684+0000","last_change":"2024-04-25T10:30:32.893743+0000","last_active":"2024-04-25T10:32:32.734684+0000","last_peered":"2024-04-25T10:32:32.734684+0000","last_clean":"2024-04-25T10:32:32.734684+0000","last_became_active":"2024-04-25T10:30:32.893360+0000","last_became_peered":"2024-04-25T10:30:32.893360+0000","last_unstale":"2024-04-25T10:32:32.734684+0000","last_undegraded":"2024-04-25T10:32:32.734684+0000","last_fullsized":"2024-04-25T10:32:32.734684+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:39.830376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869026+0000","last_change":"2024-04-25T10:30:33.645127+0000","last_active":"2024-04-25T10:32:32.869026+0000","last_peered":"2024-04-25T10:32:32.869026+0000","last_clean":"2024-04-25T10:32:32.869026+0000","last_became_active":"2024-04-25T10:30:33.644259+0000","last_became_peered":"2024-04-25T10:30:33.644259+0000","last_unstale":"2024-04-25T10:32:32.869026+0000","last_undegraded":"2024-04-25T10:32:32.869026+0000","last_fullsized":"2024-04-25T10:32:32.869026+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:56.849861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462504+0000","last_change":"2024-04-25T10:30:32.896885+0000","last_active":"2024-04-25T10:32:32.462504+0000","last_peered":"2024-04-25T10:32:32.462504+0000","last_clean":"2024-04-25T10:32:32.462504+0000","last_became_active":"2024-04-25T10:30:32.896447+0000","last_became_peered":"2024-04-25T10:30:32.896447+0000","last_unstale":"2024-04-25T10:32:32.462504+0000","last_undegraded":"2024-04-25T10:32:32.462504+0000","last_fullsized":"2024-04-25T10:32:32.462504+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:13.133549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"79'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.564914+0000","last_change":"2024-04-25T10:30:33.645016+0000","last_active":"2024-04-25T10:32:32.564914+0000","last_peered":"2024-04-25T10:32:32.564914+0000","last_clean":"2024-04-25T10:32:32.564914+0000","last_became_active":"2024-04-25T10:30:33.644661+0000","last_became_peered":"2024-04-25T10:30:33.644661+0000","last_unstale":"2024-04-25T10:32:32.564914+0000","last_undegraded":"2024-04-25T10:32:32.564914+0000","last_fullsized":"2024-04-25T10:32:32.564914+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:20.935124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.461752+0000","last_change":"2024-04-25T10:30:32.907827+0000","last_active":"2024-04-25T10:32:32.461752+0000","last_peered":"2024-04-25T10:32:32.461752+0000","last_clean":"2024-04-25T10:32:32.461752+0000","last_became_active":"2024-04-25T10:30:32.907402+0000","last_became_peered":"2024-04-25T10:30:32.907402+0000","last_unstale":"2024-04-25T10:32:32.461752+0000","last_undegraded":"2024-04-25T10:32:32.461752+0000","last_fullsized":"2024-04-25T10:32:32.461752+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:38.501617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567040+0000","last_change":"2024-04-25T10:30:33.636935+0000","last_active":"2024-04-25T10:32:32.567040+0000","last_peered":"2024-04-25T10:32:32.567040+0000","last_clean":"2024-04-25T10:32:32.567040+0000","last_became_active":"2024-04-25T10:30:33.636566+0000","last_became_peered":"2024-04-25T10:30:33.636566+0000","last_unstale":"2024-04-25T10:32:32.567040+0000","last_undegraded":"2024-04-25T10:32:32.567040+0000","last_fullsized":"2024-04-25T10:32:32.567040+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:24.556275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'2","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567152+0000","last_change":"2024-04-25T10:31:33.946786+0000","last_active":"2024-04-25T10:32:32.567152+0000","last_peered":"2024-04-25T10:32:32.567152+0000","last_clean":"2024-04-25T10:32:32.567152+0000","last_became_active":"2024-04-25T10:31:33.941298+0000","last_became_peered":"2024-04-25T10:31:33.941298+0000","last_unstale":"2024-04-25T10:32:32.567152+0000","last_undegraded":"2024-04-25T10:32:32.567152+0000","last_fullsized":"2024-04-25T10:32:32.567152+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:55.009673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.461836+0000","last_change":"2024-04-25T10:30:33.642986+0000","last_active":"2024-04-25T10:32:32.461836+0000","last_peered":"2024-04-25T10:32:32.461836+0000","last_clean":"2024-04-25T10:32:32.461836+0000","last_became_active":"2024-04-25T10:30:33.642545+0000","last_became_peered":"2024-04-25T10:30:33.642545+0000","last_unstale":"2024-04-25T10:32:32.461836+0000","last_undegraded":"2024-04-25T10:32:32.461836+0000","last_fullsized":"2024-04-25T10:32:32.461836+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:49.369648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.565008+0000","last_change":"2024-04-25T10:30:32.902239+0000","last_active":"2024-04-25T10:32:32.565008+0000","last_peered":"2024-04-25T10:32:32.565008+0000","last_clean":"2024-04-25T10:32:32.565008+0000","last_became_active":"2024-04-25T10:30:32.901653+0000","last_became_peered":"2024-04-25T10:30:32.901653+0000","last_unstale":"2024-04-25T10:32:32.565008+0000","last_undegraded":"2024-04-25T10:32:32.565008+0000","last_fullsized":"2024-04-25T10:32:32.565008+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:34.732428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734130+0000","last_change":"2024-04-25T10:30:33.639763+0000","last_active":"2024-04-25T10:32:32.734130+0000","last_peered":"2024-04-25T10:32:32.734130+0000","last_clean":"2024-04-25T10:32:32.734130+0000","last_became_active":"2024-04-25T10:30:33.638004+0000","last_became_peered":"2024-04-25T10:30:33.638004+0000","last_unstale":"2024-04-25T10:32:32.734130+0000","last_undegraded":"2024-04-25T10:32:32.734130+0000","last_fullsized":"2024-04-25T10:32:32.734130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:09.722333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734268+0000","last_change":"2024-04-25T10:30:32.894823+0000","last_active":"2024-04-25T10:32:32.734268+0000","last_peered":"2024-04-25T10:32:32.734268+0000","last_clean":"2024-04-25T10:32:32.734268+0000","last_became_active":"2024-04-25T10:30:32.893819+0000","last_became_peered":"2024-04-25T10:30:32.893819+0000","last_unstale":"2024-04-25T10:32:32.734268+0000","last_undegraded":"2024-04-25T10:32:32.734268+0000","last_fullsized":"2024-04-25T10:32:32.734268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:59.809410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734513+0000","last_change":"2024-04-25T10:30:33.637046+0000","last_active":"2024-04-25T10:32:32.734513+0000","last_peered":"2024-04-25T10:32:32.734513+0000","last_clean":"2024-04-25T10:32:32.734513+0000","last_became_active":"2024-04-25T10:30:33.634903+0000","last_became_peered":"2024-04-25T10:30:33.634903+0000","last_unstale":"2024-04-25T10:32:32.734513+0000","last_undegraded":"2024-04-25T10:32:32.734513+0000","last_fullsized":"2024-04-25T10:32:32.734513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:52.252812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'39","reported_seq":68,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.683867+0000","last_change":"2024-04-25T10:30:32.898979+0000","last_active":"2024-04-25T10:32:32.683867+0000","last_peered":"2024-04-25T10:32:32.683867+0000","last_clean":"2024-04-25T10:32:32.683867+0000","last_became_active":"2024-04-25T10:30:32.898611+0000","last_became_peered":"2024-04-25T10:30:32.898611+0000","last_unstale":"2024-04-25T10:32:32.683867+0000","last_undegraded":"2024-04-25T10:32:32.683867+0000","last_fullsized":"2024-04-25T10:32:32.683867+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.549769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734130+0000","last_change":"2024-04-25T10:30:33.651596+0000","last_active":"2024-04-25T10:32:32.734130+0000","last_peered":"2024-04-25T10:32:32.734130+0000","last_clean":"2024-04-25T10:32:32.734130+0000","last_became_active":"2024-04-25T10:30:33.651140+0000","last_became_peered":"2024-04-25T10:30:33.651140+0000","last_unstale":"2024-04-25T10:32:32.734130+0000","last_undegraded":"2024-04-25T10:32:32.734130+0000","last_fullsized":"2024-04-25T10:32:32.734130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:16.762135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869451+0000","last_change":"2024-04-25T10:30:32.909982+0000","last_active":"2024-04-25T10:32:32.869451+0000","last_peered":"2024-04-25T10:32:32.869451+0000","last_clean":"2024-04-25T10:32:32.869451+0000","last_became_active":"2024-04-25T10:30:32.909506+0000","last_became_peered":"2024-04-25T10:30:32.909506+0000","last_unstale":"2024-04-25T10:32:32.869451+0000","last_undegraded":"2024-04-25T10:32:32.869451+0000","last_fullsized":"2024-04-25T10:32:32.869451+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:47.396963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734492+0000","last_change":"2024-04-25T10:30:33.642913+0000","last_active":"2024-04-25T10:32:32.734492+0000","last_peered":"2024-04-25T10:32:32.734492+0000","last_clean":"2024-04-25T10:32:32.734492+0000","last_became_active":"2024-04-25T10:30:33.642614+0000","last_became_peered":"2024-04-25T10:30:33.642614+0000","last_unstale":"2024-04-25T10:32:32.734492+0000","last_undegraded":"2024-04-25T10:32:32.734492+0000","last_fullsized":"2024-04-25T10:32:32.734492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:56.651230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567112+0000","last_change":"2024-04-25T10:30:33.644525+0000","last_active":"2024-04-25T10:32:32.567112+0000","last_peered":"2024-04-25T10:32:32.567112+0000","last_clean":"2024-04-25T10:32:32.567112+0000","last_became_active":"2024-04-25T10:30:33.644188+0000","last_became_peered":"2024-04-25T10:30:33.644188+0000","last_unstale":"2024-04-25T10:32:32.567112+0000","last_undegraded":"2024-04-25T10:32:32.567112+0000","last_fullsized":"2024-04-25T10:32:32.567112+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:28.625695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"79'12","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734197+0000","last_change":"2024-04-25T10:31:33.946016+0000","last_active":"2024-04-25T10:32:32.734197+0000","last_peered":"2024-04-25T10:32:32.734197+0000","last_clean":"2024-04-25T10:32:32.734197+0000","last_became_active":"2024-04-25T10:31:33.945611+0000","last_became_peered":"2024-04-25T10:31:33.945611+0000","last_unstale":"2024-04-25T10:32:32.734197+0000","last_undegraded":"2024-04-25T10:32:32.734197+0000","last_fullsized":"2024-04-25T10:32:32.734197+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:02.352885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734068+0000","last_change":"2024-04-25T10:30:33.644721+0000","last_active":"2024-04-25T10:32:32.734068+0000","last_peered":"2024-04-25T10:32:32.734068+0000","last_clean":"2024-04-25T10:32:32.734068+0000","last_became_active":"2024-04-25T10:30:33.644392+0000","last_became_peered":"2024-04-25T10:30:33.644392+0000","last_unstale":"2024-04-25T10:32:32.734068+0000","last_undegraded":"2024-04-25T10:32:32.734068+0000","last_fullsized":"2024-04-25T10:32:32.734068+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:50.571318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.868724+0000","last_change":"2024-04-25T10:30:33.647970+0000","last_active":"2024-04-25T10:32:32.868724+0000","last_peered":"2024-04-25T10:32:32.868724+0000","last_clean":"2024-04-25T10:32:32.868724+0000","last_became_active":"2024-04-25T10:30:33.647547+0000","last_became_peered":"2024-04-25T10:30:33.647547+0000","last_unstale":"2024-04-25T10:32:32.868724+0000","last_undegraded":"2024-04-25T10:32:32.868724+0000","last_fullsized":"2024-04-25T10:32:32.868724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:25.609085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869336+0000","last_change":"2024-04-25T10:30:33.641618+0000","last_active":"2024-04-25T10:32:32.869336+0000","last_peered":"2024-04-25T10:32:32.869336+0000","last_clean":"2024-04-25T10:32:32.869336+0000","last_became_active":"2024-04-25T10:30:33.640830+0000","last_became_peered":"2024-04-25T10:30:33.640830+0000","last_unstale":"2024-04-25T10:32:32.869336+0000","last_undegraded":"2024-04-25T10:32:32.869336+0000","last_fullsized":"2024-04-25T10:32:32.869336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:36.870127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567255+0000","last_change":"2024-04-25T10:30:33.644010+0000","last_active":"2024-04-25T10:32:32.567255+0000","last_peered":"2024-04-25T10:32:32.567255+0000","last_clean":"2024-04-25T10:32:32.567255+0000","last_became_active":"2024-04-25T10:30:33.643639+0000","last_became_peered":"2024-04-25T10:30:33.643639+0000","last_unstale":"2024-04-25T10:32:32.567255+0000","last_undegraded":"2024-04-25T10:32:32.567255+0000","last_fullsized":"2024-04-25T10:32:32.567255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:32.820552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734555+0000","last_change":"2024-04-25T10:30:33.639727+0000","last_active":"2024-04-25T10:32:32.734555+0000","last_peered":"2024-04-25T10:32:32.734555+0000","last_clean":"2024-04-25T10:32:32.734555+0000","last_became_active":"2024-04-25T10:30:33.639185+0000","last_became_peered":"2024-04-25T10:30:33.639185+0000","last_unstale":"2024-04-25T10:32:32.734555+0000","last_undegraded":"2024-04-25T10:32:32.734555+0000","last_fullsized":"2024-04-25T10:32:32.734555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:17.860864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"79'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462081+0000","last_change":"2024-04-25T10:30:33.644650+0000","last_active":"2024-04-25T10:32:32.462081+0000","last_peered":"2024-04-25T10:32:32.462081+0000","last_clean":"2024-04-25T10:32:32.462081+0000","last_became_active":"2024-04-25T10:30:33.644264+0000","last_became_peered":"2024-04-25T10:30:33.644264+0000","last_unstale":"2024-04-25T10:32:32.462081+0000","last_undegraded":"2024-04-25T10:32:32.462081+0000","last_fullsized":"2024-04-25T10:32:32.462081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:41.122318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463411+0000","last_change":"2024-04-25T10:30:33.651875+0000","last_active":"2024-04-25T10:32:32.463411+0000","last_peered":"2024-04-25T10:32:32.463411+0000","last_clean":"2024-04-25T10:32:32.463411+0000","last_became_active":"2024-04-25T10:30:33.651486+0000","last_became_peered":"2024-04-25T10:30:33.651486+0000","last_unstale":"2024-04-25T10:32:32.463411+0000","last_undegraded":"2024-04-25T10:32:32.463411+0000","last_fullsized":"2024-04-25T10:32:32.463411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:32.598209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"79'8","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.562510+0000","last_change":"2024-04-25T10:31:33.942171+0000","last_active":"2024-04-25T10:32:32.562510+0000","last_peered":"2024-04-25T10:32:32.562510+0000","last_clean":"2024-04-25T10:32:32.562510+0000","last_became_active":"2024-04-25T10:31:33.941684+0000","last_became_peered":"2024-04-25T10:31:33.941684+0000","last_unstale":"2024-04-25T10:32:32.562510+0000","last_undegraded":"2024-04-25T10:32:32.562510+0000","last_fullsized":"2024-04-25T10:32:32.562510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:45.291952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462705+0000","last_change":"2024-04-25T10:30:33.639915+0000","last_active":"2024-04-25T10:32:32.462705+0000","last_peered":"2024-04-25T10:32:32.462705+0000","last_clean":"2024-04-25T10:32:32.462705+0000","last_became_active":"2024-04-25T10:30:33.639137+0000","last_became_peered":"2024-04-25T10:30:33.639137+0000","last_unstale":"2024-04-25T10:32:32.462705+0000","last_undegraded":"2024-04-25T10:32:32.462705+0000","last_fullsized":"2024-04-25T10:32:32.462705+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:13.143405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869541+0000","last_change":"2024-04-25T10:30:33.643547+0000","last_active":"2024-04-25T10:32:32.869541+0000","last_peered":"2024-04-25T10:32:32.869541+0000","last_clean":"2024-04-25T10:32:32.869541+0000","last_became_active":"2024-04-25T10:30:33.643158+0000","last_became_peered":"2024-04-25T10:30:33.643158+0000","last_unstale":"2024-04-25T10:32:32.869541+0000","last_undegraded":"2024-04-25T10:32:32.869541+0000","last_fullsized":"2024-04-25T10:32:32.869541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:07.517132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567168+0000","last_change":"2024-04-25T10:30:33.645746+0000","last_active":"2024-04-25T10:32:32.567168+0000","last_peered":"2024-04-25T10:32:32.567168+0000","last_clean":"2024-04-25T10:32:32.567168+0000","last_became_active":"2024-04-25T10:30:33.644646+0000","last_became_peered":"2024-04-25T10:30:33.644646+0000","last_unstale":"2024-04-25T10:32:32.567168+0000","last_undegraded":"2024-04-25T10:32:32.567168+0000","last_fullsized":"2024-04-25T10:32:32.567168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:21.591264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597107,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":809,"num_read_kb":514,"num_write":283,"num_write_kb":1433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":61068,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1791321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645350,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":856064,"data_stored":413481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841578,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37004,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708148,"statfs":{"total":95995035648,"available":95957143552,"internally_reserved":0,"allocated":1708032,"data_stored":1213757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037807,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36588,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708564,"statfs":{"total":95995035648,"available":95957569536,"internally_reserved":0,"allocated":1277952,"data_stored":815838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234036,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36584,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708568,"statfs":{"total":95995035648,"available":95957573632,"internally_reserved":0,"allocated":1273856,"data_stored":815736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430266,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712528,"statfs":{"total":95995035648,"available":95961628672,"internally_reserved":0,"allocated":1478656,"data_stored":1015004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626493,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":856064,"data_stored":413084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":942080,"data_stored":489562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40352,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704800,"statfs":{"total":95995035648,"available":95953715200,"internally_reserved":0,"allocated":876544,"data_stored":414355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":872448,"data_stored":414227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36720,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708432,"statfs":{"total":95995035648,"available":95957434368,"internally_reserved":0,"allocated":1417216,"data_stored":947138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986248,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":954368,"data_stored":490738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37104,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708048,"statfs":{"total":95995035648,"available":95957041152,"internally_reserved":0,"allocated":1626112,"data_stored":1145365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":201468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:32:37.507 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-25T10:32:37.507 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:32:37.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.654+0000 7fbf6c91f640 1 -- 172.21.15.43:0/2154313059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf6810d3a0 msgr2=0x7fbf6810f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:37.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.654+0000 7fbf6c91f640 1 --2- 172.21.15.43:0/2154313059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf6810d3a0 0x7fbf6810f7c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009ca0 tx=0x7fbf6002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.654+0000 7fbf6c91f640 1 -- 172.21.15.43:0/2154313059 shutdown_connections 2024-04-25T10:32:37.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.654+0000 7fbf6c91f640 1 --2- 172.21.15.43:0/2154313059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf6810d3a0 0x7fbf6810f7c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.654+0000 7fbf6c91f640 1 --2- 172.21.15.43:0/2154313059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf68102b60 0x7fbf68102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.654+0000 7fbf6c91f640 1 --2- 172.21.15.43:0/2154313059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf68108a40 0x7fbf68102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.654+0000 7fbf6c91f640 1 -- 172.21.15.43:0/2154313059 >> 172.21.15.43:0/2154313059 conn(0x7fbf68078040 msgr2=0x7fbf68078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:37.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.654+0000 7fbf6c91f640 1 -- 172.21.15.43:0/2154313059 shutdown_connections 2024-04-25T10:32:37.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.654+0000 7fbf6c91f640 1 -- 172.21.15.43:0/2154313059 wait complete. 2024-04-25T10:32:37.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.655+0000 7fbf6c91f640 1 Processor -- start 2024-04-25T10:32:37.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.655+0000 7fbf6c91f640 1 -- start start 2024-04-25T10:32:37.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf6c91f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf68102b60 0x7fbf6819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:37.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf6c91f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf68108a40 0x7fbf6819acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:37.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf6c91f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf6810d3a0 0x7fbf6819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:37.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf6c91f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbf68112c20 con 0x7fbf68102b60 2024-04-25T10:32:37.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf6c91f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbf68112aa0 con 0x7fbf6810d3a0 2024-04-25T10:32:37.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf6c91f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbf68112da0 con 0x7fbf68108a40 2024-04-25T10:32:37.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf6659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf68102b60 0x7fbf6819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:37.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf6659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf68102b60 0x7fbf6819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49144/0 (socket says 172.21.15.43:49144) 2024-04-25T10:32:37.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf6659c640 1 -- 172.21.15.43:0/4236729606 learned_addr learned my addr 172.21.15.43:0/4236729606 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:37.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf65d9b640 1 --2- 172.21.15.43:0/4236729606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf68108a40 0x7fbf6819acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:37.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf66d9d640 1 --2- 172.21.15.43:0/4236729606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf6810d3a0 0x7fbf6819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:37.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf6659c640 1 -- 172.21.15.43:0/4236729606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf68108a40 msgr2=0x7fbf6819acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:37.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf6659c640 1 --2- 172.21.15.43:0/4236729606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf68108a40 0x7fbf6819acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf6659c640 1 -- 172.21.15.43:0/4236729606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf6810d3a0 msgr2=0x7fbf6819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:37.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf6659c640 1 --2- 172.21.15.43:0/4236729606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf6810d3a0 0x7fbf6819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf6659c640 1 -- 172.21.15.43:0/4236729606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6819f5d0 con 0x7fbf68102b60 2024-04-25T10:32:37.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf6659c640 1 --2- 172.21.15.43:0/4236729606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf68102b60 0x7fbf6819a770 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800e9c0 tx=0x7fbf5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:37.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf577fe640 1 -- 172.21.15.43:0/4236729606 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5800cd80 con 0x7fbf68102b60 2024-04-25T10:32:37.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf577fe640 1 -- 172.21.15.43:0/4236729606 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf580054d0 con 0x7fbf68102b60 2024-04-25T10:32:37.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf6c91f640 1 -- 172.21.15.43:0/4236729606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6819f8c0 con 0x7fbf68102b60 2024-04-25T10:32:37.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf577fe640 1 -- 172.21.15.43:0/4236729606 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58010430 con 0x7fbf68102b60 2024-04-25T10:32:37.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.656+0000 7fbf6c91f640 1 -- 172.21.15.43:0/4236729606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf681a7240 con 0x7fbf68102b60 2024-04-25T10:32:37.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.657+0000 7fbf6c91f640 1 -- 172.21.15.43:0/4236729606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf6806a190 con 0x7fbf68102b60 2024-04-25T10:32:37.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.659+0000 7fbf577fe640 1 -- 172.21.15.43:0/4236729606 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 147747+0+0 (secure 0 0 0) 0x7fbf58004fb0 con 0x7fbf68102b60 2024-04-25T10:32:37.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.660+0000 7fbf577fe640 1 --2- 172.21.15.43:0/4236729606 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fbf500afe00 0x7fbf500b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:37.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.660+0000 7fbf65d9b640 1 --2- 172.21.15.43:0/4236729606 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fbf500afe00 0x7fbf500b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:37.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.660+0000 7fbf577fe640 1 -- 172.21.15.43:0/4236729606 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fbf580df970 con 0x7fbf68102b60 2024-04-25T10:32:37.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.660+0000 7fbf65d9b640 1 --2- 172.21.15.43:0/4236729606 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fbf500afe00 0x7fbf500b2310 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbf68101e70 tx=0x7fbf5c005f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:37.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.664+0000 7fbf577fe640 1 -- 172.21.15.43:0/4236729606 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf58053ff0 con 0x7fbf68102b60 2024-04-25T10:32:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.849+0000 7fbf6c91f640 1 -- 172.21.15.43:0/4236729606 --> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf680754e0 con 0x7fbf500afe00 2024-04-25T10:32:37.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.856+0000 7fbf577fe640 1 -- 172.21.15.43:0/4236729606 <== mgr.15291 v2:172.21.15.43:6800/3658924116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257726 (secure 0 0 0) 0x7fbf680754e0 con 0x7fbf500afe00 2024-04-25T10:32:37.861 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:32:37.862 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T10:32:37.863 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: Updating smithi043:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:32:37.863 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: Updating smithi077:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:32:37.863 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: Updating smithi164:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:32:37.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:32:37.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:32:37.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:32:37.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:32:37.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1446860079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:32:37.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: Updating smithi077:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:32:37.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: Updating smithi164:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:32:37.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: Updating smithi043:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:32:37.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T10:32:37.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:37.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:37.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:37.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:37.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:37.865 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:37.865 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:37.865 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:32:37.865 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:32:37.865 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:32:37.865 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:37 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:37.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.859+0000 7fbf6c91f640 1 -- 172.21.15.43:0/4236729606 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fbf500afe00 msgr2=0x7fbf500b2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:37.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.859+0000 7fbf6c91f640 1 --2- 172.21.15.43:0/4236729606 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fbf500afe00 0x7fbf500b2310 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbf68101e70 tx=0x7fbf5c005f90 comp rx=0 tx=0).stop 2024-04-25T10:32:37.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.859+0000 7fbf6c91f640 1 -- 172.21.15.43:0/4236729606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf68102b60 msgr2=0x7fbf6819a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:37.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.859+0000 7fbf6c91f640 1 --2- 172.21.15.43:0/4236729606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf68102b60 0x7fbf6819a770 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800e9c0 tx=0x7fbf5800ee80 comp rx=0 tx=0).stop 2024-04-25T10:32:37.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.859+0000 7fbf6c91f640 1 -- 172.21.15.43:0/4236729606 shutdown_connections 2024-04-25T10:32:37.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.859+0000 7fbf6c91f640 1 --2- 172.21.15.43:0/4236729606 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fbf500afe00 0x7fbf500b2310 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.859+0000 7fbf6c91f640 1 --2- 172.21.15.43:0/4236729606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf6810d3a0 0x7fbf6819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.859+0000 7fbf6c91f640 1 --2- 172.21.15.43:0/4236729606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf68108a40 0x7fbf6819acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.859+0000 7fbf6c91f640 1 --2- 172.21.15.43:0/4236729606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf68102b60 0x7fbf6819a770 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:37.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.859+0000 7fbf6c91f640 1 -- 172.21.15.43:0/4236729606 >> 172.21.15.43:0/4236729606 conn(0x7fbf68078040 msgr2=0x7fbf68074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:37.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.859+0000 7fbf6c91f640 1 -- 172.21.15.43:0/4236729606 shutdown_connections 2024-04-25T10:32:37.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:37.859+0000 7fbf6c91f640 1 -- 172.21.15.43:0/4236729606 wait complete. 2024-04-25T10:32:37.872 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-25T10:32:36.476276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187709,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":890,"num_read_kb":583,"num_write":408,"num_write_kb":3585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":3012908,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":816,"ondisk_log_size":816,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":429720,"kb_used_data":13808,"kb_used_omap":18,"kb_used_meta":415597,"kb_avail":1124512104,"statfs":{"total":1151940427776,"available":1151500394496,"internally_reserved":0,"allocated":14139392,"data_stored":8588285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":425571641},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.983241"},"pg_stats":[{"pgid":"3.3f","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567443+0000","last_change":"2024-04-25T10:30:33.645032+0000","last_active":"2024-04-25T10:32:32.567443+0000","last_peered":"2024-04-25T10:32:32.567443+0000","last_clean":"2024-04-25T10:32:32.567443+0000","last_became_active":"2024-04-25T10:30:33.644687+0000","last_became_peered":"2024-04-25T10:30:33.644687+0000","last_unstale":"2024-04-25T10:32:32.567443+0000","last_undegraded":"2024-04-25T10:32:32.567443+0000","last_fullsized":"2024-04-25T10:32:32.567443+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:39.893228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869077+0000","last_change":"2024-04-25T10:30:33.650466+0000","last_active":"2024-04-25T10:32:32.869077+0000","last_peered":"2024-04-25T10:32:32.869077+0000","last_clean":"2024-04-25T10:32:32.869077+0000","last_became_active":"2024-04-25T10:30:33.649889+0000","last_became_peered":"2024-04-25T10:30:33.649889+0000","last_unstale":"2024-04-25T10:32:32.869077+0000","last_undegraded":"2024-04-25T10:32:32.869077+0000","last_fullsized":"2024-04-25T10:32:32.869077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:53.832494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869024+0000","last_change":"2024-04-25T10:30:33.648798+0000","last_active":"2024-04-25T10:32:32.869024+0000","last_peered":"2024-04-25T10:32:32.869024+0000","last_clean":"2024-04-25T10:32:32.869024+0000","last_became_active":"2024-04-25T10:30:33.648423+0000","last_became_peered":"2024-04-25T10:30:33.648423+0000","last_unstale":"2024-04-25T10:32:32.869024+0000","last_undegraded":"2024-04-25T10:32:32.869024+0000","last_fullsized":"2024-04-25T10:32:32.869024+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:19.119189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"79'8","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.562870+0000","last_change":"2024-04-25T10:31:33.941691+0000","last_active":"2024-04-25T10:32:32.562870+0000","last_peered":"2024-04-25T10:32:32.562870+0000","last_clean":"2024-04-25T10:32:32.562870+0000","last_became_active":"2024-04-25T10:31:33.941304+0000","last_became_peered":"2024-04-25T10:31:33.941304+0000","last_unstale":"2024-04-25T10:32:32.562870+0000","last_undegraded":"2024-04-25T10:32:32.562870+0000","last_fullsized":"2024-04-25T10:32:32.562870+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:18.573452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.564974+0000","last_change":"2024-04-25T10:30:33.646786+0000","last_active":"2024-04-25T10:32:32.564974+0000","last_peered":"2024-04-25T10:32:32.564974+0000","last_clean":"2024-04-25T10:32:32.564974+0000","last_became_active":"2024-04-25T10:30:33.646456+0000","last_became_peered":"2024-04-25T10:30:33.646456+0000","last_unstale":"2024-04-25T10:32:32.564974+0000","last_undegraded":"2024-04-25T10:32:32.564974+0000","last_fullsized":"2024-04-25T10:32:32.564974+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:55.006723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.572822+0000","last_change":"2024-04-25T10:30:33.639952+0000","last_active":"2024-04-25T10:32:32.572822+0000","last_peered":"2024-04-25T10:32:32.572822+0000","last_clean":"2024-04-25T10:32:32.572822+0000","last_became_active":"2024-04-25T10:30:33.639558+0000","last_became_peered":"2024-04-25T10:30:33.639558+0000","last_unstale":"2024-04-25T10:32:32.572822+0000","last_undegraded":"2024-04-25T10:32:32.572822+0000","last_fullsized":"2024-04-25T10:32:32.572822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:17.162902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.733860+0000","last_change":"2024-04-25T10:30:33.651530+0000","last_active":"2024-04-25T10:32:32.733860+0000","last_peered":"2024-04-25T10:32:32.733860+0000","last_clean":"2024-04-25T10:32:32.733860+0000","last_became_active":"2024-04-25T10:30:33.651151+0000","last_became_peered":"2024-04-25T10:30:33.651151+0000","last_unstale":"2024-04-25T10:32:32.733860+0000","last_undegraded":"2024-04-25T10:32:32.733860+0000","last_fullsized":"2024-04-25T10:32:32.733860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:31.835367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.461672+0000","last_change":"2024-04-25T10:30:33.645146+0000","last_active":"2024-04-25T10:32:32.461672+0000","last_peered":"2024-04-25T10:32:32.461672+0000","last_clean":"2024-04-25T10:32:32.461672+0000","last_became_active":"2024-04-25T10:30:33.644310+0000","last_became_peered":"2024-04-25T10:30:33.644310+0000","last_unstale":"2024-04-25T10:32:32.461672+0000","last_undegraded":"2024-04-25T10:32:32.461672+0000","last_fullsized":"2024-04-25T10:32:32.461672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:45.034934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462982+0000","last_change":"2024-04-25T10:30:33.649385+0000","last_active":"2024-04-25T10:32:32.462982+0000","last_peered":"2024-04-25T10:32:32.462982+0000","last_clean":"2024-04-25T10:32:32.462982+0000","last_became_active":"2024-04-25T10:30:33.649002+0000","last_became_peered":"2024-04-25T10:30:33.649002+0000","last_unstale":"2024-04-25T10:32:32.462982+0000","last_undegraded":"2024-04-25T10:32:32.462982+0000","last_fullsized":"2024-04-25T10:32:32.462982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:42.259290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"79'16","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.573098+0000","last_change":"2024-04-25T10:30:33.641918+0000","last_active":"2024-04-25T10:32:32.573098+0000","last_peered":"2024-04-25T10:32:32.573098+0000","last_clean":"2024-04-25T10:32:32.573098+0000","last_became_active":"2024-04-25T10:30:33.641553+0000","last_became_peered":"2024-04-25T10:30:33.641553+0000","last_unstale":"2024-04-25T10:32:32.573098+0000","last_undegraded":"2024-04-25T10:32:32.573098+0000","last_fullsized":"2024-04-25T10:32:32.573098+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:04.755694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734376+0000","last_change":"2024-04-25T10:30:33.638054+0000","last_active":"2024-04-25T10:32:32.734376+0000","last_peered":"2024-04-25T10:32:32.734376+0000","last_clean":"2024-04-25T10:32:32.734376+0000","last_became_active":"2024-04-25T10:30:33.637720+0000","last_became_peered":"2024-04-25T10:30:33.637720+0000","last_unstale":"2024-04-25T10:32:32.734376+0000","last_undegraded":"2024-04-25T10:32:32.734376+0000","last_fullsized":"2024-04-25T10:32:32.734376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:41.557430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734895+0000","last_change":"2024-04-25T10:30:33.642705+0000","last_active":"2024-04-25T10:32:32.734895+0000","last_peered":"2024-04-25T10:32:32.734895+0000","last_clean":"2024-04-25T10:32:32.734895+0000","last_became_active":"2024-04-25T10:30:33.641936+0000","last_became_peered":"2024-04-25T10:30:33.641936+0000","last_unstale":"2024-04-25T10:32:32.734895+0000","last_undegraded":"2024-04-25T10:32:32.734895+0000","last_fullsized":"2024-04-25T10:32:32.734895+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:36.450389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869566+0000","last_change":"2024-04-25T10:30:33.648987+0000","last_active":"2024-04-25T10:32:32.869566+0000","last_peered":"2024-04-25T10:32:32.869566+0000","last_clean":"2024-04-25T10:32:32.869566+0000","last_became_active":"2024-04-25T10:30:33.648621+0000","last_became_peered":"2024-04-25T10:30:33.648621+0000","last_unstale":"2024-04-25T10:32:32.869566+0000","last_undegraded":"2024-04-25T10:32:32.869566+0000","last_fullsized":"2024-04-25T10:32:32.869566+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:19.999002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.868610+0000","last_change":"2024-04-25T10:30:33.647360+0000","last_active":"2024-04-25T10:32:32.868610+0000","last_peered":"2024-04-25T10:32:32.868610+0000","last_clean":"2024-04-25T10:32:32.868610+0000","last_became_active":"2024-04-25T10:30:33.646719+0000","last_became_peered":"2024-04-25T10:30:33.646719+0000","last_unstale":"2024-04-25T10:32:32.868610+0000","last_undegraded":"2024-04-25T10:32:32.868610+0000","last_fullsized":"2024-04-25T10:32:32.868610+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:19.016333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.868373+0000","last_change":"2024-04-25T10:30:33.646503+0000","last_active":"2024-04-25T10:32:32.868373+0000","last_peered":"2024-04-25T10:32:32.868373+0000","last_clean":"2024-04-25T10:32:32.868373+0000","last_became_active":"2024-04-25T10:30:33.646123+0000","last_became_peered":"2024-04-25T10:30:33.646123+0000","last_unstale":"2024-04-25T10:32:32.868373+0000","last_undegraded":"2024-04-25T10:32:32.868373+0000","last_fullsized":"2024-04-25T10:32:32.868373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:57.712216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462498+0000","last_change":"2024-04-25T10:30:33.640463+0000","last_active":"2024-04-25T10:32:32.462498+0000","last_peered":"2024-04-25T10:32:32.462498+0000","last_clean":"2024-04-25T10:32:32.462498+0000","last_became_active":"2024-04-25T10:30:33.639931+0000","last_became_peered":"2024-04-25T10:30:33.639931+0000","last_unstale":"2024-04-25T10:32:32.462498+0000","last_undegraded":"2024-04-25T10:32:32.462498+0000","last_fullsized":"2024-04-25T10:32:32.462498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:18.863405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567342+0000","last_change":"2024-04-25T10:30:33.645695+0000","last_active":"2024-04-25T10:32:32.567342+0000","last_peered":"2024-04-25T10:32:32.567342+0000","last_clean":"2024-04-25T10:32:32.567342+0000","last_became_active":"2024-04-25T10:30:33.645387+0000","last_became_peered":"2024-04-25T10:30:33.645387+0000","last_unstale":"2024-04-25T10:32:32.567342+0000","last_undegraded":"2024-04-25T10:32:32.567342+0000","last_fullsized":"2024-04-25T10:32:32.567342+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:20.044456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.562870+0000","last_change":"2024-04-25T10:30:33.639087+0000","last_active":"2024-04-25T10:32:32.562870+0000","last_peered":"2024-04-25T10:32:32.562870+0000","last_clean":"2024-04-25T10:32:32.562870+0000","last_became_active":"2024-04-25T10:30:33.638726+0000","last_became_peered":"2024-04-25T10:30:33.638726+0000","last_unstale":"2024-04-25T10:32:32.562870+0000","last_undegraded":"2024-04-25T10:32:32.562870+0000","last_fullsized":"2024-04-25T10:32:32.562870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:36.887289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734245+0000","last_change":"2024-04-25T10:30:33.638646+0000","last_active":"2024-04-25T10:32:32.734245+0000","last_peered":"2024-04-25T10:32:32.734245+0000","last_clean":"2024-04-25T10:32:32.734245+0000","last_became_active":"2024-04-25T10:30:33.638261+0000","last_became_peered":"2024-04-25T10:30:33.638261+0000","last_unstale":"2024-04-25T10:32:32.734245+0000","last_undegraded":"2024-04-25T10:32:32.734245+0000","last_fullsized":"2024-04-25T10:32:32.734245+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:08.212310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.563028+0000","last_change":"2024-04-25T10:30:32.899992+0000","last_active":"2024-04-25T10:32:32.563028+0000","last_peered":"2024-04-25T10:32:32.563028+0000","last_clean":"2024-04-25T10:32:32.563028+0000","last_became_active":"2024-04-25T10:30:32.899643+0000","last_became_peered":"2024-04-25T10:30:32.899643+0000","last_unstale":"2024-04-25T10:32:32.563028+0000","last_undegraded":"2024-04-25T10:32:32.563028+0000","last_fullsized":"2024-04-25T10:32:32.563028+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:34.245121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462175+0000","last_change":"2024-04-25T10:30:33.644884+0000","last_active":"2024-04-25T10:32:32.462175+0000","last_peered":"2024-04-25T10:32:32.462175+0000","last_clean":"2024-04-25T10:32:32.462175+0000","last_became_active":"2024-04-25T10:30:33.644510+0000","last_became_peered":"2024-04-25T10:30:33.644510+0000","last_unstale":"2024-04-25T10:32:32.462175+0000","last_undegraded":"2024-04-25T10:32:32.462175+0000","last_fullsized":"2024-04-25T10:32:32.462175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:09.282865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462247+0000","last_change":"2024-04-25T10:30:32.909175+0000","last_active":"2024-04-25T10:32:32.462247+0000","last_peered":"2024-04-25T10:32:32.462247+0000","last_clean":"2024-04-25T10:32:32.462247+0000","last_became_active":"2024-04-25T10:30:32.908813+0000","last_became_peered":"2024-04-25T10:30:32.908813+0000","last_unstale":"2024-04-25T10:32:32.462247+0000","last_undegraded":"2024-04-25T10:32:32.462247+0000","last_fullsized":"2024-04-25T10:32:32.462247+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:04.256452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869245+0000","last_change":"2024-04-25T10:30:33.642095+0000","last_active":"2024-04-25T10:32:32.869245+0000","last_peered":"2024-04-25T10:32:32.869245+0000","last_clean":"2024-04-25T10:32:32.869245+0000","last_became_active":"2024-04-25T10:30:33.641713+0000","last_became_peered":"2024-04-25T10:30:33.641713+0000","last_unstale":"2024-04-25T10:32:32.869245+0000","last_undegraded":"2024-04-25T10:32:32.869245+0000","last_fullsized":"2024-04-25T10:32:32.869245+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:42.425197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.565390+0000","last_change":"2024-04-25T10:30:32.904376+0000","last_active":"2024-04-25T10:32:32.565390+0000","last_peered":"2024-04-25T10:32:32.565390+0000","last_clean":"2024-04-25T10:32:32.565390+0000","last_became_active":"2024-04-25T10:30:32.903994+0000","last_became_peered":"2024-04-25T10:30:32.903994+0000","last_unstale":"2024-04-25T10:32:32.565390+0000","last_undegraded":"2024-04-25T10:32:32.565390+0000","last_fullsized":"2024-04-25T10:32:32.565390+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:52.727800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462581+0000","last_change":"2024-04-25T10:30:33.637171+0000","last_active":"2024-04-25T10:32:32.462581+0000","last_peered":"2024-04-25T10:32:32.462581+0000","last_clean":"2024-04-25T10:32:32.462581+0000","last_became_active":"2024-04-25T10:30:33.636822+0000","last_became_peered":"2024-04-25T10:30:33.636822+0000","last_unstale":"2024-04-25T10:32:32.462581+0000","last_undegraded":"2024-04-25T10:32:32.462581+0000","last_fullsized":"2024-04-25T10:32:32.462581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:54.220784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.565240+0000","last_change":"2024-04-25T10:30:32.900856+0000","last_active":"2024-04-25T10:32:32.565240+0000","last_peered":"2024-04-25T10:32:32.565240+0000","last_clean":"2024-04-25T10:32:32.565240+0000","last_became_active":"2024-04-25T10:30:32.900519+0000","last_became_peered":"2024-04-25T10:30:32.900519+0000","last_unstale":"2024-04-25T10:32:32.565240+0000","last_undegraded":"2024-04-25T10:32:32.565240+0000","last_fullsized":"2024-04-25T10:32:32.565240+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:49.738005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463759+0000","last_change":"2024-04-25T10:30:33.651736+0000","last_active":"2024-04-25T10:32:32.463759+0000","last_peered":"2024-04-25T10:32:32.463759+0000","last_clean":"2024-04-25T10:32:32.463759+0000","last_became_active":"2024-04-25T10:30:33.651395+0000","last_became_peered":"2024-04-25T10:30:33.651395+0000","last_unstale":"2024-04-25T10:32:32.463759+0000","last_undegraded":"2024-04-25T10:32:32.463759+0000","last_fullsized":"2024-04-25T10:32:32.463759+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:58.230092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":146,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.982457+0000","last_change":"2024-04-25T10:30:32.901348+0000","last_active":"2024-04-25T10:32:32.982457+0000","last_peered":"2024-04-25T10:32:32.982457+0000","last_clean":"2024-04-25T10:32:32.982457+0000","last_became_active":"2024-04-25T10:30:32.901025+0000","last_became_peered":"2024-04-25T10:30:32.901025+0000","last_unstale":"2024-04-25T10:32:32.982457+0000","last_undegraded":"2024-04-25T10:32:32.982457+0000","last_fullsized":"2024-04-25T10:32:32.982457+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:00.521934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.565033+0000","last_change":"2024-04-25T10:30:33.649206+0000","last_active":"2024-04-25T10:32:32.565033+0000","last_peered":"2024-04-25T10:32:32.565033+0000","last_clean":"2024-04-25T10:32:32.565033+0000","last_became_active":"2024-04-25T10:30:33.648883+0000","last_became_peered":"2024-04-25T10:30:33.648883+0000","last_unstale":"2024-04-25T10:32:32.565033+0000","last_undegraded":"2024-04-25T10:32:32.565033+0000","last_fullsized":"2024-04-25T10:32:32.565033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:43.105562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.572768+0000","last_change":"2024-04-25T10:30:32.901201+0000","last_active":"2024-04-25T10:32:32.572768+0000","last_peered":"2024-04-25T10:32:32.572768+0000","last_clean":"2024-04-25T10:32:32.572768+0000","last_became_active":"2024-04-25T10:30:32.900876+0000","last_became_peered":"2024-04-25T10:30:32.900876+0000","last_unstale":"2024-04-25T10:32:32.572768+0000","last_undegraded":"2024-04-25T10:32:32.572768+0000","last_fullsized":"2024-04-25T10:32:32.572768+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:58.077045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463571+0000","last_change":"2024-04-25T10:30:33.651935+0000","last_active":"2024-04-25T10:32:32.463571+0000","last_peered":"2024-04-25T10:32:32.463571+0000","last_clean":"2024-04-25T10:32:32.463571+0000","last_became_active":"2024-04-25T10:30:33.651568+0000","last_became_peered":"2024-04-25T10:30:33.651568+0000","last_unstale":"2024-04-25T10:32:32.463571+0000","last_undegraded":"2024-04-25T10:32:32.463571+0000","last_fullsized":"2024-04-25T10:32:32.463571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:58.849535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869003+0000","last_change":"2024-04-25T10:30:32.896500+0000","last_active":"2024-04-25T10:32:32.869003+0000","last_peered":"2024-04-25T10:32:32.869003+0000","last_clean":"2024-04-25T10:32:32.869003+0000","last_became_active":"2024-04-25T10:30:32.896071+0000","last_became_peered":"2024-04-25T10:30:32.896071+0000","last_unstale":"2024-04-25T10:32:32.869003+0000","last_undegraded":"2024-04-25T10:32:32.869003+0000","last_fullsized":"2024-04-25T10:32:32.869003+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:10.385443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.461754+0000","last_change":"2024-04-25T10:30:33.631730+0000","last_active":"2024-04-25T10:32:32.461754+0000","last_peered":"2024-04-25T10:32:32.461754+0000","last_clean":"2024-04-25T10:32:32.461754+0000","last_became_active":"2024-04-25T10:30:33.631374+0000","last_became_peered":"2024-04-25T10:30:33.631374+0000","last_unstale":"2024-04-25T10:32:32.461754+0000","last_undegraded":"2024-04-25T10:32:32.461754+0000","last_fullsized":"2024-04-25T10:32:32.461754+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:14.957866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463314+0000","last_change":"2024-04-25T10:30:32.900762+0000","last_active":"2024-04-25T10:32:32.463314+0000","last_peered":"2024-04-25T10:32:32.463314+0000","last_clean":"2024-04-25T10:32:32.463314+0000","last_became_active":"2024-04-25T10:30:32.900459+0000","last_became_peered":"2024-04-25T10:30:32.900459+0000","last_unstale":"2024-04-25T10:32:32.463314+0000","last_undegraded":"2024-04-25T10:32:32.463314+0000","last_fullsized":"2024-04-25T10:32:32.463314+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:22.286261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.562715+0000","last_change":"2024-04-25T10:30:33.637727+0000","last_active":"2024-04-25T10:32:32.562715+0000","last_peered":"2024-04-25T10:32:32.562715+0000","last_clean":"2024-04-25T10:32:32.562715+0000","last_became_active":"2024-04-25T10:30:33.637359+0000","last_became_peered":"2024-04-25T10:30:33.637359+0000","last_unstale":"2024-04-25T10:32:32.562715+0000","last_undegraded":"2024-04-25T10:32:32.562715+0000","last_fullsized":"2024-04-25T10:32:32.562715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:21.877562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":141,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.733608+0000","last_change":"2024-04-25T10:30:32.894210+0000","last_active":"2024-04-25T10:32:32.733608+0000","last_peered":"2024-04-25T10:32:32.733608+0000","last_clean":"2024-04-25T10:32:32.733608+0000","last_became_active":"2024-04-25T10:30:32.893241+0000","last_became_peered":"2024-04-25T10:30:32.893241+0000","last_unstale":"2024-04-25T10:32:32.733608+0000","last_undegraded":"2024-04-25T10:32:32.733608+0000","last_fullsized":"2024-04-25T10:32:32.733608+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:06.340392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463016+0000","last_change":"2024-04-25T10:30:33.632349+0000","last_active":"2024-04-25T10:32:32.463016+0000","last_peered":"2024-04-25T10:32:32.463016+0000","last_clean":"2024-04-25T10:32:32.463016+0000","last_became_active":"2024-04-25T10:30:33.631968+0000","last_became_peered":"2024-04-25T10:30:33.631968+0000","last_unstale":"2024-04-25T10:32:32.463016+0000","last_undegraded":"2024-04-25T10:32:32.463016+0000","last_fullsized":"2024-04-25T10:32:32.463016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:29.090932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734093+0000","last_change":"2024-04-25T10:30:32.900105+0000","last_active":"2024-04-25T10:32:32.734093+0000","last_peered":"2024-04-25T10:32:32.734093+0000","last_clean":"2024-04-25T10:32:32.734093+0000","last_became_active":"2024-04-25T10:30:32.899740+0000","last_became_peered":"2024-04-25T10:30:32.899740+0000","last_unstale":"2024-04-25T10:32:32.734093+0000","last_undegraded":"2024-04-25T10:32:32.734093+0000","last_fullsized":"2024-04-25T10:32:32.734093+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:49.390799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"79'9","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869399+0000","last_change":"2024-04-25T10:31:33.943769+0000","last_active":"2024-04-25T10:32:32.869399+0000","last_peered":"2024-04-25T10:32:32.869399+0000","last_clean":"2024-04-25T10:32:32.869399+0000","last_became_active":"2024-04-25T10:31:33.943331+0000","last_became_peered":"2024-04-25T10:31:33.943331+0000","last_unstale":"2024-04-25T10:32:32.869399+0000","last_undegraded":"2024-04-25T10:32:32.869399+0000","last_fullsized":"2024-04-25T10:32:32.869399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:07.183059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463831+0000","last_change":"2024-04-25T10:30:32.893453+0000","last_active":"2024-04-25T10:32:32.463831+0000","last_peered":"2024-04-25T10:32:32.463831+0000","last_clean":"2024-04-25T10:32:32.463831+0000","last_became_active":"2024-04-25T10:30:32.893106+0000","last_became_peered":"2024-04-25T10:30:32.893106+0000","last_unstale":"2024-04-25T10:32:32.463831+0000","last_undegraded":"2024-04-25T10:32:32.463831+0000","last_fullsized":"2024-04-25T10:32:32.463831+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:57.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462772+0000","last_change":"2024-04-25T10:30:33.645693+0000","last_active":"2024-04-25T10:32:32.462772+0000","last_peered":"2024-04-25T10:32:32.462772+0000","last_clean":"2024-04-25T10:32:32.462772+0000","last_became_active":"2024-04-25T10:30:33.645342+0000","last_became_peered":"2024-04-25T10:30:33.645342+0000","last_unstale":"2024-04-25T10:32:32.462772+0000","last_undegraded":"2024-04-25T10:32:32.462772+0000","last_fullsized":"2024-04-25T10:32:32.462772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:15.729860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'35","reported_seq":317,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:33.028479+0000","last_change":"2024-04-25T10:31:33.949796+0000","last_active":"2024-04-25T10:32:33.028479+0000","last_peered":"2024-04-25T10:32:33.028479+0000","last_clean":"2024-04-25T10:32:33.028479+0000","last_became_active":"2024-04-25T10:31:33.942989+0000","last_became_peered":"2024-04-25T10:31:33.942989+0000","last_unstale":"2024-04-25T10:32:33.028479+0000","last_undegraded":"2024-04-25T10:32:33.028479+0000","last_fullsized":"2024-04-25T10:32:33.028479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:04.194643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67758,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":235,"num_read_kb":184,"num_write":36,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61046,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.565318+0000","last_change":"2024-04-25T10:30:33.641768+0000","last_active":"2024-04-25T10:32:32.565318+0000","last_peered":"2024-04-25T10:32:32.565318+0000","last_clean":"2024-04-25T10:32:32.565318+0000","last_became_active":"2024-04-25T10:30:33.641415+0000","last_became_peered":"2024-04-25T10:30:33.641415+0000","last_unstale":"2024-04-25T10:32:32.565318+0000","last_undegraded":"2024-04-25T10:32:32.565318+0000","last_fullsized":"2024-04-25T10:32:32.565318+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:38.569474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734173+0000","last_change":"2024-04-25T10:30:32.898703+0000","last_active":"2024-04-25T10:32:32.734173+0000","last_peered":"2024-04-25T10:32:32.734173+0000","last_clean":"2024-04-25T10:32:32.734173+0000","last_became_active":"2024-04-25T10:30:32.898340+0000","last_became_peered":"2024-04-25T10:30:32.898340+0000","last_unstale":"2024-04-25T10:32:32.734173+0000","last_undegraded":"2024-04-25T10:32:32.734173+0000","last_fullsized":"2024-04-25T10:32:32.734173+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:27.024144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869241+0000","last_change":"2024-04-25T10:30:33.640393+0000","last_active":"2024-04-25T10:32:32.869241+0000","last_peered":"2024-04-25T10:32:32.869241+0000","last_clean":"2024-04-25T10:32:32.869241+0000","last_became_active":"2024-04-25T10:30:33.639822+0000","last_became_peered":"2024-04-25T10:30:33.639822+0000","last_unstale":"2024-04-25T10:32:32.869241+0000","last_undegraded":"2024-04-25T10:32:32.869241+0000","last_fullsized":"2024-04-25T10:32:32.869241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:08.618406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463750+0000","last_change":"2024-04-25T10:30:32.900880+0000","last_active":"2024-04-25T10:32:32.463750+0000","last_peered":"2024-04-25T10:32:32.463750+0000","last_clean":"2024-04-25T10:32:32.463750+0000","last_became_active":"2024-04-25T10:30:32.899987+0000","last_became_peered":"2024-04-25T10:30:32.899987+0000","last_unstale":"2024-04-25T10:32:32.463750+0000","last_undegraded":"2024-04-25T10:32:32.463750+0000","last_fullsized":"2024-04-25T10:32:32.463750+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:35.687773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734321+0000","last_change":"2024-04-25T10:30:33.641933+0000","last_active":"2024-04-25T10:32:32.734321+0000","last_peered":"2024-04-25T10:32:32.734321+0000","last_clean":"2024-04-25T10:32:32.734321+0000","last_became_active":"2024-04-25T10:30:33.641549+0000","last_became_peered":"2024-04-25T10:30:33.641549+0000","last_unstale":"2024-04-25T10:32:32.734321+0000","last_undegraded":"2024-04-25T10:32:32.734321+0000","last_fullsized":"2024-04-25T10:32:32.734321+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:36.696523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'5","reported_seq":145,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734534+0000","last_change":"2024-04-25T10:30:32.901178+0000","last_active":"2024-04-25T10:32:32.734534+0000","last_peered":"2024-04-25T10:32:32.734534+0000","last_clean":"2024-04-25T10:32:32.734534+0000","last_became_active":"2024-04-25T10:30:32.900817+0000","last_became_peered":"2024-04-25T10:30:32.900817+0000","last_unstale":"2024-04-25T10:32:32.734534+0000","last_undegraded":"2024-04-25T10:32:32.734534+0000","last_fullsized":"2024-04-25T10:32:32.734534+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:53.999253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"79'6","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.733811+0000","last_change":"2024-04-25T10:31:33.943470+0000","last_active":"2024-04-25T10:32:32.733811+0000","last_peered":"2024-04-25T10:32:32.733811+0000","last_clean":"2024-04-25T10:32:32.733811+0000","last_became_active":"2024-04-25T10:31:33.943098+0000","last_became_peered":"2024-04-25T10:31:33.943098+0000","last_unstale":"2024-04-25T10:32:32.733811+0000","last_undegraded":"2024-04-25T10:32:32.733811+0000","last_fullsized":"2024-04-25T10:32:32.733811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:27.707195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.564919+0000","last_change":"2024-04-25T10:30:32.902862+0000","last_active":"2024-04-25T10:32:32.564919+0000","last_peered":"2024-04-25T10:32:32.564919+0000","last_clean":"2024-04-25T10:32:32.564919+0000","last_became_active":"2024-04-25T10:30:32.902537+0000","last_became_peered":"2024-04-25T10:30:32.902537+0000","last_unstale":"2024-04-25T10:32:32.564919+0000","last_undegraded":"2024-04-25T10:32:32.564919+0000","last_fullsized":"2024-04-25T10:32:32.564919+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:55.974128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463465+0000","last_change":"2024-04-25T10:30:33.651409+0000","last_active":"2024-04-25T10:32:32.463465+0000","last_peered":"2024-04-25T10:32:32.463465+0000","last_clean":"2024-04-25T10:32:32.463465+0000","last_became_active":"2024-04-25T10:30:33.650673+0000","last_became_peered":"2024-04-25T10:30:33.650673+0000","last_unstale":"2024-04-25T10:32:32.463465+0000","last_undegraded":"2024-04-25T10:32:32.463465+0000","last_fullsized":"2024-04-25T10:32:32.463465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:48.989198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567343+0000","last_change":"2024-04-25T10:31:33.953408+0000","last_active":"2024-04-25T10:32:32.567343+0000","last_peered":"2024-04-25T10:32:32.567343+0000","last_clean":"2024-04-25T10:32:32.567343+0000","last_became_active":"2024-04-25T10:31:33.943495+0000","last_became_peered":"2024-04-25T10:31:33.943495+0000","last_unstale":"2024-04-25T10:32:32.567343+0000","last_undegraded":"2024-04-25T10:32:32.567343+0000","last_fullsized":"2024-04-25T10:32:32.567343+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:03.947967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567280+0000","last_change":"2024-04-25T10:30:33.643929+0000","last_active":"2024-04-25T10:32:32.567280+0000","last_peered":"2024-04-25T10:32:32.567280+0000","last_clean":"2024-04-25T10:32:32.567280+0000","last_became_active":"2024-04-25T10:30:33.643593+0000","last_became_peered":"2024-04-25T10:30:33.643593+0000","last_unstale":"2024-04-25T10:32:32.567280+0000","last_undegraded":"2024-04-25T10:32:32.567280+0000","last_fullsized":"2024-04-25T10:32:32.567280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:39.591022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.562806+0000","last_change":"2024-04-25T10:30:32.898937+0000","last_active":"2024-04-25T10:32:32.562806+0000","last_peered":"2024-04-25T10:32:32.562806+0000","last_clean":"2024-04-25T10:32:32.562806+0000","last_became_active":"2024-04-25T10:30:32.898570+0000","last_became_peered":"2024-04-25T10:30:32.898570+0000","last_unstale":"2024-04-25T10:32:32.562806+0000","last_undegraded":"2024-04-25T10:32:32.562806+0000","last_fullsized":"2024-04-25T10:32:32.562806+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:40.123035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.573011+0000","last_change":"2024-04-25T10:30:33.632857+0000","last_active":"2024-04-25T10:32:32.573011+0000","last_peered":"2024-04-25T10:32:32.573011+0000","last_clean":"2024-04-25T10:32:32.573011+0000","last_became_active":"2024-04-25T10:30:33.632043+0000","last_became_peered":"2024-04-25T10:30:33.632043+0000","last_unstale":"2024-04-25T10:32:32.573011+0000","last_undegraded":"2024-04-25T10:32:32.573011+0000","last_fullsized":"2024-04-25T10:32:32.573011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:12.683035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734027+0000","last_change":"2024-04-25T10:30:32.901349+0000","last_active":"2024-04-25T10:32:32.734027+0000","last_peered":"2024-04-25T10:32:32.734027+0000","last_clean":"2024-04-25T10:32:32.734027+0000","last_became_active":"2024-04-25T10:30:32.900881+0000","last_became_peered":"2024-04-25T10:30:32.900881+0000","last_unstale":"2024-04-25T10:32:32.734027+0000","last_undegraded":"2024-04-25T10:32:32.734027+0000","last_fullsized":"2024-04-25T10:32:32.734027+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:34.963564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.572921+0000","last_change":"2024-04-25T10:30:33.644376+0000","last_active":"2024-04-25T10:32:32.572921+0000","last_peered":"2024-04-25T10:32:32.572921+0000","last_clean":"2024-04-25T10:32:32.572921+0000","last_became_active":"2024-04-25T10:30:33.644055+0000","last_became_peered":"2024-04-25T10:30:33.644055+0000","last_unstale":"2024-04-25T10:32:32.572921+0000","last_undegraded":"2024-04-25T10:32:32.572921+0000","last_fullsized":"2024-04-25T10:32:32.572921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:48.134775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"79'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463526+0000","last_change":"2024-04-25T10:30:33.646312+0000","last_active":"2024-04-25T10:32:32.463526+0000","last_peered":"2024-04-25T10:32:32.463526+0000","last_clean":"2024-04-25T10:32:32.463526+0000","last_became_active":"2024-04-25T10:30:33.642338+0000","last_became_peered":"2024-04-25T10:30:33.642338+0000","last_unstale":"2024-04-25T10:32:32.463526+0000","last_undegraded":"2024-04-25T10:32:32.463526+0000","last_fullsized":"2024-04-25T10:32:32.463526+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:42.588360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":141,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.868833+0000","last_change":"2024-04-25T10:30:32.909680+0000","last_active":"2024-04-25T10:32:32.868833+0000","last_peered":"2024-04-25T10:32:32.868833+0000","last_clean":"2024-04-25T10:32:32.868833+0000","last_became_active":"2024-04-25T10:30:32.909169+0000","last_became_peered":"2024-04-25T10:30:32.909169+0000","last_unstale":"2024-04-25T10:32:32.868833+0000","last_undegraded":"2024-04-25T10:32:32.868833+0000","last_fullsized":"2024-04-25T10:32:32.868833+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:54.876475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.461954+0000","last_change":"2024-04-25T10:30:33.649524+0000","last_active":"2024-04-25T10:32:32.461954+0000","last_peered":"2024-04-25T10:32:32.461954+0000","last_clean":"2024-04-25T10:32:32.461954+0000","last_became_active":"2024-04-25T10:30:33.648958+0000","last_became_peered":"2024-04-25T10:30:33.648958+0000","last_unstale":"2024-04-25T10:32:32.461954+0000","last_undegraded":"2024-04-25T10:32:32.461954+0000","last_fullsized":"2024-04-25T10:32:32.461954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:13.407742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734705+0000","last_change":"2024-04-25T10:30:32.895270+0000","last_active":"2024-04-25T10:32:32.734705+0000","last_peered":"2024-04-25T10:32:32.734705+0000","last_clean":"2024-04-25T10:32:32.734705+0000","last_became_active":"2024-04-25T10:30:32.894899+0000","last_became_peered":"2024-04-25T10:30:32.894899+0000","last_unstale":"2024-04-25T10:32:32.734705+0000","last_undegraded":"2024-04-25T10:32:32.734705+0000","last_fullsized":"2024-04-25T10:32:32.734705+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:21.233217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.733901+0000","last_change":"2024-04-25T10:29:21.105073+0000","last_active":"2024-04-25T10:32:32.733901+0000","last_peered":"2024-04-25T10:32:32.733901+0000","last_clean":"2024-04-25T10:32:32.733901+0000","last_became_active":"2024-04-25T10:29:21.098771+0000","last_became_peered":"2024-04-25T10:29:21.098771+0000","last_unstale":"2024-04-25T10:32:32.733901+0000","last_undegraded":"2024-04-25T10:32:32.733901+0000","last_fullsized":"2024-04-25T10:32:32.733901+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:59.015440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:59.015440+0000","last_clean_scrub_stamp":"2024-04-25T10:25:59.015440+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:41.728977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.868619+0000","last_change":"2024-04-25T10:30:33.639159+0000","last_active":"2024-04-25T10:32:32.868619+0000","last_peered":"2024-04-25T10:32:32.868619+0000","last_clean":"2024-04-25T10:32:32.868619+0000","last_became_active":"2024-04-25T10:30:33.638761+0000","last_became_peered":"2024-04-25T10:30:33.638761+0000","last_unstale":"2024-04-25T10:32:32.868619+0000","last_undegraded":"2024-04-25T10:32:32.868619+0000","last_fullsized":"2024-04-25T10:32:32.868619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:43.262103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734611+0000","last_change":"2024-04-25T10:30:32.898587+0000","last_active":"2024-04-25T10:32:32.734611+0000","last_peered":"2024-04-25T10:32:32.734611+0000","last_clean":"2024-04-25T10:32:32.734611+0000","last_became_active":"2024-04-25T10:30:32.898113+0000","last_became_peered":"2024-04-25T10:30:32.898113+0000","last_unstale":"2024-04-25T10:32:32.734611+0000","last_undegraded":"2024-04-25T10:32:32.734611+0000","last_fullsized":"2024-04-25T10:32:32.734611+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:24.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.565148+0000","last_change":"2024-04-25T10:30:32.907188+0000","last_active":"2024-04-25T10:32:32.565148+0000","last_peered":"2024-04-25T10:32:32.565148+0000","last_clean":"2024-04-25T10:32:32.565148+0000","last_became_active":"2024-04-25T10:30:32.906837+0000","last_became_peered":"2024-04-25T10:30:32.906837+0000","last_unstale":"2024-04-25T10:32:32.565148+0000","last_undegraded":"2024-04-25T10:32:32.565148+0000","last_fullsized":"2024-04-25T10:32:32.565148+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:11.835053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869498+0000","last_change":"2024-04-25T10:30:33.636159+0000","last_active":"2024-04-25T10:32:32.869498+0000","last_peered":"2024-04-25T10:32:32.869498+0000","last_clean":"2024-04-25T10:32:32.869498+0000","last_became_active":"2024-04-25T10:30:33.635098+0000","last_became_peered":"2024-04-25T10:30:33.635098+0000","last_unstale":"2024-04-25T10:32:32.869498+0000","last_undegraded":"2024-04-25T10:32:32.869498+0000","last_fullsized":"2024-04-25T10:32:32.869498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:13.092368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'45","reported_seq":89,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.689001+0000","last_change":"2024-04-25T10:30:32.905469+0000","last_active":"2024-04-25T10:32:32.689001+0000","last_peered":"2024-04-25T10:32:32.689001+0000","last_clean":"2024-04-25T10:32:32.689001+0000","last_became_active":"2024-04-25T10:30:32.905123+0000","last_became_peered":"2024-04-25T10:30:32.905123+0000","last_unstale":"2024-04-25T10:32:32.689001+0000","last_undegraded":"2024-04-25T10:32:32.689001+0000","last_fullsized":"2024-04-25T10:32:32.689001+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:12.884809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462239+0000","last_change":"2024-04-25T10:30:33.640690+0000","last_active":"2024-04-25T10:32:32.462239+0000","last_peered":"2024-04-25T10:32:32.462239+0000","last_clean":"2024-04-25T10:32:32.462239+0000","last_became_active":"2024-04-25T10:30:33.640310+0000","last_became_peered":"2024-04-25T10:30:33.640310+0000","last_unstale":"2024-04-25T10:32:32.462239+0000","last_undegraded":"2024-04-25T10:32:32.462239+0000","last_fullsized":"2024-04-25T10:32:32.462239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:03.671495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734684+0000","last_change":"2024-04-25T10:30:32.893743+0000","last_active":"2024-04-25T10:32:32.734684+0000","last_peered":"2024-04-25T10:32:32.734684+0000","last_clean":"2024-04-25T10:32:32.734684+0000","last_became_active":"2024-04-25T10:30:32.893360+0000","last_became_peered":"2024-04-25T10:30:32.893360+0000","last_unstale":"2024-04-25T10:32:32.734684+0000","last_undegraded":"2024-04-25T10:32:32.734684+0000","last_fullsized":"2024-04-25T10:32:32.734684+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:39.830376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869026+0000","last_change":"2024-04-25T10:30:33.645127+0000","last_active":"2024-04-25T10:32:32.869026+0000","last_peered":"2024-04-25T10:32:32.869026+0000","last_clean":"2024-04-25T10:32:32.869026+0000","last_became_active":"2024-04-25T10:30:33.644259+0000","last_became_peered":"2024-04-25T10:30:33.644259+0000","last_unstale":"2024-04-25T10:32:32.869026+0000","last_undegraded":"2024-04-25T10:32:32.869026+0000","last_fullsized":"2024-04-25T10:32:32.869026+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:56.849861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462504+0000","last_change":"2024-04-25T10:30:32.896885+0000","last_active":"2024-04-25T10:32:32.462504+0000","last_peered":"2024-04-25T10:32:32.462504+0000","last_clean":"2024-04-25T10:32:32.462504+0000","last_became_active":"2024-04-25T10:30:32.896447+0000","last_became_peered":"2024-04-25T10:30:32.896447+0000","last_unstale":"2024-04-25T10:32:32.462504+0000","last_undegraded":"2024-04-25T10:32:32.462504+0000","last_fullsized":"2024-04-25T10:32:32.462504+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:13.133549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"79'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.564914+0000","last_change":"2024-04-25T10:30:33.645016+0000","last_active":"2024-04-25T10:32:32.564914+0000","last_peered":"2024-04-25T10:32:32.564914+0000","last_clean":"2024-04-25T10:32:32.564914+0000","last_became_active":"2024-04-25T10:30:33.644661+0000","last_became_peered":"2024-04-25T10:30:33.644661+0000","last_unstale":"2024-04-25T10:32:32.564914+0000","last_undegraded":"2024-04-25T10:32:32.564914+0000","last_fullsized":"2024-04-25T10:32:32.564914+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:20.935124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.461752+0000","last_change":"2024-04-25T10:30:32.907827+0000","last_active":"2024-04-25T10:32:32.461752+0000","last_peered":"2024-04-25T10:32:32.461752+0000","last_clean":"2024-04-25T10:32:32.461752+0000","last_became_active":"2024-04-25T10:30:32.907402+0000","last_became_peered":"2024-04-25T10:30:32.907402+0000","last_unstale":"2024-04-25T10:32:32.461752+0000","last_undegraded":"2024-04-25T10:32:32.461752+0000","last_fullsized":"2024-04-25T10:32:32.461752+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:38.501617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567040+0000","last_change":"2024-04-25T10:30:33.636935+0000","last_active":"2024-04-25T10:32:32.567040+0000","last_peered":"2024-04-25T10:32:32.567040+0000","last_clean":"2024-04-25T10:32:32.567040+0000","last_became_active":"2024-04-25T10:30:33.636566+0000","last_became_peered":"2024-04-25T10:30:33.636566+0000","last_unstale":"2024-04-25T10:32:32.567040+0000","last_undegraded":"2024-04-25T10:32:32.567040+0000","last_fullsized":"2024-04-25T10:32:32.567040+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:24.556275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'2","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567152+0000","last_change":"2024-04-25T10:31:33.946786+0000","last_active":"2024-04-25T10:32:32.567152+0000","last_peered":"2024-04-25T10:32:32.567152+0000","last_clean":"2024-04-25T10:32:32.567152+0000","last_became_active":"2024-04-25T10:31:33.941298+0000","last_became_peered":"2024-04-25T10:31:33.941298+0000","last_unstale":"2024-04-25T10:32:32.567152+0000","last_undegraded":"2024-04-25T10:32:32.567152+0000","last_fullsized":"2024-04-25T10:32:32.567152+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:55.009673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.461836+0000","last_change":"2024-04-25T10:30:33.642986+0000","last_active":"2024-04-25T10:32:32.461836+0000","last_peered":"2024-04-25T10:32:32.461836+0000","last_clean":"2024-04-25T10:32:32.461836+0000","last_became_active":"2024-04-25T10:30:33.642545+0000","last_became_peered":"2024-04-25T10:30:33.642545+0000","last_unstale":"2024-04-25T10:32:32.461836+0000","last_undegraded":"2024-04-25T10:32:32.461836+0000","last_fullsized":"2024-04-25T10:32:32.461836+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:49.369648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.565008+0000","last_change":"2024-04-25T10:30:32.902239+0000","last_active":"2024-04-25T10:32:32.565008+0000","last_peered":"2024-04-25T10:32:32.565008+0000","last_clean":"2024-04-25T10:32:32.565008+0000","last_became_active":"2024-04-25T10:30:32.901653+0000","last_became_peered":"2024-04-25T10:30:32.901653+0000","last_unstale":"2024-04-25T10:32:32.565008+0000","last_undegraded":"2024-04-25T10:32:32.565008+0000","last_fullsized":"2024-04-25T10:32:32.565008+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:34.732428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734130+0000","last_change":"2024-04-25T10:30:33.639763+0000","last_active":"2024-04-25T10:32:32.734130+0000","last_peered":"2024-04-25T10:32:32.734130+0000","last_clean":"2024-04-25T10:32:32.734130+0000","last_became_active":"2024-04-25T10:30:33.638004+0000","last_became_peered":"2024-04-25T10:30:33.638004+0000","last_unstale":"2024-04-25T10:32:32.734130+0000","last_undegraded":"2024-04-25T10:32:32.734130+0000","last_fullsized":"2024-04-25T10:32:32.734130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:09.722333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734268+0000","last_change":"2024-04-25T10:30:32.894823+0000","last_active":"2024-04-25T10:32:32.734268+0000","last_peered":"2024-04-25T10:32:32.734268+0000","last_clean":"2024-04-25T10:32:32.734268+0000","last_became_active":"2024-04-25T10:30:32.893819+0000","last_became_peered":"2024-04-25T10:30:32.893819+0000","last_unstale":"2024-04-25T10:32:32.734268+0000","last_undegraded":"2024-04-25T10:32:32.734268+0000","last_fullsized":"2024-04-25T10:32:32.734268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:59.809410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734513+0000","last_change":"2024-04-25T10:30:33.637046+0000","last_active":"2024-04-25T10:32:32.734513+0000","last_peered":"2024-04-25T10:32:32.734513+0000","last_clean":"2024-04-25T10:32:32.734513+0000","last_became_active":"2024-04-25T10:30:33.634903+0000","last_became_peered":"2024-04-25T10:30:33.634903+0000","last_unstale":"2024-04-25T10:32:32.734513+0000","last_undegraded":"2024-04-25T10:32:32.734513+0000","last_fullsized":"2024-04-25T10:32:32.734513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:52.252812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'39","reported_seq":68,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.683867+0000","last_change":"2024-04-25T10:30:32.898979+0000","last_active":"2024-04-25T10:32:32.683867+0000","last_peered":"2024-04-25T10:32:32.683867+0000","last_clean":"2024-04-25T10:32:32.683867+0000","last_became_active":"2024-04-25T10:30:32.898611+0000","last_became_peered":"2024-04-25T10:30:32.898611+0000","last_unstale":"2024-04-25T10:32:32.683867+0000","last_undegraded":"2024-04-25T10:32:32.683867+0000","last_fullsized":"2024-04-25T10:32:32.683867+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.549769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734130+0000","last_change":"2024-04-25T10:30:33.651596+0000","last_active":"2024-04-25T10:32:32.734130+0000","last_peered":"2024-04-25T10:32:32.734130+0000","last_clean":"2024-04-25T10:32:32.734130+0000","last_became_active":"2024-04-25T10:30:33.651140+0000","last_became_peered":"2024-04-25T10:30:33.651140+0000","last_unstale":"2024-04-25T10:32:32.734130+0000","last_undegraded":"2024-04-25T10:32:32.734130+0000","last_fullsized":"2024-04-25T10:32:32.734130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:16.762135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869451+0000","last_change":"2024-04-25T10:30:32.909982+0000","last_active":"2024-04-25T10:32:32.869451+0000","last_peered":"2024-04-25T10:32:32.869451+0000","last_clean":"2024-04-25T10:32:32.869451+0000","last_became_active":"2024-04-25T10:30:32.909506+0000","last_became_peered":"2024-04-25T10:30:32.909506+0000","last_unstale":"2024-04-25T10:32:32.869451+0000","last_undegraded":"2024-04-25T10:32:32.869451+0000","last_fullsized":"2024-04-25T10:32:32.869451+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.878852+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.878852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:47.396963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734492+0000","last_change":"2024-04-25T10:30:33.642913+0000","last_active":"2024-04-25T10:32:32.734492+0000","last_peered":"2024-04-25T10:32:32.734492+0000","last_clean":"2024-04-25T10:32:32.734492+0000","last_became_active":"2024-04-25T10:30:33.642614+0000","last_became_peered":"2024-04-25T10:30:33.642614+0000","last_unstale":"2024-04-25T10:32:32.734492+0000","last_undegraded":"2024-04-25T10:32:32.734492+0000","last_fullsized":"2024-04-25T10:32:32.734492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:56.651230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567112+0000","last_change":"2024-04-25T10:30:33.644525+0000","last_active":"2024-04-25T10:32:32.567112+0000","last_peered":"2024-04-25T10:32:32.567112+0000","last_clean":"2024-04-25T10:32:32.567112+0000","last_became_active":"2024-04-25T10:30:33.644188+0000","last_became_peered":"2024-04-25T10:30:33.644188+0000","last_unstale":"2024-04-25T10:32:32.567112+0000","last_undegraded":"2024-04-25T10:32:32.567112+0000","last_fullsized":"2024-04-25T10:32:32.567112+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:28.625695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"79'12","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734197+0000","last_change":"2024-04-25T10:31:33.946016+0000","last_active":"2024-04-25T10:32:32.734197+0000","last_peered":"2024-04-25T10:32:32.734197+0000","last_clean":"2024-04-25T10:32:32.734197+0000","last_became_active":"2024-04-25T10:31:33.945611+0000","last_became_peered":"2024-04-25T10:31:33.945611+0000","last_unstale":"2024-04-25T10:32:32.734197+0000","last_undegraded":"2024-04-25T10:32:32.734197+0000","last_fullsized":"2024-04-25T10:32:32.734197+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:02.352885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734068+0000","last_change":"2024-04-25T10:30:33.644721+0000","last_active":"2024-04-25T10:32:32.734068+0000","last_peered":"2024-04-25T10:32:32.734068+0000","last_clean":"2024-04-25T10:32:32.734068+0000","last_became_active":"2024-04-25T10:30:33.644392+0000","last_became_peered":"2024-04-25T10:30:33.644392+0000","last_unstale":"2024-04-25T10:32:32.734068+0000","last_undegraded":"2024-04-25T10:32:32.734068+0000","last_fullsized":"2024-04-25T10:32:32.734068+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:50.571318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.868724+0000","last_change":"2024-04-25T10:30:33.647970+0000","last_active":"2024-04-25T10:32:32.868724+0000","last_peered":"2024-04-25T10:32:32.868724+0000","last_clean":"2024-04-25T10:32:32.868724+0000","last_became_active":"2024-04-25T10:30:33.647547+0000","last_became_peered":"2024-04-25T10:30:33.647547+0000","last_unstale":"2024-04-25T10:32:32.868724+0000","last_undegraded":"2024-04-25T10:32:32.868724+0000","last_fullsized":"2024-04-25T10:32:32.868724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:25.609085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869336+0000","last_change":"2024-04-25T10:30:33.641618+0000","last_active":"2024-04-25T10:32:32.869336+0000","last_peered":"2024-04-25T10:32:32.869336+0000","last_clean":"2024-04-25T10:32:32.869336+0000","last_became_active":"2024-04-25T10:30:33.640830+0000","last_became_peered":"2024-04-25T10:30:33.640830+0000","last_unstale":"2024-04-25T10:32:32.869336+0000","last_undegraded":"2024-04-25T10:32:32.869336+0000","last_fullsized":"2024-04-25T10:32:32.869336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:36.870127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567255+0000","last_change":"2024-04-25T10:30:33.644010+0000","last_active":"2024-04-25T10:32:32.567255+0000","last_peered":"2024-04-25T10:32:32.567255+0000","last_clean":"2024-04-25T10:32:32.567255+0000","last_became_active":"2024-04-25T10:30:33.643639+0000","last_became_peered":"2024-04-25T10:30:33.643639+0000","last_unstale":"2024-04-25T10:32:32.567255+0000","last_undegraded":"2024-04-25T10:32:32.567255+0000","last_fullsized":"2024-04-25T10:32:32.567255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:32.820552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.734555+0000","last_change":"2024-04-25T10:30:33.639727+0000","last_active":"2024-04-25T10:32:32.734555+0000","last_peered":"2024-04-25T10:32:32.734555+0000","last_clean":"2024-04-25T10:32:32.734555+0000","last_became_active":"2024-04-25T10:30:33.639185+0000","last_became_peered":"2024-04-25T10:30:33.639185+0000","last_unstale":"2024-04-25T10:32:32.734555+0000","last_undegraded":"2024-04-25T10:32:32.734555+0000","last_fullsized":"2024-04-25T10:32:32.734555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:17.860864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"79'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462081+0000","last_change":"2024-04-25T10:30:33.644650+0000","last_active":"2024-04-25T10:32:32.462081+0000","last_peered":"2024-04-25T10:32:32.462081+0000","last_clean":"2024-04-25T10:32:32.462081+0000","last_became_active":"2024-04-25T10:30:33.644264+0000","last_became_peered":"2024-04-25T10:30:33.644264+0000","last_unstale":"2024-04-25T10:32:32.462081+0000","last_undegraded":"2024-04-25T10:32:32.462081+0000","last_fullsized":"2024-04-25T10:32:32.462081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:41.122318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.463411+0000","last_change":"2024-04-25T10:30:33.651875+0000","last_active":"2024-04-25T10:32:32.463411+0000","last_peered":"2024-04-25T10:32:32.463411+0000","last_clean":"2024-04-25T10:32:32.463411+0000","last_became_active":"2024-04-25T10:30:33.651486+0000","last_became_peered":"2024-04-25T10:30:33.651486+0000","last_unstale":"2024-04-25T10:32:32.463411+0000","last_undegraded":"2024-04-25T10:32:32.463411+0000","last_fullsized":"2024-04-25T10:32:32.463411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:32.598209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"79'8","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.562510+0000","last_change":"2024-04-25T10:31:33.942171+0000","last_active":"2024-04-25T10:32:32.562510+0000","last_peered":"2024-04-25T10:32:32.562510+0000","last_clean":"2024-04-25T10:32:32.562510+0000","last_became_active":"2024-04-25T10:31:33.941684+0000","last_became_peered":"2024-04-25T10:31:33.941684+0000","last_unstale":"2024-04-25T10:32:32.562510+0000","last_undegraded":"2024-04-25T10:32:32.562510+0000","last_fullsized":"2024-04-25T10:32:32.562510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:45.291952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.462705+0000","last_change":"2024-04-25T10:30:33.639915+0000","last_active":"2024-04-25T10:32:32.462705+0000","last_peered":"2024-04-25T10:32:32.462705+0000","last_clean":"2024-04-25T10:32:32.462705+0000","last_became_active":"2024-04-25T10:30:33.639137+0000","last_became_peered":"2024-04-25T10:30:33.639137+0000","last_unstale":"2024-04-25T10:32:32.462705+0000","last_undegraded":"2024-04-25T10:32:32.462705+0000","last_fullsized":"2024-04-25T10:32:32.462705+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:13.143405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.869541+0000","last_change":"2024-04-25T10:30:33.643547+0000","last_active":"2024-04-25T10:32:32.869541+0000","last_peered":"2024-04-25T10:32:32.869541+0000","last_clean":"2024-04-25T10:32:32.869541+0000","last_became_active":"2024-04-25T10:30:33.643158+0000","last_became_peered":"2024-04-25T10:30:33.643158+0000","last_unstale":"2024-04-25T10:32:32.869541+0000","last_undegraded":"2024-04-25T10:32:32.869541+0000","last_fullsized":"2024-04-25T10:32:32.869541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:07.517132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:32:32.567168+0000","last_change":"2024-04-25T10:30:33.645746+0000","last_active":"2024-04-25T10:32:32.567168+0000","last_peered":"2024-04-25T10:32:32.567168+0000","last_clean":"2024-04-25T10:32:32.567168+0000","last_became_active":"2024-04-25T10:30:33.644646+0000","last_became_peered":"2024-04-25T10:30:33.644646+0000","last_unstale":"2024-04-25T10:32:32.567168+0000","last_undegraded":"2024-04-25T10:32:32.567168+0000","last_fullsized":"2024-04-25T10:32:32.567168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:32.880504+0000","last_clean_scrub_stamp":"2024-04-25T10:30:32.880504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:21.591264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597107,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":809,"num_read_kb":514,"num_write":283,"num_write_kb":1433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":61068,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1791321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645350,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":856064,"data_stored":413481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841578,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37004,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708148,"statfs":{"total":95995035648,"available":95957143552,"internally_reserved":0,"allocated":1708032,"data_stored":1213757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037807,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36588,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708564,"statfs":{"total":95995035648,"available":95957569536,"internally_reserved":0,"allocated":1277952,"data_stored":815838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234036,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36584,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708568,"statfs":{"total":95995035648,"available":95957573632,"internally_reserved":0,"allocated":1273856,"data_stored":815736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430266,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712528,"statfs":{"total":95995035648,"available":95961628672,"internally_reserved":0,"allocated":1478656,"data_stored":1015004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626493,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":856064,"data_stored":413084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":942080,"data_stored":489562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40352,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704800,"statfs":{"total":95995035648,"available":95953715200,"internally_reserved":0,"allocated":876544,"data_stored":414355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":872448,"data_stored":414227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36720,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708432,"statfs":{"total":95995035648,"available":95957434368,"internally_reserved":0,"allocated":1417216,"data_stored":947138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986248,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":954368,"data_stored":490738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37104,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708048,"statfs":{"total":95995035648,"available":95957041152,"internally_reserved":0,"allocated":1626112,"data_stored":1145365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":201468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:32:37.877 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:32:38.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.024+0000 7f8181b72640 1 -- 172.21.15.43:0/2209549330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f817c108a20 msgr2=0x7f817c1025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:38.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.024+0000 7f8181b72640 1 --2- 172.21.15.43:0/2209549330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f817c108a20 0x7f817c1025b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f816c009900 tx=0x7f816c02a410 comp rx=0 tx=0).stop 2024-04-25T10:32:38.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.024+0000 7f8181b72640 1 -- 172.21.15.43:0/2209549330 shutdown_connections 2024-04-25T10:32:38.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.024+0000 7f8181b72640 1 --2- 172.21.15.43:0/2209549330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f817c10d380 0x7f817c10f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.024+0000 7f8181b72640 1 --2- 172.21.15.43:0/2209549330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f817c102af0 0x7f817c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.024+0000 7f8181b72640 1 --2- 172.21.15.43:0/2209549330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f817c108a20 0x7f817c1025b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.024+0000 7f8181b72640 1 -- 172.21.15.43:0/2209549330 >> 172.21.15.43:0/2209549330 conn(0x7f817c078110 msgr2=0x7f817c078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:38.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.024+0000 7f8181b72640 1 -- 172.21.15.43:0/2209549330 shutdown_connections 2024-04-25T10:32:38.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.024+0000 7f8181b72640 1 -- 172.21.15.43:0/2209549330 wait complete. 2024-04-25T10:32:38.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.025+0000 7f8181b72640 1 Processor -- start 2024-04-25T10:32:38.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.025+0000 7f8181b72640 1 -- start start 2024-04-25T10:32:38.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.025+0000 7f8181b72640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f817c102af0 0x7f817c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:38.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.025+0000 7f8181b72640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f817c108a20 0x7f817c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:38.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.025+0000 7f8181b72640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f817c10d380 0x7f817c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:38.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.025+0000 7f8181b72640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f817c112be0 con 0x7f817c108a20 2024-04-25T10:32:38.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.025+0000 7f8181b72640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f817c112a60 con 0x7f817c10d380 2024-04-25T10:32:38.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.025+0000 7f8181b72640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f817c112d60 con 0x7f817c102af0 2024-04-25T10:32:38.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.025+0000 7f817affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f817c108a20 0x7f817c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:38.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.025+0000 7f817b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f817c102af0 0x7f817c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:38.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.025+0000 7f817b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f817c102af0 0x7f817c19a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:58748/0 (socket says 172.21.15.43:58748) 2024-04-25T10:32:38.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.025+0000 7f817affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f817c108a20 0x7f817c19ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49154/0 (socket says 172.21.15.43:49154) 2024-04-25T10:32:38.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.025+0000 7f817b7fe640 1 -- 172.21.15.43:0/3960854440 learned_addr learned my addr 172.21.15.43:0/3960854440 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:38.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.025+0000 7f817bfff640 1 --2- 172.21.15.43:0/3960854440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f817c10d380 0x7f817c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:38.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.026+0000 7f817affd640 1 -- 172.21.15.43:0/3960854440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f817c102af0 msgr2=0x7f817c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:38.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.026+0000 7f817affd640 1 --2- 172.21.15.43:0/3960854440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f817c102af0 0x7f817c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.026+0000 7f817affd640 1 -- 172.21.15.43:0/3960854440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f817c10d380 msgr2=0x7f817c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:38.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.026+0000 7f817affd640 1 --2- 172.21.15.43:0/3960854440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f817c10d380 0x7f817c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.026+0000 7f817affd640 1 -- 172.21.15.43:0/3960854440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817c19f550 con 0x7f817c108a20 2024-04-25T10:32:38.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.026+0000 7f817affd640 1 --2- 172.21.15.43:0/3960854440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f817c108a20 0x7f817c19ac50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f817c075fd0 tx=0x7f817000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:38.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.026+0000 7f8178ff9640 1 -- 172.21.15.43:0/3960854440 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8170017070 con 0x7f817c108a20 2024-04-25T10:32:38.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.026+0000 7f8181b72640 1 -- 172.21.15.43:0/3960854440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f817c19f840 con 0x7f817c108a20 2024-04-25T10:32:38.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.026+0000 7f8178ff9640 1 -- 172.21.15.43:0/3960854440 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8170009e50 con 0x7f817c108a20 2024-04-25T10:32:38.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.026+0000 7f8178ff9640 1 -- 172.21.15.43:0/3960854440 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817000f3c0 con 0x7f817c108a20 2024-04-25T10:32:38.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.026+0000 7f8181b72640 1 -- 172.21.15.43:0/3960854440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f817c1a71c0 con 0x7f817c108a20 2024-04-25T10:32:38.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.027+0000 7f8181b72640 1 -- 172.21.15.43:0/3960854440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f817c103cd0 con 0x7f817c108a20 2024-04-25T10:32:38.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.028+0000 7f8178ff9640 1 -- 172.21.15.43:0/3960854440 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 147747+0+0 (secure 0 0 0) 0x7f817001a070 con 0x7f817c108a20 2024-04-25T10:32:38.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.029+0000 7f8178ff9640 1 --2- 172.21.15.43:0/3960854440 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f81640afd80 0x7f81640b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:38.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.029+0000 7f8178ff9640 1 -- 172.21.15.43:0/3960854440 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f81700debb0 con 0x7f817c108a20 2024-04-25T10:32:38.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.029+0000 7f817b7fe640 1 --2- 172.21.15.43:0/3960854440 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f81640afd80 0x7f81640b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:38.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.030+0000 7f817b7fe640 1 --2- 172.21.15.43:0/3960854440 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f81640afd80 0x7f81640b2290 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f816c005f70 tx=0x7f816c009560 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:38.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.034+0000 7f8178ff9640 1 -- 172.21.15.43:0/3960854440 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81700a6be0 con 0x7f817c108a20 2024-04-25T10:32:38.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: Updating smithi043:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:32:38.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: Updating smithi077:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:32:38.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: Updating smithi164:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:32:38.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:32:38.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:32:38.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:32:38.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:32:38.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1446860079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:32:38.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: Updating smithi077:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:32:38.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: Updating smithi164:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:32:38.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: Updating smithi043:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:32:38.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T10:32:38.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:38.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:38.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:38.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:38.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:38.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:38.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:38.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:32:38.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:32:38.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:32:38.043 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:37 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:38.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: Updating smithi043:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:32:38.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: Updating smithi077:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:32:38.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: Updating smithi164:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T10:32:38.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T10:32:38.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:32:38.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:32:38.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-04-25T10:32:38.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1446860079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:32:38.044 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: Updating smithi077:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:32:38.044 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: Updating smithi164:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:32:38.044 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: Updating smithi043:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T10:32:38.044 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T10:32:38.044 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:38.044 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:38.044 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:38.044 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:38.044 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:38.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:38.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:38.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:32:38.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:32:38.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:32:38.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:37 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:38.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.221+0000 7f8181b72640 1 -- 172.21.15.43:0/3960854440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f817c1a7390 con 0x7f817c108a20 2024-04-25T10:32:38.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.223+0000 7f8178ff9640 1 -- 172.21.15.43:0/3960854440 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22996 (secure 0 0 0) 0x7f81700aaee0 con 0x7f817c108a20 2024-04-25T10:32:38.225 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:32:38.225 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":80,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:32:32.456657+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:31.503414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:32.224334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1599315071":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6801/1953858940":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6800/1953858940":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3026776968":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/188939647":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/1287690800":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:32:38.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.225+0000 7f8181b72640 1 -- 172.21.15.43:0/3960854440 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f81640afd80 msgr2=0x7f81640b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:38.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.225+0000 7f8181b72640 1 --2- 172.21.15.43:0/3960854440 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f81640afd80 0x7f81640b2290 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f816c005f70 tx=0x7f816c009560 comp rx=0 tx=0).stop 2024-04-25T10:32:38.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.225+0000 7f8181b72640 1 -- 172.21.15.43:0/3960854440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f817c108a20 msgr2=0x7f817c19ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:38.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.225+0000 7f8181b72640 1 --2- 172.21.15.43:0/3960854440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f817c108a20 0x7f817c19ac50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f817c075fd0 tx=0x7f817000dbb0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.225+0000 7f8181b72640 1 -- 172.21.15.43:0/3960854440 shutdown_connections 2024-04-25T10:32:38.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.225+0000 7f8181b72640 1 --2- 172.21.15.43:0/3960854440 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f81640afd80 0x7f81640b2290 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.225+0000 7f8181b72640 1 --2- 172.21.15.43:0/3960854440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f817c10d380 0x7f817c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.225+0000 7f8181b72640 1 --2- 172.21.15.43:0/3960854440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f817c108a20 0x7f817c19ac50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.225+0000 7f8181b72640 1 --2- 172.21.15.43:0/3960854440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f817c102af0 0x7f817c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.225+0000 7f8181b72640 1 -- 172.21.15.43:0/3960854440 >> 172.21.15.43:0/3960854440 conn(0x7f817c078110 msgr2=0x7f817c069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:38.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.225+0000 7f8181b72640 1 -- 172.21.15.43:0/3960854440 shutdown_connections 2024-04-25T10:32:38.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.225+0000 7f8181b72640 1 -- 172.21.15.43:0/3960854440 wait complete. 2024-04-25T10:32:38.239 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:25:58.167476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:31.503414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:32.224334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:32:38.239 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:32:38.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.352+0000 7f23c5c30640 1 -- 172.21.15.43:0/162282272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23c0108a20 msgr2=0x7f23c01025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:38.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.352+0000 7f23c5c30640 1 --2- 172.21.15.43:0/162282272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23c0108a20 0x7f23c01025b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009900 tx=0x7f23b002a410 comp rx=0 tx=0).stop 2024-04-25T10:32:38.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.352+0000 7f23c5c30640 1 -- 172.21.15.43:0/162282272 shutdown_connections 2024-04-25T10:32:38.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.352+0000 7f23c5c30640 1 --2- 172.21.15.43:0/162282272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23c010d380 0x7f23c010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.352+0000 7f23c5c30640 1 --2- 172.21.15.43:0/162282272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23c0102af0 0x7f23c0102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.352+0000 7f23c5c30640 1 --2- 172.21.15.43:0/162282272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23c0108a20 0x7f23c01025b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.352+0000 7f23c5c30640 1 -- 172.21.15.43:0/162282272 >> 172.21.15.43:0/162282272 conn(0x7f23c0078100 msgr2=0x7f23c0078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:38.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.352+0000 7f23c5c30640 1 -- 172.21.15.43:0/162282272 shutdown_connections 2024-04-25T10:32:38.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.352+0000 7f23c5c30640 1 -- 172.21.15.43:0/162282272 wait complete. 2024-04-25T10:32:38.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.353+0000 7f23c5c30640 1 Processor -- start 2024-04-25T10:32:38.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.353+0000 7f23c5c30640 1 -- start start 2024-04-25T10:32:38.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.353+0000 7f23c5c30640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23c0102af0 0x7f23c00ff250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:38.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.353+0000 7f23c5c30640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23c0108a20 0x7f23c00ff790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:38.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.353+0000 7f23c5c30640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23c010d380 0x7f23c00ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:38.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.353+0000 7f23c5c30640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f23c0112be0 con 0x7f23c010d380 2024-04-25T10:32:38.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.353+0000 7f23c5c30640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f23c0112a60 con 0x7f23c0102af0 2024-04-25T10:32:38.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.353+0000 7f23c5c30640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f23c0112d60 con 0x7f23c0108a20 2024-04-25T10:32:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.353+0000 7f23bffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23c010d380 0x7f23c00ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.353+0000 7f23bffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23c010d380 0x7f23c00ffcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49166/0 (socket says 172.21.15.43:49166) 2024-04-25T10:32:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.353+0000 7f23bffff640 1 -- 172.21.15.43:0/4005161808 learned_addr learned my addr 172.21.15.43:0/4005161808 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.353+0000 7f23beffd640 1 --2- 172.21.15.43:0/4005161808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23c0108a20 0x7f23c00ff790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.353+0000 7f23bf7fe640 1 --2- 172.21.15.43:0/4005161808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23c0102af0 0x7f23c00ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.354+0000 7f23bffff640 1 -- 172.21.15.43:0/4005161808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23c0108a20 msgr2=0x7f23c00ff790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.354+0000 7f23bffff640 1 --2- 172.21.15.43:0/4005161808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23c0108a20 0x7f23c00ff790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.354+0000 7f23bffff640 1 -- 172.21.15.43:0/4005161808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23c0102af0 msgr2=0x7f23c00ff250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.354+0000 7f23bffff640 1 --2- 172.21.15.43:0/4005161808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23c0102af0 0x7f23c00ff250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.354+0000 7f23bffff640 1 -- 172.21.15.43:0/4005161808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c01a8a30 con 0x7f23c010d380 2024-04-25T10:32:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.354+0000 7f23bf7fe640 1 --2- 172.21.15.43:0/4005161808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23c0102af0 0x7f23c00ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:32:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.354+0000 7f23bffff640 1 --2- 172.21.15.43:0/4005161808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23c010d380 0x7f23c00ffcd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f23b800eaa0 tx=0x7f23b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.354+0000 7f23bcff9640 1 -- 172.21.15.43:0/4005161808 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b800cdb0 con 0x7f23c010d380 2024-04-25T10:32:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.354+0000 7f23bcff9640 1 -- 172.21.15.43:0/4005161808 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23b8005020 con 0x7f23c010d380 2024-04-25T10:32:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.354+0000 7f23c5c30640 1 -- 172.21.15.43:0/4005161808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c01a8d20 con 0x7f23c010d380 2024-04-25T10:32:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.354+0000 7f23bcff9640 1 -- 172.21.15.43:0/4005161808 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b80116d0 con 0x7f23c010d380 2024-04-25T10:32:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.354+0000 7f23c5c30640 1 -- 172.21.15.43:0/4005161808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c01a91c0 con 0x7f23c010d380 2024-04-25T10:32:38.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.355+0000 7f23c5c30640 1 -- 172.21.15.43:0/4005161808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c006afa0 con 0x7f23c010d380 2024-04-25T10:32:38.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.357+0000 7f23bcff9640 1 -- 172.21.15.43:0/4005161808 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 147747+0+0 (secure 0 0 0) 0x7f23b80051d0 con 0x7f23c010d380 2024-04-25T10:32:38.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.357+0000 7f23bcff9640 1 --2- 172.21.15.43:0/4005161808 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f23a80afd80 0x7f23a80b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:38.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.357+0000 7f23bf7fe640 1 --2- 172.21.15.43:0/4005161808 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f23a80afd80 0x7f23a80b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:38.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.357+0000 7f23bcff9640 1 -- 172.21.15.43:0/4005161808 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f23b80df8b0 con 0x7f23c010d380 2024-04-25T10:32:38.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.358+0000 7f23bf7fe640 1 --2- 172.21.15.43:0/4005161808 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f23a80afd80 0x7f23a80b2290 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f23b00096c0 tx=0x7f23b0031400 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:38.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.361+0000 7f23bcff9640 1 -- 172.21.15.43:0/4005161808 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23b8016030 con 0x7f23c010d380 2024-04-25T10:32:38.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.550+0000 7f23c5c30640 1 -- 172.21.15.43:0/4005161808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f23c01a95b0 con 0x7f23c010d380 2024-04-25T10:32:38.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.550+0000 7f23bcff9640 1 -- 172.21.15.43:0/4005161808 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7f23b80a7f30 con 0x7f23c010d380 2024-04-25T10:32:38.552 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T10:32:38.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.552+0000 7f23c5c30640 1 -- 172.21.15.43:0/4005161808 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f23a80afd80 msgr2=0x7f23a80b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:38.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.552+0000 7f23c5c30640 1 --2- 172.21.15.43:0/4005161808 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f23a80afd80 0x7f23a80b2290 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f23b00096c0 tx=0x7f23b0031400 comp rx=0 tx=0).stop 2024-04-25T10:32:38.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.552+0000 7f23c5c30640 1 -- 172.21.15.43:0/4005161808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23c010d380 msgr2=0x7f23c00ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:38.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.552+0000 7f23c5c30640 1 --2- 172.21.15.43:0/4005161808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23c010d380 0x7f23c00ffcd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f23b800eaa0 tx=0x7f23b800ef60 comp rx=0 tx=0).stop 2024-04-25T10:32:38.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.552+0000 7f23c5c30640 1 -- 172.21.15.43:0/4005161808 shutdown_connections 2024-04-25T10:32:38.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.552+0000 7f23c5c30640 1 --2- 172.21.15.43:0/4005161808 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f23a80afd80 0x7f23a80b2290 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.552+0000 7f23c5c30640 1 --2- 172.21.15.43:0/4005161808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23c010d380 0x7f23c00ffcd0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.552+0000 7f23c5c30640 1 --2- 172.21.15.43:0/4005161808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23c0108a20 0x7f23c00ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.552+0000 7f23c5c30640 1 --2- 172.21.15.43:0/4005161808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23c0102af0 0x7f23c00ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.552+0000 7f23c5c30640 1 -- 172.21.15.43:0/4005161808 >> 172.21.15.43:0/4005161808 conn(0x7f23c0078100 msgr2=0x7f23c0069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:38.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.552+0000 7f23c5c30640 1 -- 172.21.15.43:0/4005161808 shutdown_connections 2024-04-25T10:32:38.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.552+0000 7f23c5c30640 1 -- 172.21.15.43:0/4005161808 wait complete. 2024-04-25T10:32:38.566 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:32:38.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.679+0000 7f8801d5e640 1 -- 172.21.15.43:0/2532456401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87fc108a20 msgr2=0x7f87fc1025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:38.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.679+0000 7f8801d5e640 1 --2- 172.21.15.43:0/2532456401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87fc108a20 0x7f87fc1025b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009900 tx=0x7f87ec02a410 comp rx=0 tx=0).stop 2024-04-25T10:32:38.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.679+0000 7f8801d5e640 1 -- 172.21.15.43:0/2532456401 shutdown_connections 2024-04-25T10:32:38.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.679+0000 7f8801d5e640 1 --2- 172.21.15.43:0/2532456401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc10d380 0x7f87fc10f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.679+0000 7f8801d5e640 1 --2- 172.21.15.43:0/2532456401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87fc102af0 0x7f87fc102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.679+0000 7f8801d5e640 1 --2- 172.21.15.43:0/2532456401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87fc108a20 0x7f87fc1025b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.679+0000 7f8801d5e640 1 -- 172.21.15.43:0/2532456401 >> 172.21.15.43:0/2532456401 conn(0x7f87fc078100 msgr2=0x7f87fc078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:38.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.679+0000 7f8801d5e640 1 -- 172.21.15.43:0/2532456401 shutdown_connections 2024-04-25T10:32:38.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.679+0000 7f8801d5e640 1 -- 172.21.15.43:0/2532456401 wait complete. 2024-04-25T10:32:38.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.680+0000 7f8801d5e640 1 Processor -- start 2024-04-25T10:32:38.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.680+0000 7f8801d5e640 1 -- start start 2024-04-25T10:32:38.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.680+0000 7f8801d5e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc102af0 0x7f87fc19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:38.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.680+0000 7f8801d5e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87fc108a20 0x7f87fc19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:38.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.680+0000 7f8801d5e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87fc10d380 0x7f87fc19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:38.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.680+0000 7f8801d5e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc113d00 con 0x7f87fc108a20 2024-04-25T10:32:38.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.680+0000 7f8801d5e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc113b80 con 0x7f87fc102af0 2024-04-25T10:32:38.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.680+0000 7f8801d5e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc113e80 con 0x7f87fc10d380 2024-04-25T10:32:38.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.680+0000 7f87faffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87fc108a20 0x7f87fc19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:38.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.680+0000 7f87faffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87fc108a20 0x7f87fc19ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50490/0 (socket says 172.21.15.43:50490) 2024-04-25T10:32:38.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.680+0000 7f87faffd640 1 -- 172.21.15.43:0/1206886127 learned_addr learned my addr 172.21.15.43:0/1206886127 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:38.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.681+0000 7f87fb7fe640 1 --2- 172.21.15.43:0/1206886127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc102af0 0x7f87fc19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:38.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.681+0000 7f87fbfff640 1 --2- 172.21.15.43:0/1206886127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87fc10d380 0x7f87fc19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:38.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.681+0000 7f87faffd640 1 -- 172.21.15.43:0/1206886127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87fc10d380 msgr2=0x7f87fc19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:38.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.681+0000 7f87faffd640 1 --2- 172.21.15.43:0/1206886127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87fc10d380 0x7f87fc19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.681+0000 7f87faffd640 1 -- 172.21.15.43:0/1206886127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc102af0 msgr2=0x7f87fc19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:38.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.681+0000 7f87faffd640 1 --2- 172.21.15.43:0/1206886127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc102af0 0x7f87fc19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.681+0000 7f87faffd640 1 -- 172.21.15.43:0/1206886127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87fc19f560 con 0x7f87fc108a20 2024-04-25T10:32:38.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.681+0000 7f87fbfff640 1 --2- 172.21.15.43:0/1206886127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87fc10d380 0x7f87fc19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:32:38.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.681+0000 7f87faffd640 1 --2- 172.21.15.43:0/1206886127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87fc108a20 0x7f87fc19ac60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f87f0005d00 tx=0x7f87f000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:38.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.681+0000 7f87f8ff9640 1 -- 172.21.15.43:0/1206886127 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0017070 con 0x7f87fc108a20 2024-04-25T10:32:38.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.681+0000 7f8801d5e640 1 -- 172.21.15.43:0/1206886127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87fc19f850 con 0x7f87fc108a20 2024-04-25T10:32:38.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.681+0000 7f87f8ff9640 1 -- 172.21.15.43:0/1206886127 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87f0009c30 con 0x7f87fc108a20 2024-04-25T10:32:38.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.681+0000 7f87f8ff9640 1 -- 172.21.15.43:0/1206886127 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f000f620 con 0x7f87fc108a20 2024-04-25T10:32:38.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.681+0000 7f8801d5e640 1 -- 172.21.15.43:0/1206886127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87fc19fd50 con 0x7f87fc108a20 2024-04-25T10:32:38.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.682+0000 7f8801d5e640 1 -- 172.21.15.43:0/1206886127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87fc06afa0 con 0x7f87fc108a20 2024-04-25T10:32:38.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.683+0000 7f87f8ff9640 1 -- 172.21.15.43:0/1206886127 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 147747+0+0 (secure 0 0 0) 0x7f87f001a070 con 0x7f87fc108a20 2024-04-25T10:32:38.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.684+0000 7f87f8ff9640 1 --2- 172.21.15.43:0/1206886127 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f87e40afd80 0x7f87e40b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:38.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.684+0000 7f87f8ff9640 1 -- 172.21.15.43:0/1206886127 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f87f00deda0 con 0x7f87fc108a20 2024-04-25T10:32:38.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.684+0000 7f87fb7fe640 1 --2- 172.21.15.43:0/1206886127 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f87e40afd80 0x7f87e40b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:38.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.685+0000 7f87fb7fe640 1 --2- 172.21.15.43:0/1206886127 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f87e40afd80 0x7f87e40b2290 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f87ec000c00 tx=0x7f87ec031400 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:38.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.688+0000 7f87f8ff9640 1 -- 172.21.15.43:0/1206886127 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f87fc19fd50 con 0x7f87fc108a20 2024-04-25T10:32:38.834 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:38 smithi043 ceph-mon[29478]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:32:38.834 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:38 smithi043 ceph-mon[29478]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:32:38.834 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:38 smithi043 ceph-mon[29478]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:32:38.834 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:38 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3960854440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:38.834 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:38 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/4005161808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:32:38.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.877+0000 7f8801d5e640 1 -- 172.21.15.43:0/1206886127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f87fc106f40 con 0x7f87fc108a20 2024-04-25T10:32:38.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.878+0000 7f87f8ff9640 1 -- 172.21.15.43:0/1206886127 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7f87f0009de0 con 0x7f87fc108a20 2024-04-25T10:32:38.880 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T10:32:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.880+0000 7f8801d5e640 1 -- 172.21.15.43:0/1206886127 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f87e40afd80 msgr2=0x7f87e40b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.880+0000 7f8801d5e640 1 --2- 172.21.15.43:0/1206886127 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f87e40afd80 0x7f87e40b2290 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f87ec000c00 tx=0x7f87ec031400 comp rx=0 tx=0).stop 2024-04-25T10:32:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.880+0000 7f8801d5e640 1 -- 172.21.15.43:0/1206886127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87fc108a20 msgr2=0x7f87fc19ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.880+0000 7f8801d5e640 1 --2- 172.21.15.43:0/1206886127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87fc108a20 0x7f87fc19ac60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f87f0005d00 tx=0x7f87f000dde0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.880+0000 7f8801d5e640 1 -- 172.21.15.43:0/1206886127 shutdown_connections 2024-04-25T10:32:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.880+0000 7f8801d5e640 1 --2- 172.21.15.43:0/1206886127 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f87e40afd80 0x7f87e40b2290 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.880+0000 7f8801d5e640 1 --2- 172.21.15.43:0/1206886127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87fc10d380 0x7f87fc19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.880+0000 7f8801d5e640 1 --2- 172.21.15.43:0/1206886127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87fc108a20 0x7f87fc19ac60 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.880+0000 7f8801d5e640 1 --2- 172.21.15.43:0/1206886127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc102af0 0x7f87fc19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:38.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.880+0000 7f8801d5e640 1 -- 172.21.15.43:0/1206886127 >> 172.21.15.43:0/1206886127 conn(0x7f87fc078100 msgr2=0x7f87fc1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:38.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.880+0000 7f8801d5e640 1 -- 172.21.15.43:0/1206886127 shutdown_connections 2024-04-25T10:32:38.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:38.880+0000 7f8801d5e640 1 -- 172.21.15.43:0/1206886127 wait complete. 2024-04-25T10:32:38.893 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:32:39.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.006+0000 7f2d30f6c640 1 -- 172.21.15.43:0/3916472516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d2c104d10 msgr2=0x7f2d2c105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:39.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.006+0000 7f2d30f6c640 1 --2- 172.21.15.43:0/3916472516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d2c104d10 0x7f2d2c105120 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009900 tx=0x7f2d1c02a410 comp rx=0 tx=0).stop 2024-04-25T10:32:39.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.006+0000 7f2d30f6c640 1 -- 172.21.15.43:0/3916472516 shutdown_connections 2024-04-25T10:32:39.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.006+0000 7f2d30f6c640 1 --2- 172.21.15.43:0/3916472516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d2c10e0d0 0x7f2d2c110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.006+0000 7f2d30f6c640 1 --2- 172.21.15.43:0/3916472516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d2c10b770 0x7f2d2c10db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.006+0000 7f2d30f6c640 1 --2- 172.21.15.43:0/3916472516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d2c104d10 0x7f2d2c105120 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.006+0000 7f2d30f6c640 1 -- 172.21.15.43:0/3916472516 >> 172.21.15.43:0/3916472516 conn(0x7f2d2c078110 msgr2=0x7f2d2c078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:39.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.006+0000 7f2d30f6c640 1 -- 172.21.15.43:0/3916472516 shutdown_connections 2024-04-25T10:32:39.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.006+0000 7f2d30f6c640 1 -- 172.21.15.43:0/3916472516 wait complete. 2024-04-25T10:32:39.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.007+0000 7f2d30f6c640 1 Processor -- start 2024-04-25T10:32:39.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.007+0000 7f2d30f6c640 1 -- start start 2024-04-25T10:32:39.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.008+0000 7f2d30f6c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d2c104d10 0x7f2d2c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:39.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.008+0000 7f2d30f6c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d2c10b770 0x7f2d2c19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:39.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.008+0000 7f2d30f6c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d2c10e0d0 0x7f2d2c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:39.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.008+0000 7f2d30f6c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c113ce0 con 0x7f2d2c10e0d0 2024-04-25T10:32:39.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.008+0000 7f2d30f6c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c113b60 con 0x7f2d2c10b770 2024-04-25T10:32:39.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.008+0000 7f2d30f6c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c113e60 con 0x7f2d2c104d10 2024-04-25T10:32:39.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.008+0000 7f2d2ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d2c10e0d0 0x7f2d2c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:39.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.008+0000 7f2d2ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d2c10e0d0 0x7f2d2c19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50502/0 (socket says 172.21.15.43:50502) 2024-04-25T10:32:39.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.008+0000 7f2d2ad9d640 1 -- 172.21.15.43:0/2642451307 learned_addr learned my addr 172.21.15.43:0/2642451307 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:39.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.008+0000 7f2d21d9b640 1 --2- 172.21.15.43:0/2642451307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d2c10b770 0x7f2d2c19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:39.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.008+0000 7f2d2a59c640 1 --2- 172.21.15.43:0/2642451307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d2c104d10 0x7f2d2c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:39.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.008+0000 7f2d2ad9d640 1 -- 172.21.15.43:0/2642451307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d2c104d10 msgr2=0x7f2d2c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:39.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.008+0000 7f2d2ad9d640 1 --2- 172.21.15.43:0/2642451307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d2c104d10 0x7f2d2c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.008+0000 7f2d2ad9d640 1 -- 172.21.15.43:0/2642451307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d2c10b770 msgr2=0x7f2d2c19aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:39.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.008+0000 7f2d2ad9d640 1 --2- 172.21.15.43:0/2642451307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d2c10b770 0x7f2d2c19aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.008+0000 7f2d2ad9d640 1 -- 172.21.15.43:0/2642451307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2c19f5a0 con 0x7f2d2c10e0d0 2024-04-25T10:32:39.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.008+0000 7f2d21d9b640 1 --2- 172.21.15.43:0/2642451307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d2c10b770 0x7f2d2c19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:39.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.009+0000 7f2d2ad9d640 1 --2- 172.21.15.43:0/2642451307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d2c10e0d0 0x7f2d2c19ef60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400eaa0 tx=0x7f2d2400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:39.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.009+0000 7f2d23fff640 1 -- 172.21.15.43:0/2642451307 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2400cdb0 con 0x7f2d2c10e0d0 2024-04-25T10:32:39.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.009+0000 7f2d23fff640 1 -- 172.21.15.43:0/2642451307 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d24005020 con 0x7f2d2c10e0d0 2024-04-25T10:32:39.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.009+0000 7f2d30f6c640 1 -- 172.21.15.43:0/2642451307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2c19f890 con 0x7f2d2c10e0d0 2024-04-25T10:32:39.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.009+0000 7f2d23fff640 1 -- 172.21.15.43:0/2642451307 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d240116d0 con 0x7f2d2c10e0d0 2024-04-25T10:32:39.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.009+0000 7f2d30f6c640 1 -- 172.21.15.43:0/2642451307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d2c1a7210 con 0x7f2d2c10e0d0 2024-04-25T10:32:39.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.010+0000 7f2d30f6c640 1 -- 172.21.15.43:0/2642451307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d2c06a160 con 0x7f2d2c10e0d0 2024-04-25T10:32:39.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.011+0000 7f2d23fff640 1 -- 172.21.15.43:0/2642451307 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 147747+0+0 (secure 0 0 0) 0x7f2d240051d0 con 0x7f2d2c10e0d0 2024-04-25T10:32:39.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.012+0000 7f2d23fff640 1 --2- 172.21.15.43:0/2642451307 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2d180afe00 0x7f2d180b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:39.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.013+0000 7f2d2a59c640 1 --2- 172.21.15.43:0/2642451307 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2d180afe00 0x7f2d180b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:39.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.013+0000 7f2d23fff640 1 -- 172.21.15.43:0/2642451307 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f2d240df830 con 0x7f2d2c10e0d0 2024-04-25T10:32:39.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.013+0000 7f2d2a59c640 1 --2- 172.21.15.43:0/2642451307 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2d180afe00 0x7f2d180b2310 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c031440 tx=0x7f2d1c005d50 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:39.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.016+0000 7f2d23fff640 1 -- 172.21.15.43:0/2642451307 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d24016030 con 0x7f2d2c10e0d0 2024-04-25T10:32:39.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.203+0000 7f2d30f6c640 1 -- 172.21.15.43:0/2642451307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2d2c1a7410 con 0x7f2d2c10e0d0 2024-04-25T10:32:39.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.203+0000 7f2d23fff640 1 -- 172.21.15.43:0/2642451307 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7f2d240a7f30 con 0x7f2d2c10e0d0 2024-04-25T10:32:39.205 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T10:32:39.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.205+0000 7f2d30f6c640 1 -- 172.21.15.43:0/2642451307 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2d180afe00 msgr2=0x7f2d180b2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:39.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.205+0000 7f2d30f6c640 1 --2- 172.21.15.43:0/2642451307 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2d180afe00 0x7f2d180b2310 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c031440 tx=0x7f2d1c005d50 comp rx=0 tx=0).stop 2024-04-25T10:32:39.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.205+0000 7f2d30f6c640 1 -- 172.21.15.43:0/2642451307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d2c10e0d0 msgr2=0x7f2d2c19ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:39.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.205+0000 7f2d30f6c640 1 --2- 172.21.15.43:0/2642451307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d2c10e0d0 0x7f2d2c19ef60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400eaa0 tx=0x7f2d2400ef60 comp rx=0 tx=0).stop 2024-04-25T10:32:39.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.205+0000 7f2d30f6c640 1 -- 172.21.15.43:0/2642451307 shutdown_connections 2024-04-25T10:32:39.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.205+0000 7f2d30f6c640 1 --2- 172.21.15.43:0/2642451307 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2d180afe00 0x7f2d180b2310 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.205+0000 7f2d30f6c640 1 --2- 172.21.15.43:0/2642451307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d2c10b770 0x7f2d2c19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.205+0000 7f2d30f6c640 1 --2- 172.21.15.43:0/2642451307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d2c10e0d0 0x7f2d2c19ef60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.205+0000 7f2d30f6c640 1 --2- 172.21.15.43:0/2642451307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d2c104d10 0x7f2d2c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.205+0000 7f2d30f6c640 1 -- 172.21.15.43:0/2642451307 >> 172.21.15.43:0/2642451307 conn(0x7f2d2c078110 msgr2=0x7f2d2c101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:39.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.205+0000 7f2d30f6c640 1 -- 172.21.15.43:0/2642451307 shutdown_connections 2024-04-25T10:32:39.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.205+0000 7f2d30f6c640 1 -- 172.21.15.43:0/2642451307 wait complete. 2024-04-25T10:32:39.219 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:32:39.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:38 smithi077 ceph-mon[32772]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:32:39.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:38 smithi077 ceph-mon[32772]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:32:39.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:38 smithi077 ceph-mon[32772]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:32:39.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:38 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3960854440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:39.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:38 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/4005161808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:32:39.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:38 smithi164 ceph-mon[32713]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:32:39.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:38 smithi164 ceph-mon[32713]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:32:39.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:38 smithi164 ceph-mon[32713]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T10:32:39.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:38 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3960854440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:39.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:38 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/4005161808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:32:39.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.328+0000 7fc57f2a8640 1 -- 172.21.15.43:0/3280703384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc57810b680 msgr2=0x7fc57810daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:39.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.328+0000 7fc57f2a8640 1 --2- 172.21.15.43:0/3280703384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc57810b680 0x7fc57810daa0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc568009900 tx=0x7fc56802a480 comp rx=0 tx=0).stop 2024-04-25T10:32:39.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.328+0000 7fc57f2a8640 1 -- 172.21.15.43:0/3280703384 shutdown_connections 2024-04-25T10:32:39.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.328+0000 7fc57f2a8640 1 --2- 172.21.15.43:0/3280703384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc57810e070 0x7fc578110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.328+0000 7fc57f2a8640 1 --2- 172.21.15.43:0/3280703384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc57810b680 0x7fc57810daa0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.328+0000 7fc57f2a8640 1 --2- 172.21.15.43:0/3280703384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc578104c20 0x7fc578105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.328+0000 7fc57f2a8640 1 -- 172.21.15.43:0/3280703384 >> 172.21.15.43:0/3280703384 conn(0x7fc578078040 msgr2=0x7fc578078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:39.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.328+0000 7fc57f2a8640 1 -- 172.21.15.43:0/3280703384 shutdown_connections 2024-04-25T10:32:39.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.328+0000 7fc57f2a8640 1 -- 172.21.15.43:0/3280703384 wait complete. 2024-04-25T10:32:39.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.329+0000 7fc57f2a8640 1 Processor -- start 2024-04-25T10:32:39.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.329+0000 7fc57f2a8640 1 -- start start 2024-04-25T10:32:39.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.329+0000 7fc57f2a8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc578104c20 0x7fc57819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:39.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.329+0000 7fc57f2a8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc57810b680 0x7fc57819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:39.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.329+0000 7fc57f2a8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc57810e070 0x7fc57819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:39.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.329+0000 7fc57f2a8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc578113cd0 con 0x7fc578104c20 2024-04-25T10:32:39.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.329+0000 7fc57f2a8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc578113b50 con 0x7fc57810b680 2024-04-25T10:32:39.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.329+0000 7fc57f2a8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc578113e50 con 0x7fc57810e070 2024-04-25T10:32:39.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.329+0000 7fc57d044640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc578104c20 0x7fc57819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:39.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.329+0000 7fc57d044640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc578104c20 0x7fc57819a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50524/0 (socket says 172.21.15.43:50524) 2024-04-25T10:32:39.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.329+0000 7fc57d044640 1 -- 172.21.15.43:0/4257268595 learned_addr learned my addr 172.21.15.43:0/4257268595 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:39.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.330+0000 7fc57d845640 1 --2- 172.21.15.43:0/4257268595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc57810e070 0x7fc57819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:39.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.330+0000 7fc57c843640 1 --2- 172.21.15.43:0/4257268595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc57810b680 0x7fc57819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:39.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.330+0000 7fc57d044640 1 -- 172.21.15.43:0/4257268595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc57810e070 msgr2=0x7fc57819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:39.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.330+0000 7fc57d044640 1 --2- 172.21.15.43:0/4257268595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc57810e070 0x7fc57819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.330+0000 7fc57d044640 1 -- 172.21.15.43:0/4257268595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc57810b680 msgr2=0x7fc57819ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:39.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.330+0000 7fc57d044640 1 --2- 172.21.15.43:0/4257268595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc57810b680 0x7fc57819ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.330+0000 7fc57d044640 1 -- 172.21.15.43:0/4257268595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57819f610 con 0x7fc578104c20 2024-04-25T10:32:39.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.330+0000 7fc57c843640 1 --2- 172.21.15.43:0/4257268595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc57810b680 0x7fc57819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:32:39.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.330+0000 7fc57d845640 1 --2- 172.21.15.43:0/4257268595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc57810e070 0x7fc57819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:39.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.330+0000 7fc57d044640 1 --2- 172.21.15.43:0/4257268595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc578104c20 0x7fc57819a7b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc57400c970 tx=0x7fc57400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:39.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.330+0000 7fc56e7fc640 1 -- 172.21.15.43:0/4257268595 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574007bb0 con 0x7fc578104c20 2024-04-25T10:32:39.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.330+0000 7fc56e7fc640 1 -- 172.21.15.43:0/4257268595 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc574007d50 con 0x7fc578104c20 2024-04-25T10:32:39.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.330+0000 7fc57f2a8640 1 -- 172.21.15.43:0/4257268595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc57819f900 con 0x7fc578104c20 2024-04-25T10:32:39.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.330+0000 7fc56e7fc640 1 -- 172.21.15.43:0/4257268595 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57400f450 con 0x7fc578104c20 2024-04-25T10:32:39.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.330+0000 7fc57f2a8640 1 -- 172.21.15.43:0/4257268595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc57819fe00 con 0x7fc578104c20 2024-04-25T10:32:39.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.331+0000 7fc57f2a8640 1 -- 172.21.15.43:0/4257268595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc57806a190 con 0x7fc578104c20 2024-04-25T10:32:39.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.333+0000 7fc56e7fc640 1 -- 172.21.15.43:0/4257268595 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 147747+0+0 (secure 0 0 0) 0x7fc574004f00 con 0x7fc578104c20 2024-04-25T10:32:39.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.334+0000 7fc56e7fc640 1 --2- 172.21.15.43:0/4257268595 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc5640afd80 0x7fc5640b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:39.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.334+0000 7fc57c843640 1 --2- 172.21.15.43:0/4257268595 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc5640afd80 0x7fc5640b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:39.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.334+0000 7fc56e7fc640 1 -- 172.21.15.43:0/4257268595 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fc5740df4d0 con 0x7fc578104c20 2024-04-25T10:32:39.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.334+0000 7fc57c843640 1 --2- 172.21.15.43:0/4257268595 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc5640afd80 0x7fc5640b2290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc5680096c0 tx=0x7fc568031400 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:39.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.337+0000 7fc56e7fc640 1 -- 172.21.15.43:0/4257268595 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc57819fe00 con 0x7fc578104c20 2024-04-25T10:32:39.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.561+0000 7fc57f2a8640 1 -- 172.21.15.43:0/4257268595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc57819bb00 con 0x7fc578104c20 2024-04-25T10:32:39.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.562+0000 7fc56e7fc640 1 -- 172.21.15.43:0/4257268595 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11869 (secure 0 0 0) 0x7fc574014030 con 0x7fc578104c20 2024-04-25T10:32:39.564 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:32:39.564 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:32:30.234841+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2595248466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2595248466},{"type":"v1","addr":"172.21.15.77:6837","nonce":2595248466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24472,24508]},"id":1}]} 2024-04-25T10:32:39.565 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 24 2024-04-25T10:32:39.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.564+0000 7fc57f2a8640 1 -- 172.21.15.43:0/4257268595 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc5640afd80 msgr2=0x7fc5640b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:39.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.564+0000 7fc57f2a8640 1 --2- 172.21.15.43:0/4257268595 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc5640afd80 0x7fc5640b2290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc5680096c0 tx=0x7fc568031400 comp rx=0 tx=0).stop 2024-04-25T10:32:39.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.564+0000 7fc57f2a8640 1 -- 172.21.15.43:0/4257268595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc578104c20 msgr2=0x7fc57819a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:39.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.564+0000 7fc57f2a8640 1 --2- 172.21.15.43:0/4257268595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc578104c20 0x7fc57819a7b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc57400c970 tx=0x7fc57400ce30 comp rx=0 tx=0).stop 2024-04-25T10:32:39.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.564+0000 7fc57f2a8640 1 -- 172.21.15.43:0/4257268595 shutdown_connections 2024-04-25T10:32:39.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.564+0000 7fc57f2a8640 1 --2- 172.21.15.43:0/4257268595 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc5640afd80 0x7fc5640b2290 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.564+0000 7fc57f2a8640 1 --2- 172.21.15.43:0/4257268595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc57810b680 0x7fc57819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.564+0000 7fc57f2a8640 1 --2- 172.21.15.43:0/4257268595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc57810e070 0x7fc57819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.564+0000 7fc57f2a8640 1 --2- 172.21.15.43:0/4257268595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc578104c20 0x7fc57819a7b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.564+0000 7fc57f2a8640 1 -- 172.21.15.43:0/4257268595 >> 172.21.15.43:0/4257268595 conn(0x7fc578078040 msgr2=0x7fc578101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:39.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.564+0000 7fc57f2a8640 1 -- 172.21.15.43:0/4257268595 shutdown_connections 2024-04-25T10:32:39.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.564+0000 7fc57f2a8640 1 -- 172.21.15.43:0/4257268595 wait complete. 2024-04-25T10:32:39.578 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:32:39.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.688+0000 7f9df92e0640 1 -- 172.21.15.43:0/4010925454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df4102210 msgr2=0x7f9df4102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:39.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.688+0000 7f9df92e0640 1 --2- 172.21.15.43:0/4010925454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df4102210 0x7f9df4102620 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9de4009900 tx=0x7f9de402a3f0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.688+0000 7f9df92e0640 1 -- 172.21.15.43:0/4010925454 shutdown_connections 2024-04-25T10:32:39.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.689+0000 7f9df92e0640 1 --2- 172.21.15.43:0/4010925454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9df4108380 0x7f9df4108790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.689+0000 7f9df92e0640 1 --2- 172.21.15.43:0/4010925454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9df4108e00 0x7f9df4110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.689+0000 7f9df92e0640 1 --2- 172.21.15.43:0/4010925454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df4102210 0x7f9df4102620 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.689+0000 7f9df92e0640 1 -- 172.21.15.43:0/4010925454 >> 172.21.15.43:0/4010925454 conn(0x7f9df4077880 msgr2=0x7f9df4077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:39.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.689+0000 7f9df92e0640 1 -- 172.21.15.43:0/4010925454 shutdown_connections 2024-04-25T10:32:39.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.689+0000 7f9df92e0640 1 -- 172.21.15.43:0/4010925454 wait complete. 2024-04-25T10:32:39.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.689+0000 7f9df92e0640 1 Processor -- start 2024-04-25T10:32:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.689+0000 7f9df92e0640 1 -- start start 2024-04-25T10:32:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9df92e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df4102210 0x7f9df419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9df92e0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9df4108380 0x7f9df419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9df92e0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9df4108e00 0x7f9df419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9df92e0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9df4113d50 con 0x7f9df4102210 2024-04-25T10:32:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9df92e0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9df4113bd0 con 0x7f9df4108e00 2024-04-25T10:32:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9df92e0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9df4113ed0 con 0x7f9df4108380 2024-04-25T10:32:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9df2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df4102210 0x7f9df419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9df2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df4102210 0x7f9df419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50544/0 (socket says 172.21.15.43:50544) 2024-04-25T10:32:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9df2ffd640 1 -- 172.21.15.43:0/1413627031 learned_addr learned my addr 172.21.15.43:0/1413627031 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9df27fc640 1 --2- 172.21.15.43:0/1413627031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9df4108380 0x7f9df419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9df2ffd640 1 -- 172.21.15.43:0/1413627031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9df4108380 msgr2=0x7f9df419ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9df37fe640 1 --2- 172.21.15.43:0/1413627031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9df4108e00 0x7f9df419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9df2ffd640 1 --2- 172.21.15.43:0/1413627031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9df4108380 0x7f9df419ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9df2ffd640 1 -- 172.21.15.43:0/1413627031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9df4108e00 msgr2=0x7f9df419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9df2ffd640 1 --2- 172.21.15.43:0/1413627031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9df4108e00 0x7f9df419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9df2ffd640 1 -- 172.21.15.43:0/1413627031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df419f490 con 0x7f9df4102210 2024-04-25T10:32:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9df37fe640 1 --2- 172.21.15.43:0/1413627031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9df4108e00 0x7f9df419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:32:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9df27fc640 1 --2- 172.21.15.43:0/1413627031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9df4108380 0x7f9df419ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:32:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9df2ffd640 1 --2- 172.21.15.43:0/1413627031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df4102210 0x7f9df419a6b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9de4031440 tx=0x7f9de4003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9ddbfff640 1 -- 172.21.15.43:0/1413627031 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de403f070 con 0x7f9df4102210 2024-04-25T10:32:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.690+0000 7f9ddbfff640 1 -- 172.21.15.43:0/1413627031 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9de4031d60 con 0x7f9df4102210 2024-04-25T10:32:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.691+0000 7f9df92e0640 1 -- 172.21.15.43:0/1413627031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df419f720 con 0x7f9df4102210 2024-04-25T10:32:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.691+0000 7f9ddbfff640 1 -- 172.21.15.43:0/1413627031 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de4035030 con 0x7f9df4102210 2024-04-25T10:32:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.691+0000 7f9df92e0640 1 -- 172.21.15.43:0/1413627031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df419fc60 con 0x7f9df4102210 2024-04-25T10:32:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.692+0000 7f9df92e0640 1 -- 172.21.15.43:0/1413627031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9df406a190 con 0x7f9df4102210 2024-04-25T10:32:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.693+0000 7f9ddbfff640 1 -- 172.21.15.43:0/1413627031 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 147747+0+0 (secure 0 0 0) 0x7f9de4042070 con 0x7f9df4102210 2024-04-25T10:32:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.694+0000 7f9ddbfff640 1 --2- 172.21.15.43:0/1413627031 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9ddc0afe00 0x7f9ddc0b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.694+0000 7f9ddbfff640 1 -- 172.21.15.43:0/1413627031 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f9de4100240 con 0x7f9df4102210 2024-04-25T10:32:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.694+0000 7f9df27fc640 1 --2- 172.21.15.43:0/1413627031 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9ddc0afe00 0x7f9ddc0b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.694+0000 7f9df27fc640 1 --2- 172.21.15.43:0/1413627031 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9ddc0afe00 0x7f9ddc0b2310 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9de8005d00 tx=0x7f9de8005ef0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:39.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.697+0000 7f9ddbfff640 1 -- 172.21.15.43:0/1413627031 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9de40c8930 con 0x7f9df4102210 2024-04-25T10:32:39.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.887+0000 7f9df92e0640 1 -- 172.21.15.43:0/1413627031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9df410bc90 con 0x7f9df4102210 2024-04-25T10:32:39.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.888+0000 7f9ddbfff640 1 -- 172.21.15.43:0/1413627031 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22996 (secure 0 0 0) 0x7f9de40c8230 con 0x7f9df4102210 2024-04-25T10:32:39.891 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:32:39.891 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":80,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:32:32.456657+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:31.503414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:32.224334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1599315071":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6801/1953858940":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6800/1953858940":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3026776968":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/188939647":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/1287690800":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:32:39.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.890+0000 7f9df92e0640 1 -- 172.21.15.43:0/1413627031 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9ddc0afe00 msgr2=0x7f9ddc0b2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:39.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.890+0000 7f9df92e0640 1 --2- 172.21.15.43:0/1413627031 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9ddc0afe00 0x7f9ddc0b2310 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9de8005d00 tx=0x7f9de8005ef0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.890+0000 7f9df92e0640 1 -- 172.21.15.43:0/1413627031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df4102210 msgr2=0x7f9df419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:39.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.890+0000 7f9df92e0640 1 --2- 172.21.15.43:0/1413627031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df4102210 0x7f9df419a6b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9de4031440 tx=0x7f9de4003cd0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.890+0000 7f9df92e0640 1 -- 172.21.15.43:0/1413627031 shutdown_connections 2024-04-25T10:32:39.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.890+0000 7f9df92e0640 1 --2- 172.21.15.43:0/1413627031 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9ddc0afe00 0x7f9ddc0b2310 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.890+0000 7f9df92e0640 1 --2- 172.21.15.43:0/1413627031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9df4108e00 0x7f9df419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.890+0000 7f9df92e0640 1 --2- 172.21.15.43:0/1413627031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9df4108380 0x7f9df419ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.890+0000 7f9df92e0640 1 --2- 172.21.15.43:0/1413627031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df4102210 0x7f9df419a6b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:39.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.890+0000 7f9df92e0640 1 -- 172.21.15.43:0/1413627031 >> 172.21.15.43:0/1413627031 conn(0x7f9df4077880 msgr2=0x7f9df40ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:39.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.891+0000 7f9df92e0640 1 -- 172.21.15.43:0/1413627031 shutdown_connections 2024-04-25T10:32:39.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:39.891+0000 7f9df92e0640 1 -- 172.21.15.43:0/1413627031 wait complete. 2024-04-25T10:32:39.904 DEBUG:tasks.fwd_scrub:Yielding 2024-04-25T10:32:39.905 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T10:32:39.914 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T10:32:39.915 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T10:32:39.915 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-04-25T10:32:39.915 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-25T10:32:39.915 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:32:39.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:39.916 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:32:40.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.029+0000 7f515b2bd640 1 -- 172.21.15.43:0/1149282794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5154108a60 msgr2=0x7f5154102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.029+0000 7f515b2bd640 1 --2- 172.21.15.43:0/1149282794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5154108a60 0x7f5154102610 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f514c009ca0 tx=0x7f514c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.029+0000 7f515b2bd640 1 -- 172.21.15.43:0/1149282794 shutdown_connections 2024-04-25T10:32:40.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.029+0000 7f515b2bd640 1 --2- 172.21.15.43:0/1149282794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5154108f00 0x7f5154102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.029+0000 7f515b2bd640 1 --2- 172.21.15.43:0/1149282794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51541031c0 0x7f515410f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.029+0000 7f515b2bd640 1 --2- 172.21.15.43:0/1149282794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5154108a60 0x7f5154102610 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.029+0000 7f515b2bd640 1 -- 172.21.15.43:0/1149282794 >> 172.21.15.43:0/1149282794 conn(0x7f51540781b0 msgr2=0x7f51540785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:40.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.029+0000 7f515b2bd640 1 -- 172.21.15.43:0/1149282794 shutdown_connections 2024-04-25T10:32:40.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.029+0000 7f515b2bd640 1 -- 172.21.15.43:0/1149282794 wait complete. 2024-04-25T10:32:40.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.030+0000 7f515b2bd640 1 Processor -- start 2024-04-25T10:32:40.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.030+0000 7f515b2bd640 1 -- start start 2024-04-25T10:32:40.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.030+0000 7f515b2bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51541031c0 0x7f515419b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.030+0000 7f515b2bd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5154108a60 0x7f515419bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.030+0000 7f515b2bd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5154108f00 0x7f515419c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.030+0000 7f515b2bd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5154112c60 con 0x7f51541031c0 2024-04-25T10:32:40.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.030+0000 7f515b2bd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5154112ae0 con 0x7f5154108a60 2024-04-25T10:32:40.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.030+0000 7f515b2bd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5154112de0 con 0x7f5154108f00 2024-04-25T10:32:40.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.030+0000 7f5159059640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51541031c0 0x7f515419b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.030+0000 7f5159059640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51541031c0 0x7f515419b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50562/0 (socket says 172.21.15.43:50562) 2024-04-25T10:32:40.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7f5159059640 1 -- 172.21.15.43:0/1797544320 learned_addr learned my addr 172.21.15.43:0/1797544320 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:40.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7f515985a640 1 --2- 172.21.15.43:0/1797544320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5154108f00 0x7f515419c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7fc3cfb2e640 1 -- 172.21.15.43:0/941928667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3c810b690 msgr2=0x7fc3c810dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7fc3cfb2e640 1 --2- 172.21.15.43:0/941928667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3c810b690 0x7fc3c810dab0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009900 tx=0x7fc3b802a410 comp rx=0 tx=0).stop 2024-04-25T10:32:40.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7f5158858640 1 --2- 172.21.15.43:0/1797544320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5154108a60 0x7f515419bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7fc3cfb2e640 1 -- 172.21.15.43:0/941928667 shutdown_connections 2024-04-25T10:32:40.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7fc3cfb2e640 1 --2- 172.21.15.43:0/941928667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3c810e080 0x7fc3c8110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7fc3cfb2e640 1 --2- 172.21.15.43:0/941928667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3c810b690 0x7fc3c810dab0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7fc3cfb2e640 1 --2- 172.21.15.43:0/941928667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3c8104c30 0x7fc3c8105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7fc3cfb2e640 1 -- 172.21.15.43:0/941928667 >> 172.21.15.43:0/941928667 conn(0x7fc3c8078110 msgr2=0x7fc3c8078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:40.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7f5159059640 1 -- 172.21.15.43:0/1797544320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5154108f00 msgr2=0x7f515419c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7f5159059640 1 --2- 172.21.15.43:0/1797544320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5154108f00 0x7f515419c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7f5159059640 1 -- 172.21.15.43:0/1797544320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5154108a60 msgr2=0x7f515419bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7f5159059640 1 --2- 172.21.15.43:0/1797544320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5154108a60 0x7f515419bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7f5159059640 1 -- 172.21.15.43:0/1797544320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51541a68b0 con 0x7f51541031c0 2024-04-25T10:32:40.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7f5158858640 1 --2- 172.21.15.43:0/1797544320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5154108a60 0x7f515419bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:40.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7fc3cfb2e640 1 -- 172.21.15.43:0/941928667 shutdown_connections 2024-04-25T10:32:40.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7fc3cfb2e640 1 -- 172.21.15.43:0/941928667 wait complete. 2024-04-25T10:32:40.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7f5159059640 1 --2- 172.21.15.43:0/1797544320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51541031c0 0x7f515419b5a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f514c0098d0 tx=0x7f514c0317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:40.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7f51527fc640 1 -- 172.21.15.43:0/1797544320 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c003b70 con 0x7f51541031c0 2024-04-25T10:32:40.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7f51527fc640 1 -- 172.21.15.43:0/1797544320 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f514c003d10 con 0x7f51541031c0 2024-04-25T10:32:40.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7f515b2bd640 1 -- 172.21.15.43:0/1797544320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51541a6b40 con 0x7f51541031c0 2024-04-25T10:32:40.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7f51527fc640 1 -- 172.21.15.43:0/1797544320 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c02fa60 con 0x7f51541031c0 2024-04-25T10:32:40.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7f515b2bd640 1 -- 172.21.15.43:0/1797544320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51541a70e0 con 0x7f51541031c0 2024-04-25T10:32:40.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.031+0000 7fc3cfb2e640 1 Processor -- start 2024-04-25T10:32:40.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.032+0000 7fc3cfb2e640 1 -- start start 2024-04-25T10:32:40.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.032+0000 7fc3cfb2e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3c8104c30 0x7fc3c81024f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.032+0000 7fc3cfb2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3c810b690 0x7fc3c8102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.032+0000 7fc3cfb2e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3c810e080 0x7fc3c8102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.032+0000 7fc3cfb2e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c8113ce0 con 0x7fc3c810b690 2024-04-25T10:32:40.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.032+0000 7fc3cfb2e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c8113b60 con 0x7fc3c810e080 2024-04-25T10:32:40.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.032+0000 7fc3cfb2e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c8113e60 con 0x7fc3c8104c30 2024-04-25T10:32:40.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.032+0000 7fc3cd0c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3c810b690 0x7fc3c8102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.032+0000 7fc3cd0c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3c810b690 0x7fc3c8102a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50566/0 (socket says 172.21.15.43:50566) 2024-04-25T10:32:40.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.032+0000 7fc3cd0c9640 1 -- 172.21.15.43:0/1388256265 learned_addr learned my addr 172.21.15.43:0/1388256265 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:40.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.032+0000 7fc3cd8ca640 1 --2- 172.21.15.43:0/1388256265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3c8104c30 0x7fc3c81024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.032+0000 7fc3ce0cb640 1 --2- 172.21.15.43:0/1388256265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3c810e080 0x7fc3c8102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.033+0000 7fc3cd0c9640 1 -- 172.21.15.43:0/1388256265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3c8104c30 msgr2=0x7fc3c81024f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.033+0000 7fc3cd0c9640 1 --2- 172.21.15.43:0/1388256265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3c8104c30 0x7fc3c81024f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.033+0000 7fc3cd0c9640 1 -- 172.21.15.43:0/1388256265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3c810e080 msgr2=0x7fc3c8102f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.033+0000 7fc3cd0c9640 1 --2- 172.21.15.43:0/1388256265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3c810e080 0x7fc3c8102f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.033+0000 7fc3cd0c9640 1 -- 172.21.15.43:0/1388256265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c81a8aa0 con 0x7fc3c810b690 2024-04-25T10:32:40.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.033+0000 7fc3cd8ca640 1 --2- 172.21.15.43:0/1388256265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3c8104c30 0x7fc3c81024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:40.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.033+0000 7fc3ce0cb640 1 --2- 172.21.15.43:0/1388256265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3c810e080 0x7fc3c8102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:40.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.033+0000 7f515b2bd640 1 -- 172.21.15.43:0/1797544320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f515406a110 con 0x7f51541031c0 2024-04-25T10:32:40.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.033+0000 7fc3cd0c9640 1 --2- 172.21.15.43:0/1388256265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3c810b690 0x7fc3c8102a30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8005940 tx=0x7fc3b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:40.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.033+0000 7fc3beffd640 1 -- 172.21.15.43:0/1388256265 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b803f030 con 0x7fc3c810b690 2024-04-25T10:32:40.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.033+0000 7fc3beffd640 1 -- 172.21.15.43:0/1388256265 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3b8003aa0 con 0x7fc3c810b690 2024-04-25T10:32:40.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.033+0000 7fc3cfb2e640 1 -- 172.21.15.43:0/1388256265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c81a8d30 con 0x7fc3c810b690 2024-04-25T10:32:40.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.033+0000 7fc3beffd640 1 -- 172.21.15.43:0/1388256265 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8035070 con 0x7fc3c810b690 2024-04-25T10:32:40.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.033+0000 7fc3cfb2e640 1 -- 172.21.15.43:0/1388256265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c81a91d0 con 0x7fc3c810b690 2024-04-25T10:32:40.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.034+0000 7f51527fc640 1 -- 172.21.15.43:0/1797544320 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 147747+0+0 (secure 0 0 0) 0x7f514c02fc00 con 0x7f51541031c0 2024-04-25T10:32:40.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.035+0000 7f51527fc640 1 --2- 172.21.15.43:0/1797544320 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f513c0afdf0 0x7f513c0b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.035+0000 7f5158858640 1 --2- 172.21.15.43:0/1797544320 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f513c0afdf0 0x7f513c0b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.035+0000 7f51527fc640 1 -- 172.21.15.43:0/1797544320 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f514c100530 con 0x7f51541031c0 2024-04-25T10:32:40.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.036+0000 7fc3d69d9740 1 Processor -- start 2024-04-25T10:32:40.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.036+0000 7f5158858640 1 --2- 172.21.15.43:0/1797544320 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f513c0afdf0 0x7f513c0b2300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f515410e790 tx=0x7f5148005f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.036+0000 7fc3d69d9740 1 -- start start 2024-04-25T10:32:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.036+0000 7fc3d69d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afec410 0x55827b38c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.036+0000 7fc3d69d9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b38cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.036+0000 7fc3d69d9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afcb730 0x55827b38d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.036+0000 7fc3d69d9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55827b2f2b00 con 0x55827afc3550 2024-04-25T10:32:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.036+0000 7fc3d69d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55827b28e920 con 0x55827afec410 2024-04-25T10:32:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.036+0000 7fc3d69d9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55827b2eeac0 con 0x55827afcb730 2024-04-25T10:32:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.036+0000 7fc3beffd640 1 -- 172.21.15.43:0/1388256265 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 147747+0+0 (secure 0 0 0) 0x7fc3b8042030 con 0x7fc3c810b690 2024-04-25T10:32:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.037+0000 7fc3cd8ca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b38cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.037+0000 7fc3cd8ca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b38cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50582/0 (socket says 172.21.15.43:50582) 2024-04-25T10:32:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.037+0000 7fc3cd8ca640 1 -- 172.21.15.43:0/767679443 learned_addr learned my addr 172.21.15.43:0/767679443 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.037+0000 7fc3ce0cb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afec410 0x55827b38c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.037+0000 7fc3cd0c9640 1 --2- 172.21.15.43:0/767679443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afcb730 0x55827b38d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.037+0000 7fc3cd8ca640 1 -- 172.21.15.43:0/767679443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afcb730 msgr2=0x55827b38d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.037+0000 7fc3cd8ca640 1 --2- 172.21.15.43:0/767679443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afcb730 0x55827b38d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.037+0000 7fc3cd8ca640 1 -- 172.21.15.43:0/767679443 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afec410 msgr2=0x55827b38c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.037+0000 7fc3cd8ca640 1 --2- 172.21.15.43:0/767679443 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afec410 0x55827b38c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.037+0000 7fc3cd8ca640 1 -- 172.21.15.43:0/767679443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55827b0cc7e0 con 0x55827afc3550 2024-04-25T10:32:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.037+0000 7fc3ce0cb640 1 --2- 172.21.15.43:0/767679443 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afec410 0x55827b38c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.037+0000 7fc3cd8ca640 1 --2- 172.21.15.43:0/767679443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b38cb60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4010480 tx=0x7fc3c4010940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8be397d3e74602 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.037+0000 7fc3beffd640 1 --2- 172.21.15.43:0/1388256265 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc3b40afd80 0x7fc3b40b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.037+0000 7fc3ce0cb640 1 --2- 172.21.15.43:0/1388256265 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc3b40afd80 0x7fc3b40b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.037+0000 7fc3beffd640 1 -- 172.21.15.43:0/1388256265 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fc3b80ff8a0 con 0x7fc3c810b690 2024-04-25T10:32:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.038+0000 7fc3ce0cb640 1 --2- 172.21.15.43:0/1388256265 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc3b40afd80 0x7fc3b40b2290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x55827b389790 tx=0x7fc3c0009500 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.040+0000 7fc3b3fff640 1 -- 172.21.15.43:0/767679443 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4011760 con 0x55827afc3550 2024-04-25T10:32:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.040+0000 7fc3b3fff640 1 -- 172.21.15.43:0/767679443 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3c4011db0 con 0x55827afc3550 2024-04-25T10:32:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.040+0000 7fc3b3fff640 1 -- 172.21.15.43:0/767679443 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c400e430 con 0x55827afc3550 2024-04-25T10:32:40.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.040+0000 7f51527fc640 1 -- 172.21.15.43:0/1797544320 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f514c0c8cc0 con 0x7f51541031c0 2024-04-25T10:32:40.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.041+0000 7fc3d69d9740 1 -- 172.21.15.43:0/767679443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 msgr2=0x55827b38cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.041+0000 7fc3d69d9740 1 --2- 172.21.15.43:0/767679443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b38cb60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4010480 tx=0x7fc3c4010940 comp rx=0 tx=0).stop 2024-04-25T10:32:40.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.041+0000 7fc3d69d9740 1 -- 172.21.15.43:0/767679443 shutdown_connections 2024-04-25T10:32:40.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.041+0000 7fc3d69d9740 1 --2- 172.21.15.43:0/767679443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afcb730 0x55827b38d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.041+0000 7fc3d69d9740 1 --2- 172.21.15.43:0/767679443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b38cb60 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.041+0000 7fc3d69d9740 1 --2- 172.21.15.43:0/767679443 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afec410 0x55827b38c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.041+0000 7fc3d69d9740 1 -- 172.21.15.43:0/767679443 >> 172.21.15.43:0/767679443 conn(0x55827afe0560 msgr2=0x55827b388dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:40.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.041+0000 7fc3d69d9740 1 -- 172.21.15.43:0/767679443 shutdown_connections 2024-04-25T10:32:40.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.041+0000 7fc3d69d9740 1 -- 172.21.15.43:0/767679443 wait complete. 2024-04-25T10:32:40.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.042+0000 7fc3d69d9740 1 -- start start 2024-04-25T10:32:40.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.042+0000 7fc3d69d9740 1 Processor -- start 2024-04-25T10:32:40.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.042+0000 7fc3d69d9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b396be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.042+0000 7fc3d69d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afcb730 0x55827b397120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.042+0000 7fc3d69d9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afec410 0x55827b3a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.042+0000 7fc3d69d9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55827b399970 con 0x55827afc3550 2024-04-25T10:32:40.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.042+0000 7fc3d69d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55827b28c5d0 con 0x55827afcb730 2024-04-25T10:32:40.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.042+0000 7fc3d69d9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55827b399af0 con 0x55827afec410 2024-04-25T10:32:40.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.043+0000 7fc3d69d9740 10 client.0 _ll_register_callbacks cb 0x55827b27a080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:40.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.043+0000 7fc3cd8ca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b396be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.043+0000 7fc3cd8ca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b396be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50586/0 (socket says 172.21.15.43:50586) 2024-04-25T10:32:40.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.043+0000 7fc3cd8ca640 1 -- 172.21.15.43:0/1865345976 learned_addr learned my addr 172.21.15.43:0/1865345976 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:40.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.043+0000 7fc3cd0c9640 1 --2- 172.21.15.43:0/1865345976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afcb730 0x55827b397120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.043+0000 7fc3ce0cb640 1 --2- 172.21.15.43:0/1865345976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afec410 0x55827b3a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.043+0000 7fc3cd8ca640 1 -- 172.21.15.43:0/1865345976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afec410 msgr2=0x55827b3a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:40.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.043+0000 7fc3cd8ca640 1 --2- 172.21.15.43:0/1865345976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afec410 0x55827b3a2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.043+0000 7fc3cd8ca640 1 -- 172.21.15.43:0/1865345976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afcb730 msgr2=0x55827b397120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:40.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.043+0000 7fc3cd8ca640 1 --2- 172.21.15.43:0/1865345976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afcb730 0x55827b397120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.043+0000 7fc3cd8ca640 1 -- 172.21.15.43:0/1865345976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55827b12cf70 con 0x55827afc3550 2024-04-25T10:32:40.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.043+0000 7fc3cd8ca640 1 --2- 172.21.15.43:0/1865345976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b396be0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4010450 tx=0x7fc3c401c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1193f1d7c9c100f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:40.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.043+0000 7fc3b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:32:40.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.043+0000 7fc3b17fa640 1 -- 172.21.15.43:0/1865345976 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4029020 con 0x55827afc3550 2024-04-25T10:32:40.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.043+0000 7fc3b17fa640 1 -- 172.21.15.43:0/1865345976 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fc3c4004220 con 0x55827afc3550 2024-04-25T10:32:40.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.043+0000 7fc3d69d9740 1 -- 172.21.15.43:0/1865345976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3c4011db0 con 0x55827afc3550 2024-04-25T10:32:40.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.044+0000 7fc3b17fa640 1 -- 172.21.15.43:0/1865345976 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc3c4004420 con 0x55827afc3550 2024-04-25T10:32:40.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.044+0000 7fc3d69d9740 10 client.15486 fetch_fsmap learned FSMap version 24 2024-04-25T10:32:40.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.044+0000 7fc3d69d9740 1 -- 172.21.15.43:0/1865345976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55827afc4170 con 0x55827afc3550 2024-04-25T10:32:40.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.044+0000 7fc3b17fa640 1 -- 172.21.15.43:0/1865345976 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3c4011db0 con 0x55827afc3550 2024-04-25T10:32:40.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.044+0000 7fc3b17fa640 1 -- 172.21.15.43:0/1865345976 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fc3c4024070 con 0x55827afc3550 2024-04-25T10:32:40.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.044+0000 7fc3d69d9740 10 client.15486 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:32:40.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.044+0000 7fc3d69d9740 10 client.15486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:32:40.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.044+0000 7fc3d69d9740 20 client.15486 populate_metadata read hostname 'smithi043' 2024-04-25T10:32:40.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.044+0000 7fc3d69d9740 1 --2- 172.21.15.43:0/1865345976 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55827afab1f0 0x55827b3a8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.044+0000 7fc3d69d9740 4 client.15486 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:32:40.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.044+0000 7fc3d69d9740 1 -- 172.21.15.43:0/1865345976 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55827b300710 con 0x55827afab1f0 2024-04-25T10:32:40.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.044+0000 7fc3cd0c9640 1 --2- 172.21.15.43:0/1865345976 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55827afab1f0 0x55827b3a8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.045+0000 7fc3cd0c9640 1 --2- 172.21.15.43:0/1865345976 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55827afab1f0 0x55827b3a8f10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c55159fcaa939360 server_cookie=162bc1174c752fb0 in_seq=0 out_seq=0 2024-04-25T10:32:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.045+0000 7fc3b17fa640 10 client.15486 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:32:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.045+0000 7fc3b17fa640 1 -- 172.21.15.43:0/1865345976 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55827b300710 con 0x55827afab1f0 2024-04-25T10:32:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.045+0000 7fc3b17fa640 10 client.15486 handle_command_reply: tid=0 2024-04-25T10:32:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.045+0000 7fc3d69d9740 1 client.15486 shutdown 2024-04-25T10:32:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.046+0000 7fc3d69d9740 1 -- 172.21.15.43:0/1865345976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 msgr2=0x55827b396be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.046+0000 7fc3d69d9740 1 --2- 172.21.15.43:0/1865345976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b396be0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4010450 tx=0x7fc3c401c6b0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.046+0000 7fc3d69d9740 1 -- 172.21.15.43:0/1865345976 shutdown_connections 2024-04-25T10:32:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.046+0000 7fc3d69d9740 1 --2- 172.21.15.43:0/1865345976 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55827afab1f0 0x55827b3a8f10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.046+0000 7fc3d69d9740 1 --2- 172.21.15.43:0/1865345976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afcb730 0x55827b397120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.046+0000 7fc3d69d9740 1 --2- 172.21.15.43:0/1865345976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afec410 0x55827b3a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.046+0000 7fc3b17fa640 0 client.15486 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:32:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.046+0000 7fc3d69d9740 1 --2- 172.21.15.43:0/1865345976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b396be0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.046+0000 7fc3d69d9740 1 -- 172.21.15.43:0/1865345976 >> 172.21.15.43:0/1865345976 conn(0x55827afe0560 msgr2=0x55827b38e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:40.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.046+0000 7fc3d69d9740 1 -- 172.21.15.43:0/1865345976 shutdown_connections 2024-04-25T10:32:40.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.046+0000 7fc3d69d9740 1 -- 172.21.15.43:0/1865345976 wait complete. 2024-04-25T10:32:40.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.046+0000 7fc3d69d9740 20 client.15486 trim_cache size 0 max 16384 2024-04-25T10:32:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.064+0000 7fc3d69d9740 1 Processor -- start 2024-04-25T10:32:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.064+0000 7fc3d69d9740 1 -- start start 2024-04-25T10:32:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.065+0000 7fc3d69d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afab1f0 0x55827b387ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.065+0000 7fc3d69d9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b38cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.065+0000 7fc3d69d9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afcb730 0x55827b3ae510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.065+0000 7fc3d69d9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c81b4450 con 0x55827afc3550 2024-04-25T10:32:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.065+0000 7fc3d69d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c81b1d20 con 0x55827afab1f0 2024-04-25T10:32:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.065+0000 7fc3d69d9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c81afb70 con 0x55827afcb730 2024-04-25T10:32:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.065+0000 7fc3cd0c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b38cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.065+0000 7fc3cd0c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b38cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50600/0 (socket says 172.21.15.43:50600) 2024-04-25T10:32:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.065+0000 7fc3cd0c9640 1 -- 172.21.15.43:0/1384344200 learned_addr learned my addr 172.21.15.43:0/1384344200 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.065+0000 7fc3cd8ca640 1 --2- 172.21.15.43:0/1384344200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afab1f0 0x55827b387ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.065+0000 7fc3ce0cb640 1 --2- 172.21.15.43:0/1384344200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afcb730 0x55827b3ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.065+0000 7fc3cd0c9640 1 -- 172.21.15.43:0/1384344200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afcb730 msgr2=0x55827b3ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:40.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.065+0000 7fc3cd0c9640 1 --2- 172.21.15.43:0/1384344200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afcb730 0x55827b3ae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.065+0000 7fc3cd0c9640 1 -- 172.21.15.43:0/1384344200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afab1f0 msgr2=0x55827b387ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:40.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.065+0000 7fc3cd0c9640 1 --2- 172.21.15.43:0/1384344200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afab1f0 0x55827b387ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.065+0000 7fc3cd0c9640 1 -- 172.21.15.43:0/1384344200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55827b397e90 con 0x55827afc3550 2024-04-25T10:32:40.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.065+0000 7fc3cd0c9640 1 --2- 172.21.15.43:0/1384344200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b38cbd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8006010 tx=0x7fc3b80ea040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f70f4eae3fd7c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:40.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.066+0000 7fc3b37fe640 1 -- 172.21.15.43:0/1384344200 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8049070 con 0x55827afc3550 2024-04-25T10:32:40.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.066+0000 7fc3b37fe640 1 -- 172.21.15.43:0/1384344200 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3b80d0090 con 0x55827afc3550 2024-04-25T10:32:40.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.066+0000 7fc3b37fe640 1 -- 172.21.15.43:0/1384344200 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b80d0750 con 0x55827afc3550 2024-04-25T10:32:40.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.067+0000 7fc3d69d9740 1 -- 172.21.15.43:0/1384344200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 msgr2=0x55827b38cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.067+0000 7fc3d69d9740 1 --2- 172.21.15.43:0/1384344200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b38cbd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8006010 tx=0x7fc3b80ea040 comp rx=0 tx=0).stop 2024-04-25T10:32:40.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.067+0000 7fc3d69d9740 1 -- 172.21.15.43:0/1384344200 shutdown_connections 2024-04-25T10:32:40.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.067+0000 7fc3d69d9740 1 --2- 172.21.15.43:0/1384344200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afcb730 0x55827b3ae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.067+0000 7fc3d69d9740 1 --2- 172.21.15.43:0/1384344200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b38cbd0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.067+0000 7fc3d69d9740 1 --2- 172.21.15.43:0/1384344200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afab1f0 0x55827b387ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.067+0000 7fc3d69d9740 1 -- 172.21.15.43:0/1384344200 >> 172.21.15.43:0/1384344200 conn(0x55827afe0560 msgr2=0x55827b38c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:40.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.067+0000 7fc3d69d9740 1 -- 172.21.15.43:0/1384344200 shutdown_connections 2024-04-25T10:32:40.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.067+0000 7fc3d69d9740 1 -- 172.21.15.43:0/1384344200 wait complete. 2024-04-25T10:32:40.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.067+0000 7fc3d69d9740 1 -- start start 2024-04-25T10:32:40.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.067+0000 7fc3d69d9740 1 Processor -- start 2024-04-25T10:32:40.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.067+0000 7fc3d69d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afab1f0 0x55827b39a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.067+0000 7fc3d69d9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b39a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3d69d9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afcb730 0x55827b40f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3d69d9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c81b01e0 con 0x55827afc3550 2024-04-25T10:32:40.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3d69d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c81b1650 con 0x55827afab1f0 2024-04-25T10:32:40.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3d69d9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c81b08b0 con 0x55827afcb730 2024-04-25T10:32:40.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3d69d9740 10 client.0 _ll_register_callbacks cb 0x7fc3c40056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:40.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3cd0c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b39a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3cd0c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b39a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50608/0 (socket says 172.21.15.43:50608) 2024-04-25T10:32:40.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3cd0c9640 1 -- 172.21.15.43:0/2009498568 learned_addr learned my addr 172.21.15.43:0/2009498568 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:40.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3cd8ca640 1 --2- 172.21.15.43:0/2009498568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afab1f0 0x55827b39a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3ce0cb640 1 --2- 172.21.15.43:0/2009498568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afcb730 0x55827b40f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3cd0c9640 1 -- 172.21.15.43:0/2009498568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afcb730 msgr2=0x55827b40f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:40.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3cd0c9640 1 --2- 172.21.15.43:0/2009498568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afcb730 0x55827b40f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3cd0c9640 1 -- 172.21.15.43:0/2009498568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afab1f0 msgr2=0x55827b39a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:40.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3cd0c9640 1 --2- 172.21.15.43:0/2009498568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afab1f0 0x55827b39a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3cd0c9640 1 -- 172.21.15.43:0/2009498568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55827b40f5f0 con 0x55827afc3550 2024-04-25T10:32:40.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3ce0cb640 1 --2- 172.21.15.43:0/2009498568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afcb730 0x55827b40f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:40.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3cd0c9640 1 --2- 172.21.15.43:0/2009498568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b39a670 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc3b80c7880 tx=0x7fc3b803bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3752b2b7cf5c8d3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:40.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:32:40.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3b17fa640 1 -- 172.21.15.43:0/2009498568 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8049070 con 0x55827afc3550 2024-04-25T10:32:40.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3b17fa640 1 -- 172.21.15.43:0/2009498568 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fc3b80ea0b0 con 0x55827afc3550 2024-04-25T10:32:40.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3d69d9740 1 -- 172.21.15.43:0/2009498568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55827b2fa580 con 0x55827afc3550 2024-04-25T10:32:40.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3b17fa640 1 -- 172.21.15.43:0/2009498568 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc3b80c7ff0 con 0x55827afc3550 2024-04-25T10:32:40.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3d69d9740 10 client.15492 fetch_fsmap learned FSMap version 24 2024-04-25T10:32:40.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.068+0000 7fc3d69d9740 1 -- 172.21.15.43:0/2009498568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55827b410880 con 0x55827afc3550 2024-04-25T10:32:40.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.069+0000 7fc3b17fa640 1 -- 172.21.15.43:0/2009498568 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55827b2fa580 con 0x55827afc3550 2024-04-25T10:32:40.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.069+0000 7fc3b17fa640 1 -- 172.21.15.43:0/2009498568 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fc3b80d02c0 con 0x55827afc3550 2024-04-25T10:32:40.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.069+0000 7fc3d69d9740 10 client.15492 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:32:40.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.069+0000 7fc3d69d9740 10 client.15492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:32:40.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.069+0000 7fc3d69d9740 20 client.15492 populate_metadata read hostname 'smithi043' 2024-04-25T10:32:40.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.069+0000 7fc3d69d9740 1 --2- 172.21.15.43:0/2009498568 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55827afec410 0x55827b413020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.069+0000 7fc3d69d9740 4 client.15492 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:32:40.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.069+0000 7fc3d69d9740 1 -- 172.21.15.43:0/2009498568 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55827b275ac0 con 0x55827afec410 2024-04-25T10:32:40.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.069+0000 7fc3cd8ca640 1 --2- 172.21.15.43:0/2009498568 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55827afec410 0x55827b413020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.070+0000 7fc3cd8ca640 1 --2- 172.21.15.43:0/2009498568 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55827afec410 0x55827b413020 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9acfab9f84dca4f2 server_cookie=eaa81146556fd4b7 in_seq=0 out_seq=0 2024-04-25T10:32:40.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.070+0000 7fc3b17fa640 10 client.15492 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:32:40.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.071+0000 7fc3b17fa640 1 -- 172.21.15.43:0/2009498568 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55827b275ac0 con 0x55827afec410 2024-04-25T10:32:40.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.071+0000 7fc3b17fa640 10 client.15492 handle_command_reply: tid=0 2024-04-25T10:32:40.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.071+0000 7fc3d69d9740 1 client.15492 shutdown 2024-04-25T10:32:40.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.071+0000 7fc3d69d9740 1 -- 172.21.15.43:0/2009498568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 msgr2=0x55827b39a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.071+0000 7fc3d69d9740 1 --2- 172.21.15.43:0/2009498568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b39a670 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc3b80c7880 tx=0x7fc3b803bf10 comp rx=0 tx=0).stop 2024-04-25T10:32:40.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.071+0000 7fc3d69d9740 1 -- 172.21.15.43:0/2009498568 shutdown_connections 2024-04-25T10:32:40.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.071+0000 7fc3d69d9740 1 --2- 172.21.15.43:0/2009498568 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55827afec410 0x55827b413020 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.071+0000 7fc3d69d9740 1 --2- 172.21.15.43:0/2009498568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55827afcb730 0x55827b40f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.071+0000 7fc3d69d9740 1 --2- 172.21.15.43:0/2009498568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55827afc3550 0x55827b39a670 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.071+0000 7fc3b17fa640 0 client.15492 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:32:40.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.071+0000 7fc3d69d9740 1 --2- 172.21.15.43:0/2009498568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827afab1f0 0x55827b39a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.071+0000 7fc3d69d9740 1 -- 172.21.15.43:0/2009498568 >> 172.21.15.43:0/2009498568 conn(0x55827afe0560 msgr2=0x55827b38cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:40.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.071+0000 7fc3d69d9740 1 -- 172.21.15.43:0/2009498568 shutdown_connections 2024-04-25T10:32:40.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.071+0000 7fc3d69d9740 1 -- 172.21.15.43:0/2009498568 wait complete. 2024-04-25T10:32:40.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.071+0000 7fc3d69d9740 20 client.15492 trim_cache size 0 max 16384 2024-04-25T10:32:40.079 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:32:40.079 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:32:40.079 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "8c88fa0b-0bf8-4970-bab4-598574afe4fc", 2024-04-25T10:32:40.079 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:32:40.079 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:32:40.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.072+0000 7fc3cfb2e640 1 -- 172.21.15.43:0/1388256265 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc3b40afd80 msgr2=0x7fc3b40b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.072+0000 7fc3cfb2e640 1 --2- 172.21.15.43:0/1388256265 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc3b40afd80 0x7fc3b40b2290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x55827b389790 tx=0x7fc3c0009500 comp rx=0 tx=0).stop 2024-04-25T10:32:40.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.072+0000 7fc3cfb2e640 1 -- 172.21.15.43:0/1388256265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3c810b690 msgr2=0x7fc3c8102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.072+0000 7fc3cfb2e640 1 --2- 172.21.15.43:0/1388256265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3c810b690 0x7fc3c8102a30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8005940 tx=0x7fc3b8004270 comp rx=0 tx=0).stop 2024-04-25T10:32:40.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.072+0000 7fc3cfb2e640 1 -- 172.21.15.43:0/1388256265 shutdown_connections 2024-04-25T10:32:40.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.072+0000 7fc3cfb2e640 1 --2- 172.21.15.43:0/1388256265 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc3b40afd80 0x7fc3b40b2290 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.072+0000 7fc3cfb2e640 1 --2- 172.21.15.43:0/1388256265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3c810e080 0x7fc3c8102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.072+0000 7fc3cfb2e640 1 --2- 172.21.15.43:0/1388256265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3c810b690 0x7fc3c8102a30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.072+0000 7fc3cfb2e640 1 --2- 172.21.15.43:0/1388256265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3c8104c30 0x7fc3c81024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.072+0000 7fc3cfb2e640 1 -- 172.21.15.43:0/1388256265 >> 172.21.15.43:0/1388256265 conn(0x7fc3c8078110 msgr2=0x7fc3c8101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:40.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.072+0000 7fc3cfb2e640 1 -- 172.21.15.43:0/1388256265 shutdown_connections 2024-04-25T10:32:40.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.072+0000 7fc3cfb2e640 1 -- 172.21.15.43:0/1388256265 wait complete. 2024-04-25T10:32:40.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:32:40.086 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:32:40.106 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:39 smithi043 ceph-mon[29478]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-25T10:32:40.106 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:39 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1206886127' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:32:40.106 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:39 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2642451307' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:32:40.106 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:39 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/4257268595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:32:40.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.198+0000 7f52bf534640 1 -- 172.21.15.43:0/3502551824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52b8110400 msgr2=0x7f52b8112820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.198+0000 7f52bf534640 1 --2- 172.21.15.43:0/3502551824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52b8110400 0x7f52b8112820 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009ca0 tx=0x7f52b002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.198+0000 7f52bf534640 1 -- 172.21.15.43:0/3502551824 shutdown_connections 2024-04-25T10:32:40.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.198+0000 7f52bf534640 1 --2- 172.21.15.43:0/3502551824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b810daa0 0x7f52b810fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.198+0000 7f52bf534640 1 --2- 172.21.15.43:0/3502551824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52b8110400 0x7f52b8112820 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.198+0000 7f52bf534640 1 --2- 172.21.15.43:0/3502551824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52b8102860 0x7f52b8102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.198+0000 7f52bf534640 1 -- 172.21.15.43:0/3502551824 >> 172.21.15.43:0/3502551824 conn(0x7f52b8078100 msgr2=0x7f52b8078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:40.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.198+0000 7f52bf534640 1 -- 172.21.15.43:0/3502551824 shutdown_connections 2024-04-25T10:32:40.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.198+0000 7f52bf534640 1 -- 172.21.15.43:0/3502551824 wait complete. 2024-04-25T10:32:40.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.198+0000 7f52bf534640 1 Processor -- start 2024-04-25T10:32:40.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.199+0000 7f52bf534640 1 -- start start 2024-04-25T10:32:40.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.199+0000 7f52bf534640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b8102860 0x7f52b81a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.199+0000 7f52bf534640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52b810daa0 0x7f52b81a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.199+0000 7f52bf534640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52b8110400 0x7f52b819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.199+0000 7f52bf534640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8115ec0 con 0x7f52b8110400 2024-04-25T10:32:40.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.199+0000 7f52bf534640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8115d40 con 0x7f52b8102860 2024-04-25T10:32:40.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.199+0000 7f52bf534640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8116040 con 0x7f52b810daa0 2024-04-25T10:32:40.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.199+0000 7f52bdad1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52b8110400 0x7f52b819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.199+0000 7f52bdad1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52b8110400 0x7f52b819c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50628/0 (socket says 172.21.15.43:50628) 2024-04-25T10:32:40.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.199+0000 7f52bdad1640 1 -- 172.21.15.43:0/3078372155 learned_addr learned my addr 172.21.15.43:0/3078372155 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:40.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.199+0000 7f52bcacf640 1 --2- 172.21.15.43:0/3078372155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52b810daa0 0x7f52b81a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.199+0000 7f52bd2d0640 1 --2- 172.21.15.43:0/3078372155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b8102860 0x7f52b81a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.199+0000 7f52bdad1640 1 -- 172.21.15.43:0/3078372155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52b810daa0 msgr2=0x7f52b81a2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.199+0000 7f52bdad1640 1 --2- 172.21.15.43:0/3078372155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52b810daa0 0x7f52b81a2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.199+0000 7f52bdad1640 1 -- 172.21.15.43:0/3078372155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b8102860 msgr2=0x7f52b81a25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.199+0000 7f52bdad1640 1 --2- 172.21.15.43:0/3078372155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b8102860 0x7f52b81a25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.199+0000 7f52bdad1640 1 -- 172.21.15.43:0/3078372155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b819ce70 con 0x7f52b8110400 2024-04-25T10:32:40.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.199+0000 7f52bd2d0640 1 --2- 172.21.15.43:0/3078372155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b8102860 0x7f52b81a25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:40.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.199+0000 7f52bcacf640 1 --2- 172.21.15.43:0/3078372155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52b810daa0 0x7f52b81a2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:40.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.199+0000 7f52bdad1640 1 --2- 172.21.15.43:0/3078372155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52b8110400 0x7f52b819c740 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f52b0005cc0 tx=0x7f52b0031a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:40.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.200+0000 7f52ae7fc640 1 -- 172.21.15.43:0/3078372155 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0003b70 con 0x7f52b8110400 2024-04-25T10:32:40.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.200+0000 7f52bf534640 1 -- 172.21.15.43:0/3078372155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b819d0a0 con 0x7f52b8110400 2024-04-25T10:32:40.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.200+0000 7f52ae7fc640 1 -- 172.21.15.43:0/3078372155 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b0003d10 con 0x7f52b8110400 2024-04-25T10:32:40.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.200+0000 7f52ae7fc640 1 -- 172.21.15.43:0/3078372155 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b002fa50 con 0x7f52b8110400 2024-04-25T10:32:40.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.200+0000 7f52bf534640 1 -- 172.21.15.43:0/3078372155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b81a9360 con 0x7f52b8110400 2024-04-25T10:32:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.203+0000 7f52c63e5740 1 Processor -- start 2024-04-25T10:32:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.203+0000 7f52ae7fc640 1 -- 172.21.15.43:0/3078372155 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 147747+0+0 (secure 0 0 0) 0x7f52b002fbf0 con 0x7f52b8110400 2024-04-25T10:32:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.203+0000 7f52c63e5740 1 -- start start 2024-04-25T10:32:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.203+0000 7f52c63e5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52be0f390 0x55d52c1afa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.203+0000 7f52c63e5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52bde64d0 0x55d52c1affa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.203+0000 7f52c63e5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52bdee6b0 0x55d52c1bc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.203+0000 7f52c63e5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d52c115d80 con 0x55d52bdee6b0 2024-04-25T10:32:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.203+0000 7f52c63e5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d52c0cf8b0 con 0x55d52be0f390 2024-04-25T10:32:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.203+0000 7f52c63e5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d52c111d40 con 0x55d52bde64d0 2024-04-25T10:32:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.204+0000 7f52bcacf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52bdee6b0 0x55d52c1bc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.204+0000 7f52bcacf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52bdee6b0 0x55d52c1bc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50636/0 (socket says 172.21.15.43:50636) 2024-04-25T10:32:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.204+0000 7f52bcacf640 1 -- 172.21.15.43:0/2722877948 learned_addr learned my addr 172.21.15.43:0/2722877948 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.204+0000 7f52bd2d0640 1 --2- 172.21.15.43:0/2722877948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52bde64d0 0x55d52c1affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.204+0000 7f52bcacf640 1 --2- 172.21.15.43:0/2722877948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52be0f390 0x55d52c1afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.204+0000 7f52ae7fc640 1 --2- 172.21.15.43:0/3078372155 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f52a40afd80 0x7f52a40b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.204+0000 7f52bd2d0640 1 -- 172.21.15.43:0/2722877948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52be0f390 msgr2=0x55d52c1afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.204+0000 7f52bd2d0640 1 --2- 172.21.15.43:0/2722877948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52be0f390 0x55d52c1afa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.204+0000 7f52bd2d0640 1 -- 172.21.15.43:0/2722877948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52bdee6b0 msgr2=0x55d52c1bc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.204+0000 7f52bd2d0640 1 --2- 172.21.15.43:0/2722877948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52bdee6b0 0x55d52c1bc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.204+0000 7f52bd2d0640 1 -- 172.21.15.43:0/2722877948 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d52c0a0a40 con 0x55d52bde64d0 2024-04-25T10:32:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.204+0000 7f52bdad1640 1 --2- 172.21.15.43:0/3078372155 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f52a40afd80 0x7f52a40b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.204+0000 7f52bcacf640 1 --2- 172.21.15.43:0/2722877948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52bdee6b0 0x55d52c1bc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.204+0000 7f52ae7fc640 1 -- 172.21.15.43:0/3078372155 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f52b0100660 con 0x7f52b8110400 2024-04-25T10:32:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.204+0000 7f52bd2d0640 1 --2- 172.21.15.43:0/2722877948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52bde64d0 0x55d52c1affa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009c30 tx=0x7f52a80117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c6df64efcd60bd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.204+0000 7f52a37fe640 1 -- 172.21.15.43:0/2722877948 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8005020 con 0x55d52bde64d0 2024-04-25T10:32:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.205+0000 7f52bdad1640 1 --2- 172.21.15.43:0/3078372155 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f52a40afd80 0x7f52a40b2290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f52b00fddd0 tx=0x7f52b0097040 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.205+0000 7f52a37fe640 1 -- 172.21.15.43:0/2722877948 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a80047a0 con 0x55d52bde64d0 2024-04-25T10:32:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.205+0000 7f52a37fe640 1 -- 172.21.15.43:0/2722877948 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a80126b0 con 0x55d52bde64d0 2024-04-25T10:32:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.205+0000 7f52c63e5740 1 -- 172.21.15.43:0/2722877948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52bde64d0 msgr2=0x55d52c1affa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.205+0000 7f52c63e5740 1 --2- 172.21.15.43:0/2722877948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52bde64d0 0x55d52c1affa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009c30 tx=0x7f52a80117b0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.205+0000 7f52c63e5740 1 -- 172.21.15.43:0/2722877948 shutdown_connections 2024-04-25T10:32:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.205+0000 7f52c63e5740 1 --2- 172.21.15.43:0/2722877948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52bdee6b0 0x55d52c1bc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.205+0000 7f52c63e5740 1 --2- 172.21.15.43:0/2722877948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52bde64d0 0x55d52c1affa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.205+0000 7f52c63e5740 1 --2- 172.21.15.43:0/2722877948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52be0f390 0x55d52c1afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.205+0000 7f52c63e5740 1 -- 172.21.15.43:0/2722877948 >> 172.21.15.43:0/2722877948 conn(0x55d52be034e0 msgr2=0x55d52c1ac210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.206+0000 7f52c63e5740 1 -- 172.21.15.43:0/2722877948 shutdown_connections 2024-04-25T10:32:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.206+0000 7f52c63e5740 1 -- 172.21.15.43:0/2722877948 wait complete. 2024-04-25T10:32:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.206+0000 7f52c63e5740 1 -- start start 2024-04-25T10:32:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.206+0000 7f52c63e5740 1 Processor -- start 2024-04-25T10:32:40.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.206+0000 7f52c63e5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52bde64d0 0x55d52c1ba640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.206+0000 7f52c63e5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52bdee6b0 0x55d52c1bab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.206+0000 7f52c63e5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52be0f390 0x55d52c1c04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.206+0000 7f52c63e5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d52c1bcfe0 con 0x55d52bdee6b0 2024-04-25T10:32:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.206+0000 7f52c63e5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d52c09cd10 con 0x55d52bde64d0 2024-04-25T10:32:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.206+0000 7f52c63e5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d52c1bd160 con 0x55d52be0f390 2024-04-25T10:32:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.207+0000 7f52c63e5740 10 client.0 _ll_register_callbacks cb 0x55d52c0c9210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.207+0000 7f52bd2d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52bdee6b0 0x55d52c1bab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.207+0000 7f52bd2d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52bdee6b0 0x55d52c1bab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50640/0 (socket says 172.21.15.43:50640) 2024-04-25T10:32:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.207+0000 7f52bd2d0640 1 -- 172.21.15.43:0/1899733583 learned_addr learned my addr 172.21.15.43:0/1899733583 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.207+0000 7f52bcacf640 1 --2- 172.21.15.43:0/1899733583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52bde64d0 0x55d52c1ba640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.207+0000 7f52bcacf640 1 --2- 172.21.15.43:0/1899733583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52be0f390 0x55d52c1c04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.207+0000 7f52bd2d0640 1 -- 172.21.15.43:0/1899733583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52be0f390 msgr2=0x55d52c1c04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.207+0000 7f52bd2d0640 1 --2- 172.21.15.43:0/1899733583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52be0f390 0x55d52c1c04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.207+0000 7f52bd2d0640 1 -- 172.21.15.43:0/1899733583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52bde64d0 msgr2=0x55d52c1ba640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.207+0000 7f52bd2d0640 1 --2- 172.21.15.43:0/1899733583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52bde64d0 0x55d52c1ba640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.207+0000 7f52bd2d0640 1 -- 172.21.15.43:0/1899733583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d52bef17d0 con 0x55d52bdee6b0 2024-04-25T10:32:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.207+0000 7f52bcacf640 1 --2- 172.21.15.43:0/1899733583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52bde64d0 0x55d52c1ba640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:40.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.207+0000 7f52bd2d0640 1 --2- 172.21.15.43:0/1899733583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52bdee6b0 0x55d52c1bab80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009c00 tx=0x7f52a8004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7da1fd6e35b96a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:40.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.207+0000 7f52a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:32:40.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.207+0000 7f52a0ff9640 1 -- 172.21.15.43:0/1899733583 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8028020 con 0x55d52bdee6b0 2024-04-25T10:32:40.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.207+0000 7f52a0ff9640 1 -- 172.21.15.43:0/1899733583 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f52a80050f0 con 0x55d52bdee6b0 2024-04-25T10:32:40.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.207+0000 7f52c63e5740 1 -- 172.21.15.43:0/1899733583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a80047a0 con 0x55d52bdee6b0 2024-04-25T10:32:40.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.207+0000 7f52a0ff9640 1 -- 172.21.15.43:0/1899733583 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f52a8033050 con 0x55d52bdee6b0 2024-04-25T10:32:40.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.207+0000 7f52c63e5740 10 client.15504 fetch_fsmap learned FSMap version 24 2024-04-25T10:32:40.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.207+0000 7f52c63e5740 1 -- 172.21.15.43:0/1899733583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d52be273f0 con 0x55d52bdee6b0 2024-04-25T10:32:40.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.208+0000 7f52a0ff9640 1 -- 172.21.15.43:0/1899733583 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a80047a0 con 0x55d52bdee6b0 2024-04-25T10:32:40.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.208+0000 7f52a0ff9640 1 -- 172.21.15.43:0/1899733583 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f52a8024c40 con 0x55d52bdee6b0 2024-04-25T10:32:40.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.208+0000 7f52c63e5740 10 client.15504 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:32:40.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.208+0000 7f52c63e5740 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:32:40.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.208+0000 7f52c63e5740 20 client.15504 populate_metadata read hostname 'smithi043' 2024-04-25T10:32:40.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.208+0000 7f52c63e5740 1 --2- 172.21.15.43:0/1899733583 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d52bdcdf60 0x55d52c1cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.208+0000 7f52c63e5740 4 client.15504 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:32:40.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.208+0000 7f52c63e5740 1 -- 172.21.15.43:0/1899733583 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d52c123990 con 0x55d52bdcdf60 2024-04-25T10:32:40.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.208+0000 7f52bcacf640 1 --2- 172.21.15.43:0/1899733583 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d52bdcdf60 0x55d52c1cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.209+0000 7f52a0ff9640 10 client.15504 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:32:40.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.209+0000 7f52bcacf640 1 --2- 172.21.15.43:0/1899733583 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d52bdcdf60 0x55d52c1cc5c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1eaeccfa519662b server_cookie=62b87f8bf29aee54 in_seq=0 out_seq=0 2024-04-25T10:32:40.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.210+0000 7f52a0ff9640 1 -- 172.21.15.43:0/1899733583 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d52c123990 con 0x55d52bdcdf60 2024-04-25T10:32:40.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.210+0000 7f52a0ff9640 10 client.15504 handle_command_reply: tid=0 2024-04-25T10:32:40.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.210+0000 7f52c63e5740 1 client.15504 shutdown 2024-04-25T10:32:40.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.210+0000 7f52c63e5740 1 -- 172.21.15.43:0/1899733583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52bdee6b0 msgr2=0x55d52c1bab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.210+0000 7f52c63e5740 1 --2- 172.21.15.43:0/1899733583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52bdee6b0 0x55d52c1bab80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009c00 tx=0x7f52a8004e20 comp rx=0 tx=0).stop 2024-04-25T10:32:40.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.210+0000 7f52c63e5740 1 -- 172.21.15.43:0/1899733583 shutdown_connections 2024-04-25T10:32:40.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.210+0000 7f52c63e5740 1 --2- 172.21.15.43:0/1899733583 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d52bdcdf60 0x55d52c1cc5c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.210+0000 7f52c63e5740 1 --2- 172.21.15.43:0/1899733583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52be0f390 0x55d52c1c04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.210+0000 7f52c63e5740 1 --2- 172.21.15.43:0/1899733583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52bdee6b0 0x55d52c1bab80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.210+0000 7f52a0ff9640 0 client.15504 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:32:40.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.210+0000 7f52c63e5740 1 --2- 172.21.15.43:0/1899733583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52bde64d0 0x55d52c1ba640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.210+0000 7f52c63e5740 1 -- 172.21.15.43:0/1899733583 >> 172.21.15.43:0/1899733583 conn(0x55d52be034e0 msgr2=0x55d52c1b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:40.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.210+0000 7f52c63e5740 1 -- 172.21.15.43:0/1899733583 shutdown_connections 2024-04-25T10:32:40.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.210+0000 7f52c63e5740 1 -- 172.21.15.43:0/1899733583 wait complete. 2024-04-25T10:32:40.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.210+0000 7f52c63e5740 20 client.15504 trim_cache size 0 max 16384 2024-04-25T10:32:40.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.228+0000 7f52c63e5740 1 Processor -- start 2024-04-25T10:32:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.228+0000 7f52c63e5740 1 -- start start 2024-04-25T10:32:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.229+0000 7f52c63e5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52bde64d0 0x55d52c1ac280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.229+0000 7f52c63e5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52bdee6b0 0x55d52c1bc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.229+0000 7f52c63e5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52be0f390 0x55d52c1c0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.229+0000 7f52c63e5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81b4380 con 0x55d52be0f390 2024-04-25T10:32:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.229+0000 7f52c63e5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81b1c50 con 0x55d52bdee6b0 2024-04-25T10:32:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.229+0000 7f52c63e5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81afaa0 con 0x55d52bde64d0 2024-04-25T10:32:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.229+0000 7f52bcacf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52be0f390 0x55d52c1c0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.229+0000 7f52bcacf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52be0f390 0x55d52c1c0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50642/0 (socket says 172.21.15.43:50642) 2024-04-25T10:32:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.229+0000 7f52bcacf640 1 -- 172.21.15.43:0/3797819276 learned_addr learned my addr 172.21.15.43:0/3797819276 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.229+0000 7f52bcacf640 1 --2- 172.21.15.43:0/3797819276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52bde64d0 0x55d52c1ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.229+0000 7f52bd2d0640 1 --2- 172.21.15.43:0/3797819276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52bdee6b0 0x55d52c1bc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.229+0000 7f52bcacf640 1 -- 172.21.15.43:0/3797819276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52bde64d0 msgr2=0x55d52c1ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:40.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.229+0000 7f52bcacf640 1 --2- 172.21.15.43:0/3797819276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52bde64d0 0x55d52c1ac280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.229+0000 7f52bcacf640 1 -- 172.21.15.43:0/3797819276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52bdee6b0 msgr2=0x55d52c1bc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:40.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.229+0000 7f52bcacf640 1 --2- 172.21.15.43:0/3797819276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52bdee6b0 0x55d52c1bc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.229+0000 7f52bcacf640 1 -- 172.21.15.43:0/3797819276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d52bde70f0 con 0x55d52be0f390 2024-04-25T10:32:40.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.229+0000 7f52bd2d0640 1 --2- 172.21.15.43:0/3797819276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52bdee6b0 0x55d52c1bc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:40.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.229+0000 7f52bcacf640 1 --2- 172.21.15.43:0/3797819276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52be0f390 0x55d52c1c0420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f52b4000c00 tx=0x7f52b4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31f0b2089ae9400 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:40.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.230+0000 7f52a2ffd640 1 -- 172.21.15.43:0/3797819276 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b40037f0 con 0x55d52be0f390 2024-04-25T10:32:40.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.230+0000 7f52a2ffd640 1 -- 172.21.15.43:0/3797819276 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b4003e40 con 0x55d52be0f390 2024-04-25T10:32:40.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.230+0000 7f52a2ffd640 1 -- 172.21.15.43:0/3797819276 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4011440 con 0x55d52be0f390 2024-04-25T10:32:40.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.230+0000 7f52c63e5740 1 -- 172.21.15.43:0/3797819276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52be0f390 msgr2=0x55d52c1c0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.230+0000 7f52c63e5740 1 --2- 172.21.15.43:0/3797819276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52be0f390 0x55d52c1c0420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f52b4000c00 tx=0x7f52b4005f90 comp rx=0 tx=0).stop 2024-04-25T10:32:40.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.231+0000 7f52c63e5740 1 -- 172.21.15.43:0/3797819276 shutdown_connections 2024-04-25T10:32:40.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.231+0000 7f52c63e5740 1 --2- 172.21.15.43:0/3797819276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52bdee6b0 0x55d52c1bc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.231+0000 7f52c63e5740 1 --2- 172.21.15.43:0/3797819276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52be0f390 0x55d52c1c0420 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.231+0000 7f52c63e5740 1 --2- 172.21.15.43:0/3797819276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52bde64d0 0x55d52c1ac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.231+0000 7f52c63e5740 1 -- 172.21.15.43:0/3797819276 >> 172.21.15.43:0/3797819276 conn(0x55d52be034e0 msgr2=0x55d52c1bbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:40.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.231+0000 7f52c63e5740 1 -- 172.21.15.43:0/3797819276 shutdown_connections 2024-04-25T10:32:40.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.231+0000 7f52c63e5740 1 -- 172.21.15.43:0/3797819276 wait complete. 2024-04-25T10:32:40.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.231+0000 7f52c63e5740 1 -- start start 2024-04-25T10:32:40.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.231+0000 7f52c63e5740 1 Processor -- start 2024-04-25T10:32:40.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.231+0000 7f52c63e5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52bde64d0 0x55d52c1c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.231+0000 7f52c63e5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52bdee6b0 0x55d52c1c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.231+0000 7f52c63e5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52be0f390 0x55d52c1c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.231+0000 7f52c63e5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81b0110 con 0x55d52be0f390 2024-04-25T10:32:40.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.231+0000 7f52c63e5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81b1580 con 0x55d52bdee6b0 2024-04-25T10:32:40.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.231+0000 7f52c63e5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81b07e0 con 0x55d52bde64d0 2024-04-25T10:32:40.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.232+0000 7f52c63e5740 10 client.0 _ll_register_callbacks cb 0x7f52b40096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:40.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.232+0000 7f52bcacf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52be0f390 0x55d52c1c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.232+0000 7f52bd2d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52bdee6b0 0x55d52c1c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.232+0000 7f52bcacf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52bde64d0 0x55d52c1c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.232+0000 7f52bd2d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52bdee6b0 0x55d52c1c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:51598/0 (socket says 172.21.15.43:51598) 2024-04-25T10:32:40.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.232+0000 7f52bcacf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52bde64d0 0x55d52c1c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:41440/0 (socket says 172.21.15.43:41440) 2024-04-25T10:32:40.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.232+0000 7f52bd2d0640 1 -- 172.21.15.43:0/3416222715 learned_addr learned my addr 172.21.15.43:0/3416222715 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:40.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.232+0000 7f52bd2d0640 1 -- 172.21.15.43:0/3416222715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52bde64d0 msgr2=0x55d52c1c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:40.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.232+0000 7f52bd2d0640 1 --2- 172.21.15.43:0/3416222715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52bde64d0 0x55d52c1c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.232+0000 7f52bd2d0640 1 -- 172.21.15.43:0/3416222715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52be0f390 msgr2=0x55d52c1c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:40.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.232+0000 7f52bd2d0640 1 --2- 172.21.15.43:0/3416222715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52be0f390 0x55d52c1c3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.232+0000 7f52bd2d0640 1 -- 172.21.15.43:0/3416222715 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d52c0d64b0 con 0x55d52bdee6b0 2024-04-25T10:32:40.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.232+0000 7f52bd2d0640 1 --2- 172.21.15.43:0/3416222715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52bdee6b0 0x55d52c1c6d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f52a8004ca0 tx=0x7f52a8009e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c74a6a4eb0106b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:40.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.232+0000 7f52a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-04-25T10:32:40.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.232+0000 7f52a0ff9640 1 -- 172.21.15.43:0/3416222715 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a803c060 con 0x55d52bdee6b0 2024-04-25T10:32:40.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.232+0000 7f52c63e5740 1 -- 172.21.15.43:0/3416222715 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b4003e40 con 0x55d52bdee6b0 2024-04-25T10:32:40.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.232+0000 7f52a0ff9640 1 -- 172.21.15.43:0/3416222715 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f52a80050f0 con 0x55d52bdee6b0 2024-04-25T10:32:40.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.233+0000 7f52a0ff9640 1 -- 172.21.15.43:0/3416222715 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f52a8028070 con 0x55d52bdee6b0 2024-04-25T10:32:40.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.233+0000 7f52c63e5740 10 client.25256 fetch_fsmap learned FSMap version 24 2024-04-25T10:32:40.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.233+0000 7f52c63e5740 1 -- 172.21.15.43:0/3416222715 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d52c233de0 con 0x55d52bdee6b0 2024-04-25T10:32:40.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.233+0000 7f52a0ff9640 1 -- 172.21.15.43:0/3416222715 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b4003e40 con 0x55d52bdee6b0 2024-04-25T10:32:40.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.233+0000 7f52a0ff9640 1 -- 172.21.15.43:0/3416222715 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f52a8041510 con 0x55d52bdee6b0 2024-04-25T10:32:40.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.233+0000 7f52c63e5740 10 client.25256 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:32:40.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.233+0000 7f52c63e5740 10 client.25256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:32:40.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.233+0000 7f52c63e5740 20 client.25256 populate_metadata read hostname 'smithi043' 2024-04-25T10:32:40.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.233+0000 7f52c63e5740 1 --2- 172.21.15.43:0/3416222715 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d52bda33a0 0x55d52c236510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.233+0000 7f52c63e5740 4 client.25256 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:32:40.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.233+0000 7f52c63e5740 1 -- 172.21.15.43:0/3416222715 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d52c11d800 con 0x55d52bda33a0 2024-04-25T10:32:40.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.233+0000 7f52bcacf640 1 --2- 172.21.15.43:0/3416222715 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d52bda33a0 0x55d52c236510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.234+0000 7f52a0ff9640 10 client.25256 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:32:40.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.234+0000 7f52bcacf640 1 --2- 172.21.15.43:0/3416222715 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d52bda33a0 0x55d52c236510 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db6b28e6c2eb7955 server_cookie=1d31901dcd136245 in_seq=0 out_seq=0 2024-04-25T10:32:40.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.234+0000 7f52a0ff9640 1 -- 172.21.15.43:0/3416222715 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d52c11d800 con 0x55d52bda33a0 2024-04-25T10:32:40.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.234+0000 7f52a0ff9640 10 client.25256 handle_command_reply: tid=0 2024-04-25T10:32:40.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.234+0000 7f52c63e5740 1 client.25256 shutdown 2024-04-25T10:32:40.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.234+0000 7f52c63e5740 1 -- 172.21.15.43:0/3416222715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52bdee6b0 msgr2=0x55d52c1c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.234+0000 7f52c63e5740 1 --2- 172.21.15.43:0/3416222715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52bdee6b0 0x55d52c1c6d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f52a8004ca0 tx=0x7f52a8009e20 comp rx=0 tx=0).stop 2024-04-25T10:32:40.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.234+0000 7f52c63e5740 1 -- 172.21.15.43:0/3416222715 shutdown_connections 2024-04-25T10:32:40.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.234+0000 7f52c63e5740 1 --2- 172.21.15.43:0/3416222715 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d52bda33a0 0x55d52c236510 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.234+0000 7f52c63e5740 1 --2- 172.21.15.43:0/3416222715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52bdee6b0 0x55d52c1c6d80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.234+0000 7f52a0ff9640 0 client.25256 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:32:40.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.234+0000 7f52c63e5740 1 --2- 172.21.15.43:0/3416222715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52be0f390 0x55d52c1c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.234+0000 7f52c63e5740 1 --2- 172.21.15.43:0/3416222715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52bde64d0 0x55d52c1c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.234+0000 7f52c63e5740 1 -- 172.21.15.43:0/3416222715 >> 172.21.15.43:0/3416222715 conn(0x55d52be034e0 msgr2=0x55d52c1bba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:40.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.234+0000 7f52c63e5740 1 -- 172.21.15.43:0/3416222715 shutdown_connections 2024-04-25T10:32:40.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.234+0000 7f52c63e5740 1 -- 172.21.15.43:0/3416222715 wait complete. 2024-04-25T10:32:40.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.234+0000 7f52c63e5740 20 client.25256 trim_cache size 0 max 16384 2024-04-25T10:32:40.242 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:32:40.242 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:32:40.243 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:32:40.243 INFO:teuthology.orchestra.run.smithi043.stdout: "8c88fa0b-0bf8-4970-bab4-598574afe4fc": { 2024-04-25T10:32:40.243 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:32:40.243 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "8c88fa0b-0bf8-4970-bab4-598574afe4fc", 2024-04-25T10:32:40.243 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:32:40.243 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:32:40.243 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:32:40.243 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:32:40.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.235+0000 7f52bf534640 1 -- 172.21.15.43:0/3078372155 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f52a40afd80 msgr2=0x7f52a40b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.235+0000 7f52bf534640 1 --2- 172.21.15.43:0/3078372155 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f52a40afd80 0x7f52a40b2290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f52b00fddd0 tx=0x7f52b0097040 comp rx=0 tx=0).stop 2024-04-25T10:32:40.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.235+0000 7f52bf534640 1 -- 172.21.15.43:0/3078372155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52b8110400 msgr2=0x7f52b819c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.235+0000 7f52bf534640 1 --2- 172.21.15.43:0/3078372155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52b8110400 0x7f52b819c740 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f52b0005cc0 tx=0x7f52b0031a00 comp rx=0 tx=0).stop 2024-04-25T10:32:40.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.235+0000 7f52bf534640 1 -- 172.21.15.43:0/3078372155 shutdown_connections 2024-04-25T10:32:40.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.235+0000 7f52bf534640 1 --2- 172.21.15.43:0/3078372155 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f52a40afd80 0x7f52a40b2290 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.235+0000 7f52bf534640 1 --2- 172.21.15.43:0/3078372155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52b8110400 0x7f52b819c740 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.235+0000 7f52bf534640 1 --2- 172.21.15.43:0/3078372155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52b810daa0 0x7f52b81a2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.235+0000 7f52bf534640 1 --2- 172.21.15.43:0/3078372155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b8102860 0x7f52b81a25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.235+0000 7f52bf534640 1 -- 172.21.15.43:0/3078372155 >> 172.21.15.43:0/3078372155 conn(0x7f52b8078100 msgr2=0x7f52b8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:40.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.235+0000 7f52bf534640 1 -- 172.21.15.43:0/3078372155 shutdown_connections 2024-04-25T10:32:40.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.235+0000 7f52bf534640 1 -- 172.21.15.43:0/3078372155 wait complete. 2024-04-25T10:32:40.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.238+0000 7f515b2bd640 1 -- 172.21.15.43:0/1797544320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f51541069e0 con 0x7f51541031c0 2024-04-25T10:32:40.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.243+0000 7f51527fc640 1 -- 172.21.15.43:0/1797544320 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v35)=0 v35) ==== 129+0+0 (secure 0 0 0) 0x7f514c0c85c0 con 0x7f51541031c0 2024-04-25T10:32:40.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.245+0000 7f515b2bd640 1 -- 172.21.15.43:0/1797544320 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f513c0afdf0 msgr2=0x7f513c0b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.245+0000 7f515b2bd640 1 --2- 172.21.15.43:0/1797544320 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f513c0afdf0 0x7f513c0b2300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f515410e790 tx=0x7f5148005f90 comp rx=0 tx=0).stop 2024-04-25T10:32:40.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.245+0000 7f515b2bd640 1 -- 172.21.15.43:0/1797544320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51541031c0 msgr2=0x7f515419b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.245+0000 7f515b2bd640 1 --2- 172.21.15.43:0/1797544320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51541031c0 0x7f515419b5a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f514c0098d0 tx=0x7f514c0317c0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.245+0000 7f515b2bd640 1 -- 172.21.15.43:0/1797544320 shutdown_connections 2024-04-25T10:32:40.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.245+0000 7f515b2bd640 1 --2- 172.21.15.43:0/1797544320 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f513c0afdf0 0x7f513c0b2300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.245+0000 7f515b2bd640 1 --2- 172.21.15.43:0/1797544320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5154108a60 0x7f515419bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.245+0000 7f515b2bd640 1 --2- 172.21.15.43:0/1797544320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5154108f00 0x7f515419c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.245+0000 7f515b2bd640 1 --2- 172.21.15.43:0/1797544320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51541031c0 0x7f515419b5a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.245+0000 7f515b2bd640 1 -- 172.21.15.43:0/1797544320 >> 172.21.15.43:0/1797544320 conn(0x7f51540781b0 msgr2=0x7f5154104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:40.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.245+0000 7f515b2bd640 1 -- 172.21.15.43:0/1797544320 shutdown_connections 2024-04-25T10:32:40.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.245+0000 7f515b2bd640 1 -- 172.21.15.43:0/1797544320 wait complete. 2024-04-25T10:32:40.249 INFO:tasks.cephfs.filesystem:scrub status for tag:8c88fa0b-0bf8-4970-bab4-598574afe4fc - {'path': '/', 'tag': '8c88fa0b-0bf8-4970-bab4-598574afe4fc', 'options': 'recursive,force'} 2024-04-25T10:32:40.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T10:32:40.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:39 smithi077 ceph-mon[32772]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-25T10:32:40.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:39 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1206886127' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:32:40.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:39 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2642451307' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:32:40.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:39 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/4257268595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:32:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:39 smithi164 ceph-mon[32713]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-25T10:32:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:39 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1206886127' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:32:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:39 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2642451307' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:32:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:39 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/4257268595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:32:40.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.413+0000 7ff87cfba640 1 -- 172.21.15.43:0/2690869063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff878108a50 msgr2=0x7ff878102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.413+0000 7ff87cfba640 1 --2- 172.21.15.43:0/2690869063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff878108a50 0x7ff878102600 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff868009900 tx=0x7ff86802a410 comp rx=0 tx=0).stop 2024-04-25T10:32:40.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.413+0000 7ff87cfba640 1 -- 172.21.15.43:0/2690869063 shutdown_connections 2024-04-25T10:32:40.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.413+0000 7ff87cfba640 1 --2- 172.21.15.43:0/2690869063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8781031b0 0x7ff87810f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.413+0000 7ff87cfba640 1 --2- 172.21.15.43:0/2690869063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff878108ef0 0x7ff878102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.413+0000 7ff87cfba640 1 --2- 172.21.15.43:0/2690869063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff878108a50 0x7ff878102600 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.413+0000 7ff87cfba640 1 -- 172.21.15.43:0/2690869063 >> 172.21.15.43:0/2690869063 conn(0x7ff8780fe260 msgr2=0x7ff878100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:40.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.413+0000 7ff87cfba640 1 -- 172.21.15.43:0/2690869063 shutdown_connections 2024-04-25T10:32:40.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.413+0000 7ff87cfba640 1 -- 172.21.15.43:0/2690869063 wait complete. 2024-04-25T10:32:40.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.414+0000 7ff87cfba640 1 Processor -- start 2024-04-25T10:32:40.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.414+0000 7ff87cfba640 1 -- start start 2024-04-25T10:32:40.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.415+0000 7ff87cfba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8781031b0 0x7ff87819b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.415+0000 7ff87cfba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff878108a50 0x7ff87819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.415+0000 7ff87cfba640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff878108ef0 0x7ff87819c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.415+0000 7ff87cfba640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff878112c50 con 0x7ff8781031b0 2024-04-25T10:32:40.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.415+0000 7ff87cfba640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff878112ad0 con 0x7ff878108a50 2024-04-25T10:32:40.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.415+0000 7ff87cfba640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff878112dd0 con 0x7ff878108ef0 2024-04-25T10:32:40.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.415+0000 7ff87659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8781031b0 0x7ff87819b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.415+0000 7ff87659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8781031b0 0x7ff87819b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50672/0 (socket says 172.21.15.43:50672) 2024-04-25T10:32:40.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.415+0000 7ff87659c640 1 -- 172.21.15.43:0/522003913 learned_addr learned my addr 172.21.15.43:0/522003913 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:40.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.415+0000 7ff86dd9b640 1 --2- 172.21.15.43:0/522003913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff878108a50 0x7ff87819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.415+0000 7ff876d9d640 1 --2- 172.21.15.43:0/522003913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff878108ef0 0x7ff87819c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.415+0000 7ff87659c640 1 -- 172.21.15.43:0/522003913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff878108ef0 msgr2=0x7ff87819c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.415+0000 7ff87659c640 1 --2- 172.21.15.43:0/522003913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff878108ef0 0x7ff87819c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.415+0000 7ff87659c640 1 -- 172.21.15.43:0/522003913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff878108a50 msgr2=0x7ff87819baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.415+0000 7ff87659c640 1 --2- 172.21.15.43:0/522003913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff878108a50 0x7ff87819baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.415+0000 7ff87659c640 1 -- 172.21.15.43:0/522003913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8781a68f0 con 0x7ff8781031b0 2024-04-25T10:32:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.415+0000 7ff87659c640 1 --2- 172.21.15.43:0/522003913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8781031b0 0x7ff87819b5b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff868006010 tx=0x7ff868004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.415+0000 7ff86ffff640 1 -- 172.21.15.43:0/522003913 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86803f030 con 0x7ff8781031b0 2024-04-25T10:32:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.416+0000 7ff86ffff640 1 -- 172.21.15.43:0/522003913 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff868003aa0 con 0x7ff8781031b0 2024-04-25T10:32:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.416+0000 7ff87cfba640 1 -- 172.21.15.43:0/522003913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8781a6b80 con 0x7ff8781031b0 2024-04-25T10:32:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.416+0000 7ff86ffff640 1 -- 172.21.15.43:0/522003913 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff868035070 con 0x7ff8781031b0 2024-04-25T10:32:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.416+0000 7ff87cfba640 1 -- 172.21.15.43:0/522003913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8781a7070 con 0x7ff8781031b0 2024-04-25T10:32:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.417+0000 7ff87cfba640 1 -- 172.21.15.43:0/522003913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff878050740 con 0x7ff8781031b0 2024-04-25T10:32:40.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.419+0000 7ff86ffff640 1 -- 172.21.15.43:0/522003913 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 147747+0+0 (secure 0 0 0) 0x7ff868042030 con 0x7ff8781031b0 2024-04-25T10:32:40.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.420+0000 7ff86ffff640 1 --2- 172.21.15.43:0/522003913 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff8640afd70 0x7ff8640b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.420+0000 7ff86dd9b640 1 --2- 172.21.15.43:0/522003913 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff8640afd70 0x7ff8640b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.420+0000 7ff86ffff640 1 -- 172.21.15.43:0/522003913 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7ff8680ffa50 con 0x7ff8781031b0 2024-04-25T10:32:40.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.420+0000 7ff86dd9b640 1 --2- 172.21.15.43:0/522003913 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff8640afd70 0x7ff8640b2280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff8781010b0 tx=0x7ff86000b410 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:40.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.425+0000 7ff86ffff640 1 -- 172.21.15.43:0/522003913 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff8680c7af0 con 0x7ff8781031b0 2024-04-25T10:32:40.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.622+0000 7ff87cfba640 1 -- 172.21.15.43:0/522003913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7ff8781a72e0 con 0x7ff8781031b0 2024-04-25T10:32:40.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.627+0000 7ff86ffff640 1 -- 172.21.15.43:0/522003913 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v36)=0 v36) ==== 138+0+0 (secure 0 0 0) 0x7ff8680cbdf0 con 0x7ff8781031b0 2024-04-25T10:32:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.629+0000 7ff87cfba640 1 -- 172.21.15.43:0/522003913 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff8640afd70 msgr2=0x7ff8640b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.629+0000 7ff87cfba640 1 --2- 172.21.15.43:0/522003913 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff8640afd70 0x7ff8640b2280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff8781010b0 tx=0x7ff86000b410 comp rx=0 tx=0).stop 2024-04-25T10:32:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.629+0000 7ff87cfba640 1 -- 172.21.15.43:0/522003913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8781031b0 msgr2=0x7ff87819b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.629+0000 7ff87cfba640 1 --2- 172.21.15.43:0/522003913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8781031b0 0x7ff87819b5b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff868006010 tx=0x7ff868004060 comp rx=0 tx=0).stop 2024-04-25T10:32:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.629+0000 7ff87cfba640 1 -- 172.21.15.43:0/522003913 shutdown_connections 2024-04-25T10:32:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.629+0000 7ff87cfba640 1 --2- 172.21.15.43:0/522003913 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff8640afd70 0x7ff8640b2280 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.629+0000 7ff87cfba640 1 --2- 172.21.15.43:0/522003913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff878108a50 0x7ff87819baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.629+0000 7ff87cfba640 1 --2- 172.21.15.43:0/522003913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff878108ef0 0x7ff87819c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.629+0000 7ff87cfba640 1 --2- 172.21.15.43:0/522003913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8781031b0 0x7ff87819b5b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.629+0000 7ff87cfba640 1 -- 172.21.15.43:0/522003913 >> 172.21.15.43:0/522003913 conn(0x7ff8780fe260 msgr2=0x7ff8780fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.629+0000 7ff87cfba640 1 -- 172.21.15.43:0/522003913 shutdown_connections 2024-04-25T10:32:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.629+0000 7ff87cfba640 1 -- 172.21.15.43:0/522003913 wait complete. 2024-04-25T10:32:40.643 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T10:32:40.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.756+0000 7f2a1ce49640 1 -- 172.21.15.43:0/3758859288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a18104d20 msgr2=0x7f2a18105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.756+0000 7f2a1ce49640 1 --2- 172.21.15.43:0/3758859288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a18104d20 0x7f2a18105130 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2a08009900 tx=0x7f2a0802a480 comp rx=0 tx=0).stop 2024-04-25T10:32:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.756+0000 7f2a1ce49640 1 -- 172.21.15.43:0/3758859288 shutdown_connections 2024-04-25T10:32:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.756+0000 7f2a1ce49640 1 --2- 172.21.15.43:0/3758859288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a1810b780 0x7f2a1810dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.756+0000 7f2a1ce49640 1 --2- 172.21.15.43:0/3758859288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a1810e0e0 0x7f2a181105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.756+0000 7f2a1ce49640 1 --2- 172.21.15.43:0/3758859288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a18104d20 0x7f2a18105130 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.756+0000 7f2a1ce49640 1 -- 172.21.15.43:0/3758859288 >> 172.21.15.43:0/3758859288 conn(0x7f2a180fe260 msgr2=0x7f2a18100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.756+0000 7f2a1ce49640 1 -- 172.21.15.43:0/3758859288 shutdown_connections 2024-04-25T10:32:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.756+0000 7f2a1ce49640 1 -- 172.21.15.43:0/3758859288 wait complete. 2024-04-25T10:32:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.757+0000 7f2a1ce49640 1 Processor -- start 2024-04-25T10:32:40.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.757+0000 7f2a1ce49640 1 -- start start 2024-04-25T10:32:40.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.757+0000 7f2a1ce49640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a18104d20 0x7f2a1819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.757+0000 7f2a1ce49640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a1810b780 0x7f2a1819acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.758+0000 7f2a1ce49640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a1810e0e0 0x7f2a1819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.758+0000 7f2a1ce49640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a18113d70 con 0x7f2a18104d20 2024-04-25T10:32:40.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.758+0000 7f2a1ce49640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2a18113bf0 con 0x7f2a1810e0e0 2024-04-25T10:32:40.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.758+0000 7f2a1ce49640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2a18113ef0 con 0x7f2a1810b780 2024-04-25T10:32:40.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.758+0000 7f2a1659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a18104d20 0x7f2a1819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.758+0000 7f2a1659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a18104d20 0x7f2a1819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50676/0 (socket says 172.21.15.43:50676) 2024-04-25T10:32:40.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.758+0000 7f2a1659c640 1 -- 172.21.15.43:0/1338524168 learned_addr learned my addr 172.21.15.43:0/1338524168 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:40.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.758+0000 7f2a0ffff640 1 --2- 172.21.15.43:0/1338524168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a1810b780 0x7f2a1819acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.758+0000 7f2a16d9d640 1 --2- 172.21.15.43:0/1338524168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a1810e0e0 0x7f2a1819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.758+0000 7f2a1659c640 1 -- 172.21.15.43:0/1338524168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a1810b780 msgr2=0x7f2a1819acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.758+0000 7f2a1659c640 1 --2- 172.21.15.43:0/1338524168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a1810b780 0x7f2a1819acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.758+0000 7f2a1659c640 1 -- 172.21.15.43:0/1338524168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a1810e0e0 msgr2=0x7f2a1819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:40.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.758+0000 7f2a1659c640 1 --2- 172.21.15.43:0/1338524168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a1810e0e0 0x7f2a1819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:40.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.758+0000 7f2a1659c640 1 -- 172.21.15.43:0/1338524168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1819f550 con 0x7f2a18104d20 2024-04-25T10:32:40.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.758+0000 7f2a16d9d640 1 --2- 172.21.15.43:0/1338524168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a1810e0e0 0x7f2a1819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:40.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.758+0000 7f2a1659c640 1 --2- 172.21.15.43:0/1338524168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a18104d20 0x7f2a1819a770 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2a0802aa70 tx=0x7f2a08003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:40.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.758+0000 7f2a0f7fe640 1 -- 172.21.15.43:0/1338524168 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0803f070 con 0x7f2a18104d20 2024-04-25T10:32:40.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.758+0000 7f2a1ce49640 1 -- 172.21.15.43:0/1338524168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1819f7e0 con 0x7f2a18104d20 2024-04-25T10:32:40.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.758+0000 7f2a0f7fe640 1 -- 172.21.15.43:0/1338524168 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a08003ec0 con 0x7f2a18104d20 2024-04-25T10:32:40.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.758+0000 7f2a0f7fe640 1 -- 172.21.15.43:0/1338524168 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08035030 con 0x7f2a18104d20 2024-04-25T10:32:40.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.759+0000 7f2a1ce49640 1 -- 172.21.15.43:0/1338524168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a1819fcd0 con 0x7f2a18104d20 2024-04-25T10:32:40.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.759+0000 7f2a1ce49640 1 -- 172.21.15.43:0/1338524168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a18050740 con 0x7f2a18104d20 2024-04-25T10:32:40.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.761+0000 7f2a0f7fe640 1 -- 172.21.15.43:0/1338524168 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 147747+0+0 (secure 0 0 0) 0x7f2a08042070 con 0x7f2a18104d20 2024-04-25T10:32:40.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.762+0000 7f2a0f7fe640 1 --2- 172.21.15.43:0/1338524168 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2a040afd70 0x7f2a040b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:40.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.762+0000 7f2a0ffff640 1 --2- 172.21.15.43:0/1338524168 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2a040afd70 0x7f2a040b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:40.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.762+0000 7f2a0f7fe640 1 -- 172.21.15.43:0/1338524168 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f2a08100100 con 0x7f2a18104d20 2024-04-25T10:32:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.762+0000 7f2a0ffff640 1 --2- 172.21.15.43:0/1338524168 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2a040afd70 0x7f2a040b2280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2a1810ff50 tx=0x7f2a0000b410 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:40.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.766+0000 7f2a0f7fe640 1 -- 172.21.15.43:0/1338524168 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a080c8850 con 0x7f2a18104d20 2024-04-25T10:32:40.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:40.983+0000 7f2a1ce49640 1 -- 172.21.15.43:0/1338524168 --> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f2a1810ecb0 con 0x7f2a040afd70 2024-04-25T10:32:41.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.017+0000 7f2a0f7fe640 1 -- 172.21.15.43:0/1338524168 <== mgr.15291 v2:172.21.15.43:6800/3658924116 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2a1810ecb0 con 0x7f2a040afd70 2024-04-25T10:32:41.019 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path / 2024-04-25T10:32:41.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.019+0000 7f2a1ce49640 1 -- 172.21.15.43:0/1338524168 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2a040afd70 msgr2=0x7f2a040b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:41.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.019+0000 7f2a1ce49640 1 --2- 172.21.15.43:0/1338524168 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2a040afd70 0x7f2a040b2280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2a1810ff50 tx=0x7f2a0000b410 comp rx=0 tx=0).stop 2024-04-25T10:32:41.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.019+0000 7f2a1ce49640 1 -- 172.21.15.43:0/1338524168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a18104d20 msgr2=0x7f2a1819a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:41.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.019+0000 7f2a1ce49640 1 --2- 172.21.15.43:0/1338524168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a18104d20 0x7f2a1819a770 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2a0802aa70 tx=0x7f2a08003cd0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.019+0000 7f2a1ce49640 1 -- 172.21.15.43:0/1338524168 shutdown_connections 2024-04-25T10:32:41.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.019+0000 7f2a1ce49640 1 --2- 172.21.15.43:0/1338524168 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2a040afd70 0x7f2a040b2280 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.019+0000 7f2a1ce49640 1 --2- 172.21.15.43:0/1338524168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a1810e0e0 0x7f2a1819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.019+0000 7f2a1ce49640 1 --2- 172.21.15.43:0/1338524168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a1810b780 0x7f2a1819acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.019+0000 7f2a1ce49640 1 --2- 172.21.15.43:0/1338524168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a18104d20 0x7f2a1819a770 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.019+0000 7f2a1ce49640 1 -- 172.21.15.43:0/1338524168 >> 172.21.15.43:0/1338524168 conn(0x7f2a180fe260 msgr2=0x7f2a1810e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:41.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.019+0000 7f2a1ce49640 1 -- 172.21.15.43:0/1338524168 shutdown_connections 2024-04-25T10:32:41.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.019+0000 7f2a1ce49640 1 -- 172.21.15.43:0/1338524168 wait complete. 2024-04-25T10:32:41.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:40 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1413627031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:41.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:40 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:32:41.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:40 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:32:41.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:40 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:32:41.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:40 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:32:41.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:40 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1797544320' entity='client.admin' 2024-04-25T10:32:41.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:40 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:32:41.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:40 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:32:41.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:40 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:32:41.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:40 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:41.035 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:40 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/522003913' entity='client.admin' 2024-04-25T10:32:41.035 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:40 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:32:41.035 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:40 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:32:41.035 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:40 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:32:41.035 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:40 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:41.036 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T10:32:41.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.149+0000 7f9250205640 1 -- 172.21.15.43:0/3418340898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9248102820 msgr2=0x7f92481106a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:41.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.149+0000 7f9250205640 1 --2- 172.21.15.43:0/3418340898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9248102820 0x7f92481106a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9240009ca0 tx=0x7f924002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.149+0000 7f9250205640 1 -- 172.21.15.43:0/3418340898 shutdown_connections 2024-04-25T10:32:41.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.149+0000 7f9250205640 1 --2- 172.21.15.43:0/3418340898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9248102820 0x7f92481106a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.149+0000 7f9250205640 1 --2- 172.21.15.43:0/3418340898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f924810a020 0x7f9248102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.149+0000 7f9250205640 1 --2- 172.21.15.43:0/3418340898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9248109b80 0x7f9248101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.149+0000 7f9250205640 1 -- 172.21.15.43:0/3418340898 >> 172.21.15.43:0/3418340898 conn(0x7f92480fd8a0 msgr2=0x7f92480ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:41.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.149+0000 7f9250205640 1 -- 172.21.15.43:0/3418340898 shutdown_connections 2024-04-25T10:32:41.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.149+0000 7f9250205640 1 -- 172.21.15.43:0/3418340898 wait complete. 2024-04-25T10:32:41.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.149+0000 7f9250205640 1 Processor -- start 2024-04-25T10:32:41.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.149+0000 7f9250205640 1 -- start start 2024-04-25T10:32:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.150+0000 7f9250205640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9248109b80 0x7f9248199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.150+0000 7f9250205640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924819a490 0x7f924819e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.150+0000 7f9250205640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f924819ecb0 0x7f924819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.150+0000 7f9250205640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9248113d90 con 0x7f924819a490 2024-04-25T10:32:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.150+0000 7f9250205640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9248113c10 con 0x7f924819ecb0 2024-04-25T10:32:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.150+0000 7f9250205640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9248113f10 con 0x7f9248109b80 2024-04-25T10:32:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.150+0000 7f924d7a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924819a490 0x7f924819e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.150+0000 7f924d7a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924819a490 0x7f924819e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50690/0 (socket says 172.21.15.43:50690) 2024-04-25T10:32:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.150+0000 7f924d7a0640 1 -- 172.21.15.43:0/2827187911 learned_addr learned my addr 172.21.15.43:0/2827187911 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.150+0000 7f924e7a2640 1 --2- 172.21.15.43:0/2827187911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f924819ecb0 0x7f924819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.150+0000 7f924d7a0640 1 -- 172.21.15.43:0/2827187911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9248109b80 msgr2=0x7f9248199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.150+0000 7f924d7a0640 1 --2- 172.21.15.43:0/2827187911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9248109b80 0x7f9248199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.150+0000 7f924d7a0640 1 -- 172.21.15.43:0/2827187911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f924819ecb0 msgr2=0x7f924819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.150+0000 7f924d7a0640 1 --2- 172.21.15.43:0/2827187911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f924819ecb0 0x7f924819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.150+0000 7f924d7a0640 1 -- 172.21.15.43:0/2827187911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92481a6a20 con 0x7f924819a490 2024-04-25T10:32:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.150+0000 7f924e7a2640 1 --2- 172.21.15.43:0/2827187911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f924819ecb0 0x7f924819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:32:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.150+0000 7f924d7a0640 1 --2- 172.21.15.43:0/2827187911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924819a490 0x7f924819e770 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f923800e9f0 tx=0x7f923800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.150+0000 7f923effd640 1 -- 172.21.15.43:0/2827187911 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923800ccc0 con 0x7f924819a490 2024-04-25T10:32:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.151+0000 7f923effd640 1 -- 172.21.15.43:0/2827187911 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92380054d0 con 0x7f924819a490 2024-04-25T10:32:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.151+0000 7f9250205640 1 -- 172.21.15.43:0/2827187911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92481a6bf0 con 0x7f924819a490 2024-04-25T10:32:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.151+0000 7f923effd640 1 -- 172.21.15.43:0/2827187911 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238010430 con 0x7f924819a490 2024-04-25T10:32:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.151+0000 7f9250205640 1 -- 172.21.15.43:0/2827187911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92481a7060 con 0x7f924819a490 2024-04-25T10:32:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.152+0000 7f9250205640 1 -- 172.21.15.43:0/2827187911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9248050740 con 0x7f924819a490 2024-04-25T10:32:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.154+0000 7f923effd640 1 -- 172.21.15.43:0/2827187911 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 147747+0+0 (secure 0 0 0) 0x7f92380046c0 con 0x7f924819a490 2024-04-25T10:32:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.155+0000 7f923effd640 1 --2- 172.21.15.43:0/2827187911 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f92340afd70 0x7f92340b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.155+0000 7f924dfa1640 1 --2- 172.21.15.43:0/2827187911 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f92340afd70 0x7f92340b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.155+0000 7f923effd640 1 -- 172.21.15.43:0/2827187911 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f92380df930 con 0x7f924819a490 2024-04-25T10:32:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.155+0000 7f924dfa1640 1 --2- 172.21.15.43:0/2827187911 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f92340afd70 0x7f92340b2280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f92440058c0 tx=0x7f9244005850 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.158+0000 7f923effd640 1 -- 172.21.15.43:0/2827187911 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9238053ff0 con 0x7f924819a490 2024-04-25T10:32:41.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:40 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1413627031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:41.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:40 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:32:41.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:40 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:32:41.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:40 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:32:41.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:40 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:32:41.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:40 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1797544320' entity='client.admin' 2024-04-25T10:32:41.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:40 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:32:41.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:40 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:32:41.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:40 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:32:41.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:40 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:41.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:40 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/522003913' entity='client.admin' 2024-04-25T10:32:41.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:40 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:32:41.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:40 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:32:41.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:40 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:32:41.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:40 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:41.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:40 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1413627031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:32:41.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:40 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:32:41.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:40 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:32:41.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:40 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:32:41.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:40 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:32:41.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:40 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1797544320' entity='client.admin' 2024-04-25T10:32:41.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:40 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:32:41.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:40 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:32:41.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:40 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:32:41.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:40 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:41.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:40 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/522003913' entity='client.admin' 2024-04-25T10:32:41.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:40 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:32:41.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:40 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:32:41.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:40 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:32:41.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:40 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:32:41.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.369+0000 7f9250205640 1 -- 172.21.15.43:0/2827187911 --> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f9248108970 con 0x7f92340afd70 2024-04-25T10:32:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.399+0000 7f923effd640 1 -- 172.21.15.43:0/2827187911 <== mgr.15291 v2:172.21.15.43:6800/3658924116 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f9248108970 con 0x7f92340afd70 2024-04-25T10:32:41.401 INFO:teuthology.orchestra.run.smithi043.stdout:Retention added to path / 2024-04-25T10:32:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.401+0000 7f9250205640 1 -- 172.21.15.43:0/2827187911 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f92340afd70 msgr2=0x7f92340b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.401+0000 7f9250205640 1 --2- 172.21.15.43:0/2827187911 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f92340afd70 0x7f92340b2280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f92440058c0 tx=0x7f9244005850 comp rx=0 tx=0).stop 2024-04-25T10:32:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.401+0000 7f9250205640 1 -- 172.21.15.43:0/2827187911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924819a490 msgr2=0x7f924819e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.401+0000 7f9250205640 1 --2- 172.21.15.43:0/2827187911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924819a490 0x7f924819e770 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f923800e9f0 tx=0x7f923800eeb0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.401+0000 7f9250205640 1 -- 172.21.15.43:0/2827187911 shutdown_connections 2024-04-25T10:32:41.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.401+0000 7f9250205640 1 --2- 172.21.15.43:0/2827187911 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f92340afd70 0x7f92340b2280 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.401+0000 7f9250205640 1 --2- 172.21.15.43:0/2827187911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f924819ecb0 0x7f924819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.401+0000 7f9250205640 1 --2- 172.21.15.43:0/2827187911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924819a490 0x7f924819e770 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.401+0000 7f9250205640 1 --2- 172.21.15.43:0/2827187911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9248109b80 0x7f9248199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.401+0000 7f9250205640 1 -- 172.21.15.43:0/2827187911 >> 172.21.15.43:0/2827187911 conn(0x7f92480fd8a0 msgr2=0x7f9248107250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:41.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.401+0000 7f9250205640 1 -- 172.21.15.43:0/2827187911 shutdown_connections 2024-04-25T10:32:41.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.401+0000 7f9250205640 1 -- 172.21.15.43:0/2827187911 wait complete. 2024-04-25T10:32:41.414 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T10:32:41.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.525+0000 7f0180c0d640 1 -- 172.21.15.43:0/4034997993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f017c10a010 msgr2=0x7f017c103c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:41.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.525+0000 7f0180c0d640 1 --2- 172.21.15.43:0/4034997993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f017c10a010 0x7f017c103c50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0170009900 tx=0x7f017002a410 comp rx=0 tx=0).stop 2024-04-25T10:32:41.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.525+0000 7f0180c0d640 1 -- 172.21.15.43:0/4034997993 shutdown_connections 2024-04-25T10:32:41.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.525+0000 7f0180c0d640 1 --2- 172.21.15.43:0/4034997993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f017c1042c0 0x7f017c110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.525+0000 7f0180c0d640 1 --2- 172.21.15.43:0/4034997993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f017c10a010 0x7f017c103c50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.525+0000 7f0180c0d640 1 --2- 172.21.15.43:0/4034997993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f017c109b70 0x7f017c103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.525+0000 7f0180c0d640 1 -- 172.21.15.43:0/4034997993 >> 172.21.15.43:0/4034997993 conn(0x7f017c0ff210 msgr2=0x7f017c101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:41.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.525+0000 7f0180c0d640 1 -- 172.21.15.43:0/4034997993 shutdown_connections 2024-04-25T10:32:41.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.525+0000 7f0180c0d640 1 -- 172.21.15.43:0/4034997993 wait complete. 2024-04-25T10:32:41.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.526+0000 7f0180c0d640 1 Processor -- start 2024-04-25T10:32:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.526+0000 7f0180c0d640 1 -- start start 2024-04-25T10:32:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.526+0000 7f0180c0d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f017c1042c0 0x7f017c19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.526+0000 7f0180c0d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f017c109b70 0x7f017c19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.526+0000 7f0180c0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f017c10a010 0x7f017c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.526+0000 7f0180c0d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f017c113d60 con 0x7f017c10a010 2024-04-25T10:32:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.526+0000 7f0180c0d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f017c113be0 con 0x7f017c1042c0 2024-04-25T10:32:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.526+0000 7f0180c0d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f017c113ee0 con 0x7f017c109b70 2024-04-25T10:32:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.526+0000 7f017ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f017c10a010 0x7f017c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.527+0000 7f017ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f017c10a010 0x7f017c19efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50702/0 (socket says 172.21.15.43:50702) 2024-04-25T10:32:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.527+0000 7f017ad9d640 1 -- 172.21.15.43:0/1051866621 learned_addr learned my addr 172.21.15.43:0/1051866621 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.527+0000 7f017a59c640 1 --2- 172.21.15.43:0/1051866621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f017c1042c0 0x7f017c19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.527+0000 7f0179d9b640 1 --2- 172.21.15.43:0/1051866621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f017c109b70 0x7f017c19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.527+0000 7f017ad9d640 1 -- 172.21.15.43:0/1051866621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f017c109b70 msgr2=0x7f017c19ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:41.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.527+0000 7f017ad9d640 1 --2- 172.21.15.43:0/1051866621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f017c109b70 0x7f017c19ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.527+0000 7f017ad9d640 1 -- 172.21.15.43:0/1051866621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f017c1042c0 msgr2=0x7f017c19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:41.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.527+0000 7f017ad9d640 1 --2- 172.21.15.43:0/1051866621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f017c1042c0 0x7f017c19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.527+0000 7f017ad9d640 1 -- 172.21.15.43:0/1051866621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017c19f620 con 0x7f017c10a010 2024-04-25T10:32:41.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.527+0000 7f0179d9b640 1 --2- 172.21.15.43:0/1051866621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f017c109b70 0x7f017c19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:41.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.527+0000 7f017ad9d640 1 --2- 172.21.15.43:0/1051866621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f017c10a010 0x7f017c19efe0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f017400eaa0 tx=0x7f017400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:41.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.527+0000 7f016b7fe640 1 -- 172.21.15.43:0/1051866621 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017400cdb0 con 0x7f017c10a010 2024-04-25T10:32:41.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.527+0000 7f016b7fe640 1 -- 172.21.15.43:0/1051866621 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0174005020 con 0x7f017c10a010 2024-04-25T10:32:41.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.527+0000 7f0180c0d640 1 -- 172.21.15.43:0/1051866621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017c19f910 con 0x7f017c10a010 2024-04-25T10:32:41.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.527+0000 7f016b7fe640 1 -- 172.21.15.43:0/1051866621 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01740116d0 con 0x7f017c10a010 2024-04-25T10:32:41.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.527+0000 7f0180c0d640 1 -- 172.21.15.43:0/1051866621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f017c19fde0 con 0x7f017c10a010 2024-04-25T10:32:41.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.528+0000 7f0180c0d640 1 -- 172.21.15.43:0/1051866621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f017c050740 con 0x7f017c10a010 2024-04-25T10:32:41.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.530+0000 7f016b7fe640 1 -- 172.21.15.43:0/1051866621 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 147747+0+0 (secure 0 0 0) 0x7f01740051d0 con 0x7f017c10a010 2024-04-25T10:32:41.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.531+0000 7f016b7fe640 1 --2- 172.21.15.43:0/1051866621 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f01640afd70 0x7f01640b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:41.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.531+0000 7f017a59c640 1 --2- 172.21.15.43:0/1051866621 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f01640afd70 0x7f01640b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:41.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.531+0000 7f016b7fe640 1 -- 172.21.15.43:0/1051866621 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f01740df8b0 con 0x7f017c10a010 2024-04-25T10:32:41.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.531+0000 7f017a59c640 1 --2- 172.21.15.43:0/1051866621 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f01640afd70 0x7f01640b2280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f016c005fd0 tx=0x7f016c00b390 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:41.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.534+0000 7f016b7fe640 1 -- 172.21.15.43:0/1051866621 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0174016030 con 0x7f017c10a010 2024-04-25T10:32:41.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.745+0000 7f0180c0d640 1 -- 172.21.15.43:0/1051866621 --> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f017c10ff20 con 0x7f01640afd70 2024-04-25T10:32:41.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.754+0000 7f016b7fe640 1 -- 172.21.15.43:0/1051866621 <== mgr.15291 v2:172.21.15.43:6800/3658924116 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f017c10ff20 con 0x7f01640afd70 2024-04-25T10:32:41.755 INFO:teuthology.orchestra.run.smithi043.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T10:32:40", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T10:32:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.756+0000 7f0180c0d640 1 -- 172.21.15.43:0/1051866621 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f01640afd70 msgr2=0x7f01640b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.756+0000 7f0180c0d640 1 --2- 172.21.15.43:0/1051866621 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f01640afd70 0x7f01640b2280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f016c005fd0 tx=0x7f016c00b390 comp rx=0 tx=0).stop 2024-04-25T10:32:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.756+0000 7f0180c0d640 1 -- 172.21.15.43:0/1051866621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f017c10a010 msgr2=0x7f017c19efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.756+0000 7f0180c0d640 1 --2- 172.21.15.43:0/1051866621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f017c10a010 0x7f017c19efe0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f017400eaa0 tx=0x7f017400ef60 comp rx=0 tx=0).stop 2024-04-25T10:32:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.756+0000 7f0180c0d640 1 -- 172.21.15.43:0/1051866621 shutdown_connections 2024-04-25T10:32:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.756+0000 7f0180c0d640 1 --2- 172.21.15.43:0/1051866621 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f01640afd70 0x7f01640b2280 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.756+0000 7f0180c0d640 1 --2- 172.21.15.43:0/1051866621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f017c10a010 0x7f017c19efe0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.756+0000 7f0180c0d640 1 --2- 172.21.15.43:0/1051866621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f017c109b70 0x7f017c19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.756+0000 7f0180c0d640 1 --2- 172.21.15.43:0/1051866621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f017c1042c0 0x7f017c19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.756+0000 7f0180c0d640 1 -- 172.21.15.43:0/1051866621 >> 172.21.15.43:0/1051866621 conn(0x7f017c0ff210 msgr2=0x7f017c10f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.756+0000 7f0180c0d640 1 -- 172.21.15.43:0/1051866621 shutdown_connections 2024-04-25T10:32:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.756+0000 7f0180c0d640 1 -- 172.21.15.43:0/1051866621 wait complete. 2024-04-25T10:32:41.769 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T10:32:41.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.880+0000 7f048c99d640 1 -- 172.21.15.43:0/2224172692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f048810dfa0 msgr2=0x7f0488110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:41.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.880+0000 7f048c99d640 1 --2- 172.21.15.43:0/2224172692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f048810dfa0 0x7f0488110470 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0480009ca0 tx=0x7f048002a950 comp rx=0 tx=0).stop 2024-04-25T10:32:41.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.881+0000 7f048c99d640 1 -- 172.21.15.43:0/2224172692 shutdown_connections 2024-04-25T10:32:41.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.881+0000 7f048c99d640 1 --2- 172.21.15.43:0/2224172692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f048810dfa0 0x7f0488110470 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.881+0000 7f048c99d640 1 --2- 172.21.15.43:0/2224172692 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f048810b5f0 0x7f048810da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.881+0000 7f048c99d640 1 --2- 172.21.15.43:0/2224172692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0488104b90 0x7f0488104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.881+0000 7f048c99d640 1 -- 172.21.15.43:0/2224172692 >> 172.21.15.43:0/2224172692 conn(0x7f0488078040 msgr2=0x7f0488078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:41.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.881+0000 7f048c99d640 1 -- 172.21.15.43:0/2224172692 shutdown_connections 2024-04-25T10:32:41.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.881+0000 7f048c99d640 1 -- 172.21.15.43:0/2224172692 wait complete. 2024-04-25T10:32:41.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.881+0000 7f048c99d640 1 Processor -- start 2024-04-25T10:32:41.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.881+0000 7f048c99d640 1 -- start start 2024-04-25T10:32:41.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.882+0000 7f048c99d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0488104b90 0x7f048819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:41.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.882+0000 7f048c99d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f048810b5f0 0x7f048819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:41.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.882+0000 7f048c99d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f048810dfa0 0x7f048819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:41.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.882+0000 7f048c99d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0488113b40 con 0x7f048810dfa0 2024-04-25T10:32:41.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.882+0000 7f048c99d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f04881139c0 con 0x7f048810b5f0 2024-04-25T10:32:41.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.882+0000 7f048c99d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0488113cc0 con 0x7f0488104b90 2024-04-25T10:32:41.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.882+0000 7f0487fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f048810dfa0 0x7f048819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:41.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.882+0000 7f0487fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f048810dfa0 0x7f048819ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50716/0 (socket says 172.21.15.43:50716) 2024-04-25T10:32:41.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.882+0000 7f0487fff640 1 -- 172.21.15.43:0/2243784885 learned_addr learned my addr 172.21.15.43:0/2243784885 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:41.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.882+0000 7f04877fe640 1 --2- 172.21.15.43:0/2243784885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0488104b90 0x7f048819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:41.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.882+0000 7f0486ffd640 1 --2- 172.21.15.43:0/2243784885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f048810b5f0 0x7f048819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:41.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.882+0000 7f0487fff640 1 -- 172.21.15.43:0/2243784885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0488104b90 msgr2=0x7f048819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:41.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.882+0000 7f0487fff640 1 --2- 172.21.15.43:0/2243784885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0488104b90 0x7f048819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.882+0000 7f0487fff640 1 -- 172.21.15.43:0/2243784885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f048810b5f0 msgr2=0x7f048819aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:41.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.882+0000 7f0487fff640 1 --2- 172.21.15.43:0/2243784885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f048810b5f0 0x7f048819aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.882+0000 7f0487fff640 1 -- 172.21.15.43:0/2243784885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048819f340 con 0x7f048810dfa0 2024-04-25T10:32:41.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.882+0000 7f0487fff640 1 --2- 172.21.15.43:0/2243784885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f048810dfa0 0x7f048819ed80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0480006010 tx=0x7f048002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:41.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.883+0000 7f0484ff9640 1 -- 172.21.15.43:0/2243784885 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0480034040 con 0x7f048810dfa0 2024-04-25T10:32:41.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.883+0000 7f0484ff9640 1 -- 172.21.15.43:0/2243784885 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04800308f0 con 0x7f048810dfa0 2024-04-25T10:32:41.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.883+0000 7f048c99d640 1 -- 172.21.15.43:0/2243784885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f048819f5d0 con 0x7f048810dfa0 2024-04-25T10:32:41.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.883+0000 7f0484ff9640 1 -- 172.21.15.43:0/2243784885 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0480004750 con 0x7f048810dfa0 2024-04-25T10:32:41.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.883+0000 7f048c99d640 1 -- 172.21.15.43:0/2243784885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f048819fac0 con 0x7f048810dfa0 2024-04-25T10:32:41.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.884+0000 7f048c99d640 1 -- 172.21.15.43:0/2243784885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0488050740 con 0x7f048810dfa0 2024-04-25T10:32:41.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.885+0000 7f0484ff9640 1 -- 172.21.15.43:0/2243784885 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 147747+0+0 (secure 0 0 0) 0x7f0480037020 con 0x7f048810dfa0 2024-04-25T10:32:41.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.886+0000 7f0484ff9640 1 --2- 172.21.15.43:0/2243784885 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f04700afd70 0x7f04700b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:41.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.886+0000 7f04877fe640 1 --2- 172.21.15.43:0/2243784885 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f04700afd70 0x7f04700b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:41.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.886+0000 7f0484ff9640 1 -- 172.21.15.43:0/2243784885 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f04801002f0 con 0x7f048810dfa0 2024-04-25T10:32:41.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.886+0000 7f04877fe640 1 --2- 172.21.15.43:0/2243784885 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f04700afd70 0x7f04700b2280 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f047800abe0 tx=0x7f047800a2d0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:41.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:41.889+0000 7f0484ff9640 1 -- 172.21.15.43:0/2243784885 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0480080ff0 con 0x7f048810dfa0 2024-04-25T10:32:42.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:42.100+0000 7f048c99d640 1 -- 172.21.15.43:0/2243784885 --> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f0488101760 con 0x7f04700afd70 2024-04-25T10:32:42.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:42.108+0000 7f0484ff9640 1 -- 172.21.15.43:0/2243784885 <== mgr.15291 v2:172.21.15.43:6800/3658924116 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f0488101760 con 0x7f04700afd70 2024-04-25T10:32:42.110 INFO:teuthology.orchestra.run.smithi043.stdout:/ 1m 6m3h 2024-04-25T10:32:42.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:41 smithi043 ceph-mon[29478]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.0 KiB/s wr, 16 op/s 2024-04-25T10:32:42.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:41 smithi043 ceph-mon[29478]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:42.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:42.110+0000 7f048c99d640 1 -- 172.21.15.43:0/2243784885 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f04700afd70 msgr2=0x7f04700b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:42.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:42.110+0000 7f048c99d640 1 --2- 172.21.15.43:0/2243784885 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f04700afd70 0x7f04700b2280 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f047800abe0 tx=0x7f047800a2d0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:42.110+0000 7f048c99d640 1 -- 172.21.15.43:0/2243784885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f048810dfa0 msgr2=0x7f048819ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:42.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:42.110+0000 7f048c99d640 1 --2- 172.21.15.43:0/2243784885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f048810dfa0 0x7f048819ed80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0480006010 tx=0x7f048002f5d0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:42.110+0000 7f048c99d640 1 -- 172.21.15.43:0/2243784885 shutdown_connections 2024-04-25T10:32:42.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:42.110+0000 7f048c99d640 1 --2- 172.21.15.43:0/2243784885 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f04700afd70 0x7f04700b2280 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:42.110+0000 7f048c99d640 1 --2- 172.21.15.43:0/2243784885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f048810b5f0 0x7f048819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:42.110+0000 7f048c99d640 1 --2- 172.21.15.43:0/2243784885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f048810dfa0 0x7f048819ed80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:42.110+0000 7f048c99d640 1 --2- 172.21.15.43:0/2243784885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0488104b90 0x7f048819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:42.110+0000 7f048c99d640 1 -- 172.21.15.43:0/2243784885 >> 172.21.15.43:0/2243784885 conn(0x7f0488078040 msgr2=0x7f0488101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:42.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:42.110+0000 7f048c99d640 1 -- 172.21.15.43:0/2243784885 shutdown_connections 2024-04-25T10:32:42.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:32:42.110+0000 7f048c99d640 1 -- 172.21.15.43:0/2243784885 wait complete. 2024-04-25T10:32:42.123 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T10:32:42.149 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T10:32:42.177 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T10:32:42.187 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T10:32:42.187 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T10:32:42.192 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T10:32:42.205 INFO:tasks.workunit:Pulling workunits from ref 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:32:42.205 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T10:32:42.205 INFO:tasks.workunit:timeout=3h 2024-04-25T10:32:42.205 INFO:tasks.workunit:cleanup=True 2024-04-25T10:32:42.206 DEBUG:teuthology.orchestra.run.smithi043:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:32:42.222 INFO:teuthology.orchestra.run.smithi043.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T10:32:42.222 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-25T10:32:42.222 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-25T10:32:42.222 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:32:42.222 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-25T10:32:42.222 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-25 10:30:40.289613012 +0000 2024-04-25T10:32:42.222 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-25 10:30:40.289613012 +0000 2024-04-25T10:32:42.222 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-25 10:32:29.522388037 +0000 2024-04-25T10:32:42.222 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-25 10:30:40.289613012 +0000 2024-04-25T10:32:42.222 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T10:32:42.223 DEBUG:teuthology.orchestra.run.smithi043:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T10:32:42.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:41 smithi077 ceph-mon[32772]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.0 KiB/s wr, 16 op/s 2024-04-25T10:32:42.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:41 smithi077 ceph-mon[32772]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:42.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:41 smithi164 ceph-mon[32713]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.0 KiB/s wr, 16 op/s 2024-04-25T10:32:42.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:41 smithi164 ceph-mon[32713]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:42.297 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:32:42.354 INFO:tasks.workunit.client.0.smithi043.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T10:32:43.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:42 smithi043 ceph-mon[29478]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:43.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:42 smithi043 ceph-mon[29478]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:43.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:42 smithi077 ceph-mon[32772]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:43.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:42 smithi077 ceph-mon[32772]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:43.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:42 smithi164 ceph-mon[32713]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:43.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:42 smithi164 ceph-mon[32713]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:44.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:43 smithi043 ceph-mon[29478]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:44.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:43 smithi043 ceph-mon[29478]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 797 B/s wr, 12 op/s 2024-04-25T10:32:44.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:43 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:32:44.192 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:43 smithi077 ceph-mon[32772]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:44.192 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:43 smithi077 ceph-mon[32772]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 797 B/s wr, 12 op/s 2024-04-25T10:32:44.192 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:43 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:32:44.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:43 smithi164 ceph-mon[32713]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:32:44.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:43 smithi164 ceph-mon[32713]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 797 B/s wr, 12 op/s 2024-04-25T10:32:44.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:43 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:32:46.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:45 smithi043 ceph-mon[29478]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 85 KiB/s wr, 27 op/s 2024-04-25T10:32:46.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:45 smithi077 ceph-mon[32772]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 85 KiB/s wr, 27 op/s 2024-04-25T10:32:46.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:45 smithi164 ceph-mon[32713]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 85 KiB/s wr, 27 op/s 2024-04-25T10:32:48.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:47 smithi043 ceph-mon[29478]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 78 KiB/s wr, 24 op/s 2024-04-25T10:32:48.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:47 smithi077 ceph-mon[32772]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 78 KiB/s wr, 24 op/s 2024-04-25T10:32:48.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:47 smithi164 ceph-mon[32713]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 78 KiB/s wr, 24 op/s 2024-04-25T10:32:50.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:49 smithi043 ceph-mon[29478]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 78 KiB/s wr, 25 op/s 2024-04-25T10:32:50.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:49 smithi077 ceph-mon[32772]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 78 KiB/s wr, 25 op/s 2024-04-25T10:32:50.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:49 smithi164 ceph-mon[32713]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 78 KiB/s wr, 25 op/s 2024-04-25T10:32:52.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:51 smithi043 ceph-mon[29478]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 78 KiB/s wr, 26 op/s 2024-04-25T10:32:52.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:51 smithi077 ceph-mon[32772]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 78 KiB/s wr, 26 op/s 2024-04-25T10:32:52.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:51 smithi164 ceph-mon[32713]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 78 KiB/s wr, 26 op/s 2024-04-25T10:32:54.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:53 smithi077 ceph-mon[32772]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 77 KiB/s wr, 17 op/s 2024-04-25T10:32:54.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:53 smithi164 ceph-mon[32713]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 77 KiB/s wr, 17 op/s 2024-04-25T10:32:54.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:53 smithi043 ceph-mon[29478]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 77 KiB/s wr, 17 op/s 2024-04-25T10:32:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.344+0000 7f4c7915e640 1 -- 172.21.15.43:0/1921764035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c741031c0 msgr2=0x7f4c7410f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.344+0000 7f4c7915e640 1 --2- 172.21.15.43:0/1921764035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c741031c0 0x7f4c7410f4f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009900 tx=0x7f4c6c02a410 comp rx=0 tx=0).stop 2024-04-25T10:32:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.344+0000 7f4c7915e640 1 -- 172.21.15.43:0/1921764035 shutdown_connections 2024-04-25T10:32:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.344+0000 7f4c7915e640 1 --2- 172.21.15.43:0/1921764035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c74108f00 0x7f4c74102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.344+0000 7f4c7915e640 1 --2- 172.21.15.43:0/1921764035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c741031c0 0x7f4c7410f4f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.344+0000 7f4c7915e640 1 --2- 172.21.15.43:0/1921764035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c74108a60 0x7f4c74102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.344+0000 7f4c7915e640 1 -- 172.21.15.43:0/1921764035 >> 172.21.15.43:0/1921764035 conn(0x7f4c740781b0 msgr2=0x7f4c740785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.344+0000 7f4c7915e640 1 -- 172.21.15.43:0/1921764035 shutdown_connections 2024-04-25T10:32:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.344+0000 7f4c7915e640 1 -- 172.21.15.43:0/1921764035 wait complete. 2024-04-25T10:32:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.345+0000 7f4c7915e640 1 Processor -- start 2024-04-25T10:32:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.345+0000 7f4c7915e640 1 -- start start 2024-04-25T10:32:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.346+0000 7f4c7915e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c741031c0 0x7f4c741a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.346+0000 7f4c7915e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c74108a60 0x7f4c741a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.346+0000 7f4c7915e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c74108f00 0x7f4c7419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.346+0000 7f4c7915e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4c74112ca0 con 0x7f4c74108f00 2024-04-25T10:32:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.346+0000 7f4c7915e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4c74112b20 con 0x7f4c741031c0 2024-04-25T10:32:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.346+0000 7f4c7915e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4c74112e20 con 0x7f4c74108a60 2024-04-25T10:32:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.346+0000 7f4c7359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c74108f00 0x7f4c7419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.346+0000 7f4c7359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c74108f00 0x7f4c7419c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48774/0 (socket says 172.21.15.43:48774) 2024-04-25T10:32:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.346+0000 7f4c7359e640 1 -- 172.21.15.43:0/2200255204 learned_addr learned my addr 172.21.15.43:0/2200255204 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.346+0000 7f4c72d9d640 1 --2- 172.21.15.43:0/2200255204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c741031c0 0x7f4c741a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.346+0000 7f4c7259c640 1 --2- 172.21.15.43:0/2200255204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c74108a60 0x7f4c741a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.346+0000 7f4c7359e640 1 -- 172.21.15.43:0/2200255204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c74108a60 msgr2=0x7f4c741a2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.346+0000 7f4c7359e640 1 --2- 172.21.15.43:0/2200255204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c74108a60 0x7f4c741a2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.346+0000 7f4c7359e640 1 -- 172.21.15.43:0/2200255204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c741031c0 msgr2=0x7f4c741a25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.346+0000 7f4c7359e640 1 --2- 172.21.15.43:0/2200255204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c741031c0 0x7f4c741a25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.346+0000 7f4c7359e640 1 -- 172.21.15.43:0/2200255204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7419ced0 con 0x7f4c74108f00 2024-04-25T10:32:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.346+0000 7f4c7359e640 1 --2- 172.21.15.43:0/2200255204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c74108f00 0x7f4c7419c7f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c004770 tx=0x7f4c6c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.347+0000 7f4c5bfff640 1 -- 172.21.15.43:0/2200255204 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c03f030 con 0x7f4c74108f00 2024-04-25T10:32:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.347+0000 7f4c7915e640 1 -- 172.21.15.43:0/2200255204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c7419d160 con 0x7f4c74108f00 2024-04-25T10:32:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.347+0000 7f4c5bfff640 1 -- 172.21.15.43:0/2200255204 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c6c004440 con 0x7f4c74108f00 2024-04-25T10:32:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.347+0000 7f4c5bfff640 1 -- 172.21.15.43:0/2200255204 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c035070 con 0x7f4c74108f00 2024-04-25T10:32:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.347+0000 7f4c7915e640 1 -- 172.21.15.43:0/2200255204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c741a9410 con 0x7f4c74108f00 2024-04-25T10:32:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.349+0000 7f4c8000e740 1 Processor -- start 2024-04-25T10:32:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.350+0000 7f4c5bfff640 1 -- 172.21.15.43:0/2200255204 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 147747+0+0 (secure 0 0 0) 0x7f4c6c042030 con 0x7f4c74108f00 2024-04-25T10:32:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.350+0000 7f4c8000e740 1 -- start start 2024-04-25T10:32:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.350+0000 7f4c8000e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21ad11b0 0x55ba21e70af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.350+0000 7f4c8000e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21aa82f0 0x55ba21e71030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.350+0000 7f4c8000e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21ab04d0 0x55ba21e71570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.350+0000 7f4c8000e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ba21dd7460 con 0x55ba21aa82f0 2024-04-25T10:32:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.350+0000 7f4c8000e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ba21d91590 con 0x55ba21ab04d0 2024-04-25T10:32:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.350+0000 7f4c8000e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ba21dd3420 con 0x55ba21ad11b0 2024-04-25T10:32:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.350+0000 7f4c72d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21aa82f0 0x55ba21e71030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.350+0000 7f4c72d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21aa82f0 0x55ba21e71030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48784/0 (socket says 172.21.15.43:48784) 2024-04-25T10:32:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.350+0000 7f4c72d9d640 1 -- 172.21.15.43:0/2955121717 learned_addr learned my addr 172.21.15.43:0/2955121717 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.350+0000 7f4c7259c640 1 --2- 172.21.15.43:0/2955121717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21ab04d0 0x55ba21e71570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.350+0000 7f4c7259c640 1 --2- 172.21.15.43:0/2955121717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21ad11b0 0x55ba21e70af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.350+0000 7f4c5bfff640 1 --2- 172.21.15.43:0/2200255204 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f4c5c0afd70 0x7f4c5c0b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.351+0000 7f4c72d9d640 1 -- 172.21.15.43:0/2955121717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21ad11b0 msgr2=0x55ba21e70af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.351+0000 7f4c72d9d640 1 --2- 172.21.15.43:0/2955121717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21ad11b0 0x55ba21e70af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.351+0000 7f4c72d9d640 1 -- 172.21.15.43:0/2955121717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21ab04d0 msgr2=0x55ba21e71570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.351+0000 7f4c72d9d640 1 --2- 172.21.15.43:0/2955121717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21ab04d0 0x55ba21e71570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.351+0000 7f4c72d9d640 1 -- 172.21.15.43:0/2955121717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba21d9e0d0 con 0x55ba21aa82f0 2024-04-25T10:32:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.351+0000 7f4c7259c640 1 --2- 172.21.15.43:0/2955121717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21ab04d0 0x55ba21e71570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.351+0000 7f4c7359e640 1 --2- 172.21.15.43:0/2200255204 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f4c5c0afd70 0x7f4c5c0b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.351+0000 7f4c5bfff640 1 -- 172.21.15.43:0/2200255204 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f4c6c0ff530 con 0x7f4c74108f00 2024-04-25T10:32:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.351+0000 7f4c72d9d640 1 --2- 172.21.15.43:0/2955121717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21aa82f0 0x55ba21e71030 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009fb0 tx=0x7f4c640118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11f2edf4803000b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.351+0000 7f4c597fa640 1 -- 172.21.15.43:0/2955121717 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64004d20 con 0x55ba21aa82f0 2024-04-25T10:32:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.351+0000 7f4c597fa640 1 -- 172.21.15.43:0/2955121717 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c64004050 con 0x55ba21aa82f0 2024-04-25T10:32:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.351+0000 7f4c597fa640 1 -- 172.21.15.43:0/2955121717 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64005240 con 0x55ba21aa82f0 2024-04-25T10:32:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.352+0000 7f4c8000e740 1 -- 172.21.15.43:0/2955121717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21aa82f0 msgr2=0x55ba21e71030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.352+0000 7f4c8000e740 1 --2- 172.21.15.43:0/2955121717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21aa82f0 0x55ba21e71030 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009fb0 tx=0x7f4c640118d0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.352+0000 7f4c7359e640 1 --2- 172.21.15.43:0/2200255204 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f4c5c0afd70 0x7f4c5c0b2280 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0fcb90 tx=0x7f4c6c096130 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.352+0000 7f4c8000e740 1 -- 172.21.15.43:0/2955121717 shutdown_connections 2024-04-25T10:32:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.352+0000 7f4c8000e740 1 --2- 172.21.15.43:0/2955121717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21ab04d0 0x55ba21e71570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.352+0000 7f4c8000e740 1 --2- 172.21.15.43:0/2955121717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21aa82f0 0x55ba21e71030 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.352+0000 7f4c8000e740 1 --2- 172.21.15.43:0/2955121717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21ad11b0 0x55ba21e70af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.352+0000 7f4c8000e740 1 -- 172.21.15.43:0/2955121717 >> 172.21.15.43:0/2955121717 conn(0x55ba21ac5300 msgr2=0x55ba21e6d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.352+0000 7f4c8000e740 1 -- 172.21.15.43:0/2955121717 shutdown_connections 2024-04-25T10:32:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.352+0000 7f4c8000e740 1 -- 172.21.15.43:0/2955121717 wait complete. 2024-04-25T10:32:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.352+0000 7f4c8000e740 1 -- start start 2024-04-25T10:32:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.353+0000 7f4c8000e740 1 Processor -- start 2024-04-25T10:32:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.353+0000 7f4c8000e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21aa82f0 0x55ba21e7b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.353+0000 7f4c8000e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21ab04d0 0x55ba21e7b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.353+0000 7f4c8000e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ad11b0 0x55ba21e81110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.353+0000 7f4c8000e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ba21e7dc20 con 0x55ba21ad11b0 2024-04-25T10:32:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.353+0000 7f4c8000e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ba21d5eb60 con 0x55ba21aa82f0 2024-04-25T10:32:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.353+0000 7f4c8000e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ba21e7dda0 con 0x55ba21ab04d0 2024-04-25T10:32:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.353+0000 7f4c8000e740 10 client.0 _ll_register_callbacks cb 0x55ba21d832b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.353+0000 7f4c7259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ad11b0 0x55ba21e81110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.353+0000 7f4c7259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ad11b0 0x55ba21e81110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48794/0 (socket says 172.21.15.43:48794) 2024-04-25T10:32:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.353+0000 7f4c7259c640 1 -- 172.21.15.43:0/1739649760 learned_addr learned my addr 172.21.15.43:0/1739649760 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.353+0000 7f4c7259c640 1 --2- 172.21.15.43:0/1739649760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21aa82f0 0x55ba21e7b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.353+0000 7f4c72d9d640 1 --2- 172.21.15.43:0/1739649760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21ab04d0 0x55ba21e7b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.353+0000 7f4c7259c640 1 -- 172.21.15.43:0/1739649760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21ab04d0 msgr2=0x55ba21e7b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.353+0000 7f4c7259c640 1 --2- 172.21.15.43:0/1739649760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21ab04d0 0x55ba21e7b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.353+0000 7f4c7259c640 1 -- 172.21.15.43:0/1739649760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21aa82f0 msgr2=0x55ba21e7b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.353+0000 7f4c7259c640 1 --2- 172.21.15.43:0/1739649760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21aa82f0 0x55ba21e7b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.353+0000 7f4c7259c640 1 -- 172.21.15.43:0/1739649760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba21ae9210 con 0x55ba21ad11b0 2024-04-25T10:32:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.354+0000 7f4c7259c640 1 --2- 172.21.15.43:0/1739649760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ad11b0 0x55ba21e81110 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4c7406c520 tx=0x7f4c68009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5592008872a632a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.354+0000 7f4c56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:32:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.354+0000 7f4c56ff5640 1 -- 172.21.15.43:0/1739649760 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6801a040 con 0x55ba21ad11b0 2024-04-25T10:32:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.354+0000 7f4c56ff5640 1 -- 172.21.15.43:0/1739649760 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f4c6800fc50 con 0x55ba21ad11b0 2024-04-25T10:32:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.354+0000 7f4c8000e740 1 -- 172.21.15.43:0/1739649760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c64004050 con 0x55ba21ad11b0 2024-04-25T10:32:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.354+0000 7f4c56ff5640 1 -- 172.21.15.43:0/1739649760 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4c6800fe50 con 0x55ba21ad11b0 2024-04-25T10:32:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.354+0000 7f4c8000e740 10 client.15552 fetch_fsmap learned FSMap version 24 2024-04-25T10:32:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.354+0000 7f4c8000e740 1 -- 172.21.15.43:0/1739649760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ba21bb5620 con 0x55ba21ad11b0 2024-04-25T10:32:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.354+0000 7f4c56ff5640 1 -- 172.21.15.43:0/1739649760 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c64004050 con 0x55ba21ad11b0 2024-04-25T10:32:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.354+0000 7f4c56ff5640 1 -- 172.21.15.43:0/1739649760 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4c68029380 con 0x55ba21ad11b0 2024-04-25T10:32:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.354+0000 7f4c8000e740 10 client.15552 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:32:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.354+0000 7f4c8000e740 10 client.15552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:32:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.354+0000 7f4c8000e740 20 client.15552 populate_metadata read hostname 'smithi043' 2024-04-25T10:32:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.355+0000 7f4c8000e740 1 --2- 172.21.15.43:0/1739649760 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ba21a8fd80 0x55ba21e8d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.355+0000 7f4c8000e740 4 client.15552 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:32:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.355+0000 7f4c8000e740 1 -- 172.21.15.43:0/1739649760 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba21de5070 con 0x55ba21a8fd80 2024-04-25T10:32:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.355+0000 7f4c72d9d640 1 --2- 172.21.15.43:0/1739649760 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ba21a8fd80 0x55ba21e8d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.355+0000 7f4c56ff5640 10 client.15552 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:32:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.355+0000 7f4c72d9d640 1 --2- 172.21.15.43:0/1739649760 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ba21a8fd80 0x55ba21e8d780 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdd871c6a2cd8f0b server_cookie=97b4383512ddc579 in_seq=0 out_seq=0 2024-04-25T10:32:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.356+0000 7f4c56ff5640 1 -- 172.21.15.43:0/1739649760 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ba21de5070 con 0x55ba21a8fd80 2024-04-25T10:32:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.356+0000 7f4c56ff5640 10 client.15552 handle_command_reply: tid=0 2024-04-25T10:32:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.356+0000 7f4c8000e740 1 client.15552 shutdown 2024-04-25T10:32:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.356+0000 7f4c8000e740 1 -- 172.21.15.43:0/1739649760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ad11b0 msgr2=0x55ba21e81110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.356+0000 7f4c8000e740 1 --2- 172.21.15.43:0/1739649760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ad11b0 0x55ba21e81110 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4c7406c520 tx=0x7f4c68009a40 comp rx=0 tx=0).stop 2024-04-25T10:32:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.356+0000 7f4c8000e740 1 -- 172.21.15.43:0/1739649760 shutdown_connections 2024-04-25T10:32:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.356+0000 7f4c8000e740 1 --2- 172.21.15.43:0/1739649760 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ba21a8fd80 0x55ba21e8d780 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.356+0000 7f4c8000e740 1 --2- 172.21.15.43:0/1739649760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ad11b0 0x55ba21e81110 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.356+0000 7f4c8000e740 1 --2- 172.21.15.43:0/1739649760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21ab04d0 0x55ba21e7b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.356+0000 7f4c56ff5640 0 client.15552 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:32:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.356+0000 7f4c8000e740 1 --2- 172.21.15.43:0/1739649760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21aa82f0 0x55ba21e7b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.356+0000 7f4c8000e740 1 -- 172.21.15.43:0/1739649760 >> 172.21.15.43:0/1739649760 conn(0x55ba21ac5300 msgr2=0x55ba21e70920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.356+0000 7f4c8000e740 1 -- 172.21.15.43:0/1739649760 shutdown_connections 2024-04-25T10:32:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.356+0000 7f4c8000e740 1 -- 172.21.15.43:0/1739649760 wait complete. 2024-04-25T10:32:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.356+0000 7f4c8000e740 20 client.15552 trim_cache size 0 max 16384 2024-04-25T10:32:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.372+0000 7f4c8000e740 1 Processor -- start 2024-04-25T10:32:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.372+0000 7f4c8000e740 1 -- start start 2024-04-25T10:32:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.372+0000 7f4c8000e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21aa82f0 0x55ba21e92f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.372+0000 7f4c8000e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ab04d0 0x55ba21e934a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.372+0000 7f4c8000e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21ad11b0 0x55ba21e70880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.372+0000 7f4c8000e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4c741b4440 con 0x55ba21ab04d0 2024-04-25T10:32:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.372+0000 7f4c8000e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4c741b1d10 con 0x55ba21ad11b0 2024-04-25T10:32:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.372+0000 7f4c8000e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4c741afb60 con 0x55ba21aa82f0 2024-04-25T10:32:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.372+0000 7f4c72d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ab04d0 0x55ba21e934a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.372+0000 7f4c72d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ab04d0 0x55ba21e934a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48798/0 (socket says 172.21.15.43:48798) 2024-04-25T10:32:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.372+0000 7f4c72d9d640 1 -- 172.21.15.43:0/2409637668 learned_addr learned my addr 172.21.15.43:0/2409637668 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.372+0000 7f4c7259c640 1 --2- 172.21.15.43:0/2409637668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21aa82f0 0x55ba21e92f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.372+0000 7f4c7259c640 1 --2- 172.21.15.43:0/2409637668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21ad11b0 0x55ba21e70880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.372+0000 7f4c72d9d640 1 -- 172.21.15.43:0/2409637668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21aa82f0 msgr2=0x55ba21e92f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.372+0000 7f4c72d9d640 1 --2- 172.21.15.43:0/2409637668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21aa82f0 0x55ba21e92f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.372+0000 7f4c72d9d640 1 -- 172.21.15.43:0/2409637668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21ad11b0 msgr2=0x55ba21e70880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.372+0000 7f4c72d9d640 1 --2- 172.21.15.43:0/2409637668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21ad11b0 0x55ba21e70880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.372+0000 7f4c72d9d640 1 -- 172.21.15.43:0/2409637668 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba21e71ed0 con 0x55ba21ab04d0 2024-04-25T10:32:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.373+0000 7f4c72d9d640 1 --2- 172.21.15.43:0/2409637668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ab04d0 0x55ba21e934a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4c64011dd0 tx=0x7f4c64004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2260fdae91da4681 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.373+0000 7f4c58ff9640 1 -- 172.21.15.43:0/2409637668 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6402d070 con 0x55ba21ab04d0 2024-04-25T10:32:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.373+0000 7f4c58ff9640 1 -- 172.21.15.43:0/2409637668 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c64004670 con 0x55ba21ab04d0 2024-04-25T10:32:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.373+0000 7f4c58ff9640 1 -- 172.21.15.43:0/2409637668 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64003fb0 con 0x55ba21ab04d0 2024-04-25T10:32:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.374+0000 7f4c8000e740 1 -- 172.21.15.43:0/2409637668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ab04d0 msgr2=0x55ba21e934a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.374+0000 7f4c8000e740 1 --2- 172.21.15.43:0/2409637668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ab04d0 0x55ba21e934a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4c64011dd0 tx=0x7f4c64004890 comp rx=0 tx=0).stop 2024-04-25T10:32:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.374+0000 7f4c8000e740 1 -- 172.21.15.43:0/2409637668 shutdown_connections 2024-04-25T10:32:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.374+0000 7f4c8000e740 1 --2- 172.21.15.43:0/2409637668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21ad11b0 0x55ba21e70880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.374+0000 7f4c8000e740 1 --2- 172.21.15.43:0/2409637668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ab04d0 0x55ba21e934a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.374+0000 7f4c8000e740 1 --2- 172.21.15.43:0/2409637668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21aa82f0 0x55ba21e92f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.374+0000 7f4c8000e740 1 -- 172.21.15.43:0/2409637668 >> 172.21.15.43:0/2409637668 conn(0x55ba21ac5300 msgr2=0x55ba21e798b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.374+0000 7f4c8000e740 1 -- 172.21.15.43:0/2409637668 shutdown_connections 2024-04-25T10:32:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.374+0000 7f4c8000e740 1 -- 172.21.15.43:0/2409637668 wait complete. 2024-04-25T10:32:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.374+0000 7f4c8000e740 1 -- start start 2024-04-25T10:32:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.374+0000 7f4c8000e740 1 Processor -- start 2024-04-25T10:32:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c8000e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21aa82f0 0x55ba21e85030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c8000e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ab04d0 0x55ba21e85570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c8000e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21ad11b0 0x55ba21e85ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c8000e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4c741b01d0 con 0x55ba21ab04d0 2024-04-25T10:32:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c8000e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4c741b1640 con 0x55ba21ad11b0 2024-04-25T10:32:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c8000e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4c741b08a0 con 0x55ba21aa82f0 2024-04-25T10:32:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c8000e740 10 client.0 _ll_register_callbacks cb 0x7f4c680109f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c72d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ab04d0 0x55ba21e85570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c72d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ab04d0 0x55ba21e85570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48802/0 (socket says 172.21.15.43:48802) 2024-04-25T10:32:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c72d9d640 1 -- 172.21.15.43:0/886359612 learned_addr learned my addr 172.21.15.43:0/886359612 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:32:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c7259c640 1 --2- 172.21.15.43:0/886359612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21ad11b0 0x55ba21e85ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c7259c640 1 --2- 172.21.15.43:0/886359612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21aa82f0 0x55ba21e85030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c72d9d640 1 -- 172.21.15.43:0/886359612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21aa82f0 msgr2=0x55ba21e85030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c72d9d640 1 --2- 172.21.15.43:0/886359612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21aa82f0 0x55ba21e85030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c72d9d640 1 -- 172.21.15.43:0/886359612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21ad11b0 msgr2=0x55ba21e85ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c72d9d640 1 --2- 172.21.15.43:0/886359612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21ad11b0 0x55ba21e85ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c72d9d640 1 -- 172.21.15.43:0/886359612 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba21ef4b20 con 0x55ba21ab04d0 2024-04-25T10:32:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c7259c640 1 --2- 172.21.15.43:0/886359612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21aa82f0 0x55ba21e85030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c72d9d640 1 --2- 172.21.15.43:0/886359612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ab04d0 0x55ba21e85570 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009d40 tx=0x7f4c64005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f71b2e8456b129d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:32:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c56ff5640 1 -- 172.21.15.43:0/886359612 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64036070 con 0x55ba21ab04d0 2024-04-25T10:32:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.375+0000 7f4c56ff5640 1 -- 172.21.15.43:0/886359612 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f4c64003ec0 con 0x55ba21ab04d0 2024-04-25T10:32:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.376+0000 7f4c8000e740 1 -- 172.21.15.43:0/886359612 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c64004670 con 0x55ba21ab04d0 2024-04-25T10:32:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.376+0000 7f4c56ff5640 1 -- 172.21.15.43:0/886359612 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4c6402d070 con 0x55ba21ab04d0 2024-04-25T10:32:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.376+0000 7f4c8000e740 10 client.15558 fetch_fsmap learned FSMap version 24 2024-04-25T10:32:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.376+0000 7f4c8000e740 1 -- 172.21.15.43:0/886359612 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ba21ef5c20 con 0x55ba21ab04d0 2024-04-25T10:32:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.376+0000 7f4c56ff5640 1 -- 172.21.15.43:0/886359612 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c64004670 con 0x55ba21ab04d0 2024-04-25T10:32:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.376+0000 7f4c56ff5640 1 -- 172.21.15.43:0/886359612 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4c64029ce0 con 0x55ba21ab04d0 2024-04-25T10:32:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.376+0000 7f4c8000e740 10 client.15558 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:32:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.376+0000 7f4c8000e740 10 client.15558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:32:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.376+0000 7f4c8000e740 20 client.15558 populate_metadata read hostname 'smithi043' 2024-04-25T10:32:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.376+0000 7f4c8000e740 1 --2- 172.21.15.43:0/886359612 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ba21a651c0 0x55ba21ef82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.376+0000 7f4c8000e740 4 client.15558 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:32:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.376+0000 7f4c8000e740 1 -- 172.21.15.43:0/886359612 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba21ddeee0 con 0x55ba21a651c0 2024-04-25T10:32:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.376+0000 7f4c7259c640 1 --2- 172.21.15.43:0/886359612 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ba21a651c0 0x55ba21ef82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.377+0000 7f4c7259c640 1 --2- 172.21.15.43:0/886359612 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ba21a651c0 0x55ba21ef82c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98ebe3e6e9d17e2d server_cookie=8df8934f3d55b6 in_seq=0 out_seq=0 2024-04-25T10:32:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.377+0000 7f4c56ff5640 10 client.15558 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:32:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.389+0000 7f4c56ff5640 1 -- 172.21.15.43:0/886359612 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba21ddeee0 con 0x55ba21a651c0 2024-04-25T10:32:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.389+0000 7f4c56ff5640 10 client.15558 handle_command_reply: tid=0 2024-04-25T10:32:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.389+0000 7f4c8000e740 1 client.15558 shutdown 2024-04-25T10:32:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.389+0000 7f4c8000e740 1 -- 172.21.15.43:0/886359612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ab04d0 msgr2=0x55ba21e85570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.389+0000 7f4c8000e740 1 --2- 172.21.15.43:0/886359612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ab04d0 0x55ba21e85570 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009d40 tx=0x7f4c64005fa0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.390+0000 7f4c8000e740 1 -- 172.21.15.43:0/886359612 shutdown_connections 2024-04-25T10:32:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.390+0000 7f4c8000e740 1 --2- 172.21.15.43:0/886359612 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ba21a651c0 0x55ba21ef82c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.390+0000 7f4c8000e740 1 --2- 172.21.15.43:0/886359612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba21ad11b0 0x55ba21e85ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.390+0000 7f4c56ff5640 0 client.15558 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:32:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.390+0000 7f4c8000e740 1 --2- 172.21.15.43:0/886359612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba21ab04d0 0x55ba21e85570 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.390+0000 7f4c8000e740 1 --2- 172.21.15.43:0/886359612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba21aa82f0 0x55ba21e85030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.390+0000 7f4c8000e740 1 -- 172.21.15.43:0/886359612 >> 172.21.15.43:0/886359612 conn(0x55ba21ac5300 msgr2=0x55ba21e79870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.390+0000 7f4c8000e740 1 -- 172.21.15.43:0/886359612 shutdown_connections 2024-04-25T10:32:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.390+0000 7f4c8000e740 1 -- 172.21.15.43:0/886359612 wait complete. 2024-04-25T10:32:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.390+0000 7f4c8000e740 20 client.15558 trim_cache size 0 max 16384 2024-04-25T10:32:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:32:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:32:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:32:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:32:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.390+0000 7f4c7915e640 1 -- 172.21.15.43:0/2200255204 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f4c5c0afd70 msgr2=0x7f4c5c0b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.390+0000 7f4c7915e640 1 --2- 172.21.15.43:0/2200255204 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f4c5c0afd70 0x7f4c5c0b2280 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0fcb90 tx=0x7f4c6c096130 comp rx=0 tx=0).stop 2024-04-25T10:32:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.390+0000 7f4c7915e640 1 -- 172.21.15.43:0/2200255204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c74108f00 msgr2=0x7f4c7419c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.390+0000 7f4c7915e640 1 --2- 172.21.15.43:0/2200255204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c74108f00 0x7f4c7419c7f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c004770 tx=0x7f4c6c004040 comp rx=0 tx=0).stop 2024-04-25T10:32:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.390+0000 7f4c7915e640 1 -- 172.21.15.43:0/2200255204 shutdown_connections 2024-04-25T10:32:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.390+0000 7f4c7915e640 1 --2- 172.21.15.43:0/2200255204 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f4c5c0afd70 0x7f4c5c0b2280 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.390+0000 7f4c7915e640 1 --2- 172.21.15.43:0/2200255204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c74108f00 0x7f4c7419c7f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.390+0000 7f4c7915e640 1 --2- 172.21.15.43:0/2200255204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c74108a60 0x7f4c741a2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.390+0000 7f4c7915e640 1 --2- 172.21.15.43:0/2200255204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c741031c0 0x7f4c741a25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.390+0000 7f4c7915e640 1 -- 172.21.15.43:0/2200255204 >> 172.21.15.43:0/2200255204 conn(0x7f4c740781b0 msgr2=0x7f4c74101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.391+0000 7f4c7915e640 1 -- 172.21.15.43:0/2200255204 shutdown_connections 2024-04-25T10:32:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:32:55.391+0000 7f4c7915e640 1 -- 172.21.15.43:0/2200255204 wait complete. 2024-04-25T10:32:56.163 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:55 smithi043 ceph-mon[29478]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 77 KiB/s wr, 20 op/s 2024-04-25T10:32:56.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:55 smithi164 ceph-mon[32713]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 77 KiB/s wr, 20 op/s 2024-04-25T10:32:56.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:55 smithi077 ceph-mon[32772]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 77 KiB/s wr, 20 op/s 2024-04-25T10:32:58.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:57 smithi077 ceph-mon[32772]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:32:58.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:57 smithi164 ceph-mon[32713]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:32:58.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:57 smithi043 ceph-mon[29478]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:33:00.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:32:59 smithi077 ceph-mon[32772]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T10:33:00.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:32:59 smithi164 ceph-mon[32713]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T10:33:00.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:32:59 smithi043 ceph-mon[29478]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T10:33:00.860 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:33:00 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:33:00.411+0000 7fa160ac4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:33:00.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:33:00 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:33:00.411+0000 7fa160ac4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:33:00.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:33:00 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:33:00.411+0000 7fa160ac4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:33:00.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:33:00 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:33:00.411+0000 7fa160ac4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:33:00.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:33:00 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x[29708]: 2024-04-25T10:33:00.411+0000 7fa160ac4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T10:33:02.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:01 smithi077 ceph-mon[32772]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-25T10:33:02.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:01 smithi077 ceph-mon[32772]: mgrmap e25: x(active, since 28s), standbys: z, y 2024-04-25T10:33:02.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:01 smithi164 ceph-mon[32713]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-25T10:33:02.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:01 smithi164 ceph-mon[32713]: mgrmap e25: x(active, since 28s), standbys: z, y 2024-04-25T10:33:02.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:01 smithi043 ceph-mon[29478]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-25T10:33:02.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:01 smithi043 ceph-mon[29478]: mgrmap e25: x(active, since 28s), standbys: z, y 2024-04-25T10:33:04.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:03 smithi077 ceph-mon[32772]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T10:33:04.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:03 smithi164 ceph-mon[32713]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T10:33:04.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:03 smithi043 ceph-mon[29478]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T10:33:06.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:05 smithi077 ceph-mon[32772]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-25T10:33:06.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:05 smithi164 ceph-mon[32713]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-25T10:33:06.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:05 smithi043 ceph-mon[29478]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-25T10:33:08.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:07 smithi077 ceph-mon[32772]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 25 KiB/s wr, 14 op/s 2024-04-25T10:33:08.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:07 smithi164 ceph-mon[32713]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 25 KiB/s wr, 14 op/s 2024-04-25T10:33:08.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:07 smithi043 ceph-mon[29478]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 25 KiB/s wr, 14 op/s 2024-04-25T10:33:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.509+0000 7f3f12653640 1 -- 172.21.15.43:0/4158444039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f0c110410 msgr2=0x7f3f0c112830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.509+0000 7f3f12653640 1 --2- 172.21.15.43:0/4158444039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f0c110410 0x7f3f0c112830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009ca0 tx=0x7f3f0402a950 comp rx=0 tx=0).stop 2024-04-25T10:33:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.509+0000 7f3f12653640 1 -- 172.21.15.43:0/4158444039 shutdown_connections 2024-04-25T10:33:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.509+0000 7f3f12653640 1 --2- 172.21.15.43:0/4158444039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f0c110410 0x7f3f0c112830 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.509+0000 7f3f12653640 1 --2- 172.21.15.43:0/4158444039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f0c10dab0 0x7f3f0c10fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.509+0000 7f3f12653640 1 --2- 172.21.15.43:0/4158444039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f0c0752e0 0x7f3f0c0756f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.509+0000 7f3f12653640 1 -- 172.21.15.43:0/4158444039 >> 172.21.15.43:0/4158444039 conn(0x7f3f0c0792a0 msgr2=0x7f3f0c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.509+0000 7f3f12653640 1 -- 172.21.15.43:0/4158444039 shutdown_connections 2024-04-25T10:33:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.509+0000 7f3f12653640 1 -- 172.21.15.43:0/4158444039 wait complete. 2024-04-25T10:33:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.510+0000 7f3f12653640 1 Processor -- start 2024-04-25T10:33:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.510+0000 7f3f12653640 1 -- start start 2024-04-25T10:33:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.510+0000 7f3f12653640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f0c0752e0 0x7f3f0c1a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.510+0000 7f3f12653640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f0c10dab0 0x7f3f0c1a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.510+0000 7f3f12653640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f0c110410 0x7f3f0c19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.510+0000 7f3f12653640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0c115f70 con 0x7f3f0c0752e0 2024-04-25T10:33:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.510+0000 7f3f12653640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0c115df0 con 0x7f3f0c110410 2024-04-25T10:33:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.510+0000 7f3f12653640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0c1160f0 con 0x7f3f0c10dab0 2024-04-25T10:33:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.510+0000 7f3f0bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f0c0752e0 0x7f3f0c1a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.511+0000 7f3f0bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f0c0752e0 0x7f3f0c1a25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39158/0 (socket says 172.21.15.43:39158) 2024-04-25T10:33:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.511+0000 7f3f0bfff640 1 -- 172.21.15.43:0/1924551290 learned_addr learned my addr 172.21.15.43:0/1924551290 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.511+0000 7f3f0b7fe640 1 --2- 172.21.15.43:0/1924551290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f0c10dab0 0x7f3f0c1a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.511+0000 7f3f10bf0640 1 --2- 172.21.15.43:0/1924551290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f0c110410 0x7f3f0c19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.511+0000 7f3f0bfff640 1 -- 172.21.15.43:0/1924551290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f0c10dab0 msgr2=0x7f3f0c1a2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.511+0000 7f3f0bfff640 1 --2- 172.21.15.43:0/1924551290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f0c10dab0 0x7f3f0c1a2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.511+0000 7f3f0bfff640 1 -- 172.21.15.43:0/1924551290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f0c110410 msgr2=0x7f3f0c19c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.511+0000 7f3f0bfff640 1 --2- 172.21.15.43:0/1924551290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f0c110410 0x7f3f0c19c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.511+0000 7f3f0bfff640 1 -- 172.21.15.43:0/1924551290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f0c19cea0 con 0x7f3f0c0752e0 2024-04-25T10:33:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.511+0000 7f3f10bf0640 1 --2- 172.21.15.43:0/1924551290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f0c110410 0x7f3f0c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.511+0000 7f3f0bfff640 1 --2- 172.21.15.43:0/1924551290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f0c0752e0 0x7f3f0c1a25f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400e9f0 tx=0x7f3ef400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.511+0000 7f3f097fa640 1 -- 172.21.15.43:0/1924551290 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef400cd80 con 0x7f3f0c0752e0 2024-04-25T10:33:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.511+0000 7f3f12653640 1 -- 172.21.15.43:0/1924551290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f0c19d130 con 0x7f3f0c0752e0 2024-04-25T10:33:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.511+0000 7f3f097fa640 1 -- 172.21.15.43:0/1924551290 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef40054d0 con 0x7f3f0c0752e0 2024-04-25T10:33:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.511+0000 7f3f097fa640 1 -- 172.21.15.43:0/1924551290 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4010690 con 0x7f3f0c0752e0 2024-04-25T10:33:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.511+0000 7f3f12653640 1 -- 172.21.15.43:0/1924551290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f0c1a9390 con 0x7f3f0c0752e0 2024-04-25T10:33:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.515+0000 7f3f195e0740 1 Processor -- start 2024-04-25T10:33:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.515+0000 7f3f195e0740 1 -- start start 2024-04-25T10:33:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.515+0000 7f3f097fa640 1 -- 172.21.15.43:0/1924551290 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 147804+0+0 (secure 0 0 0) 0x7f3ef4010850 con 0x7f3f0c0752e0 2024-04-25T10:33:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.515+0000 7f3f195e0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f024071b0 0x557f027a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.515+0000 7f3f195e0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f023de2f0 0x557f027a7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.515+0000 7f3f195e0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023e64d0 0x557f027a7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.515+0000 7f3f195e0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557f0270d460 con 0x557f024071b0 2024-04-25T10:33:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.515+0000 7f3f195e0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557f026c7590 con 0x557f023e64d0 2024-04-25T10:33:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.515+0000 7f3f195e0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557f02709420 con 0x557f023de2f0 2024-04-25T10:33:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.515+0000 7f3f10bf0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f024071b0 0x557f027a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.515+0000 7f3f10bf0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f024071b0 0x557f027a6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39170/0 (socket says 172.21.15.43:39170) 2024-04-25T10:33:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.515+0000 7f3f10bf0640 1 -- 172.21.15.43:0/4103630291 learned_addr learned my addr 172.21.15.43:0/4103630291 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.515+0000 7f3f0bfff640 1 --2- 172.21.15.43:0/4103630291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f023de2f0 0x557f027a7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.515+0000 7f3f0b7fe640 1 --2- 172.21.15.43:0/4103630291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023e64d0 0x557f027a7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.516+0000 7f3f097fa640 1 --2- 172.21.15.43:0/1924551290 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3ef80afe00 0x7f3ef80b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.516+0000 7f3f10bf0640 1 -- 172.21.15.43:0/4103630291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f023de2f0 msgr2=0x557f027a7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.516+0000 7f3f10bf0640 1 --2- 172.21.15.43:0/4103630291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f023de2f0 0x557f027a7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.516+0000 7f3f10bf0640 1 -- 172.21.15.43:0/4103630291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023e64d0 msgr2=0x557f027a7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.516+0000 7f3f10bf0640 1 --2- 172.21.15.43:0/4103630291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023e64d0 0x557f027a7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.516+0000 7f3f10bf0640 1 -- 172.21.15.43:0/4103630291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f026d40d0 con 0x557f024071b0 2024-04-25T10:33:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.516+0000 7f3f0bfff640 1 --2- 172.21.15.43:0/4103630291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f023de2f0 0x557f027a7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.516+0000 7f3f097fa640 1 -- 172.21.15.43:0/1924551290 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f3ef40df7f0 con 0x7f3f0c0752e0 2024-04-25T10:33:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.516+0000 7f3f10bf0640 1 --2- 172.21.15.43:0/4103630291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f024071b0 0x557f027a6af0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009a50 tx=0x7f3f0402fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2374bd17fc82989b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.516+0000 7f3f10bf0640 1 --2- 172.21.15.43:0/1924551290 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3ef80afe00 0x7f3ef80b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.516+0000 7f3f027fc640 1 -- 172.21.15.43:0/4103630291 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04049040 con 0x557f024071b0 2024-04-25T10:33:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.516+0000 7f3f027fc640 1 -- 172.21.15.43:0/4103630291 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f0402f400 con 0x557f024071b0 2024-04-25T10:33:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.516+0000 7f3f027fc640 1 -- 172.21.15.43:0/4103630291 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04034040 con 0x557f024071b0 2024-04-25T10:33:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.517+0000 7f3f195e0740 1 -- 172.21.15.43:0/4103630291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f024071b0 msgr2=0x557f027a6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.517+0000 7f3f10bf0640 1 --2- 172.21.15.43:0/1924551290 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3ef80afe00 0x7f3ef80b2310 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3f04044000 tx=0x7f3f04044500 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.517+0000 7f3f195e0740 1 --2- 172.21.15.43:0/4103630291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f024071b0 0x557f027a6af0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009a50 tx=0x7f3f0402fd40 comp rx=0 tx=0).stop 2024-04-25T10:33:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.517+0000 7f3f195e0740 1 -- 172.21.15.43:0/4103630291 shutdown_connections 2024-04-25T10:33:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.517+0000 7f3f195e0740 1 --2- 172.21.15.43:0/4103630291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023e64d0 0x557f027a7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.517+0000 7f3f195e0740 1 --2- 172.21.15.43:0/4103630291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f023de2f0 0x557f027a7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.517+0000 7f3f195e0740 1 --2- 172.21.15.43:0/4103630291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f024071b0 0x557f027a6af0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.517+0000 7f3f195e0740 1 -- 172.21.15.43:0/4103630291 >> 172.21.15.43:0/4103630291 conn(0x557f023fb300 msgr2=0x557f027a32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.517+0000 7f3f195e0740 1 -- 172.21.15.43:0/4103630291 shutdown_connections 2024-04-25T10:33:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.517+0000 7f3f195e0740 1 -- 172.21.15.43:0/4103630291 wait complete. 2024-04-25T10:33:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.518+0000 7f3f195e0740 1 -- start start 2024-04-25T10:33:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.518+0000 7f3f195e0740 1 Processor -- start 2024-04-25T10:33:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.518+0000 7f3f195e0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023de2f0 0x557f027b1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.518+0000 7f3f195e0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 0x557f027b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.518+0000 7f3f195e0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f024071b0 0x557f027b7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.518+0000 7f3f195e0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557f027b3bc0 con 0x557f023e64d0 2024-04-25T10:33:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.518+0000 7f3f195e0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557f02694b60 con 0x557f023de2f0 2024-04-25T10:33:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.518+0000 7f3f195e0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557f027b3d40 con 0x557f024071b0 2024-04-25T10:33:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.518+0000 7f3f195e0740 10 client.0 _ll_register_callbacks cb 0x557f026b92b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.519+0000 7f3f0b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 0x557f027b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.519+0000 7f3f0bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023de2f0 0x557f027b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.519+0000 7f3f0b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 0x557f027b16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39186/0 (socket says 172.21.15.43:39186) 2024-04-25T10:33:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.519+0000 7f3f0b7fe640 1 -- 172.21.15.43:0/4055940381 learned_addr learned my addr 172.21.15.43:0/4055940381 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.519+0000 7f3f0bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023de2f0 0x557f027b1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:57050/0 (socket says 172.21.15.43:57050) 2024-04-25T10:33:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.519+0000 7f3f10bf0640 1 --2- 172.21.15.43:0/4055940381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f024071b0 0x557f027b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.519+0000 7f3f0b7fe640 1 -- 172.21.15.43:0/4055940381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f024071b0 msgr2=0x557f027b7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.519+0000 7f3f0b7fe640 1 --2- 172.21.15.43:0/4055940381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f024071b0 0x557f027b7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.519+0000 7f3f0b7fe640 1 -- 172.21.15.43:0/4055940381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023de2f0 msgr2=0x557f027b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.519+0000 7f3f0b7fe640 1 --2- 172.21.15.43:0/4055940381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023de2f0 0x557f027b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.519+0000 7f3f0b7fe640 1 -- 172.21.15.43:0/4055940381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f0241f210 con 0x557f023e64d0 2024-04-25T10:33:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.519+0000 7f3f0bfff640 1 --2- 172.21.15.43:0/4055940381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023de2f0 0x557f027b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.519+0000 7f3f0b7fe640 1 --2- 172.21.15.43:0/4055940381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 0x557f027b16c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3efc012700 tx=0x7f3efc012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a52d13f1d08c40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.519+0000 7f3eeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.519+0000 7f3eeb7fe640 1 -- 172.21.15.43:0/4055940381 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3efc00f920 con 0x557f023e64d0 2024-04-25T10:33:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.519+0000 7f3eeb7fe640 1 -- 172.21.15.43:0/4055940381 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f3efc00fac0 con 0x557f023e64d0 2024-04-25T10:33:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.519+0000 7f3f195e0740 1 -- 172.21.15.43:0/4055940381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f0402f400 con 0x557f023e64d0 2024-04-25T10:33:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.519+0000 7f3eeb7fe640 1 -- 172.21.15.43:0/4055940381 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3efc00fcc0 con 0x557f023e64d0 2024-04-25T10:33:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.520+0000 7f3f195e0740 10 client.15576 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.520+0000 7f3f195e0740 1 -- 172.21.15.43:0/4055940381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557f024eb620 con 0x557f023e64d0 2024-04-25T10:33:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.520+0000 7f3eeb7fe640 1 -- 172.21.15.43:0/4055940381 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f0402f400 con 0x557f023e64d0 2024-04-25T10:33:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.520+0000 7f3eeb7fe640 1 -- 172.21.15.43:0/4055940381 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3efc01c420 con 0x557f023e64d0 2024-04-25T10:33:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.520+0000 7f3f195e0740 10 client.15576 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.520+0000 7f3f195e0740 10 client.15576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.520+0000 7f3f195e0740 20 client.15576 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.520+0000 7f3f195e0740 1 --2- 172.21.15.43:0/4055940381 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557f023c5d80 0x557f027c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.520+0000 7f3f195e0740 4 client.15576 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:33:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.520+0000 7f3f195e0740 1 -- 172.21.15.43:0/4055940381 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f0271b070 con 0x557f023c5d80 2024-04-25T10:33:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.520+0000 7f3f0bfff640 1 --2- 172.21.15.43:0/4055940381 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557f023c5d80 0x557f027c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.521+0000 7f3eeb7fe640 10 client.15576 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.521+0000 7f3f0bfff640 1 --2- 172.21.15.43:0/4055940381 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557f023c5d80 0x557f027c3710 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f52f1de2fbed9a40 server_cookie=74f251994dcee313 in_seq=0 out_seq=0 2024-04-25T10:33:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.522+0000 7f3eeb7fe640 1 -- 172.21.15.43:0/4055940381 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557f0271b070 con 0x557f023c5d80 2024-04-25T10:33:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.522+0000 7f3eeb7fe640 10 client.15576 handle_command_reply: tid=0 2024-04-25T10:33:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.522+0000 7f3f195e0740 1 client.15576 shutdown 2024-04-25T10:33:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.522+0000 7f3f195e0740 1 -- 172.21.15.43:0/4055940381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 msgr2=0x557f027b16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.522+0000 7f3f195e0740 1 --2- 172.21.15.43:0/4055940381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 0x557f027b16c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3efc012700 tx=0x7f3efc012bc0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.522+0000 7f3f195e0740 1 -- 172.21.15.43:0/4055940381 shutdown_connections 2024-04-25T10:33:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.522+0000 7f3f195e0740 1 --2- 172.21.15.43:0/4055940381 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557f023c5d80 0x557f027c3710 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.522+0000 7f3f195e0740 1 --2- 172.21.15.43:0/4055940381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f024071b0 0x557f027b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.522+0000 7f3f195e0740 1 --2- 172.21.15.43:0/4055940381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 0x557f027b16c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.522+0000 7f3f195e0740 1 --2- 172.21.15.43:0/4055940381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023de2f0 0x557f027b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.522+0000 7f3eeb7fe640 0 client.15576 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.522+0000 7f3f195e0740 1 -- 172.21.15.43:0/4055940381 >> 172.21.15.43:0/4055940381 conn(0x557f023fb300 msgr2=0x557f027a6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.522+0000 7f3f195e0740 1 -- 172.21.15.43:0/4055940381 shutdown_connections 2024-04-25T10:33:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.522+0000 7f3f195e0740 1 -- 172.21.15.43:0/4055940381 wait complete. 2024-04-25T10:33:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.522+0000 7f3f195e0740 20 client.15576 trim_cache size 0 max 16384 2024-04-25T10:33:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.537+0000 7f3f195e0740 1 Processor -- start 2024-04-25T10:33:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.537+0000 7f3f195e0740 1 -- start start 2024-04-25T10:33:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f195e0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023c5d80 0x557f027c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f195e0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f023de2f0 0x557f027c9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f195e0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 0x557f027a6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f195e0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0c1b4490 con 0x557f023e64d0 2024-04-25T10:33:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f195e0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0c1b1d60 con 0x557f023c5d80 2024-04-25T10:33:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f195e0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0c1afbb0 con 0x557f023de2f0 2024-04-25T10:33:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f10bf0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 0x557f027a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f10bf0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 0x557f027a6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39198/0 (socket says 172.21.15.43:39198) 2024-04-25T10:33:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f10bf0640 1 -- 172.21.15.43:0/214992261 learned_addr learned my addr 172.21.15.43:0/214992261 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f0bfff640 1 --2- 172.21.15.43:0/214992261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023c5d80 0x557f027c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f10bf0640 1 -- 172.21.15.43:0/214992261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f023de2f0 msgr2=0x557f027c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f0b7fe640 1 --2- 172.21.15.43:0/214992261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f023de2f0 0x557f027c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f10bf0640 1 --2- 172.21.15.43:0/214992261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f023de2f0 0x557f027c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f10bf0640 1 -- 172.21.15.43:0/214992261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023c5d80 msgr2=0x557f027c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f10bf0640 1 --2- 172.21.15.43:0/214992261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023c5d80 0x557f027c8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f10bf0640 1 -- 172.21.15.43:0/214992261 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f027b3810 con 0x557f023e64d0 2024-04-25T10:33:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f0bfff640 1 --2- 172.21.15.43:0/214992261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023c5d80 0x557f027c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f0b7fe640 1 --2- 172.21.15.43:0/214992261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f023de2f0 0x557f027c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f10bf0640 1 --2- 172.21.15.43:0/214992261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 0x557f027a6880 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3f04005e30 tx=0x7f3f04050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f1d2e55ed26cf05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f01ffb640 1 -- 172.21.15.43:0/214992261 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0405f070 con 0x557f023e64d0 2024-04-25T10:33:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.538+0000 7f3f01ffb640 1 -- 172.21.15.43:0/214992261 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f04044d50 con 0x557f023e64d0 2024-04-25T10:33:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.539+0000 7f3f01ffb640 1 -- 172.21.15.43:0/214992261 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04056040 con 0x557f023e64d0 2024-04-25T10:33:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.539+0000 7f3f195e0740 1 -- 172.21.15.43:0/214992261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 msgr2=0x557f027a6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.539+0000 7f3f195e0740 1 --2- 172.21.15.43:0/214992261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 0x557f027a6880 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3f04005e30 tx=0x7f3f04050000 comp rx=0 tx=0).stop 2024-04-25T10:33:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.539+0000 7f3f195e0740 1 -- 172.21.15.43:0/214992261 shutdown_connections 2024-04-25T10:33:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.539+0000 7f3f195e0740 1 --2- 172.21.15.43:0/214992261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 0x557f027a6880 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.539+0000 7f3f195e0740 1 --2- 172.21.15.43:0/214992261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f023de2f0 0x557f027c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.539+0000 7f3f195e0740 1 --2- 172.21.15.43:0/214992261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023c5d80 0x557f027c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.539+0000 7f3f195e0740 1 -- 172.21.15.43:0/214992261 >> 172.21.15.43:0/214992261 conn(0x557f023fb300 msgr2=0x557f027af840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.539+0000 7f3f195e0740 1 -- 172.21.15.43:0/214992261 shutdown_connections 2024-04-25T10:33:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.539+0000 7f3f195e0740 1 -- 172.21.15.43:0/214992261 wait complete. 2024-04-25T10:33:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.540+0000 7f3f195e0740 1 -- start start 2024-04-25T10:33:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.540+0000 7f3f195e0740 1 Processor -- start 2024-04-25T10:33:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.540+0000 7f3f195e0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023c5d80 0x557f027bb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.540+0000 7f3f195e0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f023de2f0 0x557f027bb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.540+0000 7f3f195e0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 0x557f027bbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.540+0000 7f3f195e0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0c1b0220 con 0x557f023e64d0 2024-04-25T10:33:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.540+0000 7f3f195e0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0c1b1690 con 0x557f023c5d80 2024-04-25T10:33:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.540+0000 7f3f195e0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0c1b08f0 con 0x557f023de2f0 2024-04-25T10:33:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.540+0000 7f3f195e0740 10 client.0 _ll_register_callbacks cb 0x7f3efc009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.540+0000 7f3f10bf0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 0x557f027bbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.540+0000 7f3f10bf0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 0x557f027bbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39204/0 (socket says 172.21.15.43:39204) 2024-04-25T10:33:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.540+0000 7f3f10bf0640 1 -- 172.21.15.43:0/2332790989 learned_addr learned my addr 172.21.15.43:0/2332790989 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.540+0000 7f3f0bfff640 1 --2- 172.21.15.43:0/2332790989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023c5d80 0x557f027bb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.541+0000 7f3f0b7fe640 1 --2- 172.21.15.43:0/2332790989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f023de2f0 0x557f027bb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.541+0000 7f3f10bf0640 1 -- 172.21.15.43:0/2332790989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f023de2f0 msgr2=0x557f027bb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.541+0000 7f3f10bf0640 1 --2- 172.21.15.43:0/2332790989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f023de2f0 0x557f027bb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.541+0000 7f3f10bf0640 1 -- 172.21.15.43:0/2332790989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023c5d80 msgr2=0x557f027bb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.541+0000 7f3f10bf0640 1 --2- 172.21.15.43:0/2332790989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023c5d80 0x557f027bb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.541+0000 7f3f10bf0640 1 -- 172.21.15.43:0/2332790989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f0282aad0 con 0x557f023e64d0 2024-04-25T10:33:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.541+0000 7f3f0b7fe640 1 --2- 172.21.15.43:0/2332790989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f023de2f0 0x557f027bb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.541+0000 7f3f10bf0640 1 --2- 172.21.15.43:0/2332790989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 0x557f027bbad0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3f0402ae70 tx=0x7f3f0402f760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3269ece8c705825 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.541+0000 7f3eeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.541+0000 7f3eeb7fe640 1 -- 172.21.15.43:0/2332790989 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0405f070 con 0x557f023e64d0 2024-04-25T10:33:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.541+0000 7f3eeb7fe640 1 -- 172.21.15.43:0/2332790989 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f3f04003970 con 0x557f023e64d0 2024-04-25T10:33:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.541+0000 7f3f195e0740 1 -- 172.21.15.43:0/2332790989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f04044d50 con 0x557f023e64d0 2024-04-25T10:33:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.541+0000 7f3eeb7fe640 1 -- 172.21.15.43:0/2332790989 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3f04034040 con 0x557f023e64d0 2024-04-25T10:33:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.541+0000 7f3f195e0740 10 client.15582 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.541+0000 7f3f195e0740 1 -- 172.21.15.43:0/2332790989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557f0282bbf0 con 0x557f023e64d0 2024-04-25T10:33:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.541+0000 7f3eeb7fe640 1 -- 172.21.15.43:0/2332790989 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f04044d50 con 0x557f023e64d0 2024-04-25T10:33:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.542+0000 7f3eeb7fe640 1 -- 172.21.15.43:0/2332790989 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3f04059030 con 0x557f023e64d0 2024-04-25T10:33:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.542+0000 7f3f195e0740 10 client.15582 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.542+0000 7f3f195e0740 10 client.15582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.542+0000 7f3f195e0740 20 client.15582 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.542+0000 7f3f195e0740 1 --2- 172.21.15.43:0/2332790989 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557f024071b0 0x557f0282ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.542+0000 7f3f195e0740 4 client.15582 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:33:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.542+0000 7f3f195e0740 1 -- 172.21.15.43:0/2332790989 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f02714ee0 con 0x557f024071b0 2024-04-25T10:33:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.542+0000 7f3f0bfff640 1 --2- 172.21.15.43:0/2332790989 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557f024071b0 0x557f0282ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.542+0000 7f3f0bfff640 1 --2- 172.21.15.43:0/2332790989 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557f024071b0 0x557f0282ded0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c369ca719f2ff23e server_cookie=1f1d8acd1c91a979 in_seq=0 out_seq=0 2024-04-25T10:33:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.542+0000 7f3eeb7fe640 10 client.15582 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.551+0000 7f3eeb7fe640 1 -- 172.21.15.43:0/2332790989 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557f02714ee0 con 0x557f024071b0 2024-04-25T10:33:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.551+0000 7f3eeb7fe640 10 client.15582 handle_command_reply: tid=0 2024-04-25T10:33:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.551+0000 7f3f195e0740 1 client.15582 shutdown 2024-04-25T10:33:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.551+0000 7f3f195e0740 1 -- 172.21.15.43:0/2332790989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 msgr2=0x557f027bbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.551+0000 7f3f195e0740 1 --2- 172.21.15.43:0/2332790989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 0x557f027bbad0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3f0402ae70 tx=0x7f3f0402f760 comp rx=0 tx=0).stop 2024-04-25T10:33:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.551+0000 7f3f195e0740 1 -- 172.21.15.43:0/2332790989 shutdown_connections 2024-04-25T10:33:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.551+0000 7f3f195e0740 1 --2- 172.21.15.43:0/2332790989 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557f024071b0 0x557f0282ded0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.551+0000 7f3f195e0740 1 --2- 172.21.15.43:0/2332790989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f023e64d0 0x557f027bbad0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.551+0000 7f3f195e0740 1 --2- 172.21.15.43:0/2332790989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f023de2f0 0x557f027bb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.551+0000 7f3eeb7fe640 0 client.15582 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.551+0000 7f3f195e0740 1 --2- 172.21.15.43:0/2332790989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f023c5d80 0x557f027bb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.551+0000 7f3f195e0740 1 -- 172.21.15.43:0/2332790989 >> 172.21.15.43:0/2332790989 conn(0x557f023fb300 msgr2=0x557f027af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.551+0000 7f3f195e0740 1 -- 172.21.15.43:0/2332790989 shutdown_connections 2024-04-25T10:33:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.551+0000 7f3f195e0740 1 -- 172.21.15.43:0/2332790989 wait complete. 2024-04-25T10:33:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.551+0000 7f3f195e0740 20 client.15582 trim_cache size 0 max 16384 2024-04-25T10:33:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:33:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:33:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:33:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:33:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.552+0000 7f3f12653640 1 -- 172.21.15.43:0/1924551290 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3ef80afe00 msgr2=0x7f3ef80b2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.552+0000 7f3f12653640 1 --2- 172.21.15.43:0/1924551290 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3ef80afe00 0x7f3ef80b2310 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3f04044000 tx=0x7f3f04044500 comp rx=0 tx=0).stop 2024-04-25T10:33:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.552+0000 7f3f12653640 1 -- 172.21.15.43:0/1924551290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f0c0752e0 msgr2=0x7f3f0c1a25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.552+0000 7f3f12653640 1 --2- 172.21.15.43:0/1924551290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f0c0752e0 0x7f3f0c1a25f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400e9f0 tx=0x7f3ef400edf0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.552+0000 7f3f12653640 1 -- 172.21.15.43:0/1924551290 shutdown_connections 2024-04-25T10:33:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.552+0000 7f3f12653640 1 --2- 172.21.15.43:0/1924551290 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3ef80afe00 0x7f3ef80b2310 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.552+0000 7f3f12653640 1 --2- 172.21.15.43:0/1924551290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f0c110410 0x7f3f0c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.552+0000 7f3f12653640 1 --2- 172.21.15.43:0/1924551290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f0c10dab0 0x7f3f0c1a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.552+0000 7f3f12653640 1 --2- 172.21.15.43:0/1924551290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f0c0752e0 0x7f3f0c1a25f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.552+0000 7f3f12653640 1 -- 172.21.15.43:0/1924551290 >> 172.21.15.43:0/1924551290 conn(0x7f3f0c0792a0 msgr2=0x7f3f0c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.552+0000 7f3f12653640 1 -- 172.21.15.43:0/1924551290 shutdown_connections 2024-04-25T10:33:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:08.552+0000 7f3f12653640 1 -- 172.21.15.43:0/1924551290 wait complete. 2024-04-25T10:33:10.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:33:10.251 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:33:10.279 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:09 smithi043 ceph-mon[29478]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-04-25T10:33:10.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:09 smithi077 ceph-mon[32772]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-04-25T10:33:10.290 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:09 smithi164 ceph-mon[32713]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-04-25T10:33:10.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.380+0000 7fc1803b2640 1 -- 172.21.15.43:0/851513372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc178109f90 msgr2=0x7fc1781020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.380+0000 7fc1803b2640 1 --2- 172.21.15.43:0/851513372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc178109f90 0x7fc1781020b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc168009900 tx=0x7fc16802a410 comp rx=0 tx=0).stop 2024-04-25T10:33:10.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.380+0000 7fc1803b2640 1 -- 172.21.15.43:0/851513372 shutdown_connections 2024-04-25T10:33:10.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.380+0000 7fc1803b2640 1 --2- 172.21.15.43:0/851513372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc178102720 0x7fc178110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.380+0000 7fc1803b2640 1 --2- 172.21.15.43:0/851513372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc178109f90 0x7fc1781020b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.380+0000 7fc1803b2640 1 --2- 172.21.15.43:0/851513372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc178109af0 0x7fc178101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.380+0000 7fc1803b2640 1 -- 172.21.15.43:0/851513372 >> 172.21.15.43:0/851513372 conn(0x7fc178077880 msgr2=0x7fc178077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:10.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.380+0000 7fc1803b2640 1 -- 172.21.15.43:0/851513372 shutdown_connections 2024-04-25T10:33:10.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.380+0000 7fc1803b2640 1 -- 172.21.15.43:0/851513372 wait complete. 2024-04-25T10:33:10.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.381+0000 7fc1803b2640 1 Processor -- start 2024-04-25T10:33:10.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.381+0000 7fc1803b2640 1 -- start start 2024-04-25T10:33:10.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.381+0000 7fc1803b2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc178102720 0x7fc178102d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.381+0000 7fc1803b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc178109af0 0x7fc178103290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.381+0000 7fc1803b2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc178109f90 0x7fc1781037d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.381+0000 7fc1803b2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc178113ce0 con 0x7fc178109af0 2024-04-25T10:33:10.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.381+0000 7fc1803b2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc178113b60 con 0x7fc178109f90 2024-04-25T10:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.381+0000 7fc1803b2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc178113e60 con 0x7fc178102720 2024-04-25T10:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.381+0000 7fc17d94d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc178109af0 0x7fc178103290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.381+0000 7fc17d94d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc178109af0 0x7fc178103290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41808/0 (socket says 172.21.15.43:41808) 2024-04-25T10:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.382+0000 7fc17d94d640 1 -- 172.21.15.43:0/531257319 learned_addr learned my addr 172.21.15.43:0/531257319 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.382+0000 7fc17e94f640 1 --2- 172.21.15.43:0/531257319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc178109f90 0x7fc1781037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.382+0000 7fc17e14e640 1 --2- 172.21.15.43:0/531257319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc178102720 0x7fc178102d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.382+0000 7fc17d94d640 1 -- 172.21.15.43:0/531257319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc178102720 msgr2=0x7fc178102d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.382+0000 7fc17d94d640 1 --2- 172.21.15.43:0/531257319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc178102720 0x7fc178102d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.382+0000 7fc17d94d640 1 -- 172.21.15.43:0/531257319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc178109f90 msgr2=0x7fc1781037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.382+0000 7fc17d94d640 1 --2- 172.21.15.43:0/531257319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc178109f90 0x7fc1781037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.382+0000 7fc17d94d640 1 -- 172.21.15.43:0/531257319 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1781a8a40 con 0x7fc178109af0 2024-04-25T10:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.382+0000 7fc17e14e640 1 --2- 172.21.15.43:0/531257319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc178102720 0x7fc178102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.382+0000 7fc17d94d640 1 --2- 172.21.15.43:0/531257319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc178109af0 0x7fc178103290 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc168006010 tx=0x7fc168004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.382+0000 7fc16f7fe640 1 -- 172.21.15.43:0/531257319 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16803f030 con 0x7fc178109af0 2024-04-25T10:33:10.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.382+0000 7fc1803b2640 1 -- 172.21.15.43:0/531257319 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1781a8cd0 con 0x7fc178109af0 2024-04-25T10:33:10.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.382+0000 7fc16f7fe640 1 -- 172.21.15.43:0/531257319 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc168003aa0 con 0x7fc178109af0 2024-04-25T10:33:10.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.382+0000 7fc16f7fe640 1 -- 172.21.15.43:0/531257319 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc168035070 con 0x7fc178109af0 2024-04-25T10:33:10.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.382+0000 7fc1803b2640 1 -- 172.21.15.43:0/531257319 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1781a91c0 con 0x7fc178109af0 2024-04-25T10:33:10.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.385+0000 7fc16f7fe640 1 -- 172.21.15.43:0/531257319 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 147804+0+0 (secure 0 0 0) 0x7fc168042030 con 0x7fc178109af0 2024-04-25T10:33:10.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.385+0000 7fc187267740 1 Processor -- start 2024-04-25T10:33:10.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.386+0000 7fc187267740 1 -- start start 2024-04-25T10:33:10.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.386+0000 7fc187267740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559da0b80390 0x559da0f20a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.386+0000 7fc187267740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b574d0 0x559da0f20fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.386+0000 7fc187267740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b5f6b0 0x559da0f2d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.386+0000 7fc187267740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559da0e86d80 con 0x559da0b574d0 2024-04-25T10:33:10.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.386+0000 7fc187267740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559da0e408b0 con 0x559da0b5f6b0 2024-04-25T10:33:10.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.386+0000 7fc187267740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559da0e82d40 con 0x559da0b80390 2024-04-25T10:33:10.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.386+0000 7fc16f7fe640 1 --2- 172.21.15.43:0/531257319 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc1640afe10 0x7fc1640b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.386+0000 7fc17e14e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b574d0 0x559da0f20fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.386+0000 7fc17e14e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b574d0 0x559da0f20fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41810/0 (socket says 172.21.15.43:41810) 2024-04-25T10:33:10.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.386+0000 7fc17e14e640 1 -- 172.21.15.43:0/3928128497 learned_addr learned my addr 172.21.15.43:0/3928128497 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:10.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.386+0000 7fc17e94f640 1 --2- 172.21.15.43:0/531257319 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc1640afe10 0x7fc1640b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.386+0000 7fc17d94d640 1 --2- 172.21.15.43:0/3928128497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b5f6b0 0x559da0f2d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.386+0000 7fc17e94f640 1 --2- 172.21.15.43:0/3928128497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559da0b80390 0x559da0f20a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.386+0000 7fc16f7fe640 1 -- 172.21.15.43:0/531257319 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fc1680ff730 con 0x7fc178109af0 2024-04-25T10:33:10.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.386+0000 7fc17e14e640 1 -- 172.21.15.43:0/3928128497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559da0b80390 msgr2=0x559da0f20a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:10.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.386+0000 7fc17e14e640 1 --2- 172.21.15.43:0/3928128497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559da0b80390 0x559da0f20a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.386+0000 7fc17e14e640 1 -- 172.21.15.43:0/3928128497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b5f6b0 msgr2=0x559da0f2d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:10.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.386+0000 7fc17e14e640 1 --2- 172.21.15.43:0/3928128497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b5f6b0 0x559da0f2d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.386+0000 7fc17e14e640 1 -- 172.21.15.43:0/3928128497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559da0e11a40 con 0x559da0b574d0 2024-04-25T10:33:10.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.387+0000 7fc17e14e640 1 --2- 172.21.15.43:0/3928128497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b574d0 0x559da0f20fa0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc174009cd0 tx=0x7fc1740117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3df43067e17f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:10.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.387+0000 7fc17e94f640 1 --2- 172.21.15.43:0/531257319 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc1640afe10 0x7fc1640b2320 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x559da0f1dbd0 tx=0x7fc170009840 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:10.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.387+0000 7fc16cbfe640 1 -- 172.21.15.43:0/3928128497 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174012450 con 0x559da0b574d0 2024-04-25T10:33:10.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.387+0000 7fc16cbfe640 1 -- 172.21.15.43:0/3928128497 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc174012aa0 con 0x559da0b574d0 2024-04-25T10:33:10.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.387+0000 7fc16cbfe640 1 -- 172.21.15.43:0/3928128497 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17400fb00 con 0x559da0b574d0 2024-04-25T10:33:10.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.388+0000 7fc187267740 1 -- 172.21.15.43:0/3928128497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b574d0 msgr2=0x559da0f20fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.388+0000 7fc187267740 1 --2- 172.21.15.43:0/3928128497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b574d0 0x559da0f20fa0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc174009cd0 tx=0x7fc1740117b0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.388+0000 7fc187267740 1 -- 172.21.15.43:0/3928128497 shutdown_connections 2024-04-25T10:33:10.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.388+0000 7fc187267740 1 --2- 172.21.15.43:0/3928128497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b5f6b0 0x559da0f2d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.388+0000 7fc187267740 1 --2- 172.21.15.43:0/3928128497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b574d0 0x559da0f20fa0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.388+0000 7fc187267740 1 --2- 172.21.15.43:0/3928128497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559da0b80390 0x559da0f20a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.388+0000 7fc187267740 1 -- 172.21.15.43:0/3928128497 >> 172.21.15.43:0/3928128497 conn(0x559da0b744e0 msgr2=0x559da0f1d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:10.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.388+0000 7fc187267740 1 -- 172.21.15.43:0/3928128497 shutdown_connections 2024-04-25T10:33:10.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.388+0000 7fc187267740 1 -- 172.21.15.43:0/3928128497 wait complete. 2024-04-25T10:33:10.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.388+0000 7fc187267740 1 -- start start 2024-04-25T10:33:10.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.388+0000 7fc187267740 1 Processor -- start 2024-04-25T10:33:10.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.389+0000 7fc187267740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559da0b574d0 0x559da0f2b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.389+0000 7fc187267740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b5f6b0 0x559da0f2bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.389+0000 7fc187267740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b80390 0x559da0f31470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.389+0000 7fc187267740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559da0f2dfe0 con 0x559da0b80390 2024-04-25T10:33:10.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.389+0000 7fc187267740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559da0e0dd10 con 0x559da0b5f6b0 2024-04-25T10:33:10.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.389+0000 7fc187267740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559da0f2e160 con 0x559da0b574d0 2024-04-25T10:33:10.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.389+0000 7fc187267740 10 client.0 _ll_register_callbacks cb 0x559da0e3a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:10.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.389+0000 7fc17e94f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b80390 0x559da0f31470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.389+0000 7fc17e94f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b80390 0x559da0f31470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41824/0 (socket says 172.21.15.43:41824) 2024-04-25T10:33:10.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.389+0000 7fc17e94f640 1 -- 172.21.15.43:0/2716774646 learned_addr learned my addr 172.21.15.43:0/2716774646 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:10.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.389+0000 7fc17d94d640 1 --2- 172.21.15.43:0/2716774646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b5f6b0 0x559da0f2bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.389+0000 7fc17e14e640 1 --2- 172.21.15.43:0/2716774646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559da0b574d0 0x559da0f2b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.389+0000 7fc17e94f640 1 -- 172.21.15.43:0/2716774646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559da0b574d0 msgr2=0x559da0f2b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:10.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.389+0000 7fc17e94f640 1 --2- 172.21.15.43:0/2716774646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559da0b574d0 0x559da0f2b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.389+0000 7fc17e94f640 1 -- 172.21.15.43:0/2716774646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b5f6b0 msgr2=0x559da0f2bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:10.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.389+0000 7fc17e94f640 1 --2- 172.21.15.43:0/2716774646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b5f6b0 0x559da0f2bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.389+0000 7fc17e94f640 1 -- 172.21.15.43:0/2716774646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559da0c627d0 con 0x559da0b80390 2024-04-25T10:33:10.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.389+0000 7fc17e14e640 1 --2- 172.21.15.43:0/2716774646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559da0b574d0 0x559da0f2b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:10.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.389+0000 7fc17e94f640 1 --2- 172.21.15.43:0/2716774646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b80390 0x559da0f31470 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc170003bd0 tx=0x7fc170009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd65ba67cdcef01f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:10.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.389+0000 7fc161ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:10.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.390+0000 7fc161ffb640 1 -- 172.21.15.43:0/2716774646 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170004450 con 0x559da0b80390 2024-04-25T10:33:10.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.390+0000 7fc161ffb640 1 -- 172.21.15.43:0/2716774646 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fc1700045f0 con 0x559da0b80390 2024-04-25T10:33:10.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.390+0000 7fc187267740 1 -- 172.21.15.43:0/2716774646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc174012aa0 con 0x559da0b80390 2024-04-25T10:33:10.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.390+0000 7fc161ffb640 1 -- 172.21.15.43:0/2716774646 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc170010e70 con 0x559da0b80390 2024-04-25T10:33:10.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.390+0000 7fc187267740 10 client.15594 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:10.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.390+0000 7fc187267740 1 -- 172.21.15.43:0/2716774646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559da0b983f0 con 0x559da0b80390 2024-04-25T10:33:10.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.390+0000 7fc161ffb640 1 -- 172.21.15.43:0/2716774646 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc174012aa0 con 0x559da0b80390 2024-04-25T10:33:10.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.390+0000 7fc161ffb640 1 -- 172.21.15.43:0/2716774646 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fc17000f040 con 0x559da0b80390 2024-04-25T10:33:10.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.390+0000 7fc187267740 10 client.15594 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:10.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.390+0000 7fc187267740 10 client.15594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:10.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.390+0000 7fc187267740 20 client.15594 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:10.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.390+0000 7fc187267740 1 --2- 172.21.15.43:0/2716774646 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559da0b3ef60 0x559da0f3d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.390+0000 7fc187267740 4 client.15594 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:33:10.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.390+0000 7fc187267740 1 -- 172.21.15.43:0/2716774646 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559da0e94990 con 0x559da0b3ef60 2024-04-25T10:33:10.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.391+0000 7fc17e14e640 1 --2- 172.21.15.43:0/2716774646 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559da0b3ef60 0x559da0f3d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.391+0000 7fc161ffb640 10 client.15594 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:10.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.391+0000 7fc17e14e640 1 --2- 172.21.15.43:0/2716774646 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559da0b3ef60 0x559da0f3d730 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=321b6af603a983ce server_cookie=ca11def30e442b01 in_seq=0 out_seq=0 2024-04-25T10:33:10.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.392+0000 7fc161ffb640 1 -- 172.21.15.43:0/2716774646 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559da0e94990 con 0x559da0b3ef60 2024-04-25T10:33:10.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.392+0000 7fc161ffb640 10 client.15594 handle_command_reply: tid=0 2024-04-25T10:33:10.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.392+0000 7fc187267740 1 client.15594 shutdown 2024-04-25T10:33:10.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.392+0000 7fc187267740 1 -- 172.21.15.43:0/2716774646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b80390 msgr2=0x559da0f31470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.392+0000 7fc187267740 1 --2- 172.21.15.43:0/2716774646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b80390 0x559da0f31470 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc170003bd0 tx=0x7fc170009600 comp rx=0 tx=0).stop 2024-04-25T10:33:10.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.392+0000 7fc187267740 1 -- 172.21.15.43:0/2716774646 shutdown_connections 2024-04-25T10:33:10.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.392+0000 7fc187267740 1 --2- 172.21.15.43:0/2716774646 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559da0b3ef60 0x559da0f3d730 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.392+0000 7fc187267740 1 --2- 172.21.15.43:0/2716774646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b5f6b0 0x559da0f2bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.392+0000 7fc187267740 1 --2- 172.21.15.43:0/2716774646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b80390 0x559da0f31470 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.392+0000 7fc187267740 1 --2- 172.21.15.43:0/2716774646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559da0b574d0 0x559da0f2b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.392+0000 7fc161ffb640 0 client.15594 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:10.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.392+0000 7fc187267740 1 -- 172.21.15.43:0/2716774646 >> 172.21.15.43:0/2716774646 conn(0x559da0b744e0 msgr2=0x559da0f22780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:10.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.392+0000 7fc187267740 1 -- 172.21.15.43:0/2716774646 shutdown_connections 2024-04-25T10:33:10.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.392+0000 7fc187267740 1 -- 172.21.15.43:0/2716774646 wait complete. 2024-04-25T10:33:10.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.392+0000 7fc187267740 20 client.15594 trim_cache size 0 max 16384 2024-04-25T10:33:10.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.410+0000 7fc187267740 1 Processor -- start 2024-04-25T10:33:10.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.410+0000 7fc187267740 1 -- start start 2024-04-25T10:33:10.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.411+0000 7fc187267740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b574d0 0x559da0f20e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.411+0000 7fc187267740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b5f6b0 0x559da0f26790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.411+0000 7fc187267740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559da0b80390 0x559da0f2a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.411+0000 7fc187267740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781b4410 con 0x559da0b5f6b0 2024-04-25T10:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.411+0000 7fc187267740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781b1ce0 con 0x559da0b574d0 2024-04-25T10:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.411+0000 7fc187267740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781afb30 con 0x559da0b80390 2024-04-25T10:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.411+0000 7fc17d94d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b5f6b0 0x559da0f26790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.411+0000 7fc17d94d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b5f6b0 0x559da0f26790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41836/0 (socket says 172.21.15.43:41836) 2024-04-25T10:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.411+0000 7fc17d94d640 1 -- 172.21.15.43:0/452940166 learned_addr learned my addr 172.21.15.43:0/452940166 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.411+0000 7fc17e14e640 1 --2- 172.21.15.43:0/452940166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b574d0 0x559da0f20e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.411+0000 7fc17d94d640 1 -- 172.21.15.43:0/452940166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559da0b80390 msgr2=0x559da0f2a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.411+0000 7fc17d94d640 1 --2- 172.21.15.43:0/452940166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559da0b80390 0x559da0f2a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.411+0000 7fc17d94d640 1 -- 172.21.15.43:0/452940166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b574d0 msgr2=0x559da0f20e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.411+0000 7fc17d94d640 1 --2- 172.21.15.43:0/452940166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b574d0 0x559da0f20e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.411+0000 7fc17d94d640 1 -- 172.21.15.43:0/452940166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559da0b580f0 con 0x559da0b5f6b0 2024-04-25T10:33:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.411+0000 7fc17e14e640 1 --2- 172.21.15.43:0/452940166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b574d0 0x559da0f20e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.411+0000 7fc17d94d640 1 --2- 172.21.15.43:0/452940166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b5f6b0 0x559da0f26790 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc1680c3740 tx=0x7fc16803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7e2f001887378d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.411+0000 7fc163fff640 1 -- 172.21.15.43:0/452940166 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc168049040 con 0x559da0b5f6b0 2024-04-25T10:33:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.411+0000 7fc163fff640 1 -- 172.21.15.43:0/452940166 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1680c1e50 con 0x559da0b5f6b0 2024-04-25T10:33:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.412+0000 7fc163fff640 1 -- 172.21.15.43:0/452940166 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1680d0520 con 0x559da0b5f6b0 2024-04-25T10:33:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.412+0000 7fc187267740 1 -- 172.21.15.43:0/452940166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b5f6b0 msgr2=0x559da0f26790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.412+0000 7fc187267740 1 --2- 172.21.15.43:0/452940166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b5f6b0 0x559da0f26790 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc1680c3740 tx=0x7fc16803b000 comp rx=0 tx=0).stop 2024-04-25T10:33:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.412+0000 7fc187267740 1 -- 172.21.15.43:0/452940166 shutdown_connections 2024-04-25T10:33:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.412+0000 7fc187267740 1 --2- 172.21.15.43:0/452940166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559da0b80390 0x559da0f2a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.412+0000 7fc187267740 1 --2- 172.21.15.43:0/452940166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b5f6b0 0x559da0f26790 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.412+0000 7fc187267740 1 --2- 172.21.15.43:0/452940166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b574d0 0x559da0f20e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.412+0000 7fc187267740 1 -- 172.21.15.43:0/452940166 >> 172.21.15.43:0/452940166 conn(0x559da0b744e0 msgr2=0x559da0f29ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.412+0000 7fc187267740 1 -- 172.21.15.43:0/452940166 shutdown_connections 2024-04-25T10:33:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.412+0000 7fc187267740 1 -- 172.21.15.43:0/452940166 wait complete. 2024-04-25T10:33:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.413+0000 7fc187267740 1 -- start start 2024-04-25T10:33:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.413+0000 7fc187267740 1 Processor -- start 2024-04-25T10:33:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.413+0000 7fc187267740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559da0b574d0 0x559da0f37820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.413+0000 7fc187267740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b5f6b0 0x559da0f37d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.413+0000 7fc187267740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b80390 0x559da0f32be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.413+0000 7fc187267740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781b01a0 con 0x559da0b80390 2024-04-25T10:33:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.413+0000 7fc187267740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781b1610 con 0x559da0b5f6b0 2024-04-25T10:33:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.413+0000 7fc187267740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781b0870 con 0x559da0b574d0 2024-04-25T10:33:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.413+0000 7fc187267740 10 client.0 _ll_register_callbacks cb 0x7fc170012510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.413+0000 7fc17e94f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b80390 0x559da0f32be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.413+0000 7fc17e94f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b80390 0x559da0f32be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41842/0 (socket says 172.21.15.43:41842) 2024-04-25T10:33:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.413+0000 7fc17e94f640 1 -- 172.21.15.43:0/1061045934 learned_addr learned my addr 172.21.15.43:0/1061045934 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.413+0000 7fc17d94d640 1 --2- 172.21.15.43:0/1061045934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b5f6b0 0x559da0f37d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.413+0000 7fc17e14e640 1 --2- 172.21.15.43:0/1061045934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559da0b574d0 0x559da0f37820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.414+0000 7fc17e94f640 1 -- 172.21.15.43:0/1061045934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559da0b574d0 msgr2=0x559da0f37820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.414+0000 7fc17e94f640 1 --2- 172.21.15.43:0/1061045934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559da0b574d0 0x559da0f37820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.414+0000 7fc17e94f640 1 -- 172.21.15.43:0/1061045934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b5f6b0 msgr2=0x559da0f37d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.414+0000 7fc17e94f640 1 --2- 172.21.15.43:0/1061045934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b5f6b0 0x559da0f37d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.414+0000 7fc17e94f640 1 -- 172.21.15.43:0/1061045934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559da0e474b0 con 0x559da0b80390 2024-04-25T10:33:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.414+0000 7fc17e94f640 1 --2- 172.21.15.43:0/1061045934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b80390 0x559da0f32be0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc17001a040 tx=0x7fc170003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e847b72f577bc0fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.414+0000 7fc161ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.414+0000 7fc161ffb640 1 -- 172.21.15.43:0/1061045934 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17002abc0 con 0x559da0b80390 2024-04-25T10:33:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.414+0000 7fc161ffb640 1 -- 172.21.15.43:0/1061045934 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fc17002ad60 con 0x559da0b80390 2024-04-25T10:33:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.414+0000 7fc187267740 1 -- 172.21.15.43:0/1061045934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1680c1e50 con 0x559da0b80390 2024-04-25T10:33:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.414+0000 7fc161ffb640 1 -- 172.21.15.43:0/1061045934 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc170004b20 con 0x559da0b80390 2024-04-25T10:33:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.414+0000 7fc187267740 10 client.15600 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.414+0000 7fc187267740 1 -- 172.21.15.43:0/1061045934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559da0fa52a0 con 0x559da0b80390 2024-04-25T10:33:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.414+0000 7fc161ffb640 1 -- 172.21.15.43:0/1061045934 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1680c1e50 con 0x559da0b80390 2024-04-25T10:33:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.414+0000 7fc161ffb640 1 -- 172.21.15.43:0/1061045934 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fc170005220 con 0x559da0b80390 2024-04-25T10:33:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.415+0000 7fc187267740 10 client.15600 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.415+0000 7fc187267740 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.415+0000 7fc187267740 20 client.15600 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.415+0000 7fc187267740 1 --2- 172.21.15.43:0/1061045934 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559da0b143a0 0x559da0fa79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.415+0000 7fc187267740 4 client.15600 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:33:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.415+0000 7fc187267740 1 -- 172.21.15.43:0/1061045934 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559da0e8e800 con 0x559da0b143a0 2024-04-25T10:33:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.415+0000 7fc17e14e640 1 --2- 172.21.15.43:0/1061045934 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559da0b143a0 0x559da0fa79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.415+0000 7fc161ffb640 10 client.15600 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.415+0000 7fc17e14e640 1 --2- 172.21.15.43:0/1061045934 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559da0b143a0 0x559da0fa79a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21891674581e7f3a server_cookie=6ebfdde432044ec in_seq=0 out_seq=0 2024-04-25T10:33:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc161ffb640 1 -- 172.21.15.43:0/1061045934 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559da0e8e800 con 0x559da0b143a0 2024-04-25T10:33:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc161ffb640 10 client.15600 handle_command_reply: tid=0 2024-04-25T10:33:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc187267740 1 client.15600 shutdown 2024-04-25T10:33:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc187267740 1 -- 172.21.15.43:0/1061045934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b80390 msgr2=0x559da0f32be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc187267740 1 --2- 172.21.15.43:0/1061045934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b80390 0x559da0f32be0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc17001a040 tx=0x7fc170003fd0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc187267740 1 -- 172.21.15.43:0/1061045934 shutdown_connections 2024-04-25T10:33:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc187267740 1 --2- 172.21.15.43:0/1061045934 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559da0b143a0 0x559da0fa79a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc187267740 1 --2- 172.21.15.43:0/1061045934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0b5f6b0 0x559da0f37d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc187267740 1 --2- 172.21.15.43:0/1061045934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559da0b80390 0x559da0f32be0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc161ffb640 0 client.15600 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc187267740 1 --2- 172.21.15.43:0/1061045934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559da0b574d0 0x559da0f37820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc187267740 1 -- 172.21.15.43:0/1061045934 >> 172.21.15.43:0/1061045934 conn(0x559da0b744e0 msgr2=0x559da0f2ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc187267740 1 -- 172.21.15.43:0/1061045934 shutdown_connections 2024-04-25T10:33:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc187267740 1 -- 172.21.15.43:0/1061045934 wait complete. 2024-04-25T10:33:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc187267740 20 client.15600 trim_cache size 0 max 16384 2024-04-25T10:33:10.425 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:33:10.425 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:33:10.425 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:33:10.425 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:33:10.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc1803b2640 1 -- 172.21.15.43:0/531257319 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc1640afe10 msgr2=0x7fc1640b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc1803b2640 1 --2- 172.21.15.43:0/531257319 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc1640afe10 0x7fc1640b2320 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x559da0f1dbd0 tx=0x7fc170009840 comp rx=0 tx=0).stop 2024-04-25T10:33:10.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc1803b2640 1 -- 172.21.15.43:0/531257319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc178109af0 msgr2=0x7fc178103290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc1803b2640 1 --2- 172.21.15.43:0/531257319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc178109af0 0x7fc178103290 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc168006010 tx=0x7fc168004060 comp rx=0 tx=0).stop 2024-04-25T10:33:10.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc1803b2640 1 -- 172.21.15.43:0/531257319 shutdown_connections 2024-04-25T10:33:10.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc1803b2640 1 --2- 172.21.15.43:0/531257319 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc1640afe10 0x7fc1640b2320 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc1803b2640 1 --2- 172.21.15.43:0/531257319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc178109f90 0x7fc1781037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc1803b2640 1 --2- 172.21.15.43:0/531257319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc178109af0 0x7fc178103290 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc1803b2640 1 --2- 172.21.15.43:0/531257319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc178102720 0x7fc178102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.416+0000 7fc1803b2640 1 -- 172.21.15.43:0/531257319 >> 172.21.15.43:0/531257319 conn(0x7fc178077880 msgr2=0x7fc1781051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:10.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.417+0000 7fc1803b2640 1 -- 172.21.15.43:0/531257319 shutdown_connections 2024-04-25T10:33:10.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.417+0000 7fc1803b2640 1 -- 172.21.15.43:0/531257319 wait complete. 2024-04-25T10:33:10.430 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:33:10.430 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:33:10.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.580+0000 7fa4955ee640 1 -- 172.21.15.43:0/568069186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa490108a40 msgr2=0x7fa490102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.580+0000 7fa4955ee640 1 --2- 172.21.15.43:0/568069186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa490108a40 0x7fa490102620 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa480009900 tx=0x7fa48002a410 comp rx=0 tx=0).stop 2024-04-25T10:33:10.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.580+0000 7fa4955ee640 1 -- 172.21.15.43:0/568069186 shutdown_connections 2024-04-25T10:33:10.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.580+0000 7fa4955ee640 1 --2- 172.21.15.43:0/568069186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa490102b60 0x7fa490102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.580+0000 7fa4955ee640 1 --2- 172.21.15.43:0/568069186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa49010d3a0 0x7fa49010f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.580+0000 7fa4955ee640 1 --2- 172.21.15.43:0/568069186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa490108a40 0x7fa490102620 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.580+0000 7fa4955ee640 1 -- 172.21.15.43:0/568069186 >> 172.21.15.43:0/568069186 conn(0x7fa490078040 msgr2=0x7fa490078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:10.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.581+0000 7fa4955ee640 1 -- 172.21.15.43:0/568069186 shutdown_connections 2024-04-25T10:33:10.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.581+0000 7fa4955ee640 1 -- 172.21.15.43:0/568069186 wait complete. 2024-04-25T10:33:10.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.581+0000 7fa4955ee640 1 Processor -- start 2024-04-25T10:33:10.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.581+0000 7fa4955ee640 1 -- start start 2024-04-25T10:33:10.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.582+0000 7fa4955ee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa490102b60 0x7fa4901995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.582+0000 7fa4955ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa490108a40 0x7fa490199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.582+0000 7fa4955ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa49010d3a0 0x7fa49019de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.582+0000 7fa4955ee640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa490112c00 con 0x7fa490108a40 2024-04-25T10:33:10.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.582+0000 7fa4955ee640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa490112a80 con 0x7fa49010d3a0 2024-04-25T10:33:10.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.582+0000 7fa4955ee640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa490112d80 con 0x7fa490102b60 2024-04-25T10:33:10.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.582+0000 7fa48e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa490108a40 0x7fa490199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.582+0000 7fa48e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa490108a40 0x7fa490199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41864/0 (socket says 172.21.15.43:41864) 2024-04-25T10:33:10.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.582+0000 7fa48e7fc640 1 -- 172.21.15.43:0/191758659 learned_addr learned my addr 172.21.15.43:0/191758659 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:10.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.582+0000 7fa48f7fe640 1 --2- 172.21.15.43:0/191758659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa49010d3a0 0x7fa49019de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.582+0000 7fa48effd640 1 --2- 172.21.15.43:0/191758659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa490102b60 0x7fa4901995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.582+0000 7fa48e7fc640 1 -- 172.21.15.43:0/191758659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa490102b60 msgr2=0x7fa4901995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.582+0000 7fa48e7fc640 1 --2- 172.21.15.43:0/191758659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa490102b60 0x7fa4901995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.582+0000 7fa48e7fc640 1 -- 172.21.15.43:0/191758659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa49010d3a0 msgr2=0x7fa49019de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.582+0000 7fa48e7fc640 1 --2- 172.21.15.43:0/191758659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa49010d3a0 0x7fa49019de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.582+0000 7fa48e7fc640 1 -- 172.21.15.43:0/191758659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49019e440 con 0x7fa490108a40 2024-04-25T10:33:10.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.582+0000 7fa48f7fe640 1 --2- 172.21.15.43:0/191758659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa49010d3a0 0x7fa49019de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:10.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.582+0000 7fa48e7fc640 1 --2- 172.21.15.43:0/191758659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa490108a40 0x7fa490199b40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa484005fd0 tx=0x7fa48400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:10.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.583+0000 7fa477fff640 1 -- 172.21.15.43:0/191758659 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484017070 con 0x7fa490108a40 2024-04-25T10:33:10.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.583+0000 7fa477fff640 1 -- 172.21.15.43:0/191758659 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa484009c30 con 0x7fa490108a40 2024-04-25T10:33:10.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.583+0000 7fa4955ee640 1 -- 172.21.15.43:0/191758659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa49019e730 con 0x7fa490108a40 2024-04-25T10:33:10.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.583+0000 7fa477fff640 1 -- 172.21.15.43:0/191758659 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484003d10 con 0x7fa490108a40 2024-04-25T10:33:10.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.583+0000 7fa4955ee640 1 -- 172.21.15.43:0/191758659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4901a60b0 con 0x7fa490108a40 2024-04-25T10:33:10.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.584+0000 7fa4955ee640 1 -- 172.21.15.43:0/191758659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa49006a190 con 0x7fa490108a40 2024-04-25T10:33:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.585+0000 7fa477fff640 1 -- 172.21.15.43:0/191758659 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 147804+0+0 (secure 0 0 0) 0x7fa484003eb0 con 0x7fa490108a40 2024-04-25T10:33:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.586+0000 7fa477fff640 1 --2- 172.21.15.43:0/191758659 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa4780afe10 0x7fa4780b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.586+0000 7fa48effd640 1 --2- 172.21.15.43:0/191758659 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa4780afe10 0x7fa4780b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.586+0000 7fa477fff640 1 -- 172.21.15.43:0/191758659 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fa4840df860 con 0x7fa490108a40 2024-04-25T10:33:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.586+0000 7fa48effd640 1 --2- 172.21.15.43:0/191758659 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa4780afe10 0x7fa4780b2320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa480000c00 tx=0x7fa480031400 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:10.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.589+0000 7fa477fff640 1 -- 172.21.15.43:0/191758659 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa484020030 con 0x7fa490108a40 2024-04-25T10:33:10.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.813+0000 7fa4955ee640 1 -- 172.21.15.43:0/191758659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa490106f00 con 0x7fa490108a40 2024-04-25T10:33:10.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.814+0000 7fa477fff640 1 -- 172.21.15.43:0/191758659 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11869 (secure 0 0 0) 0x7fa484009de0 con 0x7fa490108a40 2024-04-25T10:33:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:33:10.816 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:32:30.234841+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2595248466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2595248466},{"type":"v1","addr":"172.21.15.77:6837","nonce":2595248466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24472,24508]},"id":1}]} 2024-04-25T10:33:10.817 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 24 2024-04-25T10:33:10.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.816+0000 7fa4955ee640 1 -- 172.21.15.43:0/191758659 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa4780afe10 msgr2=0x7fa4780b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.816+0000 7fa4955ee640 1 --2- 172.21.15.43:0/191758659 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa4780afe10 0x7fa4780b2320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa480000c00 tx=0x7fa480031400 comp rx=0 tx=0).stop 2024-04-25T10:33:10.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.816+0000 7fa4955ee640 1 -- 172.21.15.43:0/191758659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa490108a40 msgr2=0x7fa490199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.816+0000 7fa4955ee640 1 --2- 172.21.15.43:0/191758659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa490108a40 0x7fa490199b40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa484005fd0 tx=0x7fa48400dde0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.816+0000 7fa4955ee640 1 -- 172.21.15.43:0/191758659 shutdown_connections 2024-04-25T10:33:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.816+0000 7fa4955ee640 1 --2- 172.21.15.43:0/191758659 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa4780afe10 0x7fa4780b2320 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.816+0000 7fa4955ee640 1 --2- 172.21.15.43:0/191758659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa49010d3a0 0x7fa49019de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.816+0000 7fa4955ee640 1 --2- 172.21.15.43:0/191758659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa490108a40 0x7fa490199b40 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.816+0000 7fa4955ee640 1 --2- 172.21.15.43:0/191758659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa490102b60 0x7fa4901995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.816+0000 7fa4955ee640 1 -- 172.21.15.43:0/191758659 >> 172.21.15.43:0/191758659 conn(0x7fa490078040 msgr2=0x7fa4901040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.816+0000 7fa4955ee640 1 -- 172.21.15.43:0/191758659 shutdown_connections 2024-04-25T10:33:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.816+0000 7fa4955ee640 1 -- 172.21.15.43:0/191758659 wait complete. 2024-04-25T10:33:10.830 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:33:10.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.943+0000 7f3b8d205640 1 -- 172.21.15.43:0/3225031983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b88108eb0 msgr2=0x7f3b88102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.943+0000 7f3b8d205640 1 --2- 172.21.15.43:0/3225031983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b88108eb0 0x7f3b88102b50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009900 tx=0x7f3b7c02a410 comp rx=0 tx=0).stop 2024-04-25T10:33:10.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.943+0000 7f3b8d205640 1 -- 172.21.15.43:0/3225031983 shutdown_connections 2024-04-25T10:33:10.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.943+0000 7f3b8d205640 1 --2- 172.21.15.43:0/3225031983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b881031c0 0x7f3b8810f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.943+0000 7f3b8d205640 1 --2- 172.21.15.43:0/3225031983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b88108eb0 0x7f3b88102b50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.943+0000 7f3b8d205640 1 --2- 172.21.15.43:0/3225031983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b88108a10 0x7f3b88102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.943+0000 7f3b8d205640 1 -- 172.21.15.43:0/3225031983 >> 172.21.15.43:0/3225031983 conn(0x7f3b88078110 msgr2=0x7f3b88078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:10.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.943+0000 7f3b8d205640 1 -- 172.21.15.43:0/3225031983 shutdown_connections 2024-04-25T10:33:10.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.943+0000 7f3b8d205640 1 -- 172.21.15.43:0/3225031983 wait complete. 2024-04-25T10:33:10.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.944+0000 7f3b8d205640 1 Processor -- start 2024-04-25T10:33:10.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.944+0000 7f3b8d205640 1 -- start start 2024-04-25T10:33:10.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.944+0000 7f3b8d205640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b881031c0 0x7f3b880ff250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.944+0000 7f3b8d205640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b88108a10 0x7f3b880ff790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.944+0000 7f3b8d205640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b88108eb0 0x7f3b880ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.944+0000 7f3b8d205640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3b88112c30 con 0x7f3b88108eb0 2024-04-25T10:33:10.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.944+0000 7f3b8d205640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3b88112ab0 con 0x7f3b88108a10 2024-04-25T10:33:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.944+0000 7f3b8d205640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3b88112db0 con 0x7f3b881031c0 2024-04-25T10:33:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.944+0000 7f3b8759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b88108eb0 0x7f3b880ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.944+0000 7f3b8759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b88108eb0 0x7f3b880ffcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41882/0 (socket says 172.21.15.43:41882) 2024-04-25T10:33:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.944+0000 7f3b8759e640 1 -- 172.21.15.43:0/627802631 learned_addr learned my addr 172.21.15.43:0/627802631 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.944+0000 7f3b86d9d640 1 --2- 172.21.15.43:0/627802631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b881031c0 0x7f3b880ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.944+0000 7f3b8659c640 1 --2- 172.21.15.43:0/627802631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b88108a10 0x7f3b880ff790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.945+0000 7f3b8759e640 1 -- 172.21.15.43:0/627802631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b881031c0 msgr2=0x7f3b880ff250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.945+0000 7f3b8759e640 1 --2- 172.21.15.43:0/627802631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b881031c0 0x7f3b880ff250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.945+0000 7f3b8759e640 1 -- 172.21.15.43:0/627802631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b88108a10 msgr2=0x7f3b880ff790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.945+0000 7f3b8759e640 1 --2- 172.21.15.43:0/627802631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b88108a10 0x7f3b880ff790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.945+0000 7f3b8759e640 1 -- 172.21.15.43:0/627802631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b881a8ac0 con 0x7f3b88108eb0 2024-04-25T10:33:10.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.945+0000 7f3b8759e640 1 --2- 172.21.15.43:0/627802631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b88108eb0 0x7f3b880ffcd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000eaa0 tx=0x7f3b8000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:10.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.945+0000 7f3b6ffff640 1 -- 172.21.15.43:0/627802631 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8000cdb0 con 0x7f3b88108eb0 2024-04-25T10:33:10.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.945+0000 7f3b8d205640 1 -- 172.21.15.43:0/627802631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b881a8db0 con 0x7f3b88108eb0 2024-04-25T10:33:10.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.945+0000 7f3b6ffff640 1 -- 172.21.15.43:0/627802631 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b80005020 con 0x7f3b88108eb0 2024-04-25T10:33:10.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.945+0000 7f3b6ffff640 1 -- 172.21.15.43:0/627802631 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b800116d0 con 0x7f3b88108eb0 2024-04-25T10:33:10.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.945+0000 7f3b8d205640 1 -- 172.21.15.43:0/627802631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b881a9210 con 0x7f3b88108eb0 2024-04-25T10:33:10.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.947+0000 7f3b94194740 1 Processor -- start 2024-04-25T10:33:10.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.948+0000 7f3b6ffff640 1 -- 172.21.15.43:0/627802631 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 147804+0+0 (secure 0 0 0) 0x7f3b80004b20 con 0x7f3b88108eb0 2024-04-25T10:33:10.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.948+0000 7f3b94194740 1 -- start start 2024-04-25T10:33:10.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.948+0000 7f3b94194740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b7e390 0x557e41f1f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.948+0000 7f3b94194740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b554d0 0x557e41f1fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.948+0000 7f3b94194740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b5d6b0 0x557e41f2c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.948+0000 7f3b94194740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557e41e84d80 con 0x557e41b554d0 2024-04-25T10:33:10.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.948+0000 7f3b94194740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557e41e3ea20 con 0x557e41b7e390 2024-04-25T10:33:10.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.948+0000 7f3b94194740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557e41e80d40 con 0x557e41b5d6b0 2024-04-25T10:33:10.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.948+0000 7f3b86d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b554d0 0x557e41f1fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.948+0000 7f3b86d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b554d0 0x557e41f1fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41894/0 (socket says 172.21.15.43:41894) 2024-04-25T10:33:10.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.948+0000 7f3b86d9d640 1 -- 172.21.15.43:0/1826036437 learned_addr learned my addr 172.21.15.43:0/1826036437 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:10.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.949+0000 7f3b8659c640 1 --2- 172.21.15.43:0/1826036437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b5d6b0 0x557e41f2c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.949+0000 7f3b8659c640 1 --2- 172.21.15.43:0/1826036437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b7e390 0x557e41f1f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.949+0000 7f3b6ffff640 1 --2- 172.21.15.43:0/627802631 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3b700afe10 0x7f3b700b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.949+0000 7f3b86d9d640 1 -- 172.21.15.43:0/1826036437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b5d6b0 msgr2=0x557e41f2c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:10.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.949+0000 7f3b86d9d640 1 --2- 172.21.15.43:0/1826036437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b5d6b0 0x557e41f2c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.949+0000 7f3b86d9d640 1 -- 172.21.15.43:0/1826036437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b7e390 msgr2=0x557e41f1f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:10.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.949+0000 7f3b86d9d640 1 --2- 172.21.15.43:0/1826036437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b7e390 0x557e41f1f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.949+0000 7f3b86d9d640 1 -- 172.21.15.43:0/1826036437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e41e49c30 con 0x557e41b554d0 2024-04-25T10:33:10.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.949+0000 7f3b8659c640 1 --2- 172.21.15.43:0/1826036437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b5d6b0 0x557e41f2c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:10.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.949+0000 7f3b8759e640 1 --2- 172.21.15.43:0/627802631 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3b700afe10 0x7f3b700b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.949+0000 7f3b6ffff640 1 -- 172.21.15.43:0/627802631 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f3b800df420 con 0x7f3b88108eb0 2024-04-25T10:33:10.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.949+0000 7f3b86d9d640 1 --2- 172.21.15.43:0/1826036437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b554d0 0x557e41f1fd50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3b78004620 tx=0x7f3b7800bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c549a08bb6bc9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:10.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.949+0000 7f3b6d7fa640 1 -- 172.21.15.43:0/1826036437 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b78003ba0 con 0x557e41b554d0 2024-04-25T10:33:10.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.949+0000 7f3b6d7fa640 1 -- 172.21.15.43:0/1826036437 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b78015070 con 0x557e41b554d0 2024-04-25T10:33:10.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.949+0000 7f3b6d7fa640 1 -- 172.21.15.43:0/1826036437 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b78004c10 con 0x557e41b554d0 2024-04-25T10:33:10.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.950+0000 7f3b94194740 1 -- 172.21.15.43:0/1826036437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b554d0 msgr2=0x557e41f1fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.950+0000 7f3b94194740 1 --2- 172.21.15.43:0/1826036437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b554d0 0x557e41f1fd50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3b78004620 tx=0x7f3b7800bb20 comp rx=0 tx=0).stop 2024-04-25T10:33:10.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.950+0000 7f3b8759e640 1 --2- 172.21.15.43:0/627802631 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3b700afe10 0x7f3b700b2320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3b800dc9c0 tx=0x7f3b80074cc0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:10.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.950+0000 7f3b94194740 1 -- 172.21.15.43:0/1826036437 shutdown_connections 2024-04-25T10:33:10.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.950+0000 7f3b94194740 1 --2- 172.21.15.43:0/1826036437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b5d6b0 0x557e41f2c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.950+0000 7f3b94194740 1 --2- 172.21.15.43:0/1826036437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b554d0 0x557e41f1fd50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.950+0000 7f3b94194740 1 --2- 172.21.15.43:0/1826036437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b7e390 0x557e41f1f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.950+0000 7f3b94194740 1 -- 172.21.15.43:0/1826036437 >> 172.21.15.43:0/1826036437 conn(0x557e41b724e0 msgr2=0x557e41f1bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:10.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.950+0000 7f3b94194740 1 -- 172.21.15.43:0/1826036437 shutdown_connections 2024-04-25T10:33:10.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.950+0000 7f3b94194740 1 -- 172.21.15.43:0/1826036437 wait complete. 2024-04-25T10:33:10.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.950+0000 7f3b94194740 1 -- start start 2024-04-25T10:33:10.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.951+0000 7f3b94194740 1 Processor -- start 2024-04-25T10:33:10.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.951+0000 7f3b94194740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b554d0 0x557e41f2a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.951+0000 7f3b94194740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b5d6b0 0x557e41f2a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.951+0000 7f3b94194740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b7e390 0x557e41f36210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.951+0000 7f3b94194740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557e41f2cd90 con 0x557e41b5d6b0 2024-04-25T10:33:10.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.951+0000 7f3b94194740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557e41e0bdc0 con 0x557e41b554d0 2024-04-25T10:33:10.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.951+0000 7f3b94194740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557e41f2cf10 con 0x557e41b7e390 2024-04-25T10:33:10.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.951+0000 7f3b94194740 10 client.0 _ll_register_callbacks cb 0x557e41e27600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:10.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.951+0000 7f3b86d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b5d6b0 0x557e41f2a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.951+0000 7f3b86d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b5d6b0 0x557e41f2a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41902/0 (socket says 172.21.15.43:41902) 2024-04-25T10:33:10.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.951+0000 7f3b86d9d640 1 -- 172.21.15.43:0/2239869180 learned_addr learned my addr 172.21.15.43:0/2239869180 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:10.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.951+0000 7f3b8659c640 1 --2- 172.21.15.43:0/2239869180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b554d0 0x557e41f2a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.951+0000 7f3b8659c640 1 --2- 172.21.15.43:0/2239869180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b7e390 0x557e41f36210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.951+0000 7f3b86d9d640 1 -- 172.21.15.43:0/2239869180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b7e390 msgr2=0x557e41f36210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:10.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.951+0000 7f3b86d9d640 1 --2- 172.21.15.43:0/2239869180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b7e390 0x557e41f36210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.951+0000 7f3b86d9d640 1 -- 172.21.15.43:0/2239869180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b554d0 msgr2=0x557e41f2a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:10.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.951+0000 7f3b86d9d640 1 --2- 172.21.15.43:0/2239869180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b554d0 0x557e41f2a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.951+0000 7f3b86d9d640 1 -- 172.21.15.43:0/2239869180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e41b963f0 con 0x557e41b5d6b0 2024-04-25T10:33:10.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.951+0000 7f3b8659c640 1 --2- 172.21.15.43:0/2239869180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b7e390 0x557e41f36210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:10.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.952+0000 7f3b86d9d640 1 --2- 172.21.15.43:0/2239869180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b5d6b0 0x557e41f2a7f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3b78006010 tx=0x7f3b78003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98966aff1468a61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:10.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.952+0000 7f3b6aff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:10.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.952+0000 7f3b6aff5640 1 -- 172.21.15.43:0/2239869180 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b78028060 con 0x557e41b5d6b0 2024-04-25T10:33:10.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.952+0000 7f3b94194740 1 -- 172.21.15.43:0/2239869180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b78015070 con 0x557e41b5d6b0 2024-04-25T10:33:10.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.952+0000 7f3b6aff5640 1 -- 172.21.15.43:0/2239869180 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f3b78011850 con 0x557e41b5d6b0 2024-04-25T10:33:10.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.952+0000 7f3b6aff5640 1 -- 172.21.15.43:0/2239869180 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3b78011a50 con 0x557e41b5d6b0 2024-04-25T10:33:10.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.952+0000 7f3b94194740 10 client.15618 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:10.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.952+0000 7f3b94194740 1 -- 172.21.15.43:0/2239869180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557e41c607d0 con 0x557e41b5d6b0 2024-04-25T10:33:10.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.952+0000 7f3b6aff5640 1 -- 172.21.15.43:0/2239869180 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b78015070 con 0x557e41b5d6b0 2024-04-25T10:33:10.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.952+0000 7f3b6aff5640 1 -- 172.21.15.43:0/2239869180 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3b78022070 con 0x557e41b5d6b0 2024-04-25T10:33:10.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.952+0000 7f3b94194740 10 client.15618 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:10.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.952+0000 7f3b94194740 10 client.15618 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:33:10.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.952+0000 7f3b94194740 20 client.15618 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:10.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.952+0000 7f3b94194740 1 --2- 172.21.15.43:0/2239869180 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x557e41b3cf60 0x557e41f3c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.952+0000 7f3b94194740 4 client.15618 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:33:10.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.952+0000 7f3b94194740 1 -- 172.21.15.43:0/2239869180 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e41e92990 con 0x557e41b3cf60 2024-04-25T10:33:10.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.953+0000 7f3b8659c640 1 --2- 172.21.15.43:0/2239869180 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x557e41b3cf60 0x557e41f3c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.953+0000 7f3b8659c640 1 --2- 172.21.15.43:0/2239869180 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x557e41b3cf60 0x557e41f3c5f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7d762279b8003ce server_cookie=36fa29146756191e in_seq=0 out_seq=0 2024-04-25T10:33:10.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.953+0000 7f3b6aff5640 10 client.15618 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:33:10.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.954+0000 7f3b6aff5640 1 -- 172.21.15.43:0/2239869180 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557e41e92990 con 0x557e41b3cf60 2024-04-25T10:33:10.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.954+0000 7f3b6aff5640 10 client.15618 handle_command_reply: tid=0 2024-04-25T10:33:10.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.954+0000 7f3b94194740 1 client.15618 shutdown 2024-04-25T10:33:10.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.954+0000 7f3b94194740 1 -- 172.21.15.43:0/2239869180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b5d6b0 msgr2=0x557e41f2a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.954+0000 7f3b94194740 1 --2- 172.21.15.43:0/2239869180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b5d6b0 0x557e41f2a7f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3b78006010 tx=0x7f3b78003dd0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.954+0000 7f3b94194740 1 -- 172.21.15.43:0/2239869180 shutdown_connections 2024-04-25T10:33:10.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.954+0000 7f3b94194740 1 --2- 172.21.15.43:0/2239869180 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x557e41b3cf60 0x557e41f3c5f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.954+0000 7f3b94194740 1 --2- 172.21.15.43:0/2239869180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b7e390 0x557e41f36210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.954+0000 7f3b94194740 1 --2- 172.21.15.43:0/2239869180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b5d6b0 0x557e41f2a7f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.954+0000 7f3b6aff5640 0 client.15618 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:33:10.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.954+0000 7f3b94194740 1 --2- 172.21.15.43:0/2239869180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b554d0 0x557e41f2a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.954+0000 7f3b94194740 1 -- 172.21.15.43:0/2239869180 >> 172.21.15.43:0/2239869180 conn(0x557e41b724e0 msgr2=0x557e41f21530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:10.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.954+0000 7f3b94194740 1 -- 172.21.15.43:0/2239869180 shutdown_connections 2024-04-25T10:33:10.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.954+0000 7f3b94194740 1 -- 172.21.15.43:0/2239869180 wait complete. 2024-04-25T10:33:10.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.954+0000 7f3b94194740 20 client.15618 trim_cache size 0 max 16384 2024-04-25T10:33:10.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.968+0000 7f3b94194740 1 Processor -- start 2024-04-25T10:33:10.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.968+0000 7f3b94194740 1 -- start start 2024-04-25T10:33:10.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.968+0000 7f3b94194740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b3cf60 0x557e41f1bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.968+0000 7f3b94194740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b554d0 0x557e41f21e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.968+0000 7f3b94194740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b5d6b0 0x557e41f29980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.968+0000 7f3b94194740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3b881b4460 con 0x557e41b554d0 2024-04-25T10:33:10.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.968+0000 7f3b94194740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3b881b1d30 con 0x557e41b5d6b0 2024-04-25T10:33:10.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.968+0000 7f3b94194740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3b881afb80 con 0x557e41b3cf60 2024-04-25T10:33:10.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.968+0000 7f3b86d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b554d0 0x557e41f21e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.968+0000 7f3b86d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b554d0 0x557e41f21e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41912/0 (socket says 172.21.15.43:41912) 2024-04-25T10:33:10.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.968+0000 7f3b86d9d640 1 -- 172.21.15.43:0/1577368566 learned_addr learned my addr 172.21.15.43:0/1577368566 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:10.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.968+0000 7f3b8659c640 1 --2- 172.21.15.43:0/1577368566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b3cf60 0x557e41f1bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.968+0000 7f3b8659c640 1 --2- 172.21.15.43:0/1577368566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b5d6b0 0x557e41f29980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.968+0000 7f3b86d9d640 1 -- 172.21.15.43:0/1577368566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b3cf60 msgr2=0x557e41f1bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:10.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.968+0000 7f3b86d9d640 1 --2- 172.21.15.43:0/1577368566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b3cf60 0x557e41f1bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.968+0000 7f3b86d9d640 1 -- 172.21.15.43:0/1577368566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b5d6b0 msgr2=0x557e41f29980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:10.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.968+0000 7f3b86d9d640 1 --2- 172.21.15.43:0/1577368566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b5d6b0 0x557e41f29980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.968+0000 7f3b86d9d640 1 -- 172.21.15.43:0/1577368566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e41f162b0 con 0x557e41b554d0 2024-04-25T10:33:10.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.968+0000 7f3b8659c640 1 --2- 172.21.15.43:0/1577368566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b5d6b0 0x557e41f29980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:10.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.969+0000 7f3b86d9d640 1 --2- 172.21.15.43:0/1577368566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b554d0 0x557e41f21e40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800b3f0 tx=0x7f3b7800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe889e5f7c95356 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:10.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.969+0000 7f3b6cff9640 1 -- 172.21.15.43:0/1577368566 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b78028070 con 0x557e41b554d0 2024-04-25T10:33:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.969+0000 7f3b6cff9640 1 -- 172.21.15.43:0/1577368566 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b78015050 con 0x557e41b554d0 2024-04-25T10:33:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.969+0000 7f3b6cff9640 1 -- 172.21.15.43:0/1577368566 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b780118b0 con 0x557e41b554d0 2024-04-25T10:33:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.969+0000 7f3b94194740 1 -- 172.21.15.43:0/1577368566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b554d0 msgr2=0x557e41f21e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.969+0000 7f3b94194740 1 --2- 172.21.15.43:0/1577368566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b554d0 0x557e41f21e40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800b3f0 tx=0x7f3b7800b4a0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.970+0000 7f3b94194740 1 -- 172.21.15.43:0/1577368566 shutdown_connections 2024-04-25T10:33:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.970+0000 7f3b94194740 1 --2- 172.21.15.43:0/1577368566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b5d6b0 0x557e41f29980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.970+0000 7f3b94194740 1 --2- 172.21.15.43:0/1577368566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b554d0 0x557e41f21e40 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.970+0000 7f3b94194740 1 --2- 172.21.15.43:0/1577368566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b3cf60 0x557e41f1bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.970+0000 7f3b94194740 1 -- 172.21.15.43:0/1577368566 >> 172.21.15.43:0/1577368566 conn(0x557e41b724e0 msgr2=0x557e41f1a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.970+0000 7f3b94194740 1 -- 172.21.15.43:0/1577368566 shutdown_connections 2024-04-25T10:33:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.970+0000 7f3b94194740 1 -- 172.21.15.43:0/1577368566 wait complete. 2024-04-25T10:33:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.970+0000 7f3b94194740 1 -- start start 2024-04-25T10:33:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.970+0000 7f3b94194740 1 Processor -- start 2024-04-25T10:33:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.970+0000 7f3b94194740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b3cf60 0x557e41f3fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.970+0000 7f3b94194740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b554d0 0x557e41f3ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.970+0000 7f3b94194740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b5d6b0 0x557e41f3d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.970+0000 7f3b94194740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3b881b01f0 con 0x557e41b3cf60 2024-04-25T10:33:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.970+0000 7f3b94194740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3b881b1660 con 0x557e41b554d0 2024-04-25T10:33:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.970+0000 7f3b94194740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3b881b08c0 con 0x557e41b5d6b0 2024-04-25T10:33:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.970+0000 7f3b94194740 10 client.0 _ll_register_callbacks cb 0x7f3b78004470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.970+0000 7f3b8659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b3cf60 0x557e41f3fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.970+0000 7f3b8659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b3cf60 0x557e41f3fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41924/0 (socket says 172.21.15.43:41924) 2024-04-25T10:33:10.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.970+0000 7f3b8659c640 1 -- 172.21.15.43:0/2574119296 learned_addr learned my addr 172.21.15.43:0/2574119296 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:10.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.971+0000 7f3b8659c640 1 --2- 172.21.15.43:0/2574119296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b5d6b0 0x557e41f3d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.971+0000 7f3b86d9d640 1 --2- 172.21.15.43:0/2574119296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b554d0 0x557e41f3ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.971+0000 7f3b8659c640 1 -- 172.21.15.43:0/2574119296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b5d6b0 msgr2=0x557e41f3d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:10.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.971+0000 7f3b8659c640 1 --2- 172.21.15.43:0/2574119296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b5d6b0 0x557e41f3d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.971+0000 7f3b8659c640 1 -- 172.21.15.43:0/2574119296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b554d0 msgr2=0x557e41f3ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:10.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.971+0000 7f3b8659c640 1 --2- 172.21.15.43:0/2574119296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b554d0 0x557e41f3ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.971+0000 7f3b8659c640 1 -- 172.21.15.43:0/2574119296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e41f3dac0 con 0x557e41b3cf60 2024-04-25T10:33:10.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.971+0000 7f3b86d9d640 1 --2- 172.21.15.43:0/2574119296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b554d0 0x557e41f3ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:10.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.971+0000 7f3b8659c640 1 --2- 172.21.15.43:0/2574119296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b3cf60 0x557e41f3fb20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c032fd0 tx=0x7f3b7c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517c66cab690b037 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:10.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.971+0000 7f3b6aff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:10.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.971+0000 7f3b6aff5640 1 -- 172.21.15.43:0/2574119296 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c051060 con 0x557e41b3cf60 2024-04-25T10:33:10.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.971+0000 7f3b6aff5640 1 -- 172.21.15.43:0/2574119296 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f3b7c047040 con 0x557e41b3cf60 2024-04-25T10:33:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.971+0000 7f3b94194740 1 -- 172.21.15.43:0/2574119296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b78015050 con 0x557e41b3cf60 2024-04-25T10:33:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.971+0000 7f3b6aff5640 1 -- 172.21.15.43:0/2574119296 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3b7c035040 con 0x557e41b3cf60 2024-04-25T10:33:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.971+0000 7f3b94194740 10 client.15624 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.971+0000 7f3b94194740 1 -- 172.21.15.43:0/2574119296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557e41fa4550 con 0x557e41b3cf60 2024-04-25T10:33:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.971+0000 7f3b6aff5640 1 -- 172.21.15.43:0/2574119296 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b78015050 con 0x557e41b3cf60 2024-04-25T10:33:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.972+0000 7f3b6aff5640 1 -- 172.21.15.43:0/2574119296 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3b7c004a20 con 0x557e41b3cf60 2024-04-25T10:33:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.972+0000 7f3b94194740 10 client.15624 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.972+0000 7f3b94194740 10 client.15624 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:33:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.972+0000 7f3b94194740 20 client.15624 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.972+0000 7f3b94194740 1 --2- 172.21.15.43:0/2574119296 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x557e41b7e390 0x557e41fa6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.972+0000 7f3b94194740 4 client.15624 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:33:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.972+0000 7f3b94194740 1 -- 172.21.15.43:0/2574119296 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e41e8c800 con 0x557e41b7e390 2024-04-25T10:33:10.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.972+0000 7f3b86d9d640 1 --2- 172.21.15.43:0/2574119296 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x557e41b7e390 0x557e41fa6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:10.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.972+0000 7f3b86d9d640 1 --2- 172.21.15.43:0/2574119296 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x557e41b7e390 0x557e41fa6ca0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d2f34fbf7c39a9d server_cookie=859047e970839a8b in_seq=0 out_seq=0 2024-04-25T10:33:10.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.972+0000 7f3b6aff5640 10 client.15624 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:33:10.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b6aff5640 1 -- 172.21.15.43:0/2574119296 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e41e8c800 con 0x557e41b7e390 2024-04-25T10:33:10.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b6aff5640 10 client.15624 handle_command_reply: tid=0 2024-04-25T10:33:10.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b94194740 1 client.15624 shutdown 2024-04-25T10:33:10.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b94194740 1 -- 172.21.15.43:0/2574119296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b3cf60 msgr2=0x557e41f3fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b94194740 1 --2- 172.21.15.43:0/2574119296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b3cf60 0x557e41f3fb20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c032fd0 tx=0x7f3b7c02a410 comp rx=0 tx=0).stop 2024-04-25T10:33:10.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b94194740 1 -- 172.21.15.43:0/2574119296 shutdown_connections 2024-04-25T10:33:10.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b94194740 1 --2- 172.21.15.43:0/2574119296 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x557e41b7e390 0x557e41fa6ca0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b94194740 1 --2- 172.21.15.43:0/2574119296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e41b554d0 0x557e41f3ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b6aff5640 0 client.15624 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:33:10.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b94194740 1 --2- 172.21.15.43:0/2574119296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e41b5d6b0 0x557e41f3d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b94194740 1 --2- 172.21.15.43:0/2574119296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e41b3cf60 0x557e41f3fb20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b94194740 1 -- 172.21.15.43:0/2574119296 >> 172.21.15.43:0/2574119296 conn(0x557e41b724e0 msgr2=0x557e41f21ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:10.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b94194740 1 -- 172.21.15.43:0/2574119296 shutdown_connections 2024-04-25T10:33:10.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b94194740 1 -- 172.21.15.43:0/2574119296 wait complete. 2024-04-25T10:33:10.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b94194740 20 client.15624 trim_cache size 0 max 16384 2024-04-25T10:33:10.983 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:33:10.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b8d205640 1 -- 172.21.15.43:0/627802631 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3b700afe10 msgr2=0x7f3b700b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b8d205640 1 --2- 172.21.15.43:0/627802631 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3b700afe10 0x7f3b700b2320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3b800dc9c0 tx=0x7f3b80074cc0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b8d205640 1 -- 172.21.15.43:0/627802631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b88108eb0 msgr2=0x7f3b880ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:10.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b8d205640 1 --2- 172.21.15.43:0/627802631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b88108eb0 0x7f3b880ffcd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000eaa0 tx=0x7f3b8000ef60 comp rx=0 tx=0).stop 2024-04-25T10:33:10.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b8d205640 1 -- 172.21.15.43:0/627802631 shutdown_connections 2024-04-25T10:33:10.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b8d205640 1 --2- 172.21.15.43:0/627802631 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3b700afe10 0x7f3b700b2320 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b8d205640 1 --2- 172.21.15.43:0/627802631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b88108a10 0x7f3b880ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b8d205640 1 --2- 172.21.15.43:0/627802631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b88108eb0 0x7f3b880ffcd0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b8d205640 1 --2- 172.21.15.43:0/627802631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b881031c0 0x7f3b880ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:10.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.973+0000 7f3b8d205640 1 -- 172.21.15.43:0/627802631 >> 172.21.15.43:0/627802631 conn(0x7f3b88078110 msgr2=0x7f3b88074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:10.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.974+0000 7f3b8d205640 1 -- 172.21.15.43:0/627802631 shutdown_connections 2024-04-25T10:33:10.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:10.974+0000 7f3b8d205640 1 -- 172.21.15.43:0/627802631 wait complete. 2024-04-25T10:33:10.987 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:33:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.099+0000 7fe9d9dce640 1 -- 172.21.15.43:0/4137340211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d4109b20 msgr2=0x7fe9d41036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.099+0000 7fe9d9dce640 1 --2- 172.21.15.43:0/4137340211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d4109b20 0x7fe9d41036c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4009900 tx=0x7fe9c402a410 comp rx=0 tx=0).stop 2024-04-25T10:33:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.099+0000 7fe9d9dce640 1 -- 172.21.15.43:0/4137340211 shutdown_connections 2024-04-25T10:33:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.099+0000 7fe9d9dce640 1 --2- 172.21.15.43:0/4137340211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9d4104270 0x7fe9d4110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.099+0000 7fe9d9dce640 1 --2- 172.21.15.43:0/4137340211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d4109fc0 0x7fe9d4103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.099+0000 7fe9d9dce640 1 --2- 172.21.15.43:0/4137340211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d4109b20 0x7fe9d41036c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.099+0000 7fe9d9dce640 1 -- 172.21.15.43:0/4137340211 >> 172.21.15.43:0/4137340211 conn(0x7fe9d40ff1e0 msgr2=0x7fe9d4101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.099+0000 7fe9d9dce640 1 -- 172.21.15.43:0/4137340211 shutdown_connections 2024-04-25T10:33:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.099+0000 7fe9d9dce640 1 -- 172.21.15.43:0/4137340211 wait complete. 2024-04-25T10:33:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.099+0000 7fe9d9dce640 1 Processor -- start 2024-04-25T10:33:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.100+0000 7fe9d9dce640 1 -- start start 2024-04-25T10:33:11.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.100+0000 7fe9d9dce640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d4104270 0x7fe9d419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.100+0000 7fe9d9dce640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9d4109b20 0x7fe9d419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.100+0000 7fe9d9dce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d4109fc0 0x7fe9d419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.100+0000 7fe9d9dce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d4113d60 con 0x7fe9d4109fc0 2024-04-25T10:33:11.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.100+0000 7fe9d9dce640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d4113be0 con 0x7fe9d4109b20 2024-04-25T10:33:11.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.100+0000 7fe9d9dce640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d4113ee0 con 0x7fe9d4104270 2024-04-25T10:33:11.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.100+0000 7fe9d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d4109fc0 0x7fe9d419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.100+0000 7fe9d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d4109fc0 0x7fe9d419efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41942/0 (socket says 172.21.15.43:41942) 2024-04-25T10:33:11.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.100+0000 7fe9d3fff640 1 -- 172.21.15.43:0/1924802030 learned_addr learned my addr 172.21.15.43:0/1924802030 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:11.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.100+0000 7fe9d2ffd640 1 --2- 172.21.15.43:0/1924802030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9d4109b20 0x7fe9d419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.100+0000 7fe9d3fff640 1 -- 172.21.15.43:0/1924802030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d4104270 msgr2=0x7fe9d419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:11.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.100+0000 7fe9d37fe640 1 --2- 172.21.15.43:0/1924802030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d4104270 0x7fe9d419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.100+0000 7fe9d3fff640 1 --2- 172.21.15.43:0/1924802030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d4104270 0x7fe9d419a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.100+0000 7fe9d3fff640 1 -- 172.21.15.43:0/1924802030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9d4109b20 msgr2=0x7fe9d419ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:11.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.100+0000 7fe9d3fff640 1 --2- 172.21.15.43:0/1924802030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9d4109b20 0x7fe9d419ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.100+0000 7fe9d3fff640 1 -- 172.21.15.43:0/1924802030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d419f620 con 0x7fe9d4109fc0 2024-04-25T10:33:11.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.100+0000 7fe9d2ffd640 1 --2- 172.21.15.43:0/1924802030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9d4109b20 0x7fe9d419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:11.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.101+0000 7fe9d3fff640 1 --2- 172.21.15.43:0/1924802030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d4109fc0 0x7fe9d419efe0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009fd0 tx=0x7fe9cc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:11.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.101+0000 7fe9d0ff9640 1 -- 172.21.15.43:0/1924802030 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc017070 con 0x7fe9d4109fc0 2024-04-25T10:33:11.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.101+0000 7fe9d0ff9640 1 -- 172.21.15.43:0/1924802030 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9cc003e20 con 0x7fe9d4109fc0 2024-04-25T10:33:11.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.101+0000 7fe9d9dce640 1 -- 172.21.15.43:0/1924802030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d419f910 con 0x7fe9d4109fc0 2024-04-25T10:33:11.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.101+0000 7fe9d0ff9640 1 -- 172.21.15.43:0/1924802030 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc010460 con 0x7fe9d4109fc0 2024-04-25T10:33:11.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.101+0000 7fe9d9dce640 1 -- 172.21.15.43:0/1924802030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d419fe10 con 0x7fe9d4109fc0 2024-04-25T10:33:11.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.104+0000 7fe9e0c83740 1 Processor -- start 2024-04-25T10:33:11.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.104+0000 7fe9d0ff9640 1 -- 172.21.15.43:0/1924802030 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 147804+0+0 (secure 0 0 0) 0x7fe9cc003970 con 0x7fe9d4109fc0 2024-04-25T10:33:11.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.104+0000 7fe9e0c83740 1 -- start start 2024-04-25T10:33:11.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.104+0000 7fe9e0c83740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2e39ab390 0x55c2e3d4c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.104+0000 7fe9e0c83740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e39824d0 0x55c2e3d4cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.104+0000 7fe9e0c83740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e398a6b0 0x55c2e3d596a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.104+0000 7fe9e0c83740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c2e3cb1d80 con 0x55c2e398a6b0 2024-04-25T10:33:11.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.104+0000 7fe9e0c83740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c2e3c6ba20 con 0x55c2e39824d0 2024-04-25T10:33:11.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.104+0000 7fe9e0c83740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c2e3cadd40 con 0x55c2e39ab390 2024-04-25T10:33:11.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.105+0000 7fe9d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e398a6b0 0x55c2e3d596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.105+0000 7fe9d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e398a6b0 0x55c2e3d596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41948/0 (socket says 172.21.15.43:41948) 2024-04-25T10:33:11.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.105+0000 7fe9d2ffd640 1 -- 172.21.15.43:0/1398407819 learned_addr learned my addr 172.21.15.43:0/1398407819 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:11.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.105+0000 7fe9d37fe640 1 --2- 172.21.15.43:0/1398407819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e39824d0 0x55c2e3d4cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.105+0000 7fe9d2ffd640 1 --2- 172.21.15.43:0/1398407819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2e39ab390 0x55c2e3d4c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.105+0000 7fe9d0ff9640 1 --2- 172.21.15.43:0/1924802030 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe9bc0afe10 0x7fe9bc0b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.105+0000 7fe9d2ffd640 1 -- 172.21.15.43:0/1398407819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2e39ab390 msgr2=0x55c2e3d4c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:11.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.105+0000 7fe9d2ffd640 1 --2- 172.21.15.43:0/1398407819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2e39ab390 0x55c2e3d4c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.105+0000 7fe9d2ffd640 1 -- 172.21.15.43:0/1398407819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e39824d0 msgr2=0x55c2e3d4cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:11.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.105+0000 7fe9d2ffd640 1 --2- 172.21.15.43:0/1398407819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e39824d0 0x55c2e3d4cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.105+0000 7fe9d2ffd640 1 -- 172.21.15.43:0/1398407819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2e3c76c30 con 0x55c2e398a6b0 2024-04-25T10:33:11.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.105+0000 7fe9d3fff640 1 --2- 172.21.15.43:0/1924802030 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe9bc0afe10 0x7fe9bc0b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.105+0000 7fe9d2ffd640 1 --2- 172.21.15.43:0/1398407819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e398a6b0 0x55c2e3d596a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8012710 tx=0x7fe9c8009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7049220de4ec05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:11.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.105+0000 7fe9d0ff9640 1 -- 172.21.15.43:0/1924802030 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fe9cc0df0c0 con 0x7fe9d4109fc0 2024-04-25T10:33:11.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.105+0000 7fe9b9ffb640 1 -- 172.21.15.43:0/1398407819 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8012c90 con 0x55c2e398a6b0 2024-04-25T10:33:11.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.106+0000 7fe9b9ffb640 1 -- 172.21.15.43:0/1398407819 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9c8012e30 con 0x55c2e398a6b0 2024-04-25T10:33:11.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.106+0000 7fe9b9ffb640 1 -- 172.21.15.43:0/1398407819 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80037f0 con 0x55c2e398a6b0 2024-04-25T10:33:11.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.106+0000 7fe9e0c83740 1 -- 172.21.15.43:0/1398407819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e398a6b0 msgr2=0x55c2e3d596a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:11.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.106+0000 7fe9d3fff640 1 --2- 172.21.15.43:0/1924802030 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe9bc0afe10 0x7fe9bc0b2320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc0dc700 tx=0x7fe9cc0d5840 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:11.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.106+0000 7fe9e0c83740 1 --2- 172.21.15.43:0/1398407819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e398a6b0 0x55c2e3d596a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8012710 tx=0x7fe9c8009640 comp rx=0 tx=0).stop 2024-04-25T10:33:11.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.106+0000 7fe9e0c83740 1 -- 172.21.15.43:0/1398407819 shutdown_connections 2024-04-25T10:33:11.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.106+0000 7fe9e0c83740 1 --2- 172.21.15.43:0/1398407819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e39824d0 0x55c2e3d4cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.106+0000 7fe9e0c83740 1 --2- 172.21.15.43:0/1398407819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e398a6b0 0x55c2e3d596a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.106+0000 7fe9e0c83740 1 --2- 172.21.15.43:0/1398407819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2e39ab390 0x55c2e3d4c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.106+0000 7fe9e0c83740 1 -- 172.21.15.43:0/1398407819 >> 172.21.15.43:0/1398407819 conn(0x55c2e399f4e0 msgr2=0x55c2e3d48fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:11.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.106+0000 7fe9e0c83740 1 -- 172.21.15.43:0/1398407819 shutdown_connections 2024-04-25T10:33:11.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.106+0000 7fe9e0c83740 1 -- 172.21.15.43:0/1398407819 wait complete. 2024-04-25T10:33:11.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.107+0000 7fe9e0c83740 1 -- start start 2024-04-25T10:33:11.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.107+0000 7fe9e0c83740 1 Processor -- start 2024-04-25T10:33:11.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.107+0000 7fe9e0c83740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e39824d0 0x55c2e3d57260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.107+0000 7fe9e0c83740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e398a6b0 0x55c2e3d577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.107+0000 7fe9e0c83740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2e39ab390 0x55c2e3d63210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.107+0000 7fe9e0c83740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c2e3d59d90 con 0x55c2e39824d0 2024-04-25T10:33:11.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.107+0000 7fe9e0c83740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c2e3c38dc0 con 0x55c2e398a6b0 2024-04-25T10:33:11.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.107+0000 7fe9e0c83740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c2e3d59f10 con 0x55c2e39ab390 2024-04-25T10:33:11.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.107+0000 7fe9e0c83740 10 client.0 _ll_register_callbacks cb 0x55c2e3c54600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:11.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.107+0000 7fe9d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e39824d0 0x55c2e3d57260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.107+0000 7fe9d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e39824d0 0x55c2e3d57260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41962/0 (socket says 172.21.15.43:41962) 2024-04-25T10:33:11.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.108+0000 7fe9d2ffd640 1 -- 172.21.15.43:0/2270049566 learned_addr learned my addr 172.21.15.43:0/2270049566 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:11.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.108+0000 7fe9d37fe640 1 --2- 172.21.15.43:0/2270049566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e398a6b0 0x55c2e3d577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.108+0000 7fe9d2ffd640 1 --2- 172.21.15.43:0/2270049566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2e39ab390 0x55c2e3d63210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.108+0000 7fe9d2ffd640 1 -- 172.21.15.43:0/2270049566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2e39ab390 msgr2=0x55c2e3d63210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:11.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.108+0000 7fe9d2ffd640 1 --2- 172.21.15.43:0/2270049566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2e39ab390 0x55c2e3d63210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.108+0000 7fe9d2ffd640 1 -- 172.21.15.43:0/2270049566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e398a6b0 msgr2=0x55c2e3d577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:11.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.108+0000 7fe9d2ffd640 1 --2- 172.21.15.43:0/2270049566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e398a6b0 0x55c2e3d577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.108+0000 7fe9d2ffd640 1 -- 172.21.15.43:0/2270049566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2e39c33f0 con 0x55c2e39824d0 2024-04-25T10:33:11.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.108+0000 7fe9d2ffd640 1 --2- 172.21.15.43:0/2270049566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e39824d0 0x55c2e3d57260 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009610 tx=0x7fe9c8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb1df255a9d811c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:11.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.108+0000 7fe9b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:11.116 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:10 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/191758659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:33:11.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.108+0000 7fe9b2ffd640 1 -- 172.21.15.43:0/2270049566 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80046c0 con 0x55c2e39824d0 2024-04-25T10:33:11.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.108+0000 7fe9e0c83740 1 -- 172.21.15.43:0/2270049566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9c8012e30 con 0x55c2e39824d0 2024-04-25T10:33:11.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.108+0000 7fe9b2ffd640 1 -- 172.21.15.43:0/2270049566 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fe9c8004860 con 0x55c2e39824d0 2024-04-25T10:33:11.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.108+0000 7fe9b2ffd640 1 -- 172.21.15.43:0/2270049566 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe9c8004a60 con 0x55c2e39824d0 2024-04-25T10:33:11.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.109+0000 7fe9e0c83740 10 client.15636 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:11.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.109+0000 7fe9e0c83740 1 -- 172.21.15.43:0/2270049566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c2e3a8d7d0 con 0x55c2e39824d0 2024-04-25T10:33:11.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.109+0000 7fe9b2ffd640 1 -- 172.21.15.43:0/2270049566 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9c8012e30 con 0x55c2e39824d0 2024-04-25T10:33:11.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.109+0000 7fe9b2ffd640 1 -- 172.21.15.43:0/2270049566 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe9c801fb40 con 0x55c2e39824d0 2024-04-25T10:33:11.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.109+0000 7fe9e0c83740 10 client.15636 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:11.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.109+0000 7fe9e0c83740 10 client.15636 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:33:11.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.109+0000 7fe9e0c83740 20 client.15636 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:11.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.109+0000 7fe9e0c83740 1 --2- 172.21.15.43:0/2270049566 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55c2e3969f60 0x55c2e3d694f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.109+0000 7fe9e0c83740 4 client.15636 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:33:11.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.109+0000 7fe9e0c83740 1 -- 172.21.15.43:0/2270049566 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2e3cbf990 con 0x55c2e3969f60 2024-04-25T10:33:11.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.109+0000 7fe9d37fe640 1 --2- 172.21.15.43:0/2270049566 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55c2e3969f60 0x55c2e3d694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.110+0000 7fe9d37fe640 1 --2- 172.21.15.43:0/2270049566 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55c2e3969f60 0x55c2e3d694f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c327d24aca2241b4 server_cookie=e55910ed5f6be8e5 in_seq=0 out_seq=0 2024-04-25T10:33:11.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.110+0000 7fe9b2ffd640 10 client.15636 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:33:11.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.111+0000 7fe9b2ffd640 1 -- 172.21.15.43:0/2270049566 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c2e3cbf990 con 0x55c2e3969f60 2024-04-25T10:33:11.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.111+0000 7fe9b2ffd640 10 client.15636 handle_command_reply: tid=0 2024-04-25T10:33:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.111+0000 7fe9e0c83740 1 client.15636 shutdown 2024-04-25T10:33:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.111+0000 7fe9e0c83740 1 -- 172.21.15.43:0/2270049566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e39824d0 msgr2=0x55c2e3d57260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.111+0000 7fe9e0c83740 1 --2- 172.21.15.43:0/2270049566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e39824d0 0x55c2e3d57260 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009610 tx=0x7fe9c8001e50 comp rx=0 tx=0).stop 2024-04-25T10:33:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.111+0000 7fe9e0c83740 1 -- 172.21.15.43:0/2270049566 shutdown_connections 2024-04-25T10:33:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.111+0000 7fe9e0c83740 1 --2- 172.21.15.43:0/2270049566 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55c2e3969f60 0x55c2e3d694f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.111+0000 7fe9e0c83740 1 --2- 172.21.15.43:0/2270049566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e398a6b0 0x55c2e3d577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.111+0000 7fe9e0c83740 1 --2- 172.21.15.43:0/2270049566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2e39ab390 0x55c2e3d63210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.111+0000 7fe9b2ffd640 0 client.15636 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:33:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.111+0000 7fe9e0c83740 1 --2- 172.21.15.43:0/2270049566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e39824d0 0x55c2e3d57260 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.111+0000 7fe9e0c83740 1 -- 172.21.15.43:0/2270049566 >> 172.21.15.43:0/2270049566 conn(0x55c2e399f4e0 msgr2=0x55c2e3d4e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.111+0000 7fe9e0c83740 1 -- 172.21.15.43:0/2270049566 shutdown_connections 2024-04-25T10:33:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.111+0000 7fe9e0c83740 1 -- 172.21.15.43:0/2270049566 wait complete. 2024-04-25T10:33:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.111+0000 7fe9e0c83740 20 client.15636 trim_cache size 0 max 16384 2024-04-25T10:33:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.125+0000 7fe9e0c83740 1 Processor -- start 2024-04-25T10:33:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.125+0000 7fe9e0c83740 1 -- start start 2024-04-25T10:33:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.125+0000 7fe9e0c83740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2e3969f60 0x55c2e3d62e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.125+0000 7fe9e0c83740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e39824d0 0x55c2e3d51430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.125+0000 7fe9e0c83740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e398a6b0 0x55c2e3d6eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.125+0000 7fe9e0c83740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d41b2270 con 0x55c2e39824d0 2024-04-25T10:33:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.125+0000 7fe9e0c83740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d41afb40 con 0x55c2e398a6b0 2024-04-25T10:33:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.125+0000 7fe9e0c83740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d41ad990 con 0x55c2e3969f60 2024-04-25T10:33:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.125+0000 7fe9d37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e39824d0 0x55c2e3d51430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.125+0000 7fe9d37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e39824d0 0x55c2e3d51430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41968/0 (socket says 172.21.15.43:41968) 2024-04-25T10:33:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.125+0000 7fe9d37fe640 1 -- 172.21.15.43:0/295602916 learned_addr learned my addr 172.21.15.43:0/295602916 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.125+0000 7fe9d37fe640 1 -- 172.21.15.43:0/295602916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2e3969f60 msgr2=0x55c2e3d62e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.125+0000 7fe9d37fe640 1 --2- 172.21.15.43:0/295602916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2e3969f60 0x55c2e3d62e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.125+0000 7fe9d37fe640 1 -- 172.21.15.43:0/295602916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e398a6b0 msgr2=0x55c2e3d6eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.125+0000 7fe9d37fe640 1 --2- 172.21.15.43:0/295602916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e398a6b0 0x55c2e3d6eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.125+0000 7fe9d37fe640 1 -- 172.21.15.43:0/295602916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2e3d58510 con 0x55c2e39824d0 2024-04-25T10:33:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.125+0000 7fe9d37fe640 1 --2- 172.21.15.43:0/295602916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e39824d0 0x55c2e3d51430 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4104800 tx=0x7fe9c4031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6acc2a88d8ca1c65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.126+0000 7fe9b97fa640 1 -- 172.21.15.43:0/295602916 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4035040 con 0x55c2e39824d0 2024-04-25T10:33:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.126+0000 7fe9b97fa640 1 -- 172.21.15.43:0/295602916 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9c4049070 con 0x55c2e39824d0 2024-04-25T10:33:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.126+0000 7fe9b97fa640 1 -- 172.21.15.43:0/295602916 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4004dd0 con 0x55c2e39824d0 2024-04-25T10:33:11.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.126+0000 7fe9e0c83740 1 -- 172.21.15.43:0/295602916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e39824d0 msgr2=0x55c2e3d51430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:11.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.126+0000 7fe9e0c83740 1 --2- 172.21.15.43:0/295602916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e39824d0 0x55c2e3d51430 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4104800 tx=0x7fe9c4031400 comp rx=0 tx=0).stop 2024-04-25T10:33:11.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.126+0000 7fe9e0c83740 1 -- 172.21.15.43:0/295602916 shutdown_connections 2024-04-25T10:33:11.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.126+0000 7fe9e0c83740 1 --2- 172.21.15.43:0/295602916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e398a6b0 0x55c2e3d6eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.126+0000 7fe9e0c83740 1 --2- 172.21.15.43:0/295602916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e39824d0 0x55c2e3d51430 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.126+0000 7fe9e0c83740 1 --2- 172.21.15.43:0/295602916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2e3969f60 0x55c2e3d62e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.126+0000 7fe9e0c83740 1 -- 172.21.15.43:0/295602916 >> 172.21.15.43:0/295602916 conn(0x55c2e399f4e0 msgr2=0x55c2e3d48040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:11.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.127+0000 7fe9e0c83740 1 -- 172.21.15.43:0/295602916 shutdown_connections 2024-04-25T10:33:11.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.127+0000 7fe9e0c83740 1 -- 172.21.15.43:0/295602916 wait complete. 2024-04-25T10:33:11.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.127+0000 7fe9e0c83740 1 -- start start 2024-04-25T10:33:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.127+0000 7fe9e0c83740 1 Processor -- start 2024-04-25T10:33:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.127+0000 7fe9e0c83740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e3969f60 0x55c2e3d5adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.127+0000 7fe9e0c83740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2e39824d0 0x55c2e3d5b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.127+0000 7fe9e0c83740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e398a6b0 0x55c2e3d5b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.127+0000 7fe9e0c83740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d41ae000 con 0x55c2e398a6b0 2024-04-25T10:33:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.127+0000 7fe9e0c83740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d41af470 con 0x55c2e3969f60 2024-04-25T10:33:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.127+0000 7fe9e0c83740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d41ae6d0 con 0x55c2e39824d0 2024-04-25T10:33:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.127+0000 7fe9e0c83740 10 client.0 _ll_register_callbacks cb 0x7fe9c8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.128+0000 7fe9d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e398a6b0 0x55c2e3d5b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.128+0000 7fe9d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e398a6b0 0x55c2e3d5b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41984/0 (socket says 172.21.15.43:41984) 2024-04-25T10:33:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.128+0000 7fe9d2ffd640 1 -- 172.21.15.43:0/849403400 learned_addr learned my addr 172.21.15.43:0/849403400 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.128+0000 7fe9d2ffd640 1 --2- 172.21.15.43:0/849403400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e3969f60 0x55c2e3d5adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.128+0000 7fe9d37fe640 1 --2- 172.21.15.43:0/849403400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2e39824d0 0x55c2e3d5b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.128+0000 7fe9d2ffd640 1 -- 172.21.15.43:0/849403400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2e39824d0 msgr2=0x55c2e3d5b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.128+0000 7fe9d2ffd640 1 --2- 172.21.15.43:0/849403400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2e39824d0 0x55c2e3d5b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.128+0000 7fe9d2ffd640 1 -- 172.21.15.43:0/849403400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e3969f60 msgr2=0x55c2e3d5adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.128+0000 7fe9d2ffd640 1 --2- 172.21.15.43:0/849403400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e3969f60 0x55c2e3d5adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.128+0000 7fe9d2ffd640 1 -- 172.21.15.43:0/849403400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2e3dd0670 con 0x55c2e398a6b0 2024-04-25T10:33:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.128+0000 7fe9d2ffd640 1 --2- 172.21.15.43:0/849403400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e398a6b0 0x55c2e3d5b870 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009340 tx=0x7fe9c801ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c66e04c1a8b262 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.128+0000 7fe9b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.128+0000 7fe9b2ffd640 1 -- 172.21.15.43:0/849403400 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8036070 con 0x55c2e398a6b0 2024-04-25T10:33:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.128+0000 7fe9b2ffd640 1 -- 172.21.15.43:0/849403400 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fe9c8012d50 con 0x55c2e398a6b0 2024-04-25T10:33:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.128+0000 7fe9e0c83740 1 -- 172.21.15.43:0/849403400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9c4049070 con 0x55c2e398a6b0 2024-04-25T10:33:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.128+0000 7fe9b2ffd640 1 -- 172.21.15.43:0/849403400 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe9c8041050 con 0x55c2e398a6b0 2024-04-25T10:33:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.128+0000 7fe9e0c83740 10 client.15642 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.128+0000 7fe9e0c83740 1 -- 172.21.15.43:0/849403400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c2e3dd1880 con 0x55c2e398a6b0 2024-04-25T10:33:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.128+0000 7fe9b2ffd640 1 -- 172.21.15.43:0/849403400 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9c4049070 con 0x55c2e398a6b0 2024-04-25T10:33:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.129+0000 7fe9b2ffd640 1 -- 172.21.15.43:0/849403400 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe9c8004c00 con 0x55c2e398a6b0 2024-04-25T10:33:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.129+0000 7fe9e0c83740 10 client.15642 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.129+0000 7fe9e0c83740 10 client.15642 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:33:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.129+0000 7fe9e0c83740 20 client.15642 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.129+0000 7fe9e0c83740 1 --2- 172.21.15.43:0/849403400 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55c2e39ab390 0x55c2e3dd3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.129+0000 7fe9e0c83740 4 client.15642 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:33:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.129+0000 7fe9e0c83740 1 -- 172.21.15.43:0/849403400 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2e3cb9800 con 0x55c2e39ab390 2024-04-25T10:33:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.129+0000 7fe9d37fe640 1 --2- 172.21.15.43:0/849403400 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55c2e39ab390 0x55c2e3dd3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.129+0000 7fe9d37fe640 1 --2- 172.21.15.43:0/849403400 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55c2e39ab390 0x55c2e3dd3e90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be10aa3f77f28501 server_cookie=3a87b34edbfc012c in_seq=0 out_seq=0 2024-04-25T10:33:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.129+0000 7fe9b2ffd640 10 client.15642 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:33:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.130+0000 7fe9b2ffd640 1 -- 172.21.15.43:0/849403400 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2e3cb9800 con 0x55c2e39ab390 2024-04-25T10:33:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.130+0000 7fe9b2ffd640 10 client.15642 handle_command_reply: tid=0 2024-04-25T10:33:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.130+0000 7fe9e0c83740 1 client.15642 shutdown 2024-04-25T10:33:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.130+0000 7fe9e0c83740 1 -- 172.21.15.43:0/849403400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e398a6b0 msgr2=0x55c2e3d5b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:11.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.130+0000 7fe9e0c83740 1 --2- 172.21.15.43:0/849403400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e398a6b0 0x55c2e3d5b870 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009340 tx=0x7fe9c801ff90 comp rx=0 tx=0).stop 2024-04-25T10:33:11.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.130+0000 7fe9e0c83740 1 -- 172.21.15.43:0/849403400 shutdown_connections 2024-04-25T10:33:11.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.130+0000 7fe9e0c83740 1 --2- 172.21.15.43:0/849403400 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55c2e39ab390 0x55c2e3dd3e90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.130+0000 7fe9e0c83740 1 --2- 172.21.15.43:0/849403400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2e398a6b0 0x55c2e3d5b870 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.130+0000 7fe9e0c83740 1 --2- 172.21.15.43:0/849403400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2e39824d0 0x55c2e3d5b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.130+0000 7fe9e0c83740 1 --2- 172.21.15.43:0/849403400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e3969f60 0x55c2e3d5adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.130+0000 7fe9b2ffd640 0 client.15642 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:33:11.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.130+0000 7fe9e0c83740 1 -- 172.21.15.43:0/849403400 >> 172.21.15.43:0/849403400 conn(0x55c2e399f4e0 msgr2=0x55c2e3d568e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:11.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.130+0000 7fe9e0c83740 1 -- 172.21.15.43:0/849403400 shutdown_connections 2024-04-25T10:33:11.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.130+0000 7fe9e0c83740 1 -- 172.21.15.43:0/849403400 wait complete. 2024-04-25T10:33:11.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.130+0000 7fe9e0c83740 20 client.15642 trim_cache size 0 max 16384 2024-04-25T10:33:11.139 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:33:11.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.131+0000 7fe9d9dce640 1 -- 172.21.15.43:0/1924802030 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe9bc0afe10 msgr2=0x7fe9bc0b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:11.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.131+0000 7fe9d9dce640 1 --2- 172.21.15.43:0/1924802030 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe9bc0afe10 0x7fe9bc0b2320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc0dc700 tx=0x7fe9cc0d5840 comp rx=0 tx=0).stop 2024-04-25T10:33:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.131+0000 7fe9d9dce640 1 -- 172.21.15.43:0/1924802030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d4109fc0 msgr2=0x7fe9d419efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.131+0000 7fe9d9dce640 1 --2- 172.21.15.43:0/1924802030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d4109fc0 0x7fe9d419efe0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009fd0 tx=0x7fe9cc00b260 comp rx=0 tx=0).stop 2024-04-25T10:33:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.131+0000 7fe9d9dce640 1 -- 172.21.15.43:0/1924802030 shutdown_connections 2024-04-25T10:33:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.131+0000 7fe9d9dce640 1 --2- 172.21.15.43:0/1924802030 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe9bc0afe10 0x7fe9bc0b2320 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.131+0000 7fe9d9dce640 1 --2- 172.21.15.43:0/1924802030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9d4109b20 0x7fe9d419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.131+0000 7fe9d9dce640 1 --2- 172.21.15.43:0/1924802030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d4109fc0 0x7fe9d419efe0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.131+0000 7fe9d9dce640 1 --2- 172.21.15.43:0/1924802030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d4104270 0x7fe9d419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.131+0000 7fe9d9dce640 1 -- 172.21.15.43:0/1924802030 >> 172.21.15.43:0/1924802030 conn(0x7fe9d40ff1e0 msgr2=0x7fe9d40ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.131+0000 7fe9d9dce640 1 -- 172.21.15.43:0/1924802030 shutdown_connections 2024-04-25T10:33:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.131+0000 7fe9d9dce640 1 -- 172.21.15.43:0/1924802030 wait complete. 2024-04-25T10:33:11.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:33:11.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.256+0000 7efe16820640 1 -- 172.21.15.43:0/874836487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe10102af0 msgr2=0x7efe10102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:11.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.256+0000 7efe16820640 1 --2- 172.21.15.43:0/874836487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe10102af0 0x7efe10102f00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7efe04009900 tx=0x7efe0402a410 comp rx=0 tx=0).stop 2024-04-25T10:33:11.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.256+0000 7efe16820640 1 -- 172.21.15.43:0/874836487 shutdown_connections 2024-04-25T10:33:11.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.256+0000 7efe16820640 1 --2- 172.21.15.43:0/874836487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe1010d380 0x7efe1010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.256+0000 7efe16820640 1 --2- 172.21.15.43:0/874836487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe10102af0 0x7efe10102f00 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.256+0000 7efe16820640 1 --2- 172.21.15.43:0/874836487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe10108a20 0x7efe101025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.256+0000 7efe16820640 1 -- 172.21.15.43:0/874836487 >> 172.21.15.43:0/874836487 conn(0x7efe10078100 msgr2=0x7efe10078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:11.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.256+0000 7efe16820640 1 -- 172.21.15.43:0/874836487 shutdown_connections 2024-04-25T10:33:11.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.256+0000 7efe16820640 1 -- 172.21.15.43:0/874836487 wait complete. 2024-04-25T10:33:11.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.257+0000 7efe16820640 1 Processor -- start 2024-04-25T10:33:11.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.257+0000 7efe16820640 1 -- start start 2024-04-25T10:33:11.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.257+0000 7efe16820640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe10102af0 0x7efe100ff250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.257+0000 7efe16820640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe10108a20 0x7efe100ff790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.257+0000 7efe16820640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1010d380 0x7efe100ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.257+0000 7efe16820640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efe10112be0 con 0x7efe1010d380 2024-04-25T10:33:11.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.257+0000 7efe16820640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efe10112a60 con 0x7efe10108a20 2024-04-25T10:33:11.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.257+0000 7efe16820640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe10112d60 con 0x7efe10102af0 2024-04-25T10:33:11.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.257+0000 7efe14dbd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1010d380 0x7efe100ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.257+0000 7efe14dbd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1010d380 0x7efe100ffcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42008/0 (socket says 172.21.15.43:42008) 2024-04-25T10:33:11.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.257+0000 7efe14dbd640 1 -- 172.21.15.43:0/201807771 learned_addr learned my addr 172.21.15.43:0/201807771 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:11.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.257+0000 7efe0ffff640 1 --2- 172.21.15.43:0/201807771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe10102af0 0x7efe100ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.257+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/201807771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe10108a20 0x7efe100ff790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.258+0000 7efe14dbd640 1 -- 172.21.15.43:0/201807771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe10102af0 msgr2=0x7efe100ff250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:11.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.258+0000 7efe14dbd640 1 --2- 172.21.15.43:0/201807771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe10102af0 0x7efe100ff250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.258+0000 7efe14dbd640 1 -- 172.21.15.43:0/201807771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe10108a20 msgr2=0x7efe100ff790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:11.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.258+0000 7efe14dbd640 1 --2- 172.21.15.43:0/201807771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe10108a20 0x7efe100ff790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.258+0000 7efe14dbd640 1 -- 172.21.15.43:0/201807771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe101a8ab0 con 0x7efe1010d380 2024-04-25T10:33:11.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.258+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/201807771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe10108a20 0x7efe100ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:11.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.258+0000 7efe14dbd640 1 --2- 172.21.15.43:0/201807771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1010d380 0x7efe100ffcd0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efe0800eb10 tx=0x7efe0800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:11.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.258+0000 7efe0d7fa640 1 -- 172.21.15.43:0/201807771 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08004b20 con 0x7efe1010d380 2024-04-25T10:33:11.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.258+0000 7efe0d7fa640 1 -- 172.21.15.43:0/201807771 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe08004cc0 con 0x7efe1010d380 2024-04-25T10:33:11.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.258+0000 7efe16820640 1 -- 172.21.15.43:0/201807771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe101a8da0 con 0x7efe1010d380 2024-04-25T10:33:11.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.258+0000 7efe0d7fa640 1 -- 172.21.15.43:0/201807771 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08010670 con 0x7efe1010d380 2024-04-25T10:33:11.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.258+0000 7efe16820640 1 -- 172.21.15.43:0/201807771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe101a91b0 con 0x7efe1010d380 2024-04-25T10:33:11.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.260+0000 7efe1d7ab740 1 Processor -- start 2024-04-25T10:33:11.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.260+0000 7efe0d7fa640 1 -- 172.21.15.43:0/201807771 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 147804+0+0 (secure 0 0 0) 0x7efe08005020 con 0x7efe1010d380 2024-04-25T10:33:11.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.261+0000 7efe1d7ab740 1 -- start start 2024-04-25T10:33:11.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.261+0000 7efe1d7ab740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8b14c5390 0x55f8b1866810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.261+0000 7efe1d7ab740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b149c4d0 0x55f8b1866d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.261+0000 7efe1d7ab740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 0x55f8b18736a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.261+0000 7efe1d7ab740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f8b17cbd80 con 0x55f8b149c4d0 2024-04-25T10:33:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.261+0000 7efe1d7ab740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f8b1785a20 con 0x55f8b14c5390 2024-04-25T10:33:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.261+0000 7efe1d7ab740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f8b17c7d40 con 0x55f8b14a46b0 2024-04-25T10:33:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.261+0000 7efe0ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b149c4d0 0x55f8b1866d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.261+0000 7efe0ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b149c4d0 0x55f8b1866d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42014/0 (socket says 172.21.15.43:42014) 2024-04-25T10:33:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.261+0000 7efe0ffff640 1 -- 172.21.15.43:0/3543751641 learned_addr learned my addr 172.21.15.43:0/3543751641 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.261+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/3543751641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 0x55f8b18736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.261+0000 7efe0d7fa640 1 --2- 172.21.15.43:0/201807771 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efdf80afe10 0x7efdf80b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.261+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/3543751641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8b14c5390 0x55f8b1866810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.261+0000 7efe14dbd640 1 --2- 172.21.15.43:0/201807771 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efdf80afe10 0x7efdf80b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.262+0000 7efe0ffff640 1 -- 172.21.15.43:0/3543751641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 msgr2=0x55f8b18736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.262+0000 7efe0ffff640 1 --2- 172.21.15.43:0/3543751641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 0x55f8b18736a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.262+0000 7efe0ffff640 1 -- 172.21.15.43:0/3543751641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8b14c5390 msgr2=0x55f8b1866810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.262+0000 7efe0ffff640 1 --2- 172.21.15.43:0/3543751641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8b14c5390 0x55f8b1866810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.262+0000 7efe0ffff640 1 -- 172.21.15.43:0/3543751641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8b1790c30 con 0x55f8b149c4d0 2024-04-25T10:33:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.262+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/3543751641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 0x55f8b18736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.262+0000 7efe0d7fa640 1 -- 172.21.15.43:0/201807771 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7efe080df900 con 0x7efe1010d380 2024-04-25T10:33:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.262+0000 7efe0ffff640 1 --2- 172.21.15.43:0/3543751641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b149c4d0 0x55f8b1866d50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7efe0000b740 tx=0x7efe0000bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bba1b435fd8ba7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.262+0000 7efe14dbd640 1 --2- 172.21.15.43:0/201807771 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efdf80afe10 0x7efdf80b2320 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efe080dcea0 tx=0x7efe080ca040 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.262+0000 7efdf67fc640 1 -- 172.21.15.43:0/3543751641 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe00003dd0 con 0x55f8b149c4d0 2024-04-25T10:33:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.262+0000 7efdf67fc640 1 -- 172.21.15.43:0/3543751641 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe00015070 con 0x55f8b149c4d0 2024-04-25T10:33:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.262+0000 7efdf67fc640 1 -- 172.21.15.43:0/3543751641 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe00004cc0 con 0x55f8b149c4d0 2024-04-25T10:33:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.263+0000 7efe1d7ab740 1 -- 172.21.15.43:0/3543751641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b149c4d0 msgr2=0x55f8b1866d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.263+0000 7efe1d7ab740 1 --2- 172.21.15.43:0/3543751641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b149c4d0 0x55f8b1866d50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7efe0000b740 tx=0x7efe0000bc00 comp rx=0 tx=0).stop 2024-04-25T10:33:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.263+0000 7efe1d7ab740 1 -- 172.21.15.43:0/3543751641 shutdown_connections 2024-04-25T10:33:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.263+0000 7efe1d7ab740 1 --2- 172.21.15.43:0/3543751641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 0x55f8b18736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.263+0000 7efe1d7ab740 1 --2- 172.21.15.43:0/3543751641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b149c4d0 0x55f8b1866d50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.263+0000 7efe1d7ab740 1 --2- 172.21.15.43:0/3543751641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8b14c5390 0x55f8b1866810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.263+0000 7efe1d7ab740 1 -- 172.21.15.43:0/3543751641 >> 172.21.15.43:0/3543751641 conn(0x55f8b14b94e0 msgr2=0x55f8b1862fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.263+0000 7efe1d7ab740 1 -- 172.21.15.43:0/3543751641 shutdown_connections 2024-04-25T10:33:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.263+0000 7efe1d7ab740 1 -- 172.21.15.43:0/3543751641 wait complete. 2024-04-25T10:33:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.263+0000 7efe1d7ab740 1 -- start start 2024-04-25T10:33:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.264+0000 7efe1d7ab740 1 Processor -- start 2024-04-25T10:33:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.264+0000 7efe1d7ab740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8b149c4d0 0x55f8b18712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.264+0000 7efe1d7ab740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 0x55f8b18717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.264+0000 7efe1d7ab740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b14c5390 0x55f8b187d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.264+0000 7efe1d7ab740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f8b1873d90 con 0x55f8b14c5390 2024-04-25T10:33:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.264+0000 7efe1d7ab740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f8b1752dc0 con 0x55f8b149c4d0 2024-04-25T10:33:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.264+0000 7efe1d7ab740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f8b1873f10 con 0x55f8b14a46b0 2024-04-25T10:33:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.264+0000 7efe1d7ab740 10 client.0 _ll_register_callbacks cb 0x55f8b176e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.264+0000 7efe0f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b14c5390 0x55f8b187d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.264+0000 7efe0f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b14c5390 0x55f8b187d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42022/0 (socket says 172.21.15.43:42022) 2024-04-25T10:33:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.264+0000 7efe0f7fe640 1 -- 172.21.15.43:0/403180331 learned_addr learned my addr 172.21.15.43:0/403180331 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.264+0000 7efe0ffff640 1 --2- 172.21.15.43:0/403180331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 0x55f8b18717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.264+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/403180331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8b149c4d0 0x55f8b18712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.264+0000 7efe0f7fe640 1 -- 172.21.15.43:0/403180331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 msgr2=0x55f8b18717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.264+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/403180331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 0x55f8b18717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.264+0000 7efe0f7fe640 1 -- 172.21.15.43:0/403180331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8b149c4d0 msgr2=0x55f8b18712b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.264+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/403180331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8b149c4d0 0x55f8b18712b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.264+0000 7efe0f7fe640 1 -- 172.21.15.43:0/403180331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8b14dd3f0 con 0x55f8b14c5390 2024-04-25T10:33:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.264+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/403180331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b14c5390 0x55f8b187d210 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7efe04009900 tx=0x7efe04009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64a1b819d82001b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.264+0000 7efdef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.265+0000 7efdef7fe640 1 -- 172.21.15.43:0/403180331 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04035040 con 0x55f8b14c5390 2024-04-25T10:33:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.265+0000 7efdef7fe640 1 -- 172.21.15.43:0/403180331 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7efe04004260 con 0x55f8b14c5390 2024-04-25T10:33:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.265+0000 7efe1d7ab740 1 -- 172.21.15.43:0/403180331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe00015070 con 0x55f8b14c5390 2024-04-25T10:33:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.265+0000 7efdef7fe640 1 -- 172.21.15.43:0/403180331 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7efe04053050 con 0x55f8b14c5390 2024-04-25T10:33:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.265+0000 7efe1d7ab740 10 client.15654 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.265+0000 7efe1d7ab740 1 -- 172.21.15.43:0/403180331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f8b15a77d0 con 0x55f8b14c5390 2024-04-25T10:33:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.265+0000 7efdef7fe640 1 -- 172.21.15.43:0/403180331 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe00015070 con 0x55f8b14c5390 2024-04-25T10:33:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.265+0000 7efdef7fe640 1 -- 172.21.15.43:0/403180331 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7efe0404a070 con 0x55f8b14c5390 2024-04-25T10:33:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.265+0000 7efe1d7ab740 10 client.15654 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.265+0000 7efe1d7ab740 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.265+0000 7efe1d7ab740 20 client.15654 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:11.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.265+0000 7efe1d7ab740 1 --2- 172.21.15.43:0/403180331 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f8b1483f60 0x55f8b1883690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.265+0000 7efe1d7ab740 4 client.15654 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:33:11.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.265+0000 7efe1d7ab740 1 -- 172.21.15.43:0/403180331 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8b17d9990 con 0x55f8b1483f60 2024-04-25T10:33:11.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.266+0000 7efe0ffff640 1 --2- 172.21.15.43:0/403180331 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f8b1483f60 0x55f8b1883690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.266+0000 7efdef7fe640 10 client.15654 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:11.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.266+0000 7efe0ffff640 1 --2- 172.21.15.43:0/403180331 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f8b1483f60 0x55f8b1883690 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1114b50e88dafc4 server_cookie=9be41cba4aa628dc in_seq=0 out_seq=0 2024-04-25T10:33:11.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.267+0000 7efdef7fe640 1 -- 172.21.15.43:0/403180331 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f8b17d9990 con 0x55f8b1483f60 2024-04-25T10:33:11.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.267+0000 7efdef7fe640 10 client.15654 handle_command_reply: tid=0 2024-04-25T10:33:11.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.267+0000 7efe1d7ab740 1 client.15654 shutdown 2024-04-25T10:33:11.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.267+0000 7efe1d7ab740 1 -- 172.21.15.43:0/403180331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b14c5390 msgr2=0x55f8b187d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:11.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.267+0000 7efe1d7ab740 1 --2- 172.21.15.43:0/403180331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b14c5390 0x55f8b187d210 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7efe04009900 tx=0x7efe04009260 comp rx=0 tx=0).stop 2024-04-25T10:33:11.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.267+0000 7efe1d7ab740 1 -- 172.21.15.43:0/403180331 shutdown_connections 2024-04-25T10:33:11.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.267+0000 7efe1d7ab740 1 --2- 172.21.15.43:0/403180331 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f8b1483f60 0x55f8b1883690 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.267+0000 7efe1d7ab740 1 --2- 172.21.15.43:0/403180331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b14c5390 0x55f8b187d210 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.267+0000 7efe1d7ab740 1 --2- 172.21.15.43:0/403180331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 0x55f8b18717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.267+0000 7efdef7fe640 0 client.15654 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:11.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.267+0000 7efe1d7ab740 1 --2- 172.21.15.43:0/403180331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8b149c4d0 0x55f8b18712b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.267+0000 7efe1d7ab740 1 -- 172.21.15.43:0/403180331 >> 172.21.15.43:0/403180331 conn(0x55f8b14b94e0 msgr2=0x55f8b1868530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:11.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.267+0000 7efe1d7ab740 1 -- 172.21.15.43:0/403180331 shutdown_connections 2024-04-25T10:33:11.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.267+0000 7efe1d7ab740 1 -- 172.21.15.43:0/403180331 wait complete. 2024-04-25T10:33:11.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.267+0000 7efe1d7ab740 20 client.15654 trim_cache size 0 max 16384 2024-04-25T10:33:11.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.281+0000 7efe1d7ab740 1 Processor -- start 2024-04-25T10:33:11.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.281+0000 7efe1d7ab740 1 -- start start 2024-04-25T10:33:11.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.281+0000 7efe1d7ab740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b149c4d0 0x55f8b186c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.281+0000 7efe1d7ab740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 0x55f8b187ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.281+0000 7efe1d7ab740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8b14c5390 0x55f8b1870980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.281+0000 7efe1d7ab740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efe101b4240 con 0x55f8b149c4d0 2024-04-25T10:33:11.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.281+0000 7efe1d7ab740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efe101b1b10 con 0x55f8b14c5390 2024-04-25T10:33:11.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.281+0000 7efe1d7ab740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe101af960 con 0x55f8b14a46b0 2024-04-25T10:33:11.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.281+0000 7efe0f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b149c4d0 0x55f8b186c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.282+0000 7efe0f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b149c4d0 0x55f8b186c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42028/0 (socket says 172.21.15.43:42028) 2024-04-25T10:33:11.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.282+0000 7efe0f7fe640 1 -- 172.21.15.43:0/707554276 learned_addr learned my addr 172.21.15.43:0/707554276 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:11.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.282+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/707554276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8b14c5390 0x55f8b1870980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.282+0000 7efe0ffff640 1 --2- 172.21.15.43:0/707554276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 0x55f8b187ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.282+0000 7efe0f7fe640 1 -- 172.21.15.43:0/707554276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 msgr2=0x55f8b187ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:11.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.282+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/707554276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 0x55f8b187ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.282+0000 7efe0f7fe640 1 -- 172.21.15.43:0/707554276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8b14c5390 msgr2=0x55f8b1870980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:11.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.282+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/707554276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8b14c5390 0x55f8b1870980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.282+0000 7efe0f7fe640 1 -- 172.21.15.43:0/707554276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8b185d2b0 con 0x55f8b149c4d0 2024-04-25T10:33:11.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.282+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/707554276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b149c4d0 0x55f8b186c8f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7efe0402a930 tx=0x7efe0402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6028568873b11a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:11.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.282+0000 7efdf5ffb640 1 -- 172.21.15.43:0/707554276 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0405c070 con 0x55f8b149c4d0 2024-04-25T10:33:11.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.282+0000 7efdf5ffb640 1 -- 172.21.15.43:0/707554276 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe040319c0 con 0x55f8b149c4d0 2024-04-25T10:33:11.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.282+0000 7efdf5ffb640 1 -- 172.21.15.43:0/707554276 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04035070 con 0x55f8b149c4d0 2024-04-25T10:33:11.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.283+0000 7efe1d7ab740 1 -- 172.21.15.43:0/707554276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b149c4d0 msgr2=0x55f8b186c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:11.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.283+0000 7efe1d7ab740 1 --2- 172.21.15.43:0/707554276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b149c4d0 0x55f8b186c8f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7efe0402a930 tx=0x7efe0402a410 comp rx=0 tx=0).stop 2024-04-25T10:33:11.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.283+0000 7efe1d7ab740 1 -- 172.21.15.43:0/707554276 shutdown_connections 2024-04-25T10:33:11.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.283+0000 7efe1d7ab740 1 --2- 172.21.15.43:0/707554276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8b14c5390 0x55f8b1870980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.283+0000 7efe1d7ab740 1 --2- 172.21.15.43:0/707554276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 0x55f8b187ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.283+0000 7efe1d7ab740 1 --2- 172.21.15.43:0/707554276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b149c4d0 0x55f8b186c8f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.283+0000 7efe1d7ab740 1 -- 172.21.15.43:0/707554276 >> 172.21.15.43:0/707554276 conn(0x55f8b14b94e0 msgr2=0x55f8b1868410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:11.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.283+0000 7efe1d7ab740 1 -- 172.21.15.43:0/707554276 shutdown_connections 2024-04-25T10:33:11.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.283+0000 7efe1d7ab740 1 -- 172.21.15.43:0/707554276 wait complete. 2024-04-25T10:33:11.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.283+0000 7efe1d7ab740 1 -- start start 2024-04-25T10:33:11.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.283+0000 7efe1d7ab740 1 Processor -- start 2024-04-25T10:33:11.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.284+0000 7efe1d7ab740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8b149c4d0 0x55f8b18747f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.284+0000 7efe1d7ab740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 0x55f8b1874d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.284+0000 7efe1d7ab740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b14c5390 0x55f8b18e39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.284+0000 7efe1d7ab740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efe101affd0 con 0x55f8b14c5390 2024-04-25T10:33:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.284+0000 7efe1d7ab740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efe101b1440 con 0x55f8b149c4d0 2024-04-25T10:33:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.284+0000 7efe1d7ab740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe101b06a0 con 0x55f8b14a46b0 2024-04-25T10:33:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.284+0000 7efe1d7ab740 10 client.0 _ll_register_callbacks cb 0x7efe04009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.284+0000 7efe0f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b14c5390 0x55f8b18e39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.284+0000 7efe0f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b14c5390 0x55f8b18e39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42030/0 (socket says 172.21.15.43:42030) 2024-04-25T10:33:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.284+0000 7efe0f7fe640 1 -- 172.21.15.43:0/1604350856 learned_addr learned my addr 172.21.15.43:0/1604350856 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.284+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/1604350856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8b149c4d0 0x55f8b18747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.284+0000 7efe0ffff640 1 --2- 172.21.15.43:0/1604350856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 0x55f8b1874d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.284+0000 7efe0f7fe640 1 -- 172.21.15.43:0/1604350856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 msgr2=0x55f8b1874d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:11.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.284+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/1604350856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 0x55f8b1874d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.284+0000 7efe0f7fe640 1 -- 172.21.15.43:0/1604350856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8b149c4d0 msgr2=0x55f8b18747f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:11.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.284+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/1604350856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8b149c4d0 0x55f8b18747f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.284+0000 7efe0f7fe640 1 -- 172.21.15.43:0/1604350856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8b18e3f30 con 0x55f8b14c5390 2024-04-25T10:33:11.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.284+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/1604350856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b14c5390 0x55f8b18e39c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7efe0403d5f0 tx=0x7efe04031530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c792505b8f72498 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:11.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.284+0000 7efdef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:11.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.285+0000 7efdef7fe640 1 -- 172.21.15.43:0/1604350856 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04035030 con 0x55f8b14c5390 2024-04-25T10:33:11.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.285+0000 7efdef7fe640 1 -- 172.21.15.43:0/1604350856 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7efe0403dd60 con 0x55f8b14c5390 2024-04-25T10:33:11.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.285+0000 7efe1d7ab740 1 -- 172.21.15.43:0/1604350856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe040319c0 con 0x55f8b14c5390 2024-04-25T10:33:11.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.285+0000 7efdef7fe640 1 -- 172.21.15.43:0/1604350856 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7efe0405c040 con 0x55f8b14c5390 2024-04-25T10:33:11.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.285+0000 7efe1d7ab740 10 client.15660 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:11.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.285+0000 7efe1d7ab740 1 -- 172.21.15.43:0/1604350856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f8b18eb1f0 con 0x55f8b14c5390 2024-04-25T10:33:11.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.285+0000 7efdef7fe640 1 -- 172.21.15.43:0/1604350856 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe040319c0 con 0x55f8b14c5390 2024-04-25T10:33:11.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.285+0000 7efdef7fe640 1 -- 172.21.15.43:0/1604350856 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7efe04062040 con 0x55f8b14c5390 2024-04-25T10:33:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.285+0000 7efe1d7ab740 10 client.15660 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.285+0000 7efe1d7ab740 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.285+0000 7efe1d7ab740 20 client.15660 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.285+0000 7efe1d7ab740 1 --2- 172.21.15.43:0/1604350856 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f8b14593a0 0x55f8b18ed8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.285+0000 7efe1d7ab740 4 client.15660 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:33:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.285+0000 7efe1d7ab740 1 -- 172.21.15.43:0/1604350856 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f8b17d3800 con 0x55f8b14593a0 2024-04-25T10:33:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.285+0000 7efe0ffff640 1 --2- 172.21.15.43:0/1604350856 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f8b14593a0 0x55f8b18ed8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.286+0000 7efe0ffff640 1 --2- 172.21.15.43:0/1604350856 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f8b14593a0 0x55f8b18ed8f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acf5ed265773ff3c server_cookie=e948cd3a3013c231 in_seq=0 out_seq=0 2024-04-25T10:33:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.286+0000 7efdef7fe640 10 client.15660 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.286+0000 7efdef7fe640 1 -- 172.21.15.43:0/1604350856 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f8b17d3800 con 0x55f8b14593a0 2024-04-25T10:33:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.286+0000 7efdef7fe640 10 client.15660 handle_command_reply: tid=0 2024-04-25T10:33:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.286+0000 7efe1d7ab740 1 client.15660 shutdown 2024-04-25T10:33:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.286+0000 7efe1d7ab740 1 -- 172.21.15.43:0/1604350856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b14c5390 msgr2=0x55f8b18e39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.286+0000 7efe1d7ab740 1 --2- 172.21.15.43:0/1604350856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b14c5390 0x55f8b18e39c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7efe0403d5f0 tx=0x7efe04031530 comp rx=0 tx=0).stop 2024-04-25T10:33:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.286+0000 7efe1d7ab740 1 -- 172.21.15.43:0/1604350856 shutdown_connections 2024-04-25T10:33:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.286+0000 7efe1d7ab740 1 --2- 172.21.15.43:0/1604350856 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f8b14593a0 0x55f8b18ed8f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.286+0000 7efe1d7ab740 1 --2- 172.21.15.43:0/1604350856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8b14c5390 0x55f8b18e39c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.286+0000 7efe1d7ab740 1 --2- 172.21.15.43:0/1604350856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8b14a46b0 0x55f8b1874d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.286+0000 7efe1d7ab740 1 --2- 172.21.15.43:0/1604350856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8b149c4d0 0x55f8b18747f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.286+0000 7efdef7fe640 0 client.15660 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.286+0000 7efe1d7ab740 1 -- 172.21.15.43:0/1604350856 >> 172.21.15.43:0/1604350856 conn(0x55f8b14b94e0 msgr2=0x55f8b1868410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.286+0000 7efe1d7ab740 1 -- 172.21.15.43:0/1604350856 shutdown_connections 2024-04-25T10:33:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.286+0000 7efe1d7ab740 1 -- 172.21.15.43:0/1604350856 wait complete. 2024-04-25T10:33:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.287+0000 7efe1d7ab740 20 client.15660 trim_cache size 0 max 16384 2024-04-25T10:33:11.297 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:33:11.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.287+0000 7efe16820640 1 -- 172.21.15.43:0/201807771 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efdf80afe10 msgr2=0x7efdf80b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:11.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.287+0000 7efe16820640 1 --2- 172.21.15.43:0/201807771 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efdf80afe10 0x7efdf80b2320 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efe080dcea0 tx=0x7efe080ca040 comp rx=0 tx=0).stop 2024-04-25T10:33:11.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.287+0000 7efe16820640 1 -- 172.21.15.43:0/201807771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1010d380 msgr2=0x7efe100ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:11.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.287+0000 7efe16820640 1 --2- 172.21.15.43:0/201807771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1010d380 0x7efe100ffcd0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efe0800eb10 tx=0x7efe0800c2d0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.287+0000 7efe16820640 1 -- 172.21.15.43:0/201807771 shutdown_connections 2024-04-25T10:33:11.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.287+0000 7efe16820640 1 --2- 172.21.15.43:0/201807771 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efdf80afe10 0x7efdf80b2320 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.287+0000 7efe16820640 1 --2- 172.21.15.43:0/201807771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe10108a20 0x7efe100ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.287+0000 7efe16820640 1 --2- 172.21.15.43:0/201807771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1010d380 0x7efe100ffcd0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.287+0000 7efe16820640 1 --2- 172.21.15.43:0/201807771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe10102af0 0x7efe100ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.287+0000 7efe16820640 1 -- 172.21.15.43:0/201807771 >> 172.21.15.43:0/201807771 conn(0x7efe10078100 msgr2=0x7efe10069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:11.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.287+0000 7efe16820640 1 -- 172.21.15.43:0/201807771 shutdown_connections 2024-04-25T10:33:11.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:11.287+0000 7efe16820640 1 -- 172.21.15.43:0/201807771 wait complete. 2024-04-25T10:33:11.299 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:10 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/191758659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:33:11.300 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:10 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/191758659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:33:12.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:11 smithi077 ceph-mon[32772]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T10:33:12.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:11 smithi164 ceph-mon[32713]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T10:33:12.301 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:33:12.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:33:12.301 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:33:12.328 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:11 smithi043 ceph-mon[29478]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T10:33:12.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.430+0000 7f26a23ce640 1 -- 172.21.15.43:0/317416673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269c109fb0 msgr2=0x7f269c103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:12.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.430+0000 7f26a23ce640 1 --2- 172.21.15.43:0/317416673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269c109fb0 0x7f269c103bf0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2690009900 tx=0x7f269002a410 comp rx=0 tx=0).stop 2024-04-25T10:33:12.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.430+0000 7f26a23ce640 1 -- 172.21.15.43:0/317416673 shutdown_connections 2024-04-25T10:33:12.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.430+0000 7f26a23ce640 1 --2- 172.21.15.43:0/317416673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f269c104260 0x7f269c110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.430+0000 7f26a23ce640 1 --2- 172.21.15.43:0/317416673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269c109fb0 0x7f269c103bf0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.430+0000 7f26a23ce640 1 --2- 172.21.15.43:0/317416673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f269c109b10 0x7f269c1036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.430+0000 7f26a23ce640 1 -- 172.21.15.43:0/317416673 >> 172.21.15.43:0/317416673 conn(0x7f269c0ff210 msgr2=0x7f269c101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:12.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.430+0000 7f26a23ce640 1 -- 172.21.15.43:0/317416673 shutdown_connections 2024-04-25T10:33:12.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.430+0000 7f26a23ce640 1 -- 172.21.15.43:0/317416673 wait complete. 2024-04-25T10:33:12.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.431+0000 7f26a23ce640 1 Processor -- start 2024-04-25T10:33:12.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.431+0000 7f26a23ce640 1 -- start start 2024-04-25T10:33:12.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.431+0000 7f26a23ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269c104260 0x7f269c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.431+0000 7f26a23ce640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f269c109b10 0x7f269c19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.431+0000 7f26a23ce640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f269c109fb0 0x7f269c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.431+0000 7f26a23ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f269c113d00 con 0x7f269c104260 2024-04-25T10:33:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.431+0000 7f26a23ce640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f269c113b80 con 0x7f269c109fb0 2024-04-25T10:33:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.431+0000 7f26a23ce640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f269c113e80 con 0x7f269c109b10 2024-04-25T10:33:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.432+0000 7f269bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269c104260 0x7f269c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.432+0000 7f269b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f269c109b10 0x7f269c19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.432+0000 7f269bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269c104260 0x7f269c19a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42052/0 (socket says 172.21.15.43:42052) 2024-04-25T10:33:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.432+0000 7f269bfff640 1 -- 172.21.15.43:0/4163130220 learned_addr learned my addr 172.21.15.43:0/4163130220 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.432+0000 7f26a096b640 1 --2- 172.21.15.43:0/4163130220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f269c109fb0 0x7f269c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.432+0000 7f269bfff640 1 -- 172.21.15.43:0/4163130220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f269c109b10 msgr2=0x7f269c19aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.432+0000 7f269bfff640 1 --2- 172.21.15.43:0/4163130220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f269c109b10 0x7f269c19aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.432+0000 7f269bfff640 1 -- 172.21.15.43:0/4163130220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f269c109fb0 msgr2=0x7f269c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:12.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.432+0000 7f269bfff640 1 --2- 172.21.15.43:0/4163130220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f269c109fb0 0x7f269c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.432+0000 7f269bfff640 1 -- 172.21.15.43:0/4163130220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269c19f5a0 con 0x7f269c104260 2024-04-25T10:33:12.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.432+0000 7f269bfff640 1 --2- 172.21.15.43:0/4163130220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269c104260 0x7f269c19a740 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f268c00e9c0 tx=0x7f268c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:12.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.432+0000 7f26997fa640 1 -- 172.21.15.43:0/4163130220 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c00cd80 con 0x7f269c104260 2024-04-25T10:33:12.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.432+0000 7f26997fa640 1 -- 172.21.15.43:0/4163130220 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f268c0054d0 con 0x7f269c104260 2024-04-25T10:33:12.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.432+0000 7f26a23ce640 1 -- 172.21.15.43:0/4163130220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f269c19f890 con 0x7f269c104260 2024-04-25T10:33:12.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.432+0000 7f26997fa640 1 -- 172.21.15.43:0/4163130220 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c010430 con 0x7f269c104260 2024-04-25T10:33:12.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.432+0000 7f26a23ce640 1 -- 172.21.15.43:0/4163130220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f269c1a7210 con 0x7f269c104260 2024-04-25T10:33:12.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.435+0000 7f26997fa640 1 -- 172.21.15.43:0/4163130220 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 147804+0+0 (secure 0 0 0) 0x7f268c004fd0 con 0x7f269c104260 2024-04-25T10:33:12.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.435+0000 7f26a927d740 1 Processor -- start 2024-04-25T10:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.435+0000 7f26a927d740 1 -- start start 2024-04-25T10:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.435+0000 7f26a927d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea51410 0x55d52edf14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.435+0000 7f26a927d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea28550 0x55d52edf19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.435+0000 7f26a927d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52ea30730 0x55d52edf1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.435+0000 7f26a927d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d52ed57b00 con 0x55d52ea51410 2024-04-25T10:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.435+0000 7f26a927d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d52ecf3920 con 0x55d52ea28550 2024-04-25T10:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.435+0000 7f26a927d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d52ed53ac0 con 0x55d52ea30730 2024-04-25T10:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.435+0000 7f26a096b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea51410 0x55d52edf14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.436+0000 7f26a096b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea51410 0x55d52edf14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42056/0 (socket says 172.21.15.43:42056) 2024-04-25T10:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.436+0000 7f26a096b640 1 -- 172.21.15.43:0/1593574889 learned_addr learned my addr 172.21.15.43:0/1593574889 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:12.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.436+0000 7f269b7fe640 1 --2- 172.21.15.43:0/1593574889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52ea30730 0x55d52edf1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.436+0000 7f269bfff640 1 --2- 172.21.15.43:0/1593574889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea28550 0x55d52edf19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.436+0000 7f26997fa640 1 --2- 172.21.15.43:0/4163130220 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26840afe90 0x7f26840b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.436+0000 7f26a096b640 1 -- 172.21.15.43:0/1593574889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52ea30730 msgr2=0x55d52edf1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:12.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.436+0000 7f26a096b640 1 --2- 172.21.15.43:0/1593574889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52ea30730 0x55d52edf1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.436+0000 7f26a096b640 1 -- 172.21.15.43:0/1593574889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea28550 msgr2=0x55d52edf19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:12.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.436+0000 7f26a096b640 1 --2- 172.21.15.43:0/1593574889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea28550 0x55d52edf19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.436+0000 7f26a096b640 1 -- 172.21.15.43:0/1593574889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d52eb317e0 con 0x55d52ea51410 2024-04-25T10:33:12.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.436+0000 7f269bfff640 1 --2- 172.21.15.43:0/1593574889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea28550 0x55d52edf19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:12.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.436+0000 7f26a096b640 1 --2- 172.21.15.43:0/4163130220 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26840afe90 0x7f26840b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.436+0000 7f26997fa640 1 -- 172.21.15.43:0/4163130220 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f268c0df5f0 con 0x7f269c104260 2024-04-25T10:33:12.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.436+0000 7f26a096b640 1 --2- 172.21.15.43:0/1593574889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea51410 0x55d52edf14b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2694010510 tx=0x7f2694009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2badfc81a881f209 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:12.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.436+0000 7f26827fc640 1 -- 172.21.15.43:0/1593574889 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694023070 con 0x55d52ea51410 2024-04-25T10:33:12.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.436+0000 7f26827fc640 1 -- 172.21.15.43:0/1593574889 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f269401dab0 con 0x55d52ea51410 2024-04-25T10:33:12.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.437+0000 7f26827fc640 1 -- 172.21.15.43:0/1593574889 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269401c820 con 0x55d52ea51410 2024-04-25T10:33:12.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.437+0000 7f26a927d740 1 -- 172.21.15.43:0/1593574889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea51410 msgr2=0x55d52edf14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:12.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.437+0000 7f26a096b640 1 --2- 172.21.15.43:0/4163130220 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26840afe90 0x7f26840b23a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2694020000 tx=0x7f269401b000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:12.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.437+0000 7f26a927d740 1 --2- 172.21.15.43:0/1593574889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea51410 0x55d52edf14b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2694010510 tx=0x7f2694009f90 comp rx=0 tx=0).stop 2024-04-25T10:33:12.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.437+0000 7f26a927d740 1 -- 172.21.15.43:0/1593574889 shutdown_connections 2024-04-25T10:33:12.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.437+0000 7f26a927d740 1 --2- 172.21.15.43:0/1593574889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea28550 0x55d52edf19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.437+0000 7f26a927d740 1 --2- 172.21.15.43:0/1593574889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52ea30730 0x55d52edf1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.437+0000 7f26a927d740 1 --2- 172.21.15.43:0/1593574889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea51410 0x55d52edf14b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.437+0000 7f26a927d740 1 -- 172.21.15.43:0/1593574889 >> 172.21.15.43:0/1593574889 conn(0x55d52ea45560 msgr2=0x55d52ededdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:12.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.437+0000 7f26a927d740 1 -- 172.21.15.43:0/1593574889 shutdown_connections 2024-04-25T10:33:12.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.437+0000 7f26a927d740 1 -- 172.21.15.43:0/1593574889 wait complete. 2024-04-25T10:33:12.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.438+0000 7f26a927d740 1 -- start start 2024-04-25T10:33:12.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.438+0000 7f26a927d740 1 Processor -- start 2024-04-25T10:33:12.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.438+0000 7f26a927d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52ea28550 0x55d52ee00070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.438+0000 7f26a927d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea30730 0x55d52ee005b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.438+0000 7f26a927d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea51410 0x55d52ee0c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.438+0000 7f26a927d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d52ee02d80 con 0x55d52ea51410 2024-04-25T10:33:12.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.438+0000 7f26a927d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d52ecf15d0 con 0x55d52ea30730 2024-04-25T10:33:12.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.438+0000 7f26a927d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d52ee02f00 con 0x55d52ea28550 2024-04-25T10:33:12.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.439+0000 7f26a927d740 10 client.0 _ll_register_callbacks cb 0x55d52ecdf080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:12.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.439+0000 7f26a096b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea51410 0x55d52ee0c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.439+0000 7f26a096b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea51410 0x55d52ee0c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42064/0 (socket says 172.21.15.43:42064) 2024-04-25T10:33:12.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.439+0000 7f26a096b640 1 -- 172.21.15.43:0/285831726 learned_addr learned my addr 172.21.15.43:0/285831726 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:12.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.439+0000 7f269bfff640 1 --2- 172.21.15.43:0/285831726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52ea28550 0x55d52ee00070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.439+0000 7f269b7fe640 1 --2- 172.21.15.43:0/285831726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea30730 0x55d52ee005b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.439+0000 7f26a096b640 1 -- 172.21.15.43:0/285831726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52ea28550 msgr2=0x55d52ee00070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:12.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.439+0000 7f26a096b640 1 --2- 172.21.15.43:0/285831726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52ea28550 0x55d52ee00070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.439+0000 7f26a096b640 1 -- 172.21.15.43:0/285831726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea30730 msgr2=0x55d52ee005b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:12.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.439+0000 7f26a096b640 1 --2- 172.21.15.43:0/285831726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea30730 0x55d52ee005b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.439+0000 7f26a096b640 1 -- 172.21.15.43:0/285831726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d52eb91f70 con 0x55d52ea51410 2024-04-25T10:33:12.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.439+0000 7f269b7fe640 1 --2- 172.21.15.43:0/285831726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea30730 0x55d52ee005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:12.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.439+0000 7f26a096b640 1 --2- 172.21.15.43:0/285831726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea51410 0x55d52ee0c230 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2694009640 tx=0x7f2694005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8393ccfcaaa0c8ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:12.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.439+0000 7f267b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:12.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.439+0000 7f267b7fe640 1 -- 172.21.15.43:0/285831726 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694023050 con 0x55d52ea51410 2024-04-25T10:33:12.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.439+0000 7f267b7fe640 1 -- 172.21.15.43:0/285831726 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f2694003b70 con 0x55d52ea51410 2024-04-25T10:33:12.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.439+0000 7f26a927d740 1 -- 172.21.15.43:0/285831726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f269401dab0 con 0x55d52ea51410 2024-04-25T10:33:12.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.440+0000 7f267b7fe640 1 -- 172.21.15.43:0/285831726 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2694003d70 con 0x55d52ea51410 2024-04-25T10:33:12.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.440+0000 7f26a927d740 10 client.15672 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:12.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.440+0000 7f26a927d740 1 -- 172.21.15.43:0/285831726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d52ea29170 con 0x55d52ea51410 2024-04-25T10:33:12.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.440+0000 7f267b7fe640 1 -- 172.21.15.43:0/285831726 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f269401dab0 con 0x55d52ea51410 2024-04-25T10:33:12.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.440+0000 7f267b7fe640 1 -- 172.21.15.43:0/285831726 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f269400f040 con 0x55d52ea51410 2024-04-25T10:33:12.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.440+0000 7f26a927d740 10 client.15672 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:12.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.440+0000 7f26a927d740 10 client.15672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:12.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.440+0000 7f26a927d740 20 client.15672 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:12.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.440+0000 7f26a927d740 1 --2- 172.21.15.43:0/285831726 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d52ea101f0 0x55d52ee122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.440+0000 7f26a927d740 4 client.15672 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:33:12.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.440+0000 7f26a927d740 1 -- 172.21.15.43:0/285831726 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d52ed65710 con 0x55d52ea101f0 2024-04-25T10:33:12.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.440+0000 7f269bfff640 1 --2- 172.21.15.43:0/285831726 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d52ea101f0 0x55d52ee122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.441+0000 7f269bfff640 1 --2- 172.21.15.43:0/285831726 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d52ea101f0 0x55d52ee122a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46487759e6069b0f server_cookie=21467fde31d0359a in_seq=0 out_seq=0 2024-04-25T10:33:12.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.441+0000 7f267b7fe640 10 client.15672 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:12.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.442+0000 7f267b7fe640 1 -- 172.21.15.43:0/285831726 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d52ed65710 con 0x55d52ea101f0 2024-04-25T10:33:12.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.442+0000 7f267b7fe640 10 client.15672 handle_command_reply: tid=0 2024-04-25T10:33:12.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.442+0000 7f26a927d740 1 client.15672 shutdown 2024-04-25T10:33:12.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.442+0000 7f26a927d740 1 -- 172.21.15.43:0/285831726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea51410 msgr2=0x55d52ee0c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:12.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.442+0000 7f26a927d740 1 --2- 172.21.15.43:0/285831726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea51410 0x55d52ee0c230 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2694009640 tx=0x7f2694005f90 comp rx=0 tx=0).stop 2024-04-25T10:33:12.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.442+0000 7f26a927d740 1 -- 172.21.15.43:0/285831726 shutdown_connections 2024-04-25T10:33:12.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.442+0000 7f26a927d740 1 --2- 172.21.15.43:0/285831726 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d52ea101f0 0x55d52ee122a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.442+0000 7f26a927d740 1 --2- 172.21.15.43:0/285831726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea30730 0x55d52ee005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.442+0000 7f267b7fe640 0 client.15672 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:12.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.442+0000 7f26a927d740 1 --2- 172.21.15.43:0/285831726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea51410 0x55d52ee0c230 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.442+0000 7f26a927d740 1 --2- 172.21.15.43:0/285831726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52ea28550 0x55d52ee00070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.442+0000 7f26a927d740 1 -- 172.21.15.43:0/285831726 >> 172.21.15.43:0/285831726 conn(0x55d52ea45560 msgr2=0x55d52edf1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:12.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.442+0000 7f26a927d740 1 -- 172.21.15.43:0/285831726 shutdown_connections 2024-04-25T10:33:12.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.442+0000 7f26a927d740 1 -- 172.21.15.43:0/285831726 wait complete. 2024-04-25T10:33:12.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.442+0000 7f26a927d740 20 client.15672 trim_cache size 0 max 16384 2024-04-25T10:33:12.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.460+0000 7f26a927d740 1 Processor -- start 2024-04-25T10:33:12.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.461+0000 7f26a927d740 1 -- start start 2024-04-25T10:33:12.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.461+0000 7f26a927d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea101f0 0x55d52edf9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.461+0000 7f26a927d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea28550 0x55d52edff610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.461+0000 7f26a927d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52ea30730 0x55d52edecbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.461+0000 7f26a927d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f269c1b00f0 con 0x55d52ea28550 2024-04-25T10:33:12.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.461+0000 7f26a927d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f269c1ad9c0 con 0x55d52ea101f0 2024-04-25T10:33:12.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.461+0000 7f26a927d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f269c1ab830 con 0x55d52ea30730 2024-04-25T10:33:12.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.461+0000 7f269b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea28550 0x55d52edff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.461+0000 7f269b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea28550 0x55d52edff610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42078/0 (socket says 172.21.15.43:42078) 2024-04-25T10:33:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.461+0000 7f269b7fe640 1 -- 172.21.15.43:0/3521370340 learned_addr learned my addr 172.21.15.43:0/3521370340 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.461+0000 7f269bfff640 1 --2- 172.21.15.43:0/3521370340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea101f0 0x55d52edf9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.461+0000 7f26a096b640 1 --2- 172.21.15.43:0/3521370340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52ea30730 0x55d52edecbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.461+0000 7f269b7fe640 1 -- 172.21.15.43:0/3521370340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52ea30730 msgr2=0x55d52edecbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.461+0000 7f269b7fe640 1 --2- 172.21.15.43:0/3521370340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52ea30730 0x55d52edecbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.461+0000 7f269b7fe640 1 -- 172.21.15.43:0/3521370340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea101f0 msgr2=0x55d52edf9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.461+0000 7f269b7fe640 1 --2- 172.21.15.43:0/3521370340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea101f0 0x55d52edf9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.461+0000 7f269b7fe640 1 -- 172.21.15.43:0/3521370340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d52edf2bc0 con 0x55d52ea28550 2024-04-25T10:33:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.461+0000 7f269bfff640 1 --2- 172.21.15.43:0/3521370340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea101f0 0x55d52edf9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.461+0000 7f269b7fe640 1 --2- 172.21.15.43:0/3521370340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea28550 0x55d52edff610 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2690033000 tx=0x7f2690000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bdcd7072fe073e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.462+0000 7f2681ffb640 1 -- 172.21.15.43:0/3521370340 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2690035040 con 0x55d52ea28550 2024-04-25T10:33:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.462+0000 7f2681ffb640 1 -- 172.21.15.43:0/3521370340 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26900316a0 con 0x55d52ea28550 2024-04-25T10:33:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.462+0000 7f2681ffb640 1 -- 172.21.15.43:0/3521370340 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2690005550 con 0x55d52ea28550 2024-04-25T10:33:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.462+0000 7f26a927d740 1 -- 172.21.15.43:0/3521370340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea28550 msgr2=0x55d52edff610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.462+0000 7f26a927d740 1 --2- 172.21.15.43:0/3521370340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea28550 0x55d52edff610 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2690033000 tx=0x7f2690000bc0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.462+0000 7f26a927d740 1 -- 172.21.15.43:0/3521370340 shutdown_connections 2024-04-25T10:33:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.462+0000 7f26a927d740 1 --2- 172.21.15.43:0/3521370340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52ea30730 0x55d52edecbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.462+0000 7f26a927d740 1 --2- 172.21.15.43:0/3521370340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea28550 0x55d52edff610 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.462+0000 7f26a927d740 1 --2- 172.21.15.43:0/3521370340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea101f0 0x55d52edf9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.462+0000 7f26a927d740 1 -- 172.21.15.43:0/3521370340 >> 172.21.15.43:0/3521370340 conn(0x55d52ea45560 msgr2=0x55d52edec4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.462+0000 7f26a927d740 1 -- 172.21.15.43:0/3521370340 shutdown_connections 2024-04-25T10:33:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.462+0000 7f26a927d740 1 -- 172.21.15.43:0/3521370340 wait complete. 2024-04-25T10:33:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.463+0000 7f26a927d740 1 -- start start 2024-04-25T10:33:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.463+0000 7f26a927d740 1 Processor -- start 2024-04-25T10:33:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.463+0000 7f26a927d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea101f0 0x55d52ee0bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.463+0000 7f26a927d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea28550 0x55d52ee0c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.463+0000 7f26a927d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52ea30730 0x55d52ee0c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.463+0000 7f26a927d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f269c1abe80 con 0x55d52ea28550 2024-04-25T10:33:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.463+0000 7f26a927d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f269c1ad2f0 con 0x55d52ea101f0 2024-04-25T10:33:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.463+0000 7f26a927d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f269c1ac550 con 0x55d52ea30730 2024-04-25T10:33:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.463+0000 7f26a927d740 10 client.0 _ll_register_callbacks cb 0x7f2694010370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.463+0000 7f269b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea28550 0x55d52ee0c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.463+0000 7f269b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea28550 0x55d52ee0c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42084/0 (socket says 172.21.15.43:42084) 2024-04-25T10:33:12.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.463+0000 7f269b7fe640 1 -- 172.21.15.43:0/2896202411 learned_addr learned my addr 172.21.15.43:0/2896202411 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:12.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.464+0000 7f26a096b640 1 --2- 172.21.15.43:0/2896202411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52ea30730 0x55d52ee0c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.464+0000 7f269bfff640 1 --2- 172.21.15.43:0/2896202411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea101f0 0x55d52ee0bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.464+0000 7f269b7fe640 1 -- 172.21.15.43:0/2896202411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52ea30730 msgr2=0x55d52ee0c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:12.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.464+0000 7f269b7fe640 1 --2- 172.21.15.43:0/2896202411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52ea30730 0x55d52ee0c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.464+0000 7f269b7fe640 1 -- 172.21.15.43:0/2896202411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea101f0 msgr2=0x55d52ee0bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:12.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.464+0000 7f269b7fe640 1 --2- 172.21.15.43:0/2896202411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea101f0 0x55d52ee0bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.464+0000 7f269b7fe640 1 -- 172.21.15.43:0/2896202411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d52ee06480 con 0x55d52ea28550 2024-04-25T10:33:12.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.464+0000 7f269bfff640 1 --2- 172.21.15.43:0/2896202411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea101f0 0x55d52ee0bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.464+0000 7f269b7fe640 1 --2- 172.21.15.43:0/2896202411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea28550 0x55d52ee0c080 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f26900051f0 tx=0x7f2690032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6592e5438b5b74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.464+0000 7f267b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.464+0000 7f267b7fe640 1 -- 172.21.15.43:0/2896202411 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2690035050 con 0x55d52ea28550 2024-04-25T10:33:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.464+0000 7f267b7fe640 1 -- 172.21.15.43:0/2896202411 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f2690005490 con 0x55d52ea28550 2024-04-25T10:33:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.464+0000 7f26a927d740 1 -- 172.21.15.43:0/2896202411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26900316a0 con 0x55d52ea28550 2024-04-25T10:33:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.464+0000 7f267b7fe640 1 -- 172.21.15.43:0/2896202411 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2690004750 con 0x55d52ea28550 2024-04-25T10:33:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.464+0000 7f26a927d740 10 client.15678 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.464+0000 7f26a927d740 1 -- 172.21.15.43:0/2896202411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d52ee7a710 con 0x55d52ea28550 2024-04-25T10:33:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.464+0000 7f267b7fe640 1 -- 172.21.15.43:0/2896202411 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26900316a0 con 0x55d52ea28550 2024-04-25T10:33:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.465+0000 7f267b7fe640 1 -- 172.21.15.43:0/2896202411 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f269003c900 con 0x55d52ea28550 2024-04-25T10:33:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.465+0000 7f26a927d740 10 client.15678 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.465+0000 7f26a927d740 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.465+0000 7f26a927d740 20 client.15678 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.465+0000 7f26a927d740 1 --2- 172.21.15.43:0/2896202411 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d52ea51410 0x55d52ee7ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.465+0000 7f26a927d740 4 client.15678 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:33:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.465+0000 7f26a927d740 1 -- 172.21.15.43:0/2896202411 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d52ed5f580 con 0x55d52ea51410 2024-04-25T10:33:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.465+0000 7f269bfff640 1 --2- 172.21.15.43:0/2896202411 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d52ea51410 0x55d52ee7ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.465+0000 7f267b7fe640 10 client.15678 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.465+0000 7f269bfff640 1 --2- 172.21.15.43:0/2896202411 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d52ea51410 0x55d52ee7ce20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b225a92ec49d22a server_cookie=b439d7a2b213842d in_seq=0 out_seq=0 2024-04-25T10:33:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.466+0000 7f267b7fe640 1 -- 172.21.15.43:0/2896202411 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d52ed5f580 con 0x55d52ea51410 2024-04-25T10:33:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.466+0000 7f267b7fe640 10 client.15678 handle_command_reply: tid=0 2024-04-25T10:33:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.467+0000 7f26a927d740 1 client.15678 shutdown 2024-04-25T10:33:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.467+0000 7f26a927d740 1 -- 172.21.15.43:0/2896202411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea28550 msgr2=0x55d52ee0c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.467+0000 7f26a927d740 1 --2- 172.21.15.43:0/2896202411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea28550 0x55d52ee0c080 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f26900051f0 tx=0x7f2690032b20 comp rx=0 tx=0).stop 2024-04-25T10:33:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.467+0000 7f26a927d740 1 -- 172.21.15.43:0/2896202411 shutdown_connections 2024-04-25T10:33:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.467+0000 7f26a927d740 1 --2- 172.21.15.43:0/2896202411 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d52ea51410 0x55d52ee7ce20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.467+0000 7f26a927d740 1 --2- 172.21.15.43:0/2896202411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d52ea30730 0x55d52ee0c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.467+0000 7f26a927d740 1 --2- 172.21.15.43:0/2896202411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d52ea28550 0x55d52ee0c080 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.467+0000 7f26a927d740 1 --2- 172.21.15.43:0/2896202411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d52ea101f0 0x55d52ee0bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.467+0000 7f26a927d740 1 -- 172.21.15.43:0/2896202411 >> 172.21.15.43:0/2896202411 conn(0x55d52ea45560 msgr2=0x55d52edec4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:12.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.467+0000 7f267b7fe640 0 client.15678 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:12.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.467+0000 7f26a927d740 1 -- 172.21.15.43:0/2896202411 shutdown_connections 2024-04-25T10:33:12.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.467+0000 7f26a927d740 1 -- 172.21.15.43:0/2896202411 wait complete. 2024-04-25T10:33:12.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.467+0000 7f26a927d740 20 client.15678 trim_cache size 0 max 16384 2024-04-25T10:33:12.477 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:33:12.477 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:33:12.477 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "04c0e6db-6031-4c74-95df-5a1614d54722", 2024-04-25T10:33:12.477 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:33:12.478 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:33:12.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.467+0000 7f26a23ce640 1 -- 172.21.15.43:0/4163130220 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26840afe90 msgr2=0x7f26840b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.467+0000 7f26a23ce640 1 --2- 172.21.15.43:0/4163130220 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26840afe90 0x7f26840b23a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2694020000 tx=0x7f269401b000 comp rx=0 tx=0).stop 2024-04-25T10:33:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.467+0000 7f26a23ce640 1 -- 172.21.15.43:0/4163130220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269c104260 msgr2=0x7f269c19a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.467+0000 7f26a23ce640 1 --2- 172.21.15.43:0/4163130220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269c104260 0x7f269c19a740 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f268c00e9c0 tx=0x7f268c00ee80 comp rx=0 tx=0).stop 2024-04-25T10:33:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.468+0000 7f26a23ce640 1 -- 172.21.15.43:0/4163130220 shutdown_connections 2024-04-25T10:33:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.468+0000 7f26a23ce640 1 --2- 172.21.15.43:0/4163130220 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26840afe90 0x7f26840b23a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.468+0000 7f26a23ce640 1 --2- 172.21.15.43:0/4163130220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f269c109fb0 0x7f269c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.468+0000 7f26a23ce640 1 --2- 172.21.15.43:0/4163130220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f269c109b10 0x7f269c19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.468+0000 7f26a23ce640 1 --2- 172.21.15.43:0/4163130220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269c104260 0x7f269c19a740 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.468+0000 7f26a23ce640 1 -- 172.21.15.43:0/4163130220 >> 172.21.15.43:0/4163130220 conn(0x7f269c0ff210 msgr2=0x7f269c10f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.468+0000 7f26a23ce640 1 -- 172.21.15.43:0/4163130220 shutdown_connections 2024-04-25T10:33:12.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.468+0000 7f26a23ce640 1 -- 172.21.15.43:0/4163130220 wait complete. 2024-04-25T10:33:12.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:33:12.481 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:33:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.634+0000 7f3795a67640 1 -- 172.21.15.43:0/1679812269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3790104c70 msgr2=0x7f3790105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.634+0000 7f3795a67640 1 --2- 172.21.15.43:0/1679812269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3790104c70 0x7f3790105080 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3780009ca0 tx=0x7f378002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.634+0000 7f3795a67640 1 -- 172.21.15.43:0/1679812269 shutdown_connections 2024-04-25T10:33:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.634+0000 7f3795a67640 1 --2- 172.21.15.43:0/1679812269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f379010b6d0 0x7f379010daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.634+0000 7f3795a67640 1 --2- 172.21.15.43:0/1679812269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f379010e0c0 0x7f37901105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.634+0000 7f3795a67640 1 --2- 172.21.15.43:0/1679812269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3790104c70 0x7f3790105080 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.634+0000 7f3795a67640 1 -- 172.21.15.43:0/1679812269 >> 172.21.15.43:0/1679812269 conn(0x7f3790078110 msgr2=0x7f3790078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.634+0000 7f3795a67640 1 -- 172.21.15.43:0/1679812269 shutdown_connections 2024-04-25T10:33:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.634+0000 7f3795a67640 1 -- 172.21.15.43:0/1679812269 wait complete. 2024-04-25T10:33:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.635+0000 7f3795a67640 1 Processor -- start 2024-04-25T10:33:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.635+0000 7f3795a67640 1 -- start start 2024-04-25T10:33:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.635+0000 7f3795a67640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3790104c70 0x7f379019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.635+0000 7f3795a67640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f379010b6d0 0x7f379019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.635+0000 7f3795a67640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f379010e0c0 0x7f379019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.635+0000 7f3795a67640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3790113ce0 con 0x7f3790104c70 2024-04-25T10:33:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.635+0000 7f3795a67640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3790113b60 con 0x7f379010b6d0 2024-04-25T10:33:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.635+0000 7f3795a67640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3790113e60 con 0x7f379010e0c0 2024-04-25T10:33:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.636+0000 7f378f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3790104c70 0x7f379019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.636+0000 7f378f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3790104c70 0x7f379019a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42092/0 (socket says 172.21.15.43:42092) 2024-04-25T10:33:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.636+0000 7f378f7fe640 1 -- 172.21.15.43:0/844885428 learned_addr learned my addr 172.21.15.43:0/844885428 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.636+0000 7f378ffff640 1 --2- 172.21.15.43:0/844885428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f379010e0c0 0x7f379019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.636+0000 7f378effd640 1 --2- 172.21.15.43:0/844885428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f379010b6d0 0x7f379019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.636+0000 7f378f7fe640 1 -- 172.21.15.43:0/844885428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f379010e0c0 msgr2=0x7f379019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.636+0000 7f378f7fe640 1 --2- 172.21.15.43:0/844885428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f379010e0c0 0x7f379019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.636+0000 7f378f7fe640 1 -- 172.21.15.43:0/844885428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f379010b6d0 msgr2=0x7f379019acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.636+0000 7f378f7fe640 1 --2- 172.21.15.43:0/844885428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f379010b6d0 0x7f379019acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.636+0000 7f378f7fe640 1 -- 172.21.15.43:0/844885428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379019f5c0 con 0x7f3790104c70 2024-04-25T10:33:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.636+0000 7f378effd640 1 --2- 172.21.15.43:0/844885428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f379010b6d0 0x7f379019acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.636+0000 7f378f7fe640 1 --2- 172.21.15.43:0/844885428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3790104c70 0x7f379019a760 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3780033040 tx=0x7f37800316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.636+0000 7f378cff9640 1 -- 172.21.15.43:0/844885428 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780003b70 con 0x7f3790104c70 2024-04-25T10:33:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.636+0000 7f378cff9640 1 -- 172.21.15.43:0/844885428 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3780003d10 con 0x7f3790104c70 2024-04-25T10:33:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.636+0000 7f3795a67640 1 -- 172.21.15.43:0/844885428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379019f850 con 0x7f3790104c70 2024-04-25T10:33:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.636+0000 7f378cff9640 1 -- 172.21.15.43:0/844885428 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378002f9e0 con 0x7f3790104c70 2024-04-25T10:33:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.636+0000 7f3795a67640 1 -- 172.21.15.43:0/844885428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379019fda0 con 0x7f3790104c70 2024-04-25T10:33:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.639+0000 7f379c9f9740 1 Processor -- start 2024-04-25T10:33:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.639+0000 7f378cff9640 1 -- 172.21.15.43:0/844885428 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 147804+0+0 (secure 0 0 0) 0x7f378002fba0 con 0x7f3790104c70 2024-04-25T10:33:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.639+0000 7f379c9f9740 1 -- start start 2024-04-25T10:33:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.639+0000 7f379c9f9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a32088390 0x556a32428a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.639+0000 7f379c9f9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a3205f4d0 0x556a32428fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.639+0000 7f379c9f9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a320676b0 0x556a324358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.639+0000 7f379c9f9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556a3238ed80 con 0x556a3205f4d0 2024-04-25T10:33:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.639+0000 7f379c9f9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556a323488b0 con 0x556a32088390 2024-04-25T10:33:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.639+0000 7f379c9f9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556a3238ad40 con 0x556a320676b0 2024-04-25T10:33:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.639+0000 7f378f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a3205f4d0 0x556a32428fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.639+0000 7f378f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a3205f4d0 0x556a32428fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42106/0 (socket says 172.21.15.43:42106) 2024-04-25T10:33:12.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.639+0000 7f378f7fe640 1 -- 172.21.15.43:0/2806645787 learned_addr learned my addr 172.21.15.43:0/2806645787 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:12.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.639+0000 7f378ffff640 1 --2- 172.21.15.43:0/2806645787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a32088390 0x556a32428a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.640+0000 7f378effd640 1 --2- 172.21.15.43:0/2806645787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a320676b0 0x556a324358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.640+0000 7f378f7fe640 1 -- 172.21.15.43:0/2806645787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a320676b0 msgr2=0x556a324358f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:12.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.640+0000 7f378f7fe640 1 --2- 172.21.15.43:0/2806645787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a320676b0 0x556a324358f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.640+0000 7f378f7fe640 1 -- 172.21.15.43:0/2806645787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a32088390 msgr2=0x556a32428a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:12.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.640+0000 7f378f7fe640 1 --2- 172.21.15.43:0/2806645787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a32088390 0x556a32428a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.640+0000 7f378f7fe640 1 -- 172.21.15.43:0/2806645787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a32319a40 con 0x556a3205f4d0 2024-04-25T10:33:12.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.640+0000 7f378ffff640 1 --2- 172.21.15.43:0/2806645787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a32088390 0x556a32428a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:12.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.640+0000 7f378cff9640 1 --2- 172.21.15.43:0/844885428 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f377c0afe10 0x7f377c0b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.640+0000 7f378ffff640 1 --2- 172.21.15.43:0/844885428 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f377c0afe10 0x7f377c0b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.640+0000 7f378f7fe640 1 --2- 172.21.15.43:0/2806645787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a3205f4d0 0x556a32428fa0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f37800fdd00 tx=0x7f3780106b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54f312acf0fac8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:12.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.640+0000 7f378cff9640 1 -- 172.21.15.43:0/844885428 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f3780100710 con 0x7f3790104c70 2024-04-25T10:33:12.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.640+0000 7f3785ffb640 1 -- 172.21.15.43:0/2806645787 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37800c4670 con 0x556a3205f4d0 2024-04-25T10:33:12.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.640+0000 7f3785ffb640 1 -- 172.21.15.43:0/2806645787 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3780036070 con 0x556a3205f4d0 2024-04-25T10:33:12.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.640+0000 7f3785ffb640 1 -- 172.21.15.43:0/2806645787 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37800c7f20 con 0x556a3205f4d0 2024-04-25T10:33:12.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.641+0000 7f379c9f9740 1 -- 172.21.15.43:0/2806645787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a3205f4d0 msgr2=0x556a32428fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:12.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.641+0000 7f379c9f9740 1 --2- 172.21.15.43:0/2806645787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a3205f4d0 0x556a32428fa0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f37800fdd00 tx=0x7f3780106b40 comp rx=0 tx=0).stop 2024-04-25T10:33:12.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.641+0000 7f378ffff640 1 --2- 172.21.15.43:0/844885428 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f377c0afe10 0x7f377c0b2320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x556a32425bd0 tx=0x7f3788005f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:12.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.641+0000 7f379c9f9740 1 -- 172.21.15.43:0/2806645787 shutdown_connections 2024-04-25T10:33:12.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.641+0000 7f379c9f9740 1 --2- 172.21.15.43:0/2806645787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a320676b0 0x556a324358f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.641+0000 7f379c9f9740 1 --2- 172.21.15.43:0/2806645787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a3205f4d0 0x556a32428fa0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.641+0000 7f379c9f9740 1 --2- 172.21.15.43:0/2806645787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a32088390 0x556a32428a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.641+0000 7f379c9f9740 1 -- 172.21.15.43:0/2806645787 >> 172.21.15.43:0/2806645787 conn(0x556a3207c4e0 msgr2=0x556a32425210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:12.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.641+0000 7f379c9f9740 1 -- 172.21.15.43:0/2806645787 shutdown_connections 2024-04-25T10:33:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.641+0000 7f379c9f9740 1 -- 172.21.15.43:0/2806645787 wait complete. 2024-04-25T10:33:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.641+0000 7f379c9f9740 1 -- start start 2024-04-25T10:33:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.642+0000 7f379c9f9740 1 Processor -- start 2024-04-25T10:33:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.642+0000 7f379c9f9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a3205f4d0 0x556a32433580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.642+0000 7f379c9f9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a320676b0 0x556a32433ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.642+0000 7f379c9f9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a32088390 0x556a3243f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.642+0000 7f379c9f9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556a3238ed80 con 0x556a3205f4d0 2024-04-25T10:33:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.642+0000 7f379c9f9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556a32315d10 con 0x556a32088390 2024-04-25T10:33:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.642+0000 7f379c9f9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556a32435e30 con 0x556a320676b0 2024-04-25T10:33:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.642+0000 7f379c9f9740 10 client.0 _ll_register_callbacks cb 0x556a32342210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.642+0000 7f378f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a3205f4d0 0x556a32433580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.642+0000 7f378f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a3205f4d0 0x556a32433580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42120/0 (socket says 172.21.15.43:42120) 2024-04-25T10:33:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.642+0000 7f378f7fe640 1 -- 172.21.15.43:0/1325887294 learned_addr learned my addr 172.21.15.43:0/1325887294 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.642+0000 7f378effd640 1 --2- 172.21.15.43:0/1325887294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a320676b0 0x556a32433ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.642+0000 7f378ffff640 1 --2- 172.21.15.43:0/1325887294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a32088390 0x556a3243f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.642+0000 7f378f7fe640 1 -- 172.21.15.43:0/1325887294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a320676b0 msgr2=0x556a32433ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.642+0000 7f378f7fe640 1 --2- 172.21.15.43:0/1325887294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a320676b0 0x556a32433ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.642+0000 7f378f7fe640 1 -- 172.21.15.43:0/1325887294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a32088390 msgr2=0x556a3243f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.642+0000 7f378f7fe640 1 --2- 172.21.15.43:0/1325887294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a32088390 0x556a3243f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.643+0000 7f378f7fe640 1 -- 172.21.15.43:0/1325887294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a3216a7d0 con 0x556a3205f4d0 2024-04-25T10:33:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.643+0000 7f378f7fe640 1 --2- 172.21.15.43:0/1325887294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a3205f4d0 0x556a32433580 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f37800fdea0 tx=0x7f37800094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70932529d6c92e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.643+0000 7f376effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.643+0000 7f376effd640 1 -- 172.21.15.43:0/1325887294 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37800c8100 con 0x556a3205f4d0 2024-04-25T10:33:12.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.643+0000 7f376effd640 1 -- 172.21.15.43:0/1325887294 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f37800c8c10 con 0x556a3205f4d0 2024-04-25T10:33:12.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.643+0000 7f379c9f9740 1 -- 172.21.15.43:0/1325887294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3780036070 con 0x556a3205f4d0 2024-04-25T10:33:12.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.643+0000 7f376effd640 1 -- 172.21.15.43:0/1325887294 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f37800c8580 con 0x556a3205f4d0 2024-04-25T10:33:12.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.643+0000 7f379c9f9740 10 client.15690 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:12.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.643+0000 7f379c9f9740 1 -- 172.21.15.43:0/1325887294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556a320a03f0 con 0x556a3205f4d0 2024-04-25T10:33:12.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.643+0000 7f376effd640 1 -- 172.21.15.43:0/1325887294 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3780036070 con 0x556a3205f4d0 2024-04-25T10:33:12.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.643+0000 7f376effd640 1 -- 172.21.15.43:0/1325887294 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f37800d1180 con 0x556a3205f4d0 2024-04-25T10:33:12.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.643+0000 7f379c9f9740 10 client.15690 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:12.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.643+0000 7f379c9f9740 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:12.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.643+0000 7f379c9f9740 20 client.15690 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:12.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.643+0000 7f379c9f9740 1 --2- 172.21.15.43:0/1325887294 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556a32046f60 0x556a32445180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.643+0000 7f379c9f9740 4 client.15690 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:33:12.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.643+0000 7f379c9f9740 1 -- 172.21.15.43:0/1325887294 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a3239c990 con 0x556a32046f60 2024-04-25T10:33:12.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.644+0000 7f378effd640 1 --2- 172.21.15.43:0/1325887294 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556a32046f60 0x556a32445180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.644+0000 7f376effd640 10 client.15690 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:12.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.644+0000 7f378effd640 1 --2- 172.21.15.43:0/1325887294 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556a32046f60 0x556a32445180 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df2bf2357fa8d95f server_cookie=727b005b88beaa9f in_seq=0 out_seq=0 2024-04-25T10:33:12.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.645+0000 7f376effd640 1 -- 172.21.15.43:0/1325887294 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556a3239c990 con 0x556a32046f60 2024-04-25T10:33:12.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.645+0000 7f376effd640 10 client.15690 handle_command_reply: tid=0 2024-04-25T10:33:12.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.645+0000 7f379c9f9740 1 client.15690 shutdown 2024-04-25T10:33:12.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.645+0000 7f379c9f9740 1 -- 172.21.15.43:0/1325887294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a3205f4d0 msgr2=0x556a32433580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:12.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.645+0000 7f379c9f9740 1 --2- 172.21.15.43:0/1325887294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a3205f4d0 0x556a32433580 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f37800fdea0 tx=0x7f37800094a0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.645+0000 7f379c9f9740 1 -- 172.21.15.43:0/1325887294 shutdown_connections 2024-04-25T10:33:12.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.645+0000 7f379c9f9740 1 --2- 172.21.15.43:0/1325887294 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556a32046f60 0x556a32445180 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.645+0000 7f379c9f9740 1 --2- 172.21.15.43:0/1325887294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a32088390 0x556a3243f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.645+0000 7f379c9f9740 1 --2- 172.21.15.43:0/1325887294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a320676b0 0x556a32433ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.645+0000 7f376effd640 0 client.15690 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:12.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.645+0000 7f379c9f9740 1 --2- 172.21.15.43:0/1325887294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a3205f4d0 0x556a32433580 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.645+0000 7f379c9f9740 1 -- 172.21.15.43:0/1325887294 >> 172.21.15.43:0/1325887294 conn(0x556a3207c4e0 msgr2=0x556a3242a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:12.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.645+0000 7f379c9f9740 1 -- 172.21.15.43:0/1325887294 shutdown_connections 2024-04-25T10:33:12.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.645+0000 7f379c9f9740 1 -- 172.21.15.43:0/1325887294 wait complete. 2024-04-25T10:33:12.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.645+0000 7f379c9f9740 20 client.15690 trim_cache size 0 max 16384 2024-04-25T10:33:12.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.663+0000 7f379c9f9740 1 Processor -- start 2024-04-25T10:33:12.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.663+0000 7f379c9f9740 1 -- start start 2024-04-25T10:33:12.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.664+0000 7f379c9f9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a3205f4d0 0x556a32432690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.664+0000 7f379c9f9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a320676b0 0x556a32424ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.664+0000 7f379c9f9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a32088390 0x556a3244a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.664+0000 7f379c9f9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f37901b22d0 con 0x556a32088390 2024-04-25T10:33:12.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.664+0000 7f379c9f9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f37901afba0 con 0x556a3205f4d0 2024-04-25T10:33:12.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.664+0000 7f379c9f9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37901ad9f0 con 0x556a320676b0 2024-04-25T10:33:12.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.664+0000 7f378ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a32088390 0x556a3244a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.664+0000 7f378ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a32088390 0x556a3244a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42122/0 (socket says 172.21.15.43:42122) 2024-04-25T10:33:12.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.664+0000 7f378ffff640 1 -- 172.21.15.43:0/3528510869 learned_addr learned my addr 172.21.15.43:0/3528510869 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:12.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.664+0000 7f378f7fe640 1 --2- 172.21.15.43:0/3528510869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a3205f4d0 0x556a32432690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.664+0000 7f378effd640 1 --2- 172.21.15.43:0/3528510869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a320676b0 0x556a32424ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.664+0000 7f378ffff640 1 -- 172.21.15.43:0/3528510869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a320676b0 msgr2=0x556a32424ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:12.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.664+0000 7f378ffff640 1 --2- 172.21.15.43:0/3528510869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a320676b0 0x556a32424ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.664+0000 7f378ffff640 1 -- 172.21.15.43:0/3528510869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a3205f4d0 msgr2=0x556a32432690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:12.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.664+0000 7f378ffff640 1 --2- 172.21.15.43:0/3528510869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a3205f4d0 0x556a32432690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.664+0000 7f378ffff640 1 -- 172.21.15.43:0/3528510869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a320600f0 con 0x556a32088390 2024-04-25T10:33:12.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.664+0000 7f378ffff640 1 --2- 172.21.15.43:0/3528510869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a32088390 0x556a3244a910 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3788004170 tx=0x7f3788017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa3d0b5b92941c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:12.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.665+0000 7f37857fa640 1 -- 172.21.15.43:0/3528510869 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378802d070 con 0x556a32088390 2024-04-25T10:33:12.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.665+0000 7f37857fa640 1 -- 172.21.15.43:0/3528510869 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3788003c20 con 0x556a32088390 2024-04-25T10:33:12.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.665+0000 7f37857fa640 1 -- 172.21.15.43:0/3528510869 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788028ab0 con 0x556a32088390 2024-04-25T10:33:12.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.665+0000 7f379c9f9740 1 -- 172.21.15.43:0/3528510869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a32088390 msgr2=0x556a3244a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:12.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.665+0000 7f379c9f9740 1 --2- 172.21.15.43:0/3528510869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a32088390 0x556a3244a910 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3788004170 tx=0x7f3788017000 comp rx=0 tx=0).stop 2024-04-25T10:33:12.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.665+0000 7f379c9f9740 1 -- 172.21.15.43:0/3528510869 shutdown_connections 2024-04-25T10:33:12.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.665+0000 7f379c9f9740 1 --2- 172.21.15.43:0/3528510869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a32088390 0x556a3244a910 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.665+0000 7f379c9f9740 1 --2- 172.21.15.43:0/3528510869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a320676b0 0x556a32424ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.665+0000 7f379c9f9740 1 --2- 172.21.15.43:0/3528510869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a3205f4d0 0x556a32432690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.665+0000 7f379c9f9740 1 -- 172.21.15.43:0/3528510869 >> 172.21.15.43:0/3528510869 conn(0x556a3207c4e0 msgr2=0x556a3242acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:12.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.665+0000 7f379c9f9740 1 -- 172.21.15.43:0/3528510869 shutdown_connections 2024-04-25T10:33:12.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.665+0000 7f379c9f9740 1 -- 172.21.15.43:0/3528510869 wait complete. 2024-04-25T10:33:12.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.666+0000 7f379c9f9740 1 -- start start 2024-04-25T10:33:12.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.666+0000 7f379c9f9740 1 Processor -- start 2024-04-25T10:33:12.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.666+0000 7f379c9f9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a3205f4d0 0x556a3243b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.666+0000 7f379c9f9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a320676b0 0x556a32436850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.666+0000 7f379c9f9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a32088390 0x556a32436d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.666+0000 7f379c9f9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f37901ae060 con 0x556a32088390 2024-04-25T10:33:12.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.666+0000 7f379c9f9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f37901af4d0 con 0x556a3205f4d0 2024-04-25T10:33:12.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.666+0000 7f379c9f9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37901ae730 con 0x556a320676b0 2024-04-25T10:33:12.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.666+0000 7f379c9f9740 10 client.0 _ll_register_callbacks cb 0x7f3788009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:12.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.666+0000 7f378ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a32088390 0x556a32436d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.666+0000 7f378ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a32088390 0x556a32436d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42124/0 (socket says 172.21.15.43:42124) 2024-04-25T10:33:12.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.666+0000 7f378ffff640 1 -- 172.21.15.43:0/3806641796 learned_addr learned my addr 172.21.15.43:0/3806641796 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:12.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.667+0000 7f378f7fe640 1 --2- 172.21.15.43:0/3806641796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a3205f4d0 0x556a3243b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.667+0000 7f378effd640 1 --2- 172.21.15.43:0/3806641796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a320676b0 0x556a32436850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.667+0000 7f378ffff640 1 -- 172.21.15.43:0/3806641796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a320676b0 msgr2=0x556a32436850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:12.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.667+0000 7f378ffff640 1 --2- 172.21.15.43:0/3806641796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a320676b0 0x556a32436850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.667+0000 7f378ffff640 1 -- 172.21.15.43:0/3806641796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a3205f4d0 msgr2=0x556a3243b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:12.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.667+0000 7f378ffff640 1 --2- 172.21.15.43:0/3806641796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a3205f4d0 0x556a3243b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.667+0000 7f378ffff640 1 -- 172.21.15.43:0/3806641796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a3234f4b0 con 0x556a32088390 2024-04-25T10:33:12.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.667+0000 7f378ffff640 1 --2- 172.21.15.43:0/3806641796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a32088390 0x556a32436d90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3788019040 tx=0x7f3788009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fdd89fe13cf40f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:12.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.667+0000 7f376effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:12.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.667+0000 7f376effd640 1 -- 172.21.15.43:0/3806641796 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788036060 con 0x556a32088390 2024-04-25T10:33:12.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.667+0000 7f379c9f9740 1 -- 172.21.15.43:0/3806641796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3788003c20 con 0x556a32088390 2024-04-25T10:33:12.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.667+0000 7f376effd640 1 -- 172.21.15.43:0/3806641796 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f3788025040 con 0x556a32088390 2024-04-25T10:33:12.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.667+0000 7f376effd640 1 -- 172.21.15.43:0/3806641796 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f378802d030 con 0x556a32088390 2024-04-25T10:33:12.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.667+0000 7f379c9f9740 10 client.15696 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:12.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.667+0000 7f379c9f9740 1 -- 172.21.15.43:0/3806641796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556a324ad0b0 con 0x556a32088390 2024-04-25T10:33:12.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.668+0000 7f376effd640 1 -- 172.21.15.43:0/3806641796 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3788003c20 con 0x556a32088390 2024-04-25T10:33:12.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.668+0000 7f376effd640 1 -- 172.21.15.43:0/3806641796 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3788005240 con 0x556a32088390 2024-04-25T10:33:12.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.668+0000 7f379c9f9740 10 client.15696 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:12.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.668+0000 7f379c9f9740 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:12.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.668+0000 7f379c9f9740 20 client.15696 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:12.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.668+0000 7f379c9f9740 1 --2- 172.21.15.43:0/3806641796 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556a3201c3a0 0x556a324af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.668+0000 7f379c9f9740 4 client.15696 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:33:12.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.668+0000 7f379c9f9740 1 -- 172.21.15.43:0/3806641796 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556a32396800 con 0x556a3201c3a0 2024-04-25T10:33:12.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.668+0000 7f378f7fe640 1 --2- 172.21.15.43:0/3806641796 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556a3201c3a0 0x556a324af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.669+0000 7f376effd640 10 client.15696 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:12.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.669+0000 7f378f7fe640 1 --2- 172.21.15.43:0/3806641796 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556a3201c3a0 0x556a324af8a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8fdd9819fb1d4a6 server_cookie=8834fe627d80473b in_seq=0 out_seq=0 2024-04-25T10:33:12.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.669+0000 7f376effd640 1 -- 172.21.15.43:0/3806641796 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556a32396800 con 0x556a3201c3a0 2024-04-25T10:33:12.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.669+0000 7f376effd640 10 client.15696 handle_command_reply: tid=0 2024-04-25T10:33:12.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.669+0000 7f379c9f9740 1 client.15696 shutdown 2024-04-25T10:33:12.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.669+0000 7f379c9f9740 1 -- 172.21.15.43:0/3806641796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a32088390 msgr2=0x556a32436d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:12.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.669+0000 7f379c9f9740 1 --2- 172.21.15.43:0/3806641796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a32088390 0x556a32436d90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3788019040 tx=0x7f3788009830 comp rx=0 tx=0).stop 2024-04-25T10:33:12.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.669+0000 7f379c9f9740 1 -- 172.21.15.43:0/3806641796 shutdown_connections 2024-04-25T10:33:12.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.669+0000 7f379c9f9740 1 --2- 172.21.15.43:0/3806641796 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556a3201c3a0 0x556a324af8a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.669+0000 7f379c9f9740 1 --2- 172.21.15.43:0/3806641796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a32088390 0x556a32436d90 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.669+0000 7f379c9f9740 1 --2- 172.21.15.43:0/3806641796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a320676b0 0x556a32436850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.669+0000 7f376effd640 0 client.15696 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:12.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.669+0000 7f379c9f9740 1 --2- 172.21.15.43:0/3806641796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a3205f4d0 0x556a3243b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.669+0000 7f379c9f9740 1 -- 172.21.15.43:0/3806641796 >> 172.21.15.43:0/3806641796 conn(0x556a3207c4e0 msgr2=0x556a32435470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:12.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.669+0000 7f379c9f9740 1 -- 172.21.15.43:0/3806641796 shutdown_connections 2024-04-25T10:33:12.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.669+0000 7f379c9f9740 1 -- 172.21.15.43:0/3806641796 wait complete. 2024-04-25T10:33:12.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.669+0000 7f379c9f9740 20 client.15696 trim_cache size 0 max 16384 2024-04-25T10:33:12.678 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:33:12.678 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:33:12.678 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:33:12.679 INFO:teuthology.orchestra.run.smithi043.stdout: "04c0e6db-6031-4c74-95df-5a1614d54722": { 2024-04-25T10:33:12.679 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:33:12.679 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "04c0e6db-6031-4c74-95df-5a1614d54722", 2024-04-25T10:33:12.679 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:33:12.679 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:33:12.679 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:33:12.679 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:33:12.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.670+0000 7f3795a67640 1 -- 172.21.15.43:0/844885428 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f377c0afe10 msgr2=0x7f377c0b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:12.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.670+0000 7f3795a67640 1 --2- 172.21.15.43:0/844885428 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f377c0afe10 0x7f377c0b2320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x556a32425bd0 tx=0x7f3788005f90 comp rx=0 tx=0).stop 2024-04-25T10:33:12.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.670+0000 7f3795a67640 1 -- 172.21.15.43:0/844885428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3790104c70 msgr2=0x7f379019a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:12.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.670+0000 7f3795a67640 1 --2- 172.21.15.43:0/844885428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3790104c70 0x7f379019a760 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3780033040 tx=0x7f37800316f0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.670+0000 7f3795a67640 1 -- 172.21.15.43:0/844885428 shutdown_connections 2024-04-25T10:33:12.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.670+0000 7f3795a67640 1 --2- 172.21.15.43:0/844885428 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f377c0afe10 0x7f377c0b2320 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.670+0000 7f3795a67640 1 --2- 172.21.15.43:0/844885428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f379010b6d0 0x7f379019acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.670+0000 7f3795a67640 1 --2- 172.21.15.43:0/844885428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f379010e0c0 0x7f379019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.670+0000 7f3795a67640 1 --2- 172.21.15.43:0/844885428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3790104c70 0x7f379019a760 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.670+0000 7f3795a67640 1 -- 172.21.15.43:0/844885428 >> 172.21.15.43:0/844885428 conn(0x7f3790078110 msgr2=0x7f3790101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:12.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.670+0000 7f3795a67640 1 -- 172.21.15.43:0/844885428 shutdown_connections 2024-04-25T10:33:12.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:12.670+0000 7f3795a67640 1 -- 172.21.15.43:0/844885428 wait complete. 2024-04-25T10:33:12.683 INFO:tasks.cephfs.filesystem:scrub status for tag:04c0e6db-6031-4c74-95df-5a1614d54722 - {'path': '/', 'tag': '04c0e6db-6031-4c74-95df-5a1614d54722', 'options': 'recursive,force'} 2024-04-25T10:33:14.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:13 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:33:14.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:13 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:33:14.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:13 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:33:14.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:13 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:33:14.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:13 smithi077 ceph-mon[32772]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-04-25T10:33:14.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:13 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:33:14.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:13 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:33:14.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:13 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:33:14.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:13 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:33:14.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:13 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:33:14.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:13 smithi164 ceph-mon[32713]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-04-25T10:33:14.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:13 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:33:14.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:13 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:33:14.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:13 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:33:14.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:13 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:33:14.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:13 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:33:14.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:13 smithi043 ceph-mon[29478]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-04-25T10:33:14.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:13 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:33:16.279 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:15 smithi077 ceph-mon[32772]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 24 KiB/s wr, 16 op/s 2024-04-25T10:33:16.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:15 smithi164 ceph-mon[32713]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 24 KiB/s wr, 16 op/s 2024-04-25T10:33:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:15 smithi043 ceph-mon[29478]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 24 KiB/s wr, 16 op/s 2024-04-25T10:33:18.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:17 smithi077 ceph-mon[32772]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T10:33:18.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:17 smithi164 ceph-mon[32713]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T10:33:18.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:17 smithi043 ceph-mon[29478]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T10:33:20.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:19 smithi077 ceph-mon[32772]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-25T10:33:20.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:19 smithi164 ceph-mon[32713]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-25T10:33:20.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:19 smithi043 ceph-mon[29478]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-25T10:33:21.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.667+0000 7f7844a84640 1 -- 172.21.15.43:0/1527021890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7840109fe0 msgr2=0x7f7840103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:21.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.667+0000 7f7844a84640 1 --2- 172.21.15.43:0/1527021890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7840109fe0 0x7f7840103c20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7834009900 tx=0x7f783402a410 comp rx=0 tx=0).stop 2024-04-25T10:33:21.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.668+0000 7f7844a84640 1 -- 172.21.15.43:0/1527021890 shutdown_connections 2024-04-25T10:33:21.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.668+0000 7f7844a84640 1 --2- 172.21.15.43:0/1527021890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7840104290 0x7f7840110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.668+0000 7f7844a84640 1 --2- 172.21.15.43:0/1527021890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7840109fe0 0x7f7840103c20 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.668+0000 7f7844a84640 1 --2- 172.21.15.43:0/1527021890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7840109b40 0x7f78401036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.668+0000 7f7844a84640 1 -- 172.21.15.43:0/1527021890 >> 172.21.15.43:0/1527021890 conn(0x7f78400ff200 msgr2=0x7f7840101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.668+0000 7f7844a84640 1 -- 172.21.15.43:0/1527021890 shutdown_connections 2024-04-25T10:33:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.668+0000 7f7844a84640 1 -- 172.21.15.43:0/1527021890 wait complete. 2024-04-25T10:33:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.669+0000 7f7844a84640 1 Processor -- start 2024-04-25T10:33:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.669+0000 7f7844a84640 1 -- start start 2024-04-25T10:33:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.669+0000 7f7844a84640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7840104290 0x7f784019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.669+0000 7f7844a84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7840109b40 0x7f784019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.669+0000 7f7844a84640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7840109fe0 0x7f784019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.669+0000 7f7844a84640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7840113d30 con 0x7f7840109b40 2024-04-25T10:33:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.669+0000 7f7844a84640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7840113bb0 con 0x7f7840109fe0 2024-04-25T10:33:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.669+0000 7f7844a84640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7840113eb0 con 0x7f7840104290 2024-04-25T10:33:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.669+0000 7f783dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7840109b40 0x7f784019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.669+0000 7f783dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7840109b40 0x7f784019ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44924/0 (socket says 172.21.15.43:44924) 2024-04-25T10:33:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.669+0000 7f783dd9b640 1 -- 172.21.15.43:0/1414518832 learned_addr learned my addr 172.21.15.43:0/1414518832 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.669+0000 7f783e59c640 1 --2- 172.21.15.43:0/1414518832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7840104290 0x7f784019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.669+0000 7f783ed9d640 1 --2- 172.21.15.43:0/1414518832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7840109fe0 0x7f784019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.670+0000 7f783dd9b640 1 -- 172.21.15.43:0/1414518832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7840104290 msgr2=0x7f784019a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.670+0000 7f783dd9b640 1 --2- 172.21.15.43:0/1414518832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7840104290 0x7f784019a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.670+0000 7f783dd9b640 1 -- 172.21.15.43:0/1414518832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7840109fe0 msgr2=0x7f784019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.670+0000 7f783dd9b640 1 --2- 172.21.15.43:0/1414518832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7840109fe0 0x7f784019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.670+0000 7f783dd9b640 1 -- 172.21.15.43:0/1414518832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784019f610 con 0x7f7840109b40 2024-04-25T10:33:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.670+0000 7f783dd9b640 1 --2- 172.21.15.43:0/1414518832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7840109b40 0x7f784019ad10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7834006010 tx=0x7f7834004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.670+0000 7f782f7fe640 1 -- 172.21.15.43:0/1414518832 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783403f030 con 0x7f7840109b40 2024-04-25T10:33:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.670+0000 7f782f7fe640 1 -- 172.21.15.43:0/1414518832 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7834003aa0 con 0x7f7840109b40 2024-04-25T10:33:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.670+0000 7f782f7fe640 1 -- 172.21.15.43:0/1414518832 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834035070 con 0x7f7840109b40 2024-04-25T10:33:21.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.670+0000 7f7844a84640 1 -- 172.21.15.43:0/1414518832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784019f8a0 con 0x7f7840109b40 2024-04-25T10:33:21.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.670+0000 7f7844a84640 1 -- 172.21.15.43:0/1414518832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f784019fd90 con 0x7f7840109b40 2024-04-25T10:33:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.673+0000 7f782f7fe640 1 -- 172.21.15.43:0/1414518832 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 147804+0+0 (secure 0 0 0) 0x7f7834042030 con 0x7f7840109b40 2024-04-25T10:33:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.673+0000 7f784ba09740 1 Processor -- start 2024-04-25T10:33:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.673+0000 7f784ba09740 1 -- start start 2024-04-25T10:33:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.673+0000 7f784ba09740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e603c51b0 0x560e60764af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.673+0000 7f784ba09740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e6039c2f0 0x560e60765030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.673+0000 7f784ba09740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e603a44d0 0x560e60765570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.673+0000 7f784ba09740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560e606cb460 con 0x560e6039c2f0 2024-04-25T10:33:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.673+0000 7f784ba09740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560e60685590 con 0x560e603a44d0 2024-04-25T10:33:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.673+0000 7f784ba09740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560e606c7420 con 0x560e603c51b0 2024-04-25T10:33:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.673+0000 7f783e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e6039c2f0 0x560e60765030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.673+0000 7f783e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e6039c2f0 0x560e60765030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44940/0 (socket says 172.21.15.43:44940) 2024-04-25T10:33:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.673+0000 7f783e59c640 1 -- 172.21.15.43:0/519557139 learned_addr learned my addr 172.21.15.43:0/519557139 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.673+0000 7f782f7fe640 1 --2- 172.21.15.43:0/1414518832 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f78280afe00 0x7f78280b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.674+0000 7f783dd9b640 1 --2- 172.21.15.43:0/519557139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e603a44d0 0x560e60765570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.674+0000 7f783ed9d640 1 --2- 172.21.15.43:0/519557139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e603c51b0 0x560e60764af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.674+0000 7f782f7fe640 1 -- 172.21.15.43:0/1414518832 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f78340ff870 con 0x7f7840109b40 2024-04-25T10:33:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.674+0000 7f783e59c640 1 -- 172.21.15.43:0/519557139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e603c51b0 msgr2=0x560e60764af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.674+0000 7f783e59c640 1 --2- 172.21.15.43:0/519557139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e603c51b0 0x560e60764af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.674+0000 7f783ed9d640 1 --2- 172.21.15.43:0/1414518832 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f78280afe00 0x7f78280b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.674+0000 7f783e59c640 1 -- 172.21.15.43:0/519557139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e603a44d0 msgr2=0x560e60765570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.674+0000 7f783e59c640 1 --2- 172.21.15.43:0/519557139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e603a44d0 0x560e60765570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.674+0000 7f783e59c640 1 -- 172.21.15.43:0/519557139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e606920d0 con 0x560e6039c2f0 2024-04-25T10:33:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.674+0000 7f783dd9b640 1 --2- 172.21.15.43:0/519557139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e603a44d0 0x560e60765570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.674+0000 7f783e59c640 1 --2- 172.21.15.43:0/519557139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e6039c2f0 0x560e60765030 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f783000bc00 tx=0x7f783000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d627b4043953a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.674+0000 7f783ed9d640 1 --2- 172.21.15.43:0/1414518832 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f78280afe00 0x7f78280b2310 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x560e60761c60 tx=0x7f7838004ab0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.674+0000 7f782cff9640 1 -- 172.21.15.43:0/519557139 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830015070 con 0x560e6039c2f0 2024-04-25T10:33:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.674+0000 7f782cff9640 1 -- 172.21.15.43:0/519557139 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78300054d0 con 0x560e6039c2f0 2024-04-25T10:33:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.674+0000 7f782cff9640 1 -- 172.21.15.43:0/519557139 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830011440 con 0x560e6039c2f0 2024-04-25T10:33:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.675+0000 7f784ba09740 1 -- 172.21.15.43:0/519557139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e6039c2f0 msgr2=0x560e60765030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.675+0000 7f784ba09740 1 --2- 172.21.15.43:0/519557139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e6039c2f0 0x560e60765030 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f783000bc00 tx=0x7f783000f730 comp rx=0 tx=0).stop 2024-04-25T10:33:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.675+0000 7f784ba09740 1 -- 172.21.15.43:0/519557139 shutdown_connections 2024-04-25T10:33:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.675+0000 7f784ba09740 1 --2- 172.21.15.43:0/519557139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e603a44d0 0x560e60765570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.675+0000 7f784ba09740 1 --2- 172.21.15.43:0/519557139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e6039c2f0 0x560e60765030 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.675+0000 7f784ba09740 1 --2- 172.21.15.43:0/519557139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e603c51b0 0x560e60764af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.675+0000 7f784ba09740 1 -- 172.21.15.43:0/519557139 >> 172.21.15.43:0/519557139 conn(0x560e603b9300 msgr2=0x560e607612a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.675+0000 7f784ba09740 1 -- 172.21.15.43:0/519557139 shutdown_connections 2024-04-25T10:33:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.675+0000 7f784ba09740 1 -- 172.21.15.43:0/519557139 wait complete. 2024-04-25T10:33:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.676+0000 7f784ba09740 1 -- start start 2024-04-25T10:33:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.676+0000 7f784ba09740 1 Processor -- start 2024-04-25T10:33:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.676+0000 7f784ba09740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e6039c2f0 0x560e60770b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.676+0000 7f784ba09740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e603a44d0 0x560e60771070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.676+0000 7f784ba09740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603c51b0 0x560e6076f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.676+0000 7f784ba09740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560e60771c20 con 0x560e603c51b0 2024-04-25T10:33:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.676+0000 7f784ba09740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560e60652b60 con 0x560e603a44d0 2024-04-25T10:33:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.676+0000 7f784ba09740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560e60771da0 con 0x560e6039c2f0 2024-04-25T10:33:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.677+0000 7f784ba09740 10 client.0 _ll_register_callbacks cb 0x560e606772b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.677+0000 7f783ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603c51b0 0x560e6076f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.677+0000 7f783e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e6039c2f0 0x560e60770b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.677+0000 7f783dd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e603a44d0 0x560e60771070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.677+0000 7f783ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603c51b0 0x560e6076f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44954/0 (socket says 172.21.15.43:44954) 2024-04-25T10:33:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.677+0000 7f783ed9d640 1 -- 172.21.15.43:0/1839902671 learned_addr learned my addr 172.21.15.43:0/1839902671 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.677+0000 7f783ed9d640 1 -- 172.21.15.43:0/1839902671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e6039c2f0 msgr2=0x560e60770b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.677+0000 7f783ed9d640 1 --2- 172.21.15.43:0/1839902671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e6039c2f0 0x560e60770b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.677+0000 7f783ed9d640 1 -- 172.21.15.43:0/1839902671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e603a44d0 msgr2=0x560e60771070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.677+0000 7f783ed9d640 1 --2- 172.21.15.43:0/1839902671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e603a44d0 0x560e60771070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.677+0000 7f783ed9d640 1 -- 172.21.15.43:0/1839902671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e603dd210 con 0x560e603c51b0 2024-04-25T10:33:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.677+0000 7f783ed9d640 1 --2- 172.21.15.43:0/1839902671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603c51b0 0x560e6076f000 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f78380046d0 tx=0x7f7838004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd12300f961e9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.677+0000 7f7825ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.677+0000 7f7825ffb640 1 -- 172.21.15.43:0/1839902671 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7838012110 con 0x560e603c51b0 2024-04-25T10:33:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.677+0000 7f7825ffb640 1 -- 172.21.15.43:0/1839902671 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f783801b070 con 0x560e603c51b0 2024-04-25T10:33:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.677+0000 7f784ba09740 1 -- 172.21.15.43:0/1839902671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78300054d0 con 0x560e603c51b0 2024-04-25T10:33:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.677+0000 7f7825ffb640 1 -- 172.21.15.43:0/1839902671 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f78380122b0 con 0x560e603c51b0 2024-04-25T10:33:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.677+0000 7f784ba09740 10 client.15708 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.678+0000 7f784ba09740 1 -- 172.21.15.43:0/1839902671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560e604a9620 con 0x560e603c51b0 2024-04-25T10:33:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.678+0000 7f7825ffb640 1 -- 172.21.15.43:0/1839902671 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78300054d0 con 0x560e603c51b0 2024-04-25T10:33:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.678+0000 7f7825ffb640 1 -- 172.21.15.43:0/1839902671 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f78380054b0 con 0x560e603c51b0 2024-04-25T10:33:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.678+0000 7f784ba09740 10 client.15708 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.678+0000 7f784ba09740 10 client.15708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.678+0000 7f784ba09740 20 client.15708 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.678+0000 7f784ba09740 1 --2- 172.21.15.43:0/1839902671 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560e60383d80 0x560e607817e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.678+0000 7f784ba09740 4 client.15708 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:33:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.678+0000 7f784ba09740 1 -- 172.21.15.43:0/1839902671 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e606d9070 con 0x560e60383d80 2024-04-25T10:33:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.678+0000 7f783e59c640 1 --2- 172.21.15.43:0/1839902671 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560e60383d80 0x560e607817e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.679+0000 7f7825ffb640 10 client.15708 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.679+0000 7f783e59c640 1 --2- 172.21.15.43:0/1839902671 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560e60383d80 0x560e607817e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=255f58d5980597d9 server_cookie=51ad5ef253b4e3fa in_seq=0 out_seq=0 2024-04-25T10:33:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.679+0000 7f7825ffb640 1 -- 172.21.15.43:0/1839902671 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560e606d9070 con 0x560e60383d80 2024-04-25T10:33:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.679+0000 7f7825ffb640 10 client.15708 handle_command_reply: tid=0 2024-04-25T10:33:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.680+0000 7f784ba09740 1 client.15708 shutdown 2024-04-25T10:33:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.680+0000 7f784ba09740 1 -- 172.21.15.43:0/1839902671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603c51b0 msgr2=0x560e6076f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.680+0000 7f784ba09740 1 --2- 172.21.15.43:0/1839902671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603c51b0 0x560e6076f000 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f78380046d0 tx=0x7f7838004a40 comp rx=0 tx=0).stop 2024-04-25T10:33:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.680+0000 7f784ba09740 1 -- 172.21.15.43:0/1839902671 shutdown_connections 2024-04-25T10:33:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.680+0000 7f784ba09740 1 --2- 172.21.15.43:0/1839902671 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560e60383d80 0x560e607817e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.680+0000 7f784ba09740 1 --2- 172.21.15.43:0/1839902671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e603a44d0 0x560e60771070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.680+0000 7f784ba09740 1 --2- 172.21.15.43:0/1839902671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603c51b0 0x560e6076f000 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.680+0000 7f7825ffb640 0 client.15708 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.680+0000 7f784ba09740 1 --2- 172.21.15.43:0/1839902671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e6039c2f0 0x560e60770b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.680+0000 7f784ba09740 1 -- 172.21.15.43:0/1839902671 >> 172.21.15.43:0/1839902671 conn(0x560e603b9300 msgr2=0x560e60766810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.680+0000 7f784ba09740 1 -- 172.21.15.43:0/1839902671 shutdown_connections 2024-04-25T10:33:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.680+0000 7f784ba09740 1 -- 172.21.15.43:0/1839902671 wait complete. 2024-04-25T10:33:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.680+0000 7f784ba09740 20 client.15708 trim_cache size 0 max 16384 2024-04-25T10:33:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.695+0000 7f784ba09740 1 Processor -- start 2024-04-25T10:33:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.695+0000 7f784ba09740 1 -- start start 2024-04-25T10:33:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.695+0000 7f784ba09740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e6039c2f0 0x560e60787030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.695+0000 7f784ba09740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e603a44d0 0x560e60787570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.695+0000 7f784ba09740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603c51b0 0x560e607667a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.696+0000 7f784ba09740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78401b21d0 con 0x560e603c51b0 2024-04-25T10:33:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.696+0000 7f784ba09740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78401afaa0 con 0x560e6039c2f0 2024-04-25T10:33:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.696+0000 7f784ba09740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f78401ad8f0 con 0x560e603a44d0 2024-04-25T10:33:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.696+0000 7f783ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603c51b0 0x560e607667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.696+0000 7f783ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603c51b0 0x560e607667a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44960/0 (socket says 172.21.15.43:44960) 2024-04-25T10:33:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.696+0000 7f783ed9d640 1 -- 172.21.15.43:0/2613055268 learned_addr learned my addr 172.21.15.43:0/2613055268 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.696+0000 7f783e59c640 1 --2- 172.21.15.43:0/2613055268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e6039c2f0 0x560e60787030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.696+0000 7f783dd9b640 1 --2- 172.21.15.43:0/2613055268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e603a44d0 0x560e60787570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.696+0000 7f783ed9d640 1 -- 172.21.15.43:0/2613055268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e603a44d0 msgr2=0x560e60787570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.696+0000 7f783ed9d640 1 --2- 172.21.15.43:0/2613055268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e603a44d0 0x560e60787570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.696+0000 7f783ed9d640 1 -- 172.21.15.43:0/2613055268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e6039c2f0 msgr2=0x560e60787030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.696+0000 7f783ed9d640 1 --2- 172.21.15.43:0/2613055268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e6039c2f0 0x560e60787030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.696+0000 7f783ed9d640 1 -- 172.21.15.43:0/2613055268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e60765950 con 0x560e603c51b0 2024-04-25T10:33:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.696+0000 7f783dd9b640 1 --2- 172.21.15.43:0/2613055268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e603a44d0 0x560e60787570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.696+0000 7f783ed9d640 1 --2- 172.21.15.43:0/2613055268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603c51b0 0x560e607667a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7838012fd0 tx=0x7f783800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d35c8d8d777d31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.696+0000 7f7827fff640 1 -- 172.21.15.43:0/2613055268 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783800ec40 con 0x560e603c51b0 2024-04-25T10:33:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.696+0000 7f7827fff640 1 -- 172.21.15.43:0/2613055268 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7838004700 con 0x560e603c51b0 2024-04-25T10:33:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.696+0000 7f7827fff640 1 -- 172.21.15.43:0/2613055268 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783802ad90 con 0x560e603c51b0 2024-04-25T10:33:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.697+0000 7f784ba09740 1 -- 172.21.15.43:0/2613055268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603c51b0 msgr2=0x560e607667a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.697+0000 7f784ba09740 1 --2- 172.21.15.43:0/2613055268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603c51b0 0x560e607667a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7838012fd0 tx=0x7f783800eeb0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.697+0000 7f784ba09740 1 -- 172.21.15.43:0/2613055268 shutdown_connections 2024-04-25T10:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.697+0000 7f784ba09740 1 --2- 172.21.15.43:0/2613055268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603c51b0 0x560e607667a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.697+0000 7f784ba09740 1 --2- 172.21.15.43:0/2613055268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e603a44d0 0x560e60787570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.697+0000 7f784ba09740 1 --2- 172.21.15.43:0/2613055268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e6039c2f0 0x560e60787030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.697+0000 7f784ba09740 1 -- 172.21.15.43:0/2613055268 >> 172.21.15.43:0/2613055268 conn(0x560e603b9300 msgr2=0x560e6076d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.697+0000 7f784ba09740 1 -- 172.21.15.43:0/2613055268 shutdown_connections 2024-04-25T10:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.697+0000 7f784ba09740 1 -- 172.21.15.43:0/2613055268 wait complete. 2024-04-25T10:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.698+0000 7f784ba09740 1 -- start start 2024-04-25T10:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.698+0000 7f784ba09740 1 Processor -- start 2024-04-25T10:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.698+0000 7f784ba09740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e6039c2f0 0x560e60777410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.698+0000 7f784ba09740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603a44d0 0x560e60774780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.698+0000 7f784ba09740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e603c51b0 0x560e60774cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.698+0000 7f784ba09740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78401adf60 con 0x560e603a44d0 2024-04-25T10:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.698+0000 7f784ba09740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78401af3d0 con 0x560e603c51b0 2024-04-25T10:33:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.698+0000 7f784ba09740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f78401ae630 con 0x560e6039c2f0 2024-04-25T10:33:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.698+0000 7f784ba09740 10 client.0 _ll_register_callbacks cb 0x7f7838009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.698+0000 7f783dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603a44d0 0x560e60774780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.698+0000 7f783dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603a44d0 0x560e60774780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44976/0 (socket says 172.21.15.43:44976) 2024-04-25T10:33:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.698+0000 7f783dd9b640 1 -- 172.21.15.43:0/2913038094 learned_addr learned my addr 172.21.15.43:0/2913038094 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.698+0000 7f783ed9d640 1 --2- 172.21.15.43:0/2913038094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e603c51b0 0x560e60774cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.698+0000 7f783dd9b640 1 -- 172.21.15.43:0/2913038094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e6039c2f0 msgr2=0x560e60777410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.698+0000 7f783e59c640 1 --2- 172.21.15.43:0/2913038094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e6039c2f0 0x560e60777410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.698+0000 7f783dd9b640 1 --2- 172.21.15.43:0/2913038094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e6039c2f0 0x560e60777410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.698+0000 7f783dd9b640 1 -- 172.21.15.43:0/2913038094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e603c51b0 msgr2=0x560e60774cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.698+0000 7f783dd9b640 1 --2- 172.21.15.43:0/2913038094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e603c51b0 0x560e60774cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.698+0000 7f783dd9b640 1 -- 172.21.15.43:0/2913038094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e60775250 con 0x560e603a44d0 2024-04-25T10:33:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.698+0000 7f783ed9d640 1 --2- 172.21.15.43:0/2913038094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e603c51b0 0x560e60774cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.699+0000 7f783dd9b640 1 --2- 172.21.15.43:0/2913038094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603a44d0 0x560e60774780 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f78340045d0 tx=0x7f7834101d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e654e5c806238236 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.699+0000 7f7825ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.699+0000 7f7825ffb640 1 -- 172.21.15.43:0/2913038094 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834049040 con 0x560e603a44d0 2024-04-25T10:33:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.699+0000 7f7825ffb640 1 -- 172.21.15.43:0/2913038094 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f78340b00c0 con 0x560e603a44d0 2024-04-25T10:33:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.699+0000 7f784ba09740 1 -- 172.21.15.43:0/2913038094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7838004700 con 0x560e603a44d0 2024-04-25T10:33:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.699+0000 7f7825ffb640 1 -- 172.21.15.43:0/2913038094 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f78340c7150 con 0x560e603a44d0 2024-04-25T10:33:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.699+0000 7f784ba09740 10 client.15714 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.699+0000 7f784ba09740 1 -- 172.21.15.43:0/2913038094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560e607e9940 con 0x560e603a44d0 2024-04-25T10:33:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.699+0000 7f7825ffb640 1 -- 172.21.15.43:0/2913038094 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7838004700 con 0x560e603a44d0 2024-04-25T10:33:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.699+0000 7f7825ffb640 1 -- 172.21.15.43:0/2913038094 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f78340c7a60 con 0x560e603a44d0 2024-04-25T10:33:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.699+0000 7f784ba09740 10 client.15714 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.699+0000 7f784ba09740 10 client.15714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.699+0000 7f784ba09740 20 client.15714 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.699+0000 7f784ba09740 1 --2- 172.21.15.43:0/2913038094 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560e603591c0 0x560e607ec0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.699+0000 7f784ba09740 4 client.15714 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.699+0000 7f784ba09740 1 -- 172.21.15.43:0/2913038094 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e606d2ee0 con 0x560e603591c0 2024-04-25T10:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.700+0000 7f783e59c640 1 --2- 172.21.15.43:0/2913038094 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560e603591c0 0x560e607ec0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.700+0000 7f783e59c640 1 --2- 172.21.15.43:0/2913038094 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560e603591c0 0x560e607ec0d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cef91e7c4e9fea48 server_cookie=86407e9d4f7145a2 in_seq=0 out_seq=0 2024-04-25T10:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.700+0000 7f7825ffb640 10 client.15714 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.708+0000 7f7825ffb640 1 -- 172.21.15.43:0/2913038094 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560e606d2ee0 con 0x560e603591c0 2024-04-25T10:33:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.708+0000 7f7825ffb640 10 client.15714 handle_command_reply: tid=0 2024-04-25T10:33:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.708+0000 7f784ba09740 1 client.15714 shutdown 2024-04-25T10:33:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.708+0000 7f784ba09740 1 -- 172.21.15.43:0/2913038094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603a44d0 msgr2=0x560e60774780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.708+0000 7f784ba09740 1 --2- 172.21.15.43:0/2913038094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603a44d0 0x560e60774780 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f78340045d0 tx=0x7f7834101d00 comp rx=0 tx=0).stop 2024-04-25T10:33:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.708+0000 7f784ba09740 1 -- 172.21.15.43:0/2913038094 shutdown_connections 2024-04-25T10:33:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.708+0000 7f784ba09740 1 --2- 172.21.15.43:0/2913038094 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560e603591c0 0x560e607ec0d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.708+0000 7f784ba09740 1 --2- 172.21.15.43:0/2913038094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e603c51b0 0x560e60774cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.708+0000 7f784ba09740 1 --2- 172.21.15.43:0/2913038094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e603a44d0 0x560e60774780 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.708+0000 7f7825ffb640 0 client.15714 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.708+0000 7f784ba09740 1 --2- 172.21.15.43:0/2913038094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e6039c2f0 0x560e60777410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.708+0000 7f784ba09740 1 -- 172.21.15.43:0/2913038094 >> 172.21.15.43:0/2913038094 conn(0x560e603b9300 msgr2=0x560e60766730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.708+0000 7f784ba09740 1 -- 172.21.15.43:0/2913038094 shutdown_connections 2024-04-25T10:33:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.708+0000 7f784ba09740 1 -- 172.21.15.43:0/2913038094 wait complete. 2024-04-25T10:33:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.708+0000 7f784ba09740 20 client.15714 trim_cache size 0 max 16384 2024-04-25T10:33:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:33:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:33:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:33:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.709+0000 7f7844a84640 1 -- 172.21.15.43:0/1414518832 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f78280afe00 msgr2=0x7f78280b2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.709+0000 7f7844a84640 1 --2- 172.21.15.43:0/1414518832 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f78280afe00 0x7f78280b2310 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x560e60761c60 tx=0x7f7838004ab0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.709+0000 7f7844a84640 1 -- 172.21.15.43:0/1414518832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7840109b40 msgr2=0x7f784019ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.709+0000 7f7844a84640 1 --2- 172.21.15.43:0/1414518832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7840109b40 0x7f784019ad10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7834006010 tx=0x7f7834004060 comp rx=0 tx=0).stop 2024-04-25T10:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.709+0000 7f7844a84640 1 -- 172.21.15.43:0/1414518832 shutdown_connections 2024-04-25T10:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.709+0000 7f7844a84640 1 --2- 172.21.15.43:0/1414518832 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f78280afe00 0x7f78280b2310 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.709+0000 7f7844a84640 1 --2- 172.21.15.43:0/1414518832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7840109fe0 0x7f784019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.709+0000 7f7844a84640 1 --2- 172.21.15.43:0/1414518832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7840109b40 0x7f784019ad10 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.709+0000 7f7844a84640 1 --2- 172.21.15.43:0/1414518832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7840104290 0x7f784019a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.709+0000 7f7844a84640 1 -- 172.21.15.43:0/1414518832 >> 172.21.15.43:0/1414518832 conn(0x7f78400ff200 msgr2=0x7f784010f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.709+0000 7f7844a84640 1 -- 172.21.15.43:0/1414518832 shutdown_connections 2024-04-25T10:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:21.709+0000 7f7844a84640 1 -- 172.21.15.43:0/1414518832 wait complete. 2024-04-25T10:33:22.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:21 smithi077 ceph-mon[32772]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-04-25T10:33:22.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:21 smithi164 ceph-mon[32713]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-04-25T10:33:22.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:21 smithi043 ceph-mon[29478]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-04-25T10:33:24.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:23 smithi077 ceph-mon[32772]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-25T10:33:24.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:23 smithi164 ceph-mon[32713]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-25T10:33:24.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:23 smithi043 ceph-mon[29478]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-25T10:33:26.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:25 smithi077 ceph-mon[32772]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-04-25T10:33:26.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:25 smithi164 ceph-mon[32713]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-04-25T10:33:26.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:25 smithi043 ceph-mon[29478]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-04-25T10:33:28.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:27 smithi077 ceph-mon[32772]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-25T10:33:28.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:27 smithi164 ceph-mon[32713]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-25T10:33:28.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:27 smithi043 ceph-mon[29478]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-25T10:33:30.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:29 smithi164 ceph-mon[32713]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-25T10:33:30.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:29 smithi077 ceph-mon[32772]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-25T10:33:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:29 smithi043 ceph-mon[29478]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-25T10:33:32.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:31 smithi077 ceph-mon[32772]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-04-25T10:33:32.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:31 smithi164 ceph-mon[32713]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-04-25T10:33:32.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:31 smithi043 ceph-mon[29478]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-04-25T10:33:34.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:33 smithi077 ceph-mon[32772]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T10:33:34.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:33 smithi164 ceph-mon[32713]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T10:33:34.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:33 smithi043 ceph-mon[29478]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T10:33:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.811+0000 7f75877e3640 1 -- 172.21.15.43:0/1365739398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7580104c70 msgr2=0x7f7580105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.811+0000 7f75877e3640 1 --2- 172.21.15.43:0/1365739398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7580104c70 0x7f7580105080 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f757c009900 tx=0x7f757c02a410 comp rx=0 tx=0).stop 2024-04-25T10:33:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.811+0000 7f75877e3640 1 -- 172.21.15.43:0/1365739398 shutdown_connections 2024-04-25T10:33:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.811+0000 7f75877e3640 1 --2- 172.21.15.43:0/1365739398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f758010b6d0 0x7f758010daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.811+0000 7f75877e3640 1 --2- 172.21.15.43:0/1365739398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f758010e0c0 0x7f75801105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.811+0000 7f75877e3640 1 --2- 172.21.15.43:0/1365739398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7580104c70 0x7f7580105080 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.811+0000 7f75877e3640 1 -- 172.21.15.43:0/1365739398 >> 172.21.15.43:0/1365739398 conn(0x7f75800781b0 msgr2=0x7f75800785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.811+0000 7f75877e3640 1 -- 172.21.15.43:0/1365739398 shutdown_connections 2024-04-25T10:33:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.811+0000 7f75877e3640 1 -- 172.21.15.43:0/1365739398 wait complete. 2024-04-25T10:33:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.812+0000 7f75877e3640 1 Processor -- start 2024-04-25T10:33:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.812+0000 7f75877e3640 1 -- start start 2024-04-25T10:33:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.812+0000 7f75877e3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7580104c70 0x7f7580102530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.812+0000 7f75877e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f758010b6d0 0x7f7580102a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.812+0000 7f75877e3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f758010e0c0 0x7f7580102fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.812+0000 7f75877e3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7580113d20 con 0x7f758010b6d0 2024-04-25T10:33:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.812+0000 7f75877e3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7580113ba0 con 0x7f7580104c70 2024-04-25T10:33:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.812+0000 7f75877e3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7580113ea0 con 0x7f758010e0c0 2024-04-25T10:33:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.812+0000 7f7584d7e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f758010b6d0 0x7f7580102a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.812+0000 7f7584d7e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f758010b6d0 0x7f7580102a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57004/0 (socket says 172.21.15.43:57004) 2024-04-25T10:33:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.812+0000 7f7584d7e640 1 -- 172.21.15.43:0/2848870686 learned_addr learned my addr 172.21.15.43:0/2848870686 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.812+0000 7f758557f640 1 --2- 172.21.15.43:0/2848870686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7580104c70 0x7f7580102530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.812+0000 7f7584d7e640 1 -- 172.21.15.43:0/2848870686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f758010e0c0 msgr2=0x7f7580102fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.812+0000 7f7584d7e640 1 --2- 172.21.15.43:0/2848870686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f758010e0c0 0x7f7580102fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.812+0000 7f7584d7e640 1 -- 172.21.15.43:0/2848870686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7580104c70 msgr2=0x7f7580102530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.812+0000 7f7584d7e640 1 --2- 172.21.15.43:0/2848870686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7580104c70 0x7f7580102530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.812+0000 7f7584d7e640 1 -- 172.21.15.43:0/2848870686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75801a8a80 con 0x7f758010b6d0 2024-04-25T10:33:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.813+0000 7f7584d7e640 1 --2- 172.21.15.43:0/2848870686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f758010b6d0 0x7f7580102a70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f757000d950 tx=0x7f757000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.813+0000 7f75767fc640 1 -- 172.21.15.43:0/2848870686 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570017030 con 0x7f758010b6d0 2024-04-25T10:33:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.813+0000 7f75877e3640 1 -- 172.21.15.43:0/2848870686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75801a8d70 con 0x7f758010b6d0 2024-04-25T10:33:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.813+0000 7f75767fc640 1 -- 172.21.15.43:0/2848870686 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7570004020 con 0x7f758010b6d0 2024-04-25T10:33:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.813+0000 7f75767fc640 1 -- 172.21.15.43:0/2848870686 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75700047a0 con 0x7f758010b6d0 2024-04-25T10:33:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.813+0000 7f75877e3640 1 -- 172.21.15.43:0/2848870686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75801a9230 con 0x7f758010b6d0 2024-04-25T10:33:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.816+0000 7f758e76e740 1 Processor -- start 2024-04-25T10:33:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.816+0000 7f758e76e740 1 -- start start 2024-04-25T10:33:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.816+0000 7f75767fc640 1 -- 172.21.15.43:0/2848870686 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 147804+0+0 (secure 0 0 0) 0x7f7570009a30 con 0x7f758010b6d0 2024-04-25T10:33:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.816+0000 7f758e76e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287a91b0 0x55e328b48af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.816+0000 7f758e76e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287802f0 0x55e328b49030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.816+0000 7f758e76e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3287884d0 0x55e328b49570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.816+0000 7f758e76e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e328aaf460 con 0x55e3287802f0 2024-04-25T10:33:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.816+0000 7f758e76e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e328a69590 con 0x55e3287884d0 2024-04-25T10:33:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.816+0000 7f758e76e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e328aab420 con 0x55e3287a91b0 2024-04-25T10:33:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.816+0000 7f758557f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287802f0 0x55e328b49030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.816+0000 7f758557f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287802f0 0x55e328b49030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57006/0 (socket says 172.21.15.43:57006) 2024-04-25T10:33:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.816+0000 7f758557f640 1 -- 172.21.15.43:0/4127545897 learned_addr learned my addr 172.21.15.43:0/4127545897 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.816+0000 7f7585d80640 1 --2- 172.21.15.43:0/4127545897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287a91b0 0x55e328b48af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.817+0000 7f7584d7e640 1 --2- 172.21.15.43:0/4127545897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3287884d0 0x55e328b49570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.817+0000 7f758557f640 1 -- 172.21.15.43:0/4127545897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287a91b0 msgr2=0x55e328b48af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.817+0000 7f758557f640 1 --2- 172.21.15.43:0/4127545897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287a91b0 0x55e328b48af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.817+0000 7f758557f640 1 -- 172.21.15.43:0/4127545897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3287884d0 msgr2=0x55e328b49570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.817+0000 7f758557f640 1 --2- 172.21.15.43:0/4127545897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3287884d0 0x55e328b49570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.817+0000 7f758557f640 1 -- 172.21.15.43:0/4127545897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e328a760d0 con 0x55e3287802f0 2024-04-25T10:33:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.817+0000 7f7584d7e640 1 --2- 172.21.15.43:0/4127545897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3287884d0 0x55e328b49570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.817+0000 7f75767fc640 1 --2- 172.21.15.43:0/2848870686 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f756c0afe00 0x7f756c0b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.817+0000 7f758557f640 1 --2- 172.21.15.43:0/4127545897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287802f0 0x55e328b49030 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f757c02a930 tx=0x7f757c032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b299e5fe614ae54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.817+0000 7f7585d80640 1 --2- 172.21.15.43:0/2848870686 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f756c0afe00 0x7f756c0b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.817+0000 7f75767fc640 1 -- 172.21.15.43:0/2848870686 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f75700dedf0 con 0x7f758010b6d0 2024-04-25T10:33:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.817+0000 7f756b7fe640 1 -- 172.21.15.43:0/4127545897 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c03f070 con 0x55e3287802f0 2024-04-25T10:33:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.817+0000 7f756b7fe640 1 -- 172.21.15.43:0/4127545897 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f757c032e40 con 0x55e3287802f0 2024-04-25T10:33:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.817+0000 7f756b7fe640 1 -- 172.21.15.43:0/4127545897 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c035070 con 0x55e3287802f0 2024-04-25T10:33:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.818+0000 7f758e76e740 1 -- 172.21.15.43:0/4127545897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287802f0 msgr2=0x55e328b49030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.818+0000 7f758e76e740 1 --2- 172.21.15.43:0/4127545897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287802f0 0x55e328b49030 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f757c02a930 tx=0x7f757c032c50 comp rx=0 tx=0).stop 2024-04-25T10:33:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.818+0000 7f7585d80640 1 --2- 172.21.15.43:0/2848870686 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f756c0afe00 0x7f756c0b2310 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7578005c40 tx=0x7f7578009560 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.818+0000 7f758e76e740 1 -- 172.21.15.43:0/4127545897 shutdown_connections 2024-04-25T10:33:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.818+0000 7f758e76e740 1 --2- 172.21.15.43:0/4127545897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3287884d0 0x55e328b49570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.818+0000 7f758e76e740 1 --2- 172.21.15.43:0/4127545897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287802f0 0x55e328b49030 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.818+0000 7f758e76e740 1 --2- 172.21.15.43:0/4127545897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287a91b0 0x55e328b48af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.818+0000 7f758e76e740 1 -- 172.21.15.43:0/4127545897 >> 172.21.15.43:0/4127545897 conn(0x55e32879d300 msgr2=0x55e328b452a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.819+0000 7f758e76e740 1 -- 172.21.15.43:0/4127545897 shutdown_connections 2024-04-25T10:33:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.819+0000 7f758e76e740 1 -- 172.21.15.43:0/4127545897 wait complete. 2024-04-25T10:33:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.819+0000 7f758e76e740 1 -- start start 2024-04-25T10:33:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.819+0000 7f758e76e740 1 Processor -- start 2024-04-25T10:33:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.820+0000 7f758e76e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287802f0 0x55e328b54b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.820+0000 7f758e76e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3287884d0 0x55e328b55050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.820+0000 7f758e76e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287a91b0 0x55e328b52fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.820+0000 7f758e76e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e328b55bc0 con 0x55e3287a91b0 2024-04-25T10:33:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.820+0000 7f758e76e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e328a36b60 con 0x55e3287884d0 2024-04-25T10:33:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.820+0000 7f758e76e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e328b55d40 con 0x55e3287802f0 2024-04-25T10:33:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.820+0000 7f758e76e740 10 client.0 _ll_register_callbacks cb 0x55e328a5b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.820+0000 7f7585d80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287a91b0 0x55e328b52fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.820+0000 7f7585d80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287a91b0 0x55e328b52fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57012/0 (socket says 172.21.15.43:57012) 2024-04-25T10:33:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.820+0000 7f7585d80640 1 -- 172.21.15.43:0/141334254 learned_addr learned my addr 172.21.15.43:0/141334254 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.820+0000 7f758557f640 1 --2- 172.21.15.43:0/141334254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287802f0 0x55e328b54b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.820+0000 7f7584d7e640 1 --2- 172.21.15.43:0/141334254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3287884d0 0x55e328b55050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.820+0000 7f7585d80640 1 -- 172.21.15.43:0/141334254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287802f0 msgr2=0x55e328b54b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.820+0000 7f7585d80640 1 --2- 172.21.15.43:0/141334254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287802f0 0x55e328b54b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.820+0000 7f7585d80640 1 -- 172.21.15.43:0/141334254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3287884d0 msgr2=0x55e328b55050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.820+0000 7f7585d80640 1 --2- 172.21.15.43:0/141334254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3287884d0 0x55e328b55050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.820+0000 7f7585d80640 1 -- 172.21.15.43:0/141334254 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3287c1210 con 0x55e3287a91b0 2024-04-25T10:33:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.820+0000 7f758557f640 1 --2- 172.21.15.43:0/141334254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287802f0 0x55e328b54b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.820+0000 7f7585d80640 1 --2- 172.21.15.43:0/141334254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287a91b0 0x55e328b52fb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7578012780 tx=0x7f7578004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c94d08ae9bbc826 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.820+0000 7f7568ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.821+0000 7f7568ff9640 1 -- 172.21.15.43:0/141334254 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757801dac0 con 0x55e3287a91b0 2024-04-25T10:33:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.821+0000 7f7568ff9640 1 -- 172.21.15.43:0/141334254 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f757801dc60 con 0x55e3287a91b0 2024-04-25T10:33:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.821+0000 7f758e76e740 1 -- 172.21.15.43:0/141334254 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e328abd070 con 0x55e3287a91b0 2024-04-25T10:33:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.821+0000 7f7568ff9640 1 -- 172.21.15.43:0/141334254 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f757801de60 con 0x55e3287a91b0 2024-04-25T10:33:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.821+0000 7f758e76e740 10 client.15726 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.821+0000 7f758e76e740 1 -- 172.21.15.43:0/141334254 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e32888d620 con 0x55e3287a91b0 2024-04-25T10:33:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.821+0000 7f7568ff9640 1 -- 172.21.15.43:0/141334254 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e328abd070 con 0x55e3287a91b0 2024-04-25T10:33:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.821+0000 7f7568ff9640 1 -- 172.21.15.43:0/141334254 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f757800f040 con 0x55e3287a91b0 2024-04-25T10:33:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.821+0000 7f758e76e740 10 client.15726 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.821+0000 7f758e76e740 10 client.15726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.821+0000 7f758e76e740 20 client.15726 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.821+0000 7f758e76e740 1 --2- 172.21.15.43:0/141334254 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e328767d80 0x55e328b65720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.821+0000 7f758e76e740 4 client.15726 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:33:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.821+0000 7f758e76e740 1 -- 172.21.15.43:0/141334254 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e328ab6ee0 con 0x55e328767d80 2024-04-25T10:33:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.822+0000 7f758557f640 1 --2- 172.21.15.43:0/141334254 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e328767d80 0x55e328b65720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.822+0000 7f7568ff9640 10 client.15726 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.822+0000 7f758557f640 1 --2- 172.21.15.43:0/141334254 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e328767d80 0x55e328b65720 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c29574b239ee2989 server_cookie=ed83f168acb00779 in_seq=0 out_seq=0 2024-04-25T10:33:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.823+0000 7f7568ff9640 1 -- 172.21.15.43:0/141334254 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e328ab6ee0 con 0x55e328767d80 2024-04-25T10:33:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.823+0000 7f7568ff9640 10 client.15726 handle_command_reply: tid=0 2024-04-25T10:33:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.823+0000 7f758e76e740 1 client.15726 shutdown 2024-04-25T10:33:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.823+0000 7f758e76e740 1 -- 172.21.15.43:0/141334254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287a91b0 msgr2=0x55e328b52fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.823+0000 7f758e76e740 1 --2- 172.21.15.43:0/141334254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287a91b0 0x55e328b52fb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7578012780 tx=0x7f7578004a40 comp rx=0 tx=0).stop 2024-04-25T10:33:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.823+0000 7f758e76e740 1 -- 172.21.15.43:0/141334254 shutdown_connections 2024-04-25T10:33:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.823+0000 7f758e76e740 1 --2- 172.21.15.43:0/141334254 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e328767d80 0x55e328b65720 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.823+0000 7f7568ff9640 0 client.15726 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.823+0000 7f758e76e740 1 --2- 172.21.15.43:0/141334254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3287884d0 0x55e328b55050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.823+0000 7f758e76e740 1 --2- 172.21.15.43:0/141334254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287a91b0 0x55e328b52fb0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.823+0000 7f758e76e740 1 --2- 172.21.15.43:0/141334254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287802f0 0x55e328b54b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.823+0000 7f758e76e740 1 -- 172.21.15.43:0/141334254 >> 172.21.15.43:0/141334254 conn(0x55e32879d300 msgr2=0x55e328b48920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.823+0000 7f758e76e740 1 -- 172.21.15.43:0/141334254 shutdown_connections 2024-04-25T10:33:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.823+0000 7f758e76e740 1 -- 172.21.15.43:0/141334254 wait complete. 2024-04-25T10:33:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.823+0000 7f758e76e740 20 client.15726 trim_cache size 0 max 16384 2024-04-25T10:33:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.839+0000 7f758e76e740 1 Processor -- start 2024-04-25T10:33:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.839+0000 7f758e76e740 1 -- start start 2024-04-25T10:33:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.839+0000 7f758e76e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e328767d80 0x55e328b4c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.839+0000 7f758e76e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3287802f0 0x55e328b4ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.839+0000 7f758e76e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287884d0 0x55e328b51880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.839+0000 7f758e76e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75801b4440 con 0x55e328767d80 2024-04-25T10:33:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.839+0000 7f758e76e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f75801b1d10 con 0x55e3287802f0 2024-04-25T10:33:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.839+0000 7f758e76e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f75801afb60 con 0x55e3287884d0 2024-04-25T10:33:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.839+0000 7f758557f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e328767d80 0x55e328b4c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.839+0000 7f758557f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e328767d80 0x55e328b4c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57016/0 (socket says 172.21.15.43:57016) 2024-04-25T10:33:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.839+0000 7f758557f640 1 -- 172.21.15.43:0/3716041710 learned_addr learned my addr 172.21.15.43:0/3716041710 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.839+0000 7f7584d7e640 1 --2- 172.21.15.43:0/3716041710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3287802f0 0x55e328b4ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.840+0000 7f7585d80640 1 --2- 172.21.15.43:0/3716041710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287884d0 0x55e328b51880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.840+0000 7f758557f640 1 -- 172.21.15.43:0/3716041710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287884d0 msgr2=0x55e328b51880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.840+0000 7f758557f640 1 --2- 172.21.15.43:0/3716041710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287884d0 0x55e328b51880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.840+0000 7f758557f640 1 -- 172.21.15.43:0/3716041710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3287802f0 msgr2=0x55e328b4ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.840+0000 7f758557f640 1 --2- 172.21.15.43:0/3716041710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3287802f0 0x55e328b4ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.840+0000 7f758557f640 1 -- 172.21.15.43:0/3716041710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e328b60520 con 0x55e328767d80 2024-04-25T10:33:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.840+0000 7f7585d80640 1 --2- 172.21.15.43:0/3716041710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287884d0 0x55e328b51880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.840+0000 7f758557f640 1 --2- 172.21.15.43:0/3716041710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e328767d80 0x55e328b4c7b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f757c005f70 tx=0x7f757c032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350c22170af4595c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.840+0000 7f756affd640 1 -- 172.21.15.43:0/3716041710 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c03f040 con 0x55e328767d80 2024-04-25T10:33:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.840+0000 7f756affd640 1 -- 172.21.15.43:0/3716041710 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f757c053030 con 0x55e328767d80 2024-04-25T10:33:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.840+0000 7f756affd640 1 -- 172.21.15.43:0/3716041710 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c035040 con 0x55e328767d80 2024-04-25T10:33:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.841+0000 7f758e76e740 1 -- 172.21.15.43:0/3716041710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e328767d80 msgr2=0x55e328b4c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.841+0000 7f758e76e740 1 --2- 172.21.15.43:0/3716041710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e328767d80 0x55e328b4c7b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f757c005f70 tx=0x7f757c032dc0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.841+0000 7f758e76e740 1 -- 172.21.15.43:0/3716041710 shutdown_connections 2024-04-25T10:33:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.841+0000 7f758e76e740 1 --2- 172.21.15.43:0/3716041710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3287802f0 0x55e328b4ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.841+0000 7f758e76e740 1 --2- 172.21.15.43:0/3716041710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287884d0 0x55e328b51880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.841+0000 7f758e76e740 1 --2- 172.21.15.43:0/3716041710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e328767d80 0x55e328b4c7b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.841+0000 7f758e76e740 1 -- 172.21.15.43:0/3716041710 >> 172.21.15.43:0/3716041710 conn(0x55e32879d300 msgr2=0x55e328b4e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.841+0000 7f758e76e740 1 -- 172.21.15.43:0/3716041710 shutdown_connections 2024-04-25T10:33:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.841+0000 7f758e76e740 1 -- 172.21.15.43:0/3716041710 wait complete. 2024-04-25T10:33:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.842+0000 7f758e76e740 1 -- start start 2024-04-25T10:33:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.842+0000 7f758e76e740 1 Processor -- start 2024-04-25T10:33:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.842+0000 7f758e76e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e328767d80 0x55e328b5a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.842+0000 7f758e76e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287802f0 0x55e328b5aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.842+0000 7f758e76e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287884d0 0x55e328bc5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.842+0000 7f758e76e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75801b01d0 con 0x55e3287802f0 2024-04-25T10:33:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.842+0000 7f758e76e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f75801b1640 con 0x55e328767d80 2024-04-25T10:33:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.842+0000 7f758e76e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f75801b08a0 con 0x55e3287884d0 2024-04-25T10:33:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.842+0000 7f758e76e740 10 client.0 _ll_register_callbacks cb 0x7f7578009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.842+0000 7f7584d7e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287802f0 0x55e328b5aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.843+0000 7f758557f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e328767d80 0x55e328b5a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.843+0000 7f7584d7e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287802f0 0x55e328b5aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57026/0 (socket says 172.21.15.43:57026) 2024-04-25T10:33:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.843+0000 7f7584d7e640 1 -- 172.21.15.43:0/2760579440 learned_addr learned my addr 172.21.15.43:0/2760579440 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.843+0000 7f7585d80640 1 --2- 172.21.15.43:0/2760579440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287884d0 0x55e328bc5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.843+0000 7f7584d7e640 1 -- 172.21.15.43:0/2760579440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287884d0 msgr2=0x55e328bc5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.843+0000 7f7584d7e640 1 --2- 172.21.15.43:0/2760579440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287884d0 0x55e328bc5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.843+0000 7f7584d7e640 1 -- 172.21.15.43:0/2760579440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e328767d80 msgr2=0x55e328b5a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.843+0000 7f7584d7e640 1 --2- 172.21.15.43:0/2760579440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e328767d80 0x55e328b5a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.843+0000 7f7584d7e640 1 -- 172.21.15.43:0/2760579440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e328bc6090 con 0x55e3287802f0 2024-04-25T10:33:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.843+0000 7f7584d7e640 1 --2- 172.21.15.43:0/2760579440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287802f0 0x55e328b5aa30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f75700a3ff0 tx=0x7f7570009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8869042d2c1822 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.843+0000 7f7568ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.843+0000 7f7568ff9640 1 -- 172.21.15.43:0/2760579440 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75700a6d80 con 0x55e3287802f0 2024-04-25T10:33:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.843+0000 7f7568ff9640 1 -- 172.21.15.43:0/2760579440 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f757008f040 con 0x55e3287802f0 2024-04-25T10:33:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.843+0000 7f758e76e740 1 -- 172.21.15.43:0/2760579440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f757c053030 con 0x55e3287802f0 2024-04-25T10:33:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.843+0000 7f7568ff9640 1 -- 172.21.15.43:0/2760579440 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f75700ab130 con 0x55e3287802f0 2024-04-25T10:33:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.843+0000 7f758e76e740 10 client.15732 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.843+0000 7f758e76e740 1 -- 172.21.15.43:0/2760579440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e328bcd3c0 con 0x55e3287802f0 2024-04-25T10:33:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.844+0000 7f7568ff9640 1 -- 172.21.15.43:0/2760579440 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f757c053030 con 0x55e3287802f0 2024-04-25T10:33:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.844+0000 7f7568ff9640 1 -- 172.21.15.43:0/2760579440 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f75700a2810 con 0x55e3287802f0 2024-04-25T10:33:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.844+0000 7f758e76e740 10 client.15732 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.844+0000 7f758e76e740 10 client.15732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.844+0000 7f758e76e740 20 client.15732 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.844+0000 7f758e76e740 1 --2- 172.21.15.43:0/2760579440 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e3287a91b0 0x55e328bcfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.844+0000 7f758e76e740 4 client.15732 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:33:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.844+0000 7f758e76e740 1 -- 172.21.15.43:0/2760579440 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e328a325f0 con 0x55e3287a91b0 2024-04-25T10:33:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.844+0000 7f758557f640 1 --2- 172.21.15.43:0/2760579440 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e3287a91b0 0x55e328bcfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.845+0000 7f7568ff9640 10 client.15732 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.845+0000 7f758557f640 1 --2- 172.21.15.43:0/2760579440 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e3287a91b0 0x55e328bcfb50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d623e237c3257855 server_cookie=442756aed4bba672 in_seq=0 out_seq=0 2024-04-25T10:33:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.848+0000 7f7568ff9640 1 -- 172.21.15.43:0/2760579440 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e328a325f0 con 0x55e3287a91b0 2024-04-25T10:33:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.849+0000 7f7568ff9640 10 client.15732 handle_command_reply: tid=0 2024-04-25T10:33:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.849+0000 7f758e76e740 1 client.15732 shutdown 2024-04-25T10:33:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.849+0000 7f758e76e740 1 -- 172.21.15.43:0/2760579440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287802f0 msgr2=0x55e328b5aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.849+0000 7f758e76e740 1 --2- 172.21.15.43:0/2760579440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287802f0 0x55e328b5aa30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f75700a3ff0 tx=0x7f7570009f50 comp rx=0 tx=0).stop 2024-04-25T10:33:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.849+0000 7f758e76e740 1 -- 172.21.15.43:0/2760579440 shutdown_connections 2024-04-25T10:33:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.849+0000 7f758e76e740 1 --2- 172.21.15.43:0/2760579440 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e3287a91b0 0x55e328bcfb50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.849+0000 7f758e76e740 1 --2- 172.21.15.43:0/2760579440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3287884d0 0x55e328bc5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.849+0000 7f758e76e740 1 --2- 172.21.15.43:0/2760579440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3287802f0 0x55e328b5aa30 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.849+0000 7f7568ff9640 0 client.15732 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.849+0000 7f758e76e740 1 --2- 172.21.15.43:0/2760579440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e328767d80 0x55e328b5a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.849+0000 7f758e76e740 1 -- 172.21.15.43:0/2760579440 >> 172.21.15.43:0/2760579440 conn(0x55e32879d300 msgr2=0x55e328b4c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.849+0000 7f758e76e740 1 -- 172.21.15.43:0/2760579440 shutdown_connections 2024-04-25T10:33:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.849+0000 7f758e76e740 1 -- 172.21.15.43:0/2760579440 wait complete. 2024-04-25T10:33:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.849+0000 7f758e76e740 20 client.15732 trim_cache size 0 max 16384 2024-04-25T10:33:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:33:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:33:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:33:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:33:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.850+0000 7f75877e3640 1 -- 172.21.15.43:0/2848870686 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f756c0afe00 msgr2=0x7f756c0b2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.850+0000 7f75877e3640 1 --2- 172.21.15.43:0/2848870686 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f756c0afe00 0x7f756c0b2310 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7578005c40 tx=0x7f7578009560 comp rx=0 tx=0).stop 2024-04-25T10:33:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.850+0000 7f75877e3640 1 -- 172.21.15.43:0/2848870686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f758010b6d0 msgr2=0x7f7580102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.850+0000 7f75877e3640 1 --2- 172.21.15.43:0/2848870686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f758010b6d0 0x7f7580102a70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f757000d950 tx=0x7f757000de10 comp rx=0 tx=0).stop 2024-04-25T10:33:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.850+0000 7f75877e3640 1 -- 172.21.15.43:0/2848870686 shutdown_connections 2024-04-25T10:33:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.850+0000 7f75877e3640 1 --2- 172.21.15.43:0/2848870686 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f756c0afe00 0x7f756c0b2310 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.850+0000 7f75877e3640 1 --2- 172.21.15.43:0/2848870686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f758010e0c0 0x7f7580102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.850+0000 7f75877e3640 1 --2- 172.21.15.43:0/2848870686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f758010b6d0 0x7f7580102a70 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.850+0000 7f75877e3640 1 --2- 172.21.15.43:0/2848870686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7580104c70 0x7f7580102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.850+0000 7f75877e3640 1 -- 172.21.15.43:0/2848870686 >> 172.21.15.43:0/2848870686 conn(0x7f75800781b0 msgr2=0x7f7580101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.850+0000 7f75877e3640 1 -- 172.21.15.43:0/2848870686 shutdown_connections 2024-04-25T10:33:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:34.850+0000 7f75877e3640 1 -- 172.21.15.43:0/2848870686 wait complete. 2024-04-25T10:33:36.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:35 smithi077 ceph-mon[32772]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T10:33:36.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:35 smithi164 ceph-mon[32713]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T10:33:36.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:35 smithi043 ceph-mon[29478]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T10:33:38.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:37 smithi077 ceph-mon[32772]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:33:38.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:37 smithi164 ceph-mon[32713]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:33:38.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:37 smithi043 ceph-mon[29478]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:33:40.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:39 smithi077 ceph-mon[32772]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T10:33:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:39 smithi164 ceph-mon[32713]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T10:33:40.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:39 smithi043 ceph-mon[29478]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T10:33:41.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:40 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:33:41.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:40 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:33:41.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:40 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:33:42.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:41 smithi077 ceph-mon[32772]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T10:33:42.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:41 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:33:42.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:41 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:33:42.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:41 smithi164 ceph-mon[32713]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T10:33:42.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:41 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:33:42.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:41 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:33:42.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:41 smithi043 ceph-mon[29478]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T10:33:42.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:41 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:33:42.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:41 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:33:42.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:33:42.684 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:33:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.799+0000 7f7a972d1640 1 -- 172.21.15.43:0/4199982242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a9010d380 msgr2=0x7f7a9010f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.799+0000 7f7a972d1640 1 --2- 172.21.15.43:0/4199982242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a9010d380 0x7f7a9010f7a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009ca0 tx=0x7f7a8802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.800+0000 7f7a972d1640 1 -- 172.21.15.43:0/4199982242 shutdown_connections 2024-04-25T10:33:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.800+0000 7f7a972d1640 1 --2- 172.21.15.43:0/4199982242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a90102af0 0x7f7a90102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.800+0000 7f7a972d1640 1 --2- 172.21.15.43:0/4199982242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a9010d380 0x7f7a9010f7a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.800+0000 7f7a972d1640 1 --2- 172.21.15.43:0/4199982242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a90108a20 0x7f7a901025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.800+0000 7f7a972d1640 1 -- 172.21.15.43:0/4199982242 >> 172.21.15.43:0/4199982242 conn(0x7f7a90078100 msgr2=0x7f7a90078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.800+0000 7f7a972d1640 1 -- 172.21.15.43:0/4199982242 shutdown_connections 2024-04-25T10:33:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.800+0000 7f7a972d1640 1 -- 172.21.15.43:0/4199982242 wait complete. 2024-04-25T10:33:42.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.800+0000 7f7a972d1640 1 Processor -- start 2024-04-25T10:33:42.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.800+0000 7f7a972d1640 1 -- start start 2024-04-25T10:33:42.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.801+0000 7f7a972d1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a90102af0 0x7f7a90110470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.801+0000 7f7a972d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a90108a20 0x7f7a901109b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.801+0000 7f7a972d1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a9010d380 0x7f7a90110ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.801+0000 7f7a972d1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7a90113d10 con 0x7f7a90108a20 2024-04-25T10:33:42.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.801+0000 7f7a972d1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7a90113b90 con 0x7f7a90102af0 2024-04-25T10:33:42.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.801+0000 7f7a972d1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7a90113e90 con 0x7f7a9010d380 2024-04-25T10:33:42.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.801+0000 7f7a9486c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a90108a20 0x7f7a901109b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.801+0000 7f7a9486c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a90108a20 0x7f7a901109b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42864/0 (socket says 172.21.15.43:42864) 2024-04-25T10:33:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.801+0000 7f7a9486c640 1 -- 172.21.15.43:0/4060012750 learned_addr learned my addr 172.21.15.43:0/4060012750 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.801+0000 7f7a9506d640 1 --2- 172.21.15.43:0/4060012750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a90102af0 0x7f7a90110470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.801+0000 7f7a9586e640 1 --2- 172.21.15.43:0/4060012750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a9010d380 0x7f7a90110ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.801+0000 7f7a9506d640 1 -- 172.21.15.43:0/4060012750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a9010d380 msgr2=0x7f7a90110ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.801+0000 7f7a9506d640 1 --2- 172.21.15.43:0/4060012750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a9010d380 0x7f7a90110ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.801+0000 7f7a9506d640 1 -- 172.21.15.43:0/4060012750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a90108a20 msgr2=0x7f7a901109b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.801+0000 7f7a9506d640 1 --2- 172.21.15.43:0/4060012750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a90108a20 0x7f7a901109b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.801+0000 7f7a9506d640 1 -- 172.21.15.43:0/4060012750 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a901a8a10 con 0x7f7a90102af0 2024-04-25T10:33:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.801+0000 7f7a9486c640 1 --2- 172.21.15.43:0/4060012750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a90108a20 0x7f7a901109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.801+0000 7f7a9586e640 1 --2- 172.21.15.43:0/4060012750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a9010d380 0x7f7a90110ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.802+0000 7f7a9506d640 1 --2- 172.21.15.43:0/4060012750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a90102af0 0x7f7a90110470 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7a90103a20 tx=0x7f7a8c00dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.802+0000 7f7a867fc640 1 -- 172.21.15.43:0/4060012750 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c017070 con 0x7f7a90102af0 2024-04-25T10:33:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.802+0000 7f7a867fc640 1 -- 172.21.15.43:0/4060012750 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a8c0099d0 con 0x7f7a90102af0 2024-04-25T10:33:42.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.802+0000 7f7a972d1640 1 -- 172.21.15.43:0/4060012750 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a901a8d00 con 0x7f7a90102af0 2024-04-25T10:33:42.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.802+0000 7f7a867fc640 1 -- 172.21.15.43:0/4060012750 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c0047a0 con 0x7f7a90102af0 2024-04-25T10:33:42.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.802+0000 7f7a972d1640 1 -- 172.21.15.43:0/4060012750 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a901a91a0 con 0x7f7a90102af0 2024-04-25T10:33:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.804+0000 7f7a867fc640 1 -- 172.21.15.43:0/4060012750 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 147804+0+0 (secure 0 0 0) 0x7f7a8c004940 con 0x7f7a90102af0 2024-04-25T10:33:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.806+0000 7f7a9e17d740 1 Processor -- start 2024-04-25T10:33:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.806+0000 7f7a867fc640 1 --2- 172.21.15.43:0/4060012750 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7a7c0afe10 0x7f7a7c0b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.806+0000 7f7a9e17d740 1 -- start start 2024-04-25T10:33:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.806+0000 7f7a867fc640 1 -- 172.21.15.43:0/4060012750 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f7a8c0deef0 con 0x7f7a90102af0 2024-04-25T10:33:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.806+0000 7f7a9e17d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da8964390 0x562da8d04a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.806+0000 7f7a9586e640 1 --2- 172.21.15.43:0/4060012750 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7a7c0afe10 0x7f7a7c0b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.806+0000 7f7a9e17d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da893b4d0 0x562da8d04fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.806+0000 7f7a9e17d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da89436b0 0x562da8d118f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.806+0000 7f7a9e17d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562da8c6ad80 con 0x562da893b4d0 2024-04-25T10:33:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.806+0000 7f7a9e17d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562da8c248b0 con 0x562da8964390 2024-04-25T10:33:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.806+0000 7f7a9e17d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562da8c66d40 con 0x562da89436b0 2024-04-25T10:33:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.806+0000 7f7a9486c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da893b4d0 0x562da8d04fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.806+0000 7f7a9506d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da8964390 0x562da8d04a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.806+0000 7f7a9506d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da8964390 0x562da8d04a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:53578/0 (socket says 172.21.15.43:53578) 2024-04-25T10:33:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.806+0000 7f7a9506d640 1 -- 172.21.15.43:0/1290489289 learned_addr learned my addr 172.21.15.43:0/1290489289 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.807+0000 7f7a9506d640 1 -- 172.21.15.43:0/1290489289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da89436b0 msgr2=0x562da8d118f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.807+0000 7f7a9586e640 1 --2- 172.21.15.43:0/1290489289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da89436b0 0x562da8d118f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.807+0000 7f7a9506d640 1 --2- 172.21.15.43:0/1290489289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da89436b0 0x562da8d118f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.807+0000 7f7a9506d640 1 -- 172.21.15.43:0/1290489289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da893b4d0 msgr2=0x562da8d04fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.807+0000 7f7a9506d640 1 --2- 172.21.15.43:0/1290489289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da893b4d0 0x562da8d04fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.807+0000 7f7a9506d640 1 -- 172.21.15.43:0/1290489289 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562da8bf5a40 con 0x562da8964390 2024-04-25T10:33:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.807+0000 7f7a9586e640 1 --2- 172.21.15.43:0/1290489289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da89436b0 0x562da8d118f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.807+0000 7f7a9486c640 1 --2- 172.21.15.43:0/1290489289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da893b4d0 0x562da8d04fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.807+0000 7f7a9586e640 1 --2- 172.21.15.43:0/4060012750 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7a7c0afe10 0x7f7a7c0b2320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7a88006010 tx=0x7f7a880094a0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.807+0000 7f7a9506d640 1 --2- 172.21.15.43:0/1290489289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da8964390 0x562da8d04a60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0a6e10 tx=0x7f7a8c0a66b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3558ef5dee330531 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.807+0000 7f7a7b7fe640 1 -- 172.21.15.43:0/1290489289 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c0a6e80 con 0x562da8964390 2024-04-25T10:33:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.807+0000 7f7a7b7fe640 1 -- 172.21.15.43:0/1290489289 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a8c0ab230 con 0x562da8964390 2024-04-25T10:33:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.807+0000 7f7a7b7fe640 1 -- 172.21.15.43:0/1290489289 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c0dc740 con 0x562da8964390 2024-04-25T10:33:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.808+0000 7f7a9e17d740 1 -- 172.21.15.43:0/1290489289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da8964390 msgr2=0x562da8d04a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.808+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/1290489289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da8964390 0x562da8d04a60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0a6e10 tx=0x7f7a8c0a66b0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.808+0000 7f7a9e17d740 1 -- 172.21.15.43:0/1290489289 shutdown_connections 2024-04-25T10:33:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.808+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/1290489289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da89436b0 0x562da8d118f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.808+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/1290489289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da893b4d0 0x562da8d04fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.808+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/1290489289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da8964390 0x562da8d04a60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.808+0000 7f7a9e17d740 1 -- 172.21.15.43:0/1290489289 >> 172.21.15.43:0/1290489289 conn(0x562da89584e0 msgr2=0x562da8d01210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.808+0000 7f7a9e17d740 1 -- 172.21.15.43:0/1290489289 shutdown_connections 2024-04-25T10:33:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.808+0000 7f7a9e17d740 1 -- 172.21.15.43:0/1290489289 wait complete. 2024-04-25T10:33:42.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.809+0000 7f7a9e17d740 1 -- start start 2024-04-25T10:33:42.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.809+0000 7f7a9e17d740 1 Processor -- start 2024-04-25T10:33:42.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.810+0000 7f7a9e17d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da893b4d0 0x562da8d10ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.810+0000 7f7a9e17d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da89436b0 0x562da8d11530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.810+0000 7f7a9486c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da89436b0 0x562da8d11530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.810+0000 7f7a9486c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da89436b0 0x562da8d11530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42882/0 (socket says 172.21.15.43:42882) 2024-04-25T10:33:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.810+0000 7f7a9486c640 1 -- 172.21.15.43:0/3109623176 learned_addr learned my addr 172.21.15.43:0/3109623176 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.810+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/3109623176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da8964390 0x562da8d0f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.810+0000 7f7a9506d640 1 --2- 172.21.15.43:0/3109623176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da893b4d0 0x562da8d10ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.810+0000 7f7a9e17d740 1 -- 172.21.15.43:0/3109623176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562da8d11fe0 con 0x562da89436b0 2024-04-25T10:33:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.810+0000 7f7a9e17d740 1 -- 172.21.15.43:0/3109623176 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562da8bf1d10 con 0x562da8964390 2024-04-25T10:33:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.810+0000 7f7a9e17d740 1 -- 172.21.15.43:0/3109623176 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562da8d12160 con 0x562da893b4d0 2024-04-25T10:33:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.810+0000 7f7a9586e640 1 --2- 172.21.15.43:0/3109623176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da8964390 0x562da8d0f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.810+0000 7f7a9506d640 1 -- 172.21.15.43:0/3109623176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da8964390 msgr2=0x562da8d0f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.810+0000 7f7a9506d640 1 --2- 172.21.15.43:0/3109623176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da8964390 0x562da8d0f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.810+0000 7f7a9506d640 1 -- 172.21.15.43:0/3109623176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da89436b0 msgr2=0x562da8d11530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.811+0000 7f7a9506d640 1 --2- 172.21.15.43:0/3109623176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da89436b0 0x562da8d11530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.811+0000 7f7a9506d640 1 -- 172.21.15.43:0/3109623176 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562da8a467d0 con 0x562da893b4d0 2024-04-25T10:33:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.811+0000 7f7a9486c640 1 --2- 172.21.15.43:0/3109623176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da89436b0 0x562da8d11530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.811+0000 7f7a9e17d740 10 client.0 _ll_register_callbacks cb 0x562da8c1e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.811+0000 7f7a9e17d740 1 -- 172.21.15.43:0/3109623176 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a8c0ab230 con 0x562da893b4d0 2024-04-25T10:33:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.811+0000 7f7a9506d640 1 --2- 172.21.15.43:0/3109623176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da893b4d0 0x562da8d10ff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x562da8d01bd0 tx=0x7f7a8c0af850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=753a37eb2d736d11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.811+0000 7f7a78ff9640 10 client.25393 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-25T10:33:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.811+0000 7f7a78ff9640 1 -- 172.21.15.43:0/3109623176 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c0a3470 con 0x562da893b4d0 2024-04-25T10:33:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.811+0000 7f7a78ff9640 1 -- 172.21.15.43:0/3109623176 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f7a8c0a60e0 con 0x562da893b4d0 2024-04-25T10:33:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.811+0000 7f7a78ff9640 1 -- 172.21.15.43:0/3109623176 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7a8c0a62e0 con 0x562da893b4d0 2024-04-25T10:33:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.811+0000 7f7a9e17d740 10 client.25393 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.811+0000 7f7a9e17d740 1 -- 172.21.15.43:0/3109623176 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({fsmap=24}) -- 0x562da897c3f0 con 0x562da893b4d0 2024-04-25T10:33:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.812+0000 7f7a78ff9640 1 -- 172.21.15.43:0/3109623176 <== mon.1 v2:172.21.15.164:3300/0 4 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7a8c0dc740 con 0x562da893b4d0 2024-04-25T10:33:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.812+0000 7f7a9e17d740 10 client.25393 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.812+0000 7f7a9e17d740 10 client.25393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.812+0000 7f7a9e17d740 20 client.25393 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.812+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/3109623176 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562da8922f60 0x562da8d211b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.812+0000 7f7a9e17d740 4 client.25393 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:33:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.812+0000 7f7a9e17d740 1 -- 172.21.15.43:0/3109623176 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562da8c78990 con 0x562da8922f60 2024-04-25T10:33:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.812+0000 7f7a9486c640 1 --2- 172.21.15.43:0/3109623176 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562da8922f60 0x562da8d211b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.813+0000 7f7a78ff9640 10 client.25393 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.813+0000 7f7a9486c640 1 --2- 172.21.15.43:0/3109623176 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562da8922f60 0x562da8d211b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c214fc19571501cb server_cookie=8f41dee31e1af51b in_seq=0 out_seq=0 2024-04-25T10:33:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.813+0000 7f7a78ff9640 1 -- 172.21.15.43:0/3109623176 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562da8c78990 con 0x562da8922f60 2024-04-25T10:33:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.813+0000 7f7a78ff9640 10 client.25393 handle_command_reply: tid=0 2024-04-25T10:33:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.813+0000 7f7a9e17d740 1 client.25393 shutdown 2024-04-25T10:33:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.814+0000 7f7a9e17d740 1 -- 172.21.15.43:0/3109623176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da893b4d0 msgr2=0x562da8d10ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.814+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/3109623176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da893b4d0 0x562da8d10ff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x562da8d01bd0 tx=0x7f7a8c0af850 comp rx=0 tx=0).stop 2024-04-25T10:33:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.814+0000 7f7a9e17d740 1 -- 172.21.15.43:0/3109623176 shutdown_connections 2024-04-25T10:33:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.814+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/3109623176 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562da8922f60 0x562da8d211b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.814+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/3109623176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da8964390 0x562da8d0f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.814+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/3109623176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da89436b0 0x562da8d11530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.814+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/3109623176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da893b4d0 0x562da8d10ff0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.814+0000 7f7a9e17d740 1 -- 172.21.15.43:0/3109623176 >> 172.21.15.43:0/3109623176 conn(0x562da89584e0 msgr2=0x562da8cff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:42.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.814+0000 7f7a78ff9640 0 client.25393 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:42.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.814+0000 7f7a9e17d740 1 -- 172.21.15.43:0/3109623176 shutdown_connections 2024-04-25T10:33:42.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.814+0000 7f7a9e17d740 1 -- 172.21.15.43:0/3109623176 wait complete. 2024-04-25T10:33:42.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.814+0000 7f7a9e17d740 20 client.25393 trim_cache size 0 max 16384 2024-04-25T10:33:42.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.847+0000 7f7a9e17d740 1 Processor -- start 2024-04-25T10:33:42.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.847+0000 7f7a9e17d740 1 -- start start 2024-04-25T10:33:42.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.847+0000 7f7a9e17d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da893b4d0 0x562da8d0e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.848+0000 7f7a9e17d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da89436b0 0x562da8d0f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.848+0000 7f7a9e17d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da8964390 0x562da8d0a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.848+0000 7f7a9e17d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7a901b4410 con 0x562da893b4d0 2024-04-25T10:33:42.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.848+0000 7f7a9e17d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7a901b1ce0 con 0x562da89436b0 2024-04-25T10:33:42.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.848+0000 7f7a9e17d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7a901afb30 con 0x562da8964390 2024-04-25T10:33:42.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.848+0000 7f7a9506d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da893b4d0 0x562da8d0e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.848+0000 7f7a9506d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da893b4d0 0x562da8d0e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42896/0 (socket says 172.21.15.43:42896) 2024-04-25T10:33:42.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.848+0000 7f7a9506d640 1 -- 172.21.15.43:0/3210410696 learned_addr learned my addr 172.21.15.43:0/3210410696 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:42.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.848+0000 7f7a9486c640 1 --2- 172.21.15.43:0/3210410696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da89436b0 0x562da8d0f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.848+0000 7f7a9506d640 1 -- 172.21.15.43:0/3210410696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da8964390 msgr2=0x562da8d0a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:42.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.848+0000 7f7a9506d640 1 --2- 172.21.15.43:0/3210410696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da8964390 0x562da8d0a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.848+0000 7f7a9506d640 1 -- 172.21.15.43:0/3210410696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da89436b0 msgr2=0x562da8d0f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:42.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.848+0000 7f7a9506d640 1 --2- 172.21.15.43:0/3210410696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da89436b0 0x562da8d0f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.848+0000 7f7a9506d640 1 -- 172.21.15.43:0/3210410696 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562da893c0f0 con 0x562da893b4d0 2024-04-25T10:33:42.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.848+0000 7f7a9506d640 1 --2- 172.21.15.43:0/3210410696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da893b4d0 0x562da8d0e440 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0a7730 tx=0x7f7a8c06c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a735bd0255436148 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:42.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.849+0000 7f7a7affd640 1 -- 172.21.15.43:0/3210410696 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c0a67d0 con 0x562da893b4d0 2024-04-25T10:33:42.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.849+0000 7f7a7affd640 1 -- 172.21.15.43:0/3210410696 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a8c0ab230 con 0x562da893b4d0 2024-04-25T10:33:42.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.849+0000 7f7a7affd640 1 -- 172.21.15.43:0/3210410696 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c0a5dd0 con 0x562da893b4d0 2024-04-25T10:33:42.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.850+0000 7f7a9e17d740 1 -- 172.21.15.43:0/3210410696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da893b4d0 msgr2=0x562da8d0e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:42.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.850+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/3210410696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da893b4d0 0x562da8d0e440 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0a7730 tx=0x7f7a8c06c000 comp rx=0 tx=0).stop 2024-04-25T10:33:42.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.850+0000 7f7a9e17d740 1 -- 172.21.15.43:0/3210410696 shutdown_connections 2024-04-25T10:33:42.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.850+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/3210410696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da89436b0 0x562da8d0f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.850+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/3210410696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da8964390 0x562da8d0a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.850+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/3210410696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da893b4d0 0x562da8d0e440 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.850+0000 7f7a9e17d740 1 -- 172.21.15.43:0/3210410696 >> 172.21.15.43:0/3210410696 conn(0x562da89584e0 msgr2=0x562da8d0ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:42.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.850+0000 7f7a9e17d740 1 -- 172.21.15.43:0/3210410696 shutdown_connections 2024-04-25T10:33:42.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.850+0000 7f7a9e17d740 1 -- 172.21.15.43:0/3210410696 wait complete. 2024-04-25T10:33:42.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.851+0000 7f7a9e17d740 1 -- start start 2024-04-25T10:33:42.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.851+0000 7f7a9e17d740 1 Processor -- start 2024-04-25T10:33:42.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.852+0000 7f7a9e17d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da893b4d0 0x562da8d24110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.852+0000 7f7a9e17d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da89436b0 0x562da8d24650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.852+0000 7f7a9e17d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da8964390 0x562da8d22590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.852+0000 7f7a9506d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da893b4d0 0x562da8d24110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.852+0000 7f7a9506d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da893b4d0 0x562da8d24110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42910/0 (socket says 172.21.15.43:42910) 2024-04-25T10:33:42.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.852+0000 7f7a9506d640 1 -- 172.21.15.43:0/2137564511 learned_addr learned my addr 172.21.15.43:0/2137564511 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:42.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.852+0000 7f7a9e17d740 1 -- 172.21.15.43:0/2137564511 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7a901b01a0 con 0x562da893b4d0 2024-04-25T10:33:42.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.852+0000 7f7a9486c640 1 --2- 172.21.15.43:0/2137564511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da89436b0 0x562da8d24650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.852+0000 7f7a9e17d740 1 -- 172.21.15.43:0/2137564511 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7a901b1610 con 0x562da8964390 2024-04-25T10:33:42.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.852+0000 7f7a9e17d740 1 -- 172.21.15.43:0/2137564511 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7a901b0870 con 0x562da89436b0 2024-04-25T10:33:42.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.852+0000 7f7a9586e640 1 --2- 172.21.15.43:0/2137564511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da8964390 0x562da8d22590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.852+0000 7f7a9506d640 1 -- 172.21.15.43:0/2137564511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da89436b0 msgr2=0x562da8d24650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:42.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.852+0000 7f7a9506d640 1 --2- 172.21.15.43:0/2137564511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da89436b0 0x562da8d24650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.852+0000 7f7a9506d640 1 -- 172.21.15.43:0/2137564511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da8964390 msgr2=0x562da8d22590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:42.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.852+0000 7f7a9506d640 1 --2- 172.21.15.43:0/2137564511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da8964390 0x562da8d22590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.852+0000 7f7a9506d640 1 -- 172.21.15.43:0/2137564511 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562da8c2b4b0 con 0x562da893b4d0 2024-04-25T10:33:42.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.853+0000 7f7a9e17d740 10 client.0 _ll_register_callbacks cb 0x7f7a80009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:42.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.853+0000 7f7a9506d640 1 --2- 172.21.15.43:0/2137564511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da893b4d0 0x562da8d24110 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0a2f00 tx=0x7f7a8c0a2a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb37e54038b37e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:42.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.853+0000 7f7a9e17d740 1 -- 172.21.15.43:0/2137564511 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a8c0ab230 con 0x562da893b4d0 2024-04-25T10:33:42.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.853+0000 7f7a78ff9640 10 client.15750 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:42.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.853+0000 7f7a78ff9640 1 -- 172.21.15.43:0/2137564511 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c014070 con 0x562da893b4d0 2024-04-25T10:33:42.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.853+0000 7f7a78ff9640 1 -- 172.21.15.43:0/2137564511 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f7a8c0ac3c0 con 0x562da893b4d0 2024-04-25T10:33:42.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.853+0000 7f7a78ff9640 1 -- 172.21.15.43:0/2137564511 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7a8c0a67d0 con 0x562da893b4d0 2024-04-25T10:33:42.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.853+0000 7f7a9e17d740 10 client.15750 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:42.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.853+0000 7f7a9e17d740 1 -- 172.21.15.43:0/2137564511 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({fsmap=24}) -- 0x562da8c2b4b0 con 0x562da893b4d0 2024-04-25T10:33:42.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.853+0000 7f7a78ff9640 1 -- 172.21.15.43:0/2137564511 <== mon.0 v2:172.21.15.43:3300/0 4 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7a8c0c10c0 con 0x562da893b4d0 2024-04-25T10:33:42.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.853+0000 7f7a9e17d740 10 client.15750 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:42.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.854+0000 7f7a9e17d740 10 client.15750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:42.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.854+0000 7f7a9e17d740 20 client.15750 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:42.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.854+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/2137564511 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562da88f83a0 0x562da8d89be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.854+0000 7f7a9e17d740 4 client.15750 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:33:42.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.854+0000 7f7a9e17d740 1 -- 172.21.15.43:0/2137564511 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562da8c72800 con 0x562da88f83a0 2024-04-25T10:33:42.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.854+0000 7f7a9486c640 1 --2- 172.21.15.43:0/2137564511 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562da88f83a0 0x562da8d89be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.854+0000 7f7a9486c640 1 --2- 172.21.15.43:0/2137564511 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562da88f83a0 0x562da8d89be0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce1852b8855fb6b9 server_cookie=639d482d4767926e in_seq=0 out_seq=0 2024-04-25T10:33:42.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.854+0000 7f7a78ff9640 10 client.15750 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:42.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.855+0000 7f7a78ff9640 1 -- 172.21.15.43:0/2137564511 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562da8c72800 con 0x562da88f83a0 2024-04-25T10:33:42.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.855+0000 7f7a78ff9640 10 client.15750 handle_command_reply: tid=0 2024-04-25T10:33:42.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.855+0000 7f7a9e17d740 1 client.15750 shutdown 2024-04-25T10:33:42.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.855+0000 7f7a9e17d740 1 -- 172.21.15.43:0/2137564511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da893b4d0 msgr2=0x562da8d24110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:42.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.855+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/2137564511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da893b4d0 0x562da8d24110 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0a2f00 tx=0x7f7a8c0a2a80 comp rx=0 tx=0).stop 2024-04-25T10:33:42.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.855+0000 7f7a9e17d740 1 -- 172.21.15.43:0/2137564511 shutdown_connections 2024-04-25T10:33:42.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.855+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/2137564511 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562da88f83a0 0x562da8d89be0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.855+0000 7f7a78ff9640 0 client.15750 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:42.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.855+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/2137564511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562da8964390 0x562da8d22590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.855+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/2137564511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562da89436b0 0x562da8d24650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.855+0000 7f7a9e17d740 1 --2- 172.21.15.43:0/2137564511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562da893b4d0 0x562da8d24110 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.855+0000 7f7a9e17d740 1 -- 172.21.15.43:0/2137564511 >> 172.21.15.43:0/2137564511 conn(0x562da89584e0 msgr2=0x562da8cff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:42.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.855+0000 7f7a9e17d740 1 -- 172.21.15.43:0/2137564511 shutdown_connections 2024-04-25T10:33:42.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.855+0000 7f7a9e17d740 1 -- 172.21.15.43:0/2137564511 wait complete. 2024-04-25T10:33:42.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.855+0000 7f7a9e17d740 20 client.15750 trim_cache size 0 max 16384 2024-04-25T10:33:42.862 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:33:42.862 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:33:42.862 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:33:42.862 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:33:42.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.856+0000 7f7a972d1640 1 -- 172.21.15.43:0/4060012750 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7a7c0afe10 msgr2=0x7f7a7c0b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:42.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.856+0000 7f7a972d1640 1 --2- 172.21.15.43:0/4060012750 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7a7c0afe10 0x7f7a7c0b2320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7a88006010 tx=0x7f7a880094a0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.856+0000 7f7a972d1640 1 -- 172.21.15.43:0/4060012750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a90102af0 msgr2=0x7f7a90110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:42.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.856+0000 7f7a972d1640 1 --2- 172.21.15.43:0/4060012750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a90102af0 0x7f7a90110470 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7a90103a20 tx=0x7f7a8c00dbb0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.856+0000 7f7a972d1640 1 -- 172.21.15.43:0/4060012750 shutdown_connections 2024-04-25T10:33:42.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.856+0000 7f7a972d1640 1 --2- 172.21.15.43:0/4060012750 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7a7c0afe10 0x7f7a7c0b2320 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.856+0000 7f7a972d1640 1 --2- 172.21.15.43:0/4060012750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a9010d380 0x7f7a90110ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.856+0000 7f7a972d1640 1 --2- 172.21.15.43:0/4060012750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a90108a20 0x7f7a901109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.856+0000 7f7a972d1640 1 --2- 172.21.15.43:0/4060012750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a90102af0 0x7f7a90110470 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.856+0000 7f7a972d1640 1 -- 172.21.15.43:0/4060012750 >> 172.21.15.43:0/4060012750 conn(0x7f7a90078100 msgr2=0x7f7a901040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:42.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.856+0000 7f7a972d1640 1 -- 172.21.15.43:0/4060012750 shutdown_connections 2024-04-25T10:33:42.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.856+0000 7f7a972d1640 1 -- 172.21.15.43:0/4060012750 wait complete. 2024-04-25T10:33:42.869 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:33:42.870 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:33:42.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.984+0000 7f7797b1f640 1 -- 172.21.15.43:0/1609846094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7790109fb0 msgr2=0x7f7790103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:42.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.984+0000 7f7797b1f640 1 --2- 172.21.15.43:0/1609846094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7790109fb0 0x7f7790103c40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7780009900 tx=0x7f778002a410 comp rx=0 tx=0).stop 2024-04-25T10:33:42.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.984+0000 7f7797b1f640 1 -- 172.21.15.43:0/1609846094 shutdown_connections 2024-04-25T10:33:42.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.984+0000 7f7797b1f640 1 --2- 172.21.15.43:0/1609846094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77901042b0 0x7f7790110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.984+0000 7f7797b1f640 1 --2- 172.21.15.43:0/1609846094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7790109fb0 0x7f7790103c40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.984+0000 7f7797b1f640 1 --2- 172.21.15.43:0/1609846094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7790109b10 0x7f7790103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.984+0000 7f7797b1f640 1 -- 172.21.15.43:0/1609846094 >> 172.21.15.43:0/1609846094 conn(0x7f77900ff360 msgr2=0x7f7790101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:42.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.984+0000 7f7797b1f640 1 -- 172.21.15.43:0/1609846094 shutdown_connections 2024-04-25T10:33:42.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.984+0000 7f7797b1f640 1 -- 172.21.15.43:0/1609846094 wait complete. 2024-04-25T10:33:42.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.985+0000 7f7797b1f640 1 Processor -- start 2024-04-25T10:33:42.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.985+0000 7f7797b1f640 1 -- start start 2024-04-25T10:33:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.985+0000 7f7797b1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77901042b0 0x7f77901a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.985+0000 7f7797b1f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7790109b10 0x7f77901a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.985+0000 7f7797b1f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f779019a680 0x7f779019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.985+0000 7f77958bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77901042b0 0x7f77901a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.985+0000 7f7797b1f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7790113d00 con 0x7f77901042b0 2024-04-25T10:33:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.985+0000 7f77958bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77901042b0 0x7f77901a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42922/0 (socket says 172.21.15.43:42922) 2024-04-25T10:33:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.985+0000 7f7797b1f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7790113b80 con 0x7f7790109b10 2024-04-25T10:33:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.985+0000 7f7797b1f640 1 -- 172.21.15.43:0/1542787814 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7790113e80 con 0x7f779019a680 2024-04-25T10:33:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.985+0000 7f77958bb640 1 -- 172.21.15.43:0/1542787814 learned_addr learned my addr 172.21.15.43:0/1542787814 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.985+0000 7f77950ba640 1 --2- 172.21.15.43:0/1542787814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7790109b10 0x7f77901a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.986+0000 7f77960bc640 1 --2- 172.21.15.43:0/1542787814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f779019a680 0x7f779019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.986+0000 7f77950ba640 1 -- 172.21.15.43:0/1542787814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f779019a680 msgr2=0x7f779019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.986+0000 7f77950ba640 1 --2- 172.21.15.43:0/1542787814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f779019a680 0x7f779019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.986+0000 7f77950ba640 1 -- 172.21.15.43:0/1542787814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77901042b0 msgr2=0x7f77901a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.986+0000 7f77950ba640 1 --2- 172.21.15.43:0/1542787814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77901042b0 0x7f77901a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.986+0000 7f77950ba640 1 -- 172.21.15.43:0/1542787814 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f779019b230 con 0x7f7790109b10 2024-04-25T10:33:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.986+0000 7f77958bb640 1 --2- 172.21.15.43:0/1542787814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77901042b0 0x7f77901a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.986+0000 7f77950ba640 1 --2- 172.21.15.43:0/1542787814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7790109b10 0x7f77901a0b00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f77901047c0 tx=0x7f77800040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.986+0000 7f7786ffd640 1 -- 172.21.15.43:0/1542787814 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778003f070 con 0x7f7790109b10 2024-04-25T10:33:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.986+0000 7f7797b1f640 1 -- 172.21.15.43:0/1542787814 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f779019b4c0 con 0x7f7790109b10 2024-04-25T10:33:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.987+0000 7f7786ffd640 1 -- 172.21.15.43:0/1542787814 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77800044c0 con 0x7f7790109b10 2024-04-25T10:33:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.987+0000 7f7786ffd640 1 -- 172.21.15.43:0/1542787814 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780035070 con 0x7f7790109b10 2024-04-25T10:33:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.987+0000 7f7797b1f640 1 -- 172.21.15.43:0/1542787814 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77900750c0 con 0x7f7790109b10 2024-04-25T10:33:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.988+0000 7f7797b1f640 1 -- 172.21.15.43:0/1542787814 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7790104df0 con 0x7f7790109b10 2024-04-25T10:33:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.989+0000 7f7786ffd640 1 -- 172.21.15.43:0/1542787814 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 147804+0+0 (secure 0 0 0) 0x7f7780043400 con 0x7f7790109b10 2024-04-25T10:33:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.989+0000 7f7786ffd640 1 --2- 172.21.15.43:0/1542787814 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f777c0afe90 0x7f777c0b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.989+0000 7f7786ffd640 1 -- 172.21.15.43:0/1542787814 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f77800ff700 con 0x7f7790109b10 2024-04-25T10:33:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.990+0000 7f77958bb640 1 --2- 172.21.15.43:0/1542787814 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f777c0afe90 0x7f777c0b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:42.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.990+0000 7f77958bb640 1 --2- 172.21.15.43:0/1542787814 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f777c0afe90 0x7f777c0b23a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f778c005bc0 tx=0x7f778c00f890 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:42.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:42.994+0000 7f7786ffd640 1 -- 172.21.15.43:0/1542787814 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77800c7d90 con 0x7f7790109b10 2024-04-25T10:33:43.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:42 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:33:43.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:42 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:33:43.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:42 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:33:43.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.224+0000 7f7797b1f640 1 -- 172.21.15.43:0/1542787814 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77900752f0 con 0x7f7790109b10 2024-04-25T10:33:43.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.225+0000 7f7786ffd640 1 -- 172.21.15.43:0/1542787814 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11869 (secure 0 0 0) 0x7f77800c1e20 con 0x7f7790109b10 2024-04-25T10:33:43.227 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:33:43.227 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:32:30.234841+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2595248466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2595248466},{"type":"v1","addr":"172.21.15.77:6837","nonce":2595248466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24472,24508]},"id":1}]} 2024-04-25T10:33:43.228 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 24 2024-04-25T10:33:43.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.227+0000 7f7797b1f640 1 -- 172.21.15.43:0/1542787814 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f777c0afe90 msgr2=0x7f777c0b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.227+0000 7f7797b1f640 1 --2- 172.21.15.43:0/1542787814 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f777c0afe90 0x7f777c0b23a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f778c005bc0 tx=0x7f778c00f890 comp rx=0 tx=0).stop 2024-04-25T10:33:43.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.228+0000 7f7797b1f640 1 -- 172.21.15.43:0/1542787814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7790109b10 msgr2=0x7f77901a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.228+0000 7f7797b1f640 1 --2- 172.21.15.43:0/1542787814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7790109b10 0x7f77901a0b00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f77901047c0 tx=0x7f77800040c0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.228+0000 7f7797b1f640 1 -- 172.21.15.43:0/1542787814 shutdown_connections 2024-04-25T10:33:43.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.228+0000 7f7797b1f640 1 --2- 172.21.15.43:0/1542787814 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f777c0afe90 0x7f777c0b23a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.228+0000 7f7797b1f640 1 --2- 172.21.15.43:0/1542787814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7790109b10 0x7f77901a0b00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.228+0000 7f7797b1f640 1 --2- 172.21.15.43:0/1542787814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f779019a680 0x7f779019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.228+0000 7f7797b1f640 1 --2- 172.21.15.43:0/1542787814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77901042b0 0x7f77901a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.228+0000 7f7797b1f640 1 -- 172.21.15.43:0/1542787814 >> 172.21.15.43:0/1542787814 conn(0x7f77900ff360 msgr2=0x7f779010f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:43.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.228+0000 7f7797b1f640 1 -- 172.21.15.43:0/1542787814 shutdown_connections 2024-04-25T10:33:43.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.228+0000 7f7797b1f640 1 -- 172.21.15.43:0/1542787814 wait complete. 2024-04-25T10:33:43.242 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:33:43.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:42 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:33:43.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:42 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:33:43.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:42 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:33:43.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:42 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:33:43.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:42 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:33:43.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:42 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:33:43.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.360+0000 7f35bec34640 1 -- 172.21.15.43:0/3512064806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35b8106990 msgr2=0x7f35b8106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.360+0000 7f35bec34640 1 --2- 172.21.15.43:0/3512064806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35b8106990 0x7f35b8106dc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009900 tx=0x7f35a402a410 comp rx=0 tx=0).stop 2024-04-25T10:33:43.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.360+0000 7f35bec34640 1 -- 172.21.15.43:0/3512064806 shutdown_connections 2024-04-25T10:33:43.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.360+0000 7f35bec34640 1 --2- 172.21.15.43:0/3512064806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35b8107450 0x7f35b810ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.360+0000 7f35bec34640 1 --2- 172.21.15.43:0/3512064806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35b8106990 0x7f35b8106dc0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.360+0000 7f35bec34640 1 --2- 172.21.15.43:0/3512064806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35b8102400 0x7f35b8102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.360+0000 7f35bec34640 1 -- 172.21.15.43:0/3512064806 >> 172.21.15.43:0/3512064806 conn(0x7f35b8077880 msgr2=0x7f35b8077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:43.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.360+0000 7f35bec34640 1 -- 172.21.15.43:0/3512064806 shutdown_connections 2024-04-25T10:33:43.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.360+0000 7f35bec34640 1 -- 172.21.15.43:0/3512064806 wait complete. 2024-04-25T10:33:43.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.361+0000 7f35bec34640 1 Processor -- start 2024-04-25T10:33:43.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.361+0000 7f35bec34640 1 -- start start 2024-04-25T10:33:43.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.361+0000 7f35bec34640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35b8102400 0x7f35b8198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.361+0000 7f35bec34640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35b8106990 0x7f35b8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.362+0000 7f35bec34640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35b8107450 0x7f35b819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.362+0000 7f35bec34640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f35b8111ae0 con 0x7f35b8106990 2024-04-25T10:33:43.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.362+0000 7f35bec34640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35b8111960 con 0x7f35b8107450 2024-04-25T10:33:43.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.362+0000 7f35bec34640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f35b8111c60 con 0x7f35b8102400 2024-04-25T10:33:43.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.362+0000 7f35b7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35b8106990 0x7f35b8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.362+0000 7f35b7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35b8106990 0x7f35b8198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42928/0 (socket says 172.21.15.43:42928) 2024-04-25T10:33:43.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.362+0000 7f35b7fff640 1 -- 172.21.15.43:0/3761132161 learned_addr learned my addr 172.21.15.43:0/3761132161 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:43.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.362+0000 7f35bd1d1640 1 --2- 172.21.15.43:0/3761132161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35b8107450 0x7f35b819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.362+0000 7f35bc9d0640 1 --2- 172.21.15.43:0/3761132161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35b8102400 0x7f35b8198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.362+0000 7f35b7fff640 1 -- 172.21.15.43:0/3761132161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35b8102400 msgr2=0x7f35b8198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.362+0000 7f35b7fff640 1 --2- 172.21.15.43:0/3761132161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35b8102400 0x7f35b8198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.362+0000 7f35b7fff640 1 -- 172.21.15.43:0/3761132161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35b8107450 msgr2=0x7f35b819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.362+0000 7f35b7fff640 1 --2- 172.21.15.43:0/3761132161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35b8107450 0x7f35b819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.362+0000 7f35b7fff640 1 -- 172.21.15.43:0/3761132161 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b819d360 con 0x7f35b8106990 2024-04-25T10:33:43.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.362+0000 7f35bd1d1640 1 --2- 172.21.15.43:0/3761132161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35b8107450 0x7f35b819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:43.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.362+0000 7f35b7fff640 1 --2- 172.21.15.43:0/3761132161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35b8106990 0x7f35b8198ae0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f35a4004060 tx=0x7f35a4004090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:43.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.363+0000 7f35b5ffb640 1 -- 172.21.15.43:0/3761132161 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4046070 con 0x7f35b8106990 2024-04-25T10:33:43.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.363+0000 7f35b5ffb640 1 -- 172.21.15.43:0/3761132161 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35a4003aa0 con 0x7f35b8106990 2024-04-25T10:33:43.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.363+0000 7f35bec34640 1 -- 172.21.15.43:0/3761132161 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b819d5f0 con 0x7f35b8106990 2024-04-25T10:33:43.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.363+0000 7f35b5ffb640 1 -- 172.21.15.43:0/3761132161 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4035050 con 0x7f35b8106990 2024-04-25T10:33:43.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.363+0000 7f35bec34640 1 -- 172.21.15.43:0/3761132161 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b819db30 con 0x7f35b8106990 2024-04-25T10:33:43.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.366+0000 7f35b5ffb640 1 -- 172.21.15.43:0/3761132161 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 147804+0+0 (secure 0 0 0) 0x7f35a4049070 con 0x7f35b8106990 2024-04-25T10:33:43.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.367+0000 7f35c5bc3740 1 Processor -- start 2024-04-25T10:33:43.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.367+0000 7f35b5ffb640 1 --2- 172.21.15.43:0/3761132161 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f35a80afe90 0x7f35a80b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.367+0000 7f35c5bc3740 1 -- start start 2024-04-25T10:33:43.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.367+0000 7f35b5ffb640 1 -- 172.21.15.43:0/3761132161 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f35a4100d90 con 0x7f35b8106990 2024-04-25T10:33:43.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.367+0000 7f35bc9d0640 1 --2- 172.21.15.43:0/3761132161 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f35a80afe90 0x7f35a80b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.367+0000 7f35c5bc3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb45dca390 0x55fb4616b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.367+0000 7f35c5bc3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45da14d0 0x55fb4616bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.367+0000 7f35c5bc3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da96b0 0x55fb461786a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.367+0000 7f35c5bc3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fb460d0d80 con 0x55fb45da14d0 2024-04-25T10:33:43.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.367+0000 7f35c5bc3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fb4608aa20 con 0x55fb45da96b0 2024-04-25T10:33:43.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.367+0000 7f35c5bc3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fb460ccd40 con 0x55fb45dca390 2024-04-25T10:33:43.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.367+0000 7f35b7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45da14d0 0x55fb4616bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.367+0000 7f35b7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45da14d0 0x55fb4616bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42942/0 (socket says 172.21.15.43:42942) 2024-04-25T10:33:43.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.367+0000 7f35b7fff640 1 -- 172.21.15.43:0/4007371133 learned_addr learned my addr 172.21.15.43:0/4007371133 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:43.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.368+0000 7f35bd1d1640 1 --2- 172.21.15.43:0/4007371133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da96b0 0x55fb461786a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.368+0000 7f35b7fff640 1 -- 172.21.15.43:0/4007371133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb45dca390 msgr2=0x55fb4616b810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:33:43.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.368+0000 7f35b7fff640 1 --2- 172.21.15.43:0/4007371133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb45dca390 0x55fb4616b810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.368+0000 7f35b7fff640 1 -- 172.21.15.43:0/4007371133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da96b0 msgr2=0x55fb461786a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.368+0000 7f35b7fff640 1 --2- 172.21.15.43:0/4007371133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da96b0 0x55fb461786a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.368+0000 7f35b7fff640 1 -- 172.21.15.43:0/4007371133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb46095c30 con 0x55fb45da14d0 2024-04-25T10:33:43.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.368+0000 7f35b7fff640 1 --2- 172.21.15.43:0/4007371133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45da14d0 0x55fb4616bd50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f35a4031440 tx=0x7f35a4003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bdaef8a55f096a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:43.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.368+0000 7f35bc9d0640 1 --2- 172.21.15.43:0/3761132161 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f35a80afe90 0x7f35a80b23a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00ac90 tx=0x7f35ac010040 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:43.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.368+0000 7f35a2ffd640 1 -- 172.21.15.43:0/4007371133 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a40df020 con 0x55fb45da14d0 2024-04-25T10:33:43.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.368+0000 7f35a2ffd640 1 -- 172.21.15.43:0/4007371133 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35a40c8dd0 con 0x55fb45da14d0 2024-04-25T10:33:43.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.369+0000 7f35a2ffd640 1 -- 172.21.15.43:0/4007371133 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a40c9030 con 0x55fb45da14d0 2024-04-25T10:33:43.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.369+0000 7f35c5bc3740 1 -- 172.21.15.43:0/4007371133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45da14d0 msgr2=0x55fb4616bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.369+0000 7f35c5bc3740 1 --2- 172.21.15.43:0/4007371133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45da14d0 0x55fb4616bd50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f35a4031440 tx=0x7f35a4003f50 comp rx=0 tx=0).stop 2024-04-25T10:33:43.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.370+0000 7f35c5bc3740 1 -- 172.21.15.43:0/4007371133 shutdown_connections 2024-04-25T10:33:43.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.370+0000 7f35c5bc3740 1 --2- 172.21.15.43:0/4007371133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da96b0 0x55fb461786a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.370+0000 7f35c5bc3740 1 --2- 172.21.15.43:0/4007371133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45da14d0 0x55fb4616bd50 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.370+0000 7f35c5bc3740 1 --2- 172.21.15.43:0/4007371133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb45dca390 0x55fb4616b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.370+0000 7f35c5bc3740 1 -- 172.21.15.43:0/4007371133 >> 172.21.15.43:0/4007371133 conn(0x55fb45dbe4e0 msgr2=0x55fb46167fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:43.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.370+0000 7f35c5bc3740 1 -- 172.21.15.43:0/4007371133 shutdown_connections 2024-04-25T10:33:43.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.370+0000 7f35c5bc3740 1 -- 172.21.15.43:0/4007371133 wait complete. 2024-04-25T10:33:43.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.370+0000 7f35c5bc3740 1 -- start start 2024-04-25T10:33:43.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.371+0000 7f35c5bc3740 1 Processor -- start 2024-04-25T10:33:43.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.371+0000 7f35c5bc3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da14d0 0x55fb46175f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.372+0000 7f35c5bc3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb45da96b0 0x55fb46176460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.372+0000 7f35c5bc3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45dca390 0x55fb46182010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.372+0000 7f35bc9d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da14d0 0x55fb46175f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.372+0000 7f35bc9d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da14d0 0x55fb46175f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:53648/0 (socket says 172.21.15.43:53648) 2024-04-25T10:33:43.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.372+0000 7f35bd1d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45dca390 0x55fb46182010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.372+0000 7f35bd1d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45dca390 0x55fb46182010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42958/0 (socket says 172.21.15.43:42958) 2024-04-25T10:33:43.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.372+0000 7f35bc9d0640 1 -- 172.21.15.43:0/1704848200 learned_addr learned my addr 172.21.15.43:0/1704848200 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:43.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.372+0000 7f35b7fff640 1 --2- 172.21.15.43:0/1704848200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb45da96b0 0x55fb46176460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.372+0000 7f35c5bc3740 1 -- 172.21.15.43:0/1704848200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fb46178be0 con 0x55fb45dca390 2024-04-25T10:33:43.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.372+0000 7f35c5bc3740 1 -- 172.21.15.43:0/1704848200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fb46057dc0 con 0x55fb45da14d0 2024-04-25T10:33:43.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.372+0000 7f35c5bc3740 1 -- 172.21.15.43:0/1704848200 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fb46178d60 con 0x55fb45da96b0 2024-04-25T10:33:43.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.372+0000 7f35c5bc3740 10 client.0 _ll_register_callbacks cb 0x55fb46073600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:43.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.372+0000 7f35bd1d1640 1 -- 172.21.15.43:0/1704848200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb45da96b0 msgr2=0x55fb46176460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.372+0000 7f35bd1d1640 1 --2- 172.21.15.43:0/1704848200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb45da96b0 0x55fb46176460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.372+0000 7f35bd1d1640 1 -- 172.21.15.43:0/1704848200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da14d0 msgr2=0x55fb46175f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.372+0000 7f35bd1d1640 1 --2- 172.21.15.43:0/1704848200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da14d0 0x55fb46175f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.372+0000 7f35bd1d1640 1 -- 172.21.15.43:0/1704848200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb45de23f0 con 0x55fb45dca390 2024-04-25T10:33:43.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.373+0000 7f35bc9d0640 1 --2- 172.21.15.43:0/1704848200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da14d0 0x55fb46175f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:43.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.373+0000 7f35bd1d1640 1 --2- 172.21.15.43:0/1704848200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45dca390 0x55fb46182010 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f35b0010f10 tx=0x7f35b000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62641e4facaa759 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:43.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.373+0000 7f3597fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:43.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.373+0000 7f35c5bc3740 1 -- 172.21.15.43:0/1704848200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35a40c8dd0 con 0x55fb45dca390 2024-04-25T10:33:43.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.373+0000 7f3597fff640 1 -- 172.21.15.43:0/1704848200 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0003d80 con 0x55fb45dca390 2024-04-25T10:33:43.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.373+0000 7f3597fff640 1 -- 172.21.15.43:0/1704848200 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f35b0004050 con 0x55fb45dca390 2024-04-25T10:33:43.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.374+0000 7f3597fff640 1 -- 172.21.15.43:0/1704848200 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f35b0027050 con 0x55fb45dca390 2024-04-25T10:33:43.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.374+0000 7f35c5bc3740 10 client.15768 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:43.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.374+0000 7f35c5bc3740 1 -- 172.21.15.43:0/1704848200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({fsmap=24}) -- 0x55fb45eac7d0 con 0x55fb45dca390 2024-04-25T10:33:43.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.374+0000 7f3597fff640 1 -- 172.21.15.43:0/1704848200 <== mon.0 v2:172.21.15.43:3300/0 4 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f35b0004250 con 0x55fb45dca390 2024-04-25T10:33:43.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.374+0000 7f35c5bc3740 10 client.15768 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:43.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.374+0000 7f35c5bc3740 10 client.15768 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:33:43.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.374+0000 7f35c5bc3740 20 client.15768 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:43.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.374+0000 7f35c5bc3740 1 --2- 172.21.15.43:0/1704848200 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55fb45d88f60 0x55fb46188000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.374+0000 7f35c5bc3740 4 client.15768 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:33:43.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.375+0000 7f35c5bc3740 1 -- 172.21.15.43:0/1704848200 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f35a40c8dd0 con 0x55fb45d88f60 2024-04-25T10:33:43.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.375+0000 7f35bc9d0640 1 --2- 172.21.15.43:0/1704848200 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55fb45d88f60 0x55fb46188000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.375+0000 7f35bc9d0640 1 --2- 172.21.15.43:0/1704848200 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55fb45d88f60 0x55fb46188000 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc53b0c3e3a8c6bc server_cookie=d6f7127411e01e85 in_seq=0 out_seq=0 2024-04-25T10:33:43.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.375+0000 7f3597fff640 10 client.15768 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:33:43.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.376+0000 7f3597fff640 1 -- 172.21.15.43:0/1704848200 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f35a40c8dd0 con 0x55fb45d88f60 2024-04-25T10:33:43.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.376+0000 7f3597fff640 10 client.15768 handle_command_reply: tid=0 2024-04-25T10:33:43.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.376+0000 7f35c5bc3740 1 client.15768 shutdown 2024-04-25T10:33:43.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.376+0000 7f35c5bc3740 1 -- 172.21.15.43:0/1704848200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45dca390 msgr2=0x55fb46182010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.376+0000 7f35c5bc3740 1 --2- 172.21.15.43:0/1704848200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45dca390 0x55fb46182010 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f35b0010f10 tx=0x7f35b000f7f0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.376+0000 7f35c5bc3740 1 -- 172.21.15.43:0/1704848200 shutdown_connections 2024-04-25T10:33:43.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.376+0000 7f35c5bc3740 1 --2- 172.21.15.43:0/1704848200 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55fb45d88f60 0x55fb46188000 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.376+0000 7f35c5bc3740 1 --2- 172.21.15.43:0/1704848200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45dca390 0x55fb46182010 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.376+0000 7f3597fff640 0 client.15768 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:33:43.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.376+0000 7f35c5bc3740 1 --2- 172.21.15.43:0/1704848200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb45da96b0 0x55fb46176460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.376+0000 7f35c5bc3740 1 --2- 172.21.15.43:0/1704848200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da14d0 0x55fb46175f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.376+0000 7f35c5bc3740 1 -- 172.21.15.43:0/1704848200 >> 172.21.15.43:0/1704848200 conn(0x55fb45dbe4e0 msgr2=0x55fb4616d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:43.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.377+0000 7f35c5bc3740 1 -- 172.21.15.43:0/1704848200 shutdown_connections 2024-04-25T10:33:43.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.377+0000 7f35c5bc3740 1 -- 172.21.15.43:0/1704848200 wait complete. 2024-04-25T10:33:43.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.377+0000 7f35c5bc3740 20 client.15768 trim_cache size 0 max 16384 2024-04-25T10:33:43.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.404+0000 7f35c5bc3740 1 Processor -- start 2024-04-25T10:33:43.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.404+0000 7f35c5bc3740 1 -- start start 2024-04-25T10:33:43.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.404+0000 7f35c5bc3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb45d88f60 0x55fb4616de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.404+0000 7f35c5bc3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da14d0 0x55fb46181c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.404+0000 7f35c5bc3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45da96b0 0x55fb4617dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.404+0000 7f35c5bc3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81affa0 con 0x55fb45da96b0 2024-04-25T10:33:43.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.404+0000 7f35c5bc3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81ad870 con 0x55fb45da14d0 2024-04-25T10:33:43.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.404+0000 7f35c5bc3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81ab6c0 con 0x55fb45d88f60 2024-04-25T10:33:43.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.404+0000 7f35bd1d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45da96b0 0x55fb4617dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.404+0000 7f35b7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da14d0 0x55fb46181c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.404+0000 7f35b7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da14d0 0x55fb46181c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:53660/0 (socket says 172.21.15.43:53660) 2024-04-25T10:33:43.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.404+0000 7f35b7fff640 1 -- 172.21.15.43:0/2550794993 learned_addr learned my addr 172.21.15.43:0/2550794993 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:43.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.404+0000 7f35bd1d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45da96b0 0x55fb4617dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42970/0 (socket says 172.21.15.43:42970) 2024-04-25T10:33:43.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.404+0000 7f35bc9d0640 1 --2- 172.21.15.43:0/2550794993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb45d88f60 0x55fb4616de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.405+0000 7f35b7fff640 1 -- 172.21.15.43:0/2550794993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb45d88f60 msgr2=0x55fb4616de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.405+0000 7f35b7fff640 1 --2- 172.21.15.43:0/2550794993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb45d88f60 0x55fb4616de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.405+0000 7f35b7fff640 1 -- 172.21.15.43:0/2550794993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45da96b0 msgr2=0x55fb4617dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.405+0000 7f35b7fff640 1 --2- 172.21.15.43:0/2550794993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45da96b0 0x55fb4617dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.405+0000 7f35b7fff640 1 -- 172.21.15.43:0/2550794993 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb46168bb0 con 0x55fb45da14d0 2024-04-25T10:33:43.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.405+0000 7f35b7fff640 1 --2- 172.21.15.43:0/2550794993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da14d0 0x55fb46181c30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f35a40c6040 tx=0x7f35a4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae86330e9eedede6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:43.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.405+0000 7f35a27fc640 1 -- 172.21.15.43:0/2550794993 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a40df040 con 0x55fb45da14d0 2024-04-25T10:33:43.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.405+0000 7f35a27fc640 1 -- 172.21.15.43:0/2550794993 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35a4038380 con 0x55fb45da14d0 2024-04-25T10:33:43.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.405+0000 7f35a27fc640 1 -- 172.21.15.43:0/2550794993 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a40c9030 con 0x55fb45da14d0 2024-04-25T10:33:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.406+0000 7f35c5bc3740 1 -- 172.21.15.43:0/2550794993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da14d0 msgr2=0x55fb46181c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.406+0000 7f35c5bc3740 1 --2- 172.21.15.43:0/2550794993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da14d0 0x55fb46181c30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f35a40c6040 tx=0x7f35a4009260 comp rx=0 tx=0).stop 2024-04-25T10:33:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.406+0000 7f35c5bc3740 1 -- 172.21.15.43:0/2550794993 shutdown_connections 2024-04-25T10:33:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.406+0000 7f35c5bc3740 1 --2- 172.21.15.43:0/2550794993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da14d0 0x55fb46181c30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.406+0000 7f35c5bc3740 1 --2- 172.21.15.43:0/2550794993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45da96b0 0x55fb4617dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.406+0000 7f35c5bc3740 1 --2- 172.21.15.43:0/2550794993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb45d88f60 0x55fb4616de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.407+0000 7f35c5bc3740 1 -- 172.21.15.43:0/2550794993 >> 172.21.15.43:0/2550794993 conn(0x55fb45dbe4e0 msgr2=0x55fb46177640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.407+0000 7f35c5bc3740 1 -- 172.21.15.43:0/2550794993 shutdown_connections 2024-04-25T10:33:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.407+0000 7f35c5bc3740 1 -- 172.21.15.43:0/2550794993 wait complete. 2024-04-25T10:33:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.407+0000 7f35c5bc3740 1 -- start start 2024-04-25T10:33:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.407+0000 7f35c5bc3740 1 Processor -- start 2024-04-25T10:33:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.408+0000 7f35c5bc3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb45d88f60 0x55fb461e77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.408+0000 7f35c5bc3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45da14d0 0x55fb461e7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.408+0000 7f35c5bc3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da96b0 0x55fb4617d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.408+0000 7f35c5bc3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81abd30 con 0x55fb45da14d0 2024-04-25T10:33:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.408+0000 7f35c5bc3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81ad1a0 con 0x55fb45da96b0 2024-04-25T10:33:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.408+0000 7f35c5bc3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81ac400 con 0x55fb45d88f60 2024-04-25T10:33:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.408+0000 7f35c5bc3740 10 client.0 _ll_register_callbacks cb 0x7f35b0007dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.408+0000 7f35b7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45da14d0 0x55fb461e7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.408+0000 7f35b7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45da14d0 0x55fb461e7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42986/0 (socket says 172.21.15.43:42986) 2024-04-25T10:33:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.408+0000 7f35b7fff640 1 -- 172.21.15.43:0/2207681655 learned_addr learned my addr 172.21.15.43:0/2207681655 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.408+0000 7f35bd1d1640 1 --2- 172.21.15.43:0/2207681655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da96b0 0x55fb4617d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.408+0000 7f35bc9d0640 1 --2- 172.21.15.43:0/2207681655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb45d88f60 0x55fb461e77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.408+0000 7f35b7fff640 1 -- 172.21.15.43:0/2207681655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb45d88f60 msgr2=0x55fb461e77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.408+0000 7f35b7fff640 1 --2- 172.21.15.43:0/2207681655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb45d88f60 0x55fb461e77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.408+0000 7f35b7fff640 1 -- 172.21.15.43:0/2207681655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da96b0 msgr2=0x55fb4617d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.408+0000 7f35b7fff640 1 --2- 172.21.15.43:0/2207681655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da96b0 0x55fb4617d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.408+0000 7f35b7fff640 1 -- 172.21.15.43:0/2207681655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb4617db60 con 0x55fb45da14d0 2024-04-25T10:33:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.408+0000 7f35bd1d1640 1 --2- 172.21.15.43:0/2207681655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da96b0 0x55fb4617d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.409+0000 7f35b7fff640 1 --2- 172.21.15.43:0/2207681655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45da14d0 0x55fb461e7d20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f35a40c8ad0 tx=0x7f35a40c9260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befb5b3254560530 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.409+0000 7f3597fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.409+0000 7f3597fff640 1 -- 172.21.15.43:0/2207681655 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a40df070 con 0x55fb45da14d0 2024-04-25T10:33:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.409+0000 7f3597fff640 1 -- 172.21.15.43:0/2207681655 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f35a40c53c0 con 0x55fb45da14d0 2024-04-25T10:33:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.409+0000 7f35c5bc3740 1 -- 172.21.15.43:0/2207681655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35a4038380 con 0x55fb45da14d0 2024-04-25T10:33:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.409+0000 7f3597fff640 1 -- 172.21.15.43:0/2207681655 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f35a40d1b80 con 0x55fb45da14d0 2024-04-25T10:33:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.409+0000 7f35c5bc3740 10 client.15774 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.409+0000 7f35c5bc3740 1 -- 172.21.15.43:0/2207681655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fb461efef0 con 0x55fb45da14d0 2024-04-25T10:33:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.409+0000 7f3597fff640 1 -- 172.21.15.43:0/2207681655 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35a40c8dd0 con 0x55fb45da14d0 2024-04-25T10:33:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.409+0000 7f3597fff640 1 -- 172.21.15.43:0/2207681655 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f35a4031ac0 con 0x55fb45da14d0 2024-04-25T10:33:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.409+0000 7f35c5bc3740 10 client.15774 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.409+0000 7f35c5bc3740 10 client.15774 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:33:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.409+0000 7f35c5bc3740 20 client.15774 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.409+0000 7f35c5bc3740 1 --2- 172.21.15.43:0/2207681655 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55fb45dca390 0x55fb461f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.409+0000 7f35c5bc3740 4 client.15774 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:33:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.409+0000 7f35c5bc3740 1 -- 172.21.15.43:0/2207681655 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f35a4038380 con 0x55fb45dca390 2024-04-25T10:33:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.410+0000 7f35bc9d0640 1 --2- 172.21.15.43:0/2207681655 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55fb45dca390 0x55fb461f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.410+0000 7f3597fff640 10 client.15774 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:33:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.410+0000 7f35bc9d0640 1 --2- 172.21.15.43:0/2207681655 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55fb45dca390 0x55fb461f23d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb4a6af63842e2a7 server_cookie=e9309e7a4929fe98 in_seq=0 out_seq=0 2024-04-25T10:33:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.411+0000 7f3597fff640 1 -- 172.21.15.43:0/2207681655 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f35a4038380 con 0x55fb45dca390 2024-04-25T10:33:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.411+0000 7f3597fff640 10 client.15774 handle_command_reply: tid=0 2024-04-25T10:33:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.411+0000 7f35c5bc3740 1 client.15774 shutdown 2024-04-25T10:33:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.411+0000 7f35c5bc3740 1 -- 172.21.15.43:0/2207681655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45da14d0 msgr2=0x55fb461e7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.411+0000 7f35c5bc3740 1 --2- 172.21.15.43:0/2207681655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45da14d0 0x55fb461e7d20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f35a40c8ad0 tx=0x7f35a40c9260 comp rx=0 tx=0).stop 2024-04-25T10:33:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.411+0000 7f35c5bc3740 1 -- 172.21.15.43:0/2207681655 shutdown_connections 2024-04-25T10:33:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.411+0000 7f35c5bc3740 1 --2- 172.21.15.43:0/2207681655 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55fb45dca390 0x55fb461f23d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.411+0000 7f3597fff640 0 client.15774 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:33:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.411+0000 7f35c5bc3740 1 --2- 172.21.15.43:0/2207681655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb45da96b0 0x55fb4617d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.411+0000 7f35c5bc3740 1 --2- 172.21.15.43:0/2207681655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb45da14d0 0x55fb461e7d20 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.411+0000 7f35c5bc3740 1 --2- 172.21.15.43:0/2207681655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb45d88f60 0x55fb461e77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.411+0000 7f35c5bc3740 1 -- 172.21.15.43:0/2207681655 >> 172.21.15.43:0/2207681655 conn(0x55fb45dbe4e0 msgr2=0x55fb4616dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.411+0000 7f35c5bc3740 1 -- 172.21.15.43:0/2207681655 shutdown_connections 2024-04-25T10:33:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.411+0000 7f35c5bc3740 1 -- 172.21.15.43:0/2207681655 wait complete. 2024-04-25T10:33:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.411+0000 7f35c5bc3740 20 client.15774 trim_cache size 0 max 16384 2024-04-25T10:33:43.418 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:33:43.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.412+0000 7f35bec34640 1 -- 172.21.15.43:0/3761132161 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f35a80afe90 msgr2=0x7f35a80b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.412+0000 7f35bec34640 1 --2- 172.21.15.43:0/3761132161 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f35a80afe90 0x7f35a80b23a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00ac90 tx=0x7f35ac010040 comp rx=0 tx=0).stop 2024-04-25T10:33:43.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.412+0000 7f35bec34640 1 -- 172.21.15.43:0/3761132161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35b8106990 msgr2=0x7f35b8198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.412+0000 7f35bec34640 1 --2- 172.21.15.43:0/3761132161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35b8106990 0x7f35b8198ae0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f35a4004060 tx=0x7f35a4004090 comp rx=0 tx=0).stop 2024-04-25T10:33:43.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.412+0000 7f35bec34640 1 -- 172.21.15.43:0/3761132161 shutdown_connections 2024-04-25T10:33:43.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.412+0000 7f35bec34640 1 --2- 172.21.15.43:0/3761132161 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f35a80afe90 0x7f35a80b23a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.412+0000 7f35bec34640 1 --2- 172.21.15.43:0/3761132161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35b8107450 0x7f35b819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.412+0000 7f35bec34640 1 --2- 172.21.15.43:0/3761132161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35b8106990 0x7f35b8198ae0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.412+0000 7f35bec34640 1 --2- 172.21.15.43:0/3761132161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35b8102400 0x7f35b8198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.412+0000 7f35bec34640 1 -- 172.21.15.43:0/3761132161 >> 172.21.15.43:0/3761132161 conn(0x7f35b8077880 msgr2=0x7f35b81008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:43.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.412+0000 7f35bec34640 1 -- 172.21.15.43:0/3761132161 shutdown_connections 2024-04-25T10:33:43.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.412+0000 7f35bec34640 1 -- 172.21.15.43:0/3761132161 wait complete. 2024-04-25T10:33:43.425 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:33:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.543+0000 7f6abf2ee640 1 -- 172.21.15.43:0/3101392338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab81027c0 msgr2=0x7f6ab8110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.543+0000 7f6abf2ee640 1 --2- 172.21.15.43:0/3101392338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab81027c0 0x7f6ab8110640 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009ca0 tx=0x7f6ab002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.543+0000 7f6abf2ee640 1 -- 172.21.15.43:0/3101392338 shutdown_connections 2024-04-25T10:33:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.543+0000 7f6abf2ee640 1 --2- 172.21.15.43:0/3101392338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab81027c0 0x7f6ab8110640 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.543+0000 7f6abf2ee640 1 --2- 172.21.15.43:0/3101392338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ab8109fc0 0x7f6ab8102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.543+0000 7f6abf2ee640 1 --2- 172.21.15.43:0/3101392338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ab8109b20 0x7f6ab8101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.543+0000 7f6abf2ee640 1 -- 172.21.15.43:0/3101392338 >> 172.21.15.43:0/3101392338 conn(0x7f6ab8077880 msgr2=0x7f6ab8077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.543+0000 7f6abf2ee640 1 -- 172.21.15.43:0/3101392338 shutdown_connections 2024-04-25T10:33:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.543+0000 7f6abf2ee640 1 -- 172.21.15.43:0/3101392338 wait complete. 2024-04-25T10:33:43.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.544+0000 7f6abf2ee640 1 Processor -- start 2024-04-25T10:33:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.544+0000 7f6abf2ee640 1 -- start start 2024-04-25T10:33:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.544+0000 7f6abf2ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ab8109b20 0x7f6ab8102db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.544+0000 7f6abf2ee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ab81032f0 0x7f6ab8103700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.544+0000 7f6abf2ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab81a6130 0x7f6ab81a8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.544+0000 7f6abf2ee640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab8113d30 con 0x7f6ab81a6130 2024-04-25T10:33:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.544+0000 7f6abf2ee640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab8113bb0 con 0x7f6ab8109b20 2024-04-25T10:33:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.544+0000 7f6abf2ee640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab8113eb0 con 0x7f6ab81032f0 2024-04-25T10:33:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.544+0000 7f6abd88b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab81a6130 0x7f6ab81a8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.545+0000 7f6abd88b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab81a6130 0x7f6ab81a8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43012/0 (socket says 172.21.15.43:43012) 2024-04-25T10:33:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.545+0000 7f6abd88b640 1 -- 172.21.15.43:0/3164569614 learned_addr learned my addr 172.21.15.43:0/3164569614 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.545+0000 7f6abd08a640 1 --2- 172.21.15.43:0/3164569614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ab8109b20 0x7f6ab8102db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.545+0000 7f6abc889640 1 --2- 172.21.15.43:0/3164569614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ab81032f0 0x7f6ab8103700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.545+0000 7f6abd88b640 1 -- 172.21.15.43:0/3164569614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ab81032f0 msgr2=0x7f6ab8103700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.545+0000 7f6abd88b640 1 --2- 172.21.15.43:0/3164569614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ab81032f0 0x7f6ab8103700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.545+0000 7f6abd88b640 1 -- 172.21.15.43:0/3164569614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ab8109b20 msgr2=0x7f6ab8102db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.545+0000 7f6abd88b640 1 --2- 172.21.15.43:0/3164569614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ab8109b20 0x7f6ab8102db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.545+0000 7f6abd88b640 1 -- 172.21.15.43:0/3164569614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab81a8b10 con 0x7f6ab81a6130 2024-04-25T10:33:43.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.545+0000 7f6abd88b640 1 --2- 172.21.15.43:0/3164569614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab81a6130 0x7f6ab81a8550 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6ab00098d0 tx=0x7f6ab00317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:43.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.545+0000 7f6aae7fc640 1 -- 172.21.15.43:0/3164569614 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0003b70 con 0x7f6ab81a6130 2024-04-25T10:33:43.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.545+0000 7f6abf2ee640 1 -- 172.21.15.43:0/3164569614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab81a8da0 con 0x7f6ab81a6130 2024-04-25T10:33:43.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.545+0000 7f6aae7fc640 1 -- 172.21.15.43:0/3164569614 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ab0003d10 con 0x7f6ab81a6130 2024-04-25T10:33:43.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.546+0000 7f6aae7fc640 1 -- 172.21.15.43:0/3164569614 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab002fa00 con 0x7f6ab81a6130 2024-04-25T10:33:43.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.546+0000 7f6abf2ee640 1 -- 172.21.15.43:0/3164569614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab81a9260 con 0x7f6ab81a6130 2024-04-25T10:33:43.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.549+0000 7f6ac619e740 1 Processor -- start 2024-04-25T10:33:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.550+0000 7f6aae7fc640 1 -- 172.21.15.43:0/3164569614 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 147804+0+0 (secure 0 0 0) 0x7f6ab002fbc0 con 0x7f6ab81a6130 2024-04-25T10:33:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.550+0000 7f6ac619e740 1 -- start start 2024-04-25T10:33:43.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.550+0000 7f6ac619e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f898370390 0x55f898711810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.550+0000 7f6ac619e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8983474d0 0x55f898711d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.550+0000 7f6ac619e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f89834f6b0 0x55f89871e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.550+0000 7f6ac619e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f898676d80 con 0x55f8983474d0 2024-04-25T10:33:43.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.550+0000 7f6ac619e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f898630a20 con 0x55f898370390 2024-04-25T10:33:43.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.550+0000 7f6ac619e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f898672d40 con 0x55f89834f6b0 2024-04-25T10:33:43.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.551+0000 7f6abc889640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f898370390 0x55f898711810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.551+0000 7f6abd08a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8983474d0 0x55f898711d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.551+0000 7f6abc889640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f898370390 0x55f898711810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:53706/0 (socket says 172.21.15.43:53706) 2024-04-25T10:33:43.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.551+0000 7f6abc889640 1 -- 172.21.15.43:0/1415706794 learned_addr learned my addr 172.21.15.43:0/1415706794 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:43.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.551+0000 7f6abc889640 1 --2- 172.21.15.43:0/1415706794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f89834f6b0 0x55f89871e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.551+0000 7f6aae7fc640 1 --2- 172.21.15.43:0/3164569614 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6aa40afe90 0x7f6aa40b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.551+0000 7f6abc889640 1 -- 172.21.15.43:0/1415706794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f89834f6b0 msgr2=0x55f89871e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.551+0000 7f6abc889640 1 --2- 172.21.15.43:0/1415706794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f89834f6b0 0x55f89871e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.551+0000 7f6abc889640 1 -- 172.21.15.43:0/1415706794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8983474d0 msgr2=0x55f898711d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.551+0000 7f6aae7fc640 1 -- 172.21.15.43:0/3164569614 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f6ab01007f0 con 0x7f6ab81a6130 2024-04-25T10:33:43.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.551+0000 7f6abd88b640 1 --2- 172.21.15.43:0/3164569614 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6aa40afe90 0x7f6aa40b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.551+0000 7f6abc889640 1 --2- 172.21.15.43:0/1415706794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8983474d0 0x55f898711d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.551+0000 7f6abc889640 1 -- 172.21.15.43:0/1415706794 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f89863bc30 con 0x55f898370390 2024-04-25T10:33:43.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.551+0000 7f6abd08a640 1 --2- 172.21.15.43:0/1415706794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8983474d0 0x55f898711d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:43.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.552+0000 7f6abc889640 1 --2- 172.21.15.43:0/1415706794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f898370390 0x55f898711810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6aa80126f0 tx=0x7f6aa800b6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad62ad7c9d4f9a87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:43.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.552+0000 7f6aa37fe640 1 -- 172.21.15.43:0/1415706794 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8012c70 con 0x55f898370390 2024-04-25T10:33:43.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.552+0000 7f6aa37fe640 1 -- 172.21.15.43:0/1415706794 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aa8012e10 con 0x55f898370390 2024-04-25T10:33:43.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.552+0000 7f6aa37fe640 1 -- 172.21.15.43:0/1415706794 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa80049a0 con 0x55f898370390 2024-04-25T10:33:43.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.553+0000 7f6ac619e740 1 -- 172.21.15.43:0/1415706794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f898370390 msgr2=0x55f898711810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.553+0000 7f6ac619e740 1 --2- 172.21.15.43:0/1415706794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f898370390 0x55f898711810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6aa80126f0 tx=0x7f6aa800b6e0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.553+0000 7f6abd88b640 1 --2- 172.21.15.43:0/3164569614 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6aa40afe90 0x7f6aa40b23a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6ab00fdea0 tx=0x7f6ab0100a70 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:43.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.553+0000 7f6ac619e740 1 -- 172.21.15.43:0/1415706794 shutdown_connections 2024-04-25T10:33:43.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.553+0000 7f6ac619e740 1 --2- 172.21.15.43:0/1415706794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f89834f6b0 0x55f89871e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.553+0000 7f6ac619e740 1 --2- 172.21.15.43:0/1415706794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8983474d0 0x55f898711d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.553+0000 7f6ac619e740 1 --2- 172.21.15.43:0/1415706794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f898370390 0x55f898711810 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.553+0000 7f6ac619e740 1 -- 172.21.15.43:0/1415706794 >> 172.21.15.43:0/1415706794 conn(0x55f8983644e0 msgr2=0x55f89870dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:43.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.553+0000 7f6ac619e740 1 -- 172.21.15.43:0/1415706794 shutdown_connections 2024-04-25T10:33:43.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.553+0000 7f6ac619e740 1 -- 172.21.15.43:0/1415706794 wait complete. 2024-04-25T10:33:43.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.553+0000 7f6ac619e740 1 -- start start 2024-04-25T10:33:43.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.553+0000 7f6ac619e740 1 Processor -- start 2024-04-25T10:33:43.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.554+0000 7f6ac619e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8983474d0 0x55f89871c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.554+0000 7f6ac619e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f89834f6b0 0x55f89871c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.554+0000 7f6ac619e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f898370390 0x55f898728210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.554+0000 7f6ac619e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f89871ed90 con 0x55f898370390 2024-04-25T10:33:43.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.554+0000 7f6ac619e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f8985fddc0 con 0x55f89834f6b0 2024-04-25T10:33:43.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.554+0000 7f6ac619e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f89871ef10 con 0x55f8983474d0 2024-04-25T10:33:43.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.554+0000 7f6ac619e740 10 client.0 _ll_register_callbacks cb 0x55f898619600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:43.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.554+0000 7f6abc889640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f898370390 0x55f898728210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.554+0000 7f6abd08a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f89834f6b0 0x55f89871c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.554+0000 7f6abc889640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f898370390 0x55f898728210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43028/0 (socket says 172.21.15.43:43028) 2024-04-25T10:33:43.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.554+0000 7f6abc889640 1 -- 172.21.15.43:0/4037242858 learned_addr learned my addr 172.21.15.43:0/4037242858 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:43.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.554+0000 7f6abc889640 1 --2- 172.21.15.43:0/4037242858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8983474d0 0x55f89871c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.555+0000 7f6abc889640 1 -- 172.21.15.43:0/4037242858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8983474d0 msgr2=0x55f89871c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.555+0000 7f6abc889640 1 --2- 172.21.15.43:0/4037242858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8983474d0 0x55f89871c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.555+0000 7f6abc889640 1 -- 172.21.15.43:0/4037242858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f89834f6b0 msgr2=0x55f89871c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.555+0000 7f6abc889640 1 --2- 172.21.15.43:0/4037242858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f89834f6b0 0x55f89871c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.555+0000 7f6abc889640 1 -- 172.21.15.43:0/4037242858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8983883f0 con 0x55f898370390 2024-04-25T10:33:43.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.555+0000 7f6abc889640 1 --2- 172.21.15.43:0/4037242858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f898370390 0x55f898728210 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x55f89871b940 tx=0x7f6aa8017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75461b334630f7ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:43.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.555+0000 7f6aa0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:43.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.555+0000 7f6aa0ff9640 1 -- 172.21.15.43:0/4037242858 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8010770 con 0x55f898370390 2024-04-25T10:33:43.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.555+0000 7f6aa0ff9640 1 -- 172.21.15.43:0/4037242858 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f6aa8010910 con 0x55f898370390 2024-04-25T10:33:43.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.555+0000 7f6ac619e740 1 -- 172.21.15.43:0/4037242858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aa8012e10 con 0x55f898370390 2024-04-25T10:33:43.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.555+0000 7f6aa0ff9640 1 -- 172.21.15.43:0/4037242858 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6aa8031050 con 0x55f898370390 2024-04-25T10:33:43.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.555+0000 7f6ac619e740 10 client.15786 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:43.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.555+0000 7f6ac619e740 1 -- 172.21.15.43:0/4037242858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f8984527d0 con 0x55f898370390 2024-04-25T10:33:43.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.556+0000 7f6aa0ff9640 1 -- 172.21.15.43:0/4037242858 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aa8012e10 con 0x55f898370390 2024-04-25T10:33:43.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.556+0000 7f6aa0ff9640 1 -- 172.21.15.43:0/4037242858 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6aa8004ca0 con 0x55f898370390 2024-04-25T10:33:43.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.556+0000 7f6ac619e740 10 client.15786 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:43.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.556+0000 7f6ac619e740 10 client.15786 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:33:43.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.556+0000 7f6ac619e740 20 client.15786 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:43.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.556+0000 7f6ac619e740 1 --2- 172.21.15.43:0/4037242858 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55f89832ef60 0x55f89872e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.556+0000 7f6ac619e740 4 client.15786 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:33:43.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.556+0000 7f6ac619e740 1 -- 172.21.15.43:0/4037242858 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f898684990 con 0x55f89832ef60 2024-04-25T10:33:43.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.556+0000 7f6abd08a640 1 --2- 172.21.15.43:0/4037242858 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55f89832ef60 0x55f89872e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.557+0000 7f6abd08a640 1 --2- 172.21.15.43:0/4037242858 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55f89832ef60 0x55f89872e620 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=183bd44d9ef0681 server_cookie=23ea8257627cda49 in_seq=0 out_seq=0 2024-04-25T10:33:43.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.557+0000 7f6aa0ff9640 10 client.15786 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:33:43.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.558+0000 7f6aa0ff9640 1 -- 172.21.15.43:0/4037242858 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f898684990 con 0x55f89832ef60 2024-04-25T10:33:43.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.558+0000 7f6aa0ff9640 10 client.15786 handle_command_reply: tid=0 2024-04-25T10:33:43.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.558+0000 7f6ac619e740 1 client.15786 shutdown 2024-04-25T10:33:43.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.558+0000 7f6ac619e740 1 -- 172.21.15.43:0/4037242858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f898370390 msgr2=0x55f898728210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.558+0000 7f6ac619e740 1 --2- 172.21.15.43:0/4037242858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f898370390 0x55f898728210 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x55f89871b940 tx=0x7f6aa8017040 comp rx=0 tx=0).stop 2024-04-25T10:33:43.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.558+0000 7f6ac619e740 1 -- 172.21.15.43:0/4037242858 shutdown_connections 2024-04-25T10:33:43.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.558+0000 7f6ac619e740 1 --2- 172.21.15.43:0/4037242858 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55f89832ef60 0x55f89872e620 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.558+0000 7f6ac619e740 1 --2- 172.21.15.43:0/4037242858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f89834f6b0 0x55f89871c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.558+0000 7f6ac619e740 1 --2- 172.21.15.43:0/4037242858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f898370390 0x55f898728210 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.558+0000 7f6ac619e740 1 --2- 172.21.15.43:0/4037242858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8983474d0 0x55f89871c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.558+0000 7f6aa0ff9640 0 client.15786 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:33:43.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.558+0000 7f6ac619e740 1 -- 172.21.15.43:0/4037242858 >> 172.21.15.43:0/4037242858 conn(0x55f8983644e0 msgr2=0x55f898713530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:43.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.558+0000 7f6ac619e740 1 -- 172.21.15.43:0/4037242858 shutdown_connections 2024-04-25T10:33:43.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.558+0000 7f6ac619e740 1 -- 172.21.15.43:0/4037242858 wait complete. 2024-04-25T10:33:43.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.558+0000 7f6ac619e740 20 client.15786 trim_cache size 0 max 16384 2024-04-25T10:33:43.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.572+0000 7f6ac619e740 1 Processor -- start 2024-04-25T10:33:43.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.572+0000 7f6ac619e740 1 -- start start 2024-04-25T10:33:43.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.572+0000 7f6ac619e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8983474d0 0x55f89871b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.572+0000 7f6ac619e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f89834f6b0 0x55f898727e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.572+0000 7f6ac619e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f898370390 0x55f89870cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.573+0000 7f6ac619e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab81b4440 con 0x55f89834f6b0 2024-04-25T10:33:43.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.573+0000 7f6ac619e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab81b1d10 con 0x55f8983474d0 2024-04-25T10:33:43.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.573+0000 7f6ac619e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab81afb60 con 0x55f898370390 2024-04-25T10:33:43.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.573+0000 7f6abd08a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f89834f6b0 0x55f898727e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.573+0000 7f6abd08a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f89834f6b0 0x55f898727e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43030/0 (socket says 172.21.15.43:43030) 2024-04-25T10:33:43.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.573+0000 7f6abd08a640 1 -- 172.21.15.43:0/3638685113 learned_addr learned my addr 172.21.15.43:0/3638685113 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:43.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.573+0000 7f6abc889640 1 --2- 172.21.15.43:0/3638685113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8983474d0 0x55f89871b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.573+0000 7f6abc889640 1 --2- 172.21.15.43:0/3638685113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f898370390 0x55f89870cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.573+0000 7f6abd08a640 1 -- 172.21.15.43:0/3638685113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f898370390 msgr2=0x55f89870cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.573+0000 7f6abd08a640 1 --2- 172.21.15.43:0/3638685113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f898370390 0x55f89870cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.573+0000 7f6abd08a640 1 -- 172.21.15.43:0/3638685113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8983474d0 msgr2=0x55f89871b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.573+0000 7f6abd08a640 1 --2- 172.21.15.43:0/3638685113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8983474d0 0x55f89871b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.573+0000 7f6abd08a640 1 -- 172.21.15.43:0/3638685113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f89871ec70 con 0x55f89834f6b0 2024-04-25T10:33:43.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.573+0000 7f6abc889640 1 --2- 172.21.15.43:0/3638685113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f898370390 0x55f89870cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:43.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.573+0000 7f6abd08a640 1 --2- 172.21.15.43:0/3638685113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f89834f6b0 0x55f898727e70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x55f89870ef50 tx=0x7f6ab4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0b8611ecf05392 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:43.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.573+0000 7f6aa2ffd640 1 -- 172.21.15.43:0/3638685113 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4003de0 con 0x55f89834f6b0 2024-04-25T10:33:43.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.574+0000 7f6aa2ffd640 1 -- 172.21.15.43:0/3638685113 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ab40108e0 con 0x55f89834f6b0 2024-04-25T10:33:43.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.574+0000 7f6aa2ffd640 1 -- 172.21.15.43:0/3638685113 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4020570 con 0x55f89834f6b0 2024-04-25T10:33:43.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.574+0000 7f6ac619e740 1 -- 172.21.15.43:0/3638685113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f89834f6b0 msgr2=0x55f898727e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.574+0000 7f6ac619e740 1 --2- 172.21.15.43:0/3638685113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f89834f6b0 0x55f898727e70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x55f89870ef50 tx=0x7f6ab4005f90 comp rx=0 tx=0).stop 2024-04-25T10:33:43.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.574+0000 7f6ac619e740 1 -- 172.21.15.43:0/3638685113 shutdown_connections 2024-04-25T10:33:43.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.574+0000 7f6ac619e740 1 --2- 172.21.15.43:0/3638685113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f898370390 0x55f89870cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.574+0000 7f6ac619e740 1 --2- 172.21.15.43:0/3638685113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f89834f6b0 0x55f898727e70 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.574+0000 7f6ac619e740 1 --2- 172.21.15.43:0/3638685113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8983474d0 0x55f89871b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.574+0000 7f6ac619e740 1 -- 172.21.15.43:0/3638685113 >> 172.21.15.43:0/3638685113 conn(0x55f8983644e0 msgr2=0x55f89870c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:43.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.575+0000 7f6ac619e740 1 -- 172.21.15.43:0/3638685113 shutdown_connections 2024-04-25T10:33:43.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.575+0000 7f6ac619e740 1 -- 172.21.15.43:0/3638685113 wait complete. 2024-04-25T10:33:43.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.575+0000 7f6ac619e740 1 -- start start 2024-04-25T10:33:43.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.575+0000 7f6ac619e740 1 Processor -- start 2024-04-25T10:33:43.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.575+0000 7f6ac619e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8983474d0 0x55f898724820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.575+0000 7f6ac619e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f89834f6b0 0x55f89871f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.575+0000 7f6ac619e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f898370390 0x55f89871f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.576+0000 7f6ac619e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab81b01d0 con 0x55f898370390 2024-04-25T10:33:43.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.576+0000 7f6ac619e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab81b1640 con 0x55f89834f6b0 2024-04-25T10:33:43.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.576+0000 7f6ac619e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab81b08a0 con 0x55f8983474d0 2024-04-25T10:33:43.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.576+0000 7f6ac619e740 10 client.0 _ll_register_callbacks cb 0x7f6ab40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:43.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.576+0000 7f6abc889640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f898370390 0x55f89871f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.576+0000 7f6abc889640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f898370390 0x55f89871f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43040/0 (socket says 172.21.15.43:43040) 2024-04-25T10:33:43.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.576+0000 7f6abc889640 1 -- 172.21.15.43:0/319885534 learned_addr learned my addr 172.21.15.43:0/319885534 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:43.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.576+0000 7f6abd08a640 1 --2- 172.21.15.43:0/319885534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f89834f6b0 0x55f89871f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.576+0000 7f6abc889640 1 --2- 172.21.15.43:0/319885534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8983474d0 0x55f898724820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.576+0000 7f6abc889640 1 -- 172.21.15.43:0/319885534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8983474d0 msgr2=0x55f898724820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.576+0000 7f6abc889640 1 --2- 172.21.15.43:0/319885534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8983474d0 0x55f898724820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.576+0000 7f6abc889640 1 -- 172.21.15.43:0/319885534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f89834f6b0 msgr2=0x55f89871f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.576+0000 7f6abc889640 1 --2- 172.21.15.43:0/319885534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f89834f6b0 0x55f89871f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.576+0000 7f6abc889640 1 -- 172.21.15.43:0/319885534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f89871fee0 con 0x55f898370390 2024-04-25T10:33:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.576+0000 7f6abc889640 1 --2- 172.21.15.43:0/319885534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f898370390 0x55f89871f750 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x55f89871b940 tx=0x7f6aa8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732cc6a6b49a9fa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.576+0000 7f6aa0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.576+0000 7f6aa0ff9640 1 -- 172.21.15.43:0/319885534 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa800bdd0 con 0x55f898370390 2024-04-25T10:33:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.577+0000 7f6aa0ff9640 1 -- 172.21.15.43:0/319885534 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f6aa8012c70 con 0x55f898370390 2024-04-25T10:33:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.577+0000 7f6ac619e740 1 -- 172.21.15.43:0/319885534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ab40108e0 con 0x55f898370390 2024-04-25T10:33:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.577+0000 7f6aa0ff9640 1 -- 172.21.15.43:0/319885534 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6aa801fa00 con 0x55f898370390 2024-04-25T10:33:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.577+0000 7f6ac619e740 10 client.15792 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.577+0000 7f6ac619e740 1 -- 172.21.15.43:0/319885534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f898795680 con 0x55f898370390 2024-04-25T10:33:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.577+0000 7f6aa0ff9640 1 -- 172.21.15.43:0/319885534 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ab40108e0 con 0x55f898370390 2024-04-25T10:33:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.577+0000 7f6aa0ff9640 1 -- 172.21.15.43:0/319885534 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6aa801fc30 con 0x55f898370390 2024-04-25T10:33:43.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.577+0000 7f6ac619e740 10 client.15792 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:43.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.577+0000 7f6ac619e740 10 client.15792 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:33:43.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.577+0000 7f6ac619e740 20 client.15792 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:43.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.577+0000 7f6ac619e740 1 --2- 172.21.15.43:0/319885534 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55f8983043a0 0x55f898797ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.577+0000 7f6ac619e740 4 client.15792 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:33:43.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.577+0000 7f6ac619e740 1 -- 172.21.15.43:0/319885534 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f89867e800 con 0x55f8983043a0 2024-04-25T10:33:43.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.578+0000 7f6abd08a640 1 --2- 172.21.15.43:0/319885534 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55f8983043a0 0x55f898797ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.578+0000 7f6abd08a640 1 --2- 172.21.15.43:0/319885534 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55f8983043a0 0x55f898797ea0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=594f69d3f41d757d server_cookie=c9670819f0f10498 in_seq=0 out_seq=0 2024-04-25T10:33:43.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.578+0000 7f6aa0ff9640 10 client.15792 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:33:43.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.579+0000 7f6aa0ff9640 1 -- 172.21.15.43:0/319885534 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f89867e800 con 0x55f8983043a0 2024-04-25T10:33:43.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.579+0000 7f6aa0ff9640 10 client.15792 handle_command_reply: tid=0 2024-04-25T10:33:43.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.579+0000 7f6ac619e740 1 client.15792 shutdown 2024-04-25T10:33:43.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.579+0000 7f6ac619e740 1 -- 172.21.15.43:0/319885534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f898370390 msgr2=0x55f89871f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.579+0000 7f6ac619e740 1 --2- 172.21.15.43:0/319885534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f898370390 0x55f89871f750 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x55f89871b940 tx=0x7f6aa8000bc0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.579+0000 7f6ac619e740 1 -- 172.21.15.43:0/319885534 shutdown_connections 2024-04-25T10:33:43.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.579+0000 7f6ac619e740 1 --2- 172.21.15.43:0/319885534 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55f8983043a0 0x55f898797ea0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.579+0000 7f6ac619e740 1 --2- 172.21.15.43:0/319885534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f89834f6b0 0x55f89871f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.579+0000 7f6ac619e740 1 --2- 172.21.15.43:0/319885534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f898370390 0x55f89871f750 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.579+0000 7f6ac619e740 1 --2- 172.21.15.43:0/319885534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8983474d0 0x55f898724820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.579+0000 7f6ac619e740 1 -- 172.21.15.43:0/319885534 >> 172.21.15.43:0/319885534 conn(0x55f8983644e0 msgr2=0x55f89871b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:43.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.579+0000 7f6aa0ff9640 0 client.15792 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:33:43.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.579+0000 7f6ac619e740 1 -- 172.21.15.43:0/319885534 shutdown_connections 2024-04-25T10:33:43.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.579+0000 7f6ac619e740 1 -- 172.21.15.43:0/319885534 wait complete. 2024-04-25T10:33:43.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.579+0000 7f6ac619e740 20 client.15792 trim_cache size 0 max 16384 2024-04-25T10:33:43.586 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:33:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.580+0000 7f6abf2ee640 1 -- 172.21.15.43:0/3164569614 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6aa40afe90 msgr2=0x7f6aa40b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.580+0000 7f6abf2ee640 1 --2- 172.21.15.43:0/3164569614 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6aa40afe90 0x7f6aa40b23a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6ab00fdea0 tx=0x7f6ab0100a70 comp rx=0 tx=0).stop 2024-04-25T10:33:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.580+0000 7f6abf2ee640 1 -- 172.21.15.43:0/3164569614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab81a6130 msgr2=0x7f6ab81a8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.580+0000 7f6abf2ee640 1 --2- 172.21.15.43:0/3164569614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab81a6130 0x7f6ab81a8550 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6ab00098d0 tx=0x7f6ab00317b0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.580+0000 7f6abf2ee640 1 -- 172.21.15.43:0/3164569614 shutdown_connections 2024-04-25T10:33:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.580+0000 7f6abf2ee640 1 --2- 172.21.15.43:0/3164569614 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6aa40afe90 0x7f6aa40b23a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.580+0000 7f6abf2ee640 1 --2- 172.21.15.43:0/3164569614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab81a6130 0x7f6ab81a8550 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.580+0000 7f6abf2ee640 1 --2- 172.21.15.43:0/3164569614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ab81032f0 0x7f6ab8103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.580+0000 7f6abf2ee640 1 --2- 172.21.15.43:0/3164569614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ab8109b20 0x7f6ab8102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.580+0000 7f6abf2ee640 1 -- 172.21.15.43:0/3164569614 >> 172.21.15.43:0/3164569614 conn(0x7f6ab8077880 msgr2=0x7f6ab81071f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.580+0000 7f6abf2ee640 1 -- 172.21.15.43:0/3164569614 shutdown_connections 2024-04-25T10:33:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.580+0000 7f6abf2ee640 1 -- 172.21.15.43:0/3164569614 wait complete. 2024-04-25T10:33:43.593 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:33:43.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.707+0000 7fe6fb3c0640 1 -- 172.21.15.43:0/3747097082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f41042d0 msgr2=0x7fe6f4110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.707+0000 7fe6fb3c0640 1 --2- 172.21.15.43:0/3747097082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f41042d0 0x7fe6f4110650 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009ca0 tx=0x7fe6ec02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.707+0000 7fe6fb3c0640 1 -- 172.21.15.43:0/3747097082 shutdown_connections 2024-04-25T10:33:43.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.707+0000 7fe6fb3c0640 1 --2- 172.21.15.43:0/3747097082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f41042d0 0x7fe6f4110650 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.707+0000 7fe6fb3c0640 1 --2- 172.21.15.43:0/3747097082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6f4109fd0 0x7fe6f4103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.707+0000 7fe6fb3c0640 1 --2- 172.21.15.43:0/3747097082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6f4109b30 0x7fe6f4103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.707+0000 7fe6fb3c0640 1 -- 172.21.15.43:0/3747097082 >> 172.21.15.43:0/3747097082 conn(0x7fe6f4079280 msgr2=0x7fe6f40796c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:43.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.707+0000 7fe6fb3c0640 1 -- 172.21.15.43:0/3747097082 shutdown_connections 2024-04-25T10:33:43.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.707+0000 7fe6fb3c0640 1 -- 172.21.15.43:0/3747097082 wait complete. 2024-04-25T10:33:43.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.708+0000 7fe6fb3c0640 1 Processor -- start 2024-04-25T10:33:43.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.708+0000 7fe6fb3c0640 1 -- start start 2024-04-25T10:33:43.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.708+0000 7fe6fb3c0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6f41042d0 0x7fe6f419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.708+0000 7fe6fb3c0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6f4109b30 0x7fe6f419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.708+0000 7fe6fb3c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f4109fd0 0x7fe6f419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.708+0000 7fe6fb3c0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f4113d40 con 0x7fe6f4109fd0 2024-04-25T10:33:43.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.708+0000 7fe6fb3c0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f4113bc0 con 0x7fe6f4109b30 2024-04-25T10:33:43.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.708+0000 7fe6fb3c0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f4113ec0 con 0x7fe6f41042d0 2024-04-25T10:33:43.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.709+0000 7fe6f995d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f4109fd0 0x7fe6f419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.709+0000 7fe6f915c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6f41042d0 0x7fe6f419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.709+0000 7fe6f995d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f4109fd0 0x7fe6f419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43052/0 (socket says 172.21.15.43:43052) 2024-04-25T10:33:43.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.709+0000 7fe6f995d640 1 -- 172.21.15.43:0/3594593153 learned_addr learned my addr 172.21.15.43:0/3594593153 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:43.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.709+0000 7fe6f895b640 1 --2- 172.21.15.43:0/3594593153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6f4109b30 0x7fe6f419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.709+0000 7fe6f915c640 1 -- 172.21.15.43:0/3594593153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6f4109b30 msgr2=0x7fe6f419ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.709+0000 7fe6f915c640 1 --2- 172.21.15.43:0/3594593153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6f4109b30 0x7fe6f419ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.709+0000 7fe6f915c640 1 -- 172.21.15.43:0/3594593153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f4109fd0 msgr2=0x7fe6f419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.709+0000 7fe6f915c640 1 --2- 172.21.15.43:0/3594593153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f4109fd0 0x7fe6f419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.709+0000 7fe6f915c640 1 -- 172.21.15.43:0/3594593153 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f419f560 con 0x7fe6f41042d0 2024-04-25T10:33:43.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.709+0000 7fe6f995d640 1 --2- 172.21.15.43:0/3594593153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f4109fd0 0x7fe6f419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:43.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.709+0000 7fe6f915c640 1 --2- 172.21.15.43:0/3594593153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6f41042d0 0x7fe6f419a700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000e9c0 tx=0x7fe6f000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:43.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.710+0000 7fe6ea7fc640 1 -- 172.21.15.43:0/3594593153 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f000ccc0 con 0x7fe6f41042d0 2024-04-25T10:33:43.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.710+0000 7fe6fb3c0640 1 -- 172.21.15.43:0/3594593153 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f419f850 con 0x7fe6f41042d0 2024-04-25T10:33:43.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.710+0000 7fe6ea7fc640 1 -- 172.21.15.43:0/3594593153 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6f00108e0 con 0x7fe6f41042d0 2024-04-25T10:33:43.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.710+0000 7fe6ea7fc640 1 -- 172.21.15.43:0/3594593153 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f00187a0 con 0x7fe6f41042d0 2024-04-25T10:33:43.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.710+0000 7fe6fb3c0640 1 -- 172.21.15.43:0/3594593153 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f41a71d0 con 0x7fe6f41042d0 2024-04-25T10:33:43.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.713+0000 7fe702270740 1 Processor -- start 2024-04-25T10:33:43.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.713+0000 7fe702270740 1 -- start start 2024-04-25T10:33:43.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.713+0000 7fe6ea7fc640 1 -- 172.21.15.43:0/3594593153 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 147804+0+0 (secure 0 0 0) 0x7fe6f0010430 con 0x7fe6f41042d0 2024-04-25T10:33:43.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.713+0000 7fe702270740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe7577390 0x55dfe7918810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.713+0000 7fe702270740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 0x55dfe7918d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.713+0000 7fe702270740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe75566b0 0x55dfe79256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.713+0000 7fe702270740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dfe787dd80 con 0x55dfe754e4d0 2024-04-25T10:33:43.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.713+0000 7fe702270740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dfe7837a20 con 0x55dfe7577390 2024-04-25T10:33:43.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.714+0000 7fe702270740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55dfe7879d40 con 0x55dfe75566b0 2024-04-25T10:33:43.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.714+0000 7fe6f915c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 0x55dfe7918d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.714+0000 7fe6f915c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 0x55dfe7918d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43058/0 (socket says 172.21.15.43:43058) 2024-04-25T10:33:43.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.714+0000 7fe6f915c640 1 -- 172.21.15.43:0/3038993218 learned_addr learned my addr 172.21.15.43:0/3038993218 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:43.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.714+0000 7fe6f895b640 1 --2- 172.21.15.43:0/3038993218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe75566b0 0x55dfe79256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.714+0000 7fe6f995d640 1 --2- 172.21.15.43:0/3038993218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe7577390 0x55dfe7918810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.714+0000 7fe6f895b640 1 -- 172.21.15.43:0/3038993218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe7577390 msgr2=0x55dfe7918810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.714+0000 7fe6f895b640 1 --2- 172.21.15.43:0/3038993218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe7577390 0x55dfe7918810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.714+0000 7fe6f895b640 1 -- 172.21.15.43:0/3038993218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 msgr2=0x55dfe7918d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.714+0000 7fe6f895b640 1 --2- 172.21.15.43:0/3038993218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 0x55dfe7918d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.714+0000 7fe6f895b640 1 -- 172.21.15.43:0/3038993218 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfe7842c30 con 0x55dfe75566b0 2024-04-25T10:33:43.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.714+0000 7fe6f915c640 1 --2- 172.21.15.43:0/3038993218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 0x55dfe7918d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:43.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.714+0000 7fe6ea7fc640 1 --2- 172.21.15.43:0/3594593153 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe6e00afe10 0x7fe6e00b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.715+0000 7fe6f995d640 1 --2- 172.21.15.43:0/3594593153 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe6e00afe10 0x7fe6e00b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.715+0000 7fe6f895b640 1 --2- 172.21.15.43:0/3038993218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe75566b0 0x55dfe79256a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4005fd0 tx=0x7fe6e40107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27c35360f2216920 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:43.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.715+0000 7fe6ea7fc640 1 -- 172.21.15.43:0/3594593153 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fe6f00df610 con 0x7fe6f41042d0 2024-04-25T10:33:43.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.715+0000 7fe6df7fe640 1 -- 172.21.15.43:0/3038993218 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4005020 con 0x55dfe75566b0 2024-04-25T10:33:43.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.715+0000 7fe6df7fe640 1 -- 172.21.15.43:0/3038993218 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6e40047a0 con 0x55dfe75566b0 2024-04-25T10:33:43.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.715+0000 7fe6df7fe640 1 -- 172.21.15.43:0/3038993218 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4011660 con 0x55dfe75566b0 2024-04-25T10:33:43.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.716+0000 7fe702270740 1 -- 172.21.15.43:0/3038993218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe75566b0 msgr2=0x55dfe79256a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.716+0000 7fe702270740 1 --2- 172.21.15.43:0/3038993218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe75566b0 0x55dfe79256a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4005fd0 tx=0x7fe6e40107a0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.716+0000 7fe6f995d640 1 --2- 172.21.15.43:0/3594593153 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe6e00afe10 0x7fe6e00b2320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55dfe7915980 tx=0x7fe6ec005c50 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:43.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.716+0000 7fe702270740 1 -- 172.21.15.43:0/3038993218 shutdown_connections 2024-04-25T10:33:43.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.716+0000 7fe702270740 1 --2- 172.21.15.43:0/3038993218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe75566b0 0x55dfe79256a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.716+0000 7fe702270740 1 --2- 172.21.15.43:0/3038993218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 0x55dfe7918d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.716+0000 7fe702270740 1 --2- 172.21.15.43:0/3038993218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe7577390 0x55dfe7918810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.716+0000 7fe702270740 1 -- 172.21.15.43:0/3038993218 >> 172.21.15.43:0/3038993218 conn(0x55dfe756b4e0 msgr2=0x55dfe7914fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:43.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.716+0000 7fe702270740 1 -- 172.21.15.43:0/3038993218 shutdown_connections 2024-04-25T10:33:43.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.716+0000 7fe702270740 1 -- 172.21.15.43:0/3038993218 wait complete. 2024-04-25T10:33:43.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.716+0000 7fe702270740 1 -- start start 2024-04-25T10:33:43.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.716+0000 7fe702270740 1 Processor -- start 2024-04-25T10:33:43.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.717+0000 7fe702270740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe754e4d0 0x55dfe7923330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.717+0000 7fe702270740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe75566b0 0x55dfe7923870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.717+0000 7fe702270740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe7577390 0x55dfe792f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.717+0000 7fe702270740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dfe7925d90 con 0x55dfe7577390 2024-04-25T10:33:43.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.717+0000 7fe702270740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dfe7804dc0 con 0x55dfe754e4d0 2024-04-25T10:33:43.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.717+0000 7fe702270740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55dfe7925f10 con 0x55dfe75566b0 2024-04-25T10:33:43.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.717+0000 7fe702270740 10 client.0 _ll_register_callbacks cb 0x55dfe7820600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:43.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.717+0000 7fe6f995d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe7577390 0x55dfe792f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.717+0000 7fe6f895b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe75566b0 0x55dfe7923870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.717+0000 7fe6f995d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe7577390 0x55dfe792f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43064/0 (socket says 172.21.15.43:43064) 2024-04-25T10:33:43.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.717+0000 7fe6f995d640 1 -- 172.21.15.43:0/695252787 learned_addr learned my addr 172.21.15.43:0/695252787 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:43.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.717+0000 7fe6f895b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe75566b0 0x55dfe7923870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:57396/0 (socket says 172.21.15.43:57396) 2024-04-25T10:33:43.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.717+0000 7fe6f915c640 1 --2- 172.21.15.43:0/695252787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe754e4d0 0x55dfe7923330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.718+0000 7fe6f915c640 1 -- 172.21.15.43:0/695252787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe75566b0 msgr2=0x55dfe7923870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.718+0000 7fe6f915c640 1 --2- 172.21.15.43:0/695252787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe75566b0 0x55dfe7923870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.718+0000 7fe6f915c640 1 -- 172.21.15.43:0/695252787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe7577390 msgr2=0x55dfe792f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.718+0000 7fe6f915c640 1 --2- 172.21.15.43:0/695252787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe7577390 0x55dfe792f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.718+0000 7fe6f915c640 1 -- 172.21.15.43:0/695252787 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfe758f3f0 con 0x55dfe754e4d0 2024-04-25T10:33:43.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.718+0000 7fe6f915c640 1 --2- 172.21.15.43:0/695252787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe754e4d0 0x55dfe7923330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55dfe7915f50 tx=0x7fe6f000ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff998265f01dbbcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:43.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.718+0000 7fe6dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-04-25T10:33:43.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.718+0000 7fe6dcff9640 1 -- 172.21.15.43:0/695252787 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0010720 con 0x55dfe754e4d0 2024-04-25T10:33:43.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.718+0000 7fe6dcff9640 1 -- 172.21.15.43:0/695252787 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fe6f00900f0 con 0x55dfe754e4d0 2024-04-25T10:33:43.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.718+0000 7fe702270740 1 -- 172.21.15.43:0/695252787 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6e40047a0 con 0x55dfe754e4d0 2024-04-25T10:33:43.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.718+0000 7fe6dcff9640 1 -- 172.21.15.43:0/695252787 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe6f00a7160 con 0x55dfe754e4d0 2024-04-25T10:33:43.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.718+0000 7fe702270740 10 client.25523 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:43.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.718+0000 7fe702270740 1 -- 172.21.15.43:0/695252787 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dfe76597d0 con 0x55dfe754e4d0 2024-04-25T10:33:43.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.718+0000 7fe6dcff9640 1 -- 172.21.15.43:0/695252787 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6f00a7650 con 0x55dfe754e4d0 2024-04-25T10:33:43.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.719+0000 7fe6dcff9640 1 -- 172.21.15.43:0/695252787 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe6f00df250 con 0x55dfe754e4d0 2024-04-25T10:33:43.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.719+0000 7fe702270740 10 client.25523 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:43.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.719+0000 7fe702270740 10 client.25523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:43.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.719+0000 7fe702270740 20 client.25523 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:43.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.719+0000 7fe702270740 1 --2- 172.21.15.43:0/695252787 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55dfe7535f60 0x55dfe7939be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.719+0000 7fe702270740 4 client.25523 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:33:43.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.719+0000 7fe702270740 1 -- 172.21.15.43:0/695252787 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe6e40047a0 con 0x55dfe7535f60 2024-04-25T10:33:43.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.719+0000 7fe6f895b640 1 --2- 172.21.15.43:0/695252787 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55dfe7535f60 0x55dfe7939be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.719+0000 7fe6dcff9640 10 client.25523 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:43.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.720+0000 7fe6f895b640 1 --2- 172.21.15.43:0/695252787 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55dfe7535f60 0x55dfe7939be0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a9b73e73d41618d server_cookie=120b81c89abd719e in_seq=0 out_seq=0 2024-04-25T10:33:43.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.720+0000 7fe6dcff9640 1 -- 172.21.15.43:0/695252787 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fe6e40047a0 con 0x55dfe7535f60 2024-04-25T10:33:43.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.720+0000 7fe6dcff9640 10 client.25523 handle_command_reply: tid=0 2024-04-25T10:33:43.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.720+0000 7fe702270740 1 client.25523 shutdown 2024-04-25T10:33:43.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.720+0000 7fe702270740 1 -- 172.21.15.43:0/695252787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe754e4d0 msgr2=0x55dfe7923330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.720+0000 7fe702270740 1 --2- 172.21.15.43:0/695252787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe754e4d0 0x55dfe7923330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55dfe7915f50 tx=0x7fe6f000ce60 comp rx=0 tx=0).stop 2024-04-25T10:33:43.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.720+0000 7fe702270740 1 -- 172.21.15.43:0/695252787 shutdown_connections 2024-04-25T10:33:43.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.720+0000 7fe702270740 1 --2- 172.21.15.43:0/695252787 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55dfe7535f60 0x55dfe7939be0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.721+0000 7fe702270740 1 --2- 172.21.15.43:0/695252787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe7577390 0x55dfe792f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.721+0000 7fe6dcff9640 0 client.25523 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:43.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.721+0000 7fe702270740 1 --2- 172.21.15.43:0/695252787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe75566b0 0x55dfe7923870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.721+0000 7fe702270740 1 --2- 172.21.15.43:0/695252787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe754e4d0 0x55dfe7923330 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.721+0000 7fe702270740 1 -- 172.21.15.43:0/695252787 >> 172.21.15.43:0/695252787 conn(0x55dfe756b4e0 msgr2=0x55dfe791a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:43.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.721+0000 7fe702270740 1 -- 172.21.15.43:0/695252787 shutdown_connections 2024-04-25T10:33:43.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.721+0000 7fe702270740 1 -- 172.21.15.43:0/695252787 wait complete. 2024-04-25T10:33:43.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.721+0000 7fe702270740 20 client.25523 trim_cache size 0 max 16384 2024-04-25T10:33:43.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.735+0000 7fe702270740 1 Processor -- start 2024-04-25T10:33:43.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.735+0000 7fe702270740 1 -- start start 2024-04-25T10:33:43.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.735+0000 7fe702270740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe7535f60 0x55dfe7922420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.735+0000 7fe702270740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 0x55dfe792ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.735+0000 7fe702270740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe75566b0 0x55dfe791d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.735+0000 7fe6f895b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 0x55dfe792ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.735+0000 7fe702270740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f41b0010 con 0x55dfe754e4d0 2024-04-25T10:33:43.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.735+0000 7fe702270740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f41ad8e0 con 0x55dfe7535f60 2024-04-25T10:33:43.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.735+0000 7fe702270740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f41ab730 con 0x55dfe75566b0 2024-04-25T10:33:43.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.735+0000 7fe6f895b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 0x55dfe792ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43066/0 (socket says 172.21.15.43:43066) 2024-04-25T10:33:43.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.735+0000 7fe6f895b640 1 -- 172.21.15.43:0/1849657401 learned_addr learned my addr 172.21.15.43:0/1849657401 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:43.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.735+0000 7fe6f915c640 1 --2- 172.21.15.43:0/1849657401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe7535f60 0x55dfe7922420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.736+0000 7fe6f995d640 1 --2- 172.21.15.43:0/1849657401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe75566b0 0x55dfe791d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.736+0000 7fe6f895b640 1 -- 172.21.15.43:0/1849657401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe75566b0 msgr2=0x55dfe791d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.736+0000 7fe6f895b640 1 --2- 172.21.15.43:0/1849657401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe75566b0 0x55dfe791d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.736+0000 7fe6f895b640 1 -- 172.21.15.43:0/1849657401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe7535f60 msgr2=0x55dfe7922420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.736+0000 7fe6f895b640 1 --2- 172.21.15.43:0/1849657401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe7535f60 0x55dfe7922420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.736+0000 7fe6f895b640 1 -- 172.21.15.43:0/1849657401 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfe7921670 con 0x55dfe754e4d0 2024-04-25T10:33:43.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.736+0000 7fe6f995d640 1 --2- 172.21.15.43:0/1849657401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe75566b0 0x55dfe791d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:43.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.736+0000 7fe6f915c640 1 --2- 172.21.15.43:0/1849657401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe7535f60 0x55dfe7922420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:43.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.736+0000 7fe6f895b640 1 --2- 172.21.15.43:0/1849657401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 0x55dfe792ee70 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400b620 tx=0x7fe6e4013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7784f67c169276af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:43.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.736+0000 7fe6deffd640 1 -- 172.21.15.43:0/1849657401 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e402e070 con 0x55dfe754e4d0 2024-04-25T10:33:43.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.736+0000 7fe6deffd640 1 -- 172.21.15.43:0/1849657401 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6e4004660 con 0x55dfe754e4d0 2024-04-25T10:33:43.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.736+0000 7fe6deffd640 1 -- 172.21.15.43:0/1849657401 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4029330 con 0x55dfe754e4d0 2024-04-25T10:33:43.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.737+0000 7fe702270740 1 -- 172.21.15.43:0/1849657401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 msgr2=0x55dfe792ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.737+0000 7fe702270740 1 --2- 172.21.15.43:0/1849657401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 0x55dfe792ee70 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400b620 tx=0x7fe6e4013040 comp rx=0 tx=0).stop 2024-04-25T10:33:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.737+0000 7fe702270740 1 -- 172.21.15.43:0/1849657401 shutdown_connections 2024-04-25T10:33:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.737+0000 7fe702270740 1 --2- 172.21.15.43:0/1849657401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe75566b0 0x55dfe791d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.737+0000 7fe702270740 1 --2- 172.21.15.43:0/1849657401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 0x55dfe792ee70 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.737+0000 7fe702270740 1 --2- 172.21.15.43:0/1849657401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe7535f60 0x55dfe7922420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.737+0000 7fe702270740 1 -- 172.21.15.43:0/1849657401 >> 172.21.15.43:0/1849657401 conn(0x55dfe756b4e0 msgr2=0x55dfe7913610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.737+0000 7fe702270740 1 -- 172.21.15.43:0/1849657401 shutdown_connections 2024-04-25T10:33:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.737+0000 7fe702270740 1 -- 172.21.15.43:0/1849657401 wait complete. 2024-04-25T10:33:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.738+0000 7fe702270740 1 -- start start 2024-04-25T10:33:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.738+0000 7fe702270740 1 Processor -- start 2024-04-25T10:33:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.738+0000 7fe702270740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe7535f60 0x55dfe79267d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.738+0000 7fe702270740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 0x55dfe7926d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.738+0000 7fe702270740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe75566b0 0x55dfe7999f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.738+0000 7fe702270740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f41abda0 con 0x55dfe754e4d0 2024-04-25T10:33:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.738+0000 7fe702270740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f41ad210 con 0x55dfe75566b0 2024-04-25T10:33:43.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.738+0000 7fe702270740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f41ac470 con 0x55dfe7535f60 2024-04-25T10:33:43.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.738+0000 7fe702270740 10 client.0 _ll_register_callbacks cb 0x7fe6f00a3510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:43.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.738+0000 7fe6f895b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 0x55dfe7926d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.738+0000 7fe6f895b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 0x55dfe7926d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43078/0 (socket says 172.21.15.43:43078) 2024-04-25T10:33:43.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.738+0000 7fe6f895b640 1 -- 172.21.15.43:0/1883299261 learned_addr learned my addr 172.21.15.43:0/1883299261 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:43.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.738+0000 7fe6f995d640 1 --2- 172.21.15.43:0/1883299261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe75566b0 0x55dfe7999f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.739+0000 7fe6f895b640 1 -- 172.21.15.43:0/1883299261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe7535f60 msgr2=0x55dfe79267d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:33:43.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.739+0000 7fe6f895b640 1 --2- 172.21.15.43:0/1883299261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe7535f60 0x55dfe79267d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.739+0000 7fe6f895b640 1 -- 172.21.15.43:0/1883299261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe75566b0 msgr2=0x55dfe7999f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:43.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.739+0000 7fe6f895b640 1 --2- 172.21.15.43:0/1883299261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe75566b0 0x55dfe7999f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.739+0000 7fe6f895b640 1 -- 172.21.15.43:0/1883299261 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfe799a4d0 con 0x55dfe754e4d0 2024-04-25T10:33:43.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.739+0000 7fe6f895b640 1 --2- 172.21.15.43:0/1883299261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 0x55dfe7926d10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400b5f0 tx=0x7fe6e400b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10eac844085902a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:43.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.739+0000 7fe6dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:43.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.739+0000 7fe6dcff9640 1 -- 172.21.15.43:0/1883299261 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4039060 con 0x55dfe754e4d0 2024-04-25T10:33:43.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.739+0000 7fe6dcff9640 1 -- 172.21.15.43:0/1883299261 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7fe6e4004a10 con 0x55dfe754e4d0 2024-04-25T10:33:43.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.739+0000 7fe702270740 1 -- 172.21.15.43:0/1883299261 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6e4004660 con 0x55dfe754e4d0 2024-04-25T10:33:43.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.739+0000 7fe6dcff9640 1 -- 172.21.15.43:0/1883299261 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe6e402e040 con 0x55dfe754e4d0 2024-04-25T10:33:43.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.739+0000 7fe702270740 10 client.15810 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:43.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.739+0000 7fe702270740 1 -- 172.21.15.43:0/1883299261 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dfe79a1760 con 0x55dfe754e4d0 2024-04-25T10:33:43.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.740+0000 7fe6dcff9640 1 -- 172.21.15.43:0/1883299261 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6e4004660 con 0x55dfe754e4d0 2024-04-25T10:33:43.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.740+0000 7fe6dcff9640 1 -- 172.21.15.43:0/1883299261 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe6e403e5a0 con 0x55dfe754e4d0 2024-04-25T10:33:43.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.740+0000 7fe702270740 10 client.15810 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:43.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.740+0000 7fe702270740 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:43.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.740+0000 7fe702270740 20 client.15810 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:43.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.740+0000 7fe702270740 1 --2- 172.21.15.43:0/1883299261 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55dfe7577390 0x55dfe79a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:43.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.740+0000 7fe702270740 4 client.15810 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:33:43.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.740+0000 7fe702270740 1 -- 172.21.15.43:0/1883299261 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dfe788b990 con 0x55dfe7577390 2024-04-25T10:33:43.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.740+0000 7fe6f915c640 1 --2- 172.21.15.43:0/1883299261 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55dfe7577390 0x55dfe79a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.741+0000 7fe6dcff9640 10 client.15810 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.741+0000 7fe6f915c640 1 --2- 172.21.15.43:0/1883299261 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55dfe7577390 0x55dfe79a3c70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7179b39394141fc7 server_cookie=fb459581bda29175 in_seq=0 out_seq=0 2024-04-25T10:33:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.741+0000 7fe6dcff9640 1 -- 172.21.15.43:0/1883299261 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dfe788b990 con 0x55dfe7577390 2024-04-25T10:33:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.741+0000 7fe6dcff9640 10 client.15810 handle_command_reply: tid=0 2024-04-25T10:33:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.741+0000 7fe702270740 1 client.15810 shutdown 2024-04-25T10:33:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.741+0000 7fe702270740 1 -- 172.21.15.43:0/1883299261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 msgr2=0x55dfe7926d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.741+0000 7fe702270740 1 --2- 172.21.15.43:0/1883299261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 0x55dfe7926d10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400b5f0 tx=0x7fe6e400b780 comp rx=0 tx=0).stop 2024-04-25T10:33:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.741+0000 7fe702270740 1 -- 172.21.15.43:0/1883299261 shutdown_connections 2024-04-25T10:33:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.741+0000 7fe702270740 1 --2- 172.21.15.43:0/1883299261 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55dfe7577390 0x55dfe79a3c70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.741+0000 7fe702270740 1 --2- 172.21.15.43:0/1883299261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfe75566b0 0x55dfe7999f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.741+0000 7fe702270740 1 --2- 172.21.15.43:0/1883299261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfe754e4d0 0x55dfe7926d10 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.741+0000 7fe702270740 1 --2- 172.21.15.43:0/1883299261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfe7535f60 0x55dfe79267d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.741+0000 7fe702270740 1 -- 172.21.15.43:0/1883299261 >> 172.21.15.43:0/1883299261 conn(0x55dfe756b4e0 msgr2=0x55dfe7922180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.741+0000 7fe6dcff9640 0 client.15810 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.741+0000 7fe702270740 1 -- 172.21.15.43:0/1883299261 shutdown_connections 2024-04-25T10:33:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.741+0000 7fe702270740 1 -- 172.21.15.43:0/1883299261 wait complete. 2024-04-25T10:33:43.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.741+0000 7fe702270740 20 client.15810 trim_cache size 0 max 16384 2024-04-25T10:33:43.749 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:33:43.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.742+0000 7fe6fb3c0640 1 -- 172.21.15.43:0/3594593153 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe6e00afe10 msgr2=0x7fe6e00b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.742+0000 7fe6fb3c0640 1 --2- 172.21.15.43:0/3594593153 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe6e00afe10 0x7fe6e00b2320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55dfe7915980 tx=0x7fe6ec005c50 comp rx=0 tx=0).stop 2024-04-25T10:33:43.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.742+0000 7fe6fb3c0640 1 -- 172.21.15.43:0/3594593153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6f41042d0 msgr2=0x7fe6f419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:43.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.742+0000 7fe6fb3c0640 1 --2- 172.21.15.43:0/3594593153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6f41042d0 0x7fe6f419a700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000e9c0 tx=0x7fe6f000ee80 comp rx=0 tx=0).stop 2024-04-25T10:33:43.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.742+0000 7fe6fb3c0640 1 -- 172.21.15.43:0/3594593153 shutdown_connections 2024-04-25T10:33:43.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.742+0000 7fe6fb3c0640 1 --2- 172.21.15.43:0/3594593153 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe6e00afe10 0x7fe6e00b2320 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.742+0000 7fe6fb3c0640 1 --2- 172.21.15.43:0/3594593153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6f4109b30 0x7fe6f419ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.742+0000 7fe6fb3c0640 1 --2- 172.21.15.43:0/3594593153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f4109fd0 0x7fe6f419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.742+0000 7fe6fb3c0640 1 --2- 172.21.15.43:0/3594593153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6f41042d0 0x7fe6f419a700 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:43.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.742+0000 7fe6fb3c0640 1 -- 172.21.15.43:0/3594593153 >> 172.21.15.43:0/3594593153 conn(0x7fe6f4079280 msgr2=0x7fe6f4100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:43.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.743+0000 7fe6fb3c0640 1 -- 172.21.15.43:0/3594593153 shutdown_connections 2024-04-25T10:33:43.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:43.743+0000 7fe6fb3c0640 1 -- 172.21.15.43:0/3594593153 wait complete. 2024-04-25T10:33:44.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:43 smithi077 ceph-mon[32772]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T10:33:44.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:43 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1542787814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:33:44.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:43 smithi164 ceph-mon[32713]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T10:33:44.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:43 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1542787814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:33:44.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:43 smithi043 ceph-mon[29478]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T10:33:44.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:43 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1542787814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:33:44.756 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:33:44.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:33:44.757 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:33:44.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.884+0000 7f55fc99f640 1 -- 172.21.15.43:0/219723951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f810de80 msgr2=0x7f55f8110370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:44.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.884+0000 7f55fc99f640 1 --2- 172.21.15.43:0/219723951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f810de80 0x7f55f8110370 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009ca0 tx=0x7f55f002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.884+0000 7f55fc99f640 1 -- 172.21.15.43:0/219723951 shutdown_connections 2024-04-25T10:33:44.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.884+0000 7f55fc99f640 1 --2- 172.21.15.43:0/219723951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f810de80 0x7f55f8110370 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.884+0000 7f55fc99f640 1 --2- 172.21.15.43:0/219723951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f810b490 0x7f55f810d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.884+0000 7f55fc99f640 1 --2- 172.21.15.43:0/219723951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55f8104a30 0x7f55f8104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.884+0000 7f55fc99f640 1 -- 172.21.15.43:0/219723951 >> 172.21.15.43:0/219723951 conn(0x7f55f80fdf10 msgr2=0x7f55f8100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:44.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.884+0000 7f55fc99f640 1 -- 172.21.15.43:0/219723951 shutdown_connections 2024-04-25T10:33:44.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.884+0000 7f55fc99f640 1 -- 172.21.15.43:0/219723951 wait complete. 2024-04-25T10:33:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.885+0000 7f55fc99f640 1 Processor -- start 2024-04-25T10:33:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.885+0000 7f55fc99f640 1 -- start start 2024-04-25T10:33:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.885+0000 7f55fc99f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f8104a30 0x7f55f819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.885+0000 7f55fc99f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f810b490 0x7f55f819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.885+0000 7f55fc99f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55f810de80 0x7f55f819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.885+0000 7f55fc99f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8113ae0 con 0x7f55f8104a30 2024-04-25T10:33:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.885+0000 7f55fc99f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8113960 con 0x7f55f810de80 2024-04-25T10:33:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.885+0000 7f55fc99f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8113c60 con 0x7f55f810b490 2024-04-25T10:33:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.886+0000 7f55f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f8104a30 0x7f55f819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.886+0000 7f55f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f8104a30 0x7f55f819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43092/0 (socket says 172.21.15.43:43092) 2024-04-25T10:33:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.886+0000 7f55f77fe640 1 -- 172.21.15.43:0/24253863 learned_addr learned my addr 172.21.15.43:0/24253863 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.886+0000 7f55f6ffd640 1 --2- 172.21.15.43:0/24253863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f810b490 0x7f55f819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.886+0000 7f55f7fff640 1 --2- 172.21.15.43:0/24253863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55f810de80 0x7f55f819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.886+0000 7f55f77fe640 1 -- 172.21.15.43:0/24253863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f810b490 msgr2=0x7f55f819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.886+0000 7f55f77fe640 1 --2- 172.21.15.43:0/24253863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f810b490 0x7f55f819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.886+0000 7f55f77fe640 1 -- 172.21.15.43:0/24253863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55f810de80 msgr2=0x7f55f819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.886+0000 7f55f77fe640 1 --2- 172.21.15.43:0/24253863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55f810de80 0x7f55f819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.886+0000 7f55f77fe640 1 -- 172.21.15.43:0/24253863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f819f340 con 0x7f55f8104a30 2024-04-25T10:33:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.886+0000 7f55f6ffd640 1 --2- 172.21.15.43:0/24253863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f810b490 0x7f55f819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.886+0000 7f55f77fe640 1 --2- 172.21.15.43:0/24253863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f8104a30 0x7f55f819a4e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f55e800e9f0 tx=0x7f55e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.886+0000 7f55f4ff9640 1 -- 172.21.15.43:0/24253863 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e800cd80 con 0x7f55f8104a30 2024-04-25T10:33:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.887+0000 7f55f4ff9640 1 -- 172.21.15.43:0/24253863 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55e80108e0 con 0x7f55f8104a30 2024-04-25T10:33:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.887+0000 7f55f4ff9640 1 -- 172.21.15.43:0/24253863 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e8018980 con 0x7f55f8104a30 2024-04-25T10:33:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.887+0000 7f55fc99f640 1 -- 172.21.15.43:0/24253863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f819f630 con 0x7f55f8104a30 2024-04-25T10:33:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.887+0000 7f55fc99f640 1 -- 172.21.15.43:0/24253863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f819fb30 con 0x7f55f8104a30 2024-04-25T10:33:44.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.890+0000 7f5604b8b740 1 Processor -- start 2024-04-25T10:33:44.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.890+0000 7f5604b8b740 1 -- start start 2024-04-25T10:33:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.890+0000 7f55f4ff9640 1 -- 172.21.15.43:0/24253863 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 147804+0+0 (secure 0 0 0) 0x7f55e8018b20 con 0x7f55f8104a30 2024-04-25T10:33:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.891+0000 7f5604b8b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e42410 0x55ef681e27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.891+0000 7f5604b8b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e19550 0x55ef681e2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.891+0000 7f5604b8b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e21730 0x55ef681e3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.891+0000 7f5604b8b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ef68148b00 con 0x55ef67e21730 2024-04-25T10:33:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.891+0000 7f5604b8b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ef680e4920 con 0x55ef67e42410 2024-04-25T10:33:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.891+0000 7f5604b8b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ef68144ac0 con 0x55ef67e19550 2024-04-25T10:33:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.891+0000 7f55f7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e42410 0x55ef681e27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.891+0000 7f55f7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e42410 0x55ef681e27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:53816/0 (socket says 172.21.15.43:53816) 2024-04-25T10:33:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.891+0000 7f55f7fff640 1 -- 172.21.15.43:0/1207815142 learned_addr learned my addr 172.21.15.43:0/1207815142 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.891+0000 7f55f6ffd640 1 --2- 172.21.15.43:0/1207815142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e21730 0x55ef681e3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.891+0000 7f55f77fe640 1 --2- 172.21.15.43:0/1207815142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e19550 0x55ef681e2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.891+0000 7f55f7fff640 1 -- 172.21.15.43:0/1207815142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e19550 msgr2=0x55ef681e2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.891+0000 7f55f7fff640 1 --2- 172.21.15.43:0/1207815142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e19550 0x55ef681e2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.891+0000 7f55f7fff640 1 -- 172.21.15.43:0/1207815142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e21730 msgr2=0x55ef681e3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.891+0000 7f55f7fff640 1 --2- 172.21.15.43:0/1207815142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e21730 0x55ef681e3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.891+0000 7f55f7fff640 1 -- 172.21.15.43:0/1207815142 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef67f227e0 con 0x55ef67e42410 2024-04-25T10:33:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.891+0000 7f55f6ffd640 1 --2- 172.21.15.43:0/1207815142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e21730 0x55ef681e3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.891+0000 7f55f4ff9640 1 --2- 172.21.15.43:0/24253863 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f55e00afe80 0x7f55e00b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.892+0000 7f55f4ff9640 1 -- 172.21.15.43:0/24253863 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f55e80df5b0 con 0x7f55f8104a30 2024-04-25T10:33:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.892+0000 7f55f77fe640 1 --2- 172.21.15.43:0/24253863 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f55e00afe80 0x7f55e00b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.892+0000 7f55f7fff640 1 --2- 172.21.15.43:0/1207815142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e42410 0x55ef681e27e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f55f002f410 tx=0x7f55f002f440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=640712d888b1eaee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.892+0000 7f55ddffb640 1 -- 172.21.15.43:0/1207815142 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0031cf0 con 0x55ef67e42410 2024-04-25T10:33:44.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.892+0000 7f55ddffb640 1 -- 172.21.15.43:0/1207815142 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f0036030 con 0x55ef67e42410 2024-04-25T10:33:44.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.892+0000 7f55ddffb640 1 -- 172.21.15.43:0/1207815142 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f00042a0 con 0x55ef67e42410 2024-04-25T10:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.893+0000 7f5604b8b740 1 -- 172.21.15.43:0/1207815142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e42410 msgr2=0x55ef681e27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.893+0000 7f5604b8b740 1 --2- 172.21.15.43:0/1207815142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e42410 0x55ef681e27e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f55f002f410 tx=0x7f55f002f440 comp rx=0 tx=0).stop 2024-04-25T10:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.893+0000 7f55f77fe640 1 --2- 172.21.15.43:0/24253863 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f55e00afe80 0x7f55e00b2390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f55e80dcb50 tx=0x7f55e8000bc0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.893+0000 7f5604b8b740 1 -- 172.21.15.43:0/1207815142 shutdown_connections 2024-04-25T10:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.893+0000 7f5604b8b740 1 --2- 172.21.15.43:0/1207815142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e21730 0x55ef681e3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.893+0000 7f5604b8b740 1 --2- 172.21.15.43:0/1207815142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e19550 0x55ef681e2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.893+0000 7f5604b8b740 1 --2- 172.21.15.43:0/1207815142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e42410 0x55ef681e27e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.893+0000 7f5604b8b740 1 -- 172.21.15.43:0/1207815142 >> 172.21.15.43:0/1207815142 conn(0x55ef67e36560 msgr2=0x55ef681def90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.893+0000 7f5604b8b740 1 -- 172.21.15.43:0/1207815142 shutdown_connections 2024-04-25T10:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.893+0000 7f5604b8b740 1 -- 172.21.15.43:0/1207815142 wait complete. 2024-04-25T10:33:44.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.894+0000 7f5604b8b740 1 -- start start 2024-04-25T10:33:44.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.894+0000 7f5604b8b740 1 Processor -- start 2024-04-25T10:33:44.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.895+0000 7f5604b8b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e19550 0x55ef681ece60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.895+0000 7f5604b8b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e21730 0x55ef681ed3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.895+0000 7f5604b8b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e42410 0x55ef681f2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.895+0000 7f5604b8b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ef681ef950 con 0x55ef67e42410 2024-04-25T10:33:44.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.895+0000 7f5604b8b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ef680e25d0 con 0x55ef67e19550 2024-04-25T10:33:44.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.895+0000 7f5604b8b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ef681efad0 con 0x55ef67e21730 2024-04-25T10:33:44.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.895+0000 7f5604b8b740 10 client.0 _ll_register_callbacks cb 0x55ef680d0080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.895+0000 7f55f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e42410 0x55ef681f2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.895+0000 7f55f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e42410 0x55ef681f2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43102/0 (socket says 172.21.15.43:43102) 2024-04-25T10:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.895+0000 7f55f7fff640 1 -- 172.21.15.43:0/4223878260 learned_addr learned my addr 172.21.15.43:0/4223878260 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.895+0000 7f55f7fff640 1 --2- 172.21.15.43:0/4223878260 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e19550 0x55ef681ece60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.895+0000 7f55f6ffd640 1 --2- 172.21.15.43:0/4223878260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e21730 0x55ef681ed3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.895+0000 7f55f7fff640 1 -- 172.21.15.43:0/4223878260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e21730 msgr2=0x55ef681ed3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.895+0000 7f55f7fff640 1 --2- 172.21.15.43:0/4223878260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e21730 0x55ef681ed3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.895+0000 7f55f7fff640 1 -- 172.21.15.43:0/4223878260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e42410 msgr2=0x55ef681f2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.895+0000 7f55f7fff640 1 --2- 172.21.15.43:0/4223878260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e42410 0x55ef681f2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.895+0000 7f55f7fff640 1 -- 172.21.15.43:0/4223878260 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef67f82f70 con 0x55ef67e19550 2024-04-25T10:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.896+0000 7f55f7fff640 1 --2- 172.21.15.43:0/4223878260 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e19550 0x55ef681ece60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009620 tx=0x7f55f002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f623ef5209ddf251 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.896+0000 7f55d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-04-25T10:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.896+0000 7f55d6ffd640 1 -- 172.21.15.43:0/4223878260 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0059060 con 0x55ef67e19550 2024-04-25T10:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.896+0000 7f5604b8b740 1 -- 172.21.15.43:0/4223878260 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55f0036030 con 0x55ef67e19550 2024-04-25T10:33:44.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.896+0000 7f55d6ffd640 1 -- 172.21.15.43:0/4223878260 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f55f0004740 con 0x55ef67e19550 2024-04-25T10:33:44.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.896+0000 7f55d6ffd640 1 -- 172.21.15.43:0/4223878260 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f55f0031450 con 0x55ef67e19550 2024-04-25T10:33:44.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.896+0000 7f5604b8b740 10 client.25541 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:44.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.896+0000 7f5604b8b740 1 -- 172.21.15.43:0/4223878260 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ef67e1a170 con 0x55ef67e19550 2024-04-25T10:33:44.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.896+0000 7f55d6ffd640 1 -- 172.21.15.43:0/4223878260 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f0036030 con 0x55ef67e19550 2024-04-25T10:33:44.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.897+0000 7f55d6ffd640 1 -- 172.21.15.43:0/4223878260 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f55f0043070 con 0x55ef67e19550 2024-04-25T10:33:44.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.897+0000 7f5604b8b740 10 client.25541 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:44.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.897+0000 7f5604b8b740 10 client.25541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:44.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.897+0000 7f5604b8b740 20 client.25541 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:44.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.897+0000 7f5604b8b740 1 --2- 172.21.15.43:0/4223878260 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ef67e011f0 0x55ef681fee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.897+0000 7f5604b8b740 4 client.25541 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:33:44.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.897+0000 7f5604b8b740 1 -- 172.21.15.43:0/4223878260 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef68156710 con 0x55ef67e011f0 2024-04-25T10:33:44.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.897+0000 7f55f6ffd640 1 --2- 172.21.15.43:0/4223878260 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ef67e011f0 0x55ef681fee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.897+0000 7f55d6ffd640 10 client.25541 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:44.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.897+0000 7f55f6ffd640 1 --2- 172.21.15.43:0/4223878260 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ef67e011f0 0x55ef681fee80 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ae8c14d6232e628 server_cookie=f02c0ac67642795f in_seq=0 out_seq=0 2024-04-25T10:33:44.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.898+0000 7f55d6ffd640 1 -- 172.21.15.43:0/4223878260 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef68156710 con 0x55ef67e011f0 2024-04-25T10:33:44.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.898+0000 7f55d6ffd640 10 client.25541 handle_command_reply: tid=0 2024-04-25T10:33:44.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.898+0000 7f5604b8b740 1 client.25541 shutdown 2024-04-25T10:33:44.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.898+0000 7f5604b8b740 1 -- 172.21.15.43:0/4223878260 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e19550 msgr2=0x55ef681ece60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:44.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.898+0000 7f5604b8b740 1 --2- 172.21.15.43:0/4223878260 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e19550 0x55ef681ece60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009620 tx=0x7f55f002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.898+0000 7f5604b8b740 1 -- 172.21.15.43:0/4223878260 shutdown_connections 2024-04-25T10:33:44.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.898+0000 7f5604b8b740 1 --2- 172.21.15.43:0/4223878260 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ef67e011f0 0x55ef681fee80 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.899+0000 7f5604b8b740 1 --2- 172.21.15.43:0/4223878260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e42410 0x55ef681f2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.899+0000 7f55d6ffd640 0 client.25541 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:44.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.899+0000 7f5604b8b740 1 --2- 172.21.15.43:0/4223878260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e21730 0x55ef681ed3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.899+0000 7f5604b8b740 1 --2- 172.21.15.43:0/4223878260 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e19550 0x55ef681ece60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.899+0000 7f5604b8b740 1 -- 172.21.15.43:0/4223878260 >> 172.21.15.43:0/4223878260 conn(0x55ef67e36560 msgr2=0x55ef681e2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:44.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.899+0000 7f5604b8b740 1 -- 172.21.15.43:0/4223878260 shutdown_connections 2024-04-25T10:33:44.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.899+0000 7f5604b8b740 1 -- 172.21.15.43:0/4223878260 wait complete. 2024-04-25T10:33:44.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.899+0000 7f5604b8b740 20 client.25541 trim_cache size 0 max 16384 2024-04-25T10:33:44.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.936+0000 7f5604b8b740 1 Processor -- start 2024-04-25T10:33:44.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.936+0000 7f5604b8b740 1 -- start start 2024-04-25T10:33:44.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.936+0000 7f5604b8b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e011f0 0x55ef681e7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.936+0000 7f5604b8b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e19550 0x55ef681f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.936+0000 7f5604b8b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e21730 0x55ef681f3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.936+0000 7f5604b8b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81b1ff0 con 0x55ef67e21730 2024-04-25T10:33:44.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.936+0000 7f5604b8b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81af8c0 con 0x55ef67e011f0 2024-04-25T10:33:44.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.936+0000 7f5604b8b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81ad710 con 0x55ef67e19550 2024-04-25T10:33:44.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.937+0000 7f55f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e21730 0x55ef681f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.937+0000 7f55f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e21730 0x55ef681f3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43110/0 (socket says 172.21.15.43:43110) 2024-04-25T10:33:44.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.937+0000 7f55f7fff640 1 -- 172.21.15.43:0/3301702515 learned_addr learned my addr 172.21.15.43:0/3301702515 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:44.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.937+0000 7f55f7fff640 1 --2- 172.21.15.43:0/3301702515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e011f0 0x55ef681e7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.937+0000 7f55f6ffd640 1 --2- 172.21.15.43:0/3301702515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e19550 0x55ef681f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.937+0000 7f55f7fff640 1 -- 172.21.15.43:0/3301702515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e19550 msgr2=0x55ef681f2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:44.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.937+0000 7f55f7fff640 1 --2- 172.21.15.43:0/3301702515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e19550 0x55ef681f2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.937+0000 7f55f7fff640 1 -- 172.21.15.43:0/3301702515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e011f0 msgr2=0x55ef681e7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:44.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.937+0000 7f55f7fff640 1 --2- 172.21.15.43:0/3301702515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e011f0 0x55ef681e7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.937+0000 7f55f7fff640 1 -- 172.21.15.43:0/3301702515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef681ff1f0 con 0x55ef67e21730 2024-04-25T10:33:44.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.937+0000 7f55f6ffd640 1 --2- 172.21.15.43:0/3301702515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e19550 0x55ef681f2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:44.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.937+0000 7f55f7fff640 1 --2- 172.21.15.43:0/3301702515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e21730 0x55ef681f3280 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f55f00043d0 tx=0x7f55f0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccd46d15e961971 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:44.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.937+0000 7f55dd7fa640 1 -- 172.21.15.43:0/3301702515 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0059070 con 0x55ef67e21730 2024-04-25T10:33:44.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.938+0000 7f55dd7fa640 1 -- 172.21.15.43:0/3301702515 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f0036040 con 0x55ef67e21730 2024-04-25T10:33:44.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.938+0000 7f55dd7fa640 1 -- 172.21.15.43:0/3301702515 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0004540 con 0x55ef67e21730 2024-04-25T10:33:44.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.939+0000 7f5604b8b740 1 -- 172.21.15.43:0/3301702515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e21730 msgr2=0x55ef681f3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:44.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.939+0000 7f5604b8b740 1 --2- 172.21.15.43:0/3301702515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e21730 0x55ef681f3280 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f55f00043d0 tx=0x7f55f0003c20 comp rx=0 tx=0).stop 2024-04-25T10:33:44.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.939+0000 7f5604b8b740 1 -- 172.21.15.43:0/3301702515 shutdown_connections 2024-04-25T10:33:44.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.939+0000 7f5604b8b740 1 --2- 172.21.15.43:0/3301702515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e21730 0x55ef681f3280 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.939+0000 7f5604b8b740 1 --2- 172.21.15.43:0/3301702515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e19550 0x55ef681f2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.939+0000 7f5604b8b740 1 --2- 172.21.15.43:0/3301702515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e011f0 0x55ef681e7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.939+0000 7f5604b8b740 1 -- 172.21.15.43:0/3301702515 >> 172.21.15.43:0/3301702515 conn(0x55ef67e36560 msgr2=0x55ef681ddf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:44.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.939+0000 7f5604b8b740 1 -- 172.21.15.43:0/3301702515 shutdown_connections 2024-04-25T10:33:44.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.939+0000 7f5604b8b740 1 -- 172.21.15.43:0/3301702515 wait complete. 2024-04-25T10:33:44.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.939+0000 7f5604b8b740 1 -- start start 2024-04-25T10:33:44.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.940+0000 7f5604b8b740 1 Processor -- start 2024-04-25T10:33:44.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.940+0000 7f5604b8b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e011f0 0x55ef681f9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.940+0000 7f5604b8b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e19550 0x55ef681f9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.940+0000 7f5604b8b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e21730 0x55ef681f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.940+0000 7f55f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e011f0 0x55ef681f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.940+0000 7f55f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e011f0 0x55ef681f9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43114/0 (socket says 172.21.15.43:43114) 2024-04-25T10:33:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.940+0000 7f55f7fff640 1 -- 172.21.15.43:0/177539583 learned_addr learned my addr 172.21.15.43:0/177539583 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.940+0000 7f5604b8b740 1 -- 172.21.15.43:0/177539583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81add80 con 0x55ef67e011f0 2024-04-25T10:33:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.940+0000 7f5604b8b740 1 -- 172.21.15.43:0/177539583 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81af1f0 con 0x55ef67e21730 2024-04-25T10:33:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.940+0000 7f5604b8b740 1 -- 172.21.15.43:0/177539583 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81ae450 con 0x55ef67e19550 2024-04-25T10:33:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.940+0000 7f55f6ffd640 1 --2- 172.21.15.43:0/177539583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e19550 0x55ef681f9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.940+0000 7f5604b8b740 10 client.0 _ll_register_callbacks cb 0x7f55f0003a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.941+0000 7f55f7fff640 1 --2- 172.21.15.43:0/177539583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e21730 0x55ef681f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.941+0000 7f55f7fff640 1 -- 172.21.15.43:0/177539583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e19550 msgr2=0x55ef681f9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.941+0000 7f55f7fff640 1 --2- 172.21.15.43:0/177539583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e19550 0x55ef681f9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.941+0000 7f55f7fff640 1 -- 172.21.15.43:0/177539583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e21730 msgr2=0x55ef681f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:44.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.941+0000 7f55f7fff640 1 --2- 172.21.15.43:0/177539583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e21730 0x55ef681f45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.941+0000 7f55f7fff640 1 -- 172.21.15.43:0/177539583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef681f4c70 con 0x55ef67e011f0 2024-04-25T10:33:44.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.941+0000 7f55f7fff640 1 --2- 172.21.15.43:0/177539583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e011f0 0x55ef681f9140 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f55f0000f80 tx=0x7f55f002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4dd662cd34bfad8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:44.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.941+0000 7f55d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:44.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.941+0000 7f55d6ffd640 1 -- 172.21.15.43:0/177539583 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0059040 con 0x55ef67e011f0 2024-04-25T10:33:44.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.941+0000 7f55d6ffd640 1 -- 172.21.15.43:0/177539583 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f55f00317f0 con 0x55ef67e011f0 2024-04-25T10:33:44.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.941+0000 7f5604b8b740 1 -- 172.21.15.43:0/177539583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55f0036040 con 0x55ef67e011f0 2024-04-25T10:33:44.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.942+0000 7f55d6ffd640 1 -- 172.21.15.43:0/177539583 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f55f0004750 con 0x55ef67e011f0 2024-04-25T10:33:44.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.942+0000 7f5604b8b740 10 client.15828 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:44.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.942+0000 7f5604b8b740 1 -- 172.21.15.43:0/177539583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ef68266960 con 0x55ef67e011f0 2024-04-25T10:33:44.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.942+0000 7f55d6ffd640 1 -- 172.21.15.43:0/177539583 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f0036040 con 0x55ef67e011f0 2024-04-25T10:33:44.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.942+0000 7f55d6ffd640 1 -- 172.21.15.43:0/177539583 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f55f0043070 con 0x55ef67e011f0 2024-04-25T10:33:44.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.942+0000 7f5604b8b740 10 client.15828 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:44.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.942+0000 7f5604b8b740 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:44.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.942+0000 7f5604b8b740 20 client.15828 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:44.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.942+0000 7f5604b8b740 1 --2- 172.21.15.43:0/177539583 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ef67e42410 0x55ef68268f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.942+0000 7f5604b8b740 4 client.15828 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:33:44.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.942+0000 7f5604b8b740 1 -- 172.21.15.43:0/177539583 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ef68150580 con 0x55ef67e42410 2024-04-25T10:33:44.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.942+0000 7f55f6ffd640 1 --2- 172.21.15.43:0/177539583 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ef67e42410 0x55ef68268f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.943+0000 7f55f6ffd640 1 --2- 172.21.15.43:0/177539583 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ef67e42410 0x55ef68268f30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=502bcc1e161db381 server_cookie=3315fe8305995291 in_seq=0 out_seq=0 2024-04-25T10:33:44.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.943+0000 7f55d6ffd640 10 client.15828 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:44.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.944+0000 7f55d6ffd640 1 -- 172.21.15.43:0/177539583 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ef68150580 con 0x55ef67e42410 2024-04-25T10:33:44.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.944+0000 7f55d6ffd640 10 client.15828 handle_command_reply: tid=0 2024-04-25T10:33:44.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.944+0000 7f5604b8b740 1 client.15828 shutdown 2024-04-25T10:33:44.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.945+0000 7f5604b8b740 1 -- 172.21.15.43:0/177539583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e011f0 msgr2=0x55ef681f9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:44.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.945+0000 7f5604b8b740 1 --2- 172.21.15.43:0/177539583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e011f0 0x55ef681f9140 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f55f0000f80 tx=0x7f55f002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.945+0000 7f5604b8b740 1 -- 172.21.15.43:0/177539583 shutdown_connections 2024-04-25T10:33:44.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.945+0000 7f5604b8b740 1 --2- 172.21.15.43:0/177539583 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ef67e42410 0x55ef68268f30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.945+0000 7f55d6ffd640 0 client.15828 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:44.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.945+0000 7f5604b8b740 1 --2- 172.21.15.43:0/177539583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef67e21730 0x55ef681f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.945+0000 7f5604b8b740 1 --2- 172.21.15.43:0/177539583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ef67e19550 0x55ef681f9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.945+0000 7f5604b8b740 1 --2- 172.21.15.43:0/177539583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef67e011f0 0x55ef681f9140 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.945+0000 7f5604b8b740 1 -- 172.21.15.43:0/177539583 >> 172.21.15.43:0/177539583 conn(0x55ef67e36560 msgr2=0x55ef681ec4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.945+0000 7f5604b8b740 1 -- 172.21.15.43:0/177539583 shutdown_connections 2024-04-25T10:33:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.945+0000 7f5604b8b740 1 -- 172.21.15.43:0/177539583 wait complete. 2024-04-25T10:33:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.945+0000 7f5604b8b740 20 client.15828 trim_cache size 0 max 16384 2024-04-25T10:33:44.950 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:33:44.950 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:33:44.950 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "3103385c-e046-4f88-8173-0f1fe7c66d1a", 2024-04-25T10:33:44.950 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:33:44.950 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:33:44.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.946+0000 7f55fc99f640 1 -- 172.21.15.43:0/24253863 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f55e00afe80 msgr2=0x7f55e00b2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:44.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.946+0000 7f55fc99f640 1 --2- 172.21.15.43:0/24253863 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f55e00afe80 0x7f55e00b2390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f55e80dcb50 tx=0x7f55e8000bc0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.946+0000 7f55fc99f640 1 -- 172.21.15.43:0/24253863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f8104a30 msgr2=0x7f55f819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:44.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.946+0000 7f55fc99f640 1 --2- 172.21.15.43:0/24253863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f8104a30 0x7f55f819a4e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f55e800e9f0 tx=0x7f55e800eeb0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.946+0000 7f55fc99f640 1 -- 172.21.15.43:0/24253863 shutdown_connections 2024-04-25T10:33:44.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.946+0000 7f55fc99f640 1 --2- 172.21.15.43:0/24253863 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f55e00afe80 0x7f55e00b2390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.946+0000 7f55fc99f640 1 --2- 172.21.15.43:0/24253863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55f810de80 0x7f55f819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.946+0000 7f55fc99f640 1 --2- 172.21.15.43:0/24253863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f810b490 0x7f55f819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.946+0000 7f55fc99f640 1 --2- 172.21.15.43:0/24253863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f8104a30 0x7f55f819a4e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.946+0000 7f55fc99f640 1 -- 172.21.15.43:0/24253863 >> 172.21.15.43:0/24253863 conn(0x7f55f80fdf10 msgr2=0x7f55f810e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:44.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.946+0000 7f55fc99f640 1 -- 172.21.15.43:0/24253863 shutdown_connections 2024-04-25T10:33:44.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:44.946+0000 7f55fc99f640 1 -- 172.21.15.43:0/24253863 wait complete. 2024-04-25T10:33:44.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:33:44.964 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:33:45.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.088+0000 7f1757bce640 1 -- 172.21.15.43:0/4131414067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1750109fd0 msgr2=0x7f1750103c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.088+0000 7f1757bce640 1 --2- 172.21.15.43:0/4131414067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1750109fd0 0x7f1750103c60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1740009900 tx=0x7f174002a410 comp rx=0 tx=0).stop 2024-04-25T10:33:45.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.088+0000 7f1757bce640 1 -- 172.21.15.43:0/4131414067 shutdown_connections 2024-04-25T10:33:45.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.088+0000 7f1757bce640 1 --2- 172.21.15.43:0/4131414067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17501042d0 0x7f1750110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.088+0000 7f1757bce640 1 --2- 172.21.15.43:0/4131414067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1750109fd0 0x7f1750103c60 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.088+0000 7f1757bce640 1 --2- 172.21.15.43:0/4131414067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1750109b30 0x7f1750103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.088+0000 7f1757bce640 1 -- 172.21.15.43:0/4131414067 >> 172.21.15.43:0/4131414067 conn(0x7f1750079280 msgr2=0x7f17500796c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:45.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.088+0000 7f1757bce640 1 -- 172.21.15.43:0/4131414067 shutdown_connections 2024-04-25T10:33:45.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.088+0000 7f1757bce640 1 -- 172.21.15.43:0/4131414067 wait complete. 2024-04-25T10:33:45.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.089+0000 7f1757bce640 1 Processor -- start 2024-04-25T10:33:45.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.089+0000 7f1757bce640 1 -- start start 2024-04-25T10:33:45.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.089+0000 7f1757bce640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17501042d0 0x7f175019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.089+0000 7f1757bce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1750109b30 0x7f175019ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.089+0000 7f1757bce640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1750109fd0 0x7f175019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.089+0000 7f1757bce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1750113d20 con 0x7f1750109b30 2024-04-25T10:33:45.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.089+0000 7f1757bce640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1750113ba0 con 0x7f17501042d0 2024-04-25T10:33:45.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.089+0000 7f1757bce640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1750113ea0 con 0x7f1750109fd0 2024-04-25T10:33:45.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.089+0000 7f1755169640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1750109b30 0x7f175019ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.089+0000 7f1755169640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1750109b30 0x7f175019ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43132/0 (socket says 172.21.15.43:43132) 2024-04-25T10:33:45.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.089+0000 7f1755169640 1 -- 172.21.15.43:0/1196339086 learned_addr learned my addr 172.21.15.43:0/1196339086 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:45.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.090+0000 7f175596a640 1 --2- 172.21.15.43:0/1196339086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17501042d0 0x7f175019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.090+0000 7f175616b640 1 --2- 172.21.15.43:0/1196339086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1750109fd0 0x7f175019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.090+0000 7f1755169640 1 -- 172.21.15.43:0/1196339086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1750109fd0 msgr2=0x7f175019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.090+0000 7f1755169640 1 --2- 172.21.15.43:0/1196339086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1750109fd0 0x7f175019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.090+0000 7f1755169640 1 -- 172.21.15.43:0/1196339086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17501042d0 msgr2=0x7f175019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.090+0000 7f1755169640 1 --2- 172.21.15.43:0/1196339086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17501042d0 0x7f175019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.090+0000 7f1755169640 1 -- 172.21.15.43:0/1196339086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f175019f5e0 con 0x7f1750109b30 2024-04-25T10:33:45.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.090+0000 7f175596a640 1 --2- 172.21.15.43:0/1196339086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17501042d0 0x7f175019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:45.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.090+0000 7f1755169640 1 --2- 172.21.15.43:0/1196339086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1750109b30 0x7f175019ace0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1740006010 tx=0x7f1740004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.090+0000 7f1746ffd640 1 -- 172.21.15.43:0/1196339086 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174003f030 con 0x7f1750109b30 2024-04-25T10:33:45.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.090+0000 7f1746ffd640 1 -- 172.21.15.43:0/1196339086 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1740003aa0 con 0x7f1750109b30 2024-04-25T10:33:45.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.090+0000 7f1757bce640 1 -- 172.21.15.43:0/1196339086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f175019f870 con 0x7f1750109b30 2024-04-25T10:33:45.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.091+0000 7f1746ffd640 1 -- 172.21.15.43:0/1196339086 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1740035070 con 0x7f1750109b30 2024-04-25T10:33:45.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.091+0000 7f1757bce640 1 -- 172.21.15.43:0/1196339086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f175019fdb0 con 0x7f1750109b30 2024-04-25T10:33:45.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.093+0000 7f1746ffd640 1 -- 172.21.15.43:0/1196339086 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 147804+0+0 (secure 0 0 0) 0x7f1740042030 con 0x7f1750109b30 2024-04-25T10:33:45.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.094+0000 7f1746ffd640 1 --2- 172.21.15.43:0/1196339086 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f173c0afe90 0x7f173c0b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.095+0000 7f1746ffd640 1 -- 172.21.15.43:0/1196339086 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f17400ff030 con 0x7f1750109b30 2024-04-25T10:33:45.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.095+0000 7f175596a640 1 --2- 172.21.15.43:0/1196339086 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f173c0afe90 0x7f173c0b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.095+0000 7f175ea7d740 1 Processor -- start 2024-04-25T10:33:45.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.095+0000 7f175596a640 1 --2- 172.21.15.43:0/1196339086 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f173c0afe90 0x7f173c0b23a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f175007a310 tx=0x7f174c005f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.095+0000 7f175ea7d740 1 -- start start 2024-04-25T10:33:45.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.096+0000 7f175ea7d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c6f390 0x5630c700fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.096+0000 7f175ea7d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c464d0 0x5630c700ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.096+0000 7f175ea7d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c4e6b0 0x5630c701c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.096+0000 7f175ea7d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5630c6f75d80 con 0x5630c6c464d0 2024-04-25T10:33:45.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.096+0000 7f175ea7d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5630c6f2f8b0 con 0x5630c6c6f390 2024-04-25T10:33:45.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.096+0000 7f175ea7d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5630c6f71d40 con 0x5630c6c4e6b0 2024-04-25T10:33:45.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.096+0000 7f1755169640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c464d0 0x5630c700ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.096+0000 7f1755169640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c464d0 0x5630c700ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43138/0 (socket says 172.21.15.43:43138) 2024-04-25T10:33:45.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.096+0000 7f1755169640 1 -- 172.21.15.43:0/860989463 learned_addr learned my addr 172.21.15.43:0/860989463 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:45.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.096+0000 7f175596a640 1 --2- 172.21.15.43:0/860989463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c6f390 0x5630c700fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.096+0000 7f175616b640 1 --2- 172.21.15.43:0/860989463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c4e6b0 0x5630c701c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.096+0000 7f1755169640 1 -- 172.21.15.43:0/860989463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c4e6b0 msgr2=0x5630c701c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:45.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.096+0000 7f1755169640 1 --2- 172.21.15.43:0/860989463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c4e6b0 0x5630c701c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.096+0000 7f1755169640 1 -- 172.21.15.43:0/860989463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c6f390 msgr2=0x5630c700fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:45.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.096+0000 7f1755169640 1 --2- 172.21.15.43:0/860989463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c6f390 0x5630c700fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.096+0000 7f1755169640 1 -- 172.21.15.43:0/860989463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630c6f00a40 con 0x5630c6c464d0 2024-04-25T10:33:45.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.096+0000 7f175616b640 1 --2- 172.21.15.43:0/860989463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c4e6b0 0x5630c701c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:45.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.097+0000 7f1755169640 1 --2- 172.21.15.43:0/860989463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c464d0 0x5630c700ffa0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f17400c6d70 tx=0x7f1740000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ad130ce0c78f01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.097+0000 7f173bfff640 1 -- 172.21.15.43:0/860989463 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1740049040 con 0x5630c6c464d0 2024-04-25T10:33:45.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.097+0000 7f173bfff640 1 -- 172.21.15.43:0/860989463 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17400cb370 con 0x5630c6c464d0 2024-04-25T10:33:45.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.097+0000 7f173bfff640 1 -- 172.21.15.43:0/860989463 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17400cfbf0 con 0x5630c6c464d0 2024-04-25T10:33:45.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.098+0000 7f175ea7d740 1 -- 172.21.15.43:0/860989463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c464d0 msgr2=0x5630c700ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.098+0000 7f175ea7d740 1 --2- 172.21.15.43:0/860989463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c464d0 0x5630c700ffa0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f17400c6d70 tx=0x7f1740000bc0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.098+0000 7f175ea7d740 1 -- 172.21.15.43:0/860989463 shutdown_connections 2024-04-25T10:33:45.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.098+0000 7f175ea7d740 1 --2- 172.21.15.43:0/860989463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c4e6b0 0x5630c701c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.098+0000 7f175ea7d740 1 --2- 172.21.15.43:0/860989463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c464d0 0x5630c700ffa0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.098+0000 7f175ea7d740 1 --2- 172.21.15.43:0/860989463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c6f390 0x5630c700fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.098+0000 7f175ea7d740 1 -- 172.21.15.43:0/860989463 >> 172.21.15.43:0/860989463 conn(0x5630c6c634e0 msgr2=0x5630c700c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:45.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.098+0000 7f175ea7d740 1 -- 172.21.15.43:0/860989463 shutdown_connections 2024-04-25T10:33:45.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.099+0000 7f175ea7d740 1 -- 172.21.15.43:0/860989463 wait complete. 2024-04-25T10:33:45.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.099+0000 7f175ea7d740 1 -- start start 2024-04-25T10:33:45.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.099+0000 7f175ea7d740 1 Processor -- start 2024-04-25T10:33:45.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.100+0000 7f175ea7d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c464d0 0x5630c701bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.100+0000 7f175ea7d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c4e6b0 0x5630c701c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.100+0000 7f175ea7d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c6f390 0x5630c701a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.100+0000 7f175616b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c6f390 0x5630c701a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.100+0000 7f175616b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c6f390 0x5630c701a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43154/0 (socket says 172.21.15.43:43154) 2024-04-25T10:33:45.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.100+0000 7f175616b640 1 -- 172.21.15.43:0/3857300989 learned_addr learned my addr 172.21.15.43:0/3857300989 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:45.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.100+0000 7f1755169640 1 --2- 172.21.15.43:0/3857300989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c4e6b0 0x5630c701c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.100+0000 7f175ea7d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5630c6f75d80 con 0x5630c6c6f390 2024-04-25T10:33:45.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.100+0000 7f175ea7d740 1 -- 172.21.15.43:0/3857300989 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5630c6efcd10 con 0x5630c6c4e6b0 2024-04-25T10:33:45.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.100+0000 7f175ea7d740 1 -- 172.21.15.43:0/3857300989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5630c7021240 con 0x5630c6c464d0 2024-04-25T10:33:45.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.100+0000 7f175596a640 1 --2- 172.21.15.43:0/3857300989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c464d0 0x5630c701bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.101+0000 7f175616b640 1 -- 172.21.15.43:0/3857300989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c464d0 msgr2=0x5630c701bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:45.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.101+0000 7f175616b640 1 --2- 172.21.15.43:0/3857300989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c464d0 0x5630c701bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.101+0000 7f175616b640 1 -- 172.21.15.43:0/3857300989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c4e6b0 msgr2=0x5630c701c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:45.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.101+0000 7f175616b640 1 --2- 172.21.15.43:0/3857300989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c4e6b0 0x5630c701c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.101+0000 7f175616b640 1 -- 172.21.15.43:0/3857300989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630c6d517d0 con 0x5630c6c6f390 2024-04-25T10:33:45.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.101+0000 7f1755169640 1 --2- 172.21.15.43:0/3857300989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c4e6b0 0x5630c701c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:45.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.101+0000 7f175616b640 1 --2- 172.21.15.43:0/3857300989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c6f390 0x5630c701a500 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1748011610 tx=0x7f1748011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5300902d47f25164 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.101+0000 7f17397fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:45.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.101+0000 7f175ea7d740 10 client.0 _ll_register_callbacks cb 0x5630c6f29210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:45.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.101+0000 7f175ea7d740 1 -- 172.21.15.43:0/3857300989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17400cb370 con 0x5630c6c6f390 2024-04-25T10:33:45.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.101+0000 7f17397fa640 1 -- 172.21.15.43:0/3857300989 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174800fa70 con 0x5630c6c6f390 2024-04-25T10:33:45.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.101+0000 7f17397fa640 1 -- 172.21.15.43:0/3857300989 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f174800fc10 con 0x5630c6c6f390 2024-04-25T10:33:45.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.101+0000 7f17397fa640 1 -- 172.21.15.43:0/3857300989 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f174800fe10 con 0x5630c6c6f390 2024-04-25T10:33:45.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.101+0000 7f175ea7d740 10 client.15840 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:45.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.101+0000 7f175ea7d740 1 -- 172.21.15.43:0/3857300989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({fsmap=24}) -- 0x5630c6c873f0 con 0x5630c6c6f390 2024-04-25T10:33:45.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.102+0000 7f17397fa640 1 -- 172.21.15.43:0/3857300989 <== mon.0 v2:172.21.15.43:3300/0 4 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f174800f040 con 0x5630c6c6f390 2024-04-25T10:33:45.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.102+0000 7f175ea7d740 10 client.15840 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:45.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.102+0000 7f175ea7d740 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:45.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.102+0000 7f175ea7d740 20 client.15840 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.102+0000 7f175ea7d740 1 --2- 172.21.15.43:0/3857300989 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5630c6c2df60 0x5630c7030620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.102+0000 7f175ea7d740 4 client.15840 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:33:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.102+0000 7f175ea7d740 1 -- 172.21.15.43:0/3857300989 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f17400cb370 con 0x5630c6c2df60 2024-04-25T10:33:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.102+0000 7f175596a640 1 --2- 172.21.15.43:0/3857300989 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5630c6c2df60 0x5630c7030620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.103+0000 7f175596a640 1 --2- 172.21.15.43:0/3857300989 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5630c6c2df60 0x5630c7030620 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3d4774f347af122 server_cookie=ab9b3b300a2b6917 in_seq=0 out_seq=0 2024-04-25T10:33:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.103+0000 7f17397fa640 10 client.15840 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.103+0000 7f17397fa640 1 -- 172.21.15.43:0/3857300989 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f17400cb370 con 0x5630c6c2df60 2024-04-25T10:33:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.103+0000 7f17397fa640 10 client.15840 handle_command_reply: tid=0 2024-04-25T10:33:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.103+0000 7f175ea7d740 1 client.15840 shutdown 2024-04-25T10:33:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.104+0000 7f175ea7d740 1 -- 172.21.15.43:0/3857300989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c6f390 msgr2=0x5630c701a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.104+0000 7f175ea7d740 1 --2- 172.21.15.43:0/3857300989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c6f390 0x5630c701a500 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1748011610 tx=0x7f1748011ad0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.104+0000 7f175ea7d740 1 -- 172.21.15.43:0/3857300989 shutdown_connections 2024-04-25T10:33:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.104+0000 7f175ea7d740 1 --2- 172.21.15.43:0/3857300989 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5630c6c2df60 0x5630c7030620 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.104+0000 7f17397fa640 0 client.15840 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.104+0000 7f175ea7d740 1 --2- 172.21.15.43:0/3857300989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c4e6b0 0x5630c701c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.104+0000 7f175ea7d740 1 --2- 172.21.15.43:0/3857300989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c6f390 0x5630c701a500 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.104+0000 7f175ea7d740 1 --2- 172.21.15.43:0/3857300989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c464d0 0x5630c701bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.104+0000 7f175ea7d740 1 -- 172.21.15.43:0/3857300989 >> 172.21.15.43:0/3857300989 conn(0x5630c6c634e0 msgr2=0x5630c7011780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:45.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.104+0000 7f175ea7d740 1 -- 172.21.15.43:0/3857300989 shutdown_connections 2024-04-25T10:33:45.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.104+0000 7f175ea7d740 1 -- 172.21.15.43:0/3857300989 wait complete. 2024-04-25T10:33:45.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.104+0000 7f175ea7d740 20 client.15840 trim_cache size 0 max 16384 2024-04-25T10:33:45.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.140+0000 7f175ea7d740 1 Processor -- start 2024-04-25T10:33:45.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.141+0000 7f175ea7d740 1 -- start start 2024-04-25T10:33:45.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.141+0000 7f175ea7d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c2df60 0x5630c700fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.141+0000 7f175ea7d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c464d0 0x5630c701c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.141+0000 7f175ea7d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c4e6b0 0x5630c7035e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.141+0000 7f175ea7d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17501b0080 con 0x5630c6c2df60 2024-04-25T10:33:45.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.141+0000 7f175ea7d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17501ad950 con 0x5630c6c4e6b0 2024-04-25T10:33:45.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.141+0000 7f175ea7d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f17501ab7c0 con 0x5630c6c464d0 2024-04-25T10:33:45.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.141+0000 7f175596a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c2df60 0x5630c700fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.141+0000 7f175596a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c2df60 0x5630c700fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43168/0 (socket says 172.21.15.43:43168) 2024-04-25T10:33:45.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.141+0000 7f175596a640 1 -- 172.21.15.43:0/611832639 learned_addr learned my addr 172.21.15.43:0/611832639 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:45.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.141+0000 7f1755169640 1 --2- 172.21.15.43:0/611832639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c464d0 0x5630c701c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.141+0000 7f175616b640 1 --2- 172.21.15.43:0/611832639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c4e6b0 0x5630c7035e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.142+0000 7f175596a640 1 -- 172.21.15.43:0/611832639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c464d0 msgr2=0x5630c701c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:45.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.142+0000 7f175596a640 1 --2- 172.21.15.43:0/611832639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c464d0 0x5630c701c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.142+0000 7f175596a640 1 -- 172.21.15.43:0/611832639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c4e6b0 msgr2=0x5630c7035e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:45.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.142+0000 7f175596a640 1 --2- 172.21.15.43:0/611832639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c4e6b0 0x5630c7035e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.142+0000 7f175596a640 1 -- 172.21.15.43:0/611832639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630c6c470f0 con 0x5630c6c2df60 2024-04-25T10:33:45.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.142+0000 7f1755169640 1 --2- 172.21.15.43:0/611832639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c464d0 0x5630c701c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:45.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.142+0000 7f175616b640 1 --2- 172.21.15.43:0/611832639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c4e6b0 0x5630c7035e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:45.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.142+0000 7f175596a640 1 --2- 172.21.15.43:0/611832639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c2df60 0x5630c700fe60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f174c009e00 tx=0x7f174c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c65ca614c37bea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.142+0000 7f173b7fe640 1 -- 172.21.15.43:0/611832639 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c030070 con 0x5630c6c2df60 2024-04-25T10:33:45.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.142+0000 7f173b7fe640 1 -- 172.21.15.43:0/611832639 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f174c01dc80 con 0x5630c6c2df60 2024-04-25T10:33:45.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.142+0000 7f173b7fe640 1 -- 172.21.15.43:0/611832639 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c02b970 con 0x5630c6c2df60 2024-04-25T10:33:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.143+0000 7f175ea7d740 1 -- 172.21.15.43:0/611832639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c2df60 msgr2=0x5630c700fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.143+0000 7f175ea7d740 1 --2- 172.21.15.43:0/611832639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c2df60 0x5630c700fe60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f174c009e00 tx=0x7f174c014000 comp rx=0 tx=0).stop 2024-04-25T10:33:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.143+0000 7f175ea7d740 1 -- 172.21.15.43:0/611832639 shutdown_connections 2024-04-25T10:33:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.143+0000 7f175ea7d740 1 --2- 172.21.15.43:0/611832639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c4e6b0 0x5630c7035e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.143+0000 7f175ea7d740 1 --2- 172.21.15.43:0/611832639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c464d0 0x5630c701c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.143+0000 7f175ea7d740 1 --2- 172.21.15.43:0/611832639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c2df60 0x5630c700fe60 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.143+0000 7f175ea7d740 1 -- 172.21.15.43:0/611832639 >> 172.21.15.43:0/611832639 conn(0x5630c6c634e0 msgr2=0x5630c701bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.143+0000 7f175ea7d740 1 -- 172.21.15.43:0/611832639 shutdown_connections 2024-04-25T10:33:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.143+0000 7f175ea7d740 1 -- 172.21.15.43:0/611832639 wait complete. 2024-04-25T10:33:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.144+0000 7f175ea7d740 1 -- start start 2024-04-25T10:33:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.144+0000 7f175ea7d740 1 Processor -- start 2024-04-25T10:33:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.144+0000 7f175ea7d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c2df60 0x5630c70234b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.144+0000 7f175ea7d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c464d0 0x5630c70239f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.144+0000 7f175ea7d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c4e6b0 0x5630c70969d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.144+0000 7f175ea7d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17501abe10 con 0x5630c6c4e6b0 2024-04-25T10:33:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.144+0000 7f175ea7d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17501ad280 con 0x5630c6c2df60 2024-04-25T10:33:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.144+0000 7f175ea7d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f17501ac4e0 con 0x5630c6c464d0 2024-04-25T10:33:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.144+0000 7f175616b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c4e6b0 0x5630c70969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.144+0000 7f175616b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c4e6b0 0x5630c70969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43180/0 (socket says 172.21.15.43:43180) 2024-04-25T10:33:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.144+0000 7f175616b640 1 -- 172.21.15.43:0/4223956511 learned_addr learned my addr 172.21.15.43:0/4223956511 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.145+0000 7f175ea7d740 10 client.0 _ll_register_callbacks cb 0x7f1748009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.145+0000 7f175596a640 1 --2- 172.21.15.43:0/4223956511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c2df60 0x5630c70234b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.145+0000 7f1755169640 1 --2- 172.21.15.43:0/4223956511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c464d0 0x5630c70239f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.145+0000 7f175616b640 1 -- 172.21.15.43:0/4223956511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c464d0 msgr2=0x5630c70239f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.145+0000 7f175616b640 1 --2- 172.21.15.43:0/4223956511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c464d0 0x5630c70239f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.145+0000 7f175616b640 1 -- 172.21.15.43:0/4223956511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c2df60 msgr2=0x5630c70234b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.145+0000 7f175616b640 1 --2- 172.21.15.43:0/4223956511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c2df60 0x5630c70234b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.145+0000 7f175616b640 1 -- 172.21.15.43:0/4223956511 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630c6f364b0 con 0x5630c6c4e6b0 2024-04-25T10:33:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.145+0000 7f175616b640 1 --2- 172.21.15.43:0/4223956511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c4e6b0 0x5630c70969d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1748011610 tx=0x7f1748004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f882a98432879ed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.145+0000 7f17397fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.145+0000 7f17397fa640 1 -- 172.21.15.43:0/4223956511 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1748003970 con 0x5630c6c4e6b0 2024-04-25T10:33:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.145+0000 7f175ea7d740 1 -- 172.21.15.43:0/4223956511 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f174c01dc80 con 0x5630c6c4e6b0 2024-04-25T10:33:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.146+0000 7f17397fa640 1 -- 172.21.15.43:0/4223956511 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f1748003b10 con 0x5630c6c4e6b0 2024-04-25T10:33:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.146+0000 7f17397fa640 1 -- 172.21.15.43:0/4223956511 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f174803c050 con 0x5630c6c4e6b0 2024-04-25T10:33:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.146+0000 7f175ea7d740 10 client.15846 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.146+0000 7f175ea7d740 1 -- 172.21.15.43:0/4223956511 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5630c7097dc0 con 0x5630c6c4e6b0 2024-04-25T10:33:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.146+0000 7f17397fa640 1 -- 172.21.15.43:0/4223956511 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f174c01dc80 con 0x5630c6c4e6b0 2024-04-25T10:33:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.146+0000 7f17397fa640 1 -- 172.21.15.43:0/4223956511 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1748004030 con 0x5630c6c4e6b0 2024-04-25T10:33:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.146+0000 7f175ea7d740 10 client.15846 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.146+0000 7f175ea7d740 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:45.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.146+0000 7f175ea7d740 20 client.15846 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:45.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.146+0000 7f175ea7d740 1 --2- 172.21.15.43:0/4223956511 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5630c6c6f390 0x5630c709a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.146+0000 7f175ea7d740 4 client.15846 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:33:45.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.146+0000 7f175ea7d740 1 -- 172.21.15.43:0/4223956511 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5630c6f83990 con 0x5630c6c6f390 2024-04-25T10:33:45.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.147+0000 7f175596a640 1 --2- 172.21.15.43:0/4223956511 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5630c6c6f390 0x5630c709a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.147+0000 7f17397fa640 10 client.15846 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:45.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.147+0000 7f175596a640 1 --2- 172.21.15.43:0/4223956511 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5630c6c6f390 0x5630c709a560 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f175a200fdf20da server_cookie=1632cb2568a341c3 in_seq=0 out_seq=0 2024-04-25T10:33:45.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.147+0000 7f17397fa640 1 -- 172.21.15.43:0/4223956511 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5630c6f83990 con 0x5630c6c6f390 2024-04-25T10:33:45.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.147+0000 7f17397fa640 10 client.15846 handle_command_reply: tid=0 2024-04-25T10:33:45.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.147+0000 7f175ea7d740 1 client.15846 shutdown 2024-04-25T10:33:45.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.147+0000 7f175ea7d740 1 -- 172.21.15.43:0/4223956511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c4e6b0 msgr2=0x5630c70969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.148+0000 7f175ea7d740 1 --2- 172.21.15.43:0/4223956511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c4e6b0 0x5630c70969d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1748011610 tx=0x7f1748004a40 comp rx=0 tx=0).stop 2024-04-25T10:33:45.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.148+0000 7f175ea7d740 1 -- 172.21.15.43:0/4223956511 shutdown_connections 2024-04-25T10:33:45.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.148+0000 7f175ea7d740 1 --2- 172.21.15.43:0/4223956511 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5630c6c6f390 0x5630c709a560 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.148+0000 7f175ea7d740 1 --2- 172.21.15.43:0/4223956511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c6c4e6b0 0x5630c70969d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.148+0000 7f175ea7d740 1 --2- 172.21.15.43:0/4223956511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630c6c464d0 0x5630c70239f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.148+0000 7f17397fa640 0 client.15846 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:45.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.148+0000 7f175ea7d740 1 --2- 172.21.15.43:0/4223956511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630c6c2df60 0x5630c70234b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.148+0000 7f175ea7d740 1 -- 172.21.15.43:0/4223956511 >> 172.21.15.43:0/4223956511 conn(0x5630c6c634e0 msgr2=0x5630c701c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:45.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.148+0000 7f175ea7d740 1 -- 172.21.15.43:0/4223956511 shutdown_connections 2024-04-25T10:33:45.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.148+0000 7f175ea7d740 1 -- 172.21.15.43:0/4223956511 wait complete. 2024-04-25T10:33:45.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.148+0000 7f175ea7d740 20 client.15846 trim_cache size 0 max 16384 2024-04-25T10:33:45.154 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:33:45.155 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:33:45.155 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:33:45.155 INFO:teuthology.orchestra.run.smithi043.stdout: "3103385c-e046-4f88-8173-0f1fe7c66d1a": { 2024-04-25T10:33:45.155 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:33:45.155 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "3103385c-e046-4f88-8173-0f1fe7c66d1a", 2024-04-25T10:33:45.155 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:33:45.155 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:33:45.155 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:33:45.155 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:33:45.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.148+0000 7f1757bce640 1 -- 172.21.15.43:0/1196339086 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f173c0afe90 msgr2=0x7f173c0b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.148+0000 7f1757bce640 1 --2- 172.21.15.43:0/1196339086 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f173c0afe90 0x7f173c0b23a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f175007a310 tx=0x7f174c005f90 comp rx=0 tx=0).stop 2024-04-25T10:33:45.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.148+0000 7f1757bce640 1 -- 172.21.15.43:0/1196339086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1750109b30 msgr2=0x7f175019ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.148+0000 7f1757bce640 1 --2- 172.21.15.43:0/1196339086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1750109b30 0x7f175019ace0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1740006010 tx=0x7f1740004060 comp rx=0 tx=0).stop 2024-04-25T10:33:45.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.148+0000 7f1757bce640 1 -- 172.21.15.43:0/1196339086 shutdown_connections 2024-04-25T10:33:45.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.149+0000 7f1757bce640 1 --2- 172.21.15.43:0/1196339086 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f173c0afe90 0x7f173c0b23a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.149+0000 7f1757bce640 1 --2- 172.21.15.43:0/1196339086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1750109fd0 0x7f175019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.149+0000 7f1757bce640 1 --2- 172.21.15.43:0/1196339086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1750109b30 0x7f175019ace0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.149+0000 7f1757bce640 1 --2- 172.21.15.43:0/1196339086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17501042d0 0x7f175019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.149+0000 7f1757bce640 1 -- 172.21.15.43:0/1196339086 >> 172.21.15.43:0/1196339086 conn(0x7f1750079280 msgr2=0x7f17501022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:45.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.149+0000 7f1757bce640 1 -- 172.21.15.43:0/1196339086 shutdown_connections 2024-04-25T10:33:45.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:33:45.149+0000 7f1757bce640 1 -- 172.21.15.43:0/1196339086 wait complete. 2024-04-25T10:33:45.162 INFO:tasks.cephfs.filesystem:scrub status for tag:3103385c-e046-4f88-8173-0f1fe7c66d1a - {'path': '/', 'tag': '3103385c-e046-4f88-8173-0f1fe7c66d1a', 'options': 'recursive,force'} 2024-04-25T10:33:46.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:45 smithi077 ceph-mon[32772]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T10:33:46.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:45 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:33:46.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:45 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:33:46.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:45 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:33:46.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:45 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:33:46.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:45 smithi164 ceph-mon[32713]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T10:33:46.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:45 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:33:46.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:45 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:33:46.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:45 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:33:46.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:45 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:33:46.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:45 smithi043 ceph-mon[29478]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T10:33:46.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:45 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:33:46.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:45 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:33:46.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:45 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:33:46.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:45 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:33:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.001+0000 7f9b603f3640 1 -- 172.21.15.43:0/4082940524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b58108ef0 msgr2=0x7f9b58102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.001+0000 7f9b603f3640 1 --2- 172.21.15.43:0/4082940524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b58108ef0 0x7f9b58102b40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009860 tx=0x7f9b4802a340 comp rx=0 tx=0).stop 2024-04-25T10:33:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.001+0000 7f9b603f3640 1 -- 172.21.15.43:0/4082940524 shutdown_connections 2024-04-25T10:33:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.001+0000 7f9b603f3640 1 --2- 172.21.15.43:0/4082940524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b58108ef0 0x7f9b58102b40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.001+0000 7f9b603f3640 1 --2- 172.21.15.43:0/4082940524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b581031b0 0x7f9b5810f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.001+0000 7f9b603f3640 1 --2- 172.21.15.43:0/4082940524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b58108a50 0x7f9b58102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.001+0000 7f9b603f3640 1 -- 172.21.15.43:0/4082940524 >> 172.21.15.43:0/4082940524 conn(0x7f9b580fe260 msgr2=0x7f9b58100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.002+0000 7f9b603f3640 1 -- 172.21.15.43:0/4082940524 shutdown_connections 2024-04-25T10:33:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.002+0000 7f9b603f3640 1 -- 172.21.15.43:0/4082940524 wait complete. 2024-04-25T10:33:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.003+0000 7f9b603f3640 1 Processor -- start 2024-04-25T10:33:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.003+0000 7f9b603f3640 1 -- start start 2024-04-25T10:33:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.003+0000 7f9b603f3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b581031b0 0x7f9b58199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.003+0000 7f9b603f3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b58108a50 0x7f9b58199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.003+0000 7f9b603f3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b58108ef0 0x7f9b58199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.004+0000 7f9b603f3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9b58112c70 con 0x7f9b58108ef0 2024-04-25T10:33:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.004+0000 7f9b603f3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9b58112af0 con 0x7f9b581031b0 2024-04-25T10:33:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.004+0000 7f9b5e990640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b58108ef0 0x7f9b58199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.004+0000 7f9b5e990640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b58108ef0 0x7f9b58199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43188/0 (socket says 172.21.15.43:43188) 2024-04-25T10:33:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.004+0000 7f9b5e18f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b581031b0 0x7f9b58199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.004+0000 7f9b5e990640 1 -- 172.21.15.43:0/3974864555 learned_addr learned my addr 172.21.15.43:0/3974864555 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.004+0000 7f9b5d98e640 1 --2- 172.21.15.43:0/3974864555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b58108a50 0x7f9b58199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.004+0000 7f9b603f3640 1 -- 172.21.15.43:0/3974864555 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9b58112df0 con 0x7f9b58108a50 2024-04-25T10:33:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.004+0000 7f9b5e990640 1 -- 172.21.15.43:0/3974864555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b58108a50 msgr2=0x7f9b58199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.004+0000 7f9b5e990640 1 --2- 172.21.15.43:0/3974864555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b58108a50 0x7f9b58199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.004+0000 7f9b5e990640 1 -- 172.21.15.43:0/3974864555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b581031b0 msgr2=0x7f9b58199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.004+0000 7f9b5e990640 1 --2- 172.21.15.43:0/3974864555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b581031b0 0x7f9b58199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.004+0000 7f9b5e990640 1 -- 172.21.15.43:0/3974864555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b581a8af0 con 0x7f9b58108ef0 2024-04-25T10:33:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.004+0000 7f9b5e18f640 1 --2- 172.21.15.43:0/3974864555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b581031b0 0x7f9b58199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.004+0000 7f9b5e990640 1 --2- 172.21.15.43:0/3974864555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b58108ef0 0x7f9b58199ec0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000efc0 tx=0x7f9b5000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.005+0000 7f9b4f7fe640 1 -- 172.21.15.43:0/3974864555 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b50003970 con 0x7f9b58108ef0 2024-04-25T10:33:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.005+0000 7f9b4f7fe640 1 -- 172.21.15.43:0/3974864555 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b50004020 con 0x7f9b58108ef0 2024-04-25T10:33:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.005+0000 7f9b603f3640 1 -- 172.21.15.43:0/3974864555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b581a8de0 con 0x7f9b58108ef0 2024-04-25T10:33:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.005+0000 7f9b603f3640 1 -- 172.21.15.43:0/3974864555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b581a9190 con 0x7f9b58108ef0 2024-04-25T10:33:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.006+0000 7f9b4f7fe640 1 -- 172.21.15.43:0/3974864555 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5000b940 con 0x7f9b58108ef0 2024-04-25T10:33:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.009+0000 7f9b6737e740 1 Processor -- start 2024-04-25T10:33:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.009+0000 7f9b4f7fe640 1 -- 172.21.15.43:0/3974864555 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 147804+0+0 (secure 0 0 0) 0x7f9b5001d020 con 0x7f9b58108ef0 2024-04-25T10:33:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.009+0000 7f9b6737e740 1 -- start start 2024-04-25T10:33:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.010+0000 7f9b6737e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeeb1b0 0x5652ce28aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.010+0000 7f9b6737e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdec22f0 0x5652ce28b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.010+0000 7f9b6737e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652cdeca4d0 0x5652ce28b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.010+0000 7f9b6737e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5652ce1f1460 con 0x5652cdeeb1b0 2024-04-25T10:33:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.010+0000 7f9b6737e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5652ce1ab590 con 0x5652cdeca4d0 2024-04-25T10:33:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.010+0000 7f9b6737e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5652ce1ed420 con 0x5652cdec22f0 2024-04-25T10:33:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.010+0000 7f9b5d98e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeeb1b0 0x5652ce28aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.010+0000 7f9b5d98e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeeb1b0 0x5652ce28aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43200/0 (socket says 172.21.15.43:43200) 2024-04-25T10:33:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.010+0000 7f9b5d98e640 1 -- 172.21.15.43:0/3012714466 learned_addr learned my addr 172.21.15.43:0/3012714466 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.010+0000 7f9b5e18f640 1 --2- 172.21.15.43:0/3012714466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdec22f0 0x5652ce28b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.010+0000 7f9b4f7fe640 1 --2- 172.21.15.43:0/3974864555 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9b440afe80 0x7f9b440b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.010+0000 7f9b5e18f640 1 -- 172.21.15.43:0/3012714466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652cdeca4d0 msgr2=0x5652ce28b570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.011+0000 7f9b5e18f640 1 --2- 172.21.15.43:0/3012714466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652cdeca4d0 0x5652ce28b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.011+0000 7f9b5e18f640 1 -- 172.21.15.43:0/3012714466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeeb1b0 msgr2=0x5652ce28aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.011+0000 7f9b5e18f640 1 --2- 172.21.15.43:0/3012714466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeeb1b0 0x5652ce28aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.011+0000 7f9b5e18f640 1 -- 172.21.15.43:0/3012714466 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652ce1b80d0 con 0x5652cdec22f0 2024-04-25T10:33:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.011+0000 7f9b5d98e640 1 --2- 172.21.15.43:0/3012714466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeeb1b0 0x5652ce28aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.011+0000 7f9b5e990640 1 --2- 172.21.15.43:0/3974864555 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9b440afe80 0x7f9b440b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.011+0000 7f9b5e18f640 1 --2- 172.21.15.43:0/3012714466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdec22f0 0x5652ce28b030 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9b58103740 tx=0x7f9b540118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b20a651848bf8666 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.011+0000 7f9b4cbfe640 1 -- 172.21.15.43:0/3012714466 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5401e070 con 0x5652cdec22f0 2024-04-25T10:33:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.011+0000 7f9b4f7fe640 1 -- 172.21.15.43:0/3974864555 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f9b500df0a0 con 0x7f9b58108ef0 2024-04-25T10:33:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.011+0000 7f9b4cbfe640 1 -- 172.21.15.43:0/3012714466 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b54004310 con 0x5652cdec22f0 2024-04-25T10:33:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.012+0000 7f9b6737e740 1 -- 172.21.15.43:0/3012714466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdec22f0 msgr2=0x5652ce28b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.012+0000 7f9b5e990640 1 --2- 172.21.15.43:0/3974864555 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9b440afe80 0x7f9b440b2390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9b500dc6e0 tx=0x7f9b500e2920 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.012+0000 7f9b6737e740 1 --2- 172.21.15.43:0/3012714466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdec22f0 0x5652ce28b030 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9b58103740 tx=0x7f9b540118d0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.012+0000 7f9b4cbfe640 1 -- 172.21.15.43:0/3012714466 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5400f8a0 con 0x5652cdec22f0 2024-04-25T10:33:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.012+0000 7f9b6737e740 1 -- 172.21.15.43:0/3012714466 shutdown_connections 2024-04-25T10:33:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.012+0000 7f9b6737e740 1 --2- 172.21.15.43:0/3012714466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652cdeca4d0 0x5652ce28b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.012+0000 7f9b6737e740 1 --2- 172.21.15.43:0/3012714466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdec22f0 0x5652ce28b030 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.012+0000 7f9b6737e740 1 --2- 172.21.15.43:0/3012714466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeeb1b0 0x5652ce28aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.012+0000 7f9b6737e740 1 -- 172.21.15.43:0/3012714466 >> 172.21.15.43:0/3012714466 conn(0x5652cdedf300 msgr2=0x5652ce2872a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.013+0000 7f9b6737e740 1 -- 172.21.15.43:0/3012714466 shutdown_connections 2024-04-25T10:33:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.013+0000 7f9b6737e740 1 -- 172.21.15.43:0/3012714466 wait complete. 2024-04-25T10:33:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.013+0000 7f9b6737e740 1 -- start start 2024-04-25T10:33:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.013+0000 7f9b6737e740 1 Processor -- start 2024-04-25T10:33:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.014+0000 7f9b6737e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdec22f0 0x5652ce295110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.014+0000 7f9b6737e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeca4d0 0x5652ce295650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.014+0000 7f9b6737e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652cdeeb1b0 0x5652ce2a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.014+0000 7f9b6737e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5652ce297cc0 con 0x5652cdeca4d0 2024-04-25T10:33:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.014+0000 7f9b6737e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5652ce178b60 con 0x5652cdeeb1b0 2024-04-25T10:33:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.014+0000 7f9b6737e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5652ce297e40 con 0x5652cdec22f0 2024-04-25T10:33:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.014+0000 7f9b6737e740 10 client.0 _ll_register_callbacks cb 0x5652ce19d2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.014+0000 7f9b5e18f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeca4d0 0x5652ce295650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.014+0000 7f9b5e18f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeca4d0 0x5652ce295650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43208/0 (socket says 172.21.15.43:43208) 2024-04-25T10:33:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.014+0000 7f9b5e18f640 1 -- 172.21.15.43:0/2544967775 learned_addr learned my addr 172.21.15.43:0/2544967775 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.014+0000 7f9b5e18f640 1 -- 172.21.15.43:0/2544967775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdec22f0 msgr2=0x5652ce295110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.014+0000 7f9b5d98e640 1 --2- 172.21.15.43:0/2544967775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdec22f0 0x5652ce295110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.014+0000 7f9b5e18f640 1 --2- 172.21.15.43:0/2544967775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdec22f0 0x5652ce295110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.014+0000 7f9b5e18f640 1 -- 172.21.15.43:0/2544967775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652cdeeb1b0 msgr2=0x5652ce2a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.014+0000 7f9b5e18f640 1 --2- 172.21.15.43:0/2544967775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652cdeeb1b0 0x5652ce2a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.014+0000 7f9b5e18f640 1 -- 172.21.15.43:0/2544967775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652cdf03210 con 0x5652cdeca4d0 2024-04-25T10:33:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.015+0000 7f9b5e18f640 1 --2- 172.21.15.43:0/2544967775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeca4d0 0x5652ce295650 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400f580 tx=0x7f9b54003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e0122a7c136510 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.015+0000 7f9b41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.015+0000 7f9b41ffb640 1 -- 172.21.15.43:0/2544967775 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54027020 con 0x5652cdeca4d0 2024-04-25T10:33:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.015+0000 7f9b6737e740 1 -- 172.21.15.43:0/2544967775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b54004310 con 0x5652cdeca4d0 2024-04-25T10:33:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.015+0000 7f9b41ffb640 1 -- 172.21.15.43:0/2544967775 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f9b54005040 con 0x5652cdeca4d0 2024-04-25T10:33:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.015+0000 7f9b41ffb640 1 -- 172.21.15.43:0/2544967775 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9b54032050 con 0x5652cdeca4d0 2024-04-25T10:33:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.015+0000 7f9b6737e740 10 client.15858 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.015+0000 7f9b6737e740 1 -- 172.21.15.43:0/2544967775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5652cdfcf620 con 0x5652cdeca4d0 2024-04-25T10:33:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.015+0000 7f9b41ffb640 1 -- 172.21.15.43:0/2544967775 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b54004310 con 0x5652cdeca4d0 2024-04-25T10:33:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.016+0000 7f9b41ffb640 1 -- 172.21.15.43:0/2544967775 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f9b54004bf0 con 0x5652cdeca4d0 2024-04-25T10:33:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.017+0000 7f9b6737e740 10 client.15858 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.017+0000 7f9b6737e740 10 client.15858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.017+0000 7f9b6737e740 20 client.15858 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.017+0000 7f9b6737e740 1 --2- 172.21.15.43:0/2544967775 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5652cdea9d80 0x5652ce2a7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.017+0000 7f9b6737e740 4 client.15858 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:33:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.017+0000 7f9b6737e740 1 -- 172.21.15.43:0/2544967775 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652ce1ff070 con 0x5652cdea9d80 2024-04-25T10:33:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.017+0000 7f9b5d98e640 1 --2- 172.21.15.43:0/2544967775 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5652cdea9d80 0x5652ce2a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.018+0000 7f9b41ffb640 10 client.15858 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.018+0000 7f9b5d98e640 1 --2- 172.21.15.43:0/2544967775 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5652cdea9d80 0x5652ce2a7660 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1b4d07204f1ee23 server_cookie=ba7bae2531e053d1 in_seq=0 out_seq=0 2024-04-25T10:33:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.018+0000 7f9b41ffb640 1 -- 172.21.15.43:0/2544967775 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5652ce1ff070 con 0x5652cdea9d80 2024-04-25T10:33:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.018+0000 7f9b41ffb640 10 client.15858 handle_command_reply: tid=0 2024-04-25T10:33:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.019+0000 7f9b6737e740 1 client.15858 shutdown 2024-04-25T10:33:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.019+0000 7f9b6737e740 1 -- 172.21.15.43:0/2544967775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeca4d0 msgr2=0x5652ce295650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.019+0000 7f9b6737e740 1 --2- 172.21.15.43:0/2544967775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeca4d0 0x5652ce295650 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400f580 tx=0x7f9b54003c30 comp rx=0 tx=0).stop 2024-04-25T10:33:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.019+0000 7f9b6737e740 1 -- 172.21.15.43:0/2544967775 shutdown_connections 2024-04-25T10:33:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.019+0000 7f9b6737e740 1 --2- 172.21.15.43:0/2544967775 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5652cdea9d80 0x5652ce2a7660 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.019+0000 7f9b6737e740 1 --2- 172.21.15.43:0/2544967775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652cdeeb1b0 0x5652ce2a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.019+0000 7f9b6737e740 1 --2- 172.21.15.43:0/2544967775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeca4d0 0x5652ce295650 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.019+0000 7f9b6737e740 1 --2- 172.21.15.43:0/2544967775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdec22f0 0x5652ce295110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.019+0000 7f9b6737e740 1 -- 172.21.15.43:0/2544967775 >> 172.21.15.43:0/2544967775 conn(0x5652cdedf300 msgr2=0x5652ce28c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.019+0000 7f9b41ffb640 0 client.15858 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.019+0000 7f9b6737e740 1 -- 172.21.15.43:0/2544967775 shutdown_connections 2024-04-25T10:33:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.019+0000 7f9b6737e740 1 -- 172.21.15.43:0/2544967775 wait complete. 2024-04-25T10:33:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.019+0000 7f9b6737e740 20 client.15858 trim_cache size 0 max 16384 2024-04-25T10:33:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.050+0000 7f9b6737e740 1 Processor -- start 2024-04-25T10:33:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.050+0000 7f9b6737e740 1 -- start start 2024-04-25T10:33:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.050+0000 7f9b6737e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdec22f0 0x5652ce28b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.050+0000 7f9b6737e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdeca4d0 0x5652ce2946b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.050+0000 7f9b6737e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652cdeeb1b0 0x5652ce28e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.050+0000 7f9b5d98e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdec22f0 0x5652ce28b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.051+0000 7f9b6737e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9b581b4430 con 0x5652cdec22f0 2024-04-25T10:33:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.051+0000 7f9b5d98e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdec22f0 0x5652ce28b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43212/0 (socket says 172.21.15.43:43212) 2024-04-25T10:33:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.051+0000 7f9b5d98e640 1 -- 172.21.15.43:0/104794219 learned_addr learned my addr 172.21.15.43:0/104794219 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.051+0000 7f9b6737e740 1 -- 172.21.15.43:0/104794219 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9b581b1d00 con 0x5652cdeeb1b0 2024-04-25T10:33:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.051+0000 7f9b6737e740 1 -- 172.21.15.43:0/104794219 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9b581afb50 con 0x5652cdeca4d0 2024-04-25T10:33:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.051+0000 7f9b5e18f640 1 --2- 172.21.15.43:0/104794219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdeca4d0 0x5652ce2946b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.051+0000 7f9b5e18f640 1 -- 172.21.15.43:0/104794219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652cdeeb1b0 msgr2=0x5652ce28e850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.051+0000 7f9b5e18f640 1 --2- 172.21.15.43:0/104794219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652cdeeb1b0 0x5652ce28e850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.051+0000 7f9b5e18f640 1 -- 172.21.15.43:0/104794219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdec22f0 msgr2=0x5652ce28b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.051+0000 7f9b5e18f640 1 --2- 172.21.15.43:0/104794219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdec22f0 0x5652ce28b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.051+0000 7f9b5e18f640 1 -- 172.21.15.43:0/104794219 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652ce293450 con 0x5652cdeca4d0 2024-04-25T10:33:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.051+0000 7f9b5d98e640 1 --2- 172.21.15.43:0/104794219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdec22f0 0x5652ce28b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.051+0000 7f9b5e18f640 1 --2- 172.21.15.43:0/104794219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdeca4d0 0x5652ce2946b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400f760 tx=0x7f9b54009cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c6e6a9578761100 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.052+0000 7f9b43fff640 1 -- 172.21.15.43:0/104794219 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54027050 con 0x5652cdeca4d0 2024-04-25T10:33:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.052+0000 7f9b43fff640 1 -- 172.21.15.43:0/104794219 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b540054f0 con 0x5652cdeca4d0 2024-04-25T10:33:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.052+0000 7f9b43fff640 1 -- 172.21.15.43:0/104794219 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5401e040 con 0x5652cdeca4d0 2024-04-25T10:33:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.053+0000 7f9b6737e740 1 -- 172.21.15.43:0/104794219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdeca4d0 msgr2=0x5652ce2946b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.053+0000 7f9b6737e740 1 --2- 172.21.15.43:0/104794219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdeca4d0 0x5652ce2946b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400f760 tx=0x7f9b54009cc0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.053+0000 7f9b6737e740 1 -- 172.21.15.43:0/104794219 shutdown_connections 2024-04-25T10:33:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.053+0000 7f9b6737e740 1 --2- 172.21.15.43:0/104794219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652cdeeb1b0 0x5652ce28e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.053+0000 7f9b6737e740 1 --2- 172.21.15.43:0/104794219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdeca4d0 0x5652ce2946b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.053+0000 7f9b6737e740 1 --2- 172.21.15.43:0/104794219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdec22f0 0x5652ce28b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.053+0000 7f9b6737e740 1 -- 172.21.15.43:0/104794219 >> 172.21.15.43:0/104794219 conn(0x5652cdedf300 msgr2=0x5652ce296620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.053+0000 7f9b6737e740 1 -- 172.21.15.43:0/104794219 shutdown_connections 2024-04-25T10:33:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.053+0000 7f9b6737e740 1 -- 172.21.15.43:0/104794219 wait complete. 2024-04-25T10:33:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.053+0000 7f9b6737e740 1 -- start start 2024-04-25T10:33:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.054+0000 7f9b6737e740 1 Processor -- start 2024-04-25T10:33:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.054+0000 7f9b6737e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdec22f0 0x5652ce2980e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.054+0000 7f9b6737e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeca4d0 0x5652ce298620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.054+0000 7f9b6737e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652cdeeb1b0 0x5652ce298b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.054+0000 7f9b6737e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9b581b01c0 con 0x5652cdeca4d0 2024-04-25T10:33:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.054+0000 7f9b6737e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9b581b1630 con 0x5652cdeeb1b0 2024-04-25T10:33:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.054+0000 7f9b6737e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9b581b0890 con 0x5652cdec22f0 2024-04-25T10:33:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.054+0000 7f9b6737e740 10 client.0 _ll_register_callbacks cb 0x7f9b5401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.054+0000 7f9b5e18f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeca4d0 0x5652ce298620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.054+0000 7f9b5e18f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeca4d0 0x5652ce298620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43220/0 (socket says 172.21.15.43:43220) 2024-04-25T10:33:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.054+0000 7f9b5e18f640 1 -- 172.21.15.43:0/2266923796 learned_addr learned my addr 172.21.15.43:0/2266923796 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:33:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.054+0000 7f9b5d98e640 1 --2- 172.21.15.43:0/2266923796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdec22f0 0x5652ce2980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.055+0000 7f9b5d98e640 1 --2- 172.21.15.43:0/2266923796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652cdeeb1b0 0x5652ce298b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.055+0000 7f9b5e18f640 1 -- 172.21.15.43:0/2266923796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdec22f0 msgr2=0x5652ce2980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.055+0000 7f9b5e18f640 1 --2- 172.21.15.43:0/2266923796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdec22f0 0x5652ce2980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.055+0000 7f9b5e18f640 1 -- 172.21.15.43:0/2266923796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652cdeeb1b0 msgr2=0x5652ce298b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.055+0000 7f9b5e18f640 1 --2- 172.21.15.43:0/2266923796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652cdeeb1b0 0x5652ce298b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.055+0000 7f9b5e18f640 1 -- 172.21.15.43:0/2266923796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652ce30da30 con 0x5652cdeca4d0 2024-04-25T10:33:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.055+0000 7f9b5e18f640 1 --2- 172.21.15.43:0/2266923796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeca4d0 0x5652ce298620 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9b54005890 tx=0x7f9b54003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc9d256b6fc2411 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.055+0000 7f9b41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:33:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.055+0000 7f9b41ffb640 1 -- 172.21.15.43:0/2266923796 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5401e070 con 0x5652cdeca4d0 2024-04-25T10:33:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.055+0000 7f9b41ffb640 1 -- 172.21.15.43:0/2266923796 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f9b54005200 con 0x5652cdeca4d0 2024-04-25T10:33:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.055+0000 7f9b6737e740 1 -- 172.21.15.43:0/2266923796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b540054f0 con 0x5652cdeca4d0 2024-04-25T10:33:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.056+0000 7f9b41ffb640 1 -- 172.21.15.43:0/2266923796 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9b54027040 con 0x5652cdeca4d0 2024-04-25T10:33:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.056+0000 7f9b6737e740 10 client.15864 fetch_fsmap learned FSMap version 24 2024-04-25T10:33:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.056+0000 7f9b6737e740 1 -- 172.21.15.43:0/2266923796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5652ce30ecc0 con 0x5652cdeca4d0 2024-04-25T10:33:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.056+0000 7f9b41ffb640 1 -- 172.21.15.43:0/2266923796 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b540054f0 con 0x5652cdeca4d0 2024-04-25T10:33:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.057+0000 7f9b41ffb640 1 -- 172.21.15.43:0/2266923796 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f9b5403e060 con 0x5652cdeca4d0 2024-04-25T10:33:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.057+0000 7f9b6737e740 10 client.15864 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:33:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.057+0000 7f9b6737e740 10 client.15864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:33:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.057+0000 7f9b6737e740 20 client.15864 populate_metadata read hostname 'smithi043' 2024-04-25T10:33:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.057+0000 7f9b6737e740 1 --2- 172.21.15.43:0/2266923796 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5652cde7f1c0 0x5652ce311380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.057+0000 7f9b6737e740 4 client.15864 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:33:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.057+0000 7f9b6737e740 1 -- 172.21.15.43:0/2266923796 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5652ce1f8ee0 con 0x5652cde7f1c0 2024-04-25T10:33:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.057+0000 7f9b5d98e640 1 --2- 172.21.15.43:0/2266923796 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5652cde7f1c0 0x5652ce311380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.058+0000 7f9b41ffb640 10 client.15864 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.058+0000 7f9b5d98e640 1 --2- 172.21.15.43:0/2266923796 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5652cde7f1c0 0x5652ce311380 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be26597296de847b server_cookie=886d5e6e66f22eed in_seq=0 out_seq=0 2024-04-25T10:33:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.062+0000 7f9b41ffb640 1 -- 172.21.15.43:0/2266923796 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5652ce1f8ee0 con 0x5652cde7f1c0 2024-04-25T10:33:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.062+0000 7f9b41ffb640 10 client.15864 handle_command_reply: tid=0 2024-04-25T10:33:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.062+0000 7f9b6737e740 1 client.15864 shutdown 2024-04-25T10:33:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.062+0000 7f9b6737e740 1 -- 172.21.15.43:0/2266923796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeca4d0 msgr2=0x5652ce298620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:48.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.062+0000 7f9b6737e740 1 --2- 172.21.15.43:0/2266923796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeca4d0 0x5652ce298620 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9b54005890 tx=0x7f9b54003e40 comp rx=0 tx=0).stop 2024-04-25T10:33:48.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.062+0000 7f9b6737e740 1 -- 172.21.15.43:0/2266923796 shutdown_connections 2024-04-25T10:33:48.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.062+0000 7f9b6737e740 1 --2- 172.21.15.43:0/2266923796 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5652cde7f1c0 0x5652ce311380 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.062+0000 7f9b6737e740 1 --2- 172.21.15.43:0/2266923796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652cdeeb1b0 0x5652ce298b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.062+0000 7f9b6737e740 1 --2- 172.21.15.43:0/2266923796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652cdeca4d0 0x5652ce298620 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.062+0000 7f9b6737e740 1 --2- 172.21.15.43:0/2266923796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652cdec22f0 0x5652ce2980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.062+0000 7f9b6737e740 1 -- 172.21.15.43:0/2266923796 >> 172.21.15.43:0/2266923796 conn(0x5652cdedf300 msgr2=0x5652ce296fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:48.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.062+0000 7f9b41ffb640 0 client.15864 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:33:48.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.062+0000 7f9b6737e740 1 -- 172.21.15.43:0/2266923796 shutdown_connections 2024-04-25T10:33:48.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.062+0000 7f9b6737e740 1 -- 172.21.15.43:0/2266923796 wait complete. 2024-04-25T10:33:48.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.062+0000 7f9b6737e740 20 client.15864 trim_cache size 0 max 16384 2024-04-25T10:33:48.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:33:48.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:33:48.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:33:48.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:33:48.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.063+0000 7f9b603f3640 1 -- 172.21.15.43:0/3974864555 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9b440afe80 msgr2=0x7f9b440b2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:48.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.063+0000 7f9b603f3640 1 --2- 172.21.15.43:0/3974864555 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9b440afe80 0x7f9b440b2390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9b500dc6e0 tx=0x7f9b500e2920 comp rx=0 tx=0).stop 2024-04-25T10:33:48.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.063+0000 7f9b603f3640 1 -- 172.21.15.43:0/3974864555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b58108ef0 msgr2=0x7f9b58199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:48.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.063+0000 7f9b603f3640 1 --2- 172.21.15.43:0/3974864555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b58108ef0 0x7f9b58199ec0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000efc0 tx=0x7f9b5000c800 comp rx=0 tx=0).stop 2024-04-25T10:33:48.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.063+0000 7f9b603f3640 1 -- 172.21.15.43:0/3974864555 shutdown_connections 2024-04-25T10:33:48.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.064+0000 7f9b603f3640 1 --2- 172.21.15.43:0/3974864555 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9b440afe80 0x7f9b440b2390 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.064+0000 7f9b603f3640 1 --2- 172.21.15.43:0/3974864555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b58108ef0 0x7f9b58199ec0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.064+0000 7f9b603f3640 1 --2- 172.21.15.43:0/3974864555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b58108a50 0x7f9b58199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.064+0000 7f9b603f3640 1 --2- 172.21.15.43:0/3974864555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b581031b0 0x7f9b58199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.064+0000 7f9b603f3640 1 -- 172.21.15.43:0/3974864555 >> 172.21.15.43:0/3974864555 conn(0x7f9b580fe260 msgr2=0x7f9b580fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.064+0000 7f9b603f3640 1 -- 172.21.15.43:0/3974864555 shutdown_connections 2024-04-25T10:33:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:33:48.064+0000 7f9b603f3640 1 -- 172.21.15.43:0/3974864555 wait complete. 2024-04-25T10:33:48.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:47 smithi077 ceph-mon[32772]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T10:33:48.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:47 smithi164 ceph-mon[32713]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T10:33:48.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:47 smithi043 ceph-mon[29478]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T10:33:49.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:48 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:33:49.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:48 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:33:49.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:48 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:33:50.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:49 smithi077 ceph-mon[32772]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T10:33:50.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:49 smithi164 ceph-mon[32713]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T10:33:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:49 smithi043 ceph-mon[29478]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T10:33:52.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:51 smithi077 ceph-mon[32772]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-25T10:33:52.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:51 smithi164 ceph-mon[32713]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-25T10:33:52.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:51 smithi043 ceph-mon[29478]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-25T10:33:54.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:53 smithi077 ceph-mon[32772]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T10:33:54.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:53 smithi164 ceph-mon[32713]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T10:33:54.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:53 smithi043 ceph-mon[29478]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T10:33:56.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:55 smithi077 ceph-mon[32772]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T10:33:56.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:55 smithi164 ceph-mon[32713]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T10:33:56.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:55 smithi043 ceph-mon[29478]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T10:33:58.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:57 smithi077 ceph-mon[32772]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T10:33:58.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:57 smithi164 ceph-mon[32713]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T10:33:58.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:57 smithi043 ceph-mon[29478]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T10:34:00.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:33:59 smithi077 ceph-mon[32772]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T10:34:00.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:33:59 smithi164 ceph-mon[32713]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T10:34:00.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:33:59 smithi043 ceph-mon[29478]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T10:34:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.169+0000 7f8744ae2640 1 -- 172.21.15.43:0/3430881111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f874010db20 msgr2=0x7f874010ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.169+0000 7f8744ae2640 1 --2- 172.21.15.43:0/3430881111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f874010db20 0x7f874010ff40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8734009900 tx=0x7f873402a410 comp rx=0 tx=0).stop 2024-04-25T10:34:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.169+0000 7f8744ae2640 1 -- 172.21.15.43:0/3430881111 shutdown_connections 2024-04-25T10:34:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.169+0000 7f8744ae2640 1 --2- 172.21.15.43:0/3430881111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8740110480 0x7f87401128a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.169+0000 7f8744ae2640 1 --2- 172.21.15.43:0/3430881111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f874010db20 0x7f874010ff40 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.169+0000 7f8744ae2640 1 --2- 172.21.15.43:0/3430881111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87401028e0 0x7f8740102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.169+0000 7f8744ae2640 1 -- 172.21.15.43:0/3430881111 >> 172.21.15.43:0/3430881111 conn(0x7f87400781b0 msgr2=0x7f87400785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.169+0000 7f8744ae2640 1 -- 172.21.15.43:0/3430881111 shutdown_connections 2024-04-25T10:34:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.169+0000 7f8744ae2640 1 -- 172.21.15.43:0/3430881111 wait complete. 2024-04-25T10:34:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.170+0000 7f8744ae2640 1 Processor -- start 2024-04-25T10:34:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.170+0000 7f8744ae2640 1 -- start start 2024-04-25T10:34:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.171+0000 7f8744ae2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87401028e0 0x7f87401a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.171+0000 7f8744ae2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f874010db20 0x7f87401a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.171+0000 7f8744ae2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8740110480 0x7f874019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.171+0000 7f8744ae2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8740115f40 con 0x7f874010db20 2024-04-25T10:34:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.171+0000 7f8744ae2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8740115dc0 con 0x7f87401028e0 2024-04-25T10:34:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.171+0000 7f8744ae2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f87401160c0 con 0x7f8740110480 2024-04-25T10:34:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.171+0000 7f873dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f874010db20 0x7f87401a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.171+0000 7f873dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f874010db20 0x7f87401a2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37292/0 (socket says 172.21.15.43:37292) 2024-04-25T10:34:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.171+0000 7f873dd9b640 1 -- 172.21.15.43:0/478340590 learned_addr learned my addr 172.21.15.43:0/478340590 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.171+0000 7f873e59c640 1 --2- 172.21.15.43:0/478340590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87401028e0 0x7f87401a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.171+0000 7f873ed9d640 1 --2- 172.21.15.43:0/478340590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8740110480 0x7f874019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.171+0000 7f873dd9b640 1 -- 172.21.15.43:0/478340590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8740110480 msgr2=0x7f874019c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.171+0000 7f873dd9b640 1 --2- 172.21.15.43:0/478340590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8740110480 0x7f874019c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.171+0000 7f873dd9b640 1 -- 172.21.15.43:0/478340590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87401028e0 msgr2=0x7f87401a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.171+0000 7f873dd9b640 1 --2- 172.21.15.43:0/478340590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87401028e0 0x7f87401a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.171+0000 7f873dd9b640 1 -- 172.21.15.43:0/478340590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f874019cea0 con 0x7f874010db20 2024-04-25T10:34:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.172+0000 7f873dd9b640 1 --2- 172.21.15.43:0/478340590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f874010db20 0x7f87401a2ad0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8734004270 tx=0x7f87340042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.172+0000 7f872f7fe640 1 -- 172.21.15.43:0/478340590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873403f030 con 0x7f874010db20 2024-04-25T10:34:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.172+0000 7f8744ae2640 1 -- 172.21.15.43:0/478340590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f874019d130 con 0x7f874010db20 2024-04-25T10:34:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.172+0000 7f8744ae2640 1 -- 172.21.15.43:0/478340590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87401a93e0 con 0x7f874010db20 2024-04-25T10:34:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.172+0000 7f872f7fe640 1 -- 172.21.15.43:0/478340590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87340044c0 con 0x7f874010db20 2024-04-25T10:34:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.172+0000 7f872f7fe640 1 -- 172.21.15.43:0/478340590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734035070 con 0x7f874010db20 2024-04-25T10:34:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.175+0000 7f874b98c740 1 Processor -- start 2024-04-25T10:34:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.175+0000 7f874b98c740 1 -- start start 2024-04-25T10:34:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.176+0000 7f874b98c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e6f1b0 0x559dc120eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.176+0000 7f874b98c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e462f0 0x559dc120f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.176+0000 7f872f7fe640 1 -- 172.21.15.43:0/478340590 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 147804+0+0 (secure 0 0 0) 0x7f8734042030 con 0x7f874010db20 2024-04-25T10:34:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.176+0000 7f873e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e462f0 0x559dc120f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.176+0000 7f874b98c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 0x559dc120f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.176+0000 7f873ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e6f1b0 0x559dc120eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.176+0000 7f873ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e6f1b0 0x559dc120eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:41810/0 (socket says 172.21.15.43:41810) 2024-04-25T10:34:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.176+0000 7f874b98c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559dc1175460 con 0x559dc0e462f0 2024-04-25T10:34:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.176+0000 7f873ed9d640 1 -- 172.21.15.43:0/3939956857 learned_addr learned my addr 172.21.15.43:0/3939956857 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.176+0000 7f874b98c740 1 -- 172.21.15.43:0/3939956857 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559dc112f590 con 0x559dc0e4e4d0 2024-04-25T10:34:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.176+0000 7f874b98c740 1 -- 172.21.15.43:0/3939956857 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559dc1171420 con 0x559dc0e6f1b0 2024-04-25T10:34:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.177+0000 7f873dd9b640 1 --2- 172.21.15.43:0/3939956857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 0x559dc120f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.177+0000 7f873dd9b640 1 -- 172.21.15.43:0/3939956857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e6f1b0 msgr2=0x559dc120eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.177+0000 7f873dd9b640 1 --2- 172.21.15.43:0/3939956857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e6f1b0 0x559dc120eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.177+0000 7f873dd9b640 1 -- 172.21.15.43:0/3939956857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e462f0 msgr2=0x559dc120f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.177+0000 7f873dd9b640 1 --2- 172.21.15.43:0/3939956857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e462f0 0x559dc120f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.177+0000 7f873dd9b640 1 -- 172.21.15.43:0/3939956857 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dc113c0d0 con 0x559dc0e4e4d0 2024-04-25T10:34:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.177+0000 7f873dd9b640 1 --2- 172.21.15.43:0/3939956857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 0x559dc120f570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f87340fcc60 tx=0x7f8734105cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c53bd39e20986079 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.177+0000 7f872f7fe640 1 --2- 172.21.15.43:0/478340590 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f87280afe00 0x7f87280b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.177+0000 7f872cff9640 1 -- 172.21.15.43:0/3939956857 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87340c70b0 con 0x559dc0e4e4d0 2024-04-25T10:34:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.178+0000 7f873ed9d640 1 --2- 172.21.15.43:0/478340590 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f87280afe00 0x7f87280b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.178+0000 7f872cff9640 1 -- 172.21.15.43:0/3939956857 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f873410f020 con 0x559dc0e4e4d0 2024-04-25T10:34:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.178+0000 7f872cff9640 1 -- 172.21.15.43:0/3939956857 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87340c72a0 con 0x559dc0e4e4d0 2024-04-25T10:34:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.178+0000 7f872f7fe640 1 -- 172.21.15.43:0/478340590 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f87340ff720 con 0x7f874010db20 2024-04-25T10:34:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.179+0000 7f874b98c740 1 -- 172.21.15.43:0/3939956857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 msgr2=0x559dc120f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.179+0000 7f873ed9d640 1 --2- 172.21.15.43:0/478340590 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f87280afe00 0x7f87280b2310 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x559dc120bc60 tx=0x7f8738009600 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.179+0000 7f874b98c740 1 --2- 172.21.15.43:0/3939956857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 0x559dc120f570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f87340fcc60 tx=0x7f8734105cb0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.179+0000 7f874b98c740 1 -- 172.21.15.43:0/3939956857 shutdown_connections 2024-04-25T10:34:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.179+0000 7f874b98c740 1 --2- 172.21.15.43:0/3939956857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 0x559dc120f570 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.179+0000 7f874b98c740 1 --2- 172.21.15.43:0/3939956857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e462f0 0x559dc120f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.179+0000 7f874b98c740 1 --2- 172.21.15.43:0/3939956857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e6f1b0 0x559dc120eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.179+0000 7f874b98c740 1 -- 172.21.15.43:0/3939956857 >> 172.21.15.43:0/3939956857 conn(0x559dc0e63300 msgr2=0x559dc120b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.179+0000 7f874b98c740 1 -- 172.21.15.43:0/3939956857 shutdown_connections 2024-04-25T10:34:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.179+0000 7f874b98c740 1 -- 172.21.15.43:0/3939956857 wait complete. 2024-04-25T10:34:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.179+0000 7f874b98c740 1 -- start start 2024-04-25T10:34:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.179+0000 7f874b98c740 1 Processor -- start 2024-04-25T10:34:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.180+0000 7f874b98c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e462f0 0x559dc121ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.180+0000 7f874b98c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 0x559dc121b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.180+0000 7f874b98c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e6f1b0 0x559dc1219010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.180+0000 7f874b98c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559dc121b9e0 con 0x559dc0e6f1b0 2024-04-25T10:34:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.180+0000 7f874b98c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559dc10fcb60 con 0x559dc0e4e4d0 2024-04-25T10:34:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.180+0000 7f874b98c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559dc121bb60 con 0x559dc0e462f0 2024-04-25T10:34:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.180+0000 7f874b98c740 10 client.0 _ll_register_callbacks cb 0x559dc11212b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.180+0000 7f873e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e462f0 0x559dc121ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.180+0000 7f873e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e462f0 0x559dc121ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:41818/0 (socket says 172.21.15.43:41818) 2024-04-25T10:34:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.180+0000 7f873ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e6f1b0 0x559dc1219010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.180+0000 7f873e59c640 1 -- 172.21.15.43:0/3891159272 learned_addr learned my addr 172.21.15.43:0/3891159272 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.180+0000 7f873dd9b640 1 --2- 172.21.15.43:0/3891159272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 0x559dc121b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.181+0000 7f873e59c640 1 -- 172.21.15.43:0/3891159272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 msgr2=0x559dc121b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.181+0000 7f873e59c640 1 --2- 172.21.15.43:0/3891159272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 0x559dc121b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.181+0000 7f873e59c640 1 -- 172.21.15.43:0/3891159272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e6f1b0 msgr2=0x559dc1219010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.181+0000 7f873e59c640 1 --2- 172.21.15.43:0/3891159272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e6f1b0 0x559dc1219010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.181+0000 7f873e59c640 1 -- 172.21.15.43:0/3891159272 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dc0e87210 con 0x559dc0e462f0 2024-04-25T10:34:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.181+0000 7f873ed9d640 1 --2- 172.21.15.43:0/3891159272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e6f1b0 0x559dc1219010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.181+0000 7f873e59c640 1 --2- 172.21.15.43:0/3891159272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e462f0 0x559dc121ab50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8730005fd0 tx=0x7f87300118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=531824835fab3458 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.181+0000 7f8725ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-25T10:34:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.181+0000 7f8725ffb640 1 -- 172.21.15.43:0/3891159272 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8730004020 con 0x559dc0e462f0 2024-04-25T10:34:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.181+0000 7f874b98c740 1 -- 172.21.15.43:0/3891159272 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f873410f020 con 0x559dc0e462f0 2024-04-25T10:34:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.181+0000 7f8725ffb640 1 -- 172.21.15.43:0/3891159272 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f87300041c0 con 0x559dc0e462f0 2024-04-25T10:34:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.182+0000 7f8725ffb640 1 -- 172.21.15.43:0/3891159272 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f87300043c0 con 0x559dc0e462f0 2024-04-25T10:34:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.182+0000 7f874b98c740 10 client.25501 fetch_fsmap learned FSMap version 24 2024-04-25T10:34:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.182+0000 7f874b98c740 1 -- 172.21.15.43:0/3891159272 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559dc0f53620 con 0x559dc0e462f0 2024-04-25T10:34:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.182+0000 7f8725ffb640 1 -- 172.21.15.43:0/3891159272 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f873410f020 con 0x559dc0e462f0 2024-04-25T10:34:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.182+0000 7f8725ffb640 1 -- 172.21.15.43:0/3891159272 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8730003a00 con 0x559dc0e462f0 2024-04-25T10:34:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.182+0000 7f874b98c740 10 client.25501 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:34:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.182+0000 7f874b98c740 10 client.25501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.182+0000 7f874b98c740 20 client.25501 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.182+0000 7f874b98c740 1 --2- 172.21.15.43:0/3891159272 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559dc0e2dd80 0x559dc122b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.182+0000 7f874b98c740 4 client.25501 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:34:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.182+0000 7f874b98c740 1 -- 172.21.15.43:0/3891159272 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dc1183070 con 0x559dc0e2dd80 2024-04-25T10:34:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.182+0000 7f873dd9b640 1 --2- 172.21.15.43:0/3891159272 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559dc0e2dd80 0x559dc122b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.183+0000 7f8725ffb640 10 client.25501 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.183+0000 7f873dd9b640 1 --2- 172.21.15.43:0/3891159272 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559dc0e2dd80 0x559dc122b4f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdf7cb8a36cb8596 server_cookie=8abd720b0c20f209 in_seq=0 out_seq=0 2024-04-25T10:34:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.184+0000 7f8725ffb640 1 -- 172.21.15.43:0/3891159272 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559dc1183070 con 0x559dc0e2dd80 2024-04-25T10:34:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.184+0000 7f8725ffb640 10 client.25501 handle_command_reply: tid=0 2024-04-25T10:34:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.184+0000 7f874b98c740 1 client.25501 shutdown 2024-04-25T10:34:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.184+0000 7f874b98c740 1 -- 172.21.15.43:0/3891159272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e462f0 msgr2=0x559dc121ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.184+0000 7f874b98c740 1 --2- 172.21.15.43:0/3891159272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e462f0 0x559dc121ab50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8730005fd0 tx=0x7f87300118d0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.184+0000 7f874b98c740 1 -- 172.21.15.43:0/3891159272 shutdown_connections 2024-04-25T10:34:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.184+0000 7f874b98c740 1 --2- 172.21.15.43:0/3891159272 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559dc0e2dd80 0x559dc122b4f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.184+0000 7f8725ffb640 0 client.25501 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.184+0000 7f874b98c740 1 --2- 172.21.15.43:0/3891159272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 0x559dc121b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.184+0000 7f874b98c740 1 --2- 172.21.15.43:0/3891159272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e6f1b0 0x559dc1219010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.184+0000 7f874b98c740 1 --2- 172.21.15.43:0/3891159272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e462f0 0x559dc121ab50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.184+0000 7f874b98c740 1 -- 172.21.15.43:0/3891159272 >> 172.21.15.43:0/3891159272 conn(0x559dc0e63300 msgr2=0x559dc1210810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.184+0000 7f874b98c740 1 -- 172.21.15.43:0/3891159272 shutdown_connections 2024-04-25T10:34:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.184+0000 7f874b98c740 1 -- 172.21.15.43:0/3891159272 wait complete. 2024-04-25T10:34:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.184+0000 7f874b98c740 20 client.25501 trim_cache size 0 max 16384 2024-04-25T10:34:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.199+0000 7f874b98c740 1 Processor -- start 2024-04-25T10:34:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.199+0000 7f874b98c740 1 -- start start 2024-04-25T10:34:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.200+0000 7f874b98c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e2dd80 0x559dc1218fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.200+0000 7f874b98c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e462f0 0x559dc1230d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.200+0000 7f874b98c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 0x559dc1231280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.200+0000 7f874b98c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f87401b4510 con 0x559dc0e462f0 2024-04-25T10:34:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.200+0000 7f874b98c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87401b1de0 con 0x559dc0e4e4d0 2024-04-25T10:34:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.200+0000 7f874b98c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f87401afc30 con 0x559dc0e2dd80 2024-04-25T10:34:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.200+0000 7f873dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e462f0 0x559dc1230d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.200+0000 7f873dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e462f0 0x559dc1230d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37318/0 (socket says 172.21.15.43:37318) 2024-04-25T10:34:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.200+0000 7f873dd9b640 1 -- 172.21.15.43:0/2978363023 learned_addr learned my addr 172.21.15.43:0/2978363023 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.200+0000 7f873e59c640 1 --2- 172.21.15.43:0/2978363023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e2dd80 0x559dc1218fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.200+0000 7f873ed9d640 1 --2- 172.21.15.43:0/2978363023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 0x559dc1231280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.200+0000 7f873dd9b640 1 -- 172.21.15.43:0/2978363023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e2dd80 msgr2=0x559dc1218fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.200+0000 7f873dd9b640 1 --2- 172.21.15.43:0/2978363023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e2dd80 0x559dc1218fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.200+0000 7f873dd9b640 1 -- 172.21.15.43:0/2978363023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 msgr2=0x559dc1231280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.200+0000 7f873dd9b640 1 --2- 172.21.15.43:0/2978363023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 0x559dc1231280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.200+0000 7f873dd9b640 1 -- 172.21.15.43:0/2978363023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dc122bab0 con 0x559dc0e462f0 2024-04-25T10:34:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.200+0000 7f873ed9d640 1 --2- 172.21.15.43:0/2978363023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 0x559dc1231280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.201+0000 7f873dd9b640 1 --2- 172.21.15.43:0/2978363023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e462f0 0x559dc1230d40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f87340fcda0 tx=0x7f873402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ddc12863f34c40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.201+0000 7f8727fff640 1 -- 172.21.15.43:0/2978363023 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734049070 con 0x559dc0e462f0 2024-04-25T10:34:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.201+0000 7f8727fff640 1 -- 172.21.15.43:0/2978363023 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87340c7270 con 0x559dc0e462f0 2024-04-25T10:34:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.201+0000 7f8727fff640 1 -- 172.21.15.43:0/2978363023 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87340c7530 con 0x559dc0e462f0 2024-04-25T10:34:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.202+0000 7f874b98c740 1 -- 172.21.15.43:0/2978363023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e462f0 msgr2=0x559dc1230d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.202+0000 7f874b98c740 1 --2- 172.21.15.43:0/2978363023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e462f0 0x559dc1230d40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f87340fcda0 tx=0x7f873402a410 comp rx=0 tx=0).stop 2024-04-25T10:34:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.202+0000 7f874b98c740 1 -- 172.21.15.43:0/2978363023 shutdown_connections 2024-04-25T10:34:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.202+0000 7f874b98c740 1 --2- 172.21.15.43:0/2978363023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 0x559dc1231280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.202+0000 7f874b98c740 1 --2- 172.21.15.43:0/2978363023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e462f0 0x559dc1230d40 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.202+0000 7f874b98c740 1 --2- 172.21.15.43:0/2978363023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e2dd80 0x559dc1218fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.202+0000 7f874b98c740 1 -- 172.21.15.43:0/2978363023 >> 172.21.15.43:0/2978363023 conn(0x559dc0e63300 msgr2=0x559dc12188a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.202+0000 7f874b98c740 1 -- 172.21.15.43:0/2978363023 shutdown_connections 2024-04-25T10:34:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.202+0000 7f874b98c740 1 -- 172.21.15.43:0/2978363023 wait complete. 2024-04-25T10:34:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.202+0000 7f874b98c740 1 -- start start 2024-04-25T10:34:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.202+0000 7f874b98c740 1 Processor -- start 2024-04-25T10:34:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.203+0000 7f874b98c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e2dd80 0x559dc121d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.203+0000 7f874b98c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e462f0 0x559dc121de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.203+0000 7f874b98c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 0x559dc121e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.203+0000 7f873dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e462f0 0x559dc121de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.203+0000 7f873dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e462f0 0x559dc121de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37332/0 (socket says 172.21.15.43:37332) 2024-04-25T10:34:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.203+0000 7f873dd9b640 1 -- 172.21.15.43:0/2243897837 learned_addr learned my addr 172.21.15.43:0/2243897837 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.203+0000 7f874b98c740 1 -- 172.21.15.43:0/2243897837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f87401b02a0 con 0x559dc0e462f0 2024-04-25T10:34:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.203+0000 7f874b98c740 1 -- 172.21.15.43:0/2243897837 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87401b1710 con 0x559dc0e4e4d0 2024-04-25T10:34:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.203+0000 7f874b98c740 1 -- 172.21.15.43:0/2243897837 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f87401b0970 con 0x559dc0e2dd80 2024-04-25T10:34:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.203+0000 7f874b98c740 10 client.0 _ll_register_callbacks cb 0x7f873000bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.203+0000 7f873e59c640 1 --2- 172.21.15.43:0/2243897837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e2dd80 0x559dc121d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.203+0000 7f873ed9d640 1 --2- 172.21.15.43:0/2243897837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 0x559dc121e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.203+0000 7f873dd9b640 1 -- 172.21.15.43:0/2243897837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e2dd80 msgr2=0x559dc121d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.203+0000 7f873dd9b640 1 --2- 172.21.15.43:0/2243897837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e2dd80 0x559dc121d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.203+0000 7f873dd9b640 1 -- 172.21.15.43:0/2243897837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 msgr2=0x559dc121e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.203+0000 7f873dd9b640 1 --2- 172.21.15.43:0/2243897837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 0x559dc121e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.203+0000 7f873dd9b640 1 -- 172.21.15.43:0/2243897837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dc12918f0 con 0x559dc0e462f0 2024-04-25T10:34:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.203+0000 7f873ed9d640 1 --2- 172.21.15.43:0/2243897837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 0x559dc121e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.203+0000 7f873dd9b640 1 --2- 172.21.15.43:0/2243897837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e462f0 0x559dc121de50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8734104040 tx=0x7f8734009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9093e90dfb05e7e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.204+0000 7f8725ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.204+0000 7f8725ffb640 1 -- 172.21.15.43:0/2243897837 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873410e020 con 0x559dc0e462f0 2024-04-25T10:34:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.204+0000 7f874b98c740 1 -- 172.21.15.43:0/2243897837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87340c7270 con 0x559dc0e462f0 2024-04-25T10:34:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.204+0000 7f8725ffb640 1 -- 172.21.15.43:0/2243897837 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8469+0+0 (secure 0 0 0) 0x7f87340aebe0 con 0x559dc0e462f0 2024-04-25T10:34:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.204+0000 7f8725ffb640 1 -- 172.21.15.43:0/2243897837 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8734049070 con 0x559dc0e462f0 2024-04-25T10:34:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.204+0000 7f874b98c740 10 client.15882 fetch_fsmap learned FSMap version 24 2024-04-25T10:34:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.204+0000 7f874b98c740 1 -- 172.21.15.43:0/2243897837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559dc12929d0 con 0x559dc0e462f0 2024-04-25T10:34:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.204+0000 7f8725ffb640 1 -- 172.21.15.43:0/2243897837 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87340c3270 con 0x559dc0e462f0 2024-04-25T10:34:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.205+0000 7f8725ffb640 1 -- 172.21.15.43:0/2243897837 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f87340c39f0 con 0x559dc0e462f0 2024-04-25T10:34:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.205+0000 7f874b98c740 10 client.15882 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:34:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.205+0000 7f874b98c740 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.205+0000 7f874b98c740 20 client.15882 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.205+0000 7f874b98c740 1 --2- 172.21.15.43:0/2243897837 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559dc0e6f1b0 0x559dc1294d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.205+0000 7f874b98c740 4 client.15882 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:34:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.205+0000 7f874b98c740 1 -- 172.21.15.43:0/2243897837 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f87340c7270 con 0x559dc0e6f1b0 2024-04-25T10:34:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.205+0000 7f873e59c640 1 --2- 172.21.15.43:0/2243897837 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559dc0e6f1b0 0x559dc1294d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.206+0000 7f8725ffb640 10 client.15882 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:01.206+0000 7f873e59c640 1 --2- 172.21.15.43:0/2243897837 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559dc0e6f1b0 0x559dc1294d70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4148f26fb3266c4b server_cookie=846a180490f7922b in_seq=0 out_seq=0 2024-04-25T10:34:02.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:01 smithi077 ceph-mon[32772]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:34:02.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:01 smithi164 ceph-mon[32713]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:34:02.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:01 smithi043 ceph-mon[29478]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:34:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.705+0000 7f8725ffb640 1 -- 172.21.15.43:0/2243897837 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f87340c7270 con 0x559dc0e6f1b0 2024-04-25T10:34:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.705+0000 7f8725ffb640 10 client.15882 handle_command_reply: tid=0 2024-04-25T10:34:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.705+0000 7f874b98c740 1 client.15882 shutdown 2024-04-25T10:34:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.705+0000 7f874b98c740 1 -- 172.21.15.43:0/2243897837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e462f0 msgr2=0x559dc121de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.705+0000 7f874b98c740 1 --2- 172.21.15.43:0/2243897837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e462f0 0x559dc121de50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8734104040 tx=0x7f8734009260 comp rx=0 tx=0).stop 2024-04-25T10:34:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.705+0000 7f874b98c740 1 -- 172.21.15.43:0/2243897837 shutdown_connections 2024-04-25T10:34:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.705+0000 7f874b98c740 1 --2- 172.21.15.43:0/2243897837 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559dc0e6f1b0 0x559dc1294d70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.705+0000 7f874b98c740 1 --2- 172.21.15.43:0/2243897837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dc0e4e4d0 0x559dc121e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.705+0000 7f874b98c740 1 --2- 172.21.15.43:0/2243897837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc0e462f0 0x559dc121de50 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.705+0000 7f874b98c740 1 --2- 172.21.15.43:0/2243897837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559dc0e2dd80 0x559dc121d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.706+0000 7f8725ffb640 0 client.15882 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.706+0000 7f874b98c740 1 -- 172.21.15.43:0/2243897837 >> 172.21.15.43:0/2243897837 conn(0x559dc0e63300 msgr2=0x559dc121a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.706+0000 7f874b98c740 1 -- 172.21.15.43:0/2243897837 shutdown_connections 2024-04-25T10:34:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.706+0000 7f874b98c740 1 -- 172.21.15.43:0/2243897837 wait complete. 2024-04-25T10:34:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.706+0000 7f874b98c740 20 client.15882 trim_cache size 0 max 16384 2024-04-25T10:34:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:34:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:34:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:34:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:34:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.706+0000 7f8744ae2640 1 -- 172.21.15.43:0/478340590 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f87280afe00 msgr2=0x7f87280b2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.706+0000 7f8744ae2640 1 --2- 172.21.15.43:0/478340590 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f87280afe00 0x7f87280b2310 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x559dc120bc60 tx=0x7f8738009600 comp rx=0 tx=0).stop 2024-04-25T10:34:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.706+0000 7f8744ae2640 1 -- 172.21.15.43:0/478340590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f874010db20 msgr2=0x7f87401a2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.706+0000 7f8744ae2640 1 --2- 172.21.15.43:0/478340590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f874010db20 0x7f87401a2ad0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8734004270 tx=0x7f87340042a0 comp rx=0 tx=0).stop 2024-04-25T10:34:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.707+0000 7f8744ae2640 1 -- 172.21.15.43:0/478340590 shutdown_connections 2024-04-25T10:34:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.707+0000 7f8744ae2640 1 --2- 172.21.15.43:0/478340590 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f87280afe00 0x7f87280b2310 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.707+0000 7f8744ae2640 1 --2- 172.21.15.43:0/478340590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8740110480 0x7f874019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.707+0000 7f8744ae2640 1 --2- 172.21.15.43:0/478340590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f874010db20 0x7f87401a2ad0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.707+0000 7f8744ae2640 1 --2- 172.21.15.43:0/478340590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87401028e0 0x7f87401a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.707+0000 7f8744ae2640 1 -- 172.21.15.43:0/478340590 >> 172.21.15.43:0/478340590 conn(0x7f87400781b0 msgr2=0x7f8740101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.707+0000 7f8744ae2640 1 -- 172.21.15.43:0/478340590 shutdown_connections 2024-04-25T10:34:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:02.707+0000 7f8744ae2640 1 -- 172.21.15.43:0/478340590 wait complete. 2024-04-25T10:34:03.516 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: *** Caught signal (Segmentation fault) ** 2024-04-25T10:34:03.517 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: in thread 7fa625ce4700 thread_name:md_log_replay 2024-04-25T10:34:03.517 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T10:34:03.517 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa6370a5d20] 2024-04-25T10:34:03.517 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x33) [0x55794339140b] 2024-04-25T10:34:03.517 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x5579433918a2] 2024-04-25T10:34:03.517 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x55794338bf19] 2024-04-25T10:34:03.517 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x55794304d4d9] 2024-04-25T10:34:03.517 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 6: (MDLog::_replay_thread()+0x1f38) [0x5579433f1c4a] 2024-04-25T10:34:03.517 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 7: (MDLog::ReplayThread::entry()+0x11) [0x5579433fb09b] 2024-04-25T10:34:03.517 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 8: (Thread::entry_wrapper()+0x43) [0x7fa6384f772d] 2024-04-25T10:34:03.518 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 9: (Thread::_entry_func(void*)+0xd) [0x7fa6384f7749] 2024-04-25T10:34:03.518 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7fa63709b1ca] 2024-04-25T10:34:03.518 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 11: clone() 2024-04-25T10:34:03.518 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 2024-04-25T10:34:03.168+0000 7fa625ce4700 -1 *** Caught signal (Segmentation fault) ** 2024-04-25T10:34:03.518 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: in thread 7fa625ce4700 thread_name:md_log_replay 2024-04-25T10:34:03.518 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 2024-04-25T10:34:03.518 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T10:34:03.518 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa6370a5d20] 2024-04-25T10:34:03.518 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x33) [0x55794339140b] 2024-04-25T10:34:03.519 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x5579433918a2] 2024-04-25T10:34:03.519 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x55794338bf19] 2024-04-25T10:34:03.519 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x55794304d4d9] 2024-04-25T10:34:03.519 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 6: (MDLog::_replay_thread()+0x1f38) [0x5579433f1c4a] 2024-04-25T10:34:03.519 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 7: (MDLog::ReplayThread::entry()+0x11) [0x5579433fb09b] 2024-04-25T10:34:03.519 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 8: (Thread::entry_wrapper()+0x43) [0x7fa6384f772d] 2024-04-25T10:34:03.519 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 9: (Thread::_entry_func(void*)+0xd) [0x7fa6384f7749] 2024-04-25T10:34:03.519 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7fa63709b1ca] 2024-04-25T10:34:03.519 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 11: clone() 2024-04-25T10:34:03.519 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-25T10:34:03.519 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 2024-04-25T10:34:03.519 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 0> 2024-04-25T10:34:03.168+0000 7fa625ce4700 -1 *** Caught signal (Segmentation fault) ** 2024-04-25T10:34:03.519 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: in thread 7fa625ce4700 thread_name:md_log_replay 2024-04-25T10:34:03.520 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 2024-04-25T10:34:03.520 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T10:34:03.520 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa6370a5d20] 2024-04-25T10:34:03.520 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x33) [0x55794339140b] 2024-04-25T10:34:03.520 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x5579433918a2] 2024-04-25T10:34:03.520 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x55794338bf19] 2024-04-25T10:34:03.520 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x55794304d4d9] 2024-04-25T10:34:03.520 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 6: (MDLog::_replay_thread()+0x1f38) [0x5579433f1c4a] 2024-04-25T10:34:03.520 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 7: (MDLog::ReplayThread::entry()+0x11) [0x5579433fb09b] 2024-04-25T10:34:03.520 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 8: (Thread::entry_wrapper()+0x43) [0x7fa6384f772d] 2024-04-25T10:34:03.520 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 9: (Thread::_entry_func(void*)+0xd) [0x7fa6384f7749] 2024-04-25T10:34:03.520 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7fa63709b1ca] 2024-04-25T10:34:03.520 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 11: clone() 2024-04-25T10:34:03.521 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-25T10:34:03.521 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 2024-04-25T10:34:03.521 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 0> 2024-04-25T10:34:03.168+0000 7fa625ce4700 -1 *** Caught signal (Segmentation fault) ** 2024-04-25T10:34:03.521 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: in thread 7fa625ce4700 thread_name:md_log_replay 2024-04-25T10:34:03.521 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 2024-04-25T10:34:03.521 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T10:34:03.521 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa6370a5d20] 2024-04-25T10:34:03.521 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x33) [0x55794339140b] 2024-04-25T10:34:03.521 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x5579433918a2] 2024-04-25T10:34:03.522 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x55794338bf19] 2024-04-25T10:34:03.522 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x55794304d4d9] 2024-04-25T10:34:03.522 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 6: (MDLog::_replay_thread()+0x1f38) [0x5579433f1c4a] 2024-04-25T10:34:03.522 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 7: (MDLog::ReplayThread::entry()+0x11) [0x5579433fb09b] 2024-04-25T10:34:03.522 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 8: (Thread::entry_wrapper()+0x43) [0x7fa6384f772d] 2024-04-25T10:34:03.522 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 9: (Thread::_entry_func(void*)+0xd) [0x7fa6384f7749] 2024-04-25T10:34:03.522 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7fa63709b1ca] 2024-04-25T10:34:03.522 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 11: clone() 2024-04-25T10:34:03.522 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-25T10:34:03.522 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[63370]: 2024-04-25T10:34:03.522 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 podman[66783]: 2024-04-25 10:34:03.231870237 +0000 UTC m=+0.022813782 container died 26ed057570417f0a4730800b3c8d82b38ff400ecd946f78f0483626bcf0f7083 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-25T10:34:04.169 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 podman[66783]: 2024-04-25 10:34:03.91551383 +0000 UTC m=+0.706457359 container remove 26ed057570417f0a4730800b3c8d82b38ff400ecd946f78f0483626bcf0f7083 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:34:04.169 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:03 smithi077 systemd[1]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.h.service: Main process exited, code=exited, status=139/n/a 2024-04-25T10:34:04.169 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:03 smithi077 ceph-mon[32772]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:34:04.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:03 smithi164 ceph-mon[32713]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:34:04.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:03 smithi043 ceph-mon[29478]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:34:04.540 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:04 smithi077 systemd[1]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.h.service: Failed with result 'exit-code'. 2024-04-25T10:34:06.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:05 smithi077 ceph-mon[32772]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-04-25T10:34:06.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:05 smithi077 ceph-mon[32772]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T10:34:06.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:05 smithi164 ceph-mon[32713]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-04-25T10:34:06.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:05 smithi164 ceph-mon[32713]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T10:34:06.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:05 smithi043 ceph-mon[29478]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-04-25T10:34:06.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:05 smithi043 ceph-mon[29478]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T10:34:07.914 INFO:tasks.workunit.client.0.smithi043.stderr:Updating files: 93% (11442/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-25T10:34:08.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:07 smithi077 ceph-mon[32772]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-04-25T10:34:08.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:07 smithi164 ceph-mon[32713]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-04-25T10:34:08.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:07 smithi043 ceph-mon[29478]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-04-25T10:34:08.790 INFO:tasks.workunit.client.0.smithi043.stderr:Note: switching to '994e2f20102a20fa9a27966c14b625cc01884df8'. 2024-04-25T10:34:08.791 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-04-25T10:34:08.791 INFO:tasks.workunit.client.0.smithi043.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T10:34:08.791 INFO:tasks.workunit.client.0.smithi043.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T10:34:08.791 INFO:tasks.workunit.client.0.smithi043.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T10:34:08.791 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-04-25T10:34:08.791 INFO:tasks.workunit.client.0.smithi043.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T10:34:08.791 INFO:tasks.workunit.client.0.smithi043.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T10:34:08.791 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-04-25T10:34:08.791 INFO:tasks.workunit.client.0.smithi043.stderr: git switch -c 2024-04-25T10:34:08.792 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-04-25T10:34:08.792 INFO:tasks.workunit.client.0.smithi043.stderr:Or undo this operation with: 2024-04-25T10:34:08.792 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-04-25T10:34:08.792 INFO:tasks.workunit.client.0.smithi043.stderr: git switch - 2024-04-25T10:34:08.792 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-04-25T10:34:08.792 INFO:tasks.workunit.client.0.smithi043.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T10:34:08.792 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-04-25T10:34:08.792 INFO:tasks.workunit.client.0.smithi043.stderr:HEAD is now at 994e2f20102 Merge PR #56810 into wip-pdonnell-testing-20240425.015853-debug 2024-04-25T10:34:08.797 DEBUG:teuthology.orchestra.run.smithi043:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T10:34:08.860 INFO:tasks.workunit.client.0.smithi043.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T10:34:08.862 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T10:34:08.862 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T10:34:09.508 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T10:34:09.563 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T10:34:09.604 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T10:34:09.606 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T10:34:09.606 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T10:34:09.646 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T10:34:09.650 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T10:34:09.651 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T10:34:09.707 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-04-25T10:34:09.708 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-04-25T10:34:09.708 DEBUG:teuthology.orchestra.run.smithi043:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-25T10:34:09.768 INFO:tasks.workunit.client.0.smithi043.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-25T10:34:09.768 INFO:tasks.workunit.client.0.smithi043.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-04-25T10:34:09.769 INFO:tasks.workunit.client.0.smithi043.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-04-25T10:34:10.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:09 smithi077 ceph-mon[32772]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-04-25T10:34:10.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:09 smithi164 ceph-mon[32713]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-04-25T10:34:10.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:09 smithi043 ceph-mon[29478]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-04-25T10:34:12.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:11 smithi077 ceph-mon[32772]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-04-25T10:34:12.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:11 smithi164 ceph-mon[32713]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-04-25T10:34:12.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:11 smithi043 ceph-mon[29478]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-04-25T10:34:14.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:14 smithi077 ceph-mon[32772]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-04-25T10:34:14.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:14 smithi164 ceph-mon[32713]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-04-25T10:34:14.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:14 smithi043 ceph-mon[29478]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-04-25T10:34:14.657 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:14 smithi077 systemd[1]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.h.service: Scheduled restart job, restart counter is at 1. 2024-04-25T10:34:14.657 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:14 smithi077 systemd[1]: Stopped Ceph mds.h for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:34:14.657 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:14 smithi077 systemd[1]: Starting Ceph mds.h for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:34:14.657 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:14 smithi077 podman[66876]: 2024-04-25 10:34:14.432631109 +0000 UTC m=+0.018582326 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:34:14.657 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:14 smithi077 podman[66876]: 2024-04-25 10:34:14.533139195 +0000 UTC m=+0.119090437 container create d3ab0de6929c97bc8752ff1b8e17cedaa1160fc2bc4263eb1f68f0e0cb7ab47a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T10:34:15.004 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:14 smithi077 podman[66876]: 2024-04-25 10:34:14.75240012 +0000 UTC m=+0.338351319 container init d3ab0de6929c97bc8752ff1b8e17cedaa1160fc2bc4263eb1f68f0e0cb7ab47a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T10:34:15.004 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:14 smithi077 podman[66876]: 2024-04-25 10:34:14.755216835 +0000 UTC m=+0.341168034 container start d3ab0de6929c97bc8752ff1b8e17cedaa1160fc2bc4263eb1f68f0e0cb7ab47a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T10:34:15.004 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:14 smithi077 ceph-mds[66894]: -- 172.21.15.77:0/1373758423 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55981f429a40 con 0x55981f471180 2024-04-25T10:34:15.005 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:14 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[66890]: starting mds.h at 2024-04-25T10:34:15.005 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:14 smithi077 bash[66876]: d3ab0de6929c97bc8752ff1b8e17cedaa1160fc2bc4263eb1f68f0e0cb7ab47a 2024-04-25T10:34:15.005 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:34:14 smithi077 systemd[1]: Started Ceph mds.h for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:34:15.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:34:15.163 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:34:15.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:15 smithi077 ceph-mon[32772]: daemon mds.h restarted 2024-04-25T10:34:15.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:15 smithi164 ceph-mon[32713]: daemon mds.h restarted 2024-04-25T10:34:15.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.290+0000 7f843fb27640 1 -- 172.21.15.43:0/4266157718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8438109af0 msgr2=0x7f8438101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.290+0000 7f843fb27640 1 --2- 172.21.15.43:0/4266157718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8438109af0 0x7f8438101b70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8434009900 tx=0x7f843402a410 comp rx=0 tx=0).stop 2024-04-25T10:34:15.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.290+0000 7f843fb27640 1 -- 172.21.15.43:0/4266157718 shutdown_connections 2024-04-25T10:34:15.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.290+0000 7f843fb27640 1 --2- 172.21.15.43:0/4266157718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8438109f90 0x7f84381020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.290+0000 7f843fb27640 1 --2- 172.21.15.43:0/4266157718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8438102720 0x7f8438110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.290+0000 7f843fb27640 1 --2- 172.21.15.43:0/4266157718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8438109af0 0x7f8438101b70 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.290+0000 7f843fb27640 1 -- 172.21.15.43:0/4266157718 >> 172.21.15.43:0/4266157718 conn(0x7f8438077880 msgr2=0x7f8438077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:15.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.290+0000 7f843fb27640 1 -- 172.21.15.43:0/4266157718 shutdown_connections 2024-04-25T10:34:15.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.290+0000 7f843fb27640 1 -- 172.21.15.43:0/4266157718 wait complete. 2024-04-25T10:34:15.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.291+0000 7f843fb27640 1 Processor -- start 2024-04-25T10:34:15.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.291+0000 7f843fb27640 1 -- start start 2024-04-25T10:34:15.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.291+0000 7f843fb27640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8438102720 0x7f843819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.291+0000 7f843fb27640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8438109af0 0x7f843819ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.291+0000 7f843fb27640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8438109f90 0x7f843819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.291+0000 7f843fb27640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8438113d00 con 0x7f8438102720 2024-04-25T10:34:15.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.291+0000 7f843fb27640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8438113b80 con 0x7f8438109af0 2024-04-25T10:34:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.291+0000 7f843fb27640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8438113e80 con 0x7f8438109f90 2024-04-25T10:34:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.291+0000 7f843d8c3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8438102720 0x7f843819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.292+0000 7f843d0c2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8438109af0 0x7f843819ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.292+0000 7f843d8c3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8438102720 0x7f843819a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41702/0 (socket says 172.21.15.43:41702) 2024-04-25T10:34:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.292+0000 7f843d8c3640 1 -- 172.21.15.43:0/2146141754 learned_addr learned my addr 172.21.15.43:0/2146141754 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.292+0000 7f843e0c4640 1 --2- 172.21.15.43:0/2146141754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8438109f90 0x7f843819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.292+0000 7f843d0c2640 1 -- 172.21.15.43:0/2146141754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8438109f90 msgr2=0x7f843819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.292+0000 7f843d0c2640 1 --2- 172.21.15.43:0/2146141754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8438109f90 0x7f843819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.292+0000 7f843d0c2640 1 -- 172.21.15.43:0/2146141754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8438102720 msgr2=0x7f843819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.292+0000 7f843d0c2640 1 --2- 172.21.15.43:0/2146141754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8438102720 0x7f843819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.292+0000 7f843d0c2640 1 -- 172.21.15.43:0/2146141754 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843819f540 con 0x7f8438109af0 2024-04-25T10:34:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.292+0000 7f843d8c3640 1 --2- 172.21.15.43:0/2146141754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8438102720 0x7f843819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.292+0000 7f843d0c2640 1 --2- 172.21.15.43:0/2146141754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8438109af0 0x7f843819ac40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f842800e9c0 tx=0x7f842800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.292+0000 7f842effd640 1 -- 172.21.15.43:0/2146141754 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842800ccc0 con 0x7f8438109af0 2024-04-25T10:34:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.292+0000 7f842effd640 1 -- 172.21.15.43:0/2146141754 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84280108e0 con 0x7f8438109af0 2024-04-25T10:34:15.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.292+0000 7f843fb27640 1 -- 172.21.15.43:0/2146141754 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843819f830 con 0x7f8438109af0 2024-04-25T10:34:15.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.292+0000 7f842effd640 1 -- 172.21.15.43:0/2146141754 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84280187c0 con 0x7f8438109af0 2024-04-25T10:34:15.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.292+0000 7f843fb27640 1 -- 172.21.15.43:0/2146141754 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84381a71b0 con 0x7f8438109af0 2024-04-25T10:34:15.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.295+0000 7f84469d7740 1 Processor -- start 2024-04-25T10:34:15.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.295+0000 7f842effd640 1 -- 172.21.15.43:0/2146141754 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f8428010430 con 0x7f8438109af0 2024-04-25T10:34:15.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.295+0000 7f84469d7740 1 -- start start 2024-04-25T10:34:15.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.296+0000 7f84469d7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792f9390 0x564779699a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.296+0000 7f84469d7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647792d04d0 0x564779699fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.296+0000 7f84469d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792d86b0 0x5647796a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.296+0000 7f84469d7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5647795ffd80 con 0x5647792f9390 2024-04-25T10:34:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.296+0000 7f84469d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5647795b98b0 con 0x5647792d86b0 2024-04-25T10:34:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.296+0000 7f84469d7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5647795fbd40 con 0x5647792d04d0 2024-04-25T10:34:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.296+0000 7f843e0c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792f9390 0x564779699a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.296+0000 7f843d0c2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792d86b0 0x5647796a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.296+0000 7f843e0c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792f9390 0x564779699a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41706/0 (socket says 172.21.15.43:41706) 2024-04-25T10:34:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.296+0000 7f843e0c4640 1 -- 172.21.15.43:0/4248792985 learned_addr learned my addr 172.21.15.43:0/4248792985 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.296+0000 7f843d8c3640 1 --2- 172.21.15.43:0/4248792985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647792d04d0 0x564779699fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.296+0000 7f842effd640 1 --2- 172.21.15.43:0/2146141754 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f84240afc60 0x7f84240b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.296+0000 7f843e0c4640 1 -- 172.21.15.43:0/4248792985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647792d04d0 msgr2=0x564779699fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.296+0000 7f843e0c4640 1 --2- 172.21.15.43:0/4248792985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647792d04d0 0x564779699fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.296+0000 7f843e0c4640 1 -- 172.21.15.43:0/4248792985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792d86b0 msgr2=0x5647796a68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.296+0000 7f843e0c4640 1 --2- 172.21.15.43:0/4248792985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792d86b0 0x5647796a68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.296+0000 7f843e0c4640 1 -- 172.21.15.43:0/4248792985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56477958aa40 con 0x5647792f9390 2024-04-25T10:34:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.296+0000 7f843d0c2640 1 --2- 172.21.15.43:0/4248792985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792d86b0 0x5647796a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:15.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.296+0000 7f842effd640 1 -- 172.21.15.43:0/2146141754 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f84280df480 con 0x7f8438109af0 2024-04-25T10:34:15.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.296+0000 7f843e0c4640 1 --2- 172.21.15.43:0/2146141754 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f84240afc60 0x7f84240b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.297+0000 7f843e0c4640 1 --2- 172.21.15.43:0/4248792985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792f9390 0x564779699a60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8430009eb0 tx=0x7f8430010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05c2c4945517de7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:15.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.297+0000 7f8423fff640 1 -- 172.21.15.43:0/4248792985 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430021070 con 0x5647792f9390 2024-04-25T10:34:15.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.297+0000 7f8423fff640 1 -- 172.21.15.43:0/4248792985 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f843001be40 con 0x5647792f9390 2024-04-25T10:34:15.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.297+0000 7f8423fff640 1 -- 172.21.15.43:0/4248792985 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430016070 con 0x5647792f9390 2024-04-25T10:34:15.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.298+0000 7f84469d7740 1 -- 172.21.15.43:0/4248792985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792f9390 msgr2=0x564779699a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.298+0000 7f843e0c4640 1 --2- 172.21.15.43:0/2146141754 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f84240afc60 0x7f84240b2170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f843001e000 tx=0x7f8430021000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:15.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.298+0000 7f84469d7740 1 --2- 172.21.15.43:0/4248792985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792f9390 0x564779699a60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8430009eb0 tx=0x7f8430010740 comp rx=0 tx=0).stop 2024-04-25T10:34:15.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.298+0000 7f84469d7740 1 -- 172.21.15.43:0/4248792985 shutdown_connections 2024-04-25T10:34:15.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.298+0000 7f84469d7740 1 --2- 172.21.15.43:0/4248792985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792d86b0 0x5647796a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.298+0000 7f84469d7740 1 --2- 172.21.15.43:0/4248792985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647792d04d0 0x564779699fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.298+0000 7f84469d7740 1 --2- 172.21.15.43:0/4248792985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792f9390 0x564779699a60 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.298+0000 7f84469d7740 1 -- 172.21.15.43:0/4248792985 >> 172.21.15.43:0/4248792985 conn(0x5647792ed4e0 msgr2=0x564779696210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:15.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.298+0000 7f84469d7740 1 -- 172.21.15.43:0/4248792985 shutdown_connections 2024-04-25T10:34:15.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.298+0000 7f84469d7740 1 -- 172.21.15.43:0/4248792985 wait complete. 2024-04-25T10:34:15.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.298+0000 7f84469d7740 1 -- start start 2024-04-25T10:34:15.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.298+0000 7f84469d7740 1 Processor -- start 2024-04-25T10:34:15.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.299+0000 7f84469d7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647792d04d0 0x5647796a4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.299+0000 7f84469d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792d86b0 0x5647796a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.299+0000 7f84469d7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792f9390 0x5647796aa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.299+0000 7f84469d7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5647796a6fe0 con 0x5647792f9390 2024-04-25T10:34:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.299+0000 7f84469d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564779586d10 con 0x5647792d86b0 2024-04-25T10:34:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.299+0000 7f84469d7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5647796a7160 con 0x5647792d04d0 2024-04-25T10:34:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.299+0000 7f84469d7740 10 client.0 _ll_register_callbacks cb 0x5647795b3210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.299+0000 7f843e0c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792f9390 0x5647796aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.299+0000 7f843e0c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792f9390 0x5647796aa430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41722/0 (socket says 172.21.15.43:41722) 2024-04-25T10:34:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.299+0000 7f843e0c4640 1 -- 172.21.15.43:0/2322900266 learned_addr learned my addr 172.21.15.43:0/2322900266 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.299+0000 7f843d0c2640 1 --2- 172.21.15.43:0/2322900266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792d86b0 0x5647796a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.299+0000 7f843d8c3640 1 --2- 172.21.15.43:0/2322900266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647792d04d0 0x5647796a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.299+0000 7f843e0c4640 1 -- 172.21.15.43:0/2322900266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647792d04d0 msgr2=0x5647796a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.299+0000 7f843e0c4640 1 --2- 172.21.15.43:0/2322900266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647792d04d0 0x5647796a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.299+0000 7f843e0c4640 1 -- 172.21.15.43:0/2322900266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792d86b0 msgr2=0x5647796a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.299+0000 7f843e0c4640 1 --2- 172.21.15.43:0/2322900266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792d86b0 0x5647796a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.299+0000 7f843e0c4640 1 -- 172.21.15.43:0/2322900266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647793db7d0 con 0x5647792f9390 2024-04-25T10:34:15.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.299+0000 7f843d0c2640 1 --2- 172.21.15.43:0/2322900266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792d86b0 0x5647796a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:15.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.300+0000 7f843e0c4640 1 --2- 172.21.15.43:0/2322900266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792f9390 0x5647796aa430 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f84300119d0 tx=0x7f8430010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242eb9365e48d405 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:15.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.300+0000 7f84217fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:15.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.300+0000 7f84217fa640 1 -- 172.21.15.43:0/2322900266 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430034060 con 0x5647792f9390 2024-04-25T10:34:15.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.300+0000 7f84217fa640 1 -- 172.21.15.43:0/2322900266 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f84300126f0 con 0x5647792f9390 2024-04-25T10:34:15.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.300+0000 7f84469d7740 1 -- 172.21.15.43:0/2322900266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56477960d990 con 0x5647792f9390 2024-04-25T10:34:15.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.300+0000 7f84217fa640 1 -- 172.21.15.43:0/2322900266 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8430016040 con 0x5647792f9390 2024-04-25T10:34:15.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.300+0000 7f84469d7740 10 client.15897 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:15.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.300+0000 7f84469d7740 1 -- 172.21.15.43:0/2322900266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5647793113f0 con 0x5647792f9390 2024-04-25T10:34:15.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.300+0000 7f84217fa640 1 -- 172.21.15.43:0/2322900266 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56477960d990 con 0x5647792f9390 2024-04-25T10:34:15.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.300+0000 7f84217fa640 1 -- 172.21.15.43:0/2322900266 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f843002e030 con 0x5647792f9390 2024-04-25T10:34:15.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.300+0000 7f84469d7740 10 client.15897 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:15.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.300+0000 7f84469d7740 10 client.15897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.300+0000 7f84469d7740 20 client.15897 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.300+0000 7f84469d7740 1 --2- 172.21.15.43:0/2322900266 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5647792b7f60 0x5647796b6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.300+0000 7f84469d7740 4 client.15897 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:34:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.300+0000 7f84469d7740 1 -- 172.21.15.43:0/2322900266 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564779607800 con 0x5647792b7f60 2024-04-25T10:34:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.301+0000 7f843d8c3640 1 --2- 172.21.15.43:0/2322900266 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5647792b7f60 0x5647796b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.301+0000 7f84217fa640 10 client.15897 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.301+0000 7f843d8c3640 1 --2- 172.21.15.43:0/2322900266 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5647792b7f60 0x5647796b6580 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eab8abf6596dd70 server_cookie=57d953499a721074 in_seq=0 out_seq=0 2024-04-25T10:34:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.302+0000 7f84217fa640 1 -- 172.21.15.43:0/2322900266 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564779607800 con 0x5647792b7f60 2024-04-25T10:34:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.302+0000 7f84217fa640 10 client.15897 handle_command_reply: tid=0 2024-04-25T10:34:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.302+0000 7f84469d7740 1 client.15897 shutdown 2024-04-25T10:34:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.302+0000 7f84469d7740 1 -- 172.21.15.43:0/2322900266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792f9390 msgr2=0x5647796aa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.302+0000 7f84469d7740 1 --2- 172.21.15.43:0/2322900266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792f9390 0x5647796aa430 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f84300119d0 tx=0x7f8430010f10 comp rx=0 tx=0).stop 2024-04-25T10:34:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.302+0000 7f84469d7740 1 -- 172.21.15.43:0/2322900266 shutdown_connections 2024-04-25T10:34:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.302+0000 7f84469d7740 1 --2- 172.21.15.43:0/2322900266 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5647792b7f60 0x5647796b6580 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.302+0000 7f84469d7740 1 --2- 172.21.15.43:0/2322900266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792d86b0 0x5647796a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.302+0000 7f84469d7740 1 --2- 172.21.15.43:0/2322900266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792f9390 0x5647796aa430 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.302+0000 7f84469d7740 1 --2- 172.21.15.43:0/2322900266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647792d04d0 0x5647796a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.302+0000 7f84217fa640 0 client.15897 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:15.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.302+0000 7f84469d7740 1 -- 172.21.15.43:0/2322900266 >> 172.21.15.43:0/2322900266 conn(0x5647792ed4e0 msgr2=0x56477969b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:15.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.302+0000 7f84469d7740 1 -- 172.21.15.43:0/2322900266 shutdown_connections 2024-04-25T10:34:15.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.302+0000 7f84469d7740 1 -- 172.21.15.43:0/2322900266 wait complete. 2024-04-25T10:34:15.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.302+0000 7f84469d7740 20 client.15897 trim_cache size 0 max 16384 2024-04-25T10:34:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.320+0000 7f84469d7740 1 Processor -- start 2024-04-25T10:34:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.320+0000 7f84469d7740 1 -- start start 2024-04-25T10:34:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.320+0000 7f84469d7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647792d04d0 0x5647796a6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.320+0000 7f84469d7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792d86b0 0x5647796a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.320+0000 7f84469d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792f9390 0x5647796bbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.320+0000 7f84469d7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f84381b2290 con 0x5647792d86b0 2024-04-25T10:34:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.320+0000 7f84469d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f84381afb60 con 0x5647792f9390 2024-04-25T10:34:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.320+0000 7f84469d7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f84381ad9b0 con 0x5647792d04d0 2024-04-25T10:34:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.321+0000 7f843d0c2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792d86b0 0x5647796a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.321+0000 7f843d0c2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792d86b0 0x5647796a3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41726/0 (socket says 172.21.15.43:41726) 2024-04-25T10:34:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.321+0000 7f843d0c2640 1 -- 172.21.15.43:0/90348311 learned_addr learned my addr 172.21.15.43:0/90348311 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.321+0000 7f843e0c4640 1 --2- 172.21.15.43:0/90348311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792f9390 0x5647796bbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.321+0000 7f843d0c2640 1 -- 172.21.15.43:0/90348311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647792d04d0 msgr2=0x5647796a6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.321+0000 7f843d8c3640 1 --2- 172.21.15.43:0/90348311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647792d04d0 0x5647796a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.321+0000 7f843d0c2640 1 --2- 172.21.15.43:0/90348311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647792d04d0 0x5647796a6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.321+0000 7f843d0c2640 1 -- 172.21.15.43:0/90348311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792f9390 msgr2=0x5647796bbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.321+0000 7f843d0c2640 1 --2- 172.21.15.43:0/90348311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792f9390 0x5647796bbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.321+0000 7f843d0c2640 1 -- 172.21.15.43:0/90348311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647792d10f0 con 0x5647792d86b0 2024-04-25T10:34:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.321+0000 7f843e0c4640 1 --2- 172.21.15.43:0/90348311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792f9390 0x5647796bbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.321+0000 7f843d8c3640 1 --2- 172.21.15.43:0/90348311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647792d04d0 0x5647796a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.321+0000 7f843d0c2640 1 --2- 172.21.15.43:0/90348311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792d86b0 0x5647796a3bd0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f84280e2cd0 tx=0x7f842800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756c7f4b76822f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.321+0000 7f84237fe640 1 -- 172.21.15.43:0/90348311 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8428010720 con 0x5647792d86b0 2024-04-25T10:34:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.321+0000 7f84237fe640 1 -- 172.21.15.43:0/90348311 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84280a7100 con 0x5647792d86b0 2024-04-25T10:34:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.321+0000 7f84237fe640 1 -- 172.21.15.43:0/90348311 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84280b0450 con 0x5647792d86b0 2024-04-25T10:34:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.322+0000 7f84469d7740 1 -- 172.21.15.43:0/90348311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792d86b0 msgr2=0x5647796a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.322+0000 7f84469d7740 1 --2- 172.21.15.43:0/90348311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792d86b0 0x5647796a3bd0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f84280e2cd0 tx=0x7f842800ce60 comp rx=0 tx=0).stop 2024-04-25T10:34:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.322+0000 7f84469d7740 1 -- 172.21.15.43:0/90348311 shutdown_connections 2024-04-25T10:34:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.322+0000 7f84469d7740 1 --2- 172.21.15.43:0/90348311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792f9390 0x5647796bbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.322+0000 7f84469d7740 1 --2- 172.21.15.43:0/90348311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792d86b0 0x5647796a3bd0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.322+0000 7f84469d7740 1 --2- 172.21.15.43:0/90348311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647792d04d0 0x5647796a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.322+0000 7f84469d7740 1 -- 172.21.15.43:0/90348311 >> 172.21.15.43:0/90348311 conn(0x5647792ed4e0 msgr2=0x56477969bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.322+0000 7f84469d7740 1 -- 172.21.15.43:0/90348311 shutdown_connections 2024-04-25T10:34:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.322+0000 7f84469d7740 1 -- 172.21.15.43:0/90348311 wait complete. 2024-04-25T10:34:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.322+0000 7f84469d7740 1 -- start start 2024-04-25T10:34:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.323+0000 7f84469d7740 1 Processor -- start 2024-04-25T10:34:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.323+0000 7f84469d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792d04d0 0x5647796abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.323+0000 7f84469d7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792d86b0 0x5647796ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.323+0000 7f84469d7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647792f9390 0x56477971c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.323+0000 7f84469d7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f84381ae020 con 0x5647792d86b0 2024-04-25T10:34:15.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.323+0000 7f84469d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f84381af490 con 0x5647792d04d0 2024-04-25T10:34:15.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.323+0000 7f84469d7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f84381ae6f0 con 0x5647792f9390 2024-04-25T10:34:15.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.323+0000 7f84469d7740 10 client.0 _ll_register_callbacks cb 0x7f843001f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:15.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.323+0000 7f843d0c2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792d86b0 0x5647796ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.323+0000 7f843d0c2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792d86b0 0x5647796ac1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41736/0 (socket says 172.21.15.43:41736) 2024-04-25T10:34:15.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.323+0000 7f843d0c2640 1 -- 172.21.15.43:0/4053003323 learned_addr learned my addr 172.21.15.43:0/4053003323 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:15.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.323+0000 7f843d8c3640 1 --2- 172.21.15.43:0/4053003323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792d04d0 0x5647796abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.323+0000 7f843d0c2640 1 -- 172.21.15.43:0/4053003323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647792f9390 msgr2=0x56477971c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.323+0000 7f843d0c2640 1 --2- 172.21.15.43:0/4053003323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647792f9390 0x56477971c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.323+0000 7f843d0c2640 1 -- 172.21.15.43:0/4053003323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792d04d0 msgr2=0x5647796abcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.323+0000 7f843d0c2640 1 --2- 172.21.15.43:0/4053003323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792d04d0 0x5647796abcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.323+0000 7f843d0c2640 1 -- 172.21.15.43:0/4053003323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647795c04b0 con 0x5647792d86b0 2024-04-25T10:34:15.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.323+0000 7f843d8c3640 1 --2- 172.21.15.43:0/4053003323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792d04d0 0x5647796abcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:15.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.324+0000 7f843d0c2640 1 --2- 172.21.15.43:0/4053003323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792d86b0 0x5647796ac1f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f84280a4860 tx=0x7f84280ab9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba75e17c387e95e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:15.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.324+0000 7f84217fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:15.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.324+0000 7f84217fa640 1 -- 172.21.15.43:0/4053003323 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84280a6a50 con 0x5647792d86b0 2024-04-25T10:34:15.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.324+0000 7f84217fa640 1 -- 172.21.15.43:0/4053003323 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f84280a2f60 con 0x5647792d86b0 2024-04-25T10:34:15.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.324+0000 7f84469d7740 1 -- 172.21.15.43:0/4053003323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84280a7100 con 0x5647792d86b0 2024-04-25T10:34:15.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.324+0000 7f84217fa640 1 -- 172.21.15.43:0/4053003323 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f84280a75f0 con 0x5647792d86b0 2024-04-25T10:34:15.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.324+0000 7f84469d7740 10 client.15903 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:15.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.324+0000 7f84469d7740 1 -- 172.21.15.43:0/4053003323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56477971dd30 con 0x5647792d86b0 2024-04-25T10:34:15.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.324+0000 7f84217fa640 1 -- 172.21.15.43:0/4053003323 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84280a7100 con 0x5647792d86b0 2024-04-25T10:34:15.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.324+0000 7f84217fa640 1 -- 172.21.15.43:0/4053003323 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f84280de070 con 0x5647792d86b0 2024-04-25T10:34:15.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.324+0000 7f84469d7740 10 client.15903 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:15.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.324+0000 7f84469d7740 10 client.15903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:15.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.324+0000 7f84469d7740 20 client.15903 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:15.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.324+0000 7f84469d7740 1 --2- 172.21.15.43:0/4053003323 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56477928d3a0 0x564779720420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.324+0000 7f84469d7740 4 client.15903 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:34:15.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.324+0000 7f84469d7740 1 -- 172.21.15.43:0/4053003323 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5647795824a0 con 0x56477928d3a0 2024-04-25T10:34:15.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.325+0000 7f843d8c3640 1 --2- 172.21.15.43:0/4053003323 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56477928d3a0 0x564779720420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.325+0000 7f84217fa640 10 client.15903 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:15.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.325+0000 7f843d8c3640 1 --2- 172.21.15.43:0/4053003323 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56477928d3a0 0x564779720420 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=266834e56d316589 server_cookie=5887e2cba0575461 in_seq=0 out_seq=0 2024-04-25T10:34:15.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.325+0000 7f84217fa640 1 -- 172.21.15.43:0/4053003323 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5647795824a0 con 0x56477928d3a0 2024-04-25T10:34:15.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.325+0000 7f84217fa640 10 client.15903 handle_command_reply: tid=0 2024-04-25T10:34:15.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.325+0000 7f84469d7740 1 client.15903 shutdown 2024-04-25T10:34:15.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.325+0000 7f84469d7740 1 -- 172.21.15.43:0/4053003323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792d86b0 msgr2=0x5647796ac1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.325+0000 7f84469d7740 1 --2- 172.21.15.43:0/4053003323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792d86b0 0x5647796ac1f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f84280a4860 tx=0x7f84280ab9a0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.325+0000 7f84469d7740 1 -- 172.21.15.43:0/4053003323 shutdown_connections 2024-04-25T10:34:15.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.325+0000 7f84469d7740 1 --2- 172.21.15.43:0/4053003323 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56477928d3a0 0x564779720420 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.325+0000 7f84469d7740 1 --2- 172.21.15.43:0/4053003323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647792f9390 0x56477971c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.325+0000 7f84469d7740 1 --2- 172.21.15.43:0/4053003323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647792d86b0 0x5647796ac1f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.325+0000 7f84217fa640 0 client.15903 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:15.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.325+0000 7f84469d7740 1 --2- 172.21.15.43:0/4053003323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5647792d04d0 0x5647796abcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.325+0000 7f84469d7740 1 -- 172.21.15.43:0/4053003323 >> 172.21.15.43:0/4053003323 conn(0x5647792ed4e0 msgr2=0x5647796a6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:15.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.326+0000 7f84469d7740 1 -- 172.21.15.43:0/4053003323 shutdown_connections 2024-04-25T10:34:15.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.326+0000 7f84469d7740 1 -- 172.21.15.43:0/4053003323 wait complete. 2024-04-25T10:34:15.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.326+0000 7f84469d7740 20 client.15903 trim_cache size 0 max 16384 2024-04-25T10:34:15.334 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:34:15.334 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:34:15.334 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:34:15.334 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:34:15.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.326+0000 7f843fb27640 1 -- 172.21.15.43:0/2146141754 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f84240afc60 msgr2=0x7f84240b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.326+0000 7f843fb27640 1 --2- 172.21.15.43:0/2146141754 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f84240afc60 0x7f84240b2170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f843001e000 tx=0x7f8430021000 comp rx=0 tx=0).stop 2024-04-25T10:34:15.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.326+0000 7f843fb27640 1 -- 172.21.15.43:0/2146141754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8438109af0 msgr2=0x7f843819ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.326+0000 7f843fb27640 1 --2- 172.21.15.43:0/2146141754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8438109af0 0x7f843819ac40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f842800e9c0 tx=0x7f842800ee80 comp rx=0 tx=0).stop 2024-04-25T10:34:15.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.326+0000 7f843fb27640 1 -- 172.21.15.43:0/2146141754 shutdown_connections 2024-04-25T10:34:15.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.326+0000 7f843fb27640 1 --2- 172.21.15.43:0/2146141754 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f84240afc60 0x7f84240b2170 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.326+0000 7f843fb27640 1 --2- 172.21.15.43:0/2146141754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8438109af0 0x7f843819ac40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.326+0000 7f843fb27640 1 --2- 172.21.15.43:0/2146141754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8438109f90 0x7f843819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.326+0000 7f843fb27640 1 --2- 172.21.15.43:0/2146141754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8438102720 0x7f843819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.326+0000 7f843fb27640 1 -- 172.21.15.43:0/2146141754 >> 172.21.15.43:0/2146141754 conn(0x7f8438077880 msgr2=0x7f84381051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:15.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.326+0000 7f843fb27640 1 -- 172.21.15.43:0/2146141754 shutdown_connections 2024-04-25T10:34:15.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.326+0000 7f843fb27640 1 -- 172.21.15.43:0/2146141754 wait complete. 2024-04-25T10:34:15.338 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:15 smithi043 ceph-mon[29478]: daemon mds.h restarted 2024-04-25T10:34:15.340 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:34:15.340 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:34:15.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.450+0000 7f9ccfc96640 1 -- 172.21.15.43:0/2684117713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cc8106990 msgr2=0x7f9cc8106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.450+0000 7f9ccfc96640 1 --2- 172.21.15.43:0/2684117713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cc8106990 0x7f9cc8106dc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009900 tx=0x7f9cb802a410 comp rx=0 tx=0).stop 2024-04-25T10:34:15.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.450+0000 7f9ccfc96640 1 -- 172.21.15.43:0/2684117713 shutdown_connections 2024-04-25T10:34:15.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.450+0000 7f9ccfc96640 1 --2- 172.21.15.43:0/2684117713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9cc8107450 0x7f9cc810ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.450+0000 7f9ccfc96640 1 --2- 172.21.15.43:0/2684117713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cc8106990 0x7f9cc8106dc0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.450+0000 7f9ccfc96640 1 --2- 172.21.15.43:0/2684117713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cc8102400 0x7f9cc8102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.450+0000 7f9ccfc96640 1 -- 172.21.15.43:0/2684117713 >> 172.21.15.43:0/2684117713 conn(0x7f9cc8077890 msgr2=0x7f9cc8077cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:15.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.450+0000 7f9ccfc96640 1 -- 172.21.15.43:0/2684117713 shutdown_connections 2024-04-25T10:34:15.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.450+0000 7f9ccfc96640 1 -- 172.21.15.43:0/2684117713 wait complete. 2024-04-25T10:34:15.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.450+0000 7f9ccfc96640 1 Processor -- start 2024-04-25T10:34:15.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.450+0000 7f9ccfc96640 1 -- start start 2024-04-25T10:34:15.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.451+0000 7f9ccfc96640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9cc8102400 0x7f9cc819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.451+0000 7f9ccfc96640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cc8106990 0x7f9cc819aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.451+0000 7f9ccfc96640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cc8107450 0x7f9cc819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.451+0000 7f9ccfc96640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8111ae0 con 0x7f9cc8107450 2024-04-25T10:34:15.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.451+0000 7f9ccfc96640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8111960 con 0x7f9cc8102400 2024-04-25T10:34:15.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.451+0000 7f9ccfc96640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8111c60 con 0x7f9cc8106990 2024-04-25T10:34:15.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.451+0000 7f9cce233640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cc8107450 0x7f9cc819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.451+0000 7f9cce233640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cc8107450 0x7f9cc819ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41762/0 (socket says 172.21.15.43:41762) 2024-04-25T10:34:15.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.451+0000 7f9cce233640 1 -- 172.21.15.43:0/2112350978 learned_addr learned my addr 172.21.15.43:0/2112350978 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:15.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.451+0000 7f9ccda32640 1 --2- 172.21.15.43:0/2112350978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9cc8102400 0x7f9cc819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.451+0000 7f9ccd231640 1 --2- 172.21.15.43:0/2112350978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cc8106990 0x7f9cc819aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.451+0000 7f9cce233640 1 -- 172.21.15.43:0/2112350978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cc8106990 msgr2=0x7f9cc819aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.451+0000 7f9cce233640 1 --2- 172.21.15.43:0/2112350978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cc8106990 0x7f9cc819aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.451+0000 7f9cce233640 1 -- 172.21.15.43:0/2112350978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9cc8102400 msgr2=0x7f9cc819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.451+0000 7f9cce233640 1 --2- 172.21.15.43:0/2112350978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9cc8102400 0x7f9cc819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.451+0000 7f9cce233640 1 -- 172.21.15.43:0/2112350978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc819f5a0 con 0x7f9cc8107450 2024-04-25T10:34:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.451+0000 7f9cce233640 1 --2- 172.21.15.43:0/2112350978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cc8107450 0x7f9cc819ef60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000af90 tx=0x7f9cc000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.451+0000 7f9cbeffd640 1 -- 172.21.15.43:0/2112350978 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0003b70 con 0x7f9cc8107450 2024-04-25T10:34:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.452+0000 7f9cbeffd640 1 -- 172.21.15.43:0/2112350978 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cc0003d10 con 0x7f9cc8107450 2024-04-25T10:34:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.452+0000 7f9cbeffd640 1 -- 172.21.15.43:0/2112350978 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000ba30 con 0x7f9cc8107450 2024-04-25T10:34:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.452+0000 7f9ccfc96640 1 -- 172.21.15.43:0/2112350978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc819f890 con 0x7f9cc8107450 2024-04-25T10:34:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.452+0000 7f9ccfc96640 1 -- 172.21.15.43:0/2112350978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc819fd60 con 0x7f9cc8107450 2024-04-25T10:34:15.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.453+0000 7f9ccfc96640 1 -- 172.21.15.43:0/2112350978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc806a190 con 0x7f9cc8107450 2024-04-25T10:34:15.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.454+0000 7f9cbeffd640 1 -- 172.21.15.43:0/2112350978 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f9cc000bbd0 con 0x7f9cc8107450 2024-04-25T10:34:15.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.455+0000 7f9cbeffd640 1 --2- 172.21.15.43:0/2112350978 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9cb40afc60 0x7f9cb40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.455+0000 7f9ccda32640 1 --2- 172.21.15.43:0/2112350978 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9cb40afc60 0x7f9cb40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.455+0000 7f9cbeffd640 1 -- 172.21.15.43:0/2112350978 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f9cc00df620 con 0x7f9cc8107450 2024-04-25T10:34:15.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.455+0000 7f9ccda32640 1 --2- 172.21.15.43:0/2112350978 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9cb40afc60 0x7f9cb40b2170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009bd0 tx=0x7f9cc4005f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:15.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.458+0000 7f9cbeffd640 1 -- 172.21.15.43:0/2112350978 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9cc00a7f80 con 0x7f9cc8107450 2024-04-25T10:34:15.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.683+0000 7f9ccfc96640 1 -- 172.21.15.43:0/2112350978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cc819ba90 con 0x7f9cc8107450 2024-04-25T10:34:15.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.683+0000 7f9cbeffd640 1 -- 172.21.15.43:0/2112350978 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7f9cc00a7880 con 0x7f9cc8107450 2024-04-25T10:34:15.686 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:34:15.686 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:34:15.686 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:34:15.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.685+0000 7f9ccfc96640 1 -- 172.21.15.43:0/2112350978 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9cb40afc60 msgr2=0x7f9cb40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.685+0000 7f9ccfc96640 1 --2- 172.21.15.43:0/2112350978 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9cb40afc60 0x7f9cb40b2170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009bd0 tx=0x7f9cc4005f90 comp rx=0 tx=0).stop 2024-04-25T10:34:15.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.685+0000 7f9ccfc96640 1 -- 172.21.15.43:0/2112350978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cc8107450 msgr2=0x7f9cc819ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.685+0000 7f9ccfc96640 1 --2- 172.21.15.43:0/2112350978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cc8107450 0x7f9cc819ef60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000af90 tx=0x7f9cc000c7a0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.686+0000 7f9ccfc96640 1 -- 172.21.15.43:0/2112350978 shutdown_connections 2024-04-25T10:34:15.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.686+0000 7f9ccfc96640 1 --2- 172.21.15.43:0/2112350978 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9cb40afc60 0x7f9cb40b2170 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.686+0000 7f9ccfc96640 1 --2- 172.21.15.43:0/2112350978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cc8107450 0x7f9cc819ef60 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.686+0000 7f9ccfc96640 1 --2- 172.21.15.43:0/2112350978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cc8106990 0x7f9cc819aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.686+0000 7f9ccfc96640 1 --2- 172.21.15.43:0/2112350978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9cc8102400 0x7f9cc819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.686+0000 7f9ccfc96640 1 -- 172.21.15.43:0/2112350978 >> 172.21.15.43:0/2112350978 conn(0x7f9cc8077890 msgr2=0x7f9cc810ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:15.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.686+0000 7f9ccfc96640 1 -- 172.21.15.43:0/2112350978 shutdown_connections 2024-04-25T10:34:15.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.686+0000 7f9ccfc96640 1 -- 172.21.15.43:0/2112350978 wait complete. 2024-04-25T10:34:15.699 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:34:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.813+0000 7fad3639e640 1 -- 172.21.15.43:0/557492785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad3010b6d0 msgr2=0x7fad3010daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.813+0000 7fad3639e640 1 --2- 172.21.15.43:0/557492785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad3010b6d0 0x7fad3010daf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fad24009900 tx=0x7fad2402a3f0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.813+0000 7fad3639e640 1 -- 172.21.15.43:0/557492785 shutdown_connections 2024-04-25T10:34:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.813+0000 7fad3639e640 1 --2- 172.21.15.43:0/557492785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad3010e0c0 0x7fad301105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.813+0000 7fad3639e640 1 --2- 172.21.15.43:0/557492785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad3010b6d0 0x7fad3010daf0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.813+0000 7fad3639e640 1 --2- 172.21.15.43:0/557492785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad30104c70 0x7fad30105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.813+0000 7fad3639e640 1 -- 172.21.15.43:0/557492785 >> 172.21.15.43:0/557492785 conn(0x7fad300781b0 msgr2=0x7fad300785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:15.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.813+0000 7f6f3b772640 1 -- 172.21.15.43:0/3774170253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f34108a20 msgr2=0x7f6f341025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.813+0000 7f6f3b772640 1 --2- 172.21.15.43:0/3774170253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f34108a20 0x7f6f341025b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009900 tx=0x7f6f3002a410 comp rx=0 tx=0).stop 2024-04-25T10:34:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.813+0000 7fad3639e640 1 -- 172.21.15.43:0/557492785 shutdown_connections 2024-04-25T10:34:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.813+0000 7fad3639e640 1 -- 172.21.15.43:0/557492785 wait complete. 2024-04-25T10:34:15.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.813+0000 7f6f3b772640 1 -- 172.21.15.43:0/3774170253 shutdown_connections 2024-04-25T10:34:15.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.813+0000 7f6f3b772640 1 --2- 172.21.15.43:0/3774170253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f3410d380 0x7f6f3410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.813+0000 7f6f3b772640 1 --2- 172.21.15.43:0/3774170253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f34102af0 0x7f6f34102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.813+0000 7f6f3b772640 1 --2- 172.21.15.43:0/3774170253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f34108a20 0x7f6f341025b0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.813+0000 7f6f3b772640 1 -- 172.21.15.43:0/3774170253 >> 172.21.15.43:0/3774170253 conn(0x7f6f34078100 msgr2=0x7f6f34078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:15.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.813+0000 7f6f3b772640 1 -- 172.21.15.43:0/3774170253 shutdown_connections 2024-04-25T10:34:15.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.813+0000 7f6f3b772640 1 -- 172.21.15.43:0/3774170253 wait complete. 2024-04-25T10:34:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.814+0000 7fad3639e640 1 Processor -- start 2024-04-25T10:34:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.814+0000 7fad3639e640 1 -- start start 2024-04-25T10:34:15.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.814+0000 7f6f3b772640 1 Processor -- start 2024-04-25T10:34:15.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.814+0000 7f6f3b772640 1 -- start start 2024-04-25T10:34:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.814+0000 7fad3639e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad30104c70 0x7fad301a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.814+0000 7fad3639e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad3010b6d0 0x7fad301a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.814+0000 7fad3639e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad3010e0c0 0x7fad3019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.814+0000 7fad3639e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fad30113d20 con 0x7fad30104c70 2024-04-25T10:34:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.814+0000 7fad3639e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fad30113ba0 con 0x7fad3010b6d0 2024-04-25T10:34:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.814+0000 7fad3639e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fad30113ea0 con 0x7fad3010e0c0 2024-04-25T10:34:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.814+0000 7fad2ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad30104c70 0x7fad301a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.814+0000 7f6f3b772640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f34102af0 0x7f6f3419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.814+0000 7f6f3b772640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f34108a20 0x7f6f3419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.814+0000 7f6f3b772640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f3410d380 0x7f6f3419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.814+0000 7f6f3b772640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34112be0 con 0x7f6f34102af0 2024-04-25T10:34:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.814+0000 7f6f3b772640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34112a60 con 0x7f6f34108a20 2024-04-25T10:34:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.814+0000 7f6f3b772640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34112d60 con 0x7f6f3410d380 2024-04-25T10:34:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.814+0000 7fad2ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad30104c70 0x7fad301a2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41794/0 (socket says 172.21.15.43:41794) 2024-04-25T10:34:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.814+0000 7fad2ffff640 1 -- 172.21.15.43:0/4123427899 learned_addr learned my addr 172.21.15.43:0/4123427899 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.814+0000 7fad2f7fe640 1 --2- 172.21.15.43:0/4123427899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad3010b6d0 0x7fad301a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.814+0000 7f6f3950e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f34102af0 0x7f6f3419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.815+0000 7fad3493b640 1 --2- 172.21.15.43:0/4123427899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad3010e0c0 0x7fad3019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.815+0000 7f6f38d0d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f34108a20 0x7f6f3419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.815+0000 7f6f3950e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f34102af0 0x7f6f3419a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41796/0 (socket says 172.21.15.43:41796) 2024-04-25T10:34:15.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.815+0000 7f6f38d0d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f34108a20 0x7f6f3419ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:47336/0 (socket says 172.21.15.43:47336) 2024-04-25T10:34:15.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.815+0000 7f6f3950e640 1 -- 172.21.15.43:0/2044103526 learned_addr learned my addr 172.21.15.43:0/2044103526 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:15.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.815+0000 7f6f39d0f640 1 --2- 172.21.15.43:0/2044103526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f3410d380 0x7f6f3419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.815+0000 7fad2ffff640 1 -- 172.21.15.43:0/4123427899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad3010e0c0 msgr2=0x7fad3019c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.815+0000 7fad2ffff640 1 --2- 172.21.15.43:0/4123427899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad3010e0c0 0x7fad3019c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.815+0000 7fad2ffff640 1 -- 172.21.15.43:0/4123427899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad3010b6d0 msgr2=0x7fad301a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.815+0000 7fad2ffff640 1 --2- 172.21.15.43:0/4123427899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad3010b6d0 0x7fad301a2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.815+0000 7fad2ffff640 1 -- 172.21.15.43:0/4123427899 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3019cea0 con 0x7fad30104c70 2024-04-25T10:34:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.815+0000 7fad2f7fe640 1 --2- 172.21.15.43:0/4123427899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad3010b6d0 0x7fad301a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:15.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.815+0000 7f6f3950e640 1 -- 172.21.15.43:0/2044103526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f3410d380 msgr2=0x7f6f3419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.815+0000 7f6f3950e640 1 --2- 172.21.15.43:0/2044103526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f3410d380 0x7f6f3419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.815+0000 7f6f3950e640 1 -- 172.21.15.43:0/2044103526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f34108a20 msgr2=0x7f6f3419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.815+0000 7f6f3950e640 1 --2- 172.21.15.43:0/2044103526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f34108a20 0x7f6f3419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.815+0000 7f6f3950e640 1 -- 172.21.15.43:0/2044103526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3419f610 con 0x7f6f34102af0 2024-04-25T10:34:15.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.815+0000 7f6f38d0d640 1 --2- 172.21.15.43:0/2044103526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f34108a20 0x7f6f3419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.815+0000 7fad2ffff640 1 --2- 172.21.15.43:0/4123427899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad30104c70 0x7fad301a2590 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fad2000d950 tx=0x7fad2000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.815+0000 7fad2d7fa640 1 -- 172.21.15.43:0/4123427899 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20017030 con 0x7fad30104c70 2024-04-25T10:34:15.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.815+0000 7f6f3950e640 1 --2- 172.21.15.43:0/2044103526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f34102af0 0x7f6f3419a7b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6f30032fd0 tx=0x7f6f30031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.815+0000 7fad3639e640 1 -- 172.21.15.43:0/4123427899 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad3019d190 con 0x7fad30104c70 2024-04-25T10:34:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.815+0000 7fad2d7fa640 1 -- 172.21.15.43:0/4123427899 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad20004020 con 0x7fad30104c70 2024-04-25T10:34:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.815+0000 7fad2d7fa640 1 -- 172.21.15.43:0/4123427899 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20005560 con 0x7fad30104c70 2024-04-25T10:34:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.815+0000 7fad3639e640 1 -- 172.21.15.43:0/4123427899 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad301a93e0 con 0x7fad30104c70 2024-04-25T10:34:15.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.816+0000 7f6f2a7fc640 1 -- 172.21.15.43:0/2044103526 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3003f030 con 0x7f6f34102af0 2024-04-25T10:34:15.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.816+0000 7f6f2a7fc640 1 -- 172.21.15.43:0/2044103526 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f300043d0 con 0x7f6f34102af0 2024-04-25T10:34:15.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.816+0000 7f6f3b772640 1 -- 172.21.15.43:0/2044103526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3419f8a0 con 0x7f6f34102af0 2024-04-25T10:34:15.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.816+0000 7f6f2a7fc640 1 -- 172.21.15.43:0/2044103526 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30035070 con 0x7f6f34102af0 2024-04-25T10:34:15.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.816+0000 7f6f3b772640 1 -- 172.21.15.43:0/2044103526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f3419fde0 con 0x7f6f34102af0 2024-04-25T10:34:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.818+0000 7fad3d24b740 1 Processor -- start 2024-04-25T10:34:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.818+0000 7fad3d24b740 1 -- start start 2024-04-25T10:34:15.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.818+0000 7f6f42626740 1 Processor -- start 2024-04-25T10:34:15.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.819+0000 7f6f42626740 1 -- start start 2024-04-25T10:34:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.819+0000 7fad2d7fa640 1 -- 172.21.15.43:0/4123427899 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fad200047a0 con 0x7fad30104c70 2024-04-25T10:34:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.819+0000 7fad3d24b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575f01b0 0x55c85798faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.819+0000 7fad3d24b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8575c72f0 0x55c857990030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.819+0000 7fad3d24b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575cf4d0 0x55c857990570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.819+0000 7fad3d24b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c8578f6460 con 0x55c8575cf4d0 2024-04-25T10:34:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.819+0000 7fad3d24b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c8578b0590 con 0x55c8575f01b0 2024-04-25T10:34:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.819+0000 7fad3d24b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c8578f2420 con 0x55c8575c72f0 2024-04-25T10:34:15.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.819+0000 7f6f42626740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720f7390 0x5598724a7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.819+0000 7f6f42626740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720ce4d0 0x5598724a8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.819+0000 7fad3493b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575f01b0 0x55c85798faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.819+0000 7f6f42626740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598720d66b0 0x5598724a2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.819+0000 7f6f42626740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5598723fdd80 con 0x5598720f7390 2024-04-25T10:34:15.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.819+0000 7f6f42626740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5598723b7a20 con 0x5598720ce4d0 2024-04-25T10:34:15.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.819+0000 7f6f42626740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5598723f9d40 con 0x5598720d66b0 2024-04-25T10:34:15.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.819+0000 7fad3493b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575f01b0 0x55c85798faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:47338/0 (socket says 172.21.15.43:47338) 2024-04-25T10:34:15.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.819+0000 7fad3493b640 1 -- 172.21.15.43:0/337063096 learned_addr learned my addr 172.21.15.43:0/337063096 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:15.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.819+0000 7fad2ffff640 1 --2- 172.21.15.43:0/337063096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8575c72f0 0x55c857990030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.819+0000 7fad2f7fe640 1 --2- 172.21.15.43:0/337063096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575cf4d0 0x55c857990570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.819+0000 7f6f39d0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720f7390 0x5598724a7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.819+0000 7f6f39d0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720f7390 0x5598724a7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41820/0 (socket says 172.21.15.43:41820) 2024-04-25T10:34:15.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.819+0000 7f6f38d0d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598720d66b0 0x5598724a2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.819+0000 7f6f39d0f640 1 -- 172.21.15.43:0/2183930830 learned_addr learned my addr 172.21.15.43:0/2183930830 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.819+0000 7fad3493b640 1 -- 172.21.15.43:0/337063096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8575c72f0 msgr2=0x55c857990030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.819+0000 7fad3493b640 1 --2- 172.21.15.43:0/337063096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8575c72f0 0x55c857990030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.819+0000 7fad3493b640 1 -- 172.21.15.43:0/337063096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575cf4d0 msgr2=0x55c857990570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.819+0000 7fad3493b640 1 --2- 172.21.15.43:0/337063096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575cf4d0 0x55c857990570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.819+0000 7fad3493b640 1 -- 172.21.15.43:0/337063096 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8578bd0d0 con 0x55c8575f01b0 2024-04-25T10:34:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.819+0000 7fad2f7fe640 1 --2- 172.21.15.43:0/337063096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575cf4d0 0x55c857990570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:15.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.819+0000 7f6f39d0f640 1 -- 172.21.15.43:0/2183930830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598720d66b0 msgr2=0x5598724a2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.819+0000 7f6f39d0f640 1 --2- 172.21.15.43:0/2183930830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598720d66b0 0x5598724a2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.819+0000 7f6f39d0f640 1 -- 172.21.15.43:0/2183930830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720ce4d0 msgr2=0x5598724a8430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:34:15.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.819+0000 7f6f39d0f640 1 --2- 172.21.15.43:0/2183930830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720ce4d0 0x5598724a8430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.819+0000 7f6f39d0f640 1 -- 172.21.15.43:0/2183930830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598723c2c30 con 0x5598720f7390 2024-04-25T10:34:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.820+0000 7fad3493b640 1 --2- 172.21.15.43:0/337063096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575f01b0 0x55c85798faf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fad28009f60 tx=0x7fad28009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=923fdfc375fa1a92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:15.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.820+0000 7f6f39d0f640 1 --2- 172.21.15.43:0/2183930830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720f7390 0x5598724a7ef0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009f60 tx=0x7f6f2c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169b9a05a8cbf094 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:15.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.820+0000 7f6f2a7fc640 1 -- 172.21.15.43:0/2044103526 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f6f30042030 con 0x7f6f34102af0 2024-04-25T10:34:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.820+0000 7fad167fc640 1 -- 172.21.15.43:0/337063096 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2801e070 con 0x55c8575f01b0 2024-04-25T10:34:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.820+0000 7fad2d7fa640 1 --2- 172.21.15.43:0/4123427899 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fad180afc50 0x7fad180b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.820+0000 7fad167fc640 1 -- 172.21.15.43:0/337063096 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad28003c30 con 0x55c8575f01b0 2024-04-25T10:34:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.820+0000 7fad167fc640 1 -- 172.21.15.43:0/337063096 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2800f8a0 con 0x55c8575f01b0 2024-04-25T10:34:15.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.820+0000 7f6f1f7fe640 1 -- 172.21.15.43:0/2183930830 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c01e070 con 0x5598720f7390 2024-04-25T10:34:15.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.820+0000 7f6f1f7fe640 1 -- 172.21.15.43:0/2183930830 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f2c003c30 con 0x5598720f7390 2024-04-25T10:34:15.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.820+0000 7f6f1f7fe640 1 -- 172.21.15.43:0/2183930830 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c00f860 con 0x5598720f7390 2024-04-25T10:34:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.820+0000 7fad2ffff640 1 --2- 172.21.15.43:0/4123427899 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fad180afc50 0x7fad180b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.820+0000 7fad2d7fa640 1 -- 172.21.15.43:0/4123427899 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fad200de930 con 0x7fad30104c70 2024-04-25T10:34:15.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.821+0000 7f6f2a7fc640 1 --2- 172.21.15.43:0/2044103526 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6f200afce0 0x7f6f200b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.821+0000 7f6f3950e640 1 --2- 172.21.15.43:0/2044103526 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6f200afce0 0x7f6f200b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.821+0000 7f6f2a7fc640 1 -- 172.21.15.43:0/2044103526 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6f300fef30 con 0x7f6f34102af0 2024-04-25T10:34:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.821+0000 7fad3d24b740 1 -- 172.21.15.43:0/337063096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575f01b0 msgr2=0x55c85798faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.821+0000 7fad2ffff640 1 --2- 172.21.15.43:0/4123427899 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fad180afc50 0x7fad180b2160 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fad200dc160 tx=0x7fad200e1f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.821+0000 7fad3d24b740 1 --2- 172.21.15.43:0/337063096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575f01b0 0x55c85798faf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fad28009f60 tx=0x7fad28009f90 comp rx=0 tx=0).stop 2024-04-25T10:34:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.821+0000 7fad3d24b740 1 -- 172.21.15.43:0/337063096 shutdown_connections 2024-04-25T10:34:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.821+0000 7fad3d24b740 1 --2- 172.21.15.43:0/337063096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575cf4d0 0x55c857990570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.821+0000 7fad3d24b740 1 --2- 172.21.15.43:0/337063096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8575c72f0 0x55c857990030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.821+0000 7fad3d24b740 1 --2- 172.21.15.43:0/337063096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575f01b0 0x55c85798faf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.821+0000 7fad3d24b740 1 -- 172.21.15.43:0/337063096 >> 172.21.15.43:0/337063096 conn(0x55c8575e4300 msgr2=0x55c85798c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:15.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.821+0000 7f6f42626740 1 -- 172.21.15.43:0/2183930830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720f7390 msgr2=0x5598724a7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.821+0000 7f6f42626740 1 --2- 172.21.15.43:0/2183930830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720f7390 0x5598724a7ef0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009f60 tx=0x7f6f2c009f90 comp rx=0 tx=0).stop 2024-04-25T10:34:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.821+0000 7fad3d24b740 1 -- 172.21.15.43:0/337063096 shutdown_connections 2024-04-25T10:34:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.821+0000 7fad3d24b740 1 -- 172.21.15.43:0/337063096 wait complete. 2024-04-25T10:34:15.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.821+0000 7f6f42626740 1 -- 172.21.15.43:0/2183930830 shutdown_connections 2024-04-25T10:34:15.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.821+0000 7f6f42626740 1 --2- 172.21.15.43:0/2183930830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720ce4d0 0x5598724a8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.821+0000 7f6f42626740 1 --2- 172.21.15.43:0/2183930830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598720d66b0 0x5598724a2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.821+0000 7f6f42626740 1 --2- 172.21.15.43:0/2183930830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720f7390 0x5598724a7ef0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.821+0000 7f6f42626740 1 -- 172.21.15.43:0/2183930830 >> 172.21.15.43:0/2183930830 conn(0x5598720eb4e0 msgr2=0x559872494960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:15.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.821+0000 7f6f3950e640 1 --2- 172.21.15.43:0/2044103526 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6f200afce0 0x7f6f200b21f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5598724958c0 tx=0x7f6f30094040 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:15.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.821+0000 7f6f42626740 1 -- 172.21.15.43:0/2183930830 shutdown_connections 2024-04-25T10:34:15.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.821+0000 7f6f42626740 1 -- 172.21.15.43:0/2183930830 wait complete. 2024-04-25T10:34:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.822+0000 7fad3d24b740 1 -- start start 2024-04-25T10:34:15.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.822+0000 7f6f42626740 1 -- start start 2024-04-25T10:34:15.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.822+0000 7f6f42626740 1 Processor -- start 2024-04-25T10:34:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.822+0000 7fad3d24b740 1 Processor -- start 2024-04-25T10:34:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.822+0000 7fad3d24b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575c72f0 0x55c85799a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.822+0000 7fad3d24b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575cf4d0 0x55c85799a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.822+0000 7fad3d24b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8575f01b0 0x55c8579a0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.822+0000 7fad3d24b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c85799cc20 con 0x55c8575cf4d0 2024-04-25T10:34:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.822+0000 7fad3d24b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c85787db60 con 0x55c8575c72f0 2024-04-25T10:34:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.822+0000 7fad3d24b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c85799cda0 con 0x55c8575f01b0 2024-04-25T10:34:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.822+0000 7fad3d24b740 10 client.0 _ll_register_callbacks cb 0x55c8578a22b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.822+0000 7fad2f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575cf4d0 0x55c85799a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.822+0000 7f6f42626740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720ce4d0 0x5598724b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.822+0000 7fad2f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575cf4d0 0x55c85799a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41824/0 (socket says 172.21.15.43:41824) 2024-04-25T10:34:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.822+0000 7fad2f7fe640 1 -- 172.21.15.43:0/1545121972 learned_addr learned my addr 172.21.15.43:0/1545121972 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.822+0000 7fad3493b640 1 --2- 172.21.15.43:0/1545121972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575c72f0 0x55c85799a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.822+0000 7f6f42626740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720d66b0 0x5598724b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.823+0000 7f6f42626740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598720f7390 0x5598724abda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.823+0000 7f6f42626740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5598724a2f50 con 0x5598720d66b0 2024-04-25T10:34:15.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.823+0000 7f6f42626740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559872384dc0 con 0x5598720ce4d0 2024-04-25T10:34:15.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.823+0000 7f6f42626740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5598724a30d0 con 0x5598720f7390 2024-04-25T10:34:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.823+0000 7fad3493b640 1 --2- 172.21.15.43:0/1545121972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8575f01b0 0x55c8579a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.823+0000 7f6f39d0f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720ce4d0 0x5598724b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.823+0000 7f6f39d0f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720ce4d0 0x5598724b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:47354/0 (socket says 172.21.15.43:47354) 2024-04-25T10:34:15.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.823+0000 7f6f39d0f640 1 -- 172.21.15.43:0/2500914549 learned_addr learned my addr 172.21.15.43:0/2500914549 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:15.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.823+0000 7f6f42626740 10 client.0 _ll_register_callbacks cb 0x5598723a0600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:15.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.823+0000 7f6f39d0f640 1 --2- 172.21.15.43:0/2500914549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598720f7390 0x5598724abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.823+0000 7f6f38d0d640 1 --2- 172.21.15.43:0/2500914549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720d66b0 0x5598724b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.823+0000 7fad2f7fe640 1 -- 172.21.15.43:0/1545121972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8575f01b0 msgr2=0x55c8579a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.823+0000 7fad2f7fe640 1 --2- 172.21.15.43:0/1545121972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8575f01b0 0x55c8579a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.823+0000 7fad2f7fe640 1 -- 172.21.15.43:0/1545121972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575c72f0 msgr2=0x55c85799a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.823+0000 7fad2f7fe640 1 --2- 172.21.15.43:0/1545121972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575c72f0 0x55c85799a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.823+0000 7fad2f7fe640 1 -- 172.21.15.43:0/1545121972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c857608210 con 0x55c8575cf4d0 2024-04-25T10:34:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.823+0000 7fad3493b640 1 --2- 172.21.15.43:0/1545121972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575c72f0 0x55c85799a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:15.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.823+0000 7f6f39d0f640 1 -- 172.21.15.43:0/2500914549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598720f7390 msgr2=0x5598724abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.823+0000 7f6f39d0f640 1 --2- 172.21.15.43:0/2500914549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598720f7390 0x5598724abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.823+0000 7f6f39d0f640 1 -- 172.21.15.43:0/2500914549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720d66b0 msgr2=0x5598724b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.823+0000 7f6f39d0f640 1 --2- 172.21.15.43:0/2500914549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720d66b0 0x5598724b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.823+0000 7f6f39d0f640 1 -- 172.21.15.43:0/2500914549 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55987210f3f0 con 0x5598720ce4d0 2024-04-25T10:34:15.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.823+0000 7f6f38d0d640 1 --2- 172.21.15.43:0/2500914549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720d66b0 0x5598724b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.823+0000 7fad2f7fe640 1 --2- 172.21.15.43:0/1545121972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575cf4d0 0x55c85799a750 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fad24009900 tx=0x7fad240316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea88bc3fc401088 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.823+0000 7fad0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:15.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.823+0000 7f6f39d0f640 1 --2- 172.21.15.43:0/2500914549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720ce4d0 0x5598724b19b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c004810 tx=0x7f6f2c00f480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1391de83cd0833e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:15.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.823+0000 7f6f1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-04-25T10:34:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.823+0000 7fad0f7fe640 1 -- 172.21.15.43:0/1545121972 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24035040 con 0x55c8575cf4d0 2024-04-25T10:34:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.823+0000 7fad0f7fe640 1 -- 172.21.15.43:0/1545121972 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fad24004460 con 0x55c8575cf4d0 2024-04-25T10:34:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.823+0000 7fad3d24b740 1 -- 172.21.15.43:0/1545121972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad28003c30 con 0x55c8575cf4d0 2024-04-25T10:34:15.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.824+0000 7f6f1cff9640 1 -- 172.21.15.43:0/2500914549 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c02e060 con 0x5598720ce4d0 2024-04-25T10:34:15.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.824+0000 7f6f1cff9640 1 -- 172.21.15.43:0/2500914549 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6f2c00f560 con 0x5598720ce4d0 2024-04-25T10:34:15.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.824+0000 7f6f42626740 1 -- 172.21.15.43:0/2500914549 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f2c003c30 con 0x5598720ce4d0 2024-04-25T10:34:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.824+0000 7fad0f7fe640 1 -- 172.21.15.43:0/1545121972 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fad24031900 con 0x55c8575cf4d0 2024-04-25T10:34:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.824+0000 7fad3d24b740 10 client.15930 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.824+0000 7fad3d24b740 1 -- 172.21.15.43:0/1545121972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c8576d4620 con 0x55c8575cf4d0 2024-04-25T10:34:15.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.824+0000 7f6f1cff9640 1 -- 172.21.15.43:0/2500914549 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6f2c01e070 con 0x5598720ce4d0 2024-04-25T10:34:15.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.824+0000 7f6f42626740 10 client.25637 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:15.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.824+0000 7f6f42626740 1 -- 172.21.15.43:0/2500914549 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5598721d97d0 con 0x5598720ce4d0 2024-04-25T10:34:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.824+0000 7fad0f7fe640 1 -- 172.21.15.43:0/1545121972 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad28003c30 con 0x55c8575cf4d0 2024-04-25T10:34:15.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.824+0000 7f6f1cff9640 1 -- 172.21.15.43:0/2500914549 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f2c003c30 con 0x5598720ce4d0 2024-04-25T10:34:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.824+0000 7fad0f7fe640 1 -- 172.21.15.43:0/1545121972 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fad240035f0 con 0x55c8575cf4d0 2024-04-25T10:34:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.824+0000 7fad3d24b740 10 client.15930 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.824+0000 7fad3d24b740 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.824+0000 7fad3d24b740 20 client.15930 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:15.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.824+0000 7f6f1cff9640 1 -- 172.21.15.43:0/2500914549 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6f2c0335a0 con 0x5598720ce4d0 2024-04-25T10:34:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.824+0000 7fad3d24b740 1 --2- 172.21.15.43:0/1545121972 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c8575aed80 0x55c8579ac730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.824+0000 7fad3d24b740 4 client.15930 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:34:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.824+0000 7fad3d24b740 1 -- 172.21.15.43:0/1545121972 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c857904070 con 0x55c8575aed80 2024-04-25T10:34:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.824+0000 7f6f42626740 10 client.25637 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.824+0000 7f6f42626740 10 client.25637 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:34:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.824+0000 7f6f42626740 20 client.25637 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.824+0000 7f6f42626740 1 --2- 172.21.15.43:0/2500914549 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x5598720b5f60 0x5598724bdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.824+0000 7f6f42626740 4 client.25637 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:34:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.824+0000 7f6f42626740 1 -- 172.21.15.43:0/2500914549 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55987240b990 con 0x5598720b5f60 2024-04-25T10:34:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.824+0000 7fad3493b640 1 --2- 172.21.15.43:0/1545121972 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c8575aed80 0x55c8579ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.825+0000 7f6f38d0d640 1 --2- 172.21.15.43:0/2500914549 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x5598720b5f60 0x5598724bdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.825+0000 7fad0f7fe640 10 client.15930 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.825+0000 7fad3493b640 1 --2- 172.21.15.43:0/1545121972 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c8575aed80 0x55c8579ac730 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cfb4b21d81879fc server_cookie=bbb89c863b78e020 in_seq=0 out_seq=0 2024-04-25T10:34:15.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.825+0000 7f6f38d0d640 1 --2- 172.21.15.43:0/2500914549 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x5598720b5f60 0x5598724bdd50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2612f4db34c782a8 server_cookie=a20b9956c6d449e9 in_seq=0 out_seq=0 2024-04-25T10:34:15.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.825+0000 7f6f1cff9640 10 client.25637 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:34:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.825+0000 7fad0f7fe640 1 -- 172.21.15.43:0/1545121972 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c857904070 con 0x55c8575aed80 2024-04-25T10:34:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.825+0000 7fad0f7fe640 10 client.15930 handle_command_reply: tid=0 2024-04-25T10:34:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.825+0000 7fad3d24b740 1 client.15930 shutdown 2024-04-25T10:34:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.826+0000 7fad3d24b740 1 -- 172.21.15.43:0/1545121972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575cf4d0 msgr2=0x55c85799a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.826+0000 7fad3d24b740 1 --2- 172.21.15.43:0/1545121972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575cf4d0 0x55c85799a750 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fad24009900 tx=0x7fad240316d0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.826+0000 7fad3d24b740 1 -- 172.21.15.43:0/1545121972 shutdown_connections 2024-04-25T10:34:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.826+0000 7fad3d24b740 1 --2- 172.21.15.43:0/1545121972 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c8575aed80 0x55c8579ac730 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.826+0000 7fad3d24b740 1 --2- 172.21.15.43:0/1545121972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8575f01b0 0x55c8579a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.826+0000 7fad3d24b740 1 --2- 172.21.15.43:0/1545121972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575cf4d0 0x55c85799a750 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.826+0000 7fad0f7fe640 0 client.15930 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.826+0000 7fad3d24b740 1 --2- 172.21.15.43:0/1545121972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575c72f0 0x55c85799a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.826+0000 7fad3d24b740 1 -- 172.21.15.43:0/1545121972 >> 172.21.15.43:0/1545121972 conn(0x55c8575e4300 msgr2=0x55c85798f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.826+0000 7f6f1cff9640 1 -- 172.21.15.43:0/2500914549 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55987240b990 con 0x5598720b5f60 2024-04-25T10:34:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.826+0000 7f6f1cff9640 10 client.25637 handle_command_reply: tid=0 2024-04-25T10:34:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.826+0000 7fad3d24b740 1 -- 172.21.15.43:0/1545121972 shutdown_connections 2024-04-25T10:34:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.826+0000 7fad3d24b740 1 -- 172.21.15.43:0/1545121972 wait complete. 2024-04-25T10:34:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.826+0000 7f6f42626740 1 client.25637 shutdown 2024-04-25T10:34:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.826+0000 7fad3d24b740 20 client.15930 trim_cache size 0 max 16384 2024-04-25T10:34:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.826+0000 7f6f42626740 1 -- 172.21.15.43:0/2500914549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720ce4d0 msgr2=0x5598724b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.826+0000 7f6f42626740 1 --2- 172.21.15.43:0/2500914549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720ce4d0 0x5598724b19b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c004810 tx=0x7f6f2c00f480 comp rx=0 tx=0).stop 2024-04-25T10:34:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.826+0000 7f6f42626740 1 -- 172.21.15.43:0/2500914549 shutdown_connections 2024-04-25T10:34:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.826+0000 7f6f42626740 1 --2- 172.21.15.43:0/2500914549 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x5598720b5f60 0x5598724bdd50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.826+0000 7f6f42626740 1 --2- 172.21.15.43:0/2500914549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598720f7390 0x5598724abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.826+0000 7f6f42626740 1 --2- 172.21.15.43:0/2500914549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720d66b0 0x5598724b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.826+0000 7f6f1cff9640 0 client.25637 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:34:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.826+0000 7f6f42626740 1 --2- 172.21.15.43:0/2500914549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720ce4d0 0x5598724b19b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.826+0000 7f6f42626740 1 -- 172.21.15.43:0/2500914549 >> 172.21.15.43:0/2500914549 conn(0x5598720eb4e0 msgr2=0x559872494960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.826+0000 7f6f42626740 1 -- 172.21.15.43:0/2500914549 shutdown_connections 2024-04-25T10:34:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.826+0000 7f6f42626740 1 -- 172.21.15.43:0/2500914549 wait complete. 2024-04-25T10:34:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.826+0000 7f6f42626740 20 client.25637 trim_cache size 0 max 16384 2024-04-25T10:34:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.840+0000 7f6f42626740 1 Processor -- start 2024-04-25T10:34:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.840+0000 7f6f42626740 1 -- start start 2024-04-25T10:34:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.840+0000 7f6f42626740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720ce4d0 0x5598724abd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.840+0000 7f6f42626740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598720d66b0 0x559872494620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.840+0000 7f6f42626740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720f7390 0x5598724c3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.840+0000 7f6f42626740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341b0070 con 0x5598720ce4d0 2024-04-25T10:34:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.840+0000 7f6f42626740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341ad940 con 0x5598720f7390 2024-04-25T10:34:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.840+0000 7f6f42626740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341ab790 con 0x5598720d66b0 2024-04-25T10:34:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.841+0000 7f6f39d0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720ce4d0 0x5598724abd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.841+0000 7f6f39d0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720ce4d0 0x5598724abd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41848/0 (socket says 172.21.15.43:41848) 2024-04-25T10:34:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.841+0000 7f6f39d0f640 1 -- 172.21.15.43:0/2149452258 learned_addr learned my addr 172.21.15.43:0/2149452258 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.841+0000 7f6f39d0f640 1 --2- 172.21.15.43:0/2149452258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720f7390 0x5598724c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.841+0000 7f6f39d0f640 1 -- 172.21.15.43:0/2149452258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598720d66b0 msgr2=0x559872494620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.841+0000 7f6f39d0f640 1 --2- 172.21.15.43:0/2149452258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598720d66b0 0x559872494620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.841+0000 7f6f39d0f640 1 -- 172.21.15.43:0/2149452258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720f7390 msgr2=0x5598724c3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.841+0000 7f6f39d0f640 1 --2- 172.21.15.43:0/2149452258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720f7390 0x5598724c3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.841+0000 7f6f39d0f640 1 -- 172.21.15.43:0/2149452258 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598724a2e30 con 0x5598720ce4d0 2024-04-25T10:34:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.841+0000 7f6f39d0f640 1 --2- 172.21.15.43:0/2149452258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720ce4d0 0x5598724abd40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c005d70 tx=0x7f6f2c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad6a178d6ac57a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.841+0000 7f6f1effd640 1 -- 172.21.15.43:0/2149452258 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c02e050 con 0x5598720ce4d0 2024-04-25T10:34:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.841+0000 7f6f1effd640 1 -- 172.21.15.43:0/2149452258 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f2c033ce0 con 0x5598720ce4d0 2024-04-25T10:34:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.841+0000 7f6f1effd640 1 -- 172.21.15.43:0/2149452258 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c00f4f0 con 0x5598720ce4d0 2024-04-25T10:34:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.842+0000 7fad3d24b740 1 Processor -- start 2024-04-25T10:34:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.842+0000 7fad3d24b740 1 -- start start 2024-04-25T10:34:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.842+0000 7fad3d24b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8575c72f0 0x55c8579b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.842+0000 7fad3d24b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575cf4d0 0x55c8579b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.842+0000 7fad3d24b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575f01b0 0x55c8579937c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.842+0000 7fad3d24b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fad301b4530 con 0x55c8575f01b0 2024-04-25T10:34:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.842+0000 7fad3d24b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fad301b1e00 con 0x55c8575cf4d0 2024-04-25T10:34:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.842+0000 7fad3d24b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fad301afc50 con 0x55c8575c72f0 2024-04-25T10:34:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.842+0000 7f6f42626740 1 -- 172.21.15.43:0/2149452258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720ce4d0 msgr2=0x5598724abd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.842+0000 7f6f42626740 1 --2- 172.21.15.43:0/2149452258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720ce4d0 0x5598724abd40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c005d70 tx=0x7f6f2c009900 comp rx=0 tx=0).stop 2024-04-25T10:34:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.842+0000 7f6f42626740 1 -- 172.21.15.43:0/2149452258 shutdown_connections 2024-04-25T10:34:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.842+0000 7f6f42626740 1 --2- 172.21.15.43:0/2149452258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720f7390 0x5598724c3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.842+0000 7f6f42626740 1 --2- 172.21.15.43:0/2149452258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598720d66b0 0x559872494620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.842+0000 7f6f42626740 1 --2- 172.21.15.43:0/2149452258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720ce4d0 0x5598724abd40 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.842+0000 7f6f42626740 1 -- 172.21.15.43:0/2149452258 >> 172.21.15.43:0/2149452258 conn(0x5598720eb4e0 msgr2=0x5598724ab630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.842+0000 7fad3493b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575f01b0 0x55c8579937c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.842+0000 7fad2f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575cf4d0 0x55c8579b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.842+0000 7fad3493b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575f01b0 0x55c8579937c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41860/0 (socket says 172.21.15.43:41860) 2024-04-25T10:34:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.842+0000 7fad2f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575cf4d0 0x55c8579b2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:47372/0 (socket says 172.21.15.43:47372) 2024-04-25T10:34:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.842+0000 7fad3493b640 1 -- 172.21.15.43:0/2490245040 learned_addr learned my addr 172.21.15.43:0/2490245040 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.842+0000 7fad3493b640 1 --2- 172.21.15.43:0/2490245040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8575c72f0 0x55c8579b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.842+0000 7f6f42626740 1 -- 172.21.15.43:0/2149452258 shutdown_connections 2024-04-25T10:34:15.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.842+0000 7f6f42626740 1 -- 172.21.15.43:0/2149452258 wait complete. 2024-04-25T10:34:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.843+0000 7fad2f7fe640 1 -- 172.21.15.43:0/2490245040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8575c72f0 msgr2=0x55c8579b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.843+0000 7fad2f7fe640 1 --2- 172.21.15.43:0/2490245040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8575c72f0 0x55c8579b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.843+0000 7fad2f7fe640 1 -- 172.21.15.43:0/2490245040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575f01b0 msgr2=0x55c8579937c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.843+0000 7fad2f7fe640 1 --2- 172.21.15.43:0/2490245040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575f01b0 0x55c8579937c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.843+0000 7fad2f7fe640 1 -- 172.21.15.43:0/2490245040 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c857998530 con 0x55c8575cf4d0 2024-04-25T10:34:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.843+0000 7fad3493b640 1 --2- 172.21.15.43:0/2490245040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575f01b0 0x55c8579937c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.843+0000 7f6f42626740 1 -- start start 2024-04-25T10:34:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.843+0000 7fad2f7fe640 1 --2- 172.21.15.43:0/2490245040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575cf4d0 0x55c8579b2470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fad24003b10 tx=0x7fad2402a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2772b6079a4f206e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.843+0000 7f6f42626740 1 Processor -- start 2024-04-25T10:34:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.843+0000 7fad15ffb640 1 -- 172.21.15.43:0/2490245040 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24035030 con 0x55c8575cf4d0 2024-04-25T10:34:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.843+0000 7fad15ffb640 1 -- 172.21.15.43:0/2490245040 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad24040070 con 0x55c8575cf4d0 2024-04-25T10:34:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.843+0000 7fad15ffb640 1 -- 172.21.15.43:0/2490245040 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24045930 con 0x55c8575cf4d0 2024-04-25T10:34:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.843+0000 7f6f42626740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720ce4d0 0x5598724b8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.843+0000 7f6f42626740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598720d66b0 0x5598724b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.843+0000 7f6f42626740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720f7390 0x5598724b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.843+0000 7f6f42626740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341abe00 con 0x5598720f7390 2024-04-25T10:34:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.843+0000 7f6f42626740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341ad270 con 0x5598720ce4d0 2024-04-25T10:34:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.843+0000 7f6f42626740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341ac4d0 con 0x5598720d66b0 2024-04-25T10:34:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.843+0000 7f6f42626740 10 client.0 _ll_register_callbacks cb 0x7f6f2c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.843+0000 7f6f39d0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720f7390 0x5598724b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.843+0000 7f6f39d0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720f7390 0x5598724b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41864/0 (socket says 172.21.15.43:41864) 2024-04-25T10:34:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.844+0000 7f6f39d0f640 1 -- 172.21.15.43:0/1416683009 learned_addr learned my addr 172.21.15.43:0/1416683009 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.844+0000 7f6f38d0d640 1 --2- 172.21.15.43:0/1416683009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598720d66b0 0x5598724b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.844+0000 7f6f39d0f640 1 --2- 172.21.15.43:0/1416683009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720ce4d0 0x5598724b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.844+0000 7fad3d24b740 1 -- 172.21.15.43:0/2490245040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575cf4d0 msgr2=0x55c8579b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.844+0000 7fad3d24b740 1 --2- 172.21.15.43:0/2490245040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575cf4d0 0x55c8579b2470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fad24003b10 tx=0x7fad2402a3f0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.844+0000 7fad3d24b740 1 -- 172.21.15.43:0/2490245040 shutdown_connections 2024-04-25T10:34:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.844+0000 7fad3d24b740 1 --2- 172.21.15.43:0/2490245040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575cf4d0 0x55c8579b2470 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.844+0000 7fad3d24b740 1 --2- 172.21.15.43:0/2490245040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575f01b0 0x55c8579937c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.844+0000 7fad3d24b740 1 --2- 172.21.15.43:0/2490245040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8575c72f0 0x55c8579b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.844+0000 7fad3d24b740 1 -- 172.21.15.43:0/2490245040 >> 172.21.15.43:0/2490245040 conn(0x55c8575e4300 msgr2=0x55c8579988b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.844+0000 7f6f39d0f640 1 -- 172.21.15.43:0/1416683009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598720d66b0 msgr2=0x5598724b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.844+0000 7f6f39d0f640 1 --2- 172.21.15.43:0/1416683009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598720d66b0 0x5598724b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.844+0000 7f6f39d0f640 1 -- 172.21.15.43:0/1416683009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720ce4d0 msgr2=0x5598724b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.844+0000 7f6f39d0f640 1 --2- 172.21.15.43:0/1416683009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720ce4d0 0x5598724b8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.844+0000 7f6f39d0f640 1 -- 172.21.15.43:0/1416683009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598724b52f0 con 0x5598720f7390 2024-04-25T10:34:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.844+0000 7fad3d24b740 1 -- 172.21.15.43:0/2490245040 shutdown_connections 2024-04-25T10:34:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.844+0000 7f6f38d0d640 1 --2- 172.21.15.43:0/1416683009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598720d66b0 0x5598724b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.844+0000 7fad3d24b740 1 -- 172.21.15.43:0/2490245040 wait complete. 2024-04-25T10:34:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.844+0000 7f6f39d0f640 1 --2- 172.21.15.43:0/1416683009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720f7390 0x5598724b4bb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009d30 tx=0x7f6f2c035040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6736259a8c806b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.844+0000 7f6f1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.844+0000 7fad3d24b740 1 -- start start 2024-04-25T10:34:15.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.844+0000 7f6f1cff9640 1 -- 172.21.15.43:0/1416683009 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c02e030 con 0x5598720f7390 2024-04-25T10:34:15.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.844+0000 7f6f1cff9640 1 -- 172.21.15.43:0/1416683009 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6f2c004490 con 0x5598720f7390 2024-04-25T10:34:15.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.844+0000 7f6f42626740 1 -- 172.21.15.43:0/1416683009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f2c033ce0 con 0x5598720f7390 2024-04-25T10:34:15.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.845+0000 7f6f1cff9640 1 -- 172.21.15.43:0/1416683009 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6f2c004690 con 0x5598720f7390 2024-04-25T10:34:15.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.845+0000 7f6f42626740 10 client.15942 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.845+0000 7fad3d24b740 1 Processor -- start 2024-04-25T10:34:15.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.845+0000 7f6f42626740 1 -- 172.21.15.43:0/1416683009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559872524f10 con 0x5598720f7390 2024-04-25T10:34:15.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.845+0000 7f6f1cff9640 1 -- 172.21.15.43:0/1416683009 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f2c033ce0 con 0x5598720f7390 2024-04-25T10:34:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.845+0000 7fad3d24b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8575c72f0 0x55c8579a3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.845+0000 7fad3d24b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575cf4d0 0x55c8579a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.845+0000 7fad3d24b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575f01b0 0x55c857a0cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.845+0000 7fad3d24b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fad301b02c0 con 0x55c8575f01b0 2024-04-25T10:34:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.845+0000 7fad3d24b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fad301b1730 con 0x55c8575cf4d0 2024-04-25T10:34:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.845+0000 7fad3d24b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fad301b0990 con 0x55c8575c72f0 2024-04-25T10:34:15.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.845+0000 7f6f1cff9640 1 -- 172.21.15.43:0/1416683009 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6f2c029440 con 0x5598720f7390 2024-04-25T10:34:15.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.845+0000 7f6f42626740 10 client.15942 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:15.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.845+0000 7f6f42626740 10 client.15942 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:34:15.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.845+0000 7f6f42626740 20 client.15942 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:15.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.845+0000 7f6f42626740 1 --2- 172.21.15.43:0/1416683009 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55987208b3a0 0x559872527550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.845+0000 7fad3d24b740 10 client.0 _ll_register_callbacks cb 0x7fad24009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:15.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.845+0000 7f6f42626740 4 client.15942 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:34:15.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.845+0000 7f6f42626740 1 -- 172.21.15.43:0/1416683009 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559872405800 con 0x55987208b3a0 2024-04-25T10:34:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.845+0000 7fad3493b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575f01b0 0x55c857a0cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.845+0000 7fad3493b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575f01b0 0x55c857a0cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41876/0 (socket says 172.21.15.43:41876) 2024-04-25T10:34:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.845+0000 7fad3493b640 1 -- 172.21.15.43:0/2706154685 learned_addr learned my addr 172.21.15.43:0/2706154685 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.845+0000 7fad2f7fe640 1 --2- 172.21.15.43:0/2706154685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575cf4d0 0x55c8579a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.845+0000 7fad3493b640 1 --2- 172.21.15.43:0/2706154685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8575c72f0 0x55c8579a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.845+0000 7fad3493b640 1 -- 172.21.15.43:0/2706154685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8575c72f0 msgr2=0x55c8579a3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.845+0000 7fad3493b640 1 --2- 172.21.15.43:0/2706154685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8575c72f0 0x55c8579a3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.845+0000 7fad3493b640 1 -- 172.21.15.43:0/2706154685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575cf4d0 msgr2=0x55c8579a3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.845+0000 7fad3493b640 1 --2- 172.21.15.43:0/2706154685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575cf4d0 0x55c8579a3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.845+0000 7f6f38d0d640 1 --2- 172.21.15.43:0/1416683009 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55987208b3a0 0x559872527550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.845+0000 7fad3493b640 1 -- 172.21.15.43:0/2706154685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c857a0d100 con 0x55c8575f01b0 2024-04-25T10:34:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.846+0000 7fad3493b640 1 --2- 172.21.15.43:0/2706154685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575f01b0 0x55c857a0cb90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fad28006010 tx=0x7fad2802a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f712616dd331924 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.846+0000 7fad0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.846+0000 7fad0f7fe640 1 -- 172.21.15.43:0/2706154685 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28035020 con 0x55c8575f01b0 2024-04-25T10:34:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.846+0000 7fad0f7fe640 1 -- 172.21.15.43:0/2706154685 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fad280045e0 con 0x55c8575f01b0 2024-04-25T10:34:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.846+0000 7fad3d24b740 1 -- 172.21.15.43:0/2706154685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad24040070 con 0x55c8575f01b0 2024-04-25T10:34:15.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.846+0000 7f6f1cff9640 10 client.15942 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:34:15.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.846+0000 7f6f38d0d640 1 --2- 172.21.15.43:0/1416683009 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55987208b3a0 0x559872527550 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38d0d365c88b0b10 server_cookie=b100861d6eca9fc7 in_seq=0 out_seq=0 2024-04-25T10:34:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.846+0000 7fad0f7fe640 1 -- 172.21.15.43:0/2706154685 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fad2801e070 con 0x55c8575f01b0 2024-04-25T10:34:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.846+0000 7fad3d24b740 10 client.15945 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.846+0000 7fad3d24b740 1 -- 172.21.15.43:0/2706154685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c857a14410 con 0x55c8575f01b0 2024-04-25T10:34:15.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.846+0000 7f6f1cff9640 1 -- 172.21.15.43:0/1416683009 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559872405800 con 0x55987208b3a0 2024-04-25T10:34:15.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.846+0000 7f6f1cff9640 10 client.15942 handle_command_reply: tid=0 2024-04-25T10:34:15.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.846+0000 7f6f42626740 1 client.15942 shutdown 2024-04-25T10:34:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.846+0000 7fad0f7fe640 1 -- 172.21.15.43:0/2706154685 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad24040070 con 0x55c8575f01b0 2024-04-25T10:34:15.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.846+0000 7f6f42626740 1 -- 172.21.15.43:0/1416683009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720f7390 msgr2=0x5598724b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.846+0000 7f6f42626740 1 --2- 172.21.15.43:0/1416683009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720f7390 0x5598724b4bb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009d30 tx=0x7f6f2c035040 comp rx=0 tx=0).stop 2024-04-25T10:34:15.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.846+0000 7f6f42626740 1 -- 172.21.15.43:0/1416683009 shutdown_connections 2024-04-25T10:34:15.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.846+0000 7f6f42626740 1 --2- 172.21.15.43:0/1416683009 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55987208b3a0 0x559872527550 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.846+0000 7f6f42626740 1 --2- 172.21.15.43:0/1416683009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598720f7390 0x5598724b4bb0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.846+0000 7f6f1cff9640 0 client.15942 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:34:15.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.846+0000 7f6f42626740 1 --2- 172.21.15.43:0/1416683009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598720d66b0 0x5598724b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.846+0000 7f6f42626740 1 --2- 172.21.15.43:0/1416683009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598720ce4d0 0x5598724b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.846+0000 7f6f42626740 1 -- 172.21.15.43:0/1416683009 >> 172.21.15.43:0/1416683009 conn(0x5598720eb4e0 msgr2=0x5598724a14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.846+0000 7fad0f7fe640 1 -- 172.21.15.43:0/2706154685 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fad28019430 con 0x55c8575f01b0 2024-04-25T10:34:15.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.847+0000 7f6f42626740 1 -- 172.21.15.43:0/1416683009 shutdown_connections 2024-04-25T10:34:15.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.847+0000 7f6f42626740 1 -- 172.21.15.43:0/1416683009 wait complete. 2024-04-25T10:34:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.847+0000 7fad3d24b740 10 client.15945 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.847+0000 7fad3d24b740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.847+0000 7fad3d24b740 20 client.15945 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.847+0000 7fad3d24b740 1 --2- 172.21.15.43:0/2706154685 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c8575841c0 0x55c857a16950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.847+0000 7fad3d24b740 4 client.15945 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:34:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.847+0000 7fad3d24b740 1 -- 172.21.15.43:0/2706154685 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8578fdee0 con 0x55c8575841c0 2024-04-25T10:34:15.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.847+0000 7f6f42626740 20 client.15942 trim_cache size 0 max 16384 2024-04-25T10:34:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.847+0000 7fad2f7fe640 1 --2- 172.21.15.43:0/2706154685 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c8575841c0 0x55c857a16950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:15.891 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:34:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.847+0000 7fad0f7fe640 10 client.15945 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.847+0000 7fad2f7fe640 1 --2- 172.21.15.43:0/2706154685 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c8575841c0 0x55c857a16950 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f799118366392a9f server_cookie=cd831436fb3f6031 in_seq=0 out_seq=0 2024-04-25T10:34:15.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.847+0000 7f6f3b772640 1 -- 172.21.15.43:0/2044103526 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6f200afce0 msgr2=0x7f6f200b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.847+0000 7f6f3b772640 1 --2- 172.21.15.43:0/2044103526 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6f200afce0 0x7f6f200b21f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5598724958c0 tx=0x7f6f30094040 comp rx=0 tx=0).stop 2024-04-25T10:34:15.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.847+0000 7f6f3b772640 1 -- 172.21.15.43:0/2044103526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f34102af0 msgr2=0x7f6f3419a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.847+0000 7f6f3b772640 1 --2- 172.21.15.43:0/2044103526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f34102af0 0x7f6f3419a7b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6f30032fd0 tx=0x7f6f30031a20 comp rx=0 tx=0).stop 2024-04-25T10:34:15.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.847+0000 7f6f3b772640 1 -- 172.21.15.43:0/2044103526 shutdown_connections 2024-04-25T10:34:15.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.847+0000 7f6f3b772640 1 --2- 172.21.15.43:0/2044103526 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6f200afce0 0x7f6f200b21f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.847+0000 7f6f3b772640 1 --2- 172.21.15.43:0/2044103526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f34108a20 0x7f6f3419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.847+0000 7f6f3b772640 1 --2- 172.21.15.43:0/2044103526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f3410d380 0x7f6f3419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.847+0000 7f6f3b772640 1 --2- 172.21.15.43:0/2044103526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f34102af0 0x7f6f3419a7b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.847+0000 7f6f3b772640 1 -- 172.21.15.43:0/2044103526 >> 172.21.15.43:0/2044103526 conn(0x7f6f34078100 msgr2=0x7f6f34069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:15.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.848+0000 7f6f3b772640 1 -- 172.21.15.43:0/2044103526 shutdown_connections 2024-04-25T10:34:15.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:15.848+0000 7f6f3b772640 1 -- 172.21.15.43:0/2044103526 wait complete. 2024-04-25T10:34:15.894 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.855+0000 7fad0f7fe640 1 -- 172.21.15.43:0/2706154685 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c8578fdee0 con 0x55c8575841c0 2024-04-25T10:34:15.894 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.855+0000 7fad0f7fe640 10 client.15945 handle_command_reply: tid=0 2024-04-25T10:34:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.855+0000 7fad3d24b740 1 client.15945 shutdown 2024-04-25T10:34:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.856+0000 7fad3d24b740 1 -- 172.21.15.43:0/2706154685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575f01b0 msgr2=0x55c857a0cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.856+0000 7fad3d24b740 1 --2- 172.21.15.43:0/2706154685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575f01b0 0x55c857a0cb90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fad28006010 tx=0x7fad2802a040 comp rx=0 tx=0).stop 2024-04-25T10:34:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.856+0000 7fad3d24b740 1 -- 172.21.15.43:0/2706154685 shutdown_connections 2024-04-25T10:34:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.856+0000 7fad3d24b740 1 --2- 172.21.15.43:0/2706154685 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c8575841c0 0x55c857a16950 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.856+0000 7fad3d24b740 1 --2- 172.21.15.43:0/2706154685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c8575cf4d0 0x55c8579a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.856+0000 7fad0f7fe640 0 client.15945 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.856+0000 7fad3d24b740 1 --2- 172.21.15.43:0/2706154685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8575f01b0 0x55c857a0cb90 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.856+0000 7fad3d24b740 1 --2- 172.21.15.43:0/2706154685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8575c72f0 0x55c8579a3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.856+0000 7fad3d24b740 1 -- 172.21.15.43:0/2706154685 >> 172.21.15.43:0/2706154685 conn(0x55c8575e4300 msgr2=0x55c857998850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.856+0000 7fad3d24b740 1 -- 172.21.15.43:0/2706154685 shutdown_connections 2024-04-25T10:34:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.856+0000 7fad3d24b740 1 -- 172.21.15.43:0/2706154685 wait complete. 2024-04-25T10:34:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.856+0000 7fad3d24b740 20 client.15945 trim_cache size 0 max 16384 2024-04-25T10:34:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:34:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:34:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:34:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:34:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.857+0000 7fad3639e640 1 -- 172.21.15.43:0/4123427899 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fad180afc50 msgr2=0x7fad180b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.857+0000 7fad3639e640 1 --2- 172.21.15.43:0/4123427899 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fad180afc50 0x7fad180b2160 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fad200dc160 tx=0x7fad200e1f90 comp rx=0 tx=0).stop 2024-04-25T10:34:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.857+0000 7fad3639e640 1 -- 172.21.15.43:0/4123427899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad30104c70 msgr2=0x7fad301a2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.857+0000 7fad3639e640 1 --2- 172.21.15.43:0/4123427899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad30104c70 0x7fad301a2590 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fad2000d950 tx=0x7fad2000dd50 comp rx=0 tx=0).stop 2024-04-25T10:34:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.857+0000 7fad3639e640 1 -- 172.21.15.43:0/4123427899 shutdown_connections 2024-04-25T10:34:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.857+0000 7fad3639e640 1 --2- 172.21.15.43:0/4123427899 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fad180afc50 0x7fad180b2160 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.857+0000 7fad3639e640 1 --2- 172.21.15.43:0/4123427899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad3010b6d0 0x7fad301a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.857+0000 7fad3639e640 1 --2- 172.21.15.43:0/4123427899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad3010e0c0 0x7fad3019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.857+0000 7fad3639e640 1 --2- 172.21.15.43:0/4123427899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad30104c70 0x7fad301a2590 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.857+0000 7fad3639e640 1 -- 172.21.15.43:0/4123427899 >> 172.21.15.43:0/4123427899 conn(0x7fad300781b0 msgr2=0x7fad30109f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.857+0000 7fad3639e640 1 -- 172.21.15.43:0/4123427899 shutdown_connections 2024-04-25T10:34:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:15.857+0000 7fad3639e640 1 -- 172.21.15.43:0/4123427899 wait complete. 2024-04-25T10:34:15.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:34:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.011+0000 7f093bc96640 1 -- 172.21.15.43:0/2382822952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0934102850 msgr2=0x7f0934102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.011+0000 7f093bc96640 1 --2- 172.21.15.43:0/2382822952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0934102850 0x7f0934102c60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0930009900 tx=0x7f093002a410 comp rx=0 tx=0).stop 2024-04-25T10:34:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.011+0000 7f093bc96640 1 -- 172.21.15.43:0/2382822952 shutdown_connections 2024-04-25T10:34:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.011+0000 7f093bc96640 1 --2- 172.21.15.43:0/2382822952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f093410da90 0x7f093410feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.011+0000 7f093bc96640 1 --2- 172.21.15.43:0/2382822952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09341103f0 0x7f0934112810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.011+0000 7f093bc96640 1 --2- 172.21.15.43:0/2382822952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0934102850 0x7f0934102c60 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.012+0000 7f093bc96640 1 -- 172.21.15.43:0/2382822952 >> 172.21.15.43:0/2382822952 conn(0x7f09340fe260 msgr2=0x7f0934100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.012+0000 7f093bc96640 1 -- 172.21.15.43:0/2382822952 shutdown_connections 2024-04-25T10:34:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.012+0000 7f093bc96640 1 -- 172.21.15.43:0/2382822952 wait complete. 2024-04-25T10:34:16.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.012+0000 7f093bc96640 1 Processor -- start 2024-04-25T10:34:16.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.012+0000 7f093bc96640 1 -- start start 2024-04-25T10:34:16.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f093bc96640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0934102850 0x7f09341a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f093bc96640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f093410da90 0x7f09341a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f093bc96640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09341103f0 0x7f093419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f093bc96640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0934115f00 con 0x7f093410da90 2024-04-25T10:34:16.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f093bc96640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0934115d80 con 0x7f0934102850 2024-04-25T10:34:16.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f093bc96640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0934116080 con 0x7f09341103f0 2024-04-25T10:34:16.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f0939231640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f093410da90 0x7f09341a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f0939a32640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0934102850 0x7f09341a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f0939231640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f093410da90 0x7f09341a2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41898/0 (socket says 172.21.15.43:41898) 2024-04-25T10:34:16.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f0939a32640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0934102850 0x7f09341a2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:47414/0 (socket says 172.21.15.43:47414) 2024-04-25T10:34:16.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f0939231640 1 -- 172.21.15.43:0/1816874808 learned_addr learned my addr 172.21.15.43:0/1816874808 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:16.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f093a233640 1 --2- 172.21.15.43:0/1816874808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09341103f0 0x7f093419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f0939231640 1 -- 172.21.15.43:0/1816874808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09341103f0 msgr2=0x7f093419c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f0939231640 1 --2- 172.21.15.43:0/1816874808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09341103f0 0x7f093419c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f0939231640 1 -- 172.21.15.43:0/1816874808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0934102850 msgr2=0x7f09341a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f0939231640 1 --2- 172.21.15.43:0/1816874808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0934102850 0x7f09341a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f0939231640 1 -- 172.21.15.43:0/1816874808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093419ce90 con 0x7f093410da90 2024-04-25T10:34:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f0939a32640 1 --2- 172.21.15.43:0/1816874808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0934102850 0x7f09341a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f0939231640 1 --2- 172.21.15.43:0/1816874808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f093410da90 0x7f09341a2ad0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f092400e9f0 tx=0x7f092400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f092affd640 1 -- 172.21.15.43:0/1816874808 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092400cd80 con 0x7f093410da90 2024-04-25T10:34:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f093bc96640 1 -- 172.21.15.43:0/1816874808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093419d120 con 0x7f093410da90 2024-04-25T10:34:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f092affd640 1 -- 172.21.15.43:0/1816874808 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09240108e0 con 0x7f093410da90 2024-04-25T10:34:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f092affd640 1 -- 172.21.15.43:0/1816874808 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0924018980 con 0x7f093410da90 2024-04-25T10:34:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.013+0000 7f093bc96640 1 -- 172.21.15.43:0/1816874808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09341a9330 con 0x7f093410da90 2024-04-25T10:34:16.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.016+0000 7f092affd640 1 -- 172.21.15.43:0/1816874808 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f0924010430 con 0x7f093410da90 2024-04-25T10:34:16.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.016+0000 7f0942c23740 1 Processor -- start 2024-04-25T10:34:16.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.016+0000 7f0942c23740 1 -- start start 2024-04-25T10:34:16.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.017+0000 7f0942c23740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d05d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.017+0000 7f0942c23740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc934d0 0x55c91d05dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.017+0000 7f0942c23740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc9b6b0 0x55c91d06a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.017+0000 7f0942c23740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c91cfc2d80 con 0x55c91ccbc390 2024-04-25T10:34:16.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.017+0000 7f0942c23740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c91cf7ca20 con 0x55c91cc934d0 2024-04-25T10:34:16.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.017+0000 7f0942c23740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c91cfbed40 con 0x55c91cc9b6b0 2024-04-25T10:34:16.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.017+0000 7f092affd640 1 --2- 172.21.15.43:0/1816874808 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f09200afc60 0x7f09200b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.017+0000 7f093a233640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d05d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.017+0000 7f093a233640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d05d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41910/0 (socket says 172.21.15.43:41910) 2024-04-25T10:34:16.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.017+0000 7f0939a32640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc934d0 0x55c91d05dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.017+0000 7f093a233640 1 -- 172.21.15.43:0/3117032660 learned_addr learned my addr 172.21.15.43:0/3117032660 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:16.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.017+0000 7f092affd640 1 -- 172.21.15.43:0/1816874808 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f09240df3d0 con 0x7f093410da90 2024-04-25T10:34:16.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.017+0000 7f093a233640 1 --2- 172.21.15.43:0/1816874808 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f09200afc60 0x7f09200b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.017+0000 7f0939231640 1 --2- 172.21.15.43:0/3117032660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc9b6b0 0x55c91d06a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.017+0000 7f0939a32640 1 -- 172.21.15.43:0/3117032660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc9b6b0 msgr2=0x55c91d06a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:16.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.017+0000 7f0939a32640 1 --2- 172.21.15.43:0/3117032660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc9b6b0 0x55c91d06a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.017+0000 7f0939a32640 1 -- 172.21.15.43:0/3117032660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 msgr2=0x55c91d05d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:16.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.017+0000 7f0939a32640 1 --2- 172.21.15.43:0/3117032660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d05d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.017+0000 7f0939a32640 1 -- 172.21.15.43:0/3117032660 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c91cf87c30 con 0x55c91cc934d0 2024-04-25T10:34:16.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.018+0000 7f0939a32640 1 --2- 172.21.15.43:0/3117032660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc934d0 0x55c91d05dd50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0930009230 tx=0x7f0930005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcc5ff91a3070927 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:16.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.018+0000 7f093a233640 1 --2- 172.21.15.43:0/1816874808 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f09200afc60 0x7f09200b2170 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55c91d05a980 tx=0x7f092c005bd0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:16.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.018+0000 7f091ffff640 1 -- 172.21.15.43:0/3117032660 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930040070 con 0x55c91cc934d0 2024-04-25T10:34:16.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.018+0000 7f091ffff640 1 -- 172.21.15.43:0/3117032660 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0930031820 con 0x55c91cc934d0 2024-04-25T10:34:16.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.018+0000 7f091ffff640 1 -- 172.21.15.43:0/3117032660 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930035040 con 0x55c91cc934d0 2024-04-25T10:34:16.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.019+0000 7f0942c23740 1 -- 172.21.15.43:0/3117032660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc934d0 msgr2=0x55c91d05dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:16.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.019+0000 7f0942c23740 1 --2- 172.21.15.43:0/3117032660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc934d0 0x55c91d05dd50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0930009230 tx=0x7f0930005ed0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.019+0000 7f0942c23740 1 -- 172.21.15.43:0/3117032660 shutdown_connections 2024-04-25T10:34:16.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.019+0000 7f0942c23740 1 --2- 172.21.15.43:0/3117032660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc934d0 0x55c91d05dd50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.019+0000 7f0942c23740 1 --2- 172.21.15.43:0/3117032660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc9b6b0 0x55c91d06a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.019+0000 7f0942c23740 1 --2- 172.21.15.43:0/3117032660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d05d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.019+0000 7f0942c23740 1 -- 172.21.15.43:0/3117032660 >> 172.21.15.43:0/3117032660 conn(0x55c91ccb04e0 msgr2=0x55c91d059fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:16.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.019+0000 7f0942c23740 1 -- 172.21.15.43:0/3117032660 shutdown_connections 2024-04-25T10:34:16.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.019+0000 7f0942c23740 1 -- 172.21.15.43:0/3117032660 wait complete. 2024-04-25T10:34:16.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.019+0000 7f0942c23740 1 -- start start 2024-04-25T10:34:16.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.019+0000 7f0942c23740 1 Processor -- start 2024-04-25T10:34:16.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f0942c23740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc934d0 0x55c91d068290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f0942c23740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc9b6b0 0x55c91d0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f0942c23740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d0741b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f0942c23740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c91d06ad30 con 0x55c91ccbc390 2024-04-25T10:34:16.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f0942c23740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c91cf49dc0 con 0x55c91cc9b6b0 2024-04-25T10:34:16.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f0942c23740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c91d06aeb0 con 0x55c91cc934d0 2024-04-25T10:34:16.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f0942c23740 10 client.0 _ll_register_callbacks cb 0x55c91cf65600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:16.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f093a233640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d0741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f093a233640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d0741b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41916/0 (socket says 172.21.15.43:41916) 2024-04-25T10:34:16.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f093a233640 1 -- 172.21.15.43:0/2806543987 learned_addr learned my addr 172.21.15.43:0/2806543987 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:16.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f0939a32640 1 --2- 172.21.15.43:0/2806543987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc934d0 0x55c91d068290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f0939231640 1 --2- 172.21.15.43:0/2806543987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc9b6b0 0x55c91d0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f093a233640 1 -- 172.21.15.43:0/2806543987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc934d0 msgr2=0x55c91d068290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:16.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f093a233640 1 --2- 172.21.15.43:0/2806543987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc934d0 0x55c91d068290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f093a233640 1 -- 172.21.15.43:0/2806543987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc9b6b0 msgr2=0x55c91d0687d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f093a233640 1 --2- 172.21.15.43:0/2806543987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc9b6b0 0x55c91d0687d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f093a233640 1 -- 172.21.15.43:0/2806543987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c91ccd43f0 con 0x55c91ccbc390 2024-04-25T10:34:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f0939231640 1 --2- 172.21.15.43:0/2806543987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc9b6b0 0x55c91d0687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f0939a32640 1 --2- 172.21.15.43:0/2806543987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc934d0 0x55c91d068290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f093a233640 1 --2- 172.21.15.43:0/2806543987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d0741b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f092c005c40 tx=0x7f092c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95a420dc3e813f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.020+0000 7f091d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.021+0000 7f091d7fa640 1 -- 172.21.15.43:0/2806543987 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c025030 con 0x55c91ccbc390 2024-04-25T10:34:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.021+0000 7f091d7fa640 1 -- 172.21.15.43:0/2806543987 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f092c01b030 con 0x55c91ccbc390 2024-04-25T10:34:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.021+0000 7f0942c23740 1 -- 172.21.15.43:0/2806543987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0930031820 con 0x55c91ccbc390 2024-04-25T10:34:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.021+0000 7f091d7fa640 1 -- 172.21.15.43:0/2806543987 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f092c02f050 con 0x55c91ccbc390 2024-04-25T10:34:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.021+0000 7f0942c23740 10 client.15957 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:16.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.021+0000 7f0942c23740 1 -- 172.21.15.43:0/2806543987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c91cd9e7d0 con 0x55c91ccbc390 2024-04-25T10:34:16.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.021+0000 7f091d7fa640 1 -- 172.21.15.43:0/2806543987 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0930031820 con 0x55c91ccbc390 2024-04-25T10:34:16.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.021+0000 7f091d7fa640 1 -- 172.21.15.43:0/2806543987 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f092c005020 con 0x55c91ccbc390 2024-04-25T10:34:16.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.021+0000 7f0942c23740 10 client.15957 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:16.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.021+0000 7f0942c23740 10 client.15957 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:34:16.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.021+0000 7f0942c23740 20 client.15957 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:16.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.021+0000 7f0942c23740 1 --2- 172.21.15.43:0/2806543987 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55c91cc7af60 0x55c91d07a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.021+0000 7f0942c23740 4 client.15957 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:34:16.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.021+0000 7f0942c23740 1 -- 172.21.15.43:0/2806543987 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c91cfd0990 con 0x55c91cc7af60 2024-04-25T10:34:16.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.022+0000 7f0939a32640 1 --2- 172.21.15.43:0/2806543987 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55c91cc7af60 0x55c91d07a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.022+0000 7f091d7fa640 10 client.15957 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:34:16.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.022+0000 7f0939a32640 1 --2- 172.21.15.43:0/2806543987 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55c91cc7af60 0x55c91d07a720 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8837667f3978f4fa server_cookie=3821fda39e9eb8a3 in_seq=0 out_seq=0 2024-04-25T10:34:16.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.023+0000 7f091d7fa640 1 -- 172.21.15.43:0/2806543987 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c91cfd0990 con 0x55c91cc7af60 2024-04-25T10:34:16.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.023+0000 7f091d7fa640 10 client.15957 handle_command_reply: tid=0 2024-04-25T10:34:16.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.023+0000 7f0942c23740 1 client.15957 shutdown 2024-04-25T10:34:16.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.023+0000 7f0942c23740 1 -- 172.21.15.43:0/2806543987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 msgr2=0x55c91d0741b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:16.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.023+0000 7f0942c23740 1 --2- 172.21.15.43:0/2806543987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d0741b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f092c005c40 tx=0x7f092c004a40 comp rx=0 tx=0).stop 2024-04-25T10:34:16.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.023+0000 7f0942c23740 1 -- 172.21.15.43:0/2806543987 shutdown_connections 2024-04-25T10:34:16.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.023+0000 7f0942c23740 1 --2- 172.21.15.43:0/2806543987 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55c91cc7af60 0x55c91d07a720 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.023+0000 7f0942c23740 1 --2- 172.21.15.43:0/2806543987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc9b6b0 0x55c91d0687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.023+0000 7f091d7fa640 0 client.15957 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:34:16.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.023+0000 7f0942c23740 1 --2- 172.21.15.43:0/2806543987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d0741b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.023+0000 7f0942c23740 1 --2- 172.21.15.43:0/2806543987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc934d0 0x55c91d068290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.023+0000 7f0942c23740 1 -- 172.21.15.43:0/2806543987 >> 172.21.15.43:0/2806543987 conn(0x55c91ccb04e0 msgr2=0x55c91d05f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.023+0000 7f0942c23740 1 -- 172.21.15.43:0/2806543987 shutdown_connections 2024-04-25T10:34:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.023+0000 7f0942c23740 1 -- 172.21.15.43:0/2806543987 wait complete. 2024-04-25T10:34:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.023+0000 7f0942c23740 20 client.15957 trim_cache size 0 max 16384 2024-04-25T10:34:16.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.037+0000 7f0942c23740 1 Processor -- start 2024-04-25T10:34:16.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.037+0000 7f0942c23740 1 -- start start 2024-04-25T10:34:16.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.037+0000 7f0942c23740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc934d0 0x55c91d05dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.037+0000 7f0942c23740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc9b6b0 0x55c91d073e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.037+0000 7f0942c23740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d062430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.037+0000 7f0942c23740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f09341b4420 con 0x55c91ccbc390 2024-04-25T10:34:16.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.037+0000 7f0942c23740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f09341b1cf0 con 0x55c91cc934d0 2024-04-25T10:34:16.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.037+0000 7f0942c23740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f09341afb40 con 0x55c91cc9b6b0 2024-04-25T10:34:16.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.037+0000 7f093a233640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d062430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.037+0000 7f0939a32640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc934d0 0x55c91d05dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.037+0000 7f0939a32640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc934d0 0x55c91d05dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:47448/0 (socket says 172.21.15.43:47448) 2024-04-25T10:34:16.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.037+0000 7f0939a32640 1 -- 172.21.15.43:0/515416694 learned_addr learned my addr 172.21.15.43:0/515416694 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:16.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.037+0000 7f0939231640 1 --2- 172.21.15.43:0/515416694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc9b6b0 0x55c91d073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.038+0000 7f093a233640 1 -- 172.21.15.43:0/515416694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc9b6b0 msgr2=0x55c91d073e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:16.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.038+0000 7f093a233640 1 --2- 172.21.15.43:0/515416694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc9b6b0 0x55c91d073e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.038+0000 7f093a233640 1 -- 172.21.15.43:0/515416694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc934d0 msgr2=0x55c91d05dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:16.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.038+0000 7f093a233640 1 --2- 172.21.15.43:0/515416694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc934d0 0x55c91d05dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.038+0000 7f093a233640 1 -- 172.21.15.43:0/515416694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c91d067ff0 con 0x55c91ccbc390 2024-04-25T10:34:16.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.038+0000 7f0939a32640 1 --2- 172.21.15.43:0/515416694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc934d0 0x55c91d05dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.038+0000 7f093a233640 1 --2- 172.21.15.43:0/515416694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d062430 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f092c005630 tx=0x7f092c012400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373d9b517e4a5605 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.038+0000 7f091f7fe640 1 -- 172.21.15.43:0/515416694 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c025040 con 0x55c91ccbc390 2024-04-25T10:34:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.038+0000 7f091f7fe640 1 -- 172.21.15.43:0/515416694 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f092c00dd60 con 0x55c91ccbc390 2024-04-25T10:34:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.038+0000 7f091f7fe640 1 -- 172.21.15.43:0/515416694 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c004cb0 con 0x55c91ccbc390 2024-04-25T10:34:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.039+0000 7f0942c23740 1 -- 172.21.15.43:0/515416694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 msgr2=0x55c91d062430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.039+0000 7f0942c23740 1 --2- 172.21.15.43:0/515416694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d062430 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f092c005630 tx=0x7f092c012400 comp rx=0 tx=0).stop 2024-04-25T10:34:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.039+0000 7f0942c23740 1 -- 172.21.15.43:0/515416694 shutdown_connections 2024-04-25T10:34:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.039+0000 7f0942c23740 1 --2- 172.21.15.43:0/515416694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d062430 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.039+0000 7f0942c23740 1 --2- 172.21.15.43:0/515416694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc9b6b0 0x55c91d073e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.039+0000 7f0942c23740 1 --2- 172.21.15.43:0/515416694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc934d0 0x55c91d05dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.039+0000 7f0942c23740 1 -- 172.21.15.43:0/515416694 >> 172.21.15.43:0/515416694 conn(0x55c91ccb04e0 msgr2=0x55c91d05d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.039+0000 7f0942c23740 1 -- 172.21.15.43:0/515416694 shutdown_connections 2024-04-25T10:34:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.039+0000 7f0942c23740 1 -- 172.21.15.43:0/515416694 wait complete. 2024-04-25T10:34:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.039+0000 7f0942c23740 1 -- start start 2024-04-25T10:34:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.039+0000 7f0942c23740 1 Processor -- start 2024-04-25T10:34:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f0942c23740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc934d0 0x55c91d06b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f0942c23740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc9b6b0 0x55c91d06b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f0942c23740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d06bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f0942c23740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f09341b01b0 con 0x55c91ccbc390 2024-04-25T10:34:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f0942c23740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f09341b1620 con 0x55c91cc934d0 2024-04-25T10:34:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f0942c23740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f09341b0880 con 0x55c91cc9b6b0 2024-04-25T10:34:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f0942c23740 10 client.0 _ll_register_callbacks cb 0x7f092c009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f093a233640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d06bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f093a233640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d06bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41938/0 (socket says 172.21.15.43:41938) 2024-04-25T10:34:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f093a233640 1 -- 172.21.15.43:0/1386686336 learned_addr learned my addr 172.21.15.43:0/1386686336 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f0939231640 1 --2- 172.21.15.43:0/1386686336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc9b6b0 0x55c91d06b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f0939a32640 1 --2- 172.21.15.43:0/1386686336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc934d0 0x55c91d06b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f093a233640 1 -- 172.21.15.43:0/1386686336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc9b6b0 msgr2=0x55c91d06b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:16.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f093a233640 1 --2- 172.21.15.43:0/1386686336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc9b6b0 0x55c91d06b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f093a233640 1 -- 172.21.15.43:0/1386686336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc934d0 msgr2=0x55c91d06b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:16.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f093a233640 1 --2- 172.21.15.43:0/1386686336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc934d0 0x55c91d06b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f093a233640 1 -- 172.21.15.43:0/1386686336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c91d0e0bb0 con 0x55c91ccbc390 2024-04-25T10:34:16.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f0939a32640 1 --2- 172.21.15.43:0/1386686336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc934d0 0x55c91d06b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:16.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f093a233640 1 --2- 172.21.15.43:0/1386686336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d06bbd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f092c006010 tx=0x7f092c0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2bf3e05684aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:16.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f091d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:16.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f091d7fa640 1 -- 172.21.15.43:0/1386686336 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c01b040 con 0x55c91ccbc390 2024-04-25T10:34:16.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f091d7fa640 1 -- 172.21.15.43:0/1386686336 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f092c021950 con 0x55c91ccbc390 2024-04-25T10:34:16.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.040+0000 7f0942c23740 1 -- 172.21.15.43:0/1386686336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f092c00dd60 con 0x55c91ccbc390 2024-04-25T10:34:16.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.041+0000 7f091d7fa640 1 -- 172.21.15.43:0/1386686336 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f092c025040 con 0x55c91ccbc390 2024-04-25T10:34:16.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.041+0000 7f0942c23740 10 client.15963 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.041+0000 7f0942c23740 1 -- 172.21.15.43:0/1386686336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c91d0e1e30 con 0x55c91ccbc390 2024-04-25T10:34:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.041+0000 7f091d7fa640 1 -- 172.21.15.43:0/1386686336 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f092c00dd60 con 0x55c91ccbc390 2024-04-25T10:34:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.041+0000 7f091d7fa640 1 -- 172.21.15.43:0/1386686336 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f092c02a400 con 0x55c91ccbc390 2024-04-25T10:34:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.041+0000 7f0942c23740 10 client.15963 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.041+0000 7f0942c23740 10 client.15963 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:34:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.041+0000 7f0942c23740 20 client.15963 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.041+0000 7f0942c23740 1 --2- 172.21.15.43:0/1386686336 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55c91cc503a0 0x55c91d0e4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.041+0000 7f0942c23740 4 client.15963 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:34:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.041+0000 7f0942c23740 1 -- 172.21.15.43:0/1386686336 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c91cfca800 con 0x55c91cc503a0 2024-04-25T10:34:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.041+0000 7f0939a32640 1 --2- 172.21.15.43:0/1386686336 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55c91cc503a0 0x55c91d0e4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.042+0000 7f091d7fa640 10 client.15963 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:34:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.042+0000 7f0939a32640 1 --2- 172.21.15.43:0/1386686336 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55c91cc503a0 0x55c91d0e4440 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6aafec1a5cad84d7 server_cookie=963c110caa18fa68 in_seq=0 out_seq=0 2024-04-25T10:34:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.042+0000 7f091d7fa640 1 -- 172.21.15.43:0/1386686336 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c91cfca800 con 0x55c91cc503a0 2024-04-25T10:34:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.042+0000 7f091d7fa640 10 client.15963 handle_command_reply: tid=0 2024-04-25T10:34:16.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.042+0000 7f0942c23740 1 client.15963 shutdown 2024-04-25T10:34:16.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.042+0000 7f0942c23740 1 -- 172.21.15.43:0/1386686336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 msgr2=0x55c91d06bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:16.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.042+0000 7f0942c23740 1 --2- 172.21.15.43:0/1386686336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d06bbd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f092c006010 tx=0x7f092c0099b0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.042+0000 7f0942c23740 1 -- 172.21.15.43:0/1386686336 shutdown_connections 2024-04-25T10:34:16.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.042+0000 7f0942c23740 1 --2- 172.21.15.43:0/1386686336 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55c91cc503a0 0x55c91d0e4440 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.042+0000 7f0942c23740 1 --2- 172.21.15.43:0/1386686336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91ccbc390 0x55c91d06bbd0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.042+0000 7f0942c23740 1 --2- 172.21.15.43:0/1386686336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c91cc9b6b0 0x55c91d06b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.042+0000 7f091d7fa640 0 client.15963 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:34:16.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.042+0000 7f0942c23740 1 --2- 172.21.15.43:0/1386686336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c91cc934d0 0x55c91d06b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.042+0000 7f0942c23740 1 -- 172.21.15.43:0/1386686336 >> 172.21.15.43:0/1386686336 conn(0x55c91ccb04e0 msgr2=0x55c91d069730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:16.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.042+0000 7f0942c23740 1 -- 172.21.15.43:0/1386686336 shutdown_connections 2024-04-25T10:34:16.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.042+0000 7f0942c23740 1 -- 172.21.15.43:0/1386686336 wait complete. 2024-04-25T10:34:16.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.043+0000 7f0942c23740 20 client.15963 trim_cache size 0 max 16384 2024-04-25T10:34:16.061 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:34:16.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.043+0000 7f093bc96640 1 -- 172.21.15.43:0/1816874808 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f09200afc60 msgr2=0x7f09200b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:16.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.043+0000 7f093bc96640 1 --2- 172.21.15.43:0/1816874808 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f09200afc60 0x7f09200b2170 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55c91d05a980 tx=0x7f092c005bd0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.043+0000 7f093bc96640 1 -- 172.21.15.43:0/1816874808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f093410da90 msgr2=0x7f09341a2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:16.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.043+0000 7f093bc96640 1 --2- 172.21.15.43:0/1816874808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f093410da90 0x7f09341a2ad0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f092400e9f0 tx=0x7f092400eeb0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.043+0000 7f093bc96640 1 -- 172.21.15.43:0/1816874808 shutdown_connections 2024-04-25T10:34:16.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.043+0000 7f093bc96640 1 --2- 172.21.15.43:0/1816874808 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f09200afc60 0x7f09200b2170 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.043+0000 7f093bc96640 1 --2- 172.21.15.43:0/1816874808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09341103f0 0x7f093419c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.043+0000 7f093bc96640 1 --2- 172.21.15.43:0/1816874808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f093410da90 0x7f09341a2ad0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.043+0000 7f093bc96640 1 --2- 172.21.15.43:0/1816874808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0934102850 0x7f09341a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.043+0000 7f093bc96640 1 -- 172.21.15.43:0/1816874808 >> 172.21.15.43:0/1816874808 conn(0x7f09340fe260 msgr2=0x7f0934100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:16.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.043+0000 7f093bc96640 1 -- 172.21.15.43:0/1816874808 shutdown_connections 2024-04-25T10:34:16.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.043+0000 7f093bc96640 1 -- 172.21.15.43:0/1816874808 wait complete. 2024-04-25T10:34:16.064 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:34:16.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.175+0000 7f581bf35640 1 -- 172.21.15.43:0/2700483954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58141032c0 msgr2=0x7f58141036d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:16.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.175+0000 7f581bf35640 1 --2- 172.21.15.43:0/2700483954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58141032c0 0x7f58141036d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5804009900 tx=0x7f580402a410 comp rx=0 tx=0).stop 2024-04-25T10:34:16.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.175+0000 7f581bf35640 1 -- 172.21.15.43:0/2700483954 shutdown_connections 2024-04-25T10:34:16.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.175+0000 7f581bf35640 1 --2- 172.21.15.43:0/2700483954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f581410d440 0x7f581410f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.175+0000 7f581bf35640 1 --2- 172.21.15.43:0/2700483954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58141032c0 0x7f58141036d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.175+0000 7f581bf35640 1 --2- 172.21.15.43:0/2700483954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5814107950 0x7f5814102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.175+0000 7f581bf35640 1 -- 172.21.15.43:0/2700483954 >> 172.21.15.43:0/2700483954 conn(0x7f5814077920 msgr2=0x7f5814077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:16.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.175+0000 7f581bf35640 1 -- 172.21.15.43:0/2700483954 shutdown_connections 2024-04-25T10:34:16.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.175+0000 7f581bf35640 1 -- 172.21.15.43:0/2700483954 wait complete. 2024-04-25T10:34:16.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.176+0000 7f581bf35640 1 Processor -- start 2024-04-25T10:34:16.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.176+0000 7f581bf35640 1 -- start start 2024-04-25T10:34:16.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.176+0000 7f581bf35640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58141032c0 0x7f581419b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.176+0000 7f581bf35640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5814107950 0x7f581419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.176+0000 7f581bf35640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f581410d440 0x7f581419c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.176+0000 7f581bf35640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5814112c00 con 0x7f5814107950 2024-04-25T10:34:16.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.176+0000 7f581bf35640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5814112a80 con 0x7f581410d440 2024-04-25T10:34:16.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.176+0000 7f581bf35640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5814112d80 con 0x7f58141032c0 2024-04-25T10:34:16.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.176+0000 7f58194d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5814107950 0x7f581419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.176+0000 7f58194d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5814107950 0x7f581419bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41960/0 (socket says 172.21.15.43:41960) 2024-04-25T10:34:16.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.176+0000 7f58194d0640 1 -- 172.21.15.43:0/1312231390 learned_addr learned my addr 172.21.15.43:0/1312231390 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:16.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.177+0000 7f5819cd1640 1 --2- 172.21.15.43:0/1312231390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58141032c0 0x7f581419b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.177+0000 7f581a4d2640 1 --2- 172.21.15.43:0/1312231390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f581410d440 0x7f581419c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.177+0000 7f58194d0640 1 -- 172.21.15.43:0/1312231390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58141032c0 msgr2=0x7f581419b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:16.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.177+0000 7f58194d0640 1 --2- 172.21.15.43:0/1312231390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58141032c0 0x7f581419b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.177+0000 7f58194d0640 1 -- 172.21.15.43:0/1312231390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f581410d440 msgr2=0x7f581419c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:16.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.177+0000 7f58194d0640 1 --2- 172.21.15.43:0/1312231390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f581410d440 0x7f581419c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.177+0000 7f58194d0640 1 -- 172.21.15.43:0/1312231390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58141a68e0 con 0x7f5814107950 2024-04-25T10:34:16.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.177+0000 7f58194d0640 1 --2- 172.21.15.43:0/1312231390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5814107950 0x7f581419bb60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5804032fd0 tx=0x7f5804004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:16.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.177+0000 7f580affd640 1 -- 172.21.15.43:0/1312231390 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580403f030 con 0x7f5814107950 2024-04-25T10:34:16.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.177+0000 7f581bf35640 1 -- 172.21.15.43:0/1312231390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58141a6b70 con 0x7f5814107950 2024-04-25T10:34:16.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.177+0000 7f580affd640 1 -- 172.21.15.43:0/1312231390 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5804004490 con 0x7f5814107950 2024-04-25T10:34:16.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.177+0000 7f580affd640 1 -- 172.21.15.43:0/1312231390 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804035070 con 0x7f5814107950 2024-04-25T10:34:16.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.177+0000 7f581bf35640 1 -- 172.21.15.43:0/1312231390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58141a70b0 con 0x7f5814107950 2024-04-25T10:34:16.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.180+0000 7f5822de0740 1 Processor -- start 2024-04-25T10:34:16.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.180+0000 7f5822de0740 1 -- start start 2024-04-25T10:34:16.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.180+0000 7f580affd640 1 -- 172.21.15.43:0/1312231390 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f5804042030 con 0x7f5814107950 2024-04-25T10:34:16.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.180+0000 7f5822de0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 0x55fedd106810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.180+0000 7f5822de0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd3c4d0 0x55fedd106d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.180+0000 7f5822de0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd446b0 0x55fedd1136a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.180+0000 7f5822de0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fedd06bd80 con 0x55fedcd65390 2024-04-25T10:34:16.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.180+0000 7f5822de0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fedd025a20 con 0x55fedcd446b0 2024-04-25T10:34:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.180+0000 7f5822de0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fedd067d40 con 0x55fedcd3c4d0 2024-04-25T10:34:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.181+0000 7f581a4d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 0x55fedd106810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.181+0000 7f581a4d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 0x55fedd106810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41966/0 (socket says 172.21.15.43:41966) 2024-04-25T10:34:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.181+0000 7f581a4d2640 1 -- 172.21.15.43:0/3684051186 learned_addr learned my addr 172.21.15.43:0/3684051186 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.181+0000 7f58194d0640 1 --2- 172.21.15.43:0/3684051186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd446b0 0x55fedd1136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.181+0000 7f5819cd1640 1 --2- 172.21.15.43:0/3684051186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd3c4d0 0x55fedd106d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.181+0000 7f581a4d2640 1 -- 172.21.15.43:0/3684051186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd3c4d0 msgr2=0x55fedd106d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.181+0000 7f581a4d2640 1 --2- 172.21.15.43:0/3684051186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd3c4d0 0x55fedd106d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.181+0000 7f581a4d2640 1 -- 172.21.15.43:0/3684051186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd446b0 msgr2=0x55fedd1136a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.181+0000 7f581a4d2640 1 --2- 172.21.15.43:0/3684051186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd446b0 0x55fedd1136a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.181+0000 7f580affd640 1 --2- 172.21.15.43:0/1312231390 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f58000afc60 0x7f58000b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.181+0000 7f581a4d2640 1 -- 172.21.15.43:0/3684051186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fedd030c30 con 0x55fedcd65390 2024-04-25T10:34:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.181+0000 7f5819cd1640 1 --2- 172.21.15.43:0/1312231390 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f58000afc60 0x7f58000b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.181+0000 7f580affd640 1 -- 172.21.15.43:0/1312231390 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f58040ff500 con 0x7f5814107950 2024-04-25T10:34:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.181+0000 7f581a4d2640 1 --2- 172.21.15.43:0/3684051186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 0x55fedd106810 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f580c009ef0 tx=0x7f580c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c139f07280f3a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.181+0000 7f57fffff640 1 -- 172.21.15.43:0/3684051186 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c015070 con 0x55fedcd65390 2024-04-25T10:34:16.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.182+0000 7f57fffff640 1 -- 172.21.15.43:0/3684051186 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f580c0118f0 con 0x55fedcd65390 2024-04-25T10:34:16.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.182+0000 7f57fffff640 1 -- 172.21.15.43:0/3684051186 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c0107f0 con 0x55fedcd65390 2024-04-25T10:34:16.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.182+0000 7f5822de0740 1 -- 172.21.15.43:0/3684051186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 msgr2=0x55fedd106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:16.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.182+0000 7f5819cd1640 1 --2- 172.21.15.43:0/1312231390 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f58000afc60 0x7f58000b2170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55fedd107fe0 tx=0x7f581000b390 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:16.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.182+0000 7f5822de0740 1 --2- 172.21.15.43:0/3684051186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 0x55fedd106810 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f580c009ef0 tx=0x7f580c00f730 comp rx=0 tx=0).stop 2024-04-25T10:34:16.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.182+0000 7f5822de0740 1 -- 172.21.15.43:0/3684051186 shutdown_connections 2024-04-25T10:34:16.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.182+0000 7f5822de0740 1 --2- 172.21.15.43:0/3684051186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd446b0 0x55fedd1136a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.182+0000 7f5822de0740 1 --2- 172.21.15.43:0/3684051186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd3c4d0 0x55fedd106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.182+0000 7f5822de0740 1 --2- 172.21.15.43:0/3684051186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 0x55fedd106810 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.182+0000 7f5822de0740 1 -- 172.21.15.43:0/3684051186 >> 172.21.15.43:0/3684051186 conn(0x55fedcd594e0 msgr2=0x55fedd102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:16.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.183+0000 7f5822de0740 1 -- 172.21.15.43:0/3684051186 shutdown_connections 2024-04-25T10:34:16.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.183+0000 7f5822de0740 1 -- 172.21.15.43:0/3684051186 wait complete. 2024-04-25T10:34:16.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.183+0000 7f5822de0740 1 -- start start 2024-04-25T10:34:16.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.183+0000 7f5822de0740 1 Processor -- start 2024-04-25T10:34:16.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.183+0000 7f5822de0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd3c4d0 0x55fedd1112f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.183+0000 7f5822de0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd446b0 0x55fedd111830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.183+0000 7f5822de0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd65390 0x55fedd11d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f5822de0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fedd113d90 con 0x55fedcd446b0 2024-04-25T10:34:16.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f5822de0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fedcff2dc0 con 0x55fedcd3c4d0 2024-04-25T10:34:16.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f5822de0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fedd113f10 con 0x55fedcd65390 2024-04-25T10:34:16.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f5822de0740 10 client.0 _ll_register_callbacks cb 0x55fedd00e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:16.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f58194d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd446b0 0x55fedd111830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f58194d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd446b0 0x55fedd111830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41982/0 (socket says 172.21.15.43:41982) 2024-04-25T10:34:16.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f58194d0640 1 -- 172.21.15.43:0/769020339 learned_addr learned my addr 172.21.15.43:0/769020339 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:16.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f5819cd1640 1 --2- 172.21.15.43:0/769020339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd3c4d0 0x55fedd1112f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f581a4d2640 1 --2- 172.21.15.43:0/769020339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd65390 0x55fedd11d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f58194d0640 1 -- 172.21.15.43:0/769020339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd65390 msgr2=0x55fedd11d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f58194d0640 1 --2- 172.21.15.43:0/769020339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd65390 0x55fedd11d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f58194d0640 1 -- 172.21.15.43:0/769020339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd3c4d0 msgr2=0x55fedd1112f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f58194d0640 1 --2- 172.21.15.43:0/769020339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd3c4d0 0x55fedd1112f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f58194d0640 1 -- 172.21.15.43:0/769020339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fedcd7d3f0 con 0x55fedcd446b0 2024-04-25T10:34:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f581a4d2640 1 --2- 172.21.15.43:0/769020339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd65390 0x55fedd11d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f5819cd1640 1 --2- 172.21.15.43:0/769020339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd3c4d0 0x55fedd1112f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f58194d0640 1 --2- 172.21.15.43:0/769020339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd446b0 0x55fedd111830 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f58040fcce0 tx=0x7f5804031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8643aef27218fedd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f57fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f57fd7fa640 1 -- 172.21.15.43:0/769020339 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804049020 con 0x55fedcd446b0 2024-04-25T10:34:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f57fd7fa640 1 -- 172.21.15.43:0/769020339 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f58040c3cc0 con 0x55fedcd446b0 2024-04-25T10:34:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f5822de0740 1 -- 172.21.15.43:0/769020339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f580c0118f0 con 0x55fedcd446b0 2024-04-25T10:34:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f57fd7fa640 1 -- 172.21.15.43:0/769020339 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f58040c1e60 con 0x55fedcd446b0 2024-04-25T10:34:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f5822de0740 10 client.15975 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.184+0000 7f5822de0740 1 -- 172.21.15.43:0/769020339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fedce477d0 con 0x55fedcd446b0 2024-04-25T10:34:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.185+0000 7f57fd7fa640 1 -- 172.21.15.43:0/769020339 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f580c0118f0 con 0x55fedcd446b0 2024-04-25T10:34:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.185+0000 7f57fd7fa640 1 -- 172.21.15.43:0/769020339 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f58040c3960 con 0x55fedcd446b0 2024-04-25T10:34:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.185+0000 7f5822de0740 10 client.15975 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.185+0000 7f5822de0740 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.185+0000 7f5822de0740 20 client.15975 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.185+0000 7f5822de0740 1 --2- 172.21.15.43:0/769020339 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fedcd23f60 0x55fedd123690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.185+0000 7f5822de0740 4 client.15975 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:34:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.185+0000 7f5822de0740 1 -- 172.21.15.43:0/769020339 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fedd079990 con 0x55fedcd23f60 2024-04-25T10:34:16.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.185+0000 7f5819cd1640 1 --2- 172.21.15.43:0/769020339 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fedcd23f60 0x55fedd123690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.186+0000 7f57fd7fa640 10 client.15975 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:16.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.186+0000 7f5819cd1640 1 --2- 172.21.15.43:0/769020339 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fedcd23f60 0x55fedd123690 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b1da03d7f9234ee server_cookie=d7e1aceee2bdb804 in_seq=0 out_seq=0 2024-04-25T10:34:16.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.186+0000 7f57fd7fa640 1 -- 172.21.15.43:0/769020339 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fedd079990 con 0x55fedcd23f60 2024-04-25T10:34:16.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.186+0000 7f57fd7fa640 10 client.15975 handle_command_reply: tid=0 2024-04-25T10:34:16.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.186+0000 7f5822de0740 1 client.15975 shutdown 2024-04-25T10:34:16.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.187+0000 7f5822de0740 1 -- 172.21.15.43:0/769020339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd446b0 msgr2=0x55fedd111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:16.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.187+0000 7f5822de0740 1 --2- 172.21.15.43:0/769020339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd446b0 0x55fedd111830 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f58040fcce0 tx=0x7f5804031a10 comp rx=0 tx=0).stop 2024-04-25T10:34:16.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.187+0000 7f5822de0740 1 -- 172.21.15.43:0/769020339 shutdown_connections 2024-04-25T10:34:16.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.187+0000 7f5822de0740 1 --2- 172.21.15.43:0/769020339 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fedcd23f60 0x55fedd123690 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.187+0000 7f5822de0740 1 --2- 172.21.15.43:0/769020339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd65390 0x55fedd11d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.187+0000 7f5822de0740 1 --2- 172.21.15.43:0/769020339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd446b0 0x55fedd111830 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.187+0000 7f5822de0740 1 --2- 172.21.15.43:0/769020339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd3c4d0 0x55fedd1112f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.187+0000 7f57fd7fa640 0 client.15975 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:16.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.187+0000 7f5822de0740 1 -- 172.21.15.43:0/769020339 >> 172.21.15.43:0/769020339 conn(0x55fedcd594e0 msgr2=0x55fedd108530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:16.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.187+0000 7f5822de0740 1 -- 172.21.15.43:0/769020339 shutdown_connections 2024-04-25T10:34:16.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.187+0000 7f5822de0740 1 -- 172.21.15.43:0/769020339 wait complete. 2024-04-25T10:34:16.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.187+0000 7f5822de0740 20 client.15975 trim_cache size 0 max 16384 2024-04-25T10:34:16.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.200+0000 7f5822de0740 1 Processor -- start 2024-04-25T10:34:16.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.200+0000 7f5822de0740 1 -- start start 2024-04-25T10:34:16.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.201+0000 7f5822de0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd3c4d0 0x55fedd106c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.201+0000 7f5822de0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd446b0 0x55fedd11ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.201+0000 7f5822de0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 0x55fedd128e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.201+0000 7f5822de0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f58141b2250 con 0x55fedcd65390 2024-04-25T10:34:16.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.201+0000 7f5822de0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f58141afb20 con 0x55fedcd446b0 2024-04-25T10:34:16.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.201+0000 7f5822de0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f58141ad970 con 0x55fedcd3c4d0 2024-04-25T10:34:16.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.201+0000 7f581a4d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 0x55fedd128e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.201+0000 7f581a4d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 0x55fedd128e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41994/0 (socket says 172.21.15.43:41994) 2024-04-25T10:34:16.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.201+0000 7f581a4d2640 1 -- 172.21.15.43:0/433275206 learned_addr learned my addr 172.21.15.43:0/433275206 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:16.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.201+0000 7f5819cd1640 1 --2- 172.21.15.43:0/433275206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd3c4d0 0x55fedd106c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.201+0000 7f58194d0640 1 --2- 172.21.15.43:0/433275206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd446b0 0x55fedd11ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.201+0000 7f581a4d2640 1 -- 172.21.15.43:0/433275206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd3c4d0 msgr2=0x55fedd106c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:16.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.201+0000 7f581a4d2640 1 --2- 172.21.15.43:0/433275206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd3c4d0 0x55fedd106c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.201+0000 7f581a4d2640 1 -- 172.21.15.43:0/433275206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd446b0 msgr2=0x55fedd11ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:16.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.201+0000 7f581a4d2640 1 --2- 172.21.15.43:0/433275206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd446b0 0x55fedd11ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.201+0000 7f581a4d2640 1 -- 172.21.15.43:0/433275206 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fedd108010 con 0x55fedcd65390 2024-04-25T10:34:16.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.201+0000 7f58194d0640 1 --2- 172.21.15.43:0/433275206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd446b0 0x55fedd11ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:16.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.201+0000 7f581a4d2640 1 --2- 172.21.15.43:0/433275206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 0x55fedd128e90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f580c00fc30 tx=0x7f580c009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12513e8dde7f137d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:16.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.202+0000 7f57ff7fe640 1 -- 172.21.15.43:0/433275206 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c029070 con 0x55fedcd65390 2024-04-25T10:34:16.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.202+0000 7f57ff7fe640 1 -- 172.21.15.43:0/433275206 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f580c003bf0 con 0x55fedcd65390 2024-04-25T10:34:16.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.202+0000 7f57ff7fe640 1 -- 172.21.15.43:0/433275206 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c015040 con 0x55fedcd65390 2024-04-25T10:34:16.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.202+0000 7f5822de0740 1 -- 172.21.15.43:0/433275206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 msgr2=0x55fedd128e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:16.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.202+0000 7f5822de0740 1 --2- 172.21.15.43:0/433275206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 0x55fedd128e90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f580c00fc30 tx=0x7f580c009c10 comp rx=0 tx=0).stop 2024-04-25T10:34:16.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.202+0000 7f5822de0740 1 -- 172.21.15.43:0/433275206 shutdown_connections 2024-04-25T10:34:16.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.202+0000 7f5822de0740 1 --2- 172.21.15.43:0/433275206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd446b0 0x55fedd11ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.202+0000 7f5822de0740 1 --2- 172.21.15.43:0/433275206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 0x55fedd128e90 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.202+0000 7f5822de0740 1 --2- 172.21.15.43:0/433275206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd3c4d0 0x55fedd106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.202+0000 7f5822de0740 1 -- 172.21.15.43:0/433275206 >> 172.21.15.43:0/433275206 conn(0x55fedcd594e0 msgr2=0x55fedd10f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:16.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.202+0000 7f5822de0740 1 -- 172.21.15.43:0/433275206 shutdown_connections 2024-04-25T10:34:16.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.202+0000 7f5822de0740 1 -- 172.21.15.43:0/433275206 wait complete. 2024-04-25T10:34:16.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.203+0000 7f5822de0740 1 -- start start 2024-04-25T10:34:16.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.203+0000 7f5822de0740 1 Processor -- start 2024-04-25T10:34:16.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.203+0000 7f5822de0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd3c4d0 0x55fedd119560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.203+0000 7f5822de0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd446b0 0x55fedd1148b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.203+0000 7f5822de0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 0x55fedd114df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.203+0000 7f5822de0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f58141adfe0 con 0x55fedcd65390 2024-04-25T10:34:16.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.203+0000 7f5822de0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f58141af450 con 0x55fedcd446b0 2024-04-25T10:34:16.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.203+0000 7f5822de0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f58141ae6b0 con 0x55fedcd3c4d0 2024-04-25T10:34:16.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.203+0000 7f5822de0740 10 client.0 _ll_register_callbacks cb 0x7f5804005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:16.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.203+0000 7f581a4d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 0x55fedd114df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.203+0000 7f581a4d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 0x55fedd114df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42010/0 (socket says 172.21.15.43:42010) 2024-04-25T10:34:16.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.203+0000 7f581a4d2640 1 -- 172.21.15.43:0/3974396925 learned_addr learned my addr 172.21.15.43:0/3974396925 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:16.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.203+0000 7f5819cd1640 1 --2- 172.21.15.43:0/3974396925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd3c4d0 0x55fedd119560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.204+0000 7f58194d0640 1 --2- 172.21.15.43:0/3974396925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd446b0 0x55fedd1148b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.204+0000 7f581a4d2640 1 -- 172.21.15.43:0/3974396925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd3c4d0 msgr2=0x55fedd119560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:16.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.204+0000 7f581a4d2640 1 --2- 172.21.15.43:0/3974396925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd3c4d0 0x55fedd119560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.204+0000 7f581a4d2640 1 -- 172.21.15.43:0/3974396925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd446b0 msgr2=0x55fedd1148b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:16.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.204+0000 7f581a4d2640 1 --2- 172.21.15.43:0/3974396925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd446b0 0x55fedd1148b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.204+0000 7f581a4d2640 1 -- 172.21.15.43:0/3974396925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fedd10cca0 con 0x55fedcd65390 2024-04-25T10:34:16.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.204+0000 7f581a4d2640 1 --2- 172.21.15.43:0/3974396925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 0x55fedd114df0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f580c009c80 tx=0x7f580c009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b6f0af620d14e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:16.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.204+0000 7f57fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:16.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.204+0000 7f57fd7fa640 1 -- 172.21.15.43:0/3974396925 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c033060 con 0x55fedcd65390 2024-04-25T10:34:16.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.204+0000 7f57fd7fa640 1 -- 172.21.15.43:0/3974396925 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f580c005260 con 0x55fedcd65390 2024-04-25T10:34:16.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.204+0000 7f5822de0740 1 -- 172.21.15.43:0/3974396925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f580c003bf0 con 0x55fedcd65390 2024-04-25T10:34:16.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.204+0000 7f57fd7fa640 1 -- 172.21.15.43:0/3974396925 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f580c015070 con 0x55fedcd65390 2024-04-25T10:34:16.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.204+0000 7f5822de0740 10 client.15981 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:16.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.204+0000 7f5822de0740 1 -- 172.21.15.43:0/3974396925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fedd0fe710 con 0x55fedcd65390 2024-04-25T10:34:16.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.204+0000 7f57fd7fa640 1 -- 172.21.15.43:0/3974396925 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f580c003bf0 con 0x55fedcd65390 2024-04-25T10:34:16.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.205+0000 7f57fd7fa640 1 -- 172.21.15.43:0/3974396925 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f580c023030 con 0x55fedcd65390 2024-04-25T10:34:16.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.205+0000 7f5822de0740 10 client.15981 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:16.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.205+0000 7f5822de0740 10 client.15981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:16.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.205+0000 7f5822de0740 20 client.15981 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:16.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.205+0000 7f5822de0740 1 --2- 172.21.15.43:0/3974396925 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fedccf93a0 0x55fedd18d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:16.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.205+0000 7f5822de0740 4 client.15981 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:34:16.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.205+0000 7f5822de0740 1 -- 172.21.15.43:0/3974396925 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fedd073800 con 0x55fedccf93a0 2024-04-25T10:34:16.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.205+0000 7f5819cd1640 1 --2- 172.21.15.43:0/3974396925 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fedccf93a0 0x55fedd18d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:16.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.205+0000 7f57fd7fa640 10 client.15981 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:16.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.205+0000 7f5819cd1640 1 --2- 172.21.15.43:0/3974396925 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fedccf93a0 0x55fedd18d850 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bfa883eaa25b8b1 server_cookie=640a94b3d528853f in_seq=0 out_seq=0 2024-04-25T10:34:16.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.205+0000 7f57fd7fa640 1 -- 172.21.15.43:0/3974396925 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fedd073800 con 0x55fedccf93a0 2024-04-25T10:34:16.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.205+0000 7f57fd7fa640 10 client.15981 handle_command_reply: tid=0 2024-04-25T10:34:16.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.205+0000 7f5822de0740 1 client.15981 shutdown 2024-04-25T10:34:16.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f5822de0740 1 -- 172.21.15.43:0/3974396925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 msgr2=0x55fedd114df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:16.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f5822de0740 1 --2- 172.21.15.43:0/3974396925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 0x55fedd114df0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f580c009c80 tx=0x7f580c009e70 comp rx=0 tx=0).stop 2024-04-25T10:34:16.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f5822de0740 1 -- 172.21.15.43:0/3974396925 shutdown_connections 2024-04-25T10:34:16.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f5822de0740 1 --2- 172.21.15.43:0/3974396925 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fedccf93a0 0x55fedd18d850 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f5822de0740 1 --2- 172.21.15.43:0/3974396925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fedcd446b0 0x55fedd1148b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f5822de0740 1 --2- 172.21.15.43:0/3974396925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fedcd65390 0x55fedd114df0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f5822de0740 1 --2- 172.21.15.43:0/3974396925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fedcd3c4d0 0x55fedd119560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f57fd7fa640 0 client.15981 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:16.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f5822de0740 1 -- 172.21.15.43:0/3974396925 >> 172.21.15.43:0/3974396925 conn(0x55fedcd594e0 msgr2=0x55fedd108410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:16.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f5822de0740 1 -- 172.21.15.43:0/3974396925 shutdown_connections 2024-04-25T10:34:16.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f5822de0740 1 -- 172.21.15.43:0/3974396925 wait complete. 2024-04-25T10:34:16.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f5822de0740 20 client.15981 trim_cache size 0 max 16384 2024-04-25T10:34:16.216 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:34:16.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f581bf35640 1 -- 172.21.15.43:0/1312231390 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f58000afc60 msgr2=0x7f58000b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:16.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f581bf35640 1 --2- 172.21.15.43:0/1312231390 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f58000afc60 0x7f58000b2170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55fedd107fe0 tx=0x7f581000b390 comp rx=0 tx=0).stop 2024-04-25T10:34:16.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f581bf35640 1 -- 172.21.15.43:0/1312231390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5814107950 msgr2=0x7f581419bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:16.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f581bf35640 1 --2- 172.21.15.43:0/1312231390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5814107950 0x7f581419bb60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5804032fd0 tx=0x7f5804004270 comp rx=0 tx=0).stop 2024-04-25T10:34:16.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f581bf35640 1 -- 172.21.15.43:0/1312231390 shutdown_connections 2024-04-25T10:34:16.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f581bf35640 1 --2- 172.21.15.43:0/1312231390 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f58000afc60 0x7f58000b2170 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f581bf35640 1 --2- 172.21.15.43:0/1312231390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f581410d440 0x7f581419c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f581bf35640 1 --2- 172.21.15.43:0/1312231390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5814107950 0x7f581419bb60 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f581bf35640 1 --2- 172.21.15.43:0/1312231390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58141032c0 0x7f581419b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:16.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f581bf35640 1 -- 172.21.15.43:0/1312231390 >> 172.21.15.43:0/1312231390 conn(0x7f5814077920 msgr2=0x7f58141008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:16.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f581bf35640 1 -- 172.21.15.43:0/1312231390 shutdown_connections 2024-04-25T10:34:16.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:16.206+0000 7f581bf35640 1 -- 172.21.15.43:0/1312231390 wait complete. 2024-04-25T10:34:16.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:16 smithi077 ceph-mon[32772]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-04-25T10:34:16.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:16 smithi077 ceph-mon[32772]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T10:34:16.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:16 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T10:34:16.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:16 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.77:6836/3696941757,v1:172.21.15.77:6837/3696941757] up:boot 2024-04-25T10:34:16.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:16 smithi077 ceph-mon[32772]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T10:34:16.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:16 smithi077 ceph-mon[32772]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T10:34:16.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:16 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2112350978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:34:16.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:16 smithi164 ceph-mon[32713]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-04-25T10:34:16.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:16 smithi164 ceph-mon[32713]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T10:34:16.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:16 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T10:34:16.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:16 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.77:6836/3696941757,v1:172.21.15.77:6837/3696941757] up:boot 2024-04-25T10:34:16.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:16 smithi164 ceph-mon[32713]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T10:34:16.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:16 smithi164 ceph-mon[32713]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T10:34:16.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:16 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2112350978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:34:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:16 smithi043 ceph-mon[29478]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-04-25T10:34:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:16 smithi043 ceph-mon[29478]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T10:34:16.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:16 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T10:34:16.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:16 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.77:6836/3696941757,v1:172.21.15.77:6837/3696941757] up:boot 2024-04-25T10:34:16.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:16 smithi043 ceph-mon[29478]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T10:34:16.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:16 smithi043 ceph-mon[29478]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T10:34:16.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:16 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2112350978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:34:17.220 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:34:17.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:34:17.220 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:34:17.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.349+0000 7f53e2a9a640 1 -- 172.21.15.43:0/2896890162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53dc10b6b0 msgr2=0x7f53dc10dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:17.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.349+0000 7f53e2a9a640 1 --2- 172.21.15.43:0/2896890162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53dc10b6b0 0x7f53dc10dad0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009900 tx=0x7f53cc02a510 comp rx=0 tx=0).stop 2024-04-25T10:34:17.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.349+0000 7f53e2a9a640 1 -- 172.21.15.43:0/2896890162 shutdown_connections 2024-04-25T10:34:17.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.349+0000 7f53e2a9a640 1 --2- 172.21.15.43:0/2896890162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53dc10e060 0x7f53dc110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.349+0000 7f53e2a9a640 1 --2- 172.21.15.43:0/2896890162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53dc10b6b0 0x7f53dc10dad0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.349+0000 7f53e2a9a640 1 --2- 172.21.15.43:0/2896890162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53dc104c50 0x7f53dc105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.349+0000 7f53e2a9a640 1 -- 172.21.15.43:0/2896890162 >> 172.21.15.43:0/2896890162 conn(0x7f53dc078100 msgr2=0x7f53dc078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:17.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.349+0000 7f53e2a9a640 1 -- 172.21.15.43:0/2896890162 shutdown_connections 2024-04-25T10:34:17.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.349+0000 7f53e2a9a640 1 -- 172.21.15.43:0/2896890162 wait complete. 2024-04-25T10:34:17.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.350+0000 7f53e2a9a640 1 Processor -- start 2024-04-25T10:34:17.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.350+0000 7f53e2a9a640 1 -- start start 2024-04-25T10:34:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.350+0000 7f53e2a9a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53dc104c50 0x7f53dc19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.350+0000 7f53e2a9a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53dc10b6b0 0x7f53dc19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.350+0000 7f53e2a9a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53dc10e060 0x7f53dc19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.350+0000 7f53e2a9a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc113cc0 con 0x7f53dc10b6b0 2024-04-25T10:34:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.350+0000 7f53e2a9a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc113b40 con 0x7f53dc10e060 2024-04-25T10:34:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.350+0000 7f53e2a9a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc113e40 con 0x7f53dc104c50 2024-04-25T10:34:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.350+0000 7f53d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53dc10b6b0 0x7f53dc19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.350+0000 7f53d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53dc10b6b0 0x7f53dc19acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42026/0 (socket says 172.21.15.43:42026) 2024-04-25T10:34:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.350+0000 7f53d3fff640 1 -- 172.21.15.43:0/2050451834 learned_addr learned my addr 172.21.15.43:0/2050451834 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.351+0000 7f53e0836640 1 --2- 172.21.15.43:0/2050451834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53dc104c50 0x7f53dc19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.351+0000 7f53e1037640 1 --2- 172.21.15.43:0/2050451834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53dc10e060 0x7f53dc19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.351+0000 7f53d3fff640 1 -- 172.21.15.43:0/2050451834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53dc104c50 msgr2=0x7f53dc19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.351+0000 7f53d3fff640 1 --2- 172.21.15.43:0/2050451834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53dc104c50 0x7f53dc19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.351+0000 7f53d3fff640 1 -- 172.21.15.43:0/2050451834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53dc10e060 msgr2=0x7f53dc19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.351+0000 7f53d3fff640 1 --2- 172.21.15.43:0/2050451834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53dc10e060 0x7f53dc19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.351+0000 7f53d3fff640 1 -- 172.21.15.43:0/2050451834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53dc19f5c0 con 0x7f53dc10b6b0 2024-04-25T10:34:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.351+0000 7f53e1037640 1 --2- 172.21.15.43:0/2050451834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53dc10e060 0x7f53dc19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.351+0000 7f53e0836640 1 --2- 172.21.15.43:0/2050451834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53dc104c50 0x7f53dc19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.351+0000 7f53d3fff640 1 --2- 172.21.15.43:0/2050451834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53dc10b6b0 0x7f53dc19acc0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f53cc005940 tx=0x7f53cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.351+0000 7f53d1ffb640 1 -- 172.21.15.43:0/2050451834 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc03f030 con 0x7f53dc10b6b0 2024-04-25T10:34:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.351+0000 7f53e2a9a640 1 -- 172.21.15.43:0/2050451834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53dc19f850 con 0x7f53dc10b6b0 2024-04-25T10:34:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.351+0000 7f53d1ffb640 1 -- 172.21.15.43:0/2050451834 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53cc003cb0 con 0x7f53dc10b6b0 2024-04-25T10:34:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.351+0000 7f53d1ffb640 1 -- 172.21.15.43:0/2050451834 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc035070 con 0x7f53dc10b6b0 2024-04-25T10:34:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.352+0000 7f53e2a9a640 1 -- 172.21.15.43:0/2050451834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53dc19fd40 con 0x7f53dc10b6b0 2024-04-25T10:34:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.354+0000 7f53e9a1f740 1 Processor -- start 2024-04-25T10:34:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.354+0000 7f53e9a1f740 1 -- start start 2024-04-25T10:34:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.354+0000 7f53d1ffb640 1 -- 172.21.15.43:0/2050451834 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f53cc042030 con 0x7f53dc10b6b0 2024-04-25T10:34:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.354+0000 7f53e9a1f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591d8d9b410 0x5591d913b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.354+0000 7f53e9a1f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d72550 0x5591d913b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.354+0000 7f53e9a1f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d7a730 0x5591d913bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.354+0000 7f53e9a1f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5591d90a1b00 con 0x5591d8d7a730 2024-04-25T10:34:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.354+0000 7f53e9a1f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5591d903d920 con 0x5591d8d9b410 2024-04-25T10:34:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.354+0000 7f53e9a1f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5591d909dac0 con 0x5591d8d72550 2024-04-25T10:34:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.355+0000 7f53d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d7a730 0x5591d913bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.355+0000 7f53e1037640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591d8d9b410 0x5591d913b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.355+0000 7f53d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d7a730 0x5591d913bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42032/0 (socket says 172.21.15.43:42032) 2024-04-25T10:34:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.355+0000 7f53e1037640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591d8d9b410 0x5591d913b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:47526/0 (socket says 172.21.15.43:47526) 2024-04-25T10:34:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.355+0000 7f53d3fff640 1 -- 172.21.15.43:0/2019514978 learned_addr learned my addr 172.21.15.43:0/2019514978 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.355+0000 7f53e0836640 1 --2- 172.21.15.43:0/2019514978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d72550 0x5591d913b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.355+0000 7f53d3fff640 1 -- 172.21.15.43:0/2019514978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d72550 msgr2=0x5591d913b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.355+0000 7f53d3fff640 1 --2- 172.21.15.43:0/2019514978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d72550 0x5591d913b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.355+0000 7f53d3fff640 1 -- 172.21.15.43:0/2019514978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591d8d9b410 msgr2=0x5591d913b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.355+0000 7f53d3fff640 1 --2- 172.21.15.43:0/2019514978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591d8d9b410 0x5591d913b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.355+0000 7f53d3fff640 1 -- 172.21.15.43:0/2019514978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591d8e7b7e0 con 0x5591d8d7a730 2024-04-25T10:34:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.355+0000 7f53d1ffb640 1 --2- 172.21.15.43:0/2050451834 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f53c40afc60 0x7f53c40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.355+0000 7f53d3fff640 1 --2- 172.21.15.43:0/2019514978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d7a730 0x5591d913bf30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0fd030 tx=0x7f53cc105c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716a1f7996ca0233 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.355+0000 7f53e1037640 1 --2- 172.21.15.43:0/2050451834 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f53c40afc60 0x7f53c40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.355+0000 7f53d1ffb640 1 -- 172.21.15.43:0/2050451834 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f53cc0ff940 con 0x7f53dc10b6b0 2024-04-25T10:34:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.356+0000 7f53c2ffd640 1 -- 172.21.15.43:0/2019514978 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0c39d0 con 0x5591d8d7a730 2024-04-25T10:34:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.356+0000 7f53c2ffd640 1 -- 172.21.15.43:0/2019514978 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53cc0c7620 con 0x5591d8d7a730 2024-04-25T10:34:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.356+0000 7f53c2ffd640 1 -- 172.21.15.43:0/2019514978 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0c7280 con 0x5591d8d7a730 2024-04-25T10:34:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.356+0000 7f53e9a1f740 1 -- 172.21.15.43:0/2019514978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d7a730 msgr2=0x5591d913bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.356+0000 7f53e1037640 1 --2- 172.21.15.43:0/2050451834 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f53c40afc60 0x7f53c40b2170 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x5591d91387c0 tx=0x7f53d4009780 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.356+0000 7f53e9a1f740 1 --2- 172.21.15.43:0/2019514978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d7a730 0x5591d913bf30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0fd030 tx=0x7f53cc105c70 comp rx=0 tx=0).stop 2024-04-25T10:34:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.356+0000 7f53e9a1f740 1 -- 172.21.15.43:0/2019514978 shutdown_connections 2024-04-25T10:34:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.356+0000 7f53e9a1f740 1 --2- 172.21.15.43:0/2019514978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d7a730 0x5591d913bf30 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.356+0000 7f53e9a1f740 1 --2- 172.21.15.43:0/2019514978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d72550 0x5591d913b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.356+0000 7f53e9a1f740 1 --2- 172.21.15.43:0/2019514978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591d8d9b410 0x5591d913b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.357+0000 7f53e9a1f740 1 -- 172.21.15.43:0/2019514978 >> 172.21.15.43:0/2019514978 conn(0x5591d8d8f560 msgr2=0x5591d9137dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.357+0000 7f53e9a1f740 1 -- 172.21.15.43:0/2019514978 shutdown_connections 2024-04-25T10:34:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.357+0000 7f53e9a1f740 1 -- 172.21.15.43:0/2019514978 wait complete. 2024-04-25T10:34:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.357+0000 7f53e9a1f740 1 -- start start 2024-04-25T10:34:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.357+0000 7f53e9a1f740 1 Processor -- start 2024-04-25T10:34:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53e9a1f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591d8d72550 0x5591d914a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53e9a1f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d7a730 0x5591d914a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53e9a1f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d9b410 0x5591d914ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53e9a1f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5591d914cb40 con 0x5591d8d7a730 2024-04-25T10:34:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53e9a1f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5591d903b5d0 con 0x5591d8d72550 2024-04-25T10:34:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53e9a1f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5591d914ccc0 con 0x5591d8d9b410 2024-04-25T10:34:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53e9a1f740 10 client.0 _ll_register_callbacks cb 0x5591d9029080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d7a730 0x5591d914a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d7a730 0x5591d914a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42044/0 (socket says 172.21.15.43:42044) 2024-04-25T10:34:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53d3fff640 1 -- 172.21.15.43:0/682313323 learned_addr learned my addr 172.21.15.43:0/682313323 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53e0836640 1 --2- 172.21.15.43:0/682313323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591d8d72550 0x5591d914a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53e1037640 1 --2- 172.21.15.43:0/682313323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d9b410 0x5591d914ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53d3fff640 1 -- 172.21.15.43:0/682313323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d9b410 msgr2=0x5591d914ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53d3fff640 1 --2- 172.21.15.43:0/682313323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d9b410 0x5591d914ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53d3fff640 1 -- 172.21.15.43:0/682313323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591d8d72550 msgr2=0x5591d914a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53d3fff640 1 --2- 172.21.15.43:0/682313323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591d8d72550 0x5591d914a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53d3fff640 1 -- 172.21.15.43:0/682313323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591d8edbf70 con 0x5591d8d7a730 2024-04-25T10:34:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53d3fff640 1 --2- 172.21.15.43:0/682313323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d7a730 0x5591d914a630 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0c3b90 tx=0x7f53cc032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df757a8fb5b09354 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53bbfff640 1 -- 172.21.15.43:0/682313323 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc049070 con 0x5591d8d7a730 2024-04-25T10:34:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53bbfff640 1 -- 172.21.15.43:0/682313323 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f53cc005d50 con 0x5591d8d7a730 2024-04-25T10:34:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.358+0000 7f53e9a1f740 1 -- 172.21.15.43:0/682313323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53cc0c7620 con 0x5591d8d7a730 2024-04-25T10:34:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.359+0000 7f53bbfff640 1 -- 172.21.15.43:0/682313323 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f53cc0c7460 con 0x5591d8d7a730 2024-04-25T10:34:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.359+0000 7f53e9a1f740 10 client.15993 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.359+0000 7f53e9a1f740 1 -- 172.21.15.43:0/682313323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5591d8d73170 con 0x5591d8d7a730 2024-04-25T10:34:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.359+0000 7f53bbfff640 1 -- 172.21.15.43:0/682313323 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53cc0c7620 con 0x5591d8d7a730 2024-04-25T10:34:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.359+0000 7f53bbfff640 1 -- 172.21.15.43:0/682313323 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f53cc0fd340 con 0x5591d8d7a730 2024-04-25T10:34:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.359+0000 7f53e9a1f740 10 client.15993 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.359+0000 7f53e9a1f740 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.359+0000 7f53e9a1f740 20 client.15993 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.359+0000 7f53e9a1f740 1 --2- 172.21.15.43:0/682313323 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5591d8d5a1f0 0x5591d915c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.359+0000 7f53e9a1f740 4 client.15993 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:34:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.359+0000 7f53e9a1f740 1 -- 172.21.15.43:0/682313323 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591d90af710 con 0x5591d8d5a1f0 2024-04-25T10:34:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.359+0000 7f53e0836640 1 --2- 172.21.15.43:0/682313323 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5591d8d5a1f0 0x5591d915c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.360+0000 7f53e0836640 1 --2- 172.21.15.43:0/682313323 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5591d8d5a1f0 0x5591d915c070 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc3b97af87dc0936 server_cookie=4293d405643b2355 in_seq=0 out_seq=0 2024-04-25T10:34:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.360+0000 7f53bbfff640 10 client.15993 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.360+0000 7f53bbfff640 1 -- 172.21.15.43:0/682313323 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5591d90af710 con 0x5591d8d5a1f0 2024-04-25T10:34:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.361+0000 7f53bbfff640 10 client.15993 handle_command_reply: tid=0 2024-04-25T10:34:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.361+0000 7f53e9a1f740 1 client.15993 shutdown 2024-04-25T10:34:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.361+0000 7f53e9a1f740 1 -- 172.21.15.43:0/682313323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d7a730 msgr2=0x5591d914a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.361+0000 7f53e9a1f740 1 --2- 172.21.15.43:0/682313323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d7a730 0x5591d914a630 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0c3b90 tx=0x7f53cc032f60 comp rx=0 tx=0).stop 2024-04-25T10:34:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.361+0000 7f53e9a1f740 1 -- 172.21.15.43:0/682313323 shutdown_connections 2024-04-25T10:34:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.361+0000 7f53e9a1f740 1 --2- 172.21.15.43:0/682313323 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5591d8d5a1f0 0x5591d915c070 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.361+0000 7f53e9a1f740 1 --2- 172.21.15.43:0/682313323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d9b410 0x5591d914ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.361+0000 7f53e9a1f740 1 --2- 172.21.15.43:0/682313323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d7a730 0x5591d914a630 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.361+0000 7f53bbfff640 0 client.15993 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.361+0000 7f53e9a1f740 1 --2- 172.21.15.43:0/682313323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591d8d72550 0x5591d914a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.361+0000 7f53e9a1f740 1 -- 172.21.15.43:0/682313323 >> 172.21.15.43:0/682313323 conn(0x5591d8d8f560 msgr2=0x5591d913d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.361+0000 7f53e9a1f740 1 -- 172.21.15.43:0/682313323 shutdown_connections 2024-04-25T10:34:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.361+0000 7f53e9a1f740 1 -- 172.21.15.43:0/682313323 wait complete. 2024-04-25T10:34:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.361+0000 7f53e9a1f740 20 client.15993 trim_cache size 0 max 16384 2024-04-25T10:34:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.380+0000 7f53e9a1f740 1 Processor -- start 2024-04-25T10:34:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.380+0000 7f53e9a1f740 1 -- start start 2024-04-25T10:34:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.380+0000 7f53e9a1f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d72550 0x5591d913bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.380+0000 7f53e9a1f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591d8d7a730 0x5591d9149690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.380+0000 7f53e9a1f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d9b410 0x5591d9143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.380+0000 7f53e9a1f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1b0060 con 0x5591d8d72550 2024-04-25T10:34:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.380+0000 7f53e9a1f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1ad930 con 0x5591d8d7a730 2024-04-25T10:34:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.380+0000 7f53e9a1f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1ab780 con 0x5591d8d9b410 2024-04-25T10:34:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.380+0000 7f53e0836640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d72550 0x5591d913bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.380+0000 7f53e0836640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d72550 0x5591d913bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42046/0 (socket says 172.21.15.43:42046) 2024-04-25T10:34:17.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.380+0000 7f53e0836640 1 -- 172.21.15.43:0/4191287292 learned_addr learned my addr 172.21.15.43:0/4191287292 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:17.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.380+0000 7f53e1037640 1 --2- 172.21.15.43:0/4191287292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d9b410 0x5591d9143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.380+0000 7f53e0836640 1 -- 172.21.15.43:0/4191287292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d9b410 msgr2=0x5591d9143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:17.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.380+0000 7f53e0836640 1 --2- 172.21.15.43:0/4191287292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d9b410 0x5591d9143860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.380+0000 7f53e0836640 1 -- 172.21.15.43:0/4191287292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591d8d7a730 msgr2=0x5591d9149690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:17.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.380+0000 7f53e0836640 1 --2- 172.21.15.43:0/4191287292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591d8d7a730 0x5591d9149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.380+0000 7f53e0836640 1 -- 172.21.15.43:0/4191287292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591d9137b70 con 0x5591d8d72550 2024-04-25T10:34:17.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.380+0000 7f53e1037640 1 --2- 172.21.15.43:0/4191287292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d9b410 0x5591d9143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:17.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.380+0000 7f53e0836640 1 --2- 172.21.15.43:0/4191287292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d72550 0x5591d913bde0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x5591d9138e80 tx=0x7f53d8011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d91da0f2ec16427 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:17.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.381+0000 7f53c27fc640 1 -- 172.21.15.43:0/4191287292 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d80042f0 con 0x5591d8d72550 2024-04-25T10:34:17.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.381+0000 7f53c27fc640 1 -- 172.21.15.43:0/4191287292 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53d8021070 con 0x5591d8d72550 2024-04-25T10:34:17.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.381+0000 7f53c27fc640 1 -- 172.21.15.43:0/4191287292 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8003c20 con 0x5591d8d72550 2024-04-25T10:34:17.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.381+0000 7f53e9a1f740 1 -- 172.21.15.43:0/4191287292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d72550 msgr2=0x5591d913bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:17.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.381+0000 7f53e9a1f740 1 --2- 172.21.15.43:0/4191287292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d72550 0x5591d913bde0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x5591d9138e80 tx=0x7f53d8011980 comp rx=0 tx=0).stop 2024-04-25T10:34:17.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.381+0000 7f53e9a1f740 1 -- 172.21.15.43:0/4191287292 shutdown_connections 2024-04-25T10:34:17.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.381+0000 7f53e9a1f740 1 --2- 172.21.15.43:0/4191287292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591d8d7a730 0x5591d9149690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.381+0000 7f53e9a1f740 1 --2- 172.21.15.43:0/4191287292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d9b410 0x5591d9143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.381+0000 7f53e9a1f740 1 --2- 172.21.15.43:0/4191287292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d72550 0x5591d913bde0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.381+0000 7f53e9a1f740 1 -- 172.21.15.43:0/4191287292 >> 172.21.15.43:0/4191287292 conn(0x5591d8d8f560 msgr2=0x5591d9136b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:17.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.381+0000 7f53e9a1f740 1 -- 172.21.15.43:0/4191287292 shutdown_connections 2024-04-25T10:34:17.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.382+0000 7f53e9a1f740 1 -- 172.21.15.43:0/4191287292 wait complete. 2024-04-25T10:34:17.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.382+0000 7f53e9a1f740 1 -- start start 2024-04-25T10:34:17.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.382+0000 7f53e9a1f740 1 Processor -- start 2024-04-25T10:34:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.382+0000 7f53e9a1f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d72550 0x5591d9156a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.382+0000 7f53e9a1f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591d8d7a730 0x5591d914fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.382+0000 7f53e9a1f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d9b410 0x5591d9150170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.382+0000 7f53e9a1f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1abdf0 con 0x5591d8d9b410 2024-04-25T10:34:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.382+0000 7f53e9a1f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1ad260 con 0x5591d8d7a730 2024-04-25T10:34:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.382+0000 7f53e9a1f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1ac4c0 con 0x5591d8d72550 2024-04-25T10:34:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.382+0000 7f53e9a1f740 10 client.0 _ll_register_callbacks cb 0x7f53d800b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.383+0000 7f53e1037640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d9b410 0x5591d9150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.383+0000 7f53e1037640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d9b410 0x5591d9150170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42058/0 (socket says 172.21.15.43:42058) 2024-04-25T10:34:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.383+0000 7f53e1037640 1 -- 172.21.15.43:0/3471073273 learned_addr learned my addr 172.21.15.43:0/3471073273 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.383+0000 7f53d3fff640 1 --2- 172.21.15.43:0/3471073273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591d8d7a730 0x5591d914fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.383+0000 7f53e0836640 1 --2- 172.21.15.43:0/3471073273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d72550 0x5591d9156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.383+0000 7f53e1037640 1 -- 172.21.15.43:0/3471073273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d72550 msgr2=0x5591d9156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.383+0000 7f53e1037640 1 --2- 172.21.15.43:0/3471073273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d72550 0x5591d9156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.383+0000 7f53e1037640 1 -- 172.21.15.43:0/3471073273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591d8d7a730 msgr2=0x5591d914fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.383+0000 7f53e1037640 1 --2- 172.21.15.43:0/3471073273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591d8d7a730 0x5591d914fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.383+0000 7f53e1037640 1 -- 172.21.15.43:0/3471073273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591d91508b0 con 0x5591d8d9b410 2024-04-25T10:34:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.383+0000 7f53e1037640 1 --2- 172.21.15.43:0/3471073273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d9b410 0x5591d9150170 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f53d4011950 tx=0x7f53d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d29587816399be6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.383+0000 7f53bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.383+0000 7f53bbfff640 1 -- 172.21.15.43:0/3471073273 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d40228e0 con 0x5591d8d9b410 2024-04-25T10:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.383+0000 7f53bbfff640 1 -- 172.21.15.43:0/3471073273 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f53d4022a80 con 0x5591d8d9b410 2024-04-25T10:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.383+0000 7f53e9a1f740 1 -- 172.21.15.43:0/3471073273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53d8021070 con 0x5591d8d9b410 2024-04-25T10:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.383+0000 7f53bbfff640 1 -- 172.21.15.43:0/3471073273 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f53d4022c80 con 0x5591d8d9b410 2024-04-25T10:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.384+0000 7f53e9a1f740 10 client.15999 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.384+0000 7f53e9a1f740 1 -- 172.21.15.43:0/3471073273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5591d91c4220 con 0x5591d8d9b410 2024-04-25T10:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.384+0000 7f53bbfff640 1 -- 172.21.15.43:0/3471073273 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53d8021070 con 0x5591d8d9b410 2024-04-25T10:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.384+0000 7f53bbfff640 1 -- 172.21.15.43:0/3471073273 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f53d400f040 con 0x5591d8d9b410 2024-04-25T10:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.384+0000 7f53e9a1f740 10 client.15999 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.384+0000 7f53e9a1f740 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.384+0000 7f53e9a1f740 20 client.15999 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.384+0000 7f53e9a1f740 1 --2- 172.21.15.43:0/3471073273 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5591d8d2f630 0x5591d91c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.384+0000 7f53e9a1f740 4 client.15999 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:34:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.384+0000 7f53e9a1f740 1 -- 172.21.15.43:0/3471073273 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5591d90a9580 con 0x5591d8d2f630 2024-04-25T10:34:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.384+0000 7f53e0836640 1 --2- 172.21.15.43:0/3471073273 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5591d8d2f630 0x5591d91c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.385+0000 7f53e0836640 1 --2- 172.21.15.43:0/3471073273 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5591d8d2f630 0x5591d91c6a40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a098e4b0293adccc server_cookie=a512d6305b0f3063 in_seq=0 out_seq=0 2024-04-25T10:34:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.385+0000 7f53bbfff640 10 client.15999 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.386+0000 7f53bbfff640 1 -- 172.21.15.43:0/3471073273 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5591d90a9580 con 0x5591d8d2f630 2024-04-25T10:34:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.386+0000 7f53bbfff640 10 client.15999 handle_command_reply: tid=0 2024-04-25T10:34:17.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.386+0000 7f53e9a1f740 1 client.15999 shutdown 2024-04-25T10:34:17.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.386+0000 7f53e9a1f740 1 -- 172.21.15.43:0/3471073273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d9b410 msgr2=0x5591d9150170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:17.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.386+0000 7f53e9a1f740 1 --2- 172.21.15.43:0/3471073273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d9b410 0x5591d9150170 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f53d4011950 tx=0x7f53d4004a40 comp rx=0 tx=0).stop 2024-04-25T10:34:17.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.386+0000 7f53e9a1f740 1 -- 172.21.15.43:0/3471073273 shutdown_connections 2024-04-25T10:34:17.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.386+0000 7f53e9a1f740 1 --2- 172.21.15.43:0/3471073273 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5591d8d2f630 0x5591d91c6a40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.386+0000 7f53e9a1f740 1 --2- 172.21.15.43:0/3471073273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591d8d7a730 0x5591d914fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.386+0000 7f53bbfff640 0 client.15999 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:17.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.386+0000 7f53e9a1f740 1 --2- 172.21.15.43:0/3471073273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591d8d9b410 0x5591d9150170 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.386+0000 7f53e9a1f740 1 --2- 172.21.15.43:0/3471073273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591d8d72550 0x5591d9156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.386+0000 7f53e9a1f740 1 -- 172.21.15.43:0/3471073273 >> 172.21.15.43:0/3471073273 conn(0x5591d8d8f560 msgr2=0x5591d913d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:17.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.386+0000 7f53e9a1f740 1 -- 172.21.15.43:0/3471073273 shutdown_connections 2024-04-25T10:34:17.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.386+0000 7f53e9a1f740 1 -- 172.21.15.43:0/3471073273 wait complete. 2024-04-25T10:34:17.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.386+0000 7f53e9a1f740 20 client.15999 trim_cache size 0 max 16384 2024-04-25T10:34:17.396 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:34:17.396 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:34:17.396 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "49ecf0e9-2aed-4412-a977-64c12422bcfd", 2024-04-25T10:34:17.396 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:34:17.396 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:34:17.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.387+0000 7f53e2a9a640 1 -- 172.21.15.43:0/2050451834 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f53c40afc60 msgr2=0x7f53c40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:17.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.387+0000 7f53e2a9a640 1 --2- 172.21.15.43:0/2050451834 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f53c40afc60 0x7f53c40b2170 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x5591d91387c0 tx=0x7f53d4009780 comp rx=0 tx=0).stop 2024-04-25T10:34:17.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.387+0000 7f53e2a9a640 1 -- 172.21.15.43:0/2050451834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53dc10b6b0 msgr2=0x7f53dc19acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:17.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.387+0000 7f53e2a9a640 1 --2- 172.21.15.43:0/2050451834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53dc10b6b0 0x7f53dc19acc0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f53cc005940 tx=0x7f53cc0042f0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.387+0000 7f53e2a9a640 1 -- 172.21.15.43:0/2050451834 shutdown_connections 2024-04-25T10:34:17.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.387+0000 7f53e2a9a640 1 --2- 172.21.15.43:0/2050451834 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f53c40afc60 0x7f53c40b2170 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.387+0000 7f53e2a9a640 1 --2- 172.21.15.43:0/2050451834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53dc10e060 0x7f53dc19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.387+0000 7f53e2a9a640 1 --2- 172.21.15.43:0/2050451834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53dc10b6b0 0x7f53dc19acc0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.387+0000 7f53e2a9a640 1 --2- 172.21.15.43:0/2050451834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53dc104c50 0x7f53dc19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.387+0000 7f53e2a9a640 1 -- 172.21.15.43:0/2050451834 >> 172.21.15.43:0/2050451834 conn(0x7f53dc078100 msgr2=0x7f53dc101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:17.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.387+0000 7f53e2a9a640 1 -- 172.21.15.43:0/2050451834 shutdown_connections 2024-04-25T10:34:17.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.387+0000 7f53e2a9a640 1 -- 172.21.15.43:0/2050451834 wait complete. 2024-04-25T10:34:17.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:34:17.400 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:34:17.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.512+0000 7f78a9ae9640 1 -- 172.21.15.43:0/1105466132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a4109fc0 msgr2=0x7f78a4102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:17.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.512+0000 7f78a9ae9640 1 --2- 172.21.15.43:0/1105466132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a4109fc0 0x7f78a4102150 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7898009900 tx=0x7f789802a410 comp rx=0 tx=0).stop 2024-04-25T10:34:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.512+0000 7f78a9ae9640 1 -- 172.21.15.43:0/1105466132 shutdown_connections 2024-04-25T10:34:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.512+0000 7f78a9ae9640 1 --2- 172.21.15.43:0/1105466132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78a41027c0 0x7f78a4110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.512+0000 7f78a9ae9640 1 --2- 172.21.15.43:0/1105466132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a4109fc0 0x7f78a4102150 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.512+0000 7f78a9ae9640 1 --2- 172.21.15.43:0/1105466132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78a4109b20 0x7f78a4101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.512+0000 7f78a9ae9640 1 -- 172.21.15.43:0/1105466132 >> 172.21.15.43:0/1105466132 conn(0x7f78a4077880 msgr2=0x7f78a4077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.512+0000 7f78a9ae9640 1 -- 172.21.15.43:0/1105466132 shutdown_connections 2024-04-25T10:34:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.512+0000 7f78a9ae9640 1 -- 172.21.15.43:0/1105466132 wait complete. 2024-04-25T10:34:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.513+0000 7f78a9ae9640 1 Processor -- start 2024-04-25T10:34:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.513+0000 7f78a9ae9640 1 -- start start 2024-04-25T10:34:17.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.513+0000 7f78a9ae9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78a41027c0 0x7f78a4102df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.513+0000 7f78a9ae9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78a4109b20 0x7f78a4103330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.513+0000 7f78a9ae9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a4109fc0 0x7f78a4103870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.513+0000 7f78a9ae9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78a4113ce0 con 0x7f78a4109fc0 2024-04-25T10:34:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.513+0000 7f78a9ae9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78a4113b60 con 0x7f78a41027c0 2024-04-25T10:34:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.513+0000 7f78a9ae9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f78a4113e60 con 0x7f78a4109b20 2024-04-25T10:34:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.513+0000 7f78a3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a4109fc0 0x7f78a4103870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.513+0000 7f78a3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a4109fc0 0x7f78a4103870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42080/0 (socket says 172.21.15.43:42080) 2024-04-25T10:34:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.513+0000 7f78a3fff640 1 -- 172.21.15.43:0/145020697 learned_addr learned my addr 172.21.15.43:0/145020697 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.513+0000 7f78a37fe640 1 --2- 172.21.15.43:0/145020697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78a41027c0 0x7f78a4102df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.513+0000 7f78a2ffd640 1 --2- 172.21.15.43:0/145020697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78a4109b20 0x7f78a4103330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.514+0000 7f78a3fff640 1 -- 172.21.15.43:0/145020697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78a4109b20 msgr2=0x7f78a4103330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.514+0000 7f78a3fff640 1 --2- 172.21.15.43:0/145020697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78a4109b20 0x7f78a4103330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.514+0000 7f78a3fff640 1 -- 172.21.15.43:0/145020697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78a41027c0 msgr2=0x7f78a4102df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.514+0000 7f78a3fff640 1 --2- 172.21.15.43:0/145020697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78a41027c0 0x7f78a4102df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.514+0000 7f78a3fff640 1 -- 172.21.15.43:0/145020697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a41a8ac0 con 0x7f78a4109fc0 2024-04-25T10:34:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.514+0000 7f78a2ffd640 1 --2- 172.21.15.43:0/145020697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78a4109b20 0x7f78a4103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.514+0000 7f78a3fff640 1 --2- 172.21.15.43:0/145020697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a4109fc0 0x7f78a4103870 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f789c00e9d0 tx=0x7f789c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.514+0000 7f78a0ff9640 1 -- 172.21.15.43:0/145020697 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c00cd80 con 0x7f78a4109fc0 2024-04-25T10:34:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.514+0000 7f78a9ae9640 1 -- 172.21.15.43:0/145020697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a41a8db0 con 0x7f78a4109fc0 2024-04-25T10:34:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.514+0000 7f78a0ff9640 1 -- 172.21.15.43:0/145020697 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f789c005020 con 0x7f78a4109fc0 2024-04-25T10:34:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.514+0000 7f78a0ff9640 1 -- 172.21.15.43:0/145020697 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c0116d0 con 0x7f78a4109fc0 2024-04-25T10:34:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.514+0000 7f78a9ae9640 1 -- 172.21.15.43:0/145020697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a41a9270 con 0x7f78a4109fc0 2024-04-25T10:34:17.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.517+0000 7f78a0ff9640 1 -- 172.21.15.43:0/145020697 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f789c0051d0 con 0x7f78a4109fc0 2024-04-25T10:34:17.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.517+0000 7f78b0995740 1 Processor -- start 2024-04-25T10:34:17.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.517+0000 7f78b0995740 1 -- start start 2024-04-25T10:34:17.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.517+0000 7f78b0995740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70493e390 0x55e704cdea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.517+0000 7f78b0995740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7049154d0 0x55e704cdefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.517+0000 7f78b0995740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70491d6b0 0x55e704ceb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.517+0000 7f78b0995740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e704c44d80 con 0x55e70491d6b0 2024-04-25T10:34:17.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.517+0000 7f78b0995740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e704bfe8b0 con 0x55e70493e390 2024-04-25T10:34:17.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.517+0000 7f78b0995740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e704c40d40 con 0x55e7049154d0 2024-04-25T10:34:17.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.517+0000 7f78a0ff9640 1 --2- 172.21.15.43:0/145020697 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f788c0afc60 0x7f788c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.517+0000 7f78a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70491d6b0 0x55e704ceb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.517+0000 7f78a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70491d6b0 0x55e704ceb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42088/0 (socket says 172.21.15.43:42088) 2024-04-25T10:34:17.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.517+0000 7f78a2ffd640 1 -- 172.21.15.43:0/2244116104 learned_addr learned my addr 172.21.15.43:0/2244116104 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:17.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.517+0000 7f78a2ffd640 1 --2- 172.21.15.43:0/2244116104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70493e390 0x55e704cdea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.517+0000 7f78a3fff640 1 --2- 172.21.15.43:0/145020697 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f788c0afc60 0x7f788c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.517+0000 7f78a37fe640 1 --2- 172.21.15.43:0/2244116104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7049154d0 0x55e704cdefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.517+0000 7f78a0ff9640 1 -- 172.21.15.43:0/145020697 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f789c0df580 con 0x7f78a4109fc0 2024-04-25T10:34:17.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.518+0000 7f78a2ffd640 1 -- 172.21.15.43:0/2244116104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7049154d0 msgr2=0x55e704cdefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:17.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.518+0000 7f78a2ffd640 1 --2- 172.21.15.43:0/2244116104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7049154d0 0x55e704cdefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.518+0000 7f78a2ffd640 1 -- 172.21.15.43:0/2244116104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70493e390 msgr2=0x55e704cdea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:17.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.518+0000 7f78a2ffd640 1 --2- 172.21.15.43:0/2244116104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70493e390 0x55e704cdea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.518+0000 7f78a2ffd640 1 -- 172.21.15.43:0/2244116104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e704bcfa40 con 0x55e70491d6b0 2024-04-25T10:34:17.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.518+0000 7f78a2ffd640 1 --2- 172.21.15.43:0/2244116104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70491d6b0 0x55e704ceb8f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x55e704cdfb60 tx=0x7f7898031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5786852b233ba64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:17.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.518+0000 7f78a3fff640 1 --2- 172.21.15.43:0/145020697 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f788c0afc60 0x7f788c0b2170 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f789c0a49c0 tx=0x7f789c09c040 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.518+0000 7f7889ffb640 1 -- 172.21.15.43:0/2244116104 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898035040 con 0x55e70491d6b0 2024-04-25T10:34:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.518+0000 7f7889ffb640 1 -- 172.21.15.43:0/2244116104 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7898031d10 con 0x55e70491d6b0 2024-04-25T10:34:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.518+0000 7f7889ffb640 1 -- 172.21.15.43:0/2244116104 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78980052b0 con 0x55e70491d6b0 2024-04-25T10:34:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.519+0000 7f78b0995740 1 -- 172.21.15.43:0/2244116104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70491d6b0 msgr2=0x55e704ceb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.519+0000 7f78b0995740 1 --2- 172.21.15.43:0/2244116104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70491d6b0 0x55e704ceb8f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x55e704cdfb60 tx=0x7f7898031400 comp rx=0 tx=0).stop 2024-04-25T10:34:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.519+0000 7f78b0995740 1 -- 172.21.15.43:0/2244116104 shutdown_connections 2024-04-25T10:34:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.519+0000 7f78b0995740 1 --2- 172.21.15.43:0/2244116104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70491d6b0 0x55e704ceb8f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.519+0000 7f78b0995740 1 --2- 172.21.15.43:0/2244116104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7049154d0 0x55e704cdefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.519+0000 7f78b0995740 1 --2- 172.21.15.43:0/2244116104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70493e390 0x55e704cdea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.519+0000 7f78b0995740 1 -- 172.21.15.43:0/2244116104 >> 172.21.15.43:0/2244116104 conn(0x55e7049324e0 msgr2=0x55e704cdb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.519+0000 7f78b0995740 1 -- 172.21.15.43:0/2244116104 shutdown_connections 2024-04-25T10:34:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.519+0000 7f78b0995740 1 -- 172.21.15.43:0/2244116104 wait complete. 2024-04-25T10:34:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.519+0000 7f78b0995740 1 -- start start 2024-04-25T10:34:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.520+0000 7f78b0995740 1 Processor -- start 2024-04-25T10:34:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.520+0000 7f78b0995740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7049154d0 0x55e704ce9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.520+0000 7f78b0995740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70491d6b0 0x55e704ce9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.520+0000 7f78b0995740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70493e390 0x55e704cef420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.520+0000 7f78b0995740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e704cebf20 con 0x55e70491d6b0 2024-04-25T10:34:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.520+0000 7f78b0995740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e704bcbd10 con 0x55e70493e390 2024-04-25T10:34:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.520+0000 7f78b0995740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e704cec0a0 con 0x55e7049154d0 2024-04-25T10:34:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.520+0000 7f78b0995740 10 client.0 _ll_register_callbacks cb 0x55e704bf8210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.520+0000 7f78a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70491d6b0 0x55e704ce9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.520+0000 7f78a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70491d6b0 0x55e704ce9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42104/0 (socket says 172.21.15.43:42104) 2024-04-25T10:34:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.520+0000 7f78a37fe640 1 -- 172.21.15.43:0/3257186040 learned_addr learned my addr 172.21.15.43:0/3257186040 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.520+0000 7f78a2ffd640 1 --2- 172.21.15.43:0/3257186040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7049154d0 0x55e704ce9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.520+0000 7f78a2ffd640 1 --2- 172.21.15.43:0/3257186040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70493e390 0x55e704cef420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.520+0000 7f78a37fe640 1 -- 172.21.15.43:0/3257186040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7049154d0 msgr2=0x55e704ce9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.520+0000 7f78a37fe640 1 --2- 172.21.15.43:0/3257186040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7049154d0 0x55e704ce9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.520+0000 7f78a37fe640 1 -- 172.21.15.43:0/3257186040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70493e390 msgr2=0x55e704cef420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.520+0000 7f78a37fe640 1 --2- 172.21.15.43:0/3257186040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70493e390 0x55e704cef420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.520+0000 7f78a37fe640 1 -- 172.21.15.43:0/3257186040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e704a207d0 con 0x55e70491d6b0 2024-04-25T10:34:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.520+0000 7f78a2ffd640 1 --2- 172.21.15.43:0/3257186040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70493e390 0x55e704cef420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.521+0000 7f78a37fe640 1 --2- 172.21.15.43:0/3257186040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70491d6b0 0x55e704ce9b60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7894012700 tx=0x7f7894012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72ac1f993a22bd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.521+0000 7f7882ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.521+0000 7f7882ffd640 1 -- 172.21.15.43:0/3257186040 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789400f8c0 con 0x55e70491d6b0 2024-04-25T10:34:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.521+0000 7f7882ffd640 1 -- 172.21.15.43:0/3257186040 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f789400fa60 con 0x55e70491d6b0 2024-04-25T10:34:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.521+0000 7f78b0995740 1 -- 172.21.15.43:0/3257186040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7898031d10 con 0x55e70491d6b0 2024-04-25T10:34:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.521+0000 7f7882ffd640 1 -- 172.21.15.43:0/3257186040 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f789400fca0 con 0x55e70491d6b0 2024-04-25T10:34:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.521+0000 7f78b0995740 10 client.16011 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.521+0000 7f78b0995740 1 -- 172.21.15.43:0/3257186040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e7049563f0 con 0x55e70491d6b0 2024-04-25T10:34:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.521+0000 7f7882ffd640 1 -- 172.21.15.43:0/3257186040 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7898031d10 con 0x55e70491d6b0 2024-04-25T10:34:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.521+0000 7f7882ffd640 1 -- 172.21.15.43:0/3257186040 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f789401c420 con 0x55e70491d6b0 2024-04-25T10:34:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.521+0000 7f78b0995740 10 client.16011 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.521+0000 7f78b0995740 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.521+0000 7f78b0995740 20 client.16011 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.521+0000 7f78b0995740 1 --2- 172.21.15.43:0/3257186040 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e7048fcf60 0x55e704cfb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.521+0000 7f78b0995740 4 client.16011 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:34:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.521+0000 7f78b0995740 1 -- 172.21.15.43:0/3257186040 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e704c52990 con 0x55e7048fcf60 2024-04-25T10:34:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.522+0000 7f78a2ffd640 1 --2- 172.21.15.43:0/3257186040 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e7048fcf60 0x55e704cfb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.522+0000 7f7882ffd640 10 client.16011 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.522+0000 7f78a2ffd640 1 --2- 172.21.15.43:0/3257186040 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e7048fcf60 0x55e704cfb760 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffddf0fa087bb081 server_cookie=70c982903811298f in_seq=0 out_seq=0 2024-04-25T10:34:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.523+0000 7f7882ffd640 1 -- 172.21.15.43:0/3257186040 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e704c52990 con 0x55e7048fcf60 2024-04-25T10:34:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.523+0000 7f7882ffd640 10 client.16011 handle_command_reply: tid=0 2024-04-25T10:34:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.523+0000 7f78b0995740 1 client.16011 shutdown 2024-04-25T10:34:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.523+0000 7f78b0995740 1 -- 172.21.15.43:0/3257186040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70491d6b0 msgr2=0x55e704ce9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.523+0000 7f78b0995740 1 --2- 172.21.15.43:0/3257186040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70491d6b0 0x55e704ce9b60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7894012700 tx=0x7f7894012bc0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.523+0000 7f78b0995740 1 -- 172.21.15.43:0/3257186040 shutdown_connections 2024-04-25T10:34:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.523+0000 7f78b0995740 1 --2- 172.21.15.43:0/3257186040 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e7048fcf60 0x55e704cfb760 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.523+0000 7f78b0995740 1 --2- 172.21.15.43:0/3257186040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70493e390 0x55e704cef420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.523+0000 7f78b0995740 1 --2- 172.21.15.43:0/3257186040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70491d6b0 0x55e704ce9b60 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.523+0000 7f7882ffd640 0 client.16011 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.523+0000 7f78b0995740 1 --2- 172.21.15.43:0/3257186040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7049154d0 0x55e704ce9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.523+0000 7f78b0995740 1 -- 172.21.15.43:0/3257186040 >> 172.21.15.43:0/3257186040 conn(0x55e7049324e0 msgr2=0x55e704ce0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.523+0000 7f78b0995740 1 -- 172.21.15.43:0/3257186040 shutdown_connections 2024-04-25T10:34:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.523+0000 7f78b0995740 1 -- 172.21.15.43:0/3257186040 wait complete. 2024-04-25T10:34:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.523+0000 7f78b0995740 20 client.16011 trim_cache size 0 max 16384 2024-04-25T10:34:17.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.541+0000 7f78b0995740 1 Processor -- start 2024-04-25T10:34:17.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.541+0000 7f78b0995740 1 -- start start 2024-04-25T10:34:17.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.542+0000 7f78b0995740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7049154d0 0x55e704ce7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.542+0000 7f78b0995740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70491d6b0 0x55e704ce8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.542+0000 7f78b0995740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e70493e390 0x55e704d00f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.542+0000 7f78b0995740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41b4460 con 0x55e7049154d0 2024-04-25T10:34:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.542+0000 7f78b0995740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41b1d30 con 0x55e70491d6b0 2024-04-25T10:34:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.542+0000 7f78b0995740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41afb80 con 0x55e70493e390 2024-04-25T10:34:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.542+0000 7f78a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7049154d0 0x55e704ce7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.542+0000 7f78a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7049154d0 0x55e704ce7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42116/0 (socket says 172.21.15.43:42116) 2024-04-25T10:34:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.542+0000 7f78a2ffd640 1 -- 172.21.15.43:0/3489424436 learned_addr learned my addr 172.21.15.43:0/3489424436 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.542+0000 7f78a37fe640 1 --2- 172.21.15.43:0/3489424436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70491d6b0 0x55e704ce8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.542+0000 7f78a2ffd640 1 -- 172.21.15.43:0/3489424436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e70493e390 msgr2=0x55e704d00f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.542+0000 7f78a2ffd640 1 --2- 172.21.15.43:0/3489424436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e70493e390 0x55e704d00f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.542+0000 7f78a2ffd640 1 -- 172.21.15.43:0/3489424436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70491d6b0 msgr2=0x55e704ce8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.542+0000 7f78a2ffd640 1 --2- 172.21.15.43:0/3489424436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70491d6b0 0x55e704ce8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.542+0000 7f78a2ffd640 1 -- 172.21.15.43:0/3489424436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7049160f0 con 0x55e7049154d0 2024-04-25T10:34:17.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.542+0000 7f78a2ffd640 1 --2- 172.21.15.43:0/3489424436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7049154d0 0x55e704ce7ce0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7898033000 tx=0x7f7898005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0adafff1347b2d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:17.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.542+0000 7f78897fa640 1 -- 172.21.15.43:0/3489424436 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898054070 con 0x55e7049154d0 2024-04-25T10:34:17.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.542+0000 7f78897fa640 1 -- 172.21.15.43:0/3489424436 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7898005500 con 0x55e7049154d0 2024-04-25T10:34:17.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.542+0000 7f78897fa640 1 -- 172.21.15.43:0/3489424436 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898035040 con 0x55e7049154d0 2024-04-25T10:34:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.543+0000 7f78b0995740 1 -- 172.21.15.43:0/3489424436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7049154d0 msgr2=0x55e704ce7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.543+0000 7f78b0995740 1 --2- 172.21.15.43:0/3489424436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7049154d0 0x55e704ce7ce0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7898033000 tx=0x7f7898005de0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.543+0000 7f78b0995740 1 -- 172.21.15.43:0/3489424436 shutdown_connections 2024-04-25T10:34:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.543+0000 7f78b0995740 1 --2- 172.21.15.43:0/3489424436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70491d6b0 0x55e704ce8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.543+0000 7f78b0995740 1 --2- 172.21.15.43:0/3489424436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e70493e390 0x55e704d00f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.543+0000 7f78b0995740 1 --2- 172.21.15.43:0/3489424436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7049154d0 0x55e704ce7ce0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.543+0000 7f78b0995740 1 -- 172.21.15.43:0/3489424436 >> 172.21.15.43:0/3489424436 conn(0x55e7049324e0 msgr2=0x55e704ce45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.543+0000 7f78b0995740 1 -- 172.21.15.43:0/3489424436 shutdown_connections 2024-04-25T10:34:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.543+0000 7f78b0995740 1 -- 172.21.15.43:0/3489424436 wait complete. 2024-04-25T10:34:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.544+0000 7f78b0995740 1 -- start start 2024-04-25T10:34:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.544+0000 7f78b0995740 1 Processor -- start 2024-04-25T10:34:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.544+0000 7f78b0995740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7049154d0 0x55e704cf0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.544+0000 7f78b0995740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70491d6b0 0x55e704cf0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.544+0000 7f78b0995740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70493e390 0x55e704cf10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.544+0000 7f78b0995740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41b01f0 con 0x55e70493e390 2024-04-25T10:34:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.544+0000 7f78b0995740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41b1660 con 0x55e70491d6b0 2024-04-25T10:34:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.544+0000 7f78b0995740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41b08c0 con 0x55e7049154d0 2024-04-25T10:34:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.544+0000 7f78b0995740 10 client.0 _ll_register_callbacks cb 0x7f7894009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.544+0000 7f78a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70493e390 0x55e704cf10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.544+0000 7f78a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70493e390 0x55e704cf10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42128/0 (socket says 172.21.15.43:42128) 2024-04-25T10:34:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.544+0000 7f78a2ffd640 1 -- 172.21.15.43:0/2087753839 learned_addr learned my addr 172.21.15.43:0/2087753839 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.544+0000 7f78a37fe640 1 --2- 172.21.15.43:0/2087753839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70491d6b0 0x55e704cf0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.544+0000 7f78a2ffd640 1 --2- 172.21.15.43:0/2087753839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7049154d0 0x55e704cf0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.545+0000 7f78a2ffd640 1 -- 172.21.15.43:0/2087753839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7049154d0 msgr2=0x55e704cf0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.545+0000 7f78a2ffd640 1 --2- 172.21.15.43:0/2087753839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7049154d0 0x55e704cf0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.545+0000 7f78a2ffd640 1 -- 172.21.15.43:0/2087753839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70491d6b0 msgr2=0x55e704cf0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.545+0000 7f78a2ffd640 1 --2- 172.21.15.43:0/2087753839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70491d6b0 0x55e704cf0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.545+0000 7f78a2ffd640 1 -- 172.21.15.43:0/2087753839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e704c054b0 con 0x55e70493e390 2024-04-25T10:34:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.545+0000 7f78a37fe640 1 --2- 172.21.15.43:0/2087753839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70491d6b0 0x55e704cf0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.545+0000 7f78a2ffd640 1 --2- 172.21.15.43:0/2087753839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70493e390 0x55e704cf10f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7898052040 tx=0x7f78980047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47421a8f6cb46bac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.545+0000 7f7882ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.545+0000 7f7882ffd640 1 -- 172.21.15.43:0/2087753839 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898035070 con 0x55e70493e390 2024-04-25T10:34:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.545+0000 7f7882ffd640 1 -- 172.21.15.43:0/2087753839 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f789804e470 con 0x55e70493e390 2024-04-25T10:34:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.545+0000 7f78b0995740 1 -- 172.21.15.43:0/2087753839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7898005500 con 0x55e70493e390 2024-04-25T10:34:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.545+0000 7f7882ffd640 1 -- 172.21.15.43:0/2087753839 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7898054040 con 0x55e70493e390 2024-04-25T10:34:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.545+0000 7f78b0995740 10 client.16017 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.545+0000 7f78b0995740 1 -- 172.21.15.43:0/2087753839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e704d62d30 con 0x55e70493e390 2024-04-25T10:34:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.545+0000 7f7882ffd640 1 -- 172.21.15.43:0/2087753839 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7898005500 con 0x55e70493e390 2024-04-25T10:34:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.546+0000 7f7882ffd640 1 -- 172.21.15.43:0/2087753839 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f789805a020 con 0x55e70493e390 2024-04-25T10:34:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.546+0000 7f78b0995740 10 client.16017 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.546+0000 7f78b0995740 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.546+0000 7f78b0995740 20 client.16017 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.546+0000 7f78b0995740 1 --2- 172.21.15.43:0/2087753839 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e7048d23a0 0x55e704d65020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.546+0000 7f78b0995740 4 client.16017 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:34:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.546+0000 7f78b0995740 1 -- 172.21.15.43:0/2087753839 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e704c4c800 con 0x55e7048d23a0 2024-04-25T10:34:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.546+0000 7f78a37fe640 1 --2- 172.21.15.43:0/2087753839 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e7048d23a0 0x55e704d65020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.546+0000 7f7882ffd640 10 client.16017 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.546+0000 7f78a37fe640 1 --2- 172.21.15.43:0/2087753839 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e7048d23a0 0x55e704d65020 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13ae28094ed4c516 server_cookie=da7c967ce908e7bc in_seq=0 out_seq=0 2024-04-25T10:34:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.547+0000 7f7882ffd640 1 -- 172.21.15.43:0/2087753839 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e704c4c800 con 0x55e7048d23a0 2024-04-25T10:34:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.547+0000 7f7882ffd640 10 client.16017 handle_command_reply: tid=0 2024-04-25T10:34:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.547+0000 7f78b0995740 1 client.16017 shutdown 2024-04-25T10:34:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.547+0000 7f78b0995740 1 -- 172.21.15.43:0/2087753839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70493e390 msgr2=0x55e704cf10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.547+0000 7f78b0995740 1 --2- 172.21.15.43:0/2087753839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70493e390 0x55e704cf10f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7898052040 tx=0x7f78980047d0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.547+0000 7f78b0995740 1 -- 172.21.15.43:0/2087753839 shutdown_connections 2024-04-25T10:34:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.547+0000 7f78b0995740 1 --2- 172.21.15.43:0/2087753839 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e7048d23a0 0x55e704d65020 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.547+0000 7f78b0995740 1 --2- 172.21.15.43:0/2087753839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e70491d6b0 0x55e704cf0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.547+0000 7f78b0995740 1 --2- 172.21.15.43:0/2087753839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70493e390 0x55e704cf10f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.547+0000 7f78b0995740 1 --2- 172.21.15.43:0/2087753839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7049154d0 0x55e704cf0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.547+0000 7f78b0995740 1 -- 172.21.15.43:0/2087753839 >> 172.21.15.43:0/2087753839 conn(0x55e7049324e0 msgr2=0x55e704ce45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.547+0000 7f7882ffd640 0 client.16017 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.547+0000 7f78b0995740 1 -- 172.21.15.43:0/2087753839 shutdown_connections 2024-04-25T10:34:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.547+0000 7f78b0995740 1 -- 172.21.15.43:0/2087753839 wait complete. 2024-04-25T10:34:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.547+0000 7f78b0995740 20 client.16017 trim_cache size 0 max 16384 2024-04-25T10:34:17.557 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:34:17.557 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:34:17.557 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:34:17.557 INFO:teuthology.orchestra.run.smithi043.stdout: "49ecf0e9-2aed-4412-a977-64c12422bcfd": { 2024-04-25T10:34:17.557 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:34:17.557 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "49ecf0e9-2aed-4412-a977-64c12422bcfd", 2024-04-25T10:34:17.557 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:34:17.557 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:34:17.557 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:34:17.557 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:34:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.547+0000 7f78a9ae9640 1 -- 172.21.15.43:0/145020697 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f788c0afc60 msgr2=0x7f788c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.548+0000 7f78a9ae9640 1 --2- 172.21.15.43:0/145020697 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f788c0afc60 0x7f788c0b2170 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f789c0a49c0 tx=0x7f789c09c040 comp rx=0 tx=0).stop 2024-04-25T10:34:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.548+0000 7f78a9ae9640 1 -- 172.21.15.43:0/145020697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a4109fc0 msgr2=0x7f78a4103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.548+0000 7f78a9ae9640 1 --2- 172.21.15.43:0/145020697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a4109fc0 0x7f78a4103870 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f789c00e9d0 tx=0x7f789c00ee90 comp rx=0 tx=0).stop 2024-04-25T10:34:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.548+0000 7f78a9ae9640 1 -- 172.21.15.43:0/145020697 shutdown_connections 2024-04-25T10:34:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.548+0000 7f78a9ae9640 1 --2- 172.21.15.43:0/145020697 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f788c0afc60 0x7f788c0b2170 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.548+0000 7f78a9ae9640 1 --2- 172.21.15.43:0/145020697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a4109fc0 0x7f78a4103870 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.548+0000 7f78a9ae9640 1 --2- 172.21.15.43:0/145020697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78a4109b20 0x7f78a4103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.548+0000 7f78a9ae9640 1 --2- 172.21.15.43:0/145020697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78a41027c0 0x7f78a4102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.548+0000 7f78a9ae9640 1 -- 172.21.15.43:0/145020697 >> 172.21.15.43:0/145020697 conn(0x7f78a4077880 msgr2=0x7f78a40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.548+0000 7f78a9ae9640 1 -- 172.21.15.43:0/145020697 shutdown_connections 2024-04-25T10:34:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:17.548+0000 7f78a9ae9640 1 -- 172.21.15.43:0/145020697 wait complete. 2024-04-25T10:34:17.561 INFO:tasks.cephfs.filesystem:scrub status for tag:49ecf0e9-2aed-4412-a977-64c12422bcfd - {'path': '/', 'tag': '49ecf0e9-2aed-4412-a977-64c12422bcfd', 'options': 'recursive,force'} 2024-04-25T10:34:18.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:18 smithi077 ceph-mon[32772]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 7.3 KiB/s wr, 5 op/s 2024-04-25T10:34:18.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:18 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:34:18.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:18 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:34:18.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:18 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:34:18.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:18 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:34:18.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:18 smithi164 ceph-mon[32713]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 7.3 KiB/s wr, 5 op/s 2024-04-25T10:34:18.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:18 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:34:18.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:18 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:34:18.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:18 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:34:18.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:18 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:34:18.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:18 smithi043 ceph-mon[29478]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 7.3 KiB/s wr, 5 op/s 2024-04-25T10:34:18.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:18 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:34:18.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:18 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:34:18.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:18 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:34:18.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:18 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:34:19.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:19 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:34:19.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:19 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:34:19.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:19 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:34:20.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:20 smithi077 ceph-mon[32772]: pgmap v57: 97 pgs: 97 active+clean; 1.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-25T10:34:20.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:20 smithi164 ceph-mon[32713]: pgmap v57: 97 pgs: 97 active+clean; 1.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-25T10:34:20.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:20 smithi043 ceph-mon[29478]: pgmap v57: 97 pgs: 97 active+clean; 1.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-25T10:34:22.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:22 smithi077 ceph-mon[32772]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-25T10:34:22.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:22 smithi164 ceph-mon[32713]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-25T10:34:22.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:22 smithi043 ceph-mon[29478]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-25T10:34:24.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:24 smithi077 ceph-mon[32772]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-25T10:34:24.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:24 smithi164 ceph-mon[32713]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-25T10:34:24.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:24 smithi043 ceph-mon[29478]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-25T10:34:26.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:26 smithi077 ceph-mon[32772]: pgmap v60: 97 pgs: 97 active+clean; 1.6 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 KiB/s wr, 16 op/s 2024-04-25T10:34:26.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:26 smithi164 ceph-mon[32713]: pgmap v60: 97 pgs: 97 active+clean; 1.6 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 KiB/s wr, 16 op/s 2024-04-25T10:34:26.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:26 smithi043 ceph-mon[29478]: pgmap v60: 97 pgs: 97 active+clean; 1.6 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 KiB/s wr, 16 op/s 2024-04-25T10:34:28.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:28 smithi077 ceph-mon[32772]: pgmap v61: 97 pgs: 97 active+clean; 1.6 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-04-25T10:34:28.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:28 smithi164 ceph-mon[32713]: pgmap v61: 97 pgs: 97 active+clean; 1.6 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-04-25T10:34:28.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:28 smithi043 ceph-mon[29478]: pgmap v61: 97 pgs: 97 active+clean; 1.6 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-04-25T10:34:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.968+0000 7fb870b5e640 1 -- 172.21.15.43:0/1948140569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c108ef0 msgr2=0x7fb86c102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.968+0000 7fb870b5e640 1 --2- 172.21.15.43:0/1948140569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c108ef0 0x7fb86c102b40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb860009900 tx=0x7fb86002a410 comp rx=0 tx=0).stop 2024-04-25T10:34:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.969+0000 7fb870b5e640 1 -- 172.21.15.43:0/1948140569 shutdown_connections 2024-04-25T10:34:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.969+0000 7fb870b5e640 1 --2- 172.21.15.43:0/1948140569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb86c1031b0 0x7fb86c10f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.969+0000 7fb870b5e640 1 --2- 172.21.15.43:0/1948140569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c108ef0 0x7fb86c102b40 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.969+0000 7fb870b5e640 1 --2- 172.21.15.43:0/1948140569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb86c108a50 0x7fb86c102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.969+0000 7fb870b5e640 1 -- 172.21.15.43:0/1948140569 >> 172.21.15.43:0/1948140569 conn(0x7fb86c0fe260 msgr2=0x7fb86c100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.969+0000 7fb870b5e640 1 -- 172.21.15.43:0/1948140569 shutdown_connections 2024-04-25T10:34:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.969+0000 7fb870b5e640 1 -- 172.21.15.43:0/1948140569 wait complete. 2024-04-25T10:34:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.969+0000 7fb870b5e640 1 Processor -- start 2024-04-25T10:34:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.970+0000 7fb870b5e640 1 -- start start 2024-04-25T10:34:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.970+0000 7fb870b5e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb86c1031b0 0x7fb86c0fe740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.970+0000 7fb870b5e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb86c108a50 0x7fb86c0fec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.970+0000 7fb870b5e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c108ef0 0x7fb86c0ff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.970+0000 7fb870b5e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c112c50 con 0x7fb86c108ef0 2024-04-25T10:34:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.970+0000 7fb870b5e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c112ad0 con 0x7fb86c108a50 2024-04-25T10:34:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.970+0000 7fb870b5e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c112dd0 con 0x7fb86c1031b0 2024-04-25T10:34:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.970+0000 7fb86ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c108ef0 0x7fb86c0ff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.970+0000 7fb86ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c108ef0 0x7fb86c0ff1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38170/0 (socket says 172.21.15.43:38170) 2024-04-25T10:34:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.970+0000 7fb86ad9d640 1 -- 172.21.15.43:0/67051112 learned_addr learned my addr 172.21.15.43:0/67051112 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.970+0000 7fb869d9b640 1 --2- 172.21.15.43:0/67051112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb86c108a50 0x7fb86c0fec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.970+0000 7fb86a59c640 1 --2- 172.21.15.43:0/67051112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb86c1031b0 0x7fb86c0fe740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.970+0000 7fb86ad9d640 1 -- 172.21.15.43:0/67051112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb86c1031b0 msgr2=0x7fb86c0fe740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.970+0000 7fb86ad9d640 1 --2- 172.21.15.43:0/67051112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb86c1031b0 0x7fb86c0fe740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.970+0000 7fb86ad9d640 1 -- 172.21.15.43:0/67051112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb86c108a50 msgr2=0x7fb86c0fec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.970+0000 7fb86ad9d640 1 --2- 172.21.15.43:0/67051112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb86c108a50 0x7fb86c0fec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.970+0000 7fb86ad9d640 1 -- 172.21.15.43:0/67051112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86c1a8b60 con 0x7fb86c108ef0 2024-04-25T10:34:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.970+0000 7fb86a59c640 1 --2- 172.21.15.43:0/67051112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb86c1031b0 0x7fb86c0fe740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.971+0000 7fb86ad9d640 1 --2- 172.21.15.43:0/67051112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c108ef0 0x7fb86c0ff1c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb86400eb10 tx=0x7fb86400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.971+0000 7fb85b7fe640 1 -- 172.21.15.43:0/67051112 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864004b20 con 0x7fb86c108ef0 2024-04-25T10:34:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.971+0000 7fb870b5e640 1 -- 172.21.15.43:0/67051112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86c1a8e50 con 0x7fb86c108ef0 2024-04-25T10:34:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.971+0000 7fb85b7fe640 1 -- 172.21.15.43:0/67051112 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb864004cc0 con 0x7fb86c108ef0 2024-04-25T10:34:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.971+0000 7fb85b7fe640 1 -- 172.21.15.43:0/67051112 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864010670 con 0x7fb86c108ef0 2024-04-25T10:34:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.971+0000 7fb870b5e640 1 -- 172.21.15.43:0/67051112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb86c1a9290 con 0x7fb86c108ef0 2024-04-25T10:34:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.974+0000 7fb877a0d740 1 Processor -- start 2024-04-25T10:34:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.974+0000 7fb85b7fe640 1 -- 172.21.15.43:0/67051112 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fb864005020 con 0x7fb86c108ef0 2024-04-25T10:34:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.975+0000 7fb877a0d740 1 -- start start 2024-04-25T10:34:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.975+0000 7fb877a0d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9be211b0 0x562b9c1c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.975+0000 7fb877a0d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9bdf82f0 0x562b9c1c1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.975+0000 7fb877a0d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be004d0 0x562b9c1c1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.975+0000 7fb877a0d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562b9c127460 con 0x562b9be004d0 2024-04-25T10:34:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.975+0000 7fb877a0d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562b9c0e1590 con 0x562b9bdf82f0 2024-04-25T10:34:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.975+0000 7fb877a0d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562b9c123420 con 0x562b9be211b0 2024-04-25T10:34:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.975+0000 7fb869d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be004d0 0x562b9c1c1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.975+0000 7fb869d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be004d0 0x562b9c1c1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38176/0 (socket says 172.21.15.43:38176) 2024-04-25T10:34:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.975+0000 7fb869d9b640 1 -- 172.21.15.43:0/1149444997 learned_addr learned my addr 172.21.15.43:0/1149444997 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.975+0000 7fb86a59c640 1 --2- 172.21.15.43:0/1149444997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9bdf82f0 0x562b9c1c1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.975+0000 7fb869d9b640 1 --2- 172.21.15.43:0/1149444997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9be211b0 0x562b9c1c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.975+0000 7fb85b7fe640 1 --2- 172.21.15.43:0/67051112 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb8540afc50 0x7fb8540b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.975+0000 7fb869d9b640 1 -- 172.21.15.43:0/1149444997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9be211b0 msgr2=0x562b9c1c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.975+0000 7fb869d9b640 1 --2- 172.21.15.43:0/1149444997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9be211b0 0x562b9c1c0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.975+0000 7fb869d9b640 1 -- 172.21.15.43:0/1149444997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9bdf82f0 msgr2=0x562b9c1c1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.975+0000 7fb869d9b640 1 --2- 172.21.15.43:0/1149444997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9bdf82f0 0x562b9c1c1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.975+0000 7fb869d9b640 1 -- 172.21.15.43:0/1149444997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b9c0ee0d0 con 0x562b9be004d0 2024-04-25T10:34:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.975+0000 7fb86a59c640 1 --2- 172.21.15.43:0/1149444997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9bdf82f0 0x562b9c1c1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.975+0000 7fb86ad9d640 1 --2- 172.21.15.43:0/67051112 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb8540afc50 0x7fb8540b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:28.981 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.976+0000 7fb85b7fe640 1 -- 172.21.15.43:0/67051112 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fb8640df740 con 0x7fb86c108ef0 2024-04-25T10:34:28.981 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.976+0000 7fb869d9b640 1 --2- 172.21.15.43:0/1149444997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be004d0 0x562b9c1c1570 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb860005940 tx=0x7fb8600058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ccd38c917655c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:28.981 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.976+0000 7fb858ff9640 1 -- 172.21.15.43:0/1149444997 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860035040 con 0x562b9be004d0 2024-04-25T10:34:28.981 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.976+0000 7fb858ff9640 1 -- 172.21.15.43:0/1149444997 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb860004030 con 0x562b9be004d0 2024-04-25T10:34:28.981 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.976+0000 7fb858ff9640 1 -- 172.21.15.43:0/1149444997 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8600038e0 con 0x562b9be004d0 2024-04-25T10:34:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.977+0000 7fb877a0d740 1 -- 172.21.15.43:0/1149444997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be004d0 msgr2=0x562b9c1c1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.977+0000 7fb86ad9d640 1 --2- 172.21.15.43:0/67051112 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb8540afc50 0x7fb8540b2160 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb8640a3210 tx=0x7fb86406b040 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.977+0000 7fb877a0d740 1 --2- 172.21.15.43:0/1149444997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be004d0 0x562b9c1c1570 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb860005940 tx=0x7fb8600058b0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.977+0000 7fb877a0d740 1 -- 172.21.15.43:0/1149444997 shutdown_connections 2024-04-25T10:34:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.977+0000 7fb877a0d740 1 --2- 172.21.15.43:0/1149444997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9bdf82f0 0x562b9c1c1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.977+0000 7fb877a0d740 1 --2- 172.21.15.43:0/1149444997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be004d0 0x562b9c1c1570 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.977+0000 7fb877a0d740 1 --2- 172.21.15.43:0/1149444997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9be211b0 0x562b9c1c0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.977+0000 7fb877a0d740 1 -- 172.21.15.43:0/1149444997 >> 172.21.15.43:0/1149444997 conn(0x562b9be15300 msgr2=0x562b9c1bd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.977+0000 7fb877a0d740 1 -- 172.21.15.43:0/1149444997 shutdown_connections 2024-04-25T10:34:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.977+0000 7fb877a0d740 1 -- 172.21.15.43:0/1149444997 wait complete. 2024-04-25T10:34:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.977+0000 7fb877a0d740 1 -- start start 2024-04-25T10:34:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.977+0000 7fb877a0d740 1 Processor -- start 2024-04-25T10:34:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.978+0000 7fb877a0d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9bdf82f0 0x562b9c1ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.978+0000 7fb877a0d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9be004d0 0x562b9c1cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.978+0000 7fb877a0d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 0x562b9c1cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.978+0000 7fb877a0d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562b9c1cdbc0 con 0x562b9be211b0 2024-04-25T10:34:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.978+0000 7fb877a0d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562b9c0aeb60 con 0x562b9be004d0 2024-04-25T10:34:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.978+0000 7fb877a0d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562b9c1cdd40 con 0x562b9bdf82f0 2024-04-25T10:34:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.978+0000 7fb877a0d740 10 client.0 _ll_register_callbacks cb 0x562b9c0d32b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.978+0000 7fb869d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 0x562b9c1cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.978+0000 7fb869d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 0x562b9c1cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38180/0 (socket says 172.21.15.43:38180) 2024-04-25T10:34:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.978+0000 7fb869d9b640 1 -- 172.21.15.43:0/3752944103 learned_addr learned my addr 172.21.15.43:0/3752944103 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.978+0000 7fb869d9b640 1 --2- 172.21.15.43:0/3752944103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9bdf82f0 0x562b9c1ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.978+0000 7fb86a59c640 1 --2- 172.21.15.43:0/3752944103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9be004d0 0x562b9c1cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.978+0000 7fb869d9b640 1 -- 172.21.15.43:0/3752944103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9bdf82f0 msgr2=0x562b9c1ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.978+0000 7fb869d9b640 1 --2- 172.21.15.43:0/3752944103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9bdf82f0 0x562b9c1ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.978+0000 7fb869d9b640 1 -- 172.21.15.43:0/3752944103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9be004d0 msgr2=0x562b9c1cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.978+0000 7fb869d9b640 1 --2- 172.21.15.43:0/3752944103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9be004d0 0x562b9c1cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.978+0000 7fb869d9b640 1 -- 172.21.15.43:0/3752944103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b9be39210 con 0x562b9be211b0 2024-04-25T10:34:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.978+0000 7fb86a59c640 1 --2- 172.21.15.43:0/3752944103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9be004d0 0x562b9c1cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.979+0000 7fb869d9b640 1 --2- 172.21.15.43:0/3752944103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 0x562b9c1cb010 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb860004610 tx=0x7fb8600095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5795b0c1eab9be84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.979+0000 7fb851ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.979+0000 7fb851ffb640 1 -- 172.21.15.43:0/3752944103 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860035030 con 0x562b9be211b0 2024-04-25T10:34:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.979+0000 7fb877a0d740 1 -- 172.21.15.43:0/3752944103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb860004030 con 0x562b9be211b0 2024-04-25T10:34:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.979+0000 7fb851ffb640 1 -- 172.21.15.43:0/3752944103 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fb860051040 con 0x562b9be211b0 2024-04-25T10:34:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.979+0000 7fb851ffb640 1 -- 172.21.15.43:0/3752944103 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb860003aa0 con 0x562b9be211b0 2024-04-25T10:34:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.979+0000 7fb877a0d740 10 client.16029 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.979+0000 7fb877a0d740 1 -- 172.21.15.43:0/3752944103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562b9bf05620 con 0x562b9be211b0 2024-04-25T10:34:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.979+0000 7fb851ffb640 1 -- 172.21.15.43:0/3752944103 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb860004030 con 0x562b9be211b0 2024-04-25T10:34:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.979+0000 7fb851ffb640 1 -- 172.21.15.43:0/3752944103 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb86003a040 con 0x562b9be211b0 2024-04-25T10:34:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.980+0000 7fb877a0d740 10 client.16029 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.980+0000 7fb877a0d740 10 client.16029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.980+0000 7fb877a0d740 20 client.16029 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.980+0000 7fb877a0d740 1 --2- 172.21.15.43:0/3752944103 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562b9bddfd80 0x562b9c1dd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.980+0000 7fb877a0d740 4 client.16029 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:34:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.980+0000 7fb877a0d740 1 -- 172.21.15.43:0/3752944103 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b9c135070 con 0x562b9bddfd80 2024-04-25T10:34:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.980+0000 7fb86a59c640 1 --2- 172.21.15.43:0/3752944103 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562b9bddfd80 0x562b9c1dd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.980+0000 7fb86a59c640 1 --2- 172.21.15.43:0/3752944103 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562b9bddfd80 0x562b9c1dd5d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49b57ef12e6d56de server_cookie=ce2468e13834d7fa in_seq=0 out_seq=0 2024-04-25T10:34:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.980+0000 7fb851ffb640 10 client.16029 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.981+0000 7fb851ffb640 1 -- 172.21.15.43:0/3752944103 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562b9c135070 con 0x562b9bddfd80 2024-04-25T10:34:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.981+0000 7fb851ffb640 10 client.16029 handle_command_reply: tid=0 2024-04-25T10:34:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.981+0000 7fb877a0d740 1 client.16029 shutdown 2024-04-25T10:34:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.981+0000 7fb877a0d740 1 -- 172.21.15.43:0/3752944103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 msgr2=0x562b9c1cb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.981+0000 7fb877a0d740 1 --2- 172.21.15.43:0/3752944103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 0x562b9c1cb010 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb860004610 tx=0x7fb8600095f0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.981+0000 7fb877a0d740 1 -- 172.21.15.43:0/3752944103 shutdown_connections 2024-04-25T10:34:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.981+0000 7fb877a0d740 1 --2- 172.21.15.43:0/3752944103 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562b9bddfd80 0x562b9c1dd5d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.981+0000 7fb877a0d740 1 --2- 172.21.15.43:0/3752944103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9be004d0 0x562b9c1cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.981+0000 7fb877a0d740 1 --2- 172.21.15.43:0/3752944103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 0x562b9c1cb010 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.981+0000 7fb851ffb640 0 client.16029 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.981+0000 7fb877a0d740 1 --2- 172.21.15.43:0/3752944103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9bdf82f0 0x562b9c1ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.981+0000 7fb877a0d740 1 -- 172.21.15.43:0/3752944103 >> 172.21.15.43:0/3752944103 conn(0x562b9be15300 msgr2=0x562b9c1c2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.981+0000 7fb877a0d740 1 -- 172.21.15.43:0/3752944103 shutdown_connections 2024-04-25T10:34:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.982+0000 7fb877a0d740 1 -- 172.21.15.43:0/3752944103 wait complete. 2024-04-25T10:34:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.982+0000 7fb877a0d740 20 client.16029 trim_cache size 0 max 16384 2024-04-25T10:34:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.997+0000 7fb877a0d740 1 Processor -- start 2024-04-25T10:34:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.997+0000 7fb877a0d740 1 -- start start 2024-04-25T10:34:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.997+0000 7fb877a0d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9bdf82f0 0x562b9c1e0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.997+0000 7fb877a0d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9be004d0 0x562b9c1e0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.997+0000 7fb877a0d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 0x562b9c1c2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.998+0000 7fb877a0d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c1b4480 con 0x562b9be211b0 2024-04-25T10:34:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.998+0000 7fb877a0d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c1b1d50 con 0x562b9be004d0 2024-04-25T10:34:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.998+0000 7fb877a0d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c1afba0 con 0x562b9bdf82f0 2024-04-25T10:34:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.998+0000 7fb869d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 0x562b9c1c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.998+0000 7fb869d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 0x562b9c1c2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38182/0 (socket says 172.21.15.43:38182) 2024-04-25T10:34:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.998+0000 7fb869d9b640 1 -- 172.21.15.43:0/2324484393 learned_addr learned my addr 172.21.15.43:0/2324484393 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.998+0000 7fb869d9b640 1 --2- 172.21.15.43:0/2324484393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9bdf82f0 0x562b9c1e0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.998+0000 7fb86a59c640 1 --2- 172.21.15.43:0/2324484393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9be004d0 0x562b9c1e0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.998+0000 7fb869d9b640 1 -- 172.21.15.43:0/2324484393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9bdf82f0 msgr2=0x562b9c1e0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.998+0000 7fb869d9b640 1 --2- 172.21.15.43:0/2324484393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9bdf82f0 0x562b9c1e0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.998+0000 7fb869d9b640 1 -- 172.21.15.43:0/2324484393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9be004d0 msgr2=0x562b9c1e0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.998+0000 7fb869d9b640 1 --2- 172.21.15.43:0/2324484393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9be004d0 0x562b9c1e0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.998+0000 7fb869d9b640 1 -- 172.21.15.43:0/2324484393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b9c1c2340 con 0x562b9be211b0 2024-04-25T10:34:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.998+0000 7fb869d9b640 1 --2- 172.21.15.43:0/2324484393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 0x562b9c1c2730 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x562b9c1cc0d0 tx=0x7fb86002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3e7080c79f9c79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.998+0000 7fb853fff640 1 -- 172.21.15.43:0/2324484393 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860035070 con 0x562b9be211b0 2024-04-25T10:34:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.998+0000 7fb853fff640 1 -- 172.21.15.43:0/2324484393 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb860004430 con 0x562b9be211b0 2024-04-25T10:34:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.998+0000 7fb853fff640 1 -- 172.21.15.43:0/2324484393 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86003d690 con 0x562b9be211b0 2024-04-25T10:34:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.999+0000 7fb877a0d740 1 -- 172.21.15.43:0/2324484393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 msgr2=0x562b9c1c2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.999+0000 7fb877a0d740 1 --2- 172.21.15.43:0/2324484393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 0x562b9c1c2730 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x562b9c1cc0d0 tx=0x7fb86002a410 comp rx=0 tx=0).stop 2024-04-25T10:34:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.999+0000 7fb877a0d740 1 -- 172.21.15.43:0/2324484393 shutdown_connections 2024-04-25T10:34:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.999+0000 7fb877a0d740 1 --2- 172.21.15.43:0/2324484393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9be004d0 0x562b9c1e0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.999+0000 7fb877a0d740 1 --2- 172.21.15.43:0/2324484393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 0x562b9c1c2730 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.999+0000 7fb877a0d740 1 --2- 172.21.15.43:0/2324484393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9bdf82f0 0x562b9c1e0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.999+0000 7fb877a0d740 1 -- 172.21.15.43:0/2324484393 >> 172.21.15.43:0/2324484393 conn(0x562b9be15300 msgr2=0x562b9c1c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.999+0000 7fb877a0d740 1 -- 172.21.15.43:0/2324484393 shutdown_connections 2024-04-25T10:34:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.999+0000 7fb877a0d740 1 -- 172.21.15.43:0/2324484393 wait complete. 2024-04-25T10:34:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:28.999+0000 7fb877a0d740 1 -- start start 2024-04-25T10:34:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.000+0000 7fb877a0d740 1 Processor -- start 2024-04-25T10:34:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.000+0000 7fb877a0d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9bdf82f0 0x562b9c1dfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.000+0000 7fb877a0d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9be004d0 0x562b9c1e0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.000+0000 7fb877a0d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 0x562b9c23d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.000+0000 7fb877a0d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c1b0210 con 0x562b9be211b0 2024-04-25T10:34:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.000+0000 7fb877a0d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c1b1680 con 0x562b9bdf82f0 2024-04-25T10:34:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.000+0000 7fb877a0d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c1b08e0 con 0x562b9be004d0 2024-04-25T10:34:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.000+0000 7fb877a0d740 10 client.0 _ll_register_callbacks cb 0x7fb860009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.000+0000 7fb869d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 0x562b9c23d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.000+0000 7fb869d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 0x562b9c23d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38188/0 (socket says 172.21.15.43:38188) 2024-04-25T10:34:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.000+0000 7fb869d9b640 1 -- 172.21.15.43:0/3167118551 learned_addr learned my addr 172.21.15.43:0/3167118551 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.000+0000 7fb86a59c640 1 --2- 172.21.15.43:0/3167118551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9be004d0 0x562b9c1e0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.000+0000 7fb869d9b640 1 --2- 172.21.15.43:0/3167118551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9bdf82f0 0x562b9c1dfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.000+0000 7fb869d9b640 1 -- 172.21.15.43:0/3167118551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9be004d0 msgr2=0x562b9c1e0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.000+0000 7fb869d9b640 1 --2- 172.21.15.43:0/3167118551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9be004d0 0x562b9c1e0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.000+0000 7fb869d9b640 1 -- 172.21.15.43:0/3167118551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9bdf82f0 msgr2=0x562b9c1dfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.000+0000 7fb869d9b640 1 --2- 172.21.15.43:0/3167118551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9bdf82f0 0x562b9c1dfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.000+0000 7fb869d9b640 1 -- 172.21.15.43:0/3167118551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b9c23dd90 con 0x562b9be211b0 2024-04-25T10:34:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.000+0000 7fb86a59c640 1 --2- 172.21.15.43:0/3167118551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9be004d0 0x562b9c1e0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.001+0000 7fb869d9b640 1 --2- 172.21.15.43:0/3167118551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 0x562b9c23d820 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb860009900 tx=0x7fb860032e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4614e34d725bab63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.001+0000 7fb851ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.001+0000 7fb851ffb640 1 -- 172.21.15.43:0/3167118551 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860035040 con 0x562b9be211b0 2024-04-25T10:34:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.001+0000 7fb851ffb640 1 -- 172.21.15.43:0/3167118551 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fb8600317b0 con 0x562b9be211b0 2024-04-25T10:34:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.001+0000 7fb877a0d740 1 -- 172.21.15.43:0/3167118551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb860004430 con 0x562b9be211b0 2024-04-25T10:34:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.001+0000 7fb851ffb640 1 -- 172.21.15.43:0/3167118551 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb860009520 con 0x562b9be211b0 2024-04-25T10:34:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.001+0000 7fb877a0d740 10 client.16035 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.001+0000 7fb877a0d740 1 -- 172.21.15.43:0/3167118551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562b9c245040 con 0x562b9be211b0 2024-04-25T10:34:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.001+0000 7fb851ffb640 1 -- 172.21.15.43:0/3167118551 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb860004430 con 0x562b9be211b0 2024-04-25T10:34:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.001+0000 7fb851ffb640 1 -- 172.21.15.43:0/3167118551 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb860048580 con 0x562b9be211b0 2024-04-25T10:34:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.001+0000 7fb877a0d740 10 client.16035 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.001+0000 7fb877a0d740 10 client.16035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.001+0000 7fb877a0d740 20 client.16035 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.001+0000 7fb877a0d740 1 --2- 172.21.15.43:0/3167118551 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562b9bdb51c0 0x562b9c247370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.001+0000 7fb877a0d740 4 client.16035 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:34:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.001+0000 7fb877a0d740 1 -- 172.21.15.43:0/3167118551 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b9c12eee0 con 0x562b9bdb51c0 2024-04-25T10:34:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.002+0000 7fb86a59c640 1 --2- 172.21.15.43:0/3167118551 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562b9bdb51c0 0x562b9c247370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.002+0000 7fb86a59c640 1 --2- 172.21.15.43:0/3167118551 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562b9bdb51c0 0x562b9c247370 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=838ac43188845d48 server_cookie=cf5317f9f34d416b in_seq=0 out_seq=0 2024-04-25T10:34:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.002+0000 7fb851ffb640 10 client.16035 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb851ffb640 1 -- 172.21.15.43:0/3167118551 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562b9c12eee0 con 0x562b9bdb51c0 2024-04-25T10:34:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb851ffb640 10 client.16035 handle_command_reply: tid=0 2024-04-25T10:34:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb877a0d740 1 client.16035 shutdown 2024-04-25T10:34:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb877a0d740 1 -- 172.21.15.43:0/3167118551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 msgr2=0x562b9c23d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb877a0d740 1 --2- 172.21.15.43:0/3167118551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 0x562b9c23d820 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb860009900 tx=0x7fb860032e80 comp rx=0 tx=0).stop 2024-04-25T10:34:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb877a0d740 1 -- 172.21.15.43:0/3167118551 shutdown_connections 2024-04-25T10:34:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb877a0d740 1 --2- 172.21.15.43:0/3167118551 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562b9bdb51c0 0x562b9c247370 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb877a0d740 1 --2- 172.21.15.43:0/3167118551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b9be211b0 0x562b9c23d820 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb877a0d740 1 --2- 172.21.15.43:0/3167118551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b9be004d0 0x562b9c1e0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb851ffb640 0 client.16035 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb877a0d740 1 --2- 172.21.15.43:0/3167118551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b9bdf82f0 0x562b9c1dfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb877a0d740 1 -- 172.21.15.43:0/3167118551 >> 172.21.15.43:0/3167118551 conn(0x562b9be15300 msgr2=0x562b9c1c2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb877a0d740 1 -- 172.21.15.43:0/3167118551 shutdown_connections 2024-04-25T10:34:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb877a0d740 1 -- 172.21.15.43:0/3167118551 wait complete. 2024-04-25T10:34:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb877a0d740 20 client.16035 trim_cache size 0 max 16384 2024-04-25T10:34:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:34:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:34:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:34:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:34:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb870b5e640 1 -- 172.21.15.43:0/67051112 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb8540afc50 msgr2=0x7fb8540b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb870b5e640 1 --2- 172.21.15.43:0/67051112 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb8540afc50 0x7fb8540b2160 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb8640a3210 tx=0x7fb86406b040 comp rx=0 tx=0).stop 2024-04-25T10:34:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb870b5e640 1 -- 172.21.15.43:0/67051112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c108ef0 msgr2=0x7fb86c0ff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb870b5e640 1 --2- 172.21.15.43:0/67051112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c108ef0 0x7fb86c0ff1c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb86400eb10 tx=0x7fb86400c2d0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb870b5e640 1 -- 172.21.15.43:0/67051112 shutdown_connections 2024-04-25T10:34:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb870b5e640 1 --2- 172.21.15.43:0/67051112 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb8540afc50 0x7fb8540b2160 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb870b5e640 1 --2- 172.21.15.43:0/67051112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb86c108a50 0x7fb86c0fec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb870b5e640 1 --2- 172.21.15.43:0/67051112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c108ef0 0x7fb86c0ff1c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb870b5e640 1 --2- 172.21.15.43:0/67051112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb86c1031b0 0x7fb86c0fe740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb870b5e640 1 -- 172.21.15.43:0/67051112 >> 172.21.15.43:0/67051112 conn(0x7fb86c0fe260 msgr2=0x7fb86c100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb870b5e640 1 -- 172.21.15.43:0/67051112 shutdown_connections 2024-04-25T10:34:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:29.017+0000 7fb870b5e640 1 -- 172.21.15.43:0/67051112 wait complete. 2024-04-25T10:34:30.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:30 smithi077 ceph-mon[32772]: pgmap v62: 97 pgs: 97 active+clean; 1.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 27 KiB/s wr, 18 op/s 2024-04-25T10:34:30.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:30 smithi164 ceph-mon[32713]: pgmap v62: 97 pgs: 97 active+clean; 1.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 27 KiB/s wr, 18 op/s 2024-04-25T10:34:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:30 smithi043 ceph-mon[29478]: pgmap v62: 97 pgs: 97 active+clean; 1.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 27 KiB/s wr, 18 op/s 2024-04-25T10:34:32.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:32 smithi077 ceph-mon[32772]: pgmap v63: 97 pgs: 97 active+clean; 1.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-04-25T10:34:32.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:32 smithi164 ceph-mon[32713]: pgmap v63: 97 pgs: 97 active+clean; 1.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-04-25T10:34:32.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:32 smithi043 ceph-mon[29478]: pgmap v63: 97 pgs: 97 active+clean; 1.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-04-25T10:34:34.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:34 smithi077 ceph-mon[32772]: pgmap v64: 97 pgs: 97 active+clean; 1.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-04-25T10:34:34.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:34 smithi164 ceph-mon[32713]: pgmap v64: 97 pgs: 97 active+clean; 1.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-04-25T10:34:34.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:34 smithi043 ceph-mon[29478]: pgmap v64: 97 pgs: 97 active+clean; 1.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-04-25T10:34:36.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:36 smithi077 ceph-mon[32772]: pgmap v65: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-25T10:34:36.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:36 smithi164 ceph-mon[32713]: pgmap v65: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-25T10:34:36.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:36 smithi043 ceph-mon[29478]: pgmap v65: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-25T10:34:38.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:38 smithi043 ceph-mon[29478]: pgmap v66: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-04-25T10:34:38.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:38 smithi077 ceph-mon[32772]: pgmap v66: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-04-25T10:34:38.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:38 smithi164 ceph-mon[32713]: pgmap v66: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-04-25T10:34:40.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:40 smithi043 ceph-mon[29478]: pgmap v67: 97 pgs: 97 active+clean; 1.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-04-25T10:34:40.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:40 smithi077 ceph-mon[32772]: pgmap v67: 97 pgs: 97 active+clean; 1.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-04-25T10:34:40.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:40 smithi164 ceph-mon[32713]: pgmap v67: 97 pgs: 97 active+clean; 1.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-04-25T10:34:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.132+0000 7fddec101640 1 -- 172.21.15.43:0/3108119336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdde410dab0 msgr2=0x7fdde410fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.132+0000 7fddec101640 1 --2- 172.21.15.43:0/3108119336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdde410dab0 0x7fdde410fed0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009900 tx=0x7fddd402a410 comp rx=0 tx=0).stop 2024-04-25T10:34:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.132+0000 7fddec101640 1 -- 172.21.15.43:0/3108119336 shutdown_connections 2024-04-25T10:34:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.132+0000 7fddec101640 1 --2- 172.21.15.43:0/3108119336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdde4110410 0x7fdde4112830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.132+0000 7fddec101640 1 --2- 172.21.15.43:0/3108119336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdde410dab0 0x7fdde410fed0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.132+0000 7fddec101640 1 --2- 172.21.15.43:0/3108119336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdde4102870 0x7fdde4102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.132+0000 7fddec101640 1 -- 172.21.15.43:0/3108119336 >> 172.21.15.43:0/3108119336 conn(0x7fdde40fe260 msgr2=0x7fdde4100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.132+0000 7fddec101640 1 -- 172.21.15.43:0/3108119336 shutdown_connections 2024-04-25T10:34:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.132+0000 7fddec101640 1 -- 172.21.15.43:0/3108119336 wait complete. 2024-04-25T10:34:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.132+0000 7fddec101640 1 Processor -- start 2024-04-25T10:34:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.133+0000 7fddec101640 1 -- start start 2024-04-25T10:34:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.133+0000 7fddec101640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdde4102870 0x7fdde41a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.133+0000 7fddec101640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdde410dab0 0x7fdde41a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.133+0000 7fddec101640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdde4110410 0x7fdde419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.133+0000 7fddec101640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdde4115f20 con 0x7fdde4110410 2024-04-25T10:34:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.133+0000 7fddec101640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdde4115da0 con 0x7fdde410dab0 2024-04-25T10:34:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.133+0000 7fddec101640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41160a0 con 0x7fdde4102870 2024-04-25T10:34:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.133+0000 7fddea69e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdde4110410 0x7fdde419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.133+0000 7fddea69e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdde4110410 0x7fdde419c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46818/0 (socket says 172.21.15.43:46818) 2024-04-25T10:34:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.133+0000 7fddea69e640 1 -- 172.21.15.43:0/3790868723 learned_addr learned my addr 172.21.15.43:0/3790868723 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.133+0000 7fdde9e9d640 1 --2- 172.21.15.43:0/3790868723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdde4102870 0x7fdde41a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.133+0000 7fdde969c640 1 --2- 172.21.15.43:0/3790868723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdde410dab0 0x7fdde41a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.133+0000 7fddea69e640 1 -- 172.21.15.43:0/3790868723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdde4102870 msgr2=0x7fdde41a2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.133+0000 7fddea69e640 1 --2- 172.21.15.43:0/3790868723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdde4102870 0x7fdde41a2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.133+0000 7fddea69e640 1 -- 172.21.15.43:0/3790868723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdde410dab0 msgr2=0x7fdde41a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.133+0000 7fddea69e640 1 --2- 172.21.15.43:0/3790868723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdde410dab0 0x7fdde41a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.133+0000 7fddea69e640 1 -- 172.21.15.43:0/3790868723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde419cee0 con 0x7fdde4110410 2024-04-25T10:34:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.134+0000 7fddea69e640 1 --2- 172.21.15.43:0/3790868723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdde4110410 0x7fdde419c800 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00eaa0 tx=0x7fdddc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.134+0000 7fdddaffd640 1 -- 172.21.15.43:0/3790868723 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc00cdb0 con 0x7fdde4110410 2024-04-25T10:34:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.134+0000 7fddec101640 1 -- 172.21.15.43:0/3790868723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde419d1d0 con 0x7fdde4110410 2024-04-25T10:34:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.134+0000 7fdddaffd640 1 -- 172.21.15.43:0/3790868723 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdddc005020 con 0x7fdde4110410 2024-04-25T10:34:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.134+0000 7fdddaffd640 1 -- 172.21.15.43:0/3790868723 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc0116d0 con 0x7fdde4110410 2024-04-25T10:34:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.134+0000 7fddec101640 1 -- 172.21.15.43:0/3790868723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde41a93d0 con 0x7fdde4110410 2024-04-25T10:34:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.136+0000 7fddf2fb0740 1 Processor -- start 2024-04-25T10:34:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.137+0000 7fdddaffd640 1 -- 172.21.15.43:0/3790868723 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fdddc0054a0 con 0x7fdde4110410 2024-04-25T10:34:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.137+0000 7fddf2fb0740 1 -- start start 2024-04-25T10:34:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.137+0000 7fddf2fb0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be6f511b0 0x564be72f0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.137+0000 7fddf2fb0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f282f0 0x564be72f1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.137+0000 7fddf2fb0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f304d0 0x564be72f1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.137+0000 7fddf2fb0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564be7257460 con 0x564be6f304d0 2024-04-25T10:34:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.137+0000 7fddf2fb0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564be7211590 con 0x564be6f282f0 2024-04-25T10:34:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.137+0000 7fddf2fb0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564be7253420 con 0x564be6f511b0 2024-04-25T10:34:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.137+0000 7fdde969c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f304d0 0x564be72f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.137+0000 7fdde969c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f304d0 0x564be72f1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46828/0 (socket says 172.21.15.43:46828) 2024-04-25T10:34:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.137+0000 7fdde969c640 1 -- 172.21.15.43:0/4024608631 learned_addr learned my addr 172.21.15.43:0/4024608631 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.138+0000 7fdde969c640 1 --2- 172.21.15.43:0/4024608631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be6f511b0 0x564be72f0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.138+0000 7fdde9e9d640 1 --2- 172.21.15.43:0/4024608631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f282f0 0x564be72f1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.138+0000 7fdddaffd640 1 --2- 172.21.15.43:0/3790868723 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fddd00afc50 0x7fddd00b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.138+0000 7fddea69e640 1 --2- 172.21.15.43:0/3790868723 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fddd00afc50 0x7fddd00b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.138+0000 7fdde969c640 1 -- 172.21.15.43:0/4024608631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be6f511b0 msgr2=0x564be72f0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.138+0000 7fdde969c640 1 --2- 172.21.15.43:0/4024608631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be6f511b0 0x564be72f0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.138+0000 7fdde969c640 1 -- 172.21.15.43:0/4024608631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f282f0 msgr2=0x564be72f1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.138+0000 7fdde969c640 1 --2- 172.21.15.43:0/4024608631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f282f0 0x564be72f1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.138+0000 7fdde969c640 1 -- 172.21.15.43:0/4024608631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564be721e0d0 con 0x564be6f304d0 2024-04-25T10:34:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.138+0000 7fdddaffd640 1 -- 172.21.15.43:0/3790868723 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fdddc0df0c0 con 0x7fdde4110410 2024-04-25T10:34:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.138+0000 7fdde969c640 1 --2- 172.21.15.43:0/4024608631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f304d0 0x564be72f1570 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fddd40057c0 tx=0x7fddd40058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d2e1f77099071b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.138+0000 7fddcffff640 1 -- 172.21.15.43:0/4024608631 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4035040 con 0x564be6f304d0 2024-04-25T10:34:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.138+0000 7fddcffff640 1 -- 172.21.15.43:0/4024608631 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddd40035f0 con 0x564be6f304d0 2024-04-25T10:34:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.138+0000 7fddcffff640 1 -- 172.21.15.43:0/4024608631 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd40044e0 con 0x564be6f304d0 2024-04-25T10:34:42.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.139+0000 7fddf2fb0740 1 -- 172.21.15.43:0/4024608631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f304d0 msgr2=0x564be72f1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.139+0000 7fddea69e640 1 --2- 172.21.15.43:0/3790868723 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fddd00afc50 0x7fddd00b2160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0dc990 tx=0x7fdddc075e30 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:42.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.139+0000 7fddf2fb0740 1 --2- 172.21.15.43:0/4024608631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f304d0 0x564be72f1570 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fddd40057c0 tx=0x7fddd40058d0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.139+0000 7fddf2fb0740 1 -- 172.21.15.43:0/4024608631 shutdown_connections 2024-04-25T10:34:42.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.139+0000 7fddf2fb0740 1 --2- 172.21.15.43:0/4024608631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f282f0 0x564be72f1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.139+0000 7fddf2fb0740 1 --2- 172.21.15.43:0/4024608631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f304d0 0x564be72f1570 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.139+0000 7fddf2fb0740 1 --2- 172.21.15.43:0/4024608631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be6f511b0 0x564be72f0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.139+0000 7fddf2fb0740 1 -- 172.21.15.43:0/4024608631 >> 172.21.15.43:0/4024608631 conn(0x564be6f45300 msgr2=0x564be72ed2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.139+0000 7fddf2fb0740 1 -- 172.21.15.43:0/4024608631 shutdown_connections 2024-04-25T10:34:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.139+0000 7fddf2fb0740 1 -- 172.21.15.43:0/4024608631 wait complete. 2024-04-25T10:34:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.140+0000 7fddf2fb0740 1 -- start start 2024-04-25T10:34:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.140+0000 7fddf2fb0740 1 Processor -- start 2024-04-25T10:34:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.140+0000 7fddf2fb0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f282f0 0x564be72fcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.140+0000 7fddf2fb0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f304d0 0x564be72fd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.140+0000 7fddf2fb0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be6f511b0 0x564be72fb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.140+0000 7fddf2fb0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564be72fdbc0 con 0x564be6f282f0 2024-04-25T10:34:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.140+0000 7fddf2fb0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564be71deb60 con 0x564be6f304d0 2024-04-25T10:34:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.140+0000 7fddf2fb0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564be72fdd40 con 0x564be6f511b0 2024-04-25T10:34:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.140+0000 7fddf2fb0740 10 client.0 _ll_register_callbacks cb 0x564be72032b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.141+0000 7fdde969c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f282f0 0x564be72fcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.141+0000 7fdde969c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f282f0 0x564be72fcb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46830/0 (socket says 172.21.15.43:46830) 2024-04-25T10:34:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.141+0000 7fdde9e9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f304d0 0x564be72fd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.141+0000 7fdde969c640 1 -- 172.21.15.43:0/2772023464 learned_addr learned my addr 172.21.15.43:0/2772023464 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.141+0000 7fdde969c640 1 --2- 172.21.15.43:0/2772023464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be6f511b0 0x564be72fb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.141+0000 7fdde969c640 1 -- 172.21.15.43:0/2772023464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be6f511b0 msgr2=0x564be72fb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.141+0000 7fdde969c640 1 --2- 172.21.15.43:0/2772023464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be6f511b0 0x564be72fb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.141+0000 7fdde969c640 1 -- 172.21.15.43:0/2772023464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f304d0 msgr2=0x564be72fd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.141+0000 7fdde969c640 1 --2- 172.21.15.43:0/2772023464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f304d0 0x564be72fd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.141+0000 7fdde969c640 1 -- 172.21.15.43:0/2772023464 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564be6f69210 con 0x564be6f282f0 2024-04-25T10:34:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.141+0000 7fdde9e9d640 1 --2- 172.21.15.43:0/2772023464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f304d0 0x564be72fd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.141+0000 7fdde969c640 1 --2- 172.21.15.43:0/2772023464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f282f0 0x564be72fcb40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fddd402a930 tx=0x7fddd4003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d648247edb9e735 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.141+0000 7fddcd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.141+0000 7fddcd7fa640 1 -- 172.21.15.43:0/2772023464 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4035050 con 0x564be6f282f0 2024-04-25T10:34:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.141+0000 7fddcd7fa640 1 -- 172.21.15.43:0/2772023464 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fddd40043f0 con 0x564be6f282f0 2024-04-25T10:34:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.141+0000 7fddf2fb0740 1 -- 172.21.15.43:0/2772023464 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddd40035f0 con 0x564be6f282f0 2024-04-25T10:34:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.142+0000 7fddcd7fa640 1 -- 172.21.15.43:0/2772023464 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fddd40037a0 con 0x564be6f282f0 2024-04-25T10:34:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.142+0000 7fddf2fb0740 10 client.16047 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.142+0000 7fddf2fb0740 1 -- 172.21.15.43:0/2772023464 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564be7035620 con 0x564be6f282f0 2024-04-25T10:34:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.142+0000 7fddcd7fa640 1 -- 172.21.15.43:0/2772023464 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddd40035f0 con 0x564be6f282f0 2024-04-25T10:34:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.142+0000 7fddcd7fa640 1 -- 172.21.15.43:0/2772023464 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fddd403a040 con 0x564be6f282f0 2024-04-25T10:34:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.142+0000 7fddf2fb0740 10 client.16047 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.142+0000 7fddf2fb0740 10 client.16047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.142+0000 7fddf2fb0740 20 client.16047 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.142+0000 7fddf2fb0740 1 --2- 172.21.15.43:0/2772023464 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564be6f0fd80 0x564be730d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.142+0000 7fddf2fb0740 4 client.16047 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:34:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.142+0000 7fddf2fb0740 1 -- 172.21.15.43:0/2772023464 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564be7265070 con 0x564be6f0fd80 2024-04-25T10:34:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.142+0000 7fdde9e9d640 1 --2- 172.21.15.43:0/2772023464 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564be6f0fd80 0x564be730d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.143+0000 7fdde9e9d640 1 --2- 172.21.15.43:0/2772023464 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564be6f0fd80 0x564be730d5a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdbfd5d283e25cc4 server_cookie=b57f7de534499c34 in_seq=0 out_seq=0 2024-04-25T10:34:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.143+0000 7fddcd7fa640 10 client.16047 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.144+0000 7fddcd7fa640 1 -- 172.21.15.43:0/2772023464 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564be7265070 con 0x564be6f0fd80 2024-04-25T10:34:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.144+0000 7fddcd7fa640 10 client.16047 handle_command_reply: tid=0 2024-04-25T10:34:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.144+0000 7fddf2fb0740 1 client.16047 shutdown 2024-04-25T10:34:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.144+0000 7fddf2fb0740 1 -- 172.21.15.43:0/2772023464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f282f0 msgr2=0x564be72fcb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.144+0000 7fddf2fb0740 1 --2- 172.21.15.43:0/2772023464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f282f0 0x564be72fcb40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fddd402a930 tx=0x7fddd4003ed0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.144+0000 7fddf2fb0740 1 -- 172.21.15.43:0/2772023464 shutdown_connections 2024-04-25T10:34:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.144+0000 7fddf2fb0740 1 --2- 172.21.15.43:0/2772023464 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564be6f0fd80 0x564be730d5a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.144+0000 7fddf2fb0740 1 --2- 172.21.15.43:0/2772023464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f304d0 0x564be72fd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.144+0000 7fddf2fb0740 1 --2- 172.21.15.43:0/2772023464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be6f511b0 0x564be72fb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.144+0000 7fddf2fb0740 1 --2- 172.21.15.43:0/2772023464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f282f0 0x564be72fcb40 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.144+0000 7fddf2fb0740 1 -- 172.21.15.43:0/2772023464 >> 172.21.15.43:0/2772023464 conn(0x564be6f45300 msgr2=0x564be72f2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.144+0000 7fddcd7fa640 0 client.16047 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.144+0000 7fddf2fb0740 1 -- 172.21.15.43:0/2772023464 shutdown_connections 2024-04-25T10:34:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.144+0000 7fddf2fb0740 1 -- 172.21.15.43:0/2772023464 wait complete. 2024-04-25T10:34:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.144+0000 7fddf2fb0740 20 client.16047 trim_cache size 0 max 16384 2024-04-25T10:34:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.160+0000 7fddf2fb0740 1 Processor -- start 2024-04-25T10:34:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.160+0000 7fddf2fb0740 1 -- start start 2024-04-25T10:34:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.160+0000 7fddf2fb0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be6f282f0 0x564be72f9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.160+0000 7fddf2fb0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f304d0 0x564be72f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.160+0000 7fddf2fb0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f511b0 0x564be73105d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.160+0000 7fddf2fb0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41b4480 con 0x564be6f511b0 2024-04-25T10:34:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.160+0000 7fddf2fb0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41b1d50 con 0x564be6f304d0 2024-04-25T10:34:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.160+0000 7fddf2fb0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41afba0 con 0x564be6f282f0 2024-04-25T10:34:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.160+0000 7fdde969c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f511b0 0x564be73105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.160+0000 7fdde969c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f511b0 0x564be73105d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46846/0 (socket says 172.21.15.43:46846) 2024-04-25T10:34:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.160+0000 7fdde969c640 1 -- 172.21.15.43:0/818397699 learned_addr learned my addr 172.21.15.43:0/818397699 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.160+0000 7fdde969c640 1 --2- 172.21.15.43:0/818397699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be6f282f0 0x564be72f9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.160+0000 7fdde9e9d640 1 --2- 172.21.15.43:0/818397699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f304d0 0x564be72f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.160+0000 7fdde969c640 1 -- 172.21.15.43:0/818397699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be6f282f0 msgr2=0x564be72f9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.160+0000 7fdde969c640 1 --2- 172.21.15.43:0/818397699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be6f282f0 0x564be72f9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.160+0000 7fdde969c640 1 -- 172.21.15.43:0/818397699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f304d0 msgr2=0x564be72f2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.160+0000 7fdde969c640 1 --2- 172.21.15.43:0/818397699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f304d0 0x564be72f2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.160+0000 7fdde969c640 1 -- 172.21.15.43:0/818397699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564be72f1ed0 con 0x564be6f511b0 2024-04-25T10:34:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.160+0000 7fdde9e9d640 1 --2- 172.21.15.43:0/818397699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f304d0 0x564be72f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.161+0000 7fdde969c640 1 --2- 172.21.15.43:0/818397699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f511b0 0x564be73105d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fddd4031740 tx=0x7fddd404c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a7caa5133452cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.161+0000 7fddcf7fe640 1 -- 172.21.15.43:0/818397699 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4035070 con 0x564be6f511b0 2024-04-25T10:34:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.161+0000 7fddcf7fe640 1 -- 172.21.15.43:0/818397699 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddd40035f0 con 0x564be6f511b0 2024-04-25T10:34:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.161+0000 7fddcf7fe640 1 -- 172.21.15.43:0/818397699 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4055310 con 0x564be6f511b0 2024-04-25T10:34:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.162+0000 7fddf2fb0740 1 -- 172.21.15.43:0/818397699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f511b0 msgr2=0x564be73105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.162+0000 7fddf2fb0740 1 --2- 172.21.15.43:0/818397699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f511b0 0x564be73105d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fddd4031740 tx=0x7fddd404c040 comp rx=0 tx=0).stop 2024-04-25T10:34:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.162+0000 7fddf2fb0740 1 -- 172.21.15.43:0/818397699 shutdown_connections 2024-04-25T10:34:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.162+0000 7fddf2fb0740 1 --2- 172.21.15.43:0/818397699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f304d0 0x564be72f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.162+0000 7fddf2fb0740 1 --2- 172.21.15.43:0/818397699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f511b0 0x564be73105d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.162+0000 7fddf2fb0740 1 --2- 172.21.15.43:0/818397699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be6f282f0 0x564be72f9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.162+0000 7fddf2fb0740 1 -- 172.21.15.43:0/818397699 >> 172.21.15.43:0/818397699 conn(0x564be6f45300 msgr2=0x564be72f27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.162+0000 7fddf2fb0740 1 -- 172.21.15.43:0/818397699 shutdown_connections 2024-04-25T10:34:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.162+0000 7fddf2fb0740 1 -- 172.21.15.43:0/818397699 wait complete. 2024-04-25T10:34:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.162+0000 7fddf2fb0740 1 -- start start 2024-04-25T10:34:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.162+0000 7fddf2fb0740 1 Processor -- start 2024-04-25T10:34:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.163+0000 7fddf2fb0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f282f0 0x564be7310c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.163+0000 7fddf2fb0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be6f304d0 0x564be730efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.163+0000 7fddf2fb0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f511b0 0x564be730f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.163+0000 7fddf2fb0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41b0210 con 0x564be6f282f0 2024-04-25T10:34:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.163+0000 7fddf2fb0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41b1680 con 0x564be6f511b0 2024-04-25T10:34:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.163+0000 7fddf2fb0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41b08e0 con 0x564be6f304d0 2024-04-25T10:34:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.163+0000 7fddf2fb0740 10 client.0 _ll_register_callbacks cb 0x7fdde000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.163+0000 7fdde969c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f282f0 0x564be7310c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.163+0000 7fdde969c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f282f0 0x564be7310c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46854/0 (socket says 172.21.15.43:46854) 2024-04-25T10:34:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.163+0000 7fdde969c640 1 -- 172.21.15.43:0/3227532408 learned_addr learned my addr 172.21.15.43:0/3227532408 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.163+0000 7fdde9e9d640 1 --2- 172.21.15.43:0/3227532408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be6f304d0 0x564be730efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.163+0000 7fdde969c640 1 -- 172.21.15.43:0/3227532408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be6f304d0 msgr2=0x564be730efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.163+0000 7fdde969c640 1 --2- 172.21.15.43:0/3227532408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be6f304d0 0x564be730efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.163+0000 7fdde969c640 1 -- 172.21.15.43:0/3227532408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f511b0 msgr2=0x564be730f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.163+0000 7fdde969c640 1 --2- 172.21.15.43:0/3227532408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f511b0 0x564be730f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.163+0000 7fdde969c640 1 -- 172.21.15.43:0/3227532408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564be730fce0 con 0x564be6f282f0 2024-04-25T10:34:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.163+0000 7fdde969c640 1 --2- 172.21.15.43:0/3227532408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f282f0 0x564be7310c00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fddd402a930 tx=0x7fddd4003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b143131e902ee555 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.163+0000 7fddcd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.163+0000 7fddcd7fa640 1 -- 172.21.15.43:0/3227532408 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4035040 con 0x564be6f282f0 2024-04-25T10:34:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.163+0000 7fddcd7fa640 1 -- 172.21.15.43:0/3227532408 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fddd4032ba0 con 0x564be6f282f0 2024-04-25T10:34:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.164+0000 7fddf2fb0740 1 -- 172.21.15.43:0/3227532408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddd40035f0 con 0x564be6f282f0 2024-04-25T10:34:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.164+0000 7fddcd7fa640 1 -- 172.21.15.43:0/3227532408 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fddd40041f0 con 0x564be6f282f0 2024-04-25T10:34:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.164+0000 7fddf2fb0740 10 client.16053 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.164+0000 7fddf2fb0740 1 -- 172.21.15.43:0/3227532408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564be73745d0 con 0x564be6f282f0 2024-04-25T10:34:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.164+0000 7fddcd7fa640 1 -- 172.21.15.43:0/3227532408 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddd40035f0 con 0x564be6f282f0 2024-04-25T10:34:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.164+0000 7fddcd7fa640 1 -- 172.21.15.43:0/3227532408 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fddd4031b20 con 0x564be6f282f0 2024-04-25T10:34:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.164+0000 7fddf2fb0740 10 client.16053 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.164+0000 7fddf2fb0740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.164+0000 7fddf2fb0740 20 client.16053 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.164+0000 7fddf2fb0740 1 --2- 172.21.15.43:0/3227532408 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564be6ee51c0 0x564be7376ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.164+0000 7fddf2fb0740 4 client.16053 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:34:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.164+0000 7fddf2fb0740 1 -- 172.21.15.43:0/3227532408 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564be725eee0 con 0x564be6ee51c0 2024-04-25T10:34:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.164+0000 7fdde9e9d640 1 --2- 172.21.15.43:0/3227532408 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564be6ee51c0 0x564be7376ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.165+0000 7fdde9e9d640 1 --2- 172.21.15.43:0/3227532408 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564be6ee51c0 0x564be7376ca0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19de9b16e3b5f5d6 server_cookie=dcb92cb38f6a7aee in_seq=0 out_seq=0 2024-04-25T10:34:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.165+0000 7fddcd7fa640 10 client.16053 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.179+0000 7fddcd7fa640 1 -- 172.21.15.43:0/3227532408 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564be725eee0 con 0x564be6ee51c0 2024-04-25T10:34:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.179+0000 7fddcd7fa640 10 client.16053 handle_command_reply: tid=0 2024-04-25T10:34:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.179+0000 7fddf2fb0740 1 client.16053 shutdown 2024-04-25T10:34:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.180+0000 7fddf2fb0740 1 -- 172.21.15.43:0/3227532408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f282f0 msgr2=0x564be7310c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.180+0000 7fddf2fb0740 1 --2- 172.21.15.43:0/3227532408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f282f0 0x564be7310c00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fddd402a930 tx=0x7fddd4003ed0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.180+0000 7fddf2fb0740 1 -- 172.21.15.43:0/3227532408 shutdown_connections 2024-04-25T10:34:42.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.180+0000 7fddf2fb0740 1 --2- 172.21.15.43:0/3227532408 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564be6ee51c0 0x564be7376ca0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.180+0000 7fddf2fb0740 1 --2- 172.21.15.43:0/3227532408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be6f511b0 0x564be730f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.180+0000 7fddf2fb0740 1 --2- 172.21.15.43:0/3227532408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be6f304d0 0x564be730efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.180+0000 7fddcd7fa640 0 client.16053 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:42.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.180+0000 7fddf2fb0740 1 --2- 172.21.15.43:0/3227532408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564be6f282f0 0x564be7310c00 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.180+0000 7fddf2fb0740 1 -- 172.21.15.43:0/3227532408 >> 172.21.15.43:0/3227532408 conn(0x564be6f45300 msgr2=0x564be72fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:42.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.180+0000 7fddf2fb0740 1 -- 172.21.15.43:0/3227532408 shutdown_connections 2024-04-25T10:34:42.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.180+0000 7fddf2fb0740 1 -- 172.21.15.43:0/3227532408 wait complete. 2024-04-25T10:34:42.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.180+0000 7fddf2fb0740 20 client.16053 trim_cache size 0 max 16384 2024-04-25T10:34:42.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:34:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:34:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:34:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:34:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.181+0000 7fddec101640 1 -- 172.21.15.43:0/3790868723 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fddd00afc50 msgr2=0x7fddd00b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.181+0000 7fddec101640 1 --2- 172.21.15.43:0/3790868723 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fddd00afc50 0x7fddd00b2160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0dc990 tx=0x7fdddc075e30 comp rx=0 tx=0).stop 2024-04-25T10:34:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.181+0000 7fddec101640 1 -- 172.21.15.43:0/3790868723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdde4110410 msgr2=0x7fdde419c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.181+0000 7fddec101640 1 --2- 172.21.15.43:0/3790868723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdde4110410 0x7fdde419c800 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00eaa0 tx=0x7fdddc00ef60 comp rx=0 tx=0).stop 2024-04-25T10:34:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.181+0000 7fddec101640 1 -- 172.21.15.43:0/3790868723 shutdown_connections 2024-04-25T10:34:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.181+0000 7fddec101640 1 --2- 172.21.15.43:0/3790868723 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fddd00afc50 0x7fddd00b2160 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.181+0000 7fddec101640 1 --2- 172.21.15.43:0/3790868723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdde410dab0 0x7fdde41a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.181+0000 7fddec101640 1 --2- 172.21.15.43:0/3790868723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdde4110410 0x7fdde419c800 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.181+0000 7fddec101640 1 --2- 172.21.15.43:0/3790868723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdde4102870 0x7fdde41a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.181+0000 7fddec101640 1 -- 172.21.15.43:0/3790868723 >> 172.21.15.43:0/3790868723 conn(0x7fdde40fe260 msgr2=0x7fdde410f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.181+0000 7fddec101640 1 -- 172.21.15.43:0/3790868723 shutdown_connections 2024-04-25T10:34:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:42.181+0000 7fddec101640 1 -- 172.21.15.43:0/3790868723 wait complete. 2024-04-25T10:34:42.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:42 smithi043 ceph-mon[29478]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-25T10:34:42.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:42 smithi077 ceph-mon[32772]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-25T10:34:42.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:42 smithi164 ceph-mon[32713]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-25T10:34:43.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:43 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:34:43.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:43 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:34:43.540 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:43 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:34:44.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:44 smithi043 ceph-mon[29478]: pgmap v69: 97 pgs: 97 active+clean; 1.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-25T10:34:44.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:44 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:34:44.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:44 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:34:44.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:44 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:34:44.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:44 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:34:44.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:44 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:34:44.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:44 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:34:44.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:44 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:34:44.441 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:44 smithi077 ceph-mon[32772]: pgmap v69: 97 pgs: 97 active+clean; 1.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-25T10:34:44.441 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:44 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:34:44.441 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:44 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:34:44.441 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:44 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:34:44.441 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:44 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:34:44.441 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:44 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:34:44.441 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:44 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:34:44.441 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:44 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:34:44.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:44 smithi164 ceph-mon[32713]: pgmap v69: 97 pgs: 97 active+clean; 1.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-25T10:34:44.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:44 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:34:44.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:44 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:34:44.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:44 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:34:44.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:44 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:34:44.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:44 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:34:44.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:44 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:34:44.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:44 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:34:46.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:46 smithi043 ceph-mon[29478]: pgmap v70: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 19 op/s 2024-04-25T10:34:46.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:46 smithi077 ceph-mon[32772]: pgmap v70: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 19 op/s 2024-04-25T10:34:46.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:46 smithi164 ceph-mon[32713]: pgmap v70: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 19 op/s 2024-04-25T10:34:47.562 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:34:47.563 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:34:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.691+0000 7f3aa90e2640 1 -- 172.21.15.43:0/2406840193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa4102b00 msgr2=0x7f3aa4102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.691+0000 7f3aa90e2640 1 --2- 172.21.15.43:0/2406840193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa4102b00 0x7f3aa4102f10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009900 tx=0x7f3a9802a410 comp rx=0 tx=0).stop 2024-04-25T10:34:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.691+0000 7f3aa90e2640 1 -- 172.21.15.43:0/2406840193 shutdown_connections 2024-04-25T10:34:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.691+0000 7f3aa90e2640 1 --2- 172.21.15.43:0/2406840193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3aa410d390 0x7f3aa410f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.691+0000 7f3aa90e2640 1 --2- 172.21.15.43:0/2406840193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa4102b00 0x7f3aa4102f10 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.691+0000 7f3aa90e2640 1 --2- 172.21.15.43:0/2406840193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3aa4108a30 0x7f3aa41025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.691+0000 7f3aa90e2640 1 -- 172.21.15.43:0/2406840193 >> 172.21.15.43:0/2406840193 conn(0x7f3aa40fe260 msgr2=0x7f3aa4100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.691+0000 7f3aa90e2640 1 -- 172.21.15.43:0/2406840193 shutdown_connections 2024-04-25T10:34:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.691+0000 7f3aa90e2640 1 -- 172.21.15.43:0/2406840193 wait complete. 2024-04-25T10:34:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.692+0000 7f3aa90e2640 1 Processor -- start 2024-04-25T10:34:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.692+0000 7f3aa90e2640 1 -- start start 2024-04-25T10:34:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.692+0000 7f3aa90e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3aa4102b00 0x7f3aa419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.692+0000 7f3aa90e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa4108a30 0x7f3aa419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.692+0000 7f3aa90e2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3aa410d390 0x7f3aa419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.692+0000 7f3aa90e2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa4112bf0 con 0x7f3aa4108a30 2024-04-25T10:34:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.692+0000 7f3aa90e2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa4112a70 con 0x7f3aa4102b00 2024-04-25T10:34:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.692+0000 7f3aa90e2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa4112d70 con 0x7f3aa410d390 2024-04-25T10:34:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.692+0000 7f3aa259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa4108a30 0x7f3aa419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.692+0000 7f3aa259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa4108a30 0x7f3aa419aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46880/0 (socket says 172.21.15.43:46880) 2024-04-25T10:34:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.692+0000 7f3aa259c640 1 -- 172.21.15.43:0/4120764445 learned_addr learned my addr 172.21.15.43:0/4120764445 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.692+0000 7f3aa359e640 1 --2- 172.21.15.43:0/4120764445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3aa410d390 0x7f3aa419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.692+0000 7f3aa2d9d640 1 --2- 172.21.15.43:0/4120764445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3aa4102b00 0x7f3aa419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.693+0000 7f3aa259c640 1 -- 172.21.15.43:0/4120764445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3aa410d390 msgr2=0x7f3aa419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.693+0000 7f3aa259c640 1 --2- 172.21.15.43:0/4120764445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3aa410d390 0x7f3aa419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.693+0000 7f3aa259c640 1 -- 172.21.15.43:0/4120764445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3aa4102b00 msgr2=0x7f3aa419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.693+0000 7f3aa259c640 1 --2- 172.21.15.43:0/4120764445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3aa4102b00 0x7f3aa419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.693+0000 7f3aa259c640 1 -- 172.21.15.43:0/4120764445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa419f5a0 con 0x7f3aa4108a30 2024-04-25T10:34:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.693+0000 7f3aa259c640 1 --2- 172.21.15.43:0/4120764445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa4108a30 0x7f3aa419aca0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3a9802a930 tx=0x7f3a98004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.693+0000 7f3a8bfff640 1 -- 172.21.15.43:0/4120764445 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9803f030 con 0x7f3aa4108a30 2024-04-25T10:34:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.693+0000 7f3aa90e2640 1 -- 172.21.15.43:0/4120764445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa419f830 con 0x7f3aa4108a30 2024-04-25T10:34:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.693+0000 7f3a8bfff640 1 -- 172.21.15.43:0/4120764445 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a98003aa0 con 0x7f3aa4108a30 2024-04-25T10:34:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.693+0000 7f3a8bfff640 1 -- 172.21.15.43:0/4120764445 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98035070 con 0x7f3aa4108a30 2024-04-25T10:34:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.693+0000 7f3aa90e2640 1 -- 172.21.15.43:0/4120764445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa419fd20 con 0x7f3aa4108a30 2024-04-25T10:34:47.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.696+0000 7f3aaff8c740 1 Processor -- start 2024-04-25T10:34:47.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.696+0000 7f3a8bfff640 1 -- 172.21.15.43:0/4120764445 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f3a98042030 con 0x7f3aa4108a30 2024-04-25T10:34:47.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.696+0000 7f3aaff8c740 1 -- start start 2024-04-25T10:34:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.696+0000 7f3aaff8c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4911d9390 0x55b491579a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.696+0000 7f3aaff8c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4911b04d0 0x55b491579fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.696+0000 7f3aaff8c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 0x55b4915868f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.696+0000 7f3aaff8c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b4914dfd80 con 0x55b4911b86b0 2024-04-25T10:34:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.696+0000 7f3aaff8c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b4914998b0 con 0x55b4911d9390 2024-04-25T10:34:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.696+0000 7f3aaff8c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b4914dbd40 con 0x55b4911b04d0 2024-04-25T10:34:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.696+0000 7f3aa259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 0x55b4915868f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.696+0000 7f3aa259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 0x55b4915868f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46888/0 (socket says 172.21.15.43:46888) 2024-04-25T10:34:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.696+0000 7f3aa259c640 1 -- 172.21.15.43:0/2669953349 learned_addr learned my addr 172.21.15.43:0/2669953349 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.696+0000 7f3aa359e640 1 --2- 172.21.15.43:0/2669953349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4911d9390 0x55b491579a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.696+0000 7f3aa2d9d640 1 --2- 172.21.15.43:0/2669953349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4911b04d0 0x55b491579fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.697+0000 7f3a8bfff640 1 --2- 172.21.15.43:0/4120764445 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3a8c0afc60 0x7f3a8c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.697+0000 7f3aa259c640 1 -- 172.21.15.43:0/2669953349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4911b04d0 msgr2=0x55b491579fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.697+0000 7f3aa259c640 1 --2- 172.21.15.43:0/2669953349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4911b04d0 0x55b491579fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.697+0000 7f3aa259c640 1 -- 172.21.15.43:0/2669953349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4911d9390 msgr2=0x55b491579a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.697+0000 7f3aa259c640 1 --2- 172.21.15.43:0/2669953349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4911d9390 0x55b491579a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.697+0000 7f3aa259c640 1 -- 172.21.15.43:0/2669953349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b49146aa40 con 0x55b4911b86b0 2024-04-25T10:34:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.697+0000 7f3aa359e640 1 --2- 172.21.15.43:0/4120764445 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3a8c0afc60 0x7f3a8c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.697+0000 7f3a8bfff640 1 -- 172.21.15.43:0/4120764445 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f3a980ff490 con 0x7f3aa4108a30 2024-04-25T10:34:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.697+0000 7f3aa259c640 1 --2- 172.21.15.43:0/2669953349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 0x55b4915868f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3a980fcb80 tx=0x7f3a9808dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7599d093b5eda0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.697+0000 7f3a897fa640 1 -- 172.21.15.43:0/2669953349 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a980c74e0 con 0x55b4911b86b0 2024-04-25T10:34:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.697+0000 7f3a897fa640 1 -- 172.21.15.43:0/2669953349 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a980c7220 con 0x55b4911b86b0 2024-04-25T10:34:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.697+0000 7f3a897fa640 1 -- 172.21.15.43:0/2669953349 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a980c6e80 con 0x55b4911b86b0 2024-04-25T10:34:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.698+0000 7f3aaff8c740 1 -- 172.21.15.43:0/2669953349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 msgr2=0x55b4915868f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.698+0000 7f3aaff8c740 1 --2- 172.21.15.43:0/2669953349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 0x55b4915868f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3a980fcb80 tx=0x7f3a9808dbe0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.698+0000 7f3aa359e640 1 --2- 172.21.15.43:0/4120764445 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3a8c0afc60 0x7f3a8c0b2170 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55b491576bd0 tx=0x7f3a94004a40 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.698+0000 7f3aaff8c740 1 -- 172.21.15.43:0/2669953349 shutdown_connections 2024-04-25T10:34:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.698+0000 7f3aaff8c740 1 --2- 172.21.15.43:0/2669953349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 0x55b4915868f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.698+0000 7f3aaff8c740 1 --2- 172.21.15.43:0/2669953349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4911b04d0 0x55b491579fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.698+0000 7f3aaff8c740 1 --2- 172.21.15.43:0/2669953349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4911d9390 0x55b491579a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.698+0000 7f3aaff8c740 1 -- 172.21.15.43:0/2669953349 >> 172.21.15.43:0/2669953349 conn(0x55b4911cd4e0 msgr2=0x55b491576210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.698+0000 7f3aaff8c740 1 -- 172.21.15.43:0/2669953349 shutdown_connections 2024-04-25T10:34:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.698+0000 7f3aaff8c740 1 -- 172.21.15.43:0/2669953349 wait complete. 2024-04-25T10:34:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.699+0000 7f3aaff8c740 1 -- start start 2024-04-25T10:34:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.699+0000 7f3aaff8c740 1 Processor -- start 2024-04-25T10:34:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.699+0000 7f3aaff8c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4911b04d0 0x55b4915845e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.699+0000 7f3aaff8c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 0x55b491584b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.699+0000 7f3aaff8c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4911d9390 0x55b49158a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.699+0000 7f3aaff8c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b491586e30 con 0x55b4911b86b0 2024-04-25T10:34:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.699+0000 7f3aaff8c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b491466d10 con 0x55b4911b04d0 2024-04-25T10:34:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.699+0000 7f3aaff8c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b491586fb0 con 0x55b4911d9390 2024-04-25T10:34:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.699+0000 7f3aaff8c740 10 client.0 _ll_register_callbacks cb 0x55b491493210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.699+0000 7f3aa259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 0x55b491584b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.699+0000 7f3aa259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 0x55b491584b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46902/0 (socket says 172.21.15.43:46902) 2024-04-25T10:34:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.699+0000 7f3aa259c640 1 -- 172.21.15.43:0/12116863 learned_addr learned my addr 172.21.15.43:0/12116863 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.699+0000 7f3aa359e640 1 --2- 172.21.15.43:0/12116863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4911d9390 0x55b49158a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.699+0000 7f3aa2d9d640 1 --2- 172.21.15.43:0/12116863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4911b04d0 0x55b4915845e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.699+0000 7f3aa259c640 1 -- 172.21.15.43:0/12116863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4911d9390 msgr2=0x55b49158a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.699+0000 7f3aa259c640 1 --2- 172.21.15.43:0/12116863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4911d9390 0x55b49158a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.699+0000 7f3aa259c640 1 -- 172.21.15.43:0/12116863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4911b04d0 msgr2=0x55b4915845e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.699+0000 7f3aa259c640 1 --2- 172.21.15.43:0/12116863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4911b04d0 0x55b4915845e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.699+0000 7f3aa259c640 1 -- 172.21.15.43:0/12116863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4912bb7d0 con 0x55b4911b86b0 2024-04-25T10:34:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.699+0000 7f3aa2d9d640 1 --2- 172.21.15.43:0/12116863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4911b04d0 0x55b4915845e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.700+0000 7f3aa259c640 1 --2- 172.21.15.43:0/12116863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 0x55b491584b20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3a980fccc0 tx=0x7f3a98031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f82da7711a9f79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.700+0000 7f3a86ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.700+0000 7f3a86ff5640 1 -- 172.21.15.43:0/12116863 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98049030 con 0x55b4911b86b0 2024-04-25T10:34:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.700+0000 7f3a86ff5640 1 -- 172.21.15.43:0/12116863 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f3a980fce90 con 0x55b4911b86b0 2024-04-25T10:34:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.700+0000 7f3aaff8c740 1 -- 172.21.15.43:0/12116863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a980c7220 con 0x55b4911b86b0 2024-04-25T10:34:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.700+0000 7f3a86ff5640 1 -- 172.21.15.43:0/12116863 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3a980c7060 con 0x55b4911b86b0 2024-04-25T10:34:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.700+0000 7f3aaff8c740 10 client.16065 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.700+0000 7f3aaff8c740 1 -- 172.21.15.43:0/12116863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b4911f13f0 con 0x55b4911b86b0 2024-04-25T10:34:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.700+0000 7f3a86ff5640 1 -- 172.21.15.43:0/12116863 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a980c7220 con 0x55b4911b86b0 2024-04-25T10:34:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.700+0000 7f3a86ff5640 1 -- 172.21.15.43:0/12116863 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3a980c39a0 con 0x55b4911b86b0 2024-04-25T10:34:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.700+0000 7f3aaff8c740 10 client.16065 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:47.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.700+0000 7f3aaff8c740 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:47.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.700+0000 7f3aaff8c740 20 client.16065 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:47.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.700+0000 7f3aaff8c740 1 --2- 172.21.15.43:0/12116863 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b491197f60 0x55b491596340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.700+0000 7f3aaff8c740 4 client.16065 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:34:47.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.700+0000 7f3aaff8c740 1 -- 172.21.15.43:0/12116863 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4914ed990 con 0x55b491197f60 2024-04-25T10:34:47.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.701+0000 7f3aa2d9d640 1 --2- 172.21.15.43:0/12116863 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b491197f60 0x55b491596340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:47.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.701+0000 7f3a86ff5640 10 client.16065 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:47.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.701+0000 7f3aa2d9d640 1 --2- 172.21.15.43:0/12116863 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b491197f60 0x55b491596340 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14e5139dc341dcc2 server_cookie=4209d1279cdef641 in_seq=0 out_seq=0 2024-04-25T10:34:47.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.702+0000 7f3a86ff5640 1 -- 172.21.15.43:0/12116863 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b4914ed990 con 0x55b491197f60 2024-04-25T10:34:47.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.702+0000 7f3a86ff5640 10 client.16065 handle_command_reply: tid=0 2024-04-25T10:34:47.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.702+0000 7f3aaff8c740 1 client.16065 shutdown 2024-04-25T10:34:47.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.702+0000 7f3aaff8c740 1 -- 172.21.15.43:0/12116863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 msgr2=0x55b491584b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:47.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.702+0000 7f3aaff8c740 1 --2- 172.21.15.43:0/12116863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 0x55b491584b20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3a980fccc0 tx=0x7f3a98031a10 comp rx=0 tx=0).stop 2024-04-25T10:34:47.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.702+0000 7f3aaff8c740 1 -- 172.21.15.43:0/12116863 shutdown_connections 2024-04-25T10:34:47.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.702+0000 7f3aaff8c740 1 --2- 172.21.15.43:0/12116863 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b491197f60 0x55b491596340 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.702+0000 7f3aaff8c740 1 --2- 172.21.15.43:0/12116863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4911d9390 0x55b49158a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.702+0000 7f3a86ff5640 0 client.16065 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:47.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.702+0000 7f3aaff8c740 1 --2- 172.21.15.43:0/12116863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 0x55b491584b20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.702+0000 7f3aaff8c740 1 --2- 172.21.15.43:0/12116863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4911b04d0 0x55b4915845e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.702+0000 7f3aaff8c740 1 -- 172.21.15.43:0/12116863 >> 172.21.15.43:0/12116863 conn(0x55b4911cd4e0 msgr2=0x55b49157b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:47.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.702+0000 7f3aaff8c740 1 -- 172.21.15.43:0/12116863 shutdown_connections 2024-04-25T10:34:47.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.702+0000 7f3aaff8c740 1 -- 172.21.15.43:0/12116863 wait complete. 2024-04-25T10:34:47.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.703+0000 7f3aaff8c740 20 client.16065 trim_cache size 0 max 16384 2024-04-25T10:34:47.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.721+0000 7f3aaff8c740 1 Processor -- start 2024-04-25T10:34:47.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.721+0000 7f3aaff8c740 1 -- start start 2024-04-25T10:34:47.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.721+0000 7f3aaff8c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b04d0 0x55b491579e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.721+0000 7f3aaff8c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4911b86b0 0x55b49158a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.721+0000 7f3aaff8c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4911d9390 0x55b49158a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.721+0000 7f3aaff8c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa41b2110 con 0x55b4911b04d0 2024-04-25T10:34:47.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.721+0000 7f3aaff8c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa41af9e0 con 0x55b4911b86b0 2024-04-25T10:34:47.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.721+0000 7f3aaff8c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa41ad830 con 0x55b4911d9390 2024-04-25T10:34:47.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.721+0000 7f3aa2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b04d0 0x55b491579e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:47.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.721+0000 7f3aa2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b04d0 0x55b491579e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46918/0 (socket says 172.21.15.43:46918) 2024-04-25T10:34:47.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.721+0000 7f3aa2d9d640 1 -- 172.21.15.43:0/2991619674 learned_addr learned my addr 172.21.15.43:0/2991619674 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:47.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.721+0000 7f3aa2d9d640 1 -- 172.21.15.43:0/2991619674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4911d9390 msgr2=0x55b49158a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:47.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.721+0000 7f3aa2d9d640 1 --2- 172.21.15.43:0/2991619674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4911d9390 0x55b49158a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.721+0000 7f3aa2d9d640 1 -- 172.21.15.43:0/2991619674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4911b86b0 msgr2=0x55b49158a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:47.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.721+0000 7f3aa2d9d640 1 --2- 172.21.15.43:0/2991619674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4911b86b0 0x55b49158a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.721+0000 7f3aa2d9d640 1 -- 172.21.15.43:0/2991619674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4911b10f0 con 0x55b4911b04d0 2024-04-25T10:34:47.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.722+0000 7f3aa2d9d640 1 --2- 172.21.15.43:0/2991619674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b04d0 0x55b491579e60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c005d00 tx=0x7f3a9c00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f68790c0a31d91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:47.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.722+0000 7f3a88ff9640 1 -- 172.21.15.43:0/2991619674 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c004220 con 0x55b4911b04d0 2024-04-25T10:34:47.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.722+0000 7f3a88ff9640 1 -- 172.21.15.43:0/2991619674 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a9c023030 con 0x55b4911b04d0 2024-04-25T10:34:47.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.722+0000 7f3a88ff9640 1 -- 172.21.15.43:0/2991619674 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c01d840 con 0x55b4911b04d0 2024-04-25T10:34:47.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.722+0000 7f3aaff8c740 1 -- 172.21.15.43:0/2991619674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b04d0 msgr2=0x55b491579e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:47.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.722+0000 7f3aaff8c740 1 --2- 172.21.15.43:0/2991619674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b04d0 0x55b491579e60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c005d00 tx=0x7f3a9c00b410 comp rx=0 tx=0).stop 2024-04-25T10:34:47.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.723+0000 7f3aaff8c740 1 -- 172.21.15.43:0/2991619674 shutdown_connections 2024-04-25T10:34:47.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.723+0000 7f3aaff8c740 1 --2- 172.21.15.43:0/2991619674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4911b86b0 0x55b49158a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.723+0000 7f3aaff8c740 1 --2- 172.21.15.43:0/2991619674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4911d9390 0x55b49158a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.723+0000 7f3aaff8c740 1 --2- 172.21.15.43:0/2991619674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b04d0 0x55b491579e60 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.723+0000 7f3aaff8c740 1 -- 172.21.15.43:0/2991619674 >> 172.21.15.43:0/2991619674 conn(0x55b4911cd4e0 msgr2=0x55b4915749c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:47.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.723+0000 7f3aaff8c740 1 -- 172.21.15.43:0/2991619674 shutdown_connections 2024-04-25T10:34:47.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.723+0000 7f3aaff8c740 1 -- 172.21.15.43:0/2991619674 wait complete. 2024-04-25T10:34:47.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.723+0000 7f3aaff8c740 1 -- start start 2024-04-25T10:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.723+0000 7f3aaff8c740 1 Processor -- start 2024-04-25T10:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.723+0000 7f3aaff8c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4911b04d0 0x55b4915906b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.723+0000 7f3aaff8c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 0x55b49158da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.723+0000 7f3aaff8c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4911d9390 0x55b49158df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.723+0000 7f3aaff8c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa41adea0 con 0x55b4911b86b0 2024-04-25T10:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.723+0000 7f3aaff8c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa41af310 con 0x55b4911d9390 2024-04-25T10:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.723+0000 7f3aaff8c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa41ae570 con 0x55b4911b04d0 2024-04-25T10:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.723+0000 7f3aaff8c740 10 client.0 _ll_register_callbacks cb 0x7f3a94009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.724+0000 7f3aa259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 0x55b49158da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.724+0000 7f3aa259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 0x55b49158da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46930/0 (socket says 172.21.15.43:46930) 2024-04-25T10:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.724+0000 7f3aa259c640 1 -- 172.21.15.43:0/2938902274 learned_addr learned my addr 172.21.15.43:0/2938902274 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.724+0000 7f3aa2d9d640 1 --2- 172.21.15.43:0/2938902274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4911b04d0 0x55b4915906b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.724+0000 7f3aa359e640 1 --2- 172.21.15.43:0/2938902274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4911d9390 0x55b49158df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.724+0000 7f3aa259c640 1 -- 172.21.15.43:0/2938902274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4911b04d0 msgr2=0x55b4915906b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.724+0000 7f3aa259c640 1 --2- 172.21.15.43:0/2938902274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4911b04d0 0x55b4915906b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.724+0000 7f3aa259c640 1 -- 172.21.15.43:0/2938902274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4911d9390 msgr2=0x55b49158df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.724+0000 7f3aa259c640 1 --2- 172.21.15.43:0/2938902274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4911d9390 0x55b49158df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.724+0000 7f3aa259c640 1 -- 172.21.15.43:0/2938902274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4914a04b0 con 0x55b4911b86b0 2024-04-25T10:34:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.724+0000 7f3aa2d9d640 1 --2- 172.21.15.43:0/2938902274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4911b04d0 0x55b4915906b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.724+0000 7f3aa259c640 1 --2- 172.21.15.43:0/2938902274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 0x55b49158da20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3a980fcc70 tx=0x7f3a9803b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27ee23de67698bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.724+0000 7f3a86ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.724+0000 7f3a86ff5640 1 -- 172.21.15.43:0/2938902274 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98049020 con 0x55b4911b86b0 2024-04-25T10:34:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.724+0000 7f3a86ff5640 1 -- 172.21.15.43:0/2938902274 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f3a980c7970 con 0x55b4911b86b0 2024-04-25T10:34:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.724+0000 7f3aaff8c740 1 -- 172.21.15.43:0/2938902274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a9c023030 con 0x55b4911b86b0 2024-04-25T10:34:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.725+0000 7f3a86ff5640 1 -- 172.21.15.43:0/2938902274 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3a980c74e0 con 0x55b4911b86b0 2024-04-25T10:34:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.725+0000 7f3aaff8c740 10 client.16071 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.725+0000 7f3aaff8c740 1 -- 172.21.15.43:0/2938902274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b4914a04b0 con 0x55b4911b86b0 2024-04-25T10:34:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.725+0000 7f3a86ff5640 1 -- 172.21.15.43:0/2938902274 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a9c023030 con 0x55b4911b86b0 2024-04-25T10:34:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.725+0000 7f3a86ff5640 1 -- 172.21.15.43:0/2938902274 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3a980d0030 con 0x55b4911b86b0 2024-04-25T10:34:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.725+0000 7f3aaff8c740 10 client.16071 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.725+0000 7f3aaff8c740 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.725+0000 7f3aaff8c740 20 client.16071 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.725+0000 7f3aaff8c740 1 --2- 172.21.15.43:0/2938902274 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b49116d3a0 0x55b4916003d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.725+0000 7f3aaff8c740 4 client.16071 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:34:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.725+0000 7f3aaff8c740 1 -- 172.21.15.43:0/2938902274 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b4914e7800 con 0x55b49116d3a0 2024-04-25T10:34:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.725+0000 7f3aa2d9d640 1 --2- 172.21.15.43:0/2938902274 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b49116d3a0 0x55b4916003d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.726+0000 7f3a86ff5640 10 client.16071 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.726+0000 7f3aa2d9d640 1 --2- 172.21.15.43:0/2938902274 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b49116d3a0 0x55b4916003d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab9ad731d6e387ab server_cookie=996ace3e806c027a in_seq=0 out_seq=0 2024-04-25T10:34:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.726+0000 7f3a86ff5640 1 -- 172.21.15.43:0/2938902274 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b4914e7800 con 0x55b49116d3a0 2024-04-25T10:34:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.726+0000 7f3a86ff5640 10 client.16071 handle_command_reply: tid=0 2024-04-25T10:34:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.726+0000 7f3aaff8c740 1 client.16071 shutdown 2024-04-25T10:34:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.726+0000 7f3aaff8c740 1 -- 172.21.15.43:0/2938902274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 msgr2=0x55b49158da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.726+0000 7f3aaff8c740 1 --2- 172.21.15.43:0/2938902274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 0x55b49158da20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3a980fcc70 tx=0x7f3a9803b6b0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.726+0000 7f3aaff8c740 1 -- 172.21.15.43:0/2938902274 shutdown_connections 2024-04-25T10:34:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.726+0000 7f3aaff8c740 1 --2- 172.21.15.43:0/2938902274 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b49116d3a0 0x55b4916003d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.726+0000 7f3aaff8c740 1 --2- 172.21.15.43:0/2938902274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4911d9390 0x55b49158df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.726+0000 7f3aaff8c740 1 --2- 172.21.15.43:0/2938902274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4911b86b0 0x55b49158da20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.726+0000 7f3aaff8c740 1 --2- 172.21.15.43:0/2938902274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4911b04d0 0x55b4915906b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.726+0000 7f3a86ff5640 0 client.16071 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.726+0000 7f3aaff8c740 1 -- 172.21.15.43:0/2938902274 >> 172.21.15.43:0/2938902274 conn(0x55b4911cd4e0 msgr2=0x55b4915753f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.726+0000 7f3aaff8c740 1 -- 172.21.15.43:0/2938902274 shutdown_connections 2024-04-25T10:34:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.726+0000 7f3aaff8c740 1 -- 172.21.15.43:0/2938902274 wait complete. 2024-04-25T10:34:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.726+0000 7f3aaff8c740 20 client.16071 trim_cache size 0 max 16384 2024-04-25T10:34:47.735 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:34:47.735 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:34:47.735 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:34:47.735 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:34:47.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.727+0000 7f3aa90e2640 1 -- 172.21.15.43:0/4120764445 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3a8c0afc60 msgr2=0x7f3a8c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:47.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.727+0000 7f3aa90e2640 1 --2- 172.21.15.43:0/4120764445 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3a8c0afc60 0x7f3a8c0b2170 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55b491576bd0 tx=0x7f3a94004a40 comp rx=0 tx=0).stop 2024-04-25T10:34:47.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.727+0000 7f3aa90e2640 1 -- 172.21.15.43:0/4120764445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa4108a30 msgr2=0x7f3aa419aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:47.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.727+0000 7f3aa90e2640 1 --2- 172.21.15.43:0/4120764445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa4108a30 0x7f3aa419aca0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3a9802a930 tx=0x7f3a98004270 comp rx=0 tx=0).stop 2024-04-25T10:34:47.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.727+0000 7f3aa90e2640 1 -- 172.21.15.43:0/4120764445 shutdown_connections 2024-04-25T10:34:47.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.727+0000 7f3aa90e2640 1 --2- 172.21.15.43:0/4120764445 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3a8c0afc60 0x7f3a8c0b2170 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.727+0000 7f3aa90e2640 1 --2- 172.21.15.43:0/4120764445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3aa410d390 0x7f3aa419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.727+0000 7f3aa90e2640 1 --2- 172.21.15.43:0/4120764445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa4108a30 0x7f3aa419aca0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.727+0000 7f3aa90e2640 1 --2- 172.21.15.43:0/4120764445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3aa4102b00 0x7f3aa419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.727+0000 7f3aa90e2640 1 -- 172.21.15.43:0/4120764445 >> 172.21.15.43:0/4120764445 conn(0x7f3aa40fe260 msgr2=0x7f3aa4100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:47.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.727+0000 7f3aa90e2640 1 -- 172.21.15.43:0/4120764445 shutdown_connections 2024-04-25T10:34:47.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.727+0000 7f3aa90e2640 1 -- 172.21.15.43:0/4120764445 wait complete. 2024-04-25T10:34:47.740 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:34:47.741 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:34:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.850+0000 7fd2e9692640 1 -- 172.21.15.43:0/3450882317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e41090b0 msgr2=0x7fd2e4103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.850+0000 7fd2e9692640 1 --2- 172.21.15.43:0/3450882317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e41090b0 0x7fd2e4103220 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009900 tx=0x7fd2d802a410 comp rx=0 tx=0).stop 2024-04-25T10:34:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.850+0000 7fd2e9692640 1 -- 172.21.15.43:0/3450882317 shutdown_connections 2024-04-25T10:34:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.850+0000 7fd2e9692640 1 --2- 172.21.15.43:0/3450882317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2e4103890 0x7fd2e410f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.850+0000 7fd2e9692640 1 --2- 172.21.15.43:0/3450882317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e41090b0 0x7fd2e4103220 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.850+0000 7fd2e9692640 1 --2- 172.21.15.43:0/3450882317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2e4108c10 0x7fd2e4102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.850+0000 7fd2e9692640 1 -- 172.21.15.43:0/3450882317 >> 172.21.15.43:0/3450882317 conn(0x7fd2e40fd840 msgr2=0x7fd2e40ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:47.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.850+0000 7fd2e9692640 1 -- 172.21.15.43:0/3450882317 shutdown_connections 2024-04-25T10:34:47.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.850+0000 7fd2e9692640 1 -- 172.21.15.43:0/3450882317 wait complete. 2024-04-25T10:34:47.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.850+0000 7fd2e9692640 1 Processor -- start 2024-04-25T10:34:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.851+0000 7fd2e9692640 1 -- start start 2024-04-25T10:34:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.851+0000 7fd2e9692640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2e4103890 0x7fd2e41a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.851+0000 7fd2e9692640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e4108c10 0x7fd2e41a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.851+0000 7fd2e9692640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2e41090b0 0x7fd2e419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.851+0000 7fd2e9692640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4112e30 con 0x7fd2e4108c10 2024-04-25T10:34:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.851+0000 7fd2e9692640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4112cb0 con 0x7fd2e4103890 2024-04-25T10:34:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.851+0000 7fd2e9692640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4112fb0 con 0x7fd2e41090b0 2024-04-25T10:34:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.851+0000 7fd2e27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e4108c10 0x7fd2e41a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.851+0000 7fd2e27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e4108c10 0x7fd2e41a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46948/0 (socket says 172.21.15.43:46948) 2024-04-25T10:34:47.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.851+0000 7fd2e27fc640 1 -- 172.21.15.43:0/1941846271 learned_addr learned my addr 172.21.15.43:0/1941846271 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:47.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.851+0000 7fd2e2ffd640 1 --2- 172.21.15.43:0/1941846271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2e4103890 0x7fd2e41a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:47.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.851+0000 7fd2e27fc640 1 -- 172.21.15.43:0/1941846271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2e41090b0 msgr2=0x7fd2e419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:47.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.851+0000 7fd2e27fc640 1 --2- 172.21.15.43:0/1941846271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2e41090b0 0x7fd2e419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.851+0000 7fd2e27fc640 1 -- 172.21.15.43:0/1941846271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2e4103890 msgr2=0x7fd2e41a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:47.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.851+0000 7fd2e27fc640 1 --2- 172.21.15.43:0/1941846271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2e4103890 0x7fd2e41a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:47.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.851+0000 7fd2e27fc640 1 -- 172.21.15.43:0/1941846271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e419ae70 con 0x7fd2e4108c10 2024-04-25T10:34:47.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.851+0000 7fd2e2ffd640 1 --2- 172.21.15.43:0/1941846271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2e4103890 0x7fd2e41a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:47.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.852+0000 7fd2e27fc640 1 --2- 172.21.15.43:0/1941846271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e4108c10 0x7fd2e41a0b00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd2d802a930 tx=0x7fd2d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:47.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.852+0000 7fd2cbfff640 1 -- 172.21.15.43:0/1941846271 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d803f030 con 0x7fd2e4108c10 2024-04-25T10:34:47.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.852+0000 7fd2e9692640 1 -- 172.21.15.43:0/1941846271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e419b0a0 con 0x7fd2e4108c10 2024-04-25T10:34:47.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.852+0000 7fd2cbfff640 1 -- 172.21.15.43:0/1941846271 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2d8003aa0 con 0x7fd2e4108c10 2024-04-25T10:34:47.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.852+0000 7fd2cbfff640 1 -- 172.21.15.43:0/1941846271 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8035070 con 0x7fd2e4108c10 2024-04-25T10:34:47.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.852+0000 7fd2e9692640 1 -- 172.21.15.43:0/1941846271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e41a7360 con 0x7fd2e4108c10 2024-04-25T10:34:47.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.853+0000 7fd2e9692640 1 -- 172.21.15.43:0/1941846271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e4103f90 con 0x7fd2e4108c10 2024-04-25T10:34:47.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.854+0000 7fd2cbfff640 1 -- 172.21.15.43:0/1941846271 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fd2d8042030 con 0x7fd2e4108c10 2024-04-25T10:34:47.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.855+0000 7fd2cbfff640 1 --2- 172.21.15.43:0/1941846271 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd2cc0afc60 0x7fd2cc0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:47.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.855+0000 7fd2e2ffd640 1 --2- 172.21.15.43:0/1941846271 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd2cc0afc60 0x7fd2cc0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:47.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.855+0000 7fd2cbfff640 1 -- 172.21.15.43:0/1941846271 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fd2d80ff5c0 con 0x7fd2e4108c10 2024-04-25T10:34:47.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.855+0000 7fd2e2ffd640 1 --2- 172.21.15.43:0/1941846271 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd2cc0afc60 0x7fd2cc0b2170 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4005d00 tx=0x7fd2d4009550 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:47.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:47.860+0000 7fd2cbfff640 1 -- 172.21.15.43:0/1941846271 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd2d80c7e90 con 0x7fd2e4108c10 2024-04-25T10:34:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.085+0000 7fd2e9692640 1 -- 172.21.15.43:0/1941846271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2e406a190 con 0x7fd2e4108c10 2024-04-25T10:34:48.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.086+0000 7fd2cbfff640 1 -- 172.21.15.43:0/1941846271 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7fd2d80c7790 con 0x7fd2e4108c10 2024-04-25T10:34:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:34:48.088 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:34:48.088 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:34:48.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.088+0000 7fd2e9692640 1 -- 172.21.15.43:0/1941846271 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd2cc0afc60 msgr2=0x7fd2cc0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.088+0000 7fd2e9692640 1 --2- 172.21.15.43:0/1941846271 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd2cc0afc60 0x7fd2cc0b2170 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4005d00 tx=0x7fd2d4009550 comp rx=0 tx=0).stop 2024-04-25T10:34:48.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.088+0000 7fd2e9692640 1 -- 172.21.15.43:0/1941846271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e4108c10 msgr2=0x7fd2e41a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.088+0000 7fd2e9692640 1 --2- 172.21.15.43:0/1941846271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e4108c10 0x7fd2e41a0b00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd2d802a930 tx=0x7fd2d8004270 comp rx=0 tx=0).stop 2024-04-25T10:34:48.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.088+0000 7fd2e9692640 1 -- 172.21.15.43:0/1941846271 shutdown_connections 2024-04-25T10:34:48.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.088+0000 7fd2e9692640 1 --2- 172.21.15.43:0/1941846271 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd2cc0afc60 0x7fd2cc0b2170 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.088+0000 7fd2e9692640 1 --2- 172.21.15.43:0/1941846271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2e41090b0 0x7fd2e419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.088+0000 7fd2e9692640 1 --2- 172.21.15.43:0/1941846271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e4108c10 0x7fd2e41a0b00 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.088+0000 7fd2e9692640 1 --2- 172.21.15.43:0/1941846271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2e4103890 0x7fd2e41a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.088+0000 7fd2e9692640 1 -- 172.21.15.43:0/1941846271 >> 172.21.15.43:0/1941846271 conn(0x7fd2e40fd840 msgr2=0x7fd2e40fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:48.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.088+0000 7fd2e9692640 1 -- 172.21.15.43:0/1941846271 shutdown_connections 2024-04-25T10:34:48.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.088+0000 7fd2e9692640 1 -- 172.21.15.43:0/1941846271 wait complete. 2024-04-25T10:34:48.102 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:34:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.214+0000 7f17363b7640 1 -- 172.21.15.43:0/428862357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1730109fd0 msgr2=0x7f1730103c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.214+0000 7f17363b7640 1 --2- 172.21.15.43:0/428862357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1730109fd0 0x7f1730103c60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1724009900 tx=0x7f172402a410 comp rx=0 tx=0).stop 2024-04-25T10:34:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.214+0000 7f17363b7640 1 -- 172.21.15.43:0/428862357 shutdown_connections 2024-04-25T10:34:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.214+0000 7f17363b7640 1 --2- 172.21.15.43:0/428862357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17301042d0 0x7f1730110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.214+0000 7f17363b7640 1 --2- 172.21.15.43:0/428862357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1730109fd0 0x7f1730103c60 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.214+0000 7f17363b7640 1 --2- 172.21.15.43:0/428862357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1730109b30 0x7f1730103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.214+0000 7f17363b7640 1 -- 172.21.15.43:0/428862357 >> 172.21.15.43:0/428862357 conn(0x7f1730079280 msgr2=0x7f17300796c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.214+0000 7f17363b7640 1 -- 172.21.15.43:0/428862357 shutdown_connections 2024-04-25T10:34:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.214+0000 7f17363b7640 1 -- 172.21.15.43:0/428862357 wait complete. 2024-04-25T10:34:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.215+0000 7f17363b7640 1 Processor -- start 2024-04-25T10:34:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.215+0000 7f17363b7640 1 -- start start 2024-04-25T10:34:48.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.215+0000 7f17363b7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17301042d0 0x7f173019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.215+0000 7f17363b7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1730109b30 0x7f173019ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.215+0000 7f17363b7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1730109fd0 0x7f173019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.215+0000 7f17363b7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1730113d20 con 0x7f1730109fd0 2024-04-25T10:34:48.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.215+0000 7f17363b7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1730113ba0 con 0x7f17301042d0 2024-04-25T10:34:48.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.215+0000 7f17363b7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1730113ea0 con 0x7f1730109b30 2024-04-25T10:34:48.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.215+0000 7f1734954640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1730109fd0 0x7f173019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.215+0000 7f1734954640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1730109fd0 0x7f173019efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46960/0 (socket says 172.21.15.43:46960) 2024-04-25T10:34:48.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.215+0000 7f1734954640 1 -- 172.21.15.43:0/1846474850 learned_addr learned my addr 172.21.15.43:0/1846474850 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:48.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.215+0000 7f172f7fe640 1 --2- 172.21.15.43:0/1846474850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1730109b30 0x7f173019ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.216+0000 7f172ffff640 1 --2- 172.21.15.43:0/1846474850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17301042d0 0x7f173019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.216+0000 7f1734954640 1 -- 172.21.15.43:0/1846474850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1730109b30 msgr2=0x7f173019ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.216+0000 7f1734954640 1 --2- 172.21.15.43:0/1846474850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1730109b30 0x7f173019ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.216+0000 7f1734954640 1 -- 172.21.15.43:0/1846474850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17301042d0 msgr2=0x7f173019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.216+0000 7f1734954640 1 --2- 172.21.15.43:0/1846474850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17301042d0 0x7f173019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.216+0000 7f1734954640 1 -- 172.21.15.43:0/1846474850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173019f580 con 0x7f1730109fd0 2024-04-25T10:34:48.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.216+0000 7f172f7fe640 1 --2- 172.21.15.43:0/1846474850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1730109b30 0x7f173019ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:48.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.216+0000 7f172ffff640 1 --2- 172.21.15.43:0/1846474850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17301042d0 0x7f173019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:48.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.216+0000 7f1734954640 1 --2- 172.21.15.43:0/1846474850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1730109fd0 0x7f173019efc0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f172800db20 tx=0x7f172800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:48.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.216+0000 7f172d7fa640 1 -- 172.21.15.43:0/1846474850 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1728017070 con 0x7f1730109fd0 2024-04-25T10:34:48.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.216+0000 7f17363b7640 1 -- 172.21.15.43:0/1846474850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f173019f870 con 0x7f1730109fd0 2024-04-25T10:34:48.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.216+0000 7f172d7fa640 1 -- 172.21.15.43:0/1846474850 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1728003e20 con 0x7f1730109fd0 2024-04-25T10:34:48.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.216+0000 7f172d7fa640 1 -- 172.21.15.43:0/1846474850 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1728010460 con 0x7f1730109fd0 2024-04-25T10:34:48.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.216+0000 7f17363b7640 1 -- 172.21.15.43:0/1846474850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f173019fd70 con 0x7f1730109fd0 2024-04-25T10:34:48.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.219+0000 7f173d26b740 1 Processor -- start 2024-04-25T10:34:48.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.219+0000 7f172d7fa640 1 -- 172.21.15.43:0/1846474850 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f1728003970 con 0x7f1730109fd0 2024-04-25T10:34:48.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.219+0000 7f173d26b740 1 -- start start 2024-04-25T10:34:48.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.220+0000 7f173d26b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb97bbf390 0x55eb97f60810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.220+0000 7f173d26b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97b964d0 0x55eb97f60d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.220+0000 7f173d26b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb97b9e6b0 0x55eb97f6d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.220+0000 7f173d26b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55eb97ec5d80 con 0x55eb97b964d0 2024-04-25T10:34:48.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.220+0000 7f173d26b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55eb97e7fa20 con 0x55eb97b9e6b0 2024-04-25T10:34:48.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.220+0000 7f173d26b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55eb97ec1d40 con 0x55eb97bbf390 2024-04-25T10:34:48.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.220+0000 7f172ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97b964d0 0x55eb97f60d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.220+0000 7f172ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97b964d0 0x55eb97f60d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46962/0 (socket says 172.21.15.43:46962) 2024-04-25T10:34:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.220+0000 7f172ffff640 1 -- 172.21.15.43:0/3890537735 learned_addr learned my addr 172.21.15.43:0/3890537735 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.220+0000 7f172f7fe640 1 --2- 172.21.15.43:0/3890537735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb97bbf390 0x55eb97f60810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.220+0000 7f172d7fa640 1 --2- 172.21.15.43:0/1846474850 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f17180afc60 0x7f17180b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.220+0000 7f172f7fe640 1 --2- 172.21.15.43:0/3890537735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb97b9e6b0 0x55eb97f6d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.220+0000 7f1734954640 1 --2- 172.21.15.43:0/1846474850 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f17180afc60 0x7f17180b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.220+0000 7f172ffff640 1 -- 172.21.15.43:0/3890537735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb97bbf390 msgr2=0x55eb97f60810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.220+0000 7f172ffff640 1 --2- 172.21.15.43:0/3890537735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb97bbf390 0x55eb97f60810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.220+0000 7f172ffff640 1 -- 172.21.15.43:0/3890537735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb97b9e6b0 msgr2=0x55eb97f6d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.220+0000 7f172ffff640 1 --2- 172.21.15.43:0/3890537735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb97b9e6b0 0x55eb97f6d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.220+0000 7f172d7fa640 1 -- 172.21.15.43:0/1846474850 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f17280dee90 con 0x7f1730109fd0 2024-04-25T10:34:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.220+0000 7f172ffff640 1 -- 172.21.15.43:0/3890537735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb97e8ac30 con 0x55eb97b964d0 2024-04-25T10:34:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.221+0000 7f172ffff640 1 --2- 172.21.15.43:0/3890537735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97b964d0 0x55eb97f60d50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f172000bb60 tx=0x7f172000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7c71c320315174 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.221+0000 7f1734954640 1 --2- 172.21.15.43:0/1846474850 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f17180afc60 0x7f17180b2170 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f17280dc6c0 tx=0x7f17280748c0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:48.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.221+0000 7f17167fc640 1 -- 172.21.15.43:0/3890537735 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172000be60 con 0x55eb97b964d0 2024-04-25T10:34:48.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.221+0000 7f17167fc640 1 -- 172.21.15.43:0/3890537735 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1720017ad0 con 0x55eb97b964d0 2024-04-25T10:34:48.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.221+0000 7f17167fc640 1 -- 172.21.15.43:0/3890537735 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172000fac0 con 0x55eb97b964d0 2024-04-25T10:34:48.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.222+0000 7f173d26b740 1 -- 172.21.15.43:0/3890537735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97b964d0 msgr2=0x55eb97f60d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.222+0000 7f173d26b740 1 --2- 172.21.15.43:0/3890537735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97b964d0 0x55eb97f60d50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f172000bb60 tx=0x7f172000d710 comp rx=0 tx=0).stop 2024-04-25T10:34:48.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.222+0000 7f173d26b740 1 -- 172.21.15.43:0/3890537735 shutdown_connections 2024-04-25T10:34:48.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.222+0000 7f173d26b740 1 --2- 172.21.15.43:0/3890537735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb97b9e6b0 0x55eb97f6d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.222+0000 7f173d26b740 1 --2- 172.21.15.43:0/3890537735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97b964d0 0x55eb97f60d50 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.222+0000 7f173d26b740 1 --2- 172.21.15.43:0/3890537735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb97bbf390 0x55eb97f60810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.222+0000 7f173d26b740 1 -- 172.21.15.43:0/3890537735 >> 172.21.15.43:0/3890537735 conn(0x55eb97bb34e0 msgr2=0x55eb97f5cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:48.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.222+0000 7f173d26b740 1 -- 172.21.15.43:0/3890537735 shutdown_connections 2024-04-25T10:34:48.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.222+0000 7f173d26b740 1 -- 172.21.15.43:0/3890537735 wait complete. 2024-04-25T10:34:48.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.222+0000 7f173d26b740 1 -- start start 2024-04-25T10:34:48.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.222+0000 7f173d26b740 1 Processor -- start 2024-04-25T10:34:48.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f173d26b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb97b964d0 0x55eb97f6b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f173d26b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb97b9e6b0 0x55eb97f6b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f173d26b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97bbf390 0x55eb97f77210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f173d26b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55eb97f6dd90 con 0x55eb97bbf390 2024-04-25T10:34:48.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f173d26b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55eb97e4cdc0 con 0x55eb97b9e6b0 2024-04-25T10:34:48.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f173d26b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55eb97f6df10 con 0x55eb97b964d0 2024-04-25T10:34:48.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f173d26b740 10 client.0 _ll_register_callbacks cb 0x55eb97e68600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:48.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f172f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97bbf390 0x55eb97f77210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f172f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97bbf390 0x55eb97f77210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46966/0 (socket says 172.21.15.43:46966) 2024-04-25T10:34:48.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f172f7fe640 1 -- 172.21.15.43:0/3172633383 learned_addr learned my addr 172.21.15.43:0/3172633383 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:48.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f172ffff640 1 --2- 172.21.15.43:0/3172633383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb97b9e6b0 0x55eb97f6b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f172f7fe640 1 --2- 172.21.15.43:0/3172633383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb97b964d0 0x55eb97f6b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f172f7fe640 1 -- 172.21.15.43:0/3172633383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb97b964d0 msgr2=0x55eb97f6b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f172f7fe640 1 --2- 172.21.15.43:0/3172633383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb97b964d0 0x55eb97f6b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f172f7fe640 1 -- 172.21.15.43:0/3172633383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb97b9e6b0 msgr2=0x55eb97f6b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f172f7fe640 1 --2- 172.21.15.43:0/3172633383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb97b9e6b0 0x55eb97f6b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f172f7fe640 1 -- 172.21.15.43:0/3172633383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb97bd73f0 con 0x55eb97bbf390 2024-04-25T10:34:48.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f172f7fe640 1 --2- 172.21.15.43:0/3172633383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97bbf390 0x55eb97f77210 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1724005f70 tx=0x7f1724009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f71ef1744497b62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:48.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f170f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:48.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f170f7fe640 1 -- 172.21.15.43:0/3172633383 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724035040 con 0x55eb97bbf390 2024-04-25T10:34:48.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f173d26b740 1 -- 172.21.15.43:0/3172633383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1720017ad0 con 0x55eb97bbf390 2024-04-25T10:34:48.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.223+0000 7f170f7fe640 1 -- 172.21.15.43:0/3172633383 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f1724004230 con 0x55eb97bbf390 2024-04-25T10:34:48.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.224+0000 7f170f7fe640 1 -- 172.21.15.43:0/3172633383 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1724004430 con 0x55eb97bbf390 2024-04-25T10:34:48.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.224+0000 7f173d26b740 10 client.16089 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:48.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.224+0000 7f173d26b740 1 -- 172.21.15.43:0/3172633383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eb97ca17d0 con 0x55eb97bbf390 2024-04-25T10:34:48.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.224+0000 7f170f7fe640 1 -- 172.21.15.43:0/3172633383 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1720017ad0 con 0x55eb97bbf390 2024-04-25T10:34:48.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.224+0000 7f170f7fe640 1 -- 172.21.15.43:0/3172633383 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f172403a030 con 0x55eb97bbf390 2024-04-25T10:34:48.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.224+0000 7f173d26b740 10 client.16089 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:48.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.224+0000 7f173d26b740 10 client.16089 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:34:48.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.224+0000 7f173d26b740 20 client.16089 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:48.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.224+0000 7f173d26b740 1 --2- 172.21.15.43:0/3172633383 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55eb97b7df60 0x55eb97f7d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.224+0000 7f173d26b740 4 client.16089 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:34:48.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.224+0000 7f173d26b740 1 -- 172.21.15.43:0/3172633383 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb97ed3990 con 0x55eb97b7df60 2024-04-25T10:34:48.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.224+0000 7f172ffff640 1 --2- 172.21.15.43:0/3172633383 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55eb97b7df60 0x55eb97f7d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.225+0000 7f170f7fe640 10 client.16089 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:34:48.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.225+0000 7f172ffff640 1 --2- 172.21.15.43:0/3172633383 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55eb97b7df60 0x55eb97f7d5e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=523c3f3099c17078 server_cookie=5b4116c12f76fca3 in_seq=0 out_seq=0 2024-04-25T10:34:48.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.226+0000 7f170f7fe640 1 -- 172.21.15.43:0/3172633383 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eb97ed3990 con 0x55eb97b7df60 2024-04-25T10:34:48.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.226+0000 7f170f7fe640 10 client.16089 handle_command_reply: tid=0 2024-04-25T10:34:48.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.226+0000 7f173d26b740 1 client.16089 shutdown 2024-04-25T10:34:48.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.226+0000 7f173d26b740 1 -- 172.21.15.43:0/3172633383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97bbf390 msgr2=0x55eb97f77210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.226+0000 7f173d26b740 1 --2- 172.21.15.43:0/3172633383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97bbf390 0x55eb97f77210 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1724005f70 tx=0x7f1724009260 comp rx=0 tx=0).stop 2024-04-25T10:34:48.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.226+0000 7f173d26b740 1 -- 172.21.15.43:0/3172633383 shutdown_connections 2024-04-25T10:34:48.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.226+0000 7f173d26b740 1 --2- 172.21.15.43:0/3172633383 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55eb97b7df60 0x55eb97f7d5e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.226+0000 7f173d26b740 1 --2- 172.21.15.43:0/3172633383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb97b9e6b0 0x55eb97f6b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.226+0000 7f173d26b740 1 --2- 172.21.15.43:0/3172633383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97bbf390 0x55eb97f77210 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.226+0000 7f170f7fe640 0 client.16089 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:34:48.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.226+0000 7f173d26b740 1 --2- 172.21.15.43:0/3172633383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb97b964d0 0x55eb97f6b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.226+0000 7f173d26b740 1 -- 172.21.15.43:0/3172633383 >> 172.21.15.43:0/3172633383 conn(0x55eb97bb34e0 msgr2=0x55eb97f62530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:48.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.226+0000 7f173d26b740 1 -- 172.21.15.43:0/3172633383 shutdown_connections 2024-04-25T10:34:48.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.226+0000 7f173d26b740 1 -- 172.21.15.43:0/3172633383 wait complete. 2024-04-25T10:34:48.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.226+0000 7f173d26b740 20 client.16089 trim_cache size 0 max 16384 2024-04-25T10:34:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.240+0000 7f173d26b740 1 Processor -- start 2024-04-25T10:34:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.240+0000 7f173d26b740 1 -- start start 2024-04-25T10:34:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.240+0000 7f173d26b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb97b964d0 0x55eb97f60cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.240+0000 7f173d26b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb97b9e6b0 0x55eb97f76e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.240+0000 7f173d26b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97bbf390 0x55eb97f82da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.240+0000 7f173d26b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17301b2250 con 0x55eb97bbf390 2024-04-25T10:34:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.240+0000 7f173d26b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17301afb20 con 0x55eb97b964d0 2024-04-25T10:34:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.240+0000 7f173d26b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f17301ad970 con 0x55eb97b9e6b0 2024-04-25T10:34:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.240+0000 7f172f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97bbf390 0x55eb97f82da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.240+0000 7f172f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97bbf390 0x55eb97f82da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46970/0 (socket says 172.21.15.43:46970) 2024-04-25T10:34:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.240+0000 7f172f7fe640 1 -- 172.21.15.43:0/647718201 learned_addr learned my addr 172.21.15.43:0/647718201 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.240+0000 7f172f7fe640 1 --2- 172.21.15.43:0/647718201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb97b964d0 0x55eb97f60cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.240+0000 7f172f7fe640 1 -- 172.21.15.43:0/647718201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb97b9e6b0 msgr2=0x55eb97f76e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.240+0000 7f172f7fe640 1 --2- 172.21.15.43:0/647718201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb97b9e6b0 0x55eb97f76e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.240+0000 7f172f7fe640 1 -- 172.21.15.43:0/647718201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb97b964d0 msgr2=0x55eb97f60cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.240+0000 7f172f7fe640 1 --2- 172.21.15.43:0/647718201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb97b964d0 0x55eb97f60cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.240+0000 7f172f7fe640 1 -- 172.21.15.43:0/647718201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb97f6b050 con 0x55eb97bbf390 2024-04-25T10:34:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.241+0000 7f172f7fe640 1 --2- 172.21.15.43:0/647718201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97bbf390 0x55eb97f82da0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1724003800 tx=0x7f172402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d31557a6f0126fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.241+0000 7f1715ffb640 1 -- 172.21.15.43:0/647718201 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724035070 con 0x55eb97bbf390 2024-04-25T10:34:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.241+0000 7f1715ffb640 1 -- 172.21.15.43:0/647718201 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1724057070 con 0x55eb97bbf390 2024-04-25T10:34:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.241+0000 7f1715ffb640 1 -- 172.21.15.43:0/647718201 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724032580 con 0x55eb97bbf390 2024-04-25T10:34:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.242+0000 7f173d26b740 1 -- 172.21.15.43:0/647718201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97bbf390 msgr2=0x55eb97f82da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.242+0000 7f173d26b740 1 --2- 172.21.15.43:0/647718201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97bbf390 0x55eb97f82da0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1724003800 tx=0x7f172402a410 comp rx=0 tx=0).stop 2024-04-25T10:34:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.242+0000 7f173d26b740 1 -- 172.21.15.43:0/647718201 shutdown_connections 2024-04-25T10:34:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.242+0000 7f173d26b740 1 --2- 172.21.15.43:0/647718201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97bbf390 0x55eb97f82da0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.242+0000 7f173d26b740 1 --2- 172.21.15.43:0/647718201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb97b9e6b0 0x55eb97f76e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.242+0000 7f173d26b740 1 --2- 172.21.15.43:0/647718201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb97b964d0 0x55eb97f60cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.242+0000 7f173d26b740 1 -- 172.21.15.43:0/647718201 >> 172.21.15.43:0/647718201 conn(0x55eb97bb34e0 msgr2=0x55eb97f605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.242+0000 7f173d26b740 1 -- 172.21.15.43:0/647718201 shutdown_connections 2024-04-25T10:34:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.242+0000 7f173d26b740 1 -- 172.21.15.43:0/647718201 wait complete. 2024-04-25T10:34:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.242+0000 7f173d26b740 1 -- start start 2024-04-25T10:34:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.242+0000 7f173d26b740 1 Processor -- start 2024-04-25T10:34:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.242+0000 7f173d26b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb97b964d0 0x55eb97f73420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.242+0000 7f173d26b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97b9e6b0 0x55eb97f6e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f173d26b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb97bbf390 0x55eb97f6ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f173d26b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17301adfe0 con 0x55eb97b9e6b0 2024-04-25T10:34:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f173d26b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17301af450 con 0x55eb97bbf390 2024-04-25T10:34:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f173d26b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f17301ae6b0 con 0x55eb97b964d0 2024-04-25T10:34:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f173d26b740 10 client.0 _ll_register_callbacks cb 0x7f1724031590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f172ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97b9e6b0 0x55eb97f6e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f172f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb97b964d0 0x55eb97f73420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f172ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97b9e6b0 0x55eb97f6e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46972/0 (socket says 172.21.15.43:46972) 2024-04-25T10:34:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f172ffff640 1 -- 172.21.15.43:0/516851586 learned_addr learned my addr 172.21.15.43:0/516851586 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f172f7fe640 1 --2- 172.21.15.43:0/516851586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb97bbf390 0x55eb97f6ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f172ffff640 1 -- 172.21.15.43:0/516851586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb97b964d0 msgr2=0x55eb97f73420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f172ffff640 1 --2- 172.21.15.43:0/516851586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb97b964d0 0x55eb97f73420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f172ffff640 1 -- 172.21.15.43:0/516851586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb97bbf390 msgr2=0x55eb97f6ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f172ffff640 1 --2- 172.21.15.43:0/516851586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb97bbf390 0x55eb97f6ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f172ffff640 1 -- 172.21.15.43:0/516851586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb97f6f1f0 con 0x55eb97b9e6b0 2024-04-25T10:34:48.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f172ffff640 1 --2- 172.21.15.43:0/516851586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97b9e6b0 0x55eb97f6e770 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f172000ba80 tx=0x7f172000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2012f5a879309ec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:48.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f170f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:48.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f170f7fe640 1 -- 172.21.15.43:0/516851586 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720029070 con 0x55eb97b9e6b0 2024-04-25T10:34:48.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f170f7fe640 1 -- 172.21.15.43:0/516851586 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f1720005040 con 0x55eb97b9e6b0 2024-04-25T10:34:48.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f173d26b740 1 -- 172.21.15.43:0/516851586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1724057070 con 0x55eb97b9e6b0 2024-04-25T10:34:48.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.243+0000 7f170f7fe640 1 -- 172.21.15.43:0/516851586 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1720039050 con 0x55eb97b9e6b0 2024-04-25T10:34:48.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.244+0000 7f173d26b740 10 client.16095 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:48.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.244+0000 7f173d26b740 1 -- 172.21.15.43:0/516851586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eb97fe54d0 con 0x55eb97b9e6b0 2024-04-25T10:34:48.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.244+0000 7f170f7fe640 1 -- 172.21.15.43:0/516851586 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1724057070 con 0x55eb97b9e6b0 2024-04-25T10:34:48.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.244+0000 7f170f7fe640 1 -- 172.21.15.43:0/516851586 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1720004660 con 0x55eb97b9e6b0 2024-04-25T10:34:48.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.244+0000 7f173d26b740 10 client.16095 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:48.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.244+0000 7f173d26b740 10 client.16095 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:34:48.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.244+0000 7f173d26b740 20 client.16095 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:48.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.244+0000 7f173d26b740 1 --2- 172.21.15.43:0/516851586 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55eb97b533a0 0x55eb97fe78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.244+0000 7f173d26b740 4 client.16095 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:34:48.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.244+0000 7f173d26b740 1 -- 172.21.15.43:0/516851586 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb97ecd800 con 0x55eb97b533a0 2024-04-25T10:34:48.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.244+0000 7f172f7fe640 1 --2- 172.21.15.43:0/516851586 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55eb97b533a0 0x55eb97fe78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.245+0000 7f170f7fe640 10 client.16095 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:34:48.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.245+0000 7f172f7fe640 1 --2- 172.21.15.43:0/516851586 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55eb97b533a0 0x55eb97fe78d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73a0c577b1873b8a server_cookie=f02be90be4f649f6 in_seq=0 out_seq=0 2024-04-25T10:34:48.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.245+0000 7f170f7fe640 1 -- 172.21.15.43:0/516851586 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb97ecd800 con 0x55eb97b533a0 2024-04-25T10:34:48.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.245+0000 7f170f7fe640 10 client.16095 handle_command_reply: tid=0 2024-04-25T10:34:48.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.245+0000 7f173d26b740 1 client.16095 shutdown 2024-04-25T10:34:48.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.245+0000 7f173d26b740 1 -- 172.21.15.43:0/516851586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97b9e6b0 msgr2=0x55eb97f6e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.245+0000 7f173d26b740 1 --2- 172.21.15.43:0/516851586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97b9e6b0 0x55eb97f6e770 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f172000ba80 tx=0x7f172000b410 comp rx=0 tx=0).stop 2024-04-25T10:34:48.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.245+0000 7f173d26b740 1 -- 172.21.15.43:0/516851586 shutdown_connections 2024-04-25T10:34:48.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.245+0000 7f173d26b740 1 --2- 172.21.15.43:0/516851586 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55eb97b533a0 0x55eb97fe78d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.245+0000 7f173d26b740 1 --2- 172.21.15.43:0/516851586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb97bbf390 0x55eb97f6ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.245+0000 7f170f7fe640 0 client.16095 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:34:48.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.245+0000 7f173d26b740 1 --2- 172.21.15.43:0/516851586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb97b9e6b0 0x55eb97f6e770 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.245+0000 7f173d26b740 1 --2- 172.21.15.43:0/516851586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb97b964d0 0x55eb97f73420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.245+0000 7f173d26b740 1 -- 172.21.15.43:0/516851586 >> 172.21.15.43:0/516851586 conn(0x55eb97bb34e0 msgr2=0x55eb97f62410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:48.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.245+0000 7f173d26b740 1 -- 172.21.15.43:0/516851586 shutdown_connections 2024-04-25T10:34:48.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.245+0000 7f173d26b740 1 -- 172.21.15.43:0/516851586 wait complete. 2024-04-25T10:34:48.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.245+0000 7f173d26b740 20 client.16095 trim_cache size 0 max 16384 2024-04-25T10:34:48.253 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:34:48.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.246+0000 7f17363b7640 1 -- 172.21.15.43:0/1846474850 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f17180afc60 msgr2=0x7f17180b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.246+0000 7f17363b7640 1 --2- 172.21.15.43:0/1846474850 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f17180afc60 0x7f17180b2170 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f17280dc6c0 tx=0x7f17280748c0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.246+0000 7f17363b7640 1 -- 172.21.15.43:0/1846474850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1730109fd0 msgr2=0x7f173019efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.246+0000 7f17363b7640 1 --2- 172.21.15.43:0/1846474850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1730109fd0 0x7f173019efc0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f172800db20 tx=0x7f172800b260 comp rx=0 tx=0).stop 2024-04-25T10:34:48.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.246+0000 7f17363b7640 1 -- 172.21.15.43:0/1846474850 shutdown_connections 2024-04-25T10:34:48.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.246+0000 7f17363b7640 1 --2- 172.21.15.43:0/1846474850 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f17180afc60 0x7f17180b2170 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.246+0000 7f17363b7640 1 --2- 172.21.15.43:0/1846474850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1730109fd0 0x7f173019efc0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.246+0000 7f17363b7640 1 --2- 172.21.15.43:0/1846474850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1730109b30 0x7f173019ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.246+0000 7f17363b7640 1 --2- 172.21.15.43:0/1846474850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17301042d0 0x7f173019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.246+0000 7f17363b7640 1 -- 172.21.15.43:0/1846474850 >> 172.21.15.43:0/1846474850 conn(0x7f1730079280 msgr2=0x7f17301022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:48.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.246+0000 7f17363b7640 1 -- 172.21.15.43:0/1846474850 shutdown_connections 2024-04-25T10:34:48.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.246+0000 7f17363b7640 1 -- 172.21.15.43:0/1846474850 wait complete. 2024-04-25T10:34:48.259 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:34:48.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:48 smithi043 ceph-mon[29478]: pgmap v71: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-25T10:34:48.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.370+0000 7ff216eb1640 1 -- 172.21.15.43:0/2014991058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff210108a20 msgr2=0x7ff210102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.370+0000 7ff216eb1640 1 --2- 172.21.15.43:0/2014991058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff210108a20 0x7ff210102620 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff204009900 tx=0x7ff20402a410 comp rx=0 tx=0).stop 2024-04-25T10:34:48.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.370+0000 7ff216eb1640 1 -- 172.21.15.43:0/2014991058 shutdown_connections 2024-04-25T10:34:48.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.370+0000 7ff216eb1640 1 --2- 172.21.15.43:0/2014991058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff210108ec0 0x7ff210102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.370+0000 7ff216eb1640 1 --2- 172.21.15.43:0/2014991058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2101031d0 0x7ff21010f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.370+0000 7ff216eb1640 1 --2- 172.21.15.43:0/2014991058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff210108a20 0x7ff210102620 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.370+0000 7ff216eb1640 1 -- 172.21.15.43:0/2014991058 >> 172.21.15.43:0/2014991058 conn(0x7ff210078180 msgr2=0x7ff2100785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:48.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.370+0000 7ff216eb1640 1 -- 172.21.15.43:0/2014991058 shutdown_connections 2024-04-25T10:34:48.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.370+0000 7ff216eb1640 1 -- 172.21.15.43:0/2014991058 wait complete. 2024-04-25T10:34:48.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.371+0000 7ff216eb1640 1 Processor -- start 2024-04-25T10:34:48.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.371+0000 7ff216eb1640 1 -- start start 2024-04-25T10:34:48.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.371+0000 7ff216eb1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2101031d0 0x7ff21019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.371+0000 7ff216eb1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff210108a20 0x7ff21019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.371+0000 7ff216eb1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff210108ec0 0x7ff21019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.371+0000 7ff216eb1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff210112c20 con 0x7ff210108ec0 2024-04-25T10:34:48.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.371+0000 7ff216eb1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff210112aa0 con 0x7ff210108a20 2024-04-25T10:34:48.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.371+0000 7ff216eb1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff210112da0 con 0x7ff2101031d0 2024-04-25T10:34:48.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.371+0000 7ff21544e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff210108ec0 0x7ff21019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.372+0000 7ff21544e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff210108ec0 0x7ff21019ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46998/0 (socket says 172.21.15.43:46998) 2024-04-25T10:34:48.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.372+0000 7ff21544e640 1 -- 172.21.15.43:0/142955741 learned_addr learned my addr 172.21.15.43:0/142955741 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:48.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.372+0000 7ff214c4d640 1 --2- 172.21.15.43:0/142955741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2101031d0 0x7ff21019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.372+0000 7ff20ffff640 1 --2- 172.21.15.43:0/142955741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff210108a20 0x7ff21019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.372+0000 7ff21544e640 1 -- 172.21.15.43:0/142955741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2101031d0 msgr2=0x7ff21019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.372+0000 7ff21544e640 1 --2- 172.21.15.43:0/142955741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2101031d0 0x7ff21019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.372+0000 7ff21544e640 1 -- 172.21.15.43:0/142955741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff210108a20 msgr2=0x7ff21019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.372+0000 7ff21544e640 1 --2- 172.21.15.43:0/142955741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff210108a20 0x7ff21019ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.372+0000 7ff21544e640 1 -- 172.21.15.43:0/142955741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21019f590 con 0x7ff210108ec0 2024-04-25T10:34:48.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.372+0000 7ff21544e640 1 --2- 172.21.15.43:0/142955741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff210108ec0 0x7ff21019ef50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff20800cad0 tx=0x7ff20800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:48.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.372+0000 7ff20dffb640 1 -- 172.21.15.43:0/142955741 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20800ae20 con 0x7ff210108ec0 2024-04-25T10:34:48.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.372+0000 7ff20dffb640 1 -- 172.21.15.43:0/142955741 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff208005020 con 0x7ff210108ec0 2024-04-25T10:34:48.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.372+0000 7ff216eb1640 1 -- 172.21.15.43:0/142955741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21019f880 con 0x7ff210108ec0 2024-04-25T10:34:48.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.372+0000 7ff20dffb640 1 -- 172.21.15.43:0/142955741 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20800f6b0 con 0x7ff210108ec0 2024-04-25T10:34:48.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.373+0000 7ff216eb1640 1 -- 172.21.15.43:0/142955741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2101a7200 con 0x7ff210108ec0 2024-04-25T10:34:48.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.375+0000 7ff20dffb640 1 -- 172.21.15.43:0/142955741 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7ff2080051d0 con 0x7ff210108ec0 2024-04-25T10:34:48.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.375+0000 7ff21d9fd740 1 Processor -- start 2024-04-25T10:34:48.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.375+0000 7ff21d9fd740 1 -- start start 2024-04-25T10:34:48.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.375+0000 7ff21d9fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 0x564ef5f0c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.375+0000 7ff21d9fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b424d0 0x564ef5f0cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.375+0000 7ff21d9fd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ef5b4a6b0 0x564ef5f196a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.375+0000 7ff21d9fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564ef5e71d80 con 0x564ef5b6b390 2024-04-25T10:34:48.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.375+0000 7ff21d9fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564ef5e2ba20 con 0x564ef5b424d0 2024-04-25T10:34:48.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.375+0000 7ff21d9fd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564ef5e6dd40 con 0x564ef5b4a6b0 2024-04-25T10:34:48.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.376+0000 7ff20ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 0x564ef5f0c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.376+0000 7ff20ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 0x564ef5f0c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47010/0 (socket says 172.21.15.43:47010) 2024-04-25T10:34:48.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.376+0000 7ff20ffff640 1 -- 172.21.15.43:0/3821076658 learned_addr learned my addr 172.21.15.43:0/3821076658 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:48.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.376+0000 7ff214c4d640 1 --2- 172.21.15.43:0/3821076658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b424d0 0x564ef5f0cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.376+0000 7ff20dffb640 1 --2- 172.21.15.43:0/142955741 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff2000afc60 0x7ff2000b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.376+0000 7ff20ffff640 1 --2- 172.21.15.43:0/3821076658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ef5b4a6b0 0x564ef5f196a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.376+0000 7ff20ffff640 1 -- 172.21.15.43:0/3821076658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ef5b4a6b0 msgr2=0x564ef5f196a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.376+0000 7ff20ffff640 1 --2- 172.21.15.43:0/3821076658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ef5b4a6b0 0x564ef5f196a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.376+0000 7ff20ffff640 1 -- 172.21.15.43:0/3821076658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b424d0 msgr2=0x564ef5f0cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.376+0000 7ff20ffff640 1 --2- 172.21.15.43:0/3821076658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b424d0 0x564ef5f0cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.376+0000 7ff20ffff640 1 -- 172.21.15.43:0/3821076658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef5e36c30 con 0x564ef5b6b390 2024-04-25T10:34:48.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.376+0000 7ff21544e640 1 --2- 172.21.15.43:0/142955741 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff2000afc60 0x7ff2000b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.376+0000 7ff20dffb640 1 -- 172.21.15.43:0/142955741 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7ff2080def90 con 0x7ff210108ec0 2024-04-25T10:34:48.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.376+0000 7ff20ffff640 1 --2- 172.21.15.43:0/3821076658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 0x564ef5f0c810 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00bf90 tx=0x7ff1fc00b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56050ba1f26aa1a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:48.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.376+0000 7ff1faffd640 1 -- 172.21.15.43:0/3821076658 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc004020 con 0x564ef5b6b390 2024-04-25T10:34:48.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.376+0000 7ff1faffd640 1 -- 172.21.15.43:0/3821076658 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1fc0054d0 con 0x564ef5b6b390 2024-04-25T10:34:48.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.376+0000 7ff1faffd640 1 -- 172.21.15.43:0/3821076658 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc004a90 con 0x564ef5b6b390 2024-04-25T10:34:48.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.377+0000 7ff21d9fd740 1 -- 172.21.15.43:0/3821076658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 msgr2=0x564ef5f0c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.377+0000 7ff21544e640 1 --2- 172.21.15.43:0/142955741 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff2000afc60 0x7ff2000b2170 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff2080a6ef0 tx=0x7ff208075cf0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:48.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.377+0000 7ff21d9fd740 1 --2- 172.21.15.43:0/3821076658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 0x564ef5f0c810 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00bf90 tx=0x7ff1fc00b7a0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.377+0000 7ff21d9fd740 1 -- 172.21.15.43:0/3821076658 shutdown_connections 2024-04-25T10:34:48.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.377+0000 7ff21d9fd740 1 --2- 172.21.15.43:0/3821076658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b424d0 0x564ef5f0cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.377+0000 7ff21d9fd740 1 --2- 172.21.15.43:0/3821076658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ef5b4a6b0 0x564ef5f196a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.377+0000 7ff21d9fd740 1 --2- 172.21.15.43:0/3821076658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 0x564ef5f0c810 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.377+0000 7ff21d9fd740 1 -- 172.21.15.43:0/3821076658 >> 172.21.15.43:0/3821076658 conn(0x564ef5b5f4e0 msgr2=0x564ef5f08fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:48.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.377+0000 7ff21d9fd740 1 -- 172.21.15.43:0/3821076658 shutdown_connections 2024-04-25T10:34:48.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.377+0000 7ff21d9fd740 1 -- 172.21.15.43:0/3821076658 wait complete. 2024-04-25T10:34:48.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.378+0000 7ff21d9fd740 1 -- start start 2024-04-25T10:34:48.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.378+0000 7ff21d9fd740 1 Processor -- start 2024-04-25T10:34:48.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.378+0000 7ff21d9fd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ef5b424d0 0x564ef5f17260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.378+0000 7ff21d9fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b4a6b0 0x564ef5f177a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.378+0000 7ff21d9fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 0x564ef5f23210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.378+0000 7ff21d9fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564ef5f19d90 con 0x564ef5b6b390 2024-04-25T10:34:48.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.378+0000 7ff21d9fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564ef5df8dc0 con 0x564ef5b4a6b0 2024-04-25T10:34:48.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.378+0000 7ff21d9fd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564ef5f19f10 con 0x564ef5b424d0 2024-04-25T10:34:48.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.378+0000 7ff21d9fd740 10 client.0 _ll_register_callbacks cb 0x564ef5e14600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.378+0000 7ff20ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 0x564ef5f23210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.378+0000 7ff20ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 0x564ef5f23210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47026/0 (socket says 172.21.15.43:47026) 2024-04-25T10:34:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.378+0000 7ff20ffff640 1 -- 172.21.15.43:0/2074320937 learned_addr learned my addr 172.21.15.43:0/2074320937 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.379+0000 7ff214c4d640 1 --2- 172.21.15.43:0/2074320937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b4a6b0 0x564ef5f177a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.379+0000 7ff20ffff640 1 --2- 172.21.15.43:0/2074320937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ef5b424d0 0x564ef5f17260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.379+0000 7ff20ffff640 1 -- 172.21.15.43:0/2074320937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ef5b424d0 msgr2=0x564ef5f17260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.379+0000 7ff20ffff640 1 --2- 172.21.15.43:0/2074320937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ef5b424d0 0x564ef5f17260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.379+0000 7ff20ffff640 1 -- 172.21.15.43:0/2074320937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b4a6b0 msgr2=0x564ef5f177a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.379+0000 7ff20ffff640 1 --2- 172.21.15.43:0/2074320937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b4a6b0 0x564ef5f177a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.379+0000 7ff20ffff640 1 -- 172.21.15.43:0/2074320937 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef5b833f0 con 0x564ef5b6b390 2024-04-25T10:34:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.379+0000 7ff20ffff640 1 --2- 172.21.15.43:0/2074320937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 0x564ef5f23210 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc012280 tx=0x7ff1fc0044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ff9152c1410af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.379+0000 7ff1effff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.379+0000 7ff1effff640 1 -- 172.21.15.43:0/2074320937 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc027040 con 0x564ef5b6b390 2024-04-25T10:34:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.379+0000 7ff1effff640 1 -- 172.21.15.43:0/2074320937 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7ff1fc0037f0 con 0x564ef5b6b390 2024-04-25T10:34:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.379+0000 7ff21d9fd740 1 -- 172.21.15.43:0/2074320937 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1fc0054d0 con 0x564ef5b6b390 2024-04-25T10:34:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.379+0000 7ff1effff640 1 -- 172.21.15.43:0/2074320937 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff1fc004250 con 0x564ef5b6b390 2024-04-25T10:34:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.379+0000 7ff21d9fd740 10 client.16107 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.379+0000 7ff21d9fd740 1 -- 172.21.15.43:0/2074320937 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564ef5c4d7d0 con 0x564ef5b6b390 2024-04-25T10:34:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.380+0000 7ff1effff640 1 -- 172.21.15.43:0/2074320937 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1fc0054d0 con 0x564ef5b6b390 2024-04-25T10:34:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.380+0000 7ff1effff640 1 -- 172.21.15.43:0/2074320937 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff1fc00f9f0 con 0x564ef5b6b390 2024-04-25T10:34:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.380+0000 7ff21d9fd740 10 client.16107 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.380+0000 7ff21d9fd740 10 client.16107 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:34:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.380+0000 7ff21d9fd740 20 client.16107 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.380+0000 7ff21d9fd740 1 --2- 172.21.15.43:0/2074320937 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x564ef5b29f60 0x564ef5f294f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.380+0000 7ff21d9fd740 4 client.16107 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:34:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.380+0000 7ff21d9fd740 1 -- 172.21.15.43:0/2074320937 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ef5e7f990 con 0x564ef5b29f60 2024-04-25T10:34:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.380+0000 7ff214c4d640 1 --2- 172.21.15.43:0/2074320937 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x564ef5b29f60 0x564ef5f294f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.381+0000 7ff1effff640 10 client.16107 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:34:48.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.381+0000 7ff214c4d640 1 --2- 172.21.15.43:0/2074320937 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x564ef5b29f60 0x564ef5f294f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=96d00e690f864b97 server_cookie=c8b1ae82efacd938 in_seq=0 out_seq=0 2024-04-25T10:34:48.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.381+0000 7ff1effff640 1 -- 172.21.15.43:0/2074320937 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564ef5e7f990 con 0x564ef5b29f60 2024-04-25T10:34:48.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.381+0000 7ff1effff640 10 client.16107 handle_command_reply: tid=0 2024-04-25T10:34:48.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.381+0000 7ff21d9fd740 1 client.16107 shutdown 2024-04-25T10:34:48.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.382+0000 7ff21d9fd740 1 -- 172.21.15.43:0/2074320937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 msgr2=0x564ef5f23210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.382+0000 7ff21d9fd740 1 --2- 172.21.15.43:0/2074320937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 0x564ef5f23210 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc012280 tx=0x7ff1fc0044e0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.382+0000 7ff21d9fd740 1 -- 172.21.15.43:0/2074320937 shutdown_connections 2024-04-25T10:34:48.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.382+0000 7ff21d9fd740 1 --2- 172.21.15.43:0/2074320937 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x564ef5b29f60 0x564ef5f294f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.382+0000 7ff21d9fd740 1 --2- 172.21.15.43:0/2074320937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b4a6b0 0x564ef5f177a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.382+0000 7ff21d9fd740 1 --2- 172.21.15.43:0/2074320937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 0x564ef5f23210 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.382+0000 7ff1effff640 0 client.16107 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:34:48.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.382+0000 7ff21d9fd740 1 --2- 172.21.15.43:0/2074320937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ef5b424d0 0x564ef5f17260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.382+0000 7ff21d9fd740 1 -- 172.21.15.43:0/2074320937 >> 172.21.15.43:0/2074320937 conn(0x564ef5b5f4e0 msgr2=0x564ef5f0e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:48.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.382+0000 7ff21d9fd740 1 -- 172.21.15.43:0/2074320937 shutdown_connections 2024-04-25T10:34:48.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.382+0000 7ff21d9fd740 1 -- 172.21.15.43:0/2074320937 wait complete. 2024-04-25T10:34:48.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.382+0000 7ff21d9fd740 20 client.16107 trim_cache size 0 max 16384 2024-04-25T10:34:48.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.395+0000 7ff21d9fd740 1 Processor -- start 2024-04-25T10:34:48.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.395+0000 7ff21d9fd740 1 -- start start 2024-04-25T10:34:48.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff21d9fd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ef5b424d0 0x564ef5f23200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff21d9fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b4a6b0 0x564ef5f0ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff21d9fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 0x564ef5f14290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff21d9fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101b2210 con 0x564ef5b6b390 2024-04-25T10:34:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff21d9fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101afae0 con 0x564ef5b4a6b0 2024-04-25T10:34:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff21d9fd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101ad930 con 0x564ef5b424d0 2024-04-25T10:34:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff20ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 0x564ef5f14290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff20ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 0x564ef5f14290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47038/0 (socket says 172.21.15.43:47038) 2024-04-25T10:34:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff20ffff640 1 -- 172.21.15.43:0/3540236242 learned_addr learned my addr 172.21.15.43:0/3540236242 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff20ffff640 1 --2- 172.21.15.43:0/3540236242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ef5b424d0 0x564ef5f23200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff214c4d640 1 --2- 172.21.15.43:0/3540236242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b4a6b0 0x564ef5f0ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff20ffff640 1 -- 172.21.15.43:0/3540236242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ef5b424d0 msgr2=0x564ef5f23200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff20ffff640 1 --2- 172.21.15.43:0/3540236242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ef5b424d0 0x564ef5f23200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff20ffff640 1 -- 172.21.15.43:0/3540236242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b4a6b0 msgr2=0x564ef5f0ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff20ffff640 1 --2- 172.21.15.43:0/3540236242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b4a6b0 0x564ef5f0ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff20ffff640 1 -- 172.21.15.43:0/3540236242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef5f2f4c0 con 0x564ef5b6b390 2024-04-25T10:34:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff214c4d640 1 --2- 172.21.15.43:0/3540236242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b4a6b0 0x564ef5f0ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff20ffff640 1 --2- 172.21.15.43:0/3540236242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 0x564ef5f14290 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00ffb0 tx=0x7ff1fc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9466f8f5972bfb3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff1fa7fc640 1 -- 172.21.15.43:0/3540236242 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc022070 con 0x564ef5b6b390 2024-04-25T10:34:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff1fa7fc640 1 -- 172.21.15.43:0/3540236242 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1fc00b390 con 0x564ef5b6b390 2024-04-25T10:34:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.396+0000 7ff1fa7fc640 1 -- 172.21.15.43:0/3540236242 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc02d920 con 0x564ef5b6b390 2024-04-25T10:34:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.397+0000 7ff21d9fd740 1 -- 172.21.15.43:0/3540236242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 msgr2=0x564ef5f14290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.397+0000 7ff21d9fd740 1 --2- 172.21.15.43:0/3540236242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 0x564ef5f14290 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00ffb0 tx=0x7ff1fc000bc0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.397+0000 7ff21d9fd740 1 -- 172.21.15.43:0/3540236242 shutdown_connections 2024-04-25T10:34:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.397+0000 7ff21d9fd740 1 --2- 172.21.15.43:0/3540236242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b4a6b0 0x564ef5f0ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.397+0000 7ff21d9fd740 1 --2- 172.21.15.43:0/3540236242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b6b390 0x564ef5f14290 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.397+0000 7ff21d9fd740 1 --2- 172.21.15.43:0/3540236242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ef5b424d0 0x564ef5f23200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.397+0000 7ff21d9fd740 1 -- 172.21.15.43:0/3540236242 >> 172.21.15.43:0/3540236242 conn(0x564ef5b5f4e0 msgr2=0x564ef5f0e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.397+0000 7ff21d9fd740 1 -- 172.21.15.43:0/3540236242 shutdown_connections 2024-04-25T10:34:48.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.397+0000 7ff21d9fd740 1 -- 172.21.15.43:0/3540236242 wait complete. 2024-04-25T10:34:48.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.397+0000 7ff21d9fd740 1 -- start start 2024-04-25T10:34:48.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.398+0000 7ff21d9fd740 1 Processor -- start 2024-04-25T10:34:48.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.398+0000 7ff21d9fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b424d0 0x564ef5f2b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.398+0000 7ff21d9fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b4a6b0 0x564ef5f2b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.398+0000 7ff21d9fd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ef5b6b390 0x564ef5f8f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.398+0000 7ff21d9fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101adfa0 con 0x564ef5b4a6b0 2024-04-25T10:34:48.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.398+0000 7ff21d9fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101af410 con 0x564ef5b424d0 2024-04-25T10:34:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.398+0000 7ff21d9fd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101ae670 con 0x564ef5b6b390 2024-04-25T10:34:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.398+0000 7ff21d9fd740 10 client.0 _ll_register_callbacks cb 0x7ff204009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.398+0000 7ff214c4d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b4a6b0 0x564ef5f2b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.398+0000 7ff214c4d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b4a6b0 0x564ef5f2b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47040/0 (socket says 172.21.15.43:47040) 2024-04-25T10:34:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.398+0000 7ff214c4d640 1 -- 172.21.15.43:0/3948101568 learned_addr learned my addr 172.21.15.43:0/3948101568 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.398+0000 7ff20ffff640 1 --2- 172.21.15.43:0/3948101568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b424d0 0x564ef5f2b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.398+0000 7ff214c4d640 1 -- 172.21.15.43:0/3948101568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ef5b6b390 msgr2=0x564ef5f8f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.398+0000 7ff214c4d640 1 --2- 172.21.15.43:0/3948101568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ef5b6b390 0x564ef5f8f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.398+0000 7ff214c4d640 1 -- 172.21.15.43:0/3948101568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b424d0 msgr2=0x564ef5f2b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.398+0000 7ff214c4d640 1 --2- 172.21.15.43:0/3948101568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b424d0 0x564ef5f2b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.398+0000 7ff214c4d640 1 -- 172.21.15.43:0/3948101568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef5f2c290 con 0x564ef5b4a6b0 2024-04-25T10:34:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.399+0000 7ff214c4d640 1 --2- 172.21.15.43:0/3948101568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b4a6b0 0x564ef5f2b9c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff204005940 tx=0x7ff204004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336045d71d91ed25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.399+0000 7ff1effff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.399+0000 7ff1effff640 1 -- 172.21.15.43:0/3948101568 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204035040 con 0x564ef5b4a6b0 2024-04-25T10:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.399+0000 7ff1effff640 1 -- 172.21.15.43:0/3948101568 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7ff204005240 con 0x564ef5b4a6b0 2024-04-25T10:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.399+0000 7ff21d9fd740 1 -- 172.21.15.43:0/3948101568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1fc00b390 con 0x564ef5b4a6b0 2024-04-25T10:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.399+0000 7ff1effff640 1 -- 172.21.15.43:0/3948101568 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff204005440 con 0x564ef5b4a6b0 2024-04-25T10:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.399+0000 7ff21d9fd740 10 client.16113 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.399+0000 7ff21d9fd740 1 -- 172.21.15.43:0/3948101568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564ef5f90bc0 con 0x564ef5b4a6b0 2024-04-25T10:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.399+0000 7ff1effff640 1 -- 172.21.15.43:0/3948101568 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1fc00b390 con 0x564ef5b4a6b0 2024-04-25T10:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.399+0000 7ff1effff640 1 -- 172.21.15.43:0/3948101568 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff204046610 con 0x564ef5b4a6b0 2024-04-25T10:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.399+0000 7ff21d9fd740 10 client.16113 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.399+0000 7ff21d9fd740 10 client.16113 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.399+0000 7ff21d9fd740 20 client.16113 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.399+0000 7ff21d9fd740 1 --2- 172.21.15.43:0/3948101568 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x564ef5aff3a0 0x564ef5f93360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.399+0000 7ff21d9fd740 4 client.16113 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.399+0000 7ff21d9fd740 1 -- 172.21.15.43:0/3948101568 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ef5f0d0e0 con 0x564ef5aff3a0 2024-04-25T10:34:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.400+0000 7ff20ffff640 1 --2- 172.21.15.43:0/3948101568 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x564ef5aff3a0 0x564ef5f93360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.400+0000 7ff1effff640 10 client.16113 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:34:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.400+0000 7ff20ffff640 1 --2- 172.21.15.43:0/3948101568 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x564ef5aff3a0 0x564ef5f93360 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44b185683296151f server_cookie=dabd9132bc1c829b in_seq=0 out_seq=0 2024-04-25T10:34:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.400+0000 7ff1effff640 1 -- 172.21.15.43:0/3948101568 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ef5f0d0e0 con 0x564ef5aff3a0 2024-04-25T10:34:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.400+0000 7ff1effff640 10 client.16113 handle_command_reply: tid=0 2024-04-25T10:34:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.400+0000 7ff21d9fd740 1 client.16113 shutdown 2024-04-25T10:34:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.400+0000 7ff21d9fd740 1 -- 172.21.15.43:0/3948101568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b4a6b0 msgr2=0x564ef5f2b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.400+0000 7ff21d9fd740 1 --2- 172.21.15.43:0/3948101568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b4a6b0 0x564ef5f2b9c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff204005940 tx=0x7ff204004220 comp rx=0 tx=0).stop 2024-04-25T10:34:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.400+0000 7ff21d9fd740 1 -- 172.21.15.43:0/3948101568 shutdown_connections 2024-04-25T10:34:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.400+0000 7ff21d9fd740 1 --2- 172.21.15.43:0/3948101568 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x564ef5aff3a0 0x564ef5f93360 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.400+0000 7ff21d9fd740 1 --2- 172.21.15.43:0/3948101568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ef5b6b390 0x564ef5f8f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.400+0000 7ff21d9fd740 1 --2- 172.21.15.43:0/3948101568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564ef5b4a6b0 0x564ef5f2b9c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.400+0000 7ff21d9fd740 1 --2- 172.21.15.43:0/3948101568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ef5b424d0 0x564ef5f2b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.400+0000 7ff1effff640 0 client.16113 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:34:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.400+0000 7ff21d9fd740 1 -- 172.21.15.43:0/3948101568 >> 172.21.15.43:0/3948101568 conn(0x564ef5b5f4e0 msgr2=0x564ef5f19130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:48.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.400+0000 7ff21d9fd740 1 -- 172.21.15.43:0/3948101568 shutdown_connections 2024-04-25T10:34:48.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.400+0000 7ff21d9fd740 1 -- 172.21.15.43:0/3948101568 wait complete. 2024-04-25T10:34:48.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.401+0000 7ff21d9fd740 20 client.16113 trim_cache size 0 max 16384 2024-04-25T10:34:48.409 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:34:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.401+0000 7ff216eb1640 1 -- 172.21.15.43:0/142955741 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff2000afc60 msgr2=0x7ff2000b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.401+0000 7ff216eb1640 1 --2- 172.21.15.43:0/142955741 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff2000afc60 0x7ff2000b2170 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff2080a6ef0 tx=0x7ff208075cf0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.401+0000 7ff216eb1640 1 -- 172.21.15.43:0/142955741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff210108ec0 msgr2=0x7ff21019ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.401+0000 7ff216eb1640 1 --2- 172.21.15.43:0/142955741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff210108ec0 0x7ff21019ef50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff20800cad0 tx=0x7ff20800cf90 comp rx=0 tx=0).stop 2024-04-25T10:34:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.401+0000 7ff216eb1640 1 -- 172.21.15.43:0/142955741 shutdown_connections 2024-04-25T10:34:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.401+0000 7ff216eb1640 1 --2- 172.21.15.43:0/142955741 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff2000afc60 0x7ff2000b2170 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.401+0000 7ff216eb1640 1 --2- 172.21.15.43:0/142955741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff210108a20 0x7ff21019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.401+0000 7ff216eb1640 1 --2- 172.21.15.43:0/142955741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff210108ec0 0x7ff21019ef50 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.401+0000 7ff216eb1640 1 --2- 172.21.15.43:0/142955741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2101031d0 0x7ff21019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.401+0000 7ff216eb1640 1 -- 172.21.15.43:0/142955741 >> 172.21.15.43:0/142955741 conn(0x7ff210078180 msgr2=0x7ff210074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.401+0000 7ff216eb1640 1 -- 172.21.15.43:0/142955741 shutdown_connections 2024-04-25T10:34:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.401+0000 7ff216eb1640 1 -- 172.21.15.43:0/142955741 wait complete. 2024-04-25T10:34:48.414 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:34:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.527+0000 7fa2ba90d640 1 -- 172.21.15.43:0/1039991930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b4102af0 msgr2=0x7fa2b4102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.527+0000 7fa2ba90d640 1 --2- 172.21.15.43:0/1039991930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b4102af0 0x7fa2b4102f00 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009900 tx=0x7fa2a802a410 comp rx=0 tx=0).stop 2024-04-25T10:34:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.527+0000 7fa2ba90d640 1 -- 172.21.15.43:0/1039991930 shutdown_connections 2024-04-25T10:34:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.527+0000 7fa2ba90d640 1 --2- 172.21.15.43:0/1039991930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2b410d380 0x7fa2b410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.527+0000 7fa2ba90d640 1 --2- 172.21.15.43:0/1039991930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b4102af0 0x7fa2b4102f00 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.527+0000 7fa2ba90d640 1 --2- 172.21.15.43:0/1039991930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2b4108a20 0x7fa2b41025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.527+0000 7fa2ba90d640 1 -- 172.21.15.43:0/1039991930 >> 172.21.15.43:0/1039991930 conn(0x7fa2b4078100 msgr2=0x7fa2b4078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.527+0000 7fa2ba90d640 1 -- 172.21.15.43:0/1039991930 shutdown_connections 2024-04-25T10:34:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.527+0000 7fa2ba90d640 1 -- 172.21.15.43:0/1039991930 wait complete. 2024-04-25T10:34:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.528+0000 7fa2ba90d640 1 Processor -- start 2024-04-25T10:34:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.528+0000 7fa2ba90d640 1 -- start start 2024-04-25T10:34:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.528+0000 7fa2ba90d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b4102af0 0x7fa2b4199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.528+0000 7fa2ba90d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2b4108a20 0x7fa2b4199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.528+0000 7fa2ba90d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2b410d380 0x7fa2b419de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.528+0000 7fa2ba90d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b4112be0 con 0x7fa2b4102af0 2024-04-25T10:34:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.528+0000 7fa2ba90d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b4112a60 con 0x7fa2b4108a20 2024-04-25T10:34:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.528+0000 7fa2ba90d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b4112d60 con 0x7fa2b410d380 2024-04-25T10:34:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.528+0000 7fa2b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b4102af0 0x7fa2b4199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.528+0000 7fa2b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b4102af0 0x7fa2b4199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47052/0 (socket says 172.21.15.43:47052) 2024-04-25T10:34:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.528+0000 7fa2b3fff640 1 -- 172.21.15.43:0/2273427537 learned_addr learned my addr 172.21.15.43:0/2273427537 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.528+0000 7fa2b37fe640 1 --2- 172.21.15.43:0/2273427537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2b4108a20 0x7fa2b4199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.528+0000 7fa2b8eaa640 1 --2- 172.21.15.43:0/2273427537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2b410d380 0x7fa2b419de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.529+0000 7fa2b3fff640 1 -- 172.21.15.43:0/2273427537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2b410d380 msgr2=0x7fa2b419de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.529+0000 7fa2b3fff640 1 --2- 172.21.15.43:0/2273427537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2b410d380 0x7fa2b419de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.529+0000 7fa2b3fff640 1 -- 172.21.15.43:0/2273427537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2b4108a20 msgr2=0x7fa2b4199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.529+0000 7fa2b3fff640 1 --2- 172.21.15.43:0/2273427537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2b4108a20 0x7fa2b4199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.529+0000 7fa2b3fff640 1 -- 172.21.15.43:0/2273427537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b419e420 con 0x7fa2b4102af0 2024-04-25T10:34:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.529+0000 7fa2b8eaa640 1 --2- 172.21.15.43:0/2273427537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2b410d380 0x7fa2b419de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.529+0000 7fa2b3fff640 1 --2- 172.21.15.43:0/2273427537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b4102af0 0x7fa2b4199640 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400c9a0 tx=0x7fa2a400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.529+0000 7fa2b17fa640 1 -- 172.21.15.43:0/2273427537 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4014070 con 0x7fa2b4102af0 2024-04-25T10:34:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.529+0000 7fa2b17fa640 1 -- 172.21.15.43:0/2273427537 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a4004440 con 0x7fa2b4102af0 2024-04-25T10:34:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.529+0000 7fa2ba90d640 1 -- 172.21.15.43:0/2273427537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b419e710 con 0x7fa2b4102af0 2024-04-25T10:34:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.529+0000 7fa2b17fa640 1 -- 172.21.15.43:0/2273427537 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400f450 con 0x7fa2b4102af0 2024-04-25T10:34:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.529+0000 7fa2ba90d640 1 -- 172.21.15.43:0/2273427537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b419ec30 con 0x7fa2b4102af0 2024-04-25T10:34:48.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:48 smithi077 ceph-mon[32772]: pgmap v71: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-25T10:34:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.531+0000 7fa2c17c1740 1 Processor -- start 2024-04-25T10:34:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.531+0000 7fa2b17fa640 1 -- 172.21.15.43:0/2273427537 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fa2a4017070 con 0x7fa2b4102af0 2024-04-25T10:34:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.532+0000 7fa2c17c1740 1 -- start start 2024-04-25T10:34:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.532+0000 7fa2c17c1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953d5390 0x55a695776660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.532+0000 7fa2c17c1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953ac4d0 0x55a695776ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.532+0000 7fa2c17c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 0x55a6957858b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.532+0000 7fa2c17c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a6956dbd80 con 0x55a6953b46b0 2024-04-25T10:34:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.532+0000 7fa2c17c1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a695695a20 con 0x55a6953d5390 2024-04-25T10:34:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.532+0000 7fa2c17c1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a6956d7d40 con 0x55a6953ac4d0 2024-04-25T10:34:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.532+0000 7fa2b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 0x55a6957858b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.532+0000 7fa2b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 0x55a6957858b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47068/0 (socket says 172.21.15.43:47068) 2024-04-25T10:34:48.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.532+0000 7fa2b37fe640 1 -- 172.21.15.43:0/333588363 learned_addr learned my addr 172.21.15.43:0/333588363 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:48.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.532+0000 7fa2b8eaa640 1 --2- 172.21.15.43:0/333588363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953d5390 0x55a695776660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.532+0000 7fa2b3fff640 1 --2- 172.21.15.43:0/333588363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953ac4d0 0x55a695776ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.532+0000 7fa2b17fa640 1 --2- 172.21.15.43:0/2273427537 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa29c0afc60 0x7fa29c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.532+0000 7fa2b37fe640 1 -- 172.21.15.43:0/333588363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953ac4d0 msgr2=0x55a695776ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.532+0000 7fa2b37fe640 1 --2- 172.21.15.43:0/333588363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953ac4d0 0x55a695776ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.532+0000 7fa2b37fe640 1 -- 172.21.15.43:0/333588363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953d5390 msgr2=0x55a695776660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.532+0000 7fa2b37fe640 1 --2- 172.21.15.43:0/333588363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953d5390 0x55a695776660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.532+0000 7fa2b37fe640 1 -- 172.21.15.43:0/333588363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6956a0c30 con 0x55a6953b46b0 2024-04-25T10:34:48.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.532+0000 7fa2b8eaa640 1 --2- 172.21.15.43:0/2273427537 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa29c0afc60 0x7fa29c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.533+0000 7fa2b17fa640 1 -- 172.21.15.43:0/2273427537 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa2a40dec10 con 0x7fa2b4102af0 2024-04-25T10:34:48.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.533+0000 7fa2b37fe640 1 --2- 172.21.15.43:0/333588363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 0x55a6957858b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa2b406c4d0 tx=0x7fa2a8031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a21ca917877b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:48.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.533+0000 7fa29a7fc640 1 -- 172.21.15.43:0/333588363 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a803f070 con 0x55a6953b46b0 2024-04-25T10:34:48.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.533+0000 7fa29a7fc640 1 -- 172.21.15.43:0/333588363 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a8004030 con 0x55a6953b46b0 2024-04-25T10:34:48.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.533+0000 7fa29a7fc640 1 -- 172.21.15.43:0/333588363 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8035070 con 0x55a6953b46b0 2024-04-25T10:34:48.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.534+0000 7fa2c17c1740 1 -- 172.21.15.43:0/333588363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 msgr2=0x55a6957858b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.534+0000 7fa2b8eaa640 1 --2- 172.21.15.43:0/2273427537 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa29c0afc60 0x7fa29c0b2170 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55a695778490 tx=0x7fa2ac004a40 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:48.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.534+0000 7fa2c17c1740 1 --2- 172.21.15.43:0/333588363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 0x55a6957858b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa2b406c4d0 tx=0x7fa2a8031b10 comp rx=0 tx=0).stop 2024-04-25T10:34:48.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.534+0000 7fa2c17c1740 1 -- 172.21.15.43:0/333588363 shutdown_connections 2024-04-25T10:34:48.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.534+0000 7fa2c17c1740 1 --2- 172.21.15.43:0/333588363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 0x55a6957858b0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.534+0000 7fa2c17c1740 1 --2- 172.21.15.43:0/333588363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953ac4d0 0x55a695776ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.534+0000 7fa2c17c1740 1 --2- 172.21.15.43:0/333588363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953d5390 0x55a695776660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.534+0000 7fa2c17c1740 1 -- 172.21.15.43:0/333588363 >> 172.21.15.43:0/333588363 conn(0x55a6953c94e0 msgr2=0x55a695772fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:48.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.534+0000 7fa2c17c1740 1 -- 172.21.15.43:0/333588363 shutdown_connections 2024-04-25T10:34:48.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.534+0000 7fa2c17c1740 1 -- 172.21.15.43:0/333588363 wait complete. 2024-04-25T10:34:48.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.534+0000 7fa2c17c1740 1 -- start start 2024-04-25T10:34:48.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.534+0000 7fa2c17c1740 1 Processor -- start 2024-04-25T10:34:48.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2c17c1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953ac4d0 0x55a695783170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2c17c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 0x55a6957836b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2c17c1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953d5390 0x55a69578f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2c17c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a695785f40 con 0x55a6953b46b0 2024-04-25T10:34:48.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2c17c1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a695662dc0 con 0x55a6953ac4d0 2024-04-25T10:34:48.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2c17c1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a6957860c0 con 0x55a6953d5390 2024-04-25T10:34:48.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2c17c1740 10 client.0 _ll_register_callbacks cb 0x55a69567e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:48.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 0x55a6957836b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 0x55a6957836b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47076/0 (socket says 172.21.15.43:47076) 2024-04-25T10:34:48.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2b37fe640 1 -- 172.21.15.43:0/2638583073 learned_addr learned my addr 172.21.15.43:0/2638583073 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:48.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2b3fff640 1 --2- 172.21.15.43:0/2638583073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953ac4d0 0x55a695783170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2b8eaa640 1 --2- 172.21.15.43:0/2638583073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953d5390 0x55a69578f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2b37fe640 1 -- 172.21.15.43:0/2638583073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953d5390 msgr2=0x55a69578f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2b37fe640 1 --2- 172.21.15.43:0/2638583073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953d5390 0x55a69578f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2b37fe640 1 -- 172.21.15.43:0/2638583073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953ac4d0 msgr2=0x55a695783170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2b37fe640 1 --2- 172.21.15.43:0/2638583073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953ac4d0 0x55a695783170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2b37fe640 1 -- 172.21.15.43:0/2638583073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6953ed3f0 con 0x55a6953b46b0 2024-04-25T10:34:48.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2b8eaa640 1 --2- 172.21.15.43:0/2638583073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953d5390 0x55a69578f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:48.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2b3fff640 1 --2- 172.21.15.43:0/2638583073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953ac4d0 0x55a695783170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:48.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2b37fe640 1 --2- 172.21.15.43:0/2638583073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 0x55a6957836b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa2a802a930 tx=0x7fa2a80314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a215536bc3f3a5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:48.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.535+0000 7fa2937fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:48.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.536+0000 7fa2937fe640 1 -- 172.21.15.43:0/2638583073 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a804a030 con 0x55a6953b46b0 2024-04-25T10:34:48.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.536+0000 7fa2937fe640 1 -- 172.21.15.43:0/2638583073 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa2a8005020 con 0x55a6953b46b0 2024-04-25T10:34:48.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.536+0000 7fa2c17c1740 1 -- 172.21.15.43:0/2638583073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2a8004030 con 0x55a6953b46b0 2024-04-25T10:34:48.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.536+0000 7fa2937fe640 1 -- 172.21.15.43:0/2638583073 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa2a8054050 con 0x55a6953b46b0 2024-04-25T10:34:48.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.536+0000 7fa2c17c1740 10 client.16125 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:48.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.536+0000 7fa2c17c1740 1 -- 172.21.15.43:0/2638583073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a6954b77d0 con 0x55a6953b46b0 2024-04-25T10:34:48.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.536+0000 7fa2937fe640 1 -- 172.21.15.43:0/2638583073 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a8004030 con 0x55a6953b46b0 2024-04-25T10:34:48.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.536+0000 7fa2937fe640 1 -- 172.21.15.43:0/2638583073 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa2a80047a0 con 0x55a6953b46b0 2024-04-25T10:34:48.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.536+0000 7fa2c17c1740 10 client.16125 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:48.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.536+0000 7fa2c17c1740 10 client.16125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:48.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.536+0000 7fa2c17c1740 20 client.16125 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:48.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.536+0000 7fa2c17c1740 1 --2- 172.21.15.43:0/2638583073 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a695393f60 0x55a695795600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.536+0000 7fa2c17c1740 4 client.16125 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:34:48.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.536+0000 7fa2c17c1740 1 -- 172.21.15.43:0/2638583073 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6956e9990 con 0x55a695393f60 2024-04-25T10:34:48.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.537+0000 7fa2b3fff640 1 --2- 172.21.15.43:0/2638583073 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a695393f60 0x55a695795600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.537+0000 7fa2b3fff640 1 --2- 172.21.15.43:0/2638583073 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a695393f60 0x55a695795600 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49a0b0a482d88f2c server_cookie=5faf55416150922e in_seq=0 out_seq=0 2024-04-25T10:34:48.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.537+0000 7fa2937fe640 10 client.16125 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:48.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.538+0000 7fa2937fe640 1 -- 172.21.15.43:0/2638583073 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a6956e9990 con 0x55a695393f60 2024-04-25T10:34:48.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.538+0000 7fa2937fe640 10 client.16125 handle_command_reply: tid=0 2024-04-25T10:34:48.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.538+0000 7fa2c17c1740 1 client.16125 shutdown 2024-04-25T10:34:48.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.538+0000 7fa2c17c1740 1 -- 172.21.15.43:0/2638583073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 msgr2=0x55a6957836b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.538+0000 7fa2c17c1740 1 --2- 172.21.15.43:0/2638583073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 0x55a6957836b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa2a802a930 tx=0x7fa2a80314a0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.538+0000 7fa2c17c1740 1 -- 172.21.15.43:0/2638583073 shutdown_connections 2024-04-25T10:34:48.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.538+0000 7fa2c17c1740 1 --2- 172.21.15.43:0/2638583073 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a695393f60 0x55a695795600 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.538+0000 7fa2c17c1740 1 --2- 172.21.15.43:0/2638583073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953d5390 0x55a69578f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.538+0000 7fa2c17c1740 1 --2- 172.21.15.43:0/2638583073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 0x55a6957836b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.538+0000 7fa2937fe640 0 client.16125 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:48.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.538+0000 7fa2c17c1740 1 --2- 172.21.15.43:0/2638583073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953ac4d0 0x55a695783170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.538+0000 7fa2c17c1740 1 -- 172.21.15.43:0/2638583073 >> 172.21.15.43:0/2638583073 conn(0x55a6953c94e0 msgr2=0x55a6957784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:48.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.538+0000 7fa2c17c1740 1 -- 172.21.15.43:0/2638583073 shutdown_connections 2024-04-25T10:34:48.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.538+0000 7fa2c17c1740 1 -- 172.21.15.43:0/2638583073 wait complete. 2024-04-25T10:34:48.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.538+0000 7fa2c17c1740 20 client.16125 trim_cache size 0 max 16384 2024-04-25T10:34:48.549 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:48 smithi164 ceph-mon[32713]: pgmap v71: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-25T10:34:48.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.552+0000 7fa2c17c1740 1 Processor -- start 2024-04-25T10:34:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.552+0000 7fa2c17c1740 1 -- start start 2024-04-25T10:34:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.552+0000 7fa2c17c1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953ac4d0 0x55a695782340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.552+0000 7fa2c17c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 0x55a695783110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.552+0000 7fa2c17c1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953d5390 0x55a695783650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.552+0000 7fa2c17c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b41afda0 con 0x55a6953b46b0 2024-04-25T10:34:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.552+0000 7fa2c17c1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b41ad670 con 0x55a6953ac4d0 2024-04-25T10:34:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.552+0000 7fa2c17c1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b41ab4e0 con 0x55a6953d5390 2024-04-25T10:34:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.552+0000 7fa2b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 0x55a695783110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.552+0000 7fa2b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 0x55a695783110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47080/0 (socket says 172.21.15.43:47080) 2024-04-25T10:34:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.552+0000 7fa2b37fe640 1 -- 172.21.15.43:0/337261580 learned_addr learned my addr 172.21.15.43:0/337261580 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.552+0000 7fa2b3fff640 1 --2- 172.21.15.43:0/337261580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953ac4d0 0x55a695782340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.552+0000 7fa2b37fe640 1 -- 172.21.15.43:0/337261580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953d5390 msgr2=0x55a695783650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.552+0000 7fa2b37fe640 1 --2- 172.21.15.43:0/337261580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953d5390 0x55a695783650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.552+0000 7fa2b37fe640 1 -- 172.21.15.43:0/337261580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953ac4d0 msgr2=0x55a695782340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.552+0000 7fa2b37fe640 1 --2- 172.21.15.43:0/337261580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953ac4d0 0x55a695782340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.552+0000 7fa2b37fe640 1 -- 172.21.15.43:0/337261580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a69578ff20 con 0x55a6953b46b0 2024-04-25T10:34:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.553+0000 7fa2b37fe640 1 --2- 172.21.15.43:0/337261580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 0x55a695783110 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8006010 tx=0x7fa2a802ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f30181e20410326 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.553+0000 7fa299ffb640 1 -- 172.21.15.43:0/337261580 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a805e070 con 0x55a6953b46b0 2024-04-25T10:34:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.553+0000 7fa299ffb640 1 -- 172.21.15.43:0/337261580 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a8031510 con 0x55a6953b46b0 2024-04-25T10:34:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.553+0000 7fa299ffb640 1 -- 172.21.15.43:0/337261580 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a804a040 con 0x55a6953b46b0 2024-04-25T10:34:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.553+0000 7fa2c17c1740 1 -- 172.21.15.43:0/337261580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 msgr2=0x55a695783110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.553+0000 7fa2c17c1740 1 --2- 172.21.15.43:0/337261580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 0x55a695783110 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8006010 tx=0x7fa2a802ab10 comp rx=0 tx=0).stop 2024-04-25T10:34:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.553+0000 7fa2c17c1740 1 -- 172.21.15.43:0/337261580 shutdown_connections 2024-04-25T10:34:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.553+0000 7fa2c17c1740 1 --2- 172.21.15.43:0/337261580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953d5390 0x55a695783650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.553+0000 7fa2c17c1740 1 --2- 172.21.15.43:0/337261580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953b46b0 0x55a695783110 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.553+0000 7fa2c17c1740 1 --2- 172.21.15.43:0/337261580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953ac4d0 0x55a695782340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.553+0000 7fa2c17c1740 1 -- 172.21.15.43:0/337261580 >> 172.21.15.43:0/337261580 conn(0x55a6953c94e0 msgr2=0x55a695782a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.554+0000 7fa2c17c1740 1 -- 172.21.15.43:0/337261580 shutdown_connections 2024-04-25T10:34:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.554+0000 7fa2c17c1740 1 -- 172.21.15.43:0/337261580 wait complete. 2024-04-25T10:34:48.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.554+0000 7fa2c17c1740 1 -- start start 2024-04-25T10:34:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.554+0000 7fa2c17c1740 1 Processor -- start 2024-04-25T10:34:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.554+0000 7fa2c17c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953ac4d0 0x55a69578b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.554+0000 7fa2c17c1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953b46b0 0x55a695786bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.554+0000 7fa2c17c1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953d5390 0x55a695787130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.554+0000 7fa2c17c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b41abb30 con 0x55a6953ac4d0 2024-04-25T10:34:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.554+0000 7fa2c17c1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b41acfa0 con 0x55a6953d5390 2024-04-25T10:34:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.554+0000 7fa2c17c1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b41ac200 con 0x55a6953b46b0 2024-04-25T10:34:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.554+0000 7fa2c17c1740 10 client.0 _ll_register_callbacks cb 0x7fa2ac0084d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.554+0000 7fa2b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953ac4d0 0x55a69578b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.555+0000 7fa2b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953ac4d0 0x55a69578b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47086/0 (socket says 172.21.15.43:47086) 2024-04-25T10:34:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.555+0000 7fa2b3fff640 1 -- 172.21.15.43:0/2868800261 learned_addr learned my addr 172.21.15.43:0/2868800261 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.555+0000 7fa2b37fe640 1 --2- 172.21.15.43:0/2868800261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953b46b0 0x55a695786bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.555+0000 7fa2b8eaa640 1 --2- 172.21.15.43:0/2868800261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953d5390 0x55a695787130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.555+0000 7fa2b3fff640 1 -- 172.21.15.43:0/2868800261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953b46b0 msgr2=0x55a695786bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.555+0000 7fa2b3fff640 1 --2- 172.21.15.43:0/2868800261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953b46b0 0x55a695786bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.555+0000 7fa2b3fff640 1 -- 172.21.15.43:0/2868800261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953d5390 msgr2=0x55a695787130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.555+0000 7fa2b3fff640 1 --2- 172.21.15.43:0/2868800261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953d5390 0x55a695787130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.555+0000 7fa2b3fff640 1 -- 172.21.15.43:0/2868800261 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a695787820 con 0x55a6953ac4d0 2024-04-25T10:34:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.555+0000 7fa2b37fe640 1 --2- 172.21.15.43:0/2868800261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953b46b0 0x55a695786bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.555+0000 7fa2b8eaa640 1 --2- 172.21.15.43:0/2868800261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953d5390 0x55a695787130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.555+0000 7fa2b3fff640 1 --2- 172.21.15.43:0/2868800261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953ac4d0 0x55a69578b8a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa2a40dc990 tx=0x7fa2a40045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fde3132699b812 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.555+0000 7fa2937fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.555+0000 7fa2937fe640 1 -- 172.21.15.43:0/2868800261 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a40a74e0 con 0x55a6953ac4d0 2024-04-25T10:34:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.555+0000 7fa2937fe640 1 -- 172.21.15.43:0/2868800261 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa2a40a3070 con 0x55a6953ac4d0 2024-04-25T10:34:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.555+0000 7fa2c17c1740 1 -- 172.21.15.43:0/2868800261 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2a8031510 con 0x55a6953ac4d0 2024-04-25T10:34:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.555+0000 7fa2937fe640 1 -- 172.21.15.43:0/2868800261 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa2a40a6de0 con 0x55a6953ac4d0 2024-04-25T10:34:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.555+0000 7fa2c17c1740 10 client.16131 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:48.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.555+0000 7fa2c17c1740 1 -- 172.21.15.43:0/2868800261 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a6957fd670 con 0x55a6953ac4d0 2024-04-25T10:34:48.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.555+0000 7fa2937fe640 1 -- 172.21.15.43:0/2868800261 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a8031510 con 0x55a6953ac4d0 2024-04-25T10:34:48.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.556+0000 7fa2937fe640 1 -- 172.21.15.43:0/2868800261 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa2a40dcc00 con 0x55a6953ac4d0 2024-04-25T10:34:48.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.556+0000 7fa2c17c1740 10 client.16131 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:48.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.556+0000 7fa2c17c1740 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:48.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.556+0000 7fa2c17c1740 20 client.16131 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:48.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.556+0000 7fa2c17c1740 1 --2- 172.21.15.43:0/2868800261 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a6953693a0 0x55a6957ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:48.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.556+0000 7fa2c17c1740 4 client.16131 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:34:48.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.556+0000 7fa2c17c1740 1 -- 172.21.15.43:0/2868800261 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a6956e3800 con 0x55a6953693a0 2024-04-25T10:34:48.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.556+0000 7fa2b37fe640 1 --2- 172.21.15.43:0/2868800261 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a6953693a0 0x55a6957ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:48.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.556+0000 7fa2937fe640 10 client.16131 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:48.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.556+0000 7fa2b37fe640 1 --2- 172.21.15.43:0/2868800261 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a6953693a0 0x55a6957ffde0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b891cc9ff5ddd0c3 server_cookie=7475ee0902e7294e in_seq=0 out_seq=0 2024-04-25T10:34:48.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.557+0000 7fa2937fe640 1 -- 172.21.15.43:0/2868800261 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a6956e3800 con 0x55a6953693a0 2024-04-25T10:34:48.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.557+0000 7fa2937fe640 10 client.16131 handle_command_reply: tid=0 2024-04-25T10:34:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.557+0000 7fa2c17c1740 1 client.16131 shutdown 2024-04-25T10:34:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.557+0000 7fa2c17c1740 1 -- 172.21.15.43:0/2868800261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953ac4d0 msgr2=0x55a69578b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.557+0000 7fa2c17c1740 1 --2- 172.21.15.43:0/2868800261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953ac4d0 0x55a69578b8a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa2a40dc990 tx=0x7fa2a40045f0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.557+0000 7fa2c17c1740 1 -- 172.21.15.43:0/2868800261 shutdown_connections 2024-04-25T10:34:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.557+0000 7fa2c17c1740 1 --2- 172.21.15.43:0/2868800261 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a6953693a0 0x55a6957ffde0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.557+0000 7fa2c17c1740 1 --2- 172.21.15.43:0/2868800261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6953d5390 0x55a695787130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.557+0000 7fa2c17c1740 1 --2- 172.21.15.43:0/2868800261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6953b46b0 0x55a695786bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.557+0000 7fa2937fe640 0 client.16131 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.557+0000 7fa2c17c1740 1 --2- 172.21.15.43:0/2868800261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6953ac4d0 0x55a69578b8a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.557+0000 7fa2c17c1740 1 -- 172.21.15.43:0/2868800261 >> 172.21.15.43:0/2868800261 conn(0x55a6953c94e0 msgr2=0x55a6957820a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.557+0000 7fa2c17c1740 1 -- 172.21.15.43:0/2868800261 shutdown_connections 2024-04-25T10:34:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.557+0000 7fa2c17c1740 1 -- 172.21.15.43:0/2868800261 wait complete. 2024-04-25T10:34:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.557+0000 7fa2c17c1740 20 client.16131 trim_cache size 0 max 16384 2024-04-25T10:34:48.566 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:34:48.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.558+0000 7fa2ba90d640 1 -- 172.21.15.43:0/2273427537 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa29c0afc60 msgr2=0x7fa29c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.558+0000 7fa2ba90d640 1 --2- 172.21.15.43:0/2273427537 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa29c0afc60 0x7fa29c0b2170 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55a695778490 tx=0x7fa2ac004a40 comp rx=0 tx=0).stop 2024-04-25T10:34:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.558+0000 7fa2ba90d640 1 -- 172.21.15.43:0/2273427537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b4102af0 msgr2=0x7fa2b4199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.558+0000 7fa2ba90d640 1 --2- 172.21.15.43:0/2273427537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b4102af0 0x7fa2b4199640 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400c9a0 tx=0x7fa2a400ce60 comp rx=0 tx=0).stop 2024-04-25T10:34:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.558+0000 7fa2ba90d640 1 -- 172.21.15.43:0/2273427537 shutdown_connections 2024-04-25T10:34:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.558+0000 7fa2ba90d640 1 --2- 172.21.15.43:0/2273427537 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa29c0afc60 0x7fa29c0b2170 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.558+0000 7fa2ba90d640 1 --2- 172.21.15.43:0/2273427537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2b4108a20 0x7fa2b4199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.558+0000 7fa2ba90d640 1 --2- 172.21.15.43:0/2273427537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2b410d380 0x7fa2b419de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.558+0000 7fa2ba90d640 1 --2- 172.21.15.43:0/2273427537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b4102af0 0x7fa2b4199640 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.558+0000 7fa2ba90d640 1 -- 172.21.15.43:0/2273427537 >> 172.21.15.43:0/2273427537 conn(0x7fa2b4078100 msgr2=0x7fa2b41040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.558+0000 7fa2ba90d640 1 -- 172.21.15.43:0/2273427537 shutdown_connections 2024-04-25T10:34:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:48.558+0000 7fa2ba90d640 1 -- 172.21.15.43:0/2273427537 wait complete. 2024-04-25T10:34:49.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:49 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1941846271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:34:49.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:49 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1941846271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:34:49.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:49 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1941846271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:34:49.571 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:34:49.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:34:49.572 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:34:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.702+0000 7f1d4a586640 1 -- 172.21.15.43:0/2364990471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d44108a20 msgr2=0x7f1d441025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.702+0000 7f1d4a586640 1 --2- 172.21.15.43:0/2364990471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d44108a20 0x7f1d441025b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009900 tx=0x7f1d3402a410 comp rx=0 tx=0).stop 2024-04-25T10:34:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.702+0000 7f1d4a586640 1 -- 172.21.15.43:0/2364990471 shutdown_connections 2024-04-25T10:34:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.702+0000 7f1d4a586640 1 --2- 172.21.15.43:0/2364990471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d4410d380 0x7f1d4410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.702+0000 7f1d4a586640 1 --2- 172.21.15.43:0/2364990471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d44102af0 0x7f1d44102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.702+0000 7f1d4a586640 1 --2- 172.21.15.43:0/2364990471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d44108a20 0x7f1d441025b0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.702+0000 7f1d4a586640 1 -- 172.21.15.43:0/2364990471 >> 172.21.15.43:0/2364990471 conn(0x7f1d44078100 msgr2=0x7f1d44078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.702+0000 7f1d4a586640 1 -- 172.21.15.43:0/2364990471 shutdown_connections 2024-04-25T10:34:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.702+0000 7f1d4a586640 1 -- 172.21.15.43:0/2364990471 wait complete. 2024-04-25T10:34:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.703+0000 7f1d4a586640 1 Processor -- start 2024-04-25T10:34:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.703+0000 7f1d4a586640 1 -- start start 2024-04-25T10:34:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.703+0000 7f1d4a586640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d44102af0 0x7f1d440ff250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.703+0000 7f1d4a586640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d44108a20 0x7f1d440ff790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.703+0000 7f1d4a586640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d4410d380 0x7f1d440ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.703+0000 7f1d4a586640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44112be0 con 0x7f1d44102af0 2024-04-25T10:34:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.703+0000 7f1d4a586640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44112a60 con 0x7f1d44108a20 2024-04-25T10:34:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.703+0000 7f1d4a586640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44112d60 con 0x7f1d4410d380 2024-04-25T10:34:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.703+0000 7f1d43fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d44102af0 0x7f1d440ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.703+0000 7f1d43fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d44102af0 0x7f1d440ff250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47488/0 (socket says 172.21.15.43:47488) 2024-04-25T10:34:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.703+0000 7f1d43fff640 1 -- 172.21.15.43:0/2171974077 learned_addr learned my addr 172.21.15.43:0/2171974077 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.704+0000 7f1d437fe640 1 --2- 172.21.15.43:0/2171974077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d44108a20 0x7f1d440ff790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.704+0000 7f1d48b23640 1 --2- 172.21.15.43:0/2171974077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d4410d380 0x7f1d440ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.704+0000 7f1d43fff640 1 -- 172.21.15.43:0/2171974077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d4410d380 msgr2=0x7f1d440ffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.704+0000 7f1d43fff640 1 --2- 172.21.15.43:0/2171974077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d4410d380 0x7f1d440ffcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.704+0000 7f1d43fff640 1 -- 172.21.15.43:0/2171974077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d44108a20 msgr2=0x7f1d440ff790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.704+0000 7f1d43fff640 1 --2- 172.21.15.43:0/2171974077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d44108a20 0x7f1d440ff790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.704+0000 7f1d43fff640 1 -- 172.21.15.43:0/2171974077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d441a8a70 con 0x7f1d44102af0 2024-04-25T10:34:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.704+0000 7f1d43fff640 1 --2- 172.21.15.43:0/2171974077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d44102af0 0x7f1d440ff250 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1d34032fd0 tx=0x7f1d34004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.704+0000 7f1d417fa640 1 -- 172.21.15.43:0/2171974077 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3403f030 con 0x7f1d44102af0 2024-04-25T10:34:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.704+0000 7f1d4a586640 1 -- 172.21.15.43:0/2171974077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d441a8d00 con 0x7f1d44102af0 2024-04-25T10:34:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.704+0000 7f1d417fa640 1 -- 172.21.15.43:0/2171974077 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d34004490 con 0x7f1d44102af0 2024-04-25T10:34:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.704+0000 7f1d417fa640 1 -- 172.21.15.43:0/2171974077 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34035070 con 0x7f1d44102af0 2024-04-25T10:34:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.704+0000 7f1d4a586640 1 -- 172.21.15.43:0/2171974077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d441a91a0 con 0x7f1d44102af0 2024-04-25T10:34:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.707+0000 7f1d5143f740 1 Processor -- start 2024-04-25T10:34:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.707+0000 7f1d5143f740 1 -- start start 2024-04-25T10:34:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.707+0000 7f1d417fa640 1 -- 172.21.15.43:0/2171974077 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f1d34042030 con 0x7f1d44102af0 2024-04-25T10:34:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.707+0000 7f1d5143f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 0x55aae52e0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.707+0000 7f1d5143f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aae4f17550 0x55aae52e0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.707+0000 7f1d5143f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f1f730 0x55aae52e10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.708+0000 7f1d5143f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aae5246b00 con 0x55aae4f40410 2024-04-25T10:34:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.708+0000 7f1d5143f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55aae51e2920 con 0x55aae4f17550 2024-04-25T10:34:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.708+0000 7f1d5143f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55aae5242ac0 con 0x55aae4f1f730 2024-04-25T10:34:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.708+0000 7f1d48b23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 0x55aae52e0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.708+0000 7f1d48b23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 0x55aae52e0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47494/0 (socket says 172.21.15.43:47494) 2024-04-25T10:34:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.708+0000 7f1d48b23640 1 -- 172.21.15.43:0/1491979938 learned_addr learned my addr 172.21.15.43:0/1491979938 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.708+0000 7f1d43fff640 1 --2- 172.21.15.43:0/1491979938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aae4f17550 0x55aae52e0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.708+0000 7f1d437fe640 1 --2- 172.21.15.43:0/1491979938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f1f730 0x55aae52e10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.708+0000 7f1d48b23640 1 -- 172.21.15.43:0/1491979938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f1f730 msgr2=0x55aae52e10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.708+0000 7f1d48b23640 1 --2- 172.21.15.43:0/1491979938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f1f730 0x55aae52e10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.708+0000 7f1d48b23640 1 -- 172.21.15.43:0/1491979938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aae4f17550 msgr2=0x55aae52e0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.708+0000 7f1d48b23640 1 --2- 172.21.15.43:0/1491979938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aae4f17550 0x55aae52e0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.708+0000 7f1d48b23640 1 -- 172.21.15.43:0/1491979938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aae50207e0 con 0x55aae4f40410 2024-04-25T10:34:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.708+0000 7f1d437fe640 1 --2- 172.21.15.43:0/1491979938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f1f730 0x55aae52e10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.708+0000 7f1d417fa640 1 --2- 172.21.15.43:0/2171974077 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1d2c0afc60 0x7f1d2c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.708+0000 7f1d48b23640 1 --2- 172.21.15.43:0/1491979938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 0x55aae52e0620 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009f70 tx=0x7f1d3c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642bc080d82915a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.708+0000 7f1d43fff640 1 --2- 172.21.15.43:0/2171974077 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1d2c0afc60 0x7f1d2c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.708+0000 7f1d2a7fc640 1 -- 172.21.15.43:0/1491979938 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c016070 con 0x55aae4f40410 2024-04-25T10:34:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.708+0000 7f1d417fa640 1 -- 172.21.15.43:0/2171974077 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f1d340ff460 con 0x7f1d44102af0 2024-04-25T10:34:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.709+0000 7f1d2a7fc640 1 -- 172.21.15.43:0/1491979938 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d3c004d20 con 0x55aae4f40410 2024-04-25T10:34:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.709+0000 7f1d2a7fc640 1 -- 172.21.15.43:0/1491979938 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c012790 con 0x55aae4f40410 2024-04-25T10:34:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.709+0000 7f1d5143f740 1 -- 172.21.15.43:0/1491979938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 msgr2=0x55aae52e0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.709+0000 7f1d43fff640 1 --2- 172.21.15.43:0/2171974077 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1d2c0afc60 0x7f1d2c0b2170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1d340fcc90 tx=0x7f1d34105060 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.709+0000 7f1d5143f740 1 --2- 172.21.15.43:0/1491979938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 0x55aae52e0620 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009f70 tx=0x7f1d3c010800 comp rx=0 tx=0).stop 2024-04-25T10:34:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.709+0000 7f1d5143f740 1 -- 172.21.15.43:0/1491979938 shutdown_connections 2024-04-25T10:34:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.709+0000 7f1d5143f740 1 --2- 172.21.15.43:0/1491979938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aae4f17550 0x55aae52e0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.709+0000 7f1d5143f740 1 --2- 172.21.15.43:0/1491979938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f1f730 0x55aae52e10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.709+0000 7f1d5143f740 1 --2- 172.21.15.43:0/1491979938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 0x55aae52e0620 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.709+0000 7f1d5143f740 1 -- 172.21.15.43:0/1491979938 >> 172.21.15.43:0/1491979938 conn(0x55aae4f34560 msgr2=0x55aae52dcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.709+0000 7f1d5143f740 1 -- 172.21.15.43:0/1491979938 shutdown_connections 2024-04-25T10:34:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.709+0000 7f1d5143f740 1 -- 172.21.15.43:0/1491979938 wait complete. 2024-04-25T10:34:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.710+0000 7f1d5143f740 1 -- start start 2024-04-25T10:34:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.710+0000 7f1d5143f740 1 Processor -- start 2024-04-25T10:34:49.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.710+0000 7f1d5143f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f17550 0x55aae52eac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.710+0000 7f1d5143f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f1f730 0x55aae52eb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.710+0000 7f1d5143f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aae4f40410 0x55aae52f0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.710+0000 7f1d5143f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aae52ed970 con 0x55aae4f17550 2024-04-25T10:34:49.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.710+0000 7f1d5143f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55aae51e05d0 con 0x55aae4f40410 2024-04-25T10:34:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.710+0000 7f1d5143f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55aae52edaf0 con 0x55aae4f1f730 2024-04-25T10:34:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.711+0000 7f1d5143f740 10 client.0 _ll_register_callbacks cb 0x55aae51ce080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.711+0000 7f1d48b23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f17550 0x55aae52eac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.711+0000 7f1d48b23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f17550 0x55aae52eac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47504/0 (socket says 172.21.15.43:47504) 2024-04-25T10:34:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.711+0000 7f1d48b23640 1 -- 172.21.15.43:0/916837791 learned_addr learned my addr 172.21.15.43:0/916837791 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.711+0000 7f1d437fe640 1 --2- 172.21.15.43:0/916837791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f1f730 0x55aae52eb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.711+0000 7f1d48b23640 1 --2- 172.21.15.43:0/916837791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aae4f40410 0x55aae52f0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.711+0000 7f1d48b23640 1 -- 172.21.15.43:0/916837791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f1f730 msgr2=0x55aae52eb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.711+0000 7f1d48b23640 1 --2- 172.21.15.43:0/916837791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f1f730 0x55aae52eb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.711+0000 7f1d48b23640 1 -- 172.21.15.43:0/916837791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aae4f40410 msgr2=0x55aae52f0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.711+0000 7f1d48b23640 1 --2- 172.21.15.43:0/916837791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aae4f40410 0x55aae52f0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.711+0000 7f1d48b23640 1 -- 172.21.15.43:0/916837791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aae5080f70 con 0x55aae4f17550 2024-04-25T10:34:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.711+0000 7f1d48b23640 1 --2- 172.21.15.43:0/916837791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f17550 0x55aae52eac80 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009d30 tx=0x7f1d3c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85d406562eb47bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.711+0000 7f1d237fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.711+0000 7f1d237fe640 1 -- 172.21.15.43:0/916837791 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c016070 con 0x55aae4f17550 2024-04-25T10:34:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.711+0000 7f1d237fe640 1 -- 172.21.15.43:0/916837791 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f1d3c012d90 con 0x55aae4f17550 2024-04-25T10:34:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.711+0000 7f1d5143f740 1 -- 172.21.15.43:0/916837791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d3c004d20 con 0x55aae4f17550 2024-04-25T10:34:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.712+0000 7f1d237fe640 1 -- 172.21.15.43:0/916837791 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d3c02a430 con 0x55aae4f17550 2024-04-25T10:34:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.712+0000 7f1d5143f740 10 client.16143 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.712+0000 7f1d5143f740 1 -- 172.21.15.43:0/916837791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aae4f18170 con 0x55aae4f17550 2024-04-25T10:34:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.712+0000 7f1d237fe640 1 -- 172.21.15.43:0/916837791 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d3c004d20 con 0x55aae4f17550 2024-04-25T10:34:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.712+0000 7f1d237fe640 1 -- 172.21.15.43:0/916837791 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1d3c00f040 con 0x55aae4f17550 2024-04-25T10:34:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.712+0000 7f1d5143f740 10 client.16143 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.712+0000 7f1d5143f740 10 client.16143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.712+0000 7f1d5143f740 20 client.16143 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.712+0000 7f1d5143f740 1 --2- 172.21.15.43:0/916837791 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aae4eff1f0 0x55aae52fceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.712+0000 7f1d5143f740 4 client.16143 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:34:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.712+0000 7f1d5143f740 1 -- 172.21.15.43:0/916837791 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aae5254710 con 0x55aae4eff1f0 2024-04-25T10:34:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.712+0000 7f1d437fe640 1 --2- 172.21.15.43:0/916837791 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aae4eff1f0 0x55aae52fceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.713+0000 7f1d237fe640 10 client.16143 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.713+0000 7f1d437fe640 1 --2- 172.21.15.43:0/916837791 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aae4eff1f0 0x55aae52fceb0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad44c11b44b361b9 server_cookie=861e922269a03a97 in_seq=0 out_seq=0 2024-04-25T10:34:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.714+0000 7f1d237fe640 1 -- 172.21.15.43:0/916837791 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aae5254710 con 0x55aae4eff1f0 2024-04-25T10:34:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.714+0000 7f1d237fe640 10 client.16143 handle_command_reply: tid=0 2024-04-25T10:34:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.714+0000 7f1d5143f740 1 client.16143 shutdown 2024-04-25T10:34:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.714+0000 7f1d5143f740 1 -- 172.21.15.43:0/916837791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f17550 msgr2=0x55aae52eac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.714+0000 7f1d5143f740 1 --2- 172.21.15.43:0/916837791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f17550 0x55aae52eac80 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009d30 tx=0x7f1d3c009800 comp rx=0 tx=0).stop 2024-04-25T10:34:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.714+0000 7f1d5143f740 1 -- 172.21.15.43:0/916837791 shutdown_connections 2024-04-25T10:34:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.714+0000 7f1d5143f740 1 --2- 172.21.15.43:0/916837791 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aae4eff1f0 0x55aae52fceb0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.714+0000 7f1d5143f740 1 --2- 172.21.15.43:0/916837791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aae4f40410 0x55aae52f0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.714+0000 7f1d5143f740 1 --2- 172.21.15.43:0/916837791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f1f730 0x55aae52eb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.714+0000 7f1d237fe640 0 client.16143 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.714+0000 7f1d5143f740 1 --2- 172.21.15.43:0/916837791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f17550 0x55aae52eac80 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.714+0000 7f1d5143f740 1 -- 172.21.15.43:0/916837791 >> 172.21.15.43:0/916837791 conn(0x55aae4f34560 msgr2=0x55aae52e0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.714+0000 7f1d5143f740 1 -- 172.21.15.43:0/916837791 shutdown_connections 2024-04-25T10:34:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.714+0000 7f1d5143f740 1 -- 172.21.15.43:0/916837791 wait complete. 2024-04-25T10:34:49.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.714+0000 7f1d5143f740 20 client.16143 trim_cache size 0 max 16384 2024-04-25T10:34:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.733+0000 7f1d5143f740 1 Processor -- start 2024-04-25T10:34:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.733+0000 7f1d5143f740 1 -- start start 2024-04-25T10:34:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.733+0000 7f1d5143f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aae4f17550 0x55aae52e50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.733+0000 7f1d5143f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f1f730 0x55aae52e0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.733+0000 7f1d5143f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 0x55aae52e1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.733+0000 7f1d5143f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1d441b44e0 con 0x55aae4f40410 2024-04-25T10:34:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.733+0000 7f1d5143f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1d441b1db0 con 0x55aae4f17550 2024-04-25T10:34:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.733+0000 7f1d5143f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1d441afc00 con 0x55aae4f1f730 2024-04-25T10:34:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.733+0000 7f1d48b23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 0x55aae52e1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.733+0000 7f1d48b23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 0x55aae52e1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47518/0 (socket says 172.21.15.43:47518) 2024-04-25T10:34:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.733+0000 7f1d48b23640 1 -- 172.21.15.43:0/1312779 learned_addr learned my addr 172.21.15.43:0/1312779 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.733+0000 7f1d437fe640 1 --2- 172.21.15.43:0/1312779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f1f730 0x55aae52e0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.733+0000 7f1d48b23640 1 --2- 172.21.15.43:0/1312779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aae4f17550 0x55aae52e50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.733+0000 7f1d48b23640 1 -- 172.21.15.43:0/1312779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f1f730 msgr2=0x55aae52e0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.733+0000 7f1d48b23640 1 --2- 172.21.15.43:0/1312779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f1f730 0x55aae52e0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.733+0000 7f1d48b23640 1 -- 172.21.15.43:0/1312779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aae4f17550 msgr2=0x55aae52e50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.733+0000 7f1d48b23640 1 --2- 172.21.15.43:0/1312779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aae4f17550 0x55aae52e50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.733+0000 7f1d48b23640 1 -- 172.21.15.43:0/1312779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aae52eb930 con 0x55aae4f40410 2024-04-25T10:34:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.733+0000 7f1d48b23640 1 --2- 172.21.15.43:0/1312779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 0x55aae52e1000 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c010d80 tx=0x7f1d3c009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30a93a30a9dca8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.734+0000 7f1d29ffb640 1 -- 172.21.15.43:0/1312779 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c016070 con 0x55aae4f40410 2024-04-25T10:34:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.734+0000 7f1d29ffb640 1 -- 172.21.15.43:0/1312779 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d3c01f030 con 0x55aae4f40410 2024-04-25T10:34:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.734+0000 7f1d29ffb640 1 -- 172.21.15.43:0/1312779 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c0326f0 con 0x55aae4f40410 2024-04-25T10:34:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.734+0000 7f1d5143f740 1 -- 172.21.15.43:0/1312779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 msgr2=0x55aae52e1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.734+0000 7f1d5143f740 1 --2- 172.21.15.43:0/1312779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 0x55aae52e1000 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c010d80 tx=0x7f1d3c009600 comp rx=0 tx=0).stop 2024-04-25T10:34:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.734+0000 7f1d5143f740 1 -- 172.21.15.43:0/1312779 shutdown_connections 2024-04-25T10:34:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.734+0000 7f1d5143f740 1 --2- 172.21.15.43:0/1312779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 0x55aae52e1000 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.734+0000 7f1d5143f740 1 --2- 172.21.15.43:0/1312779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f1f730 0x55aae52e0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.734+0000 7f1d5143f740 1 --2- 172.21.15.43:0/1312779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aae4f17550 0x55aae52e50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.734+0000 7f1d5143f740 1 -- 172.21.15.43:0/1312779 >> 172.21.15.43:0/1312779 conn(0x55aae4f34560 msgr2=0x55aae52e03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.734+0000 7f1d5143f740 1 -- 172.21.15.43:0/1312779 shutdown_connections 2024-04-25T10:34:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.735+0000 7f1d5143f740 1 -- 172.21.15.43:0/1312779 wait complete. 2024-04-25T10:34:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.735+0000 7f1d5143f740 1 -- start start 2024-04-25T10:34:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.735+0000 7f1d5143f740 1 Processor -- start 2024-04-25T10:34:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.735+0000 7f1d5143f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f17550 0x55aae52f4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.735+0000 7f1d5143f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aae4f1f730 0x55aae52f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.735+0000 7f1d5143f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 0x55aae5362fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.735+0000 7f1d5143f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1d441b0270 con 0x55aae4f40410 2024-04-25T10:34:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.735+0000 7f1d5143f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1d441b16e0 con 0x55aae4f1f730 2024-04-25T10:34:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.735+0000 7f1d5143f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1d441b0940 con 0x55aae4f17550 2024-04-25T10:34:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.735+0000 7f1d5143f740 10 client.0 _ll_register_callbacks cb 0x7f1d3c012530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.735+0000 7f1d48b23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 0x55aae5362fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.736+0000 7f1d48b23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 0x55aae5362fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47534/0 (socket says 172.21.15.43:47534) 2024-04-25T10:34:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.736+0000 7f1d48b23640 1 -- 172.21.15.43:0/2241243969 learned_addr learned my addr 172.21.15.43:0/2241243969 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.736+0000 7f1d48b23640 1 --2- 172.21.15.43:0/2241243969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f17550 0x55aae52f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.736+0000 7f1d437fe640 1 --2- 172.21.15.43:0/2241243969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aae4f1f730 0x55aae52f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.736+0000 7f1d48b23640 1 -- 172.21.15.43:0/2241243969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f17550 msgr2=0x55aae52f4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.736+0000 7f1d48b23640 1 --2- 172.21.15.43:0/2241243969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f17550 0x55aae52f4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.736+0000 7f1d48b23640 1 -- 172.21.15.43:0/2241243969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aae4f1f730 msgr2=0x55aae52f4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.736+0000 7f1d48b23640 1 --2- 172.21.15.43:0/2241243969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aae4f1f730 0x55aae52f4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.736+0000 7f1d48b23640 1 -- 172.21.15.43:0/2241243969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aae5363510 con 0x55aae4f40410 2024-04-25T10:34:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.736+0000 7f1d48b23640 1 --2- 172.21.15.43:0/2241243969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 0x55aae5362fd0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c014000 tx=0x7f1d3c012ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d262b30d522de4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.736+0000 7f1d237fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.736+0000 7f1d237fe640 1 -- 172.21.15.43:0/2241243969 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c016040 con 0x55aae4f40410 2024-04-25T10:34:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.736+0000 7f1d5143f740 1 -- 172.21.15.43:0/2241243969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d3c01f030 con 0x55aae4f40410 2024-04-25T10:34:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.736+0000 7f1d237fe640 1 -- 172.21.15.43:0/2241243969 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f1d3c02a5d0 con 0x55aae4f40410 2024-04-25T10:34:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.736+0000 7f1d237fe640 1 -- 172.21.15.43:0/2241243969 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d3c02c0d0 con 0x55aae4f40410 2024-04-25T10:34:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.736+0000 7f1d5143f740 10 client.16149 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:49.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.737+0000 7f1d5143f740 1 -- 172.21.15.43:0/2241243969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aae5364680 con 0x55aae4f40410 2024-04-25T10:34:49.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.737+0000 7f1d237fe640 1 -- 172.21.15.43:0/2241243969 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d3c01f030 con 0x55aae4f40410 2024-04-25T10:34:49.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.737+0000 7f1d237fe640 1 -- 172.21.15.43:0/2241243969 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1d3c03d040 con 0x55aae4f40410 2024-04-25T10:34:49.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.737+0000 7f1d5143f740 10 client.16149 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:49.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.737+0000 7f1d5143f740 10 client.16149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:49.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.737+0000 7f1d5143f740 20 client.16149 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:49.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.737+0000 7f1d5143f740 1 --2- 172.21.15.43:0/2241243969 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aae4ed4630 0x55aae5366c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.737+0000 7f1d5143f740 4 client.16149 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:34:49.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.737+0000 7f1d5143f740 1 -- 172.21.15.43:0/2241243969 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55aae524e580 con 0x55aae4ed4630 2024-04-25T10:34:49.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.737+0000 7f1d437fe640 1 --2- 172.21.15.43:0/2241243969 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aae4ed4630 0x55aae5366c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.738+0000 7f1d237fe640 10 client.16149 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:49.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.738+0000 7f1d437fe640 1 --2- 172.21.15.43:0/2241243969 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aae4ed4630 0x55aae5366c50 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fad597d04e716733 server_cookie=589ccca1da0b6e52 in_seq=0 out_seq=0 2024-04-25T10:34:49.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.739+0000 7f1d237fe640 1 -- 172.21.15.43:0/2241243969 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55aae524e580 con 0x55aae4ed4630 2024-04-25T10:34:49.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.739+0000 7f1d237fe640 10 client.16149 handle_command_reply: tid=0 2024-04-25T10:34:49.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.739+0000 7f1d5143f740 1 client.16149 shutdown 2024-04-25T10:34:49.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.739+0000 7f1d5143f740 1 -- 172.21.15.43:0/2241243969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 msgr2=0x55aae5362fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:49.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.739+0000 7f1d5143f740 1 --2- 172.21.15.43:0/2241243969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 0x55aae5362fd0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c014000 tx=0x7f1d3c012ef0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.739+0000 7f1d5143f740 1 -- 172.21.15.43:0/2241243969 shutdown_connections 2024-04-25T10:34:49.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.739+0000 7f1d5143f740 1 --2- 172.21.15.43:0/2241243969 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aae4ed4630 0x55aae5366c50 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.739+0000 7f1d5143f740 1 --2- 172.21.15.43:0/2241243969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aae4f1f730 0x55aae52f4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.739+0000 7f1d5143f740 1 --2- 172.21.15.43:0/2241243969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae4f40410 0x55aae5362fd0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.739+0000 7f1d237fe640 0 client.16149 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.739+0000 7f1d5143f740 1 --2- 172.21.15.43:0/2241243969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae4f17550 0x55aae52f4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.739+0000 7f1d5143f740 1 -- 172.21.15.43:0/2241243969 >> 172.21.15.43:0/2241243969 conn(0x55aae4f34560 msgr2=0x55aae52ecb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.739+0000 7f1d5143f740 1 -- 172.21.15.43:0/2241243969 shutdown_connections 2024-04-25T10:34:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.740+0000 7f1d5143f740 1 -- 172.21.15.43:0/2241243969 wait complete. 2024-04-25T10:34:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.740+0000 7f1d5143f740 20 client.16149 trim_cache size 0 max 16384 2024-04-25T10:34:49.747 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:34:49.747 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:34:49.747 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "e1a1d8a3-cf36-404a-a554-85d36c0c4a0f", 2024-04-25T10:34:49.748 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:34:49.748 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:34:49.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.740+0000 7f1d4a586640 1 -- 172.21.15.43:0/2171974077 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1d2c0afc60 msgr2=0x7f1d2c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:49.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.740+0000 7f1d4a586640 1 --2- 172.21.15.43:0/2171974077 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1d2c0afc60 0x7f1d2c0b2170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1d340fcc90 tx=0x7f1d34105060 comp rx=0 tx=0).stop 2024-04-25T10:34:49.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.740+0000 7f1d4a586640 1 -- 172.21.15.43:0/2171974077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d44102af0 msgr2=0x7f1d440ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:49.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.740+0000 7f1d4a586640 1 --2- 172.21.15.43:0/2171974077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d44102af0 0x7f1d440ff250 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1d34032fd0 tx=0x7f1d34004270 comp rx=0 tx=0).stop 2024-04-25T10:34:49.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.740+0000 7f1d4a586640 1 -- 172.21.15.43:0/2171974077 shutdown_connections 2024-04-25T10:34:49.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.740+0000 7f1d4a586640 1 --2- 172.21.15.43:0/2171974077 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1d2c0afc60 0x7f1d2c0b2170 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.740+0000 7f1d4a586640 1 --2- 172.21.15.43:0/2171974077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d44108a20 0x7f1d440ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.740+0000 7f1d4a586640 1 --2- 172.21.15.43:0/2171974077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d4410d380 0x7f1d440ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.740+0000 7f1d4a586640 1 --2- 172.21.15.43:0/2171974077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d44102af0 0x7f1d440ff250 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.740+0000 7f1d4a586640 1 -- 172.21.15.43:0/2171974077 >> 172.21.15.43:0/2171974077 conn(0x7f1d44078100 msgr2=0x7f1d44069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:49.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.740+0000 7f1d4a586640 1 -- 172.21.15.43:0/2171974077 shutdown_connections 2024-04-25T10:34:49.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.740+0000 7f1d4a586640 1 -- 172.21.15.43:0/2171974077 wait complete. 2024-04-25T10:34:49.753 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:34:49.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:34:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.866+0000 7fc40b7ee640 1 -- 172.21.15.43:0/597652834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc404108a10 msgr2=0x7fc404102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.866+0000 7fc40b7ee640 1 --2- 172.21.15.43:0/597652834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc404108a10 0x7fc404102610 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc400009900 tx=0x7fc40002a3f0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.866+0000 7fc40b7ee640 1 -- 172.21.15.43:0/597652834 shutdown_connections 2024-04-25T10:34:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.866+0000 7fc40b7ee640 1 --2- 172.21.15.43:0/597652834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4041031c0 0x7fc40410f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.866+0000 7fc40b7ee640 1 --2- 172.21.15.43:0/597652834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc404108eb0 0x7fc404102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.866+0000 7fc40b7ee640 1 --2- 172.21.15.43:0/597652834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc404108a10 0x7fc404102610 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.866+0000 7fc40b7ee640 1 -- 172.21.15.43:0/597652834 >> 172.21.15.43:0/597652834 conn(0x7fc404078110 msgr2=0x7fc404078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.866+0000 7fc40b7ee640 1 -- 172.21.15.43:0/597652834 shutdown_connections 2024-04-25T10:34:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.866+0000 7fc40b7ee640 1 -- 172.21.15.43:0/597652834 wait complete. 2024-04-25T10:34:49.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.866+0000 7fc40b7ee640 1 Processor -- start 2024-04-25T10:34:49.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.866+0000 7fc40b7ee640 1 -- start start 2024-04-25T10:34:49.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.867+0000 7fc40b7ee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4041031c0 0x7fc40419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.867+0000 7fc40b7ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc404108a10 0x7fc40419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.867+0000 7fc40b7ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc404108eb0 0x7fc40419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.867+0000 7fc40b7ee640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc404112c10 con 0x7fc404108eb0 2024-04-25T10:34:49.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.867+0000 7fc40b7ee640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc404112a90 con 0x7fc404108a10 2024-04-25T10:34:49.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.867+0000 7fc40b7ee640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc404112d90 con 0x7fc4041031c0 2024-04-25T10:34:49.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.867+0000 7fc409d8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc404108eb0 0x7fc40419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.867+0000 7fc409d8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc404108eb0 0x7fc40419ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47540/0 (socket says 172.21.15.43:47540) 2024-04-25T10:34:49.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.867+0000 7fc409d8b640 1 -- 172.21.15.43:0/3658092480 learned_addr learned my addr 172.21.15.43:0/3658092480 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:49.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.867+0000 7fc408d89640 1 --2- 172.21.15.43:0/3658092480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc404108a10 0x7fc40419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.867+0000 7fc40958a640 1 --2- 172.21.15.43:0/3658092480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4041031c0 0x7fc40419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.867+0000 7fc409d8b640 1 -- 172.21.15.43:0/3658092480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4041031c0 msgr2=0x7fc40419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:49.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.867+0000 7fc409d8b640 1 --2- 172.21.15.43:0/3658092480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4041031c0 0x7fc40419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.867+0000 7fc409d8b640 1 -- 172.21.15.43:0/3658092480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc404108a10 msgr2=0x7fc40419ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:49.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.867+0000 7fc409d8b640 1 --2- 172.21.15.43:0/3658092480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc404108a10 0x7fc40419ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.867+0000 7fc409d8b640 1 -- 172.21.15.43:0/3658092480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40419f500 con 0x7fc404108eb0 2024-04-25T10:34:49.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.867+0000 7fc408d89640 1 --2- 172.21.15.43:0/3658092480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc404108a10 0x7fc40419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:49.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.867+0000 7fc409d8b640 1 --2- 172.21.15.43:0/3658092480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc404108eb0 0x7fc40419ef40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00cd10 tx=0x7fc3fc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:49.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.868+0000 7fc3fa7fc640 1 -- 172.21.15.43:0/3658092480 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc00f450 con 0x7fc404108eb0 2024-04-25T10:34:49.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.868+0000 7fc3fa7fc640 1 -- 172.21.15.43:0/3658092480 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3fc00faa0 con 0x7fc404108eb0 2024-04-25T10:34:49.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.868+0000 7fc40b7ee640 1 -- 172.21.15.43:0/3658092480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc40419f7f0 con 0x7fc404108eb0 2024-04-25T10:34:49.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.868+0000 7fc3fa7fc640 1 -- 172.21.15.43:0/3658092480 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc018b50 con 0x7fc404108eb0 2024-04-25T10:34:49.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.868+0000 7fc40b7ee640 1 -- 172.21.15.43:0/3658092480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc40419fd40 con 0x7fc404108eb0 2024-04-25T10:34:49.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.870+0000 7fc412772740 1 Processor -- start 2024-04-25T10:34:49.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.870+0000 7fc3fa7fc640 1 -- 172.21.15.43:0/3658092480 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fc3fc00fc50 con 0x7fc404108eb0 2024-04-25T10:34:49.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.870+0000 7fc412772740 1 -- start start 2024-04-25T10:34:49.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.871+0000 7fc412772740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd8d485390 0x55cd8d8258f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.871+0000 7fc412772740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 0x55cd8d825e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.871+0000 7fc412772740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d4646b0 0x55cd8d836d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.871+0000 7fc412772740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cd8d78bd80 con 0x55cd8d45c4d0 2024-04-25T10:34:49.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.871+0000 7fc412772740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cd8d7458b0 con 0x55cd8d485390 2024-04-25T10:34:49.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.871+0000 7fc412772740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cd8d787d40 con 0x55cd8d4646b0 2024-04-25T10:34:49.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.871+0000 7fc40958a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 0x55cd8d825e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.871+0000 7fc40958a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 0x55cd8d825e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47552/0 (socket says 172.21.15.43:47552) 2024-04-25T10:34:49.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.871+0000 7fc40958a640 1 -- 172.21.15.43:0/3004947304 learned_addr learned my addr 172.21.15.43:0/3004947304 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:49.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.871+0000 7fc408d89640 1 --2- 172.21.15.43:0/3004947304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd8d485390 0x55cd8d8258f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.871+0000 7fc408d89640 1 --2- 172.21.15.43:0/3004947304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d4646b0 0x55cd8d836d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.871+0000 7fc3fa7fc640 1 --2- 172.21.15.43:0/3658092480 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc3f00afce0 0x7fc3f00b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.871+0000 7fc40958a640 1 -- 172.21.15.43:0/3004947304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d4646b0 msgr2=0x55cd8d836d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.871+0000 7fc40958a640 1 --2- 172.21.15.43:0/3004947304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d4646b0 0x55cd8d836d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.871+0000 7fc40958a640 1 -- 172.21.15.43:0/3004947304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd8d485390 msgr2=0x55cd8d8258f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.871+0000 7fc40958a640 1 --2- 172.21.15.43:0/3004947304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd8d485390 0x55cd8d8258f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.871+0000 7fc40958a640 1 -- 172.21.15.43:0/3004947304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd8d716a40 con 0x55cd8d45c4d0 2024-04-25T10:34:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.871+0000 7fc409d8b640 1 --2- 172.21.15.43:0/3658092480 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc3f00afce0 0x7fc3f00b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.871+0000 7fc3fa7fc640 1 -- 172.21.15.43:0/3658092480 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc3fc0df410 con 0x7fc404108eb0 2024-04-25T10:34:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.871+0000 7fc40958a640 1 --2- 172.21.15.43:0/3004947304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 0x55cd8d825e30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc400000c00 tx=0x7fc400032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb05b5849bc866a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.872+0000 7fc3ef7fe640 1 -- 172.21.15.43:0/3004947304 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40003f070 con 0x55cd8d45c4d0 2024-04-25T10:34:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.872+0000 7fc3ef7fe640 1 -- 172.21.15.43:0/3004947304 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4000316a0 con 0x55cd8d45c4d0 2024-04-25T10:34:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.872+0000 7fc3ef7fe640 1 -- 172.21.15.43:0/3004947304 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400035070 con 0x55cd8d45c4d0 2024-04-25T10:34:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.873+0000 7fc412772740 1 -- 172.21.15.43:0/3004947304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 msgr2=0x55cd8d825e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.873+0000 7fc409d8b640 1 --2- 172.21.15.43:0/3658092480 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc3f00afce0 0x7fc3f00b21f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc0dcc40 tx=0x7fc3fc074e40 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.873+0000 7fc412772740 1 --2- 172.21.15.43:0/3004947304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 0x55cd8d825e30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc400000c00 tx=0x7fc400032ce0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.873+0000 7fc412772740 1 -- 172.21.15.43:0/3004947304 shutdown_connections 2024-04-25T10:34:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.873+0000 7fc412772740 1 --2- 172.21.15.43:0/3004947304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d4646b0 0x55cd8d836d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.873+0000 7fc412772740 1 --2- 172.21.15.43:0/3004947304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 0x55cd8d825e30 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.873+0000 7fc412772740 1 --2- 172.21.15.43:0/3004947304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd8d485390 0x55cd8d8258f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.873+0000 7fc412772740 1 -- 172.21.15.43:0/3004947304 >> 172.21.15.43:0/3004947304 conn(0x55cd8d4794e0 msgr2=0x55cd8d822210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.873+0000 7fc412772740 1 -- 172.21.15.43:0/3004947304 shutdown_connections 2024-04-25T10:34:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.873+0000 7fc412772740 1 -- 172.21.15.43:0/3004947304 wait complete. 2024-04-25T10:34:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.873+0000 7fc412772740 1 -- start start 2024-04-25T10:34:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.873+0000 7fc412772740 1 Processor -- start 2024-04-25T10:34:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc412772740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 0x55cd8d834980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc412772740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d4646b0 0x55cd8d834ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc412772740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd8d485390 0x55cd8d840900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc412772740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cd8d837390 con 0x55cd8d45c4d0 2024-04-25T10:34:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc412772740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cd8d712d10 con 0x55cd8d485390 2024-04-25T10:34:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc412772740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cd8d837510 con 0x55cd8d4646b0 2024-04-25T10:34:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc412772740 10 client.0 _ll_register_callbacks cb 0x55cd8d73f210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc408d89640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 0x55cd8d834980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc408d89640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 0x55cd8d834980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47568/0 (socket says 172.21.15.43:47568) 2024-04-25T10:34:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc408d89640 1 -- 172.21.15.43:0/2710045428 learned_addr learned my addr 172.21.15.43:0/2710045428 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc40958a640 1 --2- 172.21.15.43:0/2710045428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d4646b0 0x55cd8d834ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc408d89640 1 -- 172.21.15.43:0/2710045428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d4646b0 msgr2=0x55cd8d834ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc408d89640 1 --2- 172.21.15.43:0/2710045428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d4646b0 0x55cd8d834ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc408d89640 1 -- 172.21.15.43:0/2710045428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd8d485390 msgr2=0x55cd8d840900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc408d89640 1 --2- 172.21.15.43:0/2710045428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd8d485390 0x55cd8d840900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc408d89640 1 -- 172.21.15.43:0/2710045428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd8d5677d0 con 0x55cd8d45c4d0 2024-04-25T10:34:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc408d89640 1 --2- 172.21.15.43:0/2710045428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 0x55cd8d834980 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc4041092e0 tx=0x7fc3f40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e4b15f9ce1d862 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc3ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc3ecff9640 1 -- 172.21.15.43:0/2710045428 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f401e430 con 0x55cd8d45c4d0 2024-04-25T10:34:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc3ecff9640 1 -- 172.21.15.43:0/2710045428 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc3f40103d0 con 0x55cd8d45c4d0 2024-04-25T10:34:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.874+0000 7fc412772740 1 -- 172.21.15.43:0/2710045428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4000316a0 con 0x55cd8d45c4d0 2024-04-25T10:34:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.875+0000 7fc3ecff9640 1 -- 172.21.15.43:0/2710045428 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc3f401e5d0 con 0x55cd8d45c4d0 2024-04-25T10:34:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.875+0000 7fc412772740 10 client.16161 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.875+0000 7fc412772740 1 -- 172.21.15.43:0/2710045428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cd8d49d3f0 con 0x55cd8d45c4d0 2024-04-25T10:34:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.875+0000 7fc3ecff9640 1 -- 172.21.15.43:0/2710045428 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4000316a0 con 0x55cd8d45c4d0 2024-04-25T10:34:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.875+0000 7fc3ecff9640 1 -- 172.21.15.43:0/2710045428 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc3f400f040 con 0x55cd8d45c4d0 2024-04-25T10:34:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.875+0000 7fc412772740 10 client.16161 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.875+0000 7fc412772740 10 client.16161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.875+0000 7fc412772740 20 client.16161 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.875+0000 7fc412772740 1 --2- 172.21.15.43:0/2710045428 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55cd8d443f60 0x55cd8d846a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.875+0000 7fc412772740 4 client.16161 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:34:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.875+0000 7fc412772740 1 -- 172.21.15.43:0/2710045428 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd8d799990 con 0x55cd8d443f60 2024-04-25T10:34:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.875+0000 7fc40958a640 1 --2- 172.21.15.43:0/2710045428 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55cd8d443f60 0x55cd8d846a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.876+0000 7fc40958a640 1 --2- 172.21.15.43:0/2710045428 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55cd8d443f60 0x55cd8d846a50 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64d199e2e65712a1 server_cookie=42986852bd71aa41 in_seq=0 out_seq=0 2024-04-25T10:34:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.876+0000 7fc3ecff9640 10 client.16161 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.877+0000 7fc3ecff9640 1 -- 172.21.15.43:0/2710045428 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cd8d799990 con 0x55cd8d443f60 2024-04-25T10:34:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.877+0000 7fc3ecff9640 10 client.16161 handle_command_reply: tid=0 2024-04-25T10:34:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.877+0000 7fc412772740 1 client.16161 shutdown 2024-04-25T10:34:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.877+0000 7fc412772740 1 -- 172.21.15.43:0/2710045428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 msgr2=0x55cd8d834980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.877+0000 7fc412772740 1 --2- 172.21.15.43:0/2710045428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 0x55cd8d834980 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc4041092e0 tx=0x7fc3f40117b0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.877+0000 7fc412772740 1 -- 172.21.15.43:0/2710045428 shutdown_connections 2024-04-25T10:34:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.877+0000 7fc412772740 1 --2- 172.21.15.43:0/2710045428 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55cd8d443f60 0x55cd8d846a50 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.877+0000 7fc412772740 1 --2- 172.21.15.43:0/2710045428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd8d485390 0x55cd8d840900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.877+0000 7fc412772740 1 --2- 172.21.15.43:0/2710045428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d4646b0 0x55cd8d834ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.877+0000 7fc3ecff9640 0 client.16161 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.877+0000 7fc412772740 1 --2- 172.21.15.43:0/2710045428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 0x55cd8d834980 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.877+0000 7fc412772740 1 -- 172.21.15.43:0/2710045428 >> 172.21.15.43:0/2710045428 conn(0x55cd8d4794e0 msgr2=0x55cd8d827780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.877+0000 7fc412772740 1 -- 172.21.15.43:0/2710045428 shutdown_connections 2024-04-25T10:34:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.877+0000 7fc412772740 1 -- 172.21.15.43:0/2710045428 wait complete. 2024-04-25T10:34:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.877+0000 7fc412772740 20 client.16161 trim_cache size 0 max 16384 2024-04-25T10:34:49.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.895+0000 7fc412772740 1 Processor -- start 2024-04-25T10:34:49.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.895+0000 7fc412772740 1 -- start start 2024-04-25T10:34:49.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.895+0000 7fc412772740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd8d45c4d0 0x55cd8d827d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.896+0000 7fc412772740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d4646b0 0x55cd8d828290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.896+0000 7fc412772740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d485390 0x55cd8d83c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.896+0000 7fc412772740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4041afe90 con 0x55cd8d4646b0 2024-04-25T10:34:49.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.896+0000 7fc412772740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc4041ad760 con 0x55cd8d45c4d0 2024-04-25T10:34:49.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.896+0000 7fc412772740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc4041ab5d0 con 0x55cd8d485390 2024-04-25T10:34:49.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.896+0000 7fc40958a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d4646b0 0x55cd8d828290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.896+0000 7fc40958a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d4646b0 0x55cd8d828290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47580/0 (socket says 172.21.15.43:47580) 2024-04-25T10:34:49.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.896+0000 7fc40958a640 1 -- 172.21.15.43:0/3120372972 learned_addr learned my addr 172.21.15.43:0/3120372972 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:49.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.896+0000 7fc408d89640 1 --2- 172.21.15.43:0/3120372972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd8d45c4d0 0x55cd8d827d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.896+0000 7fc40958a640 1 -- 172.21.15.43:0/3120372972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d485390 msgr2=0x55cd8d83c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:49.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.896+0000 7fc408d89640 1 --2- 172.21.15.43:0/3120372972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d485390 0x55cd8d83c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.896+0000 7fc40958a640 1 --2- 172.21.15.43:0/3120372972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d485390 0x55cd8d83c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.896+0000 7fc40958a640 1 -- 172.21.15.43:0/3120372972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd8d45c4d0 msgr2=0x55cd8d827d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.896+0000 7fc40958a640 1 --2- 172.21.15.43:0/3120372972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd8d45c4d0 0x55cd8d827d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.896+0000 7fc40958a640 1 -- 172.21.15.43:0/3120372972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd8d45d0f0 con 0x55cd8d4646b0 2024-04-25T10:34:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.896+0000 7fc408d89640 1 --2- 172.21.15.43:0/3120372972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d485390 0x55cd8d83c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.896+0000 7fc40958a640 1 --2- 172.21.15.43:0/3120372972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d4646b0 0x55cd8d828290 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc400009900 tx=0x7fc400032f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3e7cb677e23366 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.896+0000 7fc3eeffd640 1 -- 172.21.15.43:0/3120372972 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40003f040 con 0x55cd8d4646b0 2024-04-25T10:34:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.896+0000 7fc3eeffd640 1 -- 172.21.15.43:0/3120372972 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc400051070 con 0x55cd8d4646b0 2024-04-25T10:34:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.896+0000 7fc3eeffd640 1 -- 172.21.15.43:0/3120372972 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400035040 con 0x55cd8d4646b0 2024-04-25T10:34:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.897+0000 7fc412772740 1 -- 172.21.15.43:0/3120372972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d4646b0 msgr2=0x55cd8d828290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.897+0000 7fc412772740 1 --2- 172.21.15.43:0/3120372972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d4646b0 0x55cd8d828290 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc400009900 tx=0x7fc400032f10 comp rx=0 tx=0).stop 2024-04-25T10:34:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.897+0000 7fc412772740 1 -- 172.21.15.43:0/3120372972 shutdown_connections 2024-04-25T10:34:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.897+0000 7fc412772740 1 --2- 172.21.15.43:0/3120372972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d485390 0x55cd8d83c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.897+0000 7fc412772740 1 --2- 172.21.15.43:0/3120372972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d4646b0 0x55cd8d828290 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.897+0000 7fc412772740 1 --2- 172.21.15.43:0/3120372972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd8d45c4d0 0x55cd8d827d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.897+0000 7fc412772740 1 -- 172.21.15.43:0/3120372972 >> 172.21.15.43:0/3120372972 conn(0x55cd8d4794e0 msgr2=0x55cd8d827640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.897+0000 7fc412772740 1 -- 172.21.15.43:0/3120372972 shutdown_connections 2024-04-25T10:34:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.897+0000 7fc412772740 1 -- 172.21.15.43:0/3120372972 wait complete. 2024-04-25T10:34:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.897+0000 7fc412772740 1 -- start start 2024-04-25T10:34:49.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.898+0000 7fc412772740 1 Processor -- start 2024-04-25T10:34:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.898+0000 7fc412772740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 0x55cd8d83c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.898+0000 7fc412772740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d4646b0 0x55cd8d83cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.898+0000 7fc412772740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd8d485390 0x55cd8d837c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.898+0000 7fc412772740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4041abc20 con 0x55cd8d45c4d0 2024-04-25T10:34:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.898+0000 7fc412772740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc4041ad090 con 0x55cd8d485390 2024-04-25T10:34:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.898+0000 7fc412772740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc4041ac2f0 con 0x55cd8d4646b0 2024-04-25T10:34:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.898+0000 7fc412772740 10 client.0 _ll_register_callbacks cb 0x7fc3f4013840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.898+0000 7fc408d89640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 0x55cd8d83c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.898+0000 7fc408d89640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 0x55cd8d83c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47586/0 (socket says 172.21.15.43:47586) 2024-04-25T10:34:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.898+0000 7fc408d89640 1 -- 172.21.15.43:0/3312823299 learned_addr learned my addr 172.21.15.43:0/3312823299 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.898+0000 7fc40958a640 1 --2- 172.21.15.43:0/3312823299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d4646b0 0x55cd8d83cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.898+0000 7fc408d89640 1 -- 172.21.15.43:0/3312823299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d4646b0 msgr2=0x55cd8d83cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.898+0000 7fc408d89640 1 --2- 172.21.15.43:0/3312823299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d4646b0 0x55cd8d83cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.898+0000 7fc408d89640 1 -- 172.21.15.43:0/3312823299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd8d485390 msgr2=0x55cd8d837c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.898+0000 7fc408d89640 1 --2- 172.21.15.43:0/3312823299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd8d485390 0x55cd8d837c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.898+0000 7fc408d89640 1 -- 172.21.15.43:0/3312823299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd8d74c4b0 con 0x55cd8d45c4d0 2024-04-25T10:34:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.899+0000 7fc408d89640 1 --2- 172.21.15.43:0/3312823299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 0x55cd8d83c720 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x55cd8d833fd0 tx=0x7fc3f400bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb910a6ff080d4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.899+0000 7fc3ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.899+0000 7fc3ecff9640 1 -- 172.21.15.43:0/3312823299 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f401e430 con 0x55cd8d45c4d0 2024-04-25T10:34:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.899+0000 7fc3ecff9640 1 -- 172.21.15.43:0/3312823299 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc3f4011d70 con 0x55cd8d45c4d0 2024-04-25T10:34:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.899+0000 7fc412772740 1 -- 172.21.15.43:0/3312823299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc400051070 con 0x55cd8d45c4d0 2024-04-25T10:34:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.899+0000 7fc3ecff9640 1 -- 172.21.15.43:0/3312823299 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc3f4010ab0 con 0x55cd8d45c4d0 2024-04-25T10:34:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.899+0000 7fc412772740 10 client.16167 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.899+0000 7fc412772740 1 -- 172.21.15.43:0/3312823299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cd8d8ae490 con 0x55cd8d45c4d0 2024-04-25T10:34:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.899+0000 7fc3ecff9640 1 -- 172.21.15.43:0/3312823299 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc400051070 con 0x55cd8d45c4d0 2024-04-25T10:34:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.899+0000 7fc3ecff9640 1 -- 172.21.15.43:0/3312823299 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc3f4010460 con 0x55cd8d45c4d0 2024-04-25T10:34:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.899+0000 7fc412772740 10 client.16167 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.899+0000 7fc412772740 10 client.16167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.899+0000 7fc412772740 20 client.16167 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.899+0000 7fc412772740 1 --2- 172.21.15.43:0/3312823299 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55cd8d4193a0 0x55cd8d8b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.899+0000 7fc412772740 4 client.16167 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:34:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.899+0000 7fc412772740 1 -- 172.21.15.43:0/3312823299 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cd8d793800 con 0x55cd8d4193a0 2024-04-25T10:34:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.900+0000 7fc40958a640 1 --2- 172.21.15.43:0/3312823299 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55cd8d4193a0 0x55cd8d8b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.900+0000 7fc3ecff9640 10 client.16167 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.900+0000 7fc40958a640 1 --2- 172.21.15.43:0/3312823299 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55cd8d4193a0 0x55cd8d8b0c60 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24f7b077eee713e8 server_cookie=100293b272535b2c in_seq=0 out_seq=0 2024-04-25T10:34:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.900+0000 7fc3ecff9640 1 -- 172.21.15.43:0/3312823299 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cd8d793800 con 0x55cd8d4193a0 2024-04-25T10:34:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.900+0000 7fc3ecff9640 10 client.16167 handle_command_reply: tid=0 2024-04-25T10:34:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.900+0000 7fc412772740 1 client.16167 shutdown 2024-04-25T10:34:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.900+0000 7fc412772740 1 -- 172.21.15.43:0/3312823299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 msgr2=0x55cd8d83c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.900+0000 7fc412772740 1 --2- 172.21.15.43:0/3312823299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 0x55cd8d83c720 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x55cd8d833fd0 tx=0x7fc3f400bb60 comp rx=0 tx=0).stop 2024-04-25T10:34:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.900+0000 7fc412772740 1 -- 172.21.15.43:0/3312823299 shutdown_connections 2024-04-25T10:34:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.900+0000 7fc412772740 1 --2- 172.21.15.43:0/3312823299 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55cd8d4193a0 0x55cd8d8b0c60 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.900+0000 7fc412772740 1 --2- 172.21.15.43:0/3312823299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd8d485390 0x55cd8d837c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.900+0000 7fc3ecff9640 0 client.16167 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.900+0000 7fc412772740 1 --2- 172.21.15.43:0/3312823299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8d4646b0 0x55cd8d83cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.900+0000 7fc412772740 1 --2- 172.21.15.43:0/3312823299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd8d45c4d0 0x55cd8d83c720 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.900+0000 7fc412772740 1 -- 172.21.15.43:0/3312823299 >> 172.21.15.43:0/3312823299 conn(0x55cd8d4794e0 msgr2=0x55cd8d828070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.901+0000 7fc412772740 1 -- 172.21.15.43:0/3312823299 shutdown_connections 2024-04-25T10:34:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.901+0000 7fc412772740 1 -- 172.21.15.43:0/3312823299 wait complete. 2024-04-25T10:34:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.901+0000 7fc412772740 20 client.16167 trim_cache size 0 max 16384 2024-04-25T10:34:49.909 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:34:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:34:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:34:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "e1a1d8a3-cf36-404a-a554-85d36c0c4a0f": { 2024-04-25T10:34:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:34:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "e1a1d8a3-cf36-404a-a554-85d36c0c4a0f", 2024-04-25T10:34:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:34:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:34:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:34:49.909 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:34:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.901+0000 7fc40b7ee640 1 -- 172.21.15.43:0/3658092480 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc3f00afce0 msgr2=0x7fc3f00b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.901+0000 7fc40b7ee640 1 --2- 172.21.15.43:0/3658092480 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc3f00afce0 0x7fc3f00b21f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc0dcc40 tx=0x7fc3fc074e40 comp rx=0 tx=0).stop 2024-04-25T10:34:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.901+0000 7fc40b7ee640 1 -- 172.21.15.43:0/3658092480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc404108eb0 msgr2=0x7fc40419ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.901+0000 7fc40b7ee640 1 --2- 172.21.15.43:0/3658092480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc404108eb0 0x7fc40419ef40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00cd10 tx=0x7fc3fc00a620 comp rx=0 tx=0).stop 2024-04-25T10:34:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.901+0000 7fc40b7ee640 1 -- 172.21.15.43:0/3658092480 shutdown_connections 2024-04-25T10:34:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.901+0000 7fc40b7ee640 1 --2- 172.21.15.43:0/3658092480 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc3f00afce0 0x7fc3f00b21f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.901+0000 7fc40b7ee640 1 --2- 172.21.15.43:0/3658092480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc404108a10 0x7fc40419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.901+0000 7fc40b7ee640 1 --2- 172.21.15.43:0/3658092480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc404108eb0 0x7fc40419ef40 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.901+0000 7fc40b7ee640 1 --2- 172.21.15.43:0/3658092480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4041031c0 0x7fc40419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.901+0000 7fc40b7ee640 1 -- 172.21.15.43:0/3658092480 >> 172.21.15.43:0/3658092480 conn(0x7fc404078110 msgr2=0x7fc404074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.901+0000 7fc40b7ee640 1 -- 172.21.15.43:0/3658092480 shutdown_connections 2024-04-25T10:34:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:34:49.901+0000 7fc40b7ee640 1 -- 172.21.15.43:0/3658092480 wait complete. 2024-04-25T10:34:49.915 INFO:tasks.cephfs.filesystem:scrub status for tag:e1a1d8a3-cf36-404a-a554-85d36c0c4a0f - {'path': '/', 'tag': 'e1a1d8a3-cf36-404a-a554-85d36c0c4a0f', 'options': 'recursive,force'} 2024-04-25T10:34:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:50 smithi043 ceph-mon[29478]: pgmap v72: 97 pgs: 97 active+clean; 2.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 17 op/s 2024-04-25T10:34:50.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:50 smithi077 ceph-mon[32772]: pgmap v72: 97 pgs: 97 active+clean; 2.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 17 op/s 2024-04-25T10:34:50.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:50 smithi164 ceph-mon[32713]: pgmap v72: 97 pgs: 97 active+clean; 2.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 17 op/s 2024-04-25T10:34:51.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:51 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:34:51.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:51 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:34:51.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:51 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:34:51.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:51 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:34:51.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:51 smithi043 ceph-mon[29478]: pgmap v73: 97 pgs: 97 active+clean; 2.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-25T10:34:51.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:51 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:34:51.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:51 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:34:51.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:51 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:34:51.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:51 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:34:51.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:51 smithi077 ceph-mon[32772]: pgmap v73: 97 pgs: 97 active+clean; 2.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-25T10:34:51.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:51 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:34:51.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:51 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:34:51.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:51 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:34:51.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:51 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:34:51.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:51 smithi164 ceph-mon[32713]: pgmap v73: 97 pgs: 97 active+clean; 2.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-25T10:34:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:53 smithi043 ceph-mon[29478]: pgmap v74: 97 pgs: 97 active+clean; 2.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-25T10:34:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:53 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:34:54.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:53 smithi077 ceph-mon[32772]: pgmap v74: 97 pgs: 97 active+clean; 2.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-25T10:34:54.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:53 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:34:54.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:53 smithi164 ceph-mon[32713]: pgmap v74: 97 pgs: 97 active+clean; 2.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-25T10:34:54.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:53 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:34:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.293+0000 7fe2b4496640 1 -- 172.21.15.43:0/3900494863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2ac10a010 msgr2=0x7fe2ac103c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.293+0000 7fe2b4496640 1 --2- 172.21.15.43:0/3900494863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2ac10a010 0x7fe2ac103c50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009900 tx=0x7fe2a002a410 comp rx=0 tx=0).stop 2024-04-25T10:34:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.293+0000 7fe2b4496640 1 -- 172.21.15.43:0/3900494863 shutdown_connections 2024-04-25T10:34:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.293+0000 7fe2b4496640 1 --2- 172.21.15.43:0/3900494863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2ac1042c0 0x7fe2ac110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.293+0000 7fe2b4496640 1 --2- 172.21.15.43:0/3900494863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2ac10a010 0x7fe2ac103c50 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.293+0000 7fe2b4496640 1 --2- 172.21.15.43:0/3900494863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2ac109b70 0x7fe2ac103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.293+0000 7fe2b4496640 1 -- 172.21.15.43:0/3900494863 >> 172.21.15.43:0/3900494863 conn(0x7fe2ac0ff210 msgr2=0x7fe2ac101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.293+0000 7fe2b4496640 1 -- 172.21.15.43:0/3900494863 shutdown_connections 2024-04-25T10:34:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.293+0000 7fe2b4496640 1 -- 172.21.15.43:0/3900494863 wait complete. 2024-04-25T10:34:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.294+0000 7fe2b4496640 1 Processor -- start 2024-04-25T10:34:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.294+0000 7fe2b4496640 1 -- start start 2024-04-25T10:34:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.294+0000 7fe2b4496640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2ac1042c0 0x7fe2ac19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.294+0000 7fe2b4496640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2ac109b70 0x7fe2ac19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.294+0000 7fe2b4496640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2ac10a010 0x7fe2ac19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.294+0000 7fe2b4496640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac113d60 con 0x7fe2ac10a010 2024-04-25T10:34:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.294+0000 7fe2b4496640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac113be0 con 0x7fe2ac109b70 2024-04-25T10:34:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.294+0000 7fe2b4496640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac113ee0 con 0x7fe2ac1042c0 2024-04-25T10:34:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.294+0000 7fe2b2a33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2ac10a010 0x7fe2ac19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.294+0000 7fe2b2a33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2ac10a010 0x7fe2ac19f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47602/0 (socket says 172.21.15.43:47602) 2024-04-25T10:34:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.294+0000 7fe2b2a33640 1 -- 172.21.15.43:0/2932450010 learned_addr learned my addr 172.21.15.43:0/2932450010 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.294+0000 7fe2b2232640 1 --2- 172.21.15.43:0/2932450010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2ac1042c0 0x7fe2ac19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.295+0000 7fe2b1a31640 1 --2- 172.21.15.43:0/2932450010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2ac109b70 0x7fe2ac19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.295+0000 7fe2b2a33640 1 -- 172.21.15.43:0/2932450010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2ac1042c0 msgr2=0x7fe2ac19a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.295+0000 7fe2b2a33640 1 --2- 172.21.15.43:0/2932450010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2ac1042c0 0x7fe2ac19a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.295+0000 7fe2b2a33640 1 -- 172.21.15.43:0/2932450010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2ac109b70 msgr2=0x7fe2ac19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.295+0000 7fe2b2a33640 1 --2- 172.21.15.43:0/2932450010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2ac109b70 0x7fe2ac19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.295+0000 7fe2b2a33640 1 -- 172.21.15.43:0/2932450010 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac19f640 con 0x7fe2ac10a010 2024-04-25T10:34:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.295+0000 7fe2b2a33640 1 --2- 172.21.15.43:0/2932450010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2ac10a010 0x7fe2ac19f000 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00e970 tx=0x7fe29c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.295+0000 7fe2ab7fe640 1 -- 172.21.15.43:0/2932450010 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c00ccc0 con 0x7fe2ac10a010 2024-04-25T10:34:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.295+0000 7fe2ab7fe640 1 -- 172.21.15.43:0/2932450010 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe29c00ce60 con 0x7fe2ac10a010 2024-04-25T10:34:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.295+0000 7fe2ab7fe640 1 -- 172.21.15.43:0/2932450010 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c010430 con 0x7fe2ac10a010 2024-04-25T10:34:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.295+0000 7fe2b4496640 1 -- 172.21.15.43:0/2932450010 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ac19f930 con 0x7fe2ac10a010 2024-04-25T10:34:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.295+0000 7fe2b4496640 1 -- 172.21.15.43:0/2932450010 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ac1a72b0 con 0x7fe2ac10a010 2024-04-25T10:34:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.298+0000 7fe2bb429740 1 Processor -- start 2024-04-25T10:34:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.298+0000 7fe2ab7fe640 1 -- 172.21.15.43:0/2932450010 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fe29c004ff0 con 0x7fe2ac10a010 2024-04-25T10:34:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.299+0000 7fe2bb429740 1 -- start start 2024-04-25T10:34:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.299+0000 7fe2bb429740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20b8ed1b0 0x55e20bc8caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.299+0000 7fe2bb429740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20b8c42f0 0x55e20bc8d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.299+0000 7fe2bb429740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8cc4d0 0x55e20bc8d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.299+0000 7fe2bb429740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e20bbf3460 con 0x55e20b8cc4d0 2024-04-25T10:34:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.299+0000 7fe2bb429740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e20bbad590 con 0x55e20b8c42f0 2024-04-25T10:34:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.299+0000 7fe2bb429740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e20bbef420 con 0x55e20b8ed1b0 2024-04-25T10:34:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.299+0000 7fe2b1a31640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8cc4d0 0x55e20bc8d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.299+0000 7fe2b2232640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20b8c42f0 0x55e20bc8d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.299+0000 7fe2b1a31640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8cc4d0 0x55e20bc8d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47612/0 (socket says 172.21.15.43:47612) 2024-04-25T10:34:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.299+0000 7fe2b1a31640 1 -- 172.21.15.43:0/108570422 learned_addr learned my addr 172.21.15.43:0/108570422 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.299+0000 7fe2b1a31640 1 --2- 172.21.15.43:0/108570422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20b8ed1b0 0x55e20bc8caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.299+0000 7fe2ab7fe640 1 --2- 172.21.15.43:0/2932450010 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe2940afcd0 0x7fe2940b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.299+0000 7fe2b2232640 1 -- 172.21.15.43:0/108570422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20b8ed1b0 msgr2=0x55e20bc8caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.299+0000 7fe2b2232640 1 --2- 172.21.15.43:0/108570422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20b8ed1b0 0x55e20bc8caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.299+0000 7fe2b2232640 1 -- 172.21.15.43:0/108570422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8cc4d0 msgr2=0x55e20bc8d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.299+0000 7fe2b2232640 1 --2- 172.21.15.43:0/108570422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8cc4d0 0x55e20bc8d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.299+0000 7fe2b2232640 1 -- 172.21.15.43:0/108570422 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e20bbba0d0 con 0x55e20b8c42f0 2024-04-25T10:34:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.300+0000 7fe2b1a31640 1 --2- 172.21.15.43:0/108570422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8cc4d0 0x55e20bc8d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.300+0000 7fe2b2a33640 1 --2- 172.21.15.43:0/2932450010 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe2940afcd0 0x7fe2940b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.300+0000 7fe2ab7fe640 1 -- 172.21.15.43:0/2932450010 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fe29c0df3a0 con 0x7fe2ac10a010 2024-04-25T10:34:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.300+0000 7fe2b2232640 1 --2- 172.21.15.43:0/108570422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20b8c42f0 0x55e20bc8d030 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4005fd0 tx=0x7fe2a40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78ba451b3736d088 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.300+0000 7fe2a8bfe640 1 -- 172.21.15.43:0/108570422 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4004d20 con 0x55e20b8c42f0 2024-04-25T10:34:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.300+0000 7fe2b2a33640 1 --2- 172.21.15.43:0/2932450010 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe2940afcd0 0x7fe2940b21e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0dcbd0 tx=0x7fe29c0049b0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.300+0000 7fe2a8bfe640 1 -- 172.21.15.43:0/108570422 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a4004050 con 0x55e20b8c42f0 2024-04-25T10:34:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.300+0000 7fe2a8bfe640 1 -- 172.21.15.43:0/108570422 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4005260 con 0x55e20b8c42f0 2024-04-25T10:34:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.301+0000 7fe2bb429740 1 -- 172.21.15.43:0/108570422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20b8c42f0 msgr2=0x55e20bc8d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.301+0000 7fe2bb429740 1 --2- 172.21.15.43:0/108570422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20b8c42f0 0x55e20bc8d030 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4005fd0 tx=0x7fe2a40118d0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.301+0000 7fe2bb429740 1 -- 172.21.15.43:0/108570422 shutdown_connections 2024-04-25T10:34:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.301+0000 7fe2bb429740 1 --2- 172.21.15.43:0/108570422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20b8c42f0 0x55e20bc8d030 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.301+0000 7fe2bb429740 1 --2- 172.21.15.43:0/108570422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8cc4d0 0x55e20bc8d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.301+0000 7fe2bb429740 1 --2- 172.21.15.43:0/108570422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20b8ed1b0 0x55e20bc8caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.301+0000 7fe2bb429740 1 -- 172.21.15.43:0/108570422 >> 172.21.15.43:0/108570422 conn(0x55e20b8e1300 msgr2=0x55e20bc892a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.301+0000 7fe2bb429740 1 -- 172.21.15.43:0/108570422 shutdown_connections 2024-04-25T10:34:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.301+0000 7fe2bb429740 1 -- 172.21.15.43:0/108570422 wait complete. 2024-04-25T10:34:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.301+0000 7fe2bb429740 1 -- start start 2024-04-25T10:34:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.302+0000 7fe2bb429740 1 Processor -- start 2024-04-25T10:34:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.302+0000 7fe2bb429740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8c42f0 0x55e20bc97180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.302+0000 7fe2bb429740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20b8cc4d0 0x55e20bc976c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.302+0000 7fe2bb429740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20b8ed1b0 0x55e20bc9d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.302+0000 7fe2bb429740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e20bc99c20 con 0x55e20b8c42f0 2024-04-25T10:34:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.302+0000 7fe2bb429740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e20bb7ab60 con 0x55e20b8cc4d0 2024-04-25T10:34:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.302+0000 7fe2bb429740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e20bc99da0 con 0x55e20b8ed1b0 2024-04-25T10:34:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.302+0000 7fe2bb429740 10 client.0 _ll_register_callbacks cb 0x55e20bb9f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.302+0000 7fe2b1a31640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8c42f0 0x55e20bc97180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.302+0000 7fe2b1a31640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8c42f0 0x55e20bc97180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47626/0 (socket says 172.21.15.43:47626) 2024-04-25T10:34:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.302+0000 7fe2b1a31640 1 -- 172.21.15.43:0/4292193322 learned_addr learned my addr 172.21.15.43:0/4292193322 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.302+0000 7fe2b2232640 1 --2- 172.21.15.43:0/4292193322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20b8cc4d0 0x55e20bc976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.302+0000 7fe2b1a31640 1 --2- 172.21.15.43:0/4292193322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20b8ed1b0 0x55e20bc9d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.302+0000 7fe2b1a31640 1 -- 172.21.15.43:0/4292193322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20b8ed1b0 msgr2=0x55e20bc9d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.302+0000 7fe2b1a31640 1 --2- 172.21.15.43:0/4292193322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20b8ed1b0 0x55e20bc9d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.302+0000 7fe2b1a31640 1 -- 172.21.15.43:0/4292193322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20b8cc4d0 msgr2=0x55e20bc976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.302+0000 7fe2b1a31640 1 --2- 172.21.15.43:0/4292193322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20b8cc4d0 0x55e20bc976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.302+0000 7fe2b1a31640 1 -- 172.21.15.43:0/4292193322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e20b905210 con 0x55e20b8c42f0 2024-04-25T10:34:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.303+0000 7fe2b1a31640 1 --2- 172.21.15.43:0/4292193322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8c42f0 0x55e20bc97180 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0006010 tx=0x7fe2a002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91c9c5a264b5353 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.303+0000 7fe291ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.303+0000 7fe291ffb640 1 -- 172.21.15.43:0/4292193322 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0035070 con 0x55e20b8c42f0 2024-04-25T10:34:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.303+0000 7fe291ffb640 1 -- 172.21.15.43:0/4292193322 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fe2a0004230 con 0x55e20b8c42f0 2024-04-25T10:34:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.303+0000 7fe2bb429740 1 -- 172.21.15.43:0/4292193322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2a4004050 con 0x55e20b8c42f0 2024-04-25T10:34:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.303+0000 7fe291ffb640 1 -- 172.21.15.43:0/4292193322 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe2a0051050 con 0x55e20b8c42f0 2024-04-25T10:34:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.303+0000 7fe2bb429740 10 client.16179 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.303+0000 7fe2bb429740 1 -- 172.21.15.43:0/4292193322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e20b9d1620 con 0x55e20b8c42f0 2024-04-25T10:34:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.303+0000 7fe291ffb640 1 -- 172.21.15.43:0/4292193322 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a4004050 con 0x55e20b8c42f0 2024-04-25T10:34:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.303+0000 7fe291ffb640 1 -- 172.21.15.43:0/4292193322 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe2a00049b0 con 0x55e20b8c42f0 2024-04-25T10:34:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.304+0000 7fe2bb429740 10 client.16179 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.304+0000 7fe2bb429740 10 client.16179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.304+0000 7fe2bb429740 20 client.16179 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.304+0000 7fe2bb429740 1 --2- 172.21.15.43:0/4292193322 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e20b8abd80 0x55e20bca9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.304+0000 7fe2bb429740 4 client.16179 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:34:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.304+0000 7fe2bb429740 1 -- 172.21.15.43:0/4292193322 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e20bc01070 con 0x55e20b8abd80 2024-04-25T10:34:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.304+0000 7fe2b2232640 1 --2- 172.21.15.43:0/4292193322 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e20b8abd80 0x55e20bca9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.304+0000 7fe291ffb640 10 client.16179 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.304+0000 7fe2b2232640 1 --2- 172.21.15.43:0/4292193322 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e20b8abd80 0x55e20bca9640 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f184c19c3f76d69d server_cookie=4e795ecc4a9963b5 in_seq=0 out_seq=0 2024-04-25T10:34:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.305+0000 7fe291ffb640 1 -- 172.21.15.43:0/4292193322 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e20bc01070 con 0x55e20b8abd80 2024-04-25T10:34:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.305+0000 7fe291ffb640 10 client.16179 handle_command_reply: tid=0 2024-04-25T10:34:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.305+0000 7fe2bb429740 1 client.16179 shutdown 2024-04-25T10:34:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.305+0000 7fe2bb429740 1 -- 172.21.15.43:0/4292193322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8c42f0 msgr2=0x55e20bc97180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.305+0000 7fe2bb429740 1 --2- 172.21.15.43:0/4292193322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8c42f0 0x55e20bc97180 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0006010 tx=0x7fe2a002a410 comp rx=0 tx=0).stop 2024-04-25T10:34:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.305+0000 7fe2bb429740 1 -- 172.21.15.43:0/4292193322 shutdown_connections 2024-04-25T10:34:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.305+0000 7fe2bb429740 1 --2- 172.21.15.43:0/4292193322 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e20b8abd80 0x55e20bca9640 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.305+0000 7fe2bb429740 1 --2- 172.21.15.43:0/4292193322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20b8cc4d0 0x55e20bc976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.305+0000 7fe2bb429740 1 --2- 172.21.15.43:0/4292193322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20b8ed1b0 0x55e20bc9d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.305+0000 7fe291ffb640 0 client.16179 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.305+0000 7fe2bb429740 1 --2- 172.21.15.43:0/4292193322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8c42f0 0x55e20bc97180 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.305+0000 7fe2bb429740 1 -- 172.21.15.43:0/4292193322 >> 172.21.15.43:0/4292193322 conn(0x55e20b8e1300 msgr2=0x55e20bc8c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.305+0000 7fe2bb429740 1 -- 172.21.15.43:0/4292193322 shutdown_connections 2024-04-25T10:34:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.305+0000 7fe2bb429740 1 -- 172.21.15.43:0/4292193322 wait complete. 2024-04-25T10:34:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.306+0000 7fe2bb429740 20 client.16179 trim_cache size 0 max 16384 2024-04-25T10:34:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.321+0000 7fe2bb429740 1 Processor -- start 2024-04-25T10:34:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.321+0000 7fe2bb429740 1 -- start start 2024-04-25T10:34:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.321+0000 7fe2bb429740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8c42f0 0x55e20bc9d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.321+0000 7fe2bb429740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20b8cc4d0 0x55e20bc8cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.321+0000 7fe2bb429740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20b8ed1b0 0x55e20bcaee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.321+0000 7fe2bb429740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac1b2210 con 0x55e20b8c42f0 2024-04-25T10:34:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.321+0000 7fe2bb429740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac1afae0 con 0x55e20b8ed1b0 2024-04-25T10:34:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.321+0000 7fe2bb429740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac1ad930 con 0x55e20b8cc4d0 2024-04-25T10:34:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.321+0000 7fe2b1a31640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8c42f0 0x55e20bc9d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.321+0000 7fe2b1a31640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8c42f0 0x55e20bc9d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47634/0 (socket says 172.21.15.43:47634) 2024-04-25T10:34:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.321+0000 7fe2b1a31640 1 -- 172.21.15.43:0/1590129348 learned_addr learned my addr 172.21.15.43:0/1590129348 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.321+0000 7fe2b1a31640 1 -- 172.21.15.43:0/1590129348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20b8cc4d0 msgr2=0x55e20bc8cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.321+0000 7fe2b1a31640 1 --2- 172.21.15.43:0/1590129348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20b8cc4d0 0x55e20bc8cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.322+0000 7fe2b1a31640 1 -- 172.21.15.43:0/1590129348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20b8ed1b0 msgr2=0x55e20bcaee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.322+0000 7fe2b1a31640 1 --2- 172.21.15.43:0/1590129348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20b8ed1b0 0x55e20bcaee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.322+0000 7fe2b1a31640 1 -- 172.21.15.43:0/1590129348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e20bc8d8a0 con 0x55e20b8c42f0 2024-04-25T10:34:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.322+0000 7fe2b1a31640 1 --2- 172.21.15.43:0/1590129348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8c42f0 0x55e20bc9d010 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0001eb0 tx=0x7fe2a0004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4f844fdfb4a199 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.322+0000 7fe293fff640 1 -- 172.21.15.43:0/1590129348 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0035030 con 0x55e20b8c42f0 2024-04-25T10:34:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.322+0000 7fe293fff640 1 -- 172.21.15.43:0/1590129348 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a0004b70 con 0x55e20b8c42f0 2024-04-25T10:34:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.322+0000 7fe293fff640 1 -- 172.21.15.43:0/1590129348 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0041070 con 0x55e20b8c42f0 2024-04-25T10:34:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.323+0000 7fe2bb429740 1 -- 172.21.15.43:0/1590129348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8c42f0 msgr2=0x55e20bc9d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.323+0000 7fe2bb429740 1 --2- 172.21.15.43:0/1590129348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8c42f0 0x55e20bc9d010 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0001eb0 tx=0x7fe2a0004f50 comp rx=0 tx=0).stop 2024-04-25T10:34:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.323+0000 7fe2bb429740 1 -- 172.21.15.43:0/1590129348 shutdown_connections 2024-04-25T10:34:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.323+0000 7fe2bb429740 1 --2- 172.21.15.43:0/1590129348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20b8ed1b0 0x55e20bcaee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.323+0000 7fe2bb429740 1 --2- 172.21.15.43:0/1590129348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20b8cc4d0 0x55e20bc8cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.323+0000 7fe2bb429740 1 --2- 172.21.15.43:0/1590129348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8c42f0 0x55e20bc9d010 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.323+0000 7fe2bb429740 1 -- 172.21.15.43:0/1590129348 >> 172.21.15.43:0/1590129348 conn(0x55e20b8e1300 msgr2=0x55e20bc8c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.323+0000 7fe2bb429740 1 -- 172.21.15.43:0/1590129348 shutdown_connections 2024-04-25T10:34:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.323+0000 7fe2bb429740 1 -- 172.21.15.43:0/1590129348 wait complete. 2024-04-25T10:34:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.323+0000 7fe2bb429740 1 -- start start 2024-04-25T10:34:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.323+0000 7fe2bb429740 1 Processor -- start 2024-04-25T10:34:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.323+0000 7fe2bb429740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20b8c42f0 0x55e20bca35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.323+0000 7fe2bb429740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20b8cc4d0 0x55e20bca0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.323+0000 7fe2bb429740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8ed1b0 0x55e20bca0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.323+0000 7fe2bb429740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac1adfa0 con 0x55e20b8ed1b0 2024-04-25T10:34:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.323+0000 7fe2bb429740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac1af410 con 0x55e20b8c42f0 2024-04-25T10:34:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.323+0000 7fe2bb429740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac1ae670 con 0x55e20b8cc4d0 2024-04-25T10:34:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.324+0000 7fe2bb429740 10 client.0 _ll_register_callbacks cb 0x7fe2a0032e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:34:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.324+0000 7fe2b1a31640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8ed1b0 0x55e20bca0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.324+0000 7fe2b1a31640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8ed1b0 0x55e20bca0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47638/0 (socket says 172.21.15.43:47638) 2024-04-25T10:34:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.324+0000 7fe2b1a31640 1 -- 172.21.15.43:0/1752577460 learned_addr learned my addr 172.21.15.43:0/1752577460 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:34:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.324+0000 7fe2b1a31640 1 --2- 172.21.15.43:0/1752577460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20b8c42f0 0x55e20bca35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.324+0000 7fe2b2232640 1 --2- 172.21.15.43:0/1752577460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20b8cc4d0 0x55e20bca0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.324+0000 7fe2b1a31640 1 -- 172.21.15.43:0/1752577460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20b8cc4d0 msgr2=0x55e20bca0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.324+0000 7fe2b1a31640 1 --2- 172.21.15.43:0/1752577460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20b8cc4d0 0x55e20bca0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.324+0000 7fe2b1a31640 1 -- 172.21.15.43:0/1752577460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20b8c42f0 msgr2=0x55e20bca35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.324+0000 7fe2b1a31640 1 --2- 172.21.15.43:0/1752577460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20b8c42f0 0x55e20bca35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.324+0000 7fe2b1a31640 1 -- 172.21.15.43:0/1752577460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e20bca15b0 con 0x55e20b8ed1b0 2024-04-25T10:34:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.324+0000 7fe2b1a31640 1 --2- 172.21.15.43:0/1752577460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8ed1b0 0x55e20bca0ec0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00038f0 tx=0x7fe2a0032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e945f6c83b9e4f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.324+0000 7fe291ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:34:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.324+0000 7fe291ffb640 1 -- 172.21.15.43:0/1752577460 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0041070 con 0x55e20b8ed1b0 2024-04-25T10:34:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.324+0000 7fe291ffb640 1 -- 172.21.15.43:0/1752577460 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fe2a0005000 con 0x55e20b8ed1b0 2024-04-25T10:34:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.324+0000 7fe2bb429740 1 -- 172.21.15.43:0/1752577460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2a0004b70 con 0x55e20b8ed1b0 2024-04-25T10:34:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.324+0000 7fe291ffb640 1 -- 172.21.15.43:0/1752577460 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe2a0035040 con 0x55e20b8ed1b0 2024-04-25T10:34:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.325+0000 7fe2bb429740 10 client.16185 fetch_fsmap learned FSMap version 26 2024-04-25T10:34:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.325+0000 7fe2bb429740 1 -- 172.21.15.43:0/1752577460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e20bd11890 con 0x55e20b8ed1b0 2024-04-25T10:34:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.325+0000 7fe291ffb640 1 -- 172.21.15.43:0/1752577460 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a0004b70 con 0x55e20b8ed1b0 2024-04-25T10:34:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.325+0000 7fe291ffb640 1 -- 172.21.15.43:0/1752577460 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe2a003d8e0 con 0x55e20b8ed1b0 2024-04-25T10:34:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.325+0000 7fe2bb429740 10 client.16185 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:34:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.325+0000 7fe2bb429740 10 client.16185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:34:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.325+0000 7fe2bb429740 20 client.16185 populate_metadata read hostname 'smithi043' 2024-04-25T10:34:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.325+0000 7fe2bb429740 1 --2- 172.21.15.43:0/1752577460 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e20b8811c0 0x55e20bd13f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.325+0000 7fe2bb429740 4 client.16185 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:34:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.325+0000 7fe2bb429740 1 -- 172.21.15.43:0/1752577460 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e20bbfaee0 con 0x55e20b8811c0 2024-04-25T10:34:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.325+0000 7fe2b2232640 1 --2- 172.21.15.43:0/1752577460 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e20b8811c0 0x55e20bd13f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.326+0000 7fe291ffb640 10 client.16185 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.326+0000 7fe2b2232640 1 --2- 172.21.15.43:0/1752577460 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e20b8811c0 0x55e20bd13f80 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b34933eec8bf152e server_cookie=417d3da104be7d50 in_seq=0 out_seq=0 2024-04-25T10:34:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.341+0000 7fe291ffb640 1 -- 172.21.15.43:0/1752577460 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e20bbfaee0 con 0x55e20b8811c0 2024-04-25T10:34:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.341+0000 7fe291ffb640 10 client.16185 handle_command_reply: tid=0 2024-04-25T10:34:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.341+0000 7fe2bb429740 1 client.16185 shutdown 2024-04-25T10:34:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.341+0000 7fe2bb429740 1 -- 172.21.15.43:0/1752577460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8ed1b0 msgr2=0x55e20bca0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.341+0000 7fe2bb429740 1 --2- 172.21.15.43:0/1752577460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8ed1b0 0x55e20bca0ec0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00038f0 tx=0x7fe2a0032c90 comp rx=0 tx=0).stop 2024-04-25T10:34:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.341+0000 7fe2bb429740 1 -- 172.21.15.43:0/1752577460 shutdown_connections 2024-04-25T10:34:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.341+0000 7fe2bb429740 1 --2- 172.21.15.43:0/1752577460 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e20b8811c0 0x55e20bd13f80 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.342+0000 7fe2bb429740 1 --2- 172.21.15.43:0/1752577460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e20b8ed1b0 0x55e20bca0ec0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.342+0000 7fe291ffb640 0 client.16185 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:34:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.342+0000 7fe2bb429740 1 --2- 172.21.15.43:0/1752577460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20b8cc4d0 0x55e20bca0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.342+0000 7fe2bb429740 1 --2- 172.21.15.43:0/1752577460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20b8c42f0 0x55e20bca35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.342+0000 7fe2bb429740 1 -- 172.21.15.43:0/1752577460 >> 172.21.15.43:0/1752577460 conn(0x55e20b8e1300 msgr2=0x55e20bc98620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.342+0000 7fe2bb429740 1 -- 172.21.15.43:0/1752577460 shutdown_connections 2024-04-25T10:34:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.342+0000 7fe2bb429740 1 -- 172.21.15.43:0/1752577460 wait complete. 2024-04-25T10:34:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.342+0000 7fe2bb429740 20 client.16185 trim_cache size 0 max 16384 2024-04-25T10:34:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:34:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:34:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:34:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:34:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.342+0000 7fe2b4496640 1 -- 172.21.15.43:0/2932450010 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe2940afcd0 msgr2=0x7fe2940b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.342+0000 7fe2b4496640 1 --2- 172.21.15.43:0/2932450010 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe2940afcd0 0x7fe2940b21e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0dcbd0 tx=0x7fe29c0049b0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.342+0000 7fe2b4496640 1 -- 172.21.15.43:0/2932450010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2ac10a010 msgr2=0x7fe2ac19f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.342+0000 7fe2b4496640 1 --2- 172.21.15.43:0/2932450010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2ac10a010 0x7fe2ac19f000 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00e970 tx=0x7fe29c00ed70 comp rx=0 tx=0).stop 2024-04-25T10:34:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.342+0000 7fe2b4496640 1 -- 172.21.15.43:0/2932450010 shutdown_connections 2024-04-25T10:34:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.342+0000 7fe2b4496640 1 --2- 172.21.15.43:0/2932450010 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe2940afcd0 0x7fe2940b21e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.342+0000 7fe2b4496640 1 --2- 172.21.15.43:0/2932450010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2ac109b70 0x7fe2ac19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.342+0000 7fe2b4496640 1 --2- 172.21.15.43:0/2932450010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2ac10a010 0x7fe2ac19f000 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.342+0000 7fe2b4496640 1 --2- 172.21.15.43:0/2932450010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2ac1042c0 0x7fe2ac19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.342+0000 7fe2b4496640 1 -- 172.21.15.43:0/2932450010 >> 172.21.15.43:0/2932450010 conn(0x7fe2ac0ff210 msgr2=0x7fe2ac10fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.342+0000 7fe2b4496640 1 -- 172.21.15.43:0/2932450010 shutdown_connections 2024-04-25T10:34:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:34:55.342+0000 7fe2b4496640 1 -- 172.21.15.43:0/2932450010 wait complete. 2024-04-25T10:34:55.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:55 smithi043 ceph-mon[29478]: pgmap v75: 97 pgs: 97 active+clean; 2.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 18 op/s 2024-04-25T10:34:56.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:55 smithi077 ceph-mon[32772]: pgmap v75: 97 pgs: 97 active+clean; 2.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 18 op/s 2024-04-25T10:34:56.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:55 smithi164 ceph-mon[32713]: pgmap v75: 97 pgs: 97 active+clean; 2.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 18 op/s 2024-04-25T10:34:57.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:57 smithi043 ceph-mon[29478]: pgmap v76: 97 pgs: 97 active+clean; 2.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-04-25T10:34:58.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:57 smithi077 ceph-mon[32772]: pgmap v76: 97 pgs: 97 active+clean; 2.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-04-25T10:34:58.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:57 smithi164 ceph-mon[32713]: pgmap v76: 97 pgs: 97 active+clean; 2.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-04-25T10:34:59.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:34:59 smithi043 ceph-mon[29478]: pgmap v77: 97 pgs: 97 active+clean; 2.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 18 op/s 2024-04-25T10:35:00.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:34:59 smithi077 ceph-mon[32772]: pgmap v77: 97 pgs: 97 active+clean; 2.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 18 op/s 2024-04-25T10:35:00.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:34:59 smithi164 ceph-mon[32713]: pgmap v77: 97 pgs: 97 active+clean; 2.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 18 op/s 2024-04-25T10:35:01.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:01 smithi043 ceph-mon[29478]: pgmap v78: 97 pgs: 97 active+clean; 2.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-25T10:35:02.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:01 smithi077 ceph-mon[32772]: pgmap v78: 97 pgs: 97 active+clean; 2.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-25T10:35:02.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:01 smithi164 ceph-mon[32713]: pgmap v78: 97 pgs: 97 active+clean; 2.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-25T10:35:03.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:03 smithi043 ceph-mon[29478]: pgmap v79: 97 pgs: 97 active+clean; 2.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-25T10:35:04.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:03 smithi077 ceph-mon[32772]: pgmap v79: 97 pgs: 97 active+clean; 2.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-25T10:35:04.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:03 smithi164 ceph-mon[32713]: pgmap v79: 97 pgs: 97 active+clean; 2.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-25T10:35:05.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:05 smithi043 ceph-mon[29478]: pgmap v80: 97 pgs: 97 active+clean; 2.4 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 53 KiB/s wr, 23 op/s 2024-04-25T10:35:06.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:05 smithi077 ceph-mon[32772]: pgmap v80: 97 pgs: 97 active+clean; 2.4 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 53 KiB/s wr, 23 op/s 2024-04-25T10:35:06.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:05 smithi164 ceph-mon[32713]: pgmap v80: 97 pgs: 97 active+clean; 2.4 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 53 KiB/s wr, 23 op/s 2024-04-25T10:35:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:07 smithi043 ceph-mon[29478]: pgmap v81: 97 pgs: 97 active+clean; 2.4 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-04-25T10:35:08.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:07 smithi077 ceph-mon[32772]: pgmap v81: 97 pgs: 97 active+clean; 2.4 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-04-25T10:35:08.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:07 smithi164 ceph-mon[32713]: pgmap v81: 97 pgs: 97 active+clean; 2.4 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-04-25T10:35:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.453+0000 7f8ffaba8640 1 -- 172.21.15.43:0/1377504692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ff4109b80 msgr2=0x7f8ff4103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.453+0000 7f8ffaba8640 1 --2- 172.21.15.43:0/1377504692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ff4109b80 0x7f8ff4103720 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009900 tx=0x7f8fe402a410 comp rx=0 tx=0).stop 2024-04-25T10:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.453+0000 7f8ffaba8640 1 -- 172.21.15.43:0/1377504692 shutdown_connections 2024-04-25T10:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.453+0000 7f8ffaba8640 1 --2- 172.21.15.43:0/1377504692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ff410a020 0x7f8ff4103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.453+0000 7f8ffaba8640 1 --2- 172.21.15.43:0/1377504692 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ff41042d0 0x7f8ff41106a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.453+0000 7f8ffaba8640 1 --2- 172.21.15.43:0/1377504692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ff4109b80 0x7f8ff4103720 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.453+0000 7f8ffaba8640 1 -- 172.21.15.43:0/1377504692 >> 172.21.15.43:0/1377504692 conn(0x7f8ff4079270 msgr2=0x7f8ff40796b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.453+0000 7f8ffaba8640 1 -- 172.21.15.43:0/1377504692 shutdown_connections 2024-04-25T10:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.453+0000 7f8ffaba8640 1 -- 172.21.15.43:0/1377504692 wait complete. 2024-04-25T10:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.454+0000 7f8ffaba8640 1 Processor -- start 2024-04-25T10:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.454+0000 7f8ffaba8640 1 -- start start 2024-04-25T10:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.454+0000 7f8ffaba8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ff41042d0 0x7f8ff4074ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.454+0000 7f8ffaba8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ff4109b80 0x7f8ff40753e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.454+0000 7f8ffaba8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ff410a020 0x7f8ff4075920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.454+0000 7f8ffaba8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff4113d70 con 0x7f8ff4109b80 2024-04-25T10:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.454+0000 7f8ffaba8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff4113bf0 con 0x7f8ff41042d0 2024-04-25T10:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.454+0000 7f8ffaba8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff4113ef0 con 0x7f8ff410a020 2024-04-25T10:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.454+0000 7f8ff3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ff4109b80 0x7f8ff40753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.454+0000 7f8ff3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ff4109b80 0x7f8ff40753e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55270/0 (socket says 172.21.15.43:55270) 2024-04-25T10:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.454+0000 7f8ff3fff640 1 -- 172.21.15.43:0/97400197 learned_addr learned my addr 172.21.15.43:0/97400197 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.454+0000 7f8ff8944640 1 --2- 172.21.15.43:0/97400197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ff41042d0 0x7f8ff4074ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.454+0000 7f8ff9145640 1 --2- 172.21.15.43:0/97400197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ff410a020 0x7f8ff4075920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.455+0000 7f8ff3fff640 1 -- 172.21.15.43:0/97400197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ff410a020 msgr2=0x7f8ff4075920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.455+0000 7f8ff3fff640 1 --2- 172.21.15.43:0/97400197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ff410a020 0x7f8ff4075920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.455+0000 7f8ff3fff640 1 -- 172.21.15.43:0/97400197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ff41042d0 msgr2=0x7f8ff4074ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.455+0000 7f8ff3fff640 1 --2- 172.21.15.43:0/97400197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ff41042d0 0x7f8ff4074ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.455+0000 7f8ff3fff640 1 -- 172.21.15.43:0/97400197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff41a8b10 con 0x7f8ff4109b80 2024-04-25T10:35:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.455+0000 7f8ff3fff640 1 --2- 172.21.15.43:0/97400197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ff4109b80 0x7f8ff40753e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00e9f0 tx=0x7f8fdc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.455+0000 7f8ff1ffb640 1 -- 172.21.15.43:0/97400197 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc00ccc0 con 0x7f8ff4109b80 2024-04-25T10:35:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.455+0000 7f8ff1ffb640 1 -- 172.21.15.43:0/97400197 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fdc0035f0 con 0x7f8ff4109b80 2024-04-25T10:35:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.455+0000 7f8ff1ffb640 1 -- 172.21.15.43:0/97400197 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc00ce60 con 0x7f8ff4109b80 2024-04-25T10:35:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.455+0000 7f8ffaba8640 1 -- 172.21.15.43:0/97400197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff41a8e00 con 0x7f8ff4109b80 2024-04-25T10:35:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.455+0000 7f8ffaba8640 1 -- 172.21.15.43:0/97400197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff41a9270 con 0x7f8ff4109b80 2024-04-25T10:35:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.458+0000 7f8ff1ffb640 1 -- 172.21.15.43:0/97400197 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f8fdc003aa0 con 0x7f8ff4109b80 2024-04-25T10:35:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.458+0000 7f9001a52740 1 Processor -- start 2024-04-25T10:35:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.458+0000 7f9001a52740 1 -- start start 2024-04-25T10:35:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.458+0000 7f9001a52740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d27a111b0 0x564d27db0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.458+0000 7f9001a52740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279e82f0 0x564d27db1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.458+0000 7f9001a52740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d279f04d0 0x564d27db1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.458+0000 7f9001a52740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564d27d17460 con 0x564d279f04d0 2024-04-25T10:35:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.458+0000 7f9001a52740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564d27cd1590 con 0x564d279e82f0 2024-04-25T10:35:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.458+0000 7f9001a52740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564d27d13420 con 0x564d27a111b0 2024-04-25T10:35:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.459+0000 7f8ff3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d279f04d0 0x564d27db1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.459+0000 7f8ff3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d279f04d0 0x564d27db1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55272/0 (socket says 172.21.15.43:55272) 2024-04-25T10:35:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.459+0000 7f8ff3fff640 1 -- 172.21.15.43:0/3051521436 learned_addr learned my addr 172.21.15.43:0/3051521436 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.459+0000 7f8ff1ffb640 1 --2- 172.21.15.43:0/97400197 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8fe00afc50 0x7f8fe00b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.459+0000 7f8ff9145640 1 --2- 172.21.15.43:0/3051521436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d27a111b0 0x564d27db0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.459+0000 7f8ff8944640 1 --2- 172.21.15.43:0/3051521436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279e82f0 0x564d27db1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.459+0000 7f8ff9145640 1 --2- 172.21.15.43:0/97400197 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8fe00afc50 0x7f8fe00b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.459+0000 7f8ff1ffb640 1 -- 172.21.15.43:0/97400197 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f8fdc0df2e0 con 0x7f8ff4109b80 2024-04-25T10:35:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.459+0000 7f8ff3fff640 1 -- 172.21.15.43:0/3051521436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d27a111b0 msgr2=0x564d27db0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.459+0000 7f8ff3fff640 1 --2- 172.21.15.43:0/3051521436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d27a111b0 0x564d27db0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.459+0000 7f8ff3fff640 1 -- 172.21.15.43:0/3051521436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279e82f0 msgr2=0x564d27db1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.459+0000 7f8ff3fff640 1 --2- 172.21.15.43:0/3051521436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279e82f0 0x564d27db1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.459+0000 7f8ff3fff640 1 -- 172.21.15.43:0/3051521436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d27cde0d0 con 0x564d279f04d0 2024-04-25T10:35:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.459+0000 7f8ff8944640 1 --2- 172.21.15.43:0/3051521436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279e82f0 0x564d27db1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:35:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.459+0000 7f8ff9145640 1 --2- 172.21.15.43:0/3051521436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d27a111b0 0x564d27db0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:35:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.459+0000 7f8ff3fff640 1 --2- 172.21.15.43:0/3051521436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d279f04d0 0x564d27db1570 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0a7150 tx=0x7f8fdc003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550ef22fbc41efb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.459+0000 7f8feaffd640 1 -- 172.21.15.43:0/3051521436 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc0a6bc0 con 0x564d279f04d0 2024-04-25T10:35:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.460+0000 7f8ff9145640 1 --2- 172.21.15.43:0/97400197 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8fe00afc50 0x7f8fe00b2160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x564d27dadc60 tx=0x7f8fec004ab0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.460+0000 7f8feaffd640 1 -- 172.21.15.43:0/3051521436 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fdc0a6f60 con 0x564d279f04d0 2024-04-25T10:35:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.460+0000 7f8feaffd640 1 -- 172.21.15.43:0/3051521436 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc0afe20 con 0x564d279f04d0 2024-04-25T10:35:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.460+0000 7f9001a52740 1 -- 172.21.15.43:0/3051521436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d279f04d0 msgr2=0x564d27db1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.460+0000 7f9001a52740 1 --2- 172.21.15.43:0/3051521436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d279f04d0 0x564d27db1570 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0a7150 tx=0x7f8fdc003a10 comp rx=0 tx=0).stop 2024-04-25T10:35:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.460+0000 7f9001a52740 1 -- 172.21.15.43:0/3051521436 shutdown_connections 2024-04-25T10:35:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.460+0000 7f9001a52740 1 --2- 172.21.15.43:0/3051521436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279e82f0 0x564d27db1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.460+0000 7f9001a52740 1 --2- 172.21.15.43:0/3051521436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d279f04d0 0x564d27db1570 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.460+0000 7f9001a52740 1 --2- 172.21.15.43:0/3051521436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d27a111b0 0x564d27db0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.460+0000 7f9001a52740 1 -- 172.21.15.43:0/3051521436 >> 172.21.15.43:0/3051521436 conn(0x564d27a05300 msgr2=0x564d27dad2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.460+0000 7f9001a52740 1 -- 172.21.15.43:0/3051521436 shutdown_connections 2024-04-25T10:35:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.460+0000 7f9001a52740 1 -- 172.21.15.43:0/3051521436 wait complete. 2024-04-25T10:35:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.461+0000 7f9001a52740 1 -- start start 2024-04-25T10:35:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.461+0000 7f9001a52740 1 Processor -- start 2024-04-25T10:35:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.461+0000 7f9001a52740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d279e82f0 0x564d27dbcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.461+0000 7f9001a52740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279f04d0 0x564d27dbd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.461+0000 7f9001a52740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d27a111b0 0x564d27dbb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.461+0000 7f9001a52740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564d27dbdb60 con 0x564d27a111b0 2024-04-25T10:35:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.461+0000 7f9001a52740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564d27c9eb60 con 0x564d279f04d0 2024-04-25T10:35:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.461+0000 7f9001a52740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564d27dbdce0 con 0x564d279e82f0 2024-04-25T10:35:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.461+0000 7f9001a52740 10 client.0 _ll_register_callbacks cb 0x564d27cc32b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.462+0000 7f8ff9145640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d27a111b0 0x564d27dbb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.462+0000 7f8ff9145640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d27a111b0 0x564d27dbb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55286/0 (socket says 172.21.15.43:55286) 2024-04-25T10:35:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.462+0000 7f8ff9145640 1 -- 172.21.15.43:0/1217654369 learned_addr learned my addr 172.21.15.43:0/1217654369 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.462+0000 7f8ff8944640 1 --2- 172.21.15.43:0/1217654369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d279e82f0 0x564d27dbcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.462+0000 7f8ff3fff640 1 --2- 172.21.15.43:0/1217654369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279f04d0 0x564d27dbd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.462+0000 7f8ff9145640 1 -- 172.21.15.43:0/1217654369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d279e82f0 msgr2=0x564d27dbcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.462+0000 7f8ff9145640 1 --2- 172.21.15.43:0/1217654369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d279e82f0 0x564d27dbcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.462+0000 7f8ff9145640 1 -- 172.21.15.43:0/1217654369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279f04d0 msgr2=0x564d27dbd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.462+0000 7f8ff9145640 1 --2- 172.21.15.43:0/1217654369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279f04d0 0x564d27dbd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.462+0000 7f8ff9145640 1 -- 172.21.15.43:0/1217654369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d27a29210 con 0x564d27a111b0 2024-04-25T10:35:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.462+0000 7f8ff3fff640 1 --2- 172.21.15.43:0/1217654369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279f04d0 0x564d27dbd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.462+0000 7f8ff9145640 1 --2- 172.21.15.43:0/1217654369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d27a111b0 0x564d27dbb030 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8fec012ce0 tx=0x7f8fec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72daff53c3f78857 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.462+0000 7f8fd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.462+0000 7f8fd3fff640 1 -- 172.21.15.43:0/1217654369 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec025020 con 0x564d27a111b0 2024-04-25T10:35:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.462+0000 7f8fd3fff640 1 -- 172.21.15.43:0/1217654369 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f8fec0045b0 con 0x564d27a111b0 2024-04-25T10:35:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.462+0000 7f9001a52740 1 -- 172.21.15.43:0/1217654369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fdc0a6f60 con 0x564d27a111b0 2024-04-25T10:35:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.463+0000 7f8fd3fff640 1 -- 172.21.15.43:0/1217654369 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8fec01a040 con 0x564d27a111b0 2024-04-25T10:35:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.463+0000 7f9001a52740 10 client.16197 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.463+0000 7f9001a52740 1 -- 172.21.15.43:0/1217654369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564d27af5620 con 0x564d27a111b0 2024-04-25T10:35:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.463+0000 7f8fd3fff640 1 -- 172.21.15.43:0/1217654369 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fdc0a6f60 con 0x564d27a111b0 2024-04-25T10:35:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.463+0000 7f8fd3fff640 1 -- 172.21.15.43:0/1217654369 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8fec020040 con 0x564d27a111b0 2024-04-25T10:35:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.463+0000 7f9001a52740 10 client.16197 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.463+0000 7f9001a52740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.463+0000 7f9001a52740 20 client.16197 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.463+0000 7f9001a52740 1 --2- 172.21.15.43:0/1217654369 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564d279cfd80 0x564d27dcd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.463+0000 7f9001a52740 4 client.16197 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:35:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.463+0000 7f9001a52740 1 -- 172.21.15.43:0/1217654369 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d27d25070 con 0x564d279cfd80 2024-04-25T10:35:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.463+0000 7f8ff8944640 1 --2- 172.21.15.43:0/1217654369 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564d279cfd80 0x564d27dcd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.464+0000 7f8ff8944640 1 --2- 172.21.15.43:0/1217654369 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564d279cfd80 0x564d27dcd7b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b47466bbba3aacb server_cookie=993aae2d5219872 in_seq=0 out_seq=0 2024-04-25T10:35:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.464+0000 7f8fd3fff640 10 client.16197 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.465+0000 7f8fd3fff640 1 -- 172.21.15.43:0/1217654369 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564d27d25070 con 0x564d279cfd80 2024-04-25T10:35:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.465+0000 7f8fd3fff640 10 client.16197 handle_command_reply: tid=0 2024-04-25T10:35:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.465+0000 7f9001a52740 1 client.16197 shutdown 2024-04-25T10:35:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.465+0000 7f9001a52740 1 -- 172.21.15.43:0/1217654369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d27a111b0 msgr2=0x564d27dbb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.465+0000 7f9001a52740 1 --2- 172.21.15.43:0/1217654369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d27a111b0 0x564d27dbb030 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8fec012ce0 tx=0x7f8fec004a40 comp rx=0 tx=0).stop 2024-04-25T10:35:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.465+0000 7f9001a52740 1 -- 172.21.15.43:0/1217654369 shutdown_connections 2024-04-25T10:35:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.465+0000 7f9001a52740 1 --2- 172.21.15.43:0/1217654369 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564d279cfd80 0x564d27dcd7b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.465+0000 7f9001a52740 1 --2- 172.21.15.43:0/1217654369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279f04d0 0x564d27dbd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.465+0000 7f9001a52740 1 --2- 172.21.15.43:0/1217654369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d27a111b0 0x564d27dbb030 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.465+0000 7f8fd3fff640 0 client.16197 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.465+0000 7f9001a52740 1 --2- 172.21.15.43:0/1217654369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d279e82f0 0x564d27dbcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.465+0000 7f9001a52740 1 -- 172.21.15.43:0/1217654369 >> 172.21.15.43:0/1217654369 conn(0x564d27a05300 msgr2=0x564d27db2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.465+0000 7f9001a52740 1 -- 172.21.15.43:0/1217654369 shutdown_connections 2024-04-25T10:35:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.465+0000 7f9001a52740 1 -- 172.21.15.43:0/1217654369 wait complete. 2024-04-25T10:35:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.465+0000 7f9001a52740 20 client.16197 trim_cache size 0 max 16384 2024-04-25T10:35:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.480+0000 7f9001a52740 1 Processor -- start 2024-04-25T10:35:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.480+0000 7f9001a52740 1 -- start start 2024-04-25T10:35:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.481+0000 7f9001a52740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279e82f0 0x564d27dd2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.481+0000 7f9001a52740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d279f04d0 0x564d27dd34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.481+0000 7f9001a52740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d27a111b0 0x564d27dc2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.481+0000 7f9001a52740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41b4590 con 0x564d279f04d0 2024-04-25T10:35:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.481+0000 7f9001a52740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41b1e60 con 0x564d279e82f0 2024-04-25T10:35:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.481+0000 7f9001a52740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41afcb0 con 0x564d27a111b0 2024-04-25T10:35:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.481+0000 7f8ff3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d279f04d0 0x564d27dd34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.481+0000 7f8ff3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d279f04d0 0x564d27dd34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55292/0 (socket says 172.21.15.43:55292) 2024-04-25T10:35:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.481+0000 7f8ff3fff640 1 -- 172.21.15.43:0/69596239 learned_addr learned my addr 172.21.15.43:0/69596239 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.481+0000 7f8ff8944640 1 --2- 172.21.15.43:0/69596239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279e82f0 0x564d27dd2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.481+0000 7f8ff3fff640 1 -- 172.21.15.43:0/69596239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d27a111b0 msgr2=0x564d27dc2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.481+0000 7f8ff3fff640 1 --2- 172.21.15.43:0/69596239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d27a111b0 0x564d27dc2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.481+0000 7f8ff3fff640 1 -- 172.21.15.43:0/69596239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279e82f0 msgr2=0x564d27dd2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.481+0000 7f8ff3fff640 1 --2- 172.21.15.43:0/69596239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279e82f0 0x564d27dd2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.481+0000 7f8ff3fff640 1 -- 172.21.15.43:0/69596239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d27dbd7d0 con 0x564d279f04d0 2024-04-25T10:35:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.481+0000 7f8ff8944640 1 --2- 172.21.15.43:0/69596239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279e82f0 0x564d27dd2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:35:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.481+0000 7f8ff3fff640 1 --2- 172.21.15.43:0/69596239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d279f04d0 0x564d27dd34f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0a46c0 tx=0x7f8fdc0a3b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b79e19dcfe7da3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.481+0000 7f8fea7fc640 1 -- 172.21.15.43:0/69596239 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc0a7b50 con 0x564d279f04d0 2024-04-25T10:35:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.481+0000 7f8fea7fc640 1 -- 172.21.15.43:0/69596239 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fdc0e4070 con 0x564d279f04d0 2024-04-25T10:35:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.481+0000 7f8fea7fc640 1 -- 172.21.15.43:0/69596239 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc0a1be0 con 0x564d279f04d0 2024-04-25T10:35:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.482+0000 7f9001a52740 1 -- 172.21.15.43:0/69596239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d279f04d0 msgr2=0x564d27dd34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.482+0000 7f9001a52740 1 --2- 172.21.15.43:0/69596239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d279f04d0 0x564d27dd34f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0a46c0 tx=0x7f8fdc0a3b00 comp rx=0 tx=0).stop 2024-04-25T10:35:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.482+0000 7f9001a52740 1 -- 172.21.15.43:0/69596239 shutdown_connections 2024-04-25T10:35:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.482+0000 7f9001a52740 1 --2- 172.21.15.43:0/69596239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d27a111b0 0x564d27dc2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.482+0000 7f9001a52740 1 --2- 172.21.15.43:0/69596239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d279f04d0 0x564d27dd34f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.482+0000 7f9001a52740 1 --2- 172.21.15.43:0/69596239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279e82f0 0x564d27dd2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.482+0000 7f9001a52740 1 -- 172.21.15.43:0/69596239 >> 172.21.15.43:0/69596239 conn(0x564d27a05300 msgr2=0x564d27db9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.482+0000 7f9001a52740 1 -- 172.21.15.43:0/69596239 shutdown_connections 2024-04-25T10:35:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.482+0000 7f9001a52740 1 -- 172.21.15.43:0/69596239 wait complete. 2024-04-25T10:35:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.483+0000 7f9001a52740 1 -- start start 2024-04-25T10:35:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.483+0000 7f9001a52740 1 Processor -- start 2024-04-25T10:35:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.483+0000 7f9001a52740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279e82f0 0x564d27dbeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.483+0000 7f9001a52740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d279f04d0 0x564d27dbf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.483+0000 7f9001a52740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d27a111b0 0x564d27dbf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.483+0000 7f9001a52740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41b0320 con 0x564d27a111b0 2024-04-25T10:35:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.483+0000 7f9001a52740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41b1790 con 0x564d279e82f0 2024-04-25T10:35:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.483+0000 7f9001a52740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41b09f0 con 0x564d279f04d0 2024-04-25T10:35:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.483+0000 7f9001a52740 10 client.0 _ll_register_callbacks cb 0x7f8fec0146a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.483+0000 7f8ff9145640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d27a111b0 0x564d27dbf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.483+0000 7f8ff9145640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d27a111b0 0x564d27dbf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55306/0 (socket says 172.21.15.43:55306) 2024-04-25T10:35:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.483+0000 7f8ff9145640 1 -- 172.21.15.43:0/1654181645 learned_addr learned my addr 172.21.15.43:0/1654181645 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.483+0000 7f8ff3fff640 1 --2- 172.21.15.43:0/1654181645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d279f04d0 0x564d27dbf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.483+0000 7f8ff8944640 1 --2- 172.21.15.43:0/1654181645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279e82f0 0x564d27dbeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.484+0000 7f8ff9145640 1 -- 172.21.15.43:0/1654181645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d279f04d0 msgr2=0x564d27dbf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.484+0000 7f8ff9145640 1 --2- 172.21.15.43:0/1654181645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d279f04d0 0x564d27dbf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.484+0000 7f8ff9145640 1 -- 172.21.15.43:0/1654181645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279e82f0 msgr2=0x564d27dbeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.484+0000 7f8ff9145640 1 --2- 172.21.15.43:0/1654181645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279e82f0 0x564d27dbeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.484+0000 7f8ff9145640 1 -- 172.21.15.43:0/1654181645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d27e34b60 con 0x564d27a111b0 2024-04-25T10:35:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.484+0000 7f8ff9145640 1 --2- 172.21.15.43:0/1654181645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d27a111b0 0x564d27dbf600 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8fec021ec0 tx=0x7f8fec009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9c8a9e4a32f32f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.484+0000 7f8fd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.484+0000 7f8fd3fff640 1 -- 172.21.15.43:0/1654181645 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec025040 con 0x564d27a111b0 2024-04-25T10:35:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.484+0000 7f8fd3fff640 1 -- 172.21.15.43:0/1654181645 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f8fec010940 con 0x564d27a111b0 2024-04-25T10:35:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.484+0000 7f9001a52740 1 -- 172.21.15.43:0/1654181645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fdc0e4070 con 0x564d27a111b0 2024-04-25T10:35:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.484+0000 7f8fd3fff640 1 -- 172.21.15.43:0/1654181645 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8fec0048c0 con 0x564d27a111b0 2024-04-25T10:35:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.484+0000 7f9001a52740 10 client.16203 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.484+0000 7f9001a52740 1 -- 172.21.15.43:0/1654181645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564d27e35d10 con 0x564d27a111b0 2024-04-25T10:35:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.484+0000 7f8fd3fff640 1 -- 172.21.15.43:0/1654181645 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fdc0e4070 con 0x564d27a111b0 2024-04-25T10:35:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.484+0000 7f8fd3fff640 1 -- 172.21.15.43:0/1654181645 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8fec020070 con 0x564d27a111b0 2024-04-25T10:35:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.485+0000 7f9001a52740 10 client.16203 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.485+0000 7f9001a52740 10 client.16203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.485+0000 7f9001a52740 20 client.16203 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.485+0000 7f9001a52740 1 --2- 172.21.15.43:0/1654181645 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564d279a51c0 0x564d27e384a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.485+0000 7f9001a52740 4 client.16203 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:35:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.485+0000 7f9001a52740 1 -- 172.21.15.43:0/1654181645 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d27d1eee0 con 0x564d279a51c0 2024-04-25T10:35:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.485+0000 7f8ff8944640 1 --2- 172.21.15.43:0/1654181645 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564d279a51c0 0x564d27e384a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.485+0000 7f8fd3fff640 10 client.16203 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.485+0000 7f8ff8944640 1 --2- 172.21.15.43:0/1654181645 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564d279a51c0 0x564d27e384a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9274e57a273a2509 server_cookie=247ddca0f448d245 in_seq=0 out_seq=0 2024-04-25T10:35:08.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.502+0000 7f8fd3fff640 1 -- 172.21.15.43:0/1654181645 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564d27d1eee0 con 0x564d279a51c0 2024-04-25T10:35:08.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.502+0000 7f8fd3fff640 10 client.16203 handle_command_reply: tid=0 2024-04-25T10:35:08.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.502+0000 7f9001a52740 1 client.16203 shutdown 2024-04-25T10:35:08.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.502+0000 7f9001a52740 1 -- 172.21.15.43:0/1654181645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d27a111b0 msgr2=0x564d27dbf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:08.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.502+0000 7f9001a52740 1 --2- 172.21.15.43:0/1654181645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d27a111b0 0x564d27dbf600 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8fec021ec0 tx=0x7f8fec009680 comp rx=0 tx=0).stop 2024-04-25T10:35:08.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.502+0000 7f9001a52740 1 -- 172.21.15.43:0/1654181645 shutdown_connections 2024-04-25T10:35:08.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.502+0000 7f9001a52740 1 --2- 172.21.15.43:0/1654181645 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564d279a51c0 0x564d27e384a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.502+0000 7f9001a52740 1 --2- 172.21.15.43:0/1654181645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d27a111b0 0x564d27dbf600 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.502+0000 7f9001a52740 1 --2- 172.21.15.43:0/1654181645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d279f04d0 0x564d27dbf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.502+0000 7f8fd3fff640 0 client.16203 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:08.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.502+0000 7f9001a52740 1 --2- 172.21.15.43:0/1654181645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d279e82f0 0x564d27dbeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.502+0000 7f9001a52740 1 -- 172.21.15.43:0/1654181645 >> 172.21.15.43:0/1654181645 conn(0x564d27a05300 msgr2=0x564d27dba860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.503+0000 7f9001a52740 1 -- 172.21.15.43:0/1654181645 shutdown_connections 2024-04-25T10:35:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.503+0000 7f9001a52740 1 -- 172.21.15.43:0/1654181645 wait complete. 2024-04-25T10:35:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.503+0000 7f9001a52740 20 client.16203 trim_cache size 0 max 16384 2024-04-25T10:35:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:35:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:35:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:35:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:35:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.503+0000 7f8ffaba8640 1 -- 172.21.15.43:0/97400197 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8fe00afc50 msgr2=0x7f8fe00b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.503+0000 7f8ffaba8640 1 --2- 172.21.15.43:0/97400197 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8fe00afc50 0x7f8fe00b2160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x564d27dadc60 tx=0x7f8fec004ab0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.503+0000 7f8ffaba8640 1 -- 172.21.15.43:0/97400197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ff4109b80 msgr2=0x7f8ff40753e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.503+0000 7f8ffaba8640 1 --2- 172.21.15.43:0/97400197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ff4109b80 0x7f8ff40753e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00e9f0 tx=0x7f8fdc00eeb0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.503+0000 7f8ffaba8640 1 -- 172.21.15.43:0/97400197 shutdown_connections 2024-04-25T10:35:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.503+0000 7f8ffaba8640 1 --2- 172.21.15.43:0/97400197 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8fe00afc50 0x7f8fe00b2160 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.503+0000 7f8ffaba8640 1 --2- 172.21.15.43:0/97400197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ff410a020 0x7f8ff4075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.503+0000 7f8ffaba8640 1 --2- 172.21.15.43:0/97400197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ff4109b80 0x7f8ff40753e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.503+0000 7f8ffaba8640 1 --2- 172.21.15.43:0/97400197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ff41042d0 0x7f8ff4074ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.503+0000 7f8ffaba8640 1 -- 172.21.15.43:0/97400197 >> 172.21.15.43:0/97400197 conn(0x7f8ff4079270 msgr2=0x7f8ff4105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.503+0000 7f8ffaba8640 1 -- 172.21.15.43:0/97400197 shutdown_connections 2024-04-25T10:35:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:08.503+0000 7f8ffaba8640 1 -- 172.21.15.43:0/97400197 wait complete. 2024-04-25T10:35:09.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:09 smithi043 ceph-mon[29478]: pgmap v82: 97 pgs: 97 active+clean; 2.5 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 KiB/s wr, 24 op/s 2024-04-25T10:35:10.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:09 smithi077 ceph-mon[32772]: pgmap v82: 97 pgs: 97 active+clean; 2.5 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 KiB/s wr, 24 op/s 2024-04-25T10:35:10.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:09 smithi164 ceph-mon[32713]: pgmap v82: 97 pgs: 97 active+clean; 2.5 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 KiB/s wr, 24 op/s 2024-04-25T10:35:11.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:11 smithi043 ceph-mon[29478]: pgmap v83: 97 pgs: 97 active+clean; 2.5 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 46 KiB/s wr, 16 op/s 2024-04-25T10:35:12.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:11 smithi077 ceph-mon[32772]: pgmap v83: 97 pgs: 97 active+clean; 2.5 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 46 KiB/s wr, 16 op/s 2024-04-25T10:35:12.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:11 smithi164 ceph-mon[32713]: pgmap v83: 97 pgs: 97 active+clean; 2.5 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 46 KiB/s wr, 16 op/s 2024-04-25T10:35:13.267 INFO:tasks.workunit.client.0.smithi043.stdout:1/600: open fd = 3 2024-04-25T10:35:13.267 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.267 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.268 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.268 INFO:tasks.workunit.client.0.smithi043.stdout:2/600: open fd = 3 2024-04-25T10:35:13.268 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.268 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.268 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.268 INFO:tasks.workunit.client.0.smithi043.stdout:3/600: open fd = 3 2024-04-25T10:35:13.268 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.268 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.268 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.268 INFO:tasks.workunit.client.0.smithi043.stdout:4/600: open fd = 3 2024-04-25T10:35:13.268 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.268 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.268 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.268 INFO:tasks.workunit.client.0.smithi043.stdout:5/600: open fd = 3 2024-04-25T10:35:13.269 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.269 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.269 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.269 INFO:tasks.workunit.client.0.smithi043.stdout:6/600: open fd = 3 2024-04-25T10:35:13.269 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.269 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.269 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.269 INFO:tasks.workunit.client.0.smithi043.stdout:7/600: open fd = 3 2024-04-25T10:35:13.269 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.269 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.269 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.269 INFO:tasks.workunit.client.0.smithi043.stdout:8/600: open fd = 3 2024-04-25T10:35:13.269 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.269 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.270 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.270 INFO:tasks.workunit.client.0.smithi043.stdout:9/600: open fd = 3 2024-04-25T10:35:13.270 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.270 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.270 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.270 INFO:tasks.workunit.client.0.smithi043.stdout:10/600: open fd = 3 2024-04-25T10:35:13.270 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.270 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.270 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.270 INFO:tasks.workunit.client.0.smithi043.stdout:11/600: open fd = 3 2024-04-25T10:35:13.270 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.270 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.270 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.270 INFO:tasks.workunit.client.0.smithi043.stdout:12/600: open fd = 3 2024-04-25T10:35:13.271 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.271 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.271 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.271 INFO:tasks.workunit.client.0.smithi043.stdout:13/600: open fd = 3 2024-04-25T10:35:13.271 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.271 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.271 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.271 INFO:tasks.workunit.client.0.smithi043.stdout:14/600: open fd = 3 2024-04-25T10:35:13.271 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.271 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.271 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.271 INFO:tasks.workunit.client.0.smithi043.stdout:15/600: open fd = 3 2024-04-25T10:35:13.271 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.271 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.272 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.272 INFO:tasks.workunit.client.0.smithi043.stdout:16/600: open fd = 3 2024-04-25T10:35:13.272 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.272 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.272 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.272 INFO:tasks.workunit.client.0.smithi043.stdout:17/600: open fd = 3 2024-04-25T10:35:13.272 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.272 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.272 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.272 INFO:tasks.workunit.client.0.smithi043.stdout:18/600: open fd = 3 2024-04-25T10:35:13.272 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.272 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.272 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.272 INFO:tasks.workunit.client.0.smithi043.stdout:19/600: open fd = 3 2024-04-25T10:35:13.273 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.273 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.273 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.273 INFO:tasks.workunit.client.0.smithi043.stdout:20/600: open fd = 3 2024-04-25T10:35:13.273 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.273 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.273 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.273 INFO:tasks.workunit.client.0.smithi043.stdout:21/600: open fd = 3 2024-04-25T10:35:13.273 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.273 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.273 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.273 INFO:tasks.workunit.client.0.smithi043.stdout:22/600: open fd = 3 2024-04-25T10:35:13.273 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.273 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.273 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.274 INFO:tasks.workunit.client.0.smithi043.stdout:23/600: open fd = 3 2024-04-25T10:35:13.274 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.274 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.274 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.274 INFO:tasks.workunit.client.0.smithi043.stdout:24/600: open fd = 3 2024-04-25T10:35:13.274 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.274 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.274 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.274 INFO:tasks.workunit.client.0.smithi043.stdout:25/600: open fd = 3 2024-04-25T10:35:13.274 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.274 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.274 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.274 INFO:tasks.workunit.client.0.smithi043.stdout:26/600: open fd = 3 2024-04-25T10:35:13.274 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.274 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.275 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.275 INFO:tasks.workunit.client.0.smithi043.stdout:27/600: open fd = 3 2024-04-25T10:35:13.275 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.275 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.275 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.275 INFO:tasks.workunit.client.0.smithi043.stdout:28/600: open fd = 3 2024-04-25T10:35:13.275 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.275 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.275 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.275 INFO:tasks.workunit.client.0.smithi043.stdout:29/600: open fd = 3 2024-04-25T10:35:13.275 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.275 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.275 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.275 INFO:tasks.workunit.client.0.smithi043.stdout:30/600: open fd = 3 2024-04-25T10:35:13.277 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.277 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.277 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.277 INFO:tasks.workunit.client.0.smithi043.stdout:31/600: open fd = 3 2024-04-25T10:35:13.277 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.277 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.277 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.277 INFO:tasks.workunit.client.0.smithi043.stdout:32/600: open fd = 3 2024-04-25T10:35:13.277 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.277 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.278 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.278 INFO:tasks.workunit.client.0.smithi043.stdout:33/600: open fd = 3 2024-04-25T10:35:13.278 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.278 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.278 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.278 INFO:tasks.workunit.client.0.smithi043.stdout:34/600: open fd = 3 2024-04-25T10:35:13.278 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.278 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.278 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.278 INFO:tasks.workunit.client.0.smithi043.stdout:35/600: open fd = 3 2024-04-25T10:35:13.278 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.278 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.278 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.279 INFO:tasks.workunit.client.0.smithi043.stdout:36/600: open fd = 3 2024-04-25T10:35:13.279 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.279 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.279 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.279 INFO:tasks.workunit.client.0.smithi043.stdout:37/600: open fd = 3 2024-04-25T10:35:13.279 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.279 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.279 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.279 INFO:tasks.workunit.client.0.smithi043.stdout:38/600: open fd = 3 2024-04-25T10:35:13.279 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.279 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.279 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.279 INFO:tasks.workunit.client.0.smithi043.stdout:39/600: open fd = 3 2024-04-25T10:35:13.279 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.280 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.280 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.280 INFO:tasks.workunit.client.0.smithi043.stdout:40/600: open fd = 3 2024-04-25T10:35:13.280 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.280 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.280 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.280 INFO:tasks.workunit.client.0.smithi043.stdout:41/600: open fd = 3 2024-04-25T10:35:13.280 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.280 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.280 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.280 INFO:tasks.workunit.client.0.smithi043.stdout:42/600: open fd = 3 2024-04-25T10:35:13.280 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.280 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.280 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.281 INFO:tasks.workunit.client.0.smithi043.stdout:43/600: open fd = 3 2024-04-25T10:35:13.281 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.281 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.281 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.281 INFO:tasks.workunit.client.0.smithi043.stdout:44/600: open fd = 3 2024-04-25T10:35:13.281 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.281 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.281 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.281 INFO:tasks.workunit.client.0.smithi043.stdout:45/600: open fd = 3 2024-04-25T10:35:13.281 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.281 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.281 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.281 INFO:tasks.workunit.client.0.smithi043.stdout:46/600: open fd = 3 2024-04-25T10:35:13.281 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.281 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.282 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.282 INFO:tasks.workunit.client.0.smithi043.stdout:47/600: open fd = 3 2024-04-25T10:35:13.282 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.282 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.282 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.282 INFO:tasks.workunit.client.0.smithi043.stdout:48/600: open fd = 3 2024-04-25T10:35:13.282 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.282 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.282 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.282 INFO:tasks.workunit.client.0.smithi043.stdout:49/600: open fd = 3 2024-04-25T10:35:13.282 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.282 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.282 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.282 INFO:tasks.workunit.client.0.smithi043.stdout:50/600: open fd = 3 2024-04-25T10:35:13.283 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.283 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.283 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.283 INFO:tasks.workunit.client.0.smithi043.stdout:51/600: open fd = 3 2024-04-25T10:35:13.283 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.283 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.283 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.283 INFO:tasks.workunit.client.0.smithi043.stdout:52/600: open fd = 3 2024-04-25T10:35:13.283 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.283 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.283 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.283 INFO:tasks.workunit.client.0.smithi043.stdout:53/600: open fd = 3 2024-04-25T10:35:13.283 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.283 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.284 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.284 INFO:tasks.workunit.client.0.smithi043.stdout:54/600: open fd = 3 2024-04-25T10:35:13.284 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.284 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.284 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.284 INFO:tasks.workunit.client.0.smithi043.stdout:55/600: open fd = 3 2024-04-25T10:35:13.284 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.284 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.284 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.284 INFO:tasks.workunit.client.0.smithi043.stdout:56/600: open fd = 3 2024-04-25T10:35:13.284 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.284 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.284 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.284 INFO:tasks.workunit.client.0.smithi043.stdout:57/600: open fd = 3 2024-04-25T10:35:13.284 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.285 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.285 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.285 INFO:tasks.workunit.client.0.smithi043.stdout:58/600: open fd = 3 2024-04-25T10:35:13.285 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.285 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.285 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.285 INFO:tasks.workunit.client.0.smithi043.stdout:59/600: open fd = 3 2024-04-25T10:35:13.285 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.285 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.285 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.285 INFO:tasks.workunit.client.0.smithi043.stdout:60/600: open fd = 3 2024-04-25T10:35:13.285 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.285 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.285 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.285 INFO:tasks.workunit.client.0.smithi043.stdout:61/600: open fd = 3 2024-04-25T10:35:13.286 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.286 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.286 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.286 INFO:tasks.workunit.client.0.smithi043.stdout:62/600: open fd = 3 2024-04-25T10:35:13.286 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.286 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.286 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.286 INFO:tasks.workunit.client.0.smithi043.stdout:63/600: open fd = 3 2024-04-25T10:35:13.286 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.286 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:35:13.286 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:35:13.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:13 smithi043 ceph-mon[29478]: pgmap v84: 97 pgs: 97 active+clean; 2.5 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 46 KiB/s wr, 16 op/s 2024-04-25T10:35:14.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:13 smithi077 ceph-mon[32772]: pgmap v84: 97 pgs: 97 active+clean; 2.5 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 46 KiB/s wr, 16 op/s 2024-04-25T10:35:14.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:13 smithi164 ceph-mon[32713]: pgmap v84: 97 pgs: 97 active+clean; 2.5 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 46 KiB/s wr, 16 op/s 2024-04-25T10:35:15.853 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:15 smithi077 ceph-mon[32772]: pgmap v85: 97 pgs: 97 active+clean; 2.6 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-25T10:35:15.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:15 smithi043 ceph-mon[29478]: pgmap v85: 97 pgs: 97 active+clean; 2.6 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-25T10:35:16.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:15 smithi164 ceph-mon[32713]: pgmap v85: 97 pgs: 97 active+clean; 2.6 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-25T10:35:17.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:17 smithi043 ceph-mon[29478]: pgmap v86: 97 pgs: 97 active+clean; 2.6 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T10:35:18.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:17 smithi077 ceph-mon[32772]: pgmap v86: 97 pgs: 97 active+clean; 2.6 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T10:35:18.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:17 smithi164 ceph-mon[32713]: pgmap v86: 97 pgs: 97 active+clean; 2.6 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T10:35:19.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:19 smithi043 ceph-mon[29478]: pgmap v87: 97 pgs: 97 active+clean; 2.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-25T10:35:19.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:35:19.916 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:35:20.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:19 smithi077 ceph-mon[32772]: pgmap v87: 97 pgs: 97 active+clean; 2.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-25T10:35:20.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:19 smithi164 ceph-mon[32713]: pgmap v87: 97 pgs: 97 active+clean; 2.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-25T10:35:20.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.043+0000 7f3224bce640 1 -- 172.21.15.43:0/1214956495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3220104cb0 msgr2=0x7f32201050c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.043+0000 7f3224bce640 1 --2- 172.21.15.43:0/1214956495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3220104cb0 0x7f32201050c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3210009ca0 tx=0x7f321002a950 comp rx=0 tx=0).stop 2024-04-25T10:35:20.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.043+0000 7f3224bce640 1 -- 172.21.15.43:0/1214956495 shutdown_connections 2024-04-25T10:35:20.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.043+0000 7f3224bce640 1 --2- 172.21.15.43:0/1214956495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f322010e070 0x7f3220110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.043+0000 7f3224bce640 1 --2- 172.21.15.43:0/1214956495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f322010b710 0x7f322010db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.043+0000 7f3224bce640 1 --2- 172.21.15.43:0/1214956495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3220104cb0 0x7f32201050c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.043+0000 7f3224bce640 1 -- 172.21.15.43:0/1214956495 >> 172.21.15.43:0/1214956495 conn(0x7f3220078100 msgr2=0x7f3220078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.043+0000 7f3224bce640 1 -- 172.21.15.43:0/1214956495 shutdown_connections 2024-04-25T10:35:20.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.043+0000 7f3224bce640 1 -- 172.21.15.43:0/1214956495 wait complete. 2024-04-25T10:35:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.044+0000 7f3224bce640 1 Processor -- start 2024-04-25T10:35:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.044+0000 7f3224bce640 1 -- start start 2024-04-25T10:35:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.044+0000 7f3224bce640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3220104cb0 0x7f322019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.044+0000 7f3224bce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f322010b710 0x7f322019acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.044+0000 7f3224bce640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f322010e070 0x7f322019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.044+0000 7f3224bce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3220113cb0 con 0x7f322010b710 2024-04-25T10:35:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.044+0000 7f3224bce640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3220113b30 con 0x7f3220104cb0 2024-04-25T10:35:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.044+0000 7f3224bce640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3220113e30 con 0x7f322010e070 2024-04-25T10:35:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.045+0000 7f321dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f322010b710 0x7f322019acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.045+0000 7f321dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f322010b710 0x7f322019acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45268/0 (socket says 172.21.15.43:45268) 2024-04-25T10:35:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.045+0000 7f321e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3220104cb0 0x7f322019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.045+0000 7f321dd9b640 1 -- 172.21.15.43:0/2842820276 learned_addr learned my addr 172.21.15.43:0/2842820276 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.045+0000 7f321ed9d640 1 --2- 172.21.15.43:0/2842820276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f322010e070 0x7f322019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.045+0000 7f321dd9b640 1 -- 172.21.15.43:0/2842820276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f322010e070 msgr2=0x7f322019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.045+0000 7f321dd9b640 1 --2- 172.21.15.43:0/2842820276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f322010e070 0x7f322019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.045+0000 7f321dd9b640 1 -- 172.21.15.43:0/2842820276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3220104cb0 msgr2=0x7f322019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.045+0000 7f321dd9b640 1 --2- 172.21.15.43:0/2842820276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3220104cb0 0x7f322019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.045+0000 7f321dd9b640 1 -- 172.21.15.43:0/2842820276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322019f530 con 0x7f322010b710 2024-04-25T10:35:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.045+0000 7f321dd9b640 1 --2- 172.21.15.43:0/2842820276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f322010b710 0x7f322019acb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f321400d7a0 tx=0x7f321400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.045+0000 7f320f7fe640 1 -- 172.21.15.43:0/2842820276 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214017070 con 0x7f322010b710 2024-04-25T10:35:20.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.045+0000 7f320f7fe640 1 -- 172.21.15.43:0/2842820276 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32140054d0 con 0x7f322010b710 2024-04-25T10:35:20.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.045+0000 7f3224bce640 1 -- 172.21.15.43:0/2842820276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322019f820 con 0x7f322010b710 2024-04-25T10:35:20.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.045+0000 7f320f7fe640 1 -- 172.21.15.43:0/2842820276 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214004c10 con 0x7f322010b710 2024-04-25T10:35:20.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.045+0000 7f3224bce640 1 -- 172.21.15.43:0/2842820276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f322019fd20 con 0x7f322010b710 2024-04-25T10:35:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.048+0000 7f320f7fe640 1 -- 172.21.15.43:0/2842820276 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f3214004df0 con 0x7f322010b710 2024-04-25T10:35:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.048+0000 7f322ba82740 1 Processor -- start 2024-04-25T10:35:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.048+0000 7f322ba82740 1 -- start start 2024-04-25T10:35:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.048+0000 7f322ba82740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e37390 0x563cc41d7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.048+0000 7f322ba82740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e0e4d0 0x563cc41d7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.048+0000 7f322ba82740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e166b0 0x563cc41e8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.048+0000 7f322ba82740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563cc413dd80 con 0x563cc3e0e4d0 2024-04-25T10:35:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.048+0000 7f322ba82740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563cc40f78b0 con 0x563cc3e37390 2024-04-25T10:35:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.048+0000 7f322ba82740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563cc4139d40 con 0x563cc3e166b0 2024-04-25T10:35:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.049+0000 7f321e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e0e4d0 0x563cc41d7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.049+0000 7f321ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e37390 0x563cc41d7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.049+0000 7f321ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e37390 0x563cc41d7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:36554/0 (socket says 172.21.15.43:36554) 2024-04-25T10:35:20.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.049+0000 7f321e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e0e4d0 0x563cc41d7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45274/0 (socket says 172.21.15.43:45274) 2024-04-25T10:35:20.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.049+0000 7f321ed9d640 1 -- 172.21.15.43:0/562297585 learned_addr learned my addr 172.21.15.43:0/562297585 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.049+0000 7f320f7fe640 1 --2- 172.21.15.43:0/2842820276 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f32080afc60 0x7f32080b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.049+0000 7f321dd9b640 1 --2- 172.21.15.43:0/562297585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e166b0 0x563cc41e8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.049+0000 7f321ed9d640 1 -- 172.21.15.43:0/562297585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e166b0 msgr2=0x563cc41e8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.049+0000 7f321ed9d640 1 --2- 172.21.15.43:0/562297585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e166b0 0x563cc41e8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.049+0000 7f321ed9d640 1 -- 172.21.15.43:0/562297585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e0e4d0 msgr2=0x563cc41d7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.049+0000 7f321ed9d640 1 --2- 172.21.15.43:0/562297585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e0e4d0 0x563cc41d7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.049+0000 7f321ed9d640 1 -- 172.21.15.43:0/562297585 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cc40c8a40 con 0x563cc3e37390 2024-04-25T10:35:20.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.049+0000 7f320f7fe640 1 -- 172.21.15.43:0/2842820276 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f32140de9a0 con 0x7f322010b710 2024-04-25T10:35:20.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.049+0000 7f321ed9d640 1 --2- 172.21.15.43:0/2842820276 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f32080afc60 0x7f32080b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.049+0000 7f321ed9d640 1 --2- 172.21.15.43:0/562297585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e37390 0x563cc41d7930 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f321800ad10 tx=0x7f321800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=931ae34d4818bf57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.050+0000 7f320cff9640 1 -- 172.21.15.43:0/562297585 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218012310 con 0x563cc3e37390 2024-04-25T10:35:20.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.050+0000 7f321ed9d640 1 --2- 172.21.15.43:0/2842820276 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f32080afc60 0x7f32080b2170 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3218021000 tx=0x7f321801e000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.050+0000 7f320cff9640 1 -- 172.21.15.43:0/562297585 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3218019820 con 0x563cc3e37390 2024-04-25T10:35:20.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.050+0000 7f320cff9640 1 -- 172.21.15.43:0/562297585 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321800d6c0 con 0x563cc3e37390 2024-04-25T10:35:20.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.050+0000 7f322ba82740 1 -- 172.21.15.43:0/562297585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e37390 msgr2=0x563cc41d7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.050+0000 7f322ba82740 1 --2- 172.21.15.43:0/562297585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e37390 0x563cc41d7930 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f321800ad10 tx=0x7f321800f790 comp rx=0 tx=0).stop 2024-04-25T10:35:20.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.050+0000 7f322ba82740 1 -- 172.21.15.43:0/562297585 shutdown_connections 2024-04-25T10:35:20.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.050+0000 7f322ba82740 1 --2- 172.21.15.43:0/562297585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e166b0 0x563cc41e8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.051+0000 7f322ba82740 1 --2- 172.21.15.43:0/562297585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e0e4d0 0x563cc41d7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.051+0000 7f322ba82740 1 --2- 172.21.15.43:0/562297585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e37390 0x563cc41d7930 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.051+0000 7f322ba82740 1 -- 172.21.15.43:0/562297585 >> 172.21.15.43:0/562297585 conn(0x563cc3e2b4e0 msgr2=0x563cc41d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.051+0000 7f322ba82740 1 -- 172.21.15.43:0/562297585 shutdown_connections 2024-04-25T10:35:20.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.051+0000 7f322ba82740 1 -- 172.21.15.43:0/562297585 wait complete. 2024-04-25T10:35:20.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.051+0000 7f322ba82740 1 -- start start 2024-04-25T10:35:20.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.051+0000 7f322ba82740 1 Processor -- start 2024-04-25T10:35:20.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.051+0000 7f322ba82740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e0e4d0 0x563cc41e6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.051+0000 7f322ba82740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e166b0 0x563cc41e6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.051+0000 7f322ba82740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e37390 0x563cc41ec900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.051+0000 7f322ba82740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563cc41e9480 con 0x563cc3e37390 2024-04-25T10:35:20.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.051+0000 7f322ba82740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563cc40c4d10 con 0x563cc3e166b0 2024-04-25T10:35:20.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.051+0000 7f322ba82740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563cc41e9600 con 0x563cc3e0e4d0 2024-04-25T10:35:20.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.051+0000 7f322ba82740 10 client.0 _ll_register_callbacks cb 0x563cc40f1210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:20.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.052+0000 7f321ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e37390 0x563cc41ec900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.052+0000 7f321dd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e166b0 0x563cc41e6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.052+0000 7f321ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e37390 0x563cc41ec900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45276/0 (socket says 172.21.15.43:45276) 2024-04-25T10:35:20.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.052+0000 7f321ed9d640 1 -- 172.21.15.43:0/2421950878 learned_addr learned my addr 172.21.15.43:0/2421950878 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.052+0000 7f321e59c640 1 --2- 172.21.15.43:0/2421950878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e0e4d0 0x563cc41e6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.052+0000 7f321ed9d640 1 -- 172.21.15.43:0/2421950878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e0e4d0 msgr2=0x563cc41e6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.052+0000 7f321ed9d640 1 --2- 172.21.15.43:0/2421950878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e0e4d0 0x563cc41e6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.052+0000 7f321ed9d640 1 -- 172.21.15.43:0/2421950878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e166b0 msgr2=0x563cc41e6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.052+0000 7f321ed9d640 1 --2- 172.21.15.43:0/2421950878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e166b0 0x563cc41e6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.052+0000 7f321ed9d640 1 -- 172.21.15.43:0/2421950878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cc3f197d0 con 0x563cc3e37390 2024-04-25T10:35:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.052+0000 7f321ed9d640 1 --2- 172.21.15.43:0/2421950878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e37390 0x563cc41ec900 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3218019c00 tx=0x7f3218019c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b07221436821dde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.052+0000 7f3205ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.052+0000 7f3205ffb640 1 -- 172.21.15.43:0/2421950878 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218013e60 con 0x563cc3e37390 2024-04-25T10:35:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.052+0000 7f322ba82740 1 -- 172.21.15.43:0/2421950878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3218019820 con 0x563cc3e37390 2024-04-25T10:35:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.052+0000 7f3205ffb640 1 -- 172.21.15.43:0/2421950878 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f3218019570 con 0x563cc3e37390 2024-04-25T10:35:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.053+0000 7f3205ffb640 1 -- 172.21.15.43:0/2421950878 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3218004600 con 0x563cc3e37390 2024-04-25T10:35:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.053+0000 7f322ba82740 10 client.16215 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.053+0000 7f322ba82740 1 -- 172.21.15.43:0/2421950878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563cc3e4f3f0 con 0x563cc3e37390 2024-04-25T10:35:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.053+0000 7f3205ffb640 1 -- 172.21.15.43:0/2421950878 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3218019820 con 0x563cc3e37390 2024-04-25T10:35:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.053+0000 7f3205ffb640 1 -- 172.21.15.43:0/2421950878 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f321801a8d0 con 0x563cc3e37390 2024-04-25T10:35:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.053+0000 7f322ba82740 10 client.16215 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:20.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.053+0000 7f322ba82740 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:20.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.053+0000 7f322ba82740 20 client.16215 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:20.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.053+0000 7f322ba82740 1 --2- 172.21.15.43:0/2421950878 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563cc3df5f60 0x563cc41f8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.053+0000 7f322ba82740 4 client.16215 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:35:20.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.053+0000 7f322ba82740 1 -- 172.21.15.43:0/2421950878 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cc414b990 con 0x563cc3df5f60 2024-04-25T10:35:20.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.053+0000 7f321e59c640 1 --2- 172.21.15.43:0/2421950878 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563cc3df5f60 0x563cc41f8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.054+0000 7f321e59c640 1 --2- 172.21.15.43:0/2421950878 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563cc3df5f60 0x563cc41f8ab0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=643cf61c2b912728 server_cookie=d24d9ffd83525ffc in_seq=0 out_seq=0 2024-04-25T10:35:20.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.054+0000 7f3205ffb640 10 client.16215 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:20.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.055+0000 7f3205ffb640 1 -- 172.21.15.43:0/2421950878 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563cc414b990 con 0x563cc3df5f60 2024-04-25T10:35:20.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.055+0000 7f3205ffb640 10 client.16215 handle_command_reply: tid=0 2024-04-25T10:35:20.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.055+0000 7f322ba82740 1 client.16215 shutdown 2024-04-25T10:35:20.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.055+0000 7f322ba82740 1 -- 172.21.15.43:0/2421950878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e37390 msgr2=0x563cc41ec900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.055+0000 7f322ba82740 1 --2- 172.21.15.43:0/2421950878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e37390 0x563cc41ec900 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3218019c00 tx=0x7f3218019c30 comp rx=0 tx=0).stop 2024-04-25T10:35:20.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.055+0000 7f322ba82740 1 -- 172.21.15.43:0/2421950878 shutdown_connections 2024-04-25T10:35:20.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.055+0000 7f322ba82740 1 --2- 172.21.15.43:0/2421950878 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563cc3df5f60 0x563cc41f8ab0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.055+0000 7f322ba82740 1 --2- 172.21.15.43:0/2421950878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e166b0 0x563cc41e6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.055+0000 7f322ba82740 1 --2- 172.21.15.43:0/2421950878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e37390 0x563cc41ec900 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.055+0000 7f322ba82740 1 --2- 172.21.15.43:0/2421950878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e0e4d0 0x563cc41e6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.055+0000 7f3205ffb640 0 client.16215 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:20.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.055+0000 7f322ba82740 1 -- 172.21.15.43:0/2421950878 >> 172.21.15.43:0/2421950878 conn(0x563cc3e2b4e0 msgr2=0x563cc41d9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.055+0000 7f322ba82740 1 -- 172.21.15.43:0/2421950878 shutdown_connections 2024-04-25T10:35:20.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.055+0000 7f322ba82740 1 -- 172.21.15.43:0/2421950878 wait complete. 2024-04-25T10:35:20.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.055+0000 7f322ba82740 20 client.16215 trim_cache size 0 max 16384 2024-04-25T10:35:20.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.073+0000 7f322ba82740 1 Processor -- start 2024-04-25T10:35:20.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.073+0000 7f322ba82740 1 -- start start 2024-04-25T10:35:20.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f322ba82740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e0e4d0 0x563cc41d4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f322ba82740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e166b0 0x563cc41d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f322ba82740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e37390 0x563cc41d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f322ba82740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f32201affd0 con 0x563cc3e0e4d0 2024-04-25T10:35:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f322ba82740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f32201ad8a0 con 0x563cc3e37390 2024-04-25T10:35:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f322ba82740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f32201ab710 con 0x563cc3e166b0 2024-04-25T10:35:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f321e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e0e4d0 0x563cc41d4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f321e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e0e4d0 0x563cc41d4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45292/0 (socket says 172.21.15.43:45292) 2024-04-25T10:35:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f321e59c640 1 -- 172.21.15.43:0/1002640863 learned_addr learned my addr 172.21.15.43:0/1002640863 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f321ed9d640 1 --2- 172.21.15.43:0/1002640863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e37390 0x563cc41d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f321e59c640 1 -- 172.21.15.43:0/1002640863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e166b0 msgr2=0x563cc41d30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f321dd9b640 1 --2- 172.21.15.43:0/1002640863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e166b0 0x563cc41d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f321e59c640 1 --2- 172.21.15.43:0/1002640863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e166b0 0x563cc41d30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f321e59c640 1 -- 172.21.15.43:0/1002640863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e37390 msgr2=0x563cc41d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f321e59c640 1 --2- 172.21.15.43:0/1002640863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e37390 0x563cc41d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f321e59c640 1 -- 172.21.15.43:0/1002640863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cc3e0f0f0 con 0x563cc3e0e4d0 2024-04-25T10:35:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f321ed9d640 1 --2- 172.21.15.43:0/1002640863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e37390 0x563cc41d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:35:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f321dd9b640 1 --2- 172.21.15.43:0/1002640863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e166b0 0x563cc41d30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:35:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f321e59c640 1 --2- 172.21.15.43:0/1002640863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e0e4d0 0x563cc41d4240 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3210006010 tx=0x7f321002f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9dff4d5b508c1a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f3207fff640 1 -- 172.21.15.43:0/1002640863 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210034040 con 0x563cc3e0e4d0 2024-04-25T10:35:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f3207fff640 1 -- 172.21.15.43:0/1002640863 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3210003e20 con 0x563cc3e0e4d0 2024-04-25T10:35:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.074+0000 7f3207fff640 1 -- 172.21.15.43:0/1002640863 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210004800 con 0x563cc3e0e4d0 2024-04-25T10:35:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.075+0000 7f322ba82740 1 -- 172.21.15.43:0/1002640863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e0e4d0 msgr2=0x563cc41d4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.075+0000 7f322ba82740 1 --2- 172.21.15.43:0/1002640863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e0e4d0 0x563cc41d4240 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3210006010 tx=0x7f321002f670 comp rx=0 tx=0).stop 2024-04-25T10:35:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.075+0000 7f322ba82740 1 -- 172.21.15.43:0/1002640863 shutdown_connections 2024-04-25T10:35:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.075+0000 7f322ba82740 1 --2- 172.21.15.43:0/1002640863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e37390 0x563cc41d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.075+0000 7f322ba82740 1 --2- 172.21.15.43:0/1002640863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e166b0 0x563cc41d30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.075+0000 7f322ba82740 1 --2- 172.21.15.43:0/1002640863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e0e4d0 0x563cc41d4240 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.075+0000 7f322ba82740 1 -- 172.21.15.43:0/1002640863 >> 172.21.15.43:0/1002640863 conn(0x563cc3e2b4e0 msgr2=0x563cc41d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.075+0000 7f322ba82740 1 -- 172.21.15.43:0/1002640863 shutdown_connections 2024-04-25T10:35:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.075+0000 7f322ba82740 1 -- 172.21.15.43:0/1002640863 wait complete. 2024-04-25T10:35:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.075+0000 7f322ba82740 1 -- start start 2024-04-25T10:35:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f322ba82740 1 Processor -- start 2024-04-25T10:35:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f322ba82740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e0e4d0 0x563cc41ee060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f322ba82740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e166b0 0x563cc41ee5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f322ba82740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e37390 0x563cc425ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f322ba82740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f32201abd60 con 0x563cc3e37390 2024-04-25T10:35:20.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f322ba82740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f32201ad1d0 con 0x563cc3e0e4d0 2024-04-25T10:35:20.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f322ba82740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f32201ac430 con 0x563cc3e166b0 2024-04-25T10:35:20.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f322ba82740 10 client.0 _ll_register_callbacks cb 0x7f3210009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:20.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f321ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e37390 0x563cc425ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f321ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e37390 0x563cc425ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45296/0 (socket says 172.21.15.43:45296) 2024-04-25T10:35:20.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f321ed9d640 1 -- 172.21.15.43:0/3503681333 learned_addr learned my addr 172.21.15.43:0/3503681333 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f321dd9b640 1 --2- 172.21.15.43:0/3503681333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e166b0 0x563cc41ee5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f321e59c640 1 --2- 172.21.15.43:0/3503681333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e0e4d0 0x563cc41ee060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f321ed9d640 1 -- 172.21.15.43:0/3503681333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e166b0 msgr2=0x563cc41ee5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f321ed9d640 1 --2- 172.21.15.43:0/3503681333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e166b0 0x563cc41ee5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f321ed9d640 1 -- 172.21.15.43:0/3503681333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e0e4d0 msgr2=0x563cc41ee060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f321ed9d640 1 --2- 172.21.15.43:0/3503681333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e0e4d0 0x563cc41ee060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f321ed9d640 1 -- 172.21.15.43:0/3503681333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cc40fe4b0 con 0x563cc3e37390 2024-04-25T10:35:20.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f321e59c640 1 --2- 172.21.15.43:0/3503681333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e0e4d0 0x563cc41ee060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:20.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f321dd9b640 1 --2- 172.21.15.43:0/3503681333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e166b0 0x563cc41ee5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:20.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f321ed9d640 1 --2- 172.21.15.43:0/3503681333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e37390 0x563cc425ece0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3218001eb0 tx=0x7f321800aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d867e3dc37a521b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.076+0000 7f3205ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:20.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.077+0000 7f3205ffb640 1 -- 172.21.15.43:0/3503681333 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321803c030 con 0x563cc3e37390 2024-04-25T10:35:20.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.077+0000 7f3205ffb640 1 -- 172.21.15.43:0/3503681333 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f321801a9f0 con 0x563cc3e37390 2024-04-25T10:35:20.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.077+0000 7f322ba82740 1 -- 172.21.15.43:0/3503681333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3210003e20 con 0x563cc3e37390 2024-04-25T10:35:20.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.077+0000 7f3205ffb640 1 -- 172.21.15.43:0/3503681333 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3218046050 con 0x563cc3e37390 2024-04-25T10:35:20.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.077+0000 7f322ba82740 10 client.16221 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:20.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.077+0000 7f322ba82740 1 -- 172.21.15.43:0/3503681333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563cc42600c0 con 0x563cc3e37390 2024-04-25T10:35:20.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.077+0000 7f3205ffb640 1 -- 172.21.15.43:0/3503681333 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3210003e20 con 0x563cc3e37390 2024-04-25T10:35:20.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.077+0000 7f3205ffb640 1 -- 172.21.15.43:0/3503681333 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f321802f4f0 con 0x563cc3e37390 2024-04-25T10:35:20.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.077+0000 7f322ba82740 10 client.16221 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:20.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.077+0000 7f322ba82740 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:20.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.077+0000 7f322ba82740 20 client.16221 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:20.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.077+0000 7f322ba82740 1 --2- 172.21.15.43:0/3503681333 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563cc3dcb3a0 0x563cc42628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.077+0000 7f322ba82740 4 client.16221 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:35:20.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.077+0000 7f322ba82740 1 -- 172.21.15.43:0/3503681333 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563cc4145800 con 0x563cc3dcb3a0 2024-04-25T10:35:20.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.078+0000 7f321e59c640 1 --2- 172.21.15.43:0/3503681333 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563cc3dcb3a0 0x563cc42628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.078+0000 7f3205ffb640 10 client.16221 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:20.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.078+0000 7f321e59c640 1 --2- 172.21.15.43:0/3503681333 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563cc3dcb3a0 0x563cc42628a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2ee995e5179896 server_cookie=5bffcf167eebc4f8 in_seq=0 out_seq=0 2024-04-25T10:35:20.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.078+0000 7f3205ffb640 1 -- 172.21.15.43:0/3503681333 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563cc4145800 con 0x563cc3dcb3a0 2024-04-25T10:35:20.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.078+0000 7f3205ffb640 10 client.16221 handle_command_reply: tid=0 2024-04-25T10:35:20.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.078+0000 7f322ba82740 1 client.16221 shutdown 2024-04-25T10:35:20.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.078+0000 7f322ba82740 1 -- 172.21.15.43:0/3503681333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e37390 msgr2=0x563cc425ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.078+0000 7f322ba82740 1 --2- 172.21.15.43:0/3503681333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e37390 0x563cc425ece0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3218001eb0 tx=0x7f321800aca0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.078+0000 7f322ba82740 1 -- 172.21.15.43:0/3503681333 shutdown_connections 2024-04-25T10:35:20.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.078+0000 7f322ba82740 1 --2- 172.21.15.43:0/3503681333 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563cc3dcb3a0 0x563cc42628a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.078+0000 7f322ba82740 1 --2- 172.21.15.43:0/3503681333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563cc3e37390 0x563cc425ece0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.078+0000 7f3205ffb640 0 client.16221 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:20.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.078+0000 7f322ba82740 1 --2- 172.21.15.43:0/3503681333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563cc3e166b0 0x563cc41ee5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.078+0000 7f322ba82740 1 --2- 172.21.15.43:0/3503681333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cc3e0e4d0 0x563cc41ee060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.078+0000 7f322ba82740 1 -- 172.21.15.43:0/3503681333 >> 172.21.15.43:0/3503681333 conn(0x563cc3e2b4e0 msgr2=0x563cc41d33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.078+0000 7f322ba82740 1 -- 172.21.15.43:0/3503681333 shutdown_connections 2024-04-25T10:35:20.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.078+0000 7f322ba82740 1 -- 172.21.15.43:0/3503681333 wait complete. 2024-04-25T10:35:20.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.079+0000 7f322ba82740 20 client.16221 trim_cache size 0 max 16384 2024-04-25T10:35:20.089 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:35:20.089 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:35:20.089 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:35:20.089 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:35:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.079+0000 7f3224bce640 1 -- 172.21.15.43:0/2842820276 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f32080afc60 msgr2=0x7f32080b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.079+0000 7f3224bce640 1 --2- 172.21.15.43:0/2842820276 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f32080afc60 0x7f32080b2170 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3218021000 tx=0x7f321801e000 comp rx=0 tx=0).stop 2024-04-25T10:35:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.079+0000 7f3224bce640 1 -- 172.21.15.43:0/2842820276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f322010b710 msgr2=0x7f322019acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.079+0000 7f3224bce640 1 --2- 172.21.15.43:0/2842820276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f322010b710 0x7f322019acb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f321400d7a0 tx=0x7f321400dc60 comp rx=0 tx=0).stop 2024-04-25T10:35:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.079+0000 7f3224bce640 1 -- 172.21.15.43:0/2842820276 shutdown_connections 2024-04-25T10:35:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.079+0000 7f3224bce640 1 --2- 172.21.15.43:0/2842820276 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f32080afc60 0x7f32080b2170 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.079+0000 7f3224bce640 1 --2- 172.21.15.43:0/2842820276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f322010e070 0x7f322019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.079+0000 7f3224bce640 1 --2- 172.21.15.43:0/2842820276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f322010b710 0x7f322019acb0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.079+0000 7f3224bce640 1 --2- 172.21.15.43:0/2842820276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3220104cb0 0x7f322019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.079+0000 7f3224bce640 1 -- 172.21.15.43:0/2842820276 >> 172.21.15.43:0/2842820276 conn(0x7f3220078100 msgr2=0x7f3220101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.079+0000 7f3224bce640 1 -- 172.21.15.43:0/2842820276 shutdown_connections 2024-04-25T10:35:20.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.079+0000 7f3224bce640 1 -- 172.21.15.43:0/2842820276 wait complete. 2024-04-25T10:35:20.093 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:35:20.093 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:35:20.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.203+0000 7f70f8572640 1 -- 172.21.15.43:0/2149683828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f0108a20 msgr2=0x7f70f01025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.203+0000 7f70f8572640 1 --2- 172.21.15.43:0/2149683828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f0108a20 0x7f70f01025b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009900 tx=0x7f70e802a410 comp rx=0 tx=0).stop 2024-04-25T10:35:20.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.203+0000 7f70f8572640 1 -- 172.21.15.43:0/2149683828 shutdown_connections 2024-04-25T10:35:20.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.203+0000 7f70f8572640 1 --2- 172.21.15.43:0/2149683828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70f0102af0 0x7f70f0102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.203+0000 7f70f8572640 1 --2- 172.21.15.43:0/2149683828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70f010d380 0x7f70f010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.203+0000 7f70f8572640 1 --2- 172.21.15.43:0/2149683828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f0108a20 0x7f70f01025b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.203+0000 7f70f8572640 1 -- 172.21.15.43:0/2149683828 >> 172.21.15.43:0/2149683828 conn(0x7f70f0078110 msgr2=0x7f70f0078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.203+0000 7f70f8572640 1 -- 172.21.15.43:0/2149683828 shutdown_connections 2024-04-25T10:35:20.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.203+0000 7f70f8572640 1 -- 172.21.15.43:0/2149683828 wait complete. 2024-04-25T10:35:20.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.204+0000 7f70f8572640 1 Processor -- start 2024-04-25T10:35:20.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.204+0000 7f70f8572640 1 -- start start 2024-04-25T10:35:20.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.204+0000 7f70f8572640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70f0102af0 0x7f70f019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.204+0000 7f70f8572640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70f0108a20 0x7f70f019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.204+0000 7f70f8572640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f010d380 0x7f70f019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.204+0000 7f70f8572640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f70f0112be0 con 0x7f70f010d380 2024-04-25T10:35:20.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.204+0000 7f70f8572640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f70f0112a60 con 0x7f70f0102af0 2024-04-25T10:35:20.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.204+0000 7f70f8572640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f70f0112d60 con 0x7f70f0108a20 2024-04-25T10:35:20.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.204+0000 7f70f6b0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f010d380 0x7f70f019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.204+0000 7f70f6b0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f010d380 0x7f70f019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45320/0 (socket says 172.21.15.43:45320) 2024-04-25T10:35:20.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.204+0000 7f70f6b0f640 1 -- 172.21.15.43:0/1921821514 learned_addr learned my addr 172.21.15.43:0/1921821514 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.204+0000 7f70f630e640 1 --2- 172.21.15.43:0/1921821514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70f0102af0 0x7f70f019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.204+0000 7f70f5b0d640 1 --2- 172.21.15.43:0/1921821514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70f0108a20 0x7f70f019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.204+0000 7f70f6b0f640 1 -- 172.21.15.43:0/1921821514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70f0108a20 msgr2=0x7f70f019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.204+0000 7f70f6b0f640 1 --2- 172.21.15.43:0/1921821514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70f0108a20 0x7f70f019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.205+0000 7f70f6b0f640 1 -- 172.21.15.43:0/1921821514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70f0102af0 msgr2=0x7f70f019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.205+0000 7f70f6b0f640 1 --2- 172.21.15.43:0/1921821514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70f0102af0 0x7f70f019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.205+0000 7f70f6b0f640 1 -- 172.21.15.43:0/1921821514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f019f530 con 0x7f70f010d380 2024-04-25T10:35:20.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.205+0000 7f70f6b0f640 1 --2- 172.21.15.43:0/1921821514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f010d380 0x7f70f019eef0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f70e000eaa0 tx=0x7f70e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.205+0000 7f70ef7fe640 1 -- 172.21.15.43:0/1921821514 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e000cdb0 con 0x7f70f010d380 2024-04-25T10:35:20.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.205+0000 7f70f8572640 1 -- 172.21.15.43:0/1921821514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f019f820 con 0x7f70f010d380 2024-04-25T10:35:20.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.205+0000 7f70ef7fe640 1 -- 172.21.15.43:0/1921821514 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70e0005020 con 0x7f70f010d380 2024-04-25T10:35:20.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.205+0000 7f70ef7fe640 1 -- 172.21.15.43:0/1921821514 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e00116d0 con 0x7f70f010d380 2024-04-25T10:35:20.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.205+0000 7f70f8572640 1 -- 172.21.15.43:0/1921821514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f01a71a0 con 0x7f70f010d380 2024-04-25T10:35:20.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.206+0000 7f70f8572640 1 -- 172.21.15.43:0/1921821514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f006afa0 con 0x7f70f010d380 2024-04-25T10:35:20.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.208+0000 7f70ef7fe640 1 -- 172.21.15.43:0/1921821514 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f70e00054a0 con 0x7f70f010d380 2024-04-25T10:35:20.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.208+0000 7f70ef7fe640 1 --2- 172.21.15.43:0/1921821514 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f70d80afc60 0x7f70d80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.208+0000 7f70ef7fe640 1 -- 172.21.15.43:0/1921821514 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f70e00df5a0 con 0x7f70f010d380 2024-04-25T10:35:20.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.208+0000 7f70f630e640 1 --2- 172.21.15.43:0/1921821514 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f70d80afc60 0x7f70d80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.209+0000 7f70f630e640 1 --2- 172.21.15.43:0/1921821514 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f70d80afc60 0x7f70d80b2170 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f70e8000c00 tx=0x7f70e8031400 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.212+0000 7f70ef7fe640 1 -- 172.21.15.43:0/1921821514 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f70e0016030 con 0x7f70f010d380 2024-04-25T10:35:20.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.440+0000 7f70f8572640 1 -- 172.21.15.43:0/1921821514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70f01a7370 con 0x7f70f010d380 2024-04-25T10:35:20.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.440+0000 7f70ef7fe640 1 -- 172.21.15.43:0/1921821514 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7f70e00a7e70 con 0x7f70f010d380 2024-04-25T10:35:20.443 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:35:20.443 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:35:20.443 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:35:20.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.442+0000 7f70f8572640 1 -- 172.21.15.43:0/1921821514 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f70d80afc60 msgr2=0x7f70d80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.442+0000 7f70f8572640 1 --2- 172.21.15.43:0/1921821514 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f70d80afc60 0x7f70d80b2170 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f70e8000c00 tx=0x7f70e8031400 comp rx=0 tx=0).stop 2024-04-25T10:35:20.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.442+0000 7f70f8572640 1 -- 172.21.15.43:0/1921821514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f010d380 msgr2=0x7f70f019eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.442+0000 7f70f8572640 1 --2- 172.21.15.43:0/1921821514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f010d380 0x7f70f019eef0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f70e000eaa0 tx=0x7f70e000ef60 comp rx=0 tx=0).stop 2024-04-25T10:35:20.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.442+0000 7f70f8572640 1 -- 172.21.15.43:0/1921821514 shutdown_connections 2024-04-25T10:35:20.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.442+0000 7f70f8572640 1 --2- 172.21.15.43:0/1921821514 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f70d80afc60 0x7f70d80b2170 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.442+0000 7f70f8572640 1 --2- 172.21.15.43:0/1921821514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f010d380 0x7f70f019eef0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.442+0000 7f70f8572640 1 --2- 172.21.15.43:0/1921821514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70f0108a20 0x7f70f019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.442+0000 7f70f8572640 1 --2- 172.21.15.43:0/1921821514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70f0102af0 0x7f70f019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.442+0000 7f70f8572640 1 -- 172.21.15.43:0/1921821514 >> 172.21.15.43:0/1921821514 conn(0x7f70f0078110 msgr2=0x7f70f0069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.443+0000 7f70f8572640 1 -- 172.21.15.43:0/1921821514 shutdown_connections 2024-04-25T10:35:20.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.443+0000 7f70f8572640 1 -- 172.21.15.43:0/1921821514 wait complete. 2024-04-25T10:35:20.456 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:35:20.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.569+0000 7f397fb08640 1 -- 172.21.15.43:0/2651743529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3978109b30 msgr2=0x7f3978103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.569+0000 7f397fb08640 1 --2- 172.21.15.43:0/2651743529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3978109b30 0x7f3978103720 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3974009900 tx=0x7f397402a410 comp rx=0 tx=0).stop 2024-04-25T10:35:20.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.569+0000 7f397fb08640 1 -- 172.21.15.43:0/2651743529 shutdown_connections 2024-04-25T10:35:20.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.569+0000 7f397fb08640 1 --2- 172.21.15.43:0/2651743529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3978109fd0 0x7f3978103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.569+0000 7f397fb08640 1 --2- 172.21.15.43:0/2651743529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39781042d0 0x7f3978110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.569+0000 7f397fb08640 1 --2- 172.21.15.43:0/2651743529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3978109b30 0x7f3978103720 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.569+0000 7f397fb08640 1 -- 172.21.15.43:0/2651743529 >> 172.21.15.43:0/2651743529 conn(0x7f3978079280 msgr2=0x7f39780796c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.569+0000 7f397fb08640 1 -- 172.21.15.43:0/2651743529 shutdown_connections 2024-04-25T10:35:20.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.569+0000 7f397fb08640 1 -- 172.21.15.43:0/2651743529 wait complete. 2024-04-25T10:35:20.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.570+0000 7f397fb08640 1 Processor -- start 2024-04-25T10:35:20.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.570+0000 7f397fb08640 1 -- start start 2024-04-25T10:35:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.570+0000 7f397fb08640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39781042d0 0x7f3978074ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.570+0000 7f397fb08640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3978109b30 0x7f39780753e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.570+0000 7f397fb08640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3978109fd0 0x7f3978075920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.570+0000 7f397fb08640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3978113d20 con 0x7f3978109b30 2024-04-25T10:35:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.570+0000 7f397fb08640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3978113ba0 con 0x7f39781042d0 2024-04-25T10:35:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.570+0000 7f397fb08640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3978113ea0 con 0x7f3978109fd0 2024-04-25T10:35:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.570+0000 7f397d0a3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3978109b30 0x7f39780753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.570+0000 7f397d0a3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3978109b30 0x7f39780753e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45326/0 (socket says 172.21.15.43:45326) 2024-04-25T10:35:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.570+0000 7f397d0a3640 1 -- 172.21.15.43:0/589612273 learned_addr learned my addr 172.21.15.43:0/589612273 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.570+0000 7f397d8a4640 1 --2- 172.21.15.43:0/589612273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39781042d0 0x7f3978074ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.571+0000 7f397e0a5640 1 --2- 172.21.15.43:0/589612273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3978109fd0 0x7f3978075920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.571+0000 7f397d0a3640 1 -- 172.21.15.43:0/589612273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3978109fd0 msgr2=0x7f3978075920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.571+0000 7f397d0a3640 1 --2- 172.21.15.43:0/589612273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3978109fd0 0x7f3978075920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.571+0000 7f397d0a3640 1 -- 172.21.15.43:0/589612273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39781042d0 msgr2=0x7f3978074ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.571+0000 7f397d0a3640 1 --2- 172.21.15.43:0/589612273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39781042d0 0x7f3978074ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.571+0000 7f397d0a3640 1 -- 172.21.15.43:0/589612273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39781a8ac0 con 0x7f3978109b30 2024-04-25T10:35:20.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.571+0000 7f397d8a4640 1 --2- 172.21.15.43:0/589612273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39781042d0 0x7f3978074ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:20.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.571+0000 7f397d0a3640 1 --2- 172.21.15.43:0/589612273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3978109b30 0x7f39780753e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f396800e9f0 tx=0x7f396800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.571+0000 7f396effd640 1 -- 172.21.15.43:0/589612273 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396800ccc0 con 0x7f3978109b30 2024-04-25T10:35:20.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.571+0000 7f396effd640 1 -- 172.21.15.43:0/589612273 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39680035f0 con 0x7f3978109b30 2024-04-25T10:35:20.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.571+0000 7f397fb08640 1 -- 172.21.15.43:0/589612273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39781a8db0 con 0x7f3978109b30 2024-04-25T10:35:20.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.571+0000 7f396effd640 1 -- 172.21.15.43:0/589612273 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396800ce60 con 0x7f3978109b30 2024-04-25T10:35:20.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.571+0000 7f397fb08640 1 -- 172.21.15.43:0/589612273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39781a9270 con 0x7f3978109b30 2024-04-25T10:35:20.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.574+0000 7f396effd640 1 -- 172.21.15.43:0/589612273 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f3968003aa0 con 0x7f3978109b30 2024-04-25T10:35:20.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.574+0000 7f39869b7740 1 Processor -- start 2024-04-25T10:35:20.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.574+0000 7f39869b7740 1 -- start start 2024-04-25T10:35:20.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.574+0000 7f39869b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659d6390 0x563265d77810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.574+0000 7f39869b7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632659ad4d0 0x563265d77d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.574+0000 7f39869b7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659b56b0 0x563265d846a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.574+0000 7f39869b7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563265cdcd80 con 0x5632659b56b0 2024-04-25T10:35:20.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.574+0000 7f39869b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563265c96a20 con 0x5632659d6390 2024-04-25T10:35:20.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.574+0000 7f39869b7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563265cd8d40 con 0x5632659ad4d0 2024-04-25T10:35:20.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.575+0000 7f397d0a3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659b56b0 0x563265d846a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.575+0000 7f397e0a5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659d6390 0x563265d77810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.575+0000 7f396effd640 1 --2- 172.21.15.43:0/589612273 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f39640afc60 0x7f39640b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.575+0000 7f397e0a5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659d6390 0x563265d77810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:36638/0 (socket says 172.21.15.43:36638) 2024-04-25T10:35:20.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.575+0000 7f397e0a5640 1 -- 172.21.15.43:0/4026794215 learned_addr learned my addr 172.21.15.43:0/4026794215 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.575+0000 7f397d8a4640 1 --2- 172.21.15.43:0/4026794215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632659ad4d0 0x563265d77d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.575+0000 7f397e0a5640 1 --2- 172.21.15.43:0/589612273 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f39640afc60 0x7f39640b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.575+0000 7f397e0a5640 1 -- 172.21.15.43:0/4026794215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632659ad4d0 msgr2=0x563265d77d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.575+0000 7f396effd640 1 -- 172.21.15.43:0/589612273 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f39680df3f0 con 0x7f3978109b30 2024-04-25T10:35:20.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.575+0000 7f397e0a5640 1 --2- 172.21.15.43:0/4026794215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632659ad4d0 0x563265d77d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.575+0000 7f397e0a5640 1 -- 172.21.15.43:0/4026794215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659b56b0 msgr2=0x563265d846a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.575+0000 7f397e0a5640 1 --2- 172.21.15.43:0/4026794215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659b56b0 0x563265d846a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.575+0000 7f397e0a5640 1 -- 172.21.15.43:0/4026794215 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563265ca1c30 con 0x5632659d6390 2024-04-25T10:35:20.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.575+0000 7f397e0a5640 1 --2- 172.21.15.43:0/4026794215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659d6390 0x563265d77810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3970013a70 tx=0x7f3970013f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2fa37c2a6df5a16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.576+0000 7f397e0a5640 1 --2- 172.21.15.43:0/589612273 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f39640afc60 0x7f39640b2170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f397001b000 tx=0x7f397001d000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.576+0000 7f3963fff640 1 -- 172.21.15.43:0/4026794215 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970022070 con 0x5632659d6390 2024-04-25T10:35:20.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.576+0000 7f3963fff640 1 -- 172.21.15.43:0/4026794215 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3970011e00 con 0x5632659d6390 2024-04-25T10:35:20.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.576+0000 7f3963fff640 1 -- 172.21.15.43:0/4026794215 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397001ddb0 con 0x5632659d6390 2024-04-25T10:35:20.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.576+0000 7f39869b7740 1 -- 172.21.15.43:0/4026794215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659d6390 msgr2=0x563265d77810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.576+0000 7f39869b7740 1 --2- 172.21.15.43:0/4026794215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659d6390 0x563265d77810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3970013a70 tx=0x7f3970013f30 comp rx=0 tx=0).stop 2024-04-25T10:35:20.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.577+0000 7f39869b7740 1 -- 172.21.15.43:0/4026794215 shutdown_connections 2024-04-25T10:35:20.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.577+0000 7f39869b7740 1 --2- 172.21.15.43:0/4026794215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659b56b0 0x563265d846a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.577+0000 7f39869b7740 1 --2- 172.21.15.43:0/4026794215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632659ad4d0 0x563265d77d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.577+0000 7f39869b7740 1 --2- 172.21.15.43:0/4026794215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659d6390 0x563265d77810 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.577+0000 7f39869b7740 1 -- 172.21.15.43:0/4026794215 >> 172.21.15.43:0/4026794215 conn(0x5632659ca4e0 msgr2=0x563265d73fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.577+0000 7f39869b7740 1 -- 172.21.15.43:0/4026794215 shutdown_connections 2024-04-25T10:35:20.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.577+0000 7f39869b7740 1 -- 172.21.15.43:0/4026794215 wait complete. 2024-04-25T10:35:20.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.577+0000 7f39869b7740 1 -- start start 2024-04-25T10:35:20.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.577+0000 7f39869b7740 1 Processor -- start 2024-04-25T10:35:20.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.578+0000 7f39869b7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632659ad4d0 0x563265d82230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.578+0000 7f39869b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659b56b0 0x563265d82770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.578+0000 7f39869b7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659d6390 0x563265d8e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.578+0000 7f39869b7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563265d84d30 con 0x5632659d6390 2024-04-25T10:35:20.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.578+0000 7f39869b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563265c63dc0 con 0x5632659b56b0 2024-04-25T10:35:20.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.578+0000 7f39869b7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563265d84eb0 con 0x5632659ad4d0 2024-04-25T10:35:20.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.578+0000 7f39869b7740 10 client.0 _ll_register_callbacks cb 0x563265c7f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:20.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.578+0000 7f397e0a5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659d6390 0x563265d8e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.578+0000 7f397e0a5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659d6390 0x563265d8e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45344/0 (socket says 172.21.15.43:45344) 2024-04-25T10:35:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.578+0000 7f397e0a5640 1 -- 172.21.15.43:0/624309340 learned_addr learned my addr 172.21.15.43:0/624309340 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.578+0000 7f397d0a3640 1 --2- 172.21.15.43:0/624309340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659b56b0 0x563265d82770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.578+0000 7f397d8a4640 1 --2- 172.21.15.43:0/624309340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632659ad4d0 0x563265d82230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.578+0000 7f397e0a5640 1 -- 172.21.15.43:0/624309340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632659ad4d0 msgr2=0x563265d82230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.578+0000 7f397e0a5640 1 --2- 172.21.15.43:0/624309340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632659ad4d0 0x563265d82230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.578+0000 7f397e0a5640 1 -- 172.21.15.43:0/624309340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659b56b0 msgr2=0x563265d82770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.578+0000 7f397e0a5640 1 --2- 172.21.15.43:0/624309340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659b56b0 0x563265d82770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.578+0000 7f397e0a5640 1 -- 172.21.15.43:0/624309340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632659ee3f0 con 0x5632659d6390 2024-04-25T10:35:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.578+0000 7f397e0a5640 1 --2- 172.21.15.43:0/624309340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659d6390 0x563265d8e1b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f397001cfa0 tx=0x7f3970009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426d8f4bc400590f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.578+0000 7f39617fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.578+0000 7f39617fa640 1 -- 172.21.15.43:0/624309340 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970022030 con 0x5632659d6390 2024-04-25T10:35:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.579+0000 7f39617fa640 1 -- 172.21.15.43:0/624309340 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f397001db50 con 0x5632659d6390 2024-04-25T10:35:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.579+0000 7f39869b7740 1 -- 172.21.15.43:0/624309340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3970011e00 con 0x5632659d6390 2024-04-25T10:35:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.579+0000 7f39617fa640 1 -- 172.21.15.43:0/624309340 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f397001dd50 con 0x5632659d6390 2024-04-25T10:35:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.579+0000 7f39869b7740 10 client.16239 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.579+0000 7f39869b7740 1 -- 172.21.15.43:0/624309340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563265ab87d0 con 0x5632659d6390 2024-04-25T10:35:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.579+0000 7f39617fa640 1 -- 172.21.15.43:0/624309340 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3970011e00 con 0x5632659d6390 2024-04-25T10:35:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.579+0000 7f39617fa640 1 -- 172.21.15.43:0/624309340 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3970005040 con 0x5632659d6390 2024-04-25T10:35:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.579+0000 7f39869b7740 10 client.16239 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.579+0000 7f39869b7740 10 client.16239 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:35:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.579+0000 7f39869b7740 20 client.16239 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.579+0000 7f39869b7740 1 --2- 172.21.15.43:0/624309340 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x563265994f60 0x563265d944b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.579+0000 7f39869b7740 4 client.16239 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:35:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.579+0000 7f39869b7740 1 -- 172.21.15.43:0/624309340 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563265cea990 con 0x563265994f60 2024-04-25T10:35:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.579+0000 7f397d8a4640 1 --2- 172.21.15.43:0/624309340 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x563265994f60 0x563265d944b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.580+0000 7f397d8a4640 1 --2- 172.21.15.43:0/624309340 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x563265994f60 0x563265d944b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a6a5d87709db931 server_cookie=76cc63b9ee1ec3de in_seq=0 out_seq=0 2024-04-25T10:35:20.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.580+0000 7f39617fa640 10 client.16239 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:35:20.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.580+0000 7f39617fa640 1 -- 172.21.15.43:0/624309340 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563265cea990 con 0x563265994f60 2024-04-25T10:35:20.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.580+0000 7f39617fa640 10 client.16239 handle_command_reply: tid=0 2024-04-25T10:35:20.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.581+0000 7f39869b7740 1 client.16239 shutdown 2024-04-25T10:35:20.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.581+0000 7f39869b7740 1 -- 172.21.15.43:0/624309340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659d6390 msgr2=0x563265d8e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.581+0000 7f39869b7740 1 --2- 172.21.15.43:0/624309340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659d6390 0x563265d8e1b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f397001cfa0 tx=0x7f3970009320 comp rx=0 tx=0).stop 2024-04-25T10:35:20.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.581+0000 7f39869b7740 1 -- 172.21.15.43:0/624309340 shutdown_connections 2024-04-25T10:35:20.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.581+0000 7f39869b7740 1 --2- 172.21.15.43:0/624309340 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x563265994f60 0x563265d944b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.581+0000 7f39869b7740 1 --2- 172.21.15.43:0/624309340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659b56b0 0x563265d82770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.581+0000 7f39869b7740 1 --2- 172.21.15.43:0/624309340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659d6390 0x563265d8e1b0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.581+0000 7f39869b7740 1 --2- 172.21.15.43:0/624309340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632659ad4d0 0x563265d82230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.581+0000 7f39617fa640 0 client.16239 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:35:20.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.581+0000 7f39869b7740 1 -- 172.21.15.43:0/624309340 >> 172.21.15.43:0/624309340 conn(0x5632659ca4e0 msgr2=0x563265d79530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.581+0000 7f39869b7740 1 -- 172.21.15.43:0/624309340 shutdown_connections 2024-04-25T10:35:20.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.581+0000 7f39869b7740 1 -- 172.21.15.43:0/624309340 wait complete. 2024-04-25T10:35:20.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.581+0000 7f39869b7740 20 client.16239 trim_cache size 0 max 16384 2024-04-25T10:35:20.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.594+0000 7f39869b7740 1 Processor -- start 2024-04-25T10:35:20.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.594+0000 7f39869b7740 1 -- start start 2024-04-25T10:35:20.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.595+0000 7f39869b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659ad4d0 0x563265d8e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.595+0000 7f39869b7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659b56b0 0x563265d80ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.595+0000 7f39869b7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632659d6390 0x563265d7c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.595+0000 7f39869b7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f39781b44b0 con 0x5632659b56b0 2024-04-25T10:35:20.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.595+0000 7f39869b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f39781b1d80 con 0x5632659ad4d0 2024-04-25T10:35:20.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.595+0000 7f39869b7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f39781afbd0 con 0x5632659d6390 2024-04-25T10:35:20.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.595+0000 7f397d0a3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659b56b0 0x563265d80ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.595+0000 7f397d0a3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659b56b0 0x563265d80ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45358/0 (socket says 172.21.15.43:45358) 2024-04-25T10:35:20.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.595+0000 7f397d0a3640 1 -- 172.21.15.43:0/2285343631 learned_addr learned my addr 172.21.15.43:0/2285343631 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.595+0000 7f397d0a3640 1 -- 172.21.15.43:0/2285343631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632659d6390 msgr2=0x563265d7c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.595+0000 7f397d0a3640 1 --2- 172.21.15.43:0/2285343631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632659d6390 0x563265d7c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.595+0000 7f397d0a3640 1 -- 172.21.15.43:0/2285343631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659ad4d0 msgr2=0x563265d8e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.595+0000 7f397d8a4640 1 --2- 172.21.15.43:0/2285343631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659ad4d0 0x563265d8e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.595+0000 7f397d0a3640 1 --2- 172.21.15.43:0/2285343631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659ad4d0 0x563265d8e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.595+0000 7f397d0a3640 1 -- 172.21.15.43:0/2285343631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563265d80570 con 0x5632659b56b0 2024-04-25T10:35:20.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.595+0000 7f397d0a3640 1 --2- 172.21.15.43:0/2285343631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659b56b0 0x563265d80ed0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f396800c6a0 tx=0x7f3968003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3879c084a38060 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.595+0000 7f39637fe640 1 -- 172.21.15.43:0/2285343631 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39680900b0 con 0x5632659b56b0 2024-04-25T10:35:20.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.595+0000 7f39637fe640 1 -- 172.21.15.43:0/2285343631 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39680a70d0 con 0x5632659b56b0 2024-04-25T10:35:20.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.595+0000 7f39637fe640 1 -- 172.21.15.43:0/2285343631 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39680a3850 con 0x5632659b56b0 2024-04-25T10:35:20.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.596+0000 7f39869b7740 1 -- 172.21.15.43:0/2285343631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659b56b0 msgr2=0x563265d80ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.596+0000 7f39869b7740 1 --2- 172.21.15.43:0/2285343631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659b56b0 0x563265d80ed0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f396800c6a0 tx=0x7f3968003a10 comp rx=0 tx=0).stop 2024-04-25T10:35:20.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.596+0000 7f39869b7740 1 -- 172.21.15.43:0/2285343631 shutdown_connections 2024-04-25T10:35:20.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.596+0000 7f39869b7740 1 --2- 172.21.15.43:0/2285343631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632659d6390 0x563265d7c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.596+0000 7f39869b7740 1 --2- 172.21.15.43:0/2285343631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659b56b0 0x563265d80ed0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.596+0000 7f39869b7740 1 --2- 172.21.15.43:0/2285343631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659ad4d0 0x563265d8e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.596+0000 7f39869b7740 1 -- 172.21.15.43:0/2285343631 >> 172.21.15.43:0/2285343631 conn(0x5632659ca4e0 msgr2=0x563265d807c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.596+0000 7f39869b7740 1 -- 172.21.15.43:0/2285343631 shutdown_connections 2024-04-25T10:35:20.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.596+0000 7f39869b7740 1 -- 172.21.15.43:0/2285343631 wait complete. 2024-04-25T10:35:20.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.597+0000 7f39869b7740 1 -- start start 2024-04-25T10:35:20.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.597+0000 7f39869b7740 1 Processor -- start 2024-04-25T10:35:20.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.597+0000 7f39869b7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659ad4d0 0x563265d97710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.597+0000 7f39869b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659b56b0 0x563265d94a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.597+0000 7f39869b7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632659d6390 0x563265d94fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.597+0000 7f39869b7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f39781b0240 con 0x5632659ad4d0 2024-04-25T10:35:20.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.597+0000 7f39869b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f39781b16b0 con 0x5632659b56b0 2024-04-25T10:35:20.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.597+0000 7f39869b7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f39781b0910 con 0x5632659d6390 2024-04-25T10:35:20.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.597+0000 7f39869b7740 10 client.0 _ll_register_callbacks cb 0x7f3970020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:20.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.597+0000 7f397d8a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659ad4d0 0x563265d97710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.597+0000 7f397d8a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659ad4d0 0x563265d97710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45366/0 (socket says 172.21.15.43:45366) 2024-04-25T10:35:20.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.597+0000 7f397d8a4640 1 -- 172.21.15.43:0/631451017 learned_addr learned my addr 172.21.15.43:0/631451017 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.597+0000 7f397d0a3640 1 --2- 172.21.15.43:0/631451017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659b56b0 0x563265d94a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.597+0000 7f397e0a5640 1 --2- 172.21.15.43:0/631451017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632659d6390 0x563265d94fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.598+0000 7f397d8a4640 1 -- 172.21.15.43:0/631451017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632659d6390 msgr2=0x563265d94fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.598+0000 7f397d8a4640 1 --2- 172.21.15.43:0/631451017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632659d6390 0x563265d94fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.598+0000 7f397d8a4640 1 -- 172.21.15.43:0/631451017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659b56b0 msgr2=0x563265d94a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.598+0000 7f397d8a4640 1 --2- 172.21.15.43:0/631451017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659b56b0 0x563265d94a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.598+0000 7f397d8a4640 1 -- 172.21.15.43:0/631451017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563265d95500 con 0x5632659ad4d0 2024-04-25T10:35:20.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.598+0000 7f397d0a3640 1 --2- 172.21.15.43:0/631451017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659b56b0 0x563265d94a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:20.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.598+0000 7f397d8a4640 1 --2- 172.21.15.43:0/631451017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659ad4d0 0x563265d97710 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f397807a310 tx=0x7f39740041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa522098e26f8a21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.598+0000 7f39617fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:20.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.598+0000 7f39617fa640 1 -- 172.21.15.43:0/631451017 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974035040 con 0x5632659ad4d0 2024-04-25T10:35:20.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.598+0000 7f39617fa640 1 -- 172.21.15.43:0/631451017 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f3974005240 con 0x5632659ad4d0 2024-04-25T10:35:20.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.598+0000 7f39869b7740 1 -- 172.21.15.43:0/631451017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39680a70d0 con 0x5632659ad4d0 2024-04-25T10:35:20.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.598+0000 7f39617fa640 1 -- 172.21.15.43:0/631451017 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f39740044e0 con 0x5632659ad4d0 2024-04-25T10:35:20.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.598+0000 7f39869b7740 10 client.16245 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:20.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.598+0000 7f39869b7740 1 -- 172.21.15.43:0/631451017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563265dfc1e0 con 0x5632659ad4d0 2024-04-25T10:35:20.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.598+0000 7f39617fa640 1 -- 172.21.15.43:0/631451017 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39680a70d0 con 0x5632659ad4d0 2024-04-25T10:35:20.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.598+0000 7f39617fa640 1 -- 172.21.15.43:0/631451017 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f39740047e0 con 0x5632659ad4d0 2024-04-25T10:35:20.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.599+0000 7f39869b7740 10 client.16245 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:20.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.599+0000 7f39869b7740 10 client.16245 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:35:20.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.599+0000 7f39869b7740 20 client.16245 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:20.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.599+0000 7f39869b7740 1 --2- 172.21.15.43:0/631451017 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x56326596a3a0 0x563265dfe950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.599+0000 7f39869b7740 4 client.16245 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:35:20.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.599+0000 7f39869b7740 1 -- 172.21.15.43:0/631451017 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563265ce4800 con 0x56326596a3a0 2024-04-25T10:35:20.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.599+0000 7f397d0a3640 1 --2- 172.21.15.43:0/631451017 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x56326596a3a0 0x563265dfe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.599+0000 7f397d0a3640 1 --2- 172.21.15.43:0/631451017 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x56326596a3a0 0x563265dfe950 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=561485fee281979b server_cookie=9bc6743414d6e002 in_seq=0 out_seq=0 2024-04-25T10:35:20.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.599+0000 7f39617fa640 10 client.16245 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:35:20.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.599+0000 7f39617fa640 1 -- 172.21.15.43:0/631451017 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563265ce4800 con 0x56326596a3a0 2024-04-25T10:35:20.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.599+0000 7f39617fa640 10 client.16245 handle_command_reply: tid=0 2024-04-25T10:35:20.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f39869b7740 1 client.16245 shutdown 2024-04-25T10:35:20.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f39869b7740 1 -- 172.21.15.43:0/631451017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659ad4d0 msgr2=0x563265d97710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f39869b7740 1 --2- 172.21.15.43:0/631451017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659ad4d0 0x563265d97710 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f397807a310 tx=0x7f39740041a0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f39869b7740 1 -- 172.21.15.43:0/631451017 shutdown_connections 2024-04-25T10:35:20.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f39869b7740 1 --2- 172.21.15.43:0/631451017 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x56326596a3a0 0x563265dfe950 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f39869b7740 1 --2- 172.21.15.43:0/631451017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632659b56b0 0x563265d94a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f39869b7740 1 --2- 172.21.15.43:0/631451017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632659d6390 0x563265d94fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f39617fa640 0 client.16245 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:35:20.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f39869b7740 1 --2- 172.21.15.43:0/631451017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632659ad4d0 0x563265d97710 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f39869b7740 1 -- 172.21.15.43:0/631451017 >> 172.21.15.43:0/631451017 conn(0x5632659ca4e0 msgr2=0x563265d72610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f39869b7740 1 -- 172.21.15.43:0/631451017 shutdown_connections 2024-04-25T10:35:20.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f39869b7740 1 -- 172.21.15.43:0/631451017 wait complete. 2024-04-25T10:35:20.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f39869b7740 20 client.16245 trim_cache size 0 max 16384 2024-04-25T10:35:20.610 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:35:20.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f397fb08640 1 -- 172.21.15.43:0/589612273 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f39640afc60 msgr2=0x7f39640b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f397fb08640 1 --2- 172.21.15.43:0/589612273 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f39640afc60 0x7f39640b2170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f397001b000 tx=0x7f397001d000 comp rx=0 tx=0).stop 2024-04-25T10:35:20.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f397fb08640 1 -- 172.21.15.43:0/589612273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3978109b30 msgr2=0x7f39780753e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f397fb08640 1 --2- 172.21.15.43:0/589612273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3978109b30 0x7f39780753e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f396800e9f0 tx=0x7f396800eeb0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f397fb08640 1 -- 172.21.15.43:0/589612273 shutdown_connections 2024-04-25T10:35:20.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f397fb08640 1 --2- 172.21.15.43:0/589612273 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f39640afc60 0x7f39640b2170 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f397fb08640 1 --2- 172.21.15.43:0/589612273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3978109fd0 0x7f3978075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f397fb08640 1 --2- 172.21.15.43:0/589612273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3978109b30 0x7f39780753e0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f397fb08640 1 --2- 172.21.15.43:0/589612273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39781042d0 0x7f3978074ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.600+0000 7f397fb08640 1 -- 172.21.15.43:0/589612273 >> 172.21.15.43:0/589612273 conn(0x7f3978079280 msgr2=0x7f39781051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.601+0000 7f397fb08640 1 -- 172.21.15.43:0/589612273 shutdown_connections 2024-04-25T10:35:20.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.601+0000 7f397fb08640 1 -- 172.21.15.43:0/589612273 wait complete. 2024-04-25T10:35:20.614 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:35:20.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.726+0000 7fd98f06e640 1 -- 172.21.15.43:0/1203391375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd988107440 msgr2=0x7fd98810ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.726+0000 7fd98f06e640 1 --2- 172.21.15.43:0/1203391375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd988107440 0x7fd98810ebe0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd980009ca0 tx=0x7fd98002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.726+0000 7fd98f06e640 1 -- 172.21.15.43:0/1203391375 shutdown_connections 2024-04-25T10:35:20.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.726+0000 7fd98f06e640 1 --2- 172.21.15.43:0/1203391375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd988106980 0x7fd988106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.726+0000 7fd98f06e640 1 --2- 172.21.15.43:0/1203391375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd988107440 0x7fd98810ebe0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.726+0000 7fd98f06e640 1 --2- 172.21.15.43:0/1203391375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9881023f0 0x7fd988102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.726+0000 7fd98f06e640 1 -- 172.21.15.43:0/1203391375 >> 172.21.15.43:0/1203391375 conn(0x7fd9880fd850 msgr2=0x7fd9880ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.726+0000 7fd98f06e640 1 -- 172.21.15.43:0/1203391375 shutdown_connections 2024-04-25T10:35:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.726+0000 7fd98f06e640 1 -- 172.21.15.43:0/1203391375 wait complete. 2024-04-25T10:35:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.727+0000 7fd98f06e640 1 Processor -- start 2024-04-25T10:35:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.727+0000 7fd98f06e640 1 -- start start 2024-04-25T10:35:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.727+0000 7fd98f06e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9881023f0 0x7fd98819b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.727+0000 7fd98f06e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd988106980 0x7fd98819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.727+0000 7fd98f06e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd988107440 0x7fd98819bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.727+0000 7fd98f06e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd988112bd0 con 0x7fd988106980 2024-04-25T10:35:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.727+0000 7fd98f06e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd988112a50 con 0x7fd9881023f0 2024-04-25T10:35:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.727+0000 7fd98f06e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd988112d50 con 0x7fd988107440 2024-04-25T10:35:20.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.727+0000 7fd987fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd988106980 0x7fd98819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.727+0000 7fd987fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd988106980 0x7fd98819ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45380/0 (socket says 172.21.15.43:45380) 2024-04-25T10:35:20.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.727+0000 7fd987fff640 1 -- 172.21.15.43:0/2886994981 learned_addr learned my addr 172.21.15.43:0/2886994981 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.727+0000 7fd98ce0a640 1 --2- 172.21.15.43:0/2886994981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9881023f0 0x7fd98819b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.727+0000 7fd98d60b640 1 --2- 172.21.15.43:0/2886994981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd988107440 0x7fd98819bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.728+0000 7fd987fff640 1 -- 172.21.15.43:0/2886994981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd988107440 msgr2=0x7fd98819bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.728+0000 7fd987fff640 1 --2- 172.21.15.43:0/2886994981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd988107440 0x7fd98819bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.728+0000 7fd987fff640 1 -- 172.21.15.43:0/2886994981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9881023f0 msgr2=0x7fd98819b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.728+0000 7fd987fff640 1 --2- 172.21.15.43:0/2886994981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9881023f0 0x7fd98819b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.728+0000 7fd987fff640 1 -- 172.21.15.43:0/2886994981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9881a6860 con 0x7fd988106980 2024-04-25T10:35:20.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.728+0000 7fd98d60b640 1 --2- 172.21.15.43:0/2886994981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd988107440 0x7fd98819bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:20.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.728+0000 7fd98ce0a640 1 --2- 172.21.15.43:0/2886994981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9881023f0 0x7fd98819b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:20.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.728+0000 7fd987fff640 1 --2- 172.21.15.43:0/2886994981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd988106980 0x7fd98819ba90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd97c00e9c0 tx=0x7fd97c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.728+0000 7fd985ffb640 1 -- 172.21.15.43:0/2886994981 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c00cd80 con 0x7fd988106980 2024-04-25T10:35:20.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.728+0000 7fd985ffb640 1 -- 172.21.15.43:0/2886994981 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd97c0054d0 con 0x7fd988106980 2024-04-25T10:35:20.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.728+0000 7fd98f06e640 1 -- 172.21.15.43:0/2886994981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9881a6b50 con 0x7fd988106980 2024-04-25T10:35:20.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.728+0000 7fd985ffb640 1 -- 172.21.15.43:0/2886994981 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c010430 con 0x7fd988106980 2024-04-25T10:35:20.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.728+0000 7fd98f06e640 1 -- 172.21.15.43:0/2886994981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9881a70a0 con 0x7fd988106980 2024-04-25T10:35:20.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.730+0000 7fd985ffb640 1 -- 172.21.15.43:0/2886994981 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fd97c004fd0 con 0x7fd988106980 2024-04-25T10:35:20.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.731+0000 7fd996002740 1 Processor -- start 2024-04-25T10:35:20.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.731+0000 7fd996002740 1 -- start start 2024-04-25T10:35:20.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.731+0000 7fd996002740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595eeae390 0x55595f24f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.731+0000 7fd996002740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595ee854d0 0x55595f24fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.731+0000 7fd996002740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee8d6b0 0x55595f25c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.731+0000 7fd996002740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55595f1b4d80 con 0x55595ee854d0 2024-04-25T10:35:20.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.731+0000 7fd996002740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55595f16ea20 con 0x55595ee8d6b0 2024-04-25T10:35:20.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.731+0000 7fd996002740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55595f1b0d40 con 0x55595eeae390 2024-04-25T10:35:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.731+0000 7fd98ce0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595ee854d0 0x55595f24fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.731+0000 7fd985ffb640 1 --2- 172.21.15.43:0/2886994981 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd9700afc60 0x7fd9700b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.731+0000 7fd98ce0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595ee854d0 0x55595f24fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45386/0 (socket says 172.21.15.43:45386) 2024-04-25T10:35:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.731+0000 7fd98ce0a640 1 -- 172.21.15.43:0/2645879057 learned_addr learned my addr 172.21.15.43:0/2645879057 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.731+0000 7fd98d60b640 1 --2- 172.21.15.43:0/2886994981 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd9700afc60 0x7fd9700b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.731+0000 7fd98d60b640 1 --2- 172.21.15.43:0/2645879057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595eeae390 0x55595f24f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.731+0000 7fd985ffb640 1 -- 172.21.15.43:0/2886994981 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fd97c0df560 con 0x7fd988106980 2024-04-25T10:35:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.731+0000 7fd987fff640 1 --2- 172.21.15.43:0/2645879057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee8d6b0 0x55595f25c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.732+0000 7fd98ce0a640 1 -- 172.21.15.43:0/2645879057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595eeae390 msgr2=0x55595f24f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.732+0000 7fd98ce0a640 1 --2- 172.21.15.43:0/2645879057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595eeae390 0x55595f24f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.732+0000 7fd98ce0a640 1 -- 172.21.15.43:0/2645879057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee8d6b0 msgr2=0x55595f25c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.732+0000 7fd98ce0a640 1 --2- 172.21.15.43:0/2645879057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee8d6b0 0x55595f25c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.732+0000 7fd98ce0a640 1 -- 172.21.15.43:0/2645879057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55595f179c30 con 0x55595ee854d0 2024-04-25T10:35:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.732+0000 7fd98d60b640 1 --2- 172.21.15.43:0/2645879057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595eeae390 0x55595f24f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.732+0000 7fd98ce0a640 1 --2- 172.21.15.43:0/2645879057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595ee854d0 0x55595f24fd50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd97800bd40 tx=0x7fd97800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2231fd4af710fd68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.732+0000 7fd98d60b640 1 --2- 172.21.15.43:0/2886994981 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd9700afc60 0x7fd9700b2170 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55595f24c980 tx=0x7fd9800095c0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.732+0000 7fd96effd640 1 -- 172.21.15.43:0/2645879057 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978015070 con 0x55595ee854d0 2024-04-25T10:35:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.732+0000 7fd96effd640 1 -- 172.21.15.43:0/2645879057 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd97800be30 con 0x55595ee854d0 2024-04-25T10:35:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.732+0000 7fd96effd640 1 -- 172.21.15.43:0/2645879057 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978011920 con 0x55595ee854d0 2024-04-25T10:35:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.733+0000 7fd996002740 1 -- 172.21.15.43:0/2645879057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595ee854d0 msgr2=0x55595f24fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.733+0000 7fd996002740 1 --2- 172.21.15.43:0/2645879057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595ee854d0 0x55595f24fd50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd97800bd40 tx=0x7fd97800f730 comp rx=0 tx=0).stop 2024-04-25T10:35:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.733+0000 7fd996002740 1 -- 172.21.15.43:0/2645879057 shutdown_connections 2024-04-25T10:35:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.733+0000 7fd996002740 1 --2- 172.21.15.43:0/2645879057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee8d6b0 0x55595f25c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.733+0000 7fd996002740 1 --2- 172.21.15.43:0/2645879057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595ee854d0 0x55595f24fd50 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.733+0000 7fd996002740 1 --2- 172.21.15.43:0/2645879057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595eeae390 0x55595f24f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.733+0000 7fd996002740 1 -- 172.21.15.43:0/2645879057 >> 172.21.15.43:0/2645879057 conn(0x55595eea24e0 msgr2=0x55595f24bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.733+0000 7fd996002740 1 -- 172.21.15.43:0/2645879057 shutdown_connections 2024-04-25T10:35:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.733+0000 7fd996002740 1 -- 172.21.15.43:0/2645879057 wait complete. 2024-04-25T10:35:20.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.733+0000 7fd996002740 1 -- start start 2024-04-25T10:35:20.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.733+0000 7fd996002740 1 Processor -- start 2024-04-25T10:35:20.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.734+0000 7fd996002740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee854d0 0x55595f25a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.734+0000 7fd996002740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595ee8d6b0 0x55595f25a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.734+0000 7fd996002740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595eeae390 0x55595f266210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.734+0000 7fd996002740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55595f25cd90 con 0x55595ee8d6b0 2024-04-25T10:35:20.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.734+0000 7fd996002740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55595f13bdc0 con 0x55595ee854d0 2024-04-25T10:35:20.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.734+0000 7fd996002740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55595f25cf10 con 0x55595eeae390 2024-04-25T10:35:20.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.734+0000 7fd996002740 10 client.0 _ll_register_callbacks cb 0x55595f157600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:20.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.734+0000 7fd987fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595ee8d6b0 0x55595f25a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.734+0000 7fd987fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595ee8d6b0 0x55595f25a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45398/0 (socket says 172.21.15.43:45398) 2024-04-25T10:35:20.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.734+0000 7fd987fff640 1 -- 172.21.15.43:0/3812000756 learned_addr learned my addr 172.21.15.43:0/3812000756 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.734+0000 7fd98d60b640 1 --2- 172.21.15.43:0/3812000756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595eeae390 0x55595f266210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.734+0000 7fd98ce0a640 1 --2- 172.21.15.43:0/3812000756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee854d0 0x55595f25a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.734+0000 7fd987fff640 1 -- 172.21.15.43:0/3812000756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595eeae390 msgr2=0x55595f266210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.734+0000 7fd987fff640 1 --2- 172.21.15.43:0/3812000756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595eeae390 0x55595f266210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.734+0000 7fd987fff640 1 -- 172.21.15.43:0/3812000756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee854d0 msgr2=0x55595f25a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.734+0000 7fd987fff640 1 --2- 172.21.15.43:0/3812000756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee854d0 0x55595f25a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.734+0000 7fd987fff640 1 -- 172.21.15.43:0/3812000756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55595eec63f0 con 0x55595ee8d6b0 2024-04-25T10:35:20.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.734+0000 7fd987fff640 1 --2- 172.21.15.43:0/3812000756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595ee8d6b0 0x55595f25a7f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0a73d0 tx=0x7fd97c0a70e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e597b42f01e1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.734+0000 7fd967fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:20.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.735+0000 7fd967fff640 1 -- 172.21.15.43:0/3812000756 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0a1e60 con 0x55595ee8d6b0 2024-04-25T10:35:20.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.735+0000 7fd967fff640 1 -- 172.21.15.43:0/3812000756 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fd97c0dcf00 con 0x55595ee8d6b0 2024-04-25T10:35:20.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.735+0000 7fd996002740 1 -- 172.21.15.43:0/3812000756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd97800be30 con 0x55595ee8d6b0 2024-04-25T10:35:20.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.735+0000 7fd967fff640 1 -- 172.21.15.43:0/3812000756 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd97c0dd100 con 0x55595ee8d6b0 2024-04-25T10:35:20.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.735+0000 7fd996002740 10 client.16257 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:20.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.735+0000 7fd996002740 1 -- 172.21.15.43:0/3812000756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55595ef907d0 con 0x55595ee8d6b0 2024-04-25T10:35:20.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.735+0000 7fd967fff640 1 -- 172.21.15.43:0/3812000756 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd97800be30 con 0x55595ee8d6b0 2024-04-25T10:35:20.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.735+0000 7fd967fff640 1 -- 172.21.15.43:0/3812000756 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd97c0a7930 con 0x55595ee8d6b0 2024-04-25T10:35:20.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.735+0000 7fd996002740 10 client.16257 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:20.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.735+0000 7fd996002740 10 client.16257 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:35:20.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.735+0000 7fd996002740 20 client.16257 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:20.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.735+0000 7fd996002740 1 --2- 172.21.15.43:0/3812000756 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55595ee6cf60 0x55595f26c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.735+0000 7fd996002740 4 client.16257 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:35:20.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.735+0000 7fd996002740 1 -- 172.21.15.43:0/3812000756 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55595f1c2990 con 0x55595ee6cf60 2024-04-25T10:35:20.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.736+0000 7fd98ce0a640 1 --2- 172.21.15.43:0/3812000756 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55595ee6cf60 0x55595f26c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.736+0000 7fd98ce0a640 1 --2- 172.21.15.43:0/3812000756 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55595ee6cf60 0x55595f26c6e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc2ecb03976caa4 server_cookie=bc91c7c49b605c90 in_seq=0 out_seq=0 2024-04-25T10:35:20.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.736+0000 7fd967fff640 10 client.16257 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:35:20.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.737+0000 7fd967fff640 1 -- 172.21.15.43:0/3812000756 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55595f1c2990 con 0x55595ee6cf60 2024-04-25T10:35:20.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.737+0000 7fd967fff640 10 client.16257 handle_command_reply: tid=0 2024-04-25T10:35:20.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.737+0000 7fd996002740 1 client.16257 shutdown 2024-04-25T10:35:20.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.737+0000 7fd996002740 1 -- 172.21.15.43:0/3812000756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595ee8d6b0 msgr2=0x55595f25a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.737+0000 7fd996002740 1 --2- 172.21.15.43:0/3812000756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595ee8d6b0 0x55595f25a7f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0a73d0 tx=0x7fd97c0a70e0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.737+0000 7fd996002740 1 -- 172.21.15.43:0/3812000756 shutdown_connections 2024-04-25T10:35:20.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.737+0000 7fd996002740 1 --2- 172.21.15.43:0/3812000756 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55595ee6cf60 0x55595f26c6e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.737+0000 7fd996002740 1 --2- 172.21.15.43:0/3812000756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595eeae390 0x55595f266210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.737+0000 7fd996002740 1 --2- 172.21.15.43:0/3812000756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595ee8d6b0 0x55595f25a7f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.737+0000 7fd996002740 1 --2- 172.21.15.43:0/3812000756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee854d0 0x55595f25a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.737+0000 7fd967fff640 0 client.16257 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:35:20.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.737+0000 7fd996002740 1 -- 172.21.15.43:0/3812000756 >> 172.21.15.43:0/3812000756 conn(0x55595eea24e0 msgr2=0x55595f251530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.737+0000 7fd996002740 1 -- 172.21.15.43:0/3812000756 shutdown_connections 2024-04-25T10:35:20.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.737+0000 7fd996002740 1 -- 172.21.15.43:0/3812000756 wait complete. 2024-04-25T10:35:20.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.738+0000 7fd996002740 20 client.16257 trim_cache size 0 max 16384 2024-04-25T10:35:20.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.751+0000 7fd996002740 1 Processor -- start 2024-04-25T10:35:20.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.751+0000 7fd996002740 1 -- start start 2024-04-25T10:35:20.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.751+0000 7fd996002740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595ee854d0 0x55595f259080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.751+0000 7fd996002740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee8d6b0 0x55595f265e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.751+0000 7fd996002740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595eeae390 0x55595f254510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.751+0000 7fd996002740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9881b2280 con 0x55595eeae390 2024-04-25T10:35:20.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.751+0000 7fd996002740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd9881afb50 con 0x55595ee8d6b0 2024-04-25T10:35:20.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.751+0000 7fd996002740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd9881ad9a0 con 0x55595ee854d0 2024-04-25T10:35:20.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.752+0000 7fd98d60b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595eeae390 0x55595f254510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.752+0000 7fd98d60b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595eeae390 0x55595f254510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45406/0 (socket says 172.21.15.43:45406) 2024-04-25T10:35:20.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.752+0000 7fd98d60b640 1 -- 172.21.15.43:0/2993621647 learned_addr learned my addr 172.21.15.43:0/2993621647 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.752+0000 7fd98ce0a640 1 --2- 172.21.15.43:0/2993621647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595ee854d0 0x55595f259080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.752+0000 7fd987fff640 1 --2- 172.21.15.43:0/2993621647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee8d6b0 0x55595f265e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.752+0000 7fd98d60b640 1 -- 172.21.15.43:0/2993621647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595ee854d0 msgr2=0x55595f259080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.752+0000 7fd98d60b640 1 --2- 172.21.15.43:0/2993621647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595ee854d0 0x55595f259080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.752+0000 7fd98d60b640 1 -- 172.21.15.43:0/2993621647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee8d6b0 msgr2=0x55595f265e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.752+0000 7fd98d60b640 1 --2- 172.21.15.43:0/2993621647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee8d6b0 0x55595f265e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.752+0000 7fd98d60b640 1 -- 172.21.15.43:0/2993621647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55595f24b620 con 0x55595eeae390 2024-04-25T10:35:20.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.752+0000 7fd987fff640 1 --2- 172.21.15.43:0/2993621647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee8d6b0 0x55595f265e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:35:20.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.752+0000 7fd98d60b640 1 --2- 172.21.15.43:0/2993621647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595eeae390 0x55595f254510 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd980009ca0 tx=0x7fd98003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c43b82e32d41c1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.752+0000 7fd96e7fc640 1 -- 172.21.15.43:0/2993621647 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980051070 con 0x55595eeae390 2024-04-25T10:35:20.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.752+0000 7fd96e7fc640 1 -- 172.21.15.43:0/2993621647 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd980045040 con 0x55595eeae390 2024-04-25T10:35:20.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.752+0000 7fd96e7fc640 1 -- 172.21.15.43:0/2993621647 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980044070 con 0x55595eeae390 2024-04-25T10:35:20.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.753+0000 7fd996002740 1 -- 172.21.15.43:0/2993621647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595eeae390 msgr2=0x55595f254510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.753+0000 7fd996002740 1 --2- 172.21.15.43:0/2993621647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595eeae390 0x55595f254510 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd980009ca0 tx=0x7fd98003e000 comp rx=0 tx=0).stop 2024-04-25T10:35:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.753+0000 7fd996002740 1 -- 172.21.15.43:0/2993621647 shutdown_connections 2024-04-25T10:35:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.753+0000 7fd996002740 1 --2- 172.21.15.43:0/2993621647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee8d6b0 0x55595f265e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.753+0000 7fd996002740 1 --2- 172.21.15.43:0/2993621647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595eeae390 0x55595f254510 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.753+0000 7fd996002740 1 --2- 172.21.15.43:0/2993621647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595ee854d0 0x55595f259080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.753+0000 7fd996002740 1 -- 172.21.15.43:0/2993621647 >> 172.21.15.43:0/2993621647 conn(0x55595eea24e0 msgr2=0x55595f258970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.753+0000 7fd996002740 1 -- 172.21.15.43:0/2993621647 shutdown_connections 2024-04-25T10:35:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.753+0000 7fd996002740 1 -- 172.21.15.43:0/2993621647 wait complete. 2024-04-25T10:35:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.753+0000 7fd996002740 1 -- start start 2024-04-25T10:35:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.754+0000 7fd996002740 1 Processor -- start 2024-04-25T10:35:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.754+0000 7fd996002740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee854d0 0x55595f262890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.754+0000 7fd996002740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595ee8d6b0 0x55595f25dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.754+0000 7fd996002740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595eeae390 0x55595f25e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.754+0000 7fd996002740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9881ae010 con 0x55595eeae390 2024-04-25T10:35:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.754+0000 7fd996002740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd9881af480 con 0x55595ee854d0 2024-04-25T10:35:20.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.754+0000 7fd996002740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd9881ae6e0 con 0x55595ee8d6b0 2024-04-25T10:35:20.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.754+0000 7fd996002740 10 client.0 _ll_register_callbacks cb 0x7fd97c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:20.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.754+0000 7fd98d60b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595eeae390 0x55595f25e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.754+0000 7fd98d60b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595eeae390 0x55595f25e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45420/0 (socket says 172.21.15.43:45420) 2024-04-25T10:35:20.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.754+0000 7fd98d60b640 1 -- 172.21.15.43:0/3816897056 learned_addr learned my addr 172.21.15.43:0/3816897056 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.754+0000 7fd98ce0a640 1 --2- 172.21.15.43:0/3816897056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee854d0 0x55595f262890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.754+0000 7fd987fff640 1 --2- 172.21.15.43:0/3816897056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595ee8d6b0 0x55595f25dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.754+0000 7fd98d60b640 1 -- 172.21.15.43:0/3816897056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595ee8d6b0 msgr2=0x55595f25dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.754+0000 7fd98d60b640 1 --2- 172.21.15.43:0/3816897056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595ee8d6b0 0x55595f25dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.754+0000 7fd98d60b640 1 -- 172.21.15.43:0/3816897056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee854d0 msgr2=0x55595f262890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.754+0000 7fd98d60b640 1 --2- 172.21.15.43:0/3816897056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee854d0 0x55595f262890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.754+0000 7fd98d60b640 1 -- 172.21.15.43:0/3816897056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55595f25e970 con 0x55595eeae390 2024-04-25T10:35:20.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.754+0000 7fd987fff640 1 --2- 172.21.15.43:0/3816897056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595ee8d6b0 0x55595f25dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:35:20.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.755+0000 7fd98d60b640 1 --2- 172.21.15.43:0/3816897056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595eeae390 0x55595f25e230 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd980006010 tx=0x7fd980009470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6193ab4ec55a5c86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.755+0000 7fd967fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:20.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.755+0000 7fd967fff640 1 -- 172.21.15.43:0/3816897056 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98005d070 con 0x55595eeae390 2024-04-25T10:35:20.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.755+0000 7fd967fff640 1 -- 172.21.15.43:0/3816897056 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fd9800040e0 con 0x55595eeae390 2024-04-25T10:35:20.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.755+0000 7fd996002740 1 -- 172.21.15.43:0/3816897056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd980045040 con 0x55595eeae390 2024-04-25T10:35:20.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.755+0000 7fd967fff640 1 -- 172.21.15.43:0/3816897056 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd980069050 con 0x55595eeae390 2024-04-25T10:35:20.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.755+0000 7fd996002740 10 client.16263 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:20.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.755+0000 7fd996002740 1 -- 172.21.15.43:0/3816897056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55595f2d49d0 con 0x55595eeae390 2024-04-25T10:35:20.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.755+0000 7fd967fff640 1 -- 172.21.15.43:0/3816897056 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd980045040 con 0x55595eeae390 2024-04-25T10:35:20.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.755+0000 7fd967fff640 1 -- 172.21.15.43:0/3816897056 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd98002fc70 con 0x55595eeae390 2024-04-25T10:35:20.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.755+0000 7fd996002740 10 client.16263 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:20.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.755+0000 7fd996002740 10 client.16263 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:35:20.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.755+0000 7fd996002740 20 client.16263 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:20.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.756+0000 7fd996002740 1 --2- 172.21.15.43:0/3816897056 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55595ee423a0 0x55595f2d70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.756+0000 7fd996002740 4 client.16263 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:35:20.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.756+0000 7fd996002740 1 -- 172.21.15.43:0/3816897056 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55595f2500e0 con 0x55595ee423a0 2024-04-25T10:35:20.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.756+0000 7fd98ce0a640 1 --2- 172.21.15.43:0/3816897056 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55595ee423a0 0x55595f2d70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.757+0000 7fd967fff640 10 client.16263 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:35:20.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.757+0000 7fd98ce0a640 1 --2- 172.21.15.43:0/3816897056 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55595ee423a0 0x55595f2d70e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1afd975291bd52af server_cookie=fdc7d3541ceb2a61 in_seq=0 out_seq=0 2024-04-25T10:35:20.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.757+0000 7fd967fff640 1 -- 172.21.15.43:0/3816897056 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55595f2500e0 con 0x55595ee423a0 2024-04-25T10:35:20.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.757+0000 7fd967fff640 10 client.16263 handle_command_reply: tid=0 2024-04-25T10:35:20.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.757+0000 7fd996002740 1 client.16263 shutdown 2024-04-25T10:35:20.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.757+0000 7fd996002740 1 -- 172.21.15.43:0/3816897056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595eeae390 msgr2=0x55595f25e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.757+0000 7fd996002740 1 --2- 172.21.15.43:0/3816897056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595eeae390 0x55595f25e230 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd980006010 tx=0x7fd980009470 comp rx=0 tx=0).stop 2024-04-25T10:35:20.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.757+0000 7fd996002740 1 -- 172.21.15.43:0/3816897056 shutdown_connections 2024-04-25T10:35:20.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.757+0000 7fd996002740 1 --2- 172.21.15.43:0/3816897056 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55595ee423a0 0x55595f2d70e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.757+0000 7fd996002740 1 --2- 172.21.15.43:0/3816897056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55595eeae390 0x55595f25e230 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.757+0000 7fd996002740 1 --2- 172.21.15.43:0/3816897056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55595ee8d6b0 0x55595f25dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.757+0000 7fd996002740 1 --2- 172.21.15.43:0/3816897056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55595ee854d0 0x55595f262890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.757+0000 7fd996002740 1 -- 172.21.15.43:0/3816897056 >> 172.21.15.43:0/3816897056 conn(0x55595eea24e0 msgr2=0x55595f24bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.757+0000 7fd967fff640 0 client.16263 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:35:20.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.757+0000 7fd996002740 1 -- 172.21.15.43:0/3816897056 shutdown_connections 2024-04-25T10:35:20.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.757+0000 7fd996002740 1 -- 172.21.15.43:0/3816897056 wait complete. 2024-04-25T10:35:20.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.757+0000 7fd996002740 20 client.16263 trim_cache size 0 max 16384 2024-04-25T10:35:20.765 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:35:20.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.758+0000 7fd98f06e640 1 -- 172.21.15.43:0/2886994981 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd9700afc60 msgr2=0x7fd9700b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.758+0000 7fd98f06e640 1 --2- 172.21.15.43:0/2886994981 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd9700afc60 0x7fd9700b2170 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55595f24c980 tx=0x7fd9800095c0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.758+0000 7fd98f06e640 1 -- 172.21.15.43:0/2886994981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd988106980 msgr2=0x7fd98819ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.758+0000 7fd98f06e640 1 --2- 172.21.15.43:0/2886994981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd988106980 0x7fd98819ba90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd97c00e9c0 tx=0x7fd97c00ee80 comp rx=0 tx=0).stop 2024-04-25T10:35:20.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.758+0000 7fd98f06e640 1 -- 172.21.15.43:0/2886994981 shutdown_connections 2024-04-25T10:35:20.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.758+0000 7fd98f06e640 1 --2- 172.21.15.43:0/2886994981 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd9700afc60 0x7fd9700b2170 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.758+0000 7fd98f06e640 1 --2- 172.21.15.43:0/2886994981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd988107440 0x7fd98819bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.758+0000 7fd98f06e640 1 --2- 172.21.15.43:0/2886994981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd988106980 0x7fd98819ba90 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.758+0000 7fd98f06e640 1 --2- 172.21.15.43:0/2886994981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9881023f0 0x7fd98819b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.758+0000 7fd98f06e640 1 -- 172.21.15.43:0/2886994981 >> 172.21.15.43:0/2886994981 conn(0x7fd9880fd850 msgr2=0x7fd988104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.758+0000 7fd98f06e640 1 -- 172.21.15.43:0/2886994981 shutdown_connections 2024-04-25T10:35:20.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.758+0000 7fd98f06e640 1 -- 172.21.15.43:0/2886994981 wait complete. 2024-04-25T10:35:20.770 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:20 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1921821514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:35:20.771 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:35:20.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.883+0000 7f432a6cd640 1 -- 172.21.15.43:0/2496206050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f432410d380 msgr2=0x7f432410f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.883+0000 7f432a6cd640 1 --2- 172.21.15.43:0/2496206050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f432410d380 0x7f432410f7a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f431c009ca0 tx=0x7f431c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.883+0000 7f432a6cd640 1 -- 172.21.15.43:0/2496206050 shutdown_connections 2024-04-25T10:35:20.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.883+0000 7f432a6cd640 1 --2- 172.21.15.43:0/2496206050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4324102af0 0x7f4324102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.883+0000 7f432a6cd640 1 --2- 172.21.15.43:0/2496206050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f432410d380 0x7f432410f7a0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.883+0000 7f432a6cd640 1 --2- 172.21.15.43:0/2496206050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4324108a20 0x7f43241025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.883+0000 7f432a6cd640 1 -- 172.21.15.43:0/2496206050 >> 172.21.15.43:0/2496206050 conn(0x7f4324078100 msgr2=0x7f4324078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.883+0000 7f432a6cd640 1 -- 172.21.15.43:0/2496206050 shutdown_connections 2024-04-25T10:35:20.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.883+0000 7f432a6cd640 1 -- 172.21.15.43:0/2496206050 wait complete. 2024-04-25T10:35:20.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.884+0000 7f432a6cd640 1 Processor -- start 2024-04-25T10:35:20.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.884+0000 7f432a6cd640 1 -- start start 2024-04-25T10:35:20.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.884+0000 7f432a6cd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4324102af0 0x7f432419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.884+0000 7f432a6cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4324108a20 0x7f432419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.884+0000 7f432a6cd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f432410d380 0x7f432419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.884+0000 7f432a6cd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4324112c50 con 0x7f4324108a20 2024-04-25T10:35:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.884+0000 7f432a6cd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4324112ad0 con 0x7f432410d380 2024-04-25T10:35:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.884+0000 7f432a6cd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4324112dd0 con 0x7f4324102af0 2024-04-25T10:35:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.884+0000 7f43237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4324108a20 0x7f432419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.884+0000 7f43237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4324108a20 0x7f432419ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45446/0 (socket says 172.21.15.43:45446) 2024-04-25T10:35:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.885+0000 7f43237fe640 1 -- 172.21.15.43:0/304533141 learned_addr learned my addr 172.21.15.43:0/304533141 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.885+0000 7f4323fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4324102af0 0x7f432419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.885+0000 7f4328c6a640 1 --2- 172.21.15.43:0/304533141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f432410d380 0x7f432419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.885+0000 7f43237fe640 1 -- 172.21.15.43:0/304533141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4324102af0 msgr2=0x7f432419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.885+0000 7f43237fe640 1 --2- 172.21.15.43:0/304533141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4324102af0 0x7f432419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.885+0000 7f43237fe640 1 -- 172.21.15.43:0/304533141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f432410d380 msgr2=0x7f432419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.885+0000 7f43237fe640 1 --2- 172.21.15.43:0/304533141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f432410d380 0x7f432419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.885+0000 7f43237fe640 1 -- 172.21.15.43:0/304533141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432419f500 con 0x7f4324108a20 2024-04-25T10:35:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.885+0000 7f43237fe640 1 --2- 172.21.15.43:0/304533141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4324108a20 0x7f432419ac00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f431800e9f0 tx=0x7f431800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.885+0000 7f43217fa640 1 -- 172.21.15.43:0/304533141 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431800ccc0 con 0x7f4324108a20 2024-04-25T10:35:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.885+0000 7f43217fa640 1 -- 172.21.15.43:0/304533141 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43180035f0 con 0x7f4324108a20 2024-04-25T10:35:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.885+0000 7f432a6cd640 1 -- 172.21.15.43:0/304533141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f432419f7f0 con 0x7f4324108a20 2024-04-25T10:35:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.885+0000 7f43217fa640 1 -- 172.21.15.43:0/304533141 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431800ce60 con 0x7f4324108a20 2024-04-25T10:35:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.885+0000 7f432a6cd640 1 -- 172.21.15.43:0/304533141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43241a7170 con 0x7f4324108a20 2024-04-25T10:35:20.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.888+0000 7f4331578740 1 Processor -- start 2024-04-25T10:35:20.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.888+0000 7f4331578740 1 -- start start 2024-04-25T10:35:20.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.889+0000 7f43217fa640 1 -- 172.21.15.43:0/304533141 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f4318003aa0 con 0x7f4324108a20 2024-04-25T10:35:20.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.889+0000 7f4331578740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd75390 0x560d20116810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.889+0000 7f4331578740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd4c4d0 0x560d20116d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.889+0000 7f4331578740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd546b0 0x560d201236a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.889+0000 7f4331578740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560d2007bd80 con 0x560d1fd546b0 2024-04-25T10:35:20.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.889+0000 7f4331578740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560d20035a20 con 0x560d1fd75390 2024-04-25T10:35:20.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.889+0000 7f4331578740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560d20077d40 con 0x560d1fd4c4d0 2024-04-25T10:35:20.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.889+0000 7f43237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd546b0 0x560d201236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.889+0000 7f43237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd546b0 0x560d201236a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45454/0 (socket says 172.21.15.43:45454) 2024-04-25T10:35:20.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.889+0000 7f43237fe640 1 -- 172.21.15.43:0/4060193421 learned_addr learned my addr 172.21.15.43:0/4060193421 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.889+0000 7f4323fff640 1 --2- 172.21.15.43:0/4060193421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd4c4d0 0x560d20116d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.889+0000 7f4328c6a640 1 --2- 172.21.15.43:0/4060193421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd75390 0x560d20116810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.889+0000 7f43237fe640 1 -- 172.21.15.43:0/4060193421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd4c4d0 msgr2=0x560d20116d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.889+0000 7f43237fe640 1 --2- 172.21.15.43:0/4060193421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd4c4d0 0x560d20116d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.889+0000 7f43237fe640 1 -- 172.21.15.43:0/4060193421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd75390 msgr2=0x560d20116810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.889+0000 7f43237fe640 1 --2- 172.21.15.43:0/4060193421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd75390 0x560d20116810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.889+0000 7f43237fe640 1 -- 172.21.15.43:0/4060193421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d20040c30 con 0x560d1fd546b0 2024-04-25T10:35:20.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.889+0000 7f43237fe640 1 --2- 172.21.15.43:0/4060193421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd546b0 0x560d201236a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f43180dcb10 tx=0x7f43180e5530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a3d67cd820bea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.890+0000 7f43217fa640 1 --2- 172.21.15.43:0/304533141 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f430c0afc60 0x7f430c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.890+0000 7f4328c6a640 1 --2- 172.21.15.43:0/304533141 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f430c0afc60 0x7f430c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.890+0000 7f430a7fc640 1 -- 172.21.15.43:0/4060193421 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43180f3070 con 0x560d1fd546b0 2024-04-25T10:35:20.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.890+0000 7f43217fa640 1 -- 172.21.15.43:0/304533141 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f43180df3f0 con 0x7f4324108a20 2024-04-25T10:35:20.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.890+0000 7f430a7fc640 1 -- 172.21.15.43:0/4060193421 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43180a70d0 con 0x560d1fd546b0 2024-04-25T10:35:20.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.890+0000 7f430a7fc640 1 -- 172.21.15.43:0/4060193421 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43180a7390 con 0x560d1fd546b0 2024-04-25T10:35:20.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.891+0000 7f4331578740 1 -- 172.21.15.43:0/4060193421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd546b0 msgr2=0x560d201236a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.891+0000 7f4331578740 1 --2- 172.21.15.43:0/4060193421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd546b0 0x560d201236a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f43180dcb10 tx=0x7f43180e5530 comp rx=0 tx=0).stop 2024-04-25T10:35:20.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.891+0000 7f4328c6a640 1 --2- 172.21.15.43:0/304533141 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f430c0afc60 0x7f430c0b2170 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x560d20113980 tx=0x7f431c02f310 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.891+0000 7f4331578740 1 -- 172.21.15.43:0/4060193421 shutdown_connections 2024-04-25T10:35:20.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.891+0000 7f4331578740 1 --2- 172.21.15.43:0/4060193421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd546b0 0x560d201236a0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.891+0000 7f4331578740 1 --2- 172.21.15.43:0/4060193421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd4c4d0 0x560d20116d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.891+0000 7f4331578740 1 --2- 172.21.15.43:0/4060193421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd75390 0x560d20116810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.891+0000 7f4331578740 1 -- 172.21.15.43:0/4060193421 >> 172.21.15.43:0/4060193421 conn(0x560d1fd694e0 msgr2=0x560d20112fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.891+0000 7f4331578740 1 -- 172.21.15.43:0/4060193421 shutdown_connections 2024-04-25T10:35:20.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.891+0000 7f4331578740 1 -- 172.21.15.43:0/4060193421 wait complete. 2024-04-25T10:35:20.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.891+0000 7f4331578740 1 -- start start 2024-04-25T10:35:20.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.891+0000 7f4331578740 1 Processor -- start 2024-04-25T10:35:20.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f4331578740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd4c4d0 0x560d201212d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f4331578740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd546b0 0x560d20121810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f4331578740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd75390 0x560d2012d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f4331578740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560d20123d90 con 0x560d1fd546b0 2024-04-25T10:35:20.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f4331578740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560d20002dc0 con 0x560d1fd75390 2024-04-25T10:35:20.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f4331578740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560d20123f10 con 0x560d1fd4c4d0 2024-04-25T10:35:20.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f4331578740 10 client.0 _ll_register_callbacks cb 0x560d2001e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:20.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f43237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd546b0 0x560d20121810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f43237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd546b0 0x560d20121810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45466/0 (socket says 172.21.15.43:45466) 2024-04-25T10:35:20.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f43237fe640 1 -- 172.21.15.43:0/1583126332 learned_addr learned my addr 172.21.15.43:0/1583126332 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f4328c6a640 1 --2- 172.21.15.43:0/1583126332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd75390 0x560d2012d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f4323fff640 1 --2- 172.21.15.43:0/1583126332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd4c4d0 0x560d201212d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f43237fe640 1 -- 172.21.15.43:0/1583126332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd4c4d0 msgr2=0x560d201212d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f43237fe640 1 --2- 172.21.15.43:0/1583126332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd4c4d0 0x560d201212d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f43237fe640 1 -- 172.21.15.43:0/1583126332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd75390 msgr2=0x560d2012d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f43237fe640 1 --2- 172.21.15.43:0/1583126332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd75390 0x560d2012d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f43237fe640 1 -- 172.21.15.43:0/1583126332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d1fd8d3f0 con 0x560d1fd546b0 2024-04-25T10:35:20.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f4323fff640 1 --2- 172.21.15.43:0/1583126332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd4c4d0 0x560d201212d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:35:20.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f43237fe640 1 --2- 172.21.15.43:0/1583126332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd546b0 0x560d20121810 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f43180dcc20 tx=0x7f43180df760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7774ffd5e6be38a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f43037fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:20.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f43037fe640 1 -- 172.21.15.43:0/1583126332 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43180a6d30 con 0x560d1fd546b0 2024-04-25T10:35:20.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f43037fe640 1 -- 172.21.15.43:0/1583126332 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f43180a3850 con 0x560d1fd546b0 2024-04-25T10:35:20.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.892+0000 7f4331578740 1 -- 172.21.15.43:0/1583126332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43180a70d0 con 0x560d1fd546b0 2024-04-25T10:35:20.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.893+0000 7f43037fe640 1 -- 172.21.15.43:0/1583126332 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f43180a6f10 con 0x560d1fd546b0 2024-04-25T10:35:20.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.893+0000 7f4331578740 10 client.16275 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:20.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.893+0000 7f4331578740 1 -- 172.21.15.43:0/1583126332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560d1fe577d0 con 0x560d1fd546b0 2024-04-25T10:35:20.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.893+0000 7f43037fe640 1 -- 172.21.15.43:0/1583126332 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43180a70d0 con 0x560d1fd546b0 2024-04-25T10:35:20.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.893+0000 7f43037fe640 1 -- 172.21.15.43:0/1583126332 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f43180a7820 con 0x560d1fd546b0 2024-04-25T10:35:20.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.893+0000 7f4331578740 10 client.16275 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:20.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.893+0000 7f4331578740 10 client.16275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:20.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.893+0000 7f4331578740 20 client.16275 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:20.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.893+0000 7f4331578740 1 --2- 172.21.15.43:0/1583126332 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560d1fd33f60 0x560d20133540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.893+0000 7f4331578740 4 client.16275 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:35:20.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.893+0000 7f4331578740 1 -- 172.21.15.43:0/1583126332 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d20089990 con 0x560d1fd33f60 2024-04-25T10:35:20.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.893+0000 7f4323fff640 1 --2- 172.21.15.43:0/1583126332 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560d1fd33f60 0x560d20133540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.894+0000 7f4323fff640 1 --2- 172.21.15.43:0/1583126332 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560d1fd33f60 0x560d20133540 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da3f9f4224e68115 server_cookie=1b02610c8659d078 in_seq=0 out_seq=0 2024-04-25T10:35:20.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.894+0000 7f43037fe640 10 client.16275 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:20.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.894+0000 7f43037fe640 1 -- 172.21.15.43:0/1583126332 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560d20089990 con 0x560d1fd33f60 2024-04-25T10:35:20.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.894+0000 7f43037fe640 10 client.16275 handle_command_reply: tid=0 2024-04-25T10:35:20.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.894+0000 7f4331578740 1 client.16275 shutdown 2024-04-25T10:35:20.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.895+0000 7f4331578740 1 -- 172.21.15.43:0/1583126332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd546b0 msgr2=0x560d20121810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.895+0000 7f4331578740 1 --2- 172.21.15.43:0/1583126332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd546b0 0x560d20121810 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f43180dcc20 tx=0x7f43180df760 comp rx=0 tx=0).stop 2024-04-25T10:35:20.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.895+0000 7f4331578740 1 -- 172.21.15.43:0/1583126332 shutdown_connections 2024-04-25T10:35:20.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.895+0000 7f4331578740 1 --2- 172.21.15.43:0/1583126332 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560d1fd33f60 0x560d20133540 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.895+0000 7f4331578740 1 --2- 172.21.15.43:0/1583126332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd75390 0x560d2012d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.895+0000 7f4331578740 1 --2- 172.21.15.43:0/1583126332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd546b0 0x560d20121810 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.895+0000 7f43037fe640 0 client.16275 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:20.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.895+0000 7f4331578740 1 --2- 172.21.15.43:0/1583126332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd4c4d0 0x560d201212d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.895+0000 7f4331578740 1 -- 172.21.15.43:0/1583126332 >> 172.21.15.43:0/1583126332 conn(0x560d1fd694e0 msgr2=0x560d20118530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.895+0000 7f4331578740 1 -- 172.21.15.43:0/1583126332 shutdown_connections 2024-04-25T10:35:20.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.895+0000 7f4331578740 1 -- 172.21.15.43:0/1583126332 wait complete. 2024-04-25T10:35:20.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.895+0000 7f4331578740 20 client.16275 trim_cache size 0 max 16384 2024-04-25T10:35:20.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.909+0000 7f4331578740 1 Processor -- start 2024-04-25T10:35:20.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.909+0000 7f4331578740 1 -- start start 2024-04-25T10:35:20.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f4331578740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd33f60 0x560d2012d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f4331578740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd4c4d0 0x560d201209a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f4331578740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd546b0 0x560d20122ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f4331578740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43241b2270 con 0x560d1fd33f60 2024-04-25T10:35:20.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f4331578740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f43241afb40 con 0x560d1fd546b0 2024-04-25T10:35:20.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f4331578740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f43241ad990 con 0x560d1fd4c4d0 2024-04-25T10:35:20.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f4323fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd33f60 0x560d2012d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f4323fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd33f60 0x560d2012d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45480/0 (socket says 172.21.15.43:45480) 2024-04-25T10:35:20.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f4323fff640 1 -- 172.21.15.43:0/2981296867 learned_addr learned my addr 172.21.15.43:0/2981296867 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f4328c6a640 1 --2- 172.21.15.43:0/2981296867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd546b0 0x560d20122ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f43237fe640 1 --2- 172.21.15.43:0/2981296867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd4c4d0 0x560d201209a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f4323fff640 1 -- 172.21.15.43:0/2981296867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd4c4d0 msgr2=0x560d201209a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f4323fff640 1 --2- 172.21.15.43:0/2981296867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd4c4d0 0x560d201209a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f4323fff640 1 -- 172.21.15.43:0/2981296867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd546b0 msgr2=0x560d20122ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f4323fff640 1 --2- 172.21.15.43:0/2981296867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd546b0 0x560d20122ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f4323fff640 1 -- 172.21.15.43:0/2981296867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d20120ee0 con 0x560d1fd33f60 2024-04-25T10:35:20.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f4328c6a640 1 --2- 172.21.15.43:0/2981296867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd546b0 0x560d20122ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:20.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f43237fe640 1 --2- 172.21.15.43:0/2981296867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd4c4d0 0x560d201209a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:20.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f4323fff640 1 --2- 172.21.15.43:0/2981296867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd33f60 0x560d2012d200 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x560d20117fe0 tx=0x7f43140096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd85251bea81feb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f4309ffb640 1 -- 172.21.15.43:0/2981296867 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43140120d0 con 0x560d1fd33f60 2024-04-25T10:35:20.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f4309ffb640 1 -- 172.21.15.43:0/2981296867 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4314023020 con 0x560d1fd33f60 2024-04-25T10:35:20.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.910+0000 7f4309ffb640 1 -- 172.21.15.43:0/2981296867 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314003b90 con 0x560d1fd33f60 2024-04-25T10:35:20.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.911+0000 7f4331578740 1 -- 172.21.15.43:0/2981296867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd33f60 msgr2=0x560d2012d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.911+0000 7f4331578740 1 --2- 172.21.15.43:0/2981296867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd33f60 0x560d2012d200 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x560d20117fe0 tx=0x7f43140096d0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.911+0000 7f4331578740 1 -- 172.21.15.43:0/2981296867 shutdown_connections 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.911+0000 7f4331578740 1 --2- 172.21.15.43:0/2981296867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd546b0 0x560d20122ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.911+0000 7f4331578740 1 --2- 172.21.15.43:0/2981296867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd4c4d0 0x560d201209a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.911+0000 7f4331578740 1 --2- 172.21.15.43:0/2981296867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd33f60 0x560d2012d200 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.911+0000 7f4331578740 1 -- 172.21.15.43:0/2981296867 >> 172.21.15.43:0/2981296867 conn(0x560d1fd694e0 msgr2=0x560d201227e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.911+0000 7f4331578740 1 -- 172.21.15.43:0/2981296867 shutdown_connections 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.911+0000 7f4331578740 1 -- 172.21.15.43:0/2981296867 wait complete. 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.912+0000 7f4331578740 1 -- start start 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.912+0000 7f4331578740 1 Processor -- start 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.912+0000 7f4331578740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd33f60 0x560d20136a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.912+0000 7f4331578740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd4c4d0 0x560d20133df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.912+0000 7f4331578740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd546b0 0x560d20134330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.912+0000 7f4331578740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43241ae000 con 0x560d1fd4c4d0 2024-04-25T10:35:20.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.912+0000 7f4331578740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f43241af470 con 0x560d1fd546b0 2024-04-25T10:35:20.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.912+0000 7f4331578740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f43241ae6d0 con 0x560d1fd33f60 2024-04-25T10:35:20.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.912+0000 7f4331578740 10 client.0 _ll_register_callbacks cb 0x7f43180f1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:20.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.912+0000 7f43237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd4c4d0 0x560d20133df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.912+0000 7f43237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd4c4d0 0x560d20133df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45492/0 (socket says 172.21.15.43:45492) 2024-04-25T10:35:20.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.912+0000 7f43237fe640 1 -- 172.21.15.43:0/890128310 learned_addr learned my addr 172.21.15.43:0/890128310 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:20.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.913+0000 7f4323fff640 1 --2- 172.21.15.43:0/890128310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd33f60 0x560d20136a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.913+0000 7f4328c6a640 1 --2- 172.21.15.43:0/890128310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd546b0 0x560d20134330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.913+0000 7f43237fe640 1 -- 172.21.15.43:0/890128310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd33f60 msgr2=0x560d20136a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.913+0000 7f43237fe640 1 --2- 172.21.15.43:0/890128310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd33f60 0x560d20136a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.913+0000 7f43237fe640 1 -- 172.21.15.43:0/890128310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd546b0 msgr2=0x560d20134330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.913+0000 7f43237fe640 1 --2- 172.21.15.43:0/890128310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd546b0 0x560d20134330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.913+0000 7f43237fe640 1 -- 172.21.15.43:0/890128310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d201349f0 con 0x560d1fd4c4d0 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.913+0000 7f43237fe640 1 --2- 172.21.15.43:0/890128310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd4c4d0 0x560d20133df0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f43180a4830 tx=0x7f43180df6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c813bc39ef15405 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.913+0000 7f43037fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.913+0000 7f43037fe640 1 -- 172.21.15.43:0/890128310 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43180a6d30 con 0x560d1fd4c4d0 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.913+0000 7f43037fe640 1 -- 172.21.15.43:0/890128310 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f43180b01a0 con 0x560d1fd4c4d0 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.913+0000 7f4331578740 1 -- 172.21.15.43:0/890128310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4314023020 con 0x560d1fd4c4d0 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.913+0000 7f43037fe640 1 -- 172.21.15.43:0/890128310 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f43180a3480 con 0x560d1fd4c4d0 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.913+0000 7f4331578740 10 client.16281 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.913+0000 7f4331578740 1 -- 172.21.15.43:0/890128310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560d2019b4e0 con 0x560d1fd4c4d0 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.914+0000 7f43037fe640 1 -- 172.21.15.43:0/890128310 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4314023020 con 0x560d1fd4c4d0 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.914+0000 7f43037fe640 1 -- 172.21.15.43:0/890128310 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4318003fb0 con 0x560d1fd4c4d0 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.914+0000 7f4331578740 10 client.16281 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.914+0000 7f4331578740 10 client.16281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:20.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.914+0000 7f4331578740 20 client.16281 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:20.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.914+0000 7f4331578740 1 --2- 172.21.15.43:0/890128310 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560d1fd75390 0x560d2019d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.914+0000 7f4331578740 4 client.16281 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:35:20.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.914+0000 7f4331578740 1 -- 172.21.15.43:0/890128310 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d201133e0 con 0x560d1fd75390 2024-04-25T10:35:20.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.914+0000 7f4323fff640 1 --2- 172.21.15.43:0/890128310 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560d1fd75390 0x560d2019d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.915+0000 7f4323fff640 1 --2- 172.21.15.43:0/890128310 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560d1fd75390 0x560d2019d9c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=293a3ed938ffe350 server_cookie=d2ec8aa803577568 in_seq=0 out_seq=0 2024-04-25T10:35:20.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.915+0000 7f43037fe640 10 client.16281 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:20.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.915+0000 7f43037fe640 1 -- 172.21.15.43:0/890128310 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d201133e0 con 0x560d1fd75390 2024-04-25T10:35:20.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.915+0000 7f43037fe640 10 client.16281 handle_command_reply: tid=0 2024-04-25T10:35:20.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.915+0000 7f4331578740 1 client.16281 shutdown 2024-04-25T10:35:20.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.915+0000 7f4331578740 1 -- 172.21.15.43:0/890128310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd4c4d0 msgr2=0x560d20133df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.915+0000 7f4331578740 1 --2- 172.21.15.43:0/890128310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd4c4d0 0x560d20133df0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f43180a4830 tx=0x7f43180df6f0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.915+0000 7f4331578740 1 -- 172.21.15.43:0/890128310 shutdown_connections 2024-04-25T10:35:20.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.915+0000 7f4331578740 1 --2- 172.21.15.43:0/890128310 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560d1fd75390 0x560d2019d9c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.915+0000 7f4331578740 1 --2- 172.21.15.43:0/890128310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1fd546b0 0x560d20134330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.915+0000 7f4331578740 1 --2- 172.21.15.43:0/890128310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1fd4c4d0 0x560d20133df0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.915+0000 7f43037fe640 0 client.16281 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:20.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.915+0000 7f4331578740 1 --2- 172.21.15.43:0/890128310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d1fd33f60 0x560d20136a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.915+0000 7f4331578740 1 -- 172.21.15.43:0/890128310 >> 172.21.15.43:0/890128310 conn(0x560d1fd694e0 msgr2=0x560d20122700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.915+0000 7f4331578740 1 -- 172.21.15.43:0/890128310 shutdown_connections 2024-04-25T10:35:20.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.915+0000 7f4331578740 1 -- 172.21.15.43:0/890128310 wait complete. 2024-04-25T10:35:20.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.915+0000 7f4331578740 20 client.16281 trim_cache size 0 max 16384 2024-04-25T10:35:20.923 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:35:20.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.916+0000 7f432a6cd640 1 -- 172.21.15.43:0/304533141 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f430c0afc60 msgr2=0x7f430c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.916+0000 7f432a6cd640 1 --2- 172.21.15.43:0/304533141 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f430c0afc60 0x7f430c0b2170 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x560d20113980 tx=0x7f431c02f310 comp rx=0 tx=0).stop 2024-04-25T10:35:20.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.916+0000 7f432a6cd640 1 -- 172.21.15.43:0/304533141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4324108a20 msgr2=0x7f432419ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.916+0000 7f432a6cd640 1 --2- 172.21.15.43:0/304533141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4324108a20 0x7f432419ac00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f431800e9f0 tx=0x7f431800eeb0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.916+0000 7f432a6cd640 1 -- 172.21.15.43:0/304533141 shutdown_connections 2024-04-25T10:35:20.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.916+0000 7f432a6cd640 1 --2- 172.21.15.43:0/304533141 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f430c0afc60 0x7f430c0b2170 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.916+0000 7f432a6cd640 1 --2- 172.21.15.43:0/304533141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f432410d380 0x7f432419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.916+0000 7f432a6cd640 1 --2- 172.21.15.43:0/304533141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4324108a20 0x7f432419ac00 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.916+0000 7f432a6cd640 1 --2- 172.21.15.43:0/304533141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4324102af0 0x7f432419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.916+0000 7f432a6cd640 1 -- 172.21.15.43:0/304533141 >> 172.21.15.43:0/304533141 conn(0x7f4324078100 msgr2=0x7f4324069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.916+0000 7f432a6cd640 1 -- 172.21.15.43:0/304533141 shutdown_connections 2024-04-25T10:35:20.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:20.916+0000 7f432a6cd640 1 -- 172.21.15.43:0/304533141 wait complete. 2024-04-25T10:35:21.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:20 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1921821514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:35:21.040 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:20 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1921821514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:35:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.616+0000 7f01018a7640 1 -- 172.21.15.43:0/678261629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00fc102900 msgr2=0x7f00fc102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.616+0000 7f01018a7640 1 --2- 172.21.15.43:0/678261629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00fc102900 0x7f00fc102d10 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009900 tx=0x7f00ec02a410 comp rx=0 tx=0).stop 2024-04-25T10:35:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.616+0000 7f01018a7640 1 -- 172.21.15.43:0/678261629 shutdown_connections 2024-04-25T10:35:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.616+0000 7f01018a7640 1 --2- 172.21.15.43:0/678261629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00fc10db40 0x7f00fc10ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.616+0000 7f01018a7640 1 --2- 172.21.15.43:0/678261629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00fc1104a0 0x7f00fc1128c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.616+0000 7f01018a7640 1 --2- 172.21.15.43:0/678261629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00fc102900 0x7f00fc102d10 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.616+0000 7f01018a7640 1 -- 172.21.15.43:0/678261629 >> 172.21.15.43:0/678261629 conn(0x7f00fc0781b0 msgr2=0x7f00fc0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.616+0000 7f01018a7640 1 -- 172.21.15.43:0/678261629 shutdown_connections 2024-04-25T10:35:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.616+0000 7f01018a7640 1 -- 172.21.15.43:0/678261629 wait complete. 2024-04-25T10:35:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.617+0000 7f01018a7640 1 Processor -- start 2024-04-25T10:35:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.617+0000 7f01018a7640 1 -- start start 2024-04-25T10:35:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.617+0000 7f01018a7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00fc102900 0x7f00fc1a2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.617+0000 7f01018a7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00fc10db40 0x7f00fc1a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.617+0000 7f01018a7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00fc1104a0 0x7f00fc19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.617+0000 7f01018a7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc115f40 con 0x7f00fc1104a0 2024-04-25T10:35:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.617+0000 7f01018a7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc115dc0 con 0x7f00fc102900 2024-04-25T10:35:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.617+0000 7f01018a7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1160c0 con 0x7f00fc10db40 2024-04-25T10:35:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.618+0000 7f00fb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00fc1104a0 0x7f00fc19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.618+0000 7f00faffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00fc102900 0x7f00fc1a2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.618+0000 7f00faffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00fc102900 0x7f00fc1a2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:36772/0 (socket says 172.21.15.43:36772) 2024-04-25T10:35:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.618+0000 7f00fb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00fc1104a0 0x7f00fc19c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45498/0 (socket says 172.21.15.43:45498) 2024-04-25T10:35:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.618+0000 7f00faffd640 1 -- 172.21.15.43:0/3105813542 learned_addr learned my addr 172.21.15.43:0/3105813542 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.618+0000 7f00fa7fc640 1 --2- 172.21.15.43:0/3105813542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00fc10db40 0x7f00fc1a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.618+0000 7f00faffd640 1 -- 172.21.15.43:0/3105813542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00fc10db40 msgr2=0x7f00fc1a2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.618+0000 7f00faffd640 1 --2- 172.21.15.43:0/3105813542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00fc10db40 0x7f00fc1a2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.618+0000 7f00faffd640 1 -- 172.21.15.43:0/3105813542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00fc1104a0 msgr2=0x7f00fc19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.618+0000 7f00faffd640 1 --2- 172.21.15.43:0/3105813542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00fc1104a0 0x7f00fc19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.618+0000 7f00faffd640 1 -- 172.21.15.43:0/3105813542 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00fc19ce80 con 0x7f00fc102900 2024-04-25T10:35:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.618+0000 7f00fb7fe640 1 --2- 172.21.15.43:0/3105813542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00fc1104a0 0x7f00fc19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:35:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.618+0000 7f00faffd640 1 --2- 172.21.15.43:0/3105813542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00fc102900 0x7f00fc1a2570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f00ec005940 tx=0x7f00ec0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.618+0000 7f01008a5640 1 -- 172.21.15.43:0/3105813542 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec03f030 con 0x7f00fc102900 2024-04-25T10:35:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.619+0000 7f01018a7640 1 -- 172.21.15.43:0/3105813542 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00fc19d110 con 0x7f00fc102900 2024-04-25T10:35:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.619+0000 7f01008a5640 1 -- 172.21.15.43:0/3105813542 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00ec0044c0 con 0x7f00fc102900 2024-04-25T10:35:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.619+0000 7f01008a5640 1 -- 172.21.15.43:0/3105813542 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec035070 con 0x7f00fc102900 2024-04-25T10:35:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.619+0000 7f01018a7640 1 -- 172.21.15.43:0/3105813542 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00fc19d600 con 0x7f00fc102900 2024-04-25T10:35:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.621+0000 7f01008a5640 1 -- 172.21.15.43:0/3105813542 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f00ec042030 con 0x7f00fc102900 2024-04-25T10:35:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.621+0000 7f0108830740 1 Processor -- start 2024-04-25T10:35:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.621+0000 7f0108830740 1 -- start start 2024-04-25T10:35:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f0108830740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d4951b0 0x55569d834af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f0108830740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d46c2f0 0x55569d835030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f0108830740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4744d0 0x55569d835570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f0108830740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55569d79b460 con 0x55569d4951b0 2024-04-25T10:35:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f0108830740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55569d755590 con 0x55569d46c2f0 2024-04-25T10:35:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f0108830740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55569d797420 con 0x55569d4744d0 2024-04-25T10:35:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f00fb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d4951b0 0x55569d834af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f00fb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d4951b0 0x55569d834af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45504/0 (socket says 172.21.15.43:45504) 2024-04-25T10:35:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f00fb7fe640 1 -- 172.21.15.43:0/1168606722 learned_addr learned my addr 172.21.15.43:0/1168606722 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f00faffd640 1 --2- 172.21.15.43:0/1168606722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d46c2f0 0x55569d835030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f01008a5640 1 --2- 172.21.15.43:0/3105813542 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f00e40afc50 0x7f00e40b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f00fa7fc640 1 --2- 172.21.15.43:0/1168606722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4744d0 0x55569d835570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f00fb7fe640 1 -- 172.21.15.43:0/1168606722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4744d0 msgr2=0x55569d835570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f00fb7fe640 1 --2- 172.21.15.43:0/1168606722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4744d0 0x55569d835570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f00fb7fe640 1 -- 172.21.15.43:0/1168606722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d46c2f0 msgr2=0x55569d835030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f00fb7fe640 1 --2- 172.21.15.43:0/1168606722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d46c2f0 0x55569d835030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f00fb7fe640 1 -- 172.21.15.43:0/1168606722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55569d7620d0 con 0x55569d4951b0 2024-04-25T10:35:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f00faffd640 1 --2- 172.21.15.43:0/1168606722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d46c2f0 0x55569d835030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f00fb7fe640 1 --2- 172.21.15.43:0/3105813542 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f00e40afc50 0x7f00e40b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f01008a5640 1 -- 172.21.15.43:0/3105813542 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f00ec0ff370 con 0x7f00fc102900 2024-04-25T10:35:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.622+0000 7f00fb7fe640 1 --2- 172.21.15.43:0/1168606722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d4951b0 0x55569d834af0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f00f40118d0 tx=0x7f00f4011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178002a54c5cd4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.623+0000 7f00e1ffb640 1 -- 172.21.15.43:0/1168606722 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f4017050 con 0x55569d4951b0 2024-04-25T10:35:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.623+0000 7f00e1ffb640 1 -- 172.21.15.43:0/1168606722 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00f401bb90 con 0x55569d4951b0 2024-04-25T10:35:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.623+0000 7f00e1ffb640 1 -- 172.21.15.43:0/1168606722 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f4026df0 con 0x55569d4951b0 2024-04-25T10:35:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.623+0000 7f00fb7fe640 1 --2- 172.21.15.43:0/3105813542 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f00e40afc50 0x7f00e40b2160 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f00f4023000 tx=0x7f00f4024000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.623+0000 7f0108830740 1 -- 172.21.15.43:0/1168606722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d4951b0 msgr2=0x55569d834af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.623+0000 7f0108830740 1 --2- 172.21.15.43:0/1168606722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d4951b0 0x55569d834af0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f00f40118d0 tx=0x7f00f4011d90 comp rx=0 tx=0).stop 2024-04-25T10:35:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.624+0000 7f0108830740 1 -- 172.21.15.43:0/1168606722 shutdown_connections 2024-04-25T10:35:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.624+0000 7f0108830740 1 --2- 172.21.15.43:0/1168606722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d46c2f0 0x55569d835030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.624+0000 7f0108830740 1 --2- 172.21.15.43:0/1168606722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4744d0 0x55569d835570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.624+0000 7f0108830740 1 --2- 172.21.15.43:0/1168606722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d4951b0 0x55569d834af0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.624+0000 7f0108830740 1 -- 172.21.15.43:0/1168606722 >> 172.21.15.43:0/1168606722 conn(0x55569d489300 msgr2=0x55569d8312a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.624+0000 7f0108830740 1 -- 172.21.15.43:0/1168606722 shutdown_connections 2024-04-25T10:35:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.624+0000 7f0108830740 1 -- 172.21.15.43:0/1168606722 wait complete. 2024-04-25T10:35:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.624+0000 7f0108830740 1 -- start start 2024-04-25T10:35:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.624+0000 7f0108830740 1 Processor -- start 2024-04-25T10:35:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f0108830740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 0x55569d83f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f0108830740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d4744d0 0x55569d83f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f0108830740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4951b0 0x55569d845180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f0108830740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55569d841c20 con 0x55569d46c2f0 2024-04-25T10:35:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f0108830740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55569d722b60 con 0x55569d4744d0 2024-04-25T10:35:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f0108830740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55569d841da0 con 0x55569d4951b0 2024-04-25T10:35:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f0108830740 10 client.0 _ll_register_callbacks cb 0x55569d7472b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f00faffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 0x55569d83f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f00faffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 0x55569d83f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45510/0 (socket says 172.21.15.43:45510) 2024-04-25T10:35:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f00faffd640 1 -- 172.21.15.43:0/397028895 learned_addr learned my addr 172.21.15.43:0/397028895 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f00fb7fe640 1 --2- 172.21.15.43:0/397028895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4951b0 0x55569d845180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f00fa7fc640 1 --2- 172.21.15.43:0/397028895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d4744d0 0x55569d83f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f00faffd640 1 -- 172.21.15.43:0/397028895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4951b0 msgr2=0x55569d845180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f00faffd640 1 --2- 172.21.15.43:0/397028895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4951b0 0x55569d845180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f00faffd640 1 -- 172.21.15.43:0/397028895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d4744d0 msgr2=0x55569d83f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f00faffd640 1 --2- 172.21.15.43:0/397028895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d4744d0 0x55569d83f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f00faffd640 1 -- 172.21.15.43:0/397028895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55569d4ad210 con 0x55569d46c2f0 2024-04-25T10:35:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f00fb7fe640 1 --2- 172.21.15.43:0/397028895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4951b0 0x55569d845180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f00faffd640 1 --2- 172.21.15.43:0/397028895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 0x55569d83f1e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0c3670 tx=0x7f00ec0c7480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d5bcc9056729f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f00daffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f00daffd640 1 -- 172.21.15.43:0/397028895 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec049070 con 0x55569d46c2f0 2024-04-25T10:35:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.625+0000 7f00daffd640 1 -- 172.21.15.43:0/397028895 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f00ec0fcd70 con 0x55569d46c2f0 2024-04-25T10:35:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.626+0000 7f0108830740 1 -- 172.21.15.43:0/397028895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00f401bb90 con 0x55569d46c2f0 2024-04-25T10:35:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.626+0000 7f00daffd640 1 -- 172.21.15.43:0/397028895 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f00ec0b00b0 con 0x55569d46c2f0 2024-04-25T10:35:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.626+0000 7f0108830740 10 client.16293 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.626+0000 7f0108830740 1 -- 172.21.15.43:0/397028895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55569d579620 con 0x55569d46c2f0 2024-04-25T10:35:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.626+0000 7f00daffd640 1 -- 172.21.15.43:0/397028895 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00f401bb90 con 0x55569d46c2f0 2024-04-25T10:35:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.626+0000 7f00daffd640 1 -- 172.21.15.43:0/397028895 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f00ec0c3880 con 0x55569d46c2f0 2024-04-25T10:35:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.626+0000 7f0108830740 10 client.16293 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.626+0000 7f0108830740 10 client.16293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.626+0000 7f0108830740 20 client.16293 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.626+0000 7f0108830740 1 --2- 172.21.15.43:0/397028895 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55569d453d80 0x55569d851750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.626+0000 7f0108830740 4 client.16293 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:35:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.626+0000 7f0108830740 1 -- 172.21.15.43:0/397028895 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55569d7a9070 con 0x55569d453d80 2024-04-25T10:35:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.627+0000 7f00fa7fc640 1 --2- 172.21.15.43:0/397028895 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55569d453d80 0x55569d851750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.627+0000 7f00daffd640 10 client.16293 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.627+0000 7f00fa7fc640 1 --2- 172.21.15.43:0/397028895 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55569d453d80 0x55569d851750 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccf541473f4f963f server_cookie=18b8dba8bdd754dd in_seq=0 out_seq=0 2024-04-25T10:35:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.628+0000 7f00daffd640 1 -- 172.21.15.43:0/397028895 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55569d7a9070 con 0x55569d453d80 2024-04-25T10:35:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.628+0000 7f00daffd640 10 client.16293 handle_command_reply: tid=0 2024-04-25T10:35:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.628+0000 7f0108830740 1 client.16293 shutdown 2024-04-25T10:35:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.628+0000 7f0108830740 1 -- 172.21.15.43:0/397028895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 msgr2=0x55569d83f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.628+0000 7f0108830740 1 --2- 172.21.15.43:0/397028895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 0x55569d83f1e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0c3670 tx=0x7f00ec0c7480 comp rx=0 tx=0).stop 2024-04-25T10:35:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.628+0000 7f0108830740 1 -- 172.21.15.43:0/397028895 shutdown_connections 2024-04-25T10:35:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.628+0000 7f0108830740 1 --2- 172.21.15.43:0/397028895 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55569d453d80 0x55569d851750 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.628+0000 7f0108830740 1 --2- 172.21.15.43:0/397028895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d4744d0 0x55569d83f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.628+0000 7f0108830740 1 --2- 172.21.15.43:0/397028895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4951b0 0x55569d845180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.628+0000 7f00daffd640 0 client.16293 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.628+0000 7f0108830740 1 --2- 172.21.15.43:0/397028895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 0x55569d83f1e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.628+0000 7f0108830740 1 -- 172.21.15.43:0/397028895 >> 172.21.15.43:0/397028895 conn(0x55569d489300 msgr2=0x55569d834920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.628+0000 7f0108830740 1 -- 172.21.15.43:0/397028895 shutdown_connections 2024-04-25T10:35:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.628+0000 7f0108830740 1 -- 172.21.15.43:0/397028895 wait complete. 2024-04-25T10:35:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.628+0000 7f0108830740 20 client.16293 trim_cache size 0 max 16384 2024-04-25T10:35:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.645+0000 7f0108830740 1 Processor -- start 2024-04-25T10:35:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.645+0000 7f0108830740 1 -- start start 2024-04-25T10:35:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.645+0000 7f0108830740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 0x55569d838ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.645+0000 7f0108830740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4744d0 0x55569d856f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.645+0000 7f0108830740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d4951b0 0x55569d857460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.645+0000 7f0108830740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1b44a0 con 0x55569d46c2f0 2024-04-25T10:35:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.645+0000 7f0108830740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1b1d70 con 0x55569d4951b0 2024-04-25T10:35:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.645+0000 7f0108830740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1afbc0 con 0x55569d4744d0 2024-04-25T10:35:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.645+0000 7f00faffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 0x55569d838ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.645+0000 7f00faffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 0x55569d838ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45524/0 (socket says 172.21.15.43:45524) 2024-04-25T10:35:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.645+0000 7f00faffd640 1 -- 172.21.15.43:0/3325890627 learned_addr learned my addr 172.21.15.43:0/3325890627 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.645+0000 7f00fa7fc640 1 --2- 172.21.15.43:0/3325890627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4744d0 0x55569d856f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.645+0000 7f00fb7fe640 1 --2- 172.21.15.43:0/3325890627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d4951b0 0x55569d857460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.646+0000 7f00faffd640 1 -- 172.21.15.43:0/3325890627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4744d0 msgr2=0x55569d856f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.646+0000 7f00faffd640 1 --2- 172.21.15.43:0/3325890627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4744d0 0x55569d856f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.646+0000 7f00faffd640 1 -- 172.21.15.43:0/3325890627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d4951b0 msgr2=0x55569d857460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.646+0000 7f00faffd640 1 --2- 172.21.15.43:0/3325890627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d4951b0 0x55569d857460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.646+0000 7f00faffd640 1 -- 172.21.15.43:0/3325890627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55569d841700 con 0x55569d46c2f0 2024-04-25T10:35:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.646+0000 7f00fa7fc640 1 --2- 172.21.15.43:0/3325890627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4744d0 0x55569d856f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:35:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.646+0000 7f00faffd640 1 --2- 172.21.15.43:0/3325890627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 0x55569d838ec0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0c7580 tx=0x7f00ec0c74f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1a66d11841ba2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.646+0000 7f00e17fa640 1 -- 172.21.15.43:0/3325890627 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec049040 con 0x55569d46c2f0 2024-04-25T10:35:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.646+0000 7f00e17fa640 1 -- 172.21.15.43:0/3325890627 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00ec0e7070 con 0x55569d46c2f0 2024-04-25T10:35:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.646+0000 7f00e17fa640 1 -- 172.21.15.43:0/3325890627 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec0c7920 con 0x55569d46c2f0 2024-04-25T10:35:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.647+0000 7f0108830740 1 -- 172.21.15.43:0/3325890627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 msgr2=0x55569d838ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.647+0000 7f0108830740 1 --2- 172.21.15.43:0/3325890627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 0x55569d838ec0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0c7580 tx=0x7f00ec0c74f0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.647+0000 7f0108830740 1 -- 172.21.15.43:0/3325890627 shutdown_connections 2024-04-25T10:35:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.647+0000 7f0108830740 1 --2- 172.21.15.43:0/3325890627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d4951b0 0x55569d857460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.647+0000 7f0108830740 1 --2- 172.21.15.43:0/3325890627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4744d0 0x55569d856f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.647+0000 7f0108830740 1 --2- 172.21.15.43:0/3325890627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 0x55569d838ec0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.647+0000 7f0108830740 1 -- 172.21.15.43:0/3325890627 >> 172.21.15.43:0/3325890627 conn(0x55569d489300 msgr2=0x55569d8387b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.647+0000 7f0108830740 1 -- 172.21.15.43:0/3325890627 shutdown_connections 2024-04-25T10:35:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.647+0000 7f0108830740 1 -- 172.21.15.43:0/3325890627 wait complete. 2024-04-25T10:35:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.648+0000 7f0108830740 1 -- start start 2024-04-25T10:35:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.648+0000 7f0108830740 1 Processor -- start 2024-04-25T10:35:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.648+0000 7f0108830740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 0x55569d84b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.648+0000 7f0108830740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d4744d0 0x55569d84bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.648+0000 7f0108830740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4951b0 0x55569d848b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.648+0000 7f0108830740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1b0230 con 0x55569d46c2f0 2024-04-25T10:35:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.648+0000 7f0108830740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1b16a0 con 0x55569d4744d0 2024-04-25T10:35:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.648+0000 7f0108830740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1b0900 con 0x55569d4951b0 2024-04-25T10:35:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.648+0000 7f0108830740 10 client.0 _ll_register_callbacks cb 0x7f00ec005ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.648+0000 7f00faffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 0x55569d84b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.648+0000 7f00fa7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d4744d0 0x55569d84bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.648+0000 7f00faffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 0x55569d84b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45540/0 (socket says 172.21.15.43:45540) 2024-04-25T10:35:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.649+0000 7f00faffd640 1 -- 172.21.15.43:0/3720541654 learned_addr learned my addr 172.21.15.43:0/3720541654 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.649+0000 7f00fa7fc640 1 --2- 172.21.15.43:0/3720541654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d4744d0 0x55569d84bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:36814/0 (socket says 172.21.15.43:36814) 2024-04-25T10:35:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.649+0000 7f00fb7fe640 1 --2- 172.21.15.43:0/3720541654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4951b0 0x55569d848b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.649+0000 7f00faffd640 1 -- 172.21.15.43:0/3720541654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4951b0 msgr2=0x55569d848b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.649+0000 7f00faffd640 1 --2- 172.21.15.43:0/3720541654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4951b0 0x55569d848b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.649+0000 7f00faffd640 1 -- 172.21.15.43:0/3720541654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d4744d0 msgr2=0x55569d84bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.649+0000 7f00faffd640 1 --2- 172.21.15.43:0/3720541654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d4744d0 0x55569d84bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.649+0000 7f00faffd640 1 -- 172.21.15.43:0/3720541654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55569d849300 con 0x55569d46c2f0 2024-04-25T10:35:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.649+0000 7f00fa7fc640 1 --2- 172.21.15.43:0/3720541654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d4744d0 0x55569d84bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:35:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.649+0000 7f00faffd640 1 --2- 172.21.15.43:0/3720541654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 0x55569d84b6c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f00ec003fd0 tx=0x7f00ec0c3b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42af14e4de85650 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.649+0000 7f00daffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.649+0000 7f00daffd640 1 -- 172.21.15.43:0/3720541654 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec0e0020 con 0x55569d46c2f0 2024-04-25T10:35:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.649+0000 7f00daffd640 1 -- 172.21.15.43:0/3720541654 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f00ec0cffb0 con 0x55569d46c2f0 2024-04-25T10:35:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.649+0000 7f0108830740 1 -- 172.21.15.43:0/3720541654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00ec0e7070 con 0x55569d46c2f0 2024-04-25T10:35:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.650+0000 7f00daffd640 1 -- 172.21.15.43:0/3720541654 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f00ec0a9050 con 0x55569d46c2f0 2024-04-25T10:35:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.650+0000 7f0108830740 10 client.16299 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.650+0000 7f0108830740 1 -- 172.21.15.43:0/3720541654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55569d849300 con 0x55569d46c2f0 2024-04-25T10:35:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.650+0000 7f00daffd640 1 -- 172.21.15.43:0/3720541654 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00ec0cb8f0 con 0x55569d46c2f0 2024-04-25T10:35:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.650+0000 7f00daffd640 1 -- 172.21.15.43:0/3720541654 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f00ec0c3120 con 0x55569d46c2f0 2024-04-25T10:35:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.650+0000 7f0108830740 10 client.16299 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.650+0000 7f0108830740 10 client.16299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.650+0000 7f0108830740 20 client.16299 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.650+0000 7f0108830740 1 --2- 172.21.15.43:0/3720541654 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55569d4291c0 0x55569d8bb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.650+0000 7f0108830740 4 client.16299 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:35:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.650+0000 7f0108830740 1 -- 172.21.15.43:0/3720541654 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f00ec0e7070 con 0x55569d4291c0 2024-04-25T10:35:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.650+0000 7f00fa7fc640 1 --2- 172.21.15.43:0/3720541654 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55569d4291c0 0x55569d8bb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.651+0000 7f00daffd640 10 client.16299 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.651+0000 7f00fa7fc640 1 --2- 172.21.15.43:0/3720541654 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55569d4291c0 0x55569d8bb940 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8785f69939ebe9c8 server_cookie=9120b314af19a1d5 in_seq=0 out_seq=0 2024-04-25T10:35:21.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.668+0000 7f00daffd640 1 -- 172.21.15.43:0/3720541654 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f00ec0e7070 con 0x55569d4291c0 2024-04-25T10:35:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.668+0000 7f00daffd640 10 client.16299 handle_command_reply: tid=0 2024-04-25T10:35:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.668+0000 7f0108830740 1 client.16299 shutdown 2024-04-25T10:35:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.668+0000 7f0108830740 1 -- 172.21.15.43:0/3720541654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 msgr2=0x55569d84b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.668+0000 7f0108830740 1 --2- 172.21.15.43:0/3720541654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 0x55569d84b6c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f00ec003fd0 tx=0x7f00ec0c3b10 comp rx=0 tx=0).stop 2024-04-25T10:35:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.668+0000 7f0108830740 1 -- 172.21.15.43:0/3720541654 shutdown_connections 2024-04-25T10:35:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.668+0000 7f0108830740 1 --2- 172.21.15.43:0/3720541654 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55569d4291c0 0x55569d8bb940 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.668+0000 7f0108830740 1 --2- 172.21.15.43:0/3720541654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55569d4744d0 0x55569d84bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.668+0000 7f0108830740 1 --2- 172.21.15.43:0/3720541654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55569d4951b0 0x55569d848b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.668+0000 7f0108830740 1 --2- 172.21.15.43:0/3720541654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55569d46c2f0 0x55569d84b6c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.668+0000 7f0108830740 1 -- 172.21.15.43:0/3720541654 >> 172.21.15.43:0/3720541654 conn(0x55569d489300 msgr2=0x55569d83ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.668+0000 7f00daffd640 0 client.16299 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.668+0000 7f0108830740 1 -- 172.21.15.43:0/3720541654 shutdown_connections 2024-04-25T10:35:21.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.668+0000 7f0108830740 1 -- 172.21.15.43:0/3720541654 wait complete. 2024-04-25T10:35:21.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.668+0000 7f0108830740 20 client.16299 trim_cache size 0 max 16384 2024-04-25T10:35:21.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:35:21.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:35:21.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:35:21.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:35:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.669+0000 7f01018a7640 1 -- 172.21.15.43:0/3105813542 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f00e40afc50 msgr2=0x7f00e40b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.669+0000 7f01018a7640 1 --2- 172.21.15.43:0/3105813542 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f00e40afc50 0x7f00e40b2160 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f00f4023000 tx=0x7f00f4024000 comp rx=0 tx=0).stop 2024-04-25T10:35:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.669+0000 7f01018a7640 1 -- 172.21.15.43:0/3105813542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00fc102900 msgr2=0x7f00fc1a2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.669+0000 7f01018a7640 1 --2- 172.21.15.43:0/3105813542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00fc102900 0x7f00fc1a2570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f00ec005940 tx=0x7f00ec0040c0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.669+0000 7f01018a7640 1 -- 172.21.15.43:0/3105813542 shutdown_connections 2024-04-25T10:35:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.669+0000 7f01018a7640 1 --2- 172.21.15.43:0/3105813542 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f00e40afc50 0x7f00e40b2160 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.669+0000 7f01018a7640 1 --2- 172.21.15.43:0/3105813542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00fc1104a0 0x7f00fc19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.669+0000 7f01018a7640 1 --2- 172.21.15.43:0/3105813542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00fc10db40 0x7f00fc1a2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.669+0000 7f01018a7640 1 --2- 172.21.15.43:0/3105813542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00fc102900 0x7f00fc1a2570 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.669+0000 7f01018a7640 1 -- 172.21.15.43:0/3105813542 >> 172.21.15.43:0/3105813542 conn(0x7f00fc0781b0 msgr2=0x7f00fc101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.669+0000 7f01018a7640 1 -- 172.21.15.43:0/3105813542 shutdown_connections 2024-04-25T10:35:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:21.669+0000 7f01018a7640 1 -- 172.21.15.43:0/3105813542 wait complete. 2024-04-25T10:35:21.929 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:35:21.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:35:21.930 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:35:21.953 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:21 smithi043 ceph-mon[29478]: pgmap v88: 97 pgs: 97 active+clean; 2.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-04-25T10:35:22.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:21 smithi077 ceph-mon[32772]: pgmap v88: 97 pgs: 97 active+clean; 2.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-04-25T10:35:22.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:21 smithi164 ceph-mon[32713]: pgmap v88: 97 pgs: 97 active+clean; 2.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-04-25T10:35:22.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.049+0000 7fa234796640 1 -- 172.21.15.43:0/402653274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa22c10e060 msgr2=0x7fa22c110550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:22.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.049+0000 7fa234796640 1 --2- 172.21.15.43:0/402653274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa22c10e060 0x7fa22c110550 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa224009ca0 tx=0x7fa22402a9e0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.049+0000 7fa234796640 1 -- 172.21.15.43:0/402653274 shutdown_connections 2024-04-25T10:35:22.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.049+0000 7fa234796640 1 --2- 172.21.15.43:0/402653274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa22c10e060 0x7fa22c110550 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.049+0000 7fa234796640 1 --2- 172.21.15.43:0/402653274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa22c10b6b0 0x7fa22c10dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.049+0000 7fa234796640 1 --2- 172.21.15.43:0/402653274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa22c104c50 0x7fa22c105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.049+0000 7fa234796640 1 -- 172.21.15.43:0/402653274 >> 172.21.15.43:0/402653274 conn(0x7fa22c078100 msgr2=0x7fa22c078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:22.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.049+0000 7fa234796640 1 -- 172.21.15.43:0/402653274 shutdown_connections 2024-04-25T10:35:22.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.049+0000 7fa234796640 1 -- 172.21.15.43:0/402653274 wait complete. 2024-04-25T10:35:22.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.050+0000 7fa234796640 1 Processor -- start 2024-04-25T10:35:22.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.050+0000 7fa234796640 1 -- start start 2024-04-25T10:35:22.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.050+0000 7fa234796640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa22c104c50 0x7fa22c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.050+0000 7fa234796640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa22c10b6b0 0x7fa22c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.050+0000 7fa234796640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa22c10e060 0x7fa22c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.050+0000 7fa234796640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c113cc0 con 0x7fa22c104c50 2024-04-25T10:35:22.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.050+0000 7fa234796640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c113b40 con 0x7fa22c10e060 2024-04-25T10:35:22.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.050+0000 7fa234796640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c113e40 con 0x7fa22c10b6b0 2024-04-25T10:35:22.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.050+0000 7fa232532640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa22c104c50 0x7fa22c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.050+0000 7fa232532640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa22c104c50 0x7fa22c19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45548/0 (socket says 172.21.15.43:45548) 2024-04-25T10:35:22.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.050+0000 7fa232532640 1 -- 172.21.15.43:0/2695479568 learned_addr learned my addr 172.21.15.43:0/2695479568 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:22.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.050+0000 7fa232d33640 1 --2- 172.21.15.43:0/2695479568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa22c10e060 0x7fa22c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.050+0000 7fa231d31640 1 --2- 172.21.15.43:0/2695479568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa22c10b6b0 0x7fa22c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.050+0000 7fa232532640 1 -- 172.21.15.43:0/2695479568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa22c10b6b0 msgr2=0x7fa22c19ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:22.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.050+0000 7fa232532640 1 --2- 172.21.15.43:0/2695479568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa22c10b6b0 0x7fa22c19ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.050+0000 7fa232532640 1 -- 172.21.15.43:0/2695479568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa22c10e060 msgr2=0x7fa22c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:22.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.050+0000 7fa232532640 1 --2- 172.21.15.43:0/2695479568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa22c10e060 0x7fa22c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.050+0000 7fa232532640 1 -- 172.21.15.43:0/2695479568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22c19f540 con 0x7fa22c104c50 2024-04-25T10:35:22.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.051+0000 7fa232532640 1 --2- 172.21.15.43:0/2695479568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa22c104c50 0x7fa22c19a6e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa22800d950 tx=0x7fa22800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:22.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.051+0000 7fa2237fe640 1 -- 172.21.15.43:0/2695479568 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228017030 con 0x7fa22c104c50 2024-04-25T10:35:22.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.051+0000 7fa234796640 1 -- 172.21.15.43:0/2695479568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa22c19f830 con 0x7fa22c104c50 2024-04-25T10:35:22.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.051+0000 7fa2237fe640 1 -- 172.21.15.43:0/2695479568 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa228004020 con 0x7fa22c104c50 2024-04-25T10:35:22.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.051+0000 7fa2237fe640 1 -- 172.21.15.43:0/2695479568 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2280047a0 con 0x7fa22c104c50 2024-04-25T10:35:22.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.051+0000 7fa234796640 1 -- 172.21.15.43:0/2695479568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa22c1a71b0 con 0x7fa22c104c50 2024-04-25T10:35:22.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.053+0000 7fa23b649740 1 Processor -- start 2024-04-25T10:35:22.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.053+0000 7fa2237fe640 1 -- 172.21.15.43:0/2695479568 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fa228009a30 con 0x7fa22c104c50 2024-04-25T10:35:22.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.053+0000 7fa23b649740 1 -- start start 2024-04-25T10:35:22.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.054+0000 7fa23b649740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6402a7410 0x55e640647620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.054+0000 7fa23b649740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e64027e550 0x55e640647b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.054+0000 7fa23b649740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e640286730 0x55e6406480a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.054+0000 7fa23b649740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e6405adb00 con 0x55e64027e550 2024-04-25T10:35:22.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.054+0000 7fa23b649740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e640549920 con 0x55e640286730 2024-04-25T10:35:22.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.054+0000 7fa23b649740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e6405a9ac0 con 0x55e6402a7410 2024-04-25T10:35:22.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.054+0000 7fa232532640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e64027e550 0x55e640647b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.054+0000 7fa232532640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e64027e550 0x55e640647b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45558/0 (socket says 172.21.15.43:45558) 2024-04-25T10:35:22.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.054+0000 7fa232532640 1 -- 172.21.15.43:0/2154433900 learned_addr learned my addr 172.21.15.43:0/2154433900 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:22.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.054+0000 7fa231d31640 1 --2- 172.21.15.43:0/2154433900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e640286730 0x55e6406480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.054+0000 7fa232d33640 1 --2- 172.21.15.43:0/2154433900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6402a7410 0x55e640647620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.054+0000 7fa232532640 1 -- 172.21.15.43:0/2154433900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6402a7410 msgr2=0x55e640647620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:22.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.054+0000 7fa232532640 1 --2- 172.21.15.43:0/2154433900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6402a7410 0x55e640647620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.054+0000 7fa232532640 1 -- 172.21.15.43:0/2154433900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e640286730 msgr2=0x55e6406480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:22.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.054+0000 7fa232532640 1 --2- 172.21.15.43:0/2154433900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e640286730 0x55e6406480a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.054+0000 7fa2237fe640 1 --2- 172.21.15.43:0/2695479568 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa2180afc60 0x7fa2180b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.054+0000 7fa232532640 1 -- 172.21.15.43:0/2154433900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6403877e0 con 0x55e64027e550 2024-04-25T10:35:22.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.054+0000 7fa232d33640 1 --2- 172.21.15.43:0/2695479568 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa2180afc60 0x7fa2180b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.054+0000 7fa232532640 1 --2- 172.21.15.43:0/2154433900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e64027e550 0x55e640647b60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa2280dc130 tx=0x7fa2280a3350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f88140fbfc531f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:22.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.055+0000 7fa2237fe640 1 -- 172.21.15.43:0/2695479568 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa2280dead0 con 0x7fa22c104c50 2024-04-25T10:35:22.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.055+0000 7fa220ff9640 1 -- 172.21.15.43:0/2154433900 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2280a2b00 con 0x55e64027e550 2024-04-25T10:35:22.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.055+0000 7fa220ff9640 1 -- 172.21.15.43:0/2154433900 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2280af3f0 con 0x55e64027e550 2024-04-25T10:35:22.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.055+0000 7fa220ff9640 1 -- 172.21.15.43:0/2154433900 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2280a46f0 con 0x55e64027e550 2024-04-25T10:35:22.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.056+0000 7fa23b649740 1 -- 172.21.15.43:0/2154433900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e64027e550 msgr2=0x55e640647b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:22.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.056+0000 7fa232d33640 1 --2- 172.21.15.43:0/2695479568 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa2180afc60 0x7fa2180b2170 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa224009a60 tx=0x7fa22403a040 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:22.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.056+0000 7fa23b649740 1 --2- 172.21.15.43:0/2154433900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e64027e550 0x55e640647b60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa2280dc130 tx=0x7fa2280a3350 comp rx=0 tx=0).stop 2024-04-25T10:35:22.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.056+0000 7fa23b649740 1 -- 172.21.15.43:0/2154433900 shutdown_connections 2024-04-25T10:35:22.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.056+0000 7fa23b649740 1 --2- 172.21.15.43:0/2154433900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e640286730 0x55e6406480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.056+0000 7fa23b649740 1 --2- 172.21.15.43:0/2154433900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e64027e550 0x55e640647b60 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.056+0000 7fa23b649740 1 --2- 172.21.15.43:0/2154433900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6402a7410 0x55e640647620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.056+0000 7fa23b649740 1 -- 172.21.15.43:0/2154433900 >> 172.21.15.43:0/2154433900 conn(0x55e64029b560 msgr2=0x55e640643dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:22.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.056+0000 7fa23b649740 1 -- 172.21.15.43:0/2154433900 shutdown_connections 2024-04-25T10:35:22.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.056+0000 7fa23b649740 1 -- 172.21.15.43:0/2154433900 wait complete. 2024-04-25T10:35:22.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.056+0000 7fa23b649740 1 -- start start 2024-04-25T10:35:22.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.056+0000 7fa23b649740 1 Processor -- start 2024-04-25T10:35:22.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.056+0000 7fa23b649740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e64027e550 0x55e640651c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.056+0000 7fa23b649740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e640286730 0x55e640652140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.056+0000 7fa23b649740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6402a7410 0x55e64065de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.056+0000 7fa23b649740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e640654970 con 0x55e6402a7410 2024-04-25T10:35:22.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.056+0000 7fa23b649740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e6405475d0 con 0x55e64027e550 2024-04-25T10:35:22.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.056+0000 7fa23b649740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e640654af0 con 0x55e640286730 2024-04-25T10:35:22.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.057+0000 7fa23b649740 10 client.0 _ll_register_callbacks cb 0x55e640535080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:22.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.057+0000 7fa232d33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6402a7410 0x55e64065de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.057+0000 7fa232d33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6402a7410 0x55e64065de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45572/0 (socket says 172.21.15.43:45572) 2024-04-25T10:35:22.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.057+0000 7fa232d33640 1 -- 172.21.15.43:0/3963703892 learned_addr learned my addr 172.21.15.43:0/3963703892 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:22.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.057+0000 7fa232532640 1 --2- 172.21.15.43:0/3963703892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e64027e550 0x55e640651c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.057+0000 7fa231d31640 1 --2- 172.21.15.43:0/3963703892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e640286730 0x55e640652140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.057+0000 7fa232d33640 1 -- 172.21.15.43:0/3963703892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e640286730 msgr2=0x55e640652140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:22.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.057+0000 7fa232d33640 1 --2- 172.21.15.43:0/3963703892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e640286730 0x55e640652140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.057+0000 7fa232d33640 1 -- 172.21.15.43:0/3963703892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e64027e550 msgr2=0x55e640651c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:22.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.057+0000 7fa232d33640 1 --2- 172.21.15.43:0/3963703892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e64027e550 0x55e640651c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.057+0000 7fa232d33640 1 -- 172.21.15.43:0/3963703892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6403e7f70 con 0x55e6402a7410 2024-04-25T10:35:22.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.057+0000 7fa232532640 1 --2- 172.21.15.43:0/3963703892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e64027e550 0x55e640651c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:22.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.057+0000 7fa232d33640 1 --2- 172.21.15.43:0/3963703892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6402a7410 0x55e64065de20 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa22402e610 tx=0x7fa22402a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773434df2c2769a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:22.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.057+0000 7fa215ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:22.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.057+0000 7fa215ffb640 1 -- 172.21.15.43:0/3963703892 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224054060 con 0x55e6402a7410 2024-04-25T10:35:22.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.057+0000 7fa215ffb640 1 -- 172.21.15.43:0/3963703892 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa224045480 con 0x55e6402a7410 2024-04-25T10:35:22.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.057+0000 7fa23b649740 1 -- 172.21.15.43:0/3963703892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2280af3f0 con 0x55e6402a7410 2024-04-25T10:35:22.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.058+0000 7fa215ffb640 1 -- 172.21.15.43:0/3963703892 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa224033040 con 0x55e6402a7410 2024-04-25T10:35:22.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.058+0000 7fa23b649740 10 client.16311 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:22.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.058+0000 7fa23b649740 1 -- 172.21.15.43:0/3963703892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e64027f170 con 0x55e6402a7410 2024-04-25T10:35:22.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.058+0000 7fa215ffb640 1 -- 172.21.15.43:0/3963703892 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2280af3f0 con 0x55e6402a7410 2024-04-25T10:35:22.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.058+0000 7fa215ffb640 1 -- 172.21.15.43:0/3963703892 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa2240387c0 con 0x55e6402a7410 2024-04-25T10:35:22.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.058+0000 7fa23b649740 10 client.16311 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:22.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.058+0000 7fa23b649740 10 client.16311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:22.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.058+0000 7fa23b649740 20 client.16311 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:22.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.058+0000 7fa23b649740 1 --2- 172.21.15.43:0/3963703892 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e6402661f0 0x55e640664030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.058+0000 7fa23b649740 4 client.16311 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:35:22.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.058+0000 7fa23b649740 1 -- 172.21.15.43:0/3963703892 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6405bb710 con 0x55e6402661f0 2024-04-25T10:35:22.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.058+0000 7fa232532640 1 --2- 172.21.15.43:0/3963703892 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e6402661f0 0x55e640664030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.059+0000 7fa215ffb640 10 client.16311 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:22.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.059+0000 7fa232532640 1 --2- 172.21.15.43:0/3963703892 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e6402661f0 0x55e640664030 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d19e60e0a35f923 server_cookie=70dcb661912258b2 in_seq=0 out_seq=0 2024-04-25T10:35:22.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.060+0000 7fa215ffb640 1 -- 172.21.15.43:0/3963703892 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e6405bb710 con 0x55e6402661f0 2024-04-25T10:35:22.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.060+0000 7fa215ffb640 10 client.16311 handle_command_reply: tid=0 2024-04-25T10:35:22.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.060+0000 7fa23b649740 1 client.16311 shutdown 2024-04-25T10:35:22.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.060+0000 7fa23b649740 1 -- 172.21.15.43:0/3963703892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6402a7410 msgr2=0x55e64065de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:22.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.060+0000 7fa23b649740 1 --2- 172.21.15.43:0/3963703892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6402a7410 0x55e64065de20 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa22402e610 tx=0x7fa22402a9e0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.060+0000 7fa23b649740 1 -- 172.21.15.43:0/3963703892 shutdown_connections 2024-04-25T10:35:22.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.060+0000 7fa23b649740 1 --2- 172.21.15.43:0/3963703892 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e6402661f0 0x55e640664030 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.060+0000 7fa23b649740 1 --2- 172.21.15.43:0/3963703892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6402a7410 0x55e64065de20 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.060+0000 7fa23b649740 1 --2- 172.21.15.43:0/3963703892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e640286730 0x55e640652140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.060+0000 7fa215ffb640 0 client.16311 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:22.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.060+0000 7fa23b649740 1 --2- 172.21.15.43:0/3963703892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e64027e550 0x55e640651c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.060+0000 7fa23b649740 1 -- 172.21.15.43:0/3963703892 >> 172.21.15.43:0/3963703892 conn(0x55e64029b560 msgr2=0x55e640647450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:22.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.060+0000 7fa23b649740 1 -- 172.21.15.43:0/3963703892 shutdown_connections 2024-04-25T10:35:22.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.060+0000 7fa23b649740 1 -- 172.21.15.43:0/3963703892 wait complete. 2024-04-25T10:35:22.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.060+0000 7fa23b649740 20 client.16311 trim_cache size 0 max 16384 2024-04-25T10:35:22.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.078+0000 7fa23b649740 1 Processor -- start 2024-04-25T10:35:22.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.078+0000 7fa23b649740 1 -- start start 2024-04-25T10:35:22.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.078+0000 7fa23b649740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e64027e550 0x55e64064b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.078+0000 7fa23b649740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e640286730 0x55e640642bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.078+0000 7fa23b649740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6402a7410 0x55e6406697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.078+0000 7fa23b649740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1b2260 con 0x55e6402a7410 2024-04-25T10:35:22.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.078+0000 7fa23b649740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1afb30 con 0x55e640286730 2024-04-25T10:35:22.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.078+0000 7fa23b649740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1ad980 con 0x55e64027e550 2024-04-25T10:35:22.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.079+0000 7fa232d33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6402a7410 0x55e6406697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.079+0000 7fa232d33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6402a7410 0x55e6406697f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45574/0 (socket says 172.21.15.43:45574) 2024-04-25T10:35:22.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.079+0000 7fa232d33640 1 -- 172.21.15.43:0/715333941 learned_addr learned my addr 172.21.15.43:0/715333941 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:22.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.079+0000 7fa232532640 1 --2- 172.21.15.43:0/715333941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e64027e550 0x55e64064b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.079+0000 7fa231d31640 1 --2- 172.21.15.43:0/715333941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e640286730 0x55e640642bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.079+0000 7fa232d33640 1 -- 172.21.15.43:0/715333941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e64027e550 msgr2=0x55e64064b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:22.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.079+0000 7fa232d33640 1 --2- 172.21.15.43:0/715333941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e64027e550 0x55e64064b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.079+0000 7fa232d33640 1 -- 172.21.15.43:0/715333941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e640286730 msgr2=0x55e640642bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:22.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.079+0000 7fa232d33640 1 --2- 172.21.15.43:0/715333941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e640286730 0x55e640642bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.079+0000 7fa232d33640 1 -- 172.21.15.43:0/715333941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6406446d0 con 0x55e6402a7410 2024-04-25T10:35:22.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.079+0000 7fa232d33640 1 --2- 172.21.15.43:0/715333941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6402a7410 0x55e6406697f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa224006010 tx=0x7fa224039fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2537925231f0eb2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:22.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.079+0000 7fa217fff640 1 -- 172.21.15.43:0/715333941 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224054040 con 0x55e6402a7410 2024-04-25T10:35:22.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.079+0000 7fa217fff640 1 -- 172.21.15.43:0/715333941 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa224003e20 con 0x55e6402a7410 2024-04-25T10:35:22.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.079+0000 7fa217fff640 1 -- 172.21.15.43:0/715333941 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2240043b0 con 0x55e6402a7410 2024-04-25T10:35:22.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.080+0000 7fa23b649740 1 -- 172.21.15.43:0/715333941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6402a7410 msgr2=0x55e6406697f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:22.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.080+0000 7fa23b649740 1 --2- 172.21.15.43:0/715333941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6402a7410 0x55e6406697f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa224006010 tx=0x7fa224039fa0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.080+0000 7fa23b649740 1 -- 172.21.15.43:0/715333941 shutdown_connections 2024-04-25T10:35:22.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.080+0000 7fa23b649740 1 --2- 172.21.15.43:0/715333941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e640286730 0x55e640642bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.080+0000 7fa23b649740 1 --2- 172.21.15.43:0/715333941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6402a7410 0x55e6406697f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.080+0000 7fa23b649740 1 --2- 172.21.15.43:0/715333941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e64027e550 0x55e64064b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.080+0000 7fa23b649740 1 -- 172.21.15.43:0/715333941 >> 172.21.15.43:0/715333941 conn(0x55e64029b560 msgr2=0x55e6406424c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:22.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.080+0000 7fa23b649740 1 -- 172.21.15.43:0/715333941 shutdown_connections 2024-04-25T10:35:22.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.080+0000 7fa23b649740 1 -- 172.21.15.43:0/715333941 wait complete. 2024-04-25T10:35:22.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.080+0000 7fa23b649740 1 -- start start 2024-04-25T10:35:22.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.080+0000 7fa23b649740 1 Processor -- start 2024-04-25T10:35:22.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa23b649740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e64027e550 0x55e64065a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa23b649740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e640286730 0x55e6406554e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa23b649740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6402a7410 0x55e640655a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa23b649740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1adff0 con 0x55e64027e550 2024-04-25T10:35:22.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa23b649740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1af460 con 0x55e640286730 2024-04-25T10:35:22.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa23b649740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1ae6c0 con 0x55e6402a7410 2024-04-25T10:35:22.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa23b649740 10 client.0 _ll_register_callbacks cb 0x7fa22402efa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:22.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa232532640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e64027e550 0x55e64065a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa232532640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e64027e550 0x55e64065a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45578/0 (socket says 172.21.15.43:45578) 2024-04-25T10:35:22.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa232532640 1 -- 172.21.15.43:0/528403989 learned_addr learned my addr 172.21.15.43:0/528403989 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:22.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa231d31640 1 --2- 172.21.15.43:0/528403989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e640286730 0x55e6406554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa232d33640 1 --2- 172.21.15.43:0/528403989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6402a7410 0x55e640655a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa232532640 1 -- 172.21.15.43:0/528403989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6402a7410 msgr2=0x55e640655a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:22.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa232532640 1 --2- 172.21.15.43:0/528403989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6402a7410 0x55e640655a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa232532640 1 -- 172.21.15.43:0/528403989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e640286730 msgr2=0x55e6406554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:22.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa232532640 1 --2- 172.21.15.43:0/528403989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e640286730 0x55e6406554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa232532640 1 -- 172.21.15.43:0/528403989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e640655fe0 con 0x55e64027e550 2024-04-25T10:35:22.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa232532640 1 --2- 172.21.15.43:0/528403989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e64027e550 0x55e64065a190 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa228005c60 tx=0x7fa2280de040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4759723fe5531649 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:22.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa215ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:22.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa215ffb640 1 -- 172.21.15.43:0/528403989 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22806c060 con 0x55e64027e550 2024-04-25T10:35:22.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa215ffb640 1 -- 172.21.15.43:0/528403989 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa2280e2270 con 0x55e64027e550 2024-04-25T10:35:22.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa23b649740 1 -- 172.21.15.43:0/528403989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa224003e20 con 0x55e64027e550 2024-04-25T10:35:22.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.081+0000 7fa215ffb640 1 -- 172.21.15.43:0/528403989 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa2280a63b0 con 0x55e64027e550 2024-04-25T10:35:22.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.082+0000 7fa23b649740 10 client.16317 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:22.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.082+0000 7fa23b649740 1 -- 172.21.15.43:0/528403989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e6406cc150 con 0x55e64027e550 2024-04-25T10:35:22.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.082+0000 7fa215ffb640 1 -- 172.21.15.43:0/528403989 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa224003e20 con 0x55e64027e550 2024-04-25T10:35:22.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.082+0000 7fa215ffb640 1 -- 172.21.15.43:0/528403989 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa2280a2ed0 con 0x55e64027e550 2024-04-25T10:35:22.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.082+0000 7fa23b649740 10 client.16317 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:22.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.082+0000 7fa23b649740 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:22.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.082+0000 7fa23b649740 20 client.16317 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:22.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.082+0000 7fa23b649740 1 --2- 172.21.15.43:0/528403989 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e64023b630 0x55e6406ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.082+0000 7fa23b649740 4 client.16317 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:35:22.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.082+0000 7fa23b649740 1 -- 172.21.15.43:0/528403989 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e6405b5580 con 0x55e64023b630 2024-04-25T10:35:22.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.082+0000 7fa231d31640 1 --2- 172.21.15.43:0/528403989 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e64023b630 0x55e6406ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.083+0000 7fa231d31640 1 --2- 172.21.15.43:0/528403989 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e64023b630 0x55e6406ce3f0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f93ee0c33944bb1 server_cookie=2c65478328df65b4 in_seq=0 out_seq=0 2024-04-25T10:35:22.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.083+0000 7fa215ffb640 10 client.16317 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:22.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.084+0000 7fa215ffb640 1 -- 172.21.15.43:0/528403989 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e6405b5580 con 0x55e64023b630 2024-04-25T10:35:22.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.084+0000 7fa215ffb640 10 client.16317 handle_command_reply: tid=0 2024-04-25T10:35:22.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.084+0000 7fa23b649740 1 client.16317 shutdown 2024-04-25T10:35:22.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.084+0000 7fa23b649740 1 -- 172.21.15.43:0/528403989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e64027e550 msgr2=0x55e64065a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:22.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.084+0000 7fa23b649740 1 --2- 172.21.15.43:0/528403989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e64027e550 0x55e64065a190 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa228005c60 tx=0x7fa2280de040 comp rx=0 tx=0).stop 2024-04-25T10:35:22.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.084+0000 7fa23b649740 1 -- 172.21.15.43:0/528403989 shutdown_connections 2024-04-25T10:35:22.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.084+0000 7fa23b649740 1 --2- 172.21.15.43:0/528403989 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e64023b630 0x55e6406ce3f0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.084+0000 7fa23b649740 1 --2- 172.21.15.43:0/528403989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e640286730 0x55e6406554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.084+0000 7fa23b649740 1 --2- 172.21.15.43:0/528403989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6402a7410 0x55e640655a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.084+0000 7fa23b649740 1 --2- 172.21.15.43:0/528403989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e64027e550 0x55e64065a190 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.084+0000 7fa23b649740 1 -- 172.21.15.43:0/528403989 >> 172.21.15.43:0/528403989 conn(0x55e64029b560 msgr2=0x55e6406473b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:22.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.084+0000 7fa215ffb640 0 client.16317 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:22.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.084+0000 7fa23b649740 1 -- 172.21.15.43:0/528403989 shutdown_connections 2024-04-25T10:35:22.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.084+0000 7fa23b649740 1 -- 172.21.15.43:0/528403989 wait complete. 2024-04-25T10:35:22.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.084+0000 7fa23b649740 20 client.16317 trim_cache size 0 max 16384 2024-04-25T10:35:22.098 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:35:22.098 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:35:22.098 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "a2a0092d-1cba-47d0-9ebc-e16f5e62d308", 2024-04-25T10:35:22.098 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:35:22.098 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:35:22.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.085+0000 7fa234796640 1 -- 172.21.15.43:0/2695479568 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa2180afc60 msgr2=0x7fa2180b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:22.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.085+0000 7fa234796640 1 --2- 172.21.15.43:0/2695479568 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa2180afc60 0x7fa2180b2170 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa224009a60 tx=0x7fa22403a040 comp rx=0 tx=0).stop 2024-04-25T10:35:22.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.085+0000 7fa234796640 1 -- 172.21.15.43:0/2695479568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa22c104c50 msgr2=0x7fa22c19a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:22.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.085+0000 7fa234796640 1 --2- 172.21.15.43:0/2695479568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa22c104c50 0x7fa22c19a6e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa22800d950 tx=0x7fa22800de10 comp rx=0 tx=0).stop 2024-04-25T10:35:22.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.085+0000 7fa234796640 1 -- 172.21.15.43:0/2695479568 shutdown_connections 2024-04-25T10:35:22.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.085+0000 7fa234796640 1 --2- 172.21.15.43:0/2695479568 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa2180afc60 0x7fa2180b2170 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.085+0000 7fa234796640 1 --2- 172.21.15.43:0/2695479568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa22c10e060 0x7fa22c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.085+0000 7fa234796640 1 --2- 172.21.15.43:0/2695479568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa22c10b6b0 0x7fa22c19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.085+0000 7fa234796640 1 --2- 172.21.15.43:0/2695479568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa22c104c50 0x7fa22c19a6e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.085+0000 7fa234796640 1 -- 172.21.15.43:0/2695479568 >> 172.21.15.43:0/2695479568 conn(0x7fa22c078100 msgr2=0x7fa22c101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:22.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.085+0000 7fa234796640 1 -- 172.21.15.43:0/2695479568 shutdown_connections 2024-04-25T10:35:22.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.085+0000 7fa234796640 1 -- 172.21.15.43:0/2695479568 wait complete. 2024-04-25T10:35:22.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:35:22.101 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:35:22.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.254+0000 7f6db4ef2640 1 -- 172.21.15.43:0/2671554868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6db010b740 msgr2=0x7f6db010db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:22.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.254+0000 7f6db4ef2640 1 --2- 172.21.15.43:0/2671554868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6db010b740 0x7f6db010db60 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009900 tx=0x7f6d9802a480 comp rx=0 tx=0).stop 2024-04-25T10:35:22.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.254+0000 7f6db4ef2640 1 -- 172.21.15.43:0/2671554868 shutdown_connections 2024-04-25T10:35:22.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.254+0000 7f6db4ef2640 1 --2- 172.21.15.43:0/2671554868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db010e0a0 0x7f6db0110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.254+0000 7f6db4ef2640 1 --2- 172.21.15.43:0/2671554868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6db010b740 0x7f6db010db60 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.254+0000 7f6db4ef2640 1 --2- 172.21.15.43:0/2671554868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6db0104ce0 0x7f6db01050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.254+0000 7f6db4ef2640 1 -- 172.21.15.43:0/2671554868 >> 172.21.15.43:0/2671554868 conn(0x7f6db00fe260 msgr2=0x7f6db0100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:22.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.255+0000 7f6db4ef2640 1 -- 172.21.15.43:0/2671554868 shutdown_connections 2024-04-25T10:35:22.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.255+0000 7f6db4ef2640 1 -- 172.21.15.43:0/2671554868 wait complete. 2024-04-25T10:35:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.255+0000 7f6db4ef2640 1 Processor -- start 2024-04-25T10:35:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.255+0000 7f6db4ef2640 1 -- start start 2024-04-25T10:35:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.256+0000 7f6db4ef2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6db0104ce0 0x7f6db019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.256+0000 7f6db4ef2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db010b740 0x7f6db019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.256+0000 7f6db4ef2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6db010e0a0 0x7f6db019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.256+0000 7f6db4ef2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6db0113d00 con 0x7f6db010e0a0 2024-04-25T10:35:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.256+0000 7f6db4ef2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6db0113b80 con 0x7f6db010b740 2024-04-25T10:35:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.256+0000 7f6db4ef2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6db0113e80 con 0x7f6db0104ce0 2024-04-25T10:35:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.256+0000 7f6daed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6db010e0a0 0x7f6db019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.256+0000 7f6daed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6db010e0a0 0x7f6db019ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45596/0 (socket says 172.21.15.43:45596) 2024-04-25T10:35:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.256+0000 7f6daed9d640 1 -- 172.21.15.43:0/3237572574 learned_addr learned my addr 172.21.15.43:0/3237572574 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.256+0000 7f6dae59c640 1 --2- 172.21.15.43:0/3237572574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6db0104ce0 0x7f6db019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.256+0000 7f6da7fff640 1 --2- 172.21.15.43:0/3237572574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db010b740 0x7f6db019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.256+0000 7f6daed9d640 1 -- 172.21.15.43:0/3237572574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6db0104ce0 msgr2=0x7f6db019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.256+0000 7f6daed9d640 1 --2- 172.21.15.43:0/3237572574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6db0104ce0 0x7f6db019a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.256+0000 7f6daed9d640 1 -- 172.21.15.43:0/3237572574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db010b740 msgr2=0x7f6db019acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.256+0000 7f6daed9d640 1 --2- 172.21.15.43:0/3237572574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db010b740 0x7f6db019acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.256+0000 7f6daed9d640 1 -- 172.21.15.43:0/3237572574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db019f5c0 con 0x7f6db010e0a0 2024-04-25T10:35:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.256+0000 7f6dae59c640 1 --2- 172.21.15.43:0/3237572574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6db0104ce0 0x7f6db019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.256+0000 7f6daed9d640 1 --2- 172.21.15.43:0/3237572574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6db010e0a0 0x7f6db019ef80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f6da800eaa0 tx=0x7f6da800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.257+0000 7f6da77fe640 1 -- 172.21.15.43:0/3237572574 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da800cdb0 con 0x7f6db010e0a0 2024-04-25T10:35:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.257+0000 7f6da77fe640 1 -- 172.21.15.43:0/3237572574 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da8005020 con 0x7f6db010e0a0 2024-04-25T10:35:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.257+0000 7f6db4ef2640 1 -- 172.21.15.43:0/3237572574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db019f8b0 con 0x7f6db010e0a0 2024-04-25T10:35:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.257+0000 7f6da77fe640 1 -- 172.21.15.43:0/3237572574 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da80116d0 con 0x7f6db010e0a0 2024-04-25T10:35:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.257+0000 7f6db4ef2640 1 -- 172.21.15.43:0/3237572574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db019fd80 con 0x7f6db010e0a0 2024-04-25T10:35:22.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.260+0000 7f6dbbda6740 1 Processor -- start 2024-04-25T10:35:22.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.260+0000 7f6da77fe640 1 -- 172.21.15.43:0/3237572574 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f6da80051d0 con 0x7f6db010e0a0 2024-04-25T10:35:22.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.260+0000 7f6dbbda6740 1 -- start start 2024-04-25T10:35:22.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.260+0000 7f6dbbda6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56405ae11390 0x56405b1b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.260+0000 7f6dbbda6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ade84d0 0x56405b1b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.260+0000 7f6dbbda6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405adf06b0 0x56405b1c2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.260+0000 7f6dbbda6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56405b117d80 con 0x56405adf06b0 2024-04-25T10:35:22.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.260+0000 7f6dbbda6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56405b0d18b0 con 0x56405ade84d0 2024-04-25T10:35:22.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.260+0000 7f6dbbda6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56405b113d40 con 0x56405ae11390 2024-04-25T10:35:22.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.261+0000 7f6da7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405adf06b0 0x56405b1c2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.261+0000 7f6dae59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ade84d0 0x56405b1b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.261+0000 7f6da7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405adf06b0 0x56405b1c2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45606/0 (socket says 172.21.15.43:45606) 2024-04-25T10:35:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.261+0000 7f6da7fff640 1 -- 172.21.15.43:0/851994308 learned_addr learned my addr 172.21.15.43:0/851994308 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.261+0000 7f6da7fff640 1 --2- 172.21.15.43:0/851994308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56405ae11390 0x56405b1b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.261+0000 7f6da77fe640 1 --2- 172.21.15.43:0/3237572574 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6d9c0afc60 0x7f6d9c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.261+0000 7f6da7fff640 1 -- 172.21.15.43:0/851994308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56405ae11390 msgr2=0x56405b1b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.261+0000 7f6da7fff640 1 --2- 172.21.15.43:0/851994308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56405ae11390 0x56405b1b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.261+0000 7f6da7fff640 1 -- 172.21.15.43:0/851994308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ade84d0 msgr2=0x56405b1b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.261+0000 7f6da7fff640 1 --2- 172.21.15.43:0/851994308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ade84d0 0x56405b1b1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.261+0000 7f6da7fff640 1 -- 172.21.15.43:0/851994308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56405b0a2a40 con 0x56405adf06b0 2024-04-25T10:35:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.261+0000 7f6daed9d640 1 --2- 172.21.15.43:0/3237572574 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6d9c0afc60 0x7f6d9c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.261+0000 7f6dae59c640 1 --2- 172.21.15.43:0/851994308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ade84d0 0x56405b1b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:35:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.261+0000 7f6da77fe640 1 -- 172.21.15.43:0/3237572574 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6da80df1c0 con 0x7f6db010e0a0 2024-04-25T10:35:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.261+0000 7f6da7fff640 1 --2- 172.21.15.43:0/851994308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405adf06b0 0x56405b1c2d00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009900 tx=0x7f6d98009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80e99060c8a0de7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.262+0000 7f6da4ff9640 1 -- 172.21.15.43:0/851994308 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98035070 con 0x56405adf06b0 2024-04-25T10:35:22.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.262+0000 7f6da4ff9640 1 -- 172.21.15.43:0/851994308 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d98045070 con 0x56405adf06b0 2024-04-25T10:35:22.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.262+0000 7f6da4ff9640 1 -- 172.21.15.43:0/851994308 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98003b80 con 0x56405adf06b0 2024-04-25T10:35:22.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.262+0000 7f6dbbda6740 1 -- 172.21.15.43:0/851994308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405adf06b0 msgr2=0x56405b1c2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:22.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.262+0000 7f6daed9d640 1 --2- 172.21.15.43:0/3237572574 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6d9c0afc60 0x7f6d9c0b2170 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6da80a33b0 tx=0x7f6da8005890 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:22.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.262+0000 7f6dbbda6740 1 --2- 172.21.15.43:0/851994308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405adf06b0 0x56405b1c2d00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009900 tx=0x7f6d98009560 comp rx=0 tx=0).stop 2024-04-25T10:35:22.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.262+0000 7f6dbbda6740 1 -- 172.21.15.43:0/851994308 shutdown_connections 2024-04-25T10:35:22.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.262+0000 7f6dbbda6740 1 --2- 172.21.15.43:0/851994308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ade84d0 0x56405b1b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.262+0000 7f6dbbda6740 1 --2- 172.21.15.43:0/851994308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405adf06b0 0x56405b1c2d00 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.262+0000 7f6dbbda6740 1 --2- 172.21.15.43:0/851994308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56405ae11390 0x56405b1b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.263+0000 7f6dbbda6740 1 -- 172.21.15.43:0/851994308 >> 172.21.15.43:0/851994308 conn(0x56405ae054e0 msgr2=0x56405b1ae210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:22.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.263+0000 7f6dbbda6740 1 -- 172.21.15.43:0/851994308 shutdown_connections 2024-04-25T10:35:22.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.263+0000 7f6dbbda6740 1 -- 172.21.15.43:0/851994308 wait complete. 2024-04-25T10:35:22.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.263+0000 7f6dbbda6740 1 -- start start 2024-04-25T10:35:22.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.263+0000 7f6dbbda6740 1 Processor -- start 2024-04-25T10:35:22.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.263+0000 7f6dbbda6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ade84d0 0x56405b1c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.263+0000 7f6dbbda6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405adf06b0 0x56405b1c0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.263+0000 7f6dbbda6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56405ae11390 0x56405b1c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.263+0000 7f6dbbda6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56405b1c3420 con 0x56405adf06b0 2024-04-25T10:35:22.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.263+0000 7f6dbbda6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56405b09ed10 con 0x56405ade84d0 2024-04-25T10:35:22.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.263+0000 7f6dbbda6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56405b1c35a0 con 0x56405ae11390 2024-04-25T10:35:22.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.263+0000 7f6dbbda6740 10 client.0 _ll_register_callbacks cb 0x56405b0cb210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:22.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.263+0000 7f6dae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405adf06b0 0x56405b1c0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.263+0000 7f6dae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405adf06b0 0x56405b1c0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45608/0 (socket says 172.21.15.43:45608) 2024-04-25T10:35:22.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.263+0000 7f6dae59c640 1 -- 172.21.15.43:0/1775212566 learned_addr learned my addr 172.21.15.43:0/1775212566 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:22.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.264+0000 7f6da7fff640 1 --2- 172.21.15.43:0/1775212566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ade84d0 0x56405b1c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.264+0000 7f6da7fff640 1 --2- 172.21.15.43:0/1775212566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56405ae11390 0x56405b1c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.264+0000 7f6dae59c640 1 -- 172.21.15.43:0/1775212566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56405ae11390 msgr2=0x56405b1c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:22.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.264+0000 7f6dae59c640 1 --2- 172.21.15.43:0/1775212566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56405ae11390 0x56405b1c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.264+0000 7f6dae59c640 1 -- 172.21.15.43:0/1775212566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ade84d0 msgr2=0x56405b1c0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:22.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.264+0000 7f6dae59c640 1 --2- 172.21.15.43:0/1775212566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ade84d0 0x56405b1c0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.264+0000 7f6dae59c640 1 -- 172.21.15.43:0/1775212566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56405aef37d0 con 0x56405adf06b0 2024-04-25T10:35:22.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.264+0000 7f6da7fff640 1 --2- 172.21.15.43:0/1775212566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ade84d0 0x56405b1c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:22.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.264+0000 7f6dae59c640 1 --2- 172.21.15.43:0/1775212566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405adf06b0 0x56405b1c0f50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6da000bf70 tx=0x7f6da00128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1283605d1449c686 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:22.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.264+0000 7f6d95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:22.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.264+0000 7f6d95ffb640 1 -- 172.21.15.43:0/1775212566 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da001f060 con 0x56405adf06b0 2024-04-25T10:35:22.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.264+0000 7f6d95ffb640 1 -- 172.21.15.43:0/1775212566 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6da0010960 con 0x56405adf06b0 2024-04-25T10:35:22.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.264+0000 7f6dbbda6740 1 -- 172.21.15.43:0/1775212566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d98045070 con 0x56405adf06b0 2024-04-25T10:35:22.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.264+0000 7f6d95ffb640 1 -- 172.21.15.43:0/1775212566 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6da0010b60 con 0x56405adf06b0 2024-04-25T10:35:22.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.264+0000 7f6dbbda6740 10 client.16329 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:22.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.264+0000 7f6dbbda6740 1 -- 172.21.15.43:0/1775212566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56405ae293f0 con 0x56405adf06b0 2024-04-25T10:35:22.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.264+0000 7f6d95ffb640 1 -- 172.21.15.43:0/1775212566 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d98045070 con 0x56405adf06b0 2024-04-25T10:35:22.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.265+0000 7f6d95ffb640 1 -- 172.21.15.43:0/1775212566 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6da0019030 con 0x56405adf06b0 2024-04-25T10:35:22.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.265+0000 7f6dbbda6740 10 client.16329 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:22.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.265+0000 7f6dbbda6740 10 client.16329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:22.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.265+0000 7f6dbbda6740 20 client.16329 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:22.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.265+0000 7f6dbbda6740 1 --2- 172.21.15.43:0/1775212566 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56405adcff60 0x56405b1d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.265+0000 7f6dbbda6740 4 client.16329 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:35:22.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.265+0000 7f6dbbda6740 1 -- 172.21.15.43:0/1775212566 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56405b125990 con 0x56405adcff60 2024-04-25T10:35:22.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.265+0000 7f6da7fff640 1 --2- 172.21.15.43:0/1775212566 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56405adcff60 0x56405b1d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.265+0000 7f6d95ffb640 10 client.16329 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:22.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.265+0000 7f6da7fff640 1 --2- 172.21.15.43:0/1775212566 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56405adcff60 0x56405b1d2b50 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29c3e5c4513464cb server_cookie=2ecdf9456e1951e6 in_seq=0 out_seq=0 2024-04-25T10:35:22.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.266+0000 7f6d95ffb640 1 -- 172.21.15.43:0/1775212566 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56405b125990 con 0x56405adcff60 2024-04-25T10:35:22.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.266+0000 7f6d95ffb640 10 client.16329 handle_command_reply: tid=0 2024-04-25T10:35:22.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.266+0000 7f6dbbda6740 1 client.16329 shutdown 2024-04-25T10:35:22.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.266+0000 7f6dbbda6740 1 -- 172.21.15.43:0/1775212566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405adf06b0 msgr2=0x56405b1c0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:22.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.266+0000 7f6dbbda6740 1 --2- 172.21.15.43:0/1775212566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405adf06b0 0x56405b1c0f50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6da000bf70 tx=0x7f6da00128e0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.266+0000 7f6dbbda6740 1 -- 172.21.15.43:0/1775212566 shutdown_connections 2024-04-25T10:35:22.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.266+0000 7f6dbbda6740 1 --2- 172.21.15.43:0/1775212566 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56405adcff60 0x56405b1d2b50 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.266+0000 7f6dbbda6740 1 --2- 172.21.15.43:0/1775212566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56405ae11390 0x56405b1c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.266+0000 7f6dbbda6740 1 --2- 172.21.15.43:0/1775212566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405adf06b0 0x56405b1c0f50 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.266+0000 7f6d95ffb640 0 client.16329 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:22.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.266+0000 7f6dbbda6740 1 --2- 172.21.15.43:0/1775212566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ade84d0 0x56405b1c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.266+0000 7f6dbbda6740 1 -- 172.21.15.43:0/1775212566 >> 172.21.15.43:0/1775212566 conn(0x56405ae054e0 msgr2=0x56405b1b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:22.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.267+0000 7f6dbbda6740 1 -- 172.21.15.43:0/1775212566 shutdown_connections 2024-04-25T10:35:22.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.267+0000 7f6dbbda6740 1 -- 172.21.15.43:0/1775212566 wait complete. 2024-04-25T10:35:22.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.267+0000 7f6dbbda6740 20 client.16329 trim_cache size 0 max 16384 2024-04-25T10:35:22.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.285+0000 7f6dbbda6740 1 Processor -- start 2024-04-25T10:35:22.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.285+0000 7f6dbbda6740 1 -- start start 2024-04-25T10:35:22.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.285+0000 7f6dbbda6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405ade84d0 0x56405b1c0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.285+0000 7f6dbbda6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56405adf06b0 0x56405b1bf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.285+0000 7f6dbbda6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ae11390 0x56405b1bba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.285+0000 7f6dbbda6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6db01affc0 con 0x56405ade84d0 2024-04-25T10:35:22.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.285+0000 7f6dbbda6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6db01ad890 con 0x56405ae11390 2024-04-25T10:35:22.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.285+0000 7f6dbbda6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6db01ab6e0 con 0x56405adf06b0 2024-04-25T10:35:22.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.285+0000 7f6da7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405ade84d0 0x56405b1c0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.285+0000 7f6da7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405ade84d0 0x56405b1c0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45610/0 (socket says 172.21.15.43:45610) 2024-04-25T10:35:22.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.285+0000 7f6da7fff640 1 -- 172.21.15.43:0/732396017 learned_addr learned my addr 172.21.15.43:0/732396017 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:22.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.285+0000 7f6da7fff640 1 --2- 172.21.15.43:0/732396017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ae11390 0x56405b1bba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.285+0000 7f6dae59c640 1 --2- 172.21.15.43:0/732396017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56405adf06b0 0x56405b1bf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.285+0000 7f6da7fff640 1 -- 172.21.15.43:0/732396017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56405adf06b0 msgr2=0x56405b1bf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:22.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.285+0000 7f6da7fff640 1 --2- 172.21.15.43:0/732396017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56405adf06b0 0x56405b1bf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.285+0000 7f6da7fff640 1 -- 172.21.15.43:0/732396017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ae11390 msgr2=0x56405b1bba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:22.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.285+0000 7f6da7fff640 1 --2- 172.21.15.43:0/732396017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ae11390 0x56405b1bba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.285+0000 7f6da7fff640 1 -- 172.21.15.43:0/732396017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56405ade90f0 con 0x56405ade84d0 2024-04-25T10:35:22.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.286+0000 7f6da7fff640 1 --2- 172.21.15.43:0/732396017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405ade84d0 0x56405b1c0450 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6d98003790 tx=0x7f6d98009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa785ab75d38db98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:22.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.286+0000 7f6d97fff640 1 -- 172.21.15.43:0/732396017 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98035070 con 0x56405ade84d0 2024-04-25T10:35:22.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.286+0000 7f6d97fff640 1 -- 172.21.15.43:0/732396017 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d98045040 con 0x56405ade84d0 2024-04-25T10:35:22.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.286+0000 7f6d97fff640 1 -- 172.21.15.43:0/732396017 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98005650 con 0x56405ade84d0 2024-04-25T10:35:22.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.286+0000 7f6dbbda6740 1 -- 172.21.15.43:0/732396017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405ade84d0 msgr2=0x56405b1c0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:22.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.286+0000 7f6dbbda6740 1 --2- 172.21.15.43:0/732396017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405ade84d0 0x56405b1c0450 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6d98003790 tx=0x7f6d98009680 comp rx=0 tx=0).stop 2024-04-25T10:35:22.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.287+0000 7f6dbbda6740 1 -- 172.21.15.43:0/732396017 shutdown_connections 2024-04-25T10:35:22.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.287+0000 7f6dbbda6740 1 --2- 172.21.15.43:0/732396017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ae11390 0x56405b1bba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.287+0000 7f6dbbda6740 1 --2- 172.21.15.43:0/732396017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56405adf06b0 0x56405b1bf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.287+0000 7f6dbbda6740 1 --2- 172.21.15.43:0/732396017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405ade84d0 0x56405b1c0450 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.287+0000 7f6dbbda6740 1 -- 172.21.15.43:0/732396017 >> 172.21.15.43:0/732396017 conn(0x56405ae054e0 msgr2=0x56405b1bfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:22.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.287+0000 7f6dbbda6740 1 -- 172.21.15.43:0/732396017 shutdown_connections 2024-04-25T10:35:22.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.287+0000 7f6dbbda6740 1 -- 172.21.15.43:0/732396017 wait complete. 2024-04-25T10:35:22.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.287+0000 7f6dbbda6740 1 -- start start 2024-04-25T10:35:22.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.287+0000 7f6dbbda6740 1 Processor -- start 2024-04-25T10:35:22.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.287+0000 7f6dbbda6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405ade84d0 0x56405b1ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.287+0000 7f6dbbda6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56405adf06b0 0x56405b1c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.287+0000 7f6dbbda6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ae11390 0x56405b1c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.287+0000 7f6dbbda6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6db01abd50 con 0x56405ade84d0 2024-04-25T10:35:22.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.287+0000 7f6dbbda6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6db01ad1c0 con 0x56405ae11390 2024-04-25T10:35:22.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.287+0000 7f6dbbda6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6db01ac420 con 0x56405adf06b0 2024-04-25T10:35:22.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.287+0000 7f6dbbda6740 10 client.0 _ll_register_callbacks cb 0x7f6da000b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:22.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.287+0000 7f6da7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405ade84d0 0x56405b1ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.287+0000 7f6da7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405ade84d0 0x56405b1ccc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45612/0 (socket says 172.21.15.43:45612) 2024-04-25T10:35:22.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.287+0000 7f6da7fff640 1 -- 172.21.15.43:0/1045825899 learned_addr learned my addr 172.21.15.43:0/1045825899 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:22.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.288+0000 7f6dae59c640 1 --2- 172.21.15.43:0/1045825899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56405adf06b0 0x56405b1c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.288+0000 7f6da7fff640 1 --2- 172.21.15.43:0/1045825899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ae11390 0x56405b1c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.288+0000 7f6da7fff640 1 -- 172.21.15.43:0/1045825899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56405adf06b0 msgr2=0x56405b1c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:22.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.288+0000 7f6da7fff640 1 --2- 172.21.15.43:0/1045825899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56405adf06b0 0x56405b1c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.288+0000 7f6da7fff640 1 -- 172.21.15.43:0/1045825899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ae11390 msgr2=0x56405b1c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:22.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.288+0000 7f6da7fff640 1 --2- 172.21.15.43:0/1045825899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ae11390 0x56405b1c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.288+0000 7f6da7fff640 1 -- 172.21.15.43:0/1045825899 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56405b0d84b0 con 0x56405ade84d0 2024-04-25T10:35:22.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.288+0000 7f6da7fff640 1 --2- 172.21.15.43:0/1045825899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405ade84d0 0x56405b1ccc00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6d98005590 tx=0x7f6d98005490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b303025349122b73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:22.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.288+0000 7f6d95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:22.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.288+0000 7f6d95ffb640 1 -- 172.21.15.43:0/1045825899 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98035070 con 0x56405ade84d0 2024-04-25T10:35:22.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.288+0000 7f6d95ffb640 1 -- 172.21.15.43:0/1045825899 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6d98052130 con 0x56405ade84d0 2024-04-25T10:35:22.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.288+0000 7f6dbbda6740 1 -- 172.21.15.43:0/1045825899 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d98045040 con 0x56405ade84d0 2024-04-25T10:35:22.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.288+0000 7f6d95ffb640 1 -- 172.21.15.43:0/1045825899 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6d98066050 con 0x56405ade84d0 2024-04-25T10:35:22.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.288+0000 7f6dbbda6740 10 client.16335 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:22.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.288+0000 7f6dbbda6740 1 -- 172.21.15.43:0/1045825899 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56405b23ad10 con 0x56405ade84d0 2024-04-25T10:35:22.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.288+0000 7f6d95ffb640 1 -- 172.21.15.43:0/1045825899 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d98045040 con 0x56405ade84d0 2024-04-25T10:35:22.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.289+0000 7f6d95ffb640 1 -- 172.21.15.43:0/1045825899 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6d9805e020 con 0x56405ade84d0 2024-04-25T10:35:22.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.289+0000 7f6dbbda6740 10 client.16335 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:22.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.289+0000 7f6dbbda6740 10 client.16335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:22.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.289+0000 7f6dbbda6740 20 client.16335 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:22.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.289+0000 7f6dbbda6740 1 --2- 172.21.15.43:0/1045825899 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56405ada53a0 0x56405b23cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:22.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.289+0000 7f6dbbda6740 4 client.16335 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:35:22.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.289+0000 7f6dbbda6740 1 -- 172.21.15.43:0/1045825899 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56405b11f800 con 0x56405ada53a0 2024-04-25T10:35:22.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.289+0000 7f6dae59c640 1 --2- 172.21.15.43:0/1045825899 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56405ada53a0 0x56405b23cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:22.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.289+0000 7f6d95ffb640 10 client.16335 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:22.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.289+0000 7f6dae59c640 1 --2- 172.21.15.43:0/1045825899 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56405ada53a0 0x56405b23cfd0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96f1a2e8c838b8ba server_cookie=14d6cf6572393ade in_seq=0 out_seq=0 2024-04-25T10:35:22.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6d95ffb640 1 -- 172.21.15.43:0/1045825899 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56405b11f800 con 0x56405ada53a0 2024-04-25T10:35:22.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6d95ffb640 10 client.16335 handle_command_reply: tid=0 2024-04-25T10:35:22.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6dbbda6740 1 client.16335 shutdown 2024-04-25T10:35:22.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6dbbda6740 1 -- 172.21.15.43:0/1045825899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405ade84d0 msgr2=0x56405b1ccc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:22.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6dbbda6740 1 --2- 172.21.15.43:0/1045825899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405ade84d0 0x56405b1ccc00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6d98005590 tx=0x7f6d98005490 comp rx=0 tx=0).stop 2024-04-25T10:35:22.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6dbbda6740 1 -- 172.21.15.43:0/1045825899 shutdown_connections 2024-04-25T10:35:22.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6dbbda6740 1 --2- 172.21.15.43:0/1045825899 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56405ada53a0 0x56405b23cfd0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6dbbda6740 1 --2- 172.21.15.43:0/1045825899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405ae11390 0x56405b1c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6dbbda6740 1 --2- 172.21.15.43:0/1045825899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56405adf06b0 0x56405b1c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6dbbda6740 1 --2- 172.21.15.43:0/1045825899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56405ade84d0 0x56405b1ccc00 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6dbbda6740 1 -- 172.21.15.43:0/1045825899 >> 172.21.15.43:0/1045825899 conn(0x56405ae054e0 msgr2=0x56405b1c1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:22.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6d95ffb640 0 client.16335 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:22.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6dbbda6740 1 -- 172.21.15.43:0/1045825899 shutdown_connections 2024-04-25T10:35:22.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6dbbda6740 1 -- 172.21.15.43:0/1045825899 wait complete. 2024-04-25T10:35:22.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6dbbda6740 20 client.16335 trim_cache size 0 max 16384 2024-04-25T10:35:22.299 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:35:22.299 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:35:22.299 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:35:22.299 INFO:teuthology.orchestra.run.smithi043.stdout: "a2a0092d-1cba-47d0-9ebc-e16f5e62d308": { 2024-04-25T10:35:22.299 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:35:22.299 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "a2a0092d-1cba-47d0-9ebc-e16f5e62d308", 2024-04-25T10:35:22.299 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:35:22.300 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:35:22.300 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:35:22.300 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:35:22.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6db4ef2640 1 -- 172.21.15.43:0/3237572574 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6d9c0afc60 msgr2=0x7f6d9c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:22.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6db4ef2640 1 --2- 172.21.15.43:0/3237572574 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6d9c0afc60 0x7f6d9c0b2170 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6da80a33b0 tx=0x7f6da8005890 comp rx=0 tx=0).stop 2024-04-25T10:35:22.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6db4ef2640 1 -- 172.21.15.43:0/3237572574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6db010e0a0 msgr2=0x7f6db019ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:22.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6db4ef2640 1 --2- 172.21.15.43:0/3237572574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6db010e0a0 0x7f6db019ef80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f6da800eaa0 tx=0x7f6da800ef60 comp rx=0 tx=0).stop 2024-04-25T10:35:22.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6db4ef2640 1 -- 172.21.15.43:0/3237572574 shutdown_connections 2024-04-25T10:35:22.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6db4ef2640 1 --2- 172.21.15.43:0/3237572574 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6d9c0afc60 0x7f6d9c0b2170 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6db4ef2640 1 --2- 172.21.15.43:0/3237572574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db010b740 0x7f6db019acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6db4ef2640 1 --2- 172.21.15.43:0/3237572574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6db010e0a0 0x7f6db019ef80 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6db4ef2640 1 --2- 172.21.15.43:0/3237572574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6db0104ce0 0x7f6db019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:22.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.290+0000 7f6db4ef2640 1 -- 172.21.15.43:0/3237572574 >> 172.21.15.43:0/3237572574 conn(0x7f6db00fe260 msgr2=0x7f6db010e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:22.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.291+0000 7f6db4ef2640 1 -- 172.21.15.43:0/3237572574 shutdown_connections 2024-04-25T10:35:22.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:22.291+0000 7f6db4ef2640 1 -- 172.21.15.43:0/3237572574 wait complete. 2024-04-25T10:35:22.304 INFO:tasks.cephfs.filesystem:scrub status for tag:a2a0092d-1cba-47d0-9ebc-e16f5e62d308 - {'path': '/', 'tag': 'a2a0092d-1cba-47d0-9ebc-e16f5e62d308', 'options': 'recursive,force'} 2024-04-25T10:35:23.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:22 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:35:23.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:22 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:35:23.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:22 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:35:23.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:22 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:35:23.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:22 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:35:23.131 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:22 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:35:23.131 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:22 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:35:23.131 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:22 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:35:23.132 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:22 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:35:23.132 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:22 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:35:23.132 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:22 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:35:23.132 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:22 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:35:24.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:23 smithi077 ceph-mon[32772]: pgmap v89: 97 pgs: 97 active+clean; 2.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-04-25T10:35:24.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:23 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:35:24.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:23 smithi164 ceph-mon[32713]: pgmap v89: 97 pgs: 97 active+clean; 2.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-04-25T10:35:24.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:23 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:35:24.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:23 smithi043 ceph-mon[29478]: pgmap v89: 97 pgs: 97 active+clean; 2.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-04-25T10:35:24.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:23 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:35:26.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:25 smithi077 ceph-mon[32772]: pgmap v90: 97 pgs: 97 active+clean; 2.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 19 op/s 2024-04-25T10:35:26.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:25 smithi164 ceph-mon[32713]: pgmap v90: 97 pgs: 97 active+clean; 2.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 19 op/s 2024-04-25T10:35:26.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:25 smithi043 ceph-mon[29478]: pgmap v90: 97 pgs: 97 active+clean; 2.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 19 op/s 2024-04-25T10:35:27.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:27 smithi043 ceph-mon[29478]: pgmap v91: 97 pgs: 97 active+clean; 2.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-25T10:35:28.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:27 smithi077 ceph-mon[32772]: pgmap v91: 97 pgs: 97 active+clean; 2.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-25T10:35:28.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:27 smithi164 ceph-mon[32713]: pgmap v91: 97 pgs: 97 active+clean; 2.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-25T10:35:30.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:29 smithi164 ceph-mon[32713]: pgmap v92: 97 pgs: 97 active+clean; 3.0 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 18 op/s 2024-04-25T10:35:30.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:29 smithi077 ceph-mon[32772]: pgmap v92: 97 pgs: 97 active+clean; 3.0 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 18 op/s 2024-04-25T10:35:30.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:29 smithi043 ceph-mon[29478]: pgmap v92: 97 pgs: 97 active+clean; 3.0 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 18 op/s 2024-04-25T10:35:32.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:31 smithi077 ceph-mon[32772]: pgmap v93: 97 pgs: 97 active+clean; 3.0 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-04-25T10:35:32.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:31 smithi164 ceph-mon[32713]: pgmap v93: 97 pgs: 97 active+clean; 3.0 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-04-25T10:35:32.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:31 smithi043 ceph-mon[29478]: pgmap v93: 97 pgs: 97 active+clean; 3.0 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-04-25T10:35:34.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:33 smithi077 ceph-mon[32772]: pgmap v94: 97 pgs: 97 active+clean; 3.0 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-04-25T10:35:34.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:33 smithi164 ceph-mon[32713]: pgmap v94: 97 pgs: 97 active+clean; 3.0 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-04-25T10:35:34.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:33 smithi043 ceph-mon[29478]: pgmap v94: 97 pgs: 97 active+clean; 3.0 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-04-25T10:35:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.780+0000 7f1510b2e640 1 -- 172.21.15.43:0/1553656861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f150c108a50 msgr2=0x7f150c102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.780+0000 7f1510b2e640 1 --2- 172.21.15.43:0/1553656861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f150c108a50 0x7f150c102600 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009900 tx=0x7f14f402a410 comp rx=0 tx=0).stop 2024-04-25T10:35:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.780+0000 7f1510b2e640 1 -- 172.21.15.43:0/1553656861 shutdown_connections 2024-04-25T10:35:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.780+0000 7f1510b2e640 1 --2- 172.21.15.43:0/1553656861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150c108ef0 0x7f150c102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.780+0000 7f1510b2e640 1 --2- 172.21.15.43:0/1553656861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f150c1031b0 0x7f150c10f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.780+0000 7f1510b2e640 1 --2- 172.21.15.43:0/1553656861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f150c108a50 0x7f150c102600 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.780+0000 7f1510b2e640 1 -- 172.21.15.43:0/1553656861 >> 172.21.15.43:0/1553656861 conn(0x7f150c0fe260 msgr2=0x7f150c100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.780+0000 7f1510b2e640 1 -- 172.21.15.43:0/1553656861 shutdown_connections 2024-04-25T10:35:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.780+0000 7f1510b2e640 1 -- 172.21.15.43:0/1553656861 wait complete. 2024-04-25T10:35:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.781+0000 7f1510b2e640 1 Processor -- start 2024-04-25T10:35:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.781+0000 7f1510b2e640 1 -- start start 2024-04-25T10:35:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.782+0000 7f1510b2e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f150c1031b0 0x7f150c110250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.782+0000 7f1510b2e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150c108a50 0x7f150c110790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.782+0000 7f1510b2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f150c108ef0 0x7f150c110cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.782+0000 7f1510b2e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f150c113d70 con 0x7f150c108ef0 2024-04-25T10:35:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.782+0000 7f1510b2e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f150c113bf0 con 0x7f150c108a50 2024-04-25T10:35:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.782+0000 7f1510b2e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f150c113ef0 con 0x7f150c1031b0 2024-04-25T10:35:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.782+0000 7f150ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f150c108ef0 0x7f150c110cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.782+0000 7f150ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f150c108ef0 0x7f150c110cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41418/0 (socket says 172.21.15.43:41418) 2024-04-25T10:35:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.782+0000 7f150ad9d640 1 -- 172.21.15.43:0/1079748389 learned_addr learned my addr 172.21.15.43:0/1079748389 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.782+0000 7f150a59c640 1 --2- 172.21.15.43:0/1079748389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f150c1031b0 0x7f150c110250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.782+0000 7f1509d9b640 1 --2- 172.21.15.43:0/1079748389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150c108a50 0x7f150c110790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.782+0000 7f150ad9d640 1 -- 172.21.15.43:0/1079748389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f150c1031b0 msgr2=0x7f150c110250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.782+0000 7f150ad9d640 1 --2- 172.21.15.43:0/1079748389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f150c1031b0 0x7f150c110250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.782+0000 7f150ad9d640 1 -- 172.21.15.43:0/1079748389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150c108a50 msgr2=0x7f150c110790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.782+0000 7f150ad9d640 1 --2- 172.21.15.43:0/1079748389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150c108a50 0x7f150c110790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.782+0000 7f150ad9d640 1 -- 172.21.15.43:0/1079748389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150c1a8ad0 con 0x7f150c108ef0 2024-04-25T10:35:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.782+0000 7f150ad9d640 1 --2- 172.21.15.43:0/1079748389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f150c108ef0 0x7f150c110cd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f150400eb10 tx=0x7f150400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.783+0000 7f15037fe640 1 -- 172.21.15.43:0/1079748389 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504004b20 con 0x7f150c108ef0 2024-04-25T10:35:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.783+0000 7f15037fe640 1 -- 172.21.15.43:0/1079748389 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1504004cc0 con 0x7f150c108ef0 2024-04-25T10:35:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.783+0000 7f1510b2e640 1 -- 172.21.15.43:0/1079748389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150c1a8dc0 con 0x7f150c108ef0 2024-04-25T10:35:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.783+0000 7f15037fe640 1 -- 172.21.15.43:0/1079748389 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504010670 con 0x7f150c108ef0 2024-04-25T10:35:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.783+0000 7f1510b2e640 1 -- 172.21.15.43:0/1079748389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f150c1a9280 con 0x7f150c108ef0 2024-04-25T10:35:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.785+0000 7f15179db740 1 Processor -- start 2024-04-25T10:35:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.785+0000 7f15037fe640 1 -- 172.21.15.43:0/1079748389 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f1504005020 con 0x7f150c108ef0 2024-04-25T10:35:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.786+0000 7f15179db740 1 -- start start 2024-04-25T10:35:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.786+0000 7f15179db740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2db1b0 0x55930e67aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.786+0000 7f15179db740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e2b22f0 0x55930e67b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.786+0000 7f15179db740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e2ba4d0 0x55930e67b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.786+0000 7f15179db740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55930e5e1460 con 0x55930e2db1b0 2024-04-25T10:35:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.786+0000 7f15179db740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55930e59b590 con 0x55930e2b22f0 2024-04-25T10:35:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.786+0000 7f15179db740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55930e5dd420 con 0x55930e2ba4d0 2024-04-25T10:35:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.786+0000 7f1509d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2db1b0 0x55930e67aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.786+0000 7f1509d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2db1b0 0x55930e67aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41420/0 (socket says 172.21.15.43:41420) 2024-04-25T10:35:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.786+0000 7f1509d9b640 1 -- 172.21.15.43:0/2903935111 learned_addr learned my addr 172.21.15.43:0/2903935111 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.786+0000 7f150a59c640 1 --2- 172.21.15.43:0/2903935111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e2b22f0 0x55930e67b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.786+0000 7f1509d9b640 1 --2- 172.21.15.43:0/2903935111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e2ba4d0 0x55930e67b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.786+0000 7f15037fe640 1 --2- 172.21.15.43:0/1079748389 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f14f80afc50 0x7f14f80b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.786+0000 7f1509d9b640 1 -- 172.21.15.43:0/2903935111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e2ba4d0 msgr2=0x55930e67b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.786+0000 7f1509d9b640 1 --2- 172.21.15.43:0/2903935111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e2ba4d0 0x55930e67b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.786+0000 7f1509d9b640 1 -- 172.21.15.43:0/2903935111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e2b22f0 msgr2=0x55930e67b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.786+0000 7f1509d9b640 1 --2- 172.21.15.43:0/2903935111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e2b22f0 0x55930e67b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.786+0000 7f1509d9b640 1 -- 172.21.15.43:0/2903935111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55930e5a80d0 con 0x55930e2db1b0 2024-04-25T10:35:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.786+0000 7f150ad9d640 1 --2- 172.21.15.43:0/1079748389 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f14f80afc50 0x7f14f80b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.787+0000 7f1509d9b640 1 --2- 172.21.15.43:0/2903935111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2db1b0 0x55930e67aaf0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f14fc013480 tx=0x7f14fc00b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8248f97d00fc88c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.787+0000 7f15037fe640 1 -- 172.21.15.43:0/1079748389 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f15040df700 con 0x7f150c108ef0 2024-04-25T10:35:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.787+0000 7f1500ff9640 1 -- 172.21.15.43:0/2903935111 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc019030 con 0x55930e2db1b0 2024-04-25T10:35:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.787+0000 7f1500ff9640 1 -- 172.21.15.43:0/2903935111 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14fc0035f0 con 0x55930e2db1b0 2024-04-25T10:35:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.787+0000 7f1500ff9640 1 -- 172.21.15.43:0/2903935111 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc0116a0 con 0x55930e2db1b0 2024-04-25T10:35:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.788+0000 7f15179db740 1 -- 172.21.15.43:0/2903935111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2db1b0 msgr2=0x55930e67aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.788+0000 7f15179db740 1 --2- 172.21.15.43:0/2903935111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2db1b0 0x55930e67aaf0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f14fc013480 tx=0x7f14fc00b5a0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.788+0000 7f150ad9d640 1 --2- 172.21.15.43:0/1079748389 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f14f80afc50 0x7f14f80b2160 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1504063250 tx=0x7f150400cf70 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.788+0000 7f15179db740 1 -- 172.21.15.43:0/2903935111 shutdown_connections 2024-04-25T10:35:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.788+0000 7f15179db740 1 --2- 172.21.15.43:0/2903935111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e2b22f0 0x55930e67b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.788+0000 7f15179db740 1 --2- 172.21.15.43:0/2903935111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e2ba4d0 0x55930e67b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.788+0000 7f15179db740 1 --2- 172.21.15.43:0/2903935111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2db1b0 0x55930e67aaf0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.788+0000 7f15179db740 1 -- 172.21.15.43:0/2903935111 >> 172.21.15.43:0/2903935111 conn(0x55930e2cf300 msgr2=0x55930e6772a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.788+0000 7f15179db740 1 -- 172.21.15.43:0/2903935111 shutdown_connections 2024-04-25T10:35:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.788+0000 7f15179db740 1 -- 172.21.15.43:0/2903935111 wait complete. 2024-04-25T10:35:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.788+0000 7f15179db740 1 -- start start 2024-04-25T10:35:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.788+0000 7f15179db740 1 Processor -- start 2024-04-25T10:35:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.789+0000 7f15179db740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e2b22f0 0x55930e6851e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.789+0000 7f15179db740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2ba4d0 0x55930e685720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.789+0000 7f15179db740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e2db1b0 0x55930e68b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.789+0000 7f15179db740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55930e687c20 con 0x55930e2ba4d0 2024-04-25T10:35:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.789+0000 7f15179db740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55930e568b60 con 0x55930e2db1b0 2024-04-25T10:35:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.789+0000 7f15179db740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55930e687da0 con 0x55930e2b22f0 2024-04-25T10:35:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.789+0000 7f15179db740 10 client.0 _ll_register_callbacks cb 0x55930e58d2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.789+0000 7f150a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2ba4d0 0x55930e685720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.789+0000 7f150a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2ba4d0 0x55930e685720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41436/0 (socket says 172.21.15.43:41436) 2024-04-25T10:35:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.789+0000 7f150a59c640 1 -- 172.21.15.43:0/1546537025 learned_addr learned my addr 172.21.15.43:0/1546537025 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.789+0000 7f1509d9b640 1 --2- 172.21.15.43:0/1546537025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e2db1b0 0x55930e68b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.789+0000 7f1509d9b640 1 --2- 172.21.15.43:0/1546537025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e2b22f0 0x55930e6851e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.789+0000 7f150a59c640 1 -- 172.21.15.43:0/1546537025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e2b22f0 msgr2=0x55930e6851e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.789+0000 7f150a59c640 1 --2- 172.21.15.43:0/1546537025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e2b22f0 0x55930e6851e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.789+0000 7f150a59c640 1 -- 172.21.15.43:0/1546537025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e2db1b0 msgr2=0x55930e68b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.789+0000 7f150a59c640 1 --2- 172.21.15.43:0/1546537025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e2db1b0 0x55930e68b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.789+0000 7f150a59c640 1 -- 172.21.15.43:0/1546537025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55930e2f3210 con 0x55930e2ba4d0 2024-04-25T10:35:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.789+0000 7f1509d9b640 1 --2- 172.21.15.43:0/1546537025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e2db1b0 0x55930e68b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.789+0000 7f150a59c640 1 --2- 172.21.15.43:0/1546537025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2ba4d0 0x55930e685720 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f150c103740 tx=0x7f14f40316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7874217abeb5625b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.789+0000 7f14f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.790+0000 7f14f1ffb640 1 -- 172.21.15.43:0/1546537025 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4035070 con 0x55930e2ba4d0 2024-04-25T10:35:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.790+0000 7f15179db740 1 -- 172.21.15.43:0/1546537025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14fc0035f0 con 0x55930e2ba4d0 2024-04-25T10:35:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.790+0000 7f14f1ffb640 1 -- 172.21.15.43:0/1546537025 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f14f4004040 con 0x55930e2ba4d0 2024-04-25T10:35:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.790+0000 7f14f1ffb640 1 -- 172.21.15.43:0/1546537025 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f14f4004240 con 0x55930e2ba4d0 2024-04-25T10:35:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.790+0000 7f15179db740 10 client.16347 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.790+0000 7f15179db740 1 -- 172.21.15.43:0/1546537025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55930e3bf620 con 0x55930e2ba4d0 2024-04-25T10:35:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.790+0000 7f14f1ffb640 1 -- 172.21.15.43:0/1546537025 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14fc0035f0 con 0x55930e2ba4d0 2024-04-25T10:35:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.790+0000 7f14f1ffb640 1 -- 172.21.15.43:0/1546537025 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f14f4004980 con 0x55930e2ba4d0 2024-04-25T10:35:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.790+0000 7f15179db740 10 client.16347 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.790+0000 7f15179db740 10 client.16347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.790+0000 7f15179db740 20 client.16347 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.790+0000 7f15179db740 1 --2- 172.21.15.43:0/1546537025 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55930e299d80 0x55930e6976e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.790+0000 7f15179db740 4 client.16347 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:35:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.790+0000 7f15179db740 1 -- 172.21.15.43:0/1546537025 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55930e5ef070 con 0x55930e299d80 2024-04-25T10:35:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.791+0000 7f1509d9b640 1 --2- 172.21.15.43:0/1546537025 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55930e299d80 0x55930e6976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.791+0000 7f14f1ffb640 10 client.16347 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.791+0000 7f1509d9b640 1 --2- 172.21.15.43:0/1546537025 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55930e299d80 0x55930e6976e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=501ec18965eea707 server_cookie=f1758424ee9dbdb in_seq=0 out_seq=0 2024-04-25T10:35:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.792+0000 7f14f1ffb640 1 -- 172.21.15.43:0/1546537025 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55930e5ef070 con 0x55930e299d80 2024-04-25T10:35:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.792+0000 7f14f1ffb640 10 client.16347 handle_command_reply: tid=0 2024-04-25T10:35:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.792+0000 7f15179db740 1 client.16347 shutdown 2024-04-25T10:35:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.792+0000 7f15179db740 1 -- 172.21.15.43:0/1546537025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2ba4d0 msgr2=0x55930e685720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.792+0000 7f15179db740 1 --2- 172.21.15.43:0/1546537025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2ba4d0 0x55930e685720 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f150c103740 tx=0x7f14f40316d0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.792+0000 7f15179db740 1 -- 172.21.15.43:0/1546537025 shutdown_connections 2024-04-25T10:35:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.792+0000 7f15179db740 1 --2- 172.21.15.43:0/1546537025 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55930e299d80 0x55930e6976e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.792+0000 7f15179db740 1 --2- 172.21.15.43:0/1546537025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e2db1b0 0x55930e68b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.792+0000 7f15179db740 1 --2- 172.21.15.43:0/1546537025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2ba4d0 0x55930e685720 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.792+0000 7f14f1ffb640 0 client.16347 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.792+0000 7f15179db740 1 --2- 172.21.15.43:0/1546537025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e2b22f0 0x55930e6851e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.792+0000 7f15179db740 1 -- 172.21.15.43:0/1546537025 >> 172.21.15.43:0/1546537025 conn(0x55930e2cf300 msgr2=0x55930e67a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.792+0000 7f15179db740 1 -- 172.21.15.43:0/1546537025 shutdown_connections 2024-04-25T10:35:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.792+0000 7f15179db740 1 -- 172.21.15.43:0/1546537025 wait complete. 2024-04-25T10:35:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.792+0000 7f15179db740 20 client.16347 trim_cache size 0 max 16384 2024-04-25T10:35:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.808+0000 7f15179db740 1 Processor -- start 2024-04-25T10:35:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.808+0000 7f15179db740 1 -- start start 2024-04-25T10:35:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.808+0000 7f15179db740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e299d80 0x55930e68b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.808+0000 7f15179db740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2b22f0 0x55930e69cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.808+0000 7f15179db740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e2ba4d0 0x55930e69d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.808+0000 7f15179db740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1b43f0 con 0x55930e2b22f0 2024-04-25T10:35:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.808+0000 7f15179db740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1b1cc0 con 0x55930e2ba4d0 2024-04-25T10:35:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.808+0000 7f15179db740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1afb10 con 0x55930e299d80 2024-04-25T10:35:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.808+0000 7f150a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2b22f0 0x55930e69cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.808+0000 7f150a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2b22f0 0x55930e69cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41438/0 (socket says 172.21.15.43:41438) 2024-04-25T10:35:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.808+0000 7f150a59c640 1 -- 172.21.15.43:0/3230771639 learned_addr learned my addr 172.21.15.43:0/3230771639 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.808+0000 7f150a59c640 1 -- 172.21.15.43:0/3230771639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e299d80 msgr2=0x55930e68b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.808+0000 7f1509d9b640 1 --2- 172.21.15.43:0/3230771639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e299d80 0x55930e68b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.808+0000 7f150a59c640 1 --2- 172.21.15.43:0/3230771639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e299d80 0x55930e68b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.808+0000 7f150a59c640 1 -- 172.21.15.43:0/3230771639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e2ba4d0 msgr2=0x55930e69d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.808+0000 7f150a59c640 1 --2- 172.21.15.43:0/3230771639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e2ba4d0 0x55930e69d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.809+0000 7f150a59c640 1 -- 172.21.15.43:0/3230771639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55930e67bbf0 con 0x55930e2b22f0 2024-04-25T10:35:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.809+0000 7f1509d9b640 1 --2- 172.21.15.43:0/3230771639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e299d80 0x55930e68b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:35:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.809+0000 7f150a59c640 1 --2- 172.21.15.43:0/3230771639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2b22f0 0x55930e69cec0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f14f402a930 tx=0x7f14f4005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8fe3ac631c8818 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.809+0000 7f14f3fff640 1 -- 172.21.15.43:0/3230771639 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4051070 con 0x55930e2b22f0 2024-04-25T10:35:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.809+0000 7f14f3fff640 1 -- 172.21.15.43:0/3230771639 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14f4003a60 con 0x55930e2b22f0 2024-04-25T10:35:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.809+0000 7f14f3fff640 1 -- 172.21.15.43:0/3230771639 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4035040 con 0x55930e2b22f0 2024-04-25T10:35:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.810+0000 7f15179db740 1 -- 172.21.15.43:0/3230771639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2b22f0 msgr2=0x55930e69cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.810+0000 7f15179db740 1 --2- 172.21.15.43:0/3230771639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2b22f0 0x55930e69cec0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f14f402a930 tx=0x7f14f4005e90 comp rx=0 tx=0).stop 2024-04-25T10:35:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.810+0000 7f15179db740 1 -- 172.21.15.43:0/3230771639 shutdown_connections 2024-04-25T10:35:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.810+0000 7f15179db740 1 --2- 172.21.15.43:0/3230771639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e2ba4d0 0x55930e69d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.810+0000 7f15179db740 1 --2- 172.21.15.43:0/3230771639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2b22f0 0x55930e69cec0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.810+0000 7f15179db740 1 --2- 172.21.15.43:0/3230771639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e299d80 0x55930e68b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.810+0000 7f15179db740 1 -- 172.21.15.43:0/3230771639 >> 172.21.15.43:0/3230771639 conn(0x55930e2cf300 msgr2=0x55930e6838a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.810+0000 7f15179db740 1 -- 172.21.15.43:0/3230771639 shutdown_connections 2024-04-25T10:35:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.810+0000 7f15179db740 1 -- 172.21.15.43:0/3230771639 wait complete. 2024-04-25T10:35:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.810+0000 7f15179db740 1 -- start start 2024-04-25T10:35:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.810+0000 7f15179db740 1 Processor -- start 2024-04-25T10:35:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.810+0000 7f15179db740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e299d80 0x55930e68d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f15179db740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e2b22f0 0x55930e68de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f15179db740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2ba4d0 0x55930e6fea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f15179db740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1b0180 con 0x55930e2ba4d0 2024-04-25T10:35:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f15179db740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1b15f0 con 0x55930e299d80 2024-04-25T10:35:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f15179db740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1b0850 con 0x55930e2b22f0 2024-04-25T10:35:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f15179db740 10 client.0 _ll_register_callbacks cb 0x7f15040093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f1509d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2ba4d0 0x55930e6fea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f1509d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e299d80 0x55930e68d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f1509d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e299d80 0x55930e68d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:36794/0 (socket says 172.21.15.43:36794) 2024-04-25T10:35:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f1509d9b640 1 -- 172.21.15.43:0/2438436390 learned_addr learned my addr 172.21.15.43:0/2438436390 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f150a59c640 1 --2- 172.21.15.43:0/2438436390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e2b22f0 0x55930e68de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f1509d9b640 1 -- 172.21.15.43:0/2438436390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e2b22f0 msgr2=0x55930e68de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f1509d9b640 1 --2- 172.21.15.43:0/2438436390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e2b22f0 0x55930e68de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f1509d9b640 1 -- 172.21.15.43:0/2438436390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e299d80 msgr2=0x55930e68d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f1509d9b640 1 --2- 172.21.15.43:0/2438436390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e299d80 0x55930e68d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f1509d9b640 1 -- 172.21.15.43:0/2438436390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55930e6fefb0 con 0x55930e2ba4d0 2024-04-25T10:35:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f1509d9b640 1 --2- 172.21.15.43:0/2438436390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2ba4d0 0x55930e6fea70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x55930e68cf10 tx=0x7f14fc020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286ca604485cb5eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f14f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f14f1ffb640 1 -- 172.21.15.43:0/2438436390 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc019040 con 0x55930e2ba4d0 2024-04-25T10:35:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f14f1ffb640 1 -- 172.21.15.43:0/2438436390 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f14fc02c350 con 0x55930e2ba4d0 2024-04-25T10:35:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.811+0000 7f15179db740 1 -- 172.21.15.43:0/2438436390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14f4003a60 con 0x55930e2ba4d0 2024-04-25T10:35:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.812+0000 7f14f1ffb640 1 -- 172.21.15.43:0/2438436390 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f14fc03d050 con 0x55930e2ba4d0 2024-04-25T10:35:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.812+0000 7f15179db740 10 client.16353 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.812+0000 7f15179db740 1 -- 172.21.15.43:0/2438436390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55930e7000e0 con 0x55930e2ba4d0 2024-04-25T10:35:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.812+0000 7f14f1ffb640 1 -- 172.21.15.43:0/2438436390 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14f4003a60 con 0x55930e2ba4d0 2024-04-25T10:35:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.812+0000 7f14f1ffb640 1 -- 172.21.15.43:0/2438436390 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f14fc02c030 con 0x55930e2ba4d0 2024-04-25T10:35:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.812+0000 7f15179db740 10 client.16353 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.812+0000 7f15179db740 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.812+0000 7f15179db740 20 client.16353 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.812+0000 7f15179db740 1 --2- 172.21.15.43:0/2438436390 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55930e2db1b0 0x55930e702730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.812+0000 7f15179db740 4 client.16353 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:35:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.812+0000 7f15179db740 1 -- 172.21.15.43:0/2438436390 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55930e5e8ee0 con 0x55930e2db1b0 2024-04-25T10:35:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.812+0000 7f150a59c640 1 --2- 172.21.15.43:0/2438436390 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55930e2db1b0 0x55930e702730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.813+0000 7f14f1ffb640 10 client.16353 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.813+0000 7f150a59c640 1 --2- 172.21.15.43:0/2438436390 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55930e2db1b0 0x55930e702730 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48c35334fc0b7d8e server_cookie=d2cffa755d4bef32 in_seq=0 out_seq=0 2024-04-25T10:35:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.827+0000 7f14f1ffb640 1 -- 172.21.15.43:0/2438436390 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55930e5e8ee0 con 0x55930e2db1b0 2024-04-25T10:35:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.827+0000 7f14f1ffb640 10 client.16353 handle_command_reply: tid=0 2024-04-25T10:35:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.827+0000 7f15179db740 1 client.16353 shutdown 2024-04-25T10:35:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.827+0000 7f15179db740 1 -- 172.21.15.43:0/2438436390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2ba4d0 msgr2=0x55930e6fea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.827+0000 7f15179db740 1 --2- 172.21.15.43:0/2438436390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2ba4d0 0x55930e6fea70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x55930e68cf10 tx=0x7f14fc020040 comp rx=0 tx=0).stop 2024-04-25T10:35:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.827+0000 7f15179db740 1 -- 172.21.15.43:0/2438436390 shutdown_connections 2024-04-25T10:35:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.827+0000 7f15179db740 1 --2- 172.21.15.43:0/2438436390 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55930e2db1b0 0x55930e702730 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.827+0000 7f15179db740 1 --2- 172.21.15.43:0/2438436390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55930e2ba4d0 0x55930e6fea70 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.827+0000 7f15179db740 1 --2- 172.21.15.43:0/2438436390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55930e2b22f0 0x55930e68de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.827+0000 7f14f1ffb640 0 client.16353 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.827+0000 7f15179db740 1 --2- 172.21.15.43:0/2438436390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55930e299d80 0x55930e68d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.827+0000 7f15179db740 1 -- 172.21.15.43:0/2438436390 >> 172.21.15.43:0/2438436390 conn(0x55930e2cf300 msgr2=0x55930e67a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.827+0000 7f15179db740 1 -- 172.21.15.43:0/2438436390 shutdown_connections 2024-04-25T10:35:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.827+0000 7f15179db740 1 -- 172.21.15.43:0/2438436390 wait complete. 2024-04-25T10:35:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.827+0000 7f15179db740 20 client.16353 trim_cache size 0 max 16384 2024-04-25T10:35:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:35:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:35:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:35:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:35:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.828+0000 7f1510b2e640 1 -- 172.21.15.43:0/1079748389 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f14f80afc50 msgr2=0x7f14f80b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.828+0000 7f1510b2e640 1 --2- 172.21.15.43:0/1079748389 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f14f80afc50 0x7f14f80b2160 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1504063250 tx=0x7f150400cf70 comp rx=0 tx=0).stop 2024-04-25T10:35:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.828+0000 7f1510b2e640 1 -- 172.21.15.43:0/1079748389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f150c108ef0 msgr2=0x7f150c110cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.828+0000 7f1510b2e640 1 --2- 172.21.15.43:0/1079748389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f150c108ef0 0x7f150c110cd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f150400eb10 tx=0x7f150400c2d0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.828+0000 7f1510b2e640 1 -- 172.21.15.43:0/1079748389 shutdown_connections 2024-04-25T10:35:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.828+0000 7f1510b2e640 1 --2- 172.21.15.43:0/1079748389 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f14f80afc50 0x7f14f80b2160 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.828+0000 7f1510b2e640 1 --2- 172.21.15.43:0/1079748389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150c108a50 0x7f150c110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.828+0000 7f1510b2e640 1 --2- 172.21.15.43:0/1079748389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f150c108ef0 0x7f150c110cd0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.828+0000 7f1510b2e640 1 --2- 172.21.15.43:0/1079748389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f150c1031b0 0x7f150c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.828+0000 7f1510b2e640 1 -- 172.21.15.43:0/1079748389 >> 172.21.15.43:0/1079748389 conn(0x7f150c0fe260 msgr2=0x7f150c0fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.828+0000 7f1510b2e640 1 -- 172.21.15.43:0/1079748389 shutdown_connections 2024-04-25T10:35:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:34.828+0000 7f1510b2e640 1 -- 172.21.15.43:0/1079748389 wait complete. 2024-04-25T10:35:36.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:35 smithi077 ceph-mon[32772]: pgmap v95: 97 pgs: 97 active+clean; 3.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-04-25T10:35:36.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:35 smithi164 ceph-mon[32713]: pgmap v95: 97 pgs: 97 active+clean; 3.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-04-25T10:35:36.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:35 smithi043 ceph-mon[29478]: pgmap v95: 97 pgs: 97 active+clean; 3.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-04-25T10:35:38.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:37 smithi077 ceph-mon[32772]: pgmap v96: 97 pgs: 97 active+clean; 3.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-04-25T10:35:38.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:37 smithi164 ceph-mon[32713]: pgmap v96: 97 pgs: 97 active+clean; 3.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-04-25T10:35:38.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:37 smithi043 ceph-mon[29478]: pgmap v96: 97 pgs: 97 active+clean; 3.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-04-25T10:35:40.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:39 smithi077 ceph-mon[32772]: pgmap v97: 97 pgs: 97 active+clean; 3.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-04-25T10:35:40.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:39 smithi164 ceph-mon[32713]: pgmap v97: 97 pgs: 97 active+clean; 3.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-04-25T10:35:40.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:39 smithi043 ceph-mon[29478]: pgmap v97: 97 pgs: 97 active+clean; 3.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-04-25T10:35:42.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:41 smithi077 ceph-mon[32772]: pgmap v98: 97 pgs: 97 active+clean; 3.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T10:35:42.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:41 smithi164 ceph-mon[32713]: pgmap v98: 97 pgs: 97 active+clean; 3.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T10:35:42.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:41 smithi043 ceph-mon[29478]: pgmap v98: 97 pgs: 97 active+clean; 3.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T10:35:44.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:43 smithi077 ceph-mon[32772]: pgmap v99: 97 pgs: 97 active+clean; 3.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T10:35:44.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:43 smithi164 ceph-mon[32713]: pgmap v99: 97 pgs: 97 active+clean; 3.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T10:35:44.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:43 smithi043 ceph-mon[29478]: pgmap v99: 97 pgs: 97 active+clean; 3.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T10:35:45.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:44 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:35:45.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:44 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:35:45.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:44 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:35:45.942 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:45 smithi077 ceph-mon[32772]: pgmap v100: 97 pgs: 97 active+clean; 3.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-04-25T10:35:45.942 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:45 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:35:45.942 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:45 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:35:45.942 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:45 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:35:45.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:45 smithi164 ceph-mon[32713]: pgmap v100: 97 pgs: 97 active+clean; 3.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-04-25T10:35:45.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:45 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:35:45.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:45 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:35:45.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:45 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:35:46.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:45 smithi043 ceph-mon[29478]: pgmap v100: 97 pgs: 97 active+clean; 3.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-04-25T10:35:46.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:45 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:35:46.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:45 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:35:46.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:45 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:35:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.941+0000 7f06ae1ce640 1 -- 172.21.15.43:0/1768596826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a8109b70 msgr2=0x7f06a8103710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.941+0000 7f06ae1ce640 1 --2- 172.21.15.43:0/1768596826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a8109b70 0x7f06a8103710 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0698009900 tx=0x7f069802a410 comp rx=0 tx=0).stop 2024-04-25T10:35:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.941+0000 7f06ae1ce640 1 -- 172.21.15.43:0/1768596826 shutdown_connections 2024-04-25T10:35:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.941+0000 7f06ae1ce640 1 --2- 172.21.15.43:0/1768596826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06a810a010 0x7f06a8103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.941+0000 7f06ae1ce640 1 --2- 172.21.15.43:0/1768596826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06a81042c0 0x7f06a8110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.941+0000 7f06ae1ce640 1 --2- 172.21.15.43:0/1768596826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a8109b70 0x7f06a8103710 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.941+0000 7f06ae1ce640 1 -- 172.21.15.43:0/1768596826 >> 172.21.15.43:0/1768596826 conn(0x7f06a80ff250 msgr2=0x7f06a81016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.941+0000 7f06ae1ce640 1 -- 172.21.15.43:0/1768596826 shutdown_connections 2024-04-25T10:35:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.941+0000 7f06ae1ce640 1 -- 172.21.15.43:0/1768596826 wait complete. 2024-04-25T10:35:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.941+0000 7f06ae1ce640 1 Processor -- start 2024-04-25T10:35:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.942+0000 7f06ae1ce640 1 -- start start 2024-04-25T10:35:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.942+0000 7f06ae1ce640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06a81042c0 0x7f06a819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.942+0000 7f06ae1ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a8109b70 0x7f06a819ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.942+0000 7f06ae1ce640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06a810a010 0x7f06a819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.942+0000 7f06ae1ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06a8113d60 con 0x7f06a8109b70 2024-04-25T10:35:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.942+0000 7f06ae1ce640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f06a8113be0 con 0x7f06a810a010 2024-04-25T10:35:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.942+0000 7f06ae1ce640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f06a8113ee0 con 0x7f06a81042c0 2024-04-25T10:35:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.942+0000 7f06a6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a8109b70 0x7f06a819ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.942+0000 7f06a6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a8109b70 0x7f06a819ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43294/0 (socket says 172.21.15.43:43294) 2024-04-25T10:35:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.942+0000 7f06a6ffd640 1 -- 172.21.15.43:0/2167264520 learned_addr learned my addr 172.21.15.43:0/2167264520 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.942+0000 7f06a77fe640 1 --2- 172.21.15.43:0/2167264520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06a81042c0 0x7f06a819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.942+0000 7f06a7fff640 1 --2- 172.21.15.43:0/2167264520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06a810a010 0x7f06a819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.942+0000 7f06a6ffd640 1 -- 172.21.15.43:0/2167264520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06a81042c0 msgr2=0x7f06a819a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.942+0000 7f06a6ffd640 1 --2- 172.21.15.43:0/2167264520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06a81042c0 0x7f06a819a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.943+0000 7f06a6ffd640 1 -- 172.21.15.43:0/2167264520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06a810a010 msgr2=0x7f06a819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.943+0000 7f06a6ffd640 1 --2- 172.21.15.43:0/2167264520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06a810a010 0x7f06a819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.943+0000 7f06a6ffd640 1 -- 172.21.15.43:0/2167264520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a819f620 con 0x7f06a8109b70 2024-04-25T10:35:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.943+0000 7f06a7fff640 1 --2- 172.21.15.43:0/2167264520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06a810a010 0x7f06a819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:35:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.943+0000 7f06a6ffd640 1 --2- 172.21.15.43:0/2167264520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a8109b70 0x7f06a819ad20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f069c00e9f0 tx=0x7f069c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.943+0000 7f06a4ff9640 1 -- 172.21.15.43:0/2167264520 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c00ccc0 con 0x7f06a8109b70 2024-04-25T10:35:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.943+0000 7f06a4ff9640 1 -- 172.21.15.43:0/2167264520 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f069c0035f0 con 0x7f06a8109b70 2024-04-25T10:35:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.943+0000 7f06ae1ce640 1 -- 172.21.15.43:0/2167264520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a819f910 con 0x7f06a8109b70 2024-04-25T10:35:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.943+0000 7f06a4ff9640 1 -- 172.21.15.43:0/2167264520 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c00ce60 con 0x7f06a8109b70 2024-04-25T10:35:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.943+0000 7f06ae1ce640 1 -- 172.21.15.43:0/2167264520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a81a7290 con 0x7f06a8109b70 2024-04-25T10:35:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.946+0000 7f06a4ff9640 1 -- 172.21.15.43:0/2167264520 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f069c003aa0 con 0x7f06a8109b70 2024-04-25T10:35:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.946+0000 7f06b5084740 1 Processor -- start 2024-04-25T10:35:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.946+0000 7f06b5084740 1 -- start start 2024-04-25T10:35:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.946+0000 7f06b5084740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b93a0d1b0 0x562b93dacaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.946+0000 7f06b5084740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b939e42f0 0x562b93dad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.946+0000 7f06b5084740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 0x562b93dad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.946+0000 7f06b5084740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562b93d13460 con 0x562b939ec4d0 2024-04-25T10:35:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.946+0000 7f06b5084740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562b93ccd590 con 0x562b939e42f0 2024-04-25T10:35:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.946+0000 7f06b5084740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562b93d0f420 con 0x562b93a0d1b0 2024-04-25T10:35:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.946+0000 7f06a6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 0x562b93dad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.946+0000 7f06a6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 0x562b93dad570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43300/0 (socket says 172.21.15.43:43300) 2024-04-25T10:35:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.946+0000 7f06a6ffd640 1 -- 172.21.15.43:0/4157343534 learned_addr learned my addr 172.21.15.43:0/4157343534 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.946+0000 7f06a4ff9640 1 --2- 172.21.15.43:0/2167264520 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f06900afc50 0x7f06900b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.947+0000 7f06a77fe640 1 --2- 172.21.15.43:0/4157343534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b939e42f0 0x562b93dad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.947+0000 7f06a7fff640 1 --2- 172.21.15.43:0/4157343534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b93a0d1b0 0x562b93dacaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.947+0000 7f06a7fff640 1 --2- 172.21.15.43:0/2167264520 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f06900afc50 0x7f06900b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.947+0000 7f06a4ff9640 1 -- 172.21.15.43:0/2167264520 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f069c0df3f0 con 0x7f06a8109b70 2024-04-25T10:35:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.947+0000 7f06a6ffd640 1 -- 172.21.15.43:0/4157343534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b93a0d1b0 msgr2=0x562b93dacaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.947+0000 7f06a6ffd640 1 --2- 172.21.15.43:0/4157343534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b93a0d1b0 0x562b93dacaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.947+0000 7f06a6ffd640 1 -- 172.21.15.43:0/4157343534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b939e42f0 msgr2=0x562b93dad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.947+0000 7f06a6ffd640 1 --2- 172.21.15.43:0/4157343534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b939e42f0 0x562b93dad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.947+0000 7f06a6ffd640 1 -- 172.21.15.43:0/4157343534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b93cda0d0 con 0x562b939ec4d0 2024-04-25T10:35:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.947+0000 7f06a77fe640 1 --2- 172.21.15.43:0/4157343534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b939e42f0 0x562b93dad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:35:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.947+0000 7f06a6ffd640 1 --2- 172.21.15.43:0/4157343534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 0x562b93dad570 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f069c0a3640 tx=0x7f069c003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27fc53a1adbf4fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.947+0000 7f06a7fff640 1 --2- 172.21.15.43:0/2167264520 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f06900afc50 0x7f06900b2160 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f06a0005c40 tx=0x7f06a0009500 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.947+0000 7f068dffb640 1 -- 172.21.15.43:0/4157343534 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c0a6d30 con 0x562b939ec4d0 2024-04-25T10:35:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.947+0000 7f068dffb640 1 -- 172.21.15.43:0/4157343534 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f069c0a70d0 con 0x562b939ec4d0 2024-04-25T10:35:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.947+0000 7f068dffb640 1 -- 172.21.15.43:0/4157343534 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c0037a0 con 0x562b939ec4d0 2024-04-25T10:35:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.948+0000 7f06b5084740 1 -- 172.21.15.43:0/4157343534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 msgr2=0x562b93dad570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.948+0000 7f06b5084740 1 --2- 172.21.15.43:0/4157343534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 0x562b93dad570 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f069c0a3640 tx=0x7f069c003a10 comp rx=0 tx=0).stop 2024-04-25T10:35:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.948+0000 7f06b5084740 1 -- 172.21.15.43:0/4157343534 shutdown_connections 2024-04-25T10:35:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.948+0000 7f06b5084740 1 --2- 172.21.15.43:0/4157343534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b939e42f0 0x562b93dad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.948+0000 7f06b5084740 1 --2- 172.21.15.43:0/4157343534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 0x562b93dad570 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.948+0000 7f06b5084740 1 --2- 172.21.15.43:0/4157343534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b93a0d1b0 0x562b93dacaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.948+0000 7f06b5084740 1 -- 172.21.15.43:0/4157343534 >> 172.21.15.43:0/4157343534 conn(0x562b93a01300 msgr2=0x562b93da92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.948+0000 7f06b5084740 1 -- 172.21.15.43:0/4157343534 shutdown_connections 2024-04-25T10:35:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.948+0000 7f06b5084740 1 -- 172.21.15.43:0/4157343534 wait complete. 2024-04-25T10:35:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.949+0000 7f06b5084740 1 -- start start 2024-04-25T10:35:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.949+0000 7f06b5084740 1 Processor -- start 2024-04-25T10:35:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.949+0000 7f06b5084740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939e42f0 0x562b93db8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.949+0000 7f06b5084740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b939ec4d0 0x562b93db9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.949+0000 7f06b5084740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b93a0d1b0 0x562b93db7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.949+0000 7f06b5084740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562b93db9b60 con 0x562b939e42f0 2024-04-25T10:35:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.949+0000 7f06b5084740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562b93c9ab60 con 0x562b939ec4d0 2024-04-25T10:35:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.949+0000 7f06b5084740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562b93db9ce0 con 0x562b93a0d1b0 2024-04-25T10:35:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.949+0000 7f06b5084740 10 client.0 _ll_register_callbacks cb 0x562b93cbf2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.949+0000 7f06a77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939e42f0 0x562b93db8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.949+0000 7f06a77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939e42f0 0x562b93db8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43310/0 (socket says 172.21.15.43:43310) 2024-04-25T10:35:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.949+0000 7f06a77fe640 1 -- 172.21.15.43:0/369044057 learned_addr learned my addr 172.21.15.43:0/369044057 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.950+0000 7f06a7fff640 1 --2- 172.21.15.43:0/369044057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b93a0d1b0 0x562b93db7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.950+0000 7f06a6ffd640 1 --2- 172.21.15.43:0/369044057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b939ec4d0 0x562b93db9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.950+0000 7f06a77fe640 1 -- 172.21.15.43:0/369044057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b93a0d1b0 msgr2=0x562b93db7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.950+0000 7f06a77fe640 1 --2- 172.21.15.43:0/369044057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b93a0d1b0 0x562b93db7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.950+0000 7f06a77fe640 1 -- 172.21.15.43:0/369044057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b939ec4d0 msgr2=0x562b93db9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.950+0000 7f06a77fe640 1 --2- 172.21.15.43:0/369044057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b939ec4d0 0x562b93db9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.950+0000 7f06a77fe640 1 -- 172.21.15.43:0/369044057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b93a25210 con 0x562b939e42f0 2024-04-25T10:35:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.950+0000 7f06a7fff640 1 --2- 172.21.15.43:0/369044057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b93a0d1b0 0x562b93db7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.950+0000 7f06a77fe640 1 --2- 172.21.15.43:0/369044057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939e42f0 0x562b93db8b50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f06a8104850 tx=0x7f06980316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2077b259a2ddfef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.950+0000 7f0686ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.950+0000 7f0686ffd640 1 -- 172.21.15.43:0/369044057 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698035040 con 0x562b939e42f0 2024-04-25T10:35:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.950+0000 7f0686ffd640 1 -- 172.21.15.43:0/369044057 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f0698004040 con 0x562b939e42f0 2024-04-25T10:35:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.950+0000 7f06b5084740 1 -- 172.21.15.43:0/369044057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f069c0a70d0 con 0x562b939e42f0 2024-04-25T10:35:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.950+0000 7f0686ffd640 1 -- 172.21.15.43:0/369044057 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0698004240 con 0x562b939e42f0 2024-04-25T10:35:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.950+0000 7f06b5084740 10 client.16365 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.950+0000 7f06b5084740 1 -- 172.21.15.43:0/369044057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562b93af1620 con 0x562b939e42f0 2024-04-25T10:35:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.951+0000 7f0686ffd640 1 -- 172.21.15.43:0/369044057 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f069c0a70d0 con 0x562b939e42f0 2024-04-25T10:35:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.951+0000 7f0686ffd640 1 -- 172.21.15.43:0/369044057 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0698004960 con 0x562b939e42f0 2024-04-25T10:35:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.951+0000 7f06b5084740 10 client.16365 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.951+0000 7f06b5084740 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.951+0000 7f06b5084740 20 client.16365 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.951+0000 7f06b5084740 1 --2- 172.21.15.43:0/369044057 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562b939cbd80 0x562b93dc96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.951+0000 7f06b5084740 4 client.16365 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:35:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.951+0000 7f06b5084740 1 -- 172.21.15.43:0/369044057 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b93d21070 con 0x562b939cbd80 2024-04-25T10:35:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.951+0000 7f06a6ffd640 1 --2- 172.21.15.43:0/369044057 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562b939cbd80 0x562b93dc96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.952+0000 7f0686ffd640 10 client.16365 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.952+0000 7f06a6ffd640 1 --2- 172.21.15.43:0/369044057 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562b939cbd80 0x562b93dc96b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6586668b1f547590 server_cookie=656da29cebe908ab in_seq=0 out_seq=0 2024-04-25T10:35:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.952+0000 7f0686ffd640 1 -- 172.21.15.43:0/369044057 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562b93d21070 con 0x562b939cbd80 2024-04-25T10:35:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.952+0000 7f0686ffd640 10 client.16365 handle_command_reply: tid=0 2024-04-25T10:35:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.953+0000 7f06b5084740 1 client.16365 shutdown 2024-04-25T10:35:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.953+0000 7f06b5084740 1 -- 172.21.15.43:0/369044057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939e42f0 msgr2=0x562b93db8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.953+0000 7f06b5084740 1 --2- 172.21.15.43:0/369044057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939e42f0 0x562b93db8b50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f06a8104850 tx=0x7f06980316d0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.953+0000 7f06b5084740 1 -- 172.21.15.43:0/369044057 shutdown_connections 2024-04-25T10:35:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.953+0000 7f06b5084740 1 --2- 172.21.15.43:0/369044057 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562b939cbd80 0x562b93dc96b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.953+0000 7f06b5084740 1 --2- 172.21.15.43:0/369044057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b939ec4d0 0x562b93db9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.953+0000 7f06b5084740 1 --2- 172.21.15.43:0/369044057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b93a0d1b0 0x562b93db7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.953+0000 7f06b5084740 1 --2- 172.21.15.43:0/369044057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939e42f0 0x562b93db8b50 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.953+0000 7f0686ffd640 0 client.16365 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.953+0000 7f06b5084740 1 -- 172.21.15.43:0/369044057 >> 172.21.15.43:0/369044057 conn(0x562b93a01300 msgr2=0x562b93dae810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.953+0000 7f06b5084740 1 -- 172.21.15.43:0/369044057 shutdown_connections 2024-04-25T10:35:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.953+0000 7f06b5084740 1 -- 172.21.15.43:0/369044057 wait complete. 2024-04-25T10:35:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.953+0000 7f06b5084740 20 client.16365 trim_cache size 0 max 16384 2024-04-25T10:35:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.969+0000 7f06b5084740 1 Processor -- start 2024-04-25T10:35:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.969+0000 7f06b5084740 1 -- start start 2024-04-25T10:35:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.969+0000 7f06b5084740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b939e42f0 0x562b93daedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.969+0000 7f06b5084740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 0x562b93db07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.969+0000 7f06b5084740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b93a0d1b0 0x562b93db0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.969+0000 7f06b5084740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81b23a0 con 0x562b939ec4d0 2024-04-25T10:35:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.969+0000 7f06b5084740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81afc70 con 0x562b93a0d1b0 2024-04-25T10:35:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.969+0000 7f06b5084740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81adac0 con 0x562b939e42f0 2024-04-25T10:35:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.969+0000 7f06a6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 0x562b93db07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.969+0000 7f06a6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 0x562b93db07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43322/0 (socket says 172.21.15.43:43322) 2024-04-25T10:35:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.969+0000 7f06a6ffd640 1 -- 172.21.15.43:0/1429654248 learned_addr learned my addr 172.21.15.43:0/1429654248 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.969+0000 7f06a7fff640 1 --2- 172.21.15.43:0/1429654248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b93a0d1b0 0x562b93db0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.969+0000 7f06a77fe640 1 --2- 172.21.15.43:0/1429654248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b939e42f0 0x562b93daedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.969+0000 7f06a6ffd640 1 -- 172.21.15.43:0/1429654248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b939e42f0 msgr2=0x562b93daedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.969+0000 7f06a6ffd640 1 --2- 172.21.15.43:0/1429654248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b939e42f0 0x562b93daedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.969+0000 7f06a6ffd640 1 -- 172.21.15.43:0/1429654248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b93a0d1b0 msgr2=0x562b93db0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.969+0000 7f06a6ffd640 1 --2- 172.21.15.43:0/1429654248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b93a0d1b0 0x562b93db0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.969+0000 7f06a6ffd640 1 -- 172.21.15.43:0/1429654248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b93db9800 con 0x562b939ec4d0 2024-04-25T10:35:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.969+0000 7f06a6ffd640 1 --2- 172.21.15.43:0/1429654248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 0x562b93db07a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f069c000c00 tx=0x7f069c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be073cbaa24a448 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.970+0000 7f068d7fa640 1 -- 172.21.15.43:0/1429654248 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c0a70d0 con 0x562b939ec4d0 2024-04-25T10:35:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.970+0000 7f068d7fa640 1 -- 172.21.15.43:0/1429654248 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f069c0a7cc0 con 0x562b939ec4d0 2024-04-25T10:35:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.970+0000 7f068d7fa640 1 -- 172.21.15.43:0/1429654248 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c0aff90 con 0x562b939ec4d0 2024-04-25T10:35:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.970+0000 7f06b5084740 1 -- 172.21.15.43:0/1429654248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 msgr2=0x562b93db07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.970+0000 7f06b5084740 1 --2- 172.21.15.43:0/1429654248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 0x562b93db07a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f069c000c00 tx=0x7f069c0045e0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.970+0000 7f06b5084740 1 -- 172.21.15.43:0/1429654248 shutdown_connections 2024-04-25T10:35:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.970+0000 7f06b5084740 1 --2- 172.21.15.43:0/1429654248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b93a0d1b0 0x562b93db0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.970+0000 7f06b5084740 1 --2- 172.21.15.43:0/1429654248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 0x562b93db07a0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.970+0000 7f06b5084740 1 --2- 172.21.15.43:0/1429654248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b939e42f0 0x562b93daedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.970+0000 7f06b5084740 1 -- 172.21.15.43:0/1429654248 >> 172.21.15.43:0/1429654248 conn(0x562b93a01300 msgr2=0x562b93dae6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.971+0000 7f06b5084740 1 -- 172.21.15.43:0/1429654248 shutdown_connections 2024-04-25T10:35:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.971+0000 7f06b5084740 1 -- 172.21.15.43:0/1429654248 wait complete. 2024-04-25T10:35:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.971+0000 7f06b5084740 1 -- start start 2024-04-25T10:35:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.971+0000 7f06b5084740 1 Processor -- start 2024-04-25T10:35:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.971+0000 7f06b5084740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b939e42f0 0x562b93e29e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.971+0000 7f06b5084740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 0x562b93e2a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.971+0000 7f06b5084740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b93a0d1b0 0x562b93dbc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f06b5084740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81ae130 con 0x562b939ec4d0 2024-04-25T10:35:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f06b5084740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81af5a0 con 0x562b939e42f0 2024-04-25T10:35:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f06b5084740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81ae800 con 0x562b93a0d1b0 2024-04-25T10:35:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f06b5084740 10 client.0 _ll_register_callbacks cb 0x7f06a00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f06a6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 0x562b93e2a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f06a6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 0x562b93e2a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43330/0 (socket says 172.21.15.43:43330) 2024-04-25T10:35:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f06a6ffd640 1 -- 172.21.15.43:0/4100644363 learned_addr learned my addr 172.21.15.43:0/4100644363 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f06a77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b939e42f0 0x562b93e29e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f06a7fff640 1 --2- 172.21.15.43:0/4100644363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b93a0d1b0 0x562b93dbc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f06a6ffd640 1 -- 172.21.15.43:0/4100644363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b93a0d1b0 msgr2=0x562b93dbc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f06a6ffd640 1 --2- 172.21.15.43:0/4100644363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b93a0d1b0 0x562b93dbc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f06a6ffd640 1 -- 172.21.15.43:0/4100644363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b939e42f0 msgr2=0x562b93e29e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f06a6ffd640 1 --2- 172.21.15.43:0/4100644363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b939e42f0 0x562b93e29e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f06a6ffd640 1 -- 172.21.15.43:0/4100644363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b93dbca20 con 0x562b939ec4d0 2024-04-25T10:35:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f06a7fff640 1 --2- 172.21.15.43:0/4100644363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b93a0d1b0 0x562b93dbc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f06a6ffd640 1 --2- 172.21.15.43:0/4100644363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 0x562b93e2a3b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f069c0a7550 tx=0x7f069c0038c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e576845ffa7853 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f0686ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f0686ffd640 1 -- 172.21.15.43:0/4100644363 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c0900c0 con 0x562b939ec4d0 2024-04-25T10:35:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f0686ffd640 1 -- 172.21.15.43:0/4100644363 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f069c0aff90 con 0x562b939ec4d0 2024-04-25T10:35:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f06b5084740 1 -- 172.21.15.43:0/4100644363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f069c0a7cc0 con 0x562b939ec4d0 2024-04-25T10:35:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.972+0000 7f0686ffd640 1 -- 172.21.15.43:0/4100644363 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f069c0a75c0 con 0x562b939ec4d0 2024-04-25T10:35:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.973+0000 7f06b5084740 10 client.16371 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.973+0000 7f06b5084740 1 -- 172.21.15.43:0/4100644363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562b93e31520 con 0x562b939ec4d0 2024-04-25T10:35:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.973+0000 7f0686ffd640 1 -- 172.21.15.43:0/4100644363 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f069c0a7cc0 con 0x562b939ec4d0 2024-04-25T10:35:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.973+0000 7f0686ffd640 1 -- 172.21.15.43:0/4100644363 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f069c0a2f30 con 0x562b939ec4d0 2024-04-25T10:35:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.973+0000 7f06b5084740 10 client.16371 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.973+0000 7f06b5084740 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.973+0000 7f06b5084740 20 client.16371 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.973+0000 7f06b5084740 1 --2- 172.21.15.43:0/4100644363 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562b939a11c0 0x562b93e337a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.973+0000 7f06b5084740 4 client.16371 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:35:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.973+0000 7f06b5084740 1 -- 172.21.15.43:0/4100644363 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b93d1aee0 con 0x562b939a11c0 2024-04-25T10:35:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.973+0000 7f06a77fe640 1 --2- 172.21.15.43:0/4100644363 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562b939a11c0 0x562b93e337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.974+0000 7f0686ffd640 10 client.16371 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.974+0000 7f06a77fe640 1 --2- 172.21.15.43:0/4100644363 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562b939a11c0 0x562b93e337a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=269f53d1b1f319f9 server_cookie=3fda48dfa864758c in_seq=0 out_seq=0 2024-04-25T10:35:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f0686ffd640 1 -- 172.21.15.43:0/4100644363 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562b93d1aee0 con 0x562b939a11c0 2024-04-25T10:35:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f0686ffd640 10 client.16371 handle_command_reply: tid=0 2024-04-25T10:35:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06b5084740 1 client.16371 shutdown 2024-04-25T10:35:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06b5084740 1 -- 172.21.15.43:0/4100644363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 msgr2=0x562b93e2a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06b5084740 1 --2- 172.21.15.43:0/4100644363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 0x562b93e2a3b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f069c0a7550 tx=0x7f069c0038c0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06b5084740 1 -- 172.21.15.43:0/4100644363 shutdown_connections 2024-04-25T10:35:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06b5084740 1 --2- 172.21.15.43:0/4100644363 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562b939a11c0 0x562b93e337a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06b5084740 1 --2- 172.21.15.43:0/4100644363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562b93a0d1b0 0x562b93dbc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06b5084740 1 --2- 172.21.15.43:0/4100644363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b939ec4d0 0x562b93e2a3b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06b5084740 1 --2- 172.21.15.43:0/4100644363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b939e42f0 0x562b93e29e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f0686ffd640 0 client.16371 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06b5084740 1 -- 172.21.15.43:0/4100644363 >> 172.21.15.43:0/4100644363 conn(0x562b93a01300 msgr2=0x562b93db85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06b5084740 1 -- 172.21.15.43:0/4100644363 shutdown_connections 2024-04-25T10:35:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06b5084740 1 -- 172.21.15.43:0/4100644363 wait complete. 2024-04-25T10:35:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06b5084740 20 client.16371 trim_cache size 0 max 16384 2024-04-25T10:35:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:35:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:35:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:35:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:35:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06ae1ce640 1 -- 172.21.15.43:0/2167264520 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f06900afc50 msgr2=0x7f06900b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06ae1ce640 1 --2- 172.21.15.43:0/2167264520 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f06900afc50 0x7f06900b2160 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f06a0005c40 tx=0x7f06a0009500 comp rx=0 tx=0).stop 2024-04-25T10:35:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06ae1ce640 1 -- 172.21.15.43:0/2167264520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a8109b70 msgr2=0x7f06a819ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06ae1ce640 1 --2- 172.21.15.43:0/2167264520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a8109b70 0x7f06a819ad20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f069c00e9f0 tx=0x7f069c00eeb0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06ae1ce640 1 -- 172.21.15.43:0/2167264520 shutdown_connections 2024-04-25T10:35:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06ae1ce640 1 --2- 172.21.15.43:0/2167264520 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f06900afc50 0x7f06900b2160 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06ae1ce640 1 --2- 172.21.15.43:0/2167264520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06a810a010 0x7f06a819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06ae1ce640 1 --2- 172.21.15.43:0/2167264520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a8109b70 0x7f06a819ad20 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06ae1ce640 1 --2- 172.21.15.43:0/2167264520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06a81042c0 0x7f06a819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06ae1ce640 1 -- 172.21.15.43:0/2167264520 >> 172.21.15.43:0/2167264520 conn(0x7f06a80ff250 msgr2=0x7f06a80ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06ae1ce640 1 -- 172.21.15.43:0/2167264520 shutdown_connections 2024-04-25T10:35:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:35:47.991+0000 7f06ae1ce640 1 -- 172.21.15.43:0/2167264520 wait complete. 2024-04-25T10:35:48.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:47 smithi077 ceph-mon[32772]: pgmap v101: 97 pgs: 97 active+clean; 3.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-04-25T10:35:48.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:47 smithi164 ceph-mon[32713]: pgmap v101: 97 pgs: 97 active+clean; 3.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-04-25T10:35:48.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:47 smithi043 ceph-mon[29478]: pgmap v101: 97 pgs: 97 active+clean; 3.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-04-25T10:35:50.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:49 smithi077 ceph-mon[32772]: pgmap v102: 97 pgs: 97 active+clean; 3.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 19 op/s 2024-04-25T10:35:50.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:49 smithi164 ceph-mon[32713]: pgmap v102: 97 pgs: 97 active+clean; 3.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 19 op/s 2024-04-25T10:35:50.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:49 smithi043 ceph-mon[29478]: pgmap v102: 97 pgs: 97 active+clean; 3.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 19 op/s 2024-04-25T10:35:52.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:51 smithi077 ceph-mon[32772]: pgmap v103: 97 pgs: 97 active+clean; 3.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T10:35:52.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:51 smithi164 ceph-mon[32713]: pgmap v103: 97 pgs: 97 active+clean; 3.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T10:35:52.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:51 smithi043 ceph-mon[29478]: pgmap v103: 97 pgs: 97 active+clean; 3.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T10:35:52.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:35:52.305 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:35:52.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.431+0000 7fc2f43ce640 1 -- 172.21.15.43:0/129791371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2ec102af0 msgr2=0x7fc2ec102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:52.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.431+0000 7fc2f43ce640 1 --2- 172.21.15.43:0/129791371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2ec102af0 0x7fc2ec102f00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009900 tx=0x7fc2e002a410 comp rx=0 tx=0).stop 2024-04-25T10:35:52.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.431+0000 7fc2f43ce640 1 -- 172.21.15.43:0/129791371 shutdown_connections 2024-04-25T10:35:52.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.431+0000 7fc2f43ce640 1 --2- 172.21.15.43:0/129791371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2ec10d380 0x7fc2ec10f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.431+0000 7fc2f43ce640 1 --2- 172.21.15.43:0/129791371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2ec102af0 0x7fc2ec102f00 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.431+0000 7fc2f43ce640 1 --2- 172.21.15.43:0/129791371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2ec108a20 0x7fc2ec1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.431+0000 7fc2f43ce640 1 -- 172.21.15.43:0/129791371 >> 172.21.15.43:0/129791371 conn(0x7fc2ec078100 msgr2=0x7fc2ec078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:52.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.431+0000 7fc2f43ce640 1 -- 172.21.15.43:0/129791371 shutdown_connections 2024-04-25T10:35:52.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.431+0000 7fc2f43ce640 1 -- 172.21.15.43:0/129791371 wait complete. 2024-04-25T10:35:52.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.432+0000 7fc2f43ce640 1 Processor -- start 2024-04-25T10:35:52.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.432+0000 7fc2f43ce640 1 -- start start 2024-04-25T10:35:52.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.432+0000 7fc2f43ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2ec102af0 0x7fc2ec19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.432+0000 7fc2f43ce640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2ec108a20 0x7fc2ec19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.432+0000 7fc2f43ce640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2ec10d380 0x7fc2ec19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.432+0000 7fc2f43ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec113d00 con 0x7fc2ec102af0 2024-04-25T10:35:52.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.432+0000 7fc2f43ce640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec113b80 con 0x7fc2ec10d380 2024-04-25T10:35:52.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.432+0000 7fc2f43ce640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec113e80 con 0x7fc2ec108a20 2024-04-25T10:35:52.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.432+0000 7fc2f216a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2ec102af0 0x7fc2ec19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.432+0000 7fc2f216a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2ec102af0 0x7fc2ec19a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42862/0 (socket says 172.21.15.43:42862) 2024-04-25T10:35:52.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.432+0000 7fc2f216a640 1 -- 172.21.15.43:0/4084819266 learned_addr learned my addr 172.21.15.43:0/4084819266 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:52.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.433+0000 7fc2f296b640 1 --2- 172.21.15.43:0/4084819266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2ec10d380 0x7fc2ec19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.433+0000 7fc2f1969640 1 --2- 172.21.15.43:0/4084819266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2ec108a20 0x7fc2ec19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.433+0000 7fc2f216a640 1 -- 172.21.15.43:0/4084819266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2ec108a20 msgr2=0x7fc2ec19ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:52.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.433+0000 7fc2f216a640 1 --2- 172.21.15.43:0/4084819266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2ec108a20 0x7fc2ec19ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.433+0000 7fc2f216a640 1 -- 172.21.15.43:0/4084819266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2ec10d380 msgr2=0x7fc2ec19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:52.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.433+0000 7fc2f216a640 1 --2- 172.21.15.43:0/4084819266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2ec10d380 0x7fc2ec19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.433+0000 7fc2f216a640 1 -- 172.21.15.43:0/4084819266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ec19f560 con 0x7fc2ec102af0 2024-04-25T10:35:52.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.433+0000 7fc2f1969640 1 --2- 172.21.15.43:0/4084819266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2ec108a20 0x7fc2ec19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:35:52.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.433+0000 7fc2f216a640 1 --2- 172.21.15.43:0/4084819266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2ec102af0 0x7fc2ec19a700 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00eb90 tx=0x7fc2dc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:52.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.433+0000 7fc2eb7fe640 1 -- 172.21.15.43:0/4084819266 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc010630 con 0x7fc2ec102af0 2024-04-25T10:35:52.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.433+0000 7fc2eb7fe640 1 -- 172.21.15.43:0/4084819266 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2dc00ee90 con 0x7fc2ec102af0 2024-04-25T10:35:52.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.433+0000 7fc2f43ce640 1 -- 172.21.15.43:0/4084819266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ec19f850 con 0x7fc2ec102af0 2024-04-25T10:35:52.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.433+0000 7fc2eb7fe640 1 -- 172.21.15.43:0/4084819266 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc018ac0 con 0x7fc2ec102af0 2024-04-25T10:35:52.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.433+0000 7fc2f43ce640 1 -- 172.21.15.43:0/4084819266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ec1a71d0 con 0x7fc2ec102af0 2024-04-25T10:35:52.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.436+0000 7fc2fb284740 1 Processor -- start 2024-04-25T10:35:52.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.437+0000 7fc2fb284740 1 -- start start 2024-04-25T10:35:52.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.437+0000 7fc2fb284740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce999b390 0x560ce9d3ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.437+0000 7fc2fb284740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce99724d0 0x560ce9d3bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.437+0000 7fc2fb284740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ce997a6b0 0x560ce9d488f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.437+0000 7fc2fb284740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560ce9ca1d80 con 0x560ce999b390 2024-04-25T10:35:52.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.437+0000 7fc2fb284740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560ce9c5b8b0 con 0x560ce997a6b0 2024-04-25T10:35:52.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.437+0000 7fc2fb284740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560ce9c9dd40 con 0x560ce99724d0 2024-04-25T10:35:52.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.437+0000 7fc2eb7fe640 1 -- 172.21.15.43:0/4084819266 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fc2dc018c60 con 0x7fc2ec102af0 2024-04-25T10:35:52.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.437+0000 7fc2f296b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce999b390 0x560ce9d3ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.437+0000 7fc2f296b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce999b390 0x560ce9d3ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42872/0 (socket says 172.21.15.43:42872) 2024-04-25T10:35:52.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.437+0000 7fc2f296b640 1 -- 172.21.15.43:0/2419655765 learned_addr learned my addr 172.21.15.43:0/2419655765 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:52.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.437+0000 7fc2f1969640 1 --2- 172.21.15.43:0/2419655765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ce997a6b0 0x560ce9d488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.437+0000 7fc2f216a640 1 --2- 172.21.15.43:0/2419655765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce99724d0 0x560ce9d3bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.438+0000 7fc2f296b640 1 -- 172.21.15.43:0/2419655765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce99724d0 msgr2=0x560ce9d3bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:52.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.438+0000 7fc2f296b640 1 --2- 172.21.15.43:0/2419655765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce99724d0 0x560ce9d3bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.438+0000 7fc2f296b640 1 -- 172.21.15.43:0/2419655765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ce997a6b0 msgr2=0x560ce9d488f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:52.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.438+0000 7fc2f296b640 1 --2- 172.21.15.43:0/2419655765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ce997a6b0 0x560ce9d488f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.438+0000 7fc2f296b640 1 -- 172.21.15.43:0/2419655765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ce9c2ca40 con 0x560ce999b390 2024-04-25T10:35:52.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.438+0000 7fc2f1969640 1 --2- 172.21.15.43:0/2419655765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ce997a6b0 0x560ce9d488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:52.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.438+0000 7fc2f296b640 1 --2- 172.21.15.43:0/2419655765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce999b390 0x560ce9d3ba60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400bc70 tx=0x7fc2e400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e413cff69128fcd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:52.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.438+0000 7fc2eb7fe640 1 --2- 172.21.15.43:0/4084819266 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc2d40afc60 0x7fc2d40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.438+0000 7fc2e8bfe640 1 -- 172.21.15.43:0/2419655765 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4015070 con 0x560ce999b390 2024-04-25T10:35:52.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.438+0000 7fc2e8bfe640 1 -- 172.21.15.43:0/2419655765 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2e40118f0 con 0x560ce999b390 2024-04-25T10:35:52.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.438+0000 7fc2f216a640 1 --2- 172.21.15.43:0/4084819266 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc2d40afc60 0x7fc2d40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.438+0000 7fc2eb7fe640 1 -- 172.21.15.43:0/4084819266 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc2dc0df780 con 0x7fc2ec102af0 2024-04-25T10:35:52.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.439+0000 7fc2e8bfe640 1 -- 172.21.15.43:0/2419655765 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4010840 con 0x560ce999b390 2024-04-25T10:35:52.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.439+0000 7fc2fb284740 1 -- 172.21.15.43:0/2419655765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce999b390 msgr2=0x560ce9d3ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:52.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.439+0000 7fc2f216a640 1 --2- 172.21.15.43:0/4084819266 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc2d40afc60 0x7fc2d40b2170 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x560ce9d391a0 tx=0x7fc2dc0e5530 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:52.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.439+0000 7fc2fb284740 1 --2- 172.21.15.43:0/2419655765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce999b390 0x560ce9d3ba60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400bc70 tx=0x7fc2e400f730 comp rx=0 tx=0).stop 2024-04-25T10:35:52.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.439+0000 7fc2fb284740 1 -- 172.21.15.43:0/2419655765 shutdown_connections 2024-04-25T10:35:52.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.439+0000 7fc2fb284740 1 --2- 172.21.15.43:0/2419655765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ce997a6b0 0x560ce9d488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.439+0000 7fc2fb284740 1 --2- 172.21.15.43:0/2419655765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce99724d0 0x560ce9d3bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.439+0000 7fc2fb284740 1 --2- 172.21.15.43:0/2419655765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce999b390 0x560ce9d3ba60 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.439+0000 7fc2fb284740 1 -- 172.21.15.43:0/2419655765 >> 172.21.15.43:0/2419655765 conn(0x560ce998f4e0 msgr2=0x560ce9d38210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:52.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.439+0000 7fc2fb284740 1 -- 172.21.15.43:0/2419655765 shutdown_connections 2024-04-25T10:35:52.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.439+0000 7fc2fb284740 1 -- 172.21.15.43:0/2419655765 wait complete. 2024-04-25T10:35:52.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.440+0000 7fc2fb284740 1 -- start start 2024-04-25T10:35:52.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.440+0000 7fc2fb284740 1 Processor -- start 2024-04-25T10:35:52.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.440+0000 7fc2fb284740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce99724d0 0x560ce9d465e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.440+0000 7fc2fb284740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce997a6b0 0x560ce9d46b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.440+0000 7fc2fb284740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ce999b390 0x560ce9d4c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.440+0000 7fc2fb284740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560ce9d48fd0 con 0x560ce99724d0 2024-04-25T10:35:52.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.440+0000 7fc2fb284740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560ce9c28d10 con 0x560ce999b390 2024-04-25T10:35:52.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.440+0000 7fc2fb284740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560ce9d49150 con 0x560ce997a6b0 2024-04-25T10:35:52.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.440+0000 7fc2fb284740 10 client.0 _ll_register_callbacks cb 0x560ce9c55210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:52.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.440+0000 7fc2f296b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce99724d0 0x560ce9d465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.440+0000 7fc2f296b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce99724d0 0x560ce9d465e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42888/0 (socket says 172.21.15.43:42888) 2024-04-25T10:35:52.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.440+0000 7fc2f296b640 1 -- 172.21.15.43:0/3488925092 learned_addr learned my addr 172.21.15.43:0/3488925092 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:52.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.441+0000 7fc2f296b640 1 --2- 172.21.15.43:0/3488925092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ce999b390 0x560ce9d4c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.441+0000 7fc2f1969640 1 --2- 172.21.15.43:0/3488925092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce997a6b0 0x560ce9d46b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.441+0000 7fc2f296b640 1 -- 172.21.15.43:0/3488925092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce997a6b0 msgr2=0x560ce9d46b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:52.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.441+0000 7fc2f296b640 1 --2- 172.21.15.43:0/3488925092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce997a6b0 0x560ce9d46b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.441+0000 7fc2f296b640 1 -- 172.21.15.43:0/3488925092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ce999b390 msgr2=0x560ce9d4c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:52.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.441+0000 7fc2f296b640 1 --2- 172.21.15.43:0/3488925092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ce999b390 0x560ce9d4c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.441+0000 7fc2f296b640 1 -- 172.21.15.43:0/3488925092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ce9a7d7d0 con 0x560ce99724d0 2024-04-25T10:35:52.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.441+0000 7fc2f1969640 1 --2- 172.21.15.43:0/3488925092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce997a6b0 0x560ce9d46b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:35:52.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.441+0000 7fc2f296b640 1 --2- 172.21.15.43:0/3488925092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce99724d0 0x560ce9d465e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4011670 tx=0x7fc2e40103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd39a4ffd21c1500 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:52.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.441+0000 7fc2d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:52.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.441+0000 7fc2d1ffb640 1 -- 172.21.15.43:0/3488925092 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4022040 con 0x560ce99724d0 2024-04-25T10:35:52.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.441+0000 7fc2d1ffb640 1 -- 172.21.15.43:0/3488925092 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc2e4005040 con 0x560ce99724d0 2024-04-25T10:35:52.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.441+0000 7fc2fb284740 1 -- 172.21.15.43:0/3488925092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2e40118f0 con 0x560ce99724d0 2024-04-25T10:35:52.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.441+0000 7fc2d1ffb640 1 -- 172.21.15.43:0/3488925092 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc2e4015040 con 0x560ce99724d0 2024-04-25T10:35:52.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.441+0000 7fc2fb284740 10 client.16383 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:52.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.441+0000 7fc2fb284740 1 -- 172.21.15.43:0/3488925092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560ce99b33f0 con 0x560ce99724d0 2024-04-25T10:35:52.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.441+0000 7fc2d1ffb640 1 -- 172.21.15.43:0/3488925092 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2e40118f0 con 0x560ce99724d0 2024-04-25T10:35:52.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.442+0000 7fc2d1ffb640 1 -- 172.21.15.43:0/3488925092 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc2e40105c0 con 0x560ce99724d0 2024-04-25T10:35:52.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.442+0000 7fc2fb284740 10 client.16383 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:52.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.442+0000 7fc2fb284740 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:52.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.442+0000 7fc2fb284740 20 client.16383 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:52.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.442+0000 7fc2fb284740 1 --2- 172.21.15.43:0/3488925092 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560ce9959f60 0x560ce9d585a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.442+0000 7fc2fb284740 4 client.16383 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:35:52.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.442+0000 7fc2fb284740 1 -- 172.21.15.43:0/3488925092 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ce9caf990 con 0x560ce9959f60 2024-04-25T10:35:52.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.442+0000 7fc2f1969640 1 --2- 172.21.15.43:0/3488925092 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560ce9959f60 0x560ce9d585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.442+0000 7fc2d1ffb640 10 client.16383 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:52.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.442+0000 7fc2f1969640 1 --2- 172.21.15.43:0/3488925092 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560ce9959f60 0x560ce9d585a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd9f981b7afada25 server_cookie=721549ac5edc4ea1 in_seq=0 out_seq=0 2024-04-25T10:35:52.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.443+0000 7fc2d1ffb640 1 -- 172.21.15.43:0/3488925092 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560ce9caf990 con 0x560ce9959f60 2024-04-25T10:35:52.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.443+0000 7fc2d1ffb640 10 client.16383 handle_command_reply: tid=0 2024-04-25T10:35:52.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.443+0000 7fc2fb284740 1 client.16383 shutdown 2024-04-25T10:35:52.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.443+0000 7fc2fb284740 1 -- 172.21.15.43:0/3488925092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce99724d0 msgr2=0x560ce9d465e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:52.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.443+0000 7fc2fb284740 1 --2- 172.21.15.43:0/3488925092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce99724d0 0x560ce9d465e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4011670 tx=0x7fc2e40103d0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.443+0000 7fc2fb284740 1 -- 172.21.15.43:0/3488925092 shutdown_connections 2024-04-25T10:35:52.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.443+0000 7fc2fb284740 1 --2- 172.21.15.43:0/3488925092 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560ce9959f60 0x560ce9d585a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.443+0000 7fc2fb284740 1 --2- 172.21.15.43:0/3488925092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ce999b390 0x560ce9d4c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.443+0000 7fc2fb284740 1 --2- 172.21.15.43:0/3488925092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce997a6b0 0x560ce9d46b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.443+0000 7fc2d1ffb640 0 client.16383 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:52.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.443+0000 7fc2fb284740 1 --2- 172.21.15.43:0/3488925092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce99724d0 0x560ce9d465e0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.443+0000 7fc2fb284740 1 -- 172.21.15.43:0/3488925092 >> 172.21.15.43:0/3488925092 conn(0x560ce998f4e0 msgr2=0x560ce9d369c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:52.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.444+0000 7fc2fb284740 1 -- 172.21.15.43:0/3488925092 shutdown_connections 2024-04-25T10:35:52.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.444+0000 7fc2fb284740 1 -- 172.21.15.43:0/3488925092 wait complete. 2024-04-25T10:35:52.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.444+0000 7fc2fb284740 20 client.16383 trim_cache size 0 max 16384 2024-04-25T10:35:52.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.462+0000 7fc2fb284740 1 Processor -- start 2024-04-25T10:35:52.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.462+0000 7fc2fb284740 1 -- start start 2024-04-25T10:35:52.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.462+0000 7fc2fb284740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce99724d0 0x560ce9d45280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.462+0000 7fc2fb284740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce997a6b0 0x560ce9d4c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.462+0000 7fc2fb284740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ce999b390 0x560ce9d4c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.462+0000 7fc2fb284740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec1b22a0 con 0x560ce99724d0 2024-04-25T10:35:52.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.462+0000 7fc2fb284740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec1afb70 con 0x560ce999b390 2024-04-25T10:35:52.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.462+0000 7fc2fb284740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec1ad9c0 con 0x560ce997a6b0 2024-04-25T10:35:52.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.462+0000 7fc2f296b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce99724d0 0x560ce9d45280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.462+0000 7fc2f296b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce99724d0 0x560ce9d45280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42890/0 (socket says 172.21.15.43:42890) 2024-04-25T10:35:52.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.462+0000 7fc2f296b640 1 -- 172.21.15.43:0/4221434653 learned_addr learned my addr 172.21.15.43:0/4221434653 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:52.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.462+0000 7fc2f1969640 1 --2- 172.21.15.43:0/4221434653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce997a6b0 0x560ce9d4c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.462+0000 7fc2f296b640 1 -- 172.21.15.43:0/4221434653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce997a6b0 msgr2=0x560ce9d4c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:52.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.462+0000 7fc2f296b640 1 --2- 172.21.15.43:0/4221434653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce997a6b0 0x560ce9d4c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.462+0000 7fc2f296b640 1 -- 172.21.15.43:0/4221434653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ce999b390 msgr2=0x560ce9d4c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:52.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.462+0000 7fc2f296b640 1 --2- 172.21.15.43:0/4221434653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ce999b390 0x560ce9d4c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.463+0000 7fc2f296b640 1 -- 172.21.15.43:0/4221434653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ce99730f0 con 0x560ce99724d0 2024-04-25T10:35:52.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.463+0000 7fc2f296b640 1 --2- 172.21.15.43:0/4221434653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce99724d0 0x560ce9d45280 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400c010 tx=0x7fc2e4011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29cc77b7b662088 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:52.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.463+0000 7fc2d3fff640 1 -- 172.21.15.43:0/4221434653 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e400fc30 con 0x560ce99724d0 2024-04-25T10:35:52.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.463+0000 7fc2d3fff640 1 -- 172.21.15.43:0/4221434653 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2e4035070 con 0x560ce99724d0 2024-04-25T10:35:52.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.463+0000 7fc2d3fff640 1 -- 172.21.15.43:0/4221434653 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e403a470 con 0x560ce99724d0 2024-04-25T10:35:52.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.464+0000 7fc2fb284740 1 -- 172.21.15.43:0/4221434653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce99724d0 msgr2=0x560ce9d45280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:52.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.464+0000 7fc2fb284740 1 --2- 172.21.15.43:0/4221434653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce99724d0 0x560ce9d45280 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400c010 tx=0x7fc2e4011750 comp rx=0 tx=0).stop 2024-04-25T10:35:52.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.464+0000 7fc2fb284740 1 -- 172.21.15.43:0/4221434653 shutdown_connections 2024-04-25T10:35:52.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.464+0000 7fc2fb284740 1 --2- 172.21.15.43:0/4221434653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ce999b390 0x560ce9d4c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.464+0000 7fc2fb284740 1 --2- 172.21.15.43:0/4221434653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce997a6b0 0x560ce9d4c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.464+0000 7fc2fb284740 1 --2- 172.21.15.43:0/4221434653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce99724d0 0x560ce9d45280 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.464+0000 7fc2fb284740 1 -- 172.21.15.43:0/4221434653 >> 172.21.15.43:0/4221434653 conn(0x560ce998f4e0 msgr2=0x560ce9d44b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:52.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.464+0000 7fc2fb284740 1 -- 172.21.15.43:0/4221434653 shutdown_connections 2024-04-25T10:35:52.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.464+0000 7fc2fb284740 1 -- 172.21.15.43:0/4221434653 wait complete. 2024-04-25T10:35:52.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.464+0000 7fc2fb284740 1 -- start start 2024-04-25T10:35:52.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.464+0000 7fc2fb284740 1 Processor -- start 2024-04-25T10:35:52.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.464+0000 7fc2fb284740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce99724d0 0x560ce9d4dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.464+0000 7fc2fb284740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce997a6b0 0x560ce9d4e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.464+0000 7fc2fb284740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ce999b390 0x560ce9dbe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.464+0000 7fc2fb284740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec1ae030 con 0x560ce997a6b0 2024-04-25T10:35:52.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.464+0000 7fc2fb284740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec1af4a0 con 0x560ce999b390 2024-04-25T10:35:52.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.464+0000 7fc2fb284740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec1ae700 con 0x560ce99724d0 2024-04-25T10:35:52.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.464+0000 7fc2fb284740 10 client.0 _ll_register_callbacks cb 0x7fc2dc0e5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:52.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.465+0000 7fc2f1969640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce997a6b0 0x560ce9d4e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.465+0000 7fc2f1969640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce997a6b0 0x560ce9d4e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42896/0 (socket says 172.21.15.43:42896) 2024-04-25T10:35:52.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.465+0000 7fc2f1969640 1 -- 172.21.15.43:0/1656275672 learned_addr learned my addr 172.21.15.43:0/1656275672 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:52.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.465+0000 7fc2f296b640 1 --2- 172.21.15.43:0/1656275672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce99724d0 0x560ce9d4dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.465+0000 7fc2f296b640 1 --2- 172.21.15.43:0/1656275672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ce999b390 0x560ce9dbe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.465+0000 7fc2f1969640 1 -- 172.21.15.43:0/1656275672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce99724d0 msgr2=0x560ce9d4dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:52.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.465+0000 7fc2f1969640 1 --2- 172.21.15.43:0/1656275672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce99724d0 0x560ce9d4dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.465+0000 7fc2f1969640 1 -- 172.21.15.43:0/1656275672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ce999b390 msgr2=0x560ce9dbe770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:52.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.465+0000 7fc2f1969640 1 --2- 172.21.15.43:0/1656275672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ce999b390 0x560ce9dbe770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.465+0000 7fc2f1969640 1 -- 172.21.15.43:0/1656275672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ce9c624b0 con 0x560ce997a6b0 2024-04-25T10:35:52.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.465+0000 7fc2f1969640 1 --2- 172.21.15.43:0/1656275672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce997a6b0 0x560ce9d4e2b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0000c00 tx=0x7fc2e0032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387bbc78a5372337 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:52.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.465+0000 7fc2d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:52.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.465+0000 7fc2d1ffb640 1 -- 172.21.15.43:0/1656275672 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e004a060 con 0x560ce997a6b0 2024-04-25T10:35:52.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.465+0000 7fc2d1ffb640 1 -- 172.21.15.43:0/1656275672 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc2e0005040 con 0x560ce997a6b0 2024-04-25T10:35:52.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.465+0000 7fc2fb284740 1 -- 172.21.15.43:0/1656275672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2e4035070 con 0x560ce997a6b0 2024-04-25T10:35:52.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.465+0000 7fc2d1ffb640 1 -- 172.21.15.43:0/1656275672 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc2e0035040 con 0x560ce997a6b0 2024-04-25T10:35:52.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.465+0000 7fc2fb284740 10 client.16389 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:52.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.465+0000 7fc2fb284740 1 -- 172.21.15.43:0/1656275672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560ce9dbfd30 con 0x560ce997a6b0 2024-04-25T10:35:52.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.466+0000 7fc2d1ffb640 1 -- 172.21.15.43:0/1656275672 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2e4035070 con 0x560ce997a6b0 2024-04-25T10:35:52.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.466+0000 7fc2d1ffb640 1 -- 172.21.15.43:0/1656275672 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc2e0044070 con 0x560ce997a6b0 2024-04-25T10:35:52.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.466+0000 7fc2fb284740 10 client.16389 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:52.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.466+0000 7fc2fb284740 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:52.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.466+0000 7fc2fb284740 20 client.16389 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:52.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.466+0000 7fc2fb284740 1 --2- 172.21.15.43:0/1656275672 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560ce992f3a0 0x560ce9dc24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.466+0000 7fc2fb284740 4 client.16389 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:35:52.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.466+0000 7fc2fb284740 1 -- 172.21.15.43:0/1656275672 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560ce9ca9800 con 0x560ce992f3a0 2024-04-25T10:35:52.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.466+0000 7fc2f296b640 1 --2- 172.21.15.43:0/1656275672 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560ce992f3a0 0x560ce9dc24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.467+0000 7fc2f296b640 1 --2- 172.21.15.43:0/1656275672 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560ce992f3a0 0x560ce9dc24d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a19ecbb4fa2bd20 server_cookie=63d9b5154499c5bc in_seq=0 out_seq=0 2024-04-25T10:35:52.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.467+0000 7fc2d1ffb640 10 client.16389 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:52.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.467+0000 7fc2d1ffb640 1 -- 172.21.15.43:0/1656275672 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560ce9ca9800 con 0x560ce992f3a0 2024-04-25T10:35:52.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.467+0000 7fc2d1ffb640 10 client.16389 handle_command_reply: tid=0 2024-04-25T10:35:52.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.467+0000 7fc2fb284740 1 client.16389 shutdown 2024-04-25T10:35:52.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.467+0000 7fc2fb284740 1 -- 172.21.15.43:0/1656275672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce997a6b0 msgr2=0x560ce9d4e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:52.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.467+0000 7fc2fb284740 1 --2- 172.21.15.43:0/1656275672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce997a6b0 0x560ce9d4e2b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0000c00 tx=0x7fc2e0032cd0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.467+0000 7fc2fb284740 1 -- 172.21.15.43:0/1656275672 shutdown_connections 2024-04-25T10:35:52.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.467+0000 7fc2fb284740 1 --2- 172.21.15.43:0/1656275672 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560ce992f3a0 0x560ce9dc24d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.467+0000 7fc2fb284740 1 --2- 172.21.15.43:0/1656275672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ce999b390 0x560ce9dbe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.467+0000 7fc2fb284740 1 --2- 172.21.15.43:0/1656275672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ce997a6b0 0x560ce9d4e2b0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.467+0000 7fc2fb284740 1 --2- 172.21.15.43:0/1656275672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ce99724d0 0x560ce9d4dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.467+0000 7fc2fb284740 1 -- 172.21.15.43:0/1656275672 >> 172.21.15.43:0/1656275672 conn(0x560ce998f4e0 msgr2=0x560ce9d3f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.467+0000 7fc2d1ffb640 0 client.16389 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.467+0000 7fc2fb284740 1 -- 172.21.15.43:0/1656275672 shutdown_connections 2024-04-25T10:35:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.467+0000 7fc2fb284740 1 -- 172.21.15.43:0/1656275672 wait complete. 2024-04-25T10:35:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.467+0000 7fc2fb284740 20 client.16389 trim_cache size 0 max 16384 2024-04-25T10:35:52.485 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:35:52.485 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:35:52.485 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:35:52.485 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:35:52.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.468+0000 7fc2f43ce640 1 -- 172.21.15.43:0/4084819266 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc2d40afc60 msgr2=0x7fc2d40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:52.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.468+0000 7fc2f43ce640 1 --2- 172.21.15.43:0/4084819266 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc2d40afc60 0x7fc2d40b2170 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x560ce9d391a0 tx=0x7fc2dc0e5530 comp rx=0 tx=0).stop 2024-04-25T10:35:52.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.468+0000 7fc2f43ce640 1 -- 172.21.15.43:0/4084819266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2ec102af0 msgr2=0x7fc2ec19a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:52.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.468+0000 7fc2f43ce640 1 --2- 172.21.15.43:0/4084819266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2ec102af0 0x7fc2ec19a700 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00eb90 tx=0x7fc2dc00c6a0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.468+0000 7fc2f43ce640 1 -- 172.21.15.43:0/4084819266 shutdown_connections 2024-04-25T10:35:52.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.468+0000 7fc2f43ce640 1 --2- 172.21.15.43:0/4084819266 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc2d40afc60 0x7fc2d40b2170 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.468+0000 7fc2f43ce640 1 --2- 172.21.15.43:0/4084819266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2ec10d380 0x7fc2ec19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.468+0000 7fc2f43ce640 1 --2- 172.21.15.43:0/4084819266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2ec108a20 0x7fc2ec19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.468+0000 7fc2f43ce640 1 --2- 172.21.15.43:0/4084819266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2ec102af0 0x7fc2ec19a700 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.468+0000 7fc2f43ce640 1 -- 172.21.15.43:0/4084819266 >> 172.21.15.43:0/4084819266 conn(0x7fc2ec078100 msgr2=0x7fc2ec1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:52.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.468+0000 7fc2f43ce640 1 -- 172.21.15.43:0/4084819266 shutdown_connections 2024-04-25T10:35:52.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.468+0000 7fc2f43ce640 1 -- 172.21.15.43:0/4084819266 wait complete. 2024-04-25T10:35:52.488 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:35:52.489 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:35:52.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.598+0000 7fc91d7b2640 1 -- 172.21.15.43:0/2400827416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9181064e0 msgr2=0x7fc918112860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:52.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.598+0000 7fc91d7b2640 1 --2- 172.21.15.43:0/2400827416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9181064e0 0x7fc918112860 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc910009ca0 tx=0x7fc91002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.599+0000 7fc91d7b2640 1 -- 172.21.15.43:0/2400827416 shutdown_connections 2024-04-25T10:35:52.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.599+0000 7fc91d7b2640 1 --2- 172.21.15.43:0/2400827416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9181064e0 0x7fc918112860 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.599+0000 7fc91d7b2640 1 --2- 172.21.15.43:0/2400827416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc91810c1e0 0x7fc918105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.599+0000 7fc91d7b2640 1 --2- 172.21.15.43:0/2400827416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc91810bd40 0x7fc918105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.599+0000 7fc91d7b2640 1 -- 172.21.15.43:0/2400827416 >> 172.21.15.43:0/2400827416 conn(0x7fc918101590 msgr2=0x7fc9181039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:52.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.599+0000 7fc91d7b2640 1 -- 172.21.15.43:0/2400827416 shutdown_connections 2024-04-25T10:35:52.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.599+0000 7fc91d7b2640 1 -- 172.21.15.43:0/2400827416 wait complete. 2024-04-25T10:35:52.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.599+0000 7fc91d7b2640 1 Processor -- start 2024-04-25T10:35:52.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.599+0000 7fc91d7b2640 1 -- start start 2024-04-25T10:35:52.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.600+0000 7fc91d7b2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9181064e0 0x7fc9181a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.600+0000 7fc91d7b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc91810bd40 0x7fc9181a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.600+0000 7fc91d7b2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc91810c1e0 0x7fc91819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.600+0000 7fc91d7b2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc918115f50 con 0x7fc91810bd40 2024-04-25T10:35:52.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.600+0000 7fc91d7b2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc918115dd0 con 0x7fc9181064e0 2024-04-25T10:35:52.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.600+0000 7fc91d7b2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc9181160d0 con 0x7fc91810c1e0 2024-04-25T10:35:52.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.600+0000 7fc9167fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc91810bd40 0x7fc9181a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.600+0000 7fc9167fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc91810bd40 0x7fc9181a2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42912/0 (socket says 172.21.15.43:42912) 2024-04-25T10:35:52.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.600+0000 7fc9167fc640 1 -- 172.21.15.43:0/4058079510 learned_addr learned my addr 172.21.15.43:0/4058079510 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:52.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.600+0000 7fc916ffd640 1 --2- 172.21.15.43:0/4058079510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9181064e0 0x7fc9181a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.600+0000 7fc9177fe640 1 --2- 172.21.15.43:0/4058079510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc91810c1e0 0x7fc91819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.600+0000 7fc9167fc640 1 -- 172.21.15.43:0/4058079510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc91810c1e0 msgr2=0x7fc91819c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:52.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.600+0000 7fc9167fc640 1 --2- 172.21.15.43:0/4058079510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc91810c1e0 0x7fc91819c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.600+0000 7fc9167fc640 1 -- 172.21.15.43:0/4058079510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9181064e0 msgr2=0x7fc9181a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:52.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.600+0000 7fc9167fc640 1 --2- 172.21.15.43:0/4058079510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9181064e0 0x7fc9181a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.600+0000 7fc9167fc640 1 -- 172.21.15.43:0/4058079510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc91819ce40 con 0x7fc91810bd40 2024-04-25T10:35:52.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.600+0000 7fc916ffd640 1 --2- 172.21.15.43:0/4058079510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9181064e0 0x7fc9181a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:52.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.600+0000 7fc9177fe640 1 --2- 172.21.15.43:0/4058079510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc91810c1e0 0x7fc91819c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:35:52.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.600+0000 7fc9167fc640 1 --2- 172.21.15.43:0/4058079510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc91810bd40 0x7fc9181a2ad0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc90c005d00 tx=0x7fc90c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:52.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.600+0000 7fc8fffff640 1 -- 172.21.15.43:0/4058079510 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90c017070 con 0x7fc91810bd40 2024-04-25T10:35:52.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.601+0000 7fc8fffff640 1 -- 172.21.15.43:0/4058079510 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc90c009c30 con 0x7fc91810bd40 2024-04-25T10:35:52.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.601+0000 7fc91d7b2640 1 -- 172.21.15.43:0/4058079510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc91819d0d0 con 0x7fc91810bd40 2024-04-25T10:35:52.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.601+0000 7fc8fffff640 1 -- 172.21.15.43:0/4058079510 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90c00f620 con 0x7fc91810bd40 2024-04-25T10:35:52.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.601+0000 7fc91d7b2640 1 -- 172.21.15.43:0/4058079510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9181a9330 con 0x7fc91810bd40 2024-04-25T10:35:52.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.602+0000 7fc91d7b2640 1 -- 172.21.15.43:0/4058079510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc918107020 con 0x7fc91810bd40 2024-04-25T10:35:52.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.603+0000 7fc8fffff640 1 -- 172.21.15.43:0/4058079510 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fc90c01a070 con 0x7fc91810bd40 2024-04-25T10:35:52.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.604+0000 7fc8fffff640 1 --2- 172.21.15.43:0/4058079510 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9000afc60 0x7fc9000b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.604+0000 7fc8fffff640 1 -- 172.21.15.43:0/4058079510 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc90c0deab0 con 0x7fc91810bd40 2024-04-25T10:35:52.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.604+0000 7fc916ffd640 1 --2- 172.21.15.43:0/4058079510 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9000afc60 0x7fc9000b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.604+0000 7fc916ffd640 1 --2- 172.21.15.43:0/4058079510 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9000afc60 0x7fc9000b2170 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc9080058c0 tx=0x7fc908005850 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:52.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.608+0000 7fc8fffff640 1 -- 172.21.15.43:0/4058079510 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc90c009de0 con 0x7fc91810bd40 2024-04-25T10:35:52.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.834+0000 7fc91d7b2640 1 -- 172.21.15.43:0/4058079510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc91810de50 con 0x7fc91810bd40 2024-04-25T10:35:52.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.835+0000 7fc8fffff640 1 -- 172.21.15.43:0/4058079510 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7fc90c0a7380 con 0x7fc91810bd40 2024-04-25T10:35:52.838 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:35:52.838 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:35:52.839 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:35:52.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.837+0000 7fc91d7b2640 1 -- 172.21.15.43:0/4058079510 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9000afc60 msgr2=0x7fc9000b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:52.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.837+0000 7fc91d7b2640 1 --2- 172.21.15.43:0/4058079510 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9000afc60 0x7fc9000b2170 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc9080058c0 tx=0x7fc908005850 comp rx=0 tx=0).stop 2024-04-25T10:35:52.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.837+0000 7fc91d7b2640 1 -- 172.21.15.43:0/4058079510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc91810bd40 msgr2=0x7fc9181a2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:52.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.837+0000 7fc91d7b2640 1 --2- 172.21.15.43:0/4058079510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc91810bd40 0x7fc9181a2ad0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc90c005d00 tx=0x7fc90c00dde0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.837+0000 7fc91d7b2640 1 -- 172.21.15.43:0/4058079510 shutdown_connections 2024-04-25T10:35:52.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.837+0000 7fc91d7b2640 1 --2- 172.21.15.43:0/4058079510 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9000afc60 0x7fc9000b2170 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.837+0000 7fc91d7b2640 1 --2- 172.21.15.43:0/4058079510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc91810c1e0 0x7fc91819c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.837+0000 7fc91d7b2640 1 --2- 172.21.15.43:0/4058079510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc91810bd40 0x7fc9181a2ad0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.837+0000 7fc91d7b2640 1 --2- 172.21.15.43:0/4058079510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9181064e0 0x7fc9181a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.837+0000 7fc91d7b2640 1 -- 172.21.15.43:0/4058079510 >> 172.21.15.43:0/4058079510 conn(0x7fc918101590 msgr2=0x7fc918103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:52.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.837+0000 7fc91d7b2640 1 -- 172.21.15.43:0/4058079510 shutdown_connections 2024-04-25T10:35:52.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.837+0000 7fc91d7b2640 1 -- 172.21.15.43:0/4058079510 wait complete. 2024-04-25T10:35:52.851 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:35:52.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.963+0000 7f688a738640 1 -- 172.21.15.43:0/2931178926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f688410d380 msgr2=0x7f688410f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:52.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.963+0000 7f688a738640 1 --2- 172.21.15.43:0/2931178926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f688410d380 0x7f688410f7a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f687c009ca0 tx=0x7f687c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.963+0000 7f688a738640 1 -- 172.21.15.43:0/2931178926 shutdown_connections 2024-04-25T10:35:52.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.963+0000 7f688a738640 1 --2- 172.21.15.43:0/2931178926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f688410d380 0x7f688410f7a0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.963+0000 7f688a738640 1 --2- 172.21.15.43:0/2931178926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6884102af0 0x7f6884102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.963+0000 7f688a738640 1 --2- 172.21.15.43:0/2931178926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6884108a20 0x7f68841025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.963+0000 7f688a738640 1 -- 172.21.15.43:0/2931178926 >> 172.21.15.43:0/2931178926 conn(0x7f6884078100 msgr2=0x7f6884078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:52.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.963+0000 7f688a738640 1 -- 172.21.15.43:0/2931178926 shutdown_connections 2024-04-25T10:35:52.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.963+0000 7f688a738640 1 -- 172.21.15.43:0/2931178926 wait complete. 2024-04-25T10:35:52.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.964+0000 7f688a738640 1 Processor -- start 2024-04-25T10:35:52.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.964+0000 7f688a738640 1 -- start start 2024-04-25T10:35:52.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.964+0000 7f688a738640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6884102af0 0x7f68841995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.964+0000 7f688a738640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6884108a20 0x7f6884199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.964+0000 7f688a738640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f688410d380 0x7f688419ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.964+0000 7f688a738640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6884112c00 con 0x7f6884102af0 2024-04-25T10:35:52.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.964+0000 7f688a738640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6884112a80 con 0x7f6884108a20 2024-04-25T10:35:52.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.964+0000 7f688a738640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6884112d80 con 0x7f688410d380 2024-04-25T10:35:52.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.964+0000 7f6883fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6884102af0 0x7f68841995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.964+0000 7f6883fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6884102af0 0x7f68841995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42938/0 (socket says 172.21.15.43:42938) 2024-04-25T10:35:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.964+0000 7f6883fff640 1 -- 172.21.15.43:0/3226130771 learned_addr learned my addr 172.21.15.43:0/3226130771 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.964+0000 7f68837fe640 1 --2- 172.21.15.43:0/3226130771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6884108a20 0x7f6884199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.964+0000 7f6888cd5640 1 --2- 172.21.15.43:0/3226130771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f688410d380 0x7f688419ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.965+0000 7f6883fff640 1 -- 172.21.15.43:0/3226130771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f688410d380 msgr2=0x7f688419ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.965+0000 7f6883fff640 1 --2- 172.21.15.43:0/3226130771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f688410d380 0x7f688419ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.965+0000 7f6883fff640 1 -- 172.21.15.43:0/3226130771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6884108a20 msgr2=0x7f6884199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.965+0000 7f6883fff640 1 --2- 172.21.15.43:0/3226130771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6884108a20 0x7f6884199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.965+0000 7f6883fff640 1 -- 172.21.15.43:0/3226130771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688419e380 con 0x7f6884102af0 2024-04-25T10:35:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.965+0000 7f6883fff640 1 --2- 172.21.15.43:0/3226130771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6884102af0 0x7f68841995a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f687400c9a0 tx=0x7f687400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.965+0000 7f68817fa640 1 -- 172.21.15.43:0/3226130771 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874014070 con 0x7f6884102af0 2024-04-25T10:35:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.965+0000 7f688a738640 1 -- 172.21.15.43:0/3226130771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f688419e670 con 0x7f6884102af0 2024-04-25T10:35:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.965+0000 7f68817fa640 1 -- 172.21.15.43:0/3226130771 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6874004440 con 0x7f6884102af0 2024-04-25T10:35:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.965+0000 7f68817fa640 1 -- 172.21.15.43:0/3226130771 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687400f450 con 0x7f6884102af0 2024-04-25T10:35:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.965+0000 7f688a738640 1 -- 172.21.15.43:0/3226130771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68841a6070 con 0x7f6884102af0 2024-04-25T10:35:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.968+0000 7f68915ed740 1 Processor -- start 2024-04-25T10:35:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.968+0000 7f68915ed740 1 -- start start 2024-04-25T10:35:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.968+0000 7f68915ed740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b35390 0x55c510ed6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.968+0000 7f68915ed740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b0c4d0 0x55c510ed6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.968+0000 7f68915ed740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c510b146b0 0x55c510ee36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.968+0000 7f68915ed740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c510e3bd80 con 0x55c510b35390 2024-04-25T10:35:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.968+0000 7f68915ed740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c510df5a20 con 0x55c510b0c4d0 2024-04-25T10:35:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.968+0000 7f68915ed740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c510e37d40 con 0x55c510b146b0 2024-04-25T10:35:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.968+0000 7f68817fa640 1 -- 172.21.15.43:0/3226130771 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f6874017070 con 0x7f6884102af0 2024-04-25T10:35:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.968+0000 7f6888cd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b35390 0x55c510ed6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.968+0000 7f6888cd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b35390 0x55c510ed6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42948/0 (socket says 172.21.15.43:42948) 2024-04-25T10:35:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.968+0000 7f6888cd5640 1 -- 172.21.15.43:0/4137009673 learned_addr learned my addr 172.21.15.43:0/4137009673 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.968+0000 7f68837fe640 1 --2- 172.21.15.43:0/4137009673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c510b146b0 0x55c510ee36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.968+0000 7f6888cd5640 1 -- 172.21.15.43:0/4137009673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c510b146b0 msgr2=0x55c510ee36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.968+0000 7f6888cd5640 1 --2- 172.21.15.43:0/4137009673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c510b146b0 0x55c510ee36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.968+0000 7f6883fff640 1 --2- 172.21.15.43:0/4137009673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b0c4d0 0x55c510ed6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.968+0000 7f6888cd5640 1 -- 172.21.15.43:0/4137009673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b0c4d0 msgr2=0x55c510ed6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.969+0000 7f6888cd5640 1 --2- 172.21.15.43:0/4137009673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b0c4d0 0x55c510ed6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.969+0000 7f6888cd5640 1 -- 172.21.15.43:0/4137009673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c510e00c30 con 0x55c510b35390 2024-04-25T10:35:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.969+0000 7f6883fff640 1 --2- 172.21.15.43:0/4137009673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b0c4d0 0x55c510ed6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:35:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.969+0000 7f68817fa640 1 --2- 172.21.15.43:0/3226130771 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f686c0afc60 0x7f686c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.969+0000 7f6888cd5640 1 --2- 172.21.15.43:0/4137009673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b35390 0x55c510ed6810 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f687c02f4d0 tx=0x7f687c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37500ba5f34bcbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.969+0000 7f6883fff640 1 --2- 172.21.15.43:0/3226130771 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f686c0afc60 0x7f686c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.969+0000 7f686a7fc640 1 -- 172.21.15.43:0/4137009673 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c031cf0 con 0x55c510b35390 2024-04-25T10:35:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.969+0000 7f686a7fc640 1 -- 172.21.15.43:0/4137009673 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f687c036040 con 0x55c510b35390 2024-04-25T10:35:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.969+0000 7f68817fa640 1 -- 172.21.15.43:0/3226130771 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f68740deb60 con 0x7f6884102af0 2024-04-25T10:35:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.969+0000 7f686a7fc640 1 -- 172.21.15.43:0/4137009673 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c004240 con 0x55c510b35390 2024-04-25T10:35:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.970+0000 7f68915ed740 1 -- 172.21.15.43:0/4137009673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b35390 msgr2=0x55c510ed6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.970+0000 7f6883fff640 1 --2- 172.21.15.43:0/3226130771 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f686c0afc60 0x7f686c0b2170 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f68740dc390 tx=0x7f6874007f50 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.970+0000 7f68915ed740 1 --2- 172.21.15.43:0/4137009673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b35390 0x55c510ed6810 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f687c02f4d0 tx=0x7f687c009800 comp rx=0 tx=0).stop 2024-04-25T10:35:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.970+0000 7f68915ed740 1 -- 172.21.15.43:0/4137009673 shutdown_connections 2024-04-25T10:35:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.970+0000 7f68915ed740 1 --2- 172.21.15.43:0/4137009673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b0c4d0 0x55c510ed6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.970+0000 7f68915ed740 1 --2- 172.21.15.43:0/4137009673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c510b146b0 0x55c510ee36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.970+0000 7f68915ed740 1 --2- 172.21.15.43:0/4137009673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b35390 0x55c510ed6810 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.970+0000 7f68915ed740 1 -- 172.21.15.43:0/4137009673 >> 172.21.15.43:0/4137009673 conn(0x55c510b294e0 msgr2=0x55c510ed2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.970+0000 7f68915ed740 1 -- 172.21.15.43:0/4137009673 shutdown_connections 2024-04-25T10:35:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.970+0000 7f68915ed740 1 -- 172.21.15.43:0/4137009673 wait complete. 2024-04-25T10:35:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.970+0000 7f68915ed740 1 -- start start 2024-04-25T10:35:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.971+0000 7f68915ed740 1 Processor -- start 2024-04-25T10:35:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.971+0000 7f68915ed740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c510b0c4d0 0x55c510ee11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.971+0000 7f68915ed740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b146b0 0x55c510ee16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.971+0000 7f68915ed740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b35390 0x55c510eed1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.971+0000 7f68915ed740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c510ee3d30 con 0x55c510b146b0 2024-04-25T10:35:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.971+0000 7f68915ed740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c510dc2dc0 con 0x55c510b35390 2024-04-25T10:35:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.971+0000 7f68915ed740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c510ee3eb0 con 0x55c510b0c4d0 2024-04-25T10:35:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.971+0000 7f68915ed740 10 client.0 _ll_register_callbacks cb 0x55c510dde600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.971+0000 7f68837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b146b0 0x55c510ee16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.971+0000 7f68837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b146b0 0x55c510ee16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42964/0 (socket says 172.21.15.43:42964) 2024-04-25T10:35:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.971+0000 7f68837fe640 1 -- 172.21.15.43:0/1244414182 learned_addr learned my addr 172.21.15.43:0/1244414182 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.971+0000 7f6888cd5640 1 --2- 172.21.15.43:0/1244414182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c510b0c4d0 0x55c510ee11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.971+0000 7f6888cd5640 1 --2- 172.21.15.43:0/1244414182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b35390 0x55c510eed1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.971+0000 7f68837fe640 1 -- 172.21.15.43:0/1244414182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c510b0c4d0 msgr2=0x55c510ee11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.971+0000 7f68837fe640 1 --2- 172.21.15.43:0/1244414182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c510b0c4d0 0x55c510ee11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.971+0000 7f68837fe640 1 -- 172.21.15.43:0/1244414182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b35390 msgr2=0x55c510eed1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.971+0000 7f68837fe640 1 --2- 172.21.15.43:0/1244414182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b35390 0x55c510eed1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.971+0000 7f68837fe640 1 -- 172.21.15.43:0/1244414182 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c510b4d3f0 con 0x55c510b146b0 2024-04-25T10:35:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.972+0000 7f68837fe640 1 --2- 172.21.15.43:0/1244414182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b146b0 0x55c510ee16f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f687800bde0 tx=0x7f6878012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef894dc0807ec9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.972+0000 7f68637fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.972+0000 7f68637fe640 1 -- 172.21.15.43:0/1244414182 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6878010850 con 0x55c510b146b0 2024-04-25T10:35:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.972+0000 7f68637fe640 1 -- 172.21.15.43:0/1244414182 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f68780109f0 con 0x55c510b146b0 2024-04-25T10:35:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.972+0000 7f68915ed740 1 -- 172.21.15.43:0/1244414182 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f687c036040 con 0x55c510b146b0 2024-04-25T10:35:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.972+0000 7f68637fe640 1 -- 172.21.15.43:0/1244414182 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6878010bf0 con 0x55c510b146b0 2024-04-25T10:35:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.972+0000 7f68915ed740 10 client.16407 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.972+0000 7f68915ed740 1 -- 172.21.15.43:0/1244414182 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c510c177d0 con 0x55c510b146b0 2024-04-25T10:35:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.972+0000 7f68637fe640 1 -- 172.21.15.43:0/1244414182 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f687c036040 con 0x55c510b146b0 2024-04-25T10:35:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.972+0000 7f68637fe640 1 -- 172.21.15.43:0/1244414182 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f687800f3c0 con 0x55c510b146b0 2024-04-25T10:35:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.972+0000 7f68915ed740 10 client.16407 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.972+0000 7f68915ed740 10 client.16407 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:35:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.972+0000 7f68915ed740 20 client.16407 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.972+0000 7f68915ed740 1 --2- 172.21.15.43:0/1244414182 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c510af3f60 0x55c510ef35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.972+0000 7f68915ed740 4 client.16407 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:35:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.972+0000 7f68915ed740 1 -- 172.21.15.43:0/1244414182 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c510e49990 con 0x55c510af3f60 2024-04-25T10:35:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.973+0000 7f6888cd5640 1 --2- 172.21.15.43:0/1244414182 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c510af3f60 0x55c510ef35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.973+0000 7f6888cd5640 1 --2- 172.21.15.43:0/1244414182 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c510af3f60 0x55c510ef35b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=86d537c4952e0226 server_cookie=6b0a82b4d1eddbb7 in_seq=0 out_seq=0 2024-04-25T10:35:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.973+0000 7f68637fe640 10 client.16407 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:35:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.974+0000 7f68637fe640 1 -- 172.21.15.43:0/1244414182 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c510e49990 con 0x55c510af3f60 2024-04-25T10:35:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.974+0000 7f68637fe640 10 client.16407 handle_command_reply: tid=0 2024-04-25T10:35:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.974+0000 7f68915ed740 1 client.16407 shutdown 2024-04-25T10:35:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.974+0000 7f68915ed740 1 -- 172.21.15.43:0/1244414182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b146b0 msgr2=0x55c510ee16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.974+0000 7f68915ed740 1 --2- 172.21.15.43:0/1244414182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b146b0 0x55c510ee16f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f687800bde0 tx=0x7f6878012820 comp rx=0 tx=0).stop 2024-04-25T10:35:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.974+0000 7f68915ed740 1 -- 172.21.15.43:0/1244414182 shutdown_connections 2024-04-25T10:35:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.974+0000 7f68915ed740 1 --2- 172.21.15.43:0/1244414182 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c510af3f60 0x55c510ef35b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.974+0000 7f68915ed740 1 --2- 172.21.15.43:0/1244414182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b35390 0x55c510eed1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.974+0000 7f68915ed740 1 --2- 172.21.15.43:0/1244414182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b146b0 0x55c510ee16f0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.974+0000 7f68915ed740 1 --2- 172.21.15.43:0/1244414182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c510b0c4d0 0x55c510ee11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.974+0000 7f68637fe640 0 client.16407 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:35:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.974+0000 7f68915ed740 1 -- 172.21.15.43:0/1244414182 >> 172.21.15.43:0/1244414182 conn(0x55c510b294e0 msgr2=0x55c510ed8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.974+0000 7f68915ed740 1 -- 172.21.15.43:0/1244414182 shutdown_connections 2024-04-25T10:35:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.974+0000 7f68915ed740 1 -- 172.21.15.43:0/1244414182 wait complete. 2024-04-25T10:35:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.974+0000 7f68915ed740 20 client.16407 trim_cache size 0 max 16384 2024-04-25T10:35:52.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.988+0000 7f68915ed740 1 Processor -- start 2024-04-25T10:35:52.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.988+0000 7f68915ed740 1 -- start start 2024-04-25T10:35:52.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.988+0000 7f68915ed740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b0c4d0 0x55c510eece10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.988+0000 7f68915ed740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b146b0 0x55c510ee9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.988+0000 7f68915ed740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c510b35390 0x55c510ef8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:52.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.988+0000 7f68915ed740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f68841b0ef0 con 0x55c510b0c4d0 2024-04-25T10:35:52.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.988+0000 7f68915ed740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68841ae7c0 con 0x55c510b146b0 2024-04-25T10:35:52.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.988+0000 7f68915ed740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f68841ac610 con 0x55c510b35390 2024-04-25T10:35:52.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.988+0000 7f6888cd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b0c4d0 0x55c510eece10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.988+0000 7f6888cd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b0c4d0 0x55c510eece10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42978/0 (socket says 172.21.15.43:42978) 2024-04-25T10:35:52.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.988+0000 7f6888cd5640 1 -- 172.21.15.43:0/3055800682 learned_addr learned my addr 172.21.15.43:0/3055800682 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:52.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.989+0000 7f6888cd5640 1 -- 172.21.15.43:0/3055800682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c510b35390 msgr2=0x55c510ef8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:52.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.989+0000 7f6888cd5640 1 --2- 172.21.15.43:0/3055800682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c510b35390 0x55c510ef8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.989+0000 7f6888cd5640 1 -- 172.21.15.43:0/3055800682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b146b0 msgr2=0x55c510ee9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:52.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.989+0000 7f68837fe640 1 --2- 172.21.15.43:0/3055800682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b146b0 0x55c510ee9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:52.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.989+0000 7f6888cd5640 1 --2- 172.21.15.43:0/3055800682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b146b0 0x55c510ee9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.989+0000 7f6888cd5640 1 -- 172.21.15.43:0/3055800682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c510ed3ee0 con 0x55c510b0c4d0 2024-04-25T10:35:52.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.989+0000 7f6888cd5640 1 --2- 172.21.15.43:0/3055800682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b0c4d0 0x55c510eece10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f687c009a60 tx=0x7f687c003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f3e5e3fb25e331 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:52.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.989+0000 7f6869ffb640 1 -- 172.21.15.43:0/3055800682 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c003de0 con 0x55c510b0c4d0 2024-04-25T10:35:52.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.989+0000 7f6869ffb640 1 -- 172.21.15.43:0/3055800682 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f687c036070 con 0x55c510b0c4d0 2024-04-25T10:35:52.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.989+0000 7f6869ffb640 1 -- 172.21.15.43:0/3055800682 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c0045a0 con 0x55c510b0c4d0 2024-04-25T10:35:52.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.990+0000 7f68915ed740 1 -- 172.21.15.43:0/3055800682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b0c4d0 msgr2=0x55c510eece10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:52.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.990+0000 7f68915ed740 1 --2- 172.21.15.43:0/3055800682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b0c4d0 0x55c510eece10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f687c009a60 tx=0x7f687c003d70 comp rx=0 tx=0).stop 2024-04-25T10:35:52.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.990+0000 7f68915ed740 1 -- 172.21.15.43:0/3055800682 shutdown_connections 2024-04-25T10:35:52.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.990+0000 7f68915ed740 1 --2- 172.21.15.43:0/3055800682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b146b0 0x55c510ee9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.990+0000 7f68915ed740 1 --2- 172.21.15.43:0/3055800682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c510b35390 0x55c510ef8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.990+0000 7f68915ed740 1 --2- 172.21.15.43:0/3055800682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b0c4d0 0x55c510eece10 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:52.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.990+0000 7f68915ed740 1 -- 172.21.15.43:0/3055800682 >> 172.21.15.43:0/3055800682 conn(0x55c510b294e0 msgr2=0x55c510ee11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:52.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.990+0000 7f68915ed740 1 -- 172.21.15.43:0/3055800682 shutdown_connections 2024-04-25T10:35:52.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.990+0000 7f68915ed740 1 -- 172.21.15.43:0/3055800682 wait complete. 2024-04-25T10:35:52.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.990+0000 7f68915ed740 1 -- start start 2024-04-25T10:35:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.990+0000 7f68915ed740 1 Processor -- start 2024-04-25T10:35:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.991+0000 7f68915ed740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b0c4d0 0x55c510f54c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.991+0000 7f68915ed740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c510b146b0 0x55c510f55180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.991+0000 7f68915ed740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b35390 0x55c510ef8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.991+0000 7f68915ed740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f68841acc80 con 0x55c510b0c4d0 2024-04-25T10:35:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.991+0000 7f68915ed740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68841ae0f0 con 0x55c510b35390 2024-04-25T10:35:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.991+0000 7f68915ed740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f68841ad350 con 0x55c510b146b0 2024-04-25T10:35:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.991+0000 7f68915ed740 10 client.0 _ll_register_callbacks cb 0x7f687800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:53.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.991+0000 7f6888cd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b0c4d0 0x55c510f54c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.991+0000 7f6888cd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b0c4d0 0x55c510f54c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42990/0 (socket says 172.21.15.43:42990) 2024-04-25T10:35:53.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.991+0000 7f6888cd5640 1 -- 172.21.15.43:0/2088688333 learned_addr learned my addr 172.21.15.43:0/2088688333 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:53.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.991+0000 7f68837fe640 1 --2- 172.21.15.43:0/2088688333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c510b146b0 0x55c510f55180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.991+0000 7f6888cd5640 1 --2- 172.21.15.43:0/2088688333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b35390 0x55c510ef8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.991+0000 7f6888cd5640 1 -- 172.21.15.43:0/2088688333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c510b146b0 msgr2=0x55c510f55180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:53.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.991+0000 7f6888cd5640 1 --2- 172.21.15.43:0/2088688333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c510b146b0 0x55c510f55180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.991+0000 7f6888cd5640 1 -- 172.21.15.43:0/2088688333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b35390 msgr2=0x55c510ef8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:53.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.991+0000 7f6888cd5640 1 --2- 172.21.15.43:0/2088688333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b35390 0x55c510ef8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.991+0000 7f6888cd5640 1 -- 172.21.15.43:0/2088688333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c510ef89d0 con 0x55c510b0c4d0 2024-04-25T10:35:53.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.991+0000 7f6888cd5640 1 --2- 172.21.15.43:0/2088688333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b0c4d0 0x55c510f54c40 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f687c009bf0 tx=0x7f687c004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb2375951daa3c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:53.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.991+0000 7f68637fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:53.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.991+0000 7f68637fe640 1 -- 172.21.15.43:0/2088688333 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c066060 con 0x55c510b0c4d0 2024-04-25T10:35:53.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.992+0000 7f68637fe640 1 -- 172.21.15.43:0/2088688333 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f687c004d40 con 0x55c510b0c4d0 2024-04-25T10:35:53.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.992+0000 7f68915ed740 1 -- 172.21.15.43:0/2088688333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f687c036070 con 0x55c510b0c4d0 2024-04-25T10:35:53.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.992+0000 7f68637fe640 1 -- 172.21.15.43:0/2088688333 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f687c031450 con 0x55c510b0c4d0 2024-04-25T10:35:53.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.992+0000 7f68915ed740 10 client.16413 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:53.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.992+0000 7f68915ed740 1 -- 172.21.15.43:0/2088688333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c510f5d3d0 con 0x55c510b0c4d0 2024-04-25T10:35:53.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.992+0000 7f68637fe640 1 -- 172.21.15.43:0/2088688333 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f687c036070 con 0x55c510b0c4d0 2024-04-25T10:35:53.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.992+0000 7f68637fe640 1 -- 172.21.15.43:0/2088688333 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f687c02f660 con 0x55c510b0c4d0 2024-04-25T10:35:53.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.992+0000 7f68915ed740 10 client.16413 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:53.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.992+0000 7f68915ed740 10 client.16413 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:35:53.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.992+0000 7f68915ed740 20 client.16413 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:53.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.992+0000 7f68915ed740 1 --2- 172.21.15.43:0/2088688333 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c510ac93a0 0x55c510f5f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.992+0000 7f68915ed740 4 client.16413 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:35:53.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.992+0000 7f68915ed740 1 -- 172.21.15.43:0/2088688333 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c510e43800 con 0x55c510ac93a0 2024-04-25T10:35:53.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.993+0000 7f68837fe640 1 --2- 172.21.15.43:0/2088688333 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c510ac93a0 0x55c510f5f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.993+0000 7f68837fe640 1 --2- 172.21.15.43:0/2088688333 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c510ac93a0 0x55c510f5f600 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf6a3f0c32fc70a6 server_cookie=ed6e3cb40af16db6 in_seq=0 out_seq=0 2024-04-25T10:35:53.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.993+0000 7f68637fe640 10 client.16413 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:35:53.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.993+0000 7f68637fe640 1 -- 172.21.15.43:0/2088688333 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c510e43800 con 0x55c510ac93a0 2024-04-25T10:35:53.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.993+0000 7f68637fe640 10 client.16413 handle_command_reply: tid=0 2024-04-25T10:35:53.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.993+0000 7f68915ed740 1 client.16413 shutdown 2024-04-25T10:35:53.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.993+0000 7f68915ed740 1 -- 172.21.15.43:0/2088688333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b0c4d0 msgr2=0x55c510f54c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.993+0000 7f68915ed740 1 --2- 172.21.15.43:0/2088688333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b0c4d0 0x55c510f54c40 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f687c009bf0 tx=0x7f687c004180 comp rx=0 tx=0).stop 2024-04-25T10:35:53.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.993+0000 7f68915ed740 1 -- 172.21.15.43:0/2088688333 shutdown_connections 2024-04-25T10:35:53.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.993+0000 7f68915ed740 1 --2- 172.21.15.43:0/2088688333 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c510ac93a0 0x55c510f5f600 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.993+0000 7f68915ed740 1 --2- 172.21.15.43:0/2088688333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c510b35390 0x55c510ef8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.993+0000 7f68915ed740 1 --2- 172.21.15.43:0/2088688333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c510b146b0 0x55c510f55180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.993+0000 7f68637fe640 0 client.16413 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:35:53.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.993+0000 7f68915ed740 1 --2- 172.21.15.43:0/2088688333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c510b0c4d0 0x55c510f54c40 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.993+0000 7f68915ed740 1 -- 172.21.15.43:0/2088688333 >> 172.21.15.43:0/2088688333 conn(0x55c510b294e0 msgr2=0x55c510ed65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:53.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.993+0000 7f68915ed740 1 -- 172.21.15.43:0/2088688333 shutdown_connections 2024-04-25T10:35:53.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.993+0000 7f68915ed740 1 -- 172.21.15.43:0/2088688333 wait complete. 2024-04-25T10:35:53.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.994+0000 7f68915ed740 20 client.16413 trim_cache size 0 max 16384 2024-04-25T10:35:53.004 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:35:53.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.994+0000 7f688a738640 1 -- 172.21.15.43:0/3226130771 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f686c0afc60 msgr2=0x7f686c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.994+0000 7f688a738640 1 --2- 172.21.15.43:0/3226130771 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f686c0afc60 0x7f686c0b2170 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f68740dc390 tx=0x7f6874007f50 comp rx=0 tx=0).stop 2024-04-25T10:35:53.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.994+0000 7f688a738640 1 -- 172.21.15.43:0/3226130771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6884102af0 msgr2=0x7f68841995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.994+0000 7f688a738640 1 --2- 172.21.15.43:0/3226130771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6884102af0 0x7f68841995a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f687400c9a0 tx=0x7f687400ce60 comp rx=0 tx=0).stop 2024-04-25T10:35:53.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.994+0000 7f688a738640 1 -- 172.21.15.43:0/3226130771 shutdown_connections 2024-04-25T10:35:53.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.994+0000 7f688a738640 1 --2- 172.21.15.43:0/3226130771 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f686c0afc60 0x7f686c0b2170 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.994+0000 7f688a738640 1 --2- 172.21.15.43:0/3226130771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6884108a20 0x7f6884199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.994+0000 7f688a738640 1 --2- 172.21.15.43:0/3226130771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f688410d380 0x7f688419ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.994+0000 7f688a738640 1 --2- 172.21.15.43:0/3226130771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6884102af0 0x7f68841995a0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.994+0000 7f688a738640 1 -- 172.21.15.43:0/3226130771 >> 172.21.15.43:0/3226130771 conn(0x7f6884078100 msgr2=0x7f68841040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:53.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.994+0000 7f688a738640 1 -- 172.21.15.43:0/3226130771 shutdown_connections 2024-04-25T10:35:53.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:52.994+0000 7f688a738640 1 -- 172.21.15.43:0/3226130771 wait complete. 2024-04-25T10:35:53.008 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:35:53.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.119+0000 7f19df984640 1 -- 172.21.15.43:0/1779426951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d8104260 msgr2=0x7f19d8110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.119+0000 7f19df984640 1 --2- 172.21.15.43:0/1779426951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d8104260 0x7f19d8110630 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009ca0 tx=0x7f19d002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.119+0000 7f19df984640 1 -- 172.21.15.43:0/1779426951 shutdown_connections 2024-04-25T10:35:53.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.119+0000 7f19df984640 1 --2- 172.21.15.43:0/1779426951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19d8109fb0 0x7f19d8103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.119+0000 7f19df984640 1 --2- 172.21.15.43:0/1779426951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d8104260 0x7f19d8110630 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.119+0000 7f19df984640 1 --2- 172.21.15.43:0/1779426951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19d8109b10 0x7f19d81036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.119+0000 7f19df984640 1 -- 172.21.15.43:0/1779426951 >> 172.21.15.43:0/1779426951 conn(0x7f19d80ff1d0 msgr2=0x7f19d8101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:53.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.119+0000 7f19df984640 1 -- 172.21.15.43:0/1779426951 shutdown_connections 2024-04-25T10:35:53.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.119+0000 7f19df984640 1 -- 172.21.15.43:0/1779426951 wait complete. 2024-04-25T10:35:53.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.120+0000 7f19df984640 1 Processor -- start 2024-04-25T10:35:53.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.120+0000 7f19df984640 1 -- start start 2024-04-25T10:35:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.120+0000 7f19df984640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19d8104260 0x7f19d8075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.120+0000 7f19df984640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19d8109b10 0x7f19d80764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.120+0000 7f19df984640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d8109fb0 0x7f19d8076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.120+0000 7f19df984640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f19d8113d50 con 0x7f19d8109fb0 2024-04-25T10:35:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.120+0000 7f19df984640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f19d8113bd0 con 0x7f19d8109b10 2024-04-25T10:35:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.120+0000 7f19df984640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f19d8113ed0 con 0x7f19d8104260 2024-04-25T10:35:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.120+0000 7f19ddf21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d8109fb0 0x7f19d8076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.120+0000 7f19ddf21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d8109fb0 0x7f19d8076a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43018/0 (socket says 172.21.15.43:43018) 2024-04-25T10:35:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.121+0000 7f19ddf21640 1 -- 172.21.15.43:0/641669777 learned_addr learned my addr 172.21.15.43:0/641669777 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.121+0000 7f19dd720640 1 --2- 172.21.15.43:0/641669777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19d8104260 0x7f19d8075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.121+0000 7f19dcf1f640 1 --2- 172.21.15.43:0/641669777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19d8109b10 0x7f19d80764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.121+0000 7f19ddf21640 1 -- 172.21.15.43:0/641669777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19d8104260 msgr2=0x7f19d8075fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.121+0000 7f19ddf21640 1 --2- 172.21.15.43:0/641669777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19d8104260 0x7f19d8075fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.121+0000 7f19ddf21640 1 -- 172.21.15.43:0/641669777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19d8109b10 msgr2=0x7f19d80764f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.121+0000 7f19ddf21640 1 --2- 172.21.15.43:0/641669777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19d8109b10 0x7f19d80764f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.121+0000 7f19ddf21640 1 -- 172.21.15.43:0/641669777 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d81a8940 con 0x7f19d8109fb0 2024-04-25T10:35:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.121+0000 7f19dcf1f640 1 --2- 172.21.15.43:0/641669777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19d8109b10 0x7f19d80764f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.121+0000 7f19ddf21640 1 --2- 172.21.15.43:0/641669777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d8109fb0 0x7f19d8076a30 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f19d0031a20 tx=0x7f19d0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.121+0000 7f19ce7fc640 1 -- 172.21.15.43:0/641669777 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0003c70 con 0x7f19d8109fb0 2024-04-25T10:35:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.121+0000 7f19ce7fc640 1 -- 172.21.15.43:0/641669777 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19d0003e10 con 0x7f19d8109fb0 2024-04-25T10:35:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.121+0000 7f19df984640 1 -- 172.21.15.43:0/641669777 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d81a8bd0 con 0x7f19d8109fb0 2024-04-25T10:35:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.121+0000 7f19ce7fc640 1 -- 172.21.15.43:0/641669777 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d002fcb0 con 0x7f19d8109fb0 2024-04-25T10:35:53.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.121+0000 7f19df984640 1 -- 172.21.15.43:0/641669777 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d81a9120 con 0x7f19d8109fb0 2024-04-25T10:35:53.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.124+0000 7f19e6832740 1 Processor -- start 2024-04-25T10:35:53.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.124+0000 7f19ce7fc640 1 -- 172.21.15.43:0/641669777 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f19d0004040 con 0x7f19d8109fb0 2024-04-25T10:35:53.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.124+0000 7f19e6832740 1 -- start start 2024-04-25T10:35:53.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19e6832740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85e1fd390 0x55f85e59e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19e6832740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1d44d0 0x55f85e59ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19e6832740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1dc6b0 0x55f85e5ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19e6832740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f85e503d80 con 0x55f85e1d44d0 2024-04-25T10:35:53.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19e6832740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f85e4bda20 con 0x55f85e1dc6b0 2024-04-25T10:35:53.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19e6832740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f85e4ffd40 con 0x55f85e1fd390 2024-04-25T10:35:53.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19dd720640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1d44d0 0x55f85e59ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19dd720640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1d44d0 0x55f85e59ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43034/0 (socket says 172.21.15.43:43034) 2024-04-25T10:35:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19dd720640 1 -- 172.21.15.43:0/630324300 learned_addr learned my addr 172.21.15.43:0/630324300 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19dcf1f640 1 --2- 172.21.15.43:0/630324300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85e1fd390 0x55f85e59e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19dcf1f640 1 --2- 172.21.15.43:0/630324300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1dc6b0 0x55f85e5ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19ce7fc640 1 --2- 172.21.15.43:0/641669777 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f19c40afc60 0x7f19c40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19dd720640 1 -- 172.21.15.43:0/630324300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85e1fd390 msgr2=0x55f85e59e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19dd720640 1 --2- 172.21.15.43:0/630324300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85e1fd390 0x55f85e59e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19dd720640 1 -- 172.21.15.43:0/630324300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1dc6b0 msgr2=0x55f85e5ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19dd720640 1 --2- 172.21.15.43:0/630324300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1dc6b0 0x55f85e5ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19dd720640 1 -- 172.21.15.43:0/630324300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f85e4c8c30 con 0x55f85e1d44d0 2024-04-25T10:35:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19dcf1f640 1 --2- 172.21.15.43:0/630324300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1dc6b0 0x55f85e5ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:35:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19ddf21640 1 --2- 172.21.15.43:0/641669777 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f19c40afc60 0x7f19c40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19dd720640 1 --2- 172.21.15.43:0/630324300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1d44d0 0x55f85e59ed50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f19d400aca0 tx=0x7f19d400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b817be1b5dc0e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.125+0000 7f19ce7fc640 1 -- 172.21.15.43:0/641669777 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f19d01005d0 con 0x7f19d8109fb0 2024-04-25T10:35:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.126+0000 7f19c37fe640 1 -- 172.21.15.43:0/630324300 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d401d070 con 0x55f85e1d44d0 2024-04-25T10:35:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.126+0000 7f19c37fe640 1 -- 172.21.15.43:0/630324300 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19d4018af0 con 0x55f85e1d44d0 2024-04-25T10:35:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.126+0000 7f19c37fe640 1 -- 172.21.15.43:0/630324300 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4010b80 con 0x55f85e1d44d0 2024-04-25T10:35:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.127+0000 7f19e6832740 1 -- 172.21.15.43:0/630324300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1d44d0 msgr2=0x55f85e59ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.127+0000 7f19ddf21640 1 --2- 172.21.15.43:0/641669777 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f19c40afc60 0x7f19c40b2170 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f19d00fdf10 tx=0x7f19d0097440 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.127+0000 7f19e6832740 1 --2- 172.21.15.43:0/630324300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1d44d0 0x55f85e59ed50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f19d400aca0 tx=0x7f19d400f790 comp rx=0 tx=0).stop 2024-04-25T10:35:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.127+0000 7f19e6832740 1 -- 172.21.15.43:0/630324300 shutdown_connections 2024-04-25T10:35:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.127+0000 7f19e6832740 1 --2- 172.21.15.43:0/630324300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1dc6b0 0x55f85e5ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.127+0000 7f19e6832740 1 --2- 172.21.15.43:0/630324300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1d44d0 0x55f85e59ed50 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.127+0000 7f19e6832740 1 --2- 172.21.15.43:0/630324300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85e1fd390 0x55f85e59e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.127+0000 7f19e6832740 1 -- 172.21.15.43:0/630324300 >> 172.21.15.43:0/630324300 conn(0x55f85e1f14e0 msgr2=0x55f85e59afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.127+0000 7f19e6832740 1 -- 172.21.15.43:0/630324300 shutdown_connections 2024-04-25T10:35:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.127+0000 7f19e6832740 1 -- 172.21.15.43:0/630324300 wait complete. 2024-04-25T10:35:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.127+0000 7f19e6832740 1 -- start start 2024-04-25T10:35:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.127+0000 7f19e6832740 1 Processor -- start 2024-04-25T10:35:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19e6832740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1d44d0 0x55f85e5a92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19e6832740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1dc6b0 0x55f85e5a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19e6832740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85e1fd390 0x55f85e5b5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19e6832740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f85e5abd90 con 0x55f85e1d44d0 2024-04-25T10:35:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19e6832740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f85e48adc0 con 0x55f85e1dc6b0 2024-04-25T10:35:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19e6832740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f85e5abf10 con 0x55f85e1fd390 2024-04-25T10:35:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19e6832740 10 client.0 _ll_register_callbacks cb 0x55f85e4a6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19dcf1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1d44d0 0x55f85e5a92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19dcf1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1d44d0 0x55f85e5a92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43040/0 (socket says 172.21.15.43:43040) 2024-04-25T10:35:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19dcf1f640 1 -- 172.21.15.43:0/3220412717 learned_addr learned my addr 172.21.15.43:0/3220412717 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19dd720640 1 --2- 172.21.15.43:0/3220412717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1dc6b0 0x55f85e5a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19dcf1f640 1 --2- 172.21.15.43:0/3220412717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85e1fd390 0x55f85e5b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19dcf1f640 1 -- 172.21.15.43:0/3220412717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85e1fd390 msgr2=0x55f85e5b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19dcf1f640 1 --2- 172.21.15.43:0/3220412717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85e1fd390 0x55f85e5b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19dcf1f640 1 -- 172.21.15.43:0/3220412717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1dc6b0 msgr2=0x55f85e5a97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19dcf1f640 1 --2- 172.21.15.43:0/3220412717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1dc6b0 0x55f85e5a97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19dcf1f640 1 -- 172.21.15.43:0/3220412717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f85e2153f0 con 0x55f85e1d44d0 2024-04-25T10:35:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19dd720640 1 --2- 172.21.15.43:0/3220412717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1dc6b0 0x55f85e5a97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19dcf1f640 1 --2- 172.21.15.43:0/3220412717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1d44d0 0x55f85e5a92b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f19d810a3e0 tx=0x7f19c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a726739afe6eebc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19c0ff9640 1 -- 172.21.15.43:0/3220412717 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8004030 con 0x55f85e1d44d0 2024-04-25T10:35:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19c0ff9640 1 -- 172.21.15.43:0/3220412717 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f19c80041d0 con 0x55f85e1d44d0 2024-04-25T10:35:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.128+0000 7f19e6832740 1 -- 172.21.15.43:0/3220412717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19d4018af0 con 0x55f85e1d44d0 2024-04-25T10:35:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.129+0000 7f19c0ff9640 1 -- 172.21.15.43:0/3220412717 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f19c80043d0 con 0x55f85e1d44d0 2024-04-25T10:35:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.129+0000 7f19e6832740 10 client.16425 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.129+0000 7f19e6832740 1 -- 172.21.15.43:0/3220412717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f85e2df7d0 con 0x55f85e1d44d0 2024-04-25T10:35:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.129+0000 7f19c0ff9640 1 -- 172.21.15.43:0/3220412717 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19d4018af0 con 0x55f85e1d44d0 2024-04-25T10:35:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.129+0000 7f19c0ff9640 1 -- 172.21.15.43:0/3220412717 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f19c801f570 con 0x55f85e1d44d0 2024-04-25T10:35:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.129+0000 7f19e6832740 10 client.16425 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.129+0000 7f19e6832740 10 client.16425 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:35:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.129+0000 7f19e6832740 20 client.16425 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.129+0000 7f19e6832740 1 --2- 172.21.15.43:0/3220412717 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55f85e1bbf60 0x55f85e5bb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.129+0000 7f19e6832740 4 client.16425 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:35:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.129+0000 7f19e6832740 1 -- 172.21.15.43:0/3220412717 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f85e511990 con 0x55f85e1bbf60 2024-04-25T10:35:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.129+0000 7f19dd720640 1 --2- 172.21.15.43:0/3220412717 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55f85e1bbf60 0x55f85e5bb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.130+0000 7f19c0ff9640 10 client.16425 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:35:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.130+0000 7f19dd720640 1 --2- 172.21.15.43:0/3220412717 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55f85e1bbf60 0x55f85e5bb710 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f09a6d6e92aa0a3 server_cookie=b57aa82c398629b5 in_seq=0 out_seq=0 2024-04-25T10:35:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.131+0000 7f19c0ff9640 1 -- 172.21.15.43:0/3220412717 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f85e511990 con 0x55f85e1bbf60 2024-04-25T10:35:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.131+0000 7f19c0ff9640 10 client.16425 handle_command_reply: tid=0 2024-04-25T10:35:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.131+0000 7f19e6832740 1 client.16425 shutdown 2024-04-25T10:35:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.131+0000 7f19e6832740 1 -- 172.21.15.43:0/3220412717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1d44d0 msgr2=0x55f85e5a92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.131+0000 7f19e6832740 1 --2- 172.21.15.43:0/3220412717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1d44d0 0x55f85e5a92b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f19d810a3e0 tx=0x7f19c80046c0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.131+0000 7f19e6832740 1 -- 172.21.15.43:0/3220412717 shutdown_connections 2024-04-25T10:35:53.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.131+0000 7f19e6832740 1 --2- 172.21.15.43:0/3220412717 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55f85e1bbf60 0x55f85e5bb710 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.131+0000 7f19e6832740 1 --2- 172.21.15.43:0/3220412717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1dc6b0 0x55f85e5a97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.131+0000 7f19e6832740 1 --2- 172.21.15.43:0/3220412717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85e1fd390 0x55f85e5b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.131+0000 7f19c0ff9640 0 client.16425 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:35:53.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.131+0000 7f19e6832740 1 --2- 172.21.15.43:0/3220412717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1d44d0 0x55f85e5a92b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.131+0000 7f19e6832740 1 -- 172.21.15.43:0/3220412717 >> 172.21.15.43:0/3220412717 conn(0x55f85e1f14e0 msgr2=0x55f85e5a0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:53.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.131+0000 7f19e6832740 1 -- 172.21.15.43:0/3220412717 shutdown_connections 2024-04-25T10:35:53.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.131+0000 7f19e6832740 1 -- 172.21.15.43:0/3220412717 wait complete. 2024-04-25T10:35:53.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.131+0000 7f19e6832740 20 client.16425 trim_cache size 0 max 16384 2024-04-25T10:35:53.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.145+0000 7f19e6832740 1 Processor -- start 2024-04-25T10:35:53.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.145+0000 7f19e6832740 1 -- start start 2024-04-25T10:35:53.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.145+0000 7f19e6832740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85e1bbf60 0x55f85e5a83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.145+0000 7f19e6832740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1d44d0 0x55f85e5b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.145+0000 7f19e6832740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1dc6b0 0x55f85e5a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.145+0000 7f19e6832740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f19d81b4210 con 0x55f85e1dc6b0 2024-04-25T10:35:53.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.145+0000 7f19e6832740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f19d81b1ae0 con 0x55f85e1d44d0 2024-04-25T10:35:53.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.145+0000 7f19e6832740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f19d81af930 con 0x55f85e1bbf60 2024-04-25T10:35:53.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.145+0000 7f19dcf1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1dc6b0 0x55f85e5a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.145+0000 7f19dcf1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1dc6b0 0x55f85e5a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43048/0 (socket says 172.21.15.43:43048) 2024-04-25T10:35:53.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.145+0000 7f19dcf1f640 1 -- 172.21.15.43:0/2830617140 learned_addr learned my addr 172.21.15.43:0/2830617140 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:53.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.145+0000 7f19dcf1f640 1 --2- 172.21.15.43:0/2830617140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85e1bbf60 0x55f85e5a83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.145+0000 7f19dd720640 1 --2- 172.21.15.43:0/2830617140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1d44d0 0x55f85e5b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.145+0000 7f19dcf1f640 1 -- 172.21.15.43:0/2830617140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85e1bbf60 msgr2=0x55f85e5a83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:53.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.146+0000 7f19dcf1f640 1 --2- 172.21.15.43:0/2830617140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85e1bbf60 0x55f85e5a83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.146+0000 7f19dcf1f640 1 -- 172.21.15.43:0/2830617140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1d44d0 msgr2=0x55f85e5b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:53.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.146+0000 7f19dcf1f640 1 --2- 172.21.15.43:0/2830617140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1d44d0 0x55f85e5b4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.146+0000 7f19dcf1f640 1 -- 172.21.15.43:0/2830617140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f85e5a3d60 con 0x55f85e1dc6b0 2024-04-25T10:35:53.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.146+0000 7f19dd720640 1 --2- 172.21.15.43:0/2830617140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1d44d0 0x55f85e5b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:35:53.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.146+0000 7f19dcf1f640 1 --2- 172.21.15.43:0/2830617140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1dc6b0 0x55f85e5a0b20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f19c800ecc0 tx=0x7f19c8011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8d52f59250493d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:53.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.146+0000 7f19c2ffd640 1 -- 172.21.15.43:0/2830617140 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c801fc70 con 0x55f85e1dc6b0 2024-04-25T10:35:53.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.146+0000 7f19c2ffd640 1 -- 172.21.15.43:0/2830617140 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19c80041c0 con 0x55f85e1dc6b0 2024-04-25T10:35:53.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.146+0000 7f19c2ffd640 1 -- 172.21.15.43:0/2830617140 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8003830 con 0x55f85e1dc6b0 2024-04-25T10:35:53.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.147+0000 7f19e6832740 1 -- 172.21.15.43:0/2830617140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1dc6b0 msgr2=0x55f85e5a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.147+0000 7f19e6832740 1 --2- 172.21.15.43:0/2830617140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1dc6b0 0x55f85e5a0b20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f19c800ecc0 tx=0x7f19c8011f50 comp rx=0 tx=0).stop 2024-04-25T10:35:53.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.147+0000 7f19e6832740 1 -- 172.21.15.43:0/2830617140 shutdown_connections 2024-04-25T10:35:53.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.147+0000 7f19e6832740 1 --2- 172.21.15.43:0/2830617140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1d44d0 0x55f85e5b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.147+0000 7f19e6832740 1 --2- 172.21.15.43:0/2830617140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1dc6b0 0x55f85e5a0b20 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.147+0000 7f19e6832740 1 --2- 172.21.15.43:0/2830617140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85e1bbf60 0x55f85e5a83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.147+0000 7f19e6832740 1 -- 172.21.15.43:0/2830617140 >> 172.21.15.43:0/2830617140 conn(0x55f85e1f14e0 msgr2=0x55f85e5a0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:53.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.147+0000 7f19e6832740 1 -- 172.21.15.43:0/2830617140 shutdown_connections 2024-04-25T10:35:53.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.147+0000 7f19e6832740 1 -- 172.21.15.43:0/2830617140 wait complete. 2024-04-25T10:35:53.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.147+0000 7f19e6832740 1 -- start start 2024-04-25T10:35:53.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.147+0000 7f19e6832740 1 Processor -- start 2024-04-25T10:35:53.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19e6832740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1bbf60 0x55f85e5ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19e6832740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1d44d0 0x55f85e5acb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19e6832740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85e1dc6b0 0x55f85e61ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19e6832740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f19d81affa0 con 0x55f85e1bbf60 2024-04-25T10:35:53.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19e6832740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f19d81b1410 con 0x55f85e1d44d0 2024-04-25T10:35:53.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19e6832740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f19d81b0670 con 0x55f85e1dc6b0 2024-04-25T10:35:53.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19e6832740 10 client.0 _ll_register_callbacks cb 0x7f19c8011b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19dcf1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1bbf60 0x55f85e5ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19dcf1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1bbf60 0x55f85e5ac5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43050/0 (socket says 172.21.15.43:43050) 2024-04-25T10:35:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19dcf1f640 1 -- 172.21.15.43:0/3750373271 learned_addr learned my addr 172.21.15.43:0/3750373271 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19dd720640 1 --2- 172.21.15.43:0/3750373271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1d44d0 0x55f85e5acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19dcf1f640 1 -- 172.21.15.43:0/3750373271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85e1dc6b0 msgr2=0x55f85e61ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19dcf1f640 1 --2- 172.21.15.43:0/3750373271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85e1dc6b0 0x55f85e61ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19dcf1f640 1 -- 172.21.15.43:0/3750373271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1d44d0 msgr2=0x55f85e5acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19dcf1f640 1 --2- 172.21.15.43:0/3750373271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1d44d0 0x55f85e5acb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19dcf1f640 1 -- 172.21.15.43:0/3750373271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f85e61bff0 con 0x55f85e1bbf60 2024-04-25T10:35:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19dcf1f640 1 --2- 172.21.15.43:0/3750373271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1bbf60 0x55f85e5ac5c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f19c800c010 tx=0x7f19c800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5319647e03c4e25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19c0ff9640 1 -- 172.21.15.43:0/3750373271 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8003fd0 con 0x55f85e1bbf60 2024-04-25T10:35:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19c0ff9640 1 -- 172.21.15.43:0/3750373271 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f19c80035f0 con 0x55f85e1bbf60 2024-04-25T10:35:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.148+0000 7f19e6832740 1 -- 172.21.15.43:0/3750373271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19c80041c0 con 0x55f85e1bbf60 2024-04-25T10:35:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.149+0000 7f19c0ff9640 1 -- 172.21.15.43:0/3750373271 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f19c80037f0 con 0x55f85e1bbf60 2024-04-25T10:35:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.149+0000 7f19e6832740 10 client.16431 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.149+0000 7f19e6832740 1 -- 172.21.15.43:0/3750373271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f85e623260 con 0x55f85e1bbf60 2024-04-25T10:35:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.149+0000 7f19c0ff9640 1 -- 172.21.15.43:0/3750373271 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19c80041c0 con 0x55f85e1bbf60 2024-04-25T10:35:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.149+0000 7f19c0ff9640 1 -- 172.21.15.43:0/3750373271 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f19c8035440 con 0x55f85e1bbf60 2024-04-25T10:35:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.149+0000 7f19e6832740 10 client.16431 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.149+0000 7f19e6832740 10 client.16431 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:35:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.149+0000 7f19e6832740 20 client.16431 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.149+0000 7f19e6832740 1 --2- 172.21.15.43:0/3750373271 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55f85e1fd390 0x55f85e625930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.149+0000 7f19e6832740 4 client.16431 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:35:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.149+0000 7f19e6832740 1 -- 172.21.15.43:0/3750373271 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f85e50b800 con 0x55f85e1fd390 2024-04-25T10:35:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.149+0000 7f19dd720640 1 --2- 172.21.15.43:0/3750373271 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55f85e1fd390 0x55f85e625930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.150+0000 7f19dd720640 1 --2- 172.21.15.43:0/3750373271 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55f85e1fd390 0x55f85e625930 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f976a15450dcb1f3 server_cookie=b1c50fa939fd0709 in_seq=0 out_seq=0 2024-04-25T10:35:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.150+0000 7f19c0ff9640 10 client.16431 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:35:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.150+0000 7f19c0ff9640 1 -- 172.21.15.43:0/3750373271 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f85e50b800 con 0x55f85e1fd390 2024-04-25T10:35:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.150+0000 7f19c0ff9640 10 client.16431 handle_command_reply: tid=0 2024-04-25T10:35:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.150+0000 7f19e6832740 1 client.16431 shutdown 2024-04-25T10:35:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.150+0000 7f19e6832740 1 -- 172.21.15.43:0/3750373271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1bbf60 msgr2=0x55f85e5ac5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.150+0000 7f19e6832740 1 --2- 172.21.15.43:0/3750373271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1bbf60 0x55f85e5ac5c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f19c800c010 tx=0x7f19c800b910 comp rx=0 tx=0).stop 2024-04-25T10:35:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.150+0000 7f19e6832740 1 -- 172.21.15.43:0/3750373271 shutdown_connections 2024-04-25T10:35:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.150+0000 7f19e6832740 1 --2- 172.21.15.43:0/3750373271 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55f85e1fd390 0x55f85e625930 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.150+0000 7f19e6832740 1 --2- 172.21.15.43:0/3750373271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f85e1d44d0 0x55f85e5acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.150+0000 7f19e6832740 1 --2- 172.21.15.43:0/3750373271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85e1dc6b0 0x55f85e61ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.150+0000 7f19c0ff9640 0 client.16431 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:35:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.150+0000 7f19e6832740 1 --2- 172.21.15.43:0/3750373271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85e1bbf60 0x55f85e5ac5c0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.150+0000 7f19e6832740 1 -- 172.21.15.43:0/3750373271 >> 172.21.15.43:0/3750373271 conn(0x55f85e1f14e0 msgr2=0x55f85e5a0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.150+0000 7f19e6832740 1 -- 172.21.15.43:0/3750373271 shutdown_connections 2024-04-25T10:35:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.150+0000 7f19e6832740 1 -- 172.21.15.43:0/3750373271 wait complete. 2024-04-25T10:35:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.150+0000 7f19e6832740 20 client.16431 trim_cache size 0 max 16384 2024-04-25T10:35:53.160 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:35:53.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.151+0000 7f19df984640 1 -- 172.21.15.43:0/641669777 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f19c40afc60 msgr2=0x7f19c40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.151+0000 7f19df984640 1 --2- 172.21.15.43:0/641669777 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f19c40afc60 0x7f19c40b2170 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f19d00fdf10 tx=0x7f19d0097440 comp rx=0 tx=0).stop 2024-04-25T10:35:53.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.151+0000 7f19df984640 1 -- 172.21.15.43:0/641669777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d8109fb0 msgr2=0x7f19d8076a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.151+0000 7f19df984640 1 --2- 172.21.15.43:0/641669777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d8109fb0 0x7f19d8076a30 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f19d0031a20 tx=0x7f19d0031a50 comp rx=0 tx=0).stop 2024-04-25T10:35:53.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.151+0000 7f19df984640 1 -- 172.21.15.43:0/641669777 shutdown_connections 2024-04-25T10:35:53.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.151+0000 7f19df984640 1 --2- 172.21.15.43:0/641669777 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f19c40afc60 0x7f19c40b2170 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.151+0000 7f19df984640 1 --2- 172.21.15.43:0/641669777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19d8109b10 0x7f19d80764f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.151+0000 7f19df984640 1 --2- 172.21.15.43:0/641669777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d8109fb0 0x7f19d8076a30 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.151+0000 7f19df984640 1 --2- 172.21.15.43:0/641669777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19d8104260 0x7f19d8075fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.151+0000 7f19df984640 1 -- 172.21.15.43:0/641669777 >> 172.21.15.43:0/641669777 conn(0x7f19d80ff1d0 msgr2=0x7f19d8106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:53.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.151+0000 7f19df984640 1 -- 172.21.15.43:0/641669777 shutdown_connections 2024-04-25T10:35:53.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.151+0000 7f19df984640 1 -- 172.21.15.43:0/641669777 wait complete. 2024-04-25T10:35:53.165 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:35:53.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.276+0000 7f20ff51f640 1 -- 172.21.15.43:0/2713232319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f810db20 msgr2=0x7f20f810ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.276+0000 7f20ff51f640 1 --2- 172.21.15.43:0/2713232319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f810db20 0x7f20f810ff40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009900 tx=0x7f20e802a410 comp rx=0 tx=0).stop 2024-04-25T10:35:53.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.276+0000 7f20ff51f640 1 -- 172.21.15.43:0/2713232319 shutdown_connections 2024-04-25T10:35:53.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.276+0000 7f20ff51f640 1 --2- 172.21.15.43:0/2713232319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20f8110480 0x7f20f81128a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.276+0000 7f20ff51f640 1 --2- 172.21.15.43:0/2713232319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f810db20 0x7f20f810ff40 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.276+0000 7f20ff51f640 1 --2- 172.21.15.43:0/2713232319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20f81028e0 0x7f20f8102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.276+0000 7f20ff51f640 1 -- 172.21.15.43:0/2713232319 >> 172.21.15.43:0/2713232319 conn(0x7f20f8078110 msgr2=0x7f20f8078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:53.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.276+0000 7f20ff51f640 1 -- 172.21.15.43:0/2713232319 shutdown_connections 2024-04-25T10:35:53.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.276+0000 7f20ff51f640 1 -- 172.21.15.43:0/2713232319 wait complete. 2024-04-25T10:35:53.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.277+0000 7f20ff51f640 1 Processor -- start 2024-04-25T10:35:53.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.277+0000 7f20ff51f640 1 -- start start 2024-04-25T10:35:53.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.277+0000 7f20ff51f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f81028e0 0x7f20f81a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.277+0000 7f20ff51f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20f810db20 0x7f20f81a2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.277+0000 7f20ff51f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20f8110480 0x7f20f819c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.277+0000 7f20ff51f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20f8115f40 con 0x7f20f81028e0 2024-04-25T10:35:53.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.277+0000 7f20ff51f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f20f8115dc0 con 0x7f20f8110480 2024-04-25T10:35:53.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.277+0000 7f20ff51f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f20f81160c0 con 0x7f20f810db20 2024-04-25T10:35:53.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.277+0000 7f20fd2bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f81028e0 0x7f20f81a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.277+0000 7f20fd2bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f81028e0 0x7f20f81a2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43074/0 (socket says 172.21.15.43:43074) 2024-04-25T10:35:53.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.277+0000 7f20fd2bb640 1 -- 172.21.15.43:0/1694161786 learned_addr learned my addr 172.21.15.43:0/1694161786 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:53.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.277+0000 7f20fcaba640 1 --2- 172.21.15.43:0/1694161786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20f810db20 0x7f20f81a2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.277+0000 7f20fdabc640 1 --2- 172.21.15.43:0/1694161786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20f8110480 0x7f20f819c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.277+0000 7f20fd2bb640 1 -- 172.21.15.43:0/1694161786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20f810db20 msgr2=0x7f20f81a2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.277+0000 7f20fd2bb640 1 --2- 172.21.15.43:0/1694161786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20f810db20 0x7f20f81a2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.277+0000 7f20fd2bb640 1 -- 172.21.15.43:0/1694161786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20f8110480 msgr2=0x7f20f819c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.277+0000 7f20fd2bb640 1 --2- 172.21.15.43:0/1694161786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20f8110480 0x7f20f819c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.277+0000 7f20fd2bb640 1 -- 172.21.15.43:0/1694161786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f819ceb0 con 0x7f20f81028e0 2024-04-25T10:35:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.278+0000 7f20fd2bb640 1 --2- 172.21.15.43:0/1694161786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f81028e0 0x7f20f81a2600 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f20f400c890 tx=0x7f20f400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.278+0000 7f20ee7fc640 1 -- 172.21.15.43:0/1694161786 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f4004220 con 0x7f20f81028e0 2024-04-25T10:35:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.278+0000 7f20ff51f640 1 -- 172.21.15.43:0/1694161786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f819d140 con 0x7f20f81028e0 2024-04-25T10:35:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.278+0000 7f20ee7fc640 1 -- 172.21.15.43:0/1694161786 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f40043c0 con 0x7f20f81028e0 2024-04-25T10:35:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.278+0000 7f20ee7fc640 1 -- 172.21.15.43:0/1694161786 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f4004a00 con 0x7f20f81028e0 2024-04-25T10:35:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.278+0000 7f20ff51f640 1 -- 172.21.15.43:0/1694161786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f81a93a0 con 0x7f20f81028e0 2024-04-25T10:35:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.281+0000 7f21063d4740 1 Processor -- start 2024-04-25T10:35:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.281+0000 7f21063d4740 1 -- start start 2024-04-25T10:35:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.281+0000 7f20ee7fc640 1 -- 172.21.15.43:0/1694161786 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f20f40079b0 con 0x7f20f81028e0 2024-04-25T10:35:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.281+0000 7f21063d4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 0x556f85eb2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.281+0000 7f21063d4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f85ae84d0 0x556f85eb2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.281+0000 7f21063d4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f85af06b0 0x556f85ebf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.281+0000 7f21063d4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556f85e17d80 con 0x556f85b11390 2024-04-25T10:35:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.281+0000 7f21063d4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556f85dd1a20 con 0x556f85ae84d0 2024-04-25T10:35:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.281+0000 7f21063d4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556f85e13d40 con 0x556f85af06b0 2024-04-25T10:35:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.281+0000 7f20fdabc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 0x556f85eb2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.281+0000 7f20fdabc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 0x556f85eb2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43082/0 (socket says 172.21.15.43:43082) 2024-04-25T10:35:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.281+0000 7f20fdabc640 1 -- 172.21.15.43:0/1723841355 learned_addr learned my addr 172.21.15.43:0/1723841355 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.281+0000 7f20fd2bb640 1 --2- 172.21.15.43:0/1723841355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f85ae84d0 0x556f85eb2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.281+0000 7f20fcaba640 1 --2- 172.21.15.43:0/1723841355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f85af06b0 0x556f85ebf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.281+0000 7f20fdabc640 1 -- 172.21.15.43:0/1723841355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f85af06b0 msgr2=0x556f85ebf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.281+0000 7f20fdabc640 1 --2- 172.21.15.43:0/1723841355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f85af06b0 0x556f85ebf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.281+0000 7f20fdabc640 1 -- 172.21.15.43:0/1723841355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f85ae84d0 msgr2=0x556f85eb2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.281+0000 7f20fdabc640 1 --2- 172.21.15.43:0/1723841355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f85ae84d0 0x556f85eb2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.281+0000 7f20fdabc640 1 -- 172.21.15.43:0/1723841355 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f85ddcc30 con 0x556f85b11390 2024-04-25T10:35:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.282+0000 7f20fdabc640 1 --2- 172.21.15.43:0/1723841355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 0x556f85eb2810 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f20f0008d40 tx=0x7f20f000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94805a909b76906 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.282+0000 7f20ee7fc640 1 --2- 172.21.15.43:0/1694161786 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f20e40afc60 0x7f20e40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.282+0000 7f20e37fe640 1 -- 172.21.15.43:0/1723841355 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f0014070 con 0x556f85b11390 2024-04-25T10:35:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.282+0000 7f20e37fe640 1 -- 172.21.15.43:0/1723841355 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f0003b70 con 0x556f85b11390 2024-04-25T10:35:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.282+0000 7f20e37fe640 1 -- 172.21.15.43:0/1723841355 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f0010670 con 0x556f85b11390 2024-04-25T10:35:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.282+0000 7f20fd2bb640 1 --2- 172.21.15.43:0/1694161786 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f20e40afc60 0x7f20e40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.282+0000 7f20ee7fc640 1 -- 172.21.15.43:0/1694161786 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f20f40deed0 con 0x7f20f81028e0 2024-04-25T10:35:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.283+0000 7f21063d4740 1 -- 172.21.15.43:0/1723841355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 msgr2=0x556f85eb2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.283+0000 7f21063d4740 1 --2- 172.21.15.43:0/1723841355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 0x556f85eb2810 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f20f0008d40 tx=0x7f20f000e720 comp rx=0 tx=0).stop 2024-04-25T10:35:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.283+0000 7f20fd2bb640 1 --2- 172.21.15.43:0/1694161786 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f20e40afc60 0x7f20e40b2170 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x556f85eaff50 tx=0x7f20f4000bc0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.283+0000 7f21063d4740 1 -- 172.21.15.43:0/1723841355 shutdown_connections 2024-04-25T10:35:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.283+0000 7f21063d4740 1 --2- 172.21.15.43:0/1723841355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f85ae84d0 0x556f85eb2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.283+0000 7f21063d4740 1 --2- 172.21.15.43:0/1723841355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f85af06b0 0x556f85ebf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.283+0000 7f21063d4740 1 --2- 172.21.15.43:0/1723841355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 0x556f85eb2810 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.283+0000 7f21063d4740 1 -- 172.21.15.43:0/1723841355 >> 172.21.15.43:0/1723841355 conn(0x556f85b054e0 msgr2=0x556f85eaefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.283+0000 7f21063d4740 1 -- 172.21.15.43:0/1723841355 shutdown_connections 2024-04-25T10:35:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.283+0000 7f21063d4740 1 -- 172.21.15.43:0/1723841355 wait complete. 2024-04-25T10:35:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.283+0000 7f21063d4740 1 -- start start 2024-04-25T10:35:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.283+0000 7f21063d4740 1 Processor -- start 2024-04-25T10:35:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.284+0000 7f21063d4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f85ae84d0 0x556f85ebd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.284+0000 7f21063d4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f85af06b0 0x556f85ebd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.284+0000 7f21063d4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 0x556f85ec9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.284+0000 7f21063d4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556f85ebfd90 con 0x556f85b11390 2024-04-25T10:35:53.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.284+0000 7f21063d4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556f85d9edc0 con 0x556f85af06b0 2024-04-25T10:35:53.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.284+0000 7f21063d4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556f85ebff10 con 0x556f85ae84d0 2024-04-25T10:35:53.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.284+0000 7f21063d4740 10 client.0 _ll_register_callbacks cb 0x556f85dba600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:53.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.284+0000 7f20fdabc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 0x556f85ec9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.284+0000 7f20fdabc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 0x556f85ec9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43098/0 (socket says 172.21.15.43:43098) 2024-04-25T10:35:53.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.284+0000 7f20fdabc640 1 -- 172.21.15.43:0/241652282 learned_addr learned my addr 172.21.15.43:0/241652282 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:53.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.284+0000 7f20fcaba640 1 --2- 172.21.15.43:0/241652282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f85af06b0 0x556f85ebd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.284+0000 7f20fdabc640 1 --2- 172.21.15.43:0/241652282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f85ae84d0 0x556f85ebd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.284+0000 7f20fdabc640 1 -- 172.21.15.43:0/241652282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f85ae84d0 msgr2=0x556f85ebd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:53.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.284+0000 7f20fdabc640 1 --2- 172.21.15.43:0/241652282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f85ae84d0 0x556f85ebd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.284+0000 7f20fdabc640 1 -- 172.21.15.43:0/241652282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f85af06b0 msgr2=0x556f85ebd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.284+0000 7f20fdabc640 1 --2- 172.21.15.43:0/241652282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f85af06b0 0x556f85ebd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.284+0000 7f20fdabc640 1 -- 172.21.15.43:0/241652282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f85b293f0 con 0x556f85b11390 2024-04-25T10:35:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.284+0000 7f20fdabc640 1 --2- 172.21.15.43:0/241652282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 0x556f85ec9210 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f20f00046e0 tx=0x7f20f00048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431803e42ef01753 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.284+0000 7f20e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.285+0000 7f20e0ff9640 1 -- 172.21.15.43:0/241652282 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f001c070 con 0x556f85b11390 2024-04-25T10:35:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.285+0000 7f20e0ff9640 1 -- 172.21.15.43:0/241652282 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f20f0011da0 con 0x556f85b11390 2024-04-25T10:35:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.285+0000 7f21063d4740 1 -- 172.21.15.43:0/241652282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20f0003b70 con 0x556f85b11390 2024-04-25T10:35:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.285+0000 7f20e0ff9640 1 -- 172.21.15.43:0/241652282 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f20f002d050 con 0x556f85b11390 2024-04-25T10:35:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.285+0000 7f21063d4740 10 client.16443 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.285+0000 7f21063d4740 1 -- 172.21.15.43:0/241652282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556f85bf37d0 con 0x556f85b11390 2024-04-25T10:35:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.285+0000 7f20e0ff9640 1 -- 172.21.15.43:0/241652282 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f0003b70 con 0x556f85b11390 2024-04-25T10:35:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.285+0000 7f20e0ff9640 1 -- 172.21.15.43:0/241652282 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f20f00041e0 con 0x556f85b11390 2024-04-25T10:35:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.285+0000 7f21063d4740 10 client.16443 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.285+0000 7f21063d4740 10 client.16443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.285+0000 7f21063d4740 20 client.16443 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.285+0000 7f21063d4740 1 --2- 172.21.15.43:0/241652282 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556f85acff60 0x556f85ecf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.285+0000 7f21063d4740 4 client.16443 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:35:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.285+0000 7f21063d4740 1 -- 172.21.15.43:0/241652282 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f85e25990 con 0x556f85acff60 2024-04-25T10:35:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.286+0000 7f20fcaba640 1 --2- 172.21.15.43:0/241652282 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556f85acff60 0x556f85ecf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.286+0000 7f20fcaba640 1 --2- 172.21.15.43:0/241652282 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556f85acff60 0x556f85ecf630 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b697f42209dcc050 server_cookie=b1b3b31c529c1f35 in_seq=0 out_seq=0 2024-04-25T10:35:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.286+0000 7f20e0ff9640 10 client.16443 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:53.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.287+0000 7f20e0ff9640 1 -- 172.21.15.43:0/241652282 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556f85e25990 con 0x556f85acff60 2024-04-25T10:35:53.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.287+0000 7f20e0ff9640 10 client.16443 handle_command_reply: tid=0 2024-04-25T10:35:53.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.287+0000 7f21063d4740 1 client.16443 shutdown 2024-04-25T10:35:53.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.287+0000 7f21063d4740 1 -- 172.21.15.43:0/241652282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 msgr2=0x556f85ec9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.287+0000 7f21063d4740 1 --2- 172.21.15.43:0/241652282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 0x556f85ec9210 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f20f00046e0 tx=0x7f20f00048a0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.287+0000 7f21063d4740 1 -- 172.21.15.43:0/241652282 shutdown_connections 2024-04-25T10:35:53.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.287+0000 7f21063d4740 1 --2- 172.21.15.43:0/241652282 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556f85acff60 0x556f85ecf630 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.287+0000 7f21063d4740 1 --2- 172.21.15.43:0/241652282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f85af06b0 0x556f85ebd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.287+0000 7f21063d4740 1 --2- 172.21.15.43:0/241652282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 0x556f85ec9210 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.287+0000 7f20e0ff9640 0 client.16443 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:53.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.287+0000 7f21063d4740 1 --2- 172.21.15.43:0/241652282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f85ae84d0 0x556f85ebd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.287+0000 7f21063d4740 1 -- 172.21.15.43:0/241652282 >> 172.21.15.43:0/241652282 conn(0x556f85b054e0 msgr2=0x556f85eb4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:53.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.287+0000 7f21063d4740 1 -- 172.21.15.43:0/241652282 shutdown_connections 2024-04-25T10:35:53.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.287+0000 7f21063d4740 1 -- 172.21.15.43:0/241652282 wait complete. 2024-04-25T10:35:53.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.287+0000 7f21063d4740 20 client.16443 trim_cache size 0 max 16384 2024-04-25T10:35:53.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.301+0000 7f21063d4740 1 Processor -- start 2024-04-25T10:35:53.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.301+0000 7f21063d4740 1 -- start start 2024-04-25T10:35:53.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.301+0000 7f21063d4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85ae84d0 0x556f85ebd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.301+0000 7f21063d4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f85af06b0 0x556f85ec8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.301+0000 7f21063d4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f85b11390 0x556f85eb4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.301+0000 7f21063d4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20f81b44c0 con 0x556f85ae84d0 2024-04-25T10:35:53.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.301+0000 7f21063d4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f20f81b1d90 con 0x556f85b11390 2024-04-25T10:35:53.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.301+0000 7f21063d4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f20f81afbe0 con 0x556f85af06b0 2024-04-25T10:35:53.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.301+0000 7f20fdabc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85ae84d0 0x556f85ebd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.301+0000 7f20fdabc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85ae84d0 0x556f85ebd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43108/0 (socket says 172.21.15.43:43108) 2024-04-25T10:35:53.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.301+0000 7f20fdabc640 1 -- 172.21.15.43:0/1287045210 learned_addr learned my addr 172.21.15.43:0/1287045210 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:53.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.301+0000 7f20fcaba640 1 --2- 172.21.15.43:0/1287045210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f85af06b0 0x556f85ec8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.301+0000 7f20fdabc640 1 --2- 172.21.15.43:0/1287045210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f85b11390 0x556f85eb4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.302+0000 7f20fdabc640 1 -- 172.21.15.43:0/1287045210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f85af06b0 msgr2=0x556f85ec8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:53.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.302+0000 7f20fdabc640 1 --2- 172.21.15.43:0/1287045210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f85af06b0 0x556f85ec8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.302+0000 7f20fdabc640 1 -- 172.21.15.43:0/1287045210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f85b11390 msgr2=0x556f85eb4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:53.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.302+0000 7f20fdabc640 1 --2- 172.21.15.43:0/1287045210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f85b11390 0x556f85eb4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.302+0000 7f20fdabc640 1 -- 172.21.15.43:0/1287045210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f85ebdfb0 con 0x556f85ae84d0 2024-04-25T10:35:53.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.302+0000 7f20fdabc640 1 --2- 172.21.15.43:0/1287045210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85ae84d0 0x556f85ebd5e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f20f001b040 tx=0x7f20f0008f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84124ffd8c72050 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:53.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.302+0000 7f20e2ffd640 1 -- 172.21.15.43:0/1287045210 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f001c070 con 0x556f85ae84d0 2024-04-25T10:35:53.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.302+0000 7f20e2ffd640 1 -- 172.21.15.43:0/1287045210 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f0011da0 con 0x556f85ae84d0 2024-04-25T10:35:53.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.302+0000 7f20e2ffd640 1 -- 172.21.15.43:0/1287045210 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f0014070 con 0x556f85ae84d0 2024-04-25T10:35:53.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.303+0000 7f21063d4740 1 -- 172.21.15.43:0/1287045210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85ae84d0 msgr2=0x556f85ebd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.303+0000 7f21063d4740 1 --2- 172.21.15.43:0/1287045210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85ae84d0 0x556f85ebd5e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f20f001b040 tx=0x7f20f0008f50 comp rx=0 tx=0).stop 2024-04-25T10:35:53.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.303+0000 7f21063d4740 1 -- 172.21.15.43:0/1287045210 shutdown_connections 2024-04-25T10:35:53.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.303+0000 7f21063d4740 1 --2- 172.21.15.43:0/1287045210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f85b11390 0x556f85eb4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.303+0000 7f21063d4740 1 --2- 172.21.15.43:0/1287045210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f85af06b0 0x556f85ec8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.303+0000 7f21063d4740 1 --2- 172.21.15.43:0/1287045210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85ae84d0 0x556f85ebd5e0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.303+0000 7f21063d4740 1 -- 172.21.15.43:0/1287045210 >> 172.21.15.43:0/1287045210 conn(0x556f85b054e0 msgr2=0x556f85eb4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:53.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.303+0000 7f21063d4740 1 -- 172.21.15.43:0/1287045210 shutdown_connections 2024-04-25T10:35:53.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.303+0000 7f21063d4740 1 -- 172.21.15.43:0/1287045210 wait complete. 2024-04-25T10:35:53.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.303+0000 7f21063d4740 1 -- start start 2024-04-25T10:35:53.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.303+0000 7f21063d4740 1 Processor -- start 2024-04-25T10:35:53.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.304+0000 7f21063d4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f85ae84d0 0x556f85ec5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.304+0000 7f21063d4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f85af06b0 0x556f85ec58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.304+0000 7f21063d4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 0x556f85ec08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.304+0000 7f21063d4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20f81b0250 con 0x556f85b11390 2024-04-25T10:35:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.304+0000 7f21063d4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f20f81b16c0 con 0x556f85af06b0 2024-04-25T10:35:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.304+0000 7f21063d4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f20f81b0920 con 0x556f85ae84d0 2024-04-25T10:35:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.304+0000 7f21063d4740 10 client.0 _ll_register_callbacks cb 0x7f20f0008b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.304+0000 7f20fdabc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 0x556f85ec08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.304+0000 7f20fdabc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 0x556f85ec08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43120/0 (socket says 172.21.15.43:43120) 2024-04-25T10:35:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.304+0000 7f20fdabc640 1 -- 172.21.15.43:0/2298695614 learned_addr learned my addr 172.21.15.43:0/2298695614 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.304+0000 7f20fdabc640 1 --2- 172.21.15.43:0/2298695614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f85ae84d0 0x556f85ec5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.304+0000 7f20fcaba640 1 --2- 172.21.15.43:0/2298695614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f85af06b0 0x556f85ec58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.304+0000 7f20fdabc640 1 -- 172.21.15.43:0/2298695614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f85ae84d0 msgr2=0x556f85ec5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.304+0000 7f20fdabc640 1 --2- 172.21.15.43:0/2298695614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f85ae84d0 0x556f85ec5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.304+0000 7f20fdabc640 1 -- 172.21.15.43:0/2298695614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f85af06b0 msgr2=0x556f85ec58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.304+0000 7f20fdabc640 1 --2- 172.21.15.43:0/2298695614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f85af06b0 0x556f85ec58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.304+0000 7f20fdabc640 1 -- 172.21.15.43:0/2298695614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f85ec1010 con 0x556f85b11390 2024-04-25T10:35:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.304+0000 7f20fdabc640 1 --2- 172.21.15.43:0/2298695614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 0x556f85ec08a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f20f000eca0 tx=0x7f20f002a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f7b8fc9b4768ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.304+0000 7f20e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.305+0000 7f20e0ff9640 1 -- 172.21.15.43:0/2298695614 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f001c030 con 0x556f85b11390 2024-04-25T10:35:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.305+0000 7f20e0ff9640 1 -- 172.21.15.43:0/2298695614 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f20f0004e10 con 0x556f85b11390 2024-04-25T10:35:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.305+0000 7f21063d4740 1 -- 172.21.15.43:0/2298695614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20f0011da0 con 0x556f85b11390 2024-04-25T10:35:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.305+0000 7f20e0ff9640 1 -- 172.21.15.43:0/2298695614 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f20f0014070 con 0x556f85b11390 2024-04-25T10:35:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.305+0000 7f21063d4740 10 client.16449 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.305+0000 7f21063d4740 1 -- 172.21.15.43:0/2298695614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556f85f37370 con 0x556f85b11390 2024-04-25T10:35:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.305+0000 7f20e0ff9640 1 -- 172.21.15.43:0/2298695614 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f0011da0 con 0x556f85b11390 2024-04-25T10:35:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.305+0000 7f20e0ff9640 1 -- 172.21.15.43:0/2298695614 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f20f0019890 con 0x556f85b11390 2024-04-25T10:35:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.305+0000 7f21063d4740 10 client.16449 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.305+0000 7f21063d4740 10 client.16449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.305+0000 7f21063d4740 20 client.16449 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:53.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.305+0000 7f21063d4740 1 --2- 172.21.15.43:0/2298695614 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556f85aa53a0 0x556f85f399a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:53.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.305+0000 7f21063d4740 4 client.16449 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:35:53.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.305+0000 7f21063d4740 1 -- 172.21.15.43:0/2298695614 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f85e1f800 con 0x556f85aa53a0 2024-04-25T10:35:53.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.305+0000 7f20fcaba640 1 --2- 172.21.15.43:0/2298695614 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556f85aa53a0 0x556f85f399a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:53.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.306+0000 7f20e0ff9640 10 client.16449 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:53.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.306+0000 7f20fcaba640 1 --2- 172.21.15.43:0/2298695614 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556f85aa53a0 0x556f85f399a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=557e1e4d3ff7c932 server_cookie=a02df1c1d00097ed in_seq=0 out_seq=0 2024-04-25T10:35:53.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.306+0000 7f20e0ff9640 1 -- 172.21.15.43:0/2298695614 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f85e1f800 con 0x556f85aa53a0 2024-04-25T10:35:53.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.306+0000 7f20e0ff9640 10 client.16449 handle_command_reply: tid=0 2024-04-25T10:35:53.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.306+0000 7f21063d4740 1 client.16449 shutdown 2024-04-25T10:35:53.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.306+0000 7f21063d4740 1 -- 172.21.15.43:0/2298695614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 msgr2=0x556f85ec08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.306+0000 7f21063d4740 1 --2- 172.21.15.43:0/2298695614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 0x556f85ec08a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f20f000eca0 tx=0x7f20f002a040 comp rx=0 tx=0).stop 2024-04-25T10:35:53.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.306+0000 7f21063d4740 1 -- 172.21.15.43:0/2298695614 shutdown_connections 2024-04-25T10:35:53.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.306+0000 7f21063d4740 1 --2- 172.21.15.43:0/2298695614 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556f85aa53a0 0x556f85f399a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.306+0000 7f21063d4740 1 --2- 172.21.15.43:0/2298695614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f85af06b0 0x556f85ec58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.306+0000 7f21063d4740 1 --2- 172.21.15.43:0/2298695614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f85b11390 0x556f85ec08a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.306+0000 7f20e0ff9640 0 client.16449 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:53.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.306+0000 7f21063d4740 1 --2- 172.21.15.43:0/2298695614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f85ae84d0 0x556f85ec5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.306+0000 7f21063d4740 1 -- 172.21.15.43:0/2298695614 >> 172.21.15.43:0/2298695614 conn(0x556f85b054e0 msgr2=0x556f85eaef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:53.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.306+0000 7f21063d4740 1 -- 172.21.15.43:0/2298695614 shutdown_connections 2024-04-25T10:35:53.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.306+0000 7f21063d4740 1 -- 172.21.15.43:0/2298695614 wait complete. 2024-04-25T10:35:53.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.306+0000 7f21063d4740 20 client.16449 trim_cache size 0 max 16384 2024-04-25T10:35:53.315 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:35:53.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.307+0000 7f20ff51f640 1 -- 172.21.15.43:0/1694161786 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f20e40afc60 msgr2=0x7f20e40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.307+0000 7f20ff51f640 1 --2- 172.21.15.43:0/1694161786 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f20e40afc60 0x7f20e40b2170 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x556f85eaff50 tx=0x7f20f4000bc0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.307+0000 7f20ff51f640 1 -- 172.21.15.43:0/1694161786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f81028e0 msgr2=0x7f20f81a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:53.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.307+0000 7f20ff51f640 1 --2- 172.21.15.43:0/1694161786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f81028e0 0x7f20f81a2600 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f20f400c890 tx=0x7f20f400cd50 comp rx=0 tx=0).stop 2024-04-25T10:35:53.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.307+0000 7f20ff51f640 1 -- 172.21.15.43:0/1694161786 shutdown_connections 2024-04-25T10:35:53.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.307+0000 7f20ff51f640 1 --2- 172.21.15.43:0/1694161786 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f20e40afc60 0x7f20e40b2170 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.307+0000 7f20ff51f640 1 --2- 172.21.15.43:0/1694161786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20f8110480 0x7f20f819c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.307+0000 7f20ff51f640 1 --2- 172.21.15.43:0/1694161786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20f810db20 0x7f20f81a2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.307+0000 7f20ff51f640 1 --2- 172.21.15.43:0/1694161786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f81028e0 0x7f20f81a2600 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:53.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.307+0000 7f20ff51f640 1 -- 172.21.15.43:0/1694161786 >> 172.21.15.43:0/1694161786 conn(0x7f20f8078110 msgr2=0x7f20f8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:53.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.307+0000 7f20ff51f640 1 -- 172.21.15.43:0/1694161786 shutdown_connections 2024-04-25T10:35:53.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:53.307+0000 7f20ff51f640 1 -- 172.21.15.43:0/1694161786 wait complete. 2024-04-25T10:35:54.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:53 smithi077 ceph-mon[32772]: pgmap v104: 97 pgs: 97 active+clean; 3.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T10:35:54.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:53 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/4058079510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:35:54.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:53 smithi164 ceph-mon[32713]: pgmap v104: 97 pgs: 97 active+clean; 3.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T10:35:54.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:53 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/4058079510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:35:54.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:53 smithi043 ceph-mon[29478]: pgmap v104: 97 pgs: 97 active+clean; 3.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T10:35:54.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:53 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/4058079510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:35:54.322 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:35:54.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:35:54.322 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:35:54.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.450+0000 7f15a64ee640 1 -- 172.21.15.43:0/3821146131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a0102af0 msgr2=0x7f15a0102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:54.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.450+0000 7f15a64ee640 1 --2- 172.21.15.43:0/3821146131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a0102af0 0x7f15a0102f00 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1594009900 tx=0x7f159402a410 comp rx=0 tx=0).stop 2024-04-25T10:35:54.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.450+0000 7f15a64ee640 1 -- 172.21.15.43:0/3821146131 shutdown_connections 2024-04-25T10:35:54.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.450+0000 7f15a64ee640 1 --2- 172.21.15.43:0/3821146131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15a010d380 0x7f15a010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.450+0000 7f15a64ee640 1 --2- 172.21.15.43:0/3821146131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a0102af0 0x7f15a0102f00 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.450+0000 7f15a64ee640 1 --2- 172.21.15.43:0/3821146131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15a0108a20 0x7f15a01025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.450+0000 7f15a64ee640 1 -- 172.21.15.43:0/3821146131 >> 172.21.15.43:0/3821146131 conn(0x7f15a0078100 msgr2=0x7f15a0078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:54.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.450+0000 7f15a64ee640 1 -- 172.21.15.43:0/3821146131 shutdown_connections 2024-04-25T10:35:54.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.450+0000 7f15a64ee640 1 -- 172.21.15.43:0/3821146131 wait complete. 2024-04-25T10:35:54.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.451+0000 7f15a64ee640 1 Processor -- start 2024-04-25T10:35:54.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.451+0000 7f15a64ee640 1 -- start start 2024-04-25T10:35:54.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.451+0000 7f15a64ee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15a0102af0 0x7f15a0199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.451+0000 7f15a64ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a0108a20 0x7f15a0199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.451+0000 7f15a64ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15a010d380 0x7f15a019de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.451+0000 7f15a64ee640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f15a0112be0 con 0x7f15a0108a20 2024-04-25T10:35:54.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.451+0000 7f15a64ee640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f15a0112a60 con 0x7f15a010d380 2024-04-25T10:35:54.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.451+0000 7f15a64ee640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f15a0112d60 con 0x7f15a0102af0 2024-04-25T10:35:54.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.452+0000 7f159f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a0108a20 0x7f15a0199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.452+0000 7f159f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a0108a20 0x7f15a0199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43124/0 (socket says 172.21.15.43:43124) 2024-04-25T10:35:54.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.452+0000 7f159f7fe640 1 -- 172.21.15.43:0/1959592627 learned_addr learned my addr 172.21.15.43:0/1959592627 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:54.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.452+0000 7f15a4a8b640 1 --2- 172.21.15.43:0/1959592627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15a010d380 0x7f15a019de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.452+0000 7f159ffff640 1 --2- 172.21.15.43:0/1959592627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15a0102af0 0x7f15a0199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.452+0000 7f159f7fe640 1 -- 172.21.15.43:0/1959592627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15a0102af0 msgr2=0x7f15a0199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:54.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.452+0000 7f159f7fe640 1 --2- 172.21.15.43:0/1959592627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15a0102af0 0x7f15a0199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.452+0000 7f159f7fe640 1 -- 172.21.15.43:0/1959592627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15a010d380 msgr2=0x7f15a019de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:54.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.452+0000 7f159f7fe640 1 --2- 172.21.15.43:0/1959592627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15a010d380 0x7f15a019de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.452+0000 7f159f7fe640 1 -- 172.21.15.43:0/1959592627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a019e4a0 con 0x7f15a0108a20 2024-04-25T10:35:54.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.452+0000 7f159f7fe640 1 --2- 172.21.15.43:0/1959592627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a0108a20 0x7f15a0199ba0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1594006010 tx=0x7f1594004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:54.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.452+0000 7f159d7fa640 1 -- 172.21.15.43:0/1959592627 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159403f030 con 0x7f15a0108a20 2024-04-25T10:35:54.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.452+0000 7f15a64ee640 1 -- 172.21.15.43:0/1959592627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a019e730 con 0x7f15a0108a20 2024-04-25T10:35:54.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.452+0000 7f159d7fa640 1 -- 172.21.15.43:0/1959592627 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1594003aa0 con 0x7f15a0108a20 2024-04-25T10:35:54.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.452+0000 7f159d7fa640 1 -- 172.21.15.43:0/1959592627 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594035070 con 0x7f15a0108a20 2024-04-25T10:35:54.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.452+0000 7f15a64ee640 1 -- 172.21.15.43:0/1959592627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a019ec20 con 0x7f15a0108a20 2024-04-25T10:35:54.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f15ad39e740 1 Processor -- start 2024-04-25T10:35:54.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f15ad39e740 1 -- start start 2024-04-25T10:35:54.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f159d7fa640 1 -- 172.21.15.43:0/1959592627 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f1594042030 con 0x7f15a0108a20 2024-04-25T10:35:54.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f15ad39e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 0x55f99870d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f15ad39e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f998344550 0x55f99870d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f15ad39e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f99834c730 0x55f99870deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f15ad39e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f998673b00 con 0x55f998344550 2024-04-25T10:35:54.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f15ad39e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f99860f920 con 0x55f99836d410 2024-04-25T10:35:54.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f15ad39e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f99866fac0 con 0x55f99834c730 2024-04-25T10:35:54.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f159ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f998344550 0x55f99870d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f159ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f998344550 0x55f99870d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43140/0 (socket says 172.21.15.43:43140) 2024-04-25T10:35:54.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f159ffff640 1 -- 172.21.15.43:0/2368861072 learned_addr learned my addr 172.21.15.43:0/2368861072 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:54.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f15a4a8b640 1 --2- 172.21.15.43:0/2368861072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 0x55f99870d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f159f7fe640 1 --2- 172.21.15.43:0/2368861072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f99834c730 0x55f99870deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f15a4a8b640 1 -- 172.21.15.43:0/2368861072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f99834c730 msgr2=0x55f99870deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:54.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f15a4a8b640 1 --2- 172.21.15.43:0/2368861072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f99834c730 0x55f99870deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f15a4a8b640 1 -- 172.21.15.43:0/2368861072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f998344550 msgr2=0x55f99870d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:54.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f15a4a8b640 1 --2- 172.21.15.43:0/2368861072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f998344550 0x55f99870d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f15a4a8b640 1 -- 172.21.15.43:0/2368861072 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f99844d7e0 con 0x55f99836d410 2024-04-25T10:35:54.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f15a4a8b640 1 --2- 172.21.15.43:0/2368861072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 0x55f99870d430 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1598009f30 tx=0x7f1598010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=470b4e02d23123b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:54.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f159d7fa640 1 --2- 172.21.15.43:0/1959592627 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f15880afc60 0x7f15880b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f159ffff640 1 --2- 172.21.15.43:0/1959592627 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f15880afc60 0x7f15880b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.456+0000 7f15867fc640 1 -- 172.21.15.43:0/2368861072 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598016070 con 0x55f99836d410 2024-04-25T10:35:54.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.457+0000 7f159d7fa640 1 -- 172.21.15.43:0/1959592627 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f15940ff4f0 con 0x7f15a0108a20 2024-04-25T10:35:54.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.457+0000 7f15867fc640 1 -- 172.21.15.43:0/2368861072 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1598004d20 con 0x55f99836d410 2024-04-25T10:35:54.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.457+0000 7f15867fc640 1 -- 172.21.15.43:0/2368861072 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598012740 con 0x55f99836d410 2024-04-25T10:35:54.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.457+0000 7f15ad39e740 1 -- 172.21.15.43:0/2368861072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 msgr2=0x55f99870d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:54.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.457+0000 7f159ffff640 1 --2- 172.21.15.43:0/1959592627 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f15880afc60 0x7f15880b2170 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x55f99870b110 tx=0x7f1590005f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:54.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.457+0000 7f15ad39e740 1 --2- 172.21.15.43:0/2368861072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 0x55f99870d430 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1598009f30 tx=0x7f1598010800 comp rx=0 tx=0).stop 2024-04-25T10:35:54.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.457+0000 7f15ad39e740 1 -- 172.21.15.43:0/2368861072 shutdown_connections 2024-04-25T10:35:54.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.457+0000 7f15ad39e740 1 --2- 172.21.15.43:0/2368861072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f99834c730 0x55f99870deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.457+0000 7f15ad39e740 1 --2- 172.21.15.43:0/2368861072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f998344550 0x55f99870d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.457+0000 7f15ad39e740 1 --2- 172.21.15.43:0/2368861072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 0x55f99870d430 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.457+0000 7f15ad39e740 1 -- 172.21.15.43:0/2368861072 >> 172.21.15.43:0/2368861072 conn(0x55f998361560 msgr2=0x55f998709dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:54.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.458+0000 7f15ad39e740 1 -- 172.21.15.43:0/2368861072 shutdown_connections 2024-04-25T10:35:54.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.458+0000 7f15ad39e740 1 -- 172.21.15.43:0/2368861072 wait complete. 2024-04-25T10:35:54.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.458+0000 7f15ad39e740 1 -- start start 2024-04-25T10:35:54.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.458+0000 7f15ad39e740 1 Processor -- start 2024-04-25T10:35:54.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f15ad39e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f998344550 0x55f998719e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f15ad39e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f99834c730 0x55f99871a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f15ad39e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 0x55f998726030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f15ad39e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f99871cb80 con 0x55f998344550 2024-04-25T10:35:54.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f15ad39e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f99860d5d0 con 0x55f99836d410 2024-04-25T10:35:54.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f15ad39e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f99871cd00 con 0x55f99834c730 2024-04-25T10:35:54.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f15ad39e740 10 client.0 _ll_register_callbacks cb 0x55f9985fb080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:54.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f159ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f998344550 0x55f998719e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f159ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f998344550 0x55f998719e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43150/0 (socket says 172.21.15.43:43150) 2024-04-25T10:35:54.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f159ffff640 1 -- 172.21.15.43:0/594886985 learned_addr learned my addr 172.21.15.43:0/594886985 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:54.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f15a4a8b640 1 --2- 172.21.15.43:0/594886985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 0x55f998726030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f159f7fe640 1 --2- 172.21.15.43:0/594886985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f99834c730 0x55f99871a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f159ffff640 1 -- 172.21.15.43:0/594886985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f99834c730 msgr2=0x55f99871a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:54.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f159ffff640 1 --2- 172.21.15.43:0/594886985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f99834c730 0x55f99871a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f159ffff640 1 -- 172.21.15.43:0/594886985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 msgr2=0x55f998726030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:54.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f159ffff640 1 --2- 172.21.15.43:0/594886985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 0x55f998726030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f159ffff640 1 -- 172.21.15.43:0/594886985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9984adf70 con 0x55f998344550 2024-04-25T10:35:54.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f159f7fe640 1 --2- 172.21.15.43:0/594886985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f99834c730 0x55f99871a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:54.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f159ffff640 1 --2- 172.21.15.43:0/594886985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f998344550 0x55f998719e20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1590013010 tx=0x7f1590004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a43d0ea2b83746a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f157f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f157f7fe640 1 -- 172.21.15.43:0/594886985 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159002d060 con 0x55f998344550 2024-04-25T10:35:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f157f7fe640 1 -- 172.21.15.43:0/594886985 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f1590018030 con 0x55f998344550 2024-04-25T10:35:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.459+0000 7f15ad39e740 1 -- 172.21.15.43:0/594886985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1598004d20 con 0x55f998344550 2024-04-25T10:35:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.460+0000 7f157f7fe640 1 -- 172.21.15.43:0/594886985 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1590011940 con 0x55f998344550 2024-04-25T10:35:54.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.460+0000 7f15ad39e740 10 client.16461 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:54.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.460+0000 7f15ad39e740 1 -- 172.21.15.43:0/594886985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f998345170 con 0x55f998344550 2024-04-25T10:35:54.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.460+0000 7f157f7fe640 1 -- 172.21.15.43:0/594886985 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1598004d20 con 0x55f998344550 2024-04-25T10:35:54.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.460+0000 7f157f7fe640 1 -- 172.21.15.43:0/594886985 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1590032c50 con 0x55f998344550 2024-04-25T10:35:54.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.460+0000 7f15ad39e740 10 client.16461 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:54.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.460+0000 7f15ad39e740 10 client.16461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:54.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.460+0000 7f15ad39e740 20 client.16461 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:54.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.460+0000 7f15ad39e740 1 --2- 172.21.15.43:0/594886985 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f99832c1f0 0x55f99872c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.460+0000 7f15ad39e740 4 client.16461 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:35:54.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.460+0000 7f15ad39e740 1 -- 172.21.15.43:0/594886985 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f998681710 con 0x55f99832c1f0 2024-04-25T10:35:54.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.461+0000 7f159f7fe640 1 --2- 172.21.15.43:0/594886985 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f99832c1f0 0x55f99872c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.461+0000 7f159f7fe640 1 --2- 172.21.15.43:0/594886985 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f99832c1f0 0x55f99872c240 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8cb54eaf73d97ac server_cookie=c14d147151c1126e in_seq=0 out_seq=0 2024-04-25T10:35:54.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.461+0000 7f157f7fe640 10 client.16461 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:54.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.462+0000 7f157f7fe640 1 -- 172.21.15.43:0/594886985 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f998681710 con 0x55f99832c1f0 2024-04-25T10:35:54.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.462+0000 7f157f7fe640 10 client.16461 handle_command_reply: tid=0 2024-04-25T10:35:54.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.462+0000 7f15ad39e740 1 client.16461 shutdown 2024-04-25T10:35:54.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.462+0000 7f15ad39e740 1 -- 172.21.15.43:0/594886985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f998344550 msgr2=0x55f998719e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:54.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.462+0000 7f15ad39e740 1 --2- 172.21.15.43:0/594886985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f998344550 0x55f998719e20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1590013010 tx=0x7f1590004730 comp rx=0 tx=0).stop 2024-04-25T10:35:54.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.462+0000 7f15ad39e740 1 -- 172.21.15.43:0/594886985 shutdown_connections 2024-04-25T10:35:54.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.462+0000 7f15ad39e740 1 --2- 172.21.15.43:0/594886985 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f99832c1f0 0x55f99872c240 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.462+0000 7f15ad39e740 1 --2- 172.21.15.43:0/594886985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 0x55f998726030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.462+0000 7f15ad39e740 1 --2- 172.21.15.43:0/594886985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f99834c730 0x55f99871a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.462+0000 7f15ad39e740 1 --2- 172.21.15.43:0/594886985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f998344550 0x55f998719e20 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.462+0000 7f157f7fe640 0 client.16461 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:54.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.462+0000 7f15ad39e740 1 -- 172.21.15.43:0/594886985 >> 172.21.15.43:0/594886985 conn(0x55f998361560 msgr2=0x55f9987084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:54.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.462+0000 7f15ad39e740 1 -- 172.21.15.43:0/594886985 shutdown_connections 2024-04-25T10:35:54.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.462+0000 7f15ad39e740 1 -- 172.21.15.43:0/594886985 wait complete. 2024-04-25T10:35:54.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.462+0000 7f15ad39e740 20 client.16461 trim_cache size 0 max 16384 2024-04-25T10:35:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.480+0000 7f15ad39e740 1 Processor -- start 2024-04-25T10:35:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.480+0000 7f15ad39e740 1 -- start start 2024-04-25T10:35:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.480+0000 7f15ad39e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f998344550 0x55f99870dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.480+0000 7f15ad39e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f99834c730 0x55f998731a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.480+0000 7f15ad39e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 0x55f998731f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.480+0000 7f15ad39e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01affc0 con 0x55f99834c730 2024-04-25T10:35:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.480+0000 7f15ad39e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01ad890 con 0x55f99836d410 2024-04-25T10:35:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.480+0000 7f15ad39e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01ab6e0 con 0x55f998344550 2024-04-25T10:35:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.481+0000 7f159f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f99834c730 0x55f998731a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.481+0000 7f159f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f99834c730 0x55f998731a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43158/0 (socket says 172.21.15.43:43158) 2024-04-25T10:35:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.481+0000 7f159f7fe640 1 -- 172.21.15.43:0/3575763614 learned_addr learned my addr 172.21.15.43:0/3575763614 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.481+0000 7f159ffff640 1 --2- 172.21.15.43:0/3575763614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f998344550 0x55f99870dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.481+0000 7f15a4a8b640 1 --2- 172.21.15.43:0/3575763614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 0x55f998731f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.481+0000 7f159f7fe640 1 -- 172.21.15.43:0/3575763614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f998344550 msgr2=0x55f99870dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.481+0000 7f159f7fe640 1 --2- 172.21.15.43:0/3575763614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f998344550 0x55f99870dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.481+0000 7f159f7fe640 1 -- 172.21.15.43:0/3575763614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 msgr2=0x55f998731f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.481+0000 7f159f7fe640 1 --2- 172.21.15.43:0/3575763614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 0x55f998731f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.481+0000 7f159f7fe640 1 -- 172.21.15.43:0/3575763614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f99871aad0 con 0x55f99834c730 2024-04-25T10:35:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.481+0000 7f159f7fe640 1 --2- 172.21.15.43:0/3575763614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f99834c730 0x55f998731a00 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f15940c3740 tx=0x7f1594101a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3e40acc31890e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.481+0000 7f1585ffb640 1 -- 172.21.15.43:0/3575763614 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594049040 con 0x55f99834c730 2024-04-25T10:35:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.481+0000 7f1585ffb640 1 -- 172.21.15.43:0/3575763614 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15940c1e50 con 0x55f99834c730 2024-04-25T10:35:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.481+0000 7f1585ffb640 1 -- 172.21.15.43:0/3575763614 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15940c76c0 con 0x55f99834c730 2024-04-25T10:35:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.482+0000 7f15ad39e740 1 -- 172.21.15.43:0/3575763614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f99834c730 msgr2=0x55f998731a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.482+0000 7f15ad39e740 1 --2- 172.21.15.43:0/3575763614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f99834c730 0x55f998731a00 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f15940c3740 tx=0x7f1594101a40 comp rx=0 tx=0).stop 2024-04-25T10:35:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.482+0000 7f15ad39e740 1 -- 172.21.15.43:0/3575763614 shutdown_connections 2024-04-25T10:35:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.482+0000 7f15ad39e740 1 --2- 172.21.15.43:0/3575763614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 0x55f998731f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.482+0000 7f15ad39e740 1 --2- 172.21.15.43:0/3575763614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f99834c730 0x55f998731a00 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.482+0000 7f15ad39e740 1 --2- 172.21.15.43:0/3575763614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f998344550 0x55f99870dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.482+0000 7f15ad39e740 1 -- 172.21.15.43:0/3575763614 >> 172.21.15.43:0/3575763614 conn(0x55f998361560 msgr2=0x55f99871b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:54.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.482+0000 7f15ad39e740 1 -- 172.21.15.43:0/3575763614 shutdown_connections 2024-04-25T10:35:54.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.482+0000 7f15ad39e740 1 -- 172.21.15.43:0/3575763614 wait complete. 2024-04-25T10:35:54.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.482+0000 7f15ad39e740 1 -- start start 2024-04-25T10:35:54.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.483+0000 7f15ad39e740 1 Processor -- start 2024-04-25T10:35:54.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.483+0000 7f15ad39e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f998344550 0x55f99871dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.483+0000 7f15ad39e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f99834c730 0x55f99871e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.483+0000 7f15ad39e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 0x55f99871e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.483+0000 7f15ad39e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01abd50 con 0x55f998344550 2024-04-25T10:35:54.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.483+0000 7f15ad39e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01ad1c0 con 0x55f99836d410 2024-04-25T10:35:54.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.483+0000 7f15ad39e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01ac420 con 0x55f99834c730 2024-04-25T10:35:54.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.483+0000 7f15ad39e740 10 client.0 _ll_register_callbacks cb 0x7f15900207d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:54.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.483+0000 7f159ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f998344550 0x55f99871dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.483+0000 7f159ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f998344550 0x55f99871dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43170/0 (socket says 172.21.15.43:43170) 2024-04-25T10:35:54.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.483+0000 7f159ffff640 1 -- 172.21.15.43:0/2023062422 learned_addr learned my addr 172.21.15.43:0/2023062422 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:54.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.483+0000 7f15a4a8b640 1 --2- 172.21.15.43:0/2023062422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 0x55f99871e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.483+0000 7f159ffff640 1 -- 172.21.15.43:0/2023062422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f99834c730 msgr2=0x55f99871e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:54.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.483+0000 7f159f7fe640 1 --2- 172.21.15.43:0/2023062422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f99834c730 0x55f99871e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.483+0000 7f159ffff640 1 --2- 172.21.15.43:0/2023062422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f99834c730 0x55f99871e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.483+0000 7f159ffff640 1 -- 172.21.15.43:0/2023062422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 msgr2=0x55f99871e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:54.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.483+0000 7f159ffff640 1 --2- 172.21.15.43:0/2023062422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 0x55f99871e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.484+0000 7f159ffff640 1 -- 172.21.15.43:0/2023062422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f998793710 con 0x55f998344550 2024-04-25T10:35:54.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.484+0000 7f159ffff640 1 --2- 172.21.15.43:0/2023062422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f998344550 0x55f99871dd80 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1590028620 tx=0x7f1590011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef28099f9e832e0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:54.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.484+0000 7f157f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:54.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.484+0000 7f157f7fe640 1 -- 172.21.15.43:0/2023062422 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1590004040 con 0x55f998344550 2024-04-25T10:35:54.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.484+0000 7f157f7fe640 1 -- 172.21.15.43:0/2023062422 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f15900041e0 con 0x55f998344550 2024-04-25T10:35:54.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.484+0000 7f15ad39e740 1 -- 172.21.15.43:0/2023062422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15940c1e50 con 0x55f998344550 2024-04-25T10:35:54.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.484+0000 7f157f7fe640 1 -- 172.21.15.43:0/2023062422 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f159002d020 con 0x55f998344550 2024-04-25T10:35:54.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.484+0000 7f15ad39e740 10 client.16467 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:54.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.484+0000 7f15ad39e740 1 -- 172.21.15.43:0/2023062422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f998794a10 con 0x55f998344550 2024-04-25T10:35:54.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.484+0000 7f157f7fe640 1 -- 172.21.15.43:0/2023062422 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15940c1e50 con 0x55f998344550 2024-04-25T10:35:54.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.484+0000 7f157f7fe640 1 -- 172.21.15.43:0/2023062422 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1590032480 con 0x55f998344550 2024-04-25T10:35:54.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.485+0000 7f15ad39e740 10 client.16467 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:54.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.485+0000 7f15ad39e740 10 client.16467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:54.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.485+0000 7f15ad39e740 20 client.16467 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:54.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.485+0000 7f15ad39e740 1 --2- 172.21.15.43:0/2023062422 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f998301630 0x55f998797200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.485+0000 7f15ad39e740 4 client.16467 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:35:54.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.485+0000 7f15ad39e740 1 -- 172.21.15.43:0/2023062422 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f99867b580 con 0x55f998301630 2024-04-25T10:35:54.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.485+0000 7f159f7fe640 1 --2- 172.21.15.43:0/2023062422 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f998301630 0x55f998797200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.485+0000 7f157f7fe640 10 client.16467 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:54.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.485+0000 7f159f7fe640 1 --2- 172.21.15.43:0/2023062422 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f998301630 0x55f998797200 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1833cedc81d4fbb7 server_cookie=167ed09965673199 in_seq=0 out_seq=0 2024-04-25T10:35:54.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.486+0000 7f157f7fe640 1 -- 172.21.15.43:0/2023062422 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f99867b580 con 0x55f998301630 2024-04-25T10:35:54.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.486+0000 7f157f7fe640 10 client.16467 handle_command_reply: tid=0 2024-04-25T10:35:54.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.486+0000 7f15ad39e740 1 client.16467 shutdown 2024-04-25T10:35:54.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15ad39e740 1 -- 172.21.15.43:0/2023062422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f998344550 msgr2=0x55f99871dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:54.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15ad39e740 1 --2- 172.21.15.43:0/2023062422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f998344550 0x55f99871dd80 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1590028620 tx=0x7f1590011d10 comp rx=0 tx=0).stop 2024-04-25T10:35:54.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15ad39e740 1 -- 172.21.15.43:0/2023062422 shutdown_connections 2024-04-25T10:35:54.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15ad39e740 1 --2- 172.21.15.43:0/2023062422 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f998301630 0x55f998797200 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15ad39e740 1 --2- 172.21.15.43:0/2023062422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f99836d410 0x55f99871e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15ad39e740 1 --2- 172.21.15.43:0/2023062422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f99834c730 0x55f99871e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15ad39e740 1 --2- 172.21.15.43:0/2023062422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f998344550 0x55f99871dd80 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f157f7fe640 0 client.16467 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:54.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15ad39e740 1 -- 172.21.15.43:0/2023062422 >> 172.21.15.43:0/2023062422 conn(0x55f998361560 msgr2=0x55f99870db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:54.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15ad39e740 1 -- 172.21.15.43:0/2023062422 shutdown_connections 2024-04-25T10:35:54.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15ad39e740 1 -- 172.21.15.43:0/2023062422 wait complete. 2024-04-25T10:35:54.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15ad39e740 20 client.16467 trim_cache size 0 max 16384 2024-04-25T10:35:54.496 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:35:54.497 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:35:54.497 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "896dafb5-b61c-4f68-8927-5d7dfe730a4e", 2024-04-25T10:35:54.497 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:35:54.497 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:35:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15a64ee640 1 -- 172.21.15.43:0/1959592627 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f15880afc60 msgr2=0x7f15880b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15a64ee640 1 --2- 172.21.15.43:0/1959592627 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f15880afc60 0x7f15880b2170 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x55f99870b110 tx=0x7f1590005f90 comp rx=0 tx=0).stop 2024-04-25T10:35:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15a64ee640 1 -- 172.21.15.43:0/1959592627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a0108a20 msgr2=0x7f15a0199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15a64ee640 1 --2- 172.21.15.43:0/1959592627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a0108a20 0x7f15a0199ba0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1594006010 tx=0x7f1594004060 comp rx=0 tx=0).stop 2024-04-25T10:35:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15a64ee640 1 -- 172.21.15.43:0/1959592627 shutdown_connections 2024-04-25T10:35:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15a64ee640 1 --2- 172.21.15.43:0/1959592627 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f15880afc60 0x7f15880b2170 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15a64ee640 1 --2- 172.21.15.43:0/1959592627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15a010d380 0x7f15a019de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15a64ee640 1 --2- 172.21.15.43:0/1959592627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a0108a20 0x7f15a0199ba0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15a64ee640 1 --2- 172.21.15.43:0/1959592627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15a0102af0 0x7f15a0199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.487+0000 7f15a64ee640 1 -- 172.21.15.43:0/1959592627 >> 172.21.15.43:0/1959592627 conn(0x7f15a0078100 msgr2=0x7f15a01040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.488+0000 7f15a64ee640 1 -- 172.21.15.43:0/1959592627 shutdown_connections 2024-04-25T10:35:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.488+0000 7f15a64ee640 1 -- 172.21.15.43:0/1959592627 wait complete. 2024-04-25T10:35:54.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:35:54.502 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:35:54.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.613+0000 7f6a5f0f9640 1 -- 172.21.15.43:0/1740455647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a5810e0c0 msgr2=0x7f6a581105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:54.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.613+0000 7f6a5f0f9640 1 --2- 172.21.15.43:0/1740455647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a5810e0c0 0x7f6a581105a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009900 tx=0x7f6a4002a3f0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.613+0000 7f6a5f0f9640 1 -- 172.21.15.43:0/1740455647 shutdown_connections 2024-04-25T10:35:54.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.613+0000 7f6a5f0f9640 1 --2- 172.21.15.43:0/1740455647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a5810b6d0 0x7f6a5810daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.613+0000 7f6a5f0f9640 1 --2- 172.21.15.43:0/1740455647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a5810e0c0 0x7f6a581105a0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.613+0000 7f6a5f0f9640 1 --2- 172.21.15.43:0/1740455647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a58104c70 0x7f6a58105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.613+0000 7f6a5f0f9640 1 -- 172.21.15.43:0/1740455647 >> 172.21.15.43:0/1740455647 conn(0x7f6a58078110 msgr2=0x7f6a58078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:54.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.613+0000 7f6a5f0f9640 1 -- 172.21.15.43:0/1740455647 shutdown_connections 2024-04-25T10:35:54.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.613+0000 7f6a5f0f9640 1 -- 172.21.15.43:0/1740455647 wait complete. 2024-04-25T10:35:54.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.613+0000 7f6a5f0f9640 1 Processor -- start 2024-04-25T10:35:54.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.613+0000 7f6a5f0f9640 1 -- start start 2024-04-25T10:35:54.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.614+0000 7f6a5f0f9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a58104c70 0x7f6a5819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.614+0000 7f6a5f0f9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a5810b6d0 0x7f6a5819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.614+0000 7f6a5f0f9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a5810e0c0 0x7f6a5819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.614+0000 7f6a5f0f9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6a58113ce0 con 0x7f6a5810b6d0 2024-04-25T10:35:54.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.614+0000 7f6a5f0f9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6a58113b60 con 0x7f6a5810e0c0 2024-04-25T10:35:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.614+0000 7f6a5f0f9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6a58113e60 con 0x7f6a58104c70 2024-04-25T10:35:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.614+0000 7f6a57fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a5810b6d0 0x7f6a5819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.614+0000 7f6a57fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a5810b6d0 0x7f6a5819ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43188/0 (socket says 172.21.15.43:43188) 2024-04-25T10:35:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.614+0000 7f6a57fff640 1 -- 172.21.15.43:0/3020198186 learned_addr learned my addr 172.21.15.43:0/3020198186 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.614+0000 7f6a5d696640 1 --2- 172.21.15.43:0/3020198186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a5810e0c0 0x7f6a5819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.614+0000 7f6a5ce95640 1 --2- 172.21.15.43:0/3020198186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a58104c70 0x7f6a5819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.614+0000 7f6a57fff640 1 -- 172.21.15.43:0/3020198186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a58104c70 msgr2=0x7f6a5819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.614+0000 7f6a57fff640 1 --2- 172.21.15.43:0/3020198186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a58104c70 0x7f6a5819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.614+0000 7f6a57fff640 1 -- 172.21.15.43:0/3020198186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a5810e0c0 msgr2=0x7f6a5819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.614+0000 7f6a57fff640 1 --2- 172.21.15.43:0/3020198186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a5810e0c0 0x7f6a5819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.614+0000 7f6a57fff640 1 -- 172.21.15.43:0/3020198186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5819f4e0 con 0x7f6a5810b6d0 2024-04-25T10:35:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.614+0000 7f6a5ce95640 1 --2- 172.21.15.43:0/3020198186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a58104c70 0x7f6a5819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.614+0000 7f6a57fff640 1 --2- 172.21.15.43:0/3020198186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a5810b6d0 0x7f6a5819ac60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800c9a0 tx=0x7f6a4800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.614+0000 7f6a55ffb640 1 -- 172.21.15.43:0/3020198186 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48014070 con 0x7f6a5810b6d0 2024-04-25T10:35:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.615+0000 7f6a55ffb640 1 -- 172.21.15.43:0/3020198186 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a48004440 con 0x7f6a5810b6d0 2024-04-25T10:35:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.615+0000 7f6a5f0f9640 1 -- 172.21.15.43:0/3020198186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5819f7d0 con 0x7f6a5810b6d0 2024-04-25T10:35:54.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.615+0000 7f6a55ffb640 1 -- 172.21.15.43:0/3020198186 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4800f450 con 0x7f6a5810b6d0 2024-04-25T10:35:54.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.615+0000 7f6a5f0f9640 1 -- 172.21.15.43:0/3020198186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a5819fd20 con 0x7f6a5810b6d0 2024-04-25T10:35:54.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.617+0000 7f6a55ffb640 1 -- 172.21.15.43:0/3020198186 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f6a48017070 con 0x7f6a5810b6d0 2024-04-25T10:35:54.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.618+0000 7f6a65fad740 1 Processor -- start 2024-04-25T10:35:54.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.618+0000 7f6a65fad740 1 -- start start 2024-04-25T10:35:54.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.618+0000 7f6a65fad740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f62ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.618+0000 7f6a65fad740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f2634d0 0x55618f62cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.618+0000 7f6a65fad740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f26b6b0 0x55618f6398f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.618+0000 7f6a65fad740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55618f592d80 con 0x55618f28c390 2024-04-25T10:35:54.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.618+0000 7f6a65fad740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55618f54c8b0 con 0x55618f26b6b0 2024-04-25T10:35:54.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.618+0000 7f6a65fad740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55618f58ed40 con 0x55618f2634d0 2024-04-25T10:35:54.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.618+0000 7f6a5d696640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f62ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.618+0000 7f6a55ffb640 1 --2- 172.21.15.43:0/3020198186 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6a440afc60 0x7f6a440b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.618+0000 7f6a5d696640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f62ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43194/0 (socket says 172.21.15.43:43194) 2024-04-25T10:35:54.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.618+0000 7f6a5d696640 1 -- 172.21.15.43:0/1807226149 learned_addr learned my addr 172.21.15.43:0/1807226149 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:54.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.618+0000 7f6a57fff640 1 --2- 172.21.15.43:0/1807226149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f26b6b0 0x55618f6398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.618+0000 7f6a5ce95640 1 --2- 172.21.15.43:0/1807226149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f2634d0 0x55618f62cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.618+0000 7f6a5d696640 1 --2- 172.21.15.43:0/3020198186 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6a440afc60 0x7f6a440b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.618+0000 7f6a5d696640 1 -- 172.21.15.43:0/1807226149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f2634d0 msgr2=0x55618f62cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:54.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.618+0000 7f6a5d696640 1 --2- 172.21.15.43:0/1807226149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f2634d0 0x55618f62cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.618+0000 7f6a5d696640 1 -- 172.21.15.43:0/1807226149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f26b6b0 msgr2=0x55618f6398f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:54.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.618+0000 7f6a55ffb640 1 -- 172.21.15.43:0/3020198186 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6a480deae0 con 0x7f6a5810b6d0 2024-04-25T10:35:54.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.619+0000 7f6a5d696640 1 --2- 172.21.15.43:0/1807226149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f26b6b0 0x55618f6398f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.619+0000 7f6a5d696640 1 -- 172.21.15.43:0/1807226149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55618f51da40 con 0x55618f28c390 2024-04-25T10:35:54.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.619+0000 7f6a5ce95640 1 --2- 172.21.15.43:0/1807226149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f2634d0 0x55618f62cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:54.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.619+0000 7f6a5d696640 1 --2- 172.21.15.43:0/1807226149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f62ca60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009900 tx=0x7f6a40032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d05698568a6e27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:54.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.619+0000 7f6a4effd640 1 -- 172.21.15.43:0/1807226149 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40035040 con 0x55618f28c390 2024-04-25T10:35:54.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.619+0000 7f6a4effd640 1 -- 172.21.15.43:0/1807226149 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a400054f0 con 0x55618f28c390 2024-04-25T10:35:54.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.619+0000 7f6a4effd640 1 -- 172.21.15.43:0/1807226149 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a400035f0 con 0x55618f28c390 2024-04-25T10:35:54.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.620+0000 7f6a65fad740 1 -- 172.21.15.43:0/1807226149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 msgr2=0x55618f62ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:54.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.620+0000 7f6a5d696640 1 --2- 172.21.15.43:0/3020198186 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6a440afc60 0x7f6a440b2170 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6a4004b000 tx=0x7f6a40049000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:54.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.620+0000 7f6a65fad740 1 --2- 172.21.15.43:0/1807226149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f62ca60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009900 tx=0x7f6a40032f90 comp rx=0 tx=0).stop 2024-04-25T10:35:54.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.620+0000 7f6a65fad740 1 -- 172.21.15.43:0/1807226149 shutdown_connections 2024-04-25T10:35:54.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.620+0000 7f6a65fad740 1 --2- 172.21.15.43:0/1807226149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f26b6b0 0x55618f6398f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.620+0000 7f6a65fad740 1 --2- 172.21.15.43:0/1807226149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f2634d0 0x55618f62cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.620+0000 7f6a65fad740 1 --2- 172.21.15.43:0/1807226149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f62ca60 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.620+0000 7f6a65fad740 1 -- 172.21.15.43:0/1807226149 >> 172.21.15.43:0/1807226149 conn(0x55618f2804e0 msgr2=0x55618f629210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:54.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.620+0000 7f6a65fad740 1 -- 172.21.15.43:0/1807226149 shutdown_connections 2024-04-25T10:35:54.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.620+0000 7f6a65fad740 1 -- 172.21.15.43:0/1807226149 wait complete. 2024-04-25T10:35:54.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.620+0000 7f6a65fad740 1 -- start start 2024-04-25T10:35:54.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.620+0000 7f6a65fad740 1 Processor -- start 2024-04-25T10:35:54.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.621+0000 7f6a65fad740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f2634d0 0x55618f638fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.621+0000 7f6a65fad740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f26b6b0 0x55618f6394f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.621+0000 7f6a65fad740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f6374a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.621+0000 7f6a65fad740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55618f639fe0 con 0x55618f28c390 2024-04-25T10:35:54.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.621+0000 7f6a65fad740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55618f519d10 con 0x55618f26b6b0 2024-04-25T10:35:54.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.621+0000 7f6a65fad740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55618f63a160 con 0x55618f2634d0 2024-04-25T10:35:54.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.621+0000 7f6a65fad740 10 client.0 _ll_register_callbacks cb 0x55618f546210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:54.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.621+0000 7f6a5d696640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f6374a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.621+0000 7f6a5d696640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f6374a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43196/0 (socket says 172.21.15.43:43196) 2024-04-25T10:35:54.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.621+0000 7f6a5d696640 1 -- 172.21.15.43:0/2851038289 learned_addr learned my addr 172.21.15.43:0/2851038289 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:54.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.621+0000 7f6a57fff640 1 --2- 172.21.15.43:0/2851038289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f26b6b0 0x55618f6394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.621+0000 7f6a5ce95640 1 --2- 172.21.15.43:0/2851038289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f2634d0 0x55618f638fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.621+0000 7f6a5d696640 1 -- 172.21.15.43:0/2851038289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f2634d0 msgr2=0x55618f638fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:54.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.621+0000 7f6a5d696640 1 --2- 172.21.15.43:0/2851038289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f2634d0 0x55618f638fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.621+0000 7f6a5d696640 1 -- 172.21.15.43:0/2851038289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f26b6b0 msgr2=0x55618f6394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:54.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.621+0000 7f6a5d696640 1 --2- 172.21.15.43:0/2851038289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f26b6b0 0x55618f6394f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.621+0000 7f6a5d696640 1 -- 172.21.15.43:0/2851038289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55618f36e7d0 con 0x55618f28c390 2024-04-25T10:35:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.621+0000 7f6a5ce95640 1 --2- 172.21.15.43:0/2851038289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f2634d0 0x55618f638fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.622+0000 7f6a5d696640 1 --2- 172.21.15.43:0/2851038289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f6374a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6a4002a430 tx=0x7f6a40004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247c49aaa5a03382 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.622+0000 7f6a37fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.622+0000 7f6a37fff640 1 -- 172.21.15.43:0/2851038289 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4005b020 con 0x55618f28c390 2024-04-25T10:35:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.622+0000 7f6a37fff640 1 -- 172.21.15.43:0/2851038289 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6a40003a50 con 0x55618f28c390 2024-04-25T10:35:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.622+0000 7f6a65fad740 1 -- 172.21.15.43:0/2851038289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a400054f0 con 0x55618f28c390 2024-04-25T10:35:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.622+0000 7f6a37fff640 1 -- 172.21.15.43:0/2851038289 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6a40066050 con 0x55618f28c390 2024-04-25T10:35:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.622+0000 7f6a65fad740 10 client.16479 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.622+0000 7f6a65fad740 1 -- 172.21.15.43:0/2851038289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55618f2a43f0 con 0x55618f28c390 2024-04-25T10:35:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.622+0000 7f6a37fff640 1 -- 172.21.15.43:0/2851038289 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a400054f0 con 0x55618f28c390 2024-04-25T10:35:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.622+0000 7f6a37fff640 1 -- 172.21.15.43:0/2851038289 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6a40003c50 con 0x55618f28c390 2024-04-25T10:35:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.622+0000 7f6a65fad740 10 client.16479 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.622+0000 7f6a65fad740 10 client.16479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.622+0000 7f6a65fad740 20 client.16479 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.622+0000 7f6a65fad740 1 --2- 172.21.15.43:0/2851038289 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55618f24af60 0x55618f6495a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.622+0000 7f6a65fad740 4 client.16479 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:35:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.622+0000 7f6a65fad740 1 -- 172.21.15.43:0/2851038289 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55618f5a0990 con 0x55618f24af60 2024-04-25T10:35:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.623+0000 7f6a5ce95640 1 --2- 172.21.15.43:0/2851038289 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55618f24af60 0x55618f6495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.623+0000 7f6a5ce95640 1 --2- 172.21.15.43:0/2851038289 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55618f24af60 0x55618f6495a0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa38933f27b68901 server_cookie=f4bbaea4c8a2b7de in_seq=0 out_seq=0 2024-04-25T10:35:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.623+0000 7f6a37fff640 10 client.16479 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.624+0000 7f6a37fff640 1 -- 172.21.15.43:0/2851038289 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55618f5a0990 con 0x55618f24af60 2024-04-25T10:35:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.624+0000 7f6a37fff640 10 client.16479 handle_command_reply: tid=0 2024-04-25T10:35:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.624+0000 7f6a65fad740 1 client.16479 shutdown 2024-04-25T10:35:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.624+0000 7f6a65fad740 1 -- 172.21.15.43:0/2851038289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 msgr2=0x55618f6374a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.624+0000 7f6a65fad740 1 --2- 172.21.15.43:0/2851038289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f6374a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6a4002a430 tx=0x7f6a40004580 comp rx=0 tx=0).stop 2024-04-25T10:35:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.624+0000 7f6a65fad740 1 -- 172.21.15.43:0/2851038289 shutdown_connections 2024-04-25T10:35:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.624+0000 7f6a65fad740 1 --2- 172.21.15.43:0/2851038289 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55618f24af60 0x55618f6495a0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.624+0000 7f6a65fad740 1 --2- 172.21.15.43:0/2851038289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f26b6b0 0x55618f6394f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.624+0000 7f6a65fad740 1 --2- 172.21.15.43:0/2851038289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f6374a0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.624+0000 7f6a37fff640 0 client.16479 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:54.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.624+0000 7f6a65fad740 1 --2- 172.21.15.43:0/2851038289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f2634d0 0x55618f638fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.624+0000 7f6a65fad740 1 -- 172.21.15.43:0/2851038289 >> 172.21.15.43:0/2851038289 conn(0x55618f2804e0 msgr2=0x55618f62e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:54.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.624+0000 7f6a65fad740 1 -- 172.21.15.43:0/2851038289 shutdown_connections 2024-04-25T10:35:54.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.624+0000 7f6a65fad740 1 -- 172.21.15.43:0/2851038289 wait complete. 2024-04-25T10:35:54.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.624+0000 7f6a65fad740 20 client.16479 trim_cache size 0 max 16384 2024-04-25T10:35:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.642+0000 7f6a65fad740 1 Processor -- start 2024-04-25T10:35:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.642+0000 7f6a65fad740 1 -- start start 2024-04-25T10:35:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a65fad740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f2634d0 0x55618f62ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a65fad740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f26b6b0 0x55618f62ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a65fad740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f64c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a65fad740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6a581b2220 con 0x55618f28c390 2024-04-25T10:35:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a65fad740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6a581afaf0 con 0x55618f2634d0 2024-04-25T10:35:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a65fad740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6a581ad940 con 0x55618f26b6b0 2024-04-25T10:35:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a5d696640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f64c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a5d696640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f64c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43210/0 (socket says 172.21.15.43:43210) 2024-04-25T10:35:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a5ce95640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f2634d0 0x55618f62ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a5d696640 1 -- 172.21.15.43:0/4125988787 learned_addr learned my addr 172.21.15.43:0/4125988787 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a57fff640 1 --2- 172.21.15.43:0/4125988787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f26b6b0 0x55618f62ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a5d696640 1 -- 172.21.15.43:0/4125988787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f26b6b0 msgr2=0x55618f62ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a5d696640 1 --2- 172.21.15.43:0/4125988787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f26b6b0 0x55618f62ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a5d696640 1 -- 172.21.15.43:0/4125988787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f2634d0 msgr2=0x55618f62ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a5d696640 1 --2- 172.21.15.43:0/4125988787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f2634d0 0x55618f62ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a5d696640 1 -- 172.21.15.43:0/4125988787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55618f2640f0 con 0x55618f28c390 2024-04-25T10:35:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a5ce95640 1 --2- 172.21.15.43:0/4125988787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f2634d0 0x55618f62ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:35:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a5d696640 1 --2- 172.21.15.43:0/4125988787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f64c5d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6a40032c20 tx=0x7f6a4002ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd7bd44f2397058 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a4e7fc640 1 -- 172.21.15.43:0/4125988787 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40035030 con 0x55618f28c390 2024-04-25T10:35:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a4e7fc640 1 -- 172.21.15.43:0/4125988787 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a40053450 con 0x55618f28c390 2024-04-25T10:35:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.643+0000 7f6a4e7fc640 1 -- 172.21.15.43:0/4125988787 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4005b070 con 0x55618f28c390 2024-04-25T10:35:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.644+0000 7f6a65fad740 1 -- 172.21.15.43:0/4125988787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 msgr2=0x55618f64c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.644+0000 7f6a65fad740 1 --2- 172.21.15.43:0/4125988787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f64c5d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6a40032c20 tx=0x7f6a4002ab70 comp rx=0 tx=0).stop 2024-04-25T10:35:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.644+0000 7f6a65fad740 1 -- 172.21.15.43:0/4125988787 shutdown_connections 2024-04-25T10:35:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.644+0000 7f6a65fad740 1 --2- 172.21.15.43:0/4125988787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f64c5d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.644+0000 7f6a65fad740 1 --2- 172.21.15.43:0/4125988787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f26b6b0 0x55618f62ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.644+0000 7f6a65fad740 1 --2- 172.21.15.43:0/4125988787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f2634d0 0x55618f62ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.644+0000 7f6a65fad740 1 -- 172.21.15.43:0/4125988787 >> 172.21.15.43:0/4125988787 conn(0x55618f2804e0 msgr2=0x55618f62e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.644+0000 7f6a65fad740 1 -- 172.21.15.43:0/4125988787 shutdown_connections 2024-04-25T10:35:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.644+0000 7f6a65fad740 1 -- 172.21.15.43:0/4125988787 wait complete. 2024-04-25T10:35:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.645+0000 7f6a65fad740 1 -- start start 2024-04-25T10:35:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.645+0000 7f6a65fad740 1 Processor -- start 2024-04-25T10:35:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.645+0000 7f6a65fad740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f2634d0 0x55618f64bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.645+0000 7f6a65fad740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f26b6b0 0x55618f64c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.645+0000 7f6a65fad740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f6a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.645+0000 7f6a65fad740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6a581adfb0 con 0x55618f28c390 2024-04-25T10:35:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.645+0000 7f6a65fad740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6a581af420 con 0x55618f26b6b0 2024-04-25T10:35:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.645+0000 7f6a65fad740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6a581ae680 con 0x55618f2634d0 2024-04-25T10:35:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.645+0000 7f6a65fad740 10 client.0 _ll_register_callbacks cb 0x7f6a480a2b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:35:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.645+0000 7f6a5d696640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f6a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.645+0000 7f6a5d696640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f6a97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43214/0 (socket says 172.21.15.43:43214) 2024-04-25T10:35:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.645+0000 7f6a5d696640 1 -- 172.21.15.43:0/3188996195 learned_addr learned my addr 172.21.15.43:0/3188996195 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:35:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.645+0000 7f6a5ce95640 1 --2- 172.21.15.43:0/3188996195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f2634d0 0x55618f64bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.645+0000 7f6a57fff640 1 --2- 172.21.15.43:0/3188996195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f26b6b0 0x55618f64c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.646+0000 7f6a5d696640 1 -- 172.21.15.43:0/3188996195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f2634d0 msgr2=0x55618f64bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.646+0000 7f6a5d696640 1 --2- 172.21.15.43:0/3188996195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f2634d0 0x55618f64bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.646+0000 7f6a5d696640 1 -- 172.21.15.43:0/3188996195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f26b6b0 msgr2=0x55618f64c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.646+0000 7f6a5d696640 1 --2- 172.21.15.43:0/3188996195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f26b6b0 0x55618f64c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.646+0000 7f6a5d696640 1 -- 172.21.15.43:0/3188996195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55618f5534b0 con 0x55618f28c390 2024-04-25T10:35:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.646+0000 7f6a5d696640 1 --2- 172.21.15.43:0/3188996195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f6a97f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009900 tx=0x7f6a40005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78e27804d51fcb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.646+0000 7f6a37fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:35:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.646+0000 7f6a37fff640 1 -- 172.21.15.43:0/3188996195 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4005b040 con 0x55618f28c390 2024-04-25T10:35:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.646+0000 7f6a37fff640 1 -- 172.21.15.43:0/3188996195 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6a400054e0 con 0x55618f28c390 2024-04-25T10:35:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.646+0000 7f6a65fad740 1 -- 172.21.15.43:0/3188996195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55618f633080 con 0x55618f28c390 2024-04-25T10:35:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.646+0000 7f6a37fff640 1 -- 172.21.15.43:0/3188996195 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6a40004990 con 0x55618f28c390 2024-04-25T10:35:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.646+0000 7f6a65fad740 10 client.16485 fetch_fsmap learned FSMap version 26 2024-04-25T10:35:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.646+0000 7f6a65fad740 1 -- 172.21.15.43:0/3188996195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55618f6b0ed0 con 0x55618f28c390 2024-04-25T10:35:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.646+0000 7f6a37fff640 1 -- 172.21.15.43:0/3188996195 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55618f633080 con 0x55618f28c390 2024-04-25T10:35:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.646+0000 7f6a37fff640 1 -- 172.21.15.43:0/3188996195 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6a40071030 con 0x55618f28c390 2024-04-25T10:35:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.647+0000 7f6a65fad740 10 client.16485 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:35:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.647+0000 7f6a65fad740 10 client.16485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:35:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.647+0000 7f6a65fad740 20 client.16485 populate_metadata read hostname 'smithi043' 2024-04-25T10:35:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.647+0000 7f6a65fad740 1 --2- 172.21.15.43:0/3188996195 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55618f2203a0 0x55618f6b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.647+0000 7f6a65fad740 4 client.16485 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:35:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.647+0000 7f6a65fad740 1 -- 172.21.15.43:0/3188996195 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55618f59a800 con 0x55618f2203a0 2024-04-25T10:35:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.647+0000 7f6a5ce95640 1 --2- 172.21.15.43:0/3188996195 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55618f2203a0 0x55618f6b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.647+0000 7f6a37fff640 10 client.16485 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.647+0000 7f6a5ce95640 1 --2- 172.21.15.43:0/3188996195 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55618f2203a0 0x55618f6b3490 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7e1209e3ac70110 server_cookie=2c2fd795412640b1 in_seq=0 out_seq=0 2024-04-25T10:35:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.647+0000 7f6a37fff640 1 -- 172.21.15.43:0/3188996195 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55618f59a800 con 0x55618f2203a0 2024-04-25T10:35:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.647+0000 7f6a37fff640 10 client.16485 handle_command_reply: tid=0 2024-04-25T10:35:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.647+0000 7f6a65fad740 1 client.16485 shutdown 2024-04-25T10:35:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.648+0000 7f6a65fad740 1 -- 172.21.15.43:0/3188996195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 msgr2=0x55618f6a97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.648+0000 7f6a65fad740 1 --2- 172.21.15.43:0/3188996195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f6a97f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009900 tx=0x7f6a40005900 comp rx=0 tx=0).stop 2024-04-25T10:35:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.648+0000 7f6a65fad740 1 -- 172.21.15.43:0/3188996195 shutdown_connections 2024-04-25T10:35:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.648+0000 7f6a65fad740 1 --2- 172.21.15.43:0/3188996195 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55618f2203a0 0x55618f6b3490 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.648+0000 7f6a65fad740 1 --2- 172.21.15.43:0/3188996195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f26b6b0 0x55618f64c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.648+0000 7f6a65fad740 1 --2- 172.21.15.43:0/3188996195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55618f28c390 0x55618f6a97f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.648+0000 7f6a37fff640 0 client.16485 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:35:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.648+0000 7f6a65fad740 1 --2- 172.21.15.43:0/3188996195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55618f2634d0 0x55618f64bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.648+0000 7f6a65fad740 1 -- 172.21.15.43:0/3188996195 >> 172.21.15.43:0/3188996195 conn(0x55618f2804e0 msgr2=0x55618f6367a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.648+0000 7f6a65fad740 1 -- 172.21.15.43:0/3188996195 shutdown_connections 2024-04-25T10:35:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.648+0000 7f6a65fad740 1 -- 172.21.15.43:0/3188996195 wait complete. 2024-04-25T10:35:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.648+0000 7f6a65fad740 20 client.16485 trim_cache size 0 max 16384 2024-04-25T10:35:54.658 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:35:54.658 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:35:54.658 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:35:54.658 INFO:teuthology.orchestra.run.smithi043.stdout: "896dafb5-b61c-4f68-8927-5d7dfe730a4e": { 2024-04-25T10:35:54.658 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:35:54.658 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "896dafb5-b61c-4f68-8927-5d7dfe730a4e", 2024-04-25T10:35:54.658 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:35:54.658 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:35:54.658 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:35:54.659 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:35:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.648+0000 7f6a5f0f9640 1 -- 172.21.15.43:0/3020198186 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6a440afc60 msgr2=0x7f6a440b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.648+0000 7f6a5f0f9640 1 --2- 172.21.15.43:0/3020198186 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6a440afc60 0x7f6a440b2170 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6a4004b000 tx=0x7f6a40049000 comp rx=0 tx=0).stop 2024-04-25T10:35:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.648+0000 7f6a5f0f9640 1 -- 172.21.15.43:0/3020198186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a5810b6d0 msgr2=0x7f6a5819ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:54.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.648+0000 7f6a5f0f9640 1 --2- 172.21.15.43:0/3020198186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a5810b6d0 0x7f6a5819ac60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800c9a0 tx=0x7f6a4800ce60 comp rx=0 tx=0).stop 2024-04-25T10:35:54.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.648+0000 7f6a5f0f9640 1 -- 172.21.15.43:0/3020198186 shutdown_connections 2024-04-25T10:35:54.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.648+0000 7f6a5f0f9640 1 --2- 172.21.15.43:0/3020198186 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6a440afc60 0x7f6a440b2170 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.648+0000 7f6a5f0f9640 1 --2- 172.21.15.43:0/3020198186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a5810e0c0 0x7f6a5819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.648+0000 7f6a5f0f9640 1 --2- 172.21.15.43:0/3020198186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a5810b6d0 0x7f6a5819ac60 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.649+0000 7f6a5f0f9640 1 --2- 172.21.15.43:0/3020198186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a58104c70 0x7f6a5819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:54.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.649+0000 7f6a5f0f9640 1 -- 172.21.15.43:0/3020198186 >> 172.21.15.43:0/3020198186 conn(0x7f6a58078110 msgr2=0x7f6a58101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:54.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.649+0000 7f6a5f0f9640 1 -- 172.21.15.43:0/3020198186 shutdown_connections 2024-04-25T10:35:54.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:35:54.649+0000 7f6a5f0f9640 1 -- 172.21.15.43:0/3020198186 wait complete. 2024-04-25T10:35:54.663 INFO:tasks.cephfs.filesystem:scrub status for tag:896dafb5-b61c-4f68-8927-5d7dfe730a4e - {'path': '/', 'tag': '896dafb5-b61c-4f68-8927-5d7dfe730a4e', 'options': 'recursive,force'} 2024-04-25T10:35:56.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:55 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:35:56.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:55 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:35:56.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:55 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:35:56.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:55 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:35:56.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:55 smithi077 ceph-mon[32772]: pgmap v105: 97 pgs: 97 active+clean; 3.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-04-25T10:35:56.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:55 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:35:56.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:55 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:35:56.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:55 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:35:56.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:55 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:35:56.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:55 smithi164 ceph-mon[32713]: pgmap v105: 97 pgs: 97 active+clean; 3.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-04-25T10:35:56.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:55 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:35:56.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:55 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:35:56.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:55 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:35:56.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:55 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:35:56.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:55 smithi043 ceph-mon[29478]: pgmap v105: 97 pgs: 97 active+clean; 3.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-04-25T10:35:58.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:57 smithi077 ceph-mon[32772]: pgmap v106: 97 pgs: 97 active+clean; 3.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-04-25T10:35:58.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:57 smithi164 ceph-mon[32713]: pgmap v106: 97 pgs: 97 active+clean; 3.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-04-25T10:35:58.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:57 smithi043 ceph-mon[29478]: pgmap v106: 97 pgs: 97 active+clean; 3.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-04-25T10:35:59.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:58 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:35:59.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:58 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:35:59.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:58 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:36:00.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:35:59 smithi077 ceph-mon[32772]: pgmap v107: 97 pgs: 97 active+clean; 3.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-04-25T10:36:00.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:35:59 smithi164 ceph-mon[32713]: pgmap v107: 97 pgs: 97 active+clean; 3.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-04-25T10:36:00.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:35:59 smithi043 ceph-mon[29478]: pgmap v107: 97 pgs: 97 active+clean; 3.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-04-25T10:36:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.104+0000 7fcc08430640 1 -- 172.21.15.43:0/4091496592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0010b6c0 msgr2=0x7fcc0010dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.104+0000 7fcc08430640 1 --2- 172.21.15.43:0/4091496592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0010b6c0 0x7fcc0010dae0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009900 tx=0x7fcbf002a410 comp rx=0 tx=0).stop 2024-04-25T10:36:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.104+0000 7fcc08430640 1 -- 172.21.15.43:0/4091496592 shutdown_connections 2024-04-25T10:36:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.104+0000 7fcc08430640 1 --2- 172.21.15.43:0/4091496592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc0010e0b0 0x7fcc001105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.104+0000 7fcc08430640 1 --2- 172.21.15.43:0/4091496592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0010b6c0 0x7fcc0010dae0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.104+0000 7fcc08430640 1 --2- 172.21.15.43:0/4091496592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc00104c60 0x7fcc00105070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.104+0000 7fcc08430640 1 -- 172.21.15.43:0/4091496592 >> 172.21.15.43:0/4091496592 conn(0x7fcc000fe260 msgr2=0x7fcc00100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:01.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.104+0000 7fcc08430640 1 -- 172.21.15.43:0/4091496592 shutdown_connections 2024-04-25T10:36:01.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.104+0000 7fcc08430640 1 -- 172.21.15.43:0/4091496592 wait complete. 2024-04-25T10:36:01.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.105+0000 7fcc08430640 1 Processor -- start 2024-04-25T10:36:01.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.105+0000 7fcc08430640 1 -- start start 2024-04-25T10:36:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.105+0000 7fcc08430640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc00104c60 0x7fcc001a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.105+0000 7fcc08430640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0010b6c0 0x7fcc001a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.105+0000 7fcc08430640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc0010e0b0 0x7fcc0019c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.105+0000 7fcc08430640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcc00113d10 con 0x7fcc0010b6c0 2024-04-25T10:36:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.105+0000 7fcc08430640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc00113b90 con 0x7fcc0010e0b0 2024-04-25T10:36:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.105+0000 7fcc08430640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcc00113e90 con 0x7fcc00104c60 2024-04-25T10:36:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.106+0000 7fcc059cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0010b6c0 0x7fcc001a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.106+0000 7fcc059cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0010b6c0 0x7fcc001a2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37804/0 (socket says 172.21.15.43:37804) 2024-04-25T10:36:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.106+0000 7fcc059cb640 1 -- 172.21.15.43:0/3301028112 learned_addr learned my addr 172.21.15.43:0/3301028112 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.106+0000 7fcc069cd640 1 --2- 172.21.15.43:0/3301028112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc0010e0b0 0x7fcc0019c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.106+0000 7fcc061cc640 1 --2- 172.21.15.43:0/3301028112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc00104c60 0x7fcc001a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.106+0000 7fcc059cb640 1 -- 172.21.15.43:0/3301028112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc00104c60 msgr2=0x7fcc001a2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.106+0000 7fcc059cb640 1 --2- 172.21.15.43:0/3301028112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc00104c60 0x7fcc001a2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.106+0000 7fcc059cb640 1 -- 172.21.15.43:0/3301028112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc0010e0b0 msgr2=0x7fcc0019c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.106+0000 7fcc059cb640 1 --2- 172.21.15.43:0/3301028112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc0010e0b0 0x7fcc0019c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.106+0000 7fcc059cb640 1 -- 172.21.15.43:0/3301028112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0019cec0 con 0x7fcc0010b6c0 2024-04-25T10:36:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.106+0000 7fcc059cb640 1 --2- 172.21.15.43:0/3301028112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0010b6c0 0x7fcc001a2b50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0005940 tx=0x7fcbf00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.106+0000 7fcbf77fe640 1 -- 172.21.15.43:0/3301028112 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf003f030 con 0x7fcc0010b6c0 2024-04-25T10:36:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.106+0000 7fcbf77fe640 1 -- 172.21.15.43:0/3301028112 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf0003cb0 con 0x7fcc0010b6c0 2024-04-25T10:36:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.106+0000 7fcc08430640 1 -- 172.21.15.43:0/3301028112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc0019d0f0 con 0x7fcc0010b6c0 2024-04-25T10:36:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.106+0000 7fcbf77fe640 1 -- 172.21.15.43:0/3301028112 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0035070 con 0x7fcc0010b6c0 2024-04-25T10:36:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.106+0000 7fcc08430640 1 -- 172.21.15.43:0/3301028112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc001a93b0 con 0x7fcc0010b6c0 2024-04-25T10:36:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.109+0000 7fcbf77fe640 1 -- 172.21.15.43:0/3301028112 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fcbf0042030 con 0x7fcc0010b6c0 2024-04-25T10:36:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.109+0000 7fcc0f3b2740 1 Processor -- start 2024-04-25T10:36:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.110+0000 7fcc0f3b2740 1 -- start start 2024-04-25T10:36:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.110+0000 7fcc0f3b2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a41ec81b0 0x556a42267af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.110+0000 7fcc0f3b2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41e9f2f0 0x556a42268030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.110+0000 7fcc0f3b2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a42268570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.110+0000 7fcc0f3b2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556a421ce460 con 0x556a41ea74d0 2024-04-25T10:36:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.110+0000 7fcc0f3b2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556a42188590 con 0x556a41e9f2f0 2024-04-25T10:36:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.110+0000 7fcc0f3b2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556a421ca420 con 0x556a41ec81b0 2024-04-25T10:36:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.110+0000 7fcc059cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a42268570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.110+0000 7fcc059cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a42268570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37814/0 (socket says 172.21.15.43:37814) 2024-04-25T10:36:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.110+0000 7fcc059cb640 1 -- 172.21.15.43:0/4158578928 learned_addr learned my addr 172.21.15.43:0/4158578928 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.110+0000 7fcc061cc640 1 --2- 172.21.15.43:0/4158578928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41e9f2f0 0x556a42268030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.110+0000 7fcc069cd640 1 --2- 172.21.15.43:0/4158578928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a41ec81b0 0x556a42267af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.110+0000 7fcbf77fe640 1 --2- 172.21.15.43:0/3301028112 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcbec0afcd0 0x7fcbec0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.111+0000 7fcc069cd640 1 --2- 172.21.15.43:0/3301028112 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcbec0afcd0 0x7fcbec0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.111+0000 7fcc059cb640 1 -- 172.21.15.43:0/4158578928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a41ec81b0 msgr2=0x556a42267af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.111+0000 7fcc059cb640 1 --2- 172.21.15.43:0/4158578928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a41ec81b0 0x556a42267af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.111+0000 7fcc059cb640 1 -- 172.21.15.43:0/4158578928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41e9f2f0 msgr2=0x556a42268030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.111+0000 7fcc059cb640 1 --2- 172.21.15.43:0/4158578928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41e9f2f0 0x556a42268030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.111+0000 7fcbf77fe640 1 -- 172.21.15.43:0/3301028112 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fcbf00ff7d0 con 0x7fcc0010b6c0 2024-04-25T10:36:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.111+0000 7fcc059cb640 1 -- 172.21.15.43:0/4158578928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a421950d0 con 0x556a41ea74d0 2024-04-25T10:36:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.111+0000 7fcc059cb640 1 --2- 172.21.15.43:0/4158578928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a42268570 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00fcec0 tx=0x7fcbf002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533cf6e141f61daa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.111+0000 7fcc069cd640 1 --2- 172.21.15.43:0/3301028112 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcbec0afcd0 0x7fcbec0b21e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x556a42264c60 tx=0x7fcbf8004ab0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.111+0000 7fcbf4bfe640 1 -- 172.21.15.43:0/4158578928 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0049070 con 0x556a41ea74d0 2024-04-25T10:36:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.111+0000 7fcbf4bfe640 1 -- 172.21.15.43:0/4158578928 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf00c7560 con 0x556a41ea74d0 2024-04-25T10:36:01.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.111+0000 7fcbf4bfe640 1 -- 172.21.15.43:0/4158578928 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf00d0800 con 0x556a41ea74d0 2024-04-25T10:36:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.112+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/4158578928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 msgr2=0x556a42268570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.112+0000 7fcc0f3b2740 1 --2- 172.21.15.43:0/4158578928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a42268570 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00fcec0 tx=0x7fcbf002a410 comp rx=0 tx=0).stop 2024-04-25T10:36:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.112+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/4158578928 shutdown_connections 2024-04-25T10:36:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.112+0000 7fcc0f3b2740 1 --2- 172.21.15.43:0/4158578928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41e9f2f0 0x556a42268030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.112+0000 7fcc0f3b2740 1 --2- 172.21.15.43:0/4158578928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a42268570 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.112+0000 7fcc0f3b2740 1 --2- 172.21.15.43:0/4158578928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a41ec81b0 0x556a42267af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.112+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/4158578928 >> 172.21.15.43:0/4158578928 conn(0x556a41ebc300 msgr2=0x556a422642a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.112+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/4158578928 shutdown_connections 2024-04-25T10:36:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.112+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/4158578928 wait complete. 2024-04-25T10:36:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.113+0000 7fcc0f3b2740 1 -- start start 2024-04-25T10:36:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.113+0000 7fcc0f3b2740 1 Processor -- start 2024-04-25T10:36:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.113+0000 7fcc0f3b2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a41e9f2f0 0x556a42272180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.113+0000 7fcc0f3b2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a422726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.113+0000 7fcc0f3b2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41ec81b0 0x556a42277d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.113+0000 7fcc0f3b2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556a422749e0 con 0x556a41ea74d0 2024-04-25T10:36:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.113+0000 7fcc0f3b2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556a42155b60 con 0x556a41ec81b0 2024-04-25T10:36:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.113+0000 7fcc0f3b2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556a42274b60 con 0x556a41e9f2f0 2024-04-25T10:36:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.113+0000 7fcc0f3b2740 10 client.0 _ll_register_callbacks cb 0x556a4217a2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.113+0000 7fcc059cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a422726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.113+0000 7fcc059cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a422726c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37826/0 (socket says 172.21.15.43:37826) 2024-04-25T10:36:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.113+0000 7fcc059cb640 1 -- 172.21.15.43:0/929255830 learned_addr learned my addr 172.21.15.43:0/929255830 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.113+0000 7fcc069cd640 1 --2- 172.21.15.43:0/929255830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41ec81b0 0x556a42277d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.113+0000 7fcc061cc640 1 --2- 172.21.15.43:0/929255830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a41e9f2f0 0x556a42272180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.114+0000 7fcc059cb640 1 -- 172.21.15.43:0/929255830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a41e9f2f0 msgr2=0x556a42272180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.114+0000 7fcc059cb640 1 --2- 172.21.15.43:0/929255830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a41e9f2f0 0x556a42272180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.114+0000 7fcc059cb640 1 -- 172.21.15.43:0/929255830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41ec81b0 msgr2=0x556a42277d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.114+0000 7fcc059cb640 1 --2- 172.21.15.43:0/929255830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41ec81b0 0x556a42277d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.114+0000 7fcc059cb640 1 -- 172.21.15.43:0/929255830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a41ee0210 con 0x556a41ea74d0 2024-04-25T10:36:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.114+0000 7fcc069cd640 1 --2- 172.21.15.43:0/929255830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41ec81b0 0x556a42277d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.114+0000 7fcc059cb640 1 --2- 172.21.15.43:0/929255830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a422726c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00fd000 tx=0x7fcbf00fcf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1589852198e0f7ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.114+0000 7fcbe9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.114+0000 7fcbe9ffb640 1 -- 172.21.15.43:0/929255830 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0049040 con 0x556a41ea74d0 2024-04-25T10:36:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.114+0000 7fcbe9ffb640 1 -- 172.21.15.43:0/929255830 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fcbf00fd1d0 con 0x556a41ea74d0 2024-04-25T10:36:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.114+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/929255830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbf00c7560 con 0x556a41ea74d0 2024-04-25T10:36:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.114+0000 7fcbe9ffb640 1 -- 172.21.15.43:0/929255830 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcbf00c7820 con 0x556a41ea74d0 2024-04-25T10:36:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.114+0000 7fcc0f3b2740 10 client.16497 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.114+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/929255830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556a41fac620 con 0x556a41ea74d0 2024-04-25T10:36:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.115+0000 7fcbe9ffb640 1 -- 172.21.15.43:0/929255830 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf00c7560 con 0x556a41ea74d0 2024-04-25T10:36:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.115+0000 7fcbe9ffb640 1 -- 172.21.15.43:0/929255830 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcbf00ea110 con 0x556a41ea74d0 2024-04-25T10:36:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.115+0000 7fcc0f3b2740 10 client.16497 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.115+0000 7fcc0f3b2740 10 client.16497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.115+0000 7fcc0f3b2740 20 client.16497 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.115+0000 7fcc0f3b2740 1 --2- 172.21.15.43:0/929255830 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556a41e86d80 0x556a42284210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.115+0000 7fcc0f3b2740 4 client.16497 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:36:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.115+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/929255830 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a421dc070 con 0x556a41e86d80 2024-04-25T10:36:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.115+0000 7fcc061cc640 1 --2- 172.21.15.43:0/929255830 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556a41e86d80 0x556a42284210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.115+0000 7fcbe9ffb640 10 client.16497 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.115+0000 7fcc061cc640 1 --2- 172.21.15.43:0/929255830 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556a41e86d80 0x556a42284210 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70dc2012440c78dc server_cookie=5703cfe59a7de8b1 in_seq=0 out_seq=0 2024-04-25T10:36:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.116+0000 7fcbe9ffb640 1 -- 172.21.15.43:0/929255830 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556a421dc070 con 0x556a41e86d80 2024-04-25T10:36:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.116+0000 7fcbe9ffb640 10 client.16497 handle_command_reply: tid=0 2024-04-25T10:36:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.116+0000 7fcc0f3b2740 1 client.16497 shutdown 2024-04-25T10:36:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.116+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/929255830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 msgr2=0x556a422726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.116+0000 7fcc0f3b2740 1 --2- 172.21.15.43:0/929255830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a422726c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00fd000 tx=0x7fcbf00fcf40 comp rx=0 tx=0).stop 2024-04-25T10:36:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.116+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/929255830 shutdown_connections 2024-04-25T10:36:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.116+0000 7fcc0f3b2740 1 --2- 172.21.15.43:0/929255830 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556a41e86d80 0x556a42284210 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.116+0000 7fcc0f3b2740 1 --2- 172.21.15.43:0/929255830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41ec81b0 0x556a42277d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.116+0000 7fcc0f3b2740 1 --2- 172.21.15.43:0/929255830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a422726c0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.116+0000 7fcc0f3b2740 1 --2- 172.21.15.43:0/929255830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a41e9f2f0 0x556a42272180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.116+0000 7fcbe9ffb640 0 client.16497 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.116+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/929255830 >> 172.21.15.43:0/929255830 conn(0x556a41ebc300 msgr2=0x556a42269810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.117+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/929255830 shutdown_connections 2024-04-25T10:36:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.117+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/929255830 wait complete. 2024-04-25T10:36:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.117+0000 7fcc0f3b2740 20 client.16497 trim_cache size 0 max 16384 2024-04-25T10:36:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.132+0000 7fcc0f3b2740 1 Processor -- start 2024-04-25T10:36:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.132+0000 7fcc0f3b2740 1 -- start start 2024-04-25T10:36:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.132+0000 7fcc0f3b2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41e9f2f0 0x556a42277c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.132+0000 7fcc0f3b2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a422724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.132+0000 7fcc0f3b2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a41ec81b0 0x556a42273da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.132+0000 7fcc0f3b2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001b43f0 con 0x556a41ea74d0 2024-04-25T10:36:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.132+0000 7fcc0f3b2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001b1cc0 con 0x556a41e9f2f0 2024-04-25T10:36:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.132+0000 7fcc0f3b2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001afb10 con 0x556a41ec81b0 2024-04-25T10:36:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.132+0000 7fcc059cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a422724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.132+0000 7fcc059cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a422724c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37838/0 (socket says 172.21.15.43:37838) 2024-04-25T10:36:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.132+0000 7fcc059cb640 1 -- 172.21.15.43:0/1062324065 learned_addr learned my addr 172.21.15.43:0/1062324065 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.133+0000 7fcc061cc640 1 --2- 172.21.15.43:0/1062324065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41e9f2f0 0x556a42277c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.133+0000 7fcc059cb640 1 -- 172.21.15.43:0/1062324065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a41ec81b0 msgr2=0x556a42273da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.133+0000 7fcc059cb640 1 --2- 172.21.15.43:0/1062324065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a41ec81b0 0x556a42273da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.133+0000 7fcc059cb640 1 -- 172.21.15.43:0/1062324065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41e9f2f0 msgr2=0x556a42277c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.133+0000 7fcc059cb640 1 --2- 172.21.15.43:0/1062324065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41e9f2f0 0x556a42277c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.133+0000 7fcc059cb640 1 -- 172.21.15.43:0/1062324065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a422704c0 con 0x556a41ea74d0 2024-04-25T10:36:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.133+0000 7fcc059cb640 1 --2- 172.21.15.43:0/1062324065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a422724c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00fcfb0 tx=0x7fcbf002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5232ef0bd458eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.133+0000 7fcbebfff640 1 -- 172.21.15.43:0/1062324065 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0049070 con 0x556a41ea74d0 2024-04-25T10:36:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.133+0000 7fcbebfff640 1 -- 172.21.15.43:0/1062324065 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf00d0800 con 0x556a41ea74d0 2024-04-25T10:36:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.133+0000 7fcbebfff640 1 -- 172.21.15.43:0/1062324065 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf00c71c0 con 0x556a41ea74d0 2024-04-25T10:36:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.134+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/1062324065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 msgr2=0x556a422724c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.134+0000 7fcc0f3b2740 1 --2- 172.21.15.43:0/1062324065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a422724c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00fcfb0 tx=0x7fcbf002a410 comp rx=0 tx=0).stop 2024-04-25T10:36:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.134+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/1062324065 shutdown_connections 2024-04-25T10:36:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.134+0000 7fcc0f3b2740 1 --2- 172.21.15.43:0/1062324065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a41ec81b0 0x556a42273da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.134+0000 7fcc0f3b2740 1 --2- 172.21.15.43:0/1062324065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a422724c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.134+0000 7fcc0f3b2740 1 --2- 172.21.15.43:0/1062324065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41e9f2f0 0x556a42277c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.134+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/1062324065 >> 172.21.15.43:0/1062324065 conn(0x556a41ebc300 msgr2=0x556a42273690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.134+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/1062324065 shutdown_connections 2024-04-25T10:36:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.134+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/1062324065 wait complete. 2024-04-25T10:36:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.134+0000 7fcc0f3b2740 1 -- start start 2024-04-25T10:36:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.134+0000 7fcc0f3b2740 1 Processor -- start 2024-04-25T10:36:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcc0f3b2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a41e9f2f0 0x556a422794f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcc0f3b2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a42279a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcc0f3b2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41ec81b0 0x556a422ea320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcc0f3b2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001b0180 con 0x556a41ea74d0 2024-04-25T10:36:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcc0f3b2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001b15f0 con 0x556a41ec81b0 2024-04-25T10:36:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcc0f3b2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001b0850 con 0x556a41e9f2f0 2024-04-25T10:36:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcc0f3b2740 10 client.0 _ll_register_callbacks cb 0x7fcbfc00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcc059cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a42279a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcc059cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a42279a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37842/0 (socket says 172.21.15.43:37842) 2024-04-25T10:36:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcc059cb640 1 -- 172.21.15.43:0/307189931 learned_addr learned my addr 172.21.15.43:0/307189931 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcc069cd640 1 --2- 172.21.15.43:0/307189931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41ec81b0 0x556a422ea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcc061cc640 1 --2- 172.21.15.43:0/307189931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a41e9f2f0 0x556a422794f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcc059cb640 1 -- 172.21.15.43:0/307189931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a41e9f2f0 msgr2=0x556a422794f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcc059cb640 1 --2- 172.21.15.43:0/307189931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a41e9f2f0 0x556a422794f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcc059cb640 1 -- 172.21.15.43:0/307189931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41ec81b0 msgr2=0x556a422ea320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcc059cb640 1 --2- 172.21.15.43:0/307189931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41ec81b0 0x556a422ea320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcc059cb640 1 -- 172.21.15.43:0/307189931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a422ea860 con 0x556a41ea74d0 2024-04-25T10:36:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcc069cd640 1 --2- 172.21.15.43:0/307189931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41ec81b0 0x556a422ea320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcc061cc640 1 --2- 172.21.15.43:0/307189931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a41e9f2f0 0x556a422794f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:36:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcc059cb640 1 --2- 172.21.15.43:0/307189931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a42279a30 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00c79e0 tx=0x7fcbf00c71c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed84fc48358d8a95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcbe9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.135+0000 7fcbe9ffb640 1 -- 172.21.15.43:0/307189931 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0049070 con 0x556a41ea74d0 2024-04-25T10:36:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.136+0000 7fcbe9ffb640 1 -- 172.21.15.43:0/307189931 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fcbf007f200 con 0x556a41ea74d0 2024-04-25T10:36:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.136+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/307189931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbf00d0800 con 0x556a41ea74d0 2024-04-25T10:36:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.136+0000 7fcbe9ffb640 1 -- 172.21.15.43:0/307189931 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcbf00c73a0 con 0x556a41ea74d0 2024-04-25T10:36:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.136+0000 7fcc0f3b2740 10 client.16503 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.136+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/307189931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556a422ea860 con 0x556a41ea74d0 2024-04-25T10:36:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.136+0000 7fcbe9ffb640 1 -- 172.21.15.43:0/307189931 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf00d0800 con 0x556a41ea74d0 2024-04-25T10:36:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.136+0000 7fcbe9ffb640 1 -- 172.21.15.43:0/307189931 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcbf00d0370 con 0x556a41ea74d0 2024-04-25T10:36:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.136+0000 7fcc0f3b2740 10 client.16503 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.136+0000 7fcc0f3b2740 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.136+0000 7fcc0f3b2740 20 client.16503 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.136+0000 7fcc0f3b2740 1 --2- 172.21.15.43:0/307189931 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556a41e5c1c0 0x556a422edce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.136+0000 7fcc0f3b2740 4 client.16503 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:36:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.136+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/307189931 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556a421d5ee0 con 0x556a41e5c1c0 2024-04-25T10:36:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.136+0000 7fcc061cc640 1 --2- 172.21.15.43:0/307189931 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556a41e5c1c0 0x556a422edce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.137+0000 7fcc061cc640 1 --2- 172.21.15.43:0/307189931 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556a41e5c1c0 0x556a422edce0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=746f58f722b7dc13 server_cookie=4226fe1b134e8f42 in_seq=0 out_seq=0 2024-04-25T10:36:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.137+0000 7fcbe9ffb640 10 client.16503 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.153+0000 7fcbe9ffb640 1 -- 172.21.15.43:0/307189931 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556a421d5ee0 con 0x556a41e5c1c0 2024-04-25T10:36:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.153+0000 7fcbe9ffb640 10 client.16503 handle_command_reply: tid=0 2024-04-25T10:36:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.153+0000 7fcc0f3b2740 1 client.16503 shutdown 2024-04-25T10:36:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.153+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/307189931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 msgr2=0x556a42279a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.153+0000 7fcc0f3b2740 1 --2- 172.21.15.43:0/307189931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a42279a30 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00c79e0 tx=0x7fcbf00c71c0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.153+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/307189931 shutdown_connections 2024-04-25T10:36:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.153+0000 7fcc0f3b2740 1 --2- 172.21.15.43:0/307189931 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556a41e5c1c0 0x556a422edce0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.153+0000 7fcc0f3b2740 1 --2- 172.21.15.43:0/307189931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a41ec81b0 0x556a422ea320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.153+0000 7fcc0f3b2740 1 --2- 172.21.15.43:0/307189931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a41ea74d0 0x556a42279a30 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.153+0000 7fcbe9ffb640 0 client.16503 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.153+0000 7fcc0f3b2740 1 --2- 172.21.15.43:0/307189931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a41e9f2f0 0x556a422794f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.153+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/307189931 >> 172.21.15.43:0/307189931 conn(0x556a41ebc300 msgr2=0x556a42269ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.153+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/307189931 shutdown_connections 2024-04-25T10:36:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.153+0000 7fcc0f3b2740 1 -- 172.21.15.43:0/307189931 wait complete. 2024-04-25T10:36:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.153+0000 7fcc0f3b2740 20 client.16503 trim_cache size 0 max 16384 2024-04-25T10:36:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:36:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:36:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:36:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:36:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.154+0000 7fcc08430640 1 -- 172.21.15.43:0/3301028112 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcbec0afcd0 msgr2=0x7fcbec0b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.154+0000 7fcc08430640 1 --2- 172.21.15.43:0/3301028112 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcbec0afcd0 0x7fcbec0b21e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x556a42264c60 tx=0x7fcbf8004ab0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.154+0000 7fcc08430640 1 -- 172.21.15.43:0/3301028112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0010b6c0 msgr2=0x7fcc001a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.154+0000 7fcc08430640 1 --2- 172.21.15.43:0/3301028112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0010b6c0 0x7fcc001a2b50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0005940 tx=0x7fcbf00042f0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.154+0000 7fcc08430640 1 -- 172.21.15.43:0/3301028112 shutdown_connections 2024-04-25T10:36:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.154+0000 7fcc08430640 1 --2- 172.21.15.43:0/3301028112 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcbec0afcd0 0x7fcbec0b21e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.154+0000 7fcc08430640 1 --2- 172.21.15.43:0/3301028112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc0010e0b0 0x7fcc0019c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.154+0000 7fcc08430640 1 --2- 172.21.15.43:0/3301028112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0010b6c0 0x7fcc001a2b50 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.154+0000 7fcc08430640 1 --2- 172.21.15.43:0/3301028112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc00104c60 0x7fcc001a2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.154+0000 7fcc08430640 1 -- 172.21.15.43:0/3301028112 >> 172.21.15.43:0/3301028112 conn(0x7fcc000fe260 msgr2=0x7fcc00109ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.154+0000 7fcc08430640 1 -- 172.21.15.43:0/3301028112 shutdown_connections 2024-04-25T10:36:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:01.154+0000 7fcc08430640 1 -- 172.21.15.43:0/3301028112 wait complete. 2024-04-25T10:36:02.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:01 smithi077 ceph-mon[32772]: pgmap v108: 97 pgs: 97 active+clean; 3.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 11 op/s 2024-04-25T10:36:02.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:01 smithi164 ceph-mon[32713]: pgmap v108: 97 pgs: 97 active+clean; 3.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 11 op/s 2024-04-25T10:36:02.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:01 smithi043 ceph-mon[29478]: pgmap v108: 97 pgs: 97 active+clean; 3.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 11 op/s 2024-04-25T10:36:04.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:03 smithi077 ceph-mon[32772]: pgmap v109: 97 pgs: 97 active+clean; 3.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 11 op/s 2024-04-25T10:36:04.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:03 smithi164 ceph-mon[32713]: pgmap v109: 97 pgs: 97 active+clean; 3.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 11 op/s 2024-04-25T10:36:04.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:03 smithi043 ceph-mon[29478]: pgmap v109: 97 pgs: 97 active+clean; 3.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 11 op/s 2024-04-25T10:36:06.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:05 smithi077 ceph-mon[32772]: pgmap v110: 97 pgs: 97 active+clean; 3.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-25T10:36:06.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:05 smithi164 ceph-mon[32713]: pgmap v110: 97 pgs: 97 active+clean; 3.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-25T10:36:06.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:05 smithi043 ceph-mon[29478]: pgmap v110: 97 pgs: 97 active+clean; 3.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-25T10:36:08.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:07 smithi077 ceph-mon[32772]: pgmap v111: 97 pgs: 97 active+clean; 3.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-25T10:36:08.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:07 smithi164 ceph-mon[32713]: pgmap v111: 97 pgs: 97 active+clean; 3.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-25T10:36:08.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:07 smithi043 ceph-mon[29478]: pgmap v111: 97 pgs: 97 active+clean; 3.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-25T10:36:10.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:09 smithi077 ceph-mon[32772]: pgmap v112: 97 pgs: 97 active+clean; 4.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 62 KiB/s wr, 23 op/s 2024-04-25T10:36:10.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:09 smithi164 ceph-mon[32713]: pgmap v112: 97 pgs: 97 active+clean; 4.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 62 KiB/s wr, 23 op/s 2024-04-25T10:36:10.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:09 smithi043 ceph-mon[29478]: pgmap v112: 97 pgs: 97 active+clean; 4.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 62 KiB/s wr, 23 op/s 2024-04-25T10:36:12.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:11 smithi077 ceph-mon[32772]: pgmap v113: 97 pgs: 97 active+clean; 4.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-04-25T10:36:12.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:11 smithi164 ceph-mon[32713]: pgmap v113: 97 pgs: 97 active+clean; 4.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-04-25T10:36:12.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:11 smithi043 ceph-mon[29478]: pgmap v113: 97 pgs: 97 active+clean; 4.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-04-25T10:36:14.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:13 smithi077 ceph-mon[32772]: pgmap v114: 97 pgs: 97 active+clean; 4.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-04-25T10:36:14.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:13 smithi164 ceph-mon[32713]: pgmap v114: 97 pgs: 97 active+clean; 4.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-04-25T10:36:14.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:13 smithi043 ceph-mon[29478]: pgmap v114: 97 pgs: 97 active+clean; 4.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-04-25T10:36:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.266+0000 7f2663984640 1 -- 172.21.15.43:0/1111048812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f265c10e0e0 msgr2=0x7f265c1105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.266+0000 7f2663984640 1 --2- 172.21.15.43:0/1111048812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f265c10e0e0 0x7f265c1105d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2654009ca0 tx=0x7f265402a9e0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.266+0000 7f2663984640 1 -- 172.21.15.43:0/1111048812 shutdown_connections 2024-04-25T10:36:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.266+0000 7f2663984640 1 --2- 172.21.15.43:0/1111048812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f265c10b730 0x7f265c10db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.266+0000 7f2663984640 1 --2- 172.21.15.43:0/1111048812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f265c10e0e0 0x7f265c1105d0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.266+0000 7f2663984640 1 --2- 172.21.15.43:0/1111048812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f265c104cd0 0x7f265c1050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.266+0000 7f2663984640 1 -- 172.21.15.43:0/1111048812 >> 172.21.15.43:0/1111048812 conn(0x7f265c0781b0 msgr2=0x7f265c0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.266+0000 7f2663984640 1 -- 172.21.15.43:0/1111048812 shutdown_connections 2024-04-25T10:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.266+0000 7f2663984640 1 -- 172.21.15.43:0/1111048812 wait complete. 2024-04-25T10:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.266+0000 7f2663984640 1 Processor -- start 2024-04-25T10:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.267+0000 7f2663984640 1 -- start start 2024-04-25T10:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.267+0000 7f2663984640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f265c104cd0 0x7f265c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.267+0000 7f2663984640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f265c10b730 0x7f265c19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.267+0000 7f2663984640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f265c10e0e0 0x7f265c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.267+0000 7f2663984640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f265c113d40 con 0x7f265c10b730 2024-04-25T10:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.267+0000 7f2663984640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f265c113bc0 con 0x7f265c10e0e0 2024-04-25T10:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.267+0000 7f2663984640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f265c113ec0 con 0x7f265c104cd0 2024-04-25T10:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.267+0000 7f2660f1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f265c10b730 0x7f265c19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.267+0000 7f2660f1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f265c10b730 0x7f265c19ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50008/0 (socket says 172.21.15.43:50008) 2024-04-25T10:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.267+0000 7f2660f1f640 1 -- 172.21.15.43:0/3737848293 learned_addr learned my addr 172.21.15.43:0/3737848293 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.267+0000 7f2661720640 1 --2- 172.21.15.43:0/3737848293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f265c104cd0 0x7f265c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.267+0000 7f2661f21640 1 --2- 172.21.15.43:0/3737848293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f265c10e0e0 0x7f265c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.267+0000 7f2660f1f640 1 -- 172.21.15.43:0/3737848293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f265c104cd0 msgr2=0x7f265c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.267+0000 7f2660f1f640 1 --2- 172.21.15.43:0/3737848293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f265c104cd0 0x7f265c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.267+0000 7f2660f1f640 1 -- 172.21.15.43:0/3737848293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f265c10e0e0 msgr2=0x7f265c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.267+0000 7f2660f1f640 1 --2- 172.21.15.43:0/3737848293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f265c10e0e0 0x7f265c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.267+0000 7f2660f1f640 1 -- 172.21.15.43:0/3737848293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265c19f560 con 0x7f265c10b730 2024-04-25T10:36:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.268+0000 7f2660f1f640 1 --2- 172.21.15.43:0/3737848293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f265c10b730 0x7f265c19ac60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f265000e9f0 tx=0x7f265000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.268+0000 7f264e7fc640 1 -- 172.21.15.43:0/3737848293 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265000cd80 con 0x7f265c10b730 2024-04-25T10:36:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.268+0000 7f264e7fc640 1 -- 172.21.15.43:0/3737848293 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26500108e0 con 0x7f265c10b730 2024-04-25T10:36:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.268+0000 7f2663984640 1 -- 172.21.15.43:0/3737848293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265c19f850 con 0x7f265c10b730 2024-04-25T10:36:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.268+0000 7f264e7fc640 1 -- 172.21.15.43:0/3737848293 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650018980 con 0x7f265c10b730 2024-04-25T10:36:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.268+0000 7f2663984640 1 -- 172.21.15.43:0/3737848293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265c1a71d0 con 0x7f265c10b730 2024-04-25T10:36:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.271+0000 7f264e7fc640 1 -- 172.21.15.43:0/3737848293 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f2650010430 con 0x7f265c10b730 2024-04-25T10:36:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.271+0000 7f266a83b740 1 Processor -- start 2024-04-25T10:36:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.271+0000 7f266a83b740 1 -- start start 2024-04-25T10:36:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.271+0000 7f266a83b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589b13311b0 0x5589b16d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.271+0000 7f266a83b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13082f0 0x5589b16d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.271+0000 7f266a83b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13104d0 0x5589b16d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.271+0000 7f266a83b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5589b1637460 con 0x5589b13082f0 2024-04-25T10:36:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.271+0000 7f266a83b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5589b15f1590 con 0x5589b13104d0 2024-04-25T10:36:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.271+0000 7f266a83b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5589b1633420 con 0x5589b13311b0 2024-04-25T10:36:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.271+0000 7f2661720640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13082f0 0x5589b16d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.271+0000 7f2661720640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13082f0 0x5589b16d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50010/0 (socket says 172.21.15.43:50010) 2024-04-25T10:36:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.271+0000 7f2661720640 1 -- 172.21.15.43:0/1485689341 learned_addr learned my addr 172.21.15.43:0/1485689341 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.271+0000 7f264e7fc640 1 --2- 172.21.15.43:0/3737848293 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26480afc50 0x7f26480b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.271+0000 7f2660f1f640 1 --2- 172.21.15.43:0/1485689341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13104d0 0x5589b16d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.271+0000 7f2661f21640 1 --2- 172.21.15.43:0/1485689341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589b13311b0 0x5589b16d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.271+0000 7f2661f21640 1 --2- 172.21.15.43:0/3737848293 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26480afc50 0x7f26480b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.272+0000 7f264e7fc640 1 -- 172.21.15.43:0/3737848293 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f26500df240 con 0x7f265c10b730 2024-04-25T10:36:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.272+0000 7f2661720640 1 -- 172.21.15.43:0/1485689341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589b13311b0 msgr2=0x5589b16d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.272+0000 7f2661720640 1 --2- 172.21.15.43:0/1485689341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589b13311b0 0x5589b16d09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.272+0000 7f2661720640 1 -- 172.21.15.43:0/1485689341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13104d0 msgr2=0x5589b16d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.272+0000 7f2661720640 1 --2- 172.21.15.43:0/1485689341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13104d0 0x5589b16d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.272+0000 7f2661720640 1 -- 172.21.15.43:0/1485689341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589b15fe0d0 con 0x5589b13082f0 2024-04-25T10:36:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.272+0000 7f2660f1f640 1 --2- 172.21.15.43:0/1485689341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13104d0 0x5589b16d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.272+0000 7f2661720640 1 --2- 172.21.15.43:0/1485689341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13082f0 0x5589b16d0ee0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f265800bcd0 tx=0x7f26580107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31176a4aac298eb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.272+0000 7f2661f21640 1 --2- 172.21.15.43:0/3737848293 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26480afc50 0x7f26480b2160 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f26540094e0 tx=0x7f26540095c0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.272+0000 7f26477fe640 1 -- 172.21.15.43:0/1485689341 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658005220 con 0x5589b13082f0 2024-04-25T10:36:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.272+0000 7f26477fe640 1 -- 172.21.15.43:0/1485689341 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2658004c50 con 0x5589b13082f0 2024-04-25T10:36:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.272+0000 7f26477fe640 1 -- 172.21.15.43:0/1485689341 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26580116a0 con 0x5589b13082f0 2024-04-25T10:36:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.273+0000 7f266a83b740 1 -- 172.21.15.43:0/1485689341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13082f0 msgr2=0x5589b16d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.273+0000 7f266a83b740 1 --2- 172.21.15.43:0/1485689341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13082f0 0x5589b16d0ee0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f265800bcd0 tx=0x7f26580107a0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.273+0000 7f266a83b740 1 -- 172.21.15.43:0/1485689341 shutdown_connections 2024-04-25T10:36:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.273+0000 7f266a83b740 1 --2- 172.21.15.43:0/1485689341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13104d0 0x5589b16d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.273+0000 7f266a83b740 1 --2- 172.21.15.43:0/1485689341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13082f0 0x5589b16d0ee0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.273+0000 7f266a83b740 1 --2- 172.21.15.43:0/1485689341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589b13311b0 0x5589b16d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.273+0000 7f266a83b740 1 -- 172.21.15.43:0/1485689341 >> 172.21.15.43:0/1485689341 conn(0x5589b1325300 msgr2=0x5589b16cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.273+0000 7f266a83b740 1 -- 172.21.15.43:0/1485689341 shutdown_connections 2024-04-25T10:36:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.273+0000 7f266a83b740 1 -- 172.21.15.43:0/1485689341 wait complete. 2024-04-25T10:36:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.273+0000 7f266a83b740 1 -- start start 2024-04-25T10:36:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.274+0000 7f266a83b740 1 Processor -- start 2024-04-25T10:36:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.274+0000 7f266a83b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13082f0 0x5589b16df410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.274+0000 7f266a83b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589b13104d0 0x5589b16df950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.274+0000 7f266a83b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13311b0 0x5589b16d6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.274+0000 7f266a83b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5589b16e2030 con 0x5589b13311b0 2024-04-25T10:36:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.274+0000 7f266a83b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5589b15beb60 con 0x5589b13082f0 2024-04-25T10:36:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.274+0000 7f266a83b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5589b16e21b0 con 0x5589b13104d0 2024-04-25T10:36:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.274+0000 7f266a83b740 10 client.0 _ll_register_callbacks cb 0x5589b15e32b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.274+0000 7f2661f21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13311b0 0x5589b16d6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.274+0000 7f2661f21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13311b0 0x5589b16d6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50022/0 (socket says 172.21.15.43:50022) 2024-04-25T10:36:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.274+0000 7f2661f21640 1 -- 172.21.15.43:0/4077629783 learned_addr learned my addr 172.21.15.43:0/4077629783 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.274+0000 7f2661720640 1 --2- 172.21.15.43:0/4077629783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13082f0 0x5589b16df410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.274+0000 7f2660f1f640 1 --2- 172.21.15.43:0/4077629783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589b13104d0 0x5589b16df950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.275+0000 7f2661f21640 1 -- 172.21.15.43:0/4077629783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589b13104d0 msgr2=0x5589b16df950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.275+0000 7f2661f21640 1 --2- 172.21.15.43:0/4077629783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589b13104d0 0x5589b16df950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.275+0000 7f2661f21640 1 -- 172.21.15.43:0/4077629783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13082f0 msgr2=0x5589b16df410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.275+0000 7f2661f21640 1 --2- 172.21.15.43:0/4077629783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13082f0 0x5589b16df410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.275+0000 7f2661f21640 1 -- 172.21.15.43:0/4077629783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589b1349210 con 0x5589b13311b0 2024-04-25T10:36:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.275+0000 7f2661f21640 1 --2- 172.21.15.43:0/4077629783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13311b0 0x5589b16d6940 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f265402e890 tx=0x7f265402a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d013ec405a3963 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.275+0000 7f2644ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.275+0000 7f2644ff9640 1 -- 172.21.15.43:0/4077629783 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654050070 con 0x5589b13311b0 2024-04-25T10:36:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.275+0000 7f2644ff9640 1 -- 172.21.15.43:0/4077629783 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f2654038650 con 0x5589b13311b0 2024-04-25T10:36:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.275+0000 7f266a83b740 1 -- 172.21.15.43:0/4077629783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2658004c50 con 0x5589b13311b0 2024-04-25T10:36:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.275+0000 7f2644ff9640 1 -- 172.21.15.43:0/4077629783 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f265405c050 con 0x5589b13311b0 2024-04-25T10:36:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.275+0000 7f266a83b740 10 client.16515 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.275+0000 7f266a83b740 1 -- 172.21.15.43:0/4077629783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5589b1415620 con 0x5589b13311b0 2024-04-25T10:36:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.276+0000 7f2644ff9640 1 -- 172.21.15.43:0/4077629783 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2658004c50 con 0x5589b13311b0 2024-04-25T10:36:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.276+0000 7f2644ff9640 1 -- 172.21.15.43:0/4077629783 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f26540479e0 con 0x5589b13311b0 2024-04-25T10:36:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.276+0000 7f266a83b740 10 client.16515 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.276+0000 7f266a83b740 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.276+0000 7f266a83b740 20 client.16515 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.276+0000 7f266a83b740 1 --2- 172.21.15.43:0/4077629783 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5589b12efd80 0x5589b16f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.276+0000 7f266a83b740 4 client.16515 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:36:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.276+0000 7f266a83b740 1 -- 172.21.15.43:0/4077629783 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589b1645070 con 0x5589b12efd80 2024-04-25T10:36:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.276+0000 7f2661720640 1 --2- 172.21.15.43:0/4077629783 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5589b12efd80 0x5589b16f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.277+0000 7f2644ff9640 10 client.16515 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.277+0000 7f2661720640 1 --2- 172.21.15.43:0/4077629783 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5589b12efd80 0x5589b16f5fb0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d554163731b03ddb server_cookie=be800e39184e59b8 in_seq=0 out_seq=0 2024-04-25T10:36:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.277+0000 7f2644ff9640 1 -- 172.21.15.43:0/4077629783 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5589b1645070 con 0x5589b12efd80 2024-04-25T10:36:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.277+0000 7f2644ff9640 10 client.16515 handle_command_reply: tid=0 2024-04-25T10:36:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.278+0000 7f266a83b740 1 client.16515 shutdown 2024-04-25T10:36:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.278+0000 7f266a83b740 1 -- 172.21.15.43:0/4077629783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13311b0 msgr2=0x5589b16d6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.278+0000 7f266a83b740 1 --2- 172.21.15.43:0/4077629783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13311b0 0x5589b16d6940 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f265402e890 tx=0x7f265402a9e0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.278+0000 7f266a83b740 1 -- 172.21.15.43:0/4077629783 shutdown_connections 2024-04-25T10:36:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.278+0000 7f266a83b740 1 --2- 172.21.15.43:0/4077629783 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5589b12efd80 0x5589b16f5fb0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.278+0000 7f266a83b740 1 --2- 172.21.15.43:0/4077629783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13311b0 0x5589b16d6940 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.278+0000 7f2644ff9640 0 client.16515 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.278+0000 7f266a83b740 1 --2- 172.21.15.43:0/4077629783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589b13104d0 0x5589b16df950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.278+0000 7f266a83b740 1 --2- 172.21.15.43:0/4077629783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13082f0 0x5589b16df410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.278+0000 7f266a83b740 1 -- 172.21.15.43:0/4077629783 >> 172.21.15.43:0/4077629783 conn(0x5589b1325300 msgr2=0x5589b16d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.278+0000 7f266a83b740 1 -- 172.21.15.43:0/4077629783 shutdown_connections 2024-04-25T10:36:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.278+0000 7f266a83b740 1 -- 172.21.15.43:0/4077629783 wait complete. 2024-04-25T10:36:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.278+0000 7f266a83b740 20 client.16515 trim_cache size 0 max 16384 2024-04-25T10:36:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.293+0000 7f266a83b740 1 Processor -- start 2024-04-25T10:36:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.293+0000 7f266a83b740 1 -- start start 2024-04-25T10:36:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.293+0000 7f266a83b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13082f0 0x5589b16df960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.293+0000 7f266a83b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13104d0 0x5589b16d2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.294+0000 7f266a83b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589b13311b0 0x5589b16cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.294+0000 7f266a83b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1b0010 con 0x5589b13104d0 2024-04-25T10:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.294+0000 7f266a83b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1ad8e0 con 0x5589b13082f0 2024-04-25T10:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.294+0000 7f266a83b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1ab730 con 0x5589b13311b0 2024-04-25T10:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.294+0000 7f2660f1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13104d0 0x5589b16d2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.294+0000 7f2660f1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13104d0 0x5589b16d2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50028/0 (socket says 172.21.15.43:50028) 2024-04-25T10:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.294+0000 7f2660f1f640 1 -- 172.21.15.43:0/1522485281 learned_addr learned my addr 172.21.15.43:0/1522485281 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.294+0000 7f2661720640 1 --2- 172.21.15.43:0/1522485281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13082f0 0x5589b16df960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.294+0000 7f2660f1f640 1 -- 172.21.15.43:0/1522485281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589b13311b0 msgr2=0x5589b16cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.294+0000 7f2660f1f640 1 --2- 172.21.15.43:0/1522485281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589b13311b0 0x5589b16cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.294+0000 7f2660f1f640 1 -- 172.21.15.43:0/1522485281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13082f0 msgr2=0x5589b16df960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.294+0000 7f2660f1f640 1 --2- 172.21.15.43:0/1522485281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13082f0 0x5589b16df960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.294+0000 7f2660f1f640 1 -- 172.21.15.43:0/1522485281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589b16de710 con 0x5589b13104d0 2024-04-25T10:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.294+0000 7f2660f1f640 1 --2- 172.21.15.43:0/1522485281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13104d0 0x5589b16d2e10 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f26500dcfc0 tx=0x7f2650000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67647ca64f07a73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.294+0000 7f2646ffd640 1 -- 172.21.15.43:0/1522485281 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26500a6fd0 con 0x5589b13104d0 2024-04-25T10:36:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.294+0000 7f2646ffd640 1 -- 172.21.15.43:0/1522485281 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26500df040 con 0x5589b13104d0 2024-04-25T10:36:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.294+0000 7f2646ffd640 1 -- 172.21.15.43:0/1522485281 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26500a6720 con 0x5589b13104d0 2024-04-25T10:36:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.295+0000 7f266a83b740 1 -- 172.21.15.43:0/1522485281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13104d0 msgr2=0x5589b16d2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.295+0000 7f266a83b740 1 --2- 172.21.15.43:0/1522485281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13104d0 0x5589b16d2e10 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f26500dcfc0 tx=0x7f2650000bc0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.295+0000 7f266a83b740 1 -- 172.21.15.43:0/1522485281 shutdown_connections 2024-04-25T10:36:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.295+0000 7f266a83b740 1 --2- 172.21.15.43:0/1522485281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589b13311b0 0x5589b16cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.295+0000 7f266a83b740 1 --2- 172.21.15.43:0/1522485281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13104d0 0x5589b16d2e10 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.295+0000 7f266a83b740 1 --2- 172.21.15.43:0/1522485281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13082f0 0x5589b16df960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.295+0000 7f266a83b740 1 -- 172.21.15.43:0/1522485281 >> 172.21.15.43:0/1522485281 conn(0x5589b1325300 msgr2=0x5589b16ccc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.295+0000 7f266a83b740 1 -- 172.21.15.43:0/1522485281 shutdown_connections 2024-04-25T10:36:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.295+0000 7f266a83b740 1 -- 172.21.15.43:0/1522485281 wait complete. 2024-04-25T10:36:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.295+0000 7f266a83b740 1 -- start start 2024-04-25T10:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.296+0000 7f266a83b740 1 Processor -- start 2024-04-25T10:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.296+0000 7f266a83b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589b13082f0 0x5589b16cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.296+0000 7f266a83b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13104d0 0x5589b16cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.296+0000 7f266a83b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13311b0 0x5589b175c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.296+0000 7f266a83b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1abda0 con 0x5589b13104d0 2024-04-25T10:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.296+0000 7f266a83b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1ad210 con 0x5589b13311b0 2024-04-25T10:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.296+0000 7f266a83b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1ac470 con 0x5589b13082f0 2024-04-25T10:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.296+0000 7f266a83b740 10 client.0 _ll_register_callbacks cb 0x7f265402e750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.296+0000 7f2660f1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13104d0 0x5589b16cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.296+0000 7f2660f1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13104d0 0x5589b16cc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50040/0 (socket says 172.21.15.43:50040) 2024-04-25T10:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.296+0000 7f2660f1f640 1 -- 172.21.15.43:0/200844907 learned_addr learned my addr 172.21.15.43:0/200844907 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.296+0000 7f2661f21640 1 --2- 172.21.15.43:0/200844907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13311b0 0x5589b175c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.296+0000 7f2661720640 1 --2- 172.21.15.43:0/200844907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589b13082f0 0x5589b16cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.297+0000 7f2660f1f640 1 -- 172.21.15.43:0/200844907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589b13082f0 msgr2=0x5589b16cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.297+0000 7f2660f1f640 1 --2- 172.21.15.43:0/200844907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589b13082f0 0x5589b16cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.297+0000 7f2660f1f640 1 -- 172.21.15.43:0/200844907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13311b0 msgr2=0x5589b175c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.297+0000 7f2660f1f640 1 --2- 172.21.15.43:0/200844907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13311b0 0x5589b175c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.297+0000 7f2660f1f640 1 -- 172.21.15.43:0/200844907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589b16cccb0 con 0x5589b13104d0 2024-04-25T10:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.297+0000 7f2661f21640 1 --2- 172.21.15.43:0/200844907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13311b0 0x5589b175c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.297+0000 7f2660f1f640 1 --2- 172.21.15.43:0/200844907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13104d0 0x5589b16cc5f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f26500a34f0 tx=0x7f26500a6fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec71db7c9c62d99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.297+0000 7f2644ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.297+0000 7f2644ff9640 1 -- 172.21.15.43:0/200844907 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26500c6060 con 0x5589b13104d0 2024-04-25T10:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.297+0000 7f2644ff9640 1 -- 172.21.15.43:0/200844907 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f26500a6920 con 0x5589b13104d0 2024-04-25T10:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.297+0000 7f266a83b740 1 -- 172.21.15.43:0/200844907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26500df040 con 0x5589b13104d0 2024-04-25T10:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.297+0000 7f2644ff9640 1 -- 172.21.15.43:0/200844907 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f26500c1070 con 0x5589b13104d0 2024-04-25T10:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.297+0000 7f266a83b740 10 client.16521 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.297+0000 7f266a83b740 1 -- 172.21.15.43:0/200844907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5589b175d8d0 con 0x5589b13104d0 2024-04-25T10:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.297+0000 7f2644ff9640 1 -- 172.21.15.43:0/200844907 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26500df040 con 0x5589b13104d0 2024-04-25T10:36:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.298+0000 7f2644ff9640 1 -- 172.21.15.43:0/200844907 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f26500a2e30 con 0x5589b13104d0 2024-04-25T10:36:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.298+0000 7f266a83b740 10 client.16521 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.298+0000 7f266a83b740 10 client.16521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.298+0000 7f266a83b740 20 client.16521 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.298+0000 7f266a83b740 1 --2- 172.21.15.43:0/200844907 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5589b12c51c0 0x5589b175ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.298+0000 7f266a83b740 4 client.16521 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:36:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.298+0000 7f266a83b740 1 -- 172.21.15.43:0/200844907 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589b163eee0 con 0x5589b12c51c0 2024-04-25T10:36:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.298+0000 7f2661720640 1 --2- 172.21.15.43:0/200844907 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5589b12c51c0 0x5589b175ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.298+0000 7f2644ff9640 10 client.16521 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.298+0000 7f2661720640 1 --2- 172.21.15.43:0/200844907 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5589b12c51c0 0x5589b175ff50 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6a3f92f93ec0e5a server_cookie=d2054b1d6afb36f2 in_seq=0 out_seq=0 2024-04-25T10:36:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.315+0000 7f2644ff9640 1 -- 172.21.15.43:0/200844907 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5589b163eee0 con 0x5589b12c51c0 2024-04-25T10:36:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.315+0000 7f2644ff9640 10 client.16521 handle_command_reply: tid=0 2024-04-25T10:36:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.315+0000 7f266a83b740 1 client.16521 shutdown 2024-04-25T10:36:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.315+0000 7f266a83b740 1 -- 172.21.15.43:0/200844907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13104d0 msgr2=0x5589b16cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.315+0000 7f266a83b740 1 --2- 172.21.15.43:0/200844907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13104d0 0x5589b16cc5f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f26500a34f0 tx=0x7f26500a6fd0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.315+0000 7f266a83b740 1 -- 172.21.15.43:0/200844907 shutdown_connections 2024-04-25T10:36:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.315+0000 7f266a83b740 1 --2- 172.21.15.43:0/200844907 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5589b12c51c0 0x5589b175ff50 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.315+0000 7f266a83b740 1 --2- 172.21.15.43:0/200844907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589b13311b0 0x5589b175c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.315+0000 7f266a83b740 1 --2- 172.21.15.43:0/200844907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589b13104d0 0x5589b16cc5f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.315+0000 7f266a83b740 1 --2- 172.21.15.43:0/200844907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589b13082f0 0x5589b16cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.315+0000 7f2644ff9640 0 client.16521 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.315+0000 7f266a83b740 1 -- 172.21.15.43:0/200844907 >> 172.21.15.43:0/200844907 conn(0x5589b1325300 msgr2=0x5589b16e0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.316+0000 7f266a83b740 1 -- 172.21.15.43:0/200844907 shutdown_connections 2024-04-25T10:36:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.316+0000 7f266a83b740 1 -- 172.21.15.43:0/200844907 wait complete. 2024-04-25T10:36:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.316+0000 7f266a83b740 20 client.16521 trim_cache size 0 max 16384 2024-04-25T10:36:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:36:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:36:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:36:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:36:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.316+0000 7f2663984640 1 -- 172.21.15.43:0/3737848293 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26480afc50 msgr2=0x7f26480b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.316+0000 7f2663984640 1 --2- 172.21.15.43:0/3737848293 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26480afc50 0x7f26480b2160 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f26540094e0 tx=0x7f26540095c0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.316+0000 7f2663984640 1 -- 172.21.15.43:0/3737848293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f265c10b730 msgr2=0x7f265c19ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.316+0000 7f2663984640 1 --2- 172.21.15.43:0/3737848293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f265c10b730 0x7f265c19ac60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f265000e9f0 tx=0x7f265000eeb0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.316+0000 7f2663984640 1 -- 172.21.15.43:0/3737848293 shutdown_connections 2024-04-25T10:36:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.316+0000 7f2663984640 1 --2- 172.21.15.43:0/3737848293 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26480afc50 0x7f26480b2160 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.316+0000 7f2663984640 1 --2- 172.21.15.43:0/3737848293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f265c10e0e0 0x7f265c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.316+0000 7f2663984640 1 --2- 172.21.15.43:0/3737848293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f265c10b730 0x7f265c19ac60 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.316+0000 7f2663984640 1 --2- 172.21.15.43:0/3737848293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f265c104cd0 0x7f265c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.316+0000 7f2663984640 1 -- 172.21.15.43:0/3737848293 >> 172.21.15.43:0/3737848293 conn(0x7f265c0781b0 msgr2=0x7f265c101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.316+0000 7f2663984640 1 -- 172.21.15.43:0/3737848293 shutdown_connections 2024-04-25T10:36:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:14.316+0000 7f2663984640 1 -- 172.21.15.43:0/3737848293 wait complete. 2024-04-25T10:36:16.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:15 smithi164 ceph-mon[32713]: pgmap v115: 97 pgs: 97 active+clean; 4.1 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 63 KiB/s wr, 22 op/s 2024-04-25T10:36:16.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:15 smithi077 ceph-mon[32772]: pgmap v115: 97 pgs: 97 active+clean; 4.1 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 63 KiB/s wr, 22 op/s 2024-04-25T10:36:16.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:15 smithi043 ceph-mon[29478]: pgmap v115: 97 pgs: 97 active+clean; 4.1 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 63 KiB/s wr, 22 op/s 2024-04-25T10:36:16.775 INFO:tasks.workunit.client.0.smithi043.stdout:64/600: open fd = 3 2024-04-25T10:36:16.776 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.776 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.776 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.776 INFO:tasks.workunit.client.0.smithi043.stdout:65/600: open fd = 3 2024-04-25T10:36:16.776 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.776 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.776 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.776 INFO:tasks.workunit.client.0.smithi043.stdout:66/600: open fd = 3 2024-04-25T10:36:16.776 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.776 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.776 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.776 INFO:tasks.workunit.client.0.smithi043.stdout:67/600: open fd = 3 2024-04-25T10:36:16.776 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.776 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.777 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.777 INFO:tasks.workunit.client.0.smithi043.stdout:68/600: open fd = 3 2024-04-25T10:36:16.777 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.777 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.777 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.777 INFO:tasks.workunit.client.0.smithi043.stdout:69/600: open fd = 3 2024-04-25T10:36:16.777 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.777 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.777 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.777 INFO:tasks.workunit.client.0.smithi043.stdout:70/600: open fd = 3 2024-04-25T10:36:16.777 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.777 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.777 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.777 INFO:tasks.workunit.client.0.smithi043.stdout:71/600: open fd = 3 2024-04-25T10:36:16.777 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.778 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.778 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.778 INFO:tasks.workunit.client.0.smithi043.stdout:72/600: open fd = 3 2024-04-25T10:36:16.778 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.778 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.778 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.778 INFO:tasks.workunit.client.0.smithi043.stdout:73/600: open fd = 3 2024-04-25T10:36:16.778 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.778 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.778 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.778 INFO:tasks.workunit.client.0.smithi043.stdout:74/600: open fd = 3 2024-04-25T10:36:16.778 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.778 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.778 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.779 INFO:tasks.workunit.client.0.smithi043.stdout:75/600: open fd = 3 2024-04-25T10:36:16.779 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.779 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.779 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.779 INFO:tasks.workunit.client.0.smithi043.stdout:76/600: open fd = 3 2024-04-25T10:36:16.779 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.779 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.779 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.779 INFO:tasks.workunit.client.0.smithi043.stdout:77/600: open fd = 3 2024-04-25T10:36:16.779 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.779 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.779 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.779 INFO:tasks.workunit.client.0.smithi043.stdout:78/600: open fd = 3 2024-04-25T10:36:16.779 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.779 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.779 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.780 INFO:tasks.workunit.client.0.smithi043.stdout:79/600: open fd = 3 2024-04-25T10:36:16.780 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.780 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.780 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.780 INFO:tasks.workunit.client.0.smithi043.stdout:80/600: open fd = 3 2024-04-25T10:36:16.780 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.780 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.780 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.780 INFO:tasks.workunit.client.0.smithi043.stdout:81/600: open fd = 3 2024-04-25T10:36:16.780 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.780 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.780 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.780 INFO:tasks.workunit.client.0.smithi043.stdout:82/600: open fd = 3 2024-04-25T10:36:16.780 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.780 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.781 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.781 INFO:tasks.workunit.client.0.smithi043.stdout:83/600: open fd = 3 2024-04-25T10:36:16.781 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.781 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.781 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.781 INFO:tasks.workunit.client.0.smithi043.stdout:84/600: open fd = 3 2024-04-25T10:36:16.781 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.781 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.781 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.781 INFO:tasks.workunit.client.0.smithi043.stdout:85/600: open fd = 3 2024-04-25T10:36:16.781 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.781 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.781 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.781 INFO:tasks.workunit.client.0.smithi043.stdout:86/600: open fd = 3 2024-04-25T10:36:16.781 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.782 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.782 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.782 INFO:tasks.workunit.client.0.smithi043.stdout:87/600: open fd = 3 2024-04-25T10:36:16.782 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.782 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.782 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.782 INFO:tasks.workunit.client.0.smithi043.stdout:88/600: open fd = 3 2024-04-25T10:36:16.782 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.782 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.782 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.782 INFO:tasks.workunit.client.0.smithi043.stdout:89/600: open fd = 3 2024-04-25T10:36:16.782 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.782 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.782 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.782 INFO:tasks.workunit.client.0.smithi043.stdout:90/600: open fd = 3 2024-04-25T10:36:16.783 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.783 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.783 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.783 INFO:tasks.workunit.client.0.smithi043.stdout:91/600: open fd = 3 2024-04-25T10:36:16.783 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.783 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.783 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.783 INFO:tasks.workunit.client.0.smithi043.stdout:92/600: open fd = 3 2024-04-25T10:36:16.783 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.783 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.783 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.783 INFO:tasks.workunit.client.0.smithi043.stdout:93/600: open fd = 3 2024-04-25T10:36:16.783 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.783 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.783 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.784 INFO:tasks.workunit.client.0.smithi043.stdout:94/600: open fd = 3 2024-04-25T10:36:16.784 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.784 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.784 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.784 INFO:tasks.workunit.client.0.smithi043.stdout:95/600: open fd = 3 2024-04-25T10:36:16.784 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.784 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.784 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.784 INFO:tasks.workunit.client.0.smithi043.stdout:96/600: open fd = 3 2024-04-25T10:36:16.784 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.784 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.784 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.784 INFO:tasks.workunit.client.0.smithi043.stdout:97/600: open fd = 3 2024-04-25T10:36:16.784 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.784 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.785 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.785 INFO:tasks.workunit.client.0.smithi043.stdout:98/600: open fd = 3 2024-04-25T10:36:16.785 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.785 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.785 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.785 INFO:tasks.workunit.client.0.smithi043.stdout:99/600: open fd = 3 2024-04-25T10:36:16.785 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.785 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.785 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.785 INFO:tasks.workunit.client.0.smithi043.stdout:100/600: open fd = 3 2024-04-25T10:36:16.785 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.785 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.785 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.785 INFO:tasks.workunit.client.0.smithi043.stdout:101/600: open fd = 3 2024-04-25T10:36:16.785 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.786 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.786 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.786 INFO:tasks.workunit.client.0.smithi043.stdout:102/600: open fd = 3 2024-04-25T10:36:16.786 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.786 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.786 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.786 INFO:tasks.workunit.client.0.smithi043.stdout:103/600: open fd = 3 2024-04-25T10:36:16.786 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.786 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.786 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.786 INFO:tasks.workunit.client.0.smithi043.stdout:104/600: open fd = 3 2024-04-25T10:36:16.786 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.786 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.786 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.787 INFO:tasks.workunit.client.0.smithi043.stdout:105/600: open fd = 3 2024-04-25T10:36:16.787 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.787 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.787 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.787 INFO:tasks.workunit.client.0.smithi043.stdout:106/600: open fd = 3 2024-04-25T10:36:16.787 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.787 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.787 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.787 INFO:tasks.workunit.client.0.smithi043.stdout:107/600: open fd = 3 2024-04-25T10:36:16.787 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.787 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.787 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.787 INFO:tasks.workunit.client.0.smithi043.stdout:108/600: open fd = 3 2024-04-25T10:36:16.787 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.787 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.787 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.788 INFO:tasks.workunit.client.0.smithi043.stdout:109/600: open fd = 3 2024-04-25T10:36:16.788 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.788 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.788 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.788 INFO:tasks.workunit.client.0.smithi043.stdout:110/600: open fd = 3 2024-04-25T10:36:16.788 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.788 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.788 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.788 INFO:tasks.workunit.client.0.smithi043.stdout:111/600: open fd = 3 2024-04-25T10:36:16.788 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.788 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.788 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.788 INFO:tasks.workunit.client.0.smithi043.stdout:112/600: open fd = 3 2024-04-25T10:36:16.788 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.788 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.789 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.789 INFO:tasks.workunit.client.0.smithi043.stdout:113/600: open fd = 3 2024-04-25T10:36:16.789 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.789 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.789 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.789 INFO:tasks.workunit.client.0.smithi043.stdout:114/600: open fd = 3 2024-04-25T10:36:16.789 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.789 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.789 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.789 INFO:tasks.workunit.client.0.smithi043.stdout:115/600: open fd = 3 2024-04-25T10:36:16.789 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.789 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.789 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.789 INFO:tasks.workunit.client.0.smithi043.stdout:116/600: open fd = 3 2024-04-25T10:36:16.790 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.790 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.790 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.790 INFO:tasks.workunit.client.0.smithi043.stdout:117/600: open fd = 3 2024-04-25T10:36:16.790 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.790 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.790 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.790 INFO:tasks.workunit.client.0.smithi043.stdout:118/600: open fd = 3 2024-04-25T10:36:16.790 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.790 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.790 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.790 INFO:tasks.workunit.client.0.smithi043.stdout:119/600: open fd = 3 2024-04-25T10:36:16.790 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.790 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.790 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.791 INFO:tasks.workunit.client.0.smithi043.stdout:120/600: open fd = 3 2024-04-25T10:36:16.791 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.791 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.791 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.791 INFO:tasks.workunit.client.0.smithi043.stdout:121/600: open fd = 3 2024-04-25T10:36:16.791 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.791 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.791 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.791 INFO:tasks.workunit.client.0.smithi043.stdout:122/600: open fd = 3 2024-04-25T10:36:16.791 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.791 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.791 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.791 INFO:tasks.workunit.client.0.smithi043.stdout:123/600: open fd = 3 2024-04-25T10:36:16.791 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.791 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.792 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.792 INFO:tasks.workunit.client.0.smithi043.stdout:124/600: open fd = 3 2024-04-25T10:36:16.792 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.792 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.792 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.792 INFO:tasks.workunit.client.0.smithi043.stdout:125/600: open fd = 3 2024-04-25T10:36:16.792 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.792 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:16.792 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:36:16.792 INFO:tasks.workunit.client.0.smithi043.stdout:126/600: open fd = 3 2024-04-25T10:36:16.792 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:36:18.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:17 smithi077 ceph-mon[32772]: pgmap v116: 97 pgs: 97 active+clean; 4.1 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-04-25T10:36:18.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:17 smithi164 ceph-mon[32713]: pgmap v116: 97 pgs: 97 active+clean; 4.1 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-04-25T10:36:18.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:17 smithi043 ceph-mon[29478]: pgmap v116: 97 pgs: 97 active+clean; 4.1 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-04-25T10:36:20.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:19 smithi077 ceph-mon[32772]: pgmap v117: 97 pgs: 97 active+clean; 4.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-25T10:36:20.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:19 smithi164 ceph-mon[32713]: pgmap v117: 97 pgs: 97 active+clean; 4.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-25T10:36:20.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:19 smithi043 ceph-mon[29478]: pgmap v117: 97 pgs: 97 active+clean; 4.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-25T10:36:22.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:21 smithi077 ceph-mon[32772]: pgmap v118: 97 pgs: 97 active+clean; 4.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-25T10:36:22.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:21 smithi164 ceph-mon[32713]: pgmap v118: 97 pgs: 97 active+clean; 4.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-25T10:36:22.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:21 smithi043 ceph-mon[29478]: pgmap v118: 97 pgs: 97 active+clean; 4.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-25T10:36:24.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:23 smithi077 ceph-mon[32772]: pgmap v119: 97 pgs: 97 active+clean; 4.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-25T10:36:24.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:23 smithi164 ceph-mon[32713]: pgmap v119: 97 pgs: 97 active+clean; 4.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-25T10:36:24.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:23 smithi043 ceph-mon[29478]: pgmap v119: 97 pgs: 97 active+clean; 4.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-25T10:36:24.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:36:24.664 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:36:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.791+0000 7f1c91851640 1 -- 172.21.15.43:0/2283160903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8c1042b0 msgr2=0x7f1c8c110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.791+0000 7f1c91851640 1 --2- 172.21.15.43:0/2283160903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8c1042b0 0x7f1c8c110630 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009ca0 tx=0x7f1c8402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.791+0000 7f1c91851640 1 -- 172.21.15.43:0/2283160903 shutdown_connections 2024-04-25T10:36:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.791+0000 7f1c91851640 1 --2- 172.21.15.43:0/2283160903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c8c109fb0 0x7f1c8c103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.791+0000 7f1c91851640 1 --2- 172.21.15.43:0/2283160903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8c1042b0 0x7f1c8c110630 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.791+0000 7f1c91851640 1 --2- 172.21.15.43:0/2283160903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c8c109b10 0x7f1c8c103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.791+0000 7f1c91851640 1 -- 172.21.15.43:0/2283160903 >> 172.21.15.43:0/2283160903 conn(0x7f1c8c0ff360 msgr2=0x7f1c8c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.792+0000 7f1c91851640 1 -- 172.21.15.43:0/2283160903 shutdown_connections 2024-04-25T10:36:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.792+0000 7f1c91851640 1 -- 172.21.15.43:0/2283160903 wait complete. 2024-04-25T10:36:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.792+0000 7f1c91851640 1 Processor -- start 2024-04-25T10:36:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.792+0000 7f1c91851640 1 -- start start 2024-04-25T10:36:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.793+0000 7f1c91851640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c8c1042b0 0x7f1c8c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.793+0000 7f1c91851640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8c109b10 0x7f1c8c19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.793+0000 7f1c91851640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c8c109fb0 0x7f1c8c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.793+0000 7f1c91851640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c113d20 con 0x7f1c8c109b10 2024-04-25T10:36:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.793+0000 7f1c91851640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c113ba0 con 0x7f1c8c1042b0 2024-04-25T10:36:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.793+0000 7f1c91851640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c113ea0 con 0x7f1c8c109fb0 2024-04-25T10:36:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.793+0000 7f1c83fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8c109b10 0x7f1c8c19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.793+0000 7f1c83fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8c109b10 0x7f1c8c19ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38594/0 (socket says 172.21.15.43:38594) 2024-04-25T10:36:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.793+0000 7f1c83fff640 1 -- 172.21.15.43:0/1140552808 learned_addr learned my addr 172.21.15.43:0/1140552808 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.793+0000 7f1c8affd640 1 --2- 172.21.15.43:0/1140552808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c8c1042b0 0x7f1c8c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.793+0000 7f1c8b7fe640 1 --2- 172.21.15.43:0/1140552808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c8c109fb0 0x7f1c8c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.793+0000 7f1c83fff640 1 -- 172.21.15.43:0/1140552808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c8c109fb0 msgr2=0x7f1c8c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.793+0000 7f1c83fff640 1 --2- 172.21.15.43:0/1140552808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c8c109fb0 0x7f1c8c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.793+0000 7f1c83fff640 1 -- 172.21.15.43:0/1140552808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c8c1042b0 msgr2=0x7f1c8c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:24.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.793+0000 7f1c83fff640 1 --2- 172.21.15.43:0/1140552808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c8c1042b0 0x7f1c8c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.793+0000 7f1c83fff640 1 -- 172.21.15.43:0/1140552808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8c19f560 con 0x7f1c8c109b10 2024-04-25T10:36:24.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.793+0000 7f1c83fff640 1 --2- 172.21.15.43:0/1140552808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8c109b10 0x7f1c8c19ac60 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1c74005fd0 tx=0x7f1c7400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:24.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.793+0000 7f1c9084f640 1 -- 172.21.15.43:0/1140552808 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74017070 con 0x7f1c8c109b10 2024-04-25T10:36:24.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.794+0000 7f1c91851640 1 -- 172.21.15.43:0/1140552808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c8c19f850 con 0x7f1c8c109b10 2024-04-25T10:36:24.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.794+0000 7f1c9084f640 1 -- 172.21.15.43:0/1140552808 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c74009c30 con 0x7f1c8c109b10 2024-04-25T10:36:24.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.794+0000 7f1c9084f640 1 -- 172.21.15.43:0/1140552808 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74003d10 con 0x7f1c8c109b10 2024-04-25T10:36:24.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.794+0000 7f1c91851640 1 -- 172.21.15.43:0/1140552808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c8c1a71d0 con 0x7f1c8c109b10 2024-04-25T10:36:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.796+0000 7f1c9084f640 1 -- 172.21.15.43:0/1140552808 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f1c74003eb0 con 0x7f1c8c109b10 2024-04-25T10:36:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.796+0000 7f1c987d6740 1 Processor -- start 2024-04-25T10:36:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.796+0000 7f1c987d6740 1 -- start start 2024-04-25T10:36:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.796+0000 7f1c987d6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858db390 0x563685c7ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.796+0000 7f1c987d6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 0x563685c7bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.796+0000 7f1c987d6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858ba6b0 0x563685c888f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.796+0000 7f1c987d6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563685be1d80 con 0x5636858b24d0 2024-04-25T10:36:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.796+0000 7f1c987d6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563685b9b8b0 con 0x5636858db390 2024-04-25T10:36:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.796+0000 7f1c987d6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563685bddd40 con 0x5636858ba6b0 2024-04-25T10:36:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.797+0000 7f1c8affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 0x563685c7bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.797+0000 7f1c8affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 0x563685c7bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38598/0 (socket says 172.21.15.43:38598) 2024-04-25T10:36:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.797+0000 7f1c8affd640 1 -- 172.21.15.43:0/586391703 learned_addr learned my addr 172.21.15.43:0/586391703 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.797+0000 7f1c8b7fe640 1 --2- 172.21.15.43:0/586391703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858db390 0x563685c7ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.797+0000 7f1c83fff640 1 --2- 172.21.15.43:0/586391703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858ba6b0 0x563685c888f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.797+0000 7f1c9084f640 1 --2- 172.21.15.43:0/1140552808 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1c780afc60 0x7f1c780b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.797+0000 7f1c8affd640 1 -- 172.21.15.43:0/586391703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858ba6b0 msgr2=0x563685c888f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.797+0000 7f1c8affd640 1 --2- 172.21.15.43:0/586391703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858ba6b0 0x563685c888f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.797+0000 7f1c8affd640 1 -- 172.21.15.43:0/586391703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858db390 msgr2=0x563685c7ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.797+0000 7f1c8affd640 1 --2- 172.21.15.43:0/586391703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858db390 0x563685c7ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.797+0000 7f1c8affd640 1 -- 172.21.15.43:0/586391703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563685b6ca40 con 0x5636858b24d0 2024-04-25T10:36:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.797+0000 7f1c8b7fe640 1 --2- 172.21.15.43:0/586391703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858db390 0x563685c7ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:36:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.797+0000 7f1c8b7fe640 1 --2- 172.21.15.43:0/1140552808 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1c780afc60 0x7f1c780b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.797+0000 7f1c9084f640 1 -- 172.21.15.43:0/1140552808 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f1c740dec50 con 0x7f1c8c109b10 2024-04-25T10:36:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.797+0000 7f1c8affd640 1 --2- 172.21.15.43:0/586391703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 0x563685c7bfa0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009ca0 tx=0x7f1c7c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd813452670b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.797+0000 7f1c81ffb640 1 -- 172.21.15.43:0/586391703 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c01e070 con 0x5636858b24d0 2024-04-25T10:36:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.797+0000 7f1c81ffb640 1 -- 172.21.15.43:0/586391703 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c7c019ae0 con 0x5636858b24d0 2024-04-25T10:36:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.798+0000 7f1c81ffb640 1 -- 172.21.15.43:0/586391703 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c00e950 con 0x5636858b24d0 2024-04-25T10:36:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.798+0000 7f1c987d6740 1 -- 172.21.15.43:0/586391703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 msgr2=0x563685c7bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.798+0000 7f1c987d6740 1 --2- 172.21.15.43:0/586391703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 0x563685c7bfa0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009ca0 tx=0x7f1c7c011810 comp rx=0 tx=0).stop 2024-04-25T10:36:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.798+0000 7f1c8b7fe640 1 --2- 172.21.15.43:0/1140552808 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1c780afc60 0x7f1c780b2170 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x563685c7cb60 tx=0x7f1c840095c0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.798+0000 7f1c987d6740 1 -- 172.21.15.43:0/586391703 shutdown_connections 2024-04-25T10:36:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.798+0000 7f1c987d6740 1 --2- 172.21.15.43:0/586391703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858ba6b0 0x563685c888f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.798+0000 7f1c987d6740 1 --2- 172.21.15.43:0/586391703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 0x563685c7bfa0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.798+0000 7f1c987d6740 1 --2- 172.21.15.43:0/586391703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858db390 0x563685c7ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.798+0000 7f1c987d6740 1 -- 172.21.15.43:0/586391703 >> 172.21.15.43:0/586391703 conn(0x5636858cf4e0 msgr2=0x563685c78210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.798+0000 7f1c987d6740 1 -- 172.21.15.43:0/586391703 shutdown_connections 2024-04-25T10:36:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.798+0000 7f1c987d6740 1 -- 172.21.15.43:0/586391703 wait complete. 2024-04-25T10:36:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.799+0000 7f1c987d6740 1 -- start start 2024-04-25T10:36:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.799+0000 7f1c987d6740 1 Processor -- start 2024-04-25T10:36:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.799+0000 7f1c987d6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 0x563685c865e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.799+0000 7f1c987d6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858ba6b0 0x563685c86b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.799+0000 7f1c987d6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858db390 0x563685c8c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.799+0000 7f1c987d6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563685c88fe0 con 0x5636858b24d0 2024-04-25T10:36:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.799+0000 7f1c987d6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563685b68d10 con 0x5636858db390 2024-04-25T10:36:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.799+0000 7f1c987d6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563685c89160 con 0x5636858ba6b0 2024-04-25T10:36:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.799+0000 7f1c987d6740 10 client.0 _ll_register_callbacks cb 0x563685b95210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.799+0000 7f1c8affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 0x563685c865e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.800+0000 7f1c8affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 0x563685c865e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38606/0 (socket says 172.21.15.43:38606) 2024-04-25T10:36:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.800+0000 7f1c8affd640 1 -- 172.21.15.43:0/1835969766 learned_addr learned my addr 172.21.15.43:0/1835969766 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.800+0000 7f1c83fff640 1 --2- 172.21.15.43:0/1835969766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858ba6b0 0x563685c86b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.800+0000 7f1c8b7fe640 1 --2- 172.21.15.43:0/1835969766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858db390 0x563685c8c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.800+0000 7f1c8affd640 1 -- 172.21.15.43:0/1835969766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858ba6b0 msgr2=0x563685c86b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.800+0000 7f1c8affd640 1 --2- 172.21.15.43:0/1835969766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858ba6b0 0x563685c86b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.800+0000 7f1c8affd640 1 -- 172.21.15.43:0/1835969766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858db390 msgr2=0x563685c8c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.800+0000 7f1c8affd640 1 --2- 172.21.15.43:0/1835969766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858db390 0x563685c8c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.800+0000 7f1c8affd640 1 -- 172.21.15.43:0/1835969766 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636859bd7d0 con 0x5636858b24d0 2024-04-25T10:36:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.800+0000 7f1c83fff640 1 --2- 172.21.15.43:0/1835969766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858ba6b0 0x563685c86b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:36:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.800+0000 7f1c8b7fe640 1 --2- 172.21.15.43:0/1835969766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858db390 0x563685c8c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:36:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.800+0000 7f1c8affd640 1 --2- 172.21.15.43:0/1835969766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 0x563685c865e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009f50 tx=0x7f1c7c00e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42439629bd9ca89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.800+0000 7f1c6affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.800+0000 7f1c6affd640 1 -- 172.21.15.43:0/1835969766 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c027020 con 0x5636858b24d0 2024-04-25T10:36:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.800+0000 7f1c6affd640 1 -- 172.21.15.43:0/1835969766 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f1c7c005460 con 0x5636858b24d0 2024-04-25T10:36:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.800+0000 7f1c987d6740 1 -- 172.21.15.43:0/1835969766 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c7c019ae0 con 0x5636858b24d0 2024-04-25T10:36:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.801+0000 7f1c6affd640 1 -- 172.21.15.43:0/1835969766 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1c7c032050 con 0x5636858b24d0 2024-04-25T10:36:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.801+0000 7f1c987d6740 10 client.16533 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.801+0000 7f1c987d6740 1 -- 172.21.15.43:0/1835969766 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5636858f33f0 con 0x5636858b24d0 2024-04-25T10:36:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.801+0000 7f1c6affd640 1 -- 172.21.15.43:0/1835969766 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c7c019ae0 con 0x5636858b24d0 2024-04-25T10:36:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.801+0000 7f1c6affd640 1 -- 172.21.15.43:0/1835969766 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1c7c00e990 con 0x5636858b24d0 2024-04-25T10:36:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.801+0000 7f1c987d6740 10 client.16533 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.801+0000 7f1c987d6740 10 client.16533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.801+0000 7f1c987d6740 20 client.16533 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.801+0000 7f1c987d6740 1 --2- 172.21.15.43:0/1835969766 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563685899f60 0x563685c985a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.801+0000 7f1c987d6740 4 client.16533 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:36:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.801+0000 7f1c987d6740 1 -- 172.21.15.43:0/1835969766 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563685bef990 con 0x563685899f60 2024-04-25T10:36:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.801+0000 7f1c83fff640 1 --2- 172.21.15.43:0/1835969766 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563685899f60 0x563685c985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.802+0000 7f1c6affd640 10 client.16533 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.802+0000 7f1c83fff640 1 --2- 172.21.15.43:0/1835969766 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563685899f60 0x563685c985a0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=598fa60fe5819317 server_cookie=2ae4b40e6e4026fd in_seq=0 out_seq=0 2024-04-25T10:36:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.803+0000 7f1c6affd640 1 -- 172.21.15.43:0/1835969766 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563685bef990 con 0x563685899f60 2024-04-25T10:36:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.803+0000 7f1c6affd640 10 client.16533 handle_command_reply: tid=0 2024-04-25T10:36:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.803+0000 7f1c987d6740 1 client.16533 shutdown 2024-04-25T10:36:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.803+0000 7f1c987d6740 1 -- 172.21.15.43:0/1835969766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 msgr2=0x563685c865e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.803+0000 7f1c987d6740 1 --2- 172.21.15.43:0/1835969766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 0x563685c865e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009f50 tx=0x7f1c7c00e3b0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.803+0000 7f1c987d6740 1 -- 172.21.15.43:0/1835969766 shutdown_connections 2024-04-25T10:36:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.803+0000 7f1c987d6740 1 --2- 172.21.15.43:0/1835969766 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563685899f60 0x563685c985a0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.803+0000 7f1c987d6740 1 --2- 172.21.15.43:0/1835969766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858db390 0x563685c8c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.803+0000 7f1c987d6740 1 --2- 172.21.15.43:0/1835969766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858ba6b0 0x563685c86b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.803+0000 7f1c987d6740 1 --2- 172.21.15.43:0/1835969766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 0x563685c865e0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.803+0000 7f1c6affd640 0 client.16533 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.803+0000 7f1c987d6740 1 -- 172.21.15.43:0/1835969766 >> 172.21.15.43:0/1835969766 conn(0x5636858cf4e0 msgr2=0x563685c7d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.803+0000 7f1c987d6740 1 -- 172.21.15.43:0/1835969766 shutdown_connections 2024-04-25T10:36:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.803+0000 7f1c987d6740 1 -- 172.21.15.43:0/1835969766 wait complete. 2024-04-25T10:36:24.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.803+0000 7f1c987d6740 20 client.16533 trim_cache size 0 max 16384 2024-04-25T10:36:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.822+0000 7f1c987d6740 1 Processor -- start 2024-04-25T10:36:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.822+0000 7f1c987d6740 1 -- start start 2024-04-25T10:36:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.822+0000 7f1c987d6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858b24d0 0x563685c88520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.822+0000 7f1c987d6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858ba6b0 0x563685c8c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.822+0000 7f1c987d6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858db390 0x563685c8c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.822+0000 7f1c987d6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1b22e0 con 0x5636858db390 2024-04-25T10:36:24.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.822+0000 7f1c987d6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1afbb0 con 0x5636858ba6b0 2024-04-25T10:36:24.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.822+0000 7f1c987d6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1ada00 con 0x5636858b24d0 2024-04-25T10:36:24.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.822+0000 7f1c8b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858db390 0x563685c8c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.822+0000 7f1c8b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858db390 0x563685c8c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38616/0 (socket says 172.21.15.43:38616) 2024-04-25T10:36:24.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.822+0000 7f1c8b7fe640 1 -- 172.21.15.43:0/2817399329 learned_addr learned my addr 172.21.15.43:0/2817399329 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:24.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.822+0000 7f1c8affd640 1 --2- 172.21.15.43:0/2817399329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858b24d0 0x563685c88520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.822+0000 7f1c83fff640 1 --2- 172.21.15.43:0/2817399329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858ba6b0 0x563685c8c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.822+0000 7f1c8b7fe640 1 -- 172.21.15.43:0/2817399329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858b24d0 msgr2=0x563685c88520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:24.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.822+0000 7f1c8b7fe640 1 --2- 172.21.15.43:0/2817399329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858b24d0 0x563685c88520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.822+0000 7f1c8b7fe640 1 -- 172.21.15.43:0/2817399329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858ba6b0 msgr2=0x563685c8c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:24.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.822+0000 7f1c8b7fe640 1 --2- 172.21.15.43:0/2817399329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858ba6b0 0x563685c8c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.822+0000 7f1c8b7fe640 1 -- 172.21.15.43:0/2817399329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636858b30f0 con 0x5636858db390 2024-04-25T10:36:24.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.822+0000 7f1c8affd640 1 --2- 172.21.15.43:0/2817399329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858b24d0 0x563685c88520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:24.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.822+0000 7f1c83fff640 1 --2- 172.21.15.43:0/2817399329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858ba6b0 0x563685c8c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:36:24.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.823+0000 7f1c8b7fe640 1 --2- 172.21.15.43:0/2817399329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858db390 0x563685c8c960 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1c84035000 tx=0x7f1c84041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ef8da47aba7213 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:24.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.823+0000 7f1c817fa640 1 -- 172.21.15.43:0/2817399329 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84003ba0 con 0x5636858db390 2024-04-25T10:36:24.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.823+0000 7f1c817fa640 1 -- 172.21.15.43:0/2817399329 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c84036040 con 0x5636858db390 2024-04-25T10:36:24.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.823+0000 7f1c817fa640 1 -- 172.21.15.43:0/2817399329 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8402f950 con 0x5636858db390 2024-04-25T10:36:24.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.823+0000 7f1c987d6740 1 -- 172.21.15.43:0/2817399329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858db390 msgr2=0x563685c8c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:24.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.823+0000 7f1c987d6740 1 --2- 172.21.15.43:0/2817399329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858db390 0x563685c8c960 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1c84035000 tx=0x7f1c84041000 comp rx=0 tx=0).stop 2024-04-25T10:36:24.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.824+0000 7f1c987d6740 1 -- 172.21.15.43:0/2817399329 shutdown_connections 2024-04-25T10:36:24.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.824+0000 7f1c987d6740 1 --2- 172.21.15.43:0/2817399329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858ba6b0 0x563685c8c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.824+0000 7f1c987d6740 1 --2- 172.21.15.43:0/2817399329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858db390 0x563685c8c960 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.824+0000 7f1c987d6740 1 --2- 172.21.15.43:0/2817399329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858b24d0 0x563685c88520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.824+0000 7f1c987d6740 1 -- 172.21.15.43:0/2817399329 >> 172.21.15.43:0/2817399329 conn(0x5636858cf4e0 msgr2=0x563685c7dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:24.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.824+0000 7f1c987d6740 1 -- 172.21.15.43:0/2817399329 shutdown_connections 2024-04-25T10:36:24.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.824+0000 7f1c987d6740 1 -- 172.21.15.43:0/2817399329 wait complete. 2024-04-25T10:36:24.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.824+0000 7f1c987d6740 1 -- start start 2024-04-25T10:36:24.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.824+0000 7f1c987d6740 1 Processor -- start 2024-04-25T10:36:24.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c987d6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 0x563685c92fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c987d6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858ba6b0 0x563685c8d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c987d6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858db390 0x563685c8d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c987d6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1ae070 con 0x5636858b24d0 2024-04-25T10:36:24.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c987d6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1af4e0 con 0x5636858ba6b0 2024-04-25T10:36:24.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c987d6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1ae740 con 0x5636858db390 2024-04-25T10:36:24.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c987d6740 10 client.0 _ll_register_callbacks cb 0x7f1c7c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:24.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c8affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 0x563685c92fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c8affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 0x563685c92fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38622/0 (socket says 172.21.15.43:38622) 2024-04-25T10:36:24.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c8affd640 1 -- 172.21.15.43:0/922251626 learned_addr learned my addr 172.21.15.43:0/922251626 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:24.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c8b7fe640 1 --2- 172.21.15.43:0/922251626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858db390 0x563685c8d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c83fff640 1 --2- 172.21.15.43:0/922251626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858ba6b0 0x563685c8d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c8affd640 1 -- 172.21.15.43:0/922251626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858db390 msgr2=0x563685c8d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c8affd640 1 --2- 172.21.15.43:0/922251626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858db390 0x563685c8d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c8affd640 1 -- 172.21.15.43:0/922251626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858ba6b0 msgr2=0x563685c8d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c8affd640 1 --2- 172.21.15.43:0/922251626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858ba6b0 0x563685c8d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c8affd640 1 -- 172.21.15.43:0/922251626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563685ba24b0 con 0x5636858b24d0 2024-04-25T10:36:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c83fff640 1 --2- 172.21.15.43:0/922251626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858ba6b0 0x563685c8d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:36:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c8affd640 1 --2- 172.21.15.43:0/922251626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 0x563685c92fe0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c014020 tx=0x7f1c7c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1bacf99783f58f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c6affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c6affd640 1 -- 172.21.15.43:0/922251626 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c03b060 con 0x5636858b24d0 2024-04-25T10:36:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c6affd640 1 -- 172.21.15.43:0/922251626 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f1c7c005260 con 0x5636858b24d0 2024-04-25T10:36:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.825+0000 7f1c987d6740 1 -- 172.21.15.43:0/922251626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c84036040 con 0x5636858b24d0 2024-04-25T10:36:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.826+0000 7f1c6affd640 1 -- 172.21.15.43:0/922251626 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1c7c027040 con 0x5636858b24d0 2024-04-25T10:36:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.826+0000 7f1c987d6740 10 client.16539 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:24.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.826+0000 7f1c987d6740 1 -- 172.21.15.43:0/922251626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563685cff520 con 0x5636858b24d0 2024-04-25T10:36:24.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.826+0000 7f1c6affd640 1 -- 172.21.15.43:0/922251626 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c84036040 con 0x5636858b24d0 2024-04-25T10:36:24.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.826+0000 7f1c6affd640 1 -- 172.21.15.43:0/922251626 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1c7c0236b0 con 0x5636858b24d0 2024-04-25T10:36:24.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.826+0000 7f1c987d6740 10 client.16539 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:24.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.826+0000 7f1c987d6740 10 client.16539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:24.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.826+0000 7f1c987d6740 20 client.16539 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:24.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.826+0000 7f1c987d6740 1 --2- 172.21.15.43:0/922251626 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56368586f3a0 0x563685d01ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.826+0000 7f1c987d6740 4 client.16539 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:36:24.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.826+0000 7f1c987d6740 1 -- 172.21.15.43:0/922251626 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563685be9800 con 0x56368586f3a0 2024-04-25T10:36:24.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.826+0000 7f1c83fff640 1 --2- 172.21.15.43:0/922251626 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56368586f3a0 0x563685d01ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.827+0000 7f1c6affd640 10 client.16539 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:24.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.827+0000 7f1c83fff640 1 --2- 172.21.15.43:0/922251626 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56368586f3a0 0x563685d01ce0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8e21e1b06c73f3a server_cookie=f0ac2d0bacd5b0d4 in_seq=0 out_seq=0 2024-04-25T10:36:24.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.827+0000 7f1c6affd640 1 -- 172.21.15.43:0/922251626 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563685be9800 con 0x56368586f3a0 2024-04-25T10:36:24.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.827+0000 7f1c6affd640 10 client.16539 handle_command_reply: tid=0 2024-04-25T10:36:24.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.827+0000 7f1c987d6740 1 client.16539 shutdown 2024-04-25T10:36:24.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.827+0000 7f1c987d6740 1 -- 172.21.15.43:0/922251626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 msgr2=0x563685c92fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:24.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.827+0000 7f1c987d6740 1 --2- 172.21.15.43:0/922251626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 0x563685c92fe0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c014020 tx=0x7f1c7c011810 comp rx=0 tx=0).stop 2024-04-25T10:36:24.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.827+0000 7f1c987d6740 1 -- 172.21.15.43:0/922251626 shutdown_connections 2024-04-25T10:36:24.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.827+0000 7f1c987d6740 1 --2- 172.21.15.43:0/922251626 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56368586f3a0 0x563685d01ce0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.827+0000 7f1c987d6740 1 --2- 172.21.15.43:0/922251626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636858ba6b0 0x563685c8d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.827+0000 7f1c987d6740 1 --2- 172.21.15.43:0/922251626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636858db390 0x563685c8d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.827+0000 7f1c987d6740 1 --2- 172.21.15.43:0/922251626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636858b24d0 0x563685c92fe0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.827+0000 7f1c6affd640 0 client.16539 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:24.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.827+0000 7f1c987d6740 1 -- 172.21.15.43:0/922251626 >> 172.21.15.43:0/922251626 conn(0x5636858cf4e0 msgr2=0x563685c7dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:24.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.827+0000 7f1c987d6740 1 -- 172.21.15.43:0/922251626 shutdown_connections 2024-04-25T10:36:24.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.827+0000 7f1c987d6740 1 -- 172.21.15.43:0/922251626 wait complete. 2024-04-25T10:36:24.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.827+0000 7f1c987d6740 20 client.16539 trim_cache size 0 max 16384 2024-04-25T10:36:24.837 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:36:24.837 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:36:24.837 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:36:24.837 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:36:24.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.828+0000 7f1c91851640 1 -- 172.21.15.43:0/1140552808 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1c780afc60 msgr2=0x7f1c780b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:24.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.828+0000 7f1c91851640 1 --2- 172.21.15.43:0/1140552808 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1c780afc60 0x7f1c780b2170 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x563685c7cb60 tx=0x7f1c840095c0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.828+0000 7f1c91851640 1 -- 172.21.15.43:0/1140552808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8c109b10 msgr2=0x7f1c8c19ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:24.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.828+0000 7f1c91851640 1 --2- 172.21.15.43:0/1140552808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8c109b10 0x7f1c8c19ac60 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1c74005fd0 tx=0x7f1c7400dde0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.828+0000 7f1c91851640 1 -- 172.21.15.43:0/1140552808 shutdown_connections 2024-04-25T10:36:24.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.828+0000 7f1c91851640 1 --2- 172.21.15.43:0/1140552808 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1c780afc60 0x7f1c780b2170 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.828+0000 7f1c91851640 1 --2- 172.21.15.43:0/1140552808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c8c109fb0 0x7f1c8c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.828+0000 7f1c91851640 1 --2- 172.21.15.43:0/1140552808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8c109b10 0x7f1c8c19ac60 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.828+0000 7f1c91851640 1 --2- 172.21.15.43:0/1140552808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c8c1042b0 0x7f1c8c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.828+0000 7f1c91851640 1 -- 172.21.15.43:0/1140552808 >> 172.21.15.43:0/1140552808 conn(0x7f1c8c0ff360 msgr2=0x7f1c8c106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:24.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.828+0000 7f1c91851640 1 -- 172.21.15.43:0/1140552808 shutdown_connections 2024-04-25T10:36:24.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.828+0000 7f1c91851640 1 -- 172.21.15.43:0/1140552808 wait complete. 2024-04-25T10:36:24.842 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:36:24.842 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:36:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.952+0000 7fd0b9bd7640 1 -- 172.21.15.43:0/1178043124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b4108a40 msgr2=0x7fd0b4102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.952+0000 7fd0b9bd7640 1 --2- 172.21.15.43:0/1178043124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b4108a40 0x7fd0b4102620 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009900 tx=0x7fd0a402a410 comp rx=0 tx=0).stop 2024-04-25T10:36:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.952+0000 7fd0b9bd7640 1 -- 172.21.15.43:0/1178043124 shutdown_connections 2024-04-25T10:36:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.952+0000 7fd0b9bd7640 1 --2- 172.21.15.43:0/1178043124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0b4102b60 0x7fd0b4102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.952+0000 7fd0b9bd7640 1 --2- 172.21.15.43:0/1178043124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0b410d3a0 0x7fd0b410f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.952+0000 7fd0b9bd7640 1 --2- 172.21.15.43:0/1178043124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b4108a40 0x7fd0b4102620 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.952+0000 7fd0b9bd7640 1 -- 172.21.15.43:0/1178043124 >> 172.21.15.43:0/1178043124 conn(0x7fd0b4078040 msgr2=0x7fd0b4078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.952+0000 7fd0b9bd7640 1 -- 172.21.15.43:0/1178043124 shutdown_connections 2024-04-25T10:36:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.952+0000 7fd0b9bd7640 1 -- 172.21.15.43:0/1178043124 wait complete. 2024-04-25T10:36:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.953+0000 7fd0b9bd7640 1 Processor -- start 2024-04-25T10:36:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.953+0000 7fd0b9bd7640 1 -- start start 2024-04-25T10:36:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.953+0000 7fd0b9bd7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0b4102b60 0x7fd0b419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.953+0000 7fd0b9bd7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b4108a40 0x7fd0b419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.953+0000 7fd0b9bd7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0b410d3a0 0x7fd0b419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.953+0000 7fd0b9bd7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b4113d20 con 0x7fd0b4108a40 2024-04-25T10:36:24.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.953+0000 7fd0b9bd7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b4113ba0 con 0x7fd0b410d3a0 2024-04-25T10:36:24.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.953+0000 7fd0b9bd7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b4113ea0 con 0x7fd0b4102b60 2024-04-25T10:36:24.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.954+0000 7fd0b2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b4108a40 0x7fd0b419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.954+0000 7fd0b2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b4108a40 0x7fd0b419ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38640/0 (socket says 172.21.15.43:38640) 2024-04-25T10:36:24.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.954+0000 7fd0b2ffd640 1 -- 172.21.15.43:0/6115003 learned_addr learned my addr 172.21.15.43:0/6115003 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.954+0000 7fd0b3fff640 1 --2- 172.21.15.43:0/6115003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0b410d3a0 0x7fd0b419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.954+0000 7fd0b37fe640 1 --2- 172.21.15.43:0/6115003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0b4102b60 0x7fd0b419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.954+0000 7fd0b2ffd640 1 -- 172.21.15.43:0/6115003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0b4102b60 msgr2=0x7fd0b419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.954+0000 7fd0b2ffd640 1 --2- 172.21.15.43:0/6115003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0b4102b60 0x7fd0b419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.954+0000 7fd0b2ffd640 1 -- 172.21.15.43:0/6115003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0b410d3a0 msgr2=0x7fd0b419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.954+0000 7fd0b2ffd640 1 --2- 172.21.15.43:0/6115003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0b410d3a0 0x7fd0b419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.954+0000 7fd0b2ffd640 1 -- 172.21.15.43:0/6115003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b419f520 con 0x7fd0b4108a40 2024-04-25T10:36:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.954+0000 7fd0b2ffd640 1 --2- 172.21.15.43:0/6115003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b4108a40 0x7fd0b419ac20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8005fd0 tx=0x7fd0a800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.954+0000 7fd0b0ff9640 1 -- 172.21.15.43:0/6115003 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8017070 con 0x7fd0b4108a40 2024-04-25T10:36:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.954+0000 7fd0b0ff9640 1 -- 172.21.15.43:0/6115003 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0a8009c30 con 0x7fd0b4108a40 2024-04-25T10:36:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.954+0000 7fd0b9bd7640 1 -- 172.21.15.43:0/6115003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b419f810 con 0x7fd0b4108a40 2024-04-25T10:36:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.954+0000 7fd0b0ff9640 1 -- 172.21.15.43:0/6115003 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8003d10 con 0x7fd0b4108a40 2024-04-25T10:36:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.954+0000 7fd0b9bd7640 1 -- 172.21.15.43:0/6115003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b41a7190 con 0x7fd0b4108a40 2024-04-25T10:36:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.955+0000 7fd0b9bd7640 1 -- 172.21.15.43:0/6115003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0b406a190 con 0x7fd0b4108a40 2024-04-25T10:36:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.957+0000 7fd0b0ff9640 1 -- 172.21.15.43:0/6115003 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fd0a8003eb0 con 0x7fd0b4108a40 2024-04-25T10:36:24.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.957+0000 7fd0b0ff9640 1 --2- 172.21.15.43:0/6115003 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd09c0afc60 0x7fd09c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:24.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.958+0000 7fd0b0ff9640 1 -- 172.21.15.43:0/6115003 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fd0a80df430 con 0x7fd0b4108a40 2024-04-25T10:36:24.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.958+0000 7fd0b37fe640 1 --2- 172.21.15.43:0/6115003 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd09c0afc60 0x7fd09c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:24.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.958+0000 7fd0b37fe640 1 --2- 172.21.15.43:0/6115003 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd09c0afc60 0x7fd09c0b2170 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4000c00 tx=0x7fd0a4031400 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:24.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:24.961+0000 7fd0b0ff9640 1 -- 172.21.15.43:0/6115003 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd0a8020030 con 0x7fd0b4108a40 2024-04-25T10:36:25.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.188+0000 7fd0b9bd7640 1 -- 172.21.15.43:0/6115003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0b4106f00 con 0x7fd0b4108a40 2024-04-25T10:36:25.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.189+0000 7fd0b0ff9640 1 -- 172.21.15.43:0/6115003 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7fd0a8009de0 con 0x7fd0b4108a40 2024-04-25T10:36:25.191 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:36:25.191 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:36:25.191 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:36:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.191+0000 7fd0b9bd7640 1 -- 172.21.15.43:0/6115003 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd09c0afc60 msgr2=0x7fd09c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.191+0000 7fd0b9bd7640 1 --2- 172.21.15.43:0/6115003 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd09c0afc60 0x7fd09c0b2170 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4000c00 tx=0x7fd0a4031400 comp rx=0 tx=0).stop 2024-04-25T10:36:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.191+0000 7fd0b9bd7640 1 -- 172.21.15.43:0/6115003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b4108a40 msgr2=0x7fd0b419ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.191+0000 7fd0b9bd7640 1 --2- 172.21.15.43:0/6115003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b4108a40 0x7fd0b419ac20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8005fd0 tx=0x7fd0a800dde0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.191+0000 7fd0b9bd7640 1 -- 172.21.15.43:0/6115003 shutdown_connections 2024-04-25T10:36:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.191+0000 7fd0b9bd7640 1 --2- 172.21.15.43:0/6115003 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd09c0afc60 0x7fd09c0b2170 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.191+0000 7fd0b9bd7640 1 --2- 172.21.15.43:0/6115003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0b410d3a0 0x7fd0b419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.191+0000 7fd0b9bd7640 1 --2- 172.21.15.43:0/6115003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b4108a40 0x7fd0b419ac20 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.191+0000 7fd0b9bd7640 1 --2- 172.21.15.43:0/6115003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0b4102b60 0x7fd0b419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.191+0000 7fd0b9bd7640 1 -- 172.21.15.43:0/6115003 >> 172.21.15.43:0/6115003 conn(0x7fd0b4078040 msgr2=0x7fd0b41040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.191+0000 7fd0b9bd7640 1 -- 172.21.15.43:0/6115003 shutdown_connections 2024-04-25T10:36:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.191+0000 7fd0b9bd7640 1 -- 172.21.15.43:0/6115003 wait complete. 2024-04-25T10:36:25.204 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:36:25.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.316+0000 7f1c17a92640 1 -- 172.21.15.43:0/2385307956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c10104c70 msgr2=0x7f1c10105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.316+0000 7f1c17a92640 1 --2- 172.21.15.43:0/2385307956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c10104c70 0x7f1c10105080 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009900 tx=0x7f1c0c02a410 comp rx=0 tx=0).stop 2024-04-25T10:36:25.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.316+0000 7f1c17a92640 1 -- 172.21.15.43:0/2385307956 shutdown_connections 2024-04-25T10:36:25.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.316+0000 7f1c17a92640 1 --2- 172.21.15.43:0/2385307956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c1010b6d0 0x7f1c1010daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.316+0000 7f1c17a92640 1 --2- 172.21.15.43:0/2385307956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c1010e0c0 0x7f1c101105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.316+0000 7f1c17a92640 1 --2- 172.21.15.43:0/2385307956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c10104c70 0x7f1c10105080 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.316+0000 7f1c17a92640 1 -- 172.21.15.43:0/2385307956 >> 172.21.15.43:0/2385307956 conn(0x7f1c10078110 msgr2=0x7f1c10078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:25.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.316+0000 7f1c17a92640 1 -- 172.21.15.43:0/2385307956 shutdown_connections 2024-04-25T10:36:25.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.316+0000 7f1c17a92640 1 -- 172.21.15.43:0/2385307956 wait complete. 2024-04-25T10:36:25.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.317+0000 7f1c17a92640 1 Processor -- start 2024-04-25T10:36:25.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.317+0000 7f1c17a92640 1 -- start start 2024-04-25T10:36:25.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.317+0000 7f1c17a92640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c10104c70 0x7f1c10102530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.317+0000 7f1c17a92640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c1010b6d0 0x7f1c10102a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.317+0000 7f1c17a92640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1010e0c0 0x7f1c10102fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.317+0000 7f1c17a92640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1c10113ce0 con 0x7f1c1010e0c0 2024-04-25T10:36:25.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.317+0000 7f1c17a92640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1c10113b60 con 0x7f1c1010b6d0 2024-04-25T10:36:25.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.317+0000 7f1c17a92640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1c10113e60 con 0x7f1c10104c70 2024-04-25T10:36:25.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.317+0000 7f1c1602f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1010e0c0 0x7f1c10102fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.317+0000 7f1c1602f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1010e0c0 0x7f1c10102fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38650/0 (socket says 172.21.15.43:38650) 2024-04-25T10:36:25.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.317+0000 7f1c1602f640 1 -- 172.21.15.43:0/775113128 learned_addr learned my addr 172.21.15.43:0/775113128 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:25.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.318+0000 7f1c1502d640 1 --2- 172.21.15.43:0/775113128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c1010b6d0 0x7f1c10102a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.318+0000 7f1c1582e640 1 --2- 172.21.15.43:0/775113128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c10104c70 0x7f1c10102530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.318+0000 7f1c1602f640 1 -- 172.21.15.43:0/775113128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c10104c70 msgr2=0x7f1c10102530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.318+0000 7f1c1602f640 1 --2- 172.21.15.43:0/775113128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c10104c70 0x7f1c10102530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.318+0000 7f1c1602f640 1 -- 172.21.15.43:0/775113128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c1010b6d0 msgr2=0x7f1c10102a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.318+0000 7f1c1602f640 1 --2- 172.21.15.43:0/775113128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c1010b6d0 0x7f1c10102a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.318+0000 7f1c1602f640 1 -- 172.21.15.43:0/775113128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c101a8a30 con 0x7f1c1010e0c0 2024-04-25T10:36:25.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.318+0000 7f1c1582e640 1 --2- 172.21.15.43:0/775113128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c10104c70 0x7f1c10102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:25.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.318+0000 7f1c1602f640 1 --2- 172.21.15.43:0/775113128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1010e0c0 0x7f1c10102fb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800eaa0 tx=0x7f1c0800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:25.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.318+0000 7f1c06ffd640 1 -- 172.21.15.43:0/775113128 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08004d20 con 0x7f1c1010e0c0 2024-04-25T10:36:25.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.318+0000 7f1c17a92640 1 -- 172.21.15.43:0/775113128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c101a8d20 con 0x7f1c1010e0c0 2024-04-25T10:36:25.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.318+0000 7f1c17a92640 1 -- 172.21.15.43:0/775113128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c101a9210 con 0x7f1c1010e0c0 2024-04-25T10:36:25.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.318+0000 7f1c06ffd640 1 -- 172.21.15.43:0/775113128 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c08005020 con 0x7f1c1010e0c0 2024-04-25T10:36:25.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.318+0000 7f1c06ffd640 1 -- 172.21.15.43:0/775113128 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c080116b0 con 0x7f1c1010e0c0 2024-04-25T10:36:25.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.322+0000 7f1c1ea11740 1 Processor -- start 2024-04-25T10:36:25.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.322+0000 7f1c06ffd640 1 -- 172.21.15.43:0/775113128 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f1c0800caf0 con 0x7f1c1010e0c0 2024-04-25T10:36:25.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.322+0000 7f1c1ea11740 1 -- start start 2024-04-25T10:36:25.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.322+0000 7f1c1ea11740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564040755390 0x564040af6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.322+0000 7f1c1ea11740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56404072c4d0 0x564040af6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.322+0000 7f1c1ea11740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5640407346b0 0x564040b036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.322+0000 7f1c1ea11740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564040a5bd80 con 0x56404072c4d0 2024-04-25T10:36:25.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.322+0000 7f1c1ea11740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564040a15a20 con 0x564040755390 2024-04-25T10:36:25.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.322+0000 7f1c1ea11740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564040a57d40 con 0x5640407346b0 2024-04-25T10:36:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.322+0000 7f1c1582e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56404072c4d0 0x564040af6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.322+0000 7f1c1582e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56404072c4d0 0x564040af6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38666/0 (socket says 172.21.15.43:38666) 2024-04-25T10:36:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.322+0000 7f1c1582e640 1 -- 172.21.15.43:0/1731277240 learned_addr learned my addr 172.21.15.43:0/1731277240 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.323+0000 7f1c06ffd640 1 --2- 172.21.15.43:0/775113128 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1bfc0afce0 0x7f1bfc0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.323+0000 7f1c1502d640 1 --2- 172.21.15.43:0/1731277240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5640407346b0 0x564040b036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.323+0000 7f1c1502d640 1 --2- 172.21.15.43:0/1731277240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564040755390 0x564040af6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.323+0000 7f1c1602f640 1 --2- 172.21.15.43:0/775113128 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1bfc0afce0 0x7f1bfc0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.323+0000 7f1c1582e640 1 -- 172.21.15.43:0/1731277240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5640407346b0 msgr2=0x564040b036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.323+0000 7f1c1582e640 1 --2- 172.21.15.43:0/1731277240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5640407346b0 0x564040b036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.323+0000 7f1c1582e640 1 -- 172.21.15.43:0/1731277240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564040755390 msgr2=0x564040af6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.323+0000 7f1c1582e640 1 --2- 172.21.15.43:0/1731277240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564040755390 0x564040af6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.323+0000 7f1c1582e640 1 -- 172.21.15.43:0/1731277240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564040a20c30 con 0x56404072c4d0 2024-04-25T10:36:25.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.323+0000 7f1c06ffd640 1 -- 172.21.15.43:0/775113128 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f1c080df0b0 con 0x7f1c1010e0c0 2024-04-25T10:36:25.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.323+0000 7f1c1582e640 1 --2- 172.21.15.43:0/1731277240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56404072c4d0 0x564040af6d50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009900 tx=0x7f1c0c032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6619115259a29b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.323+0000 7f1bfbfff640 1 -- 172.21.15.43:0/1731277240 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c03f070 con 0x56404072c4d0 2024-04-25T10:36:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.323+0000 7f1c1602f640 1 --2- 172.21.15.43:0/775113128 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1bfc0afce0 0x7f1bfc0b21f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1c080a3300 tx=0x7f1c080d59d0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.323+0000 7f1bfbfff640 1 -- 172.21.15.43:0/1731277240 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c0c0316a0 con 0x56404072c4d0 2024-04-25T10:36:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.323+0000 7f1bfbfff640 1 -- 172.21.15.43:0/1731277240 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c035070 con 0x56404072c4d0 2024-04-25T10:36:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.324+0000 7f1c1ea11740 1 -- 172.21.15.43:0/1731277240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56404072c4d0 msgr2=0x564040af6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.324+0000 7f1c1ea11740 1 --2- 172.21.15.43:0/1731277240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56404072c4d0 0x564040af6d50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009900 tx=0x7f1c0c032e70 comp rx=0 tx=0).stop 2024-04-25T10:36:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.324+0000 7f1c1ea11740 1 -- 172.21.15.43:0/1731277240 shutdown_connections 2024-04-25T10:36:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.324+0000 7f1c1ea11740 1 --2- 172.21.15.43:0/1731277240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5640407346b0 0x564040b036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.324+0000 7f1c1ea11740 1 --2- 172.21.15.43:0/1731277240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56404072c4d0 0x564040af6d50 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.324+0000 7f1c1ea11740 1 --2- 172.21.15.43:0/1731277240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564040755390 0x564040af6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.324+0000 7f1c1ea11740 1 -- 172.21.15.43:0/1731277240 >> 172.21.15.43:0/1731277240 conn(0x5640407494e0 msgr2=0x564040af2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.324+0000 7f1c1ea11740 1 -- 172.21.15.43:0/1731277240 shutdown_connections 2024-04-25T10:36:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.324+0000 7f1c1ea11740 1 -- 172.21.15.43:0/1731277240 wait complete. 2024-04-25T10:36:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.325+0000 7f1c1ea11740 1 -- start start 2024-04-25T10:36:25.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.325+0000 7f1c1ea11740 1 Processor -- start 2024-04-25T10:36:25.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.325+0000 7f1c1ea11740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56404072c4d0 0x564040b01190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.325+0000 7f1c1ea11740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 0x564040b016d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.325+0000 7f1c1ea11740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564040755390 0x564040b0d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.325+0000 7f1c1ea11740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564040b03d30 con 0x5640407346b0 2024-04-25T10:36:25.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.325+0000 7f1c1ea11740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5640409e2dc0 con 0x56404072c4d0 2024-04-25T10:36:25.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.325+0000 7f1c1ea11740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564040b03eb0 con 0x564040755390 2024-04-25T10:36:25.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.325+0000 7f1c1ea11740 10 client.0 _ll_register_callbacks cb 0x5640409fe600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:25.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.325+0000 7f1c1582e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 0x564040b016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.325+0000 7f1c1582e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 0x564040b016d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38672/0 (socket says 172.21.15.43:38672) 2024-04-25T10:36:25.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.325+0000 7f1c1582e640 1 -- 172.21.15.43:0/2588782807 learned_addr learned my addr 172.21.15.43:0/2588782807 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:25.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.325+0000 7f1c1502d640 1 --2- 172.21.15.43:0/2588782807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56404072c4d0 0x564040b01190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.326+0000 7f1c1502d640 1 --2- 172.21.15.43:0/2588782807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564040755390 0x564040b0d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.326+0000 7f1c1582e640 1 -- 172.21.15.43:0/2588782807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564040755390 msgr2=0x564040b0d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.326+0000 7f1c1582e640 1 --2- 172.21.15.43:0/2588782807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564040755390 0x564040b0d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.326+0000 7f1c1582e640 1 -- 172.21.15.43:0/2588782807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56404072c4d0 msgr2=0x564040b01190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.326+0000 7f1c1582e640 1 --2- 172.21.15.43:0/2588782807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56404072c4d0 0x564040b01190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.326+0000 7f1c1582e640 1 -- 172.21.15.43:0/2588782807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56404076d3f0 con 0x5640407346b0 2024-04-25T10:36:25.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.326+0000 7f1c1582e640 1 --2- 172.21.15.43:0/2588782807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 0x564040b016d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c02a450 tx=0x7f1c0c005f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be84591b4bdda510 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:25.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.326+0000 7f1bf97fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:25.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.326+0000 7f1bf97fa640 1 -- 172.21.15.43:0/2588782807 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c04b060 con 0x5640407346b0 2024-04-25T10:36:25.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.326+0000 7f1c1ea11740 1 -- 172.21.15.43:0/2588782807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c0c0316a0 con 0x5640407346b0 2024-04-25T10:36:25.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.326+0000 7f1bf97fa640 1 -- 172.21.15.43:0/2588782807 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f1c0c004030 con 0x5640407346b0 2024-04-25T10:36:25.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.326+0000 7f1bf97fa640 1 -- 172.21.15.43:0/2588782807 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1c0c03f030 con 0x5640407346b0 2024-04-25T10:36:25.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.326+0000 7f1c1ea11740 10 client.16557 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:25.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.326+0000 7f1c1ea11740 1 -- 172.21.15.43:0/2588782807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5640408377d0 con 0x5640407346b0 2024-04-25T10:36:25.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.327+0000 7f1bf97fa640 1 -- 172.21.15.43:0/2588782807 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c0c0316a0 con 0x5640407346b0 2024-04-25T10:36:25.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.327+0000 7f1bf97fa640 1 -- 172.21.15.43:0/2588782807 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1c0c0035f0 con 0x5640407346b0 2024-04-25T10:36:25.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.327+0000 7f1c1ea11740 10 client.16557 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:25.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.327+0000 7f1c1ea11740 10 client.16557 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:36:25.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.327+0000 7f1c1ea11740 20 client.16557 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:25.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.327+0000 7f1c1ea11740 1 --2- 172.21.15.43:0/2588782807 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x564040713f60 0x564040b134b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.327+0000 7f1c1ea11740 4 client.16557 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:36:25.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.327+0000 7f1c1ea11740 1 -- 172.21.15.43:0/2588782807 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564040a69990 con 0x564040713f60 2024-04-25T10:36:25.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.327+0000 7f1c1502d640 1 --2- 172.21.15.43:0/2588782807 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x564040713f60 0x564040b134b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.328+0000 7f1c1502d640 1 --2- 172.21.15.43:0/2588782807 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x564040713f60 0x564040b134b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76584ec565d7973a server_cookie=363b98e02c7922d in_seq=0 out_seq=0 2024-04-25T10:36:25.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.328+0000 7f1bf97fa640 10 client.16557 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:36:25.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.328+0000 7f1bf97fa640 1 -- 172.21.15.43:0/2588782807 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564040a69990 con 0x564040713f60 2024-04-25T10:36:25.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.328+0000 7f1bf97fa640 10 client.16557 handle_command_reply: tid=0 2024-04-25T10:36:25.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.328+0000 7f1c1ea11740 1 client.16557 shutdown 2024-04-25T10:36:25.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.329+0000 7f1c1ea11740 1 -- 172.21.15.43:0/2588782807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 msgr2=0x564040b016d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.329+0000 7f1c1ea11740 1 --2- 172.21.15.43:0/2588782807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 0x564040b016d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c02a450 tx=0x7f1c0c005f20 comp rx=0 tx=0).stop 2024-04-25T10:36:25.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.329+0000 7f1c1ea11740 1 -- 172.21.15.43:0/2588782807 shutdown_connections 2024-04-25T10:36:25.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.329+0000 7f1c1ea11740 1 --2- 172.21.15.43:0/2588782807 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x564040713f60 0x564040b134b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.329+0000 7f1c1ea11740 1 --2- 172.21.15.43:0/2588782807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564040755390 0x564040b0d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.329+0000 7f1c1ea11740 1 --2- 172.21.15.43:0/2588782807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 0x564040b016d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.329+0000 7f1c1ea11740 1 --2- 172.21.15.43:0/2588782807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56404072c4d0 0x564040b01190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.329+0000 7f1bf97fa640 0 client.16557 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:36:25.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.329+0000 7f1c1ea11740 1 -- 172.21.15.43:0/2588782807 >> 172.21.15.43:0/2588782807 conn(0x5640407494e0 msgr2=0x564040af8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:25.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.329+0000 7f1c1ea11740 1 -- 172.21.15.43:0/2588782807 shutdown_connections 2024-04-25T10:36:25.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.329+0000 7f1c1ea11740 1 -- 172.21.15.43:0/2588782807 wait complete. 2024-04-25T10:36:25.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.329+0000 7f1c1ea11740 20 client.16557 trim_cache size 0 max 16384 2024-04-25T10:36:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.343+0000 7f1c1ea11740 1 Processor -- start 2024-04-25T10:36:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.343+0000 7f1c1ea11740 1 -- start start 2024-04-25T10:36:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.343+0000 7f1c1ea11740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564040713f60 0x564040b00bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.343+0000 7f1c1ea11740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56404072c4d0 0x564040b0d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.343+0000 7f1c1ea11740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 0x564040b164d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.343+0000 7f1c1ea11740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf4010f80 con 0x5640407346b0 2024-04-25T10:36:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.343+0000 7f1c1ea11740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf400e850 con 0x56404072c4d0 2024-04-25T10:36:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.343+0000 7f1c1ea11740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf400c6a0 con 0x564040713f60 2024-04-25T10:36:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.343+0000 7f1c1502d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 0x564040b164d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.343+0000 7f1c1502d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 0x564040b164d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38688/0 (socket says 172.21.15.43:38688) 2024-04-25T10:36:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.343+0000 7f1c1502d640 1 -- 172.21.15.43:0/3146035367 learned_addr learned my addr 172.21.15.43:0/3146035367 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.343+0000 7f1c1502d640 1 -- 172.21.15.43:0/3146035367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564040713f60 msgr2=0x564040b00bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.343+0000 7f1c1502d640 1 --2- 172.21.15.43:0/3146035367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564040713f60 0x564040b00bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.343+0000 7f1c1502d640 1 -- 172.21.15.43:0/3146035367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56404072c4d0 msgr2=0x564040b0d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.343+0000 7f1c1502d640 1 --2- 172.21.15.43:0/3146035367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56404072c4d0 0x564040b0d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.343+0000 7f1c1502d640 1 -- 172.21.15.43:0/3146035367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564040b03ec0 con 0x5640407346b0 2024-04-25T10:36:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.343+0000 7f1c1502d640 1 --2- 172.21.15.43:0/3146035367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 0x564040b164d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x564040af7910 tx=0x7f1c00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faad83df40a75060 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.343+0000 7f1bfb7fe640 1 -- 172.21.15.43:0/3146035367 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0002b030 con 0x5640407346b0 2024-04-25T10:36:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.344+0000 7f1bfb7fe640 1 -- 172.21.15.43:0/3146035367 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c00023dd0 con 0x5640407346b0 2024-04-25T10:36:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.344+0000 7f1bfb7fe640 1 -- 172.21.15.43:0/3146035367 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c000047a0 con 0x5640407346b0 2024-04-25T10:36:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.344+0000 7f1c1ea11740 1 -- 172.21.15.43:0/3146035367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 msgr2=0x564040b164d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.344+0000 7f1c1ea11740 1 --2- 172.21.15.43:0/3146035367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 0x564040b164d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x564040af7910 tx=0x7f1c00004730 comp rx=0 tx=0).stop 2024-04-25T10:36:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.344+0000 7f1c1ea11740 1 -- 172.21.15.43:0/3146035367 shutdown_connections 2024-04-25T10:36:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.344+0000 7f1c1ea11740 1 --2- 172.21.15.43:0/3146035367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56404072c4d0 0x564040b0d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.344+0000 7f1c1ea11740 1 --2- 172.21.15.43:0/3146035367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 0x564040b164d0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.344+0000 7f1c1ea11740 1 --2- 172.21.15.43:0/3146035367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564040713f60 0x564040b00bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.344+0000 7f1c1ea11740 1 -- 172.21.15.43:0/3146035367 >> 172.21.15.43:0/3146035367 conn(0x5640407494e0 msgr2=0x564040b004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.344+0000 7f1c1ea11740 1 -- 172.21.15.43:0/3146035367 shutdown_connections 2024-04-25T10:36:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.344+0000 7f1c1ea11740 1 -- 172.21.15.43:0/3146035367 wait complete. 2024-04-25T10:36:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.345+0000 7f1c1ea11740 1 -- start start 2024-04-25T10:36:25.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.345+0000 7f1c1ea11740 1 Processor -- start 2024-04-25T10:36:25.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.345+0000 7f1c1ea11740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564040713f60 0x564040b16b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.345+0000 7f1c1ea11740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404072c4d0 0x564040b13ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.345+0000 7f1c1ea11740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 0x564040b144c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.345+0000 7f1c1ea11740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf400cd10 con 0x5640407346b0 2024-04-25T10:36:25.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.345+0000 7f1c1ea11740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf400e180 con 0x564040713f60 2024-04-25T10:36:25.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.345+0000 7f1c1ea11740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf400d3e0 con 0x56404072c4d0 2024-04-25T10:36:25.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.345+0000 7f1c1ea11740 10 client.0 _ll_register_callbacks cb 0x7f1c0000add0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:25.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.345+0000 7f1c1502d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 0x564040b144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.345+0000 7f1c1502d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 0x564040b144c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38690/0 (socket says 172.21.15.43:38690) 2024-04-25T10:36:25.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.345+0000 7f1c1502d640 1 -- 172.21.15.43:0/3356412538 learned_addr learned my addr 172.21.15.43:0/3356412538 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.345+0000 7f1c1582e640 1 --2- 172.21.15.43:0/3356412538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404072c4d0 0x564040b13ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.346+0000 7f1c1502d640 1 --2- 172.21.15.43:0/3356412538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564040713f60 0x564040b16b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.346+0000 7f1c1502d640 1 -- 172.21.15.43:0/3356412538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404072c4d0 msgr2=0x564040b13ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.346+0000 7f1c1502d640 1 --2- 172.21.15.43:0/3356412538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404072c4d0 0x564040b13ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.346+0000 7f1c1502d640 1 -- 172.21.15.43:0/3356412538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564040713f60 msgr2=0x564040b16b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.346+0000 7f1c1502d640 1 --2- 172.21.15.43:0/3356412538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564040713f60 0x564040b16b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.346+0000 7f1c1502d640 1 -- 172.21.15.43:0/3356412538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564040b14c70 con 0x5640407346b0 2024-04-25T10:36:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.346+0000 7f1c1502d640 1 --2- 172.21.15.43:0/3356412538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 0x564040b144c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000a310 tx=0x7f1c0000aad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3842c73d6f42cd49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.346+0000 7f1bf97fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.346+0000 7f1bf97fa640 1 -- 172.21.15.43:0/3356412538 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0002b040 con 0x5640407346b0 2024-04-25T10:36:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.346+0000 7f1bf97fa640 1 -- 172.21.15.43:0/3356412538 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f1c00012270 con 0x5640407346b0 2024-04-25T10:36:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.346+0000 7f1c1ea11740 1 -- 172.21.15.43:0/3356412538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c00023dd0 con 0x5640407346b0 2024-04-25T10:36:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.346+0000 7f1bf97fa640 1 -- 172.21.15.43:0/3356412538 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1c00040050 con 0x5640407346b0 2024-04-25T10:36:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.346+0000 7f1c1ea11740 10 client.16563 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.346+0000 7f1c1ea11740 1 -- 172.21.15.43:0/3356412538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564040b7b5a0 con 0x5640407346b0 2024-04-25T10:36:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.346+0000 7f1bf97fa640 1 -- 172.21.15.43:0/3356412538 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c00023dd0 con 0x5640407346b0 2024-04-25T10:36:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.347+0000 7f1bf97fa640 1 -- 172.21.15.43:0/3356412538 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1c000305f0 con 0x5640407346b0 2024-04-25T10:36:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.347+0000 7f1c1ea11740 10 client.16563 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.347+0000 7f1c1ea11740 10 client.16563 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:36:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.347+0000 7f1c1ea11740 20 client.16563 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.347+0000 7f1c1ea11740 1 --2- 172.21.15.43:0/3356412538 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x564040755390 0x564040b7dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.347+0000 7f1c1ea11740 4 client.16563 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:36:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.347+0000 7f1c1ea11740 1 -- 172.21.15.43:0/3356412538 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564040a63800 con 0x564040755390 2024-04-25T10:36:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.347+0000 7f1c1582e640 1 --2- 172.21.15.43:0/3356412538 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x564040755390 0x564040b7dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.347+0000 7f1bf97fa640 10 client.16563 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:36:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.347+0000 7f1c1582e640 1 --2- 172.21.15.43:0/3356412538 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x564040755390 0x564040b7dc50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fdbe9aeae262df58 server_cookie=77a130191036752e in_seq=0 out_seq=0 2024-04-25T10:36:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1bf97fa640 1 -- 172.21.15.43:0/3356412538 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564040a63800 con 0x564040755390 2024-04-25T10:36:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1bf97fa640 10 client.16563 handle_command_reply: tid=0 2024-04-25T10:36:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c1ea11740 1 client.16563 shutdown 2024-04-25T10:36:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c1ea11740 1 -- 172.21.15.43:0/3356412538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 msgr2=0x564040b144c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c1ea11740 1 --2- 172.21.15.43:0/3356412538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 0x564040b144c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000a310 tx=0x7f1c0000aad0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c1ea11740 1 -- 172.21.15.43:0/3356412538 shutdown_connections 2024-04-25T10:36:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c1ea11740 1 --2- 172.21.15.43:0/3356412538 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x564040755390 0x564040b7dc50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c1ea11740 1 --2- 172.21.15.43:0/3356412538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5640407346b0 0x564040b144c0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c1ea11740 1 --2- 172.21.15.43:0/3356412538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404072c4d0 0x564040b13ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c1ea11740 1 --2- 172.21.15.43:0/3356412538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564040713f60 0x564040b16b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1bf97fa640 0 client.16563 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:36:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c1ea11740 1 -- 172.21.15.43:0/3356412538 >> 172.21.15.43:0/3356412538 conn(0x5640407494e0 msgr2=0x564040af8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c1ea11740 1 -- 172.21.15.43:0/3356412538 shutdown_connections 2024-04-25T10:36:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c1ea11740 1 -- 172.21.15.43:0/3356412538 wait complete. 2024-04-25T10:36:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c1ea11740 20 client.16563 trim_cache size 0 max 16384 2024-04-25T10:36:25.374 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:36:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c17a92640 1 -- 172.21.15.43:0/775113128 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1bfc0afce0 msgr2=0x7f1bfc0b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c17a92640 1 --2- 172.21.15.43:0/775113128 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1bfc0afce0 0x7f1bfc0b21f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1c080a3300 tx=0x7f1c080d59d0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c17a92640 1 -- 172.21.15.43:0/775113128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1010e0c0 msgr2=0x7f1c10102fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c17a92640 1 --2- 172.21.15.43:0/775113128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1010e0c0 0x7f1c10102fb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800eaa0 tx=0x7f1c0800eea0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c17a92640 1 -- 172.21.15.43:0/775113128 shutdown_connections 2024-04-25T10:36:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c17a92640 1 --2- 172.21.15.43:0/775113128 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1bfc0afce0 0x7f1bfc0b21f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c17a92640 1 --2- 172.21.15.43:0/775113128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c1010b6d0 0x7f1c10102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c17a92640 1 --2- 172.21.15.43:0/775113128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1010e0c0 0x7f1c10102fb0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.348+0000 7f1c17a92640 1 --2- 172.21.15.43:0/775113128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c10104c70 0x7f1c10102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.349+0000 7f1c17a92640 1 -- 172.21.15.43:0/775113128 >> 172.21.15.43:0/775113128 conn(0x7f1c10078110 msgr2=0x7f1c10101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.349+0000 7f1c17a92640 1 -- 172.21.15.43:0/775113128 shutdown_connections 2024-04-25T10:36:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.349+0000 7f1c17a92640 1 -- 172.21.15.43:0/775113128 wait complete. 2024-04-25T10:36:25.376 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:36:25.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.488+0000 7f8b7208c640 1 -- 172.21.15.43:0/1980481704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b6c1090c0 msgr2=0x7f8b6c103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.488+0000 7f8b7208c640 1 --2- 172.21.15.43:0/1980481704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b6c1090c0 0x7f8b6c103230 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8b60009900 tx=0x7f8b6002a410 comp rx=0 tx=0).stop 2024-04-25T10:36:25.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.488+0000 7f8b7208c640 1 -- 172.21.15.43:0/1980481704 shutdown_connections 2024-04-25T10:36:25.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.488+0000 7f8b7208c640 1 --2- 172.21.15.43:0/1980481704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b6c1038a0 0x7f8b6c10f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.488+0000 7f8b7208c640 1 --2- 172.21.15.43:0/1980481704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b6c1090c0 0x7f8b6c103230 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.488+0000 7f8b7208c640 1 --2- 172.21.15.43:0/1980481704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b6c108c20 0x7f8b6c102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.488+0000 7f8b7208c640 1 -- 172.21.15.43:0/1980481704 >> 172.21.15.43:0/1980481704 conn(0x7f8b6c0fd850 msgr2=0x7f8b6c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:25.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.488+0000 7f8b7208c640 1 -- 172.21.15.43:0/1980481704 shutdown_connections 2024-04-25T10:36:25.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.488+0000 7f8b7208c640 1 -- 172.21.15.43:0/1980481704 wait complete. 2024-04-25T10:36:25.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.489+0000 7f8b7208c640 1 Processor -- start 2024-04-25T10:36:25.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.489+0000 7f8b7208c640 1 -- start start 2024-04-25T10:36:25.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.489+0000 7f8b7208c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b6c1038a0 0x7f8b6c19b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.489+0000 7f8b7208c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b6c108c20 0x7f8b6c19bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.489+0000 7f8b7208c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b6c1090c0 0x7f8b6c19c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.489+0000 7f8b7208c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c112e40 con 0x7f8b6c108c20 2024-04-25T10:36:25.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.489+0000 7f8b7208c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c112cc0 con 0x7f8b6c1090c0 2024-04-25T10:36:25.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.489+0000 7f8b7208c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c112fc0 con 0x7f8b6c1038a0 2024-04-25T10:36:25.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.489+0000 7f8b6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b6c108c20 0x7f8b6c19bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.489+0000 7f8b6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b6c108c20 0x7f8b6c19bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38714/0 (socket says 172.21.15.43:38714) 2024-04-25T10:36:25.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.489+0000 7f8b6affd640 1 -- 172.21.15.43:0/3164991289 learned_addr learned my addr 172.21.15.43:0/3164991289 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:25.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.489+0000 7f8b6bfff640 1 --2- 172.21.15.43:0/3164991289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b6c1090c0 0x7f8b6c19c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.490+0000 7f8b6b7fe640 1 --2- 172.21.15.43:0/3164991289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b6c1038a0 0x7f8b6c19b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.490+0000 7f8b6affd640 1 -- 172.21.15.43:0/3164991289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b6c1038a0 msgr2=0x7f8b6c19b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.490+0000 7f8b6affd640 1 --2- 172.21.15.43:0/3164991289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b6c1038a0 0x7f8b6c19b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.490+0000 7f8b6affd640 1 -- 172.21.15.43:0/3164991289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b6c1090c0 msgr2=0x7f8b6c19c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.490+0000 7f8b6affd640 1 --2- 172.21.15.43:0/3164991289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b6c1090c0 0x7f8b6c19c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.490+0000 7f8b6affd640 1 -- 172.21.15.43:0/3164991289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6c1a6a20 con 0x7f8b6c108c20 2024-04-25T10:36:25.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.490+0000 7f8b6affd640 1 --2- 172.21.15.43:0/3164991289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b6c108c20 0x7f8b6c19bce0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8b6002a930 tx=0x7f8b60004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:25.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.490+0000 7f8b68ff9640 1 -- 172.21.15.43:0/3164991289 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6003f030 con 0x7f8b6c108c20 2024-04-25T10:36:25.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.490+0000 7f8b68ff9640 1 -- 172.21.15.43:0/3164991289 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b60003aa0 con 0x7f8b6c108c20 2024-04-25T10:36:25.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.490+0000 7f8b7208c640 1 -- 172.21.15.43:0/3164991289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6c1a6cb0 con 0x7f8b6c108c20 2024-04-25T10:36:25.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.490+0000 7f8b68ff9640 1 -- 172.21.15.43:0/3164991289 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b60035070 con 0x7f8b6c108c20 2024-04-25T10:36:25.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.490+0000 7f8b7208c640 1 -- 172.21.15.43:0/3164991289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b6c1a71a0 con 0x7f8b6c108c20 2024-04-25T10:36:25.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.493+0000 7f8b79010740 1 Processor -- start 2024-04-25T10:36:25.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.493+0000 7f8b79010740 1 -- start start 2024-04-25T10:36:25.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.493+0000 7f8b68ff9640 1 -- 172.21.15.43:0/3164991289 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f8b60042030 con 0x7f8b6c108c20 2024-04-25T10:36:25.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.493+0000 7f8b79010740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3e2a0390 0x564b3e641810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.493+0000 7f8b79010740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b3e2774d0 0x564b3e641d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.493+0000 7f8b79010740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e27f6b0 0x564b3e64e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.493+0000 7f8b79010740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564b3e5a6d80 con 0x564b3e27f6b0 2024-04-25T10:36:25.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.493+0000 7f8b79010740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564b3e560a20 con 0x564b3e2a0390 2024-04-25T10:36:25.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.493+0000 7f8b79010740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564b3e5a2d40 con 0x564b3e2774d0 2024-04-25T10:36:25.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.493+0000 7f8b6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e27f6b0 0x564b3e64e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.493+0000 7f8b6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e27f6b0 0x564b3e64e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38720/0 (socket says 172.21.15.43:38720) 2024-04-25T10:36:25.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.493+0000 7f8b6affd640 1 -- 172.21.15.43:0/2244619948 learned_addr learned my addr 172.21.15.43:0/2244619948 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:25.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.493+0000 7f8b6b7fe640 1 --2- 172.21.15.43:0/2244619948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b3e2774d0 0x564b3e641d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.494+0000 7f8b6bfff640 1 --2- 172.21.15.43:0/2244619948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3e2a0390 0x564b3e641810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.494+0000 7f8b6affd640 1 -- 172.21.15.43:0/2244619948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b3e2774d0 msgr2=0x564b3e641d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.494+0000 7f8b68ff9640 1 --2- 172.21.15.43:0/3164991289 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8b540afc60 0x7f8b540b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.494+0000 7f8b6affd640 1 --2- 172.21.15.43:0/2244619948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b3e2774d0 0x564b3e641d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.494+0000 7f8b6affd640 1 -- 172.21.15.43:0/2244619948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3e2a0390 msgr2=0x564b3e641810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.494+0000 7f8b6affd640 1 --2- 172.21.15.43:0/2244619948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3e2a0390 0x564b3e641810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.494+0000 7f8b6affd640 1 -- 172.21.15.43:0/2244619948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b3e56bc30 con 0x564b3e27f6b0 2024-04-25T10:36:25.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.494+0000 7f8b6bfff640 1 --2- 172.21.15.43:0/2244619948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3e2a0390 0x564b3e641810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:25.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.494+0000 7f8b6bfff640 1 --2- 172.21.15.43:0/3164991289 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8b540afc60 0x7f8b540b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.494+0000 7f8b68ff9640 1 -- 172.21.15.43:0/3164991289 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f8b600ff710 con 0x7f8b6c108c20 2024-04-25T10:36:25.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.494+0000 7f8b6affd640 1 --2- 172.21.15.43:0/2244619948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e27f6b0 0x564b3e64e6a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8b600fce00 tx=0x7f8b600c37a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfec90790852c1ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:25.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.494+0000 7f8b51ffb640 1 -- 172.21.15.43:0/2244619948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b60049020 con 0x564b3e27f6b0 2024-04-25T10:36:25.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.494+0000 7f8b51ffb640 1 -- 172.21.15.43:0/2244619948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b600d0080 con 0x564b3e27f6b0 2024-04-25T10:36:25.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.494+0000 7f8b51ffb640 1 -- 172.21.15.43:0/2244619948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b600c7230 con 0x564b3e27f6b0 2024-04-25T10:36:25.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.495+0000 7f8b79010740 1 -- 172.21.15.43:0/2244619948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e27f6b0 msgr2=0x564b3e64e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.495+0000 7f8b79010740 1 --2- 172.21.15.43:0/2244619948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e27f6b0 0x564b3e64e6a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8b600fce00 tx=0x7f8b600c37a0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.495+0000 7f8b6bfff640 1 --2- 172.21.15.43:0/3164991289 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8b540afc60 0x7f8b540b2170 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8b640098b0 tx=0x7f8b64004a40 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:25.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.495+0000 7f8b79010740 1 -- 172.21.15.43:0/2244619948 shutdown_connections 2024-04-25T10:36:25.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.495+0000 7f8b79010740 1 --2- 172.21.15.43:0/2244619948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e27f6b0 0x564b3e64e6a0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.495+0000 7f8b79010740 1 --2- 172.21.15.43:0/2244619948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b3e2774d0 0x564b3e641d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.495+0000 7f8b79010740 1 --2- 172.21.15.43:0/2244619948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3e2a0390 0x564b3e641810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.495+0000 7f8b79010740 1 -- 172.21.15.43:0/2244619948 >> 172.21.15.43:0/2244619948 conn(0x564b3e2944e0 msgr2=0x564b3e63dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:25.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.495+0000 7f8b79010740 1 -- 172.21.15.43:0/2244619948 shutdown_connections 2024-04-25T10:36:25.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.495+0000 7f8b79010740 1 -- 172.21.15.43:0/2244619948 wait complete. 2024-04-25T10:36:25.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.495+0000 7f8b79010740 1 -- start start 2024-04-25T10:36:25.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.496+0000 7f8b79010740 1 Processor -- start 2024-04-25T10:36:25.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.496+0000 7f8b79010740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b3e2774d0 0x564b3e64c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.496+0000 7f8b79010740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3e27f6b0 0x564b3e64c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.496+0000 7f8b79010740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e2a0390 0x564b3e657fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.496+0000 7f8b79010740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564b3e5a6d80 con 0x564b3e2a0390 2024-04-25T10:36:25.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.496+0000 7f8b79010740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564b3e52ddc0 con 0x564b3e27f6b0 2024-04-25T10:36:25.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.496+0000 7f8b79010740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564b3e64ebe0 con 0x564b3e2774d0 2024-04-25T10:36:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.496+0000 7f8b79010740 10 client.0 _ll_register_callbacks cb 0x564b3e549600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.496+0000 7f8b6bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e2a0390 0x564b3e657fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.496+0000 7f8b6bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e2a0390 0x564b3e657fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38736/0 (socket says 172.21.15.43:38736) 2024-04-25T10:36:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.496+0000 7f8b6bfff640 1 -- 172.21.15.43:0/2162164424 learned_addr learned my addr 172.21.15.43:0/2162164424 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.496+0000 7f8b6b7fe640 1 --2- 172.21.15.43:0/2162164424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b3e2774d0 0x564b3e64c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.496+0000 7f8b6affd640 1 --2- 172.21.15.43:0/2162164424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3e27f6b0 0x564b3e64c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.496+0000 7f8b6bfff640 1 -- 172.21.15.43:0/2162164424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b3e2774d0 msgr2=0x564b3e64c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.496+0000 7f8b6bfff640 1 --2- 172.21.15.43:0/2162164424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b3e2774d0 0x564b3e64c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.496+0000 7f8b6bfff640 1 -- 172.21.15.43:0/2162164424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3e27f6b0 msgr2=0x564b3e64c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.496+0000 7f8b6bfff640 1 --2- 172.21.15.43:0/2162164424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3e27f6b0 0x564b3e64c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.496+0000 7f8b6bfff640 1 -- 172.21.15.43:0/2162164424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b3e2b83f0 con 0x564b3e2a0390 2024-04-25T10:36:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.497+0000 7f8b6affd640 1 --2- 172.21.15.43:0/2162164424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3e27f6b0 0x564b3e64c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:36:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.497+0000 7f8b6bfff640 1 --2- 172.21.15.43:0/2162164424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e2a0390 0x564b3e657fa0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8b64013860 tx=0x7f8b64004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748f3fe97f1eb5cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.497+0000 7f8b4affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.497+0000 7f8b4affd640 1 -- 172.21.15.43:0/2162164424 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6401ed70 con 0x564b3e2a0390 2024-04-25T10:36:25.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.497+0000 7f8b4affd640 1 -- 172.21.15.43:0/2162164424 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f8b64003970 con 0x564b3e2a0390 2024-04-25T10:36:25.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.497+0000 7f8b79010740 1 -- 172.21.15.43:0/2162164424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564b3e5b4990 con 0x564b3e2a0390 2024-04-25T10:36:25.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.497+0000 7f8b4affd640 1 -- 172.21.15.43:0/2162164424 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8b64003b70 con 0x564b3e2a0390 2024-04-25T10:36:25.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.497+0000 7f8b79010740 10 client.16575 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:25.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.497+0000 7f8b79010740 1 -- 172.21.15.43:0/2162164424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564b3e3827d0 con 0x564b3e2a0390 2024-04-25T10:36:25.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.497+0000 7f8b4affd640 1 -- 172.21.15.43:0/2162164424 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564b3e5b4990 con 0x564b3e2a0390 2024-04-25T10:36:25.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.497+0000 7f8b4affd640 1 -- 172.21.15.43:0/2162164424 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8b6400f040 con 0x564b3e2a0390 2024-04-25T10:36:25.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.497+0000 7f8b79010740 10 client.16575 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:25.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.497+0000 7f8b79010740 10 client.16575 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:36:25.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.497+0000 7f8b79010740 20 client.16575 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:25.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.498+0000 7f8b79010740 1 --2- 172.21.15.43:0/2162164424 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x564b3e25ef60 0x564b3e65e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.498+0000 7f8b79010740 4 client.16575 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:36:25.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.498+0000 7f8b79010740 1 -- 172.21.15.43:0/2162164424 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b3e5ae800 con 0x564b3e25ef60 2024-04-25T10:36:25.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.498+0000 7f8b6b7fe640 1 --2- 172.21.15.43:0/2162164424 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x564b3e25ef60 0x564b3e65e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.498+0000 7f8b4affd640 10 client.16575 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:36:25.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.498+0000 7f8b6b7fe640 1 --2- 172.21.15.43:0/2162164424 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x564b3e25ef60 0x564b3e65e420 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9db854d2eae4ecd server_cookie=21ec3f9e32d8e3c4 in_seq=0 out_seq=0 2024-04-25T10:36:25.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.499+0000 7f8b4affd640 1 -- 172.21.15.43:0/2162164424 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564b3e5ae800 con 0x564b3e25ef60 2024-04-25T10:36:25.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.499+0000 7f8b4affd640 10 client.16575 handle_command_reply: tid=0 2024-04-25T10:36:25.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.499+0000 7f8b79010740 1 client.16575 shutdown 2024-04-25T10:36:25.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.499+0000 7f8b79010740 1 -- 172.21.15.43:0/2162164424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e2a0390 msgr2=0x564b3e657fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.499+0000 7f8b79010740 1 --2- 172.21.15.43:0/2162164424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e2a0390 0x564b3e657fa0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8b64013860 tx=0x7f8b64004ab0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.499+0000 7f8b79010740 1 -- 172.21.15.43:0/2162164424 shutdown_connections 2024-04-25T10:36:25.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.499+0000 7f8b79010740 1 --2- 172.21.15.43:0/2162164424 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x564b3e25ef60 0x564b3e65e420 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.499+0000 7f8b79010740 1 --2- 172.21.15.43:0/2162164424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3e27f6b0 0x564b3e64c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.499+0000 7f8b79010740 1 --2- 172.21.15.43:0/2162164424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e2a0390 0x564b3e657fa0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.499+0000 7f8b4affd640 0 client.16575 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:36:25.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.499+0000 7f8b79010740 1 --2- 172.21.15.43:0/2162164424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b3e2774d0 0x564b3e64c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.499+0000 7f8b79010740 1 -- 172.21.15.43:0/2162164424 >> 172.21.15.43:0/2162164424 conn(0x564b3e2944e0 msgr2=0x564b3e643530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:25.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.499+0000 7f8b79010740 1 -- 172.21.15.43:0/2162164424 shutdown_connections 2024-04-25T10:36:25.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.499+0000 7f8b79010740 1 -- 172.21.15.43:0/2162164424 wait complete. 2024-04-25T10:36:25.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.500+0000 7f8b79010740 20 client.16575 trim_cache size 0 max 16384 2024-04-25T10:36:25.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.513+0000 7f8b79010740 1 Processor -- start 2024-04-25T10:36:25.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.513+0000 7f8b79010740 1 -- start start 2024-04-25T10:36:25.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.513+0000 7f8b79010740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3e25ef60 0x564b3e657b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.514+0000 7f8b79010740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b3e2774d0 0x564b3e663c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.514+0000 7f8b79010740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e27f6b0 0x564b3e664160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.514+0000 7f8b79010740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c1b2240 con 0x564b3e27f6b0 2024-04-25T10:36:25.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.514+0000 7f8b79010740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c1afb10 con 0x564b3e25ef60 2024-04-25T10:36:25.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.514+0000 7f8b79010740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c1ad960 con 0x564b3e2774d0 2024-04-25T10:36:25.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.514+0000 7f8b6bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e27f6b0 0x564b3e664160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.514+0000 7f8b6bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e27f6b0 0x564b3e664160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38746/0 (socket says 172.21.15.43:38746) 2024-04-25T10:36:25.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.514+0000 7f8b6bfff640 1 -- 172.21.15.43:0/4165730968 learned_addr learned my addr 172.21.15.43:0/4165730968 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:25.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.514+0000 7f8b6affd640 1 --2- 172.21.15.43:0/4165730968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b3e2774d0 0x564b3e663c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.514+0000 7f8b6bfff640 1 -- 172.21.15.43:0/4165730968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b3e2774d0 msgr2=0x564b3e663c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.514+0000 7f8b6bfff640 1 --2- 172.21.15.43:0/4165730968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b3e2774d0 0x564b3e663c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.514+0000 7f8b6bfff640 1 -- 172.21.15.43:0/4165730968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3e25ef60 msgr2=0x564b3e657b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.514+0000 7f8b6bfff640 1 --2- 172.21.15.43:0/4165730968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3e25ef60 0x564b3e657b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.514+0000 7f8b6bfff640 1 -- 172.21.15.43:0/4165730968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b3e642d90 con 0x564b3e27f6b0 2024-04-25T10:36:25.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.514+0000 7f8b6bfff640 1 --2- 172.21.15.43:0/4165730968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e27f6b0 0x564b3e664160 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8b6401f5b0 tx=0x7f8b64009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35667c4b613b4a66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:25.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.514+0000 7f8b517fa640 1 -- 172.21.15.43:0/4165730968 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6401fa20 con 0x564b3e27f6b0 2024-04-25T10:36:25.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.514+0000 7f8b517fa640 1 -- 172.21.15.43:0/4165730968 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b64023070 con 0x564b3e27f6b0 2024-04-25T10:36:25.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.514+0000 7f8b517fa640 1 -- 172.21.15.43:0/4165730968 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6400fd40 con 0x564b3e27f6b0 2024-04-25T10:36:25.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.515+0000 7f8b79010740 1 -- 172.21.15.43:0/4165730968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e27f6b0 msgr2=0x564b3e664160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.515+0000 7f8b79010740 1 --2- 172.21.15.43:0/4165730968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e27f6b0 0x564b3e664160 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8b6401f5b0 tx=0x7f8b64009600 comp rx=0 tx=0).stop 2024-04-25T10:36:25.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.515+0000 7f8b79010740 1 -- 172.21.15.43:0/4165730968 shutdown_connections 2024-04-25T10:36:25.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.515+0000 7f8b79010740 1 --2- 172.21.15.43:0/4165730968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e27f6b0 0x564b3e664160 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.515+0000 7f8b79010740 1 --2- 172.21.15.43:0/4165730968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b3e2774d0 0x564b3e663c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.515+0000 7f8b79010740 1 --2- 172.21.15.43:0/4165730968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3e25ef60 0x564b3e657b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.515+0000 7f8b79010740 1 -- 172.21.15.43:0/4165730968 >> 172.21.15.43:0/4165730968 conn(0x564b3e2944e0 msgr2=0x564b3e64d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:25.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.515+0000 7f8b79010740 1 -- 172.21.15.43:0/4165730968 shutdown_connections 2024-04-25T10:36:25.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.515+0000 7f8b79010740 1 -- 172.21.15.43:0/4165730968 wait complete. 2024-04-25T10:36:25.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.515+0000 7f8b79010740 1 -- start start 2024-04-25T10:36:25.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.516+0000 7f8b79010740 1 Processor -- start 2024-04-25T10:36:25.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.516+0000 7f8b79010740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b3e25ef60 0x564b3e6513b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.516+0000 7f8b79010740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e2774d0 0x564b3e6518f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.516+0000 7f8b79010740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3e27f6b0 0x564b3e6be810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.516+0000 7f8b79010740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c1adfd0 con 0x564b3e2774d0 2024-04-25T10:36:25.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.516+0000 7f8b79010740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c1af440 con 0x564b3e27f6b0 2024-04-25T10:36:25.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.516+0000 7f8b79010740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c1ae6a0 con 0x564b3e25ef60 2024-04-25T10:36:25.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.516+0000 7f8b79010740 10 client.0 _ll_register_callbacks cb 0x7f8b64009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:25.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.516+0000 7f8b6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e2774d0 0x564b3e6518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.516+0000 7f8b6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e2774d0 0x564b3e6518f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38760/0 (socket says 172.21.15.43:38760) 2024-04-25T10:36:25.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.516+0000 7f8b6affd640 1 -- 172.21.15.43:0/351720244 learned_addr learned my addr 172.21.15.43:0/351720244 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:25.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.516+0000 7f8b6b7fe640 1 --2- 172.21.15.43:0/351720244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b3e25ef60 0x564b3e6513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.516+0000 7f8b6affd640 1 -- 172.21.15.43:0/351720244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b3e25ef60 msgr2=0x564b3e6513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.516+0000 7f8b6affd640 1 --2- 172.21.15.43:0/351720244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b3e25ef60 0x564b3e6513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.516+0000 7f8b6affd640 1 -- 172.21.15.43:0/351720244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3e27f6b0 msgr2=0x564b3e6be810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.516+0000 7f8b6affd640 1 --2- 172.21.15.43:0/351720244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3e27f6b0 0x564b3e6be810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.516+0000 7f8b6affd640 1 -- 172.21.15.43:0/351720244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b3e6bed50 con 0x564b3e2774d0 2024-04-25T10:36:25.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.517+0000 7f8b6affd640 1 --2- 172.21.15.43:0/351720244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e2774d0 0x564b3e6518f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8b600c75e0 tx=0x7f8b6002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e3a5ca3ba702c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:25.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.517+0000 7f8b4affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:25.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.517+0000 7f8b4affd640 1 -- 172.21.15.43:0/351720244 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b60049070 con 0x564b3e2774d0 2024-04-25T10:36:25.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.517+0000 7f8b79010740 1 -- 172.21.15.43:0/351720244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b64023070 con 0x564b3e2774d0 2024-04-25T10:36:25.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.517+0000 7f8b4affd640 1 -- 172.21.15.43:0/351720244 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f8b60003c50 con 0x564b3e2774d0 2024-04-25T10:36:25.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.517+0000 7f8b4affd640 1 -- 172.21.15.43:0/351720244 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8b600c7050 con 0x564b3e2774d0 2024-04-25T10:36:25.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.517+0000 7f8b79010740 10 client.16581 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:25.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.517+0000 7f8b79010740 1 -- 172.21.15.43:0/351720244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564b3e6c60b0 con 0x564b3e2774d0 2024-04-25T10:36:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.517+0000 7f8b4affd640 1 -- 172.21.15.43:0/351720244 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b64023070 con 0x564b3e2774d0 2024-04-25T10:36:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.517+0000 7f8b4affd640 1 -- 172.21.15.43:0/351720244 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8b600fd110 con 0x564b3e2774d0 2024-04-25T10:36:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.517+0000 7f8b79010740 10 client.16581 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.517+0000 7f8b79010740 10 client.16581 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:36:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.517+0000 7f8b79010740 20 client.16581 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.517+0000 7f8b79010740 1 --2- 172.21.15.43:0/351720244 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x564b3e2a0390 0x564b3e6c8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.517+0000 7f8b79010740 4 client.16581 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:36:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.517+0000 7f8b79010740 1 -- 172.21.15.43:0/351720244 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564b3e529610 con 0x564b3e2a0390 2024-04-25T10:36:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.518+0000 7f8b6b7fe640 1 --2- 172.21.15.43:0/351720244 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x564b3e2a0390 0x564b3e6c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.518+0000 7f8b4affd640 10 client.16581 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:36:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.518+0000 7f8b6b7fe640 1 --2- 172.21.15.43:0/351720244 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x564b3e2a0390 0x564b3e6c8690 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fecb66e6bc77cb44 server_cookie=704b9f0d28415c30 in_seq=0 out_seq=0 2024-04-25T10:36:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.518+0000 7f8b4affd640 1 -- 172.21.15.43:0/351720244 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564b3e529610 con 0x564b3e2a0390 2024-04-25T10:36:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.518+0000 7f8b4affd640 10 client.16581 handle_command_reply: tid=0 2024-04-25T10:36:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.518+0000 7f8b79010740 1 client.16581 shutdown 2024-04-25T10:36:25.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b79010740 1 -- 172.21.15.43:0/351720244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e2774d0 msgr2=0x564b3e6518f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b79010740 1 --2- 172.21.15.43:0/351720244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e2774d0 0x564b3e6518f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8b600c75e0 tx=0x7f8b6002a410 comp rx=0 tx=0).stop 2024-04-25T10:36:25.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b79010740 1 -- 172.21.15.43:0/351720244 shutdown_connections 2024-04-25T10:36:25.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b79010740 1 --2- 172.21.15.43:0/351720244 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x564b3e2a0390 0x564b3e6c8690 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b79010740 1 --2- 172.21.15.43:0/351720244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3e27f6b0 0x564b3e6be810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b4affd640 0 client.16581 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:36:25.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b79010740 1 --2- 172.21.15.43:0/351720244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3e2774d0 0x564b3e6518f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b79010740 1 --2- 172.21.15.43:0/351720244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b3e25ef60 0x564b3e6513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b79010740 1 -- 172.21.15.43:0/351720244 >> 172.21.15.43:0/351720244 conn(0x564b3e2944e0 msgr2=0x564b3e64e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:25.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b79010740 1 -- 172.21.15.43:0/351720244 shutdown_connections 2024-04-25T10:36:25.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b79010740 1 -- 172.21.15.43:0/351720244 wait complete. 2024-04-25T10:36:25.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b79010740 20 client.16581 trim_cache size 0 max 16384 2024-04-25T10:36:25.527 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:36:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b7208c640 1 -- 172.21.15.43:0/3164991289 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8b540afc60 msgr2=0x7f8b540b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b7208c640 1 --2- 172.21.15.43:0/3164991289 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8b540afc60 0x7f8b540b2170 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8b640098b0 tx=0x7f8b64004a40 comp rx=0 tx=0).stop 2024-04-25T10:36:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b7208c640 1 -- 172.21.15.43:0/3164991289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b6c108c20 msgr2=0x7f8b6c19bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b7208c640 1 --2- 172.21.15.43:0/3164991289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b6c108c20 0x7f8b6c19bce0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8b6002a930 tx=0x7f8b60004270 comp rx=0 tx=0).stop 2024-04-25T10:36:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b7208c640 1 -- 172.21.15.43:0/3164991289 shutdown_connections 2024-04-25T10:36:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b7208c640 1 --2- 172.21.15.43:0/3164991289 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8b540afc60 0x7f8b540b2170 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b7208c640 1 --2- 172.21.15.43:0/3164991289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b6c1090c0 0x7f8b6c19c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b7208c640 1 --2- 172.21.15.43:0/3164991289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b6c108c20 0x7f8b6c19bce0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b7208c640 1 --2- 172.21.15.43:0/3164991289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b6c1038a0 0x7f8b6c19b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.519+0000 7f8b7208c640 1 -- 172.21.15.43:0/3164991289 >> 172.21.15.43:0/3164991289 conn(0x7f8b6c0fd850 msgr2=0x7f8b6c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.520+0000 7f8b7208c640 1 -- 172.21.15.43:0/3164991289 shutdown_connections 2024-04-25T10:36:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.520+0000 7f8b7208c640 1 -- 172.21.15.43:0/3164991289 wait complete. 2024-04-25T10:36:25.533 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:36:25.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.644+0000 7fca5f972640 1 -- 172.21.15.43:0/113062582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca58104260 msgr2=0x7fca58110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.644+0000 7fca5f972640 1 --2- 172.21.15.43:0/113062582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca58104260 0x7fca58110630 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fca50009ca0 tx=0x7fca5002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.644+0000 7fca5f972640 1 -- 172.21.15.43:0/113062582 shutdown_connections 2024-04-25T10:36:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.644+0000 7fca5f972640 1 --2- 172.21.15.43:0/113062582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca58109fb0 0x7fca58103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.644+0000 7fca5f972640 1 --2- 172.21.15.43:0/113062582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca58104260 0x7fca58110630 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.644+0000 7fca5f972640 1 --2- 172.21.15.43:0/113062582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca58109b10 0x7fca581036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.644+0000 7fca5f972640 1 -- 172.21.15.43:0/113062582 >> 172.21.15.43:0/113062582 conn(0x7fca580ff210 msgr2=0x7fca58101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.644+0000 7fca5f972640 1 -- 172.21.15.43:0/113062582 shutdown_connections 2024-04-25T10:36:25.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.644+0000 7fca5f972640 1 -- 172.21.15.43:0/113062582 wait complete. 2024-04-25T10:36:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.645+0000 7fca5f972640 1 Processor -- start 2024-04-25T10:36:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.645+0000 7fca5f972640 1 -- start start 2024-04-25T10:36:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.646+0000 7fca5f972640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca58104260 0x7fca5819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.646+0000 7fca5f972640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca58109b10 0x7fca5819aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.646+0000 7fca5f972640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca58109fb0 0x7fca5819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.646+0000 7fca5f972640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fca58113d20 con 0x7fca58104260 2024-04-25T10:36:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.646+0000 7fca5f972640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca58113ba0 con 0x7fca58109b10 2024-04-25T10:36:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.646+0000 7fca5f972640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fca58113ea0 con 0x7fca58109fb0 2024-04-25T10:36:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.646+0000 7fca5d70e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca58104260 0x7fca5819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.646+0000 7fca5d70e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca58104260 0x7fca5819a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38790/0 (socket says 172.21.15.43:38790) 2024-04-25T10:36:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.646+0000 7fca5d70e640 1 -- 172.21.15.43:0/2446315619 learned_addr learned my addr 172.21.15.43:0/2446315619 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.646+0000 7fca5cf0d640 1 --2- 172.21.15.43:0/2446315619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca58109b10 0x7fca5819aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.646+0000 7fca5df0f640 1 --2- 172.21.15.43:0/2446315619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca58109fb0 0x7fca5819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.646+0000 7fca5d70e640 1 -- 172.21.15.43:0/2446315619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca58109fb0 msgr2=0x7fca5819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.646+0000 7fca5d70e640 1 --2- 172.21.15.43:0/2446315619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca58109fb0 0x7fca5819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.646+0000 7fca5d70e640 1 -- 172.21.15.43:0/2446315619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca58109b10 msgr2=0x7fca5819aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.646+0000 7fca5d70e640 1 --2- 172.21.15.43:0/2446315619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca58109b10 0x7fca5819aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.646+0000 7fca5d70e640 1 -- 172.21.15.43:0/2446315619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5819f550 con 0x7fca58104260 2024-04-25T10:36:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.647+0000 7fca5d70e640 1 --2- 172.21.15.43:0/2446315619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca58104260 0x7fca5819a740 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fca5400e9f0 tx=0x7fca5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.647+0000 7fca4e7fc640 1 -- 172.21.15.43:0/2446315619 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5400ccc0 con 0x7fca58104260 2024-04-25T10:36:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.647+0000 7fca4e7fc640 1 -- 172.21.15.43:0/2446315619 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca540054d0 con 0x7fca58104260 2024-04-25T10:36:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.647+0000 7fca5f972640 1 -- 172.21.15.43:0/2446315619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5819f840 con 0x7fca58104260 2024-04-25T10:36:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.647+0000 7fca4e7fc640 1 -- 172.21.15.43:0/2446315619 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca54010430 con 0x7fca58104260 2024-04-25T10:36:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.647+0000 7fca5f972640 1 -- 172.21.15.43:0/2446315619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca58075ee0 con 0x7fca58104260 2024-04-25T10:36:25.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.650+0000 7fca66825740 1 Processor -- start 2024-04-25T10:36:25.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.650+0000 7fca66825740 1 -- start start 2024-04-25T10:36:25.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.650+0000 7fca66825740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b6c7390 0x55aa0ba68810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.650+0000 7fca4e7fc640 1 -- 172.21.15.43:0/2446315619 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fca540046c0 con 0x7fca58104260 2024-04-25T10:36:25.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.650+0000 7fca66825740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b69e4d0 0x55aa0ba68d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.650+0000 7fca66825740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b6a66b0 0x55aa0ba756a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.650+0000 7fca66825740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aa0b9cdd80 con 0x55aa0b6c7390 2024-04-25T10:36:25.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.650+0000 7fca66825740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55aa0b987a20 con 0x55aa0b6a66b0 2024-04-25T10:36:25.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.650+0000 7fca66825740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55aa0b9c9d40 con 0x55aa0b69e4d0 2024-04-25T10:36:25.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.651+0000 7fca5df0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b6c7390 0x55aa0ba68810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.651+0000 7fca5cf0d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b6a66b0 0x55aa0ba756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.651+0000 7fca5d70e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b69e4d0 0x55aa0ba68d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.651+0000 7fca5df0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b6c7390 0x55aa0ba68810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38798/0 (socket says 172.21.15.43:38798) 2024-04-25T10:36:25.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.651+0000 7fca5df0f640 1 -- 172.21.15.43:0/3999942289 learned_addr learned my addr 172.21.15.43:0/3999942289 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:25.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.651+0000 7fca5cf0d640 1 -- 172.21.15.43:0/3999942289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b69e4d0 msgr2=0x55aa0ba68d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.651+0000 7fca5cf0d640 1 --2- 172.21.15.43:0/3999942289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b69e4d0 0x55aa0ba68d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.651+0000 7fca5cf0d640 1 -- 172.21.15.43:0/3999942289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b6c7390 msgr2=0x55aa0ba68810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.651+0000 7fca5cf0d640 1 --2- 172.21.15.43:0/3999942289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b6c7390 0x55aa0ba68810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.651+0000 7fca5cf0d640 1 -- 172.21.15.43:0/3999942289 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa0b992c30 con 0x55aa0b6a66b0 2024-04-25T10:36:25.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.651+0000 7fca5cf0d640 1 --2- 172.21.15.43:0/3999942289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b6a66b0 0x55aa0ba756a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fca5810a3e0 tx=0x7fca48011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=febb2a97e8e9ef5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.651+0000 7fca4e7fc640 1 --2- 172.21.15.43:0/2446315619 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fca440afce0 0x7fca440b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.652+0000 7fca4e7fc640 1 -- 172.21.15.43:0/2446315619 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fca540df310 con 0x7fca58104260 2024-04-25T10:36:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.652+0000 7fca437fe640 1 -- 172.21.15.43:0/3999942289 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48009d70 con 0x55aa0b6a66b0 2024-04-25T10:36:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.652+0000 7fca5df0f640 1 --2- 172.21.15.43:0/2446315619 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fca440afce0 0x7fca440b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.652+0000 7fca437fe640 1 -- 172.21.15.43:0/3999942289 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca48005020 con 0x55aa0b6a66b0 2024-04-25T10:36:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.652+0000 7fca437fe640 1 -- 172.21.15.43:0/3999942289 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48004a00 con 0x55aa0b6a66b0 2024-04-25T10:36:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.653+0000 7fca66825740 1 -- 172.21.15.43:0/3999942289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b6a66b0 msgr2=0x55aa0ba756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.653+0000 7fca66825740 1 --2- 172.21.15.43:0/3999942289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b6a66b0 0x55aa0ba756a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fca5810a3e0 tx=0x7fca48011810 comp rx=0 tx=0).stop 2024-04-25T10:36:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.653+0000 7fca5df0f640 1 --2- 172.21.15.43:0/2446315619 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fca440afce0 0x7fca440b21f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fca5002f500 tx=0x7fca5002f310 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.653+0000 7fca66825740 1 -- 172.21.15.43:0/3999942289 shutdown_connections 2024-04-25T10:36:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.653+0000 7fca66825740 1 --2- 172.21.15.43:0/3999942289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b6a66b0 0x55aa0ba756a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.653+0000 7fca66825740 1 --2- 172.21.15.43:0/3999942289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b69e4d0 0x55aa0ba68d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.653+0000 7fca66825740 1 --2- 172.21.15.43:0/3999942289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b6c7390 0x55aa0ba68810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.653+0000 7fca66825740 1 -- 172.21.15.43:0/3999942289 >> 172.21.15.43:0/3999942289 conn(0x55aa0b6bb4e0 msgr2=0x55aa0ba64fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.653+0000 7fca66825740 1 -- 172.21.15.43:0/3999942289 shutdown_connections 2024-04-25T10:36:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.653+0000 7fca66825740 1 -- 172.21.15.43:0/3999942289 wait complete. 2024-04-25T10:36:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.654+0000 7fca66825740 1 -- start start 2024-04-25T10:36:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.654+0000 7fca66825740 1 Processor -- start 2024-04-25T10:36:25.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.654+0000 7fca66825740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b69e4d0 0x55aa0ba732d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.654+0000 7fca66825740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b6a66b0 0x55aa0ba73810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.654+0000 7fca66825740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b6c7390 0x55aa0ba7f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.654+0000 7fca66825740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aa0ba75d90 con 0x55aa0b69e4d0 2024-04-25T10:36:25.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.654+0000 7fca66825740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55aa0b954dc0 con 0x55aa0b6c7390 2024-04-25T10:36:25.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.654+0000 7fca66825740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55aa0ba75f10 con 0x55aa0b6a66b0 2024-04-25T10:36:25.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.654+0000 7fca66825740 10 client.0 _ll_register_callbacks cb 0x55aa0b970600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:25.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.654+0000 7fca5d70e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b69e4d0 0x55aa0ba732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.654+0000 7fca5d70e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b69e4d0 0x55aa0ba732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38800/0 (socket says 172.21.15.43:38800) 2024-04-25T10:36:25.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.654+0000 7fca5d70e640 1 -- 172.21.15.43:0/2701961731 learned_addr learned my addr 172.21.15.43:0/2701961731 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:25.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.654+0000 7fca5cf0d640 1 --2- 172.21.15.43:0/2701961731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b6a66b0 0x55aa0ba73810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.654+0000 7fca5df0f640 1 --2- 172.21.15.43:0/2701961731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b6c7390 0x55aa0ba7f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.655+0000 7fca5d70e640 1 -- 172.21.15.43:0/2701961731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b6a66b0 msgr2=0x55aa0ba73810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.655+0000 7fca5d70e640 1 --2- 172.21.15.43:0/2701961731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b6a66b0 0x55aa0ba73810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.655+0000 7fca5d70e640 1 -- 172.21.15.43:0/2701961731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b6c7390 msgr2=0x55aa0ba7f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.655+0000 7fca5d70e640 1 --2- 172.21.15.43:0/2701961731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b6c7390 0x55aa0ba7f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.655+0000 7fca5d70e640 1 -- 172.21.15.43:0/2701961731 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa0b6df3f0 con 0x55aa0b69e4d0 2024-04-25T10:36:25.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.655+0000 7fca5d70e640 1 --2- 172.21.15.43:0/2701961731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b69e4d0 0x55aa0ba732d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fca5400c6a0 tx=0x7fca540e2a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b5517cbb1f4aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:25.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.655+0000 7fca40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:25.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.655+0000 7fca40ff9640 1 -- 172.21.15.43:0/2701961731 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca540a72b0 con 0x55aa0b69e4d0 2024-04-25T10:36:25.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.655+0000 7fca66825740 1 -- 172.21.15.43:0/2701961731 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca48005020 con 0x55aa0b69e4d0 2024-04-25T10:36:25.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.655+0000 7fca40ff9640 1 -- 172.21.15.43:0/2701961731 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fca540b00c0 con 0x55aa0b69e4d0 2024-04-25T10:36:25.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.655+0000 7fca40ff9640 1 -- 172.21.15.43:0/2701961731 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca540a6c50 con 0x55aa0b69e4d0 2024-04-25T10:36:25.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.655+0000 7fca66825740 10 client.16593 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:25.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.655+0000 7fca66825740 1 -- 172.21.15.43:0/2701961731 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aa0b7a97d0 con 0x55aa0b69e4d0 2024-04-25T10:36:25.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.655+0000 7fca40ff9640 1 -- 172.21.15.43:0/2701961731 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca48005020 con 0x55aa0b69e4d0 2024-04-25T10:36:25.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.656+0000 7fca40ff9640 1 -- 172.21.15.43:0/2701961731 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fca54005000 con 0x55aa0b69e4d0 2024-04-25T10:36:25.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.656+0000 7fca66825740 10 client.16593 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:25.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.656+0000 7fca66825740 10 client.16593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:25.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.656+0000 7fca66825740 20 client.16593 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:25.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.656+0000 7fca66825740 1 --2- 172.21.15.43:0/2701961731 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aa0b685f60 0x55aa0ba856c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.656+0000 7fca66825740 4 client.16593 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:36:25.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.656+0000 7fca66825740 1 -- 172.21.15.43:0/2701961731 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa0b9db990 con 0x55aa0b685f60 2024-04-25T10:36:25.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.656+0000 7fca5cf0d640 1 --2- 172.21.15.43:0/2701961731 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aa0b685f60 0x55aa0ba856c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.656+0000 7fca5cf0d640 1 --2- 172.21.15.43:0/2701961731 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aa0b685f60 0x55aa0ba856c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5faff82cb5150d9 server_cookie=fd3b1c57729acdae in_seq=0 out_seq=0 2024-04-25T10:36:25.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.656+0000 7fca40ff9640 10 client.16593 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:25.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.657+0000 7fca40ff9640 1 -- 172.21.15.43:0/2701961731 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aa0b9db990 con 0x55aa0b685f60 2024-04-25T10:36:25.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.657+0000 7fca40ff9640 10 client.16593 handle_command_reply: tid=0 2024-04-25T10:36:25.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.657+0000 7fca66825740 1 client.16593 shutdown 2024-04-25T10:36:25.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.658+0000 7fca66825740 1 -- 172.21.15.43:0/2701961731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b69e4d0 msgr2=0x55aa0ba732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.658+0000 7fca66825740 1 --2- 172.21.15.43:0/2701961731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b69e4d0 0x55aa0ba732d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fca5400c6a0 tx=0x7fca540e2a50 comp rx=0 tx=0).stop 2024-04-25T10:36:25.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.658+0000 7fca66825740 1 -- 172.21.15.43:0/2701961731 shutdown_connections 2024-04-25T10:36:25.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.658+0000 7fca66825740 1 --2- 172.21.15.43:0/2701961731 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aa0b685f60 0x55aa0ba856c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.658+0000 7fca66825740 1 --2- 172.21.15.43:0/2701961731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b6c7390 0x55aa0ba7f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.658+0000 7fca40ff9640 0 client.16593 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:25.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.658+0000 7fca66825740 1 --2- 172.21.15.43:0/2701961731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b6a66b0 0x55aa0ba73810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.658+0000 7fca66825740 1 --2- 172.21.15.43:0/2701961731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b69e4d0 0x55aa0ba732d0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.658+0000 7fca66825740 1 -- 172.21.15.43:0/2701961731 >> 172.21.15.43:0/2701961731 conn(0x55aa0b6bb4e0 msgr2=0x55aa0ba6a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:25.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.658+0000 7fca66825740 1 -- 172.21.15.43:0/2701961731 shutdown_connections 2024-04-25T10:36:25.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.658+0000 7fca66825740 1 -- 172.21.15.43:0/2701961731 wait complete. 2024-04-25T10:36:25.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.658+0000 7fca66825740 20 client.16593 trim_cache size 0 max 16384 2024-04-25T10:36:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.672+0000 7fca66825740 1 Processor -- start 2024-04-25T10:36:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.672+0000 7fca66825740 1 -- start start 2024-04-25T10:36:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.672+0000 7fca66825740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b69e4d0 0x55aa0ba723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.672+0000 7fca66825740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b6a66b0 0x55aa0ba7ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.672+0000 7fca66825740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b6c7390 0x55aa0ba74e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.672+0000 7fca66825740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fca581b43f0 con 0x55aa0b6c7390 2024-04-25T10:36:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.672+0000 7fca66825740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca581b1cc0 con 0x55aa0b6a66b0 2024-04-25T10:36:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.672+0000 7fca66825740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fca581afb30 con 0x55aa0b69e4d0 2024-04-25T10:36:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.672+0000 7fca5df0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b6c7390 0x55aa0ba74e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.672+0000 7fca5df0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b6c7390 0x55aa0ba74e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38802/0 (socket says 172.21.15.43:38802) 2024-04-25T10:36:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.672+0000 7fca5df0f640 1 -- 172.21.15.43:0/2242163483 learned_addr learned my addr 172.21.15.43:0/2242163483 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.672+0000 7fca5d70e640 1 --2- 172.21.15.43:0/2242163483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b69e4d0 0x55aa0ba723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.672+0000 7fca5cf0d640 1 --2- 172.21.15.43:0/2242163483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b6a66b0 0x55aa0ba7ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.672+0000 7fca5df0f640 1 -- 172.21.15.43:0/2242163483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b69e4d0 msgr2=0x55aa0ba723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.672+0000 7fca5df0f640 1 --2- 172.21.15.43:0/2242163483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b69e4d0 0x55aa0ba723c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.672+0000 7fca5df0f640 1 -- 172.21.15.43:0/2242163483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b6a66b0 msgr2=0x55aa0ba7ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.672+0000 7fca5df0f640 1 --2- 172.21.15.43:0/2242163483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b6a66b0 0x55aa0ba7ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.672+0000 7fca5df0f640 1 -- 172.21.15.43:0/2242163483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa0ba64620 con 0x55aa0b6c7390 2024-04-25T10:36:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.672+0000 7fca5d70e640 1 --2- 172.21.15.43:0/2242163483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b69e4d0 0x55aa0ba723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.673+0000 7fca5df0f640 1 --2- 172.21.15.43:0/2242163483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b6c7390 0x55aa0ba74e80 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fca50035000 tx=0x7fca50044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d8bf0d986b7d8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.673+0000 7fca42ffd640 1 -- 172.21.15.43:0/2242163483 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50003970 con 0x55aa0b6c7390 2024-04-25T10:36:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.673+0000 7fca42ffd640 1 -- 172.21.15.43:0/2242163483 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca50036050 con 0x55aa0b6c7390 2024-04-25T10:36:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.673+0000 7fca42ffd640 1 -- 172.21.15.43:0/2242163483 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50004860 con 0x55aa0b6c7390 2024-04-25T10:36:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.674+0000 7fca66825740 1 -- 172.21.15.43:0/2242163483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b6c7390 msgr2=0x55aa0ba74e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.674+0000 7fca66825740 1 --2- 172.21.15.43:0/2242163483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b6c7390 0x55aa0ba74e80 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fca50035000 tx=0x7fca50044000 comp rx=0 tx=0).stop 2024-04-25T10:36:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.674+0000 7fca66825740 1 -- 172.21.15.43:0/2242163483 shutdown_connections 2024-04-25T10:36:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.674+0000 7fca66825740 1 --2- 172.21.15.43:0/2242163483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b6a66b0 0x55aa0ba7ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.674+0000 7fca66825740 1 --2- 172.21.15.43:0/2242163483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b6c7390 0x55aa0ba74e80 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.674+0000 7fca66825740 1 --2- 172.21.15.43:0/2242163483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b69e4d0 0x55aa0ba723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.674+0000 7fca66825740 1 -- 172.21.15.43:0/2242163483 >> 172.21.15.43:0/2242163483 conn(0x55aa0b6bb4e0 msgr2=0x55aa0ba74770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.674+0000 7fca66825740 1 -- 172.21.15.43:0/2242163483 shutdown_connections 2024-04-25T10:36:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.674+0000 7fca66825740 1 -- 172.21.15.43:0/2242163483 wait complete. 2024-04-25T10:36:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.674+0000 7fca66825740 1 -- start start 2024-04-25T10:36:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.674+0000 7fca66825740 1 Processor -- start 2024-04-25T10:36:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca66825740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b69e4d0 0x55aa0ba7b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca66825740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b6a66b0 0x55aa0ba76830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca66825740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b6c7390 0x55aa0ba76d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca66825740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fca581b0180 con 0x55aa0b6a66b0 2024-04-25T10:36:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca66825740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca581b15f0 con 0x55aa0b69e4d0 2024-04-25T10:36:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca66825740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fca581b0850 con 0x55aa0b6c7390 2024-04-25T10:36:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca66825740 10 client.0 _ll_register_callbacks cb 0x7fca5400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca5cf0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b6a66b0 0x55aa0ba76830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca5cf0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b6a66b0 0x55aa0ba76830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38818/0 (socket says 172.21.15.43:38818) 2024-04-25T10:36:25.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca5cf0d640 1 -- 172.21.15.43:0/971781396 learned_addr learned my addr 172.21.15.43:0/971781396 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:25.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca5d70e640 1 --2- 172.21.15.43:0/971781396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b69e4d0 0x55aa0ba7b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca5df0f640 1 --2- 172.21.15.43:0/971781396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b6c7390 0x55aa0ba76d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca5cf0d640 1 -- 172.21.15.43:0/971781396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b6c7390 msgr2=0x55aa0ba76d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca5cf0d640 1 --2- 172.21.15.43:0/971781396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b6c7390 0x55aa0ba76d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca5cf0d640 1 -- 172.21.15.43:0/971781396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b69e4d0 msgr2=0x55aa0ba7b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:25.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca5cf0d640 1 --2- 172.21.15.43:0/971781396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b69e4d0 0x55aa0ba7b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca5cf0d640 1 -- 172.21.15.43:0/971781396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa0ba772b0 con 0x55aa0b6a66b0 2024-04-25T10:36:25.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca5d70e640 1 --2- 172.21.15.43:0/971781396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b69e4d0 0x55aa0ba7b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:25.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca5cf0d640 1 --2- 172.21.15.43:0/971781396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b6a66b0 0x55aa0ba76830 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fca48009730 tx=0x7fca48011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bbcfa4a38cf40f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:25.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:25.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.675+0000 7fca40ff9640 1 -- 172.21.15.43:0/971781396 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4801d040 con 0x55aa0b6a66b0 2024-04-25T10:36:25.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.676+0000 7fca40ff9640 1 -- 172.21.15.43:0/971781396 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fca48016040 con 0x55aa0b6a66b0 2024-04-25T10:36:25.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.676+0000 7fca66825740 1 -- 172.21.15.43:0/971781396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca50036050 con 0x55aa0b6a66b0 2024-04-25T10:36:25.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.676+0000 7fca40ff9640 1 -- 172.21.15.43:0/971781396 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca48005560 con 0x55aa0b6a66b0 2024-04-25T10:36:25.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.676+0000 7fca66825740 10 client.16599 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:25.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.676+0000 7fca66825740 1 -- 172.21.15.43:0/971781396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aa0baed680 con 0x55aa0b6a66b0 2024-04-25T10:36:25.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.676+0000 7fca40ff9640 1 -- 172.21.15.43:0/971781396 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca50036050 con 0x55aa0b6a66b0 2024-04-25T10:36:25.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.676+0000 7fca40ff9640 1 -- 172.21.15.43:0/971781396 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fca48019450 con 0x55aa0b6a66b0 2024-04-25T10:36:25.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.676+0000 7fca66825740 10 client.16599 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:25.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.676+0000 7fca66825740 10 client.16599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:25.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.676+0000 7fca66825740 20 client.16599 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:25.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.676+0000 7fca66825740 1 --2- 172.21.15.43:0/971781396 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aa0b65b3a0 0x55aa0baefb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:25.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.676+0000 7fca66825740 4 client.16599 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:36:25.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.676+0000 7fca66825740 1 -- 172.21.15.43:0/971781396 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aa0b9d5800 con 0x55aa0b65b3a0 2024-04-25T10:36:25.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.676+0000 7fca5d70e640 1 --2- 172.21.15.43:0/971781396 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aa0b65b3a0 0x55aa0baefb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:25.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.677+0000 7fca5d70e640 1 --2- 172.21.15.43:0/971781396 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aa0b65b3a0 0x55aa0baefb70 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12759f61068f09a0 server_cookie=c3fc3c8d39e90147 in_seq=0 out_seq=0 2024-04-25T10:36:25.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.677+0000 7fca40ff9640 10 client.16599 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:25.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.677+0000 7fca40ff9640 1 -- 172.21.15.43:0/971781396 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aa0b9d5800 con 0x55aa0b65b3a0 2024-04-25T10:36:25.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.677+0000 7fca40ff9640 10 client.16599 handle_command_reply: tid=0 2024-04-25T10:36:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.677+0000 7fca66825740 1 client.16599 shutdown 2024-04-25T10:36:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.677+0000 7fca66825740 1 -- 172.21.15.43:0/971781396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b6a66b0 msgr2=0x55aa0ba76830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.677+0000 7fca66825740 1 --2- 172.21.15.43:0/971781396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b6a66b0 0x55aa0ba76830 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fca48009730 tx=0x7fca48011d10 comp rx=0 tx=0).stop 2024-04-25T10:36:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.677+0000 7fca66825740 1 -- 172.21.15.43:0/971781396 shutdown_connections 2024-04-25T10:36:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.677+0000 7fca66825740 1 --2- 172.21.15.43:0/971781396 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aa0b65b3a0 0x55aa0baefb70 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.677+0000 7fca66825740 1 --2- 172.21.15.43:0/971781396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa0b6c7390 0x55aa0ba76d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.677+0000 7fca66825740 1 --2- 172.21.15.43:0/971781396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b6a66b0 0x55aa0ba76830 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.677+0000 7fca66825740 1 --2- 172.21.15.43:0/971781396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aa0b69e4d0 0x55aa0ba7b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.677+0000 7fca40ff9640 0 client.16599 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.677+0000 7fca66825740 1 -- 172.21.15.43:0/971781396 >> 172.21.15.43:0/971781396 conn(0x55aa0b6bb4e0 msgr2=0x55aa0ba72120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.677+0000 7fca66825740 1 -- 172.21.15.43:0/971781396 shutdown_connections 2024-04-25T10:36:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.677+0000 7fca66825740 1 -- 172.21.15.43:0/971781396 wait complete. 2024-04-25T10:36:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.677+0000 7fca66825740 20 client.16599 trim_cache size 0 max 16384 2024-04-25T10:36:25.686 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:36:25.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.678+0000 7fca5f972640 1 -- 172.21.15.43:0/2446315619 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fca440afce0 msgr2=0x7fca440b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.678+0000 7fca5f972640 1 --2- 172.21.15.43:0/2446315619 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fca440afce0 0x7fca440b21f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fca5002f500 tx=0x7fca5002f310 comp rx=0 tx=0).stop 2024-04-25T10:36:25.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.678+0000 7fca5f972640 1 -- 172.21.15.43:0/2446315619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca58104260 msgr2=0x7fca5819a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:25.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.678+0000 7fca5f972640 1 --2- 172.21.15.43:0/2446315619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca58104260 0x7fca5819a740 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fca5400e9f0 tx=0x7fca5400eeb0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.678+0000 7fca5f972640 1 -- 172.21.15.43:0/2446315619 shutdown_connections 2024-04-25T10:36:25.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.678+0000 7fca5f972640 1 --2- 172.21.15.43:0/2446315619 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fca440afce0 0x7fca440b21f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.678+0000 7fca5f972640 1 --2- 172.21.15.43:0/2446315619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca58109b10 0x7fca5819aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.678+0000 7fca5f972640 1 --2- 172.21.15.43:0/2446315619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca58109fb0 0x7fca5819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.678+0000 7fca5f972640 1 --2- 172.21.15.43:0/2446315619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca58104260 0x7fca5819a740 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:25.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.678+0000 7fca5f972640 1 -- 172.21.15.43:0/2446315619 >> 172.21.15.43:0/2446315619 conn(0x7fca580ff210 msgr2=0x7fca58106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:25.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.678+0000 7fca5f972640 1 -- 172.21.15.43:0/2446315619 shutdown_connections 2024-04-25T10:36:25.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:25.678+0000 7fca5f972640 1 -- 172.21.15.43:0/2446315619 wait complete. 2024-04-25T10:36:25.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:25 smithi043 ceph-mon[29478]: pgmap v120: 97 pgs: 97 active+clean; 4.4 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-25T10:36:25.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:25 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/6115003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:36:26.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:25 smithi077 ceph-mon[32772]: pgmap v120: 97 pgs: 97 active+clean; 4.4 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-25T10:36:26.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:25 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/6115003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:36:26.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:25 smithi164 ceph-mon[32713]: pgmap v120: 97 pgs: 97 active+clean; 4.4 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-25T10:36:26.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:25 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/6115003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:36:26.692 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:36:26.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:36:26.692 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:36:26.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.822+0000 7f7f85084640 1 -- 172.21.15.43:0/331705704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f80108a10 msgr2=0x7f7f80102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:26.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.822+0000 7f7f85084640 1 --2- 172.21.15.43:0/331705704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f80108a10 0x7f7f80102610 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009900 tx=0x7f7f7002a410 comp rx=0 tx=0).stop 2024-04-25T10:36:26.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.822+0000 7f7f85084640 1 -- 172.21.15.43:0/331705704 shutdown_connections 2024-04-25T10:36:26.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.822+0000 7f7f85084640 1 --2- 172.21.15.43:0/331705704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f801031c0 0x7f7f8010f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.822+0000 7f7f85084640 1 --2- 172.21.15.43:0/331705704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f80108eb0 0x7f7f80102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.822+0000 7f7f85084640 1 --2- 172.21.15.43:0/331705704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f80108a10 0x7f7f80102610 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.822+0000 7f7f85084640 1 -- 172.21.15.43:0/331705704 >> 172.21.15.43:0/331705704 conn(0x7f7f80078110 msgr2=0x7f7f80078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:26.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.822+0000 7f7f85084640 1 -- 172.21.15.43:0/331705704 shutdown_connections 2024-04-25T10:36:26.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.822+0000 7f7f85084640 1 -- 172.21.15.43:0/331705704 wait complete. 2024-04-25T10:36:26.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.823+0000 7f7f85084640 1 Processor -- start 2024-04-25T10:36:26.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.823+0000 7f7f85084640 1 -- start start 2024-04-25T10:36:26.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.823+0000 7f7f85084640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f801031c0 0x7f7f8019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.823+0000 7f7f85084640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f80108a10 0x7f7f8019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.823+0000 7f7f85084640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f80108eb0 0x7f7f8019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.823+0000 7f7f85084640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80112c10 con 0x7f7f80108a10 2024-04-25T10:36:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.824+0000 7f7f85084640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80112a90 con 0x7f7f801031c0 2024-04-25T10:36:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.824+0000 7f7f85084640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80112d90 con 0x7f7f80108eb0 2024-04-25T10:36:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.824+0000 7f7f7e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f80108a10 0x7f7f8019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.824+0000 7f7f7e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f80108a10 0x7f7f8019acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38826/0 (socket says 172.21.15.43:38826) 2024-04-25T10:36:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.824+0000 7f7f7e59c640 1 -- 172.21.15.43:0/1169095314 learned_addr learned my addr 172.21.15.43:0/1169095314 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.824+0000 7f7f7ed9d640 1 --2- 172.21.15.43:0/1169095314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f801031c0 0x7f7f8019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.824+0000 7f7f7f59e640 1 --2- 172.21.15.43:0/1169095314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f80108eb0 0x7f7f8019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.824+0000 7f7f7e59c640 1 -- 172.21.15.43:0/1169095314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f80108eb0 msgr2=0x7f7f8019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.824+0000 7f7f7e59c640 1 --2- 172.21.15.43:0/1169095314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f80108eb0 0x7f7f8019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.824+0000 7f7f7e59c640 1 -- 172.21.15.43:0/1169095314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f801031c0 msgr2=0x7f7f8019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.824+0000 7f7f7e59c640 1 --2- 172.21.15.43:0/1169095314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f801031c0 0x7f7f8019a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.824+0000 7f7f7e59c640 1 -- 172.21.15.43:0/1169095314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f8019f5c0 con 0x7f7f80108a10 2024-04-25T10:36:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.824+0000 7f7f7e59c640 1 --2- 172.21.15.43:0/1169095314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f80108a10 0x7f7f8019acc0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7f74005d00 tx=0x7f7f7400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.825+0000 7f7f67fff640 1 -- 172.21.15.43:0/1169095314 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74017070 con 0x7f7f80108a10 2024-04-25T10:36:26.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.825+0000 7f7f85084640 1 -- 172.21.15.43:0/1169095314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f8019f8b0 con 0x7f7f80108a10 2024-04-25T10:36:26.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.825+0000 7f7f67fff640 1 -- 172.21.15.43:0/1169095314 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f74009c30 con 0x7f7f80108a10 2024-04-25T10:36:26.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.825+0000 7f7f67fff640 1 -- 172.21.15.43:0/1169095314 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7400f620 con 0x7f7f80108a10 2024-04-25T10:36:26.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.825+0000 7f7f85084640 1 -- 172.21.15.43:0/1169095314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f801a7230 con 0x7f7f80108a10 2024-04-25T10:36:26.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.827+0000 7f7f8c00c740 1 Processor -- start 2024-04-25T10:36:26.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.827+0000 7f7f8c00c740 1 -- start start 2024-04-25T10:36:26.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.827+0000 7f7f8c00c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576249c4410 0x557624d64620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.827+0000 7f7f67fff640 1 -- 172.21.15.43:0/1169095314 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f7f7401a070 con 0x7f7f80108a10 2024-04-25T10:36:26.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.827+0000 7f7f8c00c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55762499b550 0x557624d64b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.827+0000 7f7f8c00c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249a3730 0x557624d650a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.827+0000 7f7f8c00c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557624ccab00 con 0x55762499b550 2024-04-25T10:36:26.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.827+0000 7f7f8c00c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557624c66920 con 0x5576249a3730 2024-04-25T10:36:26.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.827+0000 7f7f8c00c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557624cc6ac0 con 0x5576249c4410 2024-04-25T10:36:26.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.828+0000 7f7f7ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55762499b550 0x557624d64b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.828+0000 7f7f7ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55762499b550 0x557624d64b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38830/0 (socket says 172.21.15.43:38830) 2024-04-25T10:36:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.828+0000 7f7f7ed9d640 1 -- 172.21.15.43:0/1465075502 learned_addr learned my addr 172.21.15.43:0/1465075502 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.828+0000 7f7f7e59c640 1 --2- 172.21.15.43:0/1465075502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249a3730 0x557624d650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.828+0000 7f7f7f59e640 1 --2- 172.21.15.43:0/1465075502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576249c4410 0x557624d64620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.828+0000 7f7f7ed9d640 1 -- 172.21.15.43:0/1465075502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576249c4410 msgr2=0x557624d64620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.828+0000 7f7f7ed9d640 1 --2- 172.21.15.43:0/1465075502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576249c4410 0x557624d64620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.828+0000 7f7f7ed9d640 1 -- 172.21.15.43:0/1465075502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249a3730 msgr2=0x557624d650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.828+0000 7f7f7ed9d640 1 --2- 172.21.15.43:0/1465075502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249a3730 0x557624d650a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.828+0000 7f7f7ed9d640 1 -- 172.21.15.43:0/1465075502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557624aa47e0 con 0x55762499b550 2024-04-25T10:36:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.828+0000 7f7f67fff640 1 --2- 172.21.15.43:0/1169095314 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7f680afc60 0x7f7f680b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.828+0000 7f7f7ed9d640 1 --2- 172.21.15.43:0/1465075502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55762499b550 0x557624d64b60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009900 tx=0x7f7f70000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625bcdd168dd1e8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.828+0000 7f7f7f59e640 1 --2- 172.21.15.43:0/1169095314 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7f680afc60 0x7f7f680b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.828+0000 7f7f657fa640 1 -- 172.21.15.43:0/1465075502 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70040070 con 0x55762499b550 2024-04-25T10:36:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.828+0000 7f7f67fff640 1 -- 172.21.15.43:0/1169095314 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f7f740de9f0 con 0x7f7f80108a10 2024-04-25T10:36:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.829+0000 7f7f657fa640 1 -- 172.21.15.43:0/1465075502 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f70005db0 con 0x55762499b550 2024-04-25T10:36:26.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.829+0000 7f7f657fa640 1 -- 172.21.15.43:0/1465075502 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70035040 con 0x55762499b550 2024-04-25T10:36:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.829+0000 7f7f8c00c740 1 -- 172.21.15.43:0/1465075502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55762499b550 msgr2=0x557624d64b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.829+0000 7f7f7f59e640 1 --2- 172.21.15.43:0/1169095314 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7f680afc60 0x7f7f680b2170 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x557624d61790 tx=0x7f7f7800a4d0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.829+0000 7f7f8c00c740 1 --2- 172.21.15.43:0/1465075502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55762499b550 0x557624d64b60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009900 tx=0x7f7f70000bc0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.829+0000 7f7f8c00c740 1 -- 172.21.15.43:0/1465075502 shutdown_connections 2024-04-25T10:36:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.829+0000 7f7f8c00c740 1 --2- 172.21.15.43:0/1465075502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249a3730 0x557624d650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.829+0000 7f7f8c00c740 1 --2- 172.21.15.43:0/1465075502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55762499b550 0x557624d64b60 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.829+0000 7f7f8c00c740 1 --2- 172.21.15.43:0/1465075502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576249c4410 0x557624d64620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.830+0000 7f7f8c00c740 1 -- 172.21.15.43:0/1465075502 >> 172.21.15.43:0/1465075502 conn(0x5576249b8560 msgr2=0x557624d60dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.830+0000 7f7f8c00c740 1 -- 172.21.15.43:0/1465075502 shutdown_connections 2024-04-25T10:36:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.830+0000 7f7f8c00c740 1 -- 172.21.15.43:0/1465075502 wait complete. 2024-04-25T10:36:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.830+0000 7f7f8c00c740 1 -- start start 2024-04-25T10:36:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.830+0000 7f7f8c00c740 1 Processor -- start 2024-04-25T10:36:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.830+0000 7f7f8c00c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55762499b550 0x557624d6ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.830+0000 7f7f8c00c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249a3730 0x557624d6f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.830+0000 7f7f8c00c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576249c4410 0x557624d74d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.830+0000 7f7f8c00c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557624d71910 con 0x55762499b550 2024-04-25T10:36:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.830+0000 7f7f8c00c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557624c645d0 con 0x5576249a3730 2024-04-25T10:36:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.830+0000 7f7f8c00c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557624d71a90 con 0x5576249c4410 2024-04-25T10:36:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.831+0000 7f7f8c00c740 10 client.0 _ll_register_callbacks cb 0x557624c52080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.831+0000 7f7f7ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55762499b550 0x557624d6ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.831+0000 7f7f7ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55762499b550 0x557624d6ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38836/0 (socket says 172.21.15.43:38836) 2024-04-25T10:36:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.831+0000 7f7f7ed9d640 1 -- 172.21.15.43:0/734537513 learned_addr learned my addr 172.21.15.43:0/734537513 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.831+0000 7f7f7f59e640 1 --2- 172.21.15.43:0/734537513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576249c4410 0x557624d74d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.831+0000 7f7f7e59c640 1 --2- 172.21.15.43:0/734537513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249a3730 0x557624d6f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.831+0000 7f7f7ed9d640 1 -- 172.21.15.43:0/734537513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576249c4410 msgr2=0x557624d74d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.831+0000 7f7f7ed9d640 1 --2- 172.21.15.43:0/734537513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576249c4410 0x557624d74d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.831+0000 7f7f7ed9d640 1 -- 172.21.15.43:0/734537513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249a3730 msgr2=0x557624d6f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.831+0000 7f7f7ed9d640 1 --2- 172.21.15.43:0/734537513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249a3730 0x557624d6f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.831+0000 7f7f7ed9d640 1 -- 172.21.15.43:0/734537513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557624b04f70 con 0x55762499b550 2024-04-25T10:36:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.831+0000 7f7f7f59e640 1 --2- 172.21.15.43:0/734537513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576249c4410 0x557624d74d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.831+0000 7f7f7ed9d640 1 --2- 172.21.15.43:0/734537513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55762499b550 0x557624d6ec80 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7f7002a930 tx=0x7f7f700055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff49d58793a02950 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.831+0000 7f7f62ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:26.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.831+0000 7f7f62ff5640 1 -- 172.21.15.43:0/734537513 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70035070 con 0x55762499b550 2024-04-25T10:36:26.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.831+0000 7f7f62ff5640 1 -- 172.21.15.43:0/734537513 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f7f700314a0 con 0x55762499b550 2024-04-25T10:36:26.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.831+0000 7f7f8c00c740 1 -- 172.21.15.43:0/734537513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f70005db0 con 0x55762499b550 2024-04-25T10:36:26.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.831+0000 7f7f62ff5640 1 -- 172.21.15.43:0/734537513 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7f70055050 con 0x55762499b550 2024-04-25T10:36:26.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.831+0000 7f7f8c00c740 10 client.16611 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:26.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.831+0000 7f7f8c00c740 1 -- 172.21.15.43:0/734537513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55762499c170 con 0x55762499b550 2024-04-25T10:36:26.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.832+0000 7f7f62ff5640 1 -- 172.21.15.43:0/734537513 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f70005db0 con 0x55762499b550 2024-04-25T10:36:26.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.832+0000 7f7f62ff5640 1 -- 172.21.15.43:0/734537513 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7f7003c540 con 0x55762499b550 2024-04-25T10:36:26.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.832+0000 7f7f8c00c740 10 client.16611 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:26.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.832+0000 7f7f8c00c740 10 client.16611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:26.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.832+0000 7f7f8c00c740 20 client.16611 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:26.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.832+0000 7f7f8c00c740 1 --2- 172.21.15.43:0/734537513 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5576249831f0 0x557624d80e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.832+0000 7f7f8c00c740 4 client.16611 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:36:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.832+0000 7f7f8c00c740 1 -- 172.21.15.43:0/734537513 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557624cd8710 con 0x5576249831f0 2024-04-25T10:36:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.832+0000 7f7f7e59c640 1 --2- 172.21.15.43:0/734537513 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5576249831f0 0x557624d80e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.833+0000 7f7f7e59c640 1 --2- 172.21.15.43:0/734537513 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5576249831f0 0x557624d80e50 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b24440b1f4d88d80 server_cookie=39f5d8750ed52f92 in_seq=0 out_seq=0 2024-04-25T10:36:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.833+0000 7f7f62ff5640 10 client.16611 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.833+0000 7f7f62ff5640 1 -- 172.21.15.43:0/734537513 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557624cd8710 con 0x5576249831f0 2024-04-25T10:36:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.833+0000 7f7f62ff5640 10 client.16611 handle_command_reply: tid=0 2024-04-25T10:36:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.833+0000 7f7f8c00c740 1 client.16611 shutdown 2024-04-25T10:36:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.834+0000 7f7f8c00c740 1 -- 172.21.15.43:0/734537513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55762499b550 msgr2=0x557624d6ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.834+0000 7f7f8c00c740 1 --2- 172.21.15.43:0/734537513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55762499b550 0x557624d6ec80 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7f7002a930 tx=0x7f7f700055e0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.834+0000 7f7f8c00c740 1 -- 172.21.15.43:0/734537513 shutdown_connections 2024-04-25T10:36:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.834+0000 7f7f8c00c740 1 --2- 172.21.15.43:0/734537513 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5576249831f0 0x557624d80e50 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.834+0000 7f7f8c00c740 1 --2- 172.21.15.43:0/734537513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249a3730 0x557624d6f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.834+0000 7f7f62ff5640 0 client.16611 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:26.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.834+0000 7f7f8c00c740 1 --2- 172.21.15.43:0/734537513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576249c4410 0x557624d74d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.834+0000 7f7f8c00c740 1 --2- 172.21.15.43:0/734537513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55762499b550 0x557624d6ec80 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.834+0000 7f7f8c00c740 1 -- 172.21.15.43:0/734537513 >> 172.21.15.43:0/734537513 conn(0x5576249b8560 msgr2=0x557624d64450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:26.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.834+0000 7f7f8c00c740 1 -- 172.21.15.43:0/734537513 shutdown_connections 2024-04-25T10:36:26.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.834+0000 7f7f8c00c740 1 -- 172.21.15.43:0/734537513 wait complete. 2024-04-25T10:36:26.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.834+0000 7f7f8c00c740 20 client.16611 trim_cache size 0 max 16384 2024-04-25T10:36:26.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.852+0000 7f7f8c00c740 1 Processor -- start 2024-04-25T10:36:26.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.852+0000 7f7f8c00c740 1 -- start start 2024-04-25T10:36:26.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.852+0000 7f7f8c00c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55762499b550 0x557624d5fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.852+0000 7f7f8c00c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576249a3730 0x557624d74d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.852+0000 7f7f8c00c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249c4410 0x557624d75240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.852+0000 7f7f8c00c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801b21c0 con 0x5576249a3730 2024-04-25T10:36:26.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.852+0000 7f7f8c00c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801afa90 con 0x5576249c4410 2024-04-25T10:36:26.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.852+0000 7f7f8c00c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801ad8e0 con 0x55762499b550 2024-04-25T10:36:26.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.852+0000 7f7f7e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576249a3730 0x557624d74d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.852+0000 7f7f7e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576249a3730 0x557624d74d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38840/0 (socket says 172.21.15.43:38840) 2024-04-25T10:36:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.852+0000 7f7f7e59c640 1 -- 172.21.15.43:0/755558241 learned_addr learned my addr 172.21.15.43:0/755558241 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.852+0000 7f7f7f59e640 1 --2- 172.21.15.43:0/755558241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249c4410 0x557624d75240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.853+0000 7f7f7ed9d640 1 --2- 172.21.15.43:0/755558241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55762499b550 0x557624d5fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.853+0000 7f7f7e59c640 1 -- 172.21.15.43:0/755558241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55762499b550 msgr2=0x557624d5fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.853+0000 7f7f7e59c640 1 --2- 172.21.15.43:0/755558241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55762499b550 0x557624d5fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.853+0000 7f7f7e59c640 1 -- 172.21.15.43:0/755558241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249c4410 msgr2=0x557624d75240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.853+0000 7f7f7e59c640 1 --2- 172.21.15.43:0/755558241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249c4410 0x557624d75240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.853+0000 7f7f7e59c640 1 -- 172.21.15.43:0/755558241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557624d6f930 con 0x5576249a3730 2024-04-25T10:36:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.853+0000 7f7f7ed9d640 1 --2- 172.21.15.43:0/755558241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55762499b550 0x557624d5fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:36:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.853+0000 7f7f7f59e640 1 --2- 172.21.15.43:0/755558241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249c4410 0x557624d75240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.853+0000 7f7f7e59c640 1 --2- 172.21.15.43:0/755558241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576249a3730 0x557624d74d00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7f740a3ee0 tx=0x7f7f740c9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ba04364bf0eb7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.853+0000 7f7f64ff9640 1 -- 172.21.15.43:0/755558241 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74021040 con 0x5576249a3730 2024-04-25T10:36:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.853+0000 7f7f64ff9640 1 -- 172.21.15.43:0/755558241 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f740af360 con 0x5576249a3730 2024-04-25T10:36:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.853+0000 7f7f64ff9640 1 -- 172.21.15.43:0/755558241 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f740aaf70 con 0x5576249a3730 2024-04-25T10:36:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.854+0000 7f7f8c00c740 1 -- 172.21.15.43:0/755558241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576249a3730 msgr2=0x557624d74d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.854+0000 7f7f8c00c740 1 --2- 172.21.15.43:0/755558241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576249a3730 0x557624d74d00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7f740a3ee0 tx=0x7f7f740c9040 comp rx=0 tx=0).stop 2024-04-25T10:36:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.854+0000 7f7f8c00c740 1 -- 172.21.15.43:0/755558241 shutdown_connections 2024-04-25T10:36:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.854+0000 7f7f8c00c740 1 --2- 172.21.15.43:0/755558241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249c4410 0x557624d75240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.854+0000 7f7f8c00c740 1 --2- 172.21.15.43:0/755558241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576249a3730 0x557624d74d00 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.854+0000 7f7f8c00c740 1 --2- 172.21.15.43:0/755558241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55762499b550 0x557624d5fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.854+0000 7f7f8c00c740 1 -- 172.21.15.43:0/755558241 >> 172.21.15.43:0/755558241 conn(0x5576249b8560 msgr2=0x557624d6d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.854+0000 7f7f8c00c740 1 -- 172.21.15.43:0/755558241 shutdown_connections 2024-04-25T10:36:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.854+0000 7f7f8c00c740 1 -- 172.21.15.43:0/755558241 wait complete. 2024-04-25T10:36:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.854+0000 7f7f8c00c740 1 -- start start 2024-04-25T10:36:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.854+0000 7f7f8c00c740 1 Processor -- start 2024-04-25T10:36:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f8c00c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55762499b550 0x557624d7ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f8c00c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249a3730 0x557624d781f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f8c00c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576249c4410 0x557624d78730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f8c00c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801adf50 con 0x5576249c4410 2024-04-25T10:36:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f8c00c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801af3c0 con 0x5576249a3730 2024-04-25T10:36:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f8c00c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801ae620 con 0x55762499b550 2024-04-25T10:36:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f8c00c740 10 client.0 _ll_register_callbacks cb 0x7f7f70005210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f7f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576249c4410 0x557624d78730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f7f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576249c4410 0x557624d78730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38850/0 (socket says 172.21.15.43:38850) 2024-04-25T10:36:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f7f59e640 1 -- 172.21.15.43:0/703235764 learned_addr learned my addr 172.21.15.43:0/703235764 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:26.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f7ed9d640 1 --2- 172.21.15.43:0/703235764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55762499b550 0x557624d7ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f7e59c640 1 --2- 172.21.15.43:0/703235764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249a3730 0x557624d781f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f7f59e640 1 -- 172.21.15.43:0/703235764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55762499b550 msgr2=0x557624d7ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:26.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f7f59e640 1 --2- 172.21.15.43:0/703235764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55762499b550 0x557624d7ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f7f59e640 1 -- 172.21.15.43:0/703235764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249a3730 msgr2=0x557624d781f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:26.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f7f59e640 1 --2- 172.21.15.43:0/703235764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249a3730 0x557624d781f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f7f59e640 1 -- 172.21.15.43:0/703235764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557624d653c0 con 0x5576249c4410 2024-04-25T10:36:26.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f7f59e640 1 --2- 172.21.15.43:0/703235764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576249c4410 0x557624d78730 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7f7801ef40 tx=0x7f7f78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ae8e8f2d6bbc92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:26.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f62ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:26.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f62ff5640 1 -- 172.21.15.43:0/703235764 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7802c060 con 0x5576249c4410 2024-04-25T10:36:26.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f62ff5640 1 -- 172.21.15.43:0/703235764 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f7f7801e320 con 0x5576249c4410 2024-04-25T10:36:26.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.855+0000 7f7f8c00c740 1 -- 172.21.15.43:0/703235764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557624cd2580 con 0x5576249c4410 2024-04-25T10:36:26.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.856+0000 7f7f62ff5640 1 -- 172.21.15.43:0/703235764 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7f7801cb00 con 0x5576249c4410 2024-04-25T10:36:26.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.856+0000 7f7f8c00c740 10 client.16617 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.856+0000 7f7f8c00c740 1 -- 172.21.15.43:0/703235764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557624de8b70 con 0x5576249c4410 2024-04-25T10:36:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.856+0000 7f7f62ff5640 1 -- 172.21.15.43:0/703235764 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557624cd2580 con 0x5576249c4410 2024-04-25T10:36:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.856+0000 7f7f62ff5640 1 -- 172.21.15.43:0/703235764 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7f7801d410 con 0x5576249c4410 2024-04-25T10:36:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.856+0000 7f7f8c00c740 10 client.16617 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.856+0000 7f7f8c00c740 10 client.16617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.856+0000 7f7f8c00c740 20 client.16617 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.856+0000 7f7f8c00c740 1 --2- 172.21.15.43:0/703235764 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557624958630 0x557624deb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.856+0000 7f7f8c00c740 4 client.16617 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:36:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.856+0000 7f7f8c00c740 1 -- 172.21.15.43:0/703235764 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557624c4dac0 con 0x557624958630 2024-04-25T10:36:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.856+0000 7f7f7ed9d640 1 --2- 172.21.15.43:0/703235764 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557624958630 0x557624deb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.857+0000 7f7f7ed9d640 1 --2- 172.21.15.43:0/703235764 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557624958630 0x557624deb360 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4c0280bb40cd021 server_cookie=b6f95219791790bf in_seq=0 out_seq=0 2024-04-25T10:36:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.857+0000 7f7f62ff5640 10 client.16617 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.859+0000 7f7f62ff5640 1 -- 172.21.15.43:0/703235764 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557624c4dac0 con 0x557624958630 2024-04-25T10:36:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.859+0000 7f7f62ff5640 10 client.16617 handle_command_reply: tid=0 2024-04-25T10:36:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.859+0000 7f7f8c00c740 1 client.16617 shutdown 2024-04-25T10:36:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.859+0000 7f7f8c00c740 1 -- 172.21.15.43:0/703235764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576249c4410 msgr2=0x557624d78730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.859+0000 7f7f8c00c740 1 --2- 172.21.15.43:0/703235764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576249c4410 0x557624d78730 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7f7801ef40 tx=0x7f7f78004a40 comp rx=0 tx=0).stop 2024-04-25T10:36:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.859+0000 7f7f8c00c740 1 -- 172.21.15.43:0/703235764 shutdown_connections 2024-04-25T10:36:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.859+0000 7f7f8c00c740 1 --2- 172.21.15.43:0/703235764 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557624958630 0x557624deb360 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.859+0000 7f7f8c00c740 1 --2- 172.21.15.43:0/703235764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576249a3730 0x557624d781f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.859+0000 7f7f8c00c740 1 --2- 172.21.15.43:0/703235764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576249c4410 0x557624d78730 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.859+0000 7f7f8c00c740 1 --2- 172.21.15.43:0/703235764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55762499b550 0x557624d7ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.859+0000 7f7f62ff5640 0 client.16617 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.859+0000 7f7f8c00c740 1 -- 172.21.15.43:0/703235764 >> 172.21.15.43:0/703235764 conn(0x5576249b8560 msgr2=0x557624d643b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.859+0000 7f7f8c00c740 1 -- 172.21.15.43:0/703235764 shutdown_connections 2024-04-25T10:36:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.859+0000 7f7f8c00c740 1 -- 172.21.15.43:0/703235764 wait complete. 2024-04-25T10:36:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.859+0000 7f7f8c00c740 20 client.16617 trim_cache size 0 max 16384 2024-04-25T10:36:26.866 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:36:26.866 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:36:26.866 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "c8d847ea-851f-443a-97b4-c241830dda8e", 2024-04-25T10:36:26.866 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:36:26.866 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:36:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.859+0000 7f7f85084640 1 -- 172.21.15.43:0/1169095314 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7f680afc60 msgr2=0x7f7f680b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.859+0000 7f7f85084640 1 --2- 172.21.15.43:0/1169095314 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7f680afc60 0x7f7f680b2170 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x557624d61790 tx=0x7f7f7800a4d0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.859+0000 7f7f85084640 1 -- 172.21.15.43:0/1169095314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f80108a10 msgr2=0x7f7f8019acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.859+0000 7f7f85084640 1 --2- 172.21.15.43:0/1169095314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f80108a10 0x7f7f8019acc0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7f74005d00 tx=0x7f7f7400dde0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.860+0000 7f7f85084640 1 -- 172.21.15.43:0/1169095314 shutdown_connections 2024-04-25T10:36:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.860+0000 7f7f85084640 1 --2- 172.21.15.43:0/1169095314 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7f680afc60 0x7f7f680b2170 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.860+0000 7f7f85084640 1 --2- 172.21.15.43:0/1169095314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f80108eb0 0x7f7f8019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.860+0000 7f7f85084640 1 --2- 172.21.15.43:0/1169095314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f80108a10 0x7f7f8019acc0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.860+0000 7f7f85084640 1 --2- 172.21.15.43:0/1169095314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f801031c0 0x7f7f8019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.860+0000 7f7f85084640 1 -- 172.21.15.43:0/1169095314 >> 172.21.15.43:0/1169095314 conn(0x7f7f80078110 msgr2=0x7f7f80074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.860+0000 7f7f85084640 1 -- 172.21.15.43:0/1169095314 shutdown_connections 2024-04-25T10:36:26.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.860+0000 7f7f85084640 1 -- 172.21.15.43:0/1169095314 wait complete. 2024-04-25T10:36:26.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:36:26.873 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:36:26.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.985+0000 7f48c29e3640 1 -- 172.21.15.43:0/1181307591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48bc108a30 msgr2=0x7f48bc1025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:26.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.985+0000 7f48c29e3640 1 --2- 172.21.15.43:0/1181307591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48bc108a30 0x7f48bc1025e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009900 tx=0x7f48ac02a410 comp rx=0 tx=0).stop 2024-04-25T10:36:26.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.985+0000 7f48c29e3640 1 -- 172.21.15.43:0/1181307591 shutdown_connections 2024-04-25T10:36:26.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.985+0000 7f48c29e3640 1 --2- 172.21.15.43:0/1181307591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48bc108ed0 0x7f48bc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.985+0000 7f48c29e3640 1 --2- 172.21.15.43:0/1181307591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48bc103190 0x7f48bc10f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.985+0000 7f48c29e3640 1 --2- 172.21.15.43:0/1181307591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48bc108a30 0x7f48bc1025e0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.985+0000 7f48c29e3640 1 -- 172.21.15.43:0/1181307591 >> 172.21.15.43:0/1181307591 conn(0x7f48bc0fe260 msgr2=0x7f48bc100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:26.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.985+0000 7f48c29e3640 1 -- 172.21.15.43:0/1181307591 shutdown_connections 2024-04-25T10:36:26.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.985+0000 7f48c29e3640 1 -- 172.21.15.43:0/1181307591 wait complete. 2024-04-25T10:36:26.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.986+0000 7f48c29e3640 1 Processor -- start 2024-04-25T10:36:26.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.986+0000 7f48c29e3640 1 -- start start 2024-04-25T10:36:26.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.986+0000 7f48c29e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48bc103190 0x7f48bc19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.986+0000 7f48c29e3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48bc108a30 0x7f48bc19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.986+0000 7f48c29e3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48bc108ed0 0x7f48bc19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.986+0000 7f48c29e3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc112c30 con 0x7f48bc103190 2024-04-25T10:36:26.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.986+0000 7f48c29e3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc112ab0 con 0x7f48bc108a30 2024-04-25T10:36:26.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.986+0000 7f48c29e3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc112db0 con 0x7f48bc108ed0 2024-04-25T10:36:26.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.986+0000 7f48bbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48bc103190 0x7f48bc19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.986+0000 7f48bbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48bc103190 0x7f48bc19a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38858/0 (socket says 172.21.15.43:38858) 2024-04-25T10:36:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.986+0000 7f48bbfff640 1 -- 172.21.15.43:0/806082905 learned_addr learned my addr 172.21.15.43:0/806082905 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.986+0000 7f48bb7fe640 1 --2- 172.21.15.43:0/806082905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48bc108a30 0x7f48bc19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.986+0000 7f48bbfff640 1 -- 172.21.15.43:0/806082905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48bc108ed0 msgr2=0x7f48bc19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.986+0000 7f48bbfff640 1 --2- 172.21.15.43:0/806082905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48bc108ed0 0x7f48bc19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.986+0000 7f48bbfff640 1 -- 172.21.15.43:0/806082905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48bc108a30 msgr2=0x7f48bc19ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.986+0000 7f48bbfff640 1 --2- 172.21.15.43:0/806082905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48bc108a30 0x7f48bc19ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.986+0000 7f48bbfff640 1 -- 172.21.15.43:0/806082905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48bc19f600 con 0x7f48bc103190 2024-04-25T10:36:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.986+0000 7f48bb7fe640 1 --2- 172.21.15.43:0/806082905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48bc108a30 0x7f48bc19ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:36:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.987+0000 7f48bbfff640 1 --2- 172.21.15.43:0/806082905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48bc103190 0x7f48bc19a7a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f48ac02a930 tx=0x7f48ac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.987+0000 7f48b97fa640 1 -- 172.21.15.43:0/806082905 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac03f030 con 0x7f48bc103190 2024-04-25T10:36:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.987+0000 7f48c29e3640 1 -- 172.21.15.43:0/806082905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48bc19f890 con 0x7f48bc103190 2024-04-25T10:36:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.987+0000 7f48b97fa640 1 -- 172.21.15.43:0/806082905 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48ac003aa0 con 0x7f48bc103190 2024-04-25T10:36:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.987+0000 7f48b97fa640 1 -- 172.21.15.43:0/806082905 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac035070 con 0x7f48bc103190 2024-04-25T10:36:26.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.987+0000 7f48c29e3640 1 -- 172.21.15.43:0/806082905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48bc19fd80 con 0x7f48bc103190 2024-04-25T10:36:26.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.989+0000 7f48b97fa640 1 -- 172.21.15.43:0/806082905 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f48ac042030 con 0x7f48bc103190 2024-04-25T10:36:26.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.990+0000 7f48c996a740 1 Processor -- start 2024-04-25T10:36:26.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.990+0000 7f48c996a740 1 -- start start 2024-04-25T10:36:26.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.990+0000 7f48c996a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1cf150390 0x55f1cf4f0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.990+0000 7f48c996a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf1274d0 0x55f1cf4f0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.990+0000 7f48c996a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf12f6b0 0x55f1cf4fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.990+0000 7f48c996a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f1cf456d80 con 0x55f1cf12f6b0 2024-04-25T10:36:26.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.990+0000 7f48c996a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f1cf4108b0 con 0x55f1cf1274d0 2024-04-25T10:36:26.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.990+0000 7f48c996a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f1cf452d40 con 0x55f1cf150390 2024-04-25T10:36:26.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.990+0000 7f48bb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf12f6b0 0x55f1cf4fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.990+0000 7f48b97fa640 1 --2- 172.21.15.43:0/806082905 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f48a40afc60 0x7f48a40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:26.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.990+0000 7f48bb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf12f6b0 0x55f1cf4fd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38874/0 (socket says 172.21.15.43:38874) 2024-04-25T10:36:26.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.990+0000 7f48bb7fe640 1 -- 172.21.15.43:0/1380832209 learned_addr learned my addr 172.21.15.43:0/1380832209 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:26.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.990+0000 7f48bbfff640 1 --2- 172.21.15.43:0/1380832209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf1274d0 0x55f1cf4f0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.990+0000 7f48c0f80640 1 --2- 172.21.15.43:0/1380832209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1cf150390 0x55f1cf4f0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.990+0000 7f48c0f80640 1 --2- 172.21.15.43:0/806082905 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f48a40afc60 0x7f48a40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:26.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.990+0000 7f48b97fa640 1 -- 172.21.15.43:0/806082905 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f48ac0ff440 con 0x7f48bc103190 2024-04-25T10:36:26.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.990+0000 7f48bb7fe640 1 -- 172.21.15.43:0/1380832209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1cf150390 msgr2=0x55f1cf4f0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:27.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.990+0000 7f48bb7fe640 1 --2- 172.21.15.43:0/1380832209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1cf150390 0x55f1cf4f0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.990+0000 7f48bb7fe640 1 -- 172.21.15.43:0/1380832209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf1274d0 msgr2=0x55f1cf4f0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:27.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.990+0000 7f48bb7fe640 1 --2- 172.21.15.43:0/1380832209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf1274d0 0x55f1cf4f0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.991+0000 7f48bb7fe640 1 -- 172.21.15.43:0/1380832209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1cf3e1a40 con 0x55f1cf12f6b0 2024-04-25T10:36:27.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.991+0000 7f48bbfff640 1 --2- 172.21.15.43:0/1380832209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf1274d0 0x55f1cf4f0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:27.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.991+0000 7f48bb7fe640 1 --2- 172.21.15.43:0/1380832209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf12f6b0 0x55f1cf4fd8f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009b40 tx=0x7f48b000e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3641c78fd8231a94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:27.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.991+0000 7f48c0f80640 1 --2- 172.21.15.43:0/806082905 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f48a40afc60 0x7f48a40b2170 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55f1cf4edbd0 tx=0x7f48b4005e00 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:27.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.991+0000 7f48a27fc640 1 -- 172.21.15.43:0/1380832209 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0016070 con 0x55f1cf12f6b0 2024-04-25T10:36:27.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.991+0000 7f48a27fc640 1 -- 172.21.15.43:0/1380832209 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48b000ed20 con 0x55f1cf12f6b0 2024-04-25T10:36:27.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.991+0000 7f48a27fc640 1 -- 172.21.15.43:0/1380832209 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0010650 con 0x55f1cf12f6b0 2024-04-25T10:36:27.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.992+0000 7f48c996a740 1 -- 172.21.15.43:0/1380832209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf12f6b0 msgr2=0x55f1cf4fd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:27.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.992+0000 7f48c996a740 1 --2- 172.21.15.43:0/1380832209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf12f6b0 0x55f1cf4fd8f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009b40 tx=0x7f48b000e2f0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.992+0000 7f48c996a740 1 -- 172.21.15.43:0/1380832209 shutdown_connections 2024-04-25T10:36:27.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.992+0000 7f48c996a740 1 --2- 172.21.15.43:0/1380832209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf1274d0 0x55f1cf4f0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.992+0000 7f48c996a740 1 --2- 172.21.15.43:0/1380832209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf12f6b0 0x55f1cf4fd8f0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.992+0000 7f48c996a740 1 --2- 172.21.15.43:0/1380832209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1cf150390 0x55f1cf4f0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.992+0000 7f48c996a740 1 -- 172.21.15.43:0/1380832209 >> 172.21.15.43:0/1380832209 conn(0x55f1cf1444e0 msgr2=0x55f1cf4ed210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:27.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.992+0000 7f48c996a740 1 -- 172.21.15.43:0/1380832209 shutdown_connections 2024-04-25T10:36:27.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.992+0000 7f48c996a740 1 -- 172.21.15.43:0/1380832209 wait complete. 2024-04-25T10:36:27.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.992+0000 7f48c996a740 1 -- start start 2024-04-25T10:36:27.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.993+0000 7f48c996a740 1 Processor -- start 2024-04-25T10:36:27.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.993+0000 7f48c996a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf1274d0 0x55f1cf4fb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.993+0000 7f48c996a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf12f6b0 0x55f1cf4fbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.993+0000 7f48c996a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1cf150390 0x55f1cf501430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.993+0000 7f48c996a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f1cf4fdfe0 con 0x55f1cf12f6b0 2024-04-25T10:36:27.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.993+0000 7f48c996a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f1cf3ddd10 con 0x55f1cf1274d0 2024-04-25T10:36:27.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.993+0000 7f48c996a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f1cf4fe160 con 0x55f1cf150390 2024-04-25T10:36:27.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.993+0000 7f48c996a740 10 client.0 _ll_register_callbacks cb 0x55f1cf40a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:27.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.993+0000 7f48bb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf12f6b0 0x55f1cf4fbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.993+0000 7f48bb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf12f6b0 0x55f1cf4fbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38886/0 (socket says 172.21.15.43:38886) 2024-04-25T10:36:27.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.993+0000 7f48bb7fe640 1 -- 172.21.15.43:0/1958308579 learned_addr learned my addr 172.21.15.43:0/1958308579 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:27.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.993+0000 7f48bbfff640 1 --2- 172.21.15.43:0/1958308579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf1274d0 0x55f1cf4fb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.993+0000 7f48bb7fe640 1 -- 172.21.15.43:0/1958308579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1cf150390 msgr2=0x55f1cf501430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:27.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.993+0000 7f48bb7fe640 1 --2- 172.21.15.43:0/1958308579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1cf150390 0x55f1cf501430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.993+0000 7f48bb7fe640 1 -- 172.21.15.43:0/1958308579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf1274d0 msgr2=0x55f1cf4fb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:27.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.993+0000 7f48bb7fe640 1 --2- 172.21.15.43:0/1958308579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf1274d0 0x55f1cf4fb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.993+0000 7f48bb7fe640 1 -- 172.21.15.43:0/1958308579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1cf2327d0 con 0x55f1cf12f6b0 2024-04-25T10:36:27.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.993+0000 7f48bb7fe640 1 --2- 172.21.15.43:0/1958308579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf12f6b0 0x55f1cf4fbb40 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f48b0000c00 tx=0x7f48b0005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9caeba32b7c6aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:27.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.993+0000 7f489b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:27.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.994+0000 7f489b7fe640 1 -- 172.21.15.43:0/1958308579 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0016050 con 0x55f1cf12f6b0 2024-04-25T10:36:27.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.994+0000 7f489b7fe640 1 -- 172.21.15.43:0/1958308579 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f48b0011670 con 0x55f1cf12f6b0 2024-04-25T10:36:27.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.994+0000 7f48c996a740 1 -- 172.21.15.43:0/1958308579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48b000ed20 con 0x55f1cf12f6b0 2024-04-25T10:36:27.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.994+0000 7f489b7fe640 1 -- 172.21.15.43:0/1958308579 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f48b0011870 con 0x55f1cf12f6b0 2024-04-25T10:36:27.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.994+0000 7f48c996a740 10 client.16629 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:27.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.994+0000 7f48c996a740 1 -- 172.21.15.43:0/1958308579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f1cf1683f0 con 0x55f1cf12f6b0 2024-04-25T10:36:27.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.994+0000 7f489b7fe640 1 -- 172.21.15.43:0/1958308579 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48b000ed20 con 0x55f1cf12f6b0 2024-04-25T10:36:27.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.994+0000 7f489b7fe640 1 -- 172.21.15.43:0/1958308579 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f48b000f040 con 0x55f1cf12f6b0 2024-04-25T10:36:27.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.994+0000 7f48c996a740 10 client.16629 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:27.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.994+0000 7f48c996a740 10 client.16629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:27.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.994+0000 7f48c996a740 20 client.16629 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:27.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.994+0000 7f48c996a740 1 --2- 172.21.15.43:0/1958308579 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f1cf10ef60 0x55f1cf50d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.994+0000 7f48c996a740 4 client.16629 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:36:27.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.994+0000 7f48c996a740 1 -- 172.21.15.43:0/1958308579 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1cf464990 con 0x55f1cf10ef60 2024-04-25T10:36:27.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.994+0000 7f48bbfff640 1 --2- 172.21.15.43:0/1958308579 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f1cf10ef60 0x55f1cf50d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.995+0000 7f489b7fe640 10 client.16629 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:27.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.995+0000 7f48bbfff640 1 --2- 172.21.15.43:0/1958308579 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f1cf10ef60 0x55f1cf50d4f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6cb5d04c764e914 server_cookie=69da7a1de5a65b7f in_seq=0 out_seq=0 2024-04-25T10:36:27.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.995+0000 7f489b7fe640 1 -- 172.21.15.43:0/1958308579 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f1cf464990 con 0x55f1cf10ef60 2024-04-25T10:36:27.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.996+0000 7f489b7fe640 10 client.16629 handle_command_reply: tid=0 2024-04-25T10:36:27.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.996+0000 7f48c996a740 1 client.16629 shutdown 2024-04-25T10:36:27.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.996+0000 7f48c996a740 1 -- 172.21.15.43:0/1958308579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf12f6b0 msgr2=0x55f1cf4fbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:27.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.996+0000 7f48c996a740 1 --2- 172.21.15.43:0/1958308579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf12f6b0 0x55f1cf4fbb40 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f48b0000c00 tx=0x7f48b0005750 comp rx=0 tx=0).stop 2024-04-25T10:36:27.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.996+0000 7f48c996a740 1 -- 172.21.15.43:0/1958308579 shutdown_connections 2024-04-25T10:36:27.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.996+0000 7f48c996a740 1 --2- 172.21.15.43:0/1958308579 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f1cf10ef60 0x55f1cf50d4f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.996+0000 7f48c996a740 1 --2- 172.21.15.43:0/1958308579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1cf150390 0x55f1cf501430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.996+0000 7f48c996a740 1 --2- 172.21.15.43:0/1958308579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf12f6b0 0x55f1cf4fbb40 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.996+0000 7f489b7fe640 0 client.16629 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:27.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.996+0000 7f48c996a740 1 --2- 172.21.15.43:0/1958308579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf1274d0 0x55f1cf4fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.996+0000 7f48c996a740 1 -- 172.21.15.43:0/1958308579 >> 172.21.15.43:0/1958308579 conn(0x55f1cf1444e0 msgr2=0x55f1cf4f2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:27.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.996+0000 7f48c996a740 1 -- 172.21.15.43:0/1958308579 shutdown_connections 2024-04-25T10:36:27.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.996+0000 7f48c996a740 1 -- 172.21.15.43:0/1958308579 wait complete. 2024-04-25T10:36:27.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:26.996+0000 7f48c996a740 20 client.16629 trim_cache size 0 max 16384 2024-04-25T10:36:27.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.014+0000 7f48c996a740 1 Processor -- start 2024-04-25T10:36:27.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.014+0000 7f48c996a740 1 -- start start 2024-04-25T10:36:27.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.014+0000 7f48c996a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1cf1274d0 0x55f1cf4f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.014+0000 7f48c996a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf12f6b0 0x55f1cf4faba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.014+0000 7f48c996a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf150390 0x55f1cf501420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.014+0000 7f48c996a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc1b0070 con 0x55f1cf150390 2024-04-25T10:36:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.014+0000 7f48c996a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc1ad940 con 0x55f1cf12f6b0 2024-04-25T10:36:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.014+0000 7f48c996a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc1ab7b0 con 0x55f1cf1274d0 2024-04-25T10:36:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.014+0000 7f48c0f80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf150390 0x55f1cf501420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.014+0000 7f48c0f80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf150390 0x55f1cf501420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38888/0 (socket says 172.21.15.43:38888) 2024-04-25T10:36:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.014+0000 7f48c0f80640 1 -- 172.21.15.43:0/2751746973 learned_addr learned my addr 172.21.15.43:0/2751746973 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.014+0000 7f48bbfff640 1 --2- 172.21.15.43:0/2751746973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1cf1274d0 0x55f1cf4f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.015+0000 7f48bb7fe640 1 --2- 172.21.15.43:0/2751746973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf12f6b0 0x55f1cf4faba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.015+0000 7f48c0f80640 1 -- 172.21.15.43:0/2751746973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1cf1274d0 msgr2=0x55f1cf4f2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.015+0000 7f48c0f80640 1 --2- 172.21.15.43:0/2751746973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1cf1274d0 0x55f1cf4f2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.015+0000 7f48c0f80640 1 -- 172.21.15.43:0/2751746973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf12f6b0 msgr2=0x55f1cf4faba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.015+0000 7f48c0f80640 1 --2- 172.21.15.43:0/2751746973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf12f6b0 0x55f1cf4faba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.015+0000 7f48c0f80640 1 -- 172.21.15.43:0/2751746973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1cf1280f0 con 0x55f1cf150390 2024-04-25T10:36:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.015+0000 7f48c0f80640 1 --2- 172.21.15.43:0/2751746973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf150390 0x55f1cf501420 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009950 tx=0x7f48b4005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a575e2352a7b20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.015+0000 7f48a1ffb640 1 -- 172.21.15.43:0/2751746973 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b402d070 con 0x55f1cf150390 2024-04-25T10:36:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.015+0000 7f48a1ffb640 1 -- 172.21.15.43:0/2751746973 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48b4019c50 con 0x55f1cf150390 2024-04-25T10:36:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.015+0000 7f48a1ffb640 1 -- 172.21.15.43:0/2751746973 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b4028840 con 0x55f1cf150390 2024-04-25T10:36:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.016+0000 7f48c996a740 1 -- 172.21.15.43:0/2751746973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf150390 msgr2=0x55f1cf501420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.016+0000 7f48c996a740 1 --2- 172.21.15.43:0/2751746973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf150390 0x55f1cf501420 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009950 tx=0x7f48b4005d90 comp rx=0 tx=0).stop 2024-04-25T10:36:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.016+0000 7f48c996a740 1 -- 172.21.15.43:0/2751746973 shutdown_connections 2024-04-25T10:36:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.016+0000 7f48c996a740 1 --2- 172.21.15.43:0/2751746973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf12f6b0 0x55f1cf4faba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.016+0000 7f48c996a740 1 --2- 172.21.15.43:0/2751746973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf150390 0x55f1cf501420 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.016+0000 7f48c996a740 1 --2- 172.21.15.43:0/2751746973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1cf1274d0 0x55f1cf4f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.016+0000 7f48c996a740 1 -- 172.21.15.43:0/2751746973 >> 172.21.15.43:0/2751746973 conn(0x55f1cf1444e0 msgr2=0x55f1cf4f2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.016+0000 7f48c996a740 1 -- 172.21.15.43:0/2751746973 shutdown_connections 2024-04-25T10:36:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.016+0000 7f48c996a740 1 -- 172.21.15.43:0/2751746973 wait complete. 2024-04-25T10:36:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.016+0000 7f48c996a740 1 -- start start 2024-04-25T10:36:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.016+0000 7f48c996a740 1 Processor -- start 2024-04-25T10:36:27.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.017+0000 7f48c996a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf1274d0 0x55f1cf5024a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.017+0000 7f48c996a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1cf12f6b0 0x55f1cf5029e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.017+0000 7f48c996a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf150390 0x55f1cf502f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.017+0000 7f48c996a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc1abe00 con 0x55f1cf150390 2024-04-25T10:36:27.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.017+0000 7f48c996a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc1ad270 con 0x55f1cf1274d0 2024-04-25T10:36:27.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.017+0000 7f48c996a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc1ac4d0 con 0x55f1cf12f6b0 2024-04-25T10:36:27.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.017+0000 7f48c996a740 10 client.0 _ll_register_callbacks cb 0x7f48b4010df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:27.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.017+0000 7f48c0f80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf150390 0x55f1cf502f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.017+0000 7f48c0f80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf150390 0x55f1cf502f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38890/0 (socket says 172.21.15.43:38890) 2024-04-25T10:36:27.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.017+0000 7f48c0f80640 1 -- 172.21.15.43:0/1286859102 learned_addr learned my addr 172.21.15.43:0/1286859102 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:27.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.017+0000 7f48bbfff640 1 --2- 172.21.15.43:0/1286859102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf1274d0 0x55f1cf5024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.017+0000 7f48bb7fe640 1 --2- 172.21.15.43:0/1286859102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1cf12f6b0 0x55f1cf5029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.017+0000 7f48c0f80640 1 -- 172.21.15.43:0/1286859102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1cf12f6b0 msgr2=0x55f1cf5029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:27.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.017+0000 7f48c0f80640 1 --2- 172.21.15.43:0/1286859102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1cf12f6b0 0x55f1cf5029e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.017+0000 7f48c0f80640 1 -- 172.21.15.43:0/1286859102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf1274d0 msgr2=0x55f1cf5024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:27.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.017+0000 7f48c0f80640 1 --2- 172.21.15.43:0/1286859102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf1274d0 0x55f1cf5024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.017+0000 7f48c0f80640 1 -- 172.21.15.43:0/1286859102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1cf4174b0 con 0x55f1cf150390 2024-04-25T10:36:27.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.017+0000 7f48c0f80640 1 --2- 172.21.15.43:0/1286859102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf150390 0x55f1cf502f20 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f48b400a010 tx=0x7f48b4009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff02dea42ec660c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:27.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.017+0000 7f489b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:27.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.018+0000 7f489b7fe640 1 -- 172.21.15.43:0/1286859102 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b402d040 con 0x55f1cf150390 2024-04-25T10:36:27.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.018+0000 7f489b7fe640 1 -- 172.21.15.43:0/1286859102 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f48b4023040 con 0x55f1cf150390 2024-04-25T10:36:27.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.018+0000 7f48c996a740 1 -- 172.21.15.43:0/1286859102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48b4019c50 con 0x55f1cf150390 2024-04-25T10:36:27.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.018+0000 7f489b7fe640 1 -- 172.21.15.43:0/1286859102 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f48b4019e00 con 0x55f1cf150390 2024-04-25T10:36:27.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.018+0000 7f48c996a740 10 client.16635 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.018+0000 7f48c996a740 1 -- 172.21.15.43:0/1286859102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f1cf5747e0 con 0x55f1cf150390 2024-04-25T10:36:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.018+0000 7f489b7fe640 1 -- 172.21.15.43:0/1286859102 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48b4019c50 con 0x55f1cf150390 2024-04-25T10:36:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.018+0000 7f489b7fe640 1 -- 172.21.15.43:0/1286859102 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f48b401c070 con 0x55f1cf150390 2024-04-25T10:36:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.018+0000 7f48c996a740 10 client.16635 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.018+0000 7f48c996a740 10 client.16635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.018+0000 7f48c996a740 20 client.16635 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.018+0000 7f48c996a740 1 --2- 172.21.15.43:0/1286859102 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f1cf0e43a0 0x55f1cf576e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.018+0000 7f48c996a740 4 client.16635 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:36:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.018+0000 7f48c996a740 1 -- 172.21.15.43:0/1286859102 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f1cf45e800 con 0x55f1cf0e43a0 2024-04-25T10:36:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.018+0000 7f48bbfff640 1 --2- 172.21.15.43:0/1286859102 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f1cf0e43a0 0x55f1cf576e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.019+0000 7f489b7fe640 10 client.16635 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.019+0000 7f48bbfff640 1 --2- 172.21.15.43:0/1286859102 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f1cf0e43a0 0x55f1cf576e10 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68e132480dd514a3 server_cookie=51769c6e3e571d9a in_seq=0 out_seq=0 2024-04-25T10:36:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.019+0000 7f489b7fe640 1 -- 172.21.15.43:0/1286859102 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f1cf45e800 con 0x55f1cf0e43a0 2024-04-25T10:36:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.019+0000 7f489b7fe640 10 client.16635 handle_command_reply: tid=0 2024-04-25T10:36:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.019+0000 7f48c996a740 1 client.16635 shutdown 2024-04-25T10:36:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.019+0000 7f48c996a740 1 -- 172.21.15.43:0/1286859102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf150390 msgr2=0x55f1cf502f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.019+0000 7f48c996a740 1 --2- 172.21.15.43:0/1286859102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf150390 0x55f1cf502f20 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f48b400a010 tx=0x7f48b4009880 comp rx=0 tx=0).stop 2024-04-25T10:36:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.019+0000 7f48c996a740 1 -- 172.21.15.43:0/1286859102 shutdown_connections 2024-04-25T10:36:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.019+0000 7f48c996a740 1 --2- 172.21.15.43:0/1286859102 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f1cf0e43a0 0x55f1cf576e10 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.019+0000 7f48c996a740 1 --2- 172.21.15.43:0/1286859102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1cf150390 0x55f1cf502f20 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.019+0000 7f48c996a740 1 --2- 172.21.15.43:0/1286859102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1cf12f6b0 0x55f1cf5029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.019+0000 7f48c996a740 1 --2- 172.21.15.43:0/1286859102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1cf1274d0 0x55f1cf5024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.019+0000 7f489b7fe640 0 client.16635 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.019+0000 7f48c996a740 1 -- 172.21.15.43:0/1286859102 >> 172.21.15.43:0/1286859102 conn(0x55f1cf1444e0 msgr2=0x55f1cf4fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.019+0000 7f48c996a740 1 -- 172.21.15.43:0/1286859102 shutdown_connections 2024-04-25T10:36:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.020+0000 7f48c996a740 1 -- 172.21.15.43:0/1286859102 wait complete. 2024-04-25T10:36:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.020+0000 7f48c996a740 20 client.16635 trim_cache size 0 max 16384 2024-04-25T10:36:27.028 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:36:27.028 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:36:27.028 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:36:27.028 INFO:teuthology.orchestra.run.smithi043.stdout: "c8d847ea-851f-443a-97b4-c241830dda8e": { 2024-04-25T10:36:27.028 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:36:27.028 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "c8d847ea-851f-443a-97b4-c241830dda8e", 2024-04-25T10:36:27.028 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:36:27.028 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:36:27.028 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:36:27.028 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:36:27.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.020+0000 7f48c29e3640 1 -- 172.21.15.43:0/806082905 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f48a40afc60 msgr2=0x7f48a40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:27.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.020+0000 7f48c29e3640 1 --2- 172.21.15.43:0/806082905 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f48a40afc60 0x7f48a40b2170 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55f1cf4edbd0 tx=0x7f48b4005e00 comp rx=0 tx=0).stop 2024-04-25T10:36:27.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.020+0000 7f48c29e3640 1 -- 172.21.15.43:0/806082905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48bc103190 msgr2=0x7f48bc19a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:27.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.020+0000 7f48c29e3640 1 --2- 172.21.15.43:0/806082905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48bc103190 0x7f48bc19a7a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f48ac02a930 tx=0x7f48ac004270 comp rx=0 tx=0).stop 2024-04-25T10:36:27.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.020+0000 7f48c29e3640 1 -- 172.21.15.43:0/806082905 shutdown_connections 2024-04-25T10:36:27.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.020+0000 7f48c29e3640 1 --2- 172.21.15.43:0/806082905 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f48a40afc60 0x7f48a40b2170 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.020+0000 7f48c29e3640 1 --2- 172.21.15.43:0/806082905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48bc108a30 0x7f48bc19ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.020+0000 7f48c29e3640 1 --2- 172.21.15.43:0/806082905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48bc108ed0 0x7f48bc19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.020+0000 7f48c29e3640 1 --2- 172.21.15.43:0/806082905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48bc103190 0x7f48bc19a7a0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.020+0000 7f48c29e3640 1 -- 172.21.15.43:0/806082905 >> 172.21.15.43:0/806082905 conn(0x7f48bc0fe260 msgr2=0x7f48bc074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:27.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.020+0000 7f48c29e3640 1 -- 172.21.15.43:0/806082905 shutdown_connections 2024-04-25T10:36:27.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:27.020+0000 7f48c29e3640 1 -- 172.21.15.43:0/806082905 wait complete. 2024-04-25T10:36:27.033 INFO:tasks.cephfs.filesystem:scrub status for tag:c8d847ea-851f-443a-97b4-c241830dda8e - {'path': '/', 'tag': 'c8d847ea-851f-443a-97b4-c241830dda8e', 'options': 'recursive,force'} 2024-04-25T10:36:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.418+0000 7f11e8430640 1 -- 172.21.15.43:0/2335220736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11e010a000 msgr2=0x7f11e0103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.418+0000 7f11e8430640 1 --2- 172.21.15.43:0/2335220736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11e010a000 0x7f11e0103c40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009900 tx=0x7f11d002a410 comp rx=0 tx=0).stop 2024-04-25T10:36:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.419+0000 7f11e8430640 1 -- 172.21.15.43:0/2335220736 shutdown_connections 2024-04-25T10:36:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.419+0000 7f11e8430640 1 --2- 172.21.15.43:0/2335220736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11e01042b0 0x7f11e0110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.419+0000 7f11e8430640 1 --2- 172.21.15.43:0/2335220736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11e010a000 0x7f11e0103c40 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.419+0000 7f11e8430640 1 --2- 172.21.15.43:0/2335220736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11e0109b60 0x7f11e0103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.419+0000 7f11e8430640 1 -- 172.21.15.43:0/2335220736 >> 172.21.15.43:0/2335220736 conn(0x7f11e0079170 msgr2=0x7f11e00795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.419+0000 7f11e8430640 1 -- 172.21.15.43:0/2335220736 shutdown_connections 2024-04-25T10:36:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.419+0000 7f11e8430640 1 -- 172.21.15.43:0/2335220736 wait complete. 2024-04-25T10:36:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.419+0000 7f11e8430640 1 Processor -- start 2024-04-25T10:36:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.419+0000 7f11e8430640 1 -- start start 2024-04-25T10:36:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.419+0000 7f11e8430640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11e01042b0 0x7f11e019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.420+0000 7f11e8430640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11e0109b60 0x7f11e019acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.420+0000 7f11e8430640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11e010a000 0x7f11e019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.420+0000 7f11e8430640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11e0113d50 con 0x7f11e010a000 2024-04-25T10:36:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.420+0000 7f11e8430640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f11e0113bd0 con 0x7f11e01042b0 2024-04-25T10:36:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.420+0000 7f11e8430640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f11e0113ed0 con 0x7f11e0109b60 2024-04-25T10:36:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.420+0000 7f11e69cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11e010a000 0x7f11e019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.420+0000 7f11e69cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11e010a000 0x7f11e019efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38906/0 (socket says 172.21.15.43:38906) 2024-04-25T10:36:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.420+0000 7f11e69cd640 1 -- 172.21.15.43:0/617880008 learned_addr learned my addr 172.21.15.43:0/617880008 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.420+0000 7f11e61cc640 1 --2- 172.21.15.43:0/617880008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11e01042b0 0x7f11e019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.420+0000 7f11e59cb640 1 --2- 172.21.15.43:0/617880008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11e0109b60 0x7f11e019acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.420+0000 7f11e69cd640 1 -- 172.21.15.43:0/617880008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11e0109b60 msgr2=0x7f11e019acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.420+0000 7f11e69cd640 1 --2- 172.21.15.43:0/617880008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11e0109b60 0x7f11e019acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.420+0000 7f11e69cd640 1 -- 172.21.15.43:0/617880008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11e01042b0 msgr2=0x7f11e019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.420+0000 7f11e69cd640 1 --2- 172.21.15.43:0/617880008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11e01042b0 0x7f11e019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.420+0000 7f11e69cd640 1 -- 172.21.15.43:0/617880008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e019f570 con 0x7f11e010a000 2024-04-25T10:36:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.420+0000 7f11e69cd640 1 --2- 172.21.15.43:0/617880008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11e010a000 0x7f11e019efb0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f11d800ead0 tx=0x7f11d800eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.420+0000 7f11d77fe640 1 -- 172.21.15.43:0/617880008 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8004d20 con 0x7f11e010a000 2024-04-25T10:36:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.420+0000 7f11d77fe640 1 -- 172.21.15.43:0/617880008 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d8005020 con 0x7f11e010a000 2024-04-25T10:36:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.420+0000 7f11e8430640 1 -- 172.21.15.43:0/617880008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e019f860 con 0x7f11e010a000 2024-04-25T10:36:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.420+0000 7f11d77fe640 1 -- 172.21.15.43:0/617880008 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d80116d0 con 0x7f11e010a000 2024-04-25T10:36:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.421+0000 7f11e8430640 1 -- 172.21.15.43:0/617880008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e019fd80 con 0x7f11e010a000 2024-04-25T10:36:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.423+0000 7f11ef3b2740 1 Processor -- start 2024-04-25T10:36:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.423+0000 7f11d77fe640 1 -- 172.21.15.43:0/617880008 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f11d80051d0 con 0x7f11e010a000 2024-04-25T10:36:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.423+0000 7f11ef3b2740 1 -- start start 2024-04-25T10:36:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.424+0000 7f11ef3b2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924fb1b0 0x55f49289a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.424+0000 7f11ef3b2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4924d22f0 0x55f49289aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.424+0000 7f11ef3b2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4924da4d0 0x55f49289b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.424+0000 7f11ef3b2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f492801460 con 0x55f4924fb1b0 2024-04-25T10:36:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.424+0000 7f11ef3b2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f4927bb590 con 0x55f4924da4d0 2024-04-25T10:36:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.424+0000 7f11ef3b2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f4927fd420 con 0x55f4924d22f0 2024-04-25T10:36:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.424+0000 7f11e59cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924fb1b0 0x55f49289a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.424+0000 7f11e59cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924fb1b0 0x55f49289a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38914/0 (socket says 172.21.15.43:38914) 2024-04-25T10:36:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.424+0000 7f11e59cb640 1 -- 172.21.15.43:0/3578495739 learned_addr learned my addr 172.21.15.43:0/3578495739 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.424+0000 7f11e61cc640 1 --2- 172.21.15.43:0/3578495739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4924d22f0 0x55f49289aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.424+0000 7f11d77fe640 1 --2- 172.21.15.43:0/617880008 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f11cc0afcd0 0x7f11cc0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.424+0000 7f11e59cb640 1 --2- 172.21.15.43:0/3578495739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4924da4d0 0x55f49289b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.424+0000 7f11e59cb640 1 -- 172.21.15.43:0/3578495739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4924d22f0 msgr2=0x55f49289aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.424+0000 7f11e59cb640 1 --2- 172.21.15.43:0/3578495739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4924d22f0 0x55f49289aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.424+0000 7f11e69cd640 1 --2- 172.21.15.43:0/617880008 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f11cc0afcd0 0x7f11cc0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.424+0000 7f11e59cb640 1 -- 172.21.15.43:0/3578495739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4924da4d0 msgr2=0x55f49289b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.424+0000 7f11e59cb640 1 --2- 172.21.15.43:0/3578495739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4924da4d0 0x55f49289b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.424+0000 7f11e59cb640 1 -- 172.21.15.43:0/3578495739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4927c80d0 con 0x55f4924fb1b0 2024-04-25T10:36:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.424+0000 7f11d77fe640 1 -- 172.21.15.43:0/617880008 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f11d80df1f0 con 0x7f11e010a000 2024-04-25T10:36:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.425+0000 7f11e59cb640 1 --2- 172.21.15.43:0/3578495739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924fb1b0 0x55f49289a980 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f11d0005ab0 tx=0x7f11d0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39efadf2e54c5ff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.425+0000 7f11d4bfe640 1 -- 172.21.15.43:0/3578495739 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0035050 con 0x55f4924fb1b0 2024-04-25T10:36:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.425+0000 7f11e69cd640 1 --2- 172.21.15.43:0/617880008 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f11cc0afcd0 0x7f11cc0b21e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f11d80a45d0 tx=0x7f11d8005890 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.425+0000 7f11d4bfe640 1 -- 172.21.15.43:0/3578495739 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d00316f0 con 0x55f4924fb1b0 2024-04-25T10:36:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.425+0000 7f11d4bfe640 1 -- 172.21.15.43:0/3578495739 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d00046c0 con 0x55f4924fb1b0 2024-04-25T10:36:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.426+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3578495739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924fb1b0 msgr2=0x55f49289a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.426+0000 7f11ef3b2740 1 --2- 172.21.15.43:0/3578495739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924fb1b0 0x55f49289a980 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f11d0005ab0 tx=0x7f11d0005a20 comp rx=0 tx=0).stop 2024-04-25T10:36:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.426+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3578495739 shutdown_connections 2024-04-25T10:36:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.426+0000 7f11ef3b2740 1 --2- 172.21.15.43:0/3578495739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4924da4d0 0x55f49289b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.426+0000 7f11ef3b2740 1 --2- 172.21.15.43:0/3578495739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4924d22f0 0x55f49289aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.426+0000 7f11ef3b2740 1 --2- 172.21.15.43:0/3578495739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924fb1b0 0x55f49289a980 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.426+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3578495739 >> 172.21.15.43:0/3578495739 conn(0x55f4924ef300 msgr2=0x55f4928972a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.426+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3578495739 shutdown_connections 2024-04-25T10:36:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.426+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3578495739 wait complete. 2024-04-25T10:36:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.426+0000 7f11ef3b2740 1 -- start start 2024-04-25T10:36:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.426+0000 7f11ef3b2740 1 Processor -- start 2024-04-25T10:36:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11ef3b2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 0x55f4928a95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11ef3b2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4924da4d0 0x55f4928a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11ef3b2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4924fb1b0 0x55f4928af550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11ef3b2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f4928ac030 con 0x55f4924d22f0 2024-04-25T10:36:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11ef3b2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f492788b60 con 0x55f4924da4d0 2024-04-25T10:36:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11ef3b2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f4928ac1b0 con 0x55f4924fb1b0 2024-04-25T10:36:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11ef3b2740 10 client.0 _ll_register_callbacks cb 0x55f4927ad2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11e59cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 0x55f4928a95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11e59cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 0x55f4928a95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38920/0 (socket says 172.21.15.43:38920) 2024-04-25T10:36:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11e59cb640 1 -- 172.21.15.43:0/3360902306 learned_addr learned my addr 172.21.15.43:0/3360902306 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11e59cb640 1 --2- 172.21.15.43:0/3360902306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4924fb1b0 0x55f4928af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11e61cc640 1 --2- 172.21.15.43:0/3360902306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4924da4d0 0x55f4928a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11e59cb640 1 -- 172.21.15.43:0/3360902306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4924fb1b0 msgr2=0x55f4928af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11e59cb640 1 --2- 172.21.15.43:0/3360902306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4924fb1b0 0x55f4928af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11e59cb640 1 -- 172.21.15.43:0/3360902306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4924da4d0 msgr2=0x55f4928a9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11e59cb640 1 --2- 172.21.15.43:0/3360902306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4924da4d0 0x55f4928a9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11e59cb640 1 -- 172.21.15.43:0/3360902306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f492513210 con 0x55f4924d22f0 2024-04-25T10:36:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11e59cb640 1 --2- 172.21.15.43:0/3360902306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 0x55f4928a95b0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f11d0031440 tx=0x7f11d0009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe46331eec777aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11c9ffb640 1 -- 172.21.15.43:0/3360902306 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0051060 con 0x55f4924d22f0 2024-04-25T10:36:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11c9ffb640 1 -- 172.21.15.43:0/3360902306 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f11d00035f0 con 0x55f4924d22f0 2024-04-25T10:36:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.427+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3360902306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d00316f0 con 0x55f4924d22f0 2024-04-25T10:36:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.428+0000 7f11c9ffb640 1 -- 172.21.15.43:0/3360902306 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11d0035040 con 0x55f4924d22f0 2024-04-25T10:36:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.428+0000 7f11ef3b2740 10 client.16647 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.428+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3360902306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f4925df620 con 0x55f4924d22f0 2024-04-25T10:36:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.428+0000 7f11c9ffb640 1 -- 172.21.15.43:0/3360902306 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d00316f0 con 0x55f4924d22f0 2024-04-25T10:36:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.428+0000 7f11c9ffb640 1 -- 172.21.15.43:0/3360902306 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f11d00567f0 con 0x55f4924d22f0 2024-04-25T10:36:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.428+0000 7f11ef3b2740 10 client.16647 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.428+0000 7f11ef3b2740 10 client.16647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.428+0000 7f11ef3b2740 20 client.16647 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.428+0000 7f11ef3b2740 1 --2- 172.21.15.43:0/3360902306 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f4924b9d80 0x55f4928bb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.428+0000 7f11ef3b2740 4 client.16647 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:36:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.428+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3360902306 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f49280f070 con 0x55f4924b9d80 2024-04-25T10:36:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.428+0000 7f11e61cc640 1 --2- 172.21.15.43:0/3360902306 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f4924b9d80 0x55f4928bb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.429+0000 7f11c9ffb640 10 client.16647 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.429+0000 7f11e61cc640 1 --2- 172.21.15.43:0/3360902306 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f4924b9d80 0x55f4928bb990 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fec0eafddc07bedf server_cookie=c09f8c4052e6a092 in_seq=0 out_seq=0 2024-04-25T10:36:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.430+0000 7f11c9ffb640 1 -- 172.21.15.43:0/3360902306 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f49280f070 con 0x55f4924b9d80 2024-04-25T10:36:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.430+0000 7f11c9ffb640 10 client.16647 handle_command_reply: tid=0 2024-04-25T10:36:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.430+0000 7f11ef3b2740 1 client.16647 shutdown 2024-04-25T10:36:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.430+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3360902306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 msgr2=0x55f4928a95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.430+0000 7f11ef3b2740 1 --2- 172.21.15.43:0/3360902306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 0x55f4928a95b0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f11d0031440 tx=0x7f11d0009640 comp rx=0 tx=0).stop 2024-04-25T10:36:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.430+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3360902306 shutdown_connections 2024-04-25T10:36:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.430+0000 7f11ef3b2740 1 --2- 172.21.15.43:0/3360902306 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f4924b9d80 0x55f4928bb990 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.430+0000 7f11ef3b2740 1 --2- 172.21.15.43:0/3360902306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4924da4d0 0x55f4928a9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.430+0000 7f11c9ffb640 0 client.16647 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.430+0000 7f11ef3b2740 1 --2- 172.21.15.43:0/3360902306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4924fb1b0 0x55f4928af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.430+0000 7f11ef3b2740 1 --2- 172.21.15.43:0/3360902306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 0x55f4928a95b0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.430+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3360902306 >> 172.21.15.43:0/3360902306 conn(0x55f4924ef300 msgr2=0x55f49289c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.430+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3360902306 shutdown_connections 2024-04-25T10:36:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.430+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3360902306 wait complete. 2024-04-25T10:36:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.430+0000 7f11ef3b2740 20 client.16647 trim_cache size 0 max 16384 2024-04-25T10:36:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.446+0000 7f11ef3b2740 1 Processor -- start 2024-04-25T10:36:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.446+0000 7f11ef3b2740 1 -- start start 2024-04-25T10:36:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.446+0000 7f11ef3b2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 0x55f4928af450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.446+0000 7f11ef3b2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4924da4d0 0x55f4928a82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.446+0000 7f11ef3b2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4924fb1b0 0x55f4928a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.446+0000 7f11ef3b2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01aff00 con 0x55f4924d22f0 2024-04-25T10:36:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.446+0000 7f11ef3b2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01ad7d0 con 0x55f4924fb1b0 2024-04-25T10:36:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.446+0000 7f11ef3b2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01ab640 con 0x55f4924da4d0 2024-04-25T10:36:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.446+0000 7f11e59cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 0x55f4928af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.446+0000 7f11e59cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 0x55f4928af450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38926/0 (socket says 172.21.15.43:38926) 2024-04-25T10:36:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.446+0000 7f11e59cb640 1 -- 172.21.15.43:0/3273504232 learned_addr learned my addr 172.21.15.43:0/3273504232 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.446+0000 7f11e59cb640 1 -- 172.21.15.43:0/3273504232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4924da4d0 msgr2=0x55f4928a82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.446+0000 7f11e59cb640 1 --2- 172.21.15.43:0/3273504232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4924da4d0 0x55f4928a82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.446+0000 7f11e59cb640 1 -- 172.21.15.43:0/3273504232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4924fb1b0 msgr2=0x55f4928a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.446+0000 7f11e59cb640 1 --2- 172.21.15.43:0/3273504232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4924fb1b0 0x55f4928a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.446+0000 7f11e59cb640 1 -- 172.21.15.43:0/3273504232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f49289be80 con 0x55f4924d22f0 2024-04-25T10:36:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.446+0000 7f11e59cb640 1 --2- 172.21.15.43:0/3273504232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 0x55f4928af450 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f11d0005f70 tx=0x7f11d002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ed0298e8d86afd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.447+0000 7f11cbfff640 1 -- 172.21.15.43:0/3273504232 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0051040 con 0x55f4924d22f0 2024-04-25T10:36:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.447+0000 7f11cbfff640 1 -- 172.21.15.43:0/3273504232 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d0031dc0 con 0x55f4924d22f0 2024-04-25T10:36:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.447+0000 7f11cbfff640 1 -- 172.21.15.43:0/3273504232 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0056680 con 0x55f4924d22f0 2024-04-25T10:36:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.447+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3273504232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 msgr2=0x55f4928af450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.447+0000 7f11ef3b2740 1 --2- 172.21.15.43:0/3273504232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 0x55f4928af450 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f11d0005f70 tx=0x7f11d002a410 comp rx=0 tx=0).stop 2024-04-25T10:36:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.447+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3273504232 shutdown_connections 2024-04-25T10:36:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.447+0000 7f11ef3b2740 1 --2- 172.21.15.43:0/3273504232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4924fb1b0 0x55f4928a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.447+0000 7f11ef3b2740 1 --2- 172.21.15.43:0/3273504232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4924da4d0 0x55f4928a82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.447+0000 7f11ef3b2740 1 --2- 172.21.15.43:0/3273504232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 0x55f4928af450 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.447+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3273504232 >> 172.21.15.43:0/3273504232 conn(0x55f4924ef300 msgr2=0x55f49289b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.447+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3273504232 shutdown_connections 2024-04-25T10:36:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.447+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3273504232 wait complete. 2024-04-25T10:36:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.448+0000 7f11ef3b2740 1 -- start start 2024-04-25T10:36:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.448+0000 7f11ef3b2740 1 Processor -- start 2024-04-25T10:36:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.448+0000 7f11ef3b2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 0x55f4928b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.448+0000 7f11ef3b2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4924da4d0 0x55f4928b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.448+0000 7f11ef3b2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4924fb1b0 0x55f492921ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.448+0000 7f11ef3b2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01abc90 con 0x55f4924d22f0 2024-04-25T10:36:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.448+0000 7f11ef3b2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01ad100 con 0x55f4924fb1b0 2024-04-25T10:36:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.448+0000 7f11ef3b2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01ac360 con 0x55f4924da4d0 2024-04-25T10:36:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.448+0000 7f11ef3b2740 10 client.0 _ll_register_callbacks cb 0x7f11dc00a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.448+0000 7f11e59cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 0x55f4928b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.448+0000 7f11e59cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 0x55f4928b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38930/0 (socket says 172.21.15.43:38930) 2024-04-25T10:36:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.448+0000 7f11e59cb640 1 -- 172.21.15.43:0/3421123922 learned_addr learned my addr 172.21.15.43:0/3421123922 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.448+0000 7f11e61cc640 1 --2- 172.21.15.43:0/3421123922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4924da4d0 0x55f4928b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.448+0000 7f11e59cb640 1 --2- 172.21.15.43:0/3421123922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4924fb1b0 0x55f492921ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.448+0000 7f11e59cb640 1 -- 172.21.15.43:0/3421123922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4924da4d0 msgr2=0x55f4928b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.448+0000 7f11e59cb640 1 --2- 172.21.15.43:0/3421123922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4924da4d0 0x55f4928b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.448+0000 7f11e59cb640 1 -- 172.21.15.43:0/3421123922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4924fb1b0 msgr2=0x55f492921ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.448+0000 7f11e59cb640 1 --2- 172.21.15.43:0/3421123922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4924fb1b0 0x55f492921ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.448+0000 7f11e59cb640 1 -- 172.21.15.43:0/3421123922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f492921ff0 con 0x55f4924d22f0 2024-04-25T10:36:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.449+0000 7f11e59cb640 1 --2- 172.21.15.43:0/3421123922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 0x55f4928b31e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f11d0031d90 tx=0x7f11d0057f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40c8752b4e200fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.449+0000 7f11c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.449+0000 7f11c9ffb640 1 -- 172.21.15.43:0/3421123922 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0066060 con 0x55f4924d22f0 2024-04-25T10:36:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.449+0000 7f11c9ffb640 1 -- 172.21.15.43:0/3421123922 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f11d0056690 con 0x55f4924d22f0 2024-04-25T10:36:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.449+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3421123922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d0031dc0 con 0x55f4924d22f0 2024-04-25T10:36:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.449+0000 7f11c9ffb640 1 -- 172.21.15.43:0/3421123922 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11d0051040 con 0x55f4924d22f0 2024-04-25T10:36:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.449+0000 7f11ef3b2740 10 client.16653 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.449+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3421123922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f4929231d0 con 0x55f4924d22f0 2024-04-25T10:36:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.449+0000 7f11c9ffb640 1 -- 172.21.15.43:0/3421123922 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d0031dc0 con 0x55f4924d22f0 2024-04-25T10:36:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.449+0000 7f11c9ffb640 1 -- 172.21.15.43:0/3421123922 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f11d005e070 con 0x55f4924d22f0 2024-04-25T10:36:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.449+0000 7f11ef3b2740 10 client.16653 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.449+0000 7f11ef3b2740 10 client.16653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.449+0000 7f11ef3b2740 20 client.16653 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.449+0000 7f11ef3b2740 1 --2- 172.21.15.43:0/3421123922 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f49248f1c0 0x55f492925740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.449+0000 7f11ef3b2740 4 client.16653 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:36:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.449+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3421123922 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f492808ee0 con 0x55f49248f1c0 2024-04-25T10:36:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.450+0000 7f11e61cc640 1 --2- 172.21.15.43:0/3421123922 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f49248f1c0 0x55f492925740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.450+0000 7f11e61cc640 1 --2- 172.21.15.43:0/3421123922 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f49248f1c0 0x55f492925740 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f6c500c0246aff2 server_cookie=f05b53e23656ae02 in_seq=0 out_seq=0 2024-04-25T10:36:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.450+0000 7f11c9ffb640 10 client.16653 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.466+0000 7f11c9ffb640 1 -- 172.21.15.43:0/3421123922 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f492808ee0 con 0x55f49248f1c0 2024-04-25T10:36:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.466+0000 7f11c9ffb640 10 client.16653 handle_command_reply: tid=0 2024-04-25T10:36:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.466+0000 7f11ef3b2740 1 client.16653 shutdown 2024-04-25T10:36:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.466+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3421123922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 msgr2=0x55f4928b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.466+0000 7f11ef3b2740 1 --2- 172.21.15.43:0/3421123922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 0x55f4928b31e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f11d0031d90 tx=0x7f11d0057f90 comp rx=0 tx=0).stop 2024-04-25T10:36:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.466+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3421123922 shutdown_connections 2024-04-25T10:36:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.466+0000 7f11ef3b2740 1 --2- 172.21.15.43:0/3421123922 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f49248f1c0 0x55f492925740 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.466+0000 7f11ef3b2740 1 --2- 172.21.15.43:0/3421123922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4924fb1b0 0x55f492921ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.466+0000 7f11ef3b2740 1 --2- 172.21.15.43:0/3421123922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4924da4d0 0x55f4928b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.466+0000 7f11ef3b2740 1 --2- 172.21.15.43:0/3421123922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4924d22f0 0x55f4928b31e0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.466+0000 7f11c9ffb640 0 client.16653 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.466+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3421123922 >> 172.21.15.43:0/3421123922 conn(0x55f4924ef300 msgr2=0x55f49289b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.466+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3421123922 shutdown_connections 2024-04-25T10:36:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.466+0000 7f11ef3b2740 1 -- 172.21.15.43:0/3421123922 wait complete. 2024-04-25T10:36:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.466+0000 7f11ef3b2740 20 client.16653 trim_cache size 0 max 16384 2024-04-25T10:36:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:36:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:36:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:36:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:36:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.467+0000 7f11e8430640 1 -- 172.21.15.43:0/617880008 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f11cc0afcd0 msgr2=0x7f11cc0b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.467+0000 7f11e8430640 1 --2- 172.21.15.43:0/617880008 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f11cc0afcd0 0x7f11cc0b21e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f11d80a45d0 tx=0x7f11d8005890 comp rx=0 tx=0).stop 2024-04-25T10:36:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.467+0000 7f11e8430640 1 -- 172.21.15.43:0/617880008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11e010a000 msgr2=0x7f11e019efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.467+0000 7f11e8430640 1 --2- 172.21.15.43:0/617880008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11e010a000 0x7f11e019efb0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f11d800ead0 tx=0x7f11d800eed0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.467+0000 7f11e8430640 1 -- 172.21.15.43:0/617880008 shutdown_connections 2024-04-25T10:36:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.467+0000 7f11e8430640 1 --2- 172.21.15.43:0/617880008 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f11cc0afcd0 0x7f11cc0b21e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.467+0000 7f11e8430640 1 --2- 172.21.15.43:0/617880008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11e010a000 0x7f11e019efb0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.467+0000 7f11e8430640 1 --2- 172.21.15.43:0/617880008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11e0109b60 0x7f11e019acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.467+0000 7f11e8430640 1 --2- 172.21.15.43:0/617880008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11e01042b0 0x7f11e019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.467+0000 7f11e8430640 1 -- 172.21.15.43:0/617880008 >> 172.21.15.43:0/617880008 conn(0x7f11e0079170 msgr2=0x7f11e01022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.467+0000 7f11e8430640 1 -- 172.21.15.43:0/617880008 shutdown_connections 2024-04-25T10:36:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:27.467+0000 7f11e8430640 1 -- 172.21.15.43:0/617880008 wait complete. 2024-04-25T10:36:28.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:27 smithi077 ceph-mon[32772]: pgmap v121: 97 pgs: 97 active+clean; 4.4 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-25T10:36:28.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:27 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:36:28.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:27 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:36:28.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:27 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:36:28.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:27 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:36:28.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:27 smithi164 ceph-mon[32713]: pgmap v121: 97 pgs: 97 active+clean; 4.4 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-25T10:36:28.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:27 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:36:28.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:27 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:36:28.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:27 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:36:28.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:27 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:36:28.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:27 smithi043 ceph-mon[29478]: pgmap v121: 97 pgs: 97 active+clean; 4.4 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-25T10:36:28.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:27 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:36:28.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:27 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:36:28.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:27 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:36:28.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:27 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:36:29.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:28 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:36:29.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:28 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:36:29.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:28 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:36:30.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:29 smithi077 ceph-mon[32772]: pgmap v122: 97 pgs: 97 active+clean; 4.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 48 KiB/s wr, 21 op/s 2024-04-25T10:36:30.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:29 smithi164 ceph-mon[32713]: pgmap v122: 97 pgs: 97 active+clean; 4.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 48 KiB/s wr, 21 op/s 2024-04-25T10:36:30.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:29 smithi043 ceph-mon[29478]: pgmap v122: 97 pgs: 97 active+clean; 4.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 48 KiB/s wr, 21 op/s 2024-04-25T10:36:32.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:31 smithi077 ceph-mon[32772]: pgmap v123: 97 pgs: 97 active+clean; 4.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-25T10:36:32.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:31 smithi164 ceph-mon[32713]: pgmap v123: 97 pgs: 97 active+clean; 4.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-25T10:36:32.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:31 smithi043 ceph-mon[29478]: pgmap v123: 97 pgs: 97 active+clean; 4.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-25T10:36:34.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:33 smithi077 ceph-mon[32772]: pgmap v124: 97 pgs: 97 active+clean; 4.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-25T10:36:34.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:33 smithi164 ceph-mon[32713]: pgmap v124: 97 pgs: 97 active+clean; 4.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-25T10:36:34.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:33 smithi043 ceph-mon[29478]: pgmap v124: 97 pgs: 97 active+clean; 4.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-25T10:36:36.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:35 smithi077 ceph-mon[32772]: pgmap v125: 97 pgs: 97 active+clean; 4.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-25T10:36:36.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:35 smithi164 ceph-mon[32713]: pgmap v125: 97 pgs: 97 active+clean; 4.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-25T10:36:36.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:35 smithi043 ceph-mon[29478]: pgmap v125: 97 pgs: 97 active+clean; 4.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-25T10:36:38.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:37 smithi077 ceph-mon[32772]: pgmap v126: 97 pgs: 97 active+clean; 4.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-25T10:36:38.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:37 smithi164 ceph-mon[32713]: pgmap v126: 97 pgs: 97 active+clean; 4.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-25T10:36:38.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:37 smithi043 ceph-mon[29478]: pgmap v126: 97 pgs: 97 active+clean; 4.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-25T10:36:40.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:39 smithi077 ceph-mon[32772]: pgmap v127: 97 pgs: 97 active+clean; 4.8 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-04-25T10:36:40.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:39 smithi164 ceph-mon[32713]: pgmap v127: 97 pgs: 97 active+clean; 4.8 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-04-25T10:36:40.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:39 smithi043 ceph-mon[29478]: pgmap v127: 97 pgs: 97 active+clean; 4.8 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-04-25T10:36:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.578+0000 7f379ad7a640 1 -- 172.21.15.43:0/2518373358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3794102090 msgr2=0x7f37941024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.578+0000 7f379ad7a640 1 --2- 172.21.15.43:0/2518373358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3794102090 0x7f37941024a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3784009900 tx=0x7f378402a410 comp rx=0 tx=0).stop 2024-04-25T10:36:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.578+0000 7f379ad7a640 1 -- 172.21.15.43:0/2518373358 shutdown_connections 2024-04-25T10:36:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.578+0000 7f379ad7a640 1 --2- 172.21.15.43:0/2518373358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f379410e590 0x7f37941109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.578+0000 7f379ad7a640 1 --2- 172.21.15.43:0/2518373358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3794102090 0x7f37941024a0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.578+0000 7f379ad7a640 1 --2- 172.21.15.43:0/2518373358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3794109be0 0x7f3794101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.578+0000 7f379ad7a640 1 -- 172.21.15.43:0/2518373358 >> 172.21.15.43:0/2518373358 conn(0x7f37940fd8a0 msgr2=0x7f37940ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.578+0000 7f379ad7a640 1 -- 172.21.15.43:0/2518373358 shutdown_connections 2024-04-25T10:36:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.578+0000 7f379ad7a640 1 -- 172.21.15.43:0/2518373358 wait complete. 2024-04-25T10:36:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.579+0000 7f379ad7a640 1 Processor -- start 2024-04-25T10:36:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.579+0000 7f379ad7a640 1 -- start start 2024-04-25T10:36:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.579+0000 7f379ad7a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3794102090 0x7f379419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.579+0000 7f379ad7a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3794109be0 0x7f379419acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.579+0000 7f379ad7a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f379410e590 0x7f379419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.579+0000 7f379ad7a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3794113d50 con 0x7f379410e590 2024-04-25T10:36:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.579+0000 7f379ad7a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3794113bd0 con 0x7f3794109be0 2024-04-25T10:36:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.579+0000 7f379ad7a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3794113ed0 con 0x7f3794102090 2024-04-25T10:36:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.579+0000 7f3799317640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f379410e590 0x7f379419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.579+0000 7f3799317640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f379410e590 0x7f379419efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44884/0 (socket says 172.21.15.43:44884) 2024-04-25T10:36:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.579+0000 7f3799317640 1 -- 172.21.15.43:0/145528772 learned_addr learned my addr 172.21.15.43:0/145528772 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.579+0000 7f378bfff640 1 --2- 172.21.15.43:0/145528772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3794109be0 0x7f379419acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.579+0000 7f3798b16640 1 --2- 172.21.15.43:0/145528772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3794102090 0x7f379419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.579+0000 7f3799317640 1 -- 172.21.15.43:0/145528772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3794102090 msgr2=0x7f379419a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.579+0000 7f3799317640 1 --2- 172.21.15.43:0/145528772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3794102090 0x7f379419a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.579+0000 7f3799317640 1 -- 172.21.15.43:0/145528772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3794109be0 msgr2=0x7f379419acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.579+0000 7f3799317640 1 --2- 172.21.15.43:0/145528772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3794109be0 0x7f379419acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.579+0000 7f3799317640 1 -- 172.21.15.43:0/145528772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379419f570 con 0x7f379410e590 2024-04-25T10:36:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.579+0000 7f378bfff640 1 --2- 172.21.15.43:0/145528772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3794109be0 0x7f379419acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.580+0000 7f3799317640 1 --2- 172.21.15.43:0/145528772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f379410e590 0x7f379419efb0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f378c00cce0 tx=0x7f378c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.580+0000 7f3789ffb640 1 -- 172.21.15.43:0/145528772 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c00f450 con 0x7f379410e590 2024-04-25T10:36:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.580+0000 7f3789ffb640 1 -- 172.21.15.43:0/145528772 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f378c00faa0 con 0x7f379410e590 2024-04-25T10:36:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.580+0000 7f379ad7a640 1 -- 172.21.15.43:0/145528772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379419f860 con 0x7f379410e590 2024-04-25T10:36:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.580+0000 7f3789ffb640 1 -- 172.21.15.43:0/145528772 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c018af0 con 0x7f379410e590 2024-04-25T10:36:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.580+0000 7f379ad7a640 1 -- 172.21.15.43:0/145528772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379419fd80 con 0x7f379410e590 2024-04-25T10:36:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.583+0000 7f3789ffb640 1 -- 172.21.15.43:0/145528772 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f378c00fc50 con 0x7f379410e590 2024-04-25T10:36:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.583+0000 7f37a1c22740 1 Processor -- start 2024-04-25T10:36:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.583+0000 7f37a1c22740 1 -- start start 2024-04-25T10:36:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.583+0000 7f37a1c22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7572b11b0 0x55a757650af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.583+0000 7f37a1c22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7572882f0 0x55a757651030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.583+0000 7f37a1c22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572904d0 0x55a757651570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.583+0000 7f37a1c22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a7575b7460 con 0x55a7572904d0 2024-04-25T10:36:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.583+0000 7f37a1c22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a757571590 con 0x55a7572b11b0 2024-04-25T10:36:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.583+0000 7f37a1c22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a7575b3420 con 0x55a7572882f0 2024-04-25T10:36:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.583+0000 7f3789ffb640 1 --2- 172.21.15.43:0/145528772 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f377c0afcd0 0x7f377c0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.583+0000 7f378bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572904d0 0x55a757651570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.583+0000 7f378bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572904d0 0x55a757651570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44894/0 (socket says 172.21.15.43:44894) 2024-04-25T10:36:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.583+0000 7f378bfff640 1 -- 172.21.15.43:0/2696375095 learned_addr learned my addr 172.21.15.43:0/2696375095 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.583+0000 7f3799317640 1 --2- 172.21.15.43:0/145528772 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f377c0afcd0 0x7f377c0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.583+0000 7f3798b16640 1 --2- 172.21.15.43:0/2696375095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7572882f0 0x55a757651030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.584+0000 7f3789ffb640 1 -- 172.21.15.43:0/145528772 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f378c0df7f0 con 0x7f379410e590 2024-04-25T10:36:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.584+0000 7f378bfff640 1 --2- 172.21.15.43:0/2696375095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7572b11b0 0x55a757650af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.584+0000 7f378bfff640 1 -- 172.21.15.43:0/2696375095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7572882f0 msgr2=0x55a757651030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.584+0000 7f378bfff640 1 --2- 172.21.15.43:0/2696375095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7572882f0 0x55a757651030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.584+0000 7f378bfff640 1 -- 172.21.15.43:0/2696375095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7572b11b0 msgr2=0x55a757650af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.584+0000 7f378bfff640 1 --2- 172.21.15.43:0/2696375095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7572b11b0 0x55a757650af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.584+0000 7f378bfff640 1 -- 172.21.15.43:0/2696375095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a75757e0d0 con 0x55a7572904d0 2024-04-25T10:36:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.584+0000 7f378bfff640 1 --2- 172.21.15.43:0/2696375095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572904d0 0x55a757651570 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f379406bf10 tx=0x7f37840058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b23641ef4aed13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.584+0000 7f3799317640 1 --2- 172.21.15.43:0/145528772 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f377c0afcd0 0x7f377c0b21e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f378c0dcfd0 tx=0x7f378c075220 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.584+0000 7f377affd640 1 -- 172.21.15.43:0/2696375095 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784035040 con 0x55a7572904d0 2024-04-25T10:36:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.584+0000 7f377affd640 1 -- 172.21.15.43:0/2696375095 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37840317b0 con 0x55a7572904d0 2024-04-25T10:36:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.584+0000 7f377affd640 1 -- 172.21.15.43:0/2696375095 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784003bf0 con 0x55a7572904d0 2024-04-25T10:36:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.585+0000 7f37a1c22740 1 -- 172.21.15.43:0/2696375095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572904d0 msgr2=0x55a757651570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.585+0000 7f37a1c22740 1 --2- 172.21.15.43:0/2696375095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572904d0 0x55a757651570 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f379406bf10 tx=0x7f37840058b0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.585+0000 7f37a1c22740 1 -- 172.21.15.43:0/2696375095 shutdown_connections 2024-04-25T10:36:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.585+0000 7f37a1c22740 1 --2- 172.21.15.43:0/2696375095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572904d0 0x55a757651570 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.585+0000 7f37a1c22740 1 --2- 172.21.15.43:0/2696375095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7572882f0 0x55a757651030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.585+0000 7f37a1c22740 1 --2- 172.21.15.43:0/2696375095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7572b11b0 0x55a757650af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.585+0000 7f37a1c22740 1 -- 172.21.15.43:0/2696375095 >> 172.21.15.43:0/2696375095 conn(0x55a7572a5300 msgr2=0x55a75764d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.585+0000 7f37a1c22740 1 -- 172.21.15.43:0/2696375095 shutdown_connections 2024-04-25T10:36:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.585+0000 7f37a1c22740 1 -- 172.21.15.43:0/2696375095 wait complete. 2024-04-25T10:36:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.586+0000 7f37a1c22740 1 -- start start 2024-04-25T10:36:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.586+0000 7f37a1c22740 1 Processor -- start 2024-04-25T10:36:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.586+0000 7f37a1c22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7572882f0 0x55a75765b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.586+0000 7f37a1c22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7572904d0 0x55a75765b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.586+0000 7f37a1c22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572b11b0 0x55a757661110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.586+0000 7f37a1c22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a75765dbc0 con 0x55a7572b11b0 2024-04-25T10:36:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.586+0000 7f37a1c22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a75753eb60 con 0x55a7572882f0 2024-04-25T10:36:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.586+0000 7f37a1c22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a75765dd40 con 0x55a7572904d0 2024-04-25T10:36:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.586+0000 7f37a1c22740 10 client.0 _ll_register_callbacks cb 0x55a7575632b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.586+0000 7f378bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572b11b0 0x55a757661110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.586+0000 7f378bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572b11b0 0x55a757661110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44906/0 (socket says 172.21.15.43:44906) 2024-04-25T10:36:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.586+0000 7f378bfff640 1 -- 172.21.15.43:0/1137491434 learned_addr learned my addr 172.21.15.43:0/1137491434 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.587+0000 7f378bfff640 1 --2- 172.21.15.43:0/1137491434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7572882f0 0x55a75765b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.587+0000 7f378bfff640 1 -- 172.21.15.43:0/1137491434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7572904d0 msgr2=0x55a75765b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.587+0000 7f378bfff640 1 --2- 172.21.15.43:0/1137491434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7572904d0 0x55a75765b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.587+0000 7f378bfff640 1 -- 172.21.15.43:0/1137491434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7572882f0 msgr2=0x55a75765b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.587+0000 7f378bfff640 1 --2- 172.21.15.43:0/1137491434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7572882f0 0x55a75765b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.587+0000 7f378bfff640 1 -- 172.21.15.43:0/1137491434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7572c9210 con 0x55a7572b11b0 2024-04-25T10:36:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.587+0000 7f378bfff640 1 --2- 172.21.15.43:0/1137491434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572b11b0 0x55a757661110 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x55a75765a800 tx=0x7f37840038f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b2729f5e3d6ead server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.587+0000 7f3773fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.587+0000 7f3773fff640 1 -- 172.21.15.43:0/1137491434 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784004250 con 0x55a7572b11b0 2024-04-25T10:36:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.587+0000 7f3773fff640 1 -- 172.21.15.43:0/1137491434 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f3784051040 con 0x55a7572b11b0 2024-04-25T10:36:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.587+0000 7f37a1c22740 1 -- 172.21.15.43:0/1137491434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37840317b0 con 0x55a7572b11b0 2024-04-25T10:36:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.587+0000 7f3773fff640 1 -- 172.21.15.43:0/1137491434 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f37840043f0 con 0x55a7572b11b0 2024-04-25T10:36:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.587+0000 7f37a1c22740 10 client.16665 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.587+0000 7f37a1c22740 1 -- 172.21.15.43:0/1137491434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a757395620 con 0x55a7572b11b0 2024-04-25T10:36:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.588+0000 7f3773fff640 1 -- 172.21.15.43:0/1137491434 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37840317b0 con 0x55a7572b11b0 2024-04-25T10:36:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.588+0000 7f3773fff640 1 -- 172.21.15.43:0/1137491434 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f378403a040 con 0x55a7572b11b0 2024-04-25T10:36:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.588+0000 7f37a1c22740 10 client.16665 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.588+0000 7f37a1c22740 10 client.16665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.588+0000 7f37a1c22740 20 client.16665 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.588+0000 7f37a1c22740 1 --2- 172.21.15.43:0/1137491434 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a75726fd80 0x55a75766d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.588+0000 7f37a1c22740 4 client.16665 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:36:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.588+0000 7f37a1c22740 1 -- 172.21.15.43:0/1137491434 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7575c5070 con 0x55a75726fd80 2024-04-25T10:36:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.588+0000 7f3798b16640 1 --2- 172.21.15.43:0/1137491434 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a75726fd80 0x55a75766d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.589+0000 7f3773fff640 10 client.16665 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.589+0000 7f3798b16640 1 --2- 172.21.15.43:0/1137491434 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a75726fd80 0x55a75766d590 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22bc3a735c52652b server_cookie=d9e38f47ee1cb2cd in_seq=0 out_seq=0 2024-04-25T10:36:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.589+0000 7f3773fff640 1 -- 172.21.15.43:0/1137491434 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a7575c5070 con 0x55a75726fd80 2024-04-25T10:36:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.589+0000 7f3773fff640 10 client.16665 handle_command_reply: tid=0 2024-04-25T10:36:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.590+0000 7f37a1c22740 1 client.16665 shutdown 2024-04-25T10:36:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.590+0000 7f37a1c22740 1 -- 172.21.15.43:0/1137491434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572b11b0 msgr2=0x55a757661110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.590+0000 7f37a1c22740 1 --2- 172.21.15.43:0/1137491434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572b11b0 0x55a757661110 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x55a75765a800 tx=0x7f37840038f0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.590+0000 7f37a1c22740 1 -- 172.21.15.43:0/1137491434 shutdown_connections 2024-04-25T10:36:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.590+0000 7f37a1c22740 1 --2- 172.21.15.43:0/1137491434 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a75726fd80 0x55a75766d590 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.590+0000 7f37a1c22740 1 --2- 172.21.15.43:0/1137491434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572b11b0 0x55a757661110 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.590+0000 7f37a1c22740 1 --2- 172.21.15.43:0/1137491434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7572904d0 0x55a75765b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.590+0000 7f3773fff640 0 client.16665 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.590+0000 7f37a1c22740 1 --2- 172.21.15.43:0/1137491434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7572882f0 0x55a75765b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.590+0000 7f37a1c22740 1 -- 172.21.15.43:0/1137491434 >> 172.21.15.43:0/1137491434 conn(0x55a7572a5300 msgr2=0x55a757652810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.590+0000 7f37a1c22740 1 -- 172.21.15.43:0/1137491434 shutdown_connections 2024-04-25T10:36:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.590+0000 7f37a1c22740 1 -- 172.21.15.43:0/1137491434 wait complete. 2024-04-25T10:36:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.590+0000 7f37a1c22740 20 client.16665 trim_cache size 0 max 16384 2024-04-25T10:36:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.605+0000 7f37a1c22740 1 Processor -- start 2024-04-25T10:36:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.605+0000 7f37a1c22740 1 -- start start 2024-04-25T10:36:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.606+0000 7f37a1c22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a75726fd80 0x55a757651890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.606+0000 7f37a1c22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572882f0 0x55a757661010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.606+0000 7f37a1c22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7572904d0 0x55a757652e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.606+0000 7f37a1c22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f37941b2140 con 0x55a7572882f0 2024-04-25T10:36:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.606+0000 7f37a1c22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f37941afa10 con 0x55a7572904d0 2024-04-25T10:36:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.606+0000 7f37a1c22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37941ad860 con 0x55a75726fd80 2024-04-25T10:36:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.606+0000 7f3798b16640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572882f0 0x55a757661010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.606+0000 7f3798b16640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572882f0 0x55a757661010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44912/0 (socket says 172.21.15.43:44912) 2024-04-25T10:36:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.606+0000 7f3798b16640 1 -- 172.21.15.43:0/2084024937 learned_addr learned my addr 172.21.15.43:0/2084024937 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.606+0000 7f378bfff640 1 --2- 172.21.15.43:0/2084024937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a75726fd80 0x55a757651890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.606+0000 7f378bfff640 1 --2- 172.21.15.43:0/2084024937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7572904d0 0x55a757652e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.606+0000 7f3798b16640 1 -- 172.21.15.43:0/2084024937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a75726fd80 msgr2=0x55a757651890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.606+0000 7f3798b16640 1 --2- 172.21.15.43:0/2084024937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a75726fd80 0x55a757651890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.606+0000 7f3798b16640 1 -- 172.21.15.43:0/2084024937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7572904d0 msgr2=0x55a757652e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.606+0000 7f3798b16640 1 --2- 172.21.15.43:0/2084024937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7572904d0 0x55a757652e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.606+0000 7f3798b16640 1 -- 172.21.15.43:0/2084024937 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a757653380 con 0x55a7572882f0 2024-04-25T10:36:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.606+0000 7f378bfff640 1 --2- 172.21.15.43:0/2084024937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a75726fd80 0x55a757651890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.606+0000 7f3798b16640 1 --2- 172.21.15.43:0/2084024937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572882f0 0x55a757661010 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f379406bcf0 tx=0x7f379000b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e28b1d1aac3544c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.606+0000 7f377a7fc640 1 -- 172.21.15.43:0/2084024937 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37900110d0 con 0x55a7572882f0 2024-04-25T10:36:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.607+0000 7f377a7fc640 1 -- 172.21.15.43:0/2084024937 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3790011270 con 0x55a7572882f0 2024-04-25T10:36:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.607+0000 7f377a7fc640 1 -- 172.21.15.43:0/2084024937 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790019ac0 con 0x55a7572882f0 2024-04-25T10:36:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.607+0000 7f37a1c22740 1 -- 172.21.15.43:0/2084024937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572882f0 msgr2=0x55a757661010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.607+0000 7f37a1c22740 1 --2- 172.21.15.43:0/2084024937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572882f0 0x55a757661010 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f379406bcf0 tx=0x7f379000b6a0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.607+0000 7f37a1c22740 1 -- 172.21.15.43:0/2084024937 shutdown_connections 2024-04-25T10:36:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.607+0000 7f37a1c22740 1 --2- 172.21.15.43:0/2084024937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7572904d0 0x55a757652e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.607+0000 7f37a1c22740 1 --2- 172.21.15.43:0/2084024937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572882f0 0x55a757661010 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.607+0000 7f37a1c22740 1 --2- 172.21.15.43:0/2084024937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a75726fd80 0x55a757651890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.607+0000 7f37a1c22740 1 -- 172.21.15.43:0/2084024937 >> 172.21.15.43:0/2084024937 conn(0x55a7572a5300 msgr2=0x55a757652730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.607+0000 7f37a1c22740 1 -- 172.21.15.43:0/2084024937 shutdown_connections 2024-04-25T10:36:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.607+0000 7f37a1c22740 1 -- 172.21.15.43:0/2084024937 wait complete. 2024-04-25T10:36:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.608+0000 7f37a1c22740 1 -- start start 2024-04-25T10:36:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.608+0000 7f37a1c22740 1 Processor -- start 2024-04-25T10:36:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.608+0000 7f37a1c22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a75726fd80 0x55a757664880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.608+0000 7f37a1c22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7572882f0 0x55a757664dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.608+0000 7f37a1c22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572904d0 0x55a7576d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.608+0000 7f37a1c22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f37941aded0 con 0x55a7572904d0 2024-04-25T10:36:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.608+0000 7f37a1c22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f37941af340 con 0x55a7572882f0 2024-04-25T10:36:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.608+0000 7f37a1c22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37941ae5a0 con 0x55a75726fd80 2024-04-25T10:36:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.608+0000 7f37a1c22740 10 client.0 _ll_register_callbacks cb 0x7f3790005f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.608+0000 7f378bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572904d0 0x55a7576d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.608+0000 7f378bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572904d0 0x55a7576d36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44926/0 (socket says 172.21.15.43:44926) 2024-04-25T10:36:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.608+0000 7f378bfff640 1 -- 172.21.15.43:0/1468790782 learned_addr learned my addr 172.21.15.43:0/1468790782 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.608+0000 7f3798b16640 1 --2- 172.21.15.43:0/1468790782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7572882f0 0x55a757664dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.608+0000 7f378bfff640 1 --2- 172.21.15.43:0/1468790782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a75726fd80 0x55a757664880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.609+0000 7f378bfff640 1 -- 172.21.15.43:0/1468790782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a75726fd80 msgr2=0x55a757664880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.609+0000 7f378bfff640 1 --2- 172.21.15.43:0/1468790782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a75726fd80 0x55a757664880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.609+0000 7f378bfff640 1 -- 172.21.15.43:0/1468790782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7572882f0 msgr2=0x55a757664dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.609+0000 7f378bfff640 1 --2- 172.21.15.43:0/1468790782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7572882f0 0x55a757664dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.609+0000 7f378bfff640 1 -- 172.21.15.43:0/1468790782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7576d3bf0 con 0x55a7572904d0 2024-04-25T10:36:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.609+0000 7f378bfff640 1 --2- 172.21.15.43:0/1468790782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572904d0 0x55a7576d36b0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f378403d510 tx=0x7f378403f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184c078dedd8339f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.609+0000 7f3773fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.609+0000 7f3773fff640 1 -- 172.21.15.43:0/1468790782 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784035070 con 0x55a7572904d0 2024-04-25T10:36:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.609+0000 7f37a1c22740 1 -- 172.21.15.43:0/1468790782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3790011270 con 0x55a7572904d0 2024-04-25T10:36:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.609+0000 7f3773fff640 1 -- 172.21.15.43:0/1468790782 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f3784032d70 con 0x55a7572904d0 2024-04-25T10:36:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.609+0000 7f3773fff640 1 -- 172.21.15.43:0/1468790782 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3784004250 con 0x55a7572904d0 2024-04-25T10:36:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.609+0000 7f37a1c22740 10 client.16671 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.609+0000 7f37a1c22740 1 -- 172.21.15.43:0/1468790782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a7576d4d80 con 0x55a7572904d0 2024-04-25T10:36:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.609+0000 7f3773fff640 1 -- 172.21.15.43:0/1468790782 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3790011270 con 0x55a7572904d0 2024-04-25T10:36:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.610+0000 7f3773fff640 1 -- 172.21.15.43:0/1468790782 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f378403db50 con 0x55a7572904d0 2024-04-25T10:36:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.610+0000 7f37a1c22740 10 client.16671 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.610+0000 7f37a1c22740 10 client.16671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.610+0000 7f37a1c22740 20 client.16671 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.610+0000 7f37a1c22740 1 --2- 172.21.15.43:0/1468790782 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a7572b11b0 0x55a7576d73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.610+0000 7f37a1c22740 4 client.16671 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:36:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.610+0000 7f37a1c22740 1 -- 172.21.15.43:0/1468790782 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7575beee0 con 0x55a7572b11b0 2024-04-25T10:36:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.610+0000 7f3798b16640 1 --2- 172.21.15.43:0/1468790782 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a7572b11b0 0x55a7576d73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.610+0000 7f3773fff640 10 client.16671 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.610+0000 7f3798b16640 1 --2- 172.21.15.43:0/1468790782 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a7572b11b0 0x55a7576d73f0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa5042b34d579ca1 server_cookie=8ada894d7770b2e0 in_seq=0 out_seq=0 2024-04-25T10:36:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.628+0000 7f3773fff640 1 -- 172.21.15.43:0/1468790782 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a7575beee0 con 0x55a7572b11b0 2024-04-25T10:36:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.628+0000 7f3773fff640 10 client.16671 handle_command_reply: tid=0 2024-04-25T10:36:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.628+0000 7f37a1c22740 1 client.16671 shutdown 2024-04-25T10:36:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.628+0000 7f37a1c22740 1 -- 172.21.15.43:0/1468790782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572904d0 msgr2=0x55a7576d36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.628+0000 7f37a1c22740 1 --2- 172.21.15.43:0/1468790782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572904d0 0x55a7576d36b0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f378403d510 tx=0x7f378403f040 comp rx=0 tx=0).stop 2024-04-25T10:36:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.628+0000 7f37a1c22740 1 -- 172.21.15.43:0/1468790782 shutdown_connections 2024-04-25T10:36:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.628+0000 7f37a1c22740 1 --2- 172.21.15.43:0/1468790782 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a7572b11b0 0x55a7576d73f0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.628+0000 7f37a1c22740 1 --2- 172.21.15.43:0/1468790782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7572882f0 0x55a757664dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.628+0000 7f37a1c22740 1 --2- 172.21.15.43:0/1468790782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7572904d0 0x55a7576d36b0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.628+0000 7f3773fff640 0 client.16671 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.628+0000 7f37a1c22740 1 --2- 172.21.15.43:0/1468790782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a75726fd80 0x55a757664880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.628+0000 7f37a1c22740 1 -- 172.21.15.43:0/1468790782 >> 172.21.15.43:0/1468790782 conn(0x55a7572a5300 msgr2=0x55a757660000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.628+0000 7f37a1c22740 1 -- 172.21.15.43:0/1468790782 shutdown_connections 2024-04-25T10:36:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.628+0000 7f37a1c22740 1 -- 172.21.15.43:0/1468790782 wait complete. 2024-04-25T10:36:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.628+0000 7f37a1c22740 20 client.16671 trim_cache size 0 max 16384 2024-04-25T10:36:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:36:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:36:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:36:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:36:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.629+0000 7f379ad7a640 1 -- 172.21.15.43:0/145528772 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f377c0afcd0 msgr2=0x7f377c0b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.629+0000 7f379ad7a640 1 --2- 172.21.15.43:0/145528772 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f377c0afcd0 0x7f377c0b21e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f378c0dcfd0 tx=0x7f378c075220 comp rx=0 tx=0).stop 2024-04-25T10:36:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.629+0000 7f379ad7a640 1 -- 172.21.15.43:0/145528772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f379410e590 msgr2=0x7f379419efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.629+0000 7f379ad7a640 1 --2- 172.21.15.43:0/145528772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f379410e590 0x7f379419efb0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f378c00cce0 tx=0x7f378c00a620 comp rx=0 tx=0).stop 2024-04-25T10:36:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.629+0000 7f379ad7a640 1 -- 172.21.15.43:0/145528772 shutdown_connections 2024-04-25T10:36:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.629+0000 7f379ad7a640 1 --2- 172.21.15.43:0/145528772 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f377c0afcd0 0x7f377c0b21e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.629+0000 7f379ad7a640 1 --2- 172.21.15.43:0/145528772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3794109be0 0x7f379419acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.629+0000 7f379ad7a640 1 --2- 172.21.15.43:0/145528772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f379410e590 0x7f379419efb0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.629+0000 7f379ad7a640 1 --2- 172.21.15.43:0/145528772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3794102090 0x7f379419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.629+0000 7f379ad7a640 1 -- 172.21.15.43:0/145528772 >> 172.21.15.43:0/145528772 conn(0x7f37940fd8a0 msgr2=0x7f37940ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.629+0000 7f379ad7a640 1 -- 172.21.15.43:0/145528772 shutdown_connections 2024-04-25T10:36:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:40.629+0000 7f379ad7a640 1 -- 172.21.15.43:0/145528772 wait complete. 2024-04-25T10:36:42.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:41 smithi077 ceph-mon[32772]: pgmap v128: 97 pgs: 97 active+clean; 4.8 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-04-25T10:36:42.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:41 smithi164 ceph-mon[32713]: pgmap v128: 97 pgs: 97 active+clean; 4.8 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-04-25T10:36:42.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:41 smithi043 ceph-mon[29478]: pgmap v128: 97 pgs: 97 active+clean; 4.8 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-04-25T10:36:44.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:43 smithi077 ceph-mon[32772]: pgmap v129: 97 pgs: 97 active+clean; 4.8 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-04-25T10:36:44.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:43 smithi164 ceph-mon[32713]: pgmap v129: 97 pgs: 97 active+clean; 4.8 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-04-25T10:36:44.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:43 smithi043 ceph-mon[29478]: pgmap v129: 97 pgs: 97 active+clean; 4.8 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-04-25T10:36:46.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:45 smithi077 ceph-mon[32772]: pgmap v130: 97 pgs: 97 active+clean; 5.0 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-25T10:36:46.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:45 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:36:46.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:45 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:36:46.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:45 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:36:46.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:45 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:36:46.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:45 smithi164 ceph-mon[32713]: pgmap v130: 97 pgs: 97 active+clean; 5.0 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-25T10:36:46.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:45 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:36:46.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:45 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:36:46.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:45 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:36:46.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:45 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:36:46.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:45 smithi043 ceph-mon[29478]: pgmap v130: 97 pgs: 97 active+clean; 5.0 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-25T10:36:46.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:45 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:36:46.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:45 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:36:46.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:45 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:36:46.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:45 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:36:48.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:47 smithi077 ceph-mon[32772]: pgmap v131: 97 pgs: 97 active+clean; 5.0 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-25T10:36:48.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:47 smithi164 ceph-mon[32713]: pgmap v131: 97 pgs: 97 active+clean; 5.0 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-25T10:36:48.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:47 smithi043 ceph-mon[29478]: pgmap v131: 97 pgs: 97 active+clean; 5.0 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-25T10:36:50.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:49 smithi077 ceph-mon[32772]: pgmap v132: 97 pgs: 97 active+clean; 5.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-04-25T10:36:50.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:49 smithi164 ceph-mon[32713]: pgmap v132: 97 pgs: 97 active+clean; 5.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-04-25T10:36:50.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:49 smithi043 ceph-mon[29478]: pgmap v132: 97 pgs: 97 active+clean; 5.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-04-25T10:36:52.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:51 smithi077 ceph-mon[32772]: pgmap v133: 97 pgs: 97 active+clean; 5.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-25T10:36:52.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:51 smithi164 ceph-mon[32713]: pgmap v133: 97 pgs: 97 active+clean; 5.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-25T10:36:52.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:51 smithi043 ceph-mon[29478]: pgmap v133: 97 pgs: 97 active+clean; 5.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-25T10:36:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.740+0000 7f2d2b934640 1 -- 172.21.15.43:0/4268119156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d2410d450 msgr2=0x7f2d2410f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.740+0000 7f2d2b934640 1 --2- 172.21.15.43:0/4268119156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d2410d450 0x7f2d2410f870 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009ca0 tx=0x7f2d1c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.741+0000 7f2d2b934640 1 -- 172.21.15.43:0/4268119156 shutdown_connections 2024-04-25T10:36:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.741+0000 7f2d2b934640 1 --2- 172.21.15.43:0/4268119156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d241032d0 0x7f2d241036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.741+0000 7f2d2b934640 1 --2- 172.21.15.43:0/4268119156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d2410d450 0x7f2d2410f870 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.741+0000 7f2d2b934640 1 --2- 172.21.15.43:0/4268119156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d24107960 0x7f2d24102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.741+0000 7f2d2b934640 1 -- 172.21.15.43:0/4268119156 >> 172.21.15.43:0/4268119156 conn(0x7f2d240fd8f0 msgr2=0x7f2d240ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.741+0000 7f2d2b934640 1 -- 172.21.15.43:0/4268119156 shutdown_connections 2024-04-25T10:36:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.741+0000 7f2d2b934640 1 -- 172.21.15.43:0/4268119156 wait complete. 2024-04-25T10:36:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.741+0000 7f2d2b934640 1 Processor -- start 2024-04-25T10:36:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.741+0000 7f2d2b934640 1 -- start start 2024-04-25T10:36:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.742+0000 7f2d2b934640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d241032d0 0x7f2d2419b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.742+0000 7f2d2b934640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d24107960 0x7f2d2419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.742+0000 7f2d2b934640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d2410d450 0x7f2d2419c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.742+0000 7f2d2b934640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2d24112c10 con 0x7f2d2410d450 2024-04-25T10:36:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.742+0000 7f2d2b934640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d24112a90 con 0x7f2d241032d0 2024-04-25T10:36:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.742+0000 7f2d2b934640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d24112d90 con 0x7f2d24107960 2024-04-25T10:36:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.742+0000 7f2d29ed1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d2410d450 0x7f2d2419c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.742+0000 7f2d29ed1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d2410d450 0x7f2d2419c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45870/0 (socket says 172.21.15.43:45870) 2024-04-25T10:36:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.742+0000 7f2d29ed1640 1 -- 172.21.15.43:0/2177015845 learned_addr learned my addr 172.21.15.43:0/2177015845 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.742+0000 7f2d296d0640 1 --2- 172.21.15.43:0/2177015845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d241032d0 0x7f2d2419b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.742+0000 7f2d28ecf640 1 --2- 172.21.15.43:0/2177015845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d24107960 0x7f2d2419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.742+0000 7f2d29ed1640 1 -- 172.21.15.43:0/2177015845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d24107960 msgr2=0x7f2d2419bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.742+0000 7f2d29ed1640 1 --2- 172.21.15.43:0/2177015845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d24107960 0x7f2d2419bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.742+0000 7f2d29ed1640 1 -- 172.21.15.43:0/2177015845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d241032d0 msgr2=0x7f2d2419b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.742+0000 7f2d29ed1640 1 --2- 172.21.15.43:0/2177015845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d241032d0 0x7f2d2419b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.742+0000 7f2d29ed1640 1 -- 172.21.15.43:0/2177015845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d241a68d0 con 0x7f2d2410d450 2024-04-25T10:36:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.743+0000 7f2d29ed1640 1 --2- 172.21.15.43:0/2177015845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d2410d450 0x7f2d2419c030 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009430 tx=0x7f2d1c0316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.743+0000 7f2d167fc640 1 -- 172.21.15.43:0/2177015845 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c003b70 con 0x7f2d2410d450 2024-04-25T10:36:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.743+0000 7f2d2b934640 1 -- 172.21.15.43:0/2177015845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d241a6b60 con 0x7f2d2410d450 2024-04-25T10:36:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.743+0000 7f2d167fc640 1 -- 172.21.15.43:0/2177015845 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d1c003d10 con 0x7f2d2410d450 2024-04-25T10:36:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.743+0000 7f2d167fc640 1 -- 172.21.15.43:0/2177015845 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c02fb20 con 0x7f2d2410d450 2024-04-25T10:36:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.743+0000 7f2d2b934640 1 -- 172.21.15.43:0/2177015845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d241a7100 con 0x7f2d2410d450 2024-04-25T10:36:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.746+0000 7f2d327e8740 1 Processor -- start 2024-04-25T10:36:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.746+0000 7f2d167fc640 1 -- 172.21.15.43:0/2177015845 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f2d1c004580 con 0x7f2d2410d450 2024-04-25T10:36:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.746+0000 7f2d327e8740 1 -- start start 2024-04-25T10:36:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.747+0000 7f2d327e8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5631d37b01b0 0x5631d3b4faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.747+0000 7f2d327e8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d37872f0 0x5631d3b50030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.747+0000 7f2d327e8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d378f4d0 0x5631d3b50570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.747+0000 7f2d327e8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5631d3ab6460 con 0x5631d378f4d0 2024-04-25T10:36:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.747+0000 7f2d327e8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5631d3a70590 con 0x5631d37872f0 2024-04-25T10:36:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.747+0000 7f2d327e8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5631d3ab2420 con 0x5631d37b01b0 2024-04-25T10:36:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.747+0000 7f2d28ecf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d378f4d0 0x5631d3b50570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.747+0000 7f2d28ecf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d378f4d0 0x5631d3b50570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45880/0 (socket says 172.21.15.43:45880) 2024-04-25T10:36:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.747+0000 7f2d28ecf640 1 -- 172.21.15.43:0/1545436834 learned_addr learned my addr 172.21.15.43:0/1545436834 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.747+0000 7f2d296d0640 1 --2- 172.21.15.43:0/1545436834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d37872f0 0x5631d3b50030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.747+0000 7f2d28ecf640 1 --2- 172.21.15.43:0/1545436834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5631d37b01b0 0x5631d3b4faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.747+0000 7f2d167fc640 1 --2- 172.21.15.43:0/2177015845 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2d100afc50 0x7f2d100b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.747+0000 7f2d28ecf640 1 -- 172.21.15.43:0/1545436834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5631d37b01b0 msgr2=0x5631d3b4faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.747+0000 7f2d28ecf640 1 --2- 172.21.15.43:0/1545436834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5631d37b01b0 0x5631d3b4faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.747+0000 7f2d28ecf640 1 -- 172.21.15.43:0/1545436834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d37872f0 msgr2=0x5631d3b50030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.747+0000 7f2d28ecf640 1 --2- 172.21.15.43:0/1545436834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d37872f0 0x5631d3b50030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.747+0000 7f2d29ed1640 1 --2- 172.21.15.43:0/2177015845 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2d100afc50 0x7f2d100b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.747+0000 7f2d28ecf640 1 -- 172.21.15.43:0/1545436834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631d3a7d0d0 con 0x5631d378f4d0 2024-04-25T10:36:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.747+0000 7f2d167fc640 1 -- 172.21.15.43:0/2177015845 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f2d1c100350 con 0x7f2d2410d450 2024-04-25T10:36:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.748+0000 7f2d28ecf640 1 --2- 172.21.15.43:0/1545436834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d378f4d0 0x5631d3b50570 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009560 tx=0x7f2d180096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e097d4449da0cd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.748+0000 7f2d0f7fe640 1 -- 172.21.15.43:0/1545436834 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18010690 con 0x5631d378f4d0 2024-04-25T10:36:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.748+0000 7f2d0f7fe640 1 -- 172.21.15.43:0/1545436834 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d18010ce0 con 0x5631d378f4d0 2024-04-25T10:36:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.748+0000 7f2d0f7fe640 1 -- 172.21.15.43:0/1545436834 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1801dd50 con 0x5631d378f4d0 2024-04-25T10:36:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.749+0000 7f2d327e8740 1 -- 172.21.15.43:0/1545436834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d378f4d0 msgr2=0x5631d3b50570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.749+0000 7f2d327e8740 1 --2- 172.21.15.43:0/1545436834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d378f4d0 0x5631d3b50570 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009560 tx=0x7f2d180096e0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.749+0000 7f2d29ed1640 1 --2- 172.21.15.43:0/2177015845 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2d100afc50 0x7f2d100b2160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0fdc90 tx=0x7f2d1c095e90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.749+0000 7f2d327e8740 1 -- 172.21.15.43:0/1545436834 shutdown_connections 2024-04-25T10:36:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.749+0000 7f2d327e8740 1 --2- 172.21.15.43:0/1545436834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d37872f0 0x5631d3b50030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.749+0000 7f2d327e8740 1 --2- 172.21.15.43:0/1545436834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d378f4d0 0x5631d3b50570 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.749+0000 7f2d327e8740 1 --2- 172.21.15.43:0/1545436834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5631d37b01b0 0x5631d3b4faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.749+0000 7f2d327e8740 1 -- 172.21.15.43:0/1545436834 >> 172.21.15.43:0/1545436834 conn(0x5631d37a4300 msgr2=0x5631d3b4c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.749+0000 7f2d327e8740 1 -- 172.21.15.43:0/1545436834 shutdown_connections 2024-04-25T10:36:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.749+0000 7f2d327e8740 1 -- 172.21.15.43:0/1545436834 wait complete. 2024-04-25T10:36:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.749+0000 7f2d327e8740 1 -- start start 2024-04-25T10:36:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.749+0000 7f2d327e8740 1 Processor -- start 2024-04-25T10:36:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.750+0000 7f2d327e8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5631d37872f0 0x5631d3b5a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.750+0000 7f2d327e8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d378f4d0 0x5631d3b5a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.750+0000 7f2d327e8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d37b01b0 0x5631d3b60140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.750+0000 7f2d327e8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5631d3b5cc20 con 0x5631d378f4d0 2024-04-25T10:36:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.750+0000 7f2d327e8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5631d3a3db60 con 0x5631d37b01b0 2024-04-25T10:36:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.750+0000 7f2d327e8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5631d3b5cda0 con 0x5631d37872f0 2024-04-25T10:36:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.750+0000 7f2d327e8740 10 client.0 _ll_register_callbacks cb 0x5631d3a622b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.750+0000 7f2d296d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d378f4d0 0x5631d3b5a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.750+0000 7f2d296d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d378f4d0 0x5631d3b5a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45894/0 (socket says 172.21.15.43:45894) 2024-04-25T10:36:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.750+0000 7f2d296d0640 1 -- 172.21.15.43:0/2284434906 learned_addr learned my addr 172.21.15.43:0/2284434906 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.750+0000 7f2d28ecf640 1 --2- 172.21.15.43:0/2284434906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d37b01b0 0x5631d3b60140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.750+0000 7f2d28ecf640 1 --2- 172.21.15.43:0/2284434906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5631d37872f0 0x5631d3b5a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.750+0000 7f2d296d0640 1 -- 172.21.15.43:0/2284434906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5631d37872f0 msgr2=0x5631d3b5a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.750+0000 7f2d296d0640 1 --2- 172.21.15.43:0/2284434906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5631d37872f0 0x5631d3b5a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.750+0000 7f2d296d0640 1 -- 172.21.15.43:0/2284434906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d37b01b0 msgr2=0x5631d3b60140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.750+0000 7f2d296d0640 1 --2- 172.21.15.43:0/2284434906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d37b01b0 0x5631d3b60140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.750+0000 7f2d296d0640 1 -- 172.21.15.43:0/2284434906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631d37c8210 con 0x5631d378f4d0 2024-04-25T10:36:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.750+0000 7f2d28ecf640 1 --2- 172.21.15.43:0/2284434906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d37b01b0 0x5631d3b60140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.751+0000 7f2d296d0640 1 --2- 172.21.15.43:0/2284434906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d378f4d0 0x5631d3b5a6f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000b920 tx=0x7f2d2000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a185a3909356ec24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.751+0000 7f2d0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.751+0000 7f2d0cff9640 1 -- 172.21.15.43:0/2284434906 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20005020 con 0x5631d378f4d0 2024-04-25T10:36:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.751+0000 7f2d0cff9640 1 -- 172.21.15.43:0/2284434906 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f2d200051c0 con 0x5631d378f4d0 2024-04-25T10:36:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.751+0000 7f2d327e8740 1 -- 172.21.15.43:0/2284434906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d18010ce0 con 0x5631d378f4d0 2024-04-25T10:36:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.751+0000 7f2d0cff9640 1 -- 172.21.15.43:0/2284434906 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2d200053c0 con 0x5631d378f4d0 2024-04-25T10:36:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.751+0000 7f2d327e8740 10 client.16683 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.751+0000 7f2d327e8740 1 -- 172.21.15.43:0/2284434906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5631d3894620 con 0x5631d378f4d0 2024-04-25T10:36:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.751+0000 7f2d0cff9640 1 -- 172.21.15.43:0/2284434906 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d18010ce0 con 0x5631d378f4d0 2024-04-25T10:36:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.751+0000 7f2d0cff9640 1 -- 172.21.15.43:0/2284434906 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2d2001b470 con 0x5631d378f4d0 2024-04-25T10:36:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.751+0000 7f2d327e8740 10 client.16683 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.751+0000 7f2d327e8740 10 client.16683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.751+0000 7f2d327e8740 20 client.16683 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.751+0000 7f2d327e8740 1 --2- 172.21.15.43:0/2284434906 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5631d376ed80 0x5631d3b6c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.751+0000 7f2d327e8740 4 client.16683 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:36:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.751+0000 7f2d327e8740 1 -- 172.21.15.43:0/2284434906 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631d3ac4070 con 0x5631d376ed80 2024-04-25T10:36:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.752+0000 7f2d28ecf640 1 --2- 172.21.15.43:0/2284434906 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5631d376ed80 0x5631d3b6c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.752+0000 7f2d0cff9640 10 client.16683 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.752+0000 7f2d28ecf640 1 --2- 172.21.15.43:0/2284434906 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5631d376ed80 0x5631d3b6c800 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=333dee7f80f6c00 server_cookie=f952f89087319d26 in_seq=0 out_seq=0 2024-04-25T10:36:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.753+0000 7f2d0cff9640 1 -- 172.21.15.43:0/2284434906 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5631d3ac4070 con 0x5631d376ed80 2024-04-25T10:36:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.753+0000 7f2d0cff9640 10 client.16683 handle_command_reply: tid=0 2024-04-25T10:36:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.753+0000 7f2d327e8740 1 client.16683 shutdown 2024-04-25T10:36:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.753+0000 7f2d327e8740 1 -- 172.21.15.43:0/2284434906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d378f4d0 msgr2=0x5631d3b5a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.753+0000 7f2d327e8740 1 --2- 172.21.15.43:0/2284434906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d378f4d0 0x5631d3b5a6f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000b920 tx=0x7f2d2000bde0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.753+0000 7f2d327e8740 1 -- 172.21.15.43:0/2284434906 shutdown_connections 2024-04-25T10:36:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.753+0000 7f2d327e8740 1 --2- 172.21.15.43:0/2284434906 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5631d376ed80 0x5631d3b6c800 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.753+0000 7f2d327e8740 1 --2- 172.21.15.43:0/2284434906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d37b01b0 0x5631d3b60140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.753+0000 7f2d327e8740 1 --2- 172.21.15.43:0/2284434906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d378f4d0 0x5631d3b5a6f0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.753+0000 7f2d0cff9640 0 client.16683 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.753+0000 7f2d327e8740 1 --2- 172.21.15.43:0/2284434906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5631d37872f0 0x5631d3b5a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.753+0000 7f2d327e8740 1 -- 172.21.15.43:0/2284434906 >> 172.21.15.43:0/2284434906 conn(0x5631d37a4300 msgr2=0x5631d3b51810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.753+0000 7f2d327e8740 1 -- 172.21.15.43:0/2284434906 shutdown_connections 2024-04-25T10:36:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.753+0000 7f2d327e8740 1 -- 172.21.15.43:0/2284434906 wait complete. 2024-04-25T10:36:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.753+0000 7f2d327e8740 20 client.16683 trim_cache size 0 max 16384 2024-04-25T10:36:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.769+0000 7f2d327e8740 1 Processor -- start 2024-04-25T10:36:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.769+0000 7f2d327e8740 1 -- start start 2024-04-25T10:36:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.770+0000 7f2d327e8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5631d376ed80 0x5631d3b50890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.770+0000 7f2d327e8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d37872f0 0x5631d3b60040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.770+0000 7f2d327e8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d378f4d0 0x5631d3b71fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.770+0000 7f2d327e8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241b21b0 con 0x5631d37872f0 2024-04-25T10:36:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.770+0000 7f2d327e8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241afa80 con 0x5631d378f4d0 2024-04-25T10:36:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.770+0000 7f2d327e8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241ad8d0 con 0x5631d376ed80 2024-04-25T10:36:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.770+0000 7f2d296d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d37872f0 0x5631d3b60040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.770+0000 7f2d296d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d37872f0 0x5631d3b60040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45900/0 (socket says 172.21.15.43:45900) 2024-04-25T10:36:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.770+0000 7f2d296d0640 1 -- 172.21.15.43:0/1988655590 learned_addr learned my addr 172.21.15.43:0/1988655590 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.770+0000 7f2d28ecf640 1 --2- 172.21.15.43:0/1988655590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d378f4d0 0x5631d3b71fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.770+0000 7f2d28ecf640 1 --2- 172.21.15.43:0/1988655590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5631d376ed80 0x5631d3b50890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.770+0000 7f2d296d0640 1 -- 172.21.15.43:0/1988655590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5631d376ed80 msgr2=0x5631d3b50890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.770+0000 7f2d296d0640 1 --2- 172.21.15.43:0/1988655590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5631d376ed80 0x5631d3b50890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.770+0000 7f2d296d0640 1 -- 172.21.15.43:0/1988655590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d378f4d0 msgr2=0x5631d3b71fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.770+0000 7f2d296d0640 1 --2- 172.21.15.43:0/1988655590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d378f4d0 0x5631d3b71fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.770+0000 7f2d296d0640 1 -- 172.21.15.43:0/1988655590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631d3b584f0 con 0x5631d37872f0 2024-04-25T10:36:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.770+0000 7f2d296d0640 1 --2- 172.21.15.43:0/1988655590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d37872f0 0x5631d3b60040 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2d2001ba80 tx=0x7f2d2001cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48d231ee9c7ca5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.772+0000 7f2d0effd640 1 -- 172.21.15.43:0/1988655590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20031070 con 0x5631d37872f0 2024-04-25T10:36:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.772+0000 7f2d0effd640 1 -- 172.21.15.43:0/1988655590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d20005560 con 0x5631d37872f0 2024-04-25T10:36:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.772+0000 7f2d0effd640 1 -- 172.21.15.43:0/1988655590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20028040 con 0x5631d37872f0 2024-04-25T10:36:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.773+0000 7f2d327e8740 1 -- 172.21.15.43:0/1988655590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d37872f0 msgr2=0x5631d3b60040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.773+0000 7f2d327e8740 1 --2- 172.21.15.43:0/1988655590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d37872f0 0x5631d3b60040 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2d2001ba80 tx=0x7f2d2001cec0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.773+0000 7f2d327e8740 1 -- 172.21.15.43:0/1988655590 shutdown_connections 2024-04-25T10:36:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.773+0000 7f2d327e8740 1 --2- 172.21.15.43:0/1988655590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d378f4d0 0x5631d3b71fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.773+0000 7f2d327e8740 1 --2- 172.21.15.43:0/1988655590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d37872f0 0x5631d3b60040 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.773+0000 7f2d327e8740 1 --2- 172.21.15.43:0/1988655590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5631d376ed80 0x5631d3b50890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.773+0000 7f2d327e8740 1 -- 172.21.15.43:0/1988655590 >> 172.21.15.43:0/1988655590 conn(0x5631d37a4300 msgr2=0x5631d3b58870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.773+0000 7f2d327e8740 1 -- 172.21.15.43:0/1988655590 shutdown_connections 2024-04-25T10:36:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.773+0000 7f2d327e8740 1 -- 172.21.15.43:0/1988655590 wait complete. 2024-04-25T10:36:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.774+0000 7f2d327e8740 1 -- start start 2024-04-25T10:36:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.774+0000 7f2d327e8740 1 Processor -- start 2024-04-25T10:36:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.774+0000 7f2d327e8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d376ed80 0x5631d3b61010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.774+0000 7f2d327e8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d37872f0 0x5631d3b61550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.774+0000 7f2d327e8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5631d378f4d0 0x5631d3b61a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.774+0000 7f2d327e8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241adf40 con 0x5631d376ed80 2024-04-25T10:36:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.774+0000 7f2d327e8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241af3b0 con 0x5631d37872f0 2024-04-25T10:36:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.774+0000 7f2d327e8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241ae610 con 0x5631d378f4d0 2024-04-25T10:36:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.774+0000 7f2d327e8740 10 client.0 _ll_register_callbacks cb 0x7f2d20004210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.775+0000 7f2d28ecf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d376ed80 0x5631d3b61010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.775+0000 7f2d28ecf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d376ed80 0x5631d3b61010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45902/0 (socket says 172.21.15.43:45902) 2024-04-25T10:36:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.775+0000 7f2d28ecf640 1 -- 172.21.15.43:0/1036856788 learned_addr learned my addr 172.21.15.43:0/1036856788 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.775+0000 7f2d296d0640 1 --2- 172.21.15.43:0/1036856788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d37872f0 0x5631d3b61550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.775+0000 7f2d28ecf640 1 --2- 172.21.15.43:0/1036856788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5631d378f4d0 0x5631d3b61a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.775+0000 7f2d28ecf640 1 -- 172.21.15.43:0/1036856788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5631d378f4d0 msgr2=0x5631d3b61a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.775+0000 7f2d28ecf640 1 --2- 172.21.15.43:0/1036856788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5631d378f4d0 0x5631d3b61a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.775+0000 7f2d28ecf640 1 -- 172.21.15.43:0/1036856788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d37872f0 msgr2=0x5631d3b61550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.775+0000 7f2d28ecf640 1 --2- 172.21.15.43:0/1036856788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d37872f0 0x5631d3b61550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.775+0000 7f2d28ecf640 1 -- 172.21.15.43:0/1036856788 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631d3b66060 con 0x5631d376ed80 2024-04-25T10:36:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.775+0000 7f2d28ecf640 1 --2- 172.21.15.43:0/1036856788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d376ed80 0x5631d3b61010 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2d18012e70 tx=0x7f2d200044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3bf5bdfc156a242 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.775+0000 7f2d0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.775+0000 7f2d0cff9640 1 -- 172.21.15.43:0/1036856788 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18023040 con 0x5631d376ed80 2024-04-25T10:36:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.775+0000 7f2d0cff9640 1 -- 172.21.15.43:0/1036856788 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f2d1801dc60 con 0x5631d376ed80 2024-04-25T10:36:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.775+0000 7f2d327e8740 1 -- 172.21.15.43:0/1036856788 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d20005560 con 0x5631d376ed80 2024-04-25T10:36:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.775+0000 7f2d0cff9640 1 -- 172.21.15.43:0/1036856788 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2d1801de60 con 0x5631d376ed80 2024-04-25T10:36:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.775+0000 7f2d327e8740 10 client.16689 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.775+0000 7f2d327e8740 1 -- 172.21.15.43:0/1036856788 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5631d3bd4db0 con 0x5631d376ed80 2024-04-25T10:36:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.776+0000 7f2d0cff9640 1 -- 172.21.15.43:0/1036856788 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d20005560 con 0x5631d376ed80 2024-04-25T10:36:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.776+0000 7f2d0cff9640 1 -- 172.21.15.43:0/1036856788 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2d18029040 con 0x5631d376ed80 2024-04-25T10:36:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.776+0000 7f2d327e8740 10 client.16689 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.776+0000 7f2d327e8740 10 client.16689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.776+0000 7f2d327e8740 20 client.16689 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.776+0000 7f2d327e8740 1 --2- 172.21.15.43:0/1036856788 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5631d37b01b0 0x5631d3bd7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.776+0000 7f2d327e8740 4 client.16689 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:36:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.776+0000 7f2d327e8740 1 -- 172.21.15.43:0/1036856788 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5631d3abdee0 con 0x5631d37b01b0 2024-04-25T10:36:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.776+0000 7f2d296d0640 1 --2- 172.21.15.43:0/1036856788 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5631d37b01b0 0x5631d3bd7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.777+0000 7f2d0cff9640 10 client.16689 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.777+0000 7f2d296d0640 1 --2- 172.21.15.43:0/1036856788 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5631d37b01b0 0x5631d3bd7230 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c525fde8edecfb11 server_cookie=45f77ee35f7a24fb in_seq=0 out_seq=0 2024-04-25T10:36:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.796+0000 7f2d0cff9640 1 -- 172.21.15.43:0/1036856788 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5631d3abdee0 con 0x5631d37b01b0 2024-04-25T10:36:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.796+0000 7f2d0cff9640 10 client.16689 handle_command_reply: tid=0 2024-04-25T10:36:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.796+0000 7f2d327e8740 1 client.16689 shutdown 2024-04-25T10:36:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.796+0000 7f2d327e8740 1 -- 172.21.15.43:0/1036856788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d376ed80 msgr2=0x5631d3b61010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.796+0000 7f2d327e8740 1 --2- 172.21.15.43:0/1036856788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d376ed80 0x5631d3b61010 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2d18012e70 tx=0x7f2d200044a0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.796+0000 7f2d327e8740 1 -- 172.21.15.43:0/1036856788 shutdown_connections 2024-04-25T10:36:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.796+0000 7f2d327e8740 1 --2- 172.21.15.43:0/1036856788 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5631d37b01b0 0x5631d3bd7230 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.796+0000 7f2d327e8740 1 --2- 172.21.15.43:0/1036856788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631d37872f0 0x5631d3b61550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.796+0000 7f2d327e8740 1 --2- 172.21.15.43:0/1036856788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5631d378f4d0 0x5631d3b61a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.796+0000 7f2d0cff9640 0 client.16689 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.796+0000 7f2d327e8740 1 --2- 172.21.15.43:0/1036856788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631d376ed80 0x5631d3b61010 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.796+0000 7f2d327e8740 1 -- 172.21.15.43:0/1036856788 >> 172.21.15.43:0/1036856788 conn(0x5631d37a4300 msgr2=0x5631d3b5b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.796+0000 7f2d327e8740 1 -- 172.21.15.43:0/1036856788 shutdown_connections 2024-04-25T10:36:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.796+0000 7f2d327e8740 1 -- 172.21.15.43:0/1036856788 wait complete. 2024-04-25T10:36:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.796+0000 7f2d327e8740 20 client.16689 trim_cache size 0 max 16384 2024-04-25T10:36:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:36:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:36:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:36:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:36:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.796+0000 7f2d2b934640 1 -- 172.21.15.43:0/2177015845 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2d100afc50 msgr2=0x7f2d100b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.797+0000 7f2d2b934640 1 --2- 172.21.15.43:0/2177015845 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2d100afc50 0x7f2d100b2160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0fdc90 tx=0x7f2d1c095e90 comp rx=0 tx=0).stop 2024-04-25T10:36:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.797+0000 7f2d2b934640 1 -- 172.21.15.43:0/2177015845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d2410d450 msgr2=0x7f2d2419c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.797+0000 7f2d2b934640 1 --2- 172.21.15.43:0/2177015845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d2410d450 0x7f2d2419c030 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009430 tx=0x7f2d1c0316f0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.797+0000 7f2d2b934640 1 -- 172.21.15.43:0/2177015845 shutdown_connections 2024-04-25T10:36:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.797+0000 7f2d2b934640 1 --2- 172.21.15.43:0/2177015845 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2d100afc50 0x7f2d100b2160 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.797+0000 7f2d2b934640 1 --2- 172.21.15.43:0/2177015845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d2410d450 0x7f2d2419c030 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.797+0000 7f2d2b934640 1 --2- 172.21.15.43:0/2177015845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d24107960 0x7f2d2419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.797+0000 7f2d2b934640 1 --2- 172.21.15.43:0/2177015845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d241032d0 0x7f2d2419b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.797+0000 7f2d2b934640 1 -- 172.21.15.43:0/2177015845 >> 172.21.15.43:0/2177015845 conn(0x7f2d240fd8f0 msgr2=0x7f2d2410e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.797+0000 7f2d2b934640 1 -- 172.21.15.43:0/2177015845 shutdown_connections 2024-04-25T10:36:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:36:53.797+0000 7f2d2b934640 1 -- 172.21.15.43:0/2177015845 wait complete. 2024-04-25T10:36:54.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:53 smithi077 ceph-mon[32772]: pgmap v134: 97 pgs: 97 active+clean; 5.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-25T10:36:54.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:53 smithi164 ceph-mon[32713]: pgmap v134: 97 pgs: 97 active+clean; 5.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-25T10:36:54.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:53 smithi043 ceph-mon[29478]: pgmap v134: 97 pgs: 97 active+clean; 5.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-25T10:36:56.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:55 smithi077 ceph-mon[32772]: pgmap v135: 97 pgs: 97 active+clean; 5.3 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-04-25T10:36:56.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:55 smithi164 ceph-mon[32713]: pgmap v135: 97 pgs: 97 active+clean; 5.3 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-04-25T10:36:56.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:55 smithi043 ceph-mon[29478]: pgmap v135: 97 pgs: 97 active+clean; 5.3 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-04-25T10:36:57.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:36:57.035 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:36:57.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.162+0000 7f5ba8957640 1 -- 172.21.15.43:0/1690986363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ba410b680 msgr2=0x7f5ba410daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.162+0000 7f5ba8957640 1 --2- 172.21.15.43:0/1690986363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ba410b680 0x7f5ba410daa0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5b98009900 tx=0x7f5b9802a480 comp rx=0 tx=0).stop 2024-04-25T10:36:57.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.162+0000 7f5ba8957640 1 -- 172.21.15.43:0/1690986363 shutdown_connections 2024-04-25T10:36:57.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.162+0000 7f5ba8957640 1 --2- 172.21.15.43:0/1690986363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba410e070 0x7f5ba4110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.162+0000 7f5ba8957640 1 --2- 172.21.15.43:0/1690986363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ba410b680 0x7f5ba410daa0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.162+0000 7f5ba8957640 1 --2- 172.21.15.43:0/1690986363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ba4104c20 0x7f5ba4105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.162+0000 7f5ba8957640 1 -- 172.21.15.43:0/1690986363 >> 172.21.15.43:0/1690986363 conn(0x7f5ba4078180 msgr2=0x7f5ba40785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:57.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.162+0000 7f5ba8957640 1 -- 172.21.15.43:0/1690986363 shutdown_connections 2024-04-25T10:36:57.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.162+0000 7f5ba8957640 1 -- 172.21.15.43:0/1690986363 wait complete. 2024-04-25T10:36:57.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.163+0000 7f5ba8957640 1 Processor -- start 2024-04-25T10:36:57.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.163+0000 7f5ba8957640 1 -- start start 2024-04-25T10:36:57.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.163+0000 7f5ba8957640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ba4104c20 0x7f5ba419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.163+0000 7f5ba8957640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ba410b680 0x7f5ba419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.163+0000 7f5ba8957640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba410e070 0x7f5ba419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.163+0000 7f5ba8957640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba4113cd0 con 0x7f5ba410b680 2024-04-25T10:36:57.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.163+0000 7f5ba8957640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba4113b50 con 0x7f5ba4104c20 2024-04-25T10:36:57.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.163+0000 7f5ba8957640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba4113e50 con 0x7f5ba410e070 2024-04-25T10:36:57.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.163+0000 7f5ba1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ba410b680 0x7f5ba419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.164+0000 7f5ba1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ba410b680 0x7f5ba419ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45918/0 (socket says 172.21.15.43:45918) 2024-04-25T10:36:57.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.164+0000 7f5ba1d9b640 1 -- 172.21.15.43:0/709451022 learned_addr learned my addr 172.21.15.43:0/709451022 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:57.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.164+0000 7f5ba259c640 1 --2- 172.21.15.43:0/709451022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ba4104c20 0x7f5ba419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.164+0000 7f5ba2d9d640 1 --2- 172.21.15.43:0/709451022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba410e070 0x7f5ba419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.164+0000 7f5ba1d9b640 1 -- 172.21.15.43:0/709451022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba410e070 msgr2=0x7f5ba419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.164+0000 7f5ba1d9b640 1 --2- 172.21.15.43:0/709451022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba410e070 0x7f5ba419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.164+0000 7f5ba1d9b640 1 -- 172.21.15.43:0/709451022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ba4104c20 msgr2=0x7f5ba419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.164+0000 7f5ba1d9b640 1 --2- 172.21.15.43:0/709451022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ba4104c20 0x7f5ba419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.164+0000 7f5ba1d9b640 1 -- 172.21.15.43:0/709451022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba419f590 con 0x7f5ba410b680 2024-04-25T10:36:57.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.164+0000 7f5ba1d9b640 1 --2- 172.21.15.43:0/709451022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ba410b680 0x7f5ba419ac90 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5b98005940 tx=0x7f5b98004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:57.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.164+0000 7f5b937fe640 1 -- 172.21.15.43:0/709451022 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9803f030 con 0x7f5ba410b680 2024-04-25T10:36:57.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.164+0000 7f5b937fe640 1 -- 172.21.15.43:0/709451022 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b98003aa0 con 0x7f5ba410b680 2024-04-25T10:36:57.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.164+0000 7f5ba8957640 1 -- 172.21.15.43:0/709451022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba419f820 con 0x7f5ba410b680 2024-04-25T10:36:57.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.164+0000 7f5b937fe640 1 -- 172.21.15.43:0/709451022 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b98035070 con 0x7f5ba410b680 2024-04-25T10:36:57.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.164+0000 7f5ba8957640 1 -- 172.21.15.43:0/709451022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba419fd10 con 0x7f5ba410b680 2024-04-25T10:36:57.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.167+0000 7f5b937fe640 1 -- 172.21.15.43:0/709451022 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f5b98042030 con 0x7f5ba410b680 2024-04-25T10:36:57.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.167+0000 7f5baf809740 1 Processor -- start 2024-04-25T10:36:57.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.168+0000 7f5baf809740 1 -- start start 2024-04-25T10:36:57.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.168+0000 7f5baf809740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992fd2390 0x557993372a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.168+0000 7f5baf809740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992fa94d0 0x557993372fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.168+0000 7f5baf809740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fb16b0 0x55799337f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.168+0000 7f5baf809740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5579932d8d80 con 0x557992fb16b0 2024-04-25T10:36:57.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.168+0000 7f5baf809740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5579932928b0 con 0x557992fa94d0 2024-04-25T10:36:57.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.168+0000 7f5baf809740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5579932d4d40 con 0x557992fd2390 2024-04-25T10:36:57.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.168+0000 7f5ba1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fb16b0 0x55799337f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.168+0000 7f5ba1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fb16b0 0x55799337f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45926/0 (socket says 172.21.15.43:45926) 2024-04-25T10:36:57.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.168+0000 7f5ba1d9b640 1 -- 172.21.15.43:0/620754956 learned_addr learned my addr 172.21.15.43:0/620754956 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:57.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.168+0000 7f5b937fe640 1 --2- 172.21.15.43:0/709451022 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5b8c0afc60 0x7f5b8c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.168+0000 7f5ba259c640 1 --2- 172.21.15.43:0/620754956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992fa94d0 0x557993372fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.168+0000 7f5ba2d9d640 1 --2- 172.21.15.43:0/620754956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992fd2390 0x557993372a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.169+0000 7f5ba2d9d640 1 --2- 172.21.15.43:0/709451022 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5b8c0afc60 0x7f5b8c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.169+0000 7f5b937fe640 1 -- 172.21.15.43:0/709451022 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f5b980ff660 con 0x7f5ba410b680 2024-04-25T10:36:57.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.169+0000 7f5ba1d9b640 1 -- 172.21.15.43:0/620754956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992fd2390 msgr2=0x557993372a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.169+0000 7f5ba1d9b640 1 --2- 172.21.15.43:0/620754956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992fd2390 0x557993372a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.169+0000 7f5ba1d9b640 1 -- 172.21.15.43:0/620754956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992fa94d0 msgr2=0x557993372fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.169+0000 7f5ba1d9b640 1 --2- 172.21.15.43:0/620754956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992fa94d0 0x557993372fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.169+0000 7f5ba1d9b640 1 -- 172.21.15.43:0/620754956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557993263a40 con 0x557992fb16b0 2024-04-25T10:36:57.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.169+0000 7f5ba1d9b640 1 --2- 172.21.15.43:0/620754956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fb16b0 0x55799337f8f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f5b980fcd80 tx=0x7f5b98003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f91986fa11d251 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:57.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.169+0000 7f5ba2d9d640 1 --2- 172.21.15.43:0/709451022 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5b8c0afc60 0x7f5b8c0b2170 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55799336fbd0 tx=0x7f5b9c00b2e0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:57.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.169+0000 7f5b90ff9640 1 -- 172.21.15.43:0/620754956 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b98049020 con 0x557992fb16b0 2024-04-25T10:36:57.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.169+0000 7f5b90ff9640 1 -- 172.21.15.43:0/620754956 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b980c73f0 con 0x557992fb16b0 2024-04-25T10:36:57.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.169+0000 7f5b90ff9640 1 -- 172.21.15.43:0/620754956 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b980d0690 con 0x557992fb16b0 2024-04-25T10:36:57.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.170+0000 7f5baf809740 1 -- 172.21.15.43:0/620754956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fb16b0 msgr2=0x55799337f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.170+0000 7f5baf809740 1 --2- 172.21.15.43:0/620754956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fb16b0 0x55799337f8f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f5b980fcd80 tx=0x7f5b98003f10 comp rx=0 tx=0).stop 2024-04-25T10:36:57.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.170+0000 7f5baf809740 1 -- 172.21.15.43:0/620754956 shutdown_connections 2024-04-25T10:36:57.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.170+0000 7f5baf809740 1 --2- 172.21.15.43:0/620754956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992fa94d0 0x557993372fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.170+0000 7f5baf809740 1 --2- 172.21.15.43:0/620754956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fb16b0 0x55799337f8f0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.170+0000 7f5baf809740 1 --2- 172.21.15.43:0/620754956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992fd2390 0x557993372a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.170+0000 7f5baf809740 1 -- 172.21.15.43:0/620754956 >> 172.21.15.43:0/620754956 conn(0x557992fc64e0 msgr2=0x55799336f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:57.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.170+0000 7f5baf809740 1 -- 172.21.15.43:0/620754956 shutdown_connections 2024-04-25T10:36:57.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.170+0000 7f5baf809740 1 -- 172.21.15.43:0/620754956 wait complete. 2024-04-25T10:36:57.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.171+0000 7f5baf809740 1 -- start start 2024-04-25T10:36:57.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.171+0000 7f5baf809740 1 Processor -- start 2024-04-25T10:36:57.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.171+0000 7f5baf809740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992fa94d0 0x55799337d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.171+0000 7f5baf809740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fb16b0 0x55799337db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.171+0000 7f5baf809740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992fd2390 0x557993383100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.171+0000 7f5baf809740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5579932d8d80 con 0x557992fb16b0 2024-04-25T10:36:57.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.171+0000 7f5baf809740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55799325fd10 con 0x557992fa94d0 2024-04-25T10:36:57.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.171+0000 7f5baf809740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55799337fe30 con 0x557992fd2390 2024-04-25T10:36:57.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.171+0000 7f5baf809740 10 client.0 _ll_register_callbacks cb 0x55799328c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:57.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.171+0000 7f5ba1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fb16b0 0x55799337db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.171+0000 7f5ba1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fb16b0 0x55799337db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45928/0 (socket says 172.21.15.43:45928) 2024-04-25T10:36:57.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.171+0000 7f5ba1d9b640 1 -- 172.21.15.43:0/3630021070 learned_addr learned my addr 172.21.15.43:0/3630021070 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:57.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.171+0000 7f5ba259c640 1 --2- 172.21.15.43:0/3630021070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992fa94d0 0x55799337d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.171+0000 7f5ba2d9d640 1 --2- 172.21.15.43:0/3630021070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992fd2390 0x557993383100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.172+0000 7f5ba1d9b640 1 -- 172.21.15.43:0/3630021070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992fd2390 msgr2=0x557993383100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.172+0000 7f5ba1d9b640 1 --2- 172.21.15.43:0/3630021070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992fd2390 0x557993383100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.172+0000 7f5ba1d9b640 1 -- 172.21.15.43:0/3630021070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992fa94d0 msgr2=0x55799337d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.172+0000 7f5ba1d9b640 1 --2- 172.21.15.43:0/3630021070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992fa94d0 0x55799337d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.172+0000 7f5ba1d9b640 1 -- 172.21.15.43:0/3630021070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579930b47d0 con 0x557992fb16b0 2024-04-25T10:36:57.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.172+0000 7f5ba2d9d640 1 --2- 172.21.15.43:0/3630021070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992fd2390 0x557993383100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:36:57.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.172+0000 7f5ba1d9b640 1 --2- 172.21.15.43:0/3630021070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fb16b0 0x55799337db20 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5b980c7870 tx=0x7f5b981052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e7a980ecb675f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:57.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.172+0000 7f5b89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:57.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.172+0000 7f5b89ffb640 1 -- 172.21.15.43:0/3630021070 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b98049070 con 0x557992fb16b0 2024-04-25T10:36:57.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.172+0000 7f5b89ffb640 1 -- 172.21.15.43:0/3630021070 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f5b980fd060 con 0x557992fb16b0 2024-04-25T10:36:57.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.172+0000 7f5baf809740 1 -- 172.21.15.43:0/3630021070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b980c73f0 con 0x557992fb16b0 2024-04-25T10:36:57.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.172+0000 7f5b89ffb640 1 -- 172.21.15.43:0/3630021070 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5b980e4050 con 0x557992fb16b0 2024-04-25T10:36:57.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.172+0000 7f5baf809740 10 client.16701 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:57.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.172+0000 7f5baf809740 1 -- 172.21.15.43:0/3630021070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557992fea3f0 con 0x557992fb16b0 2024-04-25T10:36:57.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.172+0000 7f5b89ffb640 1 -- 172.21.15.43:0/3630021070 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b980c73f0 con 0x557992fb16b0 2024-04-25T10:36:57.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.172+0000 7f5b89ffb640 1 -- 172.21.15.43:0/3630021070 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5b980d0200 con 0x557992fb16b0 2024-04-25T10:36:57.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.173+0000 7f5baf809740 10 client.16701 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:57.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.173+0000 7f5baf809740 10 client.16701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:57.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.173+0000 7f5baf809740 20 client.16701 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:57.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.173+0000 7f5baf809740 1 --2- 172.21.15.43:0/3630021070 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557992f90f60 0x55799338f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.173+0000 7f5baf809740 4 client.16701 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:36:57.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.173+0000 7f5baf809740 1 -- 172.21.15.43:0/3630021070 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579932e6990 con 0x557992f90f60 2024-04-25T10:36:57.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.173+0000 7f5ba259c640 1 --2- 172.21.15.43:0/3630021070 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557992f90f60 0x55799338f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.173+0000 7f5b89ffb640 10 client.16701 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:57.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.173+0000 7f5ba259c640 1 --2- 172.21.15.43:0/3630021070 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557992f90f60 0x55799338f1c0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15f6b57b98310da server_cookie=a29d719692a58cff in_seq=0 out_seq=0 2024-04-25T10:36:57.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.174+0000 7f5b89ffb640 1 -- 172.21.15.43:0/3630021070 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5579932e6990 con 0x557992f90f60 2024-04-25T10:36:57.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.174+0000 7f5b89ffb640 10 client.16701 handle_command_reply: tid=0 2024-04-25T10:36:57.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.174+0000 7f5baf809740 1 client.16701 shutdown 2024-04-25T10:36:57.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.174+0000 7f5baf809740 1 -- 172.21.15.43:0/3630021070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fb16b0 msgr2=0x55799337db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.174+0000 7f5baf809740 1 --2- 172.21.15.43:0/3630021070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fb16b0 0x55799337db20 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5b980c7870 tx=0x7f5b981052b0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.174+0000 7f5baf809740 1 -- 172.21.15.43:0/3630021070 shutdown_connections 2024-04-25T10:36:57.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.174+0000 7f5baf809740 1 --2- 172.21.15.43:0/3630021070 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557992f90f60 0x55799338f1c0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.174+0000 7f5baf809740 1 --2- 172.21.15.43:0/3630021070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992fd2390 0x557993383100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.174+0000 7f5baf809740 1 --2- 172.21.15.43:0/3630021070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fb16b0 0x55799337db20 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.174+0000 7f5b89ffb640 0 client.16701 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:57.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.174+0000 7f5baf809740 1 --2- 172.21.15.43:0/3630021070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992fa94d0 0x55799337d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.174+0000 7f5baf809740 1 -- 172.21.15.43:0/3630021070 >> 172.21.15.43:0/3630021070 conn(0x557992fc64e0 msgr2=0x557993374780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:57.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.174+0000 7f5baf809740 1 -- 172.21.15.43:0/3630021070 shutdown_connections 2024-04-25T10:36:57.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.174+0000 7f5baf809740 1 -- 172.21.15.43:0/3630021070 wait complete. 2024-04-25T10:36:57.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.174+0000 7f5baf809740 20 client.16701 trim_cache size 0 max 16384 2024-04-25T10:36:57.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.192+0000 7f5baf809740 1 Processor -- start 2024-04-25T10:36:57.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.193+0000 7f5baf809740 1 -- start start 2024-04-25T10:36:57.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.193+0000 7f5baf809740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992fc64e0 0x55799337f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.193+0000 7f5baf809740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992f90f60 0x557993394850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.193+0000 7f5baf809740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fa94d0 0x557993394d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.193+0000 7f5baf809740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba41b2250 con 0x557992fa94d0 2024-04-25T10:36:57.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.193+0000 7f5baf809740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba41afb20 con 0x557992fc64e0 2024-04-25T10:36:57.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.193+0000 7f5baf809740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba41ad970 con 0x557992f90f60 2024-04-25T10:36:57.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.193+0000 7f5ba2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fa94d0 0x557993394d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.193+0000 7f5ba2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fa94d0 0x557993394d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45930/0 (socket says 172.21.15.43:45930) 2024-04-25T10:36:57.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.193+0000 7f5ba2d9d640 1 -- 172.21.15.43:0/3566752636 learned_addr learned my addr 172.21.15.43:0/3566752636 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.193+0000 7f5ba259c640 1 --2- 172.21.15.43:0/3566752636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992fc64e0 0x55799337f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.193+0000 7f5ba1d9b640 1 --2- 172.21.15.43:0/3566752636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992f90f60 0x557993394850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.193+0000 7f5ba2d9d640 1 -- 172.21.15.43:0/3566752636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992f90f60 msgr2=0x557993394850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.193+0000 7f5ba2d9d640 1 --2- 172.21.15.43:0/3566752636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992f90f60 0x557993394850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.193+0000 7f5ba2d9d640 1 -- 172.21.15.43:0/3566752636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992fc64e0 msgr2=0x55799337f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.193+0000 7f5ba2d9d640 1 --2- 172.21.15.43:0/3566752636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992fc64e0 0x55799337f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.193+0000 7f5ba2d9d640 1 -- 172.21.15.43:0/3566752636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557992faa0f0 con 0x557992fa94d0 2024-04-25T10:36:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.193+0000 7f5ba2d9d640 1 --2- 172.21.15.43:0/3566752636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fa94d0 0x557993394d90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0124e0 tx=0x7f5b9c00b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2526f655ad5263a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.194+0000 7f5b8bfff640 1 -- 172.21.15.43:0/3566752636 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c0044d0 con 0x557992fa94d0 2024-04-25T10:36:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.194+0000 7f5b8bfff640 1 -- 172.21.15.43:0/3566752636 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b9c025070 con 0x557992fa94d0 2024-04-25T10:36:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.194+0000 7f5b8bfff640 1 -- 172.21.15.43:0/3566752636 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c004a00 con 0x557992fa94d0 2024-04-25T10:36:57.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.194+0000 7f5baf809740 1 -- 172.21.15.43:0/3566752636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fa94d0 msgr2=0x557993394d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.194+0000 7f5baf809740 1 --2- 172.21.15.43:0/3566752636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fa94d0 0x557993394d90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0124e0 tx=0x7f5b9c00b600 comp rx=0 tx=0).stop 2024-04-25T10:36:57.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.194+0000 7f5baf809740 1 -- 172.21.15.43:0/3566752636 shutdown_connections 2024-04-25T10:36:57.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.194+0000 7f5baf809740 1 --2- 172.21.15.43:0/3566752636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fa94d0 0x557993394d90 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.194+0000 7f5baf809740 1 --2- 172.21.15.43:0/3566752636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992f90f60 0x557993394850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.194+0000 7f5baf809740 1 --2- 172.21.15.43:0/3566752636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992fc64e0 0x55799337f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.194+0000 7f5baf809740 1 -- 172.21.15.43:0/3566752636 >> 172.21.15.43:0/3566752636 conn(0x557993372fc0 msgr2=0x55799337eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:57.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.194+0000 7f5baf809740 1 -- 172.21.15.43:0/3566752636 shutdown_connections 2024-04-25T10:36:57.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.194+0000 7f5baf809740 1 -- 172.21.15.43:0/3566752636 wait complete. 2024-04-25T10:36:57.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.195+0000 7f5baf809740 1 -- start start 2024-04-25T10:36:57.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.195+0000 7f5baf809740 1 Processor -- start 2024-04-25T10:36:57.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.195+0000 7f5baf809740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992f90f60 0x557993389720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.195+0000 7f5baf809740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992fa94d0 0x557993386a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.195+0000 7f5baf809740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fc64e0 0x557993386fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.195+0000 7f5baf809740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba41adfe0 con 0x557992fc64e0 2024-04-25T10:36:57.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.195+0000 7f5baf809740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba41af450 con 0x557992f90f60 2024-04-25T10:36:57.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.195+0000 7f5baf809740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba41ae6b0 con 0x557992fa94d0 2024-04-25T10:36:57.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.195+0000 7f5baf809740 10 client.0 _ll_register_callbacks cb 0x7f5b94009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:57.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.195+0000 7f5ba2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fc64e0 0x557993386fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.195+0000 7f5ba2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fc64e0 0x557993386fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45942/0 (socket says 172.21.15.43:45942) 2024-04-25T10:36:57.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.195+0000 7f5ba2d9d640 1 -- 172.21.15.43:0/865662525 learned_addr learned my addr 172.21.15.43:0/865662525 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:57.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.195+0000 7f5ba259c640 1 --2- 172.21.15.43:0/865662525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992f90f60 0x557993389720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.195+0000 7f5ba1d9b640 1 --2- 172.21.15.43:0/865662525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992fa94d0 0x557993386a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.196+0000 7f5ba2d9d640 1 -- 172.21.15.43:0/865662525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992fa94d0 msgr2=0x557993386a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.196+0000 7f5ba2d9d640 1 --2- 172.21.15.43:0/865662525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992fa94d0 0x557993386a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.196+0000 7f5ba2d9d640 1 -- 172.21.15.43:0/865662525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992f90f60 msgr2=0x557993389720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.196+0000 7f5ba2d9d640 1 --2- 172.21.15.43:0/865662525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992f90f60 0x557993389720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.196+0000 7f5ba2d9d640 1 -- 172.21.15.43:0/865662525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579932994b0 con 0x557992fc64e0 2024-04-25T10:36:57.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.196+0000 7f5ba259c640 1 --2- 172.21.15.43:0/865662525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992f90f60 0x557993389720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:36:57.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.196+0000 7f5ba2d9d640 1 --2- 172.21.15.43:0/865662525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fc64e0 0x557993386fd0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c012ff0 tx=0x7f5b9c009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba269e7a12450966 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:57.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.196+0000 7f5b89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:57.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.196+0000 7f5b89ffb640 1 -- 172.21.15.43:0/865662525 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c003800 con 0x557992fc64e0 2024-04-25T10:36:57.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.196+0000 7f5baf809740 1 -- 172.21.15.43:0/865662525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b9c025070 con 0x557992fc64e0 2024-04-25T10:36:57.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.196+0000 7f5b89ffb640 1 -- 172.21.15.43:0/865662525 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f5b9c0039a0 con 0x557992fc64e0 2024-04-25T10:36:57.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.196+0000 7f5b89ffb640 1 -- 172.21.15.43:0/865662525 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5b9c003be0 con 0x557992fc64e0 2024-04-25T10:36:57.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.196+0000 7f5baf809740 10 client.16707 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:57.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.196+0000 7f5baf809740 1 -- 172.21.15.43:0/865662525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5579933f6e60 con 0x557992fc64e0 2024-04-25T10:36:57.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.196+0000 7f5b89ffb640 1 -- 172.21.15.43:0/865662525 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b9c025070 con 0x557992fc64e0 2024-04-25T10:36:57.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.197+0000 7f5b89ffb640 1 -- 172.21.15.43:0/865662525 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5b9c030020 con 0x557992fc64e0 2024-04-25T10:36:57.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.197+0000 7f5baf809740 10 client.16707 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:57.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.197+0000 7f5baf809740 10 client.16707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:57.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.197+0000 7f5baf809740 20 client.16707 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:57.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.197+0000 7f5baf809740 1 --2- 172.21.15.43:0/865662525 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557992fb16b0 0x5579933f9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.197+0000 7f5baf809740 4 client.16707 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:36:57.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.197+0000 7f5baf809740 1 -- 172.21.15.43:0/865662525 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5579932e0800 con 0x557992fb16b0 2024-04-25T10:36:57.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.197+0000 7f5ba259c640 1 --2- 172.21.15.43:0/865662525 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557992fb16b0 0x5579933f9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.197+0000 7f5ba259c640 1 --2- 172.21.15.43:0/865662525 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557992fb16b0 0x5579933f9510 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5624f7cbd17e63ee server_cookie=ff5e6fafa47730f5 in_seq=0 out_seq=0 2024-04-25T10:36:57.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.197+0000 7f5b89ffb640 10 client.16707 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:57.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.198+0000 7f5b89ffb640 1 -- 172.21.15.43:0/865662525 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5579932e0800 con 0x557992fb16b0 2024-04-25T10:36:57.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.198+0000 7f5b89ffb640 10 client.16707 handle_command_reply: tid=0 2024-04-25T10:36:57.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.198+0000 7f5baf809740 1 client.16707 shutdown 2024-04-25T10:36:57.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.198+0000 7f5baf809740 1 -- 172.21.15.43:0/865662525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fc64e0 msgr2=0x557993386fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.198+0000 7f5baf809740 1 --2- 172.21.15.43:0/865662525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fc64e0 0x557993386fd0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c012ff0 tx=0x7f5b9c009c90 comp rx=0 tx=0).stop 2024-04-25T10:36:57.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.198+0000 7f5baf809740 1 -- 172.21.15.43:0/865662525 shutdown_connections 2024-04-25T10:36:57.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.198+0000 7f5baf809740 1 --2- 172.21.15.43:0/865662525 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557992fb16b0 0x5579933f9510 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.198+0000 7f5baf809740 1 --2- 172.21.15.43:0/865662525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557992fc64e0 0x557993386fd0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.198+0000 7f5baf809740 1 --2- 172.21.15.43:0/865662525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557992fa94d0 0x557993386a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.198+0000 7f5baf809740 1 --2- 172.21.15.43:0/865662525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557992f90f60 0x557993389720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.198+0000 7f5baf809740 1 -- 172.21.15.43:0/865662525 >> 172.21.15.43:0/865662525 conn(0x557993372fc0 msgr2=0x55799336e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:57.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.198+0000 7f5b89ffb640 0 client.16707 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:57.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.198+0000 7f5baf809740 1 -- 172.21.15.43:0/865662525 shutdown_connections 2024-04-25T10:36:57.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.198+0000 7f5baf809740 1 -- 172.21.15.43:0/865662525 wait complete. 2024-04-25T10:36:57.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.198+0000 7f5baf809740 20 client.16707 trim_cache size 0 max 16384 2024-04-25T10:36:57.206 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:36:57.206 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:36:57.206 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:36:57.207 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:36:57.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.199+0000 7f5ba8957640 1 -- 172.21.15.43:0/709451022 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5b8c0afc60 msgr2=0x7f5b8c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.199+0000 7f5ba8957640 1 --2- 172.21.15.43:0/709451022 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5b8c0afc60 0x7f5b8c0b2170 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55799336fbd0 tx=0x7f5b9c00b2e0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.199+0000 7f5ba8957640 1 -- 172.21.15.43:0/709451022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ba410b680 msgr2=0x7f5ba419ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.199+0000 7f5ba8957640 1 --2- 172.21.15.43:0/709451022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ba410b680 0x7f5ba419ac90 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5b98005940 tx=0x7f5b98004270 comp rx=0 tx=0).stop 2024-04-25T10:36:57.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.199+0000 7f5ba8957640 1 -- 172.21.15.43:0/709451022 shutdown_connections 2024-04-25T10:36:57.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.199+0000 7f5ba8957640 1 --2- 172.21.15.43:0/709451022 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5b8c0afc60 0x7f5b8c0b2170 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.199+0000 7f5ba8957640 1 --2- 172.21.15.43:0/709451022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba410e070 0x7f5ba419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.199+0000 7f5ba8957640 1 --2- 172.21.15.43:0/709451022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ba410b680 0x7f5ba419ac90 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.199+0000 7f5ba8957640 1 --2- 172.21.15.43:0/709451022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ba4104c20 0x7f5ba419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.199+0000 7f5ba8957640 1 -- 172.21.15.43:0/709451022 >> 172.21.15.43:0/709451022 conn(0x7f5ba4078180 msgr2=0x7f5ba41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:57.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.199+0000 7f5ba8957640 1 -- 172.21.15.43:0/709451022 shutdown_connections 2024-04-25T10:36:57.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.199+0000 7f5ba8957640 1 -- 172.21.15.43:0/709451022 wait complete. 2024-04-25T10:36:57.212 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:36:57.212 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:36:57.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.322+0000 7f839fc42640 1 -- 172.21.15.43:0/866729542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8398104bf0 msgr2=0x7f8398105000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.322+0000 7f839fc42640 1 --2- 172.21.15.43:0/866729542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8398104bf0 0x7f8398105000 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8394009900 tx=0x7f839402a410 comp rx=0 tx=0).stop 2024-04-25T10:36:57.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.322+0000 7f839fc42640 1 -- 172.21.15.43:0/866729542 shutdown_connections 2024-04-25T10:36:57.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.322+0000 7f839fc42640 1 --2- 172.21.15.43:0/866729542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f839810e040 0x7f8398110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.322+0000 7f839fc42640 1 --2- 172.21.15.43:0/866729542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f839810b650 0x7f839810da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.322+0000 7f839fc42640 1 --2- 172.21.15.43:0/866729542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8398104bf0 0x7f8398105000 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.322+0000 7f839fc42640 1 -- 172.21.15.43:0/866729542 >> 172.21.15.43:0/866729542 conn(0x7f8398078110 msgr2=0x7f8398078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:57.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.322+0000 7f839fc42640 1 -- 172.21.15.43:0/866729542 shutdown_connections 2024-04-25T10:36:57.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.322+0000 7f839fc42640 1 -- 172.21.15.43:0/866729542 wait complete. 2024-04-25T10:36:57.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.323+0000 7f839fc42640 1 Processor -- start 2024-04-25T10:36:57.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.323+0000 7f839fc42640 1 -- start start 2024-04-25T10:36:57.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.323+0000 7f839fc42640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8398104bf0 0x7f839819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.323+0000 7f839fc42640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f839810b650 0x7f839819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.323+0000 7f839fc42640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839810e040 0x7f839819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.323+0000 7f839fc42640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8398113ca0 con 0x7f839810e040 2024-04-25T10:36:57.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.323+0000 7f839fc42640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8398113b20 con 0x7f839810b650 2024-04-25T10:36:57.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.323+0000 7f839fc42640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8398113e20 con 0x7f8398104bf0 2024-04-25T10:36:57.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.323+0000 7f839e1df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839810e040 0x7f839819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.323+0000 7f839e1df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839810e040 0x7f839819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45970/0 (socket says 172.21.15.43:45970) 2024-04-25T10:36:57.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.323+0000 7f839e1df640 1 -- 172.21.15.43:0/305130498 learned_addr learned my addr 172.21.15.43:0/305130498 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:57.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.323+0000 7f839d9de640 1 --2- 172.21.15.43:0/305130498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8398104bf0 0x7f839819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.324+0000 7f839d1dd640 1 --2- 172.21.15.43:0/305130498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f839810b650 0x7f839819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.324+0000 7f839e1df640 1 -- 172.21.15.43:0/305130498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8398104bf0 msgr2=0x7f839819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.324+0000 7f839e1df640 1 --2- 172.21.15.43:0/305130498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8398104bf0 0x7f839819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.324+0000 7f839e1df640 1 -- 172.21.15.43:0/305130498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f839810b650 msgr2=0x7f839819ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.324+0000 7f839e1df640 1 --2- 172.21.15.43:0/305130498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f839810b650 0x7f839819ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.324+0000 7f839e1df640 1 -- 172.21.15.43:0/305130498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839819f500 con 0x7f839810e040 2024-04-25T10:36:57.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.324+0000 7f839d1dd640 1 --2- 172.21.15.43:0/305130498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f839810b650 0x7f839819ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:36:57.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.324+0000 7f839d9de640 1 --2- 172.21.15.43:0/305130498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8398104bf0 0x7f839819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:57.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.324+0000 7f839e1df640 1 --2- 172.21.15.43:0/305130498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839810e040 0x7f839819eec0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f839000eb10 tx=0x7f839000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:57.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.324+0000 7f838effd640 1 -- 172.21.15.43:0/305130498 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390004b20 con 0x7f839810e040 2024-04-25T10:36:57.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.324+0000 7f838effd640 1 -- 172.21.15.43:0/305130498 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8390004cc0 con 0x7f839810e040 2024-04-25T10:36:57.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.324+0000 7f839fc42640 1 -- 172.21.15.43:0/305130498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839819f7f0 con 0x7f839810e040 2024-04-25T10:36:57.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.324+0000 7f838effd640 1 -- 172.21.15.43:0/305130498 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390010670 con 0x7f839810e040 2024-04-25T10:36:57.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.324+0000 7f839fc42640 1 -- 172.21.15.43:0/305130498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f839819fcf0 con 0x7f839810e040 2024-04-25T10:36:57.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.325+0000 7f839fc42640 1 -- 172.21.15.43:0/305130498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839806afa0 con 0x7f839810e040 2024-04-25T10:36:57.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.326+0000 7f838effd640 1 -- 172.21.15.43:0/305130498 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f8390005000 con 0x7f839810e040 2024-04-25T10:36:57.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.327+0000 7f838effd640 1 --2- 172.21.15.43:0/305130498 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f83840afc60 0x7f83840b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.327+0000 7f839d9de640 1 --2- 172.21.15.43:0/305130498 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f83840afc60 0x7f83840b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.327+0000 7f838effd640 1 -- 172.21.15.43:0/305130498 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f83900df8a0 con 0x7f839810e040 2024-04-25T10:36:57.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.328+0000 7f839d9de640 1 --2- 172.21.15.43:0/305130498 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f83840afc60 0x7f83840b2170 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8394031440 tx=0x7f8394009680 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:57.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.331+0000 7f838effd640 1 -- 172.21.15.43:0/305130498 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8390053ff0 con 0x7f839810e040 2024-04-25T10:36:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.559+0000 7f839fc42640 1 -- 172.21.15.43:0/305130498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f839819b9f0 con 0x7f839810e040 2024-04-25T10:36:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.559+0000 7f838effd640 1 -- 172.21.15.43:0/305130498 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7f839819fcf0 con 0x7f839810e040 2024-04-25T10:36:57.562 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:36:57.562 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:36:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:36:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.561+0000 7f839fc42640 1 -- 172.21.15.43:0/305130498 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f83840afc60 msgr2=0x7f83840b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.561+0000 7f839fc42640 1 --2- 172.21.15.43:0/305130498 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f83840afc60 0x7f83840b2170 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8394031440 tx=0x7f8394009680 comp rx=0 tx=0).stop 2024-04-25T10:36:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.561+0000 7f839fc42640 1 -- 172.21.15.43:0/305130498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839810e040 msgr2=0x7f839819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.561+0000 7f839fc42640 1 --2- 172.21.15.43:0/305130498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839810e040 0x7f839819eec0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f839000eb10 tx=0x7f839000c2d0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.561+0000 7f839fc42640 1 -- 172.21.15.43:0/305130498 shutdown_connections 2024-04-25T10:36:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.561+0000 7f839fc42640 1 --2- 172.21.15.43:0/305130498 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f83840afc60 0x7f83840b2170 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.561+0000 7f839fc42640 1 --2- 172.21.15.43:0/305130498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f839810b650 0x7f839819ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.561+0000 7f839fc42640 1 --2- 172.21.15.43:0/305130498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839810e040 0x7f839819eec0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.561+0000 7f839fc42640 1 --2- 172.21.15.43:0/305130498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8398104bf0 0x7f839819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.561+0000 7f839fc42640 1 -- 172.21.15.43:0/305130498 >> 172.21.15.43:0/305130498 conn(0x7f8398078110 msgr2=0x7f8398101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.562+0000 7f839fc42640 1 -- 172.21.15.43:0/305130498 shutdown_connections 2024-04-25T10:36:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.562+0000 7f839fc42640 1 -- 172.21.15.43:0/305130498 wait complete. 2024-04-25T10:36:57.575 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:36:57.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.688+0000 7ff723c97640 1 -- 172.21.15.43:0/2439586317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff71c109b10 msgr2=0x7ff71c1036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.688+0000 7ff723c97640 1 --2- 172.21.15.43:0/2439586317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff71c109b10 0x7ff71c1036b0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff718009900 tx=0x7ff71802a410 comp rx=0 tx=0).stop 2024-04-25T10:36:57.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.688+0000 7ff723c97640 1 -- 172.21.15.43:0/2439586317 shutdown_connections 2024-04-25T10:36:57.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.688+0000 7ff723c97640 1 --2- 172.21.15.43:0/2439586317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff71c109fb0 0x7ff71c103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.688+0000 7ff723c97640 1 --2- 172.21.15.43:0/2439586317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff71c104260 0x7ff71c110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.688+0000 7ff723c97640 1 --2- 172.21.15.43:0/2439586317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff71c109b10 0x7ff71c1036b0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.688+0000 7ff723c97640 1 -- 172.21.15.43:0/2439586317 >> 172.21.15.43:0/2439586317 conn(0x7ff71c0ff210 msgr2=0x7ff71c101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:57.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.688+0000 7ff723c97640 1 -- 172.21.15.43:0/2439586317 shutdown_connections 2024-04-25T10:36:57.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.688+0000 7ff723c97640 1 -- 172.21.15.43:0/2439586317 wait complete. 2024-04-25T10:36:57.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.689+0000 7ff723c97640 1 Processor -- start 2024-04-25T10:36:57.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.689+0000 7ff723c97640 1 -- start start 2024-04-25T10:36:57.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.689+0000 7ff723c97640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff71c104260 0x7ff71c19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.689+0000 7ff723c97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff71c109b10 0x7ff71c19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.689+0000 7ff723c97640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff71c109fb0 0x7ff71c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.689+0000 7ff723c97640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c113d00 con 0x7ff71c109b10 2024-04-25T10:36:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.689+0000 7ff723c97640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c113b80 con 0x7ff71c109fb0 2024-04-25T10:36:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.689+0000 7ff723c97640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c113e80 con 0x7ff71c104260 2024-04-25T10:36:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.689+0000 7ff721232640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff71c109b10 0x7ff71c19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.689+0000 7ff721232640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff71c109b10 0x7ff71c19ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45988/0 (socket says 172.21.15.43:45988) 2024-04-25T10:36:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.690+0000 7ff721232640 1 -- 172.21.15.43:0/1733551575 learned_addr learned my addr 172.21.15.43:0/1733551575 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.690+0000 7ff721a33640 1 --2- 172.21.15.43:0/1733551575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff71c104260 0x7ff71c19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.690+0000 7ff722234640 1 --2- 172.21.15.43:0/1733551575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff71c109fb0 0x7ff71c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.690+0000 7ff721232640 1 -- 172.21.15.43:0/1733551575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff71c104260 msgr2=0x7ff71c19a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.690+0000 7ff721232640 1 --2- 172.21.15.43:0/1733551575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff71c104260 0x7ff71c19a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.690+0000 7ff721232640 1 -- 172.21.15.43:0/1733551575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff71c109fb0 msgr2=0x7ff71c19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.690+0000 7ff721232640 1 --2- 172.21.15.43:0/1733551575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff71c109fb0 0x7ff71c19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.690+0000 7ff721232640 1 -- 172.21.15.43:0/1733551575 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71c19f5e0 con 0x7ff71c109b10 2024-04-25T10:36:57.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.690+0000 7ff722234640 1 --2- 172.21.15.43:0/1733551575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff71c109fb0 0x7ff71c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:57.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.690+0000 7ff721232640 1 --2- 172.21.15.43:0/1733551575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff71c109b10 0x7ff71c19ad60 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff70c00d770 tx=0x7ff70c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:57.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.690+0000 7ff712ffd640 1 -- 172.21.15.43:0/1733551575 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c017070 con 0x7ff71c109b10 2024-04-25T10:36:57.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.690+0000 7ff712ffd640 1 -- 172.21.15.43:0/1733551575 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff70c00de80 con 0x7ff71c109b10 2024-04-25T10:36:57.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.690+0000 7ff712ffd640 1 -- 172.21.15.43:0/1733551575 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c003c20 con 0x7ff71c109b10 2024-04-25T10:36:57.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.690+0000 7ff723c97640 1 -- 172.21.15.43:0/1733551575 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71c19f8d0 con 0x7ff71c109b10 2024-04-25T10:36:57.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.690+0000 7ff723c97640 1 -- 172.21.15.43:0/1733551575 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff71c19fe20 con 0x7ff71c109b10 2024-04-25T10:36:57.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.693+0000 7ff72ac22740 1 Processor -- start 2024-04-25T10:36:57.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.693+0000 7ff72ac22740 1 -- start start 2024-04-25T10:36:57.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.693+0000 7ff712ffd640 1 -- 172.21.15.43:0/1733551575 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7ff70c003dc0 con 0x7ff71c109b10 2024-04-25T10:36:57.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.693+0000 7ff72ac22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa82390 0x56136ae23810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.693+0000 7ff72ac22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa594d0 0x56136ae23d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.693+0000 7ff72ac22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56136aa616b0 0x56136ae306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.693+0000 7ff72ac22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56136ad88d80 con 0x56136aa594d0 2024-04-25T10:36:57.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.693+0000 7ff72ac22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56136ad42a20 con 0x56136aa82390 2024-04-25T10:36:57.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.693+0000 7ff72ac22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56136ad84d40 con 0x56136aa616b0 2024-04-25T10:36:57.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.693+0000 7ff721a33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa594d0 0x56136ae23d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.694+0000 7ff721a33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa594d0 0x56136ae23d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45998/0 (socket says 172.21.15.43:45998) 2024-04-25T10:36:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.694+0000 7ff721a33640 1 -- 172.21.15.43:0/3997686088 learned_addr learned my addr 172.21.15.43:0/3997686088 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.694+0000 7ff721232640 1 --2- 172.21.15.43:0/3997686088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56136aa616b0 0x56136ae306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.694+0000 7ff722234640 1 --2- 172.21.15.43:0/3997686088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa82390 0x56136ae23810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.694+0000 7ff721a33640 1 -- 172.21.15.43:0/3997686088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56136aa616b0 msgr2=0x56136ae306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.694+0000 7ff721a33640 1 --2- 172.21.15.43:0/3997686088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56136aa616b0 0x56136ae306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.694+0000 7ff721a33640 1 -- 172.21.15.43:0/3997686088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa82390 msgr2=0x56136ae23810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.694+0000 7ff721a33640 1 --2- 172.21.15.43:0/3997686088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa82390 0x56136ae23810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.694+0000 7ff721a33640 1 -- 172.21.15.43:0/3997686088 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56136ad4dc30 con 0x56136aa594d0 2024-04-25T10:36:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.694+0000 7ff712ffd640 1 --2- 172.21.15.43:0/1733551575 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff7080afc60 0x7ff7080b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.694+0000 7ff721a33640 1 --2- 172.21.15.43:0/3997686088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa594d0 0x56136ae23d50 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff718009900 tx=0x7ff718032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a4ae5f12d6b2ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.694+0000 7ff722234640 1 --2- 172.21.15.43:0/1733551575 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff7080afc60 0x7ff7080b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.694+0000 7ff712ffd640 1 -- 172.21.15.43:0/1733551575 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7ff70c0de9f0 con 0x7ff71c109b10 2024-04-25T10:36:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.694+0000 7ff707fff640 1 -- 172.21.15.43:0/3997686088 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff718040070 con 0x56136aa594d0 2024-04-25T10:36:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.694+0000 7ff707fff640 1 -- 172.21.15.43:0/3997686088 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff718005db0 con 0x56136aa594d0 2024-04-25T10:36:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.694+0000 7ff707fff640 1 -- 172.21.15.43:0/3997686088 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff718035040 con 0x56136aa594d0 2024-04-25T10:36:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.695+0000 7ff72ac22740 1 -- 172.21.15.43:0/3997686088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa594d0 msgr2=0x56136ae23d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.695+0000 7ff722234640 1 --2- 172.21.15.43:0/1733551575 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff7080afc60 0x7ff7080b2170 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x56136ae20980 tx=0x7ff714007720 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.695+0000 7ff72ac22740 1 --2- 172.21.15.43:0/3997686088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa594d0 0x56136ae23d50 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff718009900 tx=0x7ff718032df0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.695+0000 7ff72ac22740 1 -- 172.21.15.43:0/3997686088 shutdown_connections 2024-04-25T10:36:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.695+0000 7ff72ac22740 1 --2- 172.21.15.43:0/3997686088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56136aa616b0 0x56136ae306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.695+0000 7ff72ac22740 1 --2- 172.21.15.43:0/3997686088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa594d0 0x56136ae23d50 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.695+0000 7ff72ac22740 1 --2- 172.21.15.43:0/3997686088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa82390 0x56136ae23810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.695+0000 7ff72ac22740 1 -- 172.21.15.43:0/3997686088 >> 172.21.15.43:0/3997686088 conn(0x56136aa764e0 msgr2=0x56136ae1ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.695+0000 7ff72ac22740 1 -- 172.21.15.43:0/3997686088 shutdown_connections 2024-04-25T10:36:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.695+0000 7ff72ac22740 1 -- 172.21.15.43:0/3997686088 wait complete. 2024-04-25T10:36:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.696+0000 7ff72ac22740 1 -- start start 2024-04-25T10:36:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.696+0000 7ff72ac22740 1 Processor -- start 2024-04-25T10:36:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.696+0000 7ff72ac22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56136aa594d0 0x56136ae2e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.696+0000 7ff72ac22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa616b0 0x56136ae2e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.696+0000 7ff72ac22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa82390 0x56136ae3a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.696+0000 7ff72ac22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56136ae30d30 con 0x56136aa616b0 2024-04-25T10:36:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.696+0000 7ff72ac22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56136ad0fdc0 con 0x56136aa82390 2024-04-25T10:36:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.696+0000 7ff72ac22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56136ae30eb0 con 0x56136aa594d0 2024-04-25T10:36:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.696+0000 7ff72ac22740 10 client.0 _ll_register_callbacks cb 0x56136ad2b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.696+0000 7ff721232640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa616b0 0x56136ae2e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.697+0000 7ff721232640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa616b0 0x56136ae2e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46010/0 (socket says 172.21.15.43:46010) 2024-04-25T10:36:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.697+0000 7ff721232640 1 -- 172.21.15.43:0/2628009149 learned_addr learned my addr 172.21.15.43:0/2628009149 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.697+0000 7ff722234640 1 --2- 172.21.15.43:0/2628009149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa82390 0x56136ae3a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.697+0000 7ff721a33640 1 --2- 172.21.15.43:0/2628009149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56136aa594d0 0x56136ae2e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.697+0000 7ff721232640 1 -- 172.21.15.43:0/2628009149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56136aa594d0 msgr2=0x56136ae2e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.697+0000 7ff721232640 1 --2- 172.21.15.43:0/2628009149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56136aa594d0 0x56136ae2e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.697+0000 7ff721232640 1 -- 172.21.15.43:0/2628009149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa82390 msgr2=0x56136ae3a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.697+0000 7ff721232640 1 --2- 172.21.15.43:0/2628009149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa82390 0x56136ae3a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.697+0000 7ff721232640 1 -- 172.21.15.43:0/2628009149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56136aa9a3f0 con 0x56136aa616b0 2024-04-25T10:36:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.697+0000 7ff721232640 1 --2- 172.21.15.43:0/2628009149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa616b0 0x56136ae2e7e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff70c00d740 tx=0x7ff70c009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e0c292f48c4252 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.697+0000 7ff7057fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.697+0000 7ff7057fa640 1 -- 172.21.15.43:0/2628009149 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c020070 con 0x56136aa616b0 2024-04-25T10:36:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.697+0000 7ff7057fa640 1 -- 172.21.15.43:0/2628009149 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7ff70c0a2e50 con 0x56136aa616b0 2024-04-25T10:36:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.697+0000 7ff72ac22740 1 -- 172.21.15.43:0/2628009149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff718005db0 con 0x56136aa616b0 2024-04-25T10:36:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.697+0000 7ff7057fa640 1 -- 172.21.15.43:0/2628009149 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff70c0a6510 con 0x56136aa616b0 2024-04-25T10:36:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.697+0000 7ff72ac22740 10 client.16725 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.697+0000 7ff72ac22740 1 -- 172.21.15.43:0/2628009149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56136ab647d0 con 0x56136aa616b0 2024-04-25T10:36:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.698+0000 7ff7057fa640 1 -- 172.21.15.43:0/2628009149 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff718005db0 con 0x56136aa616b0 2024-04-25T10:36:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.698+0000 7ff7057fa640 1 -- 172.21.15.43:0/2628009149 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff70c0a2530 con 0x56136aa616b0 2024-04-25T10:36:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.698+0000 7ff72ac22740 10 client.16725 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.698+0000 7ff72ac22740 10 client.16725 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:36:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.698+0000 7ff72ac22740 20 client.16725 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.698+0000 7ff72ac22740 1 --2- 172.21.15.43:0/2628009149 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x56136aa40f60 0x56136ae40690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.698+0000 7ff72ac22740 4 client.16725 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:36:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.698+0000 7ff72ac22740 1 -- 172.21.15.43:0/2628009149 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56136ad96990 con 0x56136aa40f60 2024-04-25T10:36:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.698+0000 7ff721a33640 1 --2- 172.21.15.43:0/2628009149 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x56136aa40f60 0x56136ae40690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.699+0000 7ff7057fa640 10 client.16725 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:36:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.699+0000 7ff721a33640 1 --2- 172.21.15.43:0/2628009149 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x56136aa40f60 0x56136ae40690 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6b419f56958308c4 server_cookie=82014f5ec0009db6 in_seq=0 out_seq=0 2024-04-25T10:36:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.699+0000 7ff7057fa640 1 -- 172.21.15.43:0/2628009149 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56136ad96990 con 0x56136aa40f60 2024-04-25T10:36:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.699+0000 7ff7057fa640 10 client.16725 handle_command_reply: tid=0 2024-04-25T10:36:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.699+0000 7ff72ac22740 1 client.16725 shutdown 2024-04-25T10:36:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.699+0000 7ff72ac22740 1 -- 172.21.15.43:0/2628009149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa616b0 msgr2=0x56136ae2e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.699+0000 7ff72ac22740 1 --2- 172.21.15.43:0/2628009149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa616b0 0x56136ae2e7e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff70c00d740 tx=0x7ff70c009cb0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.700+0000 7ff72ac22740 1 -- 172.21.15.43:0/2628009149 shutdown_connections 2024-04-25T10:36:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.700+0000 7ff72ac22740 1 --2- 172.21.15.43:0/2628009149 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x56136aa40f60 0x56136ae40690 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.700+0000 7ff72ac22740 1 --2- 172.21.15.43:0/2628009149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa82390 0x56136ae3a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.700+0000 7ff72ac22740 1 --2- 172.21.15.43:0/2628009149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa616b0 0x56136ae2e7e0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.700+0000 7ff7057fa640 0 client.16725 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:36:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.700+0000 7ff72ac22740 1 --2- 172.21.15.43:0/2628009149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56136aa594d0 0x56136ae2e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.700+0000 7ff72ac22740 1 -- 172.21.15.43:0/2628009149 >> 172.21.15.43:0/2628009149 conn(0x56136aa764e0 msgr2=0x56136ae25530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.700+0000 7ff72ac22740 1 -- 172.21.15.43:0/2628009149 shutdown_connections 2024-04-25T10:36:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.700+0000 7ff72ac22740 1 -- 172.21.15.43:0/2628009149 wait complete. 2024-04-25T10:36:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.700+0000 7ff72ac22740 20 client.16725 trim_cache size 0 max 16384 2024-04-25T10:36:57.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.713+0000 7ff72ac22740 1 Processor -- start 2024-04-25T10:36:57.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.713+0000 7ff72ac22740 1 -- start start 2024-04-25T10:36:57.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.714+0000 7ff72ac22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa594d0 0x56136ae1ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.714+0000 7ff72ac22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa616b0 0x56136ae39e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.714+0000 7ff72ac22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56136aa82390 0x56136ae28430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.714+0000 7ff72ac22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1b2230 con 0x56136aa616b0 2024-04-25T10:36:57.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.714+0000 7ff72ac22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1afb00 con 0x56136aa594d0 2024-04-25T10:36:57.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.714+0000 7ff72ac22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1ad950 con 0x56136aa82390 2024-04-25T10:36:57.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.714+0000 7ff721232640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa616b0 0x56136ae39e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.714+0000 7ff721232640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa616b0 0x56136ae39e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46016/0 (socket says 172.21.15.43:46016) 2024-04-25T10:36:57.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.714+0000 7ff721232640 1 -- 172.21.15.43:0/3734943936 learned_addr learned my addr 172.21.15.43:0/3734943936 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:57.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.714+0000 7ff721a33640 1 --2- 172.21.15.43:0/3734943936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa594d0 0x56136ae1ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.714+0000 7ff721232640 1 -- 172.21.15.43:0/3734943936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56136aa82390 msgr2=0x56136ae28430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.714+0000 7ff721232640 1 --2- 172.21.15.43:0/3734943936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56136aa82390 0x56136ae28430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.714+0000 7ff721232640 1 -- 172.21.15.43:0/3734943936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa594d0 msgr2=0x56136ae1ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.714+0000 7ff721232640 1 --2- 172.21.15.43:0/3734943936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa594d0 0x56136ae1ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.714+0000 7ff721232640 1 -- 172.21.15.43:0/3734943936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56136ae1a2b0 con 0x56136aa616b0 2024-04-25T10:36:57.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.714+0000 7ff721a33640 1 --2- 172.21.15.43:0/3734943936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa594d0 0x56136ae1ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:57.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.714+0000 7ff721232640 1 --2- 172.21.15.43:0/3734943936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa616b0 0x56136ae39e10 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0dc770 tx=0x7ff70c009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1151ad114d2c4efa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:57.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.714+0000 7ff7077fe640 1 -- 172.21.15.43:0/3734943936 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c020070 con 0x56136aa616b0 2024-04-25T10:36:57.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.714+0000 7ff7077fe640 1 -- 172.21.15.43:0/3734943936 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff70c0c1290 con 0x56136aa616b0 2024-04-25T10:36:57.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.714+0000 7ff7077fe640 1 -- 172.21.15.43:0/3734943936 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c0a6330 con 0x56136aa616b0 2024-04-25T10:36:57.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.715+0000 7ff72ac22740 1 -- 172.21.15.43:0/3734943936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa616b0 msgr2=0x56136ae39e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.715+0000 7ff72ac22740 1 --2- 172.21.15.43:0/3734943936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa616b0 0x56136ae39e10 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0dc770 tx=0x7ff70c009740 comp rx=0 tx=0).stop 2024-04-25T10:36:57.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.715+0000 7ff72ac22740 1 -- 172.21.15.43:0/3734943936 shutdown_connections 2024-04-25T10:36:57.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.715+0000 7ff72ac22740 1 --2- 172.21.15.43:0/3734943936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56136aa82390 0x56136ae28430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.715+0000 7ff72ac22740 1 --2- 172.21.15.43:0/3734943936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa616b0 0x56136ae39e10 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.715+0000 7ff72ac22740 1 --2- 172.21.15.43:0/3734943936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa594d0 0x56136ae1ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.715+0000 7ff72ac22740 1 -- 172.21.15.43:0/3734943936 >> 172.21.15.43:0/3734943936 conn(0x56136aa764e0 msgr2=0x56136ae1f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:57.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.715+0000 7ff72ac22740 1 -- 172.21.15.43:0/3734943936 shutdown_connections 2024-04-25T10:36:57.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.715+0000 7ff72ac22740 1 -- 172.21.15.43:0/3734943936 wait complete. 2024-04-25T10:36:57.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.716+0000 7ff72ac22740 1 -- start start 2024-04-25T10:36:57.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.716+0000 7ff72ac22740 1 Processor -- start 2024-04-25T10:36:57.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.716+0000 7ff72ac22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56136aa594d0 0x56136ae36420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.716+0000 7ff72ac22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa616b0 0x56136ae31770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.716+0000 7ff72ac22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa82390 0x56136ae31cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.716+0000 7ff72ac22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1adfc0 con 0x56136aa82390 2024-04-25T10:36:57.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.716+0000 7ff72ac22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1af430 con 0x56136aa616b0 2024-04-25T10:36:57.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.716+0000 7ff72ac22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1ae690 con 0x56136aa594d0 2024-04-25T10:36:57.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.716+0000 7ff72ac22740 10 client.0 _ll_register_callbacks cb 0x7ff70c0e2230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:57.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.716+0000 7ff722234640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa82390 0x56136ae31cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.716+0000 7ff722234640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa82390 0x56136ae31cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46024/0 (socket says 172.21.15.43:46024) 2024-04-25T10:36:57.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.716+0000 7ff722234640 1 -- 172.21.15.43:0/111627078 learned_addr learned my addr 172.21.15.43:0/111627078 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:57.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.716+0000 7ff721232640 1 --2- 172.21.15.43:0/111627078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa616b0 0x56136ae31770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.716+0000 7ff722234640 1 -- 172.21.15.43:0/111627078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56136aa594d0 msgr2=0x56136ae36420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.716+0000 7ff721a33640 1 --2- 172.21.15.43:0/111627078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56136aa594d0 0x56136ae36420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.717+0000 7ff722234640 1 --2- 172.21.15.43:0/111627078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56136aa594d0 0x56136ae36420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.717+0000 7ff722234640 1 -- 172.21.15.43:0/111627078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa616b0 msgr2=0x56136ae31770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.717+0000 7ff722234640 1 --2- 172.21.15.43:0/111627078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa616b0 0x56136ae31770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.717+0000 7ff722234640 1 -- 172.21.15.43:0/111627078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56136ae40af0 con 0x56136aa82390 2024-04-25T10:36:57.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.717+0000 7ff721232640 1 --2- 172.21.15.43:0/111627078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa616b0 0x56136ae31770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:57.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.717+0000 7ff721a33640 1 --2- 172.21.15.43:0/111627078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56136aa594d0 0x56136ae36420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:36:57.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.717+0000 7ff722234640 1 --2- 172.21.15.43:0/111627078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa82390 0x56136ae31cb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff7140045e0 tx=0x7ff714004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2076d998b3b4942e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:57.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.717+0000 7ff7057fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:57.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.717+0000 7ff7057fa640 1 -- 172.21.15.43:0/111627078 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71401fe30 con 0x56136aa82390 2024-04-25T10:36:57.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.717+0000 7ff7057fa640 1 -- 172.21.15.43:0/111627078 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7ff7140273e0 con 0x56136aa82390 2024-04-25T10:36:57.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.717+0000 7ff72ac22740 1 -- 172.21.15.43:0/111627078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56136ad90800 con 0x56136aa82390 2024-04-25T10:36:57.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.717+0000 7ff7057fa640 1 -- 172.21.15.43:0/111627078 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff7140275e0 con 0x56136aa82390 2024-04-25T10:36:57.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.717+0000 7ff72ac22740 10 client.16731 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:57.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.717+0000 7ff72ac22740 1 -- 172.21.15.43:0/111627078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56136aea8350 con 0x56136aa82390 2024-04-25T10:36:57.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.717+0000 7ff7057fa640 1 -- 172.21.15.43:0/111627078 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56136ad90800 con 0x56136aa82390 2024-04-25T10:36:57.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.717+0000 7ff7057fa640 1 -- 172.21.15.43:0/111627078 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff714026070 con 0x56136aa82390 2024-04-25T10:36:57.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.718+0000 7ff72ac22740 10 client.16731 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:57.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.718+0000 7ff72ac22740 10 client.16731 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:36:57.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.718+0000 7ff72ac22740 20 client.16731 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:57.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.718+0000 7ff72ac22740 1 --2- 172.21.15.43:0/111627078 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x56136aa163a0 0x56136aeaab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.718+0000 7ff72ac22740 4 client.16731 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:36:57.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.718+0000 7ff72ac22740 1 -- 172.21.15.43:0/111627078 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56136ad0b610 con 0x56136aa163a0 2024-04-25T10:36:57.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.718+0000 7ff721a33640 1 --2- 172.21.15.43:0/111627078 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x56136aa163a0 0x56136aeaab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.718+0000 7ff7057fa640 10 client.16731 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:36:57.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.718+0000 7ff721a33640 1 --2- 172.21.15.43:0/111627078 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x56136aa163a0 0x56136aeaab90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db59fe95754d8934 server_cookie=8a2b0b85f7594001 in_seq=0 out_seq=0 2024-04-25T10:36:57.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.718+0000 7ff7057fa640 1 -- 172.21.15.43:0/111627078 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56136ad0b610 con 0x56136aa163a0 2024-04-25T10:36:57.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.718+0000 7ff7057fa640 10 client.16731 handle_command_reply: tid=0 2024-04-25T10:36:57.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.718+0000 7ff72ac22740 1 client.16731 shutdown 2024-04-25T10:36:57.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff72ac22740 1 -- 172.21.15.43:0/111627078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa82390 msgr2=0x56136ae31cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff72ac22740 1 --2- 172.21.15.43:0/111627078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa82390 0x56136ae31cb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff7140045e0 tx=0x7ff714004ab0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff72ac22740 1 -- 172.21.15.43:0/111627078 shutdown_connections 2024-04-25T10:36:57.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff72ac22740 1 --2- 172.21.15.43:0/111627078 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x56136aa163a0 0x56136aeaab90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff72ac22740 1 --2- 172.21.15.43:0/111627078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56136aa616b0 0x56136ae31770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff72ac22740 1 --2- 172.21.15.43:0/111627078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56136aa82390 0x56136ae31cb0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff72ac22740 1 --2- 172.21.15.43:0/111627078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56136aa594d0 0x56136ae36420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff7057fa640 0 client.16731 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:36:57.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff72ac22740 1 -- 172.21.15.43:0/111627078 >> 172.21.15.43:0/111627078 conn(0x56136aa764e0 msgr2=0x56136ae30090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:57.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff72ac22740 1 -- 172.21.15.43:0/111627078 shutdown_connections 2024-04-25T10:36:57.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff72ac22740 1 -- 172.21.15.43:0/111627078 wait complete. 2024-04-25T10:36:57.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff72ac22740 20 client.16731 trim_cache size 0 max 16384 2024-04-25T10:36:57.728 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:36:57.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff723c97640 1 -- 172.21.15.43:0/1733551575 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff7080afc60 msgr2=0x7ff7080b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff723c97640 1 --2- 172.21.15.43:0/1733551575 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff7080afc60 0x7ff7080b2170 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x56136ae20980 tx=0x7ff714007720 comp rx=0 tx=0).stop 2024-04-25T10:36:57.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff723c97640 1 -- 172.21.15.43:0/1733551575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff71c109b10 msgr2=0x7ff71c19ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff723c97640 1 --2- 172.21.15.43:0/1733551575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff71c109b10 0x7ff71c19ad60 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff70c00d770 tx=0x7ff70c00dc30 comp rx=0 tx=0).stop 2024-04-25T10:36:57.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff723c97640 1 -- 172.21.15.43:0/1733551575 shutdown_connections 2024-04-25T10:36:57.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff723c97640 1 --2- 172.21.15.43:0/1733551575 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff7080afc60 0x7ff7080b2170 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff723c97640 1 --2- 172.21.15.43:0/1733551575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff71c109fb0 0x7ff71c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff723c97640 1 --2- 172.21.15.43:0/1733551575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff71c109b10 0x7ff71c19ad60 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff723c97640 1 --2- 172.21.15.43:0/1733551575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff71c104260 0x7ff71c19a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff723c97640 1 -- 172.21.15.43:0/1733551575 >> 172.21.15.43:0/1733551575 conn(0x7ff71c0ff210 msgr2=0x7ff71c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:57.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff723c97640 1 -- 172.21.15.43:0/1733551575 shutdown_connections 2024-04-25T10:36:57.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.719+0000 7ff723c97640 1 -- 172.21.15.43:0/1733551575 wait complete. 2024-04-25T10:36:57.732 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:36:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.844+0000 7f7bd6a0c640 1 -- 172.21.15.43:0/1053292185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bd0104270 msgr2=0x7f7bd0110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.844+0000 7f7bd6a0c640 1 --2- 172.21.15.43:0/1053292185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bd0104270 0x7f7bd0110640 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8009ca0 tx=0x7f7bc802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.845+0000 7f7bd6a0c640 1 -- 172.21.15.43:0/1053292185 shutdown_connections 2024-04-25T10:36:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.845+0000 7f7bd6a0c640 1 --2- 172.21.15.43:0/1053292185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bd0104270 0x7f7bd0110640 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.845+0000 7f7bd6a0c640 1 --2- 172.21.15.43:0/1053292185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bd0109fc0 0x7f7bd0103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.845+0000 7f7bd6a0c640 1 --2- 172.21.15.43:0/1053292185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bd0109b20 0x7f7bd01036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.845+0000 7f7bd6a0c640 1 -- 172.21.15.43:0/1053292185 >> 172.21.15.43:0/1053292185 conn(0x7f7bd00ff1e0 msgr2=0x7f7bd0101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.845+0000 7f7bd6a0c640 1 -- 172.21.15.43:0/1053292185 shutdown_connections 2024-04-25T10:36:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.845+0000 7f7bd6a0c640 1 -- 172.21.15.43:0/1053292185 wait complete. 2024-04-25T10:36:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.845+0000 7f7bd6a0c640 1 Processor -- start 2024-04-25T10:36:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.845+0000 7f7bd6a0c640 1 -- start start 2024-04-25T10:36:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.846+0000 7f7bd6a0c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bd0104270 0x7f7bd019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.846+0000 7f7bd6a0c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bd0109b20 0x7f7bd019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.846+0000 7f7bd6a0c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bd0109fc0 0x7f7bd019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.846+0000 7f7bd6a0c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd0113d30 con 0x7f7bd0104270 2024-04-25T10:36:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.846+0000 7f7bd6a0c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd0113bb0 con 0x7f7bd0109b20 2024-04-25T10:36:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.846+0000 7f7bd6a0c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd0113eb0 con 0x7f7bd0109fc0 2024-04-25T10:36:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.846+0000 7f7bcffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bd0104270 0x7f7bd019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.846+0000 7f7bcffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bd0104270 0x7f7bd019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46040/0 (socket says 172.21.15.43:46040) 2024-04-25T10:36:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.846+0000 7f7bcffff640 1 -- 172.21.15.43:0/3714798507 learned_addr learned my addr 172.21.15.43:0/3714798507 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.846+0000 7f7bcf7fe640 1 --2- 172.21.15.43:0/3714798507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bd0109b20 0x7f7bd019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.846+0000 7f7bd4fa9640 1 --2- 172.21.15.43:0/3714798507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bd0109fc0 0x7f7bd019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.846+0000 7f7bcffff640 1 -- 172.21.15.43:0/3714798507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bd0109fc0 msgr2=0x7f7bd019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.846+0000 7f7bcffff640 1 --2- 172.21.15.43:0/3714798507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bd0109fc0 0x7f7bd019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.846+0000 7f7bcffff640 1 -- 172.21.15.43:0/3714798507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bd0109b20 msgr2=0x7f7bd019ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.846+0000 7f7bcffff640 1 --2- 172.21.15.43:0/3714798507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bd0109b20 0x7f7bd019ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.846+0000 7f7bcffff640 1 -- 172.21.15.43:0/3714798507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd019f570 con 0x7f7bd0104270 2024-04-25T10:36:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.846+0000 7f7bd4fa9640 1 --2- 172.21.15.43:0/3714798507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bd0109fc0 0x7f7bd019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:36:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.846+0000 7f7bcffff640 1 --2- 172.21.15.43:0/3714798507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bd0104270 0x7f7bd019a710 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0005fd0 tx=0x7f7bc000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:57.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.847+0000 7f7bcd7fa640 1 -- 172.21.15.43:0/3714798507 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0017070 con 0x7f7bd0104270 2024-04-25T10:36:57.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.847+0000 7f7bcd7fa640 1 -- 172.21.15.43:0/3714798507 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc0009c30 con 0x7f7bd0104270 2024-04-25T10:36:57.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.847+0000 7f7bd6a0c640 1 -- 172.21.15.43:0/3714798507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd019f860 con 0x7f7bd0104270 2024-04-25T10:36:57.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.847+0000 7f7bcd7fa640 1 -- 172.21.15.43:0/3714798507 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0003d30 con 0x7f7bd0104270 2024-04-25T10:36:57.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.847+0000 7f7bd6a0c640 1 -- 172.21.15.43:0/3714798507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd01a71e0 con 0x7f7bd0104270 2024-04-25T10:36:57.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.850+0000 7f7bdd998740 1 Processor -- start 2024-04-25T10:36:57.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.850+0000 7f7bdd998740 1 -- start start 2024-04-25T10:36:57.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.850+0000 7f7bcd7fa640 1 -- 172.21.15.43:0/3714798507 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f7bc0003ed0 con 0x7f7bd0104270 2024-04-25T10:36:57.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.850+0000 7f7bdd998740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578670db390 0x55786747c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.850+0000 7f7bdd998740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670b24d0 0x55786747cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.850+0000 7f7bdd998740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578670ba6b0 0x5578674896a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.850+0000 7f7bdd998740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5578673e1d80 con 0x5578670b24d0 2024-04-25T10:36:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.850+0000 7f7bdd998740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55786739ba20 con 0x5578670db390 2024-04-25T10:36:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.850+0000 7f7bdd998740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5578673ddd40 con 0x5578670ba6b0 2024-04-25T10:36:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.850+0000 7f7bcffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670b24d0 0x55786747cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.850+0000 7f7bcffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670b24d0 0x55786747cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46048/0 (socket says 172.21.15.43:46048) 2024-04-25T10:36:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.850+0000 7f7bcffff640 1 -- 172.21.15.43:0/2598228116 learned_addr learned my addr 172.21.15.43:0/2598228116 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.851+0000 7f7bd4fa9640 1 --2- 172.21.15.43:0/2598228116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578670db390 0x55786747c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.851+0000 7f7bcf7fe640 1 --2- 172.21.15.43:0/2598228116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578670ba6b0 0x5578674896a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.851+0000 7f7bcffff640 1 -- 172.21.15.43:0/2598228116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578670ba6b0 msgr2=0x5578674896a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.851+0000 7f7bcffff640 1 --2- 172.21.15.43:0/2598228116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578670ba6b0 0x5578674896a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.851+0000 7f7bcffff640 1 -- 172.21.15.43:0/2598228116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578670db390 msgr2=0x55786747c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.851+0000 7f7bcffff640 1 --2- 172.21.15.43:0/2598228116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578670db390 0x55786747c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.851+0000 7f7bcffff640 1 -- 172.21.15.43:0/2598228116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578673a6c30 con 0x5578670b24d0 2024-04-25T10:36:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.851+0000 7f7bd4fa9640 1 --2- 172.21.15.43:0/2598228116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578670db390 0x55786747c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:57.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.851+0000 7f7bcd7fa640 1 --2- 172.21.15.43:0/3714798507 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7bb80afc60 0x7f7bb80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.851+0000 7f7bcffff640 1 --2- 172.21.15.43:0/2598228116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670b24d0 0x55786747cd50 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00dc440 tx=0x7f7bc00e5810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbd18b93052e3fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:57.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.851+0000 7f7bd4fa9640 1 --2- 172.21.15.43:0/3714798507 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7bb80afc60 0x7f7bb80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.851+0000 7f7bb67fc640 1 -- 172.21.15.43:0/2598228116 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc00a2de0 con 0x5578670b24d0 2024-04-25T10:36:57.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.851+0000 7f7bcd7fa640 1 -- 172.21.15.43:0/3714798507 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f7bc00ded50 con 0x7f7bd0104270 2024-04-25T10:36:57.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.851+0000 7f7bb67fc640 1 -- 172.21.15.43:0/2598228116 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc00a6a30 con 0x5578670b24d0 2024-04-25T10:36:57.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.851+0000 7f7bb67fc640 1 -- 172.21.15.43:0/2598228116 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc00a6690 con 0x5578670b24d0 2024-04-25T10:36:57.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.852+0000 7f7bdd998740 1 -- 172.21.15.43:0/2598228116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670b24d0 msgr2=0x55786747cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.852+0000 7f7bd4fa9640 1 --2- 172.21.15.43:0/3714798507 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7bb80afc60 0x7f7bb80b2170 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8005e30 tx=0x7f7bc8009920 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:57.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.852+0000 7f7bdd998740 1 --2- 172.21.15.43:0/2598228116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670b24d0 0x55786747cd50 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00dc440 tx=0x7f7bc00e5810 comp rx=0 tx=0).stop 2024-04-25T10:36:57.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.852+0000 7f7bdd998740 1 -- 172.21.15.43:0/2598228116 shutdown_connections 2024-04-25T10:36:57.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.852+0000 7f7bdd998740 1 --2- 172.21.15.43:0/2598228116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578670ba6b0 0x5578674896a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.852+0000 7f7bdd998740 1 --2- 172.21.15.43:0/2598228116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670b24d0 0x55786747cd50 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.852+0000 7f7bdd998740 1 --2- 172.21.15.43:0/2598228116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578670db390 0x55786747c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.852+0000 7f7bdd998740 1 -- 172.21.15.43:0/2598228116 >> 172.21.15.43:0/2598228116 conn(0x5578670cf4e0 msgr2=0x557867478fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:57.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.852+0000 7f7bdd998740 1 -- 172.21.15.43:0/2598228116 shutdown_connections 2024-04-25T10:36:57.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.852+0000 7f7bdd998740 1 -- 172.21.15.43:0/2598228116 wait complete. 2024-04-25T10:36:57.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.852+0000 7f7bdd998740 1 -- start start 2024-04-25T10:36:57.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.853+0000 7f7bdd998740 1 Processor -- start 2024-04-25T10:36:57.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.853+0000 7f7bdd998740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578670b24d0 0x557867487280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.853+0000 7f7bdd998740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578670ba6b0 0x5578674877c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.853+0000 7f7bdd998740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670db390 0x557867493210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.853+0000 7f7bdd998740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557867489d90 con 0x5578670db390 2024-04-25T10:36:57.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.853+0000 7f7bdd998740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557867368dc0 con 0x5578670ba6b0 2024-04-25T10:36:57.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.853+0000 7f7bdd998740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557867489f10 con 0x5578670b24d0 2024-04-25T10:36:57.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.853+0000 7f7bdd998740 10 client.0 _ll_register_callbacks cb 0x557867384600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:57.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.853+0000 7f7bd4fa9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670db390 0x557867493210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.853+0000 7f7bd4fa9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670db390 0x557867493210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46052/0 (socket says 172.21.15.43:46052) 2024-04-25T10:36:57.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.853+0000 7f7bd4fa9640 1 -- 172.21.15.43:0/938744378 learned_addr learned my addr 172.21.15.43:0/938744378 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:57.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.853+0000 7f7bcf7fe640 1 --2- 172.21.15.43:0/938744378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578670ba6b0 0x5578674877c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.853+0000 7f7bcffff640 1 --2- 172.21.15.43:0/938744378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578670b24d0 0x557867487280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.854+0000 7f7bd4fa9640 1 -- 172.21.15.43:0/938744378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578670b24d0 msgr2=0x557867487280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.854+0000 7f7bd4fa9640 1 --2- 172.21.15.43:0/938744378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578670b24d0 0x557867487280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.854+0000 7f7bd4fa9640 1 -- 172.21.15.43:0/938744378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578670ba6b0 msgr2=0x5578674877c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.854+0000 7f7bd4fa9640 1 --2- 172.21.15.43:0/938744378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578670ba6b0 0x5578674877c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.854+0000 7f7bd4fa9640 1 -- 172.21.15.43:0/938744378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578670f33f0 con 0x5578670db390 2024-04-25T10:36:57.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.854+0000 7f7bcf7fe640 1 --2- 172.21.15.43:0/938744378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578670ba6b0 0x5578674877c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:57.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.854+0000 7f7bd4fa9640 1 --2- 172.21.15.43:0/938744378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670db390 0x557867493210 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80316d0 tx=0x7f7bc80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b9a2c6cdf6f9b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:57.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.854+0000 7f7baf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:57.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.854+0000 7f7baf7fe640 1 -- 172.21.15.43:0/938744378 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc804e070 con 0x5578670db390 2024-04-25T10:36:57.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.854+0000 7f7baf7fe640 1 -- 172.21.15.43:0/938744378 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f7bc80045c0 con 0x5578670db390 2024-04-25T10:36:57.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.854+0000 7f7bdd998740 1 -- 172.21.15.43:0/938744378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bc00a6a30 con 0x5578670db390 2024-04-25T10:36:57.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.854+0000 7f7baf7fe640 1 -- 172.21.15.43:0/938744378 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7bc8031910 con 0x5578670db390 2024-04-25T10:36:57.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.854+0000 7f7bdd998740 10 client.16743 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:57.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.854+0000 7f7bdd998740 1 -- 172.21.15.43:0/938744378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5578671bd7d0 con 0x5578670db390 2024-04-25T10:36:57.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.854+0000 7f7baf7fe640 1 -- 172.21.15.43:0/938744378 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc00a6a30 con 0x5578670db390 2024-04-25T10:36:57.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.855+0000 7f7baf7fe640 1 -- 172.21.15.43:0/938744378 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7bc8043040 con 0x5578670db390 2024-04-25T10:36:57.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.855+0000 7f7bdd998740 10 client.16743 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:57.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.855+0000 7f7bdd998740 10 client.16743 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:36:57.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.855+0000 7f7bdd998740 20 client.16743 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:57.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.855+0000 7f7bdd998740 1 --2- 172.21.15.43:0/938744378 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x557867099f60 0x5578674996a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.855+0000 7f7bdd998740 4 client.16743 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:36:57.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.855+0000 7f7bdd998740 1 -- 172.21.15.43:0/938744378 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578673ef990 con 0x557867099f60 2024-04-25T10:36:57.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.855+0000 7f7bcffff640 1 --2- 172.21.15.43:0/938744378 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x557867099f60 0x5578674996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.855+0000 7f7baf7fe640 10 client.16743 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:36:57.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.855+0000 7f7bcffff640 1 --2- 172.21.15.43:0/938744378 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x557867099f60 0x5578674996a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b02ad965d9a4cb61 server_cookie=57e6992e690c73b4 in_seq=0 out_seq=0 2024-04-25T10:36:57.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.856+0000 7f7baf7fe640 1 -- 172.21.15.43:0/938744378 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5578673ef990 con 0x557867099f60 2024-04-25T10:36:57.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.856+0000 7f7baf7fe640 10 client.16743 handle_command_reply: tid=0 2024-04-25T10:36:57.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.856+0000 7f7bdd998740 1 client.16743 shutdown 2024-04-25T10:36:57.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.856+0000 7f7bdd998740 1 -- 172.21.15.43:0/938744378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670db390 msgr2=0x557867493210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.856+0000 7f7bdd998740 1 --2- 172.21.15.43:0/938744378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670db390 0x557867493210 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80316d0 tx=0x7f7bc80095c0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.856+0000 7f7bdd998740 1 -- 172.21.15.43:0/938744378 shutdown_connections 2024-04-25T10:36:57.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.856+0000 7f7bdd998740 1 --2- 172.21.15.43:0/938744378 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x557867099f60 0x5578674996a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.856+0000 7f7bdd998740 1 --2- 172.21.15.43:0/938744378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578670ba6b0 0x5578674877c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.856+0000 7f7bdd998740 1 --2- 172.21.15.43:0/938744378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670db390 0x557867493210 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.856+0000 7f7baf7fe640 0 client.16743 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:36:57.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.856+0000 7f7bdd998740 1 --2- 172.21.15.43:0/938744378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578670b24d0 0x557867487280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.856+0000 7f7bdd998740 1 -- 172.21.15.43:0/938744378 >> 172.21.15.43:0/938744378 conn(0x5578670cf4e0 msgr2=0x55786747e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:57.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.857+0000 7f7bdd998740 1 -- 172.21.15.43:0/938744378 shutdown_connections 2024-04-25T10:36:57.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.857+0000 7f7bdd998740 1 -- 172.21.15.43:0/938744378 wait complete. 2024-04-25T10:36:57.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.857+0000 7f7bdd998740 20 client.16743 trim_cache size 0 max 16384 2024-04-25T10:36:57.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.870+0000 7f7bdd998740 1 Processor -- start 2024-04-25T10:36:57.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.870+0000 7f7bdd998740 1 -- start start 2024-04-25T10:36:57.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.871+0000 7f7bdd998740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557867099f60 0x55786747ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.871+0000 7f7bdd998740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578670b24d0 0x557867492e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.871+0000 7f7bdd998740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670ba6b0 0x55786749ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.871+0000 7f7bdd998740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01b2210 con 0x5578670ba6b0 2024-04-25T10:36:57.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.871+0000 7f7bdd998740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01afae0 con 0x5578670b24d0 2024-04-25T10:36:57.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.871+0000 7f7bdd998740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01ad930 con 0x557867099f60 2024-04-25T10:36:57.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.871+0000 7f7bd4fa9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670ba6b0 0x55786749ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.871+0000 7f7bd4fa9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670ba6b0 0x55786749ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46062/0 (socket says 172.21.15.43:46062) 2024-04-25T10:36:57.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.871+0000 7f7bd4fa9640 1 -- 172.21.15.43:0/1236076021 learned_addr learned my addr 172.21.15.43:0/1236076021 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:57.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.871+0000 7f7bcf7fe640 1 --2- 172.21.15.43:0/1236076021 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578670b24d0 0x557867492e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.871+0000 7f7bcffff640 1 --2- 172.21.15.43:0/1236076021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557867099f60 0x55786747ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.871+0000 7f7bd4fa9640 1 -- 172.21.15.43:0/1236076021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557867099f60 msgr2=0x55786747ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.871+0000 7f7bd4fa9640 1 --2- 172.21.15.43:0/1236076021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557867099f60 0x55786747ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.871+0000 7f7bd4fa9640 1 -- 172.21.15.43:0/1236076021 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578670b24d0 msgr2=0x557867492e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.871+0000 7f7bd4fa9640 1 --2- 172.21.15.43:0/1236076021 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578670b24d0 0x557867492e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.871+0000 7f7bd4fa9640 1 -- 172.21.15.43:0/1236076021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557867478620 con 0x5578670ba6b0 2024-04-25T10:36:57.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.871+0000 7f7bd4fa9640 1 --2- 172.21.15.43:0/1236076021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670ba6b0 0x55786749ee60 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8009bf0 tx=0x7f7bc802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8519c8a5ab230469 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:57.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.871+0000 7f7bb5ffb640 1 -- 172.21.15.43:0/1236076021 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8031cf0 con 0x5578670ba6b0 2024-04-25T10:36:57.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.871+0000 7f7bb5ffb640 1 -- 172.21.15.43:0/1236076021 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc8036040 con 0x5578670ba6b0 2024-04-25T10:36:57.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.872+0000 7f7bb5ffb640 1 -- 172.21.15.43:0/1236076021 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc802f8c0 con 0x5578670ba6b0 2024-04-25T10:36:57.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.872+0000 7f7bdd998740 1 -- 172.21.15.43:0/1236076021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670ba6b0 msgr2=0x55786749ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.872+0000 7f7bdd998740 1 --2- 172.21.15.43:0/1236076021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670ba6b0 0x55786749ee60 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8009bf0 tx=0x7f7bc802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.872+0000 7f7bdd998740 1 -- 172.21.15.43:0/1236076021 shutdown_connections 2024-04-25T10:36:57.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.872+0000 7f7bdd998740 1 --2- 172.21.15.43:0/1236076021 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578670b24d0 0x557867492e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.872+0000 7f7bdd998740 1 --2- 172.21.15.43:0/1236076021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670ba6b0 0x55786749ee60 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.872+0000 7f7bdd998740 1 --2- 172.21.15.43:0/1236076021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557867099f60 0x55786747ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.872+0000 7f7bdd998740 1 -- 172.21.15.43:0/1236076021 >> 172.21.15.43:0/1236076021 conn(0x5578670cf4e0 msgr2=0x55786747c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:57.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.872+0000 7f7bdd998740 1 -- 172.21.15.43:0/1236076021 shutdown_connections 2024-04-25T10:36:57.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.872+0000 7f7bdd998740 1 -- 172.21.15.43:0/1236076021 wait complete. 2024-04-25T10:36:57.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.873+0000 7f7bdd998740 1 -- start start 2024-04-25T10:36:57.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.873+0000 7f7bdd998740 1 Processor -- start 2024-04-25T10:36:57.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.873+0000 7f7bdd998740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557867099f60 0x55786748a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.873+0000 7f7bdd998740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578670b24d0 0x55786748a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.873+0000 7f7bdd998740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670ba6b0 0x55786748ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.873+0000 7f7bdd998740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01adfa0 con 0x5578670ba6b0 2024-04-25T10:36:57.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.873+0000 7f7bdd998740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01af410 con 0x557867099f60 2024-04-25T10:36:57.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.873+0000 7f7bdd998740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01ae670 con 0x5578670b24d0 2024-04-25T10:36:57.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.873+0000 7f7bdd998740 10 client.0 _ll_register_callbacks cb 0x7f7bc8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:57.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.873+0000 7f7bd4fa9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670ba6b0 0x55786748ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.873+0000 7f7bd4fa9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670ba6b0 0x55786748ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46076/0 (socket says 172.21.15.43:46076) 2024-04-25T10:36:57.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.873+0000 7f7bd4fa9640 1 -- 172.21.15.43:0/1877033846 learned_addr learned my addr 172.21.15.43:0/1877033846 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:57.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.873+0000 7f7bcffff640 1 --2- 172.21.15.43:0/1877033846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557867099f60 0x55786748a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.874+0000 7f7bcf7fe640 1 --2- 172.21.15.43:0/1877033846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578670b24d0 0x55786748a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.874+0000 7f7bd4fa9640 1 -- 172.21.15.43:0/1877033846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578670b24d0 msgr2=0x55786748a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.874+0000 7f7bd4fa9640 1 --2- 172.21.15.43:0/1877033846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578670b24d0 0x55786748a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.874+0000 7f7bd4fa9640 1 -- 172.21.15.43:0/1877033846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557867099f60 msgr2=0x55786748a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:57.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.874+0000 7f7bd4fa9640 1 --2- 172.21.15.43:0/1877033846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557867099f60 0x55786748a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.874+0000 7f7bd4fa9640 1 -- 172.21.15.43:0/1877033846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578674ffa80 con 0x5578670ba6b0 2024-04-25T10:36:57.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.874+0000 7f7bcf7fe640 1 --2- 172.21.15.43:0/1877033846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578670b24d0 0x55786748a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:36:57.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.874+0000 7f7bd4fa9640 1 --2- 172.21.15.43:0/1877033846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670ba6b0 0x55786748ac30 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8041040 tx=0x7f7bc8004720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1f729e8cc04894 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:57.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.874+0000 7f7baf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:57.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.874+0000 7f7baf7fe640 1 -- 172.21.15.43:0/1877033846 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8030e00 con 0x5578670ba6b0 2024-04-25T10:36:57.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.874+0000 7f7baf7fe640 1 -- 172.21.15.43:0/1877033846 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f7bc802f660 con 0x5578670ba6b0 2024-04-25T10:36:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.874+0000 7f7bdd998740 1 -- 172.21.15.43:0/1877033846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bc8036040 con 0x5578670ba6b0 2024-04-25T10:36:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.874+0000 7f7baf7fe640 1 -- 172.21.15.43:0/1877033846 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7bc8031910 con 0x5578670ba6b0 2024-04-25T10:36:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.874+0000 7f7bdd998740 10 client.16749 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.874+0000 7f7bdd998740 1 -- 172.21.15.43:0/1877033846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557867500d00 con 0x5578670ba6b0 2024-04-25T10:36:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.874+0000 7f7baf7fe640 1 -- 172.21.15.43:0/1877033846 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc8036040 con 0x5578670ba6b0 2024-04-25T10:36:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.874+0000 7f7baf7fe640 1 -- 172.21.15.43:0/1877033846 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7bc8043070 con 0x5578670ba6b0 2024-04-25T10:36:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.875+0000 7f7bdd998740 10 client.16749 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.875+0000 7f7bdd998740 10 client.16749 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:36:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.875+0000 7f7bdd998740 20 client.16749 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.875+0000 7f7bdd998740 1 --2- 172.21.15.43:0/1877033846 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5578670db390 0x557867503380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.875+0000 7f7bdd998740 4 client.16749 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:36:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.875+0000 7f7bdd998740 1 -- 172.21.15.43:0/1877033846 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5578673e9800 con 0x5578670db390 2024-04-25T10:36:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.875+0000 7f7bcffff640 1 --2- 172.21.15.43:0/1877033846 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5578670db390 0x557867503380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.875+0000 7f7bcffff640 1 --2- 172.21.15.43:0/1877033846 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5578670db390 0x557867503380 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e48711132ccb237 server_cookie=a57b055e49515e1a in_seq=0 out_seq=0 2024-04-25T10:36:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.875+0000 7f7baf7fe640 10 client.16749 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:36:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.875+0000 7f7baf7fe640 1 -- 172.21.15.43:0/1877033846 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5578673e9800 con 0x5578670db390 2024-04-25T10:36:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.875+0000 7f7baf7fe640 10 client.16749 handle_command_reply: tid=0 2024-04-25T10:36:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.875+0000 7f7bdd998740 1 client.16749 shutdown 2024-04-25T10:36:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bdd998740 1 -- 172.21.15.43:0/1877033846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670ba6b0 msgr2=0x55786748ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bdd998740 1 --2- 172.21.15.43:0/1877033846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670ba6b0 0x55786748ac30 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8041040 tx=0x7f7bc8004720 comp rx=0 tx=0).stop 2024-04-25T10:36:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bdd998740 1 -- 172.21.15.43:0/1877033846 shutdown_connections 2024-04-25T10:36:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bdd998740 1 --2- 172.21.15.43:0/1877033846 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5578670db390 0x557867503380 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bdd998740 1 --2- 172.21.15.43:0/1877033846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578670ba6b0 0x55786748ac30 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bdd998740 1 --2- 172.21.15.43:0/1877033846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578670b24d0 0x55786748a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7baf7fe640 0 client.16749 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:36:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bdd998740 1 --2- 172.21.15.43:0/1877033846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557867099f60 0x55786748a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bdd998740 1 -- 172.21.15.43:0/1877033846 >> 172.21.15.43:0/1877033846 conn(0x5578670cf4e0 msgr2=0x557867478800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bdd998740 1 -- 172.21.15.43:0/1877033846 shutdown_connections 2024-04-25T10:36:57.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bdd998740 1 -- 172.21.15.43:0/1877033846 wait complete. 2024-04-25T10:36:57.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bdd998740 20 client.16749 trim_cache size 0 max 16384 2024-04-25T10:36:57.885 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:36:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bd6a0c640 1 -- 172.21.15.43:0/3714798507 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7bb80afc60 msgr2=0x7f7bb80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bd6a0c640 1 --2- 172.21.15.43:0/3714798507 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7bb80afc60 0x7f7bb80b2170 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8005e30 tx=0x7f7bc8009920 comp rx=0 tx=0).stop 2024-04-25T10:36:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bd6a0c640 1 -- 172.21.15.43:0/3714798507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bd0104270 msgr2=0x7f7bd019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bd6a0c640 1 --2- 172.21.15.43:0/3714798507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bd0104270 0x7f7bd019a710 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0005fd0 tx=0x7f7bc000dde0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bd6a0c640 1 -- 172.21.15.43:0/3714798507 shutdown_connections 2024-04-25T10:36:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bd6a0c640 1 --2- 172.21.15.43:0/3714798507 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7bb80afc60 0x7f7bb80b2170 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bd6a0c640 1 --2- 172.21.15.43:0/3714798507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bd0109b20 0x7f7bd019ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bd6a0c640 1 --2- 172.21.15.43:0/3714798507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bd0109fc0 0x7f7bd019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bd6a0c640 1 --2- 172.21.15.43:0/3714798507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bd0104270 0x7f7bd019a710 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.876+0000 7f7bd6a0c640 1 -- 172.21.15.43:0/3714798507 >> 172.21.15.43:0/3714798507 conn(0x7f7bd00ff1e0 msgr2=0x7f7bd0101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:57.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.877+0000 7f7bd6a0c640 1 -- 172.21.15.43:0/3714798507 shutdown_connections 2024-04-25T10:36:57.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:57.877+0000 7f7bd6a0c640 1 -- 172.21.15.43:0/3714798507 wait complete. 2024-04-25T10:36:57.890 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:36:58.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.002+0000 7f0e4734d640 1 -- 172.21.15.43:0/1579765561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e40104260 msgr2=0x7f0e40110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:58.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.002+0000 7f0e4734d640 1 --2- 172.21.15.43:0/1579765561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e40104260 0x7f0e40110630 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009ca0 tx=0x7f0e3802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.002+0000 7f0e4734d640 1 -- 172.21.15.43:0/1579765561 shutdown_connections 2024-04-25T10:36:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.002+0000 7f0e4734d640 1 --2- 172.21.15.43:0/1579765561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e40109fb0 0x7f0e40103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.002+0000 7f0e4734d640 1 --2- 172.21.15.43:0/1579765561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e40104260 0x7f0e40110630 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.002+0000 7f0e4734d640 1 --2- 172.21.15.43:0/1579765561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e40109b10 0x7f0e401036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.002+0000 7f0e4734d640 1 -- 172.21.15.43:0/1579765561 >> 172.21.15.43:0/1579765561 conn(0x7f0e400ff210 msgr2=0x7f0e40101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.002+0000 7f0e4734d640 1 -- 172.21.15.43:0/1579765561 shutdown_connections 2024-04-25T10:36:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.002+0000 7f0e4734d640 1 -- 172.21.15.43:0/1579765561 wait complete. 2024-04-25T10:36:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.002+0000 7f0e4734d640 1 Processor -- start 2024-04-25T10:36:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.002+0000 7f0e4734d640 1 -- start start 2024-04-25T10:36:58.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e4734d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e40104260 0x7f0e40075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:58.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e4734d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e40109b10 0x7f0e400764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:58.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e4734d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e40109fb0 0x7f0e40076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:58.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e4734d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0e40113d00 con 0x7f0e40104260 2024-04-25T10:36:58.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e4734d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0e40113b80 con 0x7f0e40109fb0 2024-04-25T10:36:58.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e4734d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0e40113e80 con 0x7f0e40109b10 2024-04-25T10:36:58.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e450e9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e40104260 0x7f0e40075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:58.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e450e9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e40104260 0x7f0e40075fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46096/0 (socket says 172.21.15.43:46096) 2024-04-25T10:36:58.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e450e9640 1 -- 172.21.15.43:0/286791948 learned_addr learned my addr 172.21.15.43:0/286791948 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:58.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e448e8640 1 --2- 172.21.15.43:0/286791948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e40109b10 0x7f0e400764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:58.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e458ea640 1 --2- 172.21.15.43:0/286791948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e40109fb0 0x7f0e40076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:58.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e450e9640 1 -- 172.21.15.43:0/286791948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e40109b10 msgr2=0x7f0e400764f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:58.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e450e9640 1 --2- 172.21.15.43:0/286791948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e40109b10 0x7f0e400764f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e450e9640 1 -- 172.21.15.43:0/286791948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e40109fb0 msgr2=0x7f0e40076a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:58.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e450e9640 1 --2- 172.21.15.43:0/286791948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e40109fb0 0x7f0e40076a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e450e9640 1 -- 172.21.15.43:0/286791948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e401a8a00 con 0x7f0e40104260 2024-04-25T10:36:58.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e448e8640 1 --2- 172.21.15.43:0/286791948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e40109b10 0x7f0e400764f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:36:58.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e458ea640 1 --2- 172.21.15.43:0/286791948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e40109fb0 0x7f0e40076a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:36:58.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e450e9640 1 --2- 172.21.15.43:0/286791948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e40104260 0x7f0e40075fb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c005fd0 tx=0x7f0e3c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:58.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e367fc640 1 -- 172.21.15.43:0/286791948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c017070 con 0x7f0e40104260 2024-04-25T10:36:58.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e367fc640 1 -- 172.21.15.43:0/286791948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e3c009c30 con 0x7f0e40104260 2024-04-25T10:36:58.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.003+0000 7f0e4734d640 1 -- 172.21.15.43:0/286791948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e401a8cf0 con 0x7f0e40104260 2024-04-25T10:36:58.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.004+0000 7f0e367fc640 1 -- 172.21.15.43:0/286791948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c003d30 con 0x7f0e40104260 2024-04-25T10:36:58.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.004+0000 7f0e4734d640 1 -- 172.21.15.43:0/286791948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e401a9190 con 0x7f0e40104260 2024-04-25T10:36:58.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.006+0000 7f0e4e1fe740 1 Processor -- start 2024-04-25T10:36:58.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.006+0000 7f0e367fc640 1 -- 172.21.15.43:0/286791948 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f0e3c003ed0 con 0x7f0e40104260 2024-04-25T10:36:58.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.006+0000 7f0e4e1fe740 1 -- start start 2024-04-25T10:36:58.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.007+0000 7f0e4e1fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a6ce390 0x56466aa6f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:58.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.007+0000 7f0e4e1fe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6a54d0 0x56466aa6fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:58.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.007+0000 7f0e4e1fe740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56466a6ad6b0 0x56466aa7c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:58.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.007+0000 7f0e4e1fe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56466a9d4d80 con 0x56466a6a54d0 2024-04-25T10:36:58.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.007+0000 7f0e4e1fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56466a98ea20 con 0x56466a6ce390 2024-04-25T10:36:58.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.007+0000 7f0e4e1fe740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56466a9d0d40 con 0x56466a6ad6b0 2024-04-25T10:36:58.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.007+0000 7f0e450e9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6a54d0 0x56466aa6fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:58.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.007+0000 7f0e450e9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6a54d0 0x56466aa6fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46100/0 (socket says 172.21.15.43:46100) 2024-04-25T10:36:58.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.007+0000 7f0e450e9640 1 -- 172.21.15.43:0/3264455219 learned_addr learned my addr 172.21.15.43:0/3264455219 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:58.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.007+0000 7f0e458ea640 1 --2- 172.21.15.43:0/3264455219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a6ce390 0x56466aa6f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:58.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.007+0000 7f0e448e8640 1 --2- 172.21.15.43:0/3264455219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56466a6ad6b0 0x56466aa7c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:58.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.007+0000 7f0e367fc640 1 --2- 172.21.15.43:0/286791948 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0e2c0afc60 0x7f0e2c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:58.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.007+0000 7f0e450e9640 1 -- 172.21.15.43:0/3264455219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56466a6ad6b0 msgr2=0x56466aa7c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:58.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.007+0000 7f0e450e9640 1 --2- 172.21.15.43:0/3264455219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56466a6ad6b0 0x56466aa7c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.007+0000 7f0e450e9640 1 -- 172.21.15.43:0/3264455219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a6ce390 msgr2=0x56466aa6f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:58.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.007+0000 7f0e450e9640 1 --2- 172.21.15.43:0/3264455219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a6ce390 0x56466aa6f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.007+0000 7f0e450e9640 1 -- 172.21.15.43:0/3264455219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56466a999c30 con 0x56466a6a54d0 2024-04-25T10:36:58.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.007+0000 7f0e458ea640 1 --2- 172.21.15.43:0/286791948 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0e2c0afc60 0x7f0e2c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:58.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.008+0000 7f0e367fc640 1 -- 172.21.15.43:0/286791948 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f0e3c0ded50 con 0x7f0e40104260 2024-04-25T10:36:58.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.008+0000 7f0e450e9640 1 --2- 172.21.15.43:0/3264455219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6a54d0 0x56466aa6fd50 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0dc440 tx=0x7f0e3c01f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903235e36019e11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:58.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.008+0000 7f0e2b7fe640 1 -- 172.21.15.43:0/3264455219 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c0a6cf0 con 0x56466a6a54d0 2024-04-25T10:36:58.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.008+0000 7f0e458ea640 1 --2- 172.21.15.43:0/286791948 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0e2c0afc60 0x7f0e2c0b2170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x56466aa6c980 tx=0x7f0e380095c0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:58.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.008+0000 7f0e2b7fe640 1 -- 172.21.15.43:0/3264455219 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e3c0a6a30 con 0x56466a6a54d0 2024-04-25T10:36:58.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.008+0000 7f0e2b7fe640 1 -- 172.21.15.43:0/3264455219 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c0e25b0 con 0x56466a6a54d0 2024-04-25T10:36:58.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.009+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/3264455219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6a54d0 msgr2=0x56466aa6fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:58.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.009+0000 7f0e4e1fe740 1 --2- 172.21.15.43:0/3264455219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6a54d0 0x56466aa6fd50 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0dc440 tx=0x7f0e3c01f840 comp rx=0 tx=0).stop 2024-04-25T10:36:58.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.009+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/3264455219 shutdown_connections 2024-04-25T10:36:58.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.009+0000 7f0e4e1fe740 1 --2- 172.21.15.43:0/3264455219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56466a6ad6b0 0x56466aa7c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.009+0000 7f0e4e1fe740 1 --2- 172.21.15.43:0/3264455219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6a54d0 0x56466aa6fd50 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.009+0000 7f0e4e1fe740 1 --2- 172.21.15.43:0/3264455219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a6ce390 0x56466aa6f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.009+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/3264455219 >> 172.21.15.43:0/3264455219 conn(0x56466a6c24e0 msgr2=0x56466aa6bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:58.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.009+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/3264455219 shutdown_connections 2024-04-25T10:36:58.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.009+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/3264455219 wait complete. 2024-04-25T10:36:58.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.009+0000 7f0e4e1fe740 1 -- start start 2024-04-25T10:36:58.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.009+0000 7f0e4e1fe740 1 Processor -- start 2024-04-25T10:36:58.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.010+0000 7f0e4e1fe740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56466a6a54d0 0x56466aa7a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:58.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.010+0000 7f0e4e1fe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6ad6b0 0x56466aa7a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:58.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.010+0000 7f0e4e1fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a6ce390 0x56466aa86150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:58.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.010+0000 7f0e4e1fe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56466aa7ccd0 con 0x56466a6ad6b0 2024-04-25T10:36:58.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.010+0000 7f0e4e1fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56466a95bdc0 con 0x56466a6ce390 2024-04-25T10:36:58.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.010+0000 7f0e4e1fe740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56466aa7ce50 con 0x56466a6a54d0 2024-04-25T10:36:58.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.010+0000 7f0e4e1fe740 10 client.0 _ll_register_callbacks cb 0x56466a977600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:58.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.010+0000 7f0e448e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6ad6b0 0x56466aa7a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:58.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.010+0000 7f0e448e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6ad6b0 0x56466aa7a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46116/0 (socket says 172.21.15.43:46116) 2024-04-25T10:36:58.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.010+0000 7f0e448e8640 1 -- 172.21.15.43:0/546567881 learned_addr learned my addr 172.21.15.43:0/546567881 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:58.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.010+0000 7f0e458ea640 1 --2- 172.21.15.43:0/546567881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a6ce390 0x56466aa86150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:58.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.010+0000 7f0e450e9640 1 --2- 172.21.15.43:0/546567881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56466a6a54d0 0x56466aa7a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:58.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.010+0000 7f0e448e8640 1 -- 172.21.15.43:0/546567881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56466a6a54d0 msgr2=0x56466aa7a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:58.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.010+0000 7f0e448e8640 1 --2- 172.21.15.43:0/546567881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56466a6a54d0 0x56466aa7a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.010+0000 7f0e448e8640 1 -- 172.21.15.43:0/546567881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a6ce390 msgr2=0x56466aa86150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:58.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.010+0000 7f0e448e8640 1 --2- 172.21.15.43:0/546567881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a6ce390 0x56466aa86150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.010+0000 7f0e448e8640 1 -- 172.21.15.43:0/546567881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56466a6e63f0 con 0x56466a6ad6b0 2024-04-25T10:36:58.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.011+0000 7f0e448e8640 1 --2- 172.21.15.43:0/546567881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6ad6b0 0x56466aa7a7e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0e30012420 tx=0x7f0e300128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a28ce405158b7c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:58.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.011+0000 7f0e28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:58.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.011+0000 7f0e28ff9640 1 -- 172.21.15.43:0/546567881 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e30005220 con 0x56466a6ad6b0 2024-04-25T10:36:58.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.011+0000 7f0e28ff9640 1 -- 172.21.15.43:0/546567881 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f0e300053c0 con 0x56466a6ad6b0 2024-04-25T10:36:58.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.011+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/546567881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e3c0a6a30 con 0x56466a6ad6b0 2024-04-25T10:36:58.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.011+0000 7f0e28ff9640 1 -- 172.21.15.43:0/546567881 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0e30005620 con 0x56466a6ad6b0 2024-04-25T10:36:58.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.011+0000 7f0e4e1fe740 10 client.16761 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:58.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.011+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/546567881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56466a7b07d0 con 0x56466a6ad6b0 2024-04-25T10:36:58.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.011+0000 7f0e28ff9640 1 -- 172.21.15.43:0/546567881 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e3c0a6a30 con 0x56466a6ad6b0 2024-04-25T10:36:58.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.011+0000 7f0e28ff9640 1 -- 172.21.15.43:0/546567881 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0e3000f040 con 0x56466a6ad6b0 2024-04-25T10:36:58.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.011+0000 7f0e4e1fe740 10 client.16761 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:58.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.011+0000 7f0e4e1fe740 10 client.16761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:58.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.011+0000 7f0e4e1fe740 20 client.16761 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:58.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.011+0000 7f0e4e1fe740 1 --2- 172.21.15.43:0/546567881 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56466a68cf60 0x56466aa8c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:58.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.011+0000 7f0e4e1fe740 4 client.16761 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:36:58.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.011+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/546567881 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56466a9e2990 con 0x56466a68cf60 2024-04-25T10:36:58.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.012+0000 7f0e450e9640 1 --2- 172.21.15.43:0/546567881 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56466a68cf60 0x56466aa8c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:58.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.012+0000 7f0e28ff9640 10 client.16761 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:58.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.012+0000 7f0e450e9640 1 --2- 172.21.15.43:0/546567881 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56466a68cf60 0x56466aa8c740 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fec94241a318c490 server_cookie=d5ce6877ed23067d in_seq=0 out_seq=0 2024-04-25T10:36:58.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.013+0000 7f0e28ff9640 1 -- 172.21.15.43:0/546567881 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56466a9e2990 con 0x56466a68cf60 2024-04-25T10:36:58.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.013+0000 7f0e28ff9640 10 client.16761 handle_command_reply: tid=0 2024-04-25T10:36:58.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.013+0000 7f0e4e1fe740 1 client.16761 shutdown 2024-04-25T10:36:58.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.013+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/546567881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6ad6b0 msgr2=0x56466aa7a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:58.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.013+0000 7f0e4e1fe740 1 --2- 172.21.15.43:0/546567881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6ad6b0 0x56466aa7a7e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0e30012420 tx=0x7f0e300128e0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.013+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/546567881 shutdown_connections 2024-04-25T10:36:58.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.013+0000 7f0e4e1fe740 1 --2- 172.21.15.43:0/546567881 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56466a68cf60 0x56466aa8c740 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.013+0000 7f0e4e1fe740 1 --2- 172.21.15.43:0/546567881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a6ce390 0x56466aa86150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.013+0000 7f0e28ff9640 0 client.16761 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:58.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.013+0000 7f0e4e1fe740 1 --2- 172.21.15.43:0/546567881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6ad6b0 0x56466aa7a7e0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.013+0000 7f0e4e1fe740 1 --2- 172.21.15.43:0/546567881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56466a6a54d0 0x56466aa7a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.013+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/546567881 >> 172.21.15.43:0/546567881 conn(0x56466a6c24e0 msgr2=0x56466aa71530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:58.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.013+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/546567881 shutdown_connections 2024-04-25T10:36:58.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.013+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/546567881 wait complete. 2024-04-25T10:36:58.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.013+0000 7f0e4e1fe740 20 client.16761 trim_cache size 0 max 16384 2024-04-25T10:36:58.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.027+0000 7f0e4e1fe740 1 Processor -- start 2024-04-25T10:36:58.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.027+0000 7f0e4e1fe740 1 -- start start 2024-04-25T10:36:58.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e4e1fe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a68cf60 0x56466aa85db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:58.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e4e1fe740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56466a6a54d0 0x56466aa91f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:58.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e4e1fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a6ad6b0 0x56466aa92440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:58.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e4e1fe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0e401b43e0 con 0x56466a68cf60 2024-04-25T10:36:58.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e4e1fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0e401b1cb0 con 0x56466a6ad6b0 2024-04-25T10:36:58.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e4e1fe740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0e401afb00 con 0x56466a6a54d0 2024-04-25T10:36:58.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e450e9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a68cf60 0x56466aa85db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:58.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e450e9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a68cf60 0x56466aa85db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46128/0 (socket says 172.21.15.43:46128) 2024-04-25T10:36:58.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e450e9640 1 -- 172.21.15.43:0/2199052642 learned_addr learned my addr 172.21.15.43:0/2199052642 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:58.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e458ea640 1 --2- 172.21.15.43:0/2199052642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a6ad6b0 0x56466aa92440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:58.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e448e8640 1 --2- 172.21.15.43:0/2199052642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56466a6a54d0 0x56466aa91f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:58.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e450e9640 1 -- 172.21.15.43:0/2199052642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56466a6a54d0 msgr2=0x56466aa91f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:58.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e450e9640 1 --2- 172.21.15.43:0/2199052642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56466a6a54d0 0x56466aa91f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e450e9640 1 -- 172.21.15.43:0/2199052642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a6ad6b0 msgr2=0x56466aa92440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:58.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e450e9640 1 --2- 172.21.15.43:0/2199052642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a6ad6b0 0x56466aa92440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e450e9640 1 -- 172.21.15.43:0/2199052642 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56466aa785e0 con 0x56466a68cf60 2024-04-25T10:36:58.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e458ea640 1 --2- 172.21.15.43:0/2199052642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a6ad6b0 0x56466aa92440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:58.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e450e9640 1 --2- 172.21.15.43:0/2199052642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a68cf60 0x56466aa85db0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0a6c20 tx=0x7f0e3c0e2980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af0e6ea0fcee81d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:58.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e2affd640 1 -- 172.21.15.43:0/2199052642 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c0a6690 con 0x56466a68cf60 2024-04-25T10:36:58.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e2affd640 1 -- 172.21.15.43:0/2199052642 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e3c0a7620 con 0x56466a68cf60 2024-04-25T10:36:58.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.028+0000 7f0e2affd640 1 -- 172.21.15.43:0/2199052642 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c0a6870 con 0x56466a68cf60 2024-04-25T10:36:58.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.029+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/2199052642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a68cf60 msgr2=0x56466aa85db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:58.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.029+0000 7f0e4e1fe740 1 --2- 172.21.15.43:0/2199052642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a68cf60 0x56466aa85db0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0a6c20 tx=0x7f0e3c0e2980 comp rx=0 tx=0).stop 2024-04-25T10:36:58.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.029+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/2199052642 shutdown_connections 2024-04-25T10:36:58.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.029+0000 7f0e4e1fe740 1 --2- 172.21.15.43:0/2199052642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a6ad6b0 0x56466aa92440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.029+0000 7f0e4e1fe740 1 --2- 172.21.15.43:0/2199052642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56466a6a54d0 0x56466aa91f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.029+0000 7f0e4e1fe740 1 --2- 172.21.15.43:0/2199052642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a68cf60 0x56466aa85db0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.029+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/2199052642 >> 172.21.15.43:0/2199052642 conn(0x56466a6c24e0 msgr2=0x56466aa75220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:58.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.029+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/2199052642 shutdown_connections 2024-04-25T10:36:58.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.029+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/2199052642 wait complete. 2024-04-25T10:36:58.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.030+0000 7f0e4e1fe740 1 -- start start 2024-04-25T10:36:58.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.030+0000 7f0e4e1fe740 1 Processor -- start 2024-04-25T10:36:58.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.030+0000 7f0e4e1fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a68cf60 0x56466aa82450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:58.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.030+0000 7f0e4e1fe740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56466a6a54d0 0x56466aa7d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:58.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.030+0000 7f0e4e1fe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6ad6b0 0x56466aa7dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:58.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.030+0000 7f0e4e1fe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0e401b0170 con 0x56466a6ad6b0 2024-04-25T10:36:58.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.030+0000 7f0e4e1fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0e401b15e0 con 0x56466a68cf60 2024-04-25T10:36:58.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.030+0000 7f0e4e1fe740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0e401b0840 con 0x56466a6a54d0 2024-04-25T10:36:58.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.030+0000 7f0e4e1fe740 10 client.0 _ll_register_callbacks cb 0x7f0e30009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:58.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.030+0000 7f0e458ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6ad6b0 0x56466aa7dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:58.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.030+0000 7f0e458ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6ad6b0 0x56466aa7dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46144/0 (socket says 172.21.15.43:46144) 2024-04-25T10:36:58.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.030+0000 7f0e458ea640 1 -- 172.21.15.43:0/1448691601 learned_addr learned my addr 172.21.15.43:0/1448691601 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:58.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.030+0000 7f0e448e8640 1 --2- 172.21.15.43:0/1448691601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56466a6a54d0 0x56466aa7d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:58.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.031+0000 7f0e450e9640 1 --2- 172.21.15.43:0/1448691601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a68cf60 0x56466aa82450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:58.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.031+0000 7f0e458ea640 1 -- 172.21.15.43:0/1448691601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56466a6a54d0 msgr2=0x56466aa7d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:58.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.031+0000 7f0e458ea640 1 --2- 172.21.15.43:0/1448691601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56466a6a54d0 0x56466aa7d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.031+0000 7f0e458ea640 1 -- 172.21.15.43:0/1448691601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a68cf60 msgr2=0x56466aa82450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:58.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.031+0000 7f0e458ea640 1 --2- 172.21.15.43:0/1448691601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a68cf60 0x56466aa82450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.031+0000 7f0e458ea640 1 -- 172.21.15.43:0/1448691601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56466aa7e250 con 0x56466a6ad6b0 2024-04-25T10:36:58.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.031+0000 7f0e450e9640 1 --2- 172.21.15.43:0/1448691601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a68cf60 0x56466aa82450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:58.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.031+0000 7f0e458ea640 1 --2- 172.21.15.43:0/1448691601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6ad6b0 0x56466aa7dd10 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009ca0 tx=0x7f0e3802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9c33b5939f2ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:58.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.031+0000 7f0e28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:58.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.031+0000 7f0e28ff9640 1 -- 172.21.15.43:0/1448691601 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38004490 con 0x56466a6ad6b0 2024-04-25T10:36:58.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.031+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/1448691601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e3c0a7620 con 0x56466a6ad6b0 2024-04-25T10:36:58.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.031+0000 7f0e28ff9640 1 -- 172.21.15.43:0/1448691601 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f0e38004630 con 0x56466a6ad6b0 2024-04-25T10:36:58.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.031+0000 7f0e28ff9640 1 -- 172.21.15.43:0/1448691601 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0e38004830 con 0x56466a6ad6b0 2024-04-25T10:36:58.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.031+0000 7f0e4e1fe740 10 client.16767 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:58.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.031+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/1448691601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56466aaf4740 con 0x56466a6ad6b0 2024-04-25T10:36:58.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.031+0000 7f0e28ff9640 1 -- 172.21.15.43:0/1448691601 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e3c0a7620 con 0x56466a6ad6b0 2024-04-25T10:36:58.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.032+0000 7f0e28ff9640 1 -- 172.21.15.43:0/1448691601 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0e38030780 con 0x56466a6ad6b0 2024-04-25T10:36:58.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.032+0000 7f0e4e1fe740 10 client.16767 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:58.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.032+0000 7f0e4e1fe740 10 client.16767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:58.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.032+0000 7f0e4e1fe740 20 client.16767 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:58.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.032+0000 7f0e4e1fe740 1 --2- 172.21.15.43:0/1448691601 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56466a6ce390 0x56466aaf6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:58.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.032+0000 7f0e4e1fe740 4 client.16767 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:36:58.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.032+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/1448691601 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56466a9dc800 con 0x56466a6ce390 2024-04-25T10:36:58.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.032+0000 7f0e450e9640 1 --2- 172.21.15.43:0/1448691601 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56466a6ce390 0x56466aaf6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:58.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.032+0000 7f0e28ff9640 10 client.16767 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:58.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.032+0000 7f0e450e9640 1 --2- 172.21.15.43:0/1448691601 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56466a6ce390 0x56466aaf6eb0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca34d4227bd65960 server_cookie=5d132a22361b3b8b in_seq=0 out_seq=0 2024-04-25T10:36:58.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e28ff9640 1 -- 172.21.15.43:0/1448691601 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56466a9dc800 con 0x56466a6ce390 2024-04-25T10:36:58.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e28ff9640 10 client.16767 handle_command_reply: tid=0 2024-04-25T10:36:58.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e4e1fe740 1 client.16767 shutdown 2024-04-25T10:36:58.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/1448691601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6ad6b0 msgr2=0x56466aa7dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:58.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e4e1fe740 1 --2- 172.21.15.43:0/1448691601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6ad6b0 0x56466aa7dd10 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009ca0 tx=0x7f0e3802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/1448691601 shutdown_connections 2024-04-25T10:36:58.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e4e1fe740 1 --2- 172.21.15.43:0/1448691601 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56466a6ce390 0x56466aaf6eb0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e4e1fe740 1 --2- 172.21.15.43:0/1448691601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466a6ad6b0 0x56466aa7dd10 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e28ff9640 0 client.16767 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:58.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e4e1fe740 1 --2- 172.21.15.43:0/1448691601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56466a6a54d0 0x56466aa7d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e4e1fe740 1 --2- 172.21.15.43:0/1448691601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56466a68cf60 0x56466aa82450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/1448691601 >> 172.21.15.43:0/1448691601 conn(0x56466a6c24e0 msgr2=0x56466aa6b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:58.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/1448691601 shutdown_connections 2024-04-25T10:36:58.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e4e1fe740 1 -- 172.21.15.43:0/1448691601 wait complete. 2024-04-25T10:36:58.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e4e1fe740 20 client.16767 trim_cache size 0 max 16384 2024-04-25T10:36:58.042 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:36:58.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e4734d640 1 -- 172.21.15.43:0/286791948 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0e2c0afc60 msgr2=0x7f0e2c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:58.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e4734d640 1 --2- 172.21.15.43:0/286791948 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0e2c0afc60 0x7f0e2c0b2170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x56466aa6c980 tx=0x7f0e380095c0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e4734d640 1 -- 172.21.15.43:0/286791948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e40104260 msgr2=0x7f0e40075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:58.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e4734d640 1 --2- 172.21.15.43:0/286791948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e40104260 0x7f0e40075fb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c005fd0 tx=0x7f0e3c00dde0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e4734d640 1 -- 172.21.15.43:0/286791948 shutdown_connections 2024-04-25T10:36:58.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e4734d640 1 --2- 172.21.15.43:0/286791948 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0e2c0afc60 0x7f0e2c0b2170 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e4734d640 1 --2- 172.21.15.43:0/286791948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e40109fb0 0x7f0e40076a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.033+0000 7f0e4734d640 1 --2- 172.21.15.43:0/286791948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e40109b10 0x7f0e400764f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.034+0000 7f0e4734d640 1 --2- 172.21.15.43:0/286791948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e40104260 0x7f0e40075fb0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:58.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.034+0000 7f0e4734d640 1 -- 172.21.15.43:0/286791948 >> 172.21.15.43:0/286791948 conn(0x7f0e400ff210 msgr2=0x7f0e40106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:58.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.034+0000 7f0e4734d640 1 -- 172.21.15.43:0/286791948 shutdown_connections 2024-04-25T10:36:58.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:58.034+0000 7f0e4734d640 1 -- 172.21.15.43:0/286791948 wait complete. 2024-04-25T10:36:58.044 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:57 smithi164 ceph-mon[32713]: pgmap v136: 97 pgs: 97 active+clean; 5.3 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-04-25T10:36:58.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:57 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/305130498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:36:58.045 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:57 smithi077 ceph-mon[32772]: pgmap v136: 97 pgs: 97 active+clean; 5.3 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-04-25T10:36:58.045 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:57 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/305130498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:36:58.046 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:57 smithi043 ceph-mon[29478]: pgmap v136: 97 pgs: 97 active+clean; 5.3 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-04-25T10:36:58.046 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:57 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/305130498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:36:59.048 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:36:59.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:36:59.048 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:36:59.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.179+0000 7f6d87d96640 1 -- 172.21.15.43:0/4048104503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d80108ed0 msgr2=0x7f6d80102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:59.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.179+0000 7f6d87d96640 1 --2- 172.21.15.43:0/4048104503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d80108ed0 0x7f6d80102b20 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6d70009900 tx=0x7f6d7002a410 comp rx=0 tx=0).stop 2024-04-25T10:36:59.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.179+0000 7f6d87d96640 1 -- 172.21.15.43:0/4048104503 shutdown_connections 2024-04-25T10:36:59.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.179+0000 7f6d87d96640 1 --2- 172.21.15.43:0/4048104503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d80103190 0x7f6d8010f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.179+0000 7f6d87d96640 1 --2- 172.21.15.43:0/4048104503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d80108ed0 0x7f6d80102b20 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.179+0000 7f6d87d96640 1 --2- 172.21.15.43:0/4048104503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d80108a30 0x7f6d801025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.179+0000 7f6d87d96640 1 -- 172.21.15.43:0/4048104503 >> 172.21.15.43:0/4048104503 conn(0x7f6d800fe260 msgr2=0x7f6d80100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:59.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.179+0000 7f6d87d96640 1 -- 172.21.15.43:0/4048104503 shutdown_connections 2024-04-25T10:36:59.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.179+0000 7f6d87d96640 1 -- 172.21.15.43:0/4048104503 wait complete. 2024-04-25T10:36:59.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.180+0000 7f6d87d96640 1 Processor -- start 2024-04-25T10:36:59.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.180+0000 7f6d87d96640 1 -- start start 2024-04-25T10:36:59.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.180+0000 7f6d87d96640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d80103190 0x7f6d8019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.180+0000 7f6d87d96640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d80108a30 0x7f6d8019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.180+0000 7f6d87d96640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d80108ed0 0x7f6d8019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.180+0000 7f6d87d96640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6d80112c30 con 0x7f6d80108a30 2024-04-25T10:36:59.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.180+0000 7f6d87d96640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6d80112ab0 con 0x7f6d80103190 2024-04-25T10:36:59.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.180+0000 7f6d87d96640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6d80112db0 con 0x7f6d80108ed0 2024-04-25T10:36:59.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.181+0000 7f6d85331640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d80108a30 0x7f6d8019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.181+0000 7f6d85331640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d80108a30 0x7f6d8019acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43682/0 (socket says 172.21.15.43:43682) 2024-04-25T10:36:59.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.181+0000 7f6d85331640 1 -- 172.21.15.43:0/2579591690 learned_addr learned my addr 172.21.15.43:0/2579591690 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:59.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.181+0000 7f6d86333640 1 --2- 172.21.15.43:0/2579591690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d80108ed0 0x7f6d8019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.181+0000 7f6d85b32640 1 --2- 172.21.15.43:0/2579591690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d80103190 0x7f6d8019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.181+0000 7f6d85331640 1 -- 172.21.15.43:0/2579591690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d80108ed0 msgr2=0x7f6d8019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:59.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.181+0000 7f6d85331640 1 --2- 172.21.15.43:0/2579591690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d80108ed0 0x7f6d8019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.181+0000 7f6d85331640 1 -- 172.21.15.43:0/2579591690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d80103190 msgr2=0x7f6d8019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:59.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.181+0000 7f6d85331640 1 --2- 172.21.15.43:0/2579591690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d80103190 0x7f6d8019a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.181+0000 7f6d85331640 1 -- 172.21.15.43:0/2579591690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8019f5c0 con 0x7f6d80108a30 2024-04-25T10:36:59.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.181+0000 7f6d85331640 1 --2- 172.21.15.43:0/2579591690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d80108a30 0x7f6d8019acc0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6d70006010 tx=0x7f6d70004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:59.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.181+0000 7f6d76ffd640 1 -- 172.21.15.43:0/2579591690 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7003f030 con 0x7f6d80108a30 2024-04-25T10:36:59.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.181+0000 7f6d76ffd640 1 -- 172.21.15.43:0/2579591690 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d70003aa0 con 0x7f6d80108a30 2024-04-25T10:36:59.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.181+0000 7f6d87d96640 1 -- 172.21.15.43:0/2579591690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8019f850 con 0x7f6d80108a30 2024-04-25T10:36:59.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.181+0000 7f6d76ffd640 1 -- 172.21.15.43:0/2579591690 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70035070 con 0x7f6d80108a30 2024-04-25T10:36:59.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.181+0000 7f6d87d96640 1 -- 172.21.15.43:0/2579591690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8019fd90 con 0x7f6d80108a30 2024-04-25T10:36:59.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.184+0000 7f6d8ec4a740 1 Processor -- start 2024-04-25T10:36:59.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.184+0000 7f6d76ffd640 1 -- 172.21.15.43:0/2579591690 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f6d70042030 con 0x7f6d80108a30 2024-04-25T10:36:59.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.185+0000 7f6d8ec4a740 1 -- start start 2024-04-25T10:36:59.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.185+0000 7f6d8ec4a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a025707410 0x55a025aa7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.185+0000 7f6d8ec4a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256de550 0x55a025aa7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.185+0000 7f6d8ec4a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 0x55a025aa80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.185+0000 7f6d8ec4a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a025a0db00 con 0x55a0256e6730 2024-04-25T10:36:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.185+0000 7f6d8ec4a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a0259a9920 con 0x55a0256de550 2024-04-25T10:36:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.185+0000 7f6d8ec4a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a025a09ac0 con 0x55a025707410 2024-04-25T10:36:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.185+0000 7f6d85331640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 0x55a025aa80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.185+0000 7f6d85331640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 0x55a025aa80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43698/0 (socket says 172.21.15.43:43698) 2024-04-25T10:36:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.185+0000 7f6d85331640 1 -- 172.21.15.43:0/1716839691 learned_addr learned my addr 172.21.15.43:0/1716839691 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.185+0000 7f6d85b32640 1 --2- 172.21.15.43:0/1716839691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256de550 0x55a025aa7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.185+0000 7f6d86333640 1 --2- 172.21.15.43:0/1716839691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a025707410 0x55a025aa7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.185+0000 7f6d76ffd640 1 --2- 172.21.15.43:0/2579591690 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6d6c0afce0 0x7f6d6c0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.186+0000 7f6d85331640 1 -- 172.21.15.43:0/1716839691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a025707410 msgr2=0x55a025aa7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.186+0000 7f6d85331640 1 --2- 172.21.15.43:0/1716839691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a025707410 0x55a025aa7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.186+0000 7f6d85331640 1 -- 172.21.15.43:0/1716839691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256de550 msgr2=0x55a025aa7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.186+0000 7f6d85331640 1 --2- 172.21.15.43:0/1716839691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256de550 0x55a025aa7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.186+0000 7f6d85331640 1 -- 172.21.15.43:0/1716839691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0257e77e0 con 0x55a0256e6730 2024-04-25T10:36:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.186+0000 7f6d86333640 1 --2- 172.21.15.43:0/2579591690 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6d6c0afce0 0x7f6d6c0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.186+0000 7f6d76ffd640 1 -- 172.21.15.43:0/2579591690 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6d700ff430 con 0x7f6d80108a30 2024-04-25T10:36:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.186+0000 7f6d85331640 1 --2- 172.21.15.43:0/1716839691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 0x55a025aa80a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6d700fcb20 tx=0x7f6d7008dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604fc290a79fda4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.186+0000 7f6d6bfff640 1 -- 172.21.15.43:0/1716839691 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d700c73d0 con 0x55a0256e6730 2024-04-25T10:36:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.186+0000 7f6d86333640 1 --2- 172.21.15.43:0/2579591690 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6d6c0afce0 0x7f6d6c0b21f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55a025aa4790 tx=0x7f6d78014040 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.186+0000 7f6d6bfff640 1 -- 172.21.15.43:0/1716839691 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d7008d6b0 con 0x55a0256e6730 2024-04-25T10:36:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.186+0000 7f6d6bfff640 1 -- 172.21.15.43:0/1716839691 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70095110 con 0x55a0256e6730 2024-04-25T10:36:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.187+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/1716839691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 msgr2=0x55a025aa80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.187+0000 7f6d8ec4a740 1 --2- 172.21.15.43:0/1716839691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 0x55a025aa80a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6d700fcb20 tx=0x7f6d7008dad0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.187+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/1716839691 shutdown_connections 2024-04-25T10:36:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.187+0000 7f6d8ec4a740 1 --2- 172.21.15.43:0/1716839691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256de550 0x55a025aa7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.187+0000 7f6d8ec4a740 1 --2- 172.21.15.43:0/1716839691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 0x55a025aa80a0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.187+0000 7f6d8ec4a740 1 --2- 172.21.15.43:0/1716839691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a025707410 0x55a025aa7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.187+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/1716839691 >> 172.21.15.43:0/1716839691 conn(0x55a0256fb560 msgr2=0x55a025aa3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.187+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/1716839691 shutdown_connections 2024-04-25T10:36:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.187+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/1716839691 wait complete. 2024-04-25T10:36:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.188+0000 7f6d8ec4a740 1 -- start start 2024-04-25T10:36:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.188+0000 7f6d8ec4a740 1 Processor -- start 2024-04-25T10:36:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.188+0000 7f6d8ec4a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256de550 0x55a025ab1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.188+0000 7f6d8ec4a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 0x55a025ab2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.188+0000 7f6d8ec4a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a025707410 0x55a025abdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.188+0000 7f6d8ec4a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a025ab4730 con 0x55a0256e6730 2024-04-25T10:36:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.188+0000 7f6d8ec4a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a0259a75d0 con 0x55a0256de550 2024-04-25T10:36:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.188+0000 7f6d8ec4a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a025ab48b0 con 0x55a025707410 2024-04-25T10:36:59.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.188+0000 7f6d8ec4a740 10 client.0 _ll_register_callbacks cb 0x55a025995080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:59.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.188+0000 7f6d85331640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 0x55a025ab2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.188+0000 7f6d85331640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 0x55a025ab2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43708/0 (socket says 172.21.15.43:43708) 2024-04-25T10:36:59.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.188+0000 7f6d85331640 1 -- 172.21.15.43:0/112543398 learned_addr learned my addr 172.21.15.43:0/112543398 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:59.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.189+0000 7f6d85b32640 1 --2- 172.21.15.43:0/112543398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256de550 0x55a025ab1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.189+0000 7f6d86333640 1 --2- 172.21.15.43:0/112543398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a025707410 0x55a025abdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.189+0000 7f6d85331640 1 -- 172.21.15.43:0/112543398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a025707410 msgr2=0x55a025abdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:59.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.189+0000 7f6d85331640 1 --2- 172.21.15.43:0/112543398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a025707410 0x55a025abdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.189+0000 7f6d85331640 1 -- 172.21.15.43:0/112543398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256de550 msgr2=0x55a025ab1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:59.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.189+0000 7f6d85331640 1 --2- 172.21.15.43:0/112543398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256de550 0x55a025ab1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.189+0000 7f6d85331640 1 -- 172.21.15.43:0/112543398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a025847f70 con 0x55a0256e6730 2024-04-25T10:36:59.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.189+0000 7f6d85b32640 1 --2- 172.21.15.43:0/112543398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256de550 0x55a025ab1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:59.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.189+0000 7f6d85331640 1 --2- 172.21.15.43:0/112543398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 0x55a025ab2140 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6d70032af0 tx=0x7f6d70003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e289cbbc7f1a147f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.189+0000 7f6d697fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.189+0000 7f6d697fa640 1 -- 172.21.15.43:0/112543398 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70049070 con 0x55a0256e6730 2024-04-25T10:36:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.189+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/112543398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a025a1b710 con 0x55a0256e6730 2024-04-25T10:36:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.189+0000 7f6d697fa640 1 -- 172.21.15.43:0/112543398 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6d70003c50 con 0x55a0256e6730 2024-04-25T10:36:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.189+0000 7f6d697fa640 1 -- 172.21.15.43:0/112543398 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6d70095f60 con 0x55a0256e6730 2024-04-25T10:36:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.189+0000 7f6d8ec4a740 10 client.16779 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.189+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/112543398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a0256df170 con 0x55a0256e6730 2024-04-25T10:36:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.189+0000 7f6d697fa640 1 -- 172.21.15.43:0/112543398 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a025a1b710 con 0x55a0256e6730 2024-04-25T10:36:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.190+0000 7f6d697fa640 1 -- 172.21.15.43:0/112543398 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6d700fce30 con 0x55a0256e6730 2024-04-25T10:36:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.190+0000 7f6d8ec4a740 10 client.16779 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.190+0000 7f6d8ec4a740 10 client.16779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.190+0000 7f6d8ec4a740 20 client.16779 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.190+0000 7f6d8ec4a740 1 --2- 172.21.15.43:0/112543398 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a0256c61f0 0x55a025ac3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.190+0000 7f6d8ec4a740 4 client.16779 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:36:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.190+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/112543398 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a025a15580 con 0x55a0256c61f0 2024-04-25T10:36:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.190+0000 7f6d85b32640 1 --2- 172.21.15.43:0/112543398 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a0256c61f0 0x55a025ac3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.191+0000 7f6d85b32640 1 --2- 172.21.15.43:0/112543398 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a0256c61f0 0x55a025ac3ae0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99ebb8609d7f5878 server_cookie=1bfaada15d503389 in_seq=0 out_seq=0 2024-04-25T10:36:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.191+0000 7f6d697fa640 10 client.16779 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.191+0000 7f6d697fa640 1 -- 172.21.15.43:0/112543398 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a025a15580 con 0x55a0256c61f0 2024-04-25T10:36:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.191+0000 7f6d697fa640 10 client.16779 handle_command_reply: tid=0 2024-04-25T10:36:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.191+0000 7f6d8ec4a740 1 client.16779 shutdown 2024-04-25T10:36:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.191+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/112543398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 msgr2=0x55a025ab2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.191+0000 7f6d8ec4a740 1 --2- 172.21.15.43:0/112543398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 0x55a025ab2140 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6d70032af0 tx=0x7f6d70003f50 comp rx=0 tx=0).stop 2024-04-25T10:36:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.192+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/112543398 shutdown_connections 2024-04-25T10:36:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.192+0000 7f6d8ec4a740 1 --2- 172.21.15.43:0/112543398 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a0256c61f0 0x55a025ac3ae0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.192+0000 7f6d8ec4a740 1 --2- 172.21.15.43:0/112543398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a025707410 0x55a025abdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.192+0000 7f6d8ec4a740 1 --2- 172.21.15.43:0/112543398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 0x55a025ab2140 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.192+0000 7f6d697fa640 0 client.16779 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:59.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.192+0000 7f6d8ec4a740 1 --2- 172.21.15.43:0/112543398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256de550 0x55a025ab1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.192+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/112543398 >> 172.21.15.43:0/112543398 conn(0x55a0256fb560 msgr2=0x55a025aa9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:59.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.192+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/112543398 shutdown_connections 2024-04-25T10:36:59.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.192+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/112543398 wait complete. 2024-04-25T10:36:59.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.192+0000 7f6d8ec4a740 20 client.16779 trim_cache size 0 max 16384 2024-04-25T10:36:59.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.210+0000 7f6d8ec4a740 1 Processor -- start 2024-04-25T10:36:59.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.210+0000 7f6d8ec4a740 1 -- start start 2024-04-25T10:36:59.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.210+0000 7f6d8ec4a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256c61f0 0x55a025aac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.210+0000 7f6d8ec4a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0256de550 0x55a025aa7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.210+0000 7f6d8ec4a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256e6730 0x55a025ac9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.210+0000 7f6d8ec4a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6d801b22a0 con 0x55a0256c61f0 2024-04-25T10:36:59.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.210+0000 7f6d8ec4a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6d801afb70 con 0x55a0256e6730 2024-04-25T10:36:59.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.210+0000 7f6d8ec4a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6d801ad9c0 con 0x55a0256de550 2024-04-25T10:36:59.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.210+0000 7f6d85b32640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256c61f0 0x55a025aac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.210+0000 7f6d85b32640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256c61f0 0x55a025aac2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43718/0 (socket says 172.21.15.43:43718) 2024-04-25T10:36:59.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.210+0000 7f6d85b32640 1 -- 172.21.15.43:0/2041528185 learned_addr learned my addr 172.21.15.43:0/2041528185 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.210+0000 7f6d86333640 1 --2- 172.21.15.43:0/2041528185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256e6730 0x55a025ac9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.210+0000 7f6d85b32640 1 -- 172.21.15.43:0/2041528185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0256de550 msgr2=0x55a025aa7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.210+0000 7f6d85b32640 1 --2- 172.21.15.43:0/2041528185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0256de550 0x55a025aa7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.210+0000 7f6d85b32640 1 -- 172.21.15.43:0/2041528185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256e6730 msgr2=0x55a025ac9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.210+0000 7f6d85b32640 1 --2- 172.21.15.43:0/2041528185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256e6730 0x55a025ac9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.210+0000 7f6d85b32640 1 -- 172.21.15.43:0/2041528185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a025ab2910 con 0x55a0256c61f0 2024-04-25T10:36:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.211+0000 7f6d85b32640 1 --2- 172.21.15.43:0/2041528185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256c61f0 0x55a025aac2f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c005fd0 tx=0x7f6d7c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec0c7d209fce8bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.211+0000 7f6d6b7fe640 1 -- 172.21.15.43:0/2041528185 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c01d470 con 0x55a0256c61f0 2024-04-25T10:36:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.211+0000 7f6d6b7fe640 1 -- 172.21.15.43:0/2041528185 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d7c01dac0 con 0x55a0256c61f0 2024-04-25T10:36:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.211+0000 7f6d6b7fe640 1 -- 172.21.15.43:0/2041528185 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c01c3d0 con 0x55a0256c61f0 2024-04-25T10:36:59.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.212+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/2041528185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256c61f0 msgr2=0x55a025aac2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:59.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.212+0000 7f6d8ec4a740 1 --2- 172.21.15.43:0/2041528185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256c61f0 0x55a025aac2f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c005fd0 tx=0x7f6d7c0039c0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.212+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/2041528185 shutdown_connections 2024-04-25T10:36:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.212+0000 7f6d8ec4a740 1 --2- 172.21.15.43:0/2041528185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256e6730 0x55a025ac9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.212+0000 7f6d8ec4a740 1 --2- 172.21.15.43:0/2041528185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0256de550 0x55a025aa7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.212+0000 7f6d8ec4a740 1 --2- 172.21.15.43:0/2041528185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256c61f0 0x55a025aac2f0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.212+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/2041528185 >> 172.21.15.43:0/2041528185 conn(0x55a0256fb560 msgr2=0x55a025ab3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.212+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/2041528185 shutdown_connections 2024-04-25T10:36:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.212+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/2041528185 wait complete. 2024-04-25T10:36:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.212+0000 7f6d8ec4a740 1 -- start start 2024-04-25T10:36:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.212+0000 7f6d8ec4a740 1 Processor -- start 2024-04-25T10:36:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d8ec4a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256c61f0 0x55a025ab7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d8ec4a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0256de550 0x55a025ab7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d8ec4a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 0x55a025ab80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d8ec4a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6d801ae030 con 0x55a0256e6730 2024-04-25T10:36:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d8ec4a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6d801af4a0 con 0x55a0256c61f0 2024-04-25T10:36:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d8ec4a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6d801ae700 con 0x55a0256de550 2024-04-25T10:36:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d8ec4a740 10 client.0 _ll_register_callbacks cb 0x7f6d70005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d86333640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 0x55a025ab80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d86333640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 0x55a025ab80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43720/0 (socket says 172.21.15.43:43720) 2024-04-25T10:36:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d86333640 1 -- 172.21.15.43:0/1423967566 learned_addr learned my addr 172.21.15.43:0/1423967566 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d85b32640 1 --2- 172.21.15.43:0/1423967566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256c61f0 0x55a025ab7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d85331640 1 --2- 172.21.15.43:0/1423967566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0256de550 0x55a025ab7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d86333640 1 -- 172.21.15.43:0/1423967566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0256de550 msgr2=0x55a025ab7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d86333640 1 --2- 172.21.15.43:0/1423967566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0256de550 0x55a025ab7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d86333640 1 -- 172.21.15.43:0/1423967566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256c61f0 msgr2=0x55a025ab7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d86333640 1 --2- 172.21.15.43:0/1423967566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256c61f0 0x55a025ab7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d86333640 1 -- 172.21.15.43:0/1423967566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a025aa3300 con 0x55a0256e6730 2024-04-25T10:36:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d85331640 1 --2- 172.21.15.43:0/1423967566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0256de550 0x55a025ab7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d86333640 1 --2- 172.21.15.43:0/1423967566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 0x55a025ab80c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f6d7801f910 tx=0x7f6d78009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff7ab8a5a60ce29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d697fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d697fa640 1 -- 172.21.15.43:0/1423967566 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d78010430 con 0x55a0256e6730 2024-04-25T10:36:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d697fa640 1 -- 172.21.15.43:0/1423967566 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6d780105d0 con 0x55a0256e6730 2024-04-25T10:36:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.213+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/1423967566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d7c01dac0 con 0x55a0256e6730 2024-04-25T10:36:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.214+0000 7f6d697fa640 1 -- 172.21.15.43:0/1423967566 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6d78010810 con 0x55a0256e6730 2024-04-25T10:36:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.214+0000 7f6d8ec4a740 10 client.16785 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.214+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/1423967566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a025b30390 con 0x55a0256e6730 2024-04-25T10:36:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.214+0000 7f6d697fa640 1 -- 172.21.15.43:0/1423967566 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d7c01dac0 con 0x55a0256e6730 2024-04-25T10:36:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.214+0000 7f6d697fa640 1 -- 172.21.15.43:0/1423967566 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6d78029a50 con 0x55a0256e6730 2024-04-25T10:36:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.214+0000 7f6d8ec4a740 10 client.16785 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.214+0000 7f6d8ec4a740 10 client.16785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.214+0000 7f6d8ec4a740 20 client.16785 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.214+0000 7f6d8ec4a740 1 --2- 172.21.15.43:0/1423967566 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a025707410 0x55a025b32b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.214+0000 7f6d8ec4a740 4 client.16785 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:36:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.214+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/1423967566 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a025990ac0 con 0x55a025707410 2024-04-25T10:36:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.214+0000 7f6d85b32640 1 --2- 172.21.15.43:0/1423967566 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a025707410 0x55a025b32b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.215+0000 7f6d85b32640 1 --2- 172.21.15.43:0/1423967566 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a025707410 0x55a025b32b60 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a8f07457e126023 server_cookie=d8182eb156e5c1f5 in_seq=0 out_seq=0 2024-04-25T10:36:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.215+0000 7f6d697fa640 10 client.16785 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.216+0000 7f6d697fa640 1 -- 172.21.15.43:0/1423967566 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a025990ac0 con 0x55a025707410 2024-04-25T10:36:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.216+0000 7f6d697fa640 10 client.16785 handle_command_reply: tid=0 2024-04-25T10:36:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.216+0000 7f6d8ec4a740 1 client.16785 shutdown 2024-04-25T10:36:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.216+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/1423967566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 msgr2=0x55a025ab80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.216+0000 7f6d8ec4a740 1 --2- 172.21.15.43:0/1423967566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 0x55a025ab80c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f6d7801f910 tx=0x7f6d78009500 comp rx=0 tx=0).stop 2024-04-25T10:36:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.216+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/1423967566 shutdown_connections 2024-04-25T10:36:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.216+0000 7f6d8ec4a740 1 --2- 172.21.15.43:0/1423967566 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a025707410 0x55a025b32b60 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.216+0000 7f6d8ec4a740 1 --2- 172.21.15.43:0/1423967566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0256e6730 0x55a025ab80c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.216+0000 7f6d8ec4a740 1 --2- 172.21.15.43:0/1423967566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0256de550 0x55a025ab7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.216+0000 7f6d8ec4a740 1 --2- 172.21.15.43:0/1423967566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0256c61f0 0x55a025ab7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.216+0000 7f6d697fa640 0 client.16785 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:59.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.216+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/1423967566 >> 172.21.15.43:0/1423967566 conn(0x55a0256fb560 msgr2=0x55a025aa7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:59.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.216+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/1423967566 shutdown_connections 2024-04-25T10:36:59.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.216+0000 7f6d8ec4a740 1 -- 172.21.15.43:0/1423967566 wait complete. 2024-04-25T10:36:59.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.216+0000 7f6d8ec4a740 20 client.16785 trim_cache size 0 max 16384 2024-04-25T10:36:59.223 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:36:59.223 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:36:59.223 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "216979c3-86f5-43eb-8aa2-6917ed7d1b87", 2024-04-25T10:36:59.223 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:36:59.223 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:36:59.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.217+0000 7f6d87d96640 1 -- 172.21.15.43:0/2579591690 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6d6c0afce0 msgr2=0x7f6d6c0b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:59.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.217+0000 7f6d87d96640 1 --2- 172.21.15.43:0/2579591690 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6d6c0afce0 0x7f6d6c0b21f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55a025aa4790 tx=0x7f6d78014040 comp rx=0 tx=0).stop 2024-04-25T10:36:59.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.217+0000 7f6d87d96640 1 -- 172.21.15.43:0/2579591690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d80108a30 msgr2=0x7f6d8019acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:59.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.217+0000 7f6d87d96640 1 --2- 172.21.15.43:0/2579591690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d80108a30 0x7f6d8019acc0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6d70006010 tx=0x7f6d70004060 comp rx=0 tx=0).stop 2024-04-25T10:36:59.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.217+0000 7f6d87d96640 1 -- 172.21.15.43:0/2579591690 shutdown_connections 2024-04-25T10:36:59.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.217+0000 7f6d87d96640 1 --2- 172.21.15.43:0/2579591690 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6d6c0afce0 0x7f6d6c0b21f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.217+0000 7f6d87d96640 1 --2- 172.21.15.43:0/2579591690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d80108ed0 0x7f6d8019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.217+0000 7f6d87d96640 1 --2- 172.21.15.43:0/2579591690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d80108a30 0x7f6d8019acc0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.217+0000 7f6d87d96640 1 --2- 172.21.15.43:0/2579591690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d80103190 0x7f6d8019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.217+0000 7f6d87d96640 1 -- 172.21.15.43:0/2579591690 >> 172.21.15.43:0/2579591690 conn(0x7f6d800fe260 msgr2=0x7f6d80100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:59.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.217+0000 7f6d87d96640 1 -- 172.21.15.43:0/2579591690 shutdown_connections 2024-04-25T10:36:59.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.217+0000 7f6d87d96640 1 -- 172.21.15.43:0/2579591690 wait complete. 2024-04-25T10:36:59.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:36:59.230 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:36:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.342+0000 7f2228714640 1 -- 172.21.15.43:0/749985420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2220109b10 msgr2=0x7f2220101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.342+0000 7f2228714640 1 --2- 172.21.15.43:0/749985420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2220109b10 0x7f2220101c50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f221c009900 tx=0x7f221c02a410 comp rx=0 tx=0).stop 2024-04-25T10:36:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.342+0000 7f2228714640 1 -- 172.21.15.43:0/749985420 shutdown_connections 2024-04-25T10:36:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.342+0000 7f2228714640 1 --2- 172.21.15.43:0/749985420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2220102840 0x7f2220110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.342+0000 7f2228714640 1 --2- 172.21.15.43:0/749985420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2220109fb0 0x7f22201021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.342+0000 7f2228714640 1 --2- 172.21.15.43:0/749985420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2220109b10 0x7f2220101c50 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.342+0000 7f2228714640 1 -- 172.21.15.43:0/749985420 >> 172.21.15.43:0/749985420 conn(0x7f2220077880 msgr2=0x7f2220077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.342+0000 7f2228714640 1 -- 172.21.15.43:0/749985420 shutdown_connections 2024-04-25T10:36:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.342+0000 7f2228714640 1 -- 172.21.15.43:0/749985420 wait complete. 2024-04-25T10:36:59.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.343+0000 7f2228714640 1 Processor -- start 2024-04-25T10:36:59.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.343+0000 7f2228714640 1 -- start start 2024-04-25T10:36:59.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.343+0000 7f2228714640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2220109b10 0x7f222019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.343+0000 7f2228714640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2220109fb0 0x7f222019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.343+0000 7f2228714640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f222019ecd0 0x7f222019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.343+0000 7f2228714640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2220113d10 con 0x7f2220109b10 2024-04-25T10:36:59.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.343+0000 7f2228714640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2220113b90 con 0x7f2220109fb0 2024-04-25T10:36:59.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.343+0000 7f2228714640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2220113e90 con 0x7f222019ecd0 2024-04-25T10:36:59.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.344+0000 7f22264b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2220109b10 0x7f222019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.344+0000 7f22264b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2220109b10 0x7f222019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43728/0 (socket says 172.21.15.43:43728) 2024-04-25T10:36:59.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.344+0000 7f22264b0640 1 -- 172.21.15.43:0/3981572386 learned_addr learned my addr 172.21.15.43:0/3981572386 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:59.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.344+0000 7f2225caf640 1 --2- 172.21.15.43:0/3981572386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2220109fb0 0x7f222019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.344+0000 7f2226cb1640 1 --2- 172.21.15.43:0/3981572386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f222019ecd0 0x7f222019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.344+0000 7f22264b0640 1 -- 172.21.15.43:0/3981572386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f222019ecd0 msgr2=0x7f222019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:59.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.344+0000 7f22264b0640 1 --2- 172.21.15.43:0/3981572386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f222019ecd0 0x7f222019f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.344+0000 7f22264b0640 1 -- 172.21.15.43:0/3981572386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2220109fb0 msgr2=0x7f222019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:59.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.344+0000 7f22264b0640 1 --2- 172.21.15.43:0/3981572386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2220109fb0 0x7f222019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.344+0000 7f22264b0640 1 -- 172.21.15.43:0/3981572386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222019f720 con 0x7f2220109b10 2024-04-25T10:36:59.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.344+0000 7f2225caf640 1 --2- 172.21.15.43:0/3981572386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2220109fb0 0x7f222019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:59.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.344+0000 7f22264b0640 1 --2- 172.21.15.43:0/3981572386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2220109b10 0x7f222019a4b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f221c032fd0 tx=0x7f221c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:59.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.344+0000 7f22137fe640 1 -- 172.21.15.43:0/3981572386 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c03f030 con 0x7f2220109b10 2024-04-25T10:36:59.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.344+0000 7f22137fe640 1 -- 172.21.15.43:0/3981572386 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f221c004490 con 0x7f2220109b10 2024-04-25T10:36:59.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.344+0000 7f2228714640 1 -- 172.21.15.43:0/3981572386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f222019f9b0 con 0x7f2220109b10 2024-04-25T10:36:59.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.344+0000 7f22137fe640 1 -- 172.21.15.43:0/3981572386 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c035070 con 0x7f2220109b10 2024-04-25T10:36:59.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.344+0000 7f2228714640 1 -- 172.21.15.43:0/3981572386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22201a7160 con 0x7f2220109b10 2024-04-25T10:36:59.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.347+0000 7f222f5c8740 1 Processor -- start 2024-04-25T10:36:59.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.347+0000 7f222f5c8740 1 -- start start 2024-04-25T10:36:59.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.347+0000 7f22137fe640 1 -- 172.21.15.43:0/3981572386 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f221c042030 con 0x7f2220109b10 2024-04-25T10:36:59.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.347+0000 7f222f5c8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f73711f390 0x55f7374bfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.347+0000 7f222f5c8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370f64d0 0x55f7374bffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.347+0000 7f222f5c8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370fe6b0 0x55f7374cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.347+0000 7f222f5c8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f737425d80 con 0x55f7370f64d0 2024-04-25T10:36:59.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.347+0000 7f222f5c8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f7373df8b0 con 0x55f73711f390 2024-04-25T10:36:59.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.347+0000 7f222f5c8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f737421d40 con 0x55f7370fe6b0 2024-04-25T10:36:59.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.347+0000 7f22264b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370f64d0 0x55f7374bffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.347+0000 7f22264b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370f64d0 0x55f7374bffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43730/0 (socket says 172.21.15.43:43730) 2024-04-25T10:36:59.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.347+0000 7f22264b0640 1 -- 172.21.15.43:0/173714022 learned_addr learned my addr 172.21.15.43:0/173714022 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:59.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.348+0000 7f2225caf640 1 --2- 172.21.15.43:0/173714022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370fe6b0 0x55f7374cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.348+0000 7f2226cb1640 1 --2- 172.21.15.43:0/173714022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f73711f390 0x55f7374bfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.348+0000 7f22264b0640 1 -- 172.21.15.43:0/173714022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370fe6b0 msgr2=0x55f7374cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:59.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.348+0000 7f22264b0640 1 --2- 172.21.15.43:0/173714022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370fe6b0 0x55f7374cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.348+0000 7f22264b0640 1 -- 172.21.15.43:0/173714022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f73711f390 msgr2=0x55f7374bfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:59.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.348+0000 7f22264b0640 1 --2- 172.21.15.43:0/173714022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f73711f390 0x55f7374bfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.348+0000 7f22264b0640 1 -- 172.21.15.43:0/173714022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7373b0a40 con 0x55f7370f64d0 2024-04-25T10:36:59.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.348+0000 7f2225caf640 1 --2- 172.21.15.43:0/173714022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370fe6b0 0x55f7374cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:59.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.348+0000 7f22137fe640 1 --2- 172.21.15.43:0/3981572386 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f220c0afc60 0x7f220c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.348+0000 7f2226cb1640 1 --2- 172.21.15.43:0/3981572386 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f220c0afc60 0x7f220c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.348+0000 7f22264b0640 1 --2- 172.21.15.43:0/173714022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370f64d0 0x55f7374bffa0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f221c0fccb0 tx=0x7f221c105cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be64ad5e8dd8221 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:59.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.348+0000 7f22137fe640 1 -- 172.21.15.43:0/3981572386 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f221c0ff5c0 con 0x7f2220109b10 2024-04-25T10:36:59.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.348+0000 7f2210ff9640 1 -- 172.21.15.43:0/173714022 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c049070 con 0x55f7370f64d0 2024-04-25T10:36:59.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.348+0000 7f2210ff9640 1 -- 172.21.15.43:0/173714022 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f221c0c72a0 con 0x55f7370f64d0 2024-04-25T10:36:59.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.348+0000 7f2210ff9640 1 -- 172.21.15.43:0/173714022 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c0c3650 con 0x55f7370f64d0 2024-04-25T10:36:59.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.349+0000 7f222f5c8740 1 -- 172.21.15.43:0/173714022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370f64d0 msgr2=0x55f7374bffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:59.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.349+0000 7f2226cb1640 1 --2- 172.21.15.43:0/3981572386 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f220c0afc60 0x7f220c0b2170 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55f7374bcbd0 tx=0x7f2218009520 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:59.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.349+0000 7f222f5c8740 1 --2- 172.21.15.43:0/173714022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370f64d0 0x55f7374bffa0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f221c0fccb0 tx=0x7f221c105cd0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.349+0000 7f222f5c8740 1 -- 172.21.15.43:0/173714022 shutdown_connections 2024-04-25T10:36:59.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.349+0000 7f222f5c8740 1 --2- 172.21.15.43:0/173714022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370fe6b0 0x55f7374cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.349+0000 7f222f5c8740 1 --2- 172.21.15.43:0/173714022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370f64d0 0x55f7374bffa0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.349+0000 7f222f5c8740 1 --2- 172.21.15.43:0/173714022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f73711f390 0x55f7374bfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.349+0000 7f222f5c8740 1 -- 172.21.15.43:0/173714022 >> 172.21.15.43:0/173714022 conn(0x55f7371134e0 msgr2=0x55f7374bc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:59.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.349+0000 7f222f5c8740 1 -- 172.21.15.43:0/173714022 shutdown_connections 2024-04-25T10:36:59.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.349+0000 7f222f5c8740 1 -- 172.21.15.43:0/173714022 wait complete. 2024-04-25T10:36:59.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.350+0000 7f222f5c8740 1 -- start start 2024-04-25T10:36:59.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.350+0000 7f222f5c8740 1 Processor -- start 2024-04-25T10:36:59.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.350+0000 7f222f5c8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7370f64d0 0x55f7374ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.350+0000 7f222f5c8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370fe6b0 0x55f7374caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.350+0000 7f222f5c8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f73711f390 0x55f7374d6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.350+0000 7f222f5c8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f737425d80 con 0x55f73711f390 2024-04-25T10:36:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.350+0000 7f222f5c8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f7373acd10 con 0x55f7370f64d0 2024-04-25T10:36:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.350+0000 7f222f5c8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f7374cce30 con 0x55f7370fe6b0 2024-04-25T10:36:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.350+0000 7f222f5c8740 10 client.0 _ll_register_callbacks cb 0x55f7373d9210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.350+0000 7f2226cb1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f73711f390 0x55f7374d6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.350+0000 7f2226cb1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f73711f390 0x55f7374d6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43746/0 (socket says 172.21.15.43:43746) 2024-04-25T10:36:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.350+0000 7f2226cb1640 1 -- 172.21.15.43:0/1637793851 learned_addr learned my addr 172.21.15.43:0/1637793851 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.350+0000 7f2225caf640 1 --2- 172.21.15.43:0/1637793851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370fe6b0 0x55f7374caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.351+0000 7f22264b0640 1 --2- 172.21.15.43:0/1637793851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7370f64d0 0x55f7374ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.351+0000 7f2226cb1640 1 -- 172.21.15.43:0/1637793851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370fe6b0 msgr2=0x55f7374caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.351+0000 7f2226cb1640 1 --2- 172.21.15.43:0/1637793851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370fe6b0 0x55f7374caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.351+0000 7f2226cb1640 1 -- 172.21.15.43:0/1637793851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7370f64d0 msgr2=0x55f7374ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.351+0000 7f2226cb1640 1 --2- 172.21.15.43:0/1637793851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7370f64d0 0x55f7374ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.351+0000 7f2226cb1640 1 -- 172.21.15.43:0/1637793851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7372017d0 con 0x55f73711f390 2024-04-25T10:36:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.351+0000 7f22264b0640 1 --2- 172.21.15.43:0/1637793851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7370f64d0 0x55f7374ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:36:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.351+0000 7f2226cb1640 1 --2- 172.21.15.43:0/1637793851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f73711f390 0x55f7374d6170 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f22180127d0 tx=0x7f2218004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa4ba9cf11e347 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.351+0000 7f2209ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.351+0000 7f2209ffb640 1 -- 172.21.15.43:0/1637793851 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218023070 con 0x55f73711f390 2024-04-25T10:36:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.351+0000 7f2209ffb640 1 -- 172.21.15.43:0/1637793851 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f2218011930 con 0x55f73711f390 2024-04-25T10:36:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.351+0000 7f222f5c8740 1 -- 172.21.15.43:0/1637793851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f221c0c72a0 con 0x55f73711f390 2024-04-25T10:36:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.351+0000 7f2209ffb640 1 -- 172.21.15.43:0/1637793851 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f221802f050 con 0x55f73711f390 2024-04-25T10:36:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.351+0000 7f222f5c8740 10 client.16797 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.351+0000 7f222f5c8740 1 -- 172.21.15.43:0/1637793851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f7371373f0 con 0x55f73711f390 2024-04-25T10:36:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.352+0000 7f2209ffb640 1 -- 172.21.15.43:0/1637793851 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f221c0c72a0 con 0x55f73711f390 2024-04-25T10:36:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.352+0000 7f2209ffb640 1 -- 172.21.15.43:0/1637793851 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f221800f040 con 0x55f73711f390 2024-04-25T10:36:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.352+0000 7f222f5c8740 10 client.16797 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.352+0000 7f222f5c8740 10 client.16797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.352+0000 7f222f5c8740 20 client.16797 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.352+0000 7f222f5c8740 1 --2- 172.21.15.43:0/1637793851 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f7370ddf60 0x55f7374dc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.352+0000 7f222f5c8740 4 client.16797 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:36:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.352+0000 7f222f5c8740 1 -- 172.21.15.43:0/1637793851 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f737433990 con 0x55f7370ddf60 2024-04-25T10:36:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.352+0000 7f22264b0640 1 --2- 172.21.15.43:0/1637793851 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f7370ddf60 0x55f7374dc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.352+0000 7f22264b0640 1 --2- 172.21.15.43:0/1637793851 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f7370ddf60 0x55f7374dc390 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4e7ced849de0f5f server_cookie=63bb760bc199b05e in_seq=0 out_seq=0 2024-04-25T10:36:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.352+0000 7f2209ffb640 10 client.16797 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.353+0000 7f2209ffb640 1 -- 172.21.15.43:0/1637793851 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f737433990 con 0x55f7370ddf60 2024-04-25T10:36:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.353+0000 7f2209ffb640 10 client.16797 handle_command_reply: tid=0 2024-04-25T10:36:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.353+0000 7f222f5c8740 1 client.16797 shutdown 2024-04-25T10:36:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.353+0000 7f222f5c8740 1 -- 172.21.15.43:0/1637793851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f73711f390 msgr2=0x55f7374d6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.353+0000 7f222f5c8740 1 --2- 172.21.15.43:0/1637793851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f73711f390 0x55f7374d6170 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f22180127d0 tx=0x7f2218004ab0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.353+0000 7f222f5c8740 1 -- 172.21.15.43:0/1637793851 shutdown_connections 2024-04-25T10:36:59.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.353+0000 7f222f5c8740 1 --2- 172.21.15.43:0/1637793851 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f7370ddf60 0x55f7374dc390 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.353+0000 7f222f5c8740 1 --2- 172.21.15.43:0/1637793851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f73711f390 0x55f7374d6170 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.353+0000 7f2209ffb640 0 client.16797 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:59.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.353+0000 7f222f5c8740 1 --2- 172.21.15.43:0/1637793851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370fe6b0 0x55f7374caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.353+0000 7f222f5c8740 1 --2- 172.21.15.43:0/1637793851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7370f64d0 0x55f7374ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.353+0000 7f222f5c8740 1 -- 172.21.15.43:0/1637793851 >> 172.21.15.43:0/1637793851 conn(0x55f7371134e0 msgr2=0x55f7374c1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:59.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.353+0000 7f222f5c8740 1 -- 172.21.15.43:0/1637793851 shutdown_connections 2024-04-25T10:36:59.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.353+0000 7f222f5c8740 1 -- 172.21.15.43:0/1637793851 wait complete. 2024-04-25T10:36:59.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.354+0000 7f222f5c8740 20 client.16797 trim_cache size 0 max 16384 2024-04-25T10:36:59.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.371+0000 7f222f5c8740 1 Processor -- start 2024-04-25T10:36:59.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.371+0000 7f222f5c8740 1 -- start start 2024-04-25T10:36:59.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f222f5c8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370ddf60 0x55f7374ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f222f5c8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370f64d0 0x55f7374bfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f222f5c8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7370fe6b0 0x55f7374cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f222f5c8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f22201b22f0 con 0x55f7370ddf60 2024-04-25T10:36:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f222f5c8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f22201afbc0 con 0x55f7370fe6b0 2024-04-25T10:36:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f222f5c8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22201ada10 con 0x55f7370f64d0 2024-04-25T10:36:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f22264b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370ddf60 0x55f7374ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f22264b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370ddf60 0x55f7374ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43756/0 (socket says 172.21.15.43:43756) 2024-04-25T10:36:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f22264b0640 1 -- 172.21.15.43:0/2019208681 learned_addr learned my addr 172.21.15.43:0/2019208681 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f22264b0640 1 -- 172.21.15.43:0/2019208681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370f64d0 msgr2=0x55f7374bfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f2225caf640 1 --2- 172.21.15.43:0/2019208681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370f64d0 0x55f7374bfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f22264b0640 1 --2- 172.21.15.43:0/2019208681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370f64d0 0x55f7374bfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f2226cb1640 1 --2- 172.21.15.43:0/2019208681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7370fe6b0 0x55f7374cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f22264b0640 1 -- 172.21.15.43:0/2019208681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7370fe6b0 msgr2=0x55f7374cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f22264b0640 1 --2- 172.21.15.43:0/2019208681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7370fe6b0 0x55f7374cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f22264b0640 1 -- 172.21.15.43:0/2019208681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7370f70f0 con 0x55f7370ddf60 2024-04-25T10:36:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f2225caf640 1 --2- 172.21.15.43:0/2019208681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370f64d0 0x55f7374bfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:36:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f22264b0640 1 --2- 172.21.15.43:0/2019208681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370ddf60 0x55f7374ca540 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f221c0c3810 tx=0x7f221c105730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80923366fa929e8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f220bfff640 1 -- 172.21.15.43:0/2019208681 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c049040 con 0x55f7370ddf60 2024-04-25T10:36:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f220bfff640 1 -- 172.21.15.43:0/2019208681 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f221c0c72a0 con 0x55f7370ddf60 2024-04-25T10:36:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.372+0000 7f220bfff640 1 -- 172.21.15.43:0/2019208681 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c0c6f00 con 0x55f7370ddf60 2024-04-25T10:36:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.373+0000 7f222f5c8740 1 -- 172.21.15.43:0/2019208681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370ddf60 msgr2=0x55f7374ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.373+0000 7f222f5c8740 1 --2- 172.21.15.43:0/2019208681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370ddf60 0x55f7374ca540 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f221c0c3810 tx=0x7f221c105730 comp rx=0 tx=0).stop 2024-04-25T10:36:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.373+0000 7f222f5c8740 1 -- 172.21.15.43:0/2019208681 shutdown_connections 2024-04-25T10:36:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.373+0000 7f222f5c8740 1 --2- 172.21.15.43:0/2019208681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7370fe6b0 0x55f7374cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.373+0000 7f222f5c8740 1 --2- 172.21.15.43:0/2019208681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370f64d0 0x55f7374bfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.373+0000 7f222f5c8740 1 --2- 172.21.15.43:0/2019208681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370ddf60 0x55f7374ca540 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.373+0000 7f222f5c8740 1 -- 172.21.15.43:0/2019208681 >> 172.21.15.43:0/2019208681 conn(0x55f7371134e0 msgr2=0x55f7374cba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.373+0000 7f222f5c8740 1 -- 172.21.15.43:0/2019208681 shutdown_connections 2024-04-25T10:36:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.373+0000 7f222f5c8740 1 -- 172.21.15.43:0/2019208681 wait complete. 2024-04-25T10:36:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.373+0000 7f222f5c8740 1 -- start start 2024-04-25T10:36:59.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.374+0000 7f222f5c8740 1 Processor -- start 2024-04-25T10:36:59.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.374+0000 7f222f5c8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370ddf60 0x55f7374d2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.374+0000 7f222f5c8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7370f64d0 0x55f7374d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.374+0000 7f222f5c8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370fe6b0 0x55f7374cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.374+0000 7f222f5c8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f22201ae080 con 0x55f7370fe6b0 2024-04-25T10:36:59.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.374+0000 7f222f5c8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f22201af4f0 con 0x55f7370f64d0 2024-04-25T10:36:59.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.374+0000 7f222f5c8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22201ae750 con 0x55f7370ddf60 2024-04-25T10:36:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.374+0000 7f222f5c8740 10 client.0 _ll_register_callbacks cb 0x7f2218016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:36:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.374+0000 7f2226cb1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370fe6b0 0x55f7374cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.374+0000 7f2226cb1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370fe6b0 0x55f7374cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43760/0 (socket says 172.21.15.43:43760) 2024-04-25T10:36:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.374+0000 7f2226cb1640 1 -- 172.21.15.43:0/1448165326 learned_addr learned my addr 172.21.15.43:0/1448165326 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:36:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.374+0000 7f2226cb1640 1 -- 172.21.15.43:0/1448165326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370ddf60 msgr2=0x55f7374d2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.374+0000 7f22264b0640 1 --2- 172.21.15.43:0/1448165326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370ddf60 0x55f7374d2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.374+0000 7f2226cb1640 1 --2- 172.21.15.43:0/1448165326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370ddf60 0x55f7374d2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.374+0000 7f2226cb1640 1 -- 172.21.15.43:0/1448165326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7370f64d0 msgr2=0x55f7374d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:36:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.374+0000 7f2225caf640 1 --2- 172.21.15.43:0/1448165326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7370f64d0 0x55f7374d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.374+0000 7f2226cb1640 1 --2- 172.21.15.43:0/1448165326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7370f64d0 0x55f7374d2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.374+0000 7f2226cb1640 1 -- 172.21.15.43:0/1448165326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7373e64b0 con 0x55f7370fe6b0 2024-04-25T10:36:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.374+0000 7f2226cb1640 1 --2- 172.21.15.43:0/1448165326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370fe6b0 0x55f7374cd770 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2218010cc0 tx=0x7f2218027660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399e21e4930e77f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:36:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.375+0000 7f2209ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:36:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.375+0000 7f2209ffb640 1 -- 172.21.15.43:0/1448165326 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218023040 con 0x55f7370fe6b0 2024-04-25T10:36:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.375+0000 7f2209ffb640 1 -- 172.21.15.43:0/1448165326 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f2218011730 con 0x55f7370fe6b0 2024-04-25T10:36:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.375+0000 7f222f5c8740 1 -- 172.21.15.43:0/1448165326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f221c0c72a0 con 0x55f7370fe6b0 2024-04-25T10:36:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.375+0000 7f2209ffb640 1 -- 172.21.15.43:0/1448165326 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2218043050 con 0x55f7370fe6b0 2024-04-25T10:36:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.375+0000 7f222f5c8740 10 client.16803 fetch_fsmap learned FSMap version 26 2024-04-25T10:36:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.375+0000 7f222f5c8740 1 -- 172.21.15.43:0/1448165326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f737543f20 con 0x55f7370fe6b0 2024-04-25T10:36:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.375+0000 7f2209ffb640 1 -- 172.21.15.43:0/1448165326 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f221c0c72a0 con 0x55f7370fe6b0 2024-04-25T10:36:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.375+0000 7f2209ffb640 1 -- 172.21.15.43:0/1448165326 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2218027c40 con 0x55f7370fe6b0 2024-04-25T10:36:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.375+0000 7f222f5c8740 10 client.16803 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:36:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.375+0000 7f222f5c8740 10 client.16803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:36:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.375+0000 7f222f5c8740 20 client.16803 populate_metadata read hostname 'smithi043' 2024-04-25T10:36:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.375+0000 7f222f5c8740 1 --2- 172.21.15.43:0/1448165326 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f73711f390 0x55f7375466a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:36:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.375+0000 7f222f5c8740 4 client.16803 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:36:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.375+0000 7f222f5c8740 1 -- 172.21.15.43:0/1448165326 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f7374c5020 con 0x55f73711f390 2024-04-25T10:36:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.376+0000 7f22264b0640 1 --2- 172.21.15.43:0/1448165326 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f73711f390 0x55f7375466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:36:59.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.376+0000 7f22264b0640 1 --2- 172.21.15.43:0/1448165326 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f73711f390 0x55f7375466a0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5864f20b350b3b18 server_cookie=c89e7df1184618f0 in_seq=0 out_seq=0 2024-04-25T10:36:59.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.376+0000 7f2209ffb640 10 client.16803 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:59.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.376+0000 7f2209ffb640 1 -- 172.21.15.43:0/1448165326 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f7374c5020 con 0x55f73711f390 2024-04-25T10:36:59.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.376+0000 7f2209ffb640 10 client.16803 handle_command_reply: tid=0 2024-04-25T10:36:59.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.376+0000 7f222f5c8740 1 client.16803 shutdown 2024-04-25T10:36:59.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.376+0000 7f222f5c8740 1 -- 172.21.15.43:0/1448165326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370fe6b0 msgr2=0x55f7374cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:59.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.376+0000 7f222f5c8740 1 --2- 172.21.15.43:0/1448165326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370fe6b0 0x55f7374cd770 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2218010cc0 tx=0x7f2218027660 comp rx=0 tx=0).stop 2024-04-25T10:36:59.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.376+0000 7f222f5c8740 1 -- 172.21.15.43:0/1448165326 shutdown_connections 2024-04-25T10:36:59.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.376+0000 7f222f5c8740 1 --2- 172.21.15.43:0/1448165326 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f73711f390 0x55f7375466a0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.376+0000 7f222f5c8740 1 --2- 172.21.15.43:0/1448165326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7370f64d0 0x55f7374d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.376+0000 7f222f5c8740 1 --2- 172.21.15.43:0/1448165326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7370fe6b0 0x55f7374cd770 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.376+0000 7f2209ffb640 0 client.16803 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:36:59.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.376+0000 7f222f5c8740 1 --2- 172.21.15.43:0/1448165326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7370ddf60 0x55f7374d2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.376+0000 7f222f5c8740 1 -- 172.21.15.43:0/1448165326 >> 172.21.15.43:0/1448165326 conn(0x55f7371134e0 msgr2=0x55f7374cb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:59.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.376+0000 7f222f5c8740 1 -- 172.21.15.43:0/1448165326 shutdown_connections 2024-04-25T10:36:59.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.376+0000 7f222f5c8740 1 -- 172.21.15.43:0/1448165326 wait complete. 2024-04-25T10:36:59.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.376+0000 7f222f5c8740 20 client.16803 trim_cache size 0 max 16384 2024-04-25T10:36:59.385 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:36:59.385 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:36:59.385 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:36:59.385 INFO:teuthology.orchestra.run.smithi043.stdout: "216979c3-86f5-43eb-8aa2-6917ed7d1b87": { 2024-04-25T10:36:59.385 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:36:59.385 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "216979c3-86f5-43eb-8aa2-6917ed7d1b87", 2024-04-25T10:36:59.385 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:36:59.386 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:36:59.386 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:36:59.386 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:36:59.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.377+0000 7f2228714640 1 -- 172.21.15.43:0/3981572386 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f220c0afc60 msgr2=0x7f220c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:59.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.377+0000 7f2228714640 1 --2- 172.21.15.43:0/3981572386 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f220c0afc60 0x7f220c0b2170 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55f7374bcbd0 tx=0x7f2218009520 comp rx=0 tx=0).stop 2024-04-25T10:36:59.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.377+0000 7f2228714640 1 -- 172.21.15.43:0/3981572386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2220109b10 msgr2=0x7f222019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:36:59.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.377+0000 7f2228714640 1 --2- 172.21.15.43:0/3981572386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2220109b10 0x7f222019a4b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f221c032fd0 tx=0x7f221c004270 comp rx=0 tx=0).stop 2024-04-25T10:36:59.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.377+0000 7f2228714640 1 -- 172.21.15.43:0/3981572386 shutdown_connections 2024-04-25T10:36:59.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.377+0000 7f2228714640 1 --2- 172.21.15.43:0/3981572386 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f220c0afc60 0x7f220c0b2170 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.377+0000 7f2228714640 1 --2- 172.21.15.43:0/3981572386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2220109fb0 0x7f222019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.377+0000 7f2228714640 1 --2- 172.21.15.43:0/3981572386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f222019ecd0 0x7f222019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.377+0000 7f2228714640 1 --2- 172.21.15.43:0/3981572386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2220109b10 0x7f222019a4b0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:36:59.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.377+0000 7f2228714640 1 -- 172.21.15.43:0/3981572386 >> 172.21.15.43:0/3981572386 conn(0x7f2220077880 msgr2=0x7f22201051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:36:59.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.377+0000 7f2228714640 1 -- 172.21.15.43:0/3981572386 shutdown_connections 2024-04-25T10:36:59.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:36:59.377+0000 7f2228714640 1 -- 172.21.15.43:0/3981572386 wait complete. 2024-04-25T10:36:59.390 INFO:tasks.cephfs.filesystem:scrub status for tag:216979c3-86f5-43eb-8aa2-6917ed7d1b87 - {'path': '/', 'tag': '216979c3-86f5-43eb-8aa2-6917ed7d1b87', 'options': 'recursive,force'} 2024-04-25T10:37:00.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:59 smithi077 ceph-mon[32772]: pgmap v137: 97 pgs: 97 active+clean; 1.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-25T10:37:00.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:59 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:37:00.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:59 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:37:00.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:59 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:37:00.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:36:59 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:37:00.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:59 smithi164 ceph-mon[32713]: pgmap v137: 97 pgs: 97 active+clean; 1.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-25T10:37:00.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:59 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:37:00.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:59 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:37:00.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:59 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:37:00.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:36:59 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:37:00.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:59 smithi043 ceph-mon[29478]: pgmap v137: 97 pgs: 97 active+clean; 1.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-25T10:37:00.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:59 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:37:00.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:59 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:37:00.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:59 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:37:00.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:36:59 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:37:02.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:01 smithi077 ceph-mon[32772]: pgmap v138: 97 pgs: 97 active+clean; 1.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-25T10:37:02.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:01 smithi164 ceph-mon[32713]: pgmap v138: 97 pgs: 97 active+clean; 1.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-25T10:37:02.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:01 smithi043 ceph-mon[29478]: pgmap v138: 97 pgs: 97 active+clean; 1.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-25T10:37:04.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:03 smithi077 ceph-mon[32772]: pgmap v139: 97 pgs: 97 active+clean; 1.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-25T10:37:04.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:03 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:37:04.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:03 smithi164 ceph-mon[32713]: pgmap v139: 97 pgs: 97 active+clean; 1.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-25T10:37:04.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:03 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:37:04.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:03 smithi043 ceph-mon[29478]: pgmap v139: 97 pgs: 97 active+clean; 1.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-25T10:37:04.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:03 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:37:06.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:05 smithi164 ceph-mon[32713]: pgmap v140: 97 pgs: 97 active+clean; 1.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 67 KiB/s wr, 23 op/s 2024-04-25T10:37:06.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:05 smithi043 ceph-mon[29478]: pgmap v140: 97 pgs: 97 active+clean; 1.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 67 KiB/s wr, 23 op/s 2024-04-25T10:37:06.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:05 smithi077 ceph-mon[32772]: pgmap v140: 97 pgs: 97 active+clean; 1.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 67 KiB/s wr, 23 op/s 2024-04-25T10:37:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.909+0000 7f011b4b1640 1 -- 172.21.15.43:0/758359020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01141121c0 msgr2=0x7f01141145e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.909+0000 7f011b4b1640 1 --2- 172.21.15.43:0/758359020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01141121c0 0x7f01141145e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0104009900 tx=0x7f010402a480 comp rx=0 tx=0).stop 2024-04-25T10:37:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.909+0000 7f011b4b1640 1 -- 172.21.15.43:0/758359020 shutdown_connections 2024-04-25T10:37:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.909+0000 7f011b4b1640 1 --2- 172.21.15.43:0/758359020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0114114b20 0x7f0114116f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.909+0000 7f011b4b1640 1 --2- 172.21.15.43:0/758359020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01141121c0 0x7f01141145e0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.909+0000 7f011b4b1640 1 --2- 172.21.15.43:0/758359020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0114076470 0x7f0114076880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.909+0000 7f011b4b1640 1 -- 172.21.15.43:0/758359020 >> 172.21.15.43:0/758359020 conn(0x7f0114079170 msgr2=0x7f01140795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.909+0000 7f011b4b1640 1 -- 172.21.15.43:0/758359020 shutdown_connections 2024-04-25T10:37:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.909+0000 7f011b4b1640 1 -- 172.21.15.43:0/758359020 wait complete. 2024-04-25T10:37:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.910+0000 7f011b4b1640 1 Processor -- start 2024-04-25T10:37:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.910+0000 7f011b4b1640 1 -- start start 2024-04-25T10:37:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.910+0000 7f011b4b1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0114076470 0x7f01141a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.910+0000 7f011b4b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01141121c0 0x7f01141a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.910+0000 7f011b4b1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0114114b20 0x7f01141a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.910+0000 7f011b4b1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f011411a320 con 0x7f01141121c0 2024-04-25T10:37:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.910+0000 7f011b4b1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f011411a1a0 con 0x7f0114076470 2024-04-25T10:37:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.910+0000 7f011b4b1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f011411a4a0 con 0x7f0114114b20 2024-04-25T10:37:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.910+0000 7f0118a4c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01141121c0 0x7f01141a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.910+0000 7f0118a4c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01141121c0 0x7f01141a1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43764/0 (socket says 172.21.15.43:43764) 2024-04-25T10:37:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.910+0000 7f0118a4c640 1 -- 172.21.15.43:0/2293802740 learned_addr learned my addr 172.21.15.43:0/2293802740 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.910+0000 7f0119a4e640 1 --2- 172.21.15.43:0/2293802740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0114114b20 0x7f01141a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.910+0000 7f011924d640 1 --2- 172.21.15.43:0/2293802740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0114076470 0x7f01141a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.910+0000 7f0118a4c640 1 -- 172.21.15.43:0/2293802740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0114114b20 msgr2=0x7f01141a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.910+0000 7f0118a4c640 1 --2- 172.21.15.43:0/2293802740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0114114b20 0x7f01141a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.910+0000 7f0118a4c640 1 -- 172.21.15.43:0/2293802740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0114076470 msgr2=0x7f01141a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.910+0000 7f0118a4c640 1 --2- 172.21.15.43:0/2293802740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0114076470 0x7f01141a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.911+0000 7f0118a4c640 1 -- 172.21.15.43:0/2293802740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01141a5c20 con 0x7f01141121c0 2024-04-25T10:37:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.911+0000 7f0118a4c640 1 --2- 172.21.15.43:0/2293802740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01141121c0 0x7f01141a1320 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0104005940 tx=0x7f01040040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.911+0000 7f010a7fc640 1 -- 172.21.15.43:0/2293802740 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010403f030 con 0x7f01141121c0 2024-04-25T10:37:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.911+0000 7f010a7fc640 1 -- 172.21.15.43:0/2293802740 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01040044c0 con 0x7f01141121c0 2024-04-25T10:37:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.911+0000 7f011b4b1640 1 -- 172.21.15.43:0/2293802740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01141a5eb0 con 0x7f01141121c0 2024-04-25T10:37:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.911+0000 7f010a7fc640 1 -- 172.21.15.43:0/2293802740 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104035070 con 0x7f01141121c0 2024-04-25T10:37:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.911+0000 7f011b4b1640 1 -- 172.21.15.43:0/2293802740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01141a63f0 con 0x7f01141121c0 2024-04-25T10:37:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.913+0000 7f010a7fc640 1 -- 172.21.15.43:0/2293802740 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f0104042030 con 0x7f01141121c0 2024-04-25T10:37:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.914+0000 7f0121ffd740 1 Processor -- start 2024-04-25T10:37:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.914+0000 7f0121ffd740 1 -- start start 2024-04-25T10:37:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.914+0000 7f010a7fc640 1 --2- 172.21.15.43:0/2293802740 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f01000afcd0 0x7f01000b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.914+0000 7f0121ffd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac591b0 0x56423aff8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.914+0000 7f0121ffd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac302f0 0x56423aff9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.914+0000 7f0121ffd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac384d0 0x56423aff9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.914+0000 7f0121ffd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56423af5f460 con 0x56423ac591b0 2024-04-25T10:37:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.914+0000 7f0121ffd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56423af19590 con 0x56423ac302f0 2024-04-25T10:37:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.914+0000 7f0121ffd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56423af5b420 con 0x56423ac384d0 2024-04-25T10:37:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.915+0000 7f011924d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac591b0 0x56423aff8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.915+0000 7f0119a4e640 1 --2- 172.21.15.43:0/2293802740 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f01000afcd0 0x7f01000b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.915+0000 7f010a7fc640 1 -- 172.21.15.43:0/2293802740 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f01040ff400 con 0x7f01141121c0 2024-04-25T10:37:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.915+0000 7f011924d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac591b0 0x56423aff8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43770/0 (socket says 172.21.15.43:43770) 2024-04-25T10:37:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.915+0000 7f011924d640 1 -- 172.21.15.43:0/2977191647 learned_addr learned my addr 172.21.15.43:0/2977191647 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.915+0000 7f0118a4c640 1 --2- 172.21.15.43:0/2977191647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac302f0 0x56423aff9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.915+0000 7f0119a4e640 1 --2- 172.21.15.43:0/2977191647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac384d0 0x56423aff9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.915+0000 7f011924d640 1 -- 172.21.15.43:0/2977191647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac384d0 msgr2=0x56423aff9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.915+0000 7f011924d640 1 --2- 172.21.15.43:0/2977191647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac384d0 0x56423aff9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.915+0000 7f011924d640 1 -- 172.21.15.43:0/2977191647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac302f0 msgr2=0x56423aff9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.915+0000 7f011924d640 1 --2- 172.21.15.43:0/2977191647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac302f0 0x56423aff9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.915+0000 7f011924d640 1 -- 172.21.15.43:0/2977191647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56423af260d0 con 0x56423ac591b0 2024-04-25T10:37:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.915+0000 7f011924d640 1 --2- 172.21.15.43:0/2977191647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac591b0 0x56423aff8af0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0110009c90 tx=0x7f01100117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebec2b740c8f334 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.915+0000 7f0119a4e640 1 --2- 172.21.15.43:0/2293802740 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f01000afcd0 0x7f01000b21e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f010c009fd0 tx=0x7f010c013b00 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.915+0000 7f00ff7fe640 1 -- 172.21.15.43:0/2977191647 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110012450 con 0x56423ac591b0 2024-04-25T10:37:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.915+0000 7f00ff7fe640 1 -- 172.21.15.43:0/2977191647 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0110012aa0 con 0x56423ac591b0 2024-04-25T10:37:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.915+0000 7f00ff7fe640 1 -- 172.21.15.43:0/2977191647 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011000fb60 con 0x56423ac591b0 2024-04-25T10:37:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.916+0000 7f0121ffd740 1 -- 172.21.15.43:0/2977191647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac591b0 msgr2=0x56423aff8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.916+0000 7f0121ffd740 1 --2- 172.21.15.43:0/2977191647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac591b0 0x56423aff8af0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0110009c90 tx=0x7f01100117b0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.916+0000 7f0121ffd740 1 -- 172.21.15.43:0/2977191647 shutdown_connections 2024-04-25T10:37:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.916+0000 7f0121ffd740 1 --2- 172.21.15.43:0/2977191647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac302f0 0x56423aff9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.916+0000 7f0121ffd740 1 --2- 172.21.15.43:0/2977191647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac384d0 0x56423aff9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.916+0000 7f0121ffd740 1 --2- 172.21.15.43:0/2977191647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac591b0 0x56423aff8af0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.916+0000 7f0121ffd740 1 -- 172.21.15.43:0/2977191647 >> 172.21.15.43:0/2977191647 conn(0x56423ac4d300 msgr2=0x56423aff52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.916+0000 7f0121ffd740 1 -- 172.21.15.43:0/2977191647 shutdown_connections 2024-04-25T10:37:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.916+0000 7f0121ffd740 1 -- 172.21.15.43:0/2977191647 wait complete. 2024-04-25T10:37:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.917+0000 7f0121ffd740 1 -- start start 2024-04-25T10:37:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.917+0000 7f0121ffd740 1 Processor -- start 2024-04-25T10:37:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.917+0000 7f0121ffd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac302f0 0x56423b004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.917+0000 7f0121ffd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac384d0 0x56423b005050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.917+0000 7f0121ffd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac591b0 0x56423b002fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.917+0000 7f0121ffd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56423b005c20 con 0x56423ac302f0 2024-04-25T10:37:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.917+0000 7f0121ffd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56423aee6b60 con 0x56423ac591b0 2024-04-25T10:37:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.917+0000 7f0121ffd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56423b005da0 con 0x56423ac384d0 2024-04-25T10:37:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.917+0000 7f0121ffd740 10 client.0 _ll_register_callbacks cb 0x56423af0b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.918+0000 7f011924d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac302f0 0x56423b004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.918+0000 7f011924d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac302f0 0x56423b004b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43778/0 (socket says 172.21.15.43:43778) 2024-04-25T10:37:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.918+0000 7f011924d640 1 -- 172.21.15.43:0/2912284374 learned_addr learned my addr 172.21.15.43:0/2912284374 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.918+0000 7f0119a4e640 1 --2- 172.21.15.43:0/2912284374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac591b0 0x56423b002fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.918+0000 7f0118a4c640 1 --2- 172.21.15.43:0/2912284374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac384d0 0x56423b005050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.918+0000 7f011924d640 1 -- 172.21.15.43:0/2912284374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac384d0 msgr2=0x56423b005050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.918+0000 7f011924d640 1 --2- 172.21.15.43:0/2912284374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac384d0 0x56423b005050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.918+0000 7f011924d640 1 -- 172.21.15.43:0/2912284374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac591b0 msgr2=0x56423b002fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.918+0000 7f011924d640 1 --2- 172.21.15.43:0/2912284374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac591b0 0x56423b002fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.918+0000 7f011924d640 1 -- 172.21.15.43:0/2912284374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56423ac71210 con 0x56423ac302f0 2024-04-25T10:37:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.918+0000 7f0118a4c640 1 --2- 172.21.15.43:0/2912284374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac384d0 0x56423b005050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:37:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.918+0000 7f0119a4e640 1 --2- 172.21.15.43:0/2912284374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac591b0 0x56423b002fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:37:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.918+0000 7f011924d640 1 --2- 172.21.15.43:0/2912284374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac302f0 0x56423b004b10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f01100095a0 tx=0x7f0110012930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e953e83d644ae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.918+0000 7f00fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.918+0000 7f00fcff9640 1 -- 172.21.15.43:0/2912284374 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110027020 con 0x56423ac302f0 2024-04-25T10:37:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.918+0000 7f00fcff9640 1 -- 172.21.15.43:0/2912284374 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f011000f8a0 con 0x56423ac302f0 2024-04-25T10:37:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.918+0000 7f0121ffd740 1 -- 172.21.15.43:0/2912284374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0110012aa0 con 0x56423ac302f0 2024-04-25T10:37:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.919+0000 7f00fcff9640 1 -- 172.21.15.43:0/2912284374 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0110032050 con 0x56423ac302f0 2024-04-25T10:37:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.919+0000 7f0121ffd740 10 client.16815 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.919+0000 7f0121ffd740 1 -- 172.21.15.43:0/2912284374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56423ad3d620 con 0x56423ac302f0 2024-04-25T10:37:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.919+0000 7f00fcff9640 1 -- 172.21.15.43:0/2912284374 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0110012aa0 con 0x56423ac302f0 2024-04-25T10:37:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.919+0000 7f00fcff9640 1 -- 172.21.15.43:0/2912284374 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0110022030 con 0x56423ac302f0 2024-04-25T10:37:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.919+0000 7f0121ffd740 10 client.16815 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.919+0000 7f0121ffd740 10 client.16815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:37:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.919+0000 7f0121ffd740 20 client.16815 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.919+0000 7f0121ffd740 1 --2- 172.21.15.43:0/2912284374 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56423ac17d80 0x56423b015680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.919+0000 7f0121ffd740 4 client.16815 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:37:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.919+0000 7f0121ffd740 1 -- 172.21.15.43:0/2912284374 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56423af6d070 con 0x56423ac17d80 2024-04-25T10:37:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.919+0000 7f0118a4c640 1 --2- 172.21.15.43:0/2912284374 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56423ac17d80 0x56423b015680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.920+0000 7f0118a4c640 1 --2- 172.21.15.43:0/2912284374 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56423ac17d80 0x56423b015680 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91b2ad078c333ba9 server_cookie=2d6acd7fc808573d in_seq=0 out_seq=0 2024-04-25T10:37:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.920+0000 7f00fcff9640 10 client.16815 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.921+0000 7f00fcff9640 1 -- 172.21.15.43:0/2912284374 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56423af6d070 con 0x56423ac17d80 2024-04-25T10:37:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.921+0000 7f00fcff9640 10 client.16815 handle_command_reply: tid=0 2024-04-25T10:37:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.921+0000 7f0121ffd740 1 client.16815 shutdown 2024-04-25T10:37:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.921+0000 7f0121ffd740 1 -- 172.21.15.43:0/2912284374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac302f0 msgr2=0x56423b004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.921+0000 7f0121ffd740 1 --2- 172.21.15.43:0/2912284374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac302f0 0x56423b004b10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f01100095a0 tx=0x7f0110012930 comp rx=0 tx=0).stop 2024-04-25T10:37:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.921+0000 7f0121ffd740 1 -- 172.21.15.43:0/2912284374 shutdown_connections 2024-04-25T10:37:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.921+0000 7f0121ffd740 1 --2- 172.21.15.43:0/2912284374 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56423ac17d80 0x56423b015680 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.921+0000 7f0121ffd740 1 --2- 172.21.15.43:0/2912284374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac591b0 0x56423b002fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.921+0000 7f0121ffd740 1 --2- 172.21.15.43:0/2912284374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac384d0 0x56423b005050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.921+0000 7f0121ffd740 1 --2- 172.21.15.43:0/2912284374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac302f0 0x56423b004b10 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.921+0000 7f00fcff9640 0 client.16815 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.921+0000 7f0121ffd740 1 -- 172.21.15.43:0/2912284374 >> 172.21.15.43:0/2912284374 conn(0x56423ac4d300 msgr2=0x56423aff88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.921+0000 7f0121ffd740 1 -- 172.21.15.43:0/2912284374 shutdown_connections 2024-04-25T10:37:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.921+0000 7f0121ffd740 1 -- 172.21.15.43:0/2912284374 wait complete. 2024-04-25T10:37:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.921+0000 7f0121ffd740 20 client.16815 trim_cache size 0 max 16384 2024-04-25T10:37:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.936+0000 7f0121ffd740 1 Processor -- start 2024-04-25T10:37:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.936+0000 7f0121ffd740 1 -- start start 2024-04-25T10:37:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.937+0000 7f0121ffd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac302f0 0x56423b002f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.937+0000 7f0121ffd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac384d0 0x56423b0186b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.937+0000 7f0121ffd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac591b0 0x56423b018bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.937+0000 7f0121ffd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f01141b8890 con 0x56423ac302f0 2024-04-25T10:37:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.937+0000 7f0121ffd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f01141b6160 con 0x56423ac384d0 2024-04-25T10:37:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.937+0000 7f0121ffd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f01141b3fb0 con 0x56423ac591b0 2024-04-25T10:37:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.937+0000 7f011924d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac302f0 0x56423b002f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.937+0000 7f011924d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac302f0 0x56423b002f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43788/0 (socket says 172.21.15.43:43788) 2024-04-25T10:37:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.937+0000 7f011924d640 1 -- 172.21.15.43:0/2493993320 learned_addr learned my addr 172.21.15.43:0/2493993320 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.937+0000 7f0119a4e640 1 --2- 172.21.15.43:0/2493993320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac591b0 0x56423b018bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.937+0000 7f011924d640 1 -- 172.21.15.43:0/2493993320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac591b0 msgr2=0x56423b018bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.937+0000 7f0118a4c640 1 --2- 172.21.15.43:0/2493993320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac384d0 0x56423b0186b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.937+0000 7f011924d640 1 --2- 172.21.15.43:0/2493993320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac591b0 0x56423b018bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.937+0000 7f011924d640 1 -- 172.21.15.43:0/2493993320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac384d0 msgr2=0x56423b0186b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.937+0000 7f011924d640 1 --2- 172.21.15.43:0/2493993320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac384d0 0x56423b0186b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.937+0000 7f011924d640 1 -- 172.21.15.43:0/2493993320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56423aff0a60 con 0x56423ac302f0 2024-04-25T10:37:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.937+0000 7f011924d640 1 --2- 172.21.15.43:0/2493993320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac302f0 0x56423b002f50 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0110001eb0 tx=0x7f0110004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdef2802e89e7d98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.937+0000 7f00feffd640 1 -- 172.21.15.43:0/2493993320 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110000e60 con 0x56423ac302f0 2024-04-25T10:37:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.937+0000 7f00feffd640 1 -- 172.21.15.43:0/2493993320 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0110012e50 con 0x56423ac302f0 2024-04-25T10:37:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.938+0000 7f00feffd640 1 -- 172.21.15.43:0/2493993320 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110027040 con 0x56423ac302f0 2024-04-25T10:37:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.938+0000 7f0121ffd740 1 -- 172.21.15.43:0/2493993320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac302f0 msgr2=0x56423b002f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.938+0000 7f0121ffd740 1 --2- 172.21.15.43:0/2493993320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac302f0 0x56423b002f50 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0110001eb0 tx=0x7f0110004990 comp rx=0 tx=0).stop 2024-04-25T10:37:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.938+0000 7f0121ffd740 1 -- 172.21.15.43:0/2493993320 shutdown_connections 2024-04-25T10:37:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.938+0000 7f0121ffd740 1 --2- 172.21.15.43:0/2493993320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac384d0 0x56423b0186b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.938+0000 7f0121ffd740 1 --2- 172.21.15.43:0/2493993320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac591b0 0x56423b018bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.938+0000 7f0121ffd740 1 --2- 172.21.15.43:0/2493993320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac302f0 0x56423b002f50 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.938+0000 7f0121ffd740 1 -- 172.21.15.43:0/2493993320 >> 172.21.15.43:0/2493993320 conn(0x56423ac4d300 msgr2=0x56423b002840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.938+0000 7f0121ffd740 1 -- 172.21.15.43:0/2493993320 shutdown_connections 2024-04-25T10:37:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.938+0000 7f0121ffd740 1 -- 172.21.15.43:0/2493993320 wait complete. 2024-04-25T10:37:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.939+0000 7f0121ffd740 1 -- start start 2024-04-25T10:37:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.939+0000 7f0121ffd740 1 Processor -- start 2024-04-25T10:37:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.939+0000 7f0121ffd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac302f0 0x56423b06f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.939+0000 7f0121ffd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac384d0 0x56423b06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.939+0000 7f0121ffd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac591b0 0x56423b073d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.939+0000 7f0121ffd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f01141b4620 con 0x56423ac384d0 2024-04-25T10:37:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.939+0000 7f0121ffd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f01141b5a90 con 0x56423ac302f0 2024-04-25T10:37:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.939+0000 7f0121ffd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f01141b4cf0 con 0x56423ac591b0 2024-04-25T10:37:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.939+0000 7f0121ffd740 10 client.0 _ll_register_callbacks cb 0x7f0104005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.939+0000 7f0118a4c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac384d0 0x56423b06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.940+0000 7f0118a4c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac384d0 0x56423b06fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43792/0 (socket says 172.21.15.43:43792) 2024-04-25T10:37:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.940+0000 7f0118a4c640 1 -- 172.21.15.43:0/1764422723 learned_addr learned my addr 172.21.15.43:0/1764422723 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.940+0000 7f0119a4e640 1 --2- 172.21.15.43:0/1764422723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac591b0 0x56423b073d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.940+0000 7f011924d640 1 --2- 172.21.15.43:0/1764422723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac302f0 0x56423b06f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.940+0000 7f0118a4c640 1 -- 172.21.15.43:0/1764422723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac591b0 msgr2=0x56423b073d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.940+0000 7f0118a4c640 1 --2- 172.21.15.43:0/1764422723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac591b0 0x56423b073d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.940+0000 7f0118a4c640 1 -- 172.21.15.43:0/1764422723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac302f0 msgr2=0x56423b06f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.940+0000 7f0118a4c640 1 --2- 172.21.15.43:0/1764422723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac302f0 0x56423b06f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.940+0000 7f0118a4c640 1 -- 172.21.15.43:0/1764422723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56423b074300 con 0x56423ac384d0 2024-04-25T10:37:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.940+0000 7f0118a4c640 1 --2- 172.21.15.43:0/1764422723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac384d0 0x56423b06fea0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f01040096c0 tx=0x7f01040c34a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1b71d7f531d68a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.940+0000 7f00fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.940+0000 7f00fcff9640 1 -- 172.21.15.43:0/1764422723 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104049040 con 0x56423ac384d0 2024-04-25T10:37:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.940+0000 7f00fcff9640 1 -- 172.21.15.43:0/1764422723 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f01040fcda0 con 0x56423ac384d0 2024-04-25T10:37:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.940+0000 7f0121ffd740 1 -- 172.21.15.43:0/1764422723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0110012e50 con 0x56423ac384d0 2024-04-25T10:37:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.940+0000 7f00fcff9640 1 -- 172.21.15.43:0/1764422723 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f01040fcfa0 con 0x56423ac384d0 2024-04-25T10:37:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.940+0000 7f0121ffd740 10 client.16821 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.940+0000 7f0121ffd740 1 -- 172.21.15.43:0/1764422723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56423b07c8a0 con 0x56423ac384d0 2024-04-25T10:37:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.941+0000 7f00fcff9640 1 -- 172.21.15.43:0/1764422723 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0110012e50 con 0x56423ac384d0 2024-04-25T10:37:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.941+0000 7f00fcff9640 1 -- 172.21.15.43:0/1764422723 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f01040ee030 con 0x56423ac384d0 2024-04-25T10:37:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.941+0000 7f0121ffd740 10 client.16821 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.941+0000 7f0121ffd740 10 client.16821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:37:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.941+0000 7f0121ffd740 20 client.16821 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.941+0000 7f0121ffd740 1 --2- 172.21.15.43:0/1764422723 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56423abed1c0 0x56423b07f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.941+0000 7f0121ffd740 4 client.16821 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:37:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.941+0000 7f0121ffd740 1 -- 172.21.15.43:0/1764422723 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56423af66ee0 con 0x56423abed1c0 2024-04-25T10:37:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.941+0000 7f011924d640 1 --2- 172.21.15.43:0/1764422723 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56423abed1c0 0x56423b07f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.942+0000 7f011924d640 1 --2- 172.21.15.43:0/1764422723 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56423abed1c0 0x56423b07f040 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7f12ecb96ac7331 server_cookie=3ac74e11fa9086cb in_seq=0 out_seq=0 2024-04-25T10:37:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.942+0000 7f00fcff9640 10 client.16821 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f00fcff9640 1 -- 172.21.15.43:0/1764422723 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56423af66ee0 con 0x56423abed1c0 2024-04-25T10:37:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f00fcff9640 10 client.16821 handle_command_reply: tid=0 2024-04-25T10:37:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f0121ffd740 1 client.16821 shutdown 2024-04-25T10:37:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f0121ffd740 1 -- 172.21.15.43:0/1764422723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac384d0 msgr2=0x56423b06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f0121ffd740 1 --2- 172.21.15.43:0/1764422723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac384d0 0x56423b06fea0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f01040096c0 tx=0x7f01040c34a0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f0121ffd740 1 -- 172.21.15.43:0/1764422723 shutdown_connections 2024-04-25T10:37:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f0121ffd740 1 --2- 172.21.15.43:0/1764422723 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56423abed1c0 0x56423b07f040 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f0121ffd740 1 --2- 172.21.15.43:0/1764422723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56423ac591b0 0x56423b073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f00fcff9640 0 client.16821 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f0121ffd740 1 --2- 172.21.15.43:0/1764422723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56423ac384d0 0x56423b06fea0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f0121ffd740 1 --2- 172.21.15.43:0/1764422723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423ac302f0 0x56423b06f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f0121ffd740 1 -- 172.21.15.43:0/1764422723 >> 172.21.15.43:0/1764422723 conn(0x56423ac4d300 msgr2=0x56423aff8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f0121ffd740 1 -- 172.21.15.43:0/1764422723 shutdown_connections 2024-04-25T10:37:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f0121ffd740 1 -- 172.21.15.43:0/1764422723 wait complete. 2024-04-25T10:37:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f0121ffd740 20 client.16821 trim_cache size 0 max 16384 2024-04-25T10:37:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:37:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:37:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:37:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:37:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f011b4b1640 1 -- 172.21.15.43:0/2293802740 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f01000afcd0 msgr2=0x7f01000b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f011b4b1640 1 --2- 172.21.15.43:0/2293802740 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f01000afcd0 0x7f01000b21e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f010c009fd0 tx=0x7f010c013b00 comp rx=0 tx=0).stop 2024-04-25T10:37:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f011b4b1640 1 -- 172.21.15.43:0/2293802740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01141121c0 msgr2=0x7f01141a1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f011b4b1640 1 --2- 172.21.15.43:0/2293802740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01141121c0 0x7f01141a1320 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0104005940 tx=0x7f01040040c0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f011b4b1640 1 -- 172.21.15.43:0/2293802740 shutdown_connections 2024-04-25T10:37:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f011b4b1640 1 --2- 172.21.15.43:0/2293802740 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f01000afcd0 0x7f01000b21e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f011b4b1640 1 --2- 172.21.15.43:0/2293802740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0114114b20 0x7f01141a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f011b4b1640 1 --2- 172.21.15.43:0/2293802740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01141121c0 0x7f01141a1320 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f011b4b1640 1 --2- 172.21.15.43:0/2293802740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0114076470 0x7f01141a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f011b4b1640 1 -- 172.21.15.43:0/2293802740 >> 172.21.15.43:0/2293802740 conn(0x7f0114079170 msgr2=0x7f01141125d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f011b4b1640 1 -- 172.21.15.43:0/2293802740 shutdown_connections 2024-04-25T10:37:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:06.959+0000 7f011b4b1640 1 -- 172.21.15.43:0/2293802740 wait complete. 2024-04-25T10:37:08.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:07 smithi043 ceph-mon[29478]: pgmap v141: 97 pgs: 97 active+clean; 1.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-04-25T10:37:08.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:07 smithi077 ceph-mon[32772]: pgmap v141: 97 pgs: 97 active+clean; 1.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-04-25T10:37:08.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:07 smithi164 ceph-mon[32713]: pgmap v141: 97 pgs: 97 active+clean; 1.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-04-25T10:37:10.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:09 smithi043 ceph-mon[29478]: pgmap v142: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 75 KiB/s wr, 26 op/s 2024-04-25T10:37:10.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:09 smithi077 ceph-mon[32772]: pgmap v142: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 75 KiB/s wr, 26 op/s 2024-04-25T10:37:10.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:09 smithi164 ceph-mon[32713]: pgmap v142: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 75 KiB/s wr, 26 op/s 2024-04-25T10:37:12.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:11 smithi043 ceph-mon[29478]: pgmap v143: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-04-25T10:37:12.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:11 smithi077 ceph-mon[32772]: pgmap v143: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-04-25T10:37:12.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:11 smithi164 ceph-mon[32713]: pgmap v143: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-04-25T10:37:14.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:13 smithi043 ceph-mon[29478]: pgmap v144: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-04-25T10:37:14.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:13 smithi077 ceph-mon[32772]: pgmap v144: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-04-25T10:37:14.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:13 smithi164 ceph-mon[32713]: pgmap v144: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-04-25T10:37:16.093 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:15 smithi077 ceph-mon[32772]: pgmap v145: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 72 KiB/s wr, 23 op/s 2024-04-25T10:37:16.093 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:15 smithi164 ceph-mon[32713]: pgmap v145: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 72 KiB/s wr, 23 op/s 2024-04-25T10:37:16.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:15 smithi043 ceph-mon[29478]: pgmap v145: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 72 KiB/s wr, 23 op/s 2024-04-25T10:37:18.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:17 smithi043 ceph-mon[29478]: pgmap v146: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T10:37:18.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:17 smithi077 ceph-mon[32772]: pgmap v146: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T10:37:18.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:17 smithi164 ceph-mon[32713]: pgmap v146: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T10:37:19.301 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.301 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.301 INFO:tasks.workunit.client.0.smithi043.stdout:127/600: open fd = 3 2024-04-25T10:37:19.301 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.301 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.301 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.301 INFO:tasks.workunit.client.0.smithi043.stdout:128/600: open fd = 3 2024-04-25T10:37:19.301 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.301 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.302 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.302 INFO:tasks.workunit.client.0.smithi043.stdout:129/600: open fd = 3 2024-04-25T10:37:19.302 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.302 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.302 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.302 INFO:tasks.workunit.client.0.smithi043.stdout:130/600: open fd = 3 2024-04-25T10:37:19.302 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.302 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.302 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.302 INFO:tasks.workunit.client.0.smithi043.stdout:131/600: open fd = 3 2024-04-25T10:37:19.302 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.302 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.302 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.303 INFO:tasks.workunit.client.0.smithi043.stdout:132/600: open fd = 3 2024-04-25T10:37:19.303 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.303 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.303 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.303 INFO:tasks.workunit.client.0.smithi043.stdout:133/600: open fd = 3 2024-04-25T10:37:19.303 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.303 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.303 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.303 INFO:tasks.workunit.client.0.smithi043.stdout:134/600: open fd = 3 2024-04-25T10:37:19.303 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.303 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.303 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.303 INFO:tasks.workunit.client.0.smithi043.stdout:135/600: open fd = 3 2024-04-25T10:37:19.303 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.303 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.304 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.304 INFO:tasks.workunit.client.0.smithi043.stdout:136/600: open fd = 3 2024-04-25T10:37:19.304 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.304 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.304 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.304 INFO:tasks.workunit.client.0.smithi043.stdout:137/600: open fd = 3 2024-04-25T10:37:19.304 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.304 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.304 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.304 INFO:tasks.workunit.client.0.smithi043.stdout:138/600: open fd = 3 2024-04-25T10:37:19.304 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.304 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.304 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.304 INFO:tasks.workunit.client.0.smithi043.stdout:139/600: open fd = 3 2024-04-25T10:37:19.304 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.305 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.305 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.305 INFO:tasks.workunit.client.0.smithi043.stdout:140/600: open fd = 3 2024-04-25T10:37:19.305 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.305 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.305 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.305 INFO:tasks.workunit.client.0.smithi043.stdout:141/600: open fd = 3 2024-04-25T10:37:19.305 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.305 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.305 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.305 INFO:tasks.workunit.client.0.smithi043.stdout:142/600: open fd = 3 2024-04-25T10:37:19.305 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.305 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.305 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.305 INFO:tasks.workunit.client.0.smithi043.stdout:143/600: open fd = 3 2024-04-25T10:37:19.306 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.306 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.306 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.306 INFO:tasks.workunit.client.0.smithi043.stdout:144/600: open fd = 3 2024-04-25T10:37:19.306 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.306 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.306 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.306 INFO:tasks.workunit.client.0.smithi043.stdout:145/600: open fd = 3 2024-04-25T10:37:19.306 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.306 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.306 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.306 INFO:tasks.workunit.client.0.smithi043.stdout:146/600: open fd = 3 2024-04-25T10:37:19.306 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.306 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.307 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.307 INFO:tasks.workunit.client.0.smithi043.stdout:147/600: open fd = 3 2024-04-25T10:37:19.307 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.307 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.307 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.307 INFO:tasks.workunit.client.0.smithi043.stdout:148/600: open fd = 3 2024-04-25T10:37:19.307 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.307 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.307 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.307 INFO:tasks.workunit.client.0.smithi043.stdout:149/600: open fd = 3 2024-04-25T10:37:19.307 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.307 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.307 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.307 INFO:tasks.workunit.client.0.smithi043.stdout:150/600: open fd = 3 2024-04-25T10:37:19.307 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.308 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.308 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.308 INFO:tasks.workunit.client.0.smithi043.stdout:151/600: open fd = 3 2024-04-25T10:37:19.308 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.308 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.308 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.308 INFO:tasks.workunit.client.0.smithi043.stdout:152/600: open fd = 3 2024-04-25T10:37:19.308 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.308 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.308 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.308 INFO:tasks.workunit.client.0.smithi043.stdout:153/600: open fd = 3 2024-04-25T10:37:19.308 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.308 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.308 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.308 INFO:tasks.workunit.client.0.smithi043.stdout:154/600: open fd = 3 2024-04-25T10:37:19.309 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.309 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.309 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.309 INFO:tasks.workunit.client.0.smithi043.stdout:155/600: open fd = 3 2024-04-25T10:37:19.309 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.309 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.309 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.309 INFO:tasks.workunit.client.0.smithi043.stdout:156/600: open fd = 3 2024-04-25T10:37:19.309 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.309 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.309 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.309 INFO:tasks.workunit.client.0.smithi043.stdout:157/600: open fd = 3 2024-04-25T10:37:19.309 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.309 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.309 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.310 INFO:tasks.workunit.client.0.smithi043.stdout:158/600: open fd = 3 2024-04-25T10:37:19.310 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.310 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.310 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.310 INFO:tasks.workunit.client.0.smithi043.stdout:159/600: open fd = 3 2024-04-25T10:37:19.310 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.310 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.310 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.310 INFO:tasks.workunit.client.0.smithi043.stdout:160/600: open fd = 3 2024-04-25T10:37:19.310 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.310 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.310 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.310 INFO:tasks.workunit.client.0.smithi043.stdout:161/600: open fd = 3 2024-04-25T10:37:19.310 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.311 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.311 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.311 INFO:tasks.workunit.client.0.smithi043.stdout:162/600: open fd = 3 2024-04-25T10:37:19.311 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.311 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.311 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.311 INFO:tasks.workunit.client.0.smithi043.stdout:163/600: open fd = 3 2024-04-25T10:37:19.311 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.311 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.311 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.311 INFO:tasks.workunit.client.0.smithi043.stdout:164/600: open fd = 3 2024-04-25T10:37:19.311 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.311 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.311 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.311 INFO:tasks.workunit.client.0.smithi043.stdout:165/600: open fd = 3 2024-04-25T10:37:19.311 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.312 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.312 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.312 INFO:tasks.workunit.client.0.smithi043.stdout:166/600: open fd = 3 2024-04-25T10:37:19.312 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.312 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.312 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.312 INFO:tasks.workunit.client.0.smithi043.stdout:167/600: open fd = 3 2024-04-25T10:37:19.312 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.312 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.312 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.312 INFO:tasks.workunit.client.0.smithi043.stdout:168/600: open fd = 3 2024-04-25T10:37:19.312 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.312 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.312 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.312 INFO:tasks.workunit.client.0.smithi043.stdout:169/600: open fd = 3 2024-04-25T10:37:19.313 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.313 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.313 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.313 INFO:tasks.workunit.client.0.smithi043.stdout:170/600: open fd = 3 2024-04-25T10:37:19.313 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.313 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.313 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.313 INFO:tasks.workunit.client.0.smithi043.stdout:171/600: open fd = 3 2024-04-25T10:37:19.313 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.313 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.313 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.313 INFO:tasks.workunit.client.0.smithi043.stdout:172/600: open fd = 3 2024-04-25T10:37:19.313 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.313 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.314 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.314 INFO:tasks.workunit.client.0.smithi043.stdout:173/600: open fd = 3 2024-04-25T10:37:19.314 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.314 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.314 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.314 INFO:tasks.workunit.client.0.smithi043.stdout:174/600: open fd = 3 2024-04-25T10:37:19.314 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.314 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.314 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.314 INFO:tasks.workunit.client.0.smithi043.stdout:175/600: open fd = 3 2024-04-25T10:37:19.314 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.314 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.314 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.314 INFO:tasks.workunit.client.0.smithi043.stdout:176/600: open fd = 3 2024-04-25T10:37:19.315 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.315 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.315 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.315 INFO:tasks.workunit.client.0.smithi043.stdout:177/600: open fd = 3 2024-04-25T10:37:19.315 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.315 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.315 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.315 INFO:tasks.workunit.client.0.smithi043.stdout:178/600: open fd = 3 2024-04-25T10:37:19.315 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.315 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.315 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.315 INFO:tasks.workunit.client.0.smithi043.stdout:179/600: open fd = 3 2024-04-25T10:37:19.315 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.315 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.315 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.316 INFO:tasks.workunit.client.0.smithi043.stdout:180/600: open fd = 3 2024-04-25T10:37:19.316 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.316 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.316 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.316 INFO:tasks.workunit.client.0.smithi043.stdout:181/600: open fd = 3 2024-04-25T10:37:19.316 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.316 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.316 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.316 INFO:tasks.workunit.client.0.smithi043.stdout:182/600: open fd = 3 2024-04-25T10:37:19.316 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.316 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.316 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.316 INFO:tasks.workunit.client.0.smithi043.stdout:183/600: open fd = 3 2024-04-25T10:37:19.316 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.317 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.317 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.317 INFO:tasks.workunit.client.0.smithi043.stdout:184/600: open fd = 3 2024-04-25T10:37:19.317 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.317 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.317 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.317 INFO:tasks.workunit.client.0.smithi043.stdout:185/600: open fd = 3 2024-04-25T10:37:19.317 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.317 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.317 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.317 INFO:tasks.workunit.client.0.smithi043.stdout:186/600: open fd = 3 2024-04-25T10:37:19.317 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.317 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.317 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.317 INFO:tasks.workunit.client.0.smithi043.stdout:187/600: open fd = 3 2024-04-25T10:37:19.318 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.318 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.318 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:37:19.318 INFO:tasks.workunit.client.0.smithi043.stdout:188/600: open fd = 3 2024-04-25T10:37:19.318 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:19.318 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:37:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.070+0000 7fdf1a964640 1 -- 172.21.15.43:0/4029263436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf14110480 msgr2=0x7fdf141128a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.070+0000 7fdf1a964640 1 --2- 172.21.15.43:0/4029263436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf14110480 0x7fdf141128a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009ca0 tx=0x7fdf0c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.070+0000 7fdf1a964640 1 -- 172.21.15.43:0/4029263436 shutdown_connections 2024-04-25T10:37:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.070+0000 7fdf1a964640 1 --2- 172.21.15.43:0/4029263436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf14110480 0x7fdf141128a0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.070+0000 7fdf1a964640 1 --2- 172.21.15.43:0/4029263436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1410db20 0x7fdf1410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.070+0000 7fdf1a964640 1 --2- 172.21.15.43:0/4029263436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf141028e0 0x7fdf14102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.070+0000 7fdf1a964640 1 -- 172.21.15.43:0/4029263436 >> 172.21.15.43:0/4029263436 conn(0x7fdf140781b0 msgr2=0x7fdf140785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.070+0000 7fdf1a964640 1 -- 172.21.15.43:0/4029263436 shutdown_connections 2024-04-25T10:37:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.070+0000 7fdf1a964640 1 -- 172.21.15.43:0/4029263436 wait complete. 2024-04-25T10:37:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.071+0000 7fdf1a964640 1 Processor -- start 2024-04-25T10:37:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.071+0000 7fdf1a964640 1 -- start start 2024-04-25T10:37:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.071+0000 7fdf1a964640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf141028e0 0x7fdf141a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.071+0000 7fdf1a964640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1410db20 0x7fdf141a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.071+0000 7fdf1a964640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf14110480 0x7fdf1419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.071+0000 7fdf1a964640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf14115f40 con 0x7fdf141028e0 2024-04-25T10:37:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.071+0000 7fdf1a964640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdf14115dc0 con 0x7fdf14110480 2024-04-25T10:37:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.071+0000 7fdf1a964640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141160c0 con 0x7fdf1410db20 2024-04-25T10:37:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.071+0000 7fdf13fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf141028e0 0x7fdf141a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.071+0000 7fdf13fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf141028e0 0x7fdf141a25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34642/0 (socket says 172.21.15.43:34642) 2024-04-25T10:37:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.071+0000 7fdf13fff640 1 -- 172.21.15.43:0/1595891935 learned_addr learned my addr 172.21.15.43:0/1595891935 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.072+0000 7fdf137fe640 1 --2- 172.21.15.43:0/1595891935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1410db20 0x7fdf141a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.072+0000 7fdf18f01640 1 --2- 172.21.15.43:0/1595891935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf14110480 0x7fdf1419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.072+0000 7fdf13fff640 1 -- 172.21.15.43:0/1595891935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1410db20 msgr2=0x7fdf141a2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.072+0000 7fdf13fff640 1 --2- 172.21.15.43:0/1595891935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1410db20 0x7fdf141a2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.072+0000 7fdf13fff640 1 -- 172.21.15.43:0/1595891935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf14110480 msgr2=0x7fdf1419c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.072+0000 7fdf13fff640 1 --2- 172.21.15.43:0/1595891935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf14110480 0x7fdf1419c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.072+0000 7fdf13fff640 1 -- 172.21.15.43:0/1595891935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1419ce90 con 0x7fdf141028e0 2024-04-25T10:37:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.072+0000 7fdf18f01640 1 --2- 172.21.15.43:0/1595891935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf14110480 0x7fdf1419c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:37:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.072+0000 7fdf137fe640 1 --2- 172.21.15.43:0/1595891935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1410db20 0x7fdf141a2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:37:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.072+0000 7fdf13fff640 1 --2- 172.21.15.43:0/1595891935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf141028e0 0x7fdf141a25e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00d950 tx=0x7fdefc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.072+0000 7fdf117fa640 1 -- 172.21.15.43:0/1595891935 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc017030 con 0x7fdf141028e0 2024-04-25T10:37:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.072+0000 7fdf1a964640 1 -- 172.21.15.43:0/1595891935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1419d120 con 0x7fdf141028e0 2024-04-25T10:37:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.072+0000 7fdf117fa640 1 -- 172.21.15.43:0/1595891935 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdefc004020 con 0x7fdf141028e0 2024-04-25T10:37:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.072+0000 7fdf117fa640 1 -- 172.21.15.43:0/1595891935 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc0047a0 con 0x7fdf141028e0 2024-04-25T10:37:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.072+0000 7fdf1a964640 1 -- 172.21.15.43:0/1595891935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf141a9380 con 0x7fdf141028e0 2024-04-25T10:37:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.075+0000 7fdf117fa640 1 -- 172.21.15.43:0/1595891935 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fdefc009a30 con 0x7fdf141028e0 2024-04-25T10:37:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.075+0000 7fdf21815740 1 Processor -- start 2024-04-25T10:37:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.075+0000 7fdf21815740 1 -- start start 2024-04-25T10:37:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.075+0000 7fdf21815740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622f41b0 0x557662693af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.075+0000 7fdf21815740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622cb2f0 0x557662694030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.075+0000 7fdf21815740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622d34d0 0x557662694570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.075+0000 7fdf21815740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5576625fa460 con 0x5576622d34d0 2024-04-25T10:37:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.075+0000 7fdf21815740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5576625b4590 con 0x5576622f41b0 2024-04-25T10:37:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.075+0000 7fdf21815740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5576625f6420 con 0x5576622cb2f0 2024-04-25T10:37:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.075+0000 7fdf137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622d34d0 0x557662694570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.075+0000 7fdf137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622d34d0 0x557662694570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34658/0 (socket says 172.21.15.43:34658) 2024-04-25T10:37:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.075+0000 7fdf137fe640 1 -- 172.21.15.43:0/3970809072 learned_addr learned my addr 172.21.15.43:0/3970809072 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.076+0000 7fdf18f01640 1 --2- 172.21.15.43:0/3970809072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622f41b0 0x557662693af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.076+0000 7fdf13fff640 1 --2- 172.21.15.43:0/3970809072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622cb2f0 0x557662694030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.076+0000 7fdf117fa640 1 --2- 172.21.15.43:0/1595891935 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdf000afc50 0x7fdf000b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.076+0000 7fdf137fe640 1 -- 172.21.15.43:0/3970809072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622cb2f0 msgr2=0x557662694030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.076+0000 7fdf137fe640 1 --2- 172.21.15.43:0/3970809072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622cb2f0 0x557662694030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.076+0000 7fdf137fe640 1 -- 172.21.15.43:0/3970809072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622f41b0 msgr2=0x557662693af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.076+0000 7fdf137fe640 1 --2- 172.21.15.43:0/3970809072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622f41b0 0x557662693af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.076+0000 7fdf137fe640 1 -- 172.21.15.43:0/3970809072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576625c10d0 con 0x5576622d34d0 2024-04-25T10:37:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.076+0000 7fdf13fff640 1 --2- 172.21.15.43:0/3970809072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622cb2f0 0x557662694030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:37:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.076+0000 7fdf18f01640 1 --2- 172.21.15.43:0/1595891935 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdf000afc50 0x7fdf000b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.076+0000 7fdf117fa640 1 -- 172.21.15.43:0/1595891935 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fdefc0deb00 con 0x7fdf141028e0 2024-04-25T10:37:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.076+0000 7fdf137fe640 1 --2- 172.21.15.43:0/3970809072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622d34d0 0x557662694570 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400cb90 tx=0x7fdf0400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503a0c22e4fec402 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.076+0000 7fdf0a7fc640 1 -- 172.21.15.43:0/3970809072 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04005220 con 0x5576622d34d0 2024-04-25T10:37:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.076+0000 7fdf0a7fc640 1 -- 172.21.15.43:0/3970809072 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf04004c50 con 0x5576622d34d0 2024-04-25T10:37:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.076+0000 7fdf0a7fc640 1 -- 172.21.15.43:0/3970809072 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04010730 con 0x5576622d34d0 2024-04-25T10:37:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.077+0000 7fdf21815740 1 -- 172.21.15.43:0/3970809072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622d34d0 msgr2=0x557662694570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.077+0000 7fdf21815740 1 --2- 172.21.15.43:0/3970809072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622d34d0 0x557662694570 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400cb90 tx=0x7fdf0400e720 comp rx=0 tx=0).stop 2024-04-25T10:37:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.077+0000 7fdf18f01640 1 --2- 172.21.15.43:0/1595891935 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdf000afc50 0x7fdf000b2160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x557662690c60 tx=0x7fdf0c0095c0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.077+0000 7fdf21815740 1 -- 172.21.15.43:0/3970809072 shutdown_connections 2024-04-25T10:37:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.077+0000 7fdf21815740 1 --2- 172.21.15.43:0/3970809072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622d34d0 0x557662694570 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.077+0000 7fdf21815740 1 --2- 172.21.15.43:0/3970809072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622cb2f0 0x557662694030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.077+0000 7fdf21815740 1 --2- 172.21.15.43:0/3970809072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622f41b0 0x557662693af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.077+0000 7fdf21815740 1 -- 172.21.15.43:0/3970809072 >> 172.21.15.43:0/3970809072 conn(0x5576622e8300 msgr2=0x5576626902a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.077+0000 7fdf21815740 1 -- 172.21.15.43:0/3970809072 shutdown_connections 2024-04-25T10:37:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.077+0000 7fdf21815740 1 -- 172.21.15.43:0/3970809072 wait complete. 2024-04-25T10:37:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.078+0000 7fdf21815740 1 -- start start 2024-04-25T10:37:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.078+0000 7fdf21815740 1 Processor -- start 2024-04-25T10:37:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.078+0000 7fdf21815740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622cb2f0 0x55766269e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.078+0000 7fdf21815740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622d34d0 0x55766269e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.078+0000 7fdf21815740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622f41b0 0x5576626a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.078+0000 7fdf21815740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5576626a0c20 con 0x5576622f41b0 2024-04-25T10:37:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.078+0000 7fdf21815740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557662581b60 con 0x5576622cb2f0 2024-04-25T10:37:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.078+0000 7fdf21815740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5576626a0da0 con 0x5576622d34d0 2024-04-25T10:37:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.078+0000 7fdf21815740 10 client.0 _ll_register_callbacks cb 0x5576625a62b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.079+0000 7fdf18f01640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622f41b0 0x5576626a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.079+0000 7fdf13fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622cb2f0 0x55766269e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.079+0000 7fdf13fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622cb2f0 0x55766269e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:33286/0 (socket says 172.21.15.43:33286) 2024-04-25T10:37:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.079+0000 7fdf13fff640 1 -- 172.21.15.43:0/1958209482 learned_addr learned my addr 172.21.15.43:0/1958209482 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.079+0000 7fdf137fe640 1 --2- 172.21.15.43:0/1958209482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622d34d0 0x55766269e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.079+0000 7fdf13fff640 1 -- 172.21.15.43:0/1958209482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622d34d0 msgr2=0x55766269e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.079+0000 7fdf13fff640 1 --2- 172.21.15.43:0/1958209482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622d34d0 0x55766269e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.079+0000 7fdf13fff640 1 -- 172.21.15.43:0/1958209482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622f41b0 msgr2=0x5576626a40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.079+0000 7fdf13fff640 1 --2- 172.21.15.43:0/1958209482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622f41b0 0x5576626a40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.079+0000 7fdf13fff640 1 -- 172.21.15.43:0/1958209482 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55766230c210 con 0x5576622cb2f0 2024-04-25T10:37:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.079+0000 7fdf18f01640 1 --2- 172.21.15.43:0/1958209482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622f41b0 0x5576626a40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:37:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.079+0000 7fdf13fff640 1 --2- 172.21.15.43:0/1958209482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622cb2f0 0x55766269e120 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0a3f40 tx=0x7fdefc0a6570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9242485eb233a862 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.079+0000 7fdef37fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-04-25T10:37:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.079+0000 7fdef37fe640 1 -- 172.21.15.43:0/1958209482 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc0a73d0 con 0x5576622cb2f0 2024-04-25T10:37:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.079+0000 7fdf21815740 1 -- 172.21.15.43:0/1958209482 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf04004c50 con 0x5576622cb2f0 2024-04-25T10:37:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.079+0000 7fdef37fe640 1 -- 172.21.15.43:0/1958209482 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fdefc0dc500 con 0x5576622cb2f0 2024-04-25T10:37:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.080+0000 7fdef37fe640 1 -- 172.21.15.43:0/1958209482 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdefc0a6cd0 con 0x5576622cb2f0 2024-04-25T10:37:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.080+0000 7fdf21815740 10 client.26369 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.080+0000 7fdf21815740 1 -- 172.21.15.43:0/1958209482 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5576623d8620 con 0x5576622cb2f0 2024-04-25T10:37:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.080+0000 7fdef37fe640 1 -- 172.21.15.43:0/1958209482 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf04004c50 con 0x5576622cb2f0 2024-04-25T10:37:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.080+0000 7fdef37fe640 1 -- 172.21.15.43:0/1958209482 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdefc0a2f60 con 0x5576622cb2f0 2024-04-25T10:37:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.080+0000 7fdf21815740 10 client.26369 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.080+0000 7fdf21815740 10 client.26369 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:37:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.080+0000 7fdf21815740 20 client.26369 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.080+0000 7fdf21815740 1 --2- 172.21.15.43:0/1958209482 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5576622b2d80 0x5576626b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.080+0000 7fdf21815740 4 client.26369 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:37:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.080+0000 7fdf21815740 1 -- 172.21.15.43:0/1958209482 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557662608070 con 0x5576622b2d80 2024-04-25T10:37:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.080+0000 7fdf137fe640 1 --2- 172.21.15.43:0/1958209482 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5576622b2d80 0x5576626b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.081+0000 7fdf137fe640 1 --2- 172.21.15.43:0/1958209482 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5576622b2d80 0x5576626b05e0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=421ffd667bebfd8e server_cookie=6ef8f0ad772014ba in_seq=0 out_seq=0 2024-04-25T10:37:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.081+0000 7fdef37fe640 10 client.26369 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.082+0000 7fdef37fe640 1 -- 172.21.15.43:0/1958209482 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557662608070 con 0x5576622b2d80 2024-04-25T10:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.082+0000 7fdef37fe640 10 client.26369 handle_command_reply: tid=0 2024-04-25T10:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.082+0000 7fdf21815740 1 client.26369 shutdown 2024-04-25T10:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.082+0000 7fdf21815740 1 -- 172.21.15.43:0/1958209482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622cb2f0 msgr2=0x55766269e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.082+0000 7fdf21815740 1 --2- 172.21.15.43:0/1958209482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622cb2f0 0x55766269e120 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0a3f40 tx=0x7fdefc0a6570 comp rx=0 tx=0).stop 2024-04-25T10:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.082+0000 7fdf21815740 1 -- 172.21.15.43:0/1958209482 shutdown_connections 2024-04-25T10:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.082+0000 7fdf21815740 1 --2- 172.21.15.43:0/1958209482 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5576622b2d80 0x5576626b05e0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.082+0000 7fdf21815740 1 --2- 172.21.15.43:0/1958209482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622f41b0 0x5576626a40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.082+0000 7fdf21815740 1 --2- 172.21.15.43:0/1958209482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622d34d0 0x55766269e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.082+0000 7fdf21815740 1 --2- 172.21.15.43:0/1958209482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622cb2f0 0x55766269e120 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.082+0000 7fdef37fe640 0 client.26369 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.082+0000 7fdf21815740 1 -- 172.21.15.43:0/1958209482 >> 172.21.15.43:0/1958209482 conn(0x5576622e8300 msgr2=0x557662695810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.082+0000 7fdf21815740 1 -- 172.21.15.43:0/1958209482 shutdown_connections 2024-04-25T10:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.082+0000 7fdf21815740 1 -- 172.21.15.43:0/1958209482 wait complete. 2024-04-25T10:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.082+0000 7fdf21815740 20 client.26369 trim_cache size 0 max 16384 2024-04-25T10:37:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.098+0000 7fdf21815740 1 Processor -- start 2024-04-25T10:37:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.098+0000 7fdf21815740 1 -- start start 2024-04-25T10:37:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.098+0000 7fdf21815740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622cb2f0 0x5576626a4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.098+0000 7fdf21815740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622d34d0 0x5576626a0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.098+0000 7fdf21815740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622f41b0 0x5576626977a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.098+0000 7fdf21815740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141b43d0 con 0x5576622f41b0 2024-04-25T10:37:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.098+0000 7fdf21815740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141b1ca0 con 0x5576622d34d0 2024-04-25T10:37:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.098+0000 7fdf21815740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141afaf0 con 0x5576622cb2f0 2024-04-25T10:37:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.098+0000 7fdf18f01640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622f41b0 0x5576626977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.098+0000 7fdf18f01640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622f41b0 0x5576626977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34682/0 (socket says 172.21.15.43:34682) 2024-04-25T10:37:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.098+0000 7fdf18f01640 1 -- 172.21.15.43:0/130499308 learned_addr learned my addr 172.21.15.43:0/130499308 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.098+0000 7fdf137fe640 1 --2- 172.21.15.43:0/130499308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622d34d0 0x5576626a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.098+0000 7fdf13fff640 1 --2- 172.21.15.43:0/130499308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622cb2f0 0x5576626a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.098+0000 7fdf18f01640 1 -- 172.21.15.43:0/130499308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622cb2f0 msgr2=0x5576626a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.098+0000 7fdf18f01640 1 --2- 172.21.15.43:0/130499308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622cb2f0 0x5576626a4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.098+0000 7fdf18f01640 1 -- 172.21.15.43:0/130499308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622d34d0 msgr2=0x5576626a0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.098+0000 7fdf18f01640 1 --2- 172.21.15.43:0/130499308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622d34d0 0x5576626a0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.098+0000 7fdf18f01640 1 -- 172.21.15.43:0/130499308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557662696280 con 0x5576622f41b0 2024-04-25T10:37:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.099+0000 7fdf18f01640 1 --2- 172.21.15.43:0/130499308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622f41b0 0x5576626977a0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009ca0 tx=0x7fdf0c041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3999cdae5aee241 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.099+0000 7fdf09ffb640 1 -- 172.21.15.43:0/130499308 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c031830 con 0x5576622f41b0 2024-04-25T10:37:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.099+0000 7fdf09ffb640 1 -- 172.21.15.43:0/130499308 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf0c036040 con 0x5576622f41b0 2024-04-25T10:37:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.099+0000 7fdf09ffb640 1 -- 172.21.15.43:0/130499308 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c02fae0 con 0x5576622f41b0 2024-04-25T10:37:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.100+0000 7fdf21815740 1 -- 172.21.15.43:0/130499308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622f41b0 msgr2=0x5576626977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.100+0000 7fdf21815740 1 --2- 172.21.15.43:0/130499308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622f41b0 0x5576626977a0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009ca0 tx=0x7fdf0c041000 comp rx=0 tx=0).stop 2024-04-25T10:37:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.100+0000 7fdf21815740 1 -- 172.21.15.43:0/130499308 shutdown_connections 2024-04-25T10:37:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.100+0000 7fdf21815740 1 --2- 172.21.15.43:0/130499308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622d34d0 0x5576626a0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.100+0000 7fdf21815740 1 --2- 172.21.15.43:0/130499308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622f41b0 0x5576626977a0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.100+0000 7fdf21815740 1 --2- 172.21.15.43:0/130499308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622cb2f0 0x5576626a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.100+0000 7fdf21815740 1 -- 172.21.15.43:0/130499308 >> 172.21.15.43:0/130499308 conn(0x5576622e8300 msgr2=0x55766269d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.100+0000 7fdf21815740 1 -- 172.21.15.43:0/130499308 shutdown_connections 2024-04-25T10:37:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.100+0000 7fdf21815740 1 -- 172.21.15.43:0/130499308 wait complete. 2024-04-25T10:37:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.100+0000 7fdf21815740 1 -- start start 2024-04-25T10:37:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.100+0000 7fdf21815740 1 Processor -- start 2024-04-25T10:37:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.101+0000 7fdf21815740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622cb2f0 0x5576626aab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.101+0000 7fdf21815740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622d34d0 0x5576626ab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.101+0000 7fdf21815740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622f41b0 0x5576626a6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.101+0000 7fdf21815740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141b0160 con 0x5576622d34d0 2024-04-25T10:37:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.101+0000 7fdf21815740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141b15d0 con 0x5576622f41b0 2024-04-25T10:37:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.101+0000 7fdf21815740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141b0830 con 0x5576622cb2f0 2024-04-25T10:37:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.101+0000 7fdf21815740 10 client.0 _ll_register_callbacks cb 0x7fdefc0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.101+0000 7fdf137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622d34d0 0x5576626ab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.101+0000 7fdf137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622d34d0 0x5576626ab0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34688/0 (socket says 172.21.15.43:34688) 2024-04-25T10:37:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.101+0000 7fdf137fe640 1 -- 172.21.15.43:0/3008093976 learned_addr learned my addr 172.21.15.43:0/3008093976 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.101+0000 7fdf18f01640 1 --2- 172.21.15.43:0/3008093976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622f41b0 0x5576626a6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.101+0000 7fdf13fff640 1 --2- 172.21.15.43:0/3008093976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622cb2f0 0x5576626aab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.101+0000 7fdf137fe640 1 -- 172.21.15.43:0/3008093976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622cb2f0 msgr2=0x5576626aab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.101+0000 7fdf137fe640 1 --2- 172.21.15.43:0/3008093976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622cb2f0 0x5576626aab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.101+0000 7fdf137fe640 1 -- 172.21.15.43:0/3008093976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622f41b0 msgr2=0x5576626a6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.101+0000 7fdf137fe640 1 --2- 172.21.15.43:0/3008093976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622f41b0 0x5576626a6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.101+0000 7fdf137fe640 1 -- 172.21.15.43:0/3008093976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576626a67a0 con 0x5576622d34d0 2024-04-25T10:37:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.101+0000 7fdf137fe640 1 --2- 172.21.15.43:0/3008093976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622d34d0 0x5576626ab0c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fdf04005700 tx=0x7fdf04005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b68096168f49d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.101+0000 7fdef37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.101+0000 7fdef37fe640 1 -- 172.21.15.43:0/3008093976 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04005ea0 con 0x5576622d34d0 2024-04-25T10:37:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.102+0000 7fdef37fe640 1 -- 172.21.15.43:0/3008093976 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fdf0402a570 con 0x5576622d34d0 2024-04-25T10:37:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.102+0000 7fdf21815740 1 -- 172.21.15.43:0/3008093976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf0c036040 con 0x5576622d34d0 2024-04-25T10:37:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.102+0000 7fdef37fe640 1 -- 172.21.15.43:0/3008093976 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdf04038050 con 0x5576622d34d0 2024-04-25T10:37:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.102+0000 7fdf21815740 10 client.16839 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.102+0000 7fdf21815740 1 -- 172.21.15.43:0/3008093976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5576627184a0 con 0x5576622d34d0 2024-04-25T10:37:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.102+0000 7fdef37fe640 1 -- 172.21.15.43:0/3008093976 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf0c036040 con 0x5576622d34d0 2024-04-25T10:37:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.102+0000 7fdef37fe640 1 -- 172.21.15.43:0/3008093976 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdf04033b90 con 0x5576622d34d0 2024-04-25T10:37:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.102+0000 7fdf21815740 10 client.16839 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.102+0000 7fdf21815740 10 client.16839 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:37:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.102+0000 7fdf21815740 20 client.16839 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.102+0000 7fdf21815740 1 --2- 172.21.15.43:0/3008093976 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5576622881c0 0x55766271a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.102+0000 7fdf21815740 4 client.16839 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:37:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.102+0000 7fdf21815740 1 -- 172.21.15.43:0/3008093976 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557662601ee0 con 0x5576622881c0 2024-04-25T10:37:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.103+0000 7fdf13fff640 1 --2- 172.21.15.43:0/3008093976 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5576622881c0 0x55766271a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.103+0000 7fdf13fff640 1 --2- 172.21.15.43:0/3008093976 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5576622881c0 0x55766271a890 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fbe557ad2c471d5 server_cookie=b4643771ed5a232e in_seq=0 out_seq=0 2024-04-25T10:37:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.103+0000 7fdef37fe640 10 client.16839 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:20.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:19 smithi043 ceph-mon[29478]: pgmap v147: 97 pgs: 97 active+clean; 2.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 47 KiB/s wr, 16 op/s 2024-04-25T10:37:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdef37fe640 1 -- 172.21.15.43:0/3008093976 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557662601ee0 con 0x5576622881c0 2024-04-25T10:37:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdef37fe640 10 client.16839 handle_command_reply: tid=0 2024-04-25T10:37:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf21815740 1 client.16839 shutdown 2024-04-25T10:37:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf21815740 1 -- 172.21.15.43:0/3008093976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622d34d0 msgr2=0x5576626ab0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf21815740 1 --2- 172.21.15.43:0/3008093976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622d34d0 0x5576626ab0c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fdf04005700 tx=0x7fdf04005730 comp rx=0 tx=0).stop 2024-04-25T10:37:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf21815740 1 -- 172.21.15.43:0/3008093976 shutdown_connections 2024-04-25T10:37:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf21815740 1 --2- 172.21.15.43:0/3008093976 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5576622881c0 0x55766271a890 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf21815740 1 --2- 172.21.15.43:0/3008093976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576622f41b0 0x5576626a6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf21815740 1 --2- 172.21.15.43:0/3008093976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576622d34d0 0x5576626ab0c0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdef37fe640 0 client.16839 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf21815740 1 --2- 172.21.15.43:0/3008093976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576622cb2f0 0x5576626aab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf21815740 1 -- 172.21.15.43:0/3008093976 >> 172.21.15.43:0/3008093976 conn(0x5576622e8300 msgr2=0x55766269d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf21815740 1 -- 172.21.15.43:0/3008093976 shutdown_connections 2024-04-25T10:37:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf21815740 1 -- 172.21.15.43:0/3008093976 wait complete. 2024-04-25T10:37:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf21815740 20 client.16839 trim_cache size 0 max 16384 2024-04-25T10:37:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:37:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:37:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:37:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:37:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf1a964640 1 -- 172.21.15.43:0/1595891935 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdf000afc50 msgr2=0x7fdf000b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf1a964640 1 --2- 172.21.15.43:0/1595891935 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdf000afc50 0x7fdf000b2160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x557662690c60 tx=0x7fdf0c0095c0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf1a964640 1 -- 172.21.15.43:0/1595891935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf141028e0 msgr2=0x7fdf141a25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf1a964640 1 --2- 172.21.15.43:0/1595891935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf141028e0 0x7fdf141a25e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00d950 tx=0x7fdefc00de10 comp rx=0 tx=0).stop 2024-04-25T10:37:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf1a964640 1 -- 172.21.15.43:0/1595891935 shutdown_connections 2024-04-25T10:37:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf1a964640 1 --2- 172.21.15.43:0/1595891935 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdf000afc50 0x7fdf000b2160 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf1a964640 1 --2- 172.21.15.43:0/1595891935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf14110480 0x7fdf1419c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf1a964640 1 --2- 172.21.15.43:0/1595891935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1410db20 0x7fdf141a2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf1a964640 1 --2- 172.21.15.43:0/1595891935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf141028e0 0x7fdf141a25e0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf1a964640 1 -- 172.21.15.43:0/1595891935 >> 172.21.15.43:0/1595891935 conn(0x7fdf140781b0 msgr2=0x7fdf14101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf1a964640 1 -- 172.21.15.43:0/1595891935 shutdown_connections 2024-04-25T10:37:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:20.120+0000 7fdf1a964640 1 -- 172.21.15.43:0/1595891935 wait complete. 2024-04-25T10:37:20.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:19 smithi077 ceph-mon[32772]: pgmap v147: 97 pgs: 97 active+clean; 2.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 47 KiB/s wr, 16 op/s 2024-04-25T10:37:20.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:19 smithi164 ceph-mon[32713]: pgmap v147: 97 pgs: 97 active+clean; 2.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 47 KiB/s wr, 16 op/s 2024-04-25T10:37:22.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:21 smithi043 ceph-mon[29478]: pgmap v148: 97 pgs: 97 active+clean; 2.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-04-25T10:37:22.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:21 smithi077 ceph-mon[32772]: pgmap v148: 97 pgs: 97 active+clean; 2.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-04-25T10:37:22.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:21 smithi164 ceph-mon[32713]: pgmap v148: 97 pgs: 97 active+clean; 2.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-04-25T10:37:24.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:23 smithi043 ceph-mon[29478]: pgmap v149: 97 pgs: 97 active+clean; 2.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-04-25T10:37:24.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:23 smithi077 ceph-mon[32772]: pgmap v149: 97 pgs: 97 active+clean; 2.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-04-25T10:37:24.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:23 smithi164 ceph-mon[32713]: pgmap v149: 97 pgs: 97 active+clean; 2.1 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-04-25T10:37:26.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:25 smithi043 ceph-mon[29478]: pgmap v150: 97 pgs: 97 active+clean; 2.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 48 KiB/s wr, 16 op/s 2024-04-25T10:37:26.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:25 smithi077 ceph-mon[32772]: pgmap v150: 97 pgs: 97 active+clean; 2.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 48 KiB/s wr, 16 op/s 2024-04-25T10:37:26.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:25 smithi164 ceph-mon[32713]: pgmap v150: 97 pgs: 97 active+clean; 2.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 48 KiB/s wr, 16 op/s 2024-04-25T10:37:28.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:27 smithi043 ceph-mon[29478]: pgmap v151: 97 pgs: 97 active+clean; 2.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-25T10:37:28.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:27 smithi077 ceph-mon[32772]: pgmap v151: 97 pgs: 97 active+clean; 2.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-25T10:37:28.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:27 smithi164 ceph-mon[32713]: pgmap v151: 97 pgs: 97 active+clean; 2.3 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-25T10:37:29.391 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:37:29.391 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:37:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.517+0000 7f2f42042640 1 -- 172.21.15.43:0/3099700850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c10e540 msgr2=0x7f2f3c110960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.517+0000 7f2f42042640 1 --2- 172.21.15.43:0/3099700850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c10e540 0x7f2f3c110960 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009ca0 tx=0x7f2f3402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.517+0000 7f2f42042640 1 -- 172.21.15.43:0/3099700850 shutdown_connections 2024-04-25T10:37:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.517+0000 7f2f42042640 1 --2- 172.21.15.43:0/3099700850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f3c102040 0x7f2f3c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.517+0000 7f2f42042640 1 --2- 172.21.15.43:0/3099700850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c10e540 0x7f2f3c110960 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.517+0000 7f2f42042640 1 --2- 172.21.15.43:0/3099700850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f3c109b90 0x7f2f3c101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.517+0000 7f2f42042640 1 -- 172.21.15.43:0/3099700850 >> 172.21.15.43:0/3099700850 conn(0x7f2f3c0fd850 msgr2=0x7f2f3c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.517+0000 7f2f42042640 1 -- 172.21.15.43:0/3099700850 shutdown_connections 2024-04-25T10:37:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.517+0000 7f2f42042640 1 -- 172.21.15.43:0/3099700850 wait complete. 2024-04-25T10:37:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.518+0000 7f2f42042640 1 Processor -- start 2024-04-25T10:37:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.518+0000 7f2f42042640 1 -- start start 2024-04-25T10:37:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.519+0000 7f2f42042640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c102040 0x7f2f3c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.519+0000 7f2f42042640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f3c109b90 0x7f2f3c19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.519+0000 7f2f42042640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f3c10e540 0x7f2f3c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.519+0000 7f2f42042640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c113d00 con 0x7f2f3c102040 2024-04-25T10:37:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.519+0000 7f2f42042640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c113b80 con 0x7f2f3c109b90 2024-04-25T10:37:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.519+0000 7f2f42042640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c113e80 con 0x7f2f3c10e540 2024-04-25T10:37:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.519+0000 7f2f3b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c102040 0x7f2f3c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.519+0000 7f2f3b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c102040 0x7f2f3c19a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46284/0 (socket says 172.21.15.43:46284) 2024-04-25T10:37:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.519+0000 7f2f3b7fe640 1 -- 172.21.15.43:0/822020676 learned_addr learned my addr 172.21.15.43:0/822020676 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.519+0000 7f2f3affd640 1 --2- 172.21.15.43:0/822020676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f3c109b90 0x7f2f3c19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.519+0000 7f2f3bfff640 1 --2- 172.21.15.43:0/822020676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f3c10e540 0x7f2f3c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.519+0000 7f2f3b7fe640 1 -- 172.21.15.43:0/822020676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f3c10e540 msgr2=0x7f2f3c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.519+0000 7f2f3b7fe640 1 --2- 172.21.15.43:0/822020676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f3c10e540 0x7f2f3c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.519+0000 7f2f3b7fe640 1 -- 172.21.15.43:0/822020676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f3c109b90 msgr2=0x7f2f3c19ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.519+0000 7f2f3b7fe640 1 --2- 172.21.15.43:0/822020676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f3c109b90 0x7f2f3c19ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.519+0000 7f2f3b7fe640 1 -- 172.21.15.43:0/822020676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3c19f520 con 0x7f2f3c102040 2024-04-25T10:37:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.519+0000 7f2f3b7fe640 1 --2- 172.21.15.43:0/822020676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c102040 0x7f2f3c19a6c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c005d00 tx=0x7f2f2c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.519+0000 7f2f38ff9640 1 -- 172.21.15.43:0/822020676 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c017070 con 0x7f2f3c102040 2024-04-25T10:37:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.520+0000 7f2f38ff9640 1 -- 172.21.15.43:0/822020676 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f2c009c30 con 0x7f2f3c102040 2024-04-25T10:37:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.520+0000 7f2f42042640 1 -- 172.21.15.43:0/822020676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f3c19f810 con 0x7f2f3c102040 2024-04-25T10:37:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.520+0000 7f2f38ff9640 1 -- 172.21.15.43:0/822020676 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c00f620 con 0x7f2f3c102040 2024-04-25T10:37:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.520+0000 7f2f42042640 1 -- 172.21.15.43:0/822020676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f3c19fd10 con 0x7f2f3c102040 2024-04-25T10:37:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.522+0000 7f2f48fcc740 1 Processor -- start 2024-04-25T10:37:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.522+0000 7f2f38ff9640 1 -- 172.21.15.43:0/822020676 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f2f2c01a070 con 0x7f2f3c102040 2024-04-25T10:37:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.522+0000 7f2f48fcc740 1 -- start start 2024-04-25T10:37:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.523+0000 7f2f48fcc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b6e390 0x564400f0e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.523+0000 7f2f48fcc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b454d0 0x564400f0ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.523+0000 7f2f48fcc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 0x564400f1fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.523+0000 7f2f48fcc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564400e74d80 con 0x564400b4d6b0 2024-04-25T10:37:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.523+0000 7f2f48fcc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564400e2e8b0 con 0x564400b454d0 2024-04-25T10:37:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.523+0000 7f2f48fcc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564400e70d40 con 0x564400b6e390 2024-04-25T10:37:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.523+0000 7f2f3affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 0x564400f1fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.523+0000 7f2f3affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 0x564400f1fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46286/0 (socket says 172.21.15.43:46286) 2024-04-25T10:37:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.523+0000 7f2f3affd640 1 -- 172.21.15.43:0/2332220632 learned_addr learned my addr 172.21.15.43:0/2332220632 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.523+0000 7f2f3bfff640 1 --2- 172.21.15.43:0/2332220632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b6e390 0x564400f0e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.523+0000 7f2f3b7fe640 1 --2- 172.21.15.43:0/2332220632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b454d0 0x564400f0ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.523+0000 7f2f38ff9640 1 --2- 172.21.15.43:0/822020676 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2f240afc60 0x7f2f240b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.523+0000 7f2f3affd640 1 -- 172.21.15.43:0/2332220632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b6e390 msgr2=0x564400f0e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.523+0000 7f2f3affd640 1 --2- 172.21.15.43:0/2332220632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b6e390 0x564400f0e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.523+0000 7f2f3affd640 1 -- 172.21.15.43:0/2332220632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b454d0 msgr2=0x564400f0ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.523+0000 7f2f3affd640 1 --2- 172.21.15.43:0/2332220632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b454d0 0x564400f0ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.523+0000 7f2f3affd640 1 -- 172.21.15.43:0/2332220632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564400dffa40 con 0x564400b4d6b0 2024-04-25T10:37:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.523+0000 7f2f3bfff640 1 --2- 172.21.15.43:0/822020676 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2f240afc60 0x7f2f240b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.524+0000 7f2f38ff9640 1 -- 172.21.15.43:0/822020676 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f2f2c0deaa0 con 0x7f2f3c102040 2024-04-25T10:37:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.524+0000 7f2f3affd640 1 --2- 172.21.15.43:0/2332220632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 0x564400f1fd00 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009c60 tx=0x7f2f30011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863c5390ac356e81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.524+0000 7f2f21ffb640 1 -- 172.21.15.43:0/2332220632 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f30004230 con 0x564400b4d6b0 2024-04-25T10:37:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.524+0000 7f2f21ffb640 1 -- 172.21.15.43:0/2332220632 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f300043d0 con 0x564400b4d6b0 2024-04-25T10:37:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.524+0000 7f2f21ffb640 1 -- 172.21.15.43:0/2332220632 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f30004c80 con 0x564400b4d6b0 2024-04-25T10:37:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.525+0000 7f2f48fcc740 1 -- 172.21.15.43:0/2332220632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 msgr2=0x564400f1fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.525+0000 7f2f3bfff640 1 --2- 172.21.15.43:0/822020676 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2f240afc60 0x7f2f240b2170 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2f34005e30 tx=0x7f2f340095c0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.525+0000 7f2f48fcc740 1 --2- 172.21.15.43:0/2332220632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 0x564400f1fd00 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009c60 tx=0x7f2f30011810 comp rx=0 tx=0).stop 2024-04-25T10:37:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.525+0000 7f2f48fcc740 1 -- 172.21.15.43:0/2332220632 shutdown_connections 2024-04-25T10:37:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.525+0000 7f2f48fcc740 1 --2- 172.21.15.43:0/2332220632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b454d0 0x564400f0ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.525+0000 7f2f48fcc740 1 --2- 172.21.15.43:0/2332220632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 0x564400f1fd00 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.525+0000 7f2f48fcc740 1 --2- 172.21.15.43:0/2332220632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b6e390 0x564400f0e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.525+0000 7f2f48fcc740 1 -- 172.21.15.43:0/2332220632 >> 172.21.15.43:0/2332220632 conn(0x564400b624e0 msgr2=0x564400f0b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.525+0000 7f2f48fcc740 1 -- 172.21.15.43:0/2332220632 shutdown_connections 2024-04-25T10:37:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.525+0000 7f2f48fcc740 1 -- 172.21.15.43:0/2332220632 wait complete. 2024-04-25T10:37:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.525+0000 7f2f48fcc740 1 -- start start 2024-04-25T10:37:29.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.525+0000 7f2f48fcc740 1 Processor -- start 2024-04-25T10:37:29.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.526+0000 7f2f48fcc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b454d0 0x564400f1d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.526+0000 7f2f48fcc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 0x564400f1de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.526+0000 7f2f48fcc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b6e390 0x564400f29930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.526+0000 7f2f48fcc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564400f203f0 con 0x564400b4d6b0 2024-04-25T10:37:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.526+0000 7f2f48fcc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564400dfbd10 con 0x564400b454d0 2024-04-25T10:37:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.526+0000 7f2f48fcc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564400f20570 con 0x564400b6e390 2024-04-25T10:37:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.526+0000 7f2f48fcc740 10 client.0 _ll_register_callbacks cb 0x564400e28210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.526+0000 7f2f3affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 0x564400f1de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.526+0000 7f2f3affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 0x564400f1de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46296/0 (socket says 172.21.15.43:46296) 2024-04-25T10:37:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.526+0000 7f2f3affd640 1 -- 172.21.15.43:0/3829020836 learned_addr learned my addr 172.21.15.43:0/3829020836 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.526+0000 7f2f3b7fe640 1 --2- 172.21.15.43:0/3829020836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b454d0 0x564400f1d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.526+0000 7f2f3bfff640 1 --2- 172.21.15.43:0/3829020836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b6e390 0x564400f29930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.526+0000 7f2f3affd640 1 -- 172.21.15.43:0/3829020836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b6e390 msgr2=0x564400f29930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.526+0000 7f2f3affd640 1 --2- 172.21.15.43:0/3829020836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b6e390 0x564400f29930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.526+0000 7f2f3affd640 1 -- 172.21.15.43:0/3829020836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b454d0 msgr2=0x564400f1d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.526+0000 7f2f3affd640 1 --2- 172.21.15.43:0/3829020836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b454d0 0x564400f1d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.526+0000 7f2f3affd640 1 -- 172.21.15.43:0/3829020836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564400c507d0 con 0x564400b4d6b0 2024-04-25T10:37:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.526+0000 7f2f3affd640 1 --2- 172.21.15.43:0/3829020836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 0x564400f1de90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2f30019b90 tx=0x7f2f30019bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5bb8b01605a687a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.526+0000 7f2f1affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.527+0000 7f2f1affd640 1 -- 172.21.15.43:0/3829020836 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f30028020 con 0x564400b4d6b0 2024-04-25T10:37:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.527+0000 7f2f1affd640 1 -- 172.21.15.43:0/3829020836 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f2f30024620 con 0x564400b4d6b0 2024-04-25T10:37:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.527+0000 7f2f48fcc740 1 -- 172.21.15.43:0/3829020836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f300043d0 con 0x564400b4d6b0 2024-04-25T10:37:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.527+0000 7f2f1affd640 1 -- 172.21.15.43:0/3829020836 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2f30033050 con 0x564400b4d6b0 2024-04-25T10:37:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.527+0000 7f2f48fcc740 10 client.16851 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.527+0000 7f2f48fcc740 1 -- 172.21.15.43:0/3829020836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564400b863f0 con 0x564400b4d6b0 2024-04-25T10:37:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.527+0000 7f2f1affd640 1 -- 172.21.15.43:0/3829020836 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f300043d0 con 0x564400b4d6b0 2024-04-25T10:37:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.527+0000 7f2f1affd640 1 -- 172.21.15.43:0/3829020836 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2f30018070 con 0x564400b4d6b0 2024-04-25T10:37:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.527+0000 7f2f48fcc740 10 client.16851 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.527+0000 7f2f48fcc740 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:37:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.527+0000 7f2f48fcc740 20 client.16851 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:29.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.527+0000 7f2f48fcc740 1 --2- 172.21.15.43:0/3829020836 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564400b2cf60 0x564400f2f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.527+0000 7f2f48fcc740 4 client.16851 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:37:29.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.527+0000 7f2f48fcc740 1 -- 172.21.15.43:0/3829020836 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564400e82990 con 0x564400b2cf60 2024-04-25T10:37:29.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.528+0000 7f2f3b7fe640 1 --2- 172.21.15.43:0/3829020836 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564400b2cf60 0x564400f2f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.528+0000 7f2f1affd640 10 client.16851 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:29.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.528+0000 7f2f3b7fe640 1 --2- 172.21.15.43:0/3829020836 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564400b2cf60 0x564400f2f9b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eb62290f4578840 server_cookie=abde4f79f0f124c0 in_seq=0 out_seq=0 2024-04-25T10:37:29.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.529+0000 7f2f1affd640 1 -- 172.21.15.43:0/3829020836 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564400e82990 con 0x564400b2cf60 2024-04-25T10:37:29.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.529+0000 7f2f1affd640 10 client.16851 handle_command_reply: tid=0 2024-04-25T10:37:29.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.529+0000 7f2f48fcc740 1 client.16851 shutdown 2024-04-25T10:37:29.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.529+0000 7f2f48fcc740 1 -- 172.21.15.43:0/3829020836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 msgr2=0x564400f1de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:29.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.529+0000 7f2f48fcc740 1 --2- 172.21.15.43:0/3829020836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 0x564400f1de90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2f30019b90 tx=0x7f2f30019bc0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.529+0000 7f2f48fcc740 1 -- 172.21.15.43:0/3829020836 shutdown_connections 2024-04-25T10:37:29.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.529+0000 7f2f48fcc740 1 --2- 172.21.15.43:0/3829020836 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564400b2cf60 0x564400f2f9b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.529+0000 7f2f48fcc740 1 --2- 172.21.15.43:0/3829020836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b6e390 0x564400f29930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.529+0000 7f2f48fcc740 1 --2- 172.21.15.43:0/3829020836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 0x564400f1de90 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.529+0000 7f2f48fcc740 1 --2- 172.21.15.43:0/3829020836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b454d0 0x564400f1d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.529+0000 7f2f1affd640 0 client.16851 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:29.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.529+0000 7f2f48fcc740 1 -- 172.21.15.43:0/3829020836 >> 172.21.15.43:0/3829020836 conn(0x564400b624e0 msgr2=0x564400f106f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:29.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.529+0000 7f2f48fcc740 1 -- 172.21.15.43:0/3829020836 shutdown_connections 2024-04-25T10:37:29.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.529+0000 7f2f48fcc740 1 -- 172.21.15.43:0/3829020836 wait complete. 2024-04-25T10:37:29.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.529+0000 7f2f48fcc740 20 client.16851 trim_cache size 0 max 16384 2024-04-25T10:37:29.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.547+0000 7f2f48fcc740 1 Processor -- start 2024-04-25T10:37:29.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.547+0000 7f2f48fcc740 1 -- start start 2024-04-25T10:37:29.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.548+0000 7f2f48fcc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b454d0 0x564400f1c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.548+0000 7f2f48fcc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b4d6b0 0x564400f25880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.548+0000 7f2f48fcc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b6e390 0x564400f25dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.548+0000 7f2f48fcc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c1b0090 con 0x564400b454d0 2024-04-25T10:37:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.548+0000 7f2f48fcc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c1ad960 con 0x564400b6e390 2024-04-25T10:37:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.548+0000 7f2f48fcc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c1ab7b0 con 0x564400b4d6b0 2024-04-25T10:37:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.548+0000 7f2f3b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b454d0 0x564400f1c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.548+0000 7f2f3b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b454d0 0x564400f1c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46308/0 (socket says 172.21.15.43:46308) 2024-04-25T10:37:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.548+0000 7f2f3b7fe640 1 -- 172.21.15.43:0/2979827553 learned_addr learned my addr 172.21.15.43:0/2979827553 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.548+0000 7f2f3bfff640 1 --2- 172.21.15.43:0/2979827553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b6e390 0x564400f25dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.548+0000 7f2f3affd640 1 --2- 172.21.15.43:0/2979827553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b4d6b0 0x564400f25880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.548+0000 7f2f3b7fe640 1 -- 172.21.15.43:0/2979827553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b4d6b0 msgr2=0x564400f25880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.548+0000 7f2f3b7fe640 1 --2- 172.21.15.43:0/2979827553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b4d6b0 0x564400f25880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.548+0000 7f2f3b7fe640 1 -- 172.21.15.43:0/2979827553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b6e390 msgr2=0x564400f25dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.548+0000 7f2f3b7fe640 1 --2- 172.21.15.43:0/2979827553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b6e390 0x564400f25dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.548+0000 7f2f3b7fe640 1 -- 172.21.15.43:0/2979827553 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564400b460f0 con 0x564400b454d0 2024-04-25T10:37:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.548+0000 7f2f3b7fe640 1 --2- 172.21.15.43:0/2979827553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b454d0 0x564400f1c720 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00bfd0 tx=0x7f2f2c0c9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44dbb011d29b26c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.548+0000 7f2f217fa640 1 -- 172.21.15.43:0/2979827553 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c0e4030 con 0x564400b454d0 2024-04-25T10:37:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.548+0000 7f2f217fa640 1 -- 172.21.15.43:0/2979827553 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f2c0af410 con 0x564400b454d0 2024-04-25T10:37:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.548+0000 7f2f217fa640 1 -- 172.21.15.43:0/2979827553 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c021040 con 0x564400b454d0 2024-04-25T10:37:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.549+0000 7f2f48fcc740 1 -- 172.21.15.43:0/2979827553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b454d0 msgr2=0x564400f1c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.549+0000 7f2f48fcc740 1 --2- 172.21.15.43:0/2979827553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b454d0 0x564400f1c720 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00bfd0 tx=0x7f2f2c0c9040 comp rx=0 tx=0).stop 2024-04-25T10:37:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.549+0000 7f2f48fcc740 1 -- 172.21.15.43:0/2979827553 shutdown_connections 2024-04-25T10:37:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.549+0000 7f2f48fcc740 1 --2- 172.21.15.43:0/2979827553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b6e390 0x564400f25dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.549+0000 7f2f48fcc740 1 --2- 172.21.15.43:0/2979827553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b4d6b0 0x564400f25880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.549+0000 7f2f48fcc740 1 --2- 172.21.15.43:0/2979827553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b454d0 0x564400f1c720 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.549+0000 7f2f48fcc740 1 -- 172.21.15.43:0/2979827553 >> 172.21.15.43:0/2979827553 conn(0x564400b624e0 msgr2=0x564400f1c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.549+0000 7f2f48fcc740 1 -- 172.21.15.43:0/2979827553 shutdown_connections 2024-04-25T10:37:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.549+0000 7f2f48fcc740 1 -- 172.21.15.43:0/2979827553 wait complete. 2024-04-25T10:37:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.550+0000 7f2f48fcc740 1 -- start start 2024-04-25T10:37:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.550+0000 7f2f48fcc740 1 Processor -- start 2024-04-25T10:37:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.550+0000 7f2f48fcc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b454d0 0x564400f25a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.550+0000 7f2f48fcc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 0x564400f25fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.550+0000 7f2f48fcc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b6e390 0x564400f20fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.550+0000 7f2f48fcc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c1abe20 con 0x564400b4d6b0 2024-04-25T10:37:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.550+0000 7f2f48fcc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c1ad290 con 0x564400b454d0 2024-04-25T10:37:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.550+0000 7f2f48fcc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c1ac4f0 con 0x564400b6e390 2024-04-25T10:37:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.550+0000 7f2f48fcc740 10 client.0 _ll_register_callbacks cb 0x7f2f30005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.550+0000 7f2f3affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 0x564400f25fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.550+0000 7f2f3affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 0x564400f25fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46324/0 (socket says 172.21.15.43:46324) 2024-04-25T10:37:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.550+0000 7f2f3affd640 1 -- 172.21.15.43:0/2535141601 learned_addr learned my addr 172.21.15.43:0/2535141601 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.550+0000 7f2f3bfff640 1 --2- 172.21.15.43:0/2535141601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b6e390 0x564400f20fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.551+0000 7f2f3affd640 1 -- 172.21.15.43:0/2535141601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b6e390 msgr2=0x564400f20fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.550+0000 7f2f3b7fe640 1 --2- 172.21.15.43:0/2535141601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b454d0 0x564400f25a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.551+0000 7f2f3affd640 1 --2- 172.21.15.43:0/2535141601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b6e390 0x564400f20fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.551+0000 7f2f3affd640 1 -- 172.21.15.43:0/2535141601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b454d0 msgr2=0x564400f25a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.551+0000 7f2f3affd640 1 --2- 172.21.15.43:0/2535141601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b454d0 0x564400f25a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.551+0000 7f2f3affd640 1 -- 172.21.15.43:0/2535141601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564400e354b0 con 0x564400b4d6b0 2024-04-25T10:37:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.551+0000 7f2f3bfff640 1 --2- 172.21.15.43:0/2535141601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b6e390 0x564400f20fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:37:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.551+0000 7f2f3b7fe640 1 --2- 172.21.15.43:0/2535141601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b454d0 0x564400f25a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:37:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.551+0000 7f2f3affd640 1 --2- 172.21.15.43:0/2535141601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 0x564400f25fb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f2f30001eb0 tx=0x7f2f30009fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214ec4616f430c80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.551+0000 7f2f1affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.551+0000 7f2f1affd640 1 -- 172.21.15.43:0/2535141601 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3003c060 con 0x564400b4d6b0 2024-04-25T10:37:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.551+0000 7f2f48fcc740 1 -- 172.21.15.43:0/2535141601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564400e7c800 con 0x564400b4d6b0 2024-04-25T10:37:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.551+0000 7f2f1affd640 1 -- 172.21.15.43:0/2535141601 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f2f30024420 con 0x564400b4d6b0 2024-04-25T10:37:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.551+0000 7f2f1affd640 1 -- 172.21.15.43:0/2535141601 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2f30028070 con 0x564400b4d6b0 2024-04-25T10:37:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.551+0000 7f2f48fcc740 10 client.16857 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.551+0000 7f2f48fcc740 1 -- 172.21.15.43:0/2535141601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564400f97590 con 0x564400b4d6b0 2024-04-25T10:37:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.551+0000 7f2f1affd640 1 -- 172.21.15.43:0/2535141601 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564400e7c800 con 0x564400b4d6b0 2024-04-25T10:37:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.551+0000 7f2f1affd640 1 -- 172.21.15.43:0/2535141601 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2f30018040 con 0x564400b4d6b0 2024-04-25T10:37:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.552+0000 7f2f48fcc740 10 client.16857 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.552+0000 7f2f48fcc740 10 client.16857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:37:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.552+0000 7f2f48fcc740 20 client.16857 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.552+0000 7f2f48fcc740 1 --2- 172.21.15.43:0/2535141601 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564400b023a0 0x564400f99cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.552+0000 7f2f48fcc740 4 client.16857 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:37:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.552+0000 7f2f48fcc740 1 -- 172.21.15.43:0/2535141601 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564400df74a0 con 0x564400b023a0 2024-04-25T10:37:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.552+0000 7f2f3b7fe640 1 --2- 172.21.15.43:0/2535141601 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564400b023a0 0x564400f99cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.552+0000 7f2f3b7fe640 1 --2- 172.21.15.43:0/2535141601 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564400b023a0 0x564400f99cc0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a7d6da7575ec5be server_cookie=61c965565313f968 in_seq=0 out_seq=0 2024-04-25T10:37:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.552+0000 7f2f1affd640 10 client.16857 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.553+0000 7f2f1affd640 1 -- 172.21.15.43:0/2535141601 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564400df74a0 con 0x564400b023a0 2024-04-25T10:37:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.553+0000 7f2f1affd640 10 client.16857 handle_command_reply: tid=0 2024-04-25T10:37:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.553+0000 7f2f48fcc740 1 client.16857 shutdown 2024-04-25T10:37:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.553+0000 7f2f48fcc740 1 -- 172.21.15.43:0/2535141601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 msgr2=0x564400f25fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.553+0000 7f2f48fcc740 1 --2- 172.21.15.43:0/2535141601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 0x564400f25fb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f2f30001eb0 tx=0x7f2f30009fc0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.553+0000 7f2f48fcc740 1 -- 172.21.15.43:0/2535141601 shutdown_connections 2024-04-25T10:37:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.553+0000 7f2f48fcc740 1 --2- 172.21.15.43:0/2535141601 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564400b023a0 0x564400f99cc0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.553+0000 7f2f48fcc740 1 --2- 172.21.15.43:0/2535141601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564400b6e390 0x564400f20fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.553+0000 7f2f48fcc740 1 --2- 172.21.15.43:0/2535141601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564400b4d6b0 0x564400f25fb0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.553+0000 7f2f1affd640 0 client.16857 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.553+0000 7f2f48fcc740 1 --2- 172.21.15.43:0/2535141601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564400b454d0 0x564400f25a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.553+0000 7f2f48fcc740 1 -- 172.21.15.43:0/2535141601 >> 172.21.15.43:0/2535141601 conn(0x564400b624e0 msgr2=0x564400f1edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.553+0000 7f2f48fcc740 1 -- 172.21.15.43:0/2535141601 shutdown_connections 2024-04-25T10:37:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.553+0000 7f2f48fcc740 1 -- 172.21.15.43:0/2535141601 wait complete. 2024-04-25T10:37:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.553+0000 7f2f48fcc740 20 client.16857 trim_cache size 0 max 16384 2024-04-25T10:37:29.562 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:37:29.562 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:37:29.562 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:37:29.562 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:37:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.553+0000 7f2f42042640 1 -- 172.21.15.43:0/822020676 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2f240afc60 msgr2=0x7f2f240b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.553+0000 7f2f42042640 1 --2- 172.21.15.43:0/822020676 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2f240afc60 0x7f2f240b2170 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2f34005e30 tx=0x7f2f340095c0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.553+0000 7f2f42042640 1 -- 172.21.15.43:0/822020676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c102040 msgr2=0x7f2f3c19a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.554+0000 7f2f42042640 1 --2- 172.21.15.43:0/822020676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c102040 0x7f2f3c19a6c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c005d00 tx=0x7f2f2c00dde0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.554+0000 7f2f42042640 1 -- 172.21.15.43:0/822020676 shutdown_connections 2024-04-25T10:37:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.554+0000 7f2f42042640 1 --2- 172.21.15.43:0/822020676 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f2f240afc60 0x7f2f240b2170 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.554+0000 7f2f42042640 1 --2- 172.21.15.43:0/822020676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f3c109b90 0x7f2f3c19ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.554+0000 7f2f42042640 1 --2- 172.21.15.43:0/822020676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f3c10e540 0x7f2f3c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.554+0000 7f2f42042640 1 --2- 172.21.15.43:0/822020676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c102040 0x7f2f3c19a6c0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.554+0000 7f2f42042640 1 -- 172.21.15.43:0/822020676 >> 172.21.15.43:0/822020676 conn(0x7f2f3c0fd850 msgr2=0x7f2f3c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.554+0000 7f2f42042640 1 -- 172.21.15.43:0/822020676 shutdown_connections 2024-04-25T10:37:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.554+0000 7f2f42042640 1 -- 172.21.15.43:0/822020676 wait complete. 2024-04-25T10:37:29.567 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:37:29.567 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:37:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.679+0000 7f0e236e7640 1 -- 172.21.15.43:0/3999000668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e1c10d380 msgr2=0x7f0e1c10f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.679+0000 7f0e236e7640 1 --2- 172.21.15.43:0/3999000668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e1c10d380 0x7f0e1c10f7a0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009ca0 tx=0x7f0e1402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.679+0000 7f0e236e7640 1 -- 172.21.15.43:0/3999000668 shutdown_connections 2024-04-25T10:37:29.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.679+0000 7f0e236e7640 1 --2- 172.21.15.43:0/3999000668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e1c102af0 0x7f0e1c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.679+0000 7f0e236e7640 1 --2- 172.21.15.43:0/3999000668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e1c10d380 0x7f0e1c10f7a0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.679+0000 7f0e236e7640 1 --2- 172.21.15.43:0/3999000668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e1c108a20 0x7f0e1c1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.679+0000 7f0e236e7640 1 -- 172.21.15.43:0/3999000668 >> 172.21.15.43:0/3999000668 conn(0x7f0e1c078110 msgr2=0x7f0e1c078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:29.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.679+0000 7f0e236e7640 1 -- 172.21.15.43:0/3999000668 shutdown_connections 2024-04-25T10:37:29.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.679+0000 7f0e236e7640 1 -- 172.21.15.43:0/3999000668 wait complete. 2024-04-25T10:37:29.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.680+0000 7f0e236e7640 1 Processor -- start 2024-04-25T10:37:29.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.680+0000 7f0e236e7640 1 -- start start 2024-04-25T10:37:29.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.680+0000 7f0e236e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e1c102af0 0x7f0e1c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.680+0000 7f0e236e7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e1c108a20 0x7f0e1c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.680+0000 7f0e236e7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e1c10d380 0x7f0e1c19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.680+0000 7f0e236e7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c112c00 con 0x7f0e1c102af0 2024-04-25T10:37:29.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.680+0000 7f0e236e7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c112a80 con 0x7f0e1c10d380 2024-04-25T10:37:29.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.680+0000 7f0e236e7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c112d80 con 0x7f0e1c108a20 2024-04-25T10:37:29.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.680+0000 7f0e21483640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e1c102af0 0x7f0e1c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.680+0000 7f0e21483640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e1c102af0 0x7f0e1c19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46348/0 (socket says 172.21.15.43:46348) 2024-04-25T10:37:29.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.680+0000 7f0e21483640 1 -- 172.21.15.43:0/4011957456 learned_addr learned my addr 172.21.15.43:0/4011957456 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:29.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.681+0000 7f0e20c82640 1 --2- 172.21.15.43:0/4011957456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e1c108a20 0x7f0e1c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.681+0000 7f0e21c84640 1 --2- 172.21.15.43:0/4011957456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e1c10d380 0x7f0e1c19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.681+0000 7f0e21483640 1 -- 172.21.15.43:0/4011957456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e1c108a20 msgr2=0x7f0e1c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:29.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.681+0000 7f0e21483640 1 --2- 172.21.15.43:0/4011957456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e1c108a20 0x7f0e1c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.681+0000 7f0e21483640 1 -- 172.21.15.43:0/4011957456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e1c10d380 msgr2=0x7f0e1c19ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:29.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.681+0000 7f0e21483640 1 --2- 172.21.15.43:0/4011957456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e1c10d380 0x7f0e1c19ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.681+0000 7f0e21483640 1 -- 172.21.15.43:0/4011957456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c19f430 con 0x7f0e1c102af0 2024-04-25T10:37:29.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.681+0000 7f0e21483640 1 --2- 172.21.15.43:0/4011957456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e1c102af0 0x7f0e1c19a650 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800c9a0 tx=0x7f0e1800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:29.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.681+0000 7f0e127fc640 1 -- 172.21.15.43:0/4011957456 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18007bb0 con 0x7f0e1c102af0 2024-04-25T10:37:29.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.681+0000 7f0e127fc640 1 -- 172.21.15.43:0/4011957456 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e18007d50 con 0x7f0e1c102af0 2024-04-25T10:37:29.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.681+0000 7f0e236e7640 1 -- 172.21.15.43:0/4011957456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1c19f720 con 0x7f0e1c102af0 2024-04-25T10:37:29.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.681+0000 7f0e127fc640 1 -- 172.21.15.43:0/4011957456 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18018730 con 0x7f0e1c102af0 2024-04-25T10:37:29.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.681+0000 7f0e236e7640 1 -- 172.21.15.43:0/4011957456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1c19fc70 con 0x7f0e1c102af0 2024-04-25T10:37:29.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.682+0000 7f0e236e7640 1 -- 172.21.15.43:0/4011957456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e1c103cd0 con 0x7f0e1c102af0 2024-04-25T10:37:29.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.684+0000 7f0e127fc640 1 -- 172.21.15.43:0/4011957456 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f0e1800f990 con 0x7f0e1c102af0 2024-04-25T10:37:29.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.685+0000 7f0e127fc640 1 --2- 172.21.15.43:0/4011957456 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0e080afce0 0x7f0e080b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:29.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.685+0000 7f0e20c82640 1 --2- 172.21.15.43:0/4011957456 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0e080afce0 0x7f0e080b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:29.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.685+0000 7f0e127fc640 1 -- 172.21.15.43:0/4011957456 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f0e180df360 con 0x7f0e1c102af0 2024-04-25T10:37:29.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.686+0000 7f0e20c82640 1 --2- 172.21.15.43:0/4011957456 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0e080afce0 0x7f0e080b21f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c06a9e0 tx=0x7f0e0c005f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:29.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.689+0000 7f0e127fc640 1 -- 172.21.15.43:0/4011957456 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e18014030 con 0x7f0e1c102af0 2024-04-25T10:37:29.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.916+0000 7f0e236e7640 1 -- 172.21.15.43:0/4011957456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e1c06afa0 con 0x7f0e1c102af0 2024-04-25T10:37:29.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.916+0000 7f0e127fc640 1 -- 172.21.15.43:0/4011957456 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7f0e180a7c50 con 0x7f0e1c102af0 2024-04-25T10:37:29.918 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:37:29.919 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:37:29.919 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:37:29.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.919+0000 7f0e236e7640 1 -- 172.21.15.43:0/4011957456 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0e080afce0 msgr2=0x7f0e080b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:29.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.919+0000 7f0e236e7640 1 --2- 172.21.15.43:0/4011957456 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0e080afce0 0x7f0e080b21f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c06a9e0 tx=0x7f0e0c005f90 comp rx=0 tx=0).stop 2024-04-25T10:37:29.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.919+0000 7f0e236e7640 1 -- 172.21.15.43:0/4011957456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e1c102af0 msgr2=0x7f0e1c19a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:29.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.919+0000 7f0e236e7640 1 --2- 172.21.15.43:0/4011957456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e1c102af0 0x7f0e1c19a650 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800c9a0 tx=0x7f0e1800ce60 comp rx=0 tx=0).stop 2024-04-25T10:37:29.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.919+0000 7f0e236e7640 1 -- 172.21.15.43:0/4011957456 shutdown_connections 2024-04-25T10:37:29.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.919+0000 7f0e236e7640 1 --2- 172.21.15.43:0/4011957456 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0e080afce0 0x7f0e080b21f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.919+0000 7f0e236e7640 1 --2- 172.21.15.43:0/4011957456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e1c10d380 0x7f0e1c19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.919+0000 7f0e236e7640 1 --2- 172.21.15.43:0/4011957456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e1c108a20 0x7f0e1c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.919+0000 7f0e236e7640 1 --2- 172.21.15.43:0/4011957456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e1c102af0 0x7f0e1c19a650 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:29.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.919+0000 7f0e236e7640 1 -- 172.21.15.43:0/4011957456 >> 172.21.15.43:0/4011957456 conn(0x7f0e1c078110 msgr2=0x7f0e1c10eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:29.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.919+0000 7f0e236e7640 1 -- 172.21.15.43:0/4011957456 shutdown_connections 2024-04-25T10:37:29.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:29.919+0000 7f0e236e7640 1 -- 172.21.15.43:0/4011957456 wait complete. 2024-04-25T10:37:29.932 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:37:30.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.044+0000 7f6c822aa640 1 -- 172.21.15.43:0/530289250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c109fb0 msgr2=0x7f6c7c103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.044+0000 7f6c822aa640 1 --2- 172.21.15.43:0/530289250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c109fb0 0x7f6c7c103bf0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009900 tx=0x7f6c7002a410 comp rx=0 tx=0).stop 2024-04-25T10:37:30.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.044+0000 7f6c822aa640 1 -- 172.21.15.43:0/530289250 shutdown_connections 2024-04-25T10:37:30.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.044+0000 7f6c822aa640 1 --2- 172.21.15.43:0/530289250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c7c104260 0x7f6c7c110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.044+0000 7f6c822aa640 1 --2- 172.21.15.43:0/530289250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c109fb0 0x7f6c7c103bf0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.044+0000 7f6c822aa640 1 --2- 172.21.15.43:0/530289250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c7c109b10 0x7f6c7c1036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.044+0000 7f6c822aa640 1 -- 172.21.15.43:0/530289250 >> 172.21.15.43:0/530289250 conn(0x7f6c7c0ff230 msgr2=0x7f6c7c101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:30.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.044+0000 7f6c822aa640 1 -- 172.21.15.43:0/530289250 shutdown_connections 2024-04-25T10:37:30.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.044+0000 7f6c822aa640 1 -- 172.21.15.43:0/530289250 wait complete. 2024-04-25T10:37:30.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.045+0000 7f6c822aa640 1 Processor -- start 2024-04-25T10:37:30.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.045+0000 7f6c822aa640 1 -- start start 2024-04-25T10:37:30.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c822aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c104260 0x7f6c7c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c822aa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c7c109b10 0x7f6c7c19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c822aa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c7c109fb0 0x7f6c7c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c822aa640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c113d00 con 0x7f6c7c104260 2024-04-25T10:37:30.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c822aa640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c113b80 con 0x7f6c7c109b10 2024-04-25T10:37:30.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c822aa640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c113e80 con 0x7f6c7c109fb0 2024-04-25T10:37:30.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c7bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c104260 0x7f6c7c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c7bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c104260 0x7f6c7c19a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46368/0 (socket says 172.21.15.43:46368) 2024-04-25T10:37:30.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c7bfff640 1 -- 172.21.15.43:0/3870548563 learned_addr learned my addr 172.21.15.43:0/3870548563 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:30.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c80847640 1 --2- 172.21.15.43:0/3870548563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c7c109fb0 0x7f6c7c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c7b7fe640 1 --2- 172.21.15.43:0/3870548563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c7c109b10 0x7f6c7c19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c7bfff640 1 -- 172.21.15.43:0/3870548563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c7c109fb0 msgr2=0x7f6c7c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c7bfff640 1 --2- 172.21.15.43:0/3870548563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c7c109fb0 0x7f6c7c19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c7bfff640 1 -- 172.21.15.43:0/3870548563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c7c109b10 msgr2=0x7f6c7c19acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c7bfff640 1 --2- 172.21.15.43:0/3870548563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c7c109b10 0x7f6c7c19acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c7bfff640 1 -- 172.21.15.43:0/3870548563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7c19f5c0 con 0x7f6c7c104260 2024-04-25T10:37:30.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c80847640 1 --2- 172.21.15.43:0/3870548563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c7c109fb0 0x7f6c7c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:37:30.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c7bfff640 1 --2- 172.21.15.43:0/3870548563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c104260 0x7f6c7c19a760 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c005fd0 tx=0x7f6c6c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:30.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c797fa640 1 -- 172.21.15.43:0/3870548563 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c017070 con 0x7f6c7c104260 2024-04-25T10:37:30.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c822aa640 1 -- 172.21.15.43:0/3870548563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7c19f8b0 con 0x7f6c7c104260 2024-04-25T10:37:30.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c797fa640 1 -- 172.21.15.43:0/3870548563 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c6c009c30 con 0x7f6c7c104260 2024-04-25T10:37:30.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.046+0000 7f6c797fa640 1 -- 172.21.15.43:0/3870548563 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c003d30 con 0x7f6c7c104260 2024-04-25T10:37:30.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.047+0000 7f6c822aa640 1 -- 172.21.15.43:0/3870548563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c7c1a7230 con 0x7f6c7c104260 2024-04-25T10:37:30.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.050+0000 7f6c89233740 1 Processor -- start 2024-04-25T10:37:30.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.050+0000 7f6c89233740 1 -- start start 2024-04-25T10:37:30.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.050+0000 7f6c797fa640 1 -- 172.21.15.43:0/3870548563 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f6c6c003ed0 con 0x7f6c7c104260 2024-04-25T10:37:30.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.050+0000 7f6c89233740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb465390 0x5573fb806810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.050+0000 7f6c89233740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb43c4d0 0x5573fb806d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.050+0000 7f6c89233740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb4446b0 0x5573fb8136a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.050+0000 7f6c89233740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5573fb76bd80 con 0x5573fb465390 2024-04-25T10:37:30.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.050+0000 7f6c89233740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5573fb725a20 con 0x5573fb4446b0 2024-04-25T10:37:30.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.050+0000 7f6c89233740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5573fb767d40 con 0x5573fb43c4d0 2024-04-25T10:37:30.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.050+0000 7f6c80847640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb465390 0x5573fb806810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.050+0000 7f6c80847640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb465390 0x5573fb806810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46380/0 (socket says 172.21.15.43:46380) 2024-04-25T10:37:30.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.050+0000 7f6c80847640 1 -- 172.21.15.43:0/3436677568 learned_addr learned my addr 172.21.15.43:0/3436677568 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:30.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.050+0000 7f6c7b7fe640 1 --2- 172.21.15.43:0/3436677568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb4446b0 0x5573fb8136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.050+0000 7f6c7bfff640 1 --2- 172.21.15.43:0/3436677568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb43c4d0 0x5573fb806d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.050+0000 7f6c80847640 1 -- 172.21.15.43:0/3436677568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb43c4d0 msgr2=0x5573fb806d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.050+0000 7f6c80847640 1 --2- 172.21.15.43:0/3436677568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb43c4d0 0x5573fb806d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.050+0000 7f6c80847640 1 -- 172.21.15.43:0/3436677568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb4446b0 msgr2=0x5573fb8136a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.050+0000 7f6c80847640 1 --2- 172.21.15.43:0/3436677568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb4446b0 0x5573fb8136a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.051+0000 7f6c80847640 1 -- 172.21.15.43:0/3436677568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573fb730c30 con 0x5573fb465390 2024-04-25T10:37:30.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.051+0000 7f6c7bfff640 1 --2- 172.21.15.43:0/3436677568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb43c4d0 0x5573fb806d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:37:30.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.051+0000 7f6c797fa640 1 --2- 172.21.15.43:0/3870548563 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6c640afc60 0x7f6c640b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.051+0000 7f6c80847640 1 --2- 172.21.15.43:0/3436677568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb465390 0x5573fb806810 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009fb0 tx=0x7f6c74010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61df8fd4b13afd90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:30.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.051+0000 7f6c7bfff640 1 --2- 172.21.15.43:0/3870548563 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6c640afc60 0x7f6c640b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.051+0000 7f6c627fc640 1 -- 172.21.15.43:0/3436677568 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74016070 con 0x5573fb465390 2024-04-25T10:37:30.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.051+0000 7f6c627fc640 1 -- 172.21.15.43:0/3436677568 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c74012bc0 con 0x5573fb465390 2024-04-25T10:37:30.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.051+0000 7f6c797fa640 1 -- 172.21.15.43:0/3870548563 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6c6c0ded30 con 0x7f6c7c104260 2024-04-25T10:37:30.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.051+0000 7f6c627fc640 1 -- 172.21.15.43:0/3436677568 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7401bc50 con 0x5573fb465390 2024-04-25T10:37:30.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.052+0000 7f6c89233740 1 -- 172.21.15.43:0/3436677568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb465390 msgr2=0x5573fb806810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.052+0000 7f6c7bfff640 1 --2- 172.21.15.43:0/3870548563 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6c640afc60 0x7f6c640b2170 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c0dc560 tx=0x7f6c6c00bf60 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:30.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.052+0000 7f6c89233740 1 --2- 172.21.15.43:0/3436677568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb465390 0x5573fb806810 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009fb0 tx=0x7f6c74010800 comp rx=0 tx=0).stop 2024-04-25T10:37:30.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.052+0000 7f6c89233740 1 -- 172.21.15.43:0/3436677568 shutdown_connections 2024-04-25T10:37:30.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.052+0000 7f6c89233740 1 --2- 172.21.15.43:0/3436677568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb4446b0 0x5573fb8136a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.052+0000 7f6c89233740 1 --2- 172.21.15.43:0/3436677568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb43c4d0 0x5573fb806d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.052+0000 7f6c89233740 1 --2- 172.21.15.43:0/3436677568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb465390 0x5573fb806810 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.052+0000 7f6c89233740 1 -- 172.21.15.43:0/3436677568 >> 172.21.15.43:0/3436677568 conn(0x5573fb4594e0 msgr2=0x5573fb802fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:30.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.052+0000 7f6c89233740 1 -- 172.21.15.43:0/3436677568 shutdown_connections 2024-04-25T10:37:30.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.052+0000 7f6c89233740 1 -- 172.21.15.43:0/3436677568 wait complete. 2024-04-25T10:37:30.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.052+0000 7f6c89233740 1 -- start start 2024-04-25T10:37:30.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.053+0000 7f6c89233740 1 Processor -- start 2024-04-25T10:37:30.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.053+0000 7f6c89233740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb43c4d0 0x5573fb811260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.053+0000 7f6c89233740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb4446b0 0x5573fb8117a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.053+0000 7f6c89233740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb465390 0x5573fb81d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.053+0000 7f6c89233740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5573fb813d90 con 0x5573fb465390 2024-04-25T10:37:30.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.053+0000 7f6c89233740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5573fb6f2dc0 con 0x5573fb43c4d0 2024-04-25T10:37:30.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.053+0000 7f6c89233740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5573fb813f10 con 0x5573fb4446b0 2024-04-25T10:37:30.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.053+0000 7f6c89233740 10 client.0 _ll_register_callbacks cb 0x5573fb70e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:30.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.053+0000 7f6c80847640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb465390 0x5573fb81d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.053+0000 7f6c80847640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb465390 0x5573fb81d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46384/0 (socket says 172.21.15.43:46384) 2024-04-25T10:37:30.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.053+0000 7f6c80847640 1 -- 172.21.15.43:0/389619288 learned_addr learned my addr 172.21.15.43:0/389619288 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:30.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.053+0000 7f6c80847640 1 --2- 172.21.15.43:0/389619288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb43c4d0 0x5573fb811260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.053+0000 7f6c7b7fe640 1 --2- 172.21.15.43:0/389619288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb4446b0 0x5573fb8117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.053+0000 7f6c80847640 1 -- 172.21.15.43:0/389619288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb4446b0 msgr2=0x5573fb8117a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.053+0000 7f6c80847640 1 --2- 172.21.15.43:0/389619288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb4446b0 0x5573fb8117a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.054+0000 7f6c80847640 1 -- 172.21.15.43:0/389619288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb43c4d0 msgr2=0x5573fb811260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.054+0000 7f6c80847640 1 --2- 172.21.15.43:0/389619288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb43c4d0 0x5573fb811260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.054+0000 7f6c80847640 1 -- 172.21.15.43:0/389619288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573fb47d3f0 con 0x5573fb465390 2024-04-25T10:37:30.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.054+0000 7f6c80847640 1 --2- 172.21.15.43:0/389619288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb465390 0x5573fb81d210 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f6c74005fd0 tx=0x7f6c7401b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9d9b2f29fa4771 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:30.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.054+0000 7f6c5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:30.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.054+0000 7f6c5b7fe640 1 -- 172.21.15.43:0/389619288 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74016040 con 0x5573fb465390 2024-04-25T10:37:30.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.054+0000 7f6c5b7fe640 1 -- 172.21.15.43:0/389619288 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6c7401ba30 con 0x5573fb465390 2024-04-25T10:37:30.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.054+0000 7f6c89233740 1 -- 172.21.15.43:0/389619288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c74012bc0 con 0x5573fb465390 2024-04-25T10:37:30.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.054+0000 7f6c5b7fe640 1 -- 172.21.15.43:0/389619288 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6c74030050 con 0x5573fb465390 2024-04-25T10:37:30.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.054+0000 7f6c89233740 10 client.16875 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:30.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.054+0000 7f6c89233740 1 -- 172.21.15.43:0/389619288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5573fb5477d0 con 0x5573fb465390 2024-04-25T10:37:30.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.054+0000 7f6c5b7fe640 1 -- 172.21.15.43:0/389619288 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c74012bc0 con 0x5573fb465390 2024-04-25T10:37:30.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.054+0000 7f6c5b7fe640 1 -- 172.21.15.43:0/389619288 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6c74027070 con 0x5573fb465390 2024-04-25T10:37:30.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.054+0000 7f6c89233740 10 client.16875 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:30.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.054+0000 7f6c89233740 10 client.16875 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:37:30.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.054+0000 7f6c89233740 20 client.16875 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:30.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.054+0000 7f6c89233740 1 --2- 172.21.15.43:0/389619288 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x5573fb423f60 0x5573fb8234c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.054+0000 7f6c89233740 4 client.16875 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:37:30.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.055+0000 7f6c89233740 1 -- 172.21.15.43:0/389619288 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573fb779990 con 0x5573fb423f60 2024-04-25T10:37:30.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.055+0000 7f6c7b7fe640 1 --2- 172.21.15.43:0/389619288 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x5573fb423f60 0x5573fb8234c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.055+0000 7f6c7b7fe640 1 --2- 172.21.15.43:0/389619288 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x5573fb423f60 0x5573fb8234c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1b8ff448219e1462 server_cookie=81ee364ff4a8b24c in_seq=0 out_seq=0 2024-04-25T10:37:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.055+0000 7f6c5b7fe640 10 client.16875 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:37:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.056+0000 7f6c5b7fe640 1 -- 172.21.15.43:0/389619288 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5573fb779990 con 0x5573fb423f60 2024-04-25T10:37:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.056+0000 7f6c5b7fe640 10 client.16875 handle_command_reply: tid=0 2024-04-25T10:37:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.056+0000 7f6c89233740 1 client.16875 shutdown 2024-04-25T10:37:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.056+0000 7f6c89233740 1 -- 172.21.15.43:0/389619288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb465390 msgr2=0x5573fb81d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.056+0000 7f6c89233740 1 --2- 172.21.15.43:0/389619288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb465390 0x5573fb81d210 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f6c74005fd0 tx=0x7f6c7401b890 comp rx=0 tx=0).stop 2024-04-25T10:37:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.056+0000 7f6c89233740 1 -- 172.21.15.43:0/389619288 shutdown_connections 2024-04-25T10:37:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.056+0000 7f6c89233740 1 --2- 172.21.15.43:0/389619288 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x5573fb423f60 0x5573fb8234c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.056+0000 7f6c89233740 1 --2- 172.21.15.43:0/389619288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb465390 0x5573fb81d210 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.056+0000 7f6c89233740 1 --2- 172.21.15.43:0/389619288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb4446b0 0x5573fb8117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.056+0000 7f6c89233740 1 --2- 172.21.15.43:0/389619288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb43c4d0 0x5573fb811260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.056+0000 7f6c5b7fe640 0 client.16875 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:37:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.056+0000 7f6c89233740 1 -- 172.21.15.43:0/389619288 >> 172.21.15.43:0/389619288 conn(0x5573fb4594e0 msgr2=0x5573fb808530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.057+0000 7f6c89233740 1 -- 172.21.15.43:0/389619288 shutdown_connections 2024-04-25T10:37:30.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.057+0000 7f6c89233740 1 -- 172.21.15.43:0/389619288 wait complete. 2024-04-25T10:37:30.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.057+0000 7f6c89233740 20 client.16875 trim_cache size 0 max 16384 2024-04-25T10:37:30.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.070+0000 7f6c89233740 1 Processor -- start 2024-04-25T10:37:30.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.070+0000 7f6c89233740 1 -- start start 2024-04-25T10:37:30.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.070+0000 7f6c89233740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb423f60 0x5573fb810ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.070+0000 7f6c89233740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb43c4d0 0x5573fb81ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.070+0000 7f6c89233740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb4446b0 0x5573fb80b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.070+0000 7f6c89233740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c1b22d0 con 0x5573fb43c4d0 2024-04-25T10:37:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.070+0000 7f6c89233740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c1afba0 con 0x5573fb423f60 2024-04-25T10:37:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.070+0000 7f6c89233740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c1ad9f0 con 0x5573fb4446b0 2024-04-25T10:37:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.071+0000 7f6c7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb43c4d0 0x5573fb81ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.071+0000 7f6c7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb43c4d0 0x5573fb81ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46392/0 (socket says 172.21.15.43:46392) 2024-04-25T10:37:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.071+0000 7f6c7b7fe640 1 -- 172.21.15.43:0/282241207 learned_addr learned my addr 172.21.15.43:0/282241207 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.071+0000 7f6c80847640 1 --2- 172.21.15.43:0/282241207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb4446b0 0x5573fb80b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.071+0000 7f6c80847640 1 --2- 172.21.15.43:0/282241207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb423f60 0x5573fb810ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.071+0000 7f6c7b7fe640 1 -- 172.21.15.43:0/282241207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb4446b0 msgr2=0x5573fb80b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.071+0000 7f6c7b7fe640 1 --2- 172.21.15.43:0/282241207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb4446b0 0x5573fb80b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.071+0000 7f6c7b7fe640 1 -- 172.21.15.43:0/282241207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb423f60 msgr2=0x5573fb810ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.071+0000 7f6c7b7fe640 1 --2- 172.21.15.43:0/282241207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb423f60 0x5573fb810ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.071+0000 7f6c7b7fe640 1 -- 172.21.15.43:0/282241207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573fb806e70 con 0x5573fb43c4d0 2024-04-25T10:37:30.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.071+0000 7f6c80847640 1 --2- 172.21.15.43:0/282241207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb423f60 0x5573fb810ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:37:30.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.071+0000 7f6c7b7fe640 1 --2- 172.21.15.43:0/282241207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb43c4d0 0x5573fb81ce70 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f6c70031440 tx=0x7f6c70000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3bfa4ec66f3196 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:30.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.071+0000 7f6c61ffb640 1 -- 172.21.15.43:0/282241207 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70035040 con 0x5573fb43c4d0 2024-04-25T10:37:30.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.071+0000 7f6c61ffb640 1 -- 172.21.15.43:0/282241207 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c70047070 con 0x5573fb43c4d0 2024-04-25T10:37:30.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.071+0000 7f6c61ffb640 1 -- 172.21.15.43:0/282241207 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70004e10 con 0x5573fb43c4d0 2024-04-25T10:37:30.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.072+0000 7f6c89233740 1 -- 172.21.15.43:0/282241207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb43c4d0 msgr2=0x5573fb81ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.072+0000 7f6c89233740 1 --2- 172.21.15.43:0/282241207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb43c4d0 0x5573fb81ce70 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f6c70031440 tx=0x7f6c70000bc0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.072+0000 7f6c89233740 1 -- 172.21.15.43:0/282241207 shutdown_connections 2024-04-25T10:37:30.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.072+0000 7f6c89233740 1 --2- 172.21.15.43:0/282241207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb4446b0 0x5573fb80b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.072+0000 7f6c89233740 1 --2- 172.21.15.43:0/282241207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb43c4d0 0x5573fb81ce70 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.072+0000 7f6c89233740 1 --2- 172.21.15.43:0/282241207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb423f60 0x5573fb810ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.072+0000 7f6c89233740 1 -- 172.21.15.43:0/282241207 >> 172.21.15.43:0/282241207 conn(0x5573fb4594e0 msgr2=0x5573fb810590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:30.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.072+0000 7f6c89233740 1 -- 172.21.15.43:0/282241207 shutdown_connections 2024-04-25T10:37:30.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.072+0000 7f6c89233740 1 -- 172.21.15.43:0/282241207 wait complete. 2024-04-25T10:37:30.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.072+0000 7f6c89233740 1 -- start start 2024-04-25T10:37:30.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c89233740 1 Processor -- start 2024-04-25T10:37:30.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c89233740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb423f60 0x5573fb8141b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c89233740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb43c4d0 0x5573fb8146f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c89233740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb4446b0 0x5573fb814c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c89233740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c1ae060 con 0x5573fb43c4d0 2024-04-25T10:37:30.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c89233740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c1af4d0 con 0x5573fb423f60 2024-04-25T10:37:30.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c89233740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c1ae730 con 0x5573fb4446b0 2024-04-25T10:37:30.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c89233740 10 client.0 _ll_register_callbacks cb 0x7f6c74009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:30.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb43c4d0 0x5573fb8146f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb43c4d0 0x5573fb8146f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46402/0 (socket says 172.21.15.43:46402) 2024-04-25T10:37:30.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c7b7fe640 1 -- 172.21.15.43:0/3748942862 learned_addr learned my addr 172.21.15.43:0/3748942862 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:30.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c80847640 1 --2- 172.21.15.43:0/3748942862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb423f60 0x5573fb8141b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c80847640 1 --2- 172.21.15.43:0/3748942862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb4446b0 0x5573fb814c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c7b7fe640 1 -- 172.21.15.43:0/3748942862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb4446b0 msgr2=0x5573fb814c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c7b7fe640 1 --2- 172.21.15.43:0/3748942862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb4446b0 0x5573fb814c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c7b7fe640 1 -- 172.21.15.43:0/3748942862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb423f60 msgr2=0x5573fb8141b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c7b7fe640 1 --2- 172.21.15.43:0/3748942862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb423f60 0x5573fb8141b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c7b7fe640 1 -- 172.21.15.43:0/3748942862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573fb889800 con 0x5573fb43c4d0 2024-04-25T10:37:30.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c80847640 1 --2- 172.21.15.43:0/3748942862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb423f60 0x5573fb8141b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:37:30.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c7b7fe640 1 --2- 172.21.15.43:0/3748942862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb43c4d0 0x5573fb8146f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f6c70004850 tx=0x7f6c70004880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ba32a223bd2b0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:30.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.073+0000 7f6c5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:30.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.074+0000 7f6c5b7fe640 1 -- 172.21.15.43:0/3748942862 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7004f070 con 0x5573fb43c4d0 2024-04-25T10:37:30.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.074+0000 7f6c5b7fe640 1 -- 172.21.15.43:0/3748942862 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6c70042660 con 0x5573fb43c4d0 2024-04-25T10:37:30.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.074+0000 7f6c89233740 1 -- 172.21.15.43:0/3748942862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c70047070 con 0x5573fb43c4d0 2024-04-25T10:37:30.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.074+0000 7f6c5b7fe640 1 -- 172.21.15.43:0/3748942862 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6c70035040 con 0x5573fb43c4d0 2024-04-25T10:37:30.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.074+0000 7f6c89233740 10 client.16881 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:30.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.074+0000 7f6c89233740 1 -- 172.21.15.43:0/3748942862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5573fb88a9d0 con 0x5573fb43c4d0 2024-04-25T10:37:30.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.074+0000 7f6c5b7fe640 1 -- 172.21.15.43:0/3748942862 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c70047070 con 0x5573fb43c4d0 2024-04-25T10:37:30.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.074+0000 7f6c5b7fe640 1 -- 172.21.15.43:0/3748942862 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6c70042ce0 con 0x5573fb43c4d0 2024-04-25T10:37:30.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.074+0000 7f6c89233740 10 client.16881 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:30.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.074+0000 7f6c89233740 10 client.16881 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:37:30.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.074+0000 7f6c89233740 20 client.16881 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:30.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.074+0000 7f6c89233740 1 --2- 172.21.15.43:0/3748942862 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x5573fb465390 0x5573fb88d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.074+0000 7f6c89233740 4 client.16881 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:37:30.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.074+0000 7f6c89233740 1 -- 172.21.15.43:0/3748942862 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5573fb773800 con 0x5573fb465390 2024-04-25T10:37:30.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.075+0000 7f6c80847640 1 --2- 172.21.15.43:0/3748942862 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x5573fb465390 0x5573fb88d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.075+0000 7f6c80847640 1 --2- 172.21.15.43:0/3748942862 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x5573fb465390 0x5573fb88d150 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae7c45730778ac69 server_cookie=fae8ef7239e0a21b in_seq=0 out_seq=0 2024-04-25T10:37:30.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.075+0000 7f6c5b7fe640 10 client.16881 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:37:30.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.075+0000 7f6c5b7fe640 1 -- 172.21.15.43:0/3748942862 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5573fb773800 con 0x5573fb465390 2024-04-25T10:37:30.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.075+0000 7f6c5b7fe640 10 client.16881 handle_command_reply: tid=0 2024-04-25T10:37:30.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.075+0000 7f6c89233740 1 client.16881 shutdown 2024-04-25T10:37:30.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.075+0000 7f6c89233740 1 -- 172.21.15.43:0/3748942862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb43c4d0 msgr2=0x5573fb8146f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.075+0000 7f6c89233740 1 --2- 172.21.15.43:0/3748942862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb43c4d0 0x5573fb8146f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f6c70004850 tx=0x7f6c70004880 comp rx=0 tx=0).stop 2024-04-25T10:37:30.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.075+0000 7f6c89233740 1 -- 172.21.15.43:0/3748942862 shutdown_connections 2024-04-25T10:37:30.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.075+0000 7f6c89233740 1 --2- 172.21.15.43:0/3748942862 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x5573fb465390 0x5573fb88d150 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.075+0000 7f6c89233740 1 --2- 172.21.15.43:0/3748942862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573fb4446b0 0x5573fb814c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.075+0000 7f6c89233740 1 --2- 172.21.15.43:0/3748942862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573fb43c4d0 0x5573fb8146f0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.075+0000 7f6c89233740 1 --2- 172.21.15.43:0/3748942862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573fb423f60 0x5573fb8141b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.075+0000 7f6c5b7fe640 0 client.16881 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:37:30.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.075+0000 7f6c89233740 1 -- 172.21.15.43:0/3748942862 >> 172.21.15.43:0/3748942862 conn(0x5573fb4594e0 msgr2=0x5573fb802710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:30.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.075+0000 7f6c89233740 1 -- 172.21.15.43:0/3748942862 shutdown_connections 2024-04-25T10:37:30.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.075+0000 7f6c89233740 1 -- 172.21.15.43:0/3748942862 wait complete. 2024-04-25T10:37:30.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.076+0000 7f6c89233740 20 client.16881 trim_cache size 0 max 16384 2024-04-25T10:37:30.084 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:37:30.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.076+0000 7f6c822aa640 1 -- 172.21.15.43:0/3870548563 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6c640afc60 msgr2=0x7f6c640b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.076+0000 7f6c822aa640 1 --2- 172.21.15.43:0/3870548563 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6c640afc60 0x7f6c640b2170 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c0dc560 tx=0x7f6c6c00bf60 comp rx=0 tx=0).stop 2024-04-25T10:37:30.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.076+0000 7f6c822aa640 1 -- 172.21.15.43:0/3870548563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c104260 msgr2=0x7f6c7c19a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.076+0000 7f6c822aa640 1 --2- 172.21.15.43:0/3870548563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c104260 0x7f6c7c19a760 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c005fd0 tx=0x7f6c6c00dde0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.076+0000 7f6c822aa640 1 -- 172.21.15.43:0/3870548563 shutdown_connections 2024-04-25T10:37:30.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.076+0000 7f6c822aa640 1 --2- 172.21.15.43:0/3870548563 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6c640afc60 0x7f6c640b2170 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.076+0000 7f6c822aa640 1 --2- 172.21.15.43:0/3870548563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c7c109b10 0x7f6c7c19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.076+0000 7f6c822aa640 1 --2- 172.21.15.43:0/3870548563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c7c109fb0 0x7f6c7c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.076+0000 7f6c822aa640 1 --2- 172.21.15.43:0/3870548563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c104260 0x7f6c7c19a760 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.076+0000 7f6c822aa640 1 -- 172.21.15.43:0/3870548563 >> 172.21.15.43:0/3870548563 conn(0x7f6c7c0ff230 msgr2=0x7f6c7c10fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:30.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.076+0000 7f6c822aa640 1 -- 172.21.15.43:0/3870548563 shutdown_connections 2024-04-25T10:37:30.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.076+0000 7f6c822aa640 1 -- 172.21.15.43:0/3870548563 wait complete. 2024-04-25T10:37:30.088 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:29 smithi043 ceph-mon[29478]: pgmap v152: 97 pgs: 97 active+clean; 2.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-04-25T10:37:30.089 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:37:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.201+0000 7f93e96af640 1 -- 172.21.15.43:0/3440932762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93e4109ba0 msgr2=0x7f93e4101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.201+0000 7f93e96af640 1 --2- 172.21.15.43:0/3440932762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93e4109ba0 0x7f93e4101b10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009ca0 tx=0x7f93d402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.201+0000 7f93e96af640 1 -- 172.21.15.43:0/3440932762 shutdown_connections 2024-04-25T10:37:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.201+0000 7f93e96af640 1 --2- 172.21.15.43:0/3440932762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93e410e550 0x7f93e4110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.201+0000 7f93e96af640 1 --2- 172.21.15.43:0/3440932762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93e4102050 0x7f93e4102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.201+0000 7f93e96af640 1 --2- 172.21.15.43:0/3440932762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93e4109ba0 0x7f93e4101b10 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.201+0000 7f93e96af640 1 -- 172.21.15.43:0/3440932762 >> 172.21.15.43:0/3440932762 conn(0x7f93e4077880 msgr2=0x7f93e4077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.201+0000 7f93e96af640 1 -- 172.21.15.43:0/3440932762 shutdown_connections 2024-04-25T10:37:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.201+0000 7f93e96af640 1 -- 172.21.15.43:0/3440932762 wait complete. 2024-04-25T10:37:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.202+0000 7f93e96af640 1 Processor -- start 2024-04-25T10:37:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.202+0000 7f93e96af640 1 -- start start 2024-04-25T10:37:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.202+0000 7f93e96af640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93e4102050 0x7f93e4102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.202+0000 7f93e96af640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93e4109ba0 0x7f93e4103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.202+0000 7f93e96af640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93e410e550 0x7f93e4103770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.202+0000 7f93e96af640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93e4113d10 con 0x7f93e4102050 2024-04-25T10:37:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.202+0000 7f93e96af640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93e4113b90 con 0x7f93e4109ba0 2024-04-25T10:37:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.202+0000 7f93e96af640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f93e4113e90 con 0x7f93e410e550 2024-04-25T10:37:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.202+0000 7f93e2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93e4102050 0x7f93e4102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.202+0000 7f93e2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93e4102050 0x7f93e4102cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46418/0 (socket says 172.21.15.43:46418) 2024-04-25T10:37:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.202+0000 7f93e2ffd640 1 -- 172.21.15.43:0/4184105081 learned_addr learned my addr 172.21.15.43:0/4184105081 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.202+0000 7f93e27fc640 1 --2- 172.21.15.43:0/4184105081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93e4109ba0 0x7f93e4103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.202+0000 7f93e37fe640 1 --2- 172.21.15.43:0/4184105081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93e410e550 0x7f93e4103770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.203+0000 7f93e2ffd640 1 -- 172.21.15.43:0/4184105081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93e410e550 msgr2=0x7f93e4103770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.203+0000 7f93e2ffd640 1 --2- 172.21.15.43:0/4184105081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93e410e550 0x7f93e4103770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.203+0000 7f93e2ffd640 1 -- 172.21.15.43:0/4184105081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93e4109ba0 msgr2=0x7f93e4103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.203+0000 7f93e2ffd640 1 --2- 172.21.15.43:0/4184105081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93e4109ba0 0x7f93e4103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.203+0000 7f93e2ffd640 1 -- 172.21.15.43:0/4184105081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e41a8b10 con 0x7f93e4102050 2024-04-25T10:37:30.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.203+0000 7f93e37fe640 1 --2- 172.21.15.43:0/4184105081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93e410e550 0x7f93e4103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:37:30.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.203+0000 7f93e2ffd640 1 --2- 172.21.15.43:0/4184105081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93e4102050 0x7f93e4102cf0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f93d40319f0 tx=0x7f93d4031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:30.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.203+0000 7f93cbfff640 1 -- 172.21.15.43:0/4184105081 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4003bf0 con 0x7f93e4102050 2024-04-25T10:37:30.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.203+0000 7f93cbfff640 1 -- 172.21.15.43:0/4184105081 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93d4003d90 con 0x7f93e4102050 2024-04-25T10:37:30.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.203+0000 7f93e96af640 1 -- 172.21.15.43:0/4184105081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e41a8da0 con 0x7f93e4102050 2024-04-25T10:37:30.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.203+0000 7f93cbfff640 1 -- 172.21.15.43:0/4184105081 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d402fb70 con 0x7f93e4102050 2024-04-25T10:37:30.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.203+0000 7f93e96af640 1 -- 172.21.15.43:0/4184105081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e41a9260 con 0x7f93e4102050 2024-04-25T10:37:30.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.205+0000 7f93f0563740 1 Processor -- start 2024-04-25T10:37:30.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.205+0000 7f93f0563740 1 -- start start 2024-04-25T10:37:30.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.206+0000 7f93cbfff640 1 -- 172.21.15.43:0/4184105081 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f93d402fd10 con 0x7f93e4102050 2024-04-25T10:37:30.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.206+0000 7f93f0563740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a82d390 0x55945abce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.206+0000 7f93f0563740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a8044d0 0x55945abced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.206+0000 7f93f0563740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a80c6b0 0x55945abdb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.206+0000 7f93f0563740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55945ab33d80 con 0x55945a8044d0 2024-04-25T10:37:30.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.206+0000 7f93f0563740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55945aaeda20 con 0x55945a82d390 2024-04-25T10:37:30.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.206+0000 7f93f0563740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55945ab2fd40 con 0x55945a80c6b0 2024-04-25T10:37:30.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.206+0000 7f93e2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a8044d0 0x55945abced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.206+0000 7f93e2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a8044d0 0x55945abced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46420/0 (socket says 172.21.15.43:46420) 2024-04-25T10:37:30.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.206+0000 7f93e2ffd640 1 -- 172.21.15.43:0/3958072597 learned_addr learned my addr 172.21.15.43:0/3958072597 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:30.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.206+0000 7f93e37fe640 1 --2- 172.21.15.43:0/3958072597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a82d390 0x55945abce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.206+0000 7f93e27fc640 1 --2- 172.21.15.43:0/3958072597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a80c6b0 0x55945abdb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.206+0000 7f93e2ffd640 1 -- 172.21.15.43:0/3958072597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a80c6b0 msgr2=0x55945abdb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.206+0000 7f93e2ffd640 1 --2- 172.21.15.43:0/3958072597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a80c6b0 0x55945abdb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.206+0000 7f93e2ffd640 1 -- 172.21.15.43:0/3958072597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a82d390 msgr2=0x55945abce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.206+0000 7f93e2ffd640 1 --2- 172.21.15.43:0/3958072597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a82d390 0x55945abce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.206+0000 7f93e2ffd640 1 -- 172.21.15.43:0/3958072597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55945aaf8c30 con 0x55945a8044d0 2024-04-25T10:37:30.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.206+0000 7f93e27fc640 1 --2- 172.21.15.43:0/3958072597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a80c6b0 0x55945abdb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:37:30.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.206+0000 7f93cbfff640 1 --2- 172.21.15.43:0/4184105081 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f93cc0afc60 0x7f93cc0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.206+0000 7f93e2ffd640 1 --2- 172.21.15.43:0/3958072597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a8044d0 0x55945abced50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f93d40fdca0 tx=0x7f93d41065b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3642d27fe47cc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:30.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.207+0000 7f93e37fe640 1 --2- 172.21.15.43:0/4184105081 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f93cc0afc60 0x7f93cc0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.207+0000 7f93c97fa640 1 -- 172.21.15.43:0/3958072597 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d40c4640 con 0x55945a8044d0 2024-04-25T10:37:30.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.207+0000 7f93cbfff640 1 -- 172.21.15.43:0/4184105081 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f93d41004a0 con 0x7f93e4102050 2024-04-25T10:37:30.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.207+0000 7f93c97fa640 1 -- 172.21.15.43:0/3958072597 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93d40c8290 con 0x55945a8044d0 2024-04-25T10:37:30.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.207+0000 7f93c97fa640 1 -- 172.21.15.43:0/3958072597 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d40c7ef0 con 0x55945a8044d0 2024-04-25T10:37:30.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.208+0000 7f93f0563740 1 -- 172.21.15.43:0/3958072597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a8044d0 msgr2=0x55945abced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.208+0000 7f93e37fe640 1 --2- 172.21.15.43:0/4184105081 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f93cc0afc60 0x7f93cc0b2170 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55945abcb980 tx=0x7f93dc005dc0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:30.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.208+0000 7f93f0563740 1 --2- 172.21.15.43:0/3958072597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a8044d0 0x55945abced50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f93d40fdca0 tx=0x7f93d41065b0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.208+0000 7f93f0563740 1 -- 172.21.15.43:0/3958072597 shutdown_connections 2024-04-25T10:37:30.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.208+0000 7f93f0563740 1 --2- 172.21.15.43:0/3958072597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a80c6b0 0x55945abdb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.208+0000 7f93f0563740 1 --2- 172.21.15.43:0/3958072597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a8044d0 0x55945abced50 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.208+0000 7f93f0563740 1 --2- 172.21.15.43:0/3958072597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a82d390 0x55945abce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.208+0000 7f93f0563740 1 -- 172.21.15.43:0/3958072597 >> 172.21.15.43:0/3958072597 conn(0x55945a8214e0 msgr2=0x55945abcafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:30.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.208+0000 7f93f0563740 1 -- 172.21.15.43:0/3958072597 shutdown_connections 2024-04-25T10:37:30.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.208+0000 7f93f0563740 1 -- 172.21.15.43:0/3958072597 wait complete. 2024-04-25T10:37:30.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.208+0000 7f93f0563740 1 -- start start 2024-04-25T10:37:30.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.208+0000 7f93f0563740 1 Processor -- start 2024-04-25T10:37:30.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.208+0000 7f93f0563740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a8044d0 0x55945abd9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93f0563740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a80c6b0 0x55945abd9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93f0563740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a82d390 0x55945abe4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93f0563740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55945ab33d80 con 0x55945a8044d0 2024-04-25T10:37:30.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93f0563740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55945aabadc0 con 0x55945a82d390 2024-04-25T10:37:30.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93f0563740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55945abdbbe0 con 0x55945a80c6b0 2024-04-25T10:37:30.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93f0563740 10 client.0 _ll_register_callbacks cb 0x55945aad6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:30.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93e2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a8044d0 0x55945abd9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93e2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a8044d0 0x55945abd9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46436/0 (socket says 172.21.15.43:46436) 2024-04-25T10:37:30.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93e2ffd640 1 -- 172.21.15.43:0/3470029424 learned_addr learned my addr 172.21.15.43:0/3470029424 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:30.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93e37fe640 1 --2- 172.21.15.43:0/3470029424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a82d390 0x55945abe4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93e27fc640 1 --2- 172.21.15.43:0/3470029424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a80c6b0 0x55945abd9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93e2ffd640 1 -- 172.21.15.43:0/3470029424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a80c6b0 msgr2=0x55945abd9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93e2ffd640 1 --2- 172.21.15.43:0/3470029424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a80c6b0 0x55945abd9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93e2ffd640 1 -- 172.21.15.43:0/3470029424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a82d390 msgr2=0x55945abe4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93e2ffd640 1 --2- 172.21.15.43:0/3470029424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a82d390 0x55945abe4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93e2ffd640 1 -- 172.21.15.43:0/3470029424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55945a8453f0 con 0x55945a8044d0 2024-04-25T10:37:30.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93e37fe640 1 --2- 172.21.15.43:0/3470029424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a82d390 0x55945abe4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:37:30.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93e2ffd640 1 --2- 172.21.15.43:0/3470029424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a8044d0 0x55945abd9200 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f93d40cc040 tx=0x7f93d4007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30480516cf93ed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:30.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:30.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93c6ff5640 1 -- 172.21.15.43:0/3470029424 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d40c80d0 con 0x55945a8044d0 2024-04-25T10:37:30.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93c6ff5640 1 -- 172.21.15.43:0/3470029424 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f93d40b1110 con 0x55945a8044d0 2024-04-25T10:37:30.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.209+0000 7f93f0563740 1 -- 172.21.15.43:0/3470029424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93d40c8290 con 0x55945a8044d0 2024-04-25T10:37:30.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.210+0000 7f93c6ff5640 1 -- 172.21.15.43:0/3470029424 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f93d40c8550 con 0x55945a8044d0 2024-04-25T10:37:30.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.210+0000 7f93f0563740 10 client.16893 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:30.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.210+0000 7f93f0563740 1 -- 172.21.15.43:0/3470029424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55945a90f7d0 con 0x55945a8044d0 2024-04-25T10:37:30.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.210+0000 7f93c6ff5640 1 -- 172.21.15.43:0/3470029424 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93d40c8290 con 0x55945a8044d0 2024-04-25T10:37:30.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.210+0000 7f93c6ff5640 1 -- 172.21.15.43:0/3470029424 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f93d40fdea0 con 0x55945a8044d0 2024-04-25T10:37:30.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.210+0000 7f93f0563740 10 client.16893 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:30.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.210+0000 7f93f0563740 10 client.16893 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:37:30.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.210+0000 7f93f0563740 20 client.16893 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:30.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.210+0000 7f93f0563740 1 --2- 172.21.15.43:0/3470029424 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55945a7ebf60 0x55945abeb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.210+0000 7f93f0563740 4 client.16893 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:37:30.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.210+0000 7f93f0563740 1 -- 172.21.15.43:0/3470029424 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55945ab41990 con 0x55945a7ebf60 2024-04-25T10:37:30.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.210+0000 7f93e27fc640 1 --2- 172.21.15.43:0/3470029424 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55945a7ebf60 0x55945abeb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.211+0000 7f93e27fc640 1 --2- 172.21.15.43:0/3470029424 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55945a7ebf60 0x55945abeb280 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c0d3007dba227 server_cookie=82fd216fe54a0aa2 in_seq=0 out_seq=0 2024-04-25T10:37:30.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.211+0000 7f93c6ff5640 10 client.16893 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:37:30.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.212+0000 7f93c6ff5640 1 -- 172.21.15.43:0/3470029424 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55945ab41990 con 0x55945a7ebf60 2024-04-25T10:37:30.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.212+0000 7f93c6ff5640 10 client.16893 handle_command_reply: tid=0 2024-04-25T10:37:30.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.212+0000 7f93f0563740 1 client.16893 shutdown 2024-04-25T10:37:30.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.212+0000 7f93f0563740 1 -- 172.21.15.43:0/3470029424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a8044d0 msgr2=0x55945abd9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.212+0000 7f93f0563740 1 --2- 172.21.15.43:0/3470029424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a8044d0 0x55945abd9200 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f93d40cc040 tx=0x7f93d4007f90 comp rx=0 tx=0).stop 2024-04-25T10:37:30.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.212+0000 7f93f0563740 1 -- 172.21.15.43:0/3470029424 shutdown_connections 2024-04-25T10:37:30.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.212+0000 7f93f0563740 1 --2- 172.21.15.43:0/3470029424 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55945a7ebf60 0x55945abeb280 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.212+0000 7f93f0563740 1 --2- 172.21.15.43:0/3470029424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a82d390 0x55945abe4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.212+0000 7f93f0563740 1 --2- 172.21.15.43:0/3470029424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a80c6b0 0x55945abd9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.212+0000 7f93c6ff5640 0 client.16893 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:37:30.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.212+0000 7f93f0563740 1 --2- 172.21.15.43:0/3470029424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a8044d0 0x55945abd9200 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.212+0000 7f93f0563740 1 -- 172.21.15.43:0/3470029424 >> 172.21.15.43:0/3470029424 conn(0x55945a8214e0 msgr2=0x55945abd0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:30.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.212+0000 7f93f0563740 1 -- 172.21.15.43:0/3470029424 shutdown_connections 2024-04-25T10:37:30.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.212+0000 7f93f0563740 1 -- 172.21.15.43:0/3470029424 wait complete. 2024-04-25T10:37:30.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.212+0000 7f93f0563740 20 client.16893 trim_cache size 0 max 16384 2024-04-25T10:37:30.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.226+0000 7f93f0563740 1 Processor -- start 2024-04-25T10:37:30.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.226+0000 7f93f0563740 1 -- start start 2024-04-25T10:37:30.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.226+0000 7f93f0563740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a8044d0 0x55945abcb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.226+0000 7f93f0563740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a80c6b0 0x55945abe4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.226+0000 7f93f0563740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a82d390 0x55945abc9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.226+0000 7f93f0563740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93e41b4450 con 0x55945a82d390 2024-04-25T10:37:30.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.226+0000 7f93f0563740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93e41b1d20 con 0x55945a80c6b0 2024-04-25T10:37:30.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.226+0000 7f93f0563740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f93e41afb70 con 0x55945a8044d0 2024-04-25T10:37:30.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.226+0000 7f93e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a82d390 0x55945abc9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.226+0000 7f93e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a82d390 0x55945abc9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46446/0 (socket says 172.21.15.43:46446) 2024-04-25T10:37:30.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.226+0000 7f93e37fe640 1 -- 172.21.15.43:0/3990197311 learned_addr learned my addr 172.21.15.43:0/3990197311 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:30.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.226+0000 7f93e2ffd640 1 --2- 172.21.15.43:0/3990197311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a8044d0 0x55945abcb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.226+0000 7f93e27fc640 1 --2- 172.21.15.43:0/3990197311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a80c6b0 0x55945abe4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.226+0000 7f93e37fe640 1 -- 172.21.15.43:0/3990197311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a8044d0 msgr2=0x55945abcb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.226+0000 7f93e37fe640 1 --2- 172.21.15.43:0/3990197311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a8044d0 0x55945abcb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.226+0000 7f93e37fe640 1 -- 172.21.15.43:0/3990197311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a80c6b0 msgr2=0x55945abe4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.226+0000 7f93e37fe640 1 --2- 172.21.15.43:0/3990197311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a80c6b0 0x55945abe4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.226+0000 7f93e37fe640 1 -- 172.21.15.43:0/3990197311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55945abced70 con 0x55945a82d390 2024-04-25T10:37:30.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.227+0000 7f93e27fc640 1 --2- 172.21.15.43:0/3990197311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a80c6b0 0x55945abe4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:37:30.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.227+0000 7f93e37fe640 1 --2- 172.21.15.43:0/3990197311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a82d390 0x55945abc9d20 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f93dc011eb0 tx=0x7f93dc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d396cdbd5507869 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:30.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.227+0000 7f93c8ff9640 1 -- 172.21.15.43:0/3990197311 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc012e60 con 0x55945a82d390 2024-04-25T10:37:30.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.227+0000 7f93c8ff9640 1 -- 172.21.15.43:0/3990197311 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93dc023030 con 0x55945a82d390 2024-04-25T10:37:30.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.227+0000 7f93c8ff9640 1 -- 172.21.15.43:0/3990197311 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc01f680 con 0x55945a82d390 2024-04-25T10:37:30.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.228+0000 7f93f0563740 1 -- 172.21.15.43:0/3990197311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a82d390 msgr2=0x55945abc9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.228+0000 7f93f0563740 1 --2- 172.21.15.43:0/3990197311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a82d390 0x55945abc9d20 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f93dc011eb0 tx=0x7f93dc014000 comp rx=0 tx=0).stop 2024-04-25T10:37:30.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.228+0000 7f93f0563740 1 -- 172.21.15.43:0/3990197311 shutdown_connections 2024-04-25T10:37:30.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.228+0000 7f93f0563740 1 --2- 172.21.15.43:0/3990197311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a80c6b0 0x55945abe4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.228+0000 7f93f0563740 1 --2- 172.21.15.43:0/3990197311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a82d390 0x55945abc9d20 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.228+0000 7f93f0563740 1 --2- 172.21.15.43:0/3990197311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a8044d0 0x55945abcb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.228+0000 7f93f0563740 1 -- 172.21.15.43:0/3990197311 >> 172.21.15.43:0/3990197311 conn(0x55945a8214e0 msgr2=0x55945abc9610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:30.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.228+0000 7f93f0563740 1 -- 172.21.15.43:0/3990197311 shutdown_connections 2024-04-25T10:37:30.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.228+0000 7f93f0563740 1 -- 172.21.15.43:0/3990197311 wait complete. 2024-04-25T10:37:30.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.228+0000 7f93f0563740 1 -- start start 2024-04-25T10:37:30.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.228+0000 7f93f0563740 1 Processor -- start 2024-04-25T10:37:30.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.228+0000 7f93f0563740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a8044d0 0x55945abdc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.228+0000 7f93f0563740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a80c6b0 0x55945abdc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.228+0000 7f93f0563740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a82d390 0x55945ac4b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.228+0000 7f93f0563740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93e41b01e0 con 0x55945a82d390 2024-04-25T10:37:30.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.228+0000 7f93f0563740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93e41b1650 con 0x55945a80c6b0 2024-04-25T10:37:30.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.228+0000 7f93f0563740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f93e41b08b0 con 0x55945a8044d0 2024-04-25T10:37:30.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.228+0000 7f93f0563740 10 client.0 _ll_register_callbacks cb 0x7f93d40cc070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:30.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.228+0000 7f93e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a82d390 0x55945ac4b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.228+0000 7f93e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a82d390 0x55945ac4b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46452/0 (socket says 172.21.15.43:46452) 2024-04-25T10:37:30.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.229+0000 7f93e37fe640 1 -- 172.21.15.43:0/3672680321 learned_addr learned my addr 172.21.15.43:0/3672680321 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:30.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.229+0000 7f93e2ffd640 1 --2- 172.21.15.43:0/3672680321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a8044d0 0x55945abdc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.229+0000 7f93e27fc640 1 --2- 172.21.15.43:0/3672680321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a80c6b0 0x55945abdc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.229+0000 7f93e37fe640 1 -- 172.21.15.43:0/3672680321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a8044d0 msgr2=0x55945abdc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.229+0000 7f93e37fe640 1 --2- 172.21.15.43:0/3672680321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a8044d0 0x55945abdc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.229+0000 7f93e37fe640 1 -- 172.21.15.43:0/3672680321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a80c6b0 msgr2=0x55945abdc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.229+0000 7f93e37fe640 1 --2- 172.21.15.43:0/3672680321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a80c6b0 0x55945abdc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.229+0000 7f93e37fe640 1 -- 172.21.15.43:0/3672680321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55945ac4bb90 con 0x55945a82d390 2024-04-25T10:37:30.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.229+0000 7f93e27fc640 1 --2- 172.21.15.43:0/3672680321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a80c6b0 0x55945abdc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:37:30.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.229+0000 7f93e37fe640 1 --2- 172.21.15.43:0/3672680321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a82d390 0x55945ac4b620 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f93dc011fe0 tx=0x7f93dc005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3d5cd2da40416b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:30.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.229+0000 7f93c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:30.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.229+0000 7f93c6ff5640 1 -- 172.21.15.43:0/3672680321 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc004260 con 0x55945a82d390 2024-04-25T10:37:30.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.229+0000 7f93c6ff5640 1 -- 172.21.15.43:0/3672680321 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f93dc004400 con 0x55945a82d390 2024-04-25T10:37:30.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.229+0000 7f93f0563740 1 -- 172.21.15.43:0/3672680321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93dc023030 con 0x55945a82d390 2024-04-25T10:37:30.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.229+0000 7f93c6ff5640 1 -- 172.21.15.43:0/3672680321 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f93dc0035f0 con 0x55945a82d390 2024-04-25T10:37:30.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.229+0000 7f93f0563740 10 client.16899 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:30.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.229+0000 7f93f0563740 1 -- 172.21.15.43:0/3672680321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55945ac52e20 con 0x55945a82d390 2024-04-25T10:37:30.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.229+0000 7f93c6ff5640 1 -- 172.21.15.43:0/3672680321 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93dc023030 con 0x55945a82d390 2024-04-25T10:37:30.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.230+0000 7f93c6ff5640 1 -- 172.21.15.43:0/3672680321 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f93dc00f040 con 0x55945a82d390 2024-04-25T10:37:30.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.230+0000 7f93f0563740 10 client.16899 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:30.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.230+0000 7f93f0563740 10 client.16899 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:37:30.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.230+0000 7f93f0563740 20 client.16899 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:30.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.230+0000 7f93f0563740 1 --2- 172.21.15.43:0/3672680321 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55945a7c13a0 0x55945ac55510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.230+0000 7f93f0563740 4 client.16899 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:37:30.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.230+0000 7f93f0563740 1 -- 172.21.15.43:0/3672680321 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55945abcf0e0 con 0x55945a7c13a0 2024-04-25T10:37:30.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.230+0000 7f93e2ffd640 1 --2- 172.21.15.43:0/3672680321 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55945a7c13a0 0x55945ac55510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.230+0000 7f93c6ff5640 10 client.16899 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:37:30.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.230+0000 7f93e2ffd640 1 --2- 172.21.15.43:0/3672680321 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55945a7c13a0 0x55945ac55510 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1b12aafa6117c66 server_cookie=87396e557a11fb77 in_seq=0 out_seq=0 2024-04-25T10:37:30.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.231+0000 7f93c6ff5640 1 -- 172.21.15.43:0/3672680321 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55945abcf0e0 con 0x55945a7c13a0 2024-04-25T10:37:30.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.231+0000 7f93c6ff5640 10 client.16899 handle_command_reply: tid=0 2024-04-25T10:37:30.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.231+0000 7f93f0563740 1 client.16899 shutdown 2024-04-25T10:37:30.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.231+0000 7f93f0563740 1 -- 172.21.15.43:0/3672680321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a82d390 msgr2=0x55945ac4b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.231+0000 7f93f0563740 1 --2- 172.21.15.43:0/3672680321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a82d390 0x55945ac4b620 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f93dc011fe0 tx=0x7f93dc005760 comp rx=0 tx=0).stop 2024-04-25T10:37:30.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.231+0000 7f93f0563740 1 -- 172.21.15.43:0/3672680321 shutdown_connections 2024-04-25T10:37:30.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.231+0000 7f93f0563740 1 --2- 172.21.15.43:0/3672680321 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55945a7c13a0 0x55945ac55510 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.231+0000 7f93f0563740 1 --2- 172.21.15.43:0/3672680321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55945a80c6b0 0x55945abdc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.231+0000 7f93f0563740 1 --2- 172.21.15.43:0/3672680321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55945a82d390 0x55945ac4b620 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.231+0000 7f93c6ff5640 0 client.16899 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:37:30.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.231+0000 7f93f0563740 1 --2- 172.21.15.43:0/3672680321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55945a8044d0 0x55945abdc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.231+0000 7f93f0563740 1 -- 172.21.15.43:0/3672680321 >> 172.21.15.43:0/3672680321 conn(0x55945a8214e0 msgr2=0x55945abd7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:30.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.231+0000 7f93f0563740 1 -- 172.21.15.43:0/3672680321 shutdown_connections 2024-04-25T10:37:30.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.231+0000 7f93f0563740 1 -- 172.21.15.43:0/3672680321 wait complete. 2024-04-25T10:37:30.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.231+0000 7f93f0563740 20 client.16899 trim_cache size 0 max 16384 2024-04-25T10:37:30.241 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:37:30.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.231+0000 7f93e96af640 1 -- 172.21.15.43:0/4184105081 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f93cc0afc60 msgr2=0x7f93cc0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.231+0000 7f93e96af640 1 --2- 172.21.15.43:0/4184105081 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f93cc0afc60 0x7f93cc0b2170 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55945abcb980 tx=0x7f93dc005dc0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.231+0000 7f93e96af640 1 -- 172.21.15.43:0/4184105081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93e4102050 msgr2=0x7f93e4102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.231+0000 7f93e96af640 1 --2- 172.21.15.43:0/4184105081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93e4102050 0x7f93e4102cf0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f93d40319f0 tx=0x7f93d4031a20 comp rx=0 tx=0).stop 2024-04-25T10:37:30.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.231+0000 7f93e96af640 1 -- 172.21.15.43:0/4184105081 shutdown_connections 2024-04-25T10:37:30.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.232+0000 7f93e96af640 1 --2- 172.21.15.43:0/4184105081 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f93cc0afc60 0x7f93cc0b2170 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.232+0000 7f93e96af640 1 --2- 172.21.15.43:0/4184105081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93e4109ba0 0x7f93e4103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.232+0000 7f93e96af640 1 --2- 172.21.15.43:0/4184105081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93e410e550 0x7f93e4103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.232+0000 7f93e96af640 1 --2- 172.21.15.43:0/4184105081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93e4102050 0x7f93e4102cf0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.232+0000 7f93e96af640 1 -- 172.21.15.43:0/4184105081 >> 172.21.15.43:0/4184105081 conn(0x7f93e4077880 msgr2=0x7f93e410a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:30.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.232+0000 7f93e96af640 1 -- 172.21.15.43:0/4184105081 shutdown_connections 2024-04-25T10:37:30.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.232+0000 7f93e96af640 1 -- 172.21.15.43:0/4184105081 wait complete. 2024-04-25T10:37:30.244 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:37:30.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:29 smithi077 ceph-mon[32772]: pgmap v152: 97 pgs: 97 active+clean; 2.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-04-25T10:37:30.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:29 smithi164 ceph-mon[32713]: pgmap v152: 97 pgs: 97 active+clean; 2.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-04-25T10:37:30.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.356+0000 7fa7cadba640 1 -- 172.21.15.43:0/380018105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7c410b650 msgr2=0x7fa7c410da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.356+0000 7fa7cadba640 1 --2- 172.21.15.43:0/380018105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7c410b650 0x7fa7c410da70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009900 tx=0x7fa7b002a410 comp rx=0 tx=0).stop 2024-04-25T10:37:30.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.356+0000 7fa7cadba640 1 -- 172.21.15.43:0/380018105 shutdown_connections 2024-04-25T10:37:30.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.356+0000 7fa7cadba640 1 --2- 172.21.15.43:0/380018105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7c410e040 0x7fa7c4110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.356+0000 7fa7cadba640 1 --2- 172.21.15.43:0/380018105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7c410b650 0x7fa7c410da70 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.356+0000 7fa7cadba640 1 --2- 172.21.15.43:0/380018105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7c4104bf0 0x7fa7c4105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.356+0000 7fa7cadba640 1 -- 172.21.15.43:0/380018105 >> 172.21.15.43:0/380018105 conn(0x7fa7c4078100 msgr2=0x7fa7c4078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:30.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.356+0000 7fa7cadba640 1 -- 172.21.15.43:0/380018105 shutdown_connections 2024-04-25T10:37:30.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.356+0000 7fa7cadba640 1 -- 172.21.15.43:0/380018105 wait complete. 2024-04-25T10:37:30.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.357+0000 7fa7cadba640 1 Processor -- start 2024-04-25T10:37:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.357+0000 7fa7cadba640 1 -- start start 2024-04-25T10:37:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.357+0000 7fa7cadba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7c4104bf0 0x7fa7c419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.357+0000 7fa7cadba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7c410b650 0x7fa7c419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.357+0000 7fa7cadba640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7c410e040 0x7fa7c419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.357+0000 7fa7cadba640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c4113ca0 con 0x7fa7c410b650 2024-04-25T10:37:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.357+0000 7fa7cadba640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c4113b20 con 0x7fa7c4104bf0 2024-04-25T10:37:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.357+0000 7fa7cadba640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c4113e20 con 0x7fa7c410e040 2024-04-25T10:37:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.357+0000 7fa7c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7c410b650 0x7fa7c419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.358+0000 7fa7c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7c410b650 0x7fa7c419ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46458/0 (socket says 172.21.15.43:46458) 2024-04-25T10:37:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.358+0000 7fa7c3fff640 1 -- 172.21.15.43:0/2437048275 learned_addr learned my addr 172.21.15.43:0/2437048275 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.358+0000 7fa7c8b56640 1 --2- 172.21.15.43:0/2437048275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7c4104bf0 0x7fa7c419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.358+0000 7fa7c9357640 1 --2- 172.21.15.43:0/2437048275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7c410e040 0x7fa7c419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.358+0000 7fa7c3fff640 1 -- 172.21.15.43:0/2437048275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7c410e040 msgr2=0x7fa7c419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.358+0000 7fa7c3fff640 1 --2- 172.21.15.43:0/2437048275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7c410e040 0x7fa7c419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.358+0000 7fa7c3fff640 1 -- 172.21.15.43:0/2437048275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7c4104bf0 msgr2=0x7fa7c419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.358+0000 7fa7c3fff640 1 --2- 172.21.15.43:0/2437048275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7c4104bf0 0x7fa7c419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.358+0000 7fa7c3fff640 1 -- 172.21.15.43:0/2437048275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c419f580 con 0x7fa7c410b650 2024-04-25T10:37:30.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.358+0000 7fa7c9357640 1 --2- 172.21.15.43:0/2437048275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7c410e040 0x7fa7c419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:37:30.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.358+0000 7fa7c3fff640 1 --2- 172.21.15.43:0/2437048275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7c410b650 0x7fa7c419ac80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0005940 tx=0x7fa7b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:30.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.358+0000 7fa7c1ffb640 1 -- 172.21.15.43:0/2437048275 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b003f030 con 0x7fa7c410b650 2024-04-25T10:37:30.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.358+0000 7fa7cadba640 1 -- 172.21.15.43:0/2437048275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c419f810 con 0x7fa7c410b650 2024-04-25T10:37:30.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.358+0000 7fa7c1ffb640 1 -- 172.21.15.43:0/2437048275 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7b0003cb0 con 0x7fa7c410b650 2024-04-25T10:37:30.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.358+0000 7fa7c1ffb640 1 -- 172.21.15.43:0/2437048275 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0035070 con 0x7fa7c410b650 2024-04-25T10:37:30.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.359+0000 7fa7cadba640 1 -- 172.21.15.43:0/2437048275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c419fd50 con 0x7fa7c410b650 2024-04-25T10:37:30.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.361+0000 7fa7d1c6a740 1 Processor -- start 2024-04-25T10:37:30.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.361+0000 7fa7d1c6a740 1 -- start start 2024-04-25T10:37:30.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.361+0000 7fa7c1ffb640 1 -- 172.21.15.43:0/2437048275 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fa7b0042030 con 0x7fa7c410b650 2024-04-25T10:37:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.361+0000 7fa7d1c6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d075d8390 0x564d07979810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.361+0000 7fa7d1c6a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075af4d0 0x564d07979d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.361+0000 7fa7d1c6a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075b76b0 0x564d079866a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.361+0000 7fa7d1c6a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564d078ded80 con 0x564d075b76b0 2024-04-25T10:37:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.361+0000 7fa7d1c6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564d07898a20 con 0x564d075d8390 2024-04-25T10:37:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.361+0000 7fa7d1c6a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564d078dad40 con 0x564d075af4d0 2024-04-25T10:37:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.361+0000 7fa7c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075b76b0 0x564d079866a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.361+0000 7fa7c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075b76b0 0x564d079866a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46470/0 (socket says 172.21.15.43:46470) 2024-04-25T10:37:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.361+0000 7fa7c3fff640 1 -- 172.21.15.43:0/2580130677 learned_addr learned my addr 172.21.15.43:0/2580130677 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.362+0000 7fa7c9357640 1 --2- 172.21.15.43:0/2580130677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d075d8390 0x564d07979810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.362+0000 7fa7c8b56640 1 --2- 172.21.15.43:0/2580130677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075af4d0 0x564d07979d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.362+0000 7fa7c1ffb640 1 --2- 172.21.15.43:0/2437048275 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa7b40afc60 0x7fa7b40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.362+0000 7fa7c3fff640 1 -- 172.21.15.43:0/2580130677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075af4d0 msgr2=0x564d07979d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.362+0000 7fa7c3fff640 1 --2- 172.21.15.43:0/2580130677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075af4d0 0x564d07979d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.362+0000 7fa7c3fff640 1 -- 172.21.15.43:0/2580130677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d075d8390 msgr2=0x564d07979810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.362+0000 7fa7c3fff640 1 --2- 172.21.15.43:0/2580130677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d075d8390 0x564d07979810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.362+0000 7fa7c3fff640 1 -- 172.21.15.43:0/2580130677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d078a3c30 con 0x564d075b76b0 2024-04-25T10:37:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.362+0000 7fa7c9357640 1 --2- 172.21.15.43:0/2437048275 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa7b40afc60 0x7fa7b40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.362+0000 7fa7c1ffb640 1 -- 172.21.15.43:0/2437048275 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa7b00ff890 con 0x7fa7c410b650 2024-04-25T10:37:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.362+0000 7fa7c3fff640 1 --2- 172.21.15.43:0/2580130677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075b76b0 0x564d079866a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00fcf80 tx=0x7fa7b00c3920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58aa1e35aea3bf28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.362+0000 7fa7aeffd640 1 -- 172.21.15.43:0/2580130677 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0049030 con 0x564d075b76b0 2024-04-25T10:37:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.362+0000 7fa7aeffd640 1 -- 172.21.15.43:0/2580130677 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7b00d0200 con 0x564d075b76b0 2024-04-25T10:37:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.362+0000 7fa7aeffd640 1 -- 172.21.15.43:0/2580130677 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b00c71d0 con 0x564d075b76b0 2024-04-25T10:37:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.362+0000 7fa7c9357640 1 --2- 172.21.15.43:0/2437048275 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa7b40afc60 0x7fa7b40b2170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x564d07976980 tx=0x7fa7bc004a40 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.363+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2580130677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075b76b0 msgr2=0x564d079866a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.363+0000 7fa7d1c6a740 1 --2- 172.21.15.43:0/2580130677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075b76b0 0x564d079866a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00fcf80 tx=0x7fa7b00c3920 comp rx=0 tx=0).stop 2024-04-25T10:37:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.363+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2580130677 shutdown_connections 2024-04-25T10:37:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.363+0000 7fa7d1c6a740 1 --2- 172.21.15.43:0/2580130677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075b76b0 0x564d079866a0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.363+0000 7fa7d1c6a740 1 --2- 172.21.15.43:0/2580130677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075af4d0 0x564d07979d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.363+0000 7fa7d1c6a740 1 --2- 172.21.15.43:0/2580130677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d075d8390 0x564d07979810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.363+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2580130677 >> 172.21.15.43:0/2580130677 conn(0x564d075cc4e0 msgr2=0x564d07975fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:30.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.363+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2580130677 shutdown_connections 2024-04-25T10:37:30.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.363+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2580130677 wait complete. 2024-04-25T10:37:30.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.364+0000 7fa7d1c6a740 1 -- start start 2024-04-25T10:37:30.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.364+0000 7fa7d1c6a740 1 Processor -- start 2024-04-25T10:37:30.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.364+0000 7fa7d1c6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d075af4d0 0x564d07983f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.364+0000 7fa7d1c6a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075b76b0 0x564d079844a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.364+0000 7fa7d1c6a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075d8390 0x564d0798ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.364+0000 7fa7d1c6a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564d078ded80 con 0x564d075d8390 2024-04-25T10:37:30.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.364+0000 7fa7d1c6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564d07865dc0 con 0x564d075af4d0 2024-04-25T10:37:30.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.364+0000 7fa7d1c6a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564d07986be0 con 0x564d075b76b0 2024-04-25T10:37:30.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.364+0000 7fa7d1c6a740 10 client.0 _ll_register_callbacks cb 0x564d07881600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:30.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.364+0000 7fa7c9357640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075d8390 0x564d0798ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.365+0000 7fa7c3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075b76b0 0x564d079844a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.365+0000 7fa7c3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075b76b0 0x564d079844a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:36274/0 (socket says 172.21.15.43:36274) 2024-04-25T10:37:30.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.365+0000 7fa7c9357640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075d8390 0x564d0798ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46474/0 (socket says 172.21.15.43:46474) 2024-04-25T10:37:30.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.365+0000 7fa7c3fff640 1 -- 172.21.15.43:0/2053362323 learned_addr learned my addr 172.21.15.43:0/2053362323 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:30.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.365+0000 7fa7c8b56640 1 --2- 172.21.15.43:0/2053362323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d075af4d0 0x564d07983f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.365+0000 7fa7c9357640 1 -- 172.21.15.43:0/2053362323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075b76b0 msgr2=0x564d079844a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.365+0000 7fa7c9357640 1 --2- 172.21.15.43:0/2053362323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075b76b0 0x564d079844a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.365+0000 7fa7c9357640 1 -- 172.21.15.43:0/2053362323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d075af4d0 msgr2=0x564d07983f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.365+0000 7fa7c9357640 1 --2- 172.21.15.43:0/2053362323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d075af4d0 0x564d07983f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.365+0000 7fa7c9357640 1 -- 172.21.15.43:0/2053362323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d075f03f0 con 0x564d075d8390 2024-04-25T10:37:30.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.365+0000 7fa7c8b56640 1 --2- 172.21.15.43:0/2053362323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d075af4d0 0x564d07983f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:37:30.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.365+0000 7fa7c9357640 1 --2- 172.21.15.43:0/2053362323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075d8390 0x564d0798ffa0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc012e20 tx=0x7fa7bc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b34892e5f939b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:30.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.365+0000 7fa7a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.365+0000 7fa7a3fff640 1 -- 172.21.15.43:0/2053362323 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc027670 con 0x564d075d8390 2024-04-25T10:37:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.365+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2053362323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564d078ec990 con 0x564d075d8390 2024-04-25T10:37:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.365+0000 7fa7a3fff640 1 -- 172.21.15.43:0/2053362323 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa7bc01e450 con 0x564d075d8390 2024-04-25T10:37:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.365+0000 7fa7a3fff640 1 -- 172.21.15.43:0/2053362323 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa7bc027810 con 0x564d075d8390 2024-04-25T10:37:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.365+0000 7fa7d1c6a740 10 client.16911 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.365+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2053362323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564d076ba7d0 con 0x564d075d8390 2024-04-25T10:37:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.366+0000 7fa7a3fff640 1 -- 172.21.15.43:0/2053362323 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564d078ec990 con 0x564d075d8390 2024-04-25T10:37:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.366+0000 7fa7a3fff640 1 -- 172.21.15.43:0/2053362323 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa7bc01b6e0 con 0x564d075d8390 2024-04-25T10:37:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.366+0000 7fa7d1c6a740 10 client.16911 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.366+0000 7fa7d1c6a740 10 client.16911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:37:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.366+0000 7fa7d1c6a740 20 client.16911 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.366+0000 7fa7d1c6a740 1 --2- 172.21.15.43:0/2053362323 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564d07596f60 0x564d079963c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.366+0000 7fa7d1c6a740 4 client.16911 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:37:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.366+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2053362323 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d078e6800 con 0x564d07596f60 2024-04-25T10:37:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.366+0000 7fa7c8b56640 1 --2- 172.21.15.43:0/2053362323 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564d07596f60 0x564d079963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.367+0000 7fa7a3fff640 10 client.16911 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.367+0000 7fa7c8b56640 1 --2- 172.21.15.43:0/2053362323 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564d07596f60 0x564d079963c0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=710bff0e50fde1e5 server_cookie=bbc1338ca33cdb8a in_seq=0 out_seq=0 2024-04-25T10:37:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.367+0000 7fa7a3fff640 1 -- 172.21.15.43:0/2053362323 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564d078e6800 con 0x564d07596f60 2024-04-25T10:37:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.367+0000 7fa7a3fff640 10 client.16911 handle_command_reply: tid=0 2024-04-25T10:37:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.368+0000 7fa7d1c6a740 1 client.16911 shutdown 2024-04-25T10:37:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.368+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2053362323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075d8390 msgr2=0x564d0798ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.368+0000 7fa7d1c6a740 1 --2- 172.21.15.43:0/2053362323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075d8390 0x564d0798ffa0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc012e20 tx=0x7fa7bc004ab0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.368+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2053362323 shutdown_connections 2024-04-25T10:37:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.368+0000 7fa7d1c6a740 1 --2- 172.21.15.43:0/2053362323 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564d07596f60 0x564d079963c0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.368+0000 7fa7d1c6a740 1 --2- 172.21.15.43:0/2053362323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075d8390 0x564d0798ffa0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.368+0000 7fa7d1c6a740 1 --2- 172.21.15.43:0/2053362323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075b76b0 0x564d079844a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.368+0000 7fa7a3fff640 0 client.16911 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.368+0000 7fa7d1c6a740 1 --2- 172.21.15.43:0/2053362323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d075af4d0 0x564d07983f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.368+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2053362323 >> 172.21.15.43:0/2053362323 conn(0x564d075cc4e0 msgr2=0x564d0797b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.368+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2053362323 shutdown_connections 2024-04-25T10:37:30.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.368+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2053362323 wait complete. 2024-04-25T10:37:30.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.368+0000 7fa7d1c6a740 20 client.16911 trim_cache size 0 max 16384 2024-04-25T10:37:30.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.381+0000 7fa7d1c6a740 1 Processor -- start 2024-04-25T10:37:30.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.382+0000 7fa7d1c6a740 1 -- start start 2024-04-25T10:37:30.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.382+0000 7fa7d1c6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d07596f60 0x564d0798fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.382+0000 7fa7d1c6a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075af4d0 0x564d0799bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.382+0000 7fa7d1c6a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075b76b0 0x564d0799c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.382+0000 7fa7d1c6a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41b2280 con 0x564d075b76b0 2024-04-25T10:37:30.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.382+0000 7fa7d1c6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41afb50 con 0x564d07596f60 2024-04-25T10:37:30.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.382+0000 7fa7d1c6a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41ad9a0 con 0x564d075af4d0 2024-04-25T10:37:30.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.382+0000 7fa7c9357640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075b76b0 0x564d0799c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.382+0000 7fa7c9357640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075b76b0 0x564d0799c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46484/0 (socket says 172.21.15.43:46484) 2024-04-25T10:37:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.382+0000 7fa7c9357640 1 -- 172.21.15.43:0/2881704757 learned_addr learned my addr 172.21.15.43:0/2881704757 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.382+0000 7fa7c3fff640 1 --2- 172.21.15.43:0/2881704757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075af4d0 0x564d0799bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.382+0000 7fa7c8b56640 1 --2- 172.21.15.43:0/2881704757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d07596f60 0x564d0798fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.382+0000 7fa7c9357640 1 -- 172.21.15.43:0/2881704757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075af4d0 msgr2=0x564d0799bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.382+0000 7fa7c9357640 1 --2- 172.21.15.43:0/2881704757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075af4d0 0x564d0799bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.382+0000 7fa7c9357640 1 -- 172.21.15.43:0/2881704757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d07596f60 msgr2=0x564d0798fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.382+0000 7fa7c9357640 1 --2- 172.21.15.43:0/2881704757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d07596f60 0x564d0798fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.382+0000 7fa7c9357640 1 -- 172.21.15.43:0/2881704757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d079795a0 con 0x564d075b76b0 2024-04-25T10:37:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.382+0000 7fa7c8b56640 1 --2- 172.21.15.43:0/2881704757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d07596f60 0x564d0798fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:37:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.382+0000 7fa7c9357640 1 --2- 172.21.15.43:0/2881704757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075b76b0 0x564d0799c0c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00ab30 tx=0x7fa7bc00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab48f82bda18fd9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.383+0000 7fa7ae7fc640 1 -- 172.21.15.43:0/2881704757 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc027470 con 0x564d075b76b0 2024-04-25T10:37:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.383+0000 7fa7ae7fc640 1 -- 172.21.15.43:0/2881704757 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7bc022070 con 0x564d075b76b0 2024-04-25T10:37:30.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.383+0000 7fa7ae7fc640 1 -- 172.21.15.43:0/2881704757 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc00f4b0 con 0x564d075b76b0 2024-04-25T10:37:30.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.383+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2881704757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075b76b0 msgr2=0x564d0799c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.383+0000 7fa7d1c6a740 1 --2- 172.21.15.43:0/2881704757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075b76b0 0x564d0799c0c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00ab30 tx=0x7fa7bc00f3c0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.383+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2881704757 shutdown_connections 2024-04-25T10:37:30.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.383+0000 7fa7d1c6a740 1 --2- 172.21.15.43:0/2881704757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d075b76b0 0x564d0799c0c0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.383+0000 7fa7d1c6a740 1 --2- 172.21.15.43:0/2881704757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075af4d0 0x564d0799bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.383+0000 7fa7d1c6a740 1 --2- 172.21.15.43:0/2881704757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d07596f60 0x564d0798fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.383+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2881704757 >> 172.21.15.43:0/2881704757 conn(0x564d075cc4e0 msgr2=0x564d079860b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:30.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.383+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2881704757 shutdown_connections 2024-04-25T10:37:30.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.383+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2881704757 wait complete. 2024-04-25T10:37:30.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.384+0000 7fa7d1c6a740 1 -- start start 2024-04-25T10:37:30.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.384+0000 7fa7d1c6a740 1 Processor -- start 2024-04-25T10:37:30.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.384+0000 7fa7d1c6a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d07596f60 0x564d0798c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.384+0000 7fa7d1c6a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075af4d0 0x564d07987720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.384+0000 7fa7d1c6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d075b76b0 0x564d07987c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.384+0000 7fa7d1c6a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41ae010 con 0x564d07596f60 2024-04-25T10:37:30.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.384+0000 7fa7d1c6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41af480 con 0x564d075b76b0 2024-04-25T10:37:30.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.384+0000 7fa7d1c6a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41ae6e0 con 0x564d075af4d0 2024-04-25T10:37:30.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.384+0000 7fa7d1c6a740 10 client.0 _ll_register_callbacks cb 0x7fa7bc00aa60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:30.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.384+0000 7fa7c8b56640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d07596f60 0x564d0798c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.384+0000 7fa7c8b56640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d07596f60 0x564d0798c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46498/0 (socket says 172.21.15.43:46498) 2024-04-25T10:37:30.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.384+0000 7fa7c8b56640 1 -- 172.21.15.43:0/2302717499 learned_addr learned my addr 172.21.15.43:0/2302717499 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:30.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.384+0000 7fa7c3fff640 1 --2- 172.21.15.43:0/2302717499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075af4d0 0x564d07987720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.385+0000 7fa7c9357640 1 --2- 172.21.15.43:0/2302717499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d075b76b0 0x564d07987c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.385+0000 7fa7c8b56640 1 -- 172.21.15.43:0/2302717499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075af4d0 msgr2=0x564d07987720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.385+0000 7fa7c8b56640 1 --2- 172.21.15.43:0/2302717499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075af4d0 0x564d07987720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.385+0000 7fa7c8b56640 1 -- 172.21.15.43:0/2302717499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d075b76b0 msgr2=0x564d07987c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:30.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.385+0000 7fa7c8b56640 1 --2- 172.21.15.43:0/2302717499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d075b76b0 0x564d07987c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.385+0000 7fa7c8b56640 1 -- 172.21.15.43:0/2302717499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d07990aa0 con 0x564d07596f60 2024-04-25T10:37:30.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.385+0000 7fa7c9357640 1 --2- 172.21.15.43:0/2302717499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d075b76b0 0x564d07987c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:37:30.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.385+0000 7fa7c8b56640 1 --2- 172.21.15.43:0/2302717499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d07596f60 0x564d0798c3d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x564d07976f50 tx=0x7fa7b800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d2038148835e9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:30.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.385+0000 7fa7a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:30.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.385+0000 7fa7a3fff640 1 -- 172.21.15.43:0/2302717499 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b80200e0 con 0x564d07596f60 2024-04-25T10:37:30.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.385+0000 7fa7a3fff640 1 -- 172.21.15.43:0/2302717499 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa7b8020280 con 0x564d07596f60 2024-04-25T10:37:30.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.385+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2302717499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7bc022070 con 0x564d07596f60 2024-04-25T10:37:30.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.385+0000 7fa7a3fff640 1 -- 172.21.15.43:0/2302717499 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa7b8020480 con 0x564d07596f60 2024-04-25T10:37:30.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.385+0000 7fa7d1c6a740 10 client.16917 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:30.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.385+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2302717499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564d079fe360 con 0x564d07596f60 2024-04-25T10:37:30.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.385+0000 7fa7a3fff640 1 -- 172.21.15.43:0/2302717499 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7bc022070 con 0x564d07596f60 2024-04-25T10:37:30.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.385+0000 7fa7a3fff640 1 -- 172.21.15.43:0/2302717499 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa7b8032b10 con 0x564d07596f60 2024-04-25T10:37:30.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.386+0000 7fa7d1c6a740 10 client.16917 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:30.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.386+0000 7fa7d1c6a740 10 client.16917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:37:30.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.386+0000 7fa7d1c6a740 20 client.16917 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:30.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.386+0000 7fa7d1c6a740 1 --2- 172.21.15.43:0/2302717499 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564d075d8390 0x564d07a00b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:30.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.386+0000 7fa7d1c6a740 4 client.16917 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:37:30.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.386+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2302717499 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d07861610 con 0x564d075d8390 2024-04-25T10:37:30.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.386+0000 7fa7c3fff640 1 --2- 172.21.15.43:0/2302717499 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564d075d8390 0x564d07a00b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:30.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.386+0000 7fa7a3fff640 10 client.16917 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:30.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.386+0000 7fa7c3fff640 1 --2- 172.21.15.43:0/2302717499 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564d075d8390 0x564d07a00b50 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19c3f7c13a6dfe84 server_cookie=3fef745683c0eb08 in_seq=0 out_seq=0 2024-04-25T10:37:30.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.386+0000 7fa7a3fff640 1 -- 172.21.15.43:0/2302717499 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d07861610 con 0x564d075d8390 2024-04-25T10:37:30.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.386+0000 7fa7a3fff640 10 client.16917 handle_command_reply: tid=0 2024-04-25T10:37:30.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.386+0000 7fa7d1c6a740 1 client.16917 shutdown 2024-04-25T10:37:30.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.387+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2302717499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d07596f60 msgr2=0x564d0798c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.387+0000 7fa7d1c6a740 1 --2- 172.21.15.43:0/2302717499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d07596f60 0x564d0798c3d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x564d07976f50 tx=0x7fa7b800bed0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.387+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2302717499 shutdown_connections 2024-04-25T10:37:30.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.387+0000 7fa7d1c6a740 1 --2- 172.21.15.43:0/2302717499 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564d075d8390 0x564d07a00b50 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.387+0000 7fa7d1c6a740 1 --2- 172.21.15.43:0/2302717499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d075b76b0 0x564d07987c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.387+0000 7fa7a3fff640 0 client.16917 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:30.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.387+0000 7fa7d1c6a740 1 --2- 172.21.15.43:0/2302717499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d075af4d0 0x564d07987720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.387+0000 7fa7d1c6a740 1 --2- 172.21.15.43:0/2302717499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d07596f60 0x564d0798c3d0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.387+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2302717499 >> 172.21.15.43:0/2302717499 conn(0x564d075cc4e0 msgr2=0x564d079834a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:30.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.387+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2302717499 shutdown_connections 2024-04-25T10:37:30.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.387+0000 7fa7d1c6a740 1 -- 172.21.15.43:0/2302717499 wait complete. 2024-04-25T10:37:30.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.387+0000 7fa7d1c6a740 20 client.16917 trim_cache size 0 max 16384 2024-04-25T10:37:30.398 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:37:30.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.387+0000 7fa7cadba640 1 -- 172.21.15.43:0/2437048275 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa7b40afc60 msgr2=0x7fa7b40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.387+0000 7fa7cadba640 1 --2- 172.21.15.43:0/2437048275 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa7b40afc60 0x7fa7b40b2170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x564d07976980 tx=0x7fa7bc004a40 comp rx=0 tx=0).stop 2024-04-25T10:37:30.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.387+0000 7fa7cadba640 1 -- 172.21.15.43:0/2437048275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7c410b650 msgr2=0x7fa7c419ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:30.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.387+0000 7fa7cadba640 1 --2- 172.21.15.43:0/2437048275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7c410b650 0x7fa7c419ac80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0005940 tx=0x7fa7b00042f0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.387+0000 7fa7cadba640 1 -- 172.21.15.43:0/2437048275 shutdown_connections 2024-04-25T10:37:30.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.387+0000 7fa7cadba640 1 --2- 172.21.15.43:0/2437048275 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa7b40afc60 0x7fa7b40b2170 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.387+0000 7fa7cadba640 1 --2- 172.21.15.43:0/2437048275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7c410e040 0x7fa7c419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.387+0000 7fa7cadba640 1 --2- 172.21.15.43:0/2437048275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7c410b650 0x7fa7c419ac80 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.388+0000 7fa7cadba640 1 --2- 172.21.15.43:0/2437048275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7c4104bf0 0x7fa7c419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:30.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.388+0000 7fa7cadba640 1 -- 172.21.15.43:0/2437048275 >> 172.21.15.43:0/2437048275 conn(0x7fa7c4078100 msgr2=0x7fa7c4101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:30.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.388+0000 7fa7cadba640 1 -- 172.21.15.43:0/2437048275 shutdown_connections 2024-04-25T10:37:30.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:30.388+0000 7fa7cadba640 1 -- 172.21.15.43:0/2437048275 wait complete. 2024-04-25T10:37:31.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:30 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/4011957456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:37:31.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:30 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/4011957456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:37:31.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:30 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/4011957456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:37:31.401 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:37:31.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:37:31.402 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:37:31.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.531+0000 7f6ac26b1640 1 -- 172.21.15.43:0/2496253354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6abc108c50 msgr2=0x7f6abc102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:31.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.531+0000 7f6ac26b1640 1 --2- 172.21.15.43:0/2496253354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6abc108c50 0x7f6abc102d20 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f6aac009900 tx=0x7f6aac02a410 comp rx=0 tx=0).stop 2024-04-25T10:37:31.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.531+0000 7f6ac26b1640 1 -- 172.21.15.43:0/2496253354 shutdown_connections 2024-04-25T10:37:31.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.531+0000 7f6ac26b1640 1 --2- 172.21.15.43:0/2496253354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6abc1090f0 0x7f6abc103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.531+0000 7f6ac26b1640 1 --2- 172.21.15.43:0/2496253354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6abc1038d0 0x7f6abc10f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.531+0000 7f6ac26b1640 1 --2- 172.21.15.43:0/2496253354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6abc108c50 0x7f6abc102d20 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.531+0000 7f6ac26b1640 1 -- 172.21.15.43:0/2496253354 >> 172.21.15.43:0/2496253354 conn(0x7f6abc077920 msgr2=0x7f6abc077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:31.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.531+0000 7f6ac26b1640 1 -- 172.21.15.43:0/2496253354 shutdown_connections 2024-04-25T10:37:31.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.531+0000 7f6ac26b1640 1 -- 172.21.15.43:0/2496253354 wait complete. 2024-04-25T10:37:31.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.532+0000 7f6ac26b1640 1 Processor -- start 2024-04-25T10:37:31.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.532+0000 7f6ac26b1640 1 -- start start 2024-04-25T10:37:31.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6ac26b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6abc1038d0 0x7f6abc1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6ac26b1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6abc108c50 0x7f6abc1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6ac26b1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6abc1090f0 0x7f6abc19a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6ac26b1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc112eb0 con 0x7f6abc1038d0 2024-04-25T10:37:31.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6ac26b1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc112d30 con 0x7f6abc108c50 2024-04-25T10:37:31.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6ac26b1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc113030 con 0x7f6abc1090f0 2024-04-25T10:37:31.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6abbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6abc1038d0 0x7f6abc1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6abbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6abc1038d0 0x7f6abc1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46516/0 (socket says 172.21.15.43:46516) 2024-04-25T10:37:31.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6abbfff640 1 -- 172.21.15.43:0/1208574531 learned_addr learned my addr 172.21.15.43:0/1208574531 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:31.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6abb7fe640 1 --2- 172.21.15.43:0/1208574531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6abc108c50 0x7f6abc1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6ac0c4e640 1 --2- 172.21.15.43:0/1208574531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6abc1090f0 0x7f6abc19a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6abbfff640 1 -- 172.21.15.43:0/1208574531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6abc1090f0 msgr2=0x7f6abc19a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:31.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6abbfff640 1 --2- 172.21.15.43:0/1208574531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6abc1090f0 0x7f6abc19a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6abbfff640 1 -- 172.21.15.43:0/1208574531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6abc108c50 msgr2=0x7f6abc1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:31.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6abbfff640 1 --2- 172.21.15.43:0/1208574531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6abc108c50 0x7f6abc1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6abbfff640 1 -- 172.21.15.43:0/1208574531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6abc19af20 con 0x7f6abc1038d0 2024-04-25T10:37:31.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6ac0c4e640 1 --2- 172.21.15.43:0/1208574531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6abc1090f0 0x7f6abc19a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:37:31.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6abbfff640 1 --2- 172.21.15.43:0/1208574531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6abc1038d0 0x7f6abc1a0670 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f6aac02a930 tx=0x7f6aac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:31.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6ab97fa640 1 -- 172.21.15.43:0/1208574531 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac03f030 con 0x7f6abc1038d0 2024-04-25T10:37:31.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6ab97fa640 1 -- 172.21.15.43:0/1208574531 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aac003aa0 con 0x7f6abc1038d0 2024-04-25T10:37:31.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6ac26b1640 1 -- 172.21.15.43:0/1208574531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6abc19b1b0 con 0x7f6abc1038d0 2024-04-25T10:37:31.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.533+0000 7f6ab97fa640 1 -- 172.21.15.43:0/1208574531 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac035070 con 0x7f6abc1038d0 2024-04-25T10:37:31.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.534+0000 7f6ac26b1640 1 -- 172.21.15.43:0/1208574531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6abc1a74f0 con 0x7f6abc1038d0 2024-04-25T10:37:31.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.536+0000 7f6ac9639740 1 Processor -- start 2024-04-25T10:37:31.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.536+0000 7f6ab97fa640 1 -- 172.21.15.43:0/1208574531 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f6aac042030 con 0x7f6abc1038d0 2024-04-25T10:37:31.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.536+0000 7f6ac9639740 1 -- start start 2024-04-25T10:37:31.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.537+0000 7f6ac9639740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288ec5410 0x55e289265620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.537+0000 7f6ac9639740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288e9c550 0x55e289265b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.537+0000 7f6ac9639740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288ea4730 0x55e2892660a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.537+0000 7f6ac9639740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e2891cbb00 con 0x55e288e9c550 2024-04-25T10:37:31.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.537+0000 7f6ac9639740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e289167920 con 0x55e288ea4730 2024-04-25T10:37:31.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.537+0000 7f6ac9639740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e2891c7ac0 con 0x55e288ec5410 2024-04-25T10:37:31.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.537+0000 7f6abbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288e9c550 0x55e289265b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.537+0000 7f6abbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288e9c550 0x55e289265b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46524/0 (socket says 172.21.15.43:46524) 2024-04-25T10:37:31.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.537+0000 7f6abbfff640 1 -- 172.21.15.43:0/2154323594 learned_addr learned my addr 172.21.15.43:0/2154323594 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:31.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.537+0000 7f6ac0c4e640 1 --2- 172.21.15.43:0/2154323594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288ec5410 0x55e289265620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.537+0000 7f6abb7fe640 1 --2- 172.21.15.43:0/2154323594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288ea4730 0x55e2892660a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.537+0000 7f6ab97fa640 1 --2- 172.21.15.43:0/1208574531 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6aa40afce0 0x7f6aa40b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.537+0000 7f6abbfff640 1 -- 172.21.15.43:0/2154323594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288ec5410 msgr2=0x55e289265620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:31.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.537+0000 7f6abbfff640 1 --2- 172.21.15.43:0/2154323594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288ec5410 0x55e289265620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.537+0000 7f6abbfff640 1 -- 172.21.15.43:0/2154323594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288ea4730 msgr2=0x55e2892660a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:31.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.537+0000 7f6abbfff640 1 --2- 172.21.15.43:0/2154323594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288ea4730 0x55e2892660a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.537+0000 7f6abbfff640 1 -- 172.21.15.43:0/2154323594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e288fa57e0 con 0x55e288e9c550 2024-04-25T10:37:31.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.537+0000 7f6abb7fe640 1 --2- 172.21.15.43:0/2154323594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288ea4730 0x55e2892660a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:37:31.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.537+0000 7f6ac0c4e640 1 --2- 172.21.15.43:0/1208574531 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6aa40afce0 0x7f6aa40b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.537+0000 7f6ab97fa640 1 -- 172.21.15.43:0/1208574531 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6aac0ff740 con 0x7f6abc1038d0 2024-04-25T10:37:31.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.538+0000 7f6abbfff640 1 --2- 172.21.15.43:0/2154323594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288e9c550 0x55e289265b60 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f6aac0fce30 tx=0x7f6aac08dd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fd9c3eaa46853f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:31.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.538+0000 7f6aa27fc640 1 -- 172.21.15.43:0/2154323594 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac0c7260 con 0x55e288e9c550 2024-04-25T10:37:31.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.538+0000 7f6aa27fc640 1 -- 172.21.15.43:0/2154323594 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aac0c7420 con 0x55e288e9c550 2024-04-25T10:37:31.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.538+0000 7f6aa27fc640 1 -- 172.21.15.43:0/2154323594 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac0c7080 con 0x55e288e9c550 2024-04-25T10:37:31.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.538+0000 7f6ac0c4e640 1 --2- 172.21.15.43:0/1208574531 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6aa40afce0 0x7f6aa40b21f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55e289262790 tx=0x7f6ab4004b50 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:31.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.539+0000 7f6ac9639740 1 -- 172.21.15.43:0/2154323594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288e9c550 msgr2=0x55e289265b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:31.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.539+0000 7f6ac9639740 1 --2- 172.21.15.43:0/2154323594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288e9c550 0x55e289265b60 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f6aac0fce30 tx=0x7f6aac08dd30 comp rx=0 tx=0).stop 2024-04-25T10:37:31.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.539+0000 7f6ac9639740 1 -- 172.21.15.43:0/2154323594 shutdown_connections 2024-04-25T10:37:31.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.539+0000 7f6ac9639740 1 --2- 172.21.15.43:0/2154323594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288ea4730 0x55e2892660a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.539+0000 7f6ac9639740 1 --2- 172.21.15.43:0/2154323594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288e9c550 0x55e289265b60 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.539+0000 7f6ac9639740 1 --2- 172.21.15.43:0/2154323594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288ec5410 0x55e289265620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.539+0000 7f6ac9639740 1 -- 172.21.15.43:0/2154323594 >> 172.21.15.43:0/2154323594 conn(0x55e288eb9560 msgr2=0x55e289261dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:31.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.539+0000 7f6ac9639740 1 -- 172.21.15.43:0/2154323594 shutdown_connections 2024-04-25T10:37:31.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.539+0000 7f6ac9639740 1 -- 172.21.15.43:0/2154323594 wait complete. 2024-04-25T10:37:31.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.539+0000 7f6ac9639740 1 -- start start 2024-04-25T10:37:31.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.539+0000 7f6ac9639740 1 Processor -- start 2024-04-25T10:37:31.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6ac9639740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288e9c550 0x55e28926fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6ac9639740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288ea4730 0x55e289270140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6ac9639740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288ec5410 0x55e28927ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6ac9639740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e2891cbb00 con 0x55e288e9c550 2024-04-25T10:37:31.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6ac9639740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e2891655d0 con 0x55e288ec5410 2024-04-25T10:37:31.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6ac9639740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e289272730 con 0x55e288ea4730 2024-04-25T10:37:31.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6ac9639740 10 client.0 _ll_register_callbacks cb 0x55e289153080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:31.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6abbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288e9c550 0x55e28926fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6abbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288e9c550 0x55e28926fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46526/0 (socket says 172.21.15.43:46526) 2024-04-25T10:37:31.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6abbfff640 1 -- 172.21.15.43:0/1671295131 learned_addr learned my addr 172.21.15.43:0/1671295131 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:31.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6abb7fe640 1 --2- 172.21.15.43:0/1671295131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288ea4730 0x55e289270140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6ac0c4e640 1 --2- 172.21.15.43:0/1671295131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288ec5410 0x55e28927ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6abbfff640 1 -- 172.21.15.43:0/1671295131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288ea4730 msgr2=0x55e289270140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:31.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6abbfff640 1 --2- 172.21.15.43:0/1671295131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288ea4730 0x55e289270140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6abbfff640 1 -- 172.21.15.43:0/1671295131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288ec5410 msgr2=0x55e28927ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:31.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6abbfff640 1 --2- 172.21.15.43:0/1671295131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288ec5410 0x55e28927ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6abbfff640 1 -- 172.21.15.43:0/1671295131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e289005f70 con 0x55e288e9c550 2024-04-25T10:37:31.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6abb7fe640 1 --2- 172.21.15.43:0/1671295131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288ea4730 0x55e289270140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:37:31.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6ac0c4e640 1 --2- 172.21.15.43:0/1671295131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288ec5410 0x55e28927ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:37:31.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6abbfff640 1 --2- 172.21.15.43:0/1671295131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288e9c550 0x55e28926fc00 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6aac0e4760 tx=0x7f6aac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd02db0818eb7907 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:31.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.540+0000 7f6a9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:31.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.541+0000 7f6a9b7fe640 1 -- 172.21.15.43:0/1671295131 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac049070 con 0x55e288e9c550 2024-04-25T10:37:31.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.541+0000 7f6a9b7fe640 1 -- 172.21.15.43:0/1671295131 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6aac003c50 con 0x55e288e9c550 2024-04-25T10:37:31.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.541+0000 7f6ac9639740 1 -- 172.21.15.43:0/1671295131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aac0c7420 con 0x55e288e9c550 2024-04-25T10:37:31.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.541+0000 7f6a9b7fe640 1 -- 172.21.15.43:0/1671295131 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6aac0c7080 con 0x55e288e9c550 2024-04-25T10:37:31.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.541+0000 7f6ac9639740 10 client.16929 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:31.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.541+0000 7f6ac9639740 1 -- 172.21.15.43:0/1671295131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e288e9d170 con 0x55e288e9c550 2024-04-25T10:37:31.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.541+0000 7f6a9b7fe640 1 -- 172.21.15.43:0/1671295131 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aac0c7420 con 0x55e288e9c550 2024-04-25T10:37:31.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.541+0000 7f6a9b7fe640 1 -- 172.21.15.43:0/1671295131 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6aac0fd140 con 0x55e288e9c550 2024-04-25T10:37:31.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.541+0000 7f6ac9639740 10 client.16929 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:31.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.541+0000 7f6ac9639740 10 client.16929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:37:31.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.541+0000 7f6ac9639740 20 client.16929 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:31.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.541+0000 7f6ac9639740 1 --2- 172.21.15.43:0/1671295131 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e288e841f0 0x55e289281a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.541+0000 7f6ac9639740 4 client.16929 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:37:31.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.541+0000 7f6ac9639740 1 -- 172.21.15.43:0/1671295131 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2891d9710 con 0x55e288e841f0 2024-04-25T10:37:31.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.542+0000 7f6abb7fe640 1 --2- 172.21.15.43:0/1671295131 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e288e841f0 0x55e289281a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.542+0000 7f6a9b7fe640 10 client.16929 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:31.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.542+0000 7f6abb7fe640 1 --2- 172.21.15.43:0/1671295131 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e288e841f0 0x55e289281a70 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6205506f0db3b9e0 server_cookie=2aac07d15dbb2621 in_seq=0 out_seq=0 2024-04-25T10:37:31.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.543+0000 7f6a9b7fe640 1 -- 172.21.15.43:0/1671295131 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e2891d9710 con 0x55e288e841f0 2024-04-25T10:37:31.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.543+0000 7f6a9b7fe640 10 client.16929 handle_command_reply: tid=0 2024-04-25T10:37:31.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.543+0000 7f6ac9639740 1 client.16929 shutdown 2024-04-25T10:37:31.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.543+0000 7f6ac9639740 1 -- 172.21.15.43:0/1671295131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288e9c550 msgr2=0x55e28926fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:31.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.543+0000 7f6ac9639740 1 --2- 172.21.15.43:0/1671295131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288e9c550 0x55e28926fc00 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6aac0e4760 tx=0x7f6aac000bc0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.543+0000 7f6ac9639740 1 -- 172.21.15.43:0/1671295131 shutdown_connections 2024-04-25T10:37:31.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.543+0000 7f6ac9639740 1 --2- 172.21.15.43:0/1671295131 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e288e841f0 0x55e289281a70 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.543+0000 7f6ac9639740 1 --2- 172.21.15.43:0/1671295131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288ec5410 0x55e28927ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.543+0000 7f6ac9639740 1 --2- 172.21.15.43:0/1671295131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288ea4730 0x55e289270140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.543+0000 7f6ac9639740 1 --2- 172.21.15.43:0/1671295131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288e9c550 0x55e28926fc00 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.543+0000 7f6a9b7fe640 0 client.16929 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:31.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.543+0000 7f6ac9639740 1 -- 172.21.15.43:0/1671295131 >> 172.21.15.43:0/1671295131 conn(0x55e288eb9560 msgr2=0x55e289265450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:31.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.543+0000 7f6ac9639740 1 -- 172.21.15.43:0/1671295131 shutdown_connections 2024-04-25T10:37:31.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.543+0000 7f6ac9639740 1 -- 172.21.15.43:0/1671295131 wait complete. 2024-04-25T10:37:31.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.543+0000 7f6ac9639740 20 client.16929 trim_cache size 0 max 16384 2024-04-25T10:37:31.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.562+0000 7f6ac9639740 1 Processor -- start 2024-04-25T10:37:31.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.562+0000 7f6ac9639740 1 -- start start 2024-04-25T10:37:31.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.562+0000 7f6ac9639740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288e9c550 0x55e289260ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.562+0000 7f6ac9639740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288ea4730 0x55e28926e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.562+0000 7f6ac9639740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288ec5410 0x55e289287210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.562+0000 7f6ac9639740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc1b23b0 con 0x55e288ec5410 2024-04-25T10:37:31.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.562+0000 7f6ac9639740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc1afc80 con 0x55e288e9c550 2024-04-25T10:37:31.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.562+0000 7f6ac9639740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc1adad0 con 0x55e288ea4730 2024-04-25T10:37:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.562+0000 7f6ac0c4e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288ec5410 0x55e289287210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.562+0000 7f6ac0c4e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288ec5410 0x55e289287210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46534/0 (socket says 172.21.15.43:46534) 2024-04-25T10:37:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.562+0000 7f6ac0c4e640 1 -- 172.21.15.43:0/2500140523 learned_addr learned my addr 172.21.15.43:0/2500140523 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.562+0000 7f6abbfff640 1 --2- 172.21.15.43:0/2500140523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288e9c550 0x55e289260ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.562+0000 7f6abb7fe640 1 --2- 172.21.15.43:0/2500140523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288ea4730 0x55e28926e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.562+0000 7f6ac0c4e640 1 -- 172.21.15.43:0/2500140523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288ea4730 msgr2=0x55e28926e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.562+0000 7f6ac0c4e640 1 --2- 172.21.15.43:0/2500140523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288ea4730 0x55e28926e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.562+0000 7f6ac0c4e640 1 -- 172.21.15.43:0/2500140523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288e9c550 msgr2=0x55e289260ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.562+0000 7f6ac0c4e640 1 --2- 172.21.15.43:0/2500140523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288e9c550 0x55e289260ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.562+0000 7f6ac0c4e640 1 -- 172.21.15.43:0/2500140523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e289261fb0 con 0x55e288ec5410 2024-04-25T10:37:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.562+0000 7f6abb7fe640 1 --2- 172.21.15.43:0/2500140523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288ea4730 0x55e28926e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:37:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.563+0000 7f6ac0c4e640 1 --2- 172.21.15.43:0/2500140523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288ec5410 0x55e289287210 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4011780 tx=0x7f6ab4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f774947a2fc6f56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:31.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.563+0000 7f6aa1ffb640 1 -- 172.21.15.43:0/2500140523 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4022070 con 0x55e288ec5410 2024-04-25T10:37:31.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.563+0000 7f6aa1ffb640 1 -- 172.21.15.43:0/2500140523 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ab401e420 con 0x55e288ec5410 2024-04-25T10:37:31.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.563+0000 7f6aa1ffb640 1 -- 172.21.15.43:0/2500140523 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab401d610 con 0x55e288ec5410 2024-04-25T10:37:31.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.563+0000 7f6ac9639740 1 -- 172.21.15.43:0/2500140523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288ec5410 msgr2=0x55e289287210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:31.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.563+0000 7f6ac9639740 1 --2- 172.21.15.43:0/2500140523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288ec5410 0x55e289287210 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4011780 tx=0x7f6ab4011000 comp rx=0 tx=0).stop 2024-04-25T10:37:31.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.563+0000 7f6ac9639740 1 -- 172.21.15.43:0/2500140523 shutdown_connections 2024-04-25T10:37:31.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.563+0000 7f6ac9639740 1 --2- 172.21.15.43:0/2500140523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288ec5410 0x55e289287210 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.563+0000 7f6ac9639740 1 --2- 172.21.15.43:0/2500140523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288ea4730 0x55e28926e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.563+0000 7f6ac9639740 1 --2- 172.21.15.43:0/2500140523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288e9c550 0x55e289260ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.563+0000 7f6ac9639740 1 -- 172.21.15.43:0/2500140523 >> 172.21.15.43:0/2500140523 conn(0x55e288eb9560 msgr2=0x55e28926e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:31.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.564+0000 7f6ac9639740 1 -- 172.21.15.43:0/2500140523 shutdown_connections 2024-04-25T10:37:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.564+0000 7f6ac9639740 1 -- 172.21.15.43:0/2500140523 wait complete. 2024-04-25T10:37:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.564+0000 7f6ac9639740 1 -- start start 2024-04-25T10:37:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.564+0000 7f6ac9639740 1 Processor -- start 2024-04-25T10:37:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.564+0000 7f6ac9639740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288e9c550 0x55e289278050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.564+0000 7f6ac9639740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288ea4730 0x55e2892733a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.564+0000 7f6ac9639740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288ec5410 0x55e2892739f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.564+0000 7f6ac9639740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc1ae140 con 0x55e288ec5410 2024-04-25T10:37:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.564+0000 7f6ac9639740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc1af5b0 con 0x55e288ea4730 2024-04-25T10:37:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.564+0000 7f6ac9639740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc1ae810 con 0x55e288e9c550 2024-04-25T10:37:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.564+0000 7f6ac9639740 10 client.0 _ll_register_callbacks cb 0x7f6aac005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.565+0000 7f6ac0c4e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288ec5410 0x55e2892739f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.565+0000 7f6abb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288ea4730 0x55e2892733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.565+0000 7f6ac0c4e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288ec5410 0x55e2892739f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46548/0 (socket says 172.21.15.43:46548) 2024-04-25T10:37:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.565+0000 7f6ac0c4e640 1 -- 172.21.15.43:0/1939313539 learned_addr learned my addr 172.21.15.43:0/1939313539 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.565+0000 7f6abbfff640 1 --2- 172.21.15.43:0/1939313539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288e9c550 0x55e289278050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.565+0000 7f6ac0c4e640 1 -- 172.21.15.43:0/1939313539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288e9c550 msgr2=0x55e289278050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.565+0000 7f6ac0c4e640 1 --2- 172.21.15.43:0/1939313539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288e9c550 0x55e289278050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.565+0000 7f6ac0c4e640 1 -- 172.21.15.43:0/1939313539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288ea4730 msgr2=0x55e2892733a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.565+0000 7f6ac0c4e640 1 --2- 172.21.15.43:0/1939313539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288ea4730 0x55e2892733a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.565+0000 7f6ac0c4e640 1 -- 172.21.15.43:0/1939313539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e289274130 con 0x55e288ec5410 2024-04-25T10:37:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.565+0000 7f6abb7fe640 1 --2- 172.21.15.43:0/1939313539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288ea4730 0x55e2892733a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:37:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.565+0000 7f6ac0c4e640 1 --2- 172.21.15.43:0/1939313539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288ec5410 0x55e2892739f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6ab401ee20 tx=0x7f6ab401e800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3d7a97938353bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.565+0000 7f6a9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.565+0000 7f6a9b7fe640 1 -- 172.21.15.43:0/1939313539 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4022070 con 0x55e288ec5410 2024-04-25T10:37:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.565+0000 7f6a9b7fe640 1 -- 172.21.15.43:0/1939313539 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6ab4010b50 con 0x55e288ec5410 2024-04-25T10:37:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.565+0000 7f6ac9639740 1 -- 172.21.15.43:0/1939313539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ab401e420 con 0x55e288ec5410 2024-04-25T10:37:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.566+0000 7f6a9b7fe640 1 -- 172.21.15.43:0/1939313539 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6ab4010d50 con 0x55e288ec5410 2024-04-25T10:37:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.566+0000 7f6ac9639740 10 client.16935 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.566+0000 7f6ac9639740 1 -- 172.21.15.43:0/1939313539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e2892e9d80 con 0x55e288ec5410 2024-04-25T10:37:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.566+0000 7f6a9b7fe640 1 -- 172.21.15.43:0/1939313539 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ab401e420 con 0x55e288ec5410 2024-04-25T10:37:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.566+0000 7f6a9b7fe640 1 -- 172.21.15.43:0/1939313539 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6ab4027450 con 0x55e288ec5410 2024-04-25T10:37:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.566+0000 7f6ac9639740 10 client.16935 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.566+0000 7f6ac9639740 10 client.16935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:37:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.566+0000 7f6ac9639740 20 client.16935 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.566+0000 7f6ac9639740 1 --2- 172.21.15.43:0/1939313539 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e288e59630 0x55e2892ec4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.566+0000 7f6ac9639740 4 client.16935 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:37:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.566+0000 7f6ac9639740 1 -- 172.21.15.43:0/1939313539 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e2891d3580 con 0x55e288e59630 2024-04-25T10:37:31.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.566+0000 7f6abbfff640 1 --2- 172.21.15.43:0/1939313539 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e288e59630 0x55e2892ec4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.567+0000 7f6abbfff640 1 --2- 172.21.15.43:0/1939313539 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e288e59630 0x55e2892ec4b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e392427fa054c37 server_cookie=a8cbd0da16cca759 in_seq=0 out_seq=0 2024-04-25T10:37:31.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.567+0000 7f6a9b7fe640 10 client.16935 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:31.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.568+0000 7f6a9b7fe640 1 -- 172.21.15.43:0/1939313539 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e2891d3580 con 0x55e288e59630 2024-04-25T10:37:31.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.568+0000 7f6a9b7fe640 10 client.16935 handle_command_reply: tid=0 2024-04-25T10:37:31.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.568+0000 7f6ac9639740 1 client.16935 shutdown 2024-04-25T10:37:31.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.568+0000 7f6ac9639740 1 -- 172.21.15.43:0/1939313539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288ec5410 msgr2=0x55e2892739f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:31.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.568+0000 7f6ac9639740 1 --2- 172.21.15.43:0/1939313539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288ec5410 0x55e2892739f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6ab401ee20 tx=0x7f6ab401e800 comp rx=0 tx=0).stop 2024-04-25T10:37:31.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.568+0000 7f6ac9639740 1 -- 172.21.15.43:0/1939313539 shutdown_connections 2024-04-25T10:37:31.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.568+0000 7f6ac9639740 1 --2- 172.21.15.43:0/1939313539 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e288e59630 0x55e2892ec4b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.568+0000 7f6ac9639740 1 --2- 172.21.15.43:0/1939313539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e288ea4730 0x55e2892733a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.568+0000 7f6a9b7fe640 0 client.16935 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:31.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.568+0000 7f6ac9639740 1 --2- 172.21.15.43:0/1939313539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e288ec5410 0x55e2892739f0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.568+0000 7f6ac9639740 1 --2- 172.21.15.43:0/1939313539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e288e9c550 0x55e289278050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.568+0000 7f6ac9639740 1 -- 172.21.15.43:0/1939313539 >> 172.21.15.43:0/1939313539 conn(0x55e288eb9560 msgr2=0x55e289260c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:31.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.568+0000 7f6ac9639740 1 -- 172.21.15.43:0/1939313539 shutdown_connections 2024-04-25T10:37:31.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.568+0000 7f6ac9639740 1 -- 172.21.15.43:0/1939313539 wait complete. 2024-04-25T10:37:31.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.568+0000 7f6ac9639740 20 client.16935 trim_cache size 0 max 16384 2024-04-25T10:37:31.578 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:37:31.578 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:37:31.579 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "1cbf823e-dbcc-442f-9e56-06f8ef1bc64c", 2024-04-25T10:37:31.579 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:37:31.579 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:37:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.569+0000 7f6ac26b1640 1 -- 172.21.15.43:0/1208574531 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6aa40afce0 msgr2=0x7f6aa40b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.569+0000 7f6ac26b1640 1 --2- 172.21.15.43:0/1208574531 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6aa40afce0 0x7f6aa40b21f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55e289262790 tx=0x7f6ab4004b50 comp rx=0 tx=0).stop 2024-04-25T10:37:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.569+0000 7f6ac26b1640 1 -- 172.21.15.43:0/1208574531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6abc1038d0 msgr2=0x7f6abc1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.569+0000 7f6ac26b1640 1 --2- 172.21.15.43:0/1208574531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6abc1038d0 0x7f6abc1a0670 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f6aac02a930 tx=0x7f6aac004270 comp rx=0 tx=0).stop 2024-04-25T10:37:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.569+0000 7f6ac26b1640 1 -- 172.21.15.43:0/1208574531 shutdown_connections 2024-04-25T10:37:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.569+0000 7f6ac26b1640 1 --2- 172.21.15.43:0/1208574531 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6aa40afce0 0x7f6aa40b21f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.569+0000 7f6ac26b1640 1 --2- 172.21.15.43:0/1208574531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6abc108c50 0x7f6abc1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.569+0000 7f6ac26b1640 1 --2- 172.21.15.43:0/1208574531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6abc1090f0 0x7f6abc19a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.569+0000 7f6ac26b1640 1 --2- 172.21.15.43:0/1208574531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6abc1038d0 0x7f6abc1a0670 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.569+0000 7f6ac26b1640 1 -- 172.21.15.43:0/1208574531 >> 172.21.15.43:0/1208574531 conn(0x7f6abc077920 msgr2=0x7f6abc101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.569+0000 7f6ac26b1640 1 -- 172.21.15.43:0/1208574531 shutdown_connections 2024-04-25T10:37:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.569+0000 7f6ac26b1640 1 -- 172.21.15.43:0/1208574531 wait complete. 2024-04-25T10:37:31.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:37:31.582 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:37:31.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.694+0000 7fcad8f9d640 1 -- 172.21.15.43:0/4001081785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcad410def0 msgr2=0x7fcad41103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:31.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.694+0000 7fcad8f9d640 1 --2- 172.21.15.43:0/4001081785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcad410def0 0x7fcad41103e0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009ca0 tx=0x7fcacc02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.694+0000 7fcad8f9d640 1 -- 172.21.15.43:0/4001081785 shutdown_connections 2024-04-25T10:37:31.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.694+0000 7fcad8f9d640 1 --2- 172.21.15.43:0/4001081785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcad410b500 0x7fcad410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.694+0000 7fcad8f9d640 1 --2- 172.21.15.43:0/4001081785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcad410def0 0x7fcad41103e0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.694+0000 7fcad8f9d640 1 --2- 172.21.15.43:0/4001081785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcad4104aa0 0x7fcad4104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.694+0000 7fcad8f9d640 1 -- 172.21.15.43:0/4001081785 >> 172.21.15.43:0/4001081785 conn(0x7fcad4078000 msgr2=0x7fcad4078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:31.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.694+0000 7fcad8f9d640 1 -- 172.21.15.43:0/4001081785 shutdown_connections 2024-04-25T10:37:31.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.694+0000 7fcad8f9d640 1 -- 172.21.15.43:0/4001081785 wait complete. 2024-04-25T10:37:31.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.695+0000 7fcad8f9d640 1 Processor -- start 2024-04-25T10:37:31.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.695+0000 7fcad8f9d640 1 -- start start 2024-04-25T10:37:31.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.695+0000 7fcad8f9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcad4104aa0 0x7fcad419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.695+0000 7fcad8f9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcad410b500 0x7fcad419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad8f9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcad410def0 0x7fcad419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad8f9d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4113b50 con 0x7fcad4104aa0 2024-04-25T10:37:31.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad8f9d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41139d0 con 0x7fcad410def0 2024-04-25T10:37:31.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad8f9d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4113cd0 con 0x7fcad410b500 2024-04-25T10:37:31.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcad4104aa0 0x7fcad419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcad4104aa0 0x7fcad419a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46570/0 (socket says 172.21.15.43:46570) 2024-04-25T10:37:31.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad37fe640 1 -- 172.21.15.43:0/2563920481 learned_addr learned my addr 172.21.15.43:0/2563920481 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:31.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad2ffd640 1 --2- 172.21.15.43:0/2563920481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcad410b500 0x7fcad419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad37fe640 1 -- 172.21.15.43:0/2563920481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcad410b500 msgr2=0x7fcad419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:31.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad37fe640 1 --2- 172.21.15.43:0/2563920481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcad410b500 0x7fcad419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad37fe640 1 -- 172.21.15.43:0/2563920481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcad410def0 msgr2=0x7fcad419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:31.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad3fff640 1 --2- 172.21.15.43:0/2563920481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcad410def0 0x7fcad419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad37fe640 1 --2- 172.21.15.43:0/2563920481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcad410def0 0x7fcad419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad37fe640 1 -- 172.21.15.43:0/2563920481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad419f370 con 0x7fcad4104aa0 2024-04-25T10:37:31.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad3fff640 1 --2- 172.21.15.43:0/2563920481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcad410def0 0x7fcad419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:37:31.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad37fe640 1 --2- 172.21.15.43:0/2563920481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcad4104aa0 0x7fcad419a510 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fcac400d950 tx=0x7fcac400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:31.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad0ff9640 1 -- 172.21.15.43:0/2563920481 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4017070 con 0x7fcad4104aa0 2024-04-25T10:37:31.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad0ff9640 1 -- 172.21.15.43:0/2563920481 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcac4009c30 con 0x7fcad4104aa0 2024-04-25T10:37:31.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad8f9d640 1 -- 172.21.15.43:0/2563920481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad419f660 con 0x7fcad4104aa0 2024-04-25T10:37:31.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad0ff9640 1 -- 172.21.15.43:0/2563920481 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac400f620 con 0x7fcad4104aa0 2024-04-25T10:37:31.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.696+0000 7fcad8f9d640 1 -- 172.21.15.43:0/2563920481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad41a6fe0 con 0x7fcad4104aa0 2024-04-25T10:37:31.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.699+0000 7fcae118c740 1 Processor -- start 2024-04-25T10:37:31.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.699+0000 7fcae118c740 1 -- start start 2024-04-25T10:37:31.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.699+0000 7fcad0ff9640 1 -- 172.21.15.43:0/2563920481 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fcac4010460 con 0x7fcad4104aa0 2024-04-25T10:37:31.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.699+0000 7fcae118c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020ca4390 0x55d021044c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.699+0000 7fcae118c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020c7b4d0 0x55d021045190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.699+0000 7fcae118c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c836b0 0x55d021051ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.699+0000 7fcae118c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d020faad80 con 0x55d020c836b0 2024-04-25T10:37:31.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.700+0000 7fcae118c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d020f648b0 con 0x55d020c7b4d0 2024-04-25T10:37:31.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.700+0000 7fcae118c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d020fa6d40 con 0x55d020ca4390 2024-04-25T10:37:31.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.700+0000 7fcad2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c836b0 0x55d021051ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.700+0000 7fcad37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020c7b4d0 0x55d021045190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.700+0000 7fcad37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020c7b4d0 0x55d021045190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:42232/0 (socket says 172.21.15.43:42232) 2024-04-25T10:37:31.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.700+0000 7fcad2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c836b0 0x55d021051ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46582/0 (socket says 172.21.15.43:46582) 2024-04-25T10:37:31.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.700+0000 7fcad37fe640 1 -- 172.21.15.43:0/75956162 learned_addr learned my addr 172.21.15.43:0/75956162 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:31.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.700+0000 7fcad3fff640 1 --2- 172.21.15.43:0/75956162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020ca4390 0x55d021044c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.700+0000 7fcad0ff9640 1 --2- 172.21.15.43:0/2563920481 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcabc0afcd0 0x7fcabc0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.700+0000 7fcad2ffd640 1 -- 172.21.15.43:0/75956162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020ca4390 msgr2=0x55d021044c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:31.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.700+0000 7fcad2ffd640 1 --2- 172.21.15.43:0/75956162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020ca4390 0x55d021044c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.700+0000 7fcad2ffd640 1 -- 172.21.15.43:0/75956162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020c7b4d0 msgr2=0x55d021045190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:31.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.700+0000 7fcad2ffd640 1 --2- 172.21.15.43:0/75956162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020c7b4d0 0x55d021045190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.700+0000 7fcad2ffd640 1 -- 172.21.15.43:0/75956162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d020f35a40 con 0x55d020c836b0 2024-04-25T10:37:31.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.700+0000 7fcad3fff640 1 --2- 172.21.15.43:0/2563920481 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcabc0afcd0 0x7fcabc0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.700+0000 7fcad0ff9640 1 -- 172.21.15.43:0/2563920481 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fcac40dec80 con 0x7fcad4104aa0 2024-04-25T10:37:31.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.700+0000 7fcad2ffd640 1 --2- 172.21.15.43:0/75956162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c836b0 0x55d021051ae0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fcac800bb30 tx=0x7fcac800f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d74e9e7d7a3a8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:31.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.701+0000 7fcab9ffb640 1 -- 172.21.15.43:0/75956162 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac8015070 con 0x55d020c836b0 2024-04-25T10:37:31.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.701+0000 7fcab9ffb640 1 -- 172.21.15.43:0/75956162 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcac800fd90 con 0x55d020c836b0 2024-04-25T10:37:31.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.701+0000 7fcab9ffb640 1 -- 172.21.15.43:0/75956162 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac8011440 con 0x55d020c836b0 2024-04-25T10:37:31.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.701+0000 7fcae118c740 1 -- 172.21.15.43:0/75956162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c836b0 msgr2=0x55d021051ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:31.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.701+0000 7fcad3fff640 1 --2- 172.21.15.43:0/2563920481 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcabc0afcd0 0x7fcabc0b21e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55d021041dc0 tx=0x7fcacc0095c0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:31.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.701+0000 7fcae118c740 1 --2- 172.21.15.43:0/75956162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c836b0 0x55d021051ae0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fcac800bb30 tx=0x7fcac800f360 comp rx=0 tx=0).stop 2024-04-25T10:37:31.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.701+0000 7fcae118c740 1 -- 172.21.15.43:0/75956162 shutdown_connections 2024-04-25T10:37:31.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.701+0000 7fcae118c740 1 --2- 172.21.15.43:0/75956162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020c7b4d0 0x55d021045190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.701+0000 7fcae118c740 1 --2- 172.21.15.43:0/75956162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c836b0 0x55d021051ae0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.701+0000 7fcae118c740 1 --2- 172.21.15.43:0/75956162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020ca4390 0x55d021044c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.701+0000 7fcae118c740 1 -- 172.21.15.43:0/75956162 >> 172.21.15.43:0/75956162 conn(0x55d020c984e0 msgr2=0x55d021041400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:31.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.701+0000 7fcae118c740 1 -- 172.21.15.43:0/75956162 shutdown_connections 2024-04-25T10:37:31.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.701+0000 7fcae118c740 1 -- 172.21.15.43:0/75956162 wait complete. 2024-04-25T10:37:31.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.702+0000 7fcae118c740 1 -- start start 2024-04-25T10:37:31.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.702+0000 7fcae118c740 1 Processor -- start 2024-04-25T10:37:31.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.702+0000 7fcae118c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c7b4d0 0x55d02104f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.702+0000 7fcae118c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020c836b0 0x55d02104fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.702+0000 7fcae118c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020ca4390 0x55d02105b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.702+0000 7fcae118c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d0210521d0 con 0x55d020c7b4d0 2024-04-25T10:37:31.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.702+0000 7fcae118c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d020f31d10 con 0x55d020c836b0 2024-04-25T10:37:31.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.702+0000 7fcae118c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d021052350 con 0x55d020ca4390 2024-04-25T10:37:31.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.702+0000 7fcae118c740 10 client.0 _ll_register_callbacks cb 0x55d020f5e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:31.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.703+0000 7fcad37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c7b4d0 0x55d02104f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.703+0000 7fcad37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c7b4d0 0x55d02104f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46588/0 (socket says 172.21.15.43:46588) 2024-04-25T10:37:31.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.703+0000 7fcad37fe640 1 -- 172.21.15.43:0/3660718570 learned_addr learned my addr 172.21.15.43:0/3660718570 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:31.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.703+0000 7fcad2ffd640 1 --2- 172.21.15.43:0/3660718570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020c836b0 0x55d02104fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.703+0000 7fcad3fff640 1 --2- 172.21.15.43:0/3660718570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020ca4390 0x55d02105b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.703+0000 7fcad37fe640 1 -- 172.21.15.43:0/3660718570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020ca4390 msgr2=0x55d02105b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:31.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.703+0000 7fcad37fe640 1 --2- 172.21.15.43:0/3660718570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020ca4390 0x55d02105b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.703+0000 7fcad37fe640 1 -- 172.21.15.43:0/3660718570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020c836b0 msgr2=0x55d02104fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:31.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.703+0000 7fcad37fe640 1 --2- 172.21.15.43:0/3660718570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020c836b0 0x55d02104fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.703+0000 7fcad37fe640 1 -- 172.21.15.43:0/3660718570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d020d867d0 con 0x55d020c7b4d0 2024-04-25T10:37:31.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.703+0000 7fcad2ffd640 1 --2- 172.21.15.43:0/3660718570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020c836b0 0x55d02104fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:37:31.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.703+0000 7fcad3fff640 1 --2- 172.21.15.43:0/3660718570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020ca4390 0x55d02105b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:37:31.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.703+0000 7fcad37fe640 1 --2- 172.21.15.43:0/3660718570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c7b4d0 0x55d02104f720 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fcac40a6de0 tx=0x7fcac40e23c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96b37def6980ed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:31.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.703+0000 7fcab2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:31.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.703+0000 7fcab2ffd640 1 -- 172.21.15.43:0/3660718570 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4020050 con 0x55d020c7b4d0 2024-04-25T10:37:31.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.703+0000 7fcab2ffd640 1 -- 172.21.15.43:0/3660718570 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fcac40a3440 con 0x55d020c7b4d0 2024-04-25T10:37:31.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.703+0000 7fcae118c740 1 -- 172.21.15.43:0/3660718570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcac800fd90 con 0x55d020c7b4d0 2024-04-25T10:37:31.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.703+0000 7fcab2ffd640 1 -- 172.21.15.43:0/3660718570 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcac40a65c0 con 0x55d020c7b4d0 2024-04-25T10:37:31.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.703+0000 7fcae118c740 10 client.16947 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:31.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.703+0000 7fcae118c740 1 -- 172.21.15.43:0/3660718570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d020cbc3f0 con 0x55d020c7b4d0 2024-04-25T10:37:31.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.704+0000 7fcab2ffd640 1 -- 172.21.15.43:0/3660718570 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcac800fd90 con 0x55d020c7b4d0 2024-04-25T10:37:31.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.704+0000 7fcab2ffd640 1 -- 172.21.15.43:0/3660718570 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcac40a60b0 con 0x55d020c7b4d0 2024-04-25T10:37:31.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.704+0000 7fcae118c740 10 client.16947 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:31.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.704+0000 7fcae118c740 10 client.16947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:37:31.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.704+0000 7fcae118c740 20 client.16947 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:31.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.704+0000 7fcae118c740 1 --2- 172.21.15.43:0/3660718570 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d020c62f60 0x55d021061930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.704+0000 7fcae118c740 4 client.16947 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:37:31.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.704+0000 7fcae118c740 1 -- 172.21.15.43:0/3660718570 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d020fb8990 con 0x55d020c62f60 2024-04-25T10:37:31.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.704+0000 7fcad2ffd640 1 --2- 172.21.15.43:0/3660718570 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d020c62f60 0x55d021061930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.705+0000 7fcab2ffd640 10 client.16947 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:31.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.705+0000 7fcad2ffd640 1 --2- 172.21.15.43:0/3660718570 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d020c62f60 0x55d021061930 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99c681f387e3942a server_cookie=935648253dde6e04 in_seq=0 out_seq=0 2024-04-25T10:37:31.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.705+0000 7fcab2ffd640 1 -- 172.21.15.43:0/3660718570 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d020fb8990 con 0x55d020c62f60 2024-04-25T10:37:31.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.705+0000 7fcab2ffd640 10 client.16947 handle_command_reply: tid=0 2024-04-25T10:37:31.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.705+0000 7fcae118c740 1 client.16947 shutdown 2024-04-25T10:37:31.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.706+0000 7fcae118c740 1 -- 172.21.15.43:0/3660718570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c7b4d0 msgr2=0x55d02104f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:31.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.706+0000 7fcae118c740 1 --2- 172.21.15.43:0/3660718570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c7b4d0 0x55d02104f720 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fcac40a6de0 tx=0x7fcac40e23c0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.706+0000 7fcae118c740 1 -- 172.21.15.43:0/3660718570 shutdown_connections 2024-04-25T10:37:31.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.706+0000 7fcae118c740 1 --2- 172.21.15.43:0/3660718570 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d020c62f60 0x55d021061930 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.706+0000 7fcae118c740 1 --2- 172.21.15.43:0/3660718570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020c836b0 0x55d02104fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.706+0000 7fcae118c740 1 --2- 172.21.15.43:0/3660718570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020ca4390 0x55d02105b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.706+0000 7fcae118c740 1 --2- 172.21.15.43:0/3660718570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c7b4d0 0x55d02104f720 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.706+0000 7fcab2ffd640 0 client.16947 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:31.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.706+0000 7fcae118c740 1 -- 172.21.15.43:0/3660718570 >> 172.21.15.43:0/3660718570 conn(0x55d020c984e0 msgr2=0x55d021046970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:31.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.706+0000 7fcae118c740 1 -- 172.21.15.43:0/3660718570 shutdown_connections 2024-04-25T10:37:31.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.706+0000 7fcae118c740 1 -- 172.21.15.43:0/3660718570 wait complete. 2024-04-25T10:37:31.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.706+0000 7fcae118c740 20 client.16947 trim_cache size 0 max 16384 2024-04-25T10:37:31.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.724+0000 7fcae118c740 1 Processor -- start 2024-04-25T10:37:31.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.724+0000 7fcae118c740 1 -- start start 2024-04-25T10:37:31.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.724+0000 7fcae118c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c7b4d0 0x55d021045070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.724+0000 7fcae118c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020c836b0 0x55d0210670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.724+0000 7fcae118c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020ca4390 0x55d021067630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.724+0000 7fcae118c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41b2070 con 0x55d020c7b4d0 2024-04-25T10:37:31.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.724+0000 7fcae118c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41af940 con 0x55d020ca4390 2024-04-25T10:37:31.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.724+0000 7fcae118c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41ad790 con 0x55d020c836b0 2024-04-25T10:37:31.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.724+0000 7fcad37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c7b4d0 0x55d021045070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.724+0000 7fcad37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c7b4d0 0x55d021045070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46594/0 (socket says 172.21.15.43:46594) 2024-04-25T10:37:31.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.724+0000 7fcad37fe640 1 -- 172.21.15.43:0/512245186 learned_addr learned my addr 172.21.15.43:0/512245186 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.724+0000 7fcad3fff640 1 --2- 172.21.15.43:0/512245186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020ca4390 0x55d021067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.724+0000 7fcad2ffd640 1 --2- 172.21.15.43:0/512245186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020c836b0 0x55d0210670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.724+0000 7fcad37fe640 1 -- 172.21.15.43:0/512245186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020c836b0 msgr2=0x55d0210670f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.724+0000 7fcad37fe640 1 --2- 172.21.15.43:0/512245186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020c836b0 0x55d0210670f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.725+0000 7fcad37fe640 1 -- 172.21.15.43:0/512245186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020ca4390 msgr2=0x55d021067630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.725+0000 7fcad37fe640 1 --2- 172.21.15.43:0/512245186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020ca4390 0x55d021067630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.725+0000 7fcad37fe640 1 -- 172.21.15.43:0/512245186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d020c7c0f0 con 0x55d020c7b4d0 2024-04-25T10:37:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.725+0000 7fcad2ffd640 1 --2- 172.21.15.43:0/512245186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020c836b0 0x55d0210670f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:37:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.725+0000 7fcad37fe640 1 --2- 172.21.15.43:0/512245186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c7b4d0 0x55d021045070 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fcac40a6420 tx=0x7fcac4010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f546597d6624e481 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.725+0000 7fcab97fa640 1 -- 172.21.15.43:0/512245186 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4020070 con 0x55d020c7b4d0 2024-04-25T10:37:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.725+0000 7fcab97fa640 1 -- 172.21.15.43:0/512245186 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcac40a9020 con 0x55d020c7b4d0 2024-04-25T10:37:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.725+0000 7fcab97fa640 1 -- 172.21.15.43:0/512245186 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac40a6e50 con 0x55d020c7b4d0 2024-04-25T10:37:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.726+0000 7fcae118c740 1 -- 172.21.15.43:0/512245186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c7b4d0 msgr2=0x55d021045070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.726+0000 7fcae118c740 1 --2- 172.21.15.43:0/512245186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c7b4d0 0x55d021045070 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fcac40a6420 tx=0x7fcac4010cb0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.726+0000 7fcae118c740 1 -- 172.21.15.43:0/512245186 shutdown_connections 2024-04-25T10:37:31.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.726+0000 7fcae118c740 1 --2- 172.21.15.43:0/512245186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020ca4390 0x55d021067630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.726+0000 7fcae118c740 1 --2- 172.21.15.43:0/512245186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020c836b0 0x55d0210670f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.726+0000 7fcae118c740 1 --2- 172.21.15.43:0/512245186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c7b4d0 0x55d021045070 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.726+0000 7fcae118c740 1 -- 172.21.15.43:0/512245186 >> 172.21.15.43:0/512245186 conn(0x55d020c984e0 msgr2=0x55d021041350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:31.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.726+0000 7fcae118c740 1 -- 172.21.15.43:0/512245186 shutdown_connections 2024-04-25T10:37:31.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.726+0000 7fcae118c740 1 -- 172.21.15.43:0/512245186 wait complete. 2024-04-25T10:37:31.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.726+0000 7fcae118c740 1 -- start start 2024-04-25T10:37:31.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.726+0000 7fcae118c740 1 Processor -- start 2024-04-25T10:37:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.726+0000 7fcae118c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020c7b4d0 0x55d021052660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.726+0000 7fcae118c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c836b0 0x55d021052ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.726+0000 7fcae118c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020ca4390 0x55d0210530e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.726+0000 7fcae118c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41ade00 con 0x55d020c836b0 2024-04-25T10:37:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcae118c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41af270 con 0x55d020ca4390 2024-04-25T10:37:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcae118c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41ae4d0 con 0x55d020c7b4d0 2024-04-25T10:37:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcae118c740 10 client.0 _ll_register_callbacks cb 0x7fcac40a2d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcad2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c836b0 0x55d021052ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcad2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c836b0 0x55d021052ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46610/0 (socket says 172.21.15.43:46610) 2024-04-25T10:37:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcad2ffd640 1 -- 172.21.15.43:0/2314150902 learned_addr learned my addr 172.21.15.43:0/2314150902 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcad37fe640 1 --2- 172.21.15.43:0/2314150902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020c7b4d0 0x55d021052660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcad3fff640 1 --2- 172.21.15.43:0/2314150902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020ca4390 0x55d0210530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcad2ffd640 1 -- 172.21.15.43:0/2314150902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020c7b4d0 msgr2=0x55d021052660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcad2ffd640 1 --2- 172.21.15.43:0/2314150902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020c7b4d0 0x55d021052660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcad2ffd640 1 -- 172.21.15.43:0/2314150902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020ca4390 msgr2=0x55d0210530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcad2ffd640 1 --2- 172.21.15.43:0/2314150902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020ca4390 0x55d0210530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcad2ffd640 1 -- 172.21.15.43:0/2314150902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d020f6b4b0 con 0x55d020c836b0 2024-04-25T10:37:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcad37fe640 1 --2- 172.21.15.43:0/2314150902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020c7b4d0 0x55d021052660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:37:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcad2ffd640 1 --2- 172.21.15.43:0/2314150902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c836b0 0x55d021052ba0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fcac8000c00 tx=0x7fcac8003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f378789f0692d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcab2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcab2ffd640 1 -- 172.21.15.43:0/2314150902 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac8015040 con 0x55d020c836b0 2024-04-25T10:37:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcab2ffd640 1 -- 172.21.15.43:0/2314150902 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fcac8004020 con 0x55d020c836b0 2024-04-25T10:37:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcae118c740 1 -- 172.21.15.43:0/2314150902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcac40a9020 con 0x55d020c836b0 2024-04-25T10:37:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcab2ffd640 1 -- 172.21.15.43:0/2314150902 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcac803a050 con 0x55d020c836b0 2024-04-25T10:37:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcae118c740 10 client.16953 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.727+0000 7fcae118c740 1 -- 172.21.15.43:0/2314150902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d0210c8ea0 con 0x55d020c836b0 2024-04-25T10:37:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.728+0000 7fcab2ffd640 1 -- 172.21.15.43:0/2314150902 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcac40a9020 con 0x55d020c836b0 2024-04-25T10:37:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.728+0000 7fcab2ffd640 1 -- 172.21.15.43:0/2314150902 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcac8029800 con 0x55d020c836b0 2024-04-25T10:37:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.728+0000 7fcae118c740 10 client.16953 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.728+0000 7fcae118c740 10 client.16953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:37:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.728+0000 7fcae118c740 20 client.16953 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.728+0000 7fcae118c740 1 --2- 172.21.15.43:0/2314150902 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d020c383a0 0x55d0210cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.728+0000 7fcae118c740 4 client.16953 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:37:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.728+0000 7fcae118c740 1 -- 172.21.15.43:0/2314150902 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d020fb2800 con 0x55d020c383a0 2024-04-25T10:37:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.728+0000 7fcad37fe640 1 --2- 172.21.15.43:0/2314150902 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d020c383a0 0x55d0210cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.728+0000 7fcab2ffd640 10 client.16953 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.728+0000 7fcad37fe640 1 --2- 172.21.15.43:0/2314150902 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d020c383a0 0x55d0210cb300 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c875bda032150e5c server_cookie=1be046abd3009b9 in_seq=0 out_seq=0 2024-04-25T10:37:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.729+0000 7fcab2ffd640 1 -- 172.21.15.43:0/2314150902 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d020fb2800 con 0x55d020c383a0 2024-04-25T10:37:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.729+0000 7fcab2ffd640 10 client.16953 handle_command_reply: tid=0 2024-04-25T10:37:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.729+0000 7fcae118c740 1 client.16953 shutdown 2024-04-25T10:37:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.729+0000 7fcae118c740 1 -- 172.21.15.43:0/2314150902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c836b0 msgr2=0x55d021052ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.729+0000 7fcae118c740 1 --2- 172.21.15.43:0/2314150902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c836b0 0x55d021052ba0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fcac8000c00 tx=0x7fcac8003820 comp rx=0 tx=0).stop 2024-04-25T10:37:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.729+0000 7fcae118c740 1 -- 172.21.15.43:0/2314150902 shutdown_connections 2024-04-25T10:37:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.729+0000 7fcae118c740 1 --2- 172.21.15.43:0/2314150902 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d020c383a0 0x55d0210cb300 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.729+0000 7fcae118c740 1 --2- 172.21.15.43:0/2314150902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d020ca4390 0x55d0210530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.729+0000 7fcae118c740 1 --2- 172.21.15.43:0/2314150902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d020c836b0 0x55d021052ba0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.729+0000 7fcae118c740 1 --2- 172.21.15.43:0/2314150902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d020c7b4d0 0x55d021052660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.729+0000 7fcab2ffd640 0 client.16953 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.729+0000 7fcae118c740 1 -- 172.21.15.43:0/2314150902 >> 172.21.15.43:0/2314150902 conn(0x55d020c984e0 msgr2=0x55d02103f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.729+0000 7fcae118c740 1 -- 172.21.15.43:0/2314150902 shutdown_connections 2024-04-25T10:37:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.729+0000 7fcae118c740 1 -- 172.21.15.43:0/2314150902 wait complete. 2024-04-25T10:37:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.729+0000 7fcae118c740 20 client.16953 trim_cache size 0 max 16384 2024-04-25T10:37:31.738 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:37:31.738 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:37:31.738 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:37:31.738 INFO:teuthology.orchestra.run.smithi043.stdout: "1cbf823e-dbcc-442f-9e56-06f8ef1bc64c": { 2024-04-25T10:37:31.739 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:37:31.739 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "1cbf823e-dbcc-442f-9e56-06f8ef1bc64c", 2024-04-25T10:37:31.739 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:37:31.739 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:37:31.739 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:37:31.739 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:37:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.730+0000 7fcad8f9d640 1 -- 172.21.15.43:0/2563920481 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcabc0afcd0 msgr2=0x7fcabc0b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.730+0000 7fcad8f9d640 1 --2- 172.21.15.43:0/2563920481 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcabc0afcd0 0x7fcabc0b21e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55d021041dc0 tx=0x7fcacc0095c0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.730+0000 7fcad8f9d640 1 -- 172.21.15.43:0/2563920481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcad4104aa0 msgr2=0x7fcad419a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.730+0000 7fcad8f9d640 1 --2- 172.21.15.43:0/2563920481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcad4104aa0 0x7fcad419a510 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fcac400d950 tx=0x7fcac400de10 comp rx=0 tx=0).stop 2024-04-25T10:37:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.730+0000 7fcad8f9d640 1 -- 172.21.15.43:0/2563920481 shutdown_connections 2024-04-25T10:37:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.730+0000 7fcad8f9d640 1 --2- 172.21.15.43:0/2563920481 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcabc0afcd0 0x7fcabc0b21e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.730+0000 7fcad8f9d640 1 --2- 172.21.15.43:0/2563920481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcad410def0 0x7fcad419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.730+0000 7fcad8f9d640 1 --2- 172.21.15.43:0/2563920481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcad410b500 0x7fcad419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.730+0000 7fcad8f9d640 1 --2- 172.21.15.43:0/2563920481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcad4104aa0 0x7fcad419a510 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.730+0000 7fcad8f9d640 1 -- 172.21.15.43:0/2563920481 >> 172.21.15.43:0/2563920481 conn(0x7fcad4078000 msgr2=0x7fcad4100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.730+0000 7fcad8f9d640 1 -- 172.21.15.43:0/2563920481 shutdown_connections 2024-04-25T10:37:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:37:31.730+0000 7fcad8f9d640 1 -- 172.21.15.43:0/2563920481 wait complete. 2024-04-25T10:37:31.742 INFO:tasks.cephfs.filesystem:scrub status for tag:1cbf823e-dbcc-442f-9e56-06f8ef1bc64c - {'path': '/', 'tag': '1cbf823e-dbcc-442f-9e56-06f8ef1bc64c', 'options': 'recursive,force'} 2024-04-25T10:37:32.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:31 smithi043 ceph-mon[29478]: pgmap v153: 97 pgs: 97 active+clean; 2.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-04-25T10:37:32.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:31 smithi077 ceph-mon[32772]: pgmap v153: 97 pgs: 97 active+clean; 2.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-04-25T10:37:32.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:31 smithi164 ceph-mon[32713]: pgmap v153: 97 pgs: 97 active+clean; 2.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-04-25T10:37:33.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:32 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:37:33.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:32 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:37:33.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:32 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:37:33.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:32 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:37:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.232+0000 7ff550677640 1 -- 172.21.15.43:0/657610005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff548108ea0 msgr2=0x7ff5481032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.232+0000 7ff550677640 1 --2- 172.21.15.43:0/657610005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff548108ea0 0x7ff5481032b0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff538009900 tx=0x7ff53802a480 comp rx=0 tx=0).stop 2024-04-25T10:37:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.232+0000 7ff550677640 1 -- 172.21.15.43:0/657610005 shutdown_connections 2024-04-25T10:37:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.232+0000 7ff550677640 1 --2- 172.21.15.43:0/657610005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff548103940 0x7ff54810f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.232+0000 7ff550677640 1 --2- 172.21.15.43:0/657610005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff548108ea0 0x7ff5481032b0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.232+0000 7ff550677640 1 --2- 172.21.15.43:0/657610005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff548108a90 0x7ff548102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.232+0000 7ff550677640 1 -- 172.21.15.43:0/657610005 >> 172.21.15.43:0/657610005 conn(0x7ff5480fd8f0 msgr2=0x7ff5480ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.232+0000 7ff550677640 1 -- 172.21.15.43:0/657610005 shutdown_connections 2024-04-25T10:37:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.232+0000 7ff550677640 1 -- 172.21.15.43:0/657610005 wait complete. 2024-04-25T10:37:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.232+0000 7ff550677640 1 Processor -- start 2024-04-25T10:37:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.232+0000 7ff550677640 1 -- start start 2024-04-25T10:37:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.233+0000 7ff550677640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff548108a90 0x7ff548199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.233+0000 7ff550677640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff548108ea0 0x7ff5481997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.233+0000 7ff550677640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff54819da90 0x7ff54819dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.233+0000 7ff550677640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff548112c50 con 0x7ff54819da90 2024-04-25T10:37:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.233+0000 7ff550677640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff548112ad0 con 0x7ff548108ea0 2024-04-25T10:37:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.233+0000 7ff550677640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff548112dd0 con 0x7ff548108a90 2024-04-25T10:37:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.233+0000 7ff54ec14640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff54819da90 0x7ff54819dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.233+0000 7ff54ec14640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff54819da90 0x7ff54819dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46628/0 (socket says 172.21.15.43:46628) 2024-04-25T10:37:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.233+0000 7ff54ec14640 1 -- 172.21.15.43:0/1713720249 learned_addr learned my addr 172.21.15.43:0/1713720249 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.233+0000 7ff54e413640 1 --2- 172.21.15.43:0/1713720249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff548108a90 0x7ff548199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.233+0000 7ff54dc12640 1 --2- 172.21.15.43:0/1713720249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff548108ea0 0x7ff5481997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.233+0000 7ff54ec14640 1 -- 172.21.15.43:0/1713720249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff548108a90 msgr2=0x7ff548199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.233+0000 7ff54ec14640 1 --2- 172.21.15.43:0/1713720249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff548108a90 0x7ff548199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.233+0000 7ff54ec14640 1 -- 172.21.15.43:0/1713720249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff548108ea0 msgr2=0x7ff5481997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.233+0000 7ff54ec14640 1 --2- 172.21.15.43:0/1713720249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff548108ea0 0x7ff5481997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.233+0000 7ff54ec14640 1 -- 172.21.15.43:0/1713720249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54819e460 con 0x7ff54819da90 2024-04-25T10:37:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.233+0000 7ff54e413640 1 --2- 172.21.15.43:0/1713720249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff548108a90 0x7ff548199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:37:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.233+0000 7ff54ec14640 1 --2- 172.21.15.43:0/1713720249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff54819da90 0x7ff54819dea0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff54000db20 tx=0x7ff54000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.234+0000 7ff53f7fe640 1 -- 172.21.15.43:0/1713720249 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff540017070 con 0x7ff54819da90 2024-04-25T10:37:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.234+0000 7ff53f7fe640 1 -- 172.21.15.43:0/1713720249 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff540003e20 con 0x7ff54819da90 2024-04-25T10:37:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.234+0000 7ff550677640 1 -- 172.21.15.43:0/1713720249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff54819e750 con 0x7ff54819da90 2024-04-25T10:37:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.234+0000 7ff53f7fe640 1 -- 172.21.15.43:0/1713720249 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff540010460 con 0x7ff54819da90 2024-04-25T10:37:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.234+0000 7ff550677640 1 -- 172.21.15.43:0/1713720249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5481a5f20 con 0x7ff54819da90 2024-04-25T10:37:33.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.236+0000 7ff557605740 1 Processor -- start 2024-04-25T10:37:33.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.236+0000 7ff53f7fe640 1 -- 172.21.15.43:0/1713720249 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7ff540003970 con 0x7ff54819da90 2024-04-25T10:37:33.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.237+0000 7ff557605740 1 -- start start 2024-04-25T10:37:33.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.237+0000 7ff557605740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aca3191b0 0x561aca6b8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:33.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.237+0000 7ff557605740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561aca2f02f0 0x561aca6b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:33.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.237+0000 7ff557605740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 0x561aca6b9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:33.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.237+0000 7ff557605740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561aca61f460 con 0x561aca2f84d0 2024-04-25T10:37:33.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.237+0000 7ff557605740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561aca5d9590 con 0x561aca2f02f0 2024-04-25T10:37:33.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.237+0000 7ff557605740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561aca61b420 con 0x561aca3191b0 2024-04-25T10:37:33.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.237+0000 7ff54dc12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 0x561aca6b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:33.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.237+0000 7ff54dc12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 0x561aca6b9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46642/0 (socket says 172.21.15.43:46642) 2024-04-25T10:37:33.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.237+0000 7ff54dc12640 1 -- 172.21.15.43:0/510681325 learned_addr learned my addr 172.21.15.43:0/510681325 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:33.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.237+0000 7ff53f7fe640 1 --2- 172.21.15.43:0/1713720249 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff5340afc50 0x7ff5340b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:33.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.237+0000 7ff54dc12640 1 --2- 172.21.15.43:0/510681325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aca3191b0 0x561aca6b8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:33.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.237+0000 7ff54e413640 1 --2- 172.21.15.43:0/510681325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561aca2f02f0 0x561aca6b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:33.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.238+0000 7ff54ec14640 1 --2- 172.21.15.43:0/1713720249 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff5340afc50 0x7ff5340b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:33.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.238+0000 7ff54dc12640 1 -- 172.21.15.43:0/510681325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aca3191b0 msgr2=0x561aca6b8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:33.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.238+0000 7ff54dc12640 1 --2- 172.21.15.43:0/510681325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aca3191b0 0x561aca6b8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.238+0000 7ff54dc12640 1 -- 172.21.15.43:0/510681325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561aca2f02f0 msgr2=0x561aca6b8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:33.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.238+0000 7ff53f7fe640 1 -- 172.21.15.43:0/1713720249 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7ff5400ded90 con 0x7ff54819da90 2024-04-25T10:37:33.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.238+0000 7ff54dc12640 1 --2- 172.21.15.43:0/510681325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561aca2f02f0 0x561aca6b8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.238+0000 7ff54dc12640 1 -- 172.21.15.43:0/510681325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aca5e60d0 con 0x561aca2f84d0 2024-04-25T10:37:33.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.238+0000 7ff54dc12640 1 --2- 172.21.15.43:0/510681325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 0x561aca6b9380 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff5380096c0 tx=0x7ff538005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fc782b7479bdda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:33.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.238+0000 7ff53cbfe640 1 -- 172.21.15.43:0/510681325 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff538035070 con 0x561aca2f84d0 2024-04-25T10:37:33.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.238+0000 7ff54ec14640 1 --2- 172.21.15.43:0/1713720249 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff5340afc50 0x7ff5340b2160 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff5400dc5c0 tx=0x7ff540072660 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:33.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.238+0000 7ff53cbfe640 1 -- 172.21.15.43:0/510681325 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5380035f0 con 0x561aca2f84d0 2024-04-25T10:37:33.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.238+0000 7ff53cbfe640 1 -- 172.21.15.43:0/510681325 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5380044e0 con 0x561aca2f84d0 2024-04-25T10:37:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.239+0000 7ff557605740 1 -- 172.21.15.43:0/510681325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 msgr2=0x561aca6b9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.239+0000 7ff557605740 1 --2- 172.21.15.43:0/510681325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 0x561aca6b9380 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff5380096c0 tx=0x7ff538005780 comp rx=0 tx=0).stop 2024-04-25T10:37:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.239+0000 7ff557605740 1 -- 172.21.15.43:0/510681325 shutdown_connections 2024-04-25T10:37:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.239+0000 7ff557605740 1 --2- 172.21.15.43:0/510681325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561aca2f02f0 0x561aca6b8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.239+0000 7ff557605740 1 --2- 172.21.15.43:0/510681325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 0x561aca6b9380 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.239+0000 7ff557605740 1 --2- 172.21.15.43:0/510681325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aca3191b0 0x561aca6b8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.239+0000 7ff557605740 1 -- 172.21.15.43:0/510681325 >> 172.21.15.43:0/510681325 conn(0x561aca30d300 msgr2=0x561aca6b52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.239+0000 7ff557605740 1 -- 172.21.15.43:0/510681325 shutdown_connections 2024-04-25T10:37:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.239+0000 7ff557605740 1 -- 172.21.15.43:0/510681325 wait complete. 2024-04-25T10:37:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.240+0000 7ff557605740 1 -- start start 2024-04-25T10:37:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.240+0000 7ff557605740 1 Processor -- start 2024-04-25T10:37:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.240+0000 7ff557605740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aca2f02f0 0x561aca6c5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.240+0000 7ff557605740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 0x561aca6c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.240+0000 7ff557605740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561aca3191b0 0x561aca6d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.240+0000 7ff557605740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561aca6c7e30 con 0x561aca2f84d0 2024-04-25T10:37:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.240+0000 7ff557605740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561aca5a6b60 con 0x561aca3191b0 2024-04-25T10:37:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.240+0000 7ff557605740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561aca6c7fb0 con 0x561aca2f02f0 2024-04-25T10:37:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.240+0000 7ff557605740 10 client.0 _ll_register_callbacks cb 0x561aca5cb2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.240+0000 7ff54e413640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 0x561aca6c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.240+0000 7ff54e413640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 0x561aca6c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46648/0 (socket says 172.21.15.43:46648) 2024-04-25T10:37:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.240+0000 7ff54e413640 1 -- 172.21.15.43:0/3349125861 learned_addr learned my addr 172.21.15.43:0/3349125861 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.240+0000 7ff54e413640 1 -- 172.21.15.43:0/3349125861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aca2f02f0 msgr2=0x561aca6c5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.240+0000 7ff54dc12640 1 --2- 172.21.15.43:0/3349125861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aca2f02f0 0x561aca6c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.240+0000 7ff54e413640 1 --2- 172.21.15.43:0/3349125861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aca2f02f0 0x561aca6c5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.240+0000 7ff54e413640 1 -- 172.21.15.43:0/3349125861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561aca3191b0 msgr2=0x561aca6d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.241+0000 7ff54e413640 1 --2- 172.21.15.43:0/3349125861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561aca3191b0 0x561aca6d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.241+0000 7ff54e413640 1 -- 172.21.15.43:0/3349125861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aca331210 con 0x561aca2f84d0 2024-04-25T10:37:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.241+0000 7ff54dc12640 1 --2- 172.21.15.43:0/3349125861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aca2f02f0 0x561aca6c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:37:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.241+0000 7ff54e413640 1 --2- 172.21.15.43:0/3349125861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 0x561aca6c5860 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff544011810 tx=0x7ff544011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d437e354f17f9fbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.241+0000 7ff531ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.241+0000 7ff531ffb640 1 -- 172.21.15.43:0/3349125861 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff544004460 con 0x561aca2f84d0 2024-04-25T10:37:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.241+0000 7ff531ffb640 1 -- 172.21.15.43:0/3349125861 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7ff54400f9a0 con 0x561aca2f84d0 2024-04-25T10:37:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.241+0000 7ff557605740 1 -- 172.21.15.43:0/3349125861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5380035f0 con 0x561aca2f84d0 2024-04-25T10:37:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.241+0000 7ff531ffb640 1 -- 172.21.15.43:0/3349125861 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff54400fba0 con 0x561aca2f84d0 2024-04-25T10:37:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.241+0000 7ff557605740 10 client.16965 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.241+0000 7ff557605740 1 -- 172.21.15.43:0/3349125861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561aca3fd620 con 0x561aca2f84d0 2024-04-25T10:37:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.241+0000 7ff531ffb640 1 -- 172.21.15.43:0/3349125861 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5380035f0 con 0x561aca2f84d0 2024-04-25T10:37:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.241+0000 7ff531ffb640 1 -- 172.21.15.43:0/3349125861 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff544018020 con 0x561aca2f84d0 2024-04-25T10:37:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.242+0000 7ff557605740 10 client.16965 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.242+0000 7ff557605740 10 client.16965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:37:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.242+0000 7ff557605740 20 client.16965 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.242+0000 7ff557605740 1 --2- 172.21.15.43:0/3349125861 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561aca2d7d80 0x561aca6d7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.242+0000 7ff557605740 4 client.16965 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:37:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.242+0000 7ff557605740 1 -- 172.21.15.43:0/3349125861 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561aca62d070 con 0x561aca2d7d80 2024-04-25T10:37:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.242+0000 7ff54dc12640 1 --2- 172.21.15.43:0/3349125861 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561aca2d7d80 0x561aca6d7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.242+0000 7ff54dc12640 1 --2- 172.21.15.43:0/3349125861 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561aca2d7d80 0x561aca6d7950 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b5fd318d2ba2424 server_cookie=6ed7a604644c04be in_seq=0 out_seq=0 2024-04-25T10:37:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.242+0000 7ff531ffb640 10 client.16965 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.243+0000 7ff531ffb640 1 -- 172.21.15.43:0/3349125861 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561aca62d070 con 0x561aca2d7d80 2024-04-25T10:37:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.243+0000 7ff531ffb640 10 client.16965 handle_command_reply: tid=0 2024-04-25T10:37:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.243+0000 7ff557605740 1 client.16965 shutdown 2024-04-25T10:37:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.243+0000 7ff557605740 1 -- 172.21.15.43:0/3349125861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 msgr2=0x561aca6c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.243+0000 7ff557605740 1 --2- 172.21.15.43:0/3349125861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 0x561aca6c5860 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff544011810 tx=0x7ff544011cd0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.243+0000 7ff557605740 1 -- 172.21.15.43:0/3349125861 shutdown_connections 2024-04-25T10:37:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.243+0000 7ff557605740 1 --2- 172.21.15.43:0/3349125861 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561aca2d7d80 0x561aca6d7950 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.243+0000 7ff557605740 1 --2- 172.21.15.43:0/3349125861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561aca3191b0 0x561aca6d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.243+0000 7ff557605740 1 --2- 172.21.15.43:0/3349125861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 0x561aca6c5860 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.243+0000 7ff531ffb640 0 client.16965 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.243+0000 7ff557605740 1 --2- 172.21.15.43:0/3349125861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aca2f02f0 0x561aca6c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.243+0000 7ff557605740 1 -- 172.21.15.43:0/3349125861 >> 172.21.15.43:0/3349125861 conn(0x561aca30d300 msgr2=0x561aca6ba780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.243+0000 7ff557605740 1 -- 172.21.15.43:0/3349125861 shutdown_connections 2024-04-25T10:37:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.243+0000 7ff557605740 1 -- 172.21.15.43:0/3349125861 wait complete. 2024-04-25T10:37:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.244+0000 7ff557605740 20 client.16965 trim_cache size 0 max 16384 2024-04-25T10:37:33.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.259+0000 7ff557605740 1 Processor -- start 2024-04-25T10:37:33.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.259+0000 7ff557605740 1 -- start start 2024-04-25T10:37:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.259+0000 7ff557605740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aca30d300 0x561aca6bad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.259+0000 7ff557605740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561aca2f02f0 0x561aca6d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.259+0000 7ff557605740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 0x561aca6beb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.259+0000 7ff557605740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff5481aff00 con 0x561aca2f84d0 2024-04-25T10:37:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.259+0000 7ff557605740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff5481ad7d0 con 0x561aca2f02f0 2024-04-25T10:37:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.259+0000 7ff557605740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff5481ab620 con 0x561aca30d300 2024-04-25T10:37:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.259+0000 7ff54dc12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 0x561aca6beb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.259+0000 7ff54dc12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 0x561aca6beb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46658/0 (socket says 172.21.15.43:46658) 2024-04-25T10:37:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.259+0000 7ff54dc12640 1 -- 172.21.15.43:0/441005539 learned_addr learned my addr 172.21.15.43:0/441005539 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.260+0000 7ff54e413640 1 --2- 172.21.15.43:0/441005539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561aca2f02f0 0x561aca6d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.260+0000 7ff54dc12640 1 --2- 172.21.15.43:0/441005539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aca30d300 0x561aca6bad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.260+0000 7ff54dc12640 1 -- 172.21.15.43:0/441005539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aca30d300 msgr2=0x561aca6bad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.260+0000 7ff54dc12640 1 --2- 172.21.15.43:0/441005539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aca30d300 0x561aca6bad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.260+0000 7ff54dc12640 1 -- 172.21.15.43:0/441005539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561aca2f02f0 msgr2=0x561aca6d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.260+0000 7ff54dc12640 1 --2- 172.21.15.43:0/441005539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561aca2f02f0 0x561aca6d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.260+0000 7ff54dc12640 1 -- 172.21.15.43:0/441005539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aca6c3660 con 0x561aca2f84d0 2024-04-25T10:37:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.260+0000 7ff54dc12640 1 --2- 172.21.15.43:0/441005539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 0x561aca6beb30 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff538003f30 tx=0x7ff538003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c4ea973af80eb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.260+0000 7ff533fff640 1 -- 172.21.15.43:0/441005539 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53804f030 con 0x561aca2f84d0 2024-04-25T10:37:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.260+0000 7ff533fff640 1 -- 172.21.15.43:0/441005539 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5380035f0 con 0x561aca2f84d0 2024-04-25T10:37:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.260+0000 7ff533fff640 1 -- 172.21.15.43:0/441005539 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff538035040 con 0x561aca2f84d0 2024-04-25T10:37:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.261+0000 7ff557605740 1 -- 172.21.15.43:0/441005539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 msgr2=0x561aca6beb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.261+0000 7ff557605740 1 --2- 172.21.15.43:0/441005539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 0x561aca6beb30 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff538003f30 tx=0x7ff538003f60 comp rx=0 tx=0).stop 2024-04-25T10:37:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.261+0000 7ff557605740 1 -- 172.21.15.43:0/441005539 shutdown_connections 2024-04-25T10:37:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.261+0000 7ff557605740 1 --2- 172.21.15.43:0/441005539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561aca2f02f0 0x561aca6d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.261+0000 7ff557605740 1 --2- 172.21.15.43:0/441005539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f84d0 0x561aca6beb30 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.261+0000 7ff557605740 1 --2- 172.21.15.43:0/441005539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aca30d300 0x561aca6bad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.261+0000 7ff557605740 1 -- 172.21.15.43:0/441005539 >> 172.21.15.43:0/441005539 conn(0x561aca6bfc30 msgr2=0x561aca6ba5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.261+0000 7ff557605740 1 -- 172.21.15.43:0/441005539 shutdown_connections 2024-04-25T10:37:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.261+0000 7ff557605740 1 -- 172.21.15.43:0/441005539 wait complete. 2024-04-25T10:37:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.261+0000 7ff557605740 1 -- start start 2024-04-25T10:37:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.261+0000 7ff557605740 1 Processor -- start 2024-04-25T10:37:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff557605740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f02f0 0x561aca6c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff557605740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561aca2f84d0 0x561aca6c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff557605740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aca30d300 0x561aca6c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff557605740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff5481abc90 con 0x561aca2f02f0 2024-04-25T10:37:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff557605740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff5481ad100 con 0x561aca2f84d0 2024-04-25T10:37:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff557605740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff5481ac360 con 0x561aca30d300 2024-04-25T10:37:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff557605740 10 client.0 _ll_register_callbacks cb 0x7ff544008760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff54dc12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f02f0 0x561aca6c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff54dc12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f02f0 0x561aca6c8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46670/0 (socket says 172.21.15.43:46670) 2024-04-25T10:37:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff54dc12640 1 -- 172.21.15.43:0/1733910040 learned_addr learned my addr 172.21.15.43:0/1733910040 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff54e413640 1 --2- 172.21.15.43:0/1733910040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561aca2f84d0 0x561aca6c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff54dc12640 1 -- 172.21.15.43:0/1733910040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aca30d300 msgr2=0x561aca6c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff54dc12640 1 --2- 172.21.15.43:0/1733910040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aca30d300 0x561aca6c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff54dc12640 1 -- 172.21.15.43:0/1733910040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561aca2f84d0 msgr2=0x561aca6c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff54dc12640 1 --2- 172.21.15.43:0/1733910040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561aca2f84d0 0x561aca6c92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff54dc12640 1 -- 172.21.15.43:0/1733910040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aca73ecc0 con 0x561aca2f02f0 2024-04-25T10:37:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff54dc12640 1 --2- 172.21.15.43:0/1733910040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f02f0 0x561aca6c8d60 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff538031440 tx=0x7ff538009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a85fd29bb490f98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff531ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff531ffb640 1 -- 172.21.15.43:0/1733910040 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53804f070 con 0x561aca2f02f0 2024-04-25T10:37:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff557605740 1 -- 172.21.15.43:0/1733910040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5380035f0 con 0x561aca2f02f0 2024-04-25T10:37:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.262+0000 7ff531ffb640 1 -- 172.21.15.43:0/1733910040 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7ff5380037a0 con 0x561aca2f02f0 2024-04-25T10:37:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.263+0000 7ff531ffb640 1 -- 172.21.15.43:0/1733910040 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff538035040 con 0x561aca2f02f0 2024-04-25T10:37:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.263+0000 7ff557605740 10 client.16971 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.263+0000 7ff557605740 1 -- 172.21.15.43:0/1733910040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561aca73fe00 con 0x561aca2f02f0 2024-04-25T10:37:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.263+0000 7ff531ffb640 1 -- 172.21.15.43:0/1733910040 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5380035f0 con 0x561aca2f02f0 2024-04-25T10:37:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.263+0000 7ff531ffb640 1 -- 172.21.15.43:0/1733910040 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff538051ce0 con 0x561aca2f02f0 2024-04-25T10:37:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.263+0000 7ff557605740 10 client.16971 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.263+0000 7ff557605740 10 client.16971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:37:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.263+0000 7ff557605740 20 client.16971 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.263+0000 7ff557605740 1 --2- 172.21.15.43:0/1733910040 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561aca3191b0 0x561aca742030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.263+0000 7ff557605740 4 client.16971 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:37:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.263+0000 7ff557605740 1 -- 172.21.15.43:0/1733910040 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561aca626ee0 con 0x561aca3191b0 2024-04-25T10:37:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.263+0000 7ff54e413640 1 --2- 172.21.15.43:0/1733910040 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561aca3191b0 0x561aca742030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.264+0000 7ff531ffb640 10 client.16971 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.264+0000 7ff54e413640 1 --2- 172.21.15.43:0/1733910040 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561aca3191b0 0x561aca742030 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=348f51b4883efd3 server_cookie=1a41780f975d4136 in_seq=0 out_seq=0 2024-04-25T10:37:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.279+0000 7ff531ffb640 1 -- 172.21.15.43:0/1733910040 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561aca626ee0 con 0x561aca3191b0 2024-04-25T10:37:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.279+0000 7ff531ffb640 10 client.16971 handle_command_reply: tid=0 2024-04-25T10:37:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.279+0000 7ff557605740 1 client.16971 shutdown 2024-04-25T10:37:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.279+0000 7ff557605740 1 -- 172.21.15.43:0/1733910040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f02f0 msgr2=0x561aca6c8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.279+0000 7ff557605740 1 --2- 172.21.15.43:0/1733910040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f02f0 0x561aca6c8d60 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff538031440 tx=0x7ff538009520 comp rx=0 tx=0).stop 2024-04-25T10:37:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.279+0000 7ff557605740 1 -- 172.21.15.43:0/1733910040 shutdown_connections 2024-04-25T10:37:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.279+0000 7ff557605740 1 --2- 172.21.15.43:0/1733910040 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561aca3191b0 0x561aca742030 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.279+0000 7ff557605740 1 --2- 172.21.15.43:0/1733910040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561aca2f84d0 0x561aca6c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.279+0000 7ff557605740 1 --2- 172.21.15.43:0/1733910040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aca30d300 0x561aca6c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.279+0000 7ff557605740 1 --2- 172.21.15.43:0/1733910040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aca2f02f0 0x561aca6c8d60 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.279+0000 7ff531ffb640 0 client.16971 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.279+0000 7ff557605740 1 -- 172.21.15.43:0/1733910040 >> 172.21.15.43:0/1733910040 conn(0x561aca6bfc30 msgr2=0x561aca6c6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.279+0000 7ff557605740 1 -- 172.21.15.43:0/1733910040 shutdown_connections 2024-04-25T10:37:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.279+0000 7ff557605740 1 -- 172.21.15.43:0/1733910040 wait complete. 2024-04-25T10:37:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.279+0000 7ff557605740 20 client.16971 trim_cache size 0 max 16384 2024-04-25T10:37:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:37:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:37:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:37:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:37:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.280+0000 7ff550677640 1 -- 172.21.15.43:0/1713720249 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff5340afc50 msgr2=0x7ff5340b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.280+0000 7ff550677640 1 --2- 172.21.15.43:0/1713720249 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff5340afc50 0x7ff5340b2160 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff5400dc5c0 tx=0x7ff540072660 comp rx=0 tx=0).stop 2024-04-25T10:37:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.280+0000 7ff550677640 1 -- 172.21.15.43:0/1713720249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff54819da90 msgr2=0x7ff54819dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.280+0000 7ff550677640 1 --2- 172.21.15.43:0/1713720249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff54819da90 0x7ff54819dea0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff54000db20 tx=0x7ff54000b260 comp rx=0 tx=0).stop 2024-04-25T10:37:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.280+0000 7ff550677640 1 -- 172.21.15.43:0/1713720249 shutdown_connections 2024-04-25T10:37:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.280+0000 7ff550677640 1 --2- 172.21.15.43:0/1713720249 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff5340afc50 0x7ff5340b2160 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.280+0000 7ff550677640 1 --2- 172.21.15.43:0/1713720249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff548108ea0 0x7ff5481997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.280+0000 7ff550677640 1 --2- 172.21.15.43:0/1713720249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff54819da90 0x7ff54819dea0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.280+0000 7ff550677640 1 --2- 172.21.15.43:0/1713720249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff548108a90 0x7ff548199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.280+0000 7ff550677640 1 -- 172.21.15.43:0/1713720249 >> 172.21.15.43:0/1713720249 conn(0x7ff5480fd8f0 msgr2=0x7ff5480fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.280+0000 7ff550677640 1 -- 172.21.15.43:0/1713720249 shutdown_connections 2024-04-25T10:37:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:33.280+0000 7ff550677640 1 -- 172.21.15.43:0/1713720249 wait complete. 2024-04-25T10:37:33.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:32 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:37:33.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:32 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:37:33.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:32 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:37:33.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:32 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:37:33.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:32 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:37:33.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:32 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:37:33.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:32 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:37:33.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:32 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:37:34.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:33 smithi043 ceph-mon[29478]: pgmap v154: 97 pgs: 97 active+clean; 2.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-04-25T10:37:34.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:33 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:37:34.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:33 smithi077 ceph-mon[32772]: pgmap v154: 97 pgs: 97 active+clean; 2.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-04-25T10:37:34.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:33 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:37:34.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:33 smithi164 ceph-mon[32713]: pgmap v154: 97 pgs: 97 active+clean; 2.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-04-25T10:37:34.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:33 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:37:36.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:35 smithi043 ceph-mon[29478]: pgmap v155: 97 pgs: 97 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T10:37:36.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:35 smithi077 ceph-mon[32772]: pgmap v155: 97 pgs: 97 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T10:37:36.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:35 smithi164 ceph-mon[32713]: pgmap v155: 97 pgs: 97 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T10:37:38.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:37 smithi043 ceph-mon[29478]: pgmap v156: 97 pgs: 97 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-25T10:37:38.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:37 smithi077 ceph-mon[32772]: pgmap v156: 97 pgs: 97 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-25T10:37:38.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:37 smithi164 ceph-mon[32713]: pgmap v156: 97 pgs: 97 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-25T10:37:40.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:39 smithi043 ceph-mon[29478]: pgmap v157: 97 pgs: 97 active+clean; 2.8 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-25T10:37:40.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:39 smithi077 ceph-mon[32772]: pgmap v157: 97 pgs: 97 active+clean; 2.8 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-25T10:37:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:39 smithi164 ceph-mon[32713]: pgmap v157: 97 pgs: 97 active+clean; 2.8 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-25T10:37:42.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:41 smithi043 ceph-mon[29478]: pgmap v158: 97 pgs: 97 active+clean; 2.8 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-04-25T10:37:42.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:41 smithi077 ceph-mon[32772]: pgmap v158: 97 pgs: 97 active+clean; 2.8 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-04-25T10:37:42.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:41 smithi164 ceph-mon[32713]: pgmap v158: 97 pgs: 97 active+clean; 2.8 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-04-25T10:37:44.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:43 smithi043 ceph-mon[29478]: pgmap v159: 97 pgs: 97 active+clean; 2.8 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-04-25T10:37:44.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:43 smithi077 ceph-mon[32772]: pgmap v159: 97 pgs: 97 active+clean; 2.8 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-04-25T10:37:44.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:43 smithi164 ceph-mon[32713]: pgmap v159: 97 pgs: 97 active+clean; 2.8 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-04-25T10:37:46.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:45 smithi077 ceph-mon[32772]: pgmap v160: 97 pgs: 97 active+clean; 3.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-04-25T10:37:46.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:45 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:37:46.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:45 smithi164 ceph-mon[32713]: pgmap v160: 97 pgs: 97 active+clean; 3.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-04-25T10:37:46.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:45 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:37:46.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:45 smithi043 ceph-mon[29478]: pgmap v160: 97 pgs: 97 active+clean; 3.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-04-25T10:37:46.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:45 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:37:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.393+0000 7f01398d9640 1 -- 172.21.15.43:0/1543014325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0134108a80 msgr2=0x7f0134102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.393+0000 7f01398d9640 1 --2- 172.21.15.43:0/1543014325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0134108a80 0x7f0134102620 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f0124009ca0 tx=0x7f012402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.393+0000 7f01398d9640 1 -- 172.21.15.43:0/1543014325 shutdown_connections 2024-04-25T10:37:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.393+0000 7f01398d9640 1 --2- 172.21.15.43:0/1543014325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0134108f20 0x7f0134102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.393+0000 7f01398d9640 1 --2- 172.21.15.43:0/1543014325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01341031d0 0x7f013410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.393+0000 7f01398d9640 1 --2- 172.21.15.43:0/1543014325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0134108a80 0x7f0134102620 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.393+0000 7f01398d9640 1 -- 172.21.15.43:0/1543014325 >> 172.21.15.43:0/1543014325 conn(0x7f0134079170 msgr2=0x7f01340795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.393+0000 7f01398d9640 1 -- 172.21.15.43:0/1543014325 shutdown_connections 2024-04-25T10:37:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.393+0000 7f01398d9640 1 -- 172.21.15.43:0/1543014325 wait complete. 2024-04-25T10:37:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.394+0000 7f01398d9640 1 Processor -- start 2024-04-25T10:37:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.394+0000 7f01398d9640 1 -- start start 2024-04-25T10:37:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.394+0000 7f01398d9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01341031d0 0x7f013419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.394+0000 7f01398d9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0134108a80 0x7f013419ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.394+0000 7f01398d9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0134108f20 0x7f013419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.394+0000 7f01398d9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0134112c70 con 0x7f01341031d0 2024-04-25T10:37:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.394+0000 7f01398d9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0134112af0 con 0x7f0134108a80 2024-04-25T10:37:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.394+0000 7f01398d9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0134112df0 con 0x7f0134108f20 2024-04-25T10:37:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.394+0000 7f0132ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01341031d0 0x7f013419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.394+0000 7f0132ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01341031d0 0x7f013419a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44400/0 (socket says 172.21.15.43:44400) 2024-04-25T10:37:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.394+0000 7f0132ffd640 1 -- 172.21.15.43:0/1416361633 learned_addr learned my addr 172.21.15.43:0/1416361633 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.394+0000 7f01327fc640 1 --2- 172.21.15.43:0/1416361633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0134108a80 0x7f013419ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.395+0000 7f01337fe640 1 --2- 172.21.15.43:0/1416361633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0134108f20 0x7f013419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.395+0000 7f0132ffd640 1 -- 172.21.15.43:0/1416361633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0134108f20 msgr2=0x7f013419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.395+0000 7f0132ffd640 1 --2- 172.21.15.43:0/1416361633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0134108f20 0x7f013419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.395+0000 7f0132ffd640 1 -- 172.21.15.43:0/1416361633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0134108a80 msgr2=0x7f013419ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.395+0000 7f0132ffd640 1 --2- 172.21.15.43:0/1416361633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0134108a80 0x7f013419ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.395+0000 7f0132ffd640 1 -- 172.21.15.43:0/1416361633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013419f5e0 con 0x7f01341031d0 2024-04-25T10:37:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.395+0000 7f0132ffd640 1 --2- 172.21.15.43:0/1416361633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01341031d0 0x7f013419a780 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f012402ade0 tx=0x7f0124031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.395+0000 7f01388d7640 1 -- 172.21.15.43:0/1416361633 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124003b70 con 0x7f01341031d0 2024-04-25T10:37:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.395+0000 7f01388d7640 1 -- 172.21.15.43:0/1416361633 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0124003d10 con 0x7f01341031d0 2024-04-25T10:37:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.395+0000 7f01388d7640 1 -- 172.21.15.43:0/1416361633 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012402fa50 con 0x7f01341031d0 2024-04-25T10:37:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.395+0000 7f01398d9640 1 -- 172.21.15.43:0/1416361633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013419f870 con 0x7f01341031d0 2024-04-25T10:37:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.395+0000 7f01398d9640 1 -- 172.21.15.43:0/1416361633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01341a7250 con 0x7f01341031d0 2024-04-25T10:37:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.398+0000 7f0140782740 1 Processor -- start 2024-04-25T10:37:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.398+0000 7f0140782740 1 -- start start 2024-04-25T10:37:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.398+0000 7f0140782740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035fb1b0 0x56140399aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.398+0000 7f0140782740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035d22f0 0x56140399b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.398+0000 7f0140782740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035da4d0 0x56140399b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.398+0000 7f0140782740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561403901460 con 0x5614035d22f0 2024-04-25T10:37:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.398+0000 7f0140782740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5614038bb590 con 0x5614035da4d0 2024-04-25T10:37:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.398+0000 7f0140782740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5614038fd420 con 0x5614035fb1b0 2024-04-25T10:37:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.399+0000 7f01327fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035da4d0 0x56140399b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.399+0000 7f01337fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035fb1b0 0x56140399aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.399+0000 7f01327fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035da4d0 0x56140399b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:52324/0 (socket says 172.21.15.43:52324) 2024-04-25T10:37:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.399+0000 7f01327fc640 1 -- 172.21.15.43:0/3310840193 learned_addr learned my addr 172.21.15.43:0/3310840193 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.399+0000 7f01337fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035fb1b0 0x56140399aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:54230/0 (socket says 172.21.15.43:54230) 2024-04-25T10:37:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.399+0000 7f01388d7640 1 -- 172.21.15.43:0/1416361633 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f0124004580 con 0x7f01341031d0 2024-04-25T10:37:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.399+0000 7f01327fc640 1 -- 172.21.15.43:0/3310840193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035fb1b0 msgr2=0x56140399aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.399+0000 7f01327fc640 1 --2- 172.21.15.43:0/3310840193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035fb1b0 0x56140399aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.399+0000 7f01327fc640 1 -- 172.21.15.43:0/3310840193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035d22f0 msgr2=0x56140399b030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:37:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.399+0000 7f01327fc640 1 --2- 172.21.15.43:0/3310840193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035d22f0 0x56140399b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.399+0000 7f01327fc640 1 -- 172.21.15.43:0/3310840193 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614038c80d0 con 0x5614035da4d0 2024-04-25T10:37:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.399+0000 7f01337fe640 1 --2- 172.21.15.43:0/3310840193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035fb1b0 0x56140399aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:37:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.399+0000 7f01327fc640 1 --2- 172.21.15.43:0/3310840193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035da4d0 0x56140399b570 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0128004520 tx=0x7f0128004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b70a09eb0bfce7df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.399+0000 7f0119ffb640 1 -- 172.21.15.43:0/3310840193 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128003bb0 con 0x5614035da4d0 2024-04-25T10:37:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.400+0000 7f0119ffb640 1 -- 172.21.15.43:0/3310840193 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0128015070 con 0x5614035da4d0 2024-04-25T10:37:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.400+0000 7f0119ffb640 1 -- 172.21.15.43:0/3310840193 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01280106a0 con 0x5614035da4d0 2024-04-25T10:37:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.400+0000 7f01388d7640 1 --2- 172.21.15.43:0/1416361633 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f011c0afcd0 0x7f011c0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.400+0000 7f01337fe640 1 --2- 172.21.15.43:0/1416361633 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f011c0afcd0 0x7f011c0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.400+0000 7f01388d7640 1 -- 172.21.15.43:0/1416361633 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f01241002c0 con 0x7f01341031d0 2024-04-25T10:37:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.400+0000 7f0140782740 1 -- 172.21.15.43:0/3310840193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035da4d0 msgr2=0x56140399b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.400+0000 7f01337fe640 1 --2- 172.21.15.43:0/1416361633 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f011c0afcd0 0x7f011c0b21e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x561403997c60 tx=0x7f012c005f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.400+0000 7f0140782740 1 --2- 172.21.15.43:0/3310840193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035da4d0 0x56140399b570 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0128004520 tx=0x7f0128004610 comp rx=0 tx=0).stop 2024-04-25T10:37:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.401+0000 7f0140782740 1 -- 172.21.15.43:0/3310840193 shutdown_connections 2024-04-25T10:37:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.401+0000 7f0140782740 1 --2- 172.21.15.43:0/3310840193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035da4d0 0x56140399b570 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.401+0000 7f0140782740 1 --2- 172.21.15.43:0/3310840193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035d22f0 0x56140399b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.401+0000 7f0140782740 1 --2- 172.21.15.43:0/3310840193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035fb1b0 0x56140399aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.401+0000 7f0140782740 1 -- 172.21.15.43:0/3310840193 >> 172.21.15.43:0/3310840193 conn(0x5614035ef300 msgr2=0x5614039972a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.401+0000 7f0140782740 1 -- 172.21.15.43:0/3310840193 shutdown_connections 2024-04-25T10:37:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.401+0000 7f0140782740 1 -- 172.21.15.43:0/3310840193 wait complete. 2024-04-25T10:37:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.401+0000 7f0140782740 1 -- start start 2024-04-25T10:37:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.401+0000 7f0140782740 1 Processor -- start 2024-04-25T10:37:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.402+0000 7f0140782740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035d22f0 0x5614039a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.402+0000 7f0140782740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035da4d0 0x5614039a5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.402+0000 7f0140782740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035fb1b0 0x5614039ab110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.402+0000 7f0140782740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5614039a7c20 con 0x5614035d22f0 2024-04-25T10:37:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.402+0000 7f0140782740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561403888b60 con 0x5614035fb1b0 2024-04-25T10:37:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.402+0000 7f0140782740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5614039a7da0 con 0x5614035da4d0 2024-04-25T10:37:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.402+0000 7f0140782740 10 client.0 _ll_register_callbacks cb 0x5614038ad2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.402+0000 7f0132ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035d22f0 0x5614039a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.402+0000 7f0132ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035d22f0 0x5614039a51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44428/0 (socket says 172.21.15.43:44428) 2024-04-25T10:37:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.402+0000 7f0132ffd640 1 -- 172.21.15.43:0/931903034 learned_addr learned my addr 172.21.15.43:0/931903034 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.402+0000 7f01327fc640 1 --2- 172.21.15.43:0/931903034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035da4d0 0x5614039a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.402+0000 7f01337fe640 1 --2- 172.21.15.43:0/931903034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035fb1b0 0x5614039ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.402+0000 7f0132ffd640 1 -- 172.21.15.43:0/931903034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035da4d0 msgr2=0x5614039a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.402+0000 7f0132ffd640 1 --2- 172.21.15.43:0/931903034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035da4d0 0x5614039a5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.402+0000 7f0132ffd640 1 -- 172.21.15.43:0/931903034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035fb1b0 msgr2=0x5614039ab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.402+0000 7f0132ffd640 1 --2- 172.21.15.43:0/931903034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035fb1b0 0x5614039ab110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.402+0000 7f0132ffd640 1 -- 172.21.15.43:0/931903034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561403613210 con 0x5614035d22f0 2024-04-25T10:37:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.402+0000 7f0132ffd640 1 --2- 172.21.15.43:0/931903034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035d22f0 0x5614039a51b0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f01240fe040 tx=0x7f01240c8370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7a12ba9ce29f23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.402+0000 7f0112ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.403+0000 7f0112ffd640 1 -- 172.21.15.43:0/931903034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01240c7d10 con 0x5614035d22f0 2024-04-25T10:37:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.403+0000 7f0112ffd640 1 -- 172.21.15.43:0/931903034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f012402fc10 con 0x5614035d22f0 2024-04-25T10:37:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.403+0000 7f0140782740 1 -- 172.21.15.43:0/931903034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0128015070 con 0x5614035d22f0 2024-04-25T10:37:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.403+0000 7f0112ffd640 1 -- 172.21.15.43:0/931903034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f012402fe50 con 0x5614035d22f0 2024-04-25T10:37:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.403+0000 7f0140782740 10 client.16980 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.403+0000 7f0140782740 1 -- 172.21.15.43:0/931903034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5614036df620 con 0x5614035d22f0 2024-04-25T10:37:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.403+0000 7f0112ffd640 1 -- 172.21.15.43:0/931903034 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0128015070 con 0x5614035d22f0 2024-04-25T10:37:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.403+0000 7f0112ffd640 1 -- 172.21.15.43:0/931903034 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f01240c8800 con 0x5614035d22f0 2024-04-25T10:37:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.403+0000 7f0140782740 10 client.16980 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.403+0000 7f0140782740 10 client.16980 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:37:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.403+0000 7f0140782740 20 client.16980 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.403+0000 7f0140782740 1 --2- 172.21.15.43:0/931903034 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5614035b9d80 0x5614039b7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.403+0000 7f0140782740 4 client.16980 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:37:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.403+0000 7f0140782740 1 -- 172.21.15.43:0/931903034 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56140390f070 con 0x5614035b9d80 2024-04-25T10:37:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.404+0000 7f01327fc640 1 --2- 172.21.15.43:0/931903034 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5614035b9d80 0x5614039b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.404+0000 7f01327fc640 1 --2- 172.21.15.43:0/931903034 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5614035b9d80 0x5614039b7730 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93fb48335c979cb9 server_cookie=4d383e5b9a2c2ec1 in_seq=0 out_seq=0 2024-04-25T10:37:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.404+0000 7f0112ffd640 10 client.16980 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.405+0000 7f0112ffd640 1 -- 172.21.15.43:0/931903034 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56140390f070 con 0x5614035b9d80 2024-04-25T10:37:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.405+0000 7f0112ffd640 10 client.16980 handle_command_reply: tid=0 2024-04-25T10:37:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.405+0000 7f0140782740 1 client.16980 shutdown 2024-04-25T10:37:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.405+0000 7f0140782740 1 -- 172.21.15.43:0/931903034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035d22f0 msgr2=0x5614039a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.405+0000 7f0140782740 1 --2- 172.21.15.43:0/931903034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035d22f0 0x5614039a51b0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f01240fe040 tx=0x7f01240c8370 comp rx=0 tx=0).stop 2024-04-25T10:37:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.405+0000 7f0140782740 1 -- 172.21.15.43:0/931903034 shutdown_connections 2024-04-25T10:37:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.405+0000 7f0140782740 1 --2- 172.21.15.43:0/931903034 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5614035b9d80 0x5614039b7730 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.405+0000 7f0140782740 1 --2- 172.21.15.43:0/931903034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035fb1b0 0x5614039ab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.405+0000 7f0140782740 1 --2- 172.21.15.43:0/931903034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035da4d0 0x5614039a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.405+0000 7f0112ffd640 0 client.16980 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.405+0000 7f0140782740 1 --2- 172.21.15.43:0/931903034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035d22f0 0x5614039a51b0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.405+0000 7f0140782740 1 -- 172.21.15.43:0/931903034 >> 172.21.15.43:0/931903034 conn(0x5614035ef300 msgr2=0x56140399c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.405+0000 7f0140782740 1 -- 172.21.15.43:0/931903034 shutdown_connections 2024-04-25T10:37:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.405+0000 7f0140782740 1 -- 172.21.15.43:0/931903034 wait complete. 2024-04-25T10:37:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.405+0000 7f0140782740 20 client.16980 trim_cache size 0 max 16384 2024-04-25T10:37:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.420+0000 7f0140782740 1 Processor -- start 2024-04-25T10:37:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.421+0000 7f0140782740 1 -- start start 2024-04-25T10:37:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.421+0000 7f0140782740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035d22f0 0x56140399b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.421+0000 7f0140782740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035da4d0 0x5614039a5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.421+0000 7f0140782740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035fb1b0 0x5614039ab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.421+0000 7f0140782740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f01341b2200 con 0x5614035fb1b0 2024-04-25T10:37:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.421+0000 7f0140782740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f01341afad0 con 0x5614035d22f0 2024-04-25T10:37:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.421+0000 7f0140782740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f01341ad920 con 0x5614035da4d0 2024-04-25T10:37:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.421+0000 7f01337fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035fb1b0 0x5614039ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.421+0000 7f01337fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035fb1b0 0x5614039ab010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44432/0 (socket says 172.21.15.43:44432) 2024-04-25T10:37:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.421+0000 7f01337fe640 1 -- 172.21.15.43:0/73497034 learned_addr learned my addr 172.21.15.43:0/73497034 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.421+0000 7f01327fc640 1 --2- 172.21.15.43:0/73497034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035da4d0 0x5614039a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.421+0000 7f0132ffd640 1 --2- 172.21.15.43:0/73497034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035d22f0 0x56140399b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.421+0000 7f01337fe640 1 -- 172.21.15.43:0/73497034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035da4d0 msgr2=0x5614039a5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.421+0000 7f01337fe640 1 --2- 172.21.15.43:0/73497034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035da4d0 0x5614039a5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.421+0000 7f01337fe640 1 -- 172.21.15.43:0/73497034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035d22f0 msgr2=0x56140399b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.421+0000 7f01337fe640 1 --2- 172.21.15.43:0/73497034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035d22f0 0x56140399b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.421+0000 7f01337fe640 1 -- 172.21.15.43:0/73497034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561403992a60 con 0x5614035fb1b0 2024-04-25T10:37:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.422+0000 7f01337fe640 1 --2- 172.21.15.43:0/73497034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035fb1b0 0x5614039ab010 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f012c004380 tx=0x7f012c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e810202655207d28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.422+0000 7f01197fa640 1 -- 172.21.15.43:0/73497034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c003af0 con 0x5614035fb1b0 2024-04-25T10:37:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.422+0000 7f01197fa640 1 -- 172.21.15.43:0/73497034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f012c02d070 con 0x5614035fb1b0 2024-04-25T10:37:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.422+0000 7f01197fa640 1 -- 172.21.15.43:0/73497034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c028400 con 0x5614035fb1b0 2024-04-25T10:37:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.423+0000 7f0140782740 1 -- 172.21.15.43:0/73497034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035fb1b0 msgr2=0x5614039ab010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.423+0000 7f0140782740 1 --2- 172.21.15.43:0/73497034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035fb1b0 0x5614039ab010 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f012c004380 tx=0x7f012c017000 comp rx=0 tx=0).stop 2024-04-25T10:37:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.423+0000 7f0140782740 1 -- 172.21.15.43:0/73497034 shutdown_connections 2024-04-25T10:37:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.423+0000 7f0140782740 1 --2- 172.21.15.43:0/73497034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035fb1b0 0x5614039ab010 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.423+0000 7f0140782740 1 --2- 172.21.15.43:0/73497034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035da4d0 0x5614039a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.423+0000 7f0140782740 1 --2- 172.21.15.43:0/73497034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035d22f0 0x56140399b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.423+0000 7f0140782740 1 -- 172.21.15.43:0/73497034 >> 172.21.15.43:0/73497034 conn(0x5614035ef300 msgr2=0x5614039a4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.423+0000 7f0140782740 1 -- 172.21.15.43:0/73497034 shutdown_connections 2024-04-25T10:37:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.423+0000 7f0140782740 1 -- 172.21.15.43:0/73497034 wait complete. 2024-04-25T10:37:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.423+0000 7f0140782740 1 -- start start 2024-04-25T10:37:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.423+0000 7f0140782740 1 Processor -- start 2024-04-25T10:37:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f0140782740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035d22f0 0x5614039b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f0140782740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035da4d0 0x5614039b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f0140782740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035fb1b0 0x561403996420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f0140782740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f01341adf90 con 0x5614035da4d0 2024-04-25T10:37:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f0140782740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f01341af400 con 0x5614035fb1b0 2024-04-25T10:37:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f0140782740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f01341ae660 con 0x5614035d22f0 2024-04-25T10:37:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f0140782740 10 client.0 _ll_register_callbacks cb 0x7f0124009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f01327fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035da4d0 0x5614039b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f01327fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035da4d0 0x5614039b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44442/0 (socket says 172.21.15.43:44442) 2024-04-25T10:37:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f01327fc640 1 -- 172.21.15.43:0/1073518684 learned_addr learned my addr 172.21.15.43:0/1073518684 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f0132ffd640 1 --2- 172.21.15.43:0/1073518684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035d22f0 0x5614039b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f01337fe640 1 --2- 172.21.15.43:0/1073518684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035fb1b0 0x561403996420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f01327fc640 1 -- 172.21.15.43:0/1073518684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035d22f0 msgr2=0x5614039b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f01327fc640 1 --2- 172.21.15.43:0/1073518684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035d22f0 0x5614039b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f01327fc640 1 -- 172.21.15.43:0/1073518684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035fb1b0 msgr2=0x561403996420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f01327fc640 1 --2- 172.21.15.43:0/1073518684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035fb1b0 0x561403996420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f01327fc640 1 -- 172.21.15.43:0/1073518684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561403996bb0 con 0x5614035da4d0 2024-04-25T10:37:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f01337fe640 1 --2- 172.21.15.43:0/1073518684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035fb1b0 0x561403996420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:37:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f01327fc640 1 --2- 172.21.15.43:0/1073518684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035da4d0 0x5614039b1c10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f0128000c00 tx=0x7f0128003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c0f994fd81badd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f0112ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.424+0000 7f0112ffd640 1 -- 172.21.15.43:0/1073518684 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128005270 con 0x5614035da4d0 2024-04-25T10:37:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.425+0000 7f0140782740 1 -- 172.21.15.43:0/1073518684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f012c02d070 con 0x5614035da4d0 2024-04-25T10:37:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.425+0000 7f0112ffd640 1 -- 172.21.15.43:0/1073518684 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f0128005410 con 0x5614035da4d0 2024-04-25T10:37:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.425+0000 7f0112ffd640 1 -- 172.21.15.43:0/1073518684 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0128039050 con 0x5614035da4d0 2024-04-25T10:37:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.425+0000 7f0140782740 10 client.16986 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.425+0000 7f0140782740 1 -- 172.21.15.43:0/1073518684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5614039afca0 con 0x5614035da4d0 2024-04-25T10:37:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.425+0000 7f0112ffd640 1 -- 172.21.15.43:0/1073518684 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f012c02d070 con 0x5614035da4d0 2024-04-25T10:37:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.425+0000 7f0112ffd640 1 -- 172.21.15.43:0/1073518684 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0128004840 con 0x5614035da4d0 2024-04-25T10:37:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.425+0000 7f0140782740 10 client.16986 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.425+0000 7f0140782740 10 client.16986 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:37:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.425+0000 7f0140782740 20 client.16986 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.425+0000 7f0140782740 1 --2- 172.21.15.43:0/1073518684 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56140358f1c0 0x561403a21ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.425+0000 7f0140782740 4 client.16986 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:37:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.425+0000 7f0140782740 1 -- 172.21.15.43:0/1073518684 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561403908ee0 con 0x56140358f1c0 2024-04-25T10:37:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.425+0000 7f0132ffd640 1 --2- 172.21.15.43:0/1073518684 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56140358f1c0 0x561403a21ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.426+0000 7f0112ffd640 10 client.16986 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.426+0000 7f0132ffd640 1 --2- 172.21.15.43:0/1073518684 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56140358f1c0 0x561403a21ac0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7df55a45761f7209 server_cookie=5d92b51a1f164fbe in_seq=0 out_seq=0 2024-04-25T10:37:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.439+0000 7f0112ffd640 1 -- 172.21.15.43:0/1073518684 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561403908ee0 con 0x56140358f1c0 2024-04-25T10:37:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.439+0000 7f0112ffd640 10 client.16986 handle_command_reply: tid=0 2024-04-25T10:37:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.439+0000 7f0140782740 1 client.16986 shutdown 2024-04-25T10:37:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.439+0000 7f0140782740 1 -- 172.21.15.43:0/1073518684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035da4d0 msgr2=0x5614039b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.439+0000 7f0140782740 1 --2- 172.21.15.43:0/1073518684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035da4d0 0x5614039b1c10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f0128000c00 tx=0x7f0128003ce0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.439+0000 7f0140782740 1 -- 172.21.15.43:0/1073518684 shutdown_connections 2024-04-25T10:37:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.439+0000 7f0140782740 1 --2- 172.21.15.43:0/1073518684 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56140358f1c0 0x561403a21ac0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.439+0000 7f0140782740 1 --2- 172.21.15.43:0/1073518684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614035fb1b0 0x561403996420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.439+0000 7f0112ffd640 0 client.16986 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.439+0000 7f0140782740 1 --2- 172.21.15.43:0/1073518684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614035da4d0 0x5614039b1c10 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.439+0000 7f0140782740 1 --2- 172.21.15.43:0/1073518684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614035d22f0 0x5614039b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.439+0000 7f0140782740 1 -- 172.21.15.43:0/1073518684 >> 172.21.15.43:0/1073518684 conn(0x5614035ef300 msgr2=0x5614039a64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.439+0000 7f0140782740 1 -- 172.21.15.43:0/1073518684 shutdown_connections 2024-04-25T10:37:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.439+0000 7f0140782740 1 -- 172.21.15.43:0/1073518684 wait complete. 2024-04-25T10:37:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.439+0000 7f0140782740 20 client.16986 trim_cache size 0 max 16384 2024-04-25T10:37:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:37:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:37:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:37:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:37:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.440+0000 7f01398d9640 1 -- 172.21.15.43:0/1416361633 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f011c0afcd0 msgr2=0x7f011c0b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.440+0000 7f01398d9640 1 --2- 172.21.15.43:0/1416361633 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f011c0afcd0 0x7f011c0b21e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x561403997c60 tx=0x7f012c005f90 comp rx=0 tx=0).stop 2024-04-25T10:37:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.440+0000 7f01398d9640 1 -- 172.21.15.43:0/1416361633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01341031d0 msgr2=0x7f013419a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.440+0000 7f01398d9640 1 --2- 172.21.15.43:0/1416361633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01341031d0 0x7f013419a780 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f012402ade0 tx=0x7f0124031770 comp rx=0 tx=0).stop 2024-04-25T10:37:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.440+0000 7f01398d9640 1 -- 172.21.15.43:0/1416361633 shutdown_connections 2024-04-25T10:37:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.440+0000 7f01398d9640 1 --2- 172.21.15.43:0/1416361633 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f011c0afcd0 0x7f011c0b21e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.440+0000 7f01398d9640 1 --2- 172.21.15.43:0/1416361633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0134108a80 0x7f013419ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.440+0000 7f01398d9640 1 --2- 172.21.15.43:0/1416361633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0134108f20 0x7f013419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.440+0000 7f01398d9640 1 --2- 172.21.15.43:0/1416361633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01341031d0 0x7f013419a780 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.440+0000 7f01398d9640 1 -- 172.21.15.43:0/1416361633 >> 172.21.15.43:0/1416361633 conn(0x7f0134079170 msgr2=0x7f0134074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.440+0000 7f01398d9640 1 -- 172.21.15.43:0/1416361633 shutdown_connections 2024-04-25T10:37:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:46.440+0000 7f01398d9640 1 -- 172.21.15.43:0/1416361633 wait complete. 2024-04-25T10:37:47.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:46 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:37:47.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:46 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:37:47.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:46 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:37:47.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:46 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:37:47.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:46 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:37:47.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:46 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:37:47.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:46 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:37:47.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:46 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:37:47.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:46 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:37:48.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:47 smithi077 ceph-mon[32772]: pgmap v161: 97 pgs: 97 active+clean; 3.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-04-25T10:37:48.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:47 smithi164 ceph-mon[32713]: pgmap v161: 97 pgs: 97 active+clean; 3.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-04-25T10:37:48.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:47 smithi043 ceph-mon[29478]: pgmap v161: 97 pgs: 97 active+clean; 3.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-04-25T10:37:50.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:49 smithi077 ceph-mon[32772]: pgmap v162: 97 pgs: 97 active+clean; 3.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-04-25T10:37:50.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:49 smithi164 ceph-mon[32713]: pgmap v162: 97 pgs: 97 active+clean; 3.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-04-25T10:37:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:49 smithi043 ceph-mon[29478]: pgmap v162: 97 pgs: 97 active+clean; 3.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-04-25T10:37:52.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:51 smithi077 ceph-mon[32772]: pgmap v163: 97 pgs: 97 active+clean; 3.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-04-25T10:37:52.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:51 smithi164 ceph-mon[32713]: pgmap v163: 97 pgs: 97 active+clean; 3.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-04-25T10:37:52.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:51 smithi043 ceph-mon[29478]: pgmap v163: 97 pgs: 97 active+clean; 3.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-04-25T10:37:54.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:53 smithi077 ceph-mon[32772]: pgmap v164: 97 pgs: 97 active+clean; 3.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-04-25T10:37:54.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:53 smithi164 ceph-mon[32713]: pgmap v164: 97 pgs: 97 active+clean; 3.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-04-25T10:37:54.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:53 smithi043 ceph-mon[29478]: pgmap v164: 97 pgs: 97 active+clean; 3.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-04-25T10:37:56.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:55 smithi077 ceph-mon[32772]: pgmap v165: 97 pgs: 97 active+clean; 3.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 16 op/s 2024-04-25T10:37:56.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:55 smithi164 ceph-mon[32713]: pgmap v165: 97 pgs: 97 active+clean; 3.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 16 op/s 2024-04-25T10:37:56.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:55 smithi043 ceph-mon[29478]: pgmap v165: 97 pgs: 97 active+clean; 3.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 16 op/s 2024-04-25T10:37:58.163 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:57 smithi043 ceph-mon[29478]: pgmap v166: 97 pgs: 97 active+clean; 3.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T10:37:58.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:57 smithi077 ceph-mon[32772]: pgmap v166: 97 pgs: 97 active+clean; 3.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T10:37:58.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:57 smithi164 ceph-mon[32713]: pgmap v166: 97 pgs: 97 active+clean; 3.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T10:37:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.552+0000 7f0322b46640 1 -- 172.21.15.43:0/2024585325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f031c10b6d0 msgr2=0x7f031c10daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.552+0000 7f0322b46640 1 --2- 172.21.15.43:0/2024585325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f031c10b6d0 0x7f031c10daf0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f0310009900 tx=0x7f031002a410 comp rx=0 tx=0).stop 2024-04-25T10:37:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.552+0000 7f0322b46640 1 -- 172.21.15.43:0/2024585325 shutdown_connections 2024-04-25T10:37:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.552+0000 7f0322b46640 1 --2- 172.21.15.43:0/2024585325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f031c10e0c0 0x7f031c1105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.552+0000 7f0322b46640 1 --2- 172.21.15.43:0/2024585325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f031c10b6d0 0x7f031c10daf0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.552+0000 7f0322b46640 1 --2- 172.21.15.43:0/2024585325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f031c104c70 0x7f031c105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.552+0000 7f0322b46640 1 -- 172.21.15.43:0/2024585325 >> 172.21.15.43:0/2024585325 conn(0x7f031c0781b0 msgr2=0x7f031c0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.552+0000 7f0322b46640 1 -- 172.21.15.43:0/2024585325 shutdown_connections 2024-04-25T10:37:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.552+0000 7f0322b46640 1 -- 172.21.15.43:0/2024585325 wait complete. 2024-04-25T10:37:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.553+0000 7f0322b46640 1 Processor -- start 2024-04-25T10:37:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.553+0000 7f0322b46640 1 -- start start 2024-04-25T10:37:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.553+0000 7f0322b46640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f031c104c70 0x7f031c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.553+0000 7f0322b46640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f031c10b6d0 0x7f031c19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.553+0000 7f0322b46640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f031c10e0c0 0x7f031c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.553+0000 7f0322b46640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f031c113d70 con 0x7f031c10b6d0 2024-04-25T10:37:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.553+0000 7f0322b46640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f031c113bf0 con 0x7f031c10e0c0 2024-04-25T10:37:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.553+0000 7f0322b46640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f031c113ef0 con 0x7f031c104c70 2024-04-25T10:37:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.553+0000 7f031bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f031c10b6d0 0x7f031c19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.553+0000 7f031bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f031c10b6d0 0x7f031c19ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54766/0 (socket says 172.21.15.43:54766) 2024-04-25T10:37:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.553+0000 7f031bfff640 1 -- 172.21.15.43:0/381814888 learned_addr learned my addr 172.21.15.43:0/381814888 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.553+0000 7f03208e2640 1 --2- 172.21.15.43:0/381814888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f031c104c70 0x7f031c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.554+0000 7f03210e3640 1 --2- 172.21.15.43:0/381814888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f031c10e0c0 0x7f031c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.554+0000 7f031bfff640 1 -- 172.21.15.43:0/381814888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f031c104c70 msgr2=0x7f031c19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.554+0000 7f031bfff640 1 --2- 172.21.15.43:0/381814888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f031c104c70 0x7f031c19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.554+0000 7f031bfff640 1 -- 172.21.15.43:0/381814888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f031c10e0c0 msgr2=0x7f031c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.554+0000 7f031bfff640 1 --2- 172.21.15.43:0/381814888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f031c10e0c0 0x7f031c19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.554+0000 7f031bfff640 1 -- 172.21.15.43:0/381814888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031c19f570 con 0x7f031c10b6d0 2024-04-25T10:37:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.554+0000 7f031bfff640 1 --2- 172.21.15.43:0/381814888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f031c10b6d0 0x7f031c19ac70 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f0310005ab0 tx=0x7f0310003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.554+0000 7f0319ffb640 1 -- 172.21.15.43:0/381814888 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031003f070 con 0x7f031c10b6d0 2024-04-25T10:37:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.554+0000 7f0319ffb640 1 -- 172.21.15.43:0/381814888 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0310031b60 con 0x7f031c10b6d0 2024-04-25T10:37:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.554+0000 7f0319ffb640 1 -- 172.21.15.43:0/381814888 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310035030 con 0x7f031c10b6d0 2024-04-25T10:37:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.554+0000 7f0322b46640 1 -- 172.21.15.43:0/381814888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031c19f800 con 0x7f031c10b6d0 2024-04-25T10:37:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.554+0000 7f0322b46640 1 -- 172.21.15.43:0/381814888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f031c19fcf0 con 0x7f031c10b6d0 2024-04-25T10:37:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.557+0000 7f0319ffb640 1 -- 172.21.15.43:0/381814888 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f0310042070 con 0x7f031c10b6d0 2024-04-25T10:37:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.557+0000 7f03299f9740 1 Processor -- start 2024-04-25T10:37:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.557+0000 7f03299f9740 1 -- start start 2024-04-25T10:37:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.557+0000 7f03299f9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc731b0 0x5636ec012980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.557+0000 7f03299f9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636ebc4a2f0 0x5636ec012ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.557+0000 7f03299f9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc524d0 0x5636ec013400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.557+0000 7f03299f9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5636ebf79460 con 0x5636ebc524d0 2024-04-25T10:37:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.557+0000 7f03299f9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5636ebf33590 con 0x5636ebc4a2f0 2024-04-25T10:37:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.557+0000 7f03299f9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5636ebf75420 con 0x5636ebc731b0 2024-04-25T10:37:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.558+0000 7f031bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc524d0 0x5636ec013400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.558+0000 7f03210e3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc731b0 0x5636ec012980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.558+0000 7f0319ffb640 1 --2- 172.21.15.43:0/381814888 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f03040afc50 0x7f03040b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.558+0000 7f03210e3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc731b0 0x5636ec012980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:51790/0 (socket says 172.21.15.43:51790) 2024-04-25T10:37:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.558+0000 7f031bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc524d0 0x5636ec013400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54778/0 (socket says 172.21.15.43:54778) 2024-04-25T10:37:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.558+0000 7f03210e3640 1 -- 172.21.15.43:0/3651506360 learned_addr learned my addr 172.21.15.43:0/3651506360 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.558+0000 7f03208e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636ebc4a2f0 0x5636ec012ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.558+0000 7f03210e3640 1 --2- 172.21.15.43:0/381814888 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f03040afc50 0x7f03040b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.558+0000 7f03210e3640 1 -- 172.21.15.43:0/3651506360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636ebc4a2f0 msgr2=0x5636ec012ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.558+0000 7f03210e3640 1 --2- 172.21.15.43:0/3651506360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636ebc4a2f0 0x5636ec012ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.558+0000 7f03210e3640 1 -- 172.21.15.43:0/3651506360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc524d0 msgr2=0x5636ec013400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.558+0000 7f03210e3640 1 --2- 172.21.15.43:0/3651506360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc524d0 0x5636ec013400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.558+0000 7f0319ffb640 1 -- 172.21.15.43:0/381814888 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f03100ff700 con 0x7f031c10b6d0 2024-04-25T10:37:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.558+0000 7f03210e3640 1 -- 172.21.15.43:0/3651506360 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636ebf400d0 con 0x5636ebc731b0 2024-04-25T10:37:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.558+0000 7f03210e3640 1 --2- 172.21.15.43:0/3651506360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc731b0 0x5636ec012980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0314005c40 tx=0x7f0314013f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccb5f09f46bc4d6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.559+0000 7f0302ffd640 1 -- 172.21.15.43:0/3651506360 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314022020 con 0x5636ebc731b0 2024-04-25T10:37:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.559+0000 7f03210e3640 1 --2- 172.21.15.43:0/381814888 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f03040afc50 0x7f03040b2160 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0314023000 tx=0x7f0314021000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.559+0000 7f0302ffd640 1 -- 172.21.15.43:0/3651506360 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03140134e0 con 0x5636ebc731b0 2024-04-25T10:37:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.559+0000 7f0302ffd640 1 -- 172.21.15.43:0/3651506360 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314010ae0 con 0x5636ebc731b0 2024-04-25T10:37:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.559+0000 7f03299f9740 1 -- 172.21.15.43:0/3651506360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc731b0 msgr2=0x5636ec012980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.559+0000 7f03299f9740 1 --2- 172.21.15.43:0/3651506360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc731b0 0x5636ec012980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0314005c40 tx=0x7f0314013f50 comp rx=0 tx=0).stop 2024-04-25T10:37:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.559+0000 7f03299f9740 1 -- 172.21.15.43:0/3651506360 shutdown_connections 2024-04-25T10:37:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.559+0000 7f03299f9740 1 --2- 172.21.15.43:0/3651506360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636ebc4a2f0 0x5636ec012ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.559+0000 7f03299f9740 1 --2- 172.21.15.43:0/3651506360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc524d0 0x5636ec013400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.559+0000 7f03299f9740 1 --2- 172.21.15.43:0/3651506360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc731b0 0x5636ec012980 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.559+0000 7f03299f9740 1 -- 172.21.15.43:0/3651506360 >> 172.21.15.43:0/3651506360 conn(0x5636ebc67300 msgr2=0x5636ec00f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.559+0000 7f03299f9740 1 -- 172.21.15.43:0/3651506360 shutdown_connections 2024-04-25T10:37:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.560+0000 7f03299f9740 1 -- 172.21.15.43:0/3651506360 wait complete. 2024-04-25T10:37:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.560+0000 7f03299f9740 1 -- start start 2024-04-25T10:37:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.560+0000 7f03299f9740 1 Processor -- start 2024-04-25T10:37:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.560+0000 7f03299f9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc4a2f0 0x5636ec023090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.560+0000 7f03299f9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636ebc524d0 0x5636ec0235d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.560+0000 7f03299f9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc731b0 0x5636ec0214a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.560+0000 7f03299f9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5636ec024090 con 0x5636ebc4a2f0 2024-04-25T10:37:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.560+0000 7f03299f9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5636ebf00b60 con 0x5636ebc524d0 2024-04-25T10:37:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.560+0000 7f03299f9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5636ec024210 con 0x5636ebc731b0 2024-04-25T10:37:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.561+0000 7f03299f9740 10 client.0 _ll_register_callbacks cb 0x5636ebf252b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.561+0000 7f03208e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc4a2f0 0x5636ec023090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.561+0000 7f03208e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc4a2f0 0x5636ec023090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54788/0 (socket says 172.21.15.43:54788) 2024-04-25T10:37:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.561+0000 7f03208e2640 1 -- 172.21.15.43:0/1240706796 learned_addr learned my addr 172.21.15.43:0/1240706796 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.561+0000 7f03210e3640 1 --2- 172.21.15.43:0/1240706796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc731b0 0x5636ec0214a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.561+0000 7f031bfff640 1 --2- 172.21.15.43:0/1240706796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636ebc524d0 0x5636ec0235d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.561+0000 7f03208e2640 1 -- 172.21.15.43:0/1240706796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc731b0 msgr2=0x5636ec0214a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.561+0000 7f03208e2640 1 --2- 172.21.15.43:0/1240706796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc731b0 0x5636ec0214a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.561+0000 7f03208e2640 1 -- 172.21.15.43:0/1240706796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636ebc524d0 msgr2=0x5636ec0235d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.561+0000 7f03208e2640 1 --2- 172.21.15.43:0/1240706796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636ebc524d0 0x5636ec0235d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.561+0000 7f03208e2640 1 -- 172.21.15.43:0/1240706796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636ebc8b210 con 0x5636ebc4a2f0 2024-04-25T10:37:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.561+0000 7f031bfff640 1 --2- 172.21.15.43:0/1240706796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636ebc524d0 0x5636ec0235d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:37:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.561+0000 7f03208e2640 1 --2- 172.21.15.43:0/1240706796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc4a2f0 0x5636ec023090 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f030c012700 tx=0x7f030c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ed9be83b788524 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.561+0000 7f02fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.561+0000 7f02fbfff640 1 -- 172.21.15.43:0/1240706796 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c00f8c0 con 0x5636ebc4a2f0 2024-04-25T10:37:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.561+0000 7f02fbfff640 1 -- 172.21.15.43:0/1240706796 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f030c00fa60 con 0x5636ebc4a2f0 2024-04-25T10:37:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.561+0000 7f03299f9740 1 -- 172.21.15.43:0/1240706796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5636ebf87070 con 0x5636ebc4a2f0 2024-04-25T10:37:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.562+0000 7f02fbfff640 1 -- 172.21.15.43:0/1240706796 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f030c00fca0 con 0x5636ebc4a2f0 2024-04-25T10:37:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.562+0000 7f03299f9740 10 client.16998 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.562+0000 7f03299f9740 1 -- 172.21.15.43:0/1240706796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5636ebd57620 con 0x5636ebc4a2f0 2024-04-25T10:37:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.562+0000 7f02fbfff640 1 -- 172.21.15.43:0/1240706796 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5636ebf87070 con 0x5636ebc4a2f0 2024-04-25T10:37:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.562+0000 7f02fbfff640 1 -- 172.21.15.43:0/1240706796 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f030c010430 con 0x5636ebc4a2f0 2024-04-25T10:37:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.562+0000 7f03299f9740 10 client.16998 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.562+0000 7f03299f9740 10 client.16998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:37:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.562+0000 7f03299f9740 20 client.16998 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.562+0000 7f03299f9740 1 --2- 172.21.15.43:0/1240706796 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5636ebc31d80 0x5636ec033c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.562+0000 7f03299f9740 4 client.16998 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:37:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.562+0000 7f03299f9740 1 -- 172.21.15.43:0/1240706796 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636ebf80ee0 con 0x5636ebc31d80 2024-04-25T10:37:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.562+0000 7f031bfff640 1 --2- 172.21.15.43:0/1240706796 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5636ebc31d80 0x5636ec033c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.563+0000 7f02fbfff640 10 client.16998 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.563+0000 7f031bfff640 1 --2- 172.21.15.43:0/1240706796 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5636ebc31d80 0x5636ec033c70 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=452c033e8463391e server_cookie=511e79168ac6ddcf in_seq=0 out_seq=0 2024-04-25T10:37:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.563+0000 7f02fbfff640 1 -- 172.21.15.43:0/1240706796 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5636ebf80ee0 con 0x5636ebc31d80 2024-04-25T10:37:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.564+0000 7f02fbfff640 10 client.16998 handle_command_reply: tid=0 2024-04-25T10:37:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.564+0000 7f03299f9740 1 client.16998 shutdown 2024-04-25T10:37:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.564+0000 7f03299f9740 1 -- 172.21.15.43:0/1240706796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc4a2f0 msgr2=0x5636ec023090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.564+0000 7f03299f9740 1 --2- 172.21.15.43:0/1240706796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc4a2f0 0x5636ec023090 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f030c012700 tx=0x7f030c012bc0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.564+0000 7f03299f9740 1 -- 172.21.15.43:0/1240706796 shutdown_connections 2024-04-25T10:37:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.564+0000 7f03299f9740 1 --2- 172.21.15.43:0/1240706796 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5636ebc31d80 0x5636ec033c70 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.564+0000 7f03299f9740 1 --2- 172.21.15.43:0/1240706796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636ebc524d0 0x5636ec0235d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.564+0000 7f03299f9740 1 --2- 172.21.15.43:0/1240706796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc731b0 0x5636ec0214a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.564+0000 7f03299f9740 1 --2- 172.21.15.43:0/1240706796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc4a2f0 0x5636ec023090 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.564+0000 7f03299f9740 1 -- 172.21.15.43:0/1240706796 >> 172.21.15.43:0/1240706796 conn(0x5636ebc67300 msgr2=0x5636ec0147a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.564+0000 7f02fbfff640 0 client.16998 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.564+0000 7f03299f9740 1 -- 172.21.15.43:0/1240706796 shutdown_connections 2024-04-25T10:37:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.564+0000 7f03299f9740 1 -- 172.21.15.43:0/1240706796 wait complete. 2024-04-25T10:37:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.564+0000 7f03299f9740 20 client.16998 trim_cache size 0 max 16384 2024-04-25T10:37:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.579+0000 7f03299f9740 1 Processor -- start 2024-04-25T10:37:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.579+0000 7f03299f9740 1 -- start start 2024-04-25T10:37:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.580+0000 7f03299f9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc4a2f0 0x5636ec0231b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.580+0000 7f03299f9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636ebc524d0 0x5636ec01ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.580+0000 7f03299f9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc731b0 0x5636ec01b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.580+0000 7f03299f9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1b0070 con 0x5636ebc4a2f0 2024-04-25T10:37:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.580+0000 7f03299f9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1ad940 con 0x5636ebc524d0 2024-04-25T10:37:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.580+0000 7f03299f9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1ab7b0 con 0x5636ebc731b0 2024-04-25T10:37:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.580+0000 7f03208e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc4a2f0 0x5636ec0231b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.580+0000 7f03208e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc4a2f0 0x5636ec0231b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54790/0 (socket says 172.21.15.43:54790) 2024-04-25T10:37:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.580+0000 7f03208e2640 1 -- 172.21.15.43:0/1782275540 learned_addr learned my addr 172.21.15.43:0/1782275540 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.580+0000 7f03208e2640 1 -- 172.21.15.43:0/1782275540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc731b0 msgr2=0x5636ec01b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.580+0000 7f03208e2640 1 --2- 172.21.15.43:0/1782275540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc731b0 0x5636ec01b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.580+0000 7f03208e2640 1 -- 172.21.15.43:0/1782275540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636ebc524d0 msgr2=0x5636ec01ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.580+0000 7f03208e2640 1 --2- 172.21.15.43:0/1782275540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636ebc524d0 0x5636ec01ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.580+0000 7f03208e2640 1 -- 172.21.15.43:0/1782275540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636ec00aa60 con 0x5636ebc4a2f0 2024-04-25T10:37:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.580+0000 7f03208e2640 1 --2- 172.21.15.43:0/1782275540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc4a2f0 0x5636ec0231b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f030c010850 tx=0x7f030c00ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a086746f363d1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.580+0000 7f03027fc640 1 -- 172.21.15.43:0/1782275540 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c029070 con 0x5636ebc4a2f0 2024-04-25T10:37:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.581+0000 7f03027fc640 1 -- 172.21.15.43:0/1782275540 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f030c010430 con 0x5636ebc4a2f0 2024-04-25T10:37:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.581+0000 7f03027fc640 1 -- 172.21.15.43:0/1782275540 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c0035f0 con 0x5636ebc4a2f0 2024-04-25T10:37:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.581+0000 7f03299f9740 1 -- 172.21.15.43:0/1782275540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc4a2f0 msgr2=0x5636ec0231b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.581+0000 7f03299f9740 1 --2- 172.21.15.43:0/1782275540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc4a2f0 0x5636ec0231b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f030c010850 tx=0x7f030c00ff40 comp rx=0 tx=0).stop 2024-04-25T10:37:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.581+0000 7f03299f9740 1 -- 172.21.15.43:0/1782275540 shutdown_connections 2024-04-25T10:37:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.581+0000 7f03299f9740 1 --2- 172.21.15.43:0/1782275540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636ebc524d0 0x5636ec01ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.581+0000 7f03299f9740 1 --2- 172.21.15.43:0/1782275540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc731b0 0x5636ec01b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.581+0000 7f03299f9740 1 --2- 172.21.15.43:0/1782275540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc4a2f0 0x5636ec0231b0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.581+0000 7f03299f9740 1 -- 172.21.15.43:0/1782275540 >> 172.21.15.43:0/1782275540 conn(0x5636ebc67300 msgr2=0x5636ec022aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.581+0000 7f03299f9740 1 -- 172.21.15.43:0/1782275540 shutdown_connections 2024-04-25T10:37:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.581+0000 7f03299f9740 1 -- 172.21.15.43:0/1782275540 wait complete. 2024-04-25T10:37:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.582+0000 7f03299f9740 1 -- start start 2024-04-25T10:37:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.582+0000 7f03299f9740 1 Processor -- start 2024-04-25T10:37:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.582+0000 7f03299f9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc4a2f0 0x5636ec0261d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.582+0000 7f03299f9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc524d0 0x5636ec026710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.582+0000 7f03299f9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636ebc731b0 0x5636ec026c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.582+0000 7f03299f9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1abe00 con 0x5636ebc524d0 2024-04-25T10:37:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.582+0000 7f03299f9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1ad270 con 0x5636ebc731b0 2024-04-25T10:37:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.582+0000 7f03299f9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1ac4d0 con 0x5636ebc4a2f0 2024-04-25T10:37:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.582+0000 7f03299f9740 10 client.0 _ll_register_callbacks cb 0x7f030c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:37:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.582+0000 7f031bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc524d0 0x5636ec026710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.582+0000 7f031bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc524d0 0x5636ec026710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54798/0 (socket says 172.21.15.43:54798) 2024-04-25T10:37:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.582+0000 7f031bfff640 1 -- 172.21.15.43:0/2332110575 learned_addr learned my addr 172.21.15.43:0/2332110575 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:37:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.582+0000 7f03210e3640 1 --2- 172.21.15.43:0/2332110575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636ebc731b0 0x5636ec026c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.583+0000 7f03208e2640 1 --2- 172.21.15.43:0/2332110575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc4a2f0 0x5636ec0261d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.583+0000 7f031bfff640 1 -- 172.21.15.43:0/2332110575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc4a2f0 msgr2=0x5636ec0261d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.583+0000 7f031bfff640 1 --2- 172.21.15.43:0/2332110575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc4a2f0 0x5636ec0261d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.583+0000 7f031bfff640 1 -- 172.21.15.43:0/2332110575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636ebc731b0 msgr2=0x5636ec026c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:37:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.583+0000 7f031bfff640 1 --2- 172.21.15.43:0/2332110575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636ebc731b0 0x5636ec026c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.583+0000 7f031bfff640 1 -- 172.21.15.43:0/2332110575 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636ec09a150 con 0x5636ebc524d0 2024-04-25T10:37:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.583+0000 7f031bfff640 1 --2- 172.21.15.43:0/2332110575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc524d0 0x5636ec026710 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0310000c00 tx=0x7f031002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07056cd7ce70213 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:37:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.583+0000 7f02fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:37:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.583+0000 7f02fbfff640 1 -- 172.21.15.43:0/2332110575 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310049040 con 0x5636ebc524d0 2024-04-25T10:37:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.583+0000 7f02fbfff640 1 -- 172.21.15.43:0/2332110575 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f03100fd100 con 0x5636ebc524d0 2024-04-25T10:37:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.583+0000 7f03299f9740 1 -- 172.21.15.43:0/2332110575 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f030c010430 con 0x5636ebc524d0 2024-04-25T10:37:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.583+0000 7f02fbfff640 1 -- 172.21.15.43:0/2332110575 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f03100c7490 con 0x5636ebc524d0 2024-04-25T10:37:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.583+0000 7f03299f9740 10 client.17004 fetch_fsmap learned FSMap version 26 2024-04-25T10:37:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.583+0000 7f03299f9740 1 -- 172.21.15.43:0/2332110575 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5636ec09b3a0 con 0x5636ebc524d0 2024-04-25T10:37:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.583+0000 7f02fbfff640 1 -- 172.21.15.43:0/2332110575 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f030c010430 con 0x5636ebc524d0 2024-04-25T10:37:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.584+0000 7f02fbfff640 1 -- 172.21.15.43:0/2332110575 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f031003b460 con 0x5636ebc524d0 2024-04-25T10:37:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.584+0000 7f03299f9740 10 client.17004 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:37:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.584+0000 7f03299f9740 10 client.17004 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:37:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.584+0000 7f03299f9740 20 client.17004 populate_metadata read hostname 'smithi043' 2024-04-25T10:37:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.584+0000 7f03299f9740 1 --2- 172.21.15.43:0/2332110575 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5636ebc071c0 0x5636ec09da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:37:59.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.584+0000 7f03299f9740 4 client.17004 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:37:59.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.584+0000 7f03299f9740 1 -- 172.21.15.43:0/2332110575 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5636ebefc5f0 con 0x5636ebc071c0 2024-04-25T10:37:59.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.584+0000 7f03208e2640 1 --2- 172.21.15.43:0/2332110575 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5636ebc071c0 0x5636ec09da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:37:59.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.584+0000 7f02fbfff640 10 client.17004 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:59.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.584+0000 7f03208e2640 1 --2- 172.21.15.43:0/2332110575 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5636ebc071c0 0x5636ec09da90 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c11d8d63f1e81fd6 server_cookie=79ffddaf3268a89e in_seq=0 out_seq=0 2024-04-25T10:37:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.600+0000 7f02fbfff640 1 -- 172.21.15.43:0/2332110575 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5636ebefc5f0 con 0x5636ebc071c0 2024-04-25T10:37:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.600+0000 7f02fbfff640 10 client.17004 handle_command_reply: tid=0 2024-04-25T10:37:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.600+0000 7f03299f9740 1 client.17004 shutdown 2024-04-25T10:37:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.601+0000 7f03299f9740 1 -- 172.21.15.43:0/2332110575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc524d0 msgr2=0x5636ec026710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.601+0000 7f03299f9740 1 --2- 172.21.15.43:0/2332110575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc524d0 0x5636ec026710 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0310000c00 tx=0x7f031002a410 comp rx=0 tx=0).stop 2024-04-25T10:37:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.601+0000 7f03299f9740 1 -- 172.21.15.43:0/2332110575 shutdown_connections 2024-04-25T10:37:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.601+0000 7f03299f9740 1 --2- 172.21.15.43:0/2332110575 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5636ebc071c0 0x5636ec09da90 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.601+0000 7f03299f9740 1 --2- 172.21.15.43:0/2332110575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636ebc731b0 0x5636ec026c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.601+0000 7f03299f9740 1 --2- 172.21.15.43:0/2332110575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636ebc524d0 0x5636ec026710 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.601+0000 7f02fbfff640 0 client.17004 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:37:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.601+0000 7f03299f9740 1 --2- 172.21.15.43:0/2332110575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636ebc4a2f0 0x5636ec0261d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.601+0000 7f03299f9740 1 -- 172.21.15.43:0/2332110575 >> 172.21.15.43:0/2332110575 conn(0x5636ebc67300 msgr2=0x5636ec0209e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.601+0000 7f03299f9740 1 -- 172.21.15.43:0/2332110575 shutdown_connections 2024-04-25T10:37:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.601+0000 7f03299f9740 1 -- 172.21.15.43:0/2332110575 wait complete. 2024-04-25T10:37:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.601+0000 7f03299f9740 20 client.17004 trim_cache size 0 max 16384 2024-04-25T10:37:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:37:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:37:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:37:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:37:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.602+0000 7f0322b46640 1 -- 172.21.15.43:0/381814888 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f03040afc50 msgr2=0x7f03040b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.602+0000 7f0322b46640 1 --2- 172.21.15.43:0/381814888 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f03040afc50 0x7f03040b2160 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0314023000 tx=0x7f0314021000 comp rx=0 tx=0).stop 2024-04-25T10:37:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.602+0000 7f0322b46640 1 -- 172.21.15.43:0/381814888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f031c10b6d0 msgr2=0x7f031c19ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:37:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.602+0000 7f0322b46640 1 --2- 172.21.15.43:0/381814888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f031c10b6d0 0x7f031c19ac70 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f0310005ab0 tx=0x7f0310003c10 comp rx=0 tx=0).stop 2024-04-25T10:37:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.602+0000 7f0322b46640 1 -- 172.21.15.43:0/381814888 shutdown_connections 2024-04-25T10:37:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.602+0000 7f0322b46640 1 --2- 172.21.15.43:0/381814888 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f03040afc50 0x7f03040b2160 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.602+0000 7f0322b46640 1 --2- 172.21.15.43:0/381814888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f031c10e0c0 0x7f031c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.602+0000 7f0322b46640 1 --2- 172.21.15.43:0/381814888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f031c10b6d0 0x7f031c19ac70 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.602+0000 7f0322b46640 1 --2- 172.21.15.43:0/381814888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f031c104c70 0x7f031c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:37:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.602+0000 7f0322b46640 1 -- 172.21.15.43:0/381814888 >> 172.21.15.43:0/381814888 conn(0x7f031c0781b0 msgr2=0x7f031c101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:37:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.602+0000 7f0322b46640 1 -- 172.21.15.43:0/381814888 shutdown_connections 2024-04-25T10:37:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:37:59.602+0000 7f0322b46640 1 -- 172.21.15.43:0/381814888 wait complete. 2024-04-25T10:38:00.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:37:59 smithi077 ceph-mon[32772]: pgmap v167: 97 pgs: 97 active+clean; 3.4 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 16 op/s 2024-04-25T10:38:00.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:37:59 smithi164 ceph-mon[32713]: pgmap v167: 97 pgs: 97 active+clean; 3.4 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 16 op/s 2024-04-25T10:38:00.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:37:59 smithi043 ceph-mon[29478]: pgmap v167: 97 pgs: 97 active+clean; 3.4 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 16 op/s 2024-04-25T10:38:01.743 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:38:01.744 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:38:01.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.871+0000 7fc74aea7640 1 -- 172.21.15.43:0/2361308366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc74410e0d0 msgr2=0x7fc744110570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:01.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.871+0000 7fc74aea7640 1 --2- 172.21.15.43:0/2361308366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc74410e0d0 0x7fc744110570 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009900 tx=0x7fc72c02a410 comp rx=0 tx=0).stop 2024-04-25T10:38:01.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.871+0000 7fc74aea7640 1 -- 172.21.15.43:0/2361308366 shutdown_connections 2024-04-25T10:38:01.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.871+0000 7fc74aea7640 1 --2- 172.21.15.43:0/2361308366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc74410e0d0 0x7fc744110570 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.871+0000 7fc74aea7640 1 --2- 172.21.15.43:0/2361308366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc74410b770 0x7fc74410db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.871+0000 7fc74aea7640 1 --2- 172.21.15.43:0/2361308366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc744104d10 0x7fc744105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.871+0000 7fc74aea7640 1 -- 172.21.15.43:0/2361308366 >> 172.21.15.43:0/2361308366 conn(0x7fc744078110 msgr2=0x7fc744078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:01.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.871+0000 7fc74aea7640 1 -- 172.21.15.43:0/2361308366 shutdown_connections 2024-04-25T10:38:01.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.871+0000 7fc74aea7640 1 -- 172.21.15.43:0/2361308366 wait complete. 2024-04-25T10:38:01.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.871+0000 7fc74aea7640 1 Processor -- start 2024-04-25T10:38:01.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.872+0000 7fc74aea7640 1 -- start start 2024-04-25T10:38:01.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.872+0000 7fc74aea7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc744104d10 0x7fc74419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:01.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.872+0000 7fc74aea7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc74410b770 0x7fc74419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:01.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.872+0000 7fc74aea7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc74410e0d0 0x7fc74419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:01.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.872+0000 7fc74aea7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc744113ce0 con 0x7fc744104d10 2024-04-25T10:38:01.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.872+0000 7fc74aea7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc744113b60 con 0x7fc74410b770 2024-04-25T10:38:01.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.872+0000 7fc74aea7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc744113e60 con 0x7fc74410e0d0 2024-04-25T10:38:01.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.872+0000 7fc748c43640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc744104d10 0x7fc74419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:01.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.872+0000 7fc748c43640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc744104d10 0x7fc74419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54826/0 (socket says 172.21.15.43:54826) 2024-04-25T10:38:01.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.872+0000 7fc748c43640 1 -- 172.21.15.43:0/3063259755 learned_addr learned my addr 172.21.15.43:0/3063259755 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:01.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.872+0000 7fc749444640 1 --2- 172.21.15.43:0/3063259755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc74410e0d0 0x7fc74419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:01.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.872+0000 7fc743fff640 1 --2- 172.21.15.43:0/3063259755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc74410b770 0x7fc74419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:01.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.872+0000 7fc748c43640 1 -- 172.21.15.43:0/3063259755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc74410e0d0 msgr2=0x7fc74419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:01.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.872+0000 7fc748c43640 1 --2- 172.21.15.43:0/3063259755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc74410e0d0 0x7fc74419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.872+0000 7fc748c43640 1 -- 172.21.15.43:0/3063259755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc74410b770 msgr2=0x7fc74419ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:01.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.872+0000 7fc748c43640 1 --2- 172.21.15.43:0/3063259755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc74410b770 0x7fc74419ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.872+0000 7fc748c43640 1 -- 172.21.15.43:0/3063259755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc74419f500 con 0x7fc744104d10 2024-04-25T10:38:01.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.873+0000 7fc748c43640 1 --2- 172.21.15.43:0/3063259755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc744104d10 0x7fc74419a720 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc73c00c950 tx=0x7fc73c00ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:01.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.873+0000 7fc741ffb640 1 -- 172.21.15.43:0/3063259755 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c007c70 con 0x7fc744104d10 2024-04-25T10:38:01.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.873+0000 7fc74aea7640 1 -- 172.21.15.43:0/3063259755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc74419f7f0 con 0x7fc744104d10 2024-04-25T10:38:01.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.873+0000 7fc741ffb640 1 -- 172.21.15.43:0/3063259755 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc73c007e10 con 0x7fc744104d10 2024-04-25T10:38:01.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.873+0000 7fc741ffb640 1 -- 172.21.15.43:0/3063259755 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c018800 con 0x7fc744104d10 2024-04-25T10:38:01.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.873+0000 7fc74aea7640 1 -- 172.21.15.43:0/3063259755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7441a71f0 con 0x7fc744104d10 2024-04-25T10:38:01.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.876+0000 7fc751e31740 1 Processor -- start 2024-04-25T10:38:01.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.876+0000 7fc751e31740 1 -- start start 2024-04-25T10:38:01.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.876+0000 7fc741ffb640 1 -- 172.21.15.43:0/3063259755 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fc73c0189a0 con 0x7fc744104d10 2024-04-25T10:38:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.876+0000 7fc751e31740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f12b390 0x56420f4cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.876+0000 7fc751e31740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f1024d0 0x56420f4cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.876+0000 7fc751e31740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f10a6b0 0x56420f4d88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.876+0000 7fc751e31740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56420f431d80 con 0x56420f1024d0 2024-04-25T10:38:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.876+0000 7fc751e31740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56420f3eb8b0 con 0x56420f12b390 2024-04-25T10:38:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.876+0000 7fc751e31740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56420f42dd40 con 0x56420f10a6b0 2024-04-25T10:38:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.876+0000 7fc748c43640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f1024d0 0x56420f4cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.876+0000 7fc749444640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f12b390 0x56420f4cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.876+0000 7fc748c43640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f1024d0 0x56420f4cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54828/0 (socket says 172.21.15.43:54828) 2024-04-25T10:38:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.876+0000 7fc748c43640 1 -- 172.21.15.43:0/3290150325 learned_addr learned my addr 172.21.15.43:0/3290150325 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.876+0000 7fc743fff640 1 --2- 172.21.15.43:0/3290150325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f10a6b0 0x56420f4d88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.876+0000 7fc741ffb640 1 --2- 172.21.15.43:0/3063259755 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc7300afc60 0x7fc7300b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.877+0000 7fc748c43640 1 -- 172.21.15.43:0/3290150325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f10a6b0 msgr2=0x56420f4d88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.877+0000 7fc748c43640 1 --2- 172.21.15.43:0/3290150325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f10a6b0 0x56420f4d88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.877+0000 7fc748c43640 1 -- 172.21.15.43:0/3290150325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f12b390 msgr2=0x56420f4cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:01.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.877+0000 7fc748c43640 1 --2- 172.21.15.43:0/3290150325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f12b390 0x56420f4cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.877+0000 7fc748c43640 1 -- 172.21.15.43:0/3290150325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56420f3bca40 con 0x56420f1024d0 2024-04-25T10:38:01.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.877+0000 7fc749444640 1 --2- 172.21.15.43:0/3063259755 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc7300afc60 0x7fc7300b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:01.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.877+0000 7fc741ffb640 1 -- 172.21.15.43:0/3063259755 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc73c0df210 con 0x7fc744104d10 2024-04-25T10:38:01.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.877+0000 7fc748c43640 1 --2- 172.21.15.43:0/3290150325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f1024d0 0x56420f4cbfa0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0dc930 tx=0x7fc73c06d800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f25665d846aaeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:01.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.877+0000 7fc73affd640 1 -- 172.21.15.43:0/3290150325 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c0a6d30 con 0x56420f1024d0 2024-04-25T10:38:01.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.877+0000 7fc749444640 1 --2- 172.21.15.43:0/3063259755 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc7300afc60 0x7fc7300b2170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc72c005ab0 tx=0x7fc72c009260 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:01.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.877+0000 7fc73affd640 1 -- 172.21.15.43:0/3290150325 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc73c0a6ef0 con 0x56420f1024d0 2024-04-25T10:38:01.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.877+0000 7fc73affd640 1 -- 172.21.15.43:0/3290150325 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c0a6b50 con 0x56420f1024d0 2024-04-25T10:38:01.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.878+0000 7fc751e31740 1 -- 172.21.15.43:0/3290150325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f1024d0 msgr2=0x56420f4cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:01.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.878+0000 7fc751e31740 1 --2- 172.21.15.43:0/3290150325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f1024d0 0x56420f4cbfa0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0dc930 tx=0x7fc73c06d800 comp rx=0 tx=0).stop 2024-04-25T10:38:01.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.878+0000 7fc751e31740 1 -- 172.21.15.43:0/3290150325 shutdown_connections 2024-04-25T10:38:01.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.878+0000 7fc751e31740 1 --2- 172.21.15.43:0/3290150325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f10a6b0 0x56420f4d88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.878+0000 7fc751e31740 1 --2- 172.21.15.43:0/3290150325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f1024d0 0x56420f4cbfa0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.878+0000 7fc751e31740 1 --2- 172.21.15.43:0/3290150325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f12b390 0x56420f4cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.878+0000 7fc751e31740 1 -- 172.21.15.43:0/3290150325 >> 172.21.15.43:0/3290150325 conn(0x56420f11f4e0 msgr2=0x56420f4c8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.878+0000 7fc751e31740 1 -- 172.21.15.43:0/3290150325 shutdown_connections 2024-04-25T10:38:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.878+0000 7fc751e31740 1 -- 172.21.15.43:0/3290150325 wait complete. 2024-04-25T10:38:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.879+0000 7fc751e31740 1 -- start start 2024-04-25T10:38:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.879+0000 7fc751e31740 1 Processor -- start 2024-04-25T10:38:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.879+0000 7fc751e31740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f1024d0 0x56420f4d6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.879+0000 7fc751e31740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f10a6b0 0x56420f4d6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.879+0000 7fc751e31740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f12b390 0x56420f4e23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.879+0000 7fc751e31740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56420f4d8f20 con 0x56420f12b390 2024-04-25T10:38:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.879+0000 7fc751e31740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56420f3b8d10 con 0x56420f10a6b0 2024-04-25T10:38:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.879+0000 7fc751e31740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56420f4d90a0 con 0x56420f1024d0 2024-04-25T10:38:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.879+0000 7fc751e31740 10 client.0 _ll_register_callbacks cb 0x56420f3e5210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.879+0000 7fc749444640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f12b390 0x56420f4e23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.879+0000 7fc749444640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f12b390 0x56420f4e23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54840/0 (socket says 172.21.15.43:54840) 2024-04-25T10:38:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.879+0000 7fc749444640 1 -- 172.21.15.43:0/2940542913 learned_addr learned my addr 172.21.15.43:0/2940542913 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.879+0000 7fc748c43640 1 --2- 172.21.15.43:0/2940542913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f1024d0 0x56420f4d6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.880+0000 7fc743fff640 1 --2- 172.21.15.43:0/2940542913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f10a6b0 0x56420f4d6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.880+0000 7fc749444640 1 -- 172.21.15.43:0/2940542913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f1024d0 msgr2=0x56420f4d6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.880+0000 7fc749444640 1 --2- 172.21.15.43:0/2940542913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f1024d0 0x56420f4d6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.880+0000 7fc749444640 1 -- 172.21.15.43:0/2940542913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f10a6b0 msgr2=0x56420f4d6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.880+0000 7fc749444640 1 --2- 172.21.15.43:0/2940542913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f10a6b0 0x56420f4d6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.880+0000 7fc749444640 1 -- 172.21.15.43:0/2940542913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56420f20d7d0 con 0x56420f12b390 2024-04-25T10:38:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.880+0000 7fc749444640 1 --2- 172.21.15.43:0/2940542913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f12b390 0x56420f4e23a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc72c033000 tx=0x7fc72c009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0a1c1ddd50ce3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.880+0000 7fc723fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.880+0000 7fc723fff640 1 -- 172.21.15.43:0/2940542913 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c035040 con 0x56420f12b390 2024-04-25T10:38:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.880+0000 7fc723fff640 1 -- 172.21.15.43:0/2940542913 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc72c003a00 con 0x56420f12b390 2024-04-25T10:38:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.880+0000 7fc751e31740 1 -- 172.21.15.43:0/2940542913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc73c0a6ef0 con 0x56420f12b390 2024-04-25T10:38:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.880+0000 7fc723fff640 1 -- 172.21.15.43:0/2940542913 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc72c057050 con 0x56420f12b390 2024-04-25T10:38:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.880+0000 7fc751e31740 10 client.17016 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.880+0000 7fc751e31740 1 -- 172.21.15.43:0/2940542913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56420f1433f0 con 0x56420f12b390 2024-04-25T10:38:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.881+0000 7fc723fff640 1 -- 172.21.15.43:0/2940542913 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc73c0a6ef0 con 0x56420f12b390 2024-04-25T10:38:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.881+0000 7fc723fff640 1 -- 172.21.15.43:0/2940542913 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc72c049b70 con 0x56420f12b390 2024-04-25T10:38:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.881+0000 7fc751e31740 10 client.17016 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.881+0000 7fc751e31740 10 client.17016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.881+0000 7fc751e31740 20 client.17016 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.881+0000 7fc751e31740 1 --2- 172.21.15.43:0/2940542913 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56420f0e9f60 0x56420f4e85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.881+0000 7fc751e31740 4 client.17016 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:38:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.881+0000 7fc751e31740 1 -- 172.21.15.43:0/2940542913 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56420f43f990 con 0x56420f0e9f60 2024-04-25T10:38:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.881+0000 7fc748c43640 1 --2- 172.21.15.43:0/2940542913 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56420f0e9f60 0x56420f4e85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.882+0000 7fc748c43640 1 --2- 172.21.15.43:0/2940542913 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56420f0e9f60 0x56420f4e85f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d8574d8335fa48d server_cookie=69437ab2754c8351 in_seq=0 out_seq=0 2024-04-25T10:38:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.882+0000 7fc723fff640 10 client.17016 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:01.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.882+0000 7fc723fff640 1 -- 172.21.15.43:0/2940542913 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56420f43f990 con 0x56420f0e9f60 2024-04-25T10:38:01.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.882+0000 7fc723fff640 10 client.17016 handle_command_reply: tid=0 2024-04-25T10:38:01.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.882+0000 7fc751e31740 1 client.17016 shutdown 2024-04-25T10:38:01.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.882+0000 7fc751e31740 1 -- 172.21.15.43:0/2940542913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f12b390 msgr2=0x56420f4e23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:01.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.882+0000 7fc751e31740 1 --2- 172.21.15.43:0/2940542913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f12b390 0x56420f4e23a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc72c033000 tx=0x7fc72c009640 comp rx=0 tx=0).stop 2024-04-25T10:38:01.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.883+0000 7fc751e31740 1 -- 172.21.15.43:0/2940542913 shutdown_connections 2024-04-25T10:38:01.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.883+0000 7fc751e31740 1 --2- 172.21.15.43:0/2940542913 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56420f0e9f60 0x56420f4e85f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.883+0000 7fc751e31740 1 --2- 172.21.15.43:0/2940542913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f10a6b0 0x56420f4d6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.883+0000 7fc751e31740 1 --2- 172.21.15.43:0/2940542913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f12b390 0x56420f4e23a0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.883+0000 7fc751e31740 1 --2- 172.21.15.43:0/2940542913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f1024d0 0x56420f4d6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.883+0000 7fc723fff640 0 client.17016 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:01.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.883+0000 7fc751e31740 1 -- 172.21.15.43:0/2940542913 >> 172.21.15.43:0/2940542913 conn(0x56420f11f4e0 msgr2=0x56420f4cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:01.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.883+0000 7fc751e31740 1 -- 172.21.15.43:0/2940542913 shutdown_connections 2024-04-25T10:38:01.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.883+0000 7fc751e31740 1 -- 172.21.15.43:0/2940542913 wait complete. 2024-04-25T10:38:01.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.883+0000 7fc751e31740 20 client.17016 trim_cache size 0 max 16384 2024-04-25T10:38:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.901+0000 7fc751e31740 1 Processor -- start 2024-04-25T10:38:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.901+0000 7fc751e31740 1 -- start start 2024-04-25T10:38:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.902+0000 7fc751e31740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f11f4e0 0x56420f4d5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.902+0000 7fc751e31740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f1024d0 0x56420f4de370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.902+0000 7fc751e31740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f10a6b0 0x56420f4de8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.902+0000 7fc751e31740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc7441b2240 con 0x56420f11f4e0 2024-04-25T10:38:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.902+0000 7fc751e31740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc7441afb10 con 0x56420f1024d0 2024-04-25T10:38:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.902+0000 7fc751e31740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc7441ad960 con 0x56420f10a6b0 2024-04-25T10:38:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.902+0000 7fc748c43640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f11f4e0 0x56420f4d5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.902+0000 7fc748c43640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f11f4e0 0x56420f4d5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54856/0 (socket says 172.21.15.43:54856) 2024-04-25T10:38:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.902+0000 7fc748c43640 1 -- 172.21.15.43:0/2541042823 learned_addr learned my addr 172.21.15.43:0/2541042823 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.902+0000 7fc743fff640 1 --2- 172.21.15.43:0/2541042823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f1024d0 0x56420f4de370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.902+0000 7fc749444640 1 --2- 172.21.15.43:0/2541042823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f10a6b0 0x56420f4de8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.902+0000 7fc748c43640 1 -- 172.21.15.43:0/2541042823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f10a6b0 msgr2=0x56420f4de8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.902+0000 7fc748c43640 1 --2- 172.21.15.43:0/2541042823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f10a6b0 0x56420f4de8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.902+0000 7fc748c43640 1 -- 172.21.15.43:0/2541042823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f1024d0 msgr2=0x56420f4de370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.902+0000 7fc748c43640 1 --2- 172.21.15.43:0/2541042823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f1024d0 0x56420f4de370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.902+0000 7fc748c43640 1 -- 172.21.15.43:0/2541042823 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56420f1030f0 con 0x56420f11f4e0 2024-04-25T10:38:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.902+0000 7fc749444640 1 --2- 172.21.15.43:0/2541042823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f10a6b0 0x56420f4de8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:38:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.902+0000 7fc748c43640 1 --2- 172.21.15.43:0/2541042823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f11f4e0 0x56420f4d5310 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0dcf90 tx=0x7fc73c00fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5327fb9c7f512ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.903+0000 7fc73a7fc640 1 -- 172.21.15.43:0/2541042823 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c0a6d30 con 0x56420f11f4e0 2024-04-25T10:38:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.903+0000 7fc73a7fc640 1 -- 172.21.15.43:0/2541042823 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc73c0a6ef0 con 0x56420f11f4e0 2024-04-25T10:38:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.903+0000 7fc73a7fc640 1 -- 172.21.15.43:0/2541042823 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c0ab790 con 0x56420f11f4e0 2024-04-25T10:38:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.903+0000 7fc751e31740 1 -- 172.21.15.43:0/2541042823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f11f4e0 msgr2=0x56420f4d5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.903+0000 7fc751e31740 1 --2- 172.21.15.43:0/2541042823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f11f4e0 0x56420f4d5310 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0dcf90 tx=0x7fc73c00fef0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.904+0000 7fc751e31740 1 -- 172.21.15.43:0/2541042823 shutdown_connections 2024-04-25T10:38:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.904+0000 7fc751e31740 1 --2- 172.21.15.43:0/2541042823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f1024d0 0x56420f4de370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.904+0000 7fc751e31740 1 --2- 172.21.15.43:0/2541042823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f10a6b0 0x56420f4de8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.904+0000 7fc751e31740 1 --2- 172.21.15.43:0/2541042823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f11f4e0 0x56420f4d5310 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.904+0000 7fc751e31740 1 -- 172.21.15.43:0/2541042823 >> 172.21.15.43:0/2541042823 conn(0x56420f4cbfc0 msgr2=0x56420f4d4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.904+0000 7fc751e31740 1 -- 172.21.15.43:0/2541042823 shutdown_connections 2024-04-25T10:38:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.904+0000 7fc751e31740 1 -- 172.21.15.43:0/2541042823 wait complete. 2024-04-25T10:38:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.904+0000 7fc751e31740 1 -- start start 2024-04-25T10:38:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.904+0000 7fc751e31740 1 Processor -- start 2024-04-25T10:38:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc751e31740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f1024d0 0x56420f4da170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc751e31740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f10a6b0 0x56420f4da6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc751e31740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f11f4e0 0x56420f4dabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc751e31740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc7441adfd0 con 0x56420f11f4e0 2024-04-25T10:38:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc751e31740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc7441af440 con 0x56420f10a6b0 2024-04-25T10:38:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc751e31740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc7441ae6a0 con 0x56420f1024d0 2024-04-25T10:38:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc751e31740 10 client.0 _ll_register_callbacks cb 0x7fc72c005e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc749444640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f11f4e0 0x56420f4dabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc748c43640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f1024d0 0x56420f4da170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc749444640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f11f4e0 0x56420f4dabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54858/0 (socket says 172.21.15.43:54858) 2024-04-25T10:38:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc749444640 1 -- 172.21.15.43:0/3688251249 learned_addr learned my addr 172.21.15.43:0/3688251249 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc743fff640 1 --2- 172.21.15.43:0/3688251249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f10a6b0 0x56420f4da6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc749444640 1 -- 172.21.15.43:0/3688251249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f1024d0 msgr2=0x56420f4da170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc749444640 1 --2- 172.21.15.43:0/3688251249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f1024d0 0x56420f4da170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc749444640 1 -- 172.21.15.43:0/3688251249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f10a6b0 msgr2=0x56420f4da6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc749444640 1 --2- 172.21.15.43:0/3688251249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f10a6b0 0x56420f4da6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc749444640 1 -- 172.21.15.43:0/3688251249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56420f3f24b0 con 0x56420f11f4e0 2024-04-25T10:38:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc743fff640 1 --2- 172.21.15.43:0/3688251249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f10a6b0 0x56420f4da6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:38:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc749444640 1 --2- 172.21.15.43:0/3688251249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f11f4e0 0x56420f4dabf0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc72c03d4a0 tx=0x7fc72c03d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51e833ad7fb94f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc723fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.905+0000 7fc723fff640 1 -- 172.21.15.43:0/3688251249 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c035070 con 0x56420f11f4e0 2024-04-25T10:38:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.906+0000 7fc723fff640 1 -- 172.21.15.43:0/3688251249 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc72c003800 con 0x56420f11f4e0 2024-04-25T10:38:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.906+0000 7fc751e31740 1 -- 172.21.15.43:0/3688251249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc73c0a6ef0 con 0x56420f11f4e0 2024-04-25T10:38:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.906+0000 7fc723fff640 1 -- 172.21.15.43:0/3688251249 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc72c06b050 con 0x56420f11f4e0 2024-04-25T10:38:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.906+0000 7fc751e31740 10 client.17022 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.906+0000 7fc751e31740 1 -- 172.21.15.43:0/3688251249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56420f550b20 con 0x56420f11f4e0 2024-04-25T10:38:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.906+0000 7fc723fff640 1 -- 172.21.15.43:0/3688251249 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc73c0a6ef0 con 0x56420f11f4e0 2024-04-25T10:38:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.906+0000 7fc723fff640 1 -- 172.21.15.43:0/3688251249 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc72c049810 con 0x56420f11f4e0 2024-04-25T10:38:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.906+0000 7fc751e31740 10 client.17022 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.906+0000 7fc751e31740 10 client.17022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.906+0000 7fc751e31740 20 client.17022 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.906+0000 7fc751e31740 1 --2- 172.21.15.43:0/3688251249 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56420f12b390 0x56420f5532e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.906+0000 7fc751e31740 4 client.17022 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:38:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.906+0000 7fc751e31740 1 -- 172.21.15.43:0/3688251249 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56420f439800 con 0x56420f12b390 2024-04-25T10:38:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.906+0000 7fc748c43640 1 --2- 172.21.15.43:0/3688251249 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56420f12b390 0x56420f5532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.907+0000 7fc748c43640 1 --2- 172.21.15.43:0/3688251249 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56420f12b390 0x56420f5532e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfb8e299a769726e server_cookie=7791481067e10929 in_seq=0 out_seq=0 2024-04-25T10:38:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.907+0000 7fc723fff640 10 client.17022 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.907+0000 7fc723fff640 1 -- 172.21.15.43:0/3688251249 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56420f439800 con 0x56420f12b390 2024-04-25T10:38:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.907+0000 7fc723fff640 10 client.17022 handle_command_reply: tid=0 2024-04-25T10:38:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.907+0000 7fc751e31740 1 client.17022 shutdown 2024-04-25T10:38:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.907+0000 7fc751e31740 1 -- 172.21.15.43:0/3688251249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f11f4e0 msgr2=0x56420f4dabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.907+0000 7fc751e31740 1 --2- 172.21.15.43:0/3688251249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f11f4e0 0x56420f4dabf0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc72c03d4a0 tx=0x7fc72c03d940 comp rx=0 tx=0).stop 2024-04-25T10:38:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.907+0000 7fc751e31740 1 -- 172.21.15.43:0/3688251249 shutdown_connections 2024-04-25T10:38:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.907+0000 7fc751e31740 1 --2- 172.21.15.43:0/3688251249 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56420f12b390 0x56420f5532e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.907+0000 7fc751e31740 1 --2- 172.21.15.43:0/3688251249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56420f10a6b0 0x56420f4da6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.907+0000 7fc751e31740 1 --2- 172.21.15.43:0/3688251249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56420f11f4e0 0x56420f4dabf0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.907+0000 7fc751e31740 1 --2- 172.21.15.43:0/3688251249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56420f1024d0 0x56420f4da170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.907+0000 7fc723fff640 0 client.17022 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.907+0000 7fc751e31740 1 -- 172.21.15.43:0/3688251249 >> 172.21.15.43:0/3688251249 conn(0x56420f4cbfc0 msgr2=0x56420f4d7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:01.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.907+0000 7fc751e31740 1 -- 172.21.15.43:0/3688251249 shutdown_connections 2024-04-25T10:38:01.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.907+0000 7fc751e31740 1 -- 172.21.15.43:0/3688251249 wait complete. 2024-04-25T10:38:01.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.907+0000 7fc751e31740 20 client.17022 trim_cache size 0 max 16384 2024-04-25T10:38:01.916 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:38:01.916 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:38:01.916 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:38:01.916 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:38:01.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.908+0000 7fc74aea7640 1 -- 172.21.15.43:0/3063259755 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc7300afc60 msgr2=0x7fc7300b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:01.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.908+0000 7fc74aea7640 1 --2- 172.21.15.43:0/3063259755 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc7300afc60 0x7fc7300b2170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc72c005ab0 tx=0x7fc72c009260 comp rx=0 tx=0).stop 2024-04-25T10:38:01.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.908+0000 7fc74aea7640 1 -- 172.21.15.43:0/3063259755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc744104d10 msgr2=0x7fc74419a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:01.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.908+0000 7fc74aea7640 1 --2- 172.21.15.43:0/3063259755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc744104d10 0x7fc74419a720 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc73c00c950 tx=0x7fc73c00ce10 comp rx=0 tx=0).stop 2024-04-25T10:38:01.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.908+0000 7fc74aea7640 1 -- 172.21.15.43:0/3063259755 shutdown_connections 2024-04-25T10:38:01.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.908+0000 7fc74aea7640 1 --2- 172.21.15.43:0/3063259755 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc7300afc60 0x7fc7300b2170 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.908+0000 7fc74aea7640 1 --2- 172.21.15.43:0/3063259755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc74410b770 0x7fc74419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.908+0000 7fc74aea7640 1 --2- 172.21.15.43:0/3063259755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc74410e0d0 0x7fc74419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.908+0000 7fc74aea7640 1 --2- 172.21.15.43:0/3063259755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc744104d10 0x7fc74419a720 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:01.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.908+0000 7fc74aea7640 1 -- 172.21.15.43:0/3063259755 >> 172.21.15.43:0/3063259755 conn(0x7fc744078110 msgr2=0x7fc744101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:01.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.908+0000 7fc74aea7640 1 -- 172.21.15.43:0/3063259755 shutdown_connections 2024-04-25T10:38:01.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:01.908+0000 7fc74aea7640 1 -- 172.21.15.43:0/3063259755 wait complete. 2024-04-25T10:38:01.921 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:38:01.921 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:38:02.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.031+0000 7f2362b6c640 1 -- 172.21.15.43:0/2597936803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235c104260 msgr2=0x7f235c110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.031+0000 7f2362b6c640 1 --2- 172.21.15.43:0/2597936803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235c104260 0x7f235c110630 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f234c009900 tx=0x7f234c02a410 comp rx=0 tx=0).stop 2024-04-25T10:38:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.031+0000 7f2362b6c640 1 -- 172.21.15.43:0/2597936803 shutdown_connections 2024-04-25T10:38:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.031+0000 7f2362b6c640 1 --2- 172.21.15.43:0/2597936803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235c104260 0x7f235c110630 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.031+0000 7f2362b6c640 1 --2- 172.21.15.43:0/2597936803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f235c109fb0 0x7f235c103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.031+0000 7f2362b6c640 1 --2- 172.21.15.43:0/2597936803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f235c109b10 0x7f235c1036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.031+0000 7f2362b6c640 1 -- 172.21.15.43:0/2597936803 >> 172.21.15.43:0/2597936803 conn(0x7f235c0ff1b0 msgr2=0x7f235c101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.031+0000 7f2362b6c640 1 -- 172.21.15.43:0/2597936803 shutdown_connections 2024-04-25T10:38:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.031+0000 7f2362b6c640 1 -- 172.21.15.43:0/2597936803 wait complete. 2024-04-25T10:38:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.032+0000 7f2362b6c640 1 Processor -- start 2024-04-25T10:38:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.032+0000 7f2362b6c640 1 -- start start 2024-04-25T10:38:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.032+0000 7f2362b6c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235c104260 0x7f235c075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.032+0000 7f2362b6c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f235c109b10 0x7f235c0764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.032+0000 7f2362b6c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f235c109fb0 0x7f235c076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.032+0000 7f2362b6c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f235c113d00 con 0x7f235c104260 2024-04-25T10:38:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.032+0000 7f2362b6c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f235c113b80 con 0x7f235c109fb0 2024-04-25T10:38:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.032+0000 7f2362b6c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f235c113e80 con 0x7f235c109b10 2024-04-25T10:38:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.032+0000 7f2360908640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235c104260 0x7f235c075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.032+0000 7f2360908640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235c104260 0x7f235c075fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54872/0 (socket says 172.21.15.43:54872) 2024-04-25T10:38:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.032+0000 7f2360908640 1 -- 172.21.15.43:0/12639450 learned_addr learned my addr 172.21.15.43:0/12639450 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.032+0000 7f2361109640 1 --2- 172.21.15.43:0/12639450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f235c109fb0 0x7f235c076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.033+0000 7f235bfff640 1 --2- 172.21.15.43:0/12639450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f235c109b10 0x7f235c0764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.033+0000 7f2360908640 1 -- 172.21.15.43:0/12639450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f235c109b10 msgr2=0x7f235c0764f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.033+0000 7f2360908640 1 --2- 172.21.15.43:0/12639450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f235c109b10 0x7f235c0764f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.033+0000 7f2360908640 1 -- 172.21.15.43:0/12639450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f235c109fb0 msgr2=0x7f235c076a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.033+0000 7f2360908640 1 --2- 172.21.15.43:0/12639450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f235c109fb0 0x7f235c076a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.033+0000 7f2360908640 1 -- 172.21.15.43:0/12639450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f235c1a89b0 con 0x7f235c104260 2024-04-25T10:38:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.033+0000 7f2360908640 1 --2- 172.21.15.43:0/12639450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235c104260 0x7f235c075fb0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f235400ea80 tx=0x7f235400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.033+0000 7f2359ffb640 1 -- 172.21.15.43:0/12639450 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235400ce40 con 0x7f235c104260 2024-04-25T10:38:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.033+0000 7f2359ffb640 1 -- 172.21.15.43:0/12639450 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2354011920 con 0x7f235c104260 2024-04-25T10:38:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.033+0000 7f2362b6c640 1 -- 172.21.15.43:0/12639450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f235c1a8ca0 con 0x7f235c104260 2024-04-25T10:38:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.033+0000 7f2359ffb640 1 -- 172.21.15.43:0/12639450 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235401aa20 con 0x7f235c104260 2024-04-25T10:38:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.033+0000 7f2362b6c640 1 -- 172.21.15.43:0/12639450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f235c1a91a0 con 0x7f235c104260 2024-04-25T10:38:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.034+0000 7f2362b6c640 1 -- 172.21.15.43:0/12639450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f235c06a190 con 0x7f235c104260 2024-04-25T10:38:02.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.036+0000 7f2359ffb640 1 -- 172.21.15.43:0/12639450 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f235401abc0 con 0x7f235c104260 2024-04-25T10:38:02.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.036+0000 7f2359ffb640 1 --2- 172.21.15.43:0/12639450 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f23440afc60 0x7f23440b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.037+0000 7f235bfff640 1 --2- 172.21.15.43:0/12639450 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f23440afc60 0x7f23440b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.037+0000 7f2359ffb640 1 -- 172.21.15.43:0/12639450 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f23540df820 con 0x7f235c104260 2024-04-25T10:38:02.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.037+0000 7f235bfff640 1 --2- 172.21.15.43:0/12639450 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f23440afc60 0x7f23440b2170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f235c1028f0 tx=0x7f2350005ee0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:02.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.040+0000 7f2359ffb640 1 -- 172.21.15.43:0/12639450 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2354055ff0 con 0x7f235c104260 2024-04-25T10:38:02.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:01 smithi043 ceph-mon[29478]: pgmap v168: 97 pgs: 97 active+clean; 3.4 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 9 op/s 2024-04-25T10:38:02.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.266+0000 7f2362b6c640 1 -- 172.21.15.43:0/12639450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f235c0774e0 con 0x7f235c104260 2024-04-25T10:38:02.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.267+0000 7f2359ffb640 1 -- 172.21.15.43:0/12639450 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7f2354016030 con 0x7f235c104260 2024-04-25T10:38:02.269 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:38:02.269 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:38:02.269 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:38:02.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.269+0000 7f2362b6c640 1 -- 172.21.15.43:0/12639450 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f23440afc60 msgr2=0x7f23440b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.269+0000 7f2362b6c640 1 --2- 172.21.15.43:0/12639450 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f23440afc60 0x7f23440b2170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f235c1028f0 tx=0x7f2350005ee0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.269+0000 7f2362b6c640 1 -- 172.21.15.43:0/12639450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235c104260 msgr2=0x7f235c075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.269+0000 7f2362b6c640 1 --2- 172.21.15.43:0/12639450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235c104260 0x7f235c075fb0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f235400ea80 tx=0x7f235400ef40 comp rx=0 tx=0).stop 2024-04-25T10:38:02.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.269+0000 7f2362b6c640 1 -- 172.21.15.43:0/12639450 shutdown_connections 2024-04-25T10:38:02.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.269+0000 7f2362b6c640 1 --2- 172.21.15.43:0/12639450 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f23440afc60 0x7f23440b2170 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.269+0000 7f2362b6c640 1 --2- 172.21.15.43:0/12639450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f235c109fb0 0x7f235c076a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.269+0000 7f2362b6c640 1 --2- 172.21.15.43:0/12639450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f235c109b10 0x7f235c0764f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.269+0000 7f2362b6c640 1 --2- 172.21.15.43:0/12639450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235c104260 0x7f235c075fb0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.269+0000 7f2362b6c640 1 -- 172.21.15.43:0/12639450 >> 172.21.15.43:0/12639450 conn(0x7f235c0ff1b0 msgr2=0x7f235c106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:02.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.269+0000 7f2362b6c640 1 -- 172.21.15.43:0/12639450 shutdown_connections 2024-04-25T10:38:02.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.269+0000 7f2362b6c640 1 -- 172.21.15.43:0/12639450 wait complete. 2024-04-25T10:38:02.282 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:38:02.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:01 smithi077 ceph-mon[32772]: pgmap v168: 97 pgs: 97 active+clean; 3.4 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 9 op/s 2024-04-25T10:38:02.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:01 smithi164 ceph-mon[32713]: pgmap v168: 97 pgs: 97 active+clean; 3.4 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 9 op/s 2024-04-25T10:38:02.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.395+0000 7f5d191c5640 1 -- 172.21.15.43:0/3218932066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d1410d380 msgr2=0x7f5d1410f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.395+0000 7f5d191c5640 1 --2- 172.21.15.43:0/3218932066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d1410d380 0x7f5d1410f7a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009ca0 tx=0x7f5d0c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.395+0000 7f5d191c5640 1 -- 172.21.15.43:0/3218932066 shutdown_connections 2024-04-25T10:38:02.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.395+0000 7f5d191c5640 1 --2- 172.21.15.43:0/3218932066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d14102af0 0x7f5d14102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.395+0000 7f5d191c5640 1 --2- 172.21.15.43:0/3218932066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d1410d380 0x7f5d1410f7a0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.395+0000 7f5d191c5640 1 --2- 172.21.15.43:0/3218932066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d14108a20 0x7f5d141025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.395+0000 7f5d191c5640 1 -- 172.21.15.43:0/3218932066 >> 172.21.15.43:0/3218932066 conn(0x7f5d14078100 msgr2=0x7f5d14078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:02.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.395+0000 7f5d191c5640 1 -- 172.21.15.43:0/3218932066 shutdown_connections 2024-04-25T10:38:02.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.395+0000 7f5d191c5640 1 -- 172.21.15.43:0/3218932066 wait complete. 2024-04-25T10:38:02.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.396+0000 7f5d191c5640 1 Processor -- start 2024-04-25T10:38:02.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.396+0000 7f5d191c5640 1 -- start start 2024-04-25T10:38:02.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.396+0000 7f5d191c5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d14102af0 0x7f5d140ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.396+0000 7f5d191c5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d14108a20 0x7f5d141001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.396+0000 7f5d191c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d1410d380 0x7f5d14100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.396+0000 7f5d191c5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d14112c00 con 0x7f5d1410d380 2024-04-25T10:38:02.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.396+0000 7f5d191c5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5d14112a80 con 0x7f5d14102af0 2024-04-25T10:38:02.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.396+0000 7f5d191c5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5d14112d80 con 0x7f5d14108a20 2024-04-25T10:38:02.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.396+0000 7f5d1359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d1410d380 0x7f5d14100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.396+0000 7f5d1359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d1410d380 0x7f5d14100700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54904/0 (socket says 172.21.15.43:54904) 2024-04-25T10:38:02.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.396+0000 7f5d1359e640 1 -- 172.21.15.43:0/1164355801 learned_addr learned my addr 172.21.15.43:0/1164355801 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:02.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.396+0000 7f5d12d9d640 1 --2- 172.21.15.43:0/1164355801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d14102af0 0x7f5d140ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.397+0000 7f5d0bfff640 1 --2- 172.21.15.43:0/1164355801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d14108a20 0x7f5d141001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.397+0000 7f5d1359e640 1 -- 172.21.15.43:0/1164355801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d14108a20 msgr2=0x7f5d141001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.397+0000 7f5d1359e640 1 --2- 172.21.15.43:0/1164355801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d14108a20 0x7f5d141001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.397+0000 7f5d1359e640 1 -- 172.21.15.43:0/1164355801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d14102af0 msgr2=0x7f5d140ffc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.397+0000 7f5d1359e640 1 --2- 172.21.15.43:0/1164355801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d14102af0 0x7f5d140ffc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.397+0000 7f5d1359e640 1 -- 172.21.15.43:0/1164355801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d141a8a30 con 0x7f5d1410d380 2024-04-25T10:38:02.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.397+0000 7f5d12d9d640 1 --2- 172.21.15.43:0/1164355801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d14102af0 0x7f5d140ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:38:02.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.397+0000 7f5d0bfff640 1 --2- 172.21.15.43:0/1164355801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d14108a20 0x7f5d141001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:38:02.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.397+0000 7f5d1359e640 1 --2- 172.21.15.43:0/1164355801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d1410d380 0x7f5d14100700 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0098d0 tx=0x7f5d0c0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:02.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.397+0000 7f5d10d99640 1 -- 172.21.15.43:0/1164355801 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c003b70 con 0x7f5d1410d380 2024-04-25T10:38:02.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.397+0000 7f5d10d99640 1 -- 172.21.15.43:0/1164355801 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d0c003d10 con 0x7f5d1410d380 2024-04-25T10:38:02.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.397+0000 7f5d191c5640 1 -- 172.21.15.43:0/1164355801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d141a8cc0 con 0x7f5d1410d380 2024-04-25T10:38:02.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.397+0000 7f5d10d99640 1 -- 172.21.15.43:0/1164355801 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c02fa20 con 0x7f5d1410d380 2024-04-25T10:38:02.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.397+0000 7f5d191c5640 1 -- 172.21.15.43:0/1164355801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d141a91c0 con 0x7f5d1410d380 2024-04-25T10:38:02.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.400+0000 7f5d2007a740 1 Processor -- start 2024-04-25T10:38:02.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.400+0000 7f5d10d99640 1 -- 172.21.15.43:0/1164355801 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f5d0c02fc40 con 0x7f5d1410d380 2024-04-25T10:38:02.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.400+0000 7f5d2007a740 1 -- start start 2024-04-25T10:38:02.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.400+0000 7f5d2007a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0e7698390 0x55c0e7a39810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.400+0000 7f5d2007a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e766f4d0 0x55c0e7a39d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.400+0000 7f5d2007a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e76776b0 0x55c0e7a466a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.400+0000 7f5d2007a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c0e799ed80 con 0x55c0e766f4d0 2024-04-25T10:38:02.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.400+0000 7f5d2007a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c0e7958a20 con 0x55c0e76776b0 2024-04-25T10:38:02.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.400+0000 7f5d2007a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c0e799ad40 con 0x55c0e7698390 2024-04-25T10:38:02.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.401+0000 7f5d12d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e766f4d0 0x55c0e7a39d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.401+0000 7f5d0bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e76776b0 0x55c0e7a466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.401+0000 7f5d12d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e766f4d0 0x55c0e7a39d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54914/0 (socket says 172.21.15.43:54914) 2024-04-25T10:38:02.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.401+0000 7f5d0bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e76776b0 0x55c0e7a466a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:39858/0 (socket says 172.21.15.43:39858) 2024-04-25T10:38:02.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.401+0000 7f5d12d9d640 1 -- 172.21.15.43:0/1001300899 learned_addr learned my addr 172.21.15.43:0/1001300899 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:02.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.401+0000 7f5d0bfff640 1 --2- 172.21.15.43:0/1001300899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0e7698390 0x55c0e7a39810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.401+0000 7f5d10d99640 1 --2- 172.21.15.43:0/1164355801 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5d000afc60 0x7f5d000b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.401+0000 7f5d12d9d640 1 -- 172.21.15.43:0/1001300899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0e7698390 msgr2=0x55c0e7a39810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.401+0000 7f5d12d9d640 1 --2- 172.21.15.43:0/1001300899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0e7698390 0x55c0e7a39810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.401+0000 7f5d12d9d640 1 -- 172.21.15.43:0/1001300899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e76776b0 msgr2=0x55c0e7a466a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.401+0000 7f5d12d9d640 1 --2- 172.21.15.43:0/1001300899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e76776b0 0x55c0e7a466a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.401+0000 7f5d12d9d640 1 -- 172.21.15.43:0/1001300899 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0e7963c30 con 0x55c0e766f4d0 2024-04-25T10:38:02.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.401+0000 7f5d0bfff640 1 --2- 172.21.15.43:0/1001300899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e76776b0 0x55c0e7a466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:38:02.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.401+0000 7f5d10d99640 1 -- 172.21.15.43:0/1164355801 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f5d0c100500 con 0x7f5d1410d380 2024-04-25T10:38:02.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.401+0000 7f5d1359e640 1 --2- 172.21.15.43:0/1164355801 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5d000afc60 0x7f5d000b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.401+0000 7f5d12d9d640 1 --2- 172.21.15.43:0/1001300899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e766f4d0 0x55c0e7a39d50 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009c60 tx=0x7f5d040117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936f53cd16128308 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:02.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.401+0000 7f5d097fa640 1 -- 172.21.15.43:0/1001300899 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04012450 con 0x55c0e766f4d0 2024-04-25T10:38:02.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.401+0000 7f5d1359e640 1 --2- 172.21.15.43:0/1164355801 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5d000afc60 0x7f5d000b2170 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0fddf0 tx=0x7f5d0c0095c0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:02.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.401+0000 7f5d097fa640 1 -- 172.21.15.43:0/1001300899 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d04012aa0 con 0x55c0e766f4d0 2024-04-25T10:38:02.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.401+0000 7f5d097fa640 1 -- 172.21.15.43:0/1001300899 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0400fa80 con 0x55c0e766f4d0 2024-04-25T10:38:02.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.402+0000 7f5d2007a740 1 -- 172.21.15.43:0/1001300899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e766f4d0 msgr2=0x55c0e7a39d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.402+0000 7f5d2007a740 1 --2- 172.21.15.43:0/1001300899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e766f4d0 0x55c0e7a39d50 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009c60 tx=0x7f5d040117b0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.402+0000 7f5d2007a740 1 -- 172.21.15.43:0/1001300899 shutdown_connections 2024-04-25T10:38:02.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.402+0000 7f5d2007a740 1 --2- 172.21.15.43:0/1001300899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e76776b0 0x55c0e7a466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.402+0000 7f5d2007a740 1 --2- 172.21.15.43:0/1001300899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e766f4d0 0x55c0e7a39d50 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.402+0000 7f5d2007a740 1 --2- 172.21.15.43:0/1001300899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0e7698390 0x55c0e7a39810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.402+0000 7f5d2007a740 1 -- 172.21.15.43:0/1001300899 >> 172.21.15.43:0/1001300899 conn(0x55c0e768c4e0 msgr2=0x55c0e7a35fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:02.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.402+0000 7f5d2007a740 1 -- 172.21.15.43:0/1001300899 shutdown_connections 2024-04-25T10:38:02.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.403+0000 7f5d2007a740 1 -- 172.21.15.43:0/1001300899 wait complete. 2024-04-25T10:38:02.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.403+0000 7f5d2007a740 1 -- start start 2024-04-25T10:38:02.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.403+0000 7f5d2007a740 1 Processor -- start 2024-04-25T10:38:02.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.403+0000 7f5d2007a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e766f4d0 0x55c0e7a44260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.403+0000 7f5d2007a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0e76776b0 0x55c0e7a447a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.403+0000 7f5d2007a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e7698390 0x55c0e7a50210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.403+0000 7f5d2007a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c0e7a46d90 con 0x55c0e7698390 2024-04-25T10:38:02.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.403+0000 7f5d2007a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c0e7925dc0 con 0x55c0e766f4d0 2024-04-25T10:38:02.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.403+0000 7f5d2007a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c0e7a46f10 con 0x55c0e76776b0 2024-04-25T10:38:02.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.403+0000 7f5d2007a740 10 client.0 _ll_register_callbacks cb 0x55c0e7941600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:02.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.403+0000 7f5d0bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e7698390 0x55c0e7a50210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.403+0000 7f5d0bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e7698390 0x55c0e7a50210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54926/0 (socket says 172.21.15.43:54926) 2024-04-25T10:38:02.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.403+0000 7f5d0bfff640 1 -- 172.21.15.43:0/1818957744 learned_addr learned my addr 172.21.15.43:0/1818957744 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:02.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.404+0000 7f5d0bfff640 1 --2- 172.21.15.43:0/1818957744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e766f4d0 0x55c0e7a44260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.404+0000 7f5d12d9d640 1 --2- 172.21.15.43:0/1818957744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0e76776b0 0x55c0e7a447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.404+0000 7f5d0bfff640 1 -- 172.21.15.43:0/1818957744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0e76776b0 msgr2=0x55c0e7a447a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.404+0000 7f5d0bfff640 1 --2- 172.21.15.43:0/1818957744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0e76776b0 0x55c0e7a447a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.404+0000 7f5d0bfff640 1 -- 172.21.15.43:0/1818957744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e766f4d0 msgr2=0x55c0e7a44260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.404+0000 7f5d0bfff640 1 --2- 172.21.15.43:0/1818957744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e766f4d0 0x55c0e7a44260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.404+0000 7f5d0bfff640 1 -- 172.21.15.43:0/1818957744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0e76b03f0 con 0x55c0e7698390 2024-04-25T10:38:02.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.404+0000 7f5d0bfff640 1 --2- 172.21.15.43:0/1818957744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e7698390 0x55c0e7a50210 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f5d14101ee0 tx=0x7f5cfc015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0eb4d96ccbc1970 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:02.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.404+0000 7f5cfa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:02.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.404+0000 7f5cfa7fc640 1 -- 172.21.15.43:0/1818957744 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc010cd0 con 0x55c0e7698390 2024-04-25T10:38:02.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.404+0000 7f5cfa7fc640 1 -- 172.21.15.43:0/1818957744 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f5cfc005020 con 0x55c0e7698390 2024-04-25T10:38:02.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.404+0000 7f5d2007a740 1 -- 172.21.15.43:0/1818957744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d04012aa0 con 0x55c0e7698390 2024-04-25T10:38:02.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.404+0000 7f5cfa7fc640 1 -- 172.21.15.43:0/1818957744 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5cfc005220 con 0x55c0e7698390 2024-04-25T10:38:02.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.404+0000 7f5d2007a740 10 client.17040 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:02.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.404+0000 7f5d2007a740 1 -- 172.21.15.43:0/1818957744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c0e777a7d0 con 0x55c0e7698390 2024-04-25T10:38:02.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.404+0000 7f5cfa7fc640 1 -- 172.21.15.43:0/1818957744 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d04012aa0 con 0x55c0e7698390 2024-04-25T10:38:02.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.405+0000 7f5cfa7fc640 1 -- 172.21.15.43:0/1818957744 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5cfc00f040 con 0x55c0e7698390 2024-04-25T10:38:02.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.405+0000 7f5d2007a740 10 client.17040 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:02.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.405+0000 7f5d2007a740 10 client.17040 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:38:02.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.405+0000 7f5d2007a740 20 client.17040 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:02.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.405+0000 7f5d2007a740 1 --2- 172.21.15.43:0/1818957744 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c0e7656f60 0x55c0e7a56770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.405+0000 7f5d2007a740 4 client.17040 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:38:02.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.405+0000 7f5d2007a740 1 -- 172.21.15.43:0/1818957744 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0e79ac990 con 0x55c0e7656f60 2024-04-25T10:38:02.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.405+0000 7f5d12d9d640 1 --2- 172.21.15.43:0/1818957744 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c0e7656f60 0x55c0e7a56770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.405+0000 7f5cfa7fc640 10 client.17040 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:38:02.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.405+0000 7f5d12d9d640 1 --2- 172.21.15.43:0/1818957744 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c0e7656f60 0x55c0e7a56770 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf20b0d499f33fe0 server_cookie=1cb65c2b1de1462e in_seq=0 out_seq=0 2024-04-25T10:38:02.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.406+0000 7f5cfa7fc640 1 -- 172.21.15.43:0/1818957744 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c0e79ac990 con 0x55c0e7656f60 2024-04-25T10:38:02.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.406+0000 7f5cfa7fc640 10 client.17040 handle_command_reply: tid=0 2024-04-25T10:38:02.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.406+0000 7f5d2007a740 1 client.17040 shutdown 2024-04-25T10:38:02.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.406+0000 7f5d2007a740 1 -- 172.21.15.43:0/1818957744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e7698390 msgr2=0x55c0e7a50210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.406+0000 7f5d2007a740 1 --2- 172.21.15.43:0/1818957744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e7698390 0x55c0e7a50210 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f5d14101ee0 tx=0x7f5cfc015040 comp rx=0 tx=0).stop 2024-04-25T10:38:02.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.406+0000 7f5d2007a740 1 -- 172.21.15.43:0/1818957744 shutdown_connections 2024-04-25T10:38:02.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.406+0000 7f5d2007a740 1 --2- 172.21.15.43:0/1818957744 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c0e7656f60 0x55c0e7a56770 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.406+0000 7f5d2007a740 1 --2- 172.21.15.43:0/1818957744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e7698390 0x55c0e7a50210 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.406+0000 7f5d2007a740 1 --2- 172.21.15.43:0/1818957744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0e76776b0 0x55c0e7a447a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.406+0000 7f5cfa7fc640 0 client.17040 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:38:02.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.406+0000 7f5d2007a740 1 --2- 172.21.15.43:0/1818957744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e766f4d0 0x55c0e7a44260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.406+0000 7f5d2007a740 1 -- 172.21.15.43:0/1818957744 >> 172.21.15.43:0/1818957744 conn(0x55c0e768c4e0 msgr2=0x55c0e7a3b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:02.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.406+0000 7f5d2007a740 1 -- 172.21.15.43:0/1818957744 shutdown_connections 2024-04-25T10:38:02.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.407+0000 7f5d2007a740 1 -- 172.21.15.43:0/1818957744 wait complete. 2024-04-25T10:38:02.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.407+0000 7f5d2007a740 20 client.17040 trim_cache size 0 max 16384 2024-04-25T10:38:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.420+0000 7f5d2007a740 1 Processor -- start 2024-04-25T10:38:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.420+0000 7f5d2007a740 1 -- start start 2024-04-25T10:38:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.420+0000 7f5d2007a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e7656f60 0x55c0e7a43930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.420+0000 7f5d2007a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e766f4d0 0x55c0e7a34d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.420+0000 7f5d2007a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0e76776b0 0x55c0e7a3e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.420+0000 7f5d2007a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141b43d0 con 0x55c0e766f4d0 2024-04-25T10:38:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.420+0000 7f5d2007a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141b1ca0 con 0x55c0e7656f60 2024-04-25T10:38:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.420+0000 7f5d2007a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141afaf0 con 0x55c0e76776b0 2024-04-25T10:38:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.421+0000 7f5d12d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e766f4d0 0x55c0e7a34d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.421+0000 7f5d12d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e766f4d0 0x55c0e7a34d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54932/0 (socket says 172.21.15.43:54932) 2024-04-25T10:38:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.421+0000 7f5d12d9d640 1 -- 172.21.15.43:0/4133978000 learned_addr learned my addr 172.21.15.43:0/4133978000 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.421+0000 7f5d0bfff640 1 --2- 172.21.15.43:0/4133978000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0e76776b0 0x55c0e7a3e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.421+0000 7f5d0bfff640 1 --2- 172.21.15.43:0/4133978000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e7656f60 0x55c0e7a43930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.421+0000 7f5d12d9d640 1 -- 172.21.15.43:0/4133978000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0e76776b0 msgr2=0x55c0e7a3e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.421+0000 7f5d12d9d640 1 --2- 172.21.15.43:0/4133978000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0e76776b0 0x55c0e7a3e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.421+0000 7f5d12d9d640 1 -- 172.21.15.43:0/4133978000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e7656f60 msgr2=0x55c0e7a43930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.421+0000 7f5d12d9d640 1 --2- 172.21.15.43:0/4133978000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e7656f60 0x55c0e7a43930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.421+0000 7f5d12d9d640 1 -- 172.21.15.43:0/4133978000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0e7a44f40 con 0x55c0e766f4d0 2024-04-25T10:38:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.421+0000 7f5d12d9d640 1 --2- 172.21.15.43:0/4133978000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e766f4d0 0x55c0e7a34d20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009df0 tx=0x7f5d04005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84969f2766fd49d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.421+0000 7f5d08ff9640 1 -- 172.21.15.43:0/4133978000 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0400f440 con 0x55c0e766f4d0 2024-04-25T10:38:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.421+0000 7f5d08ff9640 1 -- 172.21.15.43:0/4133978000 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d040054f0 con 0x55c0e766f4d0 2024-04-25T10:38:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.421+0000 7f5d08ff9640 1 -- 172.21.15.43:0/4133978000 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04004d80 con 0x55c0e766f4d0 2024-04-25T10:38:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.422+0000 7f5d2007a740 1 -- 172.21.15.43:0/4133978000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e766f4d0 msgr2=0x55c0e7a34d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.422+0000 7f5d2007a740 1 --2- 172.21.15.43:0/4133978000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e766f4d0 0x55c0e7a34d20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009df0 tx=0x7f5d04005f60 comp rx=0 tx=0).stop 2024-04-25T10:38:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.422+0000 7f5d2007a740 1 -- 172.21.15.43:0/4133978000 shutdown_connections 2024-04-25T10:38:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.422+0000 7f5d2007a740 1 --2- 172.21.15.43:0/4133978000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0e76776b0 0x55c0e7a3e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.422+0000 7f5d2007a740 1 --2- 172.21.15.43:0/4133978000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e766f4d0 0x55c0e7a34d20 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.422+0000 7f5d2007a740 1 --2- 172.21.15.43:0/4133978000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e7656f60 0x55c0e7a43930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.422+0000 7f5d2007a740 1 -- 172.21.15.43:0/4133978000 >> 172.21.15.43:0/4133978000 conn(0x55c0e768c4e0 msgr2=0x55c0e7a34610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.422+0000 7f5d2007a740 1 -- 172.21.15.43:0/4133978000 shutdown_connections 2024-04-25T10:38:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.422+0000 7f5d2007a740 1 -- 172.21.15.43:0/4133978000 wait complete. 2024-04-25T10:38:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.422+0000 7f5d2007a740 1 -- start start 2024-04-25T10:38:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.422+0000 7f5d2007a740 1 Processor -- start 2024-04-25T10:38:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.423+0000 7f5d2007a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e7656f60 0x55c0e7a471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.423+0000 7f5d2007a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0e766f4d0 0x55c0e7a476f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.423+0000 7f5d2007a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e76776b0 0x55c0e7a47c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.423+0000 7f5d2007a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141b0160 con 0x55c0e7656f60 2024-04-25T10:38:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.423+0000 7f5d2007a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141b15d0 con 0x55c0e76776b0 2024-04-25T10:38:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.423+0000 7f5d2007a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141b0830 con 0x55c0e766f4d0 2024-04-25T10:38:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.423+0000 7f5d2007a740 10 client.0 _ll_register_callbacks cb 0x7f5cfc0139e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.423+0000 7f5d0bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e7656f60 0x55c0e7a471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.423+0000 7f5d0bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e7656f60 0x55c0e7a471b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54946/0 (socket says 172.21.15.43:54946) 2024-04-25T10:38:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.423+0000 7f5d0bfff640 1 -- 172.21.15.43:0/1697595057 learned_addr learned my addr 172.21.15.43:0/1697595057 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.423+0000 7f5d12d9d640 1 --2- 172.21.15.43:0/1697595057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0e766f4d0 0x55c0e7a476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.423+0000 7f5d0bfff640 1 --2- 172.21.15.43:0/1697595057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e76776b0 0x55c0e7a47c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.423+0000 7f5d0bfff640 1 -- 172.21.15.43:0/1697595057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0e766f4d0 msgr2=0x55c0e7a476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.423+0000 7f5d0bfff640 1 --2- 172.21.15.43:0/1697595057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0e766f4d0 0x55c0e7a476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.423+0000 7f5d0bfff640 1 -- 172.21.15.43:0/1697595057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e76776b0 msgr2=0x55c0e7a47c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.423+0000 7f5d0bfff640 1 --2- 172.21.15.43:0/1697595057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e76776b0 0x55c0e7a47c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.423+0000 7f5d0bfff640 1 -- 172.21.15.43:0/1697595057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0e7abcbd0 con 0x55c0e7656f60 2024-04-25T10:38:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.423+0000 7f5d0bfff640 1 --2- 172.21.15.43:0/1697595057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e7656f60 0x55c0e7a471b0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc013af0 tx=0x7f5cfc015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31916a343d6246d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.423+0000 7f5cfa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.424+0000 7f5cfa7fc640 1 -- 172.21.15.43:0/1697595057 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc010e60 con 0x55c0e7656f60 2024-04-25T10:38:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.424+0000 7f5cfa7fc640 1 -- 172.21.15.43:0/1697595057 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f5cfc004a70 con 0x55c0e7656f60 2024-04-25T10:38:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.424+0000 7f5d2007a740 1 -- 172.21.15.43:0/1697595057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d040054f0 con 0x55c0e7656f60 2024-04-25T10:38:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.424+0000 7f5cfa7fc640 1 -- 172.21.15.43:0/1697595057 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5cfc02c040 con 0x55c0e7656f60 2024-04-25T10:38:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.424+0000 7f5d2007a740 10 client.17046 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.424+0000 7f5d2007a740 1 -- 172.21.15.43:0/1697595057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c0e7abde50 con 0x55c0e7656f60 2024-04-25T10:38:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.424+0000 7f5cfa7fc640 1 -- 172.21.15.43:0/1697595057 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d040054f0 con 0x55c0e7656f60 2024-04-25T10:38:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.424+0000 7f5cfa7fc640 1 -- 172.21.15.43:0/1697595057 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5cfc032040 con 0x55c0e7656f60 2024-04-25T10:38:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.424+0000 7f5d2007a740 10 client.17046 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.424+0000 7f5d2007a740 10 client.17046 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:38:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.424+0000 7f5d2007a740 20 client.17046 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.424+0000 7f5d2007a740 1 --2- 172.21.15.43:0/1697595057 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c0e7698390 0x55c0e7ac0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.424+0000 7f5d2007a740 4 client.17046 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:38:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.424+0000 7f5d2007a740 1 -- 172.21.15.43:0/1697595057 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0e79a6800 con 0x55c0e7698390 2024-04-25T10:38:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.425+0000 7f5d12d9d640 1 --2- 172.21.15.43:0/1697595057 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c0e7698390 0x55c0e7ac0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.425+0000 7f5d12d9d640 1 --2- 172.21.15.43:0/1697595057 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c0e7698390 0x55c0e7ac0580 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=707277301415bf76 server_cookie=d5a7a9e40e6a338d in_seq=0 out_seq=0 2024-04-25T10:38:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.425+0000 7f5cfa7fc640 10 client.17046 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:38:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.425+0000 7f5cfa7fc640 1 -- 172.21.15.43:0/1697595057 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0e79a6800 con 0x55c0e7698390 2024-04-25T10:38:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.425+0000 7f5cfa7fc640 10 client.17046 handle_command_reply: tid=0 2024-04-25T10:38:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.425+0000 7f5d2007a740 1 client.17046 shutdown 2024-04-25T10:38:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.425+0000 7f5d2007a740 1 -- 172.21.15.43:0/1697595057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e7656f60 msgr2=0x55c0e7a471b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.425+0000 7f5d2007a740 1 --2- 172.21.15.43:0/1697595057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e7656f60 0x55c0e7a471b0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc013af0 tx=0x7f5cfc015040 comp rx=0 tx=0).stop 2024-04-25T10:38:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.425+0000 7f5d2007a740 1 -- 172.21.15.43:0/1697595057 shutdown_connections 2024-04-25T10:38:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.425+0000 7f5d2007a740 1 --2- 172.21.15.43:0/1697595057 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c0e7698390 0x55c0e7ac0580 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.425+0000 7f5d2007a740 1 --2- 172.21.15.43:0/1697595057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0e76776b0 0x55c0e7a47c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.425+0000 7f5d2007a740 1 --2- 172.21.15.43:0/1697595057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0e766f4d0 0x55c0e7a476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.425+0000 7f5cfa7fc640 0 client.17046 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:38:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.425+0000 7f5d2007a740 1 --2- 172.21.15.43:0/1697595057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0e7656f60 0x55c0e7a471b0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.425+0000 7f5d2007a740 1 -- 172.21.15.43:0/1697595057 >> 172.21.15.43:0/1697595057 conn(0x55c0e768c4e0 msgr2=0x55c0e7a4ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.425+0000 7f5d2007a740 1 -- 172.21.15.43:0/1697595057 shutdown_connections 2024-04-25T10:38:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.425+0000 7f5d2007a740 1 -- 172.21.15.43:0/1697595057 wait complete. 2024-04-25T10:38:02.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.426+0000 7f5d2007a740 20 client.17046 trim_cache size 0 max 16384 2024-04-25T10:38:02.435 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:38:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.426+0000 7f5d191c5640 1 -- 172.21.15.43:0/1164355801 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5d000afc60 msgr2=0x7f5d000b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.426+0000 7f5d191c5640 1 --2- 172.21.15.43:0/1164355801 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5d000afc60 0x7f5d000b2170 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0fddf0 tx=0x7f5d0c0095c0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.426+0000 7f5d191c5640 1 -- 172.21.15.43:0/1164355801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d1410d380 msgr2=0x7f5d14100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.426+0000 7f5d191c5640 1 --2- 172.21.15.43:0/1164355801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d1410d380 0x7f5d14100700 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0098d0 tx=0x7f5d0c0317b0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.426+0000 7f5d191c5640 1 -- 172.21.15.43:0/1164355801 shutdown_connections 2024-04-25T10:38:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.426+0000 7f5d191c5640 1 --2- 172.21.15.43:0/1164355801 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5d000afc60 0x7f5d000b2170 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.426+0000 7f5d191c5640 1 --2- 172.21.15.43:0/1164355801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d1410d380 0x7f5d14100700 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.426+0000 7f5d191c5640 1 --2- 172.21.15.43:0/1164355801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d14108a20 0x7f5d141001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.426+0000 7f5d191c5640 1 --2- 172.21.15.43:0/1164355801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d14102af0 0x7f5d140ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.426+0000 7f5d191c5640 1 -- 172.21.15.43:0/1164355801 >> 172.21.15.43:0/1164355801 conn(0x7f5d14078100 msgr2=0x7f5d14069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.426+0000 7f5d191c5640 1 -- 172.21.15.43:0/1164355801 shutdown_connections 2024-04-25T10:38:02.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.426+0000 7f5d191c5640 1 -- 172.21.15.43:0/1164355801 wait complete. 2024-04-25T10:38:02.439 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:38:02.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.551+0000 7fcdf9b3f640 1 -- 172.21.15.43:0/768135159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf410d380 msgr2=0x7fcdf410f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.551+0000 7fcdf9b3f640 1 --2- 172.21.15.43:0/768135159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf410d380 0x7fcdf410f7a0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009ca0 tx=0x7fcdec02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.551+0000 7fcdf9b3f640 1 -- 172.21.15.43:0/768135159 shutdown_connections 2024-04-25T10:38:02.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.551+0000 7fcdf9b3f640 1 --2- 172.21.15.43:0/768135159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf410d380 0x7fcdf410f7a0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.552+0000 7fcdf9b3f640 1 --2- 172.21.15.43:0/768135159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdf4102af0 0x7fcdf4102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.552+0000 7fcdf9b3f640 1 --2- 172.21.15.43:0/768135159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcdf4108a20 0x7fcdf41025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.552+0000 7fcdf9b3f640 1 -- 172.21.15.43:0/768135159 >> 172.21.15.43:0/768135159 conn(0x7fcdf4078100 msgr2=0x7fcdf4078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:02.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.552+0000 7fcdf9b3f640 1 -- 172.21.15.43:0/768135159 shutdown_connections 2024-04-25T10:38:02.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.552+0000 7fcdf9b3f640 1 -- 172.21.15.43:0/768135159 wait complete. 2024-04-25T10:38:02.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.552+0000 7fcdf9b3f640 1 Processor -- start 2024-04-25T10:38:02.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.552+0000 7fcdf9b3f640 1 -- start start 2024-04-25T10:38:02.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.552+0000 7fcdf9b3f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcdf4102af0 0x7fcdf41101e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.552+0000 7fcdf9b3f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdf4108a20 0x7fcdf4110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.552+0000 7fcdf9b3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf410d380 0x7fcdf4110c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.552+0000 7fcdf9b3f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf4113d00 con 0x7fcdf410d380 2024-04-25T10:38:02.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.553+0000 7fcdf9b3f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf4113b80 con 0x7fcdf4102af0 2024-04-25T10:38:02.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.553+0000 7fcdf9b3f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf4113e80 con 0x7fcdf4108a20 2024-04-25T10:38:02.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.553+0000 7fcdf3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf410d380 0x7fcdf4110c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.553+0000 7fcdf3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf410d380 0x7fcdf4110c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54972/0 (socket says 172.21.15.43:54972) 2024-04-25T10:38:02.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.553+0000 7fcdf3fff640 1 -- 172.21.15.43:0/3756968658 learned_addr learned my addr 172.21.15.43:0/3756968658 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:02.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.553+0000 7fcdf37fe640 1 --2- 172.21.15.43:0/3756968658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcdf4102af0 0x7fcdf41101e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.553+0000 7fcdebfff640 1 --2- 172.21.15.43:0/3756968658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdf4108a20 0x7fcdf4110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.553+0000 7fcdf3fff640 1 -- 172.21.15.43:0/3756968658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdf4108a20 msgr2=0x7fcdf4110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.553+0000 7fcdf3fff640 1 --2- 172.21.15.43:0/3756968658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdf4108a20 0x7fcdf4110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.553+0000 7fcdf3fff640 1 -- 172.21.15.43:0/3756968658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcdf4102af0 msgr2=0x7fcdf41101e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.553+0000 7fcdf3fff640 1 --2- 172.21.15.43:0/3756968658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcdf4102af0 0x7fcdf41101e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.553+0000 7fcdf3fff640 1 -- 172.21.15.43:0/3756968658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf41a8a40 con 0x7fcdf410d380 2024-04-25T10:38:02.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.553+0000 7fcdf37fe640 1 --2- 172.21.15.43:0/3756968658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcdf4102af0 0x7fcdf41101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:38:02.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.553+0000 7fcdf3fff640 1 --2- 172.21.15.43:0/3756968658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf410d380 0x7fcdf4110c60 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0098d0 tx=0x7fcdec0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:02.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.553+0000 7fcdf17fa640 1 -- 172.21.15.43:0/3756968658 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec003b70 con 0x7fcdf410d380 2024-04-25T10:38:02.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.553+0000 7fcdf17fa640 1 -- 172.21.15.43:0/3756968658 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdec003d10 con 0x7fcdf410d380 2024-04-25T10:38:02.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.553+0000 7fcdf9b3f640 1 -- 172.21.15.43:0/3756968658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf41a8cd0 con 0x7fcdf410d380 2024-04-25T10:38:02.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.553+0000 7fcdf17fa640 1 -- 172.21.15.43:0/3756968658 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec02fa20 con 0x7fcdf410d380 2024-04-25T10:38:02.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.554+0000 7fcdf9b3f640 1 -- 172.21.15.43:0/3756968658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf41a91d0 con 0x7fcdf410d380 2024-04-25T10:38:02.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.556+0000 7fce009f2740 1 Processor -- start 2024-04-25T10:38:02.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.556+0000 7fcdf17fa640 1 -- 172.21.15.43:0/3756968658 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fcdec02fc40 con 0x7fcdf410d380 2024-04-25T10:38:02.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.556+0000 7fce009f2740 1 -- start start 2024-04-25T10:38:02.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.556+0000 7fce009f2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded8f390 0x562cdf130810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fce009f2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cded664d0 0x562cdf130d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fce009f2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cded6e6b0 0x562cdf13d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fce009f2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562cdf095d80 con 0x562cded8f390 2024-04-25T10:38:02.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fce009f2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562cdf04fa20 con 0x562cded6e6b0 2024-04-25T10:38:02.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fce009f2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562cdf091d40 con 0x562cded664d0 2024-04-25T10:38:02.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fcdebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded8f390 0x562cdf130810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fcdebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded8f390 0x562cdf130810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54980/0 (socket says 172.21.15.43:54980) 2024-04-25T10:38:02.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fcdebfff640 1 -- 172.21.15.43:0/2620454048 learned_addr learned my addr 172.21.15.43:0/2620454048 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:02.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fcdf37fe640 1 --2- 172.21.15.43:0/2620454048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cded664d0 0x562cdf130d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fcdebfff640 1 --2- 172.21.15.43:0/2620454048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cded6e6b0 0x562cdf13d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fcdf17fa640 1 --2- 172.21.15.43:0/3756968658 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcde00afc60 0x7fcde00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fcdebfff640 1 -- 172.21.15.43:0/2620454048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cded664d0 msgr2=0x562cdf130d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fcdebfff640 1 --2- 172.21.15.43:0/2620454048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cded664d0 0x562cdf130d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fcdebfff640 1 -- 172.21.15.43:0/2620454048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cded6e6b0 msgr2=0x562cdf13d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fcdebfff640 1 --2- 172.21.15.43:0/2620454048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cded6e6b0 0x562cdf13d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fcdebfff640 1 -- 172.21.15.43:0/2620454048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cdf05ac30 con 0x562cded8f390 2024-04-25T10:38:02.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fcdf37fe640 1 --2- 172.21.15.43:0/2620454048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cded664d0 0x562cdf130d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:38:02.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fcdf3fff640 1 --2- 172.21.15.43:0/3756968658 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcde00afc60 0x7fcde00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fcdf17fa640 1 -- 172.21.15.43:0/3756968658 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fcdec1004e0 con 0x7fcdf410d380 2024-04-25T10:38:02.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fcdebfff640 1 --2- 172.21.15.43:0/2620454048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded8f390 0x562cdf130810 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fcddc0134d0 tx=0x7fcddc009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45b9ce843c9bf06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:02.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.557+0000 7fcde9ffb640 1 -- 172.21.15.43:0/2620454048 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc004050 con 0x562cded8f390 2024-04-25T10:38:02.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.558+0000 7fcde9ffb640 1 -- 172.21.15.43:0/2620454048 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcddc0041f0 con 0x562cded8f390 2024-04-25T10:38:02.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.558+0000 7fcde9ffb640 1 -- 172.21.15.43:0/2620454048 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc010490 con 0x562cded8f390 2024-04-25T10:38:02.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.558+0000 7fce009f2740 1 -- 172.21.15.43:0/2620454048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded8f390 msgr2=0x562cdf130810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.558+0000 7fcdf3fff640 1 --2- 172.21.15.43:0/3756968658 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcde00afc60 0x7fcde00b2170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0fde20 tx=0x7fcdec096020 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:02.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.558+0000 7fce009f2740 1 --2- 172.21.15.43:0/2620454048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded8f390 0x562cdf130810 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fcddc0134d0 tx=0x7fcddc009710 comp rx=0 tx=0).stop 2024-04-25T10:38:02.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.558+0000 7fce009f2740 1 -- 172.21.15.43:0/2620454048 shutdown_connections 2024-04-25T10:38:02.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.558+0000 7fce009f2740 1 --2- 172.21.15.43:0/2620454048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cded6e6b0 0x562cdf13d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.558+0000 7fce009f2740 1 --2- 172.21.15.43:0/2620454048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cded664d0 0x562cdf130d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.558+0000 7fce009f2740 1 --2- 172.21.15.43:0/2620454048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded8f390 0x562cdf130810 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.558+0000 7fce009f2740 1 -- 172.21.15.43:0/2620454048 >> 172.21.15.43:0/2620454048 conn(0x562cded834e0 msgr2=0x562cdf12cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:02.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.558+0000 7fce009f2740 1 -- 172.21.15.43:0/2620454048 shutdown_connections 2024-04-25T10:38:02.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.558+0000 7fce009f2740 1 -- 172.21.15.43:0/2620454048 wait complete. 2024-04-25T10:38:02.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.559+0000 7fce009f2740 1 -- start start 2024-04-25T10:38:02.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.559+0000 7fce009f2740 1 Processor -- start 2024-04-25T10:38:02.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.559+0000 7fce009f2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cded664d0 0x562cdf13b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.559+0000 7fce009f2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cded6e6b0 0x562cdf13b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.559+0000 7fce009f2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded8f390 0x562cdf147210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.559+0000 7fce009f2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562cdf13dd90 con 0x562cded8f390 2024-04-25T10:38:02.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.559+0000 7fce009f2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562cdf01cdc0 con 0x562cded664d0 2024-04-25T10:38:02.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.559+0000 7fce009f2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562cdf13df10 con 0x562cded6e6b0 2024-04-25T10:38:02.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.559+0000 7fce009f2740 10 client.0 _ll_register_callbacks cb 0x562cdf038600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:02.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.560+0000 7fcdebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded8f390 0x562cdf147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.560+0000 7fcdebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded8f390 0x562cdf147210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54990/0 (socket says 172.21.15.43:54990) 2024-04-25T10:38:02.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.560+0000 7fcdebfff640 1 -- 172.21.15.43:0/3656492784 learned_addr learned my addr 172.21.15.43:0/3656492784 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:02.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.560+0000 7fcdf37fe640 1 --2- 172.21.15.43:0/3656492784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cded6e6b0 0x562cdf13b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.560+0000 7fcdebfff640 1 --2- 172.21.15.43:0/3656492784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cded664d0 0x562cdf13b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.560+0000 7fcdebfff640 1 -- 172.21.15.43:0/3656492784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cded6e6b0 msgr2=0x562cdf13b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.560+0000 7fcdebfff640 1 --2- 172.21.15.43:0/3656492784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cded6e6b0 0x562cdf13b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.560+0000 7fcdebfff640 1 -- 172.21.15.43:0/3656492784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cded664d0 msgr2=0x562cdf13b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.560+0000 7fcdebfff640 1 --2- 172.21.15.43:0/3656492784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cded664d0 0x562cdf13b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.560+0000 7fcdebfff640 1 -- 172.21.15.43:0/3656492784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cdeda73f0 con 0x562cded8f390 2024-04-25T10:38:02.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.560+0000 7fcdebfff640 1 --2- 172.21.15.43:0/3656492784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded8f390 0x562cdf147210 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009fd0 tx=0x7fcddc003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8145421f4edd4a1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:02.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.560+0000 7fcdd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:02.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.560+0000 7fcdd2ffd640 1 -- 172.21.15.43:0/3656492784 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc0035f0 con 0x562cded8f390 2024-04-25T10:38:02.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.560+0000 7fcdd2ffd640 1 -- 172.21.15.43:0/3656492784 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fcddc003790 con 0x562cded8f390 2024-04-25T10:38:02.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.560+0000 7fce009f2740 1 -- 172.21.15.43:0/3656492784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcddc0041f0 con 0x562cded8f390 2024-04-25T10:38:02.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.560+0000 7fcdd2ffd640 1 -- 172.21.15.43:0/3656492784 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcddc011470 con 0x562cded8f390 2024-04-25T10:38:02.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.560+0000 7fce009f2740 10 client.17058 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:02.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.560+0000 7fce009f2740 1 -- 172.21.15.43:0/3656492784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562cdee717d0 con 0x562cded8f390 2024-04-25T10:38:02.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.561+0000 7fcdd2ffd640 1 -- 172.21.15.43:0/3656492784 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcddc0041f0 con 0x562cded8f390 2024-04-25T10:38:02.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.561+0000 7fcdd2ffd640 1 -- 172.21.15.43:0/3656492784 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcddc02e430 con 0x562cded8f390 2024-04-25T10:38:02.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.561+0000 7fce009f2740 10 client.17058 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:02.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.561+0000 7fce009f2740 10 client.17058 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:38:02.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.561+0000 7fce009f2740 20 client.17058 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:02.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.561+0000 7fce009f2740 1 --2- 172.21.15.43:0/3656492784 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x562cded4df60 0x562cdf14d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.561+0000 7fce009f2740 4 client.17058 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:38:02.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.561+0000 7fce009f2740 1 -- 172.21.15.43:0/3656492784 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cdf0a3990 con 0x562cded4df60 2024-04-25T10:38:02.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.561+0000 7fcdf37fe640 1 --2- 172.21.15.43:0/3656492784 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x562cded4df60 0x562cdf14d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.562+0000 7fcdd2ffd640 10 client.17058 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:38:02.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.562+0000 7fcdf37fe640 1 --2- 172.21.15.43:0/3656492784 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x562cded4df60 0x562cdf14d5c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c51817484ce3ed78 server_cookie=dd9e72e3a49b55b9 in_seq=0 out_seq=0 2024-04-25T10:38:02.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.562+0000 7fcdd2ffd640 1 -- 172.21.15.43:0/3656492784 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562cdf0a3990 con 0x562cded4df60 2024-04-25T10:38:02.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.562+0000 7fcdd2ffd640 10 client.17058 handle_command_reply: tid=0 2024-04-25T10:38:02.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.562+0000 7fce009f2740 1 client.17058 shutdown 2024-04-25T10:38:02.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.563+0000 7fce009f2740 1 -- 172.21.15.43:0/3656492784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded8f390 msgr2=0x562cdf147210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.563+0000 7fce009f2740 1 --2- 172.21.15.43:0/3656492784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded8f390 0x562cdf147210 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009fd0 tx=0x7fcddc003c80 comp rx=0 tx=0).stop 2024-04-25T10:38:02.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.563+0000 7fce009f2740 1 -- 172.21.15.43:0/3656492784 shutdown_connections 2024-04-25T10:38:02.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.563+0000 7fce009f2740 1 --2- 172.21.15.43:0/3656492784 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x562cded4df60 0x562cdf14d5c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.563+0000 7fce009f2740 1 --2- 172.21.15.43:0/3656492784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded8f390 0x562cdf147210 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.563+0000 7fcdd2ffd640 0 client.17058 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:38:02.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.563+0000 7fce009f2740 1 --2- 172.21.15.43:0/3656492784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cded6e6b0 0x562cdf13b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.563+0000 7fce009f2740 1 --2- 172.21.15.43:0/3656492784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cded664d0 0x562cdf13b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.563+0000 7fce009f2740 1 -- 172.21.15.43:0/3656492784 >> 172.21.15.43:0/3656492784 conn(0x562cded834e0 msgr2=0x562cdf132530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:02.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.563+0000 7fce009f2740 1 -- 172.21.15.43:0/3656492784 shutdown_connections 2024-04-25T10:38:02.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.563+0000 7fce009f2740 1 -- 172.21.15.43:0/3656492784 wait complete. 2024-04-25T10:38:02.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.563+0000 7fce009f2740 20 client.17058 trim_cache size 0 max 16384 2024-04-25T10:38:02.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.577+0000 7fce009f2740 1 Processor -- start 2024-04-25T10:38:02.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.577+0000 7fce009f2740 1 -- start start 2024-04-25T10:38:02.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.577+0000 7fce009f2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded664d0 0x562cdf1368b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.577+0000 7fce009f2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cded6e6b0 0x562cdf13ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.577+0000 7fce009f2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cded8f390 0x562cdf146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.577+0000 7fce009f2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf41b43d0 con 0x562cded664d0 2024-04-25T10:38:02.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.577+0000 7fce009f2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf41b1ca0 con 0x562cded8f390 2024-04-25T10:38:02.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.577+0000 7fce009f2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf41afaf0 con 0x562cded6e6b0 2024-04-25T10:38:02.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.577+0000 7fcdebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded664d0 0x562cdf1368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.577+0000 7fcdebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded664d0 0x562cdf1368b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54994/0 (socket says 172.21.15.43:54994) 2024-04-25T10:38:02.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.577+0000 7fcdebfff640 1 -- 172.21.15.43:0/1817792290 learned_addr learned my addr 172.21.15.43:0/1817792290 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:02.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.577+0000 7fcdebfff640 1 -- 172.21.15.43:0/1817792290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cded6e6b0 msgr2=0x562cdf13ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.577+0000 7fcdebfff640 1 --2- 172.21.15.43:0/1817792290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cded6e6b0 0x562cdf13ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.577+0000 7fcdebfff640 1 -- 172.21.15.43:0/1817792290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cded8f390 msgr2=0x562cdf146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.577+0000 7fcdebfff640 1 --2- 172.21.15.43:0/1817792290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cded8f390 0x562cdf146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.577+0000 7fcdebfff640 1 -- 172.21.15.43:0/1817792290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cdf13bb90 con 0x562cded664d0 2024-04-25T10:38:02.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.578+0000 7fcdebfff640 1 --2- 172.21.15.43:0/1817792290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded664d0 0x562cdf1368b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fcddc000c00 tx=0x7fcddc009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a92d25dd03d7bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:02.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.578+0000 7fcde97fa640 1 -- 172.21.15.43:0/1817792290 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc0041f0 con 0x562cded664d0 2024-04-25T10:38:02.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.578+0000 7fcde97fa640 1 -- 172.21.15.43:0/1817792290 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcddc009b70 con 0x562cded664d0 2024-04-25T10:38:02.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.578+0000 7fcde97fa640 1 -- 172.21.15.43:0/1817792290 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc005070 con 0x562cded664d0 2024-04-25T10:38:02.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.579+0000 7fce009f2740 1 -- 172.21.15.43:0/1817792290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded664d0 msgr2=0x562cdf1368b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.579+0000 7fce009f2740 1 --2- 172.21.15.43:0/1817792290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded664d0 0x562cdf1368b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fcddc000c00 tx=0x7fcddc009ef0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.579+0000 7fce009f2740 1 -- 172.21.15.43:0/1817792290 shutdown_connections 2024-04-25T10:38:02.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.579+0000 7fce009f2740 1 --2- 172.21.15.43:0/1817792290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cded8f390 0x562cdf146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.579+0000 7fce009f2740 1 --2- 172.21.15.43:0/1817792290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cded6e6b0 0x562cdf13ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.579+0000 7fce009f2740 1 --2- 172.21.15.43:0/1817792290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded664d0 0x562cdf1368b0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.579+0000 7fce009f2740 1 -- 172.21.15.43:0/1817792290 >> 172.21.15.43:0/1817792290 conn(0x562cded834e0 msgr2=0x562cdf13a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:02.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.579+0000 7fce009f2740 1 -- 172.21.15.43:0/1817792290 shutdown_connections 2024-04-25T10:38:02.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.579+0000 7fce009f2740 1 -- 172.21.15.43:0/1817792290 wait complete. 2024-04-25T10:38:02.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.579+0000 7fce009f2740 1 -- start start 2024-04-25T10:38:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.579+0000 7fce009f2740 1 Processor -- start 2024-04-25T10:38:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fce009f2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cded664d0 0x562cdf13e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fce009f2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded6e6b0 0x562cdf13e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fce009f2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cded8f390 0x562cdf13ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fce009f2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf41b0160 con 0x562cded6e6b0 2024-04-25T10:38:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fce009f2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf41b15d0 con 0x562cded664d0 2024-04-25T10:38:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fce009f2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf41b0830 con 0x562cded8f390 2024-04-25T10:38:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fce009f2740 10 client.0 _ll_register_callbacks cb 0x7fcddc013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fcdf37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded6e6b0 0x562cdf13e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fcdf37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded6e6b0 0x562cdf13e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55000/0 (socket says 172.21.15.43:55000) 2024-04-25T10:38:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fcdf37fe640 1 -- 172.21.15.43:0/2986339652 learned_addr learned my addr 172.21.15.43:0/2986339652 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fcdebfff640 1 --2- 172.21.15.43:0/2986339652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cded664d0 0x562cdf13e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fcdebfff640 1 --2- 172.21.15.43:0/2986339652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cded8f390 0x562cdf13ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fcdf37fe640 1 -- 172.21.15.43:0/2986339652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cded8f390 msgr2=0x562cdf13ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fcdf37fe640 1 --2- 172.21.15.43:0/2986339652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cded8f390 0x562cdf13ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fcdf37fe640 1 -- 172.21.15.43:0/2986339652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cded664d0 msgr2=0x562cdf13e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fcdf37fe640 1 --2- 172.21.15.43:0/2986339652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cded664d0 0x562cdf13e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fcdf37fe640 1 -- 172.21.15.43:0/2986339652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cdf1b37f0 con 0x562cded6e6b0 2024-04-25T10:38:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fcdebfff640 1 --2- 172.21.15.43:0/2986339652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cded664d0 0x562cdf13e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:38:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fcdf37fe640 1 --2- 172.21.15.43:0/2986339652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded6e6b0 0x562cdf13e6f0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fcde4005d80 tx=0x7fcde40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19f993e638fb8e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fcdd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fcdd2ffd640 1 -- 172.21.15.43:0/2986339652 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde401c3b0 con 0x562cded6e6b0 2024-04-25T10:38:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fcdd2ffd640 1 -- 172.21.15.43:0/2986339652 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fcde401c550 con 0x562cded6e6b0 2024-04-25T10:38:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.580+0000 7fce009f2740 1 -- 172.21.15.43:0/2986339652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcddc009b70 con 0x562cded6e6b0 2024-04-25T10:38:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.581+0000 7fcdd2ffd640 1 -- 172.21.15.43:0/2986339652 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcde402a050 con 0x562cded6e6b0 2024-04-25T10:38:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.581+0000 7fce009f2740 10 client.17064 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.581+0000 7fce009f2740 1 -- 172.21.15.43:0/2986339652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562cdf1b4b00 con 0x562cded6e6b0 2024-04-25T10:38:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.581+0000 7fcdd2ffd640 1 -- 172.21.15.43:0/2986339652 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcddc009b70 con 0x562cded6e6b0 2024-04-25T10:38:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.581+0000 7fcdd2ffd640 1 -- 172.21.15.43:0/2986339652 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcde4003c30 con 0x562cded6e6b0 2024-04-25T10:38:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.581+0000 7fce009f2740 10 client.17064 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.581+0000 7fce009f2740 10 client.17064 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:38:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.581+0000 7fce009f2740 20 client.17064 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.581+0000 7fce009f2740 1 --2- 172.21.15.43:0/2986339652 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x562cded233a0 0x562cdf1b72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.581+0000 7fce009f2740 4 client.17064 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:38:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.581+0000 7fce009f2740 1 -- 172.21.15.43:0/2986339652 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562cdf147e70 con 0x562cded233a0 2024-04-25T10:38:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.581+0000 7fcdebfff640 1 --2- 172.21.15.43:0/2986339652 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x562cded233a0 0x562cdf1b72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.582+0000 7fcdd2ffd640 10 client.17064 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:38:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.582+0000 7fcdebfff640 1 --2- 172.21.15.43:0/2986339652 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x562cded233a0 0x562cdf1b72f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b6e15cdc2fc1bee server_cookie=3c87cd7c7d68dce0 in_seq=0 out_seq=0 2024-04-25T10:38:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.582+0000 7fcdd2ffd640 1 -- 172.21.15.43:0/2986339652 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562cdf147e70 con 0x562cded233a0 2024-04-25T10:38:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.582+0000 7fcdd2ffd640 10 client.17064 handle_command_reply: tid=0 2024-04-25T10:38:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.582+0000 7fce009f2740 1 client.17064 shutdown 2024-04-25T10:38:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.582+0000 7fce009f2740 1 -- 172.21.15.43:0/2986339652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded6e6b0 msgr2=0x562cdf13e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.582+0000 7fce009f2740 1 --2- 172.21.15.43:0/2986339652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded6e6b0 0x562cdf13e6f0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fcde4005d80 tx=0x7fcde40118d0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.582+0000 7fce009f2740 1 -- 172.21.15.43:0/2986339652 shutdown_connections 2024-04-25T10:38:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.582+0000 7fce009f2740 1 --2- 172.21.15.43:0/2986339652 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x562cded233a0 0x562cdf1b72f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.582+0000 7fce009f2740 1 --2- 172.21.15.43:0/2986339652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cded8f390 0x562cdf13ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.582+0000 7fce009f2740 1 --2- 172.21.15.43:0/2986339652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cded6e6b0 0x562cdf13e6f0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.582+0000 7fce009f2740 1 --2- 172.21.15.43:0/2986339652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cded664d0 0x562cdf13e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.582+0000 7fcdd2ffd640 0 client.17064 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:38:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.582+0000 7fce009f2740 1 -- 172.21.15.43:0/2986339652 >> 172.21.15.43:0/2986339652 conn(0x562cded834e0 msgr2=0x562cdf13c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.582+0000 7fce009f2740 1 -- 172.21.15.43:0/2986339652 shutdown_connections 2024-04-25T10:38:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.582+0000 7fce009f2740 1 -- 172.21.15.43:0/2986339652 wait complete. 2024-04-25T10:38:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.582+0000 7fce009f2740 20 client.17064 trim_cache size 0 max 16384 2024-04-25T10:38:02.591 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:38:02.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.583+0000 7fcdf9b3f640 1 -- 172.21.15.43:0/3756968658 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcde00afc60 msgr2=0x7fcde00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.583+0000 7fcdf9b3f640 1 --2- 172.21.15.43:0/3756968658 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcde00afc60 0x7fcde00b2170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0fde20 tx=0x7fcdec096020 comp rx=0 tx=0).stop 2024-04-25T10:38:02.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.583+0000 7fcdf9b3f640 1 -- 172.21.15.43:0/3756968658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf410d380 msgr2=0x7fcdf4110c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.583+0000 7fcdf9b3f640 1 --2- 172.21.15.43:0/3756968658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf410d380 0x7fcdf4110c60 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0098d0 tx=0x7fcdec0317b0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.583+0000 7fcdf9b3f640 1 -- 172.21.15.43:0/3756968658 shutdown_connections 2024-04-25T10:38:02.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.583+0000 7fcdf9b3f640 1 --2- 172.21.15.43:0/3756968658 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcde00afc60 0x7fcde00b2170 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.583+0000 7fcdf9b3f640 1 --2- 172.21.15.43:0/3756968658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf410d380 0x7fcdf4110c60 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.583+0000 7fcdf9b3f640 1 --2- 172.21.15.43:0/3756968658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdf4108a20 0x7fcdf4110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.583+0000 7fcdf9b3f640 1 --2- 172.21.15.43:0/3756968658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcdf4102af0 0x7fcdf41101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.583+0000 7fcdf9b3f640 1 -- 172.21.15.43:0/3756968658 >> 172.21.15.43:0/3756968658 conn(0x7fcdf4078100 msgr2=0x7fcdf41040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:02.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.583+0000 7fcdf9b3f640 1 -- 172.21.15.43:0/3756968658 shutdown_connections 2024-04-25T10:38:02.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.583+0000 7fcdf9b3f640 1 -- 172.21.15.43:0/3756968658 wait complete. 2024-04-25T10:38:02.596 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:38:02.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.709+0000 7f7836e92640 1 -- 172.21.15.43:0/167678611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7830108eb0 msgr2=0x7f7830102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.709+0000 7f7836e92640 1 --2- 172.21.15.43:0/167678611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7830108eb0 0x7f7830102b50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f7818009900 tx=0x7f781802a410 comp rx=0 tx=0).stop 2024-04-25T10:38:02.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.709+0000 7f7836e92640 1 -- 172.21.15.43:0/167678611 shutdown_connections 2024-04-25T10:38:02.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.709+0000 7f7836e92640 1 --2- 172.21.15.43:0/167678611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78301031c0 0x7f783010f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.709+0000 7f7836e92640 1 --2- 172.21.15.43:0/167678611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7830108eb0 0x7f7830102b50 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.709+0000 7f7836e92640 1 --2- 172.21.15.43:0/167678611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7830108a10 0x7f7830102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.709+0000 7f7836e92640 1 -- 172.21.15.43:0/167678611 >> 172.21.15.43:0/167678611 conn(0x7f7830078110 msgr2=0x7f7830078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:02.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.709+0000 7f7836e92640 1 -- 172.21.15.43:0/167678611 shutdown_connections 2024-04-25T10:38:02.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.709+0000 7f7836e92640 1 -- 172.21.15.43:0/167678611 wait complete. 2024-04-25T10:38:02.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.710+0000 7f7836e92640 1 Processor -- start 2024-04-25T10:38:02.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.710+0000 7f7836e92640 1 -- start start 2024-04-25T10:38:02.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.710+0000 7f7836e92640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78301031c0 0x7f783019b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.710+0000 7f7836e92640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7830108a10 0x7f783019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.710+0000 7f7836e92640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7830108eb0 0x7f783019c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.710+0000 7f7836e92640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7830112c30 con 0x7f78301031c0 2024-04-25T10:38:02.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.710+0000 7f7836e92640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7830112ab0 con 0x7f7830108a10 2024-04-25T10:38:02.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.710+0000 7f7836e92640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7830112db0 con 0x7f7830108eb0 2024-04-25T10:38:02.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.710+0000 7f7834c2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78301031c0 0x7f783019b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.711+0000 7f7834c2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78301031c0 0x7f783019b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55024/0 (socket says 172.21.15.43:55024) 2024-04-25T10:38:02.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.711+0000 7f7834c2e640 1 -- 172.21.15.43:0/2912500462 learned_addr learned my addr 172.21.15.43:0/2912500462 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:02.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.711+0000 7f782ffff640 1 --2- 172.21.15.43:0/2912500462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7830108a10 0x7f783019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.711+0000 7f783542f640 1 --2- 172.21.15.43:0/2912500462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7830108eb0 0x7f783019c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.711+0000 7f7834c2e640 1 -- 172.21.15.43:0/2912500462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7830108eb0 msgr2=0x7f783019c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.711+0000 7f7834c2e640 1 --2- 172.21.15.43:0/2912500462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7830108eb0 0x7f783019c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.711+0000 7f7834c2e640 1 -- 172.21.15.43:0/2912500462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7830108a10 msgr2=0x7f783019bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.711+0000 7f7834c2e640 1 --2- 172.21.15.43:0/2912500462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7830108a10 0x7f783019bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.711+0000 7f7834c2e640 1 -- 172.21.15.43:0/2912500462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78301a68e0 con 0x7f78301031c0 2024-04-25T10:38:02.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.711+0000 7f783542f640 1 --2- 172.21.15.43:0/2912500462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7830108eb0 0x7f783019c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:38:02.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.711+0000 7f7834c2e640 1 --2- 172.21.15.43:0/2912500462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78301031c0 0x7f783019b5d0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f782000e9c0 tx=0x7f782000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:02.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.711+0000 7f782dffb640 1 -- 172.21.15.43:0/2912500462 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782000ccc0 con 0x7f78301031c0 2024-04-25T10:38:02.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.711+0000 7f782dffb640 1 -- 172.21.15.43:0/2912500462 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78200054d0 con 0x7f78301031c0 2024-04-25T10:38:02.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.711+0000 7f7836e92640 1 -- 172.21.15.43:0/2912500462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78301a6bd0 con 0x7f78301031c0 2024-04-25T10:38:02.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.711+0000 7f782dffb640 1 -- 172.21.15.43:0/2912500462 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820010430 con 0x7f78301031c0 2024-04-25T10:38:02.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.711+0000 7f7836e92640 1 -- 172.21.15.43:0/2912500462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78301a70a0 con 0x7f78301031c0 2024-04-25T10:38:02.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.715+0000 7f783de13740 1 Processor -- start 2024-04-25T10:38:02.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.715+0000 7f783de13740 1 -- start start 2024-04-25T10:38:02.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.715+0000 7f782dffb640 1 -- 172.21.15.43:0/2912500462 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f7820005020 con 0x7f78301031c0 2024-04-25T10:38:02.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.715+0000 7f783de13740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f93390 0x55746a334810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.715+0000 7f783de13740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f6a4d0 0x55746a334d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.715+0000 7f783de13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557469f726b0 0x55746a3416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.715+0000 7f783de13740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55746a299d80 con 0x557469f6a4d0 2024-04-25T10:38:02.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.715+0000 7f783de13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55746a253a20 con 0x557469f726b0 2024-04-25T10:38:02.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.715+0000 7f783de13740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55746a295d40 con 0x557469f93390 2024-04-25T10:38:02.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.715+0000 7f7834c2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f6a4d0 0x55746a334d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.715+0000 7f7834c2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f6a4d0 0x55746a334d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55038/0 (socket says 172.21.15.43:55038) 2024-04-25T10:38:02.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.715+0000 7f7834c2e640 1 -- 172.21.15.43:0/2235186044 learned_addr learned my addr 172.21.15.43:0/2235186044 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:02.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.715+0000 7f782ffff640 1 --2- 172.21.15.43:0/2235186044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557469f726b0 0x55746a3416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.715+0000 7f783542f640 1 --2- 172.21.15.43:0/2235186044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f93390 0x55746a334810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.715+0000 7f7834c2e640 1 -- 172.21.15.43:0/2235186044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f93390 msgr2=0x55746a334810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.715+0000 7f7834c2e640 1 --2- 172.21.15.43:0/2235186044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f93390 0x55746a334810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.716+0000 7f7834c2e640 1 -- 172.21.15.43:0/2235186044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557469f726b0 msgr2=0x55746a3416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.716+0000 7f7834c2e640 1 --2- 172.21.15.43:0/2235186044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557469f726b0 0x55746a3416a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.716+0000 7f7834c2e640 1 -- 172.21.15.43:0/2235186044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55746a25ec30 con 0x557469f6a4d0 2024-04-25T10:38:02.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.716+0000 7f783542f640 1 --2- 172.21.15.43:0/2235186044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f93390 0x55746a334810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:38:02.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.716+0000 7f782dffb640 1 --2- 172.21.15.43:0/2912500462 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f781c0afc60 0x7f781c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.716+0000 7f783542f640 1 --2- 172.21.15.43:0/2912500462 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f781c0afc60 0x7f781c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.716+0000 7f7834c2e640 1 --2- 172.21.15.43:0/2235186044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f6a4d0 0x55746a334d50 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f78200dca60 tx=0x7f78200e74a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6cbe01a6aa526a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:02.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.716+0000 7f782dffb640 1 -- 172.21.15.43:0/2912500462 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f78200df370 con 0x7f78301031c0 2024-04-25T10:38:02.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.716+0000 7f7826ffd640 1 -- 172.21.15.43:0/2235186044 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78200a3400 con 0x557469f6a4d0 2024-04-25T10:38:02.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.716+0000 7f7826ffd640 1 -- 172.21.15.43:0/2235186044 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7820005310 con 0x557469f6a4d0 2024-04-25T10:38:02.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.716+0000 7f7826ffd640 1 -- 172.21.15.43:0/2235186044 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782000ce60 con 0x557469f6a4d0 2024-04-25T10:38:02.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.717+0000 7f783de13740 1 -- 172.21.15.43:0/2235186044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f6a4d0 msgr2=0x55746a334d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.717+0000 7f783542f640 1 --2- 172.21.15.43:0/2912500462 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f781c0afc60 0x7f781c0b2170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55746a331980 tx=0x7f78280095b0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:02.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.717+0000 7f783de13740 1 --2- 172.21.15.43:0/2235186044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f6a4d0 0x55746a334d50 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f78200dca60 tx=0x7f78200e74a0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.717+0000 7f783de13740 1 -- 172.21.15.43:0/2235186044 shutdown_connections 2024-04-25T10:38:02.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.717+0000 7f783de13740 1 --2- 172.21.15.43:0/2235186044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557469f726b0 0x55746a3416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.717+0000 7f783de13740 1 --2- 172.21.15.43:0/2235186044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f6a4d0 0x55746a334d50 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.717+0000 7f783de13740 1 --2- 172.21.15.43:0/2235186044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f93390 0x55746a334810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.717+0000 7f783de13740 1 -- 172.21.15.43:0/2235186044 >> 172.21.15.43:0/2235186044 conn(0x557469f874e0 msgr2=0x55746a330fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.717+0000 7f783de13740 1 -- 172.21.15.43:0/2235186044 shutdown_connections 2024-04-25T10:38:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.717+0000 7f783de13740 1 -- 172.21.15.43:0/2235186044 wait complete. 2024-04-25T10:38:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.717+0000 7f783de13740 1 -- start start 2024-04-25T10:38:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.718+0000 7f783de13740 1 Processor -- start 2024-04-25T10:38:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.718+0000 7f783de13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557469f6a4d0 0x55746a33f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.718+0000 7f783de13740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f726b0 0x55746a33f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.718+0000 7f783de13740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f93390 0x55746a34b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.718+0000 7f783de13740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55746a341d90 con 0x557469f93390 2024-04-25T10:38:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.718+0000 7f783de13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55746a220dc0 con 0x557469f6a4d0 2024-04-25T10:38:02.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.718+0000 7f783de13740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55746a341f10 con 0x557469f726b0 2024-04-25T10:38:02.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.718+0000 7f783de13740 10 client.0 _ll_register_callbacks cb 0x55746a23c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:02.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.718+0000 7f783542f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f93390 0x55746a34b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.718+0000 7f783542f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f93390 0x55746a34b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55042/0 (socket says 172.21.15.43:55042) 2024-04-25T10:38:02.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.718+0000 7f783542f640 1 -- 172.21.15.43:0/1450938198 learned_addr learned my addr 172.21.15.43:0/1450938198 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:02.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.718+0000 7f7834c2e640 1 --2- 172.21.15.43:0/1450938198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557469f6a4d0 0x55746a33f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.718+0000 7f782ffff640 1 --2- 172.21.15.43:0/1450938198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f726b0 0x55746a33f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.718+0000 7f783542f640 1 -- 172.21.15.43:0/1450938198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f726b0 msgr2=0x55746a33f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.718+0000 7f783542f640 1 --2- 172.21.15.43:0/1450938198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f726b0 0x55746a33f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.718+0000 7f783542f640 1 -- 172.21.15.43:0/1450938198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557469f6a4d0 msgr2=0x55746a33f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.719+0000 7f783542f640 1 --2- 172.21.15.43:0/1450938198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557469f6a4d0 0x55746a33f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.719+0000 7f783542f640 1 -- 172.21.15.43:0/1450938198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557469fab3f0 con 0x557469f93390 2024-04-25T10:38:02.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.719+0000 7f782ffff640 1 --2- 172.21.15.43:0/1450938198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f726b0 0x55746a33f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:38:02.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.719+0000 7f783542f640 1 --2- 172.21.15.43:0/1450938198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f93390 0x55746a34b290 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f7828012880 tx=0x7f7828004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540b76d1291ea838 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:02.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.719+0000 7f780ffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:02.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.719+0000 7f780ffff640 1 -- 172.21.15.43:0/1450938198 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782800fb60 con 0x557469f93390 2024-04-25T10:38:02.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.719+0000 7f780ffff640 1 -- 172.21.15.43:0/1450938198 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f782800fd00 con 0x557469f93390 2024-04-25T10:38:02.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.719+0000 7f783de13740 1 -- 172.21.15.43:0/1450938198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55746a2a7990 con 0x557469f93390 2024-04-25T10:38:02.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.719+0000 7f780ffff640 1 -- 172.21.15.43:0/1450938198 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f782801f420 con 0x557469f93390 2024-04-25T10:38:02.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.719+0000 7f783de13740 10 client.17076 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:02.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.719+0000 7f783de13740 1 -- 172.21.15.43:0/1450938198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55746a0757d0 con 0x557469f93390 2024-04-25T10:38:02.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.719+0000 7f780ffff640 1 -- 172.21.15.43:0/1450938198 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55746a2a7990 con 0x557469f93390 2024-04-25T10:38:02.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.720+0000 7f780ffff640 1 -- 172.21.15.43:0/1450938198 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7828030720 con 0x557469f93390 2024-04-25T10:38:02.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.720+0000 7f783de13740 10 client.17076 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:02.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.720+0000 7f783de13740 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:02.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.720+0000 7f783de13740 20 client.17076 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:02.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.720+0000 7f783de13740 1 --2- 172.21.15.43:0/1450938198 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557469f51f60 0x55746a351710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.720+0000 7f783de13740 4 client.17076 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:38:02.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.720+0000 7f783de13740 1 -- 172.21.15.43:0/1450938198 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55746a2a1800 con 0x557469f51f60 2024-04-25T10:38:02.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.720+0000 7f7834c2e640 1 --2- 172.21.15.43:0/1450938198 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557469f51f60 0x55746a351710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.720+0000 7f780ffff640 10 client.17076 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:02.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.720+0000 7f7834c2e640 1 --2- 172.21.15.43:0/1450938198 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557469f51f60 0x55746a351710 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1742efe7fa1d6a93 server_cookie=e9a2cdf258761fbd in_seq=0 out_seq=0 2024-04-25T10:38:02.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.721+0000 7f780ffff640 1 -- 172.21.15.43:0/1450938198 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55746a2a1800 con 0x557469f51f60 2024-04-25T10:38:02.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.721+0000 7f780ffff640 10 client.17076 handle_command_reply: tid=0 2024-04-25T10:38:02.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.721+0000 7f783de13740 1 client.17076 shutdown 2024-04-25T10:38:02.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.721+0000 7f783de13740 1 -- 172.21.15.43:0/1450938198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f93390 msgr2=0x55746a34b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.721+0000 7f783de13740 1 --2- 172.21.15.43:0/1450938198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f93390 0x55746a34b290 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f7828012880 tx=0x7f7828004a40 comp rx=0 tx=0).stop 2024-04-25T10:38:02.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.721+0000 7f783de13740 1 -- 172.21.15.43:0/1450938198 shutdown_connections 2024-04-25T10:38:02.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.721+0000 7f783de13740 1 --2- 172.21.15.43:0/1450938198 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557469f51f60 0x55746a351710 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.721+0000 7f783de13740 1 --2- 172.21.15.43:0/1450938198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f93390 0x55746a34b290 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.721+0000 7f783de13740 1 --2- 172.21.15.43:0/1450938198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f726b0 0x55746a33f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.721+0000 7f783de13740 1 --2- 172.21.15.43:0/1450938198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557469f6a4d0 0x55746a33f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.721+0000 7f780ffff640 0 client.17076 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:02.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.721+0000 7f783de13740 1 -- 172.21.15.43:0/1450938198 >> 172.21.15.43:0/1450938198 conn(0x557469f874e0 msgr2=0x55746a336530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:02.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.721+0000 7f783de13740 1 -- 172.21.15.43:0/1450938198 shutdown_connections 2024-04-25T10:38:02.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.721+0000 7f783de13740 1 -- 172.21.15.43:0/1450938198 wait complete. 2024-04-25T10:38:02.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.721+0000 7f783de13740 20 client.17076 trim_cache size 0 max 16384 2024-04-25T10:38:02.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.735+0000 7f783de13740 1 Processor -- start 2024-04-25T10:38:02.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.735+0000 7f783de13740 1 -- start start 2024-04-25T10:38:02.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.735+0000 7f783de13740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f6a4d0 0x55746a33a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.735+0000 7f783de13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557469f726b0 0x55746a33e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.735+0000 7f783de13740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f93390 0x55746a34ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.735+0000 7f783de13740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78301b22f0 con 0x557469f6a4d0 2024-04-25T10:38:02.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.735+0000 7f783de13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78301afbc0 con 0x557469f726b0 2024-04-25T10:38:02.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.735+0000 7f783de13740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f78301ada10 con 0x557469f93390 2024-04-25T10:38:02.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.735+0000 7f7834c2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f6a4d0 0x55746a33a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.735+0000 7f7834c2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f6a4d0 0x55746a33a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55050/0 (socket says 172.21.15.43:55050) 2024-04-25T10:38:02.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.735+0000 7f7834c2e640 1 -- 172.21.15.43:0/1871497115 learned_addr learned my addr 172.21.15.43:0/1871497115 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:02.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.736+0000 7f782ffff640 1 --2- 172.21.15.43:0/1871497115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557469f726b0 0x55746a33e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.736+0000 7f7834c2e640 1 -- 172.21.15.43:0/1871497115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f93390 msgr2=0x55746a34ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.736+0000 7f783542f640 1 --2- 172.21.15.43:0/1871497115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f93390 0x55746a34ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.736+0000 7f7834c2e640 1 --2- 172.21.15.43:0/1871497115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f93390 0x55746a34ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.736+0000 7f7834c2e640 1 -- 172.21.15.43:0/1871497115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557469f726b0 msgr2=0x55746a33e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.736+0000 7f7834c2e640 1 --2- 172.21.15.43:0/1871497115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557469f726b0 0x55746a33e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.736+0000 7f7834c2e640 1 -- 172.21.15.43:0/1871497115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55746a3345a0 con 0x557469f6a4d0 2024-04-25T10:38:02.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.736+0000 7f7834c2e640 1 --2- 172.21.15.43:0/1871497115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f6a4d0 0x55746a33a850 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f78200a7240 tx=0x7f78200900d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf224dfecdd6d9f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:02.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.736+0000 7f78267fc640 1 -- 172.21.15.43:0/1871497115 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782006c070 con 0x557469f6a4d0 2024-04-25T10:38:02.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.736+0000 7f78267fc640 1 -- 172.21.15.43:0/1871497115 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7820005680 con 0x557469f6a4d0 2024-04-25T10:38:02.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.736+0000 7f78267fc640 1 -- 172.21.15.43:0/1871497115 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78200a6cb0 con 0x557469f6a4d0 2024-04-25T10:38:02.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.737+0000 7f783de13740 1 -- 172.21.15.43:0/1871497115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f6a4d0 msgr2=0x55746a33a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.737+0000 7f783de13740 1 --2- 172.21.15.43:0/1871497115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f6a4d0 0x55746a33a850 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f78200a7240 tx=0x7f78200900d0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.737+0000 7f783de13740 1 -- 172.21.15.43:0/1871497115 shutdown_connections 2024-04-25T10:38:02.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.737+0000 7f783de13740 1 --2- 172.21.15.43:0/1871497115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557469f726b0 0x55746a33e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.737+0000 7f783de13740 1 --2- 172.21.15.43:0/1871497115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f93390 0x55746a34ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.737+0000 7f783de13740 1 --2- 172.21.15.43:0/1871497115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f6a4d0 0x55746a33a850 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.737+0000 7f783de13740 1 -- 172.21.15.43:0/1871497115 >> 172.21.15.43:0/1871497115 conn(0x557469f874e0 msgr2=0x55746a33d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:02.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.737+0000 7f783de13740 1 -- 172.21.15.43:0/1871497115 shutdown_connections 2024-04-25T10:38:02.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.737+0000 7f783de13740 1 -- 172.21.15.43:0/1871497115 wait complete. 2024-04-25T10:38:02.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.737+0000 7f783de13740 1 -- start start 2024-04-25T10:38:02.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.737+0000 7f783de13740 1 Processor -- start 2024-04-25T10:38:02.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.738+0000 7f783de13740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f6a4d0 0x55746a3b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.738+0000 7f783de13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557469f726b0 0x55746a3b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.738+0000 7f783de13740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f93390 0x55746a3425c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.738+0000 7f783de13740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78301ae080 con 0x557469f93390 2024-04-25T10:38:02.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.738+0000 7f783de13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78301af4f0 con 0x557469f726b0 2024-04-25T10:38:02.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.738+0000 7f783de13740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f78301ae750 con 0x557469f6a4d0 2024-04-25T10:38:02.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.738+0000 7f783de13740 10 client.0 _ll_register_callbacks cb 0x7f7828012500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:02.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.738+0000 7f783542f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f93390 0x55746a3425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.738+0000 7f783542f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f93390 0x55746a3425c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55060/0 (socket says 172.21.15.43:55060) 2024-04-25T10:38:02.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.738+0000 7f783542f640 1 -- 172.21.15.43:0/2791372672 learned_addr learned my addr 172.21.15.43:0/2791372672 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:02.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.738+0000 7f7834c2e640 1 --2- 172.21.15.43:0/2791372672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f6a4d0 0x55746a3b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.738+0000 7f782ffff640 1 --2- 172.21.15.43:0/2791372672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557469f726b0 0x55746a3b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.738+0000 7f783542f640 1 -- 172.21.15.43:0/2791372672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f6a4d0 msgr2=0x55746a3b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.738+0000 7f783542f640 1 --2- 172.21.15.43:0/2791372672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f6a4d0 0x55746a3b1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.738+0000 7f783542f640 1 -- 172.21.15.43:0/2791372672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557469f726b0 msgr2=0x55746a3b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:02.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.738+0000 7f783542f640 1 --2- 172.21.15.43:0/2791372672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557469f726b0 0x55746a3b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.738+0000 7f783542f640 1 -- 172.21.15.43:0/2791372672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55746a32b2b0 con 0x557469f93390 2024-04-25T10:38:02.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.738+0000 7f783542f640 1 --2- 172.21.15.43:0/2791372672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f93390 0x55746a3425c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f78280124d0 tx=0x7f782800f7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7d8503ad5ae39a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:02.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.738+0000 7f780ffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:02.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.739+0000 7f780ffff640 1 -- 172.21.15.43:0/2791372672 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782803f060 con 0x557469f93390 2024-04-25T10:38:02.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.739+0000 7f780ffff640 1 -- 172.21.15.43:0/2791372672 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f7828004ab0 con 0x557469f93390 2024-04-25T10:38:02.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.739+0000 7f783de13740 1 -- 172.21.15.43:0/2791372672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7820005680 con 0x557469f93390 2024-04-25T10:38:02.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.739+0000 7f780ffff640 1 -- 172.21.15.43:0/2791372672 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7828036070 con 0x557469f93390 2024-04-25T10:38:02.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.739+0000 7f783de13740 10 client.17082 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:02.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.739+0000 7f783de13740 1 -- 172.21.15.43:0/2791372672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55746a3b9290 con 0x557469f93390 2024-04-25T10:38:02.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.739+0000 7f780ffff640 1 -- 172.21.15.43:0/2791372672 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7820005680 con 0x557469f93390 2024-04-25T10:38:02.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.739+0000 7f780ffff640 1 -- 172.21.15.43:0/2791372672 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f782801f860 con 0x557469f93390 2024-04-25T10:38:02.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.739+0000 7f783de13740 10 client.17082 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:02.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.739+0000 7f783de13740 10 client.17082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:02.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.739+0000 7f783de13740 20 client.17082 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:02.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.739+0000 7f783de13740 1 --2- 172.21.15.43:0/2791372672 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557469f273a0 0x55746a3bb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:02.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.739+0000 7f783de13740 4 client.17082 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:38:02.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.739+0000 7f783de13740 1 -- 172.21.15.43:0/2791372672 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55746a21c610 con 0x557469f273a0 2024-04-25T10:38:02.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.740+0000 7f7834c2e640 1 --2- 172.21.15.43:0/2791372672 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557469f273a0 0x55746a3bb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:02.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.740+0000 7f780ffff640 10 client.17082 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:02.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.740+0000 7f7834c2e640 1 --2- 172.21.15.43:0/2791372672 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557469f273a0 0x55746a3bb990 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5973919dff196f9 server_cookie=69d8bd8481a34a9e in_seq=0 out_seq=0 2024-04-25T10:38:02.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.740+0000 7f780ffff640 1 -- 172.21.15.43:0/2791372672 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55746a21c610 con 0x557469f273a0 2024-04-25T10:38:02.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.740+0000 7f780ffff640 10 client.17082 handle_command_reply: tid=0 2024-04-25T10:38:02.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.740+0000 7f783de13740 1 client.17082 shutdown 2024-04-25T10:38:02.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.740+0000 7f783de13740 1 -- 172.21.15.43:0/2791372672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f93390 msgr2=0x55746a3425c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.740+0000 7f783de13740 1 --2- 172.21.15.43:0/2791372672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f93390 0x55746a3425c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f78280124d0 tx=0x7f782800f7a0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.740+0000 7f783de13740 1 -- 172.21.15.43:0/2791372672 shutdown_connections 2024-04-25T10:38:02.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.740+0000 7f783de13740 1 --2- 172.21.15.43:0/2791372672 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557469f273a0 0x55746a3bb990 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.740+0000 7f783de13740 1 --2- 172.21.15.43:0/2791372672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557469f726b0 0x55746a3b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.740+0000 7f780ffff640 0 client.17082 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:02.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.740+0000 7f783de13740 1 --2- 172.21.15.43:0/2791372672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557469f93390 0x55746a3425c0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.740+0000 7f783de13740 1 --2- 172.21.15.43:0/2791372672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557469f6a4d0 0x55746a3b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.740+0000 7f783de13740 1 -- 172.21.15.43:0/2791372672 >> 172.21.15.43:0/2791372672 conn(0x557469f874e0 msgr2=0x55746a330ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:02.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.740+0000 7f783de13740 1 -- 172.21.15.43:0/2791372672 shutdown_connections 2024-04-25T10:38:02.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.740+0000 7f783de13740 1 -- 172.21.15.43:0/2791372672 wait complete. 2024-04-25T10:38:02.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.740+0000 7f783de13740 20 client.17082 trim_cache size 0 max 16384 2024-04-25T10:38:02.751 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:38:02.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.741+0000 7f7836e92640 1 -- 172.21.15.43:0/2912500462 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f781c0afc60 msgr2=0x7f781c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.741+0000 7f7836e92640 1 --2- 172.21.15.43:0/2912500462 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f781c0afc60 0x7f781c0b2170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55746a331980 tx=0x7f78280095b0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.741+0000 7f7836e92640 1 -- 172.21.15.43:0/2912500462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78301031c0 msgr2=0x7f783019b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:02.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.741+0000 7f7836e92640 1 --2- 172.21.15.43:0/2912500462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78301031c0 0x7f783019b5d0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f782000e9c0 tx=0x7f782000ee80 comp rx=0 tx=0).stop 2024-04-25T10:38:02.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.741+0000 7f7836e92640 1 -- 172.21.15.43:0/2912500462 shutdown_connections 2024-04-25T10:38:02.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.741+0000 7f7836e92640 1 --2- 172.21.15.43:0/2912500462 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f781c0afc60 0x7f781c0b2170 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.741+0000 7f7836e92640 1 --2- 172.21.15.43:0/2912500462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7830108a10 0x7f783019bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.741+0000 7f7836e92640 1 --2- 172.21.15.43:0/2912500462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7830108eb0 0x7f783019c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.741+0000 7f7836e92640 1 --2- 172.21.15.43:0/2912500462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78301031c0 0x7f783019b5d0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:02.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.741+0000 7f7836e92640 1 -- 172.21.15.43:0/2912500462 >> 172.21.15.43:0/2912500462 conn(0x7f7830078110 msgr2=0x7f78301040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:02.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.741+0000 7f7836e92640 1 -- 172.21.15.43:0/2912500462 shutdown_connections 2024-04-25T10:38:02.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:02.741+0000 7f7836e92640 1 -- 172.21.15.43:0/2912500462 wait complete. 2024-04-25T10:38:03.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:02 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/12639450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:38:03.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:02 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/12639450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:38:03.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:02 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/12639450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:38:03.755 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:38:03.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:38:03.755 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:38:03.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.884+0000 7fcebf76b640 1 -- 172.21.15.43:0/2328654036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb810b6d0 msgr2=0x7fceb810daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:03.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.884+0000 7fcebf76b640 1 --2- 172.21.15.43:0/2328654036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb810b6d0 0x7fceb810daf0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009900 tx=0x7fcea802a410 comp rx=0 tx=0).stop 2024-04-25T10:38:03.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.884+0000 7fcebf76b640 1 -- 172.21.15.43:0/2328654036 shutdown_connections 2024-04-25T10:38:03.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.884+0000 7fcebf76b640 1 --2- 172.21.15.43:0/2328654036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fceb810e0c0 0x7fceb81105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.884+0000 7fcebf76b640 1 --2- 172.21.15.43:0/2328654036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb810b6d0 0x7fceb810daf0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.884+0000 7fcebf76b640 1 --2- 172.21.15.43:0/2328654036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fceb8104c70 0x7fceb8105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.884+0000 7fcebf76b640 1 -- 172.21.15.43:0/2328654036 >> 172.21.15.43:0/2328654036 conn(0x7fceb8078110 msgr2=0x7fceb8078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:03.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.884+0000 7fcebf76b640 1 -- 172.21.15.43:0/2328654036 shutdown_connections 2024-04-25T10:38:03.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.884+0000 7fcebf76b640 1 -- 172.21.15.43:0/2328654036 wait complete. 2024-04-25T10:38:03.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.885+0000 7fcebf76b640 1 Processor -- start 2024-04-25T10:38:03.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.885+0000 7fcebf76b640 1 -- start start 2024-04-25T10:38:03.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.885+0000 7fcebf76b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fceb8104c70 0x7fceb819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:03.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.885+0000 7fcebf76b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fceb810b6d0 0x7fceb819aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:03.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.885+0000 7fcebf76b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb810e0c0 0x7fceb819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:03.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.885+0000 7fcebf76b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fceb8113ca0 con 0x7fceb810e0c0 2024-04-25T10:38:03.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.885+0000 7fcebf76b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fceb8113b20 con 0x7fceb810b6d0 2024-04-25T10:38:03.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.885+0000 7fcebf76b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fceb8113e20 con 0x7fceb8104c70 2024-04-25T10:38:03.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.886+0000 7fcebdd08640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb810e0c0 0x7fceb819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:03.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.886+0000 7fcebd507640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fceb8104c70 0x7fceb819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:03.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.886+0000 7fcebdd08640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb810e0c0 0x7fceb819ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55074/0 (socket says 172.21.15.43:55074) 2024-04-25T10:38:03.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.886+0000 7fcebdd08640 1 -- 172.21.15.43:0/1973639242 learned_addr learned my addr 172.21.15.43:0/1973639242 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:03.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.886+0000 7fcebcd06640 1 --2- 172.21.15.43:0/1973639242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fceb810b6d0 0x7fceb819aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:03.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.886+0000 7fcebdd08640 1 -- 172.21.15.43:0/1973639242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fceb8104c70 msgr2=0x7fceb819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:03.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.886+0000 7fcebdd08640 1 --2- 172.21.15.43:0/1973639242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fceb8104c70 0x7fceb819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.886+0000 7fcebdd08640 1 -- 172.21.15.43:0/1973639242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fceb810b6d0 msgr2=0x7fceb819aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:03.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.886+0000 7fcebdd08640 1 --2- 172.21.15.43:0/1973639242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fceb810b6d0 0x7fceb819aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.886+0000 7fcebdd08640 1 -- 172.21.15.43:0/1973639242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb819f5a0 con 0x7fceb810e0c0 2024-04-25T10:38:03.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.886+0000 7fcebdd08640 1 --2- 172.21.15.43:0/1973639242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb810e0c0 0x7fceb819ef60 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fceb000e970 tx=0x7fceb000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:03.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.886+0000 7fceae7fc640 1 -- 172.21.15.43:0/1973639242 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb000ccc0 con 0x7fceb810e0c0 2024-04-25T10:38:03.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.886+0000 7fceae7fc640 1 -- 172.21.15.43:0/1973639242 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceb000ce60 con 0x7fceb810e0c0 2024-04-25T10:38:03.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.886+0000 7fceae7fc640 1 -- 172.21.15.43:0/1973639242 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0010430 con 0x7fceb810e0c0 2024-04-25T10:38:03.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.886+0000 7fcebf76b640 1 -- 172.21.15.43:0/1973639242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb819f890 con 0x7fceb810e0c0 2024-04-25T10:38:03.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.886+0000 7fcebf76b640 1 -- 172.21.15.43:0/1973639242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb81a7210 con 0x7fceb810e0c0 2024-04-25T10:38:03.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.889+0000 7fcec6618740 1 Processor -- start 2024-04-25T10:38:03.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.889+0000 7fceae7fc640 1 -- 172.21.15.43:0/1973639242 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fceb0004ff0 con 0x7fceb810e0c0 2024-04-25T10:38:03.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.889+0000 7fcec6618740 1 -- start start 2024-04-25T10:38:03.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.890+0000 7fcec6618740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685eb410 0x55ee6898b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:03.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.890+0000 7fcec6618740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee685c2550 0x55ee6898bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:03.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.890+0000 7fcec6618740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee685ca730 0x55ee6898c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:03.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.890+0000 7fcec6618740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ee688f1b00 con 0x55ee685eb410 2024-04-25T10:38:03.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.890+0000 7fcec6618740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ee6888d920 con 0x55ee685c2550 2024-04-25T10:38:03.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.890+0000 7fcec6618740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ee688edac0 con 0x55ee685ca730 2024-04-25T10:38:03.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.890+0000 7fcebcd06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685eb410 0x55ee6898b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:03.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.890+0000 7fcebcd06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685eb410 0x55ee6898b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55086/0 (socket says 172.21.15.43:55086) 2024-04-25T10:38:03.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.890+0000 7fcebcd06640 1 -- 172.21.15.43:0/2275385338 learned_addr learned my addr 172.21.15.43:0/2275385338 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:03.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.890+0000 7fceae7fc640 1 --2- 172.21.15.43:0/1973639242 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcea40afce0 0x7fcea40b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:03.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.890+0000 7fcebd507640 1 --2- 172.21.15.43:0/2275385338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee685c2550 0x55ee6898bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:03.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.890+0000 7fcebcd06640 1 --2- 172.21.15.43:0/2275385338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee685ca730 0x55ee6898c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:03.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.890+0000 7fcebcd06640 1 -- 172.21.15.43:0/2275385338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee685ca730 msgr2=0x55ee6898c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:03.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.890+0000 7fcebcd06640 1 --2- 172.21.15.43:0/2275385338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee685ca730 0x55ee6898c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.890+0000 7fcebcd06640 1 -- 172.21.15.43:0/2275385338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee685c2550 msgr2=0x55ee6898bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:03.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.890+0000 7fcebcd06640 1 --2- 172.21.15.43:0/2275385338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee685c2550 0x55ee6898bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.890+0000 7fcebdd08640 1 --2- 172.21.15.43:0/1973639242 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcea40afce0 0x7fcea40b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:03.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.890+0000 7fcebcd06640 1 -- 172.21.15.43:0/2275385338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee686cb7e0 con 0x55ee685eb410 2024-04-25T10:38:03.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.890+0000 7fceae7fc640 1 -- 172.21.15.43:0/1973639242 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fceb00df340 con 0x7fceb810e0c0 2024-04-25T10:38:03.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.891+0000 7fcebcd06640 1 --2- 172.21.15.43:0/2275385338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685eb410 0x55ee6898b620 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fcea80096c0 tx=0x7fcea8005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2c50aeb21bffdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:03.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.891+0000 7fcea37fe640 1 -- 172.21.15.43:0/2275385338 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea8044020 con 0x55ee685eb410 2024-04-25T10:38:03.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.891+0000 7fcea37fe640 1 -- 172.21.15.43:0/2275385338 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcea8031720 con 0x55ee685eb410 2024-04-25T10:38:03.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.891+0000 7fcea37fe640 1 -- 172.21.15.43:0/2275385338 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea8035040 con 0x55ee685eb410 2024-04-25T10:38:03.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.892+0000 7fcec6618740 1 -- 172.21.15.43:0/2275385338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685eb410 msgr2=0x55ee6898b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:03.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.892+0000 7fcebdd08640 1 --2- 172.21.15.43:0/1973639242 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcea40afce0 0x7fcea40b21f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fceb00dd0c0 tx=0x7fceb0090040 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:03.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.892+0000 7fcec6618740 1 --2- 172.21.15.43:0/2275385338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685eb410 0x55ee6898b620 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fcea80096c0 tx=0x7fcea8005db0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.892+0000 7fcec6618740 1 -- 172.21.15.43:0/2275385338 shutdown_connections 2024-04-25T10:38:03.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.892+0000 7fcec6618740 1 --2- 172.21.15.43:0/2275385338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee685c2550 0x55ee6898bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.892+0000 7fcec6618740 1 --2- 172.21.15.43:0/2275385338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee685ca730 0x55ee6898c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.892+0000 7fcec6618740 1 --2- 172.21.15.43:0/2275385338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685eb410 0x55ee6898b620 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.892+0000 7fcec6618740 1 -- 172.21.15.43:0/2275385338 >> 172.21.15.43:0/2275385338 conn(0x55ee685df560 msgr2=0x55ee68987dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:03.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.892+0000 7fcec6618740 1 -- 172.21.15.43:0/2275385338 shutdown_connections 2024-04-25T10:38:03.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.892+0000 7fcec6618740 1 -- 172.21.15.43:0/2275385338 wait complete. 2024-04-25T10:38:03.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.892+0000 7fcec6618740 1 -- start start 2024-04-25T10:38:03.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.892+0000 7fcec6618740 1 Processor -- start 2024-04-25T10:38:03.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.893+0000 7fcec6618740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee685c2550 0x55ee68995c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:03.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.893+0000 7fcec6618740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee685ca730 0x55ee68996180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:03.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.893+0000 7fcec6618740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685eb410 0x55ee689a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:03.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.893+0000 7fcec6618740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ee68998910 con 0x55ee685eb410 2024-04-25T10:38:03.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.893+0000 7fcec6618740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ee6888b5d0 con 0x55ee685c2550 2024-04-25T10:38:03.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.893+0000 7fcec6618740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ee68998a90 con 0x55ee685ca730 2024-04-25T10:38:03.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.893+0000 7fcec6618740 10 client.0 _ll_register_callbacks cb 0x55ee68879080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:03.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.893+0000 7fcebcd06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685eb410 0x55ee689a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:03.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.893+0000 7fcebcd06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685eb410 0x55ee689a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55090/0 (socket says 172.21.15.43:55090) 2024-04-25T10:38:03.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.893+0000 7fcebcd06640 1 -- 172.21.15.43:0/221759373 learned_addr learned my addr 172.21.15.43:0/221759373 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:03.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.893+0000 7fcebcd06640 1 --2- 172.21.15.43:0/221759373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee685c2550 0x55ee68995c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:03.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.893+0000 7fcebd507640 1 --2- 172.21.15.43:0/221759373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee685ca730 0x55ee68996180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:03.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.893+0000 7fcebcd06640 1 -- 172.21.15.43:0/221759373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee685ca730 msgr2=0x55ee68996180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:03.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.893+0000 7fcebcd06640 1 --2- 172.21.15.43:0/221759373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee685ca730 0x55ee68996180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.893+0000 7fcebcd06640 1 -- 172.21.15.43:0/221759373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee685c2550 msgr2=0x55ee68995c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:03.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.893+0000 7fcebcd06640 1 --2- 172.21.15.43:0/221759373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee685c2550 0x55ee68995c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.893+0000 7fcebcd06640 1 -- 172.21.15.43:0/221759373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee6872bf70 con 0x55ee685eb410 2024-04-25T10:38:03.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.893+0000 7fcebcd06640 1 --2- 172.21.15.43:0/221759373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685eb410 0x55ee689a1dc0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fcea8005f70 tx=0x7fcea8040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47e413267dec87e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:03.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.893+0000 7fcea0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:03.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.894+0000 7fcea0ff9640 1 -- 172.21.15.43:0/221759373 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea8035070 con 0x55ee685eb410 2024-04-25T10:38:03.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.894+0000 7fcea0ff9640 1 -- 172.21.15.43:0/221759373 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fcea80048c0 con 0x55ee685eb410 2024-04-25T10:38:03.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.894+0000 7fcec6618740 1 -- 172.21.15.43:0/221759373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcea8031720 con 0x55ee685eb410 2024-04-25T10:38:03.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.894+0000 7fcea0ff9640 1 -- 172.21.15.43:0/221759373 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcea805b050 con 0x55ee685eb410 2024-04-25T10:38:03.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.894+0000 7fcec6618740 10 client.17094 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:03.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.894+0000 7fcec6618740 1 -- 172.21.15.43:0/221759373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ee685c3170 con 0x55ee685eb410 2024-04-25T10:38:03.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.894+0000 7fcea0ff9640 1 -- 172.21.15.43:0/221759373 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcea8031720 con 0x55ee685eb410 2024-04-25T10:38:03.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.894+0000 7fcea0ff9640 1 -- 172.21.15.43:0/221759373 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcea804a8b0 con 0x55ee685eb410 2024-04-25T10:38:03.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.894+0000 7fcec6618740 10 client.17094 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:03.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.894+0000 7fcec6618740 10 client.17094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:03.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.894+0000 7fcec6618740 20 client.17094 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:03.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.894+0000 7fcec6618740 1 --2- 172.21.15.43:0/221759373 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ee685aa1f0 0x55ee689a7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:03.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.894+0000 7fcec6618740 4 client.17094 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:38:03.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.894+0000 7fcec6618740 1 -- 172.21.15.43:0/221759373 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee688ff710 con 0x55ee685aa1f0 2024-04-25T10:38:03.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.895+0000 7fcebd507640 1 --2- 172.21.15.43:0/221759373 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ee685aa1f0 0x55ee689a7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:03.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.895+0000 7fcebd507640 1 --2- 172.21.15.43:0/221759373 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ee685aa1f0 0x55ee689a7df0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c4767a316aeb98d server_cookie=c38acc5c0cb57fa9 in_seq=0 out_seq=0 2024-04-25T10:38:03.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.895+0000 7fcea0ff9640 10 client.17094 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:03.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.896+0000 7fcea0ff9640 1 -- 172.21.15.43:0/221759373 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ee688ff710 con 0x55ee685aa1f0 2024-04-25T10:38:03.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.896+0000 7fcea0ff9640 10 client.17094 handle_command_reply: tid=0 2024-04-25T10:38:03.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.896+0000 7fcec6618740 1 client.17094 shutdown 2024-04-25T10:38:03.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.896+0000 7fcec6618740 1 -- 172.21.15.43:0/221759373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685eb410 msgr2=0x55ee689a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:03.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.896+0000 7fcec6618740 1 --2- 172.21.15.43:0/221759373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685eb410 0x55ee689a1dc0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fcea8005f70 tx=0x7fcea8040040 comp rx=0 tx=0).stop 2024-04-25T10:38:03.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.896+0000 7fcec6618740 1 -- 172.21.15.43:0/221759373 shutdown_connections 2024-04-25T10:38:03.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.896+0000 7fcec6618740 1 --2- 172.21.15.43:0/221759373 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ee685aa1f0 0x55ee689a7df0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.896+0000 7fcec6618740 1 --2- 172.21.15.43:0/221759373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685eb410 0x55ee689a1dc0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.896+0000 7fcec6618740 1 --2- 172.21.15.43:0/221759373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee685ca730 0x55ee68996180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.896+0000 7fcea0ff9640 0 client.17094 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:03.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.896+0000 7fcec6618740 1 --2- 172.21.15.43:0/221759373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee685c2550 0x55ee68995c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.896+0000 7fcec6618740 1 -- 172.21.15.43:0/221759373 >> 172.21.15.43:0/221759373 conn(0x55ee685df560 msgr2=0x55ee6898b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:03.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.896+0000 7fcec6618740 1 -- 172.21.15.43:0/221759373 shutdown_connections 2024-04-25T10:38:03.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.896+0000 7fcec6618740 1 -- 172.21.15.43:0/221759373 wait complete. 2024-04-25T10:38:03.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.896+0000 7fcec6618740 20 client.17094 trim_cache size 0 max 16384 2024-04-25T10:38:03.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.915+0000 7fcec6618740 1 Processor -- start 2024-04-25T10:38:03.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.915+0000 7fcec6618740 1 -- start start 2024-04-25T10:38:03.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.915+0000 7fcec6618740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee685c2550 0x55ee68986ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:03.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.915+0000 7fcec6618740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685ca730 0x55ee68995310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:03.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.915+0000 7fcec6618740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee685eb410 0x55ee689a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:03.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.915+0000 7fcec6618740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fceb81b2230 con 0x55ee685ca730 2024-04-25T10:38:03.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.915+0000 7fcec6618740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fceb81afb00 con 0x55ee685eb410 2024-04-25T10:38:03.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.915+0000 7fcec6618740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fceb81ad950 con 0x55ee685c2550 2024-04-25T10:38:03.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.915+0000 7fcebd507640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685ca730 0x55ee68995310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:03.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.915+0000 7fcebd507640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685ca730 0x55ee68995310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55102/0 (socket says 172.21.15.43:55102) 2024-04-25T10:38:03.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.915+0000 7fcebd507640 1 -- 172.21.15.43:0/2010314102 learned_addr learned my addr 172.21.15.43:0/2010314102 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:03.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.915+0000 7fcebd507640 1 -- 172.21.15.43:0/2010314102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee685c2550 msgr2=0x55ee68986ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:03.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.915+0000 7fcebd507640 1 --2- 172.21.15.43:0/2010314102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee685c2550 0x55ee68986ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.915+0000 7fcebd507640 1 -- 172.21.15.43:0/2010314102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee685eb410 msgr2=0x55ee689a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:03.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.915+0000 7fcebd507640 1 --2- 172.21.15.43:0/2010314102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee685eb410 0x55ee689a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.915+0000 7fcebd507640 1 -- 172.21.15.43:0/2010314102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee6898cc10 con 0x55ee685ca730 2024-04-25T10:38:03.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.916+0000 7fcebd507640 1 --2- 172.21.15.43:0/2010314102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685ca730 0x55ee68995310 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fceb4003a80 tx=0x7fceb4009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8aae4d93483ebed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:03.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.916+0000 7fcea2ffd640 1 -- 172.21.15.43:0/2010314102 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb401f020 con 0x55ee685ca730 2024-04-25T10:38:03.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.916+0000 7fcea2ffd640 1 -- 172.21.15.43:0/2010314102 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceb40103e0 con 0x55ee685ca730 2024-04-25T10:38:03.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.916+0000 7fcea2ffd640 1 -- 172.21.15.43:0/2010314102 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4011640 con 0x55ee685ca730 2024-04-25T10:38:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.917+0000 7fcec6618740 1 -- 172.21.15.43:0/2010314102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685ca730 msgr2=0x55ee68995310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.917+0000 7fcec6618740 1 --2- 172.21.15.43:0/2010314102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685ca730 0x55ee68995310 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fceb4003a80 tx=0x7fceb4009680 comp rx=0 tx=0).stop 2024-04-25T10:38:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.917+0000 7fcec6618740 1 -- 172.21.15.43:0/2010314102 shutdown_connections 2024-04-25T10:38:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.917+0000 7fcec6618740 1 --2- 172.21.15.43:0/2010314102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee685eb410 0x55ee689a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.917+0000 7fcec6618740 1 --2- 172.21.15.43:0/2010314102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685ca730 0x55ee68995310 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.917+0000 7fcec6618740 1 --2- 172.21.15.43:0/2010314102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee685c2550 0x55ee68986ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.917+0000 7fcec6618740 1 -- 172.21.15.43:0/2010314102 >> 172.21.15.43:0/2010314102 conn(0x55ee685df560 msgr2=0x55ee6898f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.917+0000 7fcec6618740 1 -- 172.21.15.43:0/2010314102 shutdown_connections 2024-04-25T10:38:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.917+0000 7fcec6618740 1 -- 172.21.15.43:0/2010314102 wait complete. 2024-04-25T10:38:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.917+0000 7fcec6618740 1 -- start start 2024-04-25T10:38:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.917+0000 7fcec6618740 1 Processor -- start 2024-04-25T10:38:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.918+0000 7fcec6618740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685c2550 0x55ee689ab200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.918+0000 7fcec6618740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee685ca730 0x55ee689a8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.918+0000 7fcec6618740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee685eb410 0x55ee689a8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.918+0000 7fcec6618740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fceb81adfc0 con 0x55ee685c2550 2024-04-25T10:38:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.918+0000 7fcec6618740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fceb81af430 con 0x55ee685ca730 2024-04-25T10:38:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.918+0000 7fcec6618740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fceb81ae690 con 0x55ee685eb410 2024-04-25T10:38:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.918+0000 7fcec6618740 10 client.0 _ll_register_callbacks cb 0x7fcea8031610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.918+0000 7fcebcd06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685c2550 0x55ee689ab200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.918+0000 7fcebcd06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685c2550 0x55ee689ab200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55108/0 (socket says 172.21.15.43:55108) 2024-04-25T10:38:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.918+0000 7fcebcd06640 1 -- 172.21.15.43:0/1000065187 learned_addr learned my addr 172.21.15.43:0/1000065187 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.918+0000 7fcebcd06640 1 --2- 172.21.15.43:0/1000065187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee685eb410 0x55ee689a8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.918+0000 7fcebcd06640 1 -- 172.21.15.43:0/1000065187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee685eb410 msgr2=0x55ee689a8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.918+0000 7fcebcd06640 1 --2- 172.21.15.43:0/1000065187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee685eb410 0x55ee689a8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.918+0000 7fcebcd06640 1 -- 172.21.15.43:0/1000065187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee685ca730 msgr2=0x55ee689a8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.918+0000 7fcebd507640 1 --2- 172.21.15.43:0/1000065187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee685ca730 0x55ee689a8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.918+0000 7fcebcd06640 1 --2- 172.21.15.43:0/1000065187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee685ca730 0x55ee689a8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.918+0000 7fcebcd06640 1 -- 172.21.15.43:0/1000065187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee689a9040 con 0x55ee685c2550 2024-04-25T10:38:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.918+0000 7fcebcd06640 1 --2- 172.21.15.43:0/1000065187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685c2550 0x55ee689ab200 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009340 tx=0x7fcea8040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7becffe61a4357 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.918+0000 7fcea0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.919+0000 7fcea0ff9640 1 -- 172.21.15.43:0/1000065187 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea8035030 con 0x55ee685c2550 2024-04-25T10:38:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.919+0000 7fcea0ff9640 1 -- 172.21.15.43:0/1000065187 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fcea804a630 con 0x55ee685c2550 2024-04-25T10:38:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.919+0000 7fcec6618740 1 -- 172.21.15.43:0/1000065187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceb40103e0 con 0x55ee685c2550 2024-04-25T10:38:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.919+0000 7fcea0ff9640 1 -- 172.21.15.43:0/1000065187 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcea8044070 con 0x55ee685c2550 2024-04-25T10:38:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.919+0000 7fcec6618740 10 client.17100 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.919+0000 7fcec6618740 1 -- 172.21.15.43:0/1000065187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ee68a0fca0 con 0x55ee685c2550 2024-04-25T10:38:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.919+0000 7fcea0ff9640 1 -- 172.21.15.43:0/1000065187 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceb40103e0 con 0x55ee685c2550 2024-04-25T10:38:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.919+0000 7fcea0ff9640 1 -- 172.21.15.43:0/1000065187 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcea804a830 con 0x55ee685c2550 2024-04-25T10:38:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.919+0000 7fcec6618740 10 client.17100 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.919+0000 7fcec6618740 10 client.17100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.919+0000 7fcec6618740 20 client.17100 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.919+0000 7fcec6618740 1 --2- 172.21.15.43:0/1000065187 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ee6857f630 0x55ee68a12400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.919+0000 7fcec6618740 4 client.17100 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:38:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.919+0000 7fcec6618740 1 -- 172.21.15.43:0/1000065187 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ee688f9580 con 0x55ee6857f630 2024-04-25T10:38:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.920+0000 7fcebd507640 1 --2- 172.21.15.43:0/1000065187 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ee6857f630 0x55ee68a12400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.920+0000 7fcea0ff9640 10 client.17100 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.920+0000 7fcebd507640 1 --2- 172.21.15.43:0/1000065187 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ee6857f630 0x55ee68a12400 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac8b800642d8d1e9 server_cookie=f101c0098fcaabf in_seq=0 out_seq=0 2024-04-25T10:38:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.921+0000 7fcea0ff9640 1 -- 172.21.15.43:0/1000065187 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ee688f9580 con 0x55ee6857f630 2024-04-25T10:38:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.921+0000 7fcea0ff9640 10 client.17100 handle_command_reply: tid=0 2024-04-25T10:38:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.921+0000 7fcec6618740 1 client.17100 shutdown 2024-04-25T10:38:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.921+0000 7fcec6618740 1 -- 172.21.15.43:0/1000065187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685c2550 msgr2=0x55ee689ab200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.921+0000 7fcec6618740 1 --2- 172.21.15.43:0/1000065187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685c2550 0x55ee689ab200 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009340 tx=0x7fcea8040040 comp rx=0 tx=0).stop 2024-04-25T10:38:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.921+0000 7fcec6618740 1 -- 172.21.15.43:0/1000065187 shutdown_connections 2024-04-25T10:38:03.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.921+0000 7fcec6618740 1 --2- 172.21.15.43:0/1000065187 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ee6857f630 0x55ee68a12400 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.921+0000 7fcec6618740 1 --2- 172.21.15.43:0/1000065187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee685ca730 0x55ee689a8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.921+0000 7fcec6618740 1 --2- 172.21.15.43:0/1000065187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee685eb410 0x55ee689a8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.921+0000 7fcec6618740 1 --2- 172.21.15.43:0/1000065187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee685c2550 0x55ee689ab200 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.921+0000 7fcec6618740 1 -- 172.21.15.43:0/1000065187 >> 172.21.15.43:0/1000065187 conn(0x55ee685df560 msgr2=0x55ee6898f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:03.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.921+0000 7fcea0ff9640 0 client.17100 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:03.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.921+0000 7fcec6618740 1 -- 172.21.15.43:0/1000065187 shutdown_connections 2024-04-25T10:38:03.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.922+0000 7fcec6618740 1 -- 172.21.15.43:0/1000065187 wait complete. 2024-04-25T10:38:03.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.922+0000 7fcec6618740 20 client.17100 trim_cache size 0 max 16384 2024-04-25T10:38:03.928 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:38:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:38:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "9d574763-4568-46f2-800a-b2a9b64cd322", 2024-04-25T10:38:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:38:03.929 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:38:03.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.922+0000 7fcebf76b640 1 -- 172.21.15.43:0/1973639242 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcea40afce0 msgr2=0x7fcea40b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:03.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.922+0000 7fcebf76b640 1 --2- 172.21.15.43:0/1973639242 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcea40afce0 0x7fcea40b21f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fceb00dd0c0 tx=0x7fceb0090040 comp rx=0 tx=0).stop 2024-04-25T10:38:03.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.922+0000 7fcebf76b640 1 -- 172.21.15.43:0/1973639242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb810e0c0 msgr2=0x7fceb819ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:03.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.922+0000 7fcebf76b640 1 --2- 172.21.15.43:0/1973639242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb810e0c0 0x7fceb819ef60 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fceb000e970 tx=0x7fceb000ed70 comp rx=0 tx=0).stop 2024-04-25T10:38:03.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.922+0000 7fcebf76b640 1 -- 172.21.15.43:0/1973639242 shutdown_connections 2024-04-25T10:38:03.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.922+0000 7fcebf76b640 1 --2- 172.21.15.43:0/1973639242 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcea40afce0 0x7fcea40b21f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.922+0000 7fcebf76b640 1 --2- 172.21.15.43:0/1973639242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fceb810b6d0 0x7fceb819aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.922+0000 7fcebf76b640 1 --2- 172.21.15.43:0/1973639242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb810e0c0 0x7fceb819ef60 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.922+0000 7fcebf76b640 1 --2- 172.21.15.43:0/1973639242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fceb8104c70 0x7fceb819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:03.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.922+0000 7fcebf76b640 1 -- 172.21.15.43:0/1973639242 >> 172.21.15.43:0/1973639242 conn(0x7fceb8078110 msgr2=0x7fceb8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:03.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.922+0000 7fcebf76b640 1 -- 172.21.15.43:0/1973639242 shutdown_connections 2024-04-25T10:38:03.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:03.922+0000 7fcebf76b640 1 -- 172.21.15.43:0/1973639242 wait complete. 2024-04-25T10:38:03.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:38:03.935 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:38:04.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.047+0000 7fdbe6fe3640 1 -- 172.21.15.43:0/1918071544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe0107440 msgr2=0x7fdbe010ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:04.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.047+0000 7fdbe6fe3640 1 --2- 172.21.15.43:0/1918071544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe0107440 0x7fdbe010ebe0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009ca0 tx=0x7fdbd802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.047+0000 7fdbe6fe3640 1 -- 172.21.15.43:0/1918071544 shutdown_connections 2024-04-25T10:38:04.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.047+0000 7fdbe6fe3640 1 --2- 172.21.15.43:0/1918071544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbe0106980 0x7fdbe0106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.047+0000 7fdbe6fe3640 1 --2- 172.21.15.43:0/1918071544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe0107440 0x7fdbe010ebe0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.047+0000 7fdbe6fe3640 1 --2- 172.21.15.43:0/1918071544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe0102440 0x7fdbe0102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.047+0000 7fdbe6fe3640 1 -- 172.21.15.43:0/1918071544 >> 172.21.15.43:0/1918071544 conn(0x7fdbe0077880 msgr2=0x7fdbe0077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:04.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.047+0000 7fdbe6fe3640 1 -- 172.21.15.43:0/1918071544 shutdown_connections 2024-04-25T10:38:04.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.047+0000 7fdbe6fe3640 1 -- 172.21.15.43:0/1918071544 wait complete. 2024-04-25T10:38:04.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.048+0000 7fdbe6fe3640 1 Processor -- start 2024-04-25T10:38:04.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.048+0000 7fdbe6fe3640 1 -- start start 2024-04-25T10:38:04.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.048+0000 7fdbe6fe3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbe0102440 0x7fdbe01996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:04.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.048+0000 7fdbe6fe3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe0106980 0x7fdbe0199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:04.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.048+0000 7fdbe6fe3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe0107440 0x7fdbe019df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:04.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.048+0000 7fdbe6fe3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe0112350 con 0x7fdbe0107440 2024-04-25T10:38:04.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.048+0000 7fdbe6fe3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe01121d0 con 0x7fdbe0102440 2024-04-25T10:38:04.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.048+0000 7fdbe6fe3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe01124d0 con 0x7fdbe0106980 2024-04-25T10:38:04.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.049+0000 7fdbe5580640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe0107440 0x7fdbe019df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:04.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.049+0000 7fdbe5580640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe0107440 0x7fdbe019df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55116/0 (socket says 172.21.15.43:55116) 2024-04-25T10:38:04.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.049+0000 7fdbe5580640 1 -- 172.21.15.43:0/1043750513 learned_addr learned my addr 172.21.15.43:0/1043750513 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:04.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.049+0000 7fdbdffff640 1 --2- 172.21.15.43:0/1043750513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe0106980 0x7fdbe0199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:04.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.049+0000 7fdbe4d7f640 1 --2- 172.21.15.43:0/1043750513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbe0102440 0x7fdbe01996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:04.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.049+0000 7fdbe5580640 1 -- 172.21.15.43:0/1043750513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe0106980 msgr2=0x7fdbe0199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:04.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.049+0000 7fdbe5580640 1 --2- 172.21.15.43:0/1043750513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe0106980 0x7fdbe0199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.049+0000 7fdbe5580640 1 -- 172.21.15.43:0/1043750513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbe0102440 msgr2=0x7fdbe01996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.049+0000 7fdbe5580640 1 --2- 172.21.15.43:0/1043750513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbe0102440 0x7fdbe01996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.049+0000 7fdbe5580640 1 -- 172.21.15.43:0/1043750513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe019e540 con 0x7fdbe0107440 2024-04-25T10:38:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.049+0000 7fdbe5580640 1 --2- 172.21.15.43:0/1043750513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe0107440 0x7fdbe019df00 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8005cc0 tx=0x7fdbd802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.049+0000 7fdbddffb640 1 -- 172.21.15.43:0/1043750513 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8003bf0 con 0x7fdbe0107440 2024-04-25T10:38:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.049+0000 7fdbddffb640 1 -- 172.21.15.43:0/1043750513 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd8003d90 con 0x7fdbe0107440 2024-04-25T10:38:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.049+0000 7fdbe6fe3640 1 -- 172.21.15.43:0/1043750513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe019e7d0 con 0x7fdbe0107440 2024-04-25T10:38:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.049+0000 7fdbddffb640 1 -- 172.21.15.43:0/1043750513 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd802fbc0 con 0x7fdbe0107440 2024-04-25T10:38:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.049+0000 7fdbe6fe3640 1 -- 172.21.15.43:0/1043750513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe019ed20 con 0x7fdbe0107440 2024-04-25T10:38:04.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.052+0000 7fdbedf6c740 1 Processor -- start 2024-04-25T10:38:04.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.052+0000 7fdbddffb640 1 -- 172.21.15.43:0/1043750513 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fdbd802fd60 con 0x7fdbe0107440 2024-04-25T10:38:04.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.052+0000 7fdbedf6c740 1 -- start start 2024-04-25T10:38:04.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.052+0000 7fdbedf6c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56485a042390 0x56485a3e4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:04.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.053+0000 7fdbedf6c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 0x56485a3e51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:04.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.053+0000 7fdbedf6c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0216b0 0x56485a3f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:04.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.053+0000 7fdbedf6c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56485a348d80 con 0x56485a0194d0 2024-04-25T10:38:04.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.053+0000 7fdbedf6c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56485a3028b0 con 0x56485a042390 2024-04-25T10:38:04.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.053+0000 7fdbedf6c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56485a344d40 con 0x56485a0216b0 2024-04-25T10:38:04.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.053+0000 7fdbe4d7f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 0x56485a3e51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:04.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.053+0000 7fdbe4d7f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 0x56485a3e51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55118/0 (socket says 172.21.15.43:55118) 2024-04-25T10:38:04.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.053+0000 7fdbe4d7f640 1 -- 172.21.15.43:0/1429278945 learned_addr learned my addr 172.21.15.43:0/1429278945 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:04.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.053+0000 7fdbdffff640 1 --2- 172.21.15.43:0/1429278945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56485a042390 0x56485a3e4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:04.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.053+0000 7fdbddffb640 1 --2- 172.21.15.43:0/1043750513 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdbd00afc60 0x7fdbd00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:04.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.053+0000 7fdbdffff640 1 --2- 172.21.15.43:0/1429278945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0216b0 0x56485a3f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:04.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.053+0000 7fdbe5580640 1 --2- 172.21.15.43:0/1043750513 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdbd00afc60 0x7fdbd00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:04.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.053+0000 7fdbe4d7f640 1 -- 172.21.15.43:0/1429278945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0216b0 msgr2=0x56485a3f1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:04.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.053+0000 7fdbe4d7f640 1 --2- 172.21.15.43:0/1429278945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0216b0 0x56485a3f1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.053+0000 7fdbe4d7f640 1 -- 172.21.15.43:0/1429278945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56485a042390 msgr2=0x56485a3e4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:04.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.053+0000 7fdbe4d7f640 1 --2- 172.21.15.43:0/1429278945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56485a042390 0x56485a3e4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.053+0000 7fdbe4d7f640 1 -- 172.21.15.43:0/1429278945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56485a2d3a40 con 0x56485a0194d0 2024-04-25T10:38:04.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.053+0000 7fdbdffff640 1 --2- 172.21.15.43:0/1429278945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56485a042390 0x56485a3e4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:38:04.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.053+0000 7fdbddffb640 1 -- 172.21.15.43:0/1043750513 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fdbd8100660 con 0x7fdbe0107440 2024-04-25T10:38:04.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.053+0000 7fdbe4d7f640 1 --2- 172.21.15.43:0/1429278945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 0x56485a3e51b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400cbe0 tx=0x7fdbd400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346891462891894a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:04.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.054+0000 7fdbcaffd640 1 -- 172.21.15.43:0/1429278945 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4010630 con 0x56485a0194d0 2024-04-25T10:38:04.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.054+0000 7fdbcaffd640 1 -- 172.21.15.43:0/1429278945 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd4010c80 con 0x56485a0194d0 2024-04-25T10:38:04.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.054+0000 7fdbe5580640 1 --2- 172.21.15.43:0/1043750513 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdbd00afc60 0x7fdbd00b2170 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80fdfa0 tx=0x7fdbd802f660 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:04.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.054+0000 7fdbcaffd640 1 -- 172.21.15.43:0/1429278945 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd401ab50 con 0x56485a0194d0 2024-04-25T10:38:04.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.054+0000 7fdbedf6c740 1 -- 172.21.15.43:0/1429278945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 msgr2=0x56485a3e51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:04.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.054+0000 7fdbedf6c740 1 --2- 172.21.15.43:0/1429278945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 0x56485a3e51b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400cbe0 tx=0x7fdbd400e720 comp rx=0 tx=0).stop 2024-04-25T10:38:04.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.054+0000 7fdbedf6c740 1 -- 172.21.15.43:0/1429278945 shutdown_connections 2024-04-25T10:38:04.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.054+0000 7fdbedf6c740 1 --2- 172.21.15.43:0/1429278945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0216b0 0x56485a3f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.054+0000 7fdbedf6c740 1 --2- 172.21.15.43:0/1429278945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 0x56485a3e51b0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.054+0000 7fdbedf6c740 1 --2- 172.21.15.43:0/1429278945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56485a042390 0x56485a3e4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.054+0000 7fdbedf6c740 1 -- 172.21.15.43:0/1429278945 >> 172.21.15.43:0/1429278945 conn(0x56485a0364e0 msgr2=0x56485a3e1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:04.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.055+0000 7fdbedf6c740 1 -- 172.21.15.43:0/1429278945 shutdown_connections 2024-04-25T10:38:04.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.055+0000 7fdbedf6c740 1 -- 172.21.15.43:0/1429278945 wait complete. 2024-04-25T10:38:04.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.055+0000 7fdbedf6c740 1 -- start start 2024-04-25T10:38:04.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.055+0000 7fdbedf6c740 1 Processor -- start 2024-04-25T10:38:04.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.055+0000 7fdbedf6c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0194d0 0x56485a3ef7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:04.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.055+0000 7fdbedf6c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56485a0216b0 0x56485a3efd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:04.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.055+0000 7fdbedf6c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a042390 0x56485a3f5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:04.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.055+0000 7fdbedf6c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56485a3f21f0 con 0x56485a042390 2024-04-25T10:38:04.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.055+0000 7fdbedf6c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56485a2cfd10 con 0x56485a0216b0 2024-04-25T10:38:04.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.055+0000 7fdbedf6c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56485a3f2370 con 0x56485a0194d0 2024-04-25T10:38:04.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.056+0000 7fdbedf6c740 10 client.0 _ll_register_callbacks cb 0x56485a2fc210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:04.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.056+0000 7fdbdffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a042390 0x56485a3f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:04.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.056+0000 7fdbdffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a042390 0x56485a3f5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55128/0 (socket says 172.21.15.43:55128) 2024-04-25T10:38:04.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.056+0000 7fdbdffff640 1 -- 172.21.15.43:0/3026281942 learned_addr learned my addr 172.21.15.43:0/3026281942 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:04.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.056+0000 7fdbdffff640 1 --2- 172.21.15.43:0/3026281942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0194d0 0x56485a3ef7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:04.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.056+0000 7fdbe4d7f640 1 --2- 172.21.15.43:0/3026281942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56485a0216b0 0x56485a3efd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:04.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.056+0000 7fdbdffff640 1 -- 172.21.15.43:0/3026281942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0194d0 msgr2=0x56485a3ef7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:04.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.056+0000 7fdbdffff640 1 --2- 172.21.15.43:0/3026281942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0194d0 0x56485a3ef7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.056+0000 7fdbdffff640 1 -- 172.21.15.43:0/3026281942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56485a0216b0 msgr2=0x56485a3efd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:04.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.056+0000 7fdbdffff640 1 --2- 172.21.15.43:0/3026281942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56485a0216b0 0x56485a3efd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.056+0000 7fdbdffff640 1 -- 172.21.15.43:0/3026281942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56485a1247d0 con 0x56485a042390 2024-04-25T10:38:04.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.056+0000 7fdbdffff640 1 --2- 172.21.15.43:0/3026281942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a042390 0x56485a3f5700 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc005d00 tx=0x7fdbcc014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3a59d502d8b780 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:04.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.056+0000 7fdbbffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:04.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.056+0000 7fdbbffff640 1 -- 172.21.15.43:0/3026281942 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc010a10 con 0x56485a042390 2024-04-25T10:38:04.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.056+0000 7fdbbffff640 1 -- 172.21.15.43:0/3026281942 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fdbcc010bb0 con 0x56485a042390 2024-04-25T10:38:04.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.056+0000 7fdbedf6c740 1 -- 172.21.15.43:0/3026281942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbd4010c80 con 0x56485a042390 2024-04-25T10:38:04.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.057+0000 7fdbbffff640 1 -- 172.21.15.43:0/3026281942 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdbcc010db0 con 0x56485a042390 2024-04-25T10:38:04.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.057+0000 7fdbedf6c740 10 client.17112 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:04.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.057+0000 7fdbedf6c740 1 -- 172.21.15.43:0/3026281942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56485a05a3f0 con 0x56485a042390 2024-04-25T10:38:04.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.057+0000 7fdbbffff640 1 -- 172.21.15.43:0/3026281942 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd4010c80 con 0x56485a042390 2024-04-25T10:38:04.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.057+0000 7fdbbffff640 1 -- 172.21.15.43:0/3026281942 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdbcc028070 con 0x56485a042390 2024-04-25T10:38:04.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.057+0000 7fdbedf6c740 10 client.17112 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:04.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.057+0000 7fdbedf6c740 10 client.17112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:04.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.057+0000 7fdbedf6c740 20 client.17112 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:04.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.057+0000 7fdbedf6c740 1 --2- 172.21.15.43:0/3026281942 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56485a000f60 0x56485a401970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:04.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.057+0000 7fdbedf6c740 4 client.17112 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:38:04.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.057+0000 7fdbedf6c740 1 -- 172.21.15.43:0/3026281942 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56485a356990 con 0x56485a000f60 2024-04-25T10:38:04.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.057+0000 7fdbe4d7f640 1 --2- 172.21.15.43:0/3026281942 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56485a000f60 0x56485a401970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:04.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.058+0000 7fdbbffff640 10 client.17112 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:04.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.058+0000 7fdbe4d7f640 1 --2- 172.21.15.43:0/3026281942 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56485a000f60 0x56485a401970 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b48ba3b52525b7b4 server_cookie=47911b3670e01143 in_seq=0 out_seq=0 2024-04-25T10:38:04.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.059+0000 7fdbbffff640 1 -- 172.21.15.43:0/3026281942 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56485a356990 con 0x56485a000f60 2024-04-25T10:38:04.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.059+0000 7fdbbffff640 10 client.17112 handle_command_reply: tid=0 2024-04-25T10:38:04.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.059+0000 7fdbedf6c740 1 client.17112 shutdown 2024-04-25T10:38:04.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.059+0000 7fdbedf6c740 1 -- 172.21.15.43:0/3026281942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a042390 msgr2=0x56485a3f5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:04.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.059+0000 7fdbedf6c740 1 --2- 172.21.15.43:0/3026281942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a042390 0x56485a3f5700 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc005d00 tx=0x7fdbcc014040 comp rx=0 tx=0).stop 2024-04-25T10:38:04.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.059+0000 7fdbedf6c740 1 -- 172.21.15.43:0/3026281942 shutdown_connections 2024-04-25T10:38:04.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.059+0000 7fdbedf6c740 1 --2- 172.21.15.43:0/3026281942 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56485a000f60 0x56485a401970 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.059+0000 7fdbedf6c740 1 --2- 172.21.15.43:0/3026281942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56485a0216b0 0x56485a3efd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.059+0000 7fdbedf6c740 1 --2- 172.21.15.43:0/3026281942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a042390 0x56485a3f5700 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.059+0000 7fdbedf6c740 1 --2- 172.21.15.43:0/3026281942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0194d0 0x56485a3ef7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.059+0000 7fdbedf6c740 1 -- 172.21.15.43:0/3026281942 >> 172.21.15.43:0/3026281942 conn(0x56485a0364e0 msgr2=0x56485a3e6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:04.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.059+0000 7fdbbffff640 0 client.17112 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:04.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.059+0000 7fdbedf6c740 1 -- 172.21.15.43:0/3026281942 shutdown_connections 2024-04-25T10:38:04.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.059+0000 7fdbedf6c740 1 -- 172.21.15.43:0/3026281942 wait complete. 2024-04-25T10:38:04.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.059+0000 7fdbedf6c740 20 client.17112 trim_cache size 0 max 16384 2024-04-25T10:38:04.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.077+0000 7fdbedf6c740 1 Processor -- start 2024-04-25T10:38:04.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.077+0000 7fdbedf6c740 1 -- start start 2024-04-25T10:38:04.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.077+0000 7fdbedf6c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 0x56485a3e7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:04.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.077+0000 7fdbedf6c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0216b0 0x56485a3eeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:04.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.077+0000 7fdbedf6c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56485a042390 0x56485a3f5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:04.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.077+0000 7fdbedf6c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe01b0060 con 0x56485a0194d0 2024-04-25T10:38:04.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.077+0000 7fdbedf6c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe01ad930 con 0x56485a042390 2024-04-25T10:38:04.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.077+0000 7fdbedf6c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe01ab780 con 0x56485a0216b0 2024-04-25T10:38:04.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.077+0000 7fdbdffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 0x56485a3e7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:04.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.077+0000 7fdbdffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 0x56485a3e7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55132/0 (socket says 172.21.15.43:55132) 2024-04-25T10:38:04.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.077+0000 7fdbdffff640 1 -- 172.21.15.43:0/1178289077 learned_addr learned my addr 172.21.15.43:0/1178289077 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:04.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.077+0000 7fdbe4d7f640 1 --2- 172.21.15.43:0/1178289077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0216b0 0x56485a3eeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:04.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.078+0000 7fdbdffff640 1 -- 172.21.15.43:0/1178289077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0216b0 msgr2=0x56485a3eeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:04.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.078+0000 7fdbdffff640 1 --2- 172.21.15.43:0/1178289077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0216b0 0x56485a3eeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.078+0000 7fdbdffff640 1 -- 172.21.15.43:0/1178289077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56485a042390 msgr2=0x56485a3f5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:04.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.078+0000 7fdbdffff640 1 --2- 172.21.15.43:0/1178289077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56485a042390 0x56485a3f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.078+0000 7fdbdffff640 1 -- 172.21.15.43:0/1178289077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56485a01a0f0 con 0x56485a0194d0 2024-04-25T10:38:04.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.078+0000 7fdbdffff640 1 --2- 172.21.15.43:0/1178289077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 0x56485a3e7360 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc004b40 tx=0x7fdbcc012580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938d21685ee6849a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:04.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.078+0000 7fdbca7fc640 1 -- 172.21.15.43:0/1178289077 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc02e040 con 0x56485a0194d0 2024-04-25T10:38:04.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.078+0000 7fdbca7fc640 1 -- 172.21.15.43:0/1178289077 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbcc0047a0 con 0x56485a0194d0 2024-04-25T10:38:04.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.078+0000 7fdbca7fc640 1 -- 172.21.15.43:0/1178289077 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc003850 con 0x56485a0194d0 2024-04-25T10:38:04.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.079+0000 7fdbedf6c740 1 -- 172.21.15.43:0/1178289077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 msgr2=0x56485a3e7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:04.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.079+0000 7fdbedf6c740 1 --2- 172.21.15.43:0/1178289077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 0x56485a3e7360 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc004b40 tx=0x7fdbcc012580 comp rx=0 tx=0).stop 2024-04-25T10:38:04.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.079+0000 7fdbedf6c740 1 -- 172.21.15.43:0/1178289077 shutdown_connections 2024-04-25T10:38:04.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.079+0000 7fdbedf6c740 1 --2- 172.21.15.43:0/1178289077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56485a042390 0x56485a3f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.079+0000 7fdbedf6c740 1 --2- 172.21.15.43:0/1178289077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0216b0 0x56485a3eeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.079+0000 7fdbedf6c740 1 --2- 172.21.15.43:0/1178289077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 0x56485a3e7360 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.079+0000 7fdbedf6c740 1 -- 172.21.15.43:0/1178289077 >> 172.21.15.43:0/1178289077 conn(0x56485a0364e0 msgr2=0x56485a3e1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:04.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.079+0000 7fdbedf6c740 1 -- 172.21.15.43:0/1178289077 shutdown_connections 2024-04-25T10:38:04.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.079+0000 7fdbedf6c740 1 -- 172.21.15.43:0/1178289077 wait complete. 2024-04-25T10:38:04.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.079+0000 7fdbedf6c740 1 -- start start 2024-04-25T10:38:04.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.079+0000 7fdbedf6c740 1 Processor -- start 2024-04-25T10:38:04.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.079+0000 7fdbedf6c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 0x56485a3fbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:04.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbedf6c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0216b0 0x56485a3f7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:04.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbedf6c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56485a042390 0x56485a3f7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:04.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbedf6c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe01abdf0 con 0x56485a0194d0 2024-04-25T10:38:04.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbedf6c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe01ad260 con 0x56485a042390 2024-04-25T10:38:04.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbedf6c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe01ac4c0 con 0x56485a0216b0 2024-04-25T10:38:04.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbedf6c740 10 client.0 _ll_register_callbacks cb 0x7fdbcc00bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:04.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbdffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 0x56485a3fbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:04.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbdffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 0x56485a3fbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55136/0 (socket says 172.21.15.43:55136) 2024-04-25T10:38:04.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbdffff640 1 -- 172.21.15.43:0/4225138469 learned_addr learned my addr 172.21.15.43:0/4225138469 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:04.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbe4d7f640 1 --2- 172.21.15.43:0/4225138469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0216b0 0x56485a3f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:04.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbdffff640 1 -- 172.21.15.43:0/4225138469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0216b0 msgr2=0x56485a3f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:04.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbdffff640 1 --2- 172.21.15.43:0/4225138469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0216b0 0x56485a3f7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbdffff640 1 -- 172.21.15.43:0/4225138469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56485a042390 msgr2=0x56485a3f7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:04.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbdffff640 1 --2- 172.21.15.43:0/4225138469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56485a042390 0x56485a3f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbdffff640 1 -- 172.21.15.43:0/4225138469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56485a3094b0 con 0x56485a0194d0 2024-04-25T10:38:04.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbe4d7f640 1 --2- 172.21.15.43:0/4225138469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0216b0 0x56485a3f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:38:04.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbdffff640 1 --2- 172.21.15.43:0/4225138469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 0x56485a3fbf10 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc005d00 tx=0x7fdbcc010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd13044fe2b804da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:04.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbbffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:04.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbbffff640 1 -- 172.21.15.43:0/4225138469 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc02e070 con 0x56485a0194d0 2024-04-25T10:38:04.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbbffff640 1 -- 172.21.15.43:0/4225138469 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fdbcc0037f0 con 0x56485a0194d0 2024-04-25T10:38:04.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.080+0000 7fdbedf6c740 1 -- 172.21.15.43:0/4225138469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbcc0047a0 con 0x56485a0194d0 2024-04-25T10:38:04.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.081+0000 7fdbbffff640 1 -- 172.21.15.43:0/4225138469 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdbcc044050 con 0x56485a0194d0 2024-04-25T10:38:04.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.081+0000 7fdbedf6c740 10 client.17118 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:04.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.081+0000 7fdbedf6c740 1 -- 172.21.15.43:0/4225138469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56485a469650 con 0x56485a0194d0 2024-04-25T10:38:04.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.081+0000 7fdbbffff640 1 -- 172.21.15.43:0/4225138469 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbcc0047a0 con 0x56485a0194d0 2024-04-25T10:38:04.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.081+0000 7fdbbffff640 1 -- 172.21.15.43:0/4225138469 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdbcc028070 con 0x56485a0194d0 2024-04-25T10:38:04.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.081+0000 7fdbedf6c740 10 client.17118 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:04.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.081+0000 7fdbedf6c740 10 client.17118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:04.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.081+0000 7fdbedf6c740 20 client.17118 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:04.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.081+0000 7fdbedf6c740 1 --2- 172.21.15.43:0/4225138469 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564859fd63a0 0x56485a46bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:04.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.081+0000 7fdbedf6c740 4 client.17118 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:38:04.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.081+0000 7fdbedf6c740 1 -- 172.21.15.43:0/4225138469 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56485a350800 con 0x564859fd63a0 2024-04-25T10:38:04.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.081+0000 7fdbe4d7f640 1 --2- 172.21.15.43:0/4225138469 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564859fd63a0 0x56485a46bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:04.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.082+0000 7fdbbffff640 10 client.17118 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:04.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.082+0000 7fdbe4d7f640 1 --2- 172.21.15.43:0/4225138469 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564859fd63a0 0x56485a46bc70 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e08e20a1a3ed220 server_cookie=43c10f6660b0f194 in_seq=0 out_seq=0 2024-04-25T10:38:04.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.082+0000 7fdbbffff640 1 -- 172.21.15.43:0/4225138469 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56485a350800 con 0x564859fd63a0 2024-04-25T10:38:04.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.082+0000 7fdbbffff640 10 client.17118 handle_command_reply: tid=0 2024-04-25T10:38:04.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.082+0000 7fdbedf6c740 1 client.17118 shutdown 2024-04-25T10:38:04.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.082+0000 7fdbedf6c740 1 -- 172.21.15.43:0/4225138469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 msgr2=0x56485a3fbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:04.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.082+0000 7fdbedf6c740 1 --2- 172.21.15.43:0/4225138469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 0x56485a3fbf10 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc005d00 tx=0x7fdbcc010c00 comp rx=0 tx=0).stop 2024-04-25T10:38:04.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.082+0000 7fdbedf6c740 1 -- 172.21.15.43:0/4225138469 shutdown_connections 2024-04-25T10:38:04.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.082+0000 7fdbedf6c740 1 --2- 172.21.15.43:0/4225138469 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564859fd63a0 0x56485a46bc70 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.082+0000 7fdbedf6c740 1 --2- 172.21.15.43:0/4225138469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56485a042390 0x56485a3f7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.082+0000 7fdbedf6c740 1 --2- 172.21.15.43:0/4225138469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56485a0216b0 0x56485a3f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.082+0000 7fdbbffff640 0 client.17118 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:04.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.082+0000 7fdbedf6c740 1 --2- 172.21.15.43:0/4225138469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56485a0194d0 0x56485a3fbf10 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.082+0000 7fdbedf6c740 1 -- 172.21.15.43:0/4225138469 >> 172.21.15.43:0/4225138469 conn(0x56485a0364e0 msgr2=0x56485a3e1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:04.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.082+0000 7fdbedf6c740 1 -- 172.21.15.43:0/4225138469 shutdown_connections 2024-04-25T10:38:04.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.082+0000 7fdbedf6c740 1 -- 172.21.15.43:0/4225138469 wait complete. 2024-04-25T10:38:04.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.082+0000 7fdbedf6c740 20 client.17118 trim_cache size 0 max 16384 2024-04-25T10:38:04.091 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:38:04.091 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:38:04.091 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:38:04.091 INFO:teuthology.orchestra.run.smithi043.stdout: "9d574763-4568-46f2-800a-b2a9b64cd322": { 2024-04-25T10:38:04.091 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:38:04.091 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "9d574763-4568-46f2-800a-b2a9b64cd322", 2024-04-25T10:38:04.091 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:38:04.091 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:38:04.092 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:38:04.092 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:38:04.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.083+0000 7fdbe6fe3640 1 -- 172.21.15.43:0/1043750513 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdbd00afc60 msgr2=0x7fdbd00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:04.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.083+0000 7fdbe6fe3640 1 --2- 172.21.15.43:0/1043750513 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdbd00afc60 0x7fdbd00b2170 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80fdfa0 tx=0x7fdbd802f660 comp rx=0 tx=0).stop 2024-04-25T10:38:04.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.083+0000 7fdbe6fe3640 1 -- 172.21.15.43:0/1043750513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe0107440 msgr2=0x7fdbe019df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:04.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.083+0000 7fdbe6fe3640 1 --2- 172.21.15.43:0/1043750513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe0107440 0x7fdbe019df00 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8005cc0 tx=0x7fdbd802f380 comp rx=0 tx=0).stop 2024-04-25T10:38:04.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.083+0000 7fdbe6fe3640 1 -- 172.21.15.43:0/1043750513 shutdown_connections 2024-04-25T10:38:04.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.083+0000 7fdbe6fe3640 1 --2- 172.21.15.43:0/1043750513 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdbd00afc60 0x7fdbd00b2170 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.083+0000 7fdbe6fe3640 1 --2- 172.21.15.43:0/1043750513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe0107440 0x7fdbe019df00 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.083+0000 7fdbe6fe3640 1 --2- 172.21.15.43:0/1043750513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe0106980 0x7fdbe0199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.083+0000 7fdbe6fe3640 1 --2- 172.21.15.43:0/1043750513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbe0102440 0x7fdbe01996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:04.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.083+0000 7fdbe6fe3640 1 -- 172.21.15.43:0/1043750513 >> 172.21.15.43:0/1043750513 conn(0x7fdbe0077880 msgr2=0x7fdbe01058d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:04.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.083+0000 7fdbe6fe3640 1 -- 172.21.15.43:0/1043750513 shutdown_connections 2024-04-25T10:38:04.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:04.083+0000 7fdbe6fe3640 1 -- 172.21.15.43:0/1043750513 wait complete. 2024-04-25T10:38:04.095 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:03 smithi043 ceph-mon[29478]: pgmap v169: 97 pgs: 97 active+clean; 3.4 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 9 op/s 2024-04-25T10:38:04.096 INFO:tasks.cephfs.filesystem:scrub status for tag:9d574763-4568-46f2-800a-b2a9b64cd322 - {'path': '/', 'tag': '9d574763-4568-46f2-800a-b2a9b64cd322', 'options': 'recursive,force'} 2024-04-25T10:38:04.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:03 smithi077 ceph-mon[32772]: pgmap v169: 97 pgs: 97 active+clean; 3.4 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 9 op/s 2024-04-25T10:38:04.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:03 smithi164 ceph-mon[32713]: pgmap v169: 97 pgs: 97 active+clean; 3.4 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 9 op/s 2024-04-25T10:38:05.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:04 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:38:05.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:04 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:38:05.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:04 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:38:05.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:04 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:38:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:04 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:38:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:04 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:38:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:04 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:38:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:04 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:38:05.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:04 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:38:05.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:04 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:38:05.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:04 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:38:05.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:04 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:38:06.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:05 smithi077 ceph-mon[32772]: pgmap v170: 97 pgs: 97 active+clean; 3.7 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 73 KiB/s wr, 22 op/s 2024-04-25T10:38:06.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:05 smithi164 ceph-mon[32713]: pgmap v170: 97 pgs: 97 active+clean; 3.7 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 73 KiB/s wr, 22 op/s 2024-04-25T10:38:06.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:05 smithi043 ceph-mon[29478]: pgmap v170: 97 pgs: 97 active+clean; 3.7 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 73 KiB/s wr, 22 op/s 2024-04-25T10:38:08.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:07 smithi077 ceph-mon[32772]: pgmap v171: 97 pgs: 97 active+clean; 3.7 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-04-25T10:38:08.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:07 smithi164 ceph-mon[32713]: pgmap v171: 97 pgs: 97 active+clean; 3.7 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-04-25T10:38:08.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:07 smithi043 ceph-mon[29478]: pgmap v171: 97 pgs: 97 active+clean; 3.7 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-04-25T10:38:09.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:08 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:38:09.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:08 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:38:09.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:08 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:38:10.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:09 smithi077 ceph-mon[32772]: pgmap v172: 97 pgs: 97 active+clean; 3.9 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 75 KiB/s wr, 23 op/s 2024-04-25T10:38:10.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:09 smithi164 ceph-mon[32713]: pgmap v172: 97 pgs: 97 active+clean; 3.9 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 75 KiB/s wr, 23 op/s 2024-04-25T10:38:10.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:09 smithi043 ceph-mon[29478]: pgmap v172: 97 pgs: 97 active+clean; 3.9 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 75 KiB/s wr, 23 op/s 2024-04-25T10:38:12.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:11 smithi077 ceph-mon[32772]: pgmap v173: 97 pgs: 97 active+clean; 3.9 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 63 KiB/s wr, 18 op/s 2024-04-25T10:38:12.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:11 smithi164 ceph-mon[32713]: pgmap v173: 97 pgs: 97 active+clean; 3.9 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 63 KiB/s wr, 18 op/s 2024-04-25T10:38:12.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:11 smithi043 ceph-mon[29478]: pgmap v173: 97 pgs: 97 active+clean; 3.9 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 63 KiB/s wr, 18 op/s 2024-04-25T10:38:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.714+0000 7f9dfdd6c640 1 -- 172.21.15.43:0/2083417599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df8108f00 msgr2=0x7f9df8102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.714+0000 7f9dfdd6c640 1 --2- 172.21.15.43:0/2083417599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df8108f00 0x7f9df8102b50 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009900 tx=0x7f9dec02a410 comp rx=0 tx=0).stop 2024-04-25T10:38:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.714+0000 7f9dfdd6c640 1 -- 172.21.15.43:0/2083417599 shutdown_connections 2024-04-25T10:38:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.714+0000 7f9dfdd6c640 1 --2- 172.21.15.43:0/2083417599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9df81031c0 0x7f9df810f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.714+0000 7f9dfdd6c640 1 --2- 172.21.15.43:0/2083417599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df8108f00 0x7f9df8102b50 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.714+0000 7f9dfdd6c640 1 --2- 172.21.15.43:0/2083417599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9df8108a60 0x7f9df8102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.714+0000 7f9dfdd6c640 1 -- 172.21.15.43:0/2083417599 >> 172.21.15.43:0/2083417599 conn(0x7f9df80781b0 msgr2=0x7f9df80785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.714+0000 7f9dfdd6c640 1 -- 172.21.15.43:0/2083417599 shutdown_connections 2024-04-25T10:38:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.714+0000 7f9dfdd6c640 1 -- 172.21.15.43:0/2083417599 wait complete. 2024-04-25T10:38:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.715+0000 7f9dfdd6c640 1 Processor -- start 2024-04-25T10:38:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.715+0000 7f9dfdd6c640 1 -- start start 2024-04-25T10:38:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.715+0000 7f9dfdd6c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df81031c0 0x7f9df81a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.715+0000 7f9dfdd6c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9df8108a60 0x7f9df81a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.715+0000 7f9dfdd6c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9df8108f00 0x7f9df819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.715+0000 7f9dfdd6c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9df8112c80 con 0x7f9df81031c0 2024-04-25T10:38:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.715+0000 7f9dfdd6c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9df8112b00 con 0x7f9df8108f00 2024-04-25T10:38:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.715+0000 7f9dfdd6c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9df8112e00 con 0x7f9df8108a60 2024-04-25T10:38:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.716+0000 7f9df77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df81031c0 0x7f9df81a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.716+0000 7f9df77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df81031c0 0x7f9df81a0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39732/0 (socket says 172.21.15.43:39732) 2024-04-25T10:38:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.716+0000 7f9df77fe640 1 -- 172.21.15.43:0/1391541256 learned_addr learned my addr 172.21.15.43:0/1391541256 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.716+0000 7f9df7fff640 1 --2- 172.21.15.43:0/1391541256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9df8108f00 0x7f9df819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.716+0000 7f9df6ffd640 1 --2- 172.21.15.43:0/1391541256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9df8108a60 0x7f9df81a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.716+0000 7f9df77fe640 1 -- 172.21.15.43:0/1391541256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9df8108a60 msgr2=0x7f9df81a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.716+0000 7f9df77fe640 1 --2- 172.21.15.43:0/1391541256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9df8108a60 0x7f9df81a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.716+0000 7f9df77fe640 1 -- 172.21.15.43:0/1391541256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9df8108f00 msgr2=0x7f9df819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.716+0000 7f9df77fe640 1 --2- 172.21.15.43:0/1391541256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9df8108f00 0x7f9df819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.716+0000 7f9df77fe640 1 -- 172.21.15.43:0/1391541256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df819ae00 con 0x7f9df81031c0 2024-04-25T10:38:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.716+0000 7f9df6ffd640 1 --2- 172.21.15.43:0/1391541256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9df8108a60 0x7f9df81a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:38:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.716+0000 7f9df77fe640 1 --2- 172.21.15.43:0/1391541256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df81031c0 0x7f9df81a0550 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f9de800d740 tx=0x7f9de800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.716+0000 7f9df4ff9640 1 -- 172.21.15.43:0/1391541256 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de8017070 con 0x7f9df81031c0 2024-04-25T10:38:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.716+0000 7f9dfdd6c640 1 -- 172.21.15.43:0/1391541256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df819b090 con 0x7f9df81031c0 2024-04-25T10:38:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.716+0000 7f9df4ff9640 1 -- 172.21.15.43:0/1391541256 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9de800de50 con 0x7f9df81031c0 2024-04-25T10:38:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.716+0000 7f9df4ff9640 1 -- 172.21.15.43:0/1391541256 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de800f620 con 0x7f9df81031c0 2024-04-25T10:38:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.716+0000 7f9dfdd6c640 1 -- 172.21.15.43:0/1391541256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df81a72f0 con 0x7f9df81031c0 2024-04-25T10:38:12.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.719+0000 7f9e04c16740 1 Processor -- start 2024-04-25T10:38:12.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.719+0000 7f9e04c16740 1 -- start start 2024-04-25T10:38:12.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.719+0000 7f9df4ff9640 1 -- 172.21.15.43:0/1391541256 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f9de801a070 con 0x7f9df81031c0 2024-04-25T10:38:12.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.719+0000 7f9e04c16740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843cf1b0 0x55fe8476eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:12.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.719+0000 7f9e04c16740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843a62f0 0x55fe8476f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:12.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.719+0000 7f9e04c16740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe843ae4d0 0x55fe8476f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:12.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.719+0000 7f9e04c16740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fe846d5460 con 0x55fe843cf1b0 2024-04-25T10:38:12.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.719+0000 7f9e04c16740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fe8468f590 con 0x55fe843a62f0 2024-04-25T10:38:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.719+0000 7f9e04c16740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fe846d1420 con 0x55fe843ae4d0 2024-04-25T10:38:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.719+0000 7f9df7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843cf1b0 0x55fe8476eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.720+0000 7f9df7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843cf1b0 0x55fe8476eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39746/0 (socket says 172.21.15.43:39746) 2024-04-25T10:38:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.720+0000 7f9df7fff640 1 -- 172.21.15.43:0/3034032903 learned_addr learned my addr 172.21.15.43:0/3034032903 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.720+0000 7f9df6ffd640 1 --2- 172.21.15.43:0/3034032903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe843ae4d0 0x55fe8476f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.720+0000 7f9df77fe640 1 --2- 172.21.15.43:0/3034032903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843a62f0 0x55fe8476f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.720+0000 7f9df7fff640 1 -- 172.21.15.43:0/3034032903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe843ae4d0 msgr2=0x55fe8476f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.720+0000 7f9df7fff640 1 --2- 172.21.15.43:0/3034032903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe843ae4d0 0x55fe8476f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.720+0000 7f9df7fff640 1 -- 172.21.15.43:0/3034032903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843a62f0 msgr2=0x55fe8476f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.720+0000 7f9df7fff640 1 --2- 172.21.15.43:0/3034032903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843a62f0 0x55fe8476f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.720+0000 7f9df7fff640 1 -- 172.21.15.43:0/3034032903 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe8469c0d0 con 0x55fe843cf1b0 2024-04-25T10:38:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.720+0000 7f9df77fe640 1 --2- 172.21.15.43:0/3034032903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843a62f0 0x55fe8476f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:38:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.720+0000 7f9df4ff9640 1 --2- 172.21.15.43:0/1391541256 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9de00afc50 0x7f9de00b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.720+0000 7f9df7fff640 1 --2- 172.21.15.43:0/3034032903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843cf1b0 0x55fe8476eaf0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f9df0010370 tx=0x7f9df0010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44d002fc55e0822 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.720+0000 7f9df77fe640 1 --2- 172.21.15.43:0/1391541256 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9de00afc50 0x7f9de00b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.720+0000 7f9df4ff9640 1 -- 172.21.15.43:0/1391541256 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f9de80de9b0 con 0x7f9df81031c0 2024-04-25T10:38:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.720+0000 7f9dddffb640 1 -- 172.21.15.43:0/3034032903 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df0016070 con 0x55fe843cf1b0 2024-04-25T10:38:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.720+0000 7f9dddffb640 1 -- 172.21.15.43:0/3034032903 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9df0004d20 con 0x55fe843cf1b0 2024-04-25T10:38:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.720+0000 7f9dddffb640 1 -- 172.21.15.43:0/3034032903 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df0005420 con 0x55fe843cf1b0 2024-04-25T10:38:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.721+0000 7f9e04c16740 1 -- 172.21.15.43:0/3034032903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843cf1b0 msgr2=0x55fe8476eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.721+0000 7f9e04c16740 1 --2- 172.21.15.43:0/3034032903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843cf1b0 0x55fe8476eaf0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f9df0010370 tx=0x7f9df0010830 comp rx=0 tx=0).stop 2024-04-25T10:38:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.721+0000 7f9df77fe640 1 --2- 172.21.15.43:0/1391541256 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9de00afc50 0x7f9de00b2160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9de80dc1e0 tx=0x7f9de8010c70 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.721+0000 7f9e04c16740 1 -- 172.21.15.43:0/3034032903 shutdown_connections 2024-04-25T10:38:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.721+0000 7f9e04c16740 1 --2- 172.21.15.43:0/3034032903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843a62f0 0x55fe8476f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.721+0000 7f9e04c16740 1 --2- 172.21.15.43:0/3034032903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe843ae4d0 0x55fe8476f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.721+0000 7f9e04c16740 1 --2- 172.21.15.43:0/3034032903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843cf1b0 0x55fe8476eaf0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.721+0000 7f9e04c16740 1 -- 172.21.15.43:0/3034032903 >> 172.21.15.43:0/3034032903 conn(0x55fe843c3300 msgr2=0x55fe8476b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.721+0000 7f9e04c16740 1 -- 172.21.15.43:0/3034032903 shutdown_connections 2024-04-25T10:38:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.721+0000 7f9e04c16740 1 -- 172.21.15.43:0/3034032903 wait complete. 2024-04-25T10:38:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.722+0000 7f9e04c16740 1 -- start start 2024-04-25T10:38:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.722+0000 7f9e04c16740 1 Processor -- start 2024-04-25T10:38:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.722+0000 7f9e04c16740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843a62f0 0x55fe8477ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.722+0000 7f9e04c16740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe843ae4d0 0x55fe8477b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.722+0000 7f9e04c16740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843cf1b0 0x55fe84779000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.722+0000 7f9e04c16740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fe8477bc20 con 0x55fe843a62f0 2024-04-25T10:38:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.722+0000 7f9e04c16740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fe8465cb60 con 0x55fe843cf1b0 2024-04-25T10:38:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.722+0000 7f9e04c16740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fe8477bda0 con 0x55fe843ae4d0 2024-04-25T10:38:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.723+0000 7f9e04c16740 10 client.0 _ll_register_callbacks cb 0x55fe846812b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.723+0000 7f9df7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843a62f0 0x55fe8477ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.723+0000 7f9df7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843a62f0 0x55fe8477ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39754/0 (socket says 172.21.15.43:39754) 2024-04-25T10:38:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.723+0000 7f9df7fff640 1 -- 172.21.15.43:0/283385901 learned_addr learned my addr 172.21.15.43:0/283385901 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.723+0000 7f9df6ffd640 1 --2- 172.21.15.43:0/283385901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe843ae4d0 0x55fe8477b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.723+0000 7f9df7fff640 1 --2- 172.21.15.43:0/283385901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843cf1b0 0x55fe84779000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.723+0000 7f9df7fff640 1 -- 172.21.15.43:0/283385901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe843ae4d0 msgr2=0x55fe8477b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.723+0000 7f9df7fff640 1 --2- 172.21.15.43:0/283385901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe843ae4d0 0x55fe8477b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.723+0000 7f9df7fff640 1 -- 172.21.15.43:0/283385901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843cf1b0 msgr2=0x55fe84779000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.723+0000 7f9df7fff640 1 --2- 172.21.15.43:0/283385901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843cf1b0 0x55fe84779000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.723+0000 7f9df7fff640 1 -- 172.21.15.43:0/283385901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe843e7210 con 0x55fe843a62f0 2024-04-25T10:38:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.723+0000 7f9df7fff640 1 --2- 172.21.15.43:0/283385901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843a62f0 0x55fe8477ab30 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f9df0006010 tx=0x7f9df0010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0f81e797f8c5a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.723+0000 7f9dd6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.723+0000 7f9dd6ffd640 1 -- 172.21.15.43:0/283385901 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df0016070 con 0x55fe843a62f0 2024-04-25T10:38:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.724+0000 7f9e04c16740 1 -- 172.21.15.43:0/283385901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9df0004d20 con 0x55fe843a62f0 2024-04-25T10:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.724+0000 7f9dd6ffd640 1 -- 172.21.15.43:0/283385901 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f9df0022040 con 0x55fe843a62f0 2024-04-25T10:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.724+0000 7f9dd6ffd640 1 -- 172.21.15.43:0/283385901 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9df0003b80 con 0x55fe843a62f0 2024-04-25T10:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.724+0000 7f9e04c16740 10 client.17130 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.724+0000 7f9e04c16740 1 -- 172.21.15.43:0/283385901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fe844b3620 con 0x55fe843a62f0 2024-04-25T10:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.724+0000 7f9dd6ffd640 1 -- 172.21.15.43:0/283385901 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9df0004d20 con 0x55fe843a62f0 2024-04-25T10:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.724+0000 7f9dd6ffd640 1 -- 172.21.15.43:0/283385901 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9df000f040 con 0x55fe843a62f0 2024-04-25T10:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.724+0000 7f9e04c16740 10 client.17130 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.724+0000 7f9e04c16740 10 client.17130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.724+0000 7f9e04c16740 20 client.17130 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.724+0000 7f9e04c16740 1 --2- 172.21.15.43:0/283385901 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fe8438dd80 0x55fe8478b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.724+0000 7f9e04c16740 4 client.17130 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.724+0000 7f9e04c16740 1 -- 172.21.15.43:0/283385901 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe846e3070 con 0x55fe8438dd80 2024-04-25T10:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.725+0000 7f9df6ffd640 1 --2- 172.21.15.43:0/283385901 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fe8438dd80 0x55fe8478b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.725+0000 7f9dd6ffd640 10 client.17130 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.725+0000 7f9df6ffd640 1 --2- 172.21.15.43:0/283385901 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fe8438dd80 0x55fe8478b610 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edcd4dd6cc314b36 server_cookie=db89f4320dd99e22 in_seq=0 out_seq=0 2024-04-25T10:38:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.726+0000 7f9dd6ffd640 1 -- 172.21.15.43:0/283385901 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fe846e3070 con 0x55fe8438dd80 2024-04-25T10:38:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.726+0000 7f9dd6ffd640 10 client.17130 handle_command_reply: tid=0 2024-04-25T10:38:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.726+0000 7f9e04c16740 1 client.17130 shutdown 2024-04-25T10:38:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.726+0000 7f9e04c16740 1 -- 172.21.15.43:0/283385901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843a62f0 msgr2=0x55fe8477ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.726+0000 7f9e04c16740 1 --2- 172.21.15.43:0/283385901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843a62f0 0x55fe8477ab30 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f9df0006010 tx=0x7f9df0010fa0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.726+0000 7f9e04c16740 1 -- 172.21.15.43:0/283385901 shutdown_connections 2024-04-25T10:38:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.726+0000 7f9e04c16740 1 --2- 172.21.15.43:0/283385901 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fe8438dd80 0x55fe8478b610 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.726+0000 7f9e04c16740 1 --2- 172.21.15.43:0/283385901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843cf1b0 0x55fe84779000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.726+0000 7f9e04c16740 1 --2- 172.21.15.43:0/283385901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe843ae4d0 0x55fe8477b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.726+0000 7f9dd6ffd640 0 client.17130 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.726+0000 7f9e04c16740 1 --2- 172.21.15.43:0/283385901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843a62f0 0x55fe8477ab30 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.726+0000 7f9e04c16740 1 -- 172.21.15.43:0/283385901 >> 172.21.15.43:0/283385901 conn(0x55fe843c3300 msgr2=0x55fe8476e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.726+0000 7f9e04c16740 1 -- 172.21.15.43:0/283385901 shutdown_connections 2024-04-25T10:38:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.726+0000 7f9e04c16740 1 -- 172.21.15.43:0/283385901 wait complete. 2024-04-25T10:38:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.726+0000 7f9e04c16740 20 client.17130 trim_cache size 0 max 16384 2024-04-25T10:38:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.741+0000 7f9e04c16740 1 Processor -- start 2024-04-25T10:38:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.741+0000 7f9e04c16740 1 -- start start 2024-04-25T10:38:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.742+0000 7f9e04c16740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843a62f0 0x55fe84777f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.742+0000 7f9e04c16740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843ae4d0 0x55fe8477b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.742+0000 7f9e04c16740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe843cf1b0 0x55fe84790be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.742+0000 7f9e04c16740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9df81b2330 con 0x55fe843a62f0 2024-04-25T10:38:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.742+0000 7f9e04c16740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9df81afc00 con 0x55fe843ae4d0 2024-04-25T10:38:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.742+0000 7f9e04c16740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9df81ada50 con 0x55fe843cf1b0 2024-04-25T10:38:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.742+0000 7f9df7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843a62f0 0x55fe84777f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.742+0000 7f9df7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843a62f0 0x55fe84777f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39770/0 (socket says 172.21.15.43:39770) 2024-04-25T10:38:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.742+0000 7f9df7fff640 1 -- 172.21.15.43:0/1723191025 learned_addr learned my addr 172.21.15.43:0/1723191025 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.742+0000 7f9df7fff640 1 -- 172.21.15.43:0/1723191025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe843cf1b0 msgr2=0x55fe84790be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.742+0000 7f9df7fff640 1 --2- 172.21.15.43:0/1723191025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe843cf1b0 0x55fe84790be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.742+0000 7f9df7fff640 1 -- 172.21.15.43:0/1723191025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843ae4d0 msgr2=0x55fe8477b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.742+0000 7f9df6ffd640 1 --2- 172.21.15.43:0/1723191025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843ae4d0 0x55fe8477b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.742+0000 7f9df7fff640 1 --2- 172.21.15.43:0/1723191025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843ae4d0 0x55fe8477b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.742+0000 7f9df7fff640 1 -- 172.21.15.43:0/1723191025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe8477bb00 con 0x55fe843a62f0 2024-04-25T10:38:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.742+0000 7f9df6ffd640 1 --2- 172.21.15.43:0/1723191025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843ae4d0 0x55fe8477b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:38:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.742+0000 7f9df7fff640 1 --2- 172.21.15.43:0/1723191025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843a62f0 0x55fe84777f40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f9df00126d0 tx=0x7f9df0009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8f405d0558f246 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.743+0000 7f9ddd7fa640 1 -- 172.21.15.43:0/1723191025 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df0016040 con 0x55fe843a62f0 2024-04-25T10:38:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.743+0000 7f9ddd7fa640 1 -- 172.21.15.43:0/1723191025 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9df0037020 con 0x55fe843a62f0 2024-04-25T10:38:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.743+0000 7f9ddd7fa640 1 -- 172.21.15.43:0/1723191025 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df001b930 con 0x55fe843a62f0 2024-04-25T10:38:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.743+0000 7f9e04c16740 1 -- 172.21.15.43:0/1723191025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843a62f0 msgr2=0x55fe84777f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.743+0000 7f9e04c16740 1 --2- 172.21.15.43:0/1723191025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843a62f0 0x55fe84777f40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f9df00126d0 tx=0x7f9df0009ef0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.743+0000 7f9e04c16740 1 -- 172.21.15.43:0/1723191025 shutdown_connections 2024-04-25T10:38:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.743+0000 7f9e04c16740 1 --2- 172.21.15.43:0/1723191025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843ae4d0 0x55fe8477b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.743+0000 7f9e04c16740 1 --2- 172.21.15.43:0/1723191025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe843cf1b0 0x55fe84790be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.743+0000 7f9e04c16740 1 --2- 172.21.15.43:0/1723191025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843a62f0 0x55fe84777f40 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.743+0000 7f9e04c16740 1 -- 172.21.15.43:0/1723191025 >> 172.21.15.43:0/1723191025 conn(0x55fe843c3300 msgr2=0x55fe8477a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.743+0000 7f9e04c16740 1 -- 172.21.15.43:0/1723191025 shutdown_connections 2024-04-25T10:38:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.743+0000 7f9e04c16740 1 -- 172.21.15.43:0/1723191025 wait complete. 2024-04-25T10:38:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.744+0000 7f9e04c16740 1 -- start start 2024-04-25T10:38:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.744+0000 7f9e04c16740 1 Processor -- start 2024-04-25T10:38:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.744+0000 7f9e04c16740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843a62f0 0x55fe847e7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.744+0000 7f9e04c16740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843ae4d0 0x55fe847e7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.744+0000 7f9e04c16740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe843cf1b0 0x55fe847e81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.744+0000 7f9e04c16740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9df81ae0c0 con 0x55fe843ae4d0 2024-04-25T10:38:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.744+0000 7f9e04c16740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9df81af530 con 0x55fe843a62f0 2024-04-25T10:38:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.744+0000 7f9e04c16740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9df81ae790 con 0x55fe843cf1b0 2024-04-25T10:38:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.744+0000 7f9e04c16740 10 client.0 _ll_register_callbacks cb 0x7f9dec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.745+0000 7f9df6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843ae4d0 0x55fe847e7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.745+0000 7f9df6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843ae4d0 0x55fe847e7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39772/0 (socket says 172.21.15.43:39772) 2024-04-25T10:38:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.745+0000 7f9df6ffd640 1 -- 172.21.15.43:0/2871305937 learned_addr learned my addr 172.21.15.43:0/2871305937 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.745+0000 7f9df7fff640 1 --2- 172.21.15.43:0/2871305937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe843cf1b0 0x55fe847e81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.745+0000 7f9df7fff640 1 --2- 172.21.15.43:0/2871305937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843a62f0 0x55fe847e7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.745+0000 7f9df6ffd640 1 -- 172.21.15.43:0/2871305937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe843cf1b0 msgr2=0x55fe847e81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.745+0000 7f9df6ffd640 1 --2- 172.21.15.43:0/2871305937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe843cf1b0 0x55fe847e81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.745+0000 7f9df6ffd640 1 -- 172.21.15.43:0/2871305937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843a62f0 msgr2=0x55fe847e7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.745+0000 7f9df6ffd640 1 --2- 172.21.15.43:0/2871305937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843a62f0 0x55fe847e7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.745+0000 7f9df6ffd640 1 -- 172.21.15.43:0/2871305937 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe847f2880 con 0x55fe843ae4d0 2024-04-25T10:38:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.745+0000 7f9df6ffd640 1 --2- 172.21.15.43:0/2871305937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843ae4d0 0x55fe847e7c70 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009230 tx=0x7f9dec0042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41df57585445a57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.745+0000 7f9dd6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.745+0000 7f9dd6ffd640 1 -- 172.21.15.43:0/2871305937 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec035040 con 0x55fe843ae4d0 2024-04-25T10:38:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.745+0000 7f9e04c16740 1 -- 172.21.15.43:0/2871305937 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9df0037020 con 0x55fe843ae4d0 2024-04-25T10:38:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.745+0000 7f9dd6ffd640 1 -- 172.21.15.43:0/2871305937 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f9dec005230 con 0x55fe843ae4d0 2024-04-25T10:38:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.745+0000 7f9dd6ffd640 1 -- 172.21.15.43:0/2871305937 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9dec005430 con 0x55fe843ae4d0 2024-04-25T10:38:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.745+0000 7f9e04c16740 10 client.17136 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.745+0000 7f9e04c16740 1 -- 172.21.15.43:0/2871305937 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fe847f3a40 con 0x55fe843ae4d0 2024-04-25T10:38:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.746+0000 7f9dd6ffd640 1 -- 172.21.15.43:0/2871305937 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9df0037020 con 0x55fe843ae4d0 2024-04-25T10:38:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.746+0000 7f9dd6ffd640 1 -- 172.21.15.43:0/2871305937 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9dec0048f0 con 0x55fe843ae4d0 2024-04-25T10:38:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.746+0000 7f9e04c16740 10 client.17136 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.746+0000 7f9e04c16740 10 client.17136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.746+0000 7f9e04c16740 20 client.17136 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.746+0000 7f9e04c16740 1 --2- 172.21.15.43:0/2871305937 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fe843631c0 0x55fe847f6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.746+0000 7f9e04c16740 4 client.17136 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:38:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.746+0000 7f9e04c16740 1 -- 172.21.15.43:0/2871305937 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fe846dcee0 con 0x55fe843631c0 2024-04-25T10:38:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.746+0000 7f9df7fff640 1 --2- 172.21.15.43:0/2871305937 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fe843631c0 0x55fe847f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.747+0000 7f9dd6ffd640 10 client.17136 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.747+0000 7f9df7fff640 1 --2- 172.21.15.43:0/2871305937 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fe843631c0 0x55fe847f6250 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f7b1e237a053f7b server_cookie=a1387574fe6f8562 in_seq=0 out_seq=0 2024-04-25T10:38:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.765+0000 7f9dd6ffd640 1 -- 172.21.15.43:0/2871305937 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fe846dcee0 con 0x55fe843631c0 2024-04-25T10:38:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.765+0000 7f9dd6ffd640 10 client.17136 handle_command_reply: tid=0 2024-04-25T10:38:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.765+0000 7f9e04c16740 1 client.17136 shutdown 2024-04-25T10:38:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.765+0000 7f9e04c16740 1 -- 172.21.15.43:0/2871305937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843ae4d0 msgr2=0x55fe847e7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.765+0000 7f9e04c16740 1 --2- 172.21.15.43:0/2871305937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843ae4d0 0x55fe847e7c70 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009230 tx=0x7f9dec0042d0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.765+0000 7f9e04c16740 1 -- 172.21.15.43:0/2871305937 shutdown_connections 2024-04-25T10:38:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.765+0000 7f9e04c16740 1 --2- 172.21.15.43:0/2871305937 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fe843631c0 0x55fe847f6250 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.765+0000 7f9e04c16740 1 --2- 172.21.15.43:0/2871305937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe843cf1b0 0x55fe847e81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.765+0000 7f9e04c16740 1 --2- 172.21.15.43:0/2871305937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe843ae4d0 0x55fe847e7c70 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.765+0000 7f9e04c16740 1 --2- 172.21.15.43:0/2871305937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe843a62f0 0x55fe847e7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.765+0000 7f9dd6ffd640 0 client.17136 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.765+0000 7f9e04c16740 1 -- 172.21.15.43:0/2871305937 >> 172.21.15.43:0/2871305937 conn(0x55fe843c3300 msgr2=0x55fe8476e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.765+0000 7f9e04c16740 1 -- 172.21.15.43:0/2871305937 shutdown_connections 2024-04-25T10:38:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.765+0000 7f9e04c16740 1 -- 172.21.15.43:0/2871305937 wait complete. 2024-04-25T10:38:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.765+0000 7f9e04c16740 20 client.17136 trim_cache size 0 max 16384 2024-04-25T10:38:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:38:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:38:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:38:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:38:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.766+0000 7f9dfdd6c640 1 -- 172.21.15.43:0/1391541256 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9de00afc50 msgr2=0x7f9de00b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.766+0000 7f9dfdd6c640 1 --2- 172.21.15.43:0/1391541256 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9de00afc50 0x7f9de00b2160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9de80dc1e0 tx=0x7f9de8010c70 comp rx=0 tx=0).stop 2024-04-25T10:38:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.766+0000 7f9dfdd6c640 1 -- 172.21.15.43:0/1391541256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df81031c0 msgr2=0x7f9df81a0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.766+0000 7f9dfdd6c640 1 --2- 172.21.15.43:0/1391541256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df81031c0 0x7f9df81a0550 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f9de800d740 tx=0x7f9de800dc00 comp rx=0 tx=0).stop 2024-04-25T10:38:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.766+0000 7f9dfdd6c640 1 -- 172.21.15.43:0/1391541256 shutdown_connections 2024-04-25T10:38:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.766+0000 7f9dfdd6c640 1 --2- 172.21.15.43:0/1391541256 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9de00afc50 0x7f9de00b2160 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.766+0000 7f9dfdd6c640 1 --2- 172.21.15.43:0/1391541256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9df8108f00 0x7f9df819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.766+0000 7f9dfdd6c640 1 --2- 172.21.15.43:0/1391541256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9df8108a60 0x7f9df81a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.766+0000 7f9dfdd6c640 1 --2- 172.21.15.43:0/1391541256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df81031c0 0x7f9df81a0550 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.766+0000 7f9dfdd6c640 1 -- 172.21.15.43:0/1391541256 >> 172.21.15.43:0/1391541256 conn(0x7f9df80781b0 msgr2=0x7f9df806a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.766+0000 7f9dfdd6c640 1 -- 172.21.15.43:0/1391541256 shutdown_connections 2024-04-25T10:38:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:12.766+0000 7f9dfdd6c640 1 -- 172.21.15.43:0/1391541256 wait complete. 2024-04-25T10:38:14.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:13 smithi077 ceph-mon[32772]: pgmap v174: 97 pgs: 97 active+clean; 3.9 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 63 KiB/s wr, 18 op/s 2024-04-25T10:38:14.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:13 smithi164 ceph-mon[32713]: pgmap v174: 97 pgs: 97 active+clean; 3.9 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 63 KiB/s wr, 18 op/s 2024-04-25T10:38:14.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:13 smithi043 ceph-mon[29478]: pgmap v174: 97 pgs: 97 active+clean; 3.9 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 63 KiB/s wr, 18 op/s 2024-04-25T10:38:16.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:15 smithi077 ceph-mon[32772]: pgmap v175: 97 pgs: 97 active+clean; 4.1 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 84 KiB/s wr, 26 op/s 2024-04-25T10:38:16.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:15 smithi164 ceph-mon[32713]: pgmap v175: 97 pgs: 97 active+clean; 4.1 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 84 KiB/s wr, 26 op/s 2024-04-25T10:38:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:15 smithi043 ceph-mon[29478]: pgmap v175: 97 pgs: 97 active+clean; 4.1 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 84 KiB/s wr, 26 op/s 2024-04-25T10:38:18.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:17 smithi077 ceph-mon[32772]: pgmap v176: 97 pgs: 97 active+clean; 4.1 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-25T10:38:18.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:17 smithi164 ceph-mon[32713]: pgmap v176: 97 pgs: 97 active+clean; 4.1 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-25T10:38:18.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:17 smithi043 ceph-mon[29478]: pgmap v176: 97 pgs: 97 active+clean; 4.1 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-25T10:38:20.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:19 smithi077 ceph-mon[32772]: pgmap v177: 97 pgs: 97 active+clean; 4.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 19 op/s 2024-04-25T10:38:20.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:19 smithi164 ceph-mon[32713]: pgmap v177: 97 pgs: 97 active+clean; 4.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 19 op/s 2024-04-25T10:38:20.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:19 smithi043 ceph-mon[29478]: pgmap v177: 97 pgs: 97 active+clean; 4.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 19 op/s 2024-04-25T10:38:21.837 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.837 INFO:tasks.workunit.client.0.smithi043.stdout:189/600: open fd = 3 2024-04-25T10:38:21.837 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.838 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.838 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.838 INFO:tasks.workunit.client.0.smithi043.stdout:190/600: open fd = 3 2024-04-25T10:38:21.838 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.838 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.838 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.838 INFO:tasks.workunit.client.0.smithi043.stdout:191/600: open fd = 3 2024-04-25T10:38:21.838 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.838 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.838 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.838 INFO:tasks.workunit.client.0.smithi043.stdout:192/600: open fd = 3 2024-04-25T10:38:21.838 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.838 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.839 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.839 INFO:tasks.workunit.client.0.smithi043.stdout:193/600: open fd = 3 2024-04-25T10:38:21.839 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.839 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.839 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.839 INFO:tasks.workunit.client.0.smithi043.stdout:194/600: open fd = 3 2024-04-25T10:38:21.839 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.839 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.839 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.839 INFO:tasks.workunit.client.0.smithi043.stdout:195/600: open fd = 3 2024-04-25T10:38:21.839 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.839 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.839 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.839 INFO:tasks.workunit.client.0.smithi043.stdout:196/600: open fd = 3 2024-04-25T10:38:21.839 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.840 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.840 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.840 INFO:tasks.workunit.client.0.smithi043.stdout:197/600: open fd = 3 2024-04-25T10:38:21.840 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.840 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.840 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.840 INFO:tasks.workunit.client.0.smithi043.stdout:198/600: open fd = 3 2024-04-25T10:38:21.840 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.840 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.840 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.840 INFO:tasks.workunit.client.0.smithi043.stdout:199/600: open fd = 3 2024-04-25T10:38:21.840 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.840 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.840 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.840 INFO:tasks.workunit.client.0.smithi043.stdout:200/600: open fd = 3 2024-04-25T10:38:21.841 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.841 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.841 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.841 INFO:tasks.workunit.client.0.smithi043.stdout:201/600: open fd = 3 2024-04-25T10:38:21.841 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.841 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.841 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.841 INFO:tasks.workunit.client.0.smithi043.stdout:202/600: open fd = 3 2024-04-25T10:38:21.841 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.841 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.841 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.841 INFO:tasks.workunit.client.0.smithi043.stdout:203/600: open fd = 3 2024-04-25T10:38:21.841 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.841 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.841 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.842 INFO:tasks.workunit.client.0.smithi043.stdout:204/600: open fd = 3 2024-04-25T10:38:21.842 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.842 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.842 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.842 INFO:tasks.workunit.client.0.smithi043.stdout:205/600: open fd = 3 2024-04-25T10:38:21.842 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.842 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.842 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.842 INFO:tasks.workunit.client.0.smithi043.stdout:206/600: open fd = 3 2024-04-25T10:38:21.842 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.842 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.842 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.842 INFO:tasks.workunit.client.0.smithi043.stdout:207/600: open fd = 3 2024-04-25T10:38:21.843 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.843 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.843 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.843 INFO:tasks.workunit.client.0.smithi043.stdout:208/600: open fd = 3 2024-04-25T10:38:21.843 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.843 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.843 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.843 INFO:tasks.workunit.client.0.smithi043.stdout:209/600: open fd = 3 2024-04-25T10:38:21.843 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.843 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.843 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.843 INFO:tasks.workunit.client.0.smithi043.stdout:210/600: open fd = 3 2024-04-25T10:38:21.844 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.844 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.844 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.844 INFO:tasks.workunit.client.0.smithi043.stdout:211/600: open fd = 3 2024-04-25T10:38:21.844 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.844 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.844 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.844 INFO:tasks.workunit.client.0.smithi043.stdout:212/600: open fd = 3 2024-04-25T10:38:21.844 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.844 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.844 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.844 INFO:tasks.workunit.client.0.smithi043.stdout:213/600: open fd = 3 2024-04-25T10:38:21.844 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.844 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.844 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.845 INFO:tasks.workunit.client.0.smithi043.stdout:214/600: open fd = 3 2024-04-25T10:38:21.845 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.845 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.845 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.845 INFO:tasks.workunit.client.0.smithi043.stdout:215/600: open fd = 3 2024-04-25T10:38:21.845 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.845 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.845 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.845 INFO:tasks.workunit.client.0.smithi043.stdout:216/600: open fd = 3 2024-04-25T10:38:21.845 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.845 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.845 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.845 INFO:tasks.workunit.client.0.smithi043.stdout:217/600: open fd = 3 2024-04-25T10:38:21.845 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.845 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.846 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.846 INFO:tasks.workunit.client.0.smithi043.stdout:218/600: open fd = 3 2024-04-25T10:38:21.846 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.846 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.846 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.846 INFO:tasks.workunit.client.0.smithi043.stdout:219/600: open fd = 3 2024-04-25T10:38:21.846 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.846 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.846 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.846 INFO:tasks.workunit.client.0.smithi043.stdout:220/600: open fd = 3 2024-04-25T10:38:21.846 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.846 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.846 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.847 INFO:tasks.workunit.client.0.smithi043.stdout:221/600: open fd = 3 2024-04-25T10:38:21.847 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.847 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.847 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.847 INFO:tasks.workunit.client.0.smithi043.stdout:222/600: open fd = 3 2024-04-25T10:38:21.847 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.847 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.847 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.847 INFO:tasks.workunit.client.0.smithi043.stdout:223/600: open fd = 3 2024-04-25T10:38:21.847 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.847 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.847 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.847 INFO:tasks.workunit.client.0.smithi043.stdout:224/600: open fd = 3 2024-04-25T10:38:21.847 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.847 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.848 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.848 INFO:tasks.workunit.client.0.smithi043.stdout:225/600: open fd = 3 2024-04-25T10:38:21.848 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.848 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.848 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.848 INFO:tasks.workunit.client.0.smithi043.stdout:226/600: open fd = 3 2024-04-25T10:38:21.848 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.848 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.848 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.848 INFO:tasks.workunit.client.0.smithi043.stdout:227/600: open fd = 3 2024-04-25T10:38:21.848 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.848 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.848 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.848 INFO:tasks.workunit.client.0.smithi043.stdout:228/600: open fd = 3 2024-04-25T10:38:21.848 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.849 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.849 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.849 INFO:tasks.workunit.client.0.smithi043.stdout:229/600: open fd = 3 2024-04-25T10:38:21.849 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.849 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.849 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.849 INFO:tasks.workunit.client.0.smithi043.stdout:230/600: open fd = 3 2024-04-25T10:38:21.849 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.849 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.849 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.849 INFO:tasks.workunit.client.0.smithi043.stdout:231/600: open fd = 3 2024-04-25T10:38:21.849 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.849 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.849 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.850 INFO:tasks.workunit.client.0.smithi043.stdout:232/600: open fd = 3 2024-04-25T10:38:21.850 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.850 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.850 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.850 INFO:tasks.workunit.client.0.smithi043.stdout:233/600: open fd = 3 2024-04-25T10:38:21.850 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.850 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.850 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.850 INFO:tasks.workunit.client.0.smithi043.stdout:234/600: open fd = 3 2024-04-25T10:38:21.850 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.850 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.850 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.850 INFO:tasks.workunit.client.0.smithi043.stdout:235/600: open fd = 3 2024-04-25T10:38:21.850 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.850 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.851 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.851 INFO:tasks.workunit.client.0.smithi043.stdout:236/600: open fd = 3 2024-04-25T10:38:21.851 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.851 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.851 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.851 INFO:tasks.workunit.client.0.smithi043.stdout:237/600: open fd = 3 2024-04-25T10:38:21.851 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.851 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.851 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.851 INFO:tasks.workunit.client.0.smithi043.stdout:238/600: open fd = 3 2024-04-25T10:38:21.851 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.851 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.851 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.851 INFO:tasks.workunit.client.0.smithi043.stdout:239/600: open fd = 3 2024-04-25T10:38:21.851 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.852 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.852 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.852 INFO:tasks.workunit.client.0.smithi043.stdout:240/600: open fd = 3 2024-04-25T10:38:21.852 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.852 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.852 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.852 INFO:tasks.workunit.client.0.smithi043.stdout:241/600: open fd = 3 2024-04-25T10:38:21.852 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.852 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.852 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.852 INFO:tasks.workunit.client.0.smithi043.stdout:242/600: open fd = 3 2024-04-25T10:38:21.852 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.852 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.852 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.852 INFO:tasks.workunit.client.0.smithi043.stdout:243/600: open fd = 3 2024-04-25T10:38:21.853 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.853 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.853 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.853 INFO:tasks.workunit.client.0.smithi043.stdout:244/600: open fd = 3 2024-04-25T10:38:21.853 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.853 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.853 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.853 INFO:tasks.workunit.client.0.smithi043.stdout:245/600: open fd = 3 2024-04-25T10:38:21.853 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.853 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.853 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.853 INFO:tasks.workunit.client.0.smithi043.stdout:246/600: open fd = 3 2024-04-25T10:38:21.853 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.853 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.854 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.854 INFO:tasks.workunit.client.0.smithi043.stdout:247/600: open fd = 3 2024-04-25T10:38:21.854 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.854 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.854 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.854 INFO:tasks.workunit.client.0.smithi043.stdout:248/600: open fd = 3 2024-04-25T10:38:21.854 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.854 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.854 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.854 INFO:tasks.workunit.client.0.smithi043.stdout:249/600: open fd = 3 2024-04-25T10:38:21.854 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.854 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.854 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:38:21.854 INFO:tasks.workunit.client.0.smithi043.stdout:250/600: open fd = 3 2024-04-25T10:38:21.854 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:21.855 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:38:22.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:21 smithi077 ceph-mon[32772]: pgmap v178: 97 pgs: 97 active+clean; 4.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:38:22.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:21 smithi164 ceph-mon[32713]: pgmap v178: 97 pgs: 97 active+clean; 4.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:38:22.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:21 smithi043 ceph-mon[29478]: pgmap v178: 97 pgs: 97 active+clean; 4.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:38:24.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:23 smithi077 ceph-mon[32772]: pgmap v179: 97 pgs: 97 active+clean; 4.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:38:24.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:23 smithi164 ceph-mon[32713]: pgmap v179: 97 pgs: 97 active+clean; 4.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:38:24.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:23 smithi043 ceph-mon[29478]: pgmap v179: 97 pgs: 97 active+clean; 4.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:38:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.877+0000 7f59f5f26640 1 -- 172.21.15.43:0/2925119564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59f010a000 msgr2=0x7f59f0102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.877+0000 7f59f5f26640 1 --2- 172.21.15.43:0/2925119564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59f010a000 0x7f59f0102170 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009900 tx=0x7f59e402a410 comp rx=0 tx=0).stop 2024-04-25T10:38:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.877+0000 7f59f5f26640 1 -- 172.21.15.43:0/2925119564 shutdown_connections 2024-04-25T10:38:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.877+0000 7f59f5f26640 1 --2- 172.21.15.43:0/2925119564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f0102800 0x7f59f0110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.877+0000 7f59f5f26640 1 --2- 172.21.15.43:0/2925119564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59f010a000 0x7f59f0102170 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.877+0000 7f59f5f26640 1 --2- 172.21.15.43:0/2925119564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59f0109b60 0x7f59f0101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.877+0000 7f59f5f26640 1 -- 172.21.15.43:0/2925119564 >> 172.21.15.43:0/2925119564 conn(0x7f59f00fd8a0 msgr2=0x7f59f00ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.877+0000 7f59f5f26640 1 -- 172.21.15.43:0/2925119564 shutdown_connections 2024-04-25T10:38:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.877+0000 7f59f5f26640 1 -- 172.21.15.43:0/2925119564 wait complete. 2024-04-25T10:38:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.878+0000 7f59f5f26640 1 Processor -- start 2024-04-25T10:38:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.878+0000 7f59f5f26640 1 -- start start 2024-04-25T10:38:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.879+0000 7f59f5f26640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f0109b60 0x7f59f0199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.879+0000 7f59f5f26640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59f019a4f0 0x7f59f019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.879+0000 7f59f5f26640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59f019ed10 0x7f59f019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.879+0000 7f59f5f26640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f59f0113d50 con 0x7f59f019ed10 2024-04-25T10:38:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.879+0000 7f59f5f26640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f59f0113bd0 con 0x7f59f0109b60 2024-04-25T10:38:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.879+0000 7f59f5f26640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f59f0113ed0 con 0x7f59f019a4f0 2024-04-25T10:38:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.879+0000 7f59effff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59f019ed10 0x7f59f019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.879+0000 7f59ef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f0109b60 0x7f59f0199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.879+0000 7f59effff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59f019ed10 0x7f59f019f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51614/0 (socket says 172.21.15.43:51614) 2024-04-25T10:38:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.879+0000 7f59effff640 1 -- 172.21.15.43:0/700032136 learned_addr learned my addr 172.21.15.43:0/700032136 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.879+0000 7f59eeffd640 1 --2- 172.21.15.43:0/700032136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59f019a4f0 0x7f59f019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.879+0000 7f59effff640 1 -- 172.21.15.43:0/700032136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59f019a4f0 msgr2=0x7f59f019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.879+0000 7f59effff640 1 --2- 172.21.15.43:0/700032136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59f019a4f0 0x7f59f019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.879+0000 7f59effff640 1 -- 172.21.15.43:0/700032136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f0109b60 msgr2=0x7f59f0199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.879+0000 7f59effff640 1 --2- 172.21.15.43:0/700032136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f0109b60 0x7f59f0199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.879+0000 7f59effff640 1 -- 172.21.15.43:0/700032136 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f01a6a80 con 0x7f59f019ed10 2024-04-25T10:38:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.879+0000 7f59effff640 1 --2- 172.21.15.43:0/700032136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59f019ed10 0x7f59f019f120 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f59e800eaa0 tx=0x7f59e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.880+0000 7f59ecff9640 1 -- 172.21.15.43:0/700032136 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e800cdb0 con 0x7f59f019ed10 2024-04-25T10:38:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.880+0000 7f59f5f26640 1 -- 172.21.15.43:0/700032136 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f01a6c50 con 0x7f59f019ed10 2024-04-25T10:38:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.880+0000 7f59ecff9640 1 -- 172.21.15.43:0/700032136 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59e8005020 con 0x7f59f019ed10 2024-04-25T10:38:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.880+0000 7f59ecff9640 1 -- 172.21.15.43:0/700032136 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e80116d0 con 0x7f59f019ed10 2024-04-25T10:38:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.880+0000 7f59f5f26640 1 -- 172.21.15.43:0/700032136 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f01a7110 con 0x7f59f019ed10 2024-04-25T10:38:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.882+0000 7f59fcdd8740 1 Processor -- start 2024-04-25T10:38:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.882+0000 7f59ecff9640 1 -- 172.21.15.43:0/700032136 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f59e80051d0 con 0x7f59f019ed10 2024-04-25T10:38:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.882+0000 7f59fcdd8740 1 -- start start 2024-04-25T10:38:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.883+0000 7f59fcdd8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a91b3441b0 0x55a91b6e3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.883+0000 7f59fcdd8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a91b31b2f0 0x55a91b6e4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.883+0000 7f59fcdd8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 0x55a91b6e4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.883+0000 7f59fcdd8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a91b64a460 con 0x55a91b3234d0 2024-04-25T10:38:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.883+0000 7f59fcdd8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a91b604590 con 0x55a91b31b2f0 2024-04-25T10:38:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.883+0000 7f59fcdd8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a91b646420 con 0x55a91b3441b0 2024-04-25T10:38:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.883+0000 7f59eeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 0x55a91b6e4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.883+0000 7f59eeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 0x55a91b6e4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51630/0 (socket says 172.21.15.43:51630) 2024-04-25T10:38:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.883+0000 7f59eeffd640 1 -- 172.21.15.43:0/1989244252 learned_addr learned my addr 172.21.15.43:0/1989244252 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.883+0000 7f59ef7fe640 1 --2- 172.21.15.43:0/1989244252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a91b31b2f0 0x55a91b6e4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.883+0000 7f59eeffd640 1 --2- 172.21.15.43:0/1989244252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a91b3441b0 0x55a91b6e3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.883+0000 7f59ecff9640 1 --2- 172.21.15.43:0/700032136 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f59d80afc50 0x7f59d80b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.883+0000 7f59ef7fe640 1 -- 172.21.15.43:0/1989244252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a91b3441b0 msgr2=0x55a91b6e3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.883+0000 7f59ef7fe640 1 --2- 172.21.15.43:0/1989244252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a91b3441b0 0x55a91b6e3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.883+0000 7f59ef7fe640 1 -- 172.21.15.43:0/1989244252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 msgr2=0x55a91b6e4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.883+0000 7f59ef7fe640 1 --2- 172.21.15.43:0/1989244252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 0x55a91b6e4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.883+0000 7f59ef7fe640 1 -- 172.21.15.43:0/1989244252 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a91b6110d0 con 0x55a91b31b2f0 2024-04-25T10:38:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.883+0000 7f59eeffd640 1 --2- 172.21.15.43:0/1989244252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 0x55a91b6e4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:38:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.883+0000 7f59effff640 1 --2- 172.21.15.43:0/700032136 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f59d80afc50 0x7f59d80b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.883+0000 7f59ecff9640 1 -- 172.21.15.43:0/700032136 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f59e80df0c0 con 0x7f59f019ed10 2024-04-25T10:38:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.884+0000 7f59ef7fe640 1 --2- 172.21.15.43:0/1989244252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a91b31b2f0 0x55a91b6e4030 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f59e0005fd0 tx=0x7f59e00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=529b7e487292f5ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.884+0000 7f59d5ffb640 1 -- 172.21.15.43:0/1989244252 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e001c480 con 0x55a91b31b2f0 2024-04-25T10:38:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.884+0000 7f59effff640 1 --2- 172.21.15.43:0/700032136 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f59d80afc50 0x7f59d80b2160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f59e80dc990 tx=0x7f59e809bcd0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.884+0000 7f59d5ffb640 1 -- 172.21.15.43:0/1989244252 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59e001cad0 con 0x55a91b31b2f0 2024-04-25T10:38:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.884+0000 7f59d5ffb640 1 -- 172.21.15.43:0/1989244252 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0011930 con 0x55a91b31b2f0 2024-04-25T10:38:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.885+0000 7f59fcdd8740 1 -- 172.21.15.43:0/1989244252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a91b31b2f0 msgr2=0x55a91b6e4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.885+0000 7f59fcdd8740 1 --2- 172.21.15.43:0/1989244252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a91b31b2f0 0x55a91b6e4030 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f59e0005fd0 tx=0x7f59e00107a0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.885+0000 7f59fcdd8740 1 -- 172.21.15.43:0/1989244252 shutdown_connections 2024-04-25T10:38:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.885+0000 7f59fcdd8740 1 --2- 172.21.15.43:0/1989244252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a91b31b2f0 0x55a91b6e4030 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.885+0000 7f59fcdd8740 1 --2- 172.21.15.43:0/1989244252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 0x55a91b6e4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.885+0000 7f59fcdd8740 1 --2- 172.21.15.43:0/1989244252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a91b3441b0 0x55a91b6e3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.885+0000 7f59fcdd8740 1 -- 172.21.15.43:0/1989244252 >> 172.21.15.43:0/1989244252 conn(0x55a91b338300 msgr2=0x55a91b6e02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.885+0000 7f59fcdd8740 1 -- 172.21.15.43:0/1989244252 shutdown_connections 2024-04-25T10:38:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.885+0000 7f59fcdd8740 1 -- 172.21.15.43:0/1989244252 wait complete. 2024-04-25T10:38:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.885+0000 7f59fcdd8740 1 -- start start 2024-04-25T10:38:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.886+0000 7f59fcdd8740 1 Processor -- start 2024-04-25T10:38:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.886+0000 7f59fcdd8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a91b31b2f0 0x55a91b6efb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.886+0000 7f59fcdd8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 0x55a91b6f0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.886+0000 7f59fcdd8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a91b3441b0 0x55a91b6edfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.886+0000 7f59fcdd8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a91b6f0c20 con 0x55a91b3234d0 2024-04-25T10:38:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.886+0000 7f59fcdd8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a91b5d1b60 con 0x55a91b31b2f0 2024-04-25T10:38:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.886+0000 7f59fcdd8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a91b6f0da0 con 0x55a91b3441b0 2024-04-25T10:38:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.886+0000 7f59fcdd8740 10 client.0 _ll_register_callbacks cb 0x55a91b5f62b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.886+0000 7f59ef7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 0x55a91b6f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.886+0000 7f59ef7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 0x55a91b6f0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51634/0 (socket says 172.21.15.43:51634) 2024-04-25T10:38:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.886+0000 7f59ef7fe640 1 -- 172.21.15.43:0/3485361759 learned_addr learned my addr 172.21.15.43:0/3485361759 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.886+0000 7f59eeffd640 1 --2- 172.21.15.43:0/3485361759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a91b31b2f0 0x55a91b6efb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.886+0000 7f59eeffd640 1 --2- 172.21.15.43:0/3485361759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a91b3441b0 0x55a91b6edfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.886+0000 7f59ef7fe640 1 -- 172.21.15.43:0/3485361759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a91b3441b0 msgr2=0x55a91b6edfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.886+0000 7f59ef7fe640 1 --2- 172.21.15.43:0/3485361759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a91b3441b0 0x55a91b6edfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.886+0000 7f59ef7fe640 1 -- 172.21.15.43:0/3485361759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a91b31b2f0 msgr2=0x55a91b6efb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.886+0000 7f59ef7fe640 1 --2- 172.21.15.43:0/3485361759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a91b31b2f0 0x55a91b6efb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.886+0000 7f59ef7fe640 1 -- 172.21.15.43:0/3485361759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a91b35c210 con 0x55a91b3234d0 2024-04-25T10:38:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.886+0000 7f59eeffd640 1 --2- 172.21.15.43:0/3485361759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a91b3441b0 0x55a91b6edfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:38:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.886+0000 7f59ef7fe640 1 --2- 172.21.15.43:0/3485361759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 0x55a91b6f0050 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f59e0005890 tx=0x7f59e000bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e570b5e0ffaa7386 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.887+0000 7f59ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.887+0000 7f59ceffd640 1 -- 172.21.15.43:0/3485361759 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0004020 con 0x55a91b3234d0 2024-04-25T10:38:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.887+0000 7f59fcdd8740 1 -- 172.21.15.43:0/3485361759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59e001cad0 con 0x55a91b3234d0 2024-04-25T10:38:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.887+0000 7f59ceffd640 1 -- 172.21.15.43:0/3485361759 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f59e00041c0 con 0x55a91b3234d0 2024-04-25T10:38:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.887+0000 7f59ceffd640 1 -- 172.21.15.43:0/3485361759 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f59e00043c0 con 0x55a91b3234d0 2024-04-25T10:38:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.887+0000 7f59fcdd8740 10 client.17148 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.887+0000 7f59fcdd8740 1 -- 172.21.15.43:0/3485361759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a91b428620 con 0x55a91b3234d0 2024-04-25T10:38:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.887+0000 7f59ceffd640 1 -- 172.21.15.43:0/3485361759 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59e001cad0 con 0x55a91b3234d0 2024-04-25T10:38:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.887+0000 7f59ceffd640 1 -- 172.21.15.43:0/3485361759 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f59e0005490 con 0x55a91b3234d0 2024-04-25T10:38:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.887+0000 7f59fcdd8740 10 client.17148 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.887+0000 7f59fcdd8740 10 client.17148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.887+0000 7f59fcdd8740 20 client.17148 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.887+0000 7f59fcdd8740 1 --2- 172.21.15.43:0/3485361759 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a91b302d80 0x55a91b700660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.887+0000 7f59fcdd8740 4 client.17148 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:38:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.887+0000 7f59fcdd8740 1 -- 172.21.15.43:0/3485361759 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a91b658070 con 0x55a91b302d80 2024-04-25T10:38:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.888+0000 7f59eeffd640 1 --2- 172.21.15.43:0/3485361759 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a91b302d80 0x55a91b700660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.888+0000 7f59ceffd640 10 client.17148 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.888+0000 7f59eeffd640 1 --2- 172.21.15.43:0/3485361759 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a91b302d80 0x55a91b700660 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe69ee3fbf8d759a server_cookie=eb1990128a20e023 in_seq=0 out_seq=0 2024-04-25T10:38:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.889+0000 7f59ceffd640 1 -- 172.21.15.43:0/3485361759 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a91b658070 con 0x55a91b302d80 2024-04-25T10:38:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.889+0000 7f59ceffd640 10 client.17148 handle_command_reply: tid=0 2024-04-25T10:38:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.889+0000 7f59fcdd8740 1 client.17148 shutdown 2024-04-25T10:38:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.889+0000 7f59fcdd8740 1 -- 172.21.15.43:0/3485361759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 msgr2=0x55a91b6f0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.889+0000 7f59fcdd8740 1 --2- 172.21.15.43:0/3485361759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 0x55a91b6f0050 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f59e0005890 tx=0x7f59e000bf50 comp rx=0 tx=0).stop 2024-04-25T10:38:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.889+0000 7f59fcdd8740 1 -- 172.21.15.43:0/3485361759 shutdown_connections 2024-04-25T10:38:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.889+0000 7f59fcdd8740 1 --2- 172.21.15.43:0/3485361759 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a91b302d80 0x55a91b700660 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.889+0000 7f59fcdd8740 1 --2- 172.21.15.43:0/3485361759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a91b3441b0 0x55a91b6edfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.889+0000 7f59fcdd8740 1 --2- 172.21.15.43:0/3485361759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 0x55a91b6f0050 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.889+0000 7f59ceffd640 0 client.17148 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.889+0000 7f59fcdd8740 1 --2- 172.21.15.43:0/3485361759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a91b31b2f0 0x55a91b6efb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.889+0000 7f59fcdd8740 1 -- 172.21.15.43:0/3485361759 >> 172.21.15.43:0/3485361759 conn(0x55a91b338300 msgr2=0x55a91b6e3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.889+0000 7f59fcdd8740 1 -- 172.21.15.43:0/3485361759 shutdown_connections 2024-04-25T10:38:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.889+0000 7f59fcdd8740 1 -- 172.21.15.43:0/3485361759 wait complete. 2024-04-25T10:38:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.889+0000 7f59fcdd8740 20 client.17148 trim_cache size 0 max 16384 2024-04-25T10:38:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.905+0000 7f59fcdd8740 1 Processor -- start 2024-04-25T10:38:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.905+0000 7f59fcdd8740 1 -- start start 2024-04-25T10:38:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.905+0000 7f59fcdd8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a91b31b2f0 0x55a91b6e7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.905+0000 7f59fcdd8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 0x55a91b703a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.905+0000 7f59fcdd8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a91b3441b0 0x55a91b6e3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.905+0000 7f59fcdd8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f59f01b2330 con 0x55a91b3234d0 2024-04-25T10:38:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.905+0000 7f59fcdd8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f59f01afc00 con 0x55a91b31b2f0 2024-04-25T10:38:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.905+0000 7f59fcdd8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f59f01ada50 con 0x55a91b3441b0 2024-04-25T10:38:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.905+0000 7f59ef7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 0x55a91b703a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.905+0000 7f59ef7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 0x55a91b703a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51646/0 (socket says 172.21.15.43:51646) 2024-04-25T10:38:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.905+0000 7f59ef7fe640 1 -- 172.21.15.43:0/3721460664 learned_addr learned my addr 172.21.15.43:0/3721460664 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.905+0000 7f59ef7fe640 1 -- 172.21.15.43:0/3721460664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a91b3441b0 msgr2=0x55a91b6e3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.905+0000 7f59ef7fe640 1 --2- 172.21.15.43:0/3721460664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a91b3441b0 0x55a91b6e3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.905+0000 7f59ef7fe640 1 -- 172.21.15.43:0/3721460664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a91b31b2f0 msgr2=0x55a91b6e7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.905+0000 7f59ef7fe640 1 --2- 172.21.15.43:0/3721460664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a91b31b2f0 0x55a91b6e7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.905+0000 7f59ef7fe640 1 -- 172.21.15.43:0/3721460664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a91b6f06f0 con 0x55a91b3234d0 2024-04-25T10:38:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.905+0000 7f59ef7fe640 1 --2- 172.21.15.43:0/3721460664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 0x55a91b703a70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f59e00047c0 tx=0x7f59e000b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a698564c9c34cfb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.906+0000 7f59d57fa640 1 -- 172.21.15.43:0/3721460664 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0032070 con 0x55a91b3234d0 2024-04-25T10:38:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.906+0000 7f59d57fa640 1 -- 172.21.15.43:0/3721460664 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59e001c930 con 0x55a91b3234d0 2024-04-25T10:38:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.906+0000 7f59d57fa640 1 -- 172.21.15.43:0/3721460664 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0005360 con 0x55a91b3234d0 2024-04-25T10:38:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.906+0000 7f59fcdd8740 1 -- 172.21.15.43:0/3721460664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 msgr2=0x55a91b703a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.906+0000 7f59fcdd8740 1 --2- 172.21.15.43:0/3721460664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 0x55a91b703a70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f59e00047c0 tx=0x7f59e000b710 comp rx=0 tx=0).stop 2024-04-25T10:38:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.906+0000 7f59fcdd8740 1 -- 172.21.15.43:0/3721460664 shutdown_connections 2024-04-25T10:38:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.906+0000 7f59fcdd8740 1 --2- 172.21.15.43:0/3721460664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a91b3441b0 0x55a91b6e3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.906+0000 7f59fcdd8740 1 --2- 172.21.15.43:0/3721460664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3234d0 0x55a91b703a70 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.906+0000 7f59fcdd8740 1 --2- 172.21.15.43:0/3721460664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a91b31b2f0 0x55a91b6e7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.906+0000 7f59fcdd8740 1 -- 172.21.15.43:0/3721460664 >> 172.21.15.43:0/3721460664 conn(0x55a91b338300 msgr2=0x55a91b6e77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.906+0000 7f59fcdd8740 1 -- 172.21.15.43:0/3721460664 shutdown_connections 2024-04-25T10:38:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.906+0000 7f59fcdd8740 1 -- 172.21.15.43:0/3721460664 wait complete. 2024-04-25T10:38:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.907+0000 7f59fcdd8740 1 -- start start 2024-04-25T10:38:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.907+0000 7f59fcdd8740 1 Processor -- start 2024-04-25T10:38:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.907+0000 7f59fcdd8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a91b31b2f0 0x55a91b765770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.907+0000 7f59fcdd8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a91b3234d0 0x55a91b765cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.907+0000 7f59fcdd8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3441b0 0x55a91b75fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.907+0000 7f59fcdd8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f59f01ae0c0 con 0x55a91b3441b0 2024-04-25T10:38:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.907+0000 7f59fcdd8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f59f01af530 con 0x55a91b3234d0 2024-04-25T10:38:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.907+0000 7f59fcdd8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f59f01ae790 con 0x55a91b31b2f0 2024-04-25T10:38:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.907+0000 7f59fcdd8740 10 client.0 _ll_register_callbacks cb 0x7f59e4005e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.907+0000 7f59eeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3441b0 0x55a91b75fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.907+0000 7f59eeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3441b0 0x55a91b75fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51656/0 (socket says 172.21.15.43:51656) 2024-04-25T10:38:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.907+0000 7f59eeffd640 1 -- 172.21.15.43:0/1125776131 learned_addr learned my addr 172.21.15.43:0/1125776131 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.908+0000 7f59ef7fe640 1 --2- 172.21.15.43:0/1125776131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a91b3234d0 0x55a91b765cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.908+0000 7f59eeffd640 1 --2- 172.21.15.43:0/1125776131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a91b31b2f0 0x55a91b765770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.908+0000 7f59eeffd640 1 -- 172.21.15.43:0/1125776131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a91b31b2f0 msgr2=0x55a91b765770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.908+0000 7f59eeffd640 1 --2- 172.21.15.43:0/1125776131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a91b31b2f0 0x55a91b765770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.908+0000 7f59eeffd640 1 -- 172.21.15.43:0/1125776131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a91b3234d0 msgr2=0x55a91b765cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.908+0000 7f59eeffd640 1 --2- 172.21.15.43:0/1125776131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a91b3234d0 0x55a91b765cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.908+0000 7f59eeffd640 1 -- 172.21.15.43:0/1125776131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a91b760170 con 0x55a91b3441b0 2024-04-25T10:38:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.908+0000 7f59eeffd640 1 --2- 172.21.15.43:0/1125776131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3441b0 0x55a91b75fb60 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f59e4031440 tx=0x7f59e4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eff213c03c1b0ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.908+0000 7f59ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.908+0000 7f59ceffd640 1 -- 172.21.15.43:0/1125776131 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4035040 con 0x55a91b3441b0 2024-04-25T10:38:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.908+0000 7f59ceffd640 1 -- 172.21.15.43:0/1125776131 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f59e404c400 con 0x55a91b3441b0 2024-04-25T10:38:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.908+0000 7f59fcdd8740 1 -- 172.21.15.43:0/1125776131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59e001c930 con 0x55a91b3441b0 2024-04-25T10:38:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.908+0000 7f59ceffd640 1 -- 172.21.15.43:0/1125776131 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f59e405d050 con 0x55a91b3441b0 2024-04-25T10:38:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.908+0000 7f59fcdd8740 10 client.17154 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.908+0000 7f59fcdd8740 1 -- 172.21.15.43:0/1125776131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a91b76ce20 con 0x55a91b3441b0 2024-04-25T10:38:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.908+0000 7f59ceffd640 1 -- 172.21.15.43:0/1125776131 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59e001c930 con 0x55a91b3441b0 2024-04-25T10:38:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.909+0000 7f59ceffd640 1 -- 172.21.15.43:0/1125776131 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f59e403da60 con 0x55a91b3441b0 2024-04-25T10:38:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.909+0000 7f59fcdd8740 10 client.17154 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.909+0000 7f59fcdd8740 10 client.17154 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.909+0000 7f59fcdd8740 20 client.17154 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.909+0000 7f59fcdd8740 1 --2- 172.21.15.43:0/1125776131 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a91b2d81c0 0x55a91b76f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.909+0000 7f59fcdd8740 4 client.17154 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:38:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.909+0000 7f59fcdd8740 1 -- 172.21.15.43:0/1125776131 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a91b651ee0 con 0x55a91b2d81c0 2024-04-25T10:38:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.909+0000 7f59ef7fe640 1 --2- 172.21.15.43:0/1125776131 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a91b2d81c0 0x55a91b76f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.909+0000 7f59ceffd640 10 client.17154 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.909+0000 7f59ef7fe640 1 --2- 172.21.15.43:0/1125776131 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a91b2d81c0 0x55a91b76f560 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8496be31771c6f6 server_cookie=e847cc68e25e91c0 in_seq=0 out_seq=0 2024-04-25T10:38:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.924+0000 7f59ceffd640 1 -- 172.21.15.43:0/1125776131 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a91b651ee0 con 0x55a91b2d81c0 2024-04-25T10:38:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.924+0000 7f59ceffd640 10 client.17154 handle_command_reply: tid=0 2024-04-25T10:38:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.924+0000 7f59fcdd8740 1 client.17154 shutdown 2024-04-25T10:38:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.924+0000 7f59fcdd8740 1 -- 172.21.15.43:0/1125776131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3441b0 msgr2=0x55a91b75fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.924+0000 7f59fcdd8740 1 --2- 172.21.15.43:0/1125776131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3441b0 0x55a91b75fb60 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f59e4031440 tx=0x7f59e4009260 comp rx=0 tx=0).stop 2024-04-25T10:38:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.924+0000 7f59fcdd8740 1 -- 172.21.15.43:0/1125776131 shutdown_connections 2024-04-25T10:38:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.924+0000 7f59fcdd8740 1 --2- 172.21.15.43:0/1125776131 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a91b2d81c0 0x55a91b76f560 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.924+0000 7f59fcdd8740 1 --2- 172.21.15.43:0/1125776131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a91b3234d0 0x55a91b765cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.924+0000 7f59fcdd8740 1 --2- 172.21.15.43:0/1125776131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91b3441b0 0x55a91b75fb60 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.924+0000 7f59fcdd8740 1 --2- 172.21.15.43:0/1125776131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a91b31b2f0 0x55a91b765770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.924+0000 7f59ceffd640 0 client.17154 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.924+0000 7f59fcdd8740 1 -- 172.21.15.43:0/1125776131 >> 172.21.15.43:0/1125776131 conn(0x55a91b338300 msgr2=0x55a91b6e3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.924+0000 7f59fcdd8740 1 -- 172.21.15.43:0/1125776131 shutdown_connections 2024-04-25T10:38:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.924+0000 7f59fcdd8740 1 -- 172.21.15.43:0/1125776131 wait complete. 2024-04-25T10:38:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.924+0000 7f59fcdd8740 20 client.17154 trim_cache size 0 max 16384 2024-04-25T10:38:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:38:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:38:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:38:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:38:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.925+0000 7f59f5f26640 1 -- 172.21.15.43:0/700032136 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f59d80afc50 msgr2=0x7f59d80b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.925+0000 7f59f5f26640 1 --2- 172.21.15.43:0/700032136 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f59d80afc50 0x7f59d80b2160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f59e80dc990 tx=0x7f59e809bcd0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.925+0000 7f59f5f26640 1 -- 172.21.15.43:0/700032136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59f019ed10 msgr2=0x7f59f019f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.925+0000 7f59f5f26640 1 --2- 172.21.15.43:0/700032136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59f019ed10 0x7f59f019f120 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f59e800eaa0 tx=0x7f59e800ef60 comp rx=0 tx=0).stop 2024-04-25T10:38:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.925+0000 7f59f5f26640 1 -- 172.21.15.43:0/700032136 shutdown_connections 2024-04-25T10:38:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.925+0000 7f59f5f26640 1 --2- 172.21.15.43:0/700032136 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f59d80afc50 0x7f59d80b2160 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.925+0000 7f59f5f26640 1 --2- 172.21.15.43:0/700032136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59f019ed10 0x7f59f019f120 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.925+0000 7f59f5f26640 1 --2- 172.21.15.43:0/700032136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59f019a4f0 0x7f59f019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.925+0000 7f59f5f26640 1 --2- 172.21.15.43:0/700032136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f0109b60 0x7f59f0199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.925+0000 7f59f5f26640 1 -- 172.21.15.43:0/700032136 >> 172.21.15.43:0/700032136 conn(0x7f59f00fd8a0 msgr2=0x7f59f0100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.925+0000 7f59f5f26640 1 -- 172.21.15.43:0/700032136 shutdown_connections 2024-04-25T10:38:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:25.925+0000 7f59f5f26640 1 -- 172.21.15.43:0/700032136 wait complete. 2024-04-25T10:38:26.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:25 smithi077 ceph-mon[32772]: pgmap v180: 97 pgs: 97 active+clean; 4.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-04-25T10:38:26.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:25 smithi164 ceph-mon[32713]: pgmap v180: 97 pgs: 97 active+clean; 4.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-04-25T10:38:26.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:25 smithi043 ceph-mon[29478]: pgmap v180: 97 pgs: 97 active+clean; 4.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-04-25T10:38:28.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:27 smithi077 ceph-mon[32772]: pgmap v181: 97 pgs: 97 active+clean; 4.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-04-25T10:38:28.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:27 smithi164 ceph-mon[32713]: pgmap v181: 97 pgs: 97 active+clean; 4.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-04-25T10:38:28.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:27 smithi043 ceph-mon[29478]: pgmap v181: 97 pgs: 97 active+clean; 4.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-04-25T10:38:30.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:29 smithi077 ceph-mon[32772]: pgmap v182: 97 pgs: 97 active+clean; 4.7 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-04-25T10:38:30.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:29 smithi164 ceph-mon[32713]: pgmap v182: 97 pgs: 97 active+clean; 4.7 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-04-25T10:38:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:29 smithi043 ceph-mon[29478]: pgmap v182: 97 pgs: 97 active+clean; 4.7 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-04-25T10:38:32.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:31 smithi077 ceph-mon[32772]: pgmap v183: 97 pgs: 97 active+clean; 4.7 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-04-25T10:38:32.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:31 smithi164 ceph-mon[32713]: pgmap v183: 97 pgs: 97 active+clean; 4.7 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-04-25T10:38:32.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:31 smithi043 ceph-mon[29478]: pgmap v183: 97 pgs: 97 active+clean; 4.7 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-04-25T10:38:34.097 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:38:34.097 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:38:34.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.225+0000 7f1d17e53640 1 -- 172.21.15.43:0/3229070246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d10104c70 msgr2=0x7f1d10105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.225+0000 7f1d17e53640 1 --2- 172.21.15.43:0/3229070246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d10104c70 0x7f1d10105080 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009900 tx=0x7f1d0c02a410 comp rx=0 tx=0).stop 2024-04-25T10:38:34.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.225+0000 7f1d17e53640 1 -- 172.21.15.43:0/3229070246 shutdown_connections 2024-04-25T10:38:34.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.225+0000 7f1d17e53640 1 --2- 172.21.15.43:0/3229070246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d1010b6d0 0x7f1d1010daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.225+0000 7f1d17e53640 1 --2- 172.21.15.43:0/3229070246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d1010e0c0 0x7f1d101105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.225+0000 7f1d17e53640 1 --2- 172.21.15.43:0/3229070246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d10104c70 0x7f1d10105080 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.225+0000 7f1d17e53640 1 -- 172.21.15.43:0/3229070246 >> 172.21.15.43:0/3229070246 conn(0x7f1d10078110 msgr2=0x7f1d10078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:34.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.225+0000 7f1d17e53640 1 -- 172.21.15.43:0/3229070246 shutdown_connections 2024-04-25T10:38:34.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.225+0000 7f1d17e53640 1 -- 172.21.15.43:0/3229070246 wait complete. 2024-04-25T10:38:34.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.226+0000 7f1d17e53640 1 Processor -- start 2024-04-25T10:38:34.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.226+0000 7f1d17e53640 1 -- start start 2024-04-25T10:38:34.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.226+0000 7f1d17e53640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d10104c70 0x7f1d10102530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.226+0000 7f1d17e53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1010b6d0 0x7f1d10102a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.226+0000 7f1d17e53640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d1010e0c0 0x7f1d10102fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.226+0000 7f1d17e53640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1d10113ce0 con 0x7f1d1010b6d0 2024-04-25T10:38:34.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.226+0000 7f1d17e53640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1d10113b60 con 0x7f1d1010e0c0 2024-04-25T10:38:34.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.226+0000 7f1d17e53640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1d10113e60 con 0x7f1d10104c70 2024-04-25T10:38:34.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.226+0000 7f1d153ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1010b6d0 0x7f1d10102a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.226+0000 7f1d153ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1010b6d0 0x7f1d10102a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37646/0 (socket says 172.21.15.43:37646) 2024-04-25T10:38:34.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.226+0000 7f1d153ee640 1 -- 172.21.15.43:0/135660798 learned_addr learned my addr 172.21.15.43:0/135660798 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:34.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.226+0000 7f1d163f0640 1 --2- 172.21.15.43:0/135660798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d1010e0c0 0x7f1d10102fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.226+0000 7f1d15bef640 1 --2- 172.21.15.43:0/135660798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d10104c70 0x7f1d10102530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.227+0000 7f1d153ee640 1 -- 172.21.15.43:0/135660798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d10104c70 msgr2=0x7f1d10102530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.227+0000 7f1d153ee640 1 --2- 172.21.15.43:0/135660798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d10104c70 0x7f1d10102530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.227+0000 7f1d153ee640 1 -- 172.21.15.43:0/135660798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d1010e0c0 msgr2=0x7f1d10102fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.227+0000 7f1d153ee640 1 --2- 172.21.15.43:0/135660798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d1010e0c0 0x7f1d10102fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.227+0000 7f1d153ee640 1 -- 172.21.15.43:0/135660798 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d101a8ae0 con 0x7f1d1010b6d0 2024-04-25T10:38:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.227+0000 7f1d153ee640 1 --2- 172.21.15.43:0/135660798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1010b6d0 0x7f1d10102a70 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000d770 tx=0x7f1d0000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.227+0000 7f1d06ffd640 1 -- 172.21.15.43:0/135660798 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00017070 con 0x7f1d1010b6d0 2024-04-25T10:38:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.227+0000 7f1d06ffd640 1 -- 172.21.15.43:0/135660798 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d00009c30 con 0x7f1d1010b6d0 2024-04-25T10:38:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.227+0000 7f1d06ffd640 1 -- 172.21.15.43:0/135660798 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0000f620 con 0x7f1d1010b6d0 2024-04-25T10:38:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.227+0000 7f1d17e53640 1 -- 172.21.15.43:0/135660798 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d101a8dd0 con 0x7f1d1010b6d0 2024-04-25T10:38:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.227+0000 7f1d17e53640 1 -- 172.21.15.43:0/135660798 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d101a9240 con 0x7f1d1010b6d0 2024-04-25T10:38:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.231+0000 7f1d1ede0740 1 Processor -- start 2024-04-25T10:38:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.231+0000 7f1d1ede0740 1 -- start start 2024-04-25T10:38:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.231+0000 7f1d06ffd640 1 -- 172.21.15.43:0/135660798 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f1d0001a070 con 0x7f1d1010b6d0 2024-04-25T10:38:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.231+0000 7f1d1ede0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8854c0390 0x55e885860a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.231+0000 7f1d1ede0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8854974d0 0x55e885860fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.231+0000 7f1d1ede0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e88549f6b0 0x55e88586d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.231+0000 7f1d1ede0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e8857c6d80 con 0x55e8854c0390 2024-04-25T10:38:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.231+0000 7f1d1ede0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e8857808b0 con 0x55e8854974d0 2024-04-25T10:38:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.231+0000 7f1d1ede0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e8857c2d40 con 0x55e88549f6b0 2024-04-25T10:38:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.231+0000 7f1d163f0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8854c0390 0x55e885860a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.231+0000 7f1d163f0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8854c0390 0x55e885860a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37662/0 (socket says 172.21.15.43:37662) 2024-04-25T10:38:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.231+0000 7f1d163f0640 1 -- 172.21.15.43:0/3854269868 learned_addr learned my addr 172.21.15.43:0/3854269868 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.232+0000 7f1d15bef640 1 --2- 172.21.15.43:0/3854269868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8854974d0 0x55e885860fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.232+0000 7f1d153ee640 1 --2- 172.21.15.43:0/3854269868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e88549f6b0 0x55e88586d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.232+0000 7f1d163f0640 1 -- 172.21.15.43:0/3854269868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e88549f6b0 msgr2=0x55e88586d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.232+0000 7f1d163f0640 1 --2- 172.21.15.43:0/3854269868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e88549f6b0 0x55e88586d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.232+0000 7f1d163f0640 1 -- 172.21.15.43:0/3854269868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8854974d0 msgr2=0x55e885860fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.232+0000 7f1d163f0640 1 --2- 172.21.15.43:0/3854269868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8854974d0 0x55e885860fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.232+0000 7f1d163f0640 1 -- 172.21.15.43:0/3854269868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e885751a40 con 0x55e8854c0390 2024-04-25T10:38:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.232+0000 7f1d15bef640 1 --2- 172.21.15.43:0/3854269868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8854974d0 0x55e885860fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:38:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.232+0000 7f1d06ffd640 1 --2- 172.21.15.43:0/135660798 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1cfc0afc60 0x7f1cfc0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.232+0000 7f1d163f0640 1 --2- 172.21.15.43:0/3854269868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8854c0390 0x55e885860a60 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009f70 tx=0x7f1d08010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9e69d049f3ac33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.232+0000 7f1d15bef640 1 --2- 172.21.15.43:0/135660798 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1cfc0afc60 0x7f1cfc0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.232+0000 7f1d06ffd640 1 -- 172.21.15.43:0/135660798 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f1d000deaa0 con 0x7f1d1010b6d0 2024-04-25T10:38:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.232+0000 7f1cfbfff640 1 -- 172.21.15.43:0/3854269868 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08016070 con 0x55e8854c0390 2024-04-25T10:38:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.232+0000 7f1cfbfff640 1 -- 172.21.15.43:0/3854269868 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d08004d20 con 0x55e8854c0390 2024-04-25T10:38:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.232+0000 7f1cfbfff640 1 -- 172.21.15.43:0/3854269868 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d080127d0 con 0x55e8854c0390 2024-04-25T10:38:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.233+0000 7f1d1ede0740 1 -- 172.21.15.43:0/3854269868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8854c0390 msgr2=0x55e885860a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.233+0000 7f1d15bef640 1 --2- 172.21.15.43:0/135660798 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1cfc0afc60 0x7f1cfc0b2170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c031440 tx=0x7f1d0c005890 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:34.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.233+0000 7f1d1ede0740 1 --2- 172.21.15.43:0/3854269868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8854c0390 0x55e885860a60 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009f70 tx=0x7f1d08010800 comp rx=0 tx=0).stop 2024-04-25T10:38:34.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.233+0000 7f1d1ede0740 1 -- 172.21.15.43:0/3854269868 shutdown_connections 2024-04-25T10:38:34.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.233+0000 7f1d1ede0740 1 --2- 172.21.15.43:0/3854269868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8854974d0 0x55e885860fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.233+0000 7f1d1ede0740 1 --2- 172.21.15.43:0/3854269868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e88549f6b0 0x55e88586d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.233+0000 7f1d1ede0740 1 --2- 172.21.15.43:0/3854269868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8854c0390 0x55e885860a60 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.233+0000 7f1d1ede0740 1 -- 172.21.15.43:0/3854269868 >> 172.21.15.43:0/3854269868 conn(0x55e8854b44e0 msgr2=0x55e88585d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:34.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.233+0000 7f1d1ede0740 1 -- 172.21.15.43:0/3854269868 shutdown_connections 2024-04-25T10:38:34.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.233+0000 7f1d1ede0740 1 -- 172.21.15.43:0/3854269868 wait complete. 2024-04-25T10:38:34.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.234+0000 7f1d1ede0740 1 -- start start 2024-04-25T10:38:34.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.234+0000 7f1d1ede0740 1 Processor -- start 2024-04-25T10:38:34.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.234+0000 7f1d1ede0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8854974d0 0x55e88586b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.234+0000 7f1d1ede0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e88549f6b0 0x55e88586bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.234+0000 7f1d1ede0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8854c0390 0x55e8858714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.234+0000 7f1d1ede0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e88586dfe0 con 0x55e88549f6b0 2024-04-25T10:38:34.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.234+0000 7f1d1ede0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e88574dd10 con 0x55e8854c0390 2024-04-25T10:38:34.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.234+0000 7f1d1ede0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e88586e160 con 0x55e8854974d0 2024-04-25T10:38:34.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.234+0000 7f1d1ede0740 10 client.0 _ll_register_callbacks cb 0x55e88577a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:34.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.234+0000 7f1d153ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e88549f6b0 0x55e88586bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.235+0000 7f1d153ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e88549f6b0 0x55e88586bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37666/0 (socket says 172.21.15.43:37666) 2024-04-25T10:38:34.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.235+0000 7f1d153ee640 1 -- 172.21.15.43:0/220478620 learned_addr learned my addr 172.21.15.43:0/220478620 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:34.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.235+0000 7f1d15bef640 1 --2- 172.21.15.43:0/220478620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8854974d0 0x55e88586b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.235+0000 7f1d163f0640 1 --2- 172.21.15.43:0/220478620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8854c0390 0x55e8858714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.235+0000 7f1d153ee640 1 -- 172.21.15.43:0/220478620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8854974d0 msgr2=0x55e88586b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.235+0000 7f1d153ee640 1 --2- 172.21.15.43:0/220478620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8854974d0 0x55e88586b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.235+0000 7f1d153ee640 1 -- 172.21.15.43:0/220478620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8854c0390 msgr2=0x55e8858714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.235+0000 7f1d153ee640 1 --2- 172.21.15.43:0/220478620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8854c0390 0x55e8858714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.235+0000 7f1d153ee640 1 -- 172.21.15.43:0/220478620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8855a27d0 con 0x55e88549f6b0 2024-04-25T10:38:34.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.235+0000 7f1d153ee640 1 --2- 172.21.15.43:0/220478620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e88549f6b0 0x55e88586bb80 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f1d000a6a20 tx=0x7f1d000e21b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1acec12a0c9cb14a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:34.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.235+0000 7f1cf97fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:34.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.235+0000 7f1cf97fa640 1 -- 172.21.15.43:0/220478620 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d000e4060 con 0x55e88549f6b0 2024-04-25T10:38:34.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.235+0000 7f1cf97fa640 1 -- 172.21.15.43:0/220478620 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f1d000af850 con 0x55e88549f6b0 2024-04-25T10:38:34.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.235+0000 7f1d1ede0740 1 -- 172.21.15.43:0/220478620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d08004d20 con 0x55e88549f6b0 2024-04-25T10:38:34.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.235+0000 7f1cf97fa640 1 -- 172.21.15.43:0/220478620 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d00021070 con 0x55e88549f6b0 2024-04-25T10:38:34.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.235+0000 7f1d1ede0740 10 client.17166 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:34.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.235+0000 7f1d1ede0740 1 -- 172.21.15.43:0/220478620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e8854d83f0 con 0x55e88549f6b0 2024-04-25T10:38:34.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.236+0000 7f1cf97fa640 1 -- 172.21.15.43:0/220478620 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d08004d20 con 0x55e88549f6b0 2024-04-25T10:38:34.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.236+0000 7f1cf97fa640 1 -- 172.21.15.43:0/220478620 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1d000a2690 con 0x55e88549f6b0 2024-04-25T10:38:34.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.236+0000 7f1d1ede0740 10 client.17166 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:34.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.236+0000 7f1d1ede0740 10 client.17166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:34.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.236+0000 7f1d1ede0740 20 client.17166 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:34.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.236+0000 7f1d1ede0740 1 --2- 172.21.15.43:0/220478620 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e88547ef60 0x55e88587d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.236+0000 7f1d1ede0740 4 client.17166 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:38:34.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.236+0000 7f1d1ede0740 1 -- 172.21.15.43:0/220478620 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8857d4990 con 0x55e88547ef60 2024-04-25T10:38:34.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.236+0000 7f1d15bef640 1 --2- 172.21.15.43:0/220478620 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e88547ef60 0x55e88587d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.237+0000 7f1cf97fa640 10 client.17166 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:34.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.237+0000 7f1d15bef640 1 --2- 172.21.15.43:0/220478620 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e88547ef60 0x55e88587d760 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abf14b5aad51d538 server_cookie=a4b7314a5b2fd389 in_seq=0 out_seq=0 2024-04-25T10:38:34.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.237+0000 7f1cf97fa640 1 -- 172.21.15.43:0/220478620 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e8857d4990 con 0x55e88547ef60 2024-04-25T10:38:34.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.237+0000 7f1cf97fa640 10 client.17166 handle_command_reply: tid=0 2024-04-25T10:38:34.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.237+0000 7f1d1ede0740 1 client.17166 shutdown 2024-04-25T10:38:34.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.238+0000 7f1d1ede0740 1 -- 172.21.15.43:0/220478620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e88549f6b0 msgr2=0x55e88586bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.238+0000 7f1d1ede0740 1 --2- 172.21.15.43:0/220478620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e88549f6b0 0x55e88586bb80 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f1d000a6a20 tx=0x7f1d000e21b0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.238+0000 7f1d1ede0740 1 -- 172.21.15.43:0/220478620 shutdown_connections 2024-04-25T10:38:34.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.238+0000 7f1d1ede0740 1 --2- 172.21.15.43:0/220478620 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e88547ef60 0x55e88587d760 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.238+0000 7f1d1ede0740 1 --2- 172.21.15.43:0/220478620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8854c0390 0x55e8858714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.238+0000 7f1d1ede0740 1 --2- 172.21.15.43:0/220478620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e88549f6b0 0x55e88586bb80 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.238+0000 7f1d1ede0740 1 --2- 172.21.15.43:0/220478620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8854974d0 0x55e88586b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.238+0000 7f1cf97fa640 0 client.17166 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:34.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.238+0000 7f1d1ede0740 1 -- 172.21.15.43:0/220478620 >> 172.21.15.43:0/220478620 conn(0x55e8854b44e0 msgr2=0x55e885862780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:34.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.238+0000 7f1d1ede0740 1 -- 172.21.15.43:0/220478620 shutdown_connections 2024-04-25T10:38:34.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.238+0000 7f1d1ede0740 1 -- 172.21.15.43:0/220478620 wait complete. 2024-04-25T10:38:34.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.238+0000 7f1d1ede0740 20 client.17166 trim_cache size 0 max 16384 2024-04-25T10:38:34.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.256+0000 7f1d1ede0740 1 Processor -- start 2024-04-25T10:38:34.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.256+0000 7f1d1ede0740 1 -- start start 2024-04-25T10:38:34.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.256+0000 7f1d1ede0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8854974d0 0x55e885871420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.256+0000 7f1d1ede0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e88549f6b0 0x55e885882f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.256+0000 7f1d1ede0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8854c0390 0x55e885883460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.256+0000 7f1d1ede0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1d101b44a0 con 0x55e8854974d0 2024-04-25T10:38:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.256+0000 7f1d1ede0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1d101b1d70 con 0x55e88549f6b0 2024-04-25T10:38:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.256+0000 7f1d1ede0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1d101afbc0 con 0x55e8854c0390 2024-04-25T10:38:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.256+0000 7f1d15bef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8854974d0 0x55e885871420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.256+0000 7f1d15bef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8854974d0 0x55e885871420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37672/0 (socket says 172.21.15.43:37672) 2024-04-25T10:38:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.256+0000 7f1d15bef640 1 -- 172.21.15.43:0/3201774219 learned_addr learned my addr 172.21.15.43:0/3201774219 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.257+0000 7f1d15bef640 1 -- 172.21.15.43:0/3201774219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8854c0390 msgr2=0x55e885883460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.257+0000 7f1d15bef640 1 --2- 172.21.15.43:0/3201774219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8854c0390 0x55e885883460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.257+0000 7f1d15bef640 1 -- 172.21.15.43:0/3201774219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e88549f6b0 msgr2=0x55e885882f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.257+0000 7f1d15bef640 1 --2- 172.21.15.43:0/3201774219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e88549f6b0 0x55e885882f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.257+0000 7f1d15bef640 1 -- 172.21.15.43:0/3201774219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8854980f0 con 0x55e8854974d0 2024-04-25T10:38:34.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.257+0000 7f1d15bef640 1 --2- 172.21.15.43:0/3201774219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8854974d0 0x55e885871420 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c0315b0 tx=0x7f1d0c03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8e587d8c713124 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:34.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.257+0000 7f1cfb7fe640 1 -- 172.21.15.43:0/3201774219 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c035040 con 0x55e8854974d0 2024-04-25T10:38:34.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.257+0000 7f1cfb7fe640 1 -- 172.21.15.43:0/3201774219 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d0c055070 con 0x55e8854974d0 2024-04-25T10:38:34.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.257+0000 7f1cfb7fe640 1 -- 172.21.15.43:0/3201774219 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c0049b0 con 0x55e8854974d0 2024-04-25T10:38:34.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.258+0000 7f1d1ede0740 1 -- 172.21.15.43:0/3201774219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8854974d0 msgr2=0x55e885871420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.258+0000 7f1d1ede0740 1 --2- 172.21.15.43:0/3201774219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8854974d0 0x55e885871420 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c0315b0 tx=0x7f1d0c03e000 comp rx=0 tx=0).stop 2024-04-25T10:38:34.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.258+0000 7f1d1ede0740 1 -- 172.21.15.43:0/3201774219 shutdown_connections 2024-04-25T10:38:34.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.258+0000 7f1d1ede0740 1 --2- 172.21.15.43:0/3201774219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e88549f6b0 0x55e885882f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.258+0000 7f1d1ede0740 1 --2- 172.21.15.43:0/3201774219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8854c0390 0x55e885883460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.258+0000 7f1d1ede0740 1 --2- 172.21.15.43:0/3201774219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8854974d0 0x55e885871420 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.258+0000 7f1d1ede0740 1 -- 172.21.15.43:0/3201774219 >> 172.21.15.43:0/3201774219 conn(0x55e8854b44e0 msgr2=0x55e8858607f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:34.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.258+0000 7f1d1ede0740 1 -- 172.21.15.43:0/3201774219 shutdown_connections 2024-04-25T10:38:34.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.258+0000 7f1d1ede0740 1 -- 172.21.15.43:0/3201774219 wait complete. 2024-04-25T10:38:34.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.258+0000 7f1d1ede0740 1 -- start start 2024-04-25T10:38:34.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.258+0000 7f1d1ede0740 1 Processor -- start 2024-04-25T10:38:34.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.259+0000 7f1d1ede0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8854974d0 0x55e8858dde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.259+0000 7f1d1ede0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e88549f6b0 0x55e8858de3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.259+0000 7f1d1ede0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8854c0390 0x55e8858725e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.259+0000 7f1d1ede0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1d101b0230 con 0x55e88549f6b0 2024-04-25T10:38:34.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.259+0000 7f1d1ede0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1d101b16a0 con 0x55e8854c0390 2024-04-25T10:38:34.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.259+0000 7f1d1ede0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1d101b0900 con 0x55e8854974d0 2024-04-25T10:38:34.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.259+0000 7f1d1ede0740 10 client.0 _ll_register_callbacks cb 0x7f1d0000d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:34.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.259+0000 7f1d153ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e88549f6b0 0x55e8858de3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.259+0000 7f1d153ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e88549f6b0 0x55e8858de3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37674/0 (socket says 172.21.15.43:37674) 2024-04-25T10:38:34.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.259+0000 7f1d153ee640 1 -- 172.21.15.43:0/1786782065 learned_addr learned my addr 172.21.15.43:0/1786782065 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:34.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.259+0000 7f1d15bef640 1 --2- 172.21.15.43:0/1786782065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8854974d0 0x55e8858dde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.259+0000 7f1d163f0640 1 --2- 172.21.15.43:0/1786782065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8854c0390 0x55e8858725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.259+0000 7f1d153ee640 1 -- 172.21.15.43:0/1786782065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8854974d0 msgr2=0x55e8858dde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.259+0000 7f1d153ee640 1 --2- 172.21.15.43:0/1786782065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8854974d0 0x55e8858dde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.259+0000 7f1d153ee640 1 -- 172.21.15.43:0/1786782065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8854c0390 msgr2=0x55e8858725e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.259+0000 7f1d153ee640 1 --2- 172.21.15.43:0/1786782065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8854c0390 0x55e8858725e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.259+0000 7f1d153ee640 1 -- 172.21.15.43:0/1786782065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8857874b0 con 0x55e88549f6b0 2024-04-25T10:38:34.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.259+0000 7f1d163f0640 1 --2- 172.21.15.43:0/1786782065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8854c0390 0x55e8858725e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:38:34.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.260+0000 7f1d153ee640 1 --2- 172.21.15.43:0/1786782065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e88549f6b0 0x55e8858de3a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f1d000a2da0 tx=0x7f1d000097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4225886ed3ab45e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:34.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.260+0000 7f1cf97fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:34.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.260+0000 7f1cf97fa640 1 -- 172.21.15.43:0/1786782065 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0008f050 con 0x55e88549f6b0 2024-04-25T10:38:34.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.260+0000 7f1cf97fa640 1 -- 172.21.15.43:0/1786782065 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f1d000a5f80 con 0x55e88549f6b0 2024-04-25T10:38:34.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.260+0000 7f1d1ede0740 1 -- 172.21.15.43:0/1786782065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d0c055070 con 0x55e88549f6b0 2024-04-25T10:38:34.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.260+0000 7f1cf97fa640 1 -- 172.21.15.43:0/1786782065 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d000e4030 con 0x55e88549f6b0 2024-04-25T10:38:34.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.260+0000 7f1d1ede0740 10 client.17172 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:34.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.260+0000 7f1d1ede0740 1 -- 172.21.15.43:0/1786782065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e8858e52f0 con 0x55e88549f6b0 2024-04-25T10:38:34.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.260+0000 7f1cf97fa640 1 -- 172.21.15.43:0/1786782065 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d0c055070 con 0x55e88549f6b0 2024-04-25T10:38:34.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.260+0000 7f1cf97fa640 1 -- 172.21.15.43:0/1786782065 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1d000ac1b0 con 0x55e88549f6b0 2024-04-25T10:38:34.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.260+0000 7f1d1ede0740 10 client.17172 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:34.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.260+0000 7f1d1ede0740 10 client.17172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:34.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.260+0000 7f1d1ede0740 20 client.17172 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:34.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.260+0000 7f1d1ede0740 1 --2- 172.21.15.43:0/1786782065 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e8854543a0 0x55e8858e7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.260+0000 7f1d1ede0740 4 client.17172 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:38:34.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.260+0000 7f1d1ede0740 1 -- 172.21.15.43:0/1786782065 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e8857ce800 con 0x55e8854543a0 2024-04-25T10:38:34.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.261+0000 7f1d15bef640 1 --2- 172.21.15.43:0/1786782065 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e8854543a0 0x55e8858e7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.261+0000 7f1cf97fa640 10 client.17172 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:34.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.261+0000 7f1d15bef640 1 --2- 172.21.15.43:0/1786782065 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e8854543a0 0x55e8858e7aa0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=424e263ba828ca4f server_cookie=d8f5180d222b4b70 in_seq=0 out_seq=0 2024-04-25T10:38:34.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.261+0000 7f1cf97fa640 1 -- 172.21.15.43:0/1786782065 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e8857ce800 con 0x55e8854543a0 2024-04-25T10:38:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.261+0000 7f1cf97fa640 10 client.17172 handle_command_reply: tid=0 2024-04-25T10:38:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.261+0000 7f1d1ede0740 1 client.17172 shutdown 2024-04-25T10:38:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.261+0000 7f1d1ede0740 1 -- 172.21.15.43:0/1786782065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e88549f6b0 msgr2=0x55e8858de3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.261+0000 7f1d1ede0740 1 --2- 172.21.15.43:0/1786782065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e88549f6b0 0x55e8858de3a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f1d000a2da0 tx=0x7f1d000097b0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d1ede0740 1 -- 172.21.15.43:0/1786782065 shutdown_connections 2024-04-25T10:38:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d1ede0740 1 --2- 172.21.15.43:0/1786782065 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e8854543a0 0x55e8858e7aa0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d1ede0740 1 --2- 172.21.15.43:0/1786782065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8854c0390 0x55e8858725e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1cf97fa640 0 client.17172 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d1ede0740 1 --2- 172.21.15.43:0/1786782065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e88549f6b0 0x55e8858de3a0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d1ede0740 1 --2- 172.21.15.43:0/1786782065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8854974d0 0x55e8858dde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d1ede0740 1 -- 172.21.15.43:0/1786782065 >> 172.21.15.43:0/1786782065 conn(0x55e8854b44e0 msgr2=0x55e885862660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d1ede0740 1 -- 172.21.15.43:0/1786782065 shutdown_connections 2024-04-25T10:38:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d1ede0740 1 -- 172.21.15.43:0/1786782065 wait complete. 2024-04-25T10:38:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d1ede0740 20 client.17172 trim_cache size 0 max 16384 2024-04-25T10:38:34.285 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:38:34.285 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:38:34.286 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:38:34.286 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:38:34.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d17e53640 1 -- 172.21.15.43:0/135660798 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1cfc0afc60 msgr2=0x7f1cfc0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d17e53640 1 --2- 172.21.15.43:0/135660798 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1cfc0afc60 0x7f1cfc0b2170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c031440 tx=0x7f1d0c005890 comp rx=0 tx=0).stop 2024-04-25T10:38:34.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d17e53640 1 -- 172.21.15.43:0/135660798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1010b6d0 msgr2=0x7f1d10102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d17e53640 1 --2- 172.21.15.43:0/135660798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1010b6d0 0x7f1d10102a70 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000d770 tx=0x7f1d0000dc30 comp rx=0 tx=0).stop 2024-04-25T10:38:34.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d17e53640 1 -- 172.21.15.43:0/135660798 shutdown_connections 2024-04-25T10:38:34.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d17e53640 1 --2- 172.21.15.43:0/135660798 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1cfc0afc60 0x7f1cfc0b2170 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d17e53640 1 --2- 172.21.15.43:0/135660798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d1010e0c0 0x7f1d10102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d17e53640 1 --2- 172.21.15.43:0/135660798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1010b6d0 0x7f1d10102a70 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d17e53640 1 --2- 172.21.15.43:0/135660798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d10104c70 0x7f1d10102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d17e53640 1 -- 172.21.15.43:0/135660798 >> 172.21.15.43:0/135660798 conn(0x7f1d10078110 msgr2=0x7f1d10101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:34.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d17e53640 1 -- 172.21.15.43:0/135660798 shutdown_connections 2024-04-25T10:38:34.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.262+0000 7f1d17e53640 1 -- 172.21.15.43:0/135660798 wait complete. 2024-04-25T10:38:34.289 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:33 smithi043 ceph-mon[29478]: pgmap v184: 97 pgs: 97 active+clean; 4.7 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-04-25T10:38:34.291 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:38:34.291 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:38:34.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:33 smithi164 ceph-mon[32713]: pgmap v184: 97 pgs: 97 active+clean; 4.7 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-04-25T10:38:34.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:33 smithi077 ceph-mon[32772]: pgmap v184: 97 pgs: 97 active+clean; 4.7 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-04-25T10:38:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.402+0000 7fb482051640 1 -- 172.21.15.43:0/2307894158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c1032b0 msgr2=0x7fb47c1036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.402+0000 7fb482051640 1 --2- 172.21.15.43:0/2307894158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c1032b0 0x7fb47c1036c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb470009900 tx=0x7fb47002a410 comp rx=0 tx=0).stop 2024-04-25T10:38:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.402+0000 7fb482051640 1 -- 172.21.15.43:0/2307894158 shutdown_connections 2024-04-25T10:38:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.402+0000 7fb482051640 1 --2- 172.21.15.43:0/2307894158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb47c10d430 0x7fb47c10f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.402+0000 7fb482051640 1 --2- 172.21.15.43:0/2307894158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c1032b0 0x7fb47c1036c0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.402+0000 7fb482051640 1 --2- 172.21.15.43:0/2307894158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb47c107940 0x7fb47c102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.402+0000 7fb482051640 1 -- 172.21.15.43:0/2307894158 >> 172.21.15.43:0/2307894158 conn(0x7fb47c0fd850 msgr2=0x7fb47c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.402+0000 7fb482051640 1 -- 172.21.15.43:0/2307894158 shutdown_connections 2024-04-25T10:38:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.402+0000 7fb482051640 1 -- 172.21.15.43:0/2307894158 wait complete. 2024-04-25T10:38:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.403+0000 7fb482051640 1 Processor -- start 2024-04-25T10:38:34.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.403+0000 7fb482051640 1 -- start start 2024-04-25T10:38:34.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.403+0000 7fb482051640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb47c1032b0 0x7fb47c19b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.403+0000 7fb482051640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb47c107940 0x7fb47c19ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.403+0000 7fb482051640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c10d430 0x7fb47c19bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.403+0000 7fb482051640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c112c40 con 0x7fb47c10d430 2024-04-25T10:38:34.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.403+0000 7fb482051640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c112ac0 con 0x7fb47c107940 2024-04-25T10:38:34.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.403+0000 7fb482051640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c112dc0 con 0x7fb47c1032b0 2024-04-25T10:38:34.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.403+0000 7fb47bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c10d430 0x7fb47c19bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.403+0000 7fb47bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c10d430 0x7fb47c19bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37692/0 (socket says 172.21.15.43:37692) 2024-04-25T10:38:34.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.403+0000 7fb47bfff640 1 -- 172.21.15.43:0/3437565033 learned_addr learned my addr 172.21.15.43:0/3437565033 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:34.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.403+0000 7fb47affd640 1 --2- 172.21.15.43:0/3437565033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb47c107940 0x7fb47c19ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.403+0000 7fb47b7fe640 1 --2- 172.21.15.43:0/3437565033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb47c1032b0 0x7fb47c19b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.404+0000 7fb47bfff640 1 -- 172.21.15.43:0/3437565033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb47c1032b0 msgr2=0x7fb47c19b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.404+0000 7fb47bfff640 1 --2- 172.21.15.43:0/3437565033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb47c1032b0 0x7fb47c19b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.404+0000 7fb47bfff640 1 -- 172.21.15.43:0/3437565033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb47c107940 msgr2=0x7fb47c19ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.404+0000 7fb47bfff640 1 --2- 172.21.15.43:0/3437565033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb47c107940 0x7fb47c19ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.404+0000 7fb47bfff640 1 -- 172.21.15.43:0/3437565033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47c1a6850 con 0x7fb47c10d430 2024-04-25T10:38:34.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.404+0000 7fb47b7fe640 1 --2- 172.21.15.43:0/3437565033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb47c1032b0 0x7fb47c19b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:38:34.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.404+0000 7fb47bfff640 1 --2- 172.21.15.43:0/3437565033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c10d430 0x7fb47c19bfc0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb47400e9d0 tx=0x7fb47400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:34.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.404+0000 7fb478ff9640 1 -- 172.21.15.43:0/3437565033 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47400cd80 con 0x7fb47c10d430 2024-04-25T10:38:34.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.404+0000 7fb478ff9640 1 -- 172.21.15.43:0/3437565033 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb474005020 con 0x7fb47c10d430 2024-04-25T10:38:34.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.404+0000 7fb482051640 1 -- 172.21.15.43:0/3437565033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47c1a6b40 con 0x7fb47c10d430 2024-04-25T10:38:34.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.404+0000 7fb478ff9640 1 -- 172.21.15.43:0/3437565033 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4740116d0 con 0x7fb47c10d430 2024-04-25T10:38:34.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.404+0000 7fb482051640 1 -- 172.21.15.43:0/3437565033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb47c1a7060 con 0x7fb47c10d430 2024-04-25T10:38:34.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.405+0000 7fb482051640 1 -- 172.21.15.43:0/3437565033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb47c06a190 con 0x7fb47c10d430 2024-04-25T10:38:34.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.407+0000 7fb478ff9640 1 -- 172.21.15.43:0/3437565033 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fb4740051d0 con 0x7fb47c10d430 2024-04-25T10:38:34.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.407+0000 7fb478ff9640 1 --2- 172.21.15.43:0/3437565033 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb4640afc60 0x7fb4640b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.407+0000 7fb47b7fe640 1 --2- 172.21.15.43:0/3437565033 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb4640afc60 0x7fb4640b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.407+0000 7fb478ff9640 1 -- 172.21.15.43:0/3437565033 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fb4740df530 con 0x7fb47c10d430 2024-04-25T10:38:34.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.408+0000 7fb47b7fe640 1 --2- 172.21.15.43:0/3437565033 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb4640afc60 0x7fb4640b2170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb46c0058c0 tx=0x7fb46c005850 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:34.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.411+0000 7fb478ff9640 1 -- 172.21.15.43:0/3437565033 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb474016030 con 0x7fb47c10d430 2024-04-25T10:38:34.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.637+0000 7fb482051640 1 -- 172.21.15.43:0/3437565033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb47c102250 con 0x7fb47c10d430 2024-04-25T10:38:34.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.638+0000 7fb478ff9640 1 -- 172.21.15.43:0/3437565033 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7fb4740a7da0 con 0x7fb47c10d430 2024-04-25T10:38:34.640 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:38:34.640 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:38:34.640 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:38:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.640+0000 7fb482051640 1 -- 172.21.15.43:0/3437565033 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb4640afc60 msgr2=0x7fb4640b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.640+0000 7fb482051640 1 --2- 172.21.15.43:0/3437565033 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb4640afc60 0x7fb4640b2170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb46c0058c0 tx=0x7fb46c005850 comp rx=0 tx=0).stop 2024-04-25T10:38:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.640+0000 7fb482051640 1 -- 172.21.15.43:0/3437565033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c10d430 msgr2=0x7fb47c19bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.640+0000 7fb482051640 1 --2- 172.21.15.43:0/3437565033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c10d430 0x7fb47c19bfc0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb47400e9d0 tx=0x7fb47400ee90 comp rx=0 tx=0).stop 2024-04-25T10:38:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.640+0000 7fb482051640 1 -- 172.21.15.43:0/3437565033 shutdown_connections 2024-04-25T10:38:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.640+0000 7fb482051640 1 --2- 172.21.15.43:0/3437565033 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb4640afc60 0x7fb4640b2170 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.640+0000 7fb482051640 1 --2- 172.21.15.43:0/3437565033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb47c107940 0x7fb47c19ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.640+0000 7fb482051640 1 --2- 172.21.15.43:0/3437565033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c10d430 0x7fb47c19bfc0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.640+0000 7fb482051640 1 --2- 172.21.15.43:0/3437565033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb47c1032b0 0x7fb47c19b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.640+0000 7fb482051640 1 -- 172.21.15.43:0/3437565033 >> 172.21.15.43:0/3437565033 conn(0x7fb47c0fd850 msgr2=0x7fb47c10e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.640+0000 7fb482051640 1 -- 172.21.15.43:0/3437565033 shutdown_connections 2024-04-25T10:38:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.640+0000 7fb482051640 1 -- 172.21.15.43:0/3437565033 wait complete. 2024-04-25T10:38:34.653 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:38:34.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.765+0000 7fc92afba640 1 -- 172.21.15.43:0/3482497051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc924109fb0 msgr2=0x7fc924103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.765+0000 7fc92afba640 1 --2- 172.21.15.43:0/3482497051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc924109fb0 0x7fc924103bf0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc910009900 tx=0x7fc91002a410 comp rx=0 tx=0).stop 2024-04-25T10:38:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.765+0000 7fc92afba640 1 -- 172.21.15.43:0/3482497051 shutdown_connections 2024-04-25T10:38:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.765+0000 7fc92afba640 1 --2- 172.21.15.43:0/3482497051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc924104260 0x7fc924110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.765+0000 7fc92afba640 1 --2- 172.21.15.43:0/3482497051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc924109fb0 0x7fc924103bf0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.765+0000 7fc92afba640 1 --2- 172.21.15.43:0/3482497051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc924109b10 0x7fc9241036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.765+0000 7fc92afba640 1 -- 172.21.15.43:0/3482497051 >> 172.21.15.43:0/3482497051 conn(0x7fc9240ff210 msgr2=0x7fc924101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.765+0000 7fc92afba640 1 -- 172.21.15.43:0/3482497051 shutdown_connections 2024-04-25T10:38:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.765+0000 7fc92afba640 1 -- 172.21.15.43:0/3482497051 wait complete. 2024-04-25T10:38:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.766+0000 7fc92afba640 1 Processor -- start 2024-04-25T10:38:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.766+0000 7fc92afba640 1 -- start start 2024-04-25T10:38:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.766+0000 7fc92afba640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc924104260 0x7fc9241a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.766+0000 7fc92afba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc924109b10 0x7fc9241a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.766+0000 7fc92afba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc924109fb0 0x7fc92419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.766+0000 7fc92afba640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc924113d00 con 0x7fc924109fb0 2024-04-25T10:38:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.766+0000 7fc92afba640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc924113b80 con 0x7fc924109b10 2024-04-25T10:38:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.766+0000 7fc92afba640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc924113e80 con 0x7fc924104260 2024-04-25T10:38:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.766+0000 7fc929557640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc924109fb0 0x7fc92419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.766+0000 7fc929557640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc924109fb0 0x7fc92419a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37698/0 (socket says 172.21.15.43:37698) 2024-04-25T10:38:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.766+0000 7fc929557640 1 -- 172.21.15.43:0/4241002763 learned_addr learned my addr 172.21.15.43:0/4241002763 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.767+0000 7fc923fff640 1 --2- 172.21.15.43:0/4241002763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc924109b10 0x7fc9241a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.767+0000 7fc928d56640 1 --2- 172.21.15.43:0/4241002763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc924104260 0x7fc9241a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.767+0000 7fc929557640 1 -- 172.21.15.43:0/4241002763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc924104260 msgr2=0x7fc9241a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.767+0000 7fc929557640 1 --2- 172.21.15.43:0/4241002763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc924104260 0x7fc9241a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.767+0000 7fc929557640 1 -- 172.21.15.43:0/4241002763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc924109b10 msgr2=0x7fc9241a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.767+0000 7fc929557640 1 --2- 172.21.15.43:0/4241002763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc924109b10 0x7fc9241a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.767+0000 7fc929557640 1 -- 172.21.15.43:0/4241002763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92419acc0 con 0x7fc924109fb0 2024-04-25T10:38:34.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.767+0000 7fc923fff640 1 --2- 172.21.15.43:0/4241002763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc924109b10 0x7fc9241a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:38:34.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.767+0000 7fc928d56640 1 --2- 172.21.15.43:0/4241002763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc924104260 0x7fc9241a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:38:34.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.767+0000 7fc929557640 1 --2- 172.21.15.43:0/4241002763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc924109fb0 0x7fc92419a680 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc91c00cce0 tx=0x7fc91c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:34.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.767+0000 7fc921ffb640 1 -- 172.21.15.43:0/4241002763 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c00f450 con 0x7fc924109fb0 2024-04-25T10:38:34.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.767+0000 7fc92afba640 1 -- 172.21.15.43:0/4241002763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc92419afb0 con 0x7fc924109fb0 2024-04-25T10:38:34.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.767+0000 7fc921ffb640 1 -- 172.21.15.43:0/4241002763 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc91c00faa0 con 0x7fc924109fb0 2024-04-25T10:38:34.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.767+0000 7fc921ffb640 1 -- 172.21.15.43:0/4241002763 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c018b70 con 0x7fc924109fb0 2024-04-25T10:38:34.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.767+0000 7fc92afba640 1 -- 172.21.15.43:0/4241002763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9241a72a0 con 0x7fc924109fb0 2024-04-25T10:38:34.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.770+0000 7fc921ffb640 1 -- 172.21.15.43:0/4241002763 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fc91c018d10 con 0x7fc924109fb0 2024-04-25T10:38:34.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.770+0000 7fc931e60740 1 Processor -- start 2024-04-25T10:38:34.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.770+0000 7fc931e60740 1 -- start start 2024-04-25T10:38:34.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.770+0000 7fc931e60740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180a1390 0x55c018442810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.770+0000 7fc931e60740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180784d0 0x55c018442d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.770+0000 7fc931e60740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180806b0 0x55c01844f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.770+0000 7fc931e60740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c0183a7d80 con 0x55c0180a1390 2024-04-25T10:38:34.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.770+0000 7fc931e60740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c018361a20 con 0x55c0180784d0 2024-04-25T10:38:34.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.770+0000 7fc931e60740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c0183a3d40 con 0x55c0180806b0 2024-04-25T10:38:34.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.770+0000 7fc923fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180a1390 0x55c018442810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.770+0000 7fc923fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180a1390 0x55c018442810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37710/0 (socket says 172.21.15.43:37710) 2024-04-25T10:38:34.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.770+0000 7fc923fff640 1 -- 172.21.15.43:0/3301211046 learned_addr learned my addr 172.21.15.43:0/3301211046 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:34.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.770+0000 7fc921ffb640 1 --2- 172.21.15.43:0/4241002763 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9140afc60 0x7fc9140b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.771+0000 7fc928d56640 1 --2- 172.21.15.43:0/3301211046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180784d0 0x55c018442d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.771+0000 7fc929557640 1 --2- 172.21.15.43:0/4241002763 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9140afc60 0x7fc9140b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.771+0000 7fc923fff640 1 --2- 172.21.15.43:0/3301211046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180806b0 0x55c01844f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.771+0000 7fc921ffb640 1 -- 172.21.15.43:0/4241002763 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc91c0df560 con 0x7fc924109fb0 2024-04-25T10:38:34.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.771+0000 7fc923fff640 1 -- 172.21.15.43:0/3301211046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180806b0 msgr2=0x55c01844f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.771+0000 7fc923fff640 1 --2- 172.21.15.43:0/3301211046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180806b0 0x55c01844f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.771+0000 7fc923fff640 1 -- 172.21.15.43:0/3301211046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180784d0 msgr2=0x55c018442d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.771+0000 7fc923fff640 1 --2- 172.21.15.43:0/3301211046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180784d0 0x55c018442d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.771+0000 7fc923fff640 1 -- 172.21.15.43:0/3301211046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c01836cc30 con 0x55c0180a1390 2024-04-25T10:38:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.771+0000 7fc928d56640 1 --2- 172.21.15.43:0/3301211046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180784d0 0x55c018442d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:38:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.771+0000 7fc923fff640 1 --2- 172.21.15.43:0/3301211046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180a1390 0x55c018442810 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fc910000c00 tx=0x7fc910009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8092a7ac365b076 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.771+0000 7fc929557640 1 --2- 172.21.15.43:0/4241002763 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9140afc60 0x7fc9140b2170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0dcd90 tx=0x7fc91c074f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.771+0000 7fc90effd640 1 -- 172.21.15.43:0/3301211046 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc910045070 con 0x55c0180a1390 2024-04-25T10:38:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.771+0000 7fc90effd640 1 -- 172.21.15.43:0/3301211046 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc910005040 con 0x55c0180a1390 2024-04-25T10:38:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.771+0000 7fc90effd640 1 -- 172.21.15.43:0/3301211046 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc910035050 con 0x55c0180a1390 2024-04-25T10:38:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.772+0000 7fc931e60740 1 -- 172.21.15.43:0/3301211046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180a1390 msgr2=0x55c018442810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.772+0000 7fc931e60740 1 --2- 172.21.15.43:0/3301211046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180a1390 0x55c018442810 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fc910000c00 tx=0x7fc910009560 comp rx=0 tx=0).stop 2024-04-25T10:38:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.772+0000 7fc931e60740 1 -- 172.21.15.43:0/3301211046 shutdown_connections 2024-04-25T10:38:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.772+0000 7fc931e60740 1 --2- 172.21.15.43:0/3301211046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180784d0 0x55c018442d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.772+0000 7fc931e60740 1 --2- 172.21.15.43:0/3301211046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180806b0 0x55c01844f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.772+0000 7fc931e60740 1 --2- 172.21.15.43:0/3301211046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180a1390 0x55c018442810 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.772+0000 7fc931e60740 1 -- 172.21.15.43:0/3301211046 >> 172.21.15.43:0/3301211046 conn(0x55c0180954e0 msgr2=0x55c01843efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:34.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.772+0000 7fc931e60740 1 -- 172.21.15.43:0/3301211046 shutdown_connections 2024-04-25T10:38:34.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.772+0000 7fc931e60740 1 -- 172.21.15.43:0/3301211046 wait complete. 2024-04-25T10:38:34.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.772+0000 7fc931e60740 1 -- start start 2024-04-25T10:38:34.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.773+0000 7fc931e60740 1 Processor -- start 2024-04-25T10:38:34.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.773+0000 7fc931e60740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180784d0 0x55c01844d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.773+0000 7fc931e60740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180806b0 0x55c01844d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.773+0000 7fc931e60740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180a1390 0x55c0184591f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.773+0000 7fc931e60740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c01844fd30 con 0x55c0180a1390 2024-04-25T10:38:34.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.773+0000 7fc931e60740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c01832edc0 con 0x55c0180806b0 2024-04-25T10:38:34.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.773+0000 7fc931e60740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c01844feb0 con 0x55c0180784d0 2024-04-25T10:38:34.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.773+0000 7fc931e60740 10 client.0 _ll_register_callbacks cb 0x55c01834a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:34.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.773+0000 7fc923fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180a1390 0x55c0184591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.773+0000 7fc923fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180a1390 0x55c0184591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37726/0 (socket says 172.21.15.43:37726) 2024-04-25T10:38:34.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.773+0000 7fc923fff640 1 -- 172.21.15.43:0/2151327826 learned_addr learned my addr 172.21.15.43:0/2151327826 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:34.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.773+0000 7fc923fff640 1 --2- 172.21.15.43:0/2151327826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180784d0 0x55c01844d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.773+0000 7fc928d56640 1 --2- 172.21.15.43:0/2151327826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180806b0 0x55c01844d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.774+0000 7fc923fff640 1 -- 172.21.15.43:0/2151327826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180784d0 msgr2=0x55c01844d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.774+0000 7fc923fff640 1 --2- 172.21.15.43:0/2151327826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180784d0 0x55c01844d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.774+0000 7fc923fff640 1 -- 172.21.15.43:0/2151327826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180806b0 msgr2=0x55c01844d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.774+0000 7fc923fff640 1 --2- 172.21.15.43:0/2151327826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180806b0 0x55c01844d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.774+0000 7fc923fff640 1 -- 172.21.15.43:0/2151327826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0180b93f0 con 0x55c0180a1390 2024-04-25T10:38:34.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.774+0000 7fc923fff640 1 --2- 172.21.15.43:0/2151327826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180a1390 0x55c0184591f0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc910031940 tx=0x7fc910004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547c8344232b1aea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:34.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.774+0000 7fc903fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:34.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.774+0000 7fc903fff640 1 -- 172.21.15.43:0/2151327826 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc910035070 con 0x55c0180a1390 2024-04-25T10:38:34.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.774+0000 7fc903fff640 1 -- 172.21.15.43:0/2151327826 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc9100048c0 con 0x55c0180a1390 2024-04-25T10:38:34.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.774+0000 7fc931e60740 1 -- 172.21.15.43:0/2151327826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc910005040 con 0x55c0180a1390 2024-04-25T10:38:34.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.774+0000 7fc903fff640 1 -- 172.21.15.43:0/2151327826 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc910045070 con 0x55c0180a1390 2024-04-25T10:38:34.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.774+0000 7fc931e60740 10 client.17190 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:34.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.774+0000 7fc931e60740 1 -- 172.21.15.43:0/2151327826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c0181837d0 con 0x55c0180a1390 2024-04-25T10:38:34.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.774+0000 7fc903fff640 1 -- 172.21.15.43:0/2151327826 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc910005040 con 0x55c0180a1390 2024-04-25T10:38:34.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.775+0000 7fc903fff640 1 -- 172.21.15.43:0/2151327826 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc9100039a0 con 0x55c0180a1390 2024-04-25T10:38:34.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.775+0000 7fc931e60740 10 client.17190 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:34.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.775+0000 7fc931e60740 10 client.17190 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:38:34.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.775+0000 7fc931e60740 20 client.17190 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:34.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.775+0000 7fc931e60740 1 --2- 172.21.15.43:0/2151327826 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c01805ff60 0x55c01845f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.775+0000 7fc931e60740 4 client.17190 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:38:34.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.775+0000 7fc931e60740 1 -- 172.21.15.43:0/2151327826 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0183b5990 con 0x55c01805ff60 2024-04-25T10:38:34.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.775+0000 7fc928d56640 1 --2- 172.21.15.43:0/2151327826 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c01805ff60 0x55c01845f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.775+0000 7fc928d56640 1 --2- 172.21.15.43:0/2151327826 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c01805ff60 0x55c01845f550 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5f28f83078d9f7ba server_cookie=7846e8260d4c509a in_seq=0 out_seq=0 2024-04-25T10:38:34.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.775+0000 7fc903fff640 10 client.17190 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:38:34.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.776+0000 7fc903fff640 1 -- 172.21.15.43:0/2151327826 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c0183b5990 con 0x55c01805ff60 2024-04-25T10:38:34.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.776+0000 7fc903fff640 10 client.17190 handle_command_reply: tid=0 2024-04-25T10:38:34.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.776+0000 7fc931e60740 1 client.17190 shutdown 2024-04-25T10:38:34.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.776+0000 7fc931e60740 1 -- 172.21.15.43:0/2151327826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180a1390 msgr2=0x55c0184591f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.776+0000 7fc931e60740 1 --2- 172.21.15.43:0/2151327826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180a1390 0x55c0184591f0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc910031940 tx=0x7fc910004580 comp rx=0 tx=0).stop 2024-04-25T10:38:34.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.776+0000 7fc931e60740 1 -- 172.21.15.43:0/2151327826 shutdown_connections 2024-04-25T10:38:34.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.776+0000 7fc931e60740 1 --2- 172.21.15.43:0/2151327826 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c01805ff60 0x55c01845f550 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.776+0000 7fc931e60740 1 --2- 172.21.15.43:0/2151327826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180806b0 0x55c01844d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.776+0000 7fc931e60740 1 --2- 172.21.15.43:0/2151327826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180a1390 0x55c0184591f0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.776+0000 7fc931e60740 1 --2- 172.21.15.43:0/2151327826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180784d0 0x55c01844d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.776+0000 7fc903fff640 0 client.17190 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:38:34.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.776+0000 7fc931e60740 1 -- 172.21.15.43:0/2151327826 >> 172.21.15.43:0/2151327826 conn(0x55c0180954e0 msgr2=0x55c018444530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:34.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.777+0000 7fc931e60740 1 -- 172.21.15.43:0/2151327826 shutdown_connections 2024-04-25T10:38:34.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.777+0000 7fc931e60740 1 -- 172.21.15.43:0/2151327826 wait complete. 2024-04-25T10:38:34.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.777+0000 7fc931e60740 20 client.17190 trim_cache size 0 max 16384 2024-04-25T10:38:34.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.790+0000 7fc931e60740 1 Processor -- start 2024-04-25T10:38:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.790+0000 7fc931e60740 1 -- start start 2024-04-25T10:38:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.790+0000 7fc931e60740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180784d0 0x55c01844d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.790+0000 7fc931e60740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180806b0 0x55c018447510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.790+0000 7fc931e60740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180a1390 0x55c018447a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.790+0000 7fc931e60740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc9241b2160 con 0x55c0180806b0 2024-04-25T10:38:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.790+0000 7fc931e60740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9241afa30 con 0x55c0180784d0 2024-04-25T10:38:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.790+0000 7fc931e60740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc9241ad880 con 0x55c0180a1390 2024-04-25T10:38:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.791+0000 7fc928d56640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180806b0 0x55c018447510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.791+0000 7fc928d56640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180806b0 0x55c018447510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37738/0 (socket says 172.21.15.43:37738) 2024-04-25T10:38:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.791+0000 7fc928d56640 1 -- 172.21.15.43:0/1670292493 learned_addr learned my addr 172.21.15.43:0/1670292493 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.791+0000 7fc923fff640 1 --2- 172.21.15.43:0/1670292493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180a1390 0x55c018447a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.791+0000 7fc923fff640 1 --2- 172.21.15.43:0/1670292493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180784d0 0x55c01844d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.791+0000 7fc928d56640 1 -- 172.21.15.43:0/1670292493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180a1390 msgr2=0x55c018447a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.791+0000 7fc928d56640 1 --2- 172.21.15.43:0/1670292493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180a1390 0x55c018447a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.791+0000 7fc928d56640 1 -- 172.21.15.43:0/1670292493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180784d0 msgr2=0x55c01844d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.791+0000 7fc928d56640 1 --2- 172.21.15.43:0/1670292493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180784d0 0x55c01844d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.791+0000 7fc928d56640 1 -- 172.21.15.43:0/1670292493 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c01843a710 con 0x55c0180806b0 2024-04-25T10:38:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.791+0000 7fc923fff640 1 --2- 172.21.15.43:0/1670292493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180784d0 0x55c01844d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:38:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.791+0000 7fc928d56640 1 --2- 172.21.15.43:0/1670292493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180806b0 0x55c018447510 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x55c018443fe0 tx=0x7fc918009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582656149922c071 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.791+0000 7fc90e7fc640 1 -- 172.21.15.43:0/1670292493 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9180142d0 con 0x55c0180806b0 2024-04-25T10:38:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.791+0000 7fc90e7fc640 1 -- 172.21.15.43:0/1670292493 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc91801b330 con 0x55c0180806b0 2024-04-25T10:38:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.791+0000 7fc90e7fc640 1 -- 172.21.15.43:0/1670292493 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918003850 con 0x55c0180806b0 2024-04-25T10:38:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.792+0000 7fc931e60740 1 -- 172.21.15.43:0/1670292493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180806b0 msgr2=0x55c018447510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.792+0000 7fc931e60740 1 --2- 172.21.15.43:0/1670292493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180806b0 0x55c018447510 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x55c018443fe0 tx=0x7fc918009430 comp rx=0 tx=0).stop 2024-04-25T10:38:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.792+0000 7fc931e60740 1 -- 172.21.15.43:0/1670292493 shutdown_connections 2024-04-25T10:38:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.792+0000 7fc931e60740 1 --2- 172.21.15.43:0/1670292493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180a1390 0x55c018447a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.792+0000 7fc931e60740 1 --2- 172.21.15.43:0/1670292493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180806b0 0x55c018447510 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.792+0000 7fc931e60740 1 --2- 172.21.15.43:0/1670292493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180784d0 0x55c01844d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.792+0000 7fc931e60740 1 -- 172.21.15.43:0/1670292493 >> 172.21.15.43:0/1670292493 conn(0x55c0180954e0 msgr2=0x55c01843d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.792+0000 7fc931e60740 1 -- 172.21.15.43:0/1670292493 shutdown_connections 2024-04-25T10:38:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.792+0000 7fc931e60740 1 -- 172.21.15.43:0/1670292493 wait complete. 2024-04-25T10:38:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.792+0000 7fc931e60740 1 -- start start 2024-04-25T10:38:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.793+0000 7fc931e60740 1 Processor -- start 2024-04-25T10:38:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.793+0000 7fc931e60740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180784d0 0x55c01845f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.793+0000 7fc931e60740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180806b0 0x55c01845fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.793+0000 7fc931e60740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180a1390 0x55c0184c5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.793+0000 7fc931e60740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc9241adef0 con 0x55c0180806b0 2024-04-25T10:38:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.793+0000 7fc931e60740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9241af360 con 0x55c0180a1390 2024-04-25T10:38:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.793+0000 7fc931e60740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc9241ae5c0 con 0x55c0180784d0 2024-04-25T10:38:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.793+0000 7fc931e60740 10 client.0 _ll_register_callbacks cb 0x7fc910043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.793+0000 7fc928d56640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180806b0 0x55c01845fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.793+0000 7fc928d56640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180806b0 0x55c01845fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37748/0 (socket says 172.21.15.43:37748) 2024-04-25T10:38:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.793+0000 7fc928d56640 1 -- 172.21.15.43:0/46282012 learned_addr learned my addr 172.21.15.43:0/46282012 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.793+0000 7fc923fff640 1 --2- 172.21.15.43:0/46282012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180a1390 0x55c0184c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.793+0000 7fc923fff640 1 --2- 172.21.15.43:0/46282012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180784d0 0x55c01845f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.793+0000 7fc928d56640 1 -- 172.21.15.43:0/46282012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180784d0 msgr2=0x55c01845f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.793+0000 7fc928d56640 1 --2- 172.21.15.43:0/46282012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180784d0 0x55c01845f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.793+0000 7fc928d56640 1 -- 172.21.15.43:0/46282012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180a1390 msgr2=0x55c0184c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.793+0000 7fc928d56640 1 --2- 172.21.15.43:0/46282012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180a1390 0x55c0184c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.793+0000 7fc928d56640 1 -- 172.21.15.43:0/46282012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0184c5bb0 con 0x55c0180806b0 2024-04-25T10:38:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.793+0000 7fc923fff640 1 --2- 172.21.15.43:0/46282012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180784d0 0x55c01845f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:38:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.794+0000 7fc928d56640 1 --2- 172.21.15.43:0/46282012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180806b0 0x55c01845fcd0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fc918009ab0 tx=0x7fc918014150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511685dd7704644d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.794+0000 7fc903fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.794+0000 7fc903fff640 1 -- 172.21.15.43:0/46282012 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918031060 con 0x55c0180806b0 2024-04-25T10:38:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.794+0000 7fc903fff640 1 -- 172.21.15.43:0/46282012 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc918004020 con 0x55c0180806b0 2024-04-25T10:38:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.794+0000 7fc931e60740 1 -- 172.21.15.43:0/46282012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc91801b330 con 0x55c0180806b0 2024-04-25T10:38:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.794+0000 7fc903fff640 1 -- 172.21.15.43:0/46282012 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc918004220 con 0x55c0180806b0 2024-04-25T10:38:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.794+0000 7fc931e60740 10 client.17196 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.794+0000 7fc931e60740 1 -- 172.21.15.43:0/46282012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c0184c6bf0 con 0x55c0180806b0 2024-04-25T10:38:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.794+0000 7fc903fff640 1 -- 172.21.15.43:0/46282012 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc91801b330 con 0x55c0180806b0 2024-04-25T10:38:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.794+0000 7fc903fff640 1 -- 172.21.15.43:0/46282012 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc91800f040 con 0x55c0180806b0 2024-04-25T10:38:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.794+0000 7fc931e60740 10 client.17196 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.794+0000 7fc931e60740 10 client.17196 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:38:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.794+0000 7fc931e60740 20 client.17196 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.794+0000 7fc931e60740 1 --2- 172.21.15.43:0/46282012 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c0180353a0 0x55c0184c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.794+0000 7fc931e60740 4 client.17196 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:38:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.794+0000 7fc931e60740 1 -- 172.21.15.43:0/46282012 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0183af800 con 0x55c0180353a0 2024-04-25T10:38:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.795+0000 7fc923fff640 1 --2- 172.21.15.43:0/46282012 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c0180353a0 0x55c0184c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.795+0000 7fc923fff640 1 --2- 172.21.15.43:0/46282012 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c0180353a0 0x55c0184c92c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5733eda2c79e7a2a server_cookie=75cbbaf7cd92b4c3 in_seq=0 out_seq=0 2024-04-25T10:38:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.795+0000 7fc903fff640 10 client.17196 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:38:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.795+0000 7fc903fff640 1 -- 172.21.15.43:0/46282012 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0183af800 con 0x55c0180353a0 2024-04-25T10:38:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.795+0000 7fc903fff640 10 client.17196 handle_command_reply: tid=0 2024-04-25T10:38:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.795+0000 7fc931e60740 1 client.17196 shutdown 2024-04-25T10:38:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.795+0000 7fc931e60740 1 -- 172.21.15.43:0/46282012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180806b0 msgr2=0x55c01845fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.795+0000 7fc931e60740 1 --2- 172.21.15.43:0/46282012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180806b0 0x55c01845fcd0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fc918009ab0 tx=0x7fc918014150 comp rx=0 tx=0).stop 2024-04-25T10:38:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.795+0000 7fc931e60740 1 -- 172.21.15.43:0/46282012 shutdown_connections 2024-04-25T10:38:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.795+0000 7fc931e60740 1 --2- 172.21.15.43:0/46282012 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55c0180353a0 0x55c0184c92c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.795+0000 7fc931e60740 1 --2- 172.21.15.43:0/46282012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0180a1390 0x55c0184c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.795+0000 7fc931e60740 1 --2- 172.21.15.43:0/46282012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0180806b0 0x55c01845fcd0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.795+0000 7fc931e60740 1 --2- 172.21.15.43:0/46282012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0180784d0 0x55c01845f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.795+0000 7fc903fff640 0 client.17196 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:38:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.795+0000 7fc931e60740 1 -- 172.21.15.43:0/46282012 >> 172.21.15.43:0/46282012 conn(0x55c0180954e0 msgr2=0x55c018458190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.796+0000 7fc931e60740 1 -- 172.21.15.43:0/46282012 shutdown_connections 2024-04-25T10:38:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.796+0000 7fc931e60740 1 -- 172.21.15.43:0/46282012 wait complete. 2024-04-25T10:38:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.796+0000 7fc931e60740 20 client.17196 trim_cache size 0 max 16384 2024-04-25T10:38:34.804 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:38:34.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.796+0000 7fc92afba640 1 -- 172.21.15.43:0/4241002763 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9140afc60 msgr2=0x7fc9140b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.796+0000 7fc92afba640 1 --2- 172.21.15.43:0/4241002763 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9140afc60 0x7fc9140b2170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0dcd90 tx=0x7fc91c074f90 comp rx=0 tx=0).stop 2024-04-25T10:38:34.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.796+0000 7fc92afba640 1 -- 172.21.15.43:0/4241002763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc924109fb0 msgr2=0x7fc92419a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.796+0000 7fc92afba640 1 --2- 172.21.15.43:0/4241002763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc924109fb0 0x7fc92419a680 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc91c00cce0 tx=0x7fc91c00a620 comp rx=0 tx=0).stop 2024-04-25T10:38:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.796+0000 7fc92afba640 1 -- 172.21.15.43:0/4241002763 shutdown_connections 2024-04-25T10:38:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.796+0000 7fc92afba640 1 --2- 172.21.15.43:0/4241002763 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9140afc60 0x7fc9140b2170 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.796+0000 7fc92afba640 1 --2- 172.21.15.43:0/4241002763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc924109b10 0x7fc9241a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.796+0000 7fc92afba640 1 --2- 172.21.15.43:0/4241002763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc924109fb0 0x7fc92419a680 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.796+0000 7fc92afba640 1 --2- 172.21.15.43:0/4241002763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc924104260 0x7fc9241a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.796+0000 7fc92afba640 1 -- 172.21.15.43:0/4241002763 >> 172.21.15.43:0/4241002763 conn(0x7fc9240ff210 msgr2=0x7fc92410f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.796+0000 7fc92afba640 1 -- 172.21.15.43:0/4241002763 shutdown_connections 2024-04-25T10:38:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.796+0000 7fc92afba640 1 -- 172.21.15.43:0/4241002763 wait complete. 2024-04-25T10:38:34.809 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:38:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.921+0000 7f0bcb6e7640 1 -- 172.21.15.43:0/3845631527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc4102840 msgr2=0x7f0bc4110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.921+0000 7f0bcb6e7640 1 --2- 172.21.15.43:0/3845631527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc4102840 0x7f0bc4110620 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009ca0 tx=0x7f0bbc02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.921+0000 7f0bcb6e7640 1 -- 172.21.15.43:0/3845631527 shutdown_connections 2024-04-25T10:38:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.921+0000 7f0bcb6e7640 1 --2- 172.21.15.43:0/3845631527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bc4109fb0 0x7f0bc41021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.922+0000 7f0bcb6e7640 1 --2- 172.21.15.43:0/3845631527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc4102840 0x7f0bc4110620 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.922+0000 7f0bcb6e7640 1 --2- 172.21.15.43:0/3845631527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bc4109b10 0x7f0bc4101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.922+0000 7f0bcb6e7640 1 -- 172.21.15.43:0/3845631527 >> 172.21.15.43:0/3845631527 conn(0x7f0bc4077880 msgr2=0x7f0bc4077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.922+0000 7f0bcb6e7640 1 -- 172.21.15.43:0/3845631527 shutdown_connections 2024-04-25T10:38:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.922+0000 7f0bcb6e7640 1 -- 172.21.15.43:0/3845631527 wait complete. 2024-04-25T10:38:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.922+0000 7f0bcb6e7640 1 Processor -- start 2024-04-25T10:38:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.922+0000 7f0bcb6e7640 1 -- start start 2024-04-25T10:38:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.923+0000 7f0bcb6e7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bc4109b10 0x7f0bc419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.923+0000 7f0bcb6e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc4109fb0 0x7f0bc419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.923+0000 7f0bcb6e7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bc419eb90 0x7f0bc419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.923+0000 7f0bcb6e7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc4113d30 con 0x7f0bc4109fb0 2024-04-25T10:38:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.923+0000 7f0bcb6e7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc4113bb0 con 0x7f0bc419eb90 2024-04-25T10:38:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.923+0000 7f0bcb6e7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc4113eb0 con 0x7f0bc4109b10 2024-04-25T10:38:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.923+0000 7f0bc8c82640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc4109fb0 0x7f0bc419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.923+0000 7f0bc8c82640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc4109fb0 0x7f0bc419a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37762/0 (socket says 172.21.15.43:37762) 2024-04-25T10:38:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.923+0000 7f0bc8c82640 1 -- 172.21.15.43:0/3678586597 learned_addr learned my addr 172.21.15.43:0/3678586597 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.923+0000 7f0bc9483640 1 --2- 172.21.15.43:0/3678586597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bc4109b10 0x7f0bc419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.923+0000 7f0bc9c84640 1 --2- 172.21.15.43:0/3678586597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bc419eb90 0x7f0bc419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.923+0000 7f0bc8c82640 1 -- 172.21.15.43:0/3678586597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bc4109b10 msgr2=0x7f0bc419a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.923+0000 7f0bc8c82640 1 --2- 172.21.15.43:0/3678586597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bc4109b10 0x7f0bc419a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.923+0000 7f0bc8c82640 1 -- 172.21.15.43:0/3678586597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bc419eb90 msgr2=0x7f0bc419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.923+0000 7f0bc8c82640 1 --2- 172.21.15.43:0/3678586597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bc419eb90 0x7f0bc419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.923+0000 7f0bc8c82640 1 -- 172.21.15.43:0/3678586597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc419f5e0 con 0x7f0bc4109fb0 2024-04-25T10:38:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.923+0000 7f0bc9c84640 1 --2- 172.21.15.43:0/3678586597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bc419eb90 0x7f0bc419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:38:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.923+0000 7f0bc8c82640 1 --2- 172.21.15.43:0/3678586597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc4109fb0 0x7f0bc419a8d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400e9f0 tx=0x7f0bb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.923+0000 7f0bba7fc640 1 -- 172.21.15.43:0/3678586597 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb400ccc0 con 0x7f0bc4109fb0 2024-04-25T10:38:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.924+0000 7f0bba7fc640 1 -- 172.21.15.43:0/3678586597 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bb40108e0 con 0x7f0bc4109fb0 2024-04-25T10:38:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.924+0000 7f0bcb6e7640 1 -- 172.21.15.43:0/3678586597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc419f8d0 con 0x7f0bc4109fb0 2024-04-25T10:38:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.924+0000 7f0bba7fc640 1 -- 172.21.15.43:0/3678586597 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4018860 con 0x7f0bc4109fb0 2024-04-25T10:38:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.924+0000 7f0bcb6e7640 1 -- 172.21.15.43:0/3678586597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc41a70d0 con 0x7f0bc4109fb0 2024-04-25T10:38:34.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.926+0000 7f0bd2597740 1 Processor -- start 2024-04-25T10:38:34.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.926+0000 7f0bd2597740 1 -- start start 2024-04-25T10:38:34.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.927+0000 7f0bba7fc640 1 -- 172.21.15.43:0/3678586597 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f0bb4010430 con 0x7f0bc4109fb0 2024-04-25T10:38:34.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.927+0000 7f0bd2597740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc103e390 0x558bc13df6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.927+0000 7f0bd2597740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc10154d0 0x558bc13dfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.927+0000 7f0bd2597740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bc101d6b0 0x558bc13f0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.927+0000 7f0bd2597740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558bc1344d80 con 0x558bc103e390 2024-04-25T10:38:34.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.927+0000 7f0bd2597740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558bc12fea20 con 0x558bc10154d0 2024-04-25T10:38:34.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.927+0000 7f0bd2597740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558bc1340d40 con 0x558bc101d6b0 2024-04-25T10:38:34.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.927+0000 7f0bc9c84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc103e390 0x558bc13df6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.927+0000 7f0bc9c84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc103e390 0x558bc13df6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37768/0 (socket says 172.21.15.43:37768) 2024-04-25T10:38:34.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.927+0000 7f0bc9c84640 1 -- 172.21.15.43:0/2284610774 learned_addr learned my addr 172.21.15.43:0/2284610774 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:34.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.927+0000 7f0bc9483640 1 --2- 172.21.15.43:0/2284610774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc10154d0 0x558bc13dfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.927+0000 7f0bc8c82640 1 --2- 172.21.15.43:0/2284610774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bc101d6b0 0x558bc13f0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.927+0000 7f0bc9c84640 1 -- 172.21.15.43:0/2284610774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bc101d6b0 msgr2=0x558bc13f0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.927+0000 7f0bc9c84640 1 --2- 172.21.15.43:0/2284610774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bc101d6b0 0x558bc13f0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.927+0000 7f0bc9c84640 1 -- 172.21.15.43:0/2284610774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc10154d0 msgr2=0x558bc13dfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.927+0000 7f0bc9c84640 1 --2- 172.21.15.43:0/2284610774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc10154d0 0x558bc13dfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.927+0000 7f0bc9c84640 1 -- 172.21.15.43:0/2284610774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bc1309c30 con 0x558bc103e390 2024-04-25T10:38:34.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.927+0000 7f0bba7fc640 1 --2- 172.21.15.43:0/3678586597 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0bb00afc60 0x7f0bb00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.927+0000 7f0bc9c84640 1 --2- 172.21.15.43:0/2284610774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc103e390 0x558bc13df6c0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc006010 tx=0x7f0bbc009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86568110990f159c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:34.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.928+0000 7f0bc9483640 1 --2- 172.21.15.43:0/3678586597 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0bb00afc60 0x7f0bb00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.928+0000 7f0bba7fc640 1 -- 172.21.15.43:0/3678586597 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f0bb40df2e0 con 0x7f0bc4109fb0 2024-04-25T10:38:34.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.928+0000 7f0baf7fe640 1 -- 172.21.15.43:0/2284610774 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc031cf0 con 0x558bc103e390 2024-04-25T10:38:34.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.928+0000 7f0baf7fe640 1 -- 172.21.15.43:0/2284610774 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bbc036070 con 0x558bc103e390 2024-04-25T10:38:34.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.928+0000 7f0baf7fe640 1 -- 172.21.15.43:0/2284610774 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc0042a0 con 0x558bc103e390 2024-04-25T10:38:34.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.929+0000 7f0bd2597740 1 -- 172.21.15.43:0/2284610774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc103e390 msgr2=0x558bc13df6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.929+0000 7f0bd2597740 1 --2- 172.21.15.43:0/2284610774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc103e390 0x558bc13df6c0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc006010 tx=0x7f0bbc009800 comp rx=0 tx=0).stop 2024-04-25T10:38:34.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.929+0000 7f0bc9483640 1 --2- 172.21.15.43:0/3678586597 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0bb00afc60 0x7f0bb00b2170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x558bc13dd070 tx=0x7f0bc0005f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:34.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.929+0000 7f0bd2597740 1 -- 172.21.15.43:0/2284610774 shutdown_connections 2024-04-25T10:38:34.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.929+0000 7f0bd2597740 1 --2- 172.21.15.43:0/2284610774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc10154d0 0x558bc13dfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.929+0000 7f0bd2597740 1 --2- 172.21.15.43:0/2284610774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bc101d6b0 0x558bc13f0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.929+0000 7f0bd2597740 1 --2- 172.21.15.43:0/2284610774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc103e390 0x558bc13df6c0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.929+0000 7f0bd2597740 1 -- 172.21.15.43:0/2284610774 >> 172.21.15.43:0/2284610774 conn(0x558bc10324e0 msgr2=0x558bc13dbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:34.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.929+0000 7f0bd2597740 1 -- 172.21.15.43:0/2284610774 shutdown_connections 2024-04-25T10:38:34.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.929+0000 7f0bd2597740 1 -- 172.21.15.43:0/2284610774 wait complete. 2024-04-25T10:38:34.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.929+0000 7f0bd2597740 1 -- start start 2024-04-25T10:38:34.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.929+0000 7f0bd2597740 1 Processor -- start 2024-04-25T10:38:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bd2597740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bc10154d0 0x558bc13ee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bd2597740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc101d6b0 0x558bc13eeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bd2597740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc103e390 0x558bc13fa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bd2597740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558bc13f1140 con 0x558bc103e390 2024-04-25T10:38:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bd2597740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558bc12cbdc0 con 0x558bc101d6b0 2024-04-25T10:38:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bd2597740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558bc13f12c0 con 0x558bc10154d0 2024-04-25T10:38:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bd2597740 10 client.0 _ll_register_callbacks cb 0x558bc12e7600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bc9c84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc103e390 0x558bc13fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bc9c84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc103e390 0x558bc13fa680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37772/0 (socket says 172.21.15.43:37772) 2024-04-25T10:38:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bc9c84640 1 -- 172.21.15.43:0/3627694715 learned_addr learned my addr 172.21.15.43:0/3627694715 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bc9483640 1 --2- 172.21.15.43:0/3627694715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bc10154d0 0x558bc13ee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bc8c82640 1 --2- 172.21.15.43:0/3627694715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc101d6b0 0x558bc13eeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bc9c84640 1 -- 172.21.15.43:0/3627694715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bc10154d0 msgr2=0x558bc13ee620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bc9c84640 1 --2- 172.21.15.43:0/3627694715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bc10154d0 0x558bc13ee620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bc9c84640 1 -- 172.21.15.43:0/3627694715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc101d6b0 msgr2=0x558bc13eeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bc9c84640 1 --2- 172.21.15.43:0/3627694715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc101d6b0 0x558bc13eeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bc9c84640 1 -- 172.21.15.43:0/3627694715 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bc10563f0 con 0x558bc103e390 2024-04-25T10:38:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bc8c82640 1 --2- 172.21.15.43:0/3627694715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc101d6b0 0x558bc13eeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:38:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bc9c84640 1 --2- 172.21.15.43:0/3627694715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc103e390 0x558bc13fa680 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc02a900 tx=0x7f0bbc02f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b161fd826b0fd25d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bacff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.930+0000 7f0bacff9640 1 -- 172.21.15.43:0/3627694715 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc049040 con 0x558bc103e390 2024-04-25T10:38:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.931+0000 7f0bacff9640 1 -- 172.21.15.43:0/3627694715 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f0bbc031cf0 con 0x558bc103e390 2024-04-25T10:38:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.931+0000 7f0bd2597740 1 -- 172.21.15.43:0/3627694715 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bbc036070 con 0x558bc103e390 2024-04-25T10:38:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.931+0000 7f0bacff9640 1 -- 172.21.15.43:0/3627694715 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0bbc05b050 con 0x558bc103e390 2024-04-25T10:38:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.931+0000 7f0bd2597740 10 client.17208 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.931+0000 7f0bd2597740 1 -- 172.21.15.43:0/3627694715 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558bc11207d0 con 0x558bc103e390 2024-04-25T10:38:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.931+0000 7f0bacff9640 1 -- 172.21.15.43:0/3627694715 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bbc036070 con 0x558bc103e390 2024-04-25T10:38:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.931+0000 7f0bacff9640 1 -- 172.21.15.43:0/3627694715 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0bbc04c070 con 0x558bc103e390 2024-04-25T10:38:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.931+0000 7f0bd2597740 10 client.17208 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.931+0000 7f0bd2597740 10 client.17208 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:38:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.931+0000 7f0bd2597740 20 client.17208 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.931+0000 7f0bd2597740 1 --2- 172.21.15.43:0/3627694715 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x558bc0ffcf60 0x558bc1400900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.931+0000 7f0bd2597740 4 client.17208 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:38:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.931+0000 7f0bd2597740 1 -- 172.21.15.43:0/3627694715 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bc1352990 con 0x558bc0ffcf60 2024-04-25T10:38:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.931+0000 7f0bc9483640 1 --2- 172.21.15.43:0/3627694715 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x558bc0ffcf60 0x558bc1400900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.932+0000 7f0bc9483640 1 --2- 172.21.15.43:0/3627694715 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x558bc0ffcf60 0x558bc1400900 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=688167a26d71cfea server_cookie=efa0b75d824da176 in_seq=0 out_seq=0 2024-04-25T10:38:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.932+0000 7f0bacff9640 10 client.17208 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:38:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.933+0000 7f0bacff9640 1 -- 172.21.15.43:0/3627694715 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558bc1352990 con 0x558bc0ffcf60 2024-04-25T10:38:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.933+0000 7f0bacff9640 10 client.17208 handle_command_reply: tid=0 2024-04-25T10:38:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.933+0000 7f0bd2597740 1 client.17208 shutdown 2024-04-25T10:38:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.933+0000 7f0bd2597740 1 -- 172.21.15.43:0/3627694715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc103e390 msgr2=0x558bc13fa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.933+0000 7f0bd2597740 1 --2- 172.21.15.43:0/3627694715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc103e390 0x558bc13fa680 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc02a900 tx=0x7f0bbc02f420 comp rx=0 tx=0).stop 2024-04-25T10:38:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.933+0000 7f0bd2597740 1 -- 172.21.15.43:0/3627694715 shutdown_connections 2024-04-25T10:38:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.933+0000 7f0bd2597740 1 --2- 172.21.15.43:0/3627694715 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x558bc0ffcf60 0x558bc1400900 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.933+0000 7f0bd2597740 1 --2- 172.21.15.43:0/3627694715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc101d6b0 0x558bc13eeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.933+0000 7f0bd2597740 1 --2- 172.21.15.43:0/3627694715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc103e390 0x558bc13fa680 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.933+0000 7f0bacff9640 0 client.17208 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:38:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.933+0000 7f0bd2597740 1 --2- 172.21.15.43:0/3627694715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bc10154d0 0x558bc13ee620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.933+0000 7f0bd2597740 1 -- 172.21.15.43:0/3627694715 >> 172.21.15.43:0/3627694715 conn(0x558bc10324e0 msgr2=0x558bc13e14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.933+0000 7f0bd2597740 1 -- 172.21.15.43:0/3627694715 shutdown_connections 2024-04-25T10:38:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.933+0000 7f0bd2597740 1 -- 172.21.15.43:0/3627694715 wait complete. 2024-04-25T10:38:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.933+0000 7f0bd2597740 20 client.17208 trim_cache size 0 max 16384 2024-04-25T10:38:34.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.947+0000 7f0bd2597740 1 Processor -- start 2024-04-25T10:38:34.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.947+0000 7f0bd2597740 1 -- start start 2024-04-25T10:38:34.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.947+0000 7f0bd2597740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc10154d0 0x558bc13fa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.947+0000 7f0bd2597740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc101d6b0 0x558bc13e1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.947+0000 7f0bd2597740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bc103e390 0x558bc13eb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.947+0000 7f0bd2597740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc41afff0 con 0x558bc101d6b0 2024-04-25T10:38:34.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.947+0000 7f0bd2597740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc41ad8c0 con 0x558bc10154d0 2024-04-25T10:38:34.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.947+0000 7f0bd2597740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc41ab730 con 0x558bc103e390 2024-04-25T10:38:34.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.947+0000 7f0bc8c82640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc101d6b0 0x558bc13e1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.947+0000 7f0bc8c82640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc101d6b0 0x558bc13e1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37788/0 (socket says 172.21.15.43:37788) 2024-04-25T10:38:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.947+0000 7f0bc8c82640 1 -- 172.21.15.43:0/1328679077 learned_addr learned my addr 172.21.15.43:0/1328679077 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.947+0000 7f0bc9c84640 1 --2- 172.21.15.43:0/1328679077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bc103e390 0x558bc13eb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.947+0000 7f0bc9483640 1 --2- 172.21.15.43:0/1328679077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc10154d0 0x558bc13fa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.947+0000 7f0bc8c82640 1 -- 172.21.15.43:0/1328679077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bc103e390 msgr2=0x558bc13eb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.947+0000 7f0bc8c82640 1 --2- 172.21.15.43:0/1328679077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bc103e390 0x558bc13eb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.947+0000 7f0bc8c82640 1 -- 172.21.15.43:0/1328679077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc10154d0 msgr2=0x558bc13fa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.947+0000 7f0bc8c82640 1 --2- 172.21.15.43:0/1328679077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc10154d0 0x558bc13fa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.947+0000 7f0bc8c82640 1 -- 172.21.15.43:0/1328679077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bc13ec990 con 0x558bc101d6b0 2024-04-25T10:38:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.948+0000 7f0bc8c82640 1 --2- 172.21.15.43:0/1328679077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc101d6b0 0x558bc13e1b20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40dd060 tx=0x7f0bb4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16584a1f2373a288 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.948+0000 7f0baeffd640 1 -- 172.21.15.43:0/1328679077 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb40a6fc0 con 0x558bc101d6b0 2024-04-25T10:38:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.948+0000 7f0baeffd640 1 -- 172.21.15.43:0/1328679077 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bb40a7bb0 con 0x558bc101d6b0 2024-04-25T10:38:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.948+0000 7f0baeffd640 1 -- 172.21.15.43:0/1328679077 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb40a3740 con 0x558bc101d6b0 2024-04-25T10:38:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.949+0000 7f0bd2597740 1 -- 172.21.15.43:0/1328679077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc101d6b0 msgr2=0x558bc13e1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.949+0000 7f0bd2597740 1 --2- 172.21.15.43:0/1328679077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc101d6b0 0x558bc13e1b20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40dd060 tx=0x7f0bb4000bc0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.949+0000 7f0bd2597740 1 -- 172.21.15.43:0/1328679077 shutdown_connections 2024-04-25T10:38:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.949+0000 7f0bd2597740 1 --2- 172.21.15.43:0/1328679077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bc103e390 0x558bc13eb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.949+0000 7f0bd2597740 1 --2- 172.21.15.43:0/1328679077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc101d6b0 0x558bc13e1b20 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.949+0000 7f0bd2597740 1 --2- 172.21.15.43:0/1328679077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc10154d0 0x558bc13fa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.949+0000 7f0bd2597740 1 -- 172.21.15.43:0/1328679077 >> 172.21.15.43:0/1328679077 conn(0x558bc10324e0 msgr2=0x558bc13e1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.949+0000 7f0bd2597740 1 -- 172.21.15.43:0/1328679077 shutdown_connections 2024-04-25T10:38:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.949+0000 7f0bd2597740 1 -- 172.21.15.43:0/1328679077 wait complete. 2024-04-25T10:38:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.949+0000 7f0bd2597740 1 -- start start 2024-04-25T10:38:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.949+0000 7f0bd2597740 1 Processor -- start 2024-04-25T10:38:34.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bd2597740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc10154d0 0x558bc1460e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bd2597740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc101d6b0 0x558bc1461370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bd2597740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bc103e390 0x558bc1400dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bd2597740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc41abd80 con 0x558bc10154d0 2024-04-25T10:38:34.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bd2597740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc41ad1f0 con 0x558bc101d6b0 2024-04-25T10:38:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bd2597740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc41ac450 con 0x558bc103e390 2024-04-25T10:38:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bd2597740 10 client.0 _ll_register_callbacks cb 0x7f0bc0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bc9483640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc10154d0 0x558bc1460e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bc9483640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc10154d0 0x558bc1460e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37798/0 (socket says 172.21.15.43:37798) 2024-04-25T10:38:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bc9483640 1 -- 172.21.15.43:0/3006909316 learned_addr learned my addr 172.21.15.43:0/3006909316 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bc9c84640 1 --2- 172.21.15.43:0/3006909316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bc103e390 0x558bc1400dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bc8c82640 1 --2- 172.21.15.43:0/3006909316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc101d6b0 0x558bc1461370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bc9483640 1 -- 172.21.15.43:0/3006909316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bc103e390 msgr2=0x558bc1400dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bc9483640 1 --2- 172.21.15.43:0/3006909316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bc103e390 0x558bc1400dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bc9483640 1 -- 172.21.15.43:0/3006909316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc101d6b0 msgr2=0x558bc1461370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bc9483640 1 --2- 172.21.15.43:0/3006909316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc101d6b0 0x558bc1461370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bc9483640 1 -- 172.21.15.43:0/3006909316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bc14013e0 con 0x558bc10154d0 2024-04-25T10:38:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bc9483640 1 --2- 172.21.15.43:0/3006909316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc10154d0 0x558bc1460e30 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009b80 tx=0x7f0bc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a894940042b1821c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bacff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bacff9640 1 -- 172.21.15.43:0/3006909316 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc002b5e0 con 0x558bc10154d0 2024-04-25T10:38:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.950+0000 7f0bacff9640 1 -- 172.21.15.43:0/3006909316 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f0bc002b780 con 0x558bc10154d0 2024-04-25T10:38:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.951+0000 7f0bd2597740 1 -- 172.21.15.43:0/3006909316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bb40a7bb0 con 0x558bc10154d0 2024-04-25T10:38:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.951+0000 7f0bacff9640 1 -- 172.21.15.43:0/3006909316 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0bc002b980 con 0x558bc10154d0 2024-04-25T10:38:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.951+0000 7f0bd2597740 10 client.17214 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.951+0000 7f0bd2597740 1 -- 172.21.15.43:0/3006909316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558bc1468530 con 0x558bc10154d0 2024-04-25T10:38:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.951+0000 7f0bacff9640 1 -- 172.21.15.43:0/3006909316 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bb40a7bb0 con 0x558bc10154d0 2024-04-25T10:38:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.951+0000 7f0bacff9640 1 -- 172.21.15.43:0/3006909316 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0bc00038a0 con 0x558bc10154d0 2024-04-25T10:38:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.951+0000 7f0bd2597740 10 client.17214 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.951+0000 7f0bd2597740 10 client.17214 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:38:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.951+0000 7f0bd2597740 20 client.17214 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.951+0000 7f0bd2597740 1 --2- 172.21.15.43:0/3006909316 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x558bc0fd23a0 0x558bc146ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.951+0000 7f0bd2597740 4 client.17214 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:38:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.951+0000 7f0bd2597740 1 -- 172.21.15.43:0/3006909316 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558bc134c800 con 0x558bc0fd23a0 2024-04-25T10:38:34.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.951+0000 7f0bc8c82640 1 --2- 172.21.15.43:0/3006909316 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x558bc0fd23a0 0x558bc146ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:34.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.952+0000 7f0bc8c82640 1 --2- 172.21.15.43:0/3006909316 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x558bc0fd23a0 0x558bc146ac50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=388d743fbb6117cf server_cookie=87475bce9e9d8b7f in_seq=0 out_seq=0 2024-04-25T10:38:34.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.952+0000 7f0bacff9640 10 client.17214 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:38:34.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.952+0000 7f0bacff9640 1 -- 172.21.15.43:0/3006909316 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558bc134c800 con 0x558bc0fd23a0 2024-04-25T10:38:34.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.952+0000 7f0bacff9640 10 client.17214 handle_command_reply: tid=0 2024-04-25T10:38:34.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.952+0000 7f0bd2597740 1 client.17214 shutdown 2024-04-25T10:38:34.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.952+0000 7f0bd2597740 1 -- 172.21.15.43:0/3006909316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc10154d0 msgr2=0x558bc1460e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.952+0000 7f0bd2597740 1 --2- 172.21.15.43:0/3006909316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc10154d0 0x558bc1460e30 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009b80 tx=0x7f0bc00046c0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.952+0000 7f0bd2597740 1 -- 172.21.15.43:0/3006909316 shutdown_connections 2024-04-25T10:38:34.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.952+0000 7f0bd2597740 1 --2- 172.21.15.43:0/3006909316 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x558bc0fd23a0 0x558bc146ac50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.952+0000 7f0bd2597740 1 --2- 172.21.15.43:0/3006909316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558bc101d6b0 0x558bc1461370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.952+0000 7f0bd2597740 1 --2- 172.21.15.43:0/3006909316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bc103e390 0x558bc1400dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.952+0000 7f0bd2597740 1 --2- 172.21.15.43:0/3006909316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bc10154d0 0x558bc1460e30 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.952+0000 7f0bd2597740 1 -- 172.21.15.43:0/3006909316 >> 172.21.15.43:0/3006909316 conn(0x558bc10324e0 msgr2=0x558bc13f95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:34.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.952+0000 7f0bacff9640 0 client.17214 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:38:34.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.952+0000 7f0bd2597740 1 -- 172.21.15.43:0/3006909316 shutdown_connections 2024-04-25T10:38:34.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.952+0000 7f0bd2597740 1 -- 172.21.15.43:0/3006909316 wait complete. 2024-04-25T10:38:34.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.953+0000 7f0bd2597740 20 client.17214 trim_cache size 0 max 16384 2024-04-25T10:38:34.961 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:38:34.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.953+0000 7f0bcb6e7640 1 -- 172.21.15.43:0/3678586597 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0bb00afc60 msgr2=0x7f0bb00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.953+0000 7f0bcb6e7640 1 --2- 172.21.15.43:0/3678586597 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0bb00afc60 0x7f0bb00b2170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x558bc13dd070 tx=0x7f0bc0005f90 comp rx=0 tx=0).stop 2024-04-25T10:38:34.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.953+0000 7f0bcb6e7640 1 -- 172.21.15.43:0/3678586597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc4109fb0 msgr2=0x7f0bc419a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:34.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.953+0000 7f0bcb6e7640 1 --2- 172.21.15.43:0/3678586597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc4109fb0 0x7f0bc419a8d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400e9f0 tx=0x7f0bb400eeb0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.953+0000 7f0bcb6e7640 1 -- 172.21.15.43:0/3678586597 shutdown_connections 2024-04-25T10:38:34.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.953+0000 7f0bcb6e7640 1 --2- 172.21.15.43:0/3678586597 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0bb00afc60 0x7f0bb00b2170 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.953+0000 7f0bcb6e7640 1 --2- 172.21.15.43:0/3678586597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bc419eb90 0x7f0bc419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.953+0000 7f0bcb6e7640 1 --2- 172.21.15.43:0/3678586597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc4109fb0 0x7f0bc419a8d0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.953+0000 7f0bcb6e7640 1 --2- 172.21.15.43:0/3678586597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bc4109b10 0x7f0bc419a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:34.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.953+0000 7f0bcb6e7640 1 -- 172.21.15.43:0/3678586597 >> 172.21.15.43:0/3678586597 conn(0x7f0bc4077880 msgr2=0x7f0bc410a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:34.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.953+0000 7f0bcb6e7640 1 -- 172.21.15.43:0/3678586597 shutdown_connections 2024-04-25T10:38:34.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:34.953+0000 7f0bcb6e7640 1 -- 172.21.15.43:0/3678586597 wait complete. 2024-04-25T10:38:34.969 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:38:35.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.080+0000 7f38b669a640 1 -- 172.21.15.43:0/315246362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38b010e040 msgr2=0x7f38b0110530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:35.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.080+0000 7f38b669a640 1 --2- 172.21.15.43:0/315246362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38b010e040 0x7f38b0110530 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009ca0 tx=0x7f38a802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.080+0000 7f38b669a640 1 -- 172.21.15.43:0/315246362 shutdown_connections 2024-04-25T10:38:35.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.080+0000 7f38b669a640 1 --2- 172.21.15.43:0/315246362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38b010b650 0x7f38b010da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.081+0000 7f38b669a640 1 --2- 172.21.15.43:0/315246362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38b010e040 0x7f38b0110530 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.081+0000 7f38b669a640 1 --2- 172.21.15.43:0/315246362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38b0104bf0 0x7f38b0105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.081+0000 7f38b669a640 1 -- 172.21.15.43:0/315246362 >> 172.21.15.43:0/315246362 conn(0x7f38b0078100 msgr2=0x7f38b0078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:35.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.081+0000 7f38b669a640 1 -- 172.21.15.43:0/315246362 shutdown_connections 2024-04-25T10:38:35.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.081+0000 7f38b669a640 1 -- 172.21.15.43:0/315246362 wait complete. 2024-04-25T10:38:35.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.081+0000 7f38b669a640 1 Processor -- start 2024-04-25T10:38:35.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.081+0000 7f38b669a640 1 -- start start 2024-04-25T10:38:35.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.082+0000 7f38b669a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38b0104bf0 0x7f38b019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:35.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.082+0000 7f38b669a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38b010b650 0x7f38b019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:35.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.082+0000 7f38b669a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38b010e040 0x7f38b019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:35.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.082+0000 7f38b669a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f38b0113ca0 con 0x7f38b010e040 2024-04-25T10:38:35.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.082+0000 7f38b669a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f38b0113b20 con 0x7f38b010b650 2024-04-25T10:38:35.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.082+0000 7f38b669a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f38b0113e20 con 0x7f38b0104bf0 2024-04-25T10:38:35.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.082+0000 7f38b4c37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38b010e040 0x7f38b019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:35.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.082+0000 7f38b4c37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38b010e040 0x7f38b019eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37814/0 (socket says 172.21.15.43:37814) 2024-04-25T10:38:35.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.082+0000 7f38b4c37640 1 -- 172.21.15.43:0/3046444865 learned_addr learned my addr 172.21.15.43:0/3046444865 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:35.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.082+0000 7f38affff640 1 --2- 172.21.15.43:0/3046444865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38b0104bf0 0x7f38b019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:35.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.082+0000 7f38af7fe640 1 --2- 172.21.15.43:0/3046444865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38b010b650 0x7f38b019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:35.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.082+0000 7f38b4c37640 1 -- 172.21.15.43:0/3046444865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38b0104bf0 msgr2=0x7f38b019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:35.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.082+0000 7f38b4c37640 1 --2- 172.21.15.43:0/3046444865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38b0104bf0 0x7f38b019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.082+0000 7f38b4c37640 1 -- 172.21.15.43:0/3046444865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38b010b650 msgr2=0x7f38b019ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:35.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.082+0000 7f38b4c37640 1 --2- 172.21.15.43:0/3046444865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38b010b650 0x7f38b019ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.082+0000 7f38b4c37640 1 -- 172.21.15.43:0/3046444865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b019f520 con 0x7f38b010e040 2024-04-25T10:38:35.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.082+0000 7f38b4c37640 1 --2- 172.21.15.43:0/3046444865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38b010e040 0x7f38b019eee0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f38a8005cc0 tx=0x7f38a802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:35.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.083+0000 7f38ad7fa640 1 -- 172.21.15.43:0/3046444865 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8003bf0 con 0x7f38b010e040 2024-04-25T10:38:35.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.083+0000 7f38b669a640 1 -- 172.21.15.43:0/3046444865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b019f7b0 con 0x7f38b010e040 2024-04-25T10:38:35.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.083+0000 7f38ad7fa640 1 -- 172.21.15.43:0/3046444865 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a8003d90 con 0x7f38b010e040 2024-04-25T10:38:35.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.083+0000 7f38ad7fa640 1 -- 172.21.15.43:0/3046444865 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a802fbc0 con 0x7f38b010e040 2024-04-25T10:38:35.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.083+0000 7f38b669a640 1 -- 172.21.15.43:0/3046444865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b01a7190 con 0x7f38b010e040 2024-04-25T10:38:35.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.086+0000 7f38bd620740 1 Processor -- start 2024-04-25T10:38:35.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.086+0000 7f38ad7fa640 1 -- 172.21.15.43:0/3046444865 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f38a802fd60 con 0x7f38b010e040 2024-04-25T10:38:35.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.086+0000 7f38bd620740 1 -- start start 2024-04-25T10:38:35.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.086+0000 7f38bd620740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf16052390 0x55cf163f3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:35.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.086+0000 7f38bd620740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf160294d0 0x55cf163f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:35.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.086+0000 7f38bd620740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 0x55cf164006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:35.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.086+0000 7f38bd620740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cf16358d80 con 0x55cf160316b0 2024-04-25T10:38:35.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.086+0000 7f38bd620740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cf16312a20 con 0x55cf160294d0 2024-04-25T10:38:35.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.086+0000 7f38bd620740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cf16354d40 con 0x55cf16052390 2024-04-25T10:38:35.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.087+0000 7f38af7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 0x55cf164006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:35.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.087+0000 7f38af7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 0x55cf164006a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37830/0 (socket says 172.21.15.43:37830) 2024-04-25T10:38:35.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.087+0000 7f38af7fe640 1 -- 172.21.15.43:0/2728927726 learned_addr learned my addr 172.21.15.43:0/2728927726 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:35.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.087+0000 7f38af7fe640 1 --2- 172.21.15.43:0/2728927726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf16052390 0x55cf163f3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:35.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.087+0000 7f38ad7fa640 1 --2- 172.21.15.43:0/3046444865 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f38980afc60 0x7f38980b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:35.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.087+0000 7f38affff640 1 --2- 172.21.15.43:0/2728927726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf160294d0 0x55cf163f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:35.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.087+0000 7f38b4c37640 1 --2- 172.21.15.43:0/3046444865 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f38980afc60 0x7f38980b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:35.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.087+0000 7f38ad7fa640 1 -- 172.21.15.43:0/3046444865 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f38a81005b0 con 0x7f38b010e040 2024-04-25T10:38:35.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.087+0000 7f38af7fe640 1 -- 172.21.15.43:0/2728927726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf16052390 msgr2=0x55cf163f3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:35.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.087+0000 7f38af7fe640 1 --2- 172.21.15.43:0/2728927726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf16052390 0x55cf163f3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.087+0000 7f38af7fe640 1 -- 172.21.15.43:0/2728927726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf160294d0 msgr2=0x55cf163f3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:35.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.087+0000 7f38af7fe640 1 --2- 172.21.15.43:0/2728927726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf160294d0 0x55cf163f3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.087+0000 7f38af7fe640 1 -- 172.21.15.43:0/2728927726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf1631dc30 con 0x55cf160316b0 2024-04-25T10:38:35.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.087+0000 7f38af7fe640 1 --2- 172.21.15.43:0/2728927726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 0x55cf164006a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f38a0012710 tx=0x7f38a000b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25dd56ae3556c26f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:35.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.087+0000 7f38967fc640 1 -- 172.21.15.43:0/2728927726 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0012c90 con 0x55cf160316b0 2024-04-25T10:38:35.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.087+0000 7f38b4c37640 1 --2- 172.21.15.43:0/3046444865 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f38980afc60 0x7f38980b2170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f38a80fdef0 tx=0x7f38a802a8c0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:35.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.087+0000 7f38967fc640 1 -- 172.21.15.43:0/2728927726 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a0012e30 con 0x55cf160316b0 2024-04-25T10:38:35.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.087+0000 7f38967fc640 1 -- 172.21.15.43:0/2728927726 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a001e960 con 0x55cf160316b0 2024-04-25T10:38:35.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.088+0000 7f38bd620740 1 -- 172.21.15.43:0/2728927726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 msgr2=0x55cf164006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:35.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.088+0000 7f38bd620740 1 --2- 172.21.15.43:0/2728927726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 0x55cf164006a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f38a0012710 tx=0x7f38a000b700 comp rx=0 tx=0).stop 2024-04-25T10:38:35.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.088+0000 7f38bd620740 1 -- 172.21.15.43:0/2728927726 shutdown_connections 2024-04-25T10:38:35.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.088+0000 7f38bd620740 1 --2- 172.21.15.43:0/2728927726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf160294d0 0x55cf163f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.088+0000 7f38bd620740 1 --2- 172.21.15.43:0/2728927726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 0x55cf164006a0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.088+0000 7f38bd620740 1 --2- 172.21.15.43:0/2728927726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf16052390 0x55cf163f3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.088+0000 7f38bd620740 1 -- 172.21.15.43:0/2728927726 >> 172.21.15.43:0/2728927726 conn(0x55cf160464e0 msgr2=0x55cf163effc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:35.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.088+0000 7f38bd620740 1 -- 172.21.15.43:0/2728927726 shutdown_connections 2024-04-25T10:38:35.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.088+0000 7f38bd620740 1 -- 172.21.15.43:0/2728927726 wait complete. 2024-04-25T10:38:35.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.089+0000 7f38bd620740 1 -- start start 2024-04-25T10:38:35.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.089+0000 7f38bd620740 1 Processor -- start 2024-04-25T10:38:35.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.089+0000 7f38bd620740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf160294d0 0x55cf163fe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:35.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.089+0000 7f38bd620740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 0x55cf163fe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:35.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.089+0000 7f38bd620740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf16052390 0x55cf1640a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:35.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.089+0000 7f38bd620740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cf16400d90 con 0x55cf160316b0 2024-04-25T10:38:35.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.089+0000 7f38bd620740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cf162dfdc0 con 0x55cf16052390 2024-04-25T10:38:35.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.089+0000 7f38bd620740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cf16400f10 con 0x55cf160294d0 2024-04-25T10:38:35.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.089+0000 7f38bd620740 10 client.0 _ll_register_callbacks cb 0x55cf162fb600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:35.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.089+0000 7f38affff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 0x55cf163fe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:35.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.089+0000 7f38affff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 0x55cf163fe7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37832/0 (socket says 172.21.15.43:37832) 2024-04-25T10:38:35.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.089+0000 7f38affff640 1 -- 172.21.15.43:0/2038224878 learned_addr learned my addr 172.21.15.43:0/2038224878 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:35.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.089+0000 7f38af7fe640 1 --2- 172.21.15.43:0/2038224878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf16052390 0x55cf1640a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:35.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.090+0000 7f38af7fe640 1 --2- 172.21.15.43:0/2038224878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf160294d0 0x55cf163fe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:35.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.090+0000 7f38affff640 1 -- 172.21.15.43:0/2038224878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf160294d0 msgr2=0x55cf163fe2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:35.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.090+0000 7f38affff640 1 --2- 172.21.15.43:0/2038224878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf160294d0 0x55cf163fe2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.090+0000 7f38affff640 1 -- 172.21.15.43:0/2038224878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf16052390 msgr2=0x55cf1640a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:35.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.090+0000 7f38affff640 1 --2- 172.21.15.43:0/2038224878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf16052390 0x55cf1640a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.090+0000 7f38affff640 1 -- 172.21.15.43:0/2038224878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf1606a3f0 con 0x55cf160316b0 2024-04-25T10:38:35.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.090+0000 7f38affff640 1 --2- 172.21.15.43:0/2038224878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 0x55cf163fe7f0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f38a400ba60 tx=0x7f38a400bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb703e3efe4ffed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:35.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.090+0000 7f388f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:35.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.090+0000 7f388f7fe640 1 -- 172.21.15.43:0/2038224878 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4005020 con 0x55cf160316b0 2024-04-25T10:38:35.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.090+0000 7f38bd620740 1 -- 172.21.15.43:0/2038224878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38a0012e30 con 0x55cf160316b0 2024-04-25T10:38:35.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.090+0000 7f388f7fe640 1 -- 172.21.15.43:0/2038224878 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f38a40051c0 con 0x55cf160316b0 2024-04-25T10:38:35.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.090+0000 7f388f7fe640 1 -- 172.21.15.43:0/2038224878 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f38a4005400 con 0x55cf160316b0 2024-04-25T10:38:35.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.090+0000 7f38bd620740 10 client.17226 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:35.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.090+0000 7f38bd620740 1 -- 172.21.15.43:0/2038224878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cf161347d0 con 0x55cf160316b0 2024-04-25T10:38:35.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.091+0000 7f388f7fe640 1 -- 172.21.15.43:0/2038224878 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a0012e30 con 0x55cf160316b0 2024-04-25T10:38:35.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.091+0000 7f388f7fe640 1 -- 172.21.15.43:0/2038224878 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f38a401b6a0 con 0x55cf160316b0 2024-04-25T10:38:35.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.091+0000 7f38bd620740 10 client.17226 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:35.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.091+0000 7f38bd620740 10 client.17226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:35.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.091+0000 7f38bd620740 20 client.17226 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:35.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.091+0000 7f38bd620740 1 --2- 172.21.15.43:0/2038224878 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55cf16010f60 0x55cf16410700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:35.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.091+0000 7f38bd620740 4 client.17226 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:38:35.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.091+0000 7f38bd620740 1 -- 172.21.15.43:0/2038224878 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf16366990 con 0x55cf16010f60 2024-04-25T10:38:35.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.091+0000 7f38af7fe640 1 --2- 172.21.15.43:0/2038224878 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55cf16010f60 0x55cf16410700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:35.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.091+0000 7f388f7fe640 10 client.17226 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:35.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.091+0000 7f38af7fe640 1 --2- 172.21.15.43:0/2038224878 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55cf16010f60 0x55cf16410700 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a772ca40a4b03118 server_cookie=dd167b18698091da in_seq=0 out_seq=0 2024-04-25T10:38:35.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.092+0000 7f388f7fe640 1 -- 172.21.15.43:0/2038224878 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cf16366990 con 0x55cf16010f60 2024-04-25T10:38:35.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.092+0000 7f388f7fe640 10 client.17226 handle_command_reply: tid=0 2024-04-25T10:38:35.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.092+0000 7f38bd620740 1 client.17226 shutdown 2024-04-25T10:38:35.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.093+0000 7f38bd620740 1 -- 172.21.15.43:0/2038224878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 msgr2=0x55cf163fe7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:35.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.093+0000 7f38bd620740 1 --2- 172.21.15.43:0/2038224878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 0x55cf163fe7f0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f38a400ba60 tx=0x7f38a400bf20 comp rx=0 tx=0).stop 2024-04-25T10:38:35.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.093+0000 7f38bd620740 1 -- 172.21.15.43:0/2038224878 shutdown_connections 2024-04-25T10:38:35.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.093+0000 7f38bd620740 1 --2- 172.21.15.43:0/2038224878 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55cf16010f60 0x55cf16410700 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.093+0000 7f38bd620740 1 --2- 172.21.15.43:0/2038224878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf16052390 0x55cf1640a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.093+0000 7f38bd620740 1 --2- 172.21.15.43:0/2038224878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 0x55cf163fe7f0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.093+0000 7f38bd620740 1 --2- 172.21.15.43:0/2038224878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf160294d0 0x55cf163fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.093+0000 7f388f7fe640 0 client.17226 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:35.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.093+0000 7f38bd620740 1 -- 172.21.15.43:0/2038224878 >> 172.21.15.43:0/2038224878 conn(0x55cf160464e0 msgr2=0x55cf163f5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:35.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.093+0000 7f38bd620740 1 -- 172.21.15.43:0/2038224878 shutdown_connections 2024-04-25T10:38:35.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.093+0000 7f38bd620740 1 -- 172.21.15.43:0/2038224878 wait complete. 2024-04-25T10:38:35.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.093+0000 7f38bd620740 20 client.17226 trim_cache size 0 max 16384 2024-04-25T10:38:35.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.107+0000 7f38bd620740 1 Processor -- start 2024-04-25T10:38:35.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.107+0000 7f38bd620740 1 -- start start 2024-04-25T10:38:35.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.107+0000 7f38bd620740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf16010f60 0x55cf163effb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:35.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.107+0000 7f38bd620740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160294d0 0x55cf16409e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:35.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.107+0000 7f38bd620740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf160316b0 0x55cf163f8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:35.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.107+0000 7f38bd620740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01b2190 con 0x55cf160294d0 2024-04-25T10:38:35.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.107+0000 7f38bd620740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01afa60 con 0x55cf16010f60 2024-04-25T10:38:35.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.107+0000 7f38bd620740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01ad8b0 con 0x55cf160316b0 2024-04-25T10:38:35.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.107+0000 7f38affff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160294d0 0x55cf16409e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:35.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.107+0000 7f38affff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160294d0 0x55cf16409e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37834/0 (socket says 172.21.15.43:37834) 2024-04-25T10:38:35.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.107+0000 7f38affff640 1 -- 172.21.15.43:0/2819737215 learned_addr learned my addr 172.21.15.43:0/2819737215 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:35.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.107+0000 7f38af7fe640 1 --2- 172.21.15.43:0/2819737215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf16010f60 0x55cf163effb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:35.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.107+0000 7f38affff640 1 -- 172.21.15.43:0/2819737215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf160316b0 msgr2=0x55cf163f8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:35.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.107+0000 7f38affff640 1 --2- 172.21.15.43:0/2819737215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf160316b0 0x55cf163f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.107+0000 7f38affff640 1 -- 172.21.15.43:0/2819737215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf16010f60 msgr2=0x55cf163effb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:35.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.107+0000 7f38affff640 1 --2- 172.21.15.43:0/2819737215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf16010f60 0x55cf163effb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.107+0000 7f38affff640 1 -- 172.21.15.43:0/2819737215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf163ef620 con 0x55cf160294d0 2024-04-25T10:38:35.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.107+0000 7f38affff640 1 --2- 172.21.15.43:0/2819737215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160294d0 0x55cf16409e70 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f38a400c010 tx=0x7f38a4003b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdf23cbddd47a75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:35.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.108+0000 7f3895ffb640 1 -- 172.21.15.43:0/2819737215 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a400b400 con 0x55cf160294d0 2024-04-25T10:38:35.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.108+0000 7f3895ffb640 1 -- 172.21.15.43:0/2819737215 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a401bce0 con 0x55cf160294d0 2024-04-25T10:38:35.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.108+0000 7f3895ffb640 1 -- 172.21.15.43:0/2819737215 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4024c40 con 0x55cf160294d0 2024-04-25T10:38:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.108+0000 7f38bd620740 1 -- 172.21.15.43:0/2819737215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160294d0 msgr2=0x55cf16409e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.108+0000 7f38bd620740 1 --2- 172.21.15.43:0/2819737215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160294d0 0x55cf16409e70 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f38a400c010 tx=0x7f38a4003b40 comp rx=0 tx=0).stop 2024-04-25T10:38:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.108+0000 7f38bd620740 1 -- 172.21.15.43:0/2819737215 shutdown_connections 2024-04-25T10:38:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.108+0000 7f38bd620740 1 --2- 172.21.15.43:0/2819737215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf160316b0 0x55cf163f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.108+0000 7f38bd620740 1 --2- 172.21.15.43:0/2819737215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160294d0 0x55cf16409e70 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.108+0000 7f38bd620740 1 --2- 172.21.15.43:0/2819737215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf16010f60 0x55cf163effb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.108+0000 7f38bd620740 1 -- 172.21.15.43:0/2819737215 >> 172.21.15.43:0/2819737215 conn(0x55cf160464e0 msgr2=0x55cf163ef8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.108+0000 7f38bd620740 1 -- 172.21.15.43:0/2819737215 shutdown_connections 2024-04-25T10:38:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.109+0000 7f38bd620740 1 -- 172.21.15.43:0/2819737215 wait complete. 2024-04-25T10:38:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.109+0000 7f38bd620740 1 -- start start 2024-04-25T10:38:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.109+0000 7f38bd620740 1 Processor -- start 2024-04-25T10:38:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.109+0000 7f38bd620740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf16010f60 0x55cf16470c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.109+0000 7f38bd620740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf160294d0 0x55cf164711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.109+0000 7f38bd620740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 0x55cf16405cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.109+0000 7f38bd620740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01adf20 con 0x55cf160316b0 2024-04-25T10:38:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.109+0000 7f38bd620740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01af390 con 0x55cf16010f60 2024-04-25T10:38:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.109+0000 7f38bd620740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01ae5f0 con 0x55cf160294d0 2024-04-25T10:38:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.109+0000 7f38bd620740 10 client.0 _ll_register_callbacks cb 0x7f38a40042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.109+0000 7f38af7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 0x55cf16405cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.109+0000 7f38af7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 0x55cf16405cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37836/0 (socket says 172.21.15.43:37836) 2024-04-25T10:38:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.109+0000 7f38af7fe640 1 -- 172.21.15.43:0/3341835507 learned_addr learned my addr 172.21.15.43:0/3341835507 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.110+0000 7f38af7fe640 1 --2- 172.21.15.43:0/3341835507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf16010f60 0x55cf16470c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:35.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.110+0000 7f38affff640 1 --2- 172.21.15.43:0/3341835507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf160294d0 0x55cf164711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:35.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.110+0000 7f38af7fe640 1 -- 172.21.15.43:0/3341835507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf160294d0 msgr2=0x55cf164711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:35.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.110+0000 7f38af7fe640 1 --2- 172.21.15.43:0/3341835507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf160294d0 0x55cf164711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.110+0000 7f38af7fe640 1 -- 172.21.15.43:0/3341835507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf16010f60 msgr2=0x55cf16470c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:35.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.110+0000 7f38af7fe640 1 --2- 172.21.15.43:0/3341835507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf16010f60 0x55cf16470c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.110+0000 7f38af7fe640 1 -- 172.21.15.43:0/3341835507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf164062b0 con 0x55cf160316b0 2024-04-25T10:38:35.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.110+0000 7f38af7fe640 1 --2- 172.21.15.43:0/3341835507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 0x55cf16405cf0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f38a000b620 tx=0x7f38a0017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72316e564dc1e5e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:35.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.110+0000 7f388f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:35.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.110+0000 7f388f7fe640 1 -- 172.21.15.43:0/3341835507 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0024040 con 0x55cf160316b0 2024-04-25T10:38:35.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.110+0000 7f388f7fe640 1 -- 172.21.15.43:0/3341835507 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f38a0005030 con 0x55cf160316b0 2024-04-25T10:38:35.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.110+0000 7f38bd620740 1 -- 172.21.15.43:0/3341835507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38a401bce0 con 0x55cf160316b0 2024-04-25T10:38:35.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.110+0000 7f388f7fe640 1 -- 172.21.15.43:0/3341835507 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f38a0005230 con 0x55cf160316b0 2024-04-25T10:38:35.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.110+0000 7f38bd620740 10 client.17232 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:35.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.110+0000 7f38bd620740 1 -- 172.21.15.43:0/3341835507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cf164783d0 con 0x55cf160316b0 2024-04-25T10:38:35.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.110+0000 7f388f7fe640 1 -- 172.21.15.43:0/3341835507 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a401bce0 con 0x55cf160316b0 2024-04-25T10:38:35.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.111+0000 7f388f7fe640 1 -- 172.21.15.43:0/3341835507 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f38a002a040 con 0x55cf160316b0 2024-04-25T10:38:35.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.111+0000 7f38bd620740 10 client.17232 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:35.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.111+0000 7f38bd620740 10 client.17232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:35.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.111+0000 7f38bd620740 20 client.17232 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:35.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.111+0000 7f38bd620740 1 --2- 172.21.15.43:0/3341835507 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55cf16052390 0x55cf1647aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:35.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.111+0000 7f38bd620740 4 client.17232 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:38:35.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.111+0000 7f38bd620740 1 -- 172.21.15.43:0/3341835507 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf16360800 con 0x55cf16052390 2024-04-25T10:38:35.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.111+0000 7f38affff640 1 --2- 172.21.15.43:0/3341835507 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55cf16052390 0x55cf1647aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:35.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.111+0000 7f388f7fe640 10 client.17232 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:35.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.111+0000 7f38affff640 1 --2- 172.21.15.43:0/3341835507 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55cf16052390 0x55cf1647aa00 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7c31743f34f4f57 server_cookie=bbeacb0477c4cabf in_seq=0 out_seq=0 2024-04-25T10:38:35.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.112+0000 7f388f7fe640 1 -- 172.21.15.43:0/3341835507 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf16360800 con 0x55cf16052390 2024-04-25T10:38:35.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.112+0000 7f388f7fe640 10 client.17232 handle_command_reply: tid=0 2024-04-25T10:38:35.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.112+0000 7f38bd620740 1 client.17232 shutdown 2024-04-25T10:38:35.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.112+0000 7f38bd620740 1 -- 172.21.15.43:0/3341835507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 msgr2=0x55cf16405cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:35.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.112+0000 7f38bd620740 1 --2- 172.21.15.43:0/3341835507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 0x55cf16405cf0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f38a000b620 tx=0x7f38a0017040 comp rx=0 tx=0).stop 2024-04-25T10:38:35.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.112+0000 7f38bd620740 1 -- 172.21.15.43:0/3341835507 shutdown_connections 2024-04-25T10:38:35.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.112+0000 7f38bd620740 1 --2- 172.21.15.43:0/3341835507 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55cf16052390 0x55cf1647aa00 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.112+0000 7f38bd620740 1 --2- 172.21.15.43:0/3341835507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf160316b0 0x55cf16405cf0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.112+0000 7f38bd620740 1 --2- 172.21.15.43:0/3341835507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf160294d0 0x55cf164711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.112+0000 7f388f7fe640 0 client.17232 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:35.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.112+0000 7f38bd620740 1 --2- 172.21.15.43:0/3341835507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf16010f60 0x55cf16470c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.112+0000 7f38bd620740 1 -- 172.21.15.43:0/3341835507 >> 172.21.15.43:0/3341835507 conn(0x55cf160464e0 msgr2=0x55cf163fc970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:35.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.112+0000 7f38bd620740 1 -- 172.21.15.43:0/3341835507 shutdown_connections 2024-04-25T10:38:35.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.112+0000 7f38bd620740 1 -- 172.21.15.43:0/3341835507 wait complete. 2024-04-25T10:38:35.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.112+0000 7f38bd620740 20 client.17232 trim_cache size 0 max 16384 2024-04-25T10:38:35.121 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:38:35.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.112+0000 7f38b669a640 1 -- 172.21.15.43:0/3046444865 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f38980afc60 msgr2=0x7f38980b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:35.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.112+0000 7f38b669a640 1 --2- 172.21.15.43:0/3046444865 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f38980afc60 0x7f38980b2170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f38a80fdef0 tx=0x7f38a802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.112+0000 7f38b669a640 1 -- 172.21.15.43:0/3046444865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38b010e040 msgr2=0x7f38b019eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:35.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.112+0000 7f38b669a640 1 --2- 172.21.15.43:0/3046444865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38b010e040 0x7f38b019eee0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f38a8005cc0 tx=0x7f38a802f380 comp rx=0 tx=0).stop 2024-04-25T10:38:35.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.113+0000 7f38b669a640 1 -- 172.21.15.43:0/3046444865 shutdown_connections 2024-04-25T10:38:35.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.113+0000 7f38b669a640 1 --2- 172.21.15.43:0/3046444865 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f38980afc60 0x7f38980b2170 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.113+0000 7f38b669a640 1 --2- 172.21.15.43:0/3046444865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38b010b650 0x7f38b019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.113+0000 7f38b669a640 1 --2- 172.21.15.43:0/3046444865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38b010e040 0x7f38b019eee0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.113+0000 7f38b669a640 1 --2- 172.21.15.43:0/3046444865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38b0104bf0 0x7f38b019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:35.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.113+0000 7f38b669a640 1 -- 172.21.15.43:0/3046444865 >> 172.21.15.43:0/3046444865 conn(0x7f38b0078100 msgr2=0x7f38b0101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:35.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.113+0000 7f38b669a640 1 -- 172.21.15.43:0/3046444865 shutdown_connections 2024-04-25T10:38:35.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:35.113+0000 7f38b669a640 1 -- 172.21.15.43:0/3046444865 wait complete. 2024-04-25T10:38:35.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:34 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3437565033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:38:35.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:34 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3437565033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:38:35.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:34 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3437565033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:38:36.127 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:38:36.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:38:36.127 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:38:36.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.255+0000 7fed4b59e640 1 -- 172.21.15.43:0/4006854169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed4c107050 msgr2=0x7fed4c107480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:36.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.255+0000 7fed4b59e640 1 --2- 172.21.15.43:0/4006854169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed4c107050 0x7fed4c107480 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fed40009900 tx=0x7fed4002a410 comp rx=0 tx=0).stop 2024-04-25T10:38:36.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.255+0000 7fed4b59e640 1 -- 172.21.15.43:0/4006854169 shutdown_connections 2024-04-25T10:38:36.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.255+0000 7fed4b59e640 1 --2- 172.21.15.43:0/4006854169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed4c107b10 0x7fed4c10f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.255+0000 7fed4b59e640 1 --2- 172.21.15.43:0/4006854169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed4c107050 0x7fed4c107480 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.255+0000 7fed4b59e640 1 --2- 172.21.15.43:0/4006854169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed4c102ac0 0x7fed4c102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.255+0000 7fed4b59e640 1 -- 172.21.15.43:0/4006854169 >> 172.21.15.43:0/4006854169 conn(0x7fed4c077fa0 msgr2=0x7fed4c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:36.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.255+0000 7fed4b59e640 1 -- 172.21.15.43:0/4006854169 shutdown_connections 2024-04-25T10:38:36.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.256+0000 7fed4b59e640 1 -- 172.21.15.43:0/4006854169 wait complete. 2024-04-25T10:38:36.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.256+0000 7fed4b59e640 1 Processor -- start 2024-04-25T10:38:36.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.256+0000 7fed4b59e640 1 -- start start 2024-04-25T10:38:36.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.257+0000 7fed4b59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed4c102ac0 0x7fed4c19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.257+0000 7fed4b59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed4c107050 0x7fed4c19b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.257+0000 7fed4b59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed4c107b10 0x7fed4c19be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.257+0000 7fed4b59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c112a20 con 0x7fed4c107b10 2024-04-25T10:38:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.257+0000 7fed4b59e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c1128a0 con 0x7fed4c102ac0 2024-04-25T10:38:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.257+0000 7fed4b59e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c112ba0 con 0x7fed4c107050 2024-04-25T10:38:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.257+0000 7fed4ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed4c107b10 0x7fed4c19be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.257+0000 7fed4ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed4c107b10 0x7fed4c19be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37846/0 (socket says 172.21.15.43:37846) 2024-04-25T10:38:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.257+0000 7fed4ad9d640 1 -- 172.21.15.43:0/2815167541 learned_addr learned my addr 172.21.15.43:0/2815167541 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.257+0000 7fed49d9b640 1 --2- 172.21.15.43:0/2815167541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed4c107050 0x7fed4c19b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.257+0000 7fed4a59c640 1 --2- 172.21.15.43:0/2815167541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed4c102ac0 0x7fed4c19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.257+0000 7fed4ad9d640 1 -- 172.21.15.43:0/2815167541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed4c107050 msgr2=0x7fed4c19b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.257+0000 7fed4ad9d640 1 --2- 172.21.15.43:0/2815167541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed4c107050 0x7fed4c19b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.257+0000 7fed4ad9d640 1 -- 172.21.15.43:0/2815167541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed4c102ac0 msgr2=0x7fed4c19b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.257+0000 7fed4ad9d640 1 --2- 172.21.15.43:0/2815167541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed4c102ac0 0x7fed4c19b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.257+0000 7fed4ad9d640 1 -- 172.21.15.43:0/2815167541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4c1a6650 con 0x7fed4c107b10 2024-04-25T10:38:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.257+0000 7fed4ad9d640 1 --2- 172.21.15.43:0/2815167541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed4c107b10 0x7fed4c19be40 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fed4400eaa0 tx=0x7fed4400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:36.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.258+0000 7fed3b7fe640 1 -- 172.21.15.43:0/2815167541 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44004d20 con 0x7fed4c107b10 2024-04-25T10:38:36.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.258+0000 7fed3b7fe640 1 -- 172.21.15.43:0/2815167541 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed44005020 con 0x7fed4c107b10 2024-04-25T10:38:36.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.258+0000 7fed3b7fe640 1 -- 172.21.15.43:0/2815167541 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed440116b0 con 0x7fed4c107b10 2024-04-25T10:38:36.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.258+0000 7fed4b59e640 1 -- 172.21.15.43:0/2815167541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4c1a6940 con 0x7fed4c107b10 2024-04-25T10:38:36.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.258+0000 7fed4b59e640 1 -- 172.21.15.43:0/2815167541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed4c1a6eb0 con 0x7fed4c107b10 2024-04-25T10:38:36.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.260+0000 7fed5798a740 1 Processor -- start 2024-04-25T10:38:36.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.261+0000 7fed3b7fe640 1 -- 172.21.15.43:0/2815167541 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fed44011850 con 0x7fed4c107b10 2024-04-25T10:38:36.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.261+0000 7fed5798a740 1 -- start start 2024-04-25T10:38:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.261+0000 7fed5798a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd94410 0x5622cd1347e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.261+0000 7fed5798a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ccd6b550 0x5622cd134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.261+0000 7fed5798a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd73730 0x5622cd135260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.261+0000 7fed5798a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5622cd09ab00 con 0x5622ccd94410 2024-04-25T10:38:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.261+0000 7fed5798a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5622cd036920 con 0x5622ccd6b550 2024-04-25T10:38:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.261+0000 7fed5798a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5622cd096ac0 con 0x5622ccd73730 2024-04-25T10:38:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.261+0000 7fed49d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd94410 0x5622cd1347e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.261+0000 7fed49d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd94410 0x5622cd1347e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37860/0 (socket says 172.21.15.43:37860) 2024-04-25T10:38:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.261+0000 7fed49d9b640 1 -- 172.21.15.43:0/2005682590 learned_addr learned my addr 172.21.15.43:0/2005682590 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.262+0000 7fed4a59c640 1 --2- 172.21.15.43:0/2005682590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ccd6b550 0x5622cd134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.262+0000 7fed49d9b640 1 --2- 172.21.15.43:0/2005682590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd73730 0x5622cd135260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.262+0000 7fed49d9b640 1 -- 172.21.15.43:0/2005682590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd73730 msgr2=0x5622cd135260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.262+0000 7fed49d9b640 1 --2- 172.21.15.43:0/2005682590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd73730 0x5622cd135260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.262+0000 7fed49d9b640 1 -- 172.21.15.43:0/2005682590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ccd6b550 msgr2=0x5622cd134d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.262+0000 7fed3b7fe640 1 --2- 172.21.15.43:0/2815167541 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fed340afcd0 0x7fed340b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.262+0000 7fed49d9b640 1 --2- 172.21.15.43:0/2005682590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ccd6b550 0x5622cd134d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.262+0000 7fed49d9b640 1 -- 172.21.15.43:0/2005682590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622cce747e0 con 0x5622ccd94410 2024-04-25T10:38:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.262+0000 7fed4ad9d640 1 --2- 172.21.15.43:0/2815167541 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fed340afcd0 0x7fed340b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.262+0000 7fed49d9b640 1 --2- 172.21.15.43:0/2005682590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd94410 0x5622cd1347e0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fed40009900 tx=0x7fed40005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b3bed37e9cb4cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.262+0000 7fed3b7fe640 1 -- 172.21.15.43:0/2815167541 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fed440df090 con 0x7fed4c107b10 2024-04-25T10:38:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.262+0000 7fed38ff9640 1 -- 172.21.15.43:0/2005682590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed40044020 con 0x5622ccd94410 2024-04-25T10:38:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.262+0000 7fed38ff9640 1 -- 172.21.15.43:0/2005682590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed40031870 con 0x5622ccd94410 2024-04-25T10:38:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.262+0000 7fed38ff9640 1 -- 172.21.15.43:0/2005682590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed40035040 con 0x5622ccd94410 2024-04-25T10:38:36.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.263+0000 7fed5798a740 1 -- 172.21.15.43:0/2005682590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd94410 msgr2=0x5622cd1347e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.263+0000 7fed4ad9d640 1 --2- 172.21.15.43:0/2815167541 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fed340afcd0 0x7fed340b21e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fed440a32e0 tx=0x7fed440d59b0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.263+0000 7fed5798a740 1 --2- 172.21.15.43:0/2005682590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd94410 0x5622cd1347e0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fed40009900 tx=0x7fed40005f30 comp rx=0 tx=0).stop 2024-04-25T10:38:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.263+0000 7fed5798a740 1 -- 172.21.15.43:0/2005682590 shutdown_connections 2024-04-25T10:38:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.263+0000 7fed5798a740 1 --2- 172.21.15.43:0/2005682590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ccd6b550 0x5622cd134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.263+0000 7fed5798a740 1 --2- 172.21.15.43:0/2005682590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd73730 0x5622cd135260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.263+0000 7fed5798a740 1 --2- 172.21.15.43:0/2005682590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd94410 0x5622cd1347e0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.263+0000 7fed5798a740 1 -- 172.21.15.43:0/2005682590 >> 172.21.15.43:0/2005682590 conn(0x5622ccd88560 msgr2=0x5622cd130f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.263+0000 7fed5798a740 1 -- 172.21.15.43:0/2005682590 shutdown_connections 2024-04-25T10:38:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.263+0000 7fed5798a740 1 -- 172.21.15.43:0/2005682590 wait complete. 2024-04-25T10:38:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.264+0000 7fed5798a740 1 -- start start 2024-04-25T10:38:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.264+0000 7fed5798a740 1 Processor -- start 2024-04-25T10:38:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.264+0000 7fed5798a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd6b550 0x5622cd13ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.264+0000 7fed5798a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd73730 0x5622cd13f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.264+0000 7fed5798a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ccd94410 0x5622cd144fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.264+0000 7fed5798a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5622cd141ad0 con 0x5622ccd73730 2024-04-25T10:38:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.264+0000 7fed5798a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5622cd0345d0 con 0x5622ccd94410 2024-04-25T10:38:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.264+0000 7fed5798a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5622cd141c50 con 0x5622ccd6b550 2024-04-25T10:38:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.264+0000 7fed5798a740 10 client.0 _ll_register_callbacks cb 0x5622cd022080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.264+0000 7fed4a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd73730 0x5622cd13f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.264+0000 7fed4a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd73730 0x5622cd13f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37862/0 (socket says 172.21.15.43:37862) 2024-04-25T10:38:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.264+0000 7fed4a59c640 1 -- 172.21.15.43:0/127655498 learned_addr learned my addr 172.21.15.43:0/127655498 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.265+0000 7fed49d9b640 1 --2- 172.21.15.43:0/127655498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd6b550 0x5622cd13ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.265+0000 7fed49d9b640 1 --2- 172.21.15.43:0/127655498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ccd94410 0x5622cd144fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.265+0000 7fed4a59c640 1 -- 172.21.15.43:0/127655498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd6b550 msgr2=0x5622cd13ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.265+0000 7fed4a59c640 1 --2- 172.21.15.43:0/127655498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd6b550 0x5622cd13ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.265+0000 7fed4a59c640 1 -- 172.21.15.43:0/127655498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ccd94410 msgr2=0x5622cd144fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.265+0000 7fed4a59c640 1 --2- 172.21.15.43:0/127655498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ccd94410 0x5622cd144fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.265+0000 7fed4a59c640 1 -- 172.21.15.43:0/127655498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622cced4f70 con 0x5622ccd73730 2024-04-25T10:38:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.265+0000 7fed49d9b640 1 --2- 172.21.15.43:0/127655498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd6b550 0x5622cd13ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:38:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.265+0000 7fed4a59c640 1 --2- 172.21.15.43:0/127655498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd73730 0x5622cd13f380 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fed3c010a50 tx=0x7fed3c010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ea96f49f101191 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.265+0000 7fed31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.265+0000 7fed31ffb640 1 -- 172.21.15.43:0/127655498 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c00fb10 con 0x5622ccd73730 2024-04-25T10:38:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.265+0000 7fed5798a740 1 -- 172.21.15.43:0/127655498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed40031870 con 0x5622ccd73730 2024-04-25T10:38:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.265+0000 7fed31ffb640 1 -- 172.21.15.43:0/127655498 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fed3c00fcb0 con 0x5622ccd73730 2024-04-25T10:38:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.265+0000 7fed31ffb640 1 -- 172.21.15.43:0/127655498 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed3c005020 con 0x5622ccd73730 2024-04-25T10:38:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.265+0000 7fed5798a740 10 client.17244 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.265+0000 7fed5798a740 1 -- 172.21.15.43:0/127655498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5622ccd6c170 con 0x5622ccd73730 2024-04-25T10:38:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.266+0000 7fed31ffb640 1 -- 172.21.15.43:0/127655498 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed40031870 con 0x5622ccd73730 2024-04-25T10:38:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.266+0000 7fed31ffb640 1 -- 172.21.15.43:0/127655498 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fed3c022430 con 0x5622ccd73730 2024-04-25T10:38:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.266+0000 7fed5798a740 10 client.17244 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.266+0000 7fed5798a740 10 client.17244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.266+0000 7fed5798a740 20 client.17244 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.266+0000 7fed5798a740 1 --2- 172.21.15.43:0/127655498 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5622ccd531f0 0x5622cd1512e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.266+0000 7fed5798a740 4 client.17244 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:38:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.266+0000 7fed5798a740 1 -- 172.21.15.43:0/127655498 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622cd0a8710 con 0x5622ccd531f0 2024-04-25T10:38:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.266+0000 7fed49d9b640 1 --2- 172.21.15.43:0/127655498 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5622ccd531f0 0x5622cd1512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.267+0000 7fed31ffb640 10 client.17244 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.267+0000 7fed49d9b640 1 --2- 172.21.15.43:0/127655498 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5622ccd531f0 0x5622cd1512e0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5621d73eb6eed429 server_cookie=871c66f416e6f4aa in_seq=0 out_seq=0 2024-04-25T10:38:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.267+0000 7fed31ffb640 1 -- 172.21.15.43:0/127655498 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5622cd0a8710 con 0x5622ccd531f0 2024-04-25T10:38:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.267+0000 7fed31ffb640 10 client.17244 handle_command_reply: tid=0 2024-04-25T10:38:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.267+0000 7fed5798a740 1 client.17244 shutdown 2024-04-25T10:38:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.267+0000 7fed5798a740 1 -- 172.21.15.43:0/127655498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd73730 msgr2=0x5622cd13f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.267+0000 7fed5798a740 1 --2- 172.21.15.43:0/127655498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd73730 0x5622cd13f380 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fed3c010a50 tx=0x7fed3c010f10 comp rx=0 tx=0).stop 2024-04-25T10:38:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.268+0000 7fed5798a740 1 -- 172.21.15.43:0/127655498 shutdown_connections 2024-04-25T10:38:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.268+0000 7fed5798a740 1 --2- 172.21.15.43:0/127655498 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5622ccd531f0 0x5622cd1512e0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.268+0000 7fed5798a740 1 --2- 172.21.15.43:0/127655498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ccd94410 0x5622cd144fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.268+0000 7fed5798a740 1 --2- 172.21.15.43:0/127655498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd73730 0x5622cd13f380 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.268+0000 7fed5798a740 1 --2- 172.21.15.43:0/127655498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd6b550 0x5622cd13ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.268+0000 7fed5798a740 1 -- 172.21.15.43:0/127655498 >> 172.21.15.43:0/127655498 conn(0x5622ccd88560 msgr2=0x5622cd1345b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:36.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.268+0000 7fed31ffb640 0 client.17244 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:36.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.268+0000 7fed5798a740 1 -- 172.21.15.43:0/127655498 shutdown_connections 2024-04-25T10:38:36.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.268+0000 7fed5798a740 1 -- 172.21.15.43:0/127655498 wait complete. 2024-04-25T10:38:36.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.268+0000 7fed5798a740 20 client.17244 trim_cache size 0 max 16384 2024-04-25T10:38:36.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.287+0000 7fed5798a740 1 Processor -- start 2024-04-25T10:38:36.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.287+0000 7fed5798a740 1 -- start start 2024-04-25T10:38:36.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.287+0000 7fed5798a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd6b550 0x5622cd130fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.287+0000 7fed5798a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd73730 0x5622cd1389a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.287+0000 7fed5798a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ccd94410 0x5622cd144f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.287+0000 7fed5798a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c1b1f70 con 0x5622ccd6b550 2024-04-25T10:38:36.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.287+0000 7fed5798a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c1af840 con 0x5622ccd94410 2024-04-25T10:38:36.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.287+0000 7fed5798a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c1ad690 con 0x5622ccd73730 2024-04-25T10:38:36.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.287+0000 7fed49d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd6b550 0x5622cd130fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.287+0000 7fed49d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd6b550 0x5622cd130fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37876/0 (socket says 172.21.15.43:37876) 2024-04-25T10:38:36.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.287+0000 7fed49d9b640 1 -- 172.21.15.43:0/1875586954 learned_addr learned my addr 172.21.15.43:0/1875586954 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:36.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.287+0000 7fed4a59c640 1 --2- 172.21.15.43:0/1875586954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd73730 0x5622cd1389a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.288+0000 7fed49d9b640 1 -- 172.21.15.43:0/1875586954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd73730 msgr2=0x5622cd1389a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:36.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.288+0000 7fed49d9b640 1 --2- 172.21.15.43:0/1875586954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd73730 0x5622cd1389a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.288+0000 7fed49d9b640 1 -- 172.21.15.43:0/1875586954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ccd94410 msgr2=0x5622cd144f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:36.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.288+0000 7fed49d9b640 1 --2- 172.21.15.43:0/1875586954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ccd94410 0x5622cd144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.288+0000 7fed49d9b640 1 -- 172.21.15.43:0/1875586954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622cd13f2a0 con 0x5622ccd6b550 2024-04-25T10:38:36.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.288+0000 7fed49d9b640 1 --2- 172.21.15.43:0/1875586954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd6b550 0x5622cd130fc0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x5622cd131950 tx=0x7fed40005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561f88aa080dde14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:36.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.288+0000 7fed33fff640 1 -- 172.21.15.43:0/1875586954 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed40052070 con 0x5622ccd6b550 2024-04-25T10:38:36.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.288+0000 7fed33fff640 1 -- 172.21.15.43:0/1875586954 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed400317f0 con 0x5622ccd6b550 2024-04-25T10:38:36.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.288+0000 7fed33fff640 1 -- 172.21.15.43:0/1875586954 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed40044040 con 0x5622ccd6b550 2024-04-25T10:38:36.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:35 smithi164 ceph-mon[32713]: pgmap v185: 97 pgs: 97 active+clean; 4.8 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 17 op/s 2024-04-25T10:38:36.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:35 smithi077 ceph-mon[32772]: pgmap v185: 97 pgs: 97 active+clean; 4.8 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 17 op/s 2024-04-25T10:38:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.289+0000 7fed5798a740 1 -- 172.21.15.43:0/1875586954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd6b550 msgr2=0x5622cd130fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.289+0000 7fed5798a740 1 --2- 172.21.15.43:0/1875586954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd6b550 0x5622cd130fc0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x5622cd131950 tx=0x7fed40005410 comp rx=0 tx=0).stop 2024-04-25T10:38:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.289+0000 7fed5798a740 1 -- 172.21.15.43:0/1875586954 shutdown_connections 2024-04-25T10:38:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.289+0000 7fed5798a740 1 --2- 172.21.15.43:0/1875586954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ccd94410 0x5622cd144f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.289+0000 7fed5798a740 1 --2- 172.21.15.43:0/1875586954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd73730 0x5622cd1389a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.289+0000 7fed5798a740 1 --2- 172.21.15.43:0/1875586954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd6b550 0x5622cd130fc0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.289+0000 7fed5798a740 1 -- 172.21.15.43:0/1875586954 >> 172.21.15.43:0/1875586954 conn(0x5622ccd88560 msgr2=0x5622cd138290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.289+0000 7fed5798a740 1 -- 172.21.15.43:0/1875586954 shutdown_connections 2024-04-25T10:38:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.289+0000 7fed5798a740 1 -- 172.21.15.43:0/1875586954 wait complete. 2024-04-25T10:38:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.289+0000 7fed5798a740 1 -- start start 2024-04-25T10:38:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.289+0000 7fed5798a740 1 Processor -- start 2024-04-25T10:38:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.289+0000 7fed5798a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ccd6b550 0x5622cd14b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.289+0000 7fed5798a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd73730 0x5622cd14bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed5798a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd94410 0x5622cd146a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed5798a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c1add00 con 0x5622ccd73730 2024-04-25T10:38:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed5798a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c1af170 con 0x5622ccd6b550 2024-04-25T10:38:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed5798a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c1ae3d0 con 0x5622ccd94410 2024-04-25T10:38:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed5798a740 10 client.0 _ll_register_callbacks cb 0x7fed3c00ad00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed4a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd73730 0x5622cd14bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed4a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd73730 0x5622cd14bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37880/0 (socket says 172.21.15.43:37880) 2024-04-25T10:38:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed49d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ccd6b550 0x5622cd14b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed4a59c640 1 -- 172.21.15.43:0/270406306 learned_addr learned my addr 172.21.15.43:0/270406306 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed49d9b640 1 --2- 172.21.15.43:0/270406306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd94410 0x5622cd146a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed4a59c640 1 -- 172.21.15.43:0/270406306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd94410 msgr2=0x5622cd146a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed4a59c640 1 --2- 172.21.15.43:0/270406306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd94410 0x5622cd146a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed4a59c640 1 -- 172.21.15.43:0/270406306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ccd6b550 msgr2=0x5622cd14b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed4a59c640 1 --2- 172.21.15.43:0/270406306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ccd6b550 0x5622cd14b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed4a59c640 1 -- 172.21.15.43:0/270406306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622cd1471f0 con 0x5622ccd73730 2024-04-25T10:38:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed49d9b640 1 --2- 172.21.15.43:0/270406306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd94410 0x5622cd146a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:38:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed4a59c640 1 --2- 172.21.15.43:0/270406306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd73730 0x5622cd14bbe0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fed3c000c00 tx=0x7fed3c022730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31109df5afcabea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed31ffb640 1 -- 172.21.15.43:0/270406306 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c031060 con 0x5622ccd73730 2024-04-25T10:38:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed31ffb640 1 -- 172.21.15.43:0/270406306 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fed3c00fb10 con 0x5622ccd73730 2024-04-25T10:38:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed5798a740 1 -- 172.21.15.43:0/270406306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed400317f0 con 0x5622ccd73730 2024-04-25T10:38:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.290+0000 7fed31ffb640 1 -- 172.21.15.43:0/270406306 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed3c00fd10 con 0x5622ccd73730 2024-04-25T10:38:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.291+0000 7fed5798a740 10 client.17250 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.291+0000 7fed5798a740 1 -- 172.21.15.43:0/270406306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5622cd1b90f0 con 0x5622ccd73730 2024-04-25T10:38:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.291+0000 7fed31ffb640 1 -- 172.21.15.43:0/270406306 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed400317f0 con 0x5622ccd73730 2024-04-25T10:38:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.291+0000 7fed31ffb640 1 -- 172.21.15.43:0/270406306 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fed3c004bc0 con 0x5622ccd73730 2024-04-25T10:38:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.291+0000 7fed5798a740 10 client.17250 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.291+0000 7fed5798a740 10 client.17250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.291+0000 7fed5798a740 20 client.17250 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.291+0000 7fed5798a740 1 --2- 172.21.15.43:0/270406306 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5622ccd28630 0x5622cd1bb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.291+0000 7fed5798a740 4 client.17250 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:38:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.291+0000 7fed5798a740 1 -- 172.21.15.43:0/270406306 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5622cd0a2580 con 0x5622ccd28630 2024-04-25T10:38:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.291+0000 7fed49d9b640 1 --2- 172.21.15.43:0/270406306 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5622ccd28630 0x5622cd1bb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.292+0000 7fed31ffb640 10 client.17250 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.292+0000 7fed49d9b640 1 --2- 172.21.15.43:0/270406306 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5622ccd28630 0x5622cd1bb8e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79e8c0f0d804ddf5 server_cookie=a562477118fa307f in_seq=0 out_seq=0 2024-04-25T10:38:36.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.293+0000 7fed31ffb640 1 -- 172.21.15.43:0/270406306 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5622cd0a2580 con 0x5622ccd28630 2024-04-25T10:38:36.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.293+0000 7fed31ffb640 10 client.17250 handle_command_reply: tid=0 2024-04-25T10:38:36.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.293+0000 7fed5798a740 1 client.17250 shutdown 2024-04-25T10:38:36.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.293+0000 7fed5798a740 1 -- 172.21.15.43:0/270406306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd73730 msgr2=0x5622cd14bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:36.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.293+0000 7fed5798a740 1 --2- 172.21.15.43:0/270406306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd73730 0x5622cd14bbe0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fed3c000c00 tx=0x7fed3c022730 comp rx=0 tx=0).stop 2024-04-25T10:38:36.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.293+0000 7fed5798a740 1 -- 172.21.15.43:0/270406306 shutdown_connections 2024-04-25T10:38:36.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.293+0000 7fed5798a740 1 --2- 172.21.15.43:0/270406306 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5622ccd28630 0x5622cd1bb8e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.293+0000 7fed5798a740 1 --2- 172.21.15.43:0/270406306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622ccd94410 0x5622cd146a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.293+0000 7fed5798a740 1 --2- 172.21.15.43:0/270406306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622ccd73730 0x5622cd14bbe0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.293+0000 7fed5798a740 1 --2- 172.21.15.43:0/270406306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ccd6b550 0x5622cd14b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.293+0000 7fed5798a740 1 -- 172.21.15.43:0/270406306 >> 172.21.15.43:0/270406306 conn(0x5622ccd88560 msgr2=0x5622cd13d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:36.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.293+0000 7fed31ffb640 0 client.17250 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:36.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.293+0000 7fed5798a740 1 -- 172.21.15.43:0/270406306 shutdown_connections 2024-04-25T10:38:36.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.293+0000 7fed5798a740 1 -- 172.21.15.43:0/270406306 wait complete. 2024-04-25T10:38:36.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.293+0000 7fed5798a740 20 client.17250 trim_cache size 0 max 16384 2024-04-25T10:38:36.301 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:38:36.301 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:38:36.301 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "fd5ba958-425c-4bfe-95a0-39445510ab58", 2024-04-25T10:38:36.301 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:38:36.301 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:38:36.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.294+0000 7fed4b59e640 1 -- 172.21.15.43:0/2815167541 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fed340afcd0 msgr2=0x7fed340b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:36.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.294+0000 7fed4b59e640 1 --2- 172.21.15.43:0/2815167541 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fed340afcd0 0x7fed340b21e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fed440a32e0 tx=0x7fed440d59b0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.294+0000 7fed4b59e640 1 -- 172.21.15.43:0/2815167541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed4c107b10 msgr2=0x7fed4c19be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:36.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.294+0000 7fed4b59e640 1 --2- 172.21.15.43:0/2815167541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed4c107b10 0x7fed4c19be40 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fed4400eaa0 tx=0x7fed4400eea0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.294+0000 7fed4b59e640 1 -- 172.21.15.43:0/2815167541 shutdown_connections 2024-04-25T10:38:36.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.294+0000 7fed4b59e640 1 --2- 172.21.15.43:0/2815167541 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fed340afcd0 0x7fed340b21e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.294+0000 7fed4b59e640 1 --2- 172.21.15.43:0/2815167541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed4c107b10 0x7fed4c19be40 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.294+0000 7fed4b59e640 1 --2- 172.21.15.43:0/2815167541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed4c107050 0x7fed4c19b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.294+0000 7fed4b59e640 1 --2- 172.21.15.43:0/2815167541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed4c102ac0 0x7fed4c19b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.294+0000 7fed4b59e640 1 -- 172.21.15.43:0/2815167541 >> 172.21.15.43:0/2815167541 conn(0x7fed4c077fa0 msgr2=0x7fed4c100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:36.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.294+0000 7fed4b59e640 1 -- 172.21.15.43:0/2815167541 shutdown_connections 2024-04-25T10:38:36.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.294+0000 7fed4b59e640 1 -- 172.21.15.43:0/2815167541 wait complete. 2024-04-25T10:38:36.306 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:35 smithi043 ceph-mon[29478]: pgmap v185: 97 pgs: 97 active+clean; 4.8 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 17 op/s 2024-04-25T10:38:36.307 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:38:36.307 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:38:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.419+0000 7fa0b20f9640 1 -- 172.21.15.43:0/3166039584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0ac10d380 msgr2=0x7fa0ac10f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:36.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.419+0000 7fa0b20f9640 1 --2- 172.21.15.43:0/3166039584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0ac10d380 0x7fa0ac10f7a0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009ca0 tx=0x7fa0a402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.419+0000 7fa0b20f9640 1 -- 172.21.15.43:0/3166039584 shutdown_connections 2024-04-25T10:38:36.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.419+0000 7fa0b20f9640 1 --2- 172.21.15.43:0/3166039584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0ac10d380 0x7fa0ac10f7a0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.419+0000 7fa0b20f9640 1 --2- 172.21.15.43:0/3166039584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0ac102af0 0x7fa0ac102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.419+0000 7fa0b20f9640 1 --2- 172.21.15.43:0/3166039584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0ac108a20 0x7fa0ac1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.419+0000 7fa0b20f9640 1 -- 172.21.15.43:0/3166039584 >> 172.21.15.43:0/3166039584 conn(0x7fa0ac078100 msgr2=0x7fa0ac078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:36.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.419+0000 7fa0b20f9640 1 -- 172.21.15.43:0/3166039584 shutdown_connections 2024-04-25T10:38:36.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.419+0000 7fa0b20f9640 1 -- 172.21.15.43:0/3166039584 wait complete. 2024-04-25T10:38:36.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.420+0000 7fa0b20f9640 1 Processor -- start 2024-04-25T10:38:36.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.420+0000 7fa0b20f9640 1 -- start start 2024-04-25T10:38:36.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.420+0000 7fa0b20f9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0ac102af0 0x7fa0ac19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.420+0000 7fa0b20f9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0ac108a20 0x7fa0ac19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.420+0000 7fa0b20f9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0ac10d380 0x7fa0ac19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.420+0000 7fa0b20f9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac112be0 con 0x7fa0ac10d380 2024-04-25T10:38:36.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.420+0000 7fa0b20f9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac112a60 con 0x7fa0ac102af0 2024-04-25T10:38:36.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.420+0000 7fa0b20f9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac112d60 con 0x7fa0ac108a20 2024-04-25T10:38:36.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.420+0000 7fa0abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0ac10d380 0x7fa0ac19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.420+0000 7fa0abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0ac10d380 0x7fa0ac19ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37890/0 (socket says 172.21.15.43:37890) 2024-04-25T10:38:36.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.420+0000 7fa0abfff640 1 -- 172.21.15.43:0/3567639741 learned_addr learned my addr 172.21.15.43:0/3567639741 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:36.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.420+0000 7fa0ab7fe640 1 --2- 172.21.15.43:0/3567639741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0ac102af0 0x7fa0ac19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.420+0000 7fa0aaffd640 1 --2- 172.21.15.43:0/3567639741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0ac108a20 0x7fa0ac19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.421+0000 7fa0abfff640 1 -- 172.21.15.43:0/3567639741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0ac108a20 msgr2=0x7fa0ac19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:36.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.421+0000 7fa0abfff640 1 --2- 172.21.15.43:0/3567639741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0ac108a20 0x7fa0ac19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.421+0000 7fa0abfff640 1 -- 172.21.15.43:0/3567639741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0ac102af0 msgr2=0x7fa0ac19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:36.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.421+0000 7fa0abfff640 1 --2- 172.21.15.43:0/3567639741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0ac102af0 0x7fa0ac19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.421+0000 7fa0abfff640 1 -- 172.21.15.43:0/3567639741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ac19f450 con 0x7fa0ac10d380 2024-04-25T10:38:36.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.421+0000 7fa0abfff640 1 --2- 172.21.15.43:0/3567639741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0ac10d380 0x7fa0ac19ee90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4006010 tx=0x7fa0a402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:36.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.421+0000 7fa0a8ff9640 1 -- 172.21.15.43:0/3567639741 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4003bf0 con 0x7fa0ac10d380 2024-04-25T10:38:36.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.421+0000 7fa0b20f9640 1 -- 172.21.15.43:0/3567639741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ac19f6e0 con 0x7fa0ac10d380 2024-04-25T10:38:36.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.421+0000 7fa0a8ff9640 1 -- 172.21.15.43:0/3567639741 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0a4003d90 con 0x7fa0ac10d380 2024-04-25T10:38:36.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.421+0000 7fa0a8ff9640 1 -- 172.21.15.43:0/3567639741 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a402fba0 con 0x7fa0ac10d380 2024-04-25T10:38:36.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.421+0000 7fa0b20f9640 1 -- 172.21.15.43:0/3567639741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ac19fc30 con 0x7fa0ac10d380 2024-04-25T10:38:36.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.423+0000 7fa0b8faa740 1 Processor -- start 2024-04-25T10:38:36.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.424+0000 7fa0a8ff9640 1 -- 172.21.15.43:0/3567639741 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fa0a402fd40 con 0x7fa0ac10d380 2024-04-25T10:38:36.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.424+0000 7fa0b8faa740 1 -- start start 2024-04-25T10:38:36.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.424+0000 7fa0b8faa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55587a8ec390 0x55587ac8ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.424+0000 7fa0b8faa740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55587a8c34d0 0x55587ac8cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.424+0000 7fa0b8faa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8cb6b0 0x55587ac998f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.424+0000 7fa0b8faa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55587abf2d80 con 0x55587a8cb6b0 2024-04-25T10:38:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.424+0000 7fa0b8faa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55587abac8b0 con 0x55587a8ec390 2024-04-25T10:38:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.424+0000 7fa0b8faa740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55587abeed40 con 0x55587a8c34d0 2024-04-25T10:38:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.424+0000 7fa0aaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8cb6b0 0x55587ac998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.424+0000 7fa0aaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8cb6b0 0x55587ac998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37902/0 (socket says 172.21.15.43:37902) 2024-04-25T10:38:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.424+0000 7fa0aaffd640 1 -- 172.21.15.43:0/1352835036 learned_addr learned my addr 172.21.15.43:0/1352835036 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.424+0000 7fa0a8ff9640 1 --2- 172.21.15.43:0/3567639741 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa0940afc60 0x7fa0940b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.424+0000 7fa0aaffd640 1 --2- 172.21.15.43:0/1352835036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55587a8ec390 0x55587ac8ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.424+0000 7fa0ab7fe640 1 --2- 172.21.15.43:0/1352835036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55587a8c34d0 0x55587ac8cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.424+0000 7fa0aaffd640 1 -- 172.21.15.43:0/1352835036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55587a8c34d0 msgr2=0x55587ac8cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.424+0000 7fa0aaffd640 1 --2- 172.21.15.43:0/1352835036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55587a8c34d0 0x55587ac8cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.424+0000 7fa0aaffd640 1 -- 172.21.15.43:0/1352835036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55587a8ec390 msgr2=0x55587ac8ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.424+0000 7fa0aaffd640 1 --2- 172.21.15.43:0/1352835036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55587a8ec390 0x55587ac8ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.424+0000 7fa0aaffd640 1 -- 172.21.15.43:0/1352835036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55587ab7da40 con 0x55587a8cb6b0 2024-04-25T10:38:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.424+0000 7fa0abfff640 1 --2- 172.21.15.43:0/3567639741 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa0940afc60 0x7fa0940b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.425+0000 7fa0aaffd640 1 --2- 172.21.15.43:0/1352835036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8cb6b0 0x55587ac998f0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0012520 tx=0x7fa0a000b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1d0450a59a77eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.425+0000 7fa0a8ff9640 1 -- 172.21.15.43:0/3567639741 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa0a4100680 con 0x7fa0ac10d380 2024-04-25T10:38:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.425+0000 7fa091ffb640 1 -- 172.21.15.43:0/1352835036 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0004030 con 0x55587a8cb6b0 2024-04-25T10:38:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.425+0000 7fa091ffb640 1 -- 172.21.15.43:0/1352835036 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0a0005020 con 0x55587a8cb6b0 2024-04-25T10:38:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.425+0000 7fa091ffb640 1 -- 172.21.15.43:0/1352835036 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0004b20 con 0x55587a8cb6b0 2024-04-25T10:38:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.426+0000 7fa0b8faa740 1 -- 172.21.15.43:0/1352835036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8cb6b0 msgr2=0x55587ac998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.426+0000 7fa0abfff640 1 --2- 172.21.15.43:0/3567639741 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa0940afc60 0x7fa0940b2170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40c8660 tx=0x7fa0a40c4820 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.426+0000 7fa0b8faa740 1 --2- 172.21.15.43:0/1352835036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8cb6b0 0x55587ac998f0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0012520 tx=0x7fa0a000b5e0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.426+0000 7fa0b8faa740 1 -- 172.21.15.43:0/1352835036 shutdown_connections 2024-04-25T10:38:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.426+0000 7fa0b8faa740 1 --2- 172.21.15.43:0/1352835036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8cb6b0 0x55587ac998f0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.426+0000 7fa0b8faa740 1 --2- 172.21.15.43:0/1352835036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55587a8c34d0 0x55587ac8cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.426+0000 7fa0b8faa740 1 --2- 172.21.15.43:0/1352835036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55587a8ec390 0x55587ac8ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.426+0000 7fa0b8faa740 1 -- 172.21.15.43:0/1352835036 >> 172.21.15.43:0/1352835036 conn(0x55587a8e04e0 msgr2=0x55587ac89210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.426+0000 7fa0b8faa740 1 -- 172.21.15.43:0/1352835036 shutdown_connections 2024-04-25T10:38:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.426+0000 7fa0b8faa740 1 -- 172.21.15.43:0/1352835036 wait complete. 2024-04-25T10:38:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.426+0000 7fa0b8faa740 1 -- start start 2024-04-25T10:38:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.426+0000 7fa0b8faa740 1 Processor -- start 2024-04-25T10:38:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.427+0000 7fa0b8faa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55587a8c34d0 0x55587ac97660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.427+0000 7fa0b8faa740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55587a8cb6b0 0x55587ac97ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.427+0000 7fa0b8faa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8ec390 0x55587ac9d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.427+0000 7fa0b8faa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55587ac99fe0 con 0x55587a8ec390 2024-04-25T10:38:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.427+0000 7fa0b8faa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55587ab79d10 con 0x55587a8c34d0 2024-04-25T10:38:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.427+0000 7fa0b8faa740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55587ac9a160 con 0x55587a8cb6b0 2024-04-25T10:38:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.427+0000 7fa0b8faa740 10 client.0 _ll_register_callbacks cb 0x55587aba6210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.427+0000 7fa0aaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8ec390 0x55587ac9d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.427+0000 7fa0aaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8ec390 0x55587ac9d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37904/0 (socket says 172.21.15.43:37904) 2024-04-25T10:38:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.427+0000 7fa0aaffd640 1 -- 172.21.15.43:0/2118907970 learned_addr learned my addr 172.21.15.43:0/2118907970 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.427+0000 7fa0ab7fe640 1 --2- 172.21.15.43:0/2118907970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55587a8cb6b0 0x55587ac97ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.427+0000 7fa0aaffd640 1 --2- 172.21.15.43:0/2118907970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55587a8c34d0 0x55587ac97660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.427+0000 7fa0aaffd640 1 -- 172.21.15.43:0/2118907970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55587a8cb6b0 msgr2=0x55587ac97ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.427+0000 7fa0aaffd640 1 --2- 172.21.15.43:0/2118907970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55587a8cb6b0 0x55587ac97ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.427+0000 7fa0aaffd640 1 -- 172.21.15.43:0/2118907970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55587a8c34d0 msgr2=0x55587ac97660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.427+0000 7fa0aaffd640 1 --2- 172.21.15.43:0/2118907970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55587a8c34d0 0x55587ac97660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.427+0000 7fa0aaffd640 1 -- 172.21.15.43:0/2118907970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55587a9ce7d0 con 0x55587a8ec390 2024-04-25T10:38:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.427+0000 7fa0aaffd640 1 --2- 172.21.15.43:0/2118907970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8ec390 0x55587ac9d4a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0005e30 tx=0x7fa0a0005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330c5f419652c518 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.427+0000 7fa08affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.428+0000 7fa08affd640 1 -- 172.21.15.43:0/2118907970 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0004250 con 0x55587a8ec390 2024-04-25T10:38:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.428+0000 7fa08affd640 1 -- 172.21.15.43:0/2118907970 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa0a00043f0 con 0x55587a8ec390 2024-04-25T10:38:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.428+0000 7fa0b8faa740 1 -- 172.21.15.43:0/2118907970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0a0005020 con 0x55587a8ec390 2024-04-25T10:38:36.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.428+0000 7fa08affd640 1 -- 172.21.15.43:0/2118907970 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa0a0033050 con 0x55587a8ec390 2024-04-25T10:38:36.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.428+0000 7fa0b8faa740 10 client.17262 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:36.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.428+0000 7fa0b8faa740 1 -- 172.21.15.43:0/2118907970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55587a9043f0 con 0x55587a8ec390 2024-04-25T10:38:36.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.428+0000 7fa08affd640 1 -- 172.21.15.43:0/2118907970 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0a0005020 con 0x55587a8ec390 2024-04-25T10:38:36.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.428+0000 7fa08affd640 1 -- 172.21.15.43:0/2118907970 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa0a00035f0 con 0x55587a8ec390 2024-04-25T10:38:36.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.428+0000 7fa0b8faa740 10 client.17262 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:36.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.428+0000 7fa0b8faa740 10 client.17262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:36.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.428+0000 7fa0b8faa740 20 client.17262 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:36.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.428+0000 7fa0b8faa740 1 --2- 172.21.15.43:0/2118907970 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55587a8aaf60 0x55587aca95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.428+0000 7fa0b8faa740 4 client.17262 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:38:36.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.428+0000 7fa0b8faa740 1 -- 172.21.15.43:0/2118907970 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55587ac00990 con 0x55587a8aaf60 2024-04-25T10:38:36.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.429+0000 7fa0ab7fe640 1 --2- 172.21.15.43:0/2118907970 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55587a8aaf60 0x55587aca95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.429+0000 7fa08affd640 10 client.17262 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:36.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.429+0000 7fa0ab7fe640 1 --2- 172.21.15.43:0/2118907970 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55587a8aaf60 0x55587aca95c0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6da8ea2722a75d00 server_cookie=69eb3598fa6c7e1e in_seq=0 out_seq=0 2024-04-25T10:38:36.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.430+0000 7fa08affd640 1 -- 172.21.15.43:0/2118907970 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55587ac00990 con 0x55587a8aaf60 2024-04-25T10:38:36.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.430+0000 7fa08affd640 10 client.17262 handle_command_reply: tid=0 2024-04-25T10:38:36.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.430+0000 7fa0b8faa740 1 client.17262 shutdown 2024-04-25T10:38:36.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.430+0000 7fa0b8faa740 1 -- 172.21.15.43:0/2118907970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8ec390 msgr2=0x55587ac9d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:36.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.430+0000 7fa0b8faa740 1 --2- 172.21.15.43:0/2118907970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8ec390 0x55587ac9d4a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0005e30 tx=0x7fa0a0005d80 comp rx=0 tx=0).stop 2024-04-25T10:38:36.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.430+0000 7fa0b8faa740 1 -- 172.21.15.43:0/2118907970 shutdown_connections 2024-04-25T10:38:36.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.430+0000 7fa0b8faa740 1 --2- 172.21.15.43:0/2118907970 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55587a8aaf60 0x55587aca95c0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.430+0000 7fa0b8faa740 1 --2- 172.21.15.43:0/2118907970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8ec390 0x55587ac9d4a0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.430+0000 7fa0b8faa740 1 --2- 172.21.15.43:0/2118907970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55587a8cb6b0 0x55587ac97ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.430+0000 7fa0b8faa740 1 --2- 172.21.15.43:0/2118907970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55587a8c34d0 0x55587ac97660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.430+0000 7fa0b8faa740 1 -- 172.21.15.43:0/2118907970 >> 172.21.15.43:0/2118907970 conn(0x55587a8e04e0 msgr2=0x55587ac8e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:36.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.430+0000 7fa08affd640 0 client.17262 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:36.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.430+0000 7fa0b8faa740 1 -- 172.21.15.43:0/2118907970 shutdown_connections 2024-04-25T10:38:36.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.430+0000 7fa0b8faa740 1 -- 172.21.15.43:0/2118907970 wait complete. 2024-04-25T10:38:36.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.430+0000 7fa0b8faa740 20 client.17262 trim_cache size 0 max 16384 2024-04-25T10:38:36.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.448+0000 7fa0b8faa740 1 Processor -- start 2024-04-25T10:38:36.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.448+0000 7fa0b8faa740 1 -- start start 2024-04-25T10:38:36.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.448+0000 7fa0b8faa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55587a8c34d0 0x55587ac8ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.448+0000 7fa0b8faa740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55587a8cb6b0 0x55587ac96c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.448+0000 7fa0b8faa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8ec390 0x55587ac9d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.448+0000 7fa0b8faa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac1b20f0 con 0x55587a8ec390 2024-04-25T10:38:36.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.448+0000 7fa0b8faa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac1af9c0 con 0x55587a8c34d0 2024-04-25T10:38:36.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.448+0000 7fa0b8faa740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac1ad810 con 0x55587a8cb6b0 2024-04-25T10:38:36.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.449+0000 7fa0aaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8ec390 0x55587ac9d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.449+0000 7fa0aaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8ec390 0x55587ac9d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37920/0 (socket says 172.21.15.43:37920) 2024-04-25T10:38:36.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.449+0000 7fa0aaffd640 1 -- 172.21.15.43:0/2280230907 learned_addr learned my addr 172.21.15.43:0/2280230907 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:36.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.449+0000 7fa0aaffd640 1 --2- 172.21.15.43:0/2280230907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55587a8c34d0 0x55587ac8ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.449+0000 7fa0aaffd640 1 -- 172.21.15.43:0/2280230907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55587a8cb6b0 msgr2=0x55587ac96c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:36.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.449+0000 7fa0aaffd640 1 --2- 172.21.15.43:0/2280230907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55587a8cb6b0 0x55587ac96c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.449+0000 7fa0aaffd640 1 -- 172.21.15.43:0/2280230907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55587a8c34d0 msgr2=0x55587ac8ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:36.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.449+0000 7fa0aaffd640 1 --2- 172.21.15.43:0/2280230907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55587a8c34d0 0x55587ac8ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.449+0000 7fa0aaffd640 1 -- 172.21.15.43:0/2280230907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55587a8c40f0 con 0x55587a8ec390 2024-04-25T10:38:36.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.449+0000 7fa0aaffd640 1 --2- 172.21.15.43:0/2280230907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8ec390 0x55587ac9d420 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0012fd0 tx=0x7fa0a0012f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37857d9e5bdf2791 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:36.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.449+0000 7fa0917fa640 1 -- 172.21.15.43:0/2280230907 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a000b560 con 0x55587a8ec390 2024-04-25T10:38:36.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.449+0000 7fa0917fa640 1 -- 172.21.15.43:0/2280230907 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0a0005690 con 0x55587a8ec390 2024-04-25T10:38:36.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.449+0000 7fa0917fa640 1 -- 172.21.15.43:0/2280230907 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a00035f0 con 0x55587a8ec390 2024-04-25T10:38:36.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.450+0000 7fa0b8faa740 1 -- 172.21.15.43:0/2280230907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8ec390 msgr2=0x55587ac9d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:36.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.450+0000 7fa0b8faa740 1 --2- 172.21.15.43:0/2280230907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8ec390 0x55587ac9d420 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0012fd0 tx=0x7fa0a0012f20 comp rx=0 tx=0).stop 2024-04-25T10:38:36.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.450+0000 7fa0b8faa740 1 -- 172.21.15.43:0/2280230907 shutdown_connections 2024-04-25T10:38:36.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.450+0000 7fa0b8faa740 1 --2- 172.21.15.43:0/2280230907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8ec390 0x55587ac9d420 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.450+0000 7fa0b8faa740 1 --2- 172.21.15.43:0/2280230907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55587a8cb6b0 0x55587ac96c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.450+0000 7fa0b8faa740 1 --2- 172.21.15.43:0/2280230907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55587a8c34d0 0x55587ac8ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.450+0000 7fa0b8faa740 1 -- 172.21.15.43:0/2280230907 >> 172.21.15.43:0/2280230907 conn(0x55587a8e04e0 msgr2=0x55587ac8e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:36.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.450+0000 7fa0b8faa740 1 -- 172.21.15.43:0/2280230907 shutdown_connections 2024-04-25T10:38:36.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.450+0000 7fa0b8faa740 1 -- 172.21.15.43:0/2280230907 wait complete. 2024-04-25T10:38:36.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.450+0000 7fa0b8faa740 1 -- start start 2024-04-25T10:38:36.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.451+0000 7fa0b8faa740 1 Processor -- start 2024-04-25T10:38:36.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.451+0000 7fa0b8faa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55587a8c34d0 0x55587ac9e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.451+0000 7fa0b8faa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8cb6b0 0x55587ac9ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.451+0000 7fa0b8faa740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55587a8ec390 0x55587ac9f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.451+0000 7fa0b8faa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac1ade80 con 0x55587a8cb6b0 2024-04-25T10:38:36.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.451+0000 7fa0b8faa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac1af2f0 con 0x55587a8c34d0 2024-04-25T10:38:36.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.451+0000 7fa0b8faa740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac1ae550 con 0x55587a8ec390 2024-04-25T10:38:36.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.451+0000 7fa0b8faa740 10 client.0 _ll_register_callbacks cb 0x7fa0a000bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:36.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.451+0000 7fa0ab7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8cb6b0 0x55587ac9ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.451+0000 7fa0ab7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8cb6b0 0x55587ac9ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37930/0 (socket says 172.21.15.43:37930) 2024-04-25T10:38:36.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.451+0000 7fa0ab7fe640 1 -- 172.21.15.43:0/3557441054 learned_addr learned my addr 172.21.15.43:0/3557441054 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:36.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.451+0000 7fa0aaffd640 1 --2- 172.21.15.43:0/3557441054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55587a8c34d0 0x55587ac9e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.451+0000 7fa0aaffd640 1 --2- 172.21.15.43:0/3557441054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55587a8ec390 0x55587ac9f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.451+0000 7fa0ab7fe640 1 -- 172.21.15.43:0/3557441054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55587a8ec390 msgr2=0x55587ac9f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:36.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.451+0000 7fa0ab7fe640 1 --2- 172.21.15.43:0/3557441054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55587a8ec390 0x55587ac9f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.451+0000 7fa0ab7fe640 1 -- 172.21.15.43:0/3557441054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55587a8c34d0 msgr2=0x55587ac9e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:36.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.451+0000 7fa0ab7fe640 1 --2- 172.21.15.43:0/3557441054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55587a8c34d0 0x55587ac9e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.451+0000 7fa0ab7fe640 1 -- 172.21.15.43:0/3557441054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55587abb34b0 con 0x55587a8cb6b0 2024-04-25T10:38:36.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.451+0000 7fa0aaffd640 1 --2- 172.21.15.43:0/3557441054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55587a8ec390 0x55587ac9f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:38:36.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.452+0000 7fa0ab7fe640 1 --2- 172.21.15.43:0/3557441054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8cb6b0 0x55587ac9ed10 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa09c005860 tx=0x7fa09c00be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225a19fe095923f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:36.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.452+0000 7fa08affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:36.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.452+0000 7fa08affd640 1 -- 172.21.15.43:0/3557441054 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c003df0 con 0x55587a8cb6b0 2024-04-25T10:38:36.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.452+0000 7fa08affd640 1 -- 172.21.15.43:0/3557441054 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa09c005020 con 0x55587a8cb6b0 2024-04-25T10:38:36.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.452+0000 7fa0b8faa740 1 -- 172.21.15.43:0/3557441054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0a0005690 con 0x55587a8cb6b0 2024-04-25T10:38:36.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.452+0000 7fa08affd640 1 -- 172.21.15.43:0/3557441054 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa09c005220 con 0x55587a8cb6b0 2024-04-25T10:38:36.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.452+0000 7fa0b8faa740 10 client.17268 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:36.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.452+0000 7fa0b8faa740 1 -- 172.21.15.43:0/3557441054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55587ad10a70 con 0x55587a8cb6b0 2024-04-25T10:38:36.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.452+0000 7fa08affd640 1 -- 172.21.15.43:0/3557441054 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0a0005690 con 0x55587a8cb6b0 2024-04-25T10:38:36.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.452+0000 7fa08affd640 1 -- 172.21.15.43:0/3557441054 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa09c004830 con 0x55587a8cb6b0 2024-04-25T10:38:36.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.452+0000 7fa0b8faa740 10 client.17268 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:36.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.452+0000 7fa0b8faa740 10 client.17268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:36.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.452+0000 7fa0b8faa740 20 client.17268 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:36.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.452+0000 7fa0b8faa740 1 --2- 172.21.15.43:0/3557441054 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55587a8803a0 0x55587ad13210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:36.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.452+0000 7fa0b8faa740 4 client.17268 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:38:36.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.452+0000 7fa0b8faa740 1 -- 172.21.15.43:0/3557441054 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55587abfa800 con 0x55587a8803a0 2024-04-25T10:38:36.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.453+0000 7fa0aaffd640 1 --2- 172.21.15.43:0/3557441054 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55587a8803a0 0x55587ad13210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:36.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.453+0000 7fa08affd640 10 client.17268 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:36.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.453+0000 7fa0aaffd640 1 --2- 172.21.15.43:0/3557441054 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55587a8803a0 0x55587ad13210 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=786f03e39f1754b server_cookie=4295d050e8e8b3aa in_seq=0 out_seq=0 2024-04-25T10:38:36.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.453+0000 7fa08affd640 1 -- 172.21.15.43:0/3557441054 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55587abfa800 con 0x55587a8803a0 2024-04-25T10:38:36.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.453+0000 7fa08affd640 10 client.17268 handle_command_reply: tid=0 2024-04-25T10:38:36.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.453+0000 7fa0b8faa740 1 client.17268 shutdown 2024-04-25T10:38:36.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.453+0000 7fa0b8faa740 1 -- 172.21.15.43:0/3557441054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8cb6b0 msgr2=0x55587ac9ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:36.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.453+0000 7fa0b8faa740 1 --2- 172.21.15.43:0/3557441054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8cb6b0 0x55587ac9ed10 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa09c005860 tx=0x7fa09c00be40 comp rx=0 tx=0).stop 2024-04-25T10:38:36.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.453+0000 7fa0b8faa740 1 -- 172.21.15.43:0/3557441054 shutdown_connections 2024-04-25T10:38:36.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.453+0000 7fa0b8faa740 1 --2- 172.21.15.43:0/3557441054 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55587a8803a0 0x55587ad13210 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.453+0000 7fa0b8faa740 1 --2- 172.21.15.43:0/3557441054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55587a8ec390 0x55587ac9f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.454+0000 7fa0b8faa740 1 --2- 172.21.15.43:0/3557441054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55587a8cb6b0 0x55587ac9ed10 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.454+0000 7fa0b8faa740 1 --2- 172.21.15.43:0/3557441054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55587a8c34d0 0x55587ac9e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.454+0000 7fa08affd640 0 client.17268 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:36.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.454+0000 7fa0b8faa740 1 -- 172.21.15.43:0/3557441054 >> 172.21.15.43:0/3557441054 conn(0x55587a8e04e0 msgr2=0x55587ac879c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:36.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.454+0000 7fa0b8faa740 1 -- 172.21.15.43:0/3557441054 shutdown_connections 2024-04-25T10:38:36.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.454+0000 7fa0b8faa740 1 -- 172.21.15.43:0/3557441054 wait complete. 2024-04-25T10:38:36.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.454+0000 7fa0b8faa740 20 client.17268 trim_cache size 0 max 16384 2024-04-25T10:38:36.462 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:38:36.462 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:38:36.463 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:38:36.463 INFO:teuthology.orchestra.run.smithi043.stdout: "fd5ba958-425c-4bfe-95a0-39445510ab58": { 2024-04-25T10:38:36.463 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:38:36.463 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "fd5ba958-425c-4bfe-95a0-39445510ab58", 2024-04-25T10:38:36.463 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:38:36.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:38:36.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:38:36.463 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:38:36.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.454+0000 7fa0b20f9640 1 -- 172.21.15.43:0/3567639741 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa0940afc60 msgr2=0x7fa0940b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:36.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.454+0000 7fa0b20f9640 1 --2- 172.21.15.43:0/3567639741 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa0940afc60 0x7fa0940b2170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40c8660 tx=0x7fa0a40c4820 comp rx=0 tx=0).stop 2024-04-25T10:38:36.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.454+0000 7fa0b20f9640 1 -- 172.21.15.43:0/3567639741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0ac10d380 msgr2=0x7fa0ac19ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:36.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.454+0000 7fa0b20f9640 1 --2- 172.21.15.43:0/3567639741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0ac10d380 0x7fa0ac19ee90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4006010 tx=0x7fa0a402f380 comp rx=0 tx=0).stop 2024-04-25T10:38:36.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.454+0000 7fa0b20f9640 1 -- 172.21.15.43:0/3567639741 shutdown_connections 2024-04-25T10:38:36.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.454+0000 7fa0b20f9640 1 --2- 172.21.15.43:0/3567639741 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa0940afc60 0x7fa0940b2170 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.454+0000 7fa0b20f9640 1 --2- 172.21.15.43:0/3567639741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0ac10d380 0x7fa0ac19ee90 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.454+0000 7fa0b20f9640 1 --2- 172.21.15.43:0/3567639741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0ac108a20 0x7fa0ac19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.454+0000 7fa0b20f9640 1 --2- 172.21.15.43:0/3567639741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0ac102af0 0x7fa0ac19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:36.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.454+0000 7fa0b20f9640 1 -- 172.21.15.43:0/3567639741 >> 172.21.15.43:0/3567639741 conn(0x7fa0ac078100 msgr2=0x7fa0ac069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:36.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.454+0000 7fa0b20f9640 1 -- 172.21.15.43:0/3567639741 shutdown_connections 2024-04-25T10:38:36.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:38:36.454+0000 7fa0b20f9640 1 -- 172.21.15.43:0/3567639741 wait complete. 2024-04-25T10:38:36.467 INFO:tasks.cephfs.filesystem:scrub status for tag:fd5ba958-425c-4bfe-95a0-39445510ab58 - {'path': '/', 'tag': 'fd5ba958-425c-4bfe-95a0-39445510ab58', 'options': 'recursive,force'} 2024-04-25T10:38:37.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:36 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:38:37.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:36 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:38:37.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:36 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:38:37.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:36 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:38:37.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:36 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:38:37.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:36 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:38:37.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:36 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:38:37.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:36 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:38:37.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:36 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:38:37.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:36 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:38:37.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:36 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:38:37.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:36 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:38:38.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:37 smithi077 ceph-mon[32772]: pgmap v186: 97 pgs: 97 active+clean; 4.8 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-25T10:38:38.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:37 smithi164 ceph-mon[32713]: pgmap v186: 97 pgs: 97 active+clean; 4.8 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-25T10:38:38.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:37 smithi043 ceph-mon[29478]: pgmap v186: 97 pgs: 97 active+clean; 4.8 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-25T10:38:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.041+0000 7fe4e79ce640 1 -- 172.21.15.43:0/3986597543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e01042c0 msgr2=0x7fe4e0110690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.041+0000 7fe4e79ce640 1 --2- 172.21.15.43:0/3986597543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e01042c0 0x7fe4e0110690 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009ca0 tx=0x7fe4d802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.041+0000 7fe4e79ce640 1 -- 172.21.15.43:0/3986597543 shutdown_connections 2024-04-25T10:38:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.041+0000 7fe4e79ce640 1 --2- 172.21.15.43:0/3986597543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4e010a010 0x7fe4e0103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.041+0000 7fe4e79ce640 1 --2- 172.21.15.43:0/3986597543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e01042c0 0x7fe4e0110690 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.041+0000 7fe4e79ce640 1 --2- 172.21.15.43:0/3986597543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4e0109b70 0x7fe4e0103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.041+0000 7fe4e79ce640 1 -- 172.21.15.43:0/3986597543 >> 172.21.15.43:0/3986597543 conn(0x7fe4e00ff210 msgr2=0x7fe4e0101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.041+0000 7fe4e79ce640 1 -- 172.21.15.43:0/3986597543 shutdown_connections 2024-04-25T10:38:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.041+0000 7fe4e79ce640 1 -- 172.21.15.43:0/3986597543 wait complete. 2024-04-25T10:38:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.042+0000 7fe4e79ce640 1 Processor -- start 2024-04-25T10:38:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.042+0000 7fe4e79ce640 1 -- start start 2024-04-25T10:38:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.042+0000 7fe4e79ce640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4e01042c0 0x7fe4e019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.042+0000 7fe4e79ce640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4e0109b70 0x7fe4e019ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.042+0000 7fe4e79ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e010a010 0x7fe4e019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.042+0000 7fe4e79ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e0113d80 con 0x7fe4e010a010 2024-04-25T10:38:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.042+0000 7fe4e79ce640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e0113c00 con 0x7fe4e0109b70 2024-04-25T10:38:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.042+0000 7fe4e79ce640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e0113f00 con 0x7fe4e01042c0 2024-04-25T10:38:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.043+0000 7fe4e5f6b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e010a010 0x7fe4e019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.043+0000 7fe4e5f6b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e010a010 0x7fe4e019ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47878/0 (socket says 172.21.15.43:47878) 2024-04-25T10:38:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.043+0000 7fe4e5f6b640 1 -- 172.21.15.43:0/1350303975 learned_addr learned my addr 172.21.15.43:0/1350303975 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.043+0000 7fe4e4f69640 1 --2- 172.21.15.43:0/1350303975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4e0109b70 0x7fe4e019ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.043+0000 7fe4e576a640 1 --2- 172.21.15.43:0/1350303975 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4e01042c0 0x7fe4e019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.043+0000 7fe4e5f6b640 1 -- 172.21.15.43:0/1350303975 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4e01042c0 msgr2=0x7fe4e019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.043+0000 7fe4e5f6b640 1 --2- 172.21.15.43:0/1350303975 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4e01042c0 0x7fe4e019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.043+0000 7fe4e5f6b640 1 -- 172.21.15.43:0/1350303975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4e0109b70 msgr2=0x7fe4e019ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.043+0000 7fe4e5f6b640 1 --2- 172.21.15.43:0/1350303975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4e0109b70 0x7fe4e019ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.043+0000 7fe4e5f6b640 1 -- 172.21.15.43:0/1350303975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e019f5b0 con 0x7fe4e010a010 2024-04-25T10:38:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.043+0000 7fe4e5f6b640 1 --2- 172.21.15.43:0/1350303975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e010a010 0x7fe4e019ef70 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8005cc0 tx=0x7fe4d802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.043+0000 7fe4d67fc640 1 -- 172.21.15.43:0/1350303975 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8003bf0 con 0x7fe4e010a010 2024-04-25T10:38:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.043+0000 7fe4d67fc640 1 -- 172.21.15.43:0/1350303975 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d8003d90 con 0x7fe4e010a010 2024-04-25T10:38:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.043+0000 7fe4e79ce640 1 -- 172.21.15.43:0/1350303975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e019f840 con 0x7fe4e010a010 2024-04-25T10:38:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.043+0000 7fe4d67fc640 1 -- 172.21.15.43:0/1350303975 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d802fbc0 con 0x7fe4e010a010 2024-04-25T10:38:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.044+0000 7fe4e79ce640 1 -- 172.21.15.43:0/1350303975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e019fd90 con 0x7fe4e010a010 2024-04-25T10:38:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.046+0000 7fe4d67fc640 1 -- 172.21.15.43:0/1350303975 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fe4d802fd60 con 0x7fe4e010a010 2024-04-25T10:38:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.046+0000 7fe4ee87d740 1 Processor -- start 2024-04-25T10:38:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.047+0000 7fe4ee87d740 1 -- start start 2024-04-25T10:38:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.047+0000 7fe4ee87d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a991b0 0x55ba49e389a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.047+0000 7fe4ee87d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a702f0 0x55ba49e38ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.047+0000 7fe4ee87d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a784d0 0x55ba49e39420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.047+0000 7fe4ee87d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ba49d9f460 con 0x55ba49a991b0 2024-04-25T10:38:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.047+0000 7fe4ee87d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ba49d59590 con 0x55ba49a784d0 2024-04-25T10:38:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.047+0000 7fe4ee87d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ba49d9b420 con 0x55ba49a702f0 2024-04-25T10:38:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.047+0000 7fe4e4f69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a991b0 0x55ba49e389a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.047+0000 7fe4e4f69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a991b0 0x55ba49e389a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47894/0 (socket says 172.21.15.43:47894) 2024-04-25T10:38:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.047+0000 7fe4e4f69640 1 -- 172.21.15.43:0/4046866064 learned_addr learned my addr 172.21.15.43:0/4046866064 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.047+0000 7fe4e576a640 1 --2- 172.21.15.43:0/4046866064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a702f0 0x55ba49e38ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.047+0000 7fe4d67fc640 1 --2- 172.21.15.43:0/1350303975 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe4cc0afc50 0x7fe4cc0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.047+0000 7fe4e4f69640 1 --2- 172.21.15.43:0/4046866064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a784d0 0x55ba49e39420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.047+0000 7fe4e4f69640 1 -- 172.21.15.43:0/4046866064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a702f0 msgr2=0x55ba49e38ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.047+0000 7fe4e4f69640 1 --2- 172.21.15.43:0/4046866064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a702f0 0x55ba49e38ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.047+0000 7fe4e4f69640 1 -- 172.21.15.43:0/4046866064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a784d0 msgr2=0x55ba49e39420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.047+0000 7fe4e4f69640 1 --2- 172.21.15.43:0/4046866064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a784d0 0x55ba49e39420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.047+0000 7fe4e4f69640 1 -- 172.21.15.43:0/4046866064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba49d660d0 con 0x55ba49a991b0 2024-04-25T10:38:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.048+0000 7fe4e5f6b640 1 --2- 172.21.15.43:0/1350303975 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe4cc0afc50 0x7fe4cc0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.048+0000 7fe4d67fc640 1 -- 172.21.15.43:0/1350303975 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fe4d81005b0 con 0x7fe4e010a010 2024-04-25T10:38:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.048+0000 7fe4e4f69640 1 --2- 172.21.15.43:0/4046866064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a991b0 0x55ba49e389a0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00124f0 tx=0x7fe4d000b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f3611051078e70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.048+0000 7fe4cb7fe640 1 -- 172.21.15.43:0/4046866064 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0004020 con 0x55ba49a991b0 2024-04-25T10:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.048+0000 7fe4cb7fe640 1 -- 172.21.15.43:0/4046866064 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d00054d0 con 0x55ba49a991b0 2024-04-25T10:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.048+0000 7fe4cb7fe640 1 -- 172.21.15.43:0/4046866064 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0004a00 con 0x55ba49a991b0 2024-04-25T10:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.049+0000 7fe4ee87d740 1 -- 172.21.15.43:0/4046866064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a991b0 msgr2=0x55ba49e389a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.049+0000 7fe4ee87d740 1 --2- 172.21.15.43:0/4046866064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a991b0 0x55ba49e389a0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00124f0 tx=0x7fe4d000b740 comp rx=0 tx=0).stop 2024-04-25T10:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.049+0000 7fe4e5f6b640 1 --2- 172.21.15.43:0/1350303975 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe4cc0afc50 0x7fe4cc0b2160 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80fdef0 tx=0x7fe4d80960f0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.049+0000 7fe4ee87d740 1 -- 172.21.15.43:0/4046866064 shutdown_connections 2024-04-25T10:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.049+0000 7fe4ee87d740 1 --2- 172.21.15.43:0/4046866064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a784d0 0x55ba49e39420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.049+0000 7fe4ee87d740 1 --2- 172.21.15.43:0/4046866064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a702f0 0x55ba49e38ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.049+0000 7fe4ee87d740 1 --2- 172.21.15.43:0/4046866064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a991b0 0x55ba49e389a0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.049+0000 7fe4ee87d740 1 -- 172.21.15.43:0/4046866064 >> 172.21.15.43:0/4046866064 conn(0x55ba49a8d300 msgr2=0x55ba49e352a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.049+0000 7fe4ee87d740 1 -- 172.21.15.43:0/4046866064 shutdown_connections 2024-04-25T10:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.049+0000 7fe4ee87d740 1 -- 172.21.15.43:0/4046866064 wait complete. 2024-04-25T10:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.049+0000 7fe4ee87d740 1 -- start start 2024-04-25T10:38:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.049+0000 7fe4ee87d740 1 Processor -- start 2024-04-25T10:38:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.050+0000 7fe4ee87d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a702f0 0x55ba49e49000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.050+0000 7fe4ee87d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a784d0 0x55ba49e49540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.050+0000 7fe4ee87d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a991b0 0x55ba49e47480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.050+0000 7fe4ee87d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ba49e4a030 con 0x55ba49a784d0 2024-04-25T10:38:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.050+0000 7fe4ee87d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ba49d26b60 con 0x55ba49a991b0 2024-04-25T10:38:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.050+0000 7fe4ee87d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ba49e4a1b0 con 0x55ba49a702f0 2024-04-25T10:38:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.050+0000 7fe4ee87d740 10 client.0 _ll_register_callbacks cb 0x55ba49d4b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.050+0000 7fe4e576a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a784d0 0x55ba49e49540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.050+0000 7fe4e576a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a784d0 0x55ba49e49540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47898/0 (socket says 172.21.15.43:47898) 2024-04-25T10:38:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.050+0000 7fe4e576a640 1 -- 172.21.15.43:0/2317523137 learned_addr learned my addr 172.21.15.43:0/2317523137 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.050+0000 7fe4e4f69640 1 --2- 172.21.15.43:0/2317523137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a702f0 0x55ba49e49000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.050+0000 7fe4e576a640 1 -- 172.21.15.43:0/2317523137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a702f0 msgr2=0x55ba49e49000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.050+0000 7fe4e576a640 1 --2- 172.21.15.43:0/2317523137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a702f0 0x55ba49e49000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.050+0000 7fe4e576a640 1 -- 172.21.15.43:0/2317523137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a991b0 msgr2=0x55ba49e47480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.050+0000 7fe4e4f69640 1 --2- 172.21.15.43:0/2317523137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a991b0 0x55ba49e47480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.050+0000 7fe4e576a640 1 --2- 172.21.15.43:0/2317523137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a991b0 0x55ba49e47480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.050+0000 7fe4e576a640 1 -- 172.21.15.43:0/2317523137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba49ab1210 con 0x55ba49a784d0 2024-04-25T10:38:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.051+0000 7fe4e576a640 1 --2- 172.21.15.43:0/2317523137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a784d0 0x55ba49e49540 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00b990 tx=0x7fe4dc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279824ba4cc677ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.051+0000 7fe4c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.051+0000 7fe4c8ff9640 1 -- 172.21.15.43:0/2317523137 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc005020 con 0x55ba49a784d0 2024-04-25T10:38:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.051+0000 7fe4c8ff9640 1 -- 172.21.15.43:0/2317523137 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fe4dc0051c0 con 0x55ba49a784d0 2024-04-25T10:38:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.051+0000 7fe4ee87d740 1 -- 172.21.15.43:0/2317523137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d00054d0 con 0x55ba49a784d0 2024-04-25T10:38:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.051+0000 7fe4c8ff9640 1 -- 172.21.15.43:0/2317523137 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4dc0053c0 con 0x55ba49a784d0 2024-04-25T10:38:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.051+0000 7fe4ee87d740 10 client.17280 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.051+0000 7fe4ee87d740 1 -- 172.21.15.43:0/2317523137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ba49b7d620 con 0x55ba49a784d0 2024-04-25T10:38:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.051+0000 7fe4c8ff9640 1 -- 172.21.15.43:0/2317523137 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d00054d0 con 0x55ba49a784d0 2024-04-25T10:38:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.051+0000 7fe4c8ff9640 1 -- 172.21.15.43:0/2317523137 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe4dc01b590 con 0x55ba49a784d0 2024-04-25T10:38:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.051+0000 7fe4ee87d740 10 client.17280 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.051+0000 7fe4ee87d740 10 client.17280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.051+0000 7fe4ee87d740 20 client.17280 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.051+0000 7fe4ee87d740 1 --2- 172.21.15.43:0/2317523137 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ba49a57d80 0x55ba49e59c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.051+0000 7fe4ee87d740 4 client.17280 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:38:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.051+0000 7fe4ee87d740 1 -- 172.21.15.43:0/2317523137 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba49dad070 con 0x55ba49a57d80 2024-04-25T10:38:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.052+0000 7fe4e4f69640 1 --2- 172.21.15.43:0/2317523137 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ba49a57d80 0x55ba49e59c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.052+0000 7fe4e4f69640 1 --2- 172.21.15.43:0/2317523137 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ba49a57d80 0x55ba49e59c30 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d6b1e001152b68a server_cookie=1542b6dfc8ddd436 in_seq=0 out_seq=0 2024-04-25T10:38:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.052+0000 7fe4c8ff9640 10 client.17280 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.053+0000 7fe4c8ff9640 1 -- 172.21.15.43:0/2317523137 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ba49dad070 con 0x55ba49a57d80 2024-04-25T10:38:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.053+0000 7fe4c8ff9640 10 client.17280 handle_command_reply: tid=0 2024-04-25T10:38:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.053+0000 7fe4ee87d740 1 client.17280 shutdown 2024-04-25T10:38:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.053+0000 7fe4ee87d740 1 -- 172.21.15.43:0/2317523137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a784d0 msgr2=0x55ba49e49540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.053+0000 7fe4ee87d740 1 --2- 172.21.15.43:0/2317523137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a784d0 0x55ba49e49540 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00b990 tx=0x7fe4dc00be50 comp rx=0 tx=0).stop 2024-04-25T10:38:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.053+0000 7fe4ee87d740 1 -- 172.21.15.43:0/2317523137 shutdown_connections 2024-04-25T10:38:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.053+0000 7fe4ee87d740 1 --2- 172.21.15.43:0/2317523137 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ba49a57d80 0x55ba49e59c30 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.053+0000 7fe4ee87d740 1 --2- 172.21.15.43:0/2317523137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a991b0 0x55ba49e47480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.053+0000 7fe4ee87d740 1 --2- 172.21.15.43:0/2317523137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a784d0 0x55ba49e49540 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.053+0000 7fe4ee87d740 1 --2- 172.21.15.43:0/2317523137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a702f0 0x55ba49e49000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.053+0000 7fe4c8ff9640 0 client.17280 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.053+0000 7fe4ee87d740 1 -- 172.21.15.43:0/2317523137 >> 172.21.15.43:0/2317523137 conn(0x55ba49a8d300 msgr2=0x55ba49e3a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.053+0000 7fe4ee87d740 1 -- 172.21.15.43:0/2317523137 shutdown_connections 2024-04-25T10:38:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.053+0000 7fe4ee87d740 1 -- 172.21.15.43:0/2317523137 wait complete. 2024-04-25T10:38:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.053+0000 7fe4ee87d740 20 client.17280 trim_cache size 0 max 16384 2024-04-25T10:38:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.069+0000 7fe4ee87d740 1 Processor -- start 2024-04-25T10:38:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.069+0000 7fe4ee87d740 1 -- start start 2024-04-25T10:38:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.069+0000 7fe4ee87d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a702f0 0x55ba49e3aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.069+0000 7fe4ee87d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a784d0 0x55ba49e5f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.069+0000 7fe4ee87d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a991b0 0x55ba49e5f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.069+0000 7fe4ee87d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01affb0 con 0x55ba49a702f0 2024-04-25T10:38:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.069+0000 7fe4ee87d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01ad880 con 0x55ba49a784d0 2024-04-25T10:38:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.069+0000 7fe4ee87d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01ab6d0 con 0x55ba49a991b0 2024-04-25T10:38:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.069+0000 7fe4e4f69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a702f0 0x55ba49e3aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.069+0000 7fe4e4f69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a702f0 0x55ba49e3aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47904/0 (socket says 172.21.15.43:47904) 2024-04-25T10:38:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.069+0000 7fe4e4f69640 1 -- 172.21.15.43:0/2252398159 learned_addr learned my addr 172.21.15.43:0/2252398159 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.069+0000 7fe4e576a640 1 --2- 172.21.15.43:0/2252398159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a784d0 0x55ba49e5f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.069+0000 7fe4e4f69640 1 --2- 172.21.15.43:0/2252398159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a991b0 0x55ba49e5f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.069+0000 7fe4e4f69640 1 -- 172.21.15.43:0/2252398159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a991b0 msgr2=0x55ba49e5f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.069+0000 7fe4e4f69640 1 --2- 172.21.15.43:0/2252398159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a991b0 0x55ba49e5f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.069+0000 7fe4e4f69640 1 -- 172.21.15.43:0/2252398159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a784d0 msgr2=0x55ba49e5f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.069+0000 7fe4e4f69640 1 --2- 172.21.15.43:0/2252398159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a784d0 0x55ba49e5f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.069+0000 7fe4e4f69640 1 -- 172.21.15.43:0/2252398159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba49e459d0 con 0x55ba49a702f0 2024-04-25T10:38:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.069+0000 7fe4e576a640 1 --2- 172.21.15.43:0/2252398159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a784d0 0x55ba49e5f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:38:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.070+0000 7fe4e4f69640 1 --2- 172.21.15.43:0/2252398159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a702f0 0x55ba49e3aeb0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00057c0 tx=0x7fe4d000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c62649fae04eff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.070+0000 7fe4caffd640 1 -- 172.21.15.43:0/2252398159 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0005050 con 0x55ba49a702f0 2024-04-25T10:38:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.070+0000 7fe4caffd640 1 -- 172.21.15.43:0/2252398159 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d00051f0 con 0x55ba49a702f0 2024-04-25T10:38:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.070+0000 7fe4caffd640 1 -- 172.21.15.43:0/2252398159 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0010460 con 0x55ba49a702f0 2024-04-25T10:38:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.070+0000 7fe4ee87d740 1 -- 172.21.15.43:0/2252398159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a702f0 msgr2=0x55ba49e3aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.070+0000 7fe4ee87d740 1 --2- 172.21.15.43:0/2252398159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a702f0 0x55ba49e3aeb0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00057c0 tx=0x7fe4d000b5b0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.071+0000 7fe4ee87d740 1 -- 172.21.15.43:0/2252398159 shutdown_connections 2024-04-25T10:38:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.071+0000 7fe4ee87d740 1 --2- 172.21.15.43:0/2252398159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a784d0 0x55ba49e5f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.071+0000 7fe4ee87d740 1 --2- 172.21.15.43:0/2252398159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a991b0 0x55ba49e5f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.071+0000 7fe4ee87d740 1 --2- 172.21.15.43:0/2252398159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a702f0 0x55ba49e3aeb0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.071+0000 7fe4ee87d740 1 -- 172.21.15.43:0/2252398159 >> 172.21.15.43:0/2252398159 conn(0x55ba49a8d300 msgr2=0x55ba49e3a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.071+0000 7fe4ee87d740 1 -- 172.21.15.43:0/2252398159 shutdown_connections 2024-04-25T10:38:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.071+0000 7fe4ee87d740 1 -- 172.21.15.43:0/2252398159 wait complete. 2024-04-25T10:38:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.071+0000 7fe4ee87d740 1 -- start start 2024-04-25T10:38:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.071+0000 7fe4ee87d740 1 Processor -- start 2024-04-25T10:38:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.071+0000 7fe4ee87d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a702f0 0x55ba49e4fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.071+0000 7fe4ee87d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a784d0 0x55ba49e4aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.071+0000 7fe4ee87d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a991b0 0x55ba49e4b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.071+0000 7fe4ee87d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01abd40 con 0x55ba49a991b0 2024-04-25T10:38:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.071+0000 7fe4ee87d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01ad1b0 con 0x55ba49a702f0 2024-04-25T10:38:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.071+0000 7fe4ee87d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01ac410 con 0x55ba49a784d0 2024-04-25T10:38:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.071+0000 7fe4ee87d740 10 client.0 _ll_register_callbacks cb 0x7fe4dc0042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.072+0000 7fe4e4f69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a991b0 0x55ba49e4b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.072+0000 7fe4e4f69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a991b0 0x55ba49e4b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47910/0 (socket says 172.21.15.43:47910) 2024-04-25T10:38:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.072+0000 7fe4e4f69640 1 -- 172.21.15.43:0/662745343 learned_addr learned my addr 172.21.15.43:0/662745343 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.072+0000 7fe4e576a640 1 --2- 172.21.15.43:0/662745343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a784d0 0x55ba49e4aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.072+0000 7fe4e4f69640 1 --2- 172.21.15.43:0/662745343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a702f0 0x55ba49e4fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.072+0000 7fe4e4f69640 1 -- 172.21.15.43:0/662745343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a784d0 msgr2=0x55ba49e4aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.072+0000 7fe4e4f69640 1 --2- 172.21.15.43:0/662745343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a784d0 0x55ba49e4aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.072+0000 7fe4e4f69640 1 -- 172.21.15.43:0/662745343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a702f0 msgr2=0x55ba49e4fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.072+0000 7fe4e4f69640 1 --2- 172.21.15.43:0/662745343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a702f0 0x55ba49e4fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.072+0000 7fe4e4f69640 1 -- 172.21.15.43:0/662745343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba49e4bc80 con 0x55ba49a991b0 2024-04-25T10:38:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.072+0000 7fe4e576a640 1 --2- 172.21.15.43:0/662745343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a784d0 0x55ba49e4aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:38:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.072+0000 7fe4e4f69640 1 --2- 172.21.15.43:0/662745343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a991b0 0x55ba49e4b4f0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0012ec0 tx=0x7fe4d0004bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d032a1551734879e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.072+0000 7fe4c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.072+0000 7fe4c8ff9640 1 -- 172.21.15.43:0/662745343 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0004c70 con 0x55ba49a991b0 2024-04-25T10:38:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.072+0000 7fe4c8ff9640 1 -- 172.21.15.43:0/662745343 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fe4d00045a0 con 0x55ba49a991b0 2024-04-25T10:38:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.072+0000 7fe4ee87d740 1 -- 172.21.15.43:0/662745343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d00051f0 con 0x55ba49a991b0 2024-04-25T10:38:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.072+0000 7fe4c8ff9640 1 -- 172.21.15.43:0/662745343 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4d0004e10 con 0x55ba49a991b0 2024-04-25T10:38:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.073+0000 7fe4ee87d740 10 client.17286 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.073+0000 7fe4ee87d740 1 -- 172.21.15.43:0/662745343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ba49ec1ff0 con 0x55ba49a991b0 2024-04-25T10:38:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.073+0000 7fe4c8ff9640 1 -- 172.21.15.43:0/662745343 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d00051f0 con 0x55ba49a991b0 2024-04-25T10:38:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.073+0000 7fe4c8ff9640 1 -- 172.21.15.43:0/662745343 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe4d001c070 con 0x55ba49a991b0 2024-04-25T10:38:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.073+0000 7fe4ee87d740 10 client.17286 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.073+0000 7fe4ee87d740 10 client.17286 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.073+0000 7fe4ee87d740 20 client.17286 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.073+0000 7fe4ee87d740 1 --2- 172.21.15.43:0/662745343 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ba49a2d1c0 0x55ba49ec4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.073+0000 7fe4ee87d740 4 client.17286 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:38:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.073+0000 7fe4ee87d740 1 -- 172.21.15.43:0/662745343 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba49e43060 con 0x55ba49a2d1c0 2024-04-25T10:38:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.073+0000 7fe4e576a640 1 --2- 172.21.15.43:0/662745343 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ba49a2d1c0 0x55ba49ec4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.074+0000 7fe4c8ff9640 10 client.17286 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.074+0000 7fe4e576a640 1 --2- 172.21.15.43:0/662745343 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ba49a2d1c0 0x55ba49ec4480 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c83cb304f12aac54 server_cookie=12f7e58b561f11ef in_seq=0 out_seq=0 2024-04-25T10:38:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.087+0000 7fe4c8ff9640 1 -- 172.21.15.43:0/662745343 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba49e43060 con 0x55ba49a2d1c0 2024-04-25T10:38:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.087+0000 7fe4c8ff9640 10 client.17286 handle_command_reply: tid=0 2024-04-25T10:38:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.087+0000 7fe4ee87d740 1 client.17286 shutdown 2024-04-25T10:38:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.087+0000 7fe4ee87d740 1 -- 172.21.15.43:0/662745343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a991b0 msgr2=0x55ba49e4b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.087+0000 7fe4ee87d740 1 --2- 172.21.15.43:0/662745343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a991b0 0x55ba49e4b4f0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0012ec0 tx=0x7fe4d0004bd0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.087+0000 7fe4ee87d740 1 -- 172.21.15.43:0/662745343 shutdown_connections 2024-04-25T10:38:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.087+0000 7fe4ee87d740 1 --2- 172.21.15.43:0/662745343 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ba49a2d1c0 0x55ba49ec4480 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.087+0000 7fe4ee87d740 1 --2- 172.21.15.43:0/662745343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba49a991b0 0x55ba49e4b4f0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.087+0000 7fe4ee87d740 1 --2- 172.21.15.43:0/662745343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ba49a784d0 0x55ba49e4aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.087+0000 7fe4c8ff9640 0 client.17286 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.087+0000 7fe4ee87d740 1 --2- 172.21.15.43:0/662745343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba49a702f0 0x55ba49e4fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.087+0000 7fe4ee87d740 1 -- 172.21.15.43:0/662745343 >> 172.21.15.43:0/662745343 conn(0x55ba49a8d300 msgr2=0x55ba49e38880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.087+0000 7fe4ee87d740 1 -- 172.21.15.43:0/662745343 shutdown_connections 2024-04-25T10:38:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.087+0000 7fe4ee87d740 1 -- 172.21.15.43:0/662745343 wait complete. 2024-04-25T10:38:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.088+0000 7fe4ee87d740 20 client.17286 trim_cache size 0 max 16384 2024-04-25T10:38:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:38:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:38:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:38:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:38:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.088+0000 7fe4e79ce640 1 -- 172.21.15.43:0/1350303975 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe4cc0afc50 msgr2=0x7fe4cc0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.088+0000 7fe4e79ce640 1 --2- 172.21.15.43:0/1350303975 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe4cc0afc50 0x7fe4cc0b2160 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80fdef0 tx=0x7fe4d80960f0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.088+0000 7fe4e79ce640 1 -- 172.21.15.43:0/1350303975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e010a010 msgr2=0x7fe4e019ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.088+0000 7fe4e79ce640 1 --2- 172.21.15.43:0/1350303975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e010a010 0x7fe4e019ef70 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8005cc0 tx=0x7fe4d802f380 comp rx=0 tx=0).stop 2024-04-25T10:38:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.088+0000 7fe4e79ce640 1 -- 172.21.15.43:0/1350303975 shutdown_connections 2024-04-25T10:38:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.088+0000 7fe4e79ce640 1 --2- 172.21.15.43:0/1350303975 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe4cc0afc50 0x7fe4cc0b2160 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.088+0000 7fe4e79ce640 1 --2- 172.21.15.43:0/1350303975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4e0109b70 0x7fe4e019ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.088+0000 7fe4e79ce640 1 --2- 172.21.15.43:0/1350303975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e010a010 0x7fe4e019ef70 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.088+0000 7fe4e79ce640 1 --2- 172.21.15.43:0/1350303975 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4e01042c0 0x7fe4e019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.088+0000 7fe4e79ce640 1 -- 172.21.15.43:0/1350303975 >> 172.21.15.43:0/1350303975 conn(0x7fe4e00ff210 msgr2=0x7fe4e0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.088+0000 7fe4e79ce640 1 -- 172.21.15.43:0/1350303975 shutdown_connections 2024-04-25T10:38:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:39.088+0000 7fe4e79ce640 1 -- 172.21.15.43:0/1350303975 wait complete. 2024-04-25T10:38:39.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:38 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:38:39.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:38 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:38:39.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:38 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:38:40.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:39 smithi077 ceph-mon[32772]: pgmap v187: 97 pgs: 97 active+clean; 5.0 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 18 op/s 2024-04-25T10:38:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:39 smithi164 ceph-mon[32713]: pgmap v187: 97 pgs: 97 active+clean; 5.0 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 18 op/s 2024-04-25T10:38:40.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:39 smithi043 ceph-mon[29478]: pgmap v187: 97 pgs: 97 active+clean; 5.0 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 18 op/s 2024-04-25T10:38:42.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:41 smithi077 ceph-mon[32772]: pgmap v188: 97 pgs: 97 active+clean; 5.0 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-04-25T10:38:42.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:41 smithi164 ceph-mon[32713]: pgmap v188: 97 pgs: 97 active+clean; 5.0 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-04-25T10:38:42.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:41 smithi043 ceph-mon[29478]: pgmap v188: 97 pgs: 97 active+clean; 5.0 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-04-25T10:38:44.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:43 smithi077 ceph-mon[32772]: pgmap v189: 97 pgs: 97 active+clean; 5.0 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-04-25T10:38:44.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:43 smithi164 ceph-mon[32713]: pgmap v189: 97 pgs: 97 active+clean; 5.0 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-04-25T10:38:44.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:43 smithi043 ceph-mon[29478]: pgmap v189: 97 pgs: 97 active+clean; 5.0 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-04-25T10:38:46.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:45 smithi077 ceph-mon[32772]: pgmap v190: 97 pgs: 97 active+clean; 5.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 18 op/s 2024-04-25T10:38:46.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:45 smithi164 ceph-mon[32713]: pgmap v190: 97 pgs: 97 active+clean; 5.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 18 op/s 2024-04-25T10:38:46.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:45 smithi043 ceph-mon[29478]: pgmap v190: 97 pgs: 97 active+clean; 5.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 18 op/s 2024-04-25T10:38:47.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:46 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:38:47.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:46 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:38:47.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:46 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:38:48.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:47 smithi077 ceph-mon[32772]: pgmap v191: 97 pgs: 97 active+clean; 5.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-04-25T10:38:48.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:47 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:38:48.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:47 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:38:48.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:47 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:38:48.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:47 smithi164 ceph-mon[32713]: pgmap v191: 97 pgs: 97 active+clean; 5.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-04-25T10:38:48.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:47 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:38:48.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:47 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:38:48.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:47 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:38:48.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:47 smithi043 ceph-mon[29478]: pgmap v191: 97 pgs: 97 active+clean; 5.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-04-25T10:38:48.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:47 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:38:48.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:47 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:38:48.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:47 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:38:50.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:49 smithi077 ceph-mon[32772]: pgmap v192: 97 pgs: 97 active+clean; 5.4 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-25T10:38:50.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:49 smithi164 ceph-mon[32713]: pgmap v192: 97 pgs: 97 active+clean; 5.4 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-25T10:38:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:49 smithi043 ceph-mon[29478]: pgmap v192: 97 pgs: 97 active+clean; 5.4 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-25T10:38:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.200+0000 7fe4fc26e640 1 -- 172.21.15.43:0/2281876068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4f4109c00 msgr2=0x7fe4f4101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.200+0000 7fe4fc26e640 1 --2- 172.21.15.43:0/2281876068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4f4109c00 0x7fe4f4101b70 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009ca0 tx=0x7fe4f002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.200+0000 7fe4fc26e640 1 -- 172.21.15.43:0/2281876068 shutdown_connections 2024-04-25T10:38:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.200+0000 7fe4fc26e640 1 --2- 172.21.15.43:0/2281876068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4f41020b0 0x7fe4f41024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.200+0000 7fe4fc26e640 1 --2- 172.21.15.43:0/2281876068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f410e5b0 0x7fe4f41109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.200+0000 7fe4fc26e640 1 --2- 172.21.15.43:0/2281876068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4f4109c00 0x7fe4f4101b70 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.200+0000 7fe4fc26e640 1 -- 172.21.15.43:0/2281876068 >> 172.21.15.43:0/2281876068 conn(0x7fe4f40fd8a0 msgr2=0x7fe4f40ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.200+0000 7fe4fc26e640 1 -- 172.21.15.43:0/2281876068 shutdown_connections 2024-04-25T10:38:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.200+0000 7fe4fc26e640 1 -- 172.21.15.43:0/2281876068 wait complete. 2024-04-25T10:38:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.200+0000 7fe4fc26e640 1 Processor -- start 2024-04-25T10:38:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.201+0000 7fe4fc26e640 1 -- start start 2024-04-25T10:38:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.201+0000 7fe4fc26e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4f41020b0 0x7fe4f419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.201+0000 7fe4fc26e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4f4109c00 0x7fe4f419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.201+0000 7fe4fc26e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f410e5b0 0x7fe4f419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.201+0000 7fe4fc26e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f4113d70 con 0x7fe4f41020b0 2024-04-25T10:38:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.201+0000 7fe4fc26e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f4113bf0 con 0x7fe4f4109c00 2024-04-25T10:38:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.201+0000 7fe4fc26e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f4113ef0 con 0x7fe4f410e5b0 2024-04-25T10:38:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.201+0000 7fe4fa00a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4f41020b0 0x7fe4f419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.201+0000 7fe4fa00a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4f41020b0 0x7fe4f419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46472/0 (socket says 172.21.15.43:46472) 2024-04-25T10:38:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.201+0000 7fe4fa00a640 1 -- 172.21.15.43:0/4277360162 learned_addr learned my addr 172.21.15.43:0/4277360162 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.201+0000 7fe4fa80b640 1 --2- 172.21.15.43:0/4277360162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f410e5b0 0x7fe4f419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.201+0000 7fe4f9809640 1 --2- 172.21.15.43:0/4277360162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4f4109c00 0x7fe4f419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.201+0000 7fe4fa00a640 1 -- 172.21.15.43:0/4277360162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f410e5b0 msgr2=0x7fe4f419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.201+0000 7fe4fa00a640 1 --2- 172.21.15.43:0/4277360162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f410e5b0 0x7fe4f419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.201+0000 7fe4fa00a640 1 -- 172.21.15.43:0/4277360162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4f4109c00 msgr2=0x7fe4f419acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.201+0000 7fe4fa00a640 1 --2- 172.21.15.43:0/4277360162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4f4109c00 0x7fe4f419acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.201+0000 7fe4fa00a640 1 -- 172.21.15.43:0/4277360162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f419f5b0 con 0x7fe4f41020b0 2024-04-25T10:38:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.202+0000 7fe4fa00a640 1 --2- 172.21.15.43:0/4277360162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4f41020b0 0x7fe4f419a750 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00319f0 tx=0x7fe4f0031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.202+0000 7fe4eb7fe640 1 -- 172.21.15.43:0/4277360162 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0003bf0 con 0x7fe4f41020b0 2024-04-25T10:38:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.202+0000 7fe4eb7fe640 1 -- 172.21.15.43:0/4277360162 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f0003d90 con 0x7fe4f41020b0 2024-04-25T10:38:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.202+0000 7fe4fc26e640 1 -- 172.21.15.43:0/4277360162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f419f840 con 0x7fe4f41020b0 2024-04-25T10:38:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.202+0000 7fe4eb7fe640 1 -- 172.21.15.43:0/4277360162 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f002fb70 con 0x7fe4f41020b0 2024-04-25T10:38:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.202+0000 7fe4fc26e640 1 -- 172.21.15.43:0/4277360162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f419fd90 con 0x7fe4f41020b0 2024-04-25T10:38:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.204+0000 7fe4eb7fe640 1 -- 172.21.15.43:0/4277360162 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fe4f002fd10 con 0x7fe4f41020b0 2024-04-25T10:38:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.205+0000 7fe5031ff740 1 Processor -- start 2024-04-25T10:38:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.205+0000 7fe5031ff740 1 -- start start 2024-04-25T10:38:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.205+0000 7fe5031ff740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d4dc191b0 0x557d4dfb89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.205+0000 7fe5031ff740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dbf02f0 0x557d4dfb8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.205+0000 7fe5031ff740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dbf84d0 0x557d4dfb9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.205+0000 7fe5031ff740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557d4df1f460 con 0x557d4dbf02f0 2024-04-25T10:38:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.205+0000 7fe5031ff740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557d4ded9590 con 0x557d4dc191b0 2024-04-25T10:38:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.205+0000 7fe5031ff740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557d4df1b420 con 0x557d4dbf84d0 2024-04-25T10:38:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.205+0000 7fe4fa00a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dbf02f0 0x557d4dfb8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.205+0000 7fe4fa00a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dbf02f0 0x557d4dfb8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46476/0 (socket says 172.21.15.43:46476) 2024-04-25T10:38:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.205+0000 7fe4fa00a640 1 -- 172.21.15.43:0/367666665 learned_addr learned my addr 172.21.15.43:0/367666665 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.205+0000 7fe4fa80b640 1 --2- 172.21.15.43:0/367666665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d4dc191b0 0x557d4dfb89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.206+0000 7fe4eb7fe640 1 --2- 172.21.15.43:0/4277360162 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe4e00afc50 0x7fe4e00b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.206+0000 7fe4f9809640 1 --2- 172.21.15.43:0/367666665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dbf84d0 0x557d4dfb9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.206+0000 7fe4fa80b640 1 --2- 172.21.15.43:0/4277360162 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe4e00afc50 0x7fe4e00b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.206+0000 7fe4fa00a640 1 -- 172.21.15.43:0/367666665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dbf84d0 msgr2=0x557d4dfb9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.206+0000 7fe4fa00a640 1 --2- 172.21.15.43:0/367666665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dbf84d0 0x557d4dfb9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.206+0000 7fe4fa00a640 1 -- 172.21.15.43:0/367666665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d4dc191b0 msgr2=0x557d4dfb89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.206+0000 7fe4fa00a640 1 --2- 172.21.15.43:0/367666665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d4dc191b0 0x557d4dfb89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.206+0000 7fe4fa00a640 1 -- 172.21.15.43:0/367666665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d4dee60d0 con 0x557d4dbf02f0 2024-04-25T10:38:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.206+0000 7fe4f9809640 1 --2- 172.21.15.43:0/367666665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dbf84d0 0x557d4dfb9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:38:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.206+0000 7fe4eb7fe640 1 -- 172.21.15.43:0/4277360162 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fe4f01004a0 con 0x7fe4f41020b0 2024-04-25T10:38:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.206+0000 7fe4fa00a640 1 --2- 172.21.15.43:0/367666665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dbf02f0 0x557d4dfb8ee0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00fdca0 tx=0x7fe4f00c4f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1820529783ccfd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.206+0000 7fe4fa80b640 1 --2- 172.21.15.43:0/4277360162 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe4e00afc50 0x7fe4e00b2160 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec005d00 tx=0x7fe4ec004730 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.206+0000 7fe4e8bfe640 1 -- 172.21.15.43:0/367666665 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f00c8550 con 0x557d4dbf02f0 2024-04-25T10:38:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.206+0000 7fe4e8bfe640 1 -- 172.21.15.43:0/367666665 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f00eb0c0 con 0x557d4dbf02f0 2024-04-25T10:38:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.206+0000 7fe4e8bfe640 1 -- 172.21.15.43:0/367666665 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f00c7ef0 con 0x557d4dbf02f0 2024-04-25T10:38:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.207+0000 7fe5031ff740 1 -- 172.21.15.43:0/367666665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dbf02f0 msgr2=0x557d4dfb8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.207+0000 7fe5031ff740 1 --2- 172.21.15.43:0/367666665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dbf02f0 0x557d4dfb8ee0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00fdca0 tx=0x7fe4f00c4f00 comp rx=0 tx=0).stop 2024-04-25T10:38:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.207+0000 7fe5031ff740 1 -- 172.21.15.43:0/367666665 shutdown_connections 2024-04-25T10:38:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.207+0000 7fe5031ff740 1 --2- 172.21.15.43:0/367666665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dbf84d0 0x557d4dfb9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.207+0000 7fe5031ff740 1 --2- 172.21.15.43:0/367666665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dbf02f0 0x557d4dfb8ee0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.207+0000 7fe5031ff740 1 --2- 172.21.15.43:0/367666665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d4dc191b0 0x557d4dfb89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.207+0000 7fe5031ff740 1 -- 172.21.15.43:0/367666665 >> 172.21.15.43:0/367666665 conn(0x557d4dc0d300 msgr2=0x557d4dfb52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.207+0000 7fe5031ff740 1 -- 172.21.15.43:0/367666665 shutdown_connections 2024-04-25T10:38:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.207+0000 7fe5031ff740 1 -- 172.21.15.43:0/367666665 wait complete. 2024-04-25T10:38:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.208+0000 7fe5031ff740 1 -- start start 2024-04-25T10:38:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.208+0000 7fe5031ff740 1 Processor -- start 2024-04-25T10:38:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.208+0000 7fe5031ff740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d4dbf02f0 0x557d4dfc8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.208+0000 7fe5031ff740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dbf84d0 0x557d4dfc94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.208+0000 7fe5031ff740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dc191b0 0x557d4dfc7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.208+0000 7fe5031ff740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557d4df1f460 con 0x557d4dc191b0 2024-04-25T10:38:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.208+0000 7fe5031ff740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557d4dea6b60 con 0x557d4dbf02f0 2024-04-25T10:38:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.208+0000 7fe5031ff740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557d4dfc9df0 con 0x557d4dbf84d0 2024-04-25T10:38:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.208+0000 7fe5031ff740 10 client.0 _ll_register_callbacks cb 0x557d4decb2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.208+0000 7fe4fa80b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dc191b0 0x557d4dfc7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.208+0000 7fe4fa80b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dc191b0 0x557d4dfc7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46488/0 (socket says 172.21.15.43:46488) 2024-04-25T10:38:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.208+0000 7fe4fa80b640 1 -- 172.21.15.43:0/2022616417 learned_addr learned my addr 172.21.15.43:0/2022616417 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.209+0000 7fe4fa00a640 1 --2- 172.21.15.43:0/2022616417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d4dbf02f0 0x557d4dfc8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.209+0000 7fe4f9809640 1 --2- 172.21.15.43:0/2022616417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dbf84d0 0x557d4dfc94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.209+0000 7fe4fa80b640 1 -- 172.21.15.43:0/2022616417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dbf84d0 msgr2=0x557d4dfc94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.209+0000 7fe4fa80b640 1 --2- 172.21.15.43:0/2022616417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dbf84d0 0x557d4dfc94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.209+0000 7fe4fa80b640 1 -- 172.21.15.43:0/2022616417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d4dbf02f0 msgr2=0x557d4dfc8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.209+0000 7fe4fa80b640 1 --2- 172.21.15.43:0/2022616417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d4dbf02f0 0x557d4dfc8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.209+0000 7fe4fa80b640 1 -- 172.21.15.43:0/2022616417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d4dc31210 con 0x557d4dc191b0 2024-04-25T10:38:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.209+0000 7fe4fa80b640 1 --2- 172.21.15.43:0/2022616417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dc191b0 0x557d4dfc7420 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec012450 tx=0x7fe4ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2298c1e22b994cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.209+0000 7fe4ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.209+0000 7fe4ddffb640 1 -- 172.21.15.43:0/2022616417 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec025020 con 0x557d4dc191b0 2024-04-25T10:38:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.209+0000 7fe5031ff740 1 -- 172.21.15.43:0/2022616417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557d4df2d070 con 0x557d4dc191b0 2024-04-25T10:38:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.209+0000 7fe4ddffb640 1 -- 172.21.15.43:0/2022616417 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fe4ec021950 con 0x557d4dc191b0 2024-04-25T10:38:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.209+0000 7fe4ddffb640 1 -- 172.21.15.43:0/2022616417 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4ec030050 con 0x557d4dc191b0 2024-04-25T10:38:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.209+0000 7fe5031ff740 10 client.17298 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.209+0000 7fe5031ff740 1 -- 172.21.15.43:0/2022616417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557d4dcfd620 con 0x557d4dc191b0 2024-04-25T10:38:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.210+0000 7fe4ddffb640 1 -- 172.21.15.43:0/2022616417 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557d4df2d070 con 0x557d4dc191b0 2024-04-25T10:38:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.210+0000 7fe4ddffb640 1 -- 172.21.15.43:0/2022616417 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe4ec01e070 con 0x557d4dc191b0 2024-04-25T10:38:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.210+0000 7fe5031ff740 10 client.17298 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.210+0000 7fe5031ff740 10 client.17298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.210+0000 7fe5031ff740 20 client.17298 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.210+0000 7fe5031ff740 1 --2- 172.21.15.43:0/2022616417 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557d4dbd7d80 0x557d4dfd9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.210+0000 7fe5031ff740 4 client.17298 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:38:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.210+0000 7fe5031ff740 1 -- 172.21.15.43:0/2022616417 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d4df26ee0 con 0x557d4dbd7d80 2024-04-25T10:38:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.210+0000 7fe4fa00a640 1 --2- 172.21.15.43:0/2022616417 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557d4dbd7d80 0x557d4dfd9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.211+0000 7fe4fa00a640 1 --2- 172.21.15.43:0/2022616417 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557d4dbd7d80 0x557d4dfd9780 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=777005e6d968ac88 server_cookie=c7c4c7d5f127a343 in_seq=0 out_seq=0 2024-04-25T10:38:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.211+0000 7fe4ddffb640 10 client.17298 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.211+0000 7fe4ddffb640 1 -- 172.21.15.43:0/2022616417 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557d4df26ee0 con 0x557d4dbd7d80 2024-04-25T10:38:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.211+0000 7fe4ddffb640 10 client.17298 handle_command_reply: tid=0 2024-04-25T10:38:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.211+0000 7fe5031ff740 1 client.17298 shutdown 2024-04-25T10:38:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.212+0000 7fe5031ff740 1 -- 172.21.15.43:0/2022616417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dc191b0 msgr2=0x557d4dfc7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.212+0000 7fe5031ff740 1 --2- 172.21.15.43:0/2022616417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dc191b0 0x557d4dfc7420 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec012450 tx=0x7fe4ec0046c0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.212+0000 7fe5031ff740 1 -- 172.21.15.43:0/2022616417 shutdown_connections 2024-04-25T10:38:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.212+0000 7fe5031ff740 1 --2- 172.21.15.43:0/2022616417 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557d4dbd7d80 0x557d4dfd9780 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.212+0000 7fe5031ff740 1 --2- 172.21.15.43:0/2022616417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dc191b0 0x557d4dfc7420 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.212+0000 7fe5031ff740 1 --2- 172.21.15.43:0/2022616417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dbf84d0 0x557d4dfc94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.212+0000 7fe4ddffb640 0 client.17298 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.212+0000 7fe5031ff740 1 --2- 172.21.15.43:0/2022616417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d4dbf02f0 0x557d4dfc8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.212+0000 7fe5031ff740 1 -- 172.21.15.43:0/2022616417 >> 172.21.15.43:0/2022616417 conn(0x557d4dc0d300 msgr2=0x557d4dfba7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.212+0000 7fe5031ff740 1 -- 172.21.15.43:0/2022616417 shutdown_connections 2024-04-25T10:38:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.212+0000 7fe5031ff740 1 -- 172.21.15.43:0/2022616417 wait complete. 2024-04-25T10:38:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.212+0000 7fe5031ff740 20 client.17298 trim_cache size 0 max 16384 2024-04-25T10:38:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.227+0000 7fe5031ff740 1 Processor -- start 2024-04-25T10:38:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.227+0000 7fe5031ff740 1 -- start start 2024-04-25T10:38:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.227+0000 7fe5031ff740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dbf02f0 0x557d4dfc0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.227+0000 7fe5031ff740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d4dbf84d0 0x557d4dfc1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.227+0000 7fe5031ff740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dc191b0 0x557d4dfdef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.227+0000 7fe5031ff740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f41b0080 con 0x557d4dc191b0 2024-04-25T10:38:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.227+0000 7fe5031ff740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f41ad950 con 0x557d4dbf84d0 2024-04-25T10:38:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.227+0000 7fe5031ff740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f41ab7c0 con 0x557d4dbf02f0 2024-04-25T10:38:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.228+0000 7fe4fa80b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dc191b0 0x557d4dfdef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.228+0000 7fe4fa80b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dc191b0 0x557d4dfdef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46496/0 (socket says 172.21.15.43:46496) 2024-04-25T10:38:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.228+0000 7fe4fa80b640 1 -- 172.21.15.43:0/4151167584 learned_addr learned my addr 172.21.15.43:0/4151167584 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.228+0000 7fe4fa00a640 1 --2- 172.21.15.43:0/4151167584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dbf02f0 0x557d4dfc0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.228+0000 7fe4fa80b640 1 -- 172.21.15.43:0/4151167584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dbf02f0 msgr2=0x557d4dfc0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.228+0000 7fe4fa80b640 1 --2- 172.21.15.43:0/4151167584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dbf02f0 0x557d4dfc0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.228+0000 7fe4fa80b640 1 -- 172.21.15.43:0/4151167584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d4dbf84d0 msgr2=0x557d4dfc1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.228+0000 7fe4fa80b640 1 --2- 172.21.15.43:0/4151167584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d4dbf84d0 0x557d4dfc1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.228+0000 7fe4fa80b640 1 -- 172.21.15.43:0/4151167584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d4dfb0a60 con 0x557d4dc191b0 2024-04-25T10:38:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.228+0000 7fe4fa80b640 1 --2- 172.21.15.43:0/4151167584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dc191b0 0x557d4dfdef30 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec010cf0 tx=0x7fe4ec010ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e25e9e5bda6bf79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.228+0000 7fe4dffff640 1 -- 172.21.15.43:0/4151167584 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec025070 con 0x557d4dc191b0 2024-04-25T10:38:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.229+0000 7fe4dffff640 1 -- 172.21.15.43:0/4151167584 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4ec012ce0 con 0x557d4dc191b0 2024-04-25T10:38:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.229+0000 7fe4dffff640 1 -- 172.21.15.43:0/4151167584 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0108b0 con 0x557d4dc191b0 2024-04-25T10:38:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.229+0000 7fe5031ff740 1 -- 172.21.15.43:0/4151167584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dc191b0 msgr2=0x557d4dfdef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.229+0000 7fe5031ff740 1 --2- 172.21.15.43:0/4151167584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dc191b0 0x557d4dfdef30 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec010cf0 tx=0x7fe4ec010ee0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.229+0000 7fe5031ff740 1 -- 172.21.15.43:0/4151167584 shutdown_connections 2024-04-25T10:38:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.229+0000 7fe5031ff740 1 --2- 172.21.15.43:0/4151167584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d4dbf84d0 0x557d4dfc1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.229+0000 7fe5031ff740 1 --2- 172.21.15.43:0/4151167584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dc191b0 0x557d4dfdef30 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.229+0000 7fe5031ff740 1 --2- 172.21.15.43:0/4151167584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dbf02f0 0x557d4dfc0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.229+0000 7fe5031ff740 1 -- 172.21.15.43:0/4151167584 >> 172.21.15.43:0/4151167584 conn(0x557d4dc0d300 msgr2=0x557d4dfc5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.229+0000 7fe5031ff740 1 -- 172.21.15.43:0/4151167584 shutdown_connections 2024-04-25T10:38:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.229+0000 7fe5031ff740 1 -- 172.21.15.43:0/4151167584 wait complete. 2024-04-25T10:38:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.230+0000 7fe5031ff740 1 -- start start 2024-04-25T10:38:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.230+0000 7fe5031ff740 1 Processor -- start 2024-04-25T10:38:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.230+0000 7fe5031ff740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dbf02f0 0x557d4dfccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.230+0000 7fe5031ff740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d4dbf84d0 0x557d4dfcd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.230+0000 7fe5031ff740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dc191b0 0x557d4dfcd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.230+0000 7fe5031ff740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f41abe10 con 0x557d4dbf02f0 2024-04-25T10:38:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.230+0000 7fe5031ff740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f41ad280 con 0x557d4dbf84d0 2024-04-25T10:38:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.230+0000 7fe5031ff740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f41ac4e0 con 0x557d4dc191b0 2024-04-25T10:38:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.230+0000 7fe5031ff740 10 client.0 _ll_register_callbacks cb 0x7fe4ec00b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:38:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.230+0000 7fe4fa00a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dbf02f0 0x557d4dfccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.230+0000 7fe4fa00a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dbf02f0 0x557d4dfccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46508/0 (socket says 172.21.15.43:46508) 2024-04-25T10:38:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.230+0000 7fe4fa00a640 1 -- 172.21.15.43:0/1035600618 learned_addr learned my addr 172.21.15.43:0/1035600618 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:38:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.230+0000 7fe4f9809640 1 --2- 172.21.15.43:0/1035600618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d4dbf84d0 0x557d4dfcd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.231+0000 7fe4fa80b640 1 --2- 172.21.15.43:0/1035600618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dc191b0 0x557d4dfcd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.231+0000 7fe4fa00a640 1 -- 172.21.15.43:0/1035600618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dc191b0 msgr2=0x557d4dfcd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.231+0000 7fe4fa00a640 1 --2- 172.21.15.43:0/1035600618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dc191b0 0x557d4dfcd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.231+0000 7fe4fa00a640 1 -- 172.21.15.43:0/1035600618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d4dbf84d0 msgr2=0x557d4dfcd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:38:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.231+0000 7fe4fa00a640 1 --2- 172.21.15.43:0/1035600618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d4dbf84d0 0x557d4dfcd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.231+0000 7fe4fa00a640 1 -- 172.21.15.43:0/1035600618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d4e040c70 con 0x557d4dbf02f0 2024-04-25T10:38:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.231+0000 7fe4f9809640 1 --2- 172.21.15.43:0/1035600618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d4dbf84d0 0x557d4dfcd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:38:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.231+0000 7fe4fa00a640 1 --2- 172.21.15.43:0/1035600618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dbf02f0 0x557d4dfccd50 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00fe220 tx=0x7fe4f00c83a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2d5c171b3f7382 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:38:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.231+0000 7fe4ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:38:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.231+0000 7fe4ddffb640 1 -- 172.21.15.43:0/1035600618 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f00c80d0 con 0x557d4dbf02f0 2024-04-25T10:38:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.231+0000 7fe4ddffb640 1 -- 172.21.15.43:0/1035600618 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fe4f00b10b0 con 0x557d4dbf02f0 2024-04-25T10:38:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.231+0000 7fe5031ff740 1 -- 172.21.15.43:0/1035600618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4ec012ce0 con 0x557d4dbf02f0 2024-04-25T10:38:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.231+0000 7fe4ddffb640 1 -- 172.21.15.43:0/1035600618 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4f00c8550 con 0x557d4dbf02f0 2024-04-25T10:38:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.231+0000 7fe5031ff740 10 client.17304 fetch_fsmap learned FSMap version 26 2024-04-25T10:38:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.231+0000 7fe5031ff740 1 -- 172.21.15.43:0/1035600618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557d4e041ee0 con 0x557d4dbf02f0 2024-04-25T10:38:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.231+0000 7fe4ddffb640 1 -- 172.21.15.43:0/1035600618 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4ec012ce0 con 0x557d4dbf02f0 2024-04-25T10:38:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.232+0000 7fe4ddffb640 1 -- 172.21.15.43:0/1035600618 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe4f00afbd0 con 0x557d4dbf02f0 2024-04-25T10:38:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.232+0000 7fe5031ff740 10 client.17304 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:38:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.232+0000 7fe5031ff740 10 client.17304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:38:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.232+0000 7fe5031ff740 20 client.17304 populate_metadata read hostname 'smithi043' 2024-04-25T10:38:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.232+0000 7fe5031ff740 1 --2- 172.21.15.43:0/1035600618 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557d4dbad1c0 0x557d4e0445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:38:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.232+0000 7fe5031ff740 4 client.17304 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:38:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.232+0000 7fe5031ff740 1 -- 172.21.15.43:0/1035600618 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557d4dea25f0 con 0x557d4dbad1c0 2024-04-25T10:38:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.232+0000 7fe4f9809640 1 --2- 172.21.15.43:0/1035600618 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557d4dbad1c0 0x557d4e0445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:38:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.233+0000 7fe4f9809640 1 --2- 172.21.15.43:0/1035600618 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557d4dbad1c0 0x557d4e0445e0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=195193330fe8015d server_cookie=2093cba00bec60f7 in_seq=0 out_seq=0 2024-04-25T10:38:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.233+0000 7fe4ddffb640 10 client.17304 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.247+0000 7fe4ddffb640 1 -- 172.21.15.43:0/1035600618 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557d4dea25f0 con 0x557d4dbad1c0 2024-04-25T10:38:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.247+0000 7fe4ddffb640 10 client.17304 handle_command_reply: tid=0 2024-04-25T10:38:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.247+0000 7fe5031ff740 1 client.17304 shutdown 2024-04-25T10:38:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.247+0000 7fe5031ff740 1 -- 172.21.15.43:0/1035600618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dbf02f0 msgr2=0x557d4dfccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.247+0000 7fe5031ff740 1 --2- 172.21.15.43:0/1035600618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dbf02f0 0x557d4dfccd50 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00fe220 tx=0x7fe4f00c83a0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.247+0000 7fe5031ff740 1 -- 172.21.15.43:0/1035600618 shutdown_connections 2024-04-25T10:38:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.247+0000 7fe5031ff740 1 --2- 172.21.15.43:0/1035600618 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557d4dbad1c0 0x557d4e0445e0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.247+0000 7fe5031ff740 1 --2- 172.21.15.43:0/1035600618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d4dbf84d0 0x557d4dfcd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.247+0000 7fe5031ff740 1 --2- 172.21.15.43:0/1035600618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d4dc191b0 0x557d4dfcd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.247+0000 7fe4ddffb640 0 client.17304 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:38:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.247+0000 7fe5031ff740 1 --2- 172.21.15.43:0/1035600618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d4dbf02f0 0x557d4dfccd50 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.247+0000 7fe5031ff740 1 -- 172.21.15.43:0/1035600618 >> 172.21.15.43:0/1035600618 conn(0x557d4dc0d300 msgr2=0x557d4dfb8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.247+0000 7fe5031ff740 1 -- 172.21.15.43:0/1035600618 shutdown_connections 2024-04-25T10:38:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.247+0000 7fe5031ff740 1 -- 172.21.15.43:0/1035600618 wait complete. 2024-04-25T10:38:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.247+0000 7fe5031ff740 20 client.17304 trim_cache size 0 max 16384 2024-04-25T10:38:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:38:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:38:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:38:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:38:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.248+0000 7fe4fc26e640 1 -- 172.21.15.43:0/4277360162 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe4e00afc50 msgr2=0x7fe4e00b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.248+0000 7fe4fc26e640 1 --2- 172.21.15.43:0/4277360162 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe4e00afc50 0x7fe4e00b2160 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec005d00 tx=0x7fe4ec004730 comp rx=0 tx=0).stop 2024-04-25T10:38:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.248+0000 7fe4fc26e640 1 -- 172.21.15.43:0/4277360162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4f41020b0 msgr2=0x7fe4f419a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:38:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.248+0000 7fe4fc26e640 1 --2- 172.21.15.43:0/4277360162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4f41020b0 0x7fe4f419a750 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00319f0 tx=0x7fe4f0031a20 comp rx=0 tx=0).stop 2024-04-25T10:38:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.248+0000 7fe4fc26e640 1 -- 172.21.15.43:0/4277360162 shutdown_connections 2024-04-25T10:38:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.248+0000 7fe4fc26e640 1 --2- 172.21.15.43:0/4277360162 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe4e00afc50 0x7fe4e00b2160 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.248+0000 7fe4fc26e640 1 --2- 172.21.15.43:0/4277360162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4f4109c00 0x7fe4f419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.248+0000 7fe4fc26e640 1 --2- 172.21.15.43:0/4277360162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f410e5b0 0x7fe4f419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.248+0000 7fe4fc26e640 1 --2- 172.21.15.43:0/4277360162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4f41020b0 0x7fe4f419a750 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:38:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.248+0000 7fe4fc26e640 1 -- 172.21.15.43:0/4277360162 >> 172.21.15.43:0/4277360162 conn(0x7fe4f40fd8a0 msgr2=0x7fe4f40fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:38:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.248+0000 7fe4fc26e640 1 -- 172.21.15.43:0/4277360162 shutdown_connections 2024-04-25T10:38:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:38:52.248+0000 7fe4fc26e640 1 -- 172.21.15.43:0/4277360162 wait complete. 2024-04-25T10:38:52.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:52 smithi077 ceph-mon[32772]: pgmap v193: 97 pgs: 97 active+clean; 5.4 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-25T10:38:52.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:52 smithi164 ceph-mon[32713]: pgmap v193: 97 pgs: 97 active+clean; 5.4 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-25T10:38:52.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:52 smithi043 ceph-mon[29478]: pgmap v193: 97 pgs: 97 active+clean; 5.4 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-25T10:38:54.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:54 smithi077 ceph-mon[32772]: pgmap v194: 97 pgs: 97 active+clean; 5.4 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-25T10:38:54.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:54 smithi164 ceph-mon[32713]: pgmap v194: 97 pgs: 97 active+clean; 5.4 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-25T10:38:54.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:54 smithi043 ceph-mon[29478]: pgmap v194: 97 pgs: 97 active+clean; 5.4 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-04-25T10:38:56.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:56 smithi077 ceph-mon[32772]: pgmap v195: 97 pgs: 97 active+clean; 1.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 61 KiB/s wr, 20 op/s 2024-04-25T10:38:56.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:56 smithi164 ceph-mon[32713]: pgmap v195: 97 pgs: 97 active+clean; 1.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 61 KiB/s wr, 20 op/s 2024-04-25T10:38:56.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:56 smithi043 ceph-mon[29478]: pgmap v195: 97 pgs: 97 active+clean; 1.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 61 KiB/s wr, 20 op/s 2024-04-25T10:38:58.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:38:58 smithi077 ceph-mon[32772]: pgmap v196: 97 pgs: 97 active+clean; 1.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-25T10:38:58.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:38:58 smithi164 ceph-mon[32713]: pgmap v196: 97 pgs: 97 active+clean; 1.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-25T10:38:58.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:38:58 smithi043 ceph-mon[29478]: pgmap v196: 97 pgs: 97 active+clean; 1.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-25T10:39:00.294 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:00 smithi077 ceph-mon[32772]: pgmap v197: 97 pgs: 97 active+clean; 1.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-04-25T10:39:00.295 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:00 smithi164 ceph-mon[32713]: pgmap v197: 97 pgs: 97 active+clean; 1.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-04-25T10:39:00.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:00 smithi043 ceph-mon[29478]: pgmap v197: 97 pgs: 97 active+clean; 1.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-04-25T10:39:02.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:02 smithi077 ceph-mon[32772]: pgmap v198: 97 pgs: 97 active+clean; 1.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-25T10:39:02.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:02 smithi164 ceph-mon[32713]: pgmap v198: 97 pgs: 97 active+clean; 1.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-25T10:39:02.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:02 smithi043 ceph-mon[29478]: pgmap v198: 97 pgs: 97 active+clean; 1.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-25T10:39:04.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:04 smithi077 ceph-mon[32772]: pgmap v199: 97 pgs: 97 active+clean; 1.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-25T10:39:04.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:04 smithi164 ceph-mon[32713]: pgmap v199: 97 pgs: 97 active+clean; 1.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-25T10:39:04.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:04 smithi043 ceph-mon[29478]: pgmap v199: 97 pgs: 97 active+clean; 1.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-25T10:39:05.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:05 smithi077 ceph-mon[32772]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T10:39:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:05 smithi164 ceph-mon[32713]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T10:39:05.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:05 smithi043 ceph-mon[29478]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T10:39:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.360+0000 7f192fee0640 1 -- 172.21.15.43:0/1310457942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1928109ff0 msgr2=0x7f1928103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.360+0000 7f192fee0640 1 --2- 172.21.15.43:0/1310457942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1928109ff0 0x7f1928103c30 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f1918009900 tx=0x7f191802a410 comp rx=0 tx=0).stop 2024-04-25T10:39:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.361+0000 7f192fee0640 1 -- 172.21.15.43:0/1310457942 shutdown_connections 2024-04-25T10:39:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.361+0000 7f192fee0640 1 --2- 172.21.15.43:0/1310457942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19281042a0 0x7f1928110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.361+0000 7f192fee0640 1 --2- 172.21.15.43:0/1310457942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1928109ff0 0x7f1928103c30 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.361+0000 7f192fee0640 1 --2- 172.21.15.43:0/1310457942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1928109b50 0x7f19281036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.361+0000 7f192fee0640 1 -- 172.21.15.43:0/1310457942 >> 172.21.15.43:0/1310457942 conn(0x7f19280ff230 msgr2=0x7f1928101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.361+0000 7f192fee0640 1 -- 172.21.15.43:0/1310457942 shutdown_connections 2024-04-25T10:39:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.361+0000 7f192fee0640 1 -- 172.21.15.43:0/1310457942 wait complete. 2024-04-25T10:39:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.361+0000 7f192fee0640 1 Processor -- start 2024-04-25T10:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.361+0000 7f192fee0640 1 -- start start 2024-04-25T10:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.362+0000 7f192fee0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19281042a0 0x7f192819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.362+0000 7f192fee0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1928109b50 0x7f192819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.362+0000 7f192fee0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1928109ff0 0x7f192819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.362+0000 7f192fee0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1928113d90 con 0x7f1928109ff0 2024-04-25T10:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.362+0000 7f192fee0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1928113c10 con 0x7f19281042a0 2024-04-25T10:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.362+0000 7f192fee0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1928113f10 con 0x7f1928109b50 2024-04-25T10:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.362+0000 7f192e47d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1928109ff0 0x7f192819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.362+0000 7f192e47d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1928109ff0 0x7f192819eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49412/0 (socket says 172.21.15.43:49412) 2024-04-25T10:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.362+0000 7f192e47d640 1 -- 172.21.15.43:0/2622365729 learned_addr learned my addr 172.21.15.43:0/2622365729 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.362+0000 7f192dc7c640 1 --2- 172.21.15.43:0/2622365729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19281042a0 0x7f192819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.362+0000 7f192d47b640 1 --2- 172.21.15.43:0/2622365729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1928109b50 0x7f192819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.362+0000 7f192e47d640 1 -- 172.21.15.43:0/2622365729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1928109b50 msgr2=0x7f192819ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.362+0000 7f192e47d640 1 --2- 172.21.15.43:0/2622365729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1928109b50 0x7f192819ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.362+0000 7f192e47d640 1 -- 172.21.15.43:0/2622365729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19281042a0 msgr2=0x7f192819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.362+0000 7f192e47d640 1 --2- 172.21.15.43:0/2622365729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19281042a0 0x7f192819a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.362+0000 7f192e47d640 1 -- 172.21.15.43:0/2622365729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f192819f630 con 0x7f1928109ff0 2024-04-25T10:39:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.363+0000 7f192e47d640 1 --2- 172.21.15.43:0/2622365729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1928109ff0 0x7f192819eff0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f192000e970 tx=0x7f192000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.363+0000 7f191effd640 1 -- 172.21.15.43:0/2622365729 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192000ccc0 con 0x7f1928109ff0 2024-04-25T10:39:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.363+0000 7f191effd640 1 -- 172.21.15.43:0/2622365729 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f192000ce60 con 0x7f1928109ff0 2024-04-25T10:39:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.363+0000 7f192fee0640 1 -- 172.21.15.43:0/2622365729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f192819f920 con 0x7f1928109ff0 2024-04-25T10:39:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.363+0000 7f191effd640 1 -- 172.21.15.43:0/2622365729 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1920010430 con 0x7f1928109ff0 2024-04-25T10:39:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.363+0000 7f192fee0640 1 -- 172.21.15.43:0/2622365729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19281a72a0 con 0x7f1928109ff0 2024-04-25T10:39:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.366+0000 7f1936d90740 1 Processor -- start 2024-04-25T10:39:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.366+0000 7f191effd640 1 -- 172.21.15.43:0/2622365729 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f1920004f90 con 0x7f1928109ff0 2024-04-25T10:39:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.366+0000 7f1936d90740 1 -- start start 2024-04-25T10:39:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.366+0000 7f1936d90740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968eae1b0 0x56296924d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.366+0000 7f1936d90740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562968e852f0 0x56296924dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.366+0000 7f1936d90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562968e8d4d0 0x56296924e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.366+0000 7f1936d90740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5629691b4460 con 0x562968eae1b0 2024-04-25T10:39:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.366+0000 7f1936d90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56296916e590 con 0x562968e8d4d0 2024-04-25T10:39:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.366+0000 7f1936d90740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5629691b0420 con 0x562968e852f0 2024-04-25T10:39:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.366+0000 7f192d47b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968eae1b0 0x56296924d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.366+0000 7f192d47b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968eae1b0 0x56296924d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49428/0 (socket says 172.21.15.43:49428) 2024-04-25T10:39:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.366+0000 7f192d47b640 1 -- 172.21.15.43:0/3892898428 learned_addr learned my addr 172.21.15.43:0/3892898428 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.366+0000 7f192dc7c640 1 --2- 172.21.15.43:0/3892898428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562968e852f0 0x56296924dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.367+0000 7f191effd640 1 --2- 172.21.15.43:0/2622365729 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f19140afc50 0x7f19140b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.367+0000 7f192d47b640 1 --2- 172.21.15.43:0/3892898428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562968e8d4d0 0x56296924e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.367+0000 7f192e47d640 1 --2- 172.21.15.43:0/2622365729 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f19140afc50 0x7f19140b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.367+0000 7f192d47b640 1 -- 172.21.15.43:0/3892898428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562968e852f0 msgr2=0x56296924dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.367+0000 7f192d47b640 1 --2- 172.21.15.43:0/3892898428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562968e852f0 0x56296924dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.367+0000 7f192d47b640 1 -- 172.21.15.43:0/3892898428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562968e8d4d0 msgr2=0x56296924e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.367+0000 7f192d47b640 1 --2- 172.21.15.43:0/3892898428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562968e8d4d0 0x56296924e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.367+0000 7f192d47b640 1 -- 172.21.15.43:0/3892898428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56296917b0d0 con 0x562968eae1b0 2024-04-25T10:39:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.367+0000 7f191effd640 1 -- 172.21.15.43:0/2622365729 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7f19200df4d0 con 0x7f1928109ff0 2024-04-25T10:39:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.367+0000 7f192d47b640 1 --2- 172.21.15.43:0/3892898428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968eae1b0 0x56296924d9a0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f1918005ab0 tx=0x7f1918009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc1d3f9ff83a732 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.367+0000 7f1913fff640 1 -- 172.21.15.43:0/3892898428 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918043020 con 0x562968eae1b0 2024-04-25T10:39:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.367+0000 7f1913fff640 1 -- 172.21.15.43:0/3892898428 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19180046c0 con 0x562968eae1b0 2024-04-25T10:39:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.367+0000 7f1913fff640 1 -- 172.21.15.43:0/3892898428 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918035070 con 0x562968eae1b0 2024-04-25T10:39:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.367+0000 7f192e47d640 1 --2- 172.21.15.43:0/2622365729 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f19140afc50 0x7f19140b2160 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f19200a36d0 tx=0x7f1920076150 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.368+0000 7f1936d90740 1 -- 172.21.15.43:0/3892898428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968eae1b0 msgr2=0x56296924d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.368+0000 7f1936d90740 1 --2- 172.21.15.43:0/3892898428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968eae1b0 0x56296924d9a0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f1918005ab0 tx=0x7f1918009560 comp rx=0 tx=0).stop 2024-04-25T10:39:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.368+0000 7f1936d90740 1 -- 172.21.15.43:0/3892898428 shutdown_connections 2024-04-25T10:39:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.368+0000 7f1936d90740 1 --2- 172.21.15.43:0/3892898428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562968e8d4d0 0x56296924e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.368+0000 7f1936d90740 1 --2- 172.21.15.43:0/3892898428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562968e852f0 0x56296924dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.368+0000 7f1936d90740 1 --2- 172.21.15.43:0/3892898428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968eae1b0 0x56296924d9a0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.368+0000 7f1936d90740 1 -- 172.21.15.43:0/3892898428 >> 172.21.15.43:0/3892898428 conn(0x562968ea2300 msgr2=0x56296924a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.368+0000 7f1936d90740 1 -- 172.21.15.43:0/3892898428 shutdown_connections 2024-04-25T10:39:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.368+0000 7f1936d90740 1 -- 172.21.15.43:0/3892898428 wait complete. 2024-04-25T10:39:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.369+0000 7f1936d90740 1 -- start start 2024-04-25T10:39:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.369+0000 7f1936d90740 1 Processor -- start 2024-04-25T10:39:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.369+0000 7f1936d90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562968e852f0 0x56296925df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.369+0000 7f1936d90740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562968e8d4d0 0x56296925e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.369+0000 7f1936d90740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968eae1b0 0x56296925c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.369+0000 7f1936d90740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56296925efd0 con 0x562968eae1b0 2024-04-25T10:39:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.369+0000 7f1936d90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56296913bb60 con 0x562968e852f0 2024-04-25T10:39:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.369+0000 7f1936d90740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56296925f150 con 0x562968e8d4d0 2024-04-25T10:39:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.369+0000 7f1936d90740 10 client.0 _ll_register_callbacks cb 0x5629691602b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.370+0000 7f192d47b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968eae1b0 0x56296925c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.370+0000 7f192d47b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968eae1b0 0x56296925c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49442/0 (socket says 172.21.15.43:49442) 2024-04-25T10:39:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.370+0000 7f192d47b640 1 -- 172.21.15.43:0/4115944011 learned_addr learned my addr 172.21.15.43:0/4115944011 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.370+0000 7f192d47b640 1 --2- 172.21.15.43:0/4115944011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562968e852f0 0x56296925df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.370+0000 7f192dc7c640 1 --2- 172.21.15.43:0/4115944011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562968e8d4d0 0x56296925e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.370+0000 7f192d47b640 1 -- 172.21.15.43:0/4115944011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562968e8d4d0 msgr2=0x56296925e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.370+0000 7f192d47b640 1 --2- 172.21.15.43:0/4115944011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562968e8d4d0 0x56296925e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.370+0000 7f192d47b640 1 -- 172.21.15.43:0/4115944011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562968e852f0 msgr2=0x56296925df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.370+0000 7f192d47b640 1 --2- 172.21.15.43:0/4115944011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562968e852f0 0x56296925df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.370+0000 7f192d47b640 1 -- 172.21.15.43:0/4115944011 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562968ec6210 con 0x562968eae1b0 2024-04-25T10:39:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.370+0000 7f192d47b640 1 --2- 172.21.15.43:0/4115944011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968eae1b0 0x56296925c400 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x56296925bc60 tx=0x7f1918005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4488730aa250ed6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.370+0000 7f19117fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.370+0000 7f19117fa640 1 -- 172.21.15.43:0/4115944011 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191804c070 con 0x562968eae1b0 2024-04-25T10:39:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.370+0000 7f19117fa640 1 -- 172.21.15.43:0/4115944011 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7f19180035f0 con 0x562968eae1b0 2024-04-25T10:39:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.370+0000 7f1936d90740 1 -- 172.21.15.43:0/4115944011 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19180046c0 con 0x562968eae1b0 2024-04-25T10:39:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.371+0000 7f19117fa640 1 -- 172.21.15.43:0/4115944011 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1918035070 con 0x562968eae1b0 2024-04-25T10:39:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.371+0000 7f1936d90740 10 client.17316 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.371+0000 7f1936d90740 1 -- 172.21.15.43:0/4115944011 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562968f92620 con 0x562968eae1b0 2024-04-25T10:39:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.371+0000 7f19117fa640 1 -- 172.21.15.43:0/4115944011 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19180046c0 con 0x562968eae1b0 2024-04-25T10:39:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.371+0000 7f19117fa640 1 -- 172.21.15.43:0/4115944011 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1918031810 con 0x562968eae1b0 2024-04-25T10:39:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.371+0000 7f1936d90740 10 client.17316 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.371+0000 7f1936d90740 10 client.17316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.371+0000 7f1936d90740 20 client.17316 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.371+0000 7f1936d90740 1 --2- 172.21.15.43:0/4115944011 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562968e6cd80 0x56296926e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.371+0000 7f1936d90740 4 client.17316 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:39:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.371+0000 7f1936d90740 1 -- 172.21.15.43:0/4115944011 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629691c2070 con 0x562968e6cd80 2024-04-25T10:39:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.371+0000 7f192dc7c640 1 --2- 172.21.15.43:0/4115944011 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562968e6cd80 0x56296926e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.372+0000 7f192dc7c640 1 --2- 172.21.15.43:0/4115944011 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562968e6cd80 0x56296926e960 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f96ca69cb2d2fe2 server_cookie=c9aa37ec65dd00 in_seq=0 out_seq=0 2024-04-25T10:39:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.372+0000 7f19117fa640 10 client.17316 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.373+0000 7f19117fa640 1 -- 172.21.15.43:0/4115944011 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5629691c2070 con 0x562968e6cd80 2024-04-25T10:39:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.373+0000 7f19117fa640 10 client.17316 handle_command_reply: tid=0 2024-04-25T10:39:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.373+0000 7f1936d90740 1 client.17316 shutdown 2024-04-25T10:39:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.373+0000 7f1936d90740 1 -- 172.21.15.43:0/4115944011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968eae1b0 msgr2=0x56296925c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.373+0000 7f1936d90740 1 --2- 172.21.15.43:0/4115944011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968eae1b0 0x56296925c400 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x56296925bc60 tx=0x7f1918005f00 comp rx=0 tx=0).stop 2024-04-25T10:39:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.373+0000 7f1936d90740 1 -- 172.21.15.43:0/4115944011 shutdown_connections 2024-04-25T10:39:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.373+0000 7f1936d90740 1 --2- 172.21.15.43:0/4115944011 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562968e6cd80 0x56296926e960 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.373+0000 7f1936d90740 1 --2- 172.21.15.43:0/4115944011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968eae1b0 0x56296925c400 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.373+0000 7f1936d90740 1 --2- 172.21.15.43:0/4115944011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562968e8d4d0 0x56296925e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.373+0000 7f1936d90740 1 --2- 172.21.15.43:0/4115944011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562968e852f0 0x56296925df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.373+0000 7f19117fa640 0 client.17316 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.373+0000 7f1936d90740 1 -- 172.21.15.43:0/4115944011 >> 172.21.15.43:0/4115944011 conn(0x562968ea2300 msgr2=0x56296924f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.373+0000 7f1936d90740 1 -- 172.21.15.43:0/4115944011 shutdown_connections 2024-04-25T10:39:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.373+0000 7f1936d90740 1 -- 172.21.15.43:0/4115944011 wait complete. 2024-04-25T10:39:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.373+0000 7f1936d90740 20 client.17316 trim_cache size 0 max 16384 2024-04-25T10:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.389+0000 7f1936d90740 1 Processor -- start 2024-04-25T10:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.389+0000 7f1936d90740 1 -- start start 2024-04-25T10:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.389+0000 7f1936d90740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968e852f0 0x56296924feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.389+0000 7f1936d90740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562968e8d4d0 0x562969271990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.389+0000 7f1936d90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562968eae1b0 0x562969271ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.389+0000 7f1936d90740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f19281b0160 con 0x562968e852f0 2024-04-25T10:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.389+0000 7f1936d90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f19281ada30 con 0x562968eae1b0 2024-04-25T10:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.389+0000 7f1936d90740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f19281ab8a0 con 0x562968e8d4d0 2024-04-25T10:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.389+0000 7f192d47b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968e852f0 0x56296924feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.389+0000 7f192d47b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968e852f0 0x56296924feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49446/0 (socket says 172.21.15.43:49446) 2024-04-25T10:39:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.389+0000 7f192d47b640 1 -- 172.21.15.43:0/3851022875 learned_addr learned my addr 172.21.15.43:0/3851022875 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.389+0000 7f192d47b640 1 -- 172.21.15.43:0/3851022875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562968e8d4d0 msgr2=0x562969271990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.389+0000 7f192d47b640 1 --2- 172.21.15.43:0/3851022875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562968e8d4d0 0x562969271990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.389+0000 7f192d47b640 1 -- 172.21.15.43:0/3851022875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562968eae1b0 msgr2=0x562969271ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.389+0000 7f192d47b640 1 --2- 172.21.15.43:0/3851022875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562968eae1b0 0x562969271ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.389+0000 7f192d47b640 1 -- 172.21.15.43:0/3851022875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562969250690 con 0x562968e852f0 2024-04-25T10:39:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.389+0000 7f192d47b640 1 --2- 172.21.15.43:0/3851022875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968e852f0 0x56296924feb0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1918001eb0 tx=0x7f1918009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bdf0b74b15793a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.390+0000 7f19137fe640 1 -- 172.21.15.43:0/3851022875 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191804c070 con 0x562968e852f0 2024-04-25T10:39:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.390+0000 7f19137fe640 1 -- 172.21.15.43:0/3851022875 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1918009520 con 0x562968e852f0 2024-04-25T10:39:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.390+0000 7f19137fe640 1 -- 172.21.15.43:0/3851022875 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918043030 con 0x562968e852f0 2024-04-25T10:39:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.390+0000 7f1936d90740 1 -- 172.21.15.43:0/3851022875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968e852f0 msgr2=0x56296924feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.390+0000 7f1936d90740 1 --2- 172.21.15.43:0/3851022875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968e852f0 0x56296924feb0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1918001eb0 tx=0x7f1918009830 comp rx=0 tx=0).stop 2024-04-25T10:39:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.390+0000 7f1936d90740 1 -- 172.21.15.43:0/3851022875 shutdown_connections 2024-04-25T10:39:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.390+0000 7f1936d90740 1 --2- 172.21.15.43:0/3851022875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562968eae1b0 0x562969271ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.390+0000 7f1936d90740 1 --2- 172.21.15.43:0/3851022875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562968e8d4d0 0x562969271990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.390+0000 7f1936d90740 1 --2- 172.21.15.43:0/3851022875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968e852f0 0x56296924feb0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.390+0000 7f1936d90740 1 -- 172.21.15.43:0/3851022875 >> 172.21.15.43:0/3851022875 conn(0x562968ea2300 msgr2=0x56296924f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.391+0000 7f1936d90740 1 -- 172.21.15.43:0/3851022875 shutdown_connections 2024-04-25T10:39:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.391+0000 7f1936d90740 1 -- 172.21.15.43:0/3851022875 wait complete. 2024-04-25T10:39:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.391+0000 7f1936d90740 1 -- start start 2024-04-25T10:39:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.391+0000 7f1936d90740 1 Processor -- start 2024-04-25T10:39:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.391+0000 7f1936d90740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562968e852f0 0x5629692cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.391+0000 7f1936d90740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968e8d4d0 0x5629692cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.391+0000 7f1936d90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562968eae1b0 0x562969270d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.391+0000 7f1936d90740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f19281abef0 con 0x562968e8d4d0 2024-04-25T10:39:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.391+0000 7f1936d90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f19281ad360 con 0x562968eae1b0 2024-04-25T10:39:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.391+0000 7f1936d90740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f19281ac5c0 con 0x562968e852f0 2024-04-25T10:39:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.391+0000 7f1936d90740 10 client.0 _ll_register_callbacks cb 0x7f1924009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.392+0000 7f192dc7c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968e8d4d0 0x5629692cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.392+0000 7f192dc7c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968e8d4d0 0x5629692cf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49462/0 (socket says 172.21.15.43:49462) 2024-04-25T10:39:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.392+0000 7f192dc7c640 1 -- 172.21.15.43:0/4199578041 learned_addr learned my addr 172.21.15.43:0/4199578041 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.392+0000 7f192d47b640 1 --2- 172.21.15.43:0/4199578041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562968e852f0 0x5629692cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.392+0000 7f192d47b640 1 --2- 172.21.15.43:0/4199578041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562968eae1b0 0x562969270d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.392+0000 7f192dc7c640 1 -- 172.21.15.43:0/4199578041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562968e852f0 msgr2=0x5629692cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.392+0000 7f192dc7c640 1 --2- 172.21.15.43:0/4199578041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562968e852f0 0x5629692cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.392+0000 7f192dc7c640 1 -- 172.21.15.43:0/4199578041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562968eae1b0 msgr2=0x562969270d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.392+0000 7f192dc7c640 1 --2- 172.21.15.43:0/4199578041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562968eae1b0 0x562969270d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.392+0000 7f192dc7c640 1 -- 172.21.15.43:0/4199578041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629692713b0 con 0x562968e8d4d0 2024-04-25T10:39:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.392+0000 7f192dc7c640 1 --2- 172.21.15.43:0/4199578041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968e8d4d0 0x5629692cf390 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1924000f80 tx=0x7f19240116e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e05125f65dfe734 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.392+0000 7f19117fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.392+0000 7f19117fa640 1 -- 172.21.15.43:0/4199578041 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924023020 con 0x562968e8d4d0 2024-04-25T10:39:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.392+0000 7f19117fa640 1 -- 172.21.15.43:0/4199578041 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7f1924004450 con 0x562968e8d4d0 2024-04-25T10:39:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.392+0000 7f1936d90740 1 -- 172.21.15.43:0/4199578041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1918009520 con 0x562968e8d4d0 2024-04-25T10:39:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.393+0000 7f19117fa640 1 -- 172.21.15.43:0/4199578041 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f192402e050 con 0x562968e8d4d0 2024-04-25T10:39:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.393+0000 7f1936d90740 10 client.17322 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.393+0000 7f1936d90740 1 -- 172.21.15.43:0/4199578041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5629692d6410 con 0x562968e8d4d0 2024-04-25T10:39:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.393+0000 7f19117fa640 1 -- 172.21.15.43:0/4199578041 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1918009520 con 0x562968e8d4d0 2024-04-25T10:39:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.393+0000 7f19117fa640 1 -- 172.21.15.43:0/4199578041 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f192401e400 con 0x562968e8d4d0 2024-04-25T10:39:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.393+0000 7f1936d90740 10 client.17322 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.393+0000 7f1936d90740 10 client.17322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.393+0000 7f1936d90740 20 client.17322 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.393+0000 7f1936d90740 1 --2- 172.21.15.43:0/4199578041 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562968e421c0 0x5629692d8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.393+0000 7f1936d90740 4 client.17322 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:39:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.393+0000 7f1936d90740 1 -- 172.21.15.43:0/4199578041 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5629691bbee0 con 0x562968e421c0 2024-04-25T10:39:05.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.393+0000 7f192d47b640 1 --2- 172.21.15.43:0/4199578041 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562968e421c0 0x5629692d8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:05.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.394+0000 7f192d47b640 1 --2- 172.21.15.43:0/4199578041 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562968e421c0 0x5629692d8bd0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52ec00f33cb63c8b server_cookie=8439e92ddb1202d3 in_seq=0 out_seq=0 2024-04-25T10:39:05.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.394+0000 7f19117fa640 10 client.17322 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:05.412 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.409+0000 7f19117fa640 1 -- 172.21.15.43:0/4199578041 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5629691bbee0 con 0x562968e421c0 2024-04-25T10:39:05.412 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.409+0000 7f19117fa640 10 client.17322 handle_command_reply: tid=0 2024-04-25T10:39:05.412 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.409+0000 7f1936d90740 1 client.17322 shutdown 2024-04-25T10:39:05.412 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.409+0000 7f1936d90740 1 -- 172.21.15.43:0/4199578041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968e8d4d0 msgr2=0x5629692cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:05.412 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.409+0000 7f1936d90740 1 --2- 172.21.15.43:0/4199578041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968e8d4d0 0x5629692cf390 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1924000f80 tx=0x7f19240116e0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.412 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.409+0000 7f1936d90740 1 -- 172.21.15.43:0/4199578041 shutdown_connections 2024-04-25T10:39:05.412 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.409+0000 7f1936d90740 1 --2- 172.21.15.43:0/4199578041 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562968e421c0 0x5629692d8bd0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.412 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.409+0000 7f1936d90740 1 --2- 172.21.15.43:0/4199578041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562968eae1b0 0x562969270d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.412 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.409+0000 7f1936d90740 1 --2- 172.21.15.43:0/4199578041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562968e8d4d0 0x5629692cf390 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.410+0000 7f1936d90740 1 --2- 172.21.15.43:0/4199578041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562968e852f0 0x5629692cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.409+0000 7f19117fa640 0 client.17322 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.410+0000 7f1936d90740 1 -- 172.21.15.43:0/4199578041 >> 172.21.15.43:0/4199578041 conn(0x562968ea2300 msgr2=0x56296924d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.410+0000 7f1936d90740 1 -- 172.21.15.43:0/4199578041 shutdown_connections 2024-04-25T10:39:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.410+0000 7f1936d90740 1 -- 172.21.15.43:0/4199578041 wait complete. 2024-04-25T10:39:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.410+0000 7f1936d90740 20 client.17322 trim_cache size 0 max 16384 2024-04-25T10:39:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:39:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:39:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:39:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:39:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.410+0000 7f192fee0640 1 -- 172.21.15.43:0/2622365729 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f19140afc50 msgr2=0x7f19140b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.410+0000 7f192fee0640 1 --2- 172.21.15.43:0/2622365729 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f19140afc50 0x7f19140b2160 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f19200a36d0 tx=0x7f1920076150 comp rx=0 tx=0).stop 2024-04-25T10:39:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.410+0000 7f192fee0640 1 -- 172.21.15.43:0/2622365729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1928109ff0 msgr2=0x7f192819eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.410+0000 7f192fee0640 1 --2- 172.21.15.43:0/2622365729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1928109ff0 0x7f192819eff0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f192000e970 tx=0x7f192000ed70 comp rx=0 tx=0).stop 2024-04-25T10:39:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.410+0000 7f192fee0640 1 -- 172.21.15.43:0/2622365729 shutdown_connections 2024-04-25T10:39:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.410+0000 7f192fee0640 1 --2- 172.21.15.43:0/2622365729 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f19140afc50 0x7f19140b2160 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.410+0000 7f192fee0640 1 --2- 172.21.15.43:0/2622365729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1928109ff0 0x7f192819eff0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.410+0000 7f192fee0640 1 --2- 172.21.15.43:0/2622365729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1928109b50 0x7f192819ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.410+0000 7f192fee0640 1 --2- 172.21.15.43:0/2622365729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19281042a0 0x7f192819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.410+0000 7f192fee0640 1 -- 172.21.15.43:0/2622365729 >> 172.21.15.43:0/2622365729 conn(0x7f19280ff230 msgr2=0x7f192810f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.410+0000 7f192fee0640 1 -- 172.21.15.43:0/2622365729 shutdown_connections 2024-04-25T10:39:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:05.410+0000 7f192fee0640 1 -- 172.21.15.43:0/2622365729 wait complete. 2024-04-25T10:39:06.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:06 smithi077 ceph-mon[32772]: pgmap v201: 97 pgs: 97 active+clean; 2.0 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 95 KiB/s wr, 20 op/s 2024-04-25T10:39:06.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:06 smithi164 ceph-mon[32713]: pgmap v201: 97 pgs: 97 active+clean; 2.0 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 95 KiB/s wr, 20 op/s 2024-04-25T10:39:06.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:06 smithi043 ceph-mon[29478]: pgmap v201: 97 pgs: 97 active+clean; 2.0 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 95 KiB/s wr, 20 op/s 2024-04-25T10:39:06.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:39:06.468 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:39:06.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.594+0000 7fe25a1b2640 1 -- 172.21.15.43:0/1618898560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe254109fc0 msgr2=0x7fe254102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:06.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.594+0000 7fe25a1b2640 1 --2- 172.21.15.43:0/1618898560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe254109fc0 0x7fe254102150 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fe248009900 tx=0x7fe24802a410 comp rx=0 tx=0).stop 2024-04-25T10:39:06.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.594+0000 7fe25a1b2640 1 -- 172.21.15.43:0/1618898560 shutdown_connections 2024-04-25T10:39:06.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.594+0000 7fe25a1b2640 1 --2- 172.21.15.43:0/1618898560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2541027c0 0x7fe254110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.594+0000 7fe25a1b2640 1 --2- 172.21.15.43:0/1618898560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe254109fc0 0x7fe254102150 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.594+0000 7fe25a1b2640 1 --2- 172.21.15.43:0/1618898560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe254109b20 0x7fe254101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.594+0000 7fe25a1b2640 1 -- 172.21.15.43:0/1618898560 >> 172.21.15.43:0/1618898560 conn(0x7fe254077880 msgr2=0x7fe254077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:06.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.594+0000 7fe25a1b2640 1 -- 172.21.15.43:0/1618898560 shutdown_connections 2024-04-25T10:39:06.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.594+0000 7fe25a1b2640 1 -- 172.21.15.43:0/1618898560 wait complete. 2024-04-25T10:39:06.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.594+0000 7fe25a1b2640 1 Processor -- start 2024-04-25T10:39:06.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.595+0000 7fe25a1b2640 1 -- start start 2024-04-25T10:39:06.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.595+0000 7fe25a1b2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2541027c0 0x7fe25419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.595+0000 7fe25a1b2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe254109b20 0x7fe25419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.595+0000 7fe25a1b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe254109fc0 0x7fe25419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.595+0000 7fe25a1b2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe254113ce0 con 0x7fe254109fc0 2024-04-25T10:39:06.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.595+0000 7fe25a1b2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe254113b60 con 0x7fe254109b20 2024-04-25T10:39:06.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.595+0000 7fe25a1b2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe254113e60 con 0x7fe2541027c0 2024-04-25T10:39:06.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.595+0000 7fe253fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe254109fc0 0x7fe25419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.595+0000 7fe253fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe254109fc0 0x7fe25419ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49480/0 (socket says 172.21.15.43:49480) 2024-04-25T10:39:06.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.595+0000 7fe253fff640 1 -- 172.21.15.43:0/1542358814 learned_addr learned my addr 172.21.15.43:0/1542358814 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:06.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.595+0000 7fe252ffd640 1 --2- 172.21.15.43:0/1542358814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe254109b20 0x7fe25419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.595+0000 7fe2537fe640 1 --2- 172.21.15.43:0/1542358814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2541027c0 0x7fe25419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.595+0000 7fe253fff640 1 -- 172.21.15.43:0/1542358814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2541027c0 msgr2=0x7fe25419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:06.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.595+0000 7fe253fff640 1 --2- 172.21.15.43:0/1542358814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2541027c0 0x7fe25419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.595+0000 7fe253fff640 1 -- 172.21.15.43:0/1542358814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe254109b20 msgr2=0x7fe25419acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:06.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.595+0000 7fe253fff640 1 --2- 172.21.15.43:0/1542358814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe254109b20 0x7fe25419acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.596+0000 7fe253fff640 1 -- 172.21.15.43:0/1542358814 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25419f5c0 con 0x7fe254109fc0 2024-04-25T10:39:06.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.596+0000 7fe2537fe640 1 --2- 172.21.15.43:0/1542358814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2541027c0 0x7fe25419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:06.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.596+0000 7fe253fff640 1 --2- 172.21.15.43:0/1542358814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe254109fc0 0x7fe25419ef80 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00d720 tx=0x7fe24c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:06.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.596+0000 7fe250ff9640 1 -- 172.21.15.43:0/1542358814 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c017070 con 0x7fe254109fc0 2024-04-25T10:39:06.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.596+0000 7fe25a1b2640 1 -- 172.21.15.43:0/1542358814 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe25419f8b0 con 0x7fe254109fc0 2024-04-25T10:39:06.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.596+0000 7fe250ff9640 1 -- 172.21.15.43:0/1542358814 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe24c00de30 con 0x7fe254109fc0 2024-04-25T10:39:06.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.596+0000 7fe250ff9640 1 -- 172.21.15.43:0/1542358814 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c004d00 con 0x7fe254109fc0 2024-04-25T10:39:06.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.596+0000 7fe25a1b2640 1 -- 172.21.15.43:0/1542358814 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2541a7230 con 0x7fe254109fc0 2024-04-25T10:39:06.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.598+0000 7fe261068740 1 Processor -- start 2024-04-25T10:39:06.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.599+0000 7fe250ff9640 1 -- 172.21.15.43:0/1542358814 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fe24c010460 con 0x7fe254109fc0 2024-04-25T10:39:06.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.599+0000 7fe261068740 1 -- start start 2024-04-25T10:39:06.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.599+0000 7fe261068740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa65390 0x5572bae05a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.599+0000 7fe261068740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa3c4d0 0x5572bae05fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.599+0000 7fe261068740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa446b0 0x5572bae128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.599+0000 7fe261068740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5572bad6bd80 con 0x5572baa3c4d0 2024-04-25T10:39:06.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.599+0000 7fe261068740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5572bad258b0 con 0x5572baa65390 2024-04-25T10:39:06.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.599+0000 7fe261068740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5572bad67d40 con 0x5572baa446b0 2024-04-25T10:39:06.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.599+0000 7fe2537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa3c4d0 0x5572bae05fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.599+0000 7fe2537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa3c4d0 0x5572bae05fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49492/0 (socket says 172.21.15.43:49492) 2024-04-25T10:39:06.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.599+0000 7fe2537fe640 1 -- 172.21.15.43:0/911870832 learned_addr learned my addr 172.21.15.43:0/911870832 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:06.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.599+0000 7fe252ffd640 1 --2- 172.21.15.43:0/911870832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa65390 0x5572bae05a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.599+0000 7fe250ff9640 1 --2- 172.21.15.43:0/1542358814 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe23c0afc60 0x7fe23c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.599+0000 7fe252ffd640 1 --2- 172.21.15.43:0/911870832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa446b0 0x5572bae128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.600+0000 7fe253fff640 1 --2- 172.21.15.43:0/1542358814 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe23c0afc60 0x7fe23c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.600+0000 7fe2537fe640 1 -- 172.21.15.43:0/911870832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa446b0 msgr2=0x5572bae128f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:06.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.600+0000 7fe2537fe640 1 --2- 172.21.15.43:0/911870832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa446b0 0x5572bae128f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.600+0000 7fe2537fe640 1 -- 172.21.15.43:0/911870832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa65390 msgr2=0x5572bae05a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:06.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.600+0000 7fe2537fe640 1 --2- 172.21.15.43:0/911870832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa65390 0x5572bae05a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.600+0000 7fe2537fe640 1 -- 172.21.15.43:0/911870832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572bacf6a40 con 0x5572baa3c4d0 2024-04-25T10:39:06.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.600+0000 7fe252ffd640 1 --2- 172.21.15.43:0/911870832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa65390 0x5572bae05a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:39:06.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.600+0000 7fe250ff9640 1 -- 172.21.15.43:0/1542358814 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7fe24c0de990 con 0x7fe254109fc0 2024-04-25T10:39:06.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.600+0000 7fe2537fe640 1 --2- 172.21.15.43:0/911870832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa3c4d0 0x5572bae05fa0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe24400aec0 tx=0x7fe24400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d71ab1a0d6b198d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:06.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.600+0000 7fe239ffb640 1 -- 172.21.15.43:0/911870832 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe244004d20 con 0x5572baa3c4d0 2024-04-25T10:39:06.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.600+0000 7fe253fff640 1 --2- 172.21.15.43:0/1542358814 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe23c0afc60 0x7fe23c0b2170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0dc170 tx=0x7fe24c014000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:06.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.600+0000 7fe239ffb640 1 -- 172.21.15.43:0/911870832 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe244011b00 con 0x5572baa3c4d0 2024-04-25T10:39:06.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.600+0000 7fe239ffb640 1 -- 172.21.15.43:0/911870832 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe244005580 con 0x5572baa3c4d0 2024-04-25T10:39:06.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.601+0000 7fe261068740 1 -- 172.21.15.43:0/911870832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa3c4d0 msgr2=0x5572bae05fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:06.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.601+0000 7fe261068740 1 --2- 172.21.15.43:0/911870832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa3c4d0 0x5572bae05fa0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe24400aec0 tx=0x7fe24400f730 comp rx=0 tx=0).stop 2024-04-25T10:39:06.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.601+0000 7fe261068740 1 -- 172.21.15.43:0/911870832 shutdown_connections 2024-04-25T10:39:06.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.601+0000 7fe261068740 1 --2- 172.21.15.43:0/911870832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa446b0 0x5572bae128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.601+0000 7fe261068740 1 --2- 172.21.15.43:0/911870832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa3c4d0 0x5572bae05fa0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.601+0000 7fe261068740 1 --2- 172.21.15.43:0/911870832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa65390 0x5572bae05a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.601+0000 7fe261068740 1 -- 172.21.15.43:0/911870832 >> 172.21.15.43:0/911870832 conn(0x5572baa594e0 msgr2=0x5572bae02210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:06.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.601+0000 7fe261068740 1 -- 172.21.15.43:0/911870832 shutdown_connections 2024-04-25T10:39:06.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.601+0000 7fe261068740 1 -- 172.21.15.43:0/911870832 wait complete. 2024-04-25T10:39:06.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.601+0000 7fe261068740 1 -- start start 2024-04-25T10:39:06.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.602+0000 7fe261068740 1 Processor -- start 2024-04-25T10:39:06.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.602+0000 7fe261068740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa3c4d0 0x5572bae10640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.602+0000 7fe261068740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa446b0 0x5572bae10b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.602+0000 7fe261068740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa65390 0x5572bae164a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.602+0000 7fe261068740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5572bae12fe0 con 0x5572baa3c4d0 2024-04-25T10:39:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.602+0000 7fe261068740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5572bacf2d10 con 0x5572baa65390 2024-04-25T10:39:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.602+0000 7fe261068740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5572bae13160 con 0x5572baa446b0 2024-04-25T10:39:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.602+0000 7fe261068740 10 client.0 _ll_register_callbacks cb 0x5572bad1f210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.602+0000 7fe252ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa3c4d0 0x5572bae10640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.602+0000 7fe252ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa3c4d0 0x5572bae10640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49508/0 (socket says 172.21.15.43:49508) 2024-04-25T10:39:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.602+0000 7fe252ffd640 1 -- 172.21.15.43:0/28271879 learned_addr learned my addr 172.21.15.43:0/28271879 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.603+0000 7fe2537fe640 1 --2- 172.21.15.43:0/28271879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa446b0 0x5572bae10b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.603+0000 7fe252ffd640 1 -- 172.21.15.43:0/28271879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa446b0 msgr2=0x5572bae10b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.603+0000 7fe252ffd640 1 --2- 172.21.15.43:0/28271879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa446b0 0x5572bae10b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.603+0000 7fe252ffd640 1 -- 172.21.15.43:0/28271879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa65390 msgr2=0x5572bae164a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:06.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.603+0000 7fe252ffd640 1 --2- 172.21.15.43:0/28271879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa65390 0x5572bae164a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.603+0000 7fe252ffd640 1 -- 172.21.15.43:0/28271879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572bab477d0 con 0x5572baa3c4d0 2024-04-25T10:39:06.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.603+0000 7fe252ffd640 1 --2- 172.21.15.43:0/28271879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa3c4d0 0x5572bae10640 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fe254075740 tx=0x7fe248032ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be9ca72142f3603 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:06.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.603+0000 7fe232ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:06.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.603+0000 7fe232ffd640 1 -- 172.21.15.43:0/28271879 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe248044070 con 0x5572baa3c4d0 2024-04-25T10:39:06.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.603+0000 7fe261068740 1 -- 172.21.15.43:0/28271879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe244011b00 con 0x5572baa3c4d0 2024-04-25T10:39:06.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.603+0000 7fe232ffd640 1 -- 172.21.15.43:0/28271879 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7fe248004240 con 0x5572baa3c4d0 2024-04-25T10:39:06.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.603+0000 7fe232ffd640 1 -- 172.21.15.43:0/28271879 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe248035050 con 0x5572baa3c4d0 2024-04-25T10:39:06.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.603+0000 7fe261068740 10 client.17334 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:06.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.603+0000 7fe261068740 1 -- 172.21.15.43:0/28271879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5572baa7d3f0 con 0x5572baa3c4d0 2024-04-25T10:39:06.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.603+0000 7fe232ffd640 1 -- 172.21.15.43:0/28271879 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe244011b00 con 0x5572baa3c4d0 2024-04-25T10:39:06.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.604+0000 7fe232ffd640 1 -- 172.21.15.43:0/28271879 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe2480494f0 con 0x5572baa3c4d0 2024-04-25T10:39:06.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.604+0000 7fe261068740 10 client.17334 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:06.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.604+0000 7fe261068740 10 client.17334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:06.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.604+0000 7fe261068740 20 client.17334 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:06.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.604+0000 7fe261068740 1 --2- 172.21.15.43:0/28271879 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5572baa23f60 0x5572bae226a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.604+0000 7fe261068740 4 client.17334 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:39:06.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.604+0000 7fe261068740 1 -- 172.21.15.43:0/28271879 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572bad79990 con 0x5572baa23f60 2024-04-25T10:39:06.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.604+0000 7fe2537fe640 1 --2- 172.21.15.43:0/28271879 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5572baa23f60 0x5572bae226a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.604+0000 7fe232ffd640 10 client.17334 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:06.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.604+0000 7fe2537fe640 1 --2- 172.21.15.43:0/28271879 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5572baa23f60 0x5572bae226a0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e802987b64cb0652 server_cookie=f3b3c2c06ea29942 in_seq=0 out_seq=0 2024-04-25T10:39:06.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.605+0000 7fe232ffd640 1 -- 172.21.15.43:0/28271879 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5572bad79990 con 0x5572baa23f60 2024-04-25T10:39:06.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.605+0000 7fe232ffd640 10 client.17334 handle_command_reply: tid=0 2024-04-25T10:39:06.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.605+0000 7fe261068740 1 client.17334 shutdown 2024-04-25T10:39:06.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.605+0000 7fe261068740 1 -- 172.21.15.43:0/28271879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa3c4d0 msgr2=0x5572bae10640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:06.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.605+0000 7fe261068740 1 --2- 172.21.15.43:0/28271879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa3c4d0 0x5572bae10640 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fe254075740 tx=0x7fe248032ef0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.605+0000 7fe261068740 1 -- 172.21.15.43:0/28271879 shutdown_connections 2024-04-25T10:39:06.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.605+0000 7fe261068740 1 --2- 172.21.15.43:0/28271879 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5572baa23f60 0x5572bae226a0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.605+0000 7fe261068740 1 --2- 172.21.15.43:0/28271879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa65390 0x5572bae164a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.605+0000 7fe261068740 1 --2- 172.21.15.43:0/28271879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa446b0 0x5572bae10b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.605+0000 7fe261068740 1 --2- 172.21.15.43:0/28271879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa3c4d0 0x5572bae10640 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.605+0000 7fe261068740 1 -- 172.21.15.43:0/28271879 >> 172.21.15.43:0/28271879 conn(0x5572baa594e0 msgr2=0x5572bae07780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:06.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.605+0000 7fe232ffd640 0 client.17334 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:06.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.606+0000 7fe261068740 1 -- 172.21.15.43:0/28271879 shutdown_connections 2024-04-25T10:39:06.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.606+0000 7fe261068740 1 -- 172.21.15.43:0/28271879 wait complete. 2024-04-25T10:39:06.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.606+0000 7fe261068740 20 client.17334 trim_cache size 0 max 16384 2024-04-25T10:39:06.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.624+0000 7fe261068740 1 Processor -- start 2024-04-25T10:39:06.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.624+0000 7fe261068740 1 -- start start 2024-04-25T10:39:06.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.624+0000 7fe261068740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa3c4d0 0x5572bae0f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.624+0000 7fe261068740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa446b0 0x5572bae12110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.624+0000 7fe261068740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa65390 0x5572bae12650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.624+0000 7fe261068740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe2541b23e0 con 0x5572baa446b0 2024-04-25T10:39:06.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.624+0000 7fe261068740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe2541afcb0 con 0x5572baa3c4d0 2024-04-25T10:39:06.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.624+0000 7fe261068740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe2541adb20 con 0x5572baa65390 2024-04-25T10:39:06.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.624+0000 7fe2537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa446b0 0x5572bae12110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.624+0000 7fe2537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa446b0 0x5572bae12110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49516/0 (socket says 172.21.15.43:49516) 2024-04-25T10:39:06.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.624+0000 7fe2537fe640 1 -- 172.21.15.43:0/945764892 learned_addr learned my addr 172.21.15.43:0/945764892 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:06.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.624+0000 7fe252ffd640 1 --2- 172.21.15.43:0/945764892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa3c4d0 0x5572bae0f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.624+0000 7fe252ffd640 1 --2- 172.21.15.43:0/945764892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa65390 0x5572bae12650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.624+0000 7fe2537fe640 1 -- 172.21.15.43:0/945764892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa65390 msgr2=0x5572bae12650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:06.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.624+0000 7fe2537fe640 1 --2- 172.21.15.43:0/945764892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa65390 0x5572bae12650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.624+0000 7fe2537fe640 1 -- 172.21.15.43:0/945764892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa3c4d0 msgr2=0x5572bae0f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:06.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.624+0000 7fe2537fe640 1 --2- 172.21.15.43:0/945764892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa3c4d0 0x5572bae0f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.624+0000 7fe2537fe640 1 -- 172.21.15.43:0/945764892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572baa3d0f0 con 0x5572baa446b0 2024-04-25T10:39:06.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.625+0000 7fe252ffd640 1 --2- 172.21.15.43:0/945764892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa65390 0x5572bae12650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:39:06.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.625+0000 7fe2537fe640 1 --2- 172.21.15.43:0/945764892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa446b0 0x5572bae12110 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x5572bae031a0 tx=0x7fe244005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b7fcce158ee6c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:06.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.625+0000 7fe2397fa640 1 -- 172.21.15.43:0/945764892 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2440118d0 con 0x5572baa446b0 2024-04-25T10:39:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.625+0000 7fe2397fa640 1 -- 172.21.15.43:0/945764892 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe244004e90 con 0x5572baa446b0 2024-04-25T10:39:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.625+0000 7fe2397fa640 1 -- 172.21.15.43:0/945764892 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe244028990 con 0x5572baa446b0 2024-04-25T10:39:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.626+0000 7fe261068740 1 -- 172.21.15.43:0/945764892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa446b0 msgr2=0x5572bae12110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.626+0000 7fe261068740 1 --2- 172.21.15.43:0/945764892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa446b0 0x5572bae12110 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x5572bae031a0 tx=0x7fe244005160 comp rx=0 tx=0).stop 2024-04-25T10:39:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.626+0000 7fe261068740 1 -- 172.21.15.43:0/945764892 shutdown_connections 2024-04-25T10:39:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.626+0000 7fe261068740 1 --2- 172.21.15.43:0/945764892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa65390 0x5572bae12650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.626+0000 7fe261068740 1 --2- 172.21.15.43:0/945764892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa446b0 0x5572bae12110 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.626+0000 7fe261068740 1 --2- 172.21.15.43:0/945764892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa3c4d0 0x5572bae0f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.626+0000 7fe261068740 1 -- 172.21.15.43:0/945764892 >> 172.21.15.43:0/945764892 conn(0x5572baa594e0 msgr2=0x5572bae11a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.626+0000 7fe261068740 1 -- 172.21.15.43:0/945764892 shutdown_connections 2024-04-25T10:39:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.626+0000 7fe261068740 1 -- 172.21.15.43:0/945764892 wait complete. 2024-04-25T10:39:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.626+0000 7fe261068740 1 -- start start 2024-04-25T10:39:06.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.626+0000 7fe261068740 1 Processor -- start 2024-04-25T10:39:06.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.626+0000 7fe261068740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa3c4d0 0x5572bae178f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.626+0000 7fe261068740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa446b0 0x5572bae17e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.626+0000 7fe261068740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa65390 0x5572bae18370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.626+0000 7fe261068740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe2541ae170 con 0x5572baa65390 2024-04-25T10:39:06.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.626+0000 7fe261068740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe2541af5e0 con 0x5572baa3c4d0 2024-04-25T10:39:06.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.626+0000 7fe261068740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe2541ae840 con 0x5572baa446b0 2024-04-25T10:39:06.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.627+0000 7fe261068740 10 client.0 _ll_register_callbacks cb 0x7fe248032db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:06.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.627+0000 7fe252ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa65390 0x5572bae18370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.627+0000 7fe252ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa3c4d0 0x5572bae178f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.627+0000 7fe252ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa3c4d0 0x5572bae178f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:43870/0 (socket says 172.21.15.43:43870) 2024-04-25T10:39:06.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.627+0000 7fe252ffd640 1 -- 172.21.15.43:0/1398384547 learned_addr learned my addr 172.21.15.43:0/1398384547 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:06.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.627+0000 7fe2537fe640 1 --2- 172.21.15.43:0/1398384547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa446b0 0x5572bae17e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.627+0000 7fe252ffd640 1 -- 172.21.15.43:0/1398384547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa446b0 msgr2=0x5572bae17e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:06.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.627+0000 7fe252ffd640 1 --2- 172.21.15.43:0/1398384547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa446b0 0x5572bae17e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.627+0000 7fe252ffd640 1 -- 172.21.15.43:0/1398384547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa65390 msgr2=0x5572bae18370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.627+0000 7fe252ffd640 1 --2- 172.21.15.43:0/1398384547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa65390 0x5572bae18370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.627+0000 7fe252ffd640 1 -- 172.21.15.43:0/1398384547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572bad2c4b0 con 0x5572baa3c4d0 2024-04-25T10:39:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.627+0000 7fe2537fe640 1 --2- 172.21.15.43:0/1398384547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa446b0 0x5572bae17e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.627+0000 7fe252ffd640 1 --2- 172.21.15.43:0/1398384547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa3c4d0 0x5572bae178f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe2480057c0 tx=0x7fe248031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c472c3c583786b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.627+0000 7fe232ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-04-25T10:39:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.627+0000 7fe232ffd640 1 -- 172.21.15.43:0/1398384547 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe248056070 con 0x5572baa3c4d0 2024-04-25T10:39:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.628+0000 7fe261068740 1 -- 172.21.15.43:0/1398384547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe244004e90 con 0x5572baa3c4d0 2024-04-25T10:39:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.628+0000 7fe232ffd640 1 -- 172.21.15.43:0/1398384547 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7fe248004460 con 0x5572baa3c4d0 2024-04-25T10:39:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.628+0000 7fe232ffd640 1 -- 172.21.15.43:0/1398384547 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe248044040 con 0x5572baa3c4d0 2024-04-25T10:39:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.628+0000 7fe261068740 10 client.26762 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.628+0000 7fe261068740 1 -- 172.21.15.43:0/1398384547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5572bae89af0 con 0x5572baa3c4d0 2024-04-25T10:39:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.628+0000 7fe232ffd640 1 -- 172.21.15.43:0/1398384547 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe244004e90 con 0x5572baa3c4d0 2024-04-25T10:39:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.628+0000 7fe232ffd640 1 -- 172.21.15.43:0/1398384547 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe24803d570 con 0x5572baa3c4d0 2024-04-25T10:39:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.628+0000 7fe261068740 10 client.26762 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.628+0000 7fe261068740 10 client.26762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.628+0000 7fe261068740 20 client.26762 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.628+0000 7fe261068740 1 --2- 172.21.15.43:0/1398384547 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5572ba9f93a0 0x5572bae8c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.628+0000 7fe261068740 4 client.26762 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:39:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.628+0000 7fe261068740 1 -- 172.21.15.43:0/1398384547 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5572bad73800 con 0x5572ba9f93a0 2024-04-25T10:39:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.629+0000 7fe2537fe640 1 --2- 172.21.15.43:0/1398384547 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5572ba9f93a0 0x5572bae8c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.629+0000 7fe2537fe640 1 --2- 172.21.15.43:0/1398384547 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5572ba9f93a0 0x5572bae8c230 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8bb21cb8866a82f server_cookie=e3409a323f97e3b1 in_seq=0 out_seq=0 2024-04-25T10:39:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.629+0000 7fe232ffd640 10 client.26762 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.629+0000 7fe232ffd640 1 -- 172.21.15.43:0/1398384547 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5572bad73800 con 0x5572ba9f93a0 2024-04-25T10:39:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.629+0000 7fe232ffd640 10 client.26762 handle_command_reply: tid=0 2024-04-25T10:39:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.629+0000 7fe261068740 1 client.26762 shutdown 2024-04-25T10:39:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.629+0000 7fe261068740 1 -- 172.21.15.43:0/1398384547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa3c4d0 msgr2=0x5572bae178f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.629+0000 7fe261068740 1 --2- 172.21.15.43:0/1398384547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa3c4d0 0x5572bae178f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe2480057c0 tx=0x7fe248031400 comp rx=0 tx=0).stop 2024-04-25T10:39:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.629+0000 7fe261068740 1 -- 172.21.15.43:0/1398384547 shutdown_connections 2024-04-25T10:39:06.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.629+0000 7fe261068740 1 --2- 172.21.15.43:0/1398384547 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5572ba9f93a0 0x5572bae8c230 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.629+0000 7fe261068740 1 --2- 172.21.15.43:0/1398384547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572baa65390 0x5572bae18370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.629+0000 7fe261068740 1 --2- 172.21.15.43:0/1398384547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572baa446b0 0x5572bae17e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.629+0000 7fe261068740 1 --2- 172.21.15.43:0/1398384547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572baa3c4d0 0x5572bae178f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.629+0000 7fe232ffd640 0 client.26762 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:06.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.629+0000 7fe261068740 1 -- 172.21.15.43:0/1398384547 >> 172.21.15.43:0/1398384547 conn(0x5572baa594e0 msgr2=0x5572bae12410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:06.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.629+0000 7fe261068740 1 -- 172.21.15.43:0/1398384547 shutdown_connections 2024-04-25T10:39:06.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.629+0000 7fe261068740 1 -- 172.21.15.43:0/1398384547 wait complete. 2024-04-25T10:39:06.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.630+0000 7fe261068740 20 client.26762 trim_cache size 0 max 16384 2024-04-25T10:39:06.643 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:39:06.643 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:39:06.643 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:39:06.643 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:39:06.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.630+0000 7fe25a1b2640 1 -- 172.21.15.43:0/1542358814 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe23c0afc60 msgr2=0x7fe23c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:06.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.630+0000 7fe25a1b2640 1 --2- 172.21.15.43:0/1542358814 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe23c0afc60 0x7fe23c0b2170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0dc170 tx=0x7fe24c014000 comp rx=0 tx=0).stop 2024-04-25T10:39:06.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.630+0000 7fe25a1b2640 1 -- 172.21.15.43:0/1542358814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe254109fc0 msgr2=0x7fe25419ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:06.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.630+0000 7fe25a1b2640 1 --2- 172.21.15.43:0/1542358814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe254109fc0 0x7fe25419ef80 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00d720 tx=0x7fe24c00dbe0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.630+0000 7fe25a1b2640 1 -- 172.21.15.43:0/1542358814 shutdown_connections 2024-04-25T10:39:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.630+0000 7fe25a1b2640 1 --2- 172.21.15.43:0/1542358814 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe23c0afc60 0x7fe23c0b2170 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.630+0000 7fe25a1b2640 1 --2- 172.21.15.43:0/1542358814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe254109b20 0x7fe25419acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.630+0000 7fe25a1b2640 1 --2- 172.21.15.43:0/1542358814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe254109fc0 0x7fe25419ef80 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.630+0000 7fe25a1b2640 1 --2- 172.21.15.43:0/1542358814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2541027c0 0x7fe25419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.630+0000 7fe25a1b2640 1 -- 172.21.15.43:0/1542358814 >> 172.21.15.43:0/1542358814 conn(0x7fe254077880 msgr2=0x7fe2540fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.630+0000 7fe25a1b2640 1 -- 172.21.15.43:0/1542358814 shutdown_connections 2024-04-25T10:39:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.630+0000 7fe25a1b2640 1 -- 172.21.15.43:0/1542358814 wait complete. 2024-04-25T10:39:06.645 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:39:06.645 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:39:06.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.756+0000 7f4ef57d7640 1 -- 172.21.15.43:0/160999786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ef0102b60 msgr2=0x7f4ef0102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:06.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.756+0000 7f4ef57d7640 1 --2- 172.21.15.43:0/160999786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ef0102b60 0x7f4ef0102f70 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009900 tx=0x7f4ee402a410 comp rx=0 tx=0).stop 2024-04-25T10:39:06.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.756+0000 7f4ef57d7640 1 -- 172.21.15.43:0/160999786 shutdown_connections 2024-04-25T10:39:06.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.756+0000 7f4ef57d7640 1 --2- 172.21.15.43:0/160999786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ef010d3a0 0x7f4ef010f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.756+0000 7f4ef57d7640 1 --2- 172.21.15.43:0/160999786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ef0102b60 0x7f4ef0102f70 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.756+0000 7f4ef57d7640 1 --2- 172.21.15.43:0/160999786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ef0108a40 0x7f4ef0102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.756+0000 7f4ef57d7640 1 -- 172.21.15.43:0/160999786 >> 172.21.15.43:0/160999786 conn(0x7f4ef0078040 msgr2=0x7f4ef0078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:06.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.756+0000 7f4ef57d7640 1 -- 172.21.15.43:0/160999786 shutdown_connections 2024-04-25T10:39:06.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.756+0000 7f4ef57d7640 1 -- 172.21.15.43:0/160999786 wait complete. 2024-04-25T10:39:06.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.757+0000 7f4ef57d7640 1 Processor -- start 2024-04-25T10:39:06.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.757+0000 7f4ef57d7640 1 -- start start 2024-04-25T10:39:06.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.757+0000 7f4ef57d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ef0102b60 0x7f4ef019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.757+0000 7f4ef57d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ef0108a40 0x7f4ef019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.757+0000 7f4ef57d7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ef010d3a0 0x7f4ef019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.757+0000 7f4ef57d7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef0112c00 con 0x7f4ef0108a40 2024-04-25T10:39:06.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.758+0000 7f4ef57d7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef0112a80 con 0x7f4ef0102b60 2024-04-25T10:39:06.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.758+0000 7f4ef57d7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef0112d80 con 0x7f4ef010d3a0 2024-04-25T10:39:06.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.758+0000 7f4eee7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ef0108a40 0x7f4ef019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.758+0000 7f4eee7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ef0108a40 0x7f4ef019ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49550/0 (socket says 172.21.15.43:49550) 2024-04-25T10:39:06.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.758+0000 7f4eee7fc640 1 -- 172.21.15.43:0/3570515324 learned_addr learned my addr 172.21.15.43:0/3570515324 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.758+0000 7f4eeeffd640 1 --2- 172.21.15.43:0/3570515324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ef0102b60 0x7f4ef019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.758+0000 7f4eef7fe640 1 --2- 172.21.15.43:0/3570515324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ef010d3a0 0x7f4ef019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.758+0000 7f4eee7fc640 1 -- 172.21.15.43:0/3570515324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ef010d3a0 msgr2=0x7f4ef019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.758+0000 7f4eee7fc640 1 --2- 172.21.15.43:0/3570515324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ef010d3a0 0x7f4ef019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.758+0000 7f4eee7fc640 1 -- 172.21.15.43:0/3570515324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ef0102b60 msgr2=0x7f4ef019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.758+0000 7f4eee7fc640 1 --2- 172.21.15.43:0/3570515324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ef0102b60 0x7f4ef019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.758+0000 7f4eee7fc640 1 -- 172.21.15.43:0/3570515324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef019f570 con 0x7f4ef0108a40 2024-04-25T10:39:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.758+0000 7f4eeeffd640 1 --2- 172.21.15.43:0/3570515324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ef0102b60 0x7f4ef019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:39:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.758+0000 7f4eee7fc640 1 --2- 172.21.15.43:0/3570515324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ef0108a40 0x7f4ef019ac70 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f4ee402a930 tx=0x7f4ee4031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.758+0000 7f4ed7fff640 1 -- 172.21.15.43:0/3570515324 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee403f030 con 0x7f4ef0108a40 2024-04-25T10:39:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.758+0000 7f4ed7fff640 1 -- 172.21.15.43:0/3570515324 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ee4004490 con 0x7f4ef0108a40 2024-04-25T10:39:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.758+0000 7f4ef57d7640 1 -- 172.21.15.43:0/3570515324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef019f800 con 0x7f4ef0108a40 2024-04-25T10:39:06.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.758+0000 7f4ed7fff640 1 -- 172.21.15.43:0/3570515324 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4035070 con 0x7f4ef0108a40 2024-04-25T10:39:06.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.758+0000 7f4ef57d7640 1 -- 172.21.15.43:0/3570515324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef019fcf0 con 0x7f4ef0108a40 2024-04-25T10:39:06.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.760+0000 7f4ef57d7640 1 -- 172.21.15.43:0/3570515324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ef006a190 con 0x7f4ef0108a40 2024-04-25T10:39:06.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.761+0000 7f4ed7fff640 1 -- 172.21.15.43:0/3570515324 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f4ee4042030 con 0x7f4ef0108a40 2024-04-25T10:39:06.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.762+0000 7f4ed7fff640 1 --2- 172.21.15.43:0/3570515324 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f4ed80afc60 0x7f4ed80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:06.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.762+0000 7f4eeeffd640 1 --2- 172.21.15.43:0/3570515324 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f4ed80afc60 0x7f4ed80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:06.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.762+0000 7f4ed7fff640 1 -- 172.21.15.43:0/3570515324 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7f4ee40ff860 con 0x7f4ef0108a40 2024-04-25T10:39:06.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.762+0000 7f4eeeffd640 1 --2- 172.21.15.43:0/3570515324 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f4ed80afc60 0x7f4ed80b2170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0103a40 tx=0x7f4ee0005f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:06.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.765+0000 7f4ed7fff640 1 -- 172.21.15.43:0/3570515324 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ee40c8080 con 0x7f4ef0108a40 2024-04-25T10:39:06.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.991+0000 7f4ef57d7640 1 -- 172.21.15.43:0/3570515324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ef01a7240 con 0x7f4ef0108a40 2024-04-25T10:39:06.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.992+0000 7f4ed7fff640 1 -- 172.21.15.43:0/3570515324 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7f4ee40d0100 con 0x7f4ef0108a40 2024-04-25T10:39:06.994 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:39:06.994 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:39:06.994 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:39:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.994+0000 7f4ef57d7640 1 -- 172.21.15.43:0/3570515324 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f4ed80afc60 msgr2=0x7f4ed80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.994+0000 7f4ef57d7640 1 --2- 172.21.15.43:0/3570515324 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f4ed80afc60 0x7f4ed80b2170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0103a40 tx=0x7f4ee0005f90 comp rx=0 tx=0).stop 2024-04-25T10:39:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.994+0000 7f4ef57d7640 1 -- 172.21.15.43:0/3570515324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ef0108a40 msgr2=0x7f4ef019ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.994+0000 7f4ef57d7640 1 --2- 172.21.15.43:0/3570515324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ef0108a40 0x7f4ef019ac70 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f4ee402a930 tx=0x7f4ee4031a20 comp rx=0 tx=0).stop 2024-04-25T10:39:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.994+0000 7f4ef57d7640 1 -- 172.21.15.43:0/3570515324 shutdown_connections 2024-04-25T10:39:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.994+0000 7f4ef57d7640 1 --2- 172.21.15.43:0/3570515324 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f4ed80afc60 0x7f4ed80b2170 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.994+0000 7f4ef57d7640 1 --2- 172.21.15.43:0/3570515324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ef010d3a0 0x7f4ef019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.994+0000 7f4ef57d7640 1 --2- 172.21.15.43:0/3570515324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ef0108a40 0x7f4ef019ac70 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.994+0000 7f4ef57d7640 1 --2- 172.21.15.43:0/3570515324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ef0102b60 0x7f4ef019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.994+0000 7f4ef57d7640 1 -- 172.21.15.43:0/3570515324 >> 172.21.15.43:0/3570515324 conn(0x7f4ef0078040 msgr2=0x7f4ef0074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.994+0000 7f4ef57d7640 1 -- 172.21.15.43:0/3570515324 shutdown_connections 2024-04-25T10:39:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:06.994+0000 7f4ef57d7640 1 -- 172.21.15.43:0/3570515324 wait complete. 2024-04-25T10:39:07.007 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:39:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.119+0000 7fc30c306640 1 -- 172.21.15.43:0/4159290169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc304104260 msgr2=0x7fc304110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.119+0000 7fc30c306640 1 --2- 172.21.15.43:0/4159290169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc304104260 0x7fc304110630 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009ca0 tx=0x7fc2fc02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.119+0000 7fc30c306640 1 -- 172.21.15.43:0/4159290169 shutdown_connections 2024-04-25T10:39:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.119+0000 7fc30c306640 1 --2- 172.21.15.43:0/4159290169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc304104260 0x7fc304110630 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.119+0000 7fc30c306640 1 --2- 172.21.15.43:0/4159290169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc304109fb0 0x7fc304103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.119+0000 7fc30c306640 1 --2- 172.21.15.43:0/4159290169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc304109b10 0x7fc3041036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.119+0000 7fc30c306640 1 -- 172.21.15.43:0/4159290169 >> 172.21.15.43:0/4159290169 conn(0x7fc3040ff1d0 msgr2=0x7fc304101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.119+0000 7fc30c306640 1 -- 172.21.15.43:0/4159290169 shutdown_connections 2024-04-25T10:39:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.119+0000 7fc30c306640 1 -- 172.21.15.43:0/4159290169 wait complete. 2024-04-25T10:39:07.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.120+0000 7fc30c306640 1 Processor -- start 2024-04-25T10:39:07.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.120+0000 7fc30c306640 1 -- start start 2024-04-25T10:39:07.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.120+0000 7fc30c306640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc304104260 0x7fc304075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.120+0000 7fc30c306640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc304109b10 0x7fc3040764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.120+0000 7fc30c306640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc304109fb0 0x7fc304076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.120+0000 7fc30c306640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc304113d00 con 0x7fc304109fb0 2024-04-25T10:39:07.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.120+0000 7fc30c306640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc304113b80 con 0x7fc304109b10 2024-04-25T10:39:07.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.120+0000 7fc30c306640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc304113e80 con 0x7fc304104260 2024-04-25T10:39:07.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.121+0000 7fc30a8a3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc304109fb0 0x7fc304076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.121+0000 7fc30a8a3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc304109fb0 0x7fc304076a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49582/0 (socket says 172.21.15.43:49582) 2024-04-25T10:39:07.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.121+0000 7fc30a8a3640 1 -- 172.21.15.43:0/440312813 learned_addr learned my addr 172.21.15.43:0/440312813 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:07.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.121+0000 7fc3098a1640 1 --2- 172.21.15.43:0/440312813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc304109b10 0x7fc3040764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.121+0000 7fc30a0a2640 1 --2- 172.21.15.43:0/440312813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc304104260 0x7fc304075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.121+0000 7fc30a8a3640 1 -- 172.21.15.43:0/440312813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc304104260 msgr2=0x7fc304075fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.121+0000 7fc30a8a3640 1 --2- 172.21.15.43:0/440312813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc304104260 0x7fc304075fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.121+0000 7fc30a8a3640 1 -- 172.21.15.43:0/440312813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc304109b10 msgr2=0x7fc3040764f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.121+0000 7fc30a8a3640 1 --2- 172.21.15.43:0/440312813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc304109b10 0x7fc3040764f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.121+0000 7fc30a8a3640 1 -- 172.21.15.43:0/440312813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3041a8a40 con 0x7fc304109fb0 2024-04-25T10:39:07.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.121+0000 7fc30a0a2640 1 --2- 172.21.15.43:0/440312813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc304104260 0x7fc304075fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:07.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.121+0000 7fc30a8a3640 1 --2- 172.21.15.43:0/440312813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc304109fb0 0x7fc304076a30 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc005cc0 tx=0x7fc2fc02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:07.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.121+0000 7fc2fb7fe640 1 -- 172.21.15.43:0/440312813 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc003bf0 con 0x7fc304109fb0 2024-04-25T10:39:07.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.121+0000 7fc2fb7fe640 1 -- 172.21.15.43:0/440312813 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2fc003d90 con 0x7fc304109fb0 2024-04-25T10:39:07.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.121+0000 7fc30c306640 1 -- 172.21.15.43:0/440312813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3041a8cd0 con 0x7fc304109fb0 2024-04-25T10:39:07.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.121+0000 7fc2fb7fe640 1 -- 172.21.15.43:0/440312813 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc02fbc0 con 0x7fc304109fb0 2024-04-25T10:39:07.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.121+0000 7fc30c306640 1 -- 172.21.15.43:0/440312813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3041a91d0 con 0x7fc304109fb0 2024-04-25T10:39:07.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.124+0000 7fc3131ba740 1 Processor -- start 2024-04-25T10:39:07.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.124+0000 7fc2fb7fe640 1 -- 172.21.15.43:0/440312813 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fc2fc02fd60 con 0x7fc304109fb0 2024-04-25T10:39:07.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.125+0000 7fc3131ba740 1 -- start start 2024-04-25T10:39:07.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.125+0000 7fc3131ba740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6e43a7390 0x55e6e4748810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.125+0000 7fc3131ba740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 0x55e6e4748d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.125+0000 7fc3131ba740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6e43866b0 0x55e6e47556a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.125+0000 7fc3131ba740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e6e46add80 con 0x55e6e437e4d0 2024-04-25T10:39:07.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.125+0000 7fc3131ba740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e6e4667a20 con 0x55e6e43a7390 2024-04-25T10:39:07.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.125+0000 7fc3131ba740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e6e46a9d40 con 0x55e6e43866b0 2024-04-25T10:39:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.125+0000 7fc30a0a2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 0x55e6e4748d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.125+0000 7fc30a0a2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 0x55e6e4748d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49588/0 (socket says 172.21.15.43:49588) 2024-04-25T10:39:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.125+0000 7fc30a0a2640 1 -- 172.21.15.43:0/578395948 learned_addr learned my addr 172.21.15.43:0/578395948 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.125+0000 7fc3098a1640 1 --2- 172.21.15.43:0/578395948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6e43866b0 0x55e6e47556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.125+0000 7fc2fb7fe640 1 --2- 172.21.15.43:0/440312813 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc2f00afc60 0x7fc2f00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.125+0000 7fc3098a1640 1 --2- 172.21.15.43:0/578395948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6e43a7390 0x55e6e4748810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.125+0000 7fc30a8a3640 1 --2- 172.21.15.43:0/440312813 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc2f00afc60 0x7fc2f00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.125+0000 7fc30a0a2640 1 -- 172.21.15.43:0/578395948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6e43866b0 msgr2=0x55e6e47556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.125+0000 7fc30a0a2640 1 --2- 172.21.15.43:0/578395948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6e43866b0 0x55e6e47556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.125+0000 7fc30a0a2640 1 -- 172.21.15.43:0/578395948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6e43a7390 msgr2=0x55e6e4748810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.125+0000 7fc30a0a2640 1 --2- 172.21.15.43:0/578395948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6e43a7390 0x55e6e4748810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.125+0000 7fc30a0a2640 1 -- 172.21.15.43:0/578395948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6e4672c30 con 0x55e6e437e4d0 2024-04-25T10:39:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.125+0000 7fc3098a1640 1 --2- 172.21.15.43:0/578395948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6e43866b0 0x55e6e47556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.126+0000 7fc2fb7fe640 1 -- 172.21.15.43:0/440312813 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7fc2fc1006c0 con 0x7fc304109fb0 2024-04-25T10:39:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.126+0000 7fc30a0a2640 1 --2- 172.21.15.43:0/578395948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 0x55e6e4748d50 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fc30000cc50 tx=0x7fc30000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b08312301848117 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:07.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.126+0000 7fc2f8bfe640 1 -- 172.21.15.43:0/578395948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300014070 con 0x55e6e437e4d0 2024-04-25T10:39:07.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.126+0000 7fc2f8bfe640 1 -- 172.21.15.43:0/578395948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc300019b10 con 0x55e6e437e4d0 2024-04-25T10:39:07.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.126+0000 7fc2f8bfe640 1 -- 172.21.15.43:0/578395948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300010be0 con 0x55e6e437e4d0 2024-04-25T10:39:07.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.126+0000 7fc30a8a3640 1 --2- 172.21.15.43:0/440312813 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc2f00afc60 0x7fc2f00b2170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0fdfb0 tx=0x7fc2fc003f40 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:07.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.127+0000 7fc3131ba740 1 -- 172.21.15.43:0/578395948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 msgr2=0x55e6e4748d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.127+0000 7fc3131ba740 1 --2- 172.21.15.43:0/578395948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 0x55e6e4748d50 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fc30000cc50 tx=0x7fc30000e720 comp rx=0 tx=0).stop 2024-04-25T10:39:07.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.127+0000 7fc3131ba740 1 -- 172.21.15.43:0/578395948 shutdown_connections 2024-04-25T10:39:07.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.127+0000 7fc3131ba740 1 --2- 172.21.15.43:0/578395948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6e43866b0 0x55e6e47556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.127+0000 7fc3131ba740 1 --2- 172.21.15.43:0/578395948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 0x55e6e4748d50 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.127+0000 7fc3131ba740 1 --2- 172.21.15.43:0/578395948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6e43a7390 0x55e6e4748810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.127+0000 7fc3131ba740 1 -- 172.21.15.43:0/578395948 >> 172.21.15.43:0/578395948 conn(0x55e6e439b4e0 msgr2=0x55e6e4744fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:07.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.127+0000 7fc3131ba740 1 -- 172.21.15.43:0/578395948 shutdown_connections 2024-04-25T10:39:07.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.127+0000 7fc3131ba740 1 -- 172.21.15.43:0/578395948 wait complete. 2024-04-25T10:39:07.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.127+0000 7fc3131ba740 1 -- start start 2024-04-25T10:39:07.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.127+0000 7fc3131ba740 1 Processor -- start 2024-04-25T10:39:07.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.128+0000 7fc3131ba740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 0x55e6e4753320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.128+0000 7fc3131ba740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6e43866b0 0x55e6e4753860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.128+0000 7fc3131ba740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6e43a7390 0x55e6e475f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.128+0000 7fc3131ba740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e6e4755d90 con 0x55e6e437e4d0 2024-04-25T10:39:07.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.128+0000 7fc3131ba740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e6e4634dc0 con 0x55e6e43866b0 2024-04-25T10:39:07.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.128+0000 7fc3131ba740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e6e4755f10 con 0x55e6e43a7390 2024-04-25T10:39:07.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.128+0000 7fc3131ba740 10 client.0 _ll_register_callbacks cb 0x55e6e4650600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:07.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.128+0000 7fc3098a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 0x55e6e4753320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.128+0000 7fc3098a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 0x55e6e4753320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49602/0 (socket says 172.21.15.43:49602) 2024-04-25T10:39:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.128+0000 7fc3098a1640 1 -- 172.21.15.43:0/4008954857 learned_addr learned my addr 172.21.15.43:0/4008954857 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.128+0000 7fc30a0a2640 1 --2- 172.21.15.43:0/4008954857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6e43866b0 0x55e6e4753860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.128+0000 7fc3098a1640 1 -- 172.21.15.43:0/4008954857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6e43a7390 msgr2=0x55e6e475f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.128+0000 7fc3098a1640 1 --2- 172.21.15.43:0/4008954857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6e43a7390 0x55e6e475f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.128+0000 7fc3098a1640 1 -- 172.21.15.43:0/4008954857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6e43866b0 msgr2=0x55e6e4753860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.128+0000 7fc3098a1640 1 --2- 172.21.15.43:0/4008954857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6e43866b0 0x55e6e4753860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.128+0000 7fc3098a1640 1 -- 172.21.15.43:0/4008954857 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6e43bf3f0 con 0x55e6e437e4d0 2024-04-25T10:39:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.128+0000 7fc3098a1640 1 --2- 172.21.15.43:0/4008954857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 0x55e6e4753320 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fc30406b3e0 tx=0x7fc2f400bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a034bb8c4b398bfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.128+0000 7fc2edffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.129+0000 7fc2edffb640 1 -- 172.21.15.43:0/4008954857 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4018070 con 0x55e6e437e4d0 2024-04-25T10:39:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.129+0000 7fc2edffb640 1 -- 172.21.15.43:0/4008954857 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7fc2f4005220 con 0x55e6e437e4d0 2024-04-25T10:39:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.129+0000 7fc3131ba740 1 -- 172.21.15.43:0/4008954857 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc300019b10 con 0x55e6e437e4d0 2024-04-25T10:39:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.129+0000 7fc2edffb640 1 -- 172.21.15.43:0/4008954857 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc2f4005420 con 0x55e6e437e4d0 2024-04-25T10:39:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.129+0000 7fc3131ba740 10 client.17358 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.129+0000 7fc3131ba740 1 -- 172.21.15.43:0/4008954857 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e6e44897d0 con 0x55e6e437e4d0 2024-04-25T10:39:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.129+0000 7fc2edffb640 1 -- 172.21.15.43:0/4008954857 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc300019b10 con 0x55e6e437e4d0 2024-04-25T10:39:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.129+0000 7fc2edffb640 1 -- 172.21.15.43:0/4008954857 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc2f4012570 con 0x55e6e437e4d0 2024-04-25T10:39:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.129+0000 7fc3131ba740 10 client.17358 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.129+0000 7fc3131ba740 10 client.17358 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:39:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.129+0000 7fc3131ba740 20 client.17358 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.129+0000 7fc3131ba740 1 --2- 172.21.15.43:0/4008954857 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55e6e4365f60 0x55e6e4765780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.129+0000 7fc3131ba740 4 client.17358 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:39:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.129+0000 7fc3131ba740 1 -- 172.21.15.43:0/4008954857 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6e46bb990 con 0x55e6e4365f60 2024-04-25T10:39:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.130+0000 7fc30a0a2640 1 --2- 172.21.15.43:0/4008954857 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55e6e4365f60 0x55e6e4765780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.130+0000 7fc2edffb640 10 client.17358 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:39:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.130+0000 7fc30a0a2640 1 --2- 172.21.15.43:0/4008954857 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55e6e4365f60 0x55e6e4765780 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=81df1d86b69cad23 server_cookie=6d6520e65f132e96 in_seq=0 out_seq=0 2024-04-25T10:39:07.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.131+0000 7fc2edffb640 1 -- 172.21.15.43:0/4008954857 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e6e46bb990 con 0x55e6e4365f60 2024-04-25T10:39:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.131+0000 7fc2edffb640 10 client.17358 handle_command_reply: tid=0 2024-04-25T10:39:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.131+0000 7fc3131ba740 1 client.17358 shutdown 2024-04-25T10:39:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.131+0000 7fc3131ba740 1 -- 172.21.15.43:0/4008954857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 msgr2=0x55e6e4753320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.131+0000 7fc3131ba740 1 --2- 172.21.15.43:0/4008954857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 0x55e6e4753320 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fc30406b3e0 tx=0x7fc2f400bf30 comp rx=0 tx=0).stop 2024-04-25T10:39:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.131+0000 7fc3131ba740 1 -- 172.21.15.43:0/4008954857 shutdown_connections 2024-04-25T10:39:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.131+0000 7fc3131ba740 1 --2- 172.21.15.43:0/4008954857 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55e6e4365f60 0x55e6e4765780 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.131+0000 7fc3131ba740 1 --2- 172.21.15.43:0/4008954857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6e43866b0 0x55e6e4753860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.131+0000 7fc2edffb640 0 client.17358 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:39:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.131+0000 7fc3131ba740 1 --2- 172.21.15.43:0/4008954857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6e43a7390 0x55e6e475f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.131+0000 7fc3131ba740 1 --2- 172.21.15.43:0/4008954857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 0x55e6e4753320 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.131+0000 7fc3131ba740 1 -- 172.21.15.43:0/4008954857 >> 172.21.15.43:0/4008954857 conn(0x55e6e439b4e0 msgr2=0x55e6e474a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:07.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.131+0000 7fc3131ba740 1 -- 172.21.15.43:0/4008954857 shutdown_connections 2024-04-25T10:39:07.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.131+0000 7fc3131ba740 1 -- 172.21.15.43:0/4008954857 wait complete. 2024-04-25T10:39:07.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.132+0000 7fc3131ba740 20 client.17358 trim_cache size 0 max 16384 2024-04-25T10:39:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.145+0000 7fc3131ba740 1 Processor -- start 2024-04-25T10:39:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.145+0000 7fc3131ba740 1 -- start start 2024-04-25T10:39:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.145+0000 7fc3131ba740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 0x55e6e47520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.145+0000 7fc3131ba740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6e43866b0 0x55e6e475ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.145+0000 7fc3131ba740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6e43a7390 0x55e6e476af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.145+0000 7fc3131ba740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041b4540 con 0x55e6e437e4d0 2024-04-25T10:39:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.146+0000 7fc3131ba740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041b1e10 con 0x55e6e43a7390 2024-04-25T10:39:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.146+0000 7fc3131ba740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041afc80 con 0x55e6e43866b0 2024-04-25T10:39:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.146+0000 7fc3098a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 0x55e6e47520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.146+0000 7fc3098a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 0x55e6e47520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49606/0 (socket says 172.21.15.43:49606) 2024-04-25T10:39:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.146+0000 7fc3098a1640 1 -- 172.21.15.43:0/117571562 learned_addr learned my addr 172.21.15.43:0/117571562 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.146+0000 7fc3098a1640 1 --2- 172.21.15.43:0/117571562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6e43a7390 0x55e6e476af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.146+0000 7fc30a0a2640 1 --2- 172.21.15.43:0/117571562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6e43866b0 0x55e6e475ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.146+0000 7fc3098a1640 1 -- 172.21.15.43:0/117571562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6e43866b0 msgr2=0x55e6e475ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.146+0000 7fc3098a1640 1 --2- 172.21.15.43:0/117571562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6e43866b0 0x55e6e475ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.146+0000 7fc3098a1640 1 -- 172.21.15.43:0/117571562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6e43a7390 msgr2=0x55e6e476af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.146+0000 7fc3098a1640 1 --2- 172.21.15.43:0/117571562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6e43a7390 0x55e6e476af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.146+0000 7fc3098a1640 1 -- 172.21.15.43:0/117571562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6e473f2b0 con 0x55e6e437e4d0 2024-04-25T10:39:07.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.146+0000 7fc3098a1640 1 --2- 172.21.15.43:0/117571562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 0x55e6e47520f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc2f40129c0 tx=0x7fc2f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195f45d4164606f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:07.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.146+0000 7fc2effff640 1 -- 172.21.15.43:0/117571562 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4028070 con 0x55e6e437e4d0 2024-04-25T10:39:07.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.146+0000 7fc2effff640 1 -- 172.21.15.43:0/117571562 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f40057c0 con 0x55e6e437e4d0 2024-04-25T10:39:07.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.146+0000 7fc2effff640 1 -- 172.21.15.43:0/117571562 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4018030 con 0x55e6e437e4d0 2024-04-25T10:39:07.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.147+0000 7fc3131ba740 1 -- 172.21.15.43:0/117571562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 msgr2=0x55e6e47520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.147+0000 7fc3131ba740 1 --2- 172.21.15.43:0/117571562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 0x55e6e47520f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc2f40129c0 tx=0x7fc2f40042f0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.147+0000 7fc3131ba740 1 -- 172.21.15.43:0/117571562 shutdown_connections 2024-04-25T10:39:07.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.147+0000 7fc3131ba740 1 --2- 172.21.15.43:0/117571562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6e43a7390 0x55e6e476af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.147+0000 7fc3131ba740 1 --2- 172.21.15.43:0/117571562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6e43866b0 0x55e6e475ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.147+0000 7fc3131ba740 1 --2- 172.21.15.43:0/117571562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e437e4d0 0x55e6e47520f0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.147+0000 7fc3131ba740 1 -- 172.21.15.43:0/117571562 >> 172.21.15.43:0/117571562 conn(0x55e6e439b4e0 msgr2=0x55e6e47519e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:07.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.147+0000 7fc3131ba740 1 -- 172.21.15.43:0/117571562 shutdown_connections 2024-04-25T10:39:07.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.147+0000 7fc3131ba740 1 -- 172.21.15.43:0/117571562 wait complete. 2024-04-25T10:39:07.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.147+0000 7fc3131ba740 1 -- start start 2024-04-25T10:39:07.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.147+0000 7fc3131ba740 1 Processor -- start 2024-04-25T10:39:07.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.148+0000 7fc3131ba740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6e437e4d0 0x55e6e47561b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.148+0000 7fc3131ba740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6e43866b0 0x55e6e47566f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.148+0000 7fc3131ba740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e43a7390 0x55e6e47cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.148+0000 7fc3131ba740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041b02d0 con 0x55e6e43a7390 2024-04-25T10:39:07.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.148+0000 7fc3131ba740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041b1740 con 0x55e6e437e4d0 2024-04-25T10:39:07.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.148+0000 7fc3131ba740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041b09a0 con 0x55e6e43866b0 2024-04-25T10:39:07.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.148+0000 7fc3131ba740 10 client.0 _ll_register_callbacks cb 0x7fc2f4004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:07.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.148+0000 7fc3098a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e43a7390 0x55e6e47cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.148+0000 7fc3098a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e43a7390 0x55e6e47cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49608/0 (socket says 172.21.15.43:49608) 2024-04-25T10:39:07.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.148+0000 7fc3098a1640 1 -- 172.21.15.43:0/494733000 learned_addr learned my addr 172.21.15.43:0/494733000 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:07.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.148+0000 7fc30a0a2640 1 --2- 172.21.15.43:0/494733000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6e43866b0 0x55e6e47566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.148+0000 7fc3098a1640 1 --2- 172.21.15.43:0/494733000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6e437e4d0 0x55e6e47561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.148+0000 7fc3098a1640 1 -- 172.21.15.43:0/494733000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6e43866b0 msgr2=0x55e6e47566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.148+0000 7fc3098a1640 1 --2- 172.21.15.43:0/494733000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6e43866b0 0x55e6e47566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.148+0000 7fc3098a1640 1 -- 172.21.15.43:0/494733000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6e437e4d0 msgr2=0x55e6e47561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.148+0000 7fc3098a1640 1 --2- 172.21.15.43:0/494733000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6e437e4d0 0x55e6e47561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.148+0000 7fc3098a1640 1 -- 172.21.15.43:0/494733000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6e4756f90 con 0x55e6e43a7390 2024-04-25T10:39:07.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.148+0000 7fc3098a1640 1 --2- 172.21.15.43:0/494733000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e43a7390 0x55e6e47cbae0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4032d20 tx=0x7fc2f4005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6eb61acb32dec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:07.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.148+0000 7fc2edffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:07.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.149+0000 7fc2edffb640 1 -- 172.21.15.43:0/494733000 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4028050 con 0x55e6e43a7390 2024-04-25T10:39:07.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.149+0000 7fc2edffb640 1 -- 172.21.15.43:0/494733000 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7fc2f403b070 con 0x55e6e43a7390 2024-04-25T10:39:07.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.149+0000 7fc3131ba740 1 -- 172.21.15.43:0/494733000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f40057c0 con 0x55e6e43a7390 2024-04-25T10:39:07.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.149+0000 7fc2edffb640 1 -- 172.21.15.43:0/494733000 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc2f4045050 con 0x55e6e43a7390 2024-04-25T10:39:07.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.149+0000 7fc3131ba740 10 client.17364 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:07.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.149+0000 7fc3131ba740 1 -- 172.21.15.43:0/494733000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e6e47cd070 con 0x55e6e43a7390 2024-04-25T10:39:07.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.149+0000 7fc2edffb640 1 -- 172.21.15.43:0/494733000 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f40057c0 con 0x55e6e43a7390 2024-04-25T10:39:07.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.149+0000 7fc2edffb640 1 -- 172.21.15.43:0/494733000 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc2f403f020 con 0x55e6e43a7390 2024-04-25T10:39:07.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.149+0000 7fc3131ba740 10 client.17364 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:07.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.149+0000 7fc3131ba740 10 client.17364 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:39:07.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.149+0000 7fc3131ba740 20 client.17364 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:07.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.149+0000 7fc3131ba740 1 --2- 172.21.15.43:0/494733000 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55e6e433b3a0 0x55e6e47cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.149+0000 7fc3131ba740 4 client.17364 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:39:07.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.149+0000 7fc3131ba740 1 -- 172.21.15.43:0/494733000 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e6e47490e0 con 0x55e6e433b3a0 2024-04-25T10:39:07.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.149+0000 7fc30a0a2640 1 --2- 172.21.15.43:0/494733000 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55e6e433b3a0 0x55e6e47cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.150+0000 7fc30a0a2640 1 --2- 172.21.15.43:0/494733000 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55e6e433b3a0 0x55e6e47cf730 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1b225ab5ec59b818 server_cookie=260d447dc9e2af1b in_seq=0 out_seq=0 2024-04-25T10:39:07.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.150+0000 7fc2edffb640 10 client.17364 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:39:07.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.150+0000 7fc2edffb640 1 -- 172.21.15.43:0/494733000 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e6e47490e0 con 0x55e6e433b3a0 2024-04-25T10:39:07.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.150+0000 7fc2edffb640 10 client.17364 handle_command_reply: tid=0 2024-04-25T10:39:07.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.150+0000 7fc3131ba740 1 client.17364 shutdown 2024-04-25T10:39:07.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.150+0000 7fc3131ba740 1 -- 172.21.15.43:0/494733000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e43a7390 msgr2=0x55e6e47cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.150+0000 7fc3131ba740 1 --2- 172.21.15.43:0/494733000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e43a7390 0x55e6e47cbae0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4032d20 tx=0x7fc2f4005970 comp rx=0 tx=0).stop 2024-04-25T10:39:07.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.150+0000 7fc3131ba740 1 -- 172.21.15.43:0/494733000 shutdown_connections 2024-04-25T10:39:07.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.150+0000 7fc3131ba740 1 --2- 172.21.15.43:0/494733000 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55e6e433b3a0 0x55e6e47cf730 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.150+0000 7fc3131ba740 1 --2- 172.21.15.43:0/494733000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e6e43a7390 0x55e6e47cbae0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.150+0000 7fc2edffb640 0 client.17364 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:39:07.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.150+0000 7fc3131ba740 1 --2- 172.21.15.43:0/494733000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6e43866b0 0x55e6e47566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.150+0000 7fc3131ba740 1 --2- 172.21.15.43:0/494733000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6e437e4d0 0x55e6e47561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.150+0000 7fc3131ba740 1 -- 172.21.15.43:0/494733000 >> 172.21.15.43:0/494733000 conn(0x55e6e439b4e0 msgr2=0x55e6e474aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:07.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.150+0000 7fc3131ba740 1 -- 172.21.15.43:0/494733000 shutdown_connections 2024-04-25T10:39:07.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.150+0000 7fc3131ba740 1 -- 172.21.15.43:0/494733000 wait complete. 2024-04-25T10:39:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.150+0000 7fc3131ba740 20 client.17364 trim_cache size 0 max 16384 2024-04-25T10:39:07.159 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:39:07.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.151+0000 7fc30c306640 1 -- 172.21.15.43:0/440312813 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc2f00afc60 msgr2=0x7fc2f00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.151+0000 7fc30c306640 1 --2- 172.21.15.43:0/440312813 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc2f00afc60 0x7fc2f00b2170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0fdfb0 tx=0x7fc2fc003f40 comp rx=0 tx=0).stop 2024-04-25T10:39:07.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.151+0000 7fc30c306640 1 -- 172.21.15.43:0/440312813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc304109fb0 msgr2=0x7fc304076a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.151+0000 7fc30c306640 1 --2- 172.21.15.43:0/440312813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc304109fb0 0x7fc304076a30 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc005cc0 tx=0x7fc2fc02f380 comp rx=0 tx=0).stop 2024-04-25T10:39:07.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.151+0000 7fc30c306640 1 -- 172.21.15.43:0/440312813 shutdown_connections 2024-04-25T10:39:07.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.151+0000 7fc30c306640 1 --2- 172.21.15.43:0/440312813 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc2f00afc60 0x7fc2f00b2170 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.151+0000 7fc30c306640 1 --2- 172.21.15.43:0/440312813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc304109b10 0x7fc3040764f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.151+0000 7fc30c306640 1 --2- 172.21.15.43:0/440312813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc304109fb0 0x7fc304076a30 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.151+0000 7fc30c306640 1 --2- 172.21.15.43:0/440312813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc304104260 0x7fc304075fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.151+0000 7fc30c306640 1 -- 172.21.15.43:0/440312813 >> 172.21.15.43:0/440312813 conn(0x7fc3040ff1d0 msgr2=0x7fc304106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:07.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.151+0000 7fc30c306640 1 -- 172.21.15.43:0/440312813 shutdown_connections 2024-04-25T10:39:07.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.151+0000 7fc30c306640 1 -- 172.21.15.43:0/440312813 wait complete. 2024-04-25T10:39:07.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:39:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.275+0000 7efdbb013640 1 -- 172.21.15.43:0/2434846081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb4102070 msgr2=0x7efdb4102480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.275+0000 7efdbb013640 1 --2- 172.21.15.43:0/2434846081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb4102070 0x7efdb4102480 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7efda4009900 tx=0x7efda402a410 comp rx=0 tx=0).stop 2024-04-25T10:39:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.275+0000 7efdbb013640 1 -- 172.21.15.43:0/2434846081 shutdown_connections 2024-04-25T10:39:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.275+0000 7efdbb013640 1 --2- 172.21.15.43:0/2434846081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efdb410e570 0x7efdb4110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.275+0000 7efdbb013640 1 --2- 172.21.15.43:0/2434846081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb4102070 0x7efdb4102480 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.275+0000 7efdbb013640 1 --2- 172.21.15.43:0/2434846081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb4109bc0 0x7efdb4101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.275+0000 7efdbb013640 1 -- 172.21.15.43:0/2434846081 >> 172.21.15.43:0/2434846081 conn(0x7efdb4077920 msgr2=0x7efdb4077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.276+0000 7efdbb013640 1 -- 172.21.15.43:0/2434846081 shutdown_connections 2024-04-25T10:39:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.276+0000 7efdbb013640 1 -- 172.21.15.43:0/2434846081 wait complete. 2024-04-25T10:39:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.276+0000 7efdbb013640 1 Processor -- start 2024-04-25T10:39:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.276+0000 7efdbb013640 1 -- start start 2024-04-25T10:39:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.276+0000 7efdbb013640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb4102070 0x7efdb4102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.276+0000 7efdbb013640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efdb4109bc0 0x7efdb4103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.276+0000 7efdbb013640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb410e570 0x7efdb4103790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.276+0000 7efdbb013640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efdb4113d30 con 0x7efdb4102070 2024-04-25T10:39:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.276+0000 7efdbb013640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efdb4113bb0 con 0x7efdb410e570 2024-04-25T10:39:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.276+0000 7efdbb013640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efdb4113eb0 con 0x7efdb4109bc0 2024-04-25T10:39:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.277+0000 7efdb8daf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb4102070 0x7efdb4102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.277+0000 7efdb8daf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb4102070 0x7efdb4102d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49628/0 (socket says 172.21.15.43:49628) 2024-04-25T10:39:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.277+0000 7efdb8daf640 1 -- 172.21.15.43:0/4189406579 learned_addr learned my addr 172.21.15.43:0/4189406579 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.277+0000 7efdb95b0640 1 --2- 172.21.15.43:0/4189406579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb410e570 0x7efdb4103790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.277+0000 7efdabfff640 1 --2- 172.21.15.43:0/4189406579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efdb4109bc0 0x7efdb4103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.277+0000 7efdb8daf640 1 -- 172.21.15.43:0/4189406579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efdb4109bc0 msgr2=0x7efdb4103250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.277+0000 7efdb8daf640 1 --2- 172.21.15.43:0/4189406579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efdb4109bc0 0x7efdb4103250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.277+0000 7efdb8daf640 1 -- 172.21.15.43:0/4189406579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb410e570 msgr2=0x7efdb4103790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.277+0000 7efdb8daf640 1 --2- 172.21.15.43:0/4189406579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb410e570 0x7efdb4103790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.277+0000 7efdb8daf640 1 -- 172.21.15.43:0/4189406579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb41a8af0 con 0x7efdb4102070 2024-04-25T10:39:07.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.277+0000 7efdabfff640 1 --2- 172.21.15.43:0/4189406579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efdb4109bc0 0x7efdb4103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:39:07.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.277+0000 7efdb8daf640 1 --2- 172.21.15.43:0/4189406579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb4102070 0x7efdb4102d10 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7efdb000e9f0 tx=0x7efdb000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:07.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.277+0000 7efda9ffb640 1 -- 172.21.15.43:0/4189406579 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb000ccc0 con 0x7efdb4102070 2024-04-25T10:39:07.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.277+0000 7efda9ffb640 1 -- 172.21.15.43:0/4189406579 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdb00035f0 con 0x7efdb4102070 2024-04-25T10:39:07.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.277+0000 7efdbb013640 1 -- 172.21.15.43:0/4189406579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb41a8de0 con 0x7efdb4102070 2024-04-25T10:39:07.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.277+0000 7efda9ffb640 1 -- 172.21.15.43:0/4189406579 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb000ce60 con 0x7efdb4102070 2024-04-25T10:39:07.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.278+0000 7efdbb013640 1 -- 172.21.15.43:0/4189406579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb41a9250 con 0x7efdb4102070 2024-04-25T10:39:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.280+0000 7efdc1fa0740 1 Processor -- start 2024-04-25T10:39:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.280+0000 7efda9ffb640 1 -- 172.21.15.43:0/4189406579 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7efdb0003aa0 con 0x7efdb4102070 2024-04-25T10:39:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.280+0000 7efdc1fa0740 1 -- start start 2024-04-25T10:39:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.280+0000 7efdc1fa0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556565850390 0x556565bf1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.280+0000 7efdc1fa0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565658274d0 0x556565bf1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.280+0000 7efdc1fa0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55656582f6b0 0x556565bfe6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.280+0000 7efdc1fa0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556565b56d80 con 0x55656582f6b0 2024-04-25T10:39:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.280+0000 7efdc1fa0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556565b10a20 con 0x556565850390 2024-04-25T10:39:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.280+0000 7efdc1fa0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556565b52d40 con 0x5565658274d0 2024-04-25T10:39:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.281+0000 7efdabfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55656582f6b0 0x556565bfe6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.281+0000 7efdabfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55656582f6b0 0x556565bfe6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49636/0 (socket says 172.21.15.43:49636) 2024-04-25T10:39:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.281+0000 7efdabfff640 1 -- 172.21.15.43:0/476261480 learned_addr learned my addr 172.21.15.43:0/476261480 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.281+0000 7efdb95b0640 1 --2- 172.21.15.43:0/476261480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556565850390 0x556565bf1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.281+0000 7efdb8daf640 1 --2- 172.21.15.43:0/476261480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565658274d0 0x556565bf1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.281+0000 7efda9ffb640 1 --2- 172.21.15.43:0/4189406579 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efd9c0afc60 0x7efd9c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.281+0000 7efdabfff640 1 -- 172.21.15.43:0/476261480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565658274d0 msgr2=0x556565bf1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.281+0000 7efdabfff640 1 --2- 172.21.15.43:0/476261480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565658274d0 0x556565bf1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.281+0000 7efdabfff640 1 -- 172.21.15.43:0/476261480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556565850390 msgr2=0x556565bf1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.281+0000 7efdabfff640 1 --2- 172.21.15.43:0/476261480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556565850390 0x556565bf1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.281+0000 7efdabfff640 1 -- 172.21.15.43:0/476261480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556565b1bc30 con 0x55656582f6b0 2024-04-25T10:39:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.281+0000 7efdb95b0640 1 --2- 172.21.15.43:0/476261480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556565850390 0x556565bf1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:39:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.281+0000 7efdb95b0640 1 --2- 172.21.15.43:0/4189406579 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efd9c0afc60 0x7efd9c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.281+0000 7efda9ffb640 1 -- 172.21.15.43:0/4189406579 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7efdb00df4d0 con 0x7efdb4102070 2024-04-25T10:39:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.281+0000 7efdabfff640 1 --2- 172.21.15.43:0/476261480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55656582f6b0 0x556565bfe6a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7efda4000c00 tx=0x7efda4032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8fbfddc4bc1b01e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.282+0000 7efd9affd640 1 -- 172.21.15.43:0/476261480 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda403f070 con 0x55656582f6b0 2024-04-25T10:39:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.282+0000 7efd9affd640 1 -- 172.21.15.43:0/476261480 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efda40316a0 con 0x55656582f6b0 2024-04-25T10:39:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.282+0000 7efdb95b0640 1 --2- 172.21.15.43:0/4189406579 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efd9c0afc60 0x7efd9c0b2170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x556565bee980 tx=0x7efdac004a40 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.282+0000 7efd9affd640 1 -- 172.21.15.43:0/476261480 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4035070 con 0x55656582f6b0 2024-04-25T10:39:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.282+0000 7efdc1fa0740 1 -- 172.21.15.43:0/476261480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55656582f6b0 msgr2=0x556565bfe6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.282+0000 7efdc1fa0740 1 --2- 172.21.15.43:0/476261480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55656582f6b0 0x556565bfe6a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7efda4000c00 tx=0x7efda4032c30 comp rx=0 tx=0).stop 2024-04-25T10:39:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.282+0000 7efdc1fa0740 1 -- 172.21.15.43:0/476261480 shutdown_connections 2024-04-25T10:39:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.282+0000 7efdc1fa0740 1 --2- 172.21.15.43:0/476261480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55656582f6b0 0x556565bfe6a0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.282+0000 7efdc1fa0740 1 --2- 172.21.15.43:0/476261480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565658274d0 0x556565bf1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.282+0000 7efdc1fa0740 1 --2- 172.21.15.43:0/476261480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556565850390 0x556565bf1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.282+0000 7efdc1fa0740 1 -- 172.21.15.43:0/476261480 >> 172.21.15.43:0/476261480 conn(0x5565658444e0 msgr2=0x556565bedfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.283+0000 7efdc1fa0740 1 -- 172.21.15.43:0/476261480 shutdown_connections 2024-04-25T10:39:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.283+0000 7efdc1fa0740 1 -- 172.21.15.43:0/476261480 wait complete. 2024-04-25T10:39:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.283+0000 7efdc1fa0740 1 -- start start 2024-04-25T10:39:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.283+0000 7efdc1fa0740 1 Processor -- start 2024-04-25T10:39:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.283+0000 7efdc1fa0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565658274d0 0x556565bfc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.283+0000 7efdc1fa0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656582f6b0 0x556565bfc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efdc1fa0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556565850390 0x556565c081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efdc1fa0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556565bfed30 con 0x5565658274d0 2024-04-25T10:39:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efdc1fa0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556565adddc0 con 0x55656582f6b0 2024-04-25T10:39:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efdc1fa0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556565bfeeb0 con 0x556565850390 2024-04-25T10:39:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efdc1fa0740 10 client.0 _ll_register_callbacks cb 0x556565af9600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efdb8daf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565658274d0 0x556565bfc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efdb8daf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565658274d0 0x556565bfc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49646/0 (socket says 172.21.15.43:49646) 2024-04-25T10:39:07.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efdb8daf640 1 -- 172.21.15.43:0/2556635012 learned_addr learned my addr 172.21.15.43:0/2556635012 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:07.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efdabfff640 1 --2- 172.21.15.43:0/2556635012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656582f6b0 0x556565bfc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efdb95b0640 1 --2- 172.21.15.43:0/2556635012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556565850390 0x556565c081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efdb8daf640 1 -- 172.21.15.43:0/2556635012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556565850390 msgr2=0x556565c081b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efdb8daf640 1 --2- 172.21.15.43:0/2556635012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556565850390 0x556565c081b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efdb8daf640 1 -- 172.21.15.43:0/2556635012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656582f6b0 msgr2=0x556565bfc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efdb8daf640 1 --2- 172.21.15.43:0/2556635012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656582f6b0 0x556565bfc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efdb8daf640 1 -- 172.21.15.43:0/2556635012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565658683f0 con 0x5565658274d0 2024-04-25T10:39:07.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efdb8daf640 1 --2- 172.21.15.43:0/2556635012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565658274d0 0x556565bfc280 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7efdb0000c00 tx=0x7efdb00a3b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db56c0204b679c5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:07.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efd93fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:07.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efd93fff640 1 -- 172.21.15.43:0/2556635012 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb00a6fa0 con 0x5565658274d0 2024-04-25T10:39:07.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efd93fff640 1 -- 172.21.15.43:0/2556635012 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7efdb00b0020 con 0x5565658274d0 2024-04-25T10:39:07.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efdc1fa0740 1 -- 172.21.15.43:0/2556635012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efda40316a0 con 0x5565658274d0 2024-04-25T10:39:07.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efd93fff640 1 -- 172.21.15.43:0/2556635012 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efdb00037a0 con 0x5565658274d0 2024-04-25T10:39:07.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.284+0000 7efdc1fa0740 10 client.17376 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:07.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.285+0000 7efdc1fa0740 1 -- 172.21.15.43:0/2556635012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5565659327d0 con 0x5565658274d0 2024-04-25T10:39:07.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.285+0000 7efd93fff640 1 -- 172.21.15.43:0/2556635012 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efda40316a0 con 0x5565658274d0 2024-04-25T10:39:07.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.285+0000 7efd93fff640 1 -- 172.21.15.43:0/2556635012 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7efdb0003fb0 con 0x5565658274d0 2024-04-25T10:39:07.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.285+0000 7efdc1fa0740 10 client.17376 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:07.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.285+0000 7efdc1fa0740 10 client.17376 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:39:07.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.285+0000 7efdc1fa0740 20 client.17376 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:07.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.285+0000 7efdc1fa0740 1 --2- 172.21.15.43:0/2556635012 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55656580ef60 0x556565c0e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.285+0000 7efdc1fa0740 4 client.17376 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:39:07.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.285+0000 7efdc1fa0740 1 -- 172.21.15.43:0/2556635012 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556565b64990 con 0x55656580ef60 2024-04-25T10:39:07.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.285+0000 7efdabfff640 1 --2- 172.21.15.43:0/2556635012 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55656580ef60 0x556565c0e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.286+0000 7efd93fff640 10 client.17376 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:39:07.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.286+0000 7efdabfff640 1 --2- 172.21.15.43:0/2556635012 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55656580ef60 0x556565c0e510 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=520a72d60fca0e9c server_cookie=182432ae00660ab6 in_seq=0 out_seq=0 2024-04-25T10:39:07.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.286+0000 7efd93fff640 1 -- 172.21.15.43:0/2556635012 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556565b64990 con 0x55656580ef60 2024-04-25T10:39:07.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.286+0000 7efd93fff640 10 client.17376 handle_command_reply: tid=0 2024-04-25T10:39:07.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.286+0000 7efdc1fa0740 1 client.17376 shutdown 2024-04-25T10:39:07.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.287+0000 7efdc1fa0740 1 -- 172.21.15.43:0/2556635012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565658274d0 msgr2=0x556565bfc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.287+0000 7efdc1fa0740 1 --2- 172.21.15.43:0/2556635012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565658274d0 0x556565bfc280 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7efdb0000c00 tx=0x7efdb00a3b60 comp rx=0 tx=0).stop 2024-04-25T10:39:07.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.287+0000 7efdc1fa0740 1 -- 172.21.15.43:0/2556635012 shutdown_connections 2024-04-25T10:39:07.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.287+0000 7efdc1fa0740 1 --2- 172.21.15.43:0/2556635012 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55656580ef60 0x556565c0e510 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.287+0000 7efdc1fa0740 1 --2- 172.21.15.43:0/2556635012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656582f6b0 0x556565bfc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.287+0000 7efdc1fa0740 1 --2- 172.21.15.43:0/2556635012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556565850390 0x556565c081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.287+0000 7efd93fff640 0 client.17376 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:39:07.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.287+0000 7efdc1fa0740 1 --2- 172.21.15.43:0/2556635012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565658274d0 0x556565bfc280 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.287+0000 7efdc1fa0740 1 -- 172.21.15.43:0/2556635012 >> 172.21.15.43:0/2556635012 conn(0x5565658444e0 msgr2=0x556565bf3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:07.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.287+0000 7efdc1fa0740 1 -- 172.21.15.43:0/2556635012 shutdown_connections 2024-04-25T10:39:07.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.287+0000 7efdc1fa0740 1 -- 172.21.15.43:0/2556635012 wait complete. 2024-04-25T10:39:07.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.287+0000 7efdc1fa0740 20 client.17376 trim_cache size 0 max 16384 2024-04-25T10:39:07.296 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:07 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3570515324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:39:07.296 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:07 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3570515324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:39:07.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.301+0000 7efdc1fa0740 1 Processor -- start 2024-04-25T10:39:07.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.301+0000 7efdc1fa0740 1 -- start start 2024-04-25T10:39:07.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.301+0000 7efdc1fa0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565658274d0 0x556565c07e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.301+0000 7efdc1fa0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55656582f6b0 0x556565bfdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.301+0000 7efdc1fa0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556565850390 0x556565bf6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.301+0000 7efdc1fa0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41b4650 con 0x55656582f6b0 2024-04-25T10:39:07.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.301+0000 7efdc1fa0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41b1f20 con 0x5565658274d0 2024-04-25T10:39:07.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.301+0000 7efdc1fa0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41afd90 con 0x556565850390 2024-04-25T10:39:07.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.301+0000 7efdabfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55656582f6b0 0x556565bfdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.301+0000 7efdabfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55656582f6b0 0x556565bfdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49654/0 (socket says 172.21.15.43:49654) 2024-04-25T10:39:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.301+0000 7efdabfff640 1 -- 172.21.15.43:0/1429588664 learned_addr learned my addr 172.21.15.43:0/1429588664 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.301+0000 7efdb8daf640 1 --2- 172.21.15.43:0/1429588664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565658274d0 0x556565c07e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.301+0000 7efdabfff640 1 -- 172.21.15.43:0/1429588664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556565850390 msgr2=0x556565bf6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.301+0000 7efdabfff640 1 --2- 172.21.15.43:0/1429588664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556565850390 0x556565bf6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.301+0000 7efdabfff640 1 -- 172.21.15.43:0/1429588664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565658274d0 msgr2=0x556565c07e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.301+0000 7efdabfff640 1 --2- 172.21.15.43:0/1429588664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565658274d0 0x556565c07e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.301+0000 7efdabfff640 1 -- 172.21.15.43:0/1429588664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556565bf2fc0 con 0x55656582f6b0 2024-04-25T10:39:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.301+0000 7efdb8daf640 1 --2- 172.21.15.43:0/1429588664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565658274d0 0x556565c07e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:39:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.301+0000 7efdabfff640 1 --2- 172.21.15.43:0/1429588664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55656582f6b0 0x556565bfdea0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7efda4031440 tx=0x7efda4009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0fc568fa9402ce5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.302+0000 7efd9a7fc640 1 -- 172.21.15.43:0/1429588664 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4051070 con 0x55656582f6b0 2024-04-25T10:39:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.302+0000 7efd9a7fc640 1 -- 172.21.15.43:0/1429588664 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efda4004040 con 0x55656582f6b0 2024-04-25T10:39:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.302+0000 7efd9a7fc640 1 -- 172.21.15.43:0/1429588664 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda403f070 con 0x55656582f6b0 2024-04-25T10:39:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.302+0000 7efdc1fa0740 1 -- 172.21.15.43:0/1429588664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55656582f6b0 msgr2=0x556565bfdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.302+0000 7efdc1fa0740 1 --2- 172.21.15.43:0/1429588664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55656582f6b0 0x556565bfdea0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7efda4031440 tx=0x7efda4009230 comp rx=0 tx=0).stop 2024-04-25T10:39:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.302+0000 7efdc1fa0740 1 -- 172.21.15.43:0/1429588664 shutdown_connections 2024-04-25T10:39:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.302+0000 7efdc1fa0740 1 --2- 172.21.15.43:0/1429588664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556565850390 0x556565bf6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.302+0000 7efdc1fa0740 1 --2- 172.21.15.43:0/1429588664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55656582f6b0 0x556565bfdea0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.302+0000 7efdc1fa0740 1 --2- 172.21.15.43:0/1429588664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565658274d0 0x556565c07e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.302+0000 7efdc1fa0740 1 -- 172.21.15.43:0/1429588664 >> 172.21.15.43:0/1429588664 conn(0x5565658444e0 msgr2=0x556565bfd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:07.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.302+0000 7efdc1fa0740 1 -- 172.21.15.43:0/1429588664 shutdown_connections 2024-04-25T10:39:07.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.303+0000 7efdc1fa0740 1 -- 172.21.15.43:0/1429588664 wait complete. 2024-04-25T10:39:07.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.303+0000 7efdc1fa0740 1 -- start start 2024-04-25T10:39:07.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.303+0000 7efdc1fa0740 1 Processor -- start 2024-04-25T10:39:07.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.303+0000 7efdc1fa0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565658274d0 0x556565bff560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.303+0000 7efdc1fa0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656582f6b0 0x556565bffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.303+0000 7efdc1fa0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556565850390 0x556565c6e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.303+0000 7efdc1fa0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41b03e0 con 0x556565850390 2024-04-25T10:39:07.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.303+0000 7efdc1fa0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41b1850 con 0x5565658274d0 2024-04-25T10:39:07.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.303+0000 7efdc1fa0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41b0ab0 con 0x55656582f6b0 2024-04-25T10:39:07.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.303+0000 7efdc1fa0740 10 client.0 _ll_register_callbacks cb 0x7efdb00dd0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:07.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.304+0000 7efdb95b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556565850390 0x556565c6e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.304+0000 7efdb95b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556565850390 0x556565c6e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49658/0 (socket says 172.21.15.43:49658) 2024-04-25T10:39:07.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.304+0000 7efdb95b0640 1 -- 172.21.15.43:0/3343366740 learned_addr learned my addr 172.21.15.43:0/3343366740 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.304+0000 7efdabfff640 1 --2- 172.21.15.43:0/3343366740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656582f6b0 0x556565bffaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.304+0000 7efdb8daf640 1 --2- 172.21.15.43:0/3343366740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565658274d0 0x556565bff560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.304+0000 7efdb95b0640 1 -- 172.21.15.43:0/3343366740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656582f6b0 msgr2=0x556565bffaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.304+0000 7efdb95b0640 1 --2- 172.21.15.43:0/3343366740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656582f6b0 0x556565bffaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.304+0000 7efdb95b0640 1 -- 172.21.15.43:0/3343366740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565658274d0 msgr2=0x556565bff560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.304+0000 7efdb95b0640 1 --2- 172.21.15.43:0/3343366740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565658274d0 0x556565bff560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.304+0000 7efdb95b0640 1 -- 172.21.15.43:0/3343366740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556565c6edb0 con 0x556565850390 2024-04-25T10:39:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.304+0000 7efdb95b0640 1 --2- 172.21.15.43:0/3343366740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556565850390 0x556565c6e840 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7efdac00f7f0 tx=0x7efdac009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab814e9ebe9e926d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.304+0000 7efd93fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.304+0000 7efd93fff640 1 -- 172.21.15.43:0/3343366740 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac00fe30 con 0x556565850390 2024-04-25T10:39:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.304+0000 7efd93fff640 1 -- 172.21.15.43:0/3343366740 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7efdac01eae0 con 0x556565850390 2024-04-25T10:39:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.304+0000 7efdc1fa0740 1 -- 172.21.15.43:0/3343366740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efda4004040 con 0x556565850390 2024-04-25T10:39:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.304+0000 7efd93fff640 1 -- 172.21.15.43:0/3343366740 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efdac01ece0 con 0x556565850390 2024-04-25T10:39:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.305+0000 7efdc1fa0740 10 client.17382 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.305+0000 7efdc1fa0740 1 -- 172.21.15.43:0/3343366740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556565c76080 con 0x556565850390 2024-04-25T10:39:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.305+0000 7efd93fff640 1 -- 172.21.15.43:0/3343366740 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efda4004040 con 0x556565850390 2024-04-25T10:39:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.305+0000 7efd93fff640 1 -- 172.21.15.43:0/3343366740 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7efdac004590 con 0x556565850390 2024-04-25T10:39:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.305+0000 7efdc1fa0740 10 client.17382 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.305+0000 7efdc1fa0740 10 client.17382 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:39:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.305+0000 7efdc1fa0740 20 client.17382 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.305+0000 7efdc1fa0740 1 --2- 172.21.15.43:0/3343366740 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5565657e43a0 0x556565c788a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.305+0000 7efdc1fa0740 4 client.17382 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:39:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.305+0000 7efdc1fa0740 1 -- 172.21.15.43:0/3343366740 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556565b5e800 con 0x5565657e43a0 2024-04-25T10:39:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.305+0000 7efdb8daf640 1 --2- 172.21.15.43:0/3343366740 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5565657e43a0 0x556565c788a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.306+0000 7efdb8daf640 1 --2- 172.21.15.43:0/3343366740 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5565657e43a0 0x556565c788a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc463a1cdc0c71b2 server_cookie=7cdf957e022278c1 in_seq=0 out_seq=0 2024-04-25T10:39:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.306+0000 7efd93fff640 10 client.17382 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:39:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.306+0000 7efd93fff640 1 -- 172.21.15.43:0/3343366740 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556565b5e800 con 0x5565657e43a0 2024-04-25T10:39:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.306+0000 7efd93fff640 10 client.17382 handle_command_reply: tid=0 2024-04-25T10:39:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.306+0000 7efdc1fa0740 1 client.17382 shutdown 2024-04-25T10:39:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.306+0000 7efdc1fa0740 1 -- 172.21.15.43:0/3343366740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556565850390 msgr2=0x556565c6e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.306+0000 7efdc1fa0740 1 --2- 172.21.15.43:0/3343366740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556565850390 0x556565c6e840 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7efdac00f7f0 tx=0x7efdac009760 comp rx=0 tx=0).stop 2024-04-25T10:39:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.306+0000 7efdc1fa0740 1 -- 172.21.15.43:0/3343366740 shutdown_connections 2024-04-25T10:39:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.306+0000 7efdc1fa0740 1 --2- 172.21.15.43:0/3343366740 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5565657e43a0 0x556565c788a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.306+0000 7efdc1fa0740 1 --2- 172.21.15.43:0/3343366740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556565850390 0x556565c6e840 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.306+0000 7efdc1fa0740 1 --2- 172.21.15.43:0/3343366740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656582f6b0 0x556565bffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.306+0000 7efd93fff640 0 client.17382 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:39:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.306+0000 7efdc1fa0740 1 --2- 172.21.15.43:0/3343366740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565658274d0 0x556565bff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.306+0000 7efdc1fa0740 1 -- 172.21.15.43:0/3343366740 >> 172.21.15.43:0/3343366740 conn(0x5565658444e0 msgr2=0x556565bedf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.306+0000 7efdc1fa0740 1 -- 172.21.15.43:0/3343366740 shutdown_connections 2024-04-25T10:39:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.306+0000 7efdc1fa0740 1 -- 172.21.15.43:0/3343366740 wait complete. 2024-04-25T10:39:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.306+0000 7efdc1fa0740 20 client.17382 trim_cache size 0 max 16384 2024-04-25T10:39:07.315 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:39:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.307+0000 7efdbb013640 1 -- 172.21.15.43:0/4189406579 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efd9c0afc60 msgr2=0x7efd9c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.307+0000 7efdbb013640 1 --2- 172.21.15.43:0/4189406579 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efd9c0afc60 0x7efd9c0b2170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x556565bee980 tx=0x7efdac004a40 comp rx=0 tx=0).stop 2024-04-25T10:39:07.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.307+0000 7efdbb013640 1 -- 172.21.15.43:0/4189406579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb4102070 msgr2=0x7efdb4102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.307+0000 7efdbb013640 1 --2- 172.21.15.43:0/4189406579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb4102070 0x7efdb4102d10 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7efdb000e9f0 tx=0x7efdb000eeb0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.307+0000 7efdbb013640 1 -- 172.21.15.43:0/4189406579 shutdown_connections 2024-04-25T10:39:07.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.307+0000 7efdbb013640 1 --2- 172.21.15.43:0/4189406579 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efd9c0afc60 0x7efd9c0b2170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.307+0000 7efdbb013640 1 --2- 172.21.15.43:0/4189406579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb410e570 0x7efdb4103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.307+0000 7efdbb013640 1 --2- 172.21.15.43:0/4189406579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efdb4109bc0 0x7efdb4103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.307+0000 7efdbb013640 1 --2- 172.21.15.43:0/4189406579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb4102070 0x7efdb4102d10 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.307+0000 7efdbb013640 1 -- 172.21.15.43:0/4189406579 >> 172.21.15.43:0/4189406579 conn(0x7efdb4077920 msgr2=0x7efdb4105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:07.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.307+0000 7efdbb013640 1 -- 172.21.15.43:0/4189406579 shutdown_connections 2024-04-25T10:39:07.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.307+0000 7efdbb013640 1 -- 172.21.15.43:0/4189406579 wait complete. 2024-04-25T10:39:07.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:07 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3570515324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:39:07.320 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:39:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.432+0000 7f062056b640 1 -- 172.21.15.43:0/464739268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0618108eb0 msgr2=0x7f0618102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.432+0000 7f062056b640 1 --2- 172.21.15.43:0/464739268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0618108eb0 0x7f0618102b50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0608009860 tx=0x7f060802a340 comp rx=0 tx=0).stop 2024-04-25T10:39:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.432+0000 7f062056b640 1 -- 172.21.15.43:0/464739268 shutdown_connections 2024-04-25T10:39:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.432+0000 7f062056b640 1 --2- 172.21.15.43:0/464739268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0618108eb0 0x7f0618102b50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.432+0000 7f062056b640 1 --2- 172.21.15.43:0/464739268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06181031c0 0x7f061810f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.432+0000 7f062056b640 1 --2- 172.21.15.43:0/464739268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0618108a10 0x7f0618102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.432+0000 7f062056b640 1 -- 172.21.15.43:0/464739268 >> 172.21.15.43:0/464739268 conn(0x7f0618078110 msgr2=0x7f0618078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.432+0000 7f062056b640 1 -- 172.21.15.43:0/464739268 shutdown_connections 2024-04-25T10:39:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.432+0000 7f062056b640 1 -- 172.21.15.43:0/464739268 wait complete. 2024-04-25T10:39:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.432+0000 7f062056b640 1 Processor -- start 2024-04-25T10:39:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.433+0000 7f062056b640 1 -- start start 2024-04-25T10:39:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.433+0000 7f062056b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06181031c0 0x7f061819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.433+0000 7f062056b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0618108a10 0x7f061819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.433+0000 7f062056b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0618108eb0 0x7f061819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.433+0000 7f062056b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0618112c10 con 0x7f0618108eb0 2024-04-25T10:39:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.433+0000 7f062056b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0618112a90 con 0x7f06181031c0 2024-04-25T10:39:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.433+0000 7f062056b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0618112d90 con 0x7f0618108a10 2024-04-25T10:39:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.433+0000 7f061eb08640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0618108eb0 0x7f061819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.433+0000 7f061eb08640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0618108eb0 0x7f061819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49672/0 (socket says 172.21.15.43:49672) 2024-04-25T10:39:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.433+0000 7f061eb08640 1 -- 172.21.15.43:0/3439498652 learned_addr learned my addr 172.21.15.43:0/3439498652 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.433+0000 7f061e307640 1 --2- 172.21.15.43:0/3439498652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06181031c0 0x7f061819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.433+0000 7f061db06640 1 --2- 172.21.15.43:0/3439498652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0618108a10 0x7f061819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.433+0000 7f061eb08640 1 -- 172.21.15.43:0/3439498652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0618108a10 msgr2=0x7f061819ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.433+0000 7f061eb08640 1 --2- 172.21.15.43:0/3439498652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0618108a10 0x7f061819ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.433+0000 7f061eb08640 1 -- 172.21.15.43:0/3439498652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06181031c0 msgr2=0x7f061819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.433+0000 7f061eb08640 1 --2- 172.21.15.43:0/3439498652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06181031c0 0x7f061819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.433+0000 7f061eb08640 1 -- 172.21.15.43:0/3439498652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f061819f600 con 0x7f0618108eb0 2024-04-25T10:39:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.434+0000 7f061eb08640 1 --2- 172.21.15.43:0/3439498652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0618108eb0 0x7f061819efc0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f061000efc0 tx=0x7f061000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.434+0000 7f060f7fe640 1 -- 172.21.15.43:0/3439498652 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610003970 con 0x7f0618108eb0 2024-04-25T10:39:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.434+0000 7f062056b640 1 -- 172.21.15.43:0/3439498652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f061819f8f0 con 0x7f0618108eb0 2024-04-25T10:39:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.434+0000 7f060f7fe640 1 -- 172.21.15.43:0/3439498652 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0610004020 con 0x7f0618108eb0 2024-04-25T10:39:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.434+0000 7f060f7fe640 1 -- 172.21.15.43:0/3439498652 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061000b940 con 0x7f0618108eb0 2024-04-25T10:39:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.434+0000 7f062056b640 1 -- 172.21.15.43:0/3439498652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f061819fdc0 con 0x7f0618108eb0 2024-04-25T10:39:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.437+0000 7f0627418740 1 Processor -- start 2024-04-25T10:39:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.437+0000 7f060f7fe640 1 -- 172.21.15.43:0/3439498652 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f061001d020 con 0x7f0618108eb0 2024-04-25T10:39:07.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.437+0000 7f0627418740 1 -- start start 2024-04-25T10:39:07.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.437+0000 7f0627418740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628300a7390 0x5628304486a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.437+0000 7f0627418740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 0x562830448be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.437+0000 7f0627418740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628300866b0 0x562830459ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.437+0000 7f0627418740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5628303add80 con 0x56283007e4d0 2024-04-25T10:39:07.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.437+0000 7f0627418740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562830367a20 con 0x5628300866b0 2024-04-25T10:39:07.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.437+0000 7f0627418740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5628303a9d40 con 0x5628300a7390 2024-04-25T10:39:07.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.437+0000 7f061e307640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 0x562830448be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.438+0000 7f061e307640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 0x562830448be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49684/0 (socket says 172.21.15.43:49684) 2024-04-25T10:39:07.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.438+0000 7f061e307640 1 -- 172.21.15.43:0/3312839797 learned_addr learned my addr 172.21.15.43:0/3312839797 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.438+0000 7f061db06640 1 --2- 172.21.15.43:0/3312839797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628300a7390 0x5628304486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.438+0000 7f060f7fe640 1 --2- 172.21.15.43:0/3439498652 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f06040afc60 0x7f06040b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.438+0000 7f061db06640 1 --2- 172.21.15.43:0/3312839797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628300866b0 0x562830459ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.438+0000 7f061eb08640 1 --2- 172.21.15.43:0/3439498652 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f06040afc60 0x7f06040b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.438+0000 7f061e307640 1 -- 172.21.15.43:0/3312839797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628300a7390 msgr2=0x5628304486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.438+0000 7f061e307640 1 --2- 172.21.15.43:0/3312839797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628300a7390 0x5628304486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.438+0000 7f061e307640 1 -- 172.21.15.43:0/3312839797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628300866b0 msgr2=0x562830459ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.438+0000 7f061e307640 1 --2- 172.21.15.43:0/3312839797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628300866b0 0x562830459ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.438+0000 7f061e307640 1 -- 172.21.15.43:0/3312839797 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562830372c30 con 0x56283007e4d0 2024-04-25T10:39:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.438+0000 7f060f7fe640 1 -- 172.21.15.43:0/3439498652 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7f06100deec0 con 0x7f0618108eb0 2024-04-25T10:39:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.438+0000 7f061e307640 1 --2- 172.21.15.43:0/3312839797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 0x562830448be0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f061400bc70 tx=0x7f06140107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4966628c9c07b400 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.438+0000 7f060cbfe640 1 -- 172.21.15.43:0/3312839797 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614005220 con 0x56283007e4d0 2024-04-25T10:39:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.438+0000 7f061eb08640 1 --2- 172.21.15.43:0/3439498652 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f06040afc60 0x7f06040b2170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f06100a6d40 tx=0x7f06100047a0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.438+0000 7f060cbfe640 1 -- 172.21.15.43:0/3312839797 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f061400be20 con 0x56283007e4d0 2024-04-25T10:39:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.438+0000 7f060cbfe640 1 -- 172.21.15.43:0/3312839797 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06140116a0 con 0x56283007e4d0 2024-04-25T10:39:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.439+0000 7f0627418740 1 -- 172.21.15.43:0/3312839797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 msgr2=0x562830448be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.439+0000 7f0627418740 1 --2- 172.21.15.43:0/3312839797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 0x562830448be0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f061400bc70 tx=0x7f06140107a0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.439+0000 7f0627418740 1 -- 172.21.15.43:0/3312839797 shutdown_connections 2024-04-25T10:39:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.439+0000 7f0627418740 1 --2- 172.21.15.43:0/3312839797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628300866b0 0x562830459ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.439+0000 7f0627418740 1 --2- 172.21.15.43:0/3312839797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 0x562830448be0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.439+0000 7f0627418740 1 --2- 172.21.15.43:0/3312839797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628300a7390 0x5628304486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.439+0000 7f0627418740 1 -- 172.21.15.43:0/3312839797 >> 172.21.15.43:0/3312839797 conn(0x56283009b4e0 msgr2=0x562830444fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.439+0000 7f0627418740 1 -- 172.21.15.43:0/3312839797 shutdown_connections 2024-04-25T10:39:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.439+0000 7f0627418740 1 -- 172.21.15.43:0/3312839797 wait complete. 2024-04-25T10:39:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.440+0000 7f0627418740 1 -- start start 2024-04-25T10:39:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.440+0000 7f0627418740 1 Processor -- start 2024-04-25T10:39:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.440+0000 7f0627418740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 0x562830457690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.440+0000 7f0627418740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628300866b0 0x562830457bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.440+0000 7f0627418740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628300a7390 0x5628304636e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.440+0000 7f0627418740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56283045a1a0 con 0x56283007e4d0 2024-04-25T10:39:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.440+0000 7f0627418740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562830334dc0 con 0x5628300866b0 2024-04-25T10:39:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.440+0000 7f0627418740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56283045a320 con 0x5628300a7390 2024-04-25T10:39:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.440+0000 7f0627418740 10 client.0 _ll_register_callbacks cb 0x562830350600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.440+0000 7f061db06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 0x562830457690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.440+0000 7f061db06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 0x562830457690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49694/0 (socket says 172.21.15.43:49694) 2024-04-25T10:39:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.440+0000 7f061db06640 1 -- 172.21.15.43:0/382338347 learned_addr learned my addr 172.21.15.43:0/382338347 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.440+0000 7f061e307640 1 --2- 172.21.15.43:0/382338347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628300866b0 0x562830457bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.440+0000 7f061db06640 1 --2- 172.21.15.43:0/382338347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628300a7390 0x5628304636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.441+0000 7f061db06640 1 -- 172.21.15.43:0/382338347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628300a7390 msgr2=0x5628304636e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.441+0000 7f061db06640 1 --2- 172.21.15.43:0/382338347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628300a7390 0x5628304636e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.441+0000 7f061db06640 1 -- 172.21.15.43:0/382338347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628300866b0 msgr2=0x562830457bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.441+0000 7f061db06640 1 --2- 172.21.15.43:0/382338347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628300866b0 0x562830457bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.441+0000 7f061db06640 1 -- 172.21.15.43:0/382338347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628300bf3f0 con 0x56283007e4d0 2024-04-25T10:39:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.441+0000 7f061e307640 1 --2- 172.21.15.43:0/382338347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628300866b0 0x562830457bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.441+0000 7f061db06640 1 --2- 172.21.15.43:0/382338347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 0x562830457690 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f060802a930 tx=0x7f0608005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e095a1897f4b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.441+0000 7f0601ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.441+0000 7f0601ffb640 1 -- 172.21.15.43:0/382338347 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0608037040 con 0x56283007e4d0 2024-04-25T10:39:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.441+0000 7f0627418740 1 -- 172.21.15.43:0/382338347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f061400be20 con 0x56283007e4d0 2024-04-25T10:39:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.441+0000 7f0601ffb640 1 -- 172.21.15.43:0/382338347 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7f0608004250 con 0x56283007e4d0 2024-04-25T10:39:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.441+0000 7f0601ffb640 1 -- 172.21.15.43:0/382338347 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0608051050 con 0x56283007e4d0 2024-04-25T10:39:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.441+0000 7f0627418740 10 client.17394 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.441+0000 7f0627418740 1 -- 172.21.15.43:0/382338347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5628301897d0 con 0x56283007e4d0 2024-04-25T10:39:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.441+0000 7f0601ffb640 1 -- 172.21.15.43:0/382338347 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f061400be20 con 0x56283007e4d0 2024-04-25T10:39:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.442+0000 7f0601ffb640 1 -- 172.21.15.43:0/382338347 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f06080038c0 con 0x56283007e4d0 2024-04-25T10:39:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.442+0000 7f0627418740 10 client.17394 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.442+0000 7f0627418740 10 client.17394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.442+0000 7f0627418740 20 client.17394 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.442+0000 7f0627418740 1 --2- 172.21.15.43:0/382338347 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562830065f60 0x562830469b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.442+0000 7f0627418740 4 client.17394 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:39:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.442+0000 7f0627418740 1 -- 172.21.15.43:0/382338347 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628303bb990 con 0x562830065f60 2024-04-25T10:39:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.442+0000 7f061e307640 1 --2- 172.21.15.43:0/382338347 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562830065f60 0x562830469b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.443+0000 7f0601ffb640 10 client.17394 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.443+0000 7f061e307640 1 --2- 172.21.15.43:0/382338347 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562830065f60 0x562830469b10 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d54e62fec5786cd server_cookie=42c30ff09d8bf819 in_seq=0 out_seq=0 2024-04-25T10:39:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.443+0000 7f0601ffb640 1 -- 172.21.15.43:0/382338347 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5628303bb990 con 0x562830065f60 2024-04-25T10:39:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.443+0000 7f0601ffb640 10 client.17394 handle_command_reply: tid=0 2024-04-25T10:39:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.443+0000 7f0627418740 1 client.17394 shutdown 2024-04-25T10:39:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.443+0000 7f0627418740 1 -- 172.21.15.43:0/382338347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 msgr2=0x562830457690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.443+0000 7f0627418740 1 --2- 172.21.15.43:0/382338347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 0x562830457690 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f060802a930 tx=0x7f0608005f90 comp rx=0 tx=0).stop 2024-04-25T10:39:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.444+0000 7f0627418740 1 -- 172.21.15.43:0/382338347 shutdown_connections 2024-04-25T10:39:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.444+0000 7f0627418740 1 --2- 172.21.15.43:0/382338347 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562830065f60 0x562830469b10 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.444+0000 7f0627418740 1 --2- 172.21.15.43:0/382338347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628300866b0 0x562830457bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.444+0000 7f0627418740 1 --2- 172.21.15.43:0/382338347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628300a7390 0x5628304636e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.444+0000 7f0601ffb640 0 client.17394 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.444+0000 7f0627418740 1 --2- 172.21.15.43:0/382338347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 0x562830457690 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.444+0000 7f0627418740 1 -- 172.21.15.43:0/382338347 >> 172.21.15.43:0/382338347 conn(0x56283009b4e0 msgr2=0x56283044a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.444+0000 7f0627418740 1 -- 172.21.15.43:0/382338347 shutdown_connections 2024-04-25T10:39:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.444+0000 7f0627418740 1 -- 172.21.15.43:0/382338347 wait complete. 2024-04-25T10:39:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.444+0000 7f0627418740 20 client.17394 trim_cache size 0 max 16384 2024-04-25T10:39:07.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.457+0000 7f0627418740 1 Processor -- start 2024-04-25T10:39:07.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.457+0000 7f0627418740 1 -- start start 2024-04-25T10:39:07.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.458+0000 7f0627418740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56283007e4d0 0x562830444ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.458+0000 7f0627418740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628300866b0 0x5628304632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.458+0000 7f0627418740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628300a7390 0x56283046f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.458+0000 7f0627418740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06181b0120 con 0x5628300a7390 2024-04-25T10:39:07.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.458+0000 7f0627418740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f06181ad9f0 con 0x56283007e4d0 2024-04-25T10:39:07.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.458+0000 7f0627418740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f06181ab880 con 0x5628300866b0 2024-04-25T10:39:07.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.458+0000 7f061db06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628300a7390 0x56283046f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.458+0000 7f061db06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628300a7390 0x56283046f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49698/0 (socket says 172.21.15.43:49698) 2024-04-25T10:39:07.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.458+0000 7f061db06640 1 -- 172.21.15.43:0/2826943602 learned_addr learned my addr 172.21.15.43:0/2826943602 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:07.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.458+0000 7f061db06640 1 --2- 172.21.15.43:0/2826943602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56283007e4d0 0x562830444ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.458+0000 7f061e307640 1 --2- 172.21.15.43:0/2826943602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628300866b0 0x5628304632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.458+0000 7f061db06640 1 -- 172.21.15.43:0/2826943602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628300866b0 msgr2=0x5628304632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.458+0000 7f061db06640 1 --2- 172.21.15.43:0/2826943602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628300866b0 0x5628304632c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.458+0000 7f061db06640 1 -- 172.21.15.43:0/2826943602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56283007e4d0 msgr2=0x562830444ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.458+0000 7f061db06640 1 --2- 172.21.15.43:0/2826943602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56283007e4d0 0x562830444ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.458+0000 7f061db06640 1 -- 172.21.15.43:0/2826943602 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562830453040 con 0x5628300a7390 2024-04-25T10:39:07.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.458+0000 7f061db06640 1 --2- 172.21.15.43:0/2826943602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628300a7390 0x56283046f2d0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f060802a860 tx=0x7f060802cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdcaa0072c793a9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:07.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.458+0000 7f0603fff640 1 -- 172.21.15.43:0/2826943602 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0608037070 con 0x5628300a7390 2024-04-25T10:39:07.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.458+0000 7f0603fff640 1 -- 172.21.15.43:0/2826943602 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f060802c110 con 0x5628300a7390 2024-04-25T10:39:07.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.458+0000 7f0603fff640 1 -- 172.21.15.43:0/2826943602 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0608003850 con 0x5628300a7390 2024-04-25T10:39:07.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.459+0000 7f0627418740 1 -- 172.21.15.43:0/2826943602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628300a7390 msgr2=0x56283046f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.459+0000 7f0627418740 1 --2- 172.21.15.43:0/2826943602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628300a7390 0x56283046f2d0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f060802a860 tx=0x7f060802cd80 comp rx=0 tx=0).stop 2024-04-25T10:39:07.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.459+0000 7f0627418740 1 -- 172.21.15.43:0/2826943602 shutdown_connections 2024-04-25T10:39:07.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.459+0000 7f0627418740 1 --2- 172.21.15.43:0/2826943602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628300a7390 0x56283046f2d0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.459+0000 7f0627418740 1 --2- 172.21.15.43:0/2826943602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628300866b0 0x5628304632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.459+0000 7f0627418740 1 --2- 172.21.15.43:0/2826943602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56283007e4d0 0x562830444ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.459+0000 7f0627418740 1 -- 172.21.15.43:0/2826943602 >> 172.21.15.43:0/2826943602 conn(0x56283009b4e0 msgr2=0x5628304447c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:07.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.459+0000 7f0627418740 1 -- 172.21.15.43:0/2826943602 shutdown_connections 2024-04-25T10:39:07.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.459+0000 7f0627418740 1 -- 172.21.15.43:0/2826943602 wait complete. 2024-04-25T10:39:07.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f0627418740 1 -- start start 2024-04-25T10:39:07.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f0627418740 1 Processor -- start 2024-04-25T10:39:07.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f0627418740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 0x56283045ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f0627418740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628300866b0 0x56283045b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f0627418740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628300a7390 0x5628304cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f0627418740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06181abed0 con 0x56283007e4d0 2024-04-25T10:39:07.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f0627418740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f06181ad320 con 0x5628300866b0 2024-04-25T10:39:07.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f0627418740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f06181ac580 con 0x5628300a7390 2024-04-25T10:39:07.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f0627418740 10 client.0 _ll_register_callbacks cb 0x7f06080096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:07.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f061db06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 0x56283045ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f061db06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 0x56283045ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49714/0 (socket says 172.21.15.43:49714) 2024-04-25T10:39:07.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f061db06640 1 -- 172.21.15.43:0/1922876006 learned_addr learned my addr 172.21.15.43:0/1922876006 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:07.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f061db06640 1 --2- 172.21.15.43:0/1922876006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628300a7390 0x5628304cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f061e307640 1 --2- 172.21.15.43:0/1922876006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628300866b0 0x56283045b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f061db06640 1 -- 172.21.15.43:0/1922876006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628300a7390 msgr2=0x5628304cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f061db06640 1 --2- 172.21.15.43:0/1922876006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628300a7390 0x5628304cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f061db06640 1 -- 172.21.15.43:0/1922876006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628300866b0 msgr2=0x56283045b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:07.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f061db06640 1 --2- 172.21.15.43:0/1922876006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628300866b0 0x56283045b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f061db06640 1 -- 172.21.15.43:0/1922876006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628304d03b0 con 0x56283007e4d0 2024-04-25T10:39:07.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f061db06640 1 --2- 172.21.15.43:0/1922876006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 0x56283045ac30 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f0608035040 tx=0x7f0608009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fee999eb395dbe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:07.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.460+0000 7f0601ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:07.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.461+0000 7f0601ffb640 1 -- 172.21.15.43:0/1922876006 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0608037040 con 0x56283007e4d0 2024-04-25T10:39:07.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.461+0000 7f0601ffb640 1 -- 172.21.15.43:0/1922876006 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7f0608046040 con 0x56283007e4d0 2024-04-25T10:39:07.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.461+0000 7f0627418740 1 -- 172.21.15.43:0/1922876006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f060802c110 con 0x56283007e4d0 2024-04-25T10:39:07.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.461+0000 7f0601ffb640 1 -- 172.21.15.43:0/1922876006 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0608065050 con 0x56283007e4d0 2024-04-25T10:39:07.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.461+0000 7f0627418740 10 client.17400 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:07.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.461+0000 7f0627418740 1 -- 172.21.15.43:0/1922876006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5628304d1600 con 0x56283007e4d0 2024-04-25T10:39:07.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.461+0000 7f0601ffb640 1 -- 172.21.15.43:0/1922876006 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f060802c110 con 0x56283007e4d0 2024-04-25T10:39:07.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.461+0000 7f0601ffb640 1 -- 172.21.15.43:0/1922876006 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f060802c580 con 0x56283007e4d0 2024-04-25T10:39:07.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.461+0000 7f0627418740 10 client.17400 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:07.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.461+0000 7f0627418740 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:07.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.461+0000 7f0627418740 20 client.17400 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:07.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.461+0000 7f0627418740 1 --2- 172.21.15.43:0/1922876006 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56283003b3a0 0x5628304d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:07.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.461+0000 7f0627418740 4 client.17400 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:39:07.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.461+0000 7f0627418740 1 -- 172.21.15.43:0/1922876006 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628303b5800 con 0x56283003b3a0 2024-04-25T10:39:07.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.462+0000 7f061e307640 1 --2- 172.21.15.43:0/1922876006 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56283003b3a0 0x5628304d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:07.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.462+0000 7f061e307640 1 --2- 172.21.15.43:0/1922876006 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56283003b3a0 0x5628304d3c80 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d53ac210b46aefc server_cookie=6edb0b89a23a6c33 in_seq=0 out_seq=0 2024-04-25T10:39:07.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.462+0000 7f0601ffb640 10 client.17400 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:07.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.462+0000 7f0601ffb640 1 -- 172.21.15.43:0/1922876006 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628303b5800 con 0x56283003b3a0 2024-04-25T10:39:07.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.462+0000 7f0601ffb640 10 client.17400 handle_command_reply: tid=0 2024-04-25T10:39:07.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.462+0000 7f0627418740 1 client.17400 shutdown 2024-04-25T10:39:07.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.462+0000 7f0627418740 1 -- 172.21.15.43:0/1922876006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 msgr2=0x56283045ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.462+0000 7f0627418740 1 --2- 172.21.15.43:0/1922876006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 0x56283045ac30 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f0608035040 tx=0x7f0608009450 comp rx=0 tx=0).stop 2024-04-25T10:39:07.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.462+0000 7f0627418740 1 -- 172.21.15.43:0/1922876006 shutdown_connections 2024-04-25T10:39:07.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.462+0000 7f0627418740 1 --2- 172.21.15.43:0/1922876006 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56283003b3a0 0x5628304d3c80 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.462+0000 7f0627418740 1 --2- 172.21.15.43:0/1922876006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628300866b0 0x56283045b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.462+0000 7f0627418740 1 --2- 172.21.15.43:0/1922876006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628300a7390 0x5628304cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.462+0000 7f0601ffb640 0 client.17400 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:07.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.462+0000 7f0627418740 1 --2- 172.21.15.43:0/1922876006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56283007e4d0 0x56283045ac30 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.462+0000 7f0627418740 1 -- 172.21.15.43:0/1922876006 >> 172.21.15.43:0/1922876006 conn(0x56283009b4e0 msgr2=0x562830443610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:07.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.462+0000 7f0627418740 1 -- 172.21.15.43:0/1922876006 shutdown_connections 2024-04-25T10:39:07.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.462+0000 7f0627418740 1 -- 172.21.15.43:0/1922876006 wait complete. 2024-04-25T10:39:07.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.462+0000 7f0627418740 20 client.17400 trim_cache size 0 max 16384 2024-04-25T10:39:07.481 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:39:07.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.463+0000 7f062056b640 1 -- 172.21.15.43:0/3439498652 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f06040afc60 msgr2=0x7f06040b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.463+0000 7f062056b640 1 --2- 172.21.15.43:0/3439498652 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f06040afc60 0x7f06040b2170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f06100a6d40 tx=0x7f06100047a0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.463+0000 7f062056b640 1 -- 172.21.15.43:0/3439498652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0618108eb0 msgr2=0x7f061819efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:07.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.463+0000 7f062056b640 1 --2- 172.21.15.43:0/3439498652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0618108eb0 0x7f061819efc0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f061000efc0 tx=0x7f061000c800 comp rx=0 tx=0).stop 2024-04-25T10:39:07.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.463+0000 7f062056b640 1 -- 172.21.15.43:0/3439498652 shutdown_connections 2024-04-25T10:39:07.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.463+0000 7f062056b640 1 --2- 172.21.15.43:0/3439498652 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f06040afc60 0x7f06040b2170 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.463+0000 7f062056b640 1 --2- 172.21.15.43:0/3439498652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0618108eb0 0x7f061819efc0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.463+0000 7f062056b640 1 --2- 172.21.15.43:0/3439498652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0618108a10 0x7f061819ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.463+0000 7f062056b640 1 --2- 172.21.15.43:0/3439498652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06181031c0 0x7f061819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:07.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.463+0000 7f062056b640 1 -- 172.21.15.43:0/3439498652 >> 172.21.15.43:0/3439498652 conn(0x7f0618078110 msgr2=0x7f0618074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:07.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.463+0000 7f062056b640 1 -- 172.21.15.43:0/3439498652 shutdown_connections 2024-04-25T10:39:07.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:07.463+0000 7f062056b640 1 -- 172.21.15.43:0/3439498652 wait complete. 2024-04-25T10:39:08.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:08 smithi043 ceph-mon[29478]: pgmap v202: 97 pgs: 97 active+clean; 2.0 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 95 KiB/s wr, 20 op/s 2024-04-25T10:39:08.486 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:39:08.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:39:08.487 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:39:08.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:08 smithi077 ceph-mon[32772]: pgmap v202: 97 pgs: 97 active+clean; 2.0 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 95 KiB/s wr, 20 op/s 2024-04-25T10:39:08.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:08 smithi164 ceph-mon[32713]: pgmap v202: 97 pgs: 97 active+clean; 2.0 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 95 KiB/s wr, 20 op/s 2024-04-25T10:39:08.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.616+0000 7fe06724c640 1 -- 172.21.15.43:0/1291170530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe060109fb0 msgr2=0x7fe060103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:08.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.616+0000 7fe06724c640 1 --2- 172.21.15.43:0/1291170530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe060109fb0 0x7fe060103bf0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009900 tx=0x7fe04c02a410 comp rx=0 tx=0).stop 2024-04-25T10:39:08.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.616+0000 7fe06724c640 1 -- 172.21.15.43:0/1291170530 shutdown_connections 2024-04-25T10:39:08.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.616+0000 7fe06724c640 1 --2- 172.21.15.43:0/1291170530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe060104260 0x7fe060110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.616+0000 7fe06724c640 1 --2- 172.21.15.43:0/1291170530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe060109fb0 0x7fe060103bf0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.616+0000 7fe06724c640 1 --2- 172.21.15.43:0/1291170530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe060109b10 0x7fe0601036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.616+0000 7fe06724c640 1 -- 172.21.15.43:0/1291170530 >> 172.21.15.43:0/1291170530 conn(0x7fe0600ff210 msgr2=0x7fe060101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:08.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.617+0000 7fe06724c640 1 -- 172.21.15.43:0/1291170530 shutdown_connections 2024-04-25T10:39:08.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.617+0000 7fe06724c640 1 -- 172.21.15.43:0/1291170530 wait complete. 2024-04-25T10:39:08.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.617+0000 7fe06724c640 1 Processor -- start 2024-04-25T10:39:08.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.617+0000 7fe06724c640 1 -- start start 2024-04-25T10:39:08.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.617+0000 7fe06724c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe060104260 0x7fe0601a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.617+0000 7fe06724c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe060109b10 0x7fe0601a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.618+0000 7fe06724c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe060109fb0 0x7fe06019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.618+0000 7fe06724c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe060113d00 con 0x7fe060104260 2024-04-25T10:39:08.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.618+0000 7fe06724c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe060113b80 con 0x7fe060109fb0 2024-04-25T10:39:08.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.618+0000 7fe06724c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe060113e80 con 0x7fe060109b10 2024-04-25T10:39:08.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.618+0000 7fe064fe8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe060104260 0x7fe0601a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.618+0000 7fe064fe8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe060104260 0x7fe0601a04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49738/0 (socket says 172.21.15.43:49738) 2024-04-25T10:39:08.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.618+0000 7fe064fe8640 1 -- 172.21.15.43:0/134359143 learned_addr learned my addr 172.21.15.43:0/134359143 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:08.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.618+0000 7fe05ffff640 1 --2- 172.21.15.43:0/134359143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe060109b10 0x7fe0601a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.618+0000 7fe0657e9640 1 --2- 172.21.15.43:0/134359143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe060109fb0 0x7fe06019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.618+0000 7fe064fe8640 1 -- 172.21.15.43:0/134359143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe060109b10 msgr2=0x7fe0601a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:08.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.618+0000 7fe064fe8640 1 --2- 172.21.15.43:0/134359143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe060109b10 0x7fe0601a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.618+0000 7fe064fe8640 1 -- 172.21.15.43:0/134359143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe060109fb0 msgr2=0x7fe06019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:08.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.618+0000 7fe064fe8640 1 --2- 172.21.15.43:0/134359143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe060109fb0 0x7fe06019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.618+0000 7fe064fe8640 1 -- 172.21.15.43:0/134359143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06019ad90 con 0x7fe060104260 2024-04-25T10:39:08.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.618+0000 7fe064fe8640 1 --2- 172.21.15.43:0/134359143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe060104260 0x7fe0601a04e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fe05400d770 tx=0x7fe05400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:08.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.618+0000 7fe05dffb640 1 -- 172.21.15.43:0/134359143 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe054017070 con 0x7fe060104260 2024-04-25T10:39:08.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.618+0000 7fe05dffb640 1 -- 172.21.15.43:0/134359143 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe054009c30 con 0x7fe060104260 2024-04-25T10:39:08.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.618+0000 7fe06724c640 1 -- 172.21.15.43:0/134359143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06019b020 con 0x7fe060104260 2024-04-25T10:39:08.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.618+0000 7fe05dffb640 1 -- 172.21.15.43:0/134359143 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05400f620 con 0x7fe060104260 2024-04-25T10:39:08.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.619+0000 7fe06724c640 1 -- 172.21.15.43:0/134359143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0601a7280 con 0x7fe060104260 2024-04-25T10:39:08.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.621+0000 7fe06e1db740 1 Processor -- start 2024-04-25T10:39:08.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.621+0000 7fe06e1db740 1 -- start start 2024-04-25T10:39:08.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.621+0000 7fe05dffb640 1 -- 172.21.15.43:0/134359143 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fe05401a070 con 0x7fe060104260 2024-04-25T10:39:08.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.622+0000 7fe06e1db740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642fd410 0x56106469d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.622+0000 7fe06e1db740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642d4550 0x56106469d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.622+0000 7fe06e1db740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642dc730 0x56106469df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.622+0000 7fe06e1db740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561064603b00 con 0x5610642fd410 2024-04-25T10:39:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.622+0000 7fe06e1db740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56106459f920 con 0x5610642dc730 2024-04-25T10:39:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.622+0000 7fe06e1db740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5610645ffac0 con 0x5610642d4550 2024-04-25T10:39:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.622+0000 7fe0657e9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642fd410 0x56106469d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.622+0000 7fe0657e9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642fd410 0x56106469d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49752/0 (socket says 172.21.15.43:49752) 2024-04-25T10:39:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.622+0000 7fe0657e9640 1 -- 172.21.15.43:0/3840257931 learned_addr learned my addr 172.21.15.43:0/3840257931 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.622+0000 7fe05ffff640 1 --2- 172.21.15.43:0/3840257931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642dc730 0x56106469df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.622+0000 7fe064fe8640 1 --2- 172.21.15.43:0/3840257931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642d4550 0x56106469d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.622+0000 7fe0657e9640 1 -- 172.21.15.43:0/3840257931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642d4550 msgr2=0x56106469d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.622+0000 7fe0657e9640 1 --2- 172.21.15.43:0/3840257931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642d4550 0x56106469d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.622+0000 7fe0657e9640 1 -- 172.21.15.43:0/3840257931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642dc730 msgr2=0x56106469df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.622+0000 7fe0657e9640 1 --2- 172.21.15.43:0/3840257931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642dc730 0x56106469df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.622+0000 7fe05dffb640 1 --2- 172.21.15.43:0/134359143 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe0500afce0 0x7fe0500b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.622+0000 7fe0657e9640 1 -- 172.21.15.43:0/3840257931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610643dd7e0 con 0x5610642fd410 2024-04-25T10:39:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.622+0000 7fe05ffff640 1 --2- 172.21.15.43:0/3840257931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642dc730 0x56106469df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:39:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.622+0000 7fe05dffb640 1 -- 172.21.15.43:0/134359143 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7fe0540dea30 con 0x7fe060104260 2024-04-25T10:39:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.623+0000 7fe0657e9640 1 --2- 172.21.15.43:0/134359143 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe0500afce0 0x7fe0500b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.623+0000 7fe0657e9640 1 --2- 172.21.15.43:0/3840257931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642fd410 0x56106469d4b0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fe05800ae90 tx=0x7fe05800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338c17fe96c6e33d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.623+0000 7fe04affd640 1 -- 172.21.15.43:0/3840257931 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05801e070 con 0x5610642fd410 2024-04-25T10:39:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.623+0000 7fe04affd640 1 -- 172.21.15.43:0/3840257931 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe058010d40 con 0x5610642fd410 2024-04-25T10:39:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.623+0000 7fe04affd640 1 -- 172.21.15.43:0/3840257931 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058017860 con 0x5610642fd410 2024-04-25T10:39:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.624+0000 7fe06e1db740 1 -- 172.21.15.43:0/3840257931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642fd410 msgr2=0x56106469d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.624+0000 7fe06e1db740 1 --2- 172.21.15.43:0/3840257931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642fd410 0x56106469d4b0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fe05800ae90 tx=0x7fe05800f790 comp rx=0 tx=0).stop 2024-04-25T10:39:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.624+0000 7fe0657e9640 1 --2- 172.21.15.43:0/134359143 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe0500afce0 0x7fe0500b21f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe05801b000 tx=0x7fe058017f10 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.624+0000 7fe06e1db740 1 -- 172.21.15.43:0/3840257931 shutdown_connections 2024-04-25T10:39:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.624+0000 7fe06e1db740 1 --2- 172.21.15.43:0/3840257931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642dc730 0x56106469df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.624+0000 7fe06e1db740 1 --2- 172.21.15.43:0/3840257931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642d4550 0x56106469d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.624+0000 7fe06e1db740 1 --2- 172.21.15.43:0/3840257931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642fd410 0x56106469d4b0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.624+0000 7fe06e1db740 1 -- 172.21.15.43:0/3840257931 >> 172.21.15.43:0/3840257931 conn(0x5610642f1560 msgr2=0x561064699dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.624+0000 7fe06e1db740 1 -- 172.21.15.43:0/3840257931 shutdown_connections 2024-04-25T10:39:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.624+0000 7fe06e1db740 1 -- 172.21.15.43:0/3840257931 wait complete. 2024-04-25T10:39:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.624+0000 7fe06e1db740 1 -- start start 2024-04-25T10:39:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.624+0000 7fe06e1db740 1 Processor -- start 2024-04-25T10:39:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.625+0000 7fe06e1db740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642d4550 0x5610646ac000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.625+0000 7fe06e1db740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642dc730 0x5610646ac540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.625+0000 7fe06e1db740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642fd410 0x5610646b8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.625+0000 7fe06e1db740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5610646aed80 con 0x5610642fd410 2024-04-25T10:39:08.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.625+0000 7fe06e1db740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56106459d5d0 con 0x5610642d4550 2024-04-25T10:39:08.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.625+0000 7fe06e1db740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5610646aef00 con 0x5610642dc730 2024-04-25T10:39:08.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.625+0000 7fe06e1db740 10 client.0 _ll_register_callbacks cb 0x56106458b080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:08.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.625+0000 7fe0657e9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642fd410 0x5610646b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.625+0000 7fe0657e9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642fd410 0x5610646b8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49756/0 (socket says 172.21.15.43:49756) 2024-04-25T10:39:08.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.625+0000 7fe0657e9640 1 -- 172.21.15.43:0/425250767 learned_addr learned my addr 172.21.15.43:0/425250767 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:08.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.625+0000 7fe064fe8640 1 --2- 172.21.15.43:0/425250767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642d4550 0x5610646ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.625+0000 7fe05ffff640 1 --2- 172.21.15.43:0/425250767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642dc730 0x5610646ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.625+0000 7fe0657e9640 1 -- 172.21.15.43:0/425250767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642dc730 msgr2=0x5610646ac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:08.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.625+0000 7fe0657e9640 1 --2- 172.21.15.43:0/425250767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642dc730 0x5610646ac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.625+0000 7fe0657e9640 1 -- 172.21.15.43:0/425250767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642d4550 msgr2=0x5610646ac000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.625+0000 7fe0657e9640 1 --2- 172.21.15.43:0/425250767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642d4550 0x5610646ac000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.625+0000 7fe0657e9640 1 -- 172.21.15.43:0/425250767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56106443df70 con 0x5610642fd410 2024-04-25T10:39:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.625+0000 7fe0657e9640 1 --2- 172.21.15.43:0/425250767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642fd410 0x5610646b8230 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fe05801e1e0 tx=0x7fe058022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f28ba835939a350 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.625+0000 7fe03ffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.626+0000 7fe03ffff640 1 -- 172.21.15.43:0/425250767 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05801e040 con 0x5610642fd410 2024-04-25T10:39:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.626+0000 7fe06e1db740 1 -- 172.21.15.43:0/425250767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe058010d40 con 0x5610642fd410 2024-04-25T10:39:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.626+0000 7fe03ffff640 1 -- 172.21.15.43:0/425250767 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7fe0580169e0 con 0x5610642fd410 2024-04-25T10:39:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.626+0000 7fe03ffff640 1 -- 172.21.15.43:0/425250767 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe05803c050 con 0x5610642fd410 2024-04-25T10:39:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.626+0000 7fe06e1db740 10 client.17412 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.626+0000 7fe06e1db740 1 -- 172.21.15.43:0/425250767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5610642d5170 con 0x5610642fd410 2024-04-25T10:39:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.626+0000 7fe03ffff640 1 -- 172.21.15.43:0/425250767 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe058010d40 con 0x5610642fd410 2024-04-25T10:39:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.626+0000 7fe03ffff640 1 -- 172.21.15.43:0/425250767 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe058016c70 con 0x5610642fd410 2024-04-25T10:39:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.626+0000 7fe06e1db740 10 client.17412 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:08.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.626+0000 7fe06e1db740 10 client.17412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:08.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.626+0000 7fe06e1db740 20 client.17412 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:08.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.626+0000 7fe06e1db740 1 --2- 172.21.15.43:0/425250767 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5610642bc1f0 0x5610646be3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.626+0000 7fe06e1db740 4 client.17412 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:39:08.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.626+0000 7fe06e1db740 1 -- 172.21.15.43:0/425250767 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561064611710 con 0x5610642bc1f0 2024-04-25T10:39:08.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.627+0000 7fe064fe8640 1 --2- 172.21.15.43:0/425250767 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5610642bc1f0 0x5610646be3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.627+0000 7fe064fe8640 1 --2- 172.21.15.43:0/425250767 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5610642bc1f0 0x5610646be3b0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62a42a86853c62f server_cookie=f846352ab184303c in_seq=0 out_seq=0 2024-04-25T10:39:08.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.627+0000 7fe03ffff640 10 client.17412 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:08.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.628+0000 7fe03ffff640 1 -- 172.21.15.43:0/425250767 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561064611710 con 0x5610642bc1f0 2024-04-25T10:39:08.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.628+0000 7fe03ffff640 10 client.17412 handle_command_reply: tid=0 2024-04-25T10:39:08.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.628+0000 7fe06e1db740 1 client.17412 shutdown 2024-04-25T10:39:08.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.628+0000 7fe06e1db740 1 -- 172.21.15.43:0/425250767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642fd410 msgr2=0x5610646b8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:08.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.628+0000 7fe06e1db740 1 --2- 172.21.15.43:0/425250767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642fd410 0x5610646b8230 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fe05801e1e0 tx=0x7fe058022000 comp rx=0 tx=0).stop 2024-04-25T10:39:08.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.628+0000 7fe06e1db740 1 -- 172.21.15.43:0/425250767 shutdown_connections 2024-04-25T10:39:08.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.628+0000 7fe06e1db740 1 --2- 172.21.15.43:0/425250767 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5610642bc1f0 0x5610646be3b0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.628+0000 7fe06e1db740 1 --2- 172.21.15.43:0/425250767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642fd410 0x5610646b8230 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.628+0000 7fe06e1db740 1 --2- 172.21.15.43:0/425250767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642dc730 0x5610646ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.628+0000 7fe03ffff640 0 client.17412 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:08.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.628+0000 7fe06e1db740 1 --2- 172.21.15.43:0/425250767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642d4550 0x5610646ac000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.628+0000 7fe06e1db740 1 -- 172.21.15.43:0/425250767 >> 172.21.15.43:0/425250767 conn(0x5610642f1560 msgr2=0x56106469f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:08.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.628+0000 7fe06e1db740 1 -- 172.21.15.43:0/425250767 shutdown_connections 2024-04-25T10:39:08.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.628+0000 7fe06e1db740 1 -- 172.21.15.43:0/425250767 wait complete. 2024-04-25T10:39:08.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.628+0000 7fe06e1db740 20 client.17412 trim_cache size 0 max 16384 2024-04-25T10:39:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.646+0000 7fe06e1db740 1 Processor -- start 2024-04-25T10:39:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.647+0000 7fe06e1db740 1 -- start start 2024-04-25T10:39:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.647+0000 7fe06e1db740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642d4550 0x561064698b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.647+0000 7fe06e1db740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642dc730 0x5610646c3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.647+0000 7fe06e1db740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642fd410 0x5610646c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.647+0000 7fe06e1db740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601b0210 con 0x5610642dc730 2024-04-25T10:39:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.647+0000 7fe06e1db740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601adae0 con 0x5610642d4550 2024-04-25T10:39:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.647+0000 7fe06e1db740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601ab970 con 0x5610642fd410 2024-04-25T10:39:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.647+0000 7fe05ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642dc730 0x5610646c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.647+0000 7fe05ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642dc730 0x5610646c3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49768/0 (socket says 172.21.15.43:49768) 2024-04-25T10:39:08.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.647+0000 7fe05ffff640 1 -- 172.21.15.43:0/503038367 learned_addr learned my addr 172.21.15.43:0/503038367 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:08.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.647+0000 7fe0657e9640 1 --2- 172.21.15.43:0/503038367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642fd410 0x5610646c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.647+0000 7fe064fe8640 1 --2- 172.21.15.43:0/503038367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642d4550 0x561064698b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.647+0000 7fe05ffff640 1 -- 172.21.15.43:0/503038367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642fd410 msgr2=0x5610646c3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:08.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.647+0000 7fe05ffff640 1 --2- 172.21.15.43:0/503038367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642fd410 0x5610646c3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.647+0000 7fe05ffff640 1 -- 172.21.15.43:0/503038367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642d4550 msgr2=0x561064698b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:08.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.647+0000 7fe05ffff640 1 --2- 172.21.15.43:0/503038367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642d4550 0x561064698b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.647+0000 7fe05ffff640 1 -- 172.21.15.43:0/503038367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56106469a9a0 con 0x5610642dc730 2024-04-25T10:39:08.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.647+0000 7fe0657e9640 1 --2- 172.21.15.43:0/503038367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642fd410 0x5610646c3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:39:08.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.647+0000 7fe05ffff640 1 --2- 172.21.15.43:0/503038367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642dc730 0x5610646c3950 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009900 tx=0x7fe04c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76362434e02f57d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:08.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.648+0000 7fe04a7fc640 1 -- 172.21.15.43:0/503038367 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c035040 con 0x5610642dc730 2024-04-25T10:39:08.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.648+0000 7fe04a7fc640 1 -- 172.21.15.43:0/503038367 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe04c031dd0 con 0x5610642dc730 2024-04-25T10:39:08.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.648+0000 7fe04a7fc640 1 -- 172.21.15.43:0/503038367 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c0054e0 con 0x5610642dc730 2024-04-25T10:39:08.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.648+0000 7fe06e1db740 1 -- 172.21.15.43:0/503038367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642dc730 msgr2=0x5610646c3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:08.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.648+0000 7fe06e1db740 1 --2- 172.21.15.43:0/503038367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642dc730 0x5610646c3950 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009900 tx=0x7fe04c000bc0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.649+0000 7fe06e1db740 1 -- 172.21.15.43:0/503038367 shutdown_connections 2024-04-25T10:39:08.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.649+0000 7fe06e1db740 1 --2- 172.21.15.43:0/503038367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642fd410 0x5610646c3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.649+0000 7fe06e1db740 1 --2- 172.21.15.43:0/503038367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642dc730 0x5610646c3950 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.649+0000 7fe06e1db740 1 --2- 172.21.15.43:0/503038367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642d4550 0x561064698b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.649+0000 7fe06e1db740 1 -- 172.21.15.43:0/503038367 >> 172.21.15.43:0/503038367 conn(0x5610642f1560 msgr2=0x56106469dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.649+0000 7fe06e1db740 1 -- 172.21.15.43:0/503038367 shutdown_connections 2024-04-25T10:39:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.649+0000 7fe06e1db740 1 -- 172.21.15.43:0/503038367 wait complete. 2024-04-25T10:39:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.649+0000 7fe06e1db740 1 -- start start 2024-04-25T10:39:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.649+0000 7fe06e1db740 1 Processor -- start 2024-04-25T10:39:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.650+0000 7fe06e1db740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642d4550 0x5610646b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.650+0000 7fe06e1db740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642dc730 0x5610646af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.650+0000 7fe06e1db740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642fd410 0x5610646aff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.650+0000 7fe06e1db740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601abfc0 con 0x5610642d4550 2024-04-25T10:39:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.650+0000 7fe06e1db740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601ad410 con 0x5610642fd410 2024-04-25T10:39:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.650+0000 7fe06e1db740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601ac670 con 0x5610642dc730 2024-04-25T10:39:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.650+0000 7fe06e1db740 10 client.0 _ll_register_callbacks cb 0x7fe0540e2450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.650+0000 7fe064fe8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642d4550 0x5610646b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.650+0000 7fe064fe8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642d4550 0x5610646b4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49782/0 (socket says 172.21.15.43:49782) 2024-04-25T10:39:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.650+0000 7fe064fe8640 1 -- 172.21.15.43:0/3999232447 learned_addr learned my addr 172.21.15.43:0/3999232447 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.650+0000 7fe05ffff640 1 --2- 172.21.15.43:0/3999232447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642dc730 0x5610646af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.650+0000 7fe0657e9640 1 --2- 172.21.15.43:0/3999232447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642fd410 0x5610646aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.650+0000 7fe064fe8640 1 -- 172.21.15.43:0/3999232447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642dc730 msgr2=0x5610646af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.650+0000 7fe064fe8640 1 --2- 172.21.15.43:0/3999232447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642dc730 0x5610646af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.650+0000 7fe064fe8640 1 -- 172.21.15.43:0/3999232447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642fd410 msgr2=0x5610646aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.650+0000 7fe064fe8640 1 --2- 172.21.15.43:0/3999232447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642fd410 0x5610646aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.650+0000 7fe064fe8640 1 -- 172.21.15.43:0/3999232447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610646b0540 con 0x5610642d4550 2024-04-25T10:39:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.650+0000 7fe0657e9640 1 --2- 172.21.15.43:0/3999232447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642fd410 0x5610646aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:39:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.650+0000 7fe064fe8640 1 --2- 172.21.15.43:0/3999232447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642d4550 0x5610646b4620 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fe0540a6960 tx=0x7fe054005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b179120e41433fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.650+0000 7fe03ffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.651+0000 7fe03ffff640 1 -- 172.21.15.43:0/3999232447 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe054021040 con 0x5610642d4550 2024-04-25T10:39:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.651+0000 7fe03ffff640 1 -- 172.21.15.43:0/3999232447 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7fe0540a5ec0 con 0x5610642d4550 2024-04-25T10:39:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.651+0000 7fe06e1db740 1 -- 172.21.15.43:0/3999232447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe04c031dd0 con 0x5610642d4550 2024-04-25T10:39:08.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.651+0000 7fe03ffff640 1 -- 172.21.15.43:0/3999232447 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe0540c7e60 con 0x5610642d4550 2024-04-25T10:39:08.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.651+0000 7fe06e1db740 10 client.17418 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:08.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.651+0000 7fe06e1db740 1 -- 172.21.15.43:0/3999232447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561064726300 con 0x5610642d4550 2024-04-25T10:39:08.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.651+0000 7fe03ffff640 1 -- 172.21.15.43:0/3999232447 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe04c031dd0 con 0x5610642d4550 2024-04-25T10:39:08.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.651+0000 7fe03ffff640 1 -- 172.21.15.43:0/3999232447 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe0580106c0 con 0x5610642d4550 2024-04-25T10:39:08.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.651+0000 7fe06e1db740 10 client.17418 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:08.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.651+0000 7fe06e1db740 10 client.17418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:08.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.651+0000 7fe06e1db740 20 client.17418 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:08.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.651+0000 7fe06e1db740 1 --2- 172.21.15.43:0/3999232447 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561064291630 0x561064728a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.651+0000 7fe06e1db740 4 client.17418 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:39:08.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.651+0000 7fe06e1db740 1 -- 172.21.15.43:0/3999232447 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56106460b580 con 0x561064291630 2024-04-25T10:39:08.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.652+0000 7fe05ffff640 1 --2- 172.21.15.43:0/3999232447 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561064291630 0x561064728a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.652+0000 7fe03ffff640 10 client.17418 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:08.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.652+0000 7fe05ffff640 1 --2- 172.21.15.43:0/3999232447 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561064291630 0x561064728a40 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa52bad7318a1bb3 server_cookie=108417a4de48974a in_seq=0 out_seq=0 2024-04-25T10:39:08.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.653+0000 7fe03ffff640 1 -- 172.21.15.43:0/3999232447 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56106460b580 con 0x561064291630 2024-04-25T10:39:08.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.653+0000 7fe03ffff640 10 client.17418 handle_command_reply: tid=0 2024-04-25T10:39:08.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.653+0000 7fe06e1db740 1 client.17418 shutdown 2024-04-25T10:39:08.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.653+0000 7fe06e1db740 1 -- 172.21.15.43:0/3999232447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642d4550 msgr2=0x5610646b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:08.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.653+0000 7fe06e1db740 1 --2- 172.21.15.43:0/3999232447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642d4550 0x5610646b4620 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fe0540a6960 tx=0x7fe054005d30 comp rx=0 tx=0).stop 2024-04-25T10:39:08.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.653+0000 7fe06e1db740 1 -- 172.21.15.43:0/3999232447 shutdown_connections 2024-04-25T10:39:08.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.653+0000 7fe06e1db740 1 --2- 172.21.15.43:0/3999232447 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561064291630 0x561064728a40 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.653+0000 7fe06e1db740 1 --2- 172.21.15.43:0/3999232447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610642fd410 0x5610646aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.653+0000 7fe06e1db740 1 --2- 172.21.15.43:0/3999232447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610642dc730 0x5610646af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.653+0000 7fe03ffff640 0 client.17418 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:08.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.653+0000 7fe06e1db740 1 --2- 172.21.15.43:0/3999232447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610642d4550 0x5610646b4620 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.653+0000 7fe06e1db740 1 -- 172.21.15.43:0/3999232447 >> 172.21.15.43:0/3999232447 conn(0x5610642f1560 msgr2=0x56106469dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:08.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.654+0000 7fe06e1db740 1 -- 172.21.15.43:0/3999232447 shutdown_connections 2024-04-25T10:39:08.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.654+0000 7fe06e1db740 1 -- 172.21.15.43:0/3999232447 wait complete. 2024-04-25T10:39:08.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.654+0000 7fe06e1db740 20 client.17418 trim_cache size 0 max 16384 2024-04-25T10:39:08.661 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:39:08.661 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:39:08.661 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "01afb0fb-d3a6-46d6-8fdf-bc8f2a366cc9", 2024-04-25T10:39:08.661 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:39:08.661 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:39:08.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.654+0000 7fe06724c640 1 -- 172.21.15.43:0/134359143 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe0500afce0 msgr2=0x7fe0500b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:08.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.654+0000 7fe06724c640 1 --2- 172.21.15.43:0/134359143 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe0500afce0 0x7fe0500b21f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe05801b000 tx=0x7fe058017f10 comp rx=0 tx=0).stop 2024-04-25T10:39:08.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.654+0000 7fe06724c640 1 -- 172.21.15.43:0/134359143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe060104260 msgr2=0x7fe0601a04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:08.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.654+0000 7fe06724c640 1 --2- 172.21.15.43:0/134359143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe060104260 0x7fe0601a04e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fe05400d770 tx=0x7fe05400dc30 comp rx=0 tx=0).stop 2024-04-25T10:39:08.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.654+0000 7fe06724c640 1 -- 172.21.15.43:0/134359143 shutdown_connections 2024-04-25T10:39:08.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.654+0000 7fe06724c640 1 --2- 172.21.15.43:0/134359143 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe0500afce0 0x7fe0500b21f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.654+0000 7fe06724c640 1 --2- 172.21.15.43:0/134359143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe060109fb0 0x7fe06019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.654+0000 7fe06724c640 1 --2- 172.21.15.43:0/134359143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe060109b10 0x7fe0601a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.655+0000 7fe06724c640 1 --2- 172.21.15.43:0/134359143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe060104260 0x7fe0601a04e0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.655+0000 7fe06724c640 1 -- 172.21.15.43:0/134359143 >> 172.21.15.43:0/134359143 conn(0x7fe0600ff210 msgr2=0x7fe06010f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:08.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.655+0000 7fe06724c640 1 -- 172.21.15.43:0/134359143 shutdown_connections 2024-04-25T10:39:08.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.655+0000 7fe06724c640 1 -- 172.21.15.43:0/134359143 wait complete. 2024-04-25T10:39:08.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:39:08.668 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:39:08.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.780+0000 7fe1860af640 1 -- 172.21.15.43:0/376507235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18010b6b0 msgr2=0x7fe18010dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:08.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.780+0000 7fe1860af640 1 --2- 172.21.15.43:0/376507235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18010b6b0 0x7fe18010dad0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fe174009900 tx=0x7fe17402a510 comp rx=0 tx=0).stop 2024-04-25T10:39:08.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.780+0000 7fe1860af640 1 -- 172.21.15.43:0/376507235 shutdown_connections 2024-04-25T10:39:08.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.780+0000 7fe1860af640 1 --2- 172.21.15.43:0/376507235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe18010e060 0x7fe180110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.780+0000 7fe1860af640 1 --2- 172.21.15.43:0/376507235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18010b6b0 0x7fe18010dad0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.780+0000 7fe1860af640 1 --2- 172.21.15.43:0/376507235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe180104c50 0x7fe180105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.780+0000 7fe1860af640 1 -- 172.21.15.43:0/376507235 >> 172.21.15.43:0/376507235 conn(0x7fe180078100 msgr2=0x7fe180078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:08.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.780+0000 7fe1860af640 1 -- 172.21.15.43:0/376507235 shutdown_connections 2024-04-25T10:39:08.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.780+0000 7fe1860af640 1 -- 172.21.15.43:0/376507235 wait complete. 2024-04-25T10:39:08.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.781+0000 7fe1860af640 1 Processor -- start 2024-04-25T10:39:08.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.781+0000 7fe1860af640 1 -- start start 2024-04-25T10:39:08.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.781+0000 7fe1860af640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe180104c50 0x7fe18019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.781+0000 7fe1860af640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe18010b6b0 0x7fe18019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.781+0000 7fe1860af640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe18010e060 0x7fe18019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.781+0000 7fe1860af640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe180113cc0 con 0x7fe180104c50 2024-04-25T10:39:08.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.781+0000 7fe1860af640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe180113b40 con 0x7fe18010e060 2024-04-25T10:39:08.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.781+0000 7fe1860af640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe180113e40 con 0x7fe18010b6b0 2024-04-25T10:39:08.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.781+0000 7fe17f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe180104c50 0x7fe18019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.781+0000 7fe17f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe180104c50 0x7fe18019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58832/0 (socket says 172.21.15.43:58832) 2024-04-25T10:39:08.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.781+0000 7fe17f7fe640 1 -- 172.21.15.43:0/117860066 learned_addr learned my addr 172.21.15.43:0/117860066 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:08.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.781+0000 7fe17effd640 1 --2- 172.21.15.43:0/117860066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe18010b6b0 0x7fe18019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.782+0000 7fe17ffff640 1 --2- 172.21.15.43:0/117860066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe18010e060 0x7fe18019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.782+0000 7fe17f7fe640 1 -- 172.21.15.43:0/117860066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe18010b6b0 msgr2=0x7fe18019aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:08.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.782+0000 7fe17f7fe640 1 --2- 172.21.15.43:0/117860066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe18010b6b0 0x7fe18019aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.782+0000 7fe17f7fe640 1 -- 172.21.15.43:0/117860066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe18010e060 msgr2=0x7fe18019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:08.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.782+0000 7fe17f7fe640 1 --2- 172.21.15.43:0/117860066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe18010e060 0x7fe18019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.782+0000 7fe17f7fe640 1 -- 172.21.15.43:0/117860066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18019f5a0 con 0x7fe180104c50 2024-04-25T10:39:08.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.782+0000 7fe17ffff640 1 --2- 172.21.15.43:0/117860066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe18010e060 0x7fe18019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:08.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.782+0000 7fe17f7fe640 1 --2- 172.21.15.43:0/117860066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe180104c50 0x7fe18019a740 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fe17000e9f0 tx=0x7fe17000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:08.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.782+0000 7fe17cff9640 1 -- 172.21.15.43:0/117860066 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17000cd80 con 0x7fe180104c50 2024-04-25T10:39:08.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.782+0000 7fe17cff9640 1 -- 172.21.15.43:0/117860066 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1700054d0 con 0x7fe180104c50 2024-04-25T10:39:08.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.782+0000 7fe1860af640 1 -- 172.21.15.43:0/117860066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18019f890 con 0x7fe180104c50 2024-04-25T10:39:08.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.782+0000 7fe17cff9640 1 -- 172.21.15.43:0/117860066 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170010690 con 0x7fe180104c50 2024-04-25T10:39:08.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.782+0000 7fe1860af640 1 -- 172.21.15.43:0/117860066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe18019fd60 con 0x7fe180104c50 2024-04-25T10:39:08.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe18cf66740 1 Processor -- start 2024-04-25T10:39:08.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe17cff9640 1 -- 172.21.15.43:0/117860066 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fe170004e90 con 0x7fe180104c50 2024-04-25T10:39:08.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe18cf66740 1 -- start start 2024-04-25T10:39:08.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe18cf66740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cc8c57390 0x558cc8ff7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe18cf66740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c2e4d0 0x558cc8ff7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe18cf66740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c366b0 0x558cc90048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe18cf66740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558cc8f5dd80 con 0x558cc8c2e4d0 2024-04-25T10:39:08.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe18cf66740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558cc8f178b0 con 0x558cc8c366b0 2024-04-25T10:39:08.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe18cf66740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558cc8f59d40 con 0x558cc8c57390 2024-04-25T10:39:08.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe17f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c2e4d0 0x558cc8ff7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe17f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c2e4d0 0x558cc8ff7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58844/0 (socket says 172.21.15.43:58844) 2024-04-25T10:39:08.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe17f7fe640 1 -- 172.21.15.43:0/2216359236 learned_addr learned my addr 172.21.15.43:0/2216359236 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:08.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe17ffff640 1 --2- 172.21.15.43:0/2216359236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cc8c57390 0x558cc8ff7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe17effd640 1 --2- 172.21.15.43:0/2216359236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c366b0 0x558cc90048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe17f7fe640 1 -- 172.21.15.43:0/2216359236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cc8c57390 msgr2=0x558cc8ff7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:08.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe17f7fe640 1 --2- 172.21.15.43:0/2216359236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cc8c57390 0x558cc8ff7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe17f7fe640 1 -- 172.21.15.43:0/2216359236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c366b0 msgr2=0x558cc90048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:08.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe17f7fe640 1 --2- 172.21.15.43:0/2216359236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c366b0 0x558cc90048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe17f7fe640 1 -- 172.21.15.43:0/2216359236 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cc8ee8a40 con 0x558cc8c2e4d0 2024-04-25T10:39:08.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe17cff9640 1 --2- 172.21.15.43:0/117860066 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe1680afc60 0x7fe1680b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.785+0000 7fe17effd640 1 --2- 172.21.15.43:0/2216359236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c366b0 0x558cc90048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:39:08.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.786+0000 7fe17ffff640 1 --2- 172.21.15.43:0/117860066 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe1680afc60 0x7fe1680b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.786+0000 7fe17f7fe640 1 --2- 172.21.15.43:0/2216359236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c2e4d0 0x558cc8ff7fa0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe1700dcbc0 tx=0x7fe1700e5640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d655df27e84550 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:08.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.786+0000 7fe17cff9640 1 -- 172.21.15.43:0/117860066 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7fe1700df520 con 0x7fe180104c50 2024-04-25T10:39:08.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.786+0000 7fe165ffb640 1 -- 172.21.15.43:0/2216359236 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1700a7470 con 0x558cc8c2e4d0 2024-04-25T10:39:08.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.786+0000 7fe165ffb640 1 -- 172.21.15.43:0/2216359236 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1700a71b0 con 0x558cc8c2e4d0 2024-04-25T10:39:08.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.786+0000 7fe165ffb640 1 -- 172.21.15.43:0/2216359236 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170005180 con 0x558cc8c2e4d0 2024-04-25T10:39:08.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.787+0000 7fe18cf66740 1 -- 172.21.15.43:0/2216359236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c2e4d0 msgr2=0x558cc8ff7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:08.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.787+0000 7fe17ffff640 1 --2- 172.21.15.43:0/117860066 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe1680afc60 0x7fe1680b2170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x558cc8ff4bd0 tx=0x7fe1780095a0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:08.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.787+0000 7fe18cf66740 1 --2- 172.21.15.43:0/2216359236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c2e4d0 0x558cc8ff7fa0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe1700dcbc0 tx=0x7fe1700e5640 comp rx=0 tx=0).stop 2024-04-25T10:39:08.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.787+0000 7fe18cf66740 1 -- 172.21.15.43:0/2216359236 shutdown_connections 2024-04-25T10:39:08.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.787+0000 7fe18cf66740 1 --2- 172.21.15.43:0/2216359236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c366b0 0x558cc90048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.787+0000 7fe18cf66740 1 --2- 172.21.15.43:0/2216359236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c2e4d0 0x558cc8ff7fa0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.787+0000 7fe18cf66740 1 --2- 172.21.15.43:0/2216359236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cc8c57390 0x558cc8ff7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.787+0000 7fe18cf66740 1 -- 172.21.15.43:0/2216359236 >> 172.21.15.43:0/2216359236 conn(0x558cc8c4b4e0 msgr2=0x558cc8ff4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:08.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.787+0000 7fe18cf66740 1 -- 172.21.15.43:0/2216359236 shutdown_connections 2024-04-25T10:39:08.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.787+0000 7fe18cf66740 1 -- 172.21.15.43:0/2216359236 wait complete. 2024-04-25T10:39:08.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.787+0000 7fe18cf66740 1 -- start start 2024-04-25T10:39:08.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.787+0000 7fe18cf66740 1 Processor -- start 2024-04-25T10:39:08.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.788+0000 7fe18cf66740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c2e4d0 0x558cc9002560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.788+0000 7fe18cf66740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cc8c366b0 0x558cc9002aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.788+0000 7fe18cf66740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c57390 0x558cc900e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.788+0000 7fe18cf66740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558cc9004fe0 con 0x558cc8c2e4d0 2024-04-25T10:39:08.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.788+0000 7fe18cf66740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558cc8ee4d10 con 0x558cc8c57390 2024-04-25T10:39:08.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.788+0000 7fe18cf66740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558cc9005160 con 0x558cc8c366b0 2024-04-25T10:39:08.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.788+0000 7fe18cf66740 10 client.0 _ll_register_callbacks cb 0x558cc8f11210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:08.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.788+0000 7fe17f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c2e4d0 0x558cc9002560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.788+0000 7fe17f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c2e4d0 0x558cc9002560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58856/0 (socket says 172.21.15.43:58856) 2024-04-25T10:39:08.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.788+0000 7fe17f7fe640 1 -- 172.21.15.43:0/657257017 learned_addr learned my addr 172.21.15.43:0/657257017 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:08.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.788+0000 7fe17effd640 1 --2- 172.21.15.43:0/657257017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cc8c366b0 0x558cc9002aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.788+0000 7fe17ffff640 1 --2- 172.21.15.43:0/657257017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c57390 0x558cc900e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.788+0000 7fe17f7fe640 1 -- 172.21.15.43:0/657257017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cc8c366b0 msgr2=0x558cc9002aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:08.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.788+0000 7fe17f7fe640 1 --2- 172.21.15.43:0/657257017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cc8c366b0 0x558cc9002aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.788+0000 7fe17f7fe640 1 -- 172.21.15.43:0/657257017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c57390 msgr2=0x558cc900e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:08.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.788+0000 7fe17f7fe640 1 --2- 172.21.15.43:0/657257017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c57390 0x558cc900e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.788+0000 7fe17f7fe640 1 -- 172.21.15.43:0/657257017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cc8d397d0 con 0x558cc8c2e4d0 2024-04-25T10:39:08.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.788+0000 7fe17ffff640 1 --2- 172.21.15.43:0/657257017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c57390 0x558cc900e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:39:08.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.788+0000 7fe17f7fe640 1 --2- 172.21.15.43:0/657257017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c2e4d0 0x558cc9002560 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe1700a3720 tx=0x7fe170004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436b153f48d4acf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:08.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.788+0000 7fe15effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:08.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.789+0000 7fe15effd640 1 -- 172.21.15.43:0/657257017 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1700a6e10 con 0x558cc8c2e4d0 2024-04-25T10:39:08.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.789+0000 7fe15effd640 1 -- 172.21.15.43:0/657257017 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7fe1700a7b10 con 0x558cc8c2e4d0 2024-04-25T10:39:08.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.789+0000 7fe18cf66740 1 -- 172.21.15.43:0/657257017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1700a71b0 con 0x558cc8c2e4d0 2024-04-25T10:39:08.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.789+0000 7fe15effd640 1 -- 172.21.15.43:0/657257017 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe17006a050 con 0x558cc8c2e4d0 2024-04-25T10:39:08.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.789+0000 7fe18cf66740 10 client.17430 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:08.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.789+0000 7fe18cf66740 1 -- 172.21.15.43:0/657257017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558cc8c6f3f0 con 0x558cc8c2e4d0 2024-04-25T10:39:08.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.789+0000 7fe15effd640 1 -- 172.21.15.43:0/657257017 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1700a71b0 con 0x558cc8c2e4d0 2024-04-25T10:39:08.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.789+0000 7fe15effd640 1 -- 172.21.15.43:0/657257017 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe1700b0070 con 0x558cc8c2e4d0 2024-04-25T10:39:08.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.789+0000 7fe18cf66740 10 client.17430 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:08.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.789+0000 7fe18cf66740 10 client.17430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:08.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.789+0000 7fe18cf66740 20 client.17430 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:08.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.789+0000 7fe18cf66740 1 --2- 172.21.15.43:0/657257017 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558cc8c15f60 0x558cc90143b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.789+0000 7fe18cf66740 4 client.17430 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:39:08.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.789+0000 7fe18cf66740 1 -- 172.21.15.43:0/657257017 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cc8f6b990 con 0x558cc8c15f60 2024-04-25T10:39:08.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.790+0000 7fe17effd640 1 --2- 172.21.15.43:0/657257017 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558cc8c15f60 0x558cc90143b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.790+0000 7fe17effd640 1 --2- 172.21.15.43:0/657257017 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558cc8c15f60 0x558cc90143b0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=522fd16fba003547 server_cookie=43a4822abc2aabc4 in_seq=0 out_seq=0 2024-04-25T10:39:08.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.790+0000 7fe15effd640 10 client.17430 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:08.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.791+0000 7fe15effd640 1 -- 172.21.15.43:0/657257017 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558cc8f6b990 con 0x558cc8c15f60 2024-04-25T10:39:08.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.791+0000 7fe15effd640 10 client.17430 handle_command_reply: tid=0 2024-04-25T10:39:08.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.791+0000 7fe18cf66740 1 client.17430 shutdown 2024-04-25T10:39:08.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.791+0000 7fe18cf66740 1 -- 172.21.15.43:0/657257017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c2e4d0 msgr2=0x558cc9002560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:08.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.791+0000 7fe18cf66740 1 --2- 172.21.15.43:0/657257017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c2e4d0 0x558cc9002560 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe1700a3720 tx=0x7fe170004a00 comp rx=0 tx=0).stop 2024-04-25T10:39:08.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.791+0000 7fe18cf66740 1 -- 172.21.15.43:0/657257017 shutdown_connections 2024-04-25T10:39:08.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.791+0000 7fe18cf66740 1 --2- 172.21.15.43:0/657257017 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558cc8c15f60 0x558cc90143b0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.791+0000 7fe18cf66740 1 --2- 172.21.15.43:0/657257017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c57390 0x558cc900e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.791+0000 7fe18cf66740 1 --2- 172.21.15.43:0/657257017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cc8c366b0 0x558cc9002aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.791+0000 7fe18cf66740 1 --2- 172.21.15.43:0/657257017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c2e4d0 0x558cc9002560 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.791+0000 7fe15effd640 0 client.17430 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:08.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.791+0000 7fe18cf66740 1 -- 172.21.15.43:0/657257017 >> 172.21.15.43:0/657257017 conn(0x558cc8c4b4e0 msgr2=0x558cc8ff9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:08.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.791+0000 7fe18cf66740 1 -- 172.21.15.43:0/657257017 shutdown_connections 2024-04-25T10:39:08.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.791+0000 7fe18cf66740 1 -- 172.21.15.43:0/657257017 wait complete. 2024-04-25T10:39:08.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.791+0000 7fe18cf66740 20 client.17430 trim_cache size 0 max 16384 2024-04-25T10:39:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.809+0000 7fe18cf66740 1 Processor -- start 2024-04-25T10:39:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.809+0000 7fe18cf66740 1 -- start start 2024-04-25T10:39:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.809+0000 7fe18cf66740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cc8c2e4d0 0x558cc9001290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.809+0000 7fe18cf66740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c366b0 0x558cc8ff9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.809+0000 7fe18cf66740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c57390 0x558cc900e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.809+0000 7fe18cf66740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe1801b2380 con 0x558cc8c366b0 2024-04-25T10:39:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.809+0000 7fe18cf66740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe1801afc50 con 0x558cc8c57390 2024-04-25T10:39:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.809+0000 7fe18cf66740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe1801adac0 con 0x558cc8c2e4d0 2024-04-25T10:39:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.810+0000 7fe17effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c366b0 0x558cc8ff9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.810+0000 7fe17effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c366b0 0x558cc8ff9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58860/0 (socket says 172.21.15.43:58860) 2024-04-25T10:39:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.810+0000 7fe17effd640 1 -- 172.21.15.43:0/3257210686 learned_addr learned my addr 172.21.15.43:0/3257210686 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.810+0000 7fe17f7fe640 1 --2- 172.21.15.43:0/3257210686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cc8c2e4d0 0x558cc9001290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.810+0000 7fe17ffff640 1 --2- 172.21.15.43:0/3257210686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c57390 0x558cc900e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.810+0000 7fe17effd640 1 -- 172.21.15.43:0/3257210686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cc8c2e4d0 msgr2=0x558cc9001290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:08.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.810+0000 7fe17effd640 1 --2- 172.21.15.43:0/3257210686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cc8c2e4d0 0x558cc9001290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.810+0000 7fe17effd640 1 -- 172.21.15.43:0/3257210686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c57390 msgr2=0x558cc900e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:08.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.810+0000 7fe17effd640 1 --2- 172.21.15.43:0/3257210686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c57390 0x558cc900e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.810+0000 7fe17effd640 1 -- 172.21.15.43:0/3257210686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cc8c2f0f0 con 0x558cc8c366b0 2024-04-25T10:39:08.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.810+0000 7fe17effd640 1 --2- 172.21.15.43:0/3257210686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c366b0 0x558cc8ff9e30 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fe174031440 tx=0x7fe174005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989401c757b40238 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:08.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.810+0000 7fe1657fa640 1 -- 172.21.15.43:0/3257210686 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe174035040 con 0x558cc8c366b0 2024-04-25T10:39:08.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.810+0000 7fe1657fa640 1 -- 172.21.15.43:0/3257210686 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe174049070 con 0x558cc8c366b0 2024-04-25T10:39:08.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.810+0000 7fe1657fa640 1 -- 172.21.15.43:0/3257210686 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1740049b0 con 0x558cc8c366b0 2024-04-25T10:39:08.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.811+0000 7fe18cf66740 1 -- 172.21.15.43:0/3257210686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c366b0 msgr2=0x558cc8ff9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:08.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.811+0000 7fe18cf66740 1 --2- 172.21.15.43:0/3257210686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c366b0 0x558cc8ff9e30 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fe174031440 tx=0x7fe174005890 comp rx=0 tx=0).stop 2024-04-25T10:39:08.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.811+0000 7fe18cf66740 1 -- 172.21.15.43:0/3257210686 shutdown_connections 2024-04-25T10:39:08.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.811+0000 7fe18cf66740 1 --2- 172.21.15.43:0/3257210686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c57390 0x558cc900e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.811+0000 7fe18cf66740 1 --2- 172.21.15.43:0/3257210686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c366b0 0x558cc8ff9e30 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.811+0000 7fe18cf66740 1 --2- 172.21.15.43:0/3257210686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cc8c2e4d0 0x558cc9001290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.811+0000 7fe18cf66740 1 -- 172.21.15.43:0/3257210686 >> 172.21.15.43:0/3257210686 conn(0x558cc8c4b4e0 msgr2=0x558cc8ff9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:08.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.811+0000 7fe18cf66740 1 -- 172.21.15.43:0/3257210686 shutdown_connections 2024-04-25T10:39:08.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.811+0000 7fe18cf66740 1 -- 172.21.15.43:0/3257210686 wait complete. 2024-04-25T10:39:08.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.811+0000 7fe18cf66740 1 -- start start 2024-04-25T10:39:08.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.812+0000 7fe18cf66740 1 Processor -- start 2024-04-25T10:39:08.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.812+0000 7fe18cf66740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cc8c2e4d0 0x558cc90142a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.812+0000 7fe18cf66740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c366b0 0x558cc90147e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.812+0000 7fe18cf66740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c57390 0x558cc9014d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.812+0000 7fe18cf66740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe1801ae110 con 0x558cc8c366b0 2024-04-25T10:39:08.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.812+0000 7fe18cf66740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe1801af580 con 0x558cc8c57390 2024-04-25T10:39:08.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.812+0000 7fe18cf66740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe1801ae7e0 con 0x558cc8c2e4d0 2024-04-25T10:39:08.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.812+0000 7fe18cf66740 10 client.0 _ll_register_callbacks cb 0x7fe17000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:08.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.812+0000 7fe17effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c366b0 0x558cc90147e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.812+0000 7fe17effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c366b0 0x558cc90147e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58866/0 (socket says 172.21.15.43:58866) 2024-04-25T10:39:08.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.812+0000 7fe17effd640 1 -- 172.21.15.43:0/2687178201 learned_addr learned my addr 172.21.15.43:0/2687178201 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.812+0000 7fe17f7fe640 1 --2- 172.21.15.43:0/2687178201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cc8c2e4d0 0x558cc90142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.812+0000 7fe17ffff640 1 --2- 172.21.15.43:0/2687178201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c57390 0x558cc9014d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.813+0000 7fe17effd640 1 -- 172.21.15.43:0/2687178201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cc8c2e4d0 msgr2=0x558cc90142a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.813+0000 7fe17effd640 1 --2- 172.21.15.43:0/2687178201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cc8c2e4d0 0x558cc90142a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.813+0000 7fe17effd640 1 -- 172.21.15.43:0/2687178201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c57390 msgr2=0x558cc9014d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.813+0000 7fe17effd640 1 --2- 172.21.15.43:0/2687178201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c57390 0x558cc9014d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.813+0000 7fe17effd640 1 -- 172.21.15.43:0/2687178201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cc8f1e4b0 con 0x558cc8c366b0 2024-04-25T10:39:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.813+0000 7fe17ffff640 1 --2- 172.21.15.43:0/2687178201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c57390 0x558cc9014d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.813+0000 7fe17effd640 1 --2- 172.21.15.43:0/2687178201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c366b0 0x558cc90147e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe174009230 tx=0x7fe174032e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d52d0b25abe1804 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.813+0000 7fe15effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.813+0000 7fe15effd640 1 -- 172.21.15.43:0/2687178201 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe174051070 con 0x558cc8c366b0 2024-04-25T10:39:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.813+0000 7fe15effd640 1 -- 172.21.15.43:0/2687178201 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7fe174004310 con 0x558cc8c366b0 2024-04-25T10:39:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.813+0000 7fe18cf66740 1 -- 172.21.15.43:0/2687178201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe174049070 con 0x558cc8c366b0 2024-04-25T10:39:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.813+0000 7fe15effd640 1 -- 172.21.15.43:0/2687178201 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe174035040 con 0x558cc8c366b0 2024-04-25T10:39:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.813+0000 7fe18cf66740 10 client.17436 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.813+0000 7fe18cf66740 1 -- 172.21.15.43:0/2687178201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558cc907b7e0 con 0x558cc8c366b0 2024-04-25T10:39:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.813+0000 7fe15effd640 1 -- 172.21.15.43:0/2687178201 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe174049070 con 0x558cc8c366b0 2024-04-25T10:39:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.813+0000 7fe15effd640 1 -- 172.21.15.43:0/2687178201 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe174044ce0 con 0x558cc8c366b0 2024-04-25T10:39:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.813+0000 7fe18cf66740 10 client.17436 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.813+0000 7fe18cf66740 10 client.17436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.814+0000 7fe18cf66740 20 client.17436 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.814+0000 7fe18cf66740 1 --2- 172.21.15.43:0/2687178201 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558cc8beb3a0 0x558cc907def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.814+0000 7fe18cf66740 4 client.17436 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:39:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.814+0000 7fe18cf66740 1 -- 172.21.15.43:0/2687178201 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558cc8f65800 con 0x558cc8beb3a0 2024-04-25T10:39:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.814+0000 7fe17f7fe640 1 --2- 172.21.15.43:0/2687178201 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558cc8beb3a0 0x558cc907def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.814+0000 7fe15effd640 10 client.17436 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.814+0000 7fe17f7fe640 1 --2- 172.21.15.43:0/2687178201 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558cc8beb3a0 0x558cc907def0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe6c877214b295b6 server_cookie=4ed42fe15dd79324 in_seq=0 out_seq=0 2024-04-25T10:39:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.814+0000 7fe15effd640 1 -- 172.21.15.43:0/2687178201 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558cc8f65800 con 0x558cc8beb3a0 2024-04-25T10:39:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.814+0000 7fe15effd640 10 client.17436 handle_command_reply: tid=0 2024-04-25T10:39:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.814+0000 7fe18cf66740 1 client.17436 shutdown 2024-04-25T10:39:08.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe18cf66740 1 -- 172.21.15.43:0/2687178201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c366b0 msgr2=0x558cc90147e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:08.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe18cf66740 1 --2- 172.21.15.43:0/2687178201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c366b0 0x558cc90147e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe174009230 tx=0x7fe174032e00 comp rx=0 tx=0).stop 2024-04-25T10:39:08.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe18cf66740 1 -- 172.21.15.43:0/2687178201 shutdown_connections 2024-04-25T10:39:08.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe18cf66740 1 --2- 172.21.15.43:0/2687178201 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558cc8beb3a0 0x558cc907def0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe18cf66740 1 --2- 172.21.15.43:0/2687178201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cc8c57390 0x558cc9014d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe18cf66740 1 --2- 172.21.15.43:0/2687178201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cc8c366b0 0x558cc90147e0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe15effd640 0 client.17436 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:08.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe18cf66740 1 --2- 172.21.15.43:0/2687178201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cc8c2e4d0 0x558cc90142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe18cf66740 1 -- 172.21.15.43:0/2687178201 >> 172.21.15.43:0/2687178201 conn(0x558cc8c4b4e0 msgr2=0x558cc9004430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:08.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe18cf66740 1 -- 172.21.15.43:0/2687178201 shutdown_connections 2024-04-25T10:39:08.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe18cf66740 1 -- 172.21.15.43:0/2687178201 wait complete. 2024-04-25T10:39:08.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe18cf66740 20 client.17436 trim_cache size 0 max 16384 2024-04-25T10:39:08.825 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:39:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:39:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:39:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: "01afb0fb-d3a6-46d6-8fdf-bc8f2a366cc9": { 2024-04-25T10:39:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:39:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "01afb0fb-d3a6-46d6-8fdf-bc8f2a366cc9", 2024-04-25T10:39:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:39:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:39:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:39:08.825 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:39:08.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe1860af640 1 -- 172.21.15.43:0/117860066 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe1680afc60 msgr2=0x7fe1680b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:08.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe1860af640 1 --2- 172.21.15.43:0/117860066 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe1680afc60 0x7fe1680b2170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x558cc8ff4bd0 tx=0x7fe1780095a0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe1860af640 1 -- 172.21.15.43:0/117860066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe180104c50 msgr2=0x7fe18019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:08.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe1860af640 1 --2- 172.21.15.43:0/117860066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe180104c50 0x7fe18019a740 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fe17000e9f0 tx=0x7fe17000edf0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe1860af640 1 -- 172.21.15.43:0/117860066 shutdown_connections 2024-04-25T10:39:08.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe1860af640 1 --2- 172.21.15.43:0/117860066 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe1680afc60 0x7fe1680b2170 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe1860af640 1 --2- 172.21.15.43:0/117860066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe18010e060 0x7fe18019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe1860af640 1 --2- 172.21.15.43:0/117860066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe18010b6b0 0x7fe18019aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe1860af640 1 --2- 172.21.15.43:0/117860066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe180104c50 0x7fe18019a740 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:08.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.815+0000 7fe1860af640 1 -- 172.21.15.43:0/117860066 >> 172.21.15.43:0/117860066 conn(0x7fe180078100 msgr2=0x7fe180101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:08.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.816+0000 7fe1860af640 1 -- 172.21.15.43:0/117860066 shutdown_connections 2024-04-25T10:39:08.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:08.816+0000 7fe1860af640 1 -- 172.21.15.43:0/117860066 wait complete. 2024-04-25T10:39:08.829 INFO:tasks.cephfs.filesystem:scrub status for tag:01afb0fb-d3a6-46d6-8fdf-bc8f2a366cc9 - {'path': '/', 'tag': '01afb0fb-d3a6-46d6-8fdf-bc8f2a366cc9', 'options': 'recursive,force'} 2024-04-25T10:39:10.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:10 smithi043 ceph-mon[29478]: pgmap v203: 97 pgs: 97 active+clean; 2.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 101 KiB/s wr, 23 op/s 2024-04-25T10:39:10.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:10 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:39:10.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:10 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:39:10.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:10 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:39:10.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:10 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:39:10.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:10 smithi077 ceph-mon[32772]: pgmap v203: 97 pgs: 97 active+clean; 2.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 101 KiB/s wr, 23 op/s 2024-04-25T10:39:10.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:10 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:39:10.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:10 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:39:10.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:10 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:39:10.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:10 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:39:10.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:10 smithi164 ceph-mon[32713]: pgmap v203: 97 pgs: 97 active+clean; 2.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 101 KiB/s wr, 23 op/s 2024-04-25T10:39:10.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:10 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:39:10.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:10 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:39:10.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:10 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:39:10.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:10 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:39:12.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:12 smithi043 ceph-mon[29478]: pgmap v204: 97 pgs: 97 active+clean; 2.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 101 KiB/s wr, 23 op/s 2024-04-25T10:39:12.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:12 smithi077 ceph-mon[32772]: pgmap v204: 97 pgs: 97 active+clean; 2.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 101 KiB/s wr, 23 op/s 2024-04-25T10:39:12.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:12 smithi164 ceph-mon[32713]: pgmap v204: 97 pgs: 97 active+clean; 2.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 101 KiB/s wr, 23 op/s 2024-04-25T10:39:14.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:13 smithi077 ceph-mon[32772]: pgmap v205: 97 pgs: 97 active+clean; 2.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 101 KiB/s wr, 23 op/s 2024-04-25T10:39:14.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:13 smithi077 ceph-mon[32772]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T10:39:14.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:13 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:39:14.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:13 smithi164 ceph-mon[32713]: pgmap v205: 97 pgs: 97 active+clean; 2.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 101 KiB/s wr, 23 op/s 2024-04-25T10:39:14.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:13 smithi164 ceph-mon[32713]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T10:39:14.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:13 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:39:14.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:13 smithi043 ceph-mon[29478]: pgmap v205: 97 pgs: 97 active+clean; 2.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 101 KiB/s wr, 23 op/s 2024-04-25T10:39:14.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:13 smithi043 ceph-mon[29478]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T10:39:14.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:13 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:39:16.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:15 smithi077 ceph-mon[32772]: pgmap v207: 97 pgs: 97 active+clean; 2.4 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-04-25T10:39:16.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:15 smithi164 ceph-mon[32713]: pgmap v207: 97 pgs: 97 active+clean; 2.4 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-04-25T10:39:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:15 smithi043 ceph-mon[29478]: pgmap v207: 97 pgs: 97 active+clean; 2.4 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-04-25T10:39:18.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:17 smithi077 ceph-mon[32772]: pgmap v208: 97 pgs: 97 active+clean; 2.4 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-04-25T10:39:18.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:17 smithi164 ceph-mon[32713]: pgmap v208: 97 pgs: 97 active+clean; 2.4 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-04-25T10:39:18.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:17 smithi043 ceph-mon[29478]: pgmap v208: 97 pgs: 97 active+clean; 2.4 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-04-25T10:39:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.523+0000 7f721c97d640 1 -- 172.21.15.43:0/3363966642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7218108f00 msgr2=0x7f7218102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.523+0000 7f721c97d640 1 --2- 172.21.15.43:0/3363966642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7218108f00 0x7f7218102b50 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f720c009900 tx=0x7f720c02a410 comp rx=0 tx=0).stop 2024-04-25T10:39:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.523+0000 7f721c97d640 1 -- 172.21.15.43:0/3363966642 shutdown_connections 2024-04-25T10:39:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.523+0000 7f721c97d640 1 --2- 172.21.15.43:0/3363966642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72181031c0 0x7f721810f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.523+0000 7f721c97d640 1 --2- 172.21.15.43:0/3363966642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7218108f00 0x7f7218102b50 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.523+0000 7f721c97d640 1 --2- 172.21.15.43:0/3363966642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7218108a60 0x7f7218102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.523+0000 7f721c97d640 1 -- 172.21.15.43:0/3363966642 >> 172.21.15.43:0/3363966642 conn(0x7f72180781b0 msgr2=0x7f72180785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:18.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.523+0000 7f721c97d640 1 -- 172.21.15.43:0/3363966642 shutdown_connections 2024-04-25T10:39:18.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.523+0000 7f721c97d640 1 -- 172.21.15.43:0/3363966642 wait complete. 2024-04-25T10:39:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.524+0000 7f721c97d640 1 Processor -- start 2024-04-25T10:39:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.524+0000 7f721c97d640 1 -- start start 2024-04-25T10:39:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.524+0000 7f721c97d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72181031c0 0x7f72181a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.524+0000 7f721c97d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7218108a60 0x7f72181a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.524+0000 7f721c97d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7218108f00 0x7f721819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.524+0000 7f721c97d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7218112c60 con 0x7f7218108f00 2024-04-25T10:39:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.524+0000 7f721c97d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7218112ae0 con 0x7f7218108a60 2024-04-25T10:39:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.524+0000 7f721c97d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7218112de0 con 0x7f72181031c0 2024-04-25T10:39:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.524+0000 7f7216d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7218108f00 0x7f721819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.524+0000 7f7216d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7218108f00 0x7f721819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58892/0 (socket says 172.21.15.43:58892) 2024-04-25T10:39:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.524+0000 7f7216d9d640 1 -- 172.21.15.43:0/198678194 learned_addr learned my addr 172.21.15.43:0/198678194 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.524+0000 7f721659c640 1 --2- 172.21.15.43:0/198678194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72181031c0 0x7f72181a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.524+0000 7f7215d9b640 1 --2- 172.21.15.43:0/198678194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7218108a60 0x7f72181a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.525+0000 7f7216d9d640 1 -- 172.21.15.43:0/198678194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72181031c0 msgr2=0x7f72181a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.525+0000 7f7216d9d640 1 --2- 172.21.15.43:0/198678194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72181031c0 0x7f72181a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.525+0000 7f7216d9d640 1 -- 172.21.15.43:0/198678194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7218108a60 msgr2=0x7f72181a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.525+0000 7f7216d9d640 1 --2- 172.21.15.43:0/198678194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7218108a60 0x7f72181a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.525+0000 7f7216d9d640 1 -- 172.21.15.43:0/198678194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721819adc0 con 0x7f7218108f00 2024-04-25T10:39:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.525+0000 7f7215d9b640 1 --2- 172.21.15.43:0/198678194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7218108a60 0x7f72181a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.525+0000 7f7216d9d640 1 --2- 172.21.15.43:0/198678194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7218108f00 0x7f721819a690 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f7210009fd0 tx=0x7f721000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.525+0000 7f72077fe640 1 -- 172.21.15.43:0/198678194 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210017070 con 0x7f7218108f00 2024-04-25T10:39:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.525+0000 7f72077fe640 1 -- 172.21.15.43:0/198678194 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7210003e20 con 0x7f7218108f00 2024-04-25T10:39:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.525+0000 7f721c97d640 1 -- 172.21.15.43:0/198678194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721819b050 con 0x7f7218108f00 2024-04-25T10:39:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.525+0000 7f72077fe640 1 -- 172.21.15.43:0/198678194 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210010460 con 0x7f7218108f00 2024-04-25T10:39:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.525+0000 7f721c97d640 1 -- 172.21.15.43:0/198678194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72181a72b0 con 0x7f7218108f00 2024-04-25T10:39:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.528+0000 7f722382d740 1 Processor -- start 2024-04-25T10:39:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.529+0000 7f72077fe640 1 -- 172.21.15.43:0/198678194 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f7210003970 con 0x7f7218108f00 2024-04-25T10:39:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.529+0000 7f722382d740 1 -- start start 2024-04-25T10:39:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.529+0000 7f722382d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3801b0 0x557dbb71f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.529+0000 7f722382d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557dbb3572f0 0x557dbb71fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.529+0000 7f722382d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb35f4d0 0x557dbb7203e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.529+0000 7f722382d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557dbb686460 con 0x557dbb3801b0 2024-04-25T10:39:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.529+0000 7f722382d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557dbb640590 con 0x557dbb35f4d0 2024-04-25T10:39:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.529+0000 7f722382d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557dbb682420 con 0x557dbb3572f0 2024-04-25T10:39:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.529+0000 7f7215d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3801b0 0x557dbb71f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.529+0000 7f7215d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3801b0 0x557dbb71f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58894/0 (socket says 172.21.15.43:58894) 2024-04-25T10:39:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.529+0000 7f7215d9b640 1 -- 172.21.15.43:0/3591750786 learned_addr learned my addr 172.21.15.43:0/3591750786 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.530+0000 7f721659c640 1 --2- 172.21.15.43:0/3591750786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557dbb3572f0 0x557dbb71fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.530+0000 7f72077fe640 1 --2- 172.21.15.43:0/198678194 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f72000afc50 0x7f72000b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.530+0000 7f7215d9b640 1 --2- 172.21.15.43:0/3591750786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb35f4d0 0x557dbb7203e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.530+0000 7f7216d9d640 1 --2- 172.21.15.43:0/198678194 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f72000afc50 0x7f72000b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.530+0000 7f7215d9b640 1 -- 172.21.15.43:0/3591750786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557dbb3572f0 msgr2=0x557dbb71fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.530+0000 7f7215d9b640 1 --2- 172.21.15.43:0/3591750786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557dbb3572f0 0x557dbb71fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.530+0000 7f7215d9b640 1 -- 172.21.15.43:0/3591750786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb35f4d0 msgr2=0x557dbb7203e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.530+0000 7f7215d9b640 1 --2- 172.21.15.43:0/3591750786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb35f4d0 0x557dbb7203e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.530+0000 7f7215d9b640 1 -- 172.21.15.43:0/3591750786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dbb64d0d0 con 0x557dbb3801b0 2024-04-25T10:39:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.530+0000 7f72077fe640 1 -- 172.21.15.43:0/198678194 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f72100dee10 con 0x7f7218108f00 2024-04-25T10:39:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.530+0000 7f7215d9b640 1 --2- 172.21.15.43:0/3591750786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3801b0 0x557dbb71f960 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f720c005ab0 tx=0x7f720c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de992948b869beec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.530+0000 7f7216d9d640 1 --2- 172.21.15.43:0/198678194 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f72000afc50 0x7f72000b2160 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f72100dc610 tx=0x7f7210004020 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.530+0000 7f7204ff9640 1 -- 172.21.15.43:0/3591750786 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c035050 con 0x557dbb3801b0 2024-04-25T10:39:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.530+0000 7f7204ff9640 1 -- 172.21.15.43:0/3591750786 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f720c0316f0 con 0x557dbb3801b0 2024-04-25T10:39:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.530+0000 7f7204ff9640 1 -- 172.21.15.43:0/3591750786 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c0046c0 con 0x557dbb3801b0 2024-04-25T10:39:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.531+0000 7f722382d740 1 -- 172.21.15.43:0/3591750786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3801b0 msgr2=0x557dbb71f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.531+0000 7f722382d740 1 --2- 172.21.15.43:0/3591750786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3801b0 0x557dbb71f960 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f720c005ab0 tx=0x7f720c009560 comp rx=0 tx=0).stop 2024-04-25T10:39:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.531+0000 7f722382d740 1 -- 172.21.15.43:0/3591750786 shutdown_connections 2024-04-25T10:39:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.531+0000 7f722382d740 1 --2- 172.21.15.43:0/3591750786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb35f4d0 0x557dbb7203e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.531+0000 7f722382d740 1 --2- 172.21.15.43:0/3591750786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557dbb3572f0 0x557dbb71fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.531+0000 7f722382d740 1 --2- 172.21.15.43:0/3591750786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3801b0 0x557dbb71f960 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.531+0000 7f722382d740 1 -- 172.21.15.43:0/3591750786 >> 172.21.15.43:0/3591750786 conn(0x557dbb374300 msgr2=0x557dbb71c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.531+0000 7f722382d740 1 -- 172.21.15.43:0/3591750786 shutdown_connections 2024-04-25T10:39:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.532+0000 7f722382d740 1 -- 172.21.15.43:0/3591750786 wait complete. 2024-04-25T10:39:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.532+0000 7f722382d740 1 -- start start 2024-04-25T10:39:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.532+0000 7f722382d740 1 Processor -- start 2024-04-25T10:39:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.533+0000 7f722382d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3572f0 0x557dbb72e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.533+0000 7f722382d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb35f4d0 0x557dbb72ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.533+0000 7f722382d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557dbb3801b0 0x557dbb7344b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.533+0000 7f722382d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557dbb731030 con 0x557dbb3572f0 2024-04-25T10:39:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.533+0000 7f722382d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557dbb60db60 con 0x557dbb35f4d0 2024-04-25T10:39:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.533+0000 7f722382d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557dbb7311b0 con 0x557dbb3801b0 2024-04-25T10:39:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.533+0000 7f722382d740 10 client.0 _ll_register_callbacks cb 0x557dbb6322b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.533+0000 7f7215d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3572f0 0x557dbb72e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:18.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.533+0000 7f7215d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3572f0 0x557dbb72e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58904/0 (socket says 172.21.15.43:58904) 2024-04-25T10:39:18.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.533+0000 7f7215d9b640 1 -- 172.21.15.43:0/3887178806 learned_addr learned my addr 172.21.15.43:0/3887178806 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:18.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.533+0000 7f721659c640 1 --2- 172.21.15.43:0/3887178806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb35f4d0 0x557dbb72ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:18.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.533+0000 7f7215d9b640 1 --2- 172.21.15.43:0/3887178806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557dbb3801b0 0x557dbb7344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:18.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.533+0000 7f7215d9b640 1 -- 172.21.15.43:0/3887178806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557dbb3801b0 msgr2=0x557dbb7344b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:18.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.533+0000 7f7215d9b640 1 --2- 172.21.15.43:0/3887178806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557dbb3801b0 0x557dbb7344b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.533+0000 7f7215d9b640 1 -- 172.21.15.43:0/3887178806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb35f4d0 msgr2=0x557dbb72ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:18.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.533+0000 7f7215d9b640 1 --2- 172.21.15.43:0/3887178806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb35f4d0 0x557dbb72ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.533+0000 7f7215d9b640 1 -- 172.21.15.43:0/3887178806 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dbb398210 con 0x557dbb3572f0 2024-04-25T10:39:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.533+0000 7f721659c640 1 --2- 172.21.15.43:0/3887178806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb35f4d0 0x557dbb72ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.533+0000 7f7215d9b640 1 --2- 172.21.15.43:0/3887178806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3572f0 0x557dbb72e550 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f720c02a450 tx=0x7f720c0052a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85661d9bd0ad75d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.533+0000 7f71fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.534+0000 7f71fdffb640 1 -- 172.21.15.43:0/3887178806 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c051060 con 0x557dbb3572f0 2024-04-25T10:39:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.534+0000 7f71fdffb640 1 -- 172.21.15.43:0/3887178806 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f720c004470 con 0x557dbb3572f0 2024-04-25T10:39:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.534+0000 7f722382d740 1 -- 172.21.15.43:0/3887178806 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f720c0316f0 con 0x557dbb3572f0 2024-04-25T10:39:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.534+0000 7f71fdffb640 1 -- 172.21.15.43:0/3887178806 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f720c035040 con 0x557dbb3572f0 2024-04-25T10:39:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.534+0000 7f722382d740 10 client.17448 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.534+0000 7f722382d740 1 -- 172.21.15.43:0/3887178806 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557dbb464620 con 0x557dbb3572f0 2024-04-25T10:39:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.534+0000 7f71fdffb640 1 -- 172.21.15.43:0/3887178806 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f720c0316f0 con 0x557dbb3572f0 2024-04-25T10:39:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.534+0000 7f71fdffb640 1 -- 172.21.15.43:0/3887178806 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f720c03a6f0 con 0x557dbb3572f0 2024-04-25T10:39:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.534+0000 7f722382d740 10 client.17448 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.534+0000 7f722382d740 10 client.17448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.534+0000 7f722382d740 20 client.17448 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.534+0000 7f722382d740 1 --2- 172.21.15.43:0/3887178806 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557dbb33ed80 0x557dbb740930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.534+0000 7f722382d740 4 client.17448 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:39:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.534+0000 7f722382d740 1 -- 172.21.15.43:0/3887178806 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dbb694070 con 0x557dbb33ed80 2024-04-25T10:39:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.535+0000 7f721659c640 1 --2- 172.21.15.43:0/3887178806 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557dbb33ed80 0x557dbb740930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.535+0000 7f721659c640 1 --2- 172.21.15.43:0/3887178806 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557dbb33ed80 0x557dbb740930 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6baabbf6fd8b7115 server_cookie=693a338eb33686df in_seq=0 out_seq=0 2024-04-25T10:39:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.535+0000 7f71fdffb640 10 client.17448 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.536+0000 7f71fdffb640 1 -- 172.21.15.43:0/3887178806 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557dbb694070 con 0x557dbb33ed80 2024-04-25T10:39:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.536+0000 7f71fdffb640 10 client.17448 handle_command_reply: tid=0 2024-04-25T10:39:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.536+0000 7f722382d740 1 client.17448 shutdown 2024-04-25T10:39:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.536+0000 7f722382d740 1 -- 172.21.15.43:0/3887178806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3572f0 msgr2=0x557dbb72e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.536+0000 7f722382d740 1 --2- 172.21.15.43:0/3887178806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3572f0 0x557dbb72e550 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f720c02a450 tx=0x7f720c0052a0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.536+0000 7f722382d740 1 -- 172.21.15.43:0/3887178806 shutdown_connections 2024-04-25T10:39:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.536+0000 7f722382d740 1 --2- 172.21.15.43:0/3887178806 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557dbb33ed80 0x557dbb740930 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.536+0000 7f722382d740 1 --2- 172.21.15.43:0/3887178806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb35f4d0 0x557dbb72ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.536+0000 7f722382d740 1 --2- 172.21.15.43:0/3887178806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557dbb3801b0 0x557dbb7344b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.536+0000 7f722382d740 1 --2- 172.21.15.43:0/3887178806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3572f0 0x557dbb72e550 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.536+0000 7f71fdffb640 0 client.17448 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.536+0000 7f722382d740 1 -- 172.21.15.43:0/3887178806 >> 172.21.15.43:0/3887178806 conn(0x557dbb374300 msgr2=0x557dbb721700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.536+0000 7f722382d740 1 -- 172.21.15.43:0/3887178806 shutdown_connections 2024-04-25T10:39:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.536+0000 7f722382d740 1 -- 172.21.15.43:0/3887178806 wait complete. 2024-04-25T10:39:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.536+0000 7f722382d740 20 client.17448 trim_cache size 0 max 16384 2024-04-25T10:39:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.551+0000 7f722382d740 1 Processor -- start 2024-04-25T10:39:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.551+0000 7f722382d740 1 -- start start 2024-04-25T10:39:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.552+0000 7f722382d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557dbb3572f0 0x557dbb71b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.552+0000 7f722382d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb35f4d0 0x557dbb734420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.552+0000 7f722382d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3801b0 0x557dbb720890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.552+0000 7f722382d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f72181b0080 con 0x557dbb3801b0 2024-04-25T10:39:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.552+0000 7f722382d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f72181ad950 con 0x557dbb35f4d0 2024-04-25T10:39:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.552+0000 7f722382d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f72181ab7a0 con 0x557dbb3572f0 2024-04-25T10:39:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.552+0000 7f7215d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3801b0 0x557dbb720890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.552+0000 7f7215d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3801b0 0x557dbb720890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58918/0 (socket says 172.21.15.43:58918) 2024-04-25T10:39:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.552+0000 7f7215d9b640 1 -- 172.21.15.43:0/2346108675 learned_addr learned my addr 172.21.15.43:0/2346108675 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.552+0000 7f721659c640 1 --2- 172.21.15.43:0/2346108675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb35f4d0 0x557dbb734420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.552+0000 7f7215d9b640 1 --2- 172.21.15.43:0/2346108675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557dbb3572f0 0x557dbb71b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.552+0000 7f7215d9b640 1 -- 172.21.15.43:0/2346108675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557dbb3572f0 msgr2=0x557dbb71b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.552+0000 7f7215d9b640 1 --2- 172.21.15.43:0/2346108675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557dbb3572f0 0x557dbb71b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.552+0000 7f7215d9b640 1 -- 172.21.15.43:0/2346108675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb35f4d0 msgr2=0x557dbb734420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.552+0000 7f7215d9b640 1 --2- 172.21.15.43:0/2346108675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb35f4d0 0x557dbb734420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.552+0000 7f7215d9b640 1 -- 172.21.15.43:0/2346108675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dbb71ce70 con 0x557dbb3801b0 2024-04-25T10:39:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.552+0000 7f7215d9b640 1 --2- 172.21.15.43:0/2346108675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3801b0 0x557dbb720890 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f720c02a930 tx=0x7f720c005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440aa2519c23ec73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.553+0000 7f71fffff640 1 -- 172.21.15.43:0/2346108675 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c051040 con 0x557dbb3801b0 2024-04-25T10:39:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.553+0000 7f71fffff640 1 -- 172.21.15.43:0/2346108675 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f720c032d80 con 0x557dbb3801b0 2024-04-25T10:39:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.553+0000 7f71fffff640 1 -- 172.21.15.43:0/2346108675 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c0038e0 con 0x557dbb3801b0 2024-04-25T10:39:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.553+0000 7f722382d740 1 -- 172.21.15.43:0/2346108675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3801b0 msgr2=0x557dbb720890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.553+0000 7f722382d740 1 --2- 172.21.15.43:0/2346108675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3801b0 0x557dbb720890 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f720c02a930 tx=0x7f720c005f30 comp rx=0 tx=0).stop 2024-04-25T10:39:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.553+0000 7f722382d740 1 -- 172.21.15.43:0/2346108675 shutdown_connections 2024-04-25T10:39:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.553+0000 7f722382d740 1 --2- 172.21.15.43:0/2346108675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb35f4d0 0x557dbb734420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.553+0000 7f722382d740 1 --2- 172.21.15.43:0/2346108675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb3801b0 0x557dbb720890 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.553+0000 7f722382d740 1 --2- 172.21.15.43:0/2346108675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557dbb3572f0 0x557dbb71b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.553+0000 7f722382d740 1 -- 172.21.15.43:0/2346108675 >> 172.21.15.43:0/2346108675 conn(0x557dbb374300 msgr2=0x557dbb72fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.553+0000 7f722382d740 1 -- 172.21.15.43:0/2346108675 shutdown_connections 2024-04-25T10:39:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.553+0000 7f722382d740 1 -- 172.21.15.43:0/2346108675 wait complete. 2024-04-25T10:39:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.554+0000 7f722382d740 1 -- start start 2024-04-25T10:39:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.554+0000 7f722382d740 1 Processor -- start 2024-04-25T10:39:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.554+0000 7f722382d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb3572f0 0x557dbb73a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.554+0000 7f722382d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb35f4d0 0x557dbb73abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.554+0000 7f722382d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557dbb3801b0 0x557dbb735b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.554+0000 7f722382d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f72181abe10 con 0x557dbb35f4d0 2024-04-25T10:39:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.554+0000 7f722382d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f72181ad280 con 0x557dbb3572f0 2024-04-25T10:39:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.554+0000 7f722382d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f72181ac4e0 con 0x557dbb3801b0 2024-04-25T10:39:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.554+0000 7f722382d740 10 client.0 _ll_register_callbacks cb 0x7f720800b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.554+0000 7f721659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb35f4d0 0x557dbb73abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.554+0000 7f721659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb35f4d0 0x557dbb73abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58934/0 (socket says 172.21.15.43:58934) 2024-04-25T10:39:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.554+0000 7f721659c640 1 -- 172.21.15.43:0/1550234677 learned_addr learned my addr 172.21.15.43:0/1550234677 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.555+0000 7f7215d9b640 1 --2- 172.21.15.43:0/1550234677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557dbb3801b0 0x557dbb735b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.555+0000 7f7215d9b640 1 --2- 172.21.15.43:0/1550234677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb3572f0 0x557dbb73a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.555+0000 7f721659c640 1 -- 172.21.15.43:0/1550234677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557dbb3801b0 msgr2=0x557dbb735b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.555+0000 7f721659c640 1 --2- 172.21.15.43:0/1550234677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557dbb3801b0 0x557dbb735b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.555+0000 7f721659c640 1 -- 172.21.15.43:0/1550234677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb3572f0 msgr2=0x557dbb73a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.555+0000 7f721659c640 1 --2- 172.21.15.43:0/1550234677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb3572f0 0x557dbb73a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.555+0000 7f721659c640 1 -- 172.21.15.43:0/1550234677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dbb736240 con 0x557dbb35f4d0 2024-04-25T10:39:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.555+0000 7f721659c640 1 --2- 172.21.15.43:0/1550234677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb35f4d0 0x557dbb73abf0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f7208000c00 tx=0x7f720800bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e29c4f7f7df2e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.555+0000 7f71fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.555+0000 7f71fdffb640 1 -- 172.21.15.43:0/1550234677 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720801fad0 con 0x557dbb35f4d0 2024-04-25T10:39:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.555+0000 7f71fdffb640 1 -- 172.21.15.43:0/1550234677 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f7208005220 con 0x557dbb35f4d0 2024-04-25T10:39:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.555+0000 7f722382d740 1 -- 172.21.15.43:0/1550234677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f720c032d80 con 0x557dbb35f4d0 2024-04-25T10:39:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.555+0000 7f71fdffb640 1 -- 172.21.15.43:0/1550234677 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7208005420 con 0x557dbb35f4d0 2024-04-25T10:39:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.555+0000 7f722382d740 10 client.17454 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.555+0000 7f722382d740 1 -- 172.21.15.43:0/1550234677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557dbb7a8420 con 0x557dbb35f4d0 2024-04-25T10:39:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.555+0000 7f71fdffb640 1 -- 172.21.15.43:0/1550234677 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f720c032d80 con 0x557dbb35f4d0 2024-04-25T10:39:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.556+0000 7f71fdffb640 1 -- 172.21.15.43:0/1550234677 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7208003a00 con 0x557dbb35f4d0 2024-04-25T10:39:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.556+0000 7f722382d740 10 client.17454 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.556+0000 7f722382d740 10 client.17454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.556+0000 7f722382d740 20 client.17454 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.556+0000 7f722382d740 1 --2- 172.21.15.43:0/1550234677 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557dbb3141c0 0x557dbb7aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.556+0000 7f722382d740 4 client.17454 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:39:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.556+0000 7f722382d740 1 -- 172.21.15.43:0/1550234677 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557dbb68dee0 con 0x557dbb3141c0 2024-04-25T10:39:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.556+0000 7f7215d9b640 1 --2- 172.21.15.43:0/1550234677 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557dbb3141c0 0x557dbb7aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.556+0000 7f7215d9b640 1 --2- 172.21.15.43:0/1550234677 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557dbb3141c0 0x557dbb7aabc0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df2dcf95747af721 server_cookie=fe7e3c6f296e5dea in_seq=0 out_seq=0 2024-04-25T10:39:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.556+0000 7f71fdffb640 10 client.17454 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.572+0000 7f71fdffb640 1 -- 172.21.15.43:0/1550234677 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557dbb68dee0 con 0x557dbb3141c0 2024-04-25T10:39:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.572+0000 7f71fdffb640 10 client.17454 handle_command_reply: tid=0 2024-04-25T10:39:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.573+0000 7f722382d740 1 client.17454 shutdown 2024-04-25T10:39:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.573+0000 7f722382d740 1 -- 172.21.15.43:0/1550234677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb35f4d0 msgr2=0x557dbb73abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.573+0000 7f722382d740 1 --2- 172.21.15.43:0/1550234677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb35f4d0 0x557dbb73abf0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f7208000c00 tx=0x7f720800bd00 comp rx=0 tx=0).stop 2024-04-25T10:39:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.573+0000 7f722382d740 1 -- 172.21.15.43:0/1550234677 shutdown_connections 2024-04-25T10:39:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.573+0000 7f722382d740 1 --2- 172.21.15.43:0/1550234677 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557dbb3141c0 0x557dbb7aabc0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.573+0000 7f722382d740 1 --2- 172.21.15.43:0/1550234677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557dbb3801b0 0x557dbb735b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.573+0000 7f71fdffb640 0 client.17454 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.573+0000 7f722382d740 1 --2- 172.21.15.43:0/1550234677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dbb35f4d0 0x557dbb73abf0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.573+0000 7f722382d740 1 --2- 172.21.15.43:0/1550234677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dbb3572f0 0x557dbb73a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.573+0000 7f722382d740 1 -- 172.21.15.43:0/1550234677 >> 172.21.15.43:0/1550234677 conn(0x557dbb374300 msgr2=0x557dbb71afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.573+0000 7f722382d740 1 -- 172.21.15.43:0/1550234677 shutdown_connections 2024-04-25T10:39:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.573+0000 7f722382d740 1 -- 172.21.15.43:0/1550234677 wait complete. 2024-04-25T10:39:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.573+0000 7f722382d740 20 client.17454 trim_cache size 0 max 16384 2024-04-25T10:39:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:39:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:39:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:39:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:39:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.574+0000 7f721c97d640 1 -- 172.21.15.43:0/198678194 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f72000afc50 msgr2=0x7f72000b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.574+0000 7f721c97d640 1 --2- 172.21.15.43:0/198678194 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f72000afc50 0x7f72000b2160 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f72100dc610 tx=0x7f7210004020 comp rx=0 tx=0).stop 2024-04-25T10:39:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.574+0000 7f721c97d640 1 -- 172.21.15.43:0/198678194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7218108f00 msgr2=0x7f721819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.574+0000 7f721c97d640 1 --2- 172.21.15.43:0/198678194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7218108f00 0x7f721819a690 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f7210009fd0 tx=0x7f721000b260 comp rx=0 tx=0).stop 2024-04-25T10:39:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.574+0000 7f721c97d640 1 -- 172.21.15.43:0/198678194 shutdown_connections 2024-04-25T10:39:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.574+0000 7f721c97d640 1 --2- 172.21.15.43:0/198678194 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f72000afc50 0x7f72000b2160 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.574+0000 7f721c97d640 1 --2- 172.21.15.43:0/198678194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7218108a60 0x7f72181a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.574+0000 7f721c97d640 1 --2- 172.21.15.43:0/198678194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7218108f00 0x7f721819a690 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.574+0000 7f721c97d640 1 --2- 172.21.15.43:0/198678194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72181031c0 0x7f72181a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.574+0000 7f721c97d640 1 -- 172.21.15.43:0/198678194 >> 172.21.15.43:0/198678194 conn(0x7f72180781b0 msgr2=0x7f721806a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.574+0000 7f721c97d640 1 -- 172.21.15.43:0/198678194 shutdown_connections 2024-04-25T10:39:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:18.574+0000 7f721c97d640 1 -- 172.21.15.43:0/198678194 wait complete. 2024-04-25T10:39:20.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:19 smithi077 ceph-mon[32772]: pgmap v209: 97 pgs: 97 active+clean; 2.6 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-25T10:39:20.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:19 smithi164 ceph-mon[32713]: pgmap v209: 97 pgs: 97 active+clean; 2.6 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-25T10:39:20.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:19 smithi043 ceph-mon[29478]: pgmap v209: 97 pgs: 97 active+clean; 2.6 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-25T10:39:22.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:21 smithi077 ceph-mon[32772]: pgmap v210: 97 pgs: 97 active+clean; 2.6 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-25T10:39:22.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:21 smithi164 ceph-mon[32713]: pgmap v210: 97 pgs: 97 active+clean; 2.6 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-25T10:39:22.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:21 smithi043 ceph-mon[29478]: pgmap v210: 97 pgs: 97 active+clean; 2.6 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-25T10:39:24.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:23 smithi077 ceph-mon[32772]: pgmap v211: 97 pgs: 97 active+clean; 2.6 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-25T10:39:24.413 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:23 smithi043 ceph-mon[29478]: pgmap v211: 97 pgs: 97 active+clean; 2.6 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-25T10:39:24.413 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:23 smithi164 ceph-mon[32713]: pgmap v211: 97 pgs: 97 active+clean; 2.6 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-25T10:39:24.414 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.414 INFO:tasks.workunit.client.0.smithi043.stdout:251/600: open fd = 3 2024-04-25T10:39:24.414 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.414 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.414 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.414 INFO:tasks.workunit.client.0.smithi043.stdout:252/600: open fd = 3 2024-04-25T10:39:24.414 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.414 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.414 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.414 INFO:tasks.workunit.client.0.smithi043.stdout:253/600: open fd = 3 2024-04-25T10:39:24.414 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.414 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.415 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.415 INFO:tasks.workunit.client.0.smithi043.stdout:254/600: open fd = 3 2024-04-25T10:39:24.415 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.415 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.415 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.415 INFO:tasks.workunit.client.0.smithi043.stdout:255/600: open fd = 3 2024-04-25T10:39:24.415 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.415 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.415 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.415 INFO:tasks.workunit.client.0.smithi043.stdout:256/600: open fd = 3 2024-04-25T10:39:24.415 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.415 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.415 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.415 INFO:tasks.workunit.client.0.smithi043.stdout:257/600: open fd = 3 2024-04-25T10:39:24.416 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.416 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.416 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.416 INFO:tasks.workunit.client.0.smithi043.stdout:258/600: open fd = 3 2024-04-25T10:39:24.416 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.416 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.416 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.416 INFO:tasks.workunit.client.0.smithi043.stdout:259/600: open fd = 3 2024-04-25T10:39:24.416 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.416 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.416 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.417 INFO:tasks.workunit.client.0.smithi043.stdout:260/600: open fd = 3 2024-04-25T10:39:24.417 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.417 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.417 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.417 INFO:tasks.workunit.client.0.smithi043.stdout:261/600: open fd = 3 2024-04-25T10:39:24.417 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.417 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.417 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.417 INFO:tasks.workunit.client.0.smithi043.stdout:262/600: open fd = 3 2024-04-25T10:39:24.417 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.417 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.417 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.417 INFO:tasks.workunit.client.0.smithi043.stdout:263/600: open fd = 3 2024-04-25T10:39:24.417 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.418 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.418 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.418 INFO:tasks.workunit.client.0.smithi043.stdout:264/600: open fd = 3 2024-04-25T10:39:24.418 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.418 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.418 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.418 INFO:tasks.workunit.client.0.smithi043.stdout:265/600: open fd = 3 2024-04-25T10:39:24.418 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.418 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.418 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.418 INFO:tasks.workunit.client.0.smithi043.stdout:266/600: open fd = 3 2024-04-25T10:39:24.418 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.418 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.419 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.419 INFO:tasks.workunit.client.0.smithi043.stdout:267/600: open fd = 3 2024-04-25T10:39:24.419 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.419 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.419 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.419 INFO:tasks.workunit.client.0.smithi043.stdout:268/600: open fd = 3 2024-04-25T10:39:24.419 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.419 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.419 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.419 INFO:tasks.workunit.client.0.smithi043.stdout:269/600: open fd = 3 2024-04-25T10:39:24.419 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.419 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.419 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.419 INFO:tasks.workunit.client.0.smithi043.stdout:270/600: open fd = 3 2024-04-25T10:39:24.419 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.420 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.420 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.420 INFO:tasks.workunit.client.0.smithi043.stdout:271/600: open fd = 3 2024-04-25T10:39:24.420 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.420 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.420 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.420 INFO:tasks.workunit.client.0.smithi043.stdout:272/600: open fd = 3 2024-04-25T10:39:24.420 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.420 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.420 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.420 INFO:tasks.workunit.client.0.smithi043.stdout:273/600: open fd = 3 2024-04-25T10:39:24.420 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.421 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.421 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.421 INFO:tasks.workunit.client.0.smithi043.stdout:274/600: open fd = 3 2024-04-25T10:39:24.421 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.421 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.421 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.421 INFO:tasks.workunit.client.0.smithi043.stdout:275/600: open fd = 3 2024-04-25T10:39:24.421 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.421 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.421 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.421 INFO:tasks.workunit.client.0.smithi043.stdout:276/600: open fd = 3 2024-04-25T10:39:24.421 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.422 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.422 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.422 INFO:tasks.workunit.client.0.smithi043.stdout:277/600: open fd = 3 2024-04-25T10:39:24.422 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.422 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.422 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.422 INFO:tasks.workunit.client.0.smithi043.stdout:278/600: open fd = 3 2024-04-25T10:39:24.422 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.422 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.422 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.422 INFO:tasks.workunit.client.0.smithi043.stdout:279/600: open fd = 3 2024-04-25T10:39:24.423 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.423 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.423 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.423 INFO:tasks.workunit.client.0.smithi043.stdout:280/600: open fd = 3 2024-04-25T10:39:24.423 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.423 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.423 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.423 INFO:tasks.workunit.client.0.smithi043.stdout:281/600: open fd = 3 2024-04-25T10:39:24.423 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.423 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.423 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.424 INFO:tasks.workunit.client.0.smithi043.stdout:282/600: open fd = 3 2024-04-25T10:39:24.424 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.424 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.424 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.424 INFO:tasks.workunit.client.0.smithi043.stdout:283/600: open fd = 3 2024-04-25T10:39:24.424 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.424 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.424 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.424 INFO:tasks.workunit.client.0.smithi043.stdout:284/600: open fd = 3 2024-04-25T10:39:24.424 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.424 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.424 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.425 INFO:tasks.workunit.client.0.smithi043.stdout:285/600: open fd = 3 2024-04-25T10:39:24.425 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.425 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.425 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.425 INFO:tasks.workunit.client.0.smithi043.stdout:286/600: open fd = 3 2024-04-25T10:39:24.425 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.425 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.425 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.425 INFO:tasks.workunit.client.0.smithi043.stdout:287/600: open fd = 3 2024-04-25T10:39:24.425 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.425 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.425 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.425 INFO:tasks.workunit.client.0.smithi043.stdout:288/600: open fd = 3 2024-04-25T10:39:24.425 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.425 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.426 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.426 INFO:tasks.workunit.client.0.smithi043.stdout:289/600: open fd = 3 2024-04-25T10:39:24.426 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.426 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.426 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.426 INFO:tasks.workunit.client.0.smithi043.stdout:290/600: open fd = 3 2024-04-25T10:39:24.426 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.426 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.426 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.426 INFO:tasks.workunit.client.0.smithi043.stdout:291/600: open fd = 3 2024-04-25T10:39:24.426 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.426 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.427 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.427 INFO:tasks.workunit.client.0.smithi043.stdout:292/600: open fd = 3 2024-04-25T10:39:24.427 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.427 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.427 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.427 INFO:tasks.workunit.client.0.smithi043.stdout:293/600: open fd = 3 2024-04-25T10:39:24.427 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.427 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.427 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.427 INFO:tasks.workunit.client.0.smithi043.stdout:294/600: open fd = 3 2024-04-25T10:39:24.427 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.427 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.427 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.427 INFO:tasks.workunit.client.0.smithi043.stdout:295/600: open fd = 3 2024-04-25T10:39:24.428 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.428 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.428 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.428 INFO:tasks.workunit.client.0.smithi043.stdout:296/600: open fd = 3 2024-04-25T10:39:24.428 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.428 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.428 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.428 INFO:tasks.workunit.client.0.smithi043.stdout:297/600: open fd = 3 2024-04-25T10:39:24.428 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.428 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.428 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.428 INFO:tasks.workunit.client.0.smithi043.stdout:298/600: open fd = 3 2024-04-25T10:39:24.428 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.429 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.429 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.429 INFO:tasks.workunit.client.0.smithi043.stdout:299/600: open fd = 3 2024-04-25T10:39:24.429 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.429 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.429 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.429 INFO:tasks.workunit.client.0.smithi043.stdout:300/600: open fd = 3 2024-04-25T10:39:24.429 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.429 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.429 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.429 INFO:tasks.workunit.client.0.smithi043.stdout:301/600: open fd = 3 2024-04-25T10:39:24.429 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.429 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.430 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.430 INFO:tasks.workunit.client.0.smithi043.stdout:302/600: open fd = 3 2024-04-25T10:39:24.430 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.430 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.430 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.430 INFO:tasks.workunit.client.0.smithi043.stdout:303/600: open fd = 3 2024-04-25T10:39:24.430 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.430 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.430 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.430 INFO:tasks.workunit.client.0.smithi043.stdout:304/600: open fd = 3 2024-04-25T10:39:24.430 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.431 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.431 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.431 INFO:tasks.workunit.client.0.smithi043.stdout:305/600: open fd = 3 2024-04-25T10:39:24.431 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.431 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.431 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.431 INFO:tasks.workunit.client.0.smithi043.stdout:306/600: open fd = 3 2024-04-25T10:39:24.431 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.431 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.431 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.431 INFO:tasks.workunit.client.0.smithi043.stdout:307/600: open fd = 3 2024-04-25T10:39:24.431 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.431 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.432 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.432 INFO:tasks.workunit.client.0.smithi043.stdout:308/600: open fd = 3 2024-04-25T10:39:24.432 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.432 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.432 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.432 INFO:tasks.workunit.client.0.smithi043.stdout:309/600: open fd = 3 2024-04-25T10:39:24.432 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.432 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.432 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.432 INFO:tasks.workunit.client.0.smithi043.stdout:310/600: open fd = 3 2024-04-25T10:39:24.432 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.432 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.433 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.433 INFO:tasks.workunit.client.0.smithi043.stdout:311/600: open fd = 3 2024-04-25T10:39:24.433 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.433 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.433 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:39:24.433 INFO:tasks.workunit.client.0.smithi043.stdout:312/600: open fd = 3 2024-04-25T10:39:24.433 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:24.433 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:39:26.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:25 smithi077 ceph-mon[32772]: pgmap v212: 97 pgs: 97 active+clean; 2.8 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 19 op/s 2024-04-25T10:39:26.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:25 smithi164 ceph-mon[32713]: pgmap v212: 97 pgs: 97 active+clean; 2.8 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 19 op/s 2024-04-25T10:39:26.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:25 smithi043 ceph-mon[29478]: pgmap v212: 97 pgs: 97 active+clean; 2.8 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 19 op/s 2024-04-25T10:39:28.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:27 smithi077 ceph-mon[32772]: pgmap v213: 97 pgs: 97 active+clean; 2.8 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-04-25T10:39:28.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:27 smithi164 ceph-mon[32713]: pgmap v213: 97 pgs: 97 active+clean; 2.8 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-04-25T10:39:28.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:27 smithi043 ceph-mon[29478]: pgmap v213: 97 pgs: 97 active+clean; 2.8 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-04-25T10:39:30.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:29 smithi077 ceph-mon[32772]: pgmap v214: 97 pgs: 97 active+clean; 3.0 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 17 op/s 2024-04-25T10:39:30.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:29 smithi164 ceph-mon[32713]: pgmap v214: 97 pgs: 97 active+clean; 3.0 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 17 op/s 2024-04-25T10:39:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:29 smithi043 ceph-mon[29478]: pgmap v214: 97 pgs: 97 active+clean; 3.0 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 17 op/s 2024-04-25T10:39:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.685+0000 7f1e0059e640 1 -- 172.21.15.43:0/1888173116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df8109bf0 msgr2=0x7f1df8101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.685+0000 7f1e0059e640 1 --2- 172.21.15.43:0/1888173116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df8109bf0 0x7f1df8101b60 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009900 tx=0x7f1df402a410 comp rx=0 tx=0).stop 2024-04-25T10:39:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.686+0000 7f1e0059e640 1 -- 172.21.15.43:0/1888173116 shutdown_connections 2024-04-25T10:39:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.686+0000 7f1e0059e640 1 --2- 172.21.15.43:0/1888173116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1df810e5a0 0x7f1df81109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.686+0000 7f1e0059e640 1 --2- 172.21.15.43:0/1888173116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1df81020a0 0x7f1df81024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.686+0000 7f1e0059e640 1 --2- 172.21.15.43:0/1888173116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df8109bf0 0x7f1df8101b60 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.686+0000 7f1e0059e640 1 -- 172.21.15.43:0/1888173116 >> 172.21.15.43:0/1888173116 conn(0x7f1df80fd8b0 msgr2=0x7f1df80ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.686+0000 7f1e0059e640 1 -- 172.21.15.43:0/1888173116 shutdown_connections 2024-04-25T10:39:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.686+0000 7f1e0059e640 1 -- 172.21.15.43:0/1888173116 wait complete. 2024-04-25T10:39:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.686+0000 7f1e0059e640 1 Processor -- start 2024-04-25T10:39:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.686+0000 7f1e0059e640 1 -- start start 2024-04-25T10:39:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.687+0000 7f1e0059e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1df81020a0 0x7f1df819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.687+0000 7f1e0059e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df8109bf0 0x7f1df819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.687+0000 7f1e0059e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1df810e5a0 0x7f1df81a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.687+0000 7f1e0059e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1df8113d60 con 0x7f1df8109bf0 2024-04-25T10:39:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.687+0000 7f1e0059e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1df8113be0 con 0x7f1df810e5a0 2024-04-25T10:39:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.687+0000 7f1e0059e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1df8113ee0 con 0x7f1df81020a0 2024-04-25T10:39:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.687+0000 7f1dfdb39640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df8109bf0 0x7f1df819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.687+0000 7f1dfdb39640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df8109bf0 0x7f1df819db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48400/0 (socket says 172.21.15.43:48400) 2024-04-25T10:39:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.687+0000 7f1dfdb39640 1 -- 172.21.15.43:0/2510291609 learned_addr learned my addr 172.21.15.43:0/2510291609 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.687+0000 7f1dfeb3b640 1 --2- 172.21.15.43:0/2510291609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1df810e5a0 0x7f1df81a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.687+0000 7f1dfe33a640 1 --2- 172.21.15.43:0/2510291609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1df81020a0 0x7f1df819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.687+0000 7f1dfdb39640 1 -- 172.21.15.43:0/2510291609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1df81020a0 msgr2=0x7f1df819d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.687+0000 7f1dfdb39640 1 --2- 172.21.15.43:0/2510291609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1df81020a0 0x7f1df819d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.687+0000 7f1dfdb39640 1 -- 172.21.15.43:0/2510291609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1df810e5a0 msgr2=0x7f1df81a86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.687+0000 7f1dfdb39640 1 --2- 172.21.15.43:0/2510291609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1df810e5a0 0x7f1df81a86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.687+0000 7f1dfdb39640 1 -- 172.21.15.43:0/2510291609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df81a8be0 con 0x7f1df8109bf0 2024-04-25T10:39:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.687+0000 7f1dfdb39640 1 --2- 172.21.15.43:0/2510291609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df8109bf0 0x7f1df819db60 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1de800d740 tx=0x7f1de800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.687+0000 7f1def7fe640 1 -- 172.21.15.43:0/2510291609 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8017070 con 0x7f1df8109bf0 2024-04-25T10:39:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.688+0000 7f1def7fe640 1 -- 172.21.15.43:0/2510291609 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1de800de50 con 0x7f1df8109bf0 2024-04-25T10:39:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.688+0000 7f1e0059e640 1 -- 172.21.15.43:0/2510291609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df81a8db0 con 0x7f1df8109bf0 2024-04-25T10:39:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.688+0000 7f1def7fe640 1 -- 172.21.15.43:0/2510291609 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8004c10 con 0x7f1df8109bf0 2024-04-25T10:39:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.688+0000 7f1e0059e640 1 -- 172.21.15.43:0/2510291609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df81a9220 con 0x7f1df8109bf0 2024-04-25T10:39:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.691+0000 7f1e0744b740 1 Processor -- start 2024-04-25T10:39:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.691+0000 7f1def7fe640 1 -- 172.21.15.43:0/2510291609 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f1de8004e90 con 0x7f1df8109bf0 2024-04-25T10:39:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.691+0000 7f1e0744b740 1 -- start start 2024-04-25T10:39:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.691+0000 7f1e0744b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e2c1b0 0x55de791cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.691+0000 7f1e0744b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e032f0 0x55de791cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.691+0000 7f1e0744b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e0b4d0 0x55de791cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.691+0000 7f1e0744b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55de79132460 con 0x55de78e0b4d0 2024-04-25T10:39:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.691+0000 7f1e0744b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55de790ec590 con 0x55de78e032f0 2024-04-25T10:39:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.691+0000 7f1e0744b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55de7912e420 con 0x55de78e2c1b0 2024-04-25T10:39:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.691+0000 7f1dfdb39640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e0b4d0 0x55de791cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.691+0000 7f1dfe33a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e032f0 0x55de791cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.692+0000 7f1dfdb39640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e0b4d0 0x55de791cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48402/0 (socket says 172.21.15.43:48402) 2024-04-25T10:39:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.692+0000 7f1dfdb39640 1 -- 172.21.15.43:0/1721749428 learned_addr learned my addr 172.21.15.43:0/1721749428 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.692+0000 7f1def7fe640 1 --2- 172.21.15.43:0/2510291609 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1de40afc50 0x7f1de40b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.692+0000 7f1dfeb3b640 1 --2- 172.21.15.43:0/1721749428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e2c1b0 0x55de791cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.692+0000 7f1dfeb3b640 1 --2- 172.21.15.43:0/2510291609 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1de40afc50 0x7f1de40b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.692+0000 7f1dfe33a640 1 -- 172.21.15.43:0/1721749428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e2c1b0 msgr2=0x55de791cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.692+0000 7f1dfe33a640 1 --2- 172.21.15.43:0/1721749428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e2c1b0 0x55de791cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.692+0000 7f1dfe33a640 1 -- 172.21.15.43:0/1721749428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e0b4d0 msgr2=0x55de791cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.692+0000 7f1dfe33a640 1 --2- 172.21.15.43:0/1721749428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e0b4d0 0x55de791cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.692+0000 7f1dfe33a640 1 -- 172.21.15.43:0/1721749428 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de790f90d0 con 0x55de78e032f0 2024-04-25T10:39:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.692+0000 7f1def7fe640 1 -- 172.21.15.43:0/2510291609 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f1de80dea50 con 0x7f1df8109bf0 2024-04-25T10:39:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.692+0000 7f1dfe33a640 1 --2- 172.21.15.43:0/1721749428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e032f0 0x55de791cbf20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1df402a930 tx=0x7f1df4032ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f081a95edf2a5b4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.692+0000 7f1dfeb3b640 1 --2- 172.21.15.43:0/2510291609 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1de40afc50 0x7f1de40b2160 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55de791ccba0 tx=0x7f1df0004ab0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.692+0000 7f1decbfe640 1 -- 172.21.15.43:0/1721749428 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df403f070 con 0x55de78e032f0 2024-04-25T10:39:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.692+0000 7f1decbfe640 1 -- 172.21.15.43:0/1721749428 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df4004030 con 0x55de78e032f0 2024-04-25T10:39:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.692+0000 7f1decbfe640 1 -- 172.21.15.43:0/1721749428 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4035070 con 0x55de78e032f0 2024-04-25T10:39:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.693+0000 7f1e0744b740 1 -- 172.21.15.43:0/1721749428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e032f0 msgr2=0x55de791cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.693+0000 7f1e0744b740 1 --2- 172.21.15.43:0/1721749428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e032f0 0x55de791cbf20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1df402a930 tx=0x7f1df4032ba0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.693+0000 7f1e0744b740 1 -- 172.21.15.43:0/1721749428 shutdown_connections 2024-04-25T10:39:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.693+0000 7f1e0744b740 1 --2- 172.21.15.43:0/1721749428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e032f0 0x55de791cbf20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.693+0000 7f1e0744b740 1 --2- 172.21.15.43:0/1721749428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e0b4d0 0x55de791cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.693+0000 7f1e0744b740 1 --2- 172.21.15.43:0/1721749428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e2c1b0 0x55de791cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.693+0000 7f1e0744b740 1 -- 172.21.15.43:0/1721749428 >> 172.21.15.43:0/1721749428 conn(0x55de78e20300 msgr2=0x55de791c82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.693+0000 7f1e0744b740 1 -- 172.21.15.43:0/1721749428 shutdown_connections 2024-04-25T10:39:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.694+0000 7f1e0744b740 1 -- 172.21.15.43:0/1721749428 wait complete. 2024-04-25T10:39:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.694+0000 7f1e0744b740 1 -- start start 2024-04-25T10:39:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.694+0000 7f1e0744b740 1 Processor -- start 2024-04-25T10:39:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.694+0000 7f1e0744b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e032f0 0x55de791de990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.694+0000 7f1e0744b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e0b4d0 0x55de791deed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.694+0000 7f1e0744b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e2c1b0 0x55de791e4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.694+0000 7f1e0744b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55de791e13d0 con 0x55de78e2c1b0 2024-04-25T10:39:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.694+0000 7f1e0744b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55de790b9b60 con 0x55de78e032f0 2024-04-25T10:39:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.694+0000 7f1e0744b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55de791e1550 con 0x55de78e0b4d0 2024-04-25T10:39:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.694+0000 7f1e0744b740 10 client.0 _ll_register_callbacks cb 0x55de790de2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.694+0000 7f1dfeb3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e2c1b0 0x55de791e4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.694+0000 7f1dfeb3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e2c1b0 0x55de791e4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48418/0 (socket says 172.21.15.43:48418) 2024-04-25T10:39:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.695+0000 7f1dfeb3b640 1 -- 172.21.15.43:0/3660556941 learned_addr learned my addr 172.21.15.43:0/3660556941 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.695+0000 7f1dfdb39640 1 --2- 172.21.15.43:0/3660556941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e0b4d0 0x55de791deed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.695+0000 7f1dfe33a640 1 --2- 172.21.15.43:0/3660556941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e032f0 0x55de791de990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.695+0000 7f1dfeb3b640 1 -- 172.21.15.43:0/3660556941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e0b4d0 msgr2=0x55de791deed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.695+0000 7f1dfeb3b640 1 --2- 172.21.15.43:0/3660556941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e0b4d0 0x55de791deed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.695+0000 7f1dfeb3b640 1 -- 172.21.15.43:0/3660556941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e032f0 msgr2=0x55de791de990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.695+0000 7f1dfeb3b640 1 --2- 172.21.15.43:0/3660556941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e032f0 0x55de791de990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.695+0000 7f1dfeb3b640 1 -- 172.21.15.43:0/3660556941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de78e44210 con 0x55de78e2c1b0 2024-04-25T10:39:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.695+0000 7f1dfe33a640 1 --2- 172.21.15.43:0/3660556941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e032f0 0x55de791de990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.695+0000 7f1dfeb3b640 1 --2- 172.21.15.43:0/3660556941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e2c1b0 0x55de791e4850 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f1df000fd40 tx=0x7f1df0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8151e219c860b810 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.695+0000 7f1de1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.695+0000 7f1de1ffb640 1 -- 172.21.15.43:0/3660556941 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0019070 con 0x55de78e2c1b0 2024-04-25T10:39:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.695+0000 7f1de1ffb640 1 -- 172.21.15.43:0/3660556941 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f1df001e420 con 0x55de78e2c1b0 2024-04-25T10:39:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.695+0000 7f1e0744b740 1 -- 172.21.15.43:0/3660556941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1df4004030 con 0x55de78e2c1b0 2024-04-25T10:39:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.695+0000 7f1de1ffb640 1 -- 172.21.15.43:0/3660556941 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1df001e620 con 0x55de78e2c1b0 2024-04-25T10:39:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.695+0000 7f1e0744b740 10 client.17466 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.695+0000 7f1e0744b740 1 -- 172.21.15.43:0/3660556941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55de78f10620 con 0x55de78e2c1b0 2024-04-25T10:39:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.696+0000 7f1de1ffb640 1 -- 172.21.15.43:0/3660556941 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df4004030 con 0x55de78e2c1b0 2024-04-25T10:39:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.696+0000 7f1de1ffb640 1 -- 172.21.15.43:0/3660556941 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1df00287e0 con 0x55de78e2c1b0 2024-04-25T10:39:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.696+0000 7f1e0744b740 10 client.17466 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.696+0000 7f1e0744b740 10 client.17466 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.696+0000 7f1e0744b740 20 client.17466 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.696+0000 7f1e0744b740 1 --2- 172.21.15.43:0/3660556941 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55de78dead80 0x55de791f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.696+0000 7f1e0744b740 4 client.17466 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:39:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.696+0000 7f1e0744b740 1 -- 172.21.15.43:0/3660556941 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de79140070 con 0x55de78dead80 2024-04-25T10:39:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.696+0000 7f1dfe33a640 1 --2- 172.21.15.43:0/3660556941 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55de78dead80 0x55de791f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.697+0000 7f1de1ffb640 10 client.17466 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.697+0000 7f1dfe33a640 1 --2- 172.21.15.43:0/3660556941 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55de78dead80 0x55de791f0f70 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3849f9619ffff95e server_cookie=e18eb8a2bbc03ff9 in_seq=0 out_seq=0 2024-04-25T10:39:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.697+0000 7f1de1ffb640 1 -- 172.21.15.43:0/3660556941 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55de79140070 con 0x55de78dead80 2024-04-25T10:39:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.697+0000 7f1de1ffb640 10 client.17466 handle_command_reply: tid=0 2024-04-25T10:39:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.697+0000 7f1e0744b740 1 client.17466 shutdown 2024-04-25T10:39:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.698+0000 7f1e0744b740 1 -- 172.21.15.43:0/3660556941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e2c1b0 msgr2=0x55de791e4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.698+0000 7f1e0744b740 1 --2- 172.21.15.43:0/3660556941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e2c1b0 0x55de791e4850 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f1df000fd40 tx=0x7f1df0004a40 comp rx=0 tx=0).stop 2024-04-25T10:39:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.698+0000 7f1e0744b740 1 -- 172.21.15.43:0/3660556941 shutdown_connections 2024-04-25T10:39:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.698+0000 7f1e0744b740 1 --2- 172.21.15.43:0/3660556941 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55de78dead80 0x55de791f0f70 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.698+0000 7f1e0744b740 1 --2- 172.21.15.43:0/3660556941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e2c1b0 0x55de791e4850 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.698+0000 7f1e0744b740 1 --2- 172.21.15.43:0/3660556941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e0b4d0 0x55de791deed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.698+0000 7f1de1ffb640 0 client.17466 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.698+0000 7f1e0744b740 1 --2- 172.21.15.43:0/3660556941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e032f0 0x55de791de990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.698+0000 7f1e0744b740 1 -- 172.21.15.43:0/3660556941 >> 172.21.15.43:0/3660556941 conn(0x55de78e20300 msgr2=0x55de791cd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.698+0000 7f1e0744b740 1 -- 172.21.15.43:0/3660556941 shutdown_connections 2024-04-25T10:39:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.698+0000 7f1e0744b740 1 -- 172.21.15.43:0/3660556941 wait complete. 2024-04-25T10:39:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.698+0000 7f1e0744b740 20 client.17466 trim_cache size 0 max 16384 2024-04-25T10:39:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.713+0000 7f1e0744b740 1 Processor -- start 2024-04-25T10:39:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.713+0000 7f1e0744b740 1 -- start start 2024-04-25T10:39:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.713+0000 7f1e0744b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e032f0 0x55de791de3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.713+0000 7f1e0744b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e0b4d0 0x55de791e47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.713+0000 7f1e0744b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e2c1b0 0x55de791d8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.713+0000 7f1e0744b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81b0010 con 0x55de78e032f0 2024-04-25T10:39:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.713+0000 7f1e0744b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81ad8e0 con 0x55de78e2c1b0 2024-04-25T10:39:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.713+0000 7f1e0744b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81ab7d0 con 0x55de78e0b4d0 2024-04-25T10:39:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.713+0000 7f1dfe33a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e032f0 0x55de791de3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.713+0000 7f1dfe33a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e032f0 0x55de791de3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48428/0 (socket says 172.21.15.43:48428) 2024-04-25T10:39:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.713+0000 7f1dfe33a640 1 -- 172.21.15.43:0/3758754312 learned_addr learned my addr 172.21.15.43:0/3758754312 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.714+0000 7f1dfeb3b640 1 --2- 172.21.15.43:0/3758754312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e2c1b0 0x55de791d8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.714+0000 7f1dfdb39640 1 --2- 172.21.15.43:0/3758754312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e0b4d0 0x55de791e47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.714+0000 7f1dfe33a640 1 -- 172.21.15.43:0/3758754312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e0b4d0 msgr2=0x55de791e47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.714+0000 7f1dfe33a640 1 --2- 172.21.15.43:0/3758754312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e0b4d0 0x55de791e47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.714+0000 7f1dfe33a640 1 -- 172.21.15.43:0/3758754312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e2c1b0 msgr2=0x55de791d8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.714+0000 7f1dfe33a640 1 --2- 172.21.15.43:0/3758754312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e2c1b0 0x55de791d8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.714+0000 7f1dfe33a640 1 -- 172.21.15.43:0/3758754312 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de791df670 con 0x55de78e032f0 2024-04-25T10:39:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.714+0000 7f1dfeb3b640 1 --2- 172.21.15.43:0/3758754312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e2c1b0 0x55de791d8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.714+0000 7f1dfe33a640 1 --2- 172.21.15.43:0/3758754312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e032f0 0x55de791de3d0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f1df4005f70 tx=0x7f1df40041c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ce558e316f2798 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.714+0000 7f1de3fff640 1 -- 172.21.15.43:0/3758754312 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4053070 con 0x55de78e032f0 2024-04-25T10:39:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.714+0000 7f1de3fff640 1 -- 172.21.15.43:0/3758754312 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df4005040 con 0x55de78e032f0 2024-04-25T10:39:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.714+0000 7f1de3fff640 1 -- 172.21.15.43:0/3758754312 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4035070 con 0x55de78e032f0 2024-04-25T10:39:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.715+0000 7f1e0744b740 1 -- 172.21.15.43:0/3758754312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e032f0 msgr2=0x55de791de3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.715+0000 7f1e0744b740 1 --2- 172.21.15.43:0/3758754312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e032f0 0x55de791de3d0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f1df4005f70 tx=0x7f1df40041c0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.715+0000 7f1e0744b740 1 -- 172.21.15.43:0/3758754312 shutdown_connections 2024-04-25T10:39:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.715+0000 7f1e0744b740 1 --2- 172.21.15.43:0/3758754312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e2c1b0 0x55de791d8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.715+0000 7f1e0744b740 1 --2- 172.21.15.43:0/3758754312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e0b4d0 0x55de791e47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.715+0000 7f1e0744b740 1 --2- 172.21.15.43:0/3758754312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e032f0 0x55de791de3d0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.715+0000 7f1e0744b740 1 -- 172.21.15.43:0/3758754312 >> 172.21.15.43:0/3758754312 conn(0x55de78e20300 msgr2=0x55de791ddcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.715+0000 7f1e0744b740 1 -- 172.21.15.43:0/3758754312 shutdown_connections 2024-04-25T10:39:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.715+0000 7f1e0744b740 1 -- 172.21.15.43:0/3758754312 wait complete. 2024-04-25T10:39:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.715+0000 7f1e0744b740 1 -- start start 2024-04-25T10:39:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1e0744b740 1 Processor -- start 2024-04-25T10:39:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1e0744b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e032f0 0x55de791e6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1e0744b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e0b4d0 0x55de791e65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1e0744b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e2c1b0 0x55de79251350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1e0744b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81abe20 con 0x55de78e0b4d0 2024-04-25T10:39:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1e0744b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81ad210 con 0x55de78e2c1b0 2024-04-25T10:39:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1e0744b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81ac470 con 0x55de78e032f0 2024-04-25T10:39:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1e0744b740 10 client.0 _ll_register_callbacks cb 0x7f1df0009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1dfdb39640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e0b4d0 0x55de791e65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1dfdb39640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e0b4d0 0x55de791e65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48436/0 (socket says 172.21.15.43:48436) 2024-04-25T10:39:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1dfdb39640 1 -- 172.21.15.43:0/1553215931 learned_addr learned my addr 172.21.15.43:0/1553215931 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1dfeb3b640 1 --2- 172.21.15.43:0/1553215931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e2c1b0 0x55de79251350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1dfe33a640 1 --2- 172.21.15.43:0/1553215931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e032f0 0x55de791e6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1dfdb39640 1 -- 172.21.15.43:0/1553215931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e032f0 msgr2=0x55de791e6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1dfdb39640 1 --2- 172.21.15.43:0/1553215931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e032f0 0x55de791e6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1dfdb39640 1 -- 172.21.15.43:0/1553215931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e2c1b0 msgr2=0x55de79251350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1dfdb39640 1 --2- 172.21.15.43:0/1553215931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e2c1b0 0x55de79251350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1dfdb39640 1 -- 172.21.15.43:0/1553215931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de79251910 con 0x55de78e0b4d0 2024-04-25T10:39:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1dfe33a640 1 --2- 172.21.15.43:0/1553215931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e032f0 0x55de791e6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1dfdb39640 1 --2- 172.21.15.43:0/1553215931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e0b4d0 0x55de791e65e0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f1de80a2c70 tx=0x7f1de8009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac30baaeb5a2d1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.716+0000 7f1de1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.717+0000 7f1de1ffb640 1 -- 172.21.15.43:0/1553215931 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8020040 con 0x55de78e0b4d0 2024-04-25T10:39:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.717+0000 7f1de1ffb640 1 -- 172.21.15.43:0/1553215931 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f1de80af7c0 con 0x55de78e0b4d0 2024-04-25T10:39:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.717+0000 7f1e0744b740 1 -- 172.21.15.43:0/1553215931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1df4005040 con 0x55de78e0b4d0 2024-04-25T10:39:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.717+0000 7f1de1ffb640 1 -- 172.21.15.43:0/1553215931 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1de80a6360 con 0x55de78e0b4d0 2024-04-25T10:39:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.717+0000 7f1e0744b740 10 client.17472 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.717+0000 7f1e0744b740 1 -- 172.21.15.43:0/1553215931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55de79258c50 con 0x55de78e0b4d0 2024-04-25T10:39:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.717+0000 7f1de1ffb640 1 -- 172.21.15.43:0/1553215931 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df4005040 con 0x55de78e0b4d0 2024-04-25T10:39:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.717+0000 7f1de1ffb640 1 -- 172.21.15.43:0/1553215931 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1de80a2560 con 0x55de78e0b4d0 2024-04-25T10:39:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.717+0000 7f1e0744b740 10 client.17472 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.717+0000 7f1e0744b740 10 client.17472 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.717+0000 7f1e0744b740 20 client.17472 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.717+0000 7f1e0744b740 1 --2- 172.21.15.43:0/1553215931 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55de78dc01c0 0x55de7925b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.717+0000 7f1e0744b740 4 client.17472 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:39:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.717+0000 7f1e0744b740 1 -- 172.21.15.43:0/1553215931 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de79139ee0 con 0x55de78dc01c0 2024-04-25T10:39:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.717+0000 7f1dfe33a640 1 --2- 172.21.15.43:0/1553215931 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55de78dc01c0 0x55de7925b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.718+0000 7f1dfe33a640 1 --2- 172.21.15.43:0/1553215931 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55de78dc01c0 0x55de7925b440 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f15b16a860f08fea server_cookie=c0921b836e5352e2 in_seq=0 out_seq=0 2024-04-25T10:39:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.718+0000 7f1de1ffb640 10 client.17472 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.734+0000 7f1de1ffb640 1 -- 172.21.15.43:0/1553215931 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55de79139ee0 con 0x55de78dc01c0 2024-04-25T10:39:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.734+0000 7f1de1ffb640 10 client.17472 handle_command_reply: tid=0 2024-04-25T10:39:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.734+0000 7f1e0744b740 1 client.17472 shutdown 2024-04-25T10:39:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.734+0000 7f1e0744b740 1 -- 172.21.15.43:0/1553215931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e0b4d0 msgr2=0x55de791e65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.734+0000 7f1e0744b740 1 --2- 172.21.15.43:0/1553215931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e0b4d0 0x55de791e65e0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f1de80a2c70 tx=0x7f1de8009f60 comp rx=0 tx=0).stop 2024-04-25T10:39:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.734+0000 7f1e0744b740 1 -- 172.21.15.43:0/1553215931 shutdown_connections 2024-04-25T10:39:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.734+0000 7f1e0744b740 1 --2- 172.21.15.43:0/1553215931 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55de78dc01c0 0x55de7925b440 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.734+0000 7f1e0744b740 1 --2- 172.21.15.43:0/1553215931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de78e2c1b0 0x55de79251350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.734+0000 7f1e0744b740 1 --2- 172.21.15.43:0/1553215931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de78e0b4d0 0x55de791e65e0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.734+0000 7f1e0744b740 1 --2- 172.21.15.43:0/1553215931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de78e032f0 0x55de791e6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.734+0000 7f1de1ffb640 0 client.17472 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.734+0000 7f1e0744b740 1 -- 172.21.15.43:0/1553215931 >> 172.21.15.43:0/1553215931 conn(0x55de78e20300 msgr2=0x55de791dcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.734+0000 7f1e0744b740 1 -- 172.21.15.43:0/1553215931 shutdown_connections 2024-04-25T10:39:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.734+0000 7f1e0744b740 1 -- 172.21.15.43:0/1553215931 wait complete. 2024-04-25T10:39:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.734+0000 7f1e0744b740 20 client.17472 trim_cache size 0 max 16384 2024-04-25T10:39:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:39:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:39:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:39:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:39:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.735+0000 7f1e0059e640 1 -- 172.21.15.43:0/2510291609 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1de40afc50 msgr2=0x7f1de40b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.735+0000 7f1e0059e640 1 --2- 172.21.15.43:0/2510291609 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1de40afc50 0x7f1de40b2160 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55de791ccba0 tx=0x7f1df0004ab0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.735+0000 7f1e0059e640 1 -- 172.21.15.43:0/2510291609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df8109bf0 msgr2=0x7f1df819db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.735+0000 7f1e0059e640 1 --2- 172.21.15.43:0/2510291609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df8109bf0 0x7f1df819db60 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1de800d740 tx=0x7f1de800dc00 comp rx=0 tx=0).stop 2024-04-25T10:39:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.735+0000 7f1e0059e640 1 -- 172.21.15.43:0/2510291609 shutdown_connections 2024-04-25T10:39:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.735+0000 7f1e0059e640 1 --2- 172.21.15.43:0/2510291609 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1de40afc50 0x7f1de40b2160 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.735+0000 7f1e0059e640 1 --2- 172.21.15.43:0/2510291609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1df810e5a0 0x7f1df81a86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.735+0000 7f1e0059e640 1 --2- 172.21.15.43:0/2510291609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df8109bf0 0x7f1df819db60 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.735+0000 7f1e0059e640 1 --2- 172.21.15.43:0/2510291609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1df81020a0 0x7f1df819d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.735+0000 7f1e0059e640 1 -- 172.21.15.43:0/2510291609 >> 172.21.15.43:0/2510291609 conn(0x7f1df80fd8b0 msgr2=0x7f1df8102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.735+0000 7f1e0059e640 1 -- 172.21.15.43:0/2510291609 shutdown_connections 2024-04-25T10:39:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:31.735+0000 7f1e0059e640 1 -- 172.21.15.43:0/2510291609 wait complete. 2024-04-25T10:39:32.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:31 smithi077 ceph-mon[32772]: pgmap v215: 97 pgs: 97 active+clean; 3.0 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:39:32.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:31 smithi164 ceph-mon[32713]: pgmap v215: 97 pgs: 97 active+clean; 3.0 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:39:32.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:31 smithi043 ceph-mon[29478]: pgmap v215: 97 pgs: 97 active+clean; 3.0 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:39:34.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:33 smithi077 ceph-mon[32772]: pgmap v216: 97 pgs: 97 active+clean; 3.0 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:39:34.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:33 smithi164 ceph-mon[32713]: pgmap v216: 97 pgs: 97 active+clean; 3.0 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:39:34.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:33 smithi043 ceph-mon[29478]: pgmap v216: 97 pgs: 97 active+clean; 3.0 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:39:36.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:35 smithi077 ceph-mon[32772]: pgmap v217: 97 pgs: 97 active+clean; 3.3 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 65 KiB/s wr, 20 op/s 2024-04-25T10:39:36.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:35 smithi164 ceph-mon[32713]: pgmap v217: 97 pgs: 97 active+clean; 3.3 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 65 KiB/s wr, 20 op/s 2024-04-25T10:39:36.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:35 smithi043 ceph-mon[29478]: pgmap v217: 97 pgs: 97 active+clean; 3.3 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 65 KiB/s wr, 20 op/s 2024-04-25T10:39:38.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:37 smithi077 ceph-mon[32772]: pgmap v218: 97 pgs: 97 active+clean; 3.3 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:39:38.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:37 smithi164 ceph-mon[32713]: pgmap v218: 97 pgs: 97 active+clean; 3.3 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:39:38.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:37 smithi043 ceph-mon[29478]: pgmap v218: 97 pgs: 97 active+clean; 3.3 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:39:38.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:39:38.830 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:39:38.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.956+0000 7f5b3fb64640 1 -- 172.21.15.43:0/217774846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b38102440 msgr2=0x7f5b38102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:38.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.956+0000 7f5b3fb64640 1 --2- 172.21.15.43:0/217774846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b38102440 0x7f5b38102850 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009900 tx=0x7f5b3402a410 comp rx=0 tx=0).stop 2024-04-25T10:39:38.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.956+0000 7f5b3fb64640 1 -- 172.21.15.43:0/217774846 shutdown_connections 2024-04-25T10:39:38.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.956+0000 7f5b3fb64640 1 --2- 172.21.15.43:0/217774846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b38107440 0x7f5b3810ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.956+0000 7f5b3fb64640 1 --2- 172.21.15.43:0/217774846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b38106980 0x7f5b38106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.956+0000 7f5b3fb64640 1 --2- 172.21.15.43:0/217774846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b38102440 0x7f5b38102850 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.956+0000 7f5b3fb64640 1 -- 172.21.15.43:0/217774846 >> 172.21.15.43:0/217774846 conn(0x7f5b38077880 msgr2=0x7f5b38077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:38.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.956+0000 7f5b3fb64640 1 -- 172.21.15.43:0/217774846 shutdown_connections 2024-04-25T10:39:38.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.956+0000 7f5b3fb64640 1 -- 172.21.15.43:0/217774846 wait complete. 2024-04-25T10:39:38.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.957+0000 7f5b3fb64640 1 Processor -- start 2024-04-25T10:39:38.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.957+0000 7f5b3fb64640 1 -- start start 2024-04-25T10:39:38.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.958+0000 7f5b3fb64640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b38102440 0x7f5b38199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:38.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.958+0000 7f5b3fb64640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b38106980 0x7f5b38199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:38.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.958+0000 7f5b3fb64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b38107440 0x7f5b3819de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:38.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.958+0000 7f5b3fb64640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b38112bd0 con 0x7f5b38107440 2024-04-25T10:39:38.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.958+0000 7f5b3fb64640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5b38112a50 con 0x7f5b38102440 2024-04-25T10:39:38.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.958+0000 7f5b3fb64640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5b38112d50 con 0x7f5b38106980 2024-04-25T10:39:38.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.958+0000 7f5b3e101640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b38107440 0x7f5b3819de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:38.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.958+0000 7f5b3d900640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b38102440 0x7f5b38199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:38.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.958+0000 7f5b3d900640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b38102440 0x7f5b38199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:42188/0 (socket says 172.21.15.43:42188) 2024-04-25T10:39:38.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.958+0000 7f5b3d900640 1 -- 172.21.15.43:0/1336892905 learned_addr learned my addr 172.21.15.43:0/1336892905 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:38.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.958+0000 7f5b3d0ff640 1 --2- 172.21.15.43:0/1336892905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b38106980 0x7f5b38199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:38.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.958+0000 7f5b3d900640 1 -- 172.21.15.43:0/1336892905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b38106980 msgr2=0x7f5b38199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:38.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.958+0000 7f5b3d900640 1 --2- 172.21.15.43:0/1336892905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b38106980 0x7f5b38199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.958+0000 7f5b3d900640 1 -- 172.21.15.43:0/1336892905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b38107440 msgr2=0x7f5b3819de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:38.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.958+0000 7f5b3d900640 1 --2- 172.21.15.43:0/1336892905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b38107440 0x7f5b3819de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.958+0000 7f5b3d900640 1 -- 172.21.15.43:0/1336892905 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3819e4b0 con 0x7f5b38102440 2024-04-25T10:39:38.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.958+0000 7f5b3d900640 1 --2- 172.21.15.43:0/1336892905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b38102440 0x7f5b38199650 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5b34005940 tx=0x7f5b34004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:38.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.962+0000 7f5b2effd640 1 -- 172.21.15.43:0/1336892905 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3403f030 con 0x7f5b38102440 2024-04-25T10:39:38.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.962+0000 7f5b2effd640 1 -- 172.21.15.43:0/1336892905 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b34003aa0 con 0x7f5b38102440 2024-04-25T10:39:38.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.962+0000 7f5b3fb64640 1 -- 172.21.15.43:0/1336892905 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3819e740 con 0x7f5b38102440 2024-04-25T10:39:38.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.962+0000 7f5b2effd640 1 -- 172.21.15.43:0/1336892905 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34035070 con 0x7f5b38102440 2024-04-25T10:39:38.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.962+0000 7f5b3fb64640 1 -- 172.21.15.43:0/1336892905 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3819ec80 con 0x7f5b38102440 2024-04-25T10:39:38.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.965+0000 7f5b2effd640 1 -- 172.21.15.43:0/1336892905 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f5b34042030 con 0x7f5b38102440 2024-04-25T10:39:38.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.965+0000 7f5b46a15740 1 Processor -- start 2024-04-25T10:39:38.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.966+0000 7f5b46a15740 1 -- start start 2024-04-25T10:39:38.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.966+0000 7f5b2effd640 1 --2- 172.21.15.43:0/1336892905 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5b240afce0 0x7f5b240b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:38.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.966+0000 7f5b46a15740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 0x563c89637a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.966+0000 7f5b46a15740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c8926e4d0 0x563c89637fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.966+0000 7f5b46a15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c892766b0 0x563c896448f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.966+0000 7f5b46a15740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563c8959dd80 con 0x563c89297390 2024-04-25T10:39:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.966+0000 7f5b46a15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563c895578b0 con 0x563c892766b0 2024-04-25T10:39:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.966+0000 7f5b46a15740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563c89599d40 con 0x563c8926e4d0 2024-04-25T10:39:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.966+0000 7f5b3e101640 1 --2- 172.21.15.43:0/1336892905 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5b240afce0 0x7f5b240b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.966+0000 7f5b3d900640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 0x563c89637a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.966+0000 7f5b2effd640 1 -- 172.21.15.43:0/1336892905 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f5b340ff6e0 con 0x7f5b38102440 2024-04-25T10:39:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.966+0000 7f5b3d900640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 0x563c89637a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39900/0 (socket says 172.21.15.43:39900) 2024-04-25T10:39:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.966+0000 7f5b3d900640 1 -- 172.21.15.43:0/3937118743 learned_addr learned my addr 172.21.15.43:0/3937118743 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.967+0000 7f5b3e101640 1 --2- 172.21.15.43:0/3937118743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c892766b0 0x563c896448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.967+0000 7f5b3d0ff640 1 --2- 172.21.15.43:0/3937118743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c8926e4d0 0x563c89637fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.967+0000 7f5b3d900640 1 -- 172.21.15.43:0/3937118743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c8926e4d0 msgr2=0x563c89637fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:38.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.967+0000 7f5b3d900640 1 --2- 172.21.15.43:0/3937118743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c8926e4d0 0x563c89637fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.967+0000 7f5b3d900640 1 -- 172.21.15.43:0/3937118743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c892766b0 msgr2=0x563c896448f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:38.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.967+0000 7f5b3d900640 1 --2- 172.21.15.43:0/3937118743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c892766b0 0x563c896448f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.967+0000 7f5b3d900640 1 -- 172.21.15.43:0/3937118743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c89528a40 con 0x563c89297390 2024-04-25T10:39:38.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.967+0000 7f5b3e101640 1 --2- 172.21.15.43:0/1336892905 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5b240afce0 0x7f5b240b21f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009fd0 tx=0x7f5b30009d80 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:38.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.967+0000 7f5b3d900640 1 --2- 172.21.15.43:0/3937118743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 0x563c89637a60 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f5b34032af0 tx=0x7f5b34036f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a20965eba607e1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:38.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.967+0000 7f5b23fff640 1 -- 172.21.15.43:0/3937118743 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34049070 con 0x563c89297390 2024-04-25T10:39:38.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.967+0000 7f5b23fff640 1 -- 172.21.15.43:0/3937118743 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b340c73e0 con 0x563c89297390 2024-04-25T10:39:38.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.967+0000 7f5b23fff640 1 -- 172.21.15.43:0/3937118743 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b340d0680 con 0x563c89297390 2024-04-25T10:39:38.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.968+0000 7f5b46a15740 1 -- 172.21.15.43:0/3937118743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 msgr2=0x563c89637a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:38.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.968+0000 7f5b46a15740 1 --2- 172.21.15.43:0/3937118743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 0x563c89637a60 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f5b34032af0 tx=0x7f5b34036f20 comp rx=0 tx=0).stop 2024-04-25T10:39:38.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.968+0000 7f5b46a15740 1 -- 172.21.15.43:0/3937118743 shutdown_connections 2024-04-25T10:39:38.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.968+0000 7f5b46a15740 1 --2- 172.21.15.43:0/3937118743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c892766b0 0x563c896448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.968+0000 7f5b46a15740 1 --2- 172.21.15.43:0/3937118743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c8926e4d0 0x563c89637fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.968+0000 7f5b46a15740 1 --2- 172.21.15.43:0/3937118743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 0x563c89637a60 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.968+0000 7f5b46a15740 1 -- 172.21.15.43:0/3937118743 >> 172.21.15.43:0/3937118743 conn(0x563c8928b4e0 msgr2=0x563c89634210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:38.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.968+0000 7f5b46a15740 1 -- 172.21.15.43:0/3937118743 shutdown_connections 2024-04-25T10:39:38.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.968+0000 7f5b46a15740 1 -- 172.21.15.43:0/3937118743 wait complete. 2024-04-25T10:39:38.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.969+0000 7f5b46a15740 1 -- start start 2024-04-25T10:39:38.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.969+0000 7f5b46a15740 1 Processor -- start 2024-04-25T10:39:38.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.969+0000 7f5b46a15740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c8926e4d0 0x563c89642620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.969+0000 7f5b46a15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c892766b0 0x563c89642b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.969+0000 7f5b46a15740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 0x563c896481d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.969+0000 7f5b46a15740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563c89644e30 con 0x563c89297390 2024-04-25T10:39:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.969+0000 7f5b46a15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563c89524d10 con 0x563c892766b0 2024-04-25T10:39:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.969+0000 7f5b46a15740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563c89644fb0 con 0x563c8926e4d0 2024-04-25T10:39:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.969+0000 7f5b46a15740 10 client.0 _ll_register_callbacks cb 0x563c89551210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.969+0000 7f5b3e101640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 0x563c896481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.969+0000 7f5b3e101640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 0x563c896481d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39914/0 (socket says 172.21.15.43:39914) 2024-04-25T10:39:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.969+0000 7f5b3e101640 1 -- 172.21.15.43:0/504517072 learned_addr learned my addr 172.21.15.43:0/504517072 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.970+0000 7f5b3d0ff640 1 --2- 172.21.15.43:0/504517072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c892766b0 0x563c89642b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.970+0000 7f5b3d900640 1 --2- 172.21.15.43:0/504517072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c8926e4d0 0x563c89642620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.970+0000 7f5b3e101640 1 -- 172.21.15.43:0/504517072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c8926e4d0 msgr2=0x563c89642620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.970+0000 7f5b3e101640 1 --2- 172.21.15.43:0/504517072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c8926e4d0 0x563c89642620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.970+0000 7f5b3e101640 1 -- 172.21.15.43:0/504517072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c892766b0 msgr2=0x563c89642b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.970+0000 7f5b3e101640 1 --2- 172.21.15.43:0/504517072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c892766b0 0x563c89642b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.970+0000 7f5b3e101640 1 -- 172.21.15.43:0/504517072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c893797d0 con 0x563c89297390 2024-04-25T10:39:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.970+0000 7f5b3d0ff640 1 --2- 172.21.15.43:0/504517072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c892766b0 0x563c89642b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:39:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.970+0000 7f5b3e101640 1 --2- 172.21.15.43:0/504517072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 0x563c896481d0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009c10 tx=0x7f5b3001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78dddc9a78069862 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.970+0000 7f5b217fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.970+0000 7f5b217fa640 1 -- 172.21.15.43:0/504517072 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30013330 con 0x563c89297390 2024-04-25T10:39:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.970+0000 7f5b217fa640 1 -- 172.21.15.43:0/504517072 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f5b30003e00 con 0x563c89297390 2024-04-25T10:39:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.970+0000 7f5b46a15740 1 -- 172.21.15.43:0/504517072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b340c73e0 con 0x563c89297390 2024-04-25T10:39:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.970+0000 7f5b217fa640 1 -- 172.21.15.43:0/504517072 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5b300045b0 con 0x563c89297390 2024-04-25T10:39:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.970+0000 7f5b46a15740 10 client.17484 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.970+0000 7f5b46a15740 1 -- 172.21.15.43:0/504517072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563c892af3f0 con 0x563c89297390 2024-04-25T10:39:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.970+0000 7f5b217fa640 1 -- 172.21.15.43:0/504517072 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b340c73e0 con 0x563c89297390 2024-04-25T10:39:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.971+0000 7f5b217fa640 1 -- 172.21.15.43:0/504517072 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5b30020050 con 0x563c89297390 2024-04-25T10:39:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.971+0000 7f5b46a15740 10 client.17484 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:38.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.971+0000 7f5b46a15740 10 client.17484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:38.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.971+0000 7f5b46a15740 20 client.17484 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:38.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.971+0000 7f5b46a15740 1 --2- 172.21.15.43:0/504517072 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563c89255f60 0x563c89656700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:38.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.971+0000 7f5b46a15740 4 client.17484 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:39:38.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.971+0000 7f5b46a15740 1 -- 172.21.15.43:0/504517072 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c895ab990 con 0x563c89255f60 2024-04-25T10:39:38.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.971+0000 7f5b3d900640 1 --2- 172.21.15.43:0/504517072 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563c89255f60 0x563c89656700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:38.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.972+0000 7f5b217fa640 10 client.17484 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:38.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.972+0000 7f5b3d900640 1 --2- 172.21.15.43:0/504517072 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563c89255f60 0x563c89656700 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef88c2fa3952121a server_cookie=26129fdc093073f2 in_seq=0 out_seq=0 2024-04-25T10:39:38.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.972+0000 7f5b217fa640 1 -- 172.21.15.43:0/504517072 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563c895ab990 con 0x563c89255f60 2024-04-25T10:39:38.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.972+0000 7f5b217fa640 10 client.17484 handle_command_reply: tid=0 2024-04-25T10:39:38.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.972+0000 7f5b46a15740 1 client.17484 shutdown 2024-04-25T10:39:38.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.972+0000 7f5b46a15740 1 -- 172.21.15.43:0/504517072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 msgr2=0x563c896481d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:38.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.972+0000 7f5b46a15740 1 --2- 172.21.15.43:0/504517072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 0x563c896481d0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009c10 tx=0x7f5b3001d000 comp rx=0 tx=0).stop 2024-04-25T10:39:38.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.973+0000 7f5b46a15740 1 -- 172.21.15.43:0/504517072 shutdown_connections 2024-04-25T10:39:38.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.973+0000 7f5b46a15740 1 --2- 172.21.15.43:0/504517072 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563c89255f60 0x563c89656700 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.973+0000 7f5b46a15740 1 --2- 172.21.15.43:0/504517072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c892766b0 0x563c89642b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.973+0000 7f5b217fa640 0 client.17484 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.973+0000 7f5b46a15740 1 --2- 172.21.15.43:0/504517072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 0x563c896481d0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.973+0000 7f5b46a15740 1 --2- 172.21.15.43:0/504517072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c8926e4d0 0x563c89642620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.973+0000 7f5b46a15740 1 -- 172.21.15.43:0/504517072 >> 172.21.15.43:0/504517072 conn(0x563c8928b4e0 msgr2=0x563c89639780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.973+0000 7f5b46a15740 1 -- 172.21.15.43:0/504517072 shutdown_connections 2024-04-25T10:39:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.973+0000 7f5b46a15740 1 -- 172.21.15.43:0/504517072 wait complete. 2024-04-25T10:39:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.973+0000 7f5b46a15740 20 client.17484 trim_cache size 0 max 16384 2024-04-25T10:39:38.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.991+0000 7f5b46a15740 1 Processor -- start 2024-04-25T10:39:38.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.991+0000 7f5b46a15740 1 -- start start 2024-04-25T10:39:38.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.991+0000 7f5b46a15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c8926e4d0 0x563c89637e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:38.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.991+0000 7f5b46a15740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c892766b0 0x563c896412c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:38.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.991+0000 7f5b46a15740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 0x563c8965bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:38.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.991+0000 7f5b46a15740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b381b0080 con 0x563c89297390 2024-04-25T10:39:38.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.991+0000 7f5b46a15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5b381ad950 con 0x563c8926e4d0 2024-04-25T10:39:38.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.991+0000 7f5b46a15740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5b381ab7c0 con 0x563c892766b0 2024-04-25T10:39:38.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.991+0000 7f5b3e101640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 0x563c8965bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:38.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.991+0000 7f5b3e101640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 0x563c8965bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39930/0 (socket says 172.21.15.43:39930) 2024-04-25T10:39:38.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.991+0000 7f5b3e101640 1 -- 172.21.15.43:0/4168280539 learned_addr learned my addr 172.21.15.43:0/4168280539 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:38.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.992+0000 7f5b3d0ff640 1 --2- 172.21.15.43:0/4168280539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c892766b0 0x563c896412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:38.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.992+0000 7f5b3e101640 1 -- 172.21.15.43:0/4168280539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c892766b0 msgr2=0x563c896412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:38.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.992+0000 7f5b3d900640 1 --2- 172.21.15.43:0/4168280539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c8926e4d0 0x563c89637e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:38.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.992+0000 7f5b3e101640 1 --2- 172.21.15.43:0/4168280539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c892766b0 0x563c896412c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.992+0000 7f5b3e101640 1 -- 172.21.15.43:0/4168280539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c8926e4d0 msgr2=0x563c89637e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:38.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.992+0000 7f5b3e101640 1 --2- 172.21.15.43:0/4168280539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c8926e4d0 0x563c89637e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.992+0000 7f5b3e101640 1 -- 172.21.15.43:0/4168280539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c8926f0f0 con 0x563c89297390 2024-04-25T10:39:38.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.992+0000 7f5b3d0ff640 1 --2- 172.21.15.43:0/4168280539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c892766b0 0x563c896412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:39:38.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.992+0000 7f5b3e101640 1 --2- 172.21.15.43:0/4168280539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 0x563c8965bec0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009710 tx=0x7f5b30004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd570de0e7478928 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:38.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.992+0000 7f5b237fe640 1 -- 172.21.15.43:0/4168280539 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30038070 con 0x563c89297390 2024-04-25T10:39:38.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.992+0000 7f5b237fe640 1 -- 172.21.15.43:0/4168280539 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b30013330 con 0x563c89297390 2024-04-25T10:39:38.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.992+0000 7f5b237fe640 1 -- 172.21.15.43:0/4168280539 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30004990 con 0x563c89297390 2024-04-25T10:39:38.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.993+0000 7f5b46a15740 1 -- 172.21.15.43:0/4168280539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 msgr2=0x563c8965bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:38.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.993+0000 7f5b46a15740 1 --2- 172.21.15.43:0/4168280539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 0x563c8965bec0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009710 tx=0x7f5b30004400 comp rx=0 tx=0).stop 2024-04-25T10:39:38.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.993+0000 7f5b46a15740 1 -- 172.21.15.43:0/4168280539 shutdown_connections 2024-04-25T10:39:38.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.993+0000 7f5b46a15740 1 --2- 172.21.15.43:0/4168280539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c89297390 0x563c8965bec0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.993+0000 7f5b46a15740 1 --2- 172.21.15.43:0/4168280539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c892766b0 0x563c896412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.993+0000 7f5b46a15740 1 --2- 172.21.15.43:0/4168280539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c8926e4d0 0x563c89637e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:38.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.993+0000 7f5b46a15740 1 -- 172.21.15.43:0/4168280539 >> 172.21.15.43:0/4168280539 conn(0x563c8928b4e0 msgr2=0x563c89640bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:38.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.993+0000 7f5b46a15740 1 -- 172.21.15.43:0/4168280539 shutdown_connections 2024-04-25T10:39:38.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.993+0000 7f5b46a15740 1 -- 172.21.15.43:0/4168280539 wait complete. 2024-04-25T10:39:38.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.993+0000 7f5b46a15740 1 -- start start 2024-04-25T10:39:38.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.993+0000 7f5b46a15740 1 Processor -- start 2024-04-25T10:39:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.994+0000 7f5b46a15740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c8926e4d0 0x563c896497a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.994+0000 7f5b46a15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c892766b0 0x563c89649ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.994+0000 7f5b46a15740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c89297390 0x563c896b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.994+0000 7f5b46a15740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b381abe10 con 0x563c8926e4d0 2024-04-25T10:39:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.994+0000 7f5b46a15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5b381ad280 con 0x563c892766b0 2024-04-25T10:39:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.994+0000 7f5b46a15740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5b381ac4e0 con 0x563c89297390 2024-04-25T10:39:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.994+0000 7f5b46a15740 10 client.0 _ll_register_callbacks cb 0x7f5b3000fb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:39.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.994+0000 7f5b3d900640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c8926e4d0 0x563c896497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.994+0000 7f5b3d900640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c8926e4d0 0x563c896497a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39944/0 (socket says 172.21.15.43:39944) 2024-04-25T10:39:39.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.994+0000 7f5b3d900640 1 -- 172.21.15.43:0/3442669483 learned_addr learned my addr 172.21.15.43:0/3442669483 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:39.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.994+0000 7f5b3e101640 1 --2- 172.21.15.43:0/3442669483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c89297390 0x563c896b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.994+0000 7f5b3d0ff640 1 --2- 172.21.15.43:0/3442669483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c892766b0 0x563c89649ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.994+0000 7f5b3d900640 1 -- 172.21.15.43:0/3442669483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c89297390 msgr2=0x563c896b6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.994+0000 7f5b3d900640 1 --2- 172.21.15.43:0/3442669483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c89297390 0x563c896b6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.994+0000 7f5b3d900640 1 -- 172.21.15.43:0/3442669483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c892766b0 msgr2=0x563c89649ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.994+0000 7f5b3d900640 1 --2- 172.21.15.43:0/3442669483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c892766b0 0x563c89649ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.994+0000 7f5b3d900640 1 -- 172.21.15.43:0/3442669483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c8955e4b0 con 0x563c8926e4d0 2024-04-25T10:39:39.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.994+0000 7f5b3e101640 1 --2- 172.21.15.43:0/3442669483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c89297390 0x563c896b6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:39.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.994+0000 7f5b3d900640 1 --2- 172.21.15.43:0/3442669483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c8926e4d0 0x563c896497a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f5b340c75d0 tx=0x7f5b34031a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5155a915d84a7c63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.994+0000 7f5b217fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.995+0000 7f5b217fa640 1 -- 172.21.15.43:0/3442669483 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34049040 con 0x563c8926e4d0 2024-04-25T10:39:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.995+0000 7f5b217fa640 1 -- 172.21.15.43:0/3442669483 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f5b340c3b60 con 0x563c8926e4d0 2024-04-25T10:39:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.995+0000 7f5b46a15740 1 -- 172.21.15.43:0/3442669483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b30013330 con 0x563c8926e4d0 2024-04-25T10:39:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.995+0000 7f5b217fa640 1 -- 172.21.15.43:0/3442669483 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5b340e4070 con 0x563c8926e4d0 2024-04-25T10:39:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.995+0000 7f5b46a15740 10 client.17490 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.995+0000 7f5b46a15740 1 -- 172.21.15.43:0/3442669483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563c896be0d0 con 0x563c8926e4d0 2024-04-25T10:39:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.995+0000 7f5b217fa640 1 -- 172.21.15.43:0/3442669483 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b30013330 con 0x563c8926e4d0 2024-04-25T10:39:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.995+0000 7f5b217fa640 1 -- 172.21.15.43:0/3442669483 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5b3403bcc0 con 0x563c8926e4d0 2024-04-25T10:39:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.995+0000 7f5b46a15740 10 client.17490 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.995+0000 7f5b46a15740 10 client.17490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.996+0000 7f5b46a15740 20 client.17490 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.996+0000 7f5b46a15740 1 --2- 172.21.15.43:0/3442669483 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563c8922b3a0 0x563c896c06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.996+0000 7f5b46a15740 4 client.17490 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:39:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.996+0000 7f5b46a15740 1 -- 172.21.15.43:0/3442669483 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563c895a5800 con 0x563c8922b3a0 2024-04-25T10:39:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.996+0000 7f5b3d0ff640 1 --2- 172.21.15.43:0/3442669483 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563c8922b3a0 0x563c896c06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.996+0000 7f5b3d0ff640 1 --2- 172.21.15.43:0/3442669483 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563c8922b3a0 0x563c896c06c0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0b9d46a2d6d6db3 server_cookie=cf198445ec1603b in_seq=0 out_seq=0 2024-04-25T10:39:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.996+0000 7f5b217fa640 10 client.17490 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.996+0000 7f5b217fa640 1 -- 172.21.15.43:0/3442669483 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563c895a5800 con 0x563c8922b3a0 2024-04-25T10:39:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.996+0000 7f5b217fa640 10 client.17490 handle_command_reply: tid=0 2024-04-25T10:39:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.996+0000 7f5b46a15740 1 client.17490 shutdown 2024-04-25T10:39:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b46a15740 1 -- 172.21.15.43:0/3442669483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c8926e4d0 msgr2=0x563c896497a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b46a15740 1 --2- 172.21.15.43:0/3442669483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c8926e4d0 0x563c896497a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f5b340c75d0 tx=0x7f5b34031a00 comp rx=0 tx=0).stop 2024-04-25T10:39:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b46a15740 1 -- 172.21.15.43:0/3442669483 shutdown_connections 2024-04-25T10:39:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b46a15740 1 --2- 172.21.15.43:0/3442669483 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563c8922b3a0 0x563c896c06c0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b46a15740 1 --2- 172.21.15.43:0/3442669483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c892766b0 0x563c89649ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b46a15740 1 --2- 172.21.15.43:0/3442669483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c89297390 0x563c896b6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b46a15740 1 --2- 172.21.15.43:0/3442669483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c8926e4d0 0x563c896497a0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b217fa640 0 client.17490 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b46a15740 1 -- 172.21.15.43:0/3442669483 >> 172.21.15.43:0/3442669483 conn(0x563c8928b4e0 msgr2=0x563c896439e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b46a15740 1 -- 172.21.15.43:0/3442669483 shutdown_connections 2024-04-25T10:39:39.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b46a15740 1 -- 172.21.15.43:0/3442669483 wait complete. 2024-04-25T10:39:39.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b46a15740 20 client.17490 trim_cache size 0 max 16384 2024-04-25T10:39:39.005 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:39:39.005 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:39:39.005 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:39:39.005 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:39:39.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b3fb64640 1 -- 172.21.15.43:0/1336892905 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5b240afce0 msgr2=0x7f5b240b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b3fb64640 1 --2- 172.21.15.43:0/1336892905 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5b240afce0 0x7f5b240b21f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009fd0 tx=0x7f5b30009d80 comp rx=0 tx=0).stop 2024-04-25T10:39:39.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b3fb64640 1 -- 172.21.15.43:0/1336892905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b38102440 msgr2=0x7f5b38199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b3fb64640 1 --2- 172.21.15.43:0/1336892905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b38102440 0x7f5b38199650 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5b34005940 tx=0x7f5b34004270 comp rx=0 tx=0).stop 2024-04-25T10:39:39.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b3fb64640 1 -- 172.21.15.43:0/1336892905 shutdown_connections 2024-04-25T10:39:39.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b3fb64640 1 --2- 172.21.15.43:0/1336892905 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5b240afce0 0x7f5b240b21f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b3fb64640 1 --2- 172.21.15.43:0/1336892905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b38107440 0x7f5b3819de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b3fb64640 1 --2- 172.21.15.43:0/1336892905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b38106980 0x7f5b38199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b3fb64640 1 --2- 172.21.15.43:0/1336892905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b38102440 0x7f5b38199650 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.997+0000 7f5b3fb64640 1 -- 172.21.15.43:0/1336892905 >> 172.21.15.43:0/1336892905 conn(0x7f5b38077880 msgr2=0x7f5b38100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.998+0000 7f5b3fb64640 1 -- 172.21.15.43:0/1336892905 shutdown_connections 2024-04-25T10:39:39.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:38.998+0000 7f5b3fb64640 1 -- 172.21.15.43:0/1336892905 wait complete. 2024-04-25T10:39:39.010 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:39:39.011 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:39:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.120+0000 7f4691f8f640 1 -- 172.21.15.43:0/1277268387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468c109fc0 msgr2=0x7f468c102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.120+0000 7f4691f8f640 1 --2- 172.21.15.43:0/1277268387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468c109fc0 0x7f468c102130 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f4680009900 tx=0x7f468002a410 comp rx=0 tx=0).stop 2024-04-25T10:39:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.120+0000 7f4691f8f640 1 -- 172.21.15.43:0/1277268387 shutdown_connections 2024-04-25T10:39:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.120+0000 7f4691f8f640 1 --2- 172.21.15.43:0/1277268387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f468c1027c0 0x7f468c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.120+0000 7f4691f8f640 1 --2- 172.21.15.43:0/1277268387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468c109fc0 0x7f468c102130 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.120+0000 7f4691f8f640 1 --2- 172.21.15.43:0/1277268387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f468c109b20 0x7f468c101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.120+0000 7f4691f8f640 1 -- 172.21.15.43:0/1277268387 >> 172.21.15.43:0/1277268387 conn(0x7f468c077890 msgr2=0x7f468c077cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.121+0000 7f4691f8f640 1 -- 172.21.15.43:0/1277268387 shutdown_connections 2024-04-25T10:39:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.121+0000 7f4691f8f640 1 -- 172.21.15.43:0/1277268387 wait complete. 2024-04-25T10:39:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.121+0000 7f4691f8f640 1 Processor -- start 2024-04-25T10:39:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.121+0000 7f4691f8f640 1 -- start start 2024-04-25T10:39:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.122+0000 7f4691f8f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f468c109b20 0x7f468c199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.122+0000 7f4691f8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468c19a450 0x7f468c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.122+0000 7f4691f8f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f468c19ec70 0x7f468c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.122+0000 7f4691f8f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f468c113d10 con 0x7f468c19a450 2024-04-25T10:39:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.122+0000 7f4691f8f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f468c113b90 con 0x7f468c19ec70 2024-04-25T10:39:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.122+0000 7f4691f8f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f468c113e90 con 0x7f468c109b20 2024-04-25T10:39:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.122+0000 7f468affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468c19a450 0x7f468c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.122+0000 7f468affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468c19a450 0x7f468c19e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39948/0 (socket says 172.21.15.43:39948) 2024-04-25T10:39:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.122+0000 7f468affd640 1 -- 172.21.15.43:0/667091700 learned_addr learned my addr 172.21.15.43:0/667091700 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.122+0000 7f468b7fe640 1 --2- 172.21.15.43:0/667091700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f468c109b20 0x7f468c199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.122+0000 7f468bfff640 1 --2- 172.21.15.43:0/667091700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f468c19ec70 0x7f468c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.122+0000 7f468affd640 1 -- 172.21.15.43:0/667091700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f468c109b20 msgr2=0x7f468c199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.122+0000 7f468affd640 1 --2- 172.21.15.43:0/667091700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f468c109b20 0x7f468c199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.122+0000 7f468affd640 1 -- 172.21.15.43:0/667091700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f468c19ec70 msgr2=0x7f468c19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.122+0000 7f468affd640 1 --2- 172.21.15.43:0/667091700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f468c19ec70 0x7f468c19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.122+0000 7f468affd640 1 -- 172.21.15.43:0/667091700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468c1a69e0 con 0x7f468c19a450 2024-04-25T10:39:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.122+0000 7f468affd640 1 --2- 172.21.15.43:0/667091700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468c19a450 0x7f468c19e730 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f4680006010 tx=0x7f4680004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.122+0000 7f4688ff9640 1 -- 172.21.15.43:0/667091700 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468003f030 con 0x7f468c19a450 2024-04-25T10:39:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.123+0000 7f4688ff9640 1 -- 172.21.15.43:0/667091700 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4680003aa0 con 0x7f468c19a450 2024-04-25T10:39:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.123+0000 7f4688ff9640 1 -- 172.21.15.43:0/667091700 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680035070 con 0x7f468c19a450 2024-04-25T10:39:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.123+0000 7f4691f8f640 1 -- 172.21.15.43:0/667091700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f468c1a6bb0 con 0x7f468c19a450 2024-04-25T10:39:39.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.123+0000 7f4691f8f640 1 -- 172.21.15.43:0/667091700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f468c1a6fb0 con 0x7f468c19a450 2024-04-25T10:39:39.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.124+0000 7f4691f8f640 1 -- 172.21.15.43:0/667091700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f468c06a190 con 0x7f468c19a450 2024-04-25T10:39:39.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.125+0000 7f4688ff9640 1 -- 172.21.15.43:0/667091700 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f4680042030 con 0x7f468c19a450 2024-04-25T10:39:39.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.126+0000 7f4688ff9640 1 --2- 172.21.15.43:0/667091700 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f46740afc60 0x7f46740b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.126+0000 7f468b7fe640 1 --2- 172.21.15.43:0/667091700 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f46740afc60 0x7f46740b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.126+0000 7f4688ff9640 1 -- 172.21.15.43:0/667091700 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f46800ff800 con 0x7f468c19a450 2024-04-25T10:39:39.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.126+0000 7f468b7fe640 1 --2- 172.21.15.43:0/667091700 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f46740afc60 0x7f46740b2170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f467c005fd0 tx=0x7f467c00b390 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.129+0000 7f4688ff9640 1 -- 172.21.15.43:0/667091700 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46800c7a50 con 0x7f468c19a450 2024-04-25T10:39:39.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.357+0000 7f4691f8f640 1 -- 172.21.15.43:0/667091700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f468c19b260 con 0x7f468c19a450 2024-04-25T10:39:39.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.358+0000 7f4688ff9640 1 -- 172.21.15.43:0/667091700 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7f46800cbd50 con 0x7f468c19a450 2024-04-25T10:39:39.360 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:39:39.361 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:39:39.361 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:39:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.360+0000 7f4691f8f640 1 -- 172.21.15.43:0/667091700 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f46740afc60 msgr2=0x7f46740b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.360+0000 7f4691f8f640 1 --2- 172.21.15.43:0/667091700 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f46740afc60 0x7f46740b2170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f467c005fd0 tx=0x7f467c00b390 comp rx=0 tx=0).stop 2024-04-25T10:39:39.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.360+0000 7f4691f8f640 1 -- 172.21.15.43:0/667091700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468c19a450 msgr2=0x7f468c19e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.360+0000 7f4691f8f640 1 --2- 172.21.15.43:0/667091700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468c19a450 0x7f468c19e730 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f4680006010 tx=0x7f4680004060 comp rx=0 tx=0).stop 2024-04-25T10:39:39.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.360+0000 7f4691f8f640 1 -- 172.21.15.43:0/667091700 shutdown_connections 2024-04-25T10:39:39.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.360+0000 7f4691f8f640 1 --2- 172.21.15.43:0/667091700 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f46740afc60 0x7f46740b2170 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.360+0000 7f4691f8f640 1 --2- 172.21.15.43:0/667091700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f468c19ec70 0x7f468c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.360+0000 7f4691f8f640 1 --2- 172.21.15.43:0/667091700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468c19a450 0x7f468c19e730 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.360+0000 7f4691f8f640 1 --2- 172.21.15.43:0/667091700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f468c109b20 0x7f468c199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.360+0000 7f4691f8f640 1 -- 172.21.15.43:0/667091700 >> 172.21.15.43:0/667091700 conn(0x7f468c077890 msgr2=0x7f468c10fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.360+0000 7f4691f8f640 1 -- 172.21.15.43:0/667091700 shutdown_connections 2024-04-25T10:39:39.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.360+0000 7f4691f8f640 1 -- 172.21.15.43:0/667091700 wait complete. 2024-04-25T10:39:39.374 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:39:39.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.487+0000 7f0be564a640 1 -- 172.21.15.43:0/1783024558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be0102af0 msgr2=0x7f0be0102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.487+0000 7f0be564a640 1 --2- 172.21.15.43:0/1783024558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be0102af0 0x7f0be0102f00 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009980 tx=0x7f0bd402c620 comp rx=0 tx=0).stop 2024-04-25T10:39:39.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.487+0000 7f0be564a640 1 -- 172.21.15.43:0/1783024558 shutdown_connections 2024-04-25T10:39:39.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.487+0000 7f0be564a640 1 --2- 172.21.15.43:0/1783024558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be010d380 0x7f0be010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.487+0000 7f0be564a640 1 --2- 172.21.15.43:0/1783024558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be0102af0 0x7f0be0102f00 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.487+0000 7f0be564a640 1 --2- 172.21.15.43:0/1783024558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0be0108a20 0x7f0be01025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.487+0000 7f0be564a640 1 -- 172.21.15.43:0/1783024558 >> 172.21.15.43:0/1783024558 conn(0x7f0be0078100 msgr2=0x7f0be0078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.487+0000 7f0be564a640 1 -- 172.21.15.43:0/1783024558 shutdown_connections 2024-04-25T10:39:39.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.487+0000 7f0be564a640 1 -- 172.21.15.43:0/1783024558 wait complete. 2024-04-25T10:39:39.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.488+0000 7f0be564a640 1 Processor -- start 2024-04-25T10:39:39.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.488+0000 7f0be564a640 1 -- start start 2024-04-25T10:39:39.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.488+0000 7f0be564a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be0102af0 0x7f0be0198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.488+0000 7f0be564a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0be0108a20 0x7f0be0198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.488+0000 7f0be564a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be010d380 0x7f0be019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.488+0000 7f0be564a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0be0111ae0 con 0x7f0be0102af0 2024-04-25T10:39:39.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.488+0000 7f0be564a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0be0111960 con 0x7f0be0108a20 2024-04-25T10:39:39.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.488+0000 7f0be564a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0be0111c60 con 0x7f0be010d380 2024-04-25T10:39:39.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.488+0000 7f0bdeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be0102af0 0x7f0be0198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.488+0000 7f0bdeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be0102af0 0x7f0be0198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39970/0 (socket says 172.21.15.43:39970) 2024-04-25T10:39:39.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.488+0000 7f0bdeffd640 1 -- 172.21.15.43:0/1170320109 learned_addr learned my addr 172.21.15.43:0/1170320109 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:39.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.489+0000 7f0bde7fc640 1 --2- 172.21.15.43:0/1170320109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0be0108a20 0x7f0be0198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.489+0000 7f0bdf7fe640 1 --2- 172.21.15.43:0/1170320109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be010d380 0x7f0be019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.489+0000 7f0bdeffd640 1 -- 172.21.15.43:0/1170320109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be010d380 msgr2=0x7f0be019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.489+0000 7f0bdeffd640 1 --2- 172.21.15.43:0/1170320109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be010d380 0x7f0be019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.489+0000 7f0bdeffd640 1 -- 172.21.15.43:0/1170320109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0be0108a20 msgr2=0x7f0be0198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.489+0000 7f0bdeffd640 1 --2- 172.21.15.43:0/1170320109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0be0108a20 0x7f0be0198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.489+0000 7f0bdeffd640 1 -- 172.21.15.43:0/1170320109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be019d380 con 0x7f0be0102af0 2024-04-25T10:39:39.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.489+0000 7f0bdf7fe640 1 --2- 172.21.15.43:0/1170320109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be010d380 0x7f0be019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:39:39.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.489+0000 7f0bdeffd640 1 --2- 172.21.15.43:0/1170320109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be0102af0 0x7f0be0198520 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000dbf0 tx=0x7f0bd000b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.489+0000 7f0bc7fff640 1 -- 172.21.15.43:0/1170320109 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0017070 con 0x7f0be0102af0 2024-04-25T10:39:39.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.489+0000 7f0bc7fff640 1 -- 172.21.15.43:0/1170320109 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd0003e20 con 0x7f0be0102af0 2024-04-25T10:39:39.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.489+0000 7f0be564a640 1 -- 172.21.15.43:0/1170320109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be019d670 con 0x7f0be0102af0 2024-04-25T10:39:39.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.489+0000 7f0bc7fff640 1 -- 172.21.15.43:0/1170320109 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0010460 con 0x7f0be0102af0 2024-04-25T10:39:39.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.489+0000 7f0be564a640 1 -- 172.21.15.43:0/1170320109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be01a4ff0 con 0x7f0be0102af0 2024-04-25T10:39:39.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.492+0000 7f0bc7fff640 1 -- 172.21.15.43:0/1170320109 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f0bd00040b0 con 0x7f0be0102af0 2024-04-25T10:39:39.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.492+0000 7f0bec5dd740 1 Processor -- start 2024-04-25T10:39:39.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.492+0000 7f0bec5dd740 1 -- start start 2024-04-25T10:39:39.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.492+0000 7f0bec5dd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914c1390 0x55d891862810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.492+0000 7f0bec5dd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914984d0 0x55d891862d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.492+0000 7f0bec5dd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89186f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.492+0000 7f0bec5dd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d8917c7d80 con 0x55d8914a06b0 2024-04-25T10:39:39.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.492+0000 7f0bec5dd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d891781a20 con 0x55d8914c1390 2024-04-25T10:39:39.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.492+0000 7f0bec5dd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d8917c3d40 con 0x55d8914984d0 2024-04-25T10:39:39.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.492+0000 7f0bde7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89186f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.492+0000 7f0bde7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89186f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39980/0 (socket says 172.21.15.43:39980) 2024-04-25T10:39:39.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.492+0000 7f0bde7fc640 1 -- 172.21.15.43:0/3817341494 learned_addr learned my addr 172.21.15.43:0/3817341494 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:39.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.492+0000 7f0bdf7fe640 1 --2- 172.21.15.43:0/3817341494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914c1390 0x55d891862810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.492+0000 7f0bc7fff640 1 --2- 172.21.15.43:0/1170320109 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0bc80afc60 0x7f0bc80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.493+0000 7f0bdeffd640 1 --2- 172.21.15.43:0/3817341494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914984d0 0x55d891862d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.493+0000 7f0bdf7fe640 1 --2- 172.21.15.43:0/1170320109 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0bc80afc60 0x7f0bc80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.493+0000 7f0bde7fc640 1 -- 172.21.15.43:0/3817341494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914984d0 msgr2=0x55d891862d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.493+0000 7f0bde7fc640 1 --2- 172.21.15.43:0/3817341494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914984d0 0x55d891862d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.493+0000 7f0bde7fc640 1 -- 172.21.15.43:0/3817341494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914c1390 msgr2=0x55d891862810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.493+0000 7f0bde7fc640 1 --2- 172.21.15.43:0/3817341494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914c1390 0x55d891862810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.493+0000 7f0bde7fc640 1 -- 172.21.15.43:0/3817341494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d89178cc30 con 0x55d8914a06b0 2024-04-25T10:39:39.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.493+0000 7f0bc7fff640 1 -- 172.21.15.43:0/1170320109 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f0bd00def30 con 0x7f0be0102af0 2024-04-25T10:39:39.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.493+0000 7f0bde7fc640 1 --2- 172.21.15.43:0/3817341494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89186f6a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4000c00 tx=0x7f0bd4005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f797528e810098cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.493+0000 7f0bdf7fe640 1 --2- 172.21.15.43:0/1170320109 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0bc80afc60 0x7f0bc80b2170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8009fd0 tx=0x7f0bd8004730 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.493+0000 7f0bc57fa640 1 -- 172.21.15.43:0/3817341494 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4041070 con 0x55d8914a06b0 2024-04-25T10:39:39.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.493+0000 7f0bc57fa640 1 -- 172.21.15.43:0/3817341494 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd4004030 con 0x55d8914a06b0 2024-04-25T10:39:39.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.493+0000 7f0bc57fa640 1 -- 172.21.15.43:0/3817341494 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4037070 con 0x55d8914a06b0 2024-04-25T10:39:39.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.494+0000 7f0bec5dd740 1 -- 172.21.15.43:0/3817341494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 msgr2=0x55d89186f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.494+0000 7f0bec5dd740 1 --2- 172.21.15.43:0/3817341494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89186f6a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4000c00 tx=0x7f0bd4005f10 comp rx=0 tx=0).stop 2024-04-25T10:39:39.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.494+0000 7f0bec5dd740 1 -- 172.21.15.43:0/3817341494 shutdown_connections 2024-04-25T10:39:39.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.494+0000 7f0bec5dd740 1 --2- 172.21.15.43:0/3817341494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89186f6a0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.494+0000 7f0bec5dd740 1 --2- 172.21.15.43:0/3817341494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914984d0 0x55d891862d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.494+0000 7f0bec5dd740 1 --2- 172.21.15.43:0/3817341494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914c1390 0x55d891862810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.494+0000 7f0bec5dd740 1 -- 172.21.15.43:0/3817341494 >> 172.21.15.43:0/3817341494 conn(0x55d8914b54e0 msgr2=0x55d89185efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.494+0000 7f0bec5dd740 1 -- 172.21.15.43:0/3817341494 shutdown_connections 2024-04-25T10:39:39.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.494+0000 7f0bec5dd740 1 -- 172.21.15.43:0/3817341494 wait complete. 2024-04-25T10:39:39.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.494+0000 7f0bec5dd740 1 -- start start 2024-04-25T10:39:39.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.495+0000 7f0bec5dd740 1 Processor -- start 2024-04-25T10:39:39.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.495+0000 7f0bec5dd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914984d0 0x55d89186d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.495+0000 7f0bec5dd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89186d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.495+0000 7f0bec5dd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914c1390 0x55d8918791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.495+0000 7f0bec5dd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d89186fd30 con 0x55d8914a06b0 2024-04-25T10:39:39.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.495+0000 7f0bec5dd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d89174edc0 con 0x55d8914c1390 2024-04-25T10:39:39.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.495+0000 7f0bec5dd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d89186feb0 con 0x55d8914984d0 2024-04-25T10:39:39.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.495+0000 7f0bec5dd740 10 client.0 _ll_register_callbacks cb 0x55d89176a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:39.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.495+0000 7f0bde7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89186d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.495+0000 7f0bde7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89186d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39984/0 (socket says 172.21.15.43:39984) 2024-04-25T10:39:39.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.495+0000 7f0bde7fc640 1 -- 172.21.15.43:0/618157659 learned_addr learned my addr 172.21.15.43:0/618157659 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:39.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.495+0000 7f0bdf7fe640 1 --2- 172.21.15.43:0/618157659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914c1390 0x55d8918791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.495+0000 7f0bdeffd640 1 --2- 172.21.15.43:0/618157659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914984d0 0x55d89186d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.495+0000 7f0bde7fc640 1 -- 172.21.15.43:0/618157659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914984d0 msgr2=0x55d89186d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.495+0000 7f0bde7fc640 1 --2- 172.21.15.43:0/618157659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914984d0 0x55d89186d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.495+0000 7f0bde7fc640 1 -- 172.21.15.43:0/618157659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914c1390 msgr2=0x55d8918791b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.495+0000 7f0bde7fc640 1 --2- 172.21.15.43:0/618157659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914c1390 0x55d8918791b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.495+0000 7f0bde7fc640 1 -- 172.21.15.43:0/618157659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8914d93f0 con 0x55d8914a06b0 2024-04-25T10:39:39.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.495+0000 7f0bdf7fe640 1 --2- 172.21.15.43:0/618157659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914c1390 0x55d8918791b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:39.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.495+0000 7f0bdeffd640 1 --2- 172.21.15.43:0/618157659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914984d0 0x55d89186d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:39:39.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.496+0000 7f0bde7fc640 1 --2- 172.21.15.43:0/618157659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89186d7a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f0bd402cd70 tx=0x7f0bd4033750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2038e7f86f279a09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.496+0000 7f0bc2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:39.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.496+0000 7f0bc2ff5640 1 -- 172.21.15.43:0/618157659 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4037070 con 0x55d8914a06b0 2024-04-25T10:39:39.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.496+0000 7f0bc2ff5640 1 -- 172.21.15.43:0/618157659 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f0bd40055c0 con 0x55d8914a06b0 2024-04-25T10:39:39.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.496+0000 7f0bec5dd740 1 -- 172.21.15.43:0/618157659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bd4004030 con 0x55d8914a06b0 2024-04-25T10:39:39.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.496+0000 7f0bc2ff5640 1 -- 172.21.15.43:0/618157659 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0bd4057050 con 0x55d8914a06b0 2024-04-25T10:39:39.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.496+0000 7f0bec5dd740 10 client.17508 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:39.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.496+0000 7f0bec5dd740 1 -- 172.21.15.43:0/618157659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d8915a37d0 con 0x55d8914a06b0 2024-04-25T10:39:39.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.496+0000 7f0bc2ff5640 1 -- 172.21.15.43:0/618157659 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd4004030 con 0x55d8914a06b0 2024-04-25T10:39:39.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.496+0000 7f0bc2ff5640 1 -- 172.21.15.43:0/618157659 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0bd4004880 con 0x55d8914a06b0 2024-04-25T10:39:39.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.496+0000 7f0bec5dd740 10 client.17508 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:39.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.496+0000 7f0bec5dd740 10 client.17508 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:39:39.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.496+0000 7f0bec5dd740 20 client.17508 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:39.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.496+0000 7f0bec5dd740 1 --2- 172.21.15.43:0/618157659 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d89147ff60 0x55d89187f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.496+0000 7f0bec5dd740 4 client.17508 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:39:39.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.496+0000 7f0bec5dd740 1 -- 172.21.15.43:0/618157659 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8917d5990 con 0x55d89147ff60 2024-04-25T10:39:39.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.497+0000 7f0bdeffd640 1 --2- 172.21.15.43:0/618157659 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d89147ff60 0x55d89187f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.497+0000 7f0bdeffd640 1 --2- 172.21.15.43:0/618157659 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d89147ff60 0x55d89187f490 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc1d5adae5fe290c server_cookie=2e721af250c6948a in_seq=0 out_seq=0 2024-04-25T10:39:39.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.497+0000 7f0bc2ff5640 10 client.17508 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:39:39.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.498+0000 7f0bc2ff5640 1 -- 172.21.15.43:0/618157659 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d8917d5990 con 0x55d89147ff60 2024-04-25T10:39:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.498+0000 7f0bc2ff5640 10 client.17508 handle_command_reply: tid=0 2024-04-25T10:39:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.498+0000 7f0bec5dd740 1 client.17508 shutdown 2024-04-25T10:39:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.498+0000 7f0bec5dd740 1 -- 172.21.15.43:0/618157659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 msgr2=0x55d89186d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.498+0000 7f0bec5dd740 1 --2- 172.21.15.43:0/618157659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89186d7a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f0bd402cd70 tx=0x7f0bd4033750 comp rx=0 tx=0).stop 2024-04-25T10:39:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.498+0000 7f0bec5dd740 1 -- 172.21.15.43:0/618157659 shutdown_connections 2024-04-25T10:39:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.498+0000 7f0bec5dd740 1 --2- 172.21.15.43:0/618157659 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d89147ff60 0x55d89187f490 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.498+0000 7f0bec5dd740 1 --2- 172.21.15.43:0/618157659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914c1390 0x55d8918791b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.498+0000 7f0bec5dd740 1 --2- 172.21.15.43:0/618157659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89186d7a0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.498+0000 7f0bc2ff5640 0 client.17508 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:39:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.498+0000 7f0bec5dd740 1 --2- 172.21.15.43:0/618157659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914984d0 0x55d89186d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.498+0000 7f0bec5dd740 1 -- 172.21.15.43:0/618157659 >> 172.21.15.43:0/618157659 conn(0x55d8914b54e0 msgr2=0x55d891864530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.498+0000 7f0bec5dd740 1 -- 172.21.15.43:0/618157659 shutdown_connections 2024-04-25T10:39:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.498+0000 7f0bec5dd740 1 -- 172.21.15.43:0/618157659 wait complete. 2024-04-25T10:39:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.498+0000 7f0bec5dd740 20 client.17508 trim_cache size 0 max 16384 2024-04-25T10:39:39.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.512+0000 7f0bec5dd740 1 Processor -- start 2024-04-25T10:39:39.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.512+0000 7f0bec5dd740 1 -- start start 2024-04-25T10:39:39.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.512+0000 7f0bec5dd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914984d0 0x55d8918674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.512+0000 7f0bec5dd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89186ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.512+0000 7f0bec5dd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914c1390 0x55d89186c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.512+0000 7f0bec5dd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0be01afe60 con 0x55d8914a06b0 2024-04-25T10:39:39.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.512+0000 7f0bec5dd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0be01ad730 con 0x55d8914c1390 2024-04-25T10:39:39.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.512+0000 7f0bec5dd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0be01ab5a0 con 0x55d8914984d0 2024-04-25T10:39:39.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.512+0000 7f0bde7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89186ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.512+0000 7f0bde7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89186ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39990/0 (socket says 172.21.15.43:39990) 2024-04-25T10:39:39.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.512+0000 7f0bde7fc640 1 -- 172.21.15.43:0/599352341 learned_addr learned my addr 172.21.15.43:0/599352341 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:39.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.512+0000 7f0bdf7fe640 1 --2- 172.21.15.43:0/599352341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914c1390 0x55d89186c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.512+0000 7f0bdeffd640 1 --2- 172.21.15.43:0/599352341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914984d0 0x55d8918674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.512+0000 7f0bde7fc640 1 -- 172.21.15.43:0/599352341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914984d0 msgr2=0x55d8918674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.512+0000 7f0bde7fc640 1 --2- 172.21.15.43:0/599352341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914984d0 0x55d8918674f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.512+0000 7f0bde7fc640 1 -- 172.21.15.43:0/599352341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914c1390 msgr2=0x55d89186c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.512+0000 7f0bde7fc640 1 --2- 172.21.15.43:0/599352341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914c1390 0x55d89186c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.512+0000 7f0bde7fc640 1 -- 172.21.15.43:0/599352341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d891885460 con 0x55d8914a06b0 2024-04-25T10:39:39.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.513+0000 7f0bde7fc640 1 --2- 172.21.15.43:0/599352341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89186ee80 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4004cd0 tx=0x7f0bd4033400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63ace36c4888903 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.513+0000 7f0bc4ff9640 1 -- 172.21.15.43:0/599352341 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4037070 con 0x55d8914a06b0 2024-04-25T10:39:39.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.513+0000 7f0bc4ff9640 1 -- 172.21.15.43:0/599352341 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd404e070 con 0x55d8914a06b0 2024-04-25T10:39:39.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.513+0000 7f0bc4ff9640 1 -- 172.21.15.43:0/599352341 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4041040 con 0x55d8914a06b0 2024-04-25T10:39:39.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.513+0000 7f0bec5dd740 1 -- 172.21.15.43:0/599352341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 msgr2=0x55d89186ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.513+0000 7f0bec5dd740 1 --2- 172.21.15.43:0/599352341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89186ee80 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4004cd0 tx=0x7f0bd4033400 comp rx=0 tx=0).stop 2024-04-25T10:39:39.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.514+0000 7f0bec5dd740 1 -- 172.21.15.43:0/599352341 shutdown_connections 2024-04-25T10:39:39.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.514+0000 7f0bec5dd740 1 --2- 172.21.15.43:0/599352341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914c1390 0x55d89186c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.514+0000 7f0bec5dd740 1 --2- 172.21.15.43:0/599352341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89186ee80 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.514+0000 7f0bec5dd740 1 --2- 172.21.15.43:0/599352341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914984d0 0x55d8918674f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.514+0000 7f0bec5dd740 1 -- 172.21.15.43:0/599352341 >> 172.21.15.43:0/599352341 conn(0x55d8914b54e0 msgr2=0x55d89186e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.514+0000 7f0bec5dd740 1 -- 172.21.15.43:0/599352341 shutdown_connections 2024-04-25T10:39:39.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.514+0000 7f0bec5dd740 1 -- 172.21.15.43:0/599352341 wait complete. 2024-04-25T10:39:39.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.514+0000 7f0bec5dd740 1 -- start start 2024-04-25T10:39:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.514+0000 7f0bec5dd740 1 Processor -- start 2024-04-25T10:39:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.514+0000 7f0bec5dd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914984d0 0x55d89187f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.514+0000 7f0bec5dd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89187fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.514+0000 7f0bec5dd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914c1390 0x55d8918e55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.514+0000 7f0bec5dd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0be01abbf0 con 0x55d8914a06b0 2024-04-25T10:39:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.514+0000 7f0bec5dd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0be01ad060 con 0x55d8914984d0 2024-04-25T10:39:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.514+0000 7f0bec5dd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0be01ac2c0 con 0x55d8914c1390 2024-04-25T10:39:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.514+0000 7f0bec5dd740 10 client.0 _ll_register_callbacks cb 0x7f0bd8009310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.514+0000 7f0bde7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89187fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.514+0000 7f0bde7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89187fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40006/0 (socket says 172.21.15.43:40006) 2024-04-25T10:39:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.514+0000 7f0bde7fc640 1 -- 172.21.15.43:0/2091750235 learned_addr learned my addr 172.21.15.43:0/2091750235 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.514+0000 7f0bdeffd640 1 --2- 172.21.15.43:0/2091750235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914984d0 0x55d89187f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.515+0000 7f0bdf7fe640 1 --2- 172.21.15.43:0/2091750235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914c1390 0x55d8918e55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.515+0000 7f0bde7fc640 1 -- 172.21.15.43:0/2091750235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914c1390 msgr2=0x55d8918e55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.515+0000 7f0bde7fc640 1 --2- 172.21.15.43:0/2091750235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914c1390 0x55d8918e55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.515+0000 7f0bde7fc640 1 -- 172.21.15.43:0/2091750235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914984d0 msgr2=0x55d89187f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.515+0000 7f0bde7fc640 1 --2- 172.21.15.43:0/2091750235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914984d0 0x55d89187f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.515+0000 7f0bde7fc640 1 -- 172.21.15.43:0/2091750235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8918e5af0 con 0x55d8914a06b0 2024-04-25T10:39:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.515+0000 7f0bde7fc640 1 --2- 172.21.15.43:0/2091750235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89187fd50 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4004c50 tx=0x7f0bd4034d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d7fb6a9818e083 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.515+0000 7f0bc2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.515+0000 7f0bc2ff5640 1 -- 172.21.15.43:0/2091750235 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4041040 con 0x55d8914a06b0 2024-04-25T10:39:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.515+0000 7f0bc2ff5640 1 -- 172.21.15.43:0/2091750235 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f0bd40046c0 con 0x55d8914a06b0 2024-04-25T10:39:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.515+0000 7f0bec5dd740 1 -- 172.21.15.43:0/2091750235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bd404e070 con 0x55d8914a06b0 2024-04-25T10:39:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.515+0000 7f0bc2ff5640 1 -- 172.21.15.43:0/2091750235 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0bd4037040 con 0x55d8914a06b0 2024-04-25T10:39:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.515+0000 7f0bec5dd740 10 client.17514 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.515+0000 7f0bec5dd740 1 -- 172.21.15.43:0/2091750235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d8918e6d00 con 0x55d8914a06b0 2024-04-25T10:39:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.515+0000 7f0bc2ff5640 1 -- 172.21.15.43:0/2091750235 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd404e070 con 0x55d8914a06b0 2024-04-25T10:39:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.516+0000 7f0bc2ff5640 1 -- 172.21.15.43:0/2091750235 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0bd403d820 con 0x55d8914a06b0 2024-04-25T10:39:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.516+0000 7f0bec5dd740 10 client.17514 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.516+0000 7f0bec5dd740 10 client.17514 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:39:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.516+0000 7f0bec5dd740 20 client.17514 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.516+0000 7f0bec5dd740 1 --2- 172.21.15.43:0/2091750235 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d8914553a0 0x55d8918e9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.516+0000 7f0bec5dd740 4 client.17514 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:39:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.516+0000 7f0bec5dd740 1 -- 172.21.15.43:0/2091750235 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d8918630e0 con 0x55d8914553a0 2024-04-25T10:39:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.516+0000 7f0bdeffd640 1 --2- 172.21.15.43:0/2091750235 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d8914553a0 0x55d8918e9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.516+0000 7f0bdeffd640 1 --2- 172.21.15.43:0/2091750235 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d8914553a0 0x55d8918e9420 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5cc9188eb9b3ac1c server_cookie=7ffd2c750cf0c5ff in_seq=0 out_seq=0 2024-04-25T10:39:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.516+0000 7f0bc2ff5640 10 client.17514 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:39:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.517+0000 7f0bc2ff5640 1 -- 172.21.15.43:0/2091750235 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d8918630e0 con 0x55d8914553a0 2024-04-25T10:39:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.517+0000 7f0bc2ff5640 10 client.17514 handle_command_reply: tid=0 2024-04-25T10:39:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.517+0000 7f0bec5dd740 1 client.17514 shutdown 2024-04-25T10:39:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.517+0000 7f0bec5dd740 1 -- 172.21.15.43:0/2091750235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 msgr2=0x55d89187fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.517+0000 7f0bec5dd740 1 --2- 172.21.15.43:0/2091750235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89187fd50 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4004c50 tx=0x7f0bd4034d60 comp rx=0 tx=0).stop 2024-04-25T10:39:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.517+0000 7f0bec5dd740 1 -- 172.21.15.43:0/2091750235 shutdown_connections 2024-04-25T10:39:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.517+0000 7f0bec5dd740 1 --2- 172.21.15.43:0/2091750235 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d8914553a0 0x55d8918e9420 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.517+0000 7f0bec5dd740 1 --2- 172.21.15.43:0/2091750235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8914c1390 0x55d8918e55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.517+0000 7f0bc2ff5640 0 client.17514 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:39:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.517+0000 7f0bec5dd740 1 --2- 172.21.15.43:0/2091750235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8914a06b0 0x55d89187fd50 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.517+0000 7f0bec5dd740 1 --2- 172.21.15.43:0/2091750235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8914984d0 0x55d89187f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.517+0000 7f0bec5dd740 1 -- 172.21.15.43:0/2091750235 >> 172.21.15.43:0/2091750235 conn(0x55d8914b54e0 msgr2=0x55d89186e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.517+0000 7f0bec5dd740 1 -- 172.21.15.43:0/2091750235 shutdown_connections 2024-04-25T10:39:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.517+0000 7f0bec5dd740 1 -- 172.21.15.43:0/2091750235 wait complete. 2024-04-25T10:39:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.517+0000 7f0bec5dd740 20 client.17514 trim_cache size 0 max 16384 2024-04-25T10:39:39.525 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:39:39.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.517+0000 7f0be564a640 1 -- 172.21.15.43:0/1170320109 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0bc80afc60 msgr2=0x7f0bc80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.517+0000 7f0be564a640 1 --2- 172.21.15.43:0/1170320109 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0bc80afc60 0x7f0bc80b2170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8009fd0 tx=0x7f0bd8004730 comp rx=0 tx=0).stop 2024-04-25T10:39:39.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.517+0000 7f0be564a640 1 -- 172.21.15.43:0/1170320109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be0102af0 msgr2=0x7f0be0198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.517+0000 7f0be564a640 1 --2- 172.21.15.43:0/1170320109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be0102af0 0x7f0be0198520 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000dbf0 tx=0x7f0bd000b630 comp rx=0 tx=0).stop 2024-04-25T10:39:39.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.518+0000 7f0be564a640 1 -- 172.21.15.43:0/1170320109 shutdown_connections 2024-04-25T10:39:39.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.518+0000 7f0be564a640 1 --2- 172.21.15.43:0/1170320109 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f0bc80afc60 0x7f0bc80b2170 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.518+0000 7f0be564a640 1 --2- 172.21.15.43:0/1170320109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0be0108a20 0x7f0be0198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.518+0000 7f0be564a640 1 --2- 172.21.15.43:0/1170320109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be010d380 0x7f0be019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.518+0000 7f0be564a640 1 --2- 172.21.15.43:0/1170320109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be0102af0 0x7f0be0198520 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.518+0000 7f0be564a640 1 -- 172.21.15.43:0/1170320109 >> 172.21.15.43:0/1170320109 conn(0x7f0be0078100 msgr2=0x7f0be01040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.518+0000 7f0be564a640 1 -- 172.21.15.43:0/1170320109 shutdown_connections 2024-04-25T10:39:39.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.518+0000 7f0be564a640 1 -- 172.21.15.43:0/1170320109 wait complete. 2024-04-25T10:39:39.531 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:39:39.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.643+0000 7f47884d1640 1 -- 172.21.15.43:0/2812337949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f478010d380 msgr2=0x7f478010f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.643+0000 7f47884d1640 1 --2- 172.21.15.43:0/2812337949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f478010d380 0x7f478010f7a0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f4770009ca0 tx=0x7f477002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.643+0000 7f47884d1640 1 -- 172.21.15.43:0/2812337949 shutdown_connections 2024-04-25T10:39:39.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.643+0000 7f47884d1640 1 --2- 172.21.15.43:0/2812337949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f478010d380 0x7f478010f7a0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.643+0000 7f47884d1640 1 --2- 172.21.15.43:0/2812337949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4780102af0 0x7f4780102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.643+0000 7f47884d1640 1 --2- 172.21.15.43:0/2812337949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4780108a20 0x7f47801025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.643+0000 7f47884d1640 1 -- 172.21.15.43:0/2812337949 >> 172.21.15.43:0/2812337949 conn(0x7f4780078100 msgr2=0x7f4780078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.643+0000 7f47884d1640 1 -- 172.21.15.43:0/2812337949 shutdown_connections 2024-04-25T10:39:39.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.643+0000 7f47884d1640 1 -- 172.21.15.43:0/2812337949 wait complete. 2024-04-25T10:39:39.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.644+0000 7f47884d1640 1 Processor -- start 2024-04-25T10:39:39.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.644+0000 7f47884d1640 1 -- start start 2024-04-25T10:39:39.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.644+0000 7f47884d1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4780102af0 0x7f478019b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.644+0000 7f47884d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4780108a20 0x7f478019ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.644+0000 7f47884d1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f478010d380 0x7f478019bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.644+0000 7f47884d1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4780112c00 con 0x7f4780108a20 2024-04-25T10:39:39.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.644+0000 7f47884d1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4780112a80 con 0x7f478010d380 2024-04-25T10:39:39.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.644+0000 7f47884d1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4780112d80 con 0x7f4780102af0 2024-04-25T10:39:39.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.644+0000 7f4785a6c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4780108a20 0x7f478019ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.645+0000 7f4785a6c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4780108a20 0x7f478019ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40016/0 (socket says 172.21.15.43:40016) 2024-04-25T10:39:39.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.645+0000 7f4785a6c640 1 -- 172.21.15.43:0/1115425990 learned_addr learned my addr 172.21.15.43:0/1115425990 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:39.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.645+0000 7f4786a6e640 1 --2- 172.21.15.43:0/1115425990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f478010d380 0x7f478019bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.645+0000 7f478626d640 1 --2- 172.21.15.43:0/1115425990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4780102af0 0x7f478019b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.645+0000 7f4785a6c640 1 -- 172.21.15.43:0/1115425990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4780102af0 msgr2=0x7f478019b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.645+0000 7f4785a6c640 1 --2- 172.21.15.43:0/1115425990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4780102af0 0x7f478019b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.645+0000 7f4785a6c640 1 -- 172.21.15.43:0/1115425990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f478010d380 msgr2=0x7f478019bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.645+0000 7f4785a6c640 1 --2- 172.21.15.43:0/1115425990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f478010d380 0x7f478019bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.645+0000 7f4785a6c640 1 -- 172.21.15.43:0/1115425990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47801a67c0 con 0x7f4780108a20 2024-04-25T10:39:39.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.645+0000 7f478626d640 1 --2- 172.21.15.43:0/1115425990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4780102af0 0x7f478019b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:39:39.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.645+0000 7f4786a6e640 1 --2- 172.21.15.43:0/1115425990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f478010d380 0x7f478019bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:39.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.645+0000 7f4785a6c640 1 --2- 172.21.15.43:0/1115425990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4780108a20 0x7f478019ba80 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f4774005d00 tx=0x7f477400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.645+0000 7f477f7fe640 1 -- 172.21.15.43:0/1115425990 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774017070 con 0x7f4780108a20 2024-04-25T10:39:39.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.645+0000 7f477f7fe640 1 -- 172.21.15.43:0/1115425990 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4774009c30 con 0x7f4780108a20 2024-04-25T10:39:39.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.645+0000 7f47884d1640 1 -- 172.21.15.43:0/1115425990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47801a6ab0 con 0x7f4780108a20 2024-04-25T10:39:39.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.645+0000 7f477f7fe640 1 -- 172.21.15.43:0/1115425990 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477400f620 con 0x7f4780108a20 2024-04-25T10:39:39.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.645+0000 7f47884d1640 1 -- 172.21.15.43:0/1115425990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47801a6fb0 con 0x7f4780108a20 2024-04-25T10:39:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.648+0000 7f477f7fe640 1 -- 172.21.15.43:0/1115425990 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f477401a070 con 0x7f4780108a20 2024-04-25T10:39:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.648+0000 7f478f37d740 1 Processor -- start 2024-04-25T10:39:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.648+0000 7f478f37d740 1 -- start start 2024-04-25T10:39:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.648+0000 7f478f37d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff23c5390 0x558ff2766810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.648+0000 7f478f37d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff239c4d0 0x558ff2766d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.649+0000 7f478f37d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff23a46b0 0x558ff27736a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.649+0000 7f478f37d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558ff26cbd80 con 0x558ff239c4d0 2024-04-25T10:39:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.649+0000 7f478f37d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558ff2685a20 con 0x558ff23c5390 2024-04-25T10:39:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.649+0000 7f478f37d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558ff26c7d40 con 0x558ff23a46b0 2024-04-25T10:39:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.649+0000 7f478626d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff239c4d0 0x558ff2766d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.649+0000 7f4786a6e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff23c5390 0x558ff2766810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.649+0000 7f4786a6e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff23c5390 0x558ff2766810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:42308/0 (socket says 172.21.15.43:42308) 2024-04-25T10:39:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.649+0000 7f478626d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff239c4d0 0x558ff2766d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40022/0 (socket says 172.21.15.43:40022) 2024-04-25T10:39:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.649+0000 7f4786a6e640 1 -- 172.21.15.43:0/4251192841 learned_addr learned my addr 172.21.15.43:0/4251192841 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.649+0000 7f4785a6c640 1 --2- 172.21.15.43:0/4251192841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff23a46b0 0x558ff27736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.649+0000 7f477f7fe640 1 --2- 172.21.15.43:0/1115425990 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f47680afc60 0x7f47680b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.649+0000 7f478626d640 1 -- 172.21.15.43:0/4251192841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff23a46b0 msgr2=0x558ff27736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.649+0000 7f478626d640 1 --2- 172.21.15.43:0/4251192841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff23a46b0 0x558ff27736a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.649+0000 7f478626d640 1 -- 172.21.15.43:0/4251192841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff23c5390 msgr2=0x558ff2766810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.649+0000 7f478626d640 1 --2- 172.21.15.43:0/4251192841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff23c5390 0x558ff2766810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.649+0000 7f478626d640 1 -- 172.21.15.43:0/4251192841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff2690c30 con 0x558ff239c4d0 2024-04-25T10:39:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.649+0000 7f4786a6e640 1 --2- 172.21.15.43:0/4251192841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff23c5390 0x558ff2766810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:39:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.649+0000 7f4786a6e640 1 --2- 172.21.15.43:0/1115425990 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f47680afc60 0x7f47680b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.649+0000 7f477f7fe640 1 -- 172.21.15.43:0/1115425990 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f47740deb90 con 0x7f4780108a20 2024-04-25T10:39:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.649+0000 7f478626d640 1 --2- 172.21.15.43:0/4251192841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff239c4d0 0x558ff2766d50 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f477800bab0 tx=0x7f477800bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d131165696ab9c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.650+0000 7f477cbfe640 1 -- 172.21.15.43:0/4251192841 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4778017420 con 0x558ff239c4d0 2024-04-25T10:39:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.650+0000 7f477cbfe640 1 -- 172.21.15.43:0/4251192841 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4778017a70 con 0x558ff239c4d0 2024-04-25T10:39:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.650+0000 7f4786a6e640 1 --2- 172.21.15.43:0/1115425990 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f47680afc60 0x7f47680b2170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x558ff2763980 tx=0x7f47700095c0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.650+0000 7f477cbfe640 1 -- 172.21.15.43:0/4251192841 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477800f9e0 con 0x558ff239c4d0 2024-04-25T10:39:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.651+0000 7f478f37d740 1 -- 172.21.15.43:0/4251192841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff239c4d0 msgr2=0x558ff2766d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.651+0000 7f478f37d740 1 --2- 172.21.15.43:0/4251192841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff239c4d0 0x558ff2766d50 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f477800bab0 tx=0x7f477800bf70 comp rx=0 tx=0).stop 2024-04-25T10:39:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.651+0000 7f478f37d740 1 -- 172.21.15.43:0/4251192841 shutdown_connections 2024-04-25T10:39:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.651+0000 7f478f37d740 1 --2- 172.21.15.43:0/4251192841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff23a46b0 0x558ff27736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.651+0000 7f478f37d740 1 --2- 172.21.15.43:0/4251192841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff239c4d0 0x558ff2766d50 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.651+0000 7f478f37d740 1 --2- 172.21.15.43:0/4251192841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff23c5390 0x558ff2766810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.651+0000 7f478f37d740 1 -- 172.21.15.43:0/4251192841 >> 172.21.15.43:0/4251192841 conn(0x558ff23b94e0 msgr2=0x558ff2762fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.651+0000 7f478f37d740 1 -- 172.21.15.43:0/4251192841 shutdown_connections 2024-04-25T10:39:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.651+0000 7f478f37d740 1 -- 172.21.15.43:0/4251192841 wait complete. 2024-04-25T10:39:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.651+0000 7f478f37d740 1 -- start start 2024-04-25T10:39:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.651+0000 7f478f37d740 1 Processor -- start 2024-04-25T10:39:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.651+0000 7f478f37d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff239c4d0 0x558ff2771220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.651+0000 7f478f37d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff23a46b0 0x558ff2771760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.651+0000 7f478f37d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff23c5390 0x558ff277d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.651+0000 7f478f37d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558ff2773d90 con 0x558ff23a46b0 2024-04-25T10:39:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.651+0000 7f478f37d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558ff2652dc0 con 0x558ff239c4d0 2024-04-25T10:39:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.651+0000 7f478f37d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558ff2773f10 con 0x558ff23c5390 2024-04-25T10:39:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.652+0000 7f478f37d740 10 client.0 _ll_register_callbacks cb 0x558ff266e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.652+0000 7f4785a6c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff23a46b0 0x558ff2771760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.652+0000 7f4785a6c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff23a46b0 0x558ff2771760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40026/0 (socket says 172.21.15.43:40026) 2024-04-25T10:39:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.652+0000 7f4785a6c640 1 -- 172.21.15.43:0/2678639059 learned_addr learned my addr 172.21.15.43:0/2678639059 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.652+0000 7f478626d640 1 --2- 172.21.15.43:0/2678639059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff239c4d0 0x558ff2771220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.652+0000 7f4786a6e640 1 --2- 172.21.15.43:0/2678639059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff23c5390 0x558ff277d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.652+0000 7f4785a6c640 1 -- 172.21.15.43:0/2678639059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff23c5390 msgr2=0x558ff277d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.652+0000 7f4785a6c640 1 --2- 172.21.15.43:0/2678639059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff23c5390 0x558ff277d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.652+0000 7f4785a6c640 1 -- 172.21.15.43:0/2678639059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff239c4d0 msgr2=0x558ff2771220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.652+0000 7f4785a6c640 1 --2- 172.21.15.43:0/2678639059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff239c4d0 0x558ff2771220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.652+0000 7f4785a6c640 1 -- 172.21.15.43:0/2678639059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff23dd3f0 con 0x558ff23a46b0 2024-04-25T10:39:39.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.652+0000 7f478626d640 1 --2- 172.21.15.43:0/2678639059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff239c4d0 0x558ff2771220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:39.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.652+0000 7f4785a6c640 1 --2- 172.21.15.43:0/2678639059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff23a46b0 0x558ff2771760 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4774009610 tx=0x7f477400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6af4483debbd0ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.652+0000 7f4765ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:39.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.652+0000 7f4765ffb640 1 -- 172.21.15.43:0/2678639059 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774021050 con 0x558ff23a46b0 2024-04-25T10:39:39.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.652+0000 7f4765ffb640 1 -- 172.21.15.43:0/2678639059 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f47740a26a0 con 0x558ff23a46b0 2024-04-25T10:39:39.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.652+0000 7f478f37d740 1 -- 172.21.15.43:0/2678639059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4778017a70 con 0x558ff23a46b0 2024-04-25T10:39:39.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.652+0000 7f4765ffb640 1 -- 172.21.15.43:0/2678639059 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f47740a6680 con 0x558ff23a46b0 2024-04-25T10:39:39.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.653+0000 7f478f37d740 10 client.17526 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:39.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.653+0000 7f478f37d740 1 -- 172.21.15.43:0/2678639059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558ff24a77d0 con 0x558ff23a46b0 2024-04-25T10:39:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.653+0000 7f4765ffb640 1 -- 172.21.15.43:0/2678639059 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4778017a70 con 0x558ff23a46b0 2024-04-25T10:39:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.653+0000 7f4765ffb640 1 -- 172.21.15.43:0/2678639059 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f47740e5070 con 0x558ff23a46b0 2024-04-25T10:39:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.653+0000 7f478f37d740 10 client.17526 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.653+0000 7f478f37d740 10 client.17526 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:39:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.653+0000 7f478f37d740 20 client.17526 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.653+0000 7f478f37d740 1 --2- 172.21.15.43:0/2678639059 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x558ff2383f60 0x558ff2783620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.653+0000 7f478f37d740 4 client.17526 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:39:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.653+0000 7f478f37d740 1 -- 172.21.15.43:0/2678639059 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ff26d9990 con 0x558ff2383f60 2024-04-25T10:39:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.653+0000 7f478626d640 1 --2- 172.21.15.43:0/2678639059 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x558ff2383f60 0x558ff2783620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.654+0000 7f4765ffb640 10 client.17526 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:39:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.654+0000 7f478626d640 1 --2- 172.21.15.43:0/2678639059 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x558ff2383f60 0x558ff2783620 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=23460eaa8bc22444 server_cookie=3f2e705386e83864 in_seq=0 out_seq=0 2024-04-25T10:39:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.655+0000 7f4765ffb640 1 -- 172.21.15.43:0/2678639059 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558ff26d9990 con 0x558ff2383f60 2024-04-25T10:39:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.655+0000 7f4765ffb640 10 client.17526 handle_command_reply: tid=0 2024-04-25T10:39:39.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.655+0000 7f478f37d740 1 client.17526 shutdown 2024-04-25T10:39:39.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.655+0000 7f478f37d740 1 -- 172.21.15.43:0/2678639059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff23a46b0 msgr2=0x558ff2771760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.655+0000 7f478f37d740 1 --2- 172.21.15.43:0/2678639059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff23a46b0 0x558ff2771760 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4774009610 tx=0x7f477400bf90 comp rx=0 tx=0).stop 2024-04-25T10:39:39.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.655+0000 7f478f37d740 1 -- 172.21.15.43:0/2678639059 shutdown_connections 2024-04-25T10:39:39.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.655+0000 7f478f37d740 1 --2- 172.21.15.43:0/2678639059 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x558ff2383f60 0x558ff2783620 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.655+0000 7f478f37d740 1 --2- 172.21.15.43:0/2678639059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff23c5390 0x558ff277d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.655+0000 7f478f37d740 1 --2- 172.21.15.43:0/2678639059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff23a46b0 0x558ff2771760 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.655+0000 7f4765ffb640 0 client.17526 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:39:39.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.655+0000 7f478f37d740 1 --2- 172.21.15.43:0/2678639059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff239c4d0 0x558ff2771220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.655+0000 7f478f37d740 1 -- 172.21.15.43:0/2678639059 >> 172.21.15.43:0/2678639059 conn(0x558ff23b94e0 msgr2=0x558ff2768530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.655+0000 7f478f37d740 1 -- 172.21.15.43:0/2678639059 shutdown_connections 2024-04-25T10:39:39.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.655+0000 7f478f37d740 1 -- 172.21.15.43:0/2678639059 wait complete. 2024-04-25T10:39:39.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.655+0000 7f478f37d740 20 client.17526 trim_cache size 0 max 16384 2024-04-25T10:39:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.669+0000 7f478f37d740 1 Processor -- start 2024-04-25T10:39:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.669+0000 7f478f37d740 1 -- start start 2024-04-25T10:39:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.669+0000 7f478f37d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff239c4d0 0x558ff2770c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.669+0000 7f478f37d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff23a46b0 0x558ff277ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.669+0000 7f478f37d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff23c5390 0x558ff2788db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.669+0000 7f478f37d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f47801b2030 con 0x558ff239c4d0 2024-04-25T10:39:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.669+0000 7f478f37d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f47801af900 con 0x558ff23a46b0 2024-04-25T10:39:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.669+0000 7f478f37d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f47801ad770 con 0x558ff23c5390 2024-04-25T10:39:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.669+0000 7f478626d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff239c4d0 0x558ff2770c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.669+0000 7f478626d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff239c4d0 0x558ff2770c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40036/0 (socket says 172.21.15.43:40036) 2024-04-25T10:39:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.669+0000 7f478626d640 1 -- 172.21.15.43:0/4049758570 learned_addr learned my addr 172.21.15.43:0/4049758570 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.669+0000 7f4785a6c640 1 --2- 172.21.15.43:0/4049758570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff23a46b0 0x558ff277ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.669+0000 7f478626d640 1 -- 172.21.15.43:0/4049758570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff23c5390 msgr2=0x558ff2788db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.669+0000 7f478626d640 1 --2- 172.21.15.43:0/4049758570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff23c5390 0x558ff2788db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.669+0000 7f478626d640 1 -- 172.21.15.43:0/4049758570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff23a46b0 msgr2=0x558ff277ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.669+0000 7f478626d640 1 --2- 172.21.15.43:0/4049758570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff23a46b0 0x558ff277ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.669+0000 7f478626d640 1 -- 172.21.15.43:0/4049758570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff276cbd0 con 0x558ff239c4d0 2024-04-25T10:39:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.669+0000 7f4785a6c640 1 --2- 172.21.15.43:0/4049758570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff23a46b0 0x558ff277ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:39:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.670+0000 7f478626d640 1 --2- 172.21.15.43:0/4049758570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff239c4d0 0x558ff2770c60 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4778019040 tx=0x7f477800b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f7b7bdfaa4c5f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.670+0000 7f4767fff640 1 -- 172.21.15.43:0/4049758570 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4778005240 con 0x558ff239c4d0 2024-04-25T10:39:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.670+0000 7f4767fff640 1 -- 172.21.15.43:0/4049758570 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47780053e0 con 0x558ff239c4d0 2024-04-25T10:39:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.670+0000 7f4767fff640 1 -- 172.21.15.43:0/4049758570 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47780048c0 con 0x558ff239c4d0 2024-04-25T10:39:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.670+0000 7f478f37d740 1 -- 172.21.15.43:0/4049758570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff239c4d0 msgr2=0x558ff2770c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.670+0000 7f478f37d740 1 --2- 172.21.15.43:0/4049758570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff239c4d0 0x558ff2770c60 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4778019040 tx=0x7f477800b7d0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.670+0000 7f478f37d740 1 -- 172.21.15.43:0/4049758570 shutdown_connections 2024-04-25T10:39:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.670+0000 7f478f37d740 1 --2- 172.21.15.43:0/4049758570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff23a46b0 0x558ff277ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.670+0000 7f478f37d740 1 --2- 172.21.15.43:0/4049758570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff23c5390 0x558ff2788db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.670+0000 7f478f37d740 1 --2- 172.21.15.43:0/4049758570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff239c4d0 0x558ff2770c60 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.670+0000 7f478f37d740 1 -- 172.21.15.43:0/4049758570 >> 172.21.15.43:0/4049758570 conn(0x558ff23b94e0 msgr2=0x558ff2770550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.671+0000 7f478f37d740 1 -- 172.21.15.43:0/4049758570 shutdown_connections 2024-04-25T10:39:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.671+0000 7f478f37d740 1 -- 172.21.15.43:0/4049758570 wait complete. 2024-04-25T10:39:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.671+0000 7f478f37d740 1 -- start start 2024-04-25T10:39:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.671+0000 7f478f37d740 1 Processor -- start 2024-04-25T10:39:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.671+0000 7f478f37d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff239c4d0 0x558ff27795b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.671+0000 7f478f37d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff23a46b0 0x558ff27748b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.671+0000 7f478f37d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff23c5390 0x558ff2774df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.671+0000 7f478f37d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f47801addc0 con 0x558ff23a46b0 2024-04-25T10:39:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.671+0000 7f478f37d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f47801af230 con 0x558ff239c4d0 2024-04-25T10:39:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.671+0000 7f478f37d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f47801ae490 con 0x558ff23c5390 2024-04-25T10:39:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.671+0000 7f478f37d740 10 client.0 _ll_register_callbacks cb 0x7f47740dc770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.671+0000 7f4785a6c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff23a46b0 0x558ff27748b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.671+0000 7f4785a6c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff23a46b0 0x558ff27748b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40048/0 (socket says 172.21.15.43:40048) 2024-04-25T10:39:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.671+0000 7f4785a6c640 1 -- 172.21.15.43:0/424094853 learned_addr learned my addr 172.21.15.43:0/424094853 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.671+0000 7f4786a6e640 1 --2- 172.21.15.43:0/424094853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff23c5390 0x558ff2774df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.672+0000 7f478626d640 1 --2- 172.21.15.43:0/424094853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff239c4d0 0x558ff27795b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.672+0000 7f4785a6c640 1 -- 172.21.15.43:0/424094853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff23c5390 msgr2=0x558ff2774df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.672+0000 7f4785a6c640 1 --2- 172.21.15.43:0/424094853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff23c5390 0x558ff2774df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.672+0000 7f4785a6c640 1 -- 172.21.15.43:0/424094853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff239c4d0 msgr2=0x558ff27795b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.672+0000 7f4785a6c640 1 --2- 172.21.15.43:0/424094853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff239c4d0 0x558ff27795b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.672+0000 7f4785a6c640 1 -- 172.21.15.43:0/424094853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff2775330 con 0x558ff23a46b0 2024-04-25T10:39:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.672+0000 7f4786a6e640 1 --2- 172.21.15.43:0/424094853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff23c5390 0x558ff2774df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.672+0000 7f4785a6c640 1 --2- 172.21.15.43:0/424094853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff23a46b0 0x558ff27748b0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f4774012040 tx=0x7f47740a63d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceac4689e5e86660 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.672+0000 7f4765ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.672+0000 7f4765ffb640 1 -- 172.21.15.43:0/424094853 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774021050 con 0x558ff23a46b0 2024-04-25T10:39:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.672+0000 7f4765ffb640 1 -- 172.21.15.43:0/424094853 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f47740ac480 con 0x558ff23a46b0 2024-04-25T10:39:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.672+0000 7f478f37d740 1 -- 172.21.15.43:0/424094853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47780053e0 con 0x558ff23a46b0 2024-04-25T10:39:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.672+0000 7f4765ffb640 1 -- 172.21.15.43:0/424094853 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f47740a64a0 con 0x558ff23a46b0 2024-04-25T10:39:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.672+0000 7f478f37d740 10 client.17532 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.672+0000 7f478f37d740 1 -- 172.21.15.43:0/424094853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558ff27eb5c0 con 0x558ff23a46b0 2024-04-25T10:39:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.672+0000 7f4765ffb640 1 -- 172.21.15.43:0/424094853 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47780053e0 con 0x558ff23a46b0 2024-04-25T10:39:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.673+0000 7f4765ffb640 1 -- 172.21.15.43:0/424094853 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f47740e5070 con 0x558ff23a46b0 2024-04-25T10:39:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.673+0000 7f478f37d740 10 client.17532 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.673+0000 7f478f37d740 10 client.17532 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:39:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.673+0000 7f478f37d740 20 client.17532 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.673+0000 7f478f37d740 1 --2- 172.21.15.43:0/424094853 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x558ff23593a0 0x558ff27edd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.673+0000 7f478f37d740 4 client.17532 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:39:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.673+0000 7f478f37d740 1 -- 172.21.15.43:0/424094853 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ff26d3800 con 0x558ff23593a0 2024-04-25T10:39:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.673+0000 7f478626d640 1 --2- 172.21.15.43:0/424094853 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x558ff23593a0 0x558ff27edd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.673+0000 7f4765ffb640 10 client.17532 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:39:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.673+0000 7f478626d640 1 --2- 172.21.15.43:0/424094853 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x558ff23593a0 0x558ff27edd10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c6176cfa3f8cc9f2 server_cookie=7dd0042f50474d87 in_seq=0 out_seq=0 2024-04-25T10:39:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f4765ffb640 1 -- 172.21.15.43:0/424094853 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ff26d3800 con 0x558ff23593a0 2024-04-25T10:39:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f4765ffb640 10 client.17532 handle_command_reply: tid=0 2024-04-25T10:39:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f478f37d740 1 client.17532 shutdown 2024-04-25T10:39:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f478f37d740 1 -- 172.21.15.43:0/424094853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff23a46b0 msgr2=0x558ff27748b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f478f37d740 1 --2- 172.21.15.43:0/424094853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff23a46b0 0x558ff27748b0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f4774012040 tx=0x7f47740a63d0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f478f37d740 1 -- 172.21.15.43:0/424094853 shutdown_connections 2024-04-25T10:39:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f478f37d740 1 --2- 172.21.15.43:0/424094853 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x558ff23593a0 0x558ff27edd10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f478f37d740 1 --2- 172.21.15.43:0/424094853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff23c5390 0x558ff2774df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f478f37d740 1 --2- 172.21.15.43:0/424094853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ff23a46b0 0x558ff27748b0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f478f37d740 1 --2- 172.21.15.43:0/424094853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ff239c4d0 0x558ff27795b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f4765ffb640 0 client.17532 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:39:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f478f37d740 1 -- 172.21.15.43:0/424094853 >> 172.21.15.43:0/424094853 conn(0x558ff23b94e0 msgr2=0x558ff2762760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f478f37d740 1 -- 172.21.15.43:0/424094853 shutdown_connections 2024-04-25T10:39:39.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f478f37d740 1 -- 172.21.15.43:0/424094853 wait complete. 2024-04-25T10:39:39.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f478f37d740 20 client.17532 trim_cache size 0 max 16384 2024-04-25T10:39:39.683 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:39:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f47884d1640 1 -- 172.21.15.43:0/1115425990 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f47680afc60 msgr2=0x7f47680b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f47884d1640 1 --2- 172.21.15.43:0/1115425990 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f47680afc60 0x7f47680b2170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x558ff2763980 tx=0x7f47700095c0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f47884d1640 1 -- 172.21.15.43:0/1115425990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4780108a20 msgr2=0x7f478019ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f47884d1640 1 --2- 172.21.15.43:0/1115425990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4780108a20 0x7f478019ba80 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f4774005d00 tx=0x7f477400dde0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f47884d1640 1 -- 172.21.15.43:0/1115425990 shutdown_connections 2024-04-25T10:39:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f47884d1640 1 --2- 172.21.15.43:0/1115425990 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f47680afc60 0x7f47680b2170 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f47884d1640 1 --2- 172.21.15.43:0/1115425990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f478010d380 0x7f478019bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f47884d1640 1 --2- 172.21.15.43:0/1115425990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4780108a20 0x7f478019ba80 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f47884d1640 1 --2- 172.21.15.43:0/1115425990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4780102af0 0x7f478019b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.674+0000 7f47884d1640 1 -- 172.21.15.43:0/1115425990 >> 172.21.15.43:0/1115425990 conn(0x7f4780078100 msgr2=0x7f47801040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.675+0000 7f47884d1640 1 -- 172.21.15.43:0/1115425990 shutdown_connections 2024-04-25T10:39:39.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.675+0000 7f47884d1640 1 -- 172.21.15.43:0/1115425990 wait complete. 2024-04-25T10:39:39.687 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:39:39.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.799+0000 7fc97adb7640 1 -- 172.21.15.43:0/2152513121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc974108eb0 msgr2=0x7fc974102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.799+0000 7fc97adb7640 1 --2- 172.21.15.43:0/2152513121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc974108eb0 0x7fc974102b50 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009900 tx=0x7fc95c02a410 comp rx=0 tx=0).stop 2024-04-25T10:39:39.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.799+0000 7fc97adb7640 1 -- 172.21.15.43:0/2152513121 shutdown_connections 2024-04-25T10:39:39.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.799+0000 7fc97adb7640 1 --2- 172.21.15.43:0/2152513121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9741031c0 0x7fc97410f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.799+0000 7fc97adb7640 1 --2- 172.21.15.43:0/2152513121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc974108eb0 0x7fc974102b50 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.799+0000 7fc97adb7640 1 --2- 172.21.15.43:0/2152513121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc974108a10 0x7fc974102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.799+0000 7fc97adb7640 1 -- 172.21.15.43:0/2152513121 >> 172.21.15.43:0/2152513121 conn(0x7fc974078110 msgr2=0x7fc974078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.799+0000 7fc97adb7640 1 -- 172.21.15.43:0/2152513121 shutdown_connections 2024-04-25T10:39:39.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.799+0000 7fc97adb7640 1 -- 172.21.15.43:0/2152513121 wait complete. 2024-04-25T10:39:39.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.800+0000 7fc97adb7640 1 Processor -- start 2024-04-25T10:39:39.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.800+0000 7fc97adb7640 1 -- start start 2024-04-25T10:39:39.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.800+0000 7fc97adb7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9741031c0 0x7fc97419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.800+0000 7fc97adb7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc974108a10 0x7fc97419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.800+0000 7fc97adb7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc974108eb0 0x7fc97419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.800+0000 7fc97adb7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc974113d10 con 0x7fc974108eb0 2024-04-25T10:39:39.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.800+0000 7fc97adb7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc974113b90 con 0x7fc9741031c0 2024-04-25T10:39:39.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.800+0000 7fc97adb7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc974113e90 con 0x7fc974108a10 2024-04-25T10:39:39.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.800+0000 7fc979354640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc974108eb0 0x7fc97419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.800+0000 7fc979354640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc974108eb0 0x7fc97419ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40062/0 (socket says 172.21.15.43:40062) 2024-04-25T10:39:39.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.800+0000 7fc979354640 1 -- 172.21.15.43:0/1693526247 learned_addr learned my addr 172.21.15.43:0/1693526247 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:39.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.800+0000 7fc978b53640 1 --2- 172.21.15.43:0/1693526247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9741031c0 0x7fc97419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.800+0000 7fc973fff640 1 --2- 172.21.15.43:0/1693526247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc974108a10 0x7fc97419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.801+0000 7fc979354640 1 -- 172.21.15.43:0/1693526247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc974108a10 msgr2=0x7fc97419acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.801+0000 7fc979354640 1 --2- 172.21.15.43:0/1693526247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc974108a10 0x7fc97419acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.801+0000 7fc979354640 1 -- 172.21.15.43:0/1693526247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9741031c0 msgr2=0x7fc97419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.801+0000 7fc979354640 1 --2- 172.21.15.43:0/1693526247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9741031c0 0x7fc97419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.801+0000 7fc979354640 1 -- 172.21.15.43:0/1693526247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97419f530 con 0x7fc974108eb0 2024-04-25T10:39:39.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.801+0000 7fc979354640 1 --2- 172.21.15.43:0/1693526247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc974108eb0 0x7fc97419ef70 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fc96400d780 tx=0x7fc96400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.801+0000 7fc971ffb640 1 -- 172.21.15.43:0/1693526247 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964017070 con 0x7fc974108eb0 2024-04-25T10:39:39.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.801+0000 7fc971ffb640 1 -- 172.21.15.43:0/1693526247 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc96400de90 con 0x7fc974108eb0 2024-04-25T10:39:39.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.801+0000 7fc97adb7640 1 -- 172.21.15.43:0/1693526247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97419f820 con 0x7fc974108eb0 2024-04-25T10:39:39.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.801+0000 7fc971ffb640 1 -- 172.21.15.43:0/1693526247 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964004c30 con 0x7fc974108eb0 2024-04-25T10:39:39.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.801+0000 7fc97adb7640 1 -- 172.21.15.43:0/1693526247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc97419fd40 con 0x7fc974108eb0 2024-04-25T10:39:39.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.804+0000 7fc981c6d740 1 Processor -- start 2024-04-25T10:39:39.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.804+0000 7fc971ffb640 1 -- 172.21.15.43:0/1693526247 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fc964004dd0 con 0x7fc974108eb0 2024-04-25T10:39:39.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.804+0000 7fc981c6d740 1 -- start start 2024-04-25T10:39:39.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.805+0000 7fc981c6d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593f9353390 0x5593f96f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.805+0000 7fc981c6d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f932a4d0 0x5593f96f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.805+0000 7fc981c6d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f93326b0 0x5593f9705ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.805+0000 7fc981c6d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5593f9659d80 con 0x5593f93326b0 2024-04-25T10:39:39.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.805+0000 7fc981c6d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5593f9613a20 con 0x5593f932a4d0 2024-04-25T10:39:39.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.805+0000 7fc981c6d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5593f9655d40 con 0x5593f9353390 2024-04-25T10:39:39.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.805+0000 7fc973fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f93326b0 0x5593f9705ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.805+0000 7fc973fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f93326b0 0x5593f9705ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40072/0 (socket says 172.21.15.43:40072) 2024-04-25T10:39:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.805+0000 7fc973fff640 1 -- 172.21.15.43:0/1229800724 learned_addr learned my addr 172.21.15.43:0/1229800724 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.805+0000 7fc978b53640 1 --2- 172.21.15.43:0/1229800724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f932a4d0 0x5593f96f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.805+0000 7fc973fff640 1 --2- 172.21.15.43:0/1229800724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593f9353390 0x5593f96f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.805+0000 7fc971ffb640 1 --2- 172.21.15.43:0/1693526247 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9600afce0 0x7fc9600b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.805+0000 7fc973fff640 1 -- 172.21.15.43:0/1229800724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593f9353390 msgr2=0x5593f96f46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.805+0000 7fc973fff640 1 --2- 172.21.15.43:0/1229800724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593f9353390 0x5593f96f46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.805+0000 7fc973fff640 1 -- 172.21.15.43:0/1229800724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f932a4d0 msgr2=0x5593f96f4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.805+0000 7fc973fff640 1 --2- 172.21.15.43:0/1229800724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f932a4d0 0x5593f96f4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.805+0000 7fc973fff640 1 -- 172.21.15.43:0/1229800724 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593f961ec30 con 0x5593f93326b0 2024-04-25T10:39:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.805+0000 7fc979354640 1 --2- 172.21.15.43:0/1693526247 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9600afce0 0x7fc9600b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.805+0000 7fc973fff640 1 --2- 172.21.15.43:0/1229800724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f93326b0 0x5593f9705ab0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fc95c000c00 tx=0x7fc95c005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba5e1f540950403 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.805+0000 7fc971ffb640 1 -- 172.21.15.43:0/1693526247 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7fc9640de9e0 con 0x7fc974108eb0 2024-04-25T10:39:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.806+0000 7fc96affd640 1 -- 172.21.15.43:0/1229800724 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c035040 con 0x5593f93326b0 2024-04-25T10:39:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.806+0000 7fc96affd640 1 -- 172.21.15.43:0/1229800724 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc95c004030 con 0x5593f93326b0 2024-04-25T10:39:39.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.806+0000 7fc96affd640 1 -- 172.21.15.43:0/1229800724 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c003af0 con 0x5593f93326b0 2024-04-25T10:39:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.807+0000 7fc981c6d740 1 -- 172.21.15.43:0/1229800724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f93326b0 msgr2=0x5593f9705ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.807+0000 7fc981c6d740 1 --2- 172.21.15.43:0/1229800724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f93326b0 0x5593f9705ab0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fc95c000c00 tx=0x7fc95c005d90 comp rx=0 tx=0).stop 2024-04-25T10:39:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.807+0000 7fc979354640 1 --2- 172.21.15.43:0/1693526247 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9600afce0 0x7fc9600b21f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc9640a6ab0 tx=0x7fc96405e040 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.807+0000 7fc981c6d740 1 -- 172.21.15.43:0/1229800724 shutdown_connections 2024-04-25T10:39:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.807+0000 7fc981c6d740 1 --2- 172.21.15.43:0/1229800724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f932a4d0 0x5593f96f4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.807+0000 7fc981c6d740 1 --2- 172.21.15.43:0/1229800724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f93326b0 0x5593f9705ab0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.807+0000 7fc981c6d740 1 --2- 172.21.15.43:0/1229800724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593f9353390 0x5593f96f46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.807+0000 7fc981c6d740 1 -- 172.21.15.43:0/1229800724 >> 172.21.15.43:0/1229800724 conn(0x5593f93474e0 msgr2=0x5593f96f0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.807+0000 7fc981c6d740 1 -- 172.21.15.43:0/1229800724 shutdown_connections 2024-04-25T10:39:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.807+0000 7fc981c6d740 1 -- 172.21.15.43:0/1229800724 wait complete. 2024-04-25T10:39:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.807+0000 7fc981c6d740 1 -- start start 2024-04-25T10:39:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.807+0000 7fc981c6d740 1 Processor -- start 2024-04-25T10:39:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.808+0000 7fc981c6d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f932a4d0 0x5593f9703610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.808+0000 7fc981c6d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f93326b0 0x5593f9703b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.808+0000 7fc981c6d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593f9353390 0x5593f970f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.808+0000 7fc981c6d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5593f9706140 con 0x5593f932a4d0 2024-04-25T10:39:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.808+0000 7fc981c6d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5593f95e0dc0 con 0x5593f93326b0 2024-04-25T10:39:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.808+0000 7fc981c6d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5593f97062c0 con 0x5593f9353390 2024-04-25T10:39:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.808+0000 7fc981c6d740 10 client.0 _ll_register_callbacks cb 0x5593f95fc600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.808+0000 7fc973fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f932a4d0 0x5593f9703610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.808+0000 7fc973fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f932a4d0 0x5593f9703610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40088/0 (socket says 172.21.15.43:40088) 2024-04-25T10:39:39.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.808+0000 7fc973fff640 1 -- 172.21.15.43:0/1210310910 learned_addr learned my addr 172.21.15.43:0/1210310910 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:39.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.808+0000 7fc978b53640 1 --2- 172.21.15.43:0/1210310910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f93326b0 0x5593f9703b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.808+0000 7fc973fff640 1 --2- 172.21.15.43:0/1210310910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593f9353390 0x5593f970f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.808+0000 7fc973fff640 1 -- 172.21.15.43:0/1210310910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593f9353390 msgr2=0x5593f970f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.808+0000 7fc973fff640 1 --2- 172.21.15.43:0/1210310910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593f9353390 0x5593f970f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.808+0000 7fc973fff640 1 -- 172.21.15.43:0/1210310910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f93326b0 msgr2=0x5593f9703b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.808+0000 7fc973fff640 1 --2- 172.21.15.43:0/1210310910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f93326b0 0x5593f9703b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.808+0000 7fc973fff640 1 -- 172.21.15.43:0/1210310910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593f936b3f0 con 0x5593f932a4d0 2024-04-25T10:39:39.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.808+0000 7fc973fff640 1 --2- 172.21.15.43:0/1210310910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f932a4d0 0x5593f9703610 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fc95c032fd0 tx=0x7fc95c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5f3577abe29dea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.808+0000 7fc953fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:39.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.808+0000 7fc953fff640 1 -- 172.21.15.43:0/1210310910 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c035070 con 0x5593f932a4d0 2024-04-25T10:39:39.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.809+0000 7fc981c6d740 1 -- 172.21.15.43:0/1210310910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc95c004030 con 0x5593f932a4d0 2024-04-25T10:39:39.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.809+0000 7fc953fff640 1 -- 172.21.15.43:0/1210310910 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7fc95c003870 con 0x5593f932a4d0 2024-04-25T10:39:39.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.809+0000 7fc953fff640 1 -- 172.21.15.43:0/1210310910 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc95c059050 con 0x5593f932a4d0 2024-04-25T10:39:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.809+0000 7fc981c6d740 10 client.17544 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.809+0000 7fc981c6d740 1 -- 172.21.15.43:0/1210310910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5593f94357d0 con 0x5593f932a4d0 2024-04-25T10:39:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.809+0000 7fc953fff640 1 -- 172.21.15.43:0/1210310910 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc95c004030 con 0x5593f932a4d0 2024-04-25T10:39:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.809+0000 7fc953fff640 1 -- 172.21.15.43:0/1210310910 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc95c03a040 con 0x5593f932a4d0 2024-04-25T10:39:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.809+0000 7fc981c6d740 10 client.17544 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.809+0000 7fc981c6d740 10 client.17544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.809+0000 7fc981c6d740 20 client.17544 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.809+0000 7fc981c6d740 1 --2- 172.21.15.43:0/1210310910 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5593f9311f60 0x5593f9715930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.809+0000 7fc981c6d740 4 client.17544 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:39:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.809+0000 7fc981c6d740 1 -- 172.21.15.43:0/1210310910 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593f9667990 con 0x5593f9311f60 2024-04-25T10:39:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.809+0000 7fc978b53640 1 --2- 172.21.15.43:0/1210310910 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5593f9311f60 0x5593f9715930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.810+0000 7fc978b53640 1 --2- 172.21.15.43:0/1210310910 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5593f9311f60 0x5593f9715930 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e65ac333882385cf server_cookie=e3333924e313011 in_seq=0 out_seq=0 2024-04-25T10:39:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.810+0000 7fc953fff640 10 client.17544 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.811+0000 7fc953fff640 1 -- 172.21.15.43:0/1210310910 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5593f9667990 con 0x5593f9311f60 2024-04-25T10:39:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.811+0000 7fc953fff640 10 client.17544 handle_command_reply: tid=0 2024-04-25T10:39:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.811+0000 7fc981c6d740 1 client.17544 shutdown 2024-04-25T10:39:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.811+0000 7fc981c6d740 1 -- 172.21.15.43:0/1210310910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f932a4d0 msgr2=0x5593f9703610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.811+0000 7fc981c6d740 1 --2- 172.21.15.43:0/1210310910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f932a4d0 0x5593f9703610 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fc95c032fd0 tx=0x7fc95c009650 comp rx=0 tx=0).stop 2024-04-25T10:39:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.811+0000 7fc981c6d740 1 -- 172.21.15.43:0/1210310910 shutdown_connections 2024-04-25T10:39:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.811+0000 7fc981c6d740 1 --2- 172.21.15.43:0/1210310910 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5593f9311f60 0x5593f9715930 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.811+0000 7fc981c6d740 1 --2- 172.21.15.43:0/1210310910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f93326b0 0x5593f9703b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.811+0000 7fc981c6d740 1 --2- 172.21.15.43:0/1210310910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593f9353390 0x5593f970f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.811+0000 7fc953fff640 0 client.17544 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.811+0000 7fc981c6d740 1 --2- 172.21.15.43:0/1210310910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f932a4d0 0x5593f9703610 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.811+0000 7fc981c6d740 1 -- 172.21.15.43:0/1210310910 >> 172.21.15.43:0/1210310910 conn(0x5593f93474e0 msgr2=0x5593f96f6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.811+0000 7fc981c6d740 1 -- 172.21.15.43:0/1210310910 shutdown_connections 2024-04-25T10:39:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.811+0000 7fc981c6d740 1 -- 172.21.15.43:0/1210310910 wait complete. 2024-04-25T10:39:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.811+0000 7fc981c6d740 20 client.17544 trim_cache size 0 max 16384 2024-04-25T10:39:39.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.825+0000 7fc981c6d740 1 Processor -- start 2024-04-25T10:39:39.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.825+0000 7fc981c6d740 1 -- start start 2024-04-25T10:39:39.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.825+0000 7fc981c6d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f932a4d0 0x5593f970f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.825+0000 7fc981c6d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593f93326b0 0x5593f96f0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.825+0000 7fc981c6d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f9353390 0x5593f97022b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.825+0000 7fc981c6d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc9741aff00 con 0x5593f9353390 2024-04-25T10:39:39.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.825+0000 7fc981c6d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9741ad7d0 con 0x5593f932a4d0 2024-04-25T10:39:39.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.825+0000 7fc981c6d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc9741ab620 con 0x5593f93326b0 2024-04-25T10:39:39.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.825+0000 7fc973fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f9353390 0x5593f97022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.825+0000 7fc973fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f9353390 0x5593f97022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40090/0 (socket says 172.21.15.43:40090) 2024-04-25T10:39:39.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.825+0000 7fc973fff640 1 -- 172.21.15.43:0/1182534732 learned_addr learned my addr 172.21.15.43:0/1182534732 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:39.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.825+0000 7fc973fff640 1 --2- 172.21.15.43:0/1182534732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f932a4d0 0x5593f970f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.825+0000 7fc978b53640 1 --2- 172.21.15.43:0/1182534732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593f93326b0 0x5593f96f0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.825+0000 7fc973fff640 1 -- 172.21.15.43:0/1182534732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593f93326b0 msgr2=0x5593f96f0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.825+0000 7fc973fff640 1 --2- 172.21.15.43:0/1182534732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593f93326b0 0x5593f96f0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.825+0000 7fc973fff640 1 -- 172.21.15.43:0/1182534732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f932a4d0 msgr2=0x5593f970f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.825+0000 7fc973fff640 1 --2- 172.21.15.43:0/1182534732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f932a4d0 0x5593f970f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.825+0000 7fc973fff640 1 -- 172.21.15.43:0/1182534732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593f971b900 con 0x5593f9353390 2024-04-25T10:39:39.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.825+0000 7fc973fff640 1 --2- 172.21.15.43:0/1182534732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f9353390 0x5593f97022b0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0059d0 tx=0x7fc95c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d4b2a0c4e6d052 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.826+0000 7fc96a7fc640 1 -- 172.21.15.43:0/1182534732 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c035070 con 0x5593f9353390 2024-04-25T10:39:39.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.826+0000 7fc96a7fc640 1 -- 172.21.15.43:0/1182534732 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc95c03d900 con 0x5593f9353390 2024-04-25T10:39:39.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.826+0000 7fc96a7fc640 1 -- 172.21.15.43:0/1182534732 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c003d70 con 0x5593f9353390 2024-04-25T10:39:39.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.826+0000 7fc981c6d740 1 -- 172.21.15.43:0/1182534732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f9353390 msgr2=0x5593f97022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.826+0000 7fc981c6d740 1 --2- 172.21.15.43:0/1182534732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f9353390 0x5593f97022b0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0059d0 tx=0x7fc95c031400 comp rx=0 tx=0).stop 2024-04-25T10:39:39.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.826+0000 7fc981c6d740 1 -- 172.21.15.43:0/1182534732 shutdown_connections 2024-04-25T10:39:39.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.826+0000 7fc981c6d740 1 --2- 172.21.15.43:0/1182534732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f9353390 0x5593f97022b0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.826+0000 7fc981c6d740 1 --2- 172.21.15.43:0/1182534732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593f93326b0 0x5593f96f0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.826+0000 7fc981c6d740 1 --2- 172.21.15.43:0/1182534732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f932a4d0 0x5593f970f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.826+0000 7fc981c6d740 1 -- 172.21.15.43:0/1182534732 >> 172.21.15.43:0/1182534732 conn(0x5593f93474e0 msgr2=0x5593f9701ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.826+0000 7fc981c6d740 1 -- 172.21.15.43:0/1182534732 shutdown_connections 2024-04-25T10:39:39.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.826+0000 7fc981c6d740 1 -- 172.21.15.43:0/1182534732 wait complete. 2024-04-25T10:39:39.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.827+0000 7fc981c6d740 1 -- start start 2024-04-25T10:39:39.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.827+0000 7fc981c6d740 1 Processor -- start 2024-04-25T10:39:39.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.827+0000 7fc981c6d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593f932a4d0 0x5593f9715db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.827+0000 7fc981c6d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f93326b0 0x5593f97162f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.827+0000 7fc981c6d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f9353390 0x5593f9775a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.827+0000 7fc981c6d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc9741abc90 con 0x5593f9353390 2024-04-25T10:39:39.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.827+0000 7fc981c6d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9741ad100 con 0x5593f93326b0 2024-04-25T10:39:39.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.827+0000 7fc981c6d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc9741ac360 con 0x5593f932a4d0 2024-04-25T10:39:39.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.827+0000 7fc981c6d740 10 client.0 _ll_register_callbacks cb 0x7fc96c007910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:39.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.827+0000 7fc973fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f9353390 0x5593f9775a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.828+0000 7fc973fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f9353390 0x5593f9775a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40102/0 (socket says 172.21.15.43:40102) 2024-04-25T10:39:39.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.828+0000 7fc973fff640 1 -- 172.21.15.43:0/4011539671 learned_addr learned my addr 172.21.15.43:0/4011539671 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:39.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.828+0000 7fc973fff640 1 --2- 172.21.15.43:0/4011539671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593f932a4d0 0x5593f9715db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.828+0000 7fc978b53640 1 --2- 172.21.15.43:0/4011539671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f93326b0 0x5593f97162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.828+0000 7fc973fff640 1 -- 172.21.15.43:0/4011539671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593f932a4d0 msgr2=0x5593f9715db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.828+0000 7fc973fff640 1 --2- 172.21.15.43:0/4011539671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593f932a4d0 0x5593f9715db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.828+0000 7fc973fff640 1 -- 172.21.15.43:0/4011539671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f93326b0 msgr2=0x5593f97162f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:39.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.828+0000 7fc973fff640 1 --2- 172.21.15.43:0/4011539671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f93326b0 0x5593f97162f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.828+0000 7fc973fff640 1 -- 172.21.15.43:0/4011539671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593f9775f60 con 0x5593f9353390 2024-04-25T10:39:39.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.828+0000 7fc973fff640 1 --2- 172.21.15.43:0/4011539671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f9353390 0x5593f9775a20 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x5593f9715400 tx=0x7fc95c003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa41c3dcde557e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:39.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.828+0000 7fc953fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:39.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.828+0000 7fc953fff640 1 -- 172.21.15.43:0/4011539671 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c035030 con 0x5593f9353390 2024-04-25T10:39:39.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.828+0000 7fc981c6d740 1 -- 172.21.15.43:0/4011539671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc95c03d900 con 0x5593f9353390 2024-04-25T10:39:39.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.828+0000 7fc953fff640 1 -- 172.21.15.43:0/4011539671 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7fc95c003af0 con 0x5593f9353390 2024-04-25T10:39:39.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.828+0000 7fc953fff640 1 -- 172.21.15.43:0/4011539671 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc95c003cf0 con 0x5593f9353390 2024-04-25T10:39:39.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.828+0000 7fc981c6d740 10 client.17550 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.828+0000 7fc981c6d740 1 -- 172.21.15.43:0/4011539671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5593f977d250 con 0x5593f9353390 2024-04-25T10:39:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.829+0000 7fc953fff640 1 -- 172.21.15.43:0/4011539671 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc95c03d900 con 0x5593f9353390 2024-04-25T10:39:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.829+0000 7fc953fff640 1 -- 172.21.15.43:0/4011539671 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc95c0046c0 con 0x5593f9353390 2024-04-25T10:39:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.829+0000 7fc981c6d740 10 client.17550 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.829+0000 7fc981c6d740 10 client.17550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.829+0000 7fc981c6d740 20 client.17550 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.829+0000 7fc981c6d740 1 --2- 172.21.15.43:0/4011539671 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5593f92e73a0 0x5593f977f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.829+0000 7fc981c6d740 4 client.17550 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:39:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.829+0000 7fc981c6d740 1 -- 172.21.15.43:0/4011539671 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5593f9661800 con 0x5593f92e73a0 2024-04-25T10:39:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.829+0000 7fc978b53640 1 --2- 172.21.15.43:0/4011539671 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5593f92e73a0 0x5593f977f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.829+0000 7fc953fff640 10 client.17550 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.829+0000 7fc978b53640 1 --2- 172.21.15.43:0/4011539671 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5593f92e73a0 0x5593f977f860 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ed3f363fee32db server_cookie=38dd42a829123bd6 in_seq=0 out_seq=0 2024-04-25T10:39:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.830+0000 7fc953fff640 1 -- 172.21.15.43:0/4011539671 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593f9661800 con 0x5593f92e73a0 2024-04-25T10:39:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.830+0000 7fc953fff640 10 client.17550 handle_command_reply: tid=0 2024-04-25T10:39:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.830+0000 7fc981c6d740 1 client.17550 shutdown 2024-04-25T10:39:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.830+0000 7fc981c6d740 1 -- 172.21.15.43:0/4011539671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f9353390 msgr2=0x5593f9775a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.830+0000 7fc981c6d740 1 --2- 172.21.15.43:0/4011539671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f9353390 0x5593f9775a20 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x5593f9715400 tx=0x7fc95c003a10 comp rx=0 tx=0).stop 2024-04-25T10:39:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.830+0000 7fc981c6d740 1 -- 172.21.15.43:0/4011539671 shutdown_connections 2024-04-25T10:39:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.830+0000 7fc981c6d740 1 --2- 172.21.15.43:0/4011539671 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5593f92e73a0 0x5593f977f860 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.830+0000 7fc981c6d740 1 --2- 172.21.15.43:0/4011539671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f93326b0 0x5593f97162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.830+0000 7fc981c6d740 1 --2- 172.21.15.43:0/4011539671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5593f9353390 0x5593f9775a20 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.830+0000 7fc981c6d740 1 --2- 172.21.15.43:0/4011539671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593f932a4d0 0x5593f9715db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.830+0000 7fc981c6d740 1 -- 172.21.15.43:0/4011539671 >> 172.21.15.43:0/4011539671 conn(0x5593f93474e0 msgr2=0x5593f96efda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.830+0000 7fc953fff640 0 client.17550 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.830+0000 7fc981c6d740 1 -- 172.21.15.43:0/4011539671 shutdown_connections 2024-04-25T10:39:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.830+0000 7fc981c6d740 1 -- 172.21.15.43:0/4011539671 wait complete. 2024-04-25T10:39:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.830+0000 7fc981c6d740 20 client.17550 trim_cache size 0 max 16384 2024-04-25T10:39:39.839 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:39:39.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.830+0000 7fc97adb7640 1 -- 172.21.15.43:0/1693526247 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9600afce0 msgr2=0x7fc9600b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.830+0000 7fc97adb7640 1 --2- 172.21.15.43:0/1693526247 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9600afce0 0x7fc9600b21f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc9640a6ab0 tx=0x7fc96405e040 comp rx=0 tx=0).stop 2024-04-25T10:39:39.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.831+0000 7fc97adb7640 1 -- 172.21.15.43:0/1693526247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc974108eb0 msgr2=0x7fc97419ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:39.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.831+0000 7fc97adb7640 1 --2- 172.21.15.43:0/1693526247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc974108eb0 0x7fc97419ef70 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fc96400d780 tx=0x7fc96400dc40 comp rx=0 tx=0).stop 2024-04-25T10:39:39.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.831+0000 7fc97adb7640 1 -- 172.21.15.43:0/1693526247 shutdown_connections 2024-04-25T10:39:39.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.831+0000 7fc97adb7640 1 --2- 172.21.15.43:0/1693526247 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9600afce0 0x7fc9600b21f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.831+0000 7fc97adb7640 1 --2- 172.21.15.43:0/1693526247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc974108eb0 0x7fc97419ef70 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.831+0000 7fc97adb7640 1 --2- 172.21.15.43:0/1693526247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc974108a10 0x7fc97419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.831+0000 7fc97adb7640 1 --2- 172.21.15.43:0/1693526247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9741031c0 0x7fc97419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:39.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.831+0000 7fc97adb7640 1 -- 172.21.15.43:0/1693526247 >> 172.21.15.43:0/1693526247 conn(0x7fc974078110 msgr2=0x7fc9741040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:39.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.831+0000 7fc97adb7640 1 -- 172.21.15.43:0/1693526247 shutdown_connections 2024-04-25T10:39:39.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:39.831+0000 7fc97adb7640 1 -- 172.21.15.43:0/1693526247 wait complete. 2024-04-25T10:39:40.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:39 smithi077 ceph-mon[32772]: pgmap v219: 97 pgs: 97 active+clean; 3.5 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 17 op/s 2024-04-25T10:39:40.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:39 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/667091700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:39:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:39 smithi164 ceph-mon[32713]: pgmap v219: 97 pgs: 97 active+clean; 3.5 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 17 op/s 2024-04-25T10:39:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:39 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/667091700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:39:40.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:39 smithi043 ceph-mon[29478]: pgmap v219: 97 pgs: 97 active+clean; 3.5 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 17 op/s 2024-04-25T10:39:40.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:39 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/667091700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:39:40.844 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:39:40.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:39:40.844 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:39:40.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.975+0000 7f96db134640 1 -- 172.21.15.43:0/3028566498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d4104260 msgr2=0x7f96d4110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:40.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.975+0000 7f96db134640 1 --2- 172.21.15.43:0/3028566498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d4104260 0x7f96d4110630 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009ca0 tx=0x7f96cc02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:39:40.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.975+0000 7f96db134640 1 -- 172.21.15.43:0/3028566498 shutdown_connections 2024-04-25T10:39:40.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.975+0000 7f96db134640 1 --2- 172.21.15.43:0/3028566498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d4104260 0x7f96d4110630 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:40.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.975+0000 7f96db134640 1 --2- 172.21.15.43:0/3028566498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96d4109fb0 0x7f96d4103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:40.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.975+0000 7f96db134640 1 --2- 172.21.15.43:0/3028566498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96d4109b10 0x7f96d41036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:40.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.975+0000 7f96db134640 1 -- 172.21.15.43:0/3028566498 >> 172.21.15.43:0/3028566498 conn(0x7f96d40ff230 msgr2=0x7f96d4101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:40.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.975+0000 7f96db134640 1 -- 172.21.15.43:0/3028566498 shutdown_connections 2024-04-25T10:39:40.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.975+0000 7f96db134640 1 -- 172.21.15.43:0/3028566498 wait complete. 2024-04-25T10:39:40.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.976+0000 7f96db134640 1 Processor -- start 2024-04-25T10:39:40.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.976+0000 7f96db134640 1 -- start start 2024-04-25T10:39:40.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.976+0000 7f96db134640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96d4104260 0x7f96d419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:40.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.976+0000 7f96db134640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d4109b10 0x7f96d419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:40.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.976+0000 7f96db134640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96d4109fb0 0x7f96d419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:40.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.976+0000 7f96db134640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f96d4113d00 con 0x7f96d4109b10 2024-04-25T10:39:40.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.976+0000 7f96db134640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f96d4113b80 con 0x7f96d4109fb0 2024-04-25T10:39:40.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.976+0000 7f96db134640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f96d4113e80 con 0x7f96d4104260 2024-04-25T10:39:40.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.976+0000 7f96d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d4109b10 0x7f96d419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:40.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.976+0000 7f96d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d4109b10 0x7f96d419ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40112/0 (socket says 172.21.15.43:40112) 2024-04-25T10:39:40.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.976+0000 7f96d3fff640 1 -- 172.21.15.43:0/3147203594 learned_addr learned my addr 172.21.15.43:0/3147203594 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:40.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.976+0000 7f96d8ed0640 1 --2- 172.21.15.43:0/3147203594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96d4104260 0x7f96d419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:40.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.976+0000 7f96d96d1640 1 --2- 172.21.15.43:0/3147203594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96d4109fb0 0x7f96d419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:40.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.976+0000 7f96d3fff640 1 -- 172.21.15.43:0/3147203594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96d4104260 msgr2=0x7f96d419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:40.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.976+0000 7f96d3fff640 1 --2- 172.21.15.43:0/3147203594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96d4104260 0x7f96d419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:40.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.976+0000 7f96d3fff640 1 -- 172.21.15.43:0/3147203594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96d4109fb0 msgr2=0x7f96d419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:40.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.976+0000 7f96d3fff640 1 --2- 172.21.15.43:0/3147203594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96d4109fb0 0x7f96d419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:40.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.976+0000 7f96d3fff640 1 -- 172.21.15.43:0/3147203594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d419f520 con 0x7f96d4109b10 2024-04-25T10:39:40.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.977+0000 7f96d8ed0640 1 --2- 172.21.15.43:0/3147203594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96d4104260 0x7f96d419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:39:40.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.977+0000 7f96d96d1640 1 --2- 172.21.15.43:0/3147203594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96d4109fb0 0x7f96d419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:39:40.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.977+0000 7f96d3fff640 1 --2- 172.21.15.43:0/3147203594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d4109b10 0x7f96d419ac20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f96c800d740 tx=0x7f96c800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:40.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.977+0000 7f96d1ffb640 1 -- 172.21.15.43:0/3147203594 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8017070 con 0x7f96d4109b10 2024-04-25T10:39:40.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.977+0000 7f96d1ffb640 1 -- 172.21.15.43:0/3147203594 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96c800de50 con 0x7f96d4109b10 2024-04-25T10:39:40.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.977+0000 7f96db134640 1 -- 172.21.15.43:0/3147203594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d419f810 con 0x7f96d4109b10 2024-04-25T10:39:40.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.977+0000 7f96d1ffb640 1 -- 172.21.15.43:0/3147203594 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8003c20 con 0x7f96d4109b10 2024-04-25T10:39:40.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.977+0000 7f96db134640 1 -- 172.21.15.43:0/3147203594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d41a7190 con 0x7f96d4109b10 2024-04-25T10:39:40.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.980+0000 7f96d1ffb640 1 -- 172.21.15.43:0/3147203594 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f96c8003dc0 con 0x7f96d4109b10 2024-04-25T10:39:40.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.980+0000 7f96e1fe6740 1 Processor -- start 2024-04-25T10:39:40.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.980+0000 7f96e1fe6740 1 -- start start 2024-04-25T10:39:40.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.980+0000 7f96e1fe6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa8e410 0x5623cae2e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:40.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.980+0000 7f96e1fe6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623caa65550 0x5623cae2eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:40.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.980+0000 7f96e1fe6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623caa6d730 0x5623cae2f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:40.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.980+0000 7f96e1fe6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5623cad94b00 con 0x5623caa8e410 2024-04-25T10:39:40.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.980+0000 7f96e1fe6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5623cad30920 con 0x5623caa6d730 2024-04-25T10:39:40.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.980+0000 7f96e1fe6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5623cad90ac0 con 0x5623caa65550 2024-04-25T10:39:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.980+0000 7f96d1ffb640 1 --2- 172.21.15.43:0/3147203594 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f96bc0afc60 0x7f96bc0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.980+0000 7f96d96d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa8e410 0x5623cae2e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.980+0000 7f96d96d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa8e410 0x5623cae2e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40120/0 (socket says 172.21.15.43:40120) 2024-04-25T10:39:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.980+0000 7f96d96d1640 1 -- 172.21.15.43:0/3329368536 learned_addr learned my addr 172.21.15.43:0/3329368536 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.980+0000 7f96d8ed0640 1 --2- 172.21.15.43:0/3329368536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623caa65550 0x5623cae2eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.981+0000 7f96d96d1640 1 --2- 172.21.15.43:0/3147203594 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f96bc0afc60 0x7f96bc0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.981+0000 7f96d3fff640 1 --2- 172.21.15.43:0/3329368536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623caa6d730 0x5623cae2f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.981+0000 7f96d1ffb640 1 -- 172.21.15.43:0/3147203594 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f96c80dead0 con 0x7f96d4109b10 2024-04-25T10:39:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.981+0000 7f96d96d1640 1 -- 172.21.15.43:0/3329368536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623caa65550 msgr2=0x5623cae2eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.981+0000 7f96d96d1640 1 --2- 172.21.15.43:0/3329368536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623caa65550 0x5623cae2eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.981+0000 7f96d96d1640 1 -- 172.21.15.43:0/3329368536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623caa6d730 msgr2=0x5623cae2f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.981+0000 7f96d96d1640 1 --2- 172.21.15.43:0/3329368536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623caa6d730 0x5623cae2f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.981+0000 7f96d96d1640 1 -- 172.21.15.43:0/3329368536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623cab6e7e0 con 0x5623caa8e410 2024-04-25T10:39:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.981+0000 7f96d96d1640 1 --2- 172.21.15.43:0/3329368536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa8e410 0x5623cae2e620 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0094e0 tx=0x7f96cc0314d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d23e7392bdda0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.981+0000 7f96d96d1640 1 --2- 172.21.15.43:0/3147203594 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f96bc0afc60 0x7f96bc0b2170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f96cc031540 tx=0x7f96cc042000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.981+0000 7f96baffd640 1 -- 172.21.15.43:0/3329368536 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc04e070 con 0x5623caa8e410 2024-04-25T10:39:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.981+0000 7f96baffd640 1 -- 172.21.15.43:0/3329368536 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96cc036050 con 0x5623caa8e410 2024-04-25T10:39:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.981+0000 7f96baffd640 1 -- 172.21.15.43:0/3329368536 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc004730 con 0x5623caa8e410 2024-04-25T10:39:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.982+0000 7f96e1fe6740 1 -- 172.21.15.43:0/3329368536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa8e410 msgr2=0x5623cae2e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.982+0000 7f96e1fe6740 1 --2- 172.21.15.43:0/3329368536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa8e410 0x5623cae2e620 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0094e0 tx=0x7f96cc0314d0 comp rx=0 tx=0).stop 2024-04-25T10:39:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.982+0000 7f96e1fe6740 1 -- 172.21.15.43:0/3329368536 shutdown_connections 2024-04-25T10:39:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.982+0000 7f96e1fe6740 1 --2- 172.21.15.43:0/3329368536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623caa6d730 0x5623cae2f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.982+0000 7f96e1fe6740 1 --2- 172.21.15.43:0/3329368536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623caa65550 0x5623cae2eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.982+0000 7f96e1fe6740 1 --2- 172.21.15.43:0/3329368536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa8e410 0x5623cae2e620 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.982+0000 7f96e1fe6740 1 -- 172.21.15.43:0/3329368536 >> 172.21.15.43:0/3329368536 conn(0x5623caa82560 msgr2=0x5623cae2add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.982+0000 7f96e1fe6740 1 -- 172.21.15.43:0/3329368536 shutdown_connections 2024-04-25T10:39:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.982+0000 7f96e1fe6740 1 -- 172.21.15.43:0/3329368536 wait complete. 2024-04-25T10:39:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.983+0000 7f96e1fe6740 1 -- start start 2024-04-25T10:39:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.983+0000 7f96e1fe6740 1 Processor -- start 2024-04-25T10:39:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.983+0000 7f96e1fe6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623caa65550 0x5623cae38cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.983+0000 7f96e1fe6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623caa6d730 0x5623cae391f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.983+0000 7f96e1fe6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa8e410 0x5623cae3ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.983+0000 7f96e1fe6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5623cae3b910 con 0x5623caa8e410 2024-04-25T10:39:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.983+0000 7f96e1fe6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5623cad2e5d0 con 0x5623caa65550 2024-04-25T10:39:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.983+0000 7f96e1fe6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5623cae3ba90 con 0x5623caa6d730 2024-04-25T10:39:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.983+0000 7f96e1fe6740 10 client.0 _ll_register_callbacks cb 0x5623cad1c080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.983+0000 7f96d96d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa8e410 0x5623cae3ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.984+0000 7f96d96d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa8e410 0x5623cae3ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40130/0 (socket says 172.21.15.43:40130) 2024-04-25T10:39:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.984+0000 7f96d96d1640 1 -- 172.21.15.43:0/2230435657 learned_addr learned my addr 172.21.15.43:0/2230435657 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.984+0000 7f96d8ed0640 1 --2- 172.21.15.43:0/2230435657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623caa65550 0x5623cae38cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.984+0000 7f96d3fff640 1 --2- 172.21.15.43:0/2230435657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623caa6d730 0x5623cae391f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.984+0000 7f96d96d1640 1 -- 172.21.15.43:0/2230435657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623caa6d730 msgr2=0x5623cae391f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.984+0000 7f96d96d1640 1 --2- 172.21.15.43:0/2230435657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623caa6d730 0x5623cae391f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.984+0000 7f96d96d1640 1 -- 172.21.15.43:0/2230435657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623caa65550 msgr2=0x5623cae38cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.984+0000 7f96d96d1640 1 --2- 172.21.15.43:0/2230435657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623caa65550 0x5623cae38cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:40.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.984+0000 7f96d96d1640 1 -- 172.21.15.43:0/2230435657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623cabcef70 con 0x5623caa8e410 2024-04-25T10:39:40.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.984+0000 7f96d96d1640 1 --2- 172.21.15.43:0/2230435657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa8e410 0x5623cae3ed90 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f96cc003d40 tx=0x7f96cc0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7deb4b67cdc59a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:40.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.984+0000 7f96b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:40.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.984+0000 7f96b3fff640 1 -- 172.21.15.43:0/2230435657 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc04e070 con 0x5623caa8e410 2024-04-25T10:39:40.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.984+0000 7f96b3fff640 1 -- 172.21.15.43:0/2230435657 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f96cc0046b0 con 0x5623caa8e410 2024-04-25T10:39:40.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.984+0000 7f96e1fe6740 1 -- 172.21.15.43:0/2230435657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96cc036050 con 0x5623caa8e410 2024-04-25T10:39:40.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.984+0000 7f96b3fff640 1 -- 172.21.15.43:0/2230435657 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f96cc061050 con 0x5623caa8e410 2024-04-25T10:39:40.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.984+0000 7f96e1fe6740 10 client.17562 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:40.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.984+0000 7f96e1fe6740 1 -- 172.21.15.43:0/2230435657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5623caa66170 con 0x5623caa8e410 2024-04-25T10:39:40.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.985+0000 7f96b3fff640 1 -- 172.21.15.43:0/2230435657 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96cc036050 con 0x5623caa8e410 2024-04-25T10:39:40.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.985+0000 7f96b3fff640 1 -- 172.21.15.43:0/2230435657 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f96cc0305c0 con 0x5623caa8e410 2024-04-25T10:39:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.985+0000 7f96e1fe6740 10 client.17562 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.985+0000 7f96e1fe6740 10 client.17562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.985+0000 7f96e1fe6740 20 client.17562 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.985+0000 7f96e1fe6740 1 --2- 172.21.15.43:0/2230435657 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5623caa4d1f0 0x5623cae4ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.985+0000 7f96e1fe6740 4 client.17562 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:39:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.985+0000 7f96e1fe6740 1 -- 172.21.15.43:0/2230435657 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623cada2710 con 0x5623caa4d1f0 2024-04-25T10:39:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.985+0000 7f96d8ed0640 1 --2- 172.21.15.43:0/2230435657 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5623caa4d1f0 0x5623cae4ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.986+0000 7f96b3fff640 10 client.17562 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.986+0000 7f96d8ed0640 1 --2- 172.21.15.43:0/2230435657 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5623caa4d1f0 0x5623cae4ae80 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3f8ab1745714117 server_cookie=3e796eca32cc696b in_seq=0 out_seq=0 2024-04-25T10:39:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.987+0000 7f96b3fff640 1 -- 172.21.15.43:0/2230435657 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5623cada2710 con 0x5623caa4d1f0 2024-04-25T10:39:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.987+0000 7f96b3fff640 10 client.17562 handle_command_reply: tid=0 2024-04-25T10:39:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.987+0000 7f96e1fe6740 1 client.17562 shutdown 2024-04-25T10:39:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.987+0000 7f96e1fe6740 1 -- 172.21.15.43:0/2230435657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa8e410 msgr2=0x5623cae3ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.987+0000 7f96e1fe6740 1 --2- 172.21.15.43:0/2230435657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa8e410 0x5623cae3ed90 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f96cc003d40 tx=0x7f96cc0317b0 comp rx=0 tx=0).stop 2024-04-25T10:39:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.987+0000 7f96e1fe6740 1 -- 172.21.15.43:0/2230435657 shutdown_connections 2024-04-25T10:39:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.987+0000 7f96e1fe6740 1 --2- 172.21.15.43:0/2230435657 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5623caa4d1f0 0x5623cae4ae80 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.987+0000 7f96e1fe6740 1 --2- 172.21.15.43:0/2230435657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa8e410 0x5623cae3ed90 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.987+0000 7f96b3fff640 0 client.17562 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.987+0000 7f96e1fe6740 1 --2- 172.21.15.43:0/2230435657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623caa6d730 0x5623cae391f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.987+0000 7f96e1fe6740 1 --2- 172.21.15.43:0/2230435657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623caa65550 0x5623cae38cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.987+0000 7f96e1fe6740 1 -- 172.21.15.43:0/2230435657 >> 172.21.15.43:0/2230435657 conn(0x5623caa82560 msgr2=0x5623cae2e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.987+0000 7f96e1fe6740 1 -- 172.21.15.43:0/2230435657 shutdown_connections 2024-04-25T10:39:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.987+0000 7f96e1fe6740 1 -- 172.21.15.43:0/2230435657 wait complete. 2024-04-25T10:39:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:40.987+0000 7f96e1fe6740 20 client.17562 trim_cache size 0 max 16384 2024-04-25T10:39:41.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.005+0000 7f96e1fe6740 1 Processor -- start 2024-04-25T10:39:41.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.005+0000 7f96e1fe6740 1 -- start start 2024-04-25T10:39:41.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.006+0000 7f96e1fe6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa65550 0x5623cae327e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.006+0000 7f96e1fe6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623caa6d730 0x5623cae3ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.006+0000 7f96e1fe6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623caa8e410 0x5623cae3f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.006+0000 7f96e1fe6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f96d41b2270 con 0x5623caa65550 2024-04-25T10:39:41.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.006+0000 7f96e1fe6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f96d41afb40 con 0x5623caa8e410 2024-04-25T10:39:41.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.006+0000 7f96e1fe6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f96d41ad9b0 con 0x5623caa6d730 2024-04-25T10:39:41.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.006+0000 7f96d8ed0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa65550 0x5623cae327e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.006+0000 7f96d8ed0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa65550 0x5623cae327e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40134/0 (socket says 172.21.15.43:40134) 2024-04-25T10:39:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.006+0000 7f96d8ed0640 1 -- 172.21.15.43:0/473207326 learned_addr learned my addr 172.21.15.43:0/473207326 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.006+0000 7f96d3fff640 1 --2- 172.21.15.43:0/473207326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623caa6d730 0x5623cae3ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.006+0000 7f96d96d1640 1 --2- 172.21.15.43:0/473207326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623caa8e410 0x5623cae3f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.006+0000 7f96d8ed0640 1 -- 172.21.15.43:0/473207326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623caa6d730 msgr2=0x5623cae3ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.006+0000 7f96d8ed0640 1 --2- 172.21.15.43:0/473207326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623caa6d730 0x5623cae3ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.006+0000 7f96d8ed0640 1 -- 172.21.15.43:0/473207326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623caa8e410 msgr2=0x5623cae3f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.006+0000 7f96d8ed0640 1 --2- 172.21.15.43:0/473207326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623caa8e410 0x5623cae3f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.006+0000 7f96d8ed0640 1 -- 172.21.15.43:0/473207326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623cae2f990 con 0x5623caa65550 2024-04-25T10:39:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.006+0000 7f96d8ed0640 1 --2- 172.21.15.43:0/473207326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa65550 0x5623cae327e0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f96c4000c00 tx=0x7f96c4012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4808212c7b54acd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.006+0000 7f96ba7fc640 1 -- 172.21.15.43:0/473207326 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4022020 con 0x5623caa65550 2024-04-25T10:39:41.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.006+0000 7f96ba7fc640 1 -- 172.21.15.43:0/473207326 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96c4009e30 con 0x5623caa65550 2024-04-25T10:39:41.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.006+0000 7f96ba7fc640 1 -- 172.21.15.43:0/473207326 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4011e10 con 0x5623caa65550 2024-04-25T10:39:41.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.007+0000 7f96e1fe6740 1 -- 172.21.15.43:0/473207326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa65550 msgr2=0x5623cae327e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:41.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.007+0000 7f96e1fe6740 1 --2- 172.21.15.43:0/473207326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa65550 0x5623cae327e0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f96c4000c00 tx=0x7f96c4012820 comp rx=0 tx=0).stop 2024-04-25T10:39:41.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.007+0000 7f96e1fe6740 1 -- 172.21.15.43:0/473207326 shutdown_connections 2024-04-25T10:39:41.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.007+0000 7f96e1fe6740 1 --2- 172.21.15.43:0/473207326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623caa8e410 0x5623cae3f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.007+0000 7f96e1fe6740 1 --2- 172.21.15.43:0/473207326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623caa6d730 0x5623cae3ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.007+0000 7f96e1fe6740 1 --2- 172.21.15.43:0/473207326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa65550 0x5623cae327e0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.007+0000 7f96e1fe6740 1 -- 172.21.15.43:0/473207326 >> 172.21.15.43:0/473207326 conn(0x5623caa82560 msgr2=0x5623cae320d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:41.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.007+0000 7f96e1fe6740 1 -- 172.21.15.43:0/473207326 shutdown_connections 2024-04-25T10:39:41.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.007+0000 7f96e1fe6740 1 -- 172.21.15.43:0/473207326 wait complete. 2024-04-25T10:39:41.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.008+0000 7f96e1fe6740 1 -- start start 2024-04-25T10:39:41.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.008+0000 7f96e1fe6740 1 Processor -- start 2024-04-25T10:39:41.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.008+0000 7f96e1fe6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623caa65550 0x5623cae44d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.008+0000 7f96e1fe6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa6d730 0x5623cae420c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.008+0000 7f96e1fe6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623caa8e410 0x5623cae42600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.008+0000 7f96e1fe6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f96d41ae000 con 0x5623caa6d730 2024-04-25T10:39:41.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.008+0000 7f96e1fe6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f96d41af470 con 0x5623caa65550 2024-04-25T10:39:41.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.008+0000 7f96e1fe6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f96d41ae6d0 con 0x5623caa8e410 2024-04-25T10:39:41.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.008+0000 7f96e1fe6740 10 client.0 _ll_register_callbacks cb 0x7f96cc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:41.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.008+0000 7f96d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa6d730 0x5623cae420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.008+0000 7f96d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa6d730 0x5623cae420c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40148/0 (socket says 172.21.15.43:40148) 2024-04-25T10:39:41.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.008+0000 7f96d3fff640 1 -- 172.21.15.43:0/1584935005 learned_addr learned my addr 172.21.15.43:0/1584935005 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:41.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.008+0000 7f96d8ed0640 1 --2- 172.21.15.43:0/1584935005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623caa65550 0x5623cae44d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.009+0000 7f96d96d1640 1 --2- 172.21.15.43:0/1584935005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623caa8e410 0x5623cae42600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.009+0000 7f96d3fff640 1 -- 172.21.15.43:0/1584935005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623caa8e410 msgr2=0x5623cae42600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:41.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.009+0000 7f96d3fff640 1 --2- 172.21.15.43:0/1584935005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623caa8e410 0x5623cae42600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.009+0000 7f96d3fff640 1 -- 172.21.15.43:0/1584935005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623caa65550 msgr2=0x5623cae44d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:41.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.009+0000 7f96d3fff640 1 --2- 172.21.15.43:0/1584935005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623caa65550 0x5623cae44d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.009+0000 7f96d3fff640 1 -- 172.21.15.43:0/1584935005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623cae38ac0 con 0x5623caa6d730 2024-04-25T10:39:41.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.009+0000 7f96d3fff640 1 --2- 172.21.15.43:0/1584935005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa6d730 0x5623cae420c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f96c80a6970 tx=0x7f96c8020c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae720a4cc85a502 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:41.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.009+0000 7f96b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:41.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.009+0000 7f96b3fff640 1 -- 172.21.15.43:0/1584935005 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8020040 con 0x5623caa6d730 2024-04-25T10:39:41.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.009+0000 7f96b3fff640 1 -- 172.21.15.43:0/1584935005 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f96c80a2f00 con 0x5623caa6d730 2024-04-25T10:39:41.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.009+0000 7f96e1fe6740 1 -- 172.21.15.43:0/1584935005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96c4009e30 con 0x5623caa6d730 2024-04-25T10:39:41.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.009+0000 7f96b3fff640 1 -- 172.21.15.43:0/1584935005 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f96c80a7370 con 0x5623caa6d730 2024-04-25T10:39:41.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.009+0000 7f96e1fe6740 10 client.17568 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:41.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.009+0000 7f96e1fe6740 1 -- 172.21.15.43:0/1584935005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5623caeb2a70 con 0x5623caa6d730 2024-04-25T10:39:41.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.009+0000 7f96b3fff640 1 -- 172.21.15.43:0/1584935005 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96c4009e30 con 0x5623caa6d730 2024-04-25T10:39:41.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.009+0000 7f96b3fff640 1 -- 172.21.15.43:0/1584935005 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f96c8020cc0 con 0x5623caa6d730 2024-04-25T10:39:41.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.009+0000 7f96e1fe6740 10 client.17568 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:41.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.009+0000 7f96e1fe6740 10 client.17568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:41.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.009+0000 7f96e1fe6740 20 client.17568 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:41.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.010+0000 7f96e1fe6740 1 --2- 172.21.15.43:0/1584935005 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5623caa22630 0x5623caeb5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.010+0000 7f96e1fe6740 4 client.17568 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:39:41.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.010+0000 7f96e1fe6740 1 -- 172.21.15.43:0/1584935005 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5623cad9c580 con 0x5623caa22630 2024-04-25T10:39:41.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.010+0000 7f96d8ed0640 1 --2- 172.21.15.43:0/1584935005 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5623caa22630 0x5623caeb5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.010+0000 7f96b3fff640 10 client.17568 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:41.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.010+0000 7f96d8ed0640 1 --2- 172.21.15.43:0/1584935005 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5623caa22630 0x5623caeb5230 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63da542f16321141 server_cookie=fe0cdd8a8519c043 in_seq=0 out_seq=0 2024-04-25T10:39:41.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.011+0000 7f96b3fff640 1 -- 172.21.15.43:0/1584935005 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5623cad9c580 con 0x5623caa22630 2024-04-25T10:39:41.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.011+0000 7f96b3fff640 10 client.17568 handle_command_reply: tid=0 2024-04-25T10:39:41.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.011+0000 7f96e1fe6740 1 client.17568 shutdown 2024-04-25T10:39:41.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.011+0000 7f96e1fe6740 1 -- 172.21.15.43:0/1584935005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa6d730 msgr2=0x5623cae420c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:41.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96e1fe6740 1 --2- 172.21.15.43:0/1584935005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa6d730 0x5623cae420c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f96c80a6970 tx=0x7f96c8020c50 comp rx=0 tx=0).stop 2024-04-25T10:39:41.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96e1fe6740 1 -- 172.21.15.43:0/1584935005 shutdown_connections 2024-04-25T10:39:41.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96e1fe6740 1 --2- 172.21.15.43:0/1584935005 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5623caa22630 0x5623caeb5230 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96b3fff640 0 client.17568 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:41.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96e1fe6740 1 --2- 172.21.15.43:0/1584935005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623caa8e410 0x5623cae42600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96e1fe6740 1 --2- 172.21.15.43:0/1584935005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623caa6d730 0x5623cae420c0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96e1fe6740 1 --2- 172.21.15.43:0/1584935005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623caa65550 0x5623cae44d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96e1fe6740 1 -- 172.21.15.43:0/1584935005 >> 172.21.15.43:0/1584935005 conn(0x5623caa82560 msgr2=0x5623cae294c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:41.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96e1fe6740 1 -- 172.21.15.43:0/1584935005 shutdown_connections 2024-04-25T10:39:41.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96e1fe6740 1 -- 172.21.15.43:0/1584935005 wait complete. 2024-04-25T10:39:41.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96e1fe6740 20 client.17568 trim_cache size 0 max 16384 2024-04-25T10:39:41.021 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:39:41.021 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:39:41.021 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "8f6837ec-74c7-4fe7-9261-43ce98d13d88", 2024-04-25T10:39:41.021 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:39:41.021 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:39:41.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96db134640 1 -- 172.21.15.43:0/3147203594 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f96bc0afc60 msgr2=0x7f96bc0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:41.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96db134640 1 --2- 172.21.15.43:0/3147203594 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f96bc0afc60 0x7f96bc0b2170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f96cc031540 tx=0x7f96cc042000 comp rx=0 tx=0).stop 2024-04-25T10:39:41.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96db134640 1 -- 172.21.15.43:0/3147203594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d4109b10 msgr2=0x7f96d419ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:41.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96db134640 1 --2- 172.21.15.43:0/3147203594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d4109b10 0x7f96d419ac20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f96c800d740 tx=0x7f96c800dc00 comp rx=0 tx=0).stop 2024-04-25T10:39:41.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96db134640 1 -- 172.21.15.43:0/3147203594 shutdown_connections 2024-04-25T10:39:41.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96db134640 1 --2- 172.21.15.43:0/3147203594 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f96bc0afc60 0x7f96bc0b2170 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96db134640 1 --2- 172.21.15.43:0/3147203594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96d4109fb0 0x7f96d419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96db134640 1 --2- 172.21.15.43:0/3147203594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d4109b10 0x7f96d419ac20 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96db134640 1 --2- 172.21.15.43:0/3147203594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96d4104260 0x7f96d419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.012+0000 7f96db134640 1 -- 172.21.15.43:0/3147203594 >> 172.21.15.43:0/3147203594 conn(0x7f96d40ff230 msgr2=0x7f96d4106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:41.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.013+0000 7f96db134640 1 -- 172.21.15.43:0/3147203594 shutdown_connections 2024-04-25T10:39:41.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.013+0000 7f96db134640 1 -- 172.21.15.43:0/3147203594 wait complete. 2024-04-25T10:39:41.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:39:41.026 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:39:41.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.138+0000 7f80f59c5640 1 -- 172.21.15.43:0/1347863670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80f0108e50 msgr2=0x7f80f0103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:41.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.138+0000 7f80f59c5640 1 --2- 172.21.15.43:0/1347863670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80f0108e50 0x7f80f0103300 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009900 tx=0x7f80e402a480 comp rx=0 tx=0).stop 2024-04-25T10:39:41.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.138+0000 7f80f59c5640 1 -- 172.21.15.43:0/1347863670 shutdown_connections 2024-04-25T10:39:41.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.138+0000 7f80f59c5640 1 --2- 172.21.15.43:0/1347863670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80f0103990 0x7f80f010f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.138+0000 7f80f59c5640 1 --2- 172.21.15.43:0/1347863670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80f0108e50 0x7f80f0103300 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.138+0000 7f80f59c5640 1 --2- 172.21.15.43:0/1347863670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80f0108a40 0x7f80f0102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.138+0000 7f80f59c5640 1 -- 172.21.15.43:0/1347863670 >> 172.21.15.43:0/1347863670 conn(0x7f80f0077880 msgr2=0x7f80f0077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:41.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.139+0000 7f80f59c5640 1 -- 172.21.15.43:0/1347863670 shutdown_connections 2024-04-25T10:39:41.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.139+0000 7f80f59c5640 1 -- 172.21.15.43:0/1347863670 wait complete. 2024-04-25T10:39:41.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.139+0000 7f80f59c5640 1 Processor -- start 2024-04-25T10:39:41.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.139+0000 7f80f59c5640 1 -- start start 2024-04-25T10:39:41.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80f59c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80f0108a40 0x7f80f019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80f59c5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80f0108e50 0x7f80f019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80f59c5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80f019ebd0 0x7f80f019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80f59c5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f80f0112c10 con 0x7f80f0108a40 2024-04-25T10:39:41.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80f59c5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f80f0112a90 con 0x7f80f0108e50 2024-04-25T10:39:41.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80f59c5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f80f0112d90 con 0x7f80f019ebd0 2024-04-25T10:39:41.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80eeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80f0108a40 0x7f80f019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80eeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80f0108a40 0x7f80f019a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40174/0 (socket says 172.21.15.43:40174) 2024-04-25T10:39:41.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80eeffd640 1 -- 172.21.15.43:0/143570954 learned_addr learned my addr 172.21.15.43:0/143570954 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:41.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80ef7fe640 1 --2- 172.21.15.43:0/143570954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80f019ebd0 0x7f80f019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80ee7fc640 1 --2- 172.21.15.43:0/143570954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80f0108e50 0x7f80f019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80eeffd640 1 -- 172.21.15.43:0/143570954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80f019ebd0 msgr2=0x7f80f019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:41.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80eeffd640 1 --2- 172.21.15.43:0/143570954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80f019ebd0 0x7f80f019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80eeffd640 1 -- 172.21.15.43:0/143570954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80f0108e50 msgr2=0x7f80f019a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:41.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80eeffd640 1 --2- 172.21.15.43:0/143570954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80f0108e50 0x7f80f019a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80eeffd640 1 -- 172.21.15.43:0/143570954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f019f620 con 0x7f80f0108a40 2024-04-25T10:39:41.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80eeffd640 1 --2- 172.21.15.43:0/143570954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80f0108a40 0x7f80f019a3b0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f80e000d740 tx=0x7f80e000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:41.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80f49c3640 1 -- 172.21.15.43:0/143570954 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0017070 con 0x7f80f0108a40 2024-04-25T10:39:41.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80f59c5640 1 -- 172.21.15.43:0/143570954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f019f910 con 0x7f80f0108a40 2024-04-25T10:39:41.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80f49c3640 1 -- 172.21.15.43:0/143570954 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80e000de50 con 0x7f80f0108a40 2024-04-25T10:39:41.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.140+0000 7f80f49c3640 1 -- 172.21.15.43:0/143570954 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0004ce0 con 0x7f80f0108a40 2024-04-25T10:39:41.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.141+0000 7f80f59c5640 1 -- 172.21.15.43:0/143570954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f01a7110 con 0x7f80f0108a40 2024-04-25T10:39:41.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.143+0000 7f80f49c3640 1 -- 172.21.15.43:0/143570954 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f80e0004e80 con 0x7f80f0108a40 2024-04-25T10:39:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.143+0000 7f80fc87c740 1 Processor -- start 2024-04-25T10:39:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.143+0000 7f80fc87c740 1 -- start start 2024-04-25T10:39:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.143+0000 7f80fc87c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559454092390 0x559454432a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.143+0000 7f80fc87c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594540694d0 0x559454432fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.144+0000 7f80fc87c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5594540716b0 0x55945443f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.144+0000 7f80fc87c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559454398d80 con 0x5594540694d0 2024-04-25T10:39:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.144+0000 7f80fc87c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5594543528b0 con 0x5594540716b0 2024-04-25T10:39:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.144+0000 7f80fc87c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559454394d40 con 0x559454092390 2024-04-25T10:39:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.144+0000 7f80eeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594540694d0 0x559454432fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.144+0000 7f80eeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594540694d0 0x559454432fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40186/0 (socket says 172.21.15.43:40186) 2024-04-25T10:39:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.144+0000 7f80eeffd640 1 -- 172.21.15.43:0/4045790968 learned_addr learned my addr 172.21.15.43:0/4045790968 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.144+0000 7f80f49c3640 1 --2- 172.21.15.43:0/143570954 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f80d80afc60 0x7f80d80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.144+0000 7f80ee7fc640 1 --2- 172.21.15.43:0/4045790968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5594540716b0 0x55945443f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.144+0000 7f80ef7fe640 1 --2- 172.21.15.43:0/4045790968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559454092390 0x559454432a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.144+0000 7f80ef7fe640 1 --2- 172.21.15.43:0/143570954 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f80d80afc60 0x7f80d80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.144+0000 7f80f49c3640 1 -- 172.21.15.43:0/143570954 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f80e00de8d0 con 0x7f80f0108a40 2024-04-25T10:39:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.144+0000 7f80eeffd640 1 -- 172.21.15.43:0/4045790968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559454092390 msgr2=0x559454432a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.144+0000 7f80eeffd640 1 --2- 172.21.15.43:0/4045790968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559454092390 0x559454432a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.144+0000 7f80eeffd640 1 -- 172.21.15.43:0/4045790968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5594540716b0 msgr2=0x55945443f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.144+0000 7f80eeffd640 1 --2- 172.21.15.43:0/4045790968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5594540716b0 0x55945443f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.144+0000 7f80eeffd640 1 -- 172.21.15.43:0/4045790968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559454323a40 con 0x5594540694d0 2024-04-25T10:39:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.144+0000 7f80ee7fc640 1 --2- 172.21.15.43:0/4045790968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5594540716b0 0x55945443f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:39:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.144+0000 7f80eeffd640 1 --2- 172.21.15.43:0/4045790968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594540694d0 0x559454432fa0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f80e00a6820 tx=0x7f80e00a6910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e223cc8bb15d4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.144+0000 7f80ef7fe640 1 --2- 172.21.15.43:0/143570954 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f80d80afc60 0x7f80d80b2170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55945442fbd0 tx=0x7f80e80096a0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.145+0000 7f80d5ffb640 1 -- 172.21.15.43:0/4045790968 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0020070 con 0x5594540694d0 2024-04-25T10:39:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.145+0000 7f80d5ffb640 1 -- 172.21.15.43:0/4045790968 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80e00a6630 con 0x5594540694d0 2024-04-25T10:39:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.145+0000 7f80d5ffb640 1 -- 172.21.15.43:0/4045790968 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e00dc3f0 con 0x5594540694d0 2024-04-25T10:39:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.145+0000 7f80fc87c740 1 -- 172.21.15.43:0/4045790968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594540694d0 msgr2=0x559454432fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.145+0000 7f80fc87c740 1 --2- 172.21.15.43:0/4045790968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594540694d0 0x559454432fa0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f80e00a6820 tx=0x7f80e00a6910 comp rx=0 tx=0).stop 2024-04-25T10:39:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.145+0000 7f80fc87c740 1 -- 172.21.15.43:0/4045790968 shutdown_connections 2024-04-25T10:39:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.145+0000 7f80fc87c740 1 --2- 172.21.15.43:0/4045790968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5594540716b0 0x55945443f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.145+0000 7f80fc87c740 1 --2- 172.21.15.43:0/4045790968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594540694d0 0x559454432fa0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.145+0000 7f80fc87c740 1 --2- 172.21.15.43:0/4045790968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559454092390 0x559454432a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.145+0000 7f80fc87c740 1 -- 172.21.15.43:0/4045790968 >> 172.21.15.43:0/4045790968 conn(0x5594540864e0 msgr2=0x55945442f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.145+0000 7f80fc87c740 1 -- 172.21.15.43:0/4045790968 shutdown_connections 2024-04-25T10:39:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.145+0000 7f80fc87c740 1 -- 172.21.15.43:0/4045790968 wait complete. 2024-04-25T10:39:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.146+0000 7f80fc87c740 1 -- start start 2024-04-25T10:39:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.146+0000 7f80fc87c740 1 Processor -- start 2024-04-25T10:39:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.146+0000 7f80fc87c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594540694d0 0x55945443efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.146+0000 7f80fc87c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594540716b0 0x55945443f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.146+0000 7f80fc87c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559454092390 0x55945443d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.146+0000 7f80fc87c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55945443ff20 con 0x5594540694d0 2024-04-25T10:39:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.146+0000 7f80fc87c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55945431fd10 con 0x559454092390 2024-04-25T10:39:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.146+0000 7f80fc87c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5594544400a0 con 0x5594540716b0 2024-04-25T10:39:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.146+0000 7f80fc87c740 10 client.0 _ll_register_callbacks cb 0x55945434c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.146+0000 7f80eeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594540694d0 0x55945443efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.147+0000 7f80eeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594540694d0 0x55945443efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40200/0 (socket says 172.21.15.43:40200) 2024-04-25T10:39:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.147+0000 7f80eeffd640 1 -- 172.21.15.43:0/3907797934 learned_addr learned my addr 172.21.15.43:0/3907797934 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.147+0000 7f80ee7fc640 1 --2- 172.21.15.43:0/3907797934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594540716b0 0x55945443f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.147+0000 7f80ef7fe640 1 --2- 172.21.15.43:0/3907797934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559454092390 0x55945443d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.147+0000 7f80eeffd640 1 -- 172.21.15.43:0/3907797934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594540716b0 msgr2=0x55945443f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.147+0000 7f80eeffd640 1 --2- 172.21.15.43:0/3907797934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594540716b0 0x55945443f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.147+0000 7f80eeffd640 1 -- 172.21.15.43:0/3907797934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559454092390 msgr2=0x55945443d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.147+0000 7f80eeffd640 1 --2- 172.21.15.43:0/3907797934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559454092390 0x55945443d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.147+0000 7f80eeffd640 1 -- 172.21.15.43:0/3907797934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594541747d0 con 0x5594540694d0 2024-04-25T10:39:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.147+0000 7f80eeffd640 1 --2- 172.21.15.43:0/3907797934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594540694d0 0x55945443efc0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f80e00dc0d0 tx=0x7f80e00a6390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba449050edc0f0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.147+0000 7f80ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.147+0000 7f80ceffd640 1 -- 172.21.15.43:0/3907797934 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0020050 con 0x5594540694d0 2024-04-25T10:39:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.147+0000 7f80ceffd640 1 -- 172.21.15.43:0/3907797934 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f80e00c90c0 con 0x5594540694d0 2024-04-25T10:39:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.147+0000 7f80fc87c740 1 -- 172.21.15.43:0/3907797934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80e00a6630 con 0x5594540694d0 2024-04-25T10:39:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.147+0000 7f80ceffd640 1 -- 172.21.15.43:0/3907797934 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f80e00aae20 con 0x5594540694d0 2024-04-25T10:39:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.147+0000 7f80fc87c740 10 client.17580 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.147+0000 7f80fc87c740 1 -- 172.21.15.43:0/3907797934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5594540aa3f0 con 0x5594540694d0 2024-04-25T10:39:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.148+0000 7f80ceffd640 1 -- 172.21.15.43:0/3907797934 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80e00a6630 con 0x5594540694d0 2024-04-25T10:39:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.148+0000 7f80ceffd640 1 -- 172.21.15.43:0/3907797934 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f80e00a2490 con 0x5594540694d0 2024-04-25T10:39:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.148+0000 7f80fc87c740 10 client.17580 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.148+0000 7f80fc87c740 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.148+0000 7f80fc87c740 20 client.17580 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.148+0000 7f80fc87c740 1 --2- 172.21.15.43:0/3907797934 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559454050f60 0x55945444f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.148+0000 7f80fc87c740 4 client.17580 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:39:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.148+0000 7f80fc87c740 1 -- 172.21.15.43:0/3907797934 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594543a6990 con 0x559454050f60 2024-04-25T10:39:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.148+0000 7f80ee7fc640 1 --2- 172.21.15.43:0/3907797934 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559454050f60 0x55945444f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.149+0000 7f80ceffd640 10 client.17580 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.149+0000 7f80ee7fc640 1 --2- 172.21.15.43:0/3907797934 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559454050f60 0x55945444f500 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32fbfdd2afbacdf1 server_cookie=dfaac2ab08fd7dfe in_seq=0 out_seq=0 2024-04-25T10:39:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.149+0000 7f80ceffd640 1 -- 172.21.15.43:0/3907797934 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5594543a6990 con 0x559454050f60 2024-04-25T10:39:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.149+0000 7f80ceffd640 10 client.17580 handle_command_reply: tid=0 2024-04-25T10:39:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.149+0000 7f80fc87c740 1 client.17580 shutdown 2024-04-25T10:39:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.150+0000 7f80fc87c740 1 -- 172.21.15.43:0/3907797934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594540694d0 msgr2=0x55945443efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.150+0000 7f80fc87c740 1 --2- 172.21.15.43:0/3907797934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594540694d0 0x55945443efc0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f80e00dc0d0 tx=0x7f80e00a6390 comp rx=0 tx=0).stop 2024-04-25T10:39:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.150+0000 7f80fc87c740 1 -- 172.21.15.43:0/3907797934 shutdown_connections 2024-04-25T10:39:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.150+0000 7f80fc87c740 1 --2- 172.21.15.43:0/3907797934 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559454050f60 0x55945444f500 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.150+0000 7f80fc87c740 1 --2- 172.21.15.43:0/3907797934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559454092390 0x55945443d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.150+0000 7f80fc87c740 1 --2- 172.21.15.43:0/3907797934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594540716b0 0x55945443f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.150+0000 7f80ceffd640 0 client.17580 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.150+0000 7f80fc87c740 1 --2- 172.21.15.43:0/3907797934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594540694d0 0x55945443efc0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.150+0000 7f80fc87c740 1 -- 172.21.15.43:0/3907797934 >> 172.21.15.43:0/3907797934 conn(0x5594540864e0 msgr2=0x559454434780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.150+0000 7f80fc87c740 1 -- 172.21.15.43:0/3907797934 shutdown_connections 2024-04-25T10:39:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.150+0000 7f80fc87c740 1 -- 172.21.15.43:0/3907797934 wait complete. 2024-04-25T10:39:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.150+0000 7f80fc87c740 20 client.17580 trim_cache size 0 max 16384 2024-04-25T10:39:41.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.168+0000 7f80fc87c740 1 Processor -- start 2024-04-25T10:39:41.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.168+0000 7f80fc87c740 1 -- start start 2024-04-25T10:39:41.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.168+0000 7f80fc87c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594540694d0 0x559454434d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.168+0000 7f80fc87c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5594540716b0 0x559454452530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.168+0000 7f80fc87c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559454092390 0x559454452a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.168+0000 7f80fc87c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01b2290 con 0x559454092390 2024-04-25T10:39:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.168+0000 7f80fc87c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01afb60 con 0x5594540716b0 2024-04-25T10:39:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.168+0000 7f80fc87c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01ad9d0 con 0x5594540694d0 2024-04-25T10:39:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.168+0000 7f80ef7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559454092390 0x559454452a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.168+0000 7f80ef7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559454092390 0x559454452a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40202/0 (socket says 172.21.15.43:40202) 2024-04-25T10:39:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.168+0000 7f80ef7fe640 1 -- 172.21.15.43:0/2443592844 learned_addr learned my addr 172.21.15.43:0/2443592844 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.168+0000 7f80ee7fc640 1 --2- 172.21.15.43:0/2443592844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5594540716b0 0x559454452530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.168+0000 7f80eeffd640 1 --2- 172.21.15.43:0/2443592844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594540694d0 0x559454434d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.168+0000 7f80ef7fe640 1 -- 172.21.15.43:0/2443592844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594540694d0 msgr2=0x559454434d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.168+0000 7f80ef7fe640 1 --2- 172.21.15.43:0/2443592844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594540694d0 0x559454434d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.169+0000 7f80ef7fe640 1 -- 172.21.15.43:0/2443592844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5594540716b0 msgr2=0x559454452530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.169+0000 7f80ef7fe640 1 --2- 172.21.15.43:0/2443592844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5594540716b0 0x559454452530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.169+0000 7f80ef7fe640 1 -- 172.21.15.43:0/2443592844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55945406a0f0 con 0x559454092390 2024-04-25T10:39:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.169+0000 7f80ef7fe640 1 --2- 172.21.15.43:0/2443592844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559454092390 0x559454452a70 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f80e800fa60 tx=0x7f80e8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726ea0c7b04561a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.169+0000 7f80d57fa640 1 -- 172.21.15.43:0/2443592844 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8004b20 con 0x559454092390 2024-04-25T10:39:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.169+0000 7f80d57fa640 1 -- 172.21.15.43:0/2443592844 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80e8004cc0 con 0x559454092390 2024-04-25T10:39:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.169+0000 7f80d57fa640 1 -- 172.21.15.43:0/2443592844 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8005880 con 0x559454092390 2024-04-25T10:39:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.170+0000 7f80fc87c740 1 -- 172.21.15.43:0/2443592844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559454092390 msgr2=0x559454452a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.170+0000 7f80fc87c740 1 --2- 172.21.15.43:0/2443592844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559454092390 0x559454452a70 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f80e800fa60 tx=0x7f80e8014000 comp rx=0 tx=0).stop 2024-04-25T10:39:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.170+0000 7f80fc87c740 1 -- 172.21.15.43:0/2443592844 shutdown_connections 2024-04-25T10:39:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.170+0000 7f80fc87c740 1 --2- 172.21.15.43:0/2443592844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5594540716b0 0x559454452530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.170+0000 7f80fc87c740 1 --2- 172.21.15.43:0/2443592844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559454092390 0x559454452a70 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.170+0000 7f80fc87c740 1 --2- 172.21.15.43:0/2443592844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594540694d0 0x559454434d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.170+0000 7f80fc87c740 1 -- 172.21.15.43:0/2443592844 >> 172.21.15.43:0/2443592844 conn(0x5594540864e0 msgr2=0x559454434680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:41.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.170+0000 7f80fc87c740 1 -- 172.21.15.43:0/2443592844 shutdown_connections 2024-04-25T10:39:41.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.170+0000 7f80fc87c740 1 -- 172.21.15.43:0/2443592844 wait complete. 2024-04-25T10:39:41.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.170+0000 7f80fc87c740 1 -- start start 2024-04-25T10:39:41.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.170+0000 7f80fc87c740 1 Processor -- start 2024-04-25T10:39:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80fc87c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594540694d0 0x5594544519f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80fc87c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5594540716b0 0x559454451f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80fc87c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559454092390 0x5594544af600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80fc87c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01ae020 con 0x559454092390 2024-04-25T10:39:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80fc87c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01af490 con 0x5594540716b0 2024-04-25T10:39:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80fc87c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01ae6f0 con 0x5594540694d0 2024-04-25T10:39:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80fc87c740 10 client.0 _ll_register_callbacks cb 0x7f80e4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80ef7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559454092390 0x5594544af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80ef7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559454092390 0x5594544af600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40206/0 (socket says 172.21.15.43:40206) 2024-04-25T10:39:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80ef7fe640 1 -- 172.21.15.43:0/390146849 learned_addr learned my addr 172.21.15.43:0/390146849 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80ee7fc640 1 --2- 172.21.15.43:0/390146849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5594540716b0 0x559454451f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80eeffd640 1 --2- 172.21.15.43:0/390146849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594540694d0 0x5594544519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80ef7fe640 1 -- 172.21.15.43:0/390146849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594540694d0 msgr2=0x5594544519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80ef7fe640 1 --2- 172.21.15.43:0/390146849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594540694d0 0x5594544519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80ef7fe640 1 -- 172.21.15.43:0/390146849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5594540716b0 msgr2=0x559454451f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80ef7fe640 1 --2- 172.21.15.43:0/390146849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5594540716b0 0x559454451f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80ef7fe640 1 -- 172.21.15.43:0/390146849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594543594b0 con 0x559454092390 2024-04-25T10:39:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80ee7fc640 1 --2- 172.21.15.43:0/390146849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5594540716b0 0x559454451f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:39:41.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80ef7fe640 1 --2- 172.21.15.43:0/390146849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559454092390 0x5594544af600 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f80e800fa30 tx=0x7f80e800f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445ff1985c5deaab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:41.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:41.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.171+0000 7f80ceffd640 1 -- 172.21.15.43:0/390146849 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8037070 con 0x559454092390 2024-04-25T10:39:41.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.172+0000 7f80fc87c740 1 -- 172.21.15.43:0/390146849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80e8004cc0 con 0x559454092390 2024-04-25T10:39:41.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.172+0000 7f80ceffd640 1 -- 172.21.15.43:0/390146849 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f80e801ec80 con 0x559454092390 2024-04-25T10:39:41.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.172+0000 7f80ceffd640 1 -- 172.21.15.43:0/390146849 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f80e8004e70 con 0x559454092390 2024-04-25T10:39:41.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.172+0000 7f80fc87c740 10 client.17586 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:41.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.172+0000 7f80fc87c740 1 -- 172.21.15.43:0/390146849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5594544b6cb0 con 0x559454092390 2024-04-25T10:39:41.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.172+0000 7f80ceffd640 1 -- 172.21.15.43:0/390146849 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80e8004cc0 con 0x559454092390 2024-04-25T10:39:41.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.172+0000 7f80ceffd640 1 -- 172.21.15.43:0/390146849 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f80e801c040 con 0x559454092390 2024-04-25T10:39:41.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.172+0000 7f80fc87c740 10 client.17586 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:41.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.172+0000 7f80fc87c740 10 client.17586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:41.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.172+0000 7f80fc87c740 20 client.17586 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:41.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.172+0000 7f80fc87c740 1 --2- 172.21.15.43:0/390146849 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5594540263a0 0x5594544b9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:41.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.172+0000 7f80fc87c740 4 client.17586 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:39:41.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.172+0000 7f80fc87c740 1 -- 172.21.15.43:0/390146849 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5594543a0800 con 0x5594540263a0 2024-04-25T10:39:41.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.172+0000 7f80eeffd640 1 --2- 172.21.15.43:0/390146849 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5594540263a0 0x5594544b9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:41.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.173+0000 7f80eeffd640 1 --2- 172.21.15.43:0/390146849 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5594540263a0 0x5594544b9410 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc3dbc76a14ecea server_cookie=97ba23c89c2058a2 in_seq=0 out_seq=0 2024-04-25T10:39:41.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.173+0000 7f80ceffd640 10 client.17586 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:41.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.173+0000 7f80ceffd640 1 -- 172.21.15.43:0/390146849 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5594543a0800 con 0x5594540263a0 2024-04-25T10:39:41.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.173+0000 7f80ceffd640 10 client.17586 handle_command_reply: tid=0 2024-04-25T10:39:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.173+0000 7f80fc87c740 1 client.17586 shutdown 2024-04-25T10:39:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.173+0000 7f80fc87c740 1 -- 172.21.15.43:0/390146849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559454092390 msgr2=0x5594544af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.173+0000 7f80fc87c740 1 --2- 172.21.15.43:0/390146849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559454092390 0x5594544af600 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f80e800fa30 tx=0x7f80e800f8c0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.173+0000 7f80fc87c740 1 -- 172.21.15.43:0/390146849 shutdown_connections 2024-04-25T10:39:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.173+0000 7f80fc87c740 1 --2- 172.21.15.43:0/390146849 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5594540263a0 0x5594544b9410 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.173+0000 7f80fc87c740 1 --2- 172.21.15.43:0/390146849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5594540716b0 0x559454451f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.173+0000 7f80fc87c740 1 --2- 172.21.15.43:0/390146849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559454092390 0x5594544af600 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.173+0000 7f80fc87c740 1 --2- 172.21.15.43:0/390146849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594540694d0 0x5594544519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.173+0000 7f80ceffd640 0 client.17586 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.173+0000 7f80fc87c740 1 -- 172.21.15.43:0/390146849 >> 172.21.15.43:0/390146849 conn(0x5594540864e0 msgr2=0x55945443bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.173+0000 7f80fc87c740 1 -- 172.21.15.43:0/390146849 shutdown_connections 2024-04-25T10:39:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.173+0000 7f80fc87c740 1 -- 172.21.15.43:0/390146849 wait complete. 2024-04-25T10:39:41.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.173+0000 7f80fc87c740 20 client.17586 trim_cache size 0 max 16384 2024-04-25T10:39:41.184 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:39:41.184 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:39:41.184 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:39:41.184 INFO:teuthology.orchestra.run.smithi043.stdout: "8f6837ec-74c7-4fe7-9261-43ce98d13d88": { 2024-04-25T10:39:41.184 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:39:41.184 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "8f6837ec-74c7-4fe7-9261-43ce98d13d88", 2024-04-25T10:39:41.184 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:39:41.185 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:39:41.185 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:39:41.185 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:39:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.174+0000 7f80f59c5640 1 -- 172.21.15.43:0/143570954 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f80d80afc60 msgr2=0x7f80d80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.174+0000 7f80f59c5640 1 --2- 172.21.15.43:0/143570954 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f80d80afc60 0x7f80d80b2170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55945442fbd0 tx=0x7f80e80096a0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.174+0000 7f80f59c5640 1 -- 172.21.15.43:0/143570954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80f0108a40 msgr2=0x7f80f019a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.174+0000 7f80f59c5640 1 --2- 172.21.15.43:0/143570954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80f0108a40 0x7f80f019a3b0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f80e000d740 tx=0x7f80e000dc00 comp rx=0 tx=0).stop 2024-04-25T10:39:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.174+0000 7f80f59c5640 1 -- 172.21.15.43:0/143570954 shutdown_connections 2024-04-25T10:39:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.174+0000 7f80f59c5640 1 --2- 172.21.15.43:0/143570954 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f80d80afc60 0x7f80d80b2170 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.174+0000 7f80f59c5640 1 --2- 172.21.15.43:0/143570954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80f0108e50 0x7f80f019a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.174+0000 7f80f59c5640 1 --2- 172.21.15.43:0/143570954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80f019ebd0 0x7f80f019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.174+0000 7f80f59c5640 1 --2- 172.21.15.43:0/143570954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80f0108a40 0x7f80f019a3b0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.174+0000 7f80f59c5640 1 -- 172.21.15.43:0/143570954 >> 172.21.15.43:0/143570954 conn(0x7f80f0077880 msgr2=0x7f80f0101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.174+0000 7f80f59c5640 1 -- 172.21.15.43:0/143570954 shutdown_connections 2024-04-25T10:39:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:39:41.174+0000 7f80f59c5640 1 -- 172.21.15.43:0/143570954 wait complete. 2024-04-25T10:39:41.188 INFO:tasks.cephfs.filesystem:scrub status for tag:8f6837ec-74c7-4fe7-9261-43ce98d13d88 - {'path': '/', 'tag': '8f6837ec-74c7-4fe7-9261-43ce98d13d88', 'options': 'recursive,force'} 2024-04-25T10:39:42.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:41 smithi077 ceph-mon[32772]: pgmap v220: 97 pgs: 97 active+clean; 3.5 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:39:42.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:41 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:39:42.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:41 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:39:42.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:41 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:39:42.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:41 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:39:42.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:41 smithi164 ceph-mon[32713]: pgmap v220: 97 pgs: 97 active+clean; 3.5 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:39:42.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:41 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:39:42.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:41 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:39:42.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:41 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:39:42.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:41 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:39:42.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:41 smithi043 ceph-mon[29478]: pgmap v220: 97 pgs: 97 active+clean; 3.5 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:39:42.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:41 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:39:42.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:41 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:39:42.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:41 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:39:42.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:41 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:39:44.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:43 smithi077 ceph-mon[32772]: pgmap v221: 97 pgs: 97 active+clean; 3.5 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:39:44.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:43 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:39:44.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:43 smithi164 ceph-mon[32713]: pgmap v221: 97 pgs: 97 active+clean; 3.5 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:39:44.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:43 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:39:44.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:43 smithi043 ceph-mon[29478]: pgmap v221: 97 pgs: 97 active+clean; 3.5 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:39:44.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:43 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:39:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.847+0000 7fb4db764640 1 -- 172.21.15.43:0/2238018828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4d410a000 msgr2=0x7fb4d4103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.847+0000 7fb4db764640 1 --2- 172.21.15.43:0/2238018828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4d410a000 0x7fb4d4103c40 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009900 tx=0x7fb4d002a410 comp rx=0 tx=0).stop 2024-04-25T10:39:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.847+0000 7fb4db764640 1 -- 172.21.15.43:0/2238018828 shutdown_connections 2024-04-25T10:39:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.847+0000 7fb4db764640 1 --2- 172.21.15.43:0/2238018828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4d41042b0 0x7fb4d4110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.847+0000 7fb4db764640 1 --2- 172.21.15.43:0/2238018828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4d410a000 0x7fb4d4103c40 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.847+0000 7fb4db764640 1 --2- 172.21.15.43:0/2238018828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4d4109b60 0x7fb4d4103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.847+0000 7fb4db764640 1 -- 172.21.15.43:0/2238018828 >> 172.21.15.43:0/2238018828 conn(0x7fb4d4079170 msgr2=0x7fb4d40795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.847+0000 7fb4db764640 1 -- 172.21.15.43:0/2238018828 shutdown_connections 2024-04-25T10:39:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.847+0000 7fb4db764640 1 -- 172.21.15.43:0/2238018828 wait complete. 2024-04-25T10:39:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.848+0000 7fb4db764640 1 Processor -- start 2024-04-25T10:39:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.848+0000 7fb4db764640 1 -- start start 2024-04-25T10:39:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.848+0000 7fb4db764640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4d41042b0 0x7fb4d41a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.848+0000 7fb4db764640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4d4109b60 0x7fb4d41a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.848+0000 7fb4db764640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4d410a000 0x7fb4d419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.848+0000 7fb4db764640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d4113d50 con 0x7fb4d4109b60 2024-04-25T10:39:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.848+0000 7fb4db764640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d4113bd0 con 0x7fb4d410a000 2024-04-25T10:39:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.848+0000 7fb4db764640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d4113ed0 con 0x7fb4d41042b0 2024-04-25T10:39:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.848+0000 7fb4d8cff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4d4109b60 0x7fb4d41a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.848+0000 7fb4d8cff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4d4109b60 0x7fb4d41a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40230/0 (socket says 172.21.15.43:40230) 2024-04-25T10:39:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.848+0000 7fb4d8cff640 1 -- 172.21.15.43:0/439044964 learned_addr learned my addr 172.21.15.43:0/439044964 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.848+0000 7fb4d9d01640 1 --2- 172.21.15.43:0/439044964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4d410a000 0x7fb4d419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.848+0000 7fb4d9500640 1 --2- 172.21.15.43:0/439044964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4d41042b0 0x7fb4d41a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.848+0000 7fb4d8cff640 1 -- 172.21.15.43:0/439044964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4d41042b0 msgr2=0x7fb4d41a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.848+0000 7fb4d8cff640 1 --2- 172.21.15.43:0/439044964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4d41042b0 0x7fb4d41a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.848+0000 7fb4d8cff640 1 -- 172.21.15.43:0/439044964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4d410a000 msgr2=0x7fb4d419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.848+0000 7fb4d8cff640 1 --2- 172.21.15.43:0/439044964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4d410a000 0x7fb4d419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.848+0000 7fb4d8cff640 1 -- 172.21.15.43:0/439044964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d419adc0 con 0x7fb4d4109b60 2024-04-25T10:39:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.848+0000 7fb4d9500640 1 --2- 172.21.15.43:0/439044964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4d41042b0 0x7fb4d41a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.849+0000 7fb4d8cff640 1 --2- 172.21.15.43:0/439044964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4d4109b60 0x7fb4d41a0a50 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fb4d002a930 tx=0x7fb4d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.849+0000 7fb4c67fc640 1 -- 172.21.15.43:0/439044964 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d003f030 con 0x7fb4d4109b60 2024-04-25T10:39:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.849+0000 7fb4db764640 1 -- 172.21.15.43:0/439044964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d419aff0 con 0x7fb4d4109b60 2024-04-25T10:39:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.849+0000 7fb4c67fc640 1 -- 172.21.15.43:0/439044964 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4d0003aa0 con 0x7fb4d4109b60 2024-04-25T10:39:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.849+0000 7fb4c67fc640 1 -- 172.21.15.43:0/439044964 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0035070 con 0x7fb4d4109b60 2024-04-25T10:39:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.849+0000 7fb4db764640 1 -- 172.21.15.43:0/439044964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d41a72b0 con 0x7fb4d4109b60 2024-04-25T10:39:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.851+0000 7fb4c67fc640 1 -- 172.21.15.43:0/439044964 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fb4d0042030 con 0x7fb4d4109b60 2024-04-25T10:39:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.852+0000 7fb4e2614740 1 Processor -- start 2024-04-25T10:39:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.852+0000 7fb4e2614740 1 -- start start 2024-04-25T10:39:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.852+0000 7fb4e2614740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 0x5628e81469a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.852+0000 7fb4e2614740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628e7d7e2f0 0x5628e8146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.852+0000 7fb4e2614740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7d864d0 0x5628e8147420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.852+0000 7fb4e2614740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5628e80ad460 con 0x5628e7da71b0 2024-04-25T10:39:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.852+0000 7fb4c67fc640 1 --2- 172.21.15.43:0/439044964 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb4c00afc50 0x7fb4c00b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.852+0000 7fb4e2614740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5628e8067590 con 0x5628e7d864d0 2024-04-25T10:39:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.852+0000 7fb4e2614740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5628e80a9420 con 0x5628e7d7e2f0 2024-04-25T10:39:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.852+0000 7fb4d9d01640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 0x5628e81469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.852+0000 7fb4d9d01640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 0x5628e81469a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40236/0 (socket says 172.21.15.43:40236) 2024-04-25T10:39:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.853+0000 7fb4d9d01640 1 -- 172.21.15.43:0/745245529 learned_addr learned my addr 172.21.15.43:0/745245529 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.853+0000 7fb4c67fc640 1 -- 172.21.15.43:0/439044964 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7fb4d00ff7a0 con 0x7fb4d4109b60 2024-04-25T10:39:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.853+0000 7fb4d9d01640 1 --2- 172.21.15.43:0/439044964 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb4c00afc50 0x7fb4c00b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.853+0000 7fb4d8cff640 1 --2- 172.21.15.43:0/745245529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7d864d0 0x5628e8147420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.853+0000 7fb4d9500640 1 --2- 172.21.15.43:0/745245529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628e7d7e2f0 0x5628e8146ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.853+0000 7fb4d9d01640 1 -- 172.21.15.43:0/745245529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628e7d7e2f0 msgr2=0x5628e8146ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.853+0000 7fb4d9d01640 1 --2- 172.21.15.43:0/745245529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628e7d7e2f0 0x5628e8146ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.853+0000 7fb4d9d01640 1 -- 172.21.15.43:0/745245529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7d864d0 msgr2=0x5628e8147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.853+0000 7fb4d9d01640 1 --2- 172.21.15.43:0/745245529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7d864d0 0x5628e8147420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.853+0000 7fb4d9d01640 1 -- 172.21.15.43:0/745245529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628e80740d0 con 0x5628e7da71b0 2024-04-25T10:39:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.853+0000 7fb4d8cff640 1 --2- 172.21.15.43:0/745245529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7d864d0 0x5628e8147420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:39:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.853+0000 7fb4d9d01640 1 --2- 172.21.15.43:0/745245529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 0x5628e81469a0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc005fd0 tx=0x7fb4cc0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb3afec9cf783ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.853+0000 7fb4d9d01640 1 --2- 172.21.15.43:0/439044964 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb4c00afc50 0x7fb4c00b2160 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc011cb0 tx=0x7fb4cc016000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.853+0000 7fb4bf7fe640 1 -- 172.21.15.43:0/745245529 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc024070 con 0x5628e7da71b0 2024-04-25T10:39:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.854+0000 7fb4bf7fe640 1 -- 172.21.15.43:0/745245529 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4cc00ce20 con 0x5628e7da71b0 2024-04-25T10:39:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.854+0000 7fb4bf7fe640 1 -- 172.21.15.43:0/745245529 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc004630 con 0x5628e7da71b0 2024-04-25T10:39:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.854+0000 7fb4e2614740 1 -- 172.21.15.43:0/745245529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 msgr2=0x5628e81469a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.854+0000 7fb4e2614740 1 --2- 172.21.15.43:0/745245529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 0x5628e81469a0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc005fd0 tx=0x7fb4cc0117b0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.854+0000 7fb4e2614740 1 -- 172.21.15.43:0/745245529 shutdown_connections 2024-04-25T10:39:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.854+0000 7fb4e2614740 1 --2- 172.21.15.43:0/745245529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7d864d0 0x5628e8147420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.854+0000 7fb4e2614740 1 --2- 172.21.15.43:0/745245529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628e7d7e2f0 0x5628e8146ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.854+0000 7fb4e2614740 1 --2- 172.21.15.43:0/745245529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 0x5628e81469a0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.854+0000 7fb4e2614740 1 -- 172.21.15.43:0/745245529 >> 172.21.15.43:0/745245529 conn(0x5628e7d9b300 msgr2=0x5628e81432a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.855+0000 7fb4e2614740 1 -- 172.21.15.43:0/745245529 shutdown_connections 2024-04-25T10:39:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.855+0000 7fb4e2614740 1 -- 172.21.15.43:0/745245529 wait complete. 2024-04-25T10:39:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.855+0000 7fb4e2614740 1 -- start start 2024-04-25T10:39:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.855+0000 7fb4e2614740 1 Processor -- start 2024-04-25T10:39:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.855+0000 7fb4e2614740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628e7d7e2f0 0x5628e8157060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4e2614740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7d864d0 0x5628e81575a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4e2614740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 0x5628e81554e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4e2614740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5628e8157fd0 con 0x5628e7da71b0 2024-04-25T10:39:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4e2614740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5628e8034b60 con 0x5628e7d864d0 2024-04-25T10:39:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4e2614740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5628e8158150 con 0x5628e7d7e2f0 2024-04-25T10:39:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4e2614740 10 client.0 _ll_register_callbacks cb 0x5628e80592b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4d9d01640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 0x5628e81554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4d9d01640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 0x5628e81554e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40248/0 (socket says 172.21.15.43:40248) 2024-04-25T10:39:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4d9d01640 1 -- 172.21.15.43:0/916804855 learned_addr learned my addr 172.21.15.43:0/916804855 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4d9500640 1 --2- 172.21.15.43:0/916804855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628e7d7e2f0 0x5628e8157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4d8cff640 1 --2- 172.21.15.43:0/916804855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7d864d0 0x5628e81575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4d9d01640 1 -- 172.21.15.43:0/916804855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628e7d7e2f0 msgr2=0x5628e8157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4d9d01640 1 --2- 172.21.15.43:0/916804855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628e7d7e2f0 0x5628e8157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4d9d01640 1 -- 172.21.15.43:0/916804855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7d864d0 msgr2=0x5628e81575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4d9d01640 1 --2- 172.21.15.43:0/916804855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7d864d0 0x5628e81575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4d9d01640 1 -- 172.21.15.43:0/916804855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628e7dbf210 con 0x5628e7da71b0 2024-04-25T10:39:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4d9d01640 1 --2- 172.21.15.43:0/916804855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 0x5628e81554e0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc014730 tx=0x7fb4cc004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c370db44bed6c843 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4bcff9640 1 -- 172.21.15.43:0/916804855 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc02e070 con 0x5628e7da71b0 2024-04-25T10:39:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4e2614740 1 -- 172.21.15.43:0/916804855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4cc00ce20 con 0x5628e7da71b0 2024-04-25T10:39:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.856+0000 7fb4bcff9640 1 -- 172.21.15.43:0/916804855 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7fb4cc003b80 con 0x5628e7da71b0 2024-04-25T10:39:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.857+0000 7fb4bcff9640 1 -- 172.21.15.43:0/916804855 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb4cc024040 con 0x5628e7da71b0 2024-04-25T10:39:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.857+0000 7fb4e2614740 10 client.17598 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.857+0000 7fb4e2614740 1 -- 172.21.15.43:0/916804855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5628e7e8b620 con 0x5628e7da71b0 2024-04-25T10:39:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.857+0000 7fb4bcff9640 1 -- 172.21.15.43:0/916804855 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4cc00ce20 con 0x5628e7da71b0 2024-04-25T10:39:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.857+0000 7fb4bcff9640 1 -- 172.21.15.43:0/916804855 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb4cc00f900 con 0x5628e7da71b0 2024-04-25T10:39:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.857+0000 7fb4e2614740 10 client.17598 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.857+0000 7fb4e2614740 10 client.17598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.857+0000 7fb4e2614740 20 client.17598 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.857+0000 7fb4e2614740 1 --2- 172.21.15.43:0/916804855 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5628e7d65d80 0x5628e8167b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.857+0000 7fb4e2614740 4 client.17598 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:39:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.857+0000 7fb4e2614740 1 -- 172.21.15.43:0/916804855 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628e80bb070 con 0x5628e7d65d80 2024-04-25T10:39:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.857+0000 7fb4d9500640 1 --2- 172.21.15.43:0/916804855 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5628e7d65d80 0x5628e8167b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.858+0000 7fb4d9500640 1 --2- 172.21.15.43:0/916804855 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5628e7d65d80 0x5628e8167b90 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f0bbe4c5bc77f4b server_cookie=364ebbc6fcd9d787 in_seq=0 out_seq=0 2024-04-25T10:39:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.858+0000 7fb4bcff9640 10 client.17598 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.859+0000 7fb4bcff9640 1 -- 172.21.15.43:0/916804855 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5628e80bb070 con 0x5628e7d65d80 2024-04-25T10:39:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.859+0000 7fb4bcff9640 10 client.17598 handle_command_reply: tid=0 2024-04-25T10:39:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.859+0000 7fb4e2614740 1 client.17598 shutdown 2024-04-25T10:39:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.859+0000 7fb4e2614740 1 -- 172.21.15.43:0/916804855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 msgr2=0x5628e81554e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.859+0000 7fb4e2614740 1 --2- 172.21.15.43:0/916804855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 0x5628e81554e0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc014730 tx=0x7fb4cc004620 comp rx=0 tx=0).stop 2024-04-25T10:39:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.859+0000 7fb4e2614740 1 -- 172.21.15.43:0/916804855 shutdown_connections 2024-04-25T10:39:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.859+0000 7fb4e2614740 1 --2- 172.21.15.43:0/916804855 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5628e7d65d80 0x5628e8167b90 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.859+0000 7fb4e2614740 1 --2- 172.21.15.43:0/916804855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7d864d0 0x5628e81575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.859+0000 7fb4e2614740 1 --2- 172.21.15.43:0/916804855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 0x5628e81554e0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.859+0000 7fb4e2614740 1 --2- 172.21.15.43:0/916804855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628e7d7e2f0 0x5628e8157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.859+0000 7fb4bcff9640 0 client.17598 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.859+0000 7fb4e2614740 1 -- 172.21.15.43:0/916804855 >> 172.21.15.43:0/916804855 conn(0x5628e7d9b300 msgr2=0x5628e81487a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.859+0000 7fb4e2614740 1 -- 172.21.15.43:0/916804855 shutdown_connections 2024-04-25T10:39:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.859+0000 7fb4e2614740 1 -- 172.21.15.43:0/916804855 wait complete. 2024-04-25T10:39:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.859+0000 7fb4e2614740 20 client.17598 trim_cache size 0 max 16384 2024-04-25T10:39:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.874+0000 7fb4e2614740 1 Processor -- start 2024-04-25T10:39:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.875+0000 7fb4e2614740 1 -- start start 2024-04-25T10:39:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.875+0000 7fb4e2614740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7d7e2f0 0x5628e8146f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.875+0000 7fb4e2614740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628e7d864d0 0x5628e8152720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.875+0000 7fb4e2614740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 0x5628e8152c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.875+0000 7fb4e2614740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d41b0160 con 0x5628e7da71b0 2024-04-25T10:39:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.875+0000 7fb4e2614740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d41ada30 con 0x5628e7d7e2f0 2024-04-25T10:39:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.875+0000 7fb4e2614740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d41ab8c0 con 0x5628e7d864d0 2024-04-25T10:39:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.875+0000 7fb4d9d01640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 0x5628e8152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.875+0000 7fb4d9d01640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 0x5628e8152c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40258/0 (socket says 172.21.15.43:40258) 2024-04-25T10:39:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.875+0000 7fb4d9d01640 1 -- 172.21.15.43:0/3820115508 learned_addr learned my addr 172.21.15.43:0/3820115508 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.875+0000 7fb4d8cff640 1 --2- 172.21.15.43:0/3820115508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628e7d864d0 0x5628e8152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.875+0000 7fb4d9500640 1 --2- 172.21.15.43:0/3820115508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7d7e2f0 0x5628e8146f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.875+0000 7fb4d9d01640 1 -- 172.21.15.43:0/3820115508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628e7d864d0 msgr2=0x5628e8152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.875+0000 7fb4d9d01640 1 --2- 172.21.15.43:0/3820115508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628e7d864d0 0x5628e8152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.875+0000 7fb4d9d01640 1 -- 172.21.15.43:0/3820115508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7d7e2f0 msgr2=0x5628e8146f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.875+0000 7fb4d9d01640 1 --2- 172.21.15.43:0/3820115508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7d7e2f0 0x5628e8146f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.875+0000 7fb4d9d01640 1 -- 172.21.15.43:0/3820115508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628e813ea60 con 0x5628e7da71b0 2024-04-25T10:39:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.875+0000 7fb4d9500640 1 --2- 172.21.15.43:0/3820115508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7d7e2f0 0x5628e8146f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:39:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.875+0000 7fb4d9d01640 1 --2- 172.21.15.43:0/3820115508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 0x5628e8152c60 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00a780 tx=0x7fb4cc0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636c0fb90949e4b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.876+0000 7fb4beffd640 1 -- 172.21.15.43:0/3820115508 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc02e040 con 0x5628e7da71b0 2024-04-25T10:39:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.876+0000 7fb4beffd640 1 -- 172.21.15.43:0/3820115508 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4cc014ac0 con 0x5628e7da71b0 2024-04-25T10:39:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.876+0000 7fb4beffd640 1 -- 172.21.15.43:0/3820115508 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc00f6b0 con 0x5628e7da71b0 2024-04-25T10:39:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.876+0000 7fb4e2614740 1 -- 172.21.15.43:0/3820115508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 msgr2=0x5628e8152c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.876+0000 7fb4e2614740 1 --2- 172.21.15.43:0/3820115508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 0x5628e8152c60 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00a780 tx=0x7fb4cc0117b0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.876+0000 7fb4e2614740 1 -- 172.21.15.43:0/3820115508 shutdown_connections 2024-04-25T10:39:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.876+0000 7fb4e2614740 1 --2- 172.21.15.43:0/3820115508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7da71b0 0x5628e8152c60 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.876+0000 7fb4e2614740 1 --2- 172.21.15.43:0/3820115508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628e7d864d0 0x5628e8152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.876+0000 7fb4e2614740 1 --2- 172.21.15.43:0/3820115508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7d7e2f0 0x5628e8146f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.876+0000 7fb4e2614740 1 -- 172.21.15.43:0/3820115508 >> 172.21.15.43:0/3820115508 conn(0x5628e7d9b300 msgr2=0x5628e8146880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.876+0000 7fb4e2614740 1 -- 172.21.15.43:0/3820115508 shutdown_connections 2024-04-25T10:39:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.876+0000 7fb4e2614740 1 -- 172.21.15.43:0/3820115508 wait complete. 2024-04-25T10:39:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.877+0000 7fb4e2614740 1 -- start start 2024-04-25T10:39:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.877+0000 7fb4e2614740 1 Processor -- start 2024-04-25T10:39:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.877+0000 7fb4e2614740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628e7d7e2f0 0x5628e8160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.877+0000 7fb4e2614740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7d864d0 0x5628e81611f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.877+0000 7fb4e2614740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7da71b0 0x5628e81cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.877+0000 7fb4e2614740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d41abf10 con 0x5628e7d864d0 2024-04-25T10:39:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.877+0000 7fb4e2614740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d41ad360 con 0x5628e7da71b0 2024-04-25T10:39:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.877+0000 7fb4e2614740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d41ac5c0 con 0x5628e7d7e2f0 2024-04-25T10:39:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.877+0000 7fb4e2614740 10 client.0 _ll_register_callbacks cb 0x7fb4d0009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.878+0000 7fb4d8cff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7d864d0 0x5628e81611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.878+0000 7fb4d8cff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7d864d0 0x5628e81611f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40264/0 (socket says 172.21.15.43:40264) 2024-04-25T10:39:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.878+0000 7fb4d8cff640 1 -- 172.21.15.43:0/2051589447 learned_addr learned my addr 172.21.15.43:0/2051589447 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.878+0000 7fb4d9500640 1 --2- 172.21.15.43:0/2051589447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628e7d7e2f0 0x5628e8160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.878+0000 7fb4d9d01640 1 --2- 172.21.15.43:0/2051589447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7da71b0 0x5628e81cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.878+0000 7fb4d8cff640 1 -- 172.21.15.43:0/2051589447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628e7d7e2f0 msgr2=0x5628e8160cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.878+0000 7fb4d8cff640 1 --2- 172.21.15.43:0/2051589447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628e7d7e2f0 0x5628e8160cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.878+0000 7fb4d8cff640 1 -- 172.21.15.43:0/2051589447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7da71b0 msgr2=0x5628e81cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.878+0000 7fb4d8cff640 1 --2- 172.21.15.43:0/2051589447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7da71b0 0x5628e81cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.878+0000 7fb4d8cff640 1 -- 172.21.15.43:0/2051589447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628e81ce200 con 0x5628e7d864d0 2024-04-25T10:39:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.878+0000 7fb4d8cff640 1 --2- 172.21.15.43:0/2051589447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7d864d0 0x5628e81611f0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00c3960 tx=0x7fb4d003b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbd354cdefb34d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.878+0000 7fb4bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.878+0000 7fb4bcff9640 1 -- 172.21.15.43:0/2051589447 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0049040 con 0x5628e7d864d0 2024-04-25T10:39:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.878+0000 7fb4bcff9640 1 -- 172.21.15.43:0/2051589447 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7fb4d00ea0b0 con 0x5628e7d864d0 2024-04-25T10:39:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.878+0000 7fb4e2614740 1 -- 172.21.15.43:0/2051589447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4cc014ac0 con 0x5628e7d864d0 2024-04-25T10:39:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.878+0000 7fb4bcff9640 1 -- 172.21.15.43:0/2051589447 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb4d00c7fe0 con 0x5628e7d864d0 2024-04-25T10:39:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.878+0000 7fb4e2614740 10 client.17604 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.878+0000 7fb4e2614740 1 -- 172.21.15.43:0/2051589447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5628e81cf370 con 0x5628e7d864d0 2024-04-25T10:39:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.879+0000 7fb4bcff9640 1 -- 172.21.15.43:0/2051589447 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4cc014ac0 con 0x5628e7d864d0 2024-04-25T10:39:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.879+0000 7fb4bcff9640 1 -- 172.21.15.43:0/2051589447 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb4d00c3b70 con 0x5628e7d864d0 2024-04-25T10:39:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.879+0000 7fb4e2614740 10 client.17604 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.879+0000 7fb4e2614740 10 client.17604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.879+0000 7fb4e2614740 20 client.17604 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.879+0000 7fb4e2614740 1 --2- 172.21.15.43:0/2051589447 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5628e7d3b1c0 0x5628e81d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.879+0000 7fb4e2614740 4 client.17604 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:39:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.879+0000 7fb4e2614740 1 -- 172.21.15.43:0/2051589447 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5628e80b4ee0 con 0x5628e7d3b1c0 2024-04-25T10:39:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.879+0000 7fb4d9500640 1 --2- 172.21.15.43:0/2051589447 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5628e7d3b1c0 0x5628e81d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.880+0000 7fb4bcff9640 10 client.17604 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.880+0000 7fb4d9500640 1 --2- 172.21.15.43:0/2051589447 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5628e7d3b1c0 0x5628e81d1b30 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2839696d018bed11 server_cookie=a6e3f516e454e69 in_seq=0 out_seq=0 2024-04-25T10:39:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.895+0000 7fb4bcff9640 1 -- 172.21.15.43:0/2051589447 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5628e80b4ee0 con 0x5628e7d3b1c0 2024-04-25T10:39:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.895+0000 7fb4bcff9640 10 client.17604 handle_command_reply: tid=0 2024-04-25T10:39:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.895+0000 7fb4e2614740 1 client.17604 shutdown 2024-04-25T10:39:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.895+0000 7fb4e2614740 1 -- 172.21.15.43:0/2051589447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7d864d0 msgr2=0x5628e81611f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.895+0000 7fb4e2614740 1 --2- 172.21.15.43:0/2051589447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7d864d0 0x5628e81611f0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00c3960 tx=0x7fb4d003b6b0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.895+0000 7fb4e2614740 1 -- 172.21.15.43:0/2051589447 shutdown_connections 2024-04-25T10:39:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.895+0000 7fb4e2614740 1 --2- 172.21.15.43:0/2051589447 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5628e7d3b1c0 0x5628e81d1b30 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.895+0000 7fb4e2614740 1 --2- 172.21.15.43:0/2051589447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e7da71b0 0x5628e81cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.895+0000 7fb4e2614740 1 --2- 172.21.15.43:0/2051589447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628e7d864d0 0x5628e81611f0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.895+0000 7fb4e2614740 1 --2- 172.21.15.43:0/2051589447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628e7d7e2f0 0x5628e8160cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.895+0000 7fb4e2614740 1 -- 172.21.15.43:0/2051589447 >> 172.21.15.43:0/2051589447 conn(0x5628e7d9b300 msgr2=0x5628e8154a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.895+0000 7fb4bcff9640 0 client.17604 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.895+0000 7fb4e2614740 1 -- 172.21.15.43:0/2051589447 shutdown_connections 2024-04-25T10:39:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.895+0000 7fb4e2614740 1 -- 172.21.15.43:0/2051589447 wait complete. 2024-04-25T10:39:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.895+0000 7fb4e2614740 20 client.17604 trim_cache size 0 max 16384 2024-04-25T10:39:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:39:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:39:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:39:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:39:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.896+0000 7fb4db764640 1 -- 172.21.15.43:0/439044964 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb4c00afc50 msgr2=0x7fb4c00b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.896+0000 7fb4db764640 1 --2- 172.21.15.43:0/439044964 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb4c00afc50 0x7fb4c00b2160 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc011cb0 tx=0x7fb4cc016000 comp rx=0 tx=0).stop 2024-04-25T10:39:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.896+0000 7fb4db764640 1 -- 172.21.15.43:0/439044964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4d4109b60 msgr2=0x7fb4d41a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.896+0000 7fb4db764640 1 --2- 172.21.15.43:0/439044964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4d4109b60 0x7fb4d41a0a50 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fb4d002a930 tx=0x7fb4d0004270 comp rx=0 tx=0).stop 2024-04-25T10:39:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.896+0000 7fb4db764640 1 -- 172.21.15.43:0/439044964 shutdown_connections 2024-04-25T10:39:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.896+0000 7fb4db764640 1 --2- 172.21.15.43:0/439044964 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb4c00afc50 0x7fb4c00b2160 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.896+0000 7fb4db764640 1 --2- 172.21.15.43:0/439044964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4d410a000 0x7fb4d419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.896+0000 7fb4db764640 1 --2- 172.21.15.43:0/439044964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4d4109b60 0x7fb4d41a0a50 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.896+0000 7fb4db764640 1 --2- 172.21.15.43:0/439044964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4d41042b0 0x7fb4d41a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.896+0000 7fb4db764640 1 -- 172.21.15.43:0/439044964 >> 172.21.15.43:0/439044964 conn(0x7fb4d4079170 msgr2=0x7fb4d41022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.896+0000 7fb4db764640 1 -- 172.21.15.43:0/439044964 shutdown_connections 2024-04-25T10:39:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:44.896+0000 7fb4db764640 1 -- 172.21.15.43:0/439044964 wait complete. 2024-04-25T10:39:46.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:45 smithi077 ceph-mon[32772]: pgmap v222: 97 pgs: 97 active+clean; 3.6 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-25T10:39:46.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:45 smithi164 ceph-mon[32713]: pgmap v222: 97 pgs: 97 active+clean; 3.6 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-25T10:39:46.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:45 smithi043 ceph-mon[29478]: pgmap v222: 97 pgs: 97 active+clean; 3.6 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-25T10:39:48.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:47 smithi077 ceph-mon[32772]: pgmap v223: 97 pgs: 97 active+clean; 3.6 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-04-25T10:39:48.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:47 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:39:48.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:47 smithi164 ceph-mon[32713]: pgmap v223: 97 pgs: 97 active+clean; 3.6 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-04-25T10:39:48.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:47 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:39:48.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:47 smithi043 ceph-mon[29478]: pgmap v223: 97 pgs: 97 active+clean; 3.6 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-04-25T10:39:48.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:47 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:39:49.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:48 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:39:49.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:48 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:39:49.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:48 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:39:49.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:48 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:39:49.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:48 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:39:49.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:48 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:39:49.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:48 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:39:49.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:48 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:39:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:48 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:39:50.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:50 smithi077 ceph-mon[32772]: pgmap v224: 97 pgs: 97 active+clean; 3.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-25T10:39:50.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:50 smithi164 ceph-mon[32713]: pgmap v224: 97 pgs: 97 active+clean; 3.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-25T10:39:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:50 smithi043 ceph-mon[29478]: pgmap v224: 97 pgs: 97 active+clean; 3.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-25T10:39:52.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:52 smithi077 ceph-mon[32772]: pgmap v225: 97 pgs: 97 active+clean; 3.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-04-25T10:39:52.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:52 smithi164 ceph-mon[32713]: pgmap v225: 97 pgs: 97 active+clean; 3.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-04-25T10:39:52.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:52 smithi043 ceph-mon[29478]: pgmap v225: 97 pgs: 97 active+clean; 3.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-04-25T10:39:54.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:54 smithi077 ceph-mon[32772]: pgmap v226: 97 pgs: 97 active+clean; 3.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-04-25T10:39:54.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:54 smithi164 ceph-mon[32713]: pgmap v226: 97 pgs: 97 active+clean; 3.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-04-25T10:39:54.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:54 smithi043 ceph-mon[29478]: pgmap v226: 97 pgs: 97 active+clean; 3.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-04-25T10:39:56.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:56 smithi077 ceph-mon[32772]: pgmap v227: 97 pgs: 97 active+clean; 4.0 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 18 op/s 2024-04-25T10:39:56.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:56 smithi164 ceph-mon[32713]: pgmap v227: 97 pgs: 97 active+clean; 4.0 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 18 op/s 2024-04-25T10:39:56.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:56 smithi043 ceph-mon[29478]: pgmap v227: 97 pgs: 97 active+clean; 4.0 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 18 op/s 2024-04-25T10:39:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.008+0000 7fa60c90d640 1 -- 172.21.15.43:0/2831182359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa60810e5b0 msgr2=0x7fa6081109d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.008+0000 7fa60c90d640 1 --2- 172.21.15.43:0/2831182359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa60810e5b0 0x7fa6081109d0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fa600009ca0 tx=0x7fa60002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.008+0000 7fa60c90d640 1 -- 172.21.15.43:0/2831182359 shutdown_connections 2024-04-25T10:39:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.008+0000 7fa60c90d640 1 --2- 172.21.15.43:0/2831182359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa60810e5b0 0x7fa6081109d0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.008+0000 7fa60c90d640 1 --2- 172.21.15.43:0/2831182359 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6081020b0 0x7fa6081024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.008+0000 7fa60c90d640 1 --2- 172.21.15.43:0/2831182359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa608109c00 0x7fa608101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.009+0000 7fa60c90d640 1 -- 172.21.15.43:0/2831182359 >> 172.21.15.43:0/2831182359 conn(0x7fa6080fd8a0 msgr2=0x7fa6080ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.009+0000 7fa60c90d640 1 -- 172.21.15.43:0/2831182359 shutdown_connections 2024-04-25T10:39:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.009+0000 7fa60c90d640 1 -- 172.21.15.43:0/2831182359 wait complete. 2024-04-25T10:39:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.009+0000 7fa60c90d640 1 Processor -- start 2024-04-25T10:39:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.010+0000 7fa60c90d640 1 -- start start 2024-04-25T10:39:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.010+0000 7fa60c90d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6081020b0 0x7fa608102d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.010+0000 7fa60c90d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa608109c00 0x7fa608103290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.010+0000 7fa60c90d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa60810e5b0 0x7fa6081037d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.010+0000 7fa60c90d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa608113d70 con 0x7fa6081020b0 2024-04-25T10:39:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.010+0000 7fa60c90d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa608113bf0 con 0x7fa60810e5b0 2024-04-25T10:39:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.010+0000 7fa60c90d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa608113ef0 con 0x7fa608109c00 2024-04-25T10:39:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.010+0000 7fa60659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6081020b0 0x7fa608102d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.010+0000 7fa60659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6081020b0 0x7fa608102d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45326/0 (socket says 172.21.15.43:45326) 2024-04-25T10:39:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.010+0000 7fa60659c640 1 -- 172.21.15.43:0/3264704836 learned_addr learned my addr 172.21.15.43:0/3264704836 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.010+0000 7fa5fffff640 1 --2- 172.21.15.43:0/3264704836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa608109c00 0x7fa608103290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.010+0000 7fa606d9d640 1 --2- 172.21.15.43:0/3264704836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa60810e5b0 0x7fa6081037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.010+0000 7fa60659c640 1 -- 172.21.15.43:0/3264704836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa608109c00 msgr2=0x7fa608103290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.010+0000 7fa60659c640 1 --2- 172.21.15.43:0/3264704836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa608109c00 0x7fa608103290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.010+0000 7fa60659c640 1 -- 172.21.15.43:0/3264704836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa60810e5b0 msgr2=0x7fa6081037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.010+0000 7fa60659c640 1 --2- 172.21.15.43:0/3264704836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa60810e5b0 0x7fa6081037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.010+0000 7fa60659c640 1 -- 172.21.15.43:0/3264704836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6081a8b10 con 0x7fa6081020b0 2024-04-25T10:39:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.011+0000 7fa60659c640 1 --2- 172.21.15.43:0/3264704836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6081020b0 0x7fa608102d50 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8005fd0 tx=0x7fa5f800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.011+0000 7fa5ff7fe640 1 -- 172.21.15.43:0/3264704836 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8017070 con 0x7fa6081020b0 2024-04-25T10:39:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.011+0000 7fa60c90d640 1 -- 172.21.15.43:0/3264704836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6081a8e00 con 0x7fa6081020b0 2024-04-25T10:39:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.011+0000 7fa5ff7fe640 1 -- 172.21.15.43:0/3264704836 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5f8009c30 con 0x7fa6081020b0 2024-04-25T10:39:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.011+0000 7fa5ff7fe640 1 -- 172.21.15.43:0/3264704836 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8003d10 con 0x7fa6081020b0 2024-04-25T10:39:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.011+0000 7fa60c90d640 1 -- 172.21.15.43:0/3264704836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6081a9270 con 0x7fa6081020b0 2024-04-25T10:39:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.014+0000 7fa6137c3740 1 Processor -- start 2024-04-25T10:39:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.014+0000 7fa5ff7fe640 1 -- 172.21.15.43:0/3264704836 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fa5f8003eb0 con 0x7fa6081020b0 2024-04-25T10:39:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.014+0000 7fa6137c3740 1 -- start start 2024-04-25T10:39:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.014+0000 7fa6137c3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555720c081b0 0x555720fa7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.014+0000 7fa6137c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720bdf2f0 0x555720fa8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.014+0000 7fa6137c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720be74d0 0x555720fa8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.014+0000 7fa6137c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555720f0e460 con 0x555720bdf2f0 2024-04-25T10:39:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.014+0000 7fa6137c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555720ec8590 con 0x555720be74d0 2024-04-25T10:39:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.014+0000 7fa6137c3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555720f0a420 con 0x555720c081b0 2024-04-25T10:39:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.014+0000 7fa60659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720bdf2f0 0x555720fa8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.015+0000 7fa60659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720bdf2f0 0x555720fa8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45342/0 (socket says 172.21.15.43:45342) 2024-04-25T10:39:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.015+0000 7fa60659c640 1 -- 172.21.15.43:0/3920304636 learned_addr learned my addr 172.21.15.43:0/3920304636 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.015+0000 7fa606d9d640 1 --2- 172.21.15.43:0/3920304636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555720c081b0 0x555720fa7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.015+0000 7fa5fffff640 1 --2- 172.21.15.43:0/3920304636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720be74d0 0x555720fa8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.015+0000 7fa5ff7fe640 1 --2- 172.21.15.43:0/3264704836 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa5f40afc50 0x7fa5f40b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.015+0000 7fa60659c640 1 -- 172.21.15.43:0/3920304636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555720c081b0 msgr2=0x555720fa7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.015+0000 7fa60659c640 1 --2- 172.21.15.43:0/3920304636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555720c081b0 0x555720fa7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.015+0000 7fa606d9d640 1 --2- 172.21.15.43:0/3264704836 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa5f40afc50 0x7fa5f40b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.015+0000 7fa60659c640 1 -- 172.21.15.43:0/3920304636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720be74d0 msgr2=0x555720fa8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.015+0000 7fa60659c640 1 --2- 172.21.15.43:0/3920304636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720be74d0 0x555720fa8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.015+0000 7fa60659c640 1 -- 172.21.15.43:0/3920304636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555720ed50d0 con 0x555720bdf2f0 2024-04-25T10:39:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.015+0000 7fa5fffff640 1 --2- 172.21.15.43:0/3920304636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720be74d0 0x555720fa8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:39:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.015+0000 7fa5ff7fe640 1 -- 172.21.15.43:0/3264704836 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7fa5f80dec80 con 0x7fa6081020b0 2024-04-25T10:39:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.015+0000 7fa60659c640 1 --2- 172.21.15.43:0/3920304636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720bdf2f0 0x555720fa8030 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80dc340 tx=0x7fa5f800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef71b9cee8f86a95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.015+0000 7fa606d9d640 1 --2- 172.21.15.43:0/3264704836 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa5f40afc50 0x7fa5f40b2160 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa600005cc0 tx=0x7fa60002a8c0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.016+0000 7fa5fcff9640 1 -- 172.21.15.43:0/3920304636 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f80a6bf0 con 0x555720bdf2f0 2024-04-25T10:39:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.016+0000 7fa5fcff9640 1 -- 172.21.15.43:0/3920304636 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5f80a6930 con 0x555720bdf2f0 2024-04-25T10:39:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.016+0000 7fa5fcff9640 1 -- 172.21.15.43:0/3920304636 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f80a6590 con 0x555720bdf2f0 2024-04-25T10:39:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.017+0000 7fa6137c3740 1 -- 172.21.15.43:0/3920304636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720bdf2f0 msgr2=0x555720fa8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.017+0000 7fa6137c3740 1 --2- 172.21.15.43:0/3920304636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720bdf2f0 0x555720fa8030 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80dc340 tx=0x7fa5f800bf60 comp rx=0 tx=0).stop 2024-04-25T10:39:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.017+0000 7fa6137c3740 1 -- 172.21.15.43:0/3920304636 shutdown_connections 2024-04-25T10:39:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.017+0000 7fa6137c3740 1 --2- 172.21.15.43:0/3920304636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720be74d0 0x555720fa8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.017+0000 7fa6137c3740 1 --2- 172.21.15.43:0/3920304636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720bdf2f0 0x555720fa8030 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.017+0000 7fa6137c3740 1 --2- 172.21.15.43:0/3920304636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555720c081b0 0x555720fa7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.017+0000 7fa6137c3740 1 -- 172.21.15.43:0/3920304636 >> 172.21.15.43:0/3920304636 conn(0x555720bfc300 msgr2=0x555720fa42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.017+0000 7fa6137c3740 1 -- 172.21.15.43:0/3920304636 shutdown_connections 2024-04-25T10:39:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.017+0000 7fa6137c3740 1 -- 172.21.15.43:0/3920304636 wait complete. 2024-04-25T10:39:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.017+0000 7fa6137c3740 1 -- start start 2024-04-25T10:39:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.017+0000 7fa6137c3740 1 Processor -- start 2024-04-25T10:39:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.018+0000 7fa6137c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720bdf2f0 0x555720fb21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.018+0000 7fa6137c3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555720be74d0 0x555720fb26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.018+0000 7fa6137c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720c081b0 0x555720fb8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.018+0000 7fa6137c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555720fb4b60 con 0x555720bdf2f0 2024-04-25T10:39:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.018+0000 7fa6137c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555720e95b60 con 0x555720c081b0 2024-04-25T10:39:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.018+0000 7fa6137c3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555720fb4ce0 con 0x555720be74d0 2024-04-25T10:39:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.018+0000 7fa6137c3740 10 client.0 _ll_register_callbacks cb 0x555720eba2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.018+0000 7fa60659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720bdf2f0 0x555720fb21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.018+0000 7fa60659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720bdf2f0 0x555720fb21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45344/0 (socket says 172.21.15.43:45344) 2024-04-25T10:39:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.018+0000 7fa60659c640 1 -- 172.21.15.43:0/2787917397 learned_addr learned my addr 172.21.15.43:0/2787917397 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.018+0000 7fa606d9d640 1 --2- 172.21.15.43:0/2787917397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720c081b0 0x555720fb8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.018+0000 7fa5fffff640 1 --2- 172.21.15.43:0/2787917397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555720be74d0 0x555720fb26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.018+0000 7fa60659c640 1 -- 172.21.15.43:0/2787917397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555720be74d0 msgr2=0x555720fb26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.018+0000 7fa60659c640 1 --2- 172.21.15.43:0/2787917397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555720be74d0 0x555720fb26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.018+0000 7fa60659c640 1 -- 172.21.15.43:0/2787917397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720c081b0 msgr2=0x555720fb8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.018+0000 7fa60659c640 1 --2- 172.21.15.43:0/2787917397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720c081b0 0x555720fb8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.018+0000 7fa60659c640 1 -- 172.21.15.43:0/2787917397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555720c20210 con 0x555720bdf2f0 2024-04-25T10:39:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.018+0000 7fa606d9d640 1 --2- 172.21.15.43:0/2787917397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720c081b0 0x555720fb8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.018+0000 7fa60659c640 1 --2- 172.21.15.43:0/2787917397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720bdf2f0 0x555720fb21b0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80a4090 tx=0x7fa5f80ab240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f680d63baee27055 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.018+0000 7fa5edffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.019+0000 7fa5edffb640 1 -- 172.21.15.43:0/2787917397 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f80a6770 con 0x555720bdf2f0 2024-04-25T10:39:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.019+0000 7fa5edffb640 1 -- 172.21.15.43:0/2787917397 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7fa5f80dc650 con 0x555720bdf2f0 2024-04-25T10:39:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.019+0000 7fa6137c3740 1 -- 172.21.15.43:0/2787917397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f80a6930 con 0x555720bdf2f0 2024-04-25T10:39:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.019+0000 7fa5edffb640 1 -- 172.21.15.43:0/2787917397 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa5f80a7520 con 0x555720bdf2f0 2024-04-25T10:39:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.019+0000 7fa6137c3740 10 client.17616 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.019+0000 7fa6137c3740 1 -- 172.21.15.43:0/2787917397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555720cec620 con 0x555720bdf2f0 2024-04-25T10:39:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.019+0000 7fa5edffb640 1 -- 172.21.15.43:0/2787917397 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5f80a6930 con 0x555720bdf2f0 2024-04-25T10:39:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.019+0000 7fa5edffb640 1 -- 172.21.15.43:0/2787917397 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa5f80af7f0 con 0x555720bdf2f0 2024-04-25T10:39:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.019+0000 7fa6137c3740 10 client.17616 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.019+0000 7fa6137c3740 10 client.17616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.019+0000 7fa6137c3740 20 client.17616 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.019+0000 7fa6137c3740 1 --2- 172.21.15.43:0/2787917397 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x555720bc6d80 0x555720fc44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.019+0000 7fa6137c3740 4 client.17616 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:39:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.019+0000 7fa6137c3740 1 -- 172.21.15.43:0/2787917397 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555720f1c070 con 0x555720bc6d80 2024-04-25T10:39:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.020+0000 7fa5fffff640 1 --2- 172.21.15.43:0/2787917397 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x555720bc6d80 0x555720fc44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.020+0000 7fa5fffff640 1 --2- 172.21.15.43:0/2787917397 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x555720bc6d80 0x555720fc44b0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbefb1927b2a19d0 server_cookie=b57e6baea7a8334c in_seq=0 out_seq=0 2024-04-25T10:39:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.020+0000 7fa5edffb640 10 client.17616 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.021+0000 7fa5edffb640 1 -- 172.21.15.43:0/2787917397 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555720f1c070 con 0x555720bc6d80 2024-04-25T10:39:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.021+0000 7fa5edffb640 10 client.17616 handle_command_reply: tid=0 2024-04-25T10:39:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.021+0000 7fa6137c3740 1 client.17616 shutdown 2024-04-25T10:39:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.021+0000 7fa6137c3740 1 -- 172.21.15.43:0/2787917397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720bdf2f0 msgr2=0x555720fb21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.021+0000 7fa6137c3740 1 --2- 172.21.15.43:0/2787917397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720bdf2f0 0x555720fb21b0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80a4090 tx=0x7fa5f80ab240 comp rx=0 tx=0).stop 2024-04-25T10:39:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.021+0000 7fa6137c3740 1 -- 172.21.15.43:0/2787917397 shutdown_connections 2024-04-25T10:39:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.021+0000 7fa6137c3740 1 --2- 172.21.15.43:0/2787917397 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x555720bc6d80 0x555720fc44b0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.021+0000 7fa6137c3740 1 --2- 172.21.15.43:0/2787917397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720c081b0 0x555720fb8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.021+0000 7fa6137c3740 1 --2- 172.21.15.43:0/2787917397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555720be74d0 0x555720fb26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.021+0000 7fa5edffb640 0 client.17616 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.021+0000 7fa6137c3740 1 --2- 172.21.15.43:0/2787917397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720bdf2f0 0x555720fb21b0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.021+0000 7fa6137c3740 1 -- 172.21.15.43:0/2787917397 >> 172.21.15.43:0/2787917397 conn(0x555720bfc300 msgr2=0x555720fa9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.021+0000 7fa6137c3740 1 -- 172.21.15.43:0/2787917397 shutdown_connections 2024-04-25T10:39:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.021+0000 7fa6137c3740 1 -- 172.21.15.43:0/2787917397 wait complete. 2024-04-25T10:39:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.021+0000 7fa6137c3740 20 client.17616 trim_cache size 0 max 16384 2024-04-25T10:39:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.037+0000 7fa6137c3740 1 Processor -- start 2024-04-25T10:39:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.037+0000 7fa6137c3740 1 -- start start 2024-04-25T10:39:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.037+0000 7fa6137c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720bdf2f0 0x555720fb7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.037+0000 7fa6137c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720be74d0 0x555720fab850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.037+0000 7fa6137c3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555720c081b0 0x555720fabd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.037+0000 7fa6137c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081b4670 con 0x555720be74d0 2024-04-25T10:39:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.037+0000 7fa6137c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081b1f40 con 0x555720bdf2f0 2024-04-25T10:39:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.037+0000 7fa6137c3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081afdb0 con 0x555720c081b0 2024-04-25T10:39:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.037+0000 7fa5fffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720be74d0 0x555720fab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.037+0000 7fa5fffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720be74d0 0x555720fab850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45350/0 (socket says 172.21.15.43:45350) 2024-04-25T10:39:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.037+0000 7fa5fffff640 1 -- 172.21.15.43:0/1554963085 learned_addr learned my addr 172.21.15.43:0/1554963085 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.037+0000 7fa60659c640 1 --2- 172.21.15.43:0/1554963085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720bdf2f0 0x555720fb7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.037+0000 7fa606d9d640 1 --2- 172.21.15.43:0/1554963085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555720c081b0 0x555720fabd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.037+0000 7fa5fffff640 1 -- 172.21.15.43:0/1554963085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555720c081b0 msgr2=0x555720fabd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.037+0000 7fa5fffff640 1 --2- 172.21.15.43:0/1554963085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555720c081b0 0x555720fabd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.037+0000 7fa5fffff640 1 -- 172.21.15.43:0/1554963085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720bdf2f0 msgr2=0x555720fb7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.037+0000 7fa5fffff640 1 --2- 172.21.15.43:0/1554963085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720bdf2f0 0x555720fb7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.038+0000 7fa5fffff640 1 -- 172.21.15.43:0/1554963085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555720f9e4c0 con 0x555720be74d0 2024-04-25T10:39:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.038+0000 7fa60659c640 1 --2- 172.21.15.43:0/1554963085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720bdf2f0 0x555720fb7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.038+0000 7fa5fffff640 1 --2- 172.21.15.43:0/1554963085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720be74d0 0x555720fab850 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0004650 tx=0x7fa5f000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92a65ddb864d74c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.038+0000 7fa5effff640 1 -- 172.21.15.43:0/1554963085 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0005020 con 0x555720be74d0 2024-04-25T10:39:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.038+0000 7fa5effff640 1 -- 172.21.15.43:0/1554963085 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5f00047a0 con 0x555720be74d0 2024-04-25T10:39:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.038+0000 7fa5effff640 1 -- 172.21.15.43:0/1554963085 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f001f350 con 0x555720be74d0 2024-04-25T10:39:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.039+0000 7fa6137c3740 1 -- 172.21.15.43:0/1554963085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720be74d0 msgr2=0x555720fab850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.039+0000 7fa6137c3740 1 --2- 172.21.15.43:0/1554963085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720be74d0 0x555720fab850 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0004650 tx=0x7fa5f000b390 comp rx=0 tx=0).stop 2024-04-25T10:39:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.039+0000 7fa6137c3740 1 -- 172.21.15.43:0/1554963085 shutdown_connections 2024-04-25T10:39:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.039+0000 7fa6137c3740 1 --2- 172.21.15.43:0/1554963085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555720c081b0 0x555720fabd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.039+0000 7fa6137c3740 1 --2- 172.21.15.43:0/1554963085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720be74d0 0x555720fab850 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.039+0000 7fa6137c3740 1 --2- 172.21.15.43:0/1554963085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720bdf2f0 0x555720fb7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.039+0000 7fa6137c3740 1 -- 172.21.15.43:0/1554963085 >> 172.21.15.43:0/1554963085 conn(0x555720bfc300 msgr2=0x555720fa82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.039+0000 7fa6137c3740 1 -- 172.21.15.43:0/1554963085 shutdown_connections 2024-04-25T10:39:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.039+0000 7fa6137c3740 1 -- 172.21.15.43:0/1554963085 wait complete. 2024-04-25T10:39:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.039+0000 7fa6137c3740 1 -- start start 2024-04-25T10:39:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.039+0000 7fa6137c3740 1 Processor -- start 2024-04-25T10:39:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa6137c3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555720bdf2f0 0x555720fbea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa6137c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720be74d0 0x555720fbbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa6137c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720c081b0 0x555720fbc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa6137c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081b0400 con 0x555720be74d0 2024-04-25T10:39:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa6137c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081b1870 con 0x555720c081b0 2024-04-25T10:39:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa6137c3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081b0ad0 con 0x555720bdf2f0 2024-04-25T10:39:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa6137c3740 10 client.0 _ll_register_callbacks cb 0x7fa5f0004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:39:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa5fffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720be74d0 0x555720fbbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa5fffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720be74d0 0x555720fbbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45366/0 (socket says 172.21.15.43:45366) 2024-04-25T10:39:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa5fffff640 1 -- 172.21.15.43:0/567651509 learned_addr learned my addr 172.21.15.43:0/567651509 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:39:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa606d9d640 1 --2- 172.21.15.43:0/567651509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720c081b0 0x555720fbc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa5fffff640 1 -- 172.21.15.43:0/567651509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555720bdf2f0 msgr2=0x555720fbea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa60659c640 1 --2- 172.21.15.43:0/567651509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555720bdf2f0 0x555720fbea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa5fffff640 1 --2- 172.21.15.43:0/567651509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555720bdf2f0 0x555720fbea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa5fffff640 1 -- 172.21.15.43:0/567651509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720c081b0 msgr2=0x555720fbc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:39:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa5fffff640 1 --2- 172.21.15.43:0/567651509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720c081b0 0x555720fbc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa5fffff640 1 -- 172.21.15.43:0/567651509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555720fbc940 con 0x555720be74d0 2024-04-25T10:39:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa606d9d640 1 --2- 172.21.15.43:0/567651509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720c081b0 0x555720fbc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:39:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa5fffff640 1 --2- 172.21.15.43:0/567651509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720be74d0 0x555720fbbd90 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000b800 tx=0x7fa5f000bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b93d5385edbe36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:39:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa5edffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:39:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.040+0000 7fa5edffb640 1 -- 172.21.15.43:0/567651509 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0004d10 con 0x555720be74d0 2024-04-25T10:39:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.041+0000 7fa5edffb640 1 -- 172.21.15.43:0/567651509 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7fa5f0005000 con 0x555720be74d0 2024-04-25T10:39:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.041+0000 7fa6137c3740 1 -- 172.21.15.43:0/567651509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f00047a0 con 0x555720be74d0 2024-04-25T10:39:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.041+0000 7fa5edffb640 1 -- 172.21.15.43:0/567651509 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa5f0038050 con 0x555720be74d0 2024-04-25T10:39:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.041+0000 7fa6137c3740 10 client.17622 fetch_fsmap learned FSMap version 26 2024-04-25T10:39:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.041+0000 7fa6137c3740 1 -- 172.21.15.43:0/567651509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55572102c3f0 con 0x555720be74d0 2024-04-25T10:39:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.041+0000 7fa5edffb640 1 -- 172.21.15.43:0/567651509 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5f00047a0 con 0x555720be74d0 2024-04-25T10:39:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.041+0000 7fa5edffb640 1 -- 172.21.15.43:0/567651509 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa5f001f770 con 0x555720be74d0 2024-04-25T10:39:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.041+0000 7fa6137c3740 10 client.17622 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:39:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.041+0000 7fa6137c3740 10 client.17622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:39:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.041+0000 7fa6137c3740 20 client.17622 populate_metadata read hostname 'smithi043' 2024-04-25T10:39:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.041+0000 7fa6137c3740 1 --2- 172.21.15.43:0/567651509 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x555720b9c1c0 0x55572102ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:39:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.041+0000 7fa6137c3740 4 client.17622 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:39:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.041+0000 7fa6137c3740 1 -- 172.21.15.43:0/567651509 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555720f15ee0 con 0x555720b9c1c0 2024-04-25T10:39:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.041+0000 7fa60659c640 1 --2- 172.21.15.43:0/567651509 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x555720b9c1c0 0x55572102ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:39:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.042+0000 7fa5edffb640 10 client.17622 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.042+0000 7fa60659c640 1 --2- 172.21.15.43:0/567651509 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x555720b9c1c0 0x55572102ea70 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=763e7946261fa1c4 server_cookie=3e41f8e741712c94 in_seq=0 out_seq=0 2024-04-25T10:39:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.058+0000 7fa5edffb640 1 -- 172.21.15.43:0/567651509 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555720f15ee0 con 0x555720b9c1c0 2024-04-25T10:39:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.058+0000 7fa5edffb640 10 client.17622 handle_command_reply: tid=0 2024-04-25T10:39:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.058+0000 7fa6137c3740 1 client.17622 shutdown 2024-04-25T10:39:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa6137c3740 1 -- 172.21.15.43:0/567651509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720be74d0 msgr2=0x555720fbbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa6137c3740 1 --2- 172.21.15.43:0/567651509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720be74d0 0x555720fbbd90 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000b800 tx=0x7fa5f000bf50 comp rx=0 tx=0).stop 2024-04-25T10:39:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa6137c3740 1 -- 172.21.15.43:0/567651509 shutdown_connections 2024-04-25T10:39:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa6137c3740 1 --2- 172.21.15.43:0/567651509 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x555720b9c1c0 0x55572102ea70 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa6137c3740 1 --2- 172.21.15.43:0/567651509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555720c081b0 0x555720fbc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa5edffb640 0 client.17622 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:39:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa6137c3740 1 --2- 172.21.15.43:0/567651509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555720be74d0 0x555720fbbd90 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa6137c3740 1 --2- 172.21.15.43:0/567651509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555720bdf2f0 0x555720fbea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa6137c3740 1 -- 172.21.15.43:0/567651509 >> 172.21.15.43:0/567651509 conn(0x555720bfc300 msgr2=0x555720fb1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa6137c3740 1 -- 172.21.15.43:0/567651509 shutdown_connections 2024-04-25T10:39:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa6137c3740 1 -- 172.21.15.43:0/567651509 wait complete. 2024-04-25T10:39:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa6137c3740 20 client.17622 trim_cache size 0 max 16384 2024-04-25T10:39:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:39:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:39:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:39:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:39:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa60c90d640 1 -- 172.21.15.43:0/3264704836 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa5f40afc50 msgr2=0x7fa5f40b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa60c90d640 1 --2- 172.21.15.43:0/3264704836 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa5f40afc50 0x7fa5f40b2160 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa600005cc0 tx=0x7fa60002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa60c90d640 1 -- 172.21.15.43:0/3264704836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6081020b0 msgr2=0x7fa608102d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:39:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa60c90d640 1 --2- 172.21.15.43:0/3264704836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6081020b0 0x7fa608102d50 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8005fd0 tx=0x7fa5f800dde0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa60c90d640 1 -- 172.21.15.43:0/3264704836 shutdown_connections 2024-04-25T10:39:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa60c90d640 1 --2- 172.21.15.43:0/3264704836 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa5f40afc50 0x7fa5f40b2160 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa60c90d640 1 --2- 172.21.15.43:0/3264704836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa60810e5b0 0x7fa6081037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa60c90d640 1 --2- 172.21.15.43:0/3264704836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa608109c00 0x7fa608103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.059+0000 7fa60c90d640 1 --2- 172.21.15.43:0/3264704836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6081020b0 0x7fa608102d50 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:39:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.060+0000 7fa60c90d640 1 -- 172.21.15.43:0/3264704836 >> 172.21.15.43:0/3264704836 conn(0x7fa6080fd8a0 msgr2=0x7fa6080fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:39:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.060+0000 7fa60c90d640 1 -- 172.21.15.43:0/3264704836 shutdown_connections 2024-04-25T10:39:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:39:58.060+0000 7fa60c90d640 1 -- 172.21.15.43:0/3264704836 wait complete. 2024-04-25T10:39:58.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:39:58 smithi077 ceph-mon[32772]: pgmap v228: 97 pgs: 97 active+clean; 4.0 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-04-25T10:39:58.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:39:58 smithi164 ceph-mon[32713]: pgmap v228: 97 pgs: 97 active+clean; 4.0 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-04-25T10:39:58.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:39:58 smithi043 ceph-mon[29478]: pgmap v228: 97 pgs: 97 active+clean; 4.0 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-04-25T10:40:00.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:00 smithi077 ceph-mon[32772]: pgmap v229: 97 pgs: 97 active+clean; 4.3 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 64 KiB/s wr, 20 op/s 2024-04-25T10:40:00.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:00 smithi077 ceph-mon[32772]: overall HEALTH_OK 2024-04-25T10:40:00.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:00 smithi164 ceph-mon[32713]: pgmap v229: 97 pgs: 97 active+clean; 4.3 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 64 KiB/s wr, 20 op/s 2024-04-25T10:40:00.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:00 smithi164 ceph-mon[32713]: overall HEALTH_OK 2024-04-25T10:40:00.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:00 smithi043 ceph-mon[29478]: pgmap v229: 97 pgs: 97 active+clean; 4.3 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 64 KiB/s wr, 20 op/s 2024-04-25T10:40:00.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:00 smithi043 ceph-mon[29478]: overall HEALTH_OK 2024-04-25T10:40:02.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:02 smithi077 ceph-mon[32772]: pgmap v230: 97 pgs: 97 active+clean; 4.3 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:40:02.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:02 smithi164 ceph-mon[32713]: pgmap v230: 97 pgs: 97 active+clean; 4.3 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:40:02.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:02 smithi043 ceph-mon[29478]: pgmap v230: 97 pgs: 97 active+clean; 4.3 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:40:04.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:04 smithi043 ceph-mon[29478]: pgmap v231: 97 pgs: 97 active+clean; 4.3 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:40:04.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:04 smithi077 ceph-mon[32772]: pgmap v231: 97 pgs: 97 active+clean; 4.3 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:40:04.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:04 smithi164 ceph-mon[32713]: pgmap v231: 97 pgs: 97 active+clean; 4.3 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:40:05.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:05 smithi043 ceph-mon[29478]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T10:40:05.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:05 smithi077 ceph-mon[32772]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T10:40:05.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:05 smithi164 ceph-mon[32713]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T10:40:06.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:06 smithi043 ceph-mon[29478]: pgmap v233: 97 pgs: 97 active+clean; 4.5 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 105 KiB/s wr, 23 op/s 2024-04-25T10:40:06.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:06 smithi077 ceph-mon[32772]: pgmap v233: 97 pgs: 97 active+clean; 4.5 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 105 KiB/s wr, 23 op/s 2024-04-25T10:40:06.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:06 smithi164 ceph-mon[32713]: pgmap v233: 97 pgs: 97 active+clean; 4.5 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 105 KiB/s wr, 23 op/s 2024-04-25T10:40:08.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:08 smithi043 ceph-mon[29478]: pgmap v234: 97 pgs: 97 active+clean; 4.5 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 105 KiB/s wr, 23 op/s 2024-04-25T10:40:08.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:08 smithi077 ceph-mon[32772]: pgmap v234: 97 pgs: 97 active+clean; 4.5 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 105 KiB/s wr, 23 op/s 2024-04-25T10:40:08.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:08 smithi164 ceph-mon[32713]: pgmap v234: 97 pgs: 97 active+clean; 4.5 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 105 KiB/s wr, 23 op/s 2024-04-25T10:40:10.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:10 smithi043 ceph-mon[29478]: pgmap v235: 97 pgs: 97 active+clean; 4.7 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 20 op/s 2024-04-25T10:40:10.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:10 smithi077 ceph-mon[32772]: pgmap v235: 97 pgs: 97 active+clean; 4.7 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 20 op/s 2024-04-25T10:40:10.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:10 smithi164 ceph-mon[32713]: pgmap v235: 97 pgs: 97 active+clean; 4.7 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 20 op/s 2024-04-25T10:40:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.170+0000 7f9e8a986640 1 -- 172.21.15.43:0/2542235884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e84104d10 msgr2=0x7f9e84105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.170+0000 7f9e8a986640 1 --2- 172.21.15.43:0/2542235884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e84104d10 0x7f9e84105120 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009900 tx=0x7f9e7402a480 comp rx=0 tx=0).stop 2024-04-25T10:40:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.170+0000 7f9e8a986640 1 -- 172.21.15.43:0/2542235884 shutdown_connections 2024-04-25T10:40:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.170+0000 7f9e8a986640 1 --2- 172.21.15.43:0/2542235884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e8410b770 0x7f9e8410db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.170+0000 7f9e8a986640 1 --2- 172.21.15.43:0/2542235884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e8410e0d0 0x7f9e841105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.170+0000 7f9e8a986640 1 --2- 172.21.15.43:0/2542235884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e84104d10 0x7f9e84105120 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.170+0000 7f9e8a986640 1 -- 172.21.15.43:0/2542235884 >> 172.21.15.43:0/2542235884 conn(0x7f9e840781b0 msgr2=0x7f9e840785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.170+0000 7f9e8a986640 1 -- 172.21.15.43:0/2542235884 shutdown_connections 2024-04-25T10:40:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.171+0000 7f9e8a986640 1 -- 172.21.15.43:0/2542235884 wait complete. 2024-04-25T10:40:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.171+0000 7f9e8a986640 1 Processor -- start 2024-04-25T10:40:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.171+0000 7f9e8a986640 1 -- start start 2024-04-25T10:40:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.172+0000 7f9e8a986640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e84104d10 0x7f9e8419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.172+0000 7f9e8a986640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e8410b770 0x7f9e8419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.172+0000 7f9e8a986640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e8410e0d0 0x7f9e8419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.172+0000 7f9e8a986640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e84113d30 con 0x7f9e8410e0d0 2024-04-25T10:40:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.172+0000 7f9e8a986640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9e84113bb0 con 0x7f9e8410b770 2024-04-25T10:40:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.172+0000 7f9e8a986640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9e84113eb0 con 0x7f9e84104d10 2024-04-25T10:40:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.172+0000 7f9e88f23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e8410e0d0 0x7f9e8419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.172+0000 7f9e83fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e84104d10 0x7f9e8419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.172+0000 7f9e83fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e84104d10 0x7f9e8419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:59112/0 (socket says 172.21.15.43:59112) 2024-04-25T10:40:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.172+0000 7f9e83fff640 1 -- 172.21.15.43:0/3657657325 learned_addr learned my addr 172.21.15.43:0/3657657325 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.172+0000 7f9e837fe640 1 --2- 172.21.15.43:0/3657657325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e8410b770 0x7f9e8419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.172+0000 7f9e88f23640 1 -- 172.21.15.43:0/3657657325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e84104d10 msgr2=0x7f9e8419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.172+0000 7f9e88f23640 1 --2- 172.21.15.43:0/3657657325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e84104d10 0x7f9e8419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.172+0000 7f9e88f23640 1 -- 172.21.15.43:0/3657657325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e8410b770 msgr2=0x7f9e8419acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.172+0000 7f9e88f23640 1 --2- 172.21.15.43:0/3657657325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e8410b770 0x7f9e8419acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.172+0000 7f9e88f23640 1 -- 172.21.15.43:0/3657657325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8419f5b0 con 0x7f9e8410e0d0 2024-04-25T10:40:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.172+0000 7f9e83fff640 1 --2- 172.21.15.43:0/3657657325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e84104d10 0x7f9e8419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:40:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.173+0000 7f9e88f23640 1 --2- 172.21.15.43:0/3657657325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e8410e0d0 0x7f9e8419ef70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00e940 tx=0x7f9e7c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.173+0000 7f9e817fa640 1 -- 172.21.15.43:0/3657657325 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c004d20 con 0x7f9e8410e0d0 2024-04-25T10:40:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.173+0000 7f9e8a986640 1 -- 172.21.15.43:0/3657657325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8419f8a0 con 0x7f9e8410e0d0 2024-04-25T10:40:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.173+0000 7f9e817fa640 1 -- 172.21.15.43:0/3657657325 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e7c005020 con 0x7f9e8410e0d0 2024-04-25T10:40:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.173+0000 7f9e817fa640 1 -- 172.21.15.43:0/3657657325 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c0116d0 con 0x7f9e8410e0d0 2024-04-25T10:40:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.173+0000 7f9e8a986640 1 -- 172.21.15.43:0/3657657325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e841a7220 con 0x7f9e8410e0d0 2024-04-25T10:40:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.175+0000 7f9e9183f740 1 Processor -- start 2024-04-25T10:40:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.176+0000 7f9e817fa640 1 -- 172.21.15.43:0/3657657325 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f9e7c00cb20 con 0x7f9e8410e0d0 2024-04-25T10:40:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.176+0000 7f9e9183f740 1 -- start start 2024-04-25T10:40:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.176+0000 7f9e9183f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8521b0 0x55c62fbf1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.176+0000 7f9e9183f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8292f0 0x55c62fbf2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.176+0000 7f9e9183f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8314d0 0x55c62fbf2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.176+0000 7f9e9183f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c62fb58460 con 0x55c62f8292f0 2024-04-25T10:40:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.176+0000 7f9e9183f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c62fb12590 con 0x55c62f8521b0 2024-04-25T10:40:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.176+0000 7f9e9183f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c62fb54420 con 0x55c62f8314d0 2024-04-25T10:40:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.176+0000 7f9e83fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8292f0 0x55c62fbf2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.176+0000 7f9e83fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8292f0 0x55c62fbf2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53598/0 (socket says 172.21.15.43:53598) 2024-04-25T10:40:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.176+0000 7f9e83fff640 1 -- 172.21.15.43:0/2805424360 learned_addr learned my addr 172.21.15.43:0/2805424360 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.176+0000 7f9e837fe640 1 --2- 172.21.15.43:0/2805424360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8521b0 0x55c62fbf1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.176+0000 7f9e837fe640 1 --2- 172.21.15.43:0/2805424360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8314d0 0x55c62fbf2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.176+0000 7f9e817fa640 1 --2- 172.21.15.43:0/3657657325 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9e6c0afc50 0x7f9e6c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.176+0000 7f9e83fff640 1 -- 172.21.15.43:0/2805424360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8314d0 msgr2=0x55c62fbf2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.176+0000 7f9e83fff640 1 --2- 172.21.15.43:0/2805424360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8314d0 0x55c62fbf2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.176+0000 7f9e83fff640 1 -- 172.21.15.43:0/2805424360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8521b0 msgr2=0x55c62fbf1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.176+0000 7f9e83fff640 1 --2- 172.21.15.43:0/2805424360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8521b0 0x55c62fbf1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.176+0000 7f9e83fff640 1 -- 172.21.15.43:0/2805424360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c62fb1f0d0 con 0x55c62f8292f0 2024-04-25T10:40:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.176+0000 7f9e837fe640 1 --2- 172.21.15.43:0/2805424360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8521b0 0x55c62fbf1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.177+0000 7f9e88f23640 1 --2- 172.21.15.43:0/3657657325 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9e6c0afc50 0x7f9e6c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.177+0000 7f9e83fff640 1 --2- 172.21.15.43:0/2805424360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8292f0 0x55c62fbf2030 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f9e74005e00 tx=0x7f9e74031520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738a78e941e82ec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.177+0000 7f9e817fa640 1 -- 172.21.15.43:0/3657657325 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8557+0+0 (secure 0 0 0) 0x7f9e7c0df380 con 0x7f9e8410e0d0 2024-04-25T10:40:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.177+0000 7f9e6a7fc640 1 -- 172.21.15.43:0/2805424360 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74035040 con 0x55c62f8292f0 2024-04-25T10:40:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.177+0000 7f9e6a7fc640 1 -- 172.21.15.43:0/2805424360 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e74031910 con 0x55c62f8292f0 2024-04-25T10:40:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.177+0000 7f9e6a7fc640 1 -- 172.21.15.43:0/2805424360 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74004a00 con 0x55c62f8292f0 2024-04-25T10:40:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.178+0000 7f9e9183f740 1 -- 172.21.15.43:0/2805424360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8292f0 msgr2=0x55c62fbf2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.178+0000 7f9e88f23640 1 --2- 172.21.15.43:0/3657657325 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9e6c0afc50 0x7f9e6c0b2160 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c0a7540 tx=0x7f9e7c076040 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.178+0000 7f9e9183f740 1 --2- 172.21.15.43:0/2805424360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8292f0 0x55c62fbf2030 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f9e74005e00 tx=0x7f9e74031520 comp rx=0 tx=0).stop 2024-04-25T10:40:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.178+0000 7f9e9183f740 1 -- 172.21.15.43:0/2805424360 shutdown_connections 2024-04-25T10:40:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.178+0000 7f9e9183f740 1 --2- 172.21.15.43:0/2805424360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8314d0 0x55c62fbf2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.178+0000 7f9e9183f740 1 --2- 172.21.15.43:0/2805424360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8292f0 0x55c62fbf2030 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.178+0000 7f9e9183f740 1 --2- 172.21.15.43:0/2805424360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8521b0 0x55c62fbf1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.178+0000 7f9e9183f740 1 -- 172.21.15.43:0/2805424360 >> 172.21.15.43:0/2805424360 conn(0x55c62f846300 msgr2=0x55c62fbee2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.178+0000 7f9e9183f740 1 -- 172.21.15.43:0/2805424360 shutdown_connections 2024-04-25T10:40:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.178+0000 7f9e9183f740 1 -- 172.21.15.43:0/2805424360 wait complete. 2024-04-25T10:40:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.178+0000 7f9e9183f740 1 -- start start 2024-04-25T10:40:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.179+0000 7f9e9183f740 1 Processor -- start 2024-04-25T10:40:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.179+0000 7f9e9183f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8292f0 0x55c62fbfc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.179+0000 7f9e9183f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8314d0 0x55c62fbfc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.179+0000 7f9e9183f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8521b0 0x55c62fc020b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.179+0000 7f9e9183f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c62fbfebc0 con 0x55c62f8314d0 2024-04-25T10:40:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.179+0000 7f9e9183f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c62fadfb60 con 0x55c62f8292f0 2024-04-25T10:40:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.179+0000 7f9e9183f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c62fbfed40 con 0x55c62f8521b0 2024-04-25T10:40:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.179+0000 7f9e9183f740 10 client.0 _ll_register_callbacks cb 0x55c62fb042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.179+0000 7f9e83fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8314d0 0x55c62fbfc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.179+0000 7f9e83fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8314d0 0x55c62fbfc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53608/0 (socket says 172.21.15.43:53608) 2024-04-25T10:40:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.179+0000 7f9e83fff640 1 -- 172.21.15.43:0/830544265 learned_addr learned my addr 172.21.15.43:0/830544265 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.179+0000 7f9e837fe640 1 --2- 172.21.15.43:0/830544265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8521b0 0x55c62fc020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.179+0000 7f9e837fe640 1 --2- 172.21.15.43:0/830544265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8292f0 0x55c62fbfc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.180+0000 7f9e83fff640 1 -- 172.21.15.43:0/830544265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8521b0 msgr2=0x55c62fc020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.180+0000 7f9e83fff640 1 --2- 172.21.15.43:0/830544265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8521b0 0x55c62fc020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.180+0000 7f9e83fff640 1 -- 172.21.15.43:0/830544265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8292f0 msgr2=0x55c62fbfc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.180+0000 7f9e83fff640 1 --2- 172.21.15.43:0/830544265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8292f0 0x55c62fbfc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.180+0000 7f9e83fff640 1 -- 172.21.15.43:0/830544265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c62f86a210 con 0x55c62f8314d0 2024-04-25T10:40:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.180+0000 7f9e837fe640 1 --2- 172.21.15.43:0/830544265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8521b0 0x55c62fc020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.180+0000 7f9e83fff640 1 --2- 172.21.15.43:0/830544265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8314d0 0x55c62fbfc710 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f9e74031440 tx=0x7f9e74004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8d8b496534518 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.180+0000 7f9e637fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.180+0000 7f9e637fe640 1 -- 172.21.15.43:0/830544265 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7404e020 con 0x55c62f8314d0 2024-04-25T10:40:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.180+0000 7f9e9183f740 1 -- 172.21.15.43:0/830544265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e74031910 con 0x55c62f8314d0 2024-04-25T10:40:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.180+0000 7f9e637fe640 1 -- 172.21.15.43:0/830544265 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8557+0+0 (secure 0 0 0) 0x7f9e740035f0 con 0x55c62f8314d0 2024-04-25T10:40:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.180+0000 7f9e637fe640 1 -- 172.21.15.43:0/830544265 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9e74059050 con 0x55c62f8314d0 2024-04-25T10:40:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.180+0000 7f9e9183f740 10 client.17634 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.180+0000 7f9e9183f740 1 -- 172.21.15.43:0/830544265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c62f936620 con 0x55c62f8314d0 2024-04-25T10:40:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.180+0000 7f9e637fe640 1 -- 172.21.15.43:0/830544265 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e74031910 con 0x55c62f8314d0 2024-04-25T10:40:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.181+0000 7f9e637fe640 1 -- 172.21.15.43:0/830544265 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9e74046910 con 0x55c62f8314d0 2024-04-25T10:40:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.181+0000 7f9e9183f740 10 client.17634 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.181+0000 7f9e9183f740 10 client.17634 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.181+0000 7f9e9183f740 20 client.17634 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.181+0000 7f9e9183f740 1 --2- 172.21.15.43:0/830544265 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c62f810d80 0x55c62fc0e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.181+0000 7f9e9183f740 4 client.17634 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.181+0000 7f9e9183f740 1 -- 172.21.15.43:0/830544265 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c62fb66070 con 0x55c62f810d80 2024-04-25T10:40:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.181+0000 7f9e837fe640 1 --2- 172.21.15.43:0/830544265 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c62f810d80 0x55c62fc0e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.181+0000 7f9e637fe640 10 client.17634 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.181+0000 7f9e837fe640 1 --2- 172.21.15.43:0/830544265 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c62f810d80 0x55c62fc0e570 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2df4c6d38ae5b2 server_cookie=e66684200f424fb8 in_seq=0 out_seq=0 2024-04-25T10:40:11.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:40:11.192 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:40:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.182+0000 7f9e637fe640 1 -- 172.21.15.43:0/830544265 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c62fb66070 con 0x55c62f810d80 2024-04-25T10:40:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.182+0000 7f9e637fe640 10 client.17634 handle_command_reply: tid=0 2024-04-25T10:40:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.182+0000 7f9e9183f740 1 client.17634 shutdown 2024-04-25T10:40:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.183+0000 7f9e9183f740 1 -- 172.21.15.43:0/830544265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8314d0 msgr2=0x55c62fbfc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.183+0000 7f9e9183f740 1 --2- 172.21.15.43:0/830544265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8314d0 0x55c62fbfc710 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f9e74031440 tx=0x7f9e74004270 comp rx=0 tx=0).stop 2024-04-25T10:40:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.183+0000 7f9e9183f740 1 -- 172.21.15.43:0/830544265 shutdown_connections 2024-04-25T10:40:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.183+0000 7f9e9183f740 1 --2- 172.21.15.43:0/830544265 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c62f810d80 0x55c62fc0e570 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.183+0000 7f9e9183f740 1 --2- 172.21.15.43:0/830544265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8521b0 0x55c62fc020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.183+0000 7f9e9183f740 1 --2- 172.21.15.43:0/830544265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8314d0 0x55c62fbfc710 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.183+0000 7f9e637fe640 0 client.17634 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.183+0000 7f9e9183f740 1 --2- 172.21.15.43:0/830544265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8292f0 0x55c62fbfc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.183+0000 7f9e9183f740 1 -- 172.21.15.43:0/830544265 >> 172.21.15.43:0/830544265 conn(0x55c62f846300 msgr2=0x55c62fbf3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.183+0000 7f9e9183f740 1 -- 172.21.15.43:0/830544265 shutdown_connections 2024-04-25T10:40:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.183+0000 7f9e9183f740 1 -- 172.21.15.43:0/830544265 wait complete. 2024-04-25T10:40:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.183+0000 7f9e9183f740 20 client.17634 trim_cache size 0 max 16384 2024-04-25T10:40:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.198+0000 7f9e9183f740 1 Processor -- start 2024-04-25T10:40:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.198+0000 7f9e9183f740 1 -- start start 2024-04-25T10:40:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.199+0000 7f9e9183f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8292f0 0x55c62fbfe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.199+0000 7f9e9183f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8314d0 0x55c62fbf3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.199+0000 7f9e9183f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8521b0 0x55c62fc01fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.199+0000 7f9e9183f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e841b23d0 con 0x55c62f8521b0 2024-04-25T10:40:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.199+0000 7f9e9183f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9e841afca0 con 0x55c62f8292f0 2024-04-25T10:40:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.199+0000 7f9e9183f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9e841adb10 con 0x55c62f8314d0 2024-04-25T10:40:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.199+0000 7f9e837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8521b0 0x55c62fc01fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.199+0000 7f9e837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8521b0 0x55c62fc01fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53616/0 (socket says 172.21.15.43:53616) 2024-04-25T10:40:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.199+0000 7f9e837fe640 1 -- 172.21.15.43:0/3047129255 learned_addr learned my addr 172.21.15.43:0/3047129255 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.199+0000 7f9e83fff640 1 --2- 172.21.15.43:0/3047129255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8314d0 0x55c62fbf3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.199+0000 7f9e837fe640 1 --2- 172.21.15.43:0/3047129255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8292f0 0x55c62fbfe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.199+0000 7f9e83fff640 1 -- 172.21.15.43:0/3047129255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8292f0 msgr2=0x55c62fbfe490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.199+0000 7f9e83fff640 1 --2- 172.21.15.43:0/3047129255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8292f0 0x55c62fbfe490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.199+0000 7f9e83fff640 1 -- 172.21.15.43:0/3047129255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8521b0 msgr2=0x55c62fc01fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.199+0000 7f9e83fff640 1 --2- 172.21.15.43:0/3047129255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8521b0 0x55c62fc01fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.199+0000 7f9e83fff640 1 -- 172.21.15.43:0/3047129255 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c62fbeee90 con 0x55c62f8314d0 2024-04-25T10:40:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.199+0000 7f9e837fe640 1 --2- 172.21.15.43:0/3047129255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8521b0 0x55c62fc01fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:40:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.200+0000 7f9e83fff640 1 --2- 172.21.15.43:0/3047129255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8314d0 0x55c62fbf3eb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9e740092a0 tx=0x7f9e74005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a08c10c895162554 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.200+0000 7f9e69ffb640 1 -- 172.21.15.43:0/3047129255 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74062070 con 0x55c62f8314d0 2024-04-25T10:40:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.200+0000 7f9e69ffb640 1 -- 172.21.15.43:0/3047129255 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e74009520 con 0x55c62f8314d0 2024-04-25T10:40:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.200+0000 7f9e69ffb640 1 -- 172.21.15.43:0/3047129255 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7404e030 con 0x55c62f8314d0 2024-04-25T10:40:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.201+0000 7f9e9183f740 1 -- 172.21.15.43:0/3047129255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8314d0 msgr2=0x55c62fbf3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.201+0000 7f9e9183f740 1 --2- 172.21.15.43:0/3047129255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8314d0 0x55c62fbf3eb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9e740092a0 tx=0x7f9e74005fd0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.201+0000 7f9e9183f740 1 -- 172.21.15.43:0/3047129255 shutdown_connections 2024-04-25T10:40:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.201+0000 7f9e9183f740 1 --2- 172.21.15.43:0/3047129255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8521b0 0x55c62fc01fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.201+0000 7f9e9183f740 1 --2- 172.21.15.43:0/3047129255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8314d0 0x55c62fbf3eb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.201+0000 7f9e9183f740 1 --2- 172.21.15.43:0/3047129255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8292f0 0x55c62fbfe490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.201+0000 7f9e9183f740 1 -- 172.21.15.43:0/3047129255 >> 172.21.15.43:0/3047129255 conn(0x55c62f846300 msgr2=0x55c62fbf37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.201+0000 7f9e9183f740 1 -- 172.21.15.43:0/3047129255 shutdown_connections 2024-04-25T10:40:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.201+0000 7f9e9183f740 1 -- 172.21.15.43:0/3047129255 wait complete. 2024-04-25T10:40:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.201+0000 7f9e9183f740 1 -- start start 2024-04-25T10:40:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.202+0000 7f9e9183f740 1 Processor -- start 2024-04-25T10:40:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.202+0000 7f9e9183f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8292f0 0x55c62fc05340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.202+0000 7f9e9183f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8314d0 0x55c62fc05880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.202+0000 7f9e9183f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8521b0 0x55c62fc74690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.202+0000 7f9e9183f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e841ae160 con 0x55c62f8314d0 2024-04-25T10:40:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.202+0000 7f9e9183f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9e841af5d0 con 0x55c62f8521b0 2024-04-25T10:40:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.202+0000 7f9e9183f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9e841ae830 con 0x55c62f8292f0 2024-04-25T10:40:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.202+0000 7f9e9183f740 10 client.0 _ll_register_callbacks cb 0x7f9e74040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.202+0000 7f9e83fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8314d0 0x55c62fc05880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.202+0000 7f9e83fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8314d0 0x55c62fc05880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53628/0 (socket says 172.21.15.43:53628) 2024-04-25T10:40:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.202+0000 7f9e83fff640 1 -- 172.21.15.43:0/343826589 learned_addr learned my addr 172.21.15.43:0/343826589 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.202+0000 7f9e837fe640 1 --2- 172.21.15.43:0/343826589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8292f0 0x55c62fc05340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.202+0000 7f9e837fe640 1 --2- 172.21.15.43:0/343826589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8521b0 0x55c62fc74690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.202+0000 7f9e83fff640 1 -- 172.21.15.43:0/343826589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8292f0 msgr2=0x55c62fc05340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.202+0000 7f9e83fff640 1 --2- 172.21.15.43:0/343826589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8292f0 0x55c62fc05340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.202+0000 7f9e83fff640 1 -- 172.21.15.43:0/343826589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8521b0 msgr2=0x55c62fc74690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.202+0000 7f9e83fff640 1 --2- 172.21.15.43:0/343826589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8521b0 0x55c62fc74690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.202+0000 7f9e83fff640 1 -- 172.21.15.43:0/343826589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c62fc74bd0 con 0x55c62f8314d0 2024-04-25T10:40:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.203+0000 7f9e83fff640 1 --2- 172.21.15.43:0/343826589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8314d0 0x55c62fc05880 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f9e74046ed0 tx=0x7f9e740315a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ad816a35614447 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.203+0000 7f9e637fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.203+0000 7f9e637fe640 1 -- 172.21.15.43:0/343826589 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74062040 con 0x55c62f8314d0 2024-04-25T10:40:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.203+0000 7f9e637fe640 1 -- 172.21.15.43:0/343826589 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8557+0+0 (secure 0 0 0) 0x7f9e74046a80 con 0x55c62f8314d0 2024-04-25T10:40:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.203+0000 7f9e9183f740 1 -- 172.21.15.43:0/343826589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e74009520 con 0x55c62f8314d0 2024-04-25T10:40:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.203+0000 7f9e637fe640 1 -- 172.21.15.43:0/343826589 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9e7404e040 con 0x55c62f8314d0 2024-04-25T10:40:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.203+0000 7f9e9183f740 10 client.17640 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.203+0000 7f9e9183f740 1 -- 172.21.15.43:0/343826589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c62fc75cb0 con 0x55c62f8314d0 2024-04-25T10:40:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.203+0000 7f9e637fe640 1 -- 172.21.15.43:0/343826589 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e74009520 con 0x55c62f8314d0 2024-04-25T10:40:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.204+0000 7f9e637fe640 1 -- 172.21.15.43:0/343826589 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9e740037b0 con 0x55c62f8314d0 2024-04-25T10:40:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.204+0000 7f9e9183f740 10 client.17640 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.204+0000 7f9e9183f740 10 client.17640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.204+0000 7f9e9183f740 20 client.17640 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.204+0000 7f9e9183f740 1 --2- 172.21.15.43:0/343826589 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c62f7e61c0 0x55c62fc78430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.204+0000 7f9e9183f740 4 client.17640 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:40:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.204+0000 7f9e9183f740 1 -- 172.21.15.43:0/343826589 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c62fb5fee0 con 0x55c62f7e61c0 2024-04-25T10:40:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.204+0000 7f9e837fe640 1 --2- 172.21.15.43:0/343826589 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c62f7e61c0 0x55c62fc78430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.204+0000 7f9e637fe640 10 client.17640 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.204+0000 7f9e837fe640 1 --2- 172.21.15.43:0/343826589 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c62f7e61c0 0x55c62fc78430 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a52b5c3eeec334f server_cookie=629536955a11b36f in_seq=0 out_seq=0 2024-04-25T10:40:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.221+0000 7f9e637fe640 1 -- 172.21.15.43:0/343826589 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c62fb5fee0 con 0x55c62f7e61c0 2024-04-25T10:40:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.221+0000 7f9e637fe640 10 client.17640 handle_command_reply: tid=0 2024-04-25T10:40:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.221+0000 7f9e9183f740 1 client.17640 shutdown 2024-04-25T10:40:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.221+0000 7f9e9183f740 1 -- 172.21.15.43:0/343826589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8314d0 msgr2=0x55c62fc05880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.221+0000 7f9e9183f740 1 --2- 172.21.15.43:0/343826589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8314d0 0x55c62fc05880 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f9e74046ed0 tx=0x7f9e740315a0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.222+0000 7f9e9183f740 1 -- 172.21.15.43:0/343826589 shutdown_connections 2024-04-25T10:40:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.222+0000 7f9e9183f740 1 --2- 172.21.15.43:0/343826589 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c62f7e61c0 0x55c62fc78430 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.222+0000 7f9e9183f740 1 --2- 172.21.15.43:0/343826589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c62f8521b0 0x55c62fc74690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.222+0000 7f9e9183f740 1 --2- 172.21.15.43:0/343826589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62f8314d0 0x55c62fc05880 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.222+0000 7f9e9183f740 1 --2- 172.21.15.43:0/343826589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c62f8292f0 0x55c62fc05340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.222+0000 7f9e637fe640 0 client.17640 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.222+0000 7f9e9183f740 1 -- 172.21.15.43:0/343826589 >> 172.21.15.43:0/343826589 conn(0x55c62f846300 msgr2=0x55c62fbfd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.222+0000 7f9e9183f740 1 -- 172.21.15.43:0/343826589 shutdown_connections 2024-04-25T10:40:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.222+0000 7f9e9183f740 1 -- 172.21.15.43:0/343826589 wait complete. 2024-04-25T10:40:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.222+0000 7f9e9183f740 20 client.17640 trim_cache size 0 max 16384 2024-04-25T10:40:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:40:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:40:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:40:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:40:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.222+0000 7f9e8a986640 1 -- 172.21.15.43:0/3657657325 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9e6c0afc50 msgr2=0x7f9e6c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.222+0000 7f9e8a986640 1 --2- 172.21.15.43:0/3657657325 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9e6c0afc50 0x7f9e6c0b2160 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c0a7540 tx=0x7f9e7c076040 comp rx=0 tx=0).stop 2024-04-25T10:40:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.222+0000 7f9e8a986640 1 -- 172.21.15.43:0/3657657325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e8410e0d0 msgr2=0x7f9e8419ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.222+0000 7f9e8a986640 1 --2- 172.21.15.43:0/3657657325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e8410e0d0 0x7f9e8419ef70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00e940 tx=0x7f9e7c00ee00 comp rx=0 tx=0).stop 2024-04-25T10:40:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.223+0000 7f9e8a986640 1 -- 172.21.15.43:0/3657657325 shutdown_connections 2024-04-25T10:40:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.223+0000 7f9e8a986640 1 --2- 172.21.15.43:0/3657657325 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9e6c0afc50 0x7f9e6c0b2160 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.223+0000 7f9e8a986640 1 --2- 172.21.15.43:0/3657657325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e8410b770 0x7f9e8419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.223+0000 7f9e8a986640 1 --2- 172.21.15.43:0/3657657325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e8410e0d0 0x7f9e8419ef70 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.223+0000 7f9e8a986640 1 --2- 172.21.15.43:0/3657657325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e84104d10 0x7f9e8419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.223+0000 7f9e8a986640 1 -- 172.21.15.43:0/3657657325 >> 172.21.15.43:0/3657657325 conn(0x7f9e840781b0 msgr2=0x7f9e84101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.223+0000 7f9e8a986640 1 -- 172.21.15.43:0/3657657325 shutdown_connections 2024-04-25T10:40:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:11.223+0000 7f9e8a986640 1 -- 172.21.15.43:0/3657657325 wait complete. 2024-04-25T10:40:11.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.311+0000 7f46fb242640 1 -- 172.21.15.43:0/1936078564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46f410d3a0 msgr2=0x7f46f410f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.311+0000 7f46fb242640 1 --2- 172.21.15.43:0/1936078564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46f410d3a0 0x7f46f410f7c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009ca0 tx=0x7f46ec02a8e0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.311+0000 7f46fb242640 1 -- 172.21.15.43:0/1936078564 shutdown_connections 2024-04-25T10:40:11.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.311+0000 7f46fb242640 1 --2- 172.21.15.43:0/1936078564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46f410d3a0 0x7f46f410f7c0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.311+0000 7f46fb242640 1 --2- 172.21.15.43:0/1936078564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46f4102b10 0x7f46f4102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.311+0000 7f46fb242640 1 --2- 172.21.15.43:0/1936078564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f4108a40 0x7f46f41025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.311+0000 7f46fb242640 1 -- 172.21.15.43:0/1936078564 >> 172.21.15.43:0/1936078564 conn(0x7f46f4078180 msgr2=0x7f46f40785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:11.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.311+0000 7f46fb242640 1 -- 172.21.15.43:0/1936078564 shutdown_connections 2024-04-25T10:40:11.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.311+0000 7f46fb242640 1 -- 172.21.15.43:0/1936078564 wait complete. 2024-04-25T10:40:11.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.312+0000 7f46fb242640 1 Processor -- start 2024-04-25T10:40:11.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.312+0000 7f46fb242640 1 -- start start 2024-04-25T10:40:11.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.312+0000 7f46fb242640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46f4102b10 0x7f46f419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.312+0000 7f46fb242640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f4108a40 0x7f46f419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.312+0000 7f46fb242640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46f410d3a0 0x7f46f419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.312+0000 7f46fb242640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f46f406b8a0 con 0x7f46f4102b10 2024-04-25T10:40:11.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.312+0000 7f46fb242640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f46f406b720 con 0x7f46f4108a40 2024-04-25T10:40:11.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.312+0000 7f46fb242640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f46f406ba20 con 0x7f46f410d3a0 2024-04-25T10:40:11.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.312+0000 7f46f8fde640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46f4102b10 0x7f46f419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.312+0000 7f46f8fde640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46f4102b10 0x7f46f419a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53648/0 (socket says 172.21.15.43:53648) 2024-04-25T10:40:11.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.312+0000 7f46f8fde640 1 -- 172.21.15.43:0/4058025097 learned_addr learned my addr 172.21.15.43:0/4058025097 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:11.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.312+0000 7f46f3fff640 1 --2- 172.21.15.43:0/4058025097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f4108a40 0x7f46f419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.312+0000 7f46f97df640 1 --2- 172.21.15.43:0/4058025097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46f410d3a0 0x7f46f419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.312+0000 7f46f8fde640 1 -- 172.21.15.43:0/4058025097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46f410d3a0 msgr2=0x7f46f419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.312+0000 7f46f8fde640 1 --2- 172.21.15.43:0/4058025097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46f410d3a0 0x7f46f419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.312+0000 7f46f8fde640 1 -- 172.21.15.43:0/4058025097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f4108a40 msgr2=0x7f46f419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.312+0000 7f46f8fde640 1 --2- 172.21.15.43:0/4058025097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f4108a40 0x7f46f419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.312+0000 7f46f8fde640 1 -- 172.21.15.43:0/4058025097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f419f610 con 0x7f46f4102b10 2024-04-25T10:40:11.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.312+0000 7f46f3fff640 1 --2- 172.21.15.43:0/4058025097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f4108a40 0x7f46f419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:11.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.313+0000 7f46f8fde640 1 --2- 172.21.15.43:0/4058025097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46f4102b10 0x7f46f419a7b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f46e400b450 tx=0x7f46e400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:11.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.313+0000 7f46f1ffb640 1 -- 172.21.15.43:0/4058025097 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4015020 con 0x7f46f4102b10 2024-04-25T10:40:11.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.313+0000 7f46f1ffb640 1 -- 172.21.15.43:0/4058025097 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46e4011d40 con 0x7f46f4102b10 2024-04-25T10:40:11.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.313+0000 7f46fb242640 1 -- 172.21.15.43:0/4058025097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f419f900 con 0x7f46f4102b10 2024-04-25T10:40:11.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.313+0000 7f46f1ffb640 1 -- 172.21.15.43:0/4058025097 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e400f340 con 0x7f46f4102b10 2024-04-25T10:40:11.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.313+0000 7f46fb242640 1 -- 172.21.15.43:0/4058025097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f41a7280 con 0x7f46f4102b10 2024-04-25T10:40:11.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.316+0000 7f46f1ffb640 1 -- 172.21.15.43:0/4058025097 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f46e4018020 con 0x7f46f4102b10 2024-04-25T10:40:11.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.316+0000 7f47021d0740 1 Processor -- start 2024-04-25T10:40:11.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.316+0000 7f47021d0740 1 -- start start 2024-04-25T10:40:11.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.316+0000 7f47021d0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 0x556552874a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.316+0000 7f47021d0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565524ab4d0 0x556552874fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.316+0000 7f47021d0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565524b36b0 0x5565528818f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.316+0000 7f47021d0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5565527dad80 con 0x5565524d4390 2024-04-25T10:40:11.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.316+0000 7f47021d0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5565527948b0 con 0x5565524b36b0 2024-04-25T10:40:11.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.316+0000 7f47021d0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5565527d6d40 con 0x5565524ab4d0 2024-04-25T10:40:11.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.316+0000 7f46f97df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 0x556552874a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.316+0000 7f46f97df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 0x556552874a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53650/0 (socket says 172.21.15.43:53650) 2024-04-25T10:40:11.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.316+0000 7f46f97df640 1 -- 172.21.15.43:0/2949616613 learned_addr learned my addr 172.21.15.43:0/2949616613 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:11.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.316+0000 7f46f1ffb640 1 --2- 172.21.15.43:0/4058025097 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f46dc0afc60 0x7f46dc0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.316+0000 7f46f3fff640 1 --2- 172.21.15.43:0/2949616613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565524b36b0 0x5565528818f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.317+0000 7f46f97df640 1 --2- 172.21.15.43:0/4058025097 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f46dc0afc60 0x7f46dc0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.317+0000 7f46f1ffb640 1 -- 172.21.15.43:0/4058025097 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8557+0+0 (secure 0 0 0) 0x7f46e40de9f0 con 0x7f46f4102b10 2024-04-25T10:40:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.317+0000 7f46f8fde640 1 --2- 172.21.15.43:0/2949616613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565524ab4d0 0x556552874fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.317+0000 7f46f97df640 1 -- 172.21.15.43:0/2949616613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565524ab4d0 msgr2=0x556552874fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.317+0000 7f46f97df640 1 --2- 172.21.15.43:0/2949616613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565524ab4d0 0x556552874fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.317+0000 7f46f97df640 1 -- 172.21.15.43:0/2949616613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565524b36b0 msgr2=0x5565528818f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.317+0000 7f46f97df640 1 --2- 172.21.15.43:0/2949616613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565524b36b0 0x5565528818f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.317+0000 7f46f97df640 1 -- 172.21.15.43:0/2949616613 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556552765a40 con 0x5565524d4390 2024-04-25T10:40:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.317+0000 7f46f97df640 1 --2- 172.21.15.43:0/2949616613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 0x556552874a60 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f46ec031a00 tx=0x7f46ec031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404bc8aa5d9847b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.317+0000 7f46daffd640 1 -- 172.21.15.43:0/2949616613 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec004b40 con 0x5565524d4390 2024-04-25T10:40:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.317+0000 7f46daffd640 1 -- 172.21.15.43:0/2949616613 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46ec036070 con 0x5565524d4390 2024-04-25T10:40:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.317+0000 7f46daffd640 1 -- 172.21.15.43:0/2949616613 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec030960 con 0x5565524d4390 2024-04-25T10:40:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.317+0000 7f46f97df640 1 --2- 172.21.15.43:0/4058025097 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f46dc0afc60 0x7f46dc0b2170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f46ec04d000 tx=0x7f46ec04b000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.318+0000 7f47021d0740 1 -- 172.21.15.43:0/2949616613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 msgr2=0x556552874a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.318+0000 7f47021d0740 1 --2- 172.21.15.43:0/2949616613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 0x556552874a60 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f46ec031a00 tx=0x7f46ec031a30 comp rx=0 tx=0).stop 2024-04-25T10:40:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.318+0000 7f47021d0740 1 -- 172.21.15.43:0/2949616613 shutdown_connections 2024-04-25T10:40:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.318+0000 7f47021d0740 1 --2- 172.21.15.43:0/2949616613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565524b36b0 0x5565528818f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.318+0000 7f47021d0740 1 --2- 172.21.15.43:0/2949616613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565524ab4d0 0x556552874fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.318+0000 7f47021d0740 1 --2- 172.21.15.43:0/2949616613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 0x556552874a60 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.318+0000 7f47021d0740 1 -- 172.21.15.43:0/2949616613 >> 172.21.15.43:0/2949616613 conn(0x5565524c84e0 msgr2=0x556552871210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.318+0000 7f47021d0740 1 -- 172.21.15.43:0/2949616613 shutdown_connections 2024-04-25T10:40:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.318+0000 7f47021d0740 1 -- 172.21.15.43:0/2949616613 wait complete. 2024-04-25T10:40:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.318+0000 7f47021d0740 1 -- start start 2024-04-25T10:40:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.319+0000 7f47021d0740 1 Processor -- start 2024-04-25T10:40:11.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.319+0000 7f47021d0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565524ab4d0 0x55655287f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.319+0000 7f47021d0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565524b36b0 0x55655287fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.319+0000 7f47021d0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 0x556552885430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.319+0000 7f47021d0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556552881fe0 con 0x5565524d4390 2024-04-25T10:40:11.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.319+0000 7f47021d0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556552761d10 con 0x5565524b36b0 2024-04-25T10:40:11.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.319+0000 7f47021d0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556552882160 con 0x5565524ab4d0 2024-04-25T10:40:11.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.319+0000 7f47021d0740 10 client.0 _ll_register_callbacks cb 0x55655278e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:11.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.319+0000 7f46f97df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 0x556552885430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.319+0000 7f46f97df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 0x556552885430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53660/0 (socket says 172.21.15.43:53660) 2024-04-25T10:40:11.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.319+0000 7f46f97df640 1 -- 172.21.15.43:0/1897533315 learned_addr learned my addr 172.21.15.43:0/1897533315 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:11.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.319+0000 7f46f8fde640 1 --2- 172.21.15.43:0/1897533315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565524ab4d0 0x55655287f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.319+0000 7f46f3fff640 1 --2- 172.21.15.43:0/1897533315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565524b36b0 0x55655287fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.319+0000 7f46f97df640 1 -- 172.21.15.43:0/1897533315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565524ab4d0 msgr2=0x55655287f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.319+0000 7f46f97df640 1 --2- 172.21.15.43:0/1897533315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565524ab4d0 0x55655287f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.319+0000 7f46f97df640 1 -- 172.21.15.43:0/1897533315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565524b36b0 msgr2=0x55655287fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.319+0000 7f46f97df640 1 --2- 172.21.15.43:0/1897533315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565524b36b0 0x55655287fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.319+0000 7f46f97df640 1 -- 172.21.15.43:0/1897533315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565525b67d0 con 0x5565524d4390 2024-04-25T10:40:11.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.320+0000 7f46f97df640 1 --2- 172.21.15.43:0/1897533315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 0x556552885430 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f46ec02ae00 tx=0x7f46ec02af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c436e3884e4fb711 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:11.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.320+0000 7f46d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:11.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.320+0000 7f46d3fff640 1 -- 172.21.15.43:0/1897533315 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec030940 con 0x5565524d4390 2024-04-25T10:40:11.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.320+0000 7f46d3fff640 1 -- 172.21.15.43:0/1897533315 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8557+0+0 (secure 0 0 0) 0x7f46ec003e00 con 0x5565524d4390 2024-04-25T10:40:11.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.320+0000 7f47021d0740 1 -- 172.21.15.43:0/1897533315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46ec036070 con 0x5565524d4390 2024-04-25T10:40:11.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.320+0000 7f46d3fff640 1 -- 172.21.15.43:0/1897533315 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f46ec004410 con 0x5565524d4390 2024-04-25T10:40:11.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.320+0000 7f47021d0740 10 client.17652 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:11.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.320+0000 7f47021d0740 1 -- 172.21.15.43:0/1897533315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5565524ec3f0 con 0x5565524d4390 2024-04-25T10:40:11.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.320+0000 7f46d3fff640 1 -- 172.21.15.43:0/1897533315 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46ec036070 con 0x5565524d4390 2024-04-25T10:40:11.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.320+0000 7f46d3fff640 1 -- 172.21.15.43:0/1897533315 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f46ec049990 con 0x5565524d4390 2024-04-25T10:40:11.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.320+0000 7f47021d0740 10 client.17652 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:11.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.320+0000 7f47021d0740 10 client.17652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:11.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.320+0000 7f47021d0740 20 client.17652 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:11.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.321+0000 7f47021d0740 1 --2- 172.21.15.43:0/1897533315 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556552492f60 0x5565528914c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.321+0000 7f47021d0740 4 client.17652 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:11.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.321+0000 7f47021d0740 1 -- 172.21.15.43:0/1897533315 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565527e8990 con 0x556552492f60 2024-04-25T10:40:11.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.321+0000 7f46f8fde640 1 --2- 172.21.15.43:0/1897533315 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556552492f60 0x5565528914c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.321+0000 7f46d3fff640 10 client.17652 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:11.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.321+0000 7f46f8fde640 1 --2- 172.21.15.43:0/1897533315 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556552492f60 0x5565528914c0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6329d0b80fe62b26 server_cookie=31f2efcfc0f178a9 in_seq=0 out_seq=0 2024-04-25T10:40:11.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.322+0000 7f46d3fff640 1 -- 172.21.15.43:0/1897533315 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5565527e8990 con 0x556552492f60 2024-04-25T10:40:11.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.322+0000 7f46d3fff640 10 client.17652 handle_command_reply: tid=0 2024-04-25T10:40:11.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.322+0000 7f47021d0740 1 client.17652 shutdown 2024-04-25T10:40:11.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.322+0000 7f47021d0740 1 -- 172.21.15.43:0/1897533315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 msgr2=0x556552885430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.322+0000 7f47021d0740 1 --2- 172.21.15.43:0/1897533315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 0x556552885430 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f46ec02ae00 tx=0x7f46ec02af60 comp rx=0 tx=0).stop 2024-04-25T10:40:11.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.322+0000 7f47021d0740 1 -- 172.21.15.43:0/1897533315 shutdown_connections 2024-04-25T10:40:11.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.322+0000 7f47021d0740 1 --2- 172.21.15.43:0/1897533315 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x556552492f60 0x5565528914c0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.322+0000 7f47021d0740 1 --2- 172.21.15.43:0/1897533315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565524b36b0 0x55655287fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.322+0000 7f47021d0740 1 --2- 172.21.15.43:0/1897533315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 0x556552885430 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.322+0000 7f47021d0740 1 --2- 172.21.15.43:0/1897533315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565524ab4d0 0x55655287f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.322+0000 7f47021d0740 1 -- 172.21.15.43:0/1897533315 >> 172.21.15.43:0/1897533315 conn(0x5565524c84e0 msgr2=0x556552876780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:11.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.322+0000 7f46d3fff640 0 client.17652 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:11.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.323+0000 7f47021d0740 1 -- 172.21.15.43:0/1897533315 shutdown_connections 2024-04-25T10:40:11.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.323+0000 7f47021d0740 1 -- 172.21.15.43:0/1897533315 wait complete. 2024-04-25T10:40:11.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.323+0000 7f47021d0740 20 client.17652 trim_cache size 0 max 16384 2024-04-25T10:40:11.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.341+0000 7f47021d0740 1 Processor -- start 2024-04-25T10:40:11.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.341+0000 7f47021d0740 1 -- start start 2024-04-25T10:40:11.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.341+0000 7f47021d0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565524ab4d0 0x5565528709d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.341+0000 7f47021d0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524b36b0 0x556552870f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.341+0000 7f47021d0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565524d4390 0x55655287c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.341+0000 7f47021d0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41b2250 con 0x5565524b36b0 2024-04-25T10:40:11.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.341+0000 7f47021d0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41afb20 con 0x5565524d4390 2024-04-25T10:40:11.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.341+0000 7f47021d0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41ad990 con 0x5565524ab4d0 2024-04-25T10:40:11.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.341+0000 7f46f3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524b36b0 0x556552870f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.341+0000 7f46f3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524b36b0 0x556552870f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53666/0 (socket says 172.21.15.43:53666) 2024-04-25T10:40:11.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.341+0000 7f46f3fff640 1 -- 172.21.15.43:0/596807108 learned_addr learned my addr 172.21.15.43:0/596807108 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:11.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.342+0000 7f46f97df640 1 --2- 172.21.15.43:0/596807108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565524d4390 0x55655287c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.342+0000 7f46f8fde640 1 --2- 172.21.15.43:0/596807108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565524ab4d0 0x5565528709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.342+0000 7f46f3fff640 1 -- 172.21.15.43:0/596807108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565524ab4d0 msgr2=0x5565528709d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.342+0000 7f46f3fff640 1 --2- 172.21.15.43:0/596807108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565524ab4d0 0x5565528709d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.342+0000 7f46f3fff640 1 -- 172.21.15.43:0/596807108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565524d4390 msgr2=0x55655287c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.342+0000 7f46f3fff640 1 --2- 172.21.15.43:0/596807108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565524d4390 0x55655287c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.342+0000 7f46f3fff640 1 -- 172.21.15.43:0/596807108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565524ac0f0 con 0x5565524b36b0 2024-04-25T10:40:11.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.342+0000 7f46f3fff640 1 --2- 172.21.15.43:0/596807108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524b36b0 0x556552870f10 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f46e8007970 tx=0x7f46e8007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe499ce12c9294a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:11.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.342+0000 7f46da7fc640 1 -- 172.21.15.43:0/596807108 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8019070 con 0x5565524b36b0 2024-04-25T10:40:11.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.342+0000 7f46da7fc640 1 -- 172.21.15.43:0/596807108 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46e801e460 con 0x5565524b36b0 2024-04-25T10:40:11.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.342+0000 7f46da7fc640 1 -- 172.21.15.43:0/596807108 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8015680 con 0x5565524b36b0 2024-04-25T10:40:11.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.343+0000 7f47021d0740 1 -- 172.21.15.43:0/596807108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524b36b0 msgr2=0x556552870f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.343+0000 7f47021d0740 1 --2- 172.21.15.43:0/596807108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524b36b0 0x556552870f10 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f46e8007970 tx=0x7f46e8007e30 comp rx=0 tx=0).stop 2024-04-25T10:40:11.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.343+0000 7f47021d0740 1 -- 172.21.15.43:0/596807108 shutdown_connections 2024-04-25T10:40:11.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.343+0000 7f47021d0740 1 --2- 172.21.15.43:0/596807108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565524d4390 0x55655287c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.343+0000 7f47021d0740 1 --2- 172.21.15.43:0/596807108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524b36b0 0x556552870f10 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.343+0000 7f47021d0740 1 --2- 172.21.15.43:0/596807108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565524ab4d0 0x5565528709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.343+0000 7f47021d0740 1 -- 172.21.15.43:0/596807108 >> 172.21.15.43:0/596807108 conn(0x5565524c84e0 msgr2=0x556552876660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:11.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.343+0000 7f47021d0740 1 -- 172.21.15.43:0/596807108 shutdown_connections 2024-04-25T10:40:11.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.343+0000 7f47021d0740 1 -- 172.21.15.43:0/596807108 wait complete. 2024-04-25T10:40:11.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.343+0000 7f47021d0740 1 -- start start 2024-04-25T10:40:11.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.343+0000 7f47021d0740 1 Processor -- start 2024-04-25T10:40:11.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.344+0000 7f47021d0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565524ab4d0 0x556552888570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.344+0000 7f47021d0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565524b36b0 0x556552888ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.344+0000 7f47021d0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 0x556552888ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.344+0000 7f47021d0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41adfe0 con 0x5565524d4390 2024-04-25T10:40:11.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.344+0000 7f47021d0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41af450 con 0x5565524b36b0 2024-04-25T10:40:11.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.344+0000 7f47021d0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41ae6b0 con 0x5565524ab4d0 2024-04-25T10:40:11.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.344+0000 7f47021d0740 10 client.0 _ll_register_callbacks cb 0x7f46e80071a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:11.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.344+0000 7f46f97df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 0x556552888ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.344+0000 7f46f97df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 0x556552888ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53674/0 (socket says 172.21.15.43:53674) 2024-04-25T10:40:11.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.344+0000 7f46f97df640 1 -- 172.21.15.43:0/1325889364 learned_addr learned my addr 172.21.15.43:0/1325889364 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:11.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.344+0000 7f46f3fff640 1 --2- 172.21.15.43:0/1325889364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565524b36b0 0x556552888ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.344+0000 7f46f8fde640 1 --2- 172.21.15.43:0/1325889364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565524ab4d0 0x556552888570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.344+0000 7f46f97df640 1 -- 172.21.15.43:0/1325889364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565524ab4d0 msgr2=0x556552888570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.344+0000 7f46f97df640 1 --2- 172.21.15.43:0/1325889364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565524ab4d0 0x556552888570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.344+0000 7f46f97df640 1 -- 172.21.15.43:0/1325889364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565524b36b0 msgr2=0x556552888ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.344+0000 7f46f97df640 1 --2- 172.21.15.43:0/1325889364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565524b36b0 0x556552888ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.344+0000 7f46f97df640 1 -- 172.21.15.43:0/1325889364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55655279b4b0 con 0x5565524d4390 2024-04-25T10:40:11.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.345+0000 7f46f97df640 1 --2- 172.21.15.43:0/1325889364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 0x556552888ff0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f46ec049dd0 tx=0x7f46ec004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ec5be03f5729fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:11.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.345+0000 7f46d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:11.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.345+0000 7f46d3fff640 1 -- 172.21.15.43:0/1325889364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec069070 con 0x5565524d4390 2024-04-25T10:40:11.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.345+0000 7f47021d0740 1 -- 172.21.15.43:0/1325889364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46e801e460 con 0x5565524d4390 2024-04-25T10:40:11.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.345+0000 7f46d3fff640 1 -- 172.21.15.43:0/1325889364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8557+0+0 (secure 0 0 0) 0x7f46ec030e70 con 0x5565524d4390 2024-04-25T10:40:11.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.345+0000 7f46d3fff640 1 -- 172.21.15.43:0/1325889364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f46ec030940 con 0x5565524d4390 2024-04-25T10:40:11.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.345+0000 7f47021d0740 10 client.17658 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:11.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.345+0000 7f47021d0740 1 -- 172.21.15.43:0/1325889364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5565528f8780 con 0x5565524d4390 2024-04-25T10:40:11.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.345+0000 7f46d3fff640 1 -- 172.21.15.43:0/1325889364 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46e801e460 con 0x5565524d4390 2024-04-25T10:40:11.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.345+0000 7f46d3fff640 1 -- 172.21.15.43:0/1325889364 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f46ec065460 con 0x5565524d4390 2024-04-25T10:40:11.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.345+0000 7f47021d0740 10 client.17658 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:11.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.345+0000 7f47021d0740 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:11.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.345+0000 7f47021d0740 20 client.17658 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:11.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.345+0000 7f47021d0740 1 --2- 172.21.15.43:0/1325889364 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5565524683a0 0x5565528fae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.345+0000 7f47021d0740 4 client.17658 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:40:11.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.345+0000 7f47021d0740 1 -- 172.21.15.43:0/1325889364 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5565527e2800 con 0x5565524683a0 2024-04-25T10:40:11.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.346+0000 7f46f8fde640 1 --2- 172.21.15.43:0/1325889364 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5565524683a0 0x5565528fae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.346+0000 7f46d3fff640 10 client.17658 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:11.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.346+0000 7f46f8fde640 1 --2- 172.21.15.43:0/1325889364 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5565524683a0 0x5565528fae80 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16ee64531e3fd22a server_cookie=843294c798c36b50 in_seq=0 out_seq=0 2024-04-25T10:40:11.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.346+0000 7f46d3fff640 1 -- 172.21.15.43:0/1325889364 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5565527e2800 con 0x5565524683a0 2024-04-25T10:40:11.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.346+0000 7f46d3fff640 10 client.17658 handle_command_reply: tid=0 2024-04-25T10:40:11.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.346+0000 7f47021d0740 1 client.17658 shutdown 2024-04-25T10:40:11.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.346+0000 7f47021d0740 1 -- 172.21.15.43:0/1325889364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 msgr2=0x556552888ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.346+0000 7f47021d0740 1 --2- 172.21.15.43:0/1325889364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 0x556552888ff0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f46ec049dd0 tx=0x7f46ec004f60 comp rx=0 tx=0).stop 2024-04-25T10:40:11.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.346+0000 7f47021d0740 1 -- 172.21.15.43:0/1325889364 shutdown_connections 2024-04-25T10:40:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.346+0000 7f47021d0740 1 --2- 172.21.15.43:0/1325889364 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5565524683a0 0x5565528fae80 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.346+0000 7f47021d0740 1 --2- 172.21.15.43:0/1325889364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5565524b36b0 0x556552888ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.346+0000 7f47021d0740 1 --2- 172.21.15.43:0/1325889364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565524d4390 0x556552888ff0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.346+0000 7f47021d0740 1 --2- 172.21.15.43:0/1325889364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565524ab4d0 0x556552888570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.346+0000 7f46d3fff640 0 client.17658 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.346+0000 7f47021d0740 1 -- 172.21.15.43:0/1325889364 >> 172.21.15.43:0/1325889364 conn(0x5565524c84e0 msgr2=0x55655287dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.346+0000 7f47021d0740 1 -- 172.21.15.43:0/1325889364 shutdown_connections 2024-04-25T10:40:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.346+0000 7f47021d0740 1 -- 172.21.15.43:0/1325889364 wait complete. 2024-04-25T10:40:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.347+0000 7f47021d0740 20 client.17658 trim_cache size 0 max 16384 2024-04-25T10:40:11.356 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:40:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:40:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:40:11.357 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:40:11.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.347+0000 7f46fb242640 1 -- 172.21.15.43:0/4058025097 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f46dc0afc60 msgr2=0x7f46dc0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.347+0000 7f46fb242640 1 --2- 172.21.15.43:0/4058025097 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f46dc0afc60 0x7f46dc0b2170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f46ec04d000 tx=0x7f46ec04b000 comp rx=0 tx=0).stop 2024-04-25T10:40:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.347+0000 7f46fb242640 1 -- 172.21.15.43:0/4058025097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46f4102b10 msgr2=0x7f46f419a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.347+0000 7f46fb242640 1 --2- 172.21.15.43:0/4058025097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46f4102b10 0x7f46f419a7b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f46e400b450 tx=0x7f46e400b910 comp rx=0 tx=0).stop 2024-04-25T10:40:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.347+0000 7f46fb242640 1 -- 172.21.15.43:0/4058025097 shutdown_connections 2024-04-25T10:40:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.347+0000 7f46fb242640 1 --2- 172.21.15.43:0/4058025097 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f46dc0afc60 0x7f46dc0b2170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.347+0000 7f46fb242640 1 --2- 172.21.15.43:0/4058025097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f4108a40 0x7f46f419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.347+0000 7f46fb242640 1 --2- 172.21.15.43:0/4058025097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46f410d3a0 0x7f46f419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.347+0000 7f46fb242640 1 --2- 172.21.15.43:0/4058025097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46f4102b10 0x7f46f419a7b0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.347+0000 7f46fb242640 1 -- 172.21.15.43:0/4058025097 >> 172.21.15.43:0/4058025097 conn(0x7f46f4078180 msgr2=0x7f46f41040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.347+0000 7f46fb242640 1 -- 172.21.15.43:0/4058025097 shutdown_connections 2024-04-25T10:40:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.347+0000 7f46fb242640 1 -- 172.21.15.43:0/4058025097 wait complete. 2024-04-25T10:40:11.362 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:40:11.363 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:40:11.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.513+0000 7efdbe8c6640 1 -- 172.21.15.43:0/1494976954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb8109b30 msgr2=0x7efdb81036d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.513+0000 7efdbe8c6640 1 --2- 172.21.15.43:0/1494976954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb8109b30 0x7efdb81036d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7efda8009900 tx=0x7efda802a410 comp rx=0 tx=0).stop 2024-04-25T10:40:11.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.513+0000 7efdbe8c6640 1 -- 172.21.15.43:0/1494976954 shutdown_connections 2024-04-25T10:40:11.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.513+0000 7efdbe8c6640 1 --2- 172.21.15.43:0/1494976954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb8109fd0 0x7efdb8103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.513+0000 7efdbe8c6640 1 --2- 172.21.15.43:0/1494976954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efdb8104280 0x7efdb8110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.513+0000 7efdbe8c6640 1 --2- 172.21.15.43:0/1494976954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb8109b30 0x7efdb81036d0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.513+0000 7efdbe8c6640 1 -- 172.21.15.43:0/1494976954 >> 172.21.15.43:0/1494976954 conn(0x7efdb80ff210 msgr2=0x7efdb8101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:11.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.513+0000 7efdbe8c6640 1 -- 172.21.15.43:0/1494976954 shutdown_connections 2024-04-25T10:40:11.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.513+0000 7efdbe8c6640 1 -- 172.21.15.43:0/1494976954 wait complete. 2024-04-25T10:40:11.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.513+0000 7efdbe8c6640 1 Processor -- start 2024-04-25T10:40:11.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.513+0000 7efdbe8c6640 1 -- start start 2024-04-25T10:40:11.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.514+0000 7efdbe8c6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb8104280 0x7efdb8075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.514+0000 7efdbe8c6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb8109b30 0x7efdb80764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.514+0000 7efdbe8c6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efdb8109fd0 0x7efdb8076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.514+0000 7efdbe8c6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efdb8113d20 con 0x7efdb8104280 2024-04-25T10:40:11.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.514+0000 7efdbe8c6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efdb8113ba0 con 0x7efdb8109b30 2024-04-25T10:40:11.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.514+0000 7efdbe8c6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efdb8113ea0 con 0x7efdb8109fd0 2024-04-25T10:40:11.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.514+0000 7efdb7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb8104280 0x7efdb8075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.514+0000 7efdb7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb8104280 0x7efdb8075fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53692/0 (socket says 172.21.15.43:53692) 2024-04-25T10:40:11.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.514+0000 7efdb7fff640 1 -- 172.21.15.43:0/3835257674 learned_addr learned my addr 172.21.15.43:0/3835257674 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:11.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.514+0000 7efdb77fe640 1 --2- 172.21.15.43:0/3835257674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb8109b30 0x7efdb80764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.514+0000 7efdbce63640 1 --2- 172.21.15.43:0/3835257674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efdb8109fd0 0x7efdb8076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.514+0000 7efdb7fff640 1 -- 172.21.15.43:0/3835257674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efdb8109fd0 msgr2=0x7efdb8076a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.514+0000 7efdb7fff640 1 --2- 172.21.15.43:0/3835257674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efdb8109fd0 0x7efdb8076a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.514+0000 7efdb7fff640 1 -- 172.21.15.43:0/3835257674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb8109b30 msgr2=0x7efdb80764f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.514+0000 7efdb7fff640 1 --2- 172.21.15.43:0/3835257674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb8109b30 0x7efdb80764f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.514+0000 7efdb7fff640 1 -- 172.21.15.43:0/3835257674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb81a8ac0 con 0x7efdb8104280 2024-04-25T10:40:11.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.514+0000 7efdb77fe640 1 --2- 172.21.15.43:0/3835257674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb8109b30 0x7efdb80764f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:11.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.515+0000 7efdb7fff640 1 --2- 172.21.15.43:0/3835257674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb8104280 0x7efdb8075fb0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7efda8032fd0 tx=0x7efda8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:11.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.515+0000 7efdb57fa640 1 -- 172.21.15.43:0/3835257674 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda803f030 con 0x7efdb8104280 2024-04-25T10:40:11.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.515+0000 7efdbe8c6640 1 -- 172.21.15.43:0/3835257674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb81a8d50 con 0x7efdb8104280 2024-04-25T10:40:11.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.515+0000 7efdb57fa640 1 -- 172.21.15.43:0/3835257674 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efda8004490 con 0x7efdb8104280 2024-04-25T10:40:11.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.515+0000 7efdb57fa640 1 -- 172.21.15.43:0/3835257674 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8035070 con 0x7efdb8104280 2024-04-25T10:40:11.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.515+0000 7efdbe8c6640 1 -- 172.21.15.43:0/3835257674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb81a91d0 con 0x7efdb8104280 2024-04-25T10:40:11.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.516+0000 7efdbe8c6640 1 -- 172.21.15.43:0/3835257674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdb806a190 con 0x7efdb8104280 2024-04-25T10:40:11.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.517+0000 7efdb57fa640 1 -- 172.21.15.43:0/3835257674 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7efda8042030 con 0x7efdb8104280 2024-04-25T10:40:11.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.518+0000 7efdb57fa640 1 --2- 172.21.15.43:0/3835257674 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efda00afc60 0x7efda00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.518+0000 7efdb77fe640 1 --2- 172.21.15.43:0/3835257674 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efda00afc60 0x7efda00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.518+0000 7efdb57fa640 1 -- 172.21.15.43:0/3835257674 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8557+0+0 (secure 0 0 0) 0x7efda80ff9f0 con 0x7efdb8104280 2024-04-25T10:40:11.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.519+0000 7efdb77fe640 1 --2- 172.21.15.43:0/3835257674 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efda00afc60 0x7efda00b2170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efdac005e00 tx=0x7efdac005c90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:11.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.522+0000 7efdb57fa640 1 -- 172.21.15.43:0/3835257674 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efda80c8270 con 0x7efdb8104280 2024-04-25T10:40:11.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.749+0000 7efdbe8c6640 1 -- 172.21.15.43:0/3835257674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdb81a94a0 con 0x7efdb8104280 2024-04-25T10:40:11.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.749+0000 7efdb57fa640 1 -- 172.21.15.43:0/3835257674 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7efda80e8e20 con 0x7efdb8104280 2024-04-25T10:40:11.753 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:40:11.753 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:40:11.753 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:40:11.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.752+0000 7efdbe8c6640 1 -- 172.21.15.43:0/3835257674 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efda00afc60 msgr2=0x7efda00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.752+0000 7efdbe8c6640 1 --2- 172.21.15.43:0/3835257674 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efda00afc60 0x7efda00b2170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efdac005e00 tx=0x7efdac005c90 comp rx=0 tx=0).stop 2024-04-25T10:40:11.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.752+0000 7efdbe8c6640 1 -- 172.21.15.43:0/3835257674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb8104280 msgr2=0x7efdb8075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.752+0000 7efdbe8c6640 1 --2- 172.21.15.43:0/3835257674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb8104280 0x7efdb8075fb0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7efda8032fd0 tx=0x7efda8004270 comp rx=0 tx=0).stop 2024-04-25T10:40:11.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.752+0000 7efdbe8c6640 1 -- 172.21.15.43:0/3835257674 shutdown_connections 2024-04-25T10:40:11.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.752+0000 7efdbe8c6640 1 --2- 172.21.15.43:0/3835257674 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efda00afc60 0x7efda00b2170 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.752+0000 7efdbe8c6640 1 --2- 172.21.15.43:0/3835257674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb8109b30 0x7efdb80764f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.752+0000 7efdbe8c6640 1 --2- 172.21.15.43:0/3835257674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efdb8109fd0 0x7efdb8076a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.752+0000 7efdbe8c6640 1 --2- 172.21.15.43:0/3835257674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb8104280 0x7efdb8075fb0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.752+0000 7efdbe8c6640 1 -- 172.21.15.43:0/3835257674 >> 172.21.15.43:0/3835257674 conn(0x7efdb80ff210 msgr2=0x7efdb80ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:11.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.752+0000 7efdbe8c6640 1 -- 172.21.15.43:0/3835257674 shutdown_connections 2024-04-25T10:40:11.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.752+0000 7efdbe8c6640 1 -- 172.21.15.43:0/3835257674 wait complete. 2024-04-25T10:40:11.767 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:40:11.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.879+0000 7efcc66ee640 1 -- 172.21.15.43:0/1918500096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcc0109fc0 msgr2=0x7efcc0103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.879+0000 7efcc66ee640 1 --2- 172.21.15.43:0/1918500096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcc0109fc0 0x7efcc0103c00 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009900 tx=0x7efcb402a410 comp rx=0 tx=0).stop 2024-04-25T10:40:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.879+0000 7efcc66ee640 1 -- 172.21.15.43:0/1918500096 shutdown_connections 2024-04-25T10:40:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.879+0000 7efcc66ee640 1 --2- 172.21.15.43:0/1918500096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcc0104270 0x7efcc0110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.879+0000 7efcc66ee640 1 --2- 172.21.15.43:0/1918500096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcc0109fc0 0x7efcc0103c00 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.879+0000 7efcc66ee640 1 --2- 172.21.15.43:0/1918500096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcc0109b20 0x7efcc01036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.879+0000 7efcc66ee640 1 -- 172.21.15.43:0/1918500096 >> 172.21.15.43:0/1918500096 conn(0x7efcc00ff1e0 msgr2=0x7efcc0101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.879+0000 7efcc66ee640 1 -- 172.21.15.43:0/1918500096 shutdown_connections 2024-04-25T10:40:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.879+0000 7efcc66ee640 1 -- 172.21.15.43:0/1918500096 wait complete. 2024-04-25T10:40:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.880+0000 7efcc66ee640 1 Processor -- start 2024-04-25T10:40:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.880+0000 7efcc66ee640 1 -- start start 2024-04-25T10:40:11.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.880+0000 7efcc66ee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcc0104270 0x7efcc0075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.880+0000 7efcc66ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcc0109b20 0x7efcc00764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.880+0000 7efcc66ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcc0109fc0 0x7efcc0076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.880+0000 7efcc66ee640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efcc0113d10 con 0x7efcc0109b20 2024-04-25T10:40:11.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.880+0000 7efcc66ee640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efcc0113b90 con 0x7efcc0109fc0 2024-04-25T10:40:11.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.880+0000 7efcc66ee640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efcc0113e90 con 0x7efcc0104270 2024-04-25T10:40:11.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.880+0000 7efcbf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcc0109b20 0x7efcc00764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.880+0000 7efcbf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcc0109b20 0x7efcc00764f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53710/0 (socket says 172.21.15.43:53710) 2024-04-25T10:40:11.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.880+0000 7efcbf7fe640 1 -- 172.21.15.43:0/3035847943 learned_addr learned my addr 172.21.15.43:0/3035847943 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:11.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.880+0000 7efcc4c8b640 1 --2- 172.21.15.43:0/3035847943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcc0109fc0 0x7efcc0076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.880+0000 7efcbffff640 1 --2- 172.21.15.43:0/3035847943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcc0104270 0x7efcc0075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.880+0000 7efcbf7fe640 1 -- 172.21.15.43:0/3035847943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcc0104270 msgr2=0x7efcc0075fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.880+0000 7efcbf7fe640 1 --2- 172.21.15.43:0/3035847943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcc0104270 0x7efcc0075fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.880+0000 7efcbf7fe640 1 -- 172.21.15.43:0/3035847943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcc0109fc0 msgr2=0x7efcc0076a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.881+0000 7efcbf7fe640 1 --2- 172.21.15.43:0/3035847943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcc0109fc0 0x7efcc0076a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.881+0000 7efcbf7fe640 1 -- 172.21.15.43:0/3035847943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc01a8b30 con 0x7efcc0109b20 2024-04-25T10:40:11.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.881+0000 7efcbf7fe640 1 --2- 172.21.15.43:0/3035847943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcc0109b20 0x7efcc00764f0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7efcb4006010 tx=0x7efcb4004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:11.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.881+0000 7efcbd7fa640 1 -- 172.21.15.43:0/3035847943 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb403f030 con 0x7efcc0109b20 2024-04-25T10:40:11.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.881+0000 7efcbd7fa640 1 -- 172.21.15.43:0/3035847943 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcb4003aa0 con 0x7efcc0109b20 2024-04-25T10:40:11.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.881+0000 7efcbd7fa640 1 -- 172.21.15.43:0/3035847943 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4035070 con 0x7efcc0109b20 2024-04-25T10:40:11.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.881+0000 7efcc66ee640 1 -- 172.21.15.43:0/3035847943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc01a8dc0 con 0x7efcc0109b20 2024-04-25T10:40:11.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.881+0000 7efcc66ee640 1 -- 172.21.15.43:0/3035847943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc01a91a0 con 0x7efcc0109b20 2024-04-25T10:40:11.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efcbd7fa640 1 -- 172.21.15.43:0/3035847943 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7efcb4042030 con 0x7efcc0109b20 2024-04-25T10:40:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efccd59e740 1 Processor -- start 2024-04-25T10:40:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efccd59e740 1 -- start start 2024-04-25T10:40:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efccd59e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0e9390 0x55bc6d48a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efccd59e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0c04d0 0x55bc6d48ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efccd59e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c86b0 0x55bc6d4976a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efccd59e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bc6d3efd80 con 0x55bc6d0c04d0 2024-04-25T10:40:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efccd59e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bc6d3a9a20 con 0x55bc6d0c86b0 2024-04-25T10:40:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efccd59e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bc6d3ebd40 con 0x55bc6d0e9390 2024-04-25T10:40:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efcbffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0c04d0 0x55bc6d48ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efcbffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0c04d0 0x55bc6d48ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53724/0 (socket says 172.21.15.43:53724) 2024-04-25T10:40:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efcbffff640 1 -- 172.21.15.43:0/1143686852 learned_addr learned my addr 172.21.15.43:0/1143686852 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efcc4c8b640 1 --2- 172.21.15.43:0/1143686852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0e9390 0x55bc6d48a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efcbf7fe640 1 --2- 172.21.15.43:0/1143686852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c86b0 0x55bc6d4976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efcbd7fa640 1 --2- 172.21.15.43:0/3035847943 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efca80afc60 0x7efca80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efcbffff640 1 -- 172.21.15.43:0/1143686852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0e9390 msgr2=0x55bc6d48a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efcbffff640 1 --2- 172.21.15.43:0/1143686852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0e9390 0x55bc6d48a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efcbffff640 1 -- 172.21.15.43:0/1143686852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c86b0 msgr2=0x55bc6d4976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efcbffff640 1 --2- 172.21.15.43:0/1143686852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c86b0 0x55bc6d4976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efcbffff640 1 -- 172.21.15.43:0/1143686852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc6d3b4c30 con 0x55bc6d0c04d0 2024-04-25T10:40:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efcc4c8b640 1 --2- 172.21.15.43:0/1143686852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0e9390 0x55bc6d48a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.884+0000 7efcbf7fe640 1 --2- 172.21.15.43:0/1143686852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c86b0 0x55bc6d4976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.885+0000 7efcc4c8b640 1 --2- 172.21.15.43:0/3035847943 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efca80afc60 0x7efca80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.885+0000 7efcbd7fa640 1 -- 172.21.15.43:0/3035847943 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8557+0+0 (secure 0 0 0) 0x7efcb40ff5f0 con 0x7efcc0109b20 2024-04-25T10:40:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.885+0000 7efcbffff640 1 --2- 172.21.15.43:0/1143686852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0c04d0 0x55bc6d48ad50 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7efcb0009ac0 tx=0x7efcb0009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa00a8b57ab3b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.885+0000 7efca67fc640 1 -- 172.21.15.43:0/1143686852 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0016070 con 0x55bc6d0c04d0 2024-04-25T10:40:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.885+0000 7efca67fc640 1 -- 172.21.15.43:0/1143686852 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcb00054d0 con 0x55bc6d0c04d0 2024-04-25T10:40:11.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.885+0000 7efca67fc640 1 -- 172.21.15.43:0/1143686852 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0004dd0 con 0x55bc6d0c04d0 2024-04-25T10:40:11.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.886+0000 7efccd59e740 1 -- 172.21.15.43:0/1143686852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0c04d0 msgr2=0x55bc6d48ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.886+0000 7efccd59e740 1 --2- 172.21.15.43:0/1143686852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0c04d0 0x55bc6d48ad50 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7efcb0009ac0 tx=0x7efcb0009f80 comp rx=0 tx=0).stop 2024-04-25T10:40:11.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.886+0000 7efcc4c8b640 1 --2- 172.21.15.43:0/3035847943 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efca80afc60 0x7efca80b2170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55bc6d487980 tx=0x7efcb8004a40 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:11.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.886+0000 7efccd59e740 1 -- 172.21.15.43:0/1143686852 shutdown_connections 2024-04-25T10:40:11.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.886+0000 7efccd59e740 1 --2- 172.21.15.43:0/1143686852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c86b0 0x55bc6d4976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.886+0000 7efccd59e740 1 --2- 172.21.15.43:0/1143686852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0c04d0 0x55bc6d48ad50 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.886+0000 7efccd59e740 1 --2- 172.21.15.43:0/1143686852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0e9390 0x55bc6d48a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.886+0000 7efccd59e740 1 -- 172.21.15.43:0/1143686852 >> 172.21.15.43:0/1143686852 conn(0x55bc6d0dd4e0 msgr2=0x55bc6d486fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:11.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.886+0000 7efccd59e740 1 -- 172.21.15.43:0/1143686852 shutdown_connections 2024-04-25T10:40:11.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.886+0000 7efccd59e740 1 -- 172.21.15.43:0/1143686852 wait complete. 2024-04-25T10:40:11.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.886+0000 7efccd59e740 1 -- start start 2024-04-25T10:40:11.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.886+0000 7efccd59e740 1 Processor -- start 2024-04-25T10:40:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.887+0000 7efccd59e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0c04d0 0x55bc6d495260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.887+0000 7efccd59e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c86b0 0x55bc6d4957a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.887+0000 7efccd59e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0e9390 0x55bc6d4a1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.887+0000 7efccd59e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bc6d497d90 con 0x55bc6d0e9390 2024-04-25T10:40:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.887+0000 7efccd59e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bc6d376dc0 con 0x55bc6d0c86b0 2024-04-25T10:40:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.887+0000 7efccd59e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bc6d497f10 con 0x55bc6d0c04d0 2024-04-25T10:40:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.887+0000 7efccd59e740 10 client.0 _ll_register_callbacks cb 0x55bc6d392600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.887+0000 7efcc4c8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0e9390 0x55bc6d4a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.887+0000 7efcc4c8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0e9390 0x55bc6d4a1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53726/0 (socket says 172.21.15.43:53726) 2024-04-25T10:40:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.887+0000 7efcc4c8b640 1 -- 172.21.15.43:0/1636492811 learned_addr learned my addr 172.21.15.43:0/1636492811 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.887+0000 7efcbffff640 1 --2- 172.21.15.43:0/1636492811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0c04d0 0x55bc6d495260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.887+0000 7efcbf7fe640 1 --2- 172.21.15.43:0/1636492811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c86b0 0x55bc6d4957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.887+0000 7efcc4c8b640 1 -- 172.21.15.43:0/1636492811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0c04d0 msgr2=0x55bc6d495260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.887+0000 7efcc4c8b640 1 --2- 172.21.15.43:0/1636492811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0c04d0 0x55bc6d495260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.887+0000 7efcc4c8b640 1 -- 172.21.15.43:0/1636492811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c86b0 msgr2=0x55bc6d4957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.887+0000 7efcc4c8b640 1 --2- 172.21.15.43:0/1636492811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c86b0 0x55bc6d4957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.887+0000 7efcc4c8b640 1 -- 172.21.15.43:0/1636492811 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc6d1013f0 con 0x55bc6d0e9390 2024-04-25T10:40:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.887+0000 7efcbf7fe640 1 --2- 172.21.15.43:0/1636492811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c86b0 0x55bc6d4957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.887+0000 7efcc4c8b640 1 --2- 172.21.15.43:0/1636492811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0e9390 0x55bc6d4a1210 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7efcb8009fd0 tx=0x7efcb8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85de6b0383100e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.887+0000 7efc9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.888+0000 7efc9f7fe640 1 -- 172.21.15.43:0/1636492811 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb802a060 con 0x55bc6d0e9390 2024-04-25T10:40:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.888+0000 7efc9f7fe640 1 -- 172.21.15.43:0/1636492811 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8557+0+0 (secure 0 0 0) 0x7efcb8017070 con 0x55bc6d0e9390 2024-04-25T10:40:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.888+0000 7efccd59e740 1 -- 172.21.15.43:0/1636492811 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcb00054d0 con 0x55bc6d0e9390 2024-04-25T10:40:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.888+0000 7efc9f7fe640 1 -- 172.21.15.43:0/1636492811 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efcb8011c90 con 0x55bc6d0e9390 2024-04-25T10:40:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.888+0000 7efccd59e740 10 client.17676 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.888+0000 7efccd59e740 1 -- 172.21.15.43:0/1636492811 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bc6d1cb7d0 con 0x55bc6d0e9390 2024-04-25T10:40:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.888+0000 7efc9f7fe640 1 -- 172.21.15.43:0/1636492811 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcb00054d0 con 0x55bc6d0e9390 2024-04-25T10:40:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.888+0000 7efc9f7fe640 1 -- 172.21.15.43:0/1636492811 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7efcb8025410 con 0x55bc6d0e9390 2024-04-25T10:40:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.888+0000 7efccd59e740 10 client.17676 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.888+0000 7efccd59e740 10 client.17676 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:40:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.888+0000 7efccd59e740 20 client.17676 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.888+0000 7efccd59e740 1 --2- 172.21.15.43:0/1636492811 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55bc6d0a7f60 0x55bc6d4a7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.888+0000 7efccd59e740 4 client.17676 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.888+0000 7efccd59e740 1 -- 172.21.15.43:0/1636492811 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc6d3fd990 con 0x55bc6d0a7f60 2024-04-25T10:40:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.889+0000 7efcbffff640 1 --2- 172.21.15.43:0/1636492811 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55bc6d0a7f60 0x55bc6d4a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.889+0000 7efc9f7fe640 10 client.17676 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:40:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.889+0000 7efcbffff640 1 --2- 172.21.15.43:0/1636492811 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55bc6d0a7f60 0x55bc6d4a7770 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b892b57f6d4b7fdf server_cookie=41f8dbdb6a71e892 in_seq=0 out_seq=0 2024-04-25T10:40:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.890+0000 7efc9f7fe640 1 -- 172.21.15.43:0/1636492811 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bc6d3fd990 con 0x55bc6d0a7f60 2024-04-25T10:40:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.890+0000 7efc9f7fe640 10 client.17676 handle_command_reply: tid=0 2024-04-25T10:40:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.890+0000 7efccd59e740 1 client.17676 shutdown 2024-04-25T10:40:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.890+0000 7efccd59e740 1 -- 172.21.15.43:0/1636492811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0e9390 msgr2=0x55bc6d4a1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.890+0000 7efccd59e740 1 --2- 172.21.15.43:0/1636492811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0e9390 0x55bc6d4a1210 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7efcb8009fd0 tx=0x7efcb8004ab0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.890+0000 7efccd59e740 1 -- 172.21.15.43:0/1636492811 shutdown_connections 2024-04-25T10:40:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.890+0000 7efccd59e740 1 --2- 172.21.15.43:0/1636492811 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55bc6d0a7f60 0x55bc6d4a7770 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.890+0000 7efccd59e740 1 --2- 172.21.15.43:0/1636492811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c86b0 0x55bc6d4957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.890+0000 7efccd59e740 1 --2- 172.21.15.43:0/1636492811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0e9390 0x55bc6d4a1210 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.890+0000 7efccd59e740 1 --2- 172.21.15.43:0/1636492811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0c04d0 0x55bc6d495260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.890+0000 7efc9f7fe640 0 client.17676 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:40:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.890+0000 7efccd59e740 1 -- 172.21.15.43:0/1636492811 >> 172.21.15.43:0/1636492811 conn(0x55bc6d0dd4e0 msgr2=0x55bc6d48c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.890+0000 7efccd59e740 1 -- 172.21.15.43:0/1636492811 shutdown_connections 2024-04-25T10:40:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.890+0000 7efccd59e740 1 -- 172.21.15.43:0/1636492811 wait complete. 2024-04-25T10:40:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.890+0000 7efccd59e740 20 client.17676 trim_cache size 0 max 16384 2024-04-25T10:40:11.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.904+0000 7efccd59e740 1 Processor -- start 2024-04-25T10:40:11.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.904+0000 7efccd59e740 1 -- start start 2024-04-25T10:40:11.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.904+0000 7efccd59e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c04d0 0x55bc6d486f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.904+0000 7efccd59e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0c86b0 0x55bc6d4a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.904+0000 7efccd59e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0e9390 0x55bc6d4acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.904+0000 7efccd59e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01b4550 con 0x55bc6d0c86b0 2024-04-25T10:40:11.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.904+0000 7efccd59e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01b1e20 con 0x55bc6d0c04d0 2024-04-25T10:40:11.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.904+0000 7efccd59e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01afc90 con 0x55bc6d0e9390 2024-04-25T10:40:11.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.904+0000 7efcbf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0c86b0 0x55bc6d4a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.904+0000 7efcbf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0c86b0 0x55bc6d4a0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53740/0 (socket says 172.21.15.43:53740) 2024-04-25T10:40:11.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.904+0000 7efcbf7fe640 1 -- 172.21.15.43:0/711163816 learned_addr learned my addr 172.21.15.43:0/711163816 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:11.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.904+0000 7efcbffff640 1 --2- 172.21.15.43:0/711163816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c04d0 0x55bc6d486f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.904+0000 7efcbf7fe640 1 -- 172.21.15.43:0/711163816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0e9390 msgr2=0x55bc6d4acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.905+0000 7efcc4c8b640 1 --2- 172.21.15.43:0/711163816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0e9390 0x55bc6d4acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.905+0000 7efcbf7fe640 1 --2- 172.21.15.43:0/711163816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0e9390 0x55bc6d4acf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.905+0000 7efcbf7fe640 1 -- 172.21.15.43:0/711163816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c04d0 msgr2=0x55bc6d486f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.905+0000 7efcbf7fe640 1 --2- 172.21.15.43:0/711163816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c04d0 0x55bc6d486f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.905+0000 7efcbf7fe640 1 -- 172.21.15.43:0/711163816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc6d4a7b20 con 0x55bc6d0c86b0 2024-04-25T10:40:11.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.905+0000 7efcbffff640 1 --2- 172.21.15.43:0/711163816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c04d0 0x55bc6d486f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:11.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.905+0000 7efcbf7fe640 1 --2- 172.21.15.43:0/711163816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0c86b0 0x55bc6d4a0e70 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7efcb40c7470 tx=0x7efcb402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c860d66379d6d2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:11.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.905+0000 7efca5ffb640 1 -- 172.21.15.43:0/711163816 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4049070 con 0x55bc6d0c86b0 2024-04-25T10:40:11.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.905+0000 7efca5ffb640 1 -- 172.21.15.43:0/711163816 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcb40c7280 con 0x55bc6d0c86b0 2024-04-25T10:40:11.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.905+0000 7efca5ffb640 1 -- 172.21.15.43:0/711163816 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb40fcfa0 con 0x55bc6d0c86b0 2024-04-25T10:40:11.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.906+0000 7efccd59e740 1 -- 172.21.15.43:0/711163816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0c86b0 msgr2=0x55bc6d4a0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.906+0000 7efccd59e740 1 --2- 172.21.15.43:0/711163816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0c86b0 0x55bc6d4a0e70 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7efcb40c7470 tx=0x7efcb402a410 comp rx=0 tx=0).stop 2024-04-25T10:40:11.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.906+0000 7efccd59e740 1 -- 172.21.15.43:0/711163816 shutdown_connections 2024-04-25T10:40:11.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.906+0000 7efccd59e740 1 --2- 172.21.15.43:0/711163816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0e9390 0x55bc6d4acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.906+0000 7efccd59e740 1 --2- 172.21.15.43:0/711163816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0c86b0 0x55bc6d4a0e70 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.906+0000 7efccd59e740 1 --2- 172.21.15.43:0/711163816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c04d0 0x55bc6d486f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.906+0000 7efccd59e740 1 -- 172.21.15.43:0/711163816 >> 172.21.15.43:0/711163816 conn(0x55bc6d0dd4e0 msgr2=0x55bc6d486860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:11.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.906+0000 7efccd59e740 1 -- 172.21.15.43:0/711163816 shutdown_connections 2024-04-25T10:40:11.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.906+0000 7efccd59e740 1 -- 172.21.15.43:0/711163816 wait complete. 2024-04-25T10:40:11.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.906+0000 7efccd59e740 1 -- start start 2024-04-25T10:40:11.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.906+0000 7efccd59e740 1 Processor -- start 2024-04-25T10:40:11.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efccd59e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0c04d0 0x55bc6d49d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efccd59e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c86b0 0x55bc6d4987d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efccd59e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0e9390 0x55bc6d498d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efccd59e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01b02e0 con 0x55bc6d0e9390 2024-04-25T10:40:11.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efccd59e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01b1750 con 0x55bc6d0c86b0 2024-04-25T10:40:11.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efccd59e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01b09b0 con 0x55bc6d0c04d0 2024-04-25T10:40:11.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efccd59e740 10 client.0 _ll_register_callbacks cb 0x7efcb8009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:11.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efcc4c8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0e9390 0x55bc6d498d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efcc4c8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0e9390 0x55bc6d498d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53744/0 (socket says 172.21.15.43:53744) 2024-04-25T10:40:11.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efcc4c8b640 1 -- 172.21.15.43:0/1645135908 learned_addr learned my addr 172.21.15.43:0/1645135908 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:11.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efcbf7fe640 1 --2- 172.21.15.43:0/1645135908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c86b0 0x55bc6d4987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efcbffff640 1 --2- 172.21.15.43:0/1645135908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0c04d0 0x55bc6d49d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efcc4c8b640 1 -- 172.21.15.43:0/1645135908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0c04d0 msgr2=0x55bc6d49d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efcc4c8b640 1 --2- 172.21.15.43:0/1645135908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0c04d0 0x55bc6d49d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efcc4c8b640 1 -- 172.21.15.43:0/1645135908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c86b0 msgr2=0x55bc6d4987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efcc4c8b640 1 --2- 172.21.15.43:0/1645135908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c86b0 0x55bc6d4987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efcc4c8b640 1 -- 172.21.15.43:0/1645135908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc6d499250 con 0x55bc6d0e9390 2024-04-25T10:40:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efcc4c8b640 1 --2- 172.21.15.43:0/1645135908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0e9390 0x55bc6d498d10 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7efcb8012f40 tx=0x7efcb8009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6061661bb369418e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efc9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efc9f7fe640 1 -- 172.21.15.43:0/1645135908 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8011870 con 0x55bc6d0e9390 2024-04-25T10:40:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efc9f7fe640 1 -- 172.21.15.43:0/1645135908 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8557+0+0 (secure 0 0 0) 0x7efcb8025cb0 con 0x55bc6d0e9390 2024-04-25T10:40:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.907+0000 7efccd59e740 1 -- 172.21.15.43:0/1645135908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcb40c7280 con 0x55bc6d0e9390 2024-04-25T10:40:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.908+0000 7efc9f7fe640 1 -- 172.21.15.43:0/1645135908 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efcb803f050 con 0x55bc6d0e9390 2024-04-25T10:40:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.908+0000 7efccd59e740 10 client.17682 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.908+0000 7efccd59e740 1 -- 172.21.15.43:0/1645135908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bc6d50f6e0 con 0x55bc6d0e9390 2024-04-25T10:40:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.908+0000 7efc9f7fe640 1 -- 172.21.15.43:0/1645135908 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcb40c7280 con 0x55bc6d0e9390 2024-04-25T10:40:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.908+0000 7efc9f7fe640 1 -- 172.21.15.43:0/1645135908 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7efcb8036070 con 0x55bc6d0e9390 2024-04-25T10:40:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.908+0000 7efccd59e740 10 client.17682 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.908+0000 7efccd59e740 10 client.17682 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:40:11.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.908+0000 7efccd59e740 20 client.17682 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:11.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.908+0000 7efccd59e740 1 --2- 172.21.15.43:0/1645135908 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55bc6d07d3a0 0x55bc6d511e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:11.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.908+0000 7efccd59e740 4 client.17682 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:40:11.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.908+0000 7efccd59e740 1 -- 172.21.15.43:0/1645135908 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc6d495b70 con 0x55bc6d07d3a0 2024-04-25T10:40:11.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.908+0000 7efcbffff640 1 --2- 172.21.15.43:0/1645135908 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55bc6d07d3a0 0x55bc6d511e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:11.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.909+0000 7efc9f7fe640 10 client.17682 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:40:11.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.909+0000 7efcbffff640 1 --2- 172.21.15.43:0/1645135908 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55bc6d07d3a0 0x55bc6d511e10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ecaec3c458b5e937 server_cookie=9c7f2a16cf4e5bb8 in_seq=0 out_seq=0 2024-04-25T10:40:11.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.909+0000 7efc9f7fe640 1 -- 172.21.15.43:0/1645135908 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc6d495b70 con 0x55bc6d07d3a0 2024-04-25T10:40:11.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.909+0000 7efc9f7fe640 10 client.17682 handle_command_reply: tid=0 2024-04-25T10:40:11.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.909+0000 7efccd59e740 1 client.17682 shutdown 2024-04-25T10:40:11.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.909+0000 7efccd59e740 1 -- 172.21.15.43:0/1645135908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0e9390 msgr2=0x55bc6d498d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.909+0000 7efccd59e740 1 --2- 172.21.15.43:0/1645135908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0e9390 0x55bc6d498d10 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7efcb8012f40 tx=0x7efcb8009a10 comp rx=0 tx=0).stop 2024-04-25T10:40:11.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.909+0000 7efccd59e740 1 -- 172.21.15.43:0/1645135908 shutdown_connections 2024-04-25T10:40:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.909+0000 7efccd59e740 1 --2- 172.21.15.43:0/1645135908 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55bc6d07d3a0 0x55bc6d511e10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.909+0000 7efccd59e740 1 --2- 172.21.15.43:0/1645135908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6d0c86b0 0x55bc6d4987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.909+0000 7efccd59e740 1 --2- 172.21.15.43:0/1645135908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc6d0e9390 0x55bc6d498d10 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.909+0000 7efc9f7fe640 0 client.17682 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:40:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.909+0000 7efccd59e740 1 --2- 172.21.15.43:0/1645135908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bc6d0c04d0 0x55bc6d49d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.909+0000 7efccd59e740 1 -- 172.21.15.43:0/1645135908 >> 172.21.15.43:0/1645135908 conn(0x55bc6d0dd4e0 msgr2=0x55bc6d48c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.909+0000 7efccd59e740 1 -- 172.21.15.43:0/1645135908 shutdown_connections 2024-04-25T10:40:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.909+0000 7efccd59e740 1 -- 172.21.15.43:0/1645135908 wait complete. 2024-04-25T10:40:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.909+0000 7efccd59e740 20 client.17682 trim_cache size 0 max 16384 2024-04-25T10:40:11.919 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:40:11.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.910+0000 7efcc66ee640 1 -- 172.21.15.43:0/3035847943 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efca80afc60 msgr2=0x7efca80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.910+0000 7efcc66ee640 1 --2- 172.21.15.43:0/3035847943 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efca80afc60 0x7efca80b2170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55bc6d487980 tx=0x7efcb8004a40 comp rx=0 tx=0).stop 2024-04-25T10:40:11.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.910+0000 7efcc66ee640 1 -- 172.21.15.43:0/3035847943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcc0109b20 msgr2=0x7efcc00764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:11.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.910+0000 7efcc66ee640 1 --2- 172.21.15.43:0/3035847943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcc0109b20 0x7efcc00764f0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7efcb4006010 tx=0x7efcb4004060 comp rx=0 tx=0).stop 2024-04-25T10:40:11.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.910+0000 7efcc66ee640 1 -- 172.21.15.43:0/3035847943 shutdown_connections 2024-04-25T10:40:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.910+0000 7efcc66ee640 1 --2- 172.21.15.43:0/3035847943 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efca80afc60 0x7efca80b2170 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.910+0000 7efcc66ee640 1 --2- 172.21.15.43:0/3035847943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcc0109fc0 0x7efcc0076a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.910+0000 7efcc66ee640 1 --2- 172.21.15.43:0/3035847943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcc0109b20 0x7efcc00764f0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.910+0000 7efcc66ee640 1 --2- 172.21.15.43:0/3035847943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcc0104270 0x7efcc0075fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.910+0000 7efcc66ee640 1 -- 172.21.15.43:0/3035847943 >> 172.21.15.43:0/3035847943 conn(0x7efcc00ff1e0 msgr2=0x7efcc010f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.910+0000 7efcc66ee640 1 -- 172.21.15.43:0/3035847943 shutdown_connections 2024-04-25T10:40:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:11.910+0000 7efcc66ee640 1 -- 172.21.15.43:0/3035847943 wait complete. 2024-04-25T10:40:11.924 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:40:12.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.035+0000 7f9fa97b2640 1 -- 172.21.15.43:0/2354054417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa410d390 msgr2=0x7f9fa410f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:12.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.035+0000 7f9fa97b2640 1 --2- 172.21.15.43:0/2354054417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa410d390 0x7f9fa410f7b0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009ca0 tx=0x7f9f9c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.035+0000 7f9fa97b2640 1 -- 172.21.15.43:0/2354054417 shutdown_connections 2024-04-25T10:40:12.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.035+0000 7f9fa97b2640 1 --2- 172.21.15.43:0/2354054417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa410d390 0x7f9fa410f7b0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.035+0000 7f9fa97b2640 1 --2- 172.21.15.43:0/2354054417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fa4102b00 0x7f9fa4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.035+0000 7f9fa97b2640 1 --2- 172.21.15.43:0/2354054417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9fa4108a30 0x7f9fa41025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.035+0000 7f9fa97b2640 1 -- 172.21.15.43:0/2354054417 >> 172.21.15.43:0/2354054417 conn(0x7f9fa40fe260 msgr2=0x7f9fa4100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:12.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.035+0000 7f9fa97b2640 1 -- 172.21.15.43:0/2354054417 shutdown_connections 2024-04-25T10:40:12.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.035+0000 7f9fa97b2640 1 -- 172.21.15.43:0/2354054417 wait complete. 2024-04-25T10:40:12.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.036+0000 7f9fa97b2640 1 Processor -- start 2024-04-25T10:40:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.036+0000 7f9fa97b2640 1 -- start start 2024-04-25T10:40:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa97b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa4102b00 0x7f9fa419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa97b2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9fa4108a30 0x7f9fa419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa97b2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fa410d390 0x7f9fa419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa97b2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa4112c10 con 0x7f9fa4102b00 2024-04-25T10:40:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa97b2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa4112a90 con 0x7f9fa4108a30 2024-04-25T10:40:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa97b2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa4112d90 con 0x7f9fa410d390 2024-04-25T10:40:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa4102b00 0x7f9fa419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa4102b00 0x7f9fa419a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53770/0 (socket says 172.21.15.43:53770) 2024-04-25T10:40:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa2ffd640 1 -- 172.21.15.43:0/3380165578 learned_addr learned my addr 172.21.15.43:0/3380165578 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:12.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9f9a7fc640 1 --2- 172.21.15.43:0/3380165578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9fa4108a30 0x7f9fa419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa37fe640 1 --2- 172.21.15.43:0/3380165578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fa410d390 0x7f9fa419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa2ffd640 1 -- 172.21.15.43:0/3380165578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fa410d390 msgr2=0x7f9fa419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:12.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa2ffd640 1 --2- 172.21.15.43:0/3380165578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fa410d390 0x7f9fa419f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa2ffd640 1 -- 172.21.15.43:0/3380165578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9fa4108a30 msgr2=0x7f9fa419ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:12.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa2ffd640 1 --2- 172.21.15.43:0/3380165578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9fa4108a30 0x7f9fa419ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa2ffd640 1 -- 172.21.15.43:0/3380165578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa419f5c0 con 0x7f9fa4102b00 2024-04-25T10:40:12.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa2ffd640 1 --2- 172.21.15.43:0/3380165578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa4102b00 0x7f9fa419a7e0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400c9a0 tx=0x7f9f9400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:12.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa0ff9640 1 -- 172.21.15.43:0/3380165578 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94014070 con 0x7f9fa4102b00 2024-04-25T10:40:12.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa0ff9640 1 -- 172.21.15.43:0/3380165578 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f94004440 con 0x7f9fa4102b00 2024-04-25T10:40:12.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa97b2640 1 -- 172.21.15.43:0/3380165578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa419f8b0 con 0x7f9fa4102b00 2024-04-25T10:40:12.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa0ff9640 1 -- 172.21.15.43:0/3380165578 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9400f450 con 0x7f9fa4102b00 2024-04-25T10:40:12.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.037+0000 7f9fa97b2640 1 -- 172.21.15.43:0/3380165578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa419fe00 con 0x7f9fa4102b00 2024-04-25T10:40:12.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.040+0000 7f9fb065e740 1 Processor -- start 2024-04-25T10:40:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.040+0000 7f9fb065e740 1 -- start start 2024-04-25T10:40:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.040+0000 7f9fa0ff9640 1 -- 172.21.15.43:0/3380165578 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f9f94017070 con 0x7f9fa4102b00 2024-04-25T10:40:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.041+0000 7f9fb065e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c35390 0x55b801fd6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.041+0000 7f9fb065e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b801c0c4d0 0x55b801fd6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.041+0000 7f9fb065e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c146b0 0x55b801fe36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.041+0000 7f9fb065e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b801f3bd80 con 0x55b801c35390 2024-04-25T10:40:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.041+0000 7f9fb065e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b801ef5a20 con 0x55b801c146b0 2024-04-25T10:40:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.041+0000 7f9fb065e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b801f37d40 con 0x55b801c0c4d0 2024-04-25T10:40:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.041+0000 7f9fa37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c35390 0x55b801fd6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.041+0000 7f9fa37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c35390 0x55b801fd6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53782/0 (socket says 172.21.15.43:53782) 2024-04-25T10:40:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.041+0000 7f9fa37fe640 1 -- 172.21.15.43:0/1368176909 learned_addr learned my addr 172.21.15.43:0/1368176909 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.041+0000 7f9fa2ffd640 1 --2- 172.21.15.43:0/1368176909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b801c0c4d0 0x55b801fd6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.041+0000 7f9f9a7fc640 1 --2- 172.21.15.43:0/1368176909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c146b0 0x55b801fe36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.041+0000 7f9fa37fe640 1 -- 172.21.15.43:0/1368176909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b801c0c4d0 msgr2=0x55b801fd6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.041+0000 7f9fa37fe640 1 --2- 172.21.15.43:0/1368176909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b801c0c4d0 0x55b801fd6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.041+0000 7f9fa37fe640 1 -- 172.21.15.43:0/1368176909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c146b0 msgr2=0x55b801fe36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.041+0000 7f9fa37fe640 1 --2- 172.21.15.43:0/1368176909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c146b0 0x55b801fe36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.041+0000 7f9fa37fe640 1 -- 172.21.15.43:0/1368176909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b801f00c30 con 0x55b801c35390 2024-04-25T10:40:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.041+0000 7f9f9a7fc640 1 --2- 172.21.15.43:0/1368176909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c146b0 0x55b801fe36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.041+0000 7f9fa0ff9640 1 --2- 172.21.15.43:0/3380165578 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9f900afc60 0x7f9f900b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.041+0000 7f9fa2ffd640 1 --2- 172.21.15.43:0/3380165578 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9f900afc60 0x7f9f900b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.041+0000 7f9fa37fe640 1 --2- 172.21.15.43:0/1368176909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c35390 0x55b801fd6810 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c006010 tx=0x7f9f9c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa115ae82a8d135a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.042+0000 7f9fa0ff9640 1 -- 172.21.15.43:0/3380165578 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8557+0+0 (secure 0 0 0) 0x7f9f940deb30 con 0x7f9fa4102b00 2024-04-25T10:40:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.042+0000 7f9f997fa640 1 -- 172.21.15.43:0/1368176909 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c031cf0 con 0x55b801c35390 2024-04-25T10:40:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.042+0000 7f9f997fa640 1 -- 172.21.15.43:0/1368176909 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f9c036070 con 0x55b801c35390 2024-04-25T10:40:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.042+0000 7f9f997fa640 1 -- 172.21.15.43:0/1368176909 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c0042a0 con 0x55b801c35390 2024-04-25T10:40:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.043+0000 7f9fa2ffd640 1 --2- 172.21.15.43:0/3380165578 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9f900afc60 0x7f9f900b2170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9f940dc310 tx=0x7f9f9400cf90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.043+0000 7f9fb065e740 1 -- 172.21.15.43:0/1368176909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c35390 msgr2=0x55b801fd6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.043+0000 7f9fb065e740 1 --2- 172.21.15.43:0/1368176909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c35390 0x55b801fd6810 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c006010 tx=0x7f9f9c009800 comp rx=0 tx=0).stop 2024-04-25T10:40:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.043+0000 7f9fb065e740 1 -- 172.21.15.43:0/1368176909 shutdown_connections 2024-04-25T10:40:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.043+0000 7f9fb065e740 1 --2- 172.21.15.43:0/1368176909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c146b0 0x55b801fe36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.043+0000 7f9fb065e740 1 --2- 172.21.15.43:0/1368176909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b801c0c4d0 0x55b801fd6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.043+0000 7f9fb065e740 1 --2- 172.21.15.43:0/1368176909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c35390 0x55b801fd6810 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.043+0000 7f9fb065e740 1 -- 172.21.15.43:0/1368176909 >> 172.21.15.43:0/1368176909 conn(0x55b801c294e0 msgr2=0x55b801fd2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.043+0000 7f9fb065e740 1 -- 172.21.15.43:0/1368176909 shutdown_connections 2024-04-25T10:40:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.043+0000 7f9fb065e740 1 -- 172.21.15.43:0/1368176909 wait complete. 2024-04-25T10:40:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.043+0000 7f9fb065e740 1 -- start start 2024-04-25T10:40:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.043+0000 7f9fb065e740 1 Processor -- start 2024-04-25T10:40:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.043+0000 7f9fb065e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c0c4d0 0x55b801fe1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9fb065e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b801c146b0 0x55b801fe1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9fb065e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c35390 0x55b801fed1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9fb065e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b801fe3d30 con 0x55b801c0c4d0 2024-04-25T10:40:12.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9fb065e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b801ec2dc0 con 0x55b801c35390 2024-04-25T10:40:12.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9fb065e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b801fe3eb0 con 0x55b801c146b0 2024-04-25T10:40:12.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9fb065e740 10 client.0 _ll_register_callbacks cb 0x55b801ede600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:12.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9fa37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c0c4d0 0x55b801fe1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9fa37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c0c4d0 0x55b801fe1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53796/0 (socket says 172.21.15.43:53796) 2024-04-25T10:40:12.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9fa37fe640 1 -- 172.21.15.43:0/787531755 learned_addr learned my addr 172.21.15.43:0/787531755 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:12.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9f9a7fc640 1 --2- 172.21.15.43:0/787531755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b801c146b0 0x55b801fe1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9fa37fe640 1 --2- 172.21.15.43:0/787531755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c35390 0x55b801fed1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9fa37fe640 1 -- 172.21.15.43:0/787531755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b801c146b0 msgr2=0x55b801fe1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:12.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9fa37fe640 1 --2- 172.21.15.43:0/787531755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b801c146b0 0x55b801fe1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9fa37fe640 1 -- 172.21.15.43:0/787531755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c35390 msgr2=0x55b801fed1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9fa37fe640 1 --2- 172.21.15.43:0/787531755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c35390 0x55b801fed1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9fa37fe640 1 -- 172.21.15.43:0/787531755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b801c4d3f0 con 0x55b801c0c4d0 2024-04-25T10:40:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9fa37fe640 1 --2- 172.21.15.43:0/787531755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c0c4d0 0x55b801fe1210 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c003cf0 tx=0x7f9f9c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552e61075b13962 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9f8a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9f8a7fc640 1 -- 172.21.15.43:0/787531755 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c055060 con 0x55b801c0c4d0 2024-04-25T10:40:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9f8a7fc640 1 -- 172.21.15.43:0/787531755 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8557+0+0 (secure 0 0 0) 0x7f9f9c031860 con 0x55b801c0c4d0 2024-04-25T10:40:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.044+0000 7f9fb065e740 1 -- 172.21.15.43:0/787531755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f9c036070 con 0x55b801c0c4d0 2024-04-25T10:40:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.045+0000 7f9f8a7fc640 1 -- 172.21.15.43:0/787531755 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9f9c02f660 con 0x55b801c0c4d0 2024-04-25T10:40:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.045+0000 7f9fb065e740 10 client.17694 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.045+0000 7f9fb065e740 1 -- 172.21.15.43:0/787531755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b801d177d0 con 0x55b801c0c4d0 2024-04-25T10:40:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.045+0000 7f9f8a7fc640 1 -- 172.21.15.43:0/787531755 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f9c036070 con 0x55b801c0c4d0 2024-04-25T10:40:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.045+0000 7f9f8a7fc640 1 -- 172.21.15.43:0/787531755 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9f9c0309d0 con 0x55b801c0c4d0 2024-04-25T10:40:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.045+0000 7f9fb065e740 10 client.17694 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:12.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.045+0000 7f9fb065e740 10 client.17694 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:40:12.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.045+0000 7f9fb065e740 20 client.17694 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:12.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.045+0000 7f9fb065e740 1 --2- 172.21.15.43:0/787531755 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55b801bf3f60 0x55b801ff3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.045+0000 7f9fb065e740 4 client.17694 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:12.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.045+0000 7f9fb065e740 1 -- 172.21.15.43:0/787531755 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b801f49990 con 0x55b801bf3f60 2024-04-25T10:40:12.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.045+0000 7f9f9a7fc640 1 --2- 172.21.15.43:0/787531755 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55b801bf3f60 0x55b801ff3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.046+0000 7f9f8a7fc640 10 client.17694 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:40:12.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.046+0000 7f9f9a7fc640 1 --2- 172.21.15.43:0/787531755 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55b801bf3f60 0x55b801ff3430 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4b8b832c3aa1e4f server_cookie=6f3b2d2b6f3b121 in_seq=0 out_seq=0 2024-04-25T10:40:12.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.046+0000 7f9f8a7fc640 1 -- 172.21.15.43:0/787531755 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b801f49990 con 0x55b801bf3f60 2024-04-25T10:40:12.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.046+0000 7f9f8a7fc640 10 client.17694 handle_command_reply: tid=0 2024-04-25T10:40:12.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.046+0000 7f9fb065e740 1 client.17694 shutdown 2024-04-25T10:40:12.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.047+0000 7f9fb065e740 1 -- 172.21.15.43:0/787531755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c0c4d0 msgr2=0x55b801fe1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:12.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.047+0000 7f9fb065e740 1 --2- 172.21.15.43:0/787531755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c0c4d0 0x55b801fe1210 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c003cf0 tx=0x7f9f9c0095c0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.047+0000 7f9fb065e740 1 -- 172.21.15.43:0/787531755 shutdown_connections 2024-04-25T10:40:12.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.047+0000 7f9fb065e740 1 --2- 172.21.15.43:0/787531755 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55b801bf3f60 0x55b801ff3430 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.047+0000 7f9fb065e740 1 --2- 172.21.15.43:0/787531755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c35390 0x55b801fed1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.047+0000 7f9fb065e740 1 --2- 172.21.15.43:0/787531755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b801c146b0 0x55b801fe1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.047+0000 7f9f8a7fc640 0 client.17694 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:40:12.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.047+0000 7f9fb065e740 1 --2- 172.21.15.43:0/787531755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c0c4d0 0x55b801fe1210 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.047+0000 7f9fb065e740 1 -- 172.21.15.43:0/787531755 >> 172.21.15.43:0/787531755 conn(0x55b801c294e0 msgr2=0x55b801fd8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:12.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.047+0000 7f9fb065e740 1 -- 172.21.15.43:0/787531755 shutdown_connections 2024-04-25T10:40:12.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.047+0000 7f9fb065e740 1 -- 172.21.15.43:0/787531755 wait complete. 2024-04-25T10:40:12.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.047+0000 7f9fb065e740 20 client.17694 trim_cache size 0 max 16384 2024-04-25T10:40:12.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.061+0000 7f9fb065e740 1 Processor -- start 2024-04-25T10:40:12.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.061+0000 7f9fb065e740 1 -- start start 2024-04-25T10:40:12.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.062+0000 7f9fb065e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c0c4d0 0x55b801fed1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.062+0000 7f9fb065e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c146b0 0x55b801fe0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.062+0000 7f9fb065e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b801c35390 0x55b801fe2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.062+0000 7f9fb065e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa41b2180 con 0x55b801c146b0 2024-04-25T10:40:12.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.062+0000 7f9fb065e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa41afa50 con 0x55b801c0c4d0 2024-04-25T10:40:12.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.062+0000 7f9fb065e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa41ad8c0 con 0x55b801c35390 2024-04-25T10:40:12.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.062+0000 7f9f9a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c146b0 0x55b801fe0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.062+0000 7f9f9a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c146b0 0x55b801fe0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53800/0 (socket says 172.21.15.43:53800) 2024-04-25T10:40:12.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.062+0000 7f9f9a7fc640 1 -- 172.21.15.43:0/3349501547 learned_addr learned my addr 172.21.15.43:0/3349501547 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:12.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.062+0000 7f9fa37fe640 1 --2- 172.21.15.43:0/3349501547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c0c4d0 0x55b801fed1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.062+0000 7f9fa37fe640 1 --2- 172.21.15.43:0/3349501547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b801c35390 0x55b801fe2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.062+0000 7f9f9a7fc640 1 -- 172.21.15.43:0/3349501547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b801c35390 msgr2=0x55b801fe2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:12.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.062+0000 7f9f9a7fc640 1 --2- 172.21.15.43:0/3349501547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b801c35390 0x55b801fe2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.062+0000 7f9f9a7fc640 1 -- 172.21.15.43:0/3349501547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c0c4d0 msgr2=0x55b801fed1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:12.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.062+0000 7f9f9a7fc640 1 --2- 172.21.15.43:0/3349501547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c0c4d0 0x55b801fed1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.062+0000 7f9f9a7fc640 1 -- 172.21.15.43:0/3349501547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b801ff9400 con 0x55b801c146b0 2024-04-25T10:40:12.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.063+0000 7f9f9a7fc640 1 --2- 172.21.15.43:0/3349501547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c146b0 0x55b801fe0300 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0058c0 tx=0x7f9f8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0ad6ce530f889f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:12.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.063+0000 7f9f98ff9640 1 -- 172.21.15.43:0/3349501547 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c004230 con 0x55b801c146b0 2024-04-25T10:40:12.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.063+0000 7f9f98ff9640 1 -- 172.21.15.43:0/3349501547 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f8c022070 con 0x55b801c146b0 2024-04-25T10:40:12.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.063+0000 7f9f98ff9640 1 -- 172.21.15.43:0/3349501547 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c003a20 con 0x55b801c146b0 2024-04-25T10:40:12.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.064+0000 7f9fb065e740 1 -- 172.21.15.43:0/3349501547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c146b0 msgr2=0x55b801fe0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:12.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.064+0000 7f9fb065e740 1 --2- 172.21.15.43:0/3349501547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c146b0 0x55b801fe0300 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0058c0 tx=0x7f9f8c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.064+0000 7f9fb065e740 1 -- 172.21.15.43:0/3349501547 shutdown_connections 2024-04-25T10:40:12.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.064+0000 7f9fb065e740 1 --2- 172.21.15.43:0/3349501547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b801c35390 0x55b801fe2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.064+0000 7f9fb065e740 1 --2- 172.21.15.43:0/3349501547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c146b0 0x55b801fe0300 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.064+0000 7f9fb065e740 1 --2- 172.21.15.43:0/3349501547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c0c4d0 0x55b801fed1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.064+0000 7f9fb065e740 1 -- 172.21.15.43:0/3349501547 >> 172.21.15.43:0/3349501547 conn(0x55b801c294e0 msgr2=0x55b801fe2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:12.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.064+0000 7f9fb065e740 1 -- 172.21.15.43:0/3349501547 shutdown_connections 2024-04-25T10:40:12.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.064+0000 7f9fb065e740 1 -- 172.21.15.43:0/3349501547 wait complete. 2024-04-25T10:40:12.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.064+0000 7f9fb065e740 1 -- start start 2024-04-25T10:40:12.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.064+0000 7f9fb065e740 1 Processor -- start 2024-04-25T10:40:12.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.064+0000 7f9fb065e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c0c4d0 0x55b8020528e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.064+0000 7f9fb065e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b801c146b0 0x55b802052e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.064+0000 7f9fb065e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c35390 0x55b801ff3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.064+0000 7f9fb065e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa41adf10 con 0x55b801c35390 2024-04-25T10:40:12.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.065+0000 7f9fb065e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa41af380 con 0x55b801c0c4d0 2024-04-25T10:40:12.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.065+0000 7f9fb065e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa41ae5e0 con 0x55b801c146b0 2024-04-25T10:40:12.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.065+0000 7f9fb065e740 10 client.0 _ll_register_callbacks cb 0x7f9f9c02f3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:12.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.065+0000 7f9fa37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c35390 0x55b801ff3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.065+0000 7f9fa37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c0c4d0 0x55b8020528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.065+0000 7f9fa37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c0c4d0 0x55b8020528e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:52148/0 (socket says 172.21.15.43:52148) 2024-04-25T10:40:12.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.065+0000 7f9fa37fe640 1 -- 172.21.15.43:0/123789574 learned_addr learned my addr 172.21.15.43:0/123789574 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:12.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.065+0000 7f9f9a7fc640 1 --2- 172.21.15.43:0/123789574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b801c146b0 0x55b802052e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.065+0000 7f9fa37fe640 1 -- 172.21.15.43:0/123789574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b801c146b0 msgr2=0x55b802052e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:12.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.065+0000 7f9fa37fe640 1 --2- 172.21.15.43:0/123789574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b801c146b0 0x55b802052e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.065+0000 7f9fa37fe640 1 -- 172.21.15.43:0/123789574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c35390 msgr2=0x55b801ff3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:12.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.065+0000 7f9fa37fe640 1 --2- 172.21.15.43:0/123789574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c35390 0x55b801ff3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.065+0000 7f9fa37fe640 1 -- 172.21.15.43:0/123789574 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b801ff3f80 con 0x55b801c0c4d0 2024-04-25T10:40:12.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.065+0000 7f9fa37fe640 1 --2- 172.21.15.43:0/123789574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c0c4d0 0x55b8020528e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c02ade0 tx=0x7f9f9c000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fb21607cac1df12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:12.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.065+0000 7f9f8a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-04-25T10:40:12.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.065+0000 7f9f8a7fc640 1 -- 172.21.15.43:0/123789574 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c0305c0 con 0x55b801c0c4d0 2024-04-25T10:40:12.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.065+0000 7f9fb065e740 1 -- 172.21.15.43:0/123789574 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f8c022070 con 0x55b801c0c4d0 2024-04-25T10:40:12.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.065+0000 7f9f8a7fc640 1 -- 172.21.15.43:0/123789574 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8557+0+0 (secure 0 0 0) 0x7f9f9c030760 con 0x55b801c0c4d0 2024-04-25T10:40:12.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.066+0000 7f9f8a7fc640 1 -- 172.21.15.43:0/123789574 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9f9c030960 con 0x55b801c0c4d0 2024-04-25T10:40:12.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.066+0000 7f9fb065e740 10 client.27068 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:12.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.066+0000 7f9fb065e740 1 -- 172.21.15.43:0/123789574 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b80205af30 con 0x55b801c0c4d0 2024-04-25T10:40:12.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.066+0000 7f9f8a7fc640 1 -- 172.21.15.43:0/123789574 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f8c022070 con 0x55b801c0c4d0 2024-04-25T10:40:12.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.066+0000 7f9f8a7fc640 1 -- 172.21.15.43:0/123789574 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9f9c059720 con 0x55b801c0c4d0 2024-04-25T10:40:12.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.066+0000 7f9fb065e740 10 client.27068 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:12.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.066+0000 7f9fb065e740 10 client.27068 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:40:12.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.066+0000 7f9fb065e740 20 client.27068 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:12.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.066+0000 7f9fb065e740 1 --2- 172.21.15.43:0/123789574 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55b801bc93a0 0x55b80205d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.066+0000 7f9fb065e740 4 client.27068 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:40:12.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.066+0000 7f9fb065e740 1 -- 172.21.15.43:0/123789574 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b801f43800 con 0x55b801bc93a0 2024-04-25T10:40:12.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.066+0000 7f9f9a7fc640 1 --2- 172.21.15.43:0/123789574 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55b801bc93a0 0x55b80205d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.067+0000 7f9f8a7fc640 10 client.27068 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:40:12.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.067+0000 7f9f9a7fc640 1 --2- 172.21.15.43:0/123789574 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55b801bc93a0 0x55b80205d570 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=722c492daa412d1 server_cookie=dace18bf364096c9 in_seq=0 out_seq=0 2024-04-25T10:40:12.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.067+0000 7f9f8a7fc640 1 -- 172.21.15.43:0/123789574 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b801f43800 con 0x55b801bc93a0 2024-04-25T10:40:12.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.067+0000 7f9f8a7fc640 10 client.27068 handle_command_reply: tid=0 2024-04-25T10:40:12.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.067+0000 7f9fb065e740 1 client.27068 shutdown 2024-04-25T10:40:12.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.067+0000 7f9fb065e740 1 -- 172.21.15.43:0/123789574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c0c4d0 msgr2=0x55b8020528e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:12.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.067+0000 7f9fb065e740 1 --2- 172.21.15.43:0/123789574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c0c4d0 0x55b8020528e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c02ade0 tx=0x7f9f9c000f40 comp rx=0 tx=0).stop 2024-04-25T10:40:12.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.067+0000 7f9fb065e740 1 -- 172.21.15.43:0/123789574 shutdown_connections 2024-04-25T10:40:12.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.067+0000 7f9fb065e740 1 --2- 172.21.15.43:0/123789574 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55b801bc93a0 0x55b80205d570 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.067+0000 7f9fb065e740 1 --2- 172.21.15.43:0/123789574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b801c35390 0x55b801ff3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.067+0000 7f9fb065e740 1 --2- 172.21.15.43:0/123789574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b801c146b0 0x55b802052e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.067+0000 7f9fb065e740 1 --2- 172.21.15.43:0/123789574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b801c0c4d0 0x55b8020528e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.067+0000 7f9f8a7fc640 0 client.27068 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:40:12.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.067+0000 7f9fb065e740 1 -- 172.21.15.43:0/123789574 >> 172.21.15.43:0/123789574 conn(0x55b801c294e0 msgr2=0x55b801fe0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:12.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.068+0000 7f9fb065e740 1 -- 172.21.15.43:0/123789574 shutdown_connections 2024-04-25T10:40:12.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.068+0000 7f9fb065e740 1 -- 172.21.15.43:0/123789574 wait complete. 2024-04-25T10:40:12.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.068+0000 7f9fb065e740 20 client.27068 trim_cache size 0 max 16384 2024-04-25T10:40:12.077 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:40:12.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.068+0000 7f9fa97b2640 1 -- 172.21.15.43:0/3380165578 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9f900afc60 msgr2=0x7f9f900b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:12.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.068+0000 7f9fa97b2640 1 --2- 172.21.15.43:0/3380165578 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9f900afc60 0x7f9f900b2170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9f940dc310 tx=0x7f9f9400cf90 comp rx=0 tx=0).stop 2024-04-25T10:40:12.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.068+0000 7f9fa97b2640 1 -- 172.21.15.43:0/3380165578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa4102b00 msgr2=0x7f9fa419a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:12.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.068+0000 7f9fa97b2640 1 --2- 172.21.15.43:0/3380165578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa4102b00 0x7f9fa419a7e0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400c9a0 tx=0x7f9f9400ce60 comp rx=0 tx=0).stop 2024-04-25T10:40:12.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.068+0000 7f9fa97b2640 1 -- 172.21.15.43:0/3380165578 shutdown_connections 2024-04-25T10:40:12.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.068+0000 7f9fa97b2640 1 --2- 172.21.15.43:0/3380165578 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9f900afc60 0x7f9f900b2170 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.068+0000 7f9fa97b2640 1 --2- 172.21.15.43:0/3380165578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9fa4108a30 0x7f9fa419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.068+0000 7f9fa97b2640 1 --2- 172.21.15.43:0/3380165578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fa410d390 0x7f9fa419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.068+0000 7f9fa97b2640 1 --2- 172.21.15.43:0/3380165578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa4102b00 0x7f9fa419a7e0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.068+0000 7f9fa97b2640 1 -- 172.21.15.43:0/3380165578 >> 172.21.15.43:0/3380165578 conn(0x7f9fa40fe260 msgr2=0x7f9fa41009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:12.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.068+0000 7f9fa97b2640 1 -- 172.21.15.43:0/3380165578 shutdown_connections 2024-04-25T10:40:12.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.068+0000 7f9fa97b2640 1 -- 172.21.15.43:0/3380165578 wait complete. 2024-04-25T10:40:12.082 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:40:12.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.193+0000 7f64dbe92640 1 -- 172.21.15.43:0/300183677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d4107440 msgr2=0x7f64d410ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:12.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.193+0000 7f64dbe92640 1 --2- 172.21.15.43:0/300183677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d4107440 0x7f64d410ebe0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009ca0 tx=0x7f64cc02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.193+0000 7f64dbe92640 1 -- 172.21.15.43:0/300183677 shutdown_connections 2024-04-25T10:40:12.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.193+0000 7f64dbe92640 1 --2- 172.21.15.43:0/300183677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d4107440 0x7f64d410ebe0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.193+0000 7f64dbe92640 1 --2- 172.21.15.43:0/300183677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64d4106980 0x7f64d4106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.193+0000 7f64dbe92640 1 --2- 172.21.15.43:0/300183677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64d41023f0 0x7f64d4102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.193+0000 7f64dbe92640 1 -- 172.21.15.43:0/300183677 >> 172.21.15.43:0/300183677 conn(0x7f64d40fd850 msgr2=0x7f64d40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:12.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.193+0000 7f64dbe92640 1 -- 172.21.15.43:0/300183677 shutdown_connections 2024-04-25T10:40:12.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.193+0000 7f64dbe92640 1 -- 172.21.15.43:0/300183677 wait complete. 2024-04-25T10:40:12.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.194+0000 7f64dbe92640 1 Processor -- start 2024-04-25T10:40:12.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.194+0000 7f64dbe92640 1 -- start start 2024-04-25T10:40:12.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.194+0000 7f64dbe92640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64d41023f0 0x7f64d41996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.194+0000 7f64dbe92640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d4106980 0x7f64d4199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.194+0000 7f64dbe92640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64d4107440 0x7f64d419ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.194+0000 7f64dbe92640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f64d4112350 con 0x7f64d4106980 2024-04-25T10:40:12.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.194+0000 7f64dbe92640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41121d0 con 0x7f64d4107440 2024-04-25T10:40:12.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.194+0000 7f64dbe92640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41124d0 con 0x7f64d41023f0 2024-04-25T10:40:12.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.194+0000 7f64d942d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d4106980 0x7f64d4199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.194+0000 7f64d942d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d4106980 0x7f64d4199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53834/0 (socket says 172.21.15.43:53834) 2024-04-25T10:40:12.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.194+0000 7f64d942d640 1 -- 172.21.15.43:0/3100199427 learned_addr learned my addr 172.21.15.43:0/3100199427 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:12.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.194+0000 7f64d9c2e640 1 --2- 172.21.15.43:0/3100199427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64d41023f0 0x7f64d41996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.194+0000 7f64da42f640 1 --2- 172.21.15.43:0/3100199427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64d4107440 0x7f64d419ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.194+0000 7f64d942d640 1 -- 172.21.15.43:0/3100199427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64d41023f0 msgr2=0x7f64d41996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:12.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.194+0000 7f64d942d640 1 --2- 172.21.15.43:0/3100199427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64d41023f0 0x7f64d41996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.194+0000 7f64d942d640 1 -- 172.21.15.43:0/3100199427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64d4107440 msgr2=0x7f64d419ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:12.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.194+0000 7f64d942d640 1 --2- 172.21.15.43:0/3100199427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64d4107440 0x7f64d419ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.194+0000 7f64d942d640 1 -- 172.21.15.43:0/3100199427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d419e510 con 0x7f64d4106980 2024-04-25T10:40:12.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.194+0000 7f64da42f640 1 --2- 172.21.15.43:0/3100199427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64d4107440 0x7f64d419ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:12.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.195+0000 7f64d942d640 1 --2- 172.21.15.43:0/3100199427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d4106980 0x7f64d4199c10 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f64c400d740 tx=0x7f64c400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:12.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.195+0000 7f64caffd640 1 -- 172.21.15.43:0/3100199427 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4017070 con 0x7f64d4106980 2024-04-25T10:40:12.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.195+0000 7f64caffd640 1 -- 172.21.15.43:0/3100199427 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c400de50 con 0x7f64d4106980 2024-04-25T10:40:12.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.195+0000 7f64caffd640 1 -- 172.21.15.43:0/3100199427 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4004ce0 con 0x7f64d4106980 2024-04-25T10:40:12.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.195+0000 7f64dbe92640 1 -- 172.21.15.43:0/3100199427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d419e800 con 0x7f64d4106980 2024-04-25T10:40:12.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.195+0000 7f64dbe92640 1 -- 172.21.15.43:0/3100199427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d419ed00 con 0x7f64d4106980 2024-04-25T10:40:12.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.197+0000 7f64e2e14740 1 Processor -- start 2024-04-25T10:40:12.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.197+0000 7f64caffd640 1 -- 172.21.15.43:0/3100199427 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f64c4004e80 con 0x7f64d4106980 2024-04-25T10:40:12.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.198+0000 7f64e2e14740 1 -- start start 2024-04-25T10:40:12.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.198+0000 7f64e2e14740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 0x562167df4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.198+0000 7f64e2e14740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a2a4d0 0x562167df4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.198+0000 7f64e2e14740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a326b0 0x562167e016a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.198+0000 7f64e2e14740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562167d59d80 con 0x562167a326b0 2024-04-25T10:40:12.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.198+0000 7f64e2e14740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562167d13a20 con 0x562167a53390 2024-04-25T10:40:12.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.198+0000 7f64e2e14740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562167d55d40 con 0x562167a2a4d0 2024-04-25T10:40:12.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.198+0000 7f64d942d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a326b0 0x562167e016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.198+0000 7f64d942d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a326b0 0x562167e016a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53840/0 (socket says 172.21.15.43:53840) 2024-04-25T10:40:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.198+0000 7f64d942d640 1 -- 172.21.15.43:0/99873925 learned_addr learned my addr 172.21.15.43:0/99873925 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.198+0000 7f64caffd640 1 --2- 172.21.15.43:0/3100199427 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f64c00afc60 0x7f64c00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.198+0000 7f64da42f640 1 --2- 172.21.15.43:0/99873925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 0x562167df4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.198+0000 7f64d9c2e640 1 --2- 172.21.15.43:0/99873925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a2a4d0 0x562167df4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.198+0000 7f64da42f640 1 --2- 172.21.15.43:0/3100199427 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f64c00afc60 0x7f64c00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.198+0000 7f64caffd640 1 -- 172.21.15.43:0/3100199427 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8557+0+0 (secure 0 0 0) 0x7f64c40dea30 con 0x7f64d4106980 2024-04-25T10:40:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.198+0000 7f64d942d640 1 -- 172.21.15.43:0/99873925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a2a4d0 msgr2=0x562167df4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.199+0000 7f64d942d640 1 --2- 172.21.15.43:0/99873925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a2a4d0 0x562167df4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.199+0000 7f64d942d640 1 -- 172.21.15.43:0/99873925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 msgr2=0x562167df4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.199+0000 7f64d942d640 1 --2- 172.21.15.43:0/99873925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 0x562167df4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.199+0000 7f64d942d640 1 -- 172.21.15.43:0/99873925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562167d1ec30 con 0x562167a326b0 2024-04-25T10:40:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.199+0000 7f64da42f640 1 --2- 172.21.15.43:0/99873925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 0x562167df4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.199+0000 7f64d942d640 1 --2- 172.21.15.43:0/99873925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a326b0 0x562167e016a0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f64c40a6b40 tx=0x7f64c40a23d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad391ac04b690ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.199+0000 7f64bffff640 1 -- 172.21.15.43:0/99873925 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4020070 con 0x562167a326b0 2024-04-25T10:40:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.199+0000 7f64da42f640 1 --2- 172.21.15.43:0/3100199427 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f64c00afc60 0x7f64c00b2170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009a60 tx=0x7f64cc005fd0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.199+0000 7f64bffff640 1 -- 172.21.15.43:0/99873925 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c40a66c0 con 0x562167a326b0 2024-04-25T10:40:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.199+0000 7f64bffff640 1 -- 172.21.15.43:0/99873925 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c40c7e60 con 0x562167a326b0 2024-04-25T10:40:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.200+0000 7f64e2e14740 1 -- 172.21.15.43:0/99873925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a326b0 msgr2=0x562167e016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.200+0000 7f64e2e14740 1 --2- 172.21.15.43:0/99873925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a326b0 0x562167e016a0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f64c40a6b40 tx=0x7f64c40a23d0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.200+0000 7f64e2e14740 1 -- 172.21.15.43:0/99873925 shutdown_connections 2024-04-25T10:40:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.200+0000 7f64e2e14740 1 --2- 172.21.15.43:0/99873925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a326b0 0x562167e016a0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.200+0000 7f64e2e14740 1 --2- 172.21.15.43:0/99873925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a2a4d0 0x562167df4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.200+0000 7f64e2e14740 1 --2- 172.21.15.43:0/99873925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 0x562167df4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.200+0000 7f64e2e14740 1 -- 172.21.15.43:0/99873925 >> 172.21.15.43:0/99873925 conn(0x562167a474e0 msgr2=0x562167df0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.200+0000 7f64e2e14740 1 -- 172.21.15.43:0/99873925 shutdown_connections 2024-04-25T10:40:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.200+0000 7f64e2e14740 1 -- 172.21.15.43:0/99873925 wait complete. 2024-04-25T10:40:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.200+0000 7f64e2e14740 1 -- start start 2024-04-25T10:40:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.200+0000 7f64e2e14740 1 Processor -- start 2024-04-25T10:40:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.201+0000 7f64e2e14740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a2a4d0 0x562167dff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.201+0000 7f64e2e14740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a326b0 0x562167dff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.201+0000 7f64e2e14740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 0x562167e0b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.201+0000 7f64e2e14740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562167e01cd0 con 0x562167a2a4d0 2024-04-25T10:40:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.201+0000 7f64e2e14740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562167ce0dc0 con 0x562167a53390 2024-04-25T10:40:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.201+0000 7f64e2e14740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562167e01e50 con 0x562167a326b0 2024-04-25T10:40:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.201+0000 7f64e2e14740 10 client.0 _ll_register_callbacks cb 0x562167cfc600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.201+0000 7f64d9c2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a2a4d0 0x562167dff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.201+0000 7f64d9c2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a2a4d0 0x562167dff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53842/0 (socket says 172.21.15.43:53842) 2024-04-25T10:40:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.201+0000 7f64d9c2e640 1 -- 172.21.15.43:0/398972609 learned_addr learned my addr 172.21.15.43:0/398972609 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.201+0000 7f64da42f640 1 --2- 172.21.15.43:0/398972609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 0x562167e0b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.201+0000 7f64d942d640 1 --2- 172.21.15.43:0/398972609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a326b0 0x562167dff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.201+0000 7f64d9c2e640 1 -- 172.21.15.43:0/398972609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a326b0 msgr2=0x562167dff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.201+0000 7f64d9c2e640 1 --2- 172.21.15.43:0/398972609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a326b0 0x562167dff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.201+0000 7f64d9c2e640 1 -- 172.21.15.43:0/398972609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 msgr2=0x562167e0b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.201+0000 7f64d9c2e640 1 --2- 172.21.15.43:0/398972609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 0x562167e0b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.201+0000 7f64d9c2e640 1 -- 172.21.15.43:0/398972609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562167a6b3f0 con 0x562167a2a4d0 2024-04-25T10:40:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.202+0000 7f64d9c2e640 1 --2- 172.21.15.43:0/398972609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a2a4d0 0x562167dff2a0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f64d0010740 tx=0x7f64d0010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf7669a79082b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.202+0000 7f64bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.202+0000 7f64bd7fa640 1 -- 172.21.15.43:0/398972609 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d000cd20 con 0x562167a2a4d0 2024-04-25T10:40:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.202+0000 7f64bd7fa640 1 -- 172.21.15.43:0/398972609 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8557+0+0 (secure 0 0 0) 0x7f64d0011440 con 0x562167a2a4d0 2024-04-25T10:40:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.202+0000 7f64e2e14740 1 -- 172.21.15.43:0/398972609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64c40a66c0 con 0x562167a2a4d0 2024-04-25T10:40:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.202+0000 7f64bd7fa640 1 -- 172.21.15.43:0/398972609 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f64d0010e50 con 0x562167a2a4d0 2024-04-25T10:40:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.202+0000 7f64e2e14740 10 client.17712 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.202+0000 7f64e2e14740 1 -- 172.21.15.43:0/398972609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562167b357d0 con 0x562167a2a4d0 2024-04-25T10:40:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.202+0000 7f64bd7fa640 1 -- 172.21.15.43:0/398972609 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c40a66c0 con 0x562167a2a4d0 2024-04-25T10:40:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.202+0000 7f64bd7fa640 1 -- 172.21.15.43:0/398972609 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f64d001b740 con 0x562167a2a4d0 2024-04-25T10:40:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.202+0000 7f64e2e14740 10 client.17712 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.202+0000 7f64e2e14740 10 client.17712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.202+0000 7f64e2e14740 20 client.17712 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.202+0000 7f64e2e14740 1 --2- 172.21.15.43:0/398972609 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562167a11f60 0x562167e11670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.202+0000 7f64e2e14740 4 client.17712 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.202+0000 7f64e2e14740 1 -- 172.21.15.43:0/398972609 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562167d67990 con 0x562167a11f60 2024-04-25T10:40:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.203+0000 7f64d942d640 1 --2- 172.21.15.43:0/398972609 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562167a11f60 0x562167e11670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.203+0000 7f64bd7fa640 10 client.17712 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.203+0000 7f64d942d640 1 --2- 172.21.15.43:0/398972609 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562167a11f60 0x562167e11670 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce191a41ebb4c0f9 server_cookie=2bd9267fcb790fa1 in_seq=0 out_seq=0 2024-04-25T10:40:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.204+0000 7f64bd7fa640 1 -- 172.21.15.43:0/398972609 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562167d67990 con 0x562167a11f60 2024-04-25T10:40:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.204+0000 7f64bd7fa640 10 client.17712 handle_command_reply: tid=0 2024-04-25T10:40:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.204+0000 7f64e2e14740 1 client.17712 shutdown 2024-04-25T10:40:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.204+0000 7f64e2e14740 1 -- 172.21.15.43:0/398972609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a2a4d0 msgr2=0x562167dff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.204+0000 7f64e2e14740 1 --2- 172.21.15.43:0/398972609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a2a4d0 0x562167dff2a0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f64d0010740 tx=0x7f64d0010c00 comp rx=0 tx=0).stop 2024-04-25T10:40:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.204+0000 7f64e2e14740 1 -- 172.21.15.43:0/398972609 shutdown_connections 2024-04-25T10:40:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.204+0000 7f64e2e14740 1 --2- 172.21.15.43:0/398972609 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x562167a11f60 0x562167e11670 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.204+0000 7f64e2e14740 1 --2- 172.21.15.43:0/398972609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 0x562167e0b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.204+0000 7f64bd7fa640 0 client.17712 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.204+0000 7f64e2e14740 1 --2- 172.21.15.43:0/398972609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a326b0 0x562167dff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.204+0000 7f64e2e14740 1 --2- 172.21.15.43:0/398972609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a2a4d0 0x562167dff2a0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.204+0000 7f64e2e14740 1 -- 172.21.15.43:0/398972609 >> 172.21.15.43:0/398972609 conn(0x562167a474e0 msgr2=0x562167df6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.204+0000 7f64e2e14740 1 -- 172.21.15.43:0/398972609 shutdown_connections 2024-04-25T10:40:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.204+0000 7f64e2e14740 1 -- 172.21.15.43:0/398972609 wait complete. 2024-04-25T10:40:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.204+0000 7f64e2e14740 20 client.17712 trim_cache size 0 max 16384 2024-04-25T10:40:12.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.218+0000 7f64e2e14740 1 Processor -- start 2024-04-25T10:40:12.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.218+0000 7f64e2e14740 1 -- start start 2024-04-25T10:40:12.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.218+0000 7f64e2e14740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a2a4d0 0x562167df1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.218+0000 7f64e2e14740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a326b0 0x562167e0adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.218+0000 7f64e2e14740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 0x562167e00ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.218+0000 7f64e2e14740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41b1210 con 0x562167a2a4d0 2024-04-25T10:40:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.218+0000 7f64e2e14740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41aeae0 con 0x562167a53390 2024-04-25T10:40:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.218+0000 7f64e2e14740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41ac950 con 0x562167a326b0 2024-04-25T10:40:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.218+0000 7f64d9c2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a2a4d0 0x562167df1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.218+0000 7f64d9c2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a2a4d0 0x562167df1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53858/0 (socket says 172.21.15.43:53858) 2024-04-25T10:40:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.218+0000 7f64d9c2e640 1 -- 172.21.15.43:0/2131187264 learned_addr learned my addr 172.21.15.43:0/2131187264 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.219+0000 7f64da42f640 1 --2- 172.21.15.43:0/2131187264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 0x562167e00ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.219+0000 7f64d9c2e640 1 -- 172.21.15.43:0/2131187264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a326b0 msgr2=0x562167e0adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.219+0000 7f64d942d640 1 --2- 172.21.15.43:0/2131187264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a326b0 0x562167e0adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.219+0000 7f64d9c2e640 1 --2- 172.21.15.43:0/2131187264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a326b0 0x562167e0adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.219+0000 7f64d9c2e640 1 -- 172.21.15.43:0/2131187264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 msgr2=0x562167e00ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.219+0000 7f64d9c2e640 1 --2- 172.21.15.43:0/2131187264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 0x562167e00ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.219+0000 7f64d9c2e640 1 -- 172.21.15.43:0/2131187264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562167df1b60 con 0x562167a2a4d0 2024-04-25T10:40:12.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.219+0000 7f64da42f640 1 --2- 172.21.15.43:0/2131187264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 0x562167e00ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:12.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.219+0000 7f64d942d640 1 --2- 172.21.15.43:0/2131187264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a326b0 0x562167e0adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:12.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.219+0000 7f64d9c2e640 1 --2- 172.21.15.43:0/2131187264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a2a4d0 0x562167df1030 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f64d0000c00 tx=0x7f64d001bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18caf356052fb3f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:12.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.219+0000 7f64bf7fe640 1 -- 172.21.15.43:0/2131187264 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0027040 con 0x562167a2a4d0 2024-04-25T10:40:12.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.219+0000 7f64bf7fe640 1 -- 172.21.15.43:0/2131187264 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64d001b8e0 con 0x562167a2a4d0 2024-04-25T10:40:12.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.219+0000 7f64bf7fe640 1 -- 172.21.15.43:0/2131187264 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d00055a0 con 0x562167a2a4d0 2024-04-25T10:40:12.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.220+0000 7f64e2e14740 1 -- 172.21.15.43:0/2131187264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a2a4d0 msgr2=0x562167df1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:12.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.220+0000 7f64e2e14740 1 --2- 172.21.15.43:0/2131187264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a2a4d0 0x562167df1030 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f64d0000c00 tx=0x7f64d001bdd0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.220+0000 7f64e2e14740 1 -- 172.21.15.43:0/2131187264 shutdown_connections 2024-04-25T10:40:12.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.220+0000 7f64e2e14740 1 --2- 172.21.15.43:0/2131187264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 0x562167e00ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.220+0000 7f64e2e14740 1 --2- 172.21.15.43:0/2131187264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a326b0 0x562167e0adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.220+0000 7f64e2e14740 1 --2- 172.21.15.43:0/2131187264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a2a4d0 0x562167df1030 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.220+0000 7f64e2e14740 1 -- 172.21.15.43:0/2131187264 >> 172.21.15.43:0/2131187264 conn(0x562167a474e0 msgr2=0x562167e007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:12.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.220+0000 7f64e2e14740 1 -- 172.21.15.43:0/2131187264 shutdown_connections 2024-04-25T10:40:12.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.220+0000 7f64e2e14740 1 -- 172.21.15.43:0/2131187264 wait complete. 2024-04-25T10:40:12.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.220+0000 7f64e2e14740 1 -- start start 2024-04-25T10:40:12.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.220+0000 7f64e2e14740 1 Processor -- start 2024-04-25T10:40:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64e2e14740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a2a4d0 0x562167e04920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64e2e14740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a326b0 0x562167e04e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64e2e14740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 0x562167e06d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64e2e14740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41acfa0 con 0x562167a326b0 2024-04-25T10:40:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64e2e14740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41ae410 con 0x562167a53390 2024-04-25T10:40:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64e2e14740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41ad670 con 0x562167a2a4d0 2024-04-25T10:40:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64e2e14740 10 client.0 _ll_register_callbacks cb 0x7f64d000e390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64d942d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a326b0 0x562167e04e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64d942d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a326b0 0x562167e04e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53872/0 (socket says 172.21.15.43:53872) 2024-04-25T10:40:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64d942d640 1 -- 172.21.15.43:0/1587075042 learned_addr learned my addr 172.21.15.43:0/1587075042 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64da42f640 1 --2- 172.21.15.43:0/1587075042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 0x562167e06d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64d9c2e640 1 --2- 172.21.15.43:0/1587075042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a2a4d0 0x562167e04920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64d942d640 1 -- 172.21.15.43:0/1587075042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a2a4d0 msgr2=0x562167e04920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:12.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64d942d640 1 --2- 172.21.15.43:0/1587075042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a2a4d0 0x562167e04920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64d942d640 1 -- 172.21.15.43:0/1587075042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 msgr2=0x562167e06d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:12.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64d942d640 1 --2- 172.21.15.43:0/1587075042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 0x562167e06d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64d942d640 1 -- 172.21.15.43:0/1587075042 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562167e072b0 con 0x562167a326b0 2024-04-25T10:40:12.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64d9c2e640 1 --2- 172.21.15.43:0/1587075042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a2a4d0 0x562167e04920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:12.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64da42f640 1 --2- 172.21.15.43:0/1587075042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 0x562167e06d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:12.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64d942d640 1 --2- 172.21.15.43:0/1587075042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a326b0 0x562167e04e60 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f64c40a3e20 tx=0x7f64c40a3330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641381a2e8fcd95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:12.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.221+0000 7f64bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:12.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.222+0000 7f64bd7fa640 1 -- 172.21.15.43:0/1587075042 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4020040 con 0x562167a326b0 2024-04-25T10:40:12.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.222+0000 7f64bd7fa640 1 -- 172.21.15.43:0/1587075042 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8557+0+0 (secure 0 0 0) 0x7f64c40a5e10 con 0x562167a326b0 2024-04-25T10:40:12.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.222+0000 7f64e2e14740 1 -- 172.21.15.43:0/1587075042 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64d001b8e0 con 0x562167a326b0 2024-04-25T10:40:12.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.222+0000 7f64bd7fa640 1 -- 172.21.15.43:0/1587075042 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f64c40c7e60 con 0x562167a326b0 2024-04-25T10:40:12.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.222+0000 7f64e2e14740 10 client.17718 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:12.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.222+0000 7f64e2e14740 1 -- 172.21.15.43:0/1587075042 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562167e7b450 con 0x562167a326b0 2024-04-25T10:40:12.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.222+0000 7f64bd7fa640 1 -- 172.21.15.43:0/1587075042 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64d001b8e0 con 0x562167a326b0 2024-04-25T10:40:12.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.222+0000 7f64bd7fa640 1 -- 172.21.15.43:0/1587075042 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f64c40a5880 con 0x562167a326b0 2024-04-25T10:40:12.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.222+0000 7f64e2e14740 10 client.17718 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:12.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.222+0000 7f64e2e14740 10 client.17718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:12.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.222+0000 7f64e2e14740 20 client.17718 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:12.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.222+0000 7f64e2e14740 1 --2- 172.21.15.43:0/1587075042 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5621679e73a0 0x562167e7d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:12.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.222+0000 7f64e2e14740 4 client.17718 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:40:12.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.222+0000 7f64e2e14740 1 -- 172.21.15.43:0/1587075042 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562167d61800 con 0x5621679e73a0 2024-04-25T10:40:12.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.223+0000 7f64d9c2e640 1 --2- 172.21.15.43:0/1587075042 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5621679e73a0 0x562167e7d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:12.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.223+0000 7f64d9c2e640 1 --2- 172.21.15.43:0/1587075042 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5621679e73a0 0x562167e7d6f0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d517ac6f361851cd server_cookie=16c04d327b178116 in_seq=0 out_seq=0 2024-04-25T10:40:12.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.223+0000 7f64bd7fa640 10 client.17718 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:12.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.223+0000 7f64bd7fa640 1 -- 172.21.15.43:0/1587075042 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562167d61800 con 0x5621679e73a0 2024-04-25T10:40:12.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.223+0000 7f64bd7fa640 10 client.17718 handle_command_reply: tid=0 2024-04-25T10:40:12.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.223+0000 7f64e2e14740 1 client.17718 shutdown 2024-04-25T10:40:12.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.223+0000 7f64e2e14740 1 -- 172.21.15.43:0/1587075042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a326b0 msgr2=0x562167e04e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:12.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.223+0000 7f64e2e14740 1 --2- 172.21.15.43:0/1587075042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a326b0 0x562167e04e60 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f64c40a3e20 tx=0x7f64c40a3330 comp rx=0 tx=0).stop 2024-04-25T10:40:12.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.223+0000 7f64e2e14740 1 -- 172.21.15.43:0/1587075042 shutdown_connections 2024-04-25T10:40:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.223+0000 7f64e2e14740 1 --2- 172.21.15.43:0/1587075042 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5621679e73a0 0x562167e7d6f0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.223+0000 7f64e2e14740 1 --2- 172.21.15.43:0/1587075042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562167a53390 0x562167e06d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.223+0000 7f64e2e14740 1 --2- 172.21.15.43:0/1587075042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562167a326b0 0x562167e04e60 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.223+0000 7f64bd7fa640 0 client.17718 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.223+0000 7f64e2e14740 1 --2- 172.21.15.43:0/1587075042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562167a2a4d0 0x562167e04920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.223+0000 7f64e2e14740 1 -- 172.21.15.43:0/1587075042 >> 172.21.15.43:0/1587075042 conn(0x562167a474e0 msgr2=0x562167dfdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.223+0000 7f64e2e14740 1 -- 172.21.15.43:0/1587075042 shutdown_connections 2024-04-25T10:40:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.223+0000 7f64e2e14740 1 -- 172.21.15.43:0/1587075042 wait complete. 2024-04-25T10:40:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.224+0000 7f64e2e14740 20 client.17718 trim_cache size 0 max 16384 2024-04-25T10:40:12.237 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:40:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.224+0000 7f64dbe92640 1 -- 172.21.15.43:0/3100199427 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f64c00afc60 msgr2=0x7f64c00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.224+0000 7f64dbe92640 1 --2- 172.21.15.43:0/3100199427 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f64c00afc60 0x7f64c00b2170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009a60 tx=0x7f64cc005fd0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.224+0000 7f64dbe92640 1 -- 172.21.15.43:0/3100199427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d4106980 msgr2=0x7f64d4199c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.224+0000 7f64dbe92640 1 --2- 172.21.15.43:0/3100199427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d4106980 0x7f64d4199c10 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f64c400d740 tx=0x7f64c400dc00 comp rx=0 tx=0).stop 2024-04-25T10:40:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.224+0000 7f64dbe92640 1 -- 172.21.15.43:0/3100199427 shutdown_connections 2024-04-25T10:40:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.224+0000 7f64dbe92640 1 --2- 172.21.15.43:0/3100199427 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f64c00afc60 0x7f64c00b2170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.224+0000 7f64dbe92640 1 --2- 172.21.15.43:0/3100199427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64d4107440 0x7f64d419ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.224+0000 7f64dbe92640 1 --2- 172.21.15.43:0/3100199427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d4106980 0x7f64d4199c10 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.224+0000 7f64dbe92640 1 --2- 172.21.15.43:0/3100199427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64d41023f0 0x7f64d41996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.224+0000 7f64dbe92640 1 -- 172.21.15.43:0/3100199427 >> 172.21.15.43:0/3100199427 conn(0x7f64d40fd850 msgr2=0x7f64d4105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.224+0000 7f64dbe92640 1 -- 172.21.15.43:0/3100199427 shutdown_connections 2024-04-25T10:40:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:12.224+0000 7f64dbe92640 1 -- 172.21.15.43:0/3100199427 wait complete. 2024-04-25T10:40:12.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:12 smithi043 ceph-mon[29478]: pgmap v236: 97 pgs: 97 active+clean; 4.7 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 20 op/s 2024-04-25T10:40:12.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:12 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3835257674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:40:12.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:12 smithi077 ceph-mon[32772]: pgmap v236: 97 pgs: 97 active+clean; 4.7 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 20 op/s 2024-04-25T10:40:12.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:12 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3835257674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:40:12.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:12 smithi164 ceph-mon[32713]: pgmap v236: 97 pgs: 97 active+clean; 4.7 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 20 op/s 2024-04-25T10:40:12.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:12 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3835257674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:40:13.242 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:40:13.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:40:13.242 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:40:13.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.370+0000 7fc9d9496640 1 -- 172.21.15.43:0/1106364215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9d4109fb0 msgr2=0x7fc9d4103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.370+0000 7fc9d9496640 1 --2- 172.21.15.43:0/1106364215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9d4109fb0 0x7fc9d4103c40 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc009900 tx=0x7fc9bc02a410 comp rx=0 tx=0).stop 2024-04-25T10:40:13.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.370+0000 7fc9d9496640 1 -- 172.21.15.43:0/1106364215 shutdown_connections 2024-04-25T10:40:13.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.370+0000 7fc9d9496640 1 --2- 172.21.15.43:0/1106364215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9d41042b0 0x7fc9d4110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.370+0000 7fc9d9496640 1 --2- 172.21.15.43:0/1106364215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9d4109fb0 0x7fc9d4103c40 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.370+0000 7fc9d9496640 1 --2- 172.21.15.43:0/1106364215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d4109b10 0x7fc9d4103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.370+0000 7fc9d9496640 1 -- 172.21.15.43:0/1106364215 >> 172.21.15.43:0/1106364215 conn(0x7fc9d40ff360 msgr2=0x7fc9d4101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:13.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.370+0000 7fc9d9496640 1 -- 172.21.15.43:0/1106364215 shutdown_connections 2024-04-25T10:40:13.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.370+0000 7fc9d9496640 1 -- 172.21.15.43:0/1106364215 wait complete. 2024-04-25T10:40:13.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.371+0000 7fc9d9496640 1 Processor -- start 2024-04-25T10:40:13.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.371+0000 7fc9d9496640 1 -- start start 2024-04-25T10:40:13.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.371+0000 7fc9d9496640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9d41042b0 0x7fc9d419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.371+0000 7fc9d9496640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9d4109b10 0x7fc9d419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.371+0000 7fc9d9496640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d4109fb0 0x7fc9d419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.371+0000 7fc9d9496640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d4113d00 con 0x7fc9d41042b0 2024-04-25T10:40:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.371+0000 7fc9d9496640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d4113b80 con 0x7fc9d4109fb0 2024-04-25T10:40:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.371+0000 7fc9d9496640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d4113e80 con 0x7fc9d4109b10 2024-04-25T10:40:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.371+0000 7fc9d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9d41042b0 0x7fc9d419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.371+0000 7fc9d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9d41042b0 0x7fc9d419a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53878/0 (socket says 172.21.15.43:53878) 2024-04-25T10:40:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.371+0000 7fc9d2ffd640 1 -- 172.21.15.43:0/2876945438 learned_addr learned my addr 172.21.15.43:0/2876945438 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.372+0000 7fc9cbfff640 1 --2- 172.21.15.43:0/2876945438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9d4109b10 0x7fc9d419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.372+0000 7fc9d37fe640 1 --2- 172.21.15.43:0/2876945438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d4109fb0 0x7fc9d419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.372+0000 7fc9d2ffd640 1 -- 172.21.15.43:0/2876945438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9d4109b10 msgr2=0x7fc9d419ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.372+0000 7fc9d2ffd640 1 --2- 172.21.15.43:0/2876945438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9d4109b10 0x7fc9d419ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.372+0000 7fc9d2ffd640 1 -- 172.21.15.43:0/2876945438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d4109fb0 msgr2=0x7fc9d419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.372+0000 7fc9d2ffd640 1 --2- 172.21.15.43:0/2876945438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d4109fb0 0x7fc9d419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.372+0000 7fc9d2ffd640 1 -- 172.21.15.43:0/2876945438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d419f5a0 con 0x7fc9d41042b0 2024-04-25T10:40:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.372+0000 7fc9d2ffd640 1 --2- 172.21.15.43:0/2876945438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9d41042b0 0x7fc9d419a7c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400c9a0 tx=0x7fc9c400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.372+0000 7fc9d0ff9640 1 -- 172.21.15.43:0/2876945438 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4014070 con 0x7fc9d41042b0 2024-04-25T10:40:13.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.372+0000 7fc9d0ff9640 1 -- 172.21.15.43:0/2876945438 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9c4004440 con 0x7fc9d41042b0 2024-04-25T10:40:13.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.372+0000 7fc9d9496640 1 -- 172.21.15.43:0/2876945438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d419f890 con 0x7fc9d41042b0 2024-04-25T10:40:13.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.372+0000 7fc9d0ff9640 1 -- 172.21.15.43:0/2876945438 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c400f450 con 0x7fc9d41042b0 2024-04-25T10:40:13.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.372+0000 7fc9d9496640 1 -- 172.21.15.43:0/2876945438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d419fdb0 con 0x7fc9d41042b0 2024-04-25T10:40:13.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9e0427740 1 Processor -- start 2024-04-25T10:40:13.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9e0427740 1 -- start start 2024-04-25T10:40:13.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9d0ff9640 1 -- 172.21.15.43:0/2876945438 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fc9c4017070 con 0x7fc9d41042b0 2024-04-25T10:40:13.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9e0427740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f8c410 0x564dd832c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9e0427740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f63550 0x564dd832cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9e0427740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f6b730 0x564dd832d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9e0427740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564dd8292b00 con 0x564dd7f6b730 2024-04-25T10:40:13.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9e0427740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564dd822e920 con 0x564dd7f63550 2024-04-25T10:40:13.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9e0427740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564dd828eac0 con 0x564dd7f8c410 2024-04-25T10:40:13.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f6b730 0x564dd832d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f6b730 0x564dd832d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53892/0 (socket says 172.21.15.43:53892) 2024-04-25T10:40:13.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9cbfff640 1 -- 172.21.15.43:0/2492223427 learned_addr learned my addr 172.21.15.43:0/2492223427 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:13.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9d2ffd640 1 --2- 172.21.15.43:0/2492223427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f63550 0x564dd832cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9d37fe640 1 --2- 172.21.15.43:0/2492223427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f8c410 0x564dd832c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9cbfff640 1 -- 172.21.15.43:0/2492223427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f8c410 msgr2=0x564dd832c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:13.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9cbfff640 1 --2- 172.21.15.43:0/2492223427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f8c410 0x564dd832c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9cbfff640 1 -- 172.21.15.43:0/2492223427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f63550 msgr2=0x564dd832cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9cbfff640 1 --2- 172.21.15.43:0/2492223427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f63550 0x564dd832cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9cbfff640 1 -- 172.21.15.43:0/2492223427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dd806c7e0 con 0x564dd7f6b730 2024-04-25T10:40:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9d37fe640 1 --2- 172.21.15.43:0/2492223427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f8c410 0x564dd832c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:40:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9d0ff9640 1 --2- 172.21.15.43:0/2876945438 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9c00afc60 0x7fc9c00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9d37fe640 1 --2- 172.21.15.43:0/2876945438 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9c00afc60 0x7fc9c00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9cbfff640 1 --2- 172.21.15.43:0/2492223427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f6b730 0x564dd832d0a0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4069cf0 tx=0x7fc9bc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e30cf08740df50c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9d0ff9640 1 -- 172.21.15.43:0/2876945438 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fc9c40deb50 con 0x7fc9d41042b0 2024-04-25T10:40:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9c97fa640 1 -- 172.21.15.43:0/2492223427 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc040070 con 0x564dd7f6b730 2024-04-25T10:40:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9c97fa640 1 -- 172.21.15.43:0/2492223427 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9bc0316a0 con 0x564dd7f6b730 2024-04-25T10:40:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.376+0000 7fc9c97fa640 1 -- 172.21.15.43:0/2492223427 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc035040 con 0x564dd7f6b730 2024-04-25T10:40:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.377+0000 7fc9e0427740 1 -- 172.21.15.43:0/2492223427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f6b730 msgr2=0x564dd832d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.377+0000 7fc9d37fe640 1 --2- 172.21.15.43:0/2876945438 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9c00afc60 0x7fc9c00b2170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x564dd8329790 tx=0x7fc9cc0085f0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.377+0000 7fc9e0427740 1 --2- 172.21.15.43:0/2492223427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f6b730 0x564dd832d0a0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4069cf0 tx=0x7fc9bc000bc0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.377+0000 7fc9e0427740 1 -- 172.21.15.43:0/2492223427 shutdown_connections 2024-04-25T10:40:13.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.377+0000 7fc9e0427740 1 --2- 172.21.15.43:0/2492223427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f63550 0x564dd832cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.377+0000 7fc9e0427740 1 --2- 172.21.15.43:0/2492223427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f6b730 0x564dd832d0a0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.377+0000 7fc9e0427740 1 --2- 172.21.15.43:0/2492223427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f8c410 0x564dd832c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.377+0000 7fc9e0427740 1 -- 172.21.15.43:0/2492223427 >> 172.21.15.43:0/2492223427 conn(0x564dd7f80560 msgr2=0x564dd8328dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:13.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.378+0000 7fc9e0427740 1 -- 172.21.15.43:0/2492223427 shutdown_connections 2024-04-25T10:40:13.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.378+0000 7fc9e0427740 1 -- 172.21.15.43:0/2492223427 wait complete. 2024-04-25T10:40:13.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.378+0000 7fc9e0427740 1 -- start start 2024-04-25T10:40:13.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.378+0000 7fc9e0427740 1 Processor -- start 2024-04-25T10:40:13.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.378+0000 7fc9e0427740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f63550 0x564dd8336b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.378+0000 7fc9e0427740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f6b730 0x564dd8337060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.378+0000 7fc9e0427740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f8c410 0x564dd8342dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.378+0000 7fc9e0427740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564dd8339910 con 0x564dd7f6b730 2024-04-25T10:40:13.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.378+0000 7fc9e0427740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564dd822c5d0 con 0x564dd7f8c410 2024-04-25T10:40:13.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.378+0000 7fc9e0427740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564dd8339a90 con 0x564dd7f63550 2024-04-25T10:40:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.378+0000 7fc9e0427740 10 client.0 _ll_register_callbacks cb 0x564dd821a080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.378+0000 7fc9cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f6b730 0x564dd8337060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.378+0000 7fc9cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f6b730 0x564dd8337060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53902/0 (socket says 172.21.15.43:53902) 2024-04-25T10:40:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.379+0000 7fc9cbfff640 1 -- 172.21.15.43:0/2210048927 learned_addr learned my addr 172.21.15.43:0/2210048927 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.379+0000 7fc9d2ffd640 1 --2- 172.21.15.43:0/2210048927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f63550 0x564dd8336b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.379+0000 7fc9d37fe640 1 --2- 172.21.15.43:0/2210048927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f8c410 0x564dd8342dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.379+0000 7fc9cbfff640 1 -- 172.21.15.43:0/2210048927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f63550 msgr2=0x564dd8336b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.379+0000 7fc9cbfff640 1 --2- 172.21.15.43:0/2210048927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f63550 0x564dd8336b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.379+0000 7fc9cbfff640 1 -- 172.21.15.43:0/2210048927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f8c410 msgr2=0x564dd8342dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.379+0000 7fc9cbfff640 1 --2- 172.21.15.43:0/2210048927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f8c410 0x564dd8342dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.379+0000 7fc9cbfff640 1 -- 172.21.15.43:0/2210048927 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dd80ccf70 con 0x564dd7f6b730 2024-04-25T10:40:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.379+0000 7fc9d37fe640 1 --2- 172.21.15.43:0/2210048927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f8c410 0x564dd8342dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.379+0000 7fc9cbfff640 1 --2- 172.21.15.43:0/2210048927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f6b730 0x564dd8337060 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc009900 tx=0x7fc9bc0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1846d84b452982e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:13.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.379+0000 7fc9ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:13.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.379+0000 7fc9ba7fc640 1 -- 172.21.15.43:0/2210048927 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc04b030 con 0x564dd7f6b730 2024-04-25T10:40:13.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.379+0000 7fc9ba7fc640 1 -- 172.21.15.43:0/2210048927 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fc9bc0314a0 con 0x564dd7f6b730 2024-04-25T10:40:13.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.379+0000 7fc9e0427740 1 -- 172.21.15.43:0/2210048927 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9bc0316a0 con 0x564dd7f6b730 2024-04-25T10:40:13.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.379+0000 7fc9ba7fc640 1 -- 172.21.15.43:0/2210048927 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc9bc055050 con 0x564dd7f6b730 2024-04-25T10:40:13.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.379+0000 7fc9e0427740 10 client.17730 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:13.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.379+0000 7fc9e0427740 1 -- 172.21.15.43:0/2210048927 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564dd7f64170 con 0x564dd7f6b730 2024-04-25T10:40:13.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.380+0000 7fc9ba7fc640 1 -- 172.21.15.43:0/2210048927 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9bc0316a0 con 0x564dd7f6b730 2024-04-25T10:40:13.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.380+0000 7fc9ba7fc640 1 -- 172.21.15.43:0/2210048927 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc9bc0035f0 con 0x564dd7f6b730 2024-04-25T10:40:13.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.380+0000 7fc9e0427740 10 client.17730 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:13.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.380+0000 7fc9e0427740 10 client.17730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:13.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.380+0000 7fc9e0427740 20 client.17730 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:13.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.380+0000 7fc9e0427740 1 --2- 172.21.15.43:0/2210048927 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564dd7f4b1f0 0x564dd8348d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.380+0000 7fc9e0427740 4 client.17730 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:13.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.380+0000 7fc9e0427740 1 -- 172.21.15.43:0/2210048927 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564dd82a0710 con 0x564dd7f4b1f0 2024-04-25T10:40:13.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.380+0000 7fc9d2ffd640 1 --2- 172.21.15.43:0/2210048927 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564dd7f4b1f0 0x564dd8348d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.381+0000 7fc9d2ffd640 1 --2- 172.21.15.43:0/2210048927 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564dd7f4b1f0 0x564dd8348d00 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0c7153b596c0629 server_cookie=e3b8a81dd16b6cbf in_seq=0 out_seq=0 2024-04-25T10:40:13.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.381+0000 7fc9ba7fc640 10 client.17730 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:13.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.382+0000 7fc9ba7fc640 1 -- 172.21.15.43:0/2210048927 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564dd82a0710 con 0x564dd7f4b1f0 2024-04-25T10:40:13.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.382+0000 7fc9ba7fc640 10 client.17730 handle_command_reply: tid=0 2024-04-25T10:40:13.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.382+0000 7fc9e0427740 1 client.17730 shutdown 2024-04-25T10:40:13.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.382+0000 7fc9e0427740 1 -- 172.21.15.43:0/2210048927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f6b730 msgr2=0x564dd8337060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.382+0000 7fc9e0427740 1 --2- 172.21.15.43:0/2210048927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f6b730 0x564dd8337060 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc009900 tx=0x7fc9bc0058d0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.382+0000 7fc9e0427740 1 -- 172.21.15.43:0/2210048927 shutdown_connections 2024-04-25T10:40:13.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.382+0000 7fc9e0427740 1 --2- 172.21.15.43:0/2210048927 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564dd7f4b1f0 0x564dd8348d00 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.382+0000 7fc9e0427740 1 --2- 172.21.15.43:0/2210048927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f8c410 0x564dd8342dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.382+0000 7fc9e0427740 1 --2- 172.21.15.43:0/2210048927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f6b730 0x564dd8337060 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.382+0000 7fc9ba7fc640 0 client.17730 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:13.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.382+0000 7fc9e0427740 1 --2- 172.21.15.43:0/2210048927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f63550 0x564dd8336b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.382+0000 7fc9e0427740 1 -- 172.21.15.43:0/2210048927 >> 172.21.15.43:0/2210048927 conn(0x564dd7f80560 msgr2=0x564dd832e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:13.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.382+0000 7fc9e0427740 1 -- 172.21.15.43:0/2210048927 shutdown_connections 2024-04-25T10:40:13.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.382+0000 7fc9e0427740 1 -- 172.21.15.43:0/2210048927 wait complete. 2024-04-25T10:40:13.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.382+0000 7fc9e0427740 20 client.17730 trim_cache size 0 max 16384 2024-04-25T10:40:13.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.400+0000 7fc9e0427740 1 Processor -- start 2024-04-25T10:40:13.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.400+0000 7fc9e0427740 1 -- start start 2024-04-25T10:40:13.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.401+0000 7fc9e0427740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f63550 0x564dd8327bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.401+0000 7fc9e0427740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f6b730 0x564dd8342d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.401+0000 7fc9e0427740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f8c410 0x564dd8343270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.401+0000 7fc9e0427740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41b2250 con 0x564dd7f63550 2024-04-25T10:40:13.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.401+0000 7fc9e0427740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41afb20 con 0x564dd7f8c410 2024-04-25T10:40:13.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.401+0000 7fc9e0427740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41ad990 con 0x564dd7f6b730 2024-04-25T10:40:13.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.401+0000 7fc9d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f63550 0x564dd8327bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.401+0000 7fc9d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f63550 0x564dd8327bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53918/0 (socket says 172.21.15.43:53918) 2024-04-25T10:40:13.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.401+0000 7fc9d2ffd640 1 -- 172.21.15.43:0/613764803 learned_addr learned my addr 172.21.15.43:0/613764803 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:13.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.401+0000 7fc9d37fe640 1 --2- 172.21.15.43:0/613764803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f8c410 0x564dd8343270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.401+0000 7fc9d2ffd640 1 -- 172.21.15.43:0/613764803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f6b730 msgr2=0x564dd8342d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:13.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.401+0000 7fc9d2ffd640 1 --2- 172.21.15.43:0/613764803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f6b730 0x564dd8342d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.401+0000 7fc9d2ffd640 1 -- 172.21.15.43:0/613764803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f8c410 msgr2=0x564dd8343270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:13.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.401+0000 7fc9d2ffd640 1 --2- 172.21.15.43:0/613764803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f8c410 0x564dd8343270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.401+0000 7fc9d2ffd640 1 -- 172.21.15.43:0/613764803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dd8334e60 con 0x564dd7f63550 2024-04-25T10:40:13.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.401+0000 7fc9d37fe640 1 --2- 172.21.15.43:0/613764803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f8c410 0x564dd8343270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:13.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.401+0000 7fc9d2ffd640 1 --2- 172.21.15.43:0/613764803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f63550 0x564dd8327bd0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40a6990 tx=0x7fc9c40e1010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc6be814eefb621 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:13.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.402+0000 7fc9c8ff9640 1 -- 172.21.15.43:0/613764803 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c40a65e0 con 0x564dd7f63550 2024-04-25T10:40:13.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.402+0000 7fc9c8ff9640 1 -- 172.21.15.43:0/613764803 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9c40af440 con 0x564dd7f63550 2024-04-25T10:40:13.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.402+0000 7fc9c8ff9640 1 -- 172.21.15.43:0/613764803 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c40a6400 con 0x564dd7f63550 2024-04-25T10:40:13.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.402+0000 7fc9e0427740 1 -- 172.21.15.43:0/613764803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f63550 msgr2=0x564dd8327bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.402+0000 7fc9e0427740 1 --2- 172.21.15.43:0/613764803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f63550 0x564dd8327bd0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40a6990 tx=0x7fc9c40e1010 comp rx=0 tx=0).stop 2024-04-25T10:40:13.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.403+0000 7fc9e0427740 1 -- 172.21.15.43:0/613764803 shutdown_connections 2024-04-25T10:40:13.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.403+0000 7fc9e0427740 1 --2- 172.21.15.43:0/613764803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f8c410 0x564dd8343270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.403+0000 7fc9e0427740 1 --2- 172.21.15.43:0/613764803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f6b730 0x564dd8342d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.403+0000 7fc9e0427740 1 --2- 172.21.15.43:0/613764803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f63550 0x564dd8327bd0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.403+0000 7fc9e0427740 1 -- 172.21.15.43:0/613764803 >> 172.21.15.43:0/613764803 conn(0x564dd7f80560 msgr2=0x564dd83274c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:13.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.403+0000 7fc9e0427740 1 -- 172.21.15.43:0/613764803 shutdown_connections 2024-04-25T10:40:13.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.403+0000 7fc9e0427740 1 -- 172.21.15.43:0/613764803 wait complete. 2024-04-25T10:40:13.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.403+0000 7fc9e0427740 1 -- start start 2024-04-25T10:40:13.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.403+0000 7fc9e0427740 1 Processor -- start 2024-04-25T10:40:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.403+0000 7fc9e0427740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f63550 0x564dd834c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.403+0000 7fc9e0427740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f6b730 0x564dd83496a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.403+0000 7fc9e0427740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f8c410 0x564dd8349c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.403+0000 7fc9e0427740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41adfe0 con 0x564dd7f8c410 2024-04-25T10:40:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.403+0000 7fc9e0427740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41af450 con 0x564dd7f6b730 2024-04-25T10:40:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.403+0000 7fc9e0427740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41ae6b0 con 0x564dd7f63550 2024-04-25T10:40:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.403+0000 7fc9e0427740 10 client.0 _ll_register_callbacks cb 0x7fc9bc031d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.404+0000 7fc9d37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f8c410 0x564dd8349c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.404+0000 7fc9d37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f8c410 0x564dd8349c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53924/0 (socket says 172.21.15.43:53924) 2024-04-25T10:40:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.404+0000 7fc9d37fe640 1 -- 172.21.15.43:0/2457390429 learned_addr learned my addr 172.21.15.43:0/2457390429 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.404+0000 7fc9d2ffd640 1 --2- 172.21.15.43:0/2457390429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f63550 0x564dd834c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.404+0000 7fc9cbfff640 1 --2- 172.21.15.43:0/2457390429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f6b730 0x564dd83496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.404+0000 7fc9d37fe640 1 -- 172.21.15.43:0/2457390429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f63550 msgr2=0x564dd834c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.404+0000 7fc9d37fe640 1 --2- 172.21.15.43:0/2457390429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f63550 0x564dd834c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.404+0000 7fc9d37fe640 1 -- 172.21.15.43:0/2457390429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f6b730 msgr2=0x564dd83496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.404+0000 7fc9d37fe640 1 --2- 172.21.15.43:0/2457390429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f6b730 0x564dd83496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.404+0000 7fc9d37fe640 1 -- 172.21.15.43:0/2457390429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dd834a390 con 0x564dd7f8c410 2024-04-25T10:40:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.404+0000 7fc9cbfff640 1 --2- 172.21.15.43:0/2457390429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f6b730 0x564dd83496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.404+0000 7fc9d2ffd640 1 --2- 172.21.15.43:0/2457390429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f63550 0x564dd834c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:40:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.404+0000 7fc9d37fe640 1 --2- 172.21.15.43:0/2457390429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f8c410 0x564dd8349c00 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc008df0 tx=0x7fc9cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7b7b13cbaee9c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.404+0000 7fc9ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.404+0000 7fc9ba7fc640 1 -- 172.21.15.43:0/2457390429 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc01d810 con 0x564dd7f8c410 2024-04-25T10:40:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.404+0000 7fc9ba7fc640 1 -- 172.21.15.43:0/2457390429 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fc9cc01d9b0 con 0x564dd7f8c410 2024-04-25T10:40:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.404+0000 7fc9e0427740 1 -- 172.21.15.43:0/2457390429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9c40af440 con 0x564dd7f8c410 2024-04-25T10:40:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.405+0000 7fc9ba7fc640 1 -- 172.21.15.43:0/2457390429 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc9cc00fe90 con 0x564dd7f8c410 2024-04-25T10:40:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.405+0000 7fc9e0427740 10 client.17736 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.405+0000 7fc9e0427740 1 -- 172.21.15.43:0/2457390429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564dd83b0da0 con 0x564dd7f8c410 2024-04-25T10:40:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.405+0000 7fc9ba7fc640 1 -- 172.21.15.43:0/2457390429 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9c40af440 con 0x564dd7f8c410 2024-04-25T10:40:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.405+0000 7fc9ba7fc640 1 -- 172.21.15.43:0/2457390429 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc9cc027070 con 0x564dd7f8c410 2024-04-25T10:40:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.405+0000 7fc9e0427740 10 client.17736 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.405+0000 7fc9e0427740 10 client.17736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.405+0000 7fc9e0427740 20 client.17736 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.405+0000 7fc9e0427740 1 --2- 172.21.15.43:0/2457390429 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564dd7f20630 0x564dd83b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.405+0000 7fc9e0427740 4 client.17736 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:40:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.405+0000 7fc9e0427740 1 -- 172.21.15.43:0/2457390429 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564dd829a580 con 0x564dd7f20630 2024-04-25T10:40:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.405+0000 7fc9d2ffd640 1 --2- 172.21.15.43:0/2457390429 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564dd7f20630 0x564dd83b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.406+0000 7fc9ba7fc640 10 client.17736 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.406+0000 7fc9d2ffd640 1 --2- 172.21.15.43:0/2457390429 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564dd7f20630 0x564dd83b35e0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b26466d9f1aa0d31 server_cookie=55944d46a06ac842 in_seq=0 out_seq=0 2024-04-25T10:40:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.407+0000 7fc9ba7fc640 1 -- 172.21.15.43:0/2457390429 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564dd829a580 con 0x564dd7f20630 2024-04-25T10:40:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.407+0000 7fc9ba7fc640 10 client.17736 handle_command_reply: tid=0 2024-04-25T10:40:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.407+0000 7fc9e0427740 1 client.17736 shutdown 2024-04-25T10:40:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.407+0000 7fc9e0427740 1 -- 172.21.15.43:0/2457390429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f8c410 msgr2=0x564dd8349c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.407+0000 7fc9e0427740 1 --2- 172.21.15.43:0/2457390429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f8c410 0x564dd8349c00 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc008df0 tx=0x7fc9cc004a40 comp rx=0 tx=0).stop 2024-04-25T10:40:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.407+0000 7fc9e0427740 1 -- 172.21.15.43:0/2457390429 shutdown_connections 2024-04-25T10:40:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.407+0000 7fc9e0427740 1 --2- 172.21.15.43:0/2457390429 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564dd7f20630 0x564dd83b35e0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.407+0000 7fc9e0427740 1 --2- 172.21.15.43:0/2457390429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564dd7f6b730 0x564dd83496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.407+0000 7fc9e0427740 1 --2- 172.21.15.43:0/2457390429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dd7f8c410 0x564dd8349c00 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.407+0000 7fc9e0427740 1 --2- 172.21.15.43:0/2457390429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd7f63550 0x564dd834c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.407+0000 7fc9e0427740 1 -- 172.21.15.43:0/2457390429 >> 172.21.15.43:0/2457390429 conn(0x564dd7f80560 msgr2=0x564dd832c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:13.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.407+0000 7fc9ba7fc640 0 client.17736 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:13.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.407+0000 7fc9e0427740 1 -- 172.21.15.43:0/2457390429 shutdown_connections 2024-04-25T10:40:13.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.407+0000 7fc9e0427740 1 -- 172.21.15.43:0/2457390429 wait complete. 2024-04-25T10:40:13.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.407+0000 7fc9e0427740 20 client.17736 trim_cache size 0 max 16384 2024-04-25T10:40:13.415 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:40:13.415 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:40:13.415 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "afbd0383-3f41-41eb-b134-41aa11fb3426", 2024-04-25T10:40:13.415 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:40:13.415 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:40:13.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.408+0000 7fc9d9496640 1 -- 172.21.15.43:0/2876945438 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9c00afc60 msgr2=0x7fc9c00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.408+0000 7fc9d9496640 1 --2- 172.21.15.43:0/2876945438 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9c00afc60 0x7fc9c00b2170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x564dd8329790 tx=0x7fc9cc0085f0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.408+0000 7fc9d9496640 1 -- 172.21.15.43:0/2876945438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9d41042b0 msgr2=0x7fc9d419a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.408+0000 7fc9d9496640 1 --2- 172.21.15.43:0/2876945438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9d41042b0 0x7fc9d419a7c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400c9a0 tx=0x7fc9c400ce60 comp rx=0 tx=0).stop 2024-04-25T10:40:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.408+0000 7fc9d9496640 1 -- 172.21.15.43:0/2876945438 shutdown_connections 2024-04-25T10:40:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.408+0000 7fc9d9496640 1 --2- 172.21.15.43:0/2876945438 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9c00afc60 0x7fc9c00b2170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.408+0000 7fc9d9496640 1 --2- 172.21.15.43:0/2876945438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d4109fb0 0x7fc9d419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.408+0000 7fc9d9496640 1 --2- 172.21.15.43:0/2876945438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9d4109b10 0x7fc9d419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.408+0000 7fc9d9496640 1 --2- 172.21.15.43:0/2876945438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9d41042b0 0x7fc9d419a7c0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.408+0000 7fc9d9496640 1 -- 172.21.15.43:0/2876945438 >> 172.21.15.43:0/2876945438 conn(0x7fc9d40ff360 msgr2=0x7fc9d410f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.408+0000 7fc9d9496640 1 -- 172.21.15.43:0/2876945438 shutdown_connections 2024-04-25T10:40:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.408+0000 7fc9d9496640 1 -- 172.21.15.43:0/2876945438 wait complete. 2024-04-25T10:40:13.422 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:40:13.422 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:40:13.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.534+0000 7f1b39f26640 1 -- 172.21.15.43:0/3218905854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b3410e080 msgr2=0x7f1b34110570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.534+0000 7f1b39f26640 1 --2- 172.21.15.43:0/3218905854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b3410e080 0x7f1b34110570 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009ca0 tx=0x7f1b2c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.534+0000 7f1b39f26640 1 -- 172.21.15.43:0/3218905854 shutdown_connections 2024-04-25T10:40:13.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.534+0000 7f1b39f26640 1 --2- 172.21.15.43:0/3218905854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b3410b690 0x7f1b3410dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.534+0000 7f1b39f26640 1 --2- 172.21.15.43:0/3218905854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b3410e080 0x7f1b34110570 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.534+0000 7f1b39f26640 1 --2- 172.21.15.43:0/3218905854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b34104c30 0x7f1b34105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.534+0000 7f1b39f26640 1 -- 172.21.15.43:0/3218905854 >> 172.21.15.43:0/3218905854 conn(0x7f1b34078110 msgr2=0x7f1b34078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.534+0000 7f1b39f26640 1 -- 172.21.15.43:0/3218905854 shutdown_connections 2024-04-25T10:40:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.534+0000 7f1b39f26640 1 -- 172.21.15.43:0/3218905854 wait complete. 2024-04-25T10:40:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.534+0000 7f1b39f26640 1 Processor -- start 2024-04-25T10:40:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.535+0000 7f1b39f26640 1 -- start start 2024-04-25T10:40:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.535+0000 7f1b39f26640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b34104c30 0x7f1b3419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.535+0000 7f1b39f26640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b3410b690 0x7f1b3419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.535+0000 7f1b39f26640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b3410e080 0x7f1b3419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.535+0000 7f1b39f26640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1b34113ce0 con 0x7f1b34104c30 2024-04-25T10:40:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.535+0000 7f1b39f26640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1b34113b60 con 0x7f1b3410b690 2024-04-25T10:40:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.535+0000 7f1b39f26640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b34113e60 con 0x7f1b3410e080 2024-04-25T10:40:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.535+0000 7f1b337fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b34104c30 0x7f1b3419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.535+0000 7f1b337fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b34104c30 0x7f1b3419a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53944/0 (socket says 172.21.15.43:53944) 2024-04-25T10:40:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.535+0000 7f1b337fe640 1 -- 172.21.15.43:0/1321754803 learned_addr learned my addr 172.21.15.43:0/1321754803 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.535+0000 7f1b33fff640 1 --2- 172.21.15.43:0/1321754803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b3410e080 0x7f1b3419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.535+0000 7f1b32ffd640 1 --2- 172.21.15.43:0/1321754803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b3410b690 0x7f1b3419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.535+0000 7f1b337fe640 1 -- 172.21.15.43:0/1321754803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b3410e080 msgr2=0x7f1b3419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.535+0000 7f1b337fe640 1 --2- 172.21.15.43:0/1321754803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b3410e080 0x7f1b3419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.535+0000 7f1b337fe640 1 -- 172.21.15.43:0/1321754803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b3410b690 msgr2=0x7f1b3419ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.535+0000 7f1b337fe640 1 --2- 172.21.15.43:0/1321754803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b3410b690 0x7f1b3419ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.535+0000 7f1b337fe640 1 -- 172.21.15.43:0/1321754803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3419f520 con 0x7f1b34104c30 2024-04-25T10:40:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.535+0000 7f1b33fff640 1 --2- 172.21.15.43:0/1321754803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b3410e080 0x7f1b3419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.535+0000 7f1b337fe640 1 --2- 172.21.15.43:0/1321754803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b34104c30 0x7f1b3419a6c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400e9f0 tx=0x7f1b2400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.536+0000 7f1b30ff9640 1 -- 172.21.15.43:0/1321754803 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2400cd80 con 0x7f1b34104c30 2024-04-25T10:40:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.536+0000 7f1b30ff9640 1 -- 172.21.15.43:0/1321754803 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b240054d0 con 0x7f1b34104c30 2024-04-25T10:40:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.536+0000 7f1b39f26640 1 -- 172.21.15.43:0/1321754803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3419f810 con 0x7f1b34104c30 2024-04-25T10:40:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.536+0000 7f1b30ff9640 1 -- 172.21.15.43:0/1321754803 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24010690 con 0x7f1b34104c30 2024-04-25T10:40:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.536+0000 7f1b39f26640 1 -- 172.21.15.43:0/1321754803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3419fd10 con 0x7f1b34104c30 2024-04-25T10:40:13.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.538+0000 7f1b30ff9640 1 -- 172.21.15.43:0/1321754803 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f1b24004e90 con 0x7f1b34104c30 2024-04-25T10:40:13.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.539+0000 7f1b40dd8740 1 Processor -- start 2024-04-25T10:40:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.539+0000 7f1b40dd8740 1 -- start start 2024-04-25T10:40:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.539+0000 7f1b40dd8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde5362390 0x55bde5702a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.539+0000 7f1b40dd8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde53394d0 0x55bde5702fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.539+0000 7f1b40dd8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde53416b0 0x55bde570f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.539+0000 7f1b40dd8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bde5668d80 con 0x55bde5362390 2024-04-25T10:40:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.539+0000 7f1b40dd8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bde56228b0 con 0x55bde53416b0 2024-04-25T10:40:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.539+0000 7f1b40dd8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bde5664d40 con 0x55bde53394d0 2024-04-25T10:40:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.539+0000 7f1b33fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde5362390 0x55bde5702a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.539+0000 7f1b33fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde5362390 0x55bde5702a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53948/0 (socket says 172.21.15.43:53948) 2024-04-25T10:40:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.539+0000 7f1b33fff640 1 -- 172.21.15.43:0/506331523 learned_addr learned my addr 172.21.15.43:0/506331523 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.539+0000 7f1b30ff9640 1 --2- 172.21.15.43:0/1321754803 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1b1c0afc60 0x7f1b1c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.539+0000 7f1b32ffd640 1 --2- 172.21.15.43:0/506331523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde53416b0 0x55bde570f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.539+0000 7f1b337fe640 1 --2- 172.21.15.43:0/506331523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde53394d0 0x55bde5702fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.539+0000 7f1b33fff640 1 --2- 172.21.15.43:0/1321754803 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1b1c0afc60 0x7f1b1c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.539+0000 7f1b30ff9640 1 -- 172.21.15.43:0/1321754803 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f1b240df5a0 con 0x7f1b34104c30 2024-04-25T10:40:13.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.539+0000 7f1b33fff640 1 -- 172.21.15.43:0/506331523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde53394d0 msgr2=0x55bde5702fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:13.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.539+0000 7f1b33fff640 1 --2- 172.21.15.43:0/506331523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde53394d0 0x55bde5702fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.539+0000 7f1b33fff640 1 -- 172.21.15.43:0/506331523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde53416b0 msgr2=0x55bde570f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:13.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.540+0000 7f1b33fff640 1 --2- 172.21.15.43:0/506331523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde53416b0 0x55bde570f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.540+0000 7f1b33fff640 1 -- 172.21.15.43:0/506331523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bde55f3a40 con 0x55bde5362390 2024-04-25T10:40:13.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.540+0000 7f1b33fff640 1 --2- 172.21.15.43:0/506331523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde5362390 0x55bde5702a60 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c005e30 tx=0x7f1b2c031e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188e2e0ddaaad4e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:13.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.540+0000 7f1b19ffb640 1 -- 172.21.15.43:0/506331523 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c004d40 con 0x55bde5362390 2024-04-25T10:40:13.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.540+0000 7f1b19ffb640 1 -- 172.21.15.43:0/506331523 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b2c04d070 con 0x55bde5362390 2024-04-25T10:40:13.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.540+0000 7f1b19ffb640 1 -- 172.21.15.43:0/506331523 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c004740 con 0x55bde5362390 2024-04-25T10:40:13.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.541+0000 7f1b40dd8740 1 -- 172.21.15.43:0/506331523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde5362390 msgr2=0x55bde5702a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.541+0000 7f1b40dd8740 1 --2- 172.21.15.43:0/506331523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde5362390 0x55bde5702a60 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c005e30 tx=0x7f1b2c031e20 comp rx=0 tx=0).stop 2024-04-25T10:40:13.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.541+0000 7f1b33fff640 1 --2- 172.21.15.43:0/1321754803 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1b1c0afc60 0x7f1b1c0b2170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c04c000 tx=0x7f1b2c04a000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:13.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.541+0000 7f1b40dd8740 1 -- 172.21.15.43:0/506331523 shutdown_connections 2024-04-25T10:40:13.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.541+0000 7f1b40dd8740 1 --2- 172.21.15.43:0/506331523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde53416b0 0x55bde570f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.541+0000 7f1b40dd8740 1 --2- 172.21.15.43:0/506331523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde53394d0 0x55bde5702fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.541+0000 7f1b40dd8740 1 --2- 172.21.15.43:0/506331523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde5362390 0x55bde5702a60 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.541+0000 7f1b40dd8740 1 -- 172.21.15.43:0/506331523 >> 172.21.15.43:0/506331523 conn(0x55bde53564e0 msgr2=0x55bde56ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:13.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.541+0000 7f1b40dd8740 1 -- 172.21.15.43:0/506331523 shutdown_connections 2024-04-25T10:40:13.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.541+0000 7f1b40dd8740 1 -- 172.21.15.43:0/506331523 wait complete. 2024-04-25T10:40:13.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.541+0000 7f1b40dd8740 1 -- start start 2024-04-25T10:40:13.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b40dd8740 1 Processor -- start 2024-04-25T10:40:13.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b40dd8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde53394d0 0x55bde570d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b40dd8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde53416b0 0x55bde570db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b40dd8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde5362390 0x55bde5713430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b40dd8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bde570ffe0 con 0x55bde53394d0 2024-04-25T10:40:13.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b40dd8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bde55efd10 con 0x55bde53416b0 2024-04-25T10:40:13.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b40dd8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bde5710160 con 0x55bde5362390 2024-04-25T10:40:13.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b40dd8740 10 client.0 _ll_register_callbacks cb 0x55bde561c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:13.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b337fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde53394d0 0x55bde570d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b337fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde53394d0 0x55bde570d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53954/0 (socket says 172.21.15.43:53954) 2024-04-25T10:40:13.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b337fe640 1 -- 172.21.15.43:0/3904794340 learned_addr learned my addr 172.21.15.43:0/3904794340 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:13.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b32ffd640 1 --2- 172.21.15.43:0/3904794340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde53416b0 0x55bde570db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b33fff640 1 --2- 172.21.15.43:0/3904794340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde5362390 0x55bde5713430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b337fe640 1 -- 172.21.15.43:0/3904794340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde5362390 msgr2=0x55bde5713430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:13.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b337fe640 1 --2- 172.21.15.43:0/3904794340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde5362390 0x55bde5713430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b337fe640 1 -- 172.21.15.43:0/3904794340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde53416b0 msgr2=0x55bde570db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:13.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b337fe640 1 --2- 172.21.15.43:0/3904794340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde53416b0 0x55bde570db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b337fe640 1 -- 172.21.15.43:0/3904794340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bde54447d0 con 0x55bde53394d0 2024-04-25T10:40:13.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b33fff640 1 --2- 172.21.15.43:0/3904794340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde5362390 0x55bde5713430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:13.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b337fe640 1 --2- 172.21.15.43:0/3904794340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde53394d0 0x55bde570d5d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f1b240a7440 tx=0x7f1b2400a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0c9c27ad917351 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:13.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.542+0000 7f1b12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:13.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.543+0000 7f1b12ffd640 1 -- 172.21.15.43:0/3904794340 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24013190 con 0x55bde53394d0 2024-04-25T10:40:13.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.543+0000 7f1b12ffd640 1 -- 172.21.15.43:0/3904794340 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f1b240900b0 con 0x55bde53394d0 2024-04-25T10:40:13.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.543+0000 7f1b40dd8740 1 -- 172.21.15.43:0/3904794340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b2c04d070 con 0x55bde53394d0 2024-04-25T10:40:13.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.543+0000 7f1b12ffd640 1 -- 172.21.15.43:0/3904794340 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1b240a7250 con 0x55bde53394d0 2024-04-25T10:40:13.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.543+0000 7f1b40dd8740 10 client.17748 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:13.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.543+0000 7f1b40dd8740 1 -- 172.21.15.43:0/3904794340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bde537a3f0 con 0x55bde53394d0 2024-04-25T10:40:13.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.543+0000 7f1b12ffd640 1 -- 172.21.15.43:0/3904794340 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b24005680 con 0x55bde53394d0 2024-04-25T10:40:13.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.543+0000 7f1b12ffd640 1 -- 172.21.15.43:0/3904794340 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1b240a39d0 con 0x55bde53394d0 2024-04-25T10:40:13.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.543+0000 7f1b40dd8740 10 client.17748 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:13.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.543+0000 7f1b40dd8740 10 client.17748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:13.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.543+0000 7f1b40dd8740 20 client.17748 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.543+0000 7f1b40dd8740 1 --2- 172.21.15.43:0/3904794340 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55bde5320f60 0x55bde571f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.543+0000 7f1b40dd8740 4 client.17748 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.543+0000 7f1b40dd8740 1 -- 172.21.15.43:0/3904794340 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1b2c04d070 con 0x55bde5320f60 2024-04-25T10:40:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.544+0000 7f1b32ffd640 1 --2- 172.21.15.43:0/3904794340 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55bde5320f60 0x55bde571f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.544+0000 7f1b32ffd640 1 --2- 172.21.15.43:0/3904794340 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55bde5320f60 0x55bde571f600 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc3986b3d892e0e3 server_cookie=5e1fbfff746ea580 in_seq=0 out_seq=0 2024-04-25T10:40:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.544+0000 7f1b12ffd640 10 client.17748 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.545+0000 7f1b12ffd640 1 -- 172.21.15.43:0/3904794340 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f1b2c04d070 con 0x55bde5320f60 2024-04-25T10:40:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.545+0000 7f1b12ffd640 10 client.17748 handle_command_reply: tid=0 2024-04-25T10:40:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.545+0000 7f1b40dd8740 1 client.17748 shutdown 2024-04-25T10:40:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.545+0000 7f1b40dd8740 1 -- 172.21.15.43:0/3904794340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde53394d0 msgr2=0x55bde570d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.545+0000 7f1b40dd8740 1 --2- 172.21.15.43:0/3904794340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde53394d0 0x55bde570d5d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f1b240a7440 tx=0x7f1b2400a640 comp rx=0 tx=0).stop 2024-04-25T10:40:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.545+0000 7f1b40dd8740 1 -- 172.21.15.43:0/3904794340 shutdown_connections 2024-04-25T10:40:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.545+0000 7f1b40dd8740 1 --2- 172.21.15.43:0/3904794340 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55bde5320f60 0x55bde571f600 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.545+0000 7f1b40dd8740 1 --2- 172.21.15.43:0/3904794340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde53416b0 0x55bde570db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.545+0000 7f1b40dd8740 1 --2- 172.21.15.43:0/3904794340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde5362390 0x55bde5713430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.545+0000 7f1b12ffd640 0 client.17748 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.545+0000 7f1b40dd8740 1 --2- 172.21.15.43:0/3904794340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde53394d0 0x55bde570d5d0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.545+0000 7f1b40dd8740 1 -- 172.21.15.43:0/3904794340 >> 172.21.15.43:0/3904794340 conn(0x55bde53564e0 msgr2=0x55bde5704780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.545+0000 7f1b40dd8740 1 -- 172.21.15.43:0/3904794340 shutdown_connections 2024-04-25T10:40:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.545+0000 7f1b40dd8740 1 -- 172.21.15.43:0/3904794340 wait complete. 2024-04-25T10:40:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.545+0000 7f1b40dd8740 20 client.17748 trim_cache size 0 max 16384 2024-04-25T10:40:13.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.563+0000 7f1b40dd8740 1 Processor -- start 2024-04-25T10:40:13.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.563+0000 7f1b40dd8740 1 -- start start 2024-04-25T10:40:13.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.563+0000 7f1b40dd8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde53394d0 0x55bde570c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.563+0000 7f1b40dd8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde53416b0 0x55bde5713420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.563+0000 7f1b40dd8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde5362390 0x55bde5713960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.563+0000 7f1b40dd8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1b341b2200 con 0x55bde5362390 2024-04-25T10:40:13.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.563+0000 7f1b40dd8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1b341afad0 con 0x55bde53416b0 2024-04-25T10:40:13.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.563+0000 7f1b40dd8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b341ad940 con 0x55bde53394d0 2024-04-25T10:40:13.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.564+0000 7f1b33fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde5362390 0x55bde5713960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.564+0000 7f1b33fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde5362390 0x55bde5713960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53966/0 (socket says 172.21.15.43:53966) 2024-04-25T10:40:13.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.564+0000 7f1b33fff640 1 -- 172.21.15.43:0/149000104 learned_addr learned my addr 172.21.15.43:0/149000104 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:13.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.564+0000 7f1b32ffd640 1 --2- 172.21.15.43:0/149000104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde53416b0 0x55bde5713420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.564+0000 7f1b337fe640 1 --2- 172.21.15.43:0/149000104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde53394d0 0x55bde570c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.564+0000 7f1b33fff640 1 -- 172.21.15.43:0/149000104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde53394d0 msgr2=0x55bde570c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:13.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.564+0000 7f1b33fff640 1 --2- 172.21.15.43:0/149000104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde53394d0 0x55bde570c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.564+0000 7f1b33fff640 1 -- 172.21.15.43:0/149000104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde53416b0 msgr2=0x55bde5713420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:13.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.564+0000 7f1b33fff640 1 --2- 172.21.15.43:0/149000104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde53416b0 0x55bde5713420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.564+0000 7f1b33fff640 1 -- 172.21.15.43:0/149000104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bde533a0f0 con 0x55bde5362390 2024-04-25T10:40:13.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.564+0000 7f1b32ffd640 1 --2- 172.21.15.43:0/149000104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde53416b0 0x55bde5713420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:13.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.564+0000 7f1b33fff640 1 --2- 172.21.15.43:0/149000104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde5362390 0x55bde5713960 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c031c80 tx=0x7f1b2c031cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f542c1a60e81a03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:13.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.564+0000 7f1b197fa640 1 -- 172.21.15.43:0/149000104 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c060070 con 0x55bde5362390 2024-04-25T10:40:13.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.564+0000 7f1b197fa640 1 -- 172.21.15.43:0/149000104 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b2c04d040 con 0x55bde5362390 2024-04-25T10:40:13.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.564+0000 7f1b197fa640 1 -- 172.21.15.43:0/149000104 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c0042a0 con 0x55bde5362390 2024-04-25T10:40:13.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.565+0000 7f1b40dd8740 1 -- 172.21.15.43:0/149000104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde5362390 msgr2=0x55bde5713960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.565+0000 7f1b40dd8740 1 --2- 172.21.15.43:0/149000104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde5362390 0x55bde5713960 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c031c80 tx=0x7f1b2c031cb0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.565+0000 7f1b40dd8740 1 -- 172.21.15.43:0/149000104 shutdown_connections 2024-04-25T10:40:13.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.565+0000 7f1b40dd8740 1 --2- 172.21.15.43:0/149000104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde53416b0 0x55bde5713420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.565+0000 7f1b40dd8740 1 --2- 172.21.15.43:0/149000104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde5362390 0x55bde5713960 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.565+0000 7f1b40dd8740 1 --2- 172.21.15.43:0/149000104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde53394d0 0x55bde570c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.565+0000 7f1b40dd8740 1 -- 172.21.15.43:0/149000104 >> 172.21.15.43:0/149000104 conn(0x55bde53564e0 msgr2=0x55bde5704cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:13.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.565+0000 7f1b40dd8740 1 -- 172.21.15.43:0/149000104 shutdown_connections 2024-04-25T10:40:13.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.565+0000 7f1b40dd8740 1 -- 172.21.15.43:0/149000104 wait complete. 2024-04-25T10:40:13.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.565+0000 7f1b40dd8740 1 -- start start 2024-04-25T10:40:13.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.566+0000 7f1b40dd8740 1 Processor -- start 2024-04-25T10:40:13.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.566+0000 7f1b40dd8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde53394d0 0x55bde5714970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.566+0000 7f1b40dd8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde53416b0 0x55bde5714ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.566+0000 7f1b40dd8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde5362390 0x55bde5789d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.566+0000 7f1b40dd8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1b341adf90 con 0x55bde53394d0 2024-04-25T10:40:13.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.566+0000 7f1b40dd8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1b341af400 con 0x55bde5362390 2024-04-25T10:40:13.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.566+0000 7f1b40dd8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b341ae660 con 0x55bde53416b0 2024-04-25T10:40:13.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.566+0000 7f1b40dd8740 10 client.0 _ll_register_callbacks cb 0x7f1b2400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:13.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.566+0000 7f1b337fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde53394d0 0x55bde5714970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.566+0000 7f1b337fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde53394d0 0x55bde5714970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53972/0 (socket says 172.21.15.43:53972) 2024-04-25T10:40:13.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.566+0000 7f1b337fe640 1 -- 172.21.15.43:0/2268562062 learned_addr learned my addr 172.21.15.43:0/2268562062 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:13.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.566+0000 7f1b33fff640 1 --2- 172.21.15.43:0/2268562062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde5362390 0x55bde5789d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.566+0000 7f1b32ffd640 1 --2- 172.21.15.43:0/2268562062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde53416b0 0x55bde5714ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.566+0000 7f1b337fe640 1 -- 172.21.15.43:0/2268562062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde53416b0 msgr2=0x55bde5714ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:13.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.566+0000 7f1b337fe640 1 --2- 172.21.15.43:0/2268562062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde53416b0 0x55bde5714ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.566+0000 7f1b337fe640 1 -- 172.21.15.43:0/2268562062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde5362390 msgr2=0x55bde5789d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:13.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.566+0000 7f1b337fe640 1 --2- 172.21.15.43:0/2268562062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde5362390 0x55bde5789d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.566+0000 7f1b337fe640 1 -- 172.21.15.43:0/2268562062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bde56294b0 con 0x55bde53394d0 2024-04-25T10:40:13.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.566+0000 7f1b33fff640 1 --2- 172.21.15.43:0/2268562062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde5362390 0x55bde5789d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:13.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.566+0000 7f1b32ffd640 1 --2- 172.21.15.43:0/2268562062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde53416b0 0x55bde5714ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:13.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.567+0000 7f1b337fe640 1 --2- 172.21.15.43:0/2268562062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde53394d0 0x55bde5714970 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f1b240dd2f0 tx=0x7f1b240a7e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad8e807e171e6c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:13.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.567+0000 7f1b12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:13.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.567+0000 7f1b12ffd640 1 -- 172.21.15.43:0/2268562062 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24013190 con 0x55bde53394d0 2024-04-25T10:40:13.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.567+0000 7f1b12ffd640 1 -- 172.21.15.43:0/2268562062 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f1b240b01e0 con 0x55bde53394d0 2024-04-25T10:40:13.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.567+0000 7f1b40dd8740 1 -- 172.21.15.43:0/2268562062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b2c04d040 con 0x55bde53394d0 2024-04-25T10:40:13.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.567+0000 7f1b12ffd640 1 -- 172.21.15.43:0/2268562062 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1b240a3600 con 0x55bde53394d0 2024-04-25T10:40:13.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.567+0000 7f1b40dd8740 10 client.17754 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:13.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.567+0000 7f1b40dd8740 1 -- 172.21.15.43:0/2268562062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bde578b1d0 con 0x55bde53394d0 2024-04-25T10:40:13.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.567+0000 7f1b12ffd640 1 -- 172.21.15.43:0/2268562062 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b2c04d040 con 0x55bde53394d0 2024-04-25T10:40:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.567+0000 7f1b12ffd640 1 -- 172.21.15.43:0/2268562062 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1b240bcb90 con 0x55bde53394d0 2024-04-25T10:40:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.567+0000 7f1b40dd8740 10 client.17754 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.567+0000 7f1b40dd8740 10 client.17754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.567+0000 7f1b40dd8740 20 client.17754 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.567+0000 7f1b40dd8740 1 --2- 172.21.15.43:0/2268562062 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55bde52f63a0 0x55bde578d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.567+0000 7f1b40dd8740 4 client.17754 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:40:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.567+0000 7f1b40dd8740 1 -- 172.21.15.43:0/2268562062 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bde5676990 con 0x55bde52f63a0 2024-04-25T10:40:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.568+0000 7f1b32ffd640 1 --2- 172.21.15.43:0/2268562062 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55bde52f63a0 0x55bde578d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.568+0000 7f1b12ffd640 10 client.17754 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.568+0000 7f1b32ffd640 1 --2- 172.21.15.43:0/2268562062 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55bde52f63a0 0x55bde578d9b0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5077691edd509919 server_cookie=f61c23be8a709f26 in_seq=0 out_seq=0 2024-04-25T10:40:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.568+0000 7f1b12ffd640 1 -- 172.21.15.43:0/2268562062 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bde5676990 con 0x55bde52f63a0 2024-04-25T10:40:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.568+0000 7f1b12ffd640 10 client.17754 handle_command_reply: tid=0 2024-04-25T10:40:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.568+0000 7f1b40dd8740 1 client.17754 shutdown 2024-04-25T10:40:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.568+0000 7f1b40dd8740 1 -- 172.21.15.43:0/2268562062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde53394d0 msgr2=0x55bde5714970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.568+0000 7f1b40dd8740 1 --2- 172.21.15.43:0/2268562062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde53394d0 0x55bde5714970 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f1b240dd2f0 tx=0x7f1b240a7e40 comp rx=0 tx=0).stop 2024-04-25T10:40:13.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.568+0000 7f1b40dd8740 1 -- 172.21.15.43:0/2268562062 shutdown_connections 2024-04-25T10:40:13.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.568+0000 7f1b40dd8740 1 --2- 172.21.15.43:0/2268562062 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55bde52f63a0 0x55bde578d9b0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.568+0000 7f1b40dd8740 1 --2- 172.21.15.43:0/2268562062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bde5362390 0x55bde5789d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.568+0000 7f1b40dd8740 1 --2- 172.21.15.43:0/2268562062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bde53416b0 0x55bde5714ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.568+0000 7f1b12ffd640 0 client.17754 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:13.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.568+0000 7f1b40dd8740 1 --2- 172.21.15.43:0/2268562062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bde53394d0 0x55bde5714970 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.568+0000 7f1b40dd8740 1 -- 172.21.15.43:0/2268562062 >> 172.21.15.43:0/2268562062 conn(0x55bde53564e0 msgr2=0x55bde570ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:13.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.569+0000 7f1b40dd8740 1 -- 172.21.15.43:0/2268562062 shutdown_connections 2024-04-25T10:40:13.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.569+0000 7f1b40dd8740 1 -- 172.21.15.43:0/2268562062 wait complete. 2024-04-25T10:40:13.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.569+0000 7f1b40dd8740 20 client.17754 trim_cache size 0 max 16384 2024-04-25T10:40:13.581 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:40:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:40:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:40:13.581 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:40:13.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.569+0000 7f1b39f26640 1 -- 172.21.15.43:0/1321754803 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1b1c0afc60 msgr2=0x7f1b1c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.569+0000 7f1b39f26640 1 --2- 172.21.15.43:0/1321754803 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1b1c0afc60 0x7f1b1c0b2170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c04c000 tx=0x7f1b2c04a000 comp rx=0 tx=0).stop 2024-04-25T10:40:13.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.569+0000 7f1b39f26640 1 -- 172.21.15.43:0/1321754803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b34104c30 msgr2=0x7f1b3419a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.569+0000 7f1b39f26640 1 --2- 172.21.15.43:0/1321754803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b34104c30 0x7f1b3419a6c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400e9f0 tx=0x7f1b2400edf0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.569+0000 7f1b39f26640 1 -- 172.21.15.43:0/1321754803 shutdown_connections 2024-04-25T10:40:13.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.569+0000 7f1b39f26640 1 --2- 172.21.15.43:0/1321754803 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1b1c0afc60 0x7f1b1c0b2170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.569+0000 7f1b39f26640 1 --2- 172.21.15.43:0/1321754803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b3410b690 0x7f1b3419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.569+0000 7f1b39f26640 1 --2- 172.21.15.43:0/1321754803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b3410e080 0x7f1b3419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.569+0000 7f1b39f26640 1 --2- 172.21.15.43:0/1321754803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b34104c30 0x7f1b3419a6c0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.569+0000 7f1b39f26640 1 -- 172.21.15.43:0/1321754803 >> 172.21.15.43:0/1321754803 conn(0x7f1b34078110 msgr2=0x7f1b34101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:13.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.569+0000 7f1b39f26640 1 -- 172.21.15.43:0/1321754803 shutdown_connections 2024-04-25T10:40:13.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.569+0000 7f1b39f26640 1 -- 172.21.15.43:0/1321754803 wait complete. 2024-04-25T10:40:13.583 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:40:13.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:40:13.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.692+0000 7ff73291f640 1 -- 172.21.15.43:0/4130733082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72c102050 msgr2=0x7ff72c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.692+0000 7ff73291f640 1 --2- 172.21.15.43:0/4130733082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72c102050 0x7ff72c102460 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7ff720009900 tx=0x7ff72002a410 comp rx=0 tx=0).stop 2024-04-25T10:40:13.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.692+0000 7ff73291f640 1 -- 172.21.15.43:0/4130733082 shutdown_connections 2024-04-25T10:40:13.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.692+0000 7ff73291f640 1 --2- 172.21.15.43:0/4130733082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff72c10e550 0x7ff72c110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.692+0000 7ff73291f640 1 --2- 172.21.15.43:0/4130733082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72c102050 0x7ff72c102460 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.692+0000 7ff73291f640 1 --2- 172.21.15.43:0/4130733082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff72c109ba0 0x7ff72c101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.692+0000 7ff73291f640 1 -- 172.21.15.43:0/4130733082 >> 172.21.15.43:0/4130733082 conn(0x7ff72c0fd860 msgr2=0x7ff72c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:13.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.692+0000 7ff73291f640 1 -- 172.21.15.43:0/4130733082 shutdown_connections 2024-04-25T10:40:13.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.692+0000 7ff73291f640 1 -- 172.21.15.43:0/4130733082 wait complete. 2024-04-25T10:40:13.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.692+0000 7ff73291f640 1 Processor -- start 2024-04-25T10:40:13.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.692+0000 7ff73291f640 1 -- start start 2024-04-25T10:40:13.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.693+0000 7ff73291f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff72c102050 0x7ff72c102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.693+0000 7ff73291f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff72c109ba0 0x7ff72c103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.693+0000 7ff73291f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72c10e550 0x7ff72c103770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.693+0000 7ff73291f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c113d10 con 0x7ff72c10e550 2024-04-25T10:40:13.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.693+0000 7ff73291f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c113b90 con 0x7ff72c102050 2024-04-25T10:40:13.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.693+0000 7ff73291f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c113e90 con 0x7ff72c109ba0 2024-04-25T10:40:13.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.693+0000 7ff730ebc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72c10e550 0x7ff72c103770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.693+0000 7ff730ebc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72c10e550 0x7ff72c103770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53996/0 (socket says 172.21.15.43:53996) 2024-04-25T10:40:13.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.693+0000 7ff730ebc640 1 -- 172.21.15.43:0/2655594289 learned_addr learned my addr 172.21.15.43:0/2655594289 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:13.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.693+0000 7ff72bfff640 1 --2- 172.21.15.43:0/2655594289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff72c102050 0x7ff72c102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.693+0000 7ff72b7fe640 1 --2- 172.21.15.43:0/2655594289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff72c109ba0 0x7ff72c103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.693+0000 7ff730ebc640 1 -- 172.21.15.43:0/2655594289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff72c109ba0 msgr2=0x7ff72c103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.693+0000 7ff730ebc640 1 --2- 172.21.15.43:0/2655594289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff72c109ba0 0x7ff72c103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.693+0000 7ff730ebc640 1 -- 172.21.15.43:0/2655594289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff72c102050 msgr2=0x7ff72c102cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.693+0000 7ff730ebc640 1 --2- 172.21.15.43:0/2655594289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff72c102050 0x7ff72c102cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.693+0000 7ff730ebc640 1 -- 172.21.15.43:0/2655594289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72c1a8ab0 con 0x7ff72c10e550 2024-04-25T10:40:13.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.693+0000 7ff72b7fe640 1 --2- 172.21.15.43:0/2655594289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff72c109ba0 0x7ff72c103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:13.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.693+0000 7ff730ebc640 1 --2- 172.21.15.43:0/2655594289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72c10e550 0x7ff72c103770 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7ff72400cab0 tx=0x7ff72400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:13.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.694+0000 7ff7297fa640 1 -- 172.21.15.43:0/2655594289 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72400ae20 con 0x7ff72c10e550 2024-04-25T10:40:13.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.694+0000 7ff7297fa640 1 -- 172.21.15.43:0/2655594289 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff72400f900 con 0x7ff72c10e550 2024-04-25T10:40:13.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.694+0000 7ff73291f640 1 -- 172.21.15.43:0/2655594289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff72c1a8da0 con 0x7ff72c10e550 2024-04-25T10:40:13.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.694+0000 7ff7297fa640 1 -- 172.21.15.43:0/2655594289 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff724018a50 con 0x7ff72c10e550 2024-04-25T10:40:13.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.694+0000 7ff73291f640 1 -- 172.21.15.43:0/2655594289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff72c1a9210 con 0x7ff72c10e550 2024-04-25T10:40:13.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.695+0000 7ff73291f640 1 -- 172.21.15.43:0/2655594289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff72c06a190 con 0x7ff72c10e550 2024-04-25T10:40:13.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.697+0000 7ff7297fa640 1 -- 172.21.15.43:0/2655594289 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7ff72400f450 con 0x7ff72c10e550 2024-04-25T10:40:13.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.697+0000 7ff7297fa640 1 --2- 172.21.15.43:0/2655594289 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff7140afc60 0x7ff7140b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:13.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.697+0000 7ff72bfff640 1 --2- 172.21.15.43:0/2655594289 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff7140afc60 0x7ff7140b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:13.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.697+0000 7ff7297fa640 1 -- 172.21.15.43:0/2655594289 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7ff7240dfb10 con 0x7ff72c10e550 2024-04-25T10:40:13.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.697+0000 7ff72bfff640 1 --2- 172.21.15.43:0/2655594289 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff7140afc60 0x7ff7140b2170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff71c005db0 tx=0x7ff71c005d40 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:13.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.701+0000 7ff7297fa640 1 -- 172.21.15.43:0/2655594289 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff724014030 con 0x7ff72c10e550 2024-04-25T10:40:13.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.927+0000 7ff73291f640 1 -- 172.21.15.43:0/2655594289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff72c104220 con 0x7ff72c10e550 2024-04-25T10:40:13.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.928+0000 7ff7297fa640 1 -- 172.21.15.43:0/2655594289 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7ff7240a8440 con 0x7ff72c10e550 2024-04-25T10:40:13.930 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:40:13.930 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:40:13.931 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:40:13.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.930+0000 7ff73291f640 1 -- 172.21.15.43:0/2655594289 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff7140afc60 msgr2=0x7ff7140b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.930+0000 7ff73291f640 1 --2- 172.21.15.43:0/2655594289 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff7140afc60 0x7ff7140b2170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff71c005db0 tx=0x7ff71c005d40 comp rx=0 tx=0).stop 2024-04-25T10:40:13.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.930+0000 7ff73291f640 1 -- 172.21.15.43:0/2655594289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72c10e550 msgr2=0x7ff72c103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:13.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.930+0000 7ff73291f640 1 --2- 172.21.15.43:0/2655594289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72c10e550 0x7ff72c103770 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7ff72400cab0 tx=0x7ff72400cf70 comp rx=0 tx=0).stop 2024-04-25T10:40:13.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.930+0000 7ff73291f640 1 -- 172.21.15.43:0/2655594289 shutdown_connections 2024-04-25T10:40:13.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.930+0000 7ff73291f640 1 --2- 172.21.15.43:0/2655594289 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff7140afc60 0x7ff7140b2170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.930+0000 7ff73291f640 1 --2- 172.21.15.43:0/2655594289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72c10e550 0x7ff72c103770 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.930+0000 7ff73291f640 1 --2- 172.21.15.43:0/2655594289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff72c109ba0 0x7ff72c103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.930+0000 7ff73291f640 1 --2- 172.21.15.43:0/2655594289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff72c102050 0x7ff72c102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:13.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.930+0000 7ff73291f640 1 -- 172.21.15.43:0/2655594289 >> 172.21.15.43:0/2655594289 conn(0x7ff72c0fd860 msgr2=0x7ff72c0fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:13.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.930+0000 7ff73291f640 1 -- 172.21.15.43:0/2655594289 shutdown_connections 2024-04-25T10:40:13.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:13.930+0000 7ff73291f640 1 -- 172.21.15.43:0/2655594289 wait complete. 2024-04-25T10:40:13.945 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:40:14.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.058+0000 7f26ebce7640 1 -- 172.21.15.43:0/3815375507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e4108a10 msgr2=0x7f26e4102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.058+0000 7f26ebce7640 1 --2- 172.21.15.43:0/3815375507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e4108a10 0x7f26e4102610 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009900 tx=0x7f26e002a410 comp rx=0 tx=0).stop 2024-04-25T10:40:14.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.058+0000 7f26ebce7640 1 -- 172.21.15.43:0/3815375507 shutdown_connections 2024-04-25T10:40:14.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.058+0000 7f26ebce7640 1 --2- 172.21.15.43:0/3815375507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26e41031c0 0x7f26e410f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.058+0000 7f26ebce7640 1 --2- 172.21.15.43:0/3815375507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26e4108eb0 0x7f26e4102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.058+0000 7f26ebce7640 1 --2- 172.21.15.43:0/3815375507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e4108a10 0x7f26e4102610 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.058+0000 7f26ebce7640 1 -- 172.21.15.43:0/3815375507 >> 172.21.15.43:0/3815375507 conn(0x7f26e4078110 msgr2=0x7f26e4078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:14.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.059+0000 7f26ebce7640 1 -- 172.21.15.43:0/3815375507 shutdown_connections 2024-04-25T10:40:14.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.059+0000 7f26ebce7640 1 -- 172.21.15.43:0/3815375507 wait complete. 2024-04-25T10:40:14.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.059+0000 7f26ebce7640 1 Processor -- start 2024-04-25T10:40:14.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.059+0000 7f26ebce7640 1 -- start start 2024-04-25T10:40:14.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26ebce7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e41031c0 0x7f26e419b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26ebce7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26e4108a10 0x7f26e419bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26ebce7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26e4108eb0 0x7f26e419c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26ebce7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26e4112c10 con 0x7f26e41031c0 2024-04-25T10:40:14.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26ebce7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f26e4112a90 con 0x7f26e4108a10 2024-04-25T10:40:14.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26ebce7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f26e4112d90 con 0x7f26e4108eb0 2024-04-25T10:40:14.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26e9a83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e41031c0 0x7f26e419b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26e9a83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e41031c0 0x7f26e419b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54018/0 (socket says 172.21.15.43:54018) 2024-04-25T10:40:14.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26e9a83640 1 -- 172.21.15.43:0/2291509154 learned_addr learned my addr 172.21.15.43:0/2291509154 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:14.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26e9282640 1 --2- 172.21.15.43:0/2291509154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26e4108a10 0x7f26e419bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26ea284640 1 --2- 172.21.15.43:0/2291509154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26e4108eb0 0x7f26e419c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26e9a83640 1 -- 172.21.15.43:0/2291509154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26e4108eb0 msgr2=0x7f26e419c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26e9a83640 1 --2- 172.21.15.43:0/2291509154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26e4108eb0 0x7f26e419c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26e9a83640 1 -- 172.21.15.43:0/2291509154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26e4108a10 msgr2=0x7f26e419bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26e9a83640 1 --2- 172.21.15.43:0/2291509154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26e4108a10 0x7f26e419bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26e9a83640 1 -- 172.21.15.43:0/2291509154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e41a6850 con 0x7f26e41031c0 2024-04-25T10:40:14.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26ea284640 1 --2- 172.21.15.43:0/2291509154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26e4108eb0 0x7f26e419c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:14.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26e9a83640 1 --2- 172.21.15.43:0/2291509154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e41031c0 0x7f26e419b5d0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f26e0006010 tx=0x7f26e0004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:14.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26daffd640 1 -- 172.21.15.43:0/2291509154 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e003f030 con 0x7f26e41031c0 2024-04-25T10:40:14.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26ebce7640 1 -- 172.21.15.43:0/2291509154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e41a6ae0 con 0x7f26e41031c0 2024-04-25T10:40:14.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26daffd640 1 -- 172.21.15.43:0/2291509154 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26e0003aa0 con 0x7f26e41031c0 2024-04-25T10:40:14.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.060+0000 7f26daffd640 1 -- 172.21.15.43:0/2291509154 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0035070 con 0x7f26e41031c0 2024-04-25T10:40:14.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.061+0000 7f26ebce7640 1 -- 172.21.15.43:0/2291509154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e41a7020 con 0x7f26e41031c0 2024-04-25T10:40:14.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.063+0000 7f26daffd640 1 -- 172.21.15.43:0/2291509154 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f26e0042030 con 0x7f26e41031c0 2024-04-25T10:40:14.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.063+0000 7f26f2b97740 1 Processor -- start 2024-04-25T10:40:14.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.064+0000 7f26f2b97740 1 -- start start 2024-04-25T10:40:14.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.064+0000 7f26f2b97740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b864390 0x55e75bc056c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.064+0000 7f26f2b97740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b83b4d0 0x55e75bc05c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.064+0000 7f26f2b97740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b8436b0 0x55e75bc16ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.064+0000 7f26f2b97740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e75bb6ad80 con 0x55e75b8436b0 2024-04-25T10:40:14.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.064+0000 7f26f2b97740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e75bb24a20 con 0x55e75b864390 2024-04-25T10:40:14.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.064+0000 7f26f2b97740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e75bb66d40 con 0x55e75b83b4d0 2024-04-25T10:40:14.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.064+0000 7f26daffd640 1 --2- 172.21.15.43:0/2291509154 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26d00afc60 0x7f26d00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.064+0000 7f26e9282640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b8436b0 0x55e75bc16ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.064+0000 7f26e9282640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b8436b0 0x55e75bc16ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54026/0 (socket says 172.21.15.43:54026) 2024-04-25T10:40:14.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.064+0000 7f26e9282640 1 -- 172.21.15.43:0/315197151 learned_addr learned my addr 172.21.15.43:0/315197151 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:14.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.064+0000 7f26ea284640 1 --2- 172.21.15.43:0/2291509154 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26d00afc60 0x7f26d00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.064+0000 7f26ea284640 1 --2- 172.21.15.43:0/315197151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b864390 0x55e75bc056c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.064+0000 7f26daffd640 1 -- 172.21.15.43:0/2291509154 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f26e00ff570 con 0x7f26e41031c0 2024-04-25T10:40:14.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.064+0000 7f26e9a83640 1 --2- 172.21.15.43:0/315197151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b83b4d0 0x55e75bc05c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.064+0000 7f26e9282640 1 -- 172.21.15.43:0/315197151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b83b4d0 msgr2=0x55e75bc05c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.064+0000 7f26e9282640 1 --2- 172.21.15.43:0/315197151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b83b4d0 0x55e75bc05c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.064+0000 7f26e9282640 1 -- 172.21.15.43:0/315197151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b864390 msgr2=0x55e75bc056c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.064+0000 7f26e9282640 1 --2- 172.21.15.43:0/315197151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b864390 0x55e75bc056c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.064+0000 7f26e9282640 1 -- 172.21.15.43:0/315197151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e75bb2fc30 con 0x55e75b8436b0 2024-04-25T10:40:14.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.065+0000 7f26ea284640 1 --2- 172.21.15.43:0/2291509154 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26d00afc60 0x7f26d00b2170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f26dc005fd0 tx=0x7f26dc00a420 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:14.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.065+0000 7f26e9282640 1 --2- 172.21.15.43:0/315197151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b8436b0 0x55e75bc16ab0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009c00 tx=0x7f26d4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e8539a4bf19aee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:14.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.065+0000 7f26cffff640 1 -- 172.21.15.43:0/315197151 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d401e070 con 0x55e75b8436b0 2024-04-25T10:40:14.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.065+0000 7f26cffff640 1 -- 172.21.15.43:0/315197151 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26d4009db0 con 0x55e75b8436b0 2024-04-25T10:40:14.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.065+0000 7f26cffff640 1 -- 172.21.15.43:0/315197151 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4019450 con 0x55e75b8436b0 2024-04-25T10:40:14.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.066+0000 7f26f2b97740 1 -- 172.21.15.43:0/315197151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b8436b0 msgr2=0x55e75bc16ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.066+0000 7f26f2b97740 1 --2- 172.21.15.43:0/315197151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b8436b0 0x55e75bc16ab0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009c00 tx=0x7f26d4011810 comp rx=0 tx=0).stop 2024-04-25T10:40:14.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.066+0000 7f26f2b97740 1 -- 172.21.15.43:0/315197151 shutdown_connections 2024-04-25T10:40:14.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.066+0000 7f26f2b97740 1 --2- 172.21.15.43:0/315197151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b8436b0 0x55e75bc16ab0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.066+0000 7f26f2b97740 1 --2- 172.21.15.43:0/315197151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b83b4d0 0x55e75bc05c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.066+0000 7f26f2b97740 1 --2- 172.21.15.43:0/315197151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b864390 0x55e75bc056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.066+0000 7f26f2b97740 1 -- 172.21.15.43:0/315197151 >> 172.21.15.43:0/315197151 conn(0x55e75b8584e0 msgr2=0x55e75bc01fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:14.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.066+0000 7f26f2b97740 1 -- 172.21.15.43:0/315197151 shutdown_connections 2024-04-25T10:40:14.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.066+0000 7f26f2b97740 1 -- 172.21.15.43:0/315197151 wait complete. 2024-04-25T10:40:14.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.066+0000 7f26f2b97740 1 -- start start 2024-04-25T10:40:14.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.066+0000 7f26f2b97740 1 Processor -- start 2024-04-25T10:40:14.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.067+0000 7f26f2b97740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b83b4d0 0x55e75bc14630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.067+0000 7f26f2b97740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b8436b0 0x55e75bc14b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.067+0000 7f26f2b97740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b864390 0x55e75bc206e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.067+0000 7f26f2b97740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e75bc171a0 con 0x55e75b8436b0 2024-04-25T10:40:14.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.067+0000 7f26f2b97740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e75baf1dc0 con 0x55e75b83b4d0 2024-04-25T10:40:14.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.067+0000 7f26f2b97740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e75bc17320 con 0x55e75b864390 2024-04-25T10:40:14.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.067+0000 7f26f2b97740 10 client.0 _ll_register_callbacks cb 0x55e75bb0d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:14.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.067+0000 7f26e9282640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b8436b0 0x55e75bc14b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.067+0000 7f26e9282640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b8436b0 0x55e75bc14b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54028/0 (socket says 172.21.15.43:54028) 2024-04-25T10:40:14.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.067+0000 7f26e9282640 1 -- 172.21.15.43:0/2797483167 learned_addr learned my addr 172.21.15.43:0/2797483167 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:14.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.067+0000 7f26e9a83640 1 --2- 172.21.15.43:0/2797483167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b83b4d0 0x55e75bc14630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.067+0000 7f26ea284640 1 --2- 172.21.15.43:0/2797483167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b864390 0x55e75bc206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.067+0000 7f26e9282640 1 -- 172.21.15.43:0/2797483167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b864390 msgr2=0x55e75bc206e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.067+0000 7f26e9282640 1 --2- 172.21.15.43:0/2797483167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b864390 0x55e75bc206e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.067+0000 7f26e9282640 1 -- 172.21.15.43:0/2797483167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b83b4d0 msgr2=0x55e75bc14630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.067+0000 7f26e9282640 1 --2- 172.21.15.43:0/2797483167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b83b4d0 0x55e75bc14630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.067+0000 7f26e9282640 1 -- 172.21.15.43:0/2797483167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e75b87c3f0 con 0x55e75b8436b0 2024-04-25T10:40:14.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.067+0000 7f26e9a83640 1 --2- 172.21.15.43:0/2797483167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b83b4d0 0x55e75bc14630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:40:14.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.068+0000 7f26e9282640 1 --2- 172.21.15.43:0/2797483167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b8436b0 0x55e75bc14b70 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f26d401c040 tx=0x7f26d4005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1f5a71365b087e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:14.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.068+0000 7f26cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:14.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.068+0000 7f26cd7fa640 1 -- 172.21.15.43:0/2797483167 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d401e040 con 0x55e75b8436b0 2024-04-25T10:40:14.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.068+0000 7f26cd7fa640 1 -- 172.21.15.43:0/2797483167 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f26d40035f0 con 0x55e75b8436b0 2024-04-25T10:40:14.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.068+0000 7f26f2b97740 1 -- 172.21.15.43:0/2797483167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26d4009db0 con 0x55e75b8436b0 2024-04-25T10:40:14.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.068+0000 7f26cd7fa640 1 -- 172.21.15.43:0/2797483167 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f26d4032050 con 0x55e75b8436b0 2024-04-25T10:40:14.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.068+0000 7f26f2b97740 10 client.17772 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:14.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.068+0000 7f26f2b97740 1 -- 172.21.15.43:0/2797483167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e75b9467d0 con 0x55e75b8436b0 2024-04-25T10:40:14.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.068+0000 7f26cd7fa640 1 -- 172.21.15.43:0/2797483167 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26d4009db0 con 0x55e75b8436b0 2024-04-25T10:40:14.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.068+0000 7f26cd7fa640 1 -- 172.21.15.43:0/2797483167 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f26d4021030 con 0x55e75b8436b0 2024-04-25T10:40:14.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.068+0000 7f26f2b97740 10 client.17772 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:14.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.068+0000 7f26f2b97740 10 client.17772 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:40:14.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.068+0000 7f26f2b97740 20 client.17772 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:14.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.069+0000 7f26f2b97740 1 --2- 172.21.15.43:0/2797483167 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55e75b822f60 0x55e75bc26980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.069+0000 7f26f2b97740 4 client.17772 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.069+0000 7f26f2b97740 1 -- 172.21.15.43:0/2797483167 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e75bb78990 con 0x55e75b822f60 2024-04-25T10:40:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.069+0000 7f26e9a83640 1 --2- 172.21.15.43:0/2797483167 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55e75b822f60 0x55e75bc26980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.069+0000 7f26cd7fa640 10 client.17772 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:40:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.069+0000 7f26e9a83640 1 --2- 172.21.15.43:0/2797483167 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55e75b822f60 0x55e75bc26980 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7452d5c183558dff server_cookie=5745ef2b039b838c in_seq=0 out_seq=0 2024-04-25T10:40:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.070+0000 7f26cd7fa640 1 -- 172.21.15.43:0/2797483167 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e75bb78990 con 0x55e75b822f60 2024-04-25T10:40:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.070+0000 7f26cd7fa640 10 client.17772 handle_command_reply: tid=0 2024-04-25T10:40:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.070+0000 7f26f2b97740 1 client.17772 shutdown 2024-04-25T10:40:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.070+0000 7f26f2b97740 1 -- 172.21.15.43:0/2797483167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b8436b0 msgr2=0x55e75bc14b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.070+0000 7f26f2b97740 1 --2- 172.21.15.43:0/2797483167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b8436b0 0x55e75bc14b70 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f26d401c040 tx=0x7f26d4005720 comp rx=0 tx=0).stop 2024-04-25T10:40:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.070+0000 7f26f2b97740 1 -- 172.21.15.43:0/2797483167 shutdown_connections 2024-04-25T10:40:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.070+0000 7f26f2b97740 1 --2- 172.21.15.43:0/2797483167 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55e75b822f60 0x55e75bc26980 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.070+0000 7f26f2b97740 1 --2- 172.21.15.43:0/2797483167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b864390 0x55e75bc206e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.070+0000 7f26f2b97740 1 --2- 172.21.15.43:0/2797483167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b8436b0 0x55e75bc14b70 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.070+0000 7f26cd7fa640 0 client.17772 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:40:14.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.070+0000 7f26f2b97740 1 --2- 172.21.15.43:0/2797483167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b83b4d0 0x55e75bc14630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.070+0000 7f26f2b97740 1 -- 172.21.15.43:0/2797483167 >> 172.21.15.43:0/2797483167 conn(0x55e75b8584e0 msgr2=0x55e75bc074a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:14.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.070+0000 7f26f2b97740 1 -- 172.21.15.43:0/2797483167 shutdown_connections 2024-04-25T10:40:14.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.070+0000 7f26f2b97740 1 -- 172.21.15.43:0/2797483167 wait complete. 2024-04-25T10:40:14.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.070+0000 7f26f2b97740 20 client.17772 trim_cache size 0 max 16384 2024-04-25T10:40:14.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.084+0000 7f26f2b97740 1 Processor -- start 2024-04-25T10:40:14.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.084+0000 7f26f2b97740 1 -- start start 2024-04-25T10:40:14.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.084+0000 7f26f2b97740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b822f60 0x55e75bc07ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.084+0000 7f26f2b97740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b83b4d0 0x55e75bc202c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.084+0000 7f26f2b97740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b8436b0 0x55e75bc13bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.084+0000 7f26f2b97740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26e41affe0 con 0x55e75b83b4d0 2024-04-25T10:40:14.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.084+0000 7f26f2b97740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f26e41ad8b0 con 0x55e75b822f60 2024-04-25T10:40:14.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.084+0000 7f26f2b97740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f26e41ab720 con 0x55e75b8436b0 2024-04-25T10:40:14.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.085+0000 7f26e9282640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b83b4d0 0x55e75bc202c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.085+0000 7f26e9282640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b83b4d0 0x55e75bc202c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54034/0 (socket says 172.21.15.43:54034) 2024-04-25T10:40:14.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.085+0000 7f26e9282640 1 -- 172.21.15.43:0/1284468981 learned_addr learned my addr 172.21.15.43:0/1284468981 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:14.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.085+0000 7f26e9a83640 1 --2- 172.21.15.43:0/1284468981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b822f60 0x55e75bc07ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.085+0000 7f26e9282640 1 -- 172.21.15.43:0/1284468981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b8436b0 msgr2=0x55e75bc13bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.085+0000 7f26ea284640 1 --2- 172.21.15.43:0/1284468981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b8436b0 0x55e75bc13bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.085+0000 7f26e9282640 1 --2- 172.21.15.43:0/1284468981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b8436b0 0x55e75bc13bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.085+0000 7f26e9282640 1 -- 172.21.15.43:0/1284468981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b822f60 msgr2=0x55e75bc07ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.085+0000 7f26e9282640 1 --2- 172.21.15.43:0/1284468981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b822f60 0x55e75bc07ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.085+0000 7f26e9282640 1 -- 172.21.15.43:0/1284468981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e75bc12970 con 0x55e75b83b4d0 2024-04-25T10:40:14.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.085+0000 7f26ea284640 1 --2- 172.21.15.43:0/1284468981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b8436b0 0x55e75bc13bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:14.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.085+0000 7f26e9282640 1 --2- 172.21.15.43:0/1284468981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b83b4d0 0x55e75bc202c0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f26d4003e10 tx=0x7f26d4005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c8dbb7207ecfe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:14.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.085+0000 7f26cf7fe640 1 -- 172.21.15.43:0/1284468981 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d401e040 con 0x55e75b83b4d0 2024-04-25T10:40:14.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.085+0000 7f26cf7fe640 1 -- 172.21.15.43:0/1284468981 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26d4029070 con 0x55e75b83b4d0 2024-04-25T10:40:14.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.085+0000 7f26cf7fe640 1 -- 172.21.15.43:0/1284468981 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4004b10 con 0x55e75b83b4d0 2024-04-25T10:40:14.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.086+0000 7f26f2b97740 1 -- 172.21.15.43:0/1284468981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b83b4d0 msgr2=0x55e75bc202c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.086+0000 7f26f2b97740 1 --2- 172.21.15.43:0/1284468981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b83b4d0 0x55e75bc202c0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f26d4003e10 tx=0x7f26d4005d30 comp rx=0 tx=0).stop 2024-04-25T10:40:14.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.086+0000 7f26f2b97740 1 -- 172.21.15.43:0/1284468981 shutdown_connections 2024-04-25T10:40:14.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.086+0000 7f26f2b97740 1 --2- 172.21.15.43:0/1284468981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b8436b0 0x55e75bc13bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.086+0000 7f26f2b97740 1 --2- 172.21.15.43:0/1284468981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b83b4d0 0x55e75bc202c0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.086+0000 7f26f2b97740 1 --2- 172.21.15.43:0/1284468981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b822f60 0x55e75bc07ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.086+0000 7f26f2b97740 1 -- 172.21.15.43:0/1284468981 >> 172.21.15.43:0/1284468981 conn(0x55e75b8584e0 msgr2=0x55e75bc01e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:14.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.086+0000 7f26f2b97740 1 -- 172.21.15.43:0/1284468981 shutdown_connections 2024-04-25T10:40:14.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.086+0000 7f26f2b97740 1 -- 172.21.15.43:0/1284468981 wait complete. 2024-04-25T10:40:14.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.086+0000 7f26f2b97740 1 -- start start 2024-04-25T10:40:14.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.086+0000 7f26f2b97740 1 Processor -- start 2024-04-25T10:40:14.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.087+0000 7f26f2b97740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b822f60 0x55e75bc87010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.087+0000 7f26f2b97740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b83b4d0 0x55e75bc87550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.087+0000 7f26f2b97740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b8436b0 0x55e75bc17c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.087+0000 7f26f2b97740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26e41abd70 con 0x55e75b822f60 2024-04-25T10:40:14.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.087+0000 7f26f2b97740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f26e41ad1e0 con 0x55e75b83b4d0 2024-04-25T10:40:14.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.087+0000 7f26f2b97740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f26e41ac440 con 0x55e75b8436b0 2024-04-25T10:40:14.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.087+0000 7f26f2b97740 10 client.0 _ll_register_callbacks cb 0x7f26d401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:14.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.087+0000 7f26e9a83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b822f60 0x55e75bc87010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.087+0000 7f26e9a83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b822f60 0x55e75bc87010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54038/0 (socket says 172.21.15.43:54038) 2024-04-25T10:40:14.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.087+0000 7f26e9a83640 1 -- 172.21.15.43:0/1478659457 learned_addr learned my addr 172.21.15.43:0/1478659457 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:14.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.087+0000 7f26e9282640 1 --2- 172.21.15.43:0/1478659457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b83b4d0 0x55e75bc87550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.087+0000 7f26e9a83640 1 -- 172.21.15.43:0/1478659457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b8436b0 msgr2=0x55e75bc17c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.087+0000 7f26ea284640 1 --2- 172.21.15.43:0/1478659457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b8436b0 0x55e75bc17c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.087+0000 7f26e9a83640 1 --2- 172.21.15.43:0/1478659457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b8436b0 0x55e75bc17c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.087+0000 7f26e9a83640 1 -- 172.21.15.43:0/1478659457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b83b4d0 msgr2=0x55e75bc87550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.087+0000 7f26e9a83640 1 --2- 172.21.15.43:0/1478659457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b83b4d0 0x55e75bc87550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.087+0000 7f26e9a83640 1 -- 172.21.15.43:0/1478659457 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e75bc181f0 con 0x55e75b822f60 2024-04-25T10:40:14.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.088+0000 7f26e9a83640 1 --2- 172.21.15.43:0/1478659457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b822f60 0x55e75bc87010 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f26e00c7640 tx=0x7f26e0003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6fd9619a9208101 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:14.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.088+0000 7f26cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:14.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.088+0000 7f26cd7fa640 1 -- 172.21.15.43:0/1478659457 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0049040 con 0x55e75b822f60 2024-04-25T10:40:14.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.088+0000 7f26cd7fa640 1 -- 172.21.15.43:0/1478659457 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f26e00c3940 con 0x55e75b822f60 2024-04-25T10:40:14.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.088+0000 7f26f2b97740 1 -- 172.21.15.43:0/1478659457 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26d4029070 con 0x55e75b822f60 2024-04-25T10:40:14.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.088+0000 7f26cd7fa640 1 -- 172.21.15.43:0/1478659457 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f26e00c76b0 con 0x55e75b822f60 2024-04-25T10:40:14.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.088+0000 7f26f2b97740 10 client.17778 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:14.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.088+0000 7f26f2b97740 1 -- 172.21.15.43:0/1478659457 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e75bc8e600 con 0x55e75b822f60 2024-04-25T10:40:14.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.088+0000 7f26cd7fa640 1 -- 172.21.15.43:0/1478659457 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26d4029070 con 0x55e75b822f60 2024-04-25T10:40:14.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.088+0000 7f26cd7fa640 1 -- 172.21.15.43:0/1478659457 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f26e00c7910 con 0x55e75b822f60 2024-04-25T10:40:14.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.088+0000 7f26f2b97740 10 client.17778 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:14.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.088+0000 7f26f2b97740 10 client.17778 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:40:14.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.088+0000 7f26f2b97740 20 client.17778 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:14.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.088+0000 7f26f2b97740 1 --2- 172.21.15.43:0/1478659457 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55e75b864390 0x55e75bc90ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.088+0000 7f26f2b97740 4 client.17778 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:40:14.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.088+0000 7f26f2b97740 1 -- 172.21.15.43:0/1478659457 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e75bb72800 con 0x55e75b864390 2024-04-25T10:40:14.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.089+0000 7f26e9282640 1 --2- 172.21.15.43:0/1478659457 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55e75b864390 0x55e75bc90ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.089+0000 7f26cd7fa640 10 client.17778 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:40:14.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.089+0000 7f26e9282640 1 --2- 172.21.15.43:0/1478659457 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55e75b864390 0x55e75bc90ce0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2c326436900d18 server_cookie=1216a85c52b45472 in_seq=0 out_seq=0 2024-04-25T10:40:14.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.089+0000 7f26cd7fa640 1 -- 172.21.15.43:0/1478659457 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e75bb72800 con 0x55e75b864390 2024-04-25T10:40:14.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.089+0000 7f26cd7fa640 10 client.17778 handle_command_reply: tid=0 2024-04-25T10:40:14.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.089+0000 7f26f2b97740 1 client.17778 shutdown 2024-04-25T10:40:14.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.089+0000 7f26f2b97740 1 -- 172.21.15.43:0/1478659457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b822f60 msgr2=0x55e75bc87010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.089+0000 7f26f2b97740 1 --2- 172.21.15.43:0/1478659457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b822f60 0x55e75bc87010 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f26e00c7640 tx=0x7f26e0003f50 comp rx=0 tx=0).stop 2024-04-25T10:40:14.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.089+0000 7f26f2b97740 1 -- 172.21.15.43:0/1478659457 shutdown_connections 2024-04-25T10:40:14.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.089+0000 7f26f2b97740 1 --2- 172.21.15.43:0/1478659457 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55e75b864390 0x55e75bc90ce0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.089+0000 7f26f2b97740 1 --2- 172.21.15.43:0/1478659457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e75b83b4d0 0x55e75bc87550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.089+0000 7f26f2b97740 1 --2- 172.21.15.43:0/1478659457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75b8436b0 0x55e75bc17c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.089+0000 7f26f2b97740 1 --2- 172.21.15.43:0/1478659457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e75b822f60 0x55e75bc87010 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.089+0000 7f26cd7fa640 0 client.17778 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:40:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.089+0000 7f26f2b97740 1 -- 172.21.15.43:0/1478659457 >> 172.21.15.43:0/1478659457 conn(0x55e75b8584e0 msgr2=0x55e75bc00610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.089+0000 7f26f2b97740 1 -- 172.21.15.43:0/1478659457 shutdown_connections 2024-04-25T10:40:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.089+0000 7f26f2b97740 1 -- 172.21.15.43:0/1478659457 wait complete. 2024-04-25T10:40:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.090+0000 7f26f2b97740 20 client.17778 trim_cache size 0 max 16384 2024-04-25T10:40:14.098 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:40:14.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.090+0000 7f26ebce7640 1 -- 172.21.15.43:0/2291509154 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26d00afc60 msgr2=0x7f26d00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.090+0000 7f26ebce7640 1 --2- 172.21.15.43:0/2291509154 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26d00afc60 0x7f26d00b2170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f26dc005fd0 tx=0x7f26dc00a420 comp rx=0 tx=0).stop 2024-04-25T10:40:14.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.090+0000 7f26ebce7640 1 -- 172.21.15.43:0/2291509154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e41031c0 msgr2=0x7f26e419b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.090+0000 7f26ebce7640 1 --2- 172.21.15.43:0/2291509154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e41031c0 0x7f26e419b5d0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f26e0006010 tx=0x7f26e0004060 comp rx=0 tx=0).stop 2024-04-25T10:40:14.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.090+0000 7f26ebce7640 1 -- 172.21.15.43:0/2291509154 shutdown_connections 2024-04-25T10:40:14.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.090+0000 7f26ebce7640 1 --2- 172.21.15.43:0/2291509154 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26d00afc60 0x7f26d00b2170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.090+0000 7f26ebce7640 1 --2- 172.21.15.43:0/2291509154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26e4108a10 0x7f26e419bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.090+0000 7f26ebce7640 1 --2- 172.21.15.43:0/2291509154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26e4108eb0 0x7f26e419c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.090+0000 7f26ebce7640 1 --2- 172.21.15.43:0/2291509154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e41031c0 0x7f26e419b5d0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.090+0000 7f26ebce7640 1 -- 172.21.15.43:0/2291509154 >> 172.21.15.43:0/2291509154 conn(0x7f26e4078110 msgr2=0x7f26e41040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:14.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.090+0000 7f26ebce7640 1 -- 172.21.15.43:0/2291509154 shutdown_connections 2024-04-25T10:40:14.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.090+0000 7f26ebce7640 1 -- 172.21.15.43:0/2291509154 wait complete. 2024-04-25T10:40:14.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:40:14.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.215+0000 7f78d22d8640 1 -- 172.21.15.43:0/2714610152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78cc1032c0 msgr2=0x7f78cc1036d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.215+0000 7f78d22d8640 1 --2- 172.21.15.43:0/2714610152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78cc1032c0 0x7f78cc1036d0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009900 tx=0x7f78bc02a410 comp rx=0 tx=0).stop 2024-04-25T10:40:14.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.215+0000 7f78d22d8640 1 -- 172.21.15.43:0/2714610152 shutdown_connections 2024-04-25T10:40:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.215+0000 7f78d22d8640 1 --2- 172.21.15.43:0/2714610152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78cc10d440 0x7f78cc10f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.215+0000 7f78d22d8640 1 --2- 172.21.15.43:0/2714610152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78cc1032c0 0x7f78cc1036d0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.215+0000 7f78d22d8640 1 --2- 172.21.15.43:0/2714610152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78cc107950 0x7f78cc102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.215+0000 7f78d22d8640 1 -- 172.21.15.43:0/2714610152 >> 172.21.15.43:0/2714610152 conn(0x7f78cc077920 msgr2=0x7f78cc077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.215+0000 7f78d22d8640 1 -- 172.21.15.43:0/2714610152 shutdown_connections 2024-04-25T10:40:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.215+0000 7f78d22d8640 1 -- 172.21.15.43:0/2714610152 wait complete. 2024-04-25T10:40:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.215+0000 7f78d22d8640 1 Processor -- start 2024-04-25T10:40:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.215+0000 7f78d22d8640 1 -- start start 2024-04-25T10:40:14.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.216+0000 7f78d22d8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78cc1032c0 0x7f78cc0fe9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.216+0000 7f78d22d8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78cc107950 0x7f78cc0feee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.216+0000 7f78d22d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78cc10d440 0x7f78cc0ff420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.216+0000 7f78d22d8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc112c00 con 0x7f78cc10d440 2024-04-25T10:40:14.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.216+0000 7f78d22d8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc112a80 con 0x7f78cc1032c0 2024-04-25T10:40:14.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.216+0000 7f78d22d8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc112d80 con 0x7f78cc107950 2024-04-25T10:40:14.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.216+0000 7f78d0875640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78cc10d440 0x7f78cc0ff420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.216+0000 7f78d0875640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78cc10d440 0x7f78cc0ff420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54054/0 (socket says 172.21.15.43:54054) 2024-04-25T10:40:14.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.216+0000 7f78d0875640 1 -- 172.21.15.43:0/1875456425 learned_addr learned my addr 172.21.15.43:0/1875456425 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:14.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.216+0000 7f78cbfff640 1 --2- 172.21.15.43:0/1875456425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78cc1032c0 0x7f78cc0fe9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.216+0000 7f78cb7fe640 1 --2- 172.21.15.43:0/1875456425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78cc107950 0x7f78cc0feee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.216+0000 7f78d0875640 1 -- 172.21.15.43:0/1875456425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78cc107950 msgr2=0x7f78cc0feee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.216+0000 7f78d0875640 1 --2- 172.21.15.43:0/1875456425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78cc107950 0x7f78cc0feee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.216+0000 7f78d0875640 1 -- 172.21.15.43:0/1875456425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78cc1032c0 msgr2=0x7f78cc0fe9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.216+0000 7f78d0875640 1 --2- 172.21.15.43:0/1875456425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78cc1032c0 0x7f78cc0fe9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.216+0000 7f78d0875640 1 -- 172.21.15.43:0/1875456425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78cc1a8b10 con 0x7f78cc10d440 2024-04-25T10:40:14.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.216+0000 7f78d0875640 1 --2- 172.21.15.43:0/1875456425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78cc10d440 0x7f78cc0ff420 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f78c400eb40 tx=0x7f78c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:14.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.217+0000 7f78c97fa640 1 -- 172.21.15.43:0/1875456425 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4004b20 con 0x7f78cc10d440 2024-04-25T10:40:14.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.217+0000 7f78d22d8640 1 -- 172.21.15.43:0/1875456425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78cc1a8e00 con 0x7f78cc10d440 2024-04-25T10:40:14.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.217+0000 7f78c97fa640 1 -- 172.21.15.43:0/1875456425 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78c4004cc0 con 0x7f78cc10d440 2024-04-25T10:40:14.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.217+0000 7f78c97fa640 1 -- 172.21.15.43:0/1875456425 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4010720 con 0x7f78cc10d440 2024-04-25T10:40:14.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.217+0000 7f78d22d8640 1 -- 172.21.15.43:0/1875456425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78cc1a9260 con 0x7f78cc10d440 2024-04-25T10:40:14.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.219+0000 7f78c97fa640 1 -- 172.21.15.43:0/1875456425 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f78c4005560 con 0x7f78cc10d440 2024-04-25T10:40:14.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.219+0000 7f78d9188740 1 Processor -- start 2024-04-25T10:40:14.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.219+0000 7f78d9188740 1 -- start start 2024-04-25T10:40:14.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78d9188740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707133f390 0x5570716e0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78d9188740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570713164d0 0x5570716e0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78d9188740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707131e6b0 0x5570716ed6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78d9188740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557071645d80 con 0x5570713164d0 2024-04-25T10:40:14.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78d9188740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5570715ffa20 con 0x55707131e6b0 2024-04-25T10:40:14.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78d9188740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557071641d40 con 0x55707133f390 2024-04-25T10:40:14.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78c97fa640 1 --2- 172.21.15.43:0/1875456425 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f78b80afc60 0x7f78b80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570713164d0 0x5570716e0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570713164d0 0x5570716e0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54066/0 (socket says 172.21.15.43:54066) 2024-04-25T10:40:14.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78cbfff640 1 -- 172.21.15.43:0/1037623771 learned_addr learned my addr 172.21.15.43:0/1037623771 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:14.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78d0875640 1 --2- 172.21.15.43:0/1875456425 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f78b80afc60 0x7f78b80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78cb7fe640 1 --2- 172.21.15.43:0/1037623771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707133f390 0x5570716e0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78cb7fe640 1 --2- 172.21.15.43:0/1037623771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707131e6b0 0x5570716ed6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78c97fa640 1 -- 172.21.15.43:0/1875456425 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f78c40df8d0 con 0x7f78cc10d440 2024-04-25T10:40:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78cbfff640 1 -- 172.21.15.43:0/1037623771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707133f390 msgr2=0x5570716e0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78cbfff640 1 --2- 172.21.15.43:0/1037623771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707133f390 0x5570716e0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78cbfff640 1 -- 172.21.15.43:0/1037623771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707131e6b0 msgr2=0x5570716ed6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78cbfff640 1 --2- 172.21.15.43:0/1037623771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707131e6b0 0x5570716ed6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78cbfff640 1 -- 172.21.15.43:0/1037623771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55707160ac30 con 0x5570713164d0 2024-04-25T10:40:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78cb7fe640 1 --2- 172.21.15.43:0/1037623771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707133f390 0x5570716e0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78cbfff640 1 --2- 172.21.15.43:0/1037623771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570713164d0 0x5570716e0d50 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f78b400bcb0 tx=0x7f78b400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7cc72273f8996c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.220+0000 7f78d0875640 1 --2- 172.21.15.43:0/1875456425 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f78b80afc60 0x7f78b80b2170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f78c40a4c80 tx=0x7f78c4076510 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.221+0000 7f78c27fc640 1 -- 172.21.15.43:0/1037623771 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4015070 con 0x5570713164d0 2024-04-25T10:40:14.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.221+0000 7f78c27fc640 1 -- 172.21.15.43:0/1037623771 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78b401a910 con 0x5570713164d0 2024-04-25T10:40:14.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.221+0000 7f78c27fc640 1 -- 172.21.15.43:0/1037623771 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4011980 con 0x5570713164d0 2024-04-25T10:40:14.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.221+0000 7f78d9188740 1 -- 172.21.15.43:0/1037623771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570713164d0 msgr2=0x5570716e0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.221+0000 7f78d9188740 1 --2- 172.21.15.43:0/1037623771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570713164d0 0x5570716e0d50 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f78b400bcb0 tx=0x7f78b400f730 comp rx=0 tx=0).stop 2024-04-25T10:40:14.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.221+0000 7f78d9188740 1 -- 172.21.15.43:0/1037623771 shutdown_connections 2024-04-25T10:40:14.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.221+0000 7f78d9188740 1 --2- 172.21.15.43:0/1037623771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707131e6b0 0x5570716ed6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.221+0000 7f78d9188740 1 --2- 172.21.15.43:0/1037623771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570713164d0 0x5570716e0d50 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.221+0000 7f78d9188740 1 --2- 172.21.15.43:0/1037623771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707133f390 0x5570716e0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.221+0000 7f78d9188740 1 -- 172.21.15.43:0/1037623771 >> 172.21.15.43:0/1037623771 conn(0x5570713334e0 msgr2=0x5570716dcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:14.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.221+0000 7f78d9188740 1 -- 172.21.15.43:0/1037623771 shutdown_connections 2024-04-25T10:40:14.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.221+0000 7f78d9188740 1 -- 172.21.15.43:0/1037623771 wait complete. 2024-04-25T10:40:14.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.222+0000 7f78d9188740 1 -- start start 2024-04-25T10:40:14.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.222+0000 7f78d9188740 1 Processor -- start 2024-04-25T10:40:14.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.222+0000 7f78d9188740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570713164d0 0x5570716eb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.222+0000 7f78d9188740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707131e6b0 0x5570716eb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.222+0000 7f78d9188740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707133f390 0x5570716f7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.222+0000 7f78d9188740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5570716edd90 con 0x55707131e6b0 2024-04-25T10:40:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.222+0000 7f78d9188740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5570715ccdc0 con 0x5570713164d0 2024-04-25T10:40:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.222+0000 7f78d9188740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5570716edf10 con 0x55707133f390 2024-04-25T10:40:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.222+0000 7f78d9188740 10 client.0 _ll_register_callbacks cb 0x5570715e8600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.222+0000 7f78cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707131e6b0 0x5570716eb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.223+0000 7f78cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707131e6b0 0x5570716eb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54072/0 (socket says 172.21.15.43:54072) 2024-04-25T10:40:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.223+0000 7f78cbfff640 1 -- 172.21.15.43:0/3252241844 learned_addr learned my addr 172.21.15.43:0/3252241844 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.223+0000 7f78cb7fe640 1 --2- 172.21.15.43:0/3252241844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707133f390 0x5570716f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.223+0000 7f78cb7fe640 1 --2- 172.21.15.43:0/3252241844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570713164d0 0x5570716eb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.223+0000 7f78cbfff640 1 -- 172.21.15.43:0/3252241844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707133f390 msgr2=0x5570716f7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.223+0000 7f78cbfff640 1 --2- 172.21.15.43:0/3252241844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707133f390 0x5570716f7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.223+0000 7f78cbfff640 1 -- 172.21.15.43:0/3252241844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570713164d0 msgr2=0x5570716eb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.223+0000 7f78cbfff640 1 --2- 172.21.15.43:0/3252241844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570713164d0 0x5570716eb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.223+0000 7f78cbfff640 1 -- 172.21.15.43:0/3252241844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570713573f0 con 0x55707131e6b0 2024-04-25T10:40:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.223+0000 7f78cb7fe640 1 --2- 172.21.15.43:0/3252241844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570713164d0 0x5570716eb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:14.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.223+0000 7f78cbfff640 1 --2- 172.21.15.43:0/3252241844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707131e6b0 0x5570716eb7a0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f78b400b9f0 tx=0x7f78b400bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c65c8bfab5db85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:14.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.223+0000 7f78ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:14.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.223+0000 7f78ab7fe640 1 -- 172.21.15.43:0/3252241844 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4015070 con 0x55707131e6b0 2024-04-25T10:40:14.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.223+0000 7f78d9188740 1 -- 172.21.15.43:0/3252241844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78b401a910 con 0x55707131e6b0 2024-04-25T10:40:14.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.223+0000 7f78ab7fe640 1 -- 172.21.15.43:0/3252241844 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f78b401aac0 con 0x55707131e6b0 2024-04-25T10:40:14.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.223+0000 7f78ab7fe640 1 -- 172.21.15.43:0/3252241844 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f78b401acc0 con 0x55707131e6b0 2024-04-25T10:40:14.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.223+0000 7f78d9188740 10 client.17790 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:14.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.223+0000 7f78d9188740 1 -- 172.21.15.43:0/3252241844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5570714217d0 con 0x55707131e6b0 2024-04-25T10:40:14.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.224+0000 7f78ab7fe640 1 -- 172.21.15.43:0/3252241844 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78b401a910 con 0x55707131e6b0 2024-04-25T10:40:14.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.224+0000 7f78ab7fe640 1 -- 172.21.15.43:0/3252241844 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f78b4011a00 con 0x55707131e6b0 2024-04-25T10:40:14.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.224+0000 7f78d9188740 10 client.17790 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:14.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.224+0000 7f78d9188740 10 client.17790 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:40:14.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.224+0000 7f78d9188740 20 client.17790 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:14.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.224+0000 7f78d9188740 1 --2- 172.21.15.43:0/3252241844 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5570712fdf60 0x5570716fd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.224+0000 7f78d9188740 4 client.17790 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:14.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.224+0000 7f78d9188740 1 -- 172.21.15.43:0/3252241844 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557071653990 con 0x5570712fdf60 2024-04-25T10:40:14.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.224+0000 7f78cb7fe640 1 --2- 172.21.15.43:0/3252241844 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5570712fdf60 0x5570716fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.224+0000 7f78cb7fe640 1 --2- 172.21.15.43:0/3252241844 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5570712fdf60 0x5570716fd510 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6772c7a72359dda1 server_cookie=3b23114309f90653 in_seq=0 out_seq=0 2024-04-25T10:40:14.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.224+0000 7f78ab7fe640 10 client.17790 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:40:14.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.225+0000 7f78ab7fe640 1 -- 172.21.15.43:0/3252241844 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557071653990 con 0x5570712fdf60 2024-04-25T10:40:14.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.225+0000 7f78ab7fe640 10 client.17790 handle_command_reply: tid=0 2024-04-25T10:40:14.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.225+0000 7f78d9188740 1 client.17790 shutdown 2024-04-25T10:40:14.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.225+0000 7f78d9188740 1 -- 172.21.15.43:0/3252241844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707131e6b0 msgr2=0x5570716eb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.225+0000 7f78d9188740 1 --2- 172.21.15.43:0/3252241844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707131e6b0 0x5570716eb7a0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f78b400b9f0 tx=0x7f78b400bfb0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.225+0000 7f78d9188740 1 -- 172.21.15.43:0/3252241844 shutdown_connections 2024-04-25T10:40:14.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.225+0000 7f78d9188740 1 --2- 172.21.15.43:0/3252241844 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5570712fdf60 0x5570716fd510 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.226+0000 7f78d9188740 1 --2- 172.21.15.43:0/3252241844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707133f390 0x5570716f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.226+0000 7f78d9188740 1 --2- 172.21.15.43:0/3252241844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707131e6b0 0x5570716eb7a0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.226+0000 7f78d9188740 1 --2- 172.21.15.43:0/3252241844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570713164d0 0x5570716eb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.226+0000 7f78ab7fe640 0 client.17790 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:40:14.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.226+0000 7f78d9188740 1 -- 172.21.15.43:0/3252241844 >> 172.21.15.43:0/3252241844 conn(0x5570713334e0 msgr2=0x5570716e2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:14.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.226+0000 7f78d9188740 1 -- 172.21.15.43:0/3252241844 shutdown_connections 2024-04-25T10:40:14.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.226+0000 7f78d9188740 1 -- 172.21.15.43:0/3252241844 wait complete. 2024-04-25T10:40:14.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.226+0000 7f78d9188740 20 client.17790 trim_cache size 0 max 16384 2024-04-25T10:40:14.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.239+0000 7f78d9188740 1 Processor -- start 2024-04-25T10:40:14.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.239+0000 7f78d9188740 1 -- start start 2024-04-25T10:40:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.240+0000 7f78d9188740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570713164d0 0x5570716f7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.240+0000 7f78d9188740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707131e6b0 0x5570716ece80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.240+0000 7f78d9188740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707133f390 0x5570716ea800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.240+0000 7f78d9188740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1b4500 con 0x55707133f390 2024-04-25T10:40:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.240+0000 7f78d9188740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1b1dd0 con 0x5570713164d0 2024-04-25T10:40:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.240+0000 7f78d9188740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1afc40 con 0x55707131e6b0 2024-04-25T10:40:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.240+0000 7f78cb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707133f390 0x5570716ea800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.240+0000 7f78cb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707133f390 0x5570716ea800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54086/0 (socket says 172.21.15.43:54086) 2024-04-25T10:40:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.240+0000 7f78cb7fe640 1 -- 172.21.15.43:0/172078829 learned_addr learned my addr 172.21.15.43:0/172078829 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.240+0000 7f78cb7fe640 1 --2- 172.21.15.43:0/172078829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570713164d0 0x5570716f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.240+0000 7f78cbfff640 1 --2- 172.21.15.43:0/172078829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707131e6b0 0x5570716ece80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.240+0000 7f78cb7fe640 1 -- 172.21.15.43:0/172078829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707131e6b0 msgr2=0x5570716ece80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.240+0000 7f78cb7fe640 1 --2- 172.21.15.43:0/172078829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707131e6b0 0x5570716ece80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.240+0000 7f78cb7fe640 1 -- 172.21.15.43:0/172078829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570713164d0 msgr2=0x5570716f7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.240+0000 7f78cb7fe640 1 --2- 172.21.15.43:0/172078829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570713164d0 0x5570716f7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.240+0000 7f78cb7fe640 1 -- 172.21.15.43:0/172078829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570716de050 con 0x55707133f390 2024-04-25T10:40:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.240+0000 7f78cb7fe640 1 --2- 172.21.15.43:0/172078829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707133f390 0x5570716ea800 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f78bc005ab0 tx=0x7f78bc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e51f94ab643d16f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.240+0000 7f78c1ffb640 1 -- 172.21.15.43:0/172078829 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc035040 con 0x55707133f390 2024-04-25T10:40:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.240+0000 7f78c1ffb640 1 -- 172.21.15.43:0/172078829 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78bc04b070 con 0x55707133f390 2024-04-25T10:40:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.240+0000 7f78c1ffb640 1 -- 172.21.15.43:0/172078829 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc0457e0 con 0x55707133f390 2024-04-25T10:40:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.241+0000 7f78d9188740 1 -- 172.21.15.43:0/172078829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707133f390 msgr2=0x5570716ea800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.241+0000 7f78d9188740 1 --2- 172.21.15.43:0/172078829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707133f390 0x5570716ea800 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f78bc005ab0 tx=0x7f78bc02a410 comp rx=0 tx=0).stop 2024-04-25T10:40:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.241+0000 7f78d9188740 1 -- 172.21.15.43:0/172078829 shutdown_connections 2024-04-25T10:40:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.241+0000 7f78d9188740 1 --2- 172.21.15.43:0/172078829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707133f390 0x5570716ea800 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.241+0000 7f78d9188740 1 --2- 172.21.15.43:0/172078829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707131e6b0 0x5570716ece80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.241+0000 7f78d9188740 1 --2- 172.21.15.43:0/172078829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570713164d0 0x5570716f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.241+0000 7f78d9188740 1 -- 172.21.15.43:0/172078829 >> 172.21.15.43:0/172078829 conn(0x5570713334e0 msgr2=0x5570716ec770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.241+0000 7f78d9188740 1 -- 172.21.15.43:0/172078829 shutdown_connections 2024-04-25T10:40:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.241+0000 7f78d9188740 1 -- 172.21.15.43:0/172078829 wait complete. 2024-04-25T10:40:14.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.241+0000 7f78d9188740 1 -- start start 2024-04-25T10:40:14.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.242+0000 7f78d9188740 1 Processor -- start 2024-04-25T10:40:14.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.242+0000 7f78d9188740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570713164d0 0x557071700350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.242+0000 7f78d9188740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707131e6b0 0x557071700890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.242+0000 7f78d9188740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707133f390 0x5570716fd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.242+0000 7f78d9188740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1b0290 con 0x55707131e6b0 2024-04-25T10:40:14.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.242+0000 7f78d9188740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1b1700 con 0x5570713164d0 2024-04-25T10:40:14.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.242+0000 7f78d9188740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1b0960 con 0x55707133f390 2024-04-25T10:40:14.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.242+0000 7f78d9188740 10 client.0 _ll_register_callbacks cb 0x7f78b400bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:14.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.242+0000 7f78cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707131e6b0 0x557071700890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.242+0000 7f78cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707131e6b0 0x557071700890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54090/0 (socket says 172.21.15.43:54090) 2024-04-25T10:40:14.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.242+0000 7f78cbfff640 1 -- 172.21.15.43:0/1064845012 learned_addr learned my addr 172.21.15.43:0/1064845012 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:14.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.242+0000 7f78cb7fe640 1 --2- 172.21.15.43:0/1064845012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707133f390 0x5570716fd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.242+0000 7f78cb7fe640 1 --2- 172.21.15.43:0/1064845012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570713164d0 0x557071700350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.242+0000 7f78cbfff640 1 -- 172.21.15.43:0/1064845012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707133f390 msgr2=0x5570716fd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.242+0000 7f78cbfff640 1 --2- 172.21.15.43:0/1064845012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707133f390 0x5570716fd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.242+0000 7f78cbfff640 1 -- 172.21.15.43:0/1064845012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570713164d0 msgr2=0x557071700350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.242+0000 7f78cbfff640 1 --2- 172.21.15.43:0/1064845012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570713164d0 0x557071700350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.242+0000 7f78cbfff640 1 -- 172.21.15.43:0/1064845012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570716fde50 con 0x55707131e6b0 2024-04-25T10:40:14.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.243+0000 7f78cbfff640 1 --2- 172.21.15.43:0/1064845012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707131e6b0 0x557071700890 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f78b4006010 tx=0x7f78b401a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6938ffe2984a81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:14.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.243+0000 7f78ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:14.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.243+0000 7f78ab7fe640 1 -- 172.21.15.43:0/1064845012 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b401ae60 con 0x55707131e6b0 2024-04-25T10:40:14.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.243+0000 7f78ab7fe640 1 -- 172.21.15.43:0/1064845012 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f78b401a7f0 con 0x55707131e6b0 2024-04-25T10:40:14.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.243+0000 7f78d9188740 1 -- 172.21.15.43:0/1064845012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78bc04b070 con 0x55707131e6b0 2024-04-25T10:40:14.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.243+0000 7f78ab7fe640 1 -- 172.21.15.43:0/1064845012 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f78b403b050 con 0x55707131e6b0 2024-04-25T10:40:14.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.243+0000 7f78d9188740 10 client.17796 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:14.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.243+0000 7f78d9188740 1 -- 172.21.15.43:0/1064845012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557071764f60 con 0x55707131e6b0 2024-04-25T10:40:14.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.243+0000 7f78ab7fe640 1 -- 172.21.15.43:0/1064845012 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78bc04b070 con 0x55707131e6b0 2024-04-25T10:40:14.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.243+0000 7f78ab7fe640 1 -- 172.21.15.43:0/1064845012 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f78b40118d0 con 0x55707131e6b0 2024-04-25T10:40:14.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.243+0000 7f78d9188740 10 client.17796 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.243+0000 7f78d9188740 10 client.17796 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:40:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.243+0000 7f78d9188740 20 client.17796 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.243+0000 7f78d9188740 1 --2- 172.21.15.43:0/1064845012 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5570712d33a0 0x557071767720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.243+0000 7f78d9188740 4 client.17796 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:40:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.243+0000 7f78d9188740 1 -- 172.21.15.43:0/1064845012 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55707164d800 con 0x5570712d33a0 2024-04-25T10:40:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.244+0000 7f78cb7fe640 1 --2- 172.21.15.43:0/1064845012 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5570712d33a0 0x557071767720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.244+0000 7f78cb7fe640 1 --2- 172.21.15.43:0/1064845012 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5570712d33a0 0x557071767720 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b258e168090c2431 server_cookie=749e7b91cf0d2e12 in_seq=0 out_seq=0 2024-04-25T10:40:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.244+0000 7f78ab7fe640 10 client.17796 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:40:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.244+0000 7f78ab7fe640 1 -- 172.21.15.43:0/1064845012 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55707164d800 con 0x5570712d33a0 2024-04-25T10:40:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.244+0000 7f78ab7fe640 10 client.17796 handle_command_reply: tid=0 2024-04-25T10:40:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.244+0000 7f78d9188740 1 client.17796 shutdown 2024-04-25T10:40:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.244+0000 7f78d9188740 1 -- 172.21.15.43:0/1064845012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707131e6b0 msgr2=0x557071700890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.244+0000 7f78d9188740 1 --2- 172.21.15.43:0/1064845012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707131e6b0 0x557071700890 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f78b4006010 tx=0x7f78b401a570 comp rx=0 tx=0).stop 2024-04-25T10:40:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.244+0000 7f78d9188740 1 -- 172.21.15.43:0/1064845012 shutdown_connections 2024-04-25T10:40:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.244+0000 7f78d9188740 1 --2- 172.21.15.43:0/1064845012 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5570712d33a0 0x557071767720 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.244+0000 7f78d9188740 1 --2- 172.21.15.43:0/1064845012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55707133f390 0x5570716fd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.245+0000 7f78d9188740 1 --2- 172.21.15.43:0/1064845012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55707131e6b0 0x557071700890 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.245+0000 7f78d9188740 1 --2- 172.21.15.43:0/1064845012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570713164d0 0x557071700350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.245+0000 7f78ab7fe640 0 client.17796 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:40:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.245+0000 7f78d9188740 1 -- 172.21.15.43:0/1064845012 >> 172.21.15.43:0/1064845012 conn(0x5570713334e0 msgr2=0x5570716ed130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.245+0000 7f78d9188740 1 -- 172.21.15.43:0/1064845012 shutdown_connections 2024-04-25T10:40:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.245+0000 7f78d9188740 1 -- 172.21.15.43:0/1064845012 wait complete. 2024-04-25T10:40:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.245+0000 7f78d9188740 20 client.17796 trim_cache size 0 max 16384 2024-04-25T10:40:14.253 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:40:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.245+0000 7f78d22d8640 1 -- 172.21.15.43:0/1875456425 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f78b80afc60 msgr2=0x7f78b80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.245+0000 7f78d22d8640 1 --2- 172.21.15.43:0/1875456425 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f78b80afc60 0x7f78b80b2170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f78c40a4c80 tx=0x7f78c4076510 comp rx=0 tx=0).stop 2024-04-25T10:40:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.245+0000 7f78d22d8640 1 -- 172.21.15.43:0/1875456425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78cc10d440 msgr2=0x7f78cc0ff420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.245+0000 7f78d22d8640 1 --2- 172.21.15.43:0/1875456425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78cc10d440 0x7f78cc0ff420 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f78c400eb40 tx=0x7f78c400c2d0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.245+0000 7f78d22d8640 1 -- 172.21.15.43:0/1875456425 shutdown_connections 2024-04-25T10:40:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.245+0000 7f78d22d8640 1 --2- 172.21.15.43:0/1875456425 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f78b80afc60 0x7f78b80b2170 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.245+0000 7f78d22d8640 1 --2- 172.21.15.43:0/1875456425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78cc10d440 0x7f78cc0ff420 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.245+0000 7f78d22d8640 1 --2- 172.21.15.43:0/1875456425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78cc107950 0x7f78cc0feee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.245+0000 7f78d22d8640 1 --2- 172.21.15.43:0/1875456425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78cc1032c0 0x7f78cc0fe9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.245+0000 7f78d22d8640 1 -- 172.21.15.43:0/1875456425 >> 172.21.15.43:0/1875456425 conn(0x7f78cc077920 msgr2=0x7f78cc105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.245+0000 7f78d22d8640 1 -- 172.21.15.43:0/1875456425 shutdown_connections 2024-04-25T10:40:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.245+0000 7f78d22d8640 1 -- 172.21.15.43:0/1875456425 wait complete. 2024-04-25T10:40:14.257 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:13 smithi043 ceph-mon[29478]: pgmap v237: 97 pgs: 97 active+clean; 4.7 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 20 op/s 2024-04-25T10:40:14.257 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:13 smithi043 ceph-mon[29478]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T10:40:14.257 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:13 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:40:14.257 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:13 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:40:14.257 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:13 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:40:14.257 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:13 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:40:14.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:13 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:40:14.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:13 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2655594289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:40:14.259 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:40:14.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:13 smithi077 ceph-mon[32772]: pgmap v237: 97 pgs: 97 active+clean; 4.7 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 20 op/s 2024-04-25T10:40:14.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:13 smithi077 ceph-mon[32772]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T10:40:14.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:13 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:40:14.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:13 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:40:14.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:13 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:40:14.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:13 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:40:14.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:13 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:40:14.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:13 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2655594289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:40:14.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:13 smithi164 ceph-mon[32713]: pgmap v237: 97 pgs: 97 active+clean; 4.7 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 20 op/s 2024-04-25T10:40:14.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:13 smithi164 ceph-mon[32713]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T10:40:14.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:13 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:40:14.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:13 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:40:14.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:13 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:40:14.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:13 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:40:14.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:13 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:40:14.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:13 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2655594289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:40:14.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.370+0000 7faea78e0640 1 -- 172.21.15.43:0/4144594519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faea0102af0 msgr2=0x7faea0102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.370+0000 7faea78e0640 1 --2- 172.21.15.43:0/4144594519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faea0102af0 0x7faea0102f00 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fae90009900 tx=0x7fae9002a410 comp rx=0 tx=0).stop 2024-04-25T10:40:14.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.370+0000 7faea78e0640 1 -- 172.21.15.43:0/4144594519 shutdown_connections 2024-04-25T10:40:14.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.370+0000 7faea78e0640 1 --2- 172.21.15.43:0/4144594519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faea010d380 0x7faea010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.370+0000 7faea78e0640 1 --2- 172.21.15.43:0/4144594519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faea0102af0 0x7faea0102f00 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.370+0000 7faea78e0640 1 --2- 172.21.15.43:0/4144594519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faea0108a20 0x7faea01025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.370+0000 7faea78e0640 1 -- 172.21.15.43:0/4144594519 >> 172.21.15.43:0/4144594519 conn(0x7faea0078100 msgr2=0x7faea0078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:14.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.370+0000 7faea78e0640 1 -- 172.21.15.43:0/4144594519 shutdown_connections 2024-04-25T10:40:14.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.370+0000 7faea78e0640 1 -- 172.21.15.43:0/4144594519 wait complete. 2024-04-25T10:40:14.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.371+0000 7faea78e0640 1 Processor -- start 2024-04-25T10:40:14.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.371+0000 7faea78e0640 1 -- start start 2024-04-25T10:40:14.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.371+0000 7faea78e0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faea0102af0 0x7faea019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.371+0000 7faea78e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faea0108a20 0x7faea019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.371+0000 7faea78e0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faea010d380 0x7faea019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.371+0000 7faea78e0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faea0113d00 con 0x7faea0108a20 2024-04-25T10:40:14.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.371+0000 7faea78e0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faea0113b80 con 0x7faea0102af0 2024-04-25T10:40:14.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.371+0000 7faea78e0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faea0113e80 con 0x7faea010d380 2024-04-25T10:40:14.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.371+0000 7faea4e7b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faea0108a20 0x7faea019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.371+0000 7faea4e7b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faea0108a20 0x7faea019ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54112/0 (socket says 172.21.15.43:54112) 2024-04-25T10:40:14.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.371+0000 7faea4e7b640 1 -- 172.21.15.43:0/1352802600 learned_addr learned my addr 172.21.15.43:0/1352802600 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:14.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.371+0000 7faea567c640 1 --2- 172.21.15.43:0/1352802600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faea0102af0 0x7faea019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.371+0000 7faea5e7d640 1 --2- 172.21.15.43:0/1352802600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faea010d380 0x7faea019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.371+0000 7faea4e7b640 1 -- 172.21.15.43:0/1352802600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faea010d380 msgr2=0x7faea019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.371+0000 7faea4e7b640 1 --2- 172.21.15.43:0/1352802600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faea010d380 0x7faea019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.372+0000 7faea4e7b640 1 -- 172.21.15.43:0/1352802600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faea0102af0 msgr2=0x7faea019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.372+0000 7faea4e7b640 1 --2- 172.21.15.43:0/1352802600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faea0102af0 0x7faea019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.372+0000 7faea4e7b640 1 -- 172.21.15.43:0/1352802600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea019f560 con 0x7faea0108a20 2024-04-25T10:40:14.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.372+0000 7faea4e7b640 1 --2- 172.21.15.43:0/1352802600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faea0108a20 0x7faea019ac60 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fae90006010 tx=0x7fae90004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:14.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.372+0000 7fae967fc640 1 -- 172.21.15.43:0/1352802600 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9003f030 con 0x7faea0108a20 2024-04-25T10:40:14.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.372+0000 7fae967fc640 1 -- 172.21.15.43:0/1352802600 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae90003aa0 con 0x7faea0108a20 2024-04-25T10:40:14.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.372+0000 7faea78e0640 1 -- 172.21.15.43:0/1352802600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea019f7f0 con 0x7faea0108a20 2024-04-25T10:40:14.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.372+0000 7fae967fc640 1 -- 172.21.15.43:0/1352802600 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90035070 con 0x7faea0108a20 2024-04-25T10:40:14.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.372+0000 7faea78e0640 1 -- 172.21.15.43:0/1352802600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea019fce0 con 0x7faea0108a20 2024-04-25T10:40:14.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.375+0000 7fae967fc640 1 -- 172.21.15.43:0/1352802600 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fae90042030 con 0x7faea0108a20 2024-04-25T10:40:14.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.375+0000 7faeae78a740 1 Processor -- start 2024-04-25T10:40:14.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.375+0000 7faeae78a740 1 -- start start 2024-04-25T10:40:14.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.375+0000 7faeae78a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07ce3c390 0x55e07d1dd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.375+0000 7faeae78a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce134d0 0x55e07d1ddc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.375+0000 7faeae78a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e07ce1b6b0 0x55e07d1eeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.375+0000 7faeae78a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e07d142d80 con 0x55e07ce3c390 2024-04-25T10:40:14.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.375+0000 7faeae78a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e07d0fca20 con 0x55e07ce134d0 2024-04-25T10:40:14.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.375+0000 7faeae78a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e07d13ed40 con 0x55e07ce1b6b0 2024-04-25T10:40:14.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.375+0000 7faea5e7d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07ce3c390 0x55e07d1dd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.375+0000 7faea5e7d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07ce3c390 0x55e07d1dd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54122/0 (socket says 172.21.15.43:54122) 2024-04-25T10:40:14.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.375+0000 7faea5e7d640 1 -- 172.21.15.43:0/198792882 learned_addr learned my addr 172.21.15.43:0/198792882 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:14.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.376+0000 7faea567c640 1 --2- 172.21.15.43:0/198792882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce134d0 0x55e07d1ddc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.376+0000 7faea4e7b640 1 --2- 172.21.15.43:0/198792882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e07ce1b6b0 0x55e07d1eeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.376+0000 7fae967fc640 1 --2- 172.21.15.43:0/1352802600 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fae8c0afc60 0x7fae8c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.376+0000 7faea5e7d640 1 -- 172.21.15.43:0/198792882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e07ce1b6b0 msgr2=0x55e07d1eeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.376+0000 7faea5e7d640 1 --2- 172.21.15.43:0/198792882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e07ce1b6b0 0x55e07d1eeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.376+0000 7faea5e7d640 1 -- 172.21.15.43:0/198792882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce134d0 msgr2=0x55e07d1ddc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.376+0000 7faea5e7d640 1 --2- 172.21.15.43:0/198792882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce134d0 0x55e07d1ddc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.376+0000 7faea5e7d640 1 -- 172.21.15.43:0/198792882 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e07d107c30 con 0x55e07ce3c390 2024-04-25T10:40:14.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.376+0000 7fae967fc640 1 -- 172.21.15.43:0/1352802600 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fae900ff470 con 0x7faea0108a20 2024-04-25T10:40:14.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.376+0000 7faea5e7d640 1 --2- 172.21.15.43:0/1352802600 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fae8c0afc60 0x7fae8c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.376+0000 7faea5e7d640 1 --2- 172.21.15.43:0/198792882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07ce3c390 0x55e07d1dd6c0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fae98011590 tx=0x7fae98011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6848ea23f4e5cd9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:14.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.376+0000 7fae8b7fe640 1 -- 172.21.15.43:0/198792882 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98017050 con 0x55e07ce3c390 2024-04-25T10:40:14.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.376+0000 7fae8b7fe640 1 -- 172.21.15.43:0/198792882 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae9801bb90 con 0x55e07ce3c390 2024-04-25T10:40:14.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.376+0000 7fae8b7fe640 1 -- 172.21.15.43:0/198792882 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98025d80 con 0x55e07ce3c390 2024-04-25T10:40:14.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.377+0000 7faeae78a740 1 -- 172.21.15.43:0/198792882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07ce3c390 msgr2=0x55e07d1dd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.377+0000 7faeae78a740 1 --2- 172.21.15.43:0/198792882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07ce3c390 0x55e07d1dd6c0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fae98011590 tx=0x7fae98011a50 comp rx=0 tx=0).stop 2024-04-25T10:40:14.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.377+0000 7faea5e7d640 1 --2- 172.21.15.43:0/1352802600 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fae8c0afc60 0x7fae8c0b2170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fae98022000 tx=0x7fae98023000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:14.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.377+0000 7faeae78a740 1 -- 172.21.15.43:0/198792882 shutdown_connections 2024-04-25T10:40:14.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.377+0000 7faeae78a740 1 --2- 172.21.15.43:0/198792882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce134d0 0x55e07d1ddc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.377+0000 7faeae78a740 1 --2- 172.21.15.43:0/198792882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e07ce1b6b0 0x55e07d1eeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.377+0000 7faeae78a740 1 --2- 172.21.15.43:0/198792882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07ce3c390 0x55e07d1dd6c0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.377+0000 7faeae78a740 1 -- 172.21.15.43:0/198792882 >> 172.21.15.43:0/198792882 conn(0x55e07ce304e0 msgr2=0x55e07d1d9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:14.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.377+0000 7faeae78a740 1 -- 172.21.15.43:0/198792882 shutdown_connections 2024-04-25T10:40:14.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.377+0000 7faeae78a740 1 -- 172.21.15.43:0/198792882 wait complete. 2024-04-25T10:40:14.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.378+0000 7faeae78a740 1 -- start start 2024-04-25T10:40:14.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.378+0000 7faeae78a740 1 Processor -- start 2024-04-25T10:40:14.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.378+0000 7faeae78a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e07ce134d0 0x55e07d1ec630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.378+0000 7faeae78a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07ce1b6b0 0x55e07d1ecb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.378+0000 7faeae78a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce3c390 0x55e07d1f8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.378+0000 7faeae78a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e07d1ef1a0 con 0x55e07ce1b6b0 2024-04-25T10:40:14.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.378+0000 7faeae78a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e07d0c9dc0 con 0x55e07ce3c390 2024-04-25T10:40:14.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.378+0000 7faeae78a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e07d1ef320 con 0x55e07ce134d0 2024-04-25T10:40:14.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.378+0000 7faeae78a740 10 client.0 _ll_register_callbacks cb 0x55e07d0e5600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:14.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.378+0000 7faea4e7b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07ce1b6b0 0x55e07d1ecb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.378+0000 7faea4e7b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07ce1b6b0 0x55e07d1ecb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54136/0 (socket says 172.21.15.43:54136) 2024-04-25T10:40:14.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.378+0000 7faea4e7b640 1 -- 172.21.15.43:0/2453454077 learned_addr learned my addr 172.21.15.43:0/2453454077 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:14.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.379+0000 7faea5e7d640 1 --2- 172.21.15.43:0/2453454077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce3c390 0x55e07d1f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.379+0000 7faea567c640 1 --2- 172.21.15.43:0/2453454077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e07ce134d0 0x55e07d1ec630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.379+0000 7faea4e7b640 1 -- 172.21.15.43:0/2453454077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e07ce134d0 msgr2=0x55e07d1ec630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.379+0000 7faea4e7b640 1 --2- 172.21.15.43:0/2453454077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e07ce134d0 0x55e07d1ec630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.379+0000 7faea4e7b640 1 -- 172.21.15.43:0/2453454077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce3c390 msgr2=0x55e07d1f8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.379+0000 7faea4e7b640 1 --2- 172.21.15.43:0/2453454077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce3c390 0x55e07d1f8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.379+0000 7faea4e7b640 1 -- 172.21.15.43:0/2453454077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e07ce543f0 con 0x55e07ce1b6b0 2024-04-25T10:40:14.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.379+0000 7faea5e7d640 1 --2- 172.21.15.43:0/2453454077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce3c390 0x55e07d1f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:14.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.379+0000 7faea4e7b640 1 --2- 172.21.15.43:0/2453454077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07ce1b6b0 0x55e07d1ecb70 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fae900c3740 tx=0x7fae90009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20cec2a74997c25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:14.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.379+0000 7fae88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:14.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.379+0000 7fae88ff9640 1 -- 172.21.15.43:0/2453454077 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90049070 con 0x55e07ce1b6b0 2024-04-25T10:40:14.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.379+0000 7fae88ff9640 1 -- 172.21.15.43:0/2453454077 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fae900fce40 con 0x55e07ce1b6b0 2024-04-25T10:40:14.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.379+0000 7faeae78a740 1 -- 172.21.15.43:0/2453454077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae9801bb90 con 0x55e07ce1b6b0 2024-04-25T10:40:14.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.379+0000 7fae88ff9640 1 -- 172.21.15.43:0/2453454077 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fae900e1050 con 0x55e07ce1b6b0 2024-04-25T10:40:14.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.379+0000 7faeae78a740 10 client.17808 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:14.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.379+0000 7faeae78a740 1 -- 172.21.15.43:0/2453454077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e07cf1e7d0 con 0x55e07ce1b6b0 2024-04-25T10:40:14.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.379+0000 7fae88ff9640 1 -- 172.21.15.43:0/2453454077 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae9801bb90 con 0x55e07ce1b6b0 2024-04-25T10:40:14.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.380+0000 7fae88ff9640 1 -- 172.21.15.43:0/2453454077 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fae900c3950 con 0x55e07ce1b6b0 2024-04-25T10:40:14.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.380+0000 7faeae78a740 10 client.17808 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:14.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.380+0000 7faeae78a740 10 client.17808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:14.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.380+0000 7faeae78a740 20 client.17808 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:14.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.380+0000 7faeae78a740 1 --2- 172.21.15.43:0/2453454077 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e07cdfaf60 0x55e07d1fea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.380+0000 7faeae78a740 4 client.17808 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:14.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.380+0000 7faeae78a740 1 -- 172.21.15.43:0/2453454077 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e07d150990 con 0x55e07cdfaf60 2024-04-25T10:40:14.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.380+0000 7faea567c640 1 --2- 172.21.15.43:0/2453454077 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e07cdfaf60 0x55e07d1fea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.381+0000 7faea567c640 1 --2- 172.21.15.43:0/2453454077 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e07cdfaf60 0x55e07d1fea00 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1706f27d365635b server_cookie=50f44487e5aec0f3 in_seq=0 out_seq=0 2024-04-25T10:40:14.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.381+0000 7fae88ff9640 10 client.17808 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:14.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.381+0000 7fae88ff9640 1 -- 172.21.15.43:0/2453454077 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e07d150990 con 0x55e07cdfaf60 2024-04-25T10:40:14.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.381+0000 7fae88ff9640 10 client.17808 handle_command_reply: tid=0 2024-04-25T10:40:14.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.381+0000 7faeae78a740 1 client.17808 shutdown 2024-04-25T10:40:14.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.381+0000 7faeae78a740 1 -- 172.21.15.43:0/2453454077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07ce1b6b0 msgr2=0x55e07d1ecb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.381+0000 7faeae78a740 1 --2- 172.21.15.43:0/2453454077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07ce1b6b0 0x55e07d1ecb70 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fae900c3740 tx=0x7fae90009260 comp rx=0 tx=0).stop 2024-04-25T10:40:14.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.381+0000 7faeae78a740 1 -- 172.21.15.43:0/2453454077 shutdown_connections 2024-04-25T10:40:14.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.381+0000 7faeae78a740 1 --2- 172.21.15.43:0/2453454077 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e07cdfaf60 0x55e07d1fea00 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.381+0000 7faeae78a740 1 --2- 172.21.15.43:0/2453454077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce3c390 0x55e07d1f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.381+0000 7faeae78a740 1 --2- 172.21.15.43:0/2453454077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07ce1b6b0 0x55e07d1ecb70 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.381+0000 7fae88ff9640 0 client.17808 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:14.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.381+0000 7faeae78a740 1 --2- 172.21.15.43:0/2453454077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e07ce134d0 0x55e07d1ec630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.381+0000 7faeae78a740 1 -- 172.21.15.43:0/2453454077 >> 172.21.15.43:0/2453454077 conn(0x55e07ce304e0 msgr2=0x55e07d1df4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:14.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.382+0000 7faeae78a740 1 -- 172.21.15.43:0/2453454077 shutdown_connections 2024-04-25T10:40:14.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.382+0000 7faeae78a740 1 -- 172.21.15.43:0/2453454077 wait complete. 2024-04-25T10:40:14.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.382+0000 7faeae78a740 20 client.17808 trim_cache size 0 max 16384 2024-04-25T10:40:14.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.395+0000 7faeae78a740 1 Processor -- start 2024-04-25T10:40:14.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.395+0000 7faeae78a740 1 -- start start 2024-04-25T10:40:14.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.396+0000 7faeae78a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07cdfaf60 0x55e07d1ec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.396+0000 7faeae78a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e07ce134d0 0x55e07d1f82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.396+0000 7faeae78a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce1b6b0 0x55e07d2041c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.396+0000 7faeae78a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faea01b0080 con 0x55e07cdfaf60 2024-04-25T10:40:14.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.396+0000 7faeae78a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faea01ad950 con 0x55e07ce1b6b0 2024-04-25T10:40:14.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.396+0000 7faeae78a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faea01ab7c0 con 0x55e07ce134d0 2024-04-25T10:40:14.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.396+0000 7faea567c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07cdfaf60 0x55e07d1ec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.396+0000 7faea567c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07cdfaf60 0x55e07d1ec070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54146/0 (socket says 172.21.15.43:54146) 2024-04-25T10:40:14.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.396+0000 7faea567c640 1 -- 172.21.15.43:0/2689976054 learned_addr learned my addr 172.21.15.43:0/2689976054 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.396+0000 7faea567c640 1 -- 172.21.15.43:0/2689976054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e07ce134d0 msgr2=0x55e07d1f82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.396+0000 7faea567c640 1 --2- 172.21.15.43:0/2689976054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e07ce134d0 0x55e07d1f82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.396+0000 7faea567c640 1 -- 172.21.15.43:0/2689976054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce1b6b0 msgr2=0x55e07d2041c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.396+0000 7faea5e7d640 1 --2- 172.21.15.43:0/2689976054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce1b6b0 0x55e07d2041c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.396+0000 7faea567c640 1 --2- 172.21.15.43:0/2689976054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce1b6b0 0x55e07d2041c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.396+0000 7faea567c640 1 -- 172.21.15.43:0/2689976054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e07d1e7fe0 con 0x55e07cdfaf60 2024-04-25T10:40:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.396+0000 7faea567c640 1 --2- 172.21.15.43:0/2689976054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07cdfaf60 0x55e07d1ec070 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fae9c005fd0 tx=0x7fae9c00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687220cd17c815a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.396+0000 7fae8affd640 1 -- 172.21.15.43:0/2689976054 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c004220 con 0x55e07cdfaf60 2024-04-25T10:40:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.396+0000 7fae8affd640 1 -- 172.21.15.43:0/2689976054 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae9c0043c0 con 0x55e07cdfaf60 2024-04-25T10:40:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.396+0000 7fae8affd640 1 -- 172.21.15.43:0/2689976054 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c01db00 con 0x55e07cdfaf60 2024-04-25T10:40:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.397+0000 7faeae78a740 1 -- 172.21.15.43:0/2689976054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07cdfaf60 msgr2=0x55e07d1ec070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.397+0000 7faeae78a740 1 --2- 172.21.15.43:0/2689976054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07cdfaf60 0x55e07d1ec070 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fae9c005fd0 tx=0x7fae9c00b410 comp rx=0 tx=0).stop 2024-04-25T10:40:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.397+0000 7faeae78a740 1 -- 172.21.15.43:0/2689976054 shutdown_connections 2024-04-25T10:40:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.397+0000 7faeae78a740 1 --2- 172.21.15.43:0/2689976054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce1b6b0 0x55e07d2041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.397+0000 7faeae78a740 1 --2- 172.21.15.43:0/2689976054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e07ce134d0 0x55e07d1f82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.397+0000 7faeae78a740 1 --2- 172.21.15.43:0/2689976054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07cdfaf60 0x55e07d1ec070 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.397+0000 7faeae78a740 1 -- 172.21.15.43:0/2689976054 >> 172.21.15.43:0/2689976054 conn(0x55e07ce304e0 msgr2=0x55e07d1eb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:14.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.397+0000 7faeae78a740 1 -- 172.21.15.43:0/2689976054 shutdown_connections 2024-04-25T10:40:14.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.397+0000 7faeae78a740 1 -- 172.21.15.43:0/2689976054 wait complete. 2024-04-25T10:40:14.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.398+0000 7faeae78a740 1 -- start start 2024-04-25T10:40:14.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.398+0000 7faeae78a740 1 Processor -- start 2024-04-25T10:40:14.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.398+0000 7faeae78a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07cdfaf60 0x55e07d1f4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.398+0000 7faeae78a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce134d0 0x55e07d1efcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.398+0000 7faeae78a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e07ce1b6b0 0x55e07d1f0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.398+0000 7faeae78a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faea01abe10 con 0x55e07cdfaf60 2024-04-25T10:40:14.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.398+0000 7faeae78a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faea01ad280 con 0x55e07ce134d0 2024-04-25T10:40:14.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.398+0000 7faeae78a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faea01ac4e0 con 0x55e07ce1b6b0 2024-04-25T10:40:14.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.398+0000 7faeae78a740 10 client.0 _ll_register_callbacks cb 0x7fae90005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:14.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.398+0000 7faea567c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07cdfaf60 0x55e07d1f4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.398+0000 7faea567c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07cdfaf60 0x55e07d1f4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54154/0 (socket says 172.21.15.43:54154) 2024-04-25T10:40:14.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.398+0000 7faea567c640 1 -- 172.21.15.43:0/1911064906 learned_addr learned my addr 172.21.15.43:0/1911064906 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:14.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.398+0000 7faea5e7d640 1 --2- 172.21.15.43:0/1911064906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e07ce1b6b0 0x55e07d1f0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.398+0000 7faea4e7b640 1 --2- 172.21.15.43:0/1911064906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce134d0 0x55e07d1efcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.399+0000 7faea567c640 1 -- 172.21.15.43:0/1911064906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e07ce1b6b0 msgr2=0x55e07d1f0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.399+0000 7faea567c640 1 --2- 172.21.15.43:0/1911064906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e07ce1b6b0 0x55e07d1f0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.399+0000 7faea567c640 1 -- 172.21.15.43:0/1911064906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce134d0 msgr2=0x55e07d1efcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:14.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.399+0000 7faea567c640 1 --2- 172.21.15.43:0/1911064906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce134d0 0x55e07d1efcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.399+0000 7faea567c640 1 -- 172.21.15.43:0/1911064906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e07d1f0750 con 0x55e07cdfaf60 2024-04-25T10:40:14.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.399+0000 7faea4e7b640 1 --2- 172.21.15.43:0/1911064906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce134d0 0x55e07d1efcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:14.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.399+0000 7faea567c640 1 --2- 172.21.15.43:0/1911064906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07cdfaf60 0x55e07d1f4980 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fae9c01d720 tx=0x7fae9c004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde50ca139af6d39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:14.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.399+0000 7fae88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:14.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.399+0000 7fae88ff9640 1 -- 172.21.15.43:0/1911064906 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c005220 con 0x55e07cdfaf60 2024-04-25T10:40:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.399+0000 7fae88ff9640 1 -- 172.21.15.43:0/1911064906 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fae9c0053c0 con 0x55e07cdfaf60 2024-04-25T10:40:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.399+0000 7faeae78a740 1 -- 172.21.15.43:0/1911064906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae9c0043c0 con 0x55e07cdfaf60 2024-04-25T10:40:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.399+0000 7fae88ff9640 1 -- 172.21.15.43:0/1911064906 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fae9c0055c0 con 0x55e07cdfaf60 2024-04-25T10:40:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.399+0000 7faeae78a740 10 client.17814 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.399+0000 7faeae78a740 1 -- 172.21.15.43:0/1911064906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e07d2669e0 con 0x55e07cdfaf60 2024-04-25T10:40:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.399+0000 7fae88ff9640 1 -- 172.21.15.43:0/1911064906 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae9c0043c0 con 0x55e07cdfaf60 2024-04-25T10:40:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.400+0000 7fae88ff9640 1 -- 172.21.15.43:0/1911064906 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fae9c02f030 con 0x55e07cdfaf60 2024-04-25T10:40:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.400+0000 7faeae78a740 10 client.17814 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.400+0000 7faeae78a740 10 client.17814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.400+0000 7faeae78a740 20 client.17814 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.400+0000 7faeae78a740 1 --2- 172.21.15.43:0/1911064906 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e07ce3c390 0x55e07d269110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.400+0000 7faeae78a740 4 client.17814 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:40:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.400+0000 7faeae78a740 1 -- 172.21.15.43:0/1911064906 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e07d14a800 con 0x55e07ce3c390 2024-04-25T10:40:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.400+0000 7faea4e7b640 1 --2- 172.21.15.43:0/1911064906 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e07ce3c390 0x55e07d269110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.400+0000 7faea4e7b640 1 --2- 172.21.15.43:0/1911064906 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e07ce3c390 0x55e07d269110 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c66af21c14d649d server_cookie=38ec2eebd04911e8 in_seq=0 out_seq=0 2024-04-25T10:40:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.400+0000 7fae88ff9640 10 client.17814 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.400+0000 7fae88ff9640 1 -- 172.21.15.43:0/1911064906 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e07d14a800 con 0x55e07ce3c390 2024-04-25T10:40:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.400+0000 7fae88ff9640 10 client.17814 handle_command_reply: tid=0 2024-04-25T10:40:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.400+0000 7faeae78a740 1 client.17814 shutdown 2024-04-25T10:40:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faeae78a740 1 -- 172.21.15.43:0/1911064906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07cdfaf60 msgr2=0x55e07d1f4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faeae78a740 1 --2- 172.21.15.43:0/1911064906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07cdfaf60 0x55e07d1f4980 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fae9c01d720 tx=0x7fae9c004130 comp rx=0 tx=0).stop 2024-04-25T10:40:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faeae78a740 1 -- 172.21.15.43:0/1911064906 shutdown_connections 2024-04-25T10:40:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faeae78a740 1 --2- 172.21.15.43:0/1911064906 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e07ce3c390 0x55e07d269110 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faeae78a740 1 --2- 172.21.15.43:0/1911064906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e07ce134d0 0x55e07d1efcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faeae78a740 1 --2- 172.21.15.43:0/1911064906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e07ce1b6b0 0x55e07d1f0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faeae78a740 1 --2- 172.21.15.43:0/1911064906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e07cdfaf60 0x55e07d1f4980 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faeae78a740 1 -- 172.21.15.43:0/1911064906 >> 172.21.15.43:0/1911064906 conn(0x55e07ce304e0 msgr2=0x55e07d1eb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7fae88ff9640 0 client.17814 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:14.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faeae78a740 1 -- 172.21.15.43:0/1911064906 shutdown_connections 2024-04-25T10:40:14.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faeae78a740 1 -- 172.21.15.43:0/1911064906 wait complete. 2024-04-25T10:40:14.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faeae78a740 20 client.17814 trim_cache size 0 max 16384 2024-04-25T10:40:14.410 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:40:14.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faea78e0640 1 -- 172.21.15.43:0/1352802600 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fae8c0afc60 msgr2=0x7fae8c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faea78e0640 1 --2- 172.21.15.43:0/1352802600 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fae8c0afc60 0x7fae8c0b2170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fae98022000 tx=0x7fae98023000 comp rx=0 tx=0).stop 2024-04-25T10:40:14.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faea78e0640 1 -- 172.21.15.43:0/1352802600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faea0108a20 msgr2=0x7faea019ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:14.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faea78e0640 1 --2- 172.21.15.43:0/1352802600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faea0108a20 0x7faea019ac60 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fae90006010 tx=0x7fae90004060 comp rx=0 tx=0).stop 2024-04-25T10:40:14.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faea78e0640 1 -- 172.21.15.43:0/1352802600 shutdown_connections 2024-04-25T10:40:14.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faea78e0640 1 --2- 172.21.15.43:0/1352802600 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fae8c0afc60 0x7fae8c0b2170 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faea78e0640 1 --2- 172.21.15.43:0/1352802600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faea010d380 0x7faea019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faea78e0640 1 --2- 172.21.15.43:0/1352802600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faea0108a20 0x7faea019ac60 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faea78e0640 1 --2- 172.21.15.43:0/1352802600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faea0102af0 0x7faea019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:14.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faea78e0640 1 -- 172.21.15.43:0/1352802600 >> 172.21.15.43:0/1352802600 conn(0x7faea0078100 msgr2=0x7faea01040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:14.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faea78e0640 1 -- 172.21.15.43:0/1352802600 shutdown_connections 2024-04-25T10:40:14.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:14.401+0000 7faea78e0640 1 -- 172.21.15.43:0/1352802600 wait complete. 2024-04-25T10:40:15.418 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:40:15.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:40:15.418 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:40:15.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.547+0000 7fb924b7d640 1 -- 172.21.15.43:0/3921042317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9201031c0 msgr2=0x7fb92010f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:15.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.547+0000 7fb924b7d640 1 --2- 172.21.15.43:0/3921042317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9201031c0 0x7fb92010f4a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fb918009ca0 tx=0x7fb91802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.547+0000 7fb924b7d640 1 -- 172.21.15.43:0/3921042317 shutdown_connections 2024-04-25T10:40:15.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.547+0000 7fb924b7d640 1 --2- 172.21.15.43:0/3921042317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb920108eb0 0x7fb920102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.547+0000 7fb924b7d640 1 --2- 172.21.15.43:0/3921042317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9201031c0 0x7fb92010f4a0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.547+0000 7fb924b7d640 1 --2- 172.21.15.43:0/3921042317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb920108a10 0x7fb920102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.547+0000 7fb924b7d640 1 -- 172.21.15.43:0/3921042317 >> 172.21.15.43:0/3921042317 conn(0x7fb920078110 msgr2=0x7fb920078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:15.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.547+0000 7fb924b7d640 1 -- 172.21.15.43:0/3921042317 shutdown_connections 2024-04-25T10:40:15.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.547+0000 7fb924b7d640 1 -- 172.21.15.43:0/3921042317 wait complete. 2024-04-25T10:40:15.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.548+0000 7fb924b7d640 1 Processor -- start 2024-04-25T10:40:15.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.548+0000 7fb924b7d640 1 -- start start 2024-04-25T10:40:15.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.548+0000 7fb924b7d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9201031c0 0x7fb92019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb924b7d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb920108a10 0x7fb92019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb924b7d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb920108eb0 0x7fb92019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb924b7d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb920112c50 con 0x7fb9201031c0 2024-04-25T10:40:15.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb924b7d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb920112ad0 con 0x7fb920108a10 2024-04-25T10:40:15.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb924b7d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb920112dd0 con 0x7fb920108eb0 2024-04-25T10:40:15.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb91e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9201031c0 0x7fb92019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb91e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9201031c0 0x7fb92019a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54176/0 (socket says 172.21.15.43:54176) 2024-04-25T10:40:15.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb91e59c640 1 -- 172.21.15.43:0/3459798536 learned_addr learned my addr 172.21.15.43:0/3459798536 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:15.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb91dd9b640 1 --2- 172.21.15.43:0/3459798536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb920108a10 0x7fb92019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb91ed9d640 1 --2- 172.21.15.43:0/3459798536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb920108eb0 0x7fb92019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb91e59c640 1 -- 172.21.15.43:0/3459798536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb920108eb0 msgr2=0x7fb92019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:15.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb91e59c640 1 --2- 172.21.15.43:0/3459798536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb920108eb0 0x7fb92019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb91e59c640 1 -- 172.21.15.43:0/3459798536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb920108a10 msgr2=0x7fb92019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:15.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb91e59c640 1 --2- 172.21.15.43:0/3459798536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb920108a10 0x7fb92019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb91e59c640 1 -- 172.21.15.43:0/3459798536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92019f580 con 0x7fb9201031c0 2024-04-25T10:40:15.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb91ed9d640 1 --2- 172.21.15.43:0/3459798536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb920108eb0 0x7fb92019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:15.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb91dd9b640 1 --2- 172.21.15.43:0/3459798536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb920108a10 0x7fb92019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:15.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb91e59c640 1 --2- 172.21.15.43:0/3459798536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9201031c0 0x7fb92019a720 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fb91000d750 tx=0x7fb91000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:15.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb90f7fe640 1 -- 172.21.15.43:0/3459798536 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb910017070 con 0x7fb9201031c0 2024-04-25T10:40:15.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb90f7fe640 1 -- 172.21.15.43:0/3459798536 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9100054d0 con 0x7fb9201031c0 2024-04-25T10:40:15.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb924b7d640 1 -- 172.21.15.43:0/3459798536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92019f870 con 0x7fb9201031c0 2024-04-25T10:40:15.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.549+0000 7fb90f7fe640 1 -- 172.21.15.43:0/3459798536 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb910004c30 con 0x7fb9201031c0 2024-04-25T10:40:15.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.550+0000 7fb924b7d640 1 -- 172.21.15.43:0/3459798536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9201a71f0 con 0x7fb9201031c0 2024-04-25T10:40:15.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.552+0000 7fb92ba2b740 1 Processor -- start 2024-04-25T10:40:15.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.553+0000 7fb92ba2b740 1 -- start start 2024-04-25T10:40:15.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.553+0000 7fb90f7fe640 1 -- 172.21.15.43:0/3459798536 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fb910010460 con 0x7fb9201031c0 2024-04-25T10:40:15.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.553+0000 7fb92ba2b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5e2410 0x56468d982620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.553+0000 7fb92ba2b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56468d5b9550 0x56468d982b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.553+0000 7fb92ba2b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5c1730 0x56468d9830a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.553+0000 7fb92ba2b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56468d8e8b00 con 0x56468d5e2410 2024-04-25T10:40:15.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.553+0000 7fb92ba2b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56468d884920 con 0x56468d5c1730 2024-04-25T10:40:15.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.553+0000 7fb92ba2b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56468d8e4ac0 con 0x56468d5b9550 2024-04-25T10:40:15.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.553+0000 7fb91ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5e2410 0x56468d982620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.553+0000 7fb91ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5e2410 0x56468d982620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54178/0 (socket says 172.21.15.43:54178) 2024-04-25T10:40:15.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.553+0000 7fb91ed9d640 1 -- 172.21.15.43:0/4188080048 learned_addr learned my addr 172.21.15.43:0/4188080048 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:15.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.553+0000 7fb91dd9b640 1 --2- 172.21.15.43:0/4188080048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5c1730 0x56468d9830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.553+0000 7fb91e59c640 1 --2- 172.21.15.43:0/4188080048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56468d5b9550 0x56468d982b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.553+0000 7fb91ed9d640 1 -- 172.21.15.43:0/4188080048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56468d5b9550 msgr2=0x56468d982b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:15.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.553+0000 7fb91ed9d640 1 --2- 172.21.15.43:0/4188080048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56468d5b9550 0x56468d982b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.553+0000 7fb91ed9d640 1 -- 172.21.15.43:0/4188080048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5c1730 msgr2=0x56468d9830a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:15.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.553+0000 7fb91ed9d640 1 --2- 172.21.15.43:0/4188080048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5c1730 0x56468d9830a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.553+0000 7fb91ed9d640 1 -- 172.21.15.43:0/4188080048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468d6c27e0 con 0x56468d5e2410 2024-04-25T10:40:15.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.553+0000 7fb91dd9b640 1 --2- 172.21.15.43:0/4188080048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5c1730 0x56468d9830a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:15.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.554+0000 7fb90f7fe640 1 --2- 172.21.15.43:0/3459798536 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb9080afce0 0x7fb9080b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.554+0000 7fb91ed9d640 1 --2- 172.21.15.43:0/4188080048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5e2410 0x56468d982620 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fb918009830 tx=0x7fb918009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ecef7d9f0c36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:15.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.554+0000 7fb91e59c640 1 --2- 172.21.15.43:0/3459798536 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb9080afce0 0x7fb9080b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.554+0000 7fb90f7fe640 1 -- 172.21.15.43:0/3459798536 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fb9100dea70 con 0x7fb9201031c0 2024-04-25T10:40:15.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.554+0000 7fb90cff9640 1 -- 172.21.15.43:0/4188080048 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918031e30 con 0x56468d5e2410 2024-04-25T10:40:15.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.554+0000 7fb90cff9640 1 -- 172.21.15.43:0/4188080048 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb918036040 con 0x56468d5e2410 2024-04-25T10:40:15.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.554+0000 7fb90cff9640 1 -- 172.21.15.43:0/4188080048 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918004280 con 0x56468d5e2410 2024-04-25T10:40:15.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.555+0000 7fb92ba2b740 1 -- 172.21.15.43:0/4188080048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5e2410 msgr2=0x56468d982620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:15.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.555+0000 7fb92ba2b740 1 --2- 172.21.15.43:0/4188080048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5e2410 0x56468d982620 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fb918009830 tx=0x7fb918009860 comp rx=0 tx=0).stop 2024-04-25T10:40:15.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.555+0000 7fb91e59c640 1 --2- 172.21.15.43:0/3459798536 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb9080afce0 0x7fb9080b21f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb9100dc270 tx=0x7fb910009f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:15.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.555+0000 7fb92ba2b740 1 -- 172.21.15.43:0/4188080048 shutdown_connections 2024-04-25T10:40:15.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.555+0000 7fb92ba2b740 1 --2- 172.21.15.43:0/4188080048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5c1730 0x56468d9830a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.555+0000 7fb92ba2b740 1 --2- 172.21.15.43:0/4188080048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56468d5b9550 0x56468d982b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.555+0000 7fb92ba2b740 1 --2- 172.21.15.43:0/4188080048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5e2410 0x56468d982620 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.555+0000 7fb92ba2b740 1 -- 172.21.15.43:0/4188080048 >> 172.21.15.43:0/4188080048 conn(0x56468d5d6560 msgr2=0x56468d97edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:15.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.555+0000 7fb92ba2b740 1 -- 172.21.15.43:0/4188080048 shutdown_connections 2024-04-25T10:40:15.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.555+0000 7fb92ba2b740 1 -- 172.21.15.43:0/4188080048 wait complete. 2024-04-25T10:40:15.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.555+0000 7fb92ba2b740 1 -- start start 2024-04-25T10:40:15.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.555+0000 7fb92ba2b740 1 Processor -- start 2024-04-25T10:40:15.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.556+0000 7fb92ba2b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5b9550 0x56468d98cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.556+0000 7fb92ba2b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5c1730 0x56468d98d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.556+0000 7fb92ba2b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56468d5e2410 0x56468d998dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.556+0000 7fb92ba2b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56468d98f910 con 0x56468d5b9550 2024-04-25T10:40:15.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.556+0000 7fb92ba2b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56468d8825d0 con 0x56468d5c1730 2024-04-25T10:40:15.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.556+0000 7fb92ba2b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56468d98fa90 con 0x56468d5e2410 2024-04-25T10:40:15.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.556+0000 7fb92ba2b740 10 client.0 _ll_register_callbacks cb 0x56468d870080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:15.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.556+0000 7fb91ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5b9550 0x56468d98cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.556+0000 7fb91ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5b9550 0x56468d98cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54186/0 (socket says 172.21.15.43:54186) 2024-04-25T10:40:15.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.556+0000 7fb91ed9d640 1 -- 172.21.15.43:0/1083651193 learned_addr learned my addr 172.21.15.43:0/1083651193 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:15.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.556+0000 7fb91dd9b640 1 --2- 172.21.15.43:0/1083651193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5c1730 0x56468d98d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.556+0000 7fb91ed9d640 1 --2- 172.21.15.43:0/1083651193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56468d5e2410 0x56468d998dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.556+0000 7fb91ed9d640 1 -- 172.21.15.43:0/1083651193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56468d5e2410 msgr2=0x56468d998dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:15.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.556+0000 7fb91ed9d640 1 --2- 172.21.15.43:0/1083651193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56468d5e2410 0x56468d998dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.556+0000 7fb91ed9d640 1 -- 172.21.15.43:0/1083651193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5c1730 msgr2=0x56468d98d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:15.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.556+0000 7fb91ed9d640 1 --2- 172.21.15.43:0/1083651193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5c1730 0x56468d98d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.556+0000 7fb91ed9d640 1 -- 172.21.15.43:0/1083651193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468d722f70 con 0x56468d5b9550 2024-04-25T10:40:15.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.556+0000 7fb91ed9d640 1 --2- 172.21.15.43:0/1083651193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5b9550 0x56468d98cc00 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fb91802a900 tx=0x7fb918003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be1bd1c7307942c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:15.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.556+0000 7fb905ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:15.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.557+0000 7fb905ffb640 1 -- 172.21.15.43:0/1083651193 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918058060 con 0x56468d5b9550 2024-04-25T10:40:15.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.557+0000 7fb905ffb640 1 -- 172.21.15.43:0/1083651193 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fb918031450 con 0x56468d5b9550 2024-04-25T10:40:15.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.557+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1083651193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb918036040 con 0x56468d5b9550 2024-04-25T10:40:15.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.557+0000 7fb905ffb640 1 -- 172.21.15.43:0/1083651193 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb918031650 con 0x56468d5b9550 2024-04-25T10:40:15.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.557+0000 7fb92ba2b740 10 client.17826 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:15.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.557+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1083651193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56468d5ba170 con 0x56468d5b9550 2024-04-25T10:40:15.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.557+0000 7fb905ffb640 1 -- 172.21.15.43:0/1083651193 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb918036040 con 0x56468d5b9550 2024-04-25T10:40:15.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.557+0000 7fb905ffb640 1 -- 172.21.15.43:0/1083651193 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb918003fc0 con 0x56468d5b9550 2024-04-25T10:40:15.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.557+0000 7fb92ba2b740 10 client.17826 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:15.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.557+0000 7fb92ba2b740 10 client.17826 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:15.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.557+0000 7fb92ba2b740 20 client.17826 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:15.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.557+0000 7fb92ba2b740 1 --2- 172.21.15.43:0/1083651193 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56468d5a11f0 0x56468d99ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.557+0000 7fb92ba2b740 4 client.17826 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:15.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.557+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1083651193 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56468d8f6710 con 0x56468d5a11f0 2024-04-25T10:40:15.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.558+0000 7fb91dd9b640 1 --2- 172.21.15.43:0/1083651193 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56468d5a11f0 0x56468d99ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.558+0000 7fb905ffb640 10 client.17826 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:15.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.558+0000 7fb91dd9b640 1 --2- 172.21.15.43:0/1083651193 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56468d5a11f0 0x56468d99ee00 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1438acded8e918 server_cookie=c628d499ac9b5c9f in_seq=0 out_seq=0 2024-04-25T10:40:15.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.559+0000 7fb905ffb640 1 -- 172.21.15.43:0/1083651193 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56468d8f6710 con 0x56468d5a11f0 2024-04-25T10:40:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.559+0000 7fb905ffb640 10 client.17826 handle_command_reply: tid=0 2024-04-25T10:40:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.559+0000 7fb92ba2b740 1 client.17826 shutdown 2024-04-25T10:40:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.559+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1083651193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5b9550 msgr2=0x56468d98cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.559+0000 7fb92ba2b740 1 --2- 172.21.15.43:0/1083651193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5b9550 0x56468d98cc00 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fb91802a900 tx=0x7fb918003ee0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.559+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1083651193 shutdown_connections 2024-04-25T10:40:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.559+0000 7fb92ba2b740 1 --2- 172.21.15.43:0/1083651193 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56468d5a11f0 0x56468d99ee00 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.559+0000 7fb92ba2b740 1 --2- 172.21.15.43:0/1083651193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5c1730 0x56468d98d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.559+0000 7fb92ba2b740 1 --2- 172.21.15.43:0/1083651193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56468d5e2410 0x56468d998dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.559+0000 7fb905ffb640 0 client.17826 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.559+0000 7fb92ba2b740 1 --2- 172.21.15.43:0/1083651193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5b9550 0x56468d98cc00 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.559+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1083651193 >> 172.21.15.43:0/1083651193 conn(0x56468d5d6560 msgr2=0x56468d982450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.559+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1083651193 shutdown_connections 2024-04-25T10:40:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.559+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1083651193 wait complete. 2024-04-25T10:40:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.559+0000 7fb92ba2b740 20 client.17826 trim_cache size 0 max 16384 2024-04-25T10:40:15.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.578+0000 7fb92ba2b740 1 Processor -- start 2024-04-25T10:40:15.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.578+0000 7fb92ba2b740 1 -- start start 2024-04-25T10:40:15.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.578+0000 7fb92ba2b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56468d5d6560 0x56468d98eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.578+0000 7fb92ba2b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5b9550 0x56468d982ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.578+0000 7fb92ba2b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5c1730 0x56468d983000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.578+0000 7fb92ba2b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb9201b2270 con 0x56468d5c1730 2024-04-25T10:40:15.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.578+0000 7fb92ba2b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb9201afb40 con 0x56468d5b9550 2024-04-25T10:40:15.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.578+0000 7fb92ba2b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb9201ad9b0 con 0x56468d5d6560 2024-04-25T10:40:15.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.578+0000 7fb91ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5c1730 0x56468d983000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.578+0000 7fb91ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5c1730 0x56468d983000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54194/0 (socket says 172.21.15.43:54194) 2024-04-25T10:40:15.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.578+0000 7fb91ed9d640 1 -- 172.21.15.43:0/1774745787 learned_addr learned my addr 172.21.15.43:0/1774745787 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:15.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.578+0000 7fb91ed9d640 1 -- 172.21.15.43:0/1774745787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56468d5d6560 msgr2=0x56468d98eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:15.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.578+0000 7fb91ed9d640 1 --2- 172.21.15.43:0/1774745787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56468d5d6560 0x56468d98eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.578+0000 7fb91ed9d640 1 -- 172.21.15.43:0/1774745787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5b9550 msgr2=0x56468d982ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:15.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.578+0000 7fb91dd9b640 1 --2- 172.21.15.43:0/1774745787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5b9550 0x56468d982ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.578+0000 7fb91ed9d640 1 --2- 172.21.15.43:0/1774745787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5b9550 0x56468d982ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.578+0000 7fb91ed9d640 1 -- 172.21.15.43:0/1774745787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468d97e300 con 0x56468d5c1730 2024-04-25T10:40:15.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.578+0000 7fb91ed9d640 1 --2- 172.21.15.43:0/1774745787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5c1730 0x56468d983000 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fb9180049d0 tx=0x7fb918009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98bc2a75f0211052 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:15.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.579+0000 7fb907fff640 1 -- 172.21.15.43:0/1774745787 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918058030 con 0x56468d5c1730 2024-04-25T10:40:15.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.579+0000 7fb907fff640 1 -- 172.21.15.43:0/1774745787 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb918036070 con 0x56468d5c1730 2024-04-25T10:40:15.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.579+0000 7fb907fff640 1 -- 172.21.15.43:0/1774745787 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9180047c0 con 0x56468d5c1730 2024-04-25T10:40:15.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.579+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1774745787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5c1730 msgr2=0x56468d983000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:15.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.579+0000 7fb92ba2b740 1 --2- 172.21.15.43:0/1774745787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5c1730 0x56468d983000 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fb9180049d0 tx=0x7fb918009560 comp rx=0 tx=0).stop 2024-04-25T10:40:15.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.579+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1774745787 shutdown_connections 2024-04-25T10:40:15.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.579+0000 7fb92ba2b740 1 --2- 172.21.15.43:0/1774745787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5b9550 0x56468d982ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.579+0000 7fb92ba2b740 1 --2- 172.21.15.43:0/1774745787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5c1730 0x56468d983000 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.579+0000 7fb92ba2b740 1 --2- 172.21.15.43:0/1774745787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56468d5d6560 0x56468d98eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.579+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1774745787 >> 172.21.15.43:0/1774745787 conn(0x56468d97f020 msgr2=0x56468d9823b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:15.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.580+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1774745787 shutdown_connections 2024-04-25T10:40:15.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.580+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1774745787 wait complete. 2024-04-25T10:40:15.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.580+0000 7fb92ba2b740 1 -- start start 2024-04-25T10:40:15.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.580+0000 7fb92ba2b740 1 Processor -- start 2024-04-25T10:40:15.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.580+0000 7fb92ba2b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56468d5b9550 0x56468d9a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.580+0000 7fb92ba2b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5c1730 0x56468d99f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.580+0000 7fb92ba2b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5d6560 0x56468d99fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.580+0000 7fb92ba2b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb9201ae000 con 0x56468d5c1730 2024-04-25T10:40:15.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.580+0000 7fb92ba2b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb9201af470 con 0x56468d5d6560 2024-04-25T10:40:15.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.580+0000 7fb92ba2b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb9201ae6d0 con 0x56468d5b9550 2024-04-25T10:40:15.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.580+0000 7fb92ba2b740 10 client.0 _ll_register_callbacks cb 0x7fb918031d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:15.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.580+0000 7fb91dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5c1730 0x56468d99f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.580+0000 7fb91dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5c1730 0x56468d99f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54202/0 (socket says 172.21.15.43:54202) 2024-04-25T10:40:15.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.580+0000 7fb91dd9b640 1 -- 172.21.15.43:0/1480050387 learned_addr learned my addr 172.21.15.43:0/1480050387 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:15.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.581+0000 7fb91ed9d640 1 --2- 172.21.15.43:0/1480050387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5d6560 0x56468d99fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.581+0000 7fb91dd9b640 1 -- 172.21.15.43:0/1480050387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56468d5b9550 msgr2=0x56468d9a21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:15.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.581+0000 7fb91ed9d640 1 --2- 172.21.15.43:0/1480050387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56468d5b9550 0x56468d9a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.581+0000 7fb91dd9b640 1 --2- 172.21.15.43:0/1480050387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56468d5b9550 0x56468d9a21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.581+0000 7fb91dd9b640 1 -- 172.21.15.43:0/1480050387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5d6560 msgr2=0x56468d99fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:15.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.581+0000 7fb91dd9b640 1 --2- 172.21.15.43:0/1480050387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5d6560 0x56468d99fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.581+0000 7fb91dd9b640 1 -- 172.21.15.43:0/1480050387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468d99ffa0 con 0x56468d5c1730 2024-04-25T10:40:15.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.581+0000 7fb91ed9d640 1 --2- 172.21.15.43:0/1480050387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56468d5b9550 0x56468d9a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:15.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.581+0000 7fb91dd9b640 1 --2- 172.21.15.43:0/1480050387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5c1730 0x56468d99f520 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fb9201092e0 tx=0x7fb914012a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82206a2c33b5c77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:15.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.581+0000 7fb905ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:15.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.581+0000 7fb905ffb640 1 -- 172.21.15.43:0/1480050387 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914004220 con 0x56468d5c1730 2024-04-25T10:40:15.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.581+0000 7fb905ffb640 1 -- 172.21.15.43:0/1480050387 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fb9140043c0 con 0x56468d5c1730 2024-04-25T10:40:15.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.581+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1480050387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb918036070 con 0x56468d5c1730 2024-04-25T10:40:15.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.581+0000 7fb905ffb640 1 -- 172.21.15.43:0/1480050387 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb91402c050 con 0x56468d5c1730 2024-04-25T10:40:15.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.581+0000 7fb92ba2b740 10 client.17832 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:15.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.581+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1480050387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56468da06cc0 con 0x56468d5c1730 2024-04-25T10:40:15.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.581+0000 7fb905ffb640 1 -- 172.21.15.43:0/1480050387 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb918036070 con 0x56468d5c1730 2024-04-25T10:40:15.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.582+0000 7fb905ffb640 1 -- 172.21.15.43:0/1480050387 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb914003a70 con 0x56468d5c1730 2024-04-25T10:40:15.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.582+0000 7fb92ba2b740 10 client.17832 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:15.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.582+0000 7fb92ba2b740 10 client.17832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:15.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.582+0000 7fb92ba2b740 20 client.17832 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:15.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.582+0000 7fb92ba2b740 1 --2- 172.21.15.43:0/1480050387 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56468d5e2410 0x56468da09480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.582+0000 7fb92ba2b740 4 client.17832 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:40:15.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.582+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1480050387 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56468d8f0580 con 0x56468d5e2410 2024-04-25T10:40:15.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.582+0000 7fb91ed9d640 1 --2- 172.21.15.43:0/1480050387 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56468d5e2410 0x56468da09480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.582+0000 7fb91ed9d640 1 --2- 172.21.15.43:0/1480050387 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56468d5e2410 0x56468da09480 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=444c6e1bda71331a server_cookie=2a7f31c7df3e6ab4 in_seq=0 out_seq=0 2024-04-25T10:40:15.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.582+0000 7fb905ffb640 10 client.17832 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:15.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.584+0000 7fb905ffb640 1 -- 172.21.15.43:0/1480050387 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56468d8f0580 con 0x56468d5e2410 2024-04-25T10:40:15.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.584+0000 7fb905ffb640 10 client.17832 handle_command_reply: tid=0 2024-04-25T10:40:15.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.584+0000 7fb92ba2b740 1 client.17832 shutdown 2024-04-25T10:40:15.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.584+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1480050387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5c1730 msgr2=0x56468d99f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:15.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.584+0000 7fb92ba2b740 1 --2- 172.21.15.43:0/1480050387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5c1730 0x56468d99f520 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fb9201092e0 tx=0x7fb914012a40 comp rx=0 tx=0).stop 2024-04-25T10:40:15.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.584+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1480050387 shutdown_connections 2024-04-25T10:40:15.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.584+0000 7fb92ba2b740 1 --2- 172.21.15.43:0/1480050387 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56468d5e2410 0x56468da09480 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.584+0000 7fb92ba2b740 1 --2- 172.21.15.43:0/1480050387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56468d5d6560 0x56468d99fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.584+0000 7fb92ba2b740 1 --2- 172.21.15.43:0/1480050387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468d5c1730 0x56468d99f520 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.584+0000 7fb905ffb640 0 client.17832 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:15.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.584+0000 7fb92ba2b740 1 --2- 172.21.15.43:0/1480050387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56468d5b9550 0x56468d9a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.584+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1480050387 >> 172.21.15.43:0/1480050387 conn(0x56468d97f020 msgr2=0x56468d982de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:15.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.585+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1480050387 shutdown_connections 2024-04-25T10:40:15.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.585+0000 7fb92ba2b740 1 -- 172.21.15.43:0/1480050387 wait complete. 2024-04-25T10:40:15.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.585+0000 7fb92ba2b740 20 client.17832 trim_cache size 0 max 16384 2024-04-25T10:40:15.591 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:40:15.591 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:40:15.591 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "6a1e46be-72c9-49fe-9c3e-eb03718cb018", 2024-04-25T10:40:15.591 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:40:15.592 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:40:15.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.585+0000 7fb924b7d640 1 -- 172.21.15.43:0/3459798536 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb9080afce0 msgr2=0x7fb9080b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:15.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.585+0000 7fb924b7d640 1 --2- 172.21.15.43:0/3459798536 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb9080afce0 0x7fb9080b21f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb9100dc270 tx=0x7fb910009f90 comp rx=0 tx=0).stop 2024-04-25T10:40:15.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.585+0000 7fb924b7d640 1 -- 172.21.15.43:0/3459798536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9201031c0 msgr2=0x7fb92019a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:15.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.585+0000 7fb924b7d640 1 --2- 172.21.15.43:0/3459798536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9201031c0 0x7fb92019a720 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fb91000d750 tx=0x7fb91000dc10 comp rx=0 tx=0).stop 2024-04-25T10:40:15.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.585+0000 7fb924b7d640 1 -- 172.21.15.43:0/3459798536 shutdown_connections 2024-04-25T10:40:15.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.585+0000 7fb924b7d640 1 --2- 172.21.15.43:0/3459798536 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb9080afce0 0x7fb9080b21f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.585+0000 7fb924b7d640 1 --2- 172.21.15.43:0/3459798536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb920108a10 0x7fb92019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.585+0000 7fb924b7d640 1 --2- 172.21.15.43:0/3459798536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb920108eb0 0x7fb92019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.585+0000 7fb924b7d640 1 --2- 172.21.15.43:0/3459798536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9201031c0 0x7fb92019a720 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.585+0000 7fb924b7d640 1 -- 172.21.15.43:0/3459798536 >> 172.21.15.43:0/3459798536 conn(0x7fb920078110 msgr2=0x7fb920106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:15.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.585+0000 7fb924b7d640 1 -- 172.21.15.43:0/3459798536 shutdown_connections 2024-04-25T10:40:15.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.585+0000 7fb924b7d640 1 -- 172.21.15.43:0/3459798536 wait complete. 2024-04-25T10:40:15.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:40:15.598 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:40:15.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.710+0000 7f9f78eb6640 1 -- 172.21.15.43:0/1151431802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f74104260 msgr2=0x7f9f74110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:15.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.710+0000 7f9f78eb6640 1 --2- 172.21.15.43:0/1151431802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f74104260 0x7f9f74110630 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009ca0 tx=0x7f9f6c02a8e0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.711+0000 7f9f78eb6640 1 -- 172.21.15.43:0/1151431802 shutdown_connections 2024-04-25T10:40:15.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.711+0000 7f9f78eb6640 1 --2- 172.21.15.43:0/1151431802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f74104260 0x7f9f74110630 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.711+0000 7f9f78eb6640 1 --2- 172.21.15.43:0/1151431802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f74109fb0 0x7f9f74103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.711+0000 7f9f78eb6640 1 --2- 172.21.15.43:0/1151431802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f74109b10 0x7f9f741036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.711+0000 7f9f78eb6640 1 -- 172.21.15.43:0/1151431802 >> 172.21.15.43:0/1151431802 conn(0x7f9f740ff230 msgr2=0x7f9f74101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:15.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.711+0000 7f9f78eb6640 1 -- 172.21.15.43:0/1151431802 shutdown_connections 2024-04-25T10:40:15.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.711+0000 7f9f78eb6640 1 -- 172.21.15.43:0/1151431802 wait complete. 2024-04-25T10:40:15.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.711+0000 7f9f78eb6640 1 Processor -- start 2024-04-25T10:40:15.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.711+0000 7f9f78eb6640 1 -- start start 2024-04-25T10:40:15.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.712+0000 7f9f78eb6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f74104260 0x7f9f7419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.712+0000 7f9f78eb6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f74109b10 0x7f9f7419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.712+0000 7f9f78eb6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f74109fb0 0x7f9f7419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.712+0000 7f9f78eb6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9f74113d80 con 0x7f9f74104260 2024-04-25T10:40:15.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.712+0000 7f9f78eb6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9f74113c00 con 0x7f9f74109b10 2024-04-25T10:40:15.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.712+0000 7f9f78eb6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f74113f00 con 0x7f9f74109fb0 2024-04-25T10:40:15.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.712+0000 7f9f7259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f74104260 0x7f9f7419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.712+0000 7f9f7259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f74104260 0x7f9f7419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54220/0 (socket says 172.21.15.43:54220) 2024-04-25T10:40:15.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.712+0000 7f9f7259c640 1 -- 172.21.15.43:0/2533524936 learned_addr learned my addr 172.21.15.43:0/2533524936 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:15.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.712+0000 7f9f71d9b640 1 --2- 172.21.15.43:0/2533524936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f74109b10 0x7f9f7419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.712+0000 7f9f7259c640 1 -- 172.21.15.43:0/2533524936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f74109fb0 msgr2=0x7f9f7419ef80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:40:15.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.712+0000 7f9f7259c640 1 --2- 172.21.15.43:0/2533524936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f74109fb0 0x7f9f7419ef80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.712+0000 7f9f7259c640 1 -- 172.21.15.43:0/2533524936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f74109b10 msgr2=0x7f9f7419acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:15.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.712+0000 7f9f7259c640 1 --2- 172.21.15.43:0/2533524936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f74109b10 0x7f9f7419acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.712+0000 7f9f7259c640 1 -- 172.21.15.43:0/2533524936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7419f5c0 con 0x7f9f74104260 2024-04-25T10:40:15.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.712+0000 7f9f7259c640 1 --2- 172.21.15.43:0/2533524936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f74104260 0x7f9f7419a760 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f9f64005fd0 tx=0x7f9f6400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:15.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.712+0000 7f9f637fe640 1 -- 172.21.15.43:0/2533524936 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f64017070 con 0x7f9f74104260 2024-04-25T10:40:15.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.713+0000 7f9f637fe640 1 -- 172.21.15.43:0/2533524936 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f64009c30 con 0x7f9f74104260 2024-04-25T10:40:15.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.713+0000 7f9f78eb6640 1 -- 172.21.15.43:0/2533524936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f7419f8b0 con 0x7f9f74104260 2024-04-25T10:40:15.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.713+0000 7f9f637fe640 1 -- 172.21.15.43:0/2533524936 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f64003d30 con 0x7f9f74104260 2024-04-25T10:40:15.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.713+0000 7f9f78eb6640 1 -- 172.21.15.43:0/2533524936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f741a7230 con 0x7f9f74104260 2024-04-25T10:40:15.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.716+0000 7f9f7fd6e740 1 Processor -- start 2024-04-25T10:40:15.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.716+0000 7f9f7fd6e740 1 -- start start 2024-04-25T10:40:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.716+0000 7f9f7fd6e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8e7390 0x55593cc87a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.716+0000 7f9f7fd6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8be4d0 0x55593cc87fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.716+0000 7f9f7fd6e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8c66b0 0x55593cc948f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.716+0000 7f9f7fd6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55593cbedd80 con 0x55593c8be4d0 2024-04-25T10:40:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.716+0000 7f9f7fd6e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55593cba78b0 con 0x55593c8e7390 2024-04-25T10:40:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.716+0000 7f9f7fd6e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55593cbe9d40 con 0x55593c8c66b0 2024-04-25T10:40:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.716+0000 7f9f637fe640 1 -- 172.21.15.43:0/2533524936 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f9f64010460 con 0x7f9f74104260 2024-04-25T10:40:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.716+0000 7f9f7259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8be4d0 0x55593cc87fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.717+0000 7f9f7259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8be4d0 0x55593cc87fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54224/0 (socket says 172.21.15.43:54224) 2024-04-25T10:40:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.717+0000 7f9f71d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8c66b0 0x55593cc948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.717+0000 7f9f72d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8e7390 0x55593cc87a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.717+0000 7f9f7259c640 1 -- 172.21.15.43:0/3598823044 learned_addr learned my addr 172.21.15.43:0/3598823044 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.717+0000 7f9f7259c640 1 -- 172.21.15.43:0/3598823044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8c66b0 msgr2=0x55593cc948f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.717+0000 7f9f7259c640 1 --2- 172.21.15.43:0/3598823044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8c66b0 0x55593cc948f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.717+0000 7f9f7259c640 1 -- 172.21.15.43:0/3598823044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8e7390 msgr2=0x55593cc87a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.717+0000 7f9f7259c640 1 --2- 172.21.15.43:0/3598823044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8e7390 0x55593cc87a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.717+0000 7f9f7259c640 1 -- 172.21.15.43:0/3598823044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55593cb78a40 con 0x55593c8be4d0 2024-04-25T10:40:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.717+0000 7f9f637fe640 1 --2- 172.21.15.43:0/2533524936 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9f5c0afc60 0x7f9f5c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.717+0000 7f9f72d9d640 1 --2- 172.21.15.43:0/2533524936 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9f5c0afc60 0x7f9f5c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.717+0000 7f9f7259c640 1 --2- 172.21.15.43:0/3598823044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8be4d0 0x55593cc87fa0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f9f640dc390 tx=0x7f9f640e7810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb94ce5a9ecbc49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.717+0000 7f9f637fe640 1 -- 172.21.15.43:0/2533524936 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f9f640decd0 con 0x7f9f74104260 2024-04-25T10:40:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.717+0000 7f9f60ff9640 1 -- 172.21.15.43:0/3598823044 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f640a2d30 con 0x55593c8be4d0 2024-04-25T10:40:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.717+0000 7f9f60ff9640 1 -- 172.21.15.43:0/3598823044 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f640e0070 con 0x55593c8be4d0 2024-04-25T10:40:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.717+0000 7f9f60ff9640 1 -- 172.21.15.43:0/3598823044 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f640a65e0 con 0x55593c8be4d0 2024-04-25T10:40:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.718+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/3598823044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8be4d0 msgr2=0x55593cc87fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.718+0000 7f9f7fd6e740 1 --2- 172.21.15.43:0/3598823044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8be4d0 0x55593cc87fa0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f9f640dc390 tx=0x7f9f640e7810 comp rx=0 tx=0).stop 2024-04-25T10:40:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.718+0000 7f9f72d9d640 1 --2- 172.21.15.43:0/2533524936 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9f5c0afc60 0x7f9f5c0b2170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55593cc84bd0 tx=0x7f9f6c009440 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.718+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/3598823044 shutdown_connections 2024-04-25T10:40:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.718+0000 7f9f7fd6e740 1 --2- 172.21.15.43:0/3598823044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8c66b0 0x55593cc948f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.718+0000 7f9f7fd6e740 1 --2- 172.21.15.43:0/3598823044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8be4d0 0x55593cc87fa0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.718+0000 7f9f7fd6e740 1 --2- 172.21.15.43:0/3598823044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8e7390 0x55593cc87a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.718+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/3598823044 >> 172.21.15.43:0/3598823044 conn(0x55593c8db4e0 msgr2=0x55593cc84210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.719+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/3598823044 shutdown_connections 2024-04-25T10:40:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.719+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/3598823044 wait complete. 2024-04-25T10:40:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.719+0000 7f9f7fd6e740 1 -- start start 2024-04-25T10:40:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.719+0000 7f9f7fd6e740 1 Processor -- start 2024-04-25T10:40:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.719+0000 7f9f7fd6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8be4d0 0x55593cc93fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.719+0000 7f9f7fd6e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8c66b0 0x55593cc94520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.719+0000 7f9f7fd6e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8e7390 0x55593cc924b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.719+0000 7f9f7fd6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55593cc94fe0 con 0x55593c8be4d0 2024-04-25T10:40:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.719+0000 7f9f7fd6e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55593cb74d10 con 0x55593c8e7390 2024-04-25T10:40:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.719+0000 7f9f7fd6e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55593cc95160 con 0x55593c8c66b0 2024-04-25T10:40:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.719+0000 7f9f7fd6e740 10 client.0 _ll_register_callbacks cb 0x55593cba1210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.719+0000 7f9f7259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8be4d0 0x55593cc93fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.719+0000 7f9f7259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8be4d0 0x55593cc93fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54230/0 (socket says 172.21.15.43:54230) 2024-04-25T10:40:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.719+0000 7f9f7259c640 1 -- 172.21.15.43:0/2948219528 learned_addr learned my addr 172.21.15.43:0/2948219528 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.720+0000 7f9f71d9b640 1 --2- 172.21.15.43:0/2948219528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8c66b0 0x55593cc94520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.720+0000 7f9f72d9d640 1 --2- 172.21.15.43:0/2948219528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8e7390 0x55593cc924b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.720+0000 7f9f7259c640 1 -- 172.21.15.43:0/2948219528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8c66b0 msgr2=0x55593cc94520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.720+0000 7f9f7259c640 1 --2- 172.21.15.43:0/2948219528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8c66b0 0x55593cc94520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.720+0000 7f9f7259c640 1 -- 172.21.15.43:0/2948219528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8e7390 msgr2=0x55593cc924b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.720+0000 7f9f7259c640 1 --2- 172.21.15.43:0/2948219528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8e7390 0x55593cc924b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.720+0000 7f9f7259c640 1 -- 172.21.15.43:0/2948219528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55593c9c97d0 con 0x55593c8be4d0 2024-04-25T10:40:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.720+0000 7f9f72d9d640 1 --2- 172.21.15.43:0/2948219528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8e7390 0x55593cc924b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.720+0000 7f9f71d9b640 1 --2- 172.21.15.43:0/2948219528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8c66b0 0x55593cc94520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.720+0000 7f9f7259c640 1 --2- 172.21.15.43:0/2948219528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8be4d0 0x55593cc93fe0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f9f640dc4d0 tx=0x7f9f6400df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534a0c9877db099d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.720+0000 7f9f59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.720+0000 7f9f59ffb640 1 -- 172.21.15.43:0/2948219528 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f640a67c0 con 0x55593c8be4d0 2024-04-25T10:40:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.720+0000 7f9f59ffb640 1 -- 172.21.15.43:0/2948219528 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f9f640a3460 con 0x55593c8be4d0 2024-04-25T10:40:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.720+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/2948219528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f640e0070 con 0x55593c8be4d0 2024-04-25T10:40:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.720+0000 7f9f59ffb640 1 -- 172.21.15.43:0/2948219528 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9f6406c050 con 0x55593c8be4d0 2024-04-25T10:40:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.720+0000 7f9f7fd6e740 10 client.17844 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.720+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/2948219528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55593c8ff3f0 con 0x55593c8be4d0 2024-04-25T10:40:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.720+0000 7f9f59ffb640 1 -- 172.21.15.43:0/2948219528 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f640e0070 con 0x55593c8be4d0 2024-04-25T10:40:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.721+0000 7f9f59ffb640 1 -- 172.21.15.43:0/2948219528 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9f640a70d0 con 0x55593c8be4d0 2024-04-25T10:40:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.721+0000 7f9f7fd6e740 10 client.17844 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.721+0000 7f9f7fd6e740 10 client.17844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.721+0000 7f9f7fd6e740 20 client.17844 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.721+0000 7f9f7fd6e740 1 --2- 172.21.15.43:0/2948219528 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55593c8a5f60 0x55593cca45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.721+0000 7f9f7fd6e740 4 client.17844 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.721+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/2948219528 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55593cbfb990 con 0x55593c8a5f60 2024-04-25T10:40:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.721+0000 7f9f71d9b640 1 --2- 172.21.15.43:0/2948219528 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55593c8a5f60 0x55593cca45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.721+0000 7f9f71d9b640 1 --2- 172.21.15.43:0/2948219528 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55593c8a5f60 0x55593cca45a0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=427f597e639ca33a server_cookie=120f9ff3b75e2b10 in_seq=0 out_seq=0 2024-04-25T10:40:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.721+0000 7f9f59ffb640 10 client.17844 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.722+0000 7f9f59ffb640 1 -- 172.21.15.43:0/2948219528 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55593cbfb990 con 0x55593c8a5f60 2024-04-25T10:40:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.722+0000 7f9f59ffb640 10 client.17844 handle_command_reply: tid=0 2024-04-25T10:40:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.722+0000 7f9f7fd6e740 1 client.17844 shutdown 2024-04-25T10:40:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.722+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/2948219528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8be4d0 msgr2=0x55593cc93fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.722+0000 7f9f7fd6e740 1 --2- 172.21.15.43:0/2948219528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8be4d0 0x55593cc93fe0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f9f640dc4d0 tx=0x7f9f6400df10 comp rx=0 tx=0).stop 2024-04-25T10:40:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.722+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/2948219528 shutdown_connections 2024-04-25T10:40:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.722+0000 7f9f7fd6e740 1 --2- 172.21.15.43:0/2948219528 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55593c8a5f60 0x55593cca45a0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.722+0000 7f9f7fd6e740 1 --2- 172.21.15.43:0/2948219528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8e7390 0x55593cc924b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.722+0000 7f9f7fd6e740 1 --2- 172.21.15.43:0/2948219528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8c66b0 0x55593cc94520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.722+0000 7f9f59ffb640 0 client.17844 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.722+0000 7f9f7fd6e740 1 --2- 172.21.15.43:0/2948219528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8be4d0 0x55593cc93fe0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.722+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/2948219528 >> 172.21.15.43:0/2948219528 conn(0x55593c8db4e0 msgr2=0x55593cc89780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.722+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/2948219528 shutdown_connections 2024-04-25T10:40:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.722+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/2948219528 wait complete. 2024-04-25T10:40:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.722+0000 7f9f7fd6e740 20 client.17844 trim_cache size 0 max 16384 2024-04-25T10:40:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.740+0000 7f9f7fd6e740 1 Processor -- start 2024-04-25T10:40:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.741+0000 7f9f7fd6e740 1 -- start start 2024-04-25T10:40:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.741+0000 7f9f7fd6e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8be4d0 0x55593cca75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.741+0000 7f9f7fd6e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8c66b0 0x55593cca7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.741+0000 7f9f7fd6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8e7390 0x55593cc89740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.741+0000 7f9f7fd6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9f741b2360 con 0x55593c8e7390 2024-04-25T10:40:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.741+0000 7f9f7fd6e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9f741afc30 con 0x55593c8be4d0 2024-04-25T10:40:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.741+0000 7f9f7fd6e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f741adaa0 con 0x55593c8c66b0 2024-04-25T10:40:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.741+0000 7f9f72d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8e7390 0x55593cc89740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.741+0000 7f9f72d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8e7390 0x55593cc89740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54240/0 (socket says 172.21.15.43:54240) 2024-04-25T10:40:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.741+0000 7f9f72d9d640 1 -- 172.21.15.43:0/95499116 learned_addr learned my addr 172.21.15.43:0/95499116 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.741+0000 7f9f71d9b640 1 --2- 172.21.15.43:0/95499116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8c66b0 0x55593cca7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.741+0000 7f9f7259c640 1 --2- 172.21.15.43:0/95499116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8be4d0 0x55593cca75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.741+0000 7f9f72d9d640 1 -- 172.21.15.43:0/95499116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8c66b0 msgr2=0x55593cca7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.741+0000 7f9f72d9d640 1 --2- 172.21.15.43:0/95499116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8c66b0 0x55593cca7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.741+0000 7f9f72d9d640 1 -- 172.21.15.43:0/95499116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8be4d0 msgr2=0x55593cca75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.741+0000 7f9f72d9d640 1 --2- 172.21.15.43:0/95499116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8be4d0 0x55593cca75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.741+0000 7f9f72d9d640 1 -- 172.21.15.43:0/95499116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55593c8bf0f0 con 0x55593c8e7390 2024-04-25T10:40:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.741+0000 7f9f72d9d640 1 --2- 172.21.15.43:0/95499116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8e7390 0x55593cc89740 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009ca0 tx=0x7f9f6c03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410536b4f68176f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.742+0000 7f9f5bfff640 1 -- 172.21.15.43:0/95499116 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c003cf0 con 0x55593c8e7390 2024-04-25T10:40:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.742+0000 7f9f5bfff640 1 -- 172.21.15.43:0/95499116 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f6c04d070 con 0x55593c8e7390 2024-04-25T10:40:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.742+0000 7f9f5bfff640 1 -- 172.21.15.43:0/95499116 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c030aa0 con 0x55593c8e7390 2024-04-25T10:40:15.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.742+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/95499116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8e7390 msgr2=0x55593cc89740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:15.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.742+0000 7f9f7fd6e740 1 --2- 172.21.15.43:0/95499116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8e7390 0x55593cc89740 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009ca0 tx=0x7f9f6c03e000 comp rx=0 tx=0).stop 2024-04-25T10:40:15.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.742+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/95499116 shutdown_connections 2024-04-25T10:40:15.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.742+0000 7f9f7fd6e740 1 --2- 172.21.15.43:0/95499116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8e7390 0x55593cc89740 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.742+0000 7f9f7fd6e740 1 --2- 172.21.15.43:0/95499116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8c66b0 0x55593cca7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.742+0000 7f9f7fd6e740 1 --2- 172.21.15.43:0/95499116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8be4d0 0x55593cca75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.742+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/95499116 >> 172.21.15.43:0/95499116 conn(0x55593c8db4e0 msgr2=0x55593cc90d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.742+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/95499116 shutdown_connections 2024-04-25T10:40:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.742+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/95499116 wait complete. 2024-04-25T10:40:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.743+0000 7f9f7fd6e740 1 -- start start 2024-04-25T10:40:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.743+0000 7f9f7fd6e740 1 Processor -- start 2024-04-25T10:40:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.743+0000 7f9f7fd6e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8be4d0 0x55593cd08fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.743+0000 7f9f7fd6e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8c66b0 0x55593cd09510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.743+0000 7f9f7fd6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8e7390 0x55593cca6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.743+0000 7f9f7fd6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9f741ae0f0 con 0x55593c8e7390 2024-04-25T10:40:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.743+0000 7f9f7fd6e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9f741af560 con 0x55593c8c66b0 2024-04-25T10:40:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.743+0000 7f9f7fd6e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f741ae7c0 con 0x55593c8be4d0 2024-04-25T10:40:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.743+0000 7f9f7fd6e740 10 client.0 _ll_register_callbacks cb 0x7f9f6c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.743+0000 7f9f72d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8e7390 0x55593cca6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.743+0000 7f9f72d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8e7390 0x55593cca6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54244/0 (socket says 172.21.15.43:54244) 2024-04-25T10:40:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.743+0000 7f9f72d9d640 1 -- 172.21.15.43:0/1200261969 learned_addr learned my addr 172.21.15.43:0/1200261969 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.743+0000 7f9f7259c640 1 --2- 172.21.15.43:0/1200261969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8be4d0 0x55593cd08fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.744+0000 7f9f72d9d640 1 -- 172.21.15.43:0/1200261969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8be4d0 msgr2=0x55593cd08fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.744+0000 7f9f72d9d640 1 --2- 172.21.15.43:0/1200261969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8be4d0 0x55593cd08fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.744+0000 7f9f71d9b640 1 --2- 172.21.15.43:0/1200261969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8c66b0 0x55593cd09510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.744+0000 7f9f72d9d640 1 -- 172.21.15.43:0/1200261969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8c66b0 msgr2=0x55593cd09510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.744+0000 7f9f72d9d640 1 --2- 172.21.15.43:0/1200261969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8c66b0 0x55593cd09510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.744+0000 7f9f72d9d640 1 -- 172.21.15.43:0/1200261969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55593cbae4b0 con 0x55593c8e7390 2024-04-25T10:40:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.744+0000 7f9f72d9d640 1 --2- 172.21.15.43:0/1200261969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8e7390 0x55593cca6b80 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c005e30 tx=0x7f9f6c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492dc6ea9406d0f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.744+0000 7f9f59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.744+0000 7f9f59ffb640 1 -- 172.21.15.43:0/1200261969 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c004780 con 0x55593c8e7390 2024-04-25T10:40:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.744+0000 7f9f59ffb640 1 -- 172.21.15.43:0/1200261969 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f9f6c030720 con 0x55593c8e7390 2024-04-25T10:40:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.744+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/1200261969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f6c04d070 con 0x55593c8e7390 2024-04-25T10:40:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.744+0000 7f9f59ffb640 1 -- 172.21.15.43:0/1200261969 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9f6c069050 con 0x55593c8e7390 2024-04-25T10:40:15.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.744+0000 7f9f7fd6e740 10 client.17850 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:15.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.744+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/1200261969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55593cd104c0 con 0x55593c8e7390 2024-04-25T10:40:15.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.744+0000 7f9f59ffb640 1 -- 172.21.15.43:0/1200261969 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f6c04d070 con 0x55593c8e7390 2024-04-25T10:40:15.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.745+0000 7f9f59ffb640 1 -- 172.21.15.43:0/1200261969 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9f6c064030 con 0x55593c8e7390 2024-04-25T10:40:15.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.745+0000 7f9f7fd6e740 10 client.17850 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:15.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.745+0000 7f9f7fd6e740 10 client.17850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:15.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.745+0000 7f9f7fd6e740 20 client.17850 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:15.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.745+0000 7f9f7fd6e740 1 --2- 172.21.15.43:0/1200261969 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55593c87b3a0 0x55593cd12bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:15.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.745+0000 7f9f7fd6e740 4 client.17850 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:40:15.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.745+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/1200261969 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55593cbf5800 con 0x55593c87b3a0 2024-04-25T10:40:15.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.745+0000 7f9f7259c640 1 --2- 172.21.15.43:0/1200261969 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55593c87b3a0 0x55593cd12bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:15.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.745+0000 7f9f7259c640 1 --2- 172.21.15.43:0/1200261969 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55593c87b3a0 0x55593cd12bf0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb9d0abb37e62c66 server_cookie=33c82883e497fa92 in_seq=0 out_seq=0 2024-04-25T10:40:15.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.745+0000 7f9f59ffb640 10 client.17850 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:15.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.745+0000 7f9f59ffb640 1 -- 172.21.15.43:0/1200261969 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55593cbf5800 con 0x55593c87b3a0 2024-04-25T10:40:15.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.745+0000 7f9f59ffb640 10 client.17850 handle_command_reply: tid=0 2024-04-25T10:40:15.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.745+0000 7f9f7fd6e740 1 client.17850 shutdown 2024-04-25T10:40:15.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/1200261969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8e7390 msgr2=0x55593cca6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:15.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f7fd6e740 1 --2- 172.21.15.43:0/1200261969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8e7390 0x55593cca6b80 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c005e30 tx=0x7f9f6c0095c0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/1200261969 shutdown_connections 2024-04-25T10:40:15.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f7fd6e740 1 --2- 172.21.15.43:0/1200261969 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55593c87b3a0 0x55593cd12bf0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f7fd6e740 1 --2- 172.21.15.43:0/1200261969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55593c8c66b0 0x55593cd09510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f7fd6e740 1 --2- 172.21.15.43:0/1200261969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55593c8e7390 0x55593cca6b80 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f7fd6e740 1 --2- 172.21.15.43:0/1200261969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55593c8be4d0 0x55593cd08fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f59ffb640 0 client.17850 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:15.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/1200261969 >> 172.21.15.43:0/1200261969 conn(0x55593c8db4e0 msgr2=0x55593cc93db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:15.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/1200261969 shutdown_connections 2024-04-25T10:40:15.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f7fd6e740 1 -- 172.21.15.43:0/1200261969 wait complete. 2024-04-25T10:40:15.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f7fd6e740 20 client.17850 trim_cache size 0 max 16384 2024-04-25T10:40:15.755 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:40:15.755 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:40:15.755 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:40:15.755 INFO:teuthology.orchestra.run.smithi043.stdout: "6a1e46be-72c9-49fe-9c3e-eb03718cb018": { 2024-04-25T10:40:15.755 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:40:15.755 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "6a1e46be-72c9-49fe-9c3e-eb03718cb018", 2024-04-25T10:40:15.755 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:40:15.755 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:40:15.755 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:40:15.755 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:40:15.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f78eb6640 1 -- 172.21.15.43:0/2533524936 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9f5c0afc60 msgr2=0x7f9f5c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:15.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f78eb6640 1 --2- 172.21.15.43:0/2533524936 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9f5c0afc60 0x7f9f5c0b2170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55593cc84bd0 tx=0x7f9f6c009440 comp rx=0 tx=0).stop 2024-04-25T10:40:15.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f78eb6640 1 -- 172.21.15.43:0/2533524936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f74104260 msgr2=0x7f9f7419a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:15.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f78eb6640 1 --2- 172.21.15.43:0/2533524936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f74104260 0x7f9f7419a760 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f9f64005fd0 tx=0x7f9f6400dde0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f78eb6640 1 -- 172.21.15.43:0/2533524936 shutdown_connections 2024-04-25T10:40:15.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f78eb6640 1 --2- 172.21.15.43:0/2533524936 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9f5c0afc60 0x7f9f5c0b2170 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f78eb6640 1 --2- 172.21.15.43:0/2533524936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f74109b10 0x7f9f7419acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f78eb6640 1 --2- 172.21.15.43:0/2533524936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f74109fb0 0x7f9f7419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f78eb6640 1 --2- 172.21.15.43:0/2533524936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f74104260 0x7f9f7419a760 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:15.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f78eb6640 1 -- 172.21.15.43:0/2533524936 >> 172.21.15.43:0/2533524936 conn(0x7f9f740ff230 msgr2=0x7f9f74101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:15.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f78eb6640 1 -- 172.21.15.43:0/2533524936 shutdown_connections 2024-04-25T10:40:15.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:15.746+0000 7f9f78eb6640 1 -- 172.21.15.43:0/2533524936 wait complete. 2024-04-25T10:40:15.759 INFO:tasks.cephfs.filesystem:scrub status for tag:6a1e46be-72c9-49fe-9c3e-eb03718cb018 - {'path': '/', 'tag': '6a1e46be-72c9-49fe-9c3e-eb03718cb018', 'options': 'recursive,force'} 2024-04-25T10:40:16.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:15 smithi043 ceph-mon[29478]: pgmap v239: 97 pgs: 97 active+clean; 5.0 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 16 op/s 2024-04-25T10:40:16.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:15 smithi077 ceph-mon[32772]: pgmap v239: 97 pgs: 97 active+clean; 5.0 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 16 op/s 2024-04-25T10:40:16.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:15 smithi164 ceph-mon[32713]: pgmap v239: 97 pgs: 97 active+clean; 5.0 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 16 op/s 2024-04-25T10:40:17.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:16 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:40:17.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:16 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:40:17.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:16 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:40:17.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:16 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:40:17.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:16 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:40:17.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:16 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:40:17.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:16 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:40:17.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:16 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:40:17.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:16 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:40:17.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:16 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:40:17.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:16 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:40:17.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:16 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:40:18.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:17 smithi077 ceph-mon[32772]: pgmap v240: 97 pgs: 97 active+clean; 5.0 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 16 op/s 2024-04-25T10:40:18.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:17 smithi164 ceph-mon[32713]: pgmap v240: 97 pgs: 97 active+clean; 5.0 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 16 op/s 2024-04-25T10:40:18.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:17 smithi043 ceph-mon[29478]: pgmap v240: 97 pgs: 97 active+clean; 5.0 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 16 op/s 2024-04-25T10:40:19.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:18 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:40:19.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:18 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:40:19.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:18 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:40:20.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:19 smithi077 ceph-mon[32772]: pgmap v241: 97 pgs: 97 active+clean; 5.2 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-25T10:40:20.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:19 smithi164 ceph-mon[32713]: pgmap v241: 97 pgs: 97 active+clean; 5.2 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-25T10:40:20.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:19 smithi043 ceph-mon[29478]: pgmap v241: 97 pgs: 97 active+clean; 5.2 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-25T10:40:22.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:21 smithi077 ceph-mon[32772]: pgmap v242: 97 pgs: 97 active+clean; 5.2 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-25T10:40:22.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:21 smithi164 ceph-mon[32713]: pgmap v242: 97 pgs: 97 active+clean; 5.2 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-25T10:40:22.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:21 smithi043 ceph-mon[29478]: pgmap v242: 97 pgs: 97 active+clean; 5.2 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-25T10:40:24.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:23 smithi077 ceph-mon[32772]: pgmap v243: 97 pgs: 97 active+clean; 5.2 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-25T10:40:24.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:23 smithi164 ceph-mon[32713]: pgmap v243: 97 pgs: 97 active+clean; 5.2 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-25T10:40:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.338+0000 7fade49ce640 1 -- 172.21.15.43:0/662917297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fade010b730 msgr2=0x7fade010db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.338+0000 7fade49ce640 1 --2- 172.21.15.43:0/662917297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fade010b730 0x7fade010db50 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009900 tx=0x7fadd402a510 comp rx=0 tx=0).stop 2024-04-25T10:40:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.338+0000 7fade49ce640 1 -- 172.21.15.43:0/662917297 shutdown_connections 2024-04-25T10:40:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.338+0000 7fade49ce640 1 --2- 172.21.15.43:0/662917297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fade010e0e0 0x7fade01105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.338+0000 7fade49ce640 1 --2- 172.21.15.43:0/662917297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fade010b730 0x7fade010db50 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.338+0000 7fade49ce640 1 --2- 172.21.15.43:0/662917297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fade0104cd0 0x7fade01050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.338+0000 7fade49ce640 1 -- 172.21.15.43:0/662917297 >> 172.21.15.43:0/662917297 conn(0x7fade00781b0 msgr2=0x7fade00785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.338+0000 7fade49ce640 1 -- 172.21.15.43:0/662917297 shutdown_connections 2024-04-25T10:40:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.338+0000 7fade49ce640 1 -- 172.21.15.43:0/662917297 wait complete. 2024-04-25T10:40:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.339+0000 7fade49ce640 1 Processor -- start 2024-04-25T10:40:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.339+0000 7fade49ce640 1 -- start start 2024-04-25T10:40:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.339+0000 7fade49ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fade0104cd0 0x7fade019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.339+0000 7fade49ce640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fade010b730 0x7fade019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.339+0000 7fade49ce640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fade010e0e0 0x7fade019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.339+0000 7fade49ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fade0113d40 con 0x7fade0104cd0 2024-04-25T10:40:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.339+0000 7fade49ce640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fade0113bc0 con 0x7fade010b730 2024-04-25T10:40:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.339+0000 7fade49ce640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fade0113ec0 con 0x7fade010e0e0 2024-04-25T10:40:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.339+0000 7fadde59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fade0104cd0 0x7fade019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.339+0000 7fadde59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fade0104cd0 0x7fade019a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59712/0 (socket says 172.21.15.43:59712) 2024-04-25T10:40:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.339+0000 7fadde59c640 1 -- 172.21.15.43:0/2742626302 learned_addr learned my addr 172.21.15.43:0/2742626302 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.340+0000 7fadded9d640 1 --2- 172.21.15.43:0/2742626302 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fade010e0e0 0x7fade019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.340+0000 7fadddd9b640 1 --2- 172.21.15.43:0/2742626302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fade010b730 0x7fade019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.340+0000 7fadde59c640 1 -- 172.21.15.43:0/2742626302 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fade010e0e0 msgr2=0x7fade019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.340+0000 7fadde59c640 1 --2- 172.21.15.43:0/2742626302 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fade010e0e0 0x7fade019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.340+0000 7fadde59c640 1 -- 172.21.15.43:0/2742626302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fade010b730 msgr2=0x7fade019acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.340+0000 7fadde59c640 1 --2- 172.21.15.43:0/2742626302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fade010b730 0x7fade019acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.340+0000 7fadde59c640 1 -- 172.21.15.43:0/2742626302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade019f5c0 con 0x7fade0104cd0 2024-04-25T10:40:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.340+0000 7fadde59c640 1 --2- 172.21.15.43:0/2742626302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fade0104cd0 0x7fade019a760 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fadd800e9f0 tx=0x7fadd800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.340+0000 7fadcf7fe640 1 -- 172.21.15.43:0/2742626302 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd800cd80 con 0x7fade0104cd0 2024-04-25T10:40:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.340+0000 7fadcf7fe640 1 -- 172.21.15.43:0/2742626302 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd80108e0 con 0x7fade0104cd0 2024-04-25T10:40:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.340+0000 7fade49ce640 1 -- 172.21.15.43:0/2742626302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade019f8b0 con 0x7fade0104cd0 2024-04-25T10:40:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.340+0000 7fadcf7fe640 1 -- 172.21.15.43:0/2742626302 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8018980 con 0x7fade0104cd0 2024-04-25T10:40:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.340+0000 7fade49ce640 1 -- 172.21.15.43:0/2742626302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade01a7230 con 0x7fade0104cd0 2024-04-25T10:40:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.343+0000 7fadcf7fe640 1 -- 172.21.15.43:0/2742626302 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fadd8010430 con 0x7fade0104cd0 2024-04-25T10:40:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.343+0000 7fadeb882740 1 Processor -- start 2024-04-25T10:40:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.343+0000 7fadeb882740 1 -- start start 2024-04-25T10:40:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.343+0000 7fadeb882740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d9b28821b0 0x55d9b2c21af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.343+0000 7fadeb882740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28592f0 0x55d9b2c22030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.344+0000 7fadeb882740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28614d0 0x55d9b2c22570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.344+0000 7fadcf7fe640 1 --2- 172.21.15.43:0/2742626302 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fadc80afc50 0x7fadc80b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.344+0000 7fadeb882740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d9b2b88460 con 0x55d9b28592f0 2024-04-25T10:40:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.344+0000 7fadeb882740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d9b2b42590 con 0x55d9b28614d0 2024-04-25T10:40:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.344+0000 7fadeb882740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d9b2b84420 con 0x55d9b28821b0 2024-04-25T10:40:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.344+0000 7fadde59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28592f0 0x55d9b2c22030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.344+0000 7fadded9d640 1 --2- 172.21.15.43:0/2742626302 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fadc80afc50 0x7fadc80b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.344+0000 7fadde59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28592f0 0x55d9b2c22030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59718/0 (socket says 172.21.15.43:59718) 2024-04-25T10:40:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.344+0000 7fadde59c640 1 -- 172.21.15.43:0/3571575526 learned_addr learned my addr 172.21.15.43:0/3571575526 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.344+0000 7fadddd9b640 1 --2- 172.21.15.43:0/3571575526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28614d0 0x55d9b2c22570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.344+0000 7fadcf7fe640 1 -- 172.21.15.43:0/2742626302 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fadd80df4b0 con 0x7fade0104cd0 2024-04-25T10:40:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.344+0000 7fadded9d640 1 --2- 172.21.15.43:0/3571575526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d9b28821b0 0x55d9b2c21af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.344+0000 7fadde59c640 1 -- 172.21.15.43:0/3571575526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d9b28821b0 msgr2=0x55d9b2c21af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.344+0000 7fadde59c640 1 --2- 172.21.15.43:0/3571575526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d9b28821b0 0x55d9b2c21af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.344+0000 7fadde59c640 1 -- 172.21.15.43:0/3571575526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28614d0 msgr2=0x55d9b2c22570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.344+0000 7fadde59c640 1 --2- 172.21.15.43:0/3571575526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28614d0 0x55d9b2c22570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.344+0000 7fadde59c640 1 -- 172.21.15.43:0/3571575526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9b2b4f0d0 con 0x55d9b28592f0 2024-04-25T10:40:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.344+0000 7fadded9d640 1 --2- 172.21.15.43:0/2742626302 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fadc80afc50 0x7fadc80b2160 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fadd0005c40 tx=0x7fadd0009c90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:24.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.344+0000 7fadde59c640 1 --2- 172.21.15.43:0/3571575526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28592f0 0x55d9b2c22030 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fadd80dcc60 tx=0x7fadd8071ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d13a6ab75f95625 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:24.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.345+0000 7fadccff9640 1 -- 172.21.15.43:0/3571575526 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd80a7160 con 0x55d9b28592f0 2024-04-25T10:40:24.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.345+0000 7fadccff9640 1 -- 172.21.15.43:0/3571575526 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd80a7d50 con 0x55d9b28592f0 2024-04-25T10:40:24.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.345+0000 7fadccff9640 1 -- 172.21.15.43:0/3571575526 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd80aba00 con 0x55d9b28592f0 2024-04-25T10:40:24.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.345+0000 7fadeb882740 1 -- 172.21.15.43:0/3571575526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28592f0 msgr2=0x55d9b2c22030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:24.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.345+0000 7fadeb882740 1 --2- 172.21.15.43:0/3571575526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28592f0 0x55d9b2c22030 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fadd80dcc60 tx=0x7fadd8071ee0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.345+0000 7fadeb882740 1 -- 172.21.15.43:0/3571575526 shutdown_connections 2024-04-25T10:40:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.345+0000 7fadeb882740 1 --2- 172.21.15.43:0/3571575526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28614d0 0x55d9b2c22570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.345+0000 7fadeb882740 1 --2- 172.21.15.43:0/3571575526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28592f0 0x55d9b2c22030 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.345+0000 7fadeb882740 1 --2- 172.21.15.43:0/3571575526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d9b28821b0 0x55d9b2c21af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.346+0000 7fadeb882740 1 -- 172.21.15.43:0/3571575526 >> 172.21.15.43:0/3571575526 conn(0x55d9b2876300 msgr2=0x55d9b2c1e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.346+0000 7fadeb882740 1 -- 172.21.15.43:0/3571575526 shutdown_connections 2024-04-25T10:40:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.346+0000 7fadeb882740 1 -- 172.21.15.43:0/3571575526 wait complete. 2024-04-25T10:40:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.346+0000 7fadeb882740 1 -- start start 2024-04-25T10:40:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.346+0000 7fadeb882740 1 Processor -- start 2024-04-25T10:40:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.346+0000 7fadeb882740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28592f0 0x55d9b2c2db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.346+0000 7fadeb882740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d9b28614d0 0x55d9b2c2e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.346+0000 7fadeb882740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28821b0 0x55d9b2c2c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.346+0000 7fadeb882740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d9b2c2ec20 con 0x55d9b28821b0 2024-04-25T10:40:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.346+0000 7fadeb882740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d9b2b0fb60 con 0x55d9b28592f0 2024-04-25T10:40:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.346+0000 7fadeb882740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d9b2c2eda0 con 0x55d9b28614d0 2024-04-25T10:40:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.347+0000 7fadeb882740 10 client.0 _ll_register_callbacks cb 0x55d9b2b342b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.347+0000 7fadded9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28821b0 0x55d9b2c2c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.347+0000 7fadded9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28821b0 0x55d9b2c2c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59720/0 (socket says 172.21.15.43:59720) 2024-04-25T10:40:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.347+0000 7fadded9d640 1 -- 172.21.15.43:0/96001081 learned_addr learned my addr 172.21.15.43:0/96001081 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.347+0000 7fadde59c640 1 --2- 172.21.15.43:0/96001081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28592f0 0x55d9b2c2db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.347+0000 7fadddd9b640 1 --2- 172.21.15.43:0/96001081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d9b28614d0 0x55d9b2c2e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.347+0000 7fadded9d640 1 -- 172.21.15.43:0/96001081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d9b28614d0 msgr2=0x55d9b2c2e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.347+0000 7fadded9d640 1 --2- 172.21.15.43:0/96001081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d9b28614d0 0x55d9b2c2e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.347+0000 7fadded9d640 1 -- 172.21.15.43:0/96001081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28592f0 msgr2=0x55d9b2c2db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.347+0000 7fadded9d640 1 --2- 172.21.15.43:0/96001081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28592f0 0x55d9b2c2db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.347+0000 7fadded9d640 1 -- 172.21.15.43:0/96001081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9b289a210 con 0x55d9b28821b0 2024-04-25T10:40:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.347+0000 7fadded9d640 1 --2- 172.21.15.43:0/96001081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28821b0 0x55d9b2c2c050 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fadd0010c60 tx=0x7fadd001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd70861b5729a5f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.347+0000 7fadc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.347+0000 7fadc5ffb640 1 -- 172.21.15.43:0/96001081 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0027070 con 0x55d9b28821b0 2024-04-25T10:40:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.347+0000 7fadc5ffb640 1 -- 172.21.15.43:0/96001081 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fadd001ee00 con 0x55d9b28821b0 2024-04-25T10:40:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.347+0000 7fadeb882740 1 -- 172.21.15.43:0/96001081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd80a7d50 con 0x55d9b28821b0 2024-04-25T10:40:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.348+0000 7fadc5ffb640 1 -- 172.21.15.43:0/96001081 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fadd0033050 con 0x55d9b28821b0 2024-04-25T10:40:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.348+0000 7fadeb882740 10 client.17862 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.348+0000 7fadeb882740 1 -- 172.21.15.43:0/96001081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d9b2966620 con 0x55d9b28821b0 2024-04-25T10:40:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.348+0000 7fadc5ffb640 1 -- 172.21.15.43:0/96001081 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd80a7d50 con 0x55d9b28821b0 2024-04-25T10:40:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.348+0000 7fadc5ffb640 1 -- 172.21.15.43:0/96001081 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fadd001f070 con 0x55d9b28821b0 2024-04-25T10:40:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.348+0000 7fadeb882740 10 client.17862 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.348+0000 7fadeb882740 10 client.17862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.348+0000 7fadeb882740 20 client.17862 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.348+0000 7fadeb882740 1 --2- 172.21.15.43:0/96001081 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d9b2840d80 0x55d9b2c3e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.348+0000 7fadeb882740 4 client.17862 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.348+0000 7fadeb882740 1 -- 172.21.15.43:0/96001081 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9b2b96070 con 0x55d9b2840d80 2024-04-25T10:40:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.348+0000 7fadde59c640 1 --2- 172.21.15.43:0/96001081 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d9b2840d80 0x55d9b2c3e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.349+0000 7fadde59c640 1 --2- 172.21.15.43:0/96001081 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d9b2840d80 0x55d9b2c3e770 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bbeba0d3e571eed server_cookie=5e17213488af1f6f in_seq=0 out_seq=0 2024-04-25T10:40:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.349+0000 7fadc5ffb640 10 client.17862 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.350+0000 7fadc5ffb640 1 -- 172.21.15.43:0/96001081 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d9b2b96070 con 0x55d9b2840d80 2024-04-25T10:40:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.350+0000 7fadc5ffb640 10 client.17862 handle_command_reply: tid=0 2024-04-25T10:40:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.350+0000 7fadeb882740 1 client.17862 shutdown 2024-04-25T10:40:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.350+0000 7fadeb882740 1 -- 172.21.15.43:0/96001081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28821b0 msgr2=0x55d9b2c2c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.350+0000 7fadeb882740 1 --2- 172.21.15.43:0/96001081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28821b0 0x55d9b2c2c050 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fadd0010c60 tx=0x7fadd001c000 comp rx=0 tx=0).stop 2024-04-25T10:40:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.350+0000 7fadeb882740 1 -- 172.21.15.43:0/96001081 shutdown_connections 2024-04-25T10:40:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.350+0000 7fadeb882740 1 --2- 172.21.15.43:0/96001081 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d9b2840d80 0x55d9b2c3e770 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.350+0000 7fadeb882740 1 --2- 172.21.15.43:0/96001081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28821b0 0x55d9b2c2c050 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.350+0000 7fadc5ffb640 0 client.17862 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.350+0000 7fadeb882740 1 --2- 172.21.15.43:0/96001081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d9b28614d0 0x55d9b2c2e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.350+0000 7fadeb882740 1 --2- 172.21.15.43:0/96001081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28592f0 0x55d9b2c2db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.350+0000 7fadeb882740 1 -- 172.21.15.43:0/96001081 >> 172.21.15.43:0/96001081 conn(0x55d9b2876300 msgr2=0x55d9b2c23810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.350+0000 7fadeb882740 1 -- 172.21.15.43:0/96001081 shutdown_connections 2024-04-25T10:40:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.350+0000 7fadeb882740 1 -- 172.21.15.43:0/96001081 wait complete. 2024-04-25T10:40:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.350+0000 7fadeb882740 20 client.17862 trim_cache size 0 max 16384 2024-04-25T10:40:24.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:23 smithi043 ceph-mon[29478]: pgmap v243: 97 pgs: 97 active+clean; 5.2 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-25T10:40:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.366+0000 7fadeb882740 1 Processor -- start 2024-04-25T10:40:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.366+0000 7fadeb882740 1 -- start start 2024-04-25T10:40:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.366+0000 7fadeb882740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28592f0 0x55d9b2c22a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.366+0000 7fadeb882740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28614d0 0x55d9b2c23eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.366+0000 7fadeb882740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d9b28821b0 0x55d9b2c257a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.366+0000 7fadeb882740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fade01b2380 con 0x55d9b28614d0 2024-04-25T10:40:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.366+0000 7fadeb882740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fade01afc50 con 0x55d9b28592f0 2024-04-25T10:40:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.366+0000 7fadeb882740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fade01adac0 con 0x55d9b28821b0 2024-04-25T10:40:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.366+0000 7fadddd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28614d0 0x55d9b2c23eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.366+0000 7fadddd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28614d0 0x55d9b2c23eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59732/0 (socket says 172.21.15.43:59732) 2024-04-25T10:40:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.366+0000 7fadddd9b640 1 -- 172.21.15.43:0/2590819943 learned_addr learned my addr 172.21.15.43:0/2590819943 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.366+0000 7fadde59c640 1 --2- 172.21.15.43:0/2590819943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28592f0 0x55d9b2c22a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.366+0000 7fadddd9b640 1 -- 172.21.15.43:0/2590819943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d9b28821b0 msgr2=0x55d9b2c257a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.366+0000 7fadded9d640 1 --2- 172.21.15.43:0/2590819943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d9b28821b0 0x55d9b2c257a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.366+0000 7fadddd9b640 1 --2- 172.21.15.43:0/2590819943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d9b28821b0 0x55d9b2c257a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.366+0000 7fadddd9b640 1 -- 172.21.15.43:0/2590819943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28592f0 msgr2=0x55d9b2c22a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.366+0000 7fadddd9b640 1 --2- 172.21.15.43:0/2590819943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28592f0 0x55d9b2c22a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.366+0000 7fadddd9b640 1 -- 172.21.15.43:0/2590819943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9b2c1ee90 con 0x55d9b28614d0 2024-04-25T10:40:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.366+0000 7fadde59c640 1 --2- 172.21.15.43:0/2590819943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28592f0 0x55d9b2c22a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.367+0000 7fadddd9b640 1 --2- 172.21.15.43:0/2590819943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28614d0 0x55d9b2c23eb0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009900 tx=0x7fadd4032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457428d9047c2791 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.367+0000 7fadc7fff640 1 -- 172.21.15.43:0/2590819943 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4035040 con 0x55d9b28614d0 2024-04-25T10:40:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.367+0000 7fadc7fff640 1 -- 172.21.15.43:0/2590819943 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd40316a0 con 0x55d9b28614d0 2024-04-25T10:40:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.367+0000 7fadc7fff640 1 -- 172.21.15.43:0/2590819943 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd40046c0 con 0x55d9b28614d0 2024-04-25T10:40:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.368+0000 7fadeb882740 1 -- 172.21.15.43:0/2590819943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28614d0 msgr2=0x55d9b2c23eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.368+0000 7fadeb882740 1 --2- 172.21.15.43:0/2590819943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28614d0 0x55d9b2c23eb0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009900 tx=0x7fadd4032df0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.368+0000 7fadeb882740 1 -- 172.21.15.43:0/2590819943 shutdown_connections 2024-04-25T10:40:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.368+0000 7fadeb882740 1 --2- 172.21.15.43:0/2590819943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d9b28821b0 0x55d9b2c257a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.368+0000 7fadeb882740 1 --2- 172.21.15.43:0/2590819943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28614d0 0x55d9b2c23eb0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.368+0000 7fadeb882740 1 --2- 172.21.15.43:0/2590819943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28592f0 0x55d9b2c22a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.368+0000 7fadeb882740 1 -- 172.21.15.43:0/2590819943 >> 172.21.15.43:0/2590819943 conn(0x55d9b2876300 msgr2=0x55d9b2c237a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.368+0000 7fadeb882740 1 -- 172.21.15.43:0/2590819943 shutdown_connections 2024-04-25T10:40:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.368+0000 7fadeb882740 1 -- 172.21.15.43:0/2590819943 wait complete. 2024-04-25T10:40:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.368+0000 7fadeb882740 1 -- start start 2024-04-25T10:40:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.368+0000 7fadeb882740 1 Processor -- start 2024-04-25T10:40:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.368+0000 7fadeb882740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d9b28592f0 0x55d9b2c31720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.368+0000 7fadeb882740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28614d0 0x55d9b2c31c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.368+0000 7fadeb882740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28821b0 0x55d9b2c9dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.368+0000 7fadeb882740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fade01ae110 con 0x55d9b28821b0 2024-04-25T10:40:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.368+0000 7fadeb882740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fade01af580 con 0x55d9b28614d0 2024-04-25T10:40:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.368+0000 7fadeb882740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fade01ae7e0 con 0x55d9b28592f0 2024-04-25T10:40:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.368+0000 7fadeb882740 10 client.0 _ll_register_callbacks cb 0x7fadd0009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.369+0000 7fadded9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28821b0 0x55d9b2c9dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.369+0000 7fadded9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28821b0 0x55d9b2c9dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59736/0 (socket says 172.21.15.43:59736) 2024-04-25T10:40:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.369+0000 7fadded9d640 1 -- 172.21.15.43:0/2724344954 learned_addr learned my addr 172.21.15.43:0/2724344954 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.369+0000 7fadde59c640 1 --2- 172.21.15.43:0/2724344954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d9b28592f0 0x55d9b2c31720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.369+0000 7fadddd9b640 1 --2- 172.21.15.43:0/2724344954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28614d0 0x55d9b2c31c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.369+0000 7fadded9d640 1 -- 172.21.15.43:0/2724344954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d9b28592f0 msgr2=0x55d9b2c31720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.369+0000 7fadded9d640 1 --2- 172.21.15.43:0/2724344954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d9b28592f0 0x55d9b2c31720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.369+0000 7fadded9d640 1 -- 172.21.15.43:0/2724344954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28614d0 msgr2=0x55d9b2c31c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.369+0000 7fadded9d640 1 --2- 172.21.15.43:0/2724344954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28614d0 0x55d9b2c31c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.369+0000 7fadded9d640 1 -- 172.21.15.43:0/2724344954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9b2c9e0a0 con 0x55d9b28821b0 2024-04-25T10:40:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.369+0000 7fadded9d640 1 --2- 172.21.15.43:0/2724344954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28821b0 0x55d9b2c9dae0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009530 tx=0x7fadd0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52575e02825dcf48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.369+0000 7fadc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.369+0000 7fadc5ffb640 1 -- 172.21.15.43:0/2724344954 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0024020 con 0x55d9b28821b0 2024-04-25T10:40:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.369+0000 7fadc5ffb640 1 -- 172.21.15.43:0/2724344954 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fadd001ec50 con 0x55d9b28821b0 2024-04-25T10:40:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.369+0000 7fadeb882740 1 -- 172.21.15.43:0/2724344954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd40316a0 con 0x55d9b28821b0 2024-04-25T10:40:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.369+0000 7fadc5ffb640 1 -- 172.21.15.43:0/2724344954 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fadd0027040 con 0x55d9b28821b0 2024-04-25T10:40:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.369+0000 7fadeb882740 10 client.17868 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.369+0000 7fadeb882740 1 -- 172.21.15.43:0/2724344954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d9b2ca6510 con 0x55d9b28821b0 2024-04-25T10:40:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.370+0000 7fadc5ffb640 1 -- 172.21.15.43:0/2724344954 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd40316a0 con 0x55d9b28821b0 2024-04-25T10:40:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.370+0000 7fadc5ffb640 1 -- 172.21.15.43:0/2724344954 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fadd001fbf0 con 0x55d9b28821b0 2024-04-25T10:40:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.370+0000 7fadeb882740 10 client.17868 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.370+0000 7fadeb882740 10 client.17868 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.370+0000 7fadeb882740 20 client.17868 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.370+0000 7fadeb882740 1 --2- 172.21.15.43:0/2724344954 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d9b28161c0 0x55d9b2ca8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.370+0000 7fadeb882740 4 client.17868 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:40:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.370+0000 7fadeb882740 1 -- 172.21.15.43:0/2724344954 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d9b2b8fee0 con 0x55d9b28161c0 2024-04-25T10:40:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.370+0000 7fadde59c640 1 --2- 172.21.15.43:0/2724344954 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d9b28161c0 0x55d9b2ca8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.371+0000 7fadc5ffb640 10 client.17868 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.371+0000 7fadde59c640 1 --2- 172.21.15.43:0/2724344954 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d9b28161c0 0x55d9b2ca8d20 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbff8381d82afbbb server_cookie=715578a57f734cd6 in_seq=0 out_seq=0 2024-04-25T10:40:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.386+0000 7fadc5ffb640 1 -- 172.21.15.43:0/2724344954 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d9b2b8fee0 con 0x55d9b28161c0 2024-04-25T10:40:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.386+0000 7fadc5ffb640 10 client.17868 handle_command_reply: tid=0 2024-04-25T10:40:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.386+0000 7fadeb882740 1 client.17868 shutdown 2024-04-25T10:40:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.386+0000 7fadeb882740 1 -- 172.21.15.43:0/2724344954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28821b0 msgr2=0x55d9b2c9dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.386+0000 7fadeb882740 1 --2- 172.21.15.43:0/2724344954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28821b0 0x55d9b2c9dae0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009530 tx=0x7fadd0004ab0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.386+0000 7fadeb882740 1 -- 172.21.15.43:0/2724344954 shutdown_connections 2024-04-25T10:40:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.386+0000 7fadeb882740 1 --2- 172.21.15.43:0/2724344954 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d9b28161c0 0x55d9b2ca8d20 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.386+0000 7fadeb882740 1 --2- 172.21.15.43:0/2724344954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b28614d0 0x55d9b2c31c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.387+0000 7fadeb882740 1 --2- 172.21.15.43:0/2724344954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9b28821b0 0x55d9b2c9dae0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.386+0000 7fadc5ffb640 0 client.17868 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.387+0000 7fadeb882740 1 --2- 172.21.15.43:0/2724344954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d9b28592f0 0x55d9b2c31720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.387+0000 7fadeb882740 1 -- 172.21.15.43:0/2724344954 >> 172.21.15.43:0/2724344954 conn(0x55d9b2876300 msgr2=0x55d9b2c2b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.387+0000 7fadeb882740 1 -- 172.21.15.43:0/2724344954 shutdown_connections 2024-04-25T10:40:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.387+0000 7fadeb882740 1 -- 172.21.15.43:0/2724344954 wait complete. 2024-04-25T10:40:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.387+0000 7fadeb882740 20 client.17868 trim_cache size 0 max 16384 2024-04-25T10:40:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:40:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:40:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:40:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:40:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.387+0000 7fade49ce640 1 -- 172.21.15.43:0/2742626302 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fadc80afc50 msgr2=0x7fadc80b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.387+0000 7fade49ce640 1 --2- 172.21.15.43:0/2742626302 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fadc80afc50 0x7fadc80b2160 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fadd0005c40 tx=0x7fadd0009c90 comp rx=0 tx=0).stop 2024-04-25T10:40:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.387+0000 7fade49ce640 1 -- 172.21.15.43:0/2742626302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fade0104cd0 msgr2=0x7fade019a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.387+0000 7fade49ce640 1 --2- 172.21.15.43:0/2742626302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fade0104cd0 0x7fade019a760 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fadd800e9f0 tx=0x7fadd800eeb0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.387+0000 7fade49ce640 1 -- 172.21.15.43:0/2742626302 shutdown_connections 2024-04-25T10:40:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.387+0000 7fade49ce640 1 --2- 172.21.15.43:0/2742626302 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fadc80afc50 0x7fadc80b2160 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.387+0000 7fade49ce640 1 --2- 172.21.15.43:0/2742626302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fade010b730 0x7fade019acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.387+0000 7fade49ce640 1 --2- 172.21.15.43:0/2742626302 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fade010e0e0 0x7fade019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.387+0000 7fade49ce640 1 --2- 172.21.15.43:0/2742626302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fade0104cd0 0x7fade019a760 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.387+0000 7fade49ce640 1 -- 172.21.15.43:0/2742626302 >> 172.21.15.43:0/2742626302 conn(0x7fade00781b0 msgr2=0x7fade0101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.387+0000 7fade49ce640 1 -- 172.21.15.43:0/2742626302 shutdown_connections 2024-04-25T10:40:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:24.387+0000 7fade49ce640 1 -- 172.21.15.43:0/2742626302 wait complete. 2024-04-25T10:40:26.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:25 smithi077 ceph-mon[32772]: pgmap v244: 97 pgs: 97 active+clean; 5.3 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 61 KiB/s wr, 20 op/s 2024-04-25T10:40:26.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:25 smithi164 ceph-mon[32713]: pgmap v244: 97 pgs: 97 active+clean; 5.3 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 61 KiB/s wr, 20 op/s 2024-04-25T10:40:26.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:25 smithi043 ceph-mon[29478]: pgmap v244: 97 pgs: 97 active+clean; 5.3 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 61 KiB/s wr, 20 op/s 2024-04-25T10:40:26.973 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.973 INFO:tasks.workunit.client.0.smithi043.stdout:313/600: open fd = 3 2024-04-25T10:40:26.973 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.973 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.973 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.973 INFO:tasks.workunit.client.0.smithi043.stdout:314/600: open fd = 3 2024-04-25T10:40:26.973 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.973 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.973 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.973 INFO:tasks.workunit.client.0.smithi043.stdout:315/600: open fd = 3 2024-04-25T10:40:26.973 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.974 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.974 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.974 INFO:tasks.workunit.client.0.smithi043.stdout:316/600: open fd = 3 2024-04-25T10:40:26.974 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.974 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.974 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.974 INFO:tasks.workunit.client.0.smithi043.stdout:317/600: open fd = 3 2024-04-25T10:40:26.974 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.974 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.974 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.974 INFO:tasks.workunit.client.0.smithi043.stdout:318/600: open fd = 3 2024-04-25T10:40:26.974 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.974 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.975 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.975 INFO:tasks.workunit.client.0.smithi043.stdout:319/600: open fd = 3 2024-04-25T10:40:26.975 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.975 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.975 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.975 INFO:tasks.workunit.client.0.smithi043.stdout:320/600: open fd = 3 2024-04-25T10:40:26.975 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.975 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.975 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.975 INFO:tasks.workunit.client.0.smithi043.stdout:321/600: open fd = 3 2024-04-25T10:40:26.975 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.975 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.976 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.976 INFO:tasks.workunit.client.0.smithi043.stdout:322/600: open fd = 3 2024-04-25T10:40:26.976 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.976 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.976 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.976 INFO:tasks.workunit.client.0.smithi043.stdout:323/600: open fd = 3 2024-04-25T10:40:26.976 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.976 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.976 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.976 INFO:tasks.workunit.client.0.smithi043.stdout:324/600: open fd = 3 2024-04-25T10:40:26.976 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.976 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.976 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.976 INFO:tasks.workunit.client.0.smithi043.stdout:325/600: open fd = 3 2024-04-25T10:40:26.976 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.977 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.977 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.977 INFO:tasks.workunit.client.0.smithi043.stdout:326/600: open fd = 3 2024-04-25T10:40:26.977 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.977 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.977 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.977 INFO:tasks.workunit.client.0.smithi043.stdout:327/600: open fd = 3 2024-04-25T10:40:26.977 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.977 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.977 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.977 INFO:tasks.workunit.client.0.smithi043.stdout:328/600: open fd = 3 2024-04-25T10:40:26.977 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.977 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.977 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.978 INFO:tasks.workunit.client.0.smithi043.stdout:329/600: open fd = 3 2024-04-25T10:40:26.978 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.978 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.978 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.978 INFO:tasks.workunit.client.0.smithi043.stdout:330/600: open fd = 3 2024-04-25T10:40:26.978 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.978 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.978 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.978 INFO:tasks.workunit.client.0.smithi043.stdout:331/600: open fd = 3 2024-04-25T10:40:26.978 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.978 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.978 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.978 INFO:tasks.workunit.client.0.smithi043.stdout:332/600: open fd = 3 2024-04-25T10:40:26.978 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.978 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.979 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.979 INFO:tasks.workunit.client.0.smithi043.stdout:333/600: open fd = 3 2024-04-25T10:40:26.979 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.979 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.979 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.979 INFO:tasks.workunit.client.0.smithi043.stdout:334/600: open fd = 3 2024-04-25T10:40:26.979 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.979 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.979 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.979 INFO:tasks.workunit.client.0.smithi043.stdout:335/600: open fd = 3 2024-04-25T10:40:26.979 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.979 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.979 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.979 INFO:tasks.workunit.client.0.smithi043.stdout:336/600: open fd = 3 2024-04-25T10:40:26.980 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.980 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.980 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.980 INFO:tasks.workunit.client.0.smithi043.stdout:337/600: open fd = 3 2024-04-25T10:40:26.980 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.980 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.980 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.980 INFO:tasks.workunit.client.0.smithi043.stdout:338/600: open fd = 3 2024-04-25T10:40:26.980 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.980 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.980 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.980 INFO:tasks.workunit.client.0.smithi043.stdout:339/600: open fd = 3 2024-04-25T10:40:26.980 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.980 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.980 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.981 INFO:tasks.workunit.client.0.smithi043.stdout:340/600: open fd = 3 2024-04-25T10:40:26.981 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.981 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.981 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.981 INFO:tasks.workunit.client.0.smithi043.stdout:341/600: open fd = 3 2024-04-25T10:40:26.981 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.981 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.981 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.981 INFO:tasks.workunit.client.0.smithi043.stdout:342/600: open fd = 3 2024-04-25T10:40:26.981 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.981 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.981 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.981 INFO:tasks.workunit.client.0.smithi043.stdout:343/600: open fd = 3 2024-04-25T10:40:26.981 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.981 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.981 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.982 INFO:tasks.workunit.client.0.smithi043.stdout:344/600: open fd = 3 2024-04-25T10:40:26.982 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.982 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.982 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.982 INFO:tasks.workunit.client.0.smithi043.stdout:345/600: open fd = 3 2024-04-25T10:40:26.982 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.982 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.982 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.982 INFO:tasks.workunit.client.0.smithi043.stdout:346/600: open fd = 3 2024-04-25T10:40:26.982 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.982 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.982 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.982 INFO:tasks.workunit.client.0.smithi043.stdout:347/600: open fd = 3 2024-04-25T10:40:26.982 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.983 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.983 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.983 INFO:tasks.workunit.client.0.smithi043.stdout:348/600: open fd = 3 2024-04-25T10:40:26.983 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.983 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.983 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.983 INFO:tasks.workunit.client.0.smithi043.stdout:349/600: open fd = 3 2024-04-25T10:40:26.983 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.983 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.983 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.983 INFO:tasks.workunit.client.0.smithi043.stdout:350/600: open fd = 3 2024-04-25T10:40:26.983 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.983 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.983 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.984 INFO:tasks.workunit.client.0.smithi043.stdout:351/600: open fd = 3 2024-04-25T10:40:26.984 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.984 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.984 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.984 INFO:tasks.workunit.client.0.smithi043.stdout:352/600: open fd = 3 2024-04-25T10:40:26.984 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.984 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.984 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.984 INFO:tasks.workunit.client.0.smithi043.stdout:353/600: open fd = 3 2024-04-25T10:40:26.984 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.984 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.984 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.984 INFO:tasks.workunit.client.0.smithi043.stdout:354/600: open fd = 3 2024-04-25T10:40:26.984 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.984 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.985 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.985 INFO:tasks.workunit.client.0.smithi043.stdout:355/600: open fd = 3 2024-04-25T10:40:26.985 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.985 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.985 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.985 INFO:tasks.workunit.client.0.smithi043.stdout:356/600: open fd = 3 2024-04-25T10:40:26.985 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.985 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.985 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.985 INFO:tasks.workunit.client.0.smithi043.stdout:357/600: open fd = 3 2024-04-25T10:40:26.985 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.985 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.985 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.985 INFO:tasks.workunit.client.0.smithi043.stdout:358/600: open fd = 3 2024-04-25T10:40:26.985 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.985 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.986 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.986 INFO:tasks.workunit.client.0.smithi043.stdout:359/600: open fd = 3 2024-04-25T10:40:26.986 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.986 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.986 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.986 INFO:tasks.workunit.client.0.smithi043.stdout:360/600: open fd = 3 2024-04-25T10:40:26.986 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.986 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.986 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.986 INFO:tasks.workunit.client.0.smithi043.stdout:361/600: open fd = 3 2024-04-25T10:40:26.986 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.986 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.986 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.986 INFO:tasks.workunit.client.0.smithi043.stdout:362/600: open fd = 3 2024-04-25T10:40:26.986 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.987 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.987 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.987 INFO:tasks.workunit.client.0.smithi043.stdout:363/600: open fd = 3 2024-04-25T10:40:26.987 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.987 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.987 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.987 INFO:tasks.workunit.client.0.smithi043.stdout:364/600: open fd = 3 2024-04-25T10:40:26.987 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.987 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.987 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.987 INFO:tasks.workunit.client.0.smithi043.stdout:365/600: open fd = 3 2024-04-25T10:40:26.987 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.987 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.987 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.987 INFO:tasks.workunit.client.0.smithi043.stdout:366/600: open fd = 3 2024-04-25T10:40:26.988 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.988 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.988 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.988 INFO:tasks.workunit.client.0.smithi043.stdout:367/600: open fd = 3 2024-04-25T10:40:26.988 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.988 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.988 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.988 INFO:tasks.workunit.client.0.smithi043.stdout:368/600: open fd = 3 2024-04-25T10:40:26.988 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.988 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.988 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.988 INFO:tasks.workunit.client.0.smithi043.stdout:369/600: open fd = 3 2024-04-25T10:40:26.988 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.988 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.988 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.988 INFO:tasks.workunit.client.0.smithi043.stdout:370/600: open fd = 3 2024-04-25T10:40:26.989 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.989 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.989 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.989 INFO:tasks.workunit.client.0.smithi043.stdout:371/600: open fd = 3 2024-04-25T10:40:26.989 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.989 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.989 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.989 INFO:tasks.workunit.client.0.smithi043.stdout:372/600: open fd = 3 2024-04-25T10:40:26.989 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.989 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.989 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.989 INFO:tasks.workunit.client.0.smithi043.stdout:373/600: open fd = 3 2024-04-25T10:40:26.989 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.989 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.989 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:26.990 INFO:tasks.workunit.client.0.smithi043.stdout:374/600: open fd = 3 2024-04-25T10:40:26.990 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.990 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:40:26.990 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:40:28.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:27 smithi077 ceph-mon[32772]: pgmap v245: 97 pgs: 97 active+clean; 5.3 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-04-25T10:40:28.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:27 smithi164 ceph-mon[32713]: pgmap v245: 97 pgs: 97 active+clean; 5.3 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-04-25T10:40:28.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:27 smithi043 ceph-mon[29478]: pgmap v245: 97 pgs: 97 active+clean; 5.3 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-04-25T10:40:30.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:29 smithi077 ceph-mon[32772]: pgmap v246: 97 pgs: 97 active+clean; 1.6 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 19 op/s 2024-04-25T10:40:30.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:29 smithi164 ceph-mon[32713]: pgmap v246: 97 pgs: 97 active+clean; 1.6 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 19 op/s 2024-04-25T10:40:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:29 smithi043 ceph-mon[29478]: pgmap v246: 97 pgs: 97 active+clean; 1.6 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 19 op/s 2024-04-25T10:40:32.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:31 smithi077 ceph-mon[32772]: pgmap v247: 97 pgs: 97 active+clean; 1.6 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:40:32.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:31 smithi164 ceph-mon[32713]: pgmap v247: 97 pgs: 97 active+clean; 1.6 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:40:32.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:31 smithi043 ceph-mon[29478]: pgmap v247: 97 pgs: 97 active+clean; 1.6 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:40:34.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:33 smithi077 ceph-mon[32772]: pgmap v248: 97 pgs: 97 active+clean; 1.6 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:40:34.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:33 smithi164 ceph-mon[32713]: pgmap v248: 97 pgs: 97 active+clean; 1.6 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:40:34.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:33 smithi043 ceph-mon[29478]: pgmap v248: 97 pgs: 97 active+clean; 1.6 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:40:36.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:35 smithi077 ceph-mon[32772]: pgmap v249: 97 pgs: 97 active+clean; 1.8 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-25T10:40:36.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:35 smithi164 ceph-mon[32713]: pgmap v249: 97 pgs: 97 active+clean; 1.8 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-25T10:40:36.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:35 smithi043 ceph-mon[29478]: pgmap v249: 97 pgs: 97 active+clean; 1.8 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-25T10:40:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.499+0000 7f36b5251640 1 -- 172.21.15.43:0/560592638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36b0109b70 msgr2=0x7f36b0103710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.499+0000 7f36b5251640 1 --2- 172.21.15.43:0/560592638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36b0109b70 0x7f36b0103710 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009900 tx=0x7f36a002a410 comp rx=0 tx=0).stop 2024-04-25T10:40:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.500+0000 7f36b5251640 1 -- 172.21.15.43:0/560592638 shutdown_connections 2024-04-25T10:40:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.500+0000 7f36b5251640 1 --2- 172.21.15.43:0/560592638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36b01042c0 0x7f36b0110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.500+0000 7f36b5251640 1 --2- 172.21.15.43:0/560592638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36b010a010 0x7f36b0103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.500+0000 7f36b5251640 1 --2- 172.21.15.43:0/560592638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36b0109b70 0x7f36b0103710 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.500+0000 7f36b5251640 1 -- 172.21.15.43:0/560592638 >> 172.21.15.43:0/560592638 conn(0x7f36b00ff210 msgr2=0x7f36b0101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.500+0000 7f36b5251640 1 -- 172.21.15.43:0/560592638 shutdown_connections 2024-04-25T10:40:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.500+0000 7f36b5251640 1 -- 172.21.15.43:0/560592638 wait complete. 2024-04-25T10:40:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.500+0000 7f36b5251640 1 Processor -- start 2024-04-25T10:40:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.501+0000 7f36b5251640 1 -- start start 2024-04-25T10:40:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.501+0000 7f36b5251640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36b01042c0 0x7f36b01a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.501+0000 7f36b5251640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36b0109b70 0x7f36b01a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.501+0000 7f36b5251640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36b010a010 0x7f36b019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.501+0000 7f36b5251640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f36b0113d60 con 0x7f36b010a010 2024-04-25T10:40:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.501+0000 7f36b5251640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f36b0113be0 con 0x7f36b0109b70 2024-04-25T10:40:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.501+0000 7f36b5251640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f36b0113ee0 con 0x7f36b01042c0 2024-04-25T10:40:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.501+0000 7f36af59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36b010a010 0x7f36b019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.501+0000 7f36ae59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36b0109b70 0x7f36b01a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.501+0000 7f36af59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36b010a010 0x7f36b019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52544/0 (socket says 172.21.15.43:52544) 2024-04-25T10:40:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.501+0000 7f36ae59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36b0109b70 0x7f36b01a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:52668/0 (socket says 172.21.15.43:52668) 2024-04-25T10:40:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.501+0000 7f36af59e640 1 -- 172.21.15.43:0/215925846 learned_addr learned my addr 172.21.15.43:0/215925846 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.501+0000 7f36aed9d640 1 --2- 172.21.15.43:0/215925846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36b01042c0 0x7f36b01a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.502+0000 7f36af59e640 1 -- 172.21.15.43:0/215925846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36b01042c0 msgr2=0x7f36b01a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.502+0000 7f36af59e640 1 --2- 172.21.15.43:0/215925846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36b01042c0 0x7f36b01a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.502+0000 7f36af59e640 1 -- 172.21.15.43:0/215925846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36b0109b70 msgr2=0x7f36b01a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.502+0000 7f36af59e640 1 --2- 172.21.15.43:0/215925846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36b0109b70 0x7f36b01a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.502+0000 7f36af59e640 1 -- 172.21.15.43:0/215925846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b019ace0 con 0x7f36b010a010 2024-04-25T10:40:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.502+0000 7f36ae59c640 1 --2- 172.21.15.43:0/215925846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36b0109b70 0x7f36b01a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:40:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.502+0000 7f36af59e640 1 --2- 172.21.15.43:0/215925846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36b010a010 0x7f36b019a6a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f36a800cce0 tx=0x7f36a800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.502+0000 7f3697fff640 1 -- 172.21.15.43:0/215925846 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a800f450 con 0x7f36b010a010 2024-04-25T10:40:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.502+0000 7f3697fff640 1 -- 172.21.15.43:0/215925846 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36a800faa0 con 0x7f36b010a010 2024-04-25T10:40:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.502+0000 7f36b5251640 1 -- 172.21.15.43:0/215925846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b019afd0 con 0x7f36b010a010 2024-04-25T10:40:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.502+0000 7f3697fff640 1 -- 172.21.15.43:0/215925846 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a8018b70 con 0x7f36b010a010 2024-04-25T10:40:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.502+0000 7f36b5251640 1 -- 172.21.15.43:0/215925846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b019b4d0 con 0x7f36b010a010 2024-04-25T10:40:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.505+0000 7f36bc1e7740 1 Processor -- start 2024-04-25T10:40:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.505+0000 7f3697fff640 1 -- 172.21.15.43:0/215925846 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f36a8018d10 con 0x7f36b010a010 2024-04-25T10:40:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.505+0000 7f36bc1e7740 1 -- start start 2024-04-25T10:40:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.505+0000 7f36bc1e7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c3a0df1b0 0x564c3a47eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.505+0000 7f36bc1e7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0b62f0 0x564c3a47f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.505+0000 7f36bc1e7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0be4d0 0x564c3a47f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.505+0000 7f36bc1e7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564c3a3e5460 con 0x564c3a0be4d0 2024-04-25T10:40:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.505+0000 7f36bc1e7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564c3a39f590 con 0x564c3a0df1b0 2024-04-25T10:40:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.505+0000 7f36bc1e7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564c3a3e1420 con 0x564c3a0b62f0 2024-04-25T10:40:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.506+0000 7f36ae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0be4d0 0x564c3a47f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.506+0000 7f36ae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0be4d0 0x564c3a47f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52556/0 (socket says 172.21.15.43:52556) 2024-04-25T10:40:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.506+0000 7f36ae59c640 1 -- 172.21.15.43:0/2674975972 learned_addr learned my addr 172.21.15.43:0/2674975972 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.506+0000 7f36aed9d640 1 --2- 172.21.15.43:0/2674975972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0b62f0 0x564c3a47f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.506+0000 7f36ae59c640 1 --2- 172.21.15.43:0/2674975972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c3a0df1b0 0x564c3a47eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.506+0000 7f3697fff640 1 --2- 172.21.15.43:0/215925846 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f36980afc50 0x7f36980b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.506+0000 7f36aed9d640 1 -- 172.21.15.43:0/2674975972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c3a0df1b0 msgr2=0x564c3a47eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.506+0000 7f36aed9d640 1 --2- 172.21.15.43:0/2674975972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c3a0df1b0 0x564c3a47eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.506+0000 7f36aed9d640 1 -- 172.21.15.43:0/2674975972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0be4d0 msgr2=0x564c3a47f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.506+0000 7f36aed9d640 1 --2- 172.21.15.43:0/2674975972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0be4d0 0x564c3a47f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.506+0000 7f36aed9d640 1 -- 172.21.15.43:0/2674975972 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c3a3ac0d0 con 0x564c3a0b62f0 2024-04-25T10:40:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.506+0000 7f36ae59c640 1 --2- 172.21.15.43:0/2674975972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0be4d0 0x564c3a47f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:40:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.506+0000 7f36af59e640 1 --2- 172.21.15.43:0/215925846 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f36980afc50 0x7f36980b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.506+0000 7f3697fff640 1 -- 172.21.15.43:0/215925846 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f36a80df430 con 0x7f36b010a010 2024-04-25T10:40:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.506+0000 7f36aed9d640 1 --2- 172.21.15.43:0/2674975972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0b62f0 0x564c3a47f030 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f36a0006010 tx=0x7f36a0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6cb7b2cb6dd0ed3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.507+0000 7f36957fa640 1 -- 172.21.15.43:0/2674975972 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0040070 con 0x564c3a0b62f0 2024-04-25T10:40:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.507+0000 7f36957fa640 1 -- 172.21.15.43:0/2674975972 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36a0005db0 con 0x564c3a0b62f0 2024-04-25T10:40:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.507+0000 7f36957fa640 1 -- 172.21.15.43:0/2674975972 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0035040 con 0x564c3a0b62f0 2024-04-25T10:40:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.507+0000 7f36af59e640 1 --2- 172.21.15.43:0/215925846 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f36980afc50 0x7f36980b2160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f36a80a7560 tx=0x7f36a800ff50 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.507+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2674975972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0b62f0 msgr2=0x564c3a47f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.508+0000 7f36bc1e7740 1 --2- 172.21.15.43:0/2674975972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0b62f0 0x564c3a47f030 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f36a0006010 tx=0x7f36a0000bc0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.508+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2674975972 shutdown_connections 2024-04-25T10:40:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.508+0000 7f36bc1e7740 1 --2- 172.21.15.43:0/2674975972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0be4d0 0x564c3a47f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.508+0000 7f36bc1e7740 1 --2- 172.21.15.43:0/2674975972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0b62f0 0x564c3a47f030 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.508+0000 7f36bc1e7740 1 --2- 172.21.15.43:0/2674975972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c3a0df1b0 0x564c3a47eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.508+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2674975972 >> 172.21.15.43:0/2674975972 conn(0x564c3a0d3300 msgr2=0x564c3a47b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.508+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2674975972 shutdown_connections 2024-04-25T10:40:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.508+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2674975972 wait complete. 2024-04-25T10:40:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.508+0000 7f36bc1e7740 1 -- start start 2024-04-25T10:40:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.508+0000 7f36bc1e7740 1 Processor -- start 2024-04-25T10:40:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.508+0000 7f36bc1e7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0b62f0 0x564c3a4891e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f36bc1e7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c3a0be4d0 0x564c3a489720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f36bc1e7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0df1b0 0x564c3a48f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f36bc1e7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564c3a48bbc0 con 0x564c3a0b62f0 2024-04-25T10:40:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f36bc1e7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564c3a36cb60 con 0x564c3a0be4d0 2024-04-25T10:40:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f36bc1e7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564c3a48bd40 con 0x564c3a0df1b0 2024-04-25T10:40:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f36bc1e7740 10 client.0 _ll_register_callbacks cb 0x564c3a3912b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f36ae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0b62f0 0x564c3a4891e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f36ae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0b62f0 0x564c3a4891e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52572/0 (socket says 172.21.15.43:52572) 2024-04-25T10:40:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f36ae59c640 1 -- 172.21.15.43:0/2624431891 learned_addr learned my addr 172.21.15.43:0/2624431891 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f36aed9d640 1 --2- 172.21.15.43:0/2624431891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c3a0be4d0 0x564c3a489720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f36ae59c640 1 --2- 172.21.15.43:0/2624431891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0df1b0 0x564c3a48f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f36ae59c640 1 -- 172.21.15.43:0/2624431891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0df1b0 msgr2=0x564c3a48f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f36ae59c640 1 --2- 172.21.15.43:0/2624431891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0df1b0 0x564c3a48f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f36ae59c640 1 -- 172.21.15.43:0/2624431891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c3a0be4d0 msgr2=0x564c3a489720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f36ae59c640 1 --2- 172.21.15.43:0/2624431891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c3a0be4d0 0x564c3a489720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f36ae59c640 1 -- 172.21.15.43:0/2624431891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c3a0f7210 con 0x564c3a0b62f0 2024-04-25T10:40:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f36ae59c640 1 --2- 172.21.15.43:0/2624431891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0b62f0 0x564c3a4891e0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f36a4012db0 tx=0x7f36a4014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf57536c1ef1c663 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f3692ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f3692ff5640 1 -- 172.21.15.43:0/2624431891 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4005220 con 0x564c3a0b62f0 2024-04-25T10:40:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f3692ff5640 1 -- 172.21.15.43:0/2624431891 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f36a40053c0 con 0x564c3a0b62f0 2024-04-25T10:40:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.509+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2624431891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36a0005db0 con 0x564c3a0b62f0 2024-04-25T10:40:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.510+0000 7f3692ff5640 1 -- 172.21.15.43:0/2624431891 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f36a402b050 con 0x564c3a0b62f0 2024-04-25T10:40:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.510+0000 7f36bc1e7740 10 client.17880 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.510+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2624431891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564c3a1c3620 con 0x564c3a0b62f0 2024-04-25T10:40:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.510+0000 7f3692ff5640 1 -- 172.21.15.43:0/2624431891 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36a0005db0 con 0x564c3a0b62f0 2024-04-25T10:40:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.510+0000 7f3692ff5640 1 -- 172.21.15.43:0/2624431891 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f36a4023590 con 0x564c3a0b62f0 2024-04-25T10:40:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.510+0000 7f36bc1e7740 10 client.17880 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.510+0000 7f36bc1e7740 10 client.17880 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.510+0000 7f36bc1e7740 20 client.17880 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.510+0000 7f36bc1e7740 1 --2- 172.21.15.43:0/2624431891 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564c3a09dd80 0x564c3a49b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.510+0000 7f36bc1e7740 4 client.17880 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.510+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2624431891 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c3a3f3070 con 0x564c3a09dd80 2024-04-25T10:40:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.510+0000 7f36aed9d640 1 --2- 172.21.15.43:0/2624431891 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564c3a09dd80 0x564c3a49b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.511+0000 7f3692ff5640 10 client.17880 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.511+0000 7f36aed9d640 1 --2- 172.21.15.43:0/2624431891 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564c3a09dd80 0x564c3a49b740 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6014d6ac6fe1bc84 server_cookie=db30ca9653c90454 in_seq=0 out_seq=0 2024-04-25T10:40:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.512+0000 7f3692ff5640 1 -- 172.21.15.43:0/2624431891 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564c3a3f3070 con 0x564c3a09dd80 2024-04-25T10:40:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.512+0000 7f3692ff5640 10 client.17880 handle_command_reply: tid=0 2024-04-25T10:40:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.512+0000 7f36bc1e7740 1 client.17880 shutdown 2024-04-25T10:40:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.512+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2624431891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0b62f0 msgr2=0x564c3a4891e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.512+0000 7f36bc1e7740 1 --2- 172.21.15.43:0/2624431891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0b62f0 0x564c3a4891e0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f36a4012db0 tx=0x7f36a4014040 comp rx=0 tx=0).stop 2024-04-25T10:40:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.512+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2624431891 shutdown_connections 2024-04-25T10:40:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.512+0000 7f36bc1e7740 1 --2- 172.21.15.43:0/2624431891 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564c3a09dd80 0x564c3a49b740 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.512+0000 7f36bc1e7740 1 --2- 172.21.15.43:0/2624431891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c3a0be4d0 0x564c3a489720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.512+0000 7f3692ff5640 0 client.17880 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.512+0000 7f36bc1e7740 1 --2- 172.21.15.43:0/2624431891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0df1b0 0x564c3a48f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.512+0000 7f36bc1e7740 1 --2- 172.21.15.43:0/2624431891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0b62f0 0x564c3a4891e0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.512+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2624431891 >> 172.21.15.43:0/2624431891 conn(0x564c3a0d3300 msgr2=0x564c3a47e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.512+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2624431891 shutdown_connections 2024-04-25T10:40:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.512+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2624431891 wait complete. 2024-04-25T10:40:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.512+0000 7f36bc1e7740 20 client.17880 trim_cache size 0 max 16384 2024-04-25T10:40:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.527+0000 7f36bc1e7740 1 Processor -- start 2024-04-25T10:40:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.527+0000 7f36bc1e7740 1 -- start start 2024-04-25T10:40:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.528+0000 7f36bc1e7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0b62f0 0x564c3a47ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.528+0000 7f36bc1e7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0be4d0 0x564c3a4a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.528+0000 7f36bc1e7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c3a0df1b0 0x564c3a4a1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.528+0000 7f36bc1e7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f36b01b2230 con 0x564c3a0b62f0 2024-04-25T10:40:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.528+0000 7f36bc1e7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f36b01afb00 con 0x564c3a0df1b0 2024-04-25T10:40:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.528+0000 7f36bc1e7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f36b01ad970 con 0x564c3a0be4d0 2024-04-25T10:40:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.528+0000 7f36ae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0b62f0 0x564c3a47ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.528+0000 7f36ae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0b62f0 0x564c3a47ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52584/0 (socket says 172.21.15.43:52584) 2024-04-25T10:40:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.528+0000 7f36ae59c640 1 -- 172.21.15.43:0/2146411660 learned_addr learned my addr 172.21.15.43:0/2146411660 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.528+0000 7f36aed9d640 1 --2- 172.21.15.43:0/2146411660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0be4d0 0x564c3a4a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.528+0000 7f36ae59c640 1 -- 172.21.15.43:0/2146411660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0be4d0 msgr2=0x564c3a4a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.528+0000 7f36ae59c640 1 --2- 172.21.15.43:0/2146411660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0be4d0 0x564c3a4a0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.528+0000 7f36ae59c640 1 -- 172.21.15.43:0/2146411660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c3a0df1b0 msgr2=0x564c3a4a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.528+0000 7f36ae59c640 1 --2- 172.21.15.43:0/2146411660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c3a0df1b0 0x564c3a4a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.528+0000 7f36ae59c640 1 -- 172.21.15.43:0/2146411660 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c3a47be70 con 0x564c3a0b62f0 2024-04-25T10:40:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.528+0000 7f36aed9d640 1 --2- 172.21.15.43:0/2146411660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0be4d0 0x564c3a4a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.528+0000 7f36ae59c640 1 --2- 172.21.15.43:0/2146411660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0b62f0 0x564c3a47ef90 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f36a4023ad0 tx=0x7f36a4012370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97539f01088be00e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.528+0000 7f3694ff9640 1 -- 172.21.15.43:0/2146411660 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a40233d0 con 0x564c3a0b62f0 2024-04-25T10:40:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.528+0000 7f3694ff9640 1 -- 172.21.15.43:0/2146411660 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36a4023570 con 0x564c3a0b62f0 2024-04-25T10:40:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.528+0000 7f3694ff9640 1 -- 172.21.15.43:0/2146411660 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4024c50 con 0x564c3a0b62f0 2024-04-25T10:40:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.529+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2146411660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0b62f0 msgr2=0x564c3a47ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.529+0000 7f36bc1e7740 1 --2- 172.21.15.43:0/2146411660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0b62f0 0x564c3a47ef90 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f36a4023ad0 tx=0x7f36a4012370 comp rx=0 tx=0).stop 2024-04-25T10:40:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.529+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2146411660 shutdown_connections 2024-04-25T10:40:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.529+0000 7f36bc1e7740 1 --2- 172.21.15.43:0/2146411660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c3a0df1b0 0x564c3a4a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.529+0000 7f36bc1e7740 1 --2- 172.21.15.43:0/2146411660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0be4d0 0x564c3a4a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.529+0000 7f36bc1e7740 1 --2- 172.21.15.43:0/2146411660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0b62f0 0x564c3a47ef90 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.529+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2146411660 >> 172.21.15.43:0/2146411660 conn(0x564c3a0d3300 msgr2=0x564c3a47e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.529+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2146411660 shutdown_connections 2024-04-25T10:40:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.529+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2146411660 wait complete. 2024-04-25T10:40:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.530+0000 7f36bc1e7740 1 -- start start 2024-04-25T10:40:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.530+0000 7f36bc1e7740 1 Processor -- start 2024-04-25T10:40:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.530+0000 7f36bc1e7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0b62f0 0x564c3a495990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.530+0000 7f36bc1e7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0be4d0 0x564c3a492d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.530+0000 7f36bc1e7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c3a0df1b0 0x564c3a493240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.530+0000 7f36bc1e7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f36b01adfc0 con 0x564c3a0be4d0 2024-04-25T10:40:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.530+0000 7f36bc1e7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f36b01af430 con 0x564c3a0df1b0 2024-04-25T10:40:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.530+0000 7f36bc1e7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f36b01ae690 con 0x564c3a0b62f0 2024-04-25T10:40:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.530+0000 7f36bc1e7740 10 client.0 _ll_register_callbacks cb 0x7f36a4012a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.530+0000 7f36aed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0be4d0 0x564c3a492d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.530+0000 7f36aed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0be4d0 0x564c3a492d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52594/0 (socket says 172.21.15.43:52594) 2024-04-25T10:40:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.530+0000 7f36aed9d640 1 -- 172.21.15.43:0/2923633409 learned_addr learned my addr 172.21.15.43:0/2923633409 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.530+0000 7f36ae59c640 1 --2- 172.21.15.43:0/2923633409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c3a0df1b0 0x564c3a493240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.530+0000 7f36ae59c640 1 --2- 172.21.15.43:0/2923633409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0b62f0 0x564c3a495990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.530+0000 7f36aed9d640 1 -- 172.21.15.43:0/2923633409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0b62f0 msgr2=0x564c3a495990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.530+0000 7f36aed9d640 1 --2- 172.21.15.43:0/2923633409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0b62f0 0x564c3a495990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.530+0000 7f36aed9d640 1 -- 172.21.15.43:0/2923633409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c3a0df1b0 msgr2=0x564c3a493240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.530+0000 7f36aed9d640 1 --2- 172.21.15.43:0/2923633409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c3a0df1b0 0x564c3a493240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.530+0000 7f36aed9d640 1 -- 172.21.15.43:0/2923633409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c3a4938e0 con 0x564c3a0be4d0 2024-04-25T10:40:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.530+0000 7f36ae59c640 1 --2- 172.21.15.43:0/2923633409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c3a0df1b0 0x564c3a493240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.531+0000 7f36aed9d640 1 --2- 172.21.15.43:0/2923633409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0be4d0 0x564c3a492d00 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f36a002a450 tx=0x7f36a002ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4239057bd24b12b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.531+0000 7f3692ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.531+0000 7f3692ff5640 1 -- 172.21.15.43:0/2923633409 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0035040 con 0x564c3a0be4d0 2024-04-25T10:40:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.531+0000 7f3692ff5640 1 -- 172.21.15.43:0/2923633409 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f36a0004020 con 0x564c3a0be4d0 2024-04-25T10:40:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.531+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2923633409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36a4023570 con 0x564c3a0be4d0 2024-04-25T10:40:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.531+0000 7f3692ff5640 1 -- 172.21.15.43:0/2923633409 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f36a0040040 con 0x564c3a0be4d0 2024-04-25T10:40:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.531+0000 7f36bc1e7740 10 client.17886 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.531+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2923633409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564c3a503900 con 0x564c3a0be4d0 2024-04-25T10:40:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.531+0000 7f3692ff5640 1 -- 172.21.15.43:0/2923633409 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36a4023570 con 0x564c3a0be4d0 2024-04-25T10:40:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.531+0000 7f3692ff5640 1 -- 172.21.15.43:0/2923633409 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f36a00042b0 con 0x564c3a0be4d0 2024-04-25T10:40:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.531+0000 7f36bc1e7740 10 client.17886 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.531+0000 7f36bc1e7740 10 client.17886 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.531+0000 7f36bc1e7740 20 client.17886 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.531+0000 7f36bc1e7740 1 --2- 172.21.15.43:0/2923633409 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564c3a0731c0 0x564c3a505d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.531+0000 7f36bc1e7740 4 client.17886 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:40:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.531+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2923633409 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564c3a3ecee0 con 0x564c3a0731c0 2024-04-25T10:40:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.532+0000 7f36ae59c640 1 --2- 172.21.15.43:0/2923633409 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564c3a0731c0 0x564c3a505d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.532+0000 7f36ae59c640 1 --2- 172.21.15.43:0/2923633409 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564c3a0731c0 0x564c3a505d60 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3482df4e264bef9 server_cookie=9f43d77f40e4f844 in_seq=0 out_seq=0 2024-04-25T10:40:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.532+0000 7f3692ff5640 10 client.17886 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.549+0000 7f3692ff5640 1 -- 172.21.15.43:0/2923633409 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564c3a3ecee0 con 0x564c3a0731c0 2024-04-25T10:40:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.549+0000 7f3692ff5640 10 client.17886 handle_command_reply: tid=0 2024-04-25T10:40:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.549+0000 7f36bc1e7740 1 client.17886 shutdown 2024-04-25T10:40:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.549+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2923633409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0be4d0 msgr2=0x564c3a492d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.549+0000 7f36bc1e7740 1 --2- 172.21.15.43:0/2923633409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0be4d0 0x564c3a492d00 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f36a002a450 tx=0x7f36a002ab10 comp rx=0 tx=0).stop 2024-04-25T10:40:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.549+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2923633409 shutdown_connections 2024-04-25T10:40:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.549+0000 7f36bc1e7740 1 --2- 172.21.15.43:0/2923633409 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564c3a0731c0 0x564c3a505d60 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.549+0000 7f36bc1e7740 1 --2- 172.21.15.43:0/2923633409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c3a0df1b0 0x564c3a493240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.549+0000 7f36bc1e7740 1 --2- 172.21.15.43:0/2923633409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c3a0be4d0 0x564c3a492d00 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.549+0000 7f3692ff5640 0 client.17886 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.549+0000 7f36bc1e7740 1 --2- 172.21.15.43:0/2923633409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c3a0b62f0 0x564c3a495990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.549+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2923633409 >> 172.21.15.43:0/2923633409 conn(0x564c3a0d3300 msgr2=0x564c3a48a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.549+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2923633409 shutdown_connections 2024-04-25T10:40:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.549+0000 7f36bc1e7740 1 -- 172.21.15.43:0/2923633409 wait complete. 2024-04-25T10:40:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.549+0000 7f36bc1e7740 20 client.17886 trim_cache size 0 max 16384 2024-04-25T10:40:37.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:40:37.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:40:37.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:40:37.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:40:37.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.550+0000 7f36b5251640 1 -- 172.21.15.43:0/215925846 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f36980afc50 msgr2=0x7f36980b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:37.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.550+0000 7f36b5251640 1 --2- 172.21.15.43:0/215925846 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f36980afc50 0x7f36980b2160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f36a80a7560 tx=0x7f36a800ff50 comp rx=0 tx=0).stop 2024-04-25T10:40:37.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.550+0000 7f36b5251640 1 -- 172.21.15.43:0/215925846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36b010a010 msgr2=0x7f36b019a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:37.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.550+0000 7f36b5251640 1 --2- 172.21.15.43:0/215925846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36b010a010 0x7f36b019a6a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f36a800cce0 tx=0x7f36a800a620 comp rx=0 tx=0).stop 2024-04-25T10:40:37.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.550+0000 7f36b5251640 1 -- 172.21.15.43:0/215925846 shutdown_connections 2024-04-25T10:40:37.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.550+0000 7f36b5251640 1 --2- 172.21.15.43:0/215925846 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f36980afc50 0x7f36980b2160 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.550+0000 7f36b5251640 1 --2- 172.21.15.43:0/215925846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36b0109b70 0x7f36b01a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.550+0000 7f36b5251640 1 --2- 172.21.15.43:0/215925846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36b010a010 0x7f36b019a6a0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.550+0000 7f36b5251640 1 --2- 172.21.15.43:0/215925846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36b01042c0 0x7f36b01a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.550+0000 7f36b5251640 1 -- 172.21.15.43:0/215925846 >> 172.21.15.43:0/215925846 conn(0x7f36b00ff210 msgr2=0x7f36b00ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.550+0000 7f36b5251640 1 -- 172.21.15.43:0/215925846 shutdown_connections 2024-04-25T10:40:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:37.550+0000 7f36b5251640 1 -- 172.21.15.43:0/215925846 wait complete. 2024-04-25T10:40:38.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:37 smithi077 ceph-mon[32772]: pgmap v250: 97 pgs: 97 active+clean; 1.8 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-25T10:40:38.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:37 smithi164 ceph-mon[32713]: pgmap v250: 97 pgs: 97 active+clean; 1.8 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-25T10:40:38.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:37 smithi043 ceph-mon[29478]: pgmap v250: 97 pgs: 97 active+clean; 1.8 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-25T10:40:40.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:39 smithi077 ceph-mon[32772]: pgmap v251: 97 pgs: 97 active+clean; 2.0 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 68 KiB/s wr, 20 op/s 2024-04-25T10:40:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:39 smithi164 ceph-mon[32713]: pgmap v251: 97 pgs: 97 active+clean; 2.0 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 68 KiB/s wr, 20 op/s 2024-04-25T10:40:40.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:39 smithi043 ceph-mon[29478]: pgmap v251: 97 pgs: 97 active+clean; 2.0 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 68 KiB/s wr, 20 op/s 2024-04-25T10:40:42.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:41 smithi077 ceph-mon[32772]: pgmap v252: 97 pgs: 97 active+clean; 2.0 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:40:42.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:41 smithi164 ceph-mon[32713]: pgmap v252: 97 pgs: 97 active+clean; 2.0 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:40:42.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:41 smithi043 ceph-mon[29478]: pgmap v252: 97 pgs: 97 active+clean; 2.0 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:40:44.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:43 smithi077 ceph-mon[32772]: pgmap v253: 97 pgs: 97 active+clean; 2.0 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:40:44.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:43 smithi164 ceph-mon[32713]: pgmap v253: 97 pgs: 97 active+clean; 2.0 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:40:44.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:43 smithi043 ceph-mon[29478]: pgmap v253: 97 pgs: 97 active+clean; 2.0 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:40:45.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:40:45.760 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:40:45.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.887+0000 7f02ea897640 1 -- 172.21.15.43:0/1998384641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02e4104270 msgr2=0x7f02e4110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:45.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.887+0000 7f02ea897640 1 --2- 172.21.15.43:0/1998384641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02e4104270 0x7f02e4110640 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f02dc009ca0 tx=0x7f02dc02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.887+0000 7f02ea897640 1 -- 172.21.15.43:0/1998384641 shutdown_connections 2024-04-25T10:40:45.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.887+0000 7f02ea897640 1 --2- 172.21.15.43:0/1998384641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02e4109fc0 0x7f02e4103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.887+0000 7f02ea897640 1 --2- 172.21.15.43:0/1998384641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02e4104270 0x7f02e4110640 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.887+0000 7f02ea897640 1 --2- 172.21.15.43:0/1998384641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02e4109b20 0x7f02e41036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.887+0000 7f02ea897640 1 -- 172.21.15.43:0/1998384641 >> 172.21.15.43:0/1998384641 conn(0x7f02e40ff1e0 msgr2=0x7f02e4101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:45.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.887+0000 7f02ea897640 1 -- 172.21.15.43:0/1998384641 shutdown_connections 2024-04-25T10:40:45.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.887+0000 7f02ea897640 1 -- 172.21.15.43:0/1998384641 wait complete. 2024-04-25T10:40:45.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.888+0000 7f02ea897640 1 Processor -- start 2024-04-25T10:40:45.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.888+0000 7f02ea897640 1 -- start start 2024-04-25T10:40:45.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.888+0000 7f02ea897640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02e4104270 0x7f02e419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:45.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.888+0000 7f02ea897640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02e4109b20 0x7f02e419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:45.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.888+0000 7f02ea897640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02e4109fc0 0x7f02e419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:45.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.888+0000 7f02ea897640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f02e4113d10 con 0x7f02e4109b20 2024-04-25T10:40:45.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.888+0000 7f02ea897640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f02e4113b90 con 0x7f02e4109fc0 2024-04-25T10:40:45.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.888+0000 7f02ea897640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f02e4113e90 con 0x7f02e4104270 2024-04-25T10:40:45.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.888+0000 7f02e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02e4109b20 0x7f02e419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:45.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.888+0000 7f02e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02e4109b20 0x7f02e419ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41902/0 (socket says 172.21.15.43:41902) 2024-04-25T10:40:45.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.888+0000 7f02e37fe640 1 -- 172.21.15.43:0/873907918 learned_addr learned my addr 172.21.15.43:0/873907918 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:45.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.888+0000 7f02e3fff640 1 --2- 172.21.15.43:0/873907918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02e4104270 0x7f02e419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:45.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.888+0000 7f02e8e34640 1 --2- 172.21.15.43:0/873907918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02e4109fc0 0x7f02e419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:45.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.889+0000 7f02e37fe640 1 -- 172.21.15.43:0/873907918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02e4104270 msgr2=0x7f02e419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:45.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.889+0000 7f02e37fe640 1 --2- 172.21.15.43:0/873907918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02e4104270 0x7f02e419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.889+0000 7f02e37fe640 1 -- 172.21.15.43:0/873907918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02e4109fc0 msgr2=0x7f02e419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:45.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.889+0000 7f02e37fe640 1 --2- 172.21.15.43:0/873907918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02e4109fc0 0x7f02e419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.889+0000 7f02e37fe640 1 -- 172.21.15.43:0/873907918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e419f4d0 con 0x7f02e4109b20 2024-04-25T10:40:45.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.889+0000 7f02e3fff640 1 --2- 172.21.15.43:0/873907918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02e4104270 0x7f02e419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:45.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.889+0000 7f02e37fe640 1 --2- 172.21.15.43:0/873907918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02e4109b20 0x7f02e419ac50 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f02d800d770 tx=0x7f02d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:45.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.889+0000 7f02e17fa640 1 -- 172.21.15.43:0/873907918 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8017070 con 0x7f02e4109b20 2024-04-25T10:40:45.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.889+0000 7f02ea897640 1 -- 172.21.15.43:0/873907918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e419f7c0 con 0x7f02e4109b20 2024-04-25T10:40:45.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.889+0000 7f02e17fa640 1 -- 172.21.15.43:0/873907918 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02d800de80 con 0x7f02e4109b20 2024-04-25T10:40:45.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.889+0000 7f02e17fa640 1 -- 172.21.15.43:0/873907918 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8003c30 con 0x7f02e4109b20 2024-04-25T10:40:45.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.889+0000 7f02ea897640 1 -- 172.21.15.43:0/873907918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e419fd10 con 0x7f02e4109b20 2024-04-25T10:40:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.892+0000 7f02e17fa640 1 -- 172.21.15.43:0/873907918 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f02d8003dd0 con 0x7f02e4109b20 2024-04-25T10:40:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.892+0000 7f02f1822740 1 Processor -- start 2024-04-25T10:40:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.892+0000 7f02f1822740 1 -- start start 2024-04-25T10:40:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.892+0000 7f02f1822740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557201680390 0x557201a20910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.892+0000 7f02f1822740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572016574d0 0x557201a20e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.892+0000 7f02f1822740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55720165f6b0 0x557201a31d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.892+0000 7f02f1822740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557201986d80 con 0x557201680390 2024-04-25T10:40:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.892+0000 7f02f1822740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5572019408b0 con 0x5572016574d0 2024-04-25T10:40:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.892+0000 7f02f1822740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557201982d40 con 0x55720165f6b0 2024-04-25T10:40:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.892+0000 7f02e17fa640 1 --2- 172.21.15.43:0/873907918 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f02cc0afc60 0x7f02cc0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.892+0000 7f02e8e34640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557201680390 0x557201a20910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.893+0000 7f02e8e34640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557201680390 0x557201a20910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41916/0 (socket says 172.21.15.43:41916) 2024-04-25T10:40:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.893+0000 7f02e8e34640 1 -- 172.21.15.43:0/2417283670 learned_addr learned my addr 172.21.15.43:0/2417283670 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.893+0000 7f02e3fff640 1 --2- 172.21.15.43:0/2417283670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572016574d0 0x557201a20e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.893+0000 7f02e37fe640 1 --2- 172.21.15.43:0/2417283670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55720165f6b0 0x557201a31d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.893+0000 7f02e8e34640 1 --2- 172.21.15.43:0/873907918 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f02cc0afc60 0x7f02cc0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.893+0000 7f02e17fa640 1 -- 172.21.15.43:0/873907918 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f02d80dead0 con 0x7f02e4109b20 2024-04-25T10:40:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.893+0000 7f02e8e34640 1 -- 172.21.15.43:0/2417283670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55720165f6b0 msgr2=0x557201a31d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.893+0000 7f02e8e34640 1 --2- 172.21.15.43:0/2417283670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55720165f6b0 0x557201a31d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.893+0000 7f02e8e34640 1 -- 172.21.15.43:0/2417283670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572016574d0 msgr2=0x557201a20e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.893+0000 7f02e8e34640 1 --2- 172.21.15.43:0/2417283670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572016574d0 0x557201a20e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.893+0000 7f02e8e34640 1 -- 172.21.15.43:0/2417283670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557201911a40 con 0x557201680390 2024-04-25T10:40:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.893+0000 7f02e8e34640 1 --2- 172.21.15.43:0/2417283670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557201680390 0x557201a20910 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f02dc0094e0 tx=0x7f02dc031690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac30125ddf4e26fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.893+0000 7f02e8e34640 1 --2- 172.21.15.43:0/873907918 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f02cc0afc60 0x7f02cc0b2170 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f02dc031700 tx=0x7f02dc036000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.893+0000 7f02ca7fc640 1 -- 172.21.15.43:0/2417283670 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc04d070 con 0x557201680390 2024-04-25T10:40:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.894+0000 7f02ca7fc640 1 -- 172.21.15.43:0/2417283670 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02dc036070 con 0x557201680390 2024-04-25T10:40:45.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.894+0000 7f02ca7fc640 1 -- 172.21.15.43:0/2417283670 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc02f660 con 0x557201680390 2024-04-25T10:40:45.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.894+0000 7f02f1822740 1 -- 172.21.15.43:0/2417283670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557201680390 msgr2=0x557201a20910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:45.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.894+0000 7f02f1822740 1 --2- 172.21.15.43:0/2417283670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557201680390 0x557201a20910 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f02dc0094e0 tx=0x7f02dc031690 comp rx=0 tx=0).stop 2024-04-25T10:40:45.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.894+0000 7f02f1822740 1 -- 172.21.15.43:0/2417283670 shutdown_connections 2024-04-25T10:40:45.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.894+0000 7f02f1822740 1 --2- 172.21.15.43:0/2417283670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572016574d0 0x557201a20e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.894+0000 7f02f1822740 1 --2- 172.21.15.43:0/2417283670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55720165f6b0 0x557201a31d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.894+0000 7f02f1822740 1 --2- 172.21.15.43:0/2417283670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557201680390 0x557201a20910 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.894+0000 7f02f1822740 1 -- 172.21.15.43:0/2417283670 >> 172.21.15.43:0/2417283670 conn(0x5572016744e0 msgr2=0x557201a1d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:45.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.894+0000 7f02f1822740 1 -- 172.21.15.43:0/2417283670 shutdown_connections 2024-04-25T10:40:45.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.894+0000 7f02f1822740 1 -- 172.21.15.43:0/2417283670 wait complete. 2024-04-25T10:40:45.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.895+0000 7f02f1822740 1 -- start start 2024-04-25T10:40:45.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.895+0000 7f02f1822740 1 Processor -- start 2024-04-25T10:40:45.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.895+0000 7f02f1822740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572016574d0 0x557201a2fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:45.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.895+0000 7f02f1822740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720165f6b0 0x557201a2ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:45.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.895+0000 7f02f1822740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557201680390 0x557201a358d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:45.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.896+0000 7f02f1822740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557201a32390 con 0x55720165f6b0 2024-04-25T10:40:45.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.896+0000 7f02f1822740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55720190dd10 con 0x557201680390 2024-04-25T10:40:45.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.896+0000 7f02f1822740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557201a32510 con 0x5572016574d0 2024-04-25T10:40:45.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.896+0000 7f02f1822740 10 client.0 _ll_register_callbacks cb 0x55720193a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:45.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.896+0000 7f02e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720165f6b0 0x557201a2ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:45.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.896+0000 7f02e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720165f6b0 0x557201a2ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41930/0 (socket says 172.21.15.43:41930) 2024-04-25T10:40:45.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.896+0000 7f02e37fe640 1 -- 172.21.15.43:0/2185934606 learned_addr learned my addr 172.21.15.43:0/2185934606 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:45.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.896+0000 7f02e3fff640 1 --2- 172.21.15.43:0/2185934606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572016574d0 0x557201a2fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:45.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.896+0000 7f02e8e34640 1 --2- 172.21.15.43:0/2185934606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557201680390 0x557201a358d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:45.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.896+0000 7f02e37fe640 1 -- 172.21.15.43:0/2185934606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572016574d0 msgr2=0x557201a2fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:45.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.896+0000 7f02e37fe640 1 --2- 172.21.15.43:0/2185934606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572016574d0 0x557201a2fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.896+0000 7f02e37fe640 1 -- 172.21.15.43:0/2185934606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557201680390 msgr2=0x557201a358d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:45.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.896+0000 7f02e37fe640 1 --2- 172.21.15.43:0/2185934606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557201680390 0x557201a358d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.896+0000 7f02e37fe640 1 -- 172.21.15.43:0/2185934606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572017627d0 con 0x55720165f6b0 2024-04-25T10:40:45.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.896+0000 7f02e37fe640 1 --2- 172.21.15.43:0/2185934606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720165f6b0 0x557201a2ff80 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f02d80dc820 tx=0x7f02d8018f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcab8b56d21bb02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:45.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.896+0000 7f02c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:45.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.896+0000 7f02c37fe640 1 -- 172.21.15.43:0/2185934606 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8055e60 con 0x55720165f6b0 2024-04-25T10:40:45.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.896+0000 7f02c37fe640 1 -- 172.21.15.43:0/2185934606 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f02d80a2f00 con 0x55720165f6b0 2024-04-25T10:40:45.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.896+0000 7f02f1822740 1 -- 172.21.15.43:0/2185934606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02dc036070 con 0x55720165f6b0 2024-04-25T10:40:45.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.897+0000 7f02c37fe640 1 -- 172.21.15.43:0/2185934606 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f02d8020040 con 0x55720165f6b0 2024-04-25T10:40:45.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.897+0000 7f02f1822740 10 client.17898 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:45.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.897+0000 7f02f1822740 1 -- 172.21.15.43:0/2185934606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5572016983f0 con 0x55720165f6b0 2024-04-25T10:40:45.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.897+0000 7f02c37fe640 1 -- 172.21.15.43:0/2185934606 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02dc036070 con 0x55720165f6b0 2024-04-25T10:40:45.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.897+0000 7f02c37fe640 1 -- 172.21.15.43:0/2185934606 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f02d8020cc0 con 0x55720165f6b0 2024-04-25T10:40:45.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.897+0000 7f02f1822740 10 client.17898 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:45.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.897+0000 7f02f1822740 10 client.17898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:45.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.897+0000 7f02f1822740 20 client.17898 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:45.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.897+0000 7f02f1822740 1 --2- 172.21.15.43:0/2185934606 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55720163ef60 0x557201a41b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:45.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.897+0000 7f02f1822740 4 client.17898 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.897+0000 7f02f1822740 1 -- 172.21.15.43:0/2185934606 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557201994990 con 0x55720163ef60 2024-04-25T10:40:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.897+0000 7f02e3fff640 1 --2- 172.21.15.43:0/2185934606 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55720163ef60 0x557201a41b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.898+0000 7f02c37fe640 10 client.17898 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.898+0000 7f02e3fff640 1 --2- 172.21.15.43:0/2185934606 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55720163ef60 0x557201a41b30 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45aaa2c3f49ea6b7 server_cookie=806efc4d6b32a48a in_seq=0 out_seq=0 2024-04-25T10:40:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.898+0000 7f02c37fe640 1 -- 172.21.15.43:0/2185934606 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557201994990 con 0x55720163ef60 2024-04-25T10:40:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.898+0000 7f02c37fe640 10 client.17898 handle_command_reply: tid=0 2024-04-25T10:40:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.899+0000 7f02f1822740 1 client.17898 shutdown 2024-04-25T10:40:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.899+0000 7f02f1822740 1 -- 172.21.15.43:0/2185934606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720165f6b0 msgr2=0x557201a2ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.899+0000 7f02f1822740 1 --2- 172.21.15.43:0/2185934606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720165f6b0 0x557201a2ff80 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f02d80dc820 tx=0x7f02d8018f10 comp rx=0 tx=0).stop 2024-04-25T10:40:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.899+0000 7f02f1822740 1 -- 172.21.15.43:0/2185934606 shutdown_connections 2024-04-25T10:40:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.899+0000 7f02f1822740 1 --2- 172.21.15.43:0/2185934606 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55720163ef60 0x557201a41b30 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.899+0000 7f02f1822740 1 --2- 172.21.15.43:0/2185934606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557201680390 0x557201a358d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.899+0000 7f02f1822740 1 --2- 172.21.15.43:0/2185934606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720165f6b0 0x557201a2ff80 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.899+0000 7f02c37fe640 0 client.17898 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:45.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.899+0000 7f02f1822740 1 --2- 172.21.15.43:0/2185934606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572016574d0 0x557201a2fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.899+0000 7f02f1822740 1 -- 172.21.15.43:0/2185934606 >> 172.21.15.43:0/2185934606 conn(0x5572016744e0 msgr2=0x557201a226f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:45.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.899+0000 7f02f1822740 1 -- 172.21.15.43:0/2185934606 shutdown_connections 2024-04-25T10:40:45.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.899+0000 7f02f1822740 1 -- 172.21.15.43:0/2185934606 wait complete. 2024-04-25T10:40:45.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.899+0000 7f02f1822740 20 client.17898 trim_cache size 0 max 16384 2024-04-25T10:40:45.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.917+0000 7f02f1822740 1 Processor -- start 2024-04-25T10:40:45.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.917+0000 7f02f1822740 1 -- start start 2024-04-25T10:40:45.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.918+0000 7f02f1822740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572016744e0 0x557201a35840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:45.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.918+0000 7f02f1822740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572016574d0 0x557201a2fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:45.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.918+0000 7f02f1822740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55720165f6b0 0x557201a472d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:45.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.918+0000 7f02f1822740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f02e41b0010 con 0x5572016744e0 2024-04-25T10:40:45.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.918+0000 7f02f1822740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f02e41ad8e0 con 0x5572016574d0 2024-04-25T10:40:45.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.918+0000 7f02f1822740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f02e41ab750 con 0x55720165f6b0 2024-04-25T10:40:45.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.918+0000 7f02e3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572016744e0 0x557201a35840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:45.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.918+0000 7f02e3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572016744e0 0x557201a35840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41932/0 (socket says 172.21.15.43:41932) 2024-04-25T10:40:45.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.918+0000 7f02e3fff640 1 -- 172.21.15.43:0/4240562425 learned_addr learned my addr 172.21.15.43:0/4240562425 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:45.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.918+0000 7f02e37fe640 1 --2- 172.21.15.43:0/4240562425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572016574d0 0x557201a2fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:45.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.918+0000 7f02e8e34640 1 --2- 172.21.15.43:0/4240562425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55720165f6b0 0x557201a472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:45.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.918+0000 7f02e3fff640 1 -- 172.21.15.43:0/4240562425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55720165f6b0 msgr2=0x557201a472d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:45.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.918+0000 7f02e3fff640 1 --2- 172.21.15.43:0/4240562425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55720165f6b0 0x557201a472d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.918+0000 7f02e3fff640 1 -- 172.21.15.43:0/4240562425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572016574d0 msgr2=0x557201a2fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:45.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.918+0000 7f02e3fff640 1 --2- 172.21.15.43:0/4240562425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572016574d0 0x557201a2fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.918+0000 7f02e3fff640 1 -- 172.21.15.43:0/4240562425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572016580f0 con 0x5572016744e0 2024-04-25T10:40:45.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.918+0000 7f02e37fe640 1 --2- 172.21.15.43:0/4240562425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572016574d0 0x557201a2fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:45.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.918+0000 7f02e3fff640 1 --2- 172.21.15.43:0/4240562425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572016744e0 0x557201a35840 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f02d4005fd0 tx=0x7f02d40128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d3f5d91413d11c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:45.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.918+0000 7f02c9ffb640 1 -- 172.21.15.43:0/4240562425 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d4004020 con 0x5572016744e0 2024-04-25T10:40:45.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.919+0000 7f02c9ffb640 1 -- 172.21.15.43:0/4240562425 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02d401c8e0 con 0x5572016744e0 2024-04-25T10:40:45.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.919+0000 7f02c9ffb640 1 -- 172.21.15.43:0/4240562425 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d4003a10 con 0x5572016744e0 2024-04-25T10:40:45.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.919+0000 7f02f1822740 1 -- 172.21.15.43:0/4240562425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572016744e0 msgr2=0x557201a35840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:45.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.919+0000 7f02f1822740 1 --2- 172.21.15.43:0/4240562425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572016744e0 0x557201a35840 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f02d4005fd0 tx=0x7f02d40128e0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.919+0000 7f02f1822740 1 -- 172.21.15.43:0/4240562425 shutdown_connections 2024-04-25T10:40:45.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.919+0000 7f02f1822740 1 --2- 172.21.15.43:0/4240562425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572016574d0 0x557201a2fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.919+0000 7f02f1822740 1 --2- 172.21.15.43:0/4240562425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55720165f6b0 0x557201a472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.919+0000 7f02f1822740 1 --2- 172.21.15.43:0/4240562425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572016744e0 0x557201a35840 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.919+0000 7f02f1822740 1 -- 172.21.15.43:0/4240562425 >> 172.21.15.43:0/4240562425 conn(0x557201a1d3a0 msgr2=0x557201a2a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:45.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.919+0000 7f02f1822740 1 -- 172.21.15.43:0/4240562425 shutdown_connections 2024-04-25T10:40:45.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.919+0000 7f02f1822740 1 -- 172.21.15.43:0/4240562425 wait complete. 2024-04-25T10:40:45.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.920+0000 7f02f1822740 1 -- start start 2024-04-25T10:40:45.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.920+0000 7f02f1822740 1 Processor -- start 2024-04-25T10:40:45.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.920+0000 7f02f1822740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572016574d0 0x557201a3b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:45.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.920+0000 7f02f1822740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720165f6b0 0x557201a3b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:45.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.920+0000 7f02f1822740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572016744e0 0x557201a36950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:45.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.920+0000 7f02f1822740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f02e41abda0 con 0x55720165f6b0 2024-04-25T10:40:45.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.920+0000 7f02f1822740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f02e41ad210 con 0x5572016574d0 2024-04-25T10:40:45.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.920+0000 7f02f1822740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f02e41ac470 con 0x5572016744e0 2024-04-25T10:40:45.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.920+0000 7f02f1822740 10 client.0 _ll_register_callbacks cb 0x7f02d80a24c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:45.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.920+0000 7f02e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720165f6b0 0x557201a3b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:45.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.920+0000 7f02e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720165f6b0 0x557201a3b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41944/0 (socket says 172.21.15.43:41944) 2024-04-25T10:40:45.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.920+0000 7f02e37fe640 1 -- 172.21.15.43:0/3668591538 learned_addr learned my addr 172.21.15.43:0/3668591538 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:45.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.920+0000 7f02e8e34640 1 --2- 172.21.15.43:0/3668591538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572016744e0 0x557201a36950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:45.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.920+0000 7f02e3fff640 1 --2- 172.21.15.43:0/3668591538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572016574d0 0x557201a3b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:45.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.921+0000 7f02e37fe640 1 -- 172.21.15.43:0/3668591538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572016744e0 msgr2=0x557201a36950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:45.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.921+0000 7f02e37fe640 1 --2- 172.21.15.43:0/3668591538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572016744e0 0x557201a36950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.921+0000 7f02e37fe640 1 -- 172.21.15.43:0/3668591538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572016574d0 msgr2=0x557201a3b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:45.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.921+0000 7f02e37fe640 1 --2- 172.21.15.43:0/3668591538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572016574d0 0x557201a3b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.921+0000 7f02e37fe640 1 -- 172.21.15.43:0/3668591538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572019474b0 con 0x55720165f6b0 2024-04-25T10:40:45.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.921+0000 7f02e3fff640 1 --2- 172.21.15.43:0/3668591538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572016574d0 0x557201a3b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:45.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.921+0000 7f02e37fe640 1 --2- 172.21.15.43:0/3668591538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720165f6b0 0x557201a3b960 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f02d80a2570 tx=0x7f02d80a3110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30e7719892434e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:45.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.921+0000 7f02c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:45.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.921+0000 7f02c37fe640 1 -- 172.21.15.43:0/3668591538 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8055e60 con 0x55720165f6b0 2024-04-25T10:40:45.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.921+0000 7f02c37fe640 1 -- 172.21.15.43:0/3668591538 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f02d80a2250 con 0x55720165f6b0 2024-04-25T10:40:45.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.921+0000 7f02f1822740 1 -- 172.21.15.43:0/3668591538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02d401c8e0 con 0x55720165f6b0 2024-04-25T10:40:45.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.921+0000 7f02c37fe640 1 -- 172.21.15.43:0/3668591538 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f02d80a65c0 con 0x55720165f6b0 2024-04-25T10:40:45.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.921+0000 7f02f1822740 10 client.17904 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:45.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.921+0000 7f02f1822740 1 -- 172.21.15.43:0/3668591538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557201aa9590 con 0x55720165f6b0 2024-04-25T10:40:45.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.921+0000 7f02c37fe640 1 -- 172.21.15.43:0/3668591538 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02d401c8e0 con 0x55720165f6b0 2024-04-25T10:40:45.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.921+0000 7f02c37fe640 1 -- 172.21.15.43:0/3668591538 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f02d80a25e0 con 0x55720165f6b0 2024-04-25T10:40:45.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.922+0000 7f02f1822740 10 client.17904 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:45.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.922+0000 7f02f1822740 10 client.17904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:45.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.922+0000 7f02f1822740 20 client.17904 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:45.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.922+0000 7f02f1822740 1 --2- 172.21.15.43:0/3668591538 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557201680390 0x557201aabc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:45.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.922+0000 7f02f1822740 4 client.17904 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:40:45.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.922+0000 7f02f1822740 1 -- 172.21.15.43:0/3668591538 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55720198e800 con 0x557201680390 2024-04-25T10:40:45.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.922+0000 7f02e3fff640 1 --2- 172.21.15.43:0/3668591538 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557201680390 0x557201aabc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:45.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.922+0000 7f02c37fe640 10 client.17904 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:45.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.922+0000 7f02e3fff640 1 --2- 172.21.15.43:0/3668591538 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557201680390 0x557201aabc60 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f094e71fbd53b684 server_cookie=5e30ff924cf33f10 in_seq=0 out_seq=0 2024-04-25T10:40:45.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.922+0000 7f02c37fe640 1 -- 172.21.15.43:0/3668591538 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55720198e800 con 0x557201680390 2024-04-25T10:40:45.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.922+0000 7f02c37fe640 10 client.17904 handle_command_reply: tid=0 2024-04-25T10:40:45.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.922+0000 7f02f1822740 1 client.17904 shutdown 2024-04-25T10:40:45.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.923+0000 7f02f1822740 1 -- 172.21.15.43:0/3668591538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720165f6b0 msgr2=0x557201a3b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:45.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.923+0000 7f02f1822740 1 --2- 172.21.15.43:0/3668591538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720165f6b0 0x557201a3b960 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f02d80a2570 tx=0x7f02d80a3110 comp rx=0 tx=0).stop 2024-04-25T10:40:45.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.923+0000 7f02f1822740 1 -- 172.21.15.43:0/3668591538 shutdown_connections 2024-04-25T10:40:45.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.923+0000 7f02f1822740 1 --2- 172.21.15.43:0/3668591538 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557201680390 0x557201aabc60 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.923+0000 7f02f1822740 1 --2- 172.21.15.43:0/3668591538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572016744e0 0x557201a36950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.923+0000 7f02f1822740 1 --2- 172.21.15.43:0/3668591538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720165f6b0 0x557201a3b960 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.923+0000 7f02c37fe640 0 client.17904 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:45.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.923+0000 7f02f1822740 1 --2- 172.21.15.43:0/3668591538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572016574d0 0x557201a3b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.923+0000 7f02f1822740 1 -- 172.21.15.43:0/3668591538 >> 172.21.15.43:0/3668591538 conn(0x557201a1d3a0 msgr2=0x557201a22660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:45.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.923+0000 7f02f1822740 1 -- 172.21.15.43:0/3668591538 shutdown_connections 2024-04-25T10:40:45.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.923+0000 7f02f1822740 1 -- 172.21.15.43:0/3668591538 wait complete. 2024-04-25T10:40:45.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.923+0000 7f02f1822740 20 client.17904 trim_cache size 0 max 16384 2024-04-25T10:40:45.932 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:40:45.932 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:40:45.932 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:40:45.933 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:40:45.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.923+0000 7f02ea897640 1 -- 172.21.15.43:0/873907918 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f02cc0afc60 msgr2=0x7f02cc0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:45.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.923+0000 7f02ea897640 1 --2- 172.21.15.43:0/873907918 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f02cc0afc60 0x7f02cc0b2170 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f02dc031700 tx=0x7f02dc036000 comp rx=0 tx=0).stop 2024-04-25T10:40:45.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.923+0000 7f02ea897640 1 -- 172.21.15.43:0/873907918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02e4109b20 msgr2=0x7f02e419ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:45.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.923+0000 7f02ea897640 1 --2- 172.21.15.43:0/873907918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02e4109b20 0x7f02e419ac50 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f02d800d770 tx=0x7f02d800dc30 comp rx=0 tx=0).stop 2024-04-25T10:40:45.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.924+0000 7f02ea897640 1 -- 172.21.15.43:0/873907918 shutdown_connections 2024-04-25T10:40:45.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.924+0000 7f02ea897640 1 --2- 172.21.15.43:0/873907918 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f02cc0afc60 0x7f02cc0b2170 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.924+0000 7f02ea897640 1 --2- 172.21.15.43:0/873907918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02e4109fc0 0x7f02e419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.924+0000 7f02ea897640 1 --2- 172.21.15.43:0/873907918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02e4109b20 0x7f02e419ac50 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.924+0000 7f02ea897640 1 --2- 172.21.15.43:0/873907918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02e4104270 0x7f02e419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:45.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.924+0000 7f02ea897640 1 -- 172.21.15.43:0/873907918 >> 172.21.15.43:0/873907918 conn(0x7f02e40ff1e0 msgr2=0x7f02e41065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:45.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.924+0000 7f02ea897640 1 -- 172.21.15.43:0/873907918 shutdown_connections 2024-04-25T10:40:45.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:45.924+0000 7f02ea897640 1 -- 172.21.15.43:0/873907918 wait complete. 2024-04-25T10:40:45.937 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:40:45.937 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:40:46.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.048+0000 7fe7d72aa640 1 -- 172.21.15.43:0/414246722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d010e0e0 msgr2=0x7fe7d0110560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.048+0000 7fe7d72aa640 1 --2- 172.21.15.43:0/414246722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d010e0e0 0x7fe7d0110560 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009ca0 tx=0x7fe7c802a950 comp rx=0 tx=0).stop 2024-04-25T10:40:46.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.048+0000 7fe7d72aa640 1 -- 172.21.15.43:0/414246722 shutdown_connections 2024-04-25T10:40:46.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.048+0000 7fe7d72aa640 1 --2- 172.21.15.43:0/414246722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d010b780 0x7fe7d010dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.048+0000 7fe7d72aa640 1 --2- 172.21.15.43:0/414246722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d010e0e0 0x7fe7d0110560 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.048+0000 7fe7d72aa640 1 --2- 172.21.15.43:0/414246722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d0104d20 0x7fe7d0105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.048+0000 7fe7d72aa640 1 -- 172.21.15.43:0/414246722 >> 172.21.15.43:0/414246722 conn(0x7fe7d0078040 msgr2=0x7fe7d0078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:46.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.048+0000 7fe7d72aa640 1 -- 172.21.15.43:0/414246722 shutdown_connections 2024-04-25T10:40:46.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.048+0000 7fe7d72aa640 1 -- 172.21.15.43:0/414246722 wait complete. 2024-04-25T10:40:46.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.049+0000 7fe7d72aa640 1 Processor -- start 2024-04-25T10:40:46.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.049+0000 7fe7d72aa640 1 -- start start 2024-04-25T10:40:46.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.049+0000 7fe7d72aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d0104d20 0x7fe7d019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.049+0000 7fe7d72aa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d010b780 0x7fe7d019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.049+0000 7fe7d72aa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d010e0e0 0x7fe7d019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.049+0000 7fe7d72aa640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d0113cd0 con 0x7fe7d0104d20 2024-04-25T10:40:46.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.049+0000 7fe7d72aa640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d0113b50 con 0x7fe7d010e0e0 2024-04-25T10:40:46.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.049+0000 7fe7d72aa640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d0113e50 con 0x7fe7d010b780 2024-04-25T10:40:46.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.050+0000 7fe7d5046640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d0104d20 0x7fe7d019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.050+0000 7fe7d5046640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d0104d20 0x7fe7d019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41958/0 (socket says 172.21.15.43:41958) 2024-04-25T10:40:46.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.050+0000 7fe7d5046640 1 -- 172.21.15.43:0/331184130 learned_addr learned my addr 172.21.15.43:0/331184130 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:46.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.050+0000 7fe7d4845640 1 --2- 172.21.15.43:0/331184130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d010b780 0x7fe7d019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.050+0000 7fe7d5847640 1 --2- 172.21.15.43:0/331184130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d010e0e0 0x7fe7d019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.050+0000 7fe7d5046640 1 -- 172.21.15.43:0/331184130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d010b780 msgr2=0x7fe7d019ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.050+0000 7fe7d5046640 1 --2- 172.21.15.43:0/331184130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d010b780 0x7fe7d019ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.050+0000 7fe7d5046640 1 -- 172.21.15.43:0/331184130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d010e0e0 msgr2=0x7fe7d019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.050+0000 7fe7d5046640 1 --2- 172.21.15.43:0/331184130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d010e0e0 0x7fe7d019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.050+0000 7fe7d5046640 1 -- 172.21.15.43:0/331184130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d019f570 con 0x7fe7d0104d20 2024-04-25T10:40:46.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.050+0000 7fe7d5046640 1 --2- 172.21.15.43:0/331184130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d0104d20 0x7fe7d019a710 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00d950 tx=0x7fe7cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:46.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.050+0000 7fe7c67fc640 1 -- 172.21.15.43:0/331184130 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc017070 con 0x7fe7d0104d20 2024-04-25T10:40:46.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.050+0000 7fe7c67fc640 1 -- 172.21.15.43:0/331184130 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7cc009c30 con 0x7fe7d0104d20 2024-04-25T10:40:46.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.050+0000 7fe7d72aa640 1 -- 172.21.15.43:0/331184130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d019f860 con 0x7fe7d0104d20 2024-04-25T10:40:46.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.050+0000 7fe7c67fc640 1 -- 172.21.15.43:0/331184130 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc00f620 con 0x7fe7d0104d20 2024-04-25T10:40:46.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.050+0000 7fe7d72aa640 1 -- 172.21.15.43:0/331184130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d01a71e0 con 0x7fe7d0104d20 2024-04-25T10:40:46.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.051+0000 7fe7d72aa640 1 -- 172.21.15.43:0/331184130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d006a190 con 0x7fe7d0104d20 2024-04-25T10:40:46.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.053+0000 7fe7c67fc640 1 -- 172.21.15.43:0/331184130 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fe7cc010460 con 0x7fe7d0104d20 2024-04-25T10:40:46.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.054+0000 7fe7c67fc640 1 --2- 172.21.15.43:0/331184130 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe7bc0afc60 0x7fe7bc0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.054+0000 7fe7d4845640 1 --2- 172.21.15.43:0/331184130 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe7bc0afc60 0x7fe7bc0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.054+0000 7fe7c67fc640 1 -- 172.21.15.43:0/331184130 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fe7cc0debb0 con 0x7fe7d0104d20 2024-04-25T10:40:46.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.055+0000 7fe7d4845640 1 --2- 172.21.15.43:0/331184130 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe7bc0afc60 0x7fe7bc0b2170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe7d010f9b0 tx=0x7fe7c000b390 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:46.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.058+0000 7fe7c67fc640 1 -- 172.21.15.43:0/331184130 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe7cc009de0 con 0x7fe7d0104d20 2024-04-25T10:40:46.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:45 smithi043 ceph-mon[29478]: pgmap v254: 97 pgs: 97 active+clean; 2.3 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 63 KiB/s wr, 17 op/s 2024-04-25T10:40:46.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.284+0000 7fe7d72aa640 1 -- 172.21.15.43:0/331184130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7d01a73b0 con 0x7fe7d0104d20 2024-04-25T10:40:46.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.285+0000 7fe7c67fc640 1 -- 172.21.15.43:0/331184130 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7fe7cc0a6e00 con 0x7fe7d0104d20 2024-04-25T10:40:46.287 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:40:46.287 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:40:46.287 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:40:46.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.287+0000 7fe7d72aa640 1 -- 172.21.15.43:0/331184130 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe7bc0afc60 msgr2=0x7fe7bc0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.287+0000 7fe7d72aa640 1 --2- 172.21.15.43:0/331184130 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe7bc0afc60 0x7fe7bc0b2170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe7d010f9b0 tx=0x7fe7c000b390 comp rx=0 tx=0).stop 2024-04-25T10:40:46.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.287+0000 7fe7d72aa640 1 -- 172.21.15.43:0/331184130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d0104d20 msgr2=0x7fe7d019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.287+0000 7fe7d72aa640 1 --2- 172.21.15.43:0/331184130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d0104d20 0x7fe7d019a710 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00d950 tx=0x7fe7cc00de10 comp rx=0 tx=0).stop 2024-04-25T10:40:46.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.287+0000 7fe7d72aa640 1 -- 172.21.15.43:0/331184130 shutdown_connections 2024-04-25T10:40:46.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.287+0000 7fe7d72aa640 1 --2- 172.21.15.43:0/331184130 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe7bc0afc60 0x7fe7bc0b2170 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.287+0000 7fe7d72aa640 1 --2- 172.21.15.43:0/331184130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d010e0e0 0x7fe7d019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.287+0000 7fe7d72aa640 1 --2- 172.21.15.43:0/331184130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d010b780 0x7fe7d019ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.287+0000 7fe7d72aa640 1 --2- 172.21.15.43:0/331184130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d0104d20 0x7fe7d019a710 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.287+0000 7fe7d72aa640 1 -- 172.21.15.43:0/331184130 >> 172.21.15.43:0/331184130 conn(0x7fe7d0078040 msgr2=0x7fe7d0101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:46.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.287+0000 7fe7d72aa640 1 -- 172.21.15.43:0/331184130 shutdown_connections 2024-04-25T10:40:46.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.287+0000 7fe7d72aa640 1 -- 172.21.15.43:0/331184130 wait complete. 2024-04-25T10:40:46.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:45 smithi077 ceph-mon[32772]: pgmap v254: 97 pgs: 97 active+clean; 2.3 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 63 KiB/s wr, 17 op/s 2024-04-25T10:40:46.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:45 smithi164 ceph-mon[32713]: pgmap v254: 97 pgs: 97 active+clean; 2.3 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 63 KiB/s wr, 17 op/s 2024-04-25T10:40:46.300 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:40:46.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.411+0000 7f750e500640 1 -- 172.21.15.43:0/952964955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750810e050 msgr2=0x7f7508110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.411+0000 7f750e500640 1 --2- 172.21.15.43:0/952964955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750810e050 0x7f7508110540 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f7500009ca0 tx=0x7f750002a950 comp rx=0 tx=0).stop 2024-04-25T10:40:46.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.411+0000 7f750e500640 1 -- 172.21.15.43:0/952964955 shutdown_connections 2024-04-25T10:40:46.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.411+0000 7f750e500640 1 --2- 172.21.15.43:0/952964955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750810e050 0x7f7508110540 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.411+0000 7f750e500640 1 --2- 172.21.15.43:0/952964955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f750810b6f0 0x7f750810db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.411+0000 7f750e500640 1 --2- 172.21.15.43:0/952964955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7508104c90 0x7f75081050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.411+0000 7f750e500640 1 -- 172.21.15.43:0/952964955 >> 172.21.15.43:0/952964955 conn(0x7f7508078100 msgr2=0x7f7508078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:46.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.411+0000 7f750e500640 1 -- 172.21.15.43:0/952964955 shutdown_connections 2024-04-25T10:40:46.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.411+0000 7f750e500640 1 -- 172.21.15.43:0/952964955 wait complete. 2024-04-25T10:40:46.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.412+0000 7f750e500640 1 Processor -- start 2024-04-25T10:40:46.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.412+0000 7f750e500640 1 -- start start 2024-04-25T10:40:46.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.412+0000 7f750e500640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508104c90 0x7f7508102550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.412+0000 7f750e500640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f750810b6f0 0x7f7508102a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.412+0000 7f750e500640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f750810e050 0x7f7508102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.412+0000 7f750e500640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7508113cb0 con 0x7f7508104c90 2024-04-25T10:40:46.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.412+0000 7f750e500640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7508113b30 con 0x7f750810e050 2024-04-25T10:40:46.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.412+0000 7f750e500640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7508113e30 con 0x7f750810b6f0 2024-04-25T10:40:46.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.412+0000 7f7507fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508104c90 0x7f7508102550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.412+0000 7f7507fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508104c90 0x7f7508102550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41970/0 (socket says 172.21.15.43:41970) 2024-04-25T10:40:46.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.412+0000 7f7507fff640 1 -- 172.21.15.43:0/2903697889 learned_addr learned my addr 172.21.15.43:0/2903697889 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:46.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.412+0000 7f75077fe640 1 --2- 172.21.15.43:0/2903697889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f750810b6f0 0x7f7508102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.413+0000 7f750ca9d640 1 --2- 172.21.15.43:0/2903697889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f750810e050 0x7f7508102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.413+0000 7f7507fff640 1 -- 172.21.15.43:0/2903697889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f750810b6f0 msgr2=0x7f7508102a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.413+0000 7f7507fff640 1 --2- 172.21.15.43:0/2903697889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f750810b6f0 0x7f7508102a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.413+0000 7f7507fff640 1 -- 172.21.15.43:0/2903697889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f750810e050 msgr2=0x7f7508102fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.413+0000 7f7507fff640 1 --2- 172.21.15.43:0/2903697889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f750810e050 0x7f7508102fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.413+0000 7f7507fff640 1 -- 172.21.15.43:0/2903697889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75081a8a70 con 0x7f7508104c90 2024-04-25T10:40:46.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.413+0000 7f750ca9d640 1 --2- 172.21.15.43:0/2903697889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f750810e050 0x7f7508102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:46.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.413+0000 7f7507fff640 1 --2- 172.21.15.43:0/2903697889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508104c90 0x7f7508102550 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f74f800e9f0 tx=0x7f74f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:46.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.413+0000 7f75057fa640 1 -- 172.21.15.43:0/2903697889 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f800cd80 con 0x7f7508104c90 2024-04-25T10:40:46.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.413+0000 7f750e500640 1 -- 172.21.15.43:0/2903697889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75081a8d60 con 0x7f7508104c90 2024-04-25T10:40:46.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.413+0000 7f75057fa640 1 -- 172.21.15.43:0/2903697889 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74f80108e0 con 0x7f7508104c90 2024-04-25T10:40:46.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.413+0000 7f75057fa640 1 -- 172.21.15.43:0/2903697889 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8018980 con 0x7f7508104c90 2024-04-25T10:40:46.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.413+0000 7f750e500640 1 -- 172.21.15.43:0/2903697889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75081a9210 con 0x7f7508104c90 2024-04-25T10:40:46.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.416+0000 7f75153b3740 1 Processor -- start 2024-04-25T10:40:46.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.416+0000 7f75153b3740 1 -- start start 2024-04-25T10:40:46.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.416+0000 7f75057fa640 1 -- 172.21.15.43:0/2903697889 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f74f8010430 con 0x7f7508104c90 2024-04-25T10:40:46.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.416+0000 7f75153b3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a850a390 0x55d2a88ab810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.416+0000 7f75153b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2a84e14d0 0x55d2a88abd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.416+0000 7f75153b3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a84e96b0 0x55d2a88b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.416+0000 7f75153b3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d2a8810d80 con 0x55d2a84e96b0 2024-04-25T10:40:46.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.416+0000 7f75153b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d2a87caa20 con 0x55d2a84e14d0 2024-04-25T10:40:46.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.416+0000 7f75153b3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d2a880cd40 con 0x55d2a850a390 2024-04-25T10:40:46.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.416+0000 7f75077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a84e96b0 0x55d2a88b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.416+0000 7f75077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a84e96b0 0x55d2a88b86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41972/0 (socket says 172.21.15.43:41972) 2024-04-25T10:40:46.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.416+0000 7f75077fe640 1 -- 172.21.15.43:0/243050089 learned_addr learned my addr 172.21.15.43:0/243050089 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:46.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.416+0000 7f750ca9d640 1 --2- 172.21.15.43:0/243050089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a850a390 0x55d2a88ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.416+0000 7f7507fff640 1 --2- 172.21.15.43:0/243050089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2a84e14d0 0x55d2a88abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.416+0000 7f75077fe640 1 -- 172.21.15.43:0/243050089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a850a390 msgr2=0x55d2a88ab810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.416+0000 7f75077fe640 1 --2- 172.21.15.43:0/243050089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a850a390 0x55d2a88ab810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.416+0000 7f75077fe640 1 -- 172.21.15.43:0/243050089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2a84e14d0 msgr2=0x55d2a88abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.416+0000 7f75077fe640 1 --2- 172.21.15.43:0/243050089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2a84e14d0 0x55d2a88abd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.417+0000 7f75077fe640 1 -- 172.21.15.43:0/243050089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a87d5c30 con 0x55d2a84e96b0 2024-04-25T10:40:46.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.417+0000 7f75077fe640 1 --2- 172.21.15.43:0/243050089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a84e96b0 0x55d2a88b86a0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00b690 tx=0x7f74fc00bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4a1137003b8f4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:46.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.417+0000 7f75057fa640 1 --2- 172.21.15.43:0/2903697889 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f74f00afc60 0x7f74f00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.417+0000 7f750ca9d640 1 --2- 172.21.15.43:0/2903697889 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f74f00afc60 0x7f74f00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.417+0000 7f74ee7fc640 1 -- 172.21.15.43:0/243050089 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc003bd0 con 0x55d2a84e96b0 2024-04-25T10:40:46.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.417+0000 7f75057fa640 1 -- 172.21.15.43:0/2903697889 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f74f80df400 con 0x7f7508104c90 2024-04-25T10:40:46.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.417+0000 7f74ee7fc640 1 -- 172.21.15.43:0/243050089 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74fc015070 con 0x55d2a84e96b0 2024-04-25T10:40:46.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.417+0000 7f74ee7fc640 1 -- 172.21.15.43:0/243050089 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc004d40 con 0x55d2a84e96b0 2024-04-25T10:40:46.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.418+0000 7f75153b3740 1 -- 172.21.15.43:0/243050089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a84e96b0 msgr2=0x55d2a88b86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.418+0000 7f750ca9d640 1 --2- 172.21.15.43:0/2903697889 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f74f00afc60 0x7f74f00b2170 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7500005cc0 tx=0x7f7500005c30 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:46.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.418+0000 7f75153b3740 1 --2- 172.21.15.43:0/243050089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a84e96b0 0x55d2a88b86a0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00b690 tx=0x7f74fc00bb50 comp rx=0 tx=0).stop 2024-04-25T10:40:46.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.418+0000 7f75153b3740 1 -- 172.21.15.43:0/243050089 shutdown_connections 2024-04-25T10:40:46.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.418+0000 7f75153b3740 1 --2- 172.21.15.43:0/243050089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2a84e14d0 0x55d2a88abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.418+0000 7f75153b3740 1 --2- 172.21.15.43:0/243050089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a84e96b0 0x55d2a88b86a0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.418+0000 7f75153b3740 1 --2- 172.21.15.43:0/243050089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a850a390 0x55d2a88ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.418+0000 7f75153b3740 1 -- 172.21.15.43:0/243050089 >> 172.21.15.43:0/243050089 conn(0x55d2a84fe4e0 msgr2=0x55d2a88a7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:46.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.418+0000 7f75153b3740 1 -- 172.21.15.43:0/243050089 shutdown_connections 2024-04-25T10:40:46.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.418+0000 7f75153b3740 1 -- 172.21.15.43:0/243050089 wait complete. 2024-04-25T10:40:46.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.418+0000 7f75153b3740 1 -- start start 2024-04-25T10:40:46.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.419+0000 7f75153b3740 1 Processor -- start 2024-04-25T10:40:46.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.419+0000 7f75153b3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a84e14d0 0x55d2a88b6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.419+0000 7f75153b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2a84e96b0 0x55d2a88b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.419+0000 7f75153b3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a850a390 0x55d2a88c2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.419+0000 7f75153b3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d2a88b8d90 con 0x55d2a84e14d0 2024-04-25T10:40:46.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.419+0000 7f75153b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d2a8797dc0 con 0x55d2a84e96b0 2024-04-25T10:40:46.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.419+0000 7f75153b3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d2a88b8f10 con 0x55d2a850a390 2024-04-25T10:40:46.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.419+0000 7f75153b3740 10 client.0 _ll_register_callbacks cb 0x55d2a87b3600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:46.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.419+0000 7f7507fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a84e14d0 0x55d2a88b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.419+0000 7f7507fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a84e14d0 0x55d2a88b6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41974/0 (socket says 172.21.15.43:41974) 2024-04-25T10:40:46.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.419+0000 7f7507fff640 1 -- 172.21.15.43:0/590009474 learned_addr learned my addr 172.21.15.43:0/590009474 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:46.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.419+0000 7f750ca9d640 1 --2- 172.21.15.43:0/590009474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a850a390 0x55d2a88c2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.419+0000 7f75077fe640 1 --2- 172.21.15.43:0/590009474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2a84e96b0 0x55d2a88b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.419+0000 7f7507fff640 1 -- 172.21.15.43:0/590009474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a850a390 msgr2=0x55d2a88c2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.419+0000 7f7507fff640 1 --2- 172.21.15.43:0/590009474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a850a390 0x55d2a88c2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.419+0000 7f7507fff640 1 -- 172.21.15.43:0/590009474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2a84e96b0 msgr2=0x55d2a88b67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.419+0000 7f7507fff640 1 --2- 172.21.15.43:0/590009474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2a84e96b0 0x55d2a88b67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.419+0000 7f7507fff640 1 -- 172.21.15.43:0/590009474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a85223f0 con 0x55d2a84e14d0 2024-04-25T10:40:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.419+0000 7f7507fff640 1 --2- 172.21.15.43:0/590009474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a84e14d0 0x55d2a88b6280 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f74f800e9c0 tx=0x7f74f80a3520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21385828ef66da94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.419+0000 7f74e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.420+0000 7f74e77fe640 1 -- 172.21.15.43:0/590009474 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8010720 con 0x55d2a84e14d0 2024-04-25T10:40:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.420+0000 7f74e77fe640 1 -- 172.21.15.43:0/590009474 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f74f80e2c60 con 0x55d2a84e14d0 2024-04-25T10:40:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.420+0000 7f75153b3740 1 -- 172.21.15.43:0/590009474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74fc015070 con 0x55d2a84e14d0 2024-04-25T10:40:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.420+0000 7f74e77fe640 1 -- 172.21.15.43:0/590009474 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f74f80e2e60 con 0x55d2a84e14d0 2024-04-25T10:40:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.420+0000 7f75153b3740 10 client.17922 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:46.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.420+0000 7f75153b3740 1 -- 172.21.15.43:0/590009474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d2a85ec7d0 con 0x55d2a84e14d0 2024-04-25T10:40:46.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.420+0000 7f74e77fe640 1 -- 172.21.15.43:0/590009474 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74fc015070 con 0x55d2a84e14d0 2024-04-25T10:40:46.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.420+0000 7f74e77fe640 1 -- 172.21.15.43:0/590009474 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f74f8018b20 con 0x55d2a84e14d0 2024-04-25T10:40:46.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.420+0000 7f75153b3740 10 client.17922 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:46.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.420+0000 7f75153b3740 10 client.17922 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:40:46.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.420+0000 7f75153b3740 20 client.17922 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:46.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.420+0000 7f75153b3740 1 --2- 172.21.15.43:0/590009474 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d2a84c8f60 0x55d2a88c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.420+0000 7f75153b3740 4 client.17922 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:46.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.420+0000 7f75153b3740 1 -- 172.21.15.43:0/590009474 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2a881e990 con 0x55d2a84c8f60 2024-04-25T10:40:46.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.421+0000 7f75077fe640 1 --2- 172.21.15.43:0/590009474 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d2a84c8f60 0x55d2a88c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.421+0000 7f74e77fe640 10 client.17922 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:40:46.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.421+0000 7f75077fe640 1 --2- 172.21.15.43:0/590009474 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d2a84c8f60 0x55d2a88c8660 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=331b5745c391475 server_cookie=d94c0a3fb93d25e2 in_seq=0 out_seq=0 2024-04-25T10:40:46.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.422+0000 7f74e77fe640 1 -- 172.21.15.43:0/590009474 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d2a881e990 con 0x55d2a84c8f60 2024-04-25T10:40:46.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.422+0000 7f74e77fe640 10 client.17922 handle_command_reply: tid=0 2024-04-25T10:40:46.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.422+0000 7f75153b3740 1 client.17922 shutdown 2024-04-25T10:40:46.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.422+0000 7f75153b3740 1 -- 172.21.15.43:0/590009474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a84e14d0 msgr2=0x55d2a88b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.422+0000 7f75153b3740 1 --2- 172.21.15.43:0/590009474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a84e14d0 0x55d2a88b6280 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f74f800e9c0 tx=0x7f74f80a3520 comp rx=0 tx=0).stop 2024-04-25T10:40:46.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.422+0000 7f75153b3740 1 -- 172.21.15.43:0/590009474 shutdown_connections 2024-04-25T10:40:46.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.422+0000 7f75153b3740 1 --2- 172.21.15.43:0/590009474 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d2a84c8f60 0x55d2a88c8660 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.422+0000 7f75153b3740 1 --2- 172.21.15.43:0/590009474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2a84e96b0 0x55d2a88b67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.422+0000 7f75153b3740 1 --2- 172.21.15.43:0/590009474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a850a390 0x55d2a88c2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.422+0000 7f75153b3740 1 --2- 172.21.15.43:0/590009474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a84e14d0 0x55d2a88b6280 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.422+0000 7f74e77fe640 0 client.17922 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:40:46.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.422+0000 7f75153b3740 1 -- 172.21.15.43:0/590009474 >> 172.21.15.43:0/590009474 conn(0x55d2a84fe4e0 msgr2=0x55d2a88ad530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:46.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.422+0000 7f75153b3740 1 -- 172.21.15.43:0/590009474 shutdown_connections 2024-04-25T10:40:46.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.423+0000 7f75153b3740 1 -- 172.21.15.43:0/590009474 wait complete. 2024-04-25T10:40:46.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.423+0000 7f75153b3740 20 client.17922 trim_cache size 0 max 16384 2024-04-25T10:40:46.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.436+0000 7f75153b3740 1 Processor -- start 2024-04-25T10:40:46.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.436+0000 7f75153b3740 1 -- start start 2024-04-25T10:40:46.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.436+0000 7f75153b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2a84e14d0 0x55d2a88a7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.436+0000 7f75153b3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a84e96b0 0x55d2a88c1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.436+0000 7f75153b3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a850a390 0x55d2a88cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.436+0000 7f75153b3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75081b43d0 con 0x55d2a850a390 2024-04-25T10:40:46.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.436+0000 7f75153b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f75081b1ca0 con 0x55d2a84e14d0 2024-04-25T10:40:46.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.436+0000 7f75153b3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f75081afb10 con 0x55d2a84e96b0 2024-04-25T10:40:46.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.437+0000 7f750ca9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a850a390 0x55d2a88cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.437+0000 7f750ca9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a850a390 0x55d2a88cde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41988/0 (socket says 172.21.15.43:41988) 2024-04-25T10:40:46.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.437+0000 7f750ca9d640 1 -- 172.21.15.43:0/3823844661 learned_addr learned my addr 172.21.15.43:0/3823844661 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:46.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.437+0000 7f7507fff640 1 --2- 172.21.15.43:0/3823844661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2a84e14d0 0x55d2a88a7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.437+0000 7f75077fe640 1 --2- 172.21.15.43:0/3823844661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a84e96b0 0x55d2a88c1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.437+0000 7f750ca9d640 1 -- 172.21.15.43:0/3823844661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a84e96b0 msgr2=0x55d2a88c1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.437+0000 7f750ca9d640 1 --2- 172.21.15.43:0/3823844661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a84e96b0 0x55d2a88c1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.437+0000 7f750ca9d640 1 -- 172.21.15.43:0/3823844661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2a84e14d0 msgr2=0x55d2a88a7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.437+0000 7f750ca9d640 1 --2- 172.21.15.43:0/3823844661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2a84e14d0 0x55d2a88a7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.437+0000 7f750ca9d640 1 -- 172.21.15.43:0/3823844661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a88a84b0 con 0x55d2a850a390 2024-04-25T10:40:46.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.437+0000 7f750ca9d640 1 --2- 172.21.15.43:0/3823844661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a850a390 0x55d2a88cde20 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f7500009ca0 tx=0x7f7500040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf5cefcd3fc1919 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:46.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.437+0000 7f74edffb640 1 -- 172.21.15.43:0/3823844661 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500034040 con 0x55d2a850a390 2024-04-25T10:40:46.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.437+0000 7f74edffb640 1 -- 172.21.15.43:0/3823844661 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7500004040 con 0x55d2a850a390 2024-04-25T10:40:46.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.437+0000 7f74edffb640 1 -- 172.21.15.43:0/3823844661 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75000306a0 con 0x55d2a850a390 2024-04-25T10:40:46.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.438+0000 7f75153b3740 1 -- 172.21.15.43:0/3823844661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a850a390 msgr2=0x55d2a88cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.438+0000 7f75153b3740 1 --2- 172.21.15.43:0/3823844661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a850a390 0x55d2a88cde20 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f7500009ca0 tx=0x7f7500040000 comp rx=0 tx=0).stop 2024-04-25T10:40:46.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.438+0000 7f75153b3740 1 -- 172.21.15.43:0/3823844661 shutdown_connections 2024-04-25T10:40:46.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.438+0000 7f75153b3740 1 --2- 172.21.15.43:0/3823844661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a850a390 0x55d2a88cde20 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.438+0000 7f75153b3740 1 --2- 172.21.15.43:0/3823844661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a84e96b0 0x55d2a88c1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.438+0000 7f75153b3740 1 --2- 172.21.15.43:0/3823844661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2a84e14d0 0x55d2a88a7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.438+0000 7f75153b3740 1 -- 172.21.15.43:0/3823844661 >> 172.21.15.43:0/3823844661 conn(0x55d2a84fe4e0 msgr2=0x55d2a88a7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:46.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.438+0000 7f75153b3740 1 -- 172.21.15.43:0/3823844661 shutdown_connections 2024-04-25T10:40:46.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.438+0000 7f75153b3740 1 -- 172.21.15.43:0/3823844661 wait complete. 2024-04-25T10:40:46.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.438+0000 7f75153b3740 1 -- start start 2024-04-25T10:40:46.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.439+0000 7f75153b3740 1 Processor -- start 2024-04-25T10:40:46.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.439+0000 7f75153b3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a84e14d0 0x55d2a8928d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.439+0000 7f75153b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2a84e96b0 0x55d2a89292c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.439+0000 7f75153b3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a850a390 0x55d2a88b95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.439+0000 7f75153b3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75081b0160 con 0x55d2a850a390 2024-04-25T10:40:46.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.439+0000 7f75153b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f75081b15d0 con 0x55d2a84e96b0 2024-04-25T10:40:46.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.439+0000 7f75153b3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f75081b0830 con 0x55d2a84e14d0 2024-04-25T10:40:46.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.439+0000 7f75153b3740 10 client.0 _ll_register_callbacks cb 0x7f74f8009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:46.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.439+0000 7f750ca9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a850a390 0x55d2a88b95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.439+0000 7f750ca9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a850a390 0x55d2a88b95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41992/0 (socket says 172.21.15.43:41992) 2024-04-25T10:40:46.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.439+0000 7f750ca9d640 1 -- 172.21.15.43:0/3789412653 learned_addr learned my addr 172.21.15.43:0/3789412653 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:46.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.439+0000 7f7507fff640 1 --2- 172.21.15.43:0/3789412653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a84e14d0 0x55d2a8928d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.439+0000 7f75077fe640 1 --2- 172.21.15.43:0/3789412653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2a84e96b0 0x55d2a89292c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.439+0000 7f750ca9d640 1 -- 172.21.15.43:0/3789412653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a84e14d0 msgr2=0x55d2a8928d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.439+0000 7f750ca9d640 1 --2- 172.21.15.43:0/3789412653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a84e14d0 0x55d2a8928d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.439+0000 7f750ca9d640 1 -- 172.21.15.43:0/3789412653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2a84e96b0 msgr2=0x55d2a89292c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.439+0000 7f750ca9d640 1 --2- 172.21.15.43:0/3789412653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2a84e96b0 0x55d2a89292c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.439+0000 7f750ca9d640 1 -- 172.21.15.43:0/3789412653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a88b9bb0 con 0x55d2a850a390 2024-04-25T10:40:46.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.440+0000 7f750ca9d640 1 --2- 172.21.15.43:0/3789412653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a850a390 0x55d2a88b95c0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f750002fcc0 tx=0x7f7500004810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1465b1edefa490e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:46.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.440+0000 7f74e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:46.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.440+0000 7f74e77fe640 1 -- 172.21.15.43:0/3789412653 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500057030 con 0x55d2a850a390 2024-04-25T10:40:46.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.440+0000 7f74e77fe640 1 -- 172.21.15.43:0/3789412653 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f7500041040 con 0x55d2a850a390 2024-04-25T10:40:46.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.440+0000 7f75153b3740 1 -- 172.21.15.43:0/3789412653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7500004040 con 0x55d2a850a390 2024-04-25T10:40:46.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.440+0000 7f74e77fe640 1 -- 172.21.15.43:0/3789412653 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7500034070 con 0x55d2a850a390 2024-04-25T10:40:46.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.440+0000 7f75153b3740 10 client.17928 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:46.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.440+0000 7f75153b3740 1 -- 172.21.15.43:0/3789412653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d2a89303a0 con 0x55d2a850a390 2024-04-25T10:40:46.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.440+0000 7f74e77fe640 1 -- 172.21.15.43:0/3789412653 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7500004040 con 0x55d2a850a390 2024-04-25T10:40:46.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.440+0000 7f74e77fe640 1 -- 172.21.15.43:0/3789412653 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f75000659b0 con 0x55d2a850a390 2024-04-25T10:40:46.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.440+0000 7f75153b3740 10 client.17928 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:46.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.440+0000 7f75153b3740 10 client.17928 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.440+0000 7f75153b3740 20 client.17928 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.441+0000 7f75153b3740 1 --2- 172.21.15.43:0/3789412653 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d2a849e3a0 0x55d2a8932aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.441+0000 7f75153b3740 4 client.17928 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.441+0000 7f75153b3740 1 -- 172.21.15.43:0/3789412653 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d2a8818800 con 0x55d2a849e3a0 2024-04-25T10:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.441+0000 7f7507fff640 1 --2- 172.21.15.43:0/3789412653 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d2a849e3a0 0x55d2a8932aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.441+0000 7f74e77fe640 10 client.17928 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.441+0000 7f7507fff640 1 --2- 172.21.15.43:0/3789412653 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d2a849e3a0 0x55d2a8932aa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=40e429d1835b9438 server_cookie=63dee8546f53d2f7 in_seq=0 out_seq=0 2024-04-25T10:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.441+0000 7f74e77fe640 1 -- 172.21.15.43:0/3789412653 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d2a8818800 con 0x55d2a849e3a0 2024-04-25T10:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.441+0000 7f74e77fe640 10 client.17928 handle_command_reply: tid=0 2024-04-25T10:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.441+0000 7f75153b3740 1 client.17928 shutdown 2024-04-25T10:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f75153b3740 1 -- 172.21.15.43:0/3789412653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a850a390 msgr2=0x55d2a88b95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f75153b3740 1 --2- 172.21.15.43:0/3789412653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a850a390 0x55d2a88b95c0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f750002fcc0 tx=0x7f7500004810 comp rx=0 tx=0).stop 2024-04-25T10:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f75153b3740 1 -- 172.21.15.43:0/3789412653 shutdown_connections 2024-04-25T10:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f75153b3740 1 --2- 172.21.15.43:0/3789412653 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d2a849e3a0 0x55d2a8932aa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f75153b3740 1 --2- 172.21.15.43:0/3789412653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2a84e96b0 0x55d2a89292c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f75153b3740 1 --2- 172.21.15.43:0/3789412653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a850a390 0x55d2a88b95c0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f74e77fe640 0 client.17928 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:40:46.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f75153b3740 1 --2- 172.21.15.43:0/3789412653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a84e14d0 0x55d2a8928d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f75153b3740 1 -- 172.21.15.43:0/3789412653 >> 172.21.15.43:0/3789412653 conn(0x55d2a84fe4e0 msgr2=0x55d2a88b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:46.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f75153b3740 1 -- 172.21.15.43:0/3789412653 shutdown_connections 2024-04-25T10:40:46.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f75153b3740 1 -- 172.21.15.43:0/3789412653 wait complete. 2024-04-25T10:40:46.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f75153b3740 20 client.17928 trim_cache size 0 max 16384 2024-04-25T10:40:46.450 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:40:46.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f750e500640 1 -- 172.21.15.43:0/2903697889 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f74f00afc60 msgr2=0x7f74f00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f750e500640 1 --2- 172.21.15.43:0/2903697889 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f74f00afc60 0x7f74f00b2170 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7500005cc0 tx=0x7f7500005c30 comp rx=0 tx=0).stop 2024-04-25T10:40:46.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f750e500640 1 -- 172.21.15.43:0/2903697889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508104c90 msgr2=0x7f7508102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f750e500640 1 --2- 172.21.15.43:0/2903697889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508104c90 0x7f7508102550 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f74f800e9f0 tx=0x7f74f800eeb0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f750e500640 1 -- 172.21.15.43:0/2903697889 shutdown_connections 2024-04-25T10:40:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f750e500640 1 --2- 172.21.15.43:0/2903697889 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f74f00afc60 0x7f74f00b2170 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f750e500640 1 --2- 172.21.15.43:0/2903697889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f750810e050 0x7f7508102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f750e500640 1 --2- 172.21.15.43:0/2903697889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f750810b6f0 0x7f7508102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f750e500640 1 --2- 172.21.15.43:0/2903697889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508104c90 0x7f7508102550 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f750e500640 1 -- 172.21.15.43:0/2903697889 >> 172.21.15.43:0/2903697889 conn(0x7f7508078100 msgr2=0x7f7508101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f750e500640 1 -- 172.21.15.43:0/2903697889 shutdown_connections 2024-04-25T10:40:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.442+0000 7f750e500640 1 -- 172.21.15.43:0/2903697889 wait complete. 2024-04-25T10:40:46.455 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:40:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.567+0000 7fada46e9640 1 -- 172.21.15.43:0/3039418810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9c1069b0 msgr2=0x7fad9c106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.567+0000 7fada46e9640 1 --2- 172.21.15.43:0/3039418810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9c1069b0 0x7fad9c106de0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fad8c009900 tx=0x7fad8c02a410 comp rx=0 tx=0).stop 2024-04-25T10:40:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.567+0000 7fada46e9640 1 -- 172.21.15.43:0/3039418810 shutdown_connections 2024-04-25T10:40:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.567+0000 7fada46e9640 1 --2- 172.21.15.43:0/3039418810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad9c107470 0x7fad9c10ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.567+0000 7fada46e9640 1 --2- 172.21.15.43:0/3039418810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9c1069b0 0x7fad9c106de0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.567+0000 7fada46e9640 1 --2- 172.21.15.43:0/3039418810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c102420 0x7fad9c102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.567+0000 7fada46e9640 1 -- 172.21.15.43:0/3039418810 >> 172.21.15.43:0/3039418810 conn(0x7fad9c077920 msgr2=0x7fad9c077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.567+0000 7fada46e9640 1 -- 172.21.15.43:0/3039418810 shutdown_connections 2024-04-25T10:40:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.567+0000 7fada46e9640 1 -- 172.21.15.43:0/3039418810 wait complete. 2024-04-25T10:40:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.568+0000 7fada46e9640 1 Processor -- start 2024-04-25T10:40:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.568+0000 7fada46e9640 1 -- start start 2024-04-25T10:40:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.568+0000 7fada46e9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad9c102420 0x7fad9c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.568+0000 7fada46e9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9c1069b0 0x7fad9c19acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.568+0000 7fada46e9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c107470 0x7fad9c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.568+0000 7fada46e9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c111b00 con 0x7fad9c1069b0 2024-04-25T10:40:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.568+0000 7fada46e9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c111980 con 0x7fad9c107470 2024-04-25T10:40:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.568+0000 7fada46e9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c111c80 con 0x7fad9c102420 2024-04-25T10:40:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.568+0000 7fada1c84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9c1069b0 0x7fad9c19acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.568+0000 7fada1c84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9c1069b0 0x7fad9c19acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42010/0 (socket says 172.21.15.43:42010) 2024-04-25T10:40:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.568+0000 7fada1c84640 1 -- 172.21.15.43:0/1893253051 learned_addr learned my addr 172.21.15.43:0/1893253051 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.569+0000 7fada2485640 1 --2- 172.21.15.43:0/1893253051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad9c102420 0x7fad9c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.569+0000 7fada2c86640 1 --2- 172.21.15.43:0/1893253051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c107470 0x7fad9c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.569+0000 7fada1c84640 1 -- 172.21.15.43:0/1893253051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad9c102420 msgr2=0x7fad9c19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.569+0000 7fada1c84640 1 --2- 172.21.15.43:0/1893253051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad9c102420 0x7fad9c19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.569+0000 7fada1c84640 1 -- 172.21.15.43:0/1893253051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c107470 msgr2=0x7fad9c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.569+0000 7fada1c84640 1 --2- 172.21.15.43:0/1893253051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c107470 0x7fad9c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.569+0000 7fada1c84640 1 -- 172.21.15.43:0/1893253051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9c19f5d0 con 0x7fad9c1069b0 2024-04-25T10:40:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.569+0000 7fada2c86640 1 --2- 172.21.15.43:0/1893253051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c107470 0x7fad9c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.569+0000 7fada1c84640 1 --2- 172.21.15.43:0/1893253051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9c1069b0 0x7fad9c19acd0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fad8c032fd0 tx=0x7fad8c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.569+0000 7fad937fe640 1 -- 172.21.15.43:0/1893253051 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c03f030 con 0x7fad9c1069b0 2024-04-25T10:40:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.569+0000 7fada46e9640 1 -- 172.21.15.43:0/1893253051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad9c19f860 con 0x7fad9c1069b0 2024-04-25T10:40:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.569+0000 7fad937fe640 1 -- 172.21.15.43:0/1893253051 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad8c004490 con 0x7fad9c1069b0 2024-04-25T10:40:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.569+0000 7fad937fe640 1 -- 172.21.15.43:0/1893253051 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c035070 con 0x7fad9c1069b0 2024-04-25T10:40:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.569+0000 7fada46e9640 1 -- 172.21.15.43:0/1893253051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad9c19fda0 con 0x7fad9c1069b0 2024-04-25T10:40:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.572+0000 7fadab595740 1 Processor -- start 2024-04-25T10:40:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.572+0000 7fadab595740 1 -- start start 2024-04-25T10:40:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.572+0000 7fad937fe640 1 -- 172.21.15.43:0/1893253051 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fad8c042030 con 0x7fad9c1069b0 2024-04-25T10:40:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.572+0000 7fadab595740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe29af390 0x55bbe2d506e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.572+0000 7fadab595740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29864d0 0x55bbe2d50c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.572+0000 7fadab595740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbe298e6b0 0x55bbe2d61ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.572+0000 7fadab595740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bbe2cb5d80 con 0x55bbe29af390 2024-04-25T10:40:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.572+0000 7fadab595740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bbe2c6fa20 con 0x55bbe29864d0 2024-04-25T10:40:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.572+0000 7fadab595740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bbe2cb1d40 con 0x55bbe298e6b0 2024-04-25T10:40:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.573+0000 7fada2c86640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe29af390 0x55bbe2d506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.573+0000 7fada2c86640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe29af390 0x55bbe2d506e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42026/0 (socket says 172.21.15.43:42026) 2024-04-25T10:40:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.573+0000 7fada2c86640 1 -- 172.21.15.43:0/1233813364 learned_addr learned my addr 172.21.15.43:0/1233813364 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.573+0000 7fada2485640 1 --2- 172.21.15.43:0/1233813364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29864d0 0x55bbe2d50c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.573+0000 7fada1c84640 1 --2- 172.21.15.43:0/1233813364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbe298e6b0 0x55bbe2d61ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.573+0000 7fada2c86640 1 -- 172.21.15.43:0/1233813364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbe298e6b0 msgr2=0x55bbe2d61ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.573+0000 7fada2c86640 1 --2- 172.21.15.43:0/1233813364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbe298e6b0 0x55bbe2d61ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.573+0000 7fada2c86640 1 -- 172.21.15.43:0/1233813364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29864d0 msgr2=0x55bbe2d50c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.573+0000 7fada2c86640 1 --2- 172.21.15.43:0/1233813364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29864d0 0x55bbe2d50c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.573+0000 7fada2c86640 1 -- 172.21.15.43:0/1233813364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbe2c7ac30 con 0x55bbe29af390 2024-04-25T10:40:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.573+0000 7fad937fe640 1 --2- 172.21.15.43:0/1893253051 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fad880afc60 0x7fad880b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.573+0000 7fada2c86640 1 --2- 172.21.15.43:0/1233813364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe29af390 0x55bbe2d506e0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fad94008d50 tx=0x7fad9400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6336c1afb47e6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.573+0000 7fada2485640 1 --2- 172.21.15.43:0/1893253051 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fad880afc60 0x7fad880b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.573+0000 7fad90ff9640 1 -- 172.21.15.43:0/1233813364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9400f830 con 0x55bbe29af390 2024-04-25T10:40:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.573+0000 7fad937fe640 1 -- 172.21.15.43:0/1893253051 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fad8c0ff5f0 con 0x7fad9c1069b0 2024-04-25T10:40:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.573+0000 7fad90ff9640 1 -- 172.21.15.43:0/1233813364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad9400de20 con 0x55bbe29af390 2024-04-25T10:40:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.573+0000 7fad90ff9640 1 -- 172.21.15.43:0/1233813364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94018d60 con 0x55bbe29af390 2024-04-25T10:40:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.574+0000 7fadab595740 1 -- 172.21.15.43:0/1233813364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe29af390 msgr2=0x55bbe2d506e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.574+0000 7fadab595740 1 --2- 172.21.15.43:0/1233813364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe29af390 0x55bbe2d506e0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fad94008d50 tx=0x7fad9400d710 comp rx=0 tx=0).stop 2024-04-25T10:40:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.574+0000 7fada2485640 1 --2- 172.21.15.43:0/1893253051 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fad880afc60 0x7fad880b2170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55bbe2d4e070 tx=0x7fad9800b570 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.574+0000 7fadab595740 1 -- 172.21.15.43:0/1233813364 shutdown_connections 2024-04-25T10:40:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.574+0000 7fadab595740 1 --2- 172.21.15.43:0/1233813364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29864d0 0x55bbe2d50c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.574+0000 7fadab595740 1 --2- 172.21.15.43:0/1233813364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbe298e6b0 0x55bbe2d61ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.574+0000 7fadab595740 1 --2- 172.21.15.43:0/1233813364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe29af390 0x55bbe2d506e0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.574+0000 7fadab595740 1 -- 172.21.15.43:0/1233813364 >> 172.21.15.43:0/1233813364 conn(0x55bbe29a34e0 msgr2=0x55bbe2d4cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.574+0000 7fadab595740 1 -- 172.21.15.43:0/1233813364 shutdown_connections 2024-04-25T10:40:46.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.574+0000 7fadab595740 1 -- 172.21.15.43:0/1233813364 wait complete. 2024-04-25T10:40:46.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.575+0000 7fadab595740 1 -- start start 2024-04-25T10:40:46.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.575+0000 7fadab595740 1 Processor -- start 2024-04-25T10:40:46.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.575+0000 7fadab595740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbe29864d0 0x55bbe2d5f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.575+0000 7fadab595740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 0x55bbe2d5fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.575+0000 7fadab595740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29af390 0x55bbe2d6b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.575+0000 7fadab595740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bbe2d62230 con 0x55bbe298e6b0 2024-04-25T10:40:46.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.575+0000 7fadab595740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bbe2c3cdc0 con 0x55bbe29af390 2024-04-25T10:40:46.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.575+0000 7fadab595740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bbe2d623b0 con 0x55bbe29864d0 2024-04-25T10:40:46.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.575+0000 7fadab595740 10 client.0 _ll_register_callbacks cb 0x55bbe2c58600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:46.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.575+0000 7fada1c84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 0x55bbe2d5fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.575+0000 7fada1c84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 0x55bbe2d5fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42034/0 (socket says 172.21.15.43:42034) 2024-04-25T10:40:46.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.575+0000 7fada1c84640 1 -- 172.21.15.43:0/1311873327 learned_addr learned my addr 172.21.15.43:0/1311873327 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.576+0000 7fada2485640 1 --2- 172.21.15.43:0/1311873327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbe29864d0 0x55bbe2d5f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.576+0000 7fada2c86640 1 --2- 172.21.15.43:0/1311873327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29af390 0x55bbe2d6b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.576+0000 7fada1c84640 1 -- 172.21.15.43:0/1311873327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbe29864d0 msgr2=0x55bbe2d5f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.576+0000 7fada1c84640 1 --2- 172.21.15.43:0/1311873327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbe29864d0 0x55bbe2d5f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.576+0000 7fada1c84640 1 -- 172.21.15.43:0/1311873327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29af390 msgr2=0x55bbe2d6b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.576+0000 7fada1c84640 1 --2- 172.21.15.43:0/1311873327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29af390 0x55bbe2d6b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.576+0000 7fada1c84640 1 -- 172.21.15.43:0/1311873327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbe29c73f0 con 0x55bbe298e6b0 2024-04-25T10:40:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.576+0000 7fada1c84640 1 --2- 172.21.15.43:0/1311873327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 0x55bbe2d5fbc0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0fcda0 tx=0x7fad8c03b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fdc87078ee3cb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.576+0000 7fad85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.576+0000 7fad85ffb640 1 -- 172.21.15.43:0/1311873327 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c0c7560 con 0x55bbe298e6b0 2024-04-25T10:40:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.576+0000 7fad85ffb640 1 -- 172.21.15.43:0/1311873327 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fad8c0fcfc0 con 0x55bbe298e6b0 2024-04-25T10:40:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.576+0000 7fadab595740 1 -- 172.21.15.43:0/1311873327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad9400de20 con 0x55bbe298e6b0 2024-04-25T10:40:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.576+0000 7fad85ffb640 1 -- 172.21.15.43:0/1311873327 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fad8c049030 con 0x55bbe298e6b0 2024-04-25T10:40:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.576+0000 7fadab595740 10 client.17940 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.576+0000 7fadab595740 1 -- 172.21.15.43:0/1311873327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bbe2a917d0 con 0x55bbe298e6b0 2024-04-25T10:40:46.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.576+0000 7fad85ffb640 1 -- 172.21.15.43:0/1311873327 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad9400de20 con 0x55bbe298e6b0 2024-04-25T10:40:46.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.576+0000 7fad85ffb640 1 -- 172.21.15.43:0/1311873327 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fad8c03bc20 con 0x55bbe298e6b0 2024-04-25T10:40:46.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.577+0000 7fadab595740 10 client.17940 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:46.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.577+0000 7fadab595740 10 client.17940 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:40:46.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.577+0000 7fadab595740 20 client.17940 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:46.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.577+0000 7fadab595740 1 --2- 172.21.15.43:0/1311873327 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bbe296df60 0x55bbe2d71ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.577+0000 7fadab595740 4 client.17940 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:46.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.577+0000 7fadab595740 1 -- 172.21.15.43:0/1311873327 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbe2cc3990 con 0x55bbe296df60 2024-04-25T10:40:46.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.577+0000 7fada2485640 1 --2- 172.21.15.43:0/1311873327 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bbe296df60 0x55bbe2d71ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.577+0000 7fada2485640 1 --2- 172.21.15.43:0/1311873327 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bbe296df60 0x55bbe2d71ba0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2680d53d0e9a111 server_cookie=74a43b9c41fc10a1 in_seq=0 out_seq=0 2024-04-25T10:40:46.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.577+0000 7fad85ffb640 10 client.17940 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:40:46.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.578+0000 7fad85ffb640 1 -- 172.21.15.43:0/1311873327 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bbe2cc3990 con 0x55bbe296df60 2024-04-25T10:40:46.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.578+0000 7fad85ffb640 10 client.17940 handle_command_reply: tid=0 2024-04-25T10:40:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.578+0000 7fadab595740 1 client.17940 shutdown 2024-04-25T10:40:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.578+0000 7fadab595740 1 -- 172.21.15.43:0/1311873327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 msgr2=0x55bbe2d5fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.578+0000 7fadab595740 1 --2- 172.21.15.43:0/1311873327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 0x55bbe2d5fbc0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0fcda0 tx=0x7fad8c03b720 comp rx=0 tx=0).stop 2024-04-25T10:40:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.578+0000 7fadab595740 1 -- 172.21.15.43:0/1311873327 shutdown_connections 2024-04-25T10:40:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.578+0000 7fadab595740 1 --2- 172.21.15.43:0/1311873327 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bbe296df60 0x55bbe2d71ba0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.578+0000 7fadab595740 1 --2- 172.21.15.43:0/1311873327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29af390 0x55bbe2d6b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.578+0000 7fadab595740 1 --2- 172.21.15.43:0/1311873327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 0x55bbe2d5fbc0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.578+0000 7fad85ffb640 0 client.17940 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:40:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.578+0000 7fadab595740 1 --2- 172.21.15.43:0/1311873327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbe29864d0 0x55bbe2d5f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.578+0000 7fadab595740 1 -- 172.21.15.43:0/1311873327 >> 172.21.15.43:0/1311873327 conn(0x55bbe29a34e0 msgr2=0x55bbe2d524c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.578+0000 7fadab595740 1 -- 172.21.15.43:0/1311873327 shutdown_connections 2024-04-25T10:40:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.578+0000 7fadab595740 1 -- 172.21.15.43:0/1311873327 wait complete. 2024-04-25T10:40:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.579+0000 7fadab595740 20 client.17940 trim_cache size 0 max 16384 2024-04-25T10:40:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.592+0000 7fadab595740 1 Processor -- start 2024-04-25T10:40:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.592+0000 7fadab595740 1 -- start start 2024-04-25T10:40:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.592+0000 7fadab595740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29864d0 0x55bbe2d6b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.592+0000 7fadab595740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 0x55bbe2d77370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.592+0000 7fadab595740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbe29af390 0x55bbe2d778b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.592+0000 7fadab595740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1b0110 con 0x55bbe298e6b0 2024-04-25T10:40:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.592+0000 7fadab595740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1ad9e0 con 0x55bbe29864d0 2024-04-25T10:40:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.592+0000 7fadab595740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1ab850 con 0x55bbe29af390 2024-04-25T10:40:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.592+0000 7fada1c84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 0x55bbe2d77370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.592+0000 7fada1c84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 0x55bbe2d77370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42036/0 (socket says 172.21.15.43:42036) 2024-04-25T10:40:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.592+0000 7fada1c84640 1 -- 172.21.15.43:0/892697238 learned_addr learned my addr 172.21.15.43:0/892697238 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.593+0000 7fada2485640 1 --2- 172.21.15.43:0/892697238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29864d0 0x55bbe2d6b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.593+0000 7fada1c84640 1 -- 172.21.15.43:0/892697238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbe29af390 msgr2=0x55bbe2d778b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.593+0000 7fada1c84640 1 --2- 172.21.15.43:0/892697238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbe29af390 0x55bbe2d778b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.593+0000 7fada1c84640 1 -- 172.21.15.43:0/892697238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29864d0 msgr2=0x55bbe2d6b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.593+0000 7fada1c84640 1 --2- 172.21.15.43:0/892697238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29864d0 0x55bbe2d6b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.593+0000 7fada1c84640 1 -- 172.21.15.43:0/892697238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbe2d60390 con 0x55bbe298e6b0 2024-04-25T10:40:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.593+0000 7fada1c84640 1 --2- 172.21.15.43:0/892697238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 0x55bbe2d77370 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fad8c032af0 tx=0x7fad8c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1077d024125d120 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.593+0000 7fad87fff640 1 -- 172.21.15.43:0/892697238 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c0c6f00 con 0x55bbe298e6b0 2024-04-25T10:40:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.593+0000 7fad87fff640 1 -- 172.21.15.43:0/892697238 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad8c0ea0e0 con 0x55bbe298e6b0 2024-04-25T10:40:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.593+0000 7fad87fff640 1 -- 172.21.15.43:0/892697238 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c0c70e0 con 0x55bbe298e6b0 2024-04-25T10:40:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.594+0000 7fadab595740 1 -- 172.21.15.43:0/892697238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 msgr2=0x55bbe2d77370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.594+0000 7fadab595740 1 --2- 172.21.15.43:0/892697238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 0x55bbe2d77370 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fad8c032af0 tx=0x7fad8c02a410 comp rx=0 tx=0).stop 2024-04-25T10:40:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.594+0000 7fadab595740 1 -- 172.21.15.43:0/892697238 shutdown_connections 2024-04-25T10:40:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.594+0000 7fadab595740 1 --2- 172.21.15.43:0/892697238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbe29af390 0x55bbe2d778b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.594+0000 7fadab595740 1 --2- 172.21.15.43:0/892697238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 0x55bbe2d77370 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.594+0000 7fadab595740 1 --2- 172.21.15.43:0/892697238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29864d0 0x55bbe2d6b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.594+0000 7fadab595740 1 -- 172.21.15.43:0/892697238 >> 172.21.15.43:0/892697238 conn(0x55bbe29a34e0 msgr2=0x55bbe2d5dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.594+0000 7fadab595740 1 -- 172.21.15.43:0/892697238 shutdown_connections 2024-04-25T10:40:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.594+0000 7fadab595740 1 -- 172.21.15.43:0/892697238 wait complete. 2024-04-25T10:40:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.594+0000 7fadab595740 1 -- start start 2024-04-25T10:40:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.594+0000 7fadab595740 1 Processor -- start 2024-04-25T10:40:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.594+0000 7fadab595740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbe29864d0 0x55bbe2d67bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fadab595740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 0x55bbe2d62f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fadab595740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29af390 0x55bbe2d63490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fadab595740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1abea0 con 0x55bbe298e6b0 2024-04-25T10:40:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fadab595740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1ad310 con 0x55bbe29af390 2024-04-25T10:40:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fadab595740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1ac570 con 0x55bbe29864d0 2024-04-25T10:40:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fadab595740 10 client.0 _ll_register_callbacks cb 0x7fad8c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fada1c84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 0x55bbe2d62f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fada1c84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 0x55bbe2d62f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42048/0 (socket says 172.21.15.43:42048) 2024-04-25T10:40:46.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fada1c84640 1 -- 172.21.15.43:0/3881633445 learned_addr learned my addr 172.21.15.43:0/3881633445 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:46.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fada2485640 1 --2- 172.21.15.43:0/3881633445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbe29864d0 0x55bbe2d67bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fada2c86640 1 --2- 172.21.15.43:0/3881633445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29af390 0x55bbe2d63490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fada1c84640 1 -- 172.21.15.43:0/3881633445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbe29864d0 msgr2=0x55bbe2d67bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fada1c84640 1 --2- 172.21.15.43:0/3881633445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbe29864d0 0x55bbe2d67bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fada1c84640 1 -- 172.21.15.43:0/3881633445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29af390 msgr2=0x55bbe2d63490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fada1c84640 1 --2- 172.21.15.43:0/3881633445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29af390 0x55bbe2d63490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fada1c84640 1 -- 172.21.15.43:0/3881633445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbe2d63c00 con 0x55bbe298e6b0 2024-04-25T10:40:46.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fada2c86640 1 --2- 172.21.15.43:0/3881633445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29af390 0x55bbe2d63490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:46.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fada1c84640 1 --2- 172.21.15.43:0/3881633445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 0x55bbe2d62f50 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0c7160 tx=0x7fad8c031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c43e5d26a3d922e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:46.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fad85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:46.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fad85ffb640 1 -- 172.21.15.43:0/3881633445 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c105040 con 0x55bbe298e6b0 2024-04-25T10:40:46.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fad85ffb640 1 -- 172.21.15.43:0/3881633445 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fad8c0dcbe0 con 0x55bbe298e6b0 2024-04-25T10:40:46.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fadab595740 1 -- 172.21.15.43:0/3881633445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad8c0ea0e0 con 0x55bbe298e6b0 2024-04-25T10:40:46.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fad85ffb640 1 -- 172.21.15.43:0/3881633445 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fad8c0c7560 con 0x55bbe298e6b0 2024-04-25T10:40:46.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fadab595740 10 client.17946 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:46.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.595+0000 7fadab595740 1 -- 172.21.15.43:0/3881633445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bbe2d63c00 con 0x55bbe298e6b0 2024-04-25T10:40:46.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.596+0000 7fad85ffb640 1 -- 172.21.15.43:0/3881633445 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad8c0ea0e0 con 0x55bbe298e6b0 2024-04-25T10:40:46.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.596+0000 7fad85ffb640 1 -- 172.21.15.43:0/3881633445 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fad8c0c79f0 con 0x55bbe298e6b0 2024-04-25T10:40:46.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.596+0000 7fadab595740 10 client.17946 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:46.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.596+0000 7fadab595740 10 client.17946 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:40:46.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.596+0000 7fadab595740 20 client.17946 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:46.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.596+0000 7fadab595740 1 --2- 172.21.15.43:0/3881633445 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bbe29433a0 0x55bbe2ddc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.596+0000 7fadab595740 4 client.17946 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:40:46.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.596+0000 7fadab595740 1 -- 172.21.15.43:0/3881633445 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bbe2cbd800 con 0x55bbe29433a0 2024-04-25T10:40:46.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.596+0000 7fada2485640 1 --2- 172.21.15.43:0/3881633445 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bbe29433a0 0x55bbe2ddc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.597+0000 7fad85ffb640 10 client.17946 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:40:46.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.597+0000 7fada2485640 1 --2- 172.21.15.43:0/3881633445 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bbe29433a0 0x55bbe2ddc0f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4bd5630a1d5e1346 server_cookie=8e10eaca390b99a8 in_seq=0 out_seq=0 2024-04-25T10:40:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.597+0000 7fad85ffb640 1 -- 172.21.15.43:0/3881633445 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bbe2cbd800 con 0x55bbe29433a0 2024-04-25T10:40:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.597+0000 7fad85ffb640 10 client.17946 handle_command_reply: tid=0 2024-04-25T10:40:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.597+0000 7fadab595740 1 client.17946 shutdown 2024-04-25T10:40:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.597+0000 7fadab595740 1 -- 172.21.15.43:0/3881633445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 msgr2=0x55bbe2d62f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.597+0000 7fadab595740 1 --2- 172.21.15.43:0/3881633445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 0x55bbe2d62f50 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0c7160 tx=0x7fad8c031a10 comp rx=0 tx=0).stop 2024-04-25T10:40:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.597+0000 7fadab595740 1 -- 172.21.15.43:0/3881633445 shutdown_connections 2024-04-25T10:40:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.597+0000 7fadab595740 1 --2- 172.21.15.43:0/3881633445 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bbe29433a0 0x55bbe2ddc0f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.597+0000 7fadab595740 1 --2- 172.21.15.43:0/3881633445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bbe29af390 0x55bbe2d63490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.597+0000 7fadab595740 1 --2- 172.21.15.43:0/3881633445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbe298e6b0 0x55bbe2d62f50 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.597+0000 7fadab595740 1 --2- 172.21.15.43:0/3881633445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbe29864d0 0x55bbe2d67bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.597+0000 7fad85ffb640 0 client.17946 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:40:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.597+0000 7fadab595740 1 -- 172.21.15.43:0/3881633445 >> 172.21.15.43:0/3881633445 conn(0x55bbe29a34e0 msgr2=0x55bbe2d4c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.597+0000 7fadab595740 1 -- 172.21.15.43:0/3881633445 shutdown_connections 2024-04-25T10:40:46.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.597+0000 7fadab595740 1 -- 172.21.15.43:0/3881633445 wait complete. 2024-04-25T10:40:46.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.597+0000 7fadab595740 20 client.17946 trim_cache size 0 max 16384 2024-04-25T10:40:46.606 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:40:46.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.598+0000 7fada46e9640 1 -- 172.21.15.43:0/1893253051 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fad880afc60 msgr2=0x7fad880b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.598+0000 7fada46e9640 1 --2- 172.21.15.43:0/1893253051 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fad880afc60 0x7fad880b2170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55bbe2d4e070 tx=0x7fad9800b570 comp rx=0 tx=0).stop 2024-04-25T10:40:46.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.598+0000 7fada46e9640 1 -- 172.21.15.43:0/1893253051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9c1069b0 msgr2=0x7fad9c19acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.598+0000 7fada46e9640 1 --2- 172.21.15.43:0/1893253051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9c1069b0 0x7fad9c19acd0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fad8c032fd0 tx=0x7fad8c004270 comp rx=0 tx=0).stop 2024-04-25T10:40:46.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.598+0000 7fada46e9640 1 -- 172.21.15.43:0/1893253051 shutdown_connections 2024-04-25T10:40:46.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.598+0000 7fada46e9640 1 --2- 172.21.15.43:0/1893253051 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fad880afc60 0x7fad880b2170 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.598+0000 7fada46e9640 1 --2- 172.21.15.43:0/1893253051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c107470 0x7fad9c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.598+0000 7fada46e9640 1 --2- 172.21.15.43:0/1893253051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9c1069b0 0x7fad9c19acd0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.598+0000 7fada46e9640 1 --2- 172.21.15.43:0/1893253051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad9c102420 0x7fad9c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.598+0000 7fada46e9640 1 -- 172.21.15.43:0/1893253051 >> 172.21.15.43:0/1893253051 conn(0x7fad9c077920 msgr2=0x7fad9c1008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:46.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.598+0000 7fada46e9640 1 -- 172.21.15.43:0/1893253051 shutdown_connections 2024-04-25T10:40:46.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.598+0000 7fada46e9640 1 -- 172.21.15.43:0/1893253051 wait complete. 2024-04-25T10:40:46.611 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:40:46.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.722+0000 7f50a941e640 1 -- 172.21.15.43:0/836443710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a41027b0 msgr2=0x7f50a4110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.722+0000 7f50a941e640 1 --2- 172.21.15.43:0/836443710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a41027b0 0x7f50a4110630 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f509c009ca0 tx=0x7f509c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.722+0000 7f50a941e640 1 -- 172.21.15.43:0/836443710 shutdown_connections 2024-04-25T10:40:46.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.722+0000 7f50a941e640 1 --2- 172.21.15.43:0/836443710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50a4109fb0 0x7f50a4102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.723+0000 7f50a941e640 1 --2- 172.21.15.43:0/836443710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a41027b0 0x7f50a4110630 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.723+0000 7f50a941e640 1 --2- 172.21.15.43:0/836443710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a4109b10 0x7f50a4101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.723+0000 7f50a941e640 1 -- 172.21.15.43:0/836443710 >> 172.21.15.43:0/836443710 conn(0x7f50a40fd850 msgr2=0x7f50a40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:46.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.723+0000 7f50a941e640 1 -- 172.21.15.43:0/836443710 shutdown_connections 2024-04-25T10:40:46.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.723+0000 7f50a941e640 1 -- 172.21.15.43:0/836443710 wait complete. 2024-04-25T10:40:46.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.723+0000 7f50a941e640 1 Processor -- start 2024-04-25T10:40:46.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.723+0000 7f50a941e640 1 -- start start 2024-04-25T10:40:46.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f50a941e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50a4109b10 0x7f50a4199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f50a941e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a419a420 0x7f50a419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f50a941e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a419ec40 0x7f50a419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f50a941e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f50a4113d20 con 0x7f50a419ec40 2024-04-25T10:40:46.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f50a941e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f50a4113ba0 con 0x7f50a4109b10 2024-04-25T10:40:46.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f50a941e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f50a4113ea0 con 0x7f50a419a420 2024-04-25T10:40:46.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f50a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a419ec40 0x7f50a419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f50a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a419ec40 0x7f50a419f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42058/0 (socket says 172.21.15.43:42058) 2024-04-25T10:40:46.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f50a37fe640 1 -- 172.21.15.43:0/364405963 learned_addr learned my addr 172.21.15.43:0/364405963 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:46.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f50a2ffd640 1 --2- 172.21.15.43:0/364405963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50a4109b10 0x7f50a4199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f50a27fc640 1 --2- 172.21.15.43:0/364405963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a419a420 0x7f50a419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f50a37fe640 1 -- 172.21.15.43:0/364405963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a419a420 msgr2=0x7f50a419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f50a37fe640 1 --2- 172.21.15.43:0/364405963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a419a420 0x7f50a419e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f50a37fe640 1 -- 172.21.15.43:0/364405963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50a4109b10 msgr2=0x7f50a4199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f50a37fe640 1 --2- 172.21.15.43:0/364405963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50a4109b10 0x7f50a4199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f50a37fe640 1 -- 172.21.15.43:0/364405963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a41a69b0 con 0x7f50a419ec40 2024-04-25T10:40:46.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f50a2ffd640 1 --2- 172.21.15.43:0/364405963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50a4109b10 0x7f50a4199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:46.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f50a37fe640 1 --2- 172.21.15.43:0/364405963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a419ec40 0x7f50a419f050 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f509c005cc0 tx=0x7f509c031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:46.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f508bfff640 1 -- 172.21.15.43:0/364405963 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c003b70 con 0x7f50a419ec40 2024-04-25T10:40:46.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f50a941e640 1 -- 172.21.15.43:0/364405963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a41a6b80 con 0x7f50a419ec40 2024-04-25T10:40:46.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f508bfff640 1 -- 172.21.15.43:0/364405963 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f509c0318f0 con 0x7f50a419ec40 2024-04-25T10:40:46.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.724+0000 7f508bfff640 1 -- 172.21.15.43:0/364405963 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c02f950 con 0x7f50a419ec40 2024-04-25T10:40:46.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.725+0000 7f50a941e640 1 -- 172.21.15.43:0/364405963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a41a6fe0 con 0x7f50a419ec40 2024-04-25T10:40:46.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.727+0000 7f508bfff640 1 -- 172.21.15.43:0/364405963 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f509c02fb50 con 0x7f50a419ec40 2024-04-25T10:40:46.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.727+0000 7f50b03a8740 1 Processor -- start 2024-04-25T10:40:46.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.727+0000 7f50b03a8740 1 -- start start 2024-04-25T10:40:46.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.727+0000 7f50b03a8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc9a00390 0x55bcc9da1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.727+0000 7f50b03a8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcc99d74d0 0x55bcc9da1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.727+0000 7f50b03a8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc99df6b0 0x55bcc9dae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.727+0000 7f50b03a8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bcc9d06d80 con 0x55bcc9a00390 2024-04-25T10:40:46.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.727+0000 7f50b03a8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bcc9cc0a20 con 0x55bcc99df6b0 2024-04-25T10:40:46.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.727+0000 7f50b03a8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bcc9d02d40 con 0x55bcc99d74d0 2024-04-25T10:40:46.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.727+0000 7f50a27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc9a00390 0x55bcc9da1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.727+0000 7f50a27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc9a00390 0x55bcc9da1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42060/0 (socket says 172.21.15.43:42060) 2024-04-25T10:40:46.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.727+0000 7f50a27fc640 1 -- 172.21.15.43:0/59877600 learned_addr learned my addr 172.21.15.43:0/59877600 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:46.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.728+0000 7f50a2ffd640 1 --2- 172.21.15.43:0/59877600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcc99d74d0 0x55bcc9da1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.728+0000 7f508bfff640 1 --2- 172.21.15.43:0/364405963 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f508c0afc60 0x7f508c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.728+0000 7f50a27fc640 1 --2- 172.21.15.43:0/59877600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc99df6b0 0x55bcc9dae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.728+0000 7f50a27fc640 1 -- 172.21.15.43:0/59877600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcc99d74d0 msgr2=0x55bcc9da1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.728+0000 7f50a27fc640 1 --2- 172.21.15.43:0/59877600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcc99d74d0 0x55bcc9da1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.728+0000 7f50a27fc640 1 -- 172.21.15.43:0/59877600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc99df6b0 msgr2=0x55bcc9dae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.728+0000 7f50a27fc640 1 --2- 172.21.15.43:0/59877600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc99df6b0 0x55bcc9dae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.728+0000 7f50a37fe640 1 --2- 172.21.15.43:0/364405963 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f508c0afc60 0x7f508c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.728+0000 7f50a27fc640 1 -- 172.21.15.43:0/59877600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcc9ccbc30 con 0x55bcc9a00390 2024-04-25T10:40:46.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.728+0000 7f508bfff640 1 -- 172.21.15.43:0/364405963 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f509c1003d0 con 0x7f50a419ec40 2024-04-25T10:40:46.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.728+0000 7f50a27fc640 1 --2- 172.21.15.43:0/59877600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc9a00390 0x55bcc9da1810 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f5098009fb0 tx=0x7f50980096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc35ebf27f5c417 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:46.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.728+0000 7f50897fa640 1 -- 172.21.15.43:0/59877600 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509801a030 con 0x55bcc9a00390 2024-04-25T10:40:46.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.728+0000 7f50897fa640 1 -- 172.21.15.43:0/59877600 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5098012070 con 0x55bcc9a00390 2024-04-25T10:40:46.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.728+0000 7f50897fa640 1 -- 172.21.15.43:0/59877600 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5098003b40 con 0x55bcc9a00390 2024-04-25T10:40:46.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.729+0000 7f50b03a8740 1 -- 172.21.15.43:0/59877600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc9a00390 msgr2=0x55bcc9da1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.729+0000 7f50a37fe640 1 --2- 172.21.15.43:0/364405963 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f508c0afc60 0x7f508c0b2170 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f509c0fdce0 tx=0x7f509c0047b0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:46.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.729+0000 7f50b03a8740 1 --2- 172.21.15.43:0/59877600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc9a00390 0x55bcc9da1810 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f5098009fb0 tx=0x7f50980096e0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.729+0000 7f50b03a8740 1 -- 172.21.15.43:0/59877600 shutdown_connections 2024-04-25T10:40:46.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.729+0000 7f50b03a8740 1 --2- 172.21.15.43:0/59877600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc99df6b0 0x55bcc9dae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.729+0000 7f50b03a8740 1 --2- 172.21.15.43:0/59877600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcc99d74d0 0x55bcc9da1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.729+0000 7f50b03a8740 1 --2- 172.21.15.43:0/59877600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc9a00390 0x55bcc9da1810 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.729+0000 7f50b03a8740 1 -- 172.21.15.43:0/59877600 >> 172.21.15.43:0/59877600 conn(0x55bcc99f44e0 msgr2=0x55bcc9d9dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:46.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.729+0000 7f50b03a8740 1 -- 172.21.15.43:0/59877600 shutdown_connections 2024-04-25T10:40:46.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.729+0000 7f50b03a8740 1 -- 172.21.15.43:0/59877600 wait complete. 2024-04-25T10:40:46.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.730+0000 7f50b03a8740 1 -- start start 2024-04-25T10:40:46.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.730+0000 7f50b03a8740 1 Processor -- start 2024-04-25T10:40:46.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.730+0000 7f50b03a8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc99d74d0 0x55bcc9dac280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.730+0000 7f50b03a8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 0x55bcc9dac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.730+0000 7f50b03a8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcc9a00390 0x55bcc9db8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.730+0000 7f50b03a8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bcc9daed90 con 0x55bcc99df6b0 2024-04-25T10:40:46.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.730+0000 7f50b03a8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bcc9c8ddc0 con 0x55bcc99d74d0 2024-04-25T10:40:46.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.730+0000 7f50b03a8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bcc9daef10 con 0x55bcc9a00390 2024-04-25T10:40:46.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.730+0000 7f50b03a8740 10 client.0 _ll_register_callbacks cb 0x55bcc9ca9600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:46.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.730+0000 7f50a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 0x55bcc9dac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.730+0000 7f50a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 0x55bcc9dac7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42076/0 (socket says 172.21.15.43:42076) 2024-04-25T10:40:46.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.730+0000 7f50a2ffd640 1 -- 172.21.15.43:0/726423882 learned_addr learned my addr 172.21.15.43:0/726423882 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:46.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.730+0000 7f50a27fc640 1 --2- 172.21.15.43:0/726423882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcc9a00390 0x55bcc9db8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.730+0000 7f50a27fc640 1 --2- 172.21.15.43:0/726423882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc99d74d0 0x55bcc9dac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.730+0000 7f50a2ffd640 1 -- 172.21.15.43:0/726423882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcc9a00390 msgr2=0x55bcc9db8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.730+0000 7f50a2ffd640 1 --2- 172.21.15.43:0/726423882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcc9a00390 0x55bcc9db8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.730+0000 7f50a2ffd640 1 -- 172.21.15.43:0/726423882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc99d74d0 msgr2=0x55bcc9dac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.730+0000 7f50a2ffd640 1 --2- 172.21.15.43:0/726423882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc99d74d0 0x55bcc9dac280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.731+0000 7f50a2ffd640 1 -- 172.21.15.43:0/726423882 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcc9a183f0 con 0x55bcc99df6b0 2024-04-25T10:40:46.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.731+0000 7f50a27fc640 1 --2- 172.21.15.43:0/726423882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc99d74d0 0x55bcc9dac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:46.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.731+0000 7f50a2ffd640 1 --2- 172.21.15.43:0/726423882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 0x55bcc9dac7c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f5094013710 tx=0x7f5094013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a125831c46f1ec94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:46.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.731+0000 7f5086ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:46.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.731+0000 7f5086ff5640 1 -- 172.21.15.43:0/726423882 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5094011dd0 con 0x55bcc99df6b0 2024-04-25T10:40:46.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.731+0000 7f5086ff5640 1 -- 172.21.15.43:0/726423882 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f5094004020 con 0x55bcc99df6b0 2024-04-25T10:40:46.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.731+0000 7f50b03a8740 1 -- 172.21.15.43:0/726423882 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5098012070 con 0x55bcc99df6b0 2024-04-25T10:40:46.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.731+0000 7f5086ff5640 1 -- 172.21.15.43:0/726423882 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5094004220 con 0x55bcc99df6b0 2024-04-25T10:40:46.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.731+0000 7f50b03a8740 10 client.17958 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:46.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.731+0000 7f50b03a8740 1 -- 172.21.15.43:0/726423882 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bcc9ae27d0 con 0x55bcc99df6b0 2024-04-25T10:40:46.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.731+0000 7f5086ff5640 1 -- 172.21.15.43:0/726423882 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5098012070 con 0x55bcc99df6b0 2024-04-25T10:40:46.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.731+0000 7f5086ff5640 1 -- 172.21.15.43:0/726423882 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f509401a070 con 0x55bcc99df6b0 2024-04-25T10:40:46.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.732+0000 7f50b03a8740 10 client.17958 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:46.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.732+0000 7f50b03a8740 10 client.17958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:46.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.732+0000 7f50b03a8740 20 client.17958 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:46.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.732+0000 7f50b03a8740 1 --2- 172.21.15.43:0/726423882 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55bcc99bef60 0x55bcc9dbe7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.732+0000 7f50b03a8740 4 client.17958 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:46.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.732+0000 7f50b03a8740 1 -- 172.21.15.43:0/726423882 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcc9d14990 con 0x55bcc99bef60 2024-04-25T10:40:46.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.732+0000 7f50a27fc640 1 --2- 172.21.15.43:0/726423882 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55bcc99bef60 0x55bcc9dbe7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.732+0000 7f5086ff5640 10 client.17958 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:46.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.732+0000 7f50a27fc640 1 --2- 172.21.15.43:0/726423882 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55bcc99bef60 0x55bcc9dbe7c0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=397cdc2a02c1e068 server_cookie=2d93d3cdd21e5599 in_seq=0 out_seq=0 2024-04-25T10:40:46.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.733+0000 7f5086ff5640 1 -- 172.21.15.43:0/726423882 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bcc9d14990 con 0x55bcc99bef60 2024-04-25T10:40:46.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.733+0000 7f5086ff5640 10 client.17958 handle_command_reply: tid=0 2024-04-25T10:40:46.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.733+0000 7f50b03a8740 1 client.17958 shutdown 2024-04-25T10:40:46.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.733+0000 7f50b03a8740 1 -- 172.21.15.43:0/726423882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 msgr2=0x55bcc9dac7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.733+0000 7f50b03a8740 1 --2- 172.21.15.43:0/726423882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 0x55bcc9dac7c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f5094013710 tx=0x7f5094013bd0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.733+0000 7f50b03a8740 1 -- 172.21.15.43:0/726423882 shutdown_connections 2024-04-25T10:40:46.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.733+0000 7f50b03a8740 1 --2- 172.21.15.43:0/726423882 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55bcc99bef60 0x55bcc9dbe7c0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.733+0000 7f50b03a8740 1 --2- 172.21.15.43:0/726423882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcc9a00390 0x55bcc9db8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.733+0000 7f5086ff5640 0 client.17958 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:46.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.733+0000 7f50b03a8740 1 --2- 172.21.15.43:0/726423882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 0x55bcc9dac7c0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.733+0000 7f50b03a8740 1 --2- 172.21.15.43:0/726423882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc99d74d0 0x55bcc9dac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.733+0000 7f50b03a8740 1 -- 172.21.15.43:0/726423882 >> 172.21.15.43:0/726423882 conn(0x55bcc99f44e0 msgr2=0x55bcc9da3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:46.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.733+0000 7f50b03a8740 1 -- 172.21.15.43:0/726423882 shutdown_connections 2024-04-25T10:40:46.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.733+0000 7f50b03a8740 1 -- 172.21.15.43:0/726423882 wait complete. 2024-04-25T10:40:46.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.734+0000 7f50b03a8740 20 client.17958 trim_cache size 0 max 16384 2024-04-25T10:40:46.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.747+0000 7f50b03a8740 1 Processor -- start 2024-04-25T10:40:46.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.747+0000 7f50b03a8740 1 -- start start 2024-04-25T10:40:46.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.747+0000 7f50b03a8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcc99d74d0 0x55bcc9da1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.747+0000 7f50b03a8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 0x55bcc9db7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.747+0000 7f50b03a8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc9a00390 0x55bcc9da6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.747+0000 7f50b03a8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f50a41b2260 con 0x55bcc99df6b0 2024-04-25T10:40:46.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.747+0000 7f50b03a8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f50a41afb30 con 0x55bcc9a00390 2024-04-25T10:40:46.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.747+0000 7f50b03a8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f50a41ad9a0 con 0x55bcc99d74d0 2024-04-25T10:40:46.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.747+0000 7f50a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 0x55bcc9db7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.748+0000 7f50a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 0x55bcc9db7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42078/0 (socket says 172.21.15.43:42078) 2024-04-25T10:40:46.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.748+0000 7f50a2ffd640 1 -- 172.21.15.43:0/4074992288 learned_addr learned my addr 172.21.15.43:0/4074992288 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:46.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.748+0000 7f50a27fc640 1 --2- 172.21.15.43:0/4074992288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcc99d74d0 0x55bcc9da1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.748+0000 7f50a2ffd640 1 -- 172.21.15.43:0/4074992288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcc99d74d0 msgr2=0x55bcc9da1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.748+0000 7f50a2ffd640 1 --2- 172.21.15.43:0/4074992288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcc99d74d0 0x55bcc9da1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.748+0000 7f50a27fc640 1 --2- 172.21.15.43:0/4074992288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc9a00390 0x55bcc9da6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.748+0000 7f50a2ffd640 1 -- 172.21.15.43:0/4074992288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc9a00390 msgr2=0x55bcc9da6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.748+0000 7f50a2ffd640 1 --2- 172.21.15.43:0/4074992288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc9a00390 0x55bcc9da6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.748+0000 7f50a2ffd640 1 -- 172.21.15.43:0/4074992288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcc9da7c30 con 0x55bcc99df6b0 2024-04-25T10:40:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.748+0000 7f50a2ffd640 1 --2- 172.21.15.43:0/4074992288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 0x55bcc9db7e70 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f5094001eb0 tx=0x7f50940118e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0086de872c2d66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.748+0000 7f5088ff9640 1 -- 172.21.15.43:0/4074992288 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509402f020 con 0x55bcc99df6b0 2024-04-25T10:40:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.748+0000 7f5088ff9640 1 -- 172.21.15.43:0/4074992288 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5094023c60 con 0x55bcc99df6b0 2024-04-25T10:40:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.748+0000 7f5088ff9640 1 -- 172.21.15.43:0/4074992288 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5094000e60 con 0x55bcc99df6b0 2024-04-25T10:40:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.749+0000 7f50b03a8740 1 -- 172.21.15.43:0/4074992288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 msgr2=0x55bcc9db7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.749+0000 7f50b03a8740 1 --2- 172.21.15.43:0/4074992288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 0x55bcc9db7e70 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f5094001eb0 tx=0x7f50940118e0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.749+0000 7f50b03a8740 1 -- 172.21.15.43:0/4074992288 shutdown_connections 2024-04-25T10:40:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.749+0000 7f50b03a8740 1 --2- 172.21.15.43:0/4074992288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc9a00390 0x55bcc9da6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.749+0000 7f50b03a8740 1 --2- 172.21.15.43:0/4074992288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 0x55bcc9db7e70 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.749+0000 7f50b03a8740 1 --2- 172.21.15.43:0/4074992288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcc99d74d0 0x55bcc9da1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.749+0000 7f50b03a8740 1 -- 172.21.15.43:0/4074992288 >> 172.21.15.43:0/4074992288 conn(0x55bcc99f44e0 msgr2=0x55bcc9da15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.749+0000 7f50b03a8740 1 -- 172.21.15.43:0/4074992288 shutdown_connections 2024-04-25T10:40:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.749+0000 7f50b03a8740 1 -- 172.21.15.43:0/4074992288 wait complete. 2024-04-25T10:40:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.749+0000 7f50b03a8740 1 -- start start 2024-04-25T10:40:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.749+0000 7f50b03a8740 1 Processor -- start 2024-04-25T10:40:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f50b03a8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc99d74d0 0x55bcc9db43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f50b03a8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 0x55bcc9db48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f50b03a8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcc9a00390 0x55bcc9daf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f50b03a8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f50a41adff0 con 0x55bcc99df6b0 2024-04-25T10:40:46.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f50b03a8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f50a41af460 con 0x55bcc99d74d0 2024-04-25T10:40:46.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f50b03a8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f50a41ae6c0 con 0x55bcc9a00390 2024-04-25T10:40:46.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f50b03a8740 10 client.0 _ll_register_callbacks cb 0x7f5094009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:46.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f50a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 0x55bcc9db48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f50a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 0x55bcc9db48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42084/0 (socket says 172.21.15.43:42084) 2024-04-25T10:40:46.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f50a2ffd640 1 -- 172.21.15.43:0/3638039902 learned_addr learned my addr 172.21.15.43:0/3638039902 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:46.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f50a27fc640 1 --2- 172.21.15.43:0/3638039902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc99d74d0 0x55bcc9db43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f50a27fc640 1 --2- 172.21.15.43:0/3638039902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcc9a00390 0x55bcc9daf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f50a2ffd640 1 -- 172.21.15.43:0/3638039902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcc9a00390 msgr2=0x55bcc9daf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f50a2ffd640 1 --2- 172.21.15.43:0/3638039902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcc9a00390 0x55bcc9daf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f50a2ffd640 1 -- 172.21.15.43:0/3638039902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc99d74d0 msgr2=0x55bcc9db43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:46.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f50a2ffd640 1 --2- 172.21.15.43:0/3638039902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc99d74d0 0x55bcc9db43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f50a2ffd640 1 -- 172.21.15.43:0/3638039902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcc9db0050 con 0x55bcc99df6b0 2024-04-25T10:40:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f50a27fc640 1 --2- 172.21.15.43:0/3638039902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc99d74d0 0x55bcc9db43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f50a2ffd640 1 --2- 172.21.15.43:0/3638039902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 0x55bcc9db48e0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f509402b450 tx=0x7f5094011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6363c985caf93f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f5086ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f5086ff5640 1 -- 172.21.15.43:0/3638039902 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5094039020 con 0x55bcc99df6b0 2024-04-25T10:40:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f5086ff5640 1 -- 172.21.15.43:0/3638039902 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f509402b4f0 con 0x55bcc99df6b0 2024-04-25T10:40:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.750+0000 7f50b03a8740 1 -- 172.21.15.43:0/3638039902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5094023c60 con 0x55bcc99df6b0 2024-04-25T10:40:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.751+0000 7f5086ff5640 1 -- 172.21.15.43:0/3638039902 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5094044050 con 0x55bcc99df6b0 2024-04-25T10:40:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.751+0000 7f50b03a8740 10 client.17964 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.751+0000 7f50b03a8740 1 -- 172.21.15.43:0/3638039902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bcc9e26650 con 0x55bcc99df6b0 2024-04-25T10:40:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.751+0000 7f5086ff5640 1 -- 172.21.15.43:0/3638039902 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5094023c60 con 0x55bcc99df6b0 2024-04-25T10:40:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.751+0000 7f5086ff5640 1 -- 172.21.15.43:0/3638039902 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f509402a070 con 0x55bcc99df6b0 2024-04-25T10:40:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.751+0000 7f50b03a8740 10 client.17964 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.751+0000 7f50b03a8740 10 client.17964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:46.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.751+0000 7f50b03a8740 20 client.17964 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:46.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.751+0000 7f50b03a8740 1 --2- 172.21.15.43:0/3638039902 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55bcc99943a0 0x55bcc9e28d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:46.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.751+0000 7f50b03a8740 4 client.17964 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:40:46.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.751+0000 7f50b03a8740 1 -- 172.21.15.43:0/3638039902 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bcc9d0e800 con 0x55bcc99943a0 2024-04-25T10:40:46.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.751+0000 7f50a27fc640 1 --2- 172.21.15.43:0/3638039902 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55bcc99943a0 0x55bcc9e28d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:46.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.752+0000 7f5086ff5640 10 client.17964 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:46.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.752+0000 7f50a27fc640 1 --2- 172.21.15.43:0/3638039902 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55bcc99943a0 0x55bcc9e28d20 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccf0a7b0e3636edf server_cookie=e006e43048c63dce in_seq=0 out_seq=0 2024-04-25T10:40:46.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.752+0000 7f5086ff5640 1 -- 172.21.15.43:0/3638039902 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bcc9d0e800 con 0x55bcc99943a0 2024-04-25T10:40:46.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.752+0000 7f5086ff5640 10 client.17964 handle_command_reply: tid=0 2024-04-25T10:40:46.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.752+0000 7f50b03a8740 1 client.17964 shutdown 2024-04-25T10:40:46.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.752+0000 7f50b03a8740 1 -- 172.21.15.43:0/3638039902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 msgr2=0x55bcc9db48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.752+0000 7f50b03a8740 1 --2- 172.21.15.43:0/3638039902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 0x55bcc9db48e0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f509402b450 tx=0x7f5094011e00 comp rx=0 tx=0).stop 2024-04-25T10:40:46.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.752+0000 7f50b03a8740 1 -- 172.21.15.43:0/3638039902 shutdown_connections 2024-04-25T10:40:46.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.752+0000 7f50b03a8740 1 --2- 172.21.15.43:0/3638039902 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55bcc99943a0 0x55bcc9e28d20 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.752+0000 7f50b03a8740 1 --2- 172.21.15.43:0/3638039902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcc9a00390 0x55bcc9daf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.752+0000 7f50b03a8740 1 --2- 172.21.15.43:0/3638039902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc99df6b0 0x55bcc9db48e0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.752+0000 7f50b03a8740 1 --2- 172.21.15.43:0/3638039902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc99d74d0 0x55bcc9db43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.752+0000 7f5086ff5640 0 client.17964 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:46.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.752+0000 7f50b03a8740 1 -- 172.21.15.43:0/3638039902 >> 172.21.15.43:0/3638039902 conn(0x55bcc99f44e0 msgr2=0x55bcc9d9d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:46.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.752+0000 7f50b03a8740 1 -- 172.21.15.43:0/3638039902 shutdown_connections 2024-04-25T10:40:46.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.752+0000 7f50b03a8740 1 -- 172.21.15.43:0/3638039902 wait complete. 2024-04-25T10:40:46.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.753+0000 7f50b03a8740 20 client.17964 trim_cache size 0 max 16384 2024-04-25T10:40:46.761 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:40:46.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.753+0000 7f50a941e640 1 -- 172.21.15.43:0/364405963 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f508c0afc60 msgr2=0x7f508c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.753+0000 7f50a941e640 1 --2- 172.21.15.43:0/364405963 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f508c0afc60 0x7f508c0b2170 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f509c0fdce0 tx=0x7f509c0047b0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.753+0000 7f50a941e640 1 -- 172.21.15.43:0/364405963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a419ec40 msgr2=0x7f50a419f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:46.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.753+0000 7f50a941e640 1 --2- 172.21.15.43:0/364405963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a419ec40 0x7f50a419f050 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f509c005cc0 tx=0x7f509c031700 comp rx=0 tx=0).stop 2024-04-25T10:40:46.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.753+0000 7f50a941e640 1 -- 172.21.15.43:0/364405963 shutdown_connections 2024-04-25T10:40:46.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.753+0000 7f50a941e640 1 --2- 172.21.15.43:0/364405963 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f508c0afc60 0x7f508c0b2170 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.753+0000 7f50a941e640 1 --2- 172.21.15.43:0/364405963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a419ec40 0x7f50a419f050 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.753+0000 7f50a941e640 1 --2- 172.21.15.43:0/364405963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a419a420 0x7f50a419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.753+0000 7f50a941e640 1 --2- 172.21.15.43:0/364405963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50a4109b10 0x7f50a4199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:46.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.753+0000 7f50a941e640 1 -- 172.21.15.43:0/364405963 >> 172.21.15.43:0/364405963 conn(0x7f50a40fd850 msgr2=0x7f50a41071e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:46.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.753+0000 7f50a941e640 1 -- 172.21.15.43:0/364405963 shutdown_connections 2024-04-25T10:40:46.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:46.753+0000 7f50a941e640 1 -- 172.21.15.43:0/364405963 wait complete. 2024-04-25T10:40:47.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:46 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/331184130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:40:47.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:46 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/331184130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:40:47.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:46 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/331184130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:40:47.767 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:40:47.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:40:47.767 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:40:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.896+0000 7f735d764640 1 -- 172.21.15.43:0/988487862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7358104cb0 msgr2=0x7f73581050c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.896+0000 7f735d764640 1 --2- 172.21.15.43:0/988487862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7358104cb0 0x7f73581050c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f7348009900 tx=0x7f734802a480 comp rx=0 tx=0).stop 2024-04-25T10:40:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.896+0000 7f735d764640 1 -- 172.21.15.43:0/988487862 shutdown_connections 2024-04-25T10:40:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.896+0000 7f735d764640 1 --2- 172.21.15.43:0/988487862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f735810e070 0x7f7358110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.896+0000 7f735d764640 1 --2- 172.21.15.43:0/988487862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f735810b710 0x7f735810db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.896+0000 7f735d764640 1 --2- 172.21.15.43:0/988487862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7358104cb0 0x7f73581050c0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.896+0000 7f735d764640 1 -- 172.21.15.43:0/988487862 >> 172.21.15.43:0/988487862 conn(0x7f7358078100 msgr2=0x7f7358078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.896+0000 7f735d764640 1 -- 172.21.15.43:0/988487862 shutdown_connections 2024-04-25T10:40:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.896+0000 7f735d764640 1 -- 172.21.15.43:0/988487862 wait complete. 2024-04-25T10:40:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.897+0000 7f735d764640 1 Processor -- start 2024-04-25T10:40:47.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.897+0000 7f735d764640 1 -- start start 2024-04-25T10:40:47.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.897+0000 7f735d764640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7358104cb0 0x7f735819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:47.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.897+0000 7f735d764640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f735810b710 0x7f735819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:47.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.897+0000 7f735d764640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f735810e070 0x7f735819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:47.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.897+0000 7f735d764640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7358113cb0 con 0x7f735810e070 2024-04-25T10:40:47.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.897+0000 7f735d764640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7358113b30 con 0x7f735810b710 2024-04-25T10:40:47.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.897+0000 7f735d764640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7358113e30 con 0x7f7358104cb0 2024-04-25T10:40:47.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.898+0000 7f73577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f735810e070 0x7f735819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:47.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.898+0000 7f73577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f735810e070 0x7f735819f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42102/0 (socket says 172.21.15.43:42102) 2024-04-25T10:40:47.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.898+0000 7f73577fe640 1 -- 172.21.15.43:0/1973016942 learned_addr learned my addr 172.21.15.43:0/1973016942 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:47.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.898+0000 7f7356ffd640 1 --2- 172.21.15.43:0/1973016942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7358104cb0 0x7f735819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:47.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.898+0000 7f73567fc640 1 --2- 172.21.15.43:0/1973016942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f735810b710 0x7f735819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:47.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.898+0000 7f73577fe640 1 -- 172.21.15.43:0/1973016942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7358104cb0 msgr2=0x7f735819a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:47.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.898+0000 7f73577fe640 1 --2- 172.21.15.43:0/1973016942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7358104cb0 0x7f735819a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.898+0000 7f73577fe640 1 -- 172.21.15.43:0/1973016942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f735810b710 msgr2=0x7f735819ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:47.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.898+0000 7f73577fe640 1 --2- 172.21.15.43:0/1973016942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f735810b710 0x7f735819ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.898+0000 7f73577fe640 1 -- 172.21.15.43:0/1973016942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f735819f650 con 0x7f735810e070 2024-04-25T10:40:47.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.898+0000 7f7356ffd640 1 --2- 172.21.15.43:0/1973016942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7358104cb0 0x7f735819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:47.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.898+0000 7f73567fc640 1 --2- 172.21.15.43:0/1973016942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f735810b710 0x7f735819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:47.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.898+0000 7f73577fe640 1 --2- 172.21.15.43:0/1973016942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f735810e070 0x7f735819f010 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f7350009fd0 tx=0x7f735000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:47.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.898+0000 7f733ffff640 1 -- 172.21.15.43:0/1973016942 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7350017070 con 0x7f735810e070 2024-04-25T10:40:47.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.898+0000 7f735d764640 1 -- 172.21.15.43:0/1973016942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f735819f940 con 0x7f735810e070 2024-04-25T10:40:47.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.898+0000 7f733ffff640 1 -- 172.21.15.43:0/1973016942 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7350003e20 con 0x7f735810e070 2024-04-25T10:40:47.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.898+0000 7f733ffff640 1 -- 172.21.15.43:0/1973016942 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7350010460 con 0x7f735810e070 2024-04-25T10:40:47.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.898+0000 7f735d764640 1 -- 172.21.15.43:0/1973016942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f735819fe40 con 0x7f735810e070 2024-04-25T10:40:47.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.901+0000 7f7364612740 1 Processor -- start 2024-04-25T10:40:47.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.901+0000 7f733ffff640 1 -- 172.21.15.43:0/1973016942 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f7350003970 con 0x7f735810e070 2024-04-25T10:40:47.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.901+0000 7f7364612740 1 -- start start 2024-04-25T10:40:47.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.901+0000 7f7364612740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158cd410 0x560815c6d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:47.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.901+0000 7f7364612740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608158a4550 0x560815c6db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:47.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.901+0000 7f7364612740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608158ac730 0x560815c6e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:47.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.901+0000 7f7364612740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560815bd3b00 con 0x5608158cd410 2024-04-25T10:40:47.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.901+0000 7f7364612740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560815b6f920 con 0x5608158a4550 2024-04-25T10:40:47.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.901+0000 7f7364612740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560815bcfac0 con 0x5608158ac730 2024-04-25T10:40:47.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.901+0000 7f73567fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158cd410 0x560815c6d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:47.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.902+0000 7f73567fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158cd410 0x560815c6d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42114/0 (socket says 172.21.15.43:42114) 2024-04-25T10:40:47.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.902+0000 7f73567fc640 1 -- 172.21.15.43:0/2759434465 learned_addr learned my addr 172.21.15.43:0/2759434465 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:47.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.902+0000 7f7356ffd640 1 --2- 172.21.15.43:0/2759434465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608158a4550 0x560815c6db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:47.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.902+0000 7f73567fc640 1 --2- 172.21.15.43:0/2759434465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608158ac730 0x560815c6e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:47.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.902+0000 7f733ffff640 1 --2- 172.21.15.43:0/1973016942 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f73400afc60 0x7f73400b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:47.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.902+0000 7f73567fc640 1 -- 172.21.15.43:0/2759434465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608158ac730 msgr2=0x560815c6e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:47.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.902+0000 7f73567fc640 1 --2- 172.21.15.43:0/2759434465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608158ac730 0x560815c6e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.902+0000 7f73567fc640 1 -- 172.21.15.43:0/2759434465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608158a4550 msgr2=0x560815c6db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:47.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.902+0000 7f73567fc640 1 --2- 172.21.15.43:0/2759434465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608158a4550 0x560815c6db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.902+0000 7f73567fc640 1 -- 172.21.15.43:0/2759434465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608159ad7e0 con 0x5608158cd410 2024-04-25T10:40:47.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.902+0000 7f73577fe640 1 --2- 172.21.15.43:0/1973016942 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f73400afc60 0x7f73400b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:47.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.902+0000 7f73567fc640 1 --2- 172.21.15.43:0/2759434465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158cd410 0x560815c6d620 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f734c0124c0 tx=0x7f734c00c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd8a2f7b20462ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:47.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.902+0000 7f733ffff640 1 -- 172.21.15.43:0/1973016942 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f73500dee30 con 0x7f735810e070 2024-04-25T10:40:47.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.902+0000 7f733d7fa640 1 -- 172.21.15.43:0/2759434465 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c018030 con 0x5608158cd410 2024-04-25T10:40:47.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.902+0000 7f733d7fa640 1 -- 172.21.15.43:0/2759434465 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f734c0054d0 con 0x5608158cd410 2024-04-25T10:40:47.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.902+0000 7f733d7fa640 1 -- 172.21.15.43:0/2759434465 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c010840 con 0x5608158cd410 2024-04-25T10:40:47.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.903+0000 7f7364612740 1 -- 172.21.15.43:0/2759434465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158cd410 msgr2=0x560815c6d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:47.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.903+0000 7f73577fe640 1 --2- 172.21.15.43:0/1973016942 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f73400afc60 0x7f73400b2170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f73500a4240 tx=0x7f7350074830 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:47.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.903+0000 7f7364612740 1 --2- 172.21.15.43:0/2759434465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158cd410 0x560815c6d620 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f734c0124c0 tx=0x7f734c00c640 comp rx=0 tx=0).stop 2024-04-25T10:40:47.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.903+0000 7f7364612740 1 -- 172.21.15.43:0/2759434465 shutdown_connections 2024-04-25T10:40:47.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.903+0000 7f7364612740 1 --2- 172.21.15.43:0/2759434465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608158a4550 0x560815c6db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.903+0000 7f7364612740 1 --2- 172.21.15.43:0/2759434465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608158ac730 0x560815c6e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.903+0000 7f7364612740 1 --2- 172.21.15.43:0/2759434465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158cd410 0x560815c6d620 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.903+0000 7f7364612740 1 -- 172.21.15.43:0/2759434465 >> 172.21.15.43:0/2759434465 conn(0x5608158c1560 msgr2=0x560815c69dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:47.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.903+0000 7f7364612740 1 -- 172.21.15.43:0/2759434465 shutdown_connections 2024-04-25T10:40:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.903+0000 7f7364612740 1 -- 172.21.15.43:0/2759434465 wait complete. 2024-04-25T10:40:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.904+0000 7f7364612740 1 -- start start 2024-04-25T10:40:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.904+0000 7f7364612740 1 Processor -- start 2024-04-25T10:40:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.904+0000 7f7364612740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608158a4550 0x560815c77c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.904+0000 7f7364612740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608158ac730 0x560815c78180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.904+0000 7f7364612740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158cd410 0x560815c83e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.904+0000 7f7364612740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560815c7a970 con 0x5608158cd410 2024-04-25T10:40:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.904+0000 7f7364612740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560815b6d5d0 con 0x5608158ac730 2024-04-25T10:40:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.904+0000 7f7364612740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560815c7aaf0 con 0x5608158a4550 2024-04-25T10:40:47.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.904+0000 7f7364612740 10 client.0 _ll_register_callbacks cb 0x560815b5b080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:47.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.904+0000 7f73567fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158cd410 0x560815c83e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:47.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.904+0000 7f73567fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158cd410 0x560815c83e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42124/0 (socket says 172.21.15.43:42124) 2024-04-25T10:40:47.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.904+0000 7f73567fc640 1 -- 172.21.15.43:0/3106014948 learned_addr learned my addr 172.21.15.43:0/3106014948 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:47.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.904+0000 7f7356ffd640 1 --2- 172.21.15.43:0/3106014948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608158ac730 0x560815c78180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:47.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.905+0000 7f73567fc640 1 -- 172.21.15.43:0/3106014948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608158a4550 msgr2=0x560815c77c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:47.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.905+0000 7f73567fc640 1 --2- 172.21.15.43:0/3106014948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608158a4550 0x560815c77c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.905+0000 7f73567fc640 1 -- 172.21.15.43:0/3106014948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608158ac730 msgr2=0x560815c78180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:47.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.905+0000 7f73567fc640 1 --2- 172.21.15.43:0/3106014948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608158ac730 0x560815c78180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.905+0000 7f73567fc640 1 -- 172.21.15.43:0/3106014948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560815a0df70 con 0x5608158cd410 2024-04-25T10:40:47.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.905+0000 7f73567fc640 1 --2- 172.21.15.43:0/3106014948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158cd410 0x560815c83e20 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f734c012e10 tx=0x7f734c005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ade3f601f08f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:47.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.905+0000 7f733aff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:47.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.905+0000 7f733aff5640 1 -- 172.21.15.43:0/3106014948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c018050 con 0x5608158cd410 2024-04-25T10:40:47.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.905+0000 7f733aff5640 1 -- 172.21.15.43:0/3106014948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f734c004e10 con 0x5608158cd410 2024-04-25T10:40:47.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.905+0000 7f7364612740 1 -- 172.21.15.43:0/3106014948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f734c0054d0 con 0x5608158cd410 2024-04-25T10:40:47.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.905+0000 7f733aff5640 1 -- 172.21.15.43:0/3106014948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f734c0048d0 con 0x5608158cd410 2024-04-25T10:40:47.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.905+0000 7f7364612740 10 client.17976 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:47.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.905+0000 7f7364612740 1 -- 172.21.15.43:0/3106014948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5608158a5170 con 0x5608158cd410 2024-04-25T10:40:47.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.905+0000 7f733aff5640 1 -- 172.21.15.43:0/3106014948 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f734c0054d0 con 0x5608158cd410 2024-04-25T10:40:47.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.906+0000 7f733aff5640 1 -- 172.21.15.43:0/3106014948 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f734c01d620 con 0x5608158cd410 2024-04-25T10:40:47.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.906+0000 7f7364612740 10 client.17976 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:47.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.906+0000 7f7364612740 10 client.17976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:47.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.906+0000 7f7364612740 20 client.17976 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:47.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.906+0000 7f7364612740 1 --2- 172.21.15.43:0/3106014948 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56081588c1f0 0x560815c89f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:47.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.906+0000 7f7364612740 4 client.17976 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:47.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.906+0000 7f7364612740 1 -- 172.21.15.43:0/3106014948 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560815be1710 con 0x56081588c1f0 2024-04-25T10:40:47.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.906+0000 7f7356ffd640 1 --2- 172.21.15.43:0/3106014948 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56081588c1f0 0x560815c89f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:47.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.906+0000 7f733aff5640 10 client.17976 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:47.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.906+0000 7f7356ffd640 1 --2- 172.21.15.43:0/3106014948 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56081588c1f0 0x560815c89f20 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a0342116fc0cc5d server_cookie=e2b086db5c0cc6f7 in_seq=0 out_seq=0 2024-04-25T10:40:47.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.907+0000 7f733aff5640 1 -- 172.21.15.43:0/3106014948 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560815be1710 con 0x56081588c1f0 2024-04-25T10:40:47.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.907+0000 7f733aff5640 10 client.17976 handle_command_reply: tid=0 2024-04-25T10:40:47.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.907+0000 7f7364612740 1 client.17976 shutdown 2024-04-25T10:40:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.907+0000 7f7364612740 1 -- 172.21.15.43:0/3106014948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158cd410 msgr2=0x560815c83e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.907+0000 7f7364612740 1 --2- 172.21.15.43:0/3106014948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158cd410 0x560815c83e20 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f734c012e10 tx=0x7f734c005230 comp rx=0 tx=0).stop 2024-04-25T10:40:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.907+0000 7f7364612740 1 -- 172.21.15.43:0/3106014948 shutdown_connections 2024-04-25T10:40:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.907+0000 7f7364612740 1 --2- 172.21.15.43:0/3106014948 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56081588c1f0 0x560815c89f20 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.907+0000 7f7364612740 1 --2- 172.21.15.43:0/3106014948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608158ac730 0x560815c78180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.907+0000 7f7364612740 1 --2- 172.21.15.43:0/3106014948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158cd410 0x560815c83e20 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.907+0000 7f733aff5640 0 client.17976 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.907+0000 7f7364612740 1 --2- 172.21.15.43:0/3106014948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608158a4550 0x560815c77c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.907+0000 7f7364612740 1 -- 172.21.15.43:0/3106014948 >> 172.21.15.43:0/3106014948 conn(0x5608158c1560 msgr2=0x560815c6d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.908+0000 7f7364612740 1 -- 172.21.15.43:0/3106014948 shutdown_connections 2024-04-25T10:40:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.908+0000 7f7364612740 1 -- 172.21.15.43:0/3106014948 wait complete. 2024-04-25T10:40:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.908+0000 7f7364612740 20 client.17976 trim_cache size 0 max 16384 2024-04-25T10:40:47.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.926+0000 7f7364612740 1 Processor -- start 2024-04-25T10:40:47.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.926+0000 7f7364612740 1 -- start start 2024-04-25T10:40:47.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.926+0000 7f7364612740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56081588c1f0 0x560815c717e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:47.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.926+0000 7f7364612740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608158a4550 0x560815c8f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:47.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.926+0000 7f7364612740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158ac730 0x560815c8fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:47.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.926+0000 7f7364612740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f73581b2310 con 0x5608158ac730 2024-04-25T10:40:47.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.926+0000 7f7364612740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f73581afbe0 con 0x56081588c1f0 2024-04-25T10:40:47.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.926+0000 7f7364612740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f73581ada50 con 0x5608158a4550 2024-04-25T10:40:47.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.926+0000 7f73567fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158ac730 0x560815c8fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:47.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.926+0000 7f73567fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158ac730 0x560815c8fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42134/0 (socket says 172.21.15.43:42134) 2024-04-25T10:40:47.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.926+0000 7f73567fc640 1 -- 172.21.15.43:0/4278855689 learned_addr learned my addr 172.21.15.43:0/4278855689 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:47.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.927+0000 7f73567fc640 1 -- 172.21.15.43:0/4278855689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608158a4550 msgr2=0x560815c8f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:47.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.927+0000 7f73567fc640 1 --2- 172.21.15.43:0/4278855689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608158a4550 0x560815c8f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.927+0000 7f73567fc640 1 -- 172.21.15.43:0/4278855689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56081588c1f0 msgr2=0x560815c717e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:47.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.927+0000 7f73567fc640 1 --2- 172.21.15.43:0/4278855689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56081588c1f0 0x560815c717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.927+0000 7f73567fc640 1 -- 172.21.15.43:0/4278855689 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560815c7a850 con 0x5608158ac730 2024-04-25T10:40:47.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.927+0000 7f73567fc640 1 --2- 172.21.15.43:0/4278855689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158ac730 0x560815c8fa60 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f734c005a70 tx=0x7f734c004c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7a0dfc3c72bae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:47.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.927+0000 7f733cff9640 1 -- 172.21.15.43:0/4278855689 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c018030 con 0x5608158ac730 2024-04-25T10:40:47.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.927+0000 7f733cff9640 1 -- 172.21.15.43:0/4278855689 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f734c004c90 con 0x5608158ac730 2024-04-25T10:40:47.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.927+0000 7f733cff9640 1 -- 172.21.15.43:0/4278855689 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c010c20 con 0x5608158ac730 2024-04-25T10:40:47.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.928+0000 7f7364612740 1 -- 172.21.15.43:0/4278855689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158ac730 msgr2=0x560815c8fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:47.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.928+0000 7f7364612740 1 --2- 172.21.15.43:0/4278855689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158ac730 0x560815c8fa60 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f734c005a70 tx=0x7f734c004c20 comp rx=0 tx=0).stop 2024-04-25T10:40:47.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.928+0000 7f7364612740 1 -- 172.21.15.43:0/4278855689 shutdown_connections 2024-04-25T10:40:47.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.928+0000 7f7364612740 1 --2- 172.21.15.43:0/4278855689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158ac730 0x560815c8fa60 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.928+0000 7f7364612740 1 --2- 172.21.15.43:0/4278855689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608158a4550 0x560815c8f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.928+0000 7f7364612740 1 --2- 172.21.15.43:0/4278855689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56081588c1f0 0x560815c717e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.928+0000 7f7364612740 1 -- 172.21.15.43:0/4278855689 >> 172.21.15.43:0/4278855689 conn(0x5608158c1560 msgr2=0x560815c710d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:47.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.928+0000 7f7364612740 1 -- 172.21.15.43:0/4278855689 shutdown_connections 2024-04-25T10:40:47.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.928+0000 7f7364612740 1 -- 172.21.15.43:0/4278855689 wait complete. 2024-04-25T10:40:47.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.928+0000 7f7364612740 1 -- start start 2024-04-25T10:40:47.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.928+0000 7f7364612740 1 Processor -- start 2024-04-25T10:40:47.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.928+0000 7f7364612740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56081588c1f0 0x560815c80100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:47.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.928+0000 7f7364612740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158a4550 0x560815c7b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:47.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.928+0000 7f7364612740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608158ac730 0x560815c7b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:47.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.928+0000 7f7364612740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f73581ae0a0 con 0x5608158a4550 2024-04-25T10:40:47.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.928+0000 7f7364612740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f73581af510 con 0x5608158ac730 2024-04-25T10:40:47.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.928+0000 7f7364612740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f73581ae770 con 0x56081588c1f0 2024-04-25T10:40:47.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.928+0000 7f7364612740 10 client.0 _ll_register_callbacks cb 0x7f734c00c340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:47.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.929+0000 7f7356ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158a4550 0x560815c7b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:47.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.929+0000 7f7356ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158a4550 0x560815c7b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42150/0 (socket says 172.21.15.43:42150) 2024-04-25T10:40:47.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.929+0000 7f7356ffd640 1 -- 172.21.15.43:0/908838145 learned_addr learned my addr 172.21.15.43:0/908838145 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:47.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.929+0000 7f73567fc640 1 --2- 172.21.15.43:0/908838145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56081588c1f0 0x560815c80100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:47.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.929+0000 7f73567fc640 1 --2- 172.21.15.43:0/908838145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608158ac730 0x560815c7b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:47.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.929+0000 7f7356ffd640 1 -- 172.21.15.43:0/908838145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56081588c1f0 msgr2=0x560815c80100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:47.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.929+0000 7f7356ffd640 1 --2- 172.21.15.43:0/908838145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56081588c1f0 0x560815c80100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.929+0000 7f7356ffd640 1 -- 172.21.15.43:0/908838145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608158ac730 msgr2=0x560815c7b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:47.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.929+0000 7f7356ffd640 1 --2- 172.21.15.43:0/908838145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608158ac730 0x560815c7b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.929+0000 7f7356ffd640 1 -- 172.21.15.43:0/908838145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560815c7bf00 con 0x5608158a4550 2024-04-25T10:40:47.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.929+0000 7f7356ffd640 1 --2- 172.21.15.43:0/908838145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158a4550 0x560815c7b480 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f734802a9a0 tx=0x7f7348004210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3968718d0a74b92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:47.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.929+0000 7f733aff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:47.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.929+0000 7f733aff5640 1 -- 172.21.15.43:0/908838145 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348049060 con 0x5608158a4550 2024-04-25T10:40:47.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.929+0000 7f733aff5640 1 -- 172.21.15.43:0/908838145 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f7348005240 con 0x5608158a4550 2024-04-25T10:40:47.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.929+0000 7f7364612740 1 -- 172.21.15.43:0/908838145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f734c004c90 con 0x5608158a4550 2024-04-25T10:40:47.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.929+0000 7f733aff5640 1 -- 172.21.15.43:0/908838145 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7348035040 con 0x5608158a4550 2024-04-25T10:40:47.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.929+0000 7f7364612740 10 client.17982 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:47.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.929+0000 7f7364612740 1 -- 172.21.15.43:0/908838145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560815cf1e00 con 0x5608158a4550 2024-04-25T10:40:47.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.930+0000 7f733aff5640 1 -- 172.21.15.43:0/908838145 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f734c004c90 con 0x5608158a4550 2024-04-25T10:40:47.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.930+0000 7f733aff5640 1 -- 172.21.15.43:0/908838145 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7348004750 con 0x5608158a4550 2024-04-25T10:40:47.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.930+0000 7f7364612740 10 client.17982 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:47.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.930+0000 7f7364612740 10 client.17982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:47.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.930+0000 7f7364612740 20 client.17982 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:47.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.930+0000 7f7364612740 1 --2- 172.21.15.43:0/908838145 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5608158cd410 0x560815cf45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:47.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.930+0000 7f7364612740 4 client.17982 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:40:47.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.930+0000 7f7364612740 1 -- 172.21.15.43:0/908838145 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560815bdb580 con 0x5608158cd410 2024-04-25T10:40:47.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.930+0000 7f73567fc640 1 --2- 172.21.15.43:0/908838145 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5608158cd410 0x560815cf45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:47.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.931+0000 7f733aff5640 10 client.17982 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:47.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.931+0000 7f73567fc640 1 --2- 172.21.15.43:0/908838145 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5608158cd410 0x560815cf45f0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5ae092fb305517d server_cookie=80659f411de266a in_seq=0 out_seq=0 2024-04-25T10:40:47.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.932+0000 7f733aff5640 1 -- 172.21.15.43:0/908838145 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560815bdb580 con 0x5608158cd410 2024-04-25T10:40:47.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.932+0000 7f733aff5640 10 client.17982 handle_command_reply: tid=0 2024-04-25T10:40:47.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.932+0000 7f7364612740 1 client.17982 shutdown 2024-04-25T10:40:47.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.932+0000 7f7364612740 1 -- 172.21.15.43:0/908838145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158a4550 msgr2=0x560815c7b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:47.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.932+0000 7f7364612740 1 --2- 172.21.15.43:0/908838145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158a4550 0x560815c7b480 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f734802a9a0 tx=0x7f7348004210 comp rx=0 tx=0).stop 2024-04-25T10:40:47.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.932+0000 7f7364612740 1 -- 172.21.15.43:0/908838145 shutdown_connections 2024-04-25T10:40:47.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.932+0000 7f7364612740 1 --2- 172.21.15.43:0/908838145 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5608158cd410 0x560815cf45f0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.932+0000 7f7364612740 1 --2- 172.21.15.43:0/908838145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608158ac730 0x560815c7b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.932+0000 7f7364612740 1 --2- 172.21.15.43:0/908838145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608158a4550 0x560815c7b480 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.932+0000 7f7364612740 1 --2- 172.21.15.43:0/908838145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56081588c1f0 0x560815c80100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.932+0000 7f733aff5640 0 client.17982 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:47.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.932+0000 7f7364612740 1 -- 172.21.15.43:0/908838145 >> 172.21.15.43:0/908838145 conn(0x5608158c1560 msgr2=0x560815c684c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:47.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.932+0000 7f7364612740 1 -- 172.21.15.43:0/908838145 shutdown_connections 2024-04-25T10:40:47.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.932+0000 7f7364612740 1 -- 172.21.15.43:0/908838145 wait complete. 2024-04-25T10:40:47.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.932+0000 7f7364612740 20 client.17982 trim_cache size 0 max 16384 2024-04-25T10:40:47.941 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:40:47.942 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:40:47.942 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "c96ff49a-811c-4223-a4e5-afc3b774de51", 2024-04-25T10:40:47.942 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:40:47.942 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:40:47.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.933+0000 7f735d764640 1 -- 172.21.15.43:0/1973016942 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f73400afc60 msgr2=0x7f73400b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:47.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.933+0000 7f735d764640 1 --2- 172.21.15.43:0/1973016942 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f73400afc60 0x7f73400b2170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f73500a4240 tx=0x7f7350074830 comp rx=0 tx=0).stop 2024-04-25T10:40:47.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.933+0000 7f735d764640 1 -- 172.21.15.43:0/1973016942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f735810e070 msgr2=0x7f735819f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:47.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.933+0000 7f735d764640 1 --2- 172.21.15.43:0/1973016942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f735810e070 0x7f735819f010 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f7350009fd0 tx=0x7f735000b260 comp rx=0 tx=0).stop 2024-04-25T10:40:47.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.933+0000 7f735d764640 1 -- 172.21.15.43:0/1973016942 shutdown_connections 2024-04-25T10:40:47.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.933+0000 7f735d764640 1 --2- 172.21.15.43:0/1973016942 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f73400afc60 0x7f73400b2170 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.933+0000 7f735d764640 1 --2- 172.21.15.43:0/1973016942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f735810b710 0x7f735819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.933+0000 7f735d764640 1 --2- 172.21.15.43:0/1973016942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f735810e070 0x7f735819f010 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.933+0000 7f735d764640 1 --2- 172.21.15.43:0/1973016942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7358104cb0 0x7f735819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:47.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.933+0000 7f735d764640 1 -- 172.21.15.43:0/1973016942 >> 172.21.15.43:0/1973016942 conn(0x7f7358078100 msgr2=0x7f7358101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:47.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.933+0000 7f735d764640 1 -- 172.21.15.43:0/1973016942 shutdown_connections 2024-04-25T10:40:47.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:47.933+0000 7f735d764640 1 -- 172.21.15.43:0/1973016942 wait complete. 2024-04-25T10:40:47.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:40:47.947 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:40:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.059+0000 7fe7ddef2640 1 -- 172.21.15.43:0/279953013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8109b20 msgr2=0x7fe7d81036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.059+0000 7fe7ddef2640 1 --2- 172.21.15.43:0/279953013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8109b20 0x7fe7d81036c0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009900 tx=0x7fe7c802a410 comp rx=0 tx=0).stop 2024-04-25T10:40:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.059+0000 7fe7ddef2640 1 -- 172.21.15.43:0/279953013 shutdown_connections 2024-04-25T10:40:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.059+0000 7fe7ddef2640 1 --2- 172.21.15.43:0/279953013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d8104270 0x7fe7d8110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.059+0000 7fe7ddef2640 1 --2- 172.21.15.43:0/279953013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d8109fc0 0x7fe7d8103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.059+0000 7fe7ddef2640 1 --2- 172.21.15.43:0/279953013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8109b20 0x7fe7d81036c0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.059+0000 7fe7ddef2640 1 -- 172.21.15.43:0/279953013 >> 172.21.15.43:0/279953013 conn(0x7fe7d80ff1e0 msgr2=0x7fe7d8101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.059+0000 7fe7ddef2640 1 -- 172.21.15.43:0/279953013 shutdown_connections 2024-04-25T10:40:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.059+0000 7fe7ddef2640 1 -- 172.21.15.43:0/279953013 wait complete. 2024-04-25T10:40:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.060+0000 7fe7ddef2640 1 Processor -- start 2024-04-25T10:40:48.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.060+0000 7fe7ddef2640 1 -- start start 2024-04-25T10:40:48.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.060+0000 7fe7ddef2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d8104270 0x7fe7d819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:48.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.060+0000 7fe7ddef2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8109b20 0x7fe7d819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:48.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.060+0000 7fe7ddef2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d8109fc0 0x7fe7d819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:48.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.060+0000 7fe7ddef2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d8113d10 con 0x7fe7d8109b20 2024-04-25T10:40:48.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.060+0000 7fe7ddef2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d8113b90 con 0x7fe7d8109fc0 2024-04-25T10:40:48.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.060+0000 7fe7ddef2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d8113e90 con 0x7fe7d8104270 2024-04-25T10:40:48.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.061+0000 7fe7d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8109b20 0x7fe7d819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:48.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.061+0000 7fe7d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8109b20 0x7fe7d819ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42164/0 (socket says 172.21.15.43:42164) 2024-04-25T10:40:48.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.061+0000 7fe7d6ffd640 1 -- 172.21.15.43:0/3933610243 learned_addr learned my addr 172.21.15.43:0/3933610243 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:48.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.061+0000 7fe7d77fe640 1 --2- 172.21.15.43:0/3933610243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d8104270 0x7fe7d819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:48.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.061+0000 7fe7d7fff640 1 --2- 172.21.15.43:0/3933610243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d8109fc0 0x7fe7d819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:48.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.061+0000 7fe7d6ffd640 1 -- 172.21.15.43:0/3933610243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d8104270 msgr2=0x7fe7d819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:48.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.061+0000 7fe7d6ffd640 1 --2- 172.21.15.43:0/3933610243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d8104270 0x7fe7d819a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.061+0000 7fe7d6ffd640 1 -- 172.21.15.43:0/3933610243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d8109fc0 msgr2=0x7fe7d819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:48.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.061+0000 7fe7d6ffd640 1 --2- 172.21.15.43:0/3933610243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d8109fc0 0x7fe7d819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.061+0000 7fe7d6ffd640 1 -- 172.21.15.43:0/3933610243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d819f5b0 con 0x7fe7d8109b20 2024-04-25T10:40:48.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.061+0000 7fe7d6ffd640 1 --2- 172.21.15.43:0/3933610243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8109b20 0x7fe7d819ad30 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00d950 tx=0x7fe7cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:48.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.061+0000 7fe7d4ff9640 1 -- 172.21.15.43:0/3933610243 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc017070 con 0x7fe7d8109b20 2024-04-25T10:40:48.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.061+0000 7fe7d4ff9640 1 -- 172.21.15.43:0/3933610243 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7cc009c30 con 0x7fe7d8109b20 2024-04-25T10:40:48.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.061+0000 7fe7ddef2640 1 -- 172.21.15.43:0/3933610243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d819f8a0 con 0x7fe7d8109b20 2024-04-25T10:40:48.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.061+0000 7fe7d4ff9640 1 -- 172.21.15.43:0/3933610243 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc00f620 con 0x7fe7d8109b20 2024-04-25T10:40:48.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.061+0000 7fe7ddef2640 1 -- 172.21.15.43:0/3933610243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d819fda0 con 0x7fe7d8109b20 2024-04-25T10:40:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.064+0000 7fe7e4da6740 1 Processor -- start 2024-04-25T10:40:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.064+0000 7fe7e4da6740 1 -- start start 2024-04-25T10:40:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.064+0000 7fe7d4ff9640 1 -- 172.21.15.43:0/3933610243 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fe7cc01a070 con 0x7fe7d8109b20 2024-04-25T10:40:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.064+0000 7fe7e4da6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 0x563239ac48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.064+0000 7fe7e4da6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632396fb4d0 0x563239ac4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.064+0000 7fe7e4da6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632397036b0 0x563239ad5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.064+0000 7fe7e4da6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563239a2ad80 con 0x563239724390 2024-04-25T10:40:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.064+0000 7fe7e4da6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5632399e48b0 con 0x5632396fb4d0 2024-04-25T10:40:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.064+0000 7fe7e4da6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563239a26d40 con 0x5632397036b0 2024-04-25T10:40:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.065+0000 7fe7d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 0x563239ac48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.065+0000 7fe7d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 0x563239ac48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42178/0 (socket says 172.21.15.43:42178) 2024-04-25T10:40:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.065+0000 7fe7d7fff640 1 -- 172.21.15.43:0/3223017886 learned_addr learned my addr 172.21.15.43:0/3223017886 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.065+0000 7fe7d6ffd640 1 --2- 172.21.15.43:0/3223017886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632397036b0 0x563239ad5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.065+0000 7fe7d77fe640 1 --2- 172.21.15.43:0/3223017886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632396fb4d0 0x563239ac4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.065+0000 7fe7d7fff640 1 -- 172.21.15.43:0/3223017886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632397036b0 msgr2=0x563239ad5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.065+0000 7fe7d7fff640 1 --2- 172.21.15.43:0/3223017886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632397036b0 0x563239ad5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.065+0000 7fe7d7fff640 1 -- 172.21.15.43:0/3223017886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632396fb4d0 msgr2=0x563239ac4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.065+0000 7fe7d7fff640 1 --2- 172.21.15.43:0/3223017886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632396fb4d0 0x563239ac4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.065+0000 7fe7d7fff640 1 -- 172.21.15.43:0/3223017886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632399b5a40 con 0x563239724390 2024-04-25T10:40:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.065+0000 7fe7d4ff9640 1 --2- 172.21.15.43:0/3933610243 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe7c00afc60 0x7fe7c00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.065+0000 7fe7d7fff640 1 --2- 172.21.15.43:0/3223017886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 0x563239ac48f0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0007e50 tx=0x7fe7d000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5154a6a4d3bff4b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:48.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.065+0000 7fe7d77fe640 1 --2- 172.21.15.43:0/3933610243 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe7c00afc60 0x7fe7c00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:48.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.065+0000 7fe7d4ff9640 1 -- 172.21.15.43:0/3933610243 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fe7cc0dea40 con 0x7fe7d8109b20 2024-04-25T10:40:48.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.065+0000 7fe7bdffb640 1 -- 172.21.15.43:0/3223017886 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0014070 con 0x563239724390 2024-04-25T10:40:48.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.065+0000 7fe7bdffb640 1 -- 172.21.15.43:0/3223017886 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7d000ee90 con 0x563239724390 2024-04-25T10:40:48.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.065+0000 7fe7bdffb640 1 -- 172.21.15.43:0/3223017886 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0005380 con 0x563239724390 2024-04-25T10:40:48.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.066+0000 7fe7e4da6740 1 -- 172.21.15.43:0/3223017886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 msgr2=0x563239ac48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:48.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.066+0000 7fe7e4da6740 1 --2- 172.21.15.43:0/3223017886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 0x563239ac48f0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0007e50 tx=0x7fe7d000e720 comp rx=0 tx=0).stop 2024-04-25T10:40:48.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.066+0000 7fe7d77fe640 1 --2- 172.21.15.43:0/3933610243 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe7c00afc60 0x7fe7c00b2170 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8000c00 tx=0x7fe7c8031400 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:48.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.066+0000 7fe7e4da6740 1 -- 172.21.15.43:0/3223017886 shutdown_connections 2024-04-25T10:40:48.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.066+0000 7fe7e4da6740 1 --2- 172.21.15.43:0/3223017886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632396fb4d0 0x563239ac4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.066+0000 7fe7e4da6740 1 --2- 172.21.15.43:0/3223017886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632397036b0 0x563239ad5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.066+0000 7fe7e4da6740 1 --2- 172.21.15.43:0/3223017886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 0x563239ac48f0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.066+0000 7fe7e4da6740 1 -- 172.21.15.43:0/3223017886 >> 172.21.15.43:0/3223017886 conn(0x5632397184e0 msgr2=0x563239ac1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:48.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.066+0000 7fe7e4da6740 1 -- 172.21.15.43:0/3223017886 shutdown_connections 2024-04-25T10:40:48.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.066+0000 7fe7e4da6740 1 -- 172.21.15.43:0/3223017886 wait complete. 2024-04-25T10:40:48.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.067+0000 7fe7e4da6740 1 -- start start 2024-04-25T10:40:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.067+0000 7fe7e4da6740 1 Processor -- start 2024-04-25T10:40:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.067+0000 7fe7e4da6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632396fb4d0 0x563239ad3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.067+0000 7fe7e4da6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632397036b0 0x563239ad3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.067+0000 7fe7e4da6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 0x563239ad9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.067+0000 7fe7e4da6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563239ad63f0 con 0x563239724390 2024-04-25T10:40:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.067+0000 7fe7e4da6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5632399b1d10 con 0x5632396fb4d0 2024-04-25T10:40:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.067+0000 7fe7e4da6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563239ad6570 con 0x5632397036b0 2024-04-25T10:40:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.067+0000 7fe7e4da6740 10 client.0 _ll_register_callbacks cb 0x5632399de210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.068+0000 7fe7d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 0x563239ad9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.068+0000 7fe7d77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632396fb4d0 0x563239ad3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.068+0000 7fe7d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 0x563239ad9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42188/0 (socket says 172.21.15.43:42188) 2024-04-25T10:40:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.068+0000 7fe7d77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632396fb4d0 0x563239ad3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:39490/0 (socket says 172.21.15.43:39490) 2024-04-25T10:40:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.068+0000 7fe7d7fff640 1 -- 172.21.15.43:0/1412513686 learned_addr learned my addr 172.21.15.43:0/1412513686 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.068+0000 7fe7d6ffd640 1 --2- 172.21.15.43:0/1412513686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632397036b0 0x563239ad3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.068+0000 7fe7d7fff640 1 -- 172.21.15.43:0/1412513686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632397036b0 msgr2=0x563239ad3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.068+0000 7fe7d7fff640 1 --2- 172.21.15.43:0/1412513686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632397036b0 0x563239ad3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.068+0000 7fe7d7fff640 1 -- 172.21.15.43:0/1412513686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632396fb4d0 msgr2=0x563239ad3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.068+0000 7fe7d7fff640 1 --2- 172.21.15.43:0/1412513686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632396fb4d0 0x563239ad3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.068+0000 7fe7d7fff640 1 -- 172.21.15.43:0/1412513686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632398067d0 con 0x563239724390 2024-04-25T10:40:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.068+0000 7fe7d77fe640 1 --2- 172.21.15.43:0/1412513686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632396fb4d0 0x563239ad3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:40:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.068+0000 7fe7d7fff640 1 --2- 172.21.15.43:0/1412513686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 0x563239ad9900 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0007a00 tx=0x7fe7d0007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1072a075693ed4fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.068+0000 7fe7b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.068+0000 7fe7b6ffd640 1 -- 172.21.15.43:0/1412513686 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0027060 con 0x563239724390 2024-04-25T10:40:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.068+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1412513686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7d000ee90 con 0x563239724390 2024-04-25T10:40:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.068+0000 7fe7b6ffd640 1 -- 172.21.15.43:0/1412513686 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fe7d0010850 con 0x563239724390 2024-04-25T10:40:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.068+0000 7fe7b6ffd640 1 -- 172.21.15.43:0/1412513686 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe7d0014070 con 0x563239724390 2024-04-25T10:40:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.068+0000 7fe7e4da6740 10 client.17994 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.068+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1412513686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56323973c3f0 con 0x563239724390 2024-04-25T10:40:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.069+0000 7fe7b6ffd640 1 -- 172.21.15.43:0/1412513686 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7d000ee90 con 0x563239724390 2024-04-25T10:40:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.069+0000 7fe7b6ffd640 1 -- 172.21.15.43:0/1412513686 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe7d0018410 con 0x563239724390 2024-04-25T10:40:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.069+0000 7fe7e4da6740 10 client.17994 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.069+0000 7fe7e4da6740 10 client.17994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.069+0000 7fe7e4da6740 20 client.17994 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.069+0000 7fe7e4da6740 1 --2- 172.21.15.43:0/1412513686 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5632396e2f60 0x563239ae5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.069+0000 7fe7e4da6740 4 client.17994 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.069+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1412513686 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563239a38990 con 0x5632396e2f60 2024-04-25T10:40:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.069+0000 7fe7d77fe640 1 --2- 172.21.15.43:0/1412513686 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5632396e2f60 0x563239ae5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.070+0000 7fe7b6ffd640 10 client.17994 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.070+0000 7fe7d77fe640 1 --2- 172.21.15.43:0/1412513686 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5632396e2f60 0x563239ae5aa0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d66dcc982b94b55 server_cookie=8b9a735de3b0cdea in_seq=0 out_seq=0 2024-04-25T10:40:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.070+0000 7fe7b6ffd640 1 -- 172.21.15.43:0/1412513686 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563239a38990 con 0x5632396e2f60 2024-04-25T10:40:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.070+0000 7fe7b6ffd640 10 client.17994 handle_command_reply: tid=0 2024-04-25T10:40:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.071+0000 7fe7e4da6740 1 client.17994 shutdown 2024-04-25T10:40:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.071+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1412513686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 msgr2=0x563239ad9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.071+0000 7fe7e4da6740 1 --2- 172.21.15.43:0/1412513686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 0x563239ad9900 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0007a00 tx=0x7fe7d0007a30 comp rx=0 tx=0).stop 2024-04-25T10:40:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.071+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1412513686 shutdown_connections 2024-04-25T10:40:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.071+0000 7fe7e4da6740 1 --2- 172.21.15.43:0/1412513686 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5632396e2f60 0x563239ae5aa0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.071+0000 7fe7e4da6740 1 --2- 172.21.15.43:0/1412513686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 0x563239ad9900 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.071+0000 7fe7e4da6740 1 --2- 172.21.15.43:0/1412513686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632397036b0 0x563239ad3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.071+0000 7fe7e4da6740 1 --2- 172.21.15.43:0/1412513686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632396fb4d0 0x563239ad3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.071+0000 7fe7b6ffd640 0 client.17994 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.071+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1412513686 >> 172.21.15.43:0/1412513686 conn(0x5632397184e0 msgr2=0x563239ac66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.071+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1412513686 shutdown_connections 2024-04-25T10:40:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.071+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1412513686 wait complete. 2024-04-25T10:40:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.071+0000 7fe7e4da6740 20 client.17994 trim_cache size 0 max 16384 2024-04-25T10:40:48.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.089+0000 7fe7e4da6740 1 Processor -- start 2024-04-25T10:40:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.089+0000 7fe7e4da6740 1 -- start start 2024-04-25T10:40:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.089+0000 7fe7e4da6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632396fb4d0 0x563239acebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.089+0000 7fe7e4da6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632397036b0 0x563239ad5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.090+0000 7fe7e4da6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 0x563239ad9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.090+0000 7fe7e4da6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d81b00e0 con 0x563239724390 2024-04-25T10:40:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.090+0000 7fe7e4da6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d81ad9b0 con 0x5632397036b0 2024-04-25T10:40:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.090+0000 7fe7e4da6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d81ab820 con 0x5632396fb4d0 2024-04-25T10:40:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.090+0000 7fe7d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 0x563239ad9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.090+0000 7fe7d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 0x563239ad9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42204/0 (socket says 172.21.15.43:42204) 2024-04-25T10:40:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.090+0000 7fe7d7fff640 1 -- 172.21.15.43:0/1128903678 learned_addr learned my addr 172.21.15.43:0/1128903678 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.090+0000 7fe7d77fe640 1 --2- 172.21.15.43:0/1128903678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632396fb4d0 0x563239acebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.090+0000 7fe7d6ffd640 1 --2- 172.21.15.43:0/1128903678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632397036b0 0x563239ad5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.090+0000 7fe7d7fff640 1 -- 172.21.15.43:0/1128903678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632396fb4d0 msgr2=0x563239acebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:48.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.090+0000 7fe7d7fff640 1 --2- 172.21.15.43:0/1128903678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632396fb4d0 0x563239acebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.090+0000 7fe7d7fff640 1 -- 172.21.15.43:0/1128903678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632397036b0 msgr2=0x563239ad5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:48.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.090+0000 7fe7d7fff640 1 --2- 172.21.15.43:0/1128903678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632397036b0 0x563239ad5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.090+0000 7fe7d7fff640 1 -- 172.21.15.43:0/1128903678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632396fc0f0 con 0x563239724390 2024-04-25T10:40:48.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.090+0000 7fe7d7fff640 1 --2- 172.21.15.43:0/1128903678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 0x563239ad9870 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0000f80 tx=0x7fe7d00076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9217cd3a68e8a3cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:48.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.090+0000 7fe7bd7fa640 1 -- 172.21.15.43:0/1128903678 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0027070 con 0x563239724390 2024-04-25T10:40:48.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.090+0000 7fe7bd7fa640 1 -- 172.21.15.43:0/1128903678 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7d001c030 con 0x563239724390 2024-04-25T10:40:48.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.090+0000 7fe7bd7fa640 1 -- 172.21.15.43:0/1128903678 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0018570 con 0x563239724390 2024-04-25T10:40:48.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.091+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1128903678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 msgr2=0x563239ad9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:48.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.091+0000 7fe7e4da6740 1 --2- 172.21.15.43:0/1128903678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 0x563239ad9870 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0000f80 tx=0x7fe7d00076c0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.091+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1128903678 shutdown_connections 2024-04-25T10:40:48.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.091+0000 7fe7e4da6740 1 --2- 172.21.15.43:0/1128903678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632397036b0 0x563239ad5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.091+0000 7fe7e4da6740 1 --2- 172.21.15.43:0/1128903678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563239724390 0x563239ad9870 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.091+0000 7fe7e4da6740 1 --2- 172.21.15.43:0/1128903678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5632396fb4d0 0x563239acebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.091+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1128903678 >> 172.21.15.43:0/1128903678 conn(0x5632397184e0 msgr2=0x563239ac6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:48.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.091+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1128903678 shutdown_connections 2024-04-25T10:40:48.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.091+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1128903678 wait complete. 2024-04-25T10:40:48.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.091+0000 7fe7e4da6740 1 -- start start 2024-04-25T10:40:48.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.092+0000 7fe7e4da6740 1 Processor -- start 2024-04-25T10:40:48.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.092+0000 7fe7e4da6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632396fb4d0 0x563239ae0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:48.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.092+0000 7fe7e4da6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632397036b0 0x563239ade610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:48.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.092+0000 7fe7e4da6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563239724390 0x563239adeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:48.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.092+0000 7fe7e4da6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d81abe70 con 0x5632396fb4d0 2024-04-25T10:40:48.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.092+0000 7fe7e4da6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d81ad2e0 con 0x5632397036b0 2024-04-25T10:40:48.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.092+0000 7fe7e4da6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d81ac540 con 0x563239724390 2024-04-25T10:40:48.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.092+0000 7fe7e4da6740 10 client.0 _ll_register_callbacks cb 0x7fe7c8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:48.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.092+0000 7fe7d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632396fb4d0 0x563239ae0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:48.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.092+0000 7fe7d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632396fb4d0 0x563239ae0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42208/0 (socket says 172.21.15.43:42208) 2024-04-25T10:40:48.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.092+0000 7fe7d77fe640 1 -- 172.21.15.43:0/1784853988 learned_addr learned my addr 172.21.15.43:0/1784853988 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:48.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.092+0000 7fe7d6ffd640 1 --2- 172.21.15.43:0/1784853988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632397036b0 0x563239ade610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:48.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.092+0000 7fe7d7fff640 1 --2- 172.21.15.43:0/1784853988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563239724390 0x563239adeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:48.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.092+0000 7fe7d77fe640 1 -- 172.21.15.43:0/1784853988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563239724390 msgr2=0x563239adeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:48.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.092+0000 7fe7d77fe640 1 --2- 172.21.15.43:0/1784853988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563239724390 0x563239adeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.092+0000 7fe7d77fe640 1 -- 172.21.15.43:0/1784853988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632397036b0 msgr2=0x563239ade610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:48.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.092+0000 7fe7d77fe640 1 --2- 172.21.15.43:0/1784853988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632397036b0 0x563239ade610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.093+0000 7fe7d77fe640 1 -- 172.21.15.43:0/1784853988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632399eb4b0 con 0x5632396fb4d0 2024-04-25T10:40:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.093+0000 7fe7d77fe640 1 --2- 172.21.15.43:0/1784853988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632396fb4d0 0x563239ae0290 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009900 tx=0x7fe7c8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482724f4e33b5e6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.093+0000 7fe7b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.093+0000 7fe7b6ffd640 1 -- 172.21.15.43:0/1784853988 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8035040 con 0x5632396fb4d0 2024-04-25T10:40:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.093+0000 7fe7b6ffd640 1 -- 172.21.15.43:0/1784853988 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fe7c8005240 con 0x5632396fb4d0 2024-04-25T10:40:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.093+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1784853988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7d001c030 con 0x5632396fb4d0 2024-04-25T10:40:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.093+0000 7fe7b6ffd640 1 -- 172.21.15.43:0/1784853988 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe7c8060050 con 0x5632396fb4d0 2024-04-25T10:40:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.093+0000 7fe7e4da6740 10 client.18000 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.093+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1784853988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563239b4c8a0 con 0x5632396fb4d0 2024-04-25T10:40:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.093+0000 7fe7b6ffd640 1 -- 172.21.15.43:0/1784853988 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7d001c030 con 0x5632396fb4d0 2024-04-25T10:40:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.093+0000 7fe7b6ffd640 1 -- 172.21.15.43:0/1784853988 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe7c8051ce0 con 0x5632396fb4d0 2024-04-25T10:40:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.093+0000 7fe7e4da6740 10 client.18000 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.093+0000 7fe7e4da6740 10 client.18000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.093+0000 7fe7e4da6740 20 client.18000 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.094+0000 7fe7e4da6740 1 --2- 172.21.15.43:0/1784853988 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5632396b83a0 0x563239b4f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.094+0000 7fe7e4da6740 4 client.18000 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:40:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.094+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1784853988 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563239a32800 con 0x5632396b83a0 2024-04-25T10:40:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.094+0000 7fe7d6ffd640 1 --2- 172.21.15.43:0/1784853988 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5632396b83a0 0x563239b4f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.094+0000 7fe7b6ffd640 10 client.18000 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.094+0000 7fe7d6ffd640 1 --2- 172.21.15.43:0/1784853988 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5632396b83a0 0x563239b4f090 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=628d5abad24b56be server_cookie=f9ff983e0ecfdd6 in_seq=0 out_seq=0 2024-04-25T10:40:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.095+0000 7fe7b6ffd640 1 -- 172.21.15.43:0/1784853988 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563239a32800 con 0x5632396b83a0 2024-04-25T10:40:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.095+0000 7fe7b6ffd640 10 client.18000 handle_command_reply: tid=0 2024-04-25T10:40:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.095+0000 7fe7e4da6740 1 client.18000 shutdown 2024-04-25T10:40:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.095+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1784853988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632396fb4d0 msgr2=0x563239ae0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.095+0000 7fe7e4da6740 1 --2- 172.21.15.43:0/1784853988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632396fb4d0 0x563239ae0290 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009900 tx=0x7fe7c8009260 comp rx=0 tx=0).stop 2024-04-25T10:40:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.095+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1784853988 shutdown_connections 2024-04-25T10:40:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.095+0000 7fe7e4da6740 1 --2- 172.21.15.43:0/1784853988 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5632396b83a0 0x563239b4f090 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.095+0000 7fe7e4da6740 1 --2- 172.21.15.43:0/1784853988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632397036b0 0x563239ade610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.095+0000 7fe7e4da6740 1 --2- 172.21.15.43:0/1784853988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563239724390 0x563239adeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.095+0000 7fe7e4da6740 1 --2- 172.21.15.43:0/1784853988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632396fb4d0 0x563239ae0290 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.095+0000 7fe7b6ffd640 0 client.18000 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.095+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1784853988 >> 172.21.15.43:0/1784853988 conn(0x5632397184e0 msgr2=0x563239ac6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.095+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1784853988 shutdown_connections 2024-04-25T10:40:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.095+0000 7fe7e4da6740 1 -- 172.21.15.43:0/1784853988 wait complete. 2024-04-25T10:40:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.095+0000 7fe7e4da6740 20 client.18000 trim_cache size 0 max 16384 2024-04-25T10:40:48.103 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:40:48.103 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:40:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:40:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "c96ff49a-811c-4223-a4e5-afc3b774de51": { 2024-04-25T10:40:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:40:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "c96ff49a-811c-4223-a4e5-afc3b774de51", 2024-04-25T10:40:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:40:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:40:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:40:48.104 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:40:48.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.095+0000 7fe7ddef2640 1 -- 172.21.15.43:0/3933610243 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe7c00afc60 msgr2=0x7fe7c00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:48.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.095+0000 7fe7ddef2640 1 --2- 172.21.15.43:0/3933610243 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe7c00afc60 0x7fe7c00b2170 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8000c00 tx=0x7fe7c8031400 comp rx=0 tx=0).stop 2024-04-25T10:40:48.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.095+0000 7fe7ddef2640 1 -- 172.21.15.43:0/3933610243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8109b20 msgr2=0x7fe7d819ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:48.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.095+0000 7fe7ddef2640 1 --2- 172.21.15.43:0/3933610243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8109b20 0x7fe7d819ad30 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00d950 tx=0x7fe7cc00de10 comp rx=0 tx=0).stop 2024-04-25T10:40:48.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.096+0000 7fe7ddef2640 1 -- 172.21.15.43:0/3933610243 shutdown_connections 2024-04-25T10:40:48.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.096+0000 7fe7ddef2640 1 --2- 172.21.15.43:0/3933610243 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe7c00afc60 0x7fe7c00b2170 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.096+0000 7fe7ddef2640 1 --2- 172.21.15.43:0/3933610243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d8109fc0 0x7fe7d819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.096+0000 7fe7ddef2640 1 --2- 172.21.15.43:0/3933610243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8109b20 0x7fe7d819ad30 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.096+0000 7fe7ddef2640 1 --2- 172.21.15.43:0/3933610243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d8104270 0x7fe7d819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:48.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.096+0000 7fe7ddef2640 1 -- 172.21.15.43:0/3933610243 >> 172.21.15.43:0/3933610243 conn(0x7fe7d80ff1e0 msgr2=0x7fe7d80ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:48.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.096+0000 7fe7ddef2640 1 -- 172.21.15.43:0/3933610243 shutdown_connections 2024-04-25T10:40:48.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:40:48.096+0000 7fe7ddef2640 1 -- 172.21.15.43:0/3933610243 wait complete. 2024-04-25T10:40:48.107 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:47 smithi043 ceph-mon[29478]: pgmap v255: 97 pgs: 97 active+clean; 2.3 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:40:48.108 INFO:tasks.cephfs.filesystem:scrub status for tag:c96ff49a-811c-4223-a4e5-afc3b774de51 - {'path': '/', 'tag': 'c96ff49a-811c-4223-a4e5-afc3b774de51', 'options': 'recursive,force'} 2024-04-25T10:40:48.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:48 smithi077 ceph-mon[32772]: pgmap v255: 97 pgs: 97 active+clean; 2.3 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:40:48.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:47 smithi164 ceph-mon[32713]: pgmap v255: 97 pgs: 97 active+clean; 2.3 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:40:49.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:49 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:40:49.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:49 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:40:49.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:49 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:40:49.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:49 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:40:49.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:49 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:40:49.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:49 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:40:49.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:49 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:40:49.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:49 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:40:49.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:49 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:40:49.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:49 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:40:49.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:49 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:40:49.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:49 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:40:49.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:49 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:40:49.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:49 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:40:49.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:49 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:40:49.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:49 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:40:49.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:49 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:40:49.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:49 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:40:49.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:49 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:40:49.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:49 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:40:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:49 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:40:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:49 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:40:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:49 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:40:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:49 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:40:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:49 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:40:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:49 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:40:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:49 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:40:50.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:50 smithi077 ceph-mon[32772]: pgmap v256: 97 pgs: 97 active+clean; 2.4 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-04-25T10:40:50.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:50 smithi164 ceph-mon[32713]: pgmap v256: 97 pgs: 97 active+clean; 2.4 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-04-25T10:40:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:50 smithi043 ceph-mon[29478]: pgmap v256: 97 pgs: 97 active+clean; 2.4 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-04-25T10:40:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.662+0000 7f69d1684640 1 -- 172.21.15.43:0/2781379641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69cc1020b0 msgr2=0x7f69cc1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.662+0000 7f69d1684640 1 --2- 172.21.15.43:0/2781379641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69cc1020b0 0x7f69cc1024c0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f69c0009900 tx=0x7f69c002a410 comp rx=0 tx=0).stop 2024-04-25T10:40:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.662+0000 7f69d1684640 1 -- 172.21.15.43:0/2781379641 shutdown_connections 2024-04-25T10:40:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.662+0000 7f69d1684640 1 --2- 172.21.15.43:0/2781379641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69cc10e5b0 0x7f69cc1109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.662+0000 7f69d1684640 1 --2- 172.21.15.43:0/2781379641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69cc1020b0 0x7f69cc1024c0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.662+0000 7f69d1684640 1 --2- 172.21.15.43:0/2781379641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc109c00 0x7f69cc101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.662+0000 7f69d1684640 1 -- 172.21.15.43:0/2781379641 >> 172.21.15.43:0/2781379641 conn(0x7f69cc0fd8a0 msgr2=0x7f69cc0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.662+0000 7f69d1684640 1 -- 172.21.15.43:0/2781379641 shutdown_connections 2024-04-25T10:40:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.662+0000 7f69d1684640 1 -- 172.21.15.43:0/2781379641 wait complete. 2024-04-25T10:40:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.663+0000 7f69d1684640 1 Processor -- start 2024-04-25T10:40:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.663+0000 7f69d1684640 1 -- start start 2024-04-25T10:40:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.664+0000 7f69d1684640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc1020b0 0x7f69cc1a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.664+0000 7f69d1684640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69cc109c00 0x7f69cc1a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.664+0000 7f69d1684640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69cc10e5b0 0x7f69cc19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.664+0000 7f69d1684640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc113d70 con 0x7f69cc109c00 2024-04-25T10:40:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.664+0000 7f69d1684640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc113bf0 con 0x7f69cc10e5b0 2024-04-25T10:40:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.664+0000 7f69d1684640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc113ef0 con 0x7f69cc1020b0 2024-04-25T10:40:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.664+0000 7f69ca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69cc109c00 0x7f69cc1a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.664+0000 7f69ca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69cc109c00 0x7f69cc1a0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41208/0 (socket says 172.21.15.43:41208) 2024-04-25T10:40:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.664+0000 7f69ca7fc640 1 -- 172.21.15.43:0/3211075601 learned_addr learned my addr 172.21.15.43:0/3211075601 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.664+0000 7f69cb7fe640 1 --2- 172.21.15.43:0/3211075601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69cc10e5b0 0x7f69cc19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.664+0000 7f69caffd640 1 --2- 172.21.15.43:0/3211075601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc1020b0 0x7f69cc1a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.664+0000 7f69ca7fc640 1 -- 172.21.15.43:0/3211075601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc1020b0 msgr2=0x7f69cc1a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.664+0000 7f69ca7fc640 1 --2- 172.21.15.43:0/3211075601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc1020b0 0x7f69cc1a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.664+0000 7f69ca7fc640 1 -- 172.21.15.43:0/3211075601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69cc10e5b0 msgr2=0x7f69cc19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.664+0000 7f69ca7fc640 1 --2- 172.21.15.43:0/3211075601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69cc10e5b0 0x7f69cc19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.664+0000 7f69ca7fc640 1 -- 172.21.15.43:0/3211075601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69cc19adc0 con 0x7f69cc109c00 2024-04-25T10:40:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.664+0000 7f69ca7fc640 1 --2- 172.21.15.43:0/3211075601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69cc109c00 0x7f69cc1a0a00 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f69c0032fd0 tx=0x7f69c0031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:50.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.664+0000 7f69b3fff640 1 -- 172.21.15.43:0/3211075601 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c003f030 con 0x7f69cc109c00 2024-04-25T10:40:50.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.665+0000 7f69b3fff640 1 -- 172.21.15.43:0/3211075601 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69c0004490 con 0x7f69cc109c00 2024-04-25T10:40:50.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.665+0000 7f69b3fff640 1 -- 172.21.15.43:0/3211075601 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0035070 con 0x7f69cc109c00 2024-04-25T10:40:50.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.665+0000 7f69d1684640 1 -- 172.21.15.43:0/3211075601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69cc19b050 con 0x7f69cc109c00 2024-04-25T10:40:50.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.665+0000 7f69d1684640 1 -- 172.21.15.43:0/3211075601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69cc1a72b0 con 0x7f69cc109c00 2024-04-25T10:40:50.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.667+0000 7f69b3fff640 1 -- 172.21.15.43:0/3211075601 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f69c0042030 con 0x7f69cc109c00 2024-04-25T10:40:50.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.667+0000 7f69d860b740 1 Processor -- start 2024-04-25T10:40:50.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.668+0000 7f69d860b740 1 -- start start 2024-04-25T10:40:50.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.668+0000 7f69d860b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484da91b0 0x55a485148af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:50.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.668+0000 7f69d860b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a484d802f0 0x55a485149030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.668+0000 7f69d860b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d884d0 0x55a485149570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.668+0000 7f69d860b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a4850af460 con 0x55a484da91b0 2024-04-25T10:40:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.668+0000 7f69d860b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a485069590 con 0x55a484d802f0 2024-04-25T10:40:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.668+0000 7f69d860b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a4850ab420 con 0x55a484d884d0 2024-04-25T10:40:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.668+0000 7f69cb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484da91b0 0x55a485148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.668+0000 7f69cb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484da91b0 0x55a485148af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41222/0 (socket says 172.21.15.43:41222) 2024-04-25T10:40:50.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.668+0000 7f69cb7fe640 1 -- 172.21.15.43:0/1563432388 learned_addr learned my addr 172.21.15.43:0/1563432388 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:50.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.668+0000 7f69ca7fc640 1 --2- 172.21.15.43:0/1563432388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d884d0 0x55a485149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:50.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.668+0000 7f69caffd640 1 --2- 172.21.15.43:0/1563432388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a484d802f0 0x55a485149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:50.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.668+0000 7f69b3fff640 1 --2- 172.21.15.43:0/3211075601 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f69b40afcd0 0x7f69b40b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:50.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.668+0000 7f69cb7fe640 1 -- 172.21.15.43:0/1563432388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d884d0 msgr2=0x55a485149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.668+0000 7f69cb7fe640 1 --2- 172.21.15.43:0/1563432388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d884d0 0x55a485149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.668+0000 7f69cb7fe640 1 -- 172.21.15.43:0/1563432388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a484d802f0 msgr2=0x55a485149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.668+0000 7f69cb7fe640 1 --2- 172.21.15.43:0/1563432388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a484d802f0 0x55a485149030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.668+0000 7f69cb7fe640 1 -- 172.21.15.43:0/1563432388 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4850760d0 con 0x55a484da91b0 2024-04-25T10:40:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.669+0000 7f69b3fff640 1 -- 172.21.15.43:0/3211075601 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f69c00ff2c0 con 0x7f69cc109c00 2024-04-25T10:40:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.669+0000 7f69cb7fe640 1 --2- 172.21.15.43:0/3211075601 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f69b40afcd0 0x7f69b40b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.669+0000 7f69cb7fe640 1 --2- 172.21.15.43:0/1563432388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484da91b0 0x55a485148af0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f69c40114c0 tx=0x7f69c40118c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80adf92738cb49cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:50.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.669+0000 7f69b17fa640 1 -- 172.21.15.43:0/1563432388 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4017070 con 0x55a484da91b0 2024-04-25T10:40:50.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.669+0000 7f69b17fa640 1 -- 172.21.15.43:0/1563432388 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69c400fb90 con 0x55a484da91b0 2024-04-25T10:40:50.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.669+0000 7f69b17fa640 1 -- 172.21.15.43:0/1563432388 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c400e8b0 con 0x55a484da91b0 2024-04-25T10:40:50.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.670+0000 7f69d860b740 1 -- 172.21.15.43:0/1563432388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484da91b0 msgr2=0x55a485148af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:50.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.670+0000 7f69d860b740 1 --2- 172.21.15.43:0/1563432388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484da91b0 0x55a485148af0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f69c40114c0 tx=0x7f69c40118c0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.670+0000 7f69cb7fe640 1 --2- 172.21.15.43:0/3211075601 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f69b40afcd0 0x7f69b40b21e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f69c4022000 tx=0x7f69c4023000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.670+0000 7f69d860b740 1 -- 172.21.15.43:0/1563432388 shutdown_connections 2024-04-25T10:40:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.670+0000 7f69d860b740 1 --2- 172.21.15.43:0/1563432388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a484d802f0 0x55a485149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.670+0000 7f69d860b740 1 --2- 172.21.15.43:0/1563432388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d884d0 0x55a485149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.670+0000 7f69d860b740 1 --2- 172.21.15.43:0/1563432388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484da91b0 0x55a485148af0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.670+0000 7f69d860b740 1 -- 172.21.15.43:0/1563432388 >> 172.21.15.43:0/1563432388 conn(0x55a484d9d300 msgr2=0x55a4851452a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.670+0000 7f69d860b740 1 -- 172.21.15.43:0/1563432388 shutdown_connections 2024-04-25T10:40:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.670+0000 7f69d860b740 1 -- 172.21.15.43:0/1563432388 wait complete. 2024-04-25T10:40:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.670+0000 7f69d860b740 1 -- start start 2024-04-25T10:40:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.671+0000 7f69d860b740 1 Processor -- start 2024-04-25T10:40:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.671+0000 7f69d860b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d802f0 0x55a485153140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.671+0000 7f69d860b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d884d0 0x55a485153680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.671+0000 7f69d860b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a484da91b0 0x55a485159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.671+0000 7f69d860b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a485155c20 con 0x55a484d802f0 2024-04-25T10:40:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.671+0000 7f69d860b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a485036b60 con 0x55a484da91b0 2024-04-25T10:40:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.671+0000 7f69d860b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a485155da0 con 0x55a484d884d0 2024-04-25T10:40:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.671+0000 7f69d860b740 10 client.0 _ll_register_callbacks cb 0x55a48505b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.671+0000 7f69caffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d802f0 0x55a485153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.671+0000 7f69caffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d802f0 0x55a485153140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41226/0 (socket says 172.21.15.43:41226) 2024-04-25T10:40:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.671+0000 7f69caffd640 1 -- 172.21.15.43:0/4146383705 learned_addr learned my addr 172.21.15.43:0/4146383705 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.671+0000 7f69ca7fc640 1 --2- 172.21.15.43:0/4146383705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d884d0 0x55a485153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.671+0000 7f69cb7fe640 1 --2- 172.21.15.43:0/4146383705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a484da91b0 0x55a485159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.671+0000 7f69caffd640 1 -- 172.21.15.43:0/4146383705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d884d0 msgr2=0x55a485153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.671+0000 7f69caffd640 1 --2- 172.21.15.43:0/4146383705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d884d0 0x55a485153680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.671+0000 7f69caffd640 1 -- 172.21.15.43:0/4146383705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a484da91b0 msgr2=0x55a485159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.671+0000 7f69caffd640 1 --2- 172.21.15.43:0/4146383705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a484da91b0 0x55a485159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.671+0000 7f69caffd640 1 -- 172.21.15.43:0/4146383705 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a484dc1210 con 0x55a484d802f0 2024-04-25T10:40:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.671+0000 7f69ca7fc640 1 --2- 172.21.15.43:0/4146383705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d884d0 0x55a485153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:40:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.672+0000 7f69caffd640 1 --2- 172.21.15.43:0/4146383705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d802f0 0x55a485153140 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0126d0 tx=0x7f69bc012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e522f5361f9940c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.672+0000 7f69aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.672+0000 7f69aeff5640 1 -- 172.21.15.43:0/4146383705 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc004030 con 0x55a484d802f0 2024-04-25T10:40:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.672+0000 7f69aeff5640 1 -- 172.21.15.43:0/4146383705 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f69bc0041d0 con 0x55a484d802f0 2024-04-25T10:40:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.672+0000 7f69d860b740 1 -- 172.21.15.43:0/4146383705 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69c400fb90 con 0x55a484d802f0 2024-04-25T10:40:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.672+0000 7f69aeff5640 1 -- 172.21.15.43:0/4146383705 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f69bc0043d0 con 0x55a484d802f0 2024-04-25T10:40:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.672+0000 7f69d860b740 10 client.18012 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.672+0000 7f69d860b740 1 -- 172.21.15.43:0/4146383705 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a484e8d620 con 0x55a484d802f0 2024-04-25T10:40:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.672+0000 7f69aeff5640 1 -- 172.21.15.43:0/4146383705 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69c400fb90 con 0x55a484d802f0 2024-04-25T10:40:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.672+0000 7f69aeff5640 1 -- 172.21.15.43:0/4146383705 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f69bc003ab0 con 0x55a484d802f0 2024-04-25T10:40:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.672+0000 7f69d860b740 10 client.18012 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.672+0000 7f69d860b740 10 client.18012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.672+0000 7f69d860b740 20 client.18012 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.672+0000 7f69d860b740 1 --2- 172.21.15.43:0/4146383705 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a484d67d80 0x55a4851657e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.672+0000 7f69d860b740 4 client.18012 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:40:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.672+0000 7f69d860b740 1 -- 172.21.15.43:0/4146383705 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4850bd070 con 0x55a484d67d80 2024-04-25T10:40:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.673+0000 7f69ca7fc640 1 --2- 172.21.15.43:0/4146383705 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a484d67d80 0x55a4851657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.673+0000 7f69ca7fc640 1 --2- 172.21.15.43:0/4146383705 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a484d67d80 0x55a4851657e0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b4c7db508726029 server_cookie=53c56e413e7ebff6 in_seq=0 out_seq=0 2024-04-25T10:40:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.673+0000 7f69aeff5640 10 client.18012 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.674+0000 7f69aeff5640 1 -- 172.21.15.43:0/4146383705 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a4850bd070 con 0x55a484d67d80 2024-04-25T10:40:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.674+0000 7f69aeff5640 10 client.18012 handle_command_reply: tid=0 2024-04-25T10:40:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.674+0000 7f69d860b740 1 client.18012 shutdown 2024-04-25T10:40:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.674+0000 7f69d860b740 1 -- 172.21.15.43:0/4146383705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d802f0 msgr2=0x55a485153140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.674+0000 7f69d860b740 1 --2- 172.21.15.43:0/4146383705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d802f0 0x55a485153140 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0126d0 tx=0x7f69bc012b90 comp rx=0 tx=0).stop 2024-04-25T10:40:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.674+0000 7f69d860b740 1 -- 172.21.15.43:0/4146383705 shutdown_connections 2024-04-25T10:40:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.674+0000 7f69d860b740 1 --2- 172.21.15.43:0/4146383705 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a484d67d80 0x55a4851657e0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.674+0000 7f69d860b740 1 --2- 172.21.15.43:0/4146383705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a484da91b0 0x55a485159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.674+0000 7f69aeff5640 0 client.18012 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.674+0000 7f69d860b740 1 --2- 172.21.15.43:0/4146383705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d884d0 0x55a485153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.674+0000 7f69d860b740 1 --2- 172.21.15.43:0/4146383705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d802f0 0x55a485153140 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.674+0000 7f69d860b740 1 -- 172.21.15.43:0/4146383705 >> 172.21.15.43:0/4146383705 conn(0x55a484d9d300 msgr2=0x55a485148920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.674+0000 7f69d860b740 1 -- 172.21.15.43:0/4146383705 shutdown_connections 2024-04-25T10:40:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.674+0000 7f69d860b740 1 -- 172.21.15.43:0/4146383705 wait complete. 2024-04-25T10:40:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.674+0000 7f69d860b740 20 client.18012 trim_cache size 0 max 16384 2024-04-25T10:40:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.689+0000 7f69d860b740 1 Processor -- start 2024-04-25T10:40:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.690+0000 7f69d860b740 1 -- start start 2024-04-25T10:40:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.690+0000 7f69d860b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a484d67d80 0x55a485151e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.690+0000 7f69d860b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d802f0 0x55a485159070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.690+0000 7f69d860b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d884d0 0x55a485148f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.690+0000 7f69d860b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1b2310 con 0x55a484d884d0 2024-04-25T10:40:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.690+0000 7f69d860b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1afbe0 con 0x55a484d67d80 2024-04-25T10:40:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.690+0000 7f69d860b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1ada50 con 0x55a484d802f0 2024-04-25T10:40:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.690+0000 7f69cb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d884d0 0x55a485148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.690+0000 7f69cb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d884d0 0x55a485148f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41240/0 (socket says 172.21.15.43:41240) 2024-04-25T10:40:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.690+0000 7f69cb7fe640 1 -- 172.21.15.43:0/582130098 learned_addr learned my addr 172.21.15.43:0/582130098 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.690+0000 7f69caffd640 1 --2- 172.21.15.43:0/582130098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a484d67d80 0x55a485151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.690+0000 7f69ca7fc640 1 --2- 172.21.15.43:0/582130098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d802f0 0x55a485159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.690+0000 7f69cb7fe640 1 -- 172.21.15.43:0/582130098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d802f0 msgr2=0x55a485159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.690+0000 7f69cb7fe640 1 --2- 172.21.15.43:0/582130098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d802f0 0x55a485159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.690+0000 7f69cb7fe640 1 -- 172.21.15.43:0/582130098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a484d67d80 msgr2=0x55a485151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.690+0000 7f69cb7fe640 1 --2- 172.21.15.43:0/582130098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a484d67d80 0x55a485151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.690+0000 7f69cb7fe640 1 -- 172.21.15.43:0/582130098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a485148480 con 0x55a484d884d0 2024-04-25T10:40:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.690+0000 7f69ca7fc640 1 --2- 172.21.15.43:0/582130098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d802f0 0x55a485159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:40:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.691+0000 7f69cb7fe640 1 --2- 172.21.15.43:0/582130098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d884d0 0x55a485148f90 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009d50 tx=0x7f69c401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5393ca8cc960937 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.691+0000 7f69b0ff9640 1 -- 172.21.15.43:0/582130098 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4036070 con 0x55a484d884d0 2024-04-25T10:40:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.691+0000 7f69b0ff9640 1 -- 172.21.15.43:0/582130098 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69c400e860 con 0x55a484d884d0 2024-04-25T10:40:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.691+0000 7f69b0ff9640 1 -- 172.21.15.43:0/582130098 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4017040 con 0x55a484d884d0 2024-04-25T10:40:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.691+0000 7f69d860b740 1 -- 172.21.15.43:0/582130098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d884d0 msgr2=0x55a485148f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.691+0000 7f69d860b740 1 --2- 172.21.15.43:0/582130098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d884d0 0x55a485148f90 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009d50 tx=0x7f69c401d000 comp rx=0 tx=0).stop 2024-04-25T10:40:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.692+0000 7f69d860b740 1 -- 172.21.15.43:0/582130098 shutdown_connections 2024-04-25T10:40:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.692+0000 7f69d860b740 1 --2- 172.21.15.43:0/582130098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d884d0 0x55a485148f90 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.692+0000 7f69d860b740 1 --2- 172.21.15.43:0/582130098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d802f0 0x55a485159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.692+0000 7f69d860b740 1 --2- 172.21.15.43:0/582130098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a484d67d80 0x55a485151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.692+0000 7f69d860b740 1 -- 172.21.15.43:0/582130098 >> 172.21.15.43:0/582130098 conn(0x55a484d9d300 msgr2=0x55a485148880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.692+0000 7f69d860b740 1 -- 172.21.15.43:0/582130098 shutdown_connections 2024-04-25T10:40:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.692+0000 7f69d860b740 1 -- 172.21.15.43:0/582130098 wait complete. 2024-04-25T10:40:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.692+0000 7f69d860b740 1 -- start start 2024-04-25T10:40:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.692+0000 7f69d860b740 1 Processor -- start 2024-04-25T10:40:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.692+0000 7f69d860b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a484d67d80 0x55a4851c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.692+0000 7f69d860b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d802f0 0x55a4851c6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.692+0000 7f69d860b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d884d0 0x55a48515c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.692+0000 7f69d860b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1ae0a0 con 0x55a484d802f0 2024-04-25T10:40:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.692+0000 7f69d860b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1af510 con 0x55a484d67d80 2024-04-25T10:40:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.692+0000 7f69d860b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1ae770 con 0x55a484d884d0 2024-04-25T10:40:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.692+0000 7f69d860b740 10 client.0 _ll_register_callbacks cb 0x7f69bc009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:40:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.692+0000 7f69ca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d802f0 0x55a4851c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.692+0000 7f69ca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d802f0 0x55a4851c6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41248/0 (socket says 172.21.15.43:41248) 2024-04-25T10:40:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.692+0000 7f69ca7fc640 1 -- 172.21.15.43:0/3752739759 learned_addr learned my addr 172.21.15.43:0/3752739759 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:40:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.693+0000 7f69caffd640 1 --2- 172.21.15.43:0/3752739759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a484d67d80 0x55a4851c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.693+0000 7f69cb7fe640 1 --2- 172.21.15.43:0/3752739759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d884d0 0x55a48515c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.693+0000 7f69ca7fc640 1 -- 172.21.15.43:0/3752739759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d884d0 msgr2=0x55a48515c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.693+0000 7f69ca7fc640 1 --2- 172.21.15.43:0/3752739759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d884d0 0x55a48515c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.693+0000 7f69ca7fc640 1 -- 172.21.15.43:0/3752739759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a484d67d80 msgr2=0x55a4851c5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:40:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.693+0000 7f69ca7fc640 1 --2- 172.21.15.43:0/3752739759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a484d67d80 0x55a4851c5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.693+0000 7f69ca7fc640 1 -- 172.21.15.43:0/3752739759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a48515cfb0 con 0x55a484d802f0 2024-04-25T10:40:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.693+0000 7f69ca7fc640 1 --2- 172.21.15.43:0/3752739759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d802f0 0x55a4851c6520 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f69c00fcac0 tx=0x7f69c002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c800ff52536ba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:40:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.693+0000 7f69aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:40:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.693+0000 7f69aeff5640 1 -- 172.21.15.43:0/3752739759 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0049040 con 0x55a484d802f0 2024-04-25T10:40:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.693+0000 7f69aeff5640 1 -- 172.21.15.43:0/3752739759 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f69c00c37a0 con 0x55a484d802f0 2024-04-25T10:40:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.693+0000 7f69d860b740 1 -- 172.21.15.43:0/3752739759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69c400e860 con 0x55a484d802f0 2024-04-25T10:40:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.693+0000 7f69aeff5640 1 -- 172.21.15.43:0/3752739759 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f69c00c7020 con 0x55a484d802f0 2024-04-25T10:40:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.693+0000 7f69d860b740 10 client.18018 fetch_fsmap learned FSMap version 26 2024-04-25T10:40:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.693+0000 7f69d860b740 1 -- 172.21.15.43:0/3752739759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a4851cd600 con 0x55a484d802f0 2024-04-25T10:40:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.694+0000 7f69aeff5640 1 -- 172.21.15.43:0/3752739759 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69c400e860 con 0x55a484d802f0 2024-04-25T10:40:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.694+0000 7f69aeff5640 1 -- 172.21.15.43:0/3752739759 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f69c0102be0 con 0x55a484d802f0 2024-04-25T10:40:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.694+0000 7f69d860b740 10 client.18018 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:40:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.694+0000 7f69d860b740 10 client.18018 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:40:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.694+0000 7f69d860b740 20 client.18018 populate_metadata read hostname 'smithi043' 2024-04-25T10:40:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.694+0000 7f69d860b740 1 --2- 172.21.15.43:0/3752739759 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a484da91b0 0x55a4851cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:40:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.694+0000 7f69d860b740 4 client.18018 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:40:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.694+0000 7f69d860b740 1 -- 172.21.15.43:0/3752739759 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a4850b6ee0 con 0x55a484da91b0 2024-04-25T10:40:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.694+0000 7f69caffd640 1 --2- 172.21.15.43:0/3752739759 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a484da91b0 0x55a4851cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:40:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.694+0000 7f69aeff5640 10 client.18018 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.694+0000 7f69caffd640 1 --2- 172.21.15.43:0/3752739759 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a484da91b0 0x55a4851cfe10 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc615b34c5d34abe server_cookie=9fc994d25770327 in_seq=0 out_seq=0 2024-04-25T10:40:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.710+0000 7f69aeff5640 1 -- 172.21.15.43:0/3752739759 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a4850b6ee0 con 0x55a484da91b0 2024-04-25T10:40:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.710+0000 7f69aeff5640 10 client.18018 handle_command_reply: tid=0 2024-04-25T10:40:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.710+0000 7f69d860b740 1 client.18018 shutdown 2024-04-25T10:40:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.710+0000 7f69d860b740 1 -- 172.21.15.43:0/3752739759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d802f0 msgr2=0x55a4851c6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.710+0000 7f69d860b740 1 --2- 172.21.15.43:0/3752739759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d802f0 0x55a4851c6520 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f69c00fcac0 tx=0x7f69c002a410 comp rx=0 tx=0).stop 2024-04-25T10:40:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.710+0000 7f69d860b740 1 -- 172.21.15.43:0/3752739759 shutdown_connections 2024-04-25T10:40:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.710+0000 7f69d860b740 1 --2- 172.21.15.43:0/3752739759 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a484da91b0 0x55a4851cfe10 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.710+0000 7f69d860b740 1 --2- 172.21.15.43:0/3752739759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a484d884d0 0x55a48515c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.710+0000 7f69d860b740 1 --2- 172.21.15.43:0/3752739759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a484d802f0 0x55a4851c6520 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.710+0000 7f69aeff5640 0 client.18018 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:40:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.710+0000 7f69d860b740 1 --2- 172.21.15.43:0/3752739759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a484d67d80 0x55a4851c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.710+0000 7f69d860b740 1 -- 172.21.15.43:0/3752739759 >> 172.21.15.43:0/3752739759 conn(0x55a484d9d300 msgr2=0x55a4851545b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.710+0000 7f69d860b740 1 -- 172.21.15.43:0/3752739759 shutdown_connections 2024-04-25T10:40:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.710+0000 7f69d860b740 1 -- 172.21.15.43:0/3752739759 wait complete. 2024-04-25T10:40:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.710+0000 7f69d860b740 20 client.18018 trim_cache size 0 max 16384 2024-04-25T10:40:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:40:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:40:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:40:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:40:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.711+0000 7f69d1684640 1 -- 172.21.15.43:0/3211075601 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f69b40afcd0 msgr2=0x7f69b40b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.711+0000 7f69d1684640 1 --2- 172.21.15.43:0/3211075601 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f69b40afcd0 0x7f69b40b21e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f69c4022000 tx=0x7f69c4023000 comp rx=0 tx=0).stop 2024-04-25T10:40:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.711+0000 7f69d1684640 1 -- 172.21.15.43:0/3211075601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69cc109c00 msgr2=0x7f69cc1a0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:40:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.711+0000 7f69d1684640 1 --2- 172.21.15.43:0/3211075601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69cc109c00 0x7f69cc1a0a00 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f69c0032fd0 tx=0x7f69c0031a20 comp rx=0 tx=0).stop 2024-04-25T10:40:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.711+0000 7f69d1684640 1 -- 172.21.15.43:0/3211075601 shutdown_connections 2024-04-25T10:40:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.711+0000 7f69d1684640 1 --2- 172.21.15.43:0/3211075601 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f69b40afcd0 0x7f69b40b21e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.711+0000 7f69d1684640 1 --2- 172.21.15.43:0/3211075601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69cc10e5b0 0x7f69cc19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.711+0000 7f69d1684640 1 --2- 172.21.15.43:0/3211075601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69cc109c00 0x7f69cc1a0a00 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.711+0000 7f69d1684640 1 --2- 172.21.15.43:0/3211075601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc1020b0 0x7f69cc1a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:40:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.711+0000 7f69d1684640 1 -- 172.21.15.43:0/3211075601 >> 172.21.15.43:0/3211075601 conn(0x7f69cc0fd8a0 msgr2=0x7f69cc0fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:40:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.711+0000 7f69d1684640 1 -- 172.21.15.43:0/3211075601 shutdown_connections 2024-04-25T10:40:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:40:50.711+0000 7f69d1684640 1 -- 172.21.15.43:0/3211075601 wait complete. 2024-04-25T10:40:52.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:52 smithi077 ceph-mon[32772]: pgmap v257: 97 pgs: 97 active+clean; 2.4 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-04-25T10:40:52.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:52 smithi164 ceph-mon[32713]: pgmap v257: 97 pgs: 97 active+clean; 2.4 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-04-25T10:40:52.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:52 smithi043 ceph-mon[29478]: pgmap v257: 97 pgs: 97 active+clean; 2.4 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-04-25T10:40:54.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:54 smithi077 ceph-mon[32772]: pgmap v258: 97 pgs: 97 active+clean; 2.4 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-04-25T10:40:54.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:54 smithi164 ceph-mon[32713]: pgmap v258: 97 pgs: 97 active+clean; 2.4 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-04-25T10:40:54.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:54 smithi043 ceph-mon[29478]: pgmap v258: 97 pgs: 97 active+clean; 2.4 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-04-25T10:40:56.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:56 smithi077 ceph-mon[32772]: pgmap v259: 97 pgs: 97 active+clean; 2.7 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 60 KiB/s wr, 18 op/s 2024-04-25T10:40:56.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:56 smithi164 ceph-mon[32713]: pgmap v259: 97 pgs: 97 active+clean; 2.7 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 60 KiB/s wr, 18 op/s 2024-04-25T10:40:56.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:56 smithi043 ceph-mon[29478]: pgmap v259: 97 pgs: 97 active+clean; 2.7 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 60 KiB/s wr, 18 op/s 2024-04-25T10:40:58.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:40:58 smithi077 ceph-mon[32772]: pgmap v260: 97 pgs: 97 active+clean; 2.7 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-25T10:40:58.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:40:58 smithi164 ceph-mon[32713]: pgmap v260: 97 pgs: 97 active+clean; 2.7 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-25T10:40:58.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:40:58 smithi043 ceph-mon[29478]: pgmap v260: 97 pgs: 97 active+clean; 2.7 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-04-25T10:41:00.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:00 smithi077 ceph-mon[32772]: pgmap v261: 97 pgs: 97 active+clean; 2.9 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 60 KiB/s wr, 18 op/s 2024-04-25T10:41:00.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:00 smithi164 ceph-mon[32713]: pgmap v261: 97 pgs: 97 active+clean; 2.9 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 60 KiB/s wr, 18 op/s 2024-04-25T10:41:00.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:00 smithi043 ceph-mon[29478]: pgmap v261: 97 pgs: 97 active+clean; 2.9 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 60 KiB/s wr, 18 op/s 2024-04-25T10:41:02.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:02 smithi077 ceph-mon[32772]: pgmap v262: 97 pgs: 97 active+clean; 2.9 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-25T10:41:02.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:02 smithi164 ceph-mon[32713]: pgmap v262: 97 pgs: 97 active+clean; 2.9 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-25T10:41:02.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:02 smithi043 ceph-mon[29478]: pgmap v262: 97 pgs: 97 active+clean; 2.9 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-25T10:41:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.823+0000 7f456a7c5640 1 -- 172.21.15.43:0/2820421751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4564108a60 msgr2=0x7f4564102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.823+0000 7f456a7c5640 1 --2- 172.21.15.43:0/2820421751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4564108a60 0x7f4564102610 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f4554009900 tx=0x7f455402a410 comp rx=0 tx=0).stop 2024-04-25T10:41:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.823+0000 7f456a7c5640 1 -- 172.21.15.43:0/2820421751 shutdown_connections 2024-04-25T10:41:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.823+0000 7f456a7c5640 1 --2- 172.21.15.43:0/2820421751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45641031c0 0x7f456410f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.823+0000 7f456a7c5640 1 --2- 172.21.15.43:0/2820421751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4564108f00 0x7f4564102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.823+0000 7f456a7c5640 1 --2- 172.21.15.43:0/2820421751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4564108a60 0x7f4564102610 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.823+0000 7f456a7c5640 1 -- 172.21.15.43:0/2820421751 >> 172.21.15.43:0/2820421751 conn(0x7f45640781b0 msgr2=0x7f45640785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.823+0000 7f456a7c5640 1 -- 172.21.15.43:0/2820421751 shutdown_connections 2024-04-25T10:41:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.823+0000 7f456a7c5640 1 -- 172.21.15.43:0/2820421751 wait complete. 2024-04-25T10:41:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.824+0000 7f456a7c5640 1 Processor -- start 2024-04-25T10:41:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.824+0000 7f456a7c5640 1 -- start start 2024-04-25T10:41:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.824+0000 7f456a7c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45641031c0 0x7f456419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.824+0000 7f456a7c5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4564108a60 0x7f456419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.824+0000 7f456a7c5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4564108f00 0x7f456419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.824+0000 7f456a7c5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4564112c60 con 0x7f45641031c0 2024-04-25T10:41:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.824+0000 7f456a7c5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4564112ae0 con 0x7f4564108a60 2024-04-25T10:41:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.824+0000 7f456a7c5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4564112de0 con 0x7f4564108f00 2024-04-25T10:41:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.824+0000 7f4563fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45641031c0 0x7f456419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.825+0000 7f4563fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45641031c0 0x7f456419a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38990/0 (socket says 172.21.15.43:38990) 2024-04-25T10:41:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.825+0000 7f4563fff640 1 -- 172.21.15.43:0/3341817311 learned_addr learned my addr 172.21.15.43:0/3341817311 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.825+0000 7f4568d62640 1 --2- 172.21.15.43:0/3341817311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4564108f00 0x7f456419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.825+0000 7f45637fe640 1 --2- 172.21.15.43:0/3341817311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4564108a60 0x7f456419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.825+0000 7f4563fff640 1 -- 172.21.15.43:0/3341817311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4564108f00 msgr2=0x7f456419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.825+0000 7f4563fff640 1 --2- 172.21.15.43:0/3341817311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4564108f00 0x7f456419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.825+0000 7f4563fff640 1 -- 172.21.15.43:0/3341817311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4564108a60 msgr2=0x7f456419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.825+0000 7f4563fff640 1 --2- 172.21.15.43:0/3341817311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4564108a60 0x7f456419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.825+0000 7f4563fff640 1 -- 172.21.15.43:0/3341817311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456419f610 con 0x7f45641031c0 2024-04-25T10:41:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.825+0000 7f45637fe640 1 --2- 172.21.15.43:0/3341817311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4564108a60 0x7f456419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.825+0000 7f4563fff640 1 --2- 172.21.15.43:0/3341817311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45641031c0 0x7f456419a7b0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f455402a930 tx=0x7f4554004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.825+0000 7f45617fa640 1 -- 172.21.15.43:0/3341817311 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455403f030 con 0x7f45641031c0 2024-04-25T10:41:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.825+0000 7f45617fa640 1 -- 172.21.15.43:0/3341817311 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4554003aa0 con 0x7f45641031c0 2024-04-25T10:41:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.825+0000 7f456a7c5640 1 -- 172.21.15.43:0/3341817311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456419f8a0 con 0x7f45641031c0 2024-04-25T10:41:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.825+0000 7f45617fa640 1 -- 172.21.15.43:0/3341817311 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554035070 con 0x7f45641031c0 2024-04-25T10:41:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.825+0000 7f456a7c5640 1 -- 172.21.15.43:0/3341817311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f456419fde0 con 0x7f45641031c0 2024-04-25T10:41:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.828+0000 7f45617fa640 1 -- 172.21.15.43:0/3341817311 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f4554042030 con 0x7f45641031c0 2024-04-25T10:41:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.828+0000 7f4571677740 1 Processor -- start 2024-04-25T10:41:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.828+0000 7f4571677740 1 -- start start 2024-04-25T10:41:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.829+0000 7f4571677740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2cd271b0 0x557a2d0c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.829+0000 7f4571677740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.829+0000 7f4571677740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd064d0 0x557a2d0c7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.829+0000 7f4571677740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557a2d02d460 con 0x557a2ccfe2f0 2024-04-25T10:41:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.829+0000 7f4571677740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a2cfe7590 con 0x557a2cd271b0 2024-04-25T10:41:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.829+0000 7f4571677740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557a2d029420 con 0x557a2cd064d0 2024-04-25T10:41:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.829+0000 7f4563fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.829+0000 7f45617fa640 1 --2- 172.21.15.43:0/3341817311 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f454c0afcd0 0x7f454c0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.829+0000 7f4563fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0c7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38992/0 (socket says 172.21.15.43:38992) 2024-04-25T10:41:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.829+0000 7f4563fff640 1 -- 172.21.15.43:0/3027758853 learned_addr learned my addr 172.21.15.43:0/3027758853 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.829+0000 7f4568d62640 1 --2- 172.21.15.43:0/3027758853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2cd271b0 0x557a2d0c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.829+0000 7f45637fe640 1 --2- 172.21.15.43:0/3027758853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd064d0 0x557a2d0c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.829+0000 7f4568d62640 1 --2- 172.21.15.43:0/3341817311 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f454c0afcd0 0x7f454c0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.829+0000 7f45617fa640 1 -- 172.21.15.43:0/3341817311 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f45540ff4c0 con 0x7f45641031c0 2024-04-25T10:41:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.829+0000 7f4563fff640 1 -- 172.21.15.43:0/3027758853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd064d0 msgr2=0x557a2d0c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.829+0000 7f4563fff640 1 --2- 172.21.15.43:0/3027758853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd064d0 0x557a2d0c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.829+0000 7f4563fff640 1 -- 172.21.15.43:0/3027758853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2cd271b0 msgr2=0x557a2d0c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.829+0000 7f4563fff640 1 --2- 172.21.15.43:0/3027758853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2cd271b0 0x557a2d0c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.829+0000 7f4563fff640 1 -- 172.21.15.43:0/3027758853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a2cff40d0 con 0x557a2ccfe2f0 2024-04-25T10:41:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.829+0000 7f45637fe640 1 --2- 172.21.15.43:0/3027758853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd064d0 0x557a2d0c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:41:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.830+0000 7f4563fff640 1 --2- 172.21.15.43:0/3027758853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0c7030 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f45540fcbb0 tx=0x7f4554031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d8b681820a5220 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.830+0000 7f4568d62640 1 --2- 172.21.15.43:0/3341817311 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f454c0afcd0 0x7f454c0b21e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f455c009fd0 tx=0x7f455c0096a0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.830+0000 7f454a7fc640 1 -- 172.21.15.43:0/3027758853 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554049020 con 0x557a2ccfe2f0 2024-04-25T10:41:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.830+0000 7f454a7fc640 1 -- 172.21.15.43:0/3027758853 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45540c7d60 con 0x557a2ccfe2f0 2024-04-25T10:41:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.830+0000 7f454a7fc640 1 -- 172.21.15.43:0/3027758853 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554003c50 con 0x557a2ccfe2f0 2024-04-25T10:41:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.831+0000 7f4571677740 1 -- 172.21.15.43:0/3027758853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 msgr2=0x557a2d0c7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.831+0000 7f4571677740 1 --2- 172.21.15.43:0/3027758853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0c7030 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f45540fcbb0 tx=0x7f4554031a10 comp rx=0 tx=0).stop 2024-04-25T10:41:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.831+0000 7f4571677740 1 -- 172.21.15.43:0/3027758853 shutdown_connections 2024-04-25T10:41:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.831+0000 7f4571677740 1 --2- 172.21.15.43:0/3027758853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd064d0 0x557a2d0c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.831+0000 7f4571677740 1 --2- 172.21.15.43:0/3027758853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0c7030 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.831+0000 7f4571677740 1 --2- 172.21.15.43:0/3027758853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2cd271b0 0x557a2d0c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.831+0000 7f4571677740 1 -- 172.21.15.43:0/3027758853 >> 172.21.15.43:0/3027758853 conn(0x557a2cd1b300 msgr2=0x557a2d0c32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.831+0000 7f4571677740 1 -- 172.21.15.43:0/3027758853 shutdown_connections 2024-04-25T10:41:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.831+0000 7f4571677740 1 -- 172.21.15.43:0/3027758853 wait complete. 2024-04-25T10:41:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.831+0000 7f4571677740 1 -- start start 2024-04-25T10:41:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.831+0000 7f4571677740 1 Processor -- start 2024-04-25T10:41:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.832+0000 7f4571677740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0d1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.832+0000 7f4571677740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd064d0 0x557a2d0d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.832+0000 7f4571677740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2cd271b0 0x557a2d0d6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.832+0000 7f4571677740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557a2d02d460 con 0x557a2ccfe2f0 2024-04-25T10:41:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.832+0000 7f4571677740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a2cfb4b60 con 0x557a2cd271b0 2024-04-25T10:41:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.832+0000 7f4571677740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557a2d0d39e0 con 0x557a2cd064d0 2024-04-25T10:41:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.832+0000 7f4571677740 10 client.0 _ll_register_callbacks cb 0x557a2cfd92b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.832+0000 7f4563fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.832+0000 7f4563fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0d1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39008/0 (socket says 172.21.15.43:39008) 2024-04-25T10:41:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.832+0000 7f4563fff640 1 -- 172.21.15.43:0/3398748037 learned_addr learned my addr 172.21.15.43:0/3398748037 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.832+0000 7f4568d62640 1 --2- 172.21.15.43:0/3398748037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2cd271b0 0x557a2d0d6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.832+0000 7f45637fe640 1 --2- 172.21.15.43:0/3398748037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd064d0 0x557a2d0d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.832+0000 7f4563fff640 1 -- 172.21.15.43:0/3398748037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd064d0 msgr2=0x557a2d0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.832+0000 7f4563fff640 1 --2- 172.21.15.43:0/3398748037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd064d0 0x557a2d0d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.832+0000 7f4563fff640 1 -- 172.21.15.43:0/3398748037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2cd271b0 msgr2=0x557a2d0d6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.832+0000 7f4563fff640 1 --2- 172.21.15.43:0/3398748037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2cd271b0 0x557a2d0d6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.832+0000 7f4563fff640 1 -- 172.21.15.43:0/3398748037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a2cd3f210 con 0x557a2ccfe2f0 2024-04-25T10:41:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.832+0000 7f4563fff640 1 --2- 172.21.15.43:0/3398748037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0d1140 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f45540fccc0 tx=0x7f455402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84aa9bc42708a648 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.832+0000 7f45437fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.833+0000 7f45437fe640 1 -- 172.21.15.43:0/3398748037 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554049040 con 0x557a2ccfe2f0 2024-04-25T10:41:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.833+0000 7f45437fe640 1 -- 172.21.15.43:0/3398748037 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f45540fce90 con 0x557a2ccfe2f0 2024-04-25T10:41:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.833+0000 7f4571677740 1 -- 172.21.15.43:0/3398748037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45540c7d60 con 0x557a2ccfe2f0 2024-04-25T10:41:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.833+0000 7f45437fe640 1 -- 172.21.15.43:0/3398748037 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f45540c7660 con 0x557a2ccfe2f0 2024-04-25T10:41:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.833+0000 7f4571677740 10 client.18030 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.833+0000 7f4571677740 1 -- 172.21.15.43:0/3398748037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557a2ce0b620 con 0x557a2ccfe2f0 2024-04-25T10:41:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.833+0000 7f45437fe640 1 -- 172.21.15.43:0/3398748037 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45540c7d60 con 0x557a2ccfe2f0 2024-04-25T10:41:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.833+0000 7f45437fe640 1 -- 172.21.15.43:0/3398748037 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f45540c78c0 con 0x557a2ccfe2f0 2024-04-25T10:41:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.833+0000 7f4571677740 10 client.18030 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.833+0000 7f4571677740 10 client.18030 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.833+0000 7f4571677740 20 client.18030 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.833+0000 7f4571677740 1 --2- 172.21.15.43:0/3398748037 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557a2cce5d80 0x557a2d0e3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.833+0000 7f4571677740 4 client.18030 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:41:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.833+0000 7f4571677740 1 -- 172.21.15.43:0/3398748037 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a2d03b070 con 0x557a2cce5d80 2024-04-25T10:41:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.834+0000 7f45637fe640 1 --2- 172.21.15.43:0/3398748037 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557a2cce5d80 0x557a2d0e3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.834+0000 7f45437fe640 10 client.18030 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.834+0000 7f45637fe640 1 --2- 172.21.15.43:0/3398748037 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557a2cce5d80 0x557a2d0e3160 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de2244e43f2c4be5 server_cookie=9c104c4673c08b81 in_seq=0 out_seq=0 2024-04-25T10:41:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.835+0000 7f45437fe640 1 -- 172.21.15.43:0/3398748037 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557a2d03b070 con 0x557a2cce5d80 2024-04-25T10:41:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.835+0000 7f45437fe640 10 client.18030 handle_command_reply: tid=0 2024-04-25T10:41:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.835+0000 7f4571677740 1 client.18030 shutdown 2024-04-25T10:41:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.835+0000 7f4571677740 1 -- 172.21.15.43:0/3398748037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 msgr2=0x557a2d0d1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.835+0000 7f4571677740 1 --2- 172.21.15.43:0/3398748037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0d1140 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f45540fccc0 tx=0x7f455402a410 comp rx=0 tx=0).stop 2024-04-25T10:41:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.835+0000 7f4571677740 1 -- 172.21.15.43:0/3398748037 shutdown_connections 2024-04-25T10:41:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.835+0000 7f4571677740 1 --2- 172.21.15.43:0/3398748037 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557a2cce5d80 0x557a2d0e3160 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.835+0000 7f4571677740 1 --2- 172.21.15.43:0/3398748037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2cd271b0 0x557a2d0d6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.835+0000 7f4571677740 1 --2- 172.21.15.43:0/3398748037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd064d0 0x557a2d0d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.835+0000 7f45437fe640 0 client.18030 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.835+0000 7f4571677740 1 --2- 172.21.15.43:0/3398748037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0d1140 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.835+0000 7f4571677740 1 -- 172.21.15.43:0/3398748037 >> 172.21.15.43:0/3398748037 conn(0x557a2cd1b300 msgr2=0x557a2d0c6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.835+0000 7f4571677740 1 -- 172.21.15.43:0/3398748037 shutdown_connections 2024-04-25T10:41:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.835+0000 7f4571677740 1 -- 172.21.15.43:0/3398748037 wait complete. 2024-04-25T10:41:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.835+0000 7f4571677740 20 client.18030 trim_cache size 0 max 16384 2024-04-25T10:41:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.851+0000 7f4571677740 1 Processor -- start 2024-04-25T10:41:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.851+0000 7f4571677740 1 -- start start 2024-04-25T10:41:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.851+0000 7f4571677740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0d6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.851+0000 7f4571677740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2cd064d0 0x557a2d0cff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.851+0000 7f4571677740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd271b0 0x557a2d0e88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.851+0000 7f4571677740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45641b2360 con 0x557a2ccfe2f0 2024-04-25T10:41:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.851+0000 7f4571677740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f45641afc30 con 0x557a2cd064d0 2024-04-25T10:41:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.851+0000 7f4571677740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f45641adaa0 con 0x557a2cd271b0 2024-04-25T10:41:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.851+0000 7f4563fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0d6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.851+0000 7f4563fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0d6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39014/0 (socket says 172.21.15.43:39014) 2024-04-25T10:41:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.851+0000 7f4563fff640 1 -- 172.21.15.43:0/2626462445 learned_addr learned my addr 172.21.15.43:0/2626462445 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.852+0000 7f4568d62640 1 --2- 172.21.15.43:0/2626462445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd271b0 0x557a2d0e88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.852+0000 7f45637fe640 1 --2- 172.21.15.43:0/2626462445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2cd064d0 0x557a2d0cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.852+0000 7f4563fff640 1 -- 172.21.15.43:0/2626462445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd271b0 msgr2=0x557a2d0e88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.852+0000 7f4563fff640 1 --2- 172.21.15.43:0/2626462445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd271b0 0x557a2d0e88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.852+0000 7f4563fff640 1 -- 172.21.15.43:0/2626462445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2cd064d0 msgr2=0x557a2d0cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.852+0000 7f4563fff640 1 --2- 172.21.15.43:0/2626462445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2cd064d0 0x557a2d0cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.852+0000 7f4563fff640 1 -- 172.21.15.43:0/2626462445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a2d0c8340 con 0x557a2ccfe2f0 2024-04-25T10:41:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.852+0000 7f4563fff640 1 --2- 172.21.15.43:0/2626462445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0d6c50 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f45540fd210 tx=0x7f4554000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ab03aa9252a7fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.852+0000 7f4549ffb640 1 -- 172.21.15.43:0/2626462445 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45540c3520 con 0x557a2ccfe2f0 2024-04-25T10:41:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.852+0000 7f4549ffb640 1 -- 172.21.15.43:0/2626462445 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45540c7d60 con 0x557a2ccfe2f0 2024-04-25T10:41:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.852+0000 7f4549ffb640 1 -- 172.21.15.43:0/2626462445 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45540ccde0 con 0x557a2ccfe2f0 2024-04-25T10:41:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.853+0000 7f4571677740 1 -- 172.21.15.43:0/2626462445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 msgr2=0x557a2d0d6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.853+0000 7f4571677740 1 --2- 172.21.15.43:0/2626462445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0d6c50 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f45540fd210 tx=0x7f4554000bc0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.853+0000 7f4571677740 1 -- 172.21.15.43:0/2626462445 shutdown_connections 2024-04-25T10:41:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.853+0000 7f4571677740 1 --2- 172.21.15.43:0/2626462445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2cd064d0 0x557a2d0cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.853+0000 7f4571677740 1 --2- 172.21.15.43:0/2626462445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd271b0 0x557a2d0e88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.853+0000 7f4571677740 1 --2- 172.21.15.43:0/2626462445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0d6c50 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.853+0000 7f4571677740 1 -- 172.21.15.43:0/2626462445 >> 172.21.15.43:0/2626462445 conn(0x557a2cd1b300 msgr2=0x557a2d0cf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.853+0000 7f4571677740 1 -- 172.21.15.43:0/2626462445 shutdown_connections 2024-04-25T10:41:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.853+0000 7f4571677740 1 -- 172.21.15.43:0/2626462445 wait complete. 2024-04-25T10:41:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.853+0000 7f4571677740 1 -- start start 2024-04-25T10:41:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.854+0000 7f4571677740 1 Processor -- start 2024-04-25T10:41:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.854+0000 7f4571677740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0da4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.854+0000 7f4571677740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2cd064d0 0x557a2d0da9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.854+0000 7f4571677740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd271b0 0x557a2d143450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.854+0000 7f4571677740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45641ae0f0 con 0x557a2ccfe2f0 2024-04-25T10:41:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.854+0000 7f4571677740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f45641af560 con 0x557a2cd064d0 2024-04-25T10:41:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.854+0000 7f4571677740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f45641ae7c0 con 0x557a2cd271b0 2024-04-25T10:41:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.854+0000 7f4571677740 10 client.0 _ll_register_callbacks cb 0x7f455c0123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.854+0000 7f4563fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0da4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.854+0000 7f4563fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0da4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39018/0 (socket says 172.21.15.43:39018) 2024-04-25T10:41:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.854+0000 7f4563fff640 1 -- 172.21.15.43:0/926266639 learned_addr learned my addr 172.21.15.43:0/926266639 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.854+0000 7f45637fe640 1 --2- 172.21.15.43:0/926266639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2cd064d0 0x557a2d0da9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.854+0000 7f4568d62640 1 --2- 172.21.15.43:0/926266639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd271b0 0x557a2d143450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.854+0000 7f4563fff640 1 -- 172.21.15.43:0/926266639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd271b0 msgr2=0x557a2d143450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.854+0000 7f4563fff640 1 --2- 172.21.15.43:0/926266639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd271b0 0x557a2d143450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.854+0000 7f4563fff640 1 -- 172.21.15.43:0/926266639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2cd064d0 msgr2=0x557a2d0da9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.854+0000 7f4563fff640 1 --2- 172.21.15.43:0/926266639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2cd064d0 0x557a2d0da9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.854+0000 7f4563fff640 1 -- 172.21.15.43:0/926266639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a2d143990 con 0x557a2ccfe2f0 2024-04-25T10:41:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.855+0000 7f4563fff640 1 --2- 172.21.15.43:0/926266639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0da4a0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f45540c7360 tx=0x7f455402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b3375c9324131c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.855+0000 7f45437fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.855+0000 7f45437fe640 1 -- 172.21.15.43:0/926266639 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45540c3520 con 0x557a2ccfe2f0 2024-04-25T10:41:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.855+0000 7f45437fe640 1 -- 172.21.15.43:0/926266639 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f45540c3c50 con 0x557a2ccfe2f0 2024-04-25T10:41:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.855+0000 7f4571677740 1 -- 172.21.15.43:0/926266639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45540c7d60 con 0x557a2ccfe2f0 2024-04-25T10:41:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.855+0000 7f45437fe640 1 -- 172.21.15.43:0/926266639 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f45540d04c0 con 0x557a2ccfe2f0 2024-04-25T10:41:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.855+0000 7f4571677740 10 client.18036 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.855+0000 7f4571677740 1 -- 172.21.15.43:0/926266639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557a2d143990 con 0x557a2ccfe2f0 2024-04-25T10:41:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.855+0000 7f45437fe640 1 -- 172.21.15.43:0/926266639 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45540c7d60 con 0x557a2ccfe2f0 2024-04-25T10:41:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.855+0000 7f45437fe640 1 -- 172.21.15.43:0/926266639 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f45540c68c0 con 0x557a2ccfe2f0 2024-04-25T10:41:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.855+0000 7f4571677740 10 client.18036 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.855+0000 7f4571677740 10 client.18036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.855+0000 7f4571677740 20 client.18036 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.855+0000 7f4571677740 1 --2- 172.21.15.43:0/926266639 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557a2ccbb1c0 0x557a2d14cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.855+0000 7f4571677740 4 client.18036 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:41:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.855+0000 7f4571677740 1 -- 172.21.15.43:0/926266639 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a2d034ee0 con 0x557a2ccbb1c0 2024-04-25T10:41:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.856+0000 7f45637fe640 1 --2- 172.21.15.43:0/926266639 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557a2ccbb1c0 0x557a2d14cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.856+0000 7f45437fe640 10 client.18036 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.856+0000 7f45637fe640 1 --2- 172.21.15.43:0/926266639 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557a2ccbb1c0 0x557a2d14cfc0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc1c408c7535eff8 server_cookie=7be48dfa44ba013d in_seq=0 out_seq=0 2024-04-25T10:41:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.872+0000 7f45437fe640 1 -- 172.21.15.43:0/926266639 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a2d034ee0 con 0x557a2ccbb1c0 2024-04-25T10:41:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.872+0000 7f45437fe640 10 client.18036 handle_command_reply: tid=0 2024-04-25T10:41:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.872+0000 7f4571677740 1 client.18036 shutdown 2024-04-25T10:41:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.872+0000 7f4571677740 1 -- 172.21.15.43:0/926266639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 msgr2=0x557a2d0da4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.872+0000 7f4571677740 1 --2- 172.21.15.43:0/926266639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0da4a0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f45540c7360 tx=0x7f455402a410 comp rx=0 tx=0).stop 2024-04-25T10:41:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.872+0000 7f4571677740 1 -- 172.21.15.43:0/926266639 shutdown_connections 2024-04-25T10:41:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.872+0000 7f4571677740 1 --2- 172.21.15.43:0/926266639 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557a2ccbb1c0 0x557a2d14cfc0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.872+0000 7f4571677740 1 --2- 172.21.15.43:0/926266639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2cd064d0 0x557a2d0da9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.872+0000 7f4571677740 1 --2- 172.21.15.43:0/926266639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a2cd271b0 0x557a2d143450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.872+0000 7f45437fe640 0 client.18036 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.872+0000 7f4571677740 1 --2- 172.21.15.43:0/926266639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a2ccfe2f0 0x557a2d0da4a0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.872+0000 7f4571677740 1 -- 172.21.15.43:0/926266639 >> 172.21.15.43:0/926266639 conn(0x557a2cd1b300 msgr2=0x557a2d0d2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.872+0000 7f4571677740 1 -- 172.21.15.43:0/926266639 shutdown_connections 2024-04-25T10:41:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.873+0000 7f4571677740 1 -- 172.21.15.43:0/926266639 wait complete. 2024-04-25T10:41:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.873+0000 7f4571677740 20 client.18036 trim_cache size 0 max 16384 2024-04-25T10:41:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:41:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:41:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:41:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:41:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.873+0000 7f456a7c5640 1 -- 172.21.15.43:0/3341817311 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f454c0afcd0 msgr2=0x7f454c0b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.873+0000 7f456a7c5640 1 --2- 172.21.15.43:0/3341817311 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f454c0afcd0 0x7f454c0b21e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f455c009fd0 tx=0x7f455c0096a0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.873+0000 7f456a7c5640 1 -- 172.21.15.43:0/3341817311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45641031c0 msgr2=0x7f456419a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.873+0000 7f456a7c5640 1 --2- 172.21.15.43:0/3341817311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45641031c0 0x7f456419a7b0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f455402a930 tx=0x7f4554004270 comp rx=0 tx=0).stop 2024-04-25T10:41:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.873+0000 7f456a7c5640 1 -- 172.21.15.43:0/3341817311 shutdown_connections 2024-04-25T10:41:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.873+0000 7f456a7c5640 1 --2- 172.21.15.43:0/3341817311 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f454c0afcd0 0x7f454c0b21e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.873+0000 7f456a7c5640 1 --2- 172.21.15.43:0/3341817311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4564108a60 0x7f456419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.873+0000 7f456a7c5640 1 --2- 172.21.15.43:0/3341817311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4564108f00 0x7f456419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.873+0000 7f456a7c5640 1 --2- 172.21.15.43:0/3341817311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45641031c0 0x7f456419a7b0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.873+0000 7f456a7c5640 1 -- 172.21.15.43:0/3341817311 >> 172.21.15.43:0/3341817311 conn(0x7f45640781b0 msgr2=0x7f456406a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.873+0000 7f456a7c5640 1 -- 172.21.15.43:0/3341817311 shutdown_connections 2024-04-25T10:41:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:03.873+0000 7f456a7c5640 1 -- 172.21.15.43:0/3341817311 wait complete. 2024-04-25T10:41:04.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:04 smithi077 ceph-mon[32772]: pgmap v263: 97 pgs: 97 active+clean; 2.9 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-25T10:41:04.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:04 smithi164 ceph-mon[32713]: pgmap v263: 97 pgs: 97 active+clean; 2.9 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-25T10:41:04.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:04 smithi043 ceph-mon[29478]: pgmap v263: 97 pgs: 97 active+clean; 2.9 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-25T10:41:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:05 smithi164 ceph-mon[32713]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T10:41:05.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:05 smithi043 ceph-mon[29478]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T10:41:05.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:05 smithi077 ceph-mon[32772]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T10:41:06.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:06 smithi043 ceph-mon[29478]: pgmap v265: 97 pgs: 97 active+clean; 3.1 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 99 KiB/s wr, 20 op/s 2024-04-25T10:41:06.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:06 smithi077 ceph-mon[32772]: pgmap v265: 97 pgs: 97 active+clean; 3.1 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 99 KiB/s wr, 20 op/s 2024-04-25T10:41:06.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:06 smithi164 ceph-mon[32713]: pgmap v265: 97 pgs: 97 active+clean; 3.1 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 99 KiB/s wr, 20 op/s 2024-04-25T10:41:08.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:08 smithi043 ceph-mon[29478]: pgmap v266: 97 pgs: 97 active+clean; 3.1 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 99 KiB/s wr, 20 op/s 2024-04-25T10:41:08.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:08 smithi077 ceph-mon[32772]: pgmap v266: 97 pgs: 97 active+clean; 3.1 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 99 KiB/s wr, 20 op/s 2024-04-25T10:41:08.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:08 smithi164 ceph-mon[32713]: pgmap v266: 97 pgs: 97 active+clean; 3.1 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 99 KiB/s wr, 20 op/s 2024-04-25T10:41:10.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:10 smithi043 ceph-mon[29478]: pgmap v267: 97 pgs: 97 active+clean; 3.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 23 op/s 2024-04-25T10:41:10.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:10 smithi077 ceph-mon[32772]: pgmap v267: 97 pgs: 97 active+clean; 3.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 23 op/s 2024-04-25T10:41:10.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:10 smithi164 ceph-mon[32713]: pgmap v267: 97 pgs: 97 active+clean; 3.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 23 op/s 2024-04-25T10:41:12.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:12 smithi043 ceph-mon[29478]: pgmap v268: 97 pgs: 97 active+clean; 3.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 23 op/s 2024-04-25T10:41:12.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:12 smithi077 ceph-mon[32772]: pgmap v268: 97 pgs: 97 active+clean; 3.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 23 op/s 2024-04-25T10:41:12.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:12 smithi164 ceph-mon[32713]: pgmap v268: 97 pgs: 97 active+clean; 3.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 23 op/s 2024-04-25T10:41:14.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:13 smithi077 ceph-mon[32772]: pgmap v269: 97 pgs: 97 active+clean; 3.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 23 op/s 2024-04-25T10:41:14.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:13 smithi077 ceph-mon[32772]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T10:41:14.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:13 smithi164 ceph-mon[32713]: pgmap v269: 97 pgs: 97 active+clean; 3.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 23 op/s 2024-04-25T10:41:14.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:13 smithi164 ceph-mon[32713]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T10:41:14.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:13 smithi043 ceph-mon[29478]: pgmap v269: 97 pgs: 97 active+clean; 3.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 23 op/s 2024-04-25T10:41:14.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:13 smithi043 ceph-mon[29478]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T10:41:16.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:15 smithi077 ceph-mon[32772]: pgmap v271: 97 pgs: 97 active+clean; 3.5 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-04-25T10:41:16.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:15 smithi164 ceph-mon[32713]: pgmap v271: 97 pgs: 97 active+clean; 3.5 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-04-25T10:41:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:15 smithi043 ceph-mon[29478]: pgmap v271: 97 pgs: 97 active+clean; 3.5 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-04-25T10:41:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.987+0000 7f9a1cc77640 1 -- 172.21.15.43:0/3620162442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a18104c70 msgr2=0x7f9a18105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.987+0000 7f9a1cc77640 1 --2- 172.21.15.43:0/3620162442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a18104c70 0x7f9a18105080 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f9a08009ca0 tx=0x7f9a0802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:41:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.987+0000 7f9a1cc77640 1 -- 172.21.15.43:0/3620162442 shutdown_connections 2024-04-25T10:41:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.987+0000 7f9a1cc77640 1 --2- 172.21.15.43:0/3620162442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a1810b6d0 0x7f9a1810daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.987+0000 7f9a1cc77640 1 --2- 172.21.15.43:0/3620162442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a1810e0c0 0x7f9a181105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.987+0000 7f9a1cc77640 1 --2- 172.21.15.43:0/3620162442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a18104c70 0x7f9a18105080 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.987+0000 7f9a1cc77640 1 -- 172.21.15.43:0/3620162442 >> 172.21.15.43:0/3620162442 conn(0x7f9a180781b0 msgr2=0x7f9a180785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.987+0000 7f9a1cc77640 1 -- 172.21.15.43:0/3620162442 shutdown_connections 2024-04-25T10:41:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.987+0000 7f9a1cc77640 1 -- 172.21.15.43:0/3620162442 wait complete. 2024-04-25T10:41:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.988+0000 7f9a1cc77640 1 Processor -- start 2024-04-25T10:41:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.988+0000 7f9a1cc77640 1 -- start start 2024-04-25T10:41:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.989+0000 7f9a1cc77640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a18104c70 0x7f9a1819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.989+0000 7f9a1cc77640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a1810b6d0 0x7f9a1819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.989+0000 7f9a1cc77640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a1810e0c0 0x7f9a1819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.989+0000 7f9a1cc77640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18113d20 con 0x7f9a1810b6d0 2024-04-25T10:41:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.989+0000 7f9a1cc77640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18113ba0 con 0x7f9a1810e0c0 2024-04-25T10:41:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.989+0000 7f9a1cc77640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18113ea0 con 0x7f9a18104c70 2024-04-25T10:41:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.989+0000 7f9a15d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a1810b6d0 0x7f9a1819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.989+0000 7f9a15d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a1810b6d0 0x7f9a1819ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48166/0 (socket says 172.21.15.43:48166) 2024-04-25T10:41:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.989+0000 7f9a15d9b640 1 -- 172.21.15.43:0/111000126 learned_addr learned my addr 172.21.15.43:0/111000126 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.989+0000 7f9a16d9d640 1 --2- 172.21.15.43:0/111000126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a1810e0c0 0x7f9a1819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.989+0000 7f9a1659c640 1 --2- 172.21.15.43:0/111000126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a18104c70 0x7f9a1819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.989+0000 7f9a15d9b640 1 -- 172.21.15.43:0/111000126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a18104c70 msgr2=0x7f9a1819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.989+0000 7f9a15d9b640 1 --2- 172.21.15.43:0/111000126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a18104c70 0x7f9a1819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.989+0000 7f9a15d9b640 1 -- 172.21.15.43:0/111000126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a1810e0c0 msgr2=0x7f9a1819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.989+0000 7f9a15d9b640 1 --2- 172.21.15.43:0/111000126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a1810e0c0 0x7f9a1819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.989+0000 7f9a15d9b640 1 -- 172.21.15.43:0/111000126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1819f580 con 0x7f9a1810b6d0 2024-04-25T10:41:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.989+0000 7f9a15d9b640 1 --2- 172.21.15.43:0/111000126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a1810b6d0 0x7f9a1819ac80 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00d950 tx=0x7f9a0c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.990+0000 7f9a077fe640 1 -- 172.21.15.43:0/111000126 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c017030 con 0x7f9a1810b6d0 2024-04-25T10:41:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.990+0000 7f9a077fe640 1 -- 172.21.15.43:0/111000126 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a0c004020 con 0x7f9a1810b6d0 2024-04-25T10:41:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.990+0000 7f9a1cc77640 1 -- 172.21.15.43:0/111000126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a1819f870 con 0x7f9a1810b6d0 2024-04-25T10:41:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.990+0000 7f9a077fe640 1 -- 172.21.15.43:0/111000126 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c0047a0 con 0x7f9a1810b6d0 2024-04-25T10:41:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.990+0000 7f9a1cc77640 1 -- 172.21.15.43:0/111000126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a181a71f0 con 0x7f9a1810b6d0 2024-04-25T10:41:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.992+0000 7f9a23c05740 1 Processor -- start 2024-04-25T10:41:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.992+0000 7f9a077fe640 1 -- 172.21.15.43:0/111000126 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f9a0c009a30 con 0x7f9a1810b6d0 2024-04-25T10:41:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.993+0000 7f9a23c05740 1 -- start start 2024-04-25T10:41:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.993+0000 7f9a23c05740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636a9e7f1b0 0x5636aa21e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.993+0000 7f9a23c05740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e562f0 0x5636aa21eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.993+0000 7f9a23c05740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636a9e5e4d0 0x5636aa21f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.993+0000 7f9a23c05740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5636aa185460 con 0x5636a9e562f0 2024-04-25T10:41:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.993+0000 7f9a23c05740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5636aa13f590 con 0x5636a9e7f1b0 2024-04-25T10:41:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.993+0000 7f9a23c05740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5636aa181420 con 0x5636a9e5e4d0 2024-04-25T10:41:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.993+0000 7f9a1659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e562f0 0x5636aa21eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.993+0000 7f9a1659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e562f0 0x5636aa21eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48168/0 (socket says 172.21.15.43:48168) 2024-04-25T10:41:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.993+0000 7f9a1659c640 1 -- 172.21.15.43:0/1763659191 learned_addr learned my addr 172.21.15.43:0/1763659191 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.993+0000 7f9a15d9b640 1 --2- 172.21.15.43:0/1763659191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636a9e5e4d0 0x5636aa21f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.993+0000 7f9a077fe640 1 --2- 172.21.15.43:0/111000126 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9a000afc50 0x7f9a000b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.993+0000 7f9a16d9d640 1 --2- 172.21.15.43:0/1763659191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636a9e7f1b0 0x5636aa21e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.994+0000 7f9a1659c640 1 -- 172.21.15.43:0/1763659191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636a9e5e4d0 msgr2=0x5636aa21f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.994+0000 7f9a1659c640 1 --2- 172.21.15.43:0/1763659191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636a9e5e4d0 0x5636aa21f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.994+0000 7f9a1659c640 1 -- 172.21.15.43:0/1763659191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636a9e7f1b0 msgr2=0x5636aa21e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.994+0000 7f9a1659c640 1 --2- 172.21.15.43:0/1763659191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636a9e7f1b0 0x5636aa21e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.994+0000 7f9a1659c640 1 -- 172.21.15.43:0/1763659191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636aa14c0d0 con 0x5636a9e562f0 2024-04-25T10:41:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.994+0000 7f9a15d9b640 1 --2- 172.21.15.43:0/1763659191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636a9e5e4d0 0x5636aa21f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.994+0000 7f9a16d9d640 1 --2- 172.21.15.43:0/111000126 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9a000afc50 0x7f9a000b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.994+0000 7f9a077fe640 1 -- 172.21.15.43:0/111000126 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f9a0c0de9f0 con 0x7f9a1810b6d0 2024-04-25T10:41:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.994+0000 7f9a1659c640 1 --2- 172.21.15.43:0/1763659191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e562f0 0x5636aa21eee0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f9a08035000 tx=0x7f9a080094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c27a5fb0d6722bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.994+0000 7f9a04ff9640 1 -- 172.21.15.43:0/1763659191 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08031db0 con 0x5636a9e562f0 2024-04-25T10:41:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.994+0000 7f9a04ff9640 1 -- 172.21.15.43:0/1763659191 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a08036030 con 0x5636a9e562f0 2024-04-25T10:41:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.994+0000 7f9a04ff9640 1 -- 172.21.15.43:0/1763659191 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a080042f0 con 0x5636a9e562f0 2024-04-25T10:41:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.995+0000 7f9a23c05740 1 -- 172.21.15.43:0/1763659191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e562f0 msgr2=0x5636aa21eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.995+0000 7f9a16d9d640 1 --2- 172.21.15.43:0/111000126 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9a000afc50 0x7f9a000b2160 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000b6c0 tx=0x7f9a100046c0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.995+0000 7f9a23c05740 1 --2- 172.21.15.43:0/1763659191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e562f0 0x5636aa21eee0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f9a08035000 tx=0x7f9a080094a0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.995+0000 7f9a23c05740 1 -- 172.21.15.43:0/1763659191 shutdown_connections 2024-04-25T10:41:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.995+0000 7f9a23c05740 1 --2- 172.21.15.43:0/1763659191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636a9e5e4d0 0x5636aa21f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.995+0000 7f9a23c05740 1 --2- 172.21.15.43:0/1763659191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e562f0 0x5636aa21eee0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.995+0000 7f9a23c05740 1 --2- 172.21.15.43:0/1763659191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636a9e7f1b0 0x5636aa21e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.995+0000 7f9a23c05740 1 -- 172.21.15.43:0/1763659191 >> 172.21.15.43:0/1763659191 conn(0x5636a9e73300 msgr2=0x5636aa21b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.995+0000 7f9a23c05740 1 -- 172.21.15.43:0/1763659191 shutdown_connections 2024-04-25T10:41:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.995+0000 7f9a23c05740 1 -- 172.21.15.43:0/1763659191 wait complete. 2024-04-25T10:41:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.996+0000 7f9a23c05740 1 -- start start 2024-04-25T10:41:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.996+0000 7f9a23c05740 1 Processor -- start 2024-04-25T10:41:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.996+0000 7f9a23c05740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636a9e562f0 0x5636aa22d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.996+0000 7f9a23c05740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636a9e5e4d0 0x5636aa22db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.996+0000 7f9a23c05740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e7f1b0 0x5636aa233340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.996+0000 7f9a23c05740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5636aa22fe50 con 0x5636a9e7f1b0 2024-04-25T10:41:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.996+0000 7f9a23c05740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5636aa10cb60 con 0x5636a9e562f0 2024-04-25T10:41:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.996+0000 7f9a23c05740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5636aa22ffd0 con 0x5636a9e5e4d0 2024-04-25T10:41:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.996+0000 7f9a23c05740 10 client.0 _ll_register_callbacks cb 0x5636aa1312b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.996+0000 7f9a16d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e7f1b0 0x5636aa233340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.996+0000 7f9a16d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e7f1b0 0x5636aa233340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48174/0 (socket says 172.21.15.43:48174) 2024-04-25T10:41:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.996+0000 7f9a16d9d640 1 -- 172.21.15.43:0/1983645282 learned_addr learned my addr 172.21.15.43:0/1983645282 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.996+0000 7f9a1659c640 1 --2- 172.21.15.43:0/1983645282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636a9e562f0 0x5636aa22d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.997+0000 7f9a15d9b640 1 --2- 172.21.15.43:0/1983645282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636a9e5e4d0 0x5636aa22db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.997+0000 7f9a16d9d640 1 -- 172.21.15.43:0/1983645282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636a9e5e4d0 msgr2=0x5636aa22db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.997+0000 7f9a16d9d640 1 --2- 172.21.15.43:0/1983645282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636a9e5e4d0 0x5636aa22db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.997+0000 7f9a16d9d640 1 -- 172.21.15.43:0/1983645282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636a9e562f0 msgr2=0x5636aa22d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.997+0000 7f9a16d9d640 1 --2- 172.21.15.43:0/1983645282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636a9e562f0 0x5636aa22d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.997+0000 7f9a16d9d640 1 -- 172.21.15.43:0/1983645282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636a9e97210 con 0x5636a9e7f1b0 2024-04-25T10:41:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.997+0000 7f9a16d9d640 1 --2- 172.21.15.43:0/1983645282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e7f1b0 0x5636aa233340 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f9a10003e20 tx=0x7f9a10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8e439a0db7b775 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.997+0000 7f99fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.997+0000 7f99fdffb640 1 -- 172.21.15.43:0/1983645282 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1001b820 con 0x5636a9e7f1b0 2024-04-25T10:41:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.997+0000 7f99fdffb640 1 -- 172.21.15.43:0/1983645282 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f9a1001b9c0 con 0x5636a9e7f1b0 2024-04-25T10:41:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.997+0000 7f9a23c05740 1 -- 172.21.15.43:0/1983645282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a08036030 con 0x5636a9e7f1b0 2024-04-25T10:41:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.997+0000 7f99fdffb640 1 -- 172.21.15.43:0/1983645282 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9a10003e50 con 0x5636a9e7f1b0 2024-04-25T10:41:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.997+0000 7f9a23c05740 10 client.18048 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.997+0000 7f9a23c05740 1 -- 172.21.15.43:0/1983645282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5636a9f63620 con 0x5636a9e7f1b0 2024-04-25T10:41:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.998+0000 7f99fdffb640 1 -- 172.21.15.43:0/1983645282 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a08036030 con 0x5636a9e7f1b0 2024-04-25T10:41:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.998+0000 7f99fdffb640 1 -- 172.21.15.43:0/1983645282 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9a10025070 con 0x5636a9e7f1b0 2024-04-25T10:41:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.998+0000 7f9a23c05740 10 client.18048 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.998+0000 7f9a23c05740 10 client.18048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.998+0000 7f9a23c05740 20 client.18048 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.998+0000 7f9a23c05740 1 --2- 172.21.15.43:0/1983645282 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5636a9e3dd80 0x5636aa23f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.998+0000 7f9a23c05740 4 client.18048 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:41:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.998+0000 7f9a23c05740 1 -- 172.21.15.43:0/1983645282 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636aa193070 con 0x5636a9e3dd80 2024-04-25T10:41:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.998+0000 7f9a1659c640 1 --2- 172.21.15.43:0/1983645282 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5636a9e3dd80 0x5636aa23f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.999+0000 7f9a1659c640 1 --2- 172.21.15.43:0/1983645282 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5636a9e3dd80 0x5636aa23f900 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=788255fd87af57a2 server_cookie=95f3b4f70e11ebf in_seq=0 out_seq=0 2024-04-25T10:41:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.999+0000 7f99fdffb640 10 client.18048 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.999+0000 7f99fdffb640 1 -- 172.21.15.43:0/1983645282 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5636aa193070 con 0x5636a9e3dd80 2024-04-25T10:41:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.999+0000 7f99fdffb640 10 client.18048 handle_command_reply: tid=0 2024-04-25T10:41:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.999+0000 7f9a23c05740 1 client.18048 shutdown 2024-04-25T10:41:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.999+0000 7f9a23c05740 1 -- 172.21.15.43:0/1983645282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e7f1b0 msgr2=0x5636aa233340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.999+0000 7f9a23c05740 1 --2- 172.21.15.43:0/1983645282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e7f1b0 0x5636aa233340 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f9a10003e20 tx=0x7f9a10004730 comp rx=0 tx=0).stop 2024-04-25T10:41:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.999+0000 7f9a23c05740 1 -- 172.21.15.43:0/1983645282 shutdown_connections 2024-04-25T10:41:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.999+0000 7f9a23c05740 1 --2- 172.21.15.43:0/1983645282 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5636a9e3dd80 0x5636aa23f900 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.999+0000 7f9a23c05740 1 --2- 172.21.15.43:0/1983645282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e7f1b0 0x5636aa233340 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.999+0000 7f9a23c05740 1 --2- 172.21.15.43:0/1983645282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636a9e5e4d0 0x5636aa22db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.999+0000 7f9a23c05740 1 --2- 172.21.15.43:0/1983645282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636a9e562f0 0x5636aa22d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.999+0000 7f99fdffb640 0 client.18048 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:16.999+0000 7f9a23c05740 1 -- 172.21.15.43:0/1983645282 >> 172.21.15.43:0/1983645282 conn(0x5636a9e73300 msgr2=0x5636aa2207a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.000+0000 7f9a23c05740 1 -- 172.21.15.43:0/1983645282 shutdown_connections 2024-04-25T10:41:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.000+0000 7f9a23c05740 1 -- 172.21.15.43:0/1983645282 wait complete. 2024-04-25T10:41:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.000+0000 7f9a23c05740 20 client.18048 trim_cache size 0 max 16384 2024-04-25T10:41:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.015+0000 7f9a23c05740 1 Processor -- start 2024-04-25T10:41:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.015+0000 7f9a23c05740 1 -- start start 2024-04-25T10:41:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.015+0000 7f9a23c05740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e562f0 0x5636aa233240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.015+0000 7f9a23c05740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636a9e5e4d0 0x5636aa226d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.015+0000 7f9a23c05740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636a9e7f1b0 0x5636aa227270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.015+0000 7f9a23c05740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a181b0090 con 0x5636a9e562f0 2024-04-25T10:41:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.015+0000 7f9a23c05740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9a181ad960 con 0x5636a9e5e4d0 2024-04-25T10:41:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.015+0000 7f9a23c05740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9a181ab7d0 con 0x5636a9e7f1b0 2024-04-25T10:41:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.015+0000 7f9a1659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e562f0 0x5636aa233240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.015+0000 7f9a1659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e562f0 0x5636aa233240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48186/0 (socket says 172.21.15.43:48186) 2024-04-25T10:41:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.015+0000 7f9a1659c640 1 -- 172.21.15.43:0/2441753960 learned_addr learned my addr 172.21.15.43:0/2441753960 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.016+0000 7f9a1659c640 1 -- 172.21.15.43:0/2441753960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636a9e7f1b0 msgr2=0x5636aa227270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.016+0000 7f9a1659c640 1 --2- 172.21.15.43:0/2441753960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636a9e7f1b0 0x5636aa227270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.016+0000 7f9a1659c640 1 -- 172.21.15.43:0/2441753960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636a9e5e4d0 msgr2=0x5636aa226d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.016+0000 7f9a1659c640 1 --2- 172.21.15.43:0/2441753960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636a9e5e4d0 0x5636aa226d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.016+0000 7f9a1659c640 1 -- 172.21.15.43:0/2441753960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636aa228f20 con 0x5636a9e562f0 2024-04-25T10:41:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.016+0000 7f9a1659c640 1 --2- 172.21.15.43:0/2441753960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e562f0 0x5636aa233240 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x5636aa21c5e0 tx=0x7f9a08003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417226e454e733a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.016+0000 7f99fffff640 1 -- 172.21.15.43:0/2441753960 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0805e070 con 0x5636a9e562f0 2024-04-25T10:41:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.016+0000 7f99fffff640 1 -- 172.21.15.43:0/2441753960 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a08055070 con 0x5636a9e562f0 2024-04-25T10:41:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.016+0000 7f99fffff640 1 -- 172.21.15.43:0/2441753960 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0804a040 con 0x5636a9e562f0 2024-04-25T10:41:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.017+0000 7f9a23c05740 1 -- 172.21.15.43:0/2441753960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e562f0 msgr2=0x5636aa233240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.017+0000 7f9a23c05740 1 --2- 172.21.15.43:0/2441753960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e562f0 0x5636aa233240 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x5636aa21c5e0 tx=0x7f9a08003ea0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.017+0000 7f9a23c05740 1 -- 172.21.15.43:0/2441753960 shutdown_connections 2024-04-25T10:41:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.017+0000 7f9a23c05740 1 --2- 172.21.15.43:0/2441753960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636a9e5e4d0 0x5636aa226d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.017+0000 7f9a23c05740 1 --2- 172.21.15.43:0/2441753960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636a9e7f1b0 0x5636aa227270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.017+0000 7f9a23c05740 1 --2- 172.21.15.43:0/2441753960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e562f0 0x5636aa233240 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.017+0000 7f9a23c05740 1 -- 172.21.15.43:0/2441753960 >> 172.21.15.43:0/2441753960 conn(0x5636a9e73300 msgr2=0x5636aa2207a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.017+0000 7f9a23c05740 1 -- 172.21.15.43:0/2441753960 shutdown_connections 2024-04-25T10:41:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.017+0000 7f9a23c05740 1 -- 172.21.15.43:0/2441753960 wait complete. 2024-04-25T10:41:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.017+0000 7f9a23c05740 1 -- start start 2024-04-25T10:41:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.017+0000 7f9a23c05740 1 Processor -- start 2024-04-25T10:41:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f9a23c05740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636a9e562f0 0x5636aa239b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f9a23c05740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636a9e5e4d0 0x5636aa23a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f9a23c05740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e7f1b0 0x5636aa244ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f9a23c05740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a181abe20 con 0x5636a9e7f1b0 2024-04-25T10:41:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f9a23c05740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9a181ad290 con 0x5636a9e5e4d0 2024-04-25T10:41:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f9a23c05740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9a181ac4f0 con 0x5636a9e562f0 2024-04-25T10:41:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f9a23c05740 10 client.0 _ll_register_callbacks cb 0x7f9a10004380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f9a16d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e7f1b0 0x5636aa244ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f9a16d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e7f1b0 0x5636aa244ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48194/0 (socket says 172.21.15.43:48194) 2024-04-25T10:41:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f9a16d9d640 1 -- 172.21.15.43:0/810806129 learned_addr learned my addr 172.21.15.43:0/810806129 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f9a1659c640 1 --2- 172.21.15.43:0/810806129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636a9e562f0 0x5636aa239b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f9a15d9b640 1 --2- 172.21.15.43:0/810806129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636a9e5e4d0 0x5636aa23a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f9a16d9d640 1 -- 172.21.15.43:0/810806129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636a9e562f0 msgr2=0x5636aa239b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f9a16d9d640 1 --2- 172.21.15.43:0/810806129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636a9e562f0 0x5636aa239b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f9a16d9d640 1 -- 172.21.15.43:0/810806129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636a9e5e4d0 msgr2=0x5636aa23a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f9a16d9d640 1 --2- 172.21.15.43:0/810806129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636a9e5e4d0 0x5636aa23a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f9a16d9d640 1 -- 172.21.15.43:0/810806129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636aa2451e0 con 0x5636a9e7f1b0 2024-04-25T10:41:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f9a15d9b640 1 --2- 172.21.15.43:0/810806129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636a9e5e4d0 0x5636aa23a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f9a16d9d640 1 --2- 172.21.15.43:0/810806129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e7f1b0 0x5636aa244ba0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9a10027db0 tx=0x7f9a10030a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bfba6c25d2e83e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f99fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f99fdffb640 1 -- 172.21.15.43:0/810806129 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10003a00 con 0x5636a9e7f1b0 2024-04-25T10:41:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.018+0000 7f99fdffb640 1 -- 172.21.15.43:0/810806129 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f9a10027440 con 0x5636a9e7f1b0 2024-04-25T10:41:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.019+0000 7f9a23c05740 1 -- 172.21.15.43:0/810806129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a08055070 con 0x5636a9e7f1b0 2024-04-25T10:41:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.019+0000 7f99fdffb640 1 -- 172.21.15.43:0/810806129 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9a1001ecb0 con 0x5636a9e7f1b0 2024-04-25T10:41:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.019+0000 7f9a23c05740 10 client.18054 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.019+0000 7f9a23c05740 1 -- 172.21.15.43:0/810806129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5636aa237f40 con 0x5636a9e7f1b0 2024-04-25T10:41:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.019+0000 7f99fdffb640 1 -- 172.21.15.43:0/810806129 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a08055070 con 0x5636a9e7f1b0 2024-04-25T10:41:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.019+0000 7f99fdffb640 1 -- 172.21.15.43:0/810806129 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9a10025050 con 0x5636a9e7f1b0 2024-04-25T10:41:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.019+0000 7f9a23c05740 10 client.18054 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.019+0000 7f9a23c05740 10 client.18054 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.019+0000 7f9a23c05740 20 client.18054 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.019+0000 7f9a23c05740 1 --2- 172.21.15.43:0/810806129 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5636a9e131c0 0x5636aa2a9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.019+0000 7f9a23c05740 4 client.18054 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:41:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.019+0000 7f9a23c05740 1 -- 172.21.15.43:0/810806129 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5636aa18cee0 con 0x5636a9e131c0 2024-04-25T10:41:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.019+0000 7f9a1659c640 1 --2- 172.21.15.43:0/810806129 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5636a9e131c0 0x5636aa2a9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.020+0000 7f99fdffb640 10 client.18054 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.020+0000 7f9a1659c640 1 --2- 172.21.15.43:0/810806129 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5636a9e131c0 0x5636aa2a9d10 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3606b9678ed24922 server_cookie=75108a034d696fd6 in_seq=0 out_seq=0 2024-04-25T10:41:17.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.035+0000 7f99fdffb640 1 -- 172.21.15.43:0/810806129 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5636aa18cee0 con 0x5636a9e131c0 2024-04-25T10:41:17.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.035+0000 7f99fdffb640 10 client.18054 handle_command_reply: tid=0 2024-04-25T10:41:17.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.035+0000 7f9a23c05740 1 client.18054 shutdown 2024-04-25T10:41:17.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.035+0000 7f9a23c05740 1 -- 172.21.15.43:0/810806129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e7f1b0 msgr2=0x5636aa244ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.035+0000 7f9a23c05740 1 --2- 172.21.15.43:0/810806129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e7f1b0 0x5636aa244ba0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9a10027db0 tx=0x7f9a10030a50 comp rx=0 tx=0).stop 2024-04-25T10:41:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.035+0000 7f9a23c05740 1 -- 172.21.15.43:0/810806129 shutdown_connections 2024-04-25T10:41:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.035+0000 7f9a23c05740 1 --2- 172.21.15.43:0/810806129 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5636a9e131c0 0x5636aa2a9d10 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.035+0000 7f9a23c05740 1 --2- 172.21.15.43:0/810806129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636a9e5e4d0 0x5636aa23a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.035+0000 7f9a23c05740 1 --2- 172.21.15.43:0/810806129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636a9e7f1b0 0x5636aa244ba0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.035+0000 7f9a23c05740 1 --2- 172.21.15.43:0/810806129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636a9e562f0 0x5636aa239b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.035+0000 7f99fdffb640 0 client.18054 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.035+0000 7f9a23c05740 1 -- 172.21.15.43:0/810806129 >> 172.21.15.43:0/810806129 conn(0x5636a9e73300 msgr2=0x5636aa22bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.035+0000 7f9a23c05740 1 -- 172.21.15.43:0/810806129 shutdown_connections 2024-04-25T10:41:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.035+0000 7f9a23c05740 1 -- 172.21.15.43:0/810806129 wait complete. 2024-04-25T10:41:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.035+0000 7f9a23c05740 20 client.18054 trim_cache size 0 max 16384 2024-04-25T10:41:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:41:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:41:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:41:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:41:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.036+0000 7f9a1cc77640 1 -- 172.21.15.43:0/111000126 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9a000afc50 msgr2=0x7f9a000b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.036+0000 7f9a1cc77640 1 --2- 172.21.15.43:0/111000126 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9a000afc50 0x7f9a000b2160 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000b6c0 tx=0x7f9a100046c0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.036+0000 7f9a1cc77640 1 -- 172.21.15.43:0/111000126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a1810b6d0 msgr2=0x7f9a1819ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.036+0000 7f9a1cc77640 1 --2- 172.21.15.43:0/111000126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a1810b6d0 0x7f9a1819ac80 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00d950 tx=0x7f9a0c00de10 comp rx=0 tx=0).stop 2024-04-25T10:41:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.036+0000 7f9a1cc77640 1 -- 172.21.15.43:0/111000126 shutdown_connections 2024-04-25T10:41:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.036+0000 7f9a1cc77640 1 --2- 172.21.15.43:0/111000126 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9a000afc50 0x7f9a000b2160 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.036+0000 7f9a1cc77640 1 --2- 172.21.15.43:0/111000126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a1810e0c0 0x7f9a1819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.036+0000 7f9a1cc77640 1 --2- 172.21.15.43:0/111000126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a1810b6d0 0x7f9a1819ac80 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.036+0000 7f9a1cc77640 1 --2- 172.21.15.43:0/111000126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a18104c70 0x7f9a1819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.036+0000 7f9a1cc77640 1 -- 172.21.15.43:0/111000126 >> 172.21.15.43:0/111000126 conn(0x7f9a180781b0 msgr2=0x7f9a18101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.036+0000 7f9a1cc77640 1 -- 172.21.15.43:0/111000126 shutdown_connections 2024-04-25T10:41:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:17.036+0000 7f9a1cc77640 1 -- 172.21.15.43:0/111000126 wait complete. 2024-04-25T10:41:18.109 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:41:18.110 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:41:18.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.237+0000 7f01fdaaa640 1 -- 172.21.15.43:0/1234025515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f810d380 msgr2=0x7f01f810f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.237+0000 7f01fdaaa640 1 --2- 172.21.15.43:0/1234025515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f810d380 0x7f01f810f7a0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009ca0 tx=0x7f01f002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.237+0000 7f01fdaaa640 1 -- 172.21.15.43:0/1234025515 shutdown_connections 2024-04-25T10:41:18.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.237+0000 7f01fdaaa640 1 --2- 172.21.15.43:0/1234025515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01f8102af0 0x7f01f8102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.237+0000 7f01fdaaa640 1 --2- 172.21.15.43:0/1234025515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f810d380 0x7f01f810f7a0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.237+0000 7f01fdaaa640 1 --2- 172.21.15.43:0/1234025515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01f8108a20 0x7f01f81025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.237+0000 7f01fdaaa640 1 -- 172.21.15.43:0/1234025515 >> 172.21.15.43:0/1234025515 conn(0x7f01f8078100 msgr2=0x7f01f8078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:18.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.237+0000 7f01fdaaa640 1 -- 172.21.15.43:0/1234025515 shutdown_connections 2024-04-25T10:41:18.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.237+0000 7f01fdaaa640 1 -- 172.21.15.43:0/1234025515 wait complete. 2024-04-25T10:41:18.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.238+0000 7f01fdaaa640 1 Processor -- start 2024-04-25T10:41:18.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.238+0000 7f01fdaaa640 1 -- start start 2024-04-25T10:41:18.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.238+0000 7f01fdaaa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f8102af0 0x7f01f80ff250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.238+0000 7f01fdaaa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01f8108a20 0x7f01f80ff790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.238+0000 7f01fdaaa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01f810d380 0x7f01f80ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.238+0000 7f01fdaaa640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f01f8112c60 con 0x7f01f8102af0 2024-04-25T10:41:18.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.238+0000 7f01fdaaa640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f01f8112ae0 con 0x7f01f810d380 2024-04-25T10:41:18.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.238+0000 7f01fdaaa640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f01f8112de0 con 0x7f01f8108a20 2024-04-25T10:41:18.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.238+0000 7f01f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f8102af0 0x7f01f80ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.238+0000 7f01f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f8102af0 0x7f01f80ff250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48220/0 (socket says 172.21.15.43:48220) 2024-04-25T10:41:18.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.238+0000 7f01f77fe640 1 -- 172.21.15.43:0/3418044166 learned_addr learned my addr 172.21.15.43:0/3418044166 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:18.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.238+0000 7f01f6ffd640 1 --2- 172.21.15.43:0/3418044166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01f8108a20 0x7f01f80ff790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.239+0000 7f01f7fff640 1 --2- 172.21.15.43:0/3418044166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01f810d380 0x7f01f80ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.239+0000 7f01f77fe640 1 -- 172.21.15.43:0/3418044166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01f8108a20 msgr2=0x7f01f80ff790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.239+0000 7f01f77fe640 1 --2- 172.21.15.43:0/3418044166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01f8108a20 0x7f01f80ff790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.239+0000 7f01f77fe640 1 -- 172.21.15.43:0/3418044166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01f810d380 msgr2=0x7f01f80ffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.239+0000 7f01f77fe640 1 --2- 172.21.15.43:0/3418044166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01f810d380 0x7f01f80ffcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.239+0000 7f01f77fe640 1 -- 172.21.15.43:0/3418044166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f81a8a70 con 0x7f01f8102af0 2024-04-25T10:41:18.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.239+0000 7f01f77fe640 1 --2- 172.21.15.43:0/3418044166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f8102af0 0x7f01f80ff250 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f01e800b450 tx=0x7f01e800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:18.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.239+0000 7f01f4ff9640 1 -- 172.21.15.43:0/3418044166 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8015020 con 0x7f01f8102af0 2024-04-25T10:41:18.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.239+0000 7f01fdaaa640 1 -- 172.21.15.43:0/3418044166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f81a8d60 con 0x7f01f8102af0 2024-04-25T10:41:18.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.239+0000 7f01f4ff9640 1 -- 172.21.15.43:0/3418044166 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01e8011d40 con 0x7f01f8102af0 2024-04-25T10:41:18.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.239+0000 7f01f4ff9640 1 -- 172.21.15.43:0/3418044166 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e800f340 con 0x7f01f8102af0 2024-04-25T10:41:18.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.239+0000 7f01fdaaa640 1 -- 172.21.15.43:0/3418044166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f81a91f0 con 0x7f01f8102af0 2024-04-25T10:41:18.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.242+0000 7f0204a34740 1 Processor -- start 2024-04-25T10:41:18.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.242+0000 7f0204a34740 1 -- start start 2024-04-25T10:41:18.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.242+0000 7f01f4ff9640 1 -- 172.21.15.43:0/3418044166 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f01e8018020 con 0x7f01f8102af0 2024-04-25T10:41:18.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.242+0000 7f0204a34740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d71a390 0x55c22dabaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.242+0000 7f0204a34740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d6f14d0 0x55c22dabafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.242+0000 7f0204a34740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 0x55c22dac78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.242+0000 7f0204a34740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c22da20d80 con 0x55c22d6f96b0 2024-04-25T10:41:18.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.242+0000 7f0204a34740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c22d9da8b0 con 0x55c22d71a390 2024-04-25T10:41:18.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.242+0000 7f0204a34740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c22da1cd40 con 0x55c22d6f14d0 2024-04-25T10:41:18.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.242+0000 7f01f6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 0x55c22dac78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.242+0000 7f01f6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 0x55c22dac78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48226/0 (socket says 172.21.15.43:48226) 2024-04-25T10:41:18.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.242+0000 7f01f6ffd640 1 -- 172.21.15.43:0/3083548793 learned_addr learned my addr 172.21.15.43:0/3083548793 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:18.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.242+0000 7f01f7fff640 1 --2- 172.21.15.43:0/3083548793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d71a390 0x55c22dabaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.242+0000 7f01f77fe640 1 --2- 172.21.15.43:0/3083548793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d6f14d0 0x55c22dabafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.243+0000 7f01f6ffd640 1 -- 172.21.15.43:0/3083548793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d6f14d0 msgr2=0x55c22dabafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.243+0000 7f01f6ffd640 1 --2- 172.21.15.43:0/3083548793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d6f14d0 0x55c22dabafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.243+0000 7f01f6ffd640 1 -- 172.21.15.43:0/3083548793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d71a390 msgr2=0x55c22dabaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.243+0000 7f01f6ffd640 1 --2- 172.21.15.43:0/3083548793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d71a390 0x55c22dabaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.243+0000 7f01f6ffd640 1 -- 172.21.15.43:0/3083548793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c22d9aba40 con 0x55c22d6f96b0 2024-04-25T10:41:18.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.243+0000 7f01f7fff640 1 --2- 172.21.15.43:0/3083548793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d71a390 0x55c22dabaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:18.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.243+0000 7f01f4ff9640 1 --2- 172.21.15.43:0/3418044166 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f01e00afc60 0x7f01e00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.243+0000 7f01f6ffd640 1 --2- 172.21.15.43:0/3083548793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 0x55c22dac78f0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00cb60 tx=0x7f01ec00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad11b72917dbab91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:18.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.243+0000 7f01f7fff640 1 --2- 172.21.15.43:0/3418044166 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f01e00afc60 0x7f01e00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.243+0000 7f01f4ff9640 1 -- 172.21.15.43:0/3418044166 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f01e80dea70 con 0x7f01f8102af0 2024-04-25T10:41:18.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.243+0000 7f01ddffb640 1 -- 172.21.15.43:0/3083548793 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00ce60 con 0x55c22d6f96b0 2024-04-25T10:41:18.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.243+0000 7f01ddffb640 1 -- 172.21.15.43:0/3083548793 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01ec0047a0 con 0x55c22d6f96b0 2024-04-25T10:41:18.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.243+0000 7f01ddffb640 1 -- 172.21.15.43:0/3083548793 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec010690 con 0x55c22d6f96b0 2024-04-25T10:41:18.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.244+0000 7f0204a34740 1 -- 172.21.15.43:0/3083548793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 msgr2=0x55c22dac78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.244+0000 7f01f7fff640 1 --2- 172.21.15.43:0/3418044166 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f01e00afc60 0x7f01e00b2170 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55c22dab7bd0 tx=0x7f01f002f3c0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:18.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.244+0000 7f0204a34740 1 --2- 172.21.15.43:0/3083548793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 0x55c22dac78f0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00cb60 tx=0x7f01ec00e720 comp rx=0 tx=0).stop 2024-04-25T10:41:18.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.244+0000 7f0204a34740 1 -- 172.21.15.43:0/3083548793 shutdown_connections 2024-04-25T10:41:18.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.244+0000 7f0204a34740 1 --2- 172.21.15.43:0/3083548793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 0x55c22dac78f0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.244+0000 7f0204a34740 1 --2- 172.21.15.43:0/3083548793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d6f14d0 0x55c22dabafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.244+0000 7f0204a34740 1 --2- 172.21.15.43:0/3083548793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d71a390 0x55c22dabaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.244+0000 7f0204a34740 1 -- 172.21.15.43:0/3083548793 >> 172.21.15.43:0/3083548793 conn(0x55c22d70e4e0 msgr2=0x55c22dab7210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:18.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.244+0000 7f0204a34740 1 -- 172.21.15.43:0/3083548793 shutdown_connections 2024-04-25T10:41:18.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.244+0000 7f0204a34740 1 -- 172.21.15.43:0/3083548793 wait complete. 2024-04-25T10:41:18.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.244+0000 7f0204a34740 1 -- start start 2024-04-25T10:41:18.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.245+0000 7f0204a34740 1 Processor -- start 2024-04-25T10:41:18.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.245+0000 7f0204a34740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d6f14d0 0x55c22dac5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.245+0000 7f0204a34740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 0x55c22dac5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.245+0000 7f0204a34740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d71a390 0x55c22dad1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.245+0000 7f0204a34740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c22dac7fe0 con 0x55c22d6f96b0 2024-04-25T10:41:18.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.245+0000 7f0204a34740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c22d9a7d10 con 0x55c22d71a390 2024-04-25T10:41:18.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.245+0000 7f0204a34740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c22dac8160 con 0x55c22d6f14d0 2024-04-25T10:41:18.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.245+0000 7f0204a34740 10 client.0 _ll_register_callbacks cb 0x55c22d9d4210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:18.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.245+0000 7f01f6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 0x55c22dac5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.245+0000 7f01f6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 0x55c22dac5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48236/0 (socket says 172.21.15.43:48236) 2024-04-25T10:41:18.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.245+0000 7f01f6ffd640 1 -- 172.21.15.43:0/416672487 learned_addr learned my addr 172.21.15.43:0/416672487 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:18.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.246+0000 7f01f77fe640 1 --2- 172.21.15.43:0/416672487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d6f14d0 0x55c22dac5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.246+0000 7f01f7fff640 1 --2- 172.21.15.43:0/416672487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d71a390 0x55c22dad1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.246+0000 7f01f6ffd640 1 -- 172.21.15.43:0/416672487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d6f14d0 msgr2=0x55c22dac5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.246+0000 7f01f6ffd640 1 --2- 172.21.15.43:0/416672487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d6f14d0 0x55c22dac5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.246+0000 7f01f6ffd640 1 -- 172.21.15.43:0/416672487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d71a390 msgr2=0x55c22dad1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.246+0000 7f01f6ffd640 1 --2- 172.21.15.43:0/416672487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d71a390 0x55c22dad1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.246+0000 7f01f6ffd640 1 -- 172.21.15.43:0/416672487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c22d7fc7d0 con 0x55c22d6f96b0 2024-04-25T10:41:18.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.246+0000 7f01f6ffd640 1 --2- 172.21.15.43:0/416672487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 0x55c22dac5ad0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f01ec005a40 tx=0x7f01ec005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b817b1d156fc806 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:18.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.246+0000 7f01d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:18.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.246+0000 7f01d6ffd640 1 -- 172.21.15.43:0/416672487 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00ce60 con 0x55c22d6f96b0 2024-04-25T10:41:18.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.246+0000 7f0204a34740 1 -- 172.21.15.43:0/416672487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01ec0047a0 con 0x55c22d6f96b0 2024-04-25T10:41:18.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.246+0000 7f01d6ffd640 1 -- 172.21.15.43:0/416672487 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f01ec004020 con 0x55c22d6f96b0 2024-04-25T10:41:18.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.246+0000 7f01d6ffd640 1 -- 172.21.15.43:0/416672487 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f01ec004220 con 0x55c22d6f96b0 2024-04-25T10:41:18.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.246+0000 7f0204a34740 10 client.18066 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:18.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.246+0000 7f0204a34740 1 -- 172.21.15.43:0/416672487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c22d7323f0 con 0x55c22d6f96b0 2024-04-25T10:41:18.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.246+0000 7f01d6ffd640 1 -- 172.21.15.43:0/416672487 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01ec0047a0 con 0x55c22d6f96b0 2024-04-25T10:41:18.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.247+0000 7f01d6ffd640 1 -- 172.21.15.43:0/416672487 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f01ec010430 con 0x55c22d6f96b0 2024-04-25T10:41:18.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.247+0000 7f0204a34740 10 client.18066 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:18.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.247+0000 7f0204a34740 10 client.18066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:18.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.247+0000 7f0204a34740 20 client.18066 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:18.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.247+0000 7f0204a34740 1 --2- 172.21.15.43:0/416672487 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c22d6d8f60 0x55c22dad7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.247+0000 7f0204a34740 4 client.18066 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:41:18.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.247+0000 7f0204a34740 1 -- 172.21.15.43:0/416672487 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c22da2e990 con 0x55c22d6d8f60 2024-04-25T10:41:18.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.247+0000 7f01f77fe640 1 --2- 172.21.15.43:0/416672487 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c22d6d8f60 0x55c22dad7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.247+0000 7f01d6ffd640 10 client.18066 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:18.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.247+0000 7f01f77fe640 1 --2- 172.21.15.43:0/416672487 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c22d6d8f60 0x55c22dad7480 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adb4b1f22d4d0552 server_cookie=1a7f6a545c3e56c2 in_seq=0 out_seq=0 2024-04-25T10:41:18.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.248+0000 7f01d6ffd640 1 -- 172.21.15.43:0/416672487 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c22da2e990 con 0x55c22d6d8f60 2024-04-25T10:41:18.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.248+0000 7f01d6ffd640 10 client.18066 handle_command_reply: tid=0 2024-04-25T10:41:18.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.248+0000 7f0204a34740 1 client.18066 shutdown 2024-04-25T10:41:18.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.248+0000 7f0204a34740 1 -- 172.21.15.43:0/416672487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 msgr2=0x55c22dac5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.248+0000 7f0204a34740 1 --2- 172.21.15.43:0/416672487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 0x55c22dac5ad0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f01ec005a40 tx=0x7f01ec005230 comp rx=0 tx=0).stop 2024-04-25T10:41:18.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.248+0000 7f0204a34740 1 -- 172.21.15.43:0/416672487 shutdown_connections 2024-04-25T10:41:18.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.248+0000 7f0204a34740 1 --2- 172.21.15.43:0/416672487 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c22d6d8f60 0x55c22dad7480 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.248+0000 7f0204a34740 1 --2- 172.21.15.43:0/416672487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d71a390 0x55c22dad1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.248+0000 7f0204a34740 1 --2- 172.21.15.43:0/416672487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 0x55c22dac5ad0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.248+0000 7f0204a34740 1 --2- 172.21.15.43:0/416672487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d6f14d0 0x55c22dac5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.248+0000 7f01d6ffd640 0 client.18066 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:18.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.248+0000 7f0204a34740 1 -- 172.21.15.43:0/416672487 >> 172.21.15.43:0/416672487 conn(0x55c22d70e4e0 msgr2=0x55c22dabc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:18.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.249+0000 7f0204a34740 1 -- 172.21.15.43:0/416672487 shutdown_connections 2024-04-25T10:41:18.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.249+0000 7f0204a34740 1 -- 172.21.15.43:0/416672487 wait complete. 2024-04-25T10:41:18.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.249+0000 7f0204a34740 20 client.18066 trim_cache size 0 max 16384 2024-04-25T10:41:18.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.267+0000 7f0204a34740 1 Processor -- start 2024-04-25T10:41:18.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.267+0000 7f0204a34740 1 -- start start 2024-04-25T10:41:18.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.267+0000 7f0204a34740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d6f14d0 0x55c22dac0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.267+0000 7f0204a34740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d6f96b0 0x55c22dac4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.267+0000 7f0204a34740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d71a390 0x55c22dacd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.267+0000 7f0204a34740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f01f81b4530 con 0x55c22d71a390 2024-04-25T10:41:18.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.267+0000 7f0204a34740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f01f81b1e00 con 0x55c22d6f96b0 2024-04-25T10:41:18.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.267+0000 7f0204a34740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f01f81afc70 con 0x55c22d6f14d0 2024-04-25T10:41:18.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.267+0000 7f01f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d71a390 0x55c22dacd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.267+0000 7f01f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d71a390 0x55c22dacd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48240/0 (socket says 172.21.15.43:48240) 2024-04-25T10:41:18.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.267+0000 7f01f7fff640 1 -- 172.21.15.43:0/2075696722 learned_addr learned my addr 172.21.15.43:0/2075696722 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:18.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.267+0000 7f01f6ffd640 1 --2- 172.21.15.43:0/2075696722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d6f96b0 0x55c22dac4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.267+0000 7f01f77fe640 1 --2- 172.21.15.43:0/2075696722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d6f14d0 0x55c22dac0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.267+0000 7f01f7fff640 1 -- 172.21.15.43:0/2075696722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d6f14d0 msgr2=0x55c22dac0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.267+0000 7f01f7fff640 1 --2- 172.21.15.43:0/2075696722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d6f14d0 0x55c22dac0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.268+0000 7f01f7fff640 1 -- 172.21.15.43:0/2075696722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d6f96b0 msgr2=0x55c22dac4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.268+0000 7f01f7fff640 1 --2- 172.21.15.43:0/2075696722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d6f96b0 0x55c22dac4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.268+0000 7f01f7fff640 1 -- 172.21.15.43:0/2075696722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c22d6f20f0 con 0x55c22d71a390 2024-04-25T10:41:18.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.268+0000 7f01f7fff640 1 --2- 172.21.15.43:0/2075696722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d71a390 0x55c22dacd430 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009ca0 tx=0x7f01f0040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256edd610d554abc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:18.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.268+0000 7f01dd7fa640 1 -- 172.21.15.43:0/2075696722 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0031e60 con 0x55c22d71a390 2024-04-25T10:41:18.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.268+0000 7f01dd7fa640 1 -- 172.21.15.43:0/2075696722 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01f0036040 con 0x55c22d71a390 2024-04-25T10:41:18.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.268+0000 7f01dd7fa640 1 -- 172.21.15.43:0/2075696722 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0004830 con 0x55c22d71a390 2024-04-25T10:41:18.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.269+0000 7f0204a34740 1 -- 172.21.15.43:0/2075696722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d71a390 msgr2=0x55c22dacd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.269+0000 7f0204a34740 1 --2- 172.21.15.43:0/2075696722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d71a390 0x55c22dacd430 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009ca0 tx=0x7f01f0040000 comp rx=0 tx=0).stop 2024-04-25T10:41:18.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.269+0000 7f0204a34740 1 -- 172.21.15.43:0/2075696722 shutdown_connections 2024-04-25T10:41:18.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.269+0000 7f0204a34740 1 --2- 172.21.15.43:0/2075696722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d6f96b0 0x55c22dac4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.269+0000 7f0204a34740 1 --2- 172.21.15.43:0/2075696722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d71a390 0x55c22dacd430 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.269+0000 7f0204a34740 1 --2- 172.21.15.43:0/2075696722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d6f14d0 0x55c22dac0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.269+0000 7f0204a34740 1 -- 172.21.15.43:0/2075696722 >> 172.21.15.43:0/2075696722 conn(0x55c22d70e4e0 msgr2=0x55c22dabcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:18.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.269+0000 7f0204a34740 1 -- 172.21.15.43:0/2075696722 shutdown_connections 2024-04-25T10:41:18.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.269+0000 7f0204a34740 1 -- 172.21.15.43:0/2075696722 wait complete. 2024-04-25T10:41:18.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.269+0000 7f0204a34740 1 -- start start 2024-04-25T10:41:18.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.269+0000 7f0204a34740 1 Processor -- start 2024-04-25T10:41:18.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.270+0000 7f0204a34740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d6f14d0 0x55c22dacd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.270+0000 7f0204a34740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 0x55c22dac8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.270+0000 7f0204a34740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d71a390 0x55c22dac9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.270+0000 7f0204a34740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f01f81b02c0 con 0x55c22d6f96b0 2024-04-25T10:41:18.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.270+0000 7f0204a34740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f01f81b1730 con 0x55c22d6f14d0 2024-04-25T10:41:18.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.270+0000 7f0204a34740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f01f81b0990 con 0x55c22d71a390 2024-04-25T10:41:18.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.270+0000 7f0204a34740 10 client.0 _ll_register_callbacks cb 0x7f01ec014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:18.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.270+0000 7f01f6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 0x55c22dac8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.270+0000 7f01f6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 0x55c22dac8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48242/0 (socket says 172.21.15.43:48242) 2024-04-25T10:41:18.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.270+0000 7f01f6ffd640 1 -- 172.21.15.43:0/1873362483 learned_addr learned my addr 172.21.15.43:0/1873362483 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:18.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.270+0000 7f01f77fe640 1 --2- 172.21.15.43:0/1873362483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d6f14d0 0x55c22dacd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.270+0000 7f01f7fff640 1 --2- 172.21.15.43:0/1873362483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d71a390 0x55c22dac9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.270+0000 7f01f6ffd640 1 -- 172.21.15.43:0/1873362483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d71a390 msgr2=0x55c22dac9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.270+0000 7f01f6ffd640 1 --2- 172.21.15.43:0/1873362483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d71a390 0x55c22dac9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.270+0000 7f01f6ffd640 1 -- 172.21.15.43:0/1873362483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d6f14d0 msgr2=0x55c22dacd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.270+0000 7f01f6ffd640 1 --2- 172.21.15.43:0/1873362483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d6f14d0 0x55c22dacd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.270+0000 7f01f6ffd640 1 -- 172.21.15.43:0/1873362483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c22d9e14b0 con 0x55c22d6f96b0 2024-04-25T10:41:18.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.270+0000 7f01f7fff640 1 --2- 172.21.15.43:0/1873362483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d71a390 0x55c22dac9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:41:18.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.270+0000 7f01f6ffd640 1 --2- 172.21.15.43:0/1873362483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 0x55c22dac8ae0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f01ec005a10 tx=0x7f01ec0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f50a9d3343a6e8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:18.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.270+0000 7f01d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:18.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.271+0000 7f01d6ffd640 1 -- 172.21.15.43:0/1873362483 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00c330 con 0x55c22d6f96b0 2024-04-25T10:41:18.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.271+0000 7f01d6ffd640 1 -- 172.21.15.43:0/1873362483 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f01ec004d40 con 0x55c22d6f96b0 2024-04-25T10:41:18.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.271+0000 7f0204a34740 1 -- 172.21.15.43:0/1873362483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01f0036040 con 0x55c22d6f96b0 2024-04-25T10:41:18.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.271+0000 7f01d6ffd640 1 -- 172.21.15.43:0/1873362483 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f01ec0055e0 con 0x55c22d6f96b0 2024-04-25T10:41:18.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.271+0000 7f0204a34740 10 client.18072 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:18.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.271+0000 7f0204a34740 1 -- 172.21.15.43:0/1873362483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c22db3f020 con 0x55c22d6f96b0 2024-04-25T10:41:18.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.271+0000 7f01d6ffd640 1 -- 172.21.15.43:0/1873362483 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01f0036040 con 0x55c22d6f96b0 2024-04-25T10:41:18.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.271+0000 7f01d6ffd640 1 -- 172.21.15.43:0/1873362483 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f01ec02daa0 con 0x55c22d6f96b0 2024-04-25T10:41:18.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.271+0000 7f0204a34740 10 client.18072 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:18.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.271+0000 7f0204a34740 10 client.18072 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:18.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.271+0000 7f0204a34740 20 client.18072 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:18.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.271+0000 7f0204a34740 1 --2- 172.21.15.43:0/1873362483 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c22d6ae3a0 0x55c22db416c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.271+0000 7f0204a34740 4 client.18072 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:41:18.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.271+0000 7f0204a34740 1 -- 172.21.15.43:0/1873362483 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c22da28800 con 0x55c22d6ae3a0 2024-04-25T10:41:18.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.271+0000 7f01f77fe640 1 --2- 172.21.15.43:0/1873362483 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c22d6ae3a0 0x55c22db416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.272+0000 7f01d6ffd640 10 client.18072 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:18.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.272+0000 7f01f77fe640 1 --2- 172.21.15.43:0/1873362483 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c22d6ae3a0 0x55c22db416c0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e835878e83fdbb4 server_cookie=83ad3be7b5bfaa9 in_seq=0 out_seq=0 2024-04-25T10:41:18.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.272+0000 7f01d6ffd640 1 -- 172.21.15.43:0/1873362483 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c22da28800 con 0x55c22d6ae3a0 2024-04-25T10:41:18.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.272+0000 7f01d6ffd640 10 client.18072 handle_command_reply: tid=0 2024-04-25T10:41:18.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.272+0000 7f0204a34740 1 client.18072 shutdown 2024-04-25T10:41:18.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.272+0000 7f0204a34740 1 -- 172.21.15.43:0/1873362483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 msgr2=0x55c22dac8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.272+0000 7f0204a34740 1 --2- 172.21.15.43:0/1873362483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 0x55c22dac8ae0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f01ec005a10 tx=0x7f01ec0045f0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.272+0000 7f0204a34740 1 -- 172.21.15.43:0/1873362483 shutdown_connections 2024-04-25T10:41:18.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.272+0000 7f0204a34740 1 --2- 172.21.15.43:0/1873362483 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55c22d6ae3a0 0x55c22db416c0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.272+0000 7f0204a34740 1 --2- 172.21.15.43:0/1873362483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22d71a390 0x55c22dac9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.272+0000 7f0204a34740 1 --2- 172.21.15.43:0/1873362483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c22d6f96b0 0x55c22dac8ae0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.272+0000 7f01d6ffd640 0 client.18072 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:18.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.272+0000 7f0204a34740 1 --2- 172.21.15.43:0/1873362483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c22d6f14d0 0x55c22dacd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.272+0000 7f0204a34740 1 -- 172.21.15.43:0/1873362483 >> 172.21.15.43:0/1873362483 conn(0x55c22d70e4e0 msgr2=0x55c22dabcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:18.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.273+0000 7f0204a34740 1 -- 172.21.15.43:0/1873362483 shutdown_connections 2024-04-25T10:41:18.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.273+0000 7f0204a34740 1 -- 172.21.15.43:0/1873362483 wait complete. 2024-04-25T10:41:18.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.273+0000 7f0204a34740 20 client.18072 trim_cache size 0 max 16384 2024-04-25T10:41:18.285 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:41:18.285 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:41:18.285 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:41:18.285 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:41:18.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.273+0000 7f01fdaaa640 1 -- 172.21.15.43:0/3418044166 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f01e00afc60 msgr2=0x7f01e00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.273+0000 7f01fdaaa640 1 --2- 172.21.15.43:0/3418044166 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f01e00afc60 0x7f01e00b2170 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55c22dab7bd0 tx=0x7f01f002f3c0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.273+0000 7f01fdaaa640 1 -- 172.21.15.43:0/3418044166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f8102af0 msgr2=0x7f01f80ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.273+0000 7f01fdaaa640 1 --2- 172.21.15.43:0/3418044166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f8102af0 0x7f01f80ff250 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f01e800b450 tx=0x7f01e800b910 comp rx=0 tx=0).stop 2024-04-25T10:41:18.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.273+0000 7f01fdaaa640 1 -- 172.21.15.43:0/3418044166 shutdown_connections 2024-04-25T10:41:18.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.273+0000 7f01fdaaa640 1 --2- 172.21.15.43:0/3418044166 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f01e00afc60 0x7f01e00b2170 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.273+0000 7f01fdaaa640 1 --2- 172.21.15.43:0/3418044166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01f810d380 0x7f01f80ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.273+0000 7f01fdaaa640 1 --2- 172.21.15.43:0/3418044166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01f8108a20 0x7f01f80ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.273+0000 7f01fdaaa640 1 --2- 172.21.15.43:0/3418044166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f8102af0 0x7f01f80ff250 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.273+0000 7f01fdaaa640 1 -- 172.21.15.43:0/3418044166 >> 172.21.15.43:0/3418044166 conn(0x7f01f8078100 msgr2=0x7f01f8069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:18.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.273+0000 7f01fdaaa640 1 -- 172.21.15.43:0/3418044166 shutdown_connections 2024-04-25T10:41:18.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.273+0000 7f01fdaaa640 1 -- 172.21.15.43:0/3418044166 wait complete. 2024-04-25T10:41:18.290 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:17 smithi043 ceph-mon[29478]: pgmap v272: 97 pgs: 97 active+clean; 3.5 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-04-25T10:41:18.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:17 smithi077 ceph-mon[32772]: pgmap v272: 97 pgs: 97 active+clean; 3.5 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-04-25T10:41:18.291 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:41:18.291 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:41:18.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:17 smithi164 ceph-mon[32713]: pgmap v272: 97 pgs: 97 active+clean; 3.5 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-04-25T10:41:18.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.401+0000 7f457f7a8640 1 -- 172.21.15.43:0/3588875414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f457810d3a0 msgr2=0x7f457810f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.401+0000 7f457f7a8640 1 --2- 172.21.15.43:0/3588875414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f457810d3a0 0x7f457810f7c0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f4570009ca0 tx=0x7f457002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.402+0000 7f457f7a8640 1 -- 172.21.15.43:0/3588875414 shutdown_connections 2024-04-25T10:41:18.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.402+0000 7f457f7a8640 1 --2- 172.21.15.43:0/3588875414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f457810d3a0 0x7f457810f7c0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.402+0000 7f457f7a8640 1 --2- 172.21.15.43:0/3588875414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4578102b60 0x7f4578102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.402+0000 7f457f7a8640 1 --2- 172.21.15.43:0/3588875414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4578108a40 0x7f4578102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.402+0000 7f457f7a8640 1 -- 172.21.15.43:0/3588875414 >> 172.21.15.43:0/3588875414 conn(0x7f4578078040 msgr2=0x7f4578078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:18.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.402+0000 7f457f7a8640 1 -- 172.21.15.43:0/3588875414 shutdown_connections 2024-04-25T10:41:18.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.402+0000 7f457f7a8640 1 -- 172.21.15.43:0/3588875414 wait complete. 2024-04-25T10:41:18.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.402+0000 7f457f7a8640 1 Processor -- start 2024-04-25T10:41:18.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.402+0000 7f457f7a8640 1 -- start start 2024-04-25T10:41:18.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f457f7a8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4578102b60 0x7f45781994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f457f7a8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4578108a40 0x7f4578199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f457f7a8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f457810d3a0 0x7f457819dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f457f7a8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4578112c00 con 0x7f4578102b60 2024-04-25T10:41:18.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f457f7a8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4578112a80 con 0x7f4578108a40 2024-04-25T10:41:18.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f457f7a8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4578112d80 con 0x7f457810d3a0 2024-04-25T10:41:18.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f457d544640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4578102b60 0x7f45781994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f457d544640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4578102b60 0x7f45781994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48258/0 (socket says 172.21.15.43:48258) 2024-04-25T10:41:18.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f457d544640 1 -- 172.21.15.43:0/957862310 learned_addr learned my addr 172.21.15.43:0/957862310 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:18.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f457cd43640 1 --2- 172.21.15.43:0/957862310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4578108a40 0x7f4578199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f457dd45640 1 --2- 172.21.15.43:0/957862310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f457810d3a0 0x7f457819dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f457d544640 1 -- 172.21.15.43:0/957862310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f457810d3a0 msgr2=0x7f457819dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f457d544640 1 --2- 172.21.15.43:0/957862310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f457810d3a0 0x7f457819dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f457d544640 1 -- 172.21.15.43:0/957862310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4578108a40 msgr2=0x7f4578199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f457d544640 1 --2- 172.21.15.43:0/957862310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4578108a40 0x7f4578199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f457d544640 1 -- 172.21.15.43:0/957862310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457819e2c0 con 0x7f4578102b60 2024-04-25T10:41:18.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f457d544640 1 --2- 172.21.15.43:0/957862310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4578102b60 0x7f45781994e0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f457400c9a0 tx=0x7f457400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:18.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f456e7fc640 1 -- 172.21.15.43:0/957862310 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4574014070 con 0x7f4578102b60 2024-04-25T10:41:18.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f456e7fc640 1 -- 172.21.15.43:0/957862310 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4574004440 con 0x7f4578102b60 2024-04-25T10:41:18.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f457f7a8640 1 -- 172.21.15.43:0/957862310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457819e5b0 con 0x7f4578102b60 2024-04-25T10:41:18.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f456e7fc640 1 -- 172.21.15.43:0/957862310 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457400f450 con 0x7f4578102b60 2024-04-25T10:41:18.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.403+0000 7f457f7a8640 1 -- 172.21.15.43:0/957862310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f457819eb00 con 0x7f4578102b60 2024-04-25T10:41:18.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.404+0000 7f457f7a8640 1 -- 172.21.15.43:0/957862310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f457806a190 con 0x7f4578102b60 2024-04-25T10:41:18.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.406+0000 7f456e7fc640 1 -- 172.21.15.43:0/957862310 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f4574017070 con 0x7f4578102b60 2024-04-25T10:41:18.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.406+0000 7f456e7fc640 1 --2- 172.21.15.43:0/957862310 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f45640afc60 0x7f45640b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.407+0000 7f456e7fc640 1 -- 172.21.15.43:0/957862310 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f45740df410 con 0x7f4578102b60 2024-04-25T10:41:18.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.407+0000 7f457cd43640 1 --2- 172.21.15.43:0/957862310 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f45640afc60 0x7f45640b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.407+0000 7f457cd43640 1 --2- 172.21.15.43:0/957862310 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f45640afc60 0x7f45640b2170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f45680058c0 tx=0x7f4568005850 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:18.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.410+0000 7f456e7fc640 1 -- 172.21.15.43:0/957862310 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f457401e030 con 0x7f4578102b60 2024-04-25T10:41:18.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.642+0000 7f457f7a8640 1 -- 172.21.15.43:0/957862310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f457819a830 con 0x7f4578102b60 2024-04-25T10:41:18.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.643+0000 7f456e7fc640 1 -- 172.21.15.43:0/957862310 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7f45740a7c50 con 0x7f4578102b60 2024-04-25T10:41:18.645 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:41:18.645 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:41:18.645 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:41:18.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.645+0000 7f457f7a8640 1 -- 172.21.15.43:0/957862310 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f45640afc60 msgr2=0x7f45640b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.645+0000 7f457f7a8640 1 --2- 172.21.15.43:0/957862310 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f45640afc60 0x7f45640b2170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f45680058c0 tx=0x7f4568005850 comp rx=0 tx=0).stop 2024-04-25T10:41:18.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.645+0000 7f457f7a8640 1 -- 172.21.15.43:0/957862310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4578102b60 msgr2=0x7f45781994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.645+0000 7f457f7a8640 1 --2- 172.21.15.43:0/957862310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4578102b60 0x7f45781994e0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f457400c9a0 tx=0x7f457400ce60 comp rx=0 tx=0).stop 2024-04-25T10:41:18.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.645+0000 7f457f7a8640 1 -- 172.21.15.43:0/957862310 shutdown_connections 2024-04-25T10:41:18.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.645+0000 7f457f7a8640 1 --2- 172.21.15.43:0/957862310 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f45640afc60 0x7f45640b2170 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.645+0000 7f457f7a8640 1 --2- 172.21.15.43:0/957862310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4578108a40 0x7f4578199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.645+0000 7f457f7a8640 1 --2- 172.21.15.43:0/957862310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f457810d3a0 0x7f457819dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.645+0000 7f457f7a8640 1 --2- 172.21.15.43:0/957862310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4578102b60 0x7f45781994e0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.645+0000 7f457f7a8640 1 -- 172.21.15.43:0/957862310 >> 172.21.15.43:0/957862310 conn(0x7f4578078040 msgr2=0x7f45781040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:18.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.645+0000 7f457f7a8640 1 -- 172.21.15.43:0/957862310 shutdown_connections 2024-04-25T10:41:18.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.645+0000 7f457f7a8640 1 -- 172.21.15.43:0/957862310 wait complete. 2024-04-25T10:41:18.659 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:41:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.772+0000 7f4028b57640 1 -- 172.21.15.43:0/1475157133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4024104260 msgr2=0x7f4024110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.772+0000 7f4028b57640 1 --2- 172.21.15.43:0/1475157133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4024104260 0x7f4024110630 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f4014009ca0 tx=0x7f401402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.772+0000 7f4028b57640 1 -- 172.21.15.43:0/1475157133 shutdown_connections 2024-04-25T10:41:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.772+0000 7f4028b57640 1 --2- 172.21.15.43:0/1475157133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4024109fb0 0x7f4024103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.772+0000 7f4028b57640 1 --2- 172.21.15.43:0/1475157133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4024104260 0x7f4024110630 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.772+0000 7f4028b57640 1 --2- 172.21.15.43:0/1475157133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4024109b10 0x7f40241036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.772+0000 7f4028b57640 1 -- 172.21.15.43:0/1475157133 >> 172.21.15.43:0/1475157133 conn(0x7f40240ff210 msgr2=0x7f4024101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.772+0000 7f4028b57640 1 -- 172.21.15.43:0/1475157133 shutdown_connections 2024-04-25T10:41:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.772+0000 7f4028b57640 1 -- 172.21.15.43:0/1475157133 wait complete. 2024-04-25T10:41:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.773+0000 7f4028b57640 1 Processor -- start 2024-04-25T10:41:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.773+0000 7f4028b57640 1 -- start start 2024-04-25T10:41:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.773+0000 7f4028b57640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4024104260 0x7f4024075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.773+0000 7f4028b57640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4024109b10 0x7f40240764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.773+0000 7f4028b57640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4024109fb0 0x7f4024076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.773+0000 7f4028b57640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4024113d00 con 0x7f4024109fb0 2024-04-25T10:41:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.773+0000 7f4028b57640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4024113b80 con 0x7f4024109b10 2024-04-25T10:41:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.773+0000 7f4028b57640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4024113e80 con 0x7f4024104260 2024-04-25T10:41:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.773+0000 7f4022d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4024109fb0 0x7f4024076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.773+0000 7f4022d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4024109fb0 0x7f4024076a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48718/0 (socket says 172.21.15.43:48718) 2024-04-25T10:41:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.773+0000 7f4022d9d640 1 -- 172.21.15.43:0/3080966133 learned_addr learned my addr 172.21.15.43:0/3080966133 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.773+0000 7f402259c640 1 --2- 172.21.15.43:0/3080966133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4024104260 0x7f4024075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.773+0000 7f4021d9b640 1 --2- 172.21.15.43:0/3080966133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4024109b10 0x7f40240764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.774+0000 7f4022d9d640 1 -- 172.21.15.43:0/3080966133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4024104260 msgr2=0x7f4024075fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.774+0000 7f4022d9d640 1 --2- 172.21.15.43:0/3080966133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4024104260 0x7f4024075fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.774+0000 7f4022d9d640 1 -- 172.21.15.43:0/3080966133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4024109b10 msgr2=0x7f40240764f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.774+0000 7f4022d9d640 1 --2- 172.21.15.43:0/3080966133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4024109b10 0x7f40240764f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.774+0000 7f4022d9d640 1 -- 172.21.15.43:0/3080966133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40241a8a40 con 0x7f4024109fb0 2024-04-25T10:41:18.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.774+0000 7f4021d9b640 1 --2- 172.21.15.43:0/3080966133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4024109b10 0x7f40240764f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:41:18.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.774+0000 7f402259c640 1 --2- 172.21.15.43:0/3080966133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4024104260 0x7f4024075fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:18.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.774+0000 7f4022d9d640 1 --2- 172.21.15.43:0/3080966133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4024109fb0 0x7f4024076a30 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f40140098d0 tx=0x7f40140317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:18.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.774+0000 7f40137fe640 1 -- 172.21.15.43:0/3080966133 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014003b70 con 0x7f4024109fb0 2024-04-25T10:41:18.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.774+0000 7f4028b57640 1 -- 172.21.15.43:0/3080966133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40241a8cd0 con 0x7f4024109fb0 2024-04-25T10:41:18.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.774+0000 7f40137fe640 1 -- 172.21.15.43:0/3080966133 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4014003d10 con 0x7f4024109fb0 2024-04-25T10:41:18.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.774+0000 7f40137fe640 1 -- 172.21.15.43:0/3080966133 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401402fa00 con 0x7f4024109fb0 2024-04-25T10:41:18.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.774+0000 7f4028b57640 1 -- 172.21.15.43:0/3080966133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40241a91d0 con 0x7f4024109fb0 2024-04-25T10:41:18.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.777+0000 7f402fa08740 1 Processor -- start 2024-04-25T10:41:18.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.777+0000 7f40137fe640 1 -- 172.21.15.43:0/3080966133 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f401402fbc0 con 0x7f4024109fb0 2024-04-25T10:41:18.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.777+0000 7f402fa08740 1 -- start start 2024-04-25T10:41:18.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.778+0000 7f402fa08740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562481120390 0x5624814c1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.778+0000 7f402fa08740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624810f74d0 0x5624814c1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.778+0000 7f402fa08740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810ff6b0 0x5624814ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.778+0000 7f402fa08740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562481426d80 con 0x5624810ff6b0 2024-04-25T10:41:18.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.778+0000 7f402fa08740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5624813e0a20 con 0x562481120390 2024-04-25T10:41:18.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.778+0000 7f402fa08740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562481422d40 con 0x5624810f74d0 2024-04-25T10:41:18.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.778+0000 7f4021d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810ff6b0 0x5624814ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.778+0000 7f4021d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810ff6b0 0x5624814ce6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48734/0 (socket says 172.21.15.43:48734) 2024-04-25T10:41:18.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.778+0000 7f4021d9b640 1 -- 172.21.15.43:0/3905334406 learned_addr learned my addr 172.21.15.43:0/3905334406 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:18.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.778+0000 7f402259c640 1 --2- 172.21.15.43:0/3905334406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624810f74d0 0x5624814c1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.778+0000 7f40137fe640 1 --2- 172.21.15.43:0/3080966133 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f400c0afc60 0x7f400c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.778+0000 7f4021d9b640 1 --2- 172.21.15.43:0/3905334406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562481120390 0x5624814c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.778+0000 7f4022d9d640 1 --2- 172.21.15.43:0/3080966133 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f400c0afc60 0x7f400c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.778+0000 7f4021d9b640 1 -- 172.21.15.43:0/3905334406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624810f74d0 msgr2=0x5624814c1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.778+0000 7f4021d9b640 1 --2- 172.21.15.43:0/3905334406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624810f74d0 0x5624814c1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.778+0000 7f4021d9b640 1 -- 172.21.15.43:0/3905334406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562481120390 msgr2=0x5624814c1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.778+0000 7f4021d9b640 1 --2- 172.21.15.43:0/3905334406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562481120390 0x5624814c1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.778+0000 7f4021d9b640 1 -- 172.21.15.43:0/3905334406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624813ebc30 con 0x5624810ff6b0 2024-04-25T10:41:18.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.778+0000 7f40137fe640 1 -- 172.21.15.43:0/3080966133 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f4014100520 con 0x7f4024109fb0 2024-04-25T10:41:18.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.779+0000 7f4021d9b640 1 --2- 172.21.15.43:0/3905334406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810ff6b0 0x5624814ce6a0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f4018005f60 tx=0x7f4018009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440503fa3a7d60a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:18.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.779+0000 7f4010ff9640 1 -- 172.21.15.43:0/3905334406 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018019030 con 0x5624810ff6b0 2024-04-25T10:41:18.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.779+0000 7f4022d9d640 1 --2- 172.21.15.43:0/3080966133 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f400c0afc60 0x7f400c0b2170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f40140fde30 tx=0x7f4014096030 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:18.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.779+0000 7f4010ff9640 1 -- 172.21.15.43:0/3905334406 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40180054d0 con 0x5624810ff6b0 2024-04-25T10:41:18.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.779+0000 7f4010ff9640 1 -- 172.21.15.43:0/3905334406 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018011490 con 0x5624810ff6b0 2024-04-25T10:41:18.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.780+0000 7f402fa08740 1 -- 172.21.15.43:0/3905334406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810ff6b0 msgr2=0x5624814ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.780+0000 7f402fa08740 1 --2- 172.21.15.43:0/3905334406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810ff6b0 0x5624814ce6a0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f4018005f60 tx=0x7f4018009680 comp rx=0 tx=0).stop 2024-04-25T10:41:18.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.780+0000 7f402fa08740 1 -- 172.21.15.43:0/3905334406 shutdown_connections 2024-04-25T10:41:18.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.780+0000 7f402fa08740 1 --2- 172.21.15.43:0/3905334406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810ff6b0 0x5624814ce6a0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.780+0000 7f402fa08740 1 --2- 172.21.15.43:0/3905334406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624810f74d0 0x5624814c1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.780+0000 7f402fa08740 1 --2- 172.21.15.43:0/3905334406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562481120390 0x5624814c1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.780+0000 7f402fa08740 1 -- 172.21.15.43:0/3905334406 >> 172.21.15.43:0/3905334406 conn(0x5624811144e0 msgr2=0x5624814bdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:18.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.780+0000 7f402fa08740 1 -- 172.21.15.43:0/3905334406 shutdown_connections 2024-04-25T10:41:18.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.780+0000 7f402fa08740 1 -- 172.21.15.43:0/3905334406 wait complete. 2024-04-25T10:41:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.780+0000 7f402fa08740 1 -- start start 2024-04-25T10:41:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.780+0000 7f402fa08740 1 Processor -- start 2024-04-25T10:41:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f402fa08740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624810f74d0 0x5624814cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f402fa08740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810ff6b0 0x5624814cc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f402fa08740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562481120390 0x5624814d8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f402fa08740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5624814ced90 con 0x5624810ff6b0 2024-04-25T10:41:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f402fa08740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5624813addc0 con 0x562481120390 2024-04-25T10:41:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f402fa08740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5624814cef10 con 0x5624810f74d0 2024-04-25T10:41:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f402fa08740 10 client.0 _ll_register_callbacks cb 0x5624813c9600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f402259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810ff6b0 0x5624814cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f402259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810ff6b0 0x5624814cc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48748/0 (socket says 172.21.15.43:48748) 2024-04-25T10:41:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f402259c640 1 -- 172.21.15.43:0/2319574680 learned_addr learned my addr 172.21.15.43:0/2319574680 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f4021d9b640 1 --2- 172.21.15.43:0/2319574680 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562481120390 0x5624814d8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f4021d9b640 1 --2- 172.21.15.43:0/2319574680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624810f74d0 0x5624814cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f402259c640 1 -- 172.21.15.43:0/2319574680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624810f74d0 msgr2=0x5624814cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f402259c640 1 --2- 172.21.15.43:0/2319574680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624810f74d0 0x5624814cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f402259c640 1 -- 172.21.15.43:0/2319574680 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562481120390 msgr2=0x5624814d8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f402259c640 1 --2- 172.21.15.43:0/2319574680 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562481120390 0x5624814d8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f402259c640 1 -- 172.21.15.43:0/2319574680 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624811383f0 con 0x5624810ff6b0 2024-04-25T10:41:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f4021d9b640 1 --2- 172.21.15.43:0/2319574680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624810f74d0 0x5624814cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:41:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f402259c640 1 --2- 172.21.15.43:0/2319574680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810ff6b0 0x5624814cc800 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f401c00bb10 tx=0x7f401c0044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f7e932c400e1fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f4009ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f4009ffb640 1 -- 172.21.15.43:0/2319574680 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c01b640 con 0x5624810ff6b0 2024-04-25T10:41:18.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f4009ffb640 1 -- 172.21.15.43:0/2319574680 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f401c01b7e0 con 0x5624810ff6b0 2024-04-25T10:41:18.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.781+0000 7f402fa08740 1 -- 172.21.15.43:0/2319574680 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40180054d0 con 0x5624810ff6b0 2024-04-25T10:41:18.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.782+0000 7f4009ffb640 1 -- 172.21.15.43:0/2319574680 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f401c01ba20 con 0x5624810ff6b0 2024-04-25T10:41:18.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.782+0000 7f402fa08740 10 client.18090 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:18.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.782+0000 7f402fa08740 1 -- 172.21.15.43:0/2319574680 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5624812027d0 con 0x5624810ff6b0 2024-04-25T10:41:18.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.782+0000 7f4009ffb640 1 -- 172.21.15.43:0/2319574680 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40180054d0 con 0x5624810ff6b0 2024-04-25T10:41:18.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.782+0000 7f4009ffb640 1 -- 172.21.15.43:0/2319574680 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f401c019020 con 0x5624810ff6b0 2024-04-25T10:41:18.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.782+0000 7f402fa08740 10 client.18090 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:18.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.782+0000 7f402fa08740 10 client.18090 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:41:18.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.782+0000 7f402fa08740 20 client.18090 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:18.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.782+0000 7f402fa08740 1 --2- 172.21.15.43:0/2319574680 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x5624810def60 0x5624814de800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.782+0000 7f402fa08740 4 client.18090 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:41:18.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.782+0000 7f402fa08740 1 -- 172.21.15.43:0/2319574680 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562481434990 con 0x5624810def60 2024-04-25T10:41:18.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.782+0000 7f4021d9b640 1 --2- 172.21.15.43:0/2319574680 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x5624810def60 0x5624814de800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.783+0000 7f4021d9b640 1 --2- 172.21.15.43:0/2319574680 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x5624810def60 0x5624814de800 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=13770e9e8ce5aaef server_cookie=fd590d194d75abe1 in_seq=0 out_seq=0 2024-04-25T10:41:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.783+0000 7f4009ffb640 10 client.18090 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:41:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.783+0000 7f4009ffb640 1 -- 172.21.15.43:0/2319574680 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562481434990 con 0x5624810def60 2024-04-25T10:41:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.783+0000 7f4009ffb640 10 client.18090 handle_command_reply: tid=0 2024-04-25T10:41:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.784+0000 7f402fa08740 1 client.18090 shutdown 2024-04-25T10:41:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.784+0000 7f402fa08740 1 -- 172.21.15.43:0/2319574680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810ff6b0 msgr2=0x5624814cc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.784+0000 7f402fa08740 1 --2- 172.21.15.43:0/2319574680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810ff6b0 0x5624814cc800 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f401c00bb10 tx=0x7f401c0044f0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.784+0000 7f402fa08740 1 -- 172.21.15.43:0/2319574680 shutdown_connections 2024-04-25T10:41:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.784+0000 7f402fa08740 1 --2- 172.21.15.43:0/2319574680 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x5624810def60 0x5624814de800 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.784+0000 7f402fa08740 1 --2- 172.21.15.43:0/2319574680 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562481120390 0x5624814d8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.784+0000 7f4009ffb640 0 client.18090 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:41:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.784+0000 7f402fa08740 1 --2- 172.21.15.43:0/2319574680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810ff6b0 0x5624814cc800 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.784+0000 7f402fa08740 1 --2- 172.21.15.43:0/2319574680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624810f74d0 0x5624814cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.784+0000 7f402fa08740 1 -- 172.21.15.43:0/2319574680 >> 172.21.15.43:0/2319574680 conn(0x5624811144e0 msgr2=0x5624814c3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:18.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.784+0000 7f402fa08740 1 -- 172.21.15.43:0/2319574680 shutdown_connections 2024-04-25T10:41:18.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.784+0000 7f402fa08740 1 -- 172.21.15.43:0/2319574680 wait complete. 2024-04-25T10:41:18.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.784+0000 7f402fa08740 20 client.18090 trim_cache size 0 max 16384 2024-04-25T10:41:18.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.797+0000 7f402fa08740 1 Processor -- start 2024-04-25T10:41:18.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.797+0000 7f402fa08740 1 -- start start 2024-04-25T10:41:18.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.798+0000 7f402fa08740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810f74d0 0x5624814bdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.798+0000 7f402fa08740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624810ff6b0 0x5624814d7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.798+0000 7f402fa08740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562481120390 0x5624814c6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.798+0000 7f402fa08740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f40241b43c0 con 0x5624810f74d0 2024-04-25T10:41:18.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.798+0000 7f402fa08740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f40241b1c90 con 0x5624810ff6b0 2024-04-25T10:41:18.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.798+0000 7f402fa08740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f40241afb00 con 0x562481120390 2024-04-25T10:41:18.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.798+0000 7f4021d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810f74d0 0x5624814bdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.798+0000 7f4021d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810f74d0 0x5624814bdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48762/0 (socket says 172.21.15.43:48762) 2024-04-25T10:41:18.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.798+0000 7f4021d9b640 1 -- 172.21.15.43:0/268827491 learned_addr learned my addr 172.21.15.43:0/268827491 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:18.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.798+0000 7f4021d9b640 1 -- 172.21.15.43:0/268827491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562481120390 msgr2=0x5624814c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.798+0000 7f4021d9b640 1 --2- 172.21.15.43:0/268827491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562481120390 0x5624814c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.798+0000 7f4021d9b640 1 -- 172.21.15.43:0/268827491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624810ff6b0 msgr2=0x5624814d7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.798+0000 7f4021d9b640 1 --2- 172.21.15.43:0/268827491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624810ff6b0 0x5624814d7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.798+0000 7f4021d9b640 1 -- 172.21.15.43:0/268827491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624814c7c70 con 0x5624810f74d0 2024-04-25T10:41:18.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.798+0000 7f4021d9b640 1 --2- 172.21.15.43:0/268827491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810f74d0 0x5624814bdf70 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f4018005f60 tx=0x7f4018009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1017829caea228e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:18.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.798+0000 7f400bfff640 1 -- 172.21.15.43:0/268827491 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018019040 con 0x5624810f74d0 2024-04-25T10:41:18.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.798+0000 7f400bfff640 1 -- 172.21.15.43:0/268827491 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f401801a040 con 0x5624810f74d0 2024-04-25T10:41:18.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.798+0000 7f400bfff640 1 -- 172.21.15.43:0/268827491 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401801e990 con 0x5624810f74d0 2024-04-25T10:41:18.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.799+0000 7f402fa08740 1 -- 172.21.15.43:0/268827491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810f74d0 msgr2=0x5624814bdf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.799+0000 7f402fa08740 1 --2- 172.21.15.43:0/268827491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810f74d0 0x5624814bdf70 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f4018005f60 tx=0x7f4018009680 comp rx=0 tx=0).stop 2024-04-25T10:41:18.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.799+0000 7f402fa08740 1 -- 172.21.15.43:0/268827491 shutdown_connections 2024-04-25T10:41:18.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.799+0000 7f402fa08740 1 --2- 172.21.15.43:0/268827491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624810ff6b0 0x5624814d7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.799+0000 7f402fa08740 1 --2- 172.21.15.43:0/268827491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562481120390 0x5624814c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.799+0000 7f402fa08740 1 --2- 172.21.15.43:0/268827491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810f74d0 0x5624814bdf70 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.799+0000 7f402fa08740 1 -- 172.21.15.43:0/268827491 >> 172.21.15.43:0/268827491 conn(0x5624811144e0 msgr2=0x5624814bd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:18.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.799+0000 7f402fa08740 1 -- 172.21.15.43:0/268827491 shutdown_connections 2024-04-25T10:41:18.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.799+0000 7f402fa08740 1 -- 172.21.15.43:0/268827491 wait complete. 2024-04-25T10:41:18.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.800+0000 7f402fa08740 1 -- start start 2024-04-25T10:41:18.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.800+0000 7f402fa08740 1 Processor -- start 2024-04-25T10:41:18.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.800+0000 7f402fa08740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810f74d0 0x5624814d4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.800+0000 7f402fa08740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624810ff6b0 0x5624814cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.800+0000 7f402fa08740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562481120390 0x5624814d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.800+0000 7f402fa08740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f40241b0150 con 0x5624810f74d0 2024-04-25T10:41:18.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.800+0000 7f402fa08740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f40241b15c0 con 0x5624810ff6b0 2024-04-25T10:41:18.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.800+0000 7f402fa08740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f40241b0820 con 0x562481120390 2024-04-25T10:41:18.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.800+0000 7f402fa08740 10 client.0 _ll_register_callbacks cb 0x7f401c0043b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:18.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.800+0000 7f4021d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810f74d0 0x5624814d4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.800+0000 7f4021d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810f74d0 0x5624814d4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48768/0 (socket says 172.21.15.43:48768) 2024-04-25T10:41:18.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.800+0000 7f4021d9b640 1 -- 172.21.15.43:0/325321219 learned_addr learned my addr 172.21.15.43:0/325321219 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:18.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.800+0000 7f402259c640 1 --2- 172.21.15.43:0/325321219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624810ff6b0 0x5624814cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.800+0000 7f4021d9b640 1 --2- 172.21.15.43:0/325321219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562481120390 0x5624814d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.800+0000 7f4021d9b640 1 -- 172.21.15.43:0/325321219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562481120390 msgr2=0x5624814d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.800+0000 7f4021d9b640 1 --2- 172.21.15.43:0/325321219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562481120390 0x5624814d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.800+0000 7f4021d9b640 1 -- 172.21.15.43:0/325321219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624810ff6b0 msgr2=0x5624814cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.800+0000 7f4021d9b640 1 --2- 172.21.15.43:0/325321219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624810ff6b0 0x5624814cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.800+0000 7f4021d9b640 1 -- 172.21.15.43:0/325321219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624814d0960 con 0x5624810f74d0 2024-04-25T10:41:18.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.801+0000 7f4021d9b640 1 --2- 172.21.15.43:0/325321219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810f74d0 0x5624814d4860 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f401801e5b0 tx=0x7f401801e460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa48b3d36d0bf17f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:18.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.801+0000 7f4009ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:18.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.801+0000 7f4009ffb640 1 -- 172.21.15.43:0/325321219 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018019040 con 0x5624810f74d0 2024-04-25T10:41:18.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.801+0000 7f4009ffb640 1 -- 172.21.15.43:0/325321219 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f401802c8d0 con 0x5624810f74d0 2024-04-25T10:41:18.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.801+0000 7f402fa08740 1 -- 172.21.15.43:0/325321219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f401801a040 con 0x5624810f74d0 2024-04-25T10:41:18.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.801+0000 7f4009ffb640 1 -- 172.21.15.43:0/325321219 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f401802cad0 con 0x5624810f74d0 2024-04-25T10:41:18.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.801+0000 7f402fa08740 10 client.18096 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:18.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.801+0000 7f402fa08740 1 -- 172.21.15.43:0/325321219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562481546b30 con 0x5624810f74d0 2024-04-25T10:41:18.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.801+0000 7f4009ffb640 1 -- 172.21.15.43:0/325321219 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f401801a040 con 0x5624810f74d0 2024-04-25T10:41:18.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.801+0000 7f4009ffb640 1 -- 172.21.15.43:0/325321219 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4018011940 con 0x5624810f74d0 2024-04-25T10:41:18.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.801+0000 7f402fa08740 10 client.18096 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:18.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.801+0000 7f402fa08740 10 client.18096 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:41:18.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.801+0000 7f402fa08740 20 client.18096 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:18.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.801+0000 7f402fa08740 1 --2- 172.21.15.43:0/325321219 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x5624810b43a0 0x562481548f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.801+0000 7f402fa08740 4 client.18096 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:41:18.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.801+0000 7f402fa08740 1 -- 172.21.15.43:0/325321219 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5624814d8eb0 con 0x5624810b43a0 2024-04-25T10:41:18.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.802+0000 7f402259c640 1 --2- 172.21.15.43:0/325321219 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x5624810b43a0 0x562481548f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.802+0000 7f402259c640 1 --2- 172.21.15.43:0/325321219 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x5624810b43a0 0x562481548f90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b4b99feb3a86ec6 server_cookie=eff070f2e550c77f in_seq=0 out_seq=0 2024-04-25T10:41:18.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.802+0000 7f4009ffb640 10 client.18096 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:41:18.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.802+0000 7f4009ffb640 1 -- 172.21.15.43:0/325321219 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5624814d8eb0 con 0x5624810b43a0 2024-04-25T10:41:18.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.802+0000 7f4009ffb640 10 client.18096 handle_command_reply: tid=0 2024-04-25T10:41:18.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.802+0000 7f402fa08740 1 client.18096 shutdown 2024-04-25T10:41:18.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.802+0000 7f402fa08740 1 -- 172.21.15.43:0/325321219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810f74d0 msgr2=0x5624814d4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.802+0000 7f402fa08740 1 --2- 172.21.15.43:0/325321219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810f74d0 0x5624814d4860 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f401801e5b0 tx=0x7f401801e460 comp rx=0 tx=0).stop 2024-04-25T10:41:18.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.802+0000 7f402fa08740 1 -- 172.21.15.43:0/325321219 shutdown_connections 2024-04-25T10:41:18.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.802+0000 7f402fa08740 1 --2- 172.21.15.43:0/325321219 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x5624810b43a0 0x562481548f90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.802+0000 7f402fa08740 1 --2- 172.21.15.43:0/325321219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624810ff6b0 0x5624814cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.802+0000 7f4009ffb640 0 client.18096 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:41:18.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.802+0000 7f402fa08740 1 --2- 172.21.15.43:0/325321219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562481120390 0x5624814d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.802+0000 7f402fa08740 1 --2- 172.21.15.43:0/325321219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624810f74d0 0x5624814d4860 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.802+0000 7f402fa08740 1 -- 172.21.15.43:0/325321219 >> 172.21.15.43:0/325321219 conn(0x5624811144e0 msgr2=0x5624814ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:18.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.803+0000 7f402fa08740 1 -- 172.21.15.43:0/325321219 shutdown_connections 2024-04-25T10:41:18.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.803+0000 7f402fa08740 1 -- 172.21.15.43:0/325321219 wait complete. 2024-04-25T10:41:18.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.803+0000 7f402fa08740 20 client.18096 trim_cache size 0 max 16384 2024-04-25T10:41:18.812 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:41:18.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.803+0000 7f4028b57640 1 -- 172.21.15.43:0/3080966133 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f400c0afc60 msgr2=0x7f400c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.803+0000 7f4028b57640 1 --2- 172.21.15.43:0/3080966133 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f400c0afc60 0x7f400c0b2170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f40140fde30 tx=0x7f4014096030 comp rx=0 tx=0).stop 2024-04-25T10:41:18.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.803+0000 7f4028b57640 1 -- 172.21.15.43:0/3080966133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4024109fb0 msgr2=0x7f4024076a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.803+0000 7f4028b57640 1 --2- 172.21.15.43:0/3080966133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4024109fb0 0x7f4024076a30 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f40140098d0 tx=0x7f40140317b0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.803+0000 7f4028b57640 1 -- 172.21.15.43:0/3080966133 shutdown_connections 2024-04-25T10:41:18.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.803+0000 7f4028b57640 1 --2- 172.21.15.43:0/3080966133 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f400c0afc60 0x7f400c0b2170 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.803+0000 7f4028b57640 1 --2- 172.21.15.43:0/3080966133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4024109b10 0x7f40240764f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.803+0000 7f4028b57640 1 --2- 172.21.15.43:0/3080966133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4024109fb0 0x7f4024076a30 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.803+0000 7f4028b57640 1 --2- 172.21.15.43:0/3080966133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4024104260 0x7f4024075fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.803+0000 7f4028b57640 1 -- 172.21.15.43:0/3080966133 >> 172.21.15.43:0/3080966133 conn(0x7f40240ff210 msgr2=0x7f4024106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:18.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.803+0000 7f4028b57640 1 -- 172.21.15.43:0/3080966133 shutdown_connections 2024-04-25T10:41:18.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.803+0000 7f4028b57640 1 -- 172.21.15.43:0/3080966133 wait complete. 2024-04-25T10:41:18.816 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:41:18.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.928+0000 7fc1ed97d640 1 -- 172.21.15.43:0/1822408108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1e8102b00 msgr2=0x7fc1e8102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.929+0000 7fc1ed97d640 1 --2- 172.21.15.43:0/1822408108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1e8102b00 0x7fc1e8102f10 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0009900 tx=0x7fc1d002a410 comp rx=0 tx=0).stop 2024-04-25T10:41:18.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.929+0000 7fc1ed97d640 1 -- 172.21.15.43:0/1822408108 shutdown_connections 2024-04-25T10:41:18.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.929+0000 7fc1ed97d640 1 --2- 172.21.15.43:0/1822408108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1e810d390 0x7fc1e810f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.929+0000 7fc1ed97d640 1 --2- 172.21.15.43:0/1822408108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1e8102b00 0x7fc1e8102f10 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.929+0000 7fc1ed97d640 1 --2- 172.21.15.43:0/1822408108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1e8108a30 0x7fc1e81025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.929+0000 7fc1ed97d640 1 -- 172.21.15.43:0/1822408108 >> 172.21.15.43:0/1822408108 conn(0x7fc1e80fe260 msgr2=0x7fc1e8100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:18.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.929+0000 7fc1ed97d640 1 -- 172.21.15.43:0/1822408108 shutdown_connections 2024-04-25T10:41:18.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.929+0000 7fc1ed97d640 1 -- 172.21.15.43:0/1822408108 wait complete. 2024-04-25T10:41:18.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.929+0000 7fc1ed97d640 1 Processor -- start 2024-04-25T10:41:18.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.929+0000 7fc1ed97d640 1 -- start start 2024-04-25T10:41:18.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.930+0000 7fc1ed97d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1e8102b00 0x7fc1e80fe740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.930+0000 7fc1ed97d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1e8108a30 0x7fc1e80fec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.930+0000 7fc1ed97d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1e810d390 0x7fc1e80ff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.930+0000 7fc1ed97d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e8112bf0 con 0x7fc1e810d390 2024-04-25T10:41:18.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.930+0000 7fc1ed97d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e8112a70 con 0x7fc1e8108a30 2024-04-25T10:41:18.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.930+0000 7fc1ed97d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e8112d70 con 0x7fc1e8102b00 2024-04-25T10:41:18.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.930+0000 7fc1e77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1e810d390 0x7fc1e80ff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.930+0000 7fc1e77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1e810d390 0x7fc1e80ff1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48786/0 (socket says 172.21.15.43:48786) 2024-04-25T10:41:18.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.930+0000 7fc1e77fe640 1 -- 172.21.15.43:0/2276606877 learned_addr learned my addr 172.21.15.43:0/2276606877 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:18.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.930+0000 7fc1dffff640 1 --2- 172.21.15.43:0/2276606877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1e8108a30 0x7fc1e80fec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.930+0000 7fc1e6ffd640 1 --2- 172.21.15.43:0/2276606877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1e8102b00 0x7fc1e80fe740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.930+0000 7fc1e77fe640 1 -- 172.21.15.43:0/2276606877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1e8102b00 msgr2=0x7fc1e80fe740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.930+0000 7fc1e77fe640 1 --2- 172.21.15.43:0/2276606877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1e8102b00 0x7fc1e80fe740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.930+0000 7fc1e77fe640 1 -- 172.21.15.43:0/2276606877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1e8108a30 msgr2=0x7fc1e80fec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.930+0000 7fc1e77fe640 1 --2- 172.21.15.43:0/2276606877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1e8108a30 0x7fc1e80fec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.930+0000 7fc1e77fe640 1 -- 172.21.15.43:0/2276606877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e81a8ae0 con 0x7fc1e810d390 2024-04-25T10:41:18.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.930+0000 7fc1e6ffd640 1 --2- 172.21.15.43:0/2276606877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1e8102b00 0x7fc1e80fe740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:41:18.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.930+0000 7fc1e77fe640 1 --2- 172.21.15.43:0/2276606877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1e810d390 0x7fc1e80ff1c0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000eaa0 tx=0x7fc1e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:18.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.930+0000 7fc1ec97b640 1 -- 172.21.15.43:0/2276606877 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e000cdb0 con 0x7fc1e810d390 2024-04-25T10:41:18.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.931+0000 7fc1ec97b640 1 -- 172.21.15.43:0/2276606877 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1e0005020 con 0x7fc1e810d390 2024-04-25T10:41:18.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.931+0000 7fc1ed97d640 1 -- 172.21.15.43:0/2276606877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e81a8dd0 con 0x7fc1e810d390 2024-04-25T10:41:18.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.931+0000 7fc1ec97b640 1 -- 172.21.15.43:0/2276606877 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e00116d0 con 0x7fc1e810d390 2024-04-25T10:41:18.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.931+0000 7fc1ed97d640 1 -- 172.21.15.43:0/2276606877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e81a9210 con 0x7fc1e810d390 2024-04-25T10:41:18.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.933+0000 7fc1f482d740 1 Processor -- start 2024-04-25T10:41:18.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.933+0000 7fc1ec97b640 1 -- 172.21.15.43:0/2276606877 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fc1e00051d0 con 0x7fc1e810d390 2024-04-25T10:41:18.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.933+0000 7fc1f482d740 1 -- start start 2024-04-25T10:41:18.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.933+0000 7fc1f482d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0028f6390 0x55d002c97810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.933+0000 7fc1f482d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028cd4d0 0x55d002c97d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.933+0000 7fc1f482d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d0028d56b0 0x55d002ca46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.933+0000 7fc1f482d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d002bfcd80 con 0x55d0028cd4d0 2024-04-25T10:41:18.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.933+0000 7fc1f482d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d002bb6a20 con 0x55d0028d56b0 2024-04-25T10:41:18.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.933+0000 7fc1f482d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d002bf8d40 con 0x55d0028f6390 2024-04-25T10:41:18.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.934+0000 7fc1e6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028cd4d0 0x55d002c97d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.934+0000 7fc1e6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028cd4d0 0x55d002c97d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48802/0 (socket says 172.21.15.43:48802) 2024-04-25T10:41:18.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.934+0000 7fc1e6ffd640 1 -- 172.21.15.43:0/2559573301 learned_addr learned my addr 172.21.15.43:0/2559573301 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:18.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.934+0000 7fc1dffff640 1 --2- 172.21.15.43:0/2559573301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d0028d56b0 0x55d002ca46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.934+0000 7fc1dffff640 1 --2- 172.21.15.43:0/2559573301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0028f6390 0x55d002c97810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.934+0000 7fc1ec97b640 1 --2- 172.21.15.43:0/2276606877 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc1d40afc60 0x7fc1d40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.934+0000 7fc1e6ffd640 1 -- 172.21.15.43:0/2559573301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0028f6390 msgr2=0x55d002c97810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.934+0000 7fc1e6ffd640 1 --2- 172.21.15.43:0/2559573301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0028f6390 0x55d002c97810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.934+0000 7fc1e6ffd640 1 -- 172.21.15.43:0/2559573301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d0028d56b0 msgr2=0x55d002ca46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.934+0000 7fc1e6ffd640 1 --2- 172.21.15.43:0/2559573301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d0028d56b0 0x55d002ca46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.934+0000 7fc1e6ffd640 1 -- 172.21.15.43:0/2559573301 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d002bc1c30 con 0x55d0028cd4d0 2024-04-25T10:41:18.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.934+0000 7fc1e77fe640 1 --2- 172.21.15.43:0/2276606877 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc1d40afc60 0x7fc1d40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.934+0000 7fc1e6ffd640 1 --2- 172.21.15.43:0/2559573301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028cd4d0 0x55d002c97d50 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009c20 tx=0x7fc1d800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ccdc4c4ff336e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:18.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.934+0000 7fc1ec97b640 1 -- 172.21.15.43:0/2276606877 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fc1e00df1f0 con 0x7fc1e810d390 2024-04-25T10:41:18.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.934+0000 7fc1ddffb640 1 -- 172.21.15.43:0/2559573301 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8016070 con 0x55d0028cd4d0 2024-04-25T10:41:18.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.934+0000 7fc1ddffb640 1 -- 172.21.15.43:0/2559573301 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1d800ee70 con 0x55d0028cd4d0 2024-04-25T10:41:18.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.934+0000 7fc1ddffb640 1 -- 172.21.15.43:0/2559573301 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8010750 con 0x55d0028cd4d0 2024-04-25T10:41:18.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.935+0000 7fc1f482d740 1 -- 172.21.15.43:0/2559573301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028cd4d0 msgr2=0x55d002c97d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.935+0000 7fc1e77fe640 1 --2- 172.21.15.43:0/2276606877 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc1d40afc60 0x7fc1d40b2170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc1e00dc9f0 tx=0x7fc1e0074bf0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:18.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.935+0000 7fc1f482d740 1 --2- 172.21.15.43:0/2559573301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028cd4d0 0x55d002c97d50 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009c20 tx=0x7fc1d800e6c0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.935+0000 7fc1f482d740 1 -- 172.21.15.43:0/2559573301 shutdown_connections 2024-04-25T10:41:18.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.935+0000 7fc1f482d740 1 --2- 172.21.15.43:0/2559573301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d0028d56b0 0x55d002ca46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.935+0000 7fc1f482d740 1 --2- 172.21.15.43:0/2559573301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028cd4d0 0x55d002c97d50 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.935+0000 7fc1f482d740 1 --2- 172.21.15.43:0/2559573301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0028f6390 0x55d002c97810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.935+0000 7fc1f482d740 1 -- 172.21.15.43:0/2559573301 >> 172.21.15.43:0/2559573301 conn(0x55d0028ea4e0 msgr2=0x55d002c93fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:18.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.935+0000 7fc1f482d740 1 -- 172.21.15.43:0/2559573301 shutdown_connections 2024-04-25T10:41:18.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.935+0000 7fc1f482d740 1 -- 172.21.15.43:0/2559573301 wait complete. 2024-04-25T10:41:18.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.936+0000 7fc1f482d740 1 -- start start 2024-04-25T10:41:18.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.936+0000 7fc1f482d740 1 Processor -- start 2024-04-25T10:41:18.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.936+0000 7fc1f482d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0028cd4d0 0x55d002ca21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.936+0000 7fc1f482d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d0028d56b0 0x55d002ca2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.936+0000 7fc1f482d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028f6390 0x55d002cae210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.936+0000 7fc1f482d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d002ca4d90 con 0x55d0028f6390 2024-04-25T10:41:18.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.936+0000 7fc1f482d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d002b83dc0 con 0x55d0028d56b0 2024-04-25T10:41:18.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.936+0000 7fc1f482d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d002ca4f10 con 0x55d0028cd4d0 2024-04-25T10:41:18.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.936+0000 7fc1f482d740 10 client.0 _ll_register_callbacks cb 0x55d002b9f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:18.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.936+0000 7fc1dffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028f6390 0x55d002cae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.936+0000 7fc1dffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028f6390 0x55d002cae210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48804/0 (socket says 172.21.15.43:48804) 2024-04-25T10:41:18.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.937+0000 7fc1dffff640 1 -- 172.21.15.43:0/2295607077 learned_addr learned my addr 172.21.15.43:0/2295607077 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:18.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.937+0000 7fc1dffff640 1 --2- 172.21.15.43:0/2295607077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0028cd4d0 0x55d002ca21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.937+0000 7fc1e6ffd640 1 --2- 172.21.15.43:0/2295607077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d0028d56b0 0x55d002ca2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.937+0000 7fc1dffff640 1 -- 172.21.15.43:0/2295607077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0028cd4d0 msgr2=0x55d002ca21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.937+0000 7fc1dffff640 1 --2- 172.21.15.43:0/2295607077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0028cd4d0 0x55d002ca21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.937+0000 7fc1dffff640 1 -- 172.21.15.43:0/2295607077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d0028d56b0 msgr2=0x55d002ca2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.937+0000 7fc1dffff640 1 --2- 172.21.15.43:0/2295607077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d0028d56b0 0x55d002ca2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.937+0000 7fc1dffff640 1 -- 172.21.15.43:0/2295607077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d00290e3f0 con 0x55d0028f6390 2024-04-25T10:41:18.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.937+0000 7fc1dffff640 1 --2- 172.21.15.43:0/2295607077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028f6390 0x55d002cae210 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00096c0 tx=0x7fc1d0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b1a9eb7d505c6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:18.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.937+0000 7fc1c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:18.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.937+0000 7fc1c6ffd640 1 -- 172.21.15.43:0/2295607077 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0035040 con 0x55d0028f6390 2024-04-25T10:41:18.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.937+0000 7fc1f482d740 1 -- 172.21.15.43:0/2295607077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1d800ee70 con 0x55d0028f6390 2024-04-25T10:41:18.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.937+0000 7fc1c6ffd640 1 -- 172.21.15.43:0/2295607077 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fc1d0005020 con 0x55d0028f6390 2024-04-25T10:41:18.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.937+0000 7fc1c6ffd640 1 -- 172.21.15.43:0/2295607077 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc1d00044d0 con 0x55d0028f6390 2024-04-25T10:41:18.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.937+0000 7fc1f482d740 10 client.18108 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:18.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.937+0000 7fc1f482d740 1 -- 172.21.15.43:0/2295607077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d0029d87d0 con 0x55d0028f6390 2024-04-25T10:41:18.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.938+0000 7fc1c6ffd640 1 -- 172.21.15.43:0/2295607077 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1d800ee70 con 0x55d0028f6390 2024-04-25T10:41:18.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.938+0000 7fc1c6ffd640 1 -- 172.21.15.43:0/2295607077 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc1d00035f0 con 0x55d0028f6390 2024-04-25T10:41:18.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.938+0000 7fc1f482d740 10 client.18108 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:18.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.938+0000 7fc1f482d740 10 client.18108 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:41:18.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.938+0000 7fc1f482d740 20 client.18108 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:18.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.938+0000 7fc1f482d740 1 --2- 172.21.15.43:0/2295607077 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55d0028b4f60 0x55d002cb4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.938+0000 7fc1f482d740 4 client.18108 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:41:18.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.938+0000 7fc1f482d740 1 -- 172.21.15.43:0/2295607077 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d002c0a990 con 0x55d0028b4f60 2024-04-25T10:41:18.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.938+0000 7fc1e6ffd640 1 --2- 172.21.15.43:0/2295607077 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55d0028b4f60 0x55d002cb4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.939+0000 7fc1c6ffd640 10 client.18108 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:41:18.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.939+0000 7fc1e6ffd640 1 --2- 172.21.15.43:0/2295607077 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55d0028b4f60 0x55d002cb4520 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=257e2fabe8df33d4 server_cookie=1e582652e117edc5 in_seq=0 out_seq=0 2024-04-25T10:41:18.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.939+0000 7fc1c6ffd640 1 -- 172.21.15.43:0/2295607077 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d002c0a990 con 0x55d0028b4f60 2024-04-25T10:41:18.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.939+0000 7fc1c6ffd640 10 client.18108 handle_command_reply: tid=0 2024-04-25T10:41:18.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.939+0000 7fc1f482d740 1 client.18108 shutdown 2024-04-25T10:41:18.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.940+0000 7fc1f482d740 1 -- 172.21.15.43:0/2295607077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028f6390 msgr2=0x55d002cae210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.940+0000 7fc1f482d740 1 --2- 172.21.15.43:0/2295607077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028f6390 0x55d002cae210 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00096c0 tx=0x7fc1d0009260 comp rx=0 tx=0).stop 2024-04-25T10:41:18.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.940+0000 7fc1f482d740 1 -- 172.21.15.43:0/2295607077 shutdown_connections 2024-04-25T10:41:18.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.940+0000 7fc1f482d740 1 --2- 172.21.15.43:0/2295607077 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55d0028b4f60 0x55d002cb4520 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.940+0000 7fc1f482d740 1 --2- 172.21.15.43:0/2295607077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d0028d56b0 0x55d002ca2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.940+0000 7fc1f482d740 1 --2- 172.21.15.43:0/2295607077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028f6390 0x55d002cae210 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.940+0000 7fc1c6ffd640 0 client.18108 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:41:18.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.940+0000 7fc1f482d740 1 --2- 172.21.15.43:0/2295607077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0028cd4d0 0x55d002ca21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.940+0000 7fc1f482d740 1 -- 172.21.15.43:0/2295607077 >> 172.21.15.43:0/2295607077 conn(0x55d0028ea4e0 msgr2=0x55d002c99530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:18.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.940+0000 7fc1f482d740 1 -- 172.21.15.43:0/2295607077 shutdown_connections 2024-04-25T10:41:18.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.940+0000 7fc1f482d740 1 -- 172.21.15.43:0/2295607077 wait complete. 2024-04-25T10:41:18.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.940+0000 7fc1f482d740 20 client.18108 trim_cache size 0 max 16384 2024-04-25T10:41:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.953+0000 7fc1f482d740 1 Processor -- start 2024-04-25T10:41:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.954+0000 7fc1f482d740 1 -- start start 2024-04-25T10:41:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.954+0000 7fc1f482d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d0028cd4d0 0x55d002c93040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.954+0000 7fc1f482d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028d56b0 0x55d002cade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.954+0000 7fc1f482d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0028f6390 0x55d002ca3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.954+0000 7fc1f482d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e81b4550 con 0x55d0028d56b0 2024-04-25T10:41:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.954+0000 7fc1f482d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e81b1e20 con 0x55d0028cd4d0 2024-04-25T10:41:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.954+0000 7fc1f482d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e81afc90 con 0x55d0028f6390 2024-04-25T10:41:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.954+0000 7fc1e6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028d56b0 0x55d002cade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.954+0000 7fc1e6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028d56b0 0x55d002cade70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48820/0 (socket says 172.21.15.43:48820) 2024-04-25T10:41:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.954+0000 7fc1e6ffd640 1 -- 172.21.15.43:0/3828893518 learned_addr learned my addr 172.21.15.43:0/3828893518 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.954+0000 7fc1dffff640 1 --2- 172.21.15.43:0/3828893518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d0028cd4d0 0x55d002c93040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.954+0000 7fc1e6ffd640 1 -- 172.21.15.43:0/3828893518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0028f6390 msgr2=0x55d002ca3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.954+0000 7fc1e6ffd640 1 --2- 172.21.15.43:0/3828893518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0028f6390 0x55d002ca3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.954+0000 7fc1e6ffd640 1 -- 172.21.15.43:0/3828893518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d0028cd4d0 msgr2=0x55d002c93040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.954+0000 7fc1e6ffd640 1 --2- 172.21.15.43:0/3828893518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d0028cd4d0 0x55d002c93040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.954+0000 7fc1e6ffd640 1 -- 172.21.15.43:0/3828893518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d002c980c0 con 0x55d0028d56b0 2024-04-25T10:41:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.954+0000 7fc1e6ffd640 1 --2- 172.21.15.43:0/3828893518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028d56b0 0x55d002cade70 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fc1d800ebc0 tx=0x7fc1d8011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ec0312a5ce656 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.954+0000 7fc1dd7fa640 1 -- 172.21.15.43:0/3828893518 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8016040 con 0x55d0028d56b0 2024-04-25T10:41:18.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.955+0000 7fc1dd7fa640 1 -- 172.21.15.43:0/3828893518 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1d800ee70 con 0x55d0028d56b0 2024-04-25T10:41:18.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.955+0000 7fc1dd7fa640 1 -- 172.21.15.43:0/3828893518 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8011470 con 0x55d0028d56b0 2024-04-25T10:41:18.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.955+0000 7fc1f482d740 1 -- 172.21.15.43:0/3828893518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028d56b0 msgr2=0x55d002cade70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.955+0000 7fc1f482d740 1 --2- 172.21.15.43:0/3828893518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028d56b0 0x55d002cade70 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fc1d800ebc0 tx=0x7fc1d8011f30 comp rx=0 tx=0).stop 2024-04-25T10:41:18.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.955+0000 7fc1f482d740 1 -- 172.21.15.43:0/3828893518 shutdown_connections 2024-04-25T10:41:18.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.955+0000 7fc1f482d740 1 --2- 172.21.15.43:0/3828893518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0028f6390 0x55d002ca3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.955+0000 7fc1f482d740 1 --2- 172.21.15.43:0/3828893518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028d56b0 0x55d002cade70 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.955+0000 7fc1f482d740 1 --2- 172.21.15.43:0/3828893518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d0028cd4d0 0x55d002c93040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.955+0000 7fc1f482d740 1 -- 172.21.15.43:0/3828893518 >> 172.21.15.43:0/3828893518 conn(0x55d0028ea4e0 msgr2=0x55d002ca36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:18.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.955+0000 7fc1f482d740 1 -- 172.21.15.43:0/3828893518 shutdown_connections 2024-04-25T10:41:18.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.955+0000 7fc1f482d740 1 -- 172.21.15.43:0/3828893518 wait complete. 2024-04-25T10:41:18.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.956+0000 7fc1f482d740 1 -- start start 2024-04-25T10:41:18.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.956+0000 7fc1f482d740 1 Processor -- start 2024-04-25T10:41:18.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.956+0000 7fc1f482d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028cd4d0 0x55d002caa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.956+0000 7fc1f482d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0028d56b0 0x55d002caa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.956+0000 7fc1f482d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d0028f6390 0x55d002ca58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.956+0000 7fc1f482d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e81b02e0 con 0x55d0028cd4d0 2024-04-25T10:41:18.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.956+0000 7fc1f482d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e81b1750 con 0x55d0028f6390 2024-04-25T10:41:18.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.956+0000 7fc1f482d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e81b09b0 con 0x55d0028d56b0 2024-04-25T10:41:18.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.956+0000 7fc1f482d740 10 client.0 _ll_register_callbacks cb 0x7fc1d00316e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:18.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.956+0000 7fc1dffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028cd4d0 0x55d002caa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.956+0000 7fc1dffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028cd4d0 0x55d002caa350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48836/0 (socket says 172.21.15.43:48836) 2024-04-25T10:41:18.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.956+0000 7fc1dffff640 1 -- 172.21.15.43:0/2259256510 learned_addr learned my addr 172.21.15.43:0/2259256510 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:18.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.956+0000 7fc1e6ffd640 1 --2- 172.21.15.43:0/2259256510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0028d56b0 0x55d002caa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.957+0000 7fc1dffff640 1 -- 172.21.15.43:0/2259256510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0028d56b0 msgr2=0x55d002caa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.957+0000 7fc1dffff640 1 --2- 172.21.15.43:0/2259256510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0028d56b0 0x55d002caa890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.957+0000 7fc1dffff640 1 -- 172.21.15.43:0/2259256510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d0028f6390 msgr2=0x55d002ca58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:18.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.957+0000 7fc1dffff640 1 --2- 172.21.15.43:0/2259256510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d0028f6390 0x55d002ca58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.957+0000 7fc1dffff640 1 -- 172.21.15.43:0/2259256510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d002ca6000 con 0x55d0028cd4d0 2024-04-25T10:41:18.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.957+0000 7fc1e6ffd640 1 --2- 172.21.15.43:0/2259256510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0028d56b0 0x55d002caa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:41:18.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.957+0000 7fc1dffff640 1 --2- 172.21.15.43:0/2259256510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028cd4d0 0x55d002caa350 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0006010 tx=0x7fc1d0003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a6fcf40db754e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:18.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.957+0000 7fc1c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:18.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.957+0000 7fc1c6ffd640 1 -- 172.21.15.43:0/2259256510 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d00042d0 con 0x55d0028cd4d0 2024-04-25T10:41:18.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.957+0000 7fc1c6ffd640 1 -- 172.21.15.43:0/2259256510 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fc1d0004470 con 0x55d0028cd4d0 2024-04-25T10:41:18.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.957+0000 7fc1f482d740 1 -- 172.21.15.43:0/2259256510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1d800ee70 con 0x55d0028cd4d0 2024-04-25T10:41:18.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.957+0000 7fc1c6ffd640 1 -- 172.21.15.43:0/2259256510 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc1d0064050 con 0x55d0028cd4d0 2024-04-25T10:41:18.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.958+0000 7fc1f482d740 10 client.18114 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:18.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.958+0000 7fc1f482d740 1 -- 172.21.15.43:0/2259256510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d002d1c350 con 0x55d0028cd4d0 2024-04-25T10:41:18.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.958+0000 7fc1c6ffd640 1 -- 172.21.15.43:0/2259256510 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1d800ee70 con 0x55d0028cd4d0 2024-04-25T10:41:18.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.958+0000 7fc1c6ffd640 1 -- 172.21.15.43:0/2259256510 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc1d004bb00 con 0x55d0028cd4d0 2024-04-25T10:41:18.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.958+0000 7fc1f482d740 10 client.18114 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:18.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.958+0000 7fc1f482d740 10 client.18114 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:41:18.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.958+0000 7fc1f482d740 20 client.18114 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:18.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.958+0000 7fc1f482d740 1 --2- 172.21.15.43:0/2259256510 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55d00288a3a0 0x55d002d1ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:18.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.958+0000 7fc1f482d740 4 client.18114 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:41:18.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.958+0000 7fc1f482d740 1 -- 172.21.15.43:0/2259256510 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d002c04800 con 0x55d00288a3a0 2024-04-25T10:41:18.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.958+0000 7fc1e6ffd640 1 --2- 172.21.15.43:0/2259256510 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55d00288a3a0 0x55d002d1ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:18.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.959+0000 7fc1e6ffd640 1 --2- 172.21.15.43:0/2259256510 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55d00288a3a0 0x55d002d1ea30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47fca911998e23ff server_cookie=7f16be836012271e in_seq=0 out_seq=0 2024-04-25T10:41:18.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.959+0000 7fc1c6ffd640 10 client.18114 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:41:18.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.959+0000 7fc1c6ffd640 1 -- 172.21.15.43:0/2259256510 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d002c04800 con 0x55d00288a3a0 2024-04-25T10:41:18.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.959+0000 7fc1c6ffd640 10 client.18114 handle_command_reply: tid=0 2024-04-25T10:41:18.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.959+0000 7fc1f482d740 1 client.18114 shutdown 2024-04-25T10:41:18.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.959+0000 7fc1f482d740 1 -- 172.21.15.43:0/2259256510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028cd4d0 msgr2=0x55d002caa350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.959+0000 7fc1f482d740 1 --2- 172.21.15.43:0/2259256510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028cd4d0 0x55d002caa350 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0006010 tx=0x7fc1d0003f60 comp rx=0 tx=0).stop 2024-04-25T10:41:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.959+0000 7fc1f482d740 1 -- 172.21.15.43:0/2259256510 shutdown_connections 2024-04-25T10:41:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.959+0000 7fc1f482d740 1 --2- 172.21.15.43:0/2259256510 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55d00288a3a0 0x55d002d1ea30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.959+0000 7fc1f482d740 1 --2- 172.21.15.43:0/2259256510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d0028f6390 0x55d002ca58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.959+0000 7fc1f482d740 1 --2- 172.21.15.43:0/2259256510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0028d56b0 0x55d002caa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.959+0000 7fc1c6ffd640 0 client.18114 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:41:18.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.959+0000 7fc1f482d740 1 --2- 172.21.15.43:0/2259256510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0028cd4d0 0x55d002caa350 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.959+0000 7fc1f482d740 1 -- 172.21.15.43:0/2259256510 >> 172.21.15.43:0/2259256510 conn(0x55d0028ea4e0 msgr2=0x55d002c92da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:18.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.959+0000 7fc1f482d740 1 -- 172.21.15.43:0/2259256510 shutdown_connections 2024-04-25T10:41:18.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.959+0000 7fc1f482d740 1 -- 172.21.15.43:0/2259256510 wait complete. 2024-04-25T10:41:18.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.960+0000 7fc1f482d740 20 client.18114 trim_cache size 0 max 16384 2024-04-25T10:41:18.973 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:41:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.960+0000 7fc1ed97d640 1 -- 172.21.15.43:0/2276606877 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc1d40afc60 msgr2=0x7fc1d40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.960+0000 7fc1ed97d640 1 --2- 172.21.15.43:0/2276606877 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc1d40afc60 0x7fc1d40b2170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc1e00dc9f0 tx=0x7fc1e0074bf0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.960+0000 7fc1ed97d640 1 -- 172.21.15.43:0/2276606877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1e810d390 msgr2=0x7fc1e80ff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.960+0000 7fc1ed97d640 1 --2- 172.21.15.43:0/2276606877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1e810d390 0x7fc1e80ff1c0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000eaa0 tx=0x7fc1e000ef60 comp rx=0 tx=0).stop 2024-04-25T10:41:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.960+0000 7fc1ed97d640 1 -- 172.21.15.43:0/2276606877 shutdown_connections 2024-04-25T10:41:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.960+0000 7fc1ed97d640 1 --2- 172.21.15.43:0/2276606877 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc1d40afc60 0x7fc1d40b2170 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.960+0000 7fc1ed97d640 1 --2- 172.21.15.43:0/2276606877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1e8108a30 0x7fc1e80fec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.960+0000 7fc1ed97d640 1 --2- 172.21.15.43:0/2276606877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1e810d390 0x7fc1e80ff1c0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.960+0000 7fc1ed97d640 1 --2- 172.21.15.43:0/2276606877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1e8102b00 0x7fc1e80fe740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.960+0000 7fc1ed97d640 1 -- 172.21.15.43:0/2276606877 >> 172.21.15.43:0/2276606877 conn(0x7fc1e80fe260 msgr2=0x7fc1e8074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.960+0000 7fc1ed97d640 1 -- 172.21.15.43:0/2276606877 shutdown_connections 2024-04-25T10:41:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:18.960+0000 7fc1ed97d640 1 -- 172.21.15.43:0/2276606877 wait complete. 2024-04-25T10:41:18.976 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:41:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.087+0000 7f28ef4f9640 1 -- 172.21.15.43:0/2457419540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e8109b10 msgr2=0x7f28e81036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.087+0000 7f28ef4f9640 1 --2- 172.21.15.43:0/2457419540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e8109b10 0x7f28e81036b0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009900 tx=0x7f28e402a410 comp rx=0 tx=0).stop 2024-04-25T10:41:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.087+0000 7f28ef4f9640 1 -- 172.21.15.43:0/2457419540 shutdown_connections 2024-04-25T10:41:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.087+0000 7f28ef4f9640 1 --2- 172.21.15.43:0/2457419540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28e8104260 0x7f28e8110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.087+0000 7f28ef4f9640 1 --2- 172.21.15.43:0/2457419540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28e8109fb0 0x7f28e8103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.087+0000 7f28ef4f9640 1 --2- 172.21.15.43:0/2457419540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e8109b10 0x7f28e81036b0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.087+0000 7f28ef4f9640 1 -- 172.21.15.43:0/2457419540 >> 172.21.15.43:0/2457419540 conn(0x7f28e80ff210 msgr2=0x7f28e8101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.087+0000 7f28ef4f9640 1 -- 172.21.15.43:0/2457419540 shutdown_connections 2024-04-25T10:41:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.087+0000 7f28ef4f9640 1 -- 172.21.15.43:0/2457419540 wait complete. 2024-04-25T10:41:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.088+0000 7f28ef4f9640 1 Processor -- start 2024-04-25T10:41:19.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.088+0000 7f28ef4f9640 1 -- start start 2024-04-25T10:41:19.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.088+0000 7f28ef4f9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28e8104260 0x7f28e819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:19.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.088+0000 7f28ef4f9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e8109b10 0x7f28e819ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:19.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.088+0000 7f28ef4f9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28e8109fb0 0x7f28e819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:19.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.088+0000 7f28ef4f9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f28e8113d00 con 0x7f28e8109b10 2024-04-25T10:41:19.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.088+0000 7f28ef4f9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f28e8113b80 con 0x7f28e8109fb0 2024-04-25T10:41:19.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.088+0000 7f28ef4f9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f28e8113e80 con 0x7f28e8104260 2024-04-25T10:41:19.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.088+0000 7f28eca94640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e8109b10 0x7f28e819ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:19.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.088+0000 7f28eca94640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e8109b10 0x7f28e819ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48856/0 (socket says 172.21.15.43:48856) 2024-04-25T10:41:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.088+0000 7f28eca94640 1 -- 172.21.15.43:0/1067245943 learned_addr learned my addr 172.21.15.43:0/1067245943 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.089+0000 7f28eda96640 1 --2- 172.21.15.43:0/1067245943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28e8109fb0 0x7f28e819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.089+0000 7f28ed295640 1 --2- 172.21.15.43:0/1067245943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28e8104260 0x7f28e819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.089+0000 7f28eca94640 1 -- 172.21.15.43:0/1067245943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28e8104260 msgr2=0x7f28e819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.089+0000 7f28eca94640 1 --2- 172.21.15.43:0/1067245943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28e8104260 0x7f28e819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.089+0000 7f28eca94640 1 -- 172.21.15.43:0/1067245943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28e8109fb0 msgr2=0x7f28e819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.089+0000 7f28eca94640 1 --2- 172.21.15.43:0/1067245943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28e8109fb0 0x7f28e819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.089+0000 7f28eca94640 1 -- 172.21.15.43:0/1067245943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e819f5e0 con 0x7f28e8109b10 2024-04-25T10:41:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.089+0000 7f28eca94640 1 --2- 172.21.15.43:0/1067245943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e8109b10 0x7f28e819ace0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f28d8005fd0 tx=0x7f28d800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.089+0000 7f28de7fc640 1 -- 172.21.15.43:0/1067245943 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8017070 con 0x7f28e8109b10 2024-04-25T10:41:19.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.089+0000 7f28de7fc640 1 -- 172.21.15.43:0/1067245943 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28d8009c30 con 0x7f28e8109b10 2024-04-25T10:41:19.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.089+0000 7f28ef4f9640 1 -- 172.21.15.43:0/1067245943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e819f8d0 con 0x7f28e8109b10 2024-04-25T10:41:19.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.089+0000 7f28de7fc640 1 -- 172.21.15.43:0/1067245943 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8003d10 con 0x7f28e8109b10 2024-04-25T10:41:19.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.089+0000 7f28ef4f9640 1 -- 172.21.15.43:0/1067245943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e81a7250 con 0x7f28e8109b10 2024-04-25T10:41:19.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.092+0000 7f28de7fc640 1 -- 172.21.15.43:0/1067245943 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f28d8003eb0 con 0x7f28e8109b10 2024-04-25T10:41:19.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.092+0000 7f28f63a9740 1 Processor -- start 2024-04-25T10:41:19.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.092+0000 7f28f63a9740 1 -- start start 2024-04-25T10:41:19.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.092+0000 7f28f63a9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559146211390 0x5591465b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:19.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.092+0000 7f28f63a9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591461e84d0 0x5591465b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:19.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.092+0000 7f28f63a9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591461f06b0 0x5591465bf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:19.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.092+0000 7f28f63a9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559146517d80 con 0x559146211390 2024-04-25T10:41:19.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.092+0000 7f28f63a9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5591464d1a20 con 0x5591461f06b0 2024-04-25T10:41:19.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.092+0000 7f28f63a9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559146513d40 con 0x5591461e84d0 2024-04-25T10:41:19.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.093+0000 7f28eda96640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559146211390 0x5591465b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:19.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.093+0000 7f28eda96640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559146211390 0x5591465b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48862/0 (socket says 172.21.15.43:48862) 2024-04-25T10:41:19.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.093+0000 7f28eda96640 1 -- 172.21.15.43:0/887123593 learned_addr learned my addr 172.21.15.43:0/887123593 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:19.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.093+0000 7f28ed295640 1 --2- 172.21.15.43:0/887123593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591461e84d0 0x5591465b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:19.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.093+0000 7f28de7fc640 1 --2- 172.21.15.43:0/1067245943 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f28d40afce0 0x7f28d40b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:19.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.093+0000 7f28eca94640 1 --2- 172.21.15.43:0/887123593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591461f06b0 0x5591465bf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:19.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.093+0000 7f28eda96640 1 --2- 172.21.15.43:0/1067245943 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f28d40afce0 0x7f28d40b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:19.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.093+0000 7f28de7fc640 1 -- 172.21.15.43:0/1067245943 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f28d80dec90 con 0x7f28e8109b10 2024-04-25T10:41:19.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.093+0000 7f28eda96640 1 -- 172.21.15.43:0/887123593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591461e84d0 msgr2=0x5591465b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:19.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.093+0000 7f28eda96640 1 --2- 172.21.15.43:0/887123593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591461e84d0 0x5591465b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.093+0000 7f28eda96640 1 -- 172.21.15.43:0/887123593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591461f06b0 msgr2=0x5591465bf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:19.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.093+0000 7f28eda96640 1 --2- 172.21.15.43:0/887123593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591461f06b0 0x5591465bf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.093+0000 7f28eda96640 1 -- 172.21.15.43:0/887123593 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591464dcc30 con 0x559146211390 2024-04-25T10:41:19.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.093+0000 7f28eda96640 1 --2- 172.21.15.43:0/887123593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559146211390 0x5591465b2810 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f28e0012a70 tx=0x7f28e0012f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61693e44034a77d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:19.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.093+0000 7f28d37fe640 1 -- 172.21.15.43:0/887123593 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e0004230 con 0x559146211390 2024-04-25T10:41:19.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.093+0000 7f28d37fe640 1 -- 172.21.15.43:0/887123593 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28e0012490 con 0x559146211390 2024-04-25T10:41:19.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.093+0000 7f28d37fe640 1 -- 172.21.15.43:0/887123593 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e001fb90 con 0x559146211390 2024-04-25T10:41:19.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.094+0000 7f28f63a9740 1 -- 172.21.15.43:0/887123593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559146211390 msgr2=0x5591465b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:19.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.094+0000 7f28eda96640 1 --2- 172.21.15.43:0/1067245943 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f28d40afce0 0x7f28d40b21f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f28e0022000 tx=0x7f28e0020000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:19.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.094+0000 7f28f63a9740 1 --2- 172.21.15.43:0/887123593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559146211390 0x5591465b2810 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f28e0012a70 tx=0x7f28e0012f30 comp rx=0 tx=0).stop 2024-04-25T10:41:19.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.094+0000 7f28f63a9740 1 -- 172.21.15.43:0/887123593 shutdown_connections 2024-04-25T10:41:19.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.094+0000 7f28f63a9740 1 --2- 172.21.15.43:0/887123593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591461f06b0 0x5591465bf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.094+0000 7f28f63a9740 1 --2- 172.21.15.43:0/887123593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591461e84d0 0x5591465b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.094+0000 7f28f63a9740 1 --2- 172.21.15.43:0/887123593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559146211390 0x5591465b2810 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.094+0000 7f28f63a9740 1 -- 172.21.15.43:0/887123593 >> 172.21.15.43:0/887123593 conn(0x5591462054e0 msgr2=0x5591465aefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:19.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.094+0000 7f28f63a9740 1 -- 172.21.15.43:0/887123593 shutdown_connections 2024-04-25T10:41:19.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.094+0000 7f28f63a9740 1 -- 172.21.15.43:0/887123593 wait complete. 2024-04-25T10:41:19.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.095+0000 7f28f63a9740 1 -- start start 2024-04-25T10:41:19.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.095+0000 7f28f63a9740 1 Processor -- start 2024-04-25T10:41:19.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.095+0000 7f28f63a9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591461e84d0 0x5591465bd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:19.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.095+0000 7f28f63a9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461f06b0 0x5591465bd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:19.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.095+0000 7f28f63a9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559146211390 0x5591465c9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:19.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.095+0000 7f28f63a9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5591465bfd90 con 0x5591461f06b0 2024-04-25T10:41:19.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.095+0000 7f28f63a9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55914649edc0 con 0x5591461e84d0 2024-04-25T10:41:19.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.095+0000 7f28f63a9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5591465bff10 con 0x559146211390 2024-04-25T10:41:19.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.095+0000 7f28f63a9740 10 client.0 _ll_register_callbacks cb 0x5591464ba600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:19.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.095+0000 7f28eca94640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461f06b0 0x5591465bd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:19.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.095+0000 7f28eca94640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461f06b0 0x5591465bd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48864/0 (socket says 172.21.15.43:48864) 2024-04-25T10:41:19.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.095+0000 7f28eca94640 1 -- 172.21.15.43:0/3778556110 learned_addr learned my addr 172.21.15.43:0/3778556110 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:19.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.096+0000 7f28ed295640 1 --2- 172.21.15.43:0/3778556110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591461e84d0 0x5591465bd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:19.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.096+0000 7f28eda96640 1 --2- 172.21.15.43:0/3778556110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559146211390 0x5591465c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:19.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.096+0000 7f28eca94640 1 -- 172.21.15.43:0/3778556110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559146211390 msgr2=0x5591465c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:19.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.096+0000 7f28eca94640 1 --2- 172.21.15.43:0/3778556110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559146211390 0x5591465c9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.096+0000 7f28eca94640 1 -- 172.21.15.43:0/3778556110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591461e84d0 msgr2=0x5591465bd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:19.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.096+0000 7f28eca94640 1 --2- 172.21.15.43:0/3778556110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591461e84d0 0x5591465bd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.096+0000 7f28eca94640 1 -- 172.21.15.43:0/3778556110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591462293f0 con 0x5591461f06b0 2024-04-25T10:41:19.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.096+0000 7f28eca94640 1 --2- 172.21.15.43:0/3778556110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461f06b0 0x5591465bd780 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f28d80058c0 tx=0x7f28d806d6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604edb4331005566 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:19.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.096+0000 7f28d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:19.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.096+0000 7f28d0ff9640 1 -- 172.21.15.43:0/3778556110 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d80a7530 con 0x5591461f06b0 2024-04-25T10:41:19.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.096+0000 7f28d0ff9640 1 -- 172.21.15.43:0/3778556110 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f28d80dc660 con 0x5591461f06b0 2024-04-25T10:41:19.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.096+0000 7f28f63a9740 1 -- 172.21.15.43:0/3778556110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559146525990 con 0x5591461f06b0 2024-04-25T10:41:19.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.096+0000 7f28d0ff9640 1 -- 172.21.15.43:0/3778556110 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f28d8009de0 con 0x5591461f06b0 2024-04-25T10:41:19.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.096+0000 7f28f63a9740 10 client.18126 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:19.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.096+0000 7f28f63a9740 1 -- 172.21.15.43:0/3778556110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5591462f37d0 con 0x5591461f06b0 2024-04-25T10:41:19.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.096+0000 7f28d0ff9640 1 -- 172.21.15.43:0/3778556110 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559146525990 con 0x5591461f06b0 2024-04-25T10:41:19.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.097+0000 7f28d0ff9640 1 -- 172.21.15.43:0/3778556110 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f28d80a30c0 con 0x5591461f06b0 2024-04-25T10:41:19.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.097+0000 7f28f63a9740 10 client.18126 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:19.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.097+0000 7f28f63a9740 10 client.18126 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:19.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.097+0000 7f28f63a9740 20 client.18126 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:19.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.097+0000 7f28f63a9740 1 --2- 172.21.15.43:0/3778556110 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5591461cff60 0x5591465cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:19.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.097+0000 7f28f63a9740 4 client.18126 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:41:19.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.097+0000 7f28f63a9740 1 -- 172.21.15.43:0/3778556110 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55914651f800 con 0x5591461cff60 2024-04-25T10:41:19.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.097+0000 7f28ed295640 1 --2- 172.21.15.43:0/3778556110 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5591461cff60 0x5591465cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:19.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.098+0000 7f28ed295640 1 --2- 172.21.15.43:0/3778556110 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5591461cff60 0x5591465cf5f0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=990eccdbd11e3423 server_cookie=3e2beeda73efeca in_seq=0 out_seq=0 2024-04-25T10:41:19.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.098+0000 7f28d0ff9640 10 client.18126 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:19.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.098+0000 7f28d0ff9640 1 -- 172.21.15.43:0/3778556110 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55914651f800 con 0x5591461cff60 2024-04-25T10:41:19.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.098+0000 7f28d0ff9640 10 client.18126 handle_command_reply: tid=0 2024-04-25T10:41:19.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.098+0000 7f28f63a9740 1 client.18126 shutdown 2024-04-25T10:41:19.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.098+0000 7f28f63a9740 1 -- 172.21.15.43:0/3778556110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461f06b0 msgr2=0x5591465bd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:19.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.098+0000 7f28f63a9740 1 --2- 172.21.15.43:0/3778556110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461f06b0 0x5591465bd780 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f28d80058c0 tx=0x7f28d806d6f0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.098+0000 7f28f63a9740 1 -- 172.21.15.43:0/3778556110 shutdown_connections 2024-04-25T10:41:19.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.098+0000 7f28f63a9740 1 --2- 172.21.15.43:0/3778556110 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5591461cff60 0x5591465cf5f0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.098+0000 7f28f63a9740 1 --2- 172.21.15.43:0/3778556110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559146211390 0x5591465c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.098+0000 7f28f63a9740 1 --2- 172.21.15.43:0/3778556110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461f06b0 0x5591465bd780 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.098+0000 7f28f63a9740 1 --2- 172.21.15.43:0/3778556110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591461e84d0 0x5591465bd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.098+0000 7f28d0ff9640 0 client.18126 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:19.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.099+0000 7f28f63a9740 1 -- 172.21.15.43:0/3778556110 >> 172.21.15.43:0/3778556110 conn(0x5591462054e0 msgr2=0x5591465b4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:19.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.099+0000 7f28f63a9740 1 -- 172.21.15.43:0/3778556110 shutdown_connections 2024-04-25T10:41:19.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.099+0000 7f28f63a9740 1 -- 172.21.15.43:0/3778556110 wait complete. 2024-04-25T10:41:19.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.099+0000 7f28f63a9740 20 client.18126 trim_cache size 0 max 16384 2024-04-25T10:41:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.112+0000 7f28f63a9740 1 Processor -- start 2024-04-25T10:41:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.112+0000 7f28f63a9740 1 -- start start 2024-04-25T10:41:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28f63a9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591462054e0 0x5591465c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28f63a9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591461e84d0 0x5591465d4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28f63a9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461f06b0 0x5591465d52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28f63a9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f28e81b22f0 con 0x5591461f06b0 2024-04-25T10:41:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28f63a9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f28e81afbc0 con 0x5591462054e0 2024-04-25T10:41:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28f63a9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f28e81ada30 con 0x5591461e84d0 2024-04-25T10:41:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28eda96640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461f06b0 0x5591465d52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28eda96640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461f06b0 0x5591465d52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48880/0 (socket says 172.21.15.43:48880) 2024-04-25T10:41:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28eda96640 1 -- 172.21.15.43:0/1945921112 learned_addr learned my addr 172.21.15.43:0/1945921112 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28ed295640 1 --2- 172.21.15.43:0/1945921112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591462054e0 0x5591465c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28eca94640 1 --2- 172.21.15.43:0/1945921112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591461e84d0 0x5591465d4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28eda96640 1 -- 172.21.15.43:0/1945921112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591461e84d0 msgr2=0x5591465d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28eda96640 1 --2- 172.21.15.43:0/1945921112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591461e84d0 0x5591465d4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28eda96640 1 -- 172.21.15.43:0/1945921112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591462054e0 msgr2=0x5591465c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28eda96640 1 --2- 172.21.15.43:0/1945921112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591462054e0 0x5591465c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28eda96640 1 -- 172.21.15.43:0/1945921112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591465b8bf0 con 0x5591461f06b0 2024-04-25T10:41:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28ed295640 1 --2- 172.21.15.43:0/1945921112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591462054e0 0x5591465c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28eca94640 1 --2- 172.21.15.43:0/1945921112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591461e84d0 0x5591465d4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28eda96640 1 --2- 172.21.15.43:0/1945921112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461f06b0 0x5591465d52a0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x5591465b3910 tx=0x7f28e0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3bed207faaebed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28d2ffd640 1 -- 172.21.15.43:0/1945921112 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e0038070 con 0x5591461f06b0 2024-04-25T10:41:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28d2ffd640 1 -- 172.21.15.43:0/1945921112 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28e0009ab0 con 0x5591461f06b0 2024-04-25T10:41:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.113+0000 7f28d2ffd640 1 -- 172.21.15.43:0/1945921112 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e0003ec0 con 0x5591461f06b0 2024-04-25T10:41:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.114+0000 7f28f63a9740 1 -- 172.21.15.43:0/1945921112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461f06b0 msgr2=0x5591465d52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.114+0000 7f28f63a9740 1 --2- 172.21.15.43:0/1945921112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461f06b0 0x5591465d52a0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x5591465b3910 tx=0x7f28e0011000 comp rx=0 tx=0).stop 2024-04-25T10:41:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.114+0000 7f28f63a9740 1 -- 172.21.15.43:0/1945921112 shutdown_connections 2024-04-25T10:41:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.114+0000 7f28f63a9740 1 --2- 172.21.15.43:0/1945921112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461f06b0 0x5591465d52a0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.114+0000 7f28f63a9740 1 --2- 172.21.15.43:0/1945921112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591461e84d0 0x5591465d4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.114+0000 7f28f63a9740 1 --2- 172.21.15.43:0/1945921112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591462054e0 0x5591465c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.114+0000 7f28f63a9740 1 -- 172.21.15.43:0/1945921112 >> 172.21.15.43:0/1945921112 conn(0x5591465b2e80 msgr2=0x5591465bb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.114+0000 7f28f63a9740 1 -- 172.21.15.43:0/1945921112 shutdown_connections 2024-04-25T10:41:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.114+0000 7f28f63a9740 1 -- 172.21.15.43:0/1945921112 wait complete. 2024-04-25T10:41:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.115+0000 7f28f63a9740 1 -- start start 2024-04-25T10:41:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.115+0000 7f28f63a9740 1 Processor -- start 2024-04-25T10:41:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.115+0000 7f28f63a9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461e84d0 0x5591465c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.115+0000 7f28f63a9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591461f06b0 0x5591465c09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.115+0000 7f28f63a9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591462054e0 0x5591465c0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.115+0000 7f28f63a9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f28e81ae080 con 0x5591461e84d0 2024-04-25T10:41:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.115+0000 7f28f63a9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f28e81af4f0 con 0x5591461f06b0 2024-04-25T10:41:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.115+0000 7f28f63a9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f28e81ae750 con 0x5591462054e0 2024-04-25T10:41:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.115+0000 7f28f63a9740 10 client.0 _ll_register_callbacks cb 0x7f28d800df10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.115+0000 7f28ed295640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461e84d0 0x5591465c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.115+0000 7f28ed295640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461e84d0 0x5591465c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48890/0 (socket says 172.21.15.43:48890) 2024-04-25T10:41:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.115+0000 7f28ed295640 1 -- 172.21.15.43:0/3145628691 learned_addr learned my addr 172.21.15.43:0/3145628691 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.116+0000 7f28ed295640 1 -- 172.21.15.43:0/3145628691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591462054e0 msgr2=0x5591465c0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.116+0000 7f28eda96640 1 --2- 172.21.15.43:0/3145628691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591462054e0 0x5591465c0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.116+0000 7f28eca94640 1 --2- 172.21.15.43:0/3145628691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591461f06b0 0x5591465c09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.116+0000 7f28ed295640 1 --2- 172.21.15.43:0/3145628691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591462054e0 0x5591465c0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.116+0000 7f28ed295640 1 -- 172.21.15.43:0/3145628691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591461f06b0 msgr2=0x5591465c09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.116+0000 7f28ed295640 1 --2- 172.21.15.43:0/3145628691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591461f06b0 0x5591465c09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.116+0000 7f28ed295640 1 -- 172.21.15.43:0/3145628691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591465c1590 con 0x5591461e84d0 2024-04-25T10:41:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.116+0000 7f28eda96640 1 --2- 172.21.15.43:0/3145628691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591462054e0 0x5591465c0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:41:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.116+0000 7f28ed295640 1 --2- 172.21.15.43:0/3145628691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461e84d0 0x5591465c5690 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f28e81047f0 tx=0x7f28e402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b873506c55f939 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.116+0000 7f28d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.116+0000 7f28d0ff9640 1 -- 172.21.15.43:0/3145628691 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4049020 con 0x5591461e84d0 2024-04-25T10:41:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.116+0000 7f28d0ff9640 1 -- 172.21.15.43:0/3145628691 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f28e4005450 con 0x5591461e84d0 2024-04-25T10:41:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.116+0000 7f28f63a9740 1 -- 172.21.15.43:0/3145628691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28e0009ab0 con 0x5591461e84d0 2024-04-25T10:41:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.116+0000 7f28d0ff9640 1 -- 172.21.15.43:0/3145628691 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f28e4054050 con 0x5591461e84d0 2024-04-25T10:41:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.116+0000 7f28f63a9740 10 client.18132 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.116+0000 7f28f63a9740 1 -- 172.21.15.43:0/3145628691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5591466376e0 con 0x5591461e84d0 2024-04-25T10:41:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.116+0000 7f28d0ff9640 1 -- 172.21.15.43:0/3145628691 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28e0009ab0 con 0x5591461e84d0 2024-04-25T10:41:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.116+0000 7f28d0ff9640 1 -- 172.21.15.43:0/3145628691 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f28e40047e0 con 0x5591461e84d0 2024-04-25T10:41:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.117+0000 7f28f63a9740 10 client.18132 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.117+0000 7f28f63a9740 10 client.18132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.117+0000 7f28f63a9740 20 client.18132 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.117+0000 7f28f63a9740 1 --2- 172.21.15.43:0/3145628691 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559146211390 0x559146639ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.117+0000 7f28f63a9740 4 client.18132 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:41:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.117+0000 7f28f63a9740 1 -- 172.21.15.43:0/3145628691 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55914649a610 con 0x559146211390 2024-04-25T10:41:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.117+0000 7f28eca94640 1 --2- 172.21.15.43:0/3145628691 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559146211390 0x559146639ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.117+0000 7f28d0ff9640 10 client.18132 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.117+0000 7f28eca94640 1 --2- 172.21.15.43:0/3145628691 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559146211390 0x559146639ed0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e866d483cc7c7908 server_cookie=228d625bd633e7b0 in_seq=0 out_seq=0 2024-04-25T10:41:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.117+0000 7f28d0ff9640 1 -- 172.21.15.43:0/3145628691 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55914649a610 con 0x559146211390 2024-04-25T10:41:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.117+0000 7f28d0ff9640 10 client.18132 handle_command_reply: tid=0 2024-04-25T10:41:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.117+0000 7f28f63a9740 1 client.18132 shutdown 2024-04-25T10:41:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28f63a9740 1 -- 172.21.15.43:0/3145628691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461e84d0 msgr2=0x5591465c5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28f63a9740 1 --2- 172.21.15.43:0/3145628691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461e84d0 0x5591465c5690 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f28e81047f0 tx=0x7f28e402a410 comp rx=0 tx=0).stop 2024-04-25T10:41:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28f63a9740 1 -- 172.21.15.43:0/3145628691 shutdown_connections 2024-04-25T10:41:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28f63a9740 1 --2- 172.21.15.43:0/3145628691 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x559146211390 0x559146639ed0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28f63a9740 1 --2- 172.21.15.43:0/3145628691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591461f06b0 0x5591465c09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28f63a9740 1 --2- 172.21.15.43:0/3145628691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5591462054e0 0x5591465c0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28d0ff9640 0 client.18132 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28f63a9740 1 --2- 172.21.15.43:0/3145628691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591461e84d0 0x5591465c5690 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28f63a9740 1 -- 172.21.15.43:0/3145628691 >> 172.21.15.43:0/3145628691 conn(0x5591465b2e80 msgr2=0x5591465ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28f63a9740 1 -- 172.21.15.43:0/3145628691 shutdown_connections 2024-04-25T10:41:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28f63a9740 1 -- 172.21.15.43:0/3145628691 wait complete. 2024-04-25T10:41:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28f63a9740 20 client.18132 trim_cache size 0 max 16384 2024-04-25T10:41:19.127 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:41:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28ef4f9640 1 -- 172.21.15.43:0/1067245943 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f28d40afce0 msgr2=0x7f28d40b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28ef4f9640 1 --2- 172.21.15.43:0/1067245943 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f28d40afce0 0x7f28d40b21f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f28e0022000 tx=0x7f28e0020000 comp rx=0 tx=0).stop 2024-04-25T10:41:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28ef4f9640 1 -- 172.21.15.43:0/1067245943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e8109b10 msgr2=0x7f28e819ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28ef4f9640 1 --2- 172.21.15.43:0/1067245943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e8109b10 0x7f28e819ace0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f28d8005fd0 tx=0x7f28d800dde0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28ef4f9640 1 -- 172.21.15.43:0/1067245943 shutdown_connections 2024-04-25T10:41:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28ef4f9640 1 --2- 172.21.15.43:0/1067245943 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f28d40afce0 0x7f28d40b21f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28ef4f9640 1 --2- 172.21.15.43:0/1067245943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28e8109fb0 0x7f28e819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28ef4f9640 1 --2- 172.21.15.43:0/1067245943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e8109b10 0x7f28e819ace0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28ef4f9640 1 --2- 172.21.15.43:0/1067245943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28e8104260 0x7f28e819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28ef4f9640 1 -- 172.21.15.43:0/1067245943 >> 172.21.15.43:0/1067245943 conn(0x7f28e80ff210 msgr2=0x7f28e80ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:19.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28ef4f9640 1 -- 172.21.15.43:0/1067245943 shutdown_connections 2024-04-25T10:41:19.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:19.118+0000 7f28ef4f9640 1 -- 172.21.15.43:0/1067245943 wait complete. 2024-04-25T10:41:19.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:18 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/957862310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:41:19.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:18 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/957862310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:41:19.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:18 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/957862310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:41:20.132 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:41:20.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:41:20.133 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:41:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.260+0000 7f7af49f3640 1 -- 172.21.15.43:0/1253201691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7af0104c80 msgr2=0x7f7af0105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.260+0000 7f7af49f3640 1 --2- 172.21.15.43:0/1253201691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7af0104c80 0x7f7af0105090 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0009900 tx=0x7f7ae002a480 comp rx=0 tx=0).stop 2024-04-25T10:41:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.260+0000 7f7af49f3640 1 -- 172.21.15.43:0/1253201691 shutdown_connections 2024-04-25T10:41:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.260+0000 7f7af49f3640 1 --2- 172.21.15.43:0/1253201691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7af010e090 0x7f7af0110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.260+0000 7f7af49f3640 1 --2- 172.21.15.43:0/1253201691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7af010b6e0 0x7f7af010db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.260+0000 7f7af49f3640 1 --2- 172.21.15.43:0/1253201691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7af0104c80 0x7f7af0105090 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.260+0000 7f7af49f3640 1 -- 172.21.15.43:0/1253201691 >> 172.21.15.43:0/1253201691 conn(0x7f7af0078180 msgr2=0x7f7af00785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.260+0000 7f7af49f3640 1 -- 172.21.15.43:0/1253201691 shutdown_connections 2024-04-25T10:41:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.261+0000 7f7af49f3640 1 -- 172.21.15.43:0/1253201691 wait complete. 2024-04-25T10:41:20.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.261+0000 7f7af49f3640 1 Processor -- start 2024-04-25T10:41:20.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.262+0000 7f7af49f3640 1 -- start start 2024-04-25T10:41:20.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.262+0000 7f7af49f3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7af0104c80 0x7f7af019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.262+0000 7f7af49f3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7af010b6e0 0x7f7af019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.262+0000 7f7af49f3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7af010e090 0x7f7af019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.262+0000 7f7af49f3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7af0113d30 con 0x7f7af010e090 2024-04-25T10:41:20.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.262+0000 7f7af49f3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7af0113bb0 con 0x7f7af010b6e0 2024-04-25T10:41:20.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.262+0000 7f7af49f3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7af0113eb0 con 0x7f7af0104c80 2024-04-25T10:41:20.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.262+0000 7f7aeed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7af010e090 0x7f7af019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.262+0000 7f7aeed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7af010e090 0x7f7af019efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48904/0 (socket says 172.21.15.43:48904) 2024-04-25T10:41:20.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.262+0000 7f7aeed9d640 1 -- 172.21.15.43:0/4181928595 learned_addr learned my addr 172.21.15.43:0/4181928595 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:20.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.262+0000 7f7ae5d9b640 1 --2- 172.21.15.43:0/4181928595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7af010b6e0 0x7f7af019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.262+0000 7f7aee59c640 1 --2- 172.21.15.43:0/4181928595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7af0104c80 0x7f7af019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.262+0000 7f7aeed9d640 1 -- 172.21.15.43:0/4181928595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7af0104c80 msgr2=0x7f7af019a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:20.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.262+0000 7f7aeed9d640 1 --2- 172.21.15.43:0/4181928595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7af0104c80 0x7f7af019a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.262+0000 7f7aeed9d640 1 -- 172.21.15.43:0/4181928595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7af010b6e0 msgr2=0x7f7af019ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:20.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.262+0000 7f7aeed9d640 1 --2- 172.21.15.43:0/4181928595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7af010b6e0 0x7f7af019ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.262+0000 7f7aeed9d640 1 -- 172.21.15.43:0/4181928595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af019f610 con 0x7f7af010e090 2024-04-25T10:41:20.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.263+0000 7f7aeed9d640 1 --2- 172.21.15.43:0/4181928595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7af010e090 0x7f7af019efd0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800cad0 tx=0x7f7ae800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:20.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.263+0000 7f7ae7fff640 1 -- 172.21.15.43:0/4181928595 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae800ad60 con 0x7f7af010e090 2024-04-25T10:41:20.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.263+0000 7f7ae7fff640 1 -- 172.21.15.43:0/4181928595 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ae8005020 con 0x7f7af010e090 2024-04-25T10:41:20.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.263+0000 7f7ae7fff640 1 -- 172.21.15.43:0/4181928595 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae800f6b0 con 0x7f7af010e090 2024-04-25T10:41:20.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.263+0000 7f7af49f3640 1 -- 172.21.15.43:0/4181928595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af019f900 con 0x7f7af010e090 2024-04-25T10:41:20.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.263+0000 7f7af49f3640 1 -- 172.21.15.43:0/4181928595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af019fe00 con 0x7f7af010e090 2024-04-25T10:41:20.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.266+0000 7f7afb97b740 1 Processor -- start 2024-04-25T10:41:20.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.266+0000 7f7ae7fff640 1 -- 172.21.15.43:0/4181928595 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f7ae80051d0 con 0x7f7af010e090 2024-04-25T10:41:20.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.266+0000 7f7afb97b740 1 -- start start 2024-04-25T10:41:20.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.267+0000 7f7afb97b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 0x5557c02b3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.267+0000 7f7afb97b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bfeea550 0x5557c02b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.267+0000 7f7afb97b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557bfef2730 0x5557c02b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.267+0000 7f7afb97b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5557c0219b00 con 0x5557bff13410 2024-04-25T10:41:20.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.267+0000 7f7afb97b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5557c01b5920 con 0x5557bfef2730 2024-04-25T10:41:20.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.267+0000 7f7afb97b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5557c0215ac0 con 0x5557bfeea550 2024-04-25T10:41:20.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.267+0000 7f7ae5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 0x5557c02b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.267+0000 7f7ae5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 0x5557c02b3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48918/0 (socket says 172.21.15.43:48918) 2024-04-25T10:41:20.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.267+0000 7f7ae5d9b640 1 -- 172.21.15.43:0/2864077779 learned_addr learned my addr 172.21.15.43:0/2864077779 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:20.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.267+0000 7f7ae5d9b640 1 --2- 172.21.15.43:0/2864077779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557bfef2730 0x5557c02b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.267+0000 7f7ae7fff640 1 --2- 172.21.15.43:0/4181928595 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7adc0afc60 0x7f7adc0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.267+0000 7f7aee59c640 1 --2- 172.21.15.43:0/2864077779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bfeea550 0x5557c02b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.267+0000 7f7ae5d9b640 1 -- 172.21.15.43:0/2864077779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bfeea550 msgr2=0x5557c02b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:20.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.267+0000 7f7aeed9d640 1 --2- 172.21.15.43:0/4181928595 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7adc0afc60 0x7f7adc0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.267+0000 7f7ae5d9b640 1 --2- 172.21.15.43:0/2864077779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bfeea550 0x5557c02b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.267+0000 7f7ae5d9b640 1 -- 172.21.15.43:0/2864077779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557bfef2730 msgr2=0x5557c02b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:20.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.267+0000 7f7ae5d9b640 1 --2- 172.21.15.43:0/2864077779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557bfef2730 0x5557c02b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.267+0000 7f7ae5d9b640 1 -- 172.21.15.43:0/2864077779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557bfff37e0 con 0x5557bff13410 2024-04-25T10:41:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.267+0000 7f7aee59c640 1 --2- 172.21.15.43:0/2864077779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bfeea550 0x5557c02b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.268+0000 7f7ae7fff640 1 -- 172.21.15.43:0/4181928595 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f7ae80df0b0 con 0x7f7af010e090 2024-04-25T10:41:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.268+0000 7f7ae5d9b640 1 --2- 172.21.15.43:0/2864077779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 0x5557c02b3510 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8017700 tx=0x7f7ad80095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9213df1764e9e4ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.268+0000 7f7ae4d99640 1 -- 172.21.15.43:0/2864077779 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80107f0 con 0x5557bff13410 2024-04-25T10:41:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.268+0000 7f7ae4d99640 1 -- 172.21.15.43:0/2864077779 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ad801d070 con 0x5557bff13410 2024-04-25T10:41:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.268+0000 7f7ae4d99640 1 -- 172.21.15.43:0/2864077779 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80189c0 con 0x5557bff13410 2024-04-25T10:41:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.269+0000 7f7afb97b740 1 -- 172.21.15.43:0/2864077779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 msgr2=0x5557c02b3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.269+0000 7f7aeed9d640 1 --2- 172.21.15.43:0/4181928595 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7adc0afc60 0x7f7adc0b2170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80a44c0 tx=0x7f7ae8074ab0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.269+0000 7f7afb97b740 1 --2- 172.21.15.43:0/2864077779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 0x5557c02b3510 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8017700 tx=0x7f7ad80095d0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.269+0000 7f7afb97b740 1 -- 172.21.15.43:0/2864077779 shutdown_connections 2024-04-25T10:41:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.269+0000 7f7afb97b740 1 --2- 172.21.15.43:0/2864077779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557bfef2730 0x5557c02b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.269+0000 7f7afb97b740 1 --2- 172.21.15.43:0/2864077779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bfeea550 0x5557c02b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.269+0000 7f7afb97b740 1 --2- 172.21.15.43:0/2864077779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 0x5557c02b3510 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.269+0000 7f7afb97b740 1 -- 172.21.15.43:0/2864077779 >> 172.21.15.43:0/2864077779 conn(0x5557bff07560 msgr2=0x5557c02afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:20.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.269+0000 7f7afb97b740 1 -- 172.21.15.43:0/2864077779 shutdown_connections 2024-04-25T10:41:20.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.269+0000 7f7afb97b740 1 -- 172.21.15.43:0/2864077779 wait complete. 2024-04-25T10:41:20.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.269+0000 7f7afb97b740 1 -- start start 2024-04-25T10:41:20.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.269+0000 7f7afb97b740 1 Processor -- start 2024-04-25T10:41:20.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.270+0000 7f7afb97b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bfeea550 0x5557c02c2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.270+0000 7f7afb97b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557bfef2730 0x5557c02c2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.270+0000 7f7afb97b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 0x5557c02ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.270+0000 7f7afb97b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5557c02c4d20 con 0x5557bff13410 2024-04-25T10:41:20.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.270+0000 7f7afb97b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5557c01b35d0 con 0x5557bfef2730 2024-04-25T10:41:20.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.270+0000 7f7afb97b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5557c02c4ea0 con 0x5557bfeea550 2024-04-25T10:41:20.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.270+0000 7f7afb97b740 10 client.0 _ll_register_callbacks cb 0x5557c01a1080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:20.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.270+0000 7f7ae5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 0x5557c02ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.270+0000 7f7ae5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 0x5557c02ce1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48920/0 (socket says 172.21.15.43:48920) 2024-04-25T10:41:20.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.270+0000 7f7ae5d9b640 1 -- 172.21.15.43:0/1463019687 learned_addr learned my addr 172.21.15.43:0/1463019687 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:20.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.270+0000 7f7ae5d9b640 1 --2- 172.21.15.43:0/1463019687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bfeea550 0x5557c02c2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.270+0000 7f7aee59c640 1 --2- 172.21.15.43:0/1463019687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557bfef2730 0x5557c02c2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.270+0000 7f7ae5d9b640 1 -- 172.21.15.43:0/1463019687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bfeea550 msgr2=0x5557c02c2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:20.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.270+0000 7f7ae5d9b640 1 --2- 172.21.15.43:0/1463019687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bfeea550 0x5557c02c2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.270+0000 7f7ae5d9b640 1 -- 172.21.15.43:0/1463019687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557bfef2730 msgr2=0x5557c02c2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:20.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.270+0000 7f7ae5d9b640 1 --2- 172.21.15.43:0/1463019687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557bfef2730 0x5557c02c2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.270+0000 7f7ae5d9b640 1 -- 172.21.15.43:0/1463019687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557c0053f70 con 0x5557bff13410 2024-04-25T10:41:20.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.271+0000 7f7ae5d9b640 1 --2- 172.21.15.43:0/1463019687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 0x5557c02ce1d0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8006d30 tx=0x7f7ad8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd053a9a1083735d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:20.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.271+0000 7f7ad5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:20.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.271+0000 7f7ad5ffb640 1 -- 172.21.15.43:0/1463019687 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8019d90 con 0x5557bff13410 2024-04-25T10:41:20.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.271+0000 7f7afb97b740 1 -- 172.21.15.43:0/1463019687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ad801d070 con 0x5557bff13410 2024-04-25T10:41:20.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.271+0000 7f7ad5ffb640 1 -- 172.21.15.43:0/1463019687 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f7ad80105c0 con 0x5557bff13410 2024-04-25T10:41:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.271+0000 7f7ad5ffb640 1 -- 172.21.15.43:0/1463019687 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7ad80107c0 con 0x5557bff13410 2024-04-25T10:41:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.271+0000 7f7afb97b740 10 client.18144 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.271+0000 7f7afb97b740 1 -- 172.21.15.43:0/1463019687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5557bfeeb170 con 0x5557bff13410 2024-04-25T10:41:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.271+0000 7f7ad5ffb640 1 -- 172.21.15.43:0/1463019687 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ad801d070 con 0x5557bff13410 2024-04-25T10:41:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.271+0000 7f7ad5ffb640 1 -- 172.21.15.43:0/1463019687 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7ad80246f0 con 0x5557bff13410 2024-04-25T10:41:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.271+0000 7f7afb97b740 10 client.18144 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.271+0000 7f7afb97b740 10 client.18144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.271+0000 7f7afb97b740 20 client.18144 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.271+0000 7f7afb97b740 1 --2- 172.21.15.43:0/1463019687 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5557bfed21f0 0x5557c02d42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.271+0000 7f7afb97b740 4 client.18144 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:41:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.271+0000 7f7afb97b740 1 -- 172.21.15.43:0/1463019687 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557c0227710 con 0x5557bfed21f0 2024-04-25T10:41:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.272+0000 7f7aee59c640 1 --2- 172.21.15.43:0/1463019687 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5557bfed21f0 0x5557c02d42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.272+0000 7f7aee59c640 1 --2- 172.21.15.43:0/1463019687 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5557bfed21f0 0x5557c02d42d0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=849abe118400391f server_cookie=6229a4f62f753e0b in_seq=0 out_seq=0 2024-04-25T10:41:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.272+0000 7f7ad5ffb640 10 client.18144 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:20.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.273+0000 7f7ad5ffb640 1 -- 172.21.15.43:0/1463019687 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5557c0227710 con 0x5557bfed21f0 2024-04-25T10:41:20.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.273+0000 7f7ad5ffb640 10 client.18144 handle_command_reply: tid=0 2024-04-25T10:41:20.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.273+0000 7f7afb97b740 1 client.18144 shutdown 2024-04-25T10:41:20.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.273+0000 7f7afb97b740 1 -- 172.21.15.43:0/1463019687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 msgr2=0x5557c02ce1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:20.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.273+0000 7f7afb97b740 1 --2- 172.21.15.43:0/1463019687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 0x5557c02ce1d0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8006d30 tx=0x7f7ad8008040 comp rx=0 tx=0).stop 2024-04-25T10:41:20.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.273+0000 7f7afb97b740 1 -- 172.21.15.43:0/1463019687 shutdown_connections 2024-04-25T10:41:20.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.273+0000 7f7afb97b740 1 --2- 172.21.15.43:0/1463019687 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5557bfed21f0 0x5557c02d42d0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.273+0000 7f7afb97b740 1 --2- 172.21.15.43:0/1463019687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557bfef2730 0x5557c02c2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.273+0000 7f7afb97b740 1 --2- 172.21.15.43:0/1463019687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 0x5557c02ce1d0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.273+0000 7f7ad5ffb640 0 client.18144 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:20.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.273+0000 7f7afb97b740 1 --2- 172.21.15.43:0/1463019687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bfeea550 0x5557c02c2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.273+0000 7f7afb97b740 1 -- 172.21.15.43:0/1463019687 >> 172.21.15.43:0/1463019687 conn(0x5557bff07560 msgr2=0x5557c02b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:20.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.273+0000 7f7afb97b740 1 -- 172.21.15.43:0/1463019687 shutdown_connections 2024-04-25T10:41:20.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.274+0000 7f7afb97b740 1 -- 172.21.15.43:0/1463019687 wait complete. 2024-04-25T10:41:20.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.274+0000 7f7afb97b740 20 client.18144 trim_cache size 0 max 16384 2024-04-25T10:41:20.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:19 smithi077 ceph-mon[32772]: pgmap v273: 97 pgs: 97 active+clean; 3.8 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-25T10:41:20.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:19 smithi164 ceph-mon[32713]: pgmap v273: 97 pgs: 97 active+clean; 3.8 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-25T10:41:20.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.292+0000 7f7afb97b740 1 Processor -- start 2024-04-25T10:41:20.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.292+0000 7f7afb97b740 1 -- start start 2024-04-25T10:41:20.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.292+0000 7f7afb97b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bfeea550 0x5557c02c3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.292+0000 7f7afb97b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557bfef2730 0x5557c02d7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.292+0000 7f7afb97b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 0x5557c02d7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.292+0000 7f7afb97b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7af01b0100 con 0x5557bff13410 2024-04-25T10:41:20.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.292+0000 7f7afb97b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7af01ad9d0 con 0x5557bfef2730 2024-04-25T10:41:20.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.292+0000 7f7afb97b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7af01ab860 con 0x5557bfeea550 2024-04-25T10:41:20.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.293+0000 7f7ae5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 0x5557c02d7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.293+0000 7f7ae5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 0x5557c02d7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48922/0 (socket says 172.21.15.43:48922) 2024-04-25T10:41:20.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.293+0000 7f7ae5d9b640 1 -- 172.21.15.43:0/3568372919 learned_addr learned my addr 172.21.15.43:0/3568372919 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:20.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.293+0000 7f7ae5d9b640 1 --2- 172.21.15.43:0/3568372919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bfeea550 0x5557c02c3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.293+0000 7f7aee59c640 1 --2- 172.21.15.43:0/3568372919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557bfef2730 0x5557c02d7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.293+0000 7f7ae5d9b640 1 -- 172.21.15.43:0/3568372919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bfeea550 msgr2=0x5557c02c3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:20.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.293+0000 7f7ae5d9b640 1 --2- 172.21.15.43:0/3568372919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bfeea550 0x5557c02c3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.293+0000 7f7ae5d9b640 1 -- 172.21.15.43:0/3568372919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557bfef2730 msgr2=0x5557c02d7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:20.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.293+0000 7f7ae5d9b640 1 --2- 172.21.15.43:0/3568372919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557bfef2730 0x5557c02d7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.293+0000 7f7ae5d9b640 1 -- 172.21.15.43:0/3568372919 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557c02c46e0 con 0x5557bff13410 2024-04-25T10:41:20.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.293+0000 7f7ae5d9b640 1 --2- 172.21.15.43:0/3568372919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 0x5557c02d7840 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8009da0 tx=0x7f7ad8009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22239a731654248f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:20.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.293+0000 7f7ad7fff640 1 -- 172.21.15.43:0/3568372919 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8029040 con 0x5557bff13410 2024-04-25T10:41:20.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.293+0000 7f7ad7fff640 1 -- 172.21.15.43:0/3568372919 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ad801d070 con 0x5557bff13410 2024-04-25T10:41:20.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.293+0000 7f7ad7fff640 1 -- 172.21.15.43:0/3568372919 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8010960 con 0x5557bff13410 2024-04-25T10:41:20.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.294+0000 7f7afb97b740 1 -- 172.21.15.43:0/3568372919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 msgr2=0x5557c02d7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:20.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.294+0000 7f7afb97b740 1 --2- 172.21.15.43:0/3568372919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 0x5557c02d7840 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8009da0 tx=0x7f7ad8009e70 comp rx=0 tx=0).stop 2024-04-25T10:41:20.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.294+0000 7f7afb97b740 1 -- 172.21.15.43:0/3568372919 shutdown_connections 2024-04-25T10:41:20.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.294+0000 7f7afb97b740 1 --2- 172.21.15.43:0/3568372919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557bfef2730 0x5557c02d7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.294+0000 7f7afb97b740 1 --2- 172.21.15.43:0/3568372919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bff13410 0x5557c02d7840 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.294+0000 7f7afb97b740 1 --2- 172.21.15.43:0/3568372919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bfeea550 0x5557c02c3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.294+0000 7f7afb97b740 1 -- 172.21.15.43:0/3568372919 >> 172.21.15.43:0/3568372919 conn(0x5557bff07560 msgr2=0x5557c02c3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:20.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.294+0000 7f7afb97b740 1 -- 172.21.15.43:0/3568372919 shutdown_connections 2024-04-25T10:41:20.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.294+0000 7f7afb97b740 1 -- 172.21.15.43:0/3568372919 wait complete. 2024-04-25T10:41:20.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.294+0000 7f7afb97b740 1 -- start start 2024-04-25T10:41:20.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.294+0000 7f7afb97b740 1 Processor -- start 2024-04-25T10:41:20.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.295+0000 7f7afb97b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557bfeea550 0x5557c02d6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.295+0000 7f7afb97b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bfef2730 0x5557c02d6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.295+0000 7f7afb97b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bff13410 0x5557c0334520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.295+0000 7f7afb97b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7af01abeb0 con 0x5557bfef2730 2024-04-25T10:41:20.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.295+0000 7f7afb97b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7af01ad300 con 0x5557bfeea550 2024-04-25T10:41:20.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.295+0000 7f7afb97b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7af01ac560 con 0x5557bff13410 2024-04-25T10:41:20.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.295+0000 7f7afb97b740 10 client.0 _ll_register_callbacks cb 0x7f7ad8017380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:20.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.295+0000 7f7aee59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bfef2730 0x5557c02d6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.295+0000 7f7aee59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bfef2730 0x5557c02d6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48934/0 (socket says 172.21.15.43:48934) 2024-04-25T10:41:20.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.295+0000 7f7aee59c640 1 -- 172.21.15.43:0/1801623627 learned_addr learned my addr 172.21.15.43:0/1801623627 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:20.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.295+0000 7f7ae5d9b640 1 --2- 172.21.15.43:0/1801623627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557bfeea550 0x5557c02d6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.295+0000 7f7ae5d9b640 1 --2- 172.21.15.43:0/1801623627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bff13410 0x5557c0334520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.295+0000 7f7aee59c640 1 -- 172.21.15.43:0/1801623627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bff13410 msgr2=0x5557c0334520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:20.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.295+0000 7f7aee59c640 1 --2- 172.21.15.43:0/1801623627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bff13410 0x5557c0334520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.295+0000 7f7aee59c640 1 -- 172.21.15.43:0/1801623627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557bfeea550 msgr2=0x5557c02d6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:20.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.295+0000 7f7aee59c640 1 --2- 172.21.15.43:0/1801623627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557bfeea550 0x5557c02d6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.295+0000 7f7aee59c640 1 -- 172.21.15.43:0/1801623627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557c0334ae0 con 0x5557bfef2730 2024-04-25T10:41:20.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.295+0000 7f7aee59c640 1 --2- 172.21.15.43:0/1801623627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bfef2730 0x5557c02d6dd0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0033000 tx=0x7f7ae0004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4955dcf426bdcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:20.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.295+0000 7f7ad5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:20.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.296+0000 7f7ad5ffb640 1 -- 172.21.15.43:0/1801623627 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0048070 con 0x5557bfef2730 2024-04-25T10:41:20.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.296+0000 7f7ad5ffb640 1 -- 172.21.15.43:0/1801623627 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f7ae00316a0 con 0x5557bfef2730 2024-04-25T10:41:20.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.296+0000 7f7afb97b740 1 -- 172.21.15.43:0/1801623627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ad801d070 con 0x5557bfef2730 2024-04-25T10:41:20.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.296+0000 7f7ad5ffb640 1 -- 172.21.15.43:0/1801623627 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7ae0053050 con 0x5557bfef2730 2024-04-25T10:41:20.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.296+0000 7f7afb97b740 10 client.18150 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:20.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.296+0000 7f7afb97b740 1 -- 172.21.15.43:0/1801623627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5557c033bea0 con 0x5557bfef2730 2024-04-25T10:41:20.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.296+0000 7f7ad5ffb640 1 -- 172.21.15.43:0/1801623627 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ad801d070 con 0x5557bfef2730 2024-04-25T10:41:20.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.296+0000 7f7ad5ffb640 1 -- 172.21.15.43:0/1801623627 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7ae0003710 con 0x5557bfef2730 2024-04-25T10:41:20.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.296+0000 7f7afb97b740 10 client.18150 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:20.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.296+0000 7f7afb97b740 10 client.18150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:20.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.296+0000 7f7afb97b740 20 client.18150 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:20.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.296+0000 7f7afb97b740 1 --2- 172.21.15.43:0/1801623627 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5557bfea7630 0x5557c033e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.296+0000 7f7afb97b740 4 client.18150 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:41:20.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.296+0000 7f7afb97b740 1 -- 172.21.15.43:0/1801623627 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5557c0221580 con 0x5557bfea7630 2024-04-25T10:41:20.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.296+0000 7f7ae5d9b640 1 --2- 172.21.15.43:0/1801623627 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5557bfea7630 0x5557c033e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.297+0000 7f7ad5ffb640 10 client.18150 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:20.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.297+0000 7f7ae5d9b640 1 --2- 172.21.15.43:0/1801623627 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5557bfea7630 0x5557c033e670 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18b25539e529abc0 server_cookie=57f36474f8c449a2 in_seq=0 out_seq=0 2024-04-25T10:41:20.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.298+0000 7f7ad5ffb640 1 -- 172.21.15.43:0/1801623627 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5557c0221580 con 0x5557bfea7630 2024-04-25T10:41:20.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.298+0000 7f7ad5ffb640 10 client.18150 handle_command_reply: tid=0 2024-04-25T10:41:20.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.298+0000 7f7afb97b740 1 client.18150 shutdown 2024-04-25T10:41:20.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.298+0000 7f7afb97b740 1 -- 172.21.15.43:0/1801623627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bfef2730 msgr2=0x5557c02d6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:20.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.298+0000 7f7afb97b740 1 --2- 172.21.15.43:0/1801623627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bfef2730 0x5557c02d6dd0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0033000 tx=0x7f7ae0004200 comp rx=0 tx=0).stop 2024-04-25T10:41:20.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.298+0000 7f7afb97b740 1 -- 172.21.15.43:0/1801623627 shutdown_connections 2024-04-25T10:41:20.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.298+0000 7f7afb97b740 1 --2- 172.21.15.43:0/1801623627 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5557bfea7630 0x5557c033e670 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.299+0000 7f7afb97b740 1 --2- 172.21.15.43:0/1801623627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557bff13410 0x5557c0334520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.299+0000 7f7afb97b740 1 --2- 172.21.15.43:0/1801623627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557bfef2730 0x5557c02d6dd0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.299+0000 7f7afb97b740 1 --2- 172.21.15.43:0/1801623627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557bfeea550 0x5557c02d6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.299+0000 7f7afb97b740 1 -- 172.21.15.43:0/1801623627 >> 172.21.15.43:0/1801623627 conn(0x5557bff07560 msgr2=0x5557c02c3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:20.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.299+0000 7f7ad5ffb640 0 client.18150 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:20.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.299+0000 7f7afb97b740 1 -- 172.21.15.43:0/1801623627 shutdown_connections 2024-04-25T10:41:20.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.299+0000 7f7afb97b740 1 -- 172.21.15.43:0/1801623627 wait complete. 2024-04-25T10:41:20.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.299+0000 7f7afb97b740 20 client.18150 trim_cache size 0 max 16384 2024-04-25T10:41:20.307 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:41:20.307 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:41:20.307 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "2e54c860-ffc2-42fe-b614-9ca3875804c5", 2024-04-25T10:41:20.307 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:41:20.307 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:41:20.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.299+0000 7f7af49f3640 1 -- 172.21.15.43:0/4181928595 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7adc0afc60 msgr2=0x7f7adc0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:20.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.299+0000 7f7af49f3640 1 --2- 172.21.15.43:0/4181928595 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7adc0afc60 0x7f7adc0b2170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80a44c0 tx=0x7f7ae8074ab0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.299+0000 7f7af49f3640 1 -- 172.21.15.43:0/4181928595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7af010e090 msgr2=0x7f7af019efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:20.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.299+0000 7f7af49f3640 1 --2- 172.21.15.43:0/4181928595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7af010e090 0x7f7af019efd0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800cad0 tx=0x7f7ae800ced0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.299+0000 7f7af49f3640 1 -- 172.21.15.43:0/4181928595 shutdown_connections 2024-04-25T10:41:20.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.299+0000 7f7af49f3640 1 --2- 172.21.15.43:0/4181928595 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7adc0afc60 0x7f7adc0b2170 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.299+0000 7f7af49f3640 1 --2- 172.21.15.43:0/4181928595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7af010b6e0 0x7f7af019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.299+0000 7f7af49f3640 1 --2- 172.21.15.43:0/4181928595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7af010e090 0x7f7af019efd0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.299+0000 7f7af49f3640 1 --2- 172.21.15.43:0/4181928595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7af0104c80 0x7f7af019a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.299+0000 7f7af49f3640 1 -- 172.21.15.43:0/4181928595 >> 172.21.15.43:0/4181928595 conn(0x7f7af0078180 msgr2=0x7f7af01011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:20.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.299+0000 7f7af49f3640 1 -- 172.21.15.43:0/4181928595 shutdown_connections 2024-04-25T10:41:20.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.300+0000 7f7af49f3640 1 -- 172.21.15.43:0/4181928595 wait complete. 2024-04-25T10:41:20.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:19 smithi043 ceph-mon[29478]: pgmap v273: 97 pgs: 97 active+clean; 3.8 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-25T10:41:20.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:41:20.313 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:41:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.424+0000 7fbde07a8640 1 -- 172.21.15.43:0/3084960731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdd81031c0 msgr2=0x7fbdd810f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.424+0000 7fbde07a8640 1 --2- 172.21.15.43:0/3084960731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdd81031c0 0x7fbdd810f4a0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009ca0 tx=0x7fbdd002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.425+0000 7fbde07a8640 1 -- 172.21.15.43:0/3084960731 shutdown_connections 2024-04-25T10:41:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.425+0000 7fbde07a8640 1 --2- 172.21.15.43:0/3084960731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdd81031c0 0x7fbdd810f4a0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.425+0000 7fbde07a8640 1 --2- 172.21.15.43:0/3084960731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbdd8108eb0 0x7fbdd8102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.425+0000 7fbde07a8640 1 --2- 172.21.15.43:0/3084960731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbdd8108a10 0x7fbdd8102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.425+0000 7fbde07a8640 1 -- 172.21.15.43:0/3084960731 >> 172.21.15.43:0/3084960731 conn(0x7fbdd8078110 msgr2=0x7fbdd8078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.425+0000 7fbde07a8640 1 -- 172.21.15.43:0/3084960731 shutdown_connections 2024-04-25T10:41:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.425+0000 7fbde07a8640 1 -- 172.21.15.43:0/3084960731 wait complete. 2024-04-25T10:41:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.425+0000 7fbde07a8640 1 Processor -- start 2024-04-25T10:41:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.425+0000 7fbde07a8640 1 -- start start 2024-04-25T10:41:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.426+0000 7fbde07a8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbdd81031c0 0x7fbdd819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.426+0000 7fbde07a8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbdd8108a10 0x7fbdd819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.426+0000 7fbde07a8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdd8108eb0 0x7fbdd819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.426+0000 7fbde07a8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd8113d50 con 0x7fbdd8108eb0 2024-04-25T10:41:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.426+0000 7fbde07a8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd8113bd0 con 0x7fbdd8108a10 2024-04-25T10:41:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.426+0000 7fbde07a8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd8113ed0 con 0x7fbdd81031c0 2024-04-25T10:41:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.426+0000 7fbdded45640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdd8108eb0 0x7fbdd819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.426+0000 7fbdded45640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdd8108eb0 0x7fbdd819ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48954/0 (socket says 172.21.15.43:48954) 2024-04-25T10:41:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.426+0000 7fbdded45640 1 -- 172.21.15.43:0/3853920733 learned_addr learned my addr 172.21.15.43:0/3853920733 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.426+0000 7fbdde544640 1 --2- 172.21.15.43:0/3853920733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbdd81031c0 0x7fbdd819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.426+0000 7fbdddd43640 1 --2- 172.21.15.43:0/3853920733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbdd8108a10 0x7fbdd819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.426+0000 7fbdded45640 1 -- 172.21.15.43:0/3853920733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbdd81031c0 msgr2=0x7fbdd819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:20.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.426+0000 7fbdded45640 1 --2- 172.21.15.43:0/3853920733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbdd81031c0 0x7fbdd819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.426+0000 7fbdded45640 1 -- 172.21.15.43:0/3853920733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbdd8108a10 msgr2=0x7fbdd819ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:20.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.426+0000 7fbdded45640 1 --2- 172.21.15.43:0/3853920733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbdd8108a10 0x7fbdd819ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.426+0000 7fbdded45640 1 -- 172.21.15.43:0/3853920733 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd819f4f0 con 0x7fbdd8108eb0 2024-04-25T10:41:20.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.426+0000 7fbdde544640 1 --2- 172.21.15.43:0/3853920733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbdd81031c0 0x7fbdd819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:20.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.426+0000 7fbdddd43640 1 --2- 172.21.15.43:0/3853920733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbdd8108a10 0x7fbdd819ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:41:20.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.426+0000 7fbdded45640 1 --2- 172.21.15.43:0/3853920733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdd8108eb0 0x7fbdd819ef30 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0006010 tx=0x7fbdd002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:20.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.427+0000 7fbdcf7fe640 1 -- 172.21.15.43:0/3853920733 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0003bf0 con 0x7fbdd8108eb0 2024-04-25T10:41:20.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.427+0000 7fbde07a8640 1 -- 172.21.15.43:0/3853920733 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd819f780 con 0x7fbdd8108eb0 2024-04-25T10:41:20.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.427+0000 7fbdcf7fe640 1 -- 172.21.15.43:0/3853920733 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd0003d90 con 0x7fbdd8108eb0 2024-04-25T10:41:20.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.427+0000 7fbdcf7fe640 1 -- 172.21.15.43:0/3853920733 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd002fba0 con 0x7fbdd8108eb0 2024-04-25T10:41:20.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.427+0000 7fbde07a8640 1 -- 172.21.15.43:0/3853920733 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd819fcd0 con 0x7fbdd8108eb0 2024-04-25T10:41:20.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.429+0000 7fbde7655740 1 Processor -- start 2024-04-25T10:41:20.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.429+0000 7fbdcf7fe640 1 -- 172.21.15.43:0/3853920733 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fbdd002fd40 con 0x7fbdd8108eb0 2024-04-25T10:41:20.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.429+0000 7fbde7655740 1 -- start start 2024-04-25T10:41:20.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbde7655740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e1140390 0x5579e14e0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbde7655740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e11174d0 0x5579e14e0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbde7655740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579e111f6b0 0x5579e14ed8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbde7655740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5579e1446d80 con 0x5579e11174d0 2024-04-25T10:41:20.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbde7655740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5579e14008b0 con 0x5579e1140390 2024-04-25T10:41:20.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbde7655740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5579e1442d40 con 0x5579e111f6b0 2024-04-25T10:41:20.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbdde544640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e11174d0 0x5579e14e0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbdde544640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e11174d0 0x5579e14e0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48958/0 (socket says 172.21.15.43:48958) 2024-04-25T10:41:20.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbdde544640 1 -- 172.21.15.43:0/3847869411 learned_addr learned my addr 172.21.15.43:0/3847869411 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:20.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbdddd43640 1 --2- 172.21.15.43:0/3847869411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e1140390 0x5579e14e0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbdcf7fe640 1 --2- 172.21.15.43:0/3853920733 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fbdc40afc60 0x7fbdc40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbdddd43640 1 --2- 172.21.15.43:0/3847869411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579e111f6b0 0x5579e14ed8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbdde544640 1 -- 172.21.15.43:0/3847869411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579e111f6b0 msgr2=0x5579e14ed8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:20.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbdded45640 1 --2- 172.21.15.43:0/3853920733 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fbdc40afc60 0x7fbdc40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbdde544640 1 --2- 172.21.15.43:0/3847869411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579e111f6b0 0x5579e14ed8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbdde544640 1 -- 172.21.15.43:0/3847869411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e1140390 msgr2=0x5579e14e0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:20.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbdde544640 1 --2- 172.21.15.43:0/3847869411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e1140390 0x5579e14e0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbdde544640 1 -- 172.21.15.43:0/3847869411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579e13d1a40 con 0x5579e11174d0 2024-04-25T10:41:20.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbdddd43640 1 --2- 172.21.15.43:0/3847869411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579e111f6b0 0x5579e14ed8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:41:20.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbdcf7fe640 1 -- 172.21.15.43:0/3853920733 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fbdd0100770 con 0x7fbdd8108eb0 2024-04-25T10:41:20.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.430+0000 7fbdde544640 1 --2- 172.21.15.43:0/3847869411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e11174d0 0x5579e14e0fa0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4008c00 tx=0x7fbdd400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f1c994260ca33c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:20.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.431+0000 7fbdded45640 1 --2- 172.21.15.43:0/3853920733 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fbdc40afc60 0x7fbdc40b2170 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00fe4c0 tx=0x7fbdd00bd310 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:20.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.431+0000 7fbdccff9640 1 -- 172.21.15.43:0/3847869411 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4014070 con 0x5579e11174d0 2024-04-25T10:41:20.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.431+0000 7fbdccff9640 1 -- 172.21.15.43:0/3847869411 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd40047a0 con 0x5579e11174d0 2024-04-25T10:41:20.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.431+0000 7fbdccff9640 1 -- 172.21.15.43:0/3847869411 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4010690 con 0x5579e11174d0 2024-04-25T10:41:20.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.431+0000 7fbde7655740 1 -- 172.21.15.43:0/3847869411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e11174d0 msgr2=0x5579e14e0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:20.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.431+0000 7fbde7655740 1 --2- 172.21.15.43:0/3847869411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e11174d0 0x5579e14e0fa0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4008c00 tx=0x7fbdd400e720 comp rx=0 tx=0).stop 2024-04-25T10:41:20.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.432+0000 7fbde7655740 1 -- 172.21.15.43:0/3847869411 shutdown_connections 2024-04-25T10:41:20.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.432+0000 7fbde7655740 1 --2- 172.21.15.43:0/3847869411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579e111f6b0 0x5579e14ed8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.432+0000 7fbde7655740 1 --2- 172.21.15.43:0/3847869411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e11174d0 0x5579e14e0fa0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.432+0000 7fbde7655740 1 --2- 172.21.15.43:0/3847869411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e1140390 0x5579e14e0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.432+0000 7fbde7655740 1 -- 172.21.15.43:0/3847869411 >> 172.21.15.43:0/3847869411 conn(0x5579e11344e0 msgr2=0x5579e14dd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:20.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.432+0000 7fbde7655740 1 -- 172.21.15.43:0/3847869411 shutdown_connections 2024-04-25T10:41:20.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.432+0000 7fbde7655740 1 -- 172.21.15.43:0/3847869411 wait complete. 2024-04-25T10:41:20.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.432+0000 7fbde7655740 1 -- start start 2024-04-25T10:41:20.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.432+0000 7fbde7655740 1 Processor -- start 2024-04-25T10:41:20.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.432+0000 7fbde7655740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e11174d0 0x5579e14eb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.432+0000 7fbde7655740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579e111f6b0 0x5579e14ebbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.432+0000 7fbde7655740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e1140390 0x5579e14f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.432+0000 7fbde7655740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5579e14edfe0 con 0x5579e11174d0 2024-04-25T10:41:20.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.432+0000 7fbde7655740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5579e13cdd10 con 0x5579e1140390 2024-04-25T10:41:20.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.432+0000 7fbde7655740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5579e14ee160 con 0x5579e111f6b0 2024-04-25T10:41:20.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.432+0000 7fbde7655740 10 client.0 _ll_register_callbacks cb 0x5579e13fa210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:20.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.433+0000 7fbdddd43640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e11174d0 0x5579e14eb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.433+0000 7fbdddd43640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e11174d0 0x5579e14eb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48966/0 (socket says 172.21.15.43:48966) 2024-04-25T10:41:20.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.433+0000 7fbdddd43640 1 -- 172.21.15.43:0/3761122600 learned_addr learned my addr 172.21.15.43:0/3761122600 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:20.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.433+0000 7fbdde544640 1 --2- 172.21.15.43:0/3761122600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579e111f6b0 0x5579e14ebbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.433+0000 7fbdddd43640 1 --2- 172.21.15.43:0/3761122600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e1140390 0x5579e14f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.433+0000 7fbdddd43640 1 -- 172.21.15.43:0/3761122600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579e111f6b0 msgr2=0x5579e14ebbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:20.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.433+0000 7fbdddd43640 1 --2- 172.21.15.43:0/3761122600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579e111f6b0 0x5579e14ebbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.433+0000 7fbdddd43640 1 -- 172.21.15.43:0/3761122600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e1140390 msgr2=0x5579e14f14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:20.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.433+0000 7fbdddd43640 1 --2- 172.21.15.43:0/3761122600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e1140390 0x5579e14f14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.433+0000 7fbdddd43640 1 -- 172.21.15.43:0/3761122600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579e12227d0 con 0x5579e11174d0 2024-04-25T10:41:20.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.433+0000 7fbdddd43640 1 --2- 172.21.15.43:0/3761122600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e11174d0 0x5579e14eb690 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8100130 tx=0x7fbdc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1b9908d34d8da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:20.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.433+0000 7fbdc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:20.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.433+0000 7fbdc1ffb640 1 -- 172.21.15.43:0/3761122600 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8011d40 con 0x5579e11174d0 2024-04-25T10:41:20.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.433+0000 7fbdc1ffb640 1 -- 172.21.15.43:0/3761122600 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fbdc8005020 con 0x5579e11174d0 2024-04-25T10:41:20.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.433+0000 7fbde7655740 1 -- 172.21.15.43:0/3761122600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdd40047a0 con 0x5579e11174d0 2024-04-25T10:41:20.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.433+0000 7fbdc1ffb640 1 -- 172.21.15.43:0/3761122600 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbdc8005220 con 0x5579e11174d0 2024-04-25T10:41:20.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.433+0000 7fbde7655740 10 client.18162 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:20.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.433+0000 7fbde7655740 1 -- 172.21.15.43:0/3761122600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5579e11583f0 con 0x5579e11174d0 2024-04-25T10:41:20.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.434+0000 7fbdc1ffb640 1 -- 172.21.15.43:0/3761122600 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd40047a0 con 0x5579e11174d0 2024-04-25T10:41:20.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.434+0000 7fbdc1ffb640 1 -- 172.21.15.43:0/3761122600 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbdc800f040 con 0x5579e11174d0 2024-04-25T10:41:20.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.434+0000 7fbde7655740 10 client.18162 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:20.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.434+0000 7fbde7655740 10 client.18162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:20.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.434+0000 7fbde7655740 20 client.18162 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:20.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.434+0000 7fbde7655740 1 --2- 172.21.15.43:0/3761122600 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5579e10fef60 0x5579e14fd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.434+0000 7fbde7655740 4 client.18162 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:41:20.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.434+0000 7fbde7655740 1 -- 172.21.15.43:0/3761122600 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579e1454990 con 0x5579e10fef60 2024-04-25T10:41:20.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.434+0000 7fbdde544640 1 --2- 172.21.15.43:0/3761122600 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5579e10fef60 0x5579e14fd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.435+0000 7fbdde544640 1 --2- 172.21.15.43:0/3761122600 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5579e10fef60 0x5579e14fd5f0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6e0833660107ca server_cookie=c855dcf058e40b5b in_seq=0 out_seq=0 2024-04-25T10:41:20.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.435+0000 7fbdc1ffb640 10 client.18162 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:20.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.435+0000 7fbdc1ffb640 1 -- 172.21.15.43:0/3761122600 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5579e1454990 con 0x5579e10fef60 2024-04-25T10:41:20.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.435+0000 7fbdc1ffb640 10 client.18162 handle_command_reply: tid=0 2024-04-25T10:41:20.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.435+0000 7fbde7655740 1 client.18162 shutdown 2024-04-25T10:41:20.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.435+0000 7fbde7655740 1 -- 172.21.15.43:0/3761122600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e11174d0 msgr2=0x5579e14eb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:20.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.435+0000 7fbde7655740 1 --2- 172.21.15.43:0/3761122600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e11174d0 0x5579e14eb690 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8100130 tx=0x7fbdc8004730 comp rx=0 tx=0).stop 2024-04-25T10:41:20.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.435+0000 7fbde7655740 1 -- 172.21.15.43:0/3761122600 shutdown_connections 2024-04-25T10:41:20.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.435+0000 7fbde7655740 1 --2- 172.21.15.43:0/3761122600 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5579e10fef60 0x5579e14fd5f0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.435+0000 7fbde7655740 1 --2- 172.21.15.43:0/3761122600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e1140390 0x5579e14f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.435+0000 7fbde7655740 1 --2- 172.21.15.43:0/3761122600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579e111f6b0 0x5579e14ebbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.435+0000 7fbdc1ffb640 0 client.18162 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:20.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.435+0000 7fbde7655740 1 --2- 172.21.15.43:0/3761122600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e11174d0 0x5579e14eb690 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.435+0000 7fbde7655740 1 -- 172.21.15.43:0/3761122600 >> 172.21.15.43:0/3761122600 conn(0x5579e11344e0 msgr2=0x5579e14e2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:20.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.435+0000 7fbde7655740 1 -- 172.21.15.43:0/3761122600 shutdown_connections 2024-04-25T10:41:20.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.436+0000 7fbde7655740 1 -- 172.21.15.43:0/3761122600 wait complete. 2024-04-25T10:41:20.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.436+0000 7fbde7655740 20 client.18162 trim_cache size 0 max 16384 2024-04-25T10:41:20.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.454+0000 7fbde7655740 1 Processor -- start 2024-04-25T10:41:20.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.454+0000 7fbde7655740 1 -- start start 2024-04-25T10:41:20.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.454+0000 7fbde7655740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e11174d0 0x5579e14e9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.454+0000 7fbde7655740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e111f6b0 0x5579e14e0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.454+0000 7fbde7655740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579e1140390 0x5579e14e2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.454+0000 7fbde7655740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd81b2260 con 0x5579e111f6b0 2024-04-25T10:41:20.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.454+0000 7fbde7655740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd81afb30 con 0x5579e11174d0 2024-04-25T10:41:20.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.454+0000 7fbde7655740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd81ad9a0 con 0x5579e1140390 2024-04-25T10:41:20.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.454+0000 7fbdde544640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e111f6b0 0x5579e14e0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.454+0000 7fbdde544640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e111f6b0 0x5579e14e0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48974/0 (socket says 172.21.15.43:48974) 2024-04-25T10:41:20.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.454+0000 7fbdde544640 1 -- 172.21.15.43:0/4124794711 learned_addr learned my addr 172.21.15.43:0/4124794711 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:20.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.454+0000 7fbdddd43640 1 --2- 172.21.15.43:0/4124794711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e11174d0 0x5579e14e9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.454+0000 7fbdddd43640 1 --2- 172.21.15.43:0/4124794711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579e1140390 0x5579e14e2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.455+0000 7fbdde544640 1 -- 172.21.15.43:0/4124794711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579e1140390 msgr2=0x5579e14e2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:20.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.455+0000 7fbdde544640 1 --2- 172.21.15.43:0/4124794711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579e1140390 0x5579e14e2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.455+0000 7fbdde544640 1 -- 172.21.15.43:0/4124794711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e11174d0 msgr2=0x5579e14e9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:20.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.455+0000 7fbdde544640 1 --2- 172.21.15.43:0/4124794711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e11174d0 0x5579e14e9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.455+0000 7fbdde544640 1 -- 172.21.15.43:0/4124794711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579e11180f0 con 0x5579e111f6b0 2024-04-25T10:41:20.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.455+0000 7fbdddd43640 1 --2- 172.21.15.43:0/4124794711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e11174d0 0x5579e14e9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:20.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.455+0000 7fbdde544640 1 --2- 172.21.15.43:0/4124794711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e111f6b0 0x5579e14e0e60 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400efe0 tx=0x7fbdd4008f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6a22e7d28ded88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:20.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.455+0000 7fbdc3fff640 1 -- 172.21.15.43:0/4124794711 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd402c070 con 0x5579e111f6b0 2024-04-25T10:41:20.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.455+0000 7fbdc3fff640 1 -- 172.21.15.43:0/4124794711 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd4004240 con 0x5579e111f6b0 2024-04-25T10:41:20.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.455+0000 7fbdc3fff640 1 -- 172.21.15.43:0/4124794711 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4014070 con 0x5579e111f6b0 2024-04-25T10:41:20.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.456+0000 7fbde7655740 1 -- 172.21.15.43:0/4124794711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e111f6b0 msgr2=0x5579e14e0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:20.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.456+0000 7fbde7655740 1 --2- 172.21.15.43:0/4124794711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e111f6b0 0x5579e14e0e60 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400efe0 tx=0x7fbdd4008f00 comp rx=0 tx=0).stop 2024-04-25T10:41:20.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.456+0000 7fbde7655740 1 -- 172.21.15.43:0/4124794711 shutdown_connections 2024-04-25T10:41:20.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.456+0000 7fbde7655740 1 --2- 172.21.15.43:0/4124794711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579e1140390 0x5579e14e2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.456+0000 7fbde7655740 1 --2- 172.21.15.43:0/4124794711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e111f6b0 0x5579e14e0e60 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.456+0000 7fbde7655740 1 --2- 172.21.15.43:0/4124794711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e11174d0 0x5579e14e9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.456+0000 7fbde7655740 1 -- 172.21.15.43:0/4124794711 >> 172.21.15.43:0/4124794711 conn(0x5579e11344e0 msgr2=0x5579e14e2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:20.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.456+0000 7fbde7655740 1 -- 172.21.15.43:0/4124794711 shutdown_connections 2024-04-25T10:41:20.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.456+0000 7fbde7655740 1 -- 172.21.15.43:0/4124794711 wait complete. 2024-04-25T10:41:20.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.456+0000 7fbde7655740 1 -- start start 2024-04-25T10:41:20.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.456+0000 7fbde7655740 1 Processor -- start 2024-04-25T10:41:20.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbde7655740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579e11174d0 0x5579e14f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbde7655740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e111f6b0 0x5579e14f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbde7655740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e1140390 0x5579e14f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbde7655740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd81adff0 con 0x5579e1140390 2024-04-25T10:41:20.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbde7655740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd81af460 con 0x5579e111f6b0 2024-04-25T10:41:20.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbde7655740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd81ae6c0 con 0x5579e11174d0 2024-04-25T10:41:20.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbde7655740 10 client.0 _ll_register_callbacks cb 0x7fbdc800b7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:20.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbdddd43640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e1140390 0x5579e14f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbdddd43640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e1140390 0x5579e14f2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48980/0 (socket says 172.21.15.43:48980) 2024-04-25T10:41:20.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbdddd43640 1 -- 172.21.15.43:0/3325767967 learned_addr learned my addr 172.21.15.43:0/3325767967 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:20.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbdde544640 1 --2- 172.21.15.43:0/3325767967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e111f6b0 0x5579e14f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbdddd43640 1 -- 172.21.15.43:0/3325767967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579e11174d0 msgr2=0x5579e14f7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:20.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbdddd43640 1 --2- 172.21.15.43:0/3325767967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579e11174d0 0x5579e14f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbdddd43640 1 -- 172.21.15.43:0/3325767967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e111f6b0 msgr2=0x5579e14f80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:20.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbdddd43640 1 --2- 172.21.15.43:0/3325767967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e111f6b0 0x5579e14f80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbdddd43640 1 -- 172.21.15.43:0/3325767967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579e14074b0 con 0x5579e1140390 2024-04-25T10:41:20.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbdddd43640 1 --2- 172.21.15.43:0/3325767967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e1140390 0x5579e14f2ed0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8001eb0 tx=0x7fbdc800b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62deda689e59d47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:20.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbdc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:20.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbdc1ffb640 1 -- 172.21.15.43:0/3325767967 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8011d40 con 0x5579e1140390 2024-04-25T10:41:20.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbde7655740 1 -- 172.21.15.43:0/3325767967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdd4004240 con 0x5579e1140390 2024-04-25T10:41:20.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.457+0000 7fbdc1ffb640 1 -- 172.21.15.43:0/3325767967 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fbdc8026640 con 0x5579e1140390 2024-04-25T10:41:20.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.458+0000 7fbdc1ffb640 1 -- 172.21.15.43:0/3325767967 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbdc8026840 con 0x5579e1140390 2024-04-25T10:41:20.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.458+0000 7fbde7655740 10 client.18168 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:20.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.458+0000 7fbde7655740 1 -- 172.21.15.43:0/3325767967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5579e1569620 con 0x5579e1140390 2024-04-25T10:41:20.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.458+0000 7fbdc1ffb640 1 -- 172.21.15.43:0/3325767967 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd4004240 con 0x5579e1140390 2024-04-25T10:41:20.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.458+0000 7fbdc1ffb640 1 -- 172.21.15.43:0/3325767967 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbdc8032030 con 0x5579e1140390 2024-04-25T10:41:20.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.458+0000 7fbde7655740 10 client.18168 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:20.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.458+0000 7fbde7655740 10 client.18168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:20.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.458+0000 7fbde7655740 20 client.18168 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:20.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.458+0000 7fbde7655740 1 --2- 172.21.15.43:0/3325767967 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5579e10d43a0 0x5579e156be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:20.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.458+0000 7fbde7655740 4 client.18168 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:41:20.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.458+0000 7fbde7655740 1 -- 172.21.15.43:0/3325767967 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5579e144e800 con 0x5579e10d43a0 2024-04-25T10:41:20.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.458+0000 7fbdde544640 1 --2- 172.21.15.43:0/3325767967 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5579e10d43a0 0x5579e156be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:20.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.459+0000 7fbdc1ffb640 10 client.18168 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:20.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.459+0000 7fbdde544640 1 --2- 172.21.15.43:0/3325767967 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5579e10d43a0 0x5579e156be10 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3700fb796732cf9 server_cookie=ebbfc6a7c81ae6df in_seq=0 out_seq=0 2024-04-25T10:41:20.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.459+0000 7fbdc1ffb640 1 -- 172.21.15.43:0/3325767967 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5579e144e800 con 0x5579e10d43a0 2024-04-25T10:41:20.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.459+0000 7fbdc1ffb640 10 client.18168 handle_command_reply: tid=0 2024-04-25T10:41:20.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.459+0000 7fbde7655740 1 client.18168 shutdown 2024-04-25T10:41:20.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.459+0000 7fbde7655740 1 -- 172.21.15.43:0/3325767967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e1140390 msgr2=0x5579e14f2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:20.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.459+0000 7fbde7655740 1 --2- 172.21.15.43:0/3325767967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e1140390 0x5579e14f2ed0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8001eb0 tx=0x7fbdc800b580 comp rx=0 tx=0).stop 2024-04-25T10:41:20.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.459+0000 7fbde7655740 1 -- 172.21.15.43:0/3325767967 shutdown_connections 2024-04-25T10:41:20.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.459+0000 7fbde7655740 1 --2- 172.21.15.43:0/3325767967 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5579e10d43a0 0x5579e156be10 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.459+0000 7fbde7655740 1 --2- 172.21.15.43:0/3325767967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e111f6b0 0x5579e14f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.459+0000 7fbde7655740 1 --2- 172.21.15.43:0/3325767967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e1140390 0x5579e14f2ed0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.459+0000 7fbdc1ffb640 0 client.18168 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:20.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.459+0000 7fbde7655740 1 --2- 172.21.15.43:0/3325767967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579e11174d0 0x5579e14f7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.459+0000 7fbde7655740 1 -- 172.21.15.43:0/3325767967 >> 172.21.15.43:0/3325767967 conn(0x5579e11344e0 msgr2=0x5579e14ec940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:20.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.459+0000 7fbde7655740 1 -- 172.21.15.43:0/3325767967 shutdown_connections 2024-04-25T10:41:20.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.459+0000 7fbde7655740 1 -- 172.21.15.43:0/3325767967 wait complete. 2024-04-25T10:41:20.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.459+0000 7fbde7655740 20 client.18168 trim_cache size 0 max 16384 2024-04-25T10:41:20.468 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:41:20.468 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:41:20.468 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:41:20.468 INFO:teuthology.orchestra.run.smithi043.stdout: "2e54c860-ffc2-42fe-b614-9ca3875804c5": { 2024-04-25T10:41:20.468 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:41:20.468 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "2e54c860-ffc2-42fe-b614-9ca3875804c5", 2024-04-25T10:41:20.468 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:41:20.469 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:41:20.469 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:41:20.469 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:41:20.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.460+0000 7fbde07a8640 1 -- 172.21.15.43:0/3853920733 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fbdc40afc60 msgr2=0x7fbdc40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:20.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.460+0000 7fbde07a8640 1 --2- 172.21.15.43:0/3853920733 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fbdc40afc60 0x7fbdc40b2170 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00fe4c0 tx=0x7fbdd00bd310 comp rx=0 tx=0).stop 2024-04-25T10:41:20.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.460+0000 7fbde07a8640 1 -- 172.21.15.43:0/3853920733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdd8108eb0 msgr2=0x7fbdd819ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:20.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.460+0000 7fbde07a8640 1 --2- 172.21.15.43:0/3853920733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdd8108eb0 0x7fbdd819ef30 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0006010 tx=0x7fbdd002f380 comp rx=0 tx=0).stop 2024-04-25T10:41:20.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.460+0000 7fbde07a8640 1 -- 172.21.15.43:0/3853920733 shutdown_connections 2024-04-25T10:41:20.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.460+0000 7fbde07a8640 1 --2- 172.21.15.43:0/3853920733 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fbdc40afc60 0x7fbdc40b2170 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.460+0000 7fbde07a8640 1 --2- 172.21.15.43:0/3853920733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbdd8108a10 0x7fbdd819ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.460+0000 7fbde07a8640 1 --2- 172.21.15.43:0/3853920733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdd8108eb0 0x7fbdd819ef30 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.460+0000 7fbde07a8640 1 --2- 172.21.15.43:0/3853920733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbdd81031c0 0x7fbdd819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:20.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.460+0000 7fbde07a8640 1 -- 172.21.15.43:0/3853920733 >> 172.21.15.43:0/3853920733 conn(0x7fbdd8078110 msgr2=0x7fbdd8100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:20.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.460+0000 7fbde07a8640 1 -- 172.21.15.43:0/3853920733 shutdown_connections 2024-04-25T10:41:20.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:20.460+0000 7fbde07a8640 1 -- 172.21.15.43:0/3853920733 wait complete. 2024-04-25T10:41:20.473 INFO:tasks.cephfs.filesystem:scrub status for tag:2e54c860-ffc2-42fe-b614-9ca3875804c5 - {'path': '/', 'tag': '2e54c860-ffc2-42fe-b614-9ca3875804c5', 'options': 'recursive,force'} 2024-04-25T10:41:21.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:20 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:41:21.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:20 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:41:21.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:20 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:41:21.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:20 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:41:21.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:20 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:41:21.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:20 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:41:21.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:20 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:41:21.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:20 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:41:21.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:20 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:41:21.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:20 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:41:21.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:20 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:41:21.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:20 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:41:22.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:21 smithi077 ceph-mon[32772]: pgmap v274: 97 pgs: 97 active+clean; 3.8 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-25T10:41:22.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:21 smithi164 ceph-mon[32713]: pgmap v274: 97 pgs: 97 active+clean; 3.8 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-25T10:41:22.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:21 smithi043 ceph-mon[29478]: pgmap v274: 97 pgs: 97 active+clean; 3.8 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-25T10:41:24.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:23 smithi077 ceph-mon[32772]: pgmap v275: 97 pgs: 97 active+clean; 3.8 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-25T10:41:24.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:23 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:41:24.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:23 smithi164 ceph-mon[32713]: pgmap v275: 97 pgs: 97 active+clean; 3.8 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-25T10:41:24.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:23 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:41:24.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:23 smithi043 ceph-mon[29478]: pgmap v275: 97 pgs: 97 active+clean; 3.8 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-25T10:41:24.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:23 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:41:26.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:25 smithi077 ceph-mon[32772]: pgmap v276: 97 pgs: 97 active+clean; 4.0 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 19 op/s 2024-04-25T10:41:26.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:25 smithi164 ceph-mon[32713]: pgmap v276: 97 pgs: 97 active+clean; 4.0 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 19 op/s 2024-04-25T10:41:26.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:25 smithi043 ceph-mon[29478]: pgmap v276: 97 pgs: 97 active+clean; 4.0 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 19 op/s 2024-04-25T10:41:28.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:27 smithi077 ceph-mon[32772]: pgmap v277: 97 pgs: 97 active+clean; 4.0 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-04-25T10:41:28.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:27 smithi164 ceph-mon[32713]: pgmap v277: 97 pgs: 97 active+clean; 4.0 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-04-25T10:41:28.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:27 smithi043 ceph-mon[29478]: pgmap v277: 97 pgs: 97 active+clean; 4.0 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-04-25T10:41:29.528 INFO:tasks.workunit.client.0.smithi043.stdout:375/600: open fd = 3 2024-04-25T10:41:29.528 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.528 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.528 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.529 INFO:tasks.workunit.client.0.smithi043.stdout:376/600: open fd = 3 2024-04-25T10:41:29.529 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.529 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.529 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.529 INFO:tasks.workunit.client.0.smithi043.stdout:377/600: open fd = 3 2024-04-25T10:41:29.529 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.530 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.530 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.530 INFO:tasks.workunit.client.0.smithi043.stdout:378/600: open fd = 3 2024-04-25T10:41:29.530 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.530 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.531 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.531 INFO:tasks.workunit.client.0.smithi043.stdout:379/600: open fd = 3 2024-04-25T10:41:29.531 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.531 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.531 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.531 INFO:tasks.workunit.client.0.smithi043.stdout:380/600: open fd = 3 2024-04-25T10:41:29.532 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.532 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.532 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.532 INFO:tasks.workunit.client.0.smithi043.stdout:381/600: open fd = 3 2024-04-25T10:41:29.532 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.532 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.533 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.533 INFO:tasks.workunit.client.0.smithi043.stdout:382/600: open fd = 3 2024-04-25T10:41:29.533 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.533 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.533 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.533 INFO:tasks.workunit.client.0.smithi043.stdout:383/600: open fd = 3 2024-04-25T10:41:29.533 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.534 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.534 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.534 INFO:tasks.workunit.client.0.smithi043.stdout:384/600: open fd = 3 2024-04-25T10:41:29.534 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.534 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.534 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.534 INFO:tasks.workunit.client.0.smithi043.stdout:385/600: open fd = 3 2024-04-25T10:41:29.535 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.535 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.535 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.535 INFO:tasks.workunit.client.0.smithi043.stdout:386/600: open fd = 3 2024-04-25T10:41:29.535 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.535 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.535 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.535 INFO:tasks.workunit.client.0.smithi043.stdout:387/600: open fd = 3 2024-04-25T10:41:29.536 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.536 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.536 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.536 INFO:tasks.workunit.client.0.smithi043.stdout:388/600: open fd = 3 2024-04-25T10:41:29.536 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.536 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.536 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.536 INFO:tasks.workunit.client.0.smithi043.stdout:389/600: open fd = 3 2024-04-25T10:41:29.537 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.537 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.537 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.537 INFO:tasks.workunit.client.0.smithi043.stdout:390/600: open fd = 3 2024-04-25T10:41:29.537 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.537 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.537 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.537 INFO:tasks.workunit.client.0.smithi043.stdout:391/600: open fd = 3 2024-04-25T10:41:29.538 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.538 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.538 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.538 INFO:tasks.workunit.client.0.smithi043.stdout:392/600: open fd = 3 2024-04-25T10:41:29.538 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.538 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.539 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.539 INFO:tasks.workunit.client.0.smithi043.stdout:393/600: open fd = 3 2024-04-25T10:41:29.539 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.539 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.539 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.539 INFO:tasks.workunit.client.0.smithi043.stdout:394/600: open fd = 3 2024-04-25T10:41:29.539 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.539 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.539 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.539 INFO:tasks.workunit.client.0.smithi043.stdout:395/600: open fd = 3 2024-04-25T10:41:29.539 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.539 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.539 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.539 INFO:tasks.workunit.client.0.smithi043.stdout:396/600: open fd = 3 2024-04-25T10:41:29.539 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.540 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.540 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.540 INFO:tasks.workunit.client.0.smithi043.stdout:397/600: open fd = 3 2024-04-25T10:41:29.540 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.540 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.540 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.540 INFO:tasks.workunit.client.0.smithi043.stdout:398/600: open fd = 3 2024-04-25T10:41:29.540 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.540 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.540 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.540 INFO:tasks.workunit.client.0.smithi043.stdout:399/600: open fd = 3 2024-04-25T10:41:29.540 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.540 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.540 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.541 INFO:tasks.workunit.client.0.smithi043.stdout:400/600: open fd = 3 2024-04-25T10:41:29.541 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.541 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.541 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.541 INFO:tasks.workunit.client.0.smithi043.stdout:401/600: open fd = 3 2024-04-25T10:41:29.541 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.541 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.541 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.541 INFO:tasks.workunit.client.0.smithi043.stdout:402/600: open fd = 3 2024-04-25T10:41:29.541 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.541 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.541 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.541 INFO:tasks.workunit.client.0.smithi043.stdout:403/600: open fd = 3 2024-04-25T10:41:29.541 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.542 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.542 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.542 INFO:tasks.workunit.client.0.smithi043.stdout:404/600: open fd = 3 2024-04-25T10:41:29.542 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.542 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.542 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.542 INFO:tasks.workunit.client.0.smithi043.stdout:405/600: open fd = 3 2024-04-25T10:41:29.542 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.542 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.542 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.543 INFO:tasks.workunit.client.0.smithi043.stdout:406/600: open fd = 3 2024-04-25T10:41:29.543 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.543 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.543 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.543 INFO:tasks.workunit.client.0.smithi043.stdout:407/600: open fd = 3 2024-04-25T10:41:29.543 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.543 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.543 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.543 INFO:tasks.workunit.client.0.smithi043.stdout:408/600: open fd = 3 2024-04-25T10:41:29.543 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.543 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.543 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.543 INFO:tasks.workunit.client.0.smithi043.stdout:409/600: open fd = 3 2024-04-25T10:41:29.543 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.544 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.544 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.544 INFO:tasks.workunit.client.0.smithi043.stdout:410/600: open fd = 3 2024-04-25T10:41:29.544 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.544 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.544 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.544 INFO:tasks.workunit.client.0.smithi043.stdout:411/600: open fd = 3 2024-04-25T10:41:29.544 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.544 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.544 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.544 INFO:tasks.workunit.client.0.smithi043.stdout:412/600: open fd = 3 2024-04-25T10:41:29.544 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.544 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.544 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.545 INFO:tasks.workunit.client.0.smithi043.stdout:413/600: open fd = 3 2024-04-25T10:41:29.545 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.545 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.545 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.545 INFO:tasks.workunit.client.0.smithi043.stdout:414/600: open fd = 3 2024-04-25T10:41:29.545 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.545 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.545 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.545 INFO:tasks.workunit.client.0.smithi043.stdout:415/600: open fd = 3 2024-04-25T10:41:29.545 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.545 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.545 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.545 INFO:tasks.workunit.client.0.smithi043.stdout:416/600: open fd = 3 2024-04-25T10:41:29.545 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.545 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.546 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.546 INFO:tasks.workunit.client.0.smithi043.stdout:417/600: open fd = 3 2024-04-25T10:41:29.546 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.546 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.546 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.546 INFO:tasks.workunit.client.0.smithi043.stdout:418/600: open fd = 3 2024-04-25T10:41:29.546 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.546 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.546 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.546 INFO:tasks.workunit.client.0.smithi043.stdout:419/600: open fd = 3 2024-04-25T10:41:29.546 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.547 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.547 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.547 INFO:tasks.workunit.client.0.smithi043.stdout:420/600: open fd = 3 2024-04-25T10:41:29.547 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.547 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.547 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.547 INFO:tasks.workunit.client.0.smithi043.stdout:421/600: open fd = 3 2024-04-25T10:41:29.547 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.547 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.547 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.548 INFO:tasks.workunit.client.0.smithi043.stdout:422/600: open fd = 3 2024-04-25T10:41:29.548 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.548 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.548 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.548 INFO:tasks.workunit.client.0.smithi043.stdout:423/600: open fd = 3 2024-04-25T10:41:29.548 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.548 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.548 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.548 INFO:tasks.workunit.client.0.smithi043.stdout:424/600: open fd = 3 2024-04-25T10:41:29.548 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.548 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.548 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.548 INFO:tasks.workunit.client.0.smithi043.stdout:425/600: open fd = 3 2024-04-25T10:41:29.549 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.549 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.549 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.549 INFO:tasks.workunit.client.0.smithi043.stdout:426/600: open fd = 3 2024-04-25T10:41:29.549 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.549 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.549 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.549 INFO:tasks.workunit.client.0.smithi043.stdout:427/600: open fd = 3 2024-04-25T10:41:29.549 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.549 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.549 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.549 INFO:tasks.workunit.client.0.smithi043.stdout:428/600: open fd = 3 2024-04-25T10:41:29.549 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.550 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.550 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.550 INFO:tasks.workunit.client.0.smithi043.stdout:429/600: open fd = 3 2024-04-25T10:41:29.550 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.550 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.550 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.550 INFO:tasks.workunit.client.0.smithi043.stdout:430/600: open fd = 3 2024-04-25T10:41:29.550 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.550 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.550 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.550 INFO:tasks.workunit.client.0.smithi043.stdout:431/600: open fd = 3 2024-04-25T10:41:29.550 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.550 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.550 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.551 INFO:tasks.workunit.client.0.smithi043.stdout:432/600: open fd = 3 2024-04-25T10:41:29.551 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.551 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.551 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.551 INFO:tasks.workunit.client.0.smithi043.stdout:433/600: open fd = 3 2024-04-25T10:41:29.551 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.551 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.551 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.551 INFO:tasks.workunit.client.0.smithi043.stdout:434/600: open fd = 3 2024-04-25T10:41:29.551 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.551 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.551 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.551 INFO:tasks.workunit.client.0.smithi043.stdout:435/600: open fd = 3 2024-04-25T10:41:29.551 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.552 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.552 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:29.552 INFO:tasks.workunit.client.0.smithi043.stdout:436/600: open fd = 3 2024-04-25T10:41:29.552 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.552 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:41:29.552 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:41:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.148+0000 7fd1840aa640 1 -- 172.21.15.43:0/4216555661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd17c108a50 msgr2=0x7fd17c102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.148+0000 7fd1840aa640 1 --2- 172.21.15.43:0/4216555661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd17c108a50 0x7fd17c102600 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fd178009ca0 tx=0x7fd17802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.148+0000 7fd1840aa640 1 -- 172.21.15.43:0/4216555661 shutdown_connections 2024-04-25T10:41:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.148+0000 7fd1840aa640 1 --2- 172.21.15.43:0/4216555661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd17c108ef0 0x7fd17c102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.148+0000 7fd1840aa640 1 --2- 172.21.15.43:0/4216555661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd17c1031b0 0x7fd17c10f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.148+0000 7fd1840aa640 1 --2- 172.21.15.43:0/4216555661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd17c108a50 0x7fd17c102600 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.148+0000 7fd1840aa640 1 -- 172.21.15.43:0/4216555661 >> 172.21.15.43:0/4216555661 conn(0x7fd17c0fe260 msgr2=0x7fd17c100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.148+0000 7fd1840aa640 1 -- 172.21.15.43:0/4216555661 shutdown_connections 2024-04-25T10:41:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.148+0000 7fd1840aa640 1 -- 172.21.15.43:0/4216555661 wait complete. 2024-04-25T10:41:30.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.149+0000 7fd1840aa640 1 Processor -- start 2024-04-25T10:41:30.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.149+0000 7fd1840aa640 1 -- start start 2024-04-25T10:41:30.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.149+0000 7fd1840aa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd17c1031b0 0x7fd17c1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:30.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.149+0000 7fd1840aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd17c108a50 0x7fd17c199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:30.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.149+0000 7fd1840aa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd17c108ef0 0x7fd17c19ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:30.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.149+0000 7fd1840aa640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c112c50 con 0x7fd17c108a50 2024-04-25T10:41:30.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.149+0000 7fd1840aa640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c112ad0 con 0x7fd17c108ef0 2024-04-25T10:41:30.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.149+0000 7fd1840aa640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c112dd0 con 0x7fd17c1031b0 2024-04-25T10:41:30.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.149+0000 7fd181645640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd17c108a50 0x7fd17c199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:30.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.149+0000 7fd181645640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd17c108a50 0x7fd17c199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52898/0 (socket says 172.21.15.43:52898) 2024-04-25T10:41:30.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.149+0000 7fd181645640 1 -- 172.21.15.43:0/1108918336 learned_addr learned my addr 172.21.15.43:0/1108918336 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.149+0000 7fd181e46640 1 --2- 172.21.15.43:0/1108918336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd17c1031b0 0x7fd17c1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.149+0000 7fd182647640 1 --2- 172.21.15.43:0/1108918336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd17c108ef0 0x7fd17c19ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.149+0000 7fd181645640 1 -- 172.21.15.43:0/1108918336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd17c1031b0 msgr2=0x7fd17c1996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.149+0000 7fd181645640 1 --2- 172.21.15.43:0/1108918336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd17c1031b0 0x7fd17c1996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.149+0000 7fd181645640 1 -- 172.21.15.43:0/1108918336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd17c108ef0 msgr2=0x7fd17c19ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.149+0000 7fd181645640 1 --2- 172.21.15.43:0/1108918336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd17c108ef0 0x7fd17c19ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.149+0000 7fd181645640 1 -- 172.21.15.43:0/1108918336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17c19e510 con 0x7fd17c108a50 2024-04-25T10:41:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.150+0000 7fd182647640 1 --2- 172.21.15.43:0/1108918336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd17c108ef0 0x7fd17c19ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:41:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.150+0000 7fd181645640 1 --2- 172.21.15.43:0/1108918336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd17c108a50 0x7fd17c199c10 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00e9f0 tx=0x7fd16c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.150+0000 7fd172ffd640 1 -- 172.21.15.43:0/1108918336 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c00ccc0 con 0x7fd17c108a50 2024-04-25T10:41:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.150+0000 7fd172ffd640 1 -- 172.21.15.43:0/1108918336 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd16c0035f0 con 0x7fd17c108a50 2024-04-25T10:41:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.150+0000 7fd1840aa640 1 -- 172.21.15.43:0/1108918336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17c19e800 con 0x7fd17c108a50 2024-04-25T10:41:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.150+0000 7fd172ffd640 1 -- 172.21.15.43:0/1108918336 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c00ce60 con 0x7fd17c108a50 2024-04-25T10:41:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.150+0000 7fd1840aa640 1 -- 172.21.15.43:0/1108918336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd17c1a6180 con 0x7fd17c108a50 2024-04-25T10:41:30.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.152+0000 7fd172ffd640 1 -- 172.21.15.43:0/1108918336 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fd16c003aa0 con 0x7fd17c108a50 2024-04-25T10:41:30.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.153+0000 7fd18b033740 1 Processor -- start 2024-04-25T10:41:30.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.153+0000 7fd18b033740 1 -- start start 2024-04-25T10:41:30.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.153+0000 7fd18b033740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3561b0 0x557a6c6f5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:30.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.153+0000 7fd18b033740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a6c32d2f0 0x557a6c6f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:30.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.153+0000 7fd18b033740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3354d0 0x557a6c6f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:30.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.153+0000 7fd18b033740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557a6c65c460 con 0x557a6c3354d0 2024-04-25T10:41:30.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.153+0000 7fd18b033740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a6c616590 con 0x557a6c32d2f0 2024-04-25T10:41:30.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.153+0000 7fd18b033740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557a6c658420 con 0x557a6c3561b0 2024-04-25T10:41:30.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.153+0000 7fd181645640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3354d0 0x557a6c6f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:30.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.153+0000 7fd181645640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3354d0 0x557a6c6f63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52902/0 (socket says 172.21.15.43:52902) 2024-04-25T10:41:30.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.153+0000 7fd181645640 1 -- 172.21.15.43:0/1077844855 learned_addr learned my addr 172.21.15.43:0/1077844855 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.153+0000 7fd172ffd640 1 --2- 172.21.15.43:0/1108918336 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd1680afcd0 0x7fd1680b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.154+0000 7fd181e46640 1 --2- 172.21.15.43:0/1077844855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a6c32d2f0 0x557a6c6f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.154+0000 7fd182647640 1 --2- 172.21.15.43:0/1077844855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3561b0 0x557a6c6f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.154+0000 7fd182647640 1 --2- 172.21.15.43:0/1108918336 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd1680afcd0 0x7fd1680b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.154+0000 7fd172ffd640 1 -- 172.21.15.43:0/1108918336 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fd16c0df400 con 0x7fd17c108a50 2024-04-25T10:41:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.154+0000 7fd181645640 1 -- 172.21.15.43:0/1077844855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3561b0 msgr2=0x557a6c6f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.154+0000 7fd181645640 1 --2- 172.21.15.43:0/1077844855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3561b0 0x557a6c6f5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.154+0000 7fd181645640 1 -- 172.21.15.43:0/1077844855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a6c32d2f0 msgr2=0x557a6c6f5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.154+0000 7fd181645640 1 --2- 172.21.15.43:0/1077844855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a6c32d2f0 0x557a6c6f5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.154+0000 7fd181645640 1 -- 172.21.15.43:0/1077844855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a6c6230d0 con 0x557a6c3354d0 2024-04-25T10:41:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.154+0000 7fd182647640 1 --2- 172.21.15.43:0/1077844855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3561b0 0x557a6c6f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.154+0000 7fd181645640 1 --2- 172.21.15.43:0/1077844855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3354d0 0x557a6c6f63c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0a4810 tx=0x7fd16c003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee2f3eb106f79b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.154+0000 7fd167fff640 1 -- 172.21.15.43:0/1077844855 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c0a7370 con 0x557a6c3354d0 2024-04-25T10:41:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.154+0000 7fd182647640 1 --2- 172.21.15.43:0/1108918336 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd1680afcd0 0x7fd1680b21e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x557a6c6f3500 tx=0x7fd174005b00 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.154+0000 7fd167fff640 1 -- 172.21.15.43:0/1077844855 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd16c0a70b0 con 0x557a6c3354d0 2024-04-25T10:41:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.154+0000 7fd167fff640 1 -- 172.21.15.43:0/1077844855 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c0e2b80 con 0x557a6c3354d0 2024-04-25T10:41:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.155+0000 7fd18b033740 1 -- 172.21.15.43:0/1077844855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3354d0 msgr2=0x557a6c6f63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.155+0000 7fd18b033740 1 --2- 172.21.15.43:0/1077844855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3354d0 0x557a6c6f63c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0a4810 tx=0x7fd16c003a10 comp rx=0 tx=0).stop 2024-04-25T10:41:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.155+0000 7fd18b033740 1 -- 172.21.15.43:0/1077844855 shutdown_connections 2024-04-25T10:41:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.155+0000 7fd18b033740 1 --2- 172.21.15.43:0/1077844855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a6c32d2f0 0x557a6c6f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.155+0000 7fd18b033740 1 --2- 172.21.15.43:0/1077844855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3354d0 0x557a6c6f63c0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.155+0000 7fd18b033740 1 --2- 172.21.15.43:0/1077844855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3561b0 0x557a6c6f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.155+0000 7fd18b033740 1 -- 172.21.15.43:0/1077844855 >> 172.21.15.43:0/1077844855 conn(0x557a6c34a300 msgr2=0x557a6c6f22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.155+0000 7fd18b033740 1 -- 172.21.15.43:0/1077844855 shutdown_connections 2024-04-25T10:41:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.155+0000 7fd18b033740 1 -- 172.21.15.43:0/1077844855 wait complete. 2024-04-25T10:41:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.156+0000 7fd18b033740 1 -- start start 2024-04-25T10:41:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.156+0000 7fd18b033740 1 Processor -- start 2024-04-25T10:41:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.156+0000 7fd18b033740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c32d2f0 0x557a6c702390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.156+0000 7fd18b033740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a6c3354d0 0x557a6c7028d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.156+0000 7fd18b033740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3561b0 0x557a6c708270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.156+0000 7fd18b033740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557a6c704e30 con 0x557a6c32d2f0 2024-04-25T10:41:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.156+0000 7fd18b033740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a6c5e3b60 con 0x557a6c3354d0 2024-04-25T10:41:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.156+0000 7fd18b033740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557a6c704fb0 con 0x557a6c3561b0 2024-04-25T10:41:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.156+0000 7fd18b033740 10 client.0 _ll_register_callbacks cb 0x557a6c6082b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.157+0000 7fd181e46640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c32d2f0 0x557a6c702390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.157+0000 7fd181e46640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c32d2f0 0x557a6c702390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52910/0 (socket says 172.21.15.43:52910) 2024-04-25T10:41:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.157+0000 7fd181e46640 1 -- 172.21.15.43:0/821370941 learned_addr learned my addr 172.21.15.43:0/821370941 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.157+0000 7fd182647640 1 --2- 172.21.15.43:0/821370941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3561b0 0x557a6c708270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.157+0000 7fd181645640 1 --2- 172.21.15.43:0/821370941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a6c3354d0 0x557a6c7028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.157+0000 7fd181e46640 1 -- 172.21.15.43:0/821370941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3561b0 msgr2=0x557a6c708270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.157+0000 7fd181e46640 1 --2- 172.21.15.43:0/821370941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3561b0 0x557a6c708270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.157+0000 7fd181e46640 1 -- 172.21.15.43:0/821370941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a6c3354d0 msgr2=0x557a6c7028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.157+0000 7fd181e46640 1 --2- 172.21.15.43:0/821370941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a6c3354d0 0x557a6c7028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.157+0000 7fd181e46640 1 -- 172.21.15.43:0/821370941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a6c36e210 con 0x557a6c32d2f0 2024-04-25T10:41:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.157+0000 7fd181e46640 1 --2- 172.21.15.43:0/821370941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c32d2f0 0x557a6c702390 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fd178009ca0 tx=0x7fd17802f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c6a228e8684c56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.157+0000 7fd1657fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.157+0000 7fd1657fa640 1 -- 172.21.15.43:0/821370941 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178031660 con 0x557a6c32d2f0 2024-04-25T10:41:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.157+0000 7fd1657fa640 1 -- 172.21.15.43:0/821370941 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fd178031800 con 0x557a6c32d2f0 2024-04-25T10:41:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.157+0000 7fd18b033740 1 -- 172.21.15.43:0/821370941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd16c0a70b0 con 0x557a6c32d2f0 2024-04-25T10:41:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.157+0000 7fd1657fa640 1 -- 172.21.15.43:0/821370941 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd178004b40 con 0x557a6c32d2f0 2024-04-25T10:41:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.158+0000 7fd18b033740 10 client.18180 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.158+0000 7fd18b033740 1 -- 172.21.15.43:0/821370941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557a6c43a620 con 0x557a6c32d2f0 2024-04-25T10:41:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.158+0000 7fd1657fa640 1 -- 172.21.15.43:0/821370941 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd16c0a70b0 con 0x557a6c32d2f0 2024-04-25T10:41:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.158+0000 7fd1657fa640 1 -- 172.21.15.43:0/821370941 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd178004340 con 0x557a6c32d2f0 2024-04-25T10:41:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.158+0000 7fd18b033740 10 client.18180 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.158+0000 7fd18b033740 10 client.18180 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.158+0000 7fd18b033740 20 client.18180 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.158+0000 7fd18b033740 1 --2- 172.21.15.43:0/821370941 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557a6c314d80 0x557a6c714930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.158+0000 7fd18b033740 4 client.18180 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:41:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.158+0000 7fd18b033740 1 -- 172.21.15.43:0/821370941 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a6c66a070 con 0x557a6c314d80 2024-04-25T10:41:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.158+0000 7fd181645640 1 --2- 172.21.15.43:0/821370941 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557a6c314d80 0x557a6c714930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.159+0000 7fd181645640 1 --2- 172.21.15.43:0/821370941 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557a6c314d80 0x557a6c714930 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8334c6efef3af38 server_cookie=86d8c42ae70d0b4b in_seq=0 out_seq=0 2024-04-25T10:41:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.159+0000 7fd1657fa640 10 client.18180 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.159+0000 7fd1657fa640 1 -- 172.21.15.43:0/821370941 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557a6c66a070 con 0x557a6c314d80 2024-04-25T10:41:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.159+0000 7fd1657fa640 10 client.18180 handle_command_reply: tid=0 2024-04-25T10:41:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.160+0000 7fd18b033740 1 client.18180 shutdown 2024-04-25T10:41:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.160+0000 7fd18b033740 1 -- 172.21.15.43:0/821370941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c32d2f0 msgr2=0x557a6c702390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.160+0000 7fd18b033740 1 --2- 172.21.15.43:0/821370941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c32d2f0 0x557a6c702390 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fd178009ca0 tx=0x7fd17802f450 comp rx=0 tx=0).stop 2024-04-25T10:41:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.160+0000 7fd18b033740 1 -- 172.21.15.43:0/821370941 shutdown_connections 2024-04-25T10:41:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.160+0000 7fd18b033740 1 --2- 172.21.15.43:0/821370941 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557a6c314d80 0x557a6c714930 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.160+0000 7fd18b033740 1 --2- 172.21.15.43:0/821370941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a6c3354d0 0x557a6c7028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.160+0000 7fd18b033740 1 --2- 172.21.15.43:0/821370941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3561b0 0x557a6c708270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.160+0000 7fd1657fa640 0 client.18180 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.160+0000 7fd18b033740 1 --2- 172.21.15.43:0/821370941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c32d2f0 0x557a6c702390 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.160+0000 7fd18b033740 1 -- 172.21.15.43:0/821370941 >> 172.21.15.43:0/821370941 conn(0x557a6c34a300 msgr2=0x557a6c6f62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.160+0000 7fd18b033740 1 -- 172.21.15.43:0/821370941 shutdown_connections 2024-04-25T10:41:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.160+0000 7fd18b033740 1 -- 172.21.15.43:0/821370941 wait complete. 2024-04-25T10:41:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.160+0000 7fd18b033740 20 client.18180 trim_cache size 0 max 16384 2024-04-25T10:41:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.175+0000 7fd18b033740 1 Processor -- start 2024-04-25T10:41:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.175+0000 7fd18b033740 1 -- start start 2024-04-25T10:41:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.175+0000 7fd18b033740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a6c32d2f0 0x557a6c6f0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.175+0000 7fd18b033740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3354d0 0x557a6c6f2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.176+0000 7fd18b033740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3561b0 0x557a6c7081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.176+0000 7fd18b033740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c1b0160 con 0x557a6c3561b0 2024-04-25T10:41:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.176+0000 7fd18b033740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c1ada30 con 0x557a6c32d2f0 2024-04-25T10:41:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.176+0000 7fd18b033740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c1ab8a0 con 0x557a6c3354d0 2024-04-25T10:41:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.176+0000 7fd182647640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3561b0 0x557a6c7081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.176+0000 7fd182647640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3561b0 0x557a6c7081e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52920/0 (socket says 172.21.15.43:52920) 2024-04-25T10:41:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.176+0000 7fd182647640 1 -- 172.21.15.43:0/2682479092 learned_addr learned my addr 172.21.15.43:0/2682479092 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.176+0000 7fd181e46640 1 --2- 172.21.15.43:0/2682479092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a6c32d2f0 0x557a6c6f0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.176+0000 7fd181645640 1 --2- 172.21.15.43:0/2682479092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3354d0 0x557a6c6f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.176+0000 7fd182647640 1 -- 172.21.15.43:0/2682479092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3354d0 msgr2=0x557a6c6f2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.176+0000 7fd182647640 1 --2- 172.21.15.43:0/2682479092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3354d0 0x557a6c6f2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.176+0000 7fd182647640 1 -- 172.21.15.43:0/2682479092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a6c32d2f0 msgr2=0x557a6c6f0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.176+0000 7fd182647640 1 --2- 172.21.15.43:0/2682479092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a6c32d2f0 0x557a6c6f0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.176+0000 7fd182647640 1 -- 172.21.15.43:0/2682479092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a6c6f6890 con 0x557a6c3561b0 2024-04-25T10:41:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.176+0000 7fd182647640 1 --2- 172.21.15.43:0/2682479092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3561b0 0x557a6c7081e0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fd17400f820 tx=0x7fd17400d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e1ea08c412b646 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.176+0000 7fd1677fe640 1 -- 172.21.15.43:0/2682479092 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17401d8e0 con 0x557a6c3561b0 2024-04-25T10:41:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.176+0000 7fd1677fe640 1 -- 172.21.15.43:0/2682479092 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd174016070 con 0x557a6c3561b0 2024-04-25T10:41:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.177+0000 7fd1677fe640 1 -- 172.21.15.43:0/2682479092 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174019e50 con 0x557a6c3561b0 2024-04-25T10:41:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.177+0000 7fd18b033740 1 -- 172.21.15.43:0/2682479092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3561b0 msgr2=0x557a6c7081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.177+0000 7fd18b033740 1 --2- 172.21.15.43:0/2682479092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3561b0 0x557a6c7081e0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fd17400f820 tx=0x7fd17400d2e0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.177+0000 7fd18b033740 1 -- 172.21.15.43:0/2682479092 shutdown_connections 2024-04-25T10:41:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.177+0000 7fd18b033740 1 --2- 172.21.15.43:0/2682479092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3561b0 0x557a6c7081e0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.177+0000 7fd18b033740 1 --2- 172.21.15.43:0/2682479092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3354d0 0x557a6c6f2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.177+0000 7fd18b033740 1 --2- 172.21.15.43:0/2682479092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a6c32d2f0 0x557a6c6f0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.177+0000 7fd18b033740 1 -- 172.21.15.43:0/2682479092 >> 172.21.15.43:0/2682479092 conn(0x557a6c34a300 msgr2=0x557a6c6f1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.177+0000 7fd18b033740 1 -- 172.21.15.43:0/2682479092 shutdown_connections 2024-04-25T10:41:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.177+0000 7fd18b033740 1 -- 172.21.15.43:0/2682479092 wait complete. 2024-04-25T10:41:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.178+0000 7fd18b033740 1 -- start start 2024-04-25T10:41:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.178+0000 7fd18b033740 1 Processor -- start 2024-04-25T10:41:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.178+0000 7fd18b033740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a6c32d2f0 0x557a6c70e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.178+0000 7fd18b033740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3354d0 0x557a6c70eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.178+0000 7fd18b033740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3561b0 0x557a6c719860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.178+0000 7fd18b033740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c1abef0 con 0x557a6c3561b0 2024-04-25T10:41:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.178+0000 7fd18b033740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c1ad360 con 0x557a6c32d2f0 2024-04-25T10:41:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.178+0000 7fd18b033740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c1ac5c0 con 0x557a6c3354d0 2024-04-25T10:41:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.178+0000 7fd18b033740 10 client.0 _ll_register_callbacks cb 0x7fd178009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.178+0000 7fd182647640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3561b0 0x557a6c719860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.178+0000 7fd182647640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3561b0 0x557a6c719860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52932/0 (socket says 172.21.15.43:52932) 2024-04-25T10:41:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.178+0000 7fd182647640 1 -- 172.21.15.43:0/554094486 learned_addr learned my addr 172.21.15.43:0/554094486 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.178+0000 7fd181e46640 1 --2- 172.21.15.43:0/554094486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a6c32d2f0 0x557a6c70e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.179+0000 7fd181645640 1 --2- 172.21.15.43:0/554094486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3354d0 0x557a6c70eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.179+0000 7fd182647640 1 -- 172.21.15.43:0/554094486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3354d0 msgr2=0x557a6c70eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.179+0000 7fd182647640 1 --2- 172.21.15.43:0/554094486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3354d0 0x557a6c70eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.179+0000 7fd182647640 1 -- 172.21.15.43:0/554094486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a6c32d2f0 msgr2=0x557a6c70e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.179+0000 7fd182647640 1 --2- 172.21.15.43:0/554094486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a6c32d2f0 0x557a6c70e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.179+0000 7fd182647640 1 -- 172.21.15.43:0/554094486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a6c719ea0 con 0x557a6c3561b0 2024-04-25T10:41:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.179+0000 7fd181645640 1 --2- 172.21.15.43:0/554094486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3354d0 0x557a6c70eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.179+0000 7fd182647640 1 --2- 172.21.15.43:0/554094486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3561b0 0x557a6c719860 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd17400fda0 tx=0x7fd174019d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f032be58e616593f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.179+0000 7fd1657fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.179+0000 7fd1657fa640 1 -- 172.21.15.43:0/554094486 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174035060 con 0x557a6c3561b0 2024-04-25T10:41:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.179+0000 7fd1657fa640 1 -- 172.21.15.43:0/554094486 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fd17401d6e0 con 0x557a6c3561b0 2024-04-25T10:41:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.179+0000 7fd18b033740 1 -- 172.21.15.43:0/554094486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd174016070 con 0x557a6c3561b0 2024-04-25T10:41:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.179+0000 7fd1657fa640 1 -- 172.21.15.43:0/554094486 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd17401d8e0 con 0x557a6c3561b0 2024-04-25T10:41:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.179+0000 7fd18b033740 10 client.18186 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.179+0000 7fd18b033740 1 -- 172.21.15.43:0/554094486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557a6c70c970 con 0x557a6c3561b0 2024-04-25T10:41:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.180+0000 7fd1657fa640 1 -- 172.21.15.43:0/554094486 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd174016070 con 0x557a6c3561b0 2024-04-25T10:41:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.180+0000 7fd1657fa640 1 -- 172.21.15.43:0/554094486 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd17402f030 con 0x557a6c3561b0 2024-04-25T10:41:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.180+0000 7fd18b033740 10 client.18186 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.180+0000 7fd18b033740 10 client.18186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.180+0000 7fd18b033740 20 client.18186 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.180+0000 7fd18b033740 1 --2- 172.21.15.43:0/554094486 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557a6c2ea1c0 0x557a6c77ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.180+0000 7fd18b033740 4 client.18186 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:41:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.180+0000 7fd18b033740 1 -- 172.21.15.43:0/554094486 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a6c663ee0 con 0x557a6c2ea1c0 2024-04-25T10:41:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.180+0000 7fd181e46640 1 --2- 172.21.15.43:0/554094486 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557a6c2ea1c0 0x557a6c77ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.181+0000 7fd1657fa640 10 client.18186 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.181+0000 7fd181e46640 1 --2- 172.21.15.43:0/554094486 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557a6c2ea1c0 0x557a6c77ecf0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0984f79fcb63031 server_cookie=4af1c8df63b3181 in_seq=0 out_seq=0 2024-04-25T10:41:30.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.196+0000 7fd1657fa640 1 -- 172.21.15.43:0/554094486 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a6c663ee0 con 0x557a6c2ea1c0 2024-04-25T10:41:30.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.196+0000 7fd1657fa640 10 client.18186 handle_command_reply: tid=0 2024-04-25T10:41:30.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.196+0000 7fd18b033740 1 client.18186 shutdown 2024-04-25T10:41:30.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.196+0000 7fd18b033740 1 -- 172.21.15.43:0/554094486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3561b0 msgr2=0x557a6c719860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:30.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.196+0000 7fd18b033740 1 --2- 172.21.15.43:0/554094486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3561b0 0x557a6c719860 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd17400fda0 tx=0x7fd174019d60 comp rx=0 tx=0).stop 2024-04-25T10:41:30.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd18b033740 1 -- 172.21.15.43:0/554094486 shutdown_connections 2024-04-25T10:41:30.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd18b033740 1 --2- 172.21.15.43:0/554094486 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557a6c2ea1c0 0x557a6c77ecf0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd18b033740 1 --2- 172.21.15.43:0/554094486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6c3561b0 0x557a6c719860 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd18b033740 1 --2- 172.21.15.43:0/554094486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a6c3354d0 0x557a6c70eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd1657fa640 0 client.18186 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd18b033740 1 --2- 172.21.15.43:0/554094486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a6c32d2f0 0x557a6c70e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd18b033740 1 -- 172.21.15.43:0/554094486 >> 172.21.15.43:0/554094486 conn(0x557a6c34a300 msgr2=0x557a6c6f5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd18b033740 1 -- 172.21.15.43:0/554094486 shutdown_connections 2024-04-25T10:41:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd18b033740 1 -- 172.21.15.43:0/554094486 wait complete. 2024-04-25T10:41:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd18b033740 20 client.18186 trim_cache size 0 max 16384 2024-04-25T10:41:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:41:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:41:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:41:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:41:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd1840aa640 1 -- 172.21.15.43:0/1108918336 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd1680afcd0 msgr2=0x7fd1680b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd1840aa640 1 --2- 172.21.15.43:0/1108918336 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd1680afcd0 0x7fd1680b21e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x557a6c6f3500 tx=0x7fd174005b00 comp rx=0 tx=0).stop 2024-04-25T10:41:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd1840aa640 1 -- 172.21.15.43:0/1108918336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd17c108a50 msgr2=0x7fd17c199c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd1840aa640 1 --2- 172.21.15.43:0/1108918336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd17c108a50 0x7fd17c199c10 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00e9f0 tx=0x7fd16c00eeb0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd1840aa640 1 -- 172.21.15.43:0/1108918336 shutdown_connections 2024-04-25T10:41:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd1840aa640 1 --2- 172.21.15.43:0/1108918336 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd1680afcd0 0x7fd1680b21e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd1840aa640 1 --2- 172.21.15.43:0/1108918336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd17c108ef0 0x7fd17c19ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd1840aa640 1 --2- 172.21.15.43:0/1108918336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd17c108a50 0x7fd17c199c10 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd1840aa640 1 --2- 172.21.15.43:0/1108918336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd17c1031b0 0x7fd17c1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd1840aa640 1 -- 172.21.15.43:0/1108918336 >> 172.21.15.43:0/1108918336 conn(0x7fd17c0fe260 msgr2=0x7fd17c0fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd1840aa640 1 -- 172.21.15.43:0/1108918336 shutdown_connections 2024-04-25T10:41:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:30.197+0000 7fd1840aa640 1 -- 172.21.15.43:0/1108918336 wait complete. 2024-04-25T10:41:30.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:29 smithi077 ceph-mon[32772]: pgmap v278: 97 pgs: 97 active+clean; 4.2 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-04-25T10:41:30.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:29 smithi164 ceph-mon[32713]: pgmap v278: 97 pgs: 97 active+clean; 4.2 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-04-25T10:41:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:29 smithi043 ceph-mon[29478]: pgmap v278: 97 pgs: 97 active+clean; 4.2 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-04-25T10:41:32.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:31 smithi077 ceph-mon[32772]: pgmap v279: 97 pgs: 97 active+clean; 4.2 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-25T10:41:32.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:31 smithi164 ceph-mon[32713]: pgmap v279: 97 pgs: 97 active+clean; 4.2 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-25T10:41:32.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:31 smithi043 ceph-mon[29478]: pgmap v279: 97 pgs: 97 active+clean; 4.2 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-25T10:41:34.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:33 smithi077 ceph-mon[32772]: pgmap v280: 97 pgs: 97 active+clean; 4.2 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-25T10:41:34.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:33 smithi164 ceph-mon[32713]: pgmap v280: 97 pgs: 97 active+clean; 4.2 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-25T10:41:34.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:33 smithi043 ceph-mon[29478]: pgmap v280: 97 pgs: 97 active+clean; 4.2 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-25T10:41:36.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:35 smithi077 ceph-mon[32772]: pgmap v281: 97 pgs: 97 active+clean; 4.4 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-04-25T10:41:36.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:35 smithi164 ceph-mon[32713]: pgmap v281: 97 pgs: 97 active+clean; 4.4 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-04-25T10:41:36.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:35 smithi043 ceph-mon[29478]: pgmap v281: 97 pgs: 97 active+clean; 4.4 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-04-25T10:41:38.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:37 smithi077 ceph-mon[32772]: pgmap v282: 97 pgs: 97 active+clean; 4.4 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 11 op/s 2024-04-25T10:41:38.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:37 smithi164 ceph-mon[32713]: pgmap v282: 97 pgs: 97 active+clean; 4.4 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 11 op/s 2024-04-25T10:41:38.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:37 smithi043 ceph-mon[29478]: pgmap v282: 97 pgs: 97 active+clean; 4.4 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 11 op/s 2024-04-25T10:41:40.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:40 smithi077 ceph-mon[32772]: pgmap v283: 97 pgs: 97 active+clean; 4.6 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 16 op/s 2024-04-25T10:41:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:40 smithi164 ceph-mon[32713]: pgmap v283: 97 pgs: 97 active+clean; 4.6 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 16 op/s 2024-04-25T10:41:40.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:40 smithi043 ceph-mon[29478]: pgmap v283: 97 pgs: 97 active+clean; 4.6 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 16 op/s 2024-04-25T10:41:42.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:42 smithi077 ceph-mon[32772]: pgmap v284: 97 pgs: 97 active+clean; 4.6 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:41:42.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:42 smithi164 ceph-mon[32713]: pgmap v284: 97 pgs: 97 active+clean; 4.6 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:41:42.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:42 smithi043 ceph-mon[29478]: pgmap v284: 97 pgs: 97 active+clean; 4.6 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:41:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.309+0000 7f424412e640 1 -- 172.21.15.43:0/2396167277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f423c1069b0 msgr2=0x7f423c106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.309+0000 7f424412e640 1 --2- 172.21.15.43:0/2396167277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f423c1069b0 0x7f423c106de0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f422c009900 tx=0x7f422c02a410 comp rx=0 tx=0).stop 2024-04-25T10:41:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.309+0000 7f424412e640 1 -- 172.21.15.43:0/2396167277 shutdown_connections 2024-04-25T10:41:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.309+0000 7f424412e640 1 --2- 172.21.15.43:0/2396167277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f423c107470 0x7f423c10ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.309+0000 7f424412e640 1 --2- 172.21.15.43:0/2396167277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f423c1069b0 0x7f423c106de0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.309+0000 7f424412e640 1 --2- 172.21.15.43:0/2396167277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c102420 0x7f423c102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.309+0000 7f424412e640 1 -- 172.21.15.43:0/2396167277 >> 172.21.15.43:0/2396167277 conn(0x7f423c0fd880 msgr2=0x7f423c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.310+0000 7f424412e640 1 -- 172.21.15.43:0/2396167277 shutdown_connections 2024-04-25T10:41:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.310+0000 7f424412e640 1 -- 172.21.15.43:0/2396167277 wait complete. 2024-04-25T10:41:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.310+0000 7f424412e640 1 Processor -- start 2024-04-25T10:41:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.310+0000 7f424412e640 1 -- start start 2024-04-25T10:41:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.311+0000 7f424412e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f423c102420 0x7f423c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.311+0000 7f424412e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f423c1069b0 0x7f423c199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.311+0000 7f424412e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c107470 0x7f423c19dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.311+0000 7f424412e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f423c112350 con 0x7f423c1069b0 2024-04-25T10:41:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.311+0000 7f424412e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1121d0 con 0x7f423c102420 2024-04-25T10:41:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.311+0000 7f424412e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1124d0 con 0x7f423c107470 2024-04-25T10:41:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.311+0000 7f42416c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f423c1069b0 0x7f423c199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.311+0000 7f42416c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f423c1069b0 0x7f423c199c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48484/0 (socket says 172.21.15.43:48484) 2024-04-25T10:41:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.311+0000 7f42416c9640 1 -- 172.21.15.43:0/3993395711 learned_addr learned my addr 172.21.15.43:0/3993395711 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.311+0000 7f4241eca640 1 --2- 172.21.15.43:0/3993395711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f423c102420 0x7f423c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.311+0000 7f42426cb640 1 --2- 172.21.15.43:0/3993395711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c107470 0x7f423c19dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.311+0000 7f42416c9640 1 -- 172.21.15.43:0/3993395711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c107470 msgr2=0x7f423c19dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.311+0000 7f42416c9640 1 --2- 172.21.15.43:0/3993395711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c107470 0x7f423c19dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.311+0000 7f42416c9640 1 -- 172.21.15.43:0/3993395711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f423c102420 msgr2=0x7f423c1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.311+0000 7f42416c9640 1 --2- 172.21.15.43:0/3993395711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f423c102420 0x7f423c1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.311+0000 7f42416c9640 1 -- 172.21.15.43:0/3993395711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423c19e500 con 0x7f423c1069b0 2024-04-25T10:41:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.311+0000 7f42426cb640 1 --2- 172.21.15.43:0/3993395711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c107470 0x7f423c19dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:41:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.311+0000 7f42416c9640 1 --2- 172.21.15.43:0/3993395711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f423c1069b0 0x7f423c199c00 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f422c006010 tx=0x7f422c004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.312+0000 7f4232ffd640 1 -- 172.21.15.43:0/3993395711 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c03f030 con 0x7f423c1069b0 2024-04-25T10:41:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.312+0000 7f4232ffd640 1 -- 172.21.15.43:0/3993395711 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f422c003aa0 con 0x7f423c1069b0 2024-04-25T10:41:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.312+0000 7f424412e640 1 -- 172.21.15.43:0/3993395711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423c19e790 con 0x7f423c1069b0 2024-04-25T10:41:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.312+0000 7f4232ffd640 1 -- 172.21.15.43:0/3993395711 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c035070 con 0x7f423c1069b0 2024-04-25T10:41:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.312+0000 7f424412e640 1 -- 172.21.15.43:0/3993395711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f423c19ecd0 con 0x7f423c1069b0 2024-04-25T10:41:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.314+0000 7f424afdc740 1 Processor -- start 2024-04-25T10:41:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.314+0000 7f424afdc740 1 -- start start 2024-04-25T10:41:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.314+0000 7f4232ffd640 1 -- 172.21.15.43:0/3993395711 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f422c042030 con 0x7f423c1069b0 2024-04-25T10:41:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.315+0000 7f424afdc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804ee1b0 0x55da8088fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.315+0000 7f424afdc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804c52f0 0x55da80890240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.315+0000 7f424afdc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804cd4d0 0x55da80890780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.315+0000 7f424afdc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55da807f4460 con 0x55da804cd4d0 2024-04-25T10:41:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.315+0000 7f424afdc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55da807ae590 con 0x55da804c52f0 2024-04-25T10:41:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.315+0000 7f424afdc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55da807f0420 con 0x55da804ee1b0 2024-04-25T10:41:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.315+0000 7f42416c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804cd4d0 0x55da80890780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.315+0000 7f42416c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804cd4d0 0x55da80890780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48490/0 (socket says 172.21.15.43:48490) 2024-04-25T10:41:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.315+0000 7f42416c9640 1 -- 172.21.15.43:0/152931148 learned_addr learned my addr 172.21.15.43:0/152931148 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.315+0000 7f42426cb640 1 --2- 172.21.15.43:0/152931148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804ee1b0 0x55da8088fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.315+0000 7f4241eca640 1 --2- 172.21.15.43:0/152931148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804c52f0 0x55da80890240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.315+0000 7f42416c9640 1 -- 172.21.15.43:0/152931148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804ee1b0 msgr2=0x55da8088fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.315+0000 7f42416c9640 1 --2- 172.21.15.43:0/152931148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804ee1b0 0x55da8088fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.315+0000 7f42416c9640 1 -- 172.21.15.43:0/152931148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804c52f0 msgr2=0x55da80890240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.315+0000 7f42416c9640 1 --2- 172.21.15.43:0/152931148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804c52f0 0x55da80890240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.315+0000 7f42416c9640 1 -- 172.21.15.43:0/152931148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da807bb0d0 con 0x55da804cd4d0 2024-04-25T10:41:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.315+0000 7f4232ffd640 1 --2- 172.21.15.43:0/3993395711 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f42280afcd0 0x7f42280b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.316+0000 7f42416c9640 1 --2- 172.21.15.43:0/152931148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804cd4d0 0x55da80890780 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f422c0fcb00 tx=0x7f422c1056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed925fa8934ccda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.316+0000 7f42426cb640 1 --2- 172.21.15.43:0/3993395711 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f42280afcd0 0x7f42280b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.316+0000 7f4232ffd640 1 -- 172.21.15.43:0/3993395711 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f422c0ff410 con 0x7f423c1069b0 2024-04-25T10:41:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.316+0000 7f4227fff640 1 -- 172.21.15.43:0/152931148 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c0c3470 con 0x55da804cd4d0 2024-04-25T10:41:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.316+0000 7f4227fff640 1 -- 172.21.15.43:0/152931148 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f422c03b460 con 0x55da804cd4d0 2024-04-25T10:41:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.316+0000 7f4227fff640 1 -- 172.21.15.43:0/152931148 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c0c6d20 con 0x55da804cd4d0 2024-04-25T10:41:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.317+0000 7f424afdc740 1 -- 172.21.15.43:0/152931148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804cd4d0 msgr2=0x55da80890780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.317+0000 7f42426cb640 1 --2- 172.21.15.43:0/3993395711 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f42280afcd0 0x7f42280b21e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55da8088ce70 tx=0x7f4234009500 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.317+0000 7f424afdc740 1 --2- 172.21.15.43:0/152931148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804cd4d0 0x55da80890780 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f422c0fcb00 tx=0x7f422c1056e0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.317+0000 7f424afdc740 1 -- 172.21.15.43:0/152931148 shutdown_connections 2024-04-25T10:41:43.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.317+0000 7f424afdc740 1 --2- 172.21.15.43:0/152931148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804c52f0 0x55da80890240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.317+0000 7f424afdc740 1 --2- 172.21.15.43:0/152931148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804cd4d0 0x55da80890780 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.317+0000 7f424afdc740 1 --2- 172.21.15.43:0/152931148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804ee1b0 0x55da8088fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.317+0000 7f424afdc740 1 -- 172.21.15.43:0/152931148 >> 172.21.15.43:0/152931148 conn(0x55da804e2300 msgr2=0x55da8088c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:43.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.317+0000 7f424afdc740 1 -- 172.21.15.43:0/152931148 shutdown_connections 2024-04-25T10:41:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.317+0000 7f424afdc740 1 -- 172.21.15.43:0/152931148 wait complete. 2024-04-25T10:41:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.317+0000 7f424afdc740 1 -- start start 2024-04-25T10:41:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.318+0000 7f424afdc740 1 Processor -- start 2024-04-25T10:41:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.318+0000 7f424afdc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804c52f0 0x55da8089bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.318+0000 7f424afdc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804cd4d0 0x55da8089c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.318+0000 7f424afdc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 0x55da8089a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.318+0000 7f424afdc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55da8089cbf0 con 0x55da804ee1b0 2024-04-25T10:41:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.318+0000 7f424afdc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55da8077bb60 con 0x55da804cd4d0 2024-04-25T10:41:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.318+0000 7f424afdc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55da8089cd70 con 0x55da804c52f0 2024-04-25T10:41:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.318+0000 7f424afdc740 10 client.0 _ll_register_callbacks cb 0x55da807a02b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.318+0000 7f42426cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 0x55da8089a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.318+0000 7f42426cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 0x55da8089a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48504/0 (socket says 172.21.15.43:48504) 2024-04-25T10:41:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.318+0000 7f42426cb640 1 -- 172.21.15.43:0/1197755594 learned_addr learned my addr 172.21.15.43:0/1197755594 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.318+0000 7f4241eca640 1 --2- 172.21.15.43:0/1197755594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804c52f0 0x55da8089bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.318+0000 7f42416c9640 1 --2- 172.21.15.43:0/1197755594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804cd4d0 0x55da8089c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.319+0000 7f42426cb640 1 -- 172.21.15.43:0/1197755594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804c52f0 msgr2=0x55da8089bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.319+0000 7f42426cb640 1 --2- 172.21.15.43:0/1197755594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804c52f0 0x55da8089bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.319+0000 7f42426cb640 1 -- 172.21.15.43:0/1197755594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804cd4d0 msgr2=0x55da8089c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.319+0000 7f42426cb640 1 --2- 172.21.15.43:0/1197755594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804cd4d0 0x55da8089c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.319+0000 7f42426cb640 1 -- 172.21.15.43:0/1197755594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da80506210 con 0x55da804ee1b0 2024-04-25T10:41:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.319+0000 7f42426cb640 1 --2- 172.21.15.43:0/1197755594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 0x55da8089a220 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f4234010960 tx=0x7f4234004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1163463f53a52058 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.319+0000 7f42257fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.319+0000 7f42257fa640 1 -- 172.21.15.43:0/1197755594 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423400dce0 con 0x55da804ee1b0 2024-04-25T10:41:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.319+0000 7f424afdc740 1 -- 172.21.15.43:0/1197755594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55da80802070 con 0x55da804ee1b0 2024-04-25T10:41:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.319+0000 7f42257fa640 1 -- 172.21.15.43:0/1197755594 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f4234027400 con 0x55da804ee1b0 2024-04-25T10:41:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.319+0000 7f42257fa640 1 -- 172.21.15.43:0/1197755594 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4234027600 con 0x55da804ee1b0 2024-04-25T10:41:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.319+0000 7f424afdc740 10 client.18198 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.319+0000 7f424afdc740 1 -- 172.21.15.43:0/1197755594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55da805d2620 con 0x55da804ee1b0 2024-04-25T10:41:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.319+0000 7f42257fa640 1 -- 172.21.15.43:0/1197755594 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55da80802070 con 0x55da804ee1b0 2024-04-25T10:41:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.320+0000 7f42257fa640 1 -- 172.21.15.43:0/1197755594 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f423401f600 con 0x55da804ee1b0 2024-04-25T10:41:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.320+0000 7f424afdc740 10 client.18198 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.320+0000 7f424afdc740 10 client.18198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.320+0000 7f424afdc740 20 client.18198 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.320+0000 7f424afdc740 1 --2- 172.21.15.43:0/1197755594 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55da804acd80 0x55da808ac730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.320+0000 7f424afdc740 4 client.18198 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:41:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.320+0000 7f424afdc740 1 -- 172.21.15.43:0/1197755594 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da807fbee0 con 0x55da804acd80 2024-04-25T10:41:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.320+0000 7f4241eca640 1 --2- 172.21.15.43:0/1197755594 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55da804acd80 0x55da808ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.320+0000 7f4241eca640 1 --2- 172.21.15.43:0/1197755594 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55da804acd80 0x55da808ac730 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fba47563e5aa72c server_cookie=bd0680ddf7d5a941 in_seq=0 out_seq=0 2024-04-25T10:41:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.320+0000 7f42257fa640 10 client.18198 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.321+0000 7f42257fa640 1 -- 172.21.15.43:0/1197755594 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55da807fbee0 con 0x55da804acd80 2024-04-25T10:41:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.321+0000 7f42257fa640 10 client.18198 handle_command_reply: tid=0 2024-04-25T10:41:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.321+0000 7f424afdc740 1 client.18198 shutdown 2024-04-25T10:41:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.322+0000 7f424afdc740 1 -- 172.21.15.43:0/1197755594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 msgr2=0x55da8089a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.322+0000 7f424afdc740 1 --2- 172.21.15.43:0/1197755594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 0x55da8089a220 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f4234010960 tx=0x7f4234004a40 comp rx=0 tx=0).stop 2024-04-25T10:41:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.322+0000 7f424afdc740 1 -- 172.21.15.43:0/1197755594 shutdown_connections 2024-04-25T10:41:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.322+0000 7f424afdc740 1 --2- 172.21.15.43:0/1197755594 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55da804acd80 0x55da808ac730 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.322+0000 7f424afdc740 1 --2- 172.21.15.43:0/1197755594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804cd4d0 0x55da8089c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.322+0000 7f424afdc740 1 --2- 172.21.15.43:0/1197755594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 0x55da8089a220 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.322+0000 7f42257fa640 0 client.18198 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.322+0000 7f424afdc740 1 --2- 172.21.15.43:0/1197755594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804c52f0 0x55da8089bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.322+0000 7f424afdc740 1 -- 172.21.15.43:0/1197755594 >> 172.21.15.43:0/1197755594 conn(0x55da804e2300 msgr2=0x55da80891a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.322+0000 7f424afdc740 1 -- 172.21.15.43:0/1197755594 shutdown_connections 2024-04-25T10:41:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.322+0000 7f424afdc740 1 -- 172.21.15.43:0/1197755594 wait complete. 2024-04-25T10:41:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.322+0000 7f424afdc740 20 client.18198 trim_cache size 0 max 16384 2024-04-25T10:41:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.337+0000 7f424afdc740 1 Processor -- start 2024-04-25T10:41:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.337+0000 7f424afdc740 1 -- start start 2024-04-25T10:41:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.337+0000 7f424afdc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804c52f0 0x55da8089beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.337+0000 7f424afdc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804cd4d0 0x55da80893a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.337+0000 7f424afdc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 0x55da80893fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.337+0000 7f424afdc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1b0150 con 0x55da804ee1b0 2024-04-25T10:41:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.337+0000 7f424afdc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1ada20 con 0x55da804c52f0 2024-04-25T10:41:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.337+0000 7f424afdc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1ab890 con 0x55da804cd4d0 2024-04-25T10:41:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.338+0000 7f42426cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 0x55da80893fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.338+0000 7f42426cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 0x55da80893fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48518/0 (socket says 172.21.15.43:48518) 2024-04-25T10:41:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.338+0000 7f42426cb640 1 -- 172.21.15.43:0/3778858443 learned_addr learned my addr 172.21.15.43:0/3778858443 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.338+0000 7f42416c9640 1 --2- 172.21.15.43:0/3778858443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804cd4d0 0x55da80893a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.338+0000 7f4241eca640 1 --2- 172.21.15.43:0/3778858443 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804c52f0 0x55da8089beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.338+0000 7f42426cb640 1 -- 172.21.15.43:0/3778858443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804cd4d0 msgr2=0x55da80893a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.338+0000 7f42426cb640 1 --2- 172.21.15.43:0/3778858443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804cd4d0 0x55da80893a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.338+0000 7f42426cb640 1 -- 172.21.15.43:0/3778858443 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804c52f0 msgr2=0x55da8089beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.338+0000 7f42426cb640 1 --2- 172.21.15.43:0/3778858443 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804c52f0 0x55da8089beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.338+0000 7f42426cb640 1 -- 172.21.15.43:0/3778858443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da8089a770 con 0x55da804ee1b0 2024-04-25T10:41:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.338+0000 7f42416c9640 1 --2- 172.21.15.43:0/3778858443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804cd4d0 0x55da80893a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.338+0000 7f4241eca640 1 --2- 172.21.15.43:0/3778858443 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804c52f0 0x55da8089beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.338+0000 7f42426cb640 1 --2- 172.21.15.43:0/3778858443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 0x55da80893fa0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f423401fa20 tx=0x7f4234009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b007bd16d83e32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.338+0000 7f42277fe640 1 -- 172.21.15.43:0/3778858443 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234034040 con 0x55da804ee1b0 2024-04-25T10:41:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.338+0000 7f42277fe640 1 -- 172.21.15.43:0/3778858443 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f423403d070 con 0x55da804ee1b0 2024-04-25T10:41:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.338+0000 7f42277fe640 1 -- 172.21.15.43:0/3778858443 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234027a00 con 0x55da804ee1b0 2024-04-25T10:41:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.339+0000 7f424afdc740 1 -- 172.21.15.43:0/3778858443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 msgr2=0x55da80893fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.339+0000 7f424afdc740 1 --2- 172.21.15.43:0/3778858443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 0x55da80893fa0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f423401fa20 tx=0x7f4234009990 comp rx=0 tx=0).stop 2024-04-25T10:41:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.339+0000 7f424afdc740 1 -- 172.21.15.43:0/3778858443 shutdown_connections 2024-04-25T10:41:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.339+0000 7f424afdc740 1 --2- 172.21.15.43:0/3778858443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 0x55da80893fa0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.339+0000 7f424afdc740 1 --2- 172.21.15.43:0/3778858443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804cd4d0 0x55da80893a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.339+0000 7f424afdc740 1 --2- 172.21.15.43:0/3778858443 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804c52f0 0x55da8089beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.339+0000 7f424afdc740 1 -- 172.21.15.43:0/3778858443 >> 172.21.15.43:0/3778858443 conn(0x55da804e2300 msgr2=0x55da8089b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.339+0000 7f424afdc740 1 -- 172.21.15.43:0/3778858443 shutdown_connections 2024-04-25T10:41:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.339+0000 7f424afdc740 1 -- 172.21.15.43:0/3778858443 wait complete. 2024-04-25T10:41:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.339+0000 7f424afdc740 1 -- start start 2024-04-25T10:41:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.340+0000 7f424afdc740 1 Processor -- start 2024-04-25T10:41:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.340+0000 7f424afdc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804c52f0 0x55da808a2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.340+0000 7f424afdc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804cd4d0 0x55da808a2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.340+0000 7f424afdc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 0x55da8089d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.340+0000 7f424afdc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1abee0 con 0x55da804ee1b0 2024-04-25T10:41:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.340+0000 7f424afdc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1ad350 con 0x55da804c52f0 2024-04-25T10:41:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.340+0000 7f424afdc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1ac5b0 con 0x55da804cd4d0 2024-04-25T10:41:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.340+0000 7f424afdc740 10 client.0 _ll_register_callbacks cb 0x7f4234009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.340+0000 7f42426cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 0x55da8089d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.340+0000 7f42426cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 0x55da8089d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48522/0 (socket says 172.21.15.43:48522) 2024-04-25T10:41:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.340+0000 7f42426cb640 1 -- 172.21.15.43:0/2283203215 learned_addr learned my addr 172.21.15.43:0/2283203215 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.340+0000 7f4241eca640 1 --2- 172.21.15.43:0/2283203215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804c52f0 0x55da808a2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.340+0000 7f42416c9640 1 --2- 172.21.15.43:0/2283203215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804cd4d0 0x55da808a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.340+0000 7f42426cb640 1 -- 172.21.15.43:0/2283203215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804cd4d0 msgr2=0x55da808a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.340+0000 7f42426cb640 1 --2- 172.21.15.43:0/2283203215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804cd4d0 0x55da808a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.340+0000 7f42426cb640 1 -- 172.21.15.43:0/2283203215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804c52f0 msgr2=0x55da808a2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.340+0000 7f42426cb640 1 --2- 172.21.15.43:0/2283203215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804c52f0 0x55da808a2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.340+0000 7f42426cb640 1 -- 172.21.15.43:0/2283203215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da8089dd50 con 0x55da804ee1b0 2024-04-25T10:41:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.340+0000 7f4241eca640 1 --2- 172.21.15.43:0/2283203215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804c52f0 0x55da808a2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.341+0000 7f42426cb640 1 --2- 172.21.15.43:0/2283203215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 0x55da8089d5c0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f423401fac0 tx=0x7f423401faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7563d43805fc50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.341+0000 7f42257fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.341+0000 7f42257fa640 1 -- 172.21.15.43:0/2283203215 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234045070 con 0x55da804ee1b0 2024-04-25T10:41:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.341+0000 7f42257fa640 1 -- 172.21.15.43:0/2283203215 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f4234004a00 con 0x55da804ee1b0 2024-04-25T10:41:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.341+0000 7f424afdc740 1 -- 172.21.15.43:0/2283203215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f423403d070 con 0x55da804ee1b0 2024-04-25T10:41:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.341+0000 7f42257fa640 1 -- 172.21.15.43:0/2283203215 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4234034040 con 0x55da804ee1b0 2024-04-25T10:41:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.341+0000 7f424afdc740 10 client.18204 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.341+0000 7f424afdc740 1 -- 172.21.15.43:0/2283203215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55da80914530 con 0x55da804ee1b0 2024-04-25T10:41:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.341+0000 7f42257fa640 1 -- 172.21.15.43:0/2283203215 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f423403d070 con 0x55da804ee1b0 2024-04-25T10:41:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.341+0000 7f42257fa640 1 -- 172.21.15.43:0/2283203215 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4234037070 con 0x55da804ee1b0 2024-04-25T10:41:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.342+0000 7f424afdc740 10 client.18204 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.342+0000 7f424afdc740 10 client.18204 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.342+0000 7f424afdc740 20 client.18204 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.342+0000 7f424afdc740 1 --2- 172.21.15.43:0/2283203215 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55da804821c0 0x55da80916bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.342+0000 7f424afdc740 4 client.18204 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:41:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.342+0000 7f424afdc740 1 -- 172.21.15.43:0/2283203215 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55da807775f0 con 0x55da804821c0 2024-04-25T10:41:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.342+0000 7f4241eca640 1 --2- 172.21.15.43:0/2283203215 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55da804821c0 0x55da80916bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.342+0000 7f4241eca640 1 --2- 172.21.15.43:0/2283203215 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55da804821c0 0x55da80916bb0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c88eb0a6fdc081c server_cookie=897d36c5d13b3e29 in_seq=0 out_seq=0 2024-04-25T10:41:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.342+0000 7f42257fa640 10 client.18204 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.360+0000 7f42257fa640 1 -- 172.21.15.43:0/2283203215 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55da807775f0 con 0x55da804821c0 2024-04-25T10:41:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.360+0000 7f42257fa640 10 client.18204 handle_command_reply: tid=0 2024-04-25T10:41:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.360+0000 7f424afdc740 1 client.18204 shutdown 2024-04-25T10:41:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.360+0000 7f424afdc740 1 -- 172.21.15.43:0/2283203215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 msgr2=0x55da8089d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.360+0000 7f424afdc740 1 --2- 172.21.15.43:0/2283203215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 0x55da8089d5c0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f423401fac0 tx=0x7f423401faf0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.360+0000 7f424afdc740 1 -- 172.21.15.43:0/2283203215 shutdown_connections 2024-04-25T10:41:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.360+0000 7f424afdc740 1 --2- 172.21.15.43:0/2283203215 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55da804821c0 0x55da80916bb0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.360+0000 7f424afdc740 1 --2- 172.21.15.43:0/2283203215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da804ee1b0 0x55da8089d5c0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.360+0000 7f424afdc740 1 --2- 172.21.15.43:0/2283203215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da804cd4d0 0x55da808a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.360+0000 7f424afdc740 1 --2- 172.21.15.43:0/2283203215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da804c52f0 0x55da808a2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.360+0000 7f424afdc740 1 -- 172.21.15.43:0/2283203215 >> 172.21.15.43:0/2283203215 conn(0x55da804e2300 msgr2=0x55da8089b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.360+0000 7f42257fa640 0 client.18204 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.360+0000 7f424afdc740 1 -- 172.21.15.43:0/2283203215 shutdown_connections 2024-04-25T10:41:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.360+0000 7f424afdc740 1 -- 172.21.15.43:0/2283203215 wait complete. 2024-04-25T10:41:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.360+0000 7f424afdc740 20 client.18204 trim_cache size 0 max 16384 2024-04-25T10:41:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:41:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:41:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:41:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:41:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.361+0000 7f424412e640 1 -- 172.21.15.43:0/3993395711 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f42280afcd0 msgr2=0x7f42280b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.361+0000 7f424412e640 1 --2- 172.21.15.43:0/3993395711 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f42280afcd0 0x7f42280b21e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55da8088ce70 tx=0x7f4234009500 comp rx=0 tx=0).stop 2024-04-25T10:41:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.361+0000 7f424412e640 1 -- 172.21.15.43:0/3993395711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f423c1069b0 msgr2=0x7f423c199c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.361+0000 7f424412e640 1 --2- 172.21.15.43:0/3993395711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f423c1069b0 0x7f423c199c00 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f422c006010 tx=0x7f422c004060 comp rx=0 tx=0).stop 2024-04-25T10:41:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.361+0000 7f424412e640 1 -- 172.21.15.43:0/3993395711 shutdown_connections 2024-04-25T10:41:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.361+0000 7f424412e640 1 --2- 172.21.15.43:0/3993395711 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f42280afcd0 0x7f42280b21e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.361+0000 7f424412e640 1 --2- 172.21.15.43:0/3993395711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c107470 0x7f423c19dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.361+0000 7f424412e640 1 --2- 172.21.15.43:0/3993395711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f423c1069b0 0x7f423c199c00 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.361+0000 7f424412e640 1 --2- 172.21.15.43:0/3993395711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f423c102420 0x7f423c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.361+0000 7f424412e640 1 -- 172.21.15.43:0/3993395711 >> 172.21.15.43:0/3993395711 conn(0x7f423c0fd880 msgr2=0x7f423c105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.361+0000 7f424412e640 1 -- 172.21.15.43:0/3993395711 shutdown_connections 2024-04-25T10:41:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:43.361+0000 7f424412e640 1 -- 172.21.15.43:0/3993395711 wait complete. 2024-04-25T10:41:44.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:44 smithi077 ceph-mon[32772]: pgmap v285: 97 pgs: 97 active+clean; 4.6 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:41:44.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:44 smithi164 ceph-mon[32713]: pgmap v285: 97 pgs: 97 active+clean; 4.6 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:41:44.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:44 smithi043 ceph-mon[29478]: pgmap v285: 97 pgs: 97 active+clean; 4.6 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:41:46.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:46 smithi077 ceph-mon[32772]: pgmap v286: 97 pgs: 97 active+clean; 4.8 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 19 op/s 2024-04-25T10:41:46.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:46 smithi164 ceph-mon[32713]: pgmap v286: 97 pgs: 97 active+clean; 4.8 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 19 op/s 2024-04-25T10:41:46.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:46 smithi043 ceph-mon[29478]: pgmap v286: 97 pgs: 97 active+clean; 4.8 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 19 op/s 2024-04-25T10:41:48.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:48 smithi077 ceph-mon[32772]: pgmap v287: 97 pgs: 97 active+clean; 4.8 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:41:48.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:48 smithi164 ceph-mon[32713]: pgmap v287: 97 pgs: 97 active+clean; 4.8 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:41:48.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:48 smithi043 ceph-mon[29478]: pgmap v287: 97 pgs: 97 active+clean; 4.8 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:41:49.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:49 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:41:49.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:49 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:41:49.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:49 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:41:50.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:50 smithi077 ceph-mon[32772]: pgmap v288: 97 pgs: 97 active+clean; 5.0 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 62 KiB/s wr, 17 op/s 2024-04-25T10:41:50.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:50 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:41:50.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:50 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:41:50.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:50 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:41:50.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:50 smithi164 ceph-mon[32713]: pgmap v288: 97 pgs: 97 active+clean; 5.0 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 62 KiB/s wr, 17 op/s 2024-04-25T10:41:50.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:50 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:41:50.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:50 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:41:50.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:50 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:41:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:50 smithi043 ceph-mon[29478]: pgmap v288: 97 pgs: 97 active+clean; 5.0 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 62 KiB/s wr, 17 op/s 2024-04-25T10:41:50.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:50 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:41:50.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:50 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:41:50.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:50 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:41:50.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:41:50.474 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:41:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.599+0000 7f2694972640 1 -- 172.21.15.43:0/1952679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269010d380 msgr2=0x7f269010f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.599+0000 7f2694972640 1 --2- 172.21.15.43:0/1952679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269010d380 0x7f269010f7a0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f2688009ca0 tx=0x7f268802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.599+0000 7f2694972640 1 -- 172.21.15.43:0/1952679 shutdown_connections 2024-04-25T10:41:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.599+0000 7f2694972640 1 --2- 172.21.15.43:0/1952679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2690102af0 0x7f2690102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.599+0000 7f2694972640 1 --2- 172.21.15.43:0/1952679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269010d380 0x7f269010f7a0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.599+0000 7f2694972640 1 --2- 172.21.15.43:0/1952679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2690108a20 0x7f26901025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.599+0000 7f2694972640 1 -- 172.21.15.43:0/1952679 >> 172.21.15.43:0/1952679 conn(0x7f2690078100 msgr2=0x7f2690078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.599+0000 7f2694972640 1 -- 172.21.15.43:0/1952679 shutdown_connections 2024-04-25T10:41:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.599+0000 7f2694972640 1 -- 172.21.15.43:0/1952679 wait complete. 2024-04-25T10:41:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.600+0000 7f2694972640 1 Processor -- start 2024-04-25T10:41:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.600+0000 7f2694972640 1 -- start start 2024-04-25T10:41:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.601+0000 7f2694972640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2690102af0 0x7f269019b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.601+0000 7f2694972640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2690108a20 0x7f269019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.601+0000 7f2694972640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269010d380 0x7f269019bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.601+0000 7f2694972640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2690112c00 con 0x7f269010d380 2024-04-25T10:41:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.601+0000 7f2694972640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2690112a80 con 0x7f2690102af0 2024-04-25T10:41:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.601+0000 7f2694972640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2690112d80 con 0x7f2690108a20 2024-04-25T10:41:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.601+0000 7f268ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269010d380 0x7f269019bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.601+0000 7f268ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269010d380 0x7f269019bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34614/0 (socket says 172.21.15.43:34614) 2024-04-25T10:41:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.601+0000 7f268ed9d640 1 -- 172.21.15.43:0/1765380360 learned_addr learned my addr 172.21.15.43:0/1765380360 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.601+0000 7f268e59c640 1 --2- 172.21.15.43:0/1765380360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2690102af0 0x7f269019b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.601+0000 7f268dd9b640 1 --2- 172.21.15.43:0/1765380360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2690108a20 0x7f269019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.601+0000 7f268ed9d640 1 -- 172.21.15.43:0/1765380360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2690108a20 msgr2=0x7f269019ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.601+0000 7f268ed9d640 1 --2- 172.21.15.43:0/1765380360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2690108a20 0x7f269019ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.601+0000 7f268ed9d640 1 -- 172.21.15.43:0/1765380360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2690102af0 msgr2=0x7f269019b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.601+0000 7f268ed9d640 1 --2- 172.21.15.43:0/1765380360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2690102af0 0x7f269019b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.601+0000 7f268ed9d640 1 -- 172.21.15.43:0/1765380360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26901a6860 con 0x7f269010d380 2024-04-25T10:41:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.601+0000 7f268ed9d640 1 --2- 172.21.15.43:0/1765380360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269010d380 0x7f269019bfc0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f26880098d0 tx=0x7f26880317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.601+0000 7f267f7fe640 1 -- 172.21.15.43:0/1765380360 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2688003b70 con 0x7f269010d380 2024-04-25T10:41:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.602+0000 7f2694972640 1 -- 172.21.15.43:0/1765380360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26901a6af0 con 0x7f269010d380 2024-04-25T10:41:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.602+0000 7f267f7fe640 1 -- 172.21.15.43:0/1765380360 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2688003d10 con 0x7f269010d380 2024-04-25T10:41:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.602+0000 7f267f7fe640 1 -- 172.21.15.43:0/1765380360 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268802fa20 con 0x7f269010d380 2024-04-25T10:41:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.602+0000 7f2694972640 1 -- 172.21.15.43:0/1765380360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26901a7040 con 0x7f269010d380 2024-04-25T10:41:50.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.604+0000 7f267f7fe640 1 -- 172.21.15.43:0/1765380360 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f268802fc40 con 0x7f269010d380 2024-04-25T10:41:50.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.604+0000 7f269b81d740 1 Processor -- start 2024-04-25T10:41:50.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.604+0000 7f269b81d740 1 -- start start 2024-04-25T10:41:50.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.605+0000 7f269b81d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cbd40e390 0x558cbd7aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.605+0000 7f269b81d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cbd3e54d0 0x558cbd7aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.605+0000 7f269b81d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ed6b0 0x558cbd7bb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.605+0000 7f269b81d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558cbd714d80 con 0x558cbd3ed6b0 2024-04-25T10:41:50.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.605+0000 7f269b81d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558cbd6ce8b0 con 0x558cbd3e54d0 2024-04-25T10:41:50.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.605+0000 7f269b81d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558cbd710d40 con 0x558cbd40e390 2024-04-25T10:41:50.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.605+0000 7f268dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ed6b0 0x558cbd7bb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:50.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.605+0000 7f268dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ed6b0 0x558cbd7bb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34624/0 (socket says 172.21.15.43:34624) 2024-04-25T10:41:50.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.605+0000 7f268dd9b640 1 -- 172.21.15.43:0/816263271 learned_addr learned my addr 172.21.15.43:0/816263271 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:50.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.605+0000 7f268e59c640 1 --2- 172.21.15.43:0/816263271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cbd3e54d0 0x558cbd7aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:50.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.605+0000 7f268dd9b640 1 --2- 172.21.15.43:0/816263271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cbd40e390 0x558cbd7aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:50.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.605+0000 7f267f7fe640 1 --2- 172.21.15.43:0/1765380360 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26780afc60 0x7f26780b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.605+0000 7f268dd9b640 1 -- 172.21.15.43:0/816263271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cbd40e390 msgr2=0x558cbd7aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:50.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.605+0000 7f268dd9b640 1 --2- 172.21.15.43:0/816263271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cbd40e390 0x558cbd7aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.605+0000 7f268dd9b640 1 -- 172.21.15.43:0/816263271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cbd3e54d0 msgr2=0x558cbd7aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:50.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.605+0000 7f268dd9b640 1 --2- 172.21.15.43:0/816263271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cbd3e54d0 0x558cbd7aefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.605+0000 7f268dd9b640 1 -- 172.21.15.43:0/816263271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cbd69fa40 con 0x558cbd3ed6b0 2024-04-25T10:41:50.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.605+0000 7f268ed9d640 1 --2- 172.21.15.43:0/1765380360 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26780afc60 0x7f26780b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:50.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.605+0000 7f268dd9b640 1 --2- 172.21.15.43:0/816263271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ed6b0 0x558cbd7bb8f0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f26840058c0 tx=0x7f2684005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c3dea7918fc17f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:50.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.605+0000 7f267f7fe640 1 -- 172.21.15.43:0/1765380360 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f26881003e0 con 0x7f269010d380 2024-04-25T10:41:50.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.606+0000 7f267cff9640 1 -- 172.21.15.43:0/816263271 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2684004030 con 0x558cbd3ed6b0 2024-04-25T10:41:50.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.606+0000 7f267cff9640 1 -- 172.21.15.43:0/816263271 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2684005020 con 0x558cbd3ed6b0 2024-04-25T10:41:50.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.606+0000 7f267cff9640 1 -- 172.21.15.43:0/816263271 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268400fb60 con 0x558cbd3ed6b0 2024-04-25T10:41:50.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.607+0000 7f269b81d740 1 -- 172.21.15.43:0/816263271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ed6b0 msgr2=0x558cbd7bb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:50.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.607+0000 7f269b81d740 1 --2- 172.21.15.43:0/816263271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ed6b0 0x558cbd7bb8f0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f26840058c0 tx=0x7f2684005850 comp rx=0 tx=0).stop 2024-04-25T10:41:50.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.607+0000 7f268ed9d640 1 --2- 172.21.15.43:0/1765380360 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26780afc60 0x7f26780b2170 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f26880fdcf0 tx=0x7f268802a8c0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:50.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.607+0000 7f269b81d740 1 -- 172.21.15.43:0/816263271 shutdown_connections 2024-04-25T10:41:50.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.607+0000 7f269b81d740 1 --2- 172.21.15.43:0/816263271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cbd3e54d0 0x558cbd7aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.607+0000 7f269b81d740 1 --2- 172.21.15.43:0/816263271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ed6b0 0x558cbd7bb8f0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.607+0000 7f269b81d740 1 --2- 172.21.15.43:0/816263271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cbd40e390 0x558cbd7aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.607+0000 7f269b81d740 1 -- 172.21.15.43:0/816263271 >> 172.21.15.43:0/816263271 conn(0x558cbd4024e0 msgr2=0x558cbd7ab210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:50.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.607+0000 7f269b81d740 1 -- 172.21.15.43:0/816263271 shutdown_connections 2024-04-25T10:41:50.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.607+0000 7f269b81d740 1 -- 172.21.15.43:0/816263271 wait complete. 2024-04-25T10:41:50.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.607+0000 7f269b81d740 1 -- start start 2024-04-25T10:41:50.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.607+0000 7f269b81d740 1 Processor -- start 2024-04-25T10:41:50.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.607+0000 7f269b81d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cbd3e54d0 0x558cbd7b95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.608+0000 7f269b81d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cbd3ed6b0 0x558cbd7b9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.608+0000 7f269b81d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd40e390 0x558cbd7bf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.608+0000 7f269b81d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558cbd7bbfe0 con 0x558cbd40e390 2024-04-25T10:41:50.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.608+0000 7f269b81d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558cbd69bd10 con 0x558cbd3e54d0 2024-04-25T10:41:50.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.608+0000 7f269b81d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558cbd7bc160 con 0x558cbd3ed6b0 2024-04-25T10:41:50.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.608+0000 7f269b81d740 10 client.0 _ll_register_callbacks cb 0x558cbd6c8210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:50.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.608+0000 7f268dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd40e390 0x558cbd7bf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:50.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.608+0000 7f268dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd40e390 0x558cbd7bf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34640/0 (socket says 172.21.15.43:34640) 2024-04-25T10:41:50.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.608+0000 7f268dd9b640 1 -- 172.21.15.43:0/647363696 learned_addr learned my addr 172.21.15.43:0/647363696 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:50.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.608+0000 7f268dd9b640 1 --2- 172.21.15.43:0/647363696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cbd3e54d0 0x558cbd7b95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:50.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.608+0000 7f268e59c640 1 --2- 172.21.15.43:0/647363696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cbd3ed6b0 0x558cbd7b9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:50.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.608+0000 7f268dd9b640 1 -- 172.21.15.43:0/647363696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cbd3ed6b0 msgr2=0x558cbd7b9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:50.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.608+0000 7f268dd9b640 1 --2- 172.21.15.43:0/647363696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cbd3ed6b0 0x558cbd7b9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.608+0000 7f268dd9b640 1 -- 172.21.15.43:0/647363696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cbd3e54d0 msgr2=0x558cbd7b95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:50.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.608+0000 7f268dd9b640 1 --2- 172.21.15.43:0/647363696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cbd3e54d0 0x558cbd7b95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.608+0000 7f268dd9b640 1 -- 172.21.15.43:0/647363696 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cbd4f07d0 con 0x558cbd40e390 2024-04-25T10:41:50.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.608+0000 7f268dd9b640 1 --2- 172.21.15.43:0/647363696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd40e390 0x558cbd7bf470 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x558cbd7b8c20 tx=0x7f2684009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd47a0c0fdfa9dd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:50.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.608+0000 7f2675ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:50.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.608+0000 7f2675ffb640 1 -- 172.21.15.43:0/647363696 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2684005590 con 0x558cbd40e390 2024-04-25T10:41:50.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.608+0000 7f2675ffb640 1 -- 172.21.15.43:0/647363696 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f2684019040 con 0x558cbd40e390 2024-04-25T10:41:50.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.609+0000 7f269b81d740 1 -- 172.21.15.43:0/647363696 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2684005020 con 0x558cbd40e390 2024-04-25T10:41:50.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.609+0000 7f2675ffb640 1 -- 172.21.15.43:0/647363696 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f268400fdd0 con 0x558cbd40e390 2024-04-25T10:41:50.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.609+0000 7f269b81d740 10 client.18216 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:50.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.609+0000 7f269b81d740 1 -- 172.21.15.43:0/647363696 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558cbd4263f0 con 0x558cbd40e390 2024-04-25T10:41:50.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.609+0000 7f2675ffb640 1 -- 172.21.15.43:0/647363696 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2684005020 con 0x558cbd40e390 2024-04-25T10:41:50.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.609+0000 7f2675ffb640 1 -- 172.21.15.43:0/647363696 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2684004d00 con 0x558cbd40e390 2024-04-25T10:41:50.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.609+0000 7f269b81d740 10 client.18216 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:50.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.609+0000 7f269b81d740 10 client.18216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:50.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.609+0000 7f269b81d740 20 client.18216 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:50.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.609+0000 7f269b81d740 1 --2- 172.21.15.43:0/647363696 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558cbd3ccf60 0x558cbd7cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.609+0000 7f269b81d740 4 client.18216 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:41:50.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.609+0000 7f269b81d740 1 -- 172.21.15.43:0/647363696 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cbd722990 con 0x558cbd3ccf60 2024-04-25T10:41:50.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.610+0000 7f268e59c640 1 --2- 172.21.15.43:0/647363696 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558cbd3ccf60 0x558cbd7cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:50.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.610+0000 7f2675ffb640 10 client.18216 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:50.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.610+0000 7f268e59c640 1 --2- 172.21.15.43:0/647363696 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558cbd3ccf60 0x558cbd7cb500 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0553bd3d6442d78 server_cookie=fd786d3164b3baf1 in_seq=0 out_seq=0 2024-04-25T10:41:50.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.611+0000 7f2675ffb640 1 -- 172.21.15.43:0/647363696 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558cbd722990 con 0x558cbd3ccf60 2024-04-25T10:41:50.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.611+0000 7f2675ffb640 10 client.18216 handle_command_reply: tid=0 2024-04-25T10:41:50.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.611+0000 7f269b81d740 1 client.18216 shutdown 2024-04-25T10:41:50.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.611+0000 7f269b81d740 1 -- 172.21.15.43:0/647363696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd40e390 msgr2=0x558cbd7bf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:50.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.611+0000 7f269b81d740 1 --2- 172.21.15.43:0/647363696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd40e390 0x558cbd7bf470 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x558cbd7b8c20 tx=0x7f2684009660 comp rx=0 tx=0).stop 2024-04-25T10:41:50.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.611+0000 7f269b81d740 1 -- 172.21.15.43:0/647363696 shutdown_connections 2024-04-25T10:41:50.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.611+0000 7f269b81d740 1 --2- 172.21.15.43:0/647363696 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558cbd3ccf60 0x558cbd7cb500 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.611+0000 7f269b81d740 1 --2- 172.21.15.43:0/647363696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd40e390 0x558cbd7bf470 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.611+0000 7f269b81d740 1 --2- 172.21.15.43:0/647363696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cbd3ed6b0 0x558cbd7b9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.611+0000 7f2675ffb640 0 client.18216 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:50.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.611+0000 7f269b81d740 1 --2- 172.21.15.43:0/647363696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cbd3e54d0 0x558cbd7b95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.611+0000 7f269b81d740 1 -- 172.21.15.43:0/647363696 >> 172.21.15.43:0/647363696 conn(0x558cbd4024e0 msgr2=0x558cbd7b0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:50.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.611+0000 7f269b81d740 1 -- 172.21.15.43:0/647363696 shutdown_connections 2024-04-25T10:41:50.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.611+0000 7f269b81d740 1 -- 172.21.15.43:0/647363696 wait complete. 2024-04-25T10:41:50.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.611+0000 7f269b81d740 20 client.18216 trim_cache size 0 max 16384 2024-04-25T10:41:50.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.629+0000 7f269b81d740 1 Processor -- start 2024-04-25T10:41:50.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.629+0000 7f269b81d740 1 -- start start 2024-04-25T10:41:50.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.630+0000 7f269b81d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ccf60 0x558cbd7b8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.630+0000 7f269b81d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cbd3e54d0 0x558cbd7d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.630+0000 7f269b81d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cbd3ed6b0 0x558cbd7d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.630+0000 7f269b81d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26901b22c0 con 0x558cbd3ccf60 2024-04-25T10:41:50.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.630+0000 7f269b81d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f26901afb90 con 0x558cbd3ed6b0 2024-04-25T10:41:50.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.630+0000 7f269b81d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f26901ada00 con 0x558cbd3e54d0 2024-04-25T10:41:50.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.630+0000 7f268dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ccf60 0x558cbd7b8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:50.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.630+0000 7f268dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ccf60 0x558cbd7b8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34644/0 (socket says 172.21.15.43:34644) 2024-04-25T10:41:50.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.630+0000 7f268dd9b640 1 -- 172.21.15.43:0/1832484028 learned_addr learned my addr 172.21.15.43:0/1832484028 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:50.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.630+0000 7f268dd9b640 1 -- 172.21.15.43:0/1832484028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cbd3e54d0 msgr2=0x558cbd7d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:50.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.630+0000 7f268dd9b640 1 --2- 172.21.15.43:0/1832484028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cbd3e54d0 0x558cbd7d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.630+0000 7f268dd9b640 1 -- 172.21.15.43:0/1832484028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cbd3ed6b0 msgr2=0x558cbd7d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:50.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.630+0000 7f268dd9b640 1 --2- 172.21.15.43:0/1832484028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cbd3ed6b0 0x558cbd7d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.630+0000 7f268dd9b640 1 -- 172.21.15.43:0/1832484028 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cbd3e60f0 con 0x558cbd3ccf60 2024-04-25T10:41:50.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.630+0000 7f268dd9b640 1 --2- 172.21.15.43:0/1832484028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ccf60 0x558cbd7b8270 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f26840097c0 tx=0x7f2684003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1231eca54e876ca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:50.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.630+0000 7f2677fff640 1 -- 172.21.15.43:0/1832484028 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26840050c0 con 0x558cbd3ccf60 2024-04-25T10:41:50.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.630+0000 7f2677fff640 1 -- 172.21.15.43:0/1832484028 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2684005450 con 0x558cbd3ccf60 2024-04-25T10:41:50.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.630+0000 7f2677fff640 1 -- 172.21.15.43:0/1832484028 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2684003e50 con 0x558cbd3ccf60 2024-04-25T10:41:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.631+0000 7f269b81d740 1 -- 172.21.15.43:0/1832484028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ccf60 msgr2=0x558cbd7b8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.631+0000 7f269b81d740 1 --2- 172.21.15.43:0/1832484028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ccf60 0x558cbd7b8270 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f26840097c0 tx=0x7f2684003a10 comp rx=0 tx=0).stop 2024-04-25T10:41:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.631+0000 7f269b81d740 1 -- 172.21.15.43:0/1832484028 shutdown_connections 2024-04-25T10:41:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.631+0000 7f269b81d740 1 --2- 172.21.15.43:0/1832484028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cbd3ed6b0 0x558cbd7d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.631+0000 7f269b81d740 1 --2- 172.21.15.43:0/1832484028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cbd3e54d0 0x558cbd7d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.631+0000 7f269b81d740 1 --2- 172.21.15.43:0/1832484028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ccf60 0x558cbd7b8270 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.631+0000 7f269b81d740 1 -- 172.21.15.43:0/1832484028 >> 172.21.15.43:0/1832484028 conn(0x558cbd4024e0 msgr2=0x558cbd7b7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.631+0000 7f269b81d740 1 -- 172.21.15.43:0/1832484028 shutdown_connections 2024-04-25T10:41:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.631+0000 7f269b81d740 1 -- 172.21.15.43:0/1832484028 wait complete. 2024-04-25T10:41:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.631+0000 7f269b81d740 1 -- start start 2024-04-25T10:41:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.632+0000 7f269b81d740 1 Processor -- start 2024-04-25T10:41:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.632+0000 7f269b81d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ccf60 0x558cbd7c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.632+0000 7f269b81d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cbd3e54d0 0x558cbd7c13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.632+0000 7f269b81d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cbd3ed6b0 0x558cbd82b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.632+0000 7f269b81d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26901ae050 con 0x558cbd3ccf60 2024-04-25T10:41:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.632+0000 7f269b81d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f26901af4c0 con 0x558cbd3e54d0 2024-04-25T10:41:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.632+0000 7f269b81d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f26901ae720 con 0x558cbd3ed6b0 2024-04-25T10:41:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.632+0000 7f269b81d740 10 client.0 _ll_register_callbacks cb 0x7f2680004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.632+0000 7f268dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ccf60 0x558cbd7c0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.632+0000 7f268dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ccf60 0x558cbd7c0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34650/0 (socket says 172.21.15.43:34650) 2024-04-25T10:41:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.632+0000 7f268dd9b640 1 -- 172.21.15.43:0/3947373596 learned_addr learned my addr 172.21.15.43:0/3947373596 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.632+0000 7f268e59c640 1 --2- 172.21.15.43:0/3947373596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cbd3e54d0 0x558cbd7c13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.632+0000 7f268dd9b640 1 -- 172.21.15.43:0/3947373596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cbd3ed6b0 msgr2=0x558cbd82b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.632+0000 7f268dd9b640 1 --2- 172.21.15.43:0/3947373596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cbd3ed6b0 0x558cbd82b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.632+0000 7f268dd9b640 1 -- 172.21.15.43:0/3947373596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cbd3e54d0 msgr2=0x558cbd7c13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.632+0000 7f268dd9b640 1 --2- 172.21.15.43:0/3947373596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cbd3e54d0 0x558cbd7c13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.632+0000 7f268dd9b640 1 -- 172.21.15.43:0/3947373596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cbd6d54b0 con 0x558cbd3ccf60 2024-04-25T10:41:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.632+0000 7f268dd9b640 1 --2- 172.21.15.43:0/3947373596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ccf60 0x558cbd7c0e80 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f2684000c00 tx=0x7f268400ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26622df5561374a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.632+0000 7f2675ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.633+0000 7f2675ffb640 1 -- 172.21.15.43:0/3947373596 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268403b070 con 0x558cbd3ccf60 2024-04-25T10:41:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.633+0000 7f2675ffb640 1 -- 172.21.15.43:0/3947373596 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f2684036aa0 con 0x558cbd3ccf60 2024-04-25T10:41:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.633+0000 7f269b81d740 1 -- 172.21.15.43:0/3947373596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2684005450 con 0x558cbd3ccf60 2024-04-25T10:41:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.633+0000 7f2675ffb640 1 -- 172.21.15.43:0/3947373596 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2684009c10 con 0x558cbd3ccf60 2024-04-25T10:41:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.633+0000 7f269b81d740 10 client.18222 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.633+0000 7f269b81d740 1 -- 172.21.15.43:0/3947373596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558cbd832e70 con 0x558cbd3ccf60 2024-04-25T10:41:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.633+0000 7f2675ffb640 1 -- 172.21.15.43:0/3947373596 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2684005450 con 0x558cbd3ccf60 2024-04-25T10:41:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.633+0000 7f2675ffb640 1 -- 172.21.15.43:0/3947373596 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f268401f980 con 0x558cbd3ccf60 2024-04-25T10:41:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.633+0000 7f269b81d740 10 client.18222 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.633+0000 7f269b81d740 10 client.18222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.633+0000 7f269b81d740 20 client.18222 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.633+0000 7f269b81d740 1 --2- 172.21.15.43:0/3947373596 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558cbd40e390 0x558cbd835490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.633+0000 7f269b81d740 4 client.18222 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:41:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.633+0000 7f269b81d740 1 -- 172.21.15.43:0/3947373596 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558cbd71c800 con 0x558cbd40e390 2024-04-25T10:41:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.634+0000 7f268e59c640 1 --2- 172.21.15.43:0/3947373596 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558cbd40e390 0x558cbd835490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.634+0000 7f268e59c640 1 --2- 172.21.15.43:0/3947373596 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558cbd40e390 0x558cbd835490 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=284158b413486c9f server_cookie=364c91fbe0d6e1d7 in_seq=0 out_seq=0 2024-04-25T10:41:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.634+0000 7f2675ffb640 10 client.18222 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.634+0000 7f2675ffb640 1 -- 172.21.15.43:0/3947373596 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558cbd71c800 con 0x558cbd40e390 2024-04-25T10:41:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.634+0000 7f2675ffb640 10 client.18222 handle_command_reply: tid=0 2024-04-25T10:41:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.634+0000 7f269b81d740 1 client.18222 shutdown 2024-04-25T10:41:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.634+0000 7f269b81d740 1 -- 172.21.15.43:0/3947373596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ccf60 msgr2=0x558cbd7c0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.634+0000 7f269b81d740 1 --2- 172.21.15.43:0/3947373596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ccf60 0x558cbd7c0e80 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f2684000c00 tx=0x7f268400ff70 comp rx=0 tx=0).stop 2024-04-25T10:41:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.634+0000 7f269b81d740 1 -- 172.21.15.43:0/3947373596 shutdown_connections 2024-04-25T10:41:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.634+0000 7f269b81d740 1 --2- 172.21.15.43:0/3947373596 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558cbd40e390 0x558cbd835490 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.634+0000 7f269b81d740 1 --2- 172.21.15.43:0/3947373596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cbd3e54d0 0x558cbd7c13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.634+0000 7f269b81d740 1 --2- 172.21.15.43:0/3947373596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cbd3ed6b0 0x558cbd82b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.634+0000 7f2675ffb640 0 client.18222 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.634+0000 7f269b81d740 1 --2- 172.21.15.43:0/3947373596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558cbd3ccf60 0x558cbd7c0e80 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.634+0000 7f269b81d740 1 -- 172.21.15.43:0/3947373596 >> 172.21.15.43:0/3947373596 conn(0x558cbd4024e0 msgr2=0x558cbd7baa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:50.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.634+0000 7f269b81d740 1 -- 172.21.15.43:0/3947373596 shutdown_connections 2024-04-25T10:41:50.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.634+0000 7f269b81d740 1 -- 172.21.15.43:0/3947373596 wait complete. 2024-04-25T10:41:50.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.635+0000 7f269b81d740 20 client.18222 trim_cache size 0 max 16384 2024-04-25T10:41:50.643 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:41:50.643 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:41:50.643 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:41:50.643 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:41:50.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.635+0000 7f2694972640 1 -- 172.21.15.43:0/1765380360 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26780afc60 msgr2=0x7f26780b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:50.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.635+0000 7f2694972640 1 --2- 172.21.15.43:0/1765380360 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26780afc60 0x7f26780b2170 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f26880fdcf0 tx=0x7f268802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.635+0000 7f2694972640 1 -- 172.21.15.43:0/1765380360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269010d380 msgr2=0x7f269019bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:50.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.635+0000 7f2694972640 1 --2- 172.21.15.43:0/1765380360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269010d380 0x7f269019bfc0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f26880098d0 tx=0x7f26880317b0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.635+0000 7f2694972640 1 -- 172.21.15.43:0/1765380360 shutdown_connections 2024-04-25T10:41:50.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.635+0000 7f2694972640 1 --2- 172.21.15.43:0/1765380360 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f26780afc60 0x7f26780b2170 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.635+0000 7f2694972640 1 --2- 172.21.15.43:0/1765380360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269010d380 0x7f269019bfc0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.635+0000 7f2694972640 1 --2- 172.21.15.43:0/1765380360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2690108a20 0x7f269019ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.635+0000 7f2694972640 1 --2- 172.21.15.43:0/1765380360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2690102af0 0x7f269019b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.635+0000 7f2694972640 1 -- 172.21.15.43:0/1765380360 >> 172.21.15.43:0/1765380360 conn(0x7f2690078100 msgr2=0x7f26901040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:50.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.635+0000 7f2694972640 1 -- 172.21.15.43:0/1765380360 shutdown_connections 2024-04-25T10:41:50.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.635+0000 7f2694972640 1 -- 172.21.15.43:0/1765380360 wait complete. 2024-04-25T10:41:50.649 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:41:50.649 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:41:50.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.759+0000 7f63080e9640 1 -- 172.21.15.43:0/1080068075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6300107450 msgr2=0x7f630010ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.759+0000 7f63080e9640 1 --2- 172.21.15.43:0/1080068075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6300107450 0x7f630010ebf0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009ca0 tx=0x7f62f802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.759+0000 7f63080e9640 1 -- 172.21.15.43:0/1080068075 shutdown_connections 2024-04-25T10:41:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.759+0000 7f63080e9640 1 --2- 172.21.15.43:0/1080068075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6300106990 0x7f6300106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.759+0000 7f63080e9640 1 --2- 172.21.15.43:0/1080068075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6300107450 0x7f630010ebf0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.759+0000 7f63080e9640 1 --2- 172.21.15.43:0/1080068075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6300102400 0x7f6300102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.759+0000 7f63080e9640 1 -- 172.21.15.43:0/1080068075 >> 172.21.15.43:0/1080068075 conn(0x7f6300077890 msgr2=0x7f6300077cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.760+0000 7f63080e9640 1 -- 172.21.15.43:0/1080068075 shutdown_connections 2024-04-25T10:41:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.760+0000 7f63080e9640 1 -- 172.21.15.43:0/1080068075 wait complete. 2024-04-25T10:41:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.760+0000 7f63080e9640 1 Processor -- start 2024-04-25T10:41:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.760+0000 7f63080e9640 1 -- start start 2024-04-25T10:41:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.761+0000 7f63080e9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6300102400 0x7f630019b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.761+0000 7f63080e9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6300106990 0x7f630019baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.761+0000 7f63080e9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6300107450 0x7f630019c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.761+0000 7f63080e9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6300112be0 con 0x7f6300107450 2024-04-25T10:41:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.761+0000 7f63080e9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6300112a60 con 0x7f6300106990 2024-04-25T10:41:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.761+0000 7f63080e9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6300112d60 con 0x7f6300102400 2024-04-25T10:41:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.761+0000 7f6306686640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6300107450 0x7f630019c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.761+0000 7f6306686640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6300107450 0x7f630019c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34670/0 (socket says 172.21.15.43:34670) 2024-04-25T10:41:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.761+0000 7f6306686640 1 -- 172.21.15.43:0/1006878815 learned_addr learned my addr 172.21.15.43:0/1006878815 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.761+0000 7f6305684640 1 --2- 172.21.15.43:0/1006878815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6300106990 0x7f630019baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.761+0000 7f6305e85640 1 --2- 172.21.15.43:0/1006878815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6300102400 0x7f630019b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.761+0000 7f6306686640 1 -- 172.21.15.43:0/1006878815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6300102400 msgr2=0x7f630019b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.761+0000 7f6306686640 1 --2- 172.21.15.43:0/1006878815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6300102400 0x7f630019b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.761+0000 7f6306686640 1 -- 172.21.15.43:0/1006878815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6300106990 msgr2=0x7f630019baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:50.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.761+0000 7f6306686640 1 --2- 172.21.15.43:0/1006878815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6300106990 0x7f630019baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.761+0000 7f6306686640 1 -- 172.21.15.43:0/1006878815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63001a68a0 con 0x7f6300107450 2024-04-25T10:41:50.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.761+0000 7f6306686640 1 --2- 172.21.15.43:0/1006878815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6300107450 0x7f630019c000 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f62f8005cc0 tx=0x7f62f802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:50.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.761+0000 7f62f6ffd640 1 -- 172.21.15.43:0/1006878815 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8003bf0 con 0x7f6300107450 2024-04-25T10:41:50.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.761+0000 7f63080e9640 1 -- 172.21.15.43:0/1006878815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63001a6b30 con 0x7f6300107450 2024-04-25T10:41:50.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.761+0000 7f62f6ffd640 1 -- 172.21.15.43:0/1006878815 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62f8003d90 con 0x7f6300107450 2024-04-25T10:41:50.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.762+0000 7f62f6ffd640 1 -- 172.21.15.43:0/1006878815 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f802fbc0 con 0x7f6300107450 2024-04-25T10:41:50.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.762+0000 7f63080e9640 1 -- 172.21.15.43:0/1006878815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63001a70d0 con 0x7f6300107450 2024-04-25T10:41:50.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.763+0000 7f63080e9640 1 -- 172.21.15.43:0/1006878815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f630006a190 con 0x7f6300107450 2024-04-25T10:41:50.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.764+0000 7f62f6ffd640 1 -- 172.21.15.43:0/1006878815 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f62f802fd60 con 0x7f6300107450 2024-04-25T10:41:50.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.765+0000 7f62f6ffd640 1 --2- 172.21.15.43:0/1006878815 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f62ec0afc60 0x7f62ec0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:50.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.765+0000 7f62f6ffd640 1 -- 172.21.15.43:0/1006878815 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f62f81007f0 con 0x7f6300107450 2024-04-25T10:41:50.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.765+0000 7f6305e85640 1 --2- 172.21.15.43:0/1006878815 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f62ec0afc60 0x7f62ec0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:50.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.765+0000 7f6305e85640 1 --2- 172.21.15.43:0/1006878815 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f62ec0afc60 0x7f62ec0b2170 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f62fc005fd0 tx=0x7f62fc00b390 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:50.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.769+0000 7f62f6ffd640 1 -- 172.21.15.43:0/1006878815 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62f8080ff0 con 0x7f6300107450 2024-04-25T10:41:50.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.992+0000 7f63080e9640 1 -- 172.21.15.43:0/1006878815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f630010fea0 con 0x7f6300107450 2024-04-25T10:41:50.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.993+0000 7f62f6ffd640 1 -- 172.21.15.43:0/1006878815 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7f62f80c91a0 con 0x7f6300107450 2024-04-25T10:41:50.995 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:41:50.995 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:41:50.995 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:41:50.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.995+0000 7f63080e9640 1 -- 172.21.15.43:0/1006878815 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f62ec0afc60 msgr2=0x7f62ec0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:50.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.995+0000 7f63080e9640 1 --2- 172.21.15.43:0/1006878815 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f62ec0afc60 0x7f62ec0b2170 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f62fc005fd0 tx=0x7f62fc00b390 comp rx=0 tx=0).stop 2024-04-25T10:41:50.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.995+0000 7f63080e9640 1 -- 172.21.15.43:0/1006878815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6300107450 msgr2=0x7f630019c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:50.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.995+0000 7f63080e9640 1 --2- 172.21.15.43:0/1006878815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6300107450 0x7f630019c000 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f62f8005cc0 tx=0x7f62f802f380 comp rx=0 tx=0).stop 2024-04-25T10:41:50.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.995+0000 7f63080e9640 1 -- 172.21.15.43:0/1006878815 shutdown_connections 2024-04-25T10:41:50.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.995+0000 7f63080e9640 1 --2- 172.21.15.43:0/1006878815 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f62ec0afc60 0x7f62ec0b2170 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.995+0000 7f63080e9640 1 --2- 172.21.15.43:0/1006878815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6300106990 0x7f630019baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.995+0000 7f63080e9640 1 --2- 172.21.15.43:0/1006878815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6300107450 0x7f630019c000 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.995+0000 7f63080e9640 1 --2- 172.21.15.43:0/1006878815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6300102400 0x7f630019b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:50.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.995+0000 7f63080e9640 1 -- 172.21.15.43:0/1006878815 >> 172.21.15.43:0/1006878815 conn(0x7f6300077890 msgr2=0x7f63000ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:50.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.995+0000 7f63080e9640 1 -- 172.21.15.43:0/1006878815 shutdown_connections 2024-04-25T10:41:50.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:50.995+0000 7f63080e9640 1 -- 172.21.15.43:0/1006878815 wait complete. 2024-04-25T10:41:51.008 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:41:51.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.121+0000 7f5316fc7640 1 -- 172.21.15.43:0/2403279096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5310102af0 msgr2=0x7f5310102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.121+0000 7f5316fc7640 1 --2- 172.21.15.43:0/2403279096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5310102af0 0x7f5310102f00 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f5304009900 tx=0x7f530402a410 comp rx=0 tx=0).stop 2024-04-25T10:41:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.121+0000 7f5316fc7640 1 -- 172.21.15.43:0/2403279096 shutdown_connections 2024-04-25T10:41:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.121+0000 7f5316fc7640 1 --2- 172.21.15.43:0/2403279096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f531010d380 0x7f531010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.121+0000 7f5316fc7640 1 --2- 172.21.15.43:0/2403279096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5310102af0 0x7f5310102f00 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.121+0000 7f5316fc7640 1 --2- 172.21.15.43:0/2403279096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5310108a20 0x7f53101025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.121+0000 7f5316fc7640 1 -- 172.21.15.43:0/2403279096 >> 172.21.15.43:0/2403279096 conn(0x7f5310078100 msgr2=0x7f5310078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.121+0000 7f5316fc7640 1 -- 172.21.15.43:0/2403279096 shutdown_connections 2024-04-25T10:41:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.121+0000 7f5316fc7640 1 -- 172.21.15.43:0/2403279096 wait complete. 2024-04-25T10:41:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.122+0000 7f5316fc7640 1 Processor -- start 2024-04-25T10:41:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.122+0000 7f5316fc7640 1 -- start start 2024-04-25T10:41:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.122+0000 7f5316fc7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5310102af0 0x7f531019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.122+0000 7f5316fc7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5310108a20 0x7f531019ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.122+0000 7f5316fc7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f531010d380 0x7f531019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.122+0000 7f5316fc7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5310112c30 con 0x7f531010d380 2024-04-25T10:41:51.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.122+0000 7f5316fc7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5310112ab0 con 0x7f5310102af0 2024-04-25T10:41:51.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.122+0000 7f5316fc7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5310112db0 con 0x7f5310108a20 2024-04-25T10:41:51.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.122+0000 7f5315564640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f531010d380 0x7f531019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.122+0000 7f5315564640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f531010d380 0x7f531019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34694/0 (socket says 172.21.15.43:34694) 2024-04-25T10:41:51.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.122+0000 7f5315564640 1 -- 172.21.15.43:0/743050062 learned_addr learned my addr 172.21.15.43:0/743050062 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:51.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.122+0000 7f5314d63640 1 --2- 172.21.15.43:0/743050062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5310102af0 0x7f531019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.122+0000 7f530ffff640 1 --2- 172.21.15.43:0/743050062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5310108a20 0x7f531019ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.122+0000 7f5315564640 1 -- 172.21.15.43:0/743050062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5310108a20 msgr2=0x7f531019ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.123+0000 7f5315564640 1 --2- 172.21.15.43:0/743050062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5310108a20 0x7f531019ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.123+0000 7f5315564640 1 -- 172.21.15.43:0/743050062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5310102af0 msgr2=0x7f531019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.123+0000 7f5315564640 1 --2- 172.21.15.43:0/743050062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5310102af0 0x7f531019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.123+0000 7f5315564640 1 -- 172.21.15.43:0/743050062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f531019f5e0 con 0x7f531010d380 2024-04-25T10:41:51.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.123+0000 7f530ffff640 1 --2- 172.21.15.43:0/743050062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5310108a20 0x7f531019ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:41:51.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.123+0000 7f5315564640 1 --2- 172.21.15.43:0/743050062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f531010d380 0x7f531019efa0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f530800cad0 tx=0x7f530800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:51.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.123+0000 7f530dffb640 1 -- 172.21.15.43:0/743050062 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530800ae20 con 0x7f531010d380 2024-04-25T10:41:51.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.123+0000 7f530dffb640 1 -- 172.21.15.43:0/743050062 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5308005020 con 0x7f531010d380 2024-04-25T10:41:51.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.123+0000 7f5316fc7640 1 -- 172.21.15.43:0/743050062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f531019f8d0 con 0x7f531010d380 2024-04-25T10:41:51.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.123+0000 7f530dffb640 1 -- 172.21.15.43:0/743050062 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530800f6b0 con 0x7f531010d380 2024-04-25T10:41:51.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.123+0000 7f5316fc7640 1 -- 172.21.15.43:0/743050062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f531019fdd0 con 0x7f531010d380 2024-04-25T10:41:51.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.125+0000 7f530dffb640 1 -- 172.21.15.43:0/743050062 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f53080051d0 con 0x7f531010d380 2024-04-25T10:41:51.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.126+0000 7f531de75740 1 Processor -- start 2024-04-25T10:41:51.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.126+0000 7f531de75740 1 -- start start 2024-04-25T10:41:51.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.126+0000 7f531de75740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624acb390 0x561624e6c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.126+0000 7f531de75740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561624aa24d0 0x561624e6cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.126+0000 7f531de75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624aaa6b0 0x561624e796a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.126+0000 7f531de75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561624dd1d80 con 0x561624aaa6b0 2024-04-25T10:41:51.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.126+0000 7f531de75740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561624d8ba20 con 0x561624acb390 2024-04-25T10:41:51.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.126+0000 7f531de75740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561624dcdd40 con 0x561624aa24d0 2024-04-25T10:41:51.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.126+0000 7f530ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624aaa6b0 0x561624e796a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.126+0000 7f530ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624acb390 0x561624e6c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.126+0000 7f530ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624acb390 0x561624e6c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:54208/0 (socket says 172.21.15.43:54208) 2024-04-25T10:41:51.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.126+0000 7f530ffff640 1 -- 172.21.15.43:0/3651647343 learned_addr learned my addr 172.21.15.43:0/3651647343 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:51.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.126+0000 7f5314d63640 1 --2- 172.21.15.43:0/3651647343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561624aa24d0 0x561624e6cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.126+0000 7f530dffb640 1 --2- 172.21.15.43:0/743050062 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f52f80afc60 0x7f52f80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.126+0000 7f530ffff640 1 -- 172.21.15.43:0/3651647343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561624aa24d0 msgr2=0x561624e6cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.126+0000 7f5315564640 1 --2- 172.21.15.43:0/743050062 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f52f80afc60 0x7f52f80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.126+0000 7f530ffff640 1 --2- 172.21.15.43:0/3651647343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561624aa24d0 0x561624e6cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.127+0000 7f530ffff640 1 -- 172.21.15.43:0/3651647343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624aaa6b0 msgr2=0x561624e796a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.127+0000 7f530ffff640 1 --2- 172.21.15.43:0/3651647343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624aaa6b0 0x561624e796a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.127+0000 7f530ffff640 1 -- 172.21.15.43:0/3651647343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561624d96c30 con 0x561624acb390 2024-04-25T10:41:51.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.127+0000 7f530dffb640 1 -- 172.21.15.43:0/743050062 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f53080df090 con 0x7f531010d380 2024-04-25T10:41:51.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.127+0000 7f530ffff640 1 --2- 172.21.15.43:0/3651647343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624acb390 0x561624e6c810 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5304004650 tx=0x7f5304005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46061ae95d8a50c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:51.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.127+0000 7f5315564640 1 --2- 172.21.15.43:0/743050062 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f52f80afc60 0x7f52f80b2170 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f53080dc890 tx=0x7f5308075d30 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:51.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.127+0000 7f52f6ffd640 1 -- 172.21.15.43:0/3651647343 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5304045070 con 0x561624acb390 2024-04-25T10:41:51.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.127+0000 7f52f6ffd640 1 -- 172.21.15.43:0/3651647343 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5304003de0 con 0x561624acb390 2024-04-25T10:41:51.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.127+0000 7f52f6ffd640 1 -- 172.21.15.43:0/3651647343 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5304035040 con 0x561624acb390 2024-04-25T10:41:51.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.128+0000 7f531de75740 1 -- 172.21.15.43:0/3651647343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624acb390 msgr2=0x561624e6c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.128+0000 7f531de75740 1 --2- 172.21.15.43:0/3651647343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624acb390 0x561624e6c810 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5304004650 tx=0x7f5304005fd0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.128+0000 7f531de75740 1 -- 172.21.15.43:0/3651647343 shutdown_connections 2024-04-25T10:41:51.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.128+0000 7f531de75740 1 --2- 172.21.15.43:0/3651647343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624aaa6b0 0x561624e796a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.128+0000 7f531de75740 1 --2- 172.21.15.43:0/3651647343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561624aa24d0 0x561624e6cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.128+0000 7f531de75740 1 --2- 172.21.15.43:0/3651647343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624acb390 0x561624e6c810 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.128+0000 7f531de75740 1 -- 172.21.15.43:0/3651647343 >> 172.21.15.43:0/3651647343 conn(0x561624abf4e0 msgr2=0x561624e68fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:51.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.128+0000 7f531de75740 1 -- 172.21.15.43:0/3651647343 shutdown_connections 2024-04-25T10:41:51.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.128+0000 7f531de75740 1 -- 172.21.15.43:0/3651647343 wait complete. 2024-04-25T10:41:51.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.128+0000 7f531de75740 1 -- start start 2024-04-25T10:41:51.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.129+0000 7f531de75740 1 Processor -- start 2024-04-25T10:41:51.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.129+0000 7f531de75740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561624aa24d0 0x561624e772d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.129+0000 7f531de75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624aaa6b0 0x561624e77810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.129+0000 7f531de75740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624acb390 0x561624e875c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.129+0000 7f531de75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561624e79d30 con 0x561624aaa6b0 2024-04-25T10:41:51.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.129+0000 7f531de75740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561624d58dc0 con 0x561624acb390 2024-04-25T10:41:51.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.129+0000 7f531de75740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561624e79eb0 con 0x561624aa24d0 2024-04-25T10:41:51.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.129+0000 7f531de75740 10 client.0 _ll_register_callbacks cb 0x561624d74600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:51.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.129+0000 7f5314d63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624aaa6b0 0x561624e77810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.129+0000 7f5314d63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624aaa6b0 0x561624e77810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34716/0 (socket says 172.21.15.43:34716) 2024-04-25T10:41:51.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.129+0000 7f530ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561624aa24d0 0x561624e772d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.129+0000 7f5314d63640 1 -- 172.21.15.43:0/2354530573 learned_addr learned my addr 172.21.15.43:0/2354530573 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:51.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.129+0000 7f530ffff640 1 --2- 172.21.15.43:0/2354530573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624acb390 0x561624e875c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.129+0000 7f5314d63640 1 -- 172.21.15.43:0/2354530573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561624aa24d0 msgr2=0x561624e772d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.129+0000 7f5314d63640 1 --2- 172.21.15.43:0/2354530573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561624aa24d0 0x561624e772d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.129+0000 7f5314d63640 1 -- 172.21.15.43:0/2354530573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624acb390 msgr2=0x561624e875c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.130+0000 7f5314d63640 1 --2- 172.21.15.43:0/2354530573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624acb390 0x561624e875c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.130+0000 7f5314d63640 1 -- 172.21.15.43:0/2354530573 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561624ae33f0 con 0x561624aaa6b0 2024-04-25T10:41:51.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.130+0000 7f530ffff640 1 --2- 172.21.15.43:0/2354530573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624acb390 0x561624e875c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.130+0000 7f5314d63640 1 --2- 172.21.15.43:0/2354530573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624aaa6b0 0x561624e77810 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f5300010820 tx=0x7f5300010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99898e9f45c141a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.130+0000 7f52effff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.130+0000 7f52effff640 1 -- 172.21.15.43:0/2354530573 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530000fb10 con 0x561624aaa6b0 2024-04-25T10:41:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.130+0000 7f52effff640 1 -- 172.21.15.43:0/2354530573 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f530000fcb0 con 0x561624aaa6b0 2024-04-25T10:41:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.130+0000 7f531de75740 1 -- 172.21.15.43:0/2354530573 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5304003de0 con 0x561624aaa6b0 2024-04-25T10:41:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.130+0000 7f52effff640 1 -- 172.21.15.43:0/2354530573 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5300004020 con 0x561624aaa6b0 2024-04-25T10:41:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.130+0000 7f531de75740 10 client.18240 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.130+0000 7f531de75740 1 -- 172.21.15.43:0/2354530573 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561624bad7d0 con 0x561624aaa6b0 2024-04-25T10:41:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.130+0000 7f52effff640 1 -- 172.21.15.43:0/2354530573 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5304003de0 con 0x561624aaa6b0 2024-04-25T10:41:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.130+0000 7f52effff640 1 -- 172.21.15.43:0/2354530573 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f53000048c0 con 0x561624aaa6b0 2024-04-25T10:41:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.131+0000 7f531de75740 10 client.18240 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.131+0000 7f531de75740 10 client.18240 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:41:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.131+0000 7f531de75740 20 client.18240 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.131+0000 7f531de75740 1 --2- 172.21.15.43:0/2354530573 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x561624a89f60 0x561624e8db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.131+0000 7f531de75740 4 client.18240 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:41:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.131+0000 7f531de75740 1 -- 172.21.15.43:0/2354530573 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561624ddf990 con 0x561624a89f60 2024-04-25T10:41:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.131+0000 7f530ffff640 1 --2- 172.21.15.43:0/2354530573 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x561624a89f60 0x561624e8db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.131+0000 7f530ffff640 1 --2- 172.21.15.43:0/2354530573 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x561624a89f60 0x561624e8db80 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c883b6fd5cdda12 server_cookie=d5a6db1d245e9906 in_seq=0 out_seq=0 2024-04-25T10:41:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.131+0000 7f52effff640 10 client.18240 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:41:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.132+0000 7f52effff640 1 -- 172.21.15.43:0/2354530573 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561624ddf990 con 0x561624a89f60 2024-04-25T10:41:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.132+0000 7f52effff640 10 client.18240 handle_command_reply: tid=0 2024-04-25T10:41:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.132+0000 7f531de75740 1 client.18240 shutdown 2024-04-25T10:41:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.132+0000 7f531de75740 1 -- 172.21.15.43:0/2354530573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624aaa6b0 msgr2=0x561624e77810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.132+0000 7f531de75740 1 --2- 172.21.15.43:0/2354530573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624aaa6b0 0x561624e77810 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f5300010820 tx=0x7f5300010ce0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.132+0000 7f531de75740 1 -- 172.21.15.43:0/2354530573 shutdown_connections 2024-04-25T10:41:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.132+0000 7f531de75740 1 --2- 172.21.15.43:0/2354530573 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x561624a89f60 0x561624e8db80 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.132+0000 7f531de75740 1 --2- 172.21.15.43:0/2354530573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624acb390 0x561624e875c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.132+0000 7f531de75740 1 --2- 172.21.15.43:0/2354530573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624aaa6b0 0x561624e77810 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.132+0000 7f52effff640 0 client.18240 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:41:51.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.132+0000 7f531de75740 1 --2- 172.21.15.43:0/2354530573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561624aa24d0 0x561624e772d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.132+0000 7f531de75740 1 -- 172.21.15.43:0/2354530573 >> 172.21.15.43:0/2354530573 conn(0x561624abf4e0 msgr2=0x561624e6e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:51.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.132+0000 7f531de75740 1 -- 172.21.15.43:0/2354530573 shutdown_connections 2024-04-25T10:41:51.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.132+0000 7f531de75740 1 -- 172.21.15.43:0/2354530573 wait complete. 2024-04-25T10:41:51.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.132+0000 7f531de75740 20 client.18240 trim_cache size 0 max 16384 2024-04-25T10:41:51.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.146+0000 7f531de75740 1 Processor -- start 2024-04-25T10:41:51.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.146+0000 7f531de75740 1 -- start start 2024-04-25T10:41:51.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.146+0000 7f531de75740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561624aa24d0 0x561624e69090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.146+0000 7f531de75740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624aaa6b0 0x561624e87220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.146+0000 7f531de75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624acb390 0x561624e714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.146+0000 7f531de75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f53101afef0 con 0x561624acb390 2024-04-25T10:41:51.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.146+0000 7f531de75740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f53101ad7c0 con 0x561624aaa6b0 2024-04-25T10:41:51.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.146+0000 7f531de75740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f53101ab650 con 0x561624aa24d0 2024-04-25T10:41:51.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.146+0000 7f530ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624acb390 0x561624e714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.146+0000 7f530ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624acb390 0x561624e714f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34730/0 (socket says 172.21.15.43:34730) 2024-04-25T10:41:51.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.146+0000 7f530ffff640 1 -- 172.21.15.43:0/4215306897 learned_addr learned my addr 172.21.15.43:0/4215306897 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:51.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.147+0000 7f530ffff640 1 --2- 172.21.15.43:0/4215306897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561624aa24d0 0x561624e69090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.147+0000 7f5314d63640 1 --2- 172.21.15.43:0/4215306897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624aaa6b0 0x561624e87220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.147+0000 7f530ffff640 1 -- 172.21.15.43:0/4215306897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561624aa24d0 msgr2=0x561624e69090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.147+0000 7f530ffff640 1 --2- 172.21.15.43:0/4215306897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561624aa24d0 0x561624e69090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.147+0000 7f530ffff640 1 -- 172.21.15.43:0/4215306897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624aaa6b0 msgr2=0x561624e87220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.147+0000 7f530ffff640 1 --2- 172.21.15.43:0/4215306897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624aaa6b0 0x561624e87220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.147+0000 7f530ffff640 1 -- 172.21.15.43:0/4215306897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561624e6dfc0 con 0x561624acb390 2024-04-25T10:41:51.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.147+0000 7f530ffff640 1 --2- 172.21.15.43:0/4215306897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624acb390 0x561624e714f0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f5304009340 tx=0x7f5304005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28920ce476feb84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:51.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.147+0000 7f52f67fc640 1 -- 172.21.15.43:0/4215306897 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5304035070 con 0x561624acb390 2024-04-25T10:41:51.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.147+0000 7f52f67fc640 1 -- 172.21.15.43:0/4215306897 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5304042040 con 0x561624acb390 2024-04-25T10:41:51.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.147+0000 7f52f67fc640 1 -- 172.21.15.43:0/4215306897 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5304045040 con 0x561624acb390 2024-04-25T10:41:51.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.148+0000 7f531de75740 1 -- 172.21.15.43:0/4215306897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624acb390 msgr2=0x561624e714f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.148+0000 7f531de75740 1 --2- 172.21.15.43:0/4215306897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624acb390 0x561624e714f0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f5304009340 tx=0x7f5304005fd0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.148+0000 7f531de75740 1 -- 172.21.15.43:0/4215306897 shutdown_connections 2024-04-25T10:41:51.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.148+0000 7f531de75740 1 --2- 172.21.15.43:0/4215306897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624aaa6b0 0x561624e87220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.148+0000 7f531de75740 1 --2- 172.21.15.43:0/4215306897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624acb390 0x561624e714f0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.148+0000 7f531de75740 1 --2- 172.21.15.43:0/4215306897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561624aa24d0 0x561624e69090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.148+0000 7f531de75740 1 -- 172.21.15.43:0/4215306897 >> 172.21.15.43:0/4215306897 conn(0x561624abf4e0 msgr2=0x561624e68980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:51.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.148+0000 7f531de75740 1 -- 172.21.15.43:0/4215306897 shutdown_connections 2024-04-25T10:41:51.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.148+0000 7f531de75740 1 -- 172.21.15.43:0/4215306897 wait complete. 2024-04-25T10:41:51.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.148+0000 7f531de75740 1 -- start start 2024-04-25T10:41:51.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.148+0000 7f531de75740 1 Processor -- start 2024-04-25T10:41:51.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.149+0000 7f531de75740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624aa24d0 0x561624e83980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.149+0000 7f531de75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624aaa6b0 0x561624e7ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.149+0000 7f531de75740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561624acb390 0x561624e7f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.149+0000 7f531de75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f53101abca0 con 0x561624aaa6b0 2024-04-25T10:41:51.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.149+0000 7f531de75740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f53101ad0f0 con 0x561624aa24d0 2024-04-25T10:41:51.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.149+0000 7f531de75740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f53101ac350 con 0x561624acb390 2024-04-25T10:41:51.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.149+0000 7f531de75740 10 client.0 _ll_register_callbacks cb 0x7f530000a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:51.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.149+0000 7f5314d63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624aaa6b0 0x561624e7ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.149+0000 7f5314d63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624aaa6b0 0x561624e7ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34744/0 (socket says 172.21.15.43:34744) 2024-04-25T10:41:51.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.149+0000 7f5314d63640 1 -- 172.21.15.43:0/1700409288 learned_addr learned my addr 172.21.15.43:0/1700409288 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:51.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.149+0000 7f5314d63640 1 -- 172.21.15.43:0/1700409288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561624acb390 msgr2=0x561624e7f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.149+0000 7f5314d63640 1 --2- 172.21.15.43:0/1700409288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561624acb390 0x561624e7f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.149+0000 7f530ffff640 1 --2- 172.21.15.43:0/1700409288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624aa24d0 0x561624e83980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.149+0000 7f5314d63640 1 -- 172.21.15.43:0/1700409288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624aa24d0 msgr2=0x561624e83980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.149+0000 7f5314d63640 1 --2- 172.21.15.43:0/1700409288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624aa24d0 0x561624e83980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.149+0000 7f5314d63640 1 -- 172.21.15.43:0/1700409288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561624e7f7a0 con 0x561624aaa6b0 2024-04-25T10:41:51.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.149+0000 7f5314d63640 1 --2- 172.21.15.43:0/1700409288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624aaa6b0 0x561624e7ecd0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x561624e6d190 tx=0x7f530000fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ce834b50e1fdbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:51.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.149+0000 7f52effff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:51.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.149+0000 7f52effff640 1 -- 172.21.15.43:0/1700409288 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300030060 con 0x561624aaa6b0 2024-04-25T10:41:51.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.150+0000 7f52effff640 1 -- 172.21.15.43:0/1700409288 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f5300004870 con 0x561624aaa6b0 2024-04-25T10:41:51.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.150+0000 7f531de75740 1 -- 172.21.15.43:0/1700409288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5304042040 con 0x561624aaa6b0 2024-04-25T10:41:51.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.150+0000 7f52effff640 1 -- 172.21.15.43:0/1700409288 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5300004a70 con 0x561624aaa6b0 2024-04-25T10:41:51.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.150+0000 7f531de75740 10 client.18246 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:51.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.150+0000 7f531de75740 1 -- 172.21.15.43:0/1700409288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561624ef5c30 con 0x561624aaa6b0 2024-04-25T10:41:51.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.150+0000 7f52effff640 1 -- 172.21.15.43:0/1700409288 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5304042040 con 0x561624aaa6b0 2024-04-25T10:41:51.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.150+0000 7f52effff640 1 -- 172.21.15.43:0/1700409288 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5300003680 con 0x561624aaa6b0 2024-04-25T10:41:51.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.150+0000 7f531de75740 10 client.18246 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:51.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.150+0000 7f531de75740 10 client.18246 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:41:51.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.150+0000 7f531de75740 20 client.18246 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:51.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.150+0000 7f531de75740 1 --2- 172.21.15.43:0/1700409288 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x561624a5f3a0 0x561624ef83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.150+0000 7f531de75740 4 client.18246 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:41:51.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.150+0000 7f531de75740 1 -- 172.21.15.43:0/1700409288 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561624dd9800 con 0x561624a5f3a0 2024-04-25T10:41:51.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.150+0000 7f530ffff640 1 --2- 172.21.15.43:0/1700409288 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x561624a5f3a0 0x561624ef83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.151+0000 7f52effff640 10 client.18246 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:41:51.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.151+0000 7f530ffff640 1 --2- 172.21.15.43:0/1700409288 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x561624a5f3a0 0x561624ef83e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65aaca2e48d5abd3 server_cookie=bc685136d7b96d59 in_seq=0 out_seq=0 2024-04-25T10:41:51.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.151+0000 7f52effff640 1 -- 172.21.15.43:0/1700409288 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561624dd9800 con 0x561624a5f3a0 2024-04-25T10:41:51.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.151+0000 7f52effff640 10 client.18246 handle_command_reply: tid=0 2024-04-25T10:41:51.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.151+0000 7f531de75740 1 client.18246 shutdown 2024-04-25T10:41:51.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.151+0000 7f531de75740 1 -- 172.21.15.43:0/1700409288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624aaa6b0 msgr2=0x561624e7ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.151+0000 7f531de75740 1 --2- 172.21.15.43:0/1700409288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624aaa6b0 0x561624e7ecd0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x561624e6d190 tx=0x7f530000fdc0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.151+0000 7f531de75740 1 -- 172.21.15.43:0/1700409288 shutdown_connections 2024-04-25T10:41:51.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.151+0000 7f531de75740 1 --2- 172.21.15.43:0/1700409288 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x561624a5f3a0 0x561624ef83e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.151+0000 7f531de75740 1 --2- 172.21.15.43:0/1700409288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561624acb390 0x561624e7f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.151+0000 7f531de75740 1 --2- 172.21.15.43:0/1700409288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561624aaa6b0 0x561624e7ecd0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.151+0000 7f52effff640 0 client.18246 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:41:51.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.151+0000 7f531de75740 1 --2- 172.21.15.43:0/1700409288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561624aa24d0 0x561624e83980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.151+0000 7f531de75740 1 -- 172.21.15.43:0/1700409288 >> 172.21.15.43:0/1700409288 conn(0x561624abf4e0 msgr2=0x561624e75990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:51.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.152+0000 7f531de75740 1 -- 172.21.15.43:0/1700409288 shutdown_connections 2024-04-25T10:41:51.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.152+0000 7f531de75740 1 -- 172.21.15.43:0/1700409288 wait complete. 2024-04-25T10:41:51.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.152+0000 7f531de75740 20 client.18246 trim_cache size 0 max 16384 2024-04-25T10:41:51.161 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:41:51.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.152+0000 7f5316fc7640 1 -- 172.21.15.43:0/743050062 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f52f80afc60 msgr2=0x7f52f80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.152+0000 7f5316fc7640 1 --2- 172.21.15.43:0/743050062 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f52f80afc60 0x7f52f80b2170 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f53080dc890 tx=0x7f5308075d30 comp rx=0 tx=0).stop 2024-04-25T10:41:51.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.152+0000 7f5316fc7640 1 -- 172.21.15.43:0/743050062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f531010d380 msgr2=0x7f531019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.152+0000 7f5316fc7640 1 --2- 172.21.15.43:0/743050062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f531010d380 0x7f531019efa0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f530800cad0 tx=0x7f530800cf90 comp rx=0 tx=0).stop 2024-04-25T10:41:51.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.152+0000 7f5316fc7640 1 -- 172.21.15.43:0/743050062 shutdown_connections 2024-04-25T10:41:51.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.152+0000 7f5316fc7640 1 --2- 172.21.15.43:0/743050062 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f52f80afc60 0x7f52f80b2170 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.152+0000 7f5316fc7640 1 --2- 172.21.15.43:0/743050062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f531010d380 0x7f531019efa0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.152+0000 7f5316fc7640 1 --2- 172.21.15.43:0/743050062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5310108a20 0x7f531019ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.152+0000 7f5316fc7640 1 --2- 172.21.15.43:0/743050062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5310102af0 0x7f531019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.152+0000 7f5316fc7640 1 -- 172.21.15.43:0/743050062 >> 172.21.15.43:0/743050062 conn(0x7f5310078100 msgr2=0x7f5310069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:51.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.152+0000 7f5316fc7640 1 -- 172.21.15.43:0/743050062 shutdown_connections 2024-04-25T10:41:51.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.152+0000 7f5316fc7640 1 -- 172.21.15.43:0/743050062 wait complete. 2024-04-25T10:41:51.165 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:41:51.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.277+0000 7fca640d8640 1 -- 172.21.15.43:0/256151990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca5c108a20 msgr2=0x7fca5c1025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.277+0000 7fca640d8640 1 --2- 172.21.15.43:0/256151990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca5c108a20 0x7fca5c1025b0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fca58009900 tx=0x7fca5802a410 comp rx=0 tx=0).stop 2024-04-25T10:41:51.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.277+0000 7fca640d8640 1 -- 172.21.15.43:0/256151990 shutdown_connections 2024-04-25T10:41:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.277+0000 7fca640d8640 1 --2- 172.21.15.43:0/256151990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca5c10d380 0x7fca5c10f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.277+0000 7fca640d8640 1 --2- 172.21.15.43:0/256151990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca5c102af0 0x7fca5c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.277+0000 7fca640d8640 1 --2- 172.21.15.43:0/256151990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca5c108a20 0x7fca5c1025b0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.277+0000 7fca640d8640 1 -- 172.21.15.43:0/256151990 >> 172.21.15.43:0/256151990 conn(0x7fca5c078100 msgr2=0x7fca5c078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.277+0000 7fca640d8640 1 -- 172.21.15.43:0/256151990 shutdown_connections 2024-04-25T10:41:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.277+0000 7fca640d8640 1 -- 172.21.15.43:0/256151990 wait complete. 2024-04-25T10:41:51.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.278+0000 7fca640d8640 1 Processor -- start 2024-04-25T10:41:51.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.278+0000 7fca640d8640 1 -- start start 2024-04-25T10:41:51.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.278+0000 7fca640d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca5c102af0 0x7fca5c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.278+0000 7fca640d8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca5c108a20 0x7fca5c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.278+0000 7fca640d8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca5c10d380 0x7fca5c19de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.278+0000 7fca640d8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c112be0 con 0x7fca5c102af0 2024-04-25T10:41:51.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.278+0000 7fca640d8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c112a60 con 0x7fca5c108a20 2024-04-25T10:41:51.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.278+0000 7fca640d8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c112d60 con 0x7fca5c10d380 2024-04-25T10:41:51.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.278+0000 7fca61e74640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca5c102af0 0x7fca5c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.278+0000 7fca61e74640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca5c102af0 0x7fca5c199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34758/0 (socket says 172.21.15.43:34758) 2024-04-25T10:41:51.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.278+0000 7fca61e74640 1 -- 172.21.15.43:0/765308238 learned_addr learned my addr 172.21.15.43:0/765308238 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:51.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.278+0000 7fca62675640 1 --2- 172.21.15.43:0/765308238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca5c10d380 0x7fca5c19de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.279+0000 7fca61673640 1 --2- 172.21.15.43:0/765308238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca5c108a20 0x7fca5c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.279+0000 7fca61e74640 1 -- 172.21.15.43:0/765308238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca5c10d380 msgr2=0x7fca5c19de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.279+0000 7fca61e74640 1 --2- 172.21.15.43:0/765308238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca5c10d380 0x7fca5c19de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.279+0000 7fca61e74640 1 -- 172.21.15.43:0/765308238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca5c108a20 msgr2=0x7fca5c199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.279+0000 7fca61e74640 1 --2- 172.21.15.43:0/765308238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca5c108a20 0x7fca5c199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.279+0000 7fca61e74640 1 -- 172.21.15.43:0/765308238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5c19e470 con 0x7fca5c102af0 2024-04-25T10:41:51.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.279+0000 7fca61e74640 1 --2- 172.21.15.43:0/765308238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca5c102af0 0x7fca5c199660 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fca58006010 tx=0x7fca58004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:51.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.279+0000 7fca52ffd640 1 -- 172.21.15.43:0/765308238 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5803f030 con 0x7fca5c102af0 2024-04-25T10:41:51.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.279+0000 7fca640d8640 1 -- 172.21.15.43:0/765308238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5c19e700 con 0x7fca5c102af0 2024-04-25T10:41:51.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.279+0000 7fca52ffd640 1 -- 172.21.15.43:0/765308238 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca58003aa0 con 0x7fca5c102af0 2024-04-25T10:41:51.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.279+0000 7fca52ffd640 1 -- 172.21.15.43:0/765308238 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58035070 con 0x7fca5c102af0 2024-04-25T10:41:51.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.279+0000 7fca640d8640 1 -- 172.21.15.43:0/765308238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca5c19ebf0 con 0x7fca5c102af0 2024-04-25T10:41:51.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.282+0000 7fca6af88740 1 Processor -- start 2024-04-25T10:41:51.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.282+0000 7fca6af88740 1 -- start start 2024-04-25T10:41:51.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.283+0000 7fca52ffd640 1 -- 172.21.15.43:0/765308238 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fca58042030 con 0x7fca5c102af0 2024-04-25T10:41:51.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.283+0000 7fca6af88740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e126390 0x56454e4c7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.283+0000 7fca6af88740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e0fd4d0 0x56454e4c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.283+0000 7fca6af88740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e1056b0 0x56454e4d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.283+0000 7fca6af88740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56454e42cd80 con 0x56454e1056b0 2024-04-25T10:41:51.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.283+0000 7fca6af88740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56454e3e6a20 con 0x56454e0fd4d0 2024-04-25T10:41:51.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.283+0000 7fca6af88740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56454e428d40 con 0x56454e126390 2024-04-25T10:41:51.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.283+0000 7fca61673640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e1056b0 0x56454e4d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.283+0000 7fca61673640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e1056b0 0x56454e4d46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34770/0 (socket says 172.21.15.43:34770) 2024-04-25T10:41:51.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.283+0000 7fca61673640 1 -- 172.21.15.43:0/3999737777 learned_addr learned my addr 172.21.15.43:0/3999737777 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:51.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.283+0000 7fca62675640 1 --2- 172.21.15.43:0/3999737777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e126390 0x56454e4c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.283+0000 7fca61e74640 1 --2- 172.21.15.43:0/3999737777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e0fd4d0 0x56454e4c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.283+0000 7fca61673640 1 -- 172.21.15.43:0/3999737777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e126390 msgr2=0x56454e4c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.283+0000 7fca61673640 1 --2- 172.21.15.43:0/3999737777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e126390 0x56454e4c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.283+0000 7fca61673640 1 -- 172.21.15.43:0/3999737777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e0fd4d0 msgr2=0x56454e4c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.283+0000 7fca61673640 1 --2- 172.21.15.43:0/3999737777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e0fd4d0 0x56454e4c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.283+0000 7fca61673640 1 -- 172.21.15.43:0/3999737777 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56454e3f1c30 con 0x56454e1056b0 2024-04-25T10:41:51.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.283+0000 7fca52ffd640 1 --2- 172.21.15.43:0/765308238 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fca480afc60 0x7fca480b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.283+0000 7fca61673640 1 --2- 172.21.15.43:0/3999737777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e1056b0 0x56454e4d46a0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009bb0 tx=0x7fca4c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c44d7fb266ce252 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:51.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.283+0000 7fca62675640 1 --2- 172.21.15.43:0/765308238 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fca480afc60 0x7fca480b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.284+0000 7fca52ffd640 1 -- 172.21.15.43:0/765308238 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fca580ff4b0 con 0x7fca5c102af0 2024-04-25T10:41:51.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.284+0000 7fca47fff640 1 -- 172.21.15.43:0/3999737777 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c012450 con 0x56454e1056b0 2024-04-25T10:41:51.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.284+0000 7fca47fff640 1 -- 172.21.15.43:0/3999737777 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca4c012aa0 con 0x56454e1056b0 2024-04-25T10:41:51.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.284+0000 7fca47fff640 1 -- 172.21.15.43:0/3999737777 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c00fa80 con 0x56454e1056b0 2024-04-25T10:41:51.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.285+0000 7fca6af88740 1 -- 172.21.15.43:0/3999737777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e1056b0 msgr2=0x56454e4d46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.285+0000 7fca62675640 1 --2- 172.21.15.43:0/765308238 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fca480afc60 0x7fca480b2170 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x56454e4c4980 tx=0x7fca54009600 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:51.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.285+0000 7fca6af88740 1 --2- 172.21.15.43:0/3999737777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e1056b0 0x56454e4d46a0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009bb0 tx=0x7fca4c0117b0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.285+0000 7fca6af88740 1 -- 172.21.15.43:0/3999737777 shutdown_connections 2024-04-25T10:41:51.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.285+0000 7fca6af88740 1 --2- 172.21.15.43:0/3999737777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e0fd4d0 0x56454e4c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.285+0000 7fca6af88740 1 --2- 172.21.15.43:0/3999737777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e1056b0 0x56454e4d46a0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.285+0000 7fca6af88740 1 --2- 172.21.15.43:0/3999737777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e126390 0x56454e4c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.285+0000 7fca6af88740 1 -- 172.21.15.43:0/3999737777 >> 172.21.15.43:0/3999737777 conn(0x56454e11a4e0 msgr2=0x56454e4c3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:51.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.285+0000 7fca6af88740 1 -- 172.21.15.43:0/3999737777 shutdown_connections 2024-04-25T10:41:51.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.285+0000 7fca6af88740 1 -- 172.21.15.43:0/3999737777 wait complete. 2024-04-25T10:41:51.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.285+0000 7fca6af88740 1 -- start start 2024-04-25T10:41:51.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.285+0000 7fca6af88740 1 Processor -- start 2024-04-25T10:41:51.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca6af88740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e0fd4d0 0x56454e4d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca6af88740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e1056b0 0x56454e4d2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca6af88740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e126390 0x56454e4de210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca6af88740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56454e4d4d90 con 0x56454e0fd4d0 2024-04-25T10:41:51.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca6af88740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56454e3b3dc0 con 0x56454e126390 2024-04-25T10:41:51.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca6af88740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56454e4d4f10 con 0x56454e1056b0 2024-04-25T10:41:51.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca6af88740 10 client.0 _ll_register_callbacks cb 0x56454e3cf600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:51.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca61e74640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e0fd4d0 0x56454e4d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca61e74640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e0fd4d0 0x56454e4d21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34786/0 (socket says 172.21.15.43:34786) 2024-04-25T10:41:51.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca61e74640 1 -- 172.21.15.43:0/1279998630 learned_addr learned my addr 172.21.15.43:0/1279998630 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:51.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca61673640 1 --2- 172.21.15.43:0/1279998630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e1056b0 0x56454e4d2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca62675640 1 --2- 172.21.15.43:0/1279998630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e126390 0x56454e4de210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca61e74640 1 -- 172.21.15.43:0/1279998630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e1056b0 msgr2=0x56454e4d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca61e74640 1 --2- 172.21.15.43:0/1279998630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e1056b0 0x56454e4d2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca61e74640 1 -- 172.21.15.43:0/1279998630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e126390 msgr2=0x56454e4de210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca61e74640 1 --2- 172.21.15.43:0/1279998630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e126390 0x56454e4de210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca61e74640 1 -- 172.21.15.43:0/1279998630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56454e13e3f0 con 0x56454e0fd4d0 2024-04-25T10:41:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca61673640 1 --2- 172.21.15.43:0/1279998630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e1056b0 0x56454e4d2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca62675640 1 --2- 172.21.15.43:0/1279998630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e126390 0x56454e4de210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca61e74640 1 --2- 172.21.15.43:0/1279998630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e0fd4d0 0x56454e4d21c0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fca58031440 tx=0x7fca58003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7db9b1005d6cc68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca457fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.286+0000 7fca457fa640 1 -- 172.21.15.43:0/1279998630 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58049070 con 0x56454e0fd4d0 2024-04-25T10:41:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.287+0000 7fca457fa640 1 -- 172.21.15.43:0/1279998630 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fca580fce80 con 0x56454e0fd4d0 2024-04-25T10:41:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.287+0000 7fca6af88740 1 -- 172.21.15.43:0/1279998630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca4c012aa0 con 0x56454e0fd4d0 2024-04-25T10:41:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.287+0000 7fca457fa640 1 -- 172.21.15.43:0/1279998630 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca580c7420 con 0x56454e0fd4d0 2024-04-25T10:41:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.287+0000 7fca6af88740 10 client.18258 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.287+0000 7fca6af88740 1 -- 172.21.15.43:0/1279998630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56454e2087d0 con 0x56454e0fd4d0 2024-04-25T10:41:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.287+0000 7fca457fa640 1 -- 172.21.15.43:0/1279998630 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca4c012aa0 con 0x56454e0fd4d0 2024-04-25T10:41:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.287+0000 7fca457fa640 1 -- 172.21.15.43:0/1279998630 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fca580c38e0 con 0x56454e0fd4d0 2024-04-25T10:41:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.287+0000 7fca6af88740 10 client.18258 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.287+0000 7fca6af88740 10 client.18258 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:41:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.287+0000 7fca6af88740 20 client.18258 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.287+0000 7fca6af88740 1 --2- 172.21.15.43:0/1279998630 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x56454e0e4f60 0x56454e4e6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.287+0000 7fca6af88740 4 client.18258 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:41:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.287+0000 7fca6af88740 1 -- 172.21.15.43:0/1279998630 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56454e43a990 con 0x56454e0e4f60 2024-04-25T10:41:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.287+0000 7fca61673640 1 --2- 172.21.15.43:0/1279998630 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x56454e0e4f60 0x56454e4e6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.288+0000 7fca61673640 1 --2- 172.21.15.43:0/1279998630 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x56454e0e4f60 0x56454e4e6730 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da8258cedd230976 server_cookie=141af49d1af762a8 in_seq=0 out_seq=0 2024-04-25T10:41:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.288+0000 7fca457fa640 10 client.18258 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:41:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.289+0000 7fca457fa640 1 -- 172.21.15.43:0/1279998630 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56454e43a990 con 0x56454e0e4f60 2024-04-25T10:41:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.289+0000 7fca457fa640 10 client.18258 handle_command_reply: tid=0 2024-04-25T10:41:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.289+0000 7fca6af88740 1 client.18258 shutdown 2024-04-25T10:41:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.289+0000 7fca6af88740 1 -- 172.21.15.43:0/1279998630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e0fd4d0 msgr2=0x56454e4d21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.289+0000 7fca6af88740 1 --2- 172.21.15.43:0/1279998630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e0fd4d0 0x56454e4d21c0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fca58031440 tx=0x7fca58003f50 comp rx=0 tx=0).stop 2024-04-25T10:41:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.289+0000 7fca6af88740 1 -- 172.21.15.43:0/1279998630 shutdown_connections 2024-04-25T10:41:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.289+0000 7fca6af88740 1 --2- 172.21.15.43:0/1279998630 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x56454e0e4f60 0x56454e4e6730 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.289+0000 7fca6af88740 1 --2- 172.21.15.43:0/1279998630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e126390 0x56454e4de210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.289+0000 7fca6af88740 1 --2- 172.21.15.43:0/1279998630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e1056b0 0x56454e4d2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.289+0000 7fca6af88740 1 --2- 172.21.15.43:0/1279998630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e0fd4d0 0x56454e4d21c0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.289+0000 7fca457fa640 0 client.18258 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:41:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.289+0000 7fca6af88740 1 -- 172.21.15.43:0/1279998630 >> 172.21.15.43:0/1279998630 conn(0x56454e11a4e0 msgr2=0x56454e4c9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.289+0000 7fca6af88740 1 -- 172.21.15.43:0/1279998630 shutdown_connections 2024-04-25T10:41:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.289+0000 7fca6af88740 1 -- 172.21.15.43:0/1279998630 wait complete. 2024-04-25T10:41:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.289+0000 7fca6af88740 20 client.18258 trim_cache size 0 max 16384 2024-04-25T10:41:51.298 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:51 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1006878815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:41:51.298 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:51 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1006878815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:41:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.303+0000 7fca6af88740 1 Processor -- start 2024-04-25T10:41:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.303+0000 7fca6af88740 1 -- start start 2024-04-25T10:41:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.303+0000 7fca6af88740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e0fd4d0 0x56454e4d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.303+0000 7fca6af88740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e1056b0 0x56454e4dde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.303+0000 7fca6af88740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e126390 0x56454e4cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.303+0000 7fca6af88740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c1affe0 con 0x56454e126390 2024-04-25T10:41:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.303+0000 7fca6af88740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c1ad8b0 con 0x56454e1056b0 2024-04-25T10:41:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.303+0000 7fca6af88740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c1ab720 con 0x56454e0fd4d0 2024-04-25T10:41:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.303+0000 7fca62675640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e126390 0x56454e4cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.303+0000 7fca62675640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e126390 0x56454e4cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34798/0 (socket says 172.21.15.43:34798) 2024-04-25T10:41:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.303+0000 7fca62675640 1 -- 172.21.15.43:0/1448043724 learned_addr learned my addr 172.21.15.43:0/1448043724 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.303+0000 7fca61673640 1 --2- 172.21.15.43:0/1448043724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e1056b0 0x56454e4dde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.303+0000 7fca61e74640 1 --2- 172.21.15.43:0/1448043724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e0fd4d0 0x56454e4d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.303+0000 7fca62675640 1 -- 172.21.15.43:0/1448043724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e0fd4d0 msgr2=0x56454e4d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.303+0000 7fca62675640 1 --2- 172.21.15.43:0/1448043724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e0fd4d0 0x56454e4d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.303+0000 7fca62675640 1 -- 172.21.15.43:0/1448043724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e1056b0 msgr2=0x56454e4dde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.303+0000 7fca62675640 1 --2- 172.21.15.43:0/1448043724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e1056b0 0x56454e4dde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.303+0000 7fca62675640 1 -- 172.21.15.43:0/1448043724 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56454e4c3760 con 0x56454e126390 2024-04-25T10:41:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.303+0000 7fca61e74640 1 --2- 172.21.15.43:0/1448043724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e0fd4d0 0x56454e4d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:41:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.303+0000 7fca61673640 1 --2- 172.21.15.43:0/1448043724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e1056b0 0x56454e4dde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:41:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.304+0000 7fca62675640 1 --2- 172.21.15.43:0/1448043724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e126390 0x56454e4cc4f0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fca54003bf0 tx=0x7fca54016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7100d8981abd82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.304+0000 7fca477fe640 1 -- 172.21.15.43:0/1448043724 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca540045b0 con 0x56454e126390 2024-04-25T10:41:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.304+0000 7fca477fe640 1 -- 172.21.15.43:0/1448043724 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca54026030 con 0x56454e126390 2024-04-25T10:41:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.304+0000 7fca477fe640 1 -- 172.21.15.43:0/1448043724 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca54022810 con 0x56454e126390 2024-04-25T10:41:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.305+0000 7fca6af88740 1 -- 172.21.15.43:0/1448043724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e126390 msgr2=0x56454e4cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.305+0000 7fca6af88740 1 --2- 172.21.15.43:0/1448043724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e126390 0x56454e4cc4f0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fca54003bf0 tx=0x7fca54016000 comp rx=0 tx=0).stop 2024-04-25T10:41:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.305+0000 7fca6af88740 1 -- 172.21.15.43:0/1448043724 shutdown_connections 2024-04-25T10:41:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.305+0000 7fca6af88740 1 --2- 172.21.15.43:0/1448043724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e1056b0 0x56454e4dde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.305+0000 7fca6af88740 1 --2- 172.21.15.43:0/1448043724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e126390 0x56454e4cc4f0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.305+0000 7fca6af88740 1 --2- 172.21.15.43:0/1448043724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e0fd4d0 0x56454e4d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.305+0000 7fca6af88740 1 -- 172.21.15.43:0/1448043724 >> 172.21.15.43:0/1448043724 conn(0x56454e11a4e0 msgr2=0x56454e4c2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.305+0000 7fca6af88740 1 -- 172.21.15.43:0/1448043724 shutdown_connections 2024-04-25T10:41:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.305+0000 7fca6af88740 1 -- 172.21.15.43:0/1448043724 wait complete. 2024-04-25T10:41:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.305+0000 7fca6af88740 1 -- start start 2024-04-25T10:41:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.305+0000 7fca6af88740 1 Processor -- start 2024-04-25T10:41:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.305+0000 7fca6af88740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e0fd4d0 0x56454e4da420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.305+0000 7fca6af88740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e1056b0 0x56454e4d5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.305+0000 7fca6af88740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e126390 0x56454e4d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.305+0000 7fca6af88740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c1abd70 con 0x56454e1056b0 2024-04-25T10:41:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.305+0000 7fca6af88740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c1ad1e0 con 0x56454e0fd4d0 2024-04-25T10:41:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.305+0000 7fca6af88740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c1ac440 con 0x56454e126390 2024-04-25T10:41:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.306+0000 7fca6af88740 10 client.0 _ll_register_callbacks cb 0x7fca58005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.306+0000 7fca61673640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e1056b0 0x56454e4d5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.306+0000 7fca61673640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e1056b0 0x56454e4d5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34814/0 (socket says 172.21.15.43:34814) 2024-04-25T10:41:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.306+0000 7fca61673640 1 -- 172.21.15.43:0/722152610 learned_addr learned my addr 172.21.15.43:0/722152610 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.306+0000 7fca61e74640 1 --2- 172.21.15.43:0/722152610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e0fd4d0 0x56454e4da420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.306+0000 7fca62675640 1 --2- 172.21.15.43:0/722152610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e126390 0x56454e4d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.306+0000 7fca61673640 1 -- 172.21.15.43:0/722152610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e126390 msgr2=0x56454e4d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.306+0000 7fca61673640 1 --2- 172.21.15.43:0/722152610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e126390 0x56454e4d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.306+0000 7fca61673640 1 -- 172.21.15.43:0/722152610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e0fd4d0 msgr2=0x56454e4da420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.306+0000 7fca61673640 1 --2- 172.21.15.43:0/722152610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e0fd4d0 0x56454e4da420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.306+0000 7fca61673640 1 -- 172.21.15.43:0/722152610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56454e4d61f0 con 0x56454e1056b0 2024-04-25T10:41:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.306+0000 7fca61673640 1 --2- 172.21.15.43:0/722152610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e1056b0 0x56454e4d5770 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00f5d0 tx=0x7fca4c0095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4a249f2a2fb56b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.306+0000 7fca457fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.306+0000 7fca457fa640 1 -- 172.21.15.43:0/722152610 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c012760 con 0x56454e1056b0 2024-04-25T10:41:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.306+0000 7fca457fa640 1 -- 172.21.15.43:0/722152610 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fca4c012900 con 0x56454e1056b0 2024-04-25T10:41:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.306+0000 7fca6af88740 1 -- 172.21.15.43:0/722152610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca54026030 con 0x56454e1056b0 2024-04-25T10:41:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.306+0000 7fca457fa640 1 -- 172.21.15.43:0/722152610 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca4c005040 con 0x56454e1056b0 2024-04-25T10:41:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.306+0000 7fca6af88740 10 client.18264 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.306+0000 7fca6af88740 1 -- 172.21.15.43:0/722152610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56454e54e520 con 0x56454e1056b0 2024-04-25T10:41:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.307+0000 7fca457fa640 1 -- 172.21.15.43:0/722152610 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca54026030 con 0x56454e1056b0 2024-04-25T10:41:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.307+0000 7fca457fa640 1 -- 172.21.15.43:0/722152610 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fca4c00f8f0 con 0x56454e1056b0 2024-04-25T10:41:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.307+0000 7fca6af88740 10 client.18264 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.307+0000 7fca6af88740 10 client.18264 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:41:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.307+0000 7fca6af88740 20 client.18264 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.307+0000 7fca6af88740 1 --2- 172.21.15.43:0/722152610 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x56454e0ba3a0 0x56454e550c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.307+0000 7fca6af88740 4 client.18264 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:41:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.307+0000 7fca6af88740 1 -- 172.21.15.43:0/722152610 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56454e434800 con 0x56454e0ba3a0 2024-04-25T10:41:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.307+0000 7fca61e74640 1 --2- 172.21.15.43:0/722152610 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x56454e0ba3a0 0x56454e550c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.308+0000 7fca457fa640 10 client.18264 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:41:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.308+0000 7fca61e74640 1 --2- 172.21.15.43:0/722152610 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x56454e0ba3a0 0x56454e550c20 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c23b6bbc8d61965 server_cookie=a2fabca2eb0462d6 in_seq=0 out_seq=0 2024-04-25T10:41:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.308+0000 7fca457fa640 1 -- 172.21.15.43:0/722152610 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56454e434800 con 0x56454e0ba3a0 2024-04-25T10:41:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.308+0000 7fca457fa640 10 client.18264 handle_command_reply: tid=0 2024-04-25T10:41:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.308+0000 7fca6af88740 1 client.18264 shutdown 2024-04-25T10:41:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.308+0000 7fca6af88740 1 -- 172.21.15.43:0/722152610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e1056b0 msgr2=0x56454e4d5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.308+0000 7fca6af88740 1 --2- 172.21.15.43:0/722152610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e1056b0 0x56454e4d5770 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00f5d0 tx=0x7fca4c0095a0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.308+0000 7fca6af88740 1 -- 172.21.15.43:0/722152610 shutdown_connections 2024-04-25T10:41:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.308+0000 7fca6af88740 1 --2- 172.21.15.43:0/722152610 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x56454e0ba3a0 0x56454e550c20 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.308+0000 7fca6af88740 1 --2- 172.21.15.43:0/722152610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56454e126390 0x56454e4d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.308+0000 7fca6af88740 1 --2- 172.21.15.43:0/722152610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56454e1056b0 0x56454e4d5770 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.308+0000 7fca457fa640 0 client.18264 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:41:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.308+0000 7fca6af88740 1 --2- 172.21.15.43:0/722152610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56454e0fd4d0 0x56454e4da420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.308+0000 7fca6af88740 1 -- 172.21.15.43:0/722152610 >> 172.21.15.43:0/722152610 conn(0x56454e11a4e0 msgr2=0x56454e4c3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.308+0000 7fca6af88740 1 -- 172.21.15.43:0/722152610 shutdown_connections 2024-04-25T10:41:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.308+0000 7fca6af88740 1 -- 172.21.15.43:0/722152610 wait complete. 2024-04-25T10:41:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.308+0000 7fca6af88740 20 client.18264 trim_cache size 0 max 16384 2024-04-25T10:41:51.317 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:41:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.309+0000 7fca640d8640 1 -- 172.21.15.43:0/765308238 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fca480afc60 msgr2=0x7fca480b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.309+0000 7fca640d8640 1 --2- 172.21.15.43:0/765308238 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fca480afc60 0x7fca480b2170 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x56454e4c4980 tx=0x7fca54009600 comp rx=0 tx=0).stop 2024-04-25T10:41:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.309+0000 7fca640d8640 1 -- 172.21.15.43:0/765308238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca5c102af0 msgr2=0x7fca5c199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.309+0000 7fca640d8640 1 --2- 172.21.15.43:0/765308238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca5c102af0 0x7fca5c199660 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fca58006010 tx=0x7fca58004060 comp rx=0 tx=0).stop 2024-04-25T10:41:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.309+0000 7fca640d8640 1 -- 172.21.15.43:0/765308238 shutdown_connections 2024-04-25T10:41:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.309+0000 7fca640d8640 1 --2- 172.21.15.43:0/765308238 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fca480afc60 0x7fca480b2170 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.309+0000 7fca640d8640 1 --2- 172.21.15.43:0/765308238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca5c108a20 0x7fca5c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.309+0000 7fca640d8640 1 --2- 172.21.15.43:0/765308238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca5c10d380 0x7fca5c19de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.309+0000 7fca640d8640 1 --2- 172.21.15.43:0/765308238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca5c102af0 0x7fca5c199660 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.309+0000 7fca640d8640 1 -- 172.21.15.43:0/765308238 >> 172.21.15.43:0/765308238 conn(0x7fca5c078100 msgr2=0x7fca5c1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.309+0000 7fca640d8640 1 -- 172.21.15.43:0/765308238 shutdown_connections 2024-04-25T10:41:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.309+0000 7fca640d8640 1 -- 172.21.15.43:0/765308238 wait complete. 2024-04-25T10:41:51.320 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:51 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1006878815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:41:51.322 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:41:51.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.433+0000 7f64ccad9640 1 -- 172.21.15.43:0/2258615510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c8108a30 msgr2=0x7f64c81025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.433+0000 7f64ccad9640 1 --2- 172.21.15.43:0/2258615510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c8108a30 0x7f64c81025e0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009900 tx=0x7f64b802a410 comp rx=0 tx=0).stop 2024-04-25T10:41:51.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.433+0000 7f64ccad9640 1 -- 172.21.15.43:0/2258615510 shutdown_connections 2024-04-25T10:41:51.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.433+0000 7f64ccad9640 1 --2- 172.21.15.43:0/2258615510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64c8103190 0x7f64c810f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.433+0000 7f64ccad9640 1 --2- 172.21.15.43:0/2258615510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64c8108ed0 0x7f64c8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.433+0000 7f64ccad9640 1 --2- 172.21.15.43:0/2258615510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c8108a30 0x7f64c81025e0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.433+0000 7f64ccad9640 1 -- 172.21.15.43:0/2258615510 >> 172.21.15.43:0/2258615510 conn(0x7f64c80fe260 msgr2=0x7f64c8100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:51.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.433+0000 7f64ccad9640 1 -- 172.21.15.43:0/2258615510 shutdown_connections 2024-04-25T10:41:51.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.434+0000 7f64ccad9640 1 -- 172.21.15.43:0/2258615510 wait complete. 2024-04-25T10:41:51.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.434+0000 7f64ccad9640 1 Processor -- start 2024-04-25T10:41:51.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.434+0000 7f64ccad9640 1 -- start start 2024-04-25T10:41:51.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.435+0000 7f64ccad9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64c8103190 0x7f64c819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.435+0000 7f64ccad9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64c8108a30 0x7f64c819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.435+0000 7f64ccad9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c8108ed0 0x7f64c819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.435+0000 7f64ccad9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f64c8112c30 con 0x7f64c8108ed0 2024-04-25T10:41:51.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.435+0000 7f64ccad9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f64c8112ab0 con 0x7f64c8103190 2024-04-25T10:41:51.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.435+0000 7f64ccad9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f64c8112db0 con 0x7f64c8108a30 2024-04-25T10:41:51.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.435+0000 7f64c6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c8108ed0 0x7f64c819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.435+0000 7f64c6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c8108ed0 0x7f64c819ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34834/0 (socket says 172.21.15.43:34834) 2024-04-25T10:41:51.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.435+0000 7f64c6d9d640 1 -- 172.21.15.43:0/3371065683 learned_addr learned my addr 172.21.15.43:0/3371065683 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:51.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.435+0000 7f64c659c640 1 --2- 172.21.15.43:0/3371065683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64c8103190 0x7f64c819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.435+0000 7f64c5d9b640 1 --2- 172.21.15.43:0/3371065683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64c8108a30 0x7f64c819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.435+0000 7f64c6d9d640 1 -- 172.21.15.43:0/3371065683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64c8108a30 msgr2=0x7f64c819ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.435+0000 7f64c6d9d640 1 --2- 172.21.15.43:0/3371065683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64c8108a30 0x7f64c819ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.435+0000 7f64c6d9d640 1 -- 172.21.15.43:0/3371065683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64c8103190 msgr2=0x7f64c819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.435+0000 7f64c6d9d640 1 --2- 172.21.15.43:0/3371065683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64c8103190 0x7f64c819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.435+0000 7f64c6d9d640 1 -- 172.21.15.43:0/3371065683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c819f560 con 0x7f64c8108ed0 2024-04-25T10:41:51.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.435+0000 7f64c5d9b640 1 --2- 172.21.15.43:0/3371065683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64c8108a30 0x7f64c819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:51.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.435+0000 7f64c6d9d640 1 --2- 172.21.15.43:0/3371065683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c8108ed0 0x7f64c819ef20 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f64c000eb10 tx=0x7f64c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:51.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.435+0000 7f64b77fe640 1 -- 172.21.15.43:0/3371065683 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c0004b20 con 0x7f64c8108ed0 2024-04-25T10:41:51.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.436+0000 7f64ccad9640 1 -- 172.21.15.43:0/3371065683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c819f850 con 0x7f64c8108ed0 2024-04-25T10:41:51.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.436+0000 7f64b77fe640 1 -- 172.21.15.43:0/3371065683 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c0004cc0 con 0x7f64c8108ed0 2024-04-25T10:41:51.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.436+0000 7f64b77fe640 1 -- 172.21.15.43:0/3371065683 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c0010670 con 0x7f64c8108ed0 2024-04-25T10:41:51.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.436+0000 7f64ccad9640 1 -- 172.21.15.43:0/3371065683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c819fd50 con 0x7f64c8108ed0 2024-04-25T10:41:51.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.438+0000 7f64d3982740 1 Processor -- start 2024-04-25T10:41:51.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.438+0000 7f64b77fe640 1 -- 172.21.15.43:0/3371065683 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f64c0005020 con 0x7f64c8108ed0 2024-04-25T10:41:51.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.439+0000 7f64d3982740 1 -- start start 2024-04-25T10:41:51.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.439+0000 7f64d3982740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aeba4f8390 0x55aeba8996c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.439+0000 7f64d3982740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4cf4d0 0x55aeba899c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.439+0000 7f64d3982740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 0x55aeba8aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.439+0000 7f64d3982740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aeba7fed80 con 0x55aeba4d76b0 2024-04-25T10:41:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.439+0000 7f64d3982740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55aeba7b8a20 con 0x55aeba4cf4d0 2024-04-25T10:41:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.439+0000 7f64d3982740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55aeba7fad40 con 0x55aeba4f8390 2024-04-25T10:41:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.439+0000 7f64c5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 0x55aeba8aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.439+0000 7f64c5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 0x55aeba8aaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34838/0 (socket says 172.21.15.43:34838) 2024-04-25T10:41:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.439+0000 7f64c5d9b640 1 -- 172.21.15.43:0/379728628 learned_addr learned my addr 172.21.15.43:0/379728628 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.439+0000 7f64b77fe640 1 --2- 172.21.15.43:0/3371065683 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f64b00afc60 0x7f64b00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.439+0000 7f64c5d9b640 1 --2- 172.21.15.43:0/379728628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aeba4f8390 0x55aeba8996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.439+0000 7f64c659c640 1 --2- 172.21.15.43:0/379728628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4cf4d0 0x55aeba899c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.439+0000 7f64c5d9b640 1 -- 172.21.15.43:0/379728628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aeba4f8390 msgr2=0x55aeba8996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.439+0000 7f64c5d9b640 1 --2- 172.21.15.43:0/379728628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aeba4f8390 0x55aeba8996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.439+0000 7f64c5d9b640 1 -- 172.21.15.43:0/379728628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4cf4d0 msgr2=0x55aeba899c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.439+0000 7f64c5d9b640 1 --2- 172.21.15.43:0/379728628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4cf4d0 0x55aeba899c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.439+0000 7f64c6d9d640 1 --2- 172.21.15.43:0/3371065683 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f64b00afc60 0x7f64b00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.439+0000 7f64c5d9b640 1 -- 172.21.15.43:0/379728628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aeba7c3c30 con 0x55aeba4d76b0 2024-04-25T10:41:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.440+0000 7f64b77fe640 1 -- 172.21.15.43:0/3371065683 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f64c00df700 con 0x7f64c8108ed0 2024-04-25T10:41:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.440+0000 7f64c5d9b640 1 --2- 172.21.15.43:0/379728628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 0x55aeba8aaab0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f64bc00bfd0 tx=0x7f64bc00b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c96663b9a033c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.440+0000 7f64b4ff9640 1 -- 172.21.15.43:0/379728628 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc019070 con 0x55aeba4d76b0 2024-04-25T10:41:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.440+0000 7f64b4ff9640 1 -- 172.21.15.43:0/379728628 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64bc0047a0 con 0x55aeba4d76b0 2024-04-25T10:41:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.440+0000 7f64b4ff9640 1 -- 172.21.15.43:0/379728628 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc011780 con 0x55aeba4d76b0 2024-04-25T10:41:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.441+0000 7f64d3982740 1 -- 172.21.15.43:0/379728628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 msgr2=0x55aeba8aaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.441+0000 7f64c6d9d640 1 --2- 172.21.15.43:0/3371065683 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f64b00afc60 0x7f64b00b2170 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f64c00a4ab0 tx=0x7f64c00b0040 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.441+0000 7f64d3982740 1 --2- 172.21.15.43:0/379728628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 0x55aeba8aaab0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f64bc00bfd0 tx=0x7f64bc00b690 comp rx=0 tx=0).stop 2024-04-25T10:41:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.441+0000 7f64d3982740 1 -- 172.21.15.43:0/379728628 shutdown_connections 2024-04-25T10:41:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.441+0000 7f64d3982740 1 --2- 172.21.15.43:0/379728628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4cf4d0 0x55aeba899c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.441+0000 7f64d3982740 1 --2- 172.21.15.43:0/379728628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 0x55aeba8aaab0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.441+0000 7f64d3982740 1 --2- 172.21.15.43:0/379728628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aeba4f8390 0x55aeba8996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.441+0000 7f64d3982740 1 -- 172.21.15.43:0/379728628 >> 172.21.15.43:0/379728628 conn(0x55aeba4ec4e0 msgr2=0x55aeba895fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:51.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.441+0000 7f64d3982740 1 -- 172.21.15.43:0/379728628 shutdown_connections 2024-04-25T10:41:51.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.441+0000 7f64d3982740 1 -- 172.21.15.43:0/379728628 wait complete. 2024-04-25T10:41:51.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.441+0000 7f64d3982740 1 -- start start 2024-04-25T10:41:51.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.441+0000 7f64d3982740 1 Processor -- start 2024-04-25T10:41:51.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.442+0000 7f64d3982740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4cf4d0 0x55aeba8a86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.442+0000 7f64d3982740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aeba4d76b0 0x55aeba8a8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.442+0000 7f64d3982740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4f8390 0x55aeba8b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.442+0000 7f64d3982740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aeba8ab1a0 con 0x55aeba4f8390 2024-04-25T10:41:51.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.442+0000 7f64d3982740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55aeba785dc0 con 0x55aeba4cf4d0 2024-04-25T10:41:51.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.442+0000 7f64d3982740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55aeba8ab320 con 0x55aeba4d76b0 2024-04-25T10:41:51.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.442+0000 7f64d3982740 10 client.0 _ll_register_callbacks cb 0x55aeba7a1600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.442+0000 7f64c5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4f8390 0x55aeba8b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.442+0000 7f64c5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4f8390 0x55aeba8b46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34844/0 (socket says 172.21.15.43:34844) 2024-04-25T10:41:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.442+0000 7f64c5d9b640 1 -- 172.21.15.43:0/3855607761 learned_addr learned my addr 172.21.15.43:0/3855607761 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.442+0000 7f64c5d9b640 1 --2- 172.21.15.43:0/3855607761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4cf4d0 0x55aeba8a86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.442+0000 7f64c659c640 1 --2- 172.21.15.43:0/3855607761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aeba4d76b0 0x55aeba8a8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.442+0000 7f64c5d9b640 1 -- 172.21.15.43:0/3855607761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aeba4d76b0 msgr2=0x55aeba8a8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.442+0000 7f64c5d9b640 1 --2- 172.21.15.43:0/3855607761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aeba4d76b0 0x55aeba8a8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.442+0000 7f64c5d9b640 1 -- 172.21.15.43:0/3855607761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4cf4d0 msgr2=0x55aeba8a86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.442+0000 7f64c5d9b640 1 --2- 172.21.15.43:0/3855607761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4cf4d0 0x55aeba8a86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.442+0000 7f64c5d9b640 1 -- 172.21.15.43:0/3855607761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aeba5103f0 con 0x55aeba4f8390 2024-04-25T10:41:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.442+0000 7f64c5d9b640 1 --2- 172.21.15.43:0/3855607761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4f8390 0x55aeba8b46e0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x55aeba8a7d20 tx=0x7f64bc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458c84988279ab85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.442+0000 7f64adffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.443+0000 7f64adffb640 1 -- 172.21.15.43:0/3855607761 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc019070 con 0x55aeba4f8390 2024-04-25T10:41:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.443+0000 7f64adffb640 1 -- 172.21.15.43:0/3855607761 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f64bc004fe0 con 0x55aeba4f8390 2024-04-25T10:41:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.443+0000 7f64d3982740 1 -- 172.21.15.43:0/3855607761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64bc0047a0 con 0x55aeba4f8390 2024-04-25T10:41:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.443+0000 7f64adffb640 1 -- 172.21.15.43:0/3855607761 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f64bc0117f0 con 0x55aeba4f8390 2024-04-25T10:41:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.443+0000 7f64d3982740 10 client.18276 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.443+0000 7f64d3982740 1 -- 172.21.15.43:0/3855607761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aeba5da7d0 con 0x55aeba4f8390 2024-04-25T10:41:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.443+0000 7f64adffb640 1 -- 172.21.15.43:0/3855607761 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64bc0047a0 con 0x55aeba4f8390 2024-04-25T10:41:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.443+0000 7f64adffb640 1 -- 172.21.15.43:0/3855607761 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f64bc01d930 con 0x55aeba4f8390 2024-04-25T10:41:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.443+0000 7f64d3982740 10 client.18276 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.443+0000 7f64d3982740 10 client.18276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.443+0000 7f64d3982740 20 client.18276 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.443+0000 7f64d3982740 1 --2- 172.21.15.43:0/3855607761 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aeba4b6f60 0x55aeba8baa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.443+0000 7f64d3982740 4 client.18276 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:41:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.443+0000 7f64d3982740 1 -- 172.21.15.43:0/3855607761 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aeba80c990 con 0x55aeba4b6f60 2024-04-25T10:41:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.444+0000 7f64c659c640 1 --2- 172.21.15.43:0/3855607761 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aeba4b6f60 0x55aeba8baa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.444+0000 7f64c659c640 1 --2- 172.21.15.43:0/3855607761 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aeba4b6f60 0x55aeba8baa00 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba7e821e252a04ce server_cookie=5de013bbcce04f3c in_seq=0 out_seq=0 2024-04-25T10:41:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.444+0000 7f64adffb640 10 client.18276 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.445+0000 7f64adffb640 1 -- 172.21.15.43:0/3855607761 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aeba80c990 con 0x55aeba4b6f60 2024-04-25T10:41:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.445+0000 7f64adffb640 10 client.18276 handle_command_reply: tid=0 2024-04-25T10:41:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.445+0000 7f64d3982740 1 client.18276 shutdown 2024-04-25T10:41:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.445+0000 7f64d3982740 1 -- 172.21.15.43:0/3855607761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4f8390 msgr2=0x55aeba8b46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.445+0000 7f64d3982740 1 --2- 172.21.15.43:0/3855607761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4f8390 0x55aeba8b46e0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x55aeba8a7d20 tx=0x7f64bc005850 comp rx=0 tx=0).stop 2024-04-25T10:41:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.445+0000 7f64d3982740 1 -- 172.21.15.43:0/3855607761 shutdown_connections 2024-04-25T10:41:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.445+0000 7f64d3982740 1 --2- 172.21.15.43:0/3855607761 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aeba4b6f60 0x55aeba8baa00 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.445+0000 7f64d3982740 1 --2- 172.21.15.43:0/3855607761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4f8390 0x55aeba8b46e0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.445+0000 7f64d3982740 1 --2- 172.21.15.43:0/3855607761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aeba4d76b0 0x55aeba8a8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.445+0000 7f64adffb640 0 client.18276 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.445+0000 7f64d3982740 1 --2- 172.21.15.43:0/3855607761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4cf4d0 0x55aeba8a86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.445+0000 7f64d3982740 1 -- 172.21.15.43:0/3855607761 >> 172.21.15.43:0/3855607761 conn(0x55aeba4ec4e0 msgr2=0x55aeba89b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.445+0000 7f64d3982740 1 -- 172.21.15.43:0/3855607761 shutdown_connections 2024-04-25T10:41:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.445+0000 7f64d3982740 1 -- 172.21.15.43:0/3855607761 wait complete. 2024-04-25T10:41:51.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.445+0000 7f64d3982740 20 client.18276 trim_cache size 0 max 16384 2024-04-25T10:41:51.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.459+0000 7f64d3982740 1 Processor -- start 2024-04-25T10:41:51.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.459+0000 7f64d3982740 1 -- start start 2024-04-25T10:41:51.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.459+0000 7f64d3982740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aeba4cf4d0 0x55aeba8b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.459+0000 7f64d3982740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 0x55aeba89bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.459+0000 7f64d3982740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4f8390 0x55aeba8bda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.459+0000 7f64d3982740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f64c81b00d0 con 0x55aeba4d76b0 2024-04-25T10:41:51.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.459+0000 7f64d3982740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f64c81ad9a0 con 0x55aeba4f8390 2024-04-25T10:41:51.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.459+0000 7f64d3982740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f64c81ab810 con 0x55aeba4cf4d0 2024-04-25T10:41:51.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.459+0000 7f64c659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 0x55aeba89bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.459+0000 7f64c659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 0x55aeba89bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34850/0 (socket says 172.21.15.43:34850) 2024-04-25T10:41:51.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.459+0000 7f64c659c640 1 -- 172.21.15.43:0/2092601989 learned_addr learned my addr 172.21.15.43:0/2092601989 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:51.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.459+0000 7f64c5d9b640 1 --2- 172.21.15.43:0/2092601989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4f8390 0x55aeba8bda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.459+0000 7f64c659c640 1 -- 172.21.15.43:0/2092601989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aeba4cf4d0 msgr2=0x55aeba8b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.459+0000 7f64c5d9b640 1 --2- 172.21.15.43:0/2092601989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aeba4cf4d0 0x55aeba8b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.459+0000 7f64c659c640 1 --2- 172.21.15.43:0/2092601989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aeba4cf4d0 0x55aeba8b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.459+0000 7f64c659c640 1 -- 172.21.15.43:0/2092601989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4f8390 msgr2=0x55aeba8bda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.459+0000 7f64c659c640 1 --2- 172.21.15.43:0/2092601989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4f8390 0x55aeba8bda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.459+0000 7f64c659c640 1 -- 172.21.15.43:0/2092601989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aeba8b5300 con 0x55aeba4d76b0 2024-04-25T10:41:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.459+0000 7f64c5d9b640 1 --2- 172.21.15.43:0/2092601989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4f8390 0x55aeba8bda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:41:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.459+0000 7f64c659c640 1 --2- 172.21.15.43:0/2092601989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 0x55aeba89bbb0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f64c8103720 tx=0x7f64b8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694bb519ad901a1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.460+0000 7f64affff640 1 -- 172.21.15.43:0/2092601989 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b804a070 con 0x55aeba4d76b0 2024-04-25T10:41:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.460+0000 7f64affff640 1 -- 172.21.15.43:0/2092601989 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64b8004430 con 0x55aeba4d76b0 2024-04-25T10:41:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.460+0000 7f64affff640 1 -- 172.21.15.43:0/2092601989 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b8035040 con 0x55aeba4d76b0 2024-04-25T10:41:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.460+0000 7f64d3982740 1 -- 172.21.15.43:0/2092601989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 msgr2=0x55aeba89bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.460+0000 7f64d3982740 1 --2- 172.21.15.43:0/2092601989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 0x55aeba89bbb0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f64c8103720 tx=0x7f64b8031400 comp rx=0 tx=0).stop 2024-04-25T10:41:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.460+0000 7f64d3982740 1 -- 172.21.15.43:0/2092601989 shutdown_connections 2024-04-25T10:41:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.460+0000 7f64d3982740 1 --2- 172.21.15.43:0/2092601989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4f8390 0x55aeba8bda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.460+0000 7f64d3982740 1 --2- 172.21.15.43:0/2092601989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 0x55aeba89bbb0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.460+0000 7f64d3982740 1 --2- 172.21.15.43:0/2092601989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aeba4cf4d0 0x55aeba8b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.460+0000 7f64d3982740 1 -- 172.21.15.43:0/2092601989 >> 172.21.15.43:0/2092601989 conn(0x55aeba4ec4e0 msgr2=0x55aeba89b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.460+0000 7f64d3982740 1 -- 172.21.15.43:0/2092601989 shutdown_connections 2024-04-25T10:41:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.460+0000 7f64d3982740 1 -- 172.21.15.43:0/2092601989 wait complete. 2024-04-25T10:41:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.461+0000 7f64d3982740 1 -- start start 2024-04-25T10:41:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.461+0000 7f64d3982740 1 Processor -- start 2024-04-25T10:41:51.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.461+0000 7f64d3982740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4cf4d0 0x55aeba8bd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.461+0000 7f64d3982740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 0x55aeba8bdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.461+0000 7f64d3982740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aeba4f8390 0x55aeba8bada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.461+0000 7f64d3982740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f64c81abe60 con 0x55aeba4d76b0 2024-04-25T10:41:51.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.461+0000 7f64d3982740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f64c81ad2d0 con 0x55aeba4cf4d0 2024-04-25T10:41:51.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.461+0000 7f64d3982740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f64c81ac530 con 0x55aeba4f8390 2024-04-25T10:41:51.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.461+0000 7f64d3982740 10 client.0 _ll_register_callbacks cb 0x7f64bc013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:51.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.461+0000 7f64c659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 0x55aeba8bdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.461+0000 7f64c659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 0x55aeba8bdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34852/0 (socket says 172.21.15.43:34852) 2024-04-25T10:41:51.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.461+0000 7f64c659c640 1 -- 172.21.15.43:0/3940793823 learned_addr learned my addr 172.21.15.43:0/3940793823 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:51.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.461+0000 7f64c5d9b640 1 --2- 172.21.15.43:0/3940793823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aeba4f8390 0x55aeba8bada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.462+0000 7f64c5d9b640 1 --2- 172.21.15.43:0/3940793823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4cf4d0 0x55aeba8bd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.462+0000 7f64c659c640 1 -- 172.21.15.43:0/3940793823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aeba4f8390 msgr2=0x55aeba8bada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.462+0000 7f64c659c640 1 --2- 172.21.15.43:0/3940793823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aeba4f8390 0x55aeba8bada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.462+0000 7f64c659c640 1 -- 172.21.15.43:0/3940793823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4cf4d0 msgr2=0x55aeba8bd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:51.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.462+0000 7f64c659c640 1 --2- 172.21.15.43:0/3940793823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4cf4d0 0x55aeba8bd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.462+0000 7f64c659c640 1 -- 172.21.15.43:0/3940793823 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aeba8bb3b0 con 0x55aeba4d76b0 2024-04-25T10:41:51.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.462+0000 7f64c659c640 1 --2- 172.21.15.43:0/3940793823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 0x55aeba8bdd90 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f64b802a930 tx=0x7f64b80043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5402f51ecc4b14bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:51.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.462+0000 7f64adffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:51.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.462+0000 7f64adffb640 1 -- 172.21.15.43:0/3940793823 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b804a040 con 0x55aeba4d76b0 2024-04-25T10:41:51.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.462+0000 7f64adffb640 1 -- 172.21.15.43:0/3940793823 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f64b80456c0 con 0x55aeba4d76b0 2024-04-25T10:41:51.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.462+0000 7f64d3982740 1 -- 172.21.15.43:0/3940793823 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64b8004430 con 0x55aeba4d76b0 2024-04-25T10:41:51.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.462+0000 7f64adffb640 1 -- 172.21.15.43:0/3940793823 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f64b8035040 con 0x55aeba4d76b0 2024-04-25T10:41:51.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.462+0000 7f64d3982740 10 client.18282 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:51.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.462+0000 7f64d3982740 1 -- 172.21.15.43:0/3940793823 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aeba9223b0 con 0x55aeba4d76b0 2024-04-25T10:41:51.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.462+0000 7f64adffb640 1 -- 172.21.15.43:0/3940793823 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64b8004430 con 0x55aeba4d76b0 2024-04-25T10:41:51.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.463+0000 7f64adffb640 1 -- 172.21.15.43:0/3940793823 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f64b804d070 con 0x55aeba4d76b0 2024-04-25T10:41:51.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.463+0000 7f64d3982740 10 client.18282 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:51.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.463+0000 7f64d3982740 10 client.18282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:51.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.463+0000 7f64d3982740 20 client.18282 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:51.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.463+0000 7f64d3982740 1 --2- 172.21.15.43:0/3940793823 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aeba48c3a0 0x55aeba924ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:51.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.463+0000 7f64d3982740 4 client.18282 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:41:51.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.463+0000 7f64d3982740 1 -- 172.21.15.43:0/3940793823 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aeba806800 con 0x55aeba48c3a0 2024-04-25T10:41:51.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.463+0000 7f64c5d9b640 1 --2- 172.21.15.43:0/3940793823 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aeba48c3a0 0x55aeba924ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:51.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.463+0000 7f64adffb640 10 client.18282 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:51.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.463+0000 7f64c5d9b640 1 --2- 172.21.15.43:0/3940793823 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aeba48c3a0 0x55aeba924ad0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c32e5369203809b server_cookie=43f527fb18544bdc in_seq=0 out_seq=0 2024-04-25T10:41:51.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64adffb640 1 -- 172.21.15.43:0/3940793823 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aeba806800 con 0x55aeba48c3a0 2024-04-25T10:41:51.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64adffb640 10 client.18282 handle_command_reply: tid=0 2024-04-25T10:41:51.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64d3982740 1 client.18282 shutdown 2024-04-25T10:41:51.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64d3982740 1 -- 172.21.15.43:0/3940793823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 msgr2=0x55aeba8bdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64d3982740 1 --2- 172.21.15.43:0/3940793823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 0x55aeba8bdd90 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f64b802a930 tx=0x7f64b80043a0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64d3982740 1 -- 172.21.15.43:0/3940793823 shutdown_connections 2024-04-25T10:41:51.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64d3982740 1 --2- 172.21.15.43:0/3940793823 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55aeba48c3a0 0x55aeba924ad0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64d3982740 1 --2- 172.21.15.43:0/3940793823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aeba4f8390 0x55aeba8bada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64d3982740 1 --2- 172.21.15.43:0/3940793823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeba4d76b0 0x55aeba8bdd90 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64d3982740 1 --2- 172.21.15.43:0/3940793823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeba4cf4d0 0x55aeba8bd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64adffb640 0 client.18282 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:51.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64d3982740 1 -- 172.21.15.43:0/3940793823 >> 172.21.15.43:0/3940793823 conn(0x55aeba4ec4e0 msgr2=0x55aeba895f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:51.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64d3982740 1 -- 172.21.15.43:0/3940793823 shutdown_connections 2024-04-25T10:41:51.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64d3982740 1 -- 172.21.15.43:0/3940793823 wait complete. 2024-04-25T10:41:51.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64d3982740 20 client.18282 trim_cache size 0 max 16384 2024-04-25T10:41:51.472 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:41:51.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64ccad9640 1 -- 172.21.15.43:0/3371065683 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f64b00afc60 msgr2=0x7f64b00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64ccad9640 1 --2- 172.21.15.43:0/3371065683 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f64b00afc60 0x7f64b00b2170 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f64c00a4ab0 tx=0x7f64c00b0040 comp rx=0 tx=0).stop 2024-04-25T10:41:51.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64ccad9640 1 -- 172.21.15.43:0/3371065683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c8108ed0 msgr2=0x7f64c819ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:51.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64ccad9640 1 --2- 172.21.15.43:0/3371065683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c8108ed0 0x7f64c819ef20 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f64c000eb10 tx=0x7f64c000c2d0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64ccad9640 1 -- 172.21.15.43:0/3371065683 shutdown_connections 2024-04-25T10:41:51.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64ccad9640 1 --2- 172.21.15.43:0/3371065683 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f64b00afc60 0x7f64b00b2170 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64ccad9640 1 --2- 172.21.15.43:0/3371065683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c8108ed0 0x7f64c819ef20 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64ccad9640 1 --2- 172.21.15.43:0/3371065683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64c8108a30 0x7f64c819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64ccad9640 1 --2- 172.21.15.43:0/3371065683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64c8103190 0x7f64c819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:51.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.464+0000 7f64ccad9640 1 -- 172.21.15.43:0/3371065683 >> 172.21.15.43:0/3371065683 conn(0x7f64c80fe260 msgr2=0x7f64c80fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:51.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.465+0000 7f64ccad9640 1 -- 172.21.15.43:0/3371065683 shutdown_connections 2024-04-25T10:41:51.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:51.465+0000 7f64ccad9640 1 -- 172.21.15.43:0/3371065683 wait complete. 2024-04-25T10:41:52.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:52 smithi077 ceph-mon[32772]: pgmap v289: 97 pgs: 97 active+clean; 5.0 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-04-25T10:41:52.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:52 smithi164 ceph-mon[32713]: pgmap v289: 97 pgs: 97 active+clean; 5.0 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-04-25T10:41:52.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:52 smithi043 ceph-mon[29478]: pgmap v289: 97 pgs: 97 active+clean; 5.0 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-04-25T10:41:52.478 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:41:52.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:41:52.479 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:41:52.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.608+0000 7f90f48e7640 1 -- 172.21.15.43:0/1893156281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90f0109fc0 msgr2=0x7f90f0103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:52.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.608+0000 7f90f48e7640 1 --2- 172.21.15.43:0/1893156281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90f0109fc0 0x7f90f0103c00 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009900 tx=0x7f90e402a410 comp rx=0 tx=0).stop 2024-04-25T10:41:52.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.608+0000 7f90f48e7640 1 -- 172.21.15.43:0/1893156281 shutdown_connections 2024-04-25T10:41:52.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.608+0000 7f90f48e7640 1 --2- 172.21.15.43:0/1893156281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90f0104270 0x7f90f0110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.608+0000 7f90f48e7640 1 --2- 172.21.15.43:0/1893156281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90f0109fc0 0x7f90f0103c00 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.608+0000 7f90f48e7640 1 --2- 172.21.15.43:0/1893156281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f0109b20 0x7f90f01036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.608+0000 7f90f48e7640 1 -- 172.21.15.43:0/1893156281 >> 172.21.15.43:0/1893156281 conn(0x7f90f00ff1e0 msgr2=0x7f90f0101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:52.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.608+0000 7f90f48e7640 1 -- 172.21.15.43:0/1893156281 shutdown_connections 2024-04-25T10:41:52.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.608+0000 7f90f48e7640 1 -- 172.21.15.43:0/1893156281 wait complete. 2024-04-25T10:41:52.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.609+0000 7f90f48e7640 1 Processor -- start 2024-04-25T10:41:52.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.609+0000 7f90f48e7640 1 -- start start 2024-04-25T10:41:52.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.609+0000 7f90f48e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90f0104270 0x7f90f019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.609+0000 7f90f48e7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90f0109b20 0x7f90f019acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.609+0000 7f90f48e7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f0109fc0 0x7f90f019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.609+0000 7f90f48e7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0113d10 con 0x7f90f0104270 2024-04-25T10:41:52.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.609+0000 7f90f48e7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0113b90 con 0x7f90f0109b20 2024-04-25T10:41:52.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.609+0000 7f90f48e7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0113e90 con 0x7f90f0109fc0 2024-04-25T10:41:52.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.609+0000 7f90ee59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90f0104270 0x7f90f019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.609+0000 7f90ee59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90f0104270 0x7f90f019a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34880/0 (socket says 172.21.15.43:34880) 2024-04-25T10:41:52.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.609+0000 7f90ee59c640 1 -- 172.21.15.43:0/3200167201 learned_addr learned my addr 172.21.15.43:0/3200167201 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:52.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.609+0000 7f90edd9b640 1 --2- 172.21.15.43:0/3200167201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90f0109b20 0x7f90f019acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.610+0000 7f90ee59c640 1 -- 172.21.15.43:0/3200167201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f0109fc0 msgr2=0x7f90f019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:52.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.610+0000 7f90eed9d640 1 --2- 172.21.15.43:0/3200167201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f0109fc0 0x7f90f019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.610+0000 7f90ee59c640 1 --2- 172.21.15.43:0/3200167201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f0109fc0 0x7f90f019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.610+0000 7f90ee59c640 1 -- 172.21.15.43:0/3200167201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90f0109b20 msgr2=0x7f90f019acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:52.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.610+0000 7f90ee59c640 1 --2- 172.21.15.43:0/3200167201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90f0109b20 0x7f90f019acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.610+0000 7f90ee59c640 1 -- 172.21.15.43:0/3200167201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f019f5f0 con 0x7f90f0104270 2024-04-25T10:41:52.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.610+0000 7f90edd9b640 1 --2- 172.21.15.43:0/3200167201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90f0109b20 0x7f90f019acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:41:52.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.610+0000 7f90eed9d640 1 --2- 172.21.15.43:0/3200167201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f0109fc0 0x7f90f019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:41:52.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.610+0000 7f90ee59c640 1 --2- 172.21.15.43:0/3200167201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90f0104270 0x7f90f019a790 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f90e000d770 tx=0x7f90e000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:52.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.610+0000 7f90df7fe640 1 -- 172.21.15.43:0/3200167201 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0017070 con 0x7f90f0104270 2024-04-25T10:41:52.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.610+0000 7f90f48e7640 1 -- 172.21.15.43:0/3200167201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f019f8e0 con 0x7f90f0104270 2024-04-25T10:41:52.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.610+0000 7f90df7fe640 1 -- 172.21.15.43:0/3200167201 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e0009c30 con 0x7f90f0104270 2024-04-25T10:41:52.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.610+0000 7f90df7fe640 1 -- 172.21.15.43:0/3200167201 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e000f620 con 0x7f90f0104270 2024-04-25T10:41:52.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.610+0000 7f90f48e7640 1 -- 172.21.15.43:0/3200167201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f01a7260 con 0x7f90f0104270 2024-04-25T10:41:52.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.612+0000 7f90fb799740 1 Processor -- start 2024-04-25T10:41:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.612+0000 7f90df7fe640 1 -- 172.21.15.43:0/3200167201 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f90e001a070 con 0x7f90f0104270 2024-04-25T10:41:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.613+0000 7f90fb799740 1 -- start start 2024-04-25T10:41:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.613+0000 7f90fb799740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc2e410 0x5652affce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.613+0000 7f90fb799740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc05550 0x5652affcea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.613+0000 7f90fb799740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc0d730 0x5652affcef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.613+0000 7f90fb799740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5652aff34b00 con 0x5652afc2e410 2024-04-25T10:41:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.613+0000 7f90fb799740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5652afed0920 con 0x5652afc05550 2024-04-25T10:41:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.613+0000 7f90fb799740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5652aff30ac0 con 0x5652afc0d730 2024-04-25T10:41:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.613+0000 7f90eed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc2e410 0x5652affce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.613+0000 7f90eed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc2e410 0x5652affce4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34894/0 (socket says 172.21.15.43:34894) 2024-04-25T10:41:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.613+0000 7f90eed9d640 1 -- 172.21.15.43:0/2081656481 learned_addr learned my addr 172.21.15.43:0/2081656481 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.613+0000 7f90edd9b640 1 --2- 172.21.15.43:0/2081656481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc0d730 0x5652affcef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.613+0000 7f90df7fe640 1 --2- 172.21.15.43:0/3200167201 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f90d80afc60 0x7f90d80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.613+0000 7f90ee59c640 1 --2- 172.21.15.43:0/2081656481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc05550 0x5652affcea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.614+0000 7f90eed9d640 1 -- 172.21.15.43:0/2081656481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc0d730 msgr2=0x5652affcef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:52.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.614+0000 7f90eed9d640 1 --2- 172.21.15.43:0/2081656481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc0d730 0x5652affcef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.614+0000 7f90eed9d640 1 -- 172.21.15.43:0/2081656481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc05550 msgr2=0x5652affcea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:52.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.614+0000 7f90eed9d640 1 --2- 172.21.15.43:0/2081656481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc05550 0x5652affcea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.614+0000 7f90eed9d640 1 -- 172.21.15.43:0/2081656481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652afd0e7e0 con 0x5652afc2e410 2024-04-25T10:41:52.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.614+0000 7f90ee59c640 1 --2- 172.21.15.43:0/2081656481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc05550 0x5652affcea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:52.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.614+0000 7f90eed9d640 1 --2- 172.21.15.43:0/3200167201 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f90d80afc60 0x7f90d80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.614+0000 7f90df7fe640 1 -- 172.21.15.43:0/3200167201 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f90e00debb0 con 0x7f90f0104270 2024-04-25T10:41:52.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.614+0000 7f90eed9d640 1 --2- 172.21.15.43:0/2081656481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc2e410 0x5652affce4d0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f90e8008f50 tx=0x7f90e800d7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13058059d0457d2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:52.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.614+0000 7f90dcff9640 1 -- 172.21.15.43:0/2081656481 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e801a8e0 con 0x5652afc2e410 2024-04-25T10:41:52.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.614+0000 7f90dcff9640 1 -- 172.21.15.43:0/2081656481 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e8019410 con 0x5652afc2e410 2024-04-25T10:41:52.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.614+0000 7f90dcff9640 1 -- 172.21.15.43:0/2081656481 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e800e610 con 0x5652afc2e410 2024-04-25T10:41:52.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.614+0000 7f90eed9d640 1 --2- 172.21.15.43:0/3200167201 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f90d80afc60 0x7f90d80b2170 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f90e801e000 tx=0x7f90e801f000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:52.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.615+0000 7f90fb799740 1 -- 172.21.15.43:0/2081656481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc2e410 msgr2=0x5652affce4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:52.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.615+0000 7f90fb799740 1 --2- 172.21.15.43:0/2081656481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc2e410 0x5652affce4d0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f90e8008f50 tx=0x7f90e800d7d0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.615+0000 7f90fb799740 1 -- 172.21.15.43:0/2081656481 shutdown_connections 2024-04-25T10:41:52.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.615+0000 7f90fb799740 1 --2- 172.21.15.43:0/2081656481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc05550 0x5652affcea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.615+0000 7f90fb799740 1 --2- 172.21.15.43:0/2081656481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc0d730 0x5652affcef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.615+0000 7f90fb799740 1 --2- 172.21.15.43:0/2081656481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc2e410 0x5652affce4d0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.615+0000 7f90fb799740 1 -- 172.21.15.43:0/2081656481 >> 172.21.15.43:0/2081656481 conn(0x5652afc22560 msgr2=0x5652affcadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:52.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.615+0000 7f90fb799740 1 -- 172.21.15.43:0/2081656481 shutdown_connections 2024-04-25T10:41:52.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.615+0000 7f90fb799740 1 -- 172.21.15.43:0/2081656481 wait complete. 2024-04-25T10:41:52.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.616+0000 7f90fb799740 1 -- start start 2024-04-25T10:41:52.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.616+0000 7f90fb799740 1 Processor -- start 2024-04-25T10:41:52.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.616+0000 7f90fb799740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc05550 0x5652affdd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.616+0000 7f90fb799740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc0d730 0x5652affdd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.616+0000 7f90fb799740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc2e410 0x5652affe9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.616+0000 7f90fb799740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5652affdfd80 con 0x5652afc0d730 2024-04-25T10:41:52.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.616+0000 7f90fb799740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5652afece5d0 con 0x5652afc05550 2024-04-25T10:41:52.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.616+0000 7f90fb799740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5652affdff00 con 0x5652afc2e410 2024-04-25T10:41:52.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.616+0000 7f90fb799740 10 client.0 _ll_register_callbacks cb 0x5652afebc080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:52.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.616+0000 7f90edd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc0d730 0x5652affdd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.616+0000 7f90edd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc0d730 0x5652affdd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34904/0 (socket says 172.21.15.43:34904) 2024-04-25T10:41:52.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.616+0000 7f90edd9b640 1 -- 172.21.15.43:0/1056029665 learned_addr learned my addr 172.21.15.43:0/1056029665 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:52.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.616+0000 7f90ee59c640 1 --2- 172.21.15.43:0/1056029665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc05550 0x5652affdd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.616+0000 7f90eed9d640 1 --2- 172.21.15.43:0/1056029665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc2e410 0x5652affe9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.617+0000 7f90edd9b640 1 -- 172.21.15.43:0/1056029665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc2e410 msgr2=0x5652affe9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:52.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.617+0000 7f90edd9b640 1 --2- 172.21.15.43:0/1056029665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc2e410 0x5652affe9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.617+0000 7f90edd9b640 1 -- 172.21.15.43:0/1056029665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc05550 msgr2=0x5652affdd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:52.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.617+0000 7f90edd9b640 1 --2- 172.21.15.43:0/1056029665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc05550 0x5652affdd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.617+0000 7f90edd9b640 1 -- 172.21.15.43:0/1056029665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652afd6ef70 con 0x5652afc0d730 2024-04-25T10:41:52.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.617+0000 7f90eed9d640 1 --2- 172.21.15.43:0/1056029665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc2e410 0x5652affe9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:41:52.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.617+0000 7f90edd9b640 1 --2- 172.21.15.43:0/1056029665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc0d730 0x5652affdd580 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f90e4005ab0 tx=0x7f90e40316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4e7886965db3c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:52.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.617+0000 7f90d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:52.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.617+0000 7f90d5ffb640 1 -- 172.21.15.43:0/1056029665 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4040070 con 0x5652afc0d730 2024-04-25T10:41:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.617+0000 7f90d5ffb640 1 -- 172.21.15.43:0/1056029665 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f90e4004250 con 0x5652afc0d730 2024-04-25T10:41:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.617+0000 7f90fb799740 1 -- 172.21.15.43:0/1056029665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90e8019410 con 0x5652afc0d730 2024-04-25T10:41:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.617+0000 7f90d5ffb640 1 -- 172.21.15.43:0/1056029665 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f90e4035040 con 0x5652afc0d730 2024-04-25T10:41:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.617+0000 7f90fb799740 10 client.18294 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.617+0000 7f90fb799740 1 -- 172.21.15.43:0/1056029665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5652afc06170 con 0x5652afc0d730 2024-04-25T10:41:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.617+0000 7f90d5ffb640 1 -- 172.21.15.43:0/1056029665 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e8019410 con 0x5652afc0d730 2024-04-25T10:41:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.618+0000 7f90d5ffb640 1 -- 172.21.15.43:0/1056029665 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f90e4004ad0 con 0x5652afc0d730 2024-04-25T10:41:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.618+0000 7f90fb799740 10 client.18294 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.618+0000 7f90fb799740 10 client.18294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.618+0000 7f90fb799740 20 client.18294 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.618+0000 7f90fb799740 1 --2- 172.21.15.43:0/1056029665 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5652afbed1f0 0x5652affef410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.618+0000 7f90fb799740 4 client.18294 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:41:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.618+0000 7f90fb799740 1 -- 172.21.15.43:0/1056029665 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652aff42710 con 0x5652afbed1f0 2024-04-25T10:41:52.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.618+0000 7f90ee59c640 1 --2- 172.21.15.43:0/1056029665 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5652afbed1f0 0x5652affef410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.619+0000 7f90ee59c640 1 --2- 172.21.15.43:0/1056029665 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5652afbed1f0 0x5652affef410 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31275018535a664d server_cookie=3ae143b54ac432a9 in_seq=0 out_seq=0 2024-04-25T10:41:52.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.619+0000 7f90d5ffb640 10 client.18294 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:52.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.619+0000 7f90d5ffb640 1 -- 172.21.15.43:0/1056029665 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5652aff42710 con 0x5652afbed1f0 2024-04-25T10:41:52.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.619+0000 7f90d5ffb640 10 client.18294 handle_command_reply: tid=0 2024-04-25T10:41:52.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.619+0000 7f90fb799740 1 client.18294 shutdown 2024-04-25T10:41:52.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.620+0000 7f90fb799740 1 -- 172.21.15.43:0/1056029665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc0d730 msgr2=0x5652affdd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:52.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.620+0000 7f90fb799740 1 --2- 172.21.15.43:0/1056029665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc0d730 0x5652affdd580 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f90e4005ab0 tx=0x7f90e40316d0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.620+0000 7f90fb799740 1 -- 172.21.15.43:0/1056029665 shutdown_connections 2024-04-25T10:41:52.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.620+0000 7f90fb799740 1 --2- 172.21.15.43:0/1056029665 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5652afbed1f0 0x5652affef410 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.620+0000 7f90fb799740 1 --2- 172.21.15.43:0/1056029665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc2e410 0x5652affe9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.620+0000 7f90fb799740 1 --2- 172.21.15.43:0/1056029665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc0d730 0x5652affdd580 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.620+0000 7f90d5ffb640 0 client.18294 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:52.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.620+0000 7f90fb799740 1 --2- 172.21.15.43:0/1056029665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc05550 0x5652affdd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.620+0000 7f90fb799740 1 -- 172.21.15.43:0/1056029665 >> 172.21.15.43:0/1056029665 conn(0x5652afc22560 msgr2=0x5652affd02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:52.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.620+0000 7f90fb799740 1 -- 172.21.15.43:0/1056029665 shutdown_connections 2024-04-25T10:41:52.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.620+0000 7f90fb799740 1 -- 172.21.15.43:0/1056029665 wait complete. 2024-04-25T10:41:52.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.620+0000 7f90fb799740 20 client.18294 trim_cache size 0 max 16384 2024-04-25T10:41:52.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.638+0000 7f90fb799740 1 Processor -- start 2024-04-25T10:41:52.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.638+0000 7f90fb799740 1 -- start start 2024-04-25T10:41:52.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.639+0000 7f90fb799740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc05550 0x5652affdec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.639+0000 7f90fb799740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc0d730 0x5652affd6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.639+0000 7f90fb799740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc2e410 0x5652affd6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.639+0000 7f90fb799740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01b0210 con 0x5652afc05550 2024-04-25T10:41:52.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.639+0000 7f90fb799740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01adae0 con 0x5652afc0d730 2024-04-25T10:41:52.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.639+0000 7f90fb799740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01ab950 con 0x5652afc2e410 2024-04-25T10:41:52.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.639+0000 7f90ee59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc05550 0x5652affdec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.639+0000 7f90ee59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc05550 0x5652affdec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34914/0 (socket says 172.21.15.43:34914) 2024-04-25T10:41:52.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.639+0000 7f90ee59c640 1 -- 172.21.15.43:0/2105824074 learned_addr learned my addr 172.21.15.43:0/2105824074 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:52.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.639+0000 7f90edd9b640 1 --2- 172.21.15.43:0/2105824074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc0d730 0x5652affd6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.639+0000 7f90ee59c640 1 -- 172.21.15.43:0/2105824074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc2e410 msgr2=0x5652affd6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:52.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.639+0000 7f90ee59c640 1 --2- 172.21.15.43:0/2105824074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc2e410 0x5652affd6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.639+0000 7f90ee59c640 1 -- 172.21.15.43:0/2105824074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc0d730 msgr2=0x5652affd6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:52.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.639+0000 7f90ee59c640 1 --2- 172.21.15.43:0/2105824074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc0d730 0x5652affd6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.639+0000 7f90ee59c640 1 -- 172.21.15.43:0/2105824074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652affd86c0 con 0x5652afc05550 2024-04-25T10:41:52.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.639+0000 7f90edd9b640 1 --2- 172.21.15.43:0/2105824074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc0d730 0x5652affd6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:41:52.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.639+0000 7f90ee59c640 1 --2- 172.21.15.43:0/2105824074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc05550 0x5652affdec60 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f90e00afd70 tx=0x7f90e000df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1648864c8db0501f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:52.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.639+0000 7f90d7fff640 1 -- 172.21.15.43:0/2105824074 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0021040 con 0x5652afc05550 2024-04-25T10:41:52.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.639+0000 7f90d7fff640 1 -- 172.21.15.43:0/2105824074 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e00ab100 con 0x5652afc05550 2024-04-25T10:41:52.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.639+0000 7f90d7fff640 1 -- 172.21.15.43:0/2105824074 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e00a2920 con 0x5652afc05550 2024-04-25T10:41:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.640+0000 7f90fb799740 1 -- 172.21.15.43:0/2105824074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc05550 msgr2=0x5652affdec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.640+0000 7f90fb799740 1 --2- 172.21.15.43:0/2105824074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc05550 0x5652affdec60 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f90e00afd70 tx=0x7f90e000df10 comp rx=0 tx=0).stop 2024-04-25T10:41:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.640+0000 7f90fb799740 1 -- 172.21.15.43:0/2105824074 shutdown_connections 2024-04-25T10:41:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.640+0000 7f90fb799740 1 --2- 172.21.15.43:0/2105824074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc0d730 0x5652affd6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.640+0000 7f90fb799740 1 --2- 172.21.15.43:0/2105824074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc2e410 0x5652affd6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.640+0000 7f90fb799740 1 --2- 172.21.15.43:0/2105824074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc05550 0x5652affdec60 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.640+0000 7f90fb799740 1 -- 172.21.15.43:0/2105824074 >> 172.21.15.43:0/2105824074 conn(0x5652afc22560 msgr2=0x5652affde550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.640+0000 7f90fb799740 1 -- 172.21.15.43:0/2105824074 shutdown_connections 2024-04-25T10:41:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.640+0000 7f90fb799740 1 -- 172.21.15.43:0/2105824074 wait complete. 2024-04-25T10:41:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.640+0000 7f90fb799740 1 -- start start 2024-04-25T10:41:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.641+0000 7f90fb799740 1 Processor -- start 2024-04-25T10:41:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.641+0000 7f90fb799740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc05550 0x5652affe51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.641+0000 7f90fb799740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc0d730 0x5652affe56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.641+0000 7f90fb799740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc2e410 0x5652affe0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.641+0000 7f90fb799740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01abfa0 con 0x5652afc05550 2024-04-25T10:41:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.641+0000 7f90fb799740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01ad410 con 0x5652afc0d730 2024-04-25T10:41:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.641+0000 7f90fb799740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01ac670 con 0x5652afc2e410 2024-04-25T10:41:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.641+0000 7f90fb799740 10 client.0 _ll_register_callbacks cb 0x7f90e4032da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.641+0000 7f90ee59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc05550 0x5652affe51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.641+0000 7f90ee59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc05550 0x5652affe51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34930/0 (socket says 172.21.15.43:34930) 2024-04-25T10:41:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.641+0000 7f90ee59c640 1 -- 172.21.15.43:0/1404479068 learned_addr learned my addr 172.21.15.43:0/1404479068 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.641+0000 7f90edd9b640 1 --2- 172.21.15.43:0/1404479068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc0d730 0x5652affe56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.641+0000 7f90eed9d640 1 --2- 172.21.15.43:0/1404479068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc2e410 0x5652affe0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.641+0000 7f90ee59c640 1 -- 172.21.15.43:0/1404479068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc2e410 msgr2=0x5652affe0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.641+0000 7f90ee59c640 1 --2- 172.21.15.43:0/1404479068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc2e410 0x5652affe0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.641+0000 7f90ee59c640 1 -- 172.21.15.43:0/1404479068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc0d730 msgr2=0x5652affe56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.641+0000 7f90ee59c640 1 --2- 172.21.15.43:0/1404479068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc0d730 0x5652affe56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.641+0000 7f90ee59c640 1 -- 172.21.15.43:0/1404479068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652affe0690 con 0x5652afc05550 2024-04-25T10:41:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.641+0000 7f90eed9d640 1 --2- 172.21.15.43:0/1404479068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc2e410 0x5652affe0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:41:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.642+0000 7f90ee59c640 1 --2- 172.21.15.43:0/1404479068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc05550 0x5652affe51a0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f90e00a2d80 tx=0x7f90e000de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b58d0165a495e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.642+0000 7f90d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.642+0000 7f90d5ffb640 1 -- 172.21.15.43:0/1404479068 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e006b060 con 0x5652afc05550 2024-04-25T10:41:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.642+0000 7f90d5ffb640 1 -- 172.21.15.43:0/1404479068 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f90e00a5fb0 con 0x5652afc05550 2024-04-25T10:41:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.642+0000 7f90fb799740 1 -- 172.21.15.43:0/1404479068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90e00ab100 con 0x5652afc05550 2024-04-25T10:41:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.642+0000 7f90d5ffb640 1 -- 172.21.15.43:0/1404479068 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f90e0021070 con 0x5652afc05550 2024-04-25T10:41:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.642+0000 7f90fb799740 10 client.18300 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.642+0000 7f90fb799740 1 -- 172.21.15.43:0/1404479068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5652b0057d40 con 0x5652afc05550 2024-04-25T10:41:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.642+0000 7f90d5ffb640 1 -- 172.21.15.43:0/1404479068 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e00ab100 con 0x5652afc05550 2024-04-25T10:41:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.642+0000 7f90d5ffb640 1 -- 172.21.15.43:0/1404479068 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f90e006d300 con 0x5652afc05550 2024-04-25T10:41:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.642+0000 7f90fb799740 10 client.18300 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.642+0000 7f90fb799740 10 client.18300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.642+0000 7f90fb799740 20 client.18300 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.642+0000 7f90fb799740 1 --2- 172.21.15.43:0/1404479068 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5652afbc2630 0x5652b005a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.642+0000 7f90fb799740 4 client.18300 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:41:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.642+0000 7f90fb799740 1 -- 172.21.15.43:0/1404479068 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5652aff3c580 con 0x5652afbc2630 2024-04-25T10:41:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.643+0000 7f90edd9b640 1 --2- 172.21.15.43:0/1404479068 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5652afbc2630 0x5652b005a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.643+0000 7f90edd9b640 1 --2- 172.21.15.43:0/1404479068 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5652afbc2630 0x5652b005a2a0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=307b15007f5408ac server_cookie=663f40c7ec73952 in_seq=0 out_seq=0 2024-04-25T10:41:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.643+0000 7f90d5ffb640 10 client.18300 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.644+0000 7f90d5ffb640 1 -- 172.21.15.43:0/1404479068 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5652aff3c580 con 0x5652afbc2630 2024-04-25T10:41:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.644+0000 7f90d5ffb640 10 client.18300 handle_command_reply: tid=0 2024-04-25T10:41:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.644+0000 7f90fb799740 1 client.18300 shutdown 2024-04-25T10:41:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.644+0000 7f90fb799740 1 -- 172.21.15.43:0/1404479068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc05550 msgr2=0x5652affe51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.644+0000 7f90fb799740 1 --2- 172.21.15.43:0/1404479068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc05550 0x5652affe51a0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f90e00a2d80 tx=0x7f90e000de80 comp rx=0 tx=0).stop 2024-04-25T10:41:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90fb799740 1 -- 172.21.15.43:0/1404479068 shutdown_connections 2024-04-25T10:41:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90fb799740 1 --2- 172.21.15.43:0/1404479068 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5652afbc2630 0x5652b005a2a0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90fb799740 1 --2- 172.21.15.43:0/1404479068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652afc0d730 0x5652affe56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90fb799740 1 --2- 172.21.15.43:0/1404479068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652afc2e410 0x5652affe0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90fb799740 1 --2- 172.21.15.43:0/1404479068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652afc05550 0x5652affe51a0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90d5ffb640 0 client.18300 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:52.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90fb799740 1 -- 172.21.15.43:0/1404479068 >> 172.21.15.43:0/1404479068 conn(0x5652afc22560 msgr2=0x5652affc94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:52.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90fb799740 1 -- 172.21.15.43:0/1404479068 shutdown_connections 2024-04-25T10:41:52.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90fb799740 1 -- 172.21.15.43:0/1404479068 wait complete. 2024-04-25T10:41:52.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90fb799740 20 client.18300 trim_cache size 0 max 16384 2024-04-25T10:41:52.652 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:41:52.652 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:41:52.652 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "dc987c98-d463-4a63-9cfd-9a67211d2762", 2024-04-25T10:41:52.653 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:41:52.653 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:41:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90f48e7640 1 -- 172.21.15.43:0/3200167201 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f90d80afc60 msgr2=0x7f90d80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90f48e7640 1 --2- 172.21.15.43:0/3200167201 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f90d80afc60 0x7f90d80b2170 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f90e801e000 tx=0x7f90e801f000 comp rx=0 tx=0).stop 2024-04-25T10:41:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90f48e7640 1 -- 172.21.15.43:0/3200167201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90f0104270 msgr2=0x7f90f019a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90f48e7640 1 --2- 172.21.15.43:0/3200167201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90f0104270 0x7f90f019a790 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f90e000d770 tx=0x7f90e000dc30 comp rx=0 tx=0).stop 2024-04-25T10:41:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90f48e7640 1 -- 172.21.15.43:0/3200167201 shutdown_connections 2024-04-25T10:41:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90f48e7640 1 --2- 172.21.15.43:0/3200167201 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f90d80afc60 0x7f90d80b2170 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90f48e7640 1 --2- 172.21.15.43:0/3200167201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90f0109b20 0x7f90f019acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90f48e7640 1 --2- 172.21.15.43:0/3200167201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f0109fc0 0x7f90f019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90f48e7640 1 --2- 172.21.15.43:0/3200167201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90f0104270 0x7f90f019a790 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90f48e7640 1 -- 172.21.15.43:0/3200167201 >> 172.21.15.43:0/3200167201 conn(0x7f90f00ff1e0 msgr2=0x7f90f010f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90f48e7640 1 -- 172.21.15.43:0/3200167201 shutdown_connections 2024-04-25T10:41:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.645+0000 7f90f48e7640 1 -- 172.21.15.43:0/3200167201 wait complete. 2024-04-25T10:41:52.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:41:52.659 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:41:52.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.771+0000 7f5e30f64640 1 -- 172.21.15.43:0/2357316093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2c108a10 msgr2=0x7f5e2c102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:52.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.771+0000 7f5e30f64640 1 --2- 172.21.15.43:0/2357316093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2c108a10 0x7f5e2c102610 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009900 tx=0x7f5e1c02a410 comp rx=0 tx=0).stop 2024-04-25T10:41:52.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.771+0000 7f5e30f64640 1 -- 172.21.15.43:0/2357316093 shutdown_connections 2024-04-25T10:41:52.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.771+0000 7f5e30f64640 1 --2- 172.21.15.43:0/2357316093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e2c108eb0 0x7f5e2c102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.771+0000 7f5e30f64640 1 --2- 172.21.15.43:0/2357316093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e2c1031c0 0x7f5e2c10f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.771+0000 7f5e30f64640 1 --2- 172.21.15.43:0/2357316093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2c108a10 0x7f5e2c102610 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.771+0000 7f5e30f64640 1 -- 172.21.15.43:0/2357316093 >> 172.21.15.43:0/2357316093 conn(0x7f5e2c078110 msgr2=0x7f5e2c078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:52.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.771+0000 7f5e30f64640 1 -- 172.21.15.43:0/2357316093 shutdown_connections 2024-04-25T10:41:52.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.771+0000 7f5e30f64640 1 -- 172.21.15.43:0/2357316093 wait complete. 2024-04-25T10:41:52.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.772+0000 7f5e30f64640 1 Processor -- start 2024-04-25T10:41:52.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.772+0000 7f5e30f64640 1 -- start start 2024-04-25T10:41:52.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.772+0000 7f5e30f64640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e2c1031c0 0x7f5e2c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.772+0000 7f5e30f64640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e2c108a10 0x7f5e2c19ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.772+0000 7f5e30f64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2c108eb0 0x7f5e2c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.772+0000 7f5e30f64640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c112c30 con 0x7f5e2c108eb0 2024-04-25T10:41:52.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.772+0000 7f5e30f64640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c112ab0 con 0x7f5e2c1031c0 2024-04-25T10:41:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.772+0000 7f5e30f64640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c112db0 con 0x7f5e2c108a10 2024-04-25T10:41:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.773+0000 7f5e2ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2c108eb0 0x7f5e2c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.773+0000 7f5e2a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e2c1031c0 0x7f5e2c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.773+0000 7f5e2a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e2c1031c0 0x7f5e2c19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:54398/0 (socket says 172.21.15.43:54398) 2024-04-25T10:41:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.773+0000 7f5e2ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2c108eb0 0x7f5e2c19efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34936/0 (socket says 172.21.15.43:34936) 2024-04-25T10:41:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.773+0000 7f5e2a59c640 1 -- 172.21.15.43:0/3554393330 learned_addr learned my addr 172.21.15.43:0/3554393330 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.773+0000 7f5e29d9b640 1 --2- 172.21.15.43:0/3554393330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e2c108a10 0x7f5e2c19ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.773+0000 7f5e2ad9d640 1 -- 172.21.15.43:0/3554393330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e2c108a10 msgr2=0x7f5e2c19ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.773+0000 7f5e2ad9d640 1 --2- 172.21.15.43:0/3554393330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e2c108a10 0x7f5e2c19ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.773+0000 7f5e2ad9d640 1 -- 172.21.15.43:0/3554393330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e2c1031c0 msgr2=0x7f5e2c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.773+0000 7f5e2ad9d640 1 --2- 172.21.15.43:0/3554393330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e2c1031c0 0x7f5e2c19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.773+0000 7f5e2ad9d640 1 -- 172.21.15.43:0/3554393330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2c19f5e0 con 0x7f5e2c108eb0 2024-04-25T10:41:52.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.773+0000 7f5e2a59c640 1 --2- 172.21.15.43:0/3554393330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e2c1031c0 0x7f5e2c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:41:52.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.773+0000 7f5e2ad9d640 1 --2- 172.21.15.43:0/3554393330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2c108eb0 0x7f5e2c19efa0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400ca30 tx=0x7f5e2400cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:52.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.773+0000 7f5e1b7fe640 1 -- 172.21.15.43:0/3554393330 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24007d90 con 0x7f5e2c108eb0 2024-04-25T10:41:52.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.773+0000 7f5e30f64640 1 -- 172.21.15.43:0/3554393330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2c19f8d0 con 0x7f5e2c108eb0 2024-04-25T10:41:52.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.774+0000 7f5e1b7fe640 1 -- 172.21.15.43:0/3554393330 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e24005020 con 0x7f5e2c108eb0 2024-04-25T10:41:52.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.774+0000 7f5e1b7fe640 1 -- 172.21.15.43:0/3554393330 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400f670 con 0x7f5e2c108eb0 2024-04-25T10:41:52.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.774+0000 7f5e30f64640 1 -- 172.21.15.43:0/3554393330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e2c1a7250 con 0x7f5e2c108eb0 2024-04-25T10:41:52.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.776+0000 7f5e1b7fe640 1 -- 172.21.15.43:0/3554393330 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f5e240051d0 con 0x7f5e2c108eb0 2024-04-25T10:41:52.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.776+0000 7f5e37e15740 1 Processor -- start 2024-04-25T10:41:52.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.776+0000 7f5e37e15740 1 -- start start 2024-04-25T10:41:52.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.776+0000 7f5e37e15740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb5d390 0x561c0cefda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.777+0000 7f5e37e15740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c0cb344d0 0x561c0cefdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.777+0000 7f5e37e15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb3c6b0 0x561c0cf0a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.777+0000 7f5e37e15740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561c0ce63d80 con 0x561c0cb5d390 2024-04-25T10:41:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.777+0000 7f5e37e15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561c0ce1d8b0 con 0x561c0cb3c6b0 2024-04-25T10:41:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.777+0000 7f5e37e15740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561c0ce5fd40 con 0x561c0cb344d0 2024-04-25T10:41:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.777+0000 7f5e29d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb5d390 0x561c0cefda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.777+0000 7f5e29d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb5d390 0x561c0cefda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34952/0 (socket says 172.21.15.43:34952) 2024-04-25T10:41:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.777+0000 7f5e29d9b640 1 -- 172.21.15.43:0/1825185527 learned_addr learned my addr 172.21.15.43:0/1825185527 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.777+0000 7f5e2a59c640 1 --2- 172.21.15.43:0/1825185527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c0cb344d0 0x561c0cefdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.777+0000 7f5e1b7fe640 1 --2- 172.21.15.43:0/3554393330 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5e140afc60 0x7f5e140b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.777+0000 7f5e29d9b640 1 --2- 172.21.15.43:0/1825185527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb3c6b0 0x561c0cf0a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.777+0000 7f5e2ad9d640 1 --2- 172.21.15.43:0/3554393330 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5e140afc60 0x7f5e140b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.777+0000 7f5e2a59c640 1 -- 172.21.15.43:0/1825185527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb3c6b0 msgr2=0x561c0cf0a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.777+0000 7f5e2a59c640 1 --2- 172.21.15.43:0/1825185527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb3c6b0 0x561c0cf0a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.777+0000 7f5e2a59c640 1 -- 172.21.15.43:0/1825185527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb5d390 msgr2=0x561c0cefda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.777+0000 7f5e2a59c640 1 --2- 172.21.15.43:0/1825185527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb5d390 0x561c0cefda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.777+0000 7f5e2a59c640 1 -- 172.21.15.43:0/1825185527 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c0cdeea40 con 0x561c0cb344d0 2024-04-25T10:41:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.777+0000 7f5e29d9b640 1 --2- 172.21.15.43:0/1825185527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb3c6b0 0x561c0cf0a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.777+0000 7f5e1b7fe640 1 -- 172.21.15.43:0/3554393330 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f5e240df1e0 con 0x7f5e2c108eb0 2024-04-25T10:41:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.777+0000 7f5e2a59c640 1 --2- 172.21.15.43:0/1825185527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c0cb344d0 0x561c0cefdfa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c031440 tx=0x7f5e1c0315e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ddfd5eb4d2d1186 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.778+0000 7f5e2ad9d640 1 --2- 172.21.15.43:0/3554393330 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5e140afc60 0x7f5e140b2170 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5e240dc9e0 tx=0x7f5e24072b30 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.778+0000 7f5e18ff9640 1 -- 172.21.15.43:0/1825185527 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c035040 con 0x561c0cb344d0 2024-04-25T10:41:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.778+0000 7f5e18ff9640 1 -- 172.21.15.43:0/1825185527 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e1c005020 con 0x561c0cb344d0 2024-04-25T10:41:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.778+0000 7f5e18ff9640 1 -- 172.21.15.43:0/1825185527 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c004ad0 con 0x561c0cb344d0 2024-04-25T10:41:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.778+0000 7f5e37e15740 1 -- 172.21.15.43:0/1825185527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c0cb344d0 msgr2=0x561c0cefdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.778+0000 7f5e37e15740 1 --2- 172.21.15.43:0/1825185527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c0cb344d0 0x561c0cefdfa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c031440 tx=0x7f5e1c0315e0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.779+0000 7f5e37e15740 1 -- 172.21.15.43:0/1825185527 shutdown_connections 2024-04-25T10:41:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.779+0000 7f5e37e15740 1 --2- 172.21.15.43:0/1825185527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb3c6b0 0x561c0cf0a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.779+0000 7f5e37e15740 1 --2- 172.21.15.43:0/1825185527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c0cb344d0 0x561c0cefdfa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.779+0000 7f5e37e15740 1 --2- 172.21.15.43:0/1825185527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb5d390 0x561c0cefda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.779+0000 7f5e37e15740 1 -- 172.21.15.43:0/1825185527 >> 172.21.15.43:0/1825185527 conn(0x561c0cb514e0 msgr2=0x561c0cefa210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:52.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.779+0000 7f5e37e15740 1 -- 172.21.15.43:0/1825185527 shutdown_connections 2024-04-25T10:41:52.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.779+0000 7f5e37e15740 1 -- 172.21.15.43:0/1825185527 wait complete. 2024-04-25T10:41:52.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.779+0000 7f5e37e15740 1 -- start start 2024-04-25T10:41:52.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.779+0000 7f5e37e15740 1 Processor -- start 2024-04-25T10:41:52.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.779+0000 7f5e37e15740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb344d0 0x561c0cf085f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.779+0000 7f5e37e15740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c0cb3c6b0 0x561c0cf08b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.779+0000 7f5e37e15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb5d390 0x561c0cf0e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.779+0000 7f5e37e15740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561c0cf0af80 con 0x561c0cb344d0 2024-04-25T10:41:52.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.779+0000 7f5e37e15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561c0cdead10 con 0x561c0cb5d390 2024-04-25T10:41:52.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.779+0000 7f5e37e15740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561c0cf0b100 con 0x561c0cb3c6b0 2024-04-25T10:41:52.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.779+0000 7f5e37e15740 10 client.0 _ll_register_callbacks cb 0x561c0ce17210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:52.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.779+0000 7f5e29d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb344d0 0x561c0cf085f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.780+0000 7f5e29d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb344d0 0x561c0cf085f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34960/0 (socket says 172.21.15.43:34960) 2024-04-25T10:41:52.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.780+0000 7f5e29d9b640 1 -- 172.21.15.43:0/3518637562 learned_addr learned my addr 172.21.15.43:0/3518637562 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:52.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.780+0000 7f5e2a59c640 1 --2- 172.21.15.43:0/3518637562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c0cb3c6b0 0x561c0cf08b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.780+0000 7f5e29d9b640 1 --2- 172.21.15.43:0/3518637562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb5d390 0x561c0cf0e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.780+0000 7f5e29d9b640 1 -- 172.21.15.43:0/3518637562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c0cb3c6b0 msgr2=0x561c0cf08b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:52.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.780+0000 7f5e29d9b640 1 --2- 172.21.15.43:0/3518637562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c0cb3c6b0 0x561c0cf08b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.780+0000 7f5e29d9b640 1 -- 172.21.15.43:0/3518637562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb5d390 msgr2=0x561c0cf0e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:52.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.780+0000 7f5e29d9b640 1 --2- 172.21.15.43:0/3518637562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb5d390 0x561c0cf0e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.780+0000 7f5e29d9b640 1 -- 172.21.15.43:0/3518637562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c0cc3f7d0 con 0x561c0cb344d0 2024-04-25T10:41:52.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.780+0000 7f5e29d9b640 1 --2- 172.21.15.43:0/3518637562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb344d0 0x561c0cf085f0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f5e20012950 tx=0x7f5e2000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5fc7e72afc54c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:52.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.780+0000 7f5e11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:52.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.780+0000 7f5e11ffb640 1 -- 172.21.15.43:0/3518637562 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e20010ce0 con 0x561c0cb344d0 2024-04-25T10:41:52.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.780+0000 7f5e11ffb640 1 -- 172.21.15.43:0/3518637562 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f5e20005020 con 0x561c0cb344d0 2024-04-25T10:41:52.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.780+0000 7f5e37e15740 1 -- 172.21.15.43:0/3518637562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e1c005020 con 0x561c0cb344d0 2024-04-25T10:41:52.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.780+0000 7f5e11ffb640 1 -- 172.21.15.43:0/3518637562 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5e20010e80 con 0x561c0cb344d0 2024-04-25T10:41:52.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.780+0000 7f5e37e15740 10 client.18312 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:52.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.780+0000 7f5e37e15740 1 -- 172.21.15.43:0/3518637562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561c0cb753f0 con 0x561c0cb344d0 2024-04-25T10:41:52.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.781+0000 7f5e11ffb640 1 -- 172.21.15.43:0/3518637562 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e1c005020 con 0x561c0cb344d0 2024-04-25T10:41:52.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.781+0000 7f5e11ffb640 1 -- 172.21.15.43:0/3518637562 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5e2001e030 con 0x561c0cb344d0 2024-04-25T10:41:52.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.781+0000 7f5e37e15740 10 client.18312 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:52.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.781+0000 7f5e37e15740 10 client.18312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:52.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.781+0000 7f5e37e15740 20 client.18312 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:52.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.781+0000 7f5e37e15740 1 --2- 172.21.15.43:0/3518637562 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561c0cb1bf60 0x561c0cf1a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.781+0000 7f5e37e15740 4 client.18312 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:41:52.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.781+0000 7f5e37e15740 1 -- 172.21.15.43:0/3518637562 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c0ce71990 con 0x561c0cb1bf60 2024-04-25T10:41:52.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.781+0000 7f5e2a59c640 1 --2- 172.21.15.43:0/3518637562 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561c0cb1bf60 0x561c0cf1a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.782+0000 7f5e2a59c640 1 --2- 172.21.15.43:0/3518637562 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561c0cb1bf60 0x561c0cf1a740 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6aea7006df4f458 server_cookie=284317707864049e in_seq=0 out_seq=0 2024-04-25T10:41:52.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.782+0000 7f5e11ffb640 10 client.18312 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:52.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.782+0000 7f5e11ffb640 1 -- 172.21.15.43:0/3518637562 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561c0ce71990 con 0x561c0cb1bf60 2024-04-25T10:41:52.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.782+0000 7f5e11ffb640 10 client.18312 handle_command_reply: tid=0 2024-04-25T10:41:52.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.782+0000 7f5e37e15740 1 client.18312 shutdown 2024-04-25T10:41:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.782+0000 7f5e37e15740 1 -- 172.21.15.43:0/3518637562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb344d0 msgr2=0x561c0cf085f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.782+0000 7f5e37e15740 1 --2- 172.21.15.43:0/3518637562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb344d0 0x561c0cf085f0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f5e20012950 tx=0x7f5e2000b390 comp rx=0 tx=0).stop 2024-04-25T10:41:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.782+0000 7f5e37e15740 1 -- 172.21.15.43:0/3518637562 shutdown_connections 2024-04-25T10:41:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.782+0000 7f5e37e15740 1 --2- 172.21.15.43:0/3518637562 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561c0cb1bf60 0x561c0cf1a740 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.782+0000 7f5e37e15740 1 --2- 172.21.15.43:0/3518637562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb5d390 0x561c0cf0e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.782+0000 7f5e37e15740 1 --2- 172.21.15.43:0/3518637562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c0cb3c6b0 0x561c0cf08b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.782+0000 7f5e11ffb640 0 client.18312 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.782+0000 7f5e37e15740 1 --2- 172.21.15.43:0/3518637562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb344d0 0x561c0cf085f0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.782+0000 7f5e37e15740 1 -- 172.21.15.43:0/3518637562 >> 172.21.15.43:0/3518637562 conn(0x561c0cb514e0 msgr2=0x561c0ceff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.783+0000 7f5e37e15740 1 -- 172.21.15.43:0/3518637562 shutdown_connections 2024-04-25T10:41:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.783+0000 7f5e37e15740 1 -- 172.21.15.43:0/3518637562 wait complete. 2024-04-25T10:41:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.783+0000 7f5e37e15740 20 client.18312 trim_cache size 0 max 16384 2024-04-25T10:41:52.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.801+0000 7f5e37e15740 1 Processor -- start 2024-04-25T10:41:52.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.801+0000 7f5e37e15740 1 -- start start 2024-04-25T10:41:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.801+0000 7f5e37e15740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c0cb344d0 0x561c0cf072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.801+0000 7f5e37e15740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb3c6b0 0x561c0cf1ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.801+0000 7f5e37e15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb5d390 0x561c0cf20440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.801+0000 7f5e37e15740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1b2300 con 0x561c0cb3c6b0 2024-04-25T10:41:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.801+0000 7f5e37e15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1afbd0 con 0x561c0cb5d390 2024-04-25T10:41:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.801+0000 7f5e37e15740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1ada40 con 0x561c0cb344d0 2024-04-25T10:41:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.801+0000 7f5e2a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb3c6b0 0x561c0cf1ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.801+0000 7f5e2a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb3c6b0 0x561c0cf1ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34962/0 (socket says 172.21.15.43:34962) 2024-04-25T10:41:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.801+0000 7f5e2a59c640 1 -- 172.21.15.43:0/1503528282 learned_addr learned my addr 172.21.15.43:0/1503528282 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.801+0000 7f5e2a59c640 1 -- 172.21.15.43:0/1503528282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c0cb344d0 msgr2=0x561c0cf072b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.801+0000 7f5e2a59c640 1 --2- 172.21.15.43:0/1503528282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c0cb344d0 0x561c0cf072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.801+0000 7f5e2a59c640 1 -- 172.21.15.43:0/1503528282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb5d390 msgr2=0x561c0cf20440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.801+0000 7f5e29d9b640 1 --2- 172.21.15.43:0/1503528282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb5d390 0x561c0cf20440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.801+0000 7f5e2a59c640 1 --2- 172.21.15.43:0/1503528282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb5d390 0x561c0cf20440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.801+0000 7f5e2a59c640 1 -- 172.21.15.43:0/1503528282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c0cb350f0 con 0x561c0cb3c6b0 2024-04-25T10:41:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.801+0000 7f5e2a59c640 1 --2- 172.21.15.43:0/1503528282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb3c6b0 0x561c0cf1ff00 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009230 tx=0x7f5e1c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d1e65f8b934671 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.802+0000 7f5e13fff640 1 -- 172.21.15.43:0/1503528282 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c035040 con 0x561c0cb3c6b0 2024-04-25T10:41:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.802+0000 7f5e13fff640 1 -- 172.21.15.43:0/1503528282 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e1c0042b0 con 0x561c0cb3c6b0 2024-04-25T10:41:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.802+0000 7f5e13fff640 1 -- 172.21.15.43:0/1503528282 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c0035f0 con 0x561c0cb3c6b0 2024-04-25T10:41:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.802+0000 7f5e37e15740 1 -- 172.21.15.43:0/1503528282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb3c6b0 msgr2=0x561c0cf1ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.802+0000 7f5e37e15740 1 --2- 172.21.15.43:0/1503528282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb3c6b0 0x561c0cf1ff00 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009230 tx=0x7f5e1c004770 comp rx=0 tx=0).stop 2024-04-25T10:41:52.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.802+0000 7f5e37e15740 1 -- 172.21.15.43:0/1503528282 shutdown_connections 2024-04-25T10:41:52.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.802+0000 7f5e37e15740 1 --2- 172.21.15.43:0/1503528282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb5d390 0x561c0cf20440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.802+0000 7f5e37e15740 1 --2- 172.21.15.43:0/1503528282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb3c6b0 0x561c0cf1ff00 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.802+0000 7f5e37e15740 1 --2- 172.21.15.43:0/1503528282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c0cb344d0 0x561c0cf072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.802+0000 7f5e37e15740 1 -- 172.21.15.43:0/1503528282 >> 172.21.15.43:0/1503528282 conn(0x561c0cb514e0 msgr2=0x561c0cf06ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:52.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.802+0000 7f5e37e15740 1 -- 172.21.15.43:0/1503528282 shutdown_connections 2024-04-25T10:41:52.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.802+0000 7f5e37e15740 1 -- 172.21.15.43:0/1503528282 wait complete. 2024-04-25T10:41:52.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.803+0000 7f5e37e15740 1 -- start start 2024-04-25T10:41:52.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.803+0000 7f5e37e15740 1 Processor -- start 2024-04-25T10:41:52.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.803+0000 7f5e37e15740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c0cb344d0 0x561c0cf0f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.803+0000 7f5e37e15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb3c6b0 0x561c0cf0fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.803+0000 7f5e37e15740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb5d390 0x561c0cf80aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.803+0000 7f5e37e15740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1ae090 con 0x561c0cb5d390 2024-04-25T10:41:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.803+0000 7f5e37e15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1af500 con 0x561c0cb3c6b0 2024-04-25T10:41:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.803+0000 7f5e37e15740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1ae760 con 0x561c0cb344d0 2024-04-25T10:41:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.803+0000 7f5e37e15740 10 client.0 _ll_register_callbacks cb 0x7f5e200126f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.803+0000 7f5e29d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb5d390 0x561c0cf80aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.803+0000 7f5e29d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb5d390 0x561c0cf80aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34974/0 (socket says 172.21.15.43:34974) 2024-04-25T10:41:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.803+0000 7f5e29d9b640 1 -- 172.21.15.43:0/3614513081 learned_addr learned my addr 172.21.15.43:0/3614513081 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.804+0000 7f5e2a59c640 1 --2- 172.21.15.43:0/3614513081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb3c6b0 0x561c0cf0fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.804+0000 7f5e29d9b640 1 --2- 172.21.15.43:0/3614513081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c0cb344d0 0x561c0cf0f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.804+0000 7f5e29d9b640 1 -- 172.21.15.43:0/3614513081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c0cb344d0 msgr2=0x561c0cf0f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:52.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.804+0000 7f5e29d9b640 1 --2- 172.21.15.43:0/3614513081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c0cb344d0 0x561c0cf0f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.804+0000 7f5e29d9b640 1 -- 172.21.15.43:0/3614513081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb3c6b0 msgr2=0x561c0cf0fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:52.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.804+0000 7f5e29d9b640 1 --2- 172.21.15.43:0/3614513081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb3c6b0 0x561c0cf0fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.804+0000 7f5e29d9b640 1 -- 172.21.15.43:0/3614513081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c0ce244b0 con 0x561c0cb5d390 2024-04-25T10:41:52.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.804+0000 7f5e2a59c640 1 --2- 172.21.15.43:0/3614513081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb3c6b0 0x561c0cf0fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:52.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.804+0000 7f5e29d9b640 1 --2- 172.21.15.43:0/3614513081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb5d390 0x561c0cf80aa0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x561c0cefa840 tx=0x7f5e2000b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47afbe33a722a584 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:52.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.804+0000 7f5e11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:52.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.804+0000 7f5e11ffb640 1 -- 172.21.15.43:0/3614513081 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e20005020 con 0x561c0cb5d390 2024-04-25T10:41:52.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.804+0000 7f5e11ffb640 1 -- 172.21.15.43:0/3614513081 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f5e200051c0 con 0x561c0cb5d390 2024-04-25T10:41:52.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.804+0000 7f5e37e15740 1 -- 172.21.15.43:0/3614513081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e1c0042b0 con 0x561c0cb5d390 2024-04-25T10:41:52.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.804+0000 7f5e11ffb640 1 -- 172.21.15.43:0/3614513081 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5e200053c0 con 0x561c0cb5d390 2024-04-25T10:41:52.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.804+0000 7f5e37e15740 10 client.18318 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:52.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.804+0000 7f5e37e15740 1 -- 172.21.15.43:0/3614513081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561c0cf81e50 con 0x561c0cb5d390 2024-04-25T10:41:52.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.804+0000 7f5e11ffb640 1 -- 172.21.15.43:0/3614513081 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e1c0042b0 con 0x561c0cb5d390 2024-04-25T10:41:52.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.805+0000 7f5e11ffb640 1 -- 172.21.15.43:0/3614513081 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5e2001e070 con 0x561c0cb5d390 2024-04-25T10:41:52.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.805+0000 7f5e37e15740 10 client.18318 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:52.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.805+0000 7f5e37e15740 10 client.18318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:52.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.805+0000 7f5e37e15740 20 client.18318 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:52.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.805+0000 7f5e37e15740 1 --2- 172.21.15.43:0/3614513081 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561c0caf13a0 0x561c0cf84530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:52.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.805+0000 7f5e37e15740 4 client.18318 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:41:52.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.805+0000 7f5e37e15740 1 -- 172.21.15.43:0/3614513081 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561c0ce6b800 con 0x561c0caf13a0 2024-04-25T10:41:52.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.805+0000 7f5e2a59c640 1 --2- 172.21.15.43:0/3614513081 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561c0caf13a0 0x561c0cf84530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:52.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.805+0000 7f5e2a59c640 1 --2- 172.21.15.43:0/3614513081 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561c0caf13a0 0x561c0cf84530 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a52c3122dbed9216 server_cookie=652db911a4ed2396 in_seq=0 out_seq=0 2024-04-25T10:41:52.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.805+0000 7f5e11ffb640 10 client.18318 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:52.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.806+0000 7f5e11ffb640 1 -- 172.21.15.43:0/3614513081 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561c0ce6b800 con 0x561c0caf13a0 2024-04-25T10:41:52.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.806+0000 7f5e11ffb640 10 client.18318 handle_command_reply: tid=0 2024-04-25T10:41:52.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.806+0000 7f5e37e15740 1 client.18318 shutdown 2024-04-25T10:41:52.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.806+0000 7f5e37e15740 1 -- 172.21.15.43:0/3614513081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb5d390 msgr2=0x561c0cf80aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:52.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.806+0000 7f5e37e15740 1 --2- 172.21.15.43:0/3614513081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb5d390 0x561c0cf80aa0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x561c0cefa840 tx=0x7f5e2000b820 comp rx=0 tx=0).stop 2024-04-25T10:41:52.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.806+0000 7f5e37e15740 1 -- 172.21.15.43:0/3614513081 shutdown_connections 2024-04-25T10:41:52.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.806+0000 7f5e37e15740 1 --2- 172.21.15.43:0/3614513081 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561c0caf13a0 0x561c0cf84530 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.806+0000 7f5e37e15740 1 --2- 172.21.15.43:0/3614513081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c0cb3c6b0 0x561c0cf0fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.806+0000 7f5e37e15740 1 --2- 172.21.15.43:0/3614513081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0cb5d390 0x561c0cf80aa0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.806+0000 7f5e37e15740 1 --2- 172.21.15.43:0/3614513081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c0cb344d0 0x561c0cf0f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.806+0000 7f5e37e15740 1 -- 172.21.15.43:0/3614513081 >> 172.21.15.43:0/3614513081 conn(0x561c0cb514e0 msgr2=0x561c0cf07330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.806+0000 7f5e11ffb640 0 client.18318 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.806+0000 7f5e37e15740 1 -- 172.21.15.43:0/3614513081 shutdown_connections 2024-04-25T10:41:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.806+0000 7f5e37e15740 1 -- 172.21.15.43:0/3614513081 wait complete. 2024-04-25T10:41:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.806+0000 7f5e37e15740 20 client.18318 trim_cache size 0 max 16384 2024-04-25T10:41:52.818 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:41:52.819 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:41:52.819 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:41:52.819 INFO:teuthology.orchestra.run.smithi043.stdout: "dc987c98-d463-4a63-9cfd-9a67211d2762": { 2024-04-25T10:41:52.819 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:41:52.819 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "dc987c98-d463-4a63-9cfd-9a67211d2762", 2024-04-25T10:41:52.819 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:41:52.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:41:52.820 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:41:52.820 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:41:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.806+0000 7f5e30f64640 1 -- 172.21.15.43:0/3554393330 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5e140afc60 msgr2=0x7f5e140b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.806+0000 7f5e30f64640 1 --2- 172.21.15.43:0/3554393330 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5e140afc60 0x7f5e140b2170 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5e240dc9e0 tx=0x7f5e24072b30 comp rx=0 tx=0).stop 2024-04-25T10:41:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.806+0000 7f5e30f64640 1 -- 172.21.15.43:0/3554393330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2c108eb0 msgr2=0x7f5e2c19efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.806+0000 7f5e30f64640 1 --2- 172.21.15.43:0/3554393330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2c108eb0 0x7f5e2c19efa0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400ca30 tx=0x7f5e2400cef0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.807+0000 7f5e30f64640 1 -- 172.21.15.43:0/3554393330 shutdown_connections 2024-04-25T10:41:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.807+0000 7f5e30f64640 1 --2- 172.21.15.43:0/3554393330 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5e140afc60 0x7f5e140b2170 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.807+0000 7f5e30f64640 1 --2- 172.21.15.43:0/3554393330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2c108eb0 0x7f5e2c19efa0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.807+0000 7f5e30f64640 1 --2- 172.21.15.43:0/3554393330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e2c108a10 0x7f5e2c19ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.807+0000 7f5e30f64640 1 --2- 172.21.15.43:0/3554393330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e2c1031c0 0x7f5e2c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.807+0000 7f5e30f64640 1 -- 172.21.15.43:0/3554393330 >> 172.21.15.43:0/3554393330 conn(0x7f5e2c078110 msgr2=0x7f5e2c074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.807+0000 7f5e30f64640 1 -- 172.21.15.43:0/3554393330 shutdown_connections 2024-04-25T10:41:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:41:52.807+0000 7f5e30f64640 1 -- 172.21.15.43:0/3554393330 wait complete. 2024-04-25T10:41:52.825 INFO:tasks.cephfs.filesystem:scrub status for tag:dc987c98-d463-4a63-9cfd-9a67211d2762 - {'path': '/', 'tag': 'dc987c98-d463-4a63-9cfd-9a67211d2762', 'options': 'recursive,force'} 2024-04-25T10:41:54.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:54 smithi077 ceph-mon[32772]: pgmap v290: 97 pgs: 97 active+clean; 5.0 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-04-25T10:41:54.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:54 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:41:54.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:54 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:41:54.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:54 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:41:54.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:54 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:41:54.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:54 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:41:54.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:54 smithi164 ceph-mon[32713]: pgmap v290: 97 pgs: 97 active+clean; 5.0 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-04-25T10:41:54.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:54 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:41:54.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:54 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:41:54.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:54 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:41:54.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:54 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:41:54.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:54 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:41:54.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:54 smithi043 ceph-mon[29478]: pgmap v290: 97 pgs: 97 active+clean; 5.0 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-04-25T10:41:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:54 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:41:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:54 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:41:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:54 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:41:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:54 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:41:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:54 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:41:56.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:56 smithi077 ceph-mon[32772]: pgmap v291: 97 pgs: 97 active+clean; 5.2 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-04-25T10:41:56.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:56 smithi164 ceph-mon[32713]: pgmap v291: 97 pgs: 97 active+clean; 5.2 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-04-25T10:41:56.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:56 smithi043 ceph-mon[29478]: pgmap v291: 97 pgs: 97 active+clean; 5.2 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-04-25T10:41:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.473+0000 7fab84a4c640 1 -- 172.21.15.43:0/1019769221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab80108f00 msgr2=0x7fab80102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.473+0000 7fab84a4c640 1 --2- 172.21.15.43:0/1019769221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab80108f00 0x7fab80102b50 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fab74009900 tx=0x7fab7402a410 comp rx=0 tx=0).stop 2024-04-25T10:41:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.474+0000 7fab84a4c640 1 -- 172.21.15.43:0/1019769221 shutdown_connections 2024-04-25T10:41:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.474+0000 7fab84a4c640 1 --2- 172.21.15.43:0/1019769221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab801031c0 0x7fab8010f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.474+0000 7fab84a4c640 1 --2- 172.21.15.43:0/1019769221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab80108f00 0x7fab80102b50 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.474+0000 7fab84a4c640 1 --2- 172.21.15.43:0/1019769221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab80108a60 0x7fab80102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.474+0000 7fab84a4c640 1 -- 172.21.15.43:0/1019769221 >> 172.21.15.43:0/1019769221 conn(0x7fab800781b0 msgr2=0x7fab800785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.474+0000 7fab84a4c640 1 -- 172.21.15.43:0/1019769221 shutdown_connections 2024-04-25T10:41:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.474+0000 7fab84a4c640 1 -- 172.21.15.43:0/1019769221 wait complete. 2024-04-25T10:41:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.474+0000 7fab84a4c640 1 Processor -- start 2024-04-25T10:41:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.475+0000 7fab84a4c640 1 -- start start 2024-04-25T10:41:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.475+0000 7fab84a4c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab801031c0 0x7fab8019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.475+0000 7fab84a4c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab80108a60 0x7fab8019acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.475+0000 7fab84a4c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab80108f00 0x7fab8019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.475+0000 7fab84a4c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab80112c80 con 0x7fab801031c0 2024-04-25T10:41:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.475+0000 7fab84a4c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fab80112b00 con 0x7fab80108f00 2024-04-25T10:41:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.475+0000 7fab84a4c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab80112e00 con 0x7fab80108a60 2024-04-25T10:41:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.475+0000 7fab7e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab801031c0 0x7fab8019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.475+0000 7fab7e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab801031c0 0x7fab8019a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34994/0 (socket says 172.21.15.43:34994) 2024-04-25T10:41:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.475+0000 7fab7e59c640 1 -- 172.21.15.43:0/1016890295 learned_addr learned my addr 172.21.15.43:0/1016890295 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.475+0000 7fab7dd9b640 1 --2- 172.21.15.43:0/1016890295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab80108a60 0x7fab8019acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.475+0000 7fab7ed9d640 1 --2- 172.21.15.43:0/1016890295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab80108f00 0x7fab8019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.475+0000 7fab7e59c640 1 -- 172.21.15.43:0/1016890295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab80108a60 msgr2=0x7fab8019acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.475+0000 7fab7e59c640 1 --2- 172.21.15.43:0/1016890295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab80108a60 0x7fab8019acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.475+0000 7fab7e59c640 1 -- 172.21.15.43:0/1016890295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab80108f00 msgr2=0x7fab8019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.475+0000 7fab7e59c640 1 --2- 172.21.15.43:0/1016890295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab80108f00 0x7fab8019ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.475+0000 7fab7e59c640 1 -- 172.21.15.43:0/1016890295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8019f5b0 con 0x7fab801031c0 2024-04-25T10:41:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.476+0000 7fab7e59c640 1 --2- 172.21.15.43:0/1016890295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab801031c0 0x7fab8019a750 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fab7000d740 tx=0x7fab7000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.476+0000 7fab6f7fe640 1 -- 172.21.15.43:0/1016890295 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70017070 con 0x7fab801031c0 2024-04-25T10:41:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.476+0000 7fab6f7fe640 1 -- 172.21.15.43:0/1016890295 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab7000de50 con 0x7fab801031c0 2024-04-25T10:41:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.476+0000 7fab84a4c640 1 -- 172.21.15.43:0/1016890295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8019f8a0 con 0x7fab801031c0 2024-04-25T10:41:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.476+0000 7fab6f7fe640 1 -- 172.21.15.43:0/1016890295 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7000f620 con 0x7fab801031c0 2024-04-25T10:41:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.476+0000 7fab84a4c640 1 -- 172.21.15.43:0/1016890295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab801a7220 con 0x7fab801031c0 2024-04-25T10:41:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.478+0000 7fab6f7fe640 1 -- 172.21.15.43:0/1016890295 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fab7001a070 con 0x7fab801031c0 2024-04-25T10:41:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.478+0000 7fab8b9d9740 1 Processor -- start 2024-04-25T10:41:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.479+0000 7fab8b9d9740 1 -- start start 2024-04-25T10:41:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.479+0000 7fab8b9d9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55570f0cc1b0 0x55570f46baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.479+0000 7fab8b9d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0a32f0 0x55570f46c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.479+0000 7fab8b9d9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 0x55570f46c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.479+0000 7fab8b9d9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55570f3d2460 con 0x55570f0ab4d0 2024-04-25T10:41:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.479+0000 7fab8b9d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55570f38c590 con 0x55570f0a32f0 2024-04-25T10:41:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.479+0000 7fab8b9d9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55570f3ce420 con 0x55570f0cc1b0 2024-04-25T10:41:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.479+0000 7fab7dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 0x55570f46c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.479+0000 7fab7dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 0x55570f46c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34996/0 (socket says 172.21.15.43:34996) 2024-04-25T10:41:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.479+0000 7fab7dd9b640 1 -- 172.21.15.43:0/3414801933 learned_addr learned my addr 172.21.15.43:0/3414801933 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.479+0000 7fab6f7fe640 1 --2- 172.21.15.43:0/1016890295 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fab680afc50 0x7fab680b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.479+0000 7fab7ed9d640 1 --2- 172.21.15.43:0/3414801933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55570f0cc1b0 0x55570f46baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.479+0000 7fab7e59c640 1 --2- 172.21.15.43:0/3414801933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0a32f0 0x55570f46c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.480+0000 7fab7dd9b640 1 -- 172.21.15.43:0/3414801933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55570f0cc1b0 msgr2=0x55570f46baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.480+0000 7fab7dd9b640 1 --2- 172.21.15.43:0/3414801933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55570f0cc1b0 0x55570f46baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.480+0000 7fab7ed9d640 1 --2- 172.21.15.43:0/1016890295 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fab680afc50 0x7fab680b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.480+0000 7fab7dd9b640 1 -- 172.21.15.43:0/3414801933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0a32f0 msgr2=0x55570f46c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.480+0000 7fab7dd9b640 1 --2- 172.21.15.43:0/3414801933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0a32f0 0x55570f46c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.480+0000 7fab7dd9b640 1 -- 172.21.15.43:0/3414801933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55570f3990d0 con 0x55570f0ab4d0 2024-04-25T10:41:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.480+0000 7fab7e59c640 1 --2- 172.21.15.43:0/3414801933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0a32f0 0x55570f46c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.480+0000 7fab6f7fe640 1 -- 172.21.15.43:0/1016890295 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fab700deaf0 con 0x7fab801031c0 2024-04-25T10:41:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.480+0000 7fab7dd9b640 1 --2- 172.21.15.43:0/3414801933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 0x55570f46c570 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fab74006010 tx=0x7fab74032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4216ad31b52faf32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.480+0000 7fab7ed9d640 1 --2- 172.21.15.43:0/1016890295 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fab680afc50 0x7fab680b2160 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55570f468c60 tx=0x7fab78004ab0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.480+0000 7fab6cff9640 1 -- 172.21.15.43:0/3414801933 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7403f070 con 0x55570f0ab4d0 2024-04-25T10:41:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.480+0000 7fab6cff9640 1 -- 172.21.15.43:0/3414801933 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab74004030 con 0x55570f0ab4d0 2024-04-25T10:41:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.480+0000 7fab6cff9640 1 -- 172.21.15.43:0/3414801933 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74035070 con 0x55570f0ab4d0 2024-04-25T10:41:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.481+0000 7fab8b9d9740 1 -- 172.21.15.43:0/3414801933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 msgr2=0x55570f46c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.481+0000 7fab8b9d9740 1 --2- 172.21.15.43:0/3414801933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 0x55570f46c570 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fab74006010 tx=0x7fab74032b90 comp rx=0 tx=0).stop 2024-04-25T10:41:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.481+0000 7fab8b9d9740 1 -- 172.21.15.43:0/3414801933 shutdown_connections 2024-04-25T10:41:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.481+0000 7fab8b9d9740 1 --2- 172.21.15.43:0/3414801933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0a32f0 0x55570f46c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.481+0000 7fab8b9d9740 1 --2- 172.21.15.43:0/3414801933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 0x55570f46c570 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.481+0000 7fab8b9d9740 1 --2- 172.21.15.43:0/3414801933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55570f0cc1b0 0x55570f46baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.481+0000 7fab8b9d9740 1 -- 172.21.15.43:0/3414801933 >> 172.21.15.43:0/3414801933 conn(0x55570f0c0300 msgr2=0x55570f4682a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.481+0000 7fab8b9d9740 1 -- 172.21.15.43:0/3414801933 shutdown_connections 2024-04-25T10:41:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.481+0000 7fab8b9d9740 1 -- 172.21.15.43:0/3414801933 wait complete. 2024-04-25T10:41:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.482+0000 7fab8b9d9740 1 -- start start 2024-04-25T10:41:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.482+0000 7fab8b9d9740 1 Processor -- start 2024-04-25T10:41:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.482+0000 7fab8b9d9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55570f0a32f0 0x55570f476110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.482+0000 7fab8b9d9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 0x55570f476650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.482+0000 7fab8b9d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0cc1b0 0x55570f482070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.482+0000 7fab8b9d9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55570f478bc0 con 0x55570f0ab4d0 2024-04-25T10:41:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.482+0000 7fab8b9d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55570f359b60 con 0x55570f0cc1b0 2024-04-25T10:41:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.482+0000 7fab8b9d9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55570f478d40 con 0x55570f0a32f0 2024-04-25T10:41:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.482+0000 7fab8b9d9740 10 client.0 _ll_register_callbacks cb 0x55570f37e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.482+0000 7fab7dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 0x55570f476650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.482+0000 7fab7dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 0x55570f476650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35012/0 (socket says 172.21.15.43:35012) 2024-04-25T10:41:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.482+0000 7fab7dd9b640 1 -- 172.21.15.43:0/1657523863 learned_addr learned my addr 172.21.15.43:0/1657523863 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.482+0000 7fab7ed9d640 1 --2- 172.21.15.43:0/1657523863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0cc1b0 0x55570f482070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.482+0000 7fab7e59c640 1 --2- 172.21.15.43:0/1657523863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55570f0a32f0 0x55570f476110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.483+0000 7fab7dd9b640 1 -- 172.21.15.43:0/1657523863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55570f0a32f0 msgr2=0x55570f476110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.483+0000 7fab7dd9b640 1 --2- 172.21.15.43:0/1657523863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55570f0a32f0 0x55570f476110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.483+0000 7fab7dd9b640 1 -- 172.21.15.43:0/1657523863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0cc1b0 msgr2=0x55570f482070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.483+0000 7fab7dd9b640 1 --2- 172.21.15.43:0/1657523863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0cc1b0 0x55570f482070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.483+0000 7fab7dd9b640 1 -- 172.21.15.43:0/1657523863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55570f0e4210 con 0x55570f0ab4d0 2024-04-25T10:41:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.483+0000 7fab7dd9b640 1 --2- 172.21.15.43:0/1657523863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 0x55570f476650 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fab74009900 tx=0x7fab74004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c785a6ac10ef8ede server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.483+0000 7fab65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.483+0000 7fab65ffb640 1 -- 172.21.15.43:0/1657523863 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74035030 con 0x55570f0ab4d0 2024-04-25T10:41:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.483+0000 7fab65ffb640 1 -- 172.21.15.43:0/1657523863 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fab740314a0 con 0x55570f0ab4d0 2024-04-25T10:41:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.483+0000 7fab8b9d9740 1 -- 172.21.15.43:0/1657523863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab74004030 con 0x55570f0ab4d0 2024-04-25T10:41:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.483+0000 7fab65ffb640 1 -- 172.21.15.43:0/1657523863 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fab74055050 con 0x55570f0ab4d0 2024-04-25T10:41:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.483+0000 7fab8b9d9740 10 client.18330 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.483+0000 7fab8b9d9740 1 -- 172.21.15.43:0/1657523863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55570f1b0620 con 0x55570f0ab4d0 2024-04-25T10:41:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.483+0000 7fab65ffb640 1 -- 172.21.15.43:0/1657523863 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab74004030 con 0x55570f0ab4d0 2024-04-25T10:41:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.484+0000 7fab65ffb640 1 -- 172.21.15.43:0/1657523863 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fab740046c0 con 0x55570f0ab4d0 2024-04-25T10:41:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.484+0000 7fab8b9d9740 10 client.18330 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.484+0000 7fab8b9d9740 10 client.18330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.484+0000 7fab8b9d9740 20 client.18330 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.484+0000 7fab8b9d9740 1 --2- 172.21.15.43:0/1657523863 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55570f08ad80 0x55570f488450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.484+0000 7fab8b9d9740 4 client.18330 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:41:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.484+0000 7fab8b9d9740 1 -- 172.21.15.43:0/1657523863 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55570f3e0070 con 0x55570f08ad80 2024-04-25T10:41:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.484+0000 7fab7e59c640 1 --2- 172.21.15.43:0/1657523863 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55570f08ad80 0x55570f488450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.484+0000 7fab7e59c640 1 --2- 172.21.15.43:0/1657523863 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55570f08ad80 0x55570f488450 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c9086375676ea26 server_cookie=95daccbe48070fd3 in_seq=0 out_seq=0 2024-04-25T10:41:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.484+0000 7fab65ffb640 10 client.18330 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.485+0000 7fab65ffb640 1 -- 172.21.15.43:0/1657523863 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55570f3e0070 con 0x55570f08ad80 2024-04-25T10:41:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.485+0000 7fab65ffb640 10 client.18330 handle_command_reply: tid=0 2024-04-25T10:41:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.485+0000 7fab8b9d9740 1 client.18330 shutdown 2024-04-25T10:41:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.485+0000 7fab8b9d9740 1 -- 172.21.15.43:0/1657523863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 msgr2=0x55570f476650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.485+0000 7fab8b9d9740 1 --2- 172.21.15.43:0/1657523863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 0x55570f476650 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fab74009900 tx=0x7fab74004490 comp rx=0 tx=0).stop 2024-04-25T10:41:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.485+0000 7fab8b9d9740 1 -- 172.21.15.43:0/1657523863 shutdown_connections 2024-04-25T10:41:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.485+0000 7fab8b9d9740 1 --2- 172.21.15.43:0/1657523863 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55570f08ad80 0x55570f488450 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.485+0000 7fab8b9d9740 1 --2- 172.21.15.43:0/1657523863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0cc1b0 0x55570f482070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.485+0000 7fab8b9d9740 1 --2- 172.21.15.43:0/1657523863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 0x55570f476650 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.485+0000 7fab65ffb640 0 client.18330 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.485+0000 7fab8b9d9740 1 --2- 172.21.15.43:0/1657523863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55570f0a32f0 0x55570f476110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.485+0000 7fab8b9d9740 1 -- 172.21.15.43:0/1657523863 >> 172.21.15.43:0/1657523863 conn(0x55570f0c0300 msgr2=0x55570f46d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.485+0000 7fab8b9d9740 1 -- 172.21.15.43:0/1657523863 shutdown_connections 2024-04-25T10:41:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.485+0000 7fab8b9d9740 1 -- 172.21.15.43:0/1657523863 wait complete. 2024-04-25T10:41:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.486+0000 7fab8b9d9740 20 client.18330 trim_cache size 0 max 16384 2024-04-25T10:41:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.501+0000 7fab8b9d9740 1 Processor -- start 2024-04-25T10:41:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.501+0000 7fab8b9d9740 1 -- start start 2024-04-25T10:41:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.501+0000 7fab8b9d9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55570f0a32f0 0x55570f46dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.501+0000 7fab8b9d9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 0x55570f481fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.501+0000 7fab8b9d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0cc1b0 0x55570f46c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.501+0000 7fab8b9d9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab801b2270 con 0x55570f0ab4d0 2024-04-25T10:41:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.501+0000 7fab8b9d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fab801afb40 con 0x55570f0cc1b0 2024-04-25T10:41:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.501+0000 7fab8b9d9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab801ad9b0 con 0x55570f0a32f0 2024-04-25T10:41:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.501+0000 7fab7dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 0x55570f481fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.501+0000 7fab7dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 0x55570f481fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35028/0 (socket says 172.21.15.43:35028) 2024-04-25T10:41:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.501+0000 7fab7dd9b640 1 -- 172.21.15.43:0/4229676512 learned_addr learned my addr 172.21.15.43:0/4229676512 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.501+0000 7fab7e59c640 1 --2- 172.21.15.43:0/4229676512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55570f0a32f0 0x55570f46dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.501+0000 7fab7ed9d640 1 --2- 172.21.15.43:0/4229676512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0cc1b0 0x55570f46c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.502+0000 7fab7dd9b640 1 -- 172.21.15.43:0/4229676512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55570f0a32f0 msgr2=0x55570f46dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.502+0000 7fab7dd9b640 1 --2- 172.21.15.43:0/4229676512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55570f0a32f0 0x55570f46dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.502+0000 7fab7dd9b640 1 -- 172.21.15.43:0/4229676512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0cc1b0 msgr2=0x55570f46c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.502+0000 7fab7dd9b640 1 --2- 172.21.15.43:0/4229676512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0cc1b0 0x55570f46c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.502+0000 7fab7dd9b640 1 -- 172.21.15.43:0/4229676512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55570f474450 con 0x55570f0ab4d0 2024-04-25T10:41:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.502+0000 7fab7dd9b640 1 --2- 172.21.15.43:0/4229676512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 0x55570f481fe0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fab74005f70 tx=0x7fab74031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e032e003575a01f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.502+0000 7fab67fff640 1 -- 172.21.15.43:0/4229676512 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7405e070 con 0x55570f0ab4d0 2024-04-25T10:41:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.502+0000 7fab67fff640 1 -- 172.21.15.43:0/4229676512 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab7404c070 con 0x55570f0ab4d0 2024-04-25T10:41:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.502+0000 7fab67fff640 1 -- 172.21.15.43:0/4229676512 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74035070 con 0x55570f0ab4d0 2024-04-25T10:41:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.503+0000 7fab8b9d9740 1 -- 172.21.15.43:0/4229676512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 msgr2=0x55570f481fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.503+0000 7fab8b9d9740 1 --2- 172.21.15.43:0/4229676512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 0x55570f481fe0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fab74005f70 tx=0x7fab74031400 comp rx=0 tx=0).stop 2024-04-25T10:41:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.503+0000 7fab8b9d9740 1 -- 172.21.15.43:0/4229676512 shutdown_connections 2024-04-25T10:41:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.503+0000 7fab8b9d9740 1 --2- 172.21.15.43:0/4229676512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0cc1b0 0x55570f46c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.503+0000 7fab8b9d9740 1 --2- 172.21.15.43:0/4229676512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0ab4d0 0x55570f481fe0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.503+0000 7fab8b9d9740 1 --2- 172.21.15.43:0/4229676512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55570f0a32f0 0x55570f46dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.503+0000 7fab8b9d9740 1 -- 172.21.15.43:0/4229676512 >> 172.21.15.43:0/4229676512 conn(0x55570f0c0300 msgr2=0x55570f477620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.503+0000 7fab8b9d9740 1 -- 172.21.15.43:0/4229676512 shutdown_connections 2024-04-25T10:41:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.503+0000 7fab8b9d9740 1 -- 172.21.15.43:0/4229676512 wait complete. 2024-04-25T10:41:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.503+0000 7fab8b9d9740 1 -- start start 2024-04-25T10:41:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.503+0000 7fab8b9d9740 1 Processor -- start 2024-04-25T10:41:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.503+0000 7fab8b9d9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0a32f0 0x55570f48b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.503+0000 7fab8b9d9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55570f0ab4d0 0x55570f488a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.503+0000 7fab8b9d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0cc1b0 0x55570f488f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.503+0000 7fab8b9d9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab801ae000 con 0x55570f0a32f0 2024-04-25T10:41:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.503+0000 7fab8b9d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fab801af470 con 0x55570f0cc1b0 2024-04-25T10:41:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.503+0000 7fab8b9d9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab801ae6d0 con 0x55570f0ab4d0 2024-04-25T10:41:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.504+0000 7fab8b9d9740 10 client.0 _ll_register_callbacks cb 0x7fab78009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:41:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.504+0000 7fab7e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0a32f0 0x55570f48b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.504+0000 7fab7e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0a32f0 0x55570f48b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35030/0 (socket says 172.21.15.43:35030) 2024-04-25T10:41:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.504+0000 7fab7e59c640 1 -- 172.21.15.43:0/636609710 learned_addr learned my addr 172.21.15.43:0/636609710 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:41:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.504+0000 7fab7dd9b640 1 --2- 172.21.15.43:0/636609710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55570f0ab4d0 0x55570f488a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.504+0000 7fab7ed9d640 1 --2- 172.21.15.43:0/636609710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0cc1b0 0x55570f488f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.504+0000 7fab7e59c640 1 -- 172.21.15.43:0/636609710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55570f0ab4d0 msgr2=0x55570f488a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.504+0000 7fab7e59c640 1 --2- 172.21.15.43:0/636609710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55570f0ab4d0 0x55570f488a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.504+0000 7fab7e59c640 1 -- 172.21.15.43:0/636609710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0cc1b0 msgr2=0x55570f488f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:41:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.504+0000 7fab7e59c640 1 --2- 172.21.15.43:0/636609710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0cc1b0 0x55570f488f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.504+0000 7fab7e59c640 1 -- 172.21.15.43:0/636609710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55570f4894c0 con 0x55570f0a32f0 2024-04-25T10:41:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.504+0000 7fab7ed9d640 1 --2- 172.21.15.43:0/636609710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0cc1b0 0x55570f488f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:41:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.504+0000 7fab7e59c640 1 --2- 172.21.15.43:0/636609710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0a32f0 0x55570f48b6d0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fab70009610 tx=0x7fab700decf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14523ccff50f2df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:41:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.504+0000 7fab65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:41:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.504+0000 7fab65ffb640 1 -- 172.21.15.43:0/636609710 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70021040 con 0x55570f0a32f0 2024-04-25T10:41:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.504+0000 7fab65ffb640 1 -- 172.21.15.43:0/636609710 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fab700a2f20 con 0x55570f0a32f0 2024-04-25T10:41:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.504+0000 7fab8b9d9740 1 -- 172.21.15.43:0/636609710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab7404c070 con 0x55570f0a32f0 2024-04-25T10:41:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.504+0000 7fab65ffb640 1 -- 172.21.15.43:0/636609710 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fab700c1360 con 0x55570f0a32f0 2024-04-25T10:41:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.504+0000 7fab8b9d9740 10 client.18336 fetch_fsmap learned FSMap version 26 2024-04-25T10:41:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.504+0000 7fab8b9d9740 1 -- 172.21.15.43:0/636609710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55570f4f0150 con 0x55570f0a32f0 2024-04-25T10:41:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.505+0000 7fab65ffb640 1 -- 172.21.15.43:0/636609710 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab7404c070 con 0x55570f0a32f0 2024-04-25T10:41:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.505+0000 7fab65ffb640 1 -- 172.21.15.43:0/636609710 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fab700a2020 con 0x55570f0a32f0 2024-04-25T10:41:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.505+0000 7fab8b9d9740 10 client.18336 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:41:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.505+0000 7fab8b9d9740 10 client.18336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:41:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.505+0000 7fab8b9d9740 20 client.18336 populate_metadata read hostname 'smithi043' 2024-04-25T10:41:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.505+0000 7fab8b9d9740 1 --2- 172.21.15.43:0/636609710 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55570f0601c0 0x55570f4f2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:41:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.505+0000 7fab8b9d9740 4 client.18336 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:41:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.505+0000 7fab8b9d9740 1 -- 172.21.15.43:0/636609710 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55570f3d9ee0 con 0x55570f0601c0 2024-04-25T10:41:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.505+0000 7fab7dd9b640 1 --2- 172.21.15.43:0/636609710 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55570f0601c0 0x55570f4f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:41:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.506+0000 7fab7dd9b640 1 --2- 172.21.15.43:0/636609710 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55570f0601c0 0x55570f4f2890 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59d0663a9a8f20c8 server_cookie=17f30b304afb67bd in_seq=0 out_seq=0 2024-04-25T10:41:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.506+0000 7fab65ffb640 10 client.18336 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.520+0000 7fab65ffb640 1 -- 172.21.15.43:0/636609710 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55570f3d9ee0 con 0x55570f0601c0 2024-04-25T10:41:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.520+0000 7fab65ffb640 10 client.18336 handle_command_reply: tid=0 2024-04-25T10:41:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.520+0000 7fab8b9d9740 1 client.18336 shutdown 2024-04-25T10:41:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.520+0000 7fab8b9d9740 1 -- 172.21.15.43:0/636609710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0a32f0 msgr2=0x55570f48b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.520+0000 7fab8b9d9740 1 --2- 172.21.15.43:0/636609710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0a32f0 0x55570f48b6d0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fab70009610 tx=0x7fab700decf0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.520+0000 7fab8b9d9740 1 -- 172.21.15.43:0/636609710 shutdown_connections 2024-04-25T10:41:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.520+0000 7fab8b9d9740 1 --2- 172.21.15.43:0/636609710 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55570f0601c0 0x55570f4f2890 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.520+0000 7fab8b9d9740 1 --2- 172.21.15.43:0/636609710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55570f0cc1b0 0x55570f488f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.520+0000 7fab8b9d9740 1 --2- 172.21.15.43:0/636609710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55570f0ab4d0 0x55570f488a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.520+0000 7fab8b9d9740 1 --2- 172.21.15.43:0/636609710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55570f0a32f0 0x55570f48b6d0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.520+0000 7fab8b9d9740 1 -- 172.21.15.43:0/636609710 >> 172.21.15.43:0/636609710 conn(0x55570f0c0300 msgr2=0x55570f46dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.520+0000 7fab65ffb640 0 client.18336 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:41:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.520+0000 7fab8b9d9740 1 -- 172.21.15.43:0/636609710 shutdown_connections 2024-04-25T10:41:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.520+0000 7fab8b9d9740 1 -- 172.21.15.43:0/636609710 wait complete. 2024-04-25T10:41:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.521+0000 7fab8b9d9740 20 client.18336 trim_cache size 0 max 16384 2024-04-25T10:41:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:41:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:41:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:41:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:41:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.521+0000 7fab84a4c640 1 -- 172.21.15.43:0/1016890295 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fab680afc50 msgr2=0x7fab680b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.521+0000 7fab84a4c640 1 --2- 172.21.15.43:0/1016890295 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fab680afc50 0x7fab680b2160 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55570f468c60 tx=0x7fab78004ab0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.521+0000 7fab84a4c640 1 -- 172.21.15.43:0/1016890295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab801031c0 msgr2=0x7fab8019a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:41:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.521+0000 7fab84a4c640 1 --2- 172.21.15.43:0/1016890295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab801031c0 0x7fab8019a750 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fab7000d740 tx=0x7fab7000dc00 comp rx=0 tx=0).stop 2024-04-25T10:41:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.521+0000 7fab84a4c640 1 -- 172.21.15.43:0/1016890295 shutdown_connections 2024-04-25T10:41:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.521+0000 7fab84a4c640 1 --2- 172.21.15.43:0/1016890295 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fab680afc50 0x7fab680b2160 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.521+0000 7fab84a4c640 1 --2- 172.21.15.43:0/1016890295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab80108f00 0x7fab8019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.521+0000 7fab84a4c640 1 --2- 172.21.15.43:0/1016890295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab80108a60 0x7fab8019acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.521+0000 7fab84a4c640 1 --2- 172.21.15.43:0/1016890295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab801031c0 0x7fab8019a750 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:41:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.521+0000 7fab84a4c640 1 -- 172.21.15.43:0/1016890295 >> 172.21.15.43:0/1016890295 conn(0x7fab800781b0 msgr2=0x7fab8006a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:41:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.522+0000 7fab84a4c640 1 -- 172.21.15.43:0/1016890295 shutdown_connections 2024-04-25T10:41:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:41:56.522+0000 7fab84a4c640 1 -- 172.21.15.43:0/1016890295 wait complete. 2024-04-25T10:41:58.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:41:58 smithi043 ceph-mon[29478]: pgmap v292: 97 pgs: 97 active+clean; 5.2 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-25T10:41:58.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:41:58 smithi077 ceph-mon[32772]: pgmap v292: 97 pgs: 97 active+clean; 5.2 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-25T10:41:58.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:41:58 smithi164 ceph-mon[32713]: pgmap v292: 97 pgs: 97 active+clean; 5.2 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-25T10:42:00.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:00 smithi043 ceph-mon[29478]: pgmap v293: 97 pgs: 97 active+clean; 1.4 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-04-25T10:42:00.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:00 smithi077 ceph-mon[32772]: pgmap v293: 97 pgs: 97 active+clean; 1.4 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-04-25T10:42:00.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:00 smithi164 ceph-mon[32713]: pgmap v293: 97 pgs: 97 active+clean; 1.4 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-04-25T10:42:02.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:02 smithi043 ceph-mon[29478]: pgmap v294: 97 pgs: 97 active+clean; 1.4 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-04-25T10:42:02.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:02 smithi077 ceph-mon[32772]: pgmap v294: 97 pgs: 97 active+clean; 1.4 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-04-25T10:42:02.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:02 smithi164 ceph-mon[32713]: pgmap v294: 97 pgs: 97 active+clean; 1.4 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-04-25T10:42:04.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:04 smithi043 ceph-mon[29478]: pgmap v295: 97 pgs: 97 active+clean; 1.4 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-04-25T10:42:04.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:04 smithi077 ceph-mon[32772]: pgmap v295: 97 pgs: 97 active+clean; 1.4 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-04-25T10:42:04.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:04 smithi164 ceph-mon[32713]: pgmap v295: 97 pgs: 97 active+clean; 1.4 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-04-25T10:42:05.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:05 smithi043 ceph-mon[29478]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T10:42:05.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:05 smithi077 ceph-mon[32772]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T10:42:05.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:05 smithi164 ceph-mon[32713]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T10:42:06.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:06 smithi043 ceph-mon[29478]: pgmap v297: 97 pgs: 97 active+clean; 1.7 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 108 KiB/s wr, 24 op/s 2024-04-25T10:42:06.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:06 smithi077 ceph-mon[32772]: pgmap v297: 97 pgs: 97 active+clean; 1.7 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 108 KiB/s wr, 24 op/s 2024-04-25T10:42:06.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:06 smithi164 ceph-mon[32713]: pgmap v297: 97 pgs: 97 active+clean; 1.7 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 108 KiB/s wr, 24 op/s 2024-04-25T10:42:08.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:08 smithi043 ceph-mon[29478]: pgmap v298: 97 pgs: 97 active+clean; 1.7 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 108 KiB/s wr, 24 op/s 2024-04-25T10:42:08.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:08 smithi077 ceph-mon[32772]: pgmap v298: 97 pgs: 97 active+clean; 1.7 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 108 KiB/s wr, 24 op/s 2024-04-25T10:42:08.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:08 smithi164 ceph-mon[32713]: pgmap v298: 97 pgs: 97 active+clean; 1.7 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 108 KiB/s wr, 24 op/s 2024-04-25T10:42:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.633+0000 7f8c0a692640 1 -- 172.21.15.43:0/1306504025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c041042b0 msgr2=0x7f8c04110680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.633+0000 7f8c0a692640 1 --2- 172.21.15.43:0/1306504025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c041042b0 0x7f8c04110680 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009ca0 tx=0x7f8bfc02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.633+0000 7f8c0a692640 1 -- 172.21.15.43:0/1306504025 shutdown_connections 2024-04-25T10:42:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.633+0000 7f8c0a692640 1 --2- 172.21.15.43:0/1306504025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c041042b0 0x7f8c04110680 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.633+0000 7f8c0a692640 1 --2- 172.21.15.43:0/1306504025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c0410a000 0x7f8c04103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.633+0000 7f8c0a692640 1 --2- 172.21.15.43:0/1306504025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c04109b60 0x7f8c04103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.633+0000 7f8c0a692640 1 -- 172.21.15.43:0/1306504025 >> 172.21.15.43:0/1306504025 conn(0x7f8c04079170 msgr2=0x7f8c040795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.633+0000 7f8c0a692640 1 -- 172.21.15.43:0/1306504025 shutdown_connections 2024-04-25T10:42:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.633+0000 7f8c0a692640 1 -- 172.21.15.43:0/1306504025 wait complete. 2024-04-25T10:42:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.634+0000 7f8c0a692640 1 Processor -- start 2024-04-25T10:42:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.634+0000 7f8c0a692640 1 -- start start 2024-04-25T10:42:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.634+0000 7f8c0a692640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c041042b0 0x7f8c0419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.634+0000 7f8c0a692640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c04109b60 0x7f8c0419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.634+0000 7f8c0a692640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c0410a000 0x7f8c0419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.634+0000 7f8c0a692640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8c04113d70 con 0x7f8c04109b60 2024-04-25T10:42:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.634+0000 7f8c0a692640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8c04113bf0 con 0x7f8c041042b0 2024-04-25T10:42:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.634+0000 7f8c0a692640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8c04113ef0 con 0x7f8c0410a000 2024-04-25T10:42:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.634+0000 7f8c037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c04109b60 0x7f8c0419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.634+0000 7f8c037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c04109b60 0x7f8c0419acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52998/0 (socket says 172.21.15.43:52998) 2024-04-25T10:42:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.634+0000 7f8c037fe640 1 -- 172.21.15.43:0/4158309087 learned_addr learned my addr 172.21.15.43:0/4158309087 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.634+0000 7f8c08c2f640 1 --2- 172.21.15.43:0/4158309087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c0410a000 0x7f8c0419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.634+0000 7f8c03fff640 1 --2- 172.21.15.43:0/4158309087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c041042b0 0x7f8c0419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.635+0000 7f8c037fe640 1 -- 172.21.15.43:0/4158309087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c0410a000 msgr2=0x7f8c0419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.635+0000 7f8c037fe640 1 --2- 172.21.15.43:0/4158309087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c0410a000 0x7f8c0419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.635+0000 7f8c037fe640 1 -- 172.21.15.43:0/4158309087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c041042b0 msgr2=0x7f8c0419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.635+0000 7f8c037fe640 1 --2- 172.21.15.43:0/4158309087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c041042b0 0x7f8c0419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.635+0000 7f8c037fe640 1 -- 172.21.15.43:0/4158309087 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0419f5b0 con 0x7f8c04109b60 2024-04-25T10:42:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.635+0000 7f8c03fff640 1 --2- 172.21.15.43:0/4158309087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c041042b0 0x7f8c0419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.635+0000 7f8c08c2f640 1 --2- 172.21.15.43:0/4158309087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c0410a000 0x7f8c0419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.635+0000 7f8c037fe640 1 --2- 172.21.15.43:0/4158309087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c04109b60 0x7f8c0419acb0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8005d00 tx=0x7f8bf800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.635+0000 7f8c017fa640 1 -- 172.21.15.43:0/4158309087 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8017070 con 0x7f8c04109b60 2024-04-25T10:42:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.635+0000 7f8c0a692640 1 -- 172.21.15.43:0/4158309087 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0419f8a0 con 0x7f8c04109b60 2024-04-25T10:42:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.635+0000 7f8c017fa640 1 -- 172.21.15.43:0/4158309087 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bf8009c30 con 0x7f8c04109b60 2024-04-25T10:42:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.635+0000 7f8c017fa640 1 -- 172.21.15.43:0/4158309087 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf800f620 con 0x7f8c04109b60 2024-04-25T10:42:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.635+0000 7f8c0a692640 1 -- 172.21.15.43:0/4158309087 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c041a7220 con 0x7f8c04109b60 2024-04-25T10:42:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.638+0000 7f8c11616740 1 Processor -- start 2024-04-25T10:42:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.638+0000 7f8c017fa640 1 -- 172.21.15.43:0/4158309087 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f8bf801a070 con 0x7f8c04109b60 2024-04-25T10:42:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.638+0000 7f8c11616740 1 -- start start 2024-04-25T10:42:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.638+0000 7f8c11616740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813ebb1b0 0x55b81425aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.638+0000 7f8c11616740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813e922f0 0x55b81425b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.638+0000 7f8c11616740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e9a4d0 0x55b81425b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.638+0000 7f8c11616740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b8141c1460 con 0x55b813e9a4d0 2024-04-25T10:42:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.638+0000 7f8c11616740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b81417b590 con 0x55b813e922f0 2024-04-25T10:42:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.638+0000 7f8c11616740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b8141bd420 con 0x55b813ebb1b0 2024-04-25T10:42:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.638+0000 7f8c037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e9a4d0 0x55b81425b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.638+0000 7f8c037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e9a4d0 0x55b81425b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53000/0 (socket says 172.21.15.43:53000) 2024-04-25T10:42:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.639+0000 7f8c037fe640 1 -- 172.21.15.43:0/1250702765 learned_addr learned my addr 172.21.15.43:0/1250702765 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.639+0000 7f8c08c2f640 1 --2- 172.21.15.43:0/1250702765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813ebb1b0 0x55b81425aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.639+0000 7f8c03fff640 1 --2- 172.21.15.43:0/1250702765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813e922f0 0x55b81425b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.639+0000 7f8c017fa640 1 --2- 172.21.15.43:0/4158309087 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8bec0afc50 0x7f8bec0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.639+0000 7f8c037fe640 1 -- 172.21.15.43:0/1250702765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813ebb1b0 msgr2=0x55b81425aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.639+0000 7f8c037fe640 1 --2- 172.21.15.43:0/1250702765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813ebb1b0 0x55b81425aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.639+0000 7f8c037fe640 1 -- 172.21.15.43:0/1250702765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813e922f0 msgr2=0x55b81425b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.639+0000 7f8c037fe640 1 --2- 172.21.15.43:0/1250702765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813e922f0 0x55b81425b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.639+0000 7f8c037fe640 1 -- 172.21.15.43:0/1250702765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8141880d0 con 0x55b813e9a4d0 2024-04-25T10:42:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.639+0000 7f8c03fff640 1 --2- 172.21.15.43:0/1250702765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813e922f0 0x55b81425b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.639+0000 7f8c08c2f640 1 --2- 172.21.15.43:0/1250702765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813ebb1b0 0x55b81425aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:42:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.639+0000 7f8c08c2f640 1 --2- 172.21.15.43:0/4158309087 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8bec0afc50 0x7f8bec0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.639+0000 7f8c017fa640 1 -- 172.21.15.43:0/4158309087 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7f8bf80deb50 con 0x7f8c04109b60 2024-04-25T10:42:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.639+0000 7f8c037fe640 1 --2- 172.21.15.43:0/1250702765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e9a4d0 0x55b81425b570 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80dc1f0 tx=0x7f8bf8023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d00859b86878da0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.639+0000 7f8bea7fc640 1 -- 172.21.15.43:0/1250702765 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf80a6aa0 con 0x55b813e9a4d0 2024-04-25T10:42:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.639+0000 7f8bea7fc640 1 -- 172.21.15.43:0/1250702765 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bf80e2030 con 0x55b813e9a4d0 2024-04-25T10:42:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.639+0000 7f8bea7fc640 1 -- 172.21.15.43:0/1250702765 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8015110 con 0x55b813e9a4d0 2024-04-25T10:42:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.640+0000 7f8c11616740 1 -- 172.21.15.43:0/1250702765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e9a4d0 msgr2=0x55b81425b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.640+0000 7f8c11616740 1 --2- 172.21.15.43:0/1250702765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e9a4d0 0x55b81425b570 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80dc1f0 tx=0x7f8bf8023000 comp rx=0 tx=0).stop 2024-04-25T10:42:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.640+0000 7f8c08c2f640 1 --2- 172.21.15.43:0/4158309087 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8bec0afc50 0x7f8bec0b2160 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc004af0 tx=0x7f8bfc0095c0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.640+0000 7f8c11616740 1 -- 172.21.15.43:0/1250702765 shutdown_connections 2024-04-25T10:42:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.640+0000 7f8c11616740 1 --2- 172.21.15.43:0/1250702765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813e922f0 0x55b81425b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.640+0000 7f8c11616740 1 --2- 172.21.15.43:0/1250702765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e9a4d0 0x55b81425b570 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.640+0000 7f8c11616740 1 --2- 172.21.15.43:0/1250702765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813ebb1b0 0x55b81425aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.640+0000 7f8c11616740 1 -- 172.21.15.43:0/1250702765 >> 172.21.15.43:0/1250702765 conn(0x55b813eaf300 msgr2=0x55b8142572a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.640+0000 7f8c11616740 1 -- 172.21.15.43:0/1250702765 shutdown_connections 2024-04-25T10:42:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.640+0000 7f8c11616740 1 -- 172.21.15.43:0/1250702765 wait complete. 2024-04-25T10:42:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.641+0000 7f8c11616740 1 -- start start 2024-04-25T10:42:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.641+0000 7f8c11616740 1 Processor -- start 2024-04-25T10:42:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.641+0000 7f8c11616740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813e922f0 0x55b8142651b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.641+0000 7f8c11616740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813e9a4d0 0x55b814265710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.641+0000 7f8c11616740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813ebb1b0 0x55b81426b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.641+0000 7f8c11616740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b814267b60 con 0x55b813ebb1b0 2024-04-25T10:42:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.641+0000 7f8c11616740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b814148b60 con 0x55b813e9a4d0 2024-04-25T10:42:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.641+0000 7f8c11616740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b814267ce0 con 0x55b813e922f0 2024-04-25T10:42:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.641+0000 7f8c11616740 10 client.0 _ll_register_callbacks cb 0x55b81416d2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.642+0000 7f8c08c2f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813ebb1b0 0x55b81426b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.642+0000 7f8c08c2f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813ebb1b0 0x55b81426b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53008/0 (socket says 172.21.15.43:53008) 2024-04-25T10:42:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.642+0000 7f8c08c2f640 1 -- 172.21.15.43:0/824556622 learned_addr learned my addr 172.21.15.43:0/824556622 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.642+0000 7f8c037fe640 1 --2- 172.21.15.43:0/824556622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813e9a4d0 0x55b814265710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.642+0000 7f8c03fff640 1 --2- 172.21.15.43:0/824556622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813e922f0 0x55b8142651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.642+0000 7f8c08c2f640 1 -- 172.21.15.43:0/824556622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813e922f0 msgr2=0x55b8142651b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.642+0000 7f8c08c2f640 1 --2- 172.21.15.43:0/824556622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813e922f0 0x55b8142651b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.642+0000 7f8c08c2f640 1 -- 172.21.15.43:0/824556622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813e9a4d0 msgr2=0x55b814265710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.642+0000 7f8c08c2f640 1 --2- 172.21.15.43:0/824556622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813e9a4d0 0x55b814265710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.642+0000 7f8c08c2f640 1 -- 172.21.15.43:0/824556622 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b813ed3210 con 0x55b813ebb1b0 2024-04-25T10:42:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.642+0000 7f8c08c2f640 1 --2- 172.21.15.43:0/824556622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813ebb1b0 0x55b81426b050 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc035000 tx=0x7f8bfc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd54cc619f1d1052 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.642+0000 7f8be37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.642+0000 7f8be37fe640 1 -- 172.21.15.43:0/824556622 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc004240 con 0x55b813ebb1b0 2024-04-25T10:42:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.642+0000 7f8be37fe640 1 -- 172.21.15.43:0/824556622 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7f8bfc0043e0 con 0x55b813ebb1b0 2024-04-25T10:42:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.642+0000 7f8c11616740 1 -- 172.21.15.43:0/824556622 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b8141cf070 con 0x55b813ebb1b0 2024-04-25T10:42:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.643+0000 7f8be37fe640 1 -- 172.21.15.43:0/824556622 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8bfc0045e0 con 0x55b813ebb1b0 2024-04-25T10:42:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.643+0000 7f8c11616740 10 client.18348 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.643+0000 7f8c11616740 1 -- 172.21.15.43:0/824556622 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b813f9f620 con 0x55b813ebb1b0 2024-04-25T10:42:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.643+0000 7f8be37fe640 1 -- 172.21.15.43:0/824556622 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b8141cf070 con 0x55b813ebb1b0 2024-04-25T10:42:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.643+0000 7f8be37fe640 1 -- 172.21.15.43:0/824556622 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8bfc0305c0 con 0x55b813ebb1b0 2024-04-25T10:42:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.643+0000 7f8c11616740 10 client.18348 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.643+0000 7f8c11616740 10 client.18348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.643+0000 7f8c11616740 20 client.18348 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.643+0000 7f8c11616740 1 --2- 172.21.15.43:0/824556622 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b813e79d80 0x55b8142776e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.643+0000 7f8c11616740 4 client.18348 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:42:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.643+0000 7f8c11616740 1 -- 172.21.15.43:0/824556622 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8141c8ee0 con 0x55b813e79d80 2024-04-25T10:42:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.643+0000 7f8c03fff640 1 --2- 172.21.15.43:0/824556622 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b813e79d80 0x55b8142776e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.644+0000 7f8be37fe640 10 client.18348 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.644+0000 7f8c03fff640 1 --2- 172.21.15.43:0/824556622 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b813e79d80 0x55b8142776e0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43ce199690acb527 server_cookie=aa90ebe976a0b27 in_seq=0 out_seq=0 2024-04-25T10:42:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.644+0000 7f8be37fe640 1 -- 172.21.15.43:0/824556622 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b8141c8ee0 con 0x55b813e79d80 2024-04-25T10:42:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.644+0000 7f8be37fe640 10 client.18348 handle_command_reply: tid=0 2024-04-25T10:42:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.645+0000 7f8c11616740 1 client.18348 shutdown 2024-04-25T10:42:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.645+0000 7f8c11616740 1 -- 172.21.15.43:0/824556622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813ebb1b0 msgr2=0x55b81426b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.645+0000 7f8c11616740 1 --2- 172.21.15.43:0/824556622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813ebb1b0 0x55b81426b050 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc035000 tx=0x7f8bfc02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.645+0000 7f8c11616740 1 -- 172.21.15.43:0/824556622 shutdown_connections 2024-04-25T10:42:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.645+0000 7f8c11616740 1 --2- 172.21.15.43:0/824556622 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b813e79d80 0x55b8142776e0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.645+0000 7f8c11616740 1 --2- 172.21.15.43:0/824556622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813e9a4d0 0x55b814265710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.645+0000 7f8c11616740 1 --2- 172.21.15.43:0/824556622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813ebb1b0 0x55b81426b050 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.645+0000 7f8be37fe640 0 client.18348 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.645+0000 7f8c11616740 1 --2- 172.21.15.43:0/824556622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813e922f0 0x55b8142651b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.645+0000 7f8c11616740 1 -- 172.21.15.43:0/824556622 >> 172.21.15.43:0/824556622 conn(0x55b813eaf300 msgr2=0x55b81425c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.645+0000 7f8c11616740 1 -- 172.21.15.43:0/824556622 shutdown_connections 2024-04-25T10:42:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.645+0000 7f8c11616740 1 -- 172.21.15.43:0/824556622 wait complete. 2024-04-25T10:42:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.645+0000 7f8c11616740 20 client.18348 trim_cache size 0 max 16384 2024-04-25T10:42:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.662+0000 7f8c11616740 1 Processor -- start 2024-04-25T10:42:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.662+0000 7f8c11616740 1 -- start start 2024-04-25T10:42:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.663+0000 7f8c11616740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e922f0 0x55b81426af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.663+0000 7f8c11616740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813e9a4d0 0x55b814265510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.663+0000 7f8c11616740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813ebb1b0 0x55b81425af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.663+0000 7f8c11616740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8c041b2480 con 0x55b813e922f0 2024-04-25T10:42:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.663+0000 7f8c11616740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8c041afd50 con 0x55b813e9a4d0 2024-04-25T10:42:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.663+0000 7f8c11616740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8c041adbc0 con 0x55b813ebb1b0 2024-04-25T10:42:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.663+0000 7f8c03fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e922f0 0x55b81426af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.663+0000 7f8c03fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e922f0 0x55b81426af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53012/0 (socket says 172.21.15.43:53012) 2024-04-25T10:42:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.663+0000 7f8c03fff640 1 -- 172.21.15.43:0/3174635822 learned_addr learned my addr 172.21.15.43:0/3174635822 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.663+0000 7f8c037fe640 1 --2- 172.21.15.43:0/3174635822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813e9a4d0 0x55b814265510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.663+0000 7f8c08c2f640 1 --2- 172.21.15.43:0/3174635822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813ebb1b0 0x55b81425af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.663+0000 7f8c03fff640 1 -- 172.21.15.43:0/3174635822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813ebb1b0 msgr2=0x55b81425af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.663+0000 7f8c03fff640 1 --2- 172.21.15.43:0/3174635822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813ebb1b0 0x55b81425af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.663+0000 7f8c03fff640 1 -- 172.21.15.43:0/3174635822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813e9a4d0 msgr2=0x55b814265510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.663+0000 7f8c03fff640 1 --2- 172.21.15.43:0/3174635822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813e9a4d0 0x55b814265510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.663+0000 7f8c03fff640 1 -- 172.21.15.43:0/3174635822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b814265a50 con 0x55b813e922f0 2024-04-25T10:42:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.664+0000 7f8c03fff640 1 --2- 172.21.15.43:0/3174635822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e922f0 0x55b81426af50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x55b8142585e0 tx=0x7f8bf40149b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d60f01c590bfc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.664+0000 7f8be9ffb640 1 -- 172.21.15.43:0/3174635822 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4004030 con 0x55b813e922f0 2024-04-25T10:42:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.664+0000 7f8be9ffb640 1 -- 172.21.15.43:0/3174635822 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bf40041d0 con 0x55b813e922f0 2024-04-25T10:42:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.664+0000 7f8be9ffb640 1 -- 172.21.15.43:0/3174635822 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40126b0 con 0x55b813e922f0 2024-04-25T10:42:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.665+0000 7f8c11616740 1 -- 172.21.15.43:0/3174635822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e922f0 msgr2=0x55b81426af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.665+0000 7f8c11616740 1 --2- 172.21.15.43:0/3174635822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e922f0 0x55b81426af50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x55b8142585e0 tx=0x7f8bf40149b0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.665+0000 7f8c11616740 1 -- 172.21.15.43:0/3174635822 shutdown_connections 2024-04-25T10:42:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.665+0000 7f8c11616740 1 --2- 172.21.15.43:0/3174635822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813e9a4d0 0x55b814265510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.665+0000 7f8c11616740 1 --2- 172.21.15.43:0/3174635822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813ebb1b0 0x55b81425af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.665+0000 7f8c11616740 1 --2- 172.21.15.43:0/3174635822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e922f0 0x55b81426af50 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.665+0000 7f8c11616740 1 -- 172.21.15.43:0/3174635822 >> 172.21.15.43:0/3174635822 conn(0x55b813eaf300 msgr2=0x55b81425a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.665+0000 7f8c11616740 1 -- 172.21.15.43:0/3174635822 shutdown_connections 2024-04-25T10:42:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.665+0000 7f8c11616740 1 -- 172.21.15.43:0/3174635822 wait complete. 2024-04-25T10:42:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.665+0000 7f8c11616740 1 -- start start 2024-04-25T10:42:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.665+0000 7f8c11616740 1 Processor -- start 2024-04-25T10:42:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.666+0000 7f8c11616740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813e922f0 0x55b81425e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.666+0000 7f8c11616740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e9a4d0 0x55b81425e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.666+0000 7f8c11616740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813ebb1b0 0x55b81425ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.666+0000 7f8c11616740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8c041ae210 con 0x55b813e9a4d0 2024-04-25T10:42:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.666+0000 7f8c11616740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8c041af680 con 0x55b813ebb1b0 2024-04-25T10:42:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.666+0000 7f8c11616740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8c041ae8e0 con 0x55b813e922f0 2024-04-25T10:42:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.666+0000 7f8c11616740 10 client.0 _ll_register_callbacks cb 0x7f8bfc009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.666+0000 7f8c037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e9a4d0 0x55b81425e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.666+0000 7f8c037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e9a4d0 0x55b81425e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53016/0 (socket says 172.21.15.43:53016) 2024-04-25T10:42:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.666+0000 7f8c03fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813e922f0 0x55b81425e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.666+0000 7f8c037fe640 1 -- 172.21.15.43:0/1814552598 learned_addr learned my addr 172.21.15.43:0/1814552598 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.666+0000 7f8c08c2f640 1 --2- 172.21.15.43:0/1814552598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813ebb1b0 0x55b81425ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.666+0000 7f8c037fe640 1 -- 172.21.15.43:0/1814552598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813e922f0 msgr2=0x55b81425e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.666+0000 7f8c037fe640 1 --2- 172.21.15.43:0/1814552598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813e922f0 0x55b81425e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.666+0000 7f8c037fe640 1 -- 172.21.15.43:0/1814552598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813ebb1b0 msgr2=0x55b81425ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.666+0000 7f8c037fe640 1 --2- 172.21.15.43:0/1814552598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813ebb1b0 0x55b81425ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.666+0000 7f8c037fe640 1 -- 172.21.15.43:0/1814552598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b814271ac0 con 0x55b813e9a4d0 2024-04-25T10:42:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.666+0000 7f8c037fe640 1 --2- 172.21.15.43:0/1814552598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e9a4d0 0x55b81425e680 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80dc330 tx=0x7f8bf800df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fdd8dc41596eaa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.666+0000 7f8be37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.666+0000 7f8be37fe640 1 -- 172.21.15.43:0/1814552598 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8021040 con 0x55b813e9a4d0 2024-04-25T10:42:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.667+0000 7f8be37fe640 1 -- 172.21.15.43:0/1814552598 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7f8bf80a6f30 con 0x55b813e9a4d0 2024-04-25T10:42:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.667+0000 7f8c11616740 1 -- 172.21.15.43:0/1814552598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bf40041d0 con 0x55b813e9a4d0 2024-04-25T10:42:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.667+0000 7f8be37fe640 1 -- 172.21.15.43:0/1814552598 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8bf80a6440 con 0x55b813e9a4d0 2024-04-25T10:42:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.667+0000 7f8c11616740 10 client.18354 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.667+0000 7f8c11616740 1 -- 172.21.15.43:0/1814552598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b8142dfaf0 con 0x55b813e9a4d0 2024-04-25T10:42:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.667+0000 7f8be37fe640 1 -- 172.21.15.43:0/1814552598 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bf40041d0 con 0x55b813e9a4d0 2024-04-25T10:42:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.667+0000 7f8be37fe640 1 -- 172.21.15.43:0/1814552598 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8bf80af6a0 con 0x55b813e9a4d0 2024-04-25T10:42:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.667+0000 7f8c11616740 10 client.18354 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.667+0000 7f8c11616740 10 client.18354 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.667+0000 7f8c11616740 20 client.18354 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.667+0000 7f8c11616740 1 --2- 172.21.15.43:0/1814552598 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b813e4f1c0 0x55b8142e1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.667+0000 7f8c11616740 4 client.18354 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:42:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.667+0000 7f8c11616740 1 -- 172.21.15.43:0/1814552598 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b8141445f0 con 0x55b813e4f1c0 2024-04-25T10:42:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.667+0000 7f8c03fff640 1 --2- 172.21.15.43:0/1814552598 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b813e4f1c0 0x55b8142e1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.668+0000 7f8be37fe640 10 client.18354 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.668+0000 7f8c03fff640 1 --2- 172.21.15.43:0/1814552598 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b813e4f1c0 0x55b8142e1ec0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e231f387f21275d2 server_cookie=91cceb0a01846ffd in_seq=0 out_seq=0 2024-04-25T10:42:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.683+0000 7f8be37fe640 1 -- 172.21.15.43:0/1814552598 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b8141445f0 con 0x55b813e4f1c0 2024-04-25T10:42:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.683+0000 7f8be37fe640 10 client.18354 handle_command_reply: tid=0 2024-04-25T10:42:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.683+0000 7f8c11616740 1 client.18354 shutdown 2024-04-25T10:42:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.683+0000 7f8c11616740 1 -- 172.21.15.43:0/1814552598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e9a4d0 msgr2=0x55b81425e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.683+0000 7f8c11616740 1 --2- 172.21.15.43:0/1814552598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e9a4d0 0x55b81425e680 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80dc330 tx=0x7f8bf800df70 comp rx=0 tx=0).stop 2024-04-25T10:42:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.684+0000 7f8c11616740 1 -- 172.21.15.43:0/1814552598 shutdown_connections 2024-04-25T10:42:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.684+0000 7f8c11616740 1 --2- 172.21.15.43:0/1814552598 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b813e4f1c0 0x55b8142e1ec0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.684+0000 7f8c11616740 1 --2- 172.21.15.43:0/1814552598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b813ebb1b0 0x55b81425ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.684+0000 7f8c11616740 1 --2- 172.21.15.43:0/1814552598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b813e9a4d0 0x55b81425e680 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.684+0000 7f8c11616740 1 --2- 172.21.15.43:0/1814552598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b813e922f0 0x55b81425e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.684+0000 7f8be37fe640 0 client.18354 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.684+0000 7f8c11616740 1 -- 172.21.15.43:0/1814552598 >> 172.21.15.43:0/1814552598 conn(0x55b813eaf300 msgr2=0x55b81425b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.684+0000 7f8c11616740 1 -- 172.21.15.43:0/1814552598 shutdown_connections 2024-04-25T10:42:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.684+0000 7f8c11616740 1 -- 172.21.15.43:0/1814552598 wait complete. 2024-04-25T10:42:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.684+0000 7f8c11616740 20 client.18354 trim_cache size 0 max 16384 2024-04-25T10:42:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:42:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:42:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:42:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:42:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.684+0000 7f8c0a692640 1 -- 172.21.15.43:0/4158309087 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8bec0afc50 msgr2=0x7f8bec0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.684+0000 7f8c0a692640 1 --2- 172.21.15.43:0/4158309087 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8bec0afc50 0x7f8bec0b2160 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc004af0 tx=0x7f8bfc0095c0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.684+0000 7f8c0a692640 1 -- 172.21.15.43:0/4158309087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c04109b60 msgr2=0x7f8c0419acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.684+0000 7f8c0a692640 1 --2- 172.21.15.43:0/4158309087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c04109b60 0x7f8c0419acb0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8005d00 tx=0x7f8bf800dde0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.684+0000 7f8c0a692640 1 -- 172.21.15.43:0/4158309087 shutdown_connections 2024-04-25T10:42:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.684+0000 7f8c0a692640 1 --2- 172.21.15.43:0/4158309087 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8bec0afc50 0x7f8bec0b2160 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.684+0000 7f8c0a692640 1 --2- 172.21.15.43:0/4158309087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c0410a000 0x7f8c0419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.684+0000 7f8c0a692640 1 --2- 172.21.15.43:0/4158309087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c04109b60 0x7f8c0419acb0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.684+0000 7f8c0a692640 1 --2- 172.21.15.43:0/4158309087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c041042b0 0x7f8c0419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.684+0000 7f8c0a692640 1 -- 172.21.15.43:0/4158309087 >> 172.21.15.43:0/4158309087 conn(0x7f8c04079170 msgr2=0x7f8c04100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.685+0000 7f8c0a692640 1 -- 172.21.15.43:0/4158309087 shutdown_connections 2024-04-25T10:42:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:09.685+0000 7f8c0a692640 1 -- 172.21.15.43:0/4158309087 wait complete. 2024-04-25T10:42:10.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:10 smithi043 ceph-mon[29478]: pgmap v299: 97 pgs: 97 active+clean; 1.9 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 20 op/s 2024-04-25T10:42:10.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:10 smithi077 ceph-mon[32772]: pgmap v299: 97 pgs: 97 active+clean; 1.9 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 20 op/s 2024-04-25T10:42:10.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:10 smithi164 ceph-mon[32713]: pgmap v299: 97 pgs: 97 active+clean; 1.9 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 20 op/s 2024-04-25T10:42:12.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:12 smithi043 ceph-mon[29478]: pgmap v300: 97 pgs: 97 active+clean; 1.9 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 20 op/s 2024-04-25T10:42:12.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:12 smithi077 ceph-mon[32772]: pgmap v300: 97 pgs: 97 active+clean; 1.9 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 20 op/s 2024-04-25T10:42:12.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:12 smithi164 ceph-mon[32713]: pgmap v300: 97 pgs: 97 active+clean; 1.9 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 20 op/s 2024-04-25T10:42:14.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:13 smithi077 ceph-mon[32772]: pgmap v301: 97 pgs: 97 active+clean; 1.9 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 20 op/s 2024-04-25T10:42:14.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:13 smithi077 ceph-mon[32772]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T10:42:14.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:13 smithi164 ceph-mon[32713]: pgmap v301: 97 pgs: 97 active+clean; 1.9 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 20 op/s 2024-04-25T10:42:14.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:13 smithi164 ceph-mon[32713]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T10:42:14.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:13 smithi043 ceph-mon[29478]: pgmap v301: 97 pgs: 97 active+clean; 1.9 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 20 op/s 2024-04-25T10:42:14.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:13 smithi043 ceph-mon[29478]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T10:42:16.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:15 smithi077 ceph-mon[32772]: pgmap v303: 97 pgs: 97 active+clean; 2.1 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-04-25T10:42:16.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:15 smithi164 ceph-mon[32713]: pgmap v303: 97 pgs: 97 active+clean; 2.1 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-04-25T10:42:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:15 smithi043 ceph-mon[29478]: pgmap v303: 97 pgs: 97 active+clean; 2.1 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-04-25T10:42:18.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:17 smithi077 ceph-mon[32772]: pgmap v304: 97 pgs: 97 active+clean; 2.1 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-04-25T10:42:18.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:17 smithi164 ceph-mon[32713]: pgmap v304: 97 pgs: 97 active+clean; 2.1 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-04-25T10:42:18.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:17 smithi043 ceph-mon[29478]: pgmap v304: 97 pgs: 97 active+clean; 2.1 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-04-25T10:42:20.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:19 smithi077 ceph-mon[32772]: pgmap v305: 97 pgs: 97 active+clean; 2.4 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-04-25T10:42:20.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:19 smithi164 ceph-mon[32713]: pgmap v305: 97 pgs: 97 active+clean; 2.4 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-04-25T10:42:20.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:19 smithi043 ceph-mon[29478]: pgmap v305: 97 pgs: 97 active+clean; 2.4 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-04-25T10:42:22.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:21 smithi077 ceph-mon[32772]: pgmap v306: 97 pgs: 97 active+clean; 2.4 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-04-25T10:42:22.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:21 smithi164 ceph-mon[32713]: pgmap v306: 97 pgs: 97 active+clean; 2.4 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-04-25T10:42:22.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:21 smithi043 ceph-mon[29478]: pgmap v306: 97 pgs: 97 active+clean; 2.4 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-04-25T10:42:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.795+0000 7f0764c97640 1 -- 172.21.15.43:0/599055128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f076010b730 msgr2=0x7f076010db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.795+0000 7f0764c97640 1 --2- 172.21.15.43:0/599055128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f076010b730 0x7f076010db50 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f0748009900 tx=0x7f074802a510 comp rx=0 tx=0).stop 2024-04-25T10:42:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.795+0000 7f0764c97640 1 -- 172.21.15.43:0/599055128 shutdown_connections 2024-04-25T10:42:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.795+0000 7f0764c97640 1 --2- 172.21.15.43:0/599055128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f076010e0e0 0x7f07601105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.795+0000 7f0764c97640 1 --2- 172.21.15.43:0/599055128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f076010b730 0x7f076010db50 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.795+0000 7f0764c97640 1 --2- 172.21.15.43:0/599055128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0760104cd0 0x7f07601050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.795+0000 7f0764c97640 1 -- 172.21.15.43:0/599055128 >> 172.21.15.43:0/599055128 conn(0x7f07600781b0 msgr2=0x7f07600785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.795+0000 7f0764c97640 1 -- 172.21.15.43:0/599055128 shutdown_connections 2024-04-25T10:42:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.795+0000 7f0764c97640 1 -- 172.21.15.43:0/599055128 wait complete. 2024-04-25T10:42:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.796+0000 7f0764c97640 1 Processor -- start 2024-04-25T10:42:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.796+0000 7f0764c97640 1 -- start start 2024-04-25T10:42:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.796+0000 7f0764c97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0760104cd0 0x7f0760102590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.796+0000 7f0764c97640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f076010b730 0x7f0760102ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.796+0000 7f0764c97640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f076010e0e0 0x7f0760103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.796+0000 7f0764c97640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0760113d40 con 0x7f0760104cd0 2024-04-25T10:42:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.796+0000 7f0764c97640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0760113bc0 con 0x7f076010e0e0 2024-04-25T10:42:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.796+0000 7f0764c97640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0760113ec0 con 0x7f076010b730 2024-04-25T10:42:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.796+0000 7f075e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0760104cd0 0x7f0760102590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.796+0000 7f075e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0760104cd0 0x7f0760102590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48870/0 (socket says 172.21.15.43:48870) 2024-04-25T10:42:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.796+0000 7f075e59c640 1 -- 172.21.15.43:0/3636480805 learned_addr learned my addr 172.21.15.43:0/3636480805 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.797+0000 7f0755d9b640 1 --2- 172.21.15.43:0/3636480805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f076010b730 0x7f0760102ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.797+0000 7f075ed9d640 1 --2- 172.21.15.43:0/3636480805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f076010e0e0 0x7f0760103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.797+0000 7f075e59c640 1 -- 172.21.15.43:0/3636480805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f076010b730 msgr2=0x7f0760102ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.797+0000 7f075e59c640 1 --2- 172.21.15.43:0/3636480805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f076010b730 0x7f0760102ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.797+0000 7f075e59c640 1 -- 172.21.15.43:0/3636480805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f076010e0e0 msgr2=0x7f0760103010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.797+0000 7f075e59c640 1 --2- 172.21.15.43:0/3636480805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f076010e0e0 0x7f0760103010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.797+0000 7f075e59c640 1 -- 172.21.15.43:0/3636480805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07601a8ad0 con 0x7f0760104cd0 2024-04-25T10:42:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.797+0000 7f075e59c640 1 --2- 172.21.15.43:0/3636480805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0760104cd0 0x7f0760102590 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f075000c970 tx=0x7f075000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.797+0000 7f0757fff640 1 -- 172.21.15.43:0/3636480805 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0750007bb0 con 0x7f0760104cd0 2024-04-25T10:42:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.797+0000 7f0764c97640 1 -- 172.21.15.43:0/3636480805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07601a8dc0 con 0x7f0760104cd0 2024-04-25T10:42:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.797+0000 7f0757fff640 1 -- 172.21.15.43:0/3636480805 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0750007d50 con 0x7f0760104cd0 2024-04-25T10:42:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.797+0000 7f0757fff640 1 -- 172.21.15.43:0/3636480805 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075000f450 con 0x7f0760104cd0 2024-04-25T10:42:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.797+0000 7f0764c97640 1 -- 172.21.15.43:0/3636480805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07601a9220 con 0x7f0760104cd0 2024-04-25T10:42:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.800+0000 7f076bc22740 1 Processor -- start 2024-04-25T10:42:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.800+0000 7f0757fff640 1 -- 172.21.15.43:0/3636480805 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f0750004f20 con 0x7f0760104cd0 2024-04-25T10:42:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.800+0000 7f076bc22740 1 -- start start 2024-04-25T10:42:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.800+0000 7f076bc22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 0x5612c4148af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.800+0000 7f076bc22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d802f0 0x5612c4149030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.800+0000 7f076bc22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3d884d0 0x5612c4149570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.800+0000 7f076bc22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5612c40af460 con 0x5612c3da91b0 2024-04-25T10:42:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.800+0000 7f076bc22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5612c4069590 con 0x5612c3d802f0 2024-04-25T10:42:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.800+0000 7f076bc22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5612c40ab420 con 0x5612c3d884d0 2024-04-25T10:42:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.801+0000 7f075ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 0x5612c4148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.801+0000 7f075ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 0x5612c4148af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48880/0 (socket says 172.21.15.43:48880) 2024-04-25T10:42:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.801+0000 7f075ed9d640 1 -- 172.21.15.43:0/2771446037 learned_addr learned my addr 172.21.15.43:0/2771446037 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.801+0000 7f075e59c640 1 --2- 172.21.15.43:0/2771446037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d802f0 0x5612c4149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.801+0000 7f0755d9b640 1 --2- 172.21.15.43:0/2771446037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3d884d0 0x5612c4149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.801+0000 7f0757fff640 1 --2- 172.21.15.43:0/3636480805 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f074c0afc50 0x7f074c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.801+0000 7f075ed9d640 1 -- 172.21.15.43:0/2771446037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3d884d0 msgr2=0x5612c4149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.801+0000 7f075ed9d640 1 --2- 172.21.15.43:0/2771446037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3d884d0 0x5612c4149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.801+0000 7f075ed9d640 1 -- 172.21.15.43:0/2771446037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d802f0 msgr2=0x5612c4149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.801+0000 7f075ed9d640 1 --2- 172.21.15.43:0/2771446037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d802f0 0x5612c4149030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.801+0000 7f075ed9d640 1 -- 172.21.15.43:0/2771446037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612c40760d0 con 0x5612c3da91b0 2024-04-25T10:42:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.801+0000 7f0757fff640 1 -- 172.21.15.43:0/3636480805 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f07500deea0 con 0x7f0760104cd0 2024-04-25T10:42:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.801+0000 7f075ed9d640 1 --2- 172.21.15.43:0/3636480805 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f074c0afc50 0x7f074c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.801+0000 7f075ed9d640 1 --2- 172.21.15.43:0/2771446037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 0x5612c4148af0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f0758007f50 tx=0x7f075800e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe8d7619f862d2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.802+0000 7f0754d99640 1 -- 172.21.15.43:0/2771446037 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0758022070 con 0x5612c3da91b0 2024-04-25T10:42:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.802+0000 7f0754d99640 1 -- 172.21.15.43:0/2771446037 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0758011b70 con 0x5612c3da91b0 2024-04-25T10:42:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.802+0000 7f0754d99640 1 -- 172.21.15.43:0/2771446037 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075801cdc0 con 0x5612c3da91b0 2024-04-25T10:42:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.802+0000 7f076bc22740 1 -- 172.21.15.43:0/2771446037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 msgr2=0x5612c4148af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.802+0000 7f075ed9d640 1 --2- 172.21.15.43:0/3636480805 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f074c0afc50 0x7f074c0b2160 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f075801f000 tx=0x7f0758020000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.802+0000 7f076bc22740 1 --2- 172.21.15.43:0/2771446037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 0x5612c4148af0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f0758007f50 tx=0x7f075800e7e0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.803+0000 7f076bc22740 1 -- 172.21.15.43:0/2771446037 shutdown_connections 2024-04-25T10:42:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.803+0000 7f076bc22740 1 --2- 172.21.15.43:0/2771446037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d802f0 0x5612c4149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.803+0000 7f076bc22740 1 --2- 172.21.15.43:0/2771446037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3d884d0 0x5612c4149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.803+0000 7f076bc22740 1 --2- 172.21.15.43:0/2771446037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 0x5612c4148af0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.803+0000 7f076bc22740 1 -- 172.21.15.43:0/2771446037 >> 172.21.15.43:0/2771446037 conn(0x5612c3d9d300 msgr2=0x5612c41452a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.803+0000 7f076bc22740 1 -- 172.21.15.43:0/2771446037 shutdown_connections 2024-04-25T10:42:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.803+0000 7f076bc22740 1 -- 172.21.15.43:0/2771446037 wait complete. 2024-04-25T10:42:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.803+0000 7f076bc22740 1 -- start start 2024-04-25T10:42:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.803+0000 7f076bc22740 1 Processor -- start 2024-04-25T10:42:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.803+0000 7f076bc22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3d802f0 0x5612c4153190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.803+0000 7f076bc22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d884d0 0x5612c41536d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f076bc22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 0x5612c4159110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f076bc22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5612c4155c20 con 0x5612c3da91b0 2024-04-25T10:42:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f076bc22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5612c4036b60 con 0x5612c3d884d0 2024-04-25T10:42:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f076bc22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5612c4155da0 con 0x5612c3d802f0 2024-04-25T10:42:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f076bc22740 10 client.0 _ll_register_callbacks cb 0x5612c405b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f075ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 0x5612c4159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f075ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 0x5612c4159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48882/0 (socket says 172.21.15.43:48882) 2024-04-25T10:42:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f075ed9d640 1 -- 172.21.15.43:0/3626435033 learned_addr learned my addr 172.21.15.43:0/3626435033 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f075e59c640 1 --2- 172.21.15.43:0/3626435033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3d802f0 0x5612c4153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f0755d9b640 1 --2- 172.21.15.43:0/3626435033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d884d0 0x5612c41536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f075ed9d640 1 -- 172.21.15.43:0/3626435033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3d802f0 msgr2=0x5612c4153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f075ed9d640 1 --2- 172.21.15.43:0/3626435033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3d802f0 0x5612c4153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f075ed9d640 1 -- 172.21.15.43:0/3626435033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d884d0 msgr2=0x5612c41536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f075ed9d640 1 --2- 172.21.15.43:0/3626435033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d884d0 0x5612c41536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f075ed9d640 1 -- 172.21.15.43:0/3626435033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612c3dc1210 con 0x5612c3da91b0 2024-04-25T10:42:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f0755d9b640 1 --2- 172.21.15.43:0/3626435033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d884d0 0x5612c41536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f075ed9d640 1 --2- 172.21.15.43:0/3626435033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 0x5612c4159110 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f075801caa0 tx=0x7f0758011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a2ee992d32f1ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f0745ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f0745ffb640 1 -- 172.21.15.43:0/3626435033 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0758022040 con 0x5612c3da91b0 2024-04-25T10:42:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f0745ffb640 1 -- 172.21.15.43:0/3626435033 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f0758011970 con 0x5612c3da91b0 2024-04-25T10:42:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f076bc22740 1 -- 172.21.15.43:0/3626435033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0758011b70 con 0x5612c3da91b0 2024-04-25T10:42:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f0745ffb640 1 -- 172.21.15.43:0/3626435033 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0758007a50 con 0x5612c3da91b0 2024-04-25T10:42:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.804+0000 7f076bc22740 10 client.18366 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.805+0000 7f076bc22740 1 -- 172.21.15.43:0/3626435033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5612c3e8d620 con 0x5612c3da91b0 2024-04-25T10:42:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.805+0000 7f0745ffb640 1 -- 172.21.15.43:0/3626435033 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0758011b70 con 0x5612c3da91b0 2024-04-25T10:42:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.805+0000 7f0745ffb640 1 -- 172.21.15.43:0/3626435033 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0758019040 con 0x5612c3da91b0 2024-04-25T10:42:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.805+0000 7f076bc22740 10 client.18366 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.805+0000 7f076bc22740 10 client.18366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.805+0000 7f076bc22740 20 client.18366 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.805+0000 7f076bc22740 1 --2- 172.21.15.43:0/3626435033 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5612c3d67d80 0x5612c41655f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.805+0000 7f076bc22740 4 client.18366 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:42:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.805+0000 7f076bc22740 1 -- 172.21.15.43:0/3626435033 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612c40bd070 con 0x5612c3d67d80 2024-04-25T10:42:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.805+0000 7f075e59c640 1 --2- 172.21.15.43:0/3626435033 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5612c3d67d80 0x5612c41655f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.806+0000 7f075e59c640 1 --2- 172.21.15.43:0/3626435033 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5612c3d67d80 0x5612c41655f0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c44bcf4888ff5e4c server_cookie=4971b8946c54aa5c in_seq=0 out_seq=0 2024-04-25T10:42:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.806+0000 7f0745ffb640 10 client.18366 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.806+0000 7f0745ffb640 1 -- 172.21.15.43:0/3626435033 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5612c40bd070 con 0x5612c3d67d80 2024-04-25T10:42:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.806+0000 7f0745ffb640 10 client.18366 handle_command_reply: tid=0 2024-04-25T10:42:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.807+0000 7f076bc22740 1 client.18366 shutdown 2024-04-25T10:42:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.807+0000 7f076bc22740 1 -- 172.21.15.43:0/3626435033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 msgr2=0x5612c4159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.807+0000 7f076bc22740 1 --2- 172.21.15.43:0/3626435033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 0x5612c4159110 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f075801caa0 tx=0x7f0758011f30 comp rx=0 tx=0).stop 2024-04-25T10:42:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.807+0000 7f076bc22740 1 -- 172.21.15.43:0/3626435033 shutdown_connections 2024-04-25T10:42:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.807+0000 7f076bc22740 1 --2- 172.21.15.43:0/3626435033 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5612c3d67d80 0x5612c41655f0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.807+0000 7f076bc22740 1 --2- 172.21.15.43:0/3626435033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d884d0 0x5612c41536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.807+0000 7f076bc22740 1 --2- 172.21.15.43:0/3626435033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 0x5612c4159110 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.807+0000 7f0745ffb640 0 client.18366 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.807+0000 7f076bc22740 1 --2- 172.21.15.43:0/3626435033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3d802f0 0x5612c4153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.807+0000 7f076bc22740 1 -- 172.21.15.43:0/3626435033 >> 172.21.15.43:0/3626435033 conn(0x5612c3d9d300 msgr2=0x5612c4148920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.807+0000 7f076bc22740 1 -- 172.21.15.43:0/3626435033 shutdown_connections 2024-04-25T10:42:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.807+0000 7f076bc22740 1 -- 172.21.15.43:0/3626435033 wait complete. 2024-04-25T10:42:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.807+0000 7f076bc22740 20 client.18366 trim_cache size 0 max 16384 2024-04-25T10:42:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.822+0000 7f076bc22740 1 Processor -- start 2024-04-25T10:42:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.822+0000 7f076bc22740 1 -- start start 2024-04-25T10:42:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.822+0000 7f076bc22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d802f0 0x5612c414cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.822+0000 7f076bc22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3d884d0 0x5612c416ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.822+0000 7f076bc22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 0x5612c416b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.822+0000 7f076bc22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f07601b4470 con 0x5612c3da91b0 2024-04-25T10:42:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.822+0000 7f076bc22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f07601b1d40 con 0x5612c3d802f0 2024-04-25T10:42:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.822+0000 7f076bc22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07601afbb0 con 0x5612c3d884d0 2024-04-25T10:42:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.823+0000 7f075ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 0x5612c416b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.823+0000 7f075ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 0x5612c416b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48890/0 (socket says 172.21.15.43:48890) 2024-04-25T10:42:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.823+0000 7f075ed9d640 1 -- 172.21.15.43:0/3139743697 learned_addr learned my addr 172.21.15.43:0/3139743697 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.823+0000 7f0755d9b640 1 --2- 172.21.15.43:0/3139743697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3d884d0 0x5612c416ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.823+0000 7f075e59c640 1 --2- 172.21.15.43:0/3139743697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d802f0 0x5612c414cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.823+0000 7f075ed9d640 1 -- 172.21.15.43:0/3139743697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3d884d0 msgr2=0x5612c416ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.823+0000 7f075ed9d640 1 --2- 172.21.15.43:0/3139743697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3d884d0 0x5612c416ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.823+0000 7f075ed9d640 1 -- 172.21.15.43:0/3139743697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d802f0 msgr2=0x5612c414cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.823+0000 7f075ed9d640 1 --2- 172.21.15.43:0/3139743697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d802f0 0x5612c414cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.823+0000 7f075ed9d640 1 -- 172.21.15.43:0/3139743697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612c41514d0 con 0x5612c3da91b0 2024-04-25T10:42:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.823+0000 7f075ed9d640 1 --2- 172.21.15.43:0/3139743697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 0x5612c416b160 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f0758005b70 tx=0x7f0758007cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0ce9df156db6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.823+0000 7f0747fff640 1 -- 172.21.15.43:0/3139743697 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0758007400 con 0x5612c3da91b0 2024-04-25T10:42:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.823+0000 7f0747fff640 1 -- 172.21.15.43:0/3139743697 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07580077d0 con 0x5612c3da91b0 2024-04-25T10:42:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.823+0000 7f0747fff640 1 -- 172.21.15.43:0/3139743697 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0758011af0 con 0x5612c3da91b0 2024-04-25T10:42:22.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:42:22.828 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:42:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.824+0000 7f076bc22740 1 -- 172.21.15.43:0/3139743697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 msgr2=0x5612c416b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.824+0000 7f076bc22740 1 --2- 172.21.15.43:0/3139743697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 0x5612c416b160 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f0758005b70 tx=0x7f0758007cb0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.824+0000 7f076bc22740 1 -- 172.21.15.43:0/3139743697 shutdown_connections 2024-04-25T10:42:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.824+0000 7f076bc22740 1 --2- 172.21.15.43:0/3139743697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3da91b0 0x5612c416b160 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.824+0000 7f076bc22740 1 --2- 172.21.15.43:0/3139743697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3d884d0 0x5612c416ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.824+0000 7f076bc22740 1 --2- 172.21.15.43:0/3139743697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d802f0 0x5612c414cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.824+0000 7f076bc22740 1 -- 172.21.15.43:0/3139743697 >> 172.21.15.43:0/3139743697 conn(0x5612c3d9d300 msgr2=0x5612c414c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.824+0000 7f076bc22740 1 -- 172.21.15.43:0/3139743697 shutdown_connections 2024-04-25T10:42:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.824+0000 7f076bc22740 1 -- 172.21.15.43:0/3139743697 wait complete. 2024-04-25T10:42:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.824+0000 7f076bc22740 1 -- start start 2024-04-25T10:42:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f076bc22740 1 Processor -- start 2024-04-25T10:42:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f076bc22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d802f0 0x5612c41c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f076bc22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3d884d0 0x5612c41c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f076bc22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3da91b0 0x5612c415c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f076bc22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f07601b0200 con 0x5612c3d884d0 2024-04-25T10:42:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f076bc22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f07601b1670 con 0x5612c3d802f0 2024-04-25T10:42:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f076bc22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07601b08d0 con 0x5612c3da91b0 2024-04-25T10:42:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f076bc22740 10 client.0 _ll_register_callbacks cb 0x7f0758020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f0755d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3d884d0 0x5612c41c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f0755d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3d884d0 0x5612c41c50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48898/0 (socket says 172.21.15.43:48898) 2024-04-25T10:42:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f0755d9b640 1 -- 172.21.15.43:0/1008997848 learned_addr learned my addr 172.21.15.43:0/1008997848 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f075e59c640 1 --2- 172.21.15.43:0/1008997848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d802f0 0x5612c41c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f075ed9d640 1 --2- 172.21.15.43:0/1008997848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3da91b0 0x5612c415c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f0755d9b640 1 -- 172.21.15.43:0/1008997848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3da91b0 msgr2=0x5612c415c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f0755d9b640 1 --2- 172.21.15.43:0/1008997848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3da91b0 0x5612c415c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f0755d9b640 1 -- 172.21.15.43:0/1008997848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d802f0 msgr2=0x5612c41c4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f0755d9b640 1 --2- 172.21.15.43:0/1008997848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d802f0 0x5612c41c4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f0755d9b640 1 -- 172.21.15.43:0/1008997848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612c415cbc0 con 0x5612c3d884d0 2024-04-25T10:42:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f075ed9d640 1 --2- 172.21.15.43:0/1008997848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3da91b0 0x5612c415c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f075e59c640 1 --2- 172.21.15.43:0/1008997848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d802f0 0x5612c41c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f0755d9b640 1 --2- 172.21.15.43:0/1008997848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3d884d0 0x5612c41c50c0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f074802aa30 tx=0x7f0748031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446acd406de86146 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.825+0000 7f0745ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.826+0000 7f0745ffb640 1 -- 172.21.15.43:0/1008997848 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748035040 con 0x5612c3d884d0 2024-04-25T10:42:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.826+0000 7f0745ffb640 1 -- 172.21.15.43:0/1008997848 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f0748004310 con 0x5612c3d884d0 2024-04-25T10:42:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.826+0000 7f076bc22740 1 -- 172.21.15.43:0/1008997848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07580077d0 con 0x5612c3d884d0 2024-04-25T10:42:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.826+0000 7f0745ffb640 1 -- 172.21.15.43:0/1008997848 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0748005020 con 0x5612c3d884d0 2024-04-25T10:42:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.826+0000 7f076bc22740 10 client.18372 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.826+0000 7f076bc22740 1 -- 172.21.15.43:0/1008997848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5612c41cd1a0 con 0x5612c3d884d0 2024-04-25T10:42:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.826+0000 7f0745ffb640 1 -- 172.21.15.43:0/1008997848 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07580077d0 con 0x5612c3d884d0 2024-04-25T10:42:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.826+0000 7f0745ffb640 1 -- 172.21.15.43:0/1008997848 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f07480035f0 con 0x5612c3d884d0 2024-04-25T10:42:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.826+0000 7f076bc22740 10 client.18372 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.826+0000 7f076bc22740 10 client.18372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.826+0000 7f076bc22740 20 client.18372 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.826+0000 7f076bc22740 1 --2- 172.21.15.43:0/1008997848 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5612c3d3d1c0 0x5612c41cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.826+0000 7f076bc22740 4 client.18372 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:42:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.826+0000 7f076bc22740 1 -- 172.21.15.43:0/1008997848 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5612c40b6ee0 con 0x5612c3d3d1c0 2024-04-25T10:42:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.827+0000 7f075e59c640 1 --2- 172.21.15.43:0/1008997848 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5612c3d3d1c0 0x5612c41cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.827+0000 7f0745ffb640 10 client.18372 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.827+0000 7f075e59c640 1 --2- 172.21.15.43:0/1008997848 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5612c3d3d1c0 0x5612c41cf990 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3b37a1a896044cd server_cookie=c47ec3c0aad37130 in_seq=0 out_seq=0 2024-04-25T10:42:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.842+0000 7f0745ffb640 1 -- 172.21.15.43:0/1008997848 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5612c40b6ee0 con 0x5612c3d3d1c0 2024-04-25T10:42:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.842+0000 7f0745ffb640 10 client.18372 handle_command_reply: tid=0 2024-04-25T10:42:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.842+0000 7f076bc22740 1 client.18372 shutdown 2024-04-25T10:42:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.842+0000 7f076bc22740 1 -- 172.21.15.43:0/1008997848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3d884d0 msgr2=0x5612c41c50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.842+0000 7f076bc22740 1 --2- 172.21.15.43:0/1008997848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3d884d0 0x5612c41c50c0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f074802aa30 tx=0x7f0748031950 comp rx=0 tx=0).stop 2024-04-25T10:42:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.843+0000 7f076bc22740 1 -- 172.21.15.43:0/1008997848 shutdown_connections 2024-04-25T10:42:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.843+0000 7f076bc22740 1 --2- 172.21.15.43:0/1008997848 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5612c3d3d1c0 0x5612c41cf990 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.843+0000 7f076bc22740 1 --2- 172.21.15.43:0/1008997848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612c3da91b0 0x5612c415c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.843+0000 7f076bc22740 1 --2- 172.21.15.43:0/1008997848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612c3d884d0 0x5612c41c50c0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.843+0000 7f0745ffb640 0 client.18372 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.843+0000 7f076bc22740 1 --2- 172.21.15.43:0/1008997848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612c3d802f0 0x5612c41c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.843+0000 7f076bc22740 1 -- 172.21.15.43:0/1008997848 >> 172.21.15.43:0/1008997848 conn(0x5612c3d9d300 msgr2=0x5612c414c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.843+0000 7f076bc22740 1 -- 172.21.15.43:0/1008997848 shutdown_connections 2024-04-25T10:42:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.843+0000 7f076bc22740 1 -- 172.21.15.43:0/1008997848 wait complete. 2024-04-25T10:42:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.843+0000 7f076bc22740 20 client.18372 trim_cache size 0 max 16384 2024-04-25T10:42:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:42:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:42:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:42:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:42:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.843+0000 7f0764c97640 1 -- 172.21.15.43:0/3636480805 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f074c0afc50 msgr2=0x7f074c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.843+0000 7f0764c97640 1 --2- 172.21.15.43:0/3636480805 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f074c0afc50 0x7f074c0b2160 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f075801f000 tx=0x7f0758020000 comp rx=0 tx=0).stop 2024-04-25T10:42:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.843+0000 7f0764c97640 1 -- 172.21.15.43:0/3636480805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0760104cd0 msgr2=0x7f0760102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.843+0000 7f0764c97640 1 --2- 172.21.15.43:0/3636480805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0760104cd0 0x7f0760102590 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f075000c970 tx=0x7f075000ce30 comp rx=0 tx=0).stop 2024-04-25T10:42:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.843+0000 7f0764c97640 1 -- 172.21.15.43:0/3636480805 shutdown_connections 2024-04-25T10:42:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.843+0000 7f0764c97640 1 --2- 172.21.15.43:0/3636480805 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f074c0afc50 0x7f074c0b2160 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.843+0000 7f0764c97640 1 --2- 172.21.15.43:0/3636480805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f076010e0e0 0x7f0760103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.843+0000 7f0764c97640 1 --2- 172.21.15.43:0/3636480805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f076010b730 0x7f0760102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.843+0000 7f0764c97640 1 --2- 172.21.15.43:0/3636480805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0760104cd0 0x7f0760102590 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.843+0000 7f0764c97640 1 -- 172.21.15.43:0/3636480805 >> 172.21.15.43:0/3636480805 conn(0x7f07600781b0 msgr2=0x7f0760101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.844+0000 7f0764c97640 1 -- 172.21.15.43:0/3636480805 shutdown_connections 2024-04-25T10:42:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:22.844+0000 7f0764c97640 1 -- 172.21.15.43:0/3636480805 wait complete. 2024-04-25T10:42:22.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.946+0000 7fd1b0b0d640 1 -- 172.21.15.43:0/3102011571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac10b650 msgr2=0x7fd1ac10da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:22.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.946+0000 7fd1b0b0d640 1 --2- 172.21.15.43:0/3102011571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac10b650 0x7fd1ac10da70 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0009900 tx=0x7fd1a002a410 comp rx=0 tx=0).stop 2024-04-25T10:42:22.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.946+0000 7fd1b0b0d640 1 -- 172.21.15.43:0/3102011571 shutdown_connections 2024-04-25T10:42:22.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.946+0000 7fd1b0b0d640 1 --2- 172.21.15.43:0/3102011571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ac10e040 0x7fd1ac110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.946+0000 7fd1b0b0d640 1 --2- 172.21.15.43:0/3102011571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac10b650 0x7fd1ac10da70 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.946+0000 7fd1b0b0d640 1 --2- 172.21.15.43:0/3102011571 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1ac104bf0 0x7fd1ac105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.946+0000 7fd1b0b0d640 1 -- 172.21.15.43:0/3102011571 >> 172.21.15.43:0/3102011571 conn(0x7fd1ac078100 msgr2=0x7fd1ac078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:22.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.946+0000 7fd1b0b0d640 1 -- 172.21.15.43:0/3102011571 shutdown_connections 2024-04-25T10:42:22.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.946+0000 7fd1b0b0d640 1 -- 172.21.15.43:0/3102011571 wait complete. 2024-04-25T10:42:22.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.947+0000 7fd1b0b0d640 1 Processor -- start 2024-04-25T10:42:22.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.947+0000 7fd1b0b0d640 1 -- start start 2024-04-25T10:42:22.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.947+0000 7fd1b0b0d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1ac104bf0 0x7fd1ac19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.947+0000 7fd1b0b0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac10b650 0x7fd1ac19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.947+0000 7fd1b0b0d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ac10e040 0x7fd1ac19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.947+0000 7fd1b0b0d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac113ca0 con 0x7fd1ac10b650 2024-04-25T10:42:22.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.947+0000 7fd1b0b0d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac113b20 con 0x7fd1ac104bf0 2024-04-25T10:42:22.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.947+0000 7fd1b0b0d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac113e20 con 0x7fd1ac10e040 2024-04-25T10:42:22.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.947+0000 7fd1a9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac10b650 0x7fd1ac19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.947+0000 7fd1a9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac10b650 0x7fd1ac19aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48912/0 (socket says 172.21.15.43:48912) 2024-04-25T10:42:22.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.947+0000 7fd1a9d9b640 1 -- 172.21.15.43:0/2773174972 learned_addr learned my addr 172.21.15.43:0/2773174972 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:22.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.947+0000 7fd1aa59c640 1 --2- 172.21.15.43:0/2773174972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1ac104bf0 0x7fd1ac19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.948+0000 7fd1aad9d640 1 --2- 172.21.15.43:0/2773174972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ac10e040 0x7fd1ac19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.948+0000 7fd1a9d9b640 1 -- 172.21.15.43:0/2773174972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ac10e040 msgr2=0x7fd1ac19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:22.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.948+0000 7fd1a9d9b640 1 --2- 172.21.15.43:0/2773174972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ac10e040 0x7fd1ac19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.948+0000 7fd1a9d9b640 1 -- 172.21.15.43:0/2773174972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1ac104bf0 msgr2=0x7fd1ac19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:22.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.948+0000 7fd1a9d9b640 1 --2- 172.21.15.43:0/2773174972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1ac104bf0 0x7fd1ac19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.948+0000 7fd1a9d9b640 1 -- 172.21.15.43:0/2773174972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ac19f520 con 0x7fd1ac10b650 2024-04-25T10:42:22.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.948+0000 7fd1aa59c640 1 --2- 172.21.15.43:0/2773174972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1ac104bf0 0x7fd1ac19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:22.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.948+0000 7fd1a9d9b640 1 --2- 172.21.15.43:0/2773174972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac10b650 0x7fd1ac19aca0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0006010 tx=0x7fd1a00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:22.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.948+0000 7fd19b7fe640 1 -- 172.21.15.43:0/2773174972 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a003f030 con 0x7fd1ac10b650 2024-04-25T10:42:22.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.948+0000 7fd19b7fe640 1 -- 172.21.15.43:0/2773174972 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1a0003cb0 con 0x7fd1ac10b650 2024-04-25T10:42:22.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.948+0000 7fd1b0b0d640 1 -- 172.21.15.43:0/2773174972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1ac19f7b0 con 0x7fd1ac10b650 2024-04-25T10:42:22.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.948+0000 7fd19b7fe640 1 -- 172.21.15.43:0/2773174972 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0035070 con 0x7fd1ac10b650 2024-04-25T10:42:22.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.948+0000 7fd1b0b0d640 1 -- 172.21.15.43:0/2773174972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1ac19fcf0 con 0x7fd1ac10b650 2024-04-25T10:42:22.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.951+0000 7fd1b79c3740 1 Processor -- start 2024-04-25T10:42:22.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.951+0000 7fd19b7fe640 1 -- 172.21.15.43:0/2773174972 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fd1a0042030 con 0x7fd1ac10b650 2024-04-25T10:42:22.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.951+0000 7fd1b79c3740 1 -- start start 2024-04-25T10:42:22.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.951+0000 7fd1b79c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582edb30390 0x5582eded0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.951+0000 7fd1b79c3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb074d0 0x5582eded0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.951+0000 7fd1b79c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb0f6b0 0x5582ededd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.951+0000 7fd1b79c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5582ede36d80 con 0x5582edb0f6b0 2024-04-25T10:42:22.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.951+0000 7fd1b79c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5582eddf08b0 con 0x5582edb30390 2024-04-25T10:42:22.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.951+0000 7fd1b79c3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5582ede32d40 con 0x5582edb074d0 2024-04-25T10:42:22.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.952+0000 7fd1a9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb0f6b0 0x5582ededd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.952+0000 7fd1a9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb0f6b0 0x5582ededd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48924/0 (socket says 172.21.15.43:48924) 2024-04-25T10:42:22.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.952+0000 7fd1a9d9b640 1 -- 172.21.15.43:0/2559398238 learned_addr learned my addr 172.21.15.43:0/2559398238 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:22.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.952+0000 7fd1aad9d640 1 --2- 172.21.15.43:0/2559398238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582edb30390 0x5582eded0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.952+0000 7fd1aa59c640 1 --2- 172.21.15.43:0/2559398238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb074d0 0x5582eded0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.952+0000 7fd19b7fe640 1 --2- 172.21.15.43:0/2773174972 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd1940afce0 0x7fd1940b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.952+0000 7fd1a9d9b640 1 -- 172.21.15.43:0/2559398238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb074d0 msgr2=0x5582eded0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:22.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.952+0000 7fd1a9d9b640 1 --2- 172.21.15.43:0/2559398238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb074d0 0x5582eded0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.952+0000 7fd1a9d9b640 1 -- 172.21.15.43:0/2559398238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582edb30390 msgr2=0x5582eded0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:22.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.952+0000 7fd1a9d9b640 1 --2- 172.21.15.43:0/2559398238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582edb30390 0x5582eded0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.952+0000 7fd1a9d9b640 1 -- 172.21.15.43:0/2559398238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582eddc1a40 con 0x5582edb0f6b0 2024-04-25T10:42:22.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.952+0000 7fd1aad9d640 1 --2- 172.21.15.43:0/2773174972 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd1940afce0 0x7fd1940b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.952+0000 7fd19b7fe640 1 -- 172.21.15.43:0/2773174972 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fd1a00ff5c0 con 0x7fd1ac10b650 2024-04-25T10:42:22.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.952+0000 7fd1a9d9b640 1 --2- 172.21.15.43:0/2559398238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb0f6b0 0x5582ededd8f0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00fcc80 tx=0x7fd1a0092e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8d76c63aae01ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:22.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.952+0000 7fd198ff9640 1 -- 172.21.15.43:0/2559398238 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a00c6ed0 con 0x5582edb0f6b0 2024-04-25T10:42:22.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.952+0000 7fd198ff9640 1 -- 172.21.15.43:0/2559398238 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1a0003e60 con 0x5582edb0f6b0 2024-04-25T10:42:22.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.952+0000 7fd198ff9640 1 -- 172.21.15.43:0/2559398238 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a00c7e60 con 0x5582edb0f6b0 2024-04-25T10:42:22.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.952+0000 7fd1aad9d640 1 --2- 172.21.15.43:0/2773174972 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd1940afce0 0x7fd1940b21f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x5582edecdbd0 tx=0x7fd1a40046c0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:22.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.953+0000 7fd1b79c3740 1 -- 172.21.15.43:0/2559398238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb0f6b0 msgr2=0x5582ededd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:22.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.953+0000 7fd1b79c3740 1 --2- 172.21.15.43:0/2559398238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb0f6b0 0x5582ededd8f0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00fcc80 tx=0x7fd1a0092e60 comp rx=0 tx=0).stop 2024-04-25T10:42:22.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.953+0000 7fd1b79c3740 1 -- 172.21.15.43:0/2559398238 shutdown_connections 2024-04-25T10:42:22.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.953+0000 7fd1b79c3740 1 --2- 172.21.15.43:0/2559398238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb0f6b0 0x5582ededd8f0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.953+0000 7fd1b79c3740 1 --2- 172.21.15.43:0/2559398238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb074d0 0x5582eded0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.953+0000 7fd1b79c3740 1 --2- 172.21.15.43:0/2559398238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582edb30390 0x5582eded0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.953+0000 7fd1b79c3740 1 -- 172.21.15.43:0/2559398238 >> 172.21.15.43:0/2559398238 conn(0x5582edb244e0 msgr2=0x5582edecd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:22.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.953+0000 7fd1b79c3740 1 -- 172.21.15.43:0/2559398238 shutdown_connections 2024-04-25T10:42:22.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.953+0000 7fd1b79c3740 1 -- 172.21.15.43:0/2559398238 wait complete. 2024-04-25T10:42:22.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.954+0000 7fd1b79c3740 1 -- start start 2024-04-25T10:42:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.954+0000 7fd1b79c3740 1 Processor -- start 2024-04-25T10:42:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.954+0000 7fd1b79c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb074d0 0x5582ededb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.954+0000 7fd1b79c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582edb0f6b0 0x5582ededbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.954+0000 7fd1b79c3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb30390 0x5582edee5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.954+0000 7fd1b79c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5582ede36d80 con 0x5582edb074d0 2024-04-25T10:42:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.954+0000 7fd1b79c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5582eddbdd10 con 0x5582edb0f6b0 2024-04-25T10:42:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.954+0000 7fd1b79c3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5582edee2240 con 0x5582edb30390 2024-04-25T10:42:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.954+0000 7fd1b79c3740 10 client.0 _ll_register_callbacks cb 0x5582eddea210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.954+0000 7fd1aa59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb074d0 0x5582ededb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.954+0000 7fd1aa59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb074d0 0x5582ededb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48936/0 (socket says 172.21.15.43:48936) 2024-04-25T10:42:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.954+0000 7fd1aa59c640 1 -- 172.21.15.43:0/1440780766 learned_addr learned my addr 172.21.15.43:0/1440780766 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.955+0000 7fd1a9d9b640 1 --2- 172.21.15.43:0/1440780766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582edb0f6b0 0x5582ededbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.955+0000 7fd1aad9d640 1 --2- 172.21.15.43:0/1440780766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb30390 0x5582edee5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.955+0000 7fd1aa59c640 1 -- 172.21.15.43:0/1440780766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb30390 msgr2=0x5582edee5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.955+0000 7fd1aa59c640 1 --2- 172.21.15.43:0/1440780766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb30390 0x5582edee5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.955+0000 7fd1aa59c640 1 -- 172.21.15.43:0/1440780766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582edb0f6b0 msgr2=0x5582ededbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.955+0000 7fd1aa59c640 1 --2- 172.21.15.43:0/1440780766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582edb0f6b0 0x5582ededbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.955+0000 7fd1aa59c640 1 -- 172.21.15.43:0/1440780766 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582edc127d0 con 0x5582edb074d0 2024-04-25T10:42:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.955+0000 7fd1aad9d640 1 --2- 172.21.15.43:0/1440780766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb30390 0x5582edee5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.955+0000 7fd1aa59c640 1 --2- 172.21.15.43:0/1440780766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb074d0 0x5582ededb5d0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fd19c010ab0 tx=0x7fd19c010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce1fb0a3ccab8e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.955+0000 7fd191ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.955+0000 7fd191ffb640 1 -- 172.21.15.43:0/1440780766 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c003970 con 0x5582edb074d0 2024-04-25T10:42:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.955+0000 7fd191ffb640 1 -- 172.21.15.43:0/1440780766 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fd19c003b10 con 0x5582edb074d0 2024-04-25T10:42:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.955+0000 7fd1b79c3740 1 -- 172.21.15.43:0/1440780766 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5582ede44990 con 0x5582edb074d0 2024-04-25T10:42:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.955+0000 7fd191ffb640 1 -- 172.21.15.43:0/1440780766 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd19c003d50 con 0x5582edb074d0 2024-04-25T10:42:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.955+0000 7fd1b79c3740 10 client.18384 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.955+0000 7fd1b79c3740 1 -- 172.21.15.43:0/1440780766 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5582edb483f0 con 0x5582edb074d0 2024-04-25T10:42:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.955+0000 7fd191ffb640 1 -- 172.21.15.43:0/1440780766 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5582ede44990 con 0x5582edb074d0 2024-04-25T10:42:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.956+0000 7fd191ffb640 1 -- 172.21.15.43:0/1440780766 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd19c0245d0 con 0x5582edb074d0 2024-04-25T10:42:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.956+0000 7fd1b79c3740 10 client.18384 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.956+0000 7fd1b79c3740 10 client.18384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.956+0000 7fd1b79c3740 20 client.18384 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.956+0000 7fd1b79c3740 1 --2- 172.21.15.43:0/1440780766 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5582edaeef60 0x5582edef16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.956+0000 7fd1b79c3740 4 client.18384 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:42:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.956+0000 7fd1b79c3740 1 -- 172.21.15.43:0/1440780766 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582ede3e800 con 0x5582edaeef60 2024-04-25T10:42:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.956+0000 7fd1a9d9b640 1 --2- 172.21.15.43:0/1440780766 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5582edaeef60 0x5582edef16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.956+0000 7fd191ffb640 10 client.18384 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.956+0000 7fd1a9d9b640 1 --2- 172.21.15.43:0/1440780766 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5582edaeef60 0x5582edef16e0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ef3cf8f55b12171 server_cookie=11a42dd19ce7c6ce in_seq=0 out_seq=0 2024-04-25T10:42:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.957+0000 7fd191ffb640 1 -- 172.21.15.43:0/1440780766 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5582ede3e800 con 0x5582edaeef60 2024-04-25T10:42:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.957+0000 7fd191ffb640 10 client.18384 handle_command_reply: tid=0 2024-04-25T10:42:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.957+0000 7fd1b79c3740 1 client.18384 shutdown 2024-04-25T10:42:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.957+0000 7fd1b79c3740 1 -- 172.21.15.43:0/1440780766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb074d0 msgr2=0x5582ededb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.957+0000 7fd1b79c3740 1 --2- 172.21.15.43:0/1440780766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb074d0 0x5582ededb5d0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fd19c010ab0 tx=0x7fd19c010f70 comp rx=0 tx=0).stop 2024-04-25T10:42:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.957+0000 7fd1b79c3740 1 -- 172.21.15.43:0/1440780766 shutdown_connections 2024-04-25T10:42:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.957+0000 7fd1b79c3740 1 --2- 172.21.15.43:0/1440780766 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5582edaeef60 0x5582edef16e0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.957+0000 7fd1b79c3740 1 --2- 172.21.15.43:0/1440780766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582edb0f6b0 0x5582ededbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.957+0000 7fd1b79c3740 1 --2- 172.21.15.43:0/1440780766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb30390 0x5582edee5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.957+0000 7fd191ffb640 0 client.18384 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:22.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.957+0000 7fd1b79c3740 1 --2- 172.21.15.43:0/1440780766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb074d0 0x5582ededb5d0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.957+0000 7fd1b79c3740 1 -- 172.21.15.43:0/1440780766 >> 172.21.15.43:0/1440780766 conn(0x5582edb244e0 msgr2=0x5582eded2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:22.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.957+0000 7fd1b79c3740 1 -- 172.21.15.43:0/1440780766 shutdown_connections 2024-04-25T10:42:22.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.957+0000 7fd1b79c3740 1 -- 172.21.15.43:0/1440780766 wait complete. 2024-04-25T10:42:22.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.957+0000 7fd1b79c3740 20 client.18384 trim_cache size 0 max 16384 2024-04-25T10:42:22.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.976+0000 7fd1b79c3740 1 Processor -- start 2024-04-25T10:42:22.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.976+0000 7fd1b79c3740 1 -- start start 2024-04-25T10:42:22.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.976+0000 7fd1b79c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582edaeef60 0x5582ededa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.976+0000 7fd1b79c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb074d0 0x5582ededd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.976+0000 7fd1b79c3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb0f6b0 0x5582ededd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.976+0000 7fd1b79c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1b0050 con 0x5582edb074d0 2024-04-25T10:42:22.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.976+0000 7fd1b79c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1ad920 con 0x5582edaeef60 2024-04-25T10:42:22.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.976+0000 7fd1b79c3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1ab790 con 0x5582edb0f6b0 2024-04-25T10:42:22.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.976+0000 7fd1a9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb074d0 0x5582ededd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.976+0000 7fd1a9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb074d0 0x5582ededd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48944/0 (socket says 172.21.15.43:48944) 2024-04-25T10:42:22.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.976+0000 7fd1a9d9b640 1 -- 172.21.15.43:0/1324137391 learned_addr learned my addr 172.21.15.43:0/1324137391 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:22.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.976+0000 7fd1aad9d640 1 --2- 172.21.15.43:0/1324137391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb0f6b0 0x5582ededd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.977+0000 7fd1aa59c640 1 --2- 172.21.15.43:0/1324137391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582edaeef60 0x5582ededa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.977+0000 7fd1a9d9b640 1 -- 172.21.15.43:0/1324137391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb0f6b0 msgr2=0x5582ededd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:22.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.977+0000 7fd1a9d9b640 1 --2- 172.21.15.43:0/1324137391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb0f6b0 0x5582ededd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.977+0000 7fd1a9d9b640 1 -- 172.21.15.43:0/1324137391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582edaeef60 msgr2=0x5582ededa6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:22.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.977+0000 7fd1a9d9b640 1 --2- 172.21.15.43:0/1324137391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582edaeef60 0x5582ededa6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.977+0000 7fd1a9d9b640 1 -- 172.21.15.43:0/1324137391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582edb080f0 con 0x5582edb074d0 2024-04-25T10:42:22.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.977+0000 7fd1a9d9b640 1 --2- 172.21.15.43:0/1324137391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb074d0 0x5582ededd1f0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00c37e0 tx=0x7fd1a00b0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e60ef7902a4543 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:22.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.980+0000 7fd193fff640 1 -- 172.21.15.43:0/1324137391 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0049070 con 0x5582edb074d0 2024-04-25T10:42:22.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.980+0000 7fd193fff640 1 -- 172.21.15.43:0/1324137391 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1a00c7270 con 0x5582edb074d0 2024-04-25T10:42:22.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.980+0000 7fd193fff640 1 -- 172.21.15.43:0/1324137391 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a00c7760 con 0x5582edb074d0 2024-04-25T10:42:22.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.981+0000 7fd1b79c3740 1 -- 172.21.15.43:0/1324137391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb074d0 msgr2=0x5582ededd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:22.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.981+0000 7fd1b79c3740 1 --2- 172.21.15.43:0/1324137391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb074d0 0x5582ededd1f0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00c37e0 tx=0x7fd1a00b0040 comp rx=0 tx=0).stop 2024-04-25T10:42:22.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.981+0000 7fd1b79c3740 1 -- 172.21.15.43:0/1324137391 shutdown_connections 2024-04-25T10:42:22.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.981+0000 7fd1b79c3740 1 --2- 172.21.15.43:0/1324137391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb0f6b0 0x5582ededd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.981+0000 7fd1b79c3740 1 --2- 172.21.15.43:0/1324137391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb074d0 0x5582ededd1f0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.981+0000 7fd1b79c3740 1 --2- 172.21.15.43:0/1324137391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582edaeef60 0x5582ededa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.981+0000 7fd1b79c3740 1 -- 172.21.15.43:0/1324137391 >> 172.21.15.43:0/1324137391 conn(0x5582edb244e0 msgr2=0x5582ededcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:22.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.981+0000 7fd1b79c3740 1 -- 172.21.15.43:0/1324137391 shutdown_connections 2024-04-25T10:42:22.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.981+0000 7fd1b79c3740 1 -- 172.21.15.43:0/1324137391 wait complete. 2024-04-25T10:42:22.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.982+0000 7fd1b79c3740 1 -- start start 2024-04-25T10:42:22.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.982+0000 7fd1b79c3740 1 Processor -- start 2024-04-25T10:42:22.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.982+0000 7fd1b79c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582edaeef60 0x5582edee67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.982+0000 7fd1b79c3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb074d0 0x5582edee6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.982+0000 7fd1b79c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb0f6b0 0x5582edee7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.982+0000 7fd1b79c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1abde0 con 0x5582edb0f6b0 2024-04-25T10:42:22.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.982+0000 7fd1b79c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1ad250 con 0x5582edaeef60 2024-04-25T10:42:22.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.982+0000 7fd1b79c3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1ac4b0 con 0x5582edb074d0 2024-04-25T10:42:22.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.982+0000 7fd1b79c3740 10 client.0 _ll_register_callbacks cb 0x7fd19c0076e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:22.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.982+0000 7fd1aad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb0f6b0 0x5582edee7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.982+0000 7fd1aad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb0f6b0 0x5582edee7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48946/0 (socket says 172.21.15.43:48946) 2024-04-25T10:42:22.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.982+0000 7fd1aad9d640 1 -- 172.21.15.43:0/3581591345 learned_addr learned my addr 172.21.15.43:0/3581591345 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:22.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.982+0000 7fd1aa59c640 1 --2- 172.21.15.43:0/3581591345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582edaeef60 0x5582edee67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.982+0000 7fd1a9d9b640 1 --2- 172.21.15.43:0/3581591345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb074d0 0x5582edee6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.982+0000 7fd1aad9d640 1 -- 172.21.15.43:0/3581591345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb074d0 msgr2=0x5582edee6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:22.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.982+0000 7fd1aad9d640 1 --2- 172.21.15.43:0/3581591345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb074d0 0x5582edee6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.982+0000 7fd1aad9d640 1 -- 172.21.15.43:0/3581591345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582edaeef60 msgr2=0x5582edee67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:22.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.982+0000 7fd1aad9d640 1 --2- 172.21.15.43:0/3581591345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582edaeef60 0x5582edee67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.983+0000 7fd1aad9d640 1 -- 172.21.15.43:0/3581591345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582eddf74b0 con 0x5582edb0f6b0 2024-04-25T10:42:22.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.983+0000 7fd1aad9d640 1 --2- 172.21.15.43:0/3581591345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb0f6b0 0x5582edee7240 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4004020 tx=0x7fd1a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd99a6fe6ee25755 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:22.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.983+0000 7fd191ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:22.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.983+0000 7fd191ffb640 1 -- 172.21.15.43:0/3581591345 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a401f810 con 0x5582edb0f6b0 2024-04-25T10:42:22.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.983+0000 7fd191ffb640 1 -- 172.21.15.43:0/3581591345 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fd1a401f9b0 con 0x5582edb0f6b0 2024-04-25T10:42:22.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.983+0000 7fd1b79c3740 1 -- 172.21.15.43:0/3581591345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1a00c7270 con 0x5582edb0f6b0 2024-04-25T10:42:22.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.983+0000 7fd191ffb640 1 -- 172.21.15.43:0/3581591345 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd1a401fbb0 con 0x5582edb0f6b0 2024-04-25T10:42:22.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.983+0000 7fd1b79c3740 10 client.18390 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:22.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.983+0000 7fd1b79c3740 1 -- 172.21.15.43:0/3581591345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5582edf58c30 con 0x5582edb0f6b0 2024-04-25T10:42:22.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.983+0000 7fd191ffb640 1 -- 172.21.15.43:0/3581591345 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1a00c7270 con 0x5582edb0f6b0 2024-04-25T10:42:22.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.983+0000 7fd191ffb640 1 -- 172.21.15.43:0/3581591345 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd1a400f040 con 0x5582edb0f6b0 2024-04-25T10:42:22.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.984+0000 7fd1b79c3740 10 client.18390 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:22.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.984+0000 7fd1b79c3740 10 client.18390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:22.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.984+0000 7fd1b79c3740 20 client.18390 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:22.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.984+0000 7fd1b79c3740 1 --2- 172.21.15.43:0/3581591345 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5582edb30390 0x5582edf5b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:22.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.984+0000 7fd1b79c3740 4 client.18390 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:42:22.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.984+0000 7fd1b79c3740 1 -- 172.21.15.43:0/3581591345 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5582eddb94a0 con 0x5582edb30390 2024-04-25T10:42:22.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.984+0000 7fd1aa59c640 1 --2- 172.21.15.43:0/3581591345 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5582edb30390 0x5582edf5b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.984+0000 7fd191ffb640 10 client.18390 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.984+0000 7fd1aa59c640 1 --2- 172.21.15.43:0/3581591345 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5582edb30390 0x5582edf5b3f0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d33f55847914eefa server_cookie=68b8751ca9412366 in_seq=0 out_seq=0 2024-04-25T10:42:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.985+0000 7fd191ffb640 1 -- 172.21.15.43:0/3581591345 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5582eddb94a0 con 0x5582edb30390 2024-04-25T10:42:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.985+0000 7fd191ffb640 10 client.18390 handle_command_reply: tid=0 2024-04-25T10:42:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.985+0000 7fd1b79c3740 1 client.18390 shutdown 2024-04-25T10:42:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.985+0000 7fd1b79c3740 1 -- 172.21.15.43:0/3581591345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb0f6b0 msgr2=0x5582edee7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.985+0000 7fd1b79c3740 1 --2- 172.21.15.43:0/3581591345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb0f6b0 0x5582edee7240 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4004020 tx=0x7fd1a4004730 comp rx=0 tx=0).stop 2024-04-25T10:42:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.985+0000 7fd1b79c3740 1 -- 172.21.15.43:0/3581591345 shutdown_connections 2024-04-25T10:42:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.985+0000 7fd1b79c3740 1 --2- 172.21.15.43:0/3581591345 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5582edb30390 0x5582edf5b3f0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.985+0000 7fd1b79c3740 1 --2- 172.21.15.43:0/3581591345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582edb0f6b0 0x5582edee7240 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.985+0000 7fd1b79c3740 1 --2- 172.21.15.43:0/3581591345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5582edb074d0 0x5582edee6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.985+0000 7fd191ffb640 0 client.18390 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.985+0000 7fd1b79c3740 1 --2- 172.21.15.43:0/3581591345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582edaeef60 0x5582edee67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.985+0000 7fd1b79c3740 1 -- 172.21.15.43:0/3581591345 >> 172.21.15.43:0/3581591345 conn(0x5582edb244e0 msgr2=0x5582ededa420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.985+0000 7fd1b79c3740 1 -- 172.21.15.43:0/3581591345 shutdown_connections 2024-04-25T10:42:22.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.985+0000 7fd1b79c3740 1 -- 172.21.15.43:0/3581591345 wait complete. 2024-04-25T10:42:22.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.985+0000 7fd1b79c3740 20 client.18390 trim_cache size 0 max 16384 2024-04-25T10:42:22.992 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:42:22.992 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:42:22.992 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:42:22.992 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:42:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.985+0000 7fd1b0b0d640 1 -- 172.21.15.43:0/2773174972 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd1940afce0 msgr2=0x7fd1940b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.985+0000 7fd1b0b0d640 1 --2- 172.21.15.43:0/2773174972 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd1940afce0 0x7fd1940b21f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x5582edecdbd0 tx=0x7fd1a40046c0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.985+0000 7fd1b0b0d640 1 -- 172.21.15.43:0/2773174972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac10b650 msgr2=0x7fd1ac19aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.985+0000 7fd1b0b0d640 1 --2- 172.21.15.43:0/2773174972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac10b650 0x7fd1ac19aca0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0006010 tx=0x7fd1a00042f0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.986+0000 7fd1b0b0d640 1 -- 172.21.15.43:0/2773174972 shutdown_connections 2024-04-25T10:42:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.986+0000 7fd1b0b0d640 1 --2- 172.21.15.43:0/2773174972 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd1940afce0 0x7fd1940b21f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.986+0000 7fd1b0b0d640 1 --2- 172.21.15.43:0/2773174972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ac10e040 0x7fd1ac19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.986+0000 7fd1b0b0d640 1 --2- 172.21.15.43:0/2773174972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac10b650 0x7fd1ac19aca0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.986+0000 7fd1b0b0d640 1 --2- 172.21.15.43:0/2773174972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1ac104bf0 0x7fd1ac19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:22.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.986+0000 7fd1b0b0d640 1 -- 172.21.15.43:0/2773174972 >> 172.21.15.43:0/2773174972 conn(0x7fd1ac078100 msgr2=0x7fd1ac101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:22.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.986+0000 7fd1b0b0d640 1 -- 172.21.15.43:0/2773174972 shutdown_connections 2024-04-25T10:42:22.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:22.986+0000 7fd1b0b0d640 1 -- 172.21.15.43:0/2773174972 wait complete. 2024-04-25T10:42:22.999 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:42:22.999 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:42:23.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.149+0000 7ff0af70e640 1 -- 172.21.15.43:0/2454713459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0a8106970 msgr2=0x7ff0a8106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.149+0000 7ff0af70e640 1 --2- 172.21.15.43:0/2454713459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0a8106970 0x7ff0a8106da0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7ff09800aa10 tx=0x7ff09802b720 comp rx=0 tx=0).stop 2024-04-25T10:42:23.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.149+0000 7ff0af70e640 1 -- 172.21.15.43:0/2454713459 shutdown_connections 2024-04-25T10:42:23.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.149+0000 7ff0af70e640 1 --2- 172.21.15.43:0/2454713459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0a8107430 0x7ff0a810ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.149+0000 7ff0af70e640 1 --2- 172.21.15.43:0/2454713459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0a8106970 0x7ff0a8106da0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.149+0000 7ff0af70e640 1 --2- 172.21.15.43:0/2454713459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0a81023e0 0x7ff0a81027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.149+0000 7ff0af70e640 1 -- 172.21.15.43:0/2454713459 >> 172.21.15.43:0/2454713459 conn(0x7ff0a80fd840 msgr2=0x7ff0a80ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:23.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.149+0000 7ff0af70e640 1 -- 172.21.15.43:0/2454713459 shutdown_connections 2024-04-25T10:42:23.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.149+0000 7ff0af70e640 1 -- 172.21.15.43:0/2454713459 wait complete. 2024-04-25T10:42:23.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.150+0000 7ff0af70e640 1 Processor -- start 2024-04-25T10:42:23.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.150+0000 7ff0af70e640 1 -- start start 2024-04-25T10:42:23.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.150+0000 7ff0af70e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0a81023e0 0x7ff0a819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.150+0000 7ff0af70e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0a8106970 0x7ff0a819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.150+0000 7ff0af70e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0a8107430 0x7ff0a819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.150+0000 7ff0af70e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a8111ac0 con 0x7ff0a8107430 2024-04-25T10:42:23.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.150+0000 7ff0af70e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a8111940 con 0x7ff0a8106970 2024-04-25T10:42:23.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.150+0000 7ff0af70e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a8111c40 con 0x7ff0a81023e0 2024-04-25T10:42:23.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.150+0000 7ff0adcab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0a8107430 0x7ff0a819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.150+0000 7ff0adcab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0a8107430 0x7ff0a819ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48972/0 (socket says 172.21.15.43:48972) 2024-04-25T10:42:23.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.150+0000 7ff0adcab640 1 -- 172.21.15.43:0/1610826752 learned_addr learned my addr 172.21.15.43:0/1610826752 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:23.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.150+0000 7ff0acca9640 1 --2- 172.21.15.43:0/1610826752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0a8106970 0x7ff0a819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.150+0000 7ff0ad4aa640 1 --2- 172.21.15.43:0/1610826752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0a81023e0 0x7ff0a819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.150+0000 7ff0adcab640 1 -- 172.21.15.43:0/1610826752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0a81023e0 msgr2=0x7ff0a819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.150+0000 7ff0adcab640 1 --2- 172.21.15.43:0/1610826752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0a81023e0 0x7ff0a819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.150+0000 7ff0adcab640 1 -- 172.21.15.43:0/1610826752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0a8106970 msgr2=0x7ff0a819ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.150+0000 7ff0adcab640 1 --2- 172.21.15.43:0/1610826752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0a8106970 0x7ff0a819ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.150+0000 7ff0adcab640 1 -- 172.21.15.43:0/1610826752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a819f580 con 0x7ff0a8107430 2024-04-25T10:42:23.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.150+0000 7ff0ad4aa640 1 --2- 172.21.15.43:0/1610826752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0a81023e0 0x7ff0a819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:23.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.151+0000 7ff0adcab640 1 --2- 172.21.15.43:0/1610826752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0a8107430 0x7ff0a819ef40 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000eaa0 tx=0x7ff0a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:23.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.151+0000 7ff09e7fc640 1 -- 172.21.15.43:0/1610826752 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a000cdb0 con 0x7ff0a8107430 2024-04-25T10:42:23.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.151+0000 7ff09e7fc640 1 -- 172.21.15.43:0/1610826752 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0a0005020 con 0x7ff0a8107430 2024-04-25T10:42:23.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.151+0000 7ff09e7fc640 1 -- 172.21.15.43:0/1610826752 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a00116d0 con 0x7ff0a8107430 2024-04-25T10:42:23.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.151+0000 7ff0af70e640 1 -- 172.21.15.43:0/1610826752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a819f870 con 0x7ff0a8107430 2024-04-25T10:42:23.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.151+0000 7ff0af70e640 1 -- 172.21.15.43:0/1610826752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a81a71f0 con 0x7ff0a8107430 2024-04-25T10:42:23.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.152+0000 7ff0af70e640 1 -- 172.21.15.43:0/1610826752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0a806a190 con 0x7ff0a8107430 2024-04-25T10:42:23.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.154+0000 7ff09e7fc640 1 -- 172.21.15.43:0/1610826752 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7ff0a00051d0 con 0x7ff0a8107430 2024-04-25T10:42:23.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.155+0000 7ff09e7fc640 1 --2- 172.21.15.43:0/1610826752 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff0940afce0 0x7ff0940b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.155+0000 7ff0ad4aa640 1 --2- 172.21.15.43:0/1610826752 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff0940afce0 0x7ff0940b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.155+0000 7ff09e7fc640 1 -- 172.21.15.43:0/1610826752 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7ff0a00df580 con 0x7ff0a8107430 2024-04-25T10:42:23.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.156+0000 7ff0ad4aa640 1 --2- 172.21.15.43:0/1610826752 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff0940afce0 0x7ff0940b21f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40058c0 tx=0x7ff0a4005d30 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:23.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.159+0000 7ff09e7fc640 1 -- 172.21.15.43:0/1610826752 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff0a0016030 con 0x7ff0a8107430 2024-04-25T10:42:23.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.384+0000 7ff0af70e640 1 -- 172.21.15.43:0/1610826752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0a819ba70 con 0x7ff0a8107430 2024-04-25T10:42:23.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.385+0000 7ff09e7fc640 1 -- 172.21.15.43:0/1610826752 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7ff0a00a7e40 con 0x7ff0a8107430 2024-04-25T10:42:23.387 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:42:23.387 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:42:23.387 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:42:23.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.387+0000 7ff0af70e640 1 -- 172.21.15.43:0/1610826752 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff0940afce0 msgr2=0x7ff0940b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.387+0000 7ff0af70e640 1 --2- 172.21.15.43:0/1610826752 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff0940afce0 0x7ff0940b21f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40058c0 tx=0x7ff0a4005d30 comp rx=0 tx=0).stop 2024-04-25T10:42:23.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.387+0000 7ff0af70e640 1 -- 172.21.15.43:0/1610826752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0a8107430 msgr2=0x7ff0a819ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.387+0000 7ff0af70e640 1 --2- 172.21.15.43:0/1610826752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0a8107430 0x7ff0a819ef40 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000eaa0 tx=0x7ff0a000ef60 comp rx=0 tx=0).stop 2024-04-25T10:42:23.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.387+0000 7ff0af70e640 1 -- 172.21.15.43:0/1610826752 shutdown_connections 2024-04-25T10:42:23.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.387+0000 7ff0af70e640 1 --2- 172.21.15.43:0/1610826752 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff0940afce0 0x7ff0940b21f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.387+0000 7ff0af70e640 1 --2- 172.21.15.43:0/1610826752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0a8106970 0x7ff0a819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.387+0000 7ff0af70e640 1 --2- 172.21.15.43:0/1610826752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0a8107430 0x7ff0a819ef40 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.387+0000 7ff0af70e640 1 --2- 172.21.15.43:0/1610826752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0a81023e0 0x7ff0a819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.387+0000 7ff0af70e640 1 -- 172.21.15.43:0/1610826752 >> 172.21.15.43:0/1610826752 conn(0x7ff0a80fd840 msgr2=0x7ff0a810dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:23.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.387+0000 7ff0af70e640 1 -- 172.21.15.43:0/1610826752 shutdown_connections 2024-04-25T10:42:23.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.387+0000 7ff0af70e640 1 -- 172.21.15.43:0/1610826752 wait complete. 2024-04-25T10:42:23.400 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:42:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.512+0000 7f12f2986640 1 -- 172.21.15.43:0/2730678178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12ec102af0 msgr2=0x7f12ec102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.512+0000 7f12f2986640 1 --2- 172.21.15.43:0/2730678178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12ec102af0 0x7f12ec102f00 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009900 tx=0x7f12dc02a410 comp rx=0 tx=0).stop 2024-04-25T10:42:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.512+0000 7f12f2986640 1 -- 172.21.15.43:0/2730678178 shutdown_connections 2024-04-25T10:42:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.512+0000 7f12f2986640 1 --2- 172.21.15.43:0/2730678178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12ec10d380 0x7f12ec10f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.512+0000 7f12f2986640 1 --2- 172.21.15.43:0/2730678178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12ec102af0 0x7f12ec102f00 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.512+0000 7f12f2986640 1 --2- 172.21.15.43:0/2730678178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12ec108a20 0x7f12ec1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.512+0000 7f12f2986640 1 -- 172.21.15.43:0/2730678178 >> 172.21.15.43:0/2730678178 conn(0x7f12ec078100 msgr2=0x7f12ec078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.512+0000 7f12f2986640 1 -- 172.21.15.43:0/2730678178 shutdown_connections 2024-04-25T10:42:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.512+0000 7f12f2986640 1 -- 172.21.15.43:0/2730678178 wait complete. 2024-04-25T10:42:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.513+0000 7f12f2986640 1 Processor -- start 2024-04-25T10:42:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.513+0000 7f12f2986640 1 -- start start 2024-04-25T10:42:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.514+0000 7f12f2986640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12ec102af0 0x7f12ec199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.514+0000 7f12f2986640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12ec108a20 0x7f12ec199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.514+0000 7f12f2986640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12ec10d380 0x7f12ec19dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.514+0000 7f12f2986640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec112be0 con 0x7f12ec10d380 2024-04-25T10:42:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.514+0000 7f12f2986640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec112a60 con 0x7f12ec108a20 2024-04-25T10:42:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.514+0000 7f12f2986640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec112d60 con 0x7f12ec102af0 2024-04-25T10:42:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.514+0000 7f12f0f23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12ec10d380 0x7f12ec19dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.514+0000 7f12f0f23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12ec10d380 0x7f12ec19dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48986/0 (socket says 172.21.15.43:48986) 2024-04-25T10:42:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.514+0000 7f12f0f23640 1 -- 172.21.15.43:0/350570529 learned_addr learned my addr 172.21.15.43:0/350570529 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.514+0000 7f12eb7fe640 1 --2- 172.21.15.43:0/350570529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12ec108a20 0x7f12ec199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.514+0000 7f12ebfff640 1 --2- 172.21.15.43:0/350570529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12ec102af0 0x7f12ec199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.514+0000 7f12f0f23640 1 -- 172.21.15.43:0/350570529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12ec102af0 msgr2=0x7f12ec199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.514+0000 7f12f0f23640 1 --2- 172.21.15.43:0/350570529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12ec102af0 0x7f12ec199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.514+0000 7f12f0f23640 1 -- 172.21.15.43:0/350570529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12ec108a20 msgr2=0x7f12ec199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.514+0000 7f12f0f23640 1 --2- 172.21.15.43:0/350570529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12ec108a20 0x7f12ec199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.514+0000 7f12f0f23640 1 -- 172.21.15.43:0/350570529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ec19e460 con 0x7f12ec10d380 2024-04-25T10:42:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.514+0000 7f12f0f23640 1 --2- 172.21.15.43:0/350570529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12ec10d380 0x7f12ec19dea0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f12e400cd10 tx=0x7f12e400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.514+0000 7f12e97fa640 1 -- 172.21.15.43:0/350570529 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e400f450 con 0x7f12ec10d380 2024-04-25T10:42:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.514+0000 7f12e97fa640 1 -- 172.21.15.43:0/350570529 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12e400faa0 con 0x7f12ec10d380 2024-04-25T10:42:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.514+0000 7f12f2986640 1 -- 172.21.15.43:0/350570529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12ec19e750 con 0x7f12ec10d380 2024-04-25T10:42:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.515+0000 7f12e97fa640 1 -- 172.21.15.43:0/350570529 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e4018b50 con 0x7f12ec10d380 2024-04-25T10:42:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.515+0000 7f12f2986640 1 -- 172.21.15.43:0/350570529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12ec19ec20 con 0x7f12ec10d380 2024-04-25T10:42:23.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.517+0000 7f12e97fa640 1 -- 172.21.15.43:0/350570529 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f12e400fc50 con 0x7f12ec10d380 2024-04-25T10:42:23.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.517+0000 7f12f983e740 1 Processor -- start 2024-04-25T10:42:23.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.517+0000 7f12f983e740 1 -- start start 2024-04-25T10:42:23.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.517+0000 7f12f983e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13afb390 0x55af13e9ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.517+0000 7f12f983e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ad24d0 0x55af13e9ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.517+0000 7f12f983e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13ada6b0 0x55af13eab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.517+0000 7f12f983e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55af13e01d80 con 0x55af13ad24d0 2024-04-25T10:42:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.517+0000 7f12f983e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55af13dbba20 con 0x55af13afb390 2024-04-25T10:42:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.517+0000 7f12f983e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55af13dfdd40 con 0x55af13ada6b0 2024-04-25T10:42:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.518+0000 7f12ebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ad24d0 0x55af13e9ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.518+0000 7f12ebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ad24d0 0x55af13e9ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49002/0 (socket says 172.21.15.43:49002) 2024-04-25T10:42:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.518+0000 7f12ebfff640 1 -- 172.21.15.43:0/4122307467 learned_addr learned my addr 172.21.15.43:0/4122307467 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.518+0000 7f12eb7fe640 1 --2- 172.21.15.43:0/4122307467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13ada6b0 0x55af13eab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.518+0000 7f12eb7fe640 1 --2- 172.21.15.43:0/4122307467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13afb390 0x55af13e9ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.518+0000 7f12ebfff640 1 -- 172.21.15.43:0/4122307467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13ada6b0 msgr2=0x55af13eab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.518+0000 7f12ebfff640 1 --2- 172.21.15.43:0/4122307467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13ada6b0 0x55af13eab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.518+0000 7f12ebfff640 1 -- 172.21.15.43:0/4122307467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13afb390 msgr2=0x55af13e9ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.518+0000 7f12ebfff640 1 --2- 172.21.15.43:0/4122307467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13afb390 0x55af13e9ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.518+0000 7f12ebfff640 1 -- 172.21.15.43:0/4122307467 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af13dc6c30 con 0x55af13ad24d0 2024-04-25T10:42:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.518+0000 7f12eb7fe640 1 --2- 172.21.15.43:0/4122307467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13ada6b0 0x55af13eab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.518+0000 7f12e97fa640 1 --2- 172.21.15.43:0/350570529 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f12d40afc60 0x7f12d40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.518+0000 7f12eb7fe640 1 --2- 172.21.15.43:0/350570529 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f12d40afc60 0x7f12d40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.518+0000 7f12ebfff640 1 --2- 172.21.15.43:0/4122307467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ad24d0 0x55af13e9ef60 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f12e0009ca0 tx=0x7f12e0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93accf951e2fa317 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.518+0000 7f12e97fa640 1 -- 172.21.15.43:0/350570529 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f12e40df370 con 0x7f12ec10d380 2024-04-25T10:42:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.518+0000 7f12d27fc640 1 -- 172.21.15.43:0/4122307467 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e001e070 con 0x55af13ad24d0 2024-04-25T10:42:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.518+0000 7f12d27fc640 1 -- 172.21.15.43:0/4122307467 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12e0019ae0 con 0x55af13ad24d0 2024-04-25T10:42:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.518+0000 7f12d27fc640 1 -- 172.21.15.43:0/4122307467 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e000e950 con 0x55af13ad24d0 2024-04-25T10:42:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.519+0000 7f12f983e740 1 -- 172.21.15.43:0/4122307467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ad24d0 msgr2=0x55af13e9ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.519+0000 7f12f983e740 1 --2- 172.21.15.43:0/4122307467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ad24d0 0x55af13e9ef60 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f12e0009ca0 tx=0x7f12e0011810 comp rx=0 tx=0).stop 2024-04-25T10:42:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.519+0000 7f12eb7fe640 1 --2- 172.21.15.43:0/350570529 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f12d40afc60 0x7f12d40b2170 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009230 tx=0x7f12dc031400 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.519+0000 7f12f983e740 1 -- 172.21.15.43:0/4122307467 shutdown_connections 2024-04-25T10:42:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.519+0000 7f12f983e740 1 --2- 172.21.15.43:0/4122307467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13ada6b0 0x55af13eab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.519+0000 7f12f983e740 1 --2- 172.21.15.43:0/4122307467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ad24d0 0x55af13e9ef60 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.519+0000 7f12f983e740 1 --2- 172.21.15.43:0/4122307467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13afb390 0x55af13e9ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.519+0000 7f12f983e740 1 -- 172.21.15.43:0/4122307467 >> 172.21.15.43:0/4122307467 conn(0x55af13aef4e0 msgr2=0x55af13e9b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.519+0000 7f12f983e740 1 -- 172.21.15.43:0/4122307467 shutdown_connections 2024-04-25T10:42:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.519+0000 7f12f983e740 1 -- 172.21.15.43:0/4122307467 wait complete. 2024-04-25T10:42:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.520+0000 7f12f983e740 1 -- start start 2024-04-25T10:42:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.520+0000 7f12f983e740 1 Processor -- start 2024-04-25T10:42:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.520+0000 7f12f983e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13ad24d0 0x55af13ea9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.520+0000 7f12f983e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13ada6b0 0x55af13ea9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.520+0000 7f12f983e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13afb390 0x55af13eb54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.520+0000 7f12f983e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55af13eabfa0 con 0x55af13afb390 2024-04-25T10:42:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.520+0000 7f12f983e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55af13d88dc0 con 0x55af13ada6b0 2024-04-25T10:42:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.520+0000 7f12f983e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55af13eac120 con 0x55af13ad24d0 2024-04-25T10:42:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.520+0000 7f12f983e740 10 client.0 _ll_register_callbacks cb 0x55af13da4600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.520+0000 7f12f0f23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13afb390 0x55af13eb54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.520+0000 7f12f0f23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13afb390 0x55af13eb54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49004/0 (socket says 172.21.15.43:49004) 2024-04-25T10:42:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.520+0000 7f12f0f23640 1 -- 172.21.15.43:0/2636629483 learned_addr learned my addr 172.21.15.43:0/2636629483 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.521+0000 7f12ebfff640 1 --2- 172.21.15.43:0/2636629483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13ad24d0 0x55af13ea9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.521+0000 7f12eb7fe640 1 --2- 172.21.15.43:0/2636629483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13ada6b0 0x55af13ea9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.521+0000 7f12f0f23640 1 -- 172.21.15.43:0/2636629483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13ad24d0 msgr2=0x55af13ea9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.521+0000 7f12f0f23640 1 --2- 172.21.15.43:0/2636629483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13ad24d0 0x55af13ea9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.521+0000 7f12f0f23640 1 -- 172.21.15.43:0/2636629483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13ada6b0 msgr2=0x55af13ea9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.521+0000 7f12f0f23640 1 --2- 172.21.15.43:0/2636629483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13ada6b0 0x55af13ea9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.521+0000 7f12f0f23640 1 -- 172.21.15.43:0/2636629483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af13b133f0 con 0x55af13afb390 2024-04-25T10:42:23.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.521+0000 7f12eb7fe640 1 --2- 172.21.15.43:0/2636629483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13ada6b0 0x55af13ea9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:23.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.521+0000 7f12f0f23640 1 --2- 172.21.15.43:0/2636629483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13afb390 0x55af13eb54e0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f12e40a74a0 tx=0x7f12e40a6d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72cc21cda48c30f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:23.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.521+0000 7f12cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:23.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.521+0000 7f12cb7fe640 1 -- 172.21.15.43:0/2636629483 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e40c1060 con 0x55af13afb390 2024-04-25T10:42:23.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.521+0000 7f12cb7fe640 1 -- 172.21.15.43:0/2636629483 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f12e400f5f0 con 0x55af13afb390 2024-04-25T10:42:23.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.521+0000 7f12f983e740 1 -- 172.21.15.43:0/2636629483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12e0019ae0 con 0x55af13afb390 2024-04-25T10:42:23.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.521+0000 7f12cb7fe640 1 -- 172.21.15.43:0/2636629483 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f12e40c8050 con 0x55af13afb390 2024-04-25T10:42:23.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.521+0000 7f12f983e740 10 client.18408 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:23.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.521+0000 7f12f983e740 1 -- 172.21.15.43:0/2636629483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55af13bdd7d0 con 0x55af13afb390 2024-04-25T10:42:23.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.522+0000 7f12cb7fe640 1 -- 172.21.15.43:0/2636629483 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12e0019ae0 con 0x55af13afb390 2024-04-25T10:42:23.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.522+0000 7f12cb7fe640 1 -- 172.21.15.43:0/2636629483 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f12e4020030 con 0x55af13afb390 2024-04-25T10:42:23.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.522+0000 7f12f983e740 10 client.18408 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:23.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.522+0000 7f12f983e740 10 client.18408 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:42:23.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.522+0000 7f12f983e740 20 client.18408 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.522+0000 7f12f983e740 1 --2- 172.21.15.43:0/2636629483 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55af13ab9f60 0x55af13ebb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.522+0000 7f12f983e740 4 client.18408 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:42:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.522+0000 7f12f983e740 1 -- 172.21.15.43:0/2636629483 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af13e0f990 con 0x55af13ab9f60 2024-04-25T10:42:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.522+0000 7f12ebfff640 1 --2- 172.21.15.43:0/2636629483 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55af13ab9f60 0x55af13ebb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.523+0000 7f12cb7fe640 10 client.18408 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:42:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.523+0000 7f12ebfff640 1 --2- 172.21.15.43:0/2636629483 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55af13ab9f60 0x55af13ebb920 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73b2c20ceafa7dae server_cookie=ca8a1779b5f155dd in_seq=0 out_seq=0 2024-04-25T10:42:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.523+0000 7f12cb7fe640 1 -- 172.21.15.43:0/2636629483 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55af13e0f990 con 0x55af13ab9f60 2024-04-25T10:42:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.523+0000 7f12cb7fe640 10 client.18408 handle_command_reply: tid=0 2024-04-25T10:42:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.523+0000 7f12f983e740 1 client.18408 shutdown 2024-04-25T10:42:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.524+0000 7f12f983e740 1 -- 172.21.15.43:0/2636629483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13afb390 msgr2=0x55af13eb54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.524+0000 7f12f983e740 1 --2- 172.21.15.43:0/2636629483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13afb390 0x55af13eb54e0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f12e40a74a0 tx=0x7f12e40a6d40 comp rx=0 tx=0).stop 2024-04-25T10:42:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.524+0000 7f12f983e740 1 -- 172.21.15.43:0/2636629483 shutdown_connections 2024-04-25T10:42:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.524+0000 7f12f983e740 1 --2- 172.21.15.43:0/2636629483 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55af13ab9f60 0x55af13ebb920 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.524+0000 7f12f983e740 1 --2- 172.21.15.43:0/2636629483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13ada6b0 0x55af13ea9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.524+0000 7f12f983e740 1 --2- 172.21.15.43:0/2636629483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13afb390 0x55af13eb54e0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.524+0000 7f12f983e740 1 --2- 172.21.15.43:0/2636629483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13ad24d0 0x55af13ea9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.524+0000 7f12cb7fe640 0 client.18408 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:42:23.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.524+0000 7f12f983e740 1 -- 172.21.15.43:0/2636629483 >> 172.21.15.43:0/2636629483 conn(0x55af13aef4e0 msgr2=0x55af13ea0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:23.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.524+0000 7f12f983e740 1 -- 172.21.15.43:0/2636629483 shutdown_connections 2024-04-25T10:42:23.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.524+0000 7f12f983e740 1 -- 172.21.15.43:0/2636629483 wait complete. 2024-04-25T10:42:23.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.524+0000 7f12f983e740 20 client.18408 trim_cache size 0 max 16384 2024-04-25T10:42:23.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.538+0000 7f12f983e740 1 Processor -- start 2024-04-25T10:42:23.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.538+0000 7f12f983e740 1 -- start start 2024-04-25T10:42:23.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.538+0000 7f12f983e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ad24d0 0x55af13ea8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.538+0000 7f12f983e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13ada6b0 0x55af13eb50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.538+0000 7f12f983e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13afb390 0x55af13ea3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.538+0000 7f12f983e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec1affa0 con 0x55af13ad24d0 2024-04-25T10:42:23.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.538+0000 7f12f983e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec1ad870 con 0x55af13afb390 2024-04-25T10:42:23.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.538+0000 7f12f983e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec1ab6e0 con 0x55af13ada6b0 2024-04-25T10:42:23.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.538+0000 7f12ebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ad24d0 0x55af13ea8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.538+0000 7f12ebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ad24d0 0x55af13ea8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49014/0 (socket says 172.21.15.43:49014) 2024-04-25T10:42:23.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.538+0000 7f12ebfff640 1 -- 172.21.15.43:0/1453973253 learned_addr learned my addr 172.21.15.43:0/1453973253 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:23.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.538+0000 7f12f0f23640 1 --2- 172.21.15.43:0/1453973253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13afb390 0x55af13ea3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.538+0000 7f12eb7fe640 1 --2- 172.21.15.43:0/1453973253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13ada6b0 0x55af13eb50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.538+0000 7f12ebfff640 1 -- 172.21.15.43:0/1453973253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13ada6b0 msgr2=0x55af13eb50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.538+0000 7f12ebfff640 1 --2- 172.21.15.43:0/1453973253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13ada6b0 0x55af13eb50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.538+0000 7f12ebfff640 1 -- 172.21.15.43:0/1453973253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13afb390 msgr2=0x55af13ea3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.539+0000 7f12ebfff640 1 --2- 172.21.15.43:0/1453973253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13afb390 0x55af13ea3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.539+0000 7f12ebfff640 1 -- 172.21.15.43:0/1453973253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af13e9f130 con 0x55af13ad24d0 2024-04-25T10:42:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.539+0000 7f12f0f23640 1 --2- 172.21.15.43:0/1453973253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13afb390 0x55af13ea3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.539+0000 7f12ebfff640 1 --2- 172.21.15.43:0/1453973253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ad24d0 0x55af13ea8220 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x55af13e9c160 tx=0x7f12e00099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349b1bb094020bc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.539+0000 7f12d1ffb640 1 -- 172.21.15.43:0/1453973253 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e001e040 con 0x55af13ad24d0 2024-04-25T10:42:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.539+0000 7f12d1ffb640 1 -- 172.21.15.43:0/1453973253 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12e0030030 con 0x55af13ad24d0 2024-04-25T10:42:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.539+0000 7f12d1ffb640 1 -- 172.21.15.43:0/1453973253 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0005390 con 0x55af13ad24d0 2024-04-25T10:42:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.540+0000 7f12f983e740 1 -- 172.21.15.43:0/1453973253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ad24d0 msgr2=0x55af13ea8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.540+0000 7f12f983e740 1 --2- 172.21.15.43:0/1453973253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ad24d0 0x55af13ea8220 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x55af13e9c160 tx=0x7f12e00099e0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.540+0000 7f12f983e740 1 -- 172.21.15.43:0/1453973253 shutdown_connections 2024-04-25T10:42:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.540+0000 7f12f983e740 1 --2- 172.21.15.43:0/1453973253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13afb390 0x55af13ea3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.540+0000 7f12f983e740 1 --2- 172.21.15.43:0/1453973253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13ada6b0 0x55af13eb50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.540+0000 7f12f983e740 1 --2- 172.21.15.43:0/1453973253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ad24d0 0x55af13ea8220 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.540+0000 7f12f983e740 1 -- 172.21.15.43:0/1453973253 >> 172.21.15.43:0/1453973253 conn(0x55af13aef4e0 msgr2=0x55af13ea7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.540+0000 7f12f983e740 1 -- 172.21.15.43:0/1453973253 shutdown_connections 2024-04-25T10:42:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.540+0000 7f12f983e740 1 -- 172.21.15.43:0/1453973253 wait complete. 2024-04-25T10:42:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.540+0000 7f12f983e740 1 -- start start 2024-04-25T10:42:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.541+0000 7f12f983e740 1 Processor -- start 2024-04-25T10:42:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.541+0000 7f12f983e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13ad24d0 0x55af13eb16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.541+0000 7f12f983e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ada6b0 0x55af13eaca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.541+0000 7f12f983e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13afb390 0x55af13eacf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.541+0000 7f12f983e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec1abd30 con 0x55af13ada6b0 2024-04-25T10:42:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.541+0000 7f12f983e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec1ad1a0 con 0x55af13ad24d0 2024-04-25T10:42:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.541+0000 7f12f983e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec1ac400 con 0x55af13afb390 2024-04-25T10:42:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.541+0000 7f12f983e740 10 client.0 _ll_register_callbacks cb 0x7f12e4018510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.541+0000 7f12eb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ada6b0 0x55af13eaca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.541+0000 7f12eb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ada6b0 0x55af13eaca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49024/0 (socket says 172.21.15.43:49024) 2024-04-25T10:42:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.541+0000 7f12eb7fe640 1 -- 172.21.15.43:0/4213246384 learned_addr learned my addr 172.21.15.43:0/4213246384 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.541+0000 7f12ebfff640 1 --2- 172.21.15.43:0/4213246384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13ad24d0 0x55af13eb16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.541+0000 7f12f0f23640 1 --2- 172.21.15.43:0/4213246384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13afb390 0x55af13eacf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.541+0000 7f12eb7fe640 1 -- 172.21.15.43:0/4213246384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13afb390 msgr2=0x55af13eacf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.541+0000 7f12eb7fe640 1 --2- 172.21.15.43:0/4213246384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13afb390 0x55af13eacf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.541+0000 7f12eb7fe640 1 -- 172.21.15.43:0/4213246384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13ad24d0 msgr2=0x55af13eb16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.541+0000 7f12eb7fe640 1 --2- 172.21.15.43:0/4213246384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13ad24d0 0x55af13eb16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.541+0000 7f12eb7fe640 1 -- 172.21.15.43:0/4213246384 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af13ead480 con 0x55af13ada6b0 2024-04-25T10:42:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.542+0000 7f12eb7fe640 1 --2- 172.21.15.43:0/4213246384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ada6b0 0x55af13eaca00 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009900 tx=0x7f12dc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61aa35b354afcb2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.542+0000 7f12cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.542+0000 7f12cb7fe640 1 -- 172.21.15.43:0/4213246384 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc043070 con 0x55af13ada6b0 2024-04-25T10:42:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.542+0000 7f12f983e740 1 -- 172.21.15.43:0/4213246384 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12e0030030 con 0x55af13ada6b0 2024-04-25T10:42:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.542+0000 7f12cb7fe640 1 -- 172.21.15.43:0/4213246384 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f12dc0037f0 con 0x55af13ada6b0 2024-04-25T10:42:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.542+0000 7f12cb7fe640 1 -- 172.21.15.43:0/4213246384 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f12dc035040 con 0x55af13ada6b0 2024-04-25T10:42:23.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.542+0000 7f12f983e740 10 client.18414 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:23.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.542+0000 7f12f983e740 1 -- 172.21.15.43:0/4213246384 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55af13f237f0 con 0x55af13ada6b0 2024-04-25T10:42:23.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.542+0000 7f12cb7fe640 1 -- 172.21.15.43:0/4213246384 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12e0030030 con 0x55af13ada6b0 2024-04-25T10:42:23.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.542+0000 7f12cb7fe640 1 -- 172.21.15.43:0/4213246384 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f12dc052070 con 0x55af13ada6b0 2024-04-25T10:42:23.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.542+0000 7f12f983e740 10 client.18414 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:23.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.542+0000 7f12f983e740 10 client.18414 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:42:23.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.542+0000 7f12f983e740 20 client.18414 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:23.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.542+0000 7f12f983e740 1 --2- 172.21.15.43:0/4213246384 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55af13a8f3a0 0x55af13f25fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.542+0000 7f12f983e740 4 client.18414 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:42:23.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.542+0000 7f12f983e740 1 -- 172.21.15.43:0/4213246384 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af13e09800 con 0x55af13a8f3a0 2024-04-25T10:42:23.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.543+0000 7f12ebfff640 1 --2- 172.21.15.43:0/4213246384 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55af13a8f3a0 0x55af13f25fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.543+0000 7f12cb7fe640 10 client.18414 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:42:23.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.543+0000 7f12ebfff640 1 --2- 172.21.15.43:0/4213246384 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55af13a8f3a0 0x55af13f25fa0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca7088e41a979e0c server_cookie=8b4e9cc47ef438bd in_seq=0 out_seq=0 2024-04-25T10:42:23.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.543+0000 7f12cb7fe640 1 -- 172.21.15.43:0/4213246384 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af13e09800 con 0x55af13a8f3a0 2024-04-25T10:42:23.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.543+0000 7f12cb7fe640 10 client.18414 handle_command_reply: tid=0 2024-04-25T10:42:23.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.543+0000 7f12f983e740 1 client.18414 shutdown 2024-04-25T10:42:23.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.543+0000 7f12f983e740 1 -- 172.21.15.43:0/4213246384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ada6b0 msgr2=0x55af13eaca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.543+0000 7f12f983e740 1 --2- 172.21.15.43:0/4213246384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ada6b0 0x55af13eaca00 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009900 tx=0x7f12dc000bc0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.543+0000 7f12f983e740 1 -- 172.21.15.43:0/4213246384 shutdown_connections 2024-04-25T10:42:23.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.543+0000 7f12f983e740 1 --2- 172.21.15.43:0/4213246384 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55af13a8f3a0 0x55af13f25fa0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.543+0000 7f12f983e740 1 --2- 172.21.15.43:0/4213246384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af13afb390 0x55af13eacf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.543+0000 7f12f983e740 1 --2- 172.21.15.43:0/4213246384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af13ada6b0 0x55af13eaca00 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.543+0000 7f12f983e740 1 --2- 172.21.15.43:0/4213246384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af13ad24d0 0x55af13eb16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.543+0000 7f12cb7fe640 0 client.18414 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:42:23.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.543+0000 7f12f983e740 1 -- 172.21.15.43:0/4213246384 >> 172.21.15.43:0/4213246384 conn(0x55af13aef4e0 msgr2=0x55af13e97610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:23.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.544+0000 7f12f983e740 1 -- 172.21.15.43:0/4213246384 shutdown_connections 2024-04-25T10:42:23.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.544+0000 7f12f983e740 1 -- 172.21.15.43:0/4213246384 wait complete. 2024-04-25T10:42:23.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.544+0000 7f12f983e740 20 client.18414 trim_cache size 0 max 16384 2024-04-25T10:42:23.553 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:42:23.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.544+0000 7f12f2986640 1 -- 172.21.15.43:0/350570529 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f12d40afc60 msgr2=0x7f12d40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.544+0000 7f12f2986640 1 --2- 172.21.15.43:0/350570529 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f12d40afc60 0x7f12d40b2170 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009230 tx=0x7f12dc031400 comp rx=0 tx=0).stop 2024-04-25T10:42:23.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.544+0000 7f12f2986640 1 -- 172.21.15.43:0/350570529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12ec10d380 msgr2=0x7f12ec19dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.544+0000 7f12f2986640 1 --2- 172.21.15.43:0/350570529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12ec10d380 0x7f12ec19dea0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f12e400cd10 tx=0x7f12e400a620 comp rx=0 tx=0).stop 2024-04-25T10:42:23.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.544+0000 7f12f2986640 1 -- 172.21.15.43:0/350570529 shutdown_connections 2024-04-25T10:42:23.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.544+0000 7f12f2986640 1 --2- 172.21.15.43:0/350570529 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f12d40afc60 0x7f12d40b2170 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.544+0000 7f12f2986640 1 --2- 172.21.15.43:0/350570529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12ec108a20 0x7f12ec199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.544+0000 7f12f2986640 1 --2- 172.21.15.43:0/350570529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12ec10d380 0x7f12ec19dea0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.544+0000 7f12f2986640 1 --2- 172.21.15.43:0/350570529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12ec102af0 0x7f12ec199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.544+0000 7f12f2986640 1 -- 172.21.15.43:0/350570529 >> 172.21.15.43:0/350570529 conn(0x7f12ec078100 msgr2=0x7f12ec1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:23.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.544+0000 7f12f2986640 1 -- 172.21.15.43:0/350570529 shutdown_connections 2024-04-25T10:42:23.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.544+0000 7f12f2986640 1 -- 172.21.15.43:0/350570529 wait complete. 2024-04-25T10:42:23.557 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:42:23.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.668+0000 7f29544ee640 1 -- 172.21.15.43:0/706163577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c104cd0 msgr2=0x7f294c1050e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.668+0000 7f29544ee640 1 --2- 172.21.15.43:0/706163577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c104cd0 0x7f294c1050e0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f2948009900 tx=0x7f294802a410 comp rx=0 tx=0).stop 2024-04-25T10:42:23.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.669+0000 7f29544ee640 1 -- 172.21.15.43:0/706163577 shutdown_connections 2024-04-25T10:42:23.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.669+0000 7f29544ee640 1 --2- 172.21.15.43:0/706163577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f294c10b730 0x7f294c10db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.669+0000 7f29544ee640 1 --2- 172.21.15.43:0/706163577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f294c10e090 0x7f294c110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.669+0000 7f29544ee640 1 --2- 172.21.15.43:0/706163577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c104cd0 0x7f294c1050e0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.669+0000 7f29544ee640 1 -- 172.21.15.43:0/706163577 >> 172.21.15.43:0/706163577 conn(0x7f294c078110 msgr2=0x7f294c078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.669+0000 7f29544ee640 1 -- 172.21.15.43:0/706163577 shutdown_connections 2024-04-25T10:42:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.669+0000 7f29544ee640 1 -- 172.21.15.43:0/706163577 wait complete. 2024-04-25T10:42:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.669+0000 7f29544ee640 1 Processor -- start 2024-04-25T10:42:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.669+0000 7f29544ee640 1 -- start start 2024-04-25T10:42:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f29544ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c104cd0 0x7f294c102590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f29544ee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f294c10b730 0x7f294c102ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f29544ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f294c10e090 0x7f294c103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f29544ee640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f294c113ce0 con 0x7f294c104cd0 2024-04-25T10:42:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f29544ee640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f294c113b60 con 0x7f294c10e090 2024-04-25T10:42:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f29544ee640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f294c113e60 con 0x7f294c10b730 2024-04-25T10:42:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f295228a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c104cd0 0x7f294c102590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f295228a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c104cd0 0x7f294c102590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49044/0 (socket says 172.21.15.43:49044) 2024-04-25T10:42:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f295228a640 1 -- 172.21.15.43:0/1214249402 learned_addr learned my addr 172.21.15.43:0/1214249402 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:23.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f2952a8b640 1 --2- 172.21.15.43:0/1214249402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f294c10e090 0x7f294c103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f2951a89640 1 --2- 172.21.15.43:0/1214249402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f294c10b730 0x7f294c102ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f295228a640 1 -- 172.21.15.43:0/1214249402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f294c10b730 msgr2=0x7f294c102ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f295228a640 1 --2- 172.21.15.43:0/1214249402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f294c10b730 0x7f294c102ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f295228a640 1 -- 172.21.15.43:0/1214249402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f294c10e090 msgr2=0x7f294c103010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f295228a640 1 --2- 172.21.15.43:0/1214249402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f294c10e090 0x7f294c103010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f295228a640 1 -- 172.21.15.43:0/1214249402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294c1a8aa0 con 0x7f294c104cd0 2024-04-25T10:42:23.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f2951a89640 1 --2- 172.21.15.43:0/1214249402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f294c10b730 0x7f294c102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:23.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f295228a640 1 --2- 172.21.15.43:0/1214249402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c104cd0 0x7f294c102590 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f2948005940 tx=0x7f29480040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:23.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f29437fe640 1 -- 172.21.15.43:0/1214249402 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294803f030 con 0x7f294c104cd0 2024-04-25T10:42:23.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f29544ee640 1 -- 172.21.15.43:0/1214249402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294c1a8d30 con 0x7f294c104cd0 2024-04-25T10:42:23.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f29437fe640 1 -- 172.21.15.43:0/1214249402 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29480044c0 con 0x7f294c104cd0 2024-04-25T10:42:23.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.670+0000 7f29437fe640 1 -- 172.21.15.43:0/1214249402 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948035070 con 0x7f294c104cd0 2024-04-25T10:42:23.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.671+0000 7f29544ee640 1 -- 172.21.15.43:0/1214249402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f294c1a91c0 con 0x7f294c104cd0 2024-04-25T10:42:23.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.673+0000 7f29437fe640 1 -- 172.21.15.43:0/1214249402 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f2948042030 con 0x7f294c104cd0 2024-04-25T10:42:23.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.673+0000 7f295b39e740 1 Processor -- start 2024-04-25T10:42:23.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.673+0000 7f295b39e740 1 -- start start 2024-04-25T10:42:23.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f295b39e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 0x55bf0063f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f295b39e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf002754d0 0x55bf0063fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f295b39e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf0027d6b0 0x55bf0064c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f295b39e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bf005a4d80 con 0x55bf0029e390 2024-04-25T10:42:23.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f295b39e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bf0055ea20 con 0x55bf002754d0 2024-04-25T10:42:23.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f295b39e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bf005a0d40 con 0x55bf0027d6b0 2024-04-25T10:42:23.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f2952a8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 0x55bf0063f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f2952a8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 0x55bf0063f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49056/0 (socket says 172.21.15.43:49056) 2024-04-25T10:42:23.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f2952a8b640 1 -- 172.21.15.43:0/3516122514 learned_addr learned my addr 172.21.15.43:0/3516122514 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:23.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f29437fe640 1 --2- 172.21.15.43:0/1214249402 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f29380afc60 0x7f29380b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f2951a89640 1 --2- 172.21.15.43:0/3516122514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf0027d6b0 0x55bf0064c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f295228a640 1 --2- 172.21.15.43:0/3516122514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf002754d0 0x55bf0063fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f2952a8b640 1 -- 172.21.15.43:0/3516122514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf0027d6b0 msgr2=0x55bf0064c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f2952a8b640 1 --2- 172.21.15.43:0/3516122514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf0027d6b0 0x55bf0064c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f2952a8b640 1 -- 172.21.15.43:0/3516122514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf002754d0 msgr2=0x55bf0063fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f2952a8b640 1 --2- 172.21.15.43:0/3516122514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf002754d0 0x55bf0063fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f2952a8b640 1 -- 172.21.15.43:0/3516122514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf00569c30 con 0x55bf0029e390 2024-04-25T10:42:23.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f2951a89640 1 --2- 172.21.15.43:0/3516122514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf0027d6b0 0x55bf0064c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:23.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f295228a640 1 --2- 172.21.15.43:0/3516122514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf002754d0 0x55bf0063fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:23.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f29437fe640 1 -- 172.21.15.43:0/1214249402 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f29480ff450 con 0x7f294c104cd0 2024-04-25T10:42:23.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.674+0000 7f2952a8b640 1 --2- 172.21.15.43:0/1214249402 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f29380afc60 0x7f29380b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.675+0000 7f2952a8b640 1 --2- 172.21.15.43:0/3516122514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 0x55bf0063f810 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f2944011590 tx=0x7f2944011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09e65507ba4f1a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:23.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.675+0000 7f2940bfe640 1 -- 172.21.15.43:0/3516122514 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944017050 con 0x55bf0029e390 2024-04-25T10:42:23.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.675+0000 7f2940bfe640 1 -- 172.21.15.43:0/3516122514 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f294401bb90 con 0x55bf0029e390 2024-04-25T10:42:23.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.675+0000 7f2940bfe640 1 -- 172.21.15.43:0/3516122514 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944025d80 con 0x55bf0029e390 2024-04-25T10:42:23.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.675+0000 7f2952a8b640 1 --2- 172.21.15.43:0/1214249402 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f29380afc60 0x7f29380b2170 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2944022000 tx=0x7f2944023000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:23.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.676+0000 7f295b39e740 1 -- 172.21.15.43:0/3516122514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 msgr2=0x55bf0063f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.676+0000 7f295b39e740 1 --2- 172.21.15.43:0/3516122514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 0x55bf0063f810 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f2944011590 tx=0x7f2944011a50 comp rx=0 tx=0).stop 2024-04-25T10:42:23.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.676+0000 7f295b39e740 1 -- 172.21.15.43:0/3516122514 shutdown_connections 2024-04-25T10:42:23.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.676+0000 7f295b39e740 1 --2- 172.21.15.43:0/3516122514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf002754d0 0x55bf0063fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.676+0000 7f295b39e740 1 --2- 172.21.15.43:0/3516122514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf0027d6b0 0x55bf0064c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.676+0000 7f295b39e740 1 --2- 172.21.15.43:0/3516122514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 0x55bf0063f810 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.676+0000 7f295b39e740 1 -- 172.21.15.43:0/3516122514 >> 172.21.15.43:0/3516122514 conn(0x55bf002924e0 msgr2=0x55bf0063bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:23.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.676+0000 7f295b39e740 1 -- 172.21.15.43:0/3516122514 shutdown_connections 2024-04-25T10:42:23.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.676+0000 7f295b39e740 1 -- 172.21.15.43:0/3516122514 wait complete. 2024-04-25T10:42:23.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.676+0000 7f295b39e740 1 -- start start 2024-04-25T10:42:23.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.676+0000 7f295b39e740 1 Processor -- start 2024-04-25T10:42:23.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f295b39e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf002754d0 0x55bf0064a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f295b39e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf0027d6b0 0x55bf0064a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f295b39e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 0x55bf00656210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f295b39e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bf0064cd90 con 0x55bf0029e390 2024-04-25T10:42:23.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f295b39e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bf0052bdc0 con 0x55bf0027d6b0 2024-04-25T10:42:23.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f295b39e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bf0064cf10 con 0x55bf002754d0 2024-04-25T10:42:23.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f295b39e740 10 client.0 _ll_register_callbacks cb 0x55bf00547600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:23.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f2952a8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 0x55bf00656210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f2952a8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 0x55bf00656210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49070/0 (socket says 172.21.15.43:49070) 2024-04-25T10:42:23.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f2952a8b640 1 -- 172.21.15.43:0/3723952246 learned_addr learned my addr 172.21.15.43:0/3723952246 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:23.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f295228a640 1 --2- 172.21.15.43:0/3723952246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf002754d0 0x55bf0064a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f2951a89640 1 --2- 172.21.15.43:0/3723952246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf0027d6b0 0x55bf0064a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f2952a8b640 1 -- 172.21.15.43:0/3723952246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf002754d0 msgr2=0x55bf0064a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f2952a8b640 1 --2- 172.21.15.43:0/3723952246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf002754d0 0x55bf0064a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f2952a8b640 1 -- 172.21.15.43:0/3723952246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf0027d6b0 msgr2=0x55bf0064a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f2952a8b640 1 --2- 172.21.15.43:0/3723952246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf0027d6b0 0x55bf0064a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f2952a8b640 1 -- 172.21.15.43:0/3723952246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf002b63f0 con 0x55bf0029e390 2024-04-25T10:42:23.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f2952a8b640 1 --2- 172.21.15.43:0/3723952246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 0x55bf00656210 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f2944000f80 tx=0x7f2944009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8939a74c2b489845 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:23.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f2935ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:23.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f2935ffb640 1 -- 172.21.15.43:0/3723952246 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944025640 con 0x55bf0029e390 2024-04-25T10:42:23.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f295b39e740 1 -- 172.21.15.43:0/3723952246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f294401bb90 con 0x55bf0029e390 2024-04-25T10:42:23.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.677+0000 7f2935ffb640 1 -- 172.21.15.43:0/3723952246 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f29440257e0 con 0x55bf0029e390 2024-04-25T10:42:23.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.678+0000 7f2935ffb640 1 -- 172.21.15.43:0/3723952246 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2944017020 con 0x55bf0029e390 2024-04-25T10:42:23.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.678+0000 7f295b39e740 10 client.18426 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:23.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.678+0000 7f295b39e740 1 -- 172.21.15.43:0/3723952246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bf003807d0 con 0x55bf0029e390 2024-04-25T10:42:23.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.678+0000 7f2935ffb640 1 -- 172.21.15.43:0/3723952246 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f294401bb90 con 0x55bf0029e390 2024-04-25T10:42:23.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.678+0000 7f2935ffb640 1 -- 172.21.15.43:0/3723952246 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f294402d8a0 con 0x55bf0029e390 2024-04-25T10:42:23.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.678+0000 7f295b39e740 10 client.18426 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:23.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.678+0000 7f295b39e740 10 client.18426 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:42:23.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.678+0000 7f295b39e740 20 client.18426 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:23.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.678+0000 7f295b39e740 1 --2- 172.21.15.43:0/3723952246 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bf0025cf60 0x55bf0065c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.678+0000 7f295b39e740 4 client.18426 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:42:23.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.678+0000 7f295b39e740 1 -- 172.21.15.43:0/3723952246 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf005b2990 con 0x55bf0025cf60 2024-04-25T10:42:23.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.678+0000 7f295228a640 1 --2- 172.21.15.43:0/3723952246 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bf0025cf60 0x55bf0065c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.679+0000 7f2935ffb640 10 client.18426 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:42:23.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.679+0000 7f295228a640 1 --2- 172.21.15.43:0/3723952246 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bf0025cf60 0x55bf0065c510 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb99560cd15ea671 server_cookie=e37533283911fd03 in_seq=0 out_seq=0 2024-04-25T10:42:23.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.680+0000 7f2935ffb640 1 -- 172.21.15.43:0/3723952246 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bf005b2990 con 0x55bf0025cf60 2024-04-25T10:42:23.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.680+0000 7f2935ffb640 10 client.18426 handle_command_reply: tid=0 2024-04-25T10:42:23.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.680+0000 7f295b39e740 1 client.18426 shutdown 2024-04-25T10:42:23.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.680+0000 7f295b39e740 1 -- 172.21.15.43:0/3723952246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 msgr2=0x55bf00656210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.680+0000 7f295b39e740 1 --2- 172.21.15.43:0/3723952246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 0x55bf00656210 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f2944000f80 tx=0x7f2944009cc0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.680+0000 7f295b39e740 1 -- 172.21.15.43:0/3723952246 shutdown_connections 2024-04-25T10:42:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.680+0000 7f295b39e740 1 --2- 172.21.15.43:0/3723952246 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bf0025cf60 0x55bf0065c510 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.680+0000 7f295b39e740 1 --2- 172.21.15.43:0/3723952246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf0027d6b0 0x55bf0064a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.680+0000 7f295b39e740 1 --2- 172.21.15.43:0/3723952246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 0x55bf00656210 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.680+0000 7f2935ffb640 0 client.18426 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:42:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.680+0000 7f295b39e740 1 --2- 172.21.15.43:0/3723952246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf002754d0 0x55bf0064a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.680+0000 7f295b39e740 1 -- 172.21.15.43:0/3723952246 >> 172.21.15.43:0/3723952246 conn(0x55bf002924e0 msgr2=0x55bf00641530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.680+0000 7f295b39e740 1 -- 172.21.15.43:0/3723952246 shutdown_connections 2024-04-25T10:42:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.680+0000 7f295b39e740 1 -- 172.21.15.43:0/3723952246 wait complete. 2024-04-25T10:42:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.680+0000 7f295b39e740 20 client.18426 trim_cache size 0 max 16384 2024-04-25T10:42:23.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.694+0000 7f295b39e740 1 Processor -- start 2024-04-25T10:42:23.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.694+0000 7f295b39e740 1 -- start start 2024-04-25T10:42:23.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.694+0000 7f295b39e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf002754d0 0x55bf0063fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.694+0000 7f295b39e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0027d6b0 0x55bf00655e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.694+0000 7f295b39e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf0029e390 0x55bf00644430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.694+0000 7f295b39e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1b44d0 con 0x55bf0027d6b0 2024-04-25T10:42:23.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.694+0000 7f295b39e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1b1da0 con 0x55bf0029e390 2024-04-25T10:42:23.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.694+0000 7f295b39e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1afc10 con 0x55bf002754d0 2024-04-25T10:42:23.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.694+0000 7f2951a89640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0027d6b0 0x55bf00655e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.694+0000 7f2951a89640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0027d6b0 0x55bf00655e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49080/0 (socket says 172.21.15.43:49080) 2024-04-25T10:42:23.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.694+0000 7f2951a89640 1 -- 172.21.15.43:0/1627709044 learned_addr learned my addr 172.21.15.43:0/1627709044 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:23.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.694+0000 7f2952a8b640 1 --2- 172.21.15.43:0/1627709044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf0029e390 0x55bf00644430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.694+0000 7f295228a640 1 --2- 172.21.15.43:0/1627709044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf002754d0 0x55bf0063fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.694+0000 7f2951a89640 1 -- 172.21.15.43:0/1627709044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf002754d0 msgr2=0x55bf0063fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.694+0000 7f2951a89640 1 --2- 172.21.15.43:0/1627709044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf002754d0 0x55bf0063fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.694+0000 7f2951a89640 1 -- 172.21.15.43:0/1627709044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf0029e390 msgr2=0x55bf00644430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.694+0000 7f2951a89640 1 --2- 172.21.15.43:0/1627709044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf0029e390 0x55bf00644430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.694+0000 7f2951a89640 1 -- 172.21.15.43:0/1627709044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf0064cbe0 con 0x55bf0027d6b0 2024-04-25T10:42:23.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.695+0000 7f295228a640 1 --2- 172.21.15.43:0/1627709044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf002754d0 0x55bf0063fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:23.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.695+0000 7f2951a89640 1 --2- 172.21.15.43:0/1627709044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0027d6b0 0x55bf00655e70 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f293c009ae0 tx=0x7f293c0123f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da6873b4958f6f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:23.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.695+0000 7f2937fff640 1 -- 172.21.15.43:0/1627709044 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c020070 con 0x55bf0027d6b0 2024-04-25T10:42:23.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.695+0000 7f2937fff640 1 -- 172.21.15.43:0/1627709044 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f293c01b410 con 0x55bf0027d6b0 2024-04-25T10:42:23.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.695+0000 7f2937fff640 1 -- 172.21.15.43:0/1627709044 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c010630 con 0x55bf0027d6b0 2024-04-25T10:42:23.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.696+0000 7f295b39e740 1 -- 172.21.15.43:0/1627709044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0027d6b0 msgr2=0x55bf00655e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.696+0000 7f295b39e740 1 --2- 172.21.15.43:0/1627709044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0027d6b0 0x55bf00655e70 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f293c009ae0 tx=0x7f293c0123f0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.696+0000 7f295b39e740 1 -- 172.21.15.43:0/1627709044 shutdown_connections 2024-04-25T10:42:23.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.696+0000 7f295b39e740 1 --2- 172.21.15.43:0/1627709044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf0029e390 0x55bf00644430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.696+0000 7f295b39e740 1 --2- 172.21.15.43:0/1627709044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0027d6b0 0x55bf00655e70 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.696+0000 7f295b39e740 1 --2- 172.21.15.43:0/1627709044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf002754d0 0x55bf0063fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.696+0000 7f295b39e740 1 -- 172.21.15.43:0/1627709044 >> 172.21.15.43:0/1627709044 conn(0x55bf002924e0 msgr2=0x55bf0063f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:23.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.696+0000 7f295b39e740 1 -- 172.21.15.43:0/1627709044 shutdown_connections 2024-04-25T10:42:23.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.696+0000 7f295b39e740 1 -- 172.21.15.43:0/1627709044 wait complete. 2024-04-25T10:42:23.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.696+0000 7f295b39e740 1 -- start start 2024-04-25T10:42:23.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.696+0000 7f295b39e740 1 Processor -- start 2024-04-25T10:42:23.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.696+0000 7f295b39e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf002754d0 0x55bf0064d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.696+0000 7f295b39e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf0027d6b0 0x55bf0064db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.696+0000 7f295b39e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 0x55bf006bc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.696+0000 7f295b39e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1b0260 con 0x55bf0029e390 2024-04-25T10:42:23.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.696+0000 7f295b39e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1b16d0 con 0x55bf002754d0 2024-04-25T10:42:23.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.696+0000 7f295b39e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1b0930 con 0x55bf0027d6b0 2024-04-25T10:42:23.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.696+0000 7f295b39e740 10 client.0 _ll_register_callbacks cb 0x7f2944011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:23.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.696+0000 7f2952a8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 0x55bf006bc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.697+0000 7f2952a8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 0x55bf006bc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49090/0 (socket says 172.21.15.43:49090) 2024-04-25T10:42:23.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.697+0000 7f2952a8b640 1 -- 172.21.15.43:0/2309267108 learned_addr learned my addr 172.21.15.43:0/2309267108 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:23.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.697+0000 7f295228a640 1 --2- 172.21.15.43:0/2309267108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf002754d0 0x55bf0064d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.697+0000 7f2951a89640 1 --2- 172.21.15.43:0/2309267108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf0027d6b0 0x55bf0064db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.697+0000 7f2952a8b640 1 -- 172.21.15.43:0/2309267108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf0027d6b0 msgr2=0x55bf0064db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.697+0000 7f2952a8b640 1 --2- 172.21.15.43:0/2309267108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf0027d6b0 0x55bf0064db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.697+0000 7f2952a8b640 1 -- 172.21.15.43:0/2309267108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf002754d0 msgr2=0x55bf0064d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.697+0000 7f2952a8b640 1 --2- 172.21.15.43:0/2309267108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf002754d0 0x55bf0064d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.697+0000 7f2952a8b640 1 -- 172.21.15.43:0/2309267108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf006bcb60 con 0x55bf0029e390 2024-04-25T10:42:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.697+0000 7f295228a640 1 --2- 172.21.15.43:0/2309267108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf002754d0 0x55bf0064d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.697+0000 7f2951a89640 1 --2- 172.21.15.43:0/2309267108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf0027d6b0 0x55bf0064db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.697+0000 7f2952a8b640 1 --2- 172.21.15.43:0/2309267108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 0x55bf006bc620 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f2944005d70 tx=0x7f29440096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496d5ef3b1355ecd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.697+0000 7f2935ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.697+0000 7f2935ffb640 1 -- 172.21.15.43:0/2309267108 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944044070 con 0x55bf0029e390 2024-04-25T10:42:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.697+0000 7f2935ffb640 1 -- 172.21.15.43:0/2309267108 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f294402d450 con 0x55bf0029e390 2024-04-25T10:42:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.697+0000 7f295b39e740 1 -- 172.21.15.43:0/2309267108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f293c01b410 con 0x55bf0029e390 2024-04-25T10:42:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.697+0000 7f2935ffb640 1 -- 172.21.15.43:0/2309267108 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2944009a50 con 0x55bf0029e390 2024-04-25T10:42:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.697+0000 7f295b39e740 10 client.18432 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.697+0000 7f295b39e740 1 -- 172.21.15.43:0/2309267108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bf006c3dc0 con 0x55bf0029e390 2024-04-25T10:42:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.697+0000 7f2935ffb640 1 -- 172.21.15.43:0/2309267108 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f293c01b410 con 0x55bf0029e390 2024-04-25T10:42:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.698+0000 7f2935ffb640 1 -- 172.21.15.43:0/2309267108 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2944047030 con 0x55bf0029e390 2024-04-25T10:42:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.698+0000 7f295b39e740 10 client.18432 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.698+0000 7f295b39e740 10 client.18432 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:42:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.698+0000 7f295b39e740 20 client.18432 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.698+0000 7f295b39e740 1 --2- 172.21.15.43:0/2309267108 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bf002323a0 0x55bf006c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.698+0000 7f295b39e740 4 client.18432 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:42:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.698+0000 7f295b39e740 1 -- 172.21.15.43:0/2309267108 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf005ac800 con 0x55bf002323a0 2024-04-25T10:42:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.698+0000 7f295228a640 1 --2- 172.21.15.43:0/2309267108 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bf002323a0 0x55bf006c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.699+0000 7f2935ffb640 10 client.18432 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:42:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.699+0000 7f295228a640 1 --2- 172.21.15.43:0/2309267108 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bf002323a0 0x55bf006c6540 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b275f895f464def server_cookie=de1fbea473bebd62 in_seq=0 out_seq=0 2024-04-25T10:42:23.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.699+0000 7f2935ffb640 1 -- 172.21.15.43:0/2309267108 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf005ac800 con 0x55bf002323a0 2024-04-25T10:42:23.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.699+0000 7f2935ffb640 10 client.18432 handle_command_reply: tid=0 2024-04-25T10:42:23.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.699+0000 7f295b39e740 1 client.18432 shutdown 2024-04-25T10:42:23.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.699+0000 7f295b39e740 1 -- 172.21.15.43:0/2309267108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 msgr2=0x55bf006bc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.699+0000 7f295b39e740 1 --2- 172.21.15.43:0/2309267108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 0x55bf006bc620 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f2944005d70 tx=0x7f29440096f0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.699+0000 7f295b39e740 1 -- 172.21.15.43:0/2309267108 shutdown_connections 2024-04-25T10:42:23.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.699+0000 7f295b39e740 1 --2- 172.21.15.43:0/2309267108 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bf002323a0 0x55bf006c6540 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.699+0000 7f295b39e740 1 --2- 172.21.15.43:0/2309267108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0029e390 0x55bf006bc620 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.699+0000 7f2935ffb640 0 client.18432 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:42:23.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.699+0000 7f295b39e740 1 --2- 172.21.15.43:0/2309267108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf0027d6b0 0x55bf0064db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.699+0000 7f295b39e740 1 --2- 172.21.15.43:0/2309267108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf002754d0 0x55bf0064d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.699+0000 7f295b39e740 1 -- 172.21.15.43:0/2309267108 >> 172.21.15.43:0/2309267108 conn(0x55bf002924e0 msgr2=0x55bf0064b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:23.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.699+0000 7f295b39e740 1 -- 172.21.15.43:0/2309267108 shutdown_connections 2024-04-25T10:42:23.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.699+0000 7f295b39e740 1 -- 172.21.15.43:0/2309267108 wait complete. 2024-04-25T10:42:23.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.699+0000 7f295b39e740 20 client.18432 trim_cache size 0 max 16384 2024-04-25T10:42:23.710 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:42:23.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.700+0000 7f29544ee640 1 -- 172.21.15.43:0/1214249402 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f29380afc60 msgr2=0x7f29380b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.700+0000 7f29544ee640 1 --2- 172.21.15.43:0/1214249402 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f29380afc60 0x7f29380b2170 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2944022000 tx=0x7f2944023000 comp rx=0 tx=0).stop 2024-04-25T10:42:23.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.700+0000 7f29544ee640 1 -- 172.21.15.43:0/1214249402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c104cd0 msgr2=0x7f294c102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.700+0000 7f29544ee640 1 --2- 172.21.15.43:0/1214249402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c104cd0 0x7f294c102590 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f2948005940 tx=0x7f29480040c0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.700+0000 7f29544ee640 1 -- 172.21.15.43:0/1214249402 shutdown_connections 2024-04-25T10:42:23.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.700+0000 7f29544ee640 1 --2- 172.21.15.43:0/1214249402 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f29380afc60 0x7f29380b2170 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.700+0000 7f29544ee640 1 --2- 172.21.15.43:0/1214249402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f294c10e090 0x7f294c103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.700+0000 7f29544ee640 1 --2- 172.21.15.43:0/1214249402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f294c10b730 0x7f294c102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.700+0000 7f29544ee640 1 --2- 172.21.15.43:0/1214249402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c104cd0 0x7f294c102590 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.700+0000 7f29544ee640 1 -- 172.21.15.43:0/1214249402 >> 172.21.15.43:0/1214249402 conn(0x7f294c078110 msgr2=0x7f294c101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:23.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.700+0000 7f29544ee640 1 -- 172.21.15.43:0/1214249402 shutdown_connections 2024-04-25T10:42:23.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.700+0000 7f29544ee640 1 -- 172.21.15.43:0/1214249402 wait complete. 2024-04-25T10:42:23.713 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:42:23.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.825+0000 7f1a16084640 1 -- 172.21.15.43:0/146075400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a1010b6b0 msgr2=0x7f1a1010dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.825+0000 7f1a16084640 1 --2- 172.21.15.43:0/146075400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a1010b6b0 0x7f1a1010dad0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009900 tx=0x7f19f802a510 comp rx=0 tx=0).stop 2024-04-25T10:42:23.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.825+0000 7f1a16084640 1 -- 172.21.15.43:0/146075400 shutdown_connections 2024-04-25T10:42:23.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.825+0000 7f1a16084640 1 --2- 172.21.15.43:0/146075400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a1010e060 0x7f1a10110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.825+0000 7f1a16084640 1 --2- 172.21.15.43:0/146075400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a1010b6b0 0x7f1a1010dad0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.825+0000 7f1a16084640 1 --2- 172.21.15.43:0/146075400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a10104c50 0x7f1a10105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.825+0000 7f1a16084640 1 -- 172.21.15.43:0/146075400 >> 172.21.15.43:0/146075400 conn(0x7f1a10078100 msgr2=0x7f1a10078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:23.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.825+0000 7f1a16084640 1 -- 172.21.15.43:0/146075400 shutdown_connections 2024-04-25T10:42:23.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.825+0000 7f1a16084640 1 -- 172.21.15.43:0/146075400 wait complete. 2024-04-25T10:42:23.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.826+0000 7f1a16084640 1 Processor -- start 2024-04-25T10:42:23.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.826+0000 7f1a16084640 1 -- start start 2024-04-25T10:42:23.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.826+0000 7f1a16084640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a10104c50 0x7f1a1019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.826+0000 7f1a16084640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a1010b6b0 0x7f1a1019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.826+0000 7f1a16084640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a1010e060 0x7f1a1019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.826+0000 7f1a16084640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a10113cc0 con 0x7f1a1010e060 2024-04-25T10:42:23.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.826+0000 7f1a16084640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1a10113b40 con 0x7f1a10104c50 2024-04-25T10:42:23.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.826+0000 7f1a16084640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1a10113e40 con 0x7f1a1010b6b0 2024-04-25T10:42:23.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.826+0000 7f1a0ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a1010e060 0x7f1a1019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.826+0000 7f1a0ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a1010e060 0x7f1a1019f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49104/0 (socket says 172.21.15.43:49104) 2024-04-25T10:42:23.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.826+0000 7f1a0ffff640 1 -- 172.21.15.43:0/2113564604 learned_addr learned my addr 172.21.15.43:0/2113564604 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:23.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.826+0000 7f1a0f7fe640 1 --2- 172.21.15.43:0/2113564604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a10104c50 0x7f1a1019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.827+0000 7f1a0effd640 1 --2- 172.21.15.43:0/2113564604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a1010b6b0 0x7f1a1019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.827+0000 7f1a0ffff640 1 -- 172.21.15.43:0/2113564604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a1010b6b0 msgr2=0x7f1a1019ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.827+0000 7f1a0ffff640 1 --2- 172.21.15.43:0/2113564604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a1010b6b0 0x7f1a1019ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.827+0000 7f1a0ffff640 1 -- 172.21.15.43:0/2113564604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a10104c50 msgr2=0x7f1a1019a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.827+0000 7f1a0ffff640 1 --2- 172.21.15.43:0/2113564604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a10104c50 0x7f1a1019a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.827+0000 7f1a0ffff640 1 -- 172.21.15.43:0/2113564604 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1019f640 con 0x7f1a1010e060 2024-04-25T10:42:23.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.827+0000 7f1a0effd640 1 --2- 172.21.15.43:0/2113564604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a1010b6b0 0x7f1a1019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:23.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.827+0000 7f1a0ffff640 1 --2- 172.21.15.43:0/2113564604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a1010e060 0x7f1a1019f000 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800d750 tx=0x7f1a0800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:23.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.827+0000 7f1a0cff9640 1 -- 172.21.15.43:0/2113564604 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08017070 con 0x7f1a1010e060 2024-04-25T10:42:23.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.827+0000 7f1a0cff9640 1 -- 172.21.15.43:0/2113564604 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a080054d0 con 0x7f1a1010e060 2024-04-25T10:42:23.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.827+0000 7f1a16084640 1 -- 172.21.15.43:0/2113564604 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a1019f930 con 0x7f1a1010e060 2024-04-25T10:42:23.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.827+0000 7f1a0cff9640 1 -- 172.21.15.43:0/2113564604 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08004c60 con 0x7f1a1010e060 2024-04-25T10:42:23.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.827+0000 7f1a16084640 1 -- 172.21.15.43:0/2113564604 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a101a72b0 con 0x7f1a1010e060 2024-04-25T10:42:23.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.829+0000 7f1a0cff9640 1 -- 172.21.15.43:0/2113564604 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f1a08009bd0 con 0x7f1a1010e060 2024-04-25T10:42:23.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.830+0000 7f1a1d008740 1 Processor -- start 2024-04-25T10:42:23.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.830+0000 7f1a1d008740 1 -- start start 2024-04-25T10:42:23.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.830+0000 7f1a1d008740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc98695390 0x55fc98a36810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.830+0000 7f1a1d008740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc9866c4d0 0x55fc98a36d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.830+0000 7f1a1d008740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc986746b0 0x55fc98a436a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.830+0000 7f1a1d008740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fc9899bd80 con 0x55fc986746b0 2024-04-25T10:42:23.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.830+0000 7f1a1d008740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fc98955a20 con 0x55fc98695390 2024-04-25T10:42:23.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.830+0000 7f1a1d008740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fc98997d40 con 0x55fc9866c4d0 2024-04-25T10:42:23.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.830+0000 7f1a0effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc986746b0 0x55fc98a436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.830+0000 7f1a0cff9640 1 --2- 172.21.15.43:0/2113564604 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f19fc0afce0 0x7f19fc0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.830+0000 7f1a0effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc986746b0 0x55fc98a436a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49108/0 (socket says 172.21.15.43:49108) 2024-04-25T10:42:23.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.830+0000 7f1a0effd640 1 -- 172.21.15.43:0/2861993869 learned_addr learned my addr 172.21.15.43:0/2861993869 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:23.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.830+0000 7f1a0f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc9866c4d0 0x55fc98a36d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.830+0000 7f1a0ffff640 1 --2- 172.21.15.43:0/2113564604 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f19fc0afce0 0x7f19fc0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.830+0000 7f1a0effd640 1 --2- 172.21.15.43:0/2861993869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc98695390 0x55fc98a36810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.831+0000 7f1a0cff9640 1 -- 172.21.15.43:0/2113564604 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f1a080de9b0 con 0x7f1a1010e060 2024-04-25T10:42:23.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.831+0000 7f1a0f7fe640 1 -- 172.21.15.43:0/2861993869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc98695390 msgr2=0x55fc98a36810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.831+0000 7f1a0f7fe640 1 --2- 172.21.15.43:0/2861993869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc98695390 0x55fc98a36810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.831+0000 7f1a0f7fe640 1 -- 172.21.15.43:0/2861993869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc986746b0 msgr2=0x55fc98a436a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.831+0000 7f1a0f7fe640 1 --2- 172.21.15.43:0/2861993869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc986746b0 0x55fc98a436a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.831+0000 7f1a0f7fe640 1 -- 172.21.15.43:0/2861993869 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc98960c30 con 0x55fc9866c4d0 2024-04-25T10:42:23.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.831+0000 7f1a0effd640 1 --2- 172.21.15.43:0/2861993869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc986746b0 0x55fc98a436a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:42:23.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.831+0000 7f1a0f7fe640 1 --2- 172.21.15.43:0/2861993869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc9866c4d0 0x55fc98a36d50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009ee0 tx=0x7f1a0000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=781b769e45a528c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:23.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.831+0000 7f1a0ffff640 1 --2- 172.21.15.43:0/2113564604 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f19fc0afce0 0x7f19fc0b21f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1a080dc700 tx=0x7f1a0808f040 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:23.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.831+0000 7f1a05ffb640 1 -- 172.21.15.43:0/2861993869 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a00011640 con 0x55fc9866c4d0 2024-04-25T10:42:23.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.831+0000 7f1a05ffb640 1 -- 172.21.15.43:0/2861993869 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a00011c90 con 0x55fc9866c4d0 2024-04-25T10:42:23.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.831+0000 7f1a05ffb640 1 -- 172.21.15.43:0/2861993869 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a00010bf0 con 0x55fc9866c4d0 2024-04-25T10:42:23.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.832+0000 7f1a1d008740 1 -- 172.21.15.43:0/2861993869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc9866c4d0 msgr2=0x55fc98a36d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.832+0000 7f1a1d008740 1 --2- 172.21.15.43:0/2861993869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc9866c4d0 0x55fc98a36d50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009ee0 tx=0x7f1a0000f730 comp rx=0 tx=0).stop 2024-04-25T10:42:23.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.832+0000 7f1a1d008740 1 -- 172.21.15.43:0/2861993869 shutdown_connections 2024-04-25T10:42:23.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.832+0000 7f1a1d008740 1 --2- 172.21.15.43:0/2861993869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc986746b0 0x55fc98a436a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.832+0000 7f1a1d008740 1 --2- 172.21.15.43:0/2861993869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc9866c4d0 0x55fc98a36d50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.832+0000 7f1a1d008740 1 --2- 172.21.15.43:0/2861993869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc98695390 0x55fc98a36810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.832+0000 7f1a1d008740 1 -- 172.21.15.43:0/2861993869 >> 172.21.15.43:0/2861993869 conn(0x55fc986894e0 msgr2=0x55fc98a32fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:23.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.832+0000 7f1a1d008740 1 -- 172.21.15.43:0/2861993869 shutdown_connections 2024-04-25T10:42:23.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.832+0000 7f1a1d008740 1 -- 172.21.15.43:0/2861993869 wait complete. 2024-04-25T10:42:23.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.832+0000 7f1a1d008740 1 -- start start 2024-04-25T10:42:23.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.833+0000 7f1a1d008740 1 Processor -- start 2024-04-25T10:42:23.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.833+0000 7f1a1d008740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc9866c4d0 0x55fc98a41280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.833+0000 7f1a1d008740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc986746b0 0x55fc98a417c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.833+0000 7f1a1d008740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc98695390 0x55fc98a4d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.833+0000 7f1a1d008740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fc98a43d90 con 0x55fc9866c4d0 2024-04-25T10:42:23.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.833+0000 7f1a1d008740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fc98922dc0 con 0x55fc986746b0 2024-04-25T10:42:23.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.833+0000 7f1a1d008740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fc98a43f10 con 0x55fc98695390 2024-04-25T10:42:23.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.833+0000 7f1a1d008740 10 client.0 _ll_register_callbacks cb 0x55fc9893e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:23.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.833+0000 7f1a0effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc9866c4d0 0x55fc98a41280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.833+0000 7f1a0effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc9866c4d0 0x55fc98a41280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49114/0 (socket says 172.21.15.43:49114) 2024-04-25T10:42:23.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.833+0000 7f1a0effd640 1 -- 172.21.15.43:0/4283680291 learned_addr learned my addr 172.21.15.43:0/4283680291 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:23.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.833+0000 7f1a0f7fe640 1 --2- 172.21.15.43:0/4283680291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc986746b0 0x55fc98a417c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.833+0000 7f1a0effd640 1 --2- 172.21.15.43:0/4283680291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc98695390 0x55fc98a4d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.833+0000 7f1a0effd640 1 -- 172.21.15.43:0/4283680291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc98695390 msgr2=0x55fc98a4d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.833+0000 7f1a0effd640 1 --2- 172.21.15.43:0/4283680291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc98695390 0x55fc98a4d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.833+0000 7f1a0effd640 1 -- 172.21.15.43:0/4283680291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc986746b0 msgr2=0x55fc98a417c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.833+0000 7f1a0effd640 1 --2- 172.21.15.43:0/4283680291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc986746b0 0x55fc98a417c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.833+0000 7f1a0effd640 1 -- 172.21.15.43:0/4283680291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc986ad3f0 con 0x55fc9866c4d0 2024-04-25T10:42:23.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.833+0000 7f1a0f7fe640 1 --2- 172.21.15.43:0/4283680291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc986746b0 0x55fc98a417c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:23.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.834+0000 7f1a0effd640 1 --2- 172.21.15.43:0/4283680291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc9866c4d0 0x55fc98a41280 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f19f8032ff0 tx=0x7f19f802a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268a95398e6c39f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:23.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.834+0000 7f19eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:23.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.834+0000 7f19eeffd640 1 -- 172.21.15.43:0/4283680291 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f8035040 con 0x55fc9866c4d0 2024-04-25T10:42:23.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.834+0000 7f19eeffd640 1 -- 172.21.15.43:0/4283680291 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f19f8005040 con 0x55fc9866c4d0 2024-04-25T10:42:23.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.834+0000 7f1a1d008740 1 -- 172.21.15.43:0/4283680291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a00011c90 con 0x55fc9866c4d0 2024-04-25T10:42:23.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.834+0000 7f19eeffd640 1 -- 172.21.15.43:0/4283680291 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f19f8052050 con 0x55fc9866c4d0 2024-04-25T10:42:23.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.834+0000 7f1a1d008740 10 client.18444 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:23.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.834+0000 7f1a1d008740 1 -- 172.21.15.43:0/4283680291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc987777d0 con 0x55fc9866c4d0 2024-04-25T10:42:23.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.834+0000 7f19eeffd640 1 -- 172.21.15.43:0/4283680291 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a00011c90 con 0x55fc9866c4d0 2024-04-25T10:42:23.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.834+0000 7f19eeffd640 1 -- 172.21.15.43:0/4283680291 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f19f803a040 con 0x55fc9866c4d0 2024-04-25T10:42:23.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.834+0000 7f1a1d008740 10 client.18444 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:23.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.834+0000 7f1a1d008740 10 client.18444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:23.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.834+0000 7f1a1d008740 20 client.18444 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:23.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.834+0000 7f1a1d008740 1 --2- 172.21.15.43:0/4283680291 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fc98653f60 0x55fc98a53610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.834+0000 7f1a1d008740 4 client.18444 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:42:23.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.834+0000 7f1a1d008740 1 -- 172.21.15.43:0/4283680291 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc989a9990 con 0x55fc98653f60 2024-04-25T10:42:23.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.835+0000 7f1a0f7fe640 1 --2- 172.21.15.43:0/4283680291 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fc98653f60 0x55fc98a53610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.835+0000 7f1a0f7fe640 1 --2- 172.21.15.43:0/4283680291 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fc98653f60 0x55fc98a53610 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ad3ace3f2a9f942 server_cookie=1b55b83ec40fdce7 in_seq=0 out_seq=0 2024-04-25T10:42:23.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.835+0000 7f19eeffd640 10 client.18444 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:23.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.836+0000 7f19eeffd640 1 -- 172.21.15.43:0/4283680291 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fc989a9990 con 0x55fc98653f60 2024-04-25T10:42:23.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.836+0000 7f19eeffd640 10 client.18444 handle_command_reply: tid=0 2024-04-25T10:42:23.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.836+0000 7f1a1d008740 1 client.18444 shutdown 2024-04-25T10:42:23.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.836+0000 7f1a1d008740 1 -- 172.21.15.43:0/4283680291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc9866c4d0 msgr2=0x55fc98a41280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.836+0000 7f1a1d008740 1 --2- 172.21.15.43:0/4283680291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc9866c4d0 0x55fc98a41280 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f19f8032ff0 tx=0x7f19f802a510 comp rx=0 tx=0).stop 2024-04-25T10:42:23.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.836+0000 7f1a1d008740 1 -- 172.21.15.43:0/4283680291 shutdown_connections 2024-04-25T10:42:23.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.836+0000 7f1a1d008740 1 --2- 172.21.15.43:0/4283680291 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fc98653f60 0x55fc98a53610 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.836+0000 7f1a1d008740 1 --2- 172.21.15.43:0/4283680291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc986746b0 0x55fc98a417c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.836+0000 7f1a1d008740 1 --2- 172.21.15.43:0/4283680291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc98695390 0x55fc98a4d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.836+0000 7f19eeffd640 0 client.18444 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:23.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.836+0000 7f1a1d008740 1 --2- 172.21.15.43:0/4283680291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc9866c4d0 0x55fc98a41280 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.836+0000 7f1a1d008740 1 -- 172.21.15.43:0/4283680291 >> 172.21.15.43:0/4283680291 conn(0x55fc986894e0 msgr2=0x55fc98a38530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:23.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.836+0000 7f1a1d008740 1 -- 172.21.15.43:0/4283680291 shutdown_connections 2024-04-25T10:42:23.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.836+0000 7f1a1d008740 1 -- 172.21.15.43:0/4283680291 wait complete. 2024-04-25T10:42:23.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.836+0000 7f1a1d008740 20 client.18444 trim_cache size 0 max 16384 2024-04-25T10:42:23.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.850+0000 7f1a1d008740 1 Processor -- start 2024-04-25T10:42:23.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.850+0000 7f1a1d008740 1 -- start start 2024-04-25T10:42:23.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.850+0000 7f1a1d008740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc9866c4d0 0x55fc98a32f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.850+0000 7f1a1d008740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc986746b0 0x55fc98a4ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.850+0000 7f1a1d008740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc98695390 0x55fc98a58dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.850+0000 7f1a1d008740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101b2390 con 0x55fc986746b0 2024-04-25T10:42:23.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.850+0000 7f1a1d008740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101afc60 con 0x55fc9866c4d0 2024-04-25T10:42:23.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.850+0000 7f1a1d008740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101adad0 con 0x55fc98695390 2024-04-25T10:42:23.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.850+0000 7f1a0f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc986746b0 0x55fc98a4ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.850+0000 7f1a0f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc986746b0 0x55fc98a4ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49126/0 (socket says 172.21.15.43:49126) 2024-04-25T10:42:23.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.850+0000 7f1a0f7fe640 1 -- 172.21.15.43:0/846763174 learned_addr learned my addr 172.21.15.43:0/846763174 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:23.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.850+0000 7f1a0effd640 1 --2- 172.21.15.43:0/846763174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc9866c4d0 0x55fc98a32f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.850+0000 7f1a0f7fe640 1 -- 172.21.15.43:0/846763174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc98695390 msgr2=0x55fc98a58dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.850+0000 7f1a0f7fe640 1 --2- 172.21.15.43:0/846763174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc98695390 0x55fc98a58dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.850+0000 7f1a0f7fe640 1 -- 172.21.15.43:0/846763174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc9866c4d0 msgr2=0x55fc98a32f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.850+0000 7f1a0f7fe640 1 --2- 172.21.15.43:0/846763174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc9866c4d0 0x55fc98a32f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.850+0000 7f1a0f7fe640 1 -- 172.21.15.43:0/846763174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc98a41f80 con 0x55fc986746b0 2024-04-25T10:42:23.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.851+0000 7f1a0f7fe640 1 --2- 172.21.15.43:0/846763174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc986746b0 0x55fc98a4ce70 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009fd0 tx=0x7f1a00009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc613f4c378446a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:23.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.851+0000 7f1a057fa640 1 -- 172.21.15.43:0/846763174 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a000108c0 con 0x55fc986746b0 2024-04-25T10:42:23.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.851+0000 7f1a057fa640 1 -- 172.21.15.43:0/846763174 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a00011440 con 0x55fc986746b0 2024-04-25T10:42:23.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.851+0000 7f1a057fa640 1 -- 172.21.15.43:0/846763174 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0002a330 con 0x55fc986746b0 2024-04-25T10:42:23.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.852+0000 7f1a1d008740 1 -- 172.21.15.43:0/846763174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc986746b0 msgr2=0x55fc98a4ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.852+0000 7f1a1d008740 1 --2- 172.21.15.43:0/846763174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc986746b0 0x55fc98a4ce70 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009fd0 tx=0x7f1a00009800 comp rx=0 tx=0).stop 2024-04-25T10:42:23.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.852+0000 7f1a1d008740 1 -- 172.21.15.43:0/846763174 shutdown_connections 2024-04-25T10:42:23.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.852+0000 7f1a1d008740 1 --2- 172.21.15.43:0/846763174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc98695390 0x55fc98a58dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.852+0000 7f1a1d008740 1 --2- 172.21.15.43:0/846763174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc986746b0 0x55fc98a4ce70 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.852+0000 7f1a1d008740 1 --2- 172.21.15.43:0/846763174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc9866c4d0 0x55fc98a32f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.852+0000 7f1a1d008740 1 -- 172.21.15.43:0/846763174 >> 172.21.15.43:0/846763174 conn(0x55fc986894e0 msgr2=0x55fc98a32860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:23.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.852+0000 7f1a1d008740 1 -- 172.21.15.43:0/846763174 shutdown_connections 2024-04-25T10:42:23.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.852+0000 7f1a1d008740 1 -- 172.21.15.43:0/846763174 wait complete. 2024-04-25T10:42:23.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.852+0000 7f1a1d008740 1 -- start start 2024-04-25T10:42:23.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.852+0000 7f1a1d008740 1 Processor -- start 2024-04-25T10:42:23.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.852+0000 7f1a1d008740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc9866c4d0 0x55fc98a493b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.852+0000 7f1a1d008740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc986746b0 0x55fc98a498f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f1a1d008740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc98695390 0x55fc98a448e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f1a1d008740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101ae120 con 0x55fc98695390 2024-04-25T10:42:23.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f1a1d008740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101af590 con 0x55fc9866c4d0 2024-04-25T10:42:23.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f1a1d008740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101ae7f0 con 0x55fc986746b0 2024-04-25T10:42:23.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f1a1d008740 10 client.0 _ll_register_callbacks cb 0x7f19f8009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:23.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f1a0effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc98695390 0x55fc98a448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f1a0effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc98695390 0x55fc98a448e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49128/0 (socket says 172.21.15.43:49128) 2024-04-25T10:42:23.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f1a0effd640 1 -- 172.21.15.43:0/1627226820 learned_addr learned my addr 172.21.15.43:0/1627226820 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:23.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f1a0f7fe640 1 --2- 172.21.15.43:0/1627226820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc986746b0 0x55fc98a498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f1a0effd640 1 --2- 172.21.15.43:0/1627226820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc9866c4d0 0x55fc98a493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f1a0effd640 1 -- 172.21.15.43:0/1627226820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc986746b0 msgr2=0x55fc98a498f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f1a0effd640 1 --2- 172.21.15.43:0/1627226820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc986746b0 0x55fc98a498f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f1a0effd640 1 -- 172.21.15.43:0/1627226820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc9866c4d0 msgr2=0x55fc98a493b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:23.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f1a0effd640 1 --2- 172.21.15.43:0/1627226820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc9866c4d0 0x55fc98a493b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f1a0effd640 1 -- 172.21.15.43:0/1627226820 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc98a45070 con 0x55fc98695390 2024-04-25T10:42:23.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f1a0f7fe640 1 --2- 172.21.15.43:0/1627226820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc986746b0 0x55fc98a498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:23.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f1a0effd640 1 --2- 172.21.15.43:0/1627226820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc98695390 0x55fc98a448e0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f19f8003670 tx=0x7f19f8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d27c7b5a8b2f0b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:23.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f19eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:23.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f19eeffd640 1 -- 172.21.15.43:0/1627226820 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f8041070 con 0x55fc98695390 2024-04-25T10:42:23.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f1a1d008740 1 -- 172.21.15.43:0/1627226820 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a00011440 con 0x55fc98695390 2024-04-25T10:42:23.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f19eeffd640 1 -- 172.21.15.43:0/1627226820 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f19f8031cc0 con 0x55fc98695390 2024-04-25T10:42:23.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f19eeffd640 1 -- 172.21.15.43:0/1627226820 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f19f8035040 con 0x55fc98695390 2024-04-25T10:42:23.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f1a1d008740 10 client.18450 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:23.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.853+0000 7f1a1d008740 1 -- 172.21.15.43:0/1627226820 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc98abb4a0 con 0x55fc98695390 2024-04-25T10:42:23.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.854+0000 7f19eeffd640 1 -- 172.21.15.43:0/1627226820 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a00011440 con 0x55fc98695390 2024-04-25T10:42:23.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.854+0000 7f19eeffd640 1 -- 172.21.15.43:0/1627226820 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f19f803a040 con 0x55fc98695390 2024-04-25T10:42:23.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.854+0000 7f1a1d008740 10 client.18450 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:23.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.854+0000 7f1a1d008740 10 client.18450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:23.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.854+0000 7f1a1d008740 20 client.18450 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:23.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.854+0000 7f1a1d008740 1 --2- 172.21.15.43:0/1627226820 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fc986293a0 0x55fc98abdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:23.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.854+0000 7f1a1d008740 4 client.18450 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:42:23.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.854+0000 7f1a1d008740 1 -- 172.21.15.43:0/1627226820 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc989a3800 con 0x55fc986293a0 2024-04-25T10:42:23.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.854+0000 7f1a0f7fe640 1 --2- 172.21.15.43:0/1627226820 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fc986293a0 0x55fc98abdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:23.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.855+0000 7f1a0f7fe640 1 --2- 172.21.15.43:0/1627226820 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fc986293a0 0x55fc98abdbf0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=474adbd56a231bf1 server_cookie=dd7fdb737444fd94 in_seq=0 out_seq=0 2024-04-25T10:42:23.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.855+0000 7f19eeffd640 10 client.18450 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:23.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.855+0000 7f19eeffd640 1 -- 172.21.15.43:0/1627226820 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc989a3800 con 0x55fc986293a0 2024-04-25T10:42:23.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.855+0000 7f19eeffd640 10 client.18450 handle_command_reply: tid=0 2024-04-25T10:42:23.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.855+0000 7f1a1d008740 1 client.18450 shutdown 2024-04-25T10:42:23.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.855+0000 7f1a1d008740 1 -- 172.21.15.43:0/1627226820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc98695390 msgr2=0x55fc98a448e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.855+0000 7f1a1d008740 1 --2- 172.21.15.43:0/1627226820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc98695390 0x55fc98a448e0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f19f8003670 tx=0x7f19f8001e50 comp rx=0 tx=0).stop 2024-04-25T10:42:23.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.855+0000 7f1a1d008740 1 -- 172.21.15.43:0/1627226820 shutdown_connections 2024-04-25T10:42:23.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.855+0000 7f1a1d008740 1 --2- 172.21.15.43:0/1627226820 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55fc986293a0 0x55fc98abdbf0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.855+0000 7f1a1d008740 1 --2- 172.21.15.43:0/1627226820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc98695390 0x55fc98a448e0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.855+0000 7f1a1d008740 1 --2- 172.21.15.43:0/1627226820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc986746b0 0x55fc98a498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.855+0000 7f19eeffd640 0 client.18450 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:23.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.855+0000 7f1a1d008740 1 --2- 172.21.15.43:0/1627226820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc9866c4d0 0x55fc98a493b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.855+0000 7f1a1d008740 1 -- 172.21.15.43:0/1627226820 >> 172.21.15.43:0/1627226820 conn(0x55fc986894e0 msgr2=0x55fc98a42530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:23.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.855+0000 7f1a1d008740 1 -- 172.21.15.43:0/1627226820 shutdown_connections 2024-04-25T10:42:23.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.855+0000 7f1a1d008740 1 -- 172.21.15.43:0/1627226820 wait complete. 2024-04-25T10:42:23.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.855+0000 7f1a1d008740 20 client.18450 trim_cache size 0 max 16384 2024-04-25T10:42:23.864 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:42:23.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.856+0000 7f1a16084640 1 -- 172.21.15.43:0/2113564604 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f19fc0afce0 msgr2=0x7f19fc0b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.856+0000 7f1a16084640 1 --2- 172.21.15.43:0/2113564604 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f19fc0afce0 0x7f19fc0b21f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1a080dc700 tx=0x7f1a0808f040 comp rx=0 tx=0).stop 2024-04-25T10:42:23.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.856+0000 7f1a16084640 1 -- 172.21.15.43:0/2113564604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a1010e060 msgr2=0x7f1a1019f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:23.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.856+0000 7f1a16084640 1 --2- 172.21.15.43:0/2113564604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a1010e060 0x7f1a1019f000 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800d750 tx=0x7f1a0800dc10 comp rx=0 tx=0).stop 2024-04-25T10:42:23.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.856+0000 7f1a16084640 1 -- 172.21.15.43:0/2113564604 shutdown_connections 2024-04-25T10:42:23.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.856+0000 7f1a16084640 1 --2- 172.21.15.43:0/2113564604 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f19fc0afce0 0x7f19fc0b21f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.856+0000 7f1a16084640 1 --2- 172.21.15.43:0/2113564604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a1010e060 0x7f1a1019f000 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.856+0000 7f1a16084640 1 --2- 172.21.15.43:0/2113564604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a1010b6b0 0x7f1a1019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.856+0000 7f1a16084640 1 --2- 172.21.15.43:0/2113564604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a10104c50 0x7f1a1019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:23.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.856+0000 7f1a16084640 1 -- 172.21.15.43:0/2113564604 >> 172.21.15.43:0/2113564604 conn(0x7f1a10078100 msgr2=0x7f1a10101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:23.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.856+0000 7f1a16084640 1 -- 172.21.15.43:0/2113564604 shutdown_connections 2024-04-25T10:42:23.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:23.856+0000 7f1a16084640 1 -- 172.21.15.43:0/2113564604 wait complete. 2024-04-25T10:42:24.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:23 smithi077 ceph-mon[32772]: pgmap v307: 97 pgs: 97 active+clean; 2.4 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-04-25T10:42:24.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:23 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1610826752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:42:24.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:23 smithi164 ceph-mon[32713]: pgmap v307: 97 pgs: 97 active+clean; 2.4 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-04-25T10:42:24.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:23 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1610826752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:42:24.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:23 smithi043 ceph-mon[29478]: pgmap v307: 97 pgs: 97 active+clean; 2.4 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-04-25T10:42:24.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:23 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1610826752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:42:24.870 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:42:24.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:42:24.870 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:42:25.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.000+0000 7fb563738640 1 -- 172.21.15.43:0/1734475343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c10d380 msgr2=0x7fb55c10f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:25.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.000+0000 7fb563738640 1 --2- 172.21.15.43:0/1734475343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c10d380 0x7fb55c10f7a0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fb554009ca0 tx=0x7fb55402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.000+0000 7fb563738640 1 -- 172.21.15.43:0/1734475343 shutdown_connections 2024-04-25T10:42:25.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.000+0000 7fb563738640 1 --2- 172.21.15.43:0/1734475343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c10d380 0x7fb55c10f7a0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.000+0000 7fb563738640 1 --2- 172.21.15.43:0/1734475343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb55c102af0 0x7fb55c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.000+0000 7fb563738640 1 --2- 172.21.15.43:0/1734475343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb55c108a20 0x7fb55c1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.000+0000 7fb563738640 1 -- 172.21.15.43:0/1734475343 >> 172.21.15.43:0/1734475343 conn(0x7fb55c078100 msgr2=0x7fb55c078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:25.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.000+0000 7fb563738640 1 -- 172.21.15.43:0/1734475343 shutdown_connections 2024-04-25T10:42:25.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.000+0000 7fb563738640 1 -- 172.21.15.43:0/1734475343 wait complete. 2024-04-25T10:42:25.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.001+0000 7fb563738640 1 Processor -- start 2024-04-25T10:42:25.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.001+0000 7fb563738640 1 -- start start 2024-04-25T10:42:25.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.001+0000 7fb563738640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb55c102af0 0x7fb55c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.001+0000 7fb563738640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c108a20 0x7fb55c19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.001+0000 7fb563738640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb55c10d380 0x7fb55c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.001+0000 7fb563738640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c112c00 con 0x7fb55c108a20 2024-04-25T10:42:25.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.001+0000 7fb563738640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c112a80 con 0x7fb55c102af0 2024-04-25T10:42:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.001+0000 7fb563738640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c112d80 con 0x7fb55c10d380 2024-04-25T10:42:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.001+0000 7fb560cd3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c108a20 0x7fb55c19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.001+0000 7fb560cd3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c108a20 0x7fb55c19ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49148/0 (socket says 172.21.15.43:49148) 2024-04-25T10:42:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.001+0000 7fb560cd3640 1 -- 172.21.15.43:0/2547558915 learned_addr learned my addr 172.21.15.43:0/2547558915 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.002+0000 7fb5614d4640 1 --2- 172.21.15.43:0/2547558915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb55c102af0 0x7fb55c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.002+0000 7fb561cd5640 1 --2- 172.21.15.43:0/2547558915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb55c10d380 0x7fb55c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.002+0000 7fb560cd3640 1 -- 172.21.15.43:0/2547558915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb55c10d380 msgr2=0x7fb55c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.002+0000 7fb560cd3640 1 --2- 172.21.15.43:0/2547558915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb55c10d380 0x7fb55c19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.002+0000 7fb560cd3640 1 -- 172.21.15.43:0/2547558915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb55c102af0 msgr2=0x7fb55c19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.002+0000 7fb560cd3640 1 --2- 172.21.15.43:0/2547558915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb55c102af0 0x7fb55c19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.002+0000 7fb560cd3640 1 -- 172.21.15.43:0/2547558915 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55c19f490 con 0x7fb55c108a20 2024-04-25T10:42:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.002+0000 7fb5614d4640 1 --2- 172.21.15.43:0/2547558915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb55c102af0 0x7fb55c19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.002+0000 7fb560cd3640 1 --2- 172.21.15.43:0/2547558915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c108a20 0x7fb55c19ac10 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00d950 tx=0x7fb54c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.002+0000 7fb5527fc640 1 -- 172.21.15.43:0/2547558915 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c017070 con 0x7fb55c108a20 2024-04-25T10:42:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.002+0000 7fb563738640 1 -- 172.21.15.43:0/2547558915 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb55c19f780 con 0x7fb55c108a20 2024-04-25T10:42:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.002+0000 7fb5527fc640 1 -- 172.21.15.43:0/2547558915 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb54c009c30 con 0x7fb55c108a20 2024-04-25T10:42:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.002+0000 7fb5527fc640 1 -- 172.21.15.43:0/2547558915 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00f620 con 0x7fb55c108a20 2024-04-25T10:42:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.002+0000 7fb563738640 1 -- 172.21.15.43:0/2547558915 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb55c19fc80 con 0x7fb55c108a20 2024-04-25T10:42:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.005+0000 7fb5527fc640 1 -- 172.21.15.43:0/2547558915 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fb54c01a070 con 0x7fb55c108a20 2024-04-25T10:42:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.005+0000 7fb56a5ec740 1 Processor -- start 2024-04-25T10:42:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.005+0000 7fb56a5ec740 1 -- start start 2024-04-25T10:42:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.005+0000 7fb56a5ec740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17fe410 0x563bd1b9e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.005+0000 7fb56a5ec740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 0x563bd1b9eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.005+0000 7fb56a5ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563bd17dd730 0x563bd1b9f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.005+0000 7fb56a5ec740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563bd1b04b00 con 0x563bd17d5550 2024-04-25T10:42:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.005+0000 7fb56a5ec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563bd1aa0920 con 0x563bd17dd730 2024-04-25T10:42:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.005+0000 7fb56a5ec740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563bd1b00ac0 con 0x563bd17fe410 2024-04-25T10:42:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.005+0000 7fb5614d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 0x563bd1b9eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.006+0000 7fb5614d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 0x563bd1b9eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49150/0 (socket says 172.21.15.43:49150) 2024-04-25T10:42:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.006+0000 7fb5614d4640 1 -- 172.21.15.43:0/3695687918 learned_addr learned my addr 172.21.15.43:0/3695687918 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.006+0000 7fb560cd3640 1 --2- 172.21.15.43:0/3695687918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563bd17dd730 0x563bd1b9f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.006+0000 7fb561cd5640 1 --2- 172.21.15.43:0/3695687918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17fe410 0x563bd1b9e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.006+0000 7fb5527fc640 1 --2- 172.21.15.43:0/2547558915 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb5480afc60 0x7fb5480b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.006+0000 7fb5614d4640 1 -- 172.21.15.43:0/3695687918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17fe410 msgr2=0x563bd1b9e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.006+0000 7fb5614d4640 1 --2- 172.21.15.43:0/3695687918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17fe410 0x563bd1b9e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.006+0000 7fb5614d4640 1 -- 172.21.15.43:0/3695687918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563bd17dd730 msgr2=0x563bd1b9f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.006+0000 7fb5614d4640 1 --2- 172.21.15.43:0/3695687918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563bd17dd730 0x563bd1b9f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.006+0000 7fb5614d4640 1 -- 172.21.15.43:0/3695687918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bd18de7e0 con 0x563bd17d5550 2024-04-25T10:42:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.006+0000 7fb561cd5640 1 --2- 172.21.15.43:0/2547558915 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb5480afc60 0x7fb5480b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.006+0000 7fb5614d4640 1 --2- 172.21.15.43:0/3695687918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 0x563bd1b9eb60 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fb558008c10 tx=0x7fb55800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5235c2bf47cc27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.006+0000 7fb5527fc640 1 -- 172.21.15.43:0/2547558915 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fb54c0deb30 con 0x7fb55c108a20 2024-04-25T10:42:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.006+0000 7fb5477fe640 1 -- 172.21.15.43:0/3695687918 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558014070 con 0x563bd17d5550 2024-04-25T10:42:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.006+0000 7fb5477fe640 1 -- 172.21.15.43:0/3695687918 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5580047a0 con 0x563bd17d5550 2024-04-25T10:42:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.006+0000 7fb5477fe640 1 -- 172.21.15.43:0/3695687918 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558010690 con 0x563bd17d5550 2024-04-25T10:42:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.007+0000 7fb56a5ec740 1 -- 172.21.15.43:0/3695687918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 msgr2=0x563bd1b9eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.007+0000 7fb56a5ec740 1 --2- 172.21.15.43:0/3695687918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 0x563bd1b9eb60 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fb558008c10 tx=0x7fb55800e720 comp rx=0 tx=0).stop 2024-04-25T10:42:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.007+0000 7fb561cd5640 1 --2- 172.21.15.43:0/2547558915 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb5480afc60 0x7fb5480b2170 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb55402f500 tx=0x7fb5540095c0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.007+0000 7fb56a5ec740 1 -- 172.21.15.43:0/3695687918 shutdown_connections 2024-04-25T10:42:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.007+0000 7fb56a5ec740 1 --2- 172.21.15.43:0/3695687918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563bd17dd730 0x563bd1b9f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.007+0000 7fb56a5ec740 1 --2- 172.21.15.43:0/3695687918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 0x563bd1b9eb60 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.007+0000 7fb56a5ec740 1 --2- 172.21.15.43:0/3695687918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17fe410 0x563bd1b9e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.007+0000 7fb56a5ec740 1 -- 172.21.15.43:0/3695687918 >> 172.21.15.43:0/3695687918 conn(0x563bd17f2560 msgr2=0x563bd1b9add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.007+0000 7fb56a5ec740 1 -- 172.21.15.43:0/3695687918 shutdown_connections 2024-04-25T10:42:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.007+0000 7fb56a5ec740 1 -- 172.21.15.43:0/3695687918 wait complete. 2024-04-25T10:42:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.008+0000 7fb56a5ec740 1 -- start start 2024-04-25T10:42:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.008+0000 7fb56a5ec740 1 Processor -- start 2024-04-25T10:42:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.008+0000 7fb56a5ec740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17d5550 0x563bd1ba8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.008+0000 7fb56a5ec740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17dd730 0x563bd1ba9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.008+0000 7fb56a5ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563bd17fe410 0x563bd1bb4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.008+0000 7fb56a5ec740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563bd1bab970 con 0x563bd17dd730 2024-04-25T10:42:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.008+0000 7fb56a5ec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563bd1a9e5d0 con 0x563bd17fe410 2024-04-25T10:42:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.008+0000 7fb56a5ec740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563bd1babaf0 con 0x563bd17d5550 2024-04-25T10:42:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.008+0000 7fb56a5ec740 10 client.0 _ll_register_callbacks cb 0x563bd1a8c080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.008+0000 7fb560cd3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17dd730 0x563bd1ba9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.009+0000 7fb560cd3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17dd730 0x563bd1ba9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49152/0 (socket says 172.21.15.43:49152) 2024-04-25T10:42:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.009+0000 7fb560cd3640 1 -- 172.21.15.43:0/1743949276 learned_addr learned my addr 172.21.15.43:0/1743949276 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.009+0000 7fb5614d4640 1 --2- 172.21.15.43:0/1743949276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17d5550 0x563bd1ba8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.009+0000 7fb561cd5640 1 --2- 172.21.15.43:0/1743949276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563bd17fe410 0x563bd1bb4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.009+0000 7fb560cd3640 1 -- 172.21.15.43:0/1743949276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17d5550 msgr2=0x563bd1ba8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.009+0000 7fb560cd3640 1 --2- 172.21.15.43:0/1743949276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17d5550 0x563bd1ba8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.009+0000 7fb560cd3640 1 -- 172.21.15.43:0/1743949276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563bd17fe410 msgr2=0x563bd1bb4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.009+0000 7fb560cd3640 1 --2- 172.21.15.43:0/1743949276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563bd17fe410 0x563bd1bb4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.009+0000 7fb560cd3640 1 -- 172.21.15.43:0/1743949276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bd193ef70 con 0x563bd17dd730 2024-04-25T10:42:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.009+0000 7fb5614d4640 1 --2- 172.21.15.43:0/1743949276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17d5550 0x563bd1ba8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.009+0000 7fb560cd3640 1 --2- 172.21.15.43:0/1743949276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17dd730 0x563bd1ba9150 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009610 tx=0x7fb54c0e2270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d2ff9667c84d74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.009+0000 7fb544ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.009+0000 7fb544ff9640 1 -- 172.21.15.43:0/1743949276 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c021070 con 0x563bd17dd730 2024-04-25T10:42:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.009+0000 7fb544ff9640 1 -- 172.21.15.43:0/1743949276 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fb54c0a6140 con 0x563bd17dd730 2024-04-25T10:42:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.009+0000 7fb56a5ec740 1 -- 172.21.15.43:0/1743949276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5580047a0 con 0x563bd17dd730 2024-04-25T10:42:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.009+0000 7fb544ff9640 1 -- 172.21.15.43:0/1743949276 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb54c0a2b90 con 0x563bd17dd730 2024-04-25T10:42:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.009+0000 7fb56a5ec740 10 client.18462 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.009+0000 7fb56a5ec740 1 -- 172.21.15.43:0/1743949276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563bd17d6170 con 0x563bd17dd730 2024-04-25T10:42:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.010+0000 7fb544ff9640 1 -- 172.21.15.43:0/1743949276 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5580047a0 con 0x563bd17dd730 2024-04-25T10:42:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.010+0000 7fb544ff9640 1 -- 172.21.15.43:0/1743949276 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb54c0a59a0 con 0x563bd17dd730 2024-04-25T10:42:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.010+0000 7fb56a5ec740 10 client.18462 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.010+0000 7fb56a5ec740 10 client.18462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.010+0000 7fb56a5ec740 20 client.18462 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.010+0000 7fb56a5ec740 1 --2- 172.21.15.43:0/1743949276 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563bd17bd1f0 0x563bd1bbb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.010+0000 7fb56a5ec740 4 client.18462 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:42:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.010+0000 7fb56a5ec740 1 -- 172.21.15.43:0/1743949276 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bd1b12710 con 0x563bd17bd1f0 2024-04-25T10:42:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.010+0000 7fb5614d4640 1 --2- 172.21.15.43:0/1743949276 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563bd17bd1f0 0x563bd1bbb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.011+0000 7fb544ff9640 10 client.18462 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.011+0000 7fb5614d4640 1 --2- 172.21.15.43:0/1743949276 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563bd17bd1f0 0x563bd1bbb030 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7465428f7681809 server_cookie=63573ba45abdd78c in_seq=0 out_seq=0 2024-04-25T10:42:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.011+0000 7fb544ff9640 1 -- 172.21.15.43:0/1743949276 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563bd1b12710 con 0x563bd17bd1f0 2024-04-25T10:42:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.011+0000 7fb544ff9640 10 client.18462 handle_command_reply: tid=0 2024-04-25T10:42:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.011+0000 7fb56a5ec740 1 client.18462 shutdown 2024-04-25T10:42:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.012+0000 7fb56a5ec740 1 -- 172.21.15.43:0/1743949276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17dd730 msgr2=0x563bd1ba9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.012+0000 7fb56a5ec740 1 --2- 172.21.15.43:0/1743949276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17dd730 0x563bd1ba9150 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009610 tx=0x7fb54c0e2270 comp rx=0 tx=0).stop 2024-04-25T10:42:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.012+0000 7fb56a5ec740 1 -- 172.21.15.43:0/1743949276 shutdown_connections 2024-04-25T10:42:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.012+0000 7fb56a5ec740 1 --2- 172.21.15.43:0/1743949276 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563bd17bd1f0 0x563bd1bbb030 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.012+0000 7fb56a5ec740 1 --2- 172.21.15.43:0/1743949276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563bd17fe410 0x563bd1bb4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.012+0000 7fb544ff9640 0 client.18462 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.012+0000 7fb56a5ec740 1 --2- 172.21.15.43:0/1743949276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17dd730 0x563bd1ba9150 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.012+0000 7fb56a5ec740 1 --2- 172.21.15.43:0/1743949276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17d5550 0x563bd1ba8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.012+0000 7fb56a5ec740 1 -- 172.21.15.43:0/1743949276 >> 172.21.15.43:0/1743949276 conn(0x563bd17f2560 msgr2=0x563bd1ba0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.012+0000 7fb56a5ec740 1 -- 172.21.15.43:0/1743949276 shutdown_connections 2024-04-25T10:42:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.012+0000 7fb56a5ec740 1 -- 172.21.15.43:0/1743949276 wait complete. 2024-04-25T10:42:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.012+0000 7fb56a5ec740 20 client.18462 trim_cache size 0 max 16384 2024-04-25T10:42:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.030+0000 7fb56a5ec740 1 Processor -- start 2024-04-25T10:42:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.030+0000 7fb56a5ec740 1 -- start start 2024-04-25T10:42:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.031+0000 7fb56a5ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563bd17bd1f0 0x563bd1bc07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.031+0000 7fb56a5ec740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 0x563bd1bc0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.031+0000 7fb56a5ec740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17fe410 0x563bd1b9a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.031+0000 7fb56a5ec740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c1b2270 con 0x563bd17d5550 2024-04-25T10:42:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.031+0000 7fb56a5ec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c1afb40 con 0x563bd17bd1f0 2024-04-25T10:42:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.031+0000 7fb56a5ec740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c1ad9b0 con 0x563bd17fe410 2024-04-25T10:42:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.031+0000 7fb560cd3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 0x563bd1bc0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.031+0000 7fb560cd3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 0x563bd1bc0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49162/0 (socket says 172.21.15.43:49162) 2024-04-25T10:42:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.031+0000 7fb560cd3640 1 -- 172.21.15.43:0/2255164913 learned_addr learned my addr 172.21.15.43:0/2255164913 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.031+0000 7fb5614d4640 1 --2- 172.21.15.43:0/2255164913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563bd17bd1f0 0x563bd1bc07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.031+0000 7fb561cd5640 1 --2- 172.21.15.43:0/2255164913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17fe410 0x563bd1b9a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.031+0000 7fb560cd3640 1 -- 172.21.15.43:0/2255164913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17fe410 msgr2=0x563bd1b9a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.031+0000 7fb560cd3640 1 --2- 172.21.15.43:0/2255164913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17fe410 0x563bd1b9a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.031+0000 7fb560cd3640 1 -- 172.21.15.43:0/2255164913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563bd17bd1f0 msgr2=0x563bd1bc07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.031+0000 7fb560cd3640 1 --2- 172.21.15.43:0/2255164913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563bd17bd1f0 0x563bd1bc07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.031+0000 7fb560cd3640 1 -- 172.21.15.43:0/2255164913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bd1b9fc10 con 0x563bd17d5550 2024-04-25T10:42:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.031+0000 7fb560cd3640 1 --2- 172.21.15.43:0/2255164913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 0x563bd1bc0d30 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0a3f40 tx=0x7fb54c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8450726769ce1a09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.031+0000 7fb546ffd640 1 -- 172.21.15.43:0/2255164913 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c021070 con 0x563bd17d5550 2024-04-25T10:42:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.031+0000 7fb546ffd640 1 -- 172.21.15.43:0/2255164913 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb54c0a73d0 con 0x563bd17d5550 2024-04-25T10:42:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.031+0000 7fb546ffd640 1 -- 172.21.15.43:0/2255164913 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c0a6440 con 0x563bd17d5550 2024-04-25T10:42:25.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.032+0000 7fb56a5ec740 1 -- 172.21.15.43:0/2255164913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 msgr2=0x563bd1bc0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:25.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.032+0000 7fb56a5ec740 1 --2- 172.21.15.43:0/2255164913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 0x563bd1bc0d30 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0a3f40 tx=0x7fb54c00bf90 comp rx=0 tx=0).stop 2024-04-25T10:42:25.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.032+0000 7fb56a5ec740 1 -- 172.21.15.43:0/2255164913 shutdown_connections 2024-04-25T10:42:25.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.032+0000 7fb56a5ec740 1 --2- 172.21.15.43:0/2255164913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17fe410 0x563bd1b9a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.032+0000 7fb56a5ec740 1 --2- 172.21.15.43:0/2255164913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 0x563bd1bc0d30 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.032+0000 7fb56a5ec740 1 --2- 172.21.15.43:0/2255164913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563bd17bd1f0 0x563bd1bc07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.032+0000 7fb56a5ec740 1 -- 172.21.15.43:0/2255164913 >> 172.21.15.43:0/2255164913 conn(0x563bd17f2560 msgr2=0x563bd1ba72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:25.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.032+0000 7fb56a5ec740 1 -- 172.21.15.43:0/2255164913 shutdown_connections 2024-04-25T10:42:25.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.032+0000 7fb56a5ec740 1 -- 172.21.15.43:0/2255164913 wait complete. 2024-04-25T10:42:25.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.033+0000 7fb56a5ec740 1 -- start start 2024-04-25T10:42:25.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.033+0000 7fb56a5ec740 1 Processor -- start 2024-04-25T10:42:25.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.033+0000 7fb56a5ec740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17bd1f0 0x563bd1bb10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.033+0000 7fb56a5ec740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 0x563bd1bac420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.033+0000 7fb56a5ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563bd17fe410 0x563bd1bac960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.033+0000 7fb56a5ec740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c1ae000 con 0x563bd17d5550 2024-04-25T10:42:25.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.033+0000 7fb56a5ec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c1af470 con 0x563bd17fe410 2024-04-25T10:42:25.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.033+0000 7fb56a5ec740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c1ae6d0 con 0x563bd17bd1f0 2024-04-25T10:42:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.033+0000 7fb56a5ec740 10 client.0 _ll_register_callbacks cb 0x7fb54c0dc730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.033+0000 7fb560cd3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 0x563bd1bac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.033+0000 7fb560cd3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 0x563bd1bac420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49166/0 (socket says 172.21.15.43:49166) 2024-04-25T10:42:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.033+0000 7fb560cd3640 1 -- 172.21.15.43:0/2483923790 learned_addr learned my addr 172.21.15.43:0/2483923790 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.033+0000 7fb5614d4640 1 --2- 172.21.15.43:0/2483923790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17bd1f0 0x563bd1bb10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.034+0000 7fb561cd5640 1 --2- 172.21.15.43:0/2483923790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563bd17fe410 0x563bd1bac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.034+0000 7fb560cd3640 1 -- 172.21.15.43:0/2483923790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17bd1f0 msgr2=0x563bd1bb10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.034+0000 7fb560cd3640 1 --2- 172.21.15.43:0/2483923790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17bd1f0 0x563bd1bb10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.034+0000 7fb560cd3640 1 -- 172.21.15.43:0/2483923790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563bd17fe410 msgr2=0x563bd1bac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.034+0000 7fb560cd3640 1 --2- 172.21.15.43:0/2483923790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563bd17fe410 0x563bd1bac960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.034+0000 7fb560cd3640 1 -- 172.21.15.43:0/2483923790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bd1bacea0 con 0x563bd17d5550 2024-04-25T10:42:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.034+0000 7fb560cd3640 1 --2- 172.21.15.43:0/2483923790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 0x563bd1bac420 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0dc880 tx=0x7fb54c0a28f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1bfc321c24c3578 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.034+0000 7fb544ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.034+0000 7fb544ff9640 1 -- 172.21.15.43:0/2483923790 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c021070 con 0x563bd17d5550 2024-04-25T10:42:25.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.034+0000 7fb544ff9640 1 -- 172.21.15.43:0/2483923790 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fb54c0a32c0 con 0x563bd17d5550 2024-04-25T10:42:25.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.034+0000 7fb56a5ec740 1 -- 172.21.15.43:0/2483923790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb54c0a73d0 con 0x563bd17d5550 2024-04-25T10:42:25.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.034+0000 7fb544ff9640 1 -- 172.21.15.43:0/2483923790 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb54c0a6620 con 0x563bd17d5550 2024-04-25T10:42:25.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.034+0000 7fb56a5ec740 10 client.18468 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:25.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.034+0000 7fb56a5ec740 1 -- 172.21.15.43:0/2483923790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563bd1c23090 con 0x563bd17d5550 2024-04-25T10:42:25.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.034+0000 7fb544ff9640 1 -- 172.21.15.43:0/2483923790 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb54c0a73d0 con 0x563bd17d5550 2024-04-25T10:42:25.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.035+0000 7fb544ff9640 1 -- 172.21.15.43:0/2483923790 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb54c0ac420 con 0x563bd17d5550 2024-04-25T10:42:25.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.035+0000 7fb56a5ec740 10 client.18468 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:25.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.035+0000 7fb56a5ec740 10 client.18468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:25.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.035+0000 7fb56a5ec740 20 client.18468 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:25.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.035+0000 7fb56a5ec740 1 --2- 172.21.15.43:0/2483923790 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563bd1792630 0x563bd1c255d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.035+0000 7fb56a5ec740 4 client.18468 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:42:25.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.035+0000 7fb56a5ec740 1 -- 172.21.15.43:0/2483923790 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563bd1b0c580 con 0x563bd1792630 2024-04-25T10:42:25.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.035+0000 7fb5614d4640 1 --2- 172.21.15.43:0/2483923790 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563bd1792630 0x563bd1c255d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.035+0000 7fb544ff9640 10 client.18468 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:25.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.035+0000 7fb5614d4640 1 --2- 172.21.15.43:0/2483923790 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563bd1792630 0x563bd1c255d0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b05b3f67abeef30 server_cookie=b01ae7b7ac5ca35f in_seq=0 out_seq=0 2024-04-25T10:42:25.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.036+0000 7fb544ff9640 1 -- 172.21.15.43:0/2483923790 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563bd1b0c580 con 0x563bd1792630 2024-04-25T10:42:25.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.036+0000 7fb544ff9640 10 client.18468 handle_command_reply: tid=0 2024-04-25T10:42:25.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.036+0000 7fb56a5ec740 1 client.18468 shutdown 2024-04-25T10:42:25.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.037+0000 7fb56a5ec740 1 -- 172.21.15.43:0/2483923790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 msgr2=0x563bd1bac420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:25.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.037+0000 7fb56a5ec740 1 --2- 172.21.15.43:0/2483923790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 0x563bd1bac420 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0dc880 tx=0x7fb54c0a28f0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.037+0000 7fb56a5ec740 1 -- 172.21.15.43:0/2483923790 shutdown_connections 2024-04-25T10:42:25.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.037+0000 7fb56a5ec740 1 --2- 172.21.15.43:0/2483923790 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563bd1792630 0x563bd1c255d0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.037+0000 7fb56a5ec740 1 --2- 172.21.15.43:0/2483923790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563bd17fe410 0x563bd1bac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.037+0000 7fb56a5ec740 1 --2- 172.21.15.43:0/2483923790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd17d5550 0x563bd1bac420 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.037+0000 7fb56a5ec740 1 --2- 172.21.15.43:0/2483923790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd17bd1f0 0x563bd1bb10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.037+0000 7fb56a5ec740 1 -- 172.21.15.43:0/2483923790 >> 172.21.15.43:0/2483923790 conn(0x563bd17f2560 msgr2=0x563bd1b9a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:25.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.037+0000 7fb544ff9640 0 client.18468 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:25.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.037+0000 7fb56a5ec740 1 -- 172.21.15.43:0/2483923790 shutdown_connections 2024-04-25T10:42:25.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.037+0000 7fb56a5ec740 1 -- 172.21.15.43:0/2483923790 wait complete. 2024-04-25T10:42:25.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.037+0000 7fb56a5ec740 20 client.18468 trim_cache size 0 max 16384 2024-04-25T10:42:25.044 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:42:25.044 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:42:25.044 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "f94de101-fbdf-4d5b-a398-ad5dc1a1d9d3", 2024-04-25T10:42:25.044 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:42:25.044 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:42:25.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.037+0000 7fb563738640 1 -- 172.21.15.43:0/2547558915 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb5480afc60 msgr2=0x7fb5480b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:25.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.037+0000 7fb563738640 1 --2- 172.21.15.43:0/2547558915 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb5480afc60 0x7fb5480b2170 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb55402f500 tx=0x7fb5540095c0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.037+0000 7fb563738640 1 -- 172.21.15.43:0/2547558915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c108a20 msgr2=0x7fb55c19ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:25.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.037+0000 7fb563738640 1 --2- 172.21.15.43:0/2547558915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c108a20 0x7fb55c19ac10 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00d950 tx=0x7fb54c00de10 comp rx=0 tx=0).stop 2024-04-25T10:42:25.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.037+0000 7fb563738640 1 -- 172.21.15.43:0/2547558915 shutdown_connections 2024-04-25T10:42:25.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.037+0000 7fb563738640 1 --2- 172.21.15.43:0/2547558915 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb5480afc60 0x7fb5480b2170 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.037+0000 7fb563738640 1 --2- 172.21.15.43:0/2547558915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb55c10d380 0x7fb55c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.038+0000 7fb563738640 1 --2- 172.21.15.43:0/2547558915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c108a20 0x7fb55c19ac10 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.038+0000 7fb563738640 1 --2- 172.21.15.43:0/2547558915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb55c102af0 0x7fb55c19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.038+0000 7fb563738640 1 -- 172.21.15.43:0/2547558915 >> 172.21.15.43:0/2547558915 conn(0x7fb55c078100 msgr2=0x7fb55c069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:25.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.038+0000 7fb563738640 1 -- 172.21.15.43:0/2547558915 shutdown_connections 2024-04-25T10:42:25.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.038+0000 7fb563738640 1 -- 172.21.15.43:0/2547558915 wait complete. 2024-04-25T10:42:25.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:42:25.051 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:42:25.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.162+0000 7f18fc955640 1 -- 172.21.15.43:0/4254768424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f810b650 msgr2=0x7f18f810da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:25.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.162+0000 7f18fc955640 1 --2- 172.21.15.43:0/4254768424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f810b650 0x7f18f810da70 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009900 tx=0x7f18ec02a410 comp rx=0 tx=0).stop 2024-04-25T10:42:25.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.163+0000 7f18fc955640 1 -- 172.21.15.43:0/4254768424 shutdown_connections 2024-04-25T10:42:25.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.163+0000 7f18fc955640 1 --2- 172.21.15.43:0/4254768424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18f810e040 0x7f18f8110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.163+0000 7f18fc955640 1 --2- 172.21.15.43:0/4254768424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f810b650 0x7f18f810da70 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.163+0000 7f18fc955640 1 --2- 172.21.15.43:0/4254768424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18f8104bf0 0x7f18f8105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.163+0000 7f18fc955640 1 -- 172.21.15.43:0/4254768424 >> 172.21.15.43:0/4254768424 conn(0x7f18f8078100 msgr2=0x7f18f8078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:25.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.163+0000 7f18fc955640 1 -- 172.21.15.43:0/4254768424 shutdown_connections 2024-04-25T10:42:25.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.163+0000 7f18fc955640 1 -- 172.21.15.43:0/4254768424 wait complete. 2024-04-25T10:42:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.163+0000 7f18fc955640 1 Processor -- start 2024-04-25T10:42:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.163+0000 7f18fc955640 1 -- start start 2024-04-25T10:42:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.164+0000 7f18fc955640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18f8104bf0 0x7f18f819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.164+0000 7f18fc955640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18f810b650 0x7f18f819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.164+0000 7f18fc955640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f810e040 0x7f18f819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.164+0000 7f18fc955640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f18f8113ca0 con 0x7f18f810e040 2024-04-25T10:42:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.164+0000 7f18fc955640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f18f8113b20 con 0x7f18f810b650 2024-04-25T10:42:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.164+0000 7f18fc955640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f18f8113e20 con 0x7f18f8104bf0 2024-04-25T10:42:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.164+0000 7f18f6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f810e040 0x7f18f819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.164+0000 7f18f6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f810e040 0x7f18f819ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49172/0 (socket says 172.21.15.43:49172) 2024-04-25T10:42:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.164+0000 7f18f6d9d640 1 -- 172.21.15.43:0/1320611774 learned_addr learned my addr 172.21.15.43:0/1320611774 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.164+0000 7f18f659c640 1 --2- 172.21.15.43:0/1320611774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18f8104bf0 0x7f18f819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.164+0000 7f18f5d9b640 1 --2- 172.21.15.43:0/1320611774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18f810b650 0x7f18f819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.164+0000 7f18f6d9d640 1 -- 172.21.15.43:0/1320611774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18f8104bf0 msgr2=0x7f18f819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.164+0000 7f18f6d9d640 1 --2- 172.21.15.43:0/1320611774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18f8104bf0 0x7f18f819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.164+0000 7f18f6d9d640 1 -- 172.21.15.43:0/1320611774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18f810b650 msgr2=0x7f18f819ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.164+0000 7f18f6d9d640 1 --2- 172.21.15.43:0/1320611774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18f810b650 0x7f18f819ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.164+0000 7f18f6d9d640 1 -- 172.21.15.43:0/1320611774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f819f4e0 con 0x7f18f810e040 2024-04-25T10:42:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.164+0000 7f18f6d9d640 1 --2- 172.21.15.43:0/1320611774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f810e040 0x7f18f819ef20 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f18f000cd10 tx=0x7f18f000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.165+0000 7f18e77fe640 1 -- 172.21.15.43:0/1320611774 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f000f450 con 0x7f18f810e040 2024-04-25T10:42:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.165+0000 7f18fc955640 1 -- 172.21.15.43:0/1320611774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f819f7d0 con 0x7f18f810e040 2024-04-25T10:42:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.165+0000 7f18e77fe640 1 -- 172.21.15.43:0/1320611774 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18f000faa0 con 0x7f18f810e040 2024-04-25T10:42:25.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.165+0000 7f18e77fe640 1 -- 172.21.15.43:0/1320611774 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0018b50 con 0x7f18f810e040 2024-04-25T10:42:25.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.165+0000 7f18fc955640 1 -- 172.21.15.43:0/1320611774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f819fcf0 con 0x7f18f810e040 2024-04-25T10:42:25.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.167+0000 7f1903805740 1 Processor -- start 2024-04-25T10:42:25.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.167+0000 7f18e77fe640 1 -- 172.21.15.43:0/1320611774 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f18f000fc50 con 0x7f18f810e040 2024-04-25T10:42:25.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.167+0000 7f1903805740 1 -- start start 2024-04-25T10:42:25.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.168+0000 7f1903805740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4d7390 0x55a80e877950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.168+0000 7f1903805740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4ae4d0 0x55a80e877e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.168+0000 7f1903805740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e4b66b0 0x55a80e888d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.168+0000 7f1903805740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a80e7ddd80 con 0x55a80e4ae4d0 2024-04-25T10:42:25.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.168+0000 7f1903805740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a80e7978b0 con 0x55a80e4d7390 2024-04-25T10:42:25.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.168+0000 7f1903805740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a80e7d9d40 con 0x55a80e4b66b0 2024-04-25T10:42:25.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.168+0000 7f18f659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4ae4d0 0x55a80e877e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.168+0000 7f18f659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4ae4d0 0x55a80e877e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49178/0 (socket says 172.21.15.43:49178) 2024-04-25T10:42:25.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.168+0000 7f18f659c640 1 -- 172.21.15.43:0/3139057189 learned_addr learned my addr 172.21.15.43:0/3139057189 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:25.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.168+0000 7f18e77fe640 1 --2- 172.21.15.43:0/1320611774 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f18e00afc60 0x7f18e00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.168+0000 7f18f5d9b640 1 --2- 172.21.15.43:0/3139057189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4d7390 0x55a80e877950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.168+0000 7f18f6d9d640 1 --2- 172.21.15.43:0/1320611774 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f18e00afc60 0x7f18e00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.168+0000 7f18f5d9b640 1 --2- 172.21.15.43:0/3139057189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e4b66b0 0x55a80e888d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.168+0000 7f18f659c640 1 -- 172.21.15.43:0/3139057189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e4b66b0 msgr2=0x55a80e888d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:25.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.168+0000 7f18f659c640 1 --2- 172.21.15.43:0/3139057189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e4b66b0 0x55a80e888d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.168+0000 7f18f659c640 1 -- 172.21.15.43:0/3139057189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4d7390 msgr2=0x55a80e877950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:25.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.168+0000 7f18f659c640 1 --2- 172.21.15.43:0/3139057189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4d7390 0x55a80e877950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.168+0000 7f18f659c640 1 -- 172.21.15.43:0/3139057189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a80e768a40 con 0x55a80e4ae4d0 2024-04-25T10:42:25.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.168+0000 7f18f5d9b640 1 --2- 172.21.15.43:0/3139057189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e4b66b0 0x55a80e888d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:25.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.168+0000 7f18e77fe640 1 -- 172.21.15.43:0/1320611774 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f18f00df420 con 0x7f18f810e040 2024-04-25T10:42:25.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.169+0000 7f18f659c640 1 --2- 172.21.15.43:0/3139057189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4ae4d0 0x55a80e877e90 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f18e8009c80 tx=0x7f18e80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f37f2e7e7a444e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:25.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.169+0000 7f18e4ff9640 1 -- 172.21.15.43:0/3139057189 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e8012450 con 0x55a80e4ae4d0 2024-04-25T10:42:25.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.169+0000 7f18f6d9d640 1 --2- 172.21.15.43:0/1320611774 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f18e00afc60 0x7f18e00b2170 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f18f00dd170 tx=0x7f18f0071d90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:25.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.169+0000 7f18e4ff9640 1 -- 172.21.15.43:0/3139057189 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18e8012aa0 con 0x55a80e4ae4d0 2024-04-25T10:42:25.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.169+0000 7f18e4ff9640 1 -- 172.21.15.43:0/3139057189 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e800fad0 con 0x55a80e4ae4d0 2024-04-25T10:42:25.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.170+0000 7f1903805740 1 -- 172.21.15.43:0/3139057189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4ae4d0 msgr2=0x55a80e877e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:25.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.170+0000 7f1903805740 1 --2- 172.21.15.43:0/3139057189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4ae4d0 0x55a80e877e90 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f18e8009c80 tx=0x7f18e80117b0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.170+0000 7f1903805740 1 -- 172.21.15.43:0/3139057189 shutdown_connections 2024-04-25T10:42:25.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.170+0000 7f1903805740 1 --2- 172.21.15.43:0/3139057189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e4b66b0 0x55a80e888d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.170+0000 7f1903805740 1 --2- 172.21.15.43:0/3139057189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4ae4d0 0x55a80e877e90 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.170+0000 7f1903805740 1 --2- 172.21.15.43:0/3139057189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4d7390 0x55a80e877950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.170+0000 7f1903805740 1 -- 172.21.15.43:0/3139057189 >> 172.21.15.43:0/3139057189 conn(0x55a80e4cb4e0 msgr2=0x55a80e874210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:25.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.170+0000 7f1903805740 1 -- 172.21.15.43:0/3139057189 shutdown_connections 2024-04-25T10:42:25.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.170+0000 7f1903805740 1 -- 172.21.15.43:0/3139057189 wait complete. 2024-04-25T10:42:25.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.170+0000 7f1903805740 1 -- start start 2024-04-25T10:42:25.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.170+0000 7f1903805740 1 Processor -- start 2024-04-25T10:42:25.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.170+0000 7f1903805740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e4ae4d0 0x55a80e886950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.170+0000 7f1903805740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4b66b0 0x55a80e886e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f1903805740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4d7390 0x55a80e892930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f1903805740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a80e8893f0 con 0x55a80e4b66b0 2024-04-25T10:42:25.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f1903805740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a80e764d10 con 0x55a80e4d7390 2024-04-25T10:42:25.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f1903805740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a80e889570 con 0x55a80e4ae4d0 2024-04-25T10:42:25.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f1903805740 10 client.0 _ll_register_callbacks cb 0x55a80e791210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:25.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f18f659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4b66b0 0x55a80e886e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f18f659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4b66b0 0x55a80e886e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49194/0 (socket says 172.21.15.43:49194) 2024-04-25T10:42:25.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f18f659c640 1 -- 172.21.15.43:0/3041189783 learned_addr learned my addr 172.21.15.43:0/3041189783 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:25.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f18f5d9b640 1 --2- 172.21.15.43:0/3041189783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4d7390 0x55a80e892930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f18f5d9b640 1 --2- 172.21.15.43:0/3041189783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e4ae4d0 0x55a80e886950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f18f659c640 1 -- 172.21.15.43:0/3041189783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e4ae4d0 msgr2=0x55a80e886950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:25.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f18f659c640 1 --2- 172.21.15.43:0/3041189783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e4ae4d0 0x55a80e886950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f18f659c640 1 -- 172.21.15.43:0/3041189783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4d7390 msgr2=0x55a80e892930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:25.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f18f659c640 1 --2- 172.21.15.43:0/3041189783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4d7390 0x55a80e892930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f18f659c640 1 -- 172.21.15.43:0/3041189783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a80e5b97d0 con 0x55a80e4b66b0 2024-04-25T10:42:25.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f18f5d9b640 1 --2- 172.21.15.43:0/3041189783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4d7390 0x55a80e892930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:25.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f18f659c640 1 --2- 172.21.15.43:0/3041189783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4b66b0 0x55a80e886e90 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f18e8000c00 tx=0x7f18e8009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a16be55ae30c76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:25.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f18ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:25.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f18ddffb640 1 -- 172.21.15.43:0/3041189783 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e8004450 con 0x55a80e4b66b0 2024-04-25T10:42:25.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f18ddffb640 1 -- 172.21.15.43:0/3041189783 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f18e8012450 con 0x55a80e4b66b0 2024-04-25T10:42:25.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f1903805740 1 -- 172.21.15.43:0/3041189783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18e8012aa0 con 0x55a80e4b66b0 2024-04-25T10:42:25.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f18ddffb640 1 -- 172.21.15.43:0/3041189783 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f18e8032050 con 0x55a80e4b66b0 2024-04-25T10:42:25.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f1903805740 10 client.18480 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:25.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.171+0000 7f1903805740 1 -- 172.21.15.43:0/3041189783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a80e4ef3f0 con 0x55a80e4b66b0 2024-04-25T10:42:25.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.172+0000 7f18ddffb640 1 -- 172.21.15.43:0/3041189783 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18e8012aa0 con 0x55a80e4b66b0 2024-04-25T10:42:25.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.172+0000 7f18ddffb640 1 -- 172.21.15.43:0/3041189783 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f18e800f9d0 con 0x55a80e4b66b0 2024-04-25T10:42:25.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.172+0000 7f1903805740 10 client.18480 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:25.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.172+0000 7f1903805740 10 client.18480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:25.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.172+0000 7f1903805740 20 client.18480 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:25.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.172+0000 7f1903805740 1 --2- 172.21.15.43:0/3041189783 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a80e495f60 0x55a80e898920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.172+0000 7f1903805740 4 client.18480 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:42:25.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.172+0000 7f1903805740 1 -- 172.21.15.43:0/3041189783 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a80e7eb990 con 0x55a80e495f60 2024-04-25T10:42:25.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.172+0000 7f18f5d9b640 1 --2- 172.21.15.43:0/3041189783 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a80e495f60 0x55a80e898920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.173+0000 7f18f5d9b640 1 --2- 172.21.15.43:0/3041189783 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a80e495f60 0x55a80e898920 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f78a4b295273c68c server_cookie=d699c9fc3ad4051b in_seq=0 out_seq=0 2024-04-25T10:42:25.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.173+0000 7f18ddffb640 10 client.18480 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:25.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.173+0000 7f18ddffb640 1 -- 172.21.15.43:0/3041189783 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a80e7eb990 con 0x55a80e495f60 2024-04-25T10:42:25.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.173+0000 7f18ddffb640 10 client.18480 handle_command_reply: tid=0 2024-04-25T10:42:25.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.173+0000 7f1903805740 1 client.18480 shutdown 2024-04-25T10:42:25.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.174+0000 7f1903805740 1 -- 172.21.15.43:0/3041189783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4b66b0 msgr2=0x55a80e886e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:25.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.174+0000 7f1903805740 1 --2- 172.21.15.43:0/3041189783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4b66b0 0x55a80e886e90 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f18e8000c00 tx=0x7f18e8009fa0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.174+0000 7f1903805740 1 -- 172.21.15.43:0/3041189783 shutdown_connections 2024-04-25T10:42:25.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.174+0000 7f1903805740 1 --2- 172.21.15.43:0/3041189783 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a80e495f60 0x55a80e898920 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.174+0000 7f1903805740 1 --2- 172.21.15.43:0/3041189783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4d7390 0x55a80e892930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.174+0000 7f1903805740 1 --2- 172.21.15.43:0/3041189783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4b66b0 0x55a80e886e90 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.174+0000 7f18ddffb640 0 client.18480 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:25.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.174+0000 7f1903805740 1 --2- 172.21.15.43:0/3041189783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e4ae4d0 0x55a80e886950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.174+0000 7f1903805740 1 -- 172.21.15.43:0/3041189783 >> 172.21.15.43:0/3041189783 conn(0x55a80e4cb4e0 msgr2=0x55a80e8796f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:25.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.174+0000 7f1903805740 1 -- 172.21.15.43:0/3041189783 shutdown_connections 2024-04-25T10:42:25.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.174+0000 7f1903805740 1 -- 172.21.15.43:0/3041189783 wait complete. 2024-04-25T10:42:25.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.174+0000 7f1903805740 20 client.18480 trim_cache size 0 max 16384 2024-04-25T10:42:25.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.192+0000 7f1903805740 1 Processor -- start 2024-04-25T10:42:25.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.192+0000 7f1903805740 1 -- start start 2024-04-25T10:42:25.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.192+0000 7f1903805740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e495f60 0x55a80e8730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.192+0000 7f1903805740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4ae4d0 0x55a80e873610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.192+0000 7f1903805740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4b66b0 0x55a80e89b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.192+0000 7f1903805740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f18f81aff70 con 0x55a80e4b66b0 2024-04-25T10:42:25.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.192+0000 7f1903805740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f18f81ad840 con 0x55a80e4ae4d0 2024-04-25T10:42:25.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.192+0000 7f1903805740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f18f81ab6d0 con 0x55a80e495f60 2024-04-25T10:42:25.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.192+0000 7f18f5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4b66b0 0x55a80e89b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.192+0000 7f18f5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4b66b0 0x55a80e89b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49196/0 (socket says 172.21.15.43:49196) 2024-04-25T10:42:25.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.192+0000 7f18f5d9b640 1 -- 172.21.15.43:0/609681264 learned_addr learned my addr 172.21.15.43:0/609681264 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:25.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.192+0000 7f18f5d9b640 1 --2- 172.21.15.43:0/609681264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e495f60 0x55a80e8730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.192+0000 7f18f659c640 1 --2- 172.21.15.43:0/609681264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4ae4d0 0x55a80e873610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.192+0000 7f18f5d9b640 1 -- 172.21.15.43:0/609681264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e495f60 msgr2=0x55a80e8730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:25.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.192+0000 7f18f5d9b640 1 --2- 172.21.15.43:0/609681264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e495f60 0x55a80e8730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.192+0000 7f18f5d9b640 1 -- 172.21.15.43:0/609681264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4ae4d0 msgr2=0x55a80e873610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:25.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.192+0000 7f18f5d9b640 1 --2- 172.21.15.43:0/609681264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4ae4d0 0x55a80e873610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.192+0000 7f18f5d9b640 1 -- 172.21.15.43:0/609681264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a80e4af0f0 con 0x55a80e4b66b0 2024-04-25T10:42:25.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.193+0000 7f18f5d9b640 1 --2- 172.21.15.43:0/609681264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4b66b0 0x55a80e89b950 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f18ec031440 tx=0x7f18ec03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab92f8e9cde59d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:25.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.193+0000 7f18dffff640 1 -- 172.21.15.43:0/609681264 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec035040 con 0x55a80e4b66b0 2024-04-25T10:42:25.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.193+0000 7f18dffff640 1 -- 172.21.15.43:0/609681264 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18ec048760 con 0x55a80e4b66b0 2024-04-25T10:42:25.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.193+0000 7f18dffff640 1 -- 172.21.15.43:0/609681264 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec003990 con 0x55a80e4b66b0 2024-04-25T10:42:25.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.194+0000 7f1903805740 1 -- 172.21.15.43:0/609681264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4b66b0 msgr2=0x55a80e89b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:25.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.194+0000 7f1903805740 1 --2- 172.21.15.43:0/609681264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4b66b0 0x55a80e89b950 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f18ec031440 tx=0x7f18ec03f040 comp rx=0 tx=0).stop 2024-04-25T10:42:25.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.194+0000 7f1903805740 1 -- 172.21.15.43:0/609681264 shutdown_connections 2024-04-25T10:42:25.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.194+0000 7f1903805740 1 --2- 172.21.15.43:0/609681264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4ae4d0 0x55a80e873610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.194+0000 7f1903805740 1 --2- 172.21.15.43:0/609681264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e4b66b0 0x55a80e89b950 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.194+0000 7f1903805740 1 --2- 172.21.15.43:0/609681264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e495f60 0x55a80e8730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.194+0000 7f1903805740 1 -- 172.21.15.43:0/609681264 >> 172.21.15.43:0/609681264 conn(0x55a80e4cb4e0 msgr2=0x55a80e8729c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:25.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.194+0000 7f1903805740 1 -- 172.21.15.43:0/609681264 shutdown_connections 2024-04-25T10:42:25.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.194+0000 7f1903805740 1 -- 172.21.15.43:0/609681264 wait complete. 2024-04-25T10:42:25.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.194+0000 7f1903805740 1 -- start start 2024-04-25T10:42:25.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.194+0000 7f1903805740 1 Processor -- start 2024-04-25T10:42:25.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.194+0000 7f1903805740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e495f60 0x55a80e89b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.194+0000 7f1903805740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4ae4d0 0x55a80e89b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.194+0000 7f1903805740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e4b66b0 0x55a80e8f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.194+0000 7f1903805740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f18f81abd20 con 0x55a80e495f60 2024-04-25T10:42:25.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.194+0000 7f1903805740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f18f81ad170 con 0x55a80e4ae4d0 2024-04-25T10:42:25.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.194+0000 7f1903805740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f18f81ac3d0 con 0x55a80e4b66b0 2024-04-25T10:42:25.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.194+0000 7f1903805740 10 client.0 _ll_register_callbacks cb 0x7f18e8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:25.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.194+0000 7f18f5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e495f60 0x55a80e89b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.195+0000 7f18f5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e495f60 0x55a80e89b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49202/0 (socket says 172.21.15.43:49202) 2024-04-25T10:42:25.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.195+0000 7f18f5d9b640 1 -- 172.21.15.43:0/355171876 learned_addr learned my addr 172.21.15.43:0/355171876 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:25.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.195+0000 7f18f659c640 1 --2- 172.21.15.43:0/355171876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4ae4d0 0x55a80e89b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.195+0000 7f18f5d9b640 1 --2- 172.21.15.43:0/355171876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e4b66b0 0x55a80e8f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.195+0000 7f18f5d9b640 1 -- 172.21.15.43:0/355171876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e4b66b0 msgr2=0x55a80e8f7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:25.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.195+0000 7f18f5d9b640 1 --2- 172.21.15.43:0/355171876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e4b66b0 0x55a80e8f7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.195+0000 7f18f5d9b640 1 -- 172.21.15.43:0/355171876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4ae4d0 msgr2=0x55a80e89b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:25.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.195+0000 7f18f5d9b640 1 --2- 172.21.15.43:0/355171876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4ae4d0 0x55a80e89b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.195+0000 7f18f5d9b640 1 -- 172.21.15.43:0/355171876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a80e79e4b0 con 0x55a80e495f60 2024-04-25T10:42:25.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.195+0000 7f18f5d9b640 1 --2- 172.21.15.43:0/355171876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e495f60 0x55a80e89b3f0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f18ec02a450 tx=0x7f18ec0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe52732734ee99c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:25.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.195+0000 7f18ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:25.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.195+0000 7f18ddffb640 1 -- 172.21.15.43:0/355171876 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec035040 con 0x55a80e495f60 2024-04-25T10:42:25.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.195+0000 7f18ddffb640 1 -- 172.21.15.43:0/355171876 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f18ec0482b0 con 0x55a80e495f60 2024-04-25T10:42:25.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.195+0000 7f1903805740 1 -- 172.21.15.43:0/355171876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18ec048760 con 0x55a80e495f60 2024-04-25T10:42:25.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.195+0000 7f18ddffb640 1 -- 172.21.15.43:0/355171876 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f18ec0484b0 con 0x55a80e495f60 2024-04-25T10:42:25.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.195+0000 7f1903805740 10 client.18486 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:25.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.195+0000 7f1903805740 1 -- 172.21.15.43:0/355171876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a80e9006d0 con 0x55a80e495f60 2024-04-25T10:42:25.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.196+0000 7f18ddffb640 1 -- 172.21.15.43:0/355171876 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18ec048760 con 0x55a80e495f60 2024-04-25T10:42:25.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.196+0000 7f18ddffb640 1 -- 172.21.15.43:0/355171876 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f18ec05d070 con 0x55a80e495f60 2024-04-25T10:42:25.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.196+0000 7f1903805740 10 client.18486 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:25.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.196+0000 7f1903805740 10 client.18486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:25.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.196+0000 7f1903805740 20 client.18486 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:25.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.196+0000 7f1903805740 1 --2- 172.21.15.43:0/355171876 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a80e4d7390 0x55a80e902cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:25.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.196+0000 7f1903805740 4 client.18486 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:42:25.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.196+0000 7f1903805740 1 -- 172.21.15.43:0/355171876 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a80e7e5800 con 0x55a80e4d7390 2024-04-25T10:42:25.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.196+0000 7f18f659c640 1 --2- 172.21.15.43:0/355171876 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a80e4d7390 0x55a80e902cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:25.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.196+0000 7f18ddffb640 10 client.18486 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:25.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.196+0000 7f18f659c640 1 --2- 172.21.15.43:0/355171876 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a80e4d7390 0x55a80e902cd0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3847c083fc41ef2f server_cookie=d54f066a9923c593 in_seq=0 out_seq=0 2024-04-25T10:42:25.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f18ddffb640 1 -- 172.21.15.43:0/355171876 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a80e7e5800 con 0x55a80e4d7390 2024-04-25T10:42:25.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f18ddffb640 10 client.18486 handle_command_reply: tid=0 2024-04-25T10:42:25.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f1903805740 1 client.18486 shutdown 2024-04-25T10:42:25.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f1903805740 1 -- 172.21.15.43:0/355171876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e495f60 msgr2=0x55a80e89b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:25.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f1903805740 1 --2- 172.21.15.43:0/355171876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e495f60 0x55a80e89b3f0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f18ec02a450 tx=0x7f18ec0035f0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f1903805740 1 -- 172.21.15.43:0/355171876 shutdown_connections 2024-04-25T10:42:25.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f1903805740 1 --2- 172.21.15.43:0/355171876 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a80e4d7390 0x55a80e902cd0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f1903805740 1 --2- 172.21.15.43:0/355171876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a80e4ae4d0 0x55a80e89b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f18ddffb640 0 client.18486 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:25.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f1903805740 1 --2- 172.21.15.43:0/355171876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a80e4b66b0 0x55a80e8f7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f1903805740 1 --2- 172.21.15.43:0/355171876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a80e495f60 0x55a80e89b3f0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f1903805740 1 -- 172.21.15.43:0/355171876 >> 172.21.15.43:0/355171876 conn(0x55a80e4cb4e0 msgr2=0x55a80e8733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:25.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f1903805740 1 -- 172.21.15.43:0/355171876 shutdown_connections 2024-04-25T10:42:25.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f1903805740 1 -- 172.21.15.43:0/355171876 wait complete. 2024-04-25T10:42:25.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f1903805740 20 client.18486 trim_cache size 0 max 16384 2024-04-25T10:42:25.205 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:42:25.205 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:42:25.205 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:42:25.205 INFO:teuthology.orchestra.run.smithi043.stdout: "f94de101-fbdf-4d5b-a398-ad5dc1a1d9d3": { 2024-04-25T10:42:25.206 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:42:25.206 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "f94de101-fbdf-4d5b-a398-ad5dc1a1d9d3", 2024-04-25T10:42:25.206 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:42:25.206 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:42:25.206 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:42:25.206 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:42:25.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f18fc955640 1 -- 172.21.15.43:0/1320611774 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f18e00afc60 msgr2=0x7f18e00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:25.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f18fc955640 1 --2- 172.21.15.43:0/1320611774 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f18e00afc60 0x7f18e00b2170 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f18f00dd170 tx=0x7f18f0071d90 comp rx=0 tx=0).stop 2024-04-25T10:42:25.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f18fc955640 1 -- 172.21.15.43:0/1320611774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f810e040 msgr2=0x7f18f819ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:25.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f18fc955640 1 --2- 172.21.15.43:0/1320611774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f810e040 0x7f18f819ef20 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f18f000cd10 tx=0x7f18f000a620 comp rx=0 tx=0).stop 2024-04-25T10:42:25.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f18fc955640 1 -- 172.21.15.43:0/1320611774 shutdown_connections 2024-04-25T10:42:25.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.197+0000 7f18fc955640 1 --2- 172.21.15.43:0/1320611774 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f18e00afc60 0x7f18e00b2170 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.198+0000 7f18fc955640 1 --2- 172.21.15.43:0/1320611774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18f810b650 0x7f18f819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.198+0000 7f18fc955640 1 --2- 172.21.15.43:0/1320611774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f810e040 0x7f18f819ef20 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.198+0000 7f18fc955640 1 --2- 172.21.15.43:0/1320611774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18f8104bf0 0x7f18f819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:25.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.198+0000 7f18fc955640 1 -- 172.21.15.43:0/1320611774 >> 172.21.15.43:0/1320611774 conn(0x7f18f8078100 msgr2=0x7f18f8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:25.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.198+0000 7f18fc955640 1 -- 172.21.15.43:0/1320611774 shutdown_connections 2024-04-25T10:42:25.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:25.198+0000 7f18fc955640 1 -- 172.21.15.43:0/1320611774 wait complete. 2024-04-25T10:42:25.210 INFO:tasks.cephfs.filesystem:scrub status for tag:f94de101-fbdf-4d5b-a398-ad5dc1a1d9d3 - {'path': '/', 'tag': 'f94de101-fbdf-4d5b-a398-ad5dc1a1d9d3', 'options': 'recursive,force'} 2024-04-25T10:42:26.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:25 smithi077 ceph-mon[32772]: pgmap v308: 97 pgs: 97 active+clean; 2.6 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 70 KiB/s wr, 20 op/s 2024-04-25T10:42:26.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:25 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:42:26.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:25 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:42:26.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:25 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:42:26.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:25 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:42:26.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:25 smithi164 ceph-mon[32713]: pgmap v308: 97 pgs: 97 active+clean; 2.6 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 70 KiB/s wr, 20 op/s 2024-04-25T10:42:26.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:25 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:42:26.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:25 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:42:26.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:25 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:42:26.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:25 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:42:26.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:25 smithi043 ceph-mon[29478]: pgmap v308: 97 pgs: 97 active+clean; 2.6 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 70 KiB/s wr, 20 op/s 2024-04-25T10:42:26.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:25 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:42:26.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:25 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:42:26.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:25 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:42:26.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:25 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:42:28.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:27 smithi077 ceph-mon[32772]: pgmap v309: 97 pgs: 97 active+clean; 2.6 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:42:28.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:27 smithi164 ceph-mon[32713]: pgmap v309: 97 pgs: 97 active+clean; 2.6 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:42:28.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:27 smithi043 ceph-mon[29478]: pgmap v309: 97 pgs: 97 active+clean; 2.6 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:42:29.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:28 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:42:29.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:28 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:42:29.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:28 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:42:30.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:29 smithi077 ceph-mon[32772]: pgmap v310: 97 pgs: 97 active+clean; 2.8 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 62 KiB/s wr, 18 op/s 2024-04-25T10:42:30.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:29 smithi164 ceph-mon[32713]: pgmap v310: 97 pgs: 97 active+clean; 2.8 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 62 KiB/s wr, 18 op/s 2024-04-25T10:42:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:29 smithi043 ceph-mon[29478]: pgmap v310: 97 pgs: 97 active+clean; 2.8 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 62 KiB/s wr, 18 op/s 2024-04-25T10:42:32.089 INFO:tasks.workunit.client.0.smithi043.stdout:437/600: open fd = 3 2024-04-25T10:42:32.090 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.090 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.090 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.090 INFO:tasks.workunit.client.0.smithi043.stdout:438/600: open fd = 3 2024-04-25T10:42:32.090 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.090 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.090 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.090 INFO:tasks.workunit.client.0.smithi043.stdout:439/600: open fd = 3 2024-04-25T10:42:32.090 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.090 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.090 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.090 INFO:tasks.workunit.client.0.smithi043.stdout:440/600: open fd = 3 2024-04-25T10:42:32.090 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.091 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.091 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.091 INFO:tasks.workunit.client.0.smithi043.stdout:441/600: open fd = 3 2024-04-25T10:42:32.091 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.091 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.091 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.091 INFO:tasks.workunit.client.0.smithi043.stdout:442/600: open fd = 3 2024-04-25T10:42:32.091 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.091 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.091 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.091 INFO:tasks.workunit.client.0.smithi043.stdout:443/600: open fd = 3 2024-04-25T10:42:32.091 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.091 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.091 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.092 INFO:tasks.workunit.client.0.smithi043.stdout:444/600: open fd = 3 2024-04-25T10:42:32.092 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.092 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.092 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.092 INFO:tasks.workunit.client.0.smithi043.stdout:445/600: open fd = 3 2024-04-25T10:42:32.092 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.092 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.092 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.092 INFO:tasks.workunit.client.0.smithi043.stdout:446/600: open fd = 3 2024-04-25T10:42:32.092 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.092 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.092 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.092 INFO:tasks.workunit.client.0.smithi043.stdout:447/600: open fd = 3 2024-04-25T10:42:32.093 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.093 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.093 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.093 INFO:tasks.workunit.client.0.smithi043.stdout:448/600: open fd = 3 2024-04-25T10:42:32.093 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.093 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.093 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.093 INFO:tasks.workunit.client.0.smithi043.stdout:449/600: open fd = 3 2024-04-25T10:42:32.093 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.093 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.093 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.093 INFO:tasks.workunit.client.0.smithi043.stdout:450/600: open fd = 3 2024-04-25T10:42:32.093 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.093 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.094 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.094 INFO:tasks.workunit.client.0.smithi043.stdout:451/600: open fd = 3 2024-04-25T10:42:32.094 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.094 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.094 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.094 INFO:tasks.workunit.client.0.smithi043.stdout:452/600: open fd = 3 2024-04-25T10:42:32.094 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.094 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.094 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.094 INFO:tasks.workunit.client.0.smithi043.stdout:453/600: open fd = 3 2024-04-25T10:42:32.094 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.094 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.094 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.095 INFO:tasks.workunit.client.0.smithi043.stdout:454/600: open fd = 3 2024-04-25T10:42:32.095 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.095 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.095 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.095 INFO:tasks.workunit.client.0.smithi043.stdout:455/600: open fd = 3 2024-04-25T10:42:32.095 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.095 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.095 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.095 INFO:tasks.workunit.client.0.smithi043.stdout:456/600: open fd = 3 2024-04-25T10:42:32.095 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.095 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.095 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.095 INFO:tasks.workunit.client.0.smithi043.stdout:457/600: open fd = 3 2024-04-25T10:42:32.095 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.095 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.096 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.096 INFO:tasks.workunit.client.0.smithi043.stdout:458/600: open fd = 3 2024-04-25T10:42:32.096 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.096 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.096 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.096 INFO:tasks.workunit.client.0.smithi043.stdout:459/600: open fd = 3 2024-04-25T10:42:32.096 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.096 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.096 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.096 INFO:tasks.workunit.client.0.smithi043.stdout:460/600: open fd = 3 2024-04-25T10:42:32.096 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.096 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.096 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.096 INFO:tasks.workunit.client.0.smithi043.stdout:461/600: open fd = 3 2024-04-25T10:42:32.097 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.097 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.097 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.097 INFO:tasks.workunit.client.0.smithi043.stdout:462/600: open fd = 3 2024-04-25T10:42:32.097 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.097 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.097 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.097 INFO:tasks.workunit.client.0.smithi043.stdout:463/600: open fd = 3 2024-04-25T10:42:32.097 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.097 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.097 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.097 INFO:tasks.workunit.client.0.smithi043.stdout:464/600: open fd = 3 2024-04-25T10:42:32.097 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.097 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.098 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.098 INFO:tasks.workunit.client.0.smithi043.stdout:465/600: open fd = 3 2024-04-25T10:42:32.098 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.098 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.098 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.098 INFO:tasks.workunit.client.0.smithi043.stdout:466/600: open fd = 3 2024-04-25T10:42:32.098 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.098 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.098 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.098 INFO:tasks.workunit.client.0.smithi043.stdout:467/600: open fd = 3 2024-04-25T10:42:32.098 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.098 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.098 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.098 INFO:tasks.workunit.client.0.smithi043.stdout:468/600: open fd = 3 2024-04-25T10:42:32.099 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.099 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.099 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.099 INFO:tasks.workunit.client.0.smithi043.stdout:469/600: open fd = 3 2024-04-25T10:42:32.099 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.099 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.099 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.099 INFO:tasks.workunit.client.0.smithi043.stdout:470/600: open fd = 3 2024-04-25T10:42:32.099 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.099 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.099 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.099 INFO:tasks.workunit.client.0.smithi043.stdout:471/600: open fd = 3 2024-04-25T10:42:32.099 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.099 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.099 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.100 INFO:tasks.workunit.client.0.smithi043.stdout:472/600: open fd = 3 2024-04-25T10:42:32.100 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.100 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.100 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.100 INFO:tasks.workunit.client.0.smithi043.stdout:473/600: open fd = 3 2024-04-25T10:42:32.100 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.100 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.100 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.100 INFO:tasks.workunit.client.0.smithi043.stdout:474/600: open fd = 3 2024-04-25T10:42:32.100 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.100 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.100 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.100 INFO:tasks.workunit.client.0.smithi043.stdout:475/600: open fd = 3 2024-04-25T10:42:32.100 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.100 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.100 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.101 INFO:tasks.workunit.client.0.smithi043.stdout:476/600: open fd = 3 2024-04-25T10:42:32.101 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.101 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.101 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.101 INFO:tasks.workunit.client.0.smithi043.stdout:477/600: open fd = 3 2024-04-25T10:42:32.101 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.101 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.101 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.101 INFO:tasks.workunit.client.0.smithi043.stdout:478/600: open fd = 3 2024-04-25T10:42:32.101 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.101 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.101 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.101 INFO:tasks.workunit.client.0.smithi043.stdout:479/600: open fd = 3 2024-04-25T10:42:32.102 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.102 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.102 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.102 INFO:tasks.workunit.client.0.smithi043.stdout:480/600: open fd = 3 2024-04-25T10:42:32.102 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.102 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.102 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.102 INFO:tasks.workunit.client.0.smithi043.stdout:481/600: open fd = 3 2024-04-25T10:42:32.102 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.102 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.102 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.102 INFO:tasks.workunit.client.0.smithi043.stdout:482/600: open fd = 3 2024-04-25T10:42:32.102 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.103 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.103 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.103 INFO:tasks.workunit.client.0.smithi043.stdout:483/600: open fd = 3 2024-04-25T10:42:32.103 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.103 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.103 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.103 INFO:tasks.workunit.client.0.smithi043.stdout:484/600: open fd = 3 2024-04-25T10:42:32.103 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.103 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.103 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.103 INFO:tasks.workunit.client.0.smithi043.stdout:485/600: open fd = 3 2024-04-25T10:42:32.103 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.103 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.103 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.104 INFO:tasks.workunit.client.0.smithi043.stdout:486/600: open fd = 3 2024-04-25T10:42:32.104 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.104 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.104 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.104 INFO:tasks.workunit.client.0.smithi043.stdout:487/600: open fd = 3 2024-04-25T10:42:32.104 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.104 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.104 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.104 INFO:tasks.workunit.client.0.smithi043.stdout:488/600: open fd = 3 2024-04-25T10:42:32.104 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.104 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.104 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.104 INFO:tasks.workunit.client.0.smithi043.stdout:489/600: open fd = 3 2024-04-25T10:42:32.104 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.104 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.105 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.105 INFO:tasks.workunit.client.0.smithi043.stdout:490/600: open fd = 3 2024-04-25T10:42:32.105 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.105 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.105 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.105 INFO:tasks.workunit.client.0.smithi043.stdout:491/600: open fd = 3 2024-04-25T10:42:32.105 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.105 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.105 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.105 INFO:tasks.workunit.client.0.smithi043.stdout:492/600: open fd = 3 2024-04-25T10:42:32.105 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.105 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.105 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.105 INFO:tasks.workunit.client.0.smithi043.stdout:493/600: open fd = 3 2024-04-25T10:42:32.105 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.106 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.106 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.106 INFO:tasks.workunit.client.0.smithi043.stdout:494/600: open fd = 3 2024-04-25T10:42:32.106 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.106 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.106 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.106 INFO:tasks.workunit.client.0.smithi043.stdout:495/600: open fd = 3 2024-04-25T10:42:32.106 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.106 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.106 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.106 INFO:tasks.workunit.client.0.smithi043.stdout:496/600: open fd = 3 2024-04-25T10:42:32.106 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.106 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.106 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.107 INFO:tasks.workunit.client.0.smithi043.stdout:497/600: open fd = 3 2024-04-25T10:42:32.107 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.107 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.107 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.107 INFO:tasks.workunit.client.0.smithi043.stdout:498/600: open fd = 3 2024-04-25T10:42:32.107 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.107 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:42:32.107 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:42:32.293 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:31 smithi077 ceph-mon[32772]: pgmap v311: 97 pgs: 97 active+clean; 2.8 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-04-25T10:42:32.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:31 smithi164 ceph-mon[32713]: pgmap v311: 97 pgs: 97 active+clean; 2.8 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-04-25T10:42:32.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:31 smithi043 ceph-mon[29478]: pgmap v311: 97 pgs: 97 active+clean; 2.8 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-04-25T10:42:34.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:33 smithi077 ceph-mon[32772]: pgmap v312: 97 pgs: 97 active+clean; 2.8 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-04-25T10:42:34.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:34 smithi164 ceph-mon[32713]: pgmap v312: 97 pgs: 97 active+clean; 2.8 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-04-25T10:42:34.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:33 smithi043 ceph-mon[29478]: pgmap v312: 97 pgs: 97 active+clean; 2.8 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-04-25T10:42:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.955+0000 7fc8348d1640 1 -- 172.21.15.43:0/2998703420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc830108aa0 msgr2=0x7fc830102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.955+0000 7fc8348d1640 1 --2- 172.21.15.43:0/2998703420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc830108aa0 0x7fc830102640 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fc820009ca0 tx=0x7fc82002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.956+0000 7fc8348d1640 1 -- 172.21.15.43:0/2998703420 shutdown_connections 2024-04-25T10:42:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.956+0000 7fc8348d1640 1 --2- 172.21.15.43:0/2998703420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8301031f0 0x7fc83010f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.956+0000 7fc8348d1640 1 --2- 172.21.15.43:0/2998703420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc830108f40 0x7fc830102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.956+0000 7fc8348d1640 1 --2- 172.21.15.43:0/2998703420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc830108aa0 0x7fc830102640 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.956+0000 7fc8348d1640 1 -- 172.21.15.43:0/2998703420 >> 172.21.15.43:0/2998703420 conn(0x7fc8300781a0 msgr2=0x7fc8300785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.956+0000 7fc8348d1640 1 -- 172.21.15.43:0/2998703420 shutdown_connections 2024-04-25T10:42:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.956+0000 7fc8348d1640 1 -- 172.21.15.43:0/2998703420 wait complete. 2024-04-25T10:42:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.957+0000 7fc8348d1640 1 Processor -- start 2024-04-25T10:42:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.957+0000 7fc8348d1640 1 -- start start 2024-04-25T10:42:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.957+0000 7fc8348d1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8301031f0 0x7fc83019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.957+0000 7fc8348d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc830108aa0 0x7fc83019ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.957+0000 7fc8348d1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc830108f40 0x7fc83019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.957+0000 7fc8348d1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc830112c90 con 0x7fc830108aa0 2024-04-25T10:42:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.957+0000 7fc8348d1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc830112b10 con 0x7fc830108f40 2024-04-25T10:42:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.957+0000 7fc8348d1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc830112e10 con 0x7fc8301031f0 2024-04-25T10:42:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.957+0000 7fc82dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc830108aa0 0x7fc83019ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.957+0000 7fc82dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc830108aa0 0x7fc83019ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49918/0 (socket says 172.21.15.43:49918) 2024-04-25T10:42:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.957+0000 7fc82dd9b640 1 -- 172.21.15.43:0/2749626951 learned_addr learned my addr 172.21.15.43:0/2749626951 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.957+0000 7fc82ed9d640 1 --2- 172.21.15.43:0/2749626951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc830108f40 0x7fc83019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.957+0000 7fc82e59c640 1 --2- 172.21.15.43:0/2749626951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8301031f0 0x7fc83019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.957+0000 7fc82dd9b640 1 -- 172.21.15.43:0/2749626951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8301031f0 msgr2=0x7fc83019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.957+0000 7fc82dd9b640 1 --2- 172.21.15.43:0/2749626951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8301031f0 0x7fc83019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.957+0000 7fc82dd9b640 1 -- 172.21.15.43:0/2749626951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc830108f40 msgr2=0x7fc83019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.957+0000 7fc82dd9b640 1 --2- 172.21.15.43:0/2749626951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc830108f40 0x7fc83019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.957+0000 7fc82dd9b640 1 -- 172.21.15.43:0/2749626951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83019f5e0 con 0x7fc830108aa0 2024-04-25T10:42:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.958+0000 7fc82dd9b640 1 --2- 172.21.15.43:0/2749626951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc830108aa0 0x7fc83019ace0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fc824005fd0 tx=0x7fc82400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.958+0000 7fc81f7fe640 1 -- 172.21.15.43:0/2749626951 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc824017070 con 0x7fc830108aa0 2024-04-25T10:42:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.958+0000 7fc81f7fe640 1 -- 172.21.15.43:0/2749626951 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc824009c30 con 0x7fc830108aa0 2024-04-25T10:42:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.958+0000 7fc8348d1640 1 -- 172.21.15.43:0/2749626951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83019f8d0 con 0x7fc830108aa0 2024-04-25T10:42:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.958+0000 7fc81f7fe640 1 -- 172.21.15.43:0/2749626951 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc824003d10 con 0x7fc830108aa0 2024-04-25T10:42:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.958+0000 7fc8348d1640 1 -- 172.21.15.43:0/2749626951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8301a7250 con 0x7fc830108aa0 2024-04-25T10:42:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.961+0000 7fc83b780740 1 Processor -- start 2024-04-25T10:42:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.961+0000 7fc83b780740 1 -- start start 2024-04-25T10:42:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.962+0000 7fc81f7fe640 1 -- 172.21.15.43:0/2749626951 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fc824003eb0 con 0x7fc830108aa0 2024-04-25T10:42:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.962+0000 7fc83b780740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8fb1b0 0x563c0fc9aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.962+0000 7fc83b780740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c0f8d22f0 0x563c0fc9b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.962+0000 7fc83b780740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8da4d0 0x563c0fc9b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.962+0000 7fc83b780740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563c0fc01460 con 0x563c0f8da4d0 2024-04-25T10:42:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.962+0000 7fc83b780740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563c0fbbb590 con 0x563c0f8d22f0 2024-04-25T10:42:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.962+0000 7fc83b780740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563c0fbfd420 con 0x563c0f8fb1b0 2024-04-25T10:42:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.962+0000 7fc82dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8da4d0 0x563c0fc9b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.962+0000 7fc82dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8da4d0 0x563c0fc9b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49930/0 (socket says 172.21.15.43:49930) 2024-04-25T10:42:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.962+0000 7fc82dd9b640 1 -- 172.21.15.43:0/2715448979 learned_addr learned my addr 172.21.15.43:0/2715448979 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.962+0000 7fc82ed9d640 1 --2- 172.21.15.43:0/2715448979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8fb1b0 0x563c0fc9aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.962+0000 7fc82e59c640 1 --2- 172.21.15.43:0/2715448979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c0f8d22f0 0x563c0fc9b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.962+0000 7fc82dd9b640 1 -- 172.21.15.43:0/2715448979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8fb1b0 msgr2=0x563c0fc9aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.962+0000 7fc82dd9b640 1 --2- 172.21.15.43:0/2715448979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8fb1b0 0x563c0fc9aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.962+0000 7fc81f7fe640 1 --2- 172.21.15.43:0/2749626951 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc8180afc50 0x7fc8180b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.962+0000 7fc82dd9b640 1 -- 172.21.15.43:0/2715448979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c0f8d22f0 msgr2=0x563c0fc9b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.962+0000 7fc82dd9b640 1 --2- 172.21.15.43:0/2715448979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c0f8d22f0 0x563c0fc9b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.963+0000 7fc82dd9b640 1 -- 172.21.15.43:0/2715448979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c0fbc80d0 con 0x563c0f8da4d0 2024-04-25T10:42:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.963+0000 7fc82ed9d640 1 --2- 172.21.15.43:0/2715448979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8fb1b0 0x563c0fc9aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:42:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.963+0000 7fc82ed9d640 1 --2- 172.21.15.43:0/2749626951 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc8180afc50 0x7fc8180b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.963+0000 7fc81f7fe640 1 -- 172.21.15.43:0/2749626951 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fc8240dec80 con 0x7fc830108aa0 2024-04-25T10:42:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.963+0000 7fc82dd9b640 1 --2- 172.21.15.43:0/2715448979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8da4d0 0x563c0fc9b570 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fc8240dc340 tx=0x7fc8240b84d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15df0a8af4de8544 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.963+0000 7fc81cff9640 1 -- 172.21.15.43:0/2715448979 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8240a6bf0 con 0x563c0f8da4d0 2024-04-25T10:42:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.963+0000 7fc81cff9640 1 -- 172.21.15.43:0/2715448979 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8240a6930 con 0x563c0f8da4d0 2024-04-25T10:42:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.963+0000 7fc81cff9640 1 -- 172.21.15.43:0/2715448979 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8240a6590 con 0x563c0f8da4d0 2024-04-25T10:42:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.964+0000 7fc83b780740 1 -- 172.21.15.43:0/2715448979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8da4d0 msgr2=0x563c0fc9b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.964+0000 7fc82ed9d640 1 --2- 172.21.15.43:0/2749626951 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc8180afc50 0x7fc8180b2160 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x563c0fc97c60 tx=0x7fc828005f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.964+0000 7fc83b780740 1 --2- 172.21.15.43:0/2715448979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8da4d0 0x563c0fc9b570 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fc8240dc340 tx=0x7fc8240b84d0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.964+0000 7fc83b780740 1 -- 172.21.15.43:0/2715448979 shutdown_connections 2024-04-25T10:42:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.964+0000 7fc83b780740 1 --2- 172.21.15.43:0/2715448979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c0f8d22f0 0x563c0fc9b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.964+0000 7fc83b780740 1 --2- 172.21.15.43:0/2715448979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8da4d0 0x563c0fc9b570 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.964+0000 7fc83b780740 1 --2- 172.21.15.43:0/2715448979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8fb1b0 0x563c0fc9aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.964+0000 7fc83b780740 1 -- 172.21.15.43:0/2715448979 >> 172.21.15.43:0/2715448979 conn(0x563c0f8ef300 msgr2=0x563c0fc972a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.964+0000 7fc83b780740 1 -- 172.21.15.43:0/2715448979 shutdown_connections 2024-04-25T10:42:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.964+0000 7fc83b780740 1 -- 172.21.15.43:0/2715448979 wait complete. 2024-04-25T10:42:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.965+0000 7fc83b780740 1 -- start start 2024-04-25T10:42:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.965+0000 7fc83b780740 1 Processor -- start 2024-04-25T10:42:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.965+0000 7fc83b780740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c0f8d22f0 0x563c0fca51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.965+0000 7fc83b780740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8da4d0 0x563c0fca56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.965+0000 7fc83b780740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8fb1b0 0x563c0fcab0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.965+0000 7fc83b780740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563c0fca7bc0 con 0x563c0f8da4d0 2024-04-25T10:42:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.965+0000 7fc83b780740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563c0fb88b60 con 0x563c0f8d22f0 2024-04-25T10:42:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.965+0000 7fc83b780740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563c0fca7d40 con 0x563c0f8fb1b0 2024-04-25T10:42:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.965+0000 7fc83b780740 10 client.0 _ll_register_callbacks cb 0x563c0fbad2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.965+0000 7fc82dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8da4d0 0x563c0fca56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.965+0000 7fc82dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8da4d0 0x563c0fca56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49932/0 (socket says 172.21.15.43:49932) 2024-04-25T10:42:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.965+0000 7fc82dd9b640 1 -- 172.21.15.43:0/746012594 learned_addr learned my addr 172.21.15.43:0/746012594 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.965+0000 7fc82e59c640 1 --2- 172.21.15.43:0/746012594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c0f8d22f0 0x563c0fca51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.966+0000 7fc82ed9d640 1 --2- 172.21.15.43:0/746012594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8fb1b0 0x563c0fcab0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.966+0000 7fc82dd9b640 1 -- 172.21.15.43:0/746012594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8fb1b0 msgr2=0x563c0fcab0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.966+0000 7fc82dd9b640 1 --2- 172.21.15.43:0/746012594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8fb1b0 0x563c0fcab0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.966+0000 7fc82dd9b640 1 -- 172.21.15.43:0/746012594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c0f8d22f0 msgr2=0x563c0fca51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.966+0000 7fc82dd9b640 1 --2- 172.21.15.43:0/746012594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c0f8d22f0 0x563c0fca51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.966+0000 7fc82dd9b640 1 -- 172.21.15.43:0/746012594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c0f913210 con 0x563c0f8da4d0 2024-04-25T10:42:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.966+0000 7fc82dd9b640 1 --2- 172.21.15.43:0/746012594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8da4d0 0x563c0fca56f0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fc8240dc9d0 tx=0x7fc824005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb49fe1aa06e642c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.966+0000 7fc815ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.966+0000 7fc815ffb640 1 -- 172.21.15.43:0/746012594 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8240a6770 con 0x563c0f8da4d0 2024-04-25T10:42:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.966+0000 7fc815ffb640 1 -- 172.21.15.43:0/746012594 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fc8240a30b0 con 0x563c0f8da4d0 2024-04-25T10:42:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.966+0000 7fc83b780740 1 -- 172.21.15.43:0/746012594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8240a6930 con 0x563c0f8da4d0 2024-04-25T10:42:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.966+0000 7fc815ffb640 1 -- 172.21.15.43:0/746012594 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc8240a6590 con 0x563c0f8da4d0 2024-04-25T10:42:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.966+0000 7fc83b780740 10 client.18498 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.966+0000 7fc83b780740 1 -- 172.21.15.43:0/746012594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563c0f9df620 con 0x563c0f8da4d0 2024-04-25T10:42:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.966+0000 7fc815ffb640 1 -- 172.21.15.43:0/746012594 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8240a6930 con 0x563c0f8da4d0 2024-04-25T10:42:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.967+0000 7fc815ffb640 1 -- 172.21.15.43:0/746012594 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc8240a7080 con 0x563c0f8da4d0 2024-04-25T10:42:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.967+0000 7fc83b780740 10 client.18498 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.967+0000 7fc83b780740 10 client.18498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.967+0000 7fc83b780740 20 client.18498 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.967+0000 7fc83b780740 1 --2- 172.21.15.43:0/746012594 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563c0f8b9d80 0x563c0fcb7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.967+0000 7fc83b780740 4 client.18498 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:42:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.967+0000 7fc83b780740 1 -- 172.21.15.43:0/746012594 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c0fc0f070 con 0x563c0f8b9d80 2024-04-25T10:42:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.967+0000 7fc82e59c640 1 --2- 172.21.15.43:0/746012594 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563c0f8b9d80 0x563c0fcb7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.967+0000 7fc82e59c640 1 --2- 172.21.15.43:0/746012594 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563c0f8b9d80 0x563c0fcb7550 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8c9ff6dc81bea89 server_cookie=f3313a57fd19b107 in_seq=0 out_seq=0 2024-04-25T10:42:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.967+0000 7fc815ffb640 10 client.18498 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.968+0000 7fc815ffb640 1 -- 172.21.15.43:0/746012594 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563c0fc0f070 con 0x563c0f8b9d80 2024-04-25T10:42:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.968+0000 7fc815ffb640 10 client.18498 handle_command_reply: tid=0 2024-04-25T10:42:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.968+0000 7fc83b780740 1 client.18498 shutdown 2024-04-25T10:42:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.968+0000 7fc83b780740 1 -- 172.21.15.43:0/746012594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8da4d0 msgr2=0x563c0fca56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.968+0000 7fc83b780740 1 --2- 172.21.15.43:0/746012594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8da4d0 0x563c0fca56f0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fc8240dc9d0 tx=0x7fc824005db0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.968+0000 7fc83b780740 1 -- 172.21.15.43:0/746012594 shutdown_connections 2024-04-25T10:42:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.968+0000 7fc83b780740 1 --2- 172.21.15.43:0/746012594 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563c0f8b9d80 0x563c0fcb7550 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.968+0000 7fc83b780740 1 --2- 172.21.15.43:0/746012594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8fb1b0 0x563c0fcab0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.968+0000 7fc83b780740 1 --2- 172.21.15.43:0/746012594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8da4d0 0x563c0fca56f0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.968+0000 7fc815ffb640 0 client.18498 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.968+0000 7fc83b780740 1 --2- 172.21.15.43:0/746012594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c0f8d22f0 0x563c0fca51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.968+0000 7fc83b780740 1 -- 172.21.15.43:0/746012594 >> 172.21.15.43:0/746012594 conn(0x563c0f8ef300 msgr2=0x563c0fc9c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.968+0000 7fc83b780740 1 -- 172.21.15.43:0/746012594 shutdown_connections 2024-04-25T10:42:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.968+0000 7fc83b780740 1 -- 172.21.15.43:0/746012594 wait complete. 2024-04-25T10:42:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.969+0000 7fc83b780740 20 client.18498 trim_cache size 0 max 16384 2024-04-25T10:42:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.984+0000 7fc83b780740 1 Processor -- start 2024-04-25T10:42:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.984+0000 7fc83b780740 1 -- start start 2024-04-25T10:42:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.984+0000 7fc83b780740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c0f8b9d80 0x563c0fcaafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.984+0000 7fc83b780740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8d22f0 0x563c0fc9ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.984+0000 7fc83b780740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8da4d0 0x563c0fc9e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.984+0000 7fc83b780740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc8301b2310 con 0x563c0f8d22f0 2024-04-25T10:42:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.984+0000 7fc83b780740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc8301afbe0 con 0x563c0f8b9d80 2024-04-25T10:42:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.984+0000 7fc83b780740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc8301ada50 con 0x563c0f8da4d0 2024-04-25T10:42:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.984+0000 7fc82dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8d22f0 0x563c0fc9ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.984+0000 7fc82dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8d22f0 0x563c0fc9ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49944/0 (socket says 172.21.15.43:49944) 2024-04-25T10:42:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.984+0000 7fc82dd9b640 1 -- 172.21.15.43:0/3592516604 learned_addr learned my addr 172.21.15.43:0/3592516604 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.984+0000 7fc82dd9b640 1 -- 172.21.15.43:0/3592516604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8da4d0 msgr2=0x563c0fc9e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.984+0000 7fc82dd9b640 1 --2- 172.21.15.43:0/3592516604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8da4d0 0x563c0fc9e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.984+0000 7fc82dd9b640 1 -- 172.21.15.43:0/3592516604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c0f8b9d80 msgr2=0x563c0fcaafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.984+0000 7fc82dd9b640 1 --2- 172.21.15.43:0/3592516604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c0f8b9d80 0x563c0fcaafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.984+0000 7fc82dd9b640 1 -- 172.21.15.43:0/3592516604 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c0fc9d380 con 0x563c0f8d22f0 2024-04-25T10:42:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.985+0000 7fc82dd9b640 1 --2- 172.21.15.43:0/3592516604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8d22f0 0x563c0fc9ce40 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fc8240a2ea0 tx=0x7fc8240def80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67756e1f7f4da5f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.985+0000 7fc817fff640 1 -- 172.21.15.43:0/3592516604 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8240a6bf0 con 0x563c0f8d22f0 2024-04-25T10:42:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.985+0000 7fc817fff640 1 -- 172.21.15.43:0/3592516604 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8240e1030 con 0x563c0f8d22f0 2024-04-25T10:42:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.985+0000 7fc817fff640 1 -- 172.21.15.43:0/3592516604 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8240a7520 con 0x563c0f8d22f0 2024-04-25T10:42:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.985+0000 7fc83b780740 1 -- 172.21.15.43:0/3592516604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8d22f0 msgr2=0x563c0fc9ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.985+0000 7fc83b780740 1 --2- 172.21.15.43:0/3592516604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8d22f0 0x563c0fc9ce40 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fc8240a2ea0 tx=0x7fc8240def80 comp rx=0 tx=0).stop 2024-04-25T10:42:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.985+0000 7fc83b780740 1 -- 172.21.15.43:0/3592516604 shutdown_connections 2024-04-25T10:42:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.985+0000 7fc83b780740 1 --2- 172.21.15.43:0/3592516604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8da4d0 0x563c0fc9e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.985+0000 7fc83b780740 1 --2- 172.21.15.43:0/3592516604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8d22f0 0x563c0fc9ce40 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.985+0000 7fc83b780740 1 --2- 172.21.15.43:0/3592516604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c0f8b9d80 0x563c0fcaafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.985+0000 7fc83b780740 1 -- 172.21.15.43:0/3592516604 >> 172.21.15.43:0/3592516604 conn(0x563c0f8ef300 msgr2=0x563c0fc9c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.986+0000 7fc83b780740 1 -- 172.21.15.43:0/3592516604 shutdown_connections 2024-04-25T10:42:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.986+0000 7fc83b780740 1 -- 172.21.15.43:0/3592516604 wait complete. 2024-04-25T10:42:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.986+0000 7fc83b780740 1 -- start start 2024-04-25T10:42:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.986+0000 7fc83b780740 1 Processor -- start 2024-04-25T10:42:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.986+0000 7fc83b780740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8b9d80 0x563c0fcb1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.986+0000 7fc83b780740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8d22f0 0x563c0fcb18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.986+0000 7fc83b780740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c0f8da4d0 0x563c0fcae870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.986+0000 7fc83b780740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc8301ae0a0 con 0x563c0f8d22f0 2024-04-25T10:42:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.986+0000 7fc83b780740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc8301af510 con 0x563c0f8da4d0 2024-04-25T10:42:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.986+0000 7fc83b780740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc8301ae770 con 0x563c0f8b9d80 2024-04-25T10:42:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.986+0000 7fc83b780740 10 client.0 _ll_register_callbacks cb 0x7fc828005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.986+0000 7fc82dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8d22f0 0x563c0fcb18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.986+0000 7fc82dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8d22f0 0x563c0fcb18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49956/0 (socket says 172.21.15.43:49956) 2024-04-25T10:42:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.986+0000 7fc82dd9b640 1 -- 172.21.15.43:0/1026394287 learned_addr learned my addr 172.21.15.43:0/1026394287 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.987+0000 7fc82e59c640 1 --2- 172.21.15.43:0/1026394287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8b9d80 0x563c0fcb1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.987+0000 7fc82ed9d640 1 --2- 172.21.15.43:0/1026394287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c0f8da4d0 0x563c0fcae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.987+0000 7fc82dd9b640 1 -- 172.21.15.43:0/1026394287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8b9d80 msgr2=0x563c0fcb1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.987+0000 7fc82dd9b640 1 --2- 172.21.15.43:0/1026394287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8b9d80 0x563c0fcb1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.987+0000 7fc82dd9b640 1 -- 172.21.15.43:0/1026394287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c0f8da4d0 msgr2=0x563c0fcae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.987+0000 7fc82dd9b640 1 --2- 172.21.15.43:0/1026394287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c0f8da4d0 0x563c0fcae870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.987+0000 7fc82dd9b640 1 -- 172.21.15.43:0/1026394287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c0fcaeee0 con 0x563c0f8d22f0 2024-04-25T10:42:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.987+0000 7fc82e59c640 1 --2- 172.21.15.43:0/1026394287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8b9d80 0x563c0fcb1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.987+0000 7fc82ed9d640 1 --2- 172.21.15.43:0/1026394287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c0f8da4d0 0x563c0fcae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.987+0000 7fc82dd9b640 1 --2- 172.21.15.43:0/1026394287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8d22f0 0x563c0fcb18d0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fc8240de040 tx=0x7fc8240a6970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccdf84319945b27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.987+0000 7fc815ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.987+0000 7fc815ffb640 1 -- 172.21.15.43:0/1026394287 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8240a7520 con 0x563c0f8d22f0 2024-04-25T10:42:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.987+0000 7fc815ffb640 1 -- 172.21.15.43:0/1026394287 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fc8240a30b0 con 0x563c0f8d22f0 2024-04-25T10:42:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.987+0000 7fc83b780740 1 -- 172.21.15.43:0/1026394287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8240e1030 con 0x563c0f8d22f0 2024-04-25T10:42:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.987+0000 7fc815ffb640 1 -- 172.21.15.43:0/1026394287 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc824087050 con 0x563c0f8d22f0 2024-04-25T10:42:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.987+0000 7fc83b780740 10 client.18504 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.987+0000 7fc83b780740 1 -- 172.21.15.43:0/1026394287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563c0fcaeee0 con 0x563c0f8d22f0 2024-04-25T10:42:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.987+0000 7fc815ffb640 1 -- 172.21.15.43:0/1026394287 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8240e1030 con 0x563c0f8d22f0 2024-04-25T10:42:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.988+0000 7fc815ffb640 1 -- 172.21.15.43:0/1026394287 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc8240b7030 con 0x563c0f8d22f0 2024-04-25T10:42:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.988+0000 7fc83b780740 10 client.18504 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.988+0000 7fc83b780740 10 client.18504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.988+0000 7fc83b780740 20 client.18504 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.988+0000 7fc83b780740 1 --2- 172.21.15.43:0/1026394287 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563c0f8fb1b0 0x563c0fd257d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.988+0000 7fc83b780740 4 client.18504 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:42:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.988+0000 7fc83b780740 1 -- 172.21.15.43:0/1026394287 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563c0fc08ee0 con 0x563c0f8fb1b0 2024-04-25T10:42:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.988+0000 7fc82e59c640 1 --2- 172.21.15.43:0/1026394287 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563c0f8fb1b0 0x563c0fd257d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.988+0000 7fc815ffb640 10 client.18504 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:35.988+0000 7fc82e59c640 1 --2- 172.21.15.43:0/1026394287 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563c0f8fb1b0 0x563c0fd257d0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b734bd74f244523 server_cookie=b889afcb4cc4bdce in_seq=0 out_seq=0 2024-04-25T10:42:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.005+0000 7fc815ffb640 1 -- 172.21.15.43:0/1026394287 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563c0fc08ee0 con 0x563c0f8fb1b0 2024-04-25T10:42:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.005+0000 7fc815ffb640 10 client.18504 handle_command_reply: tid=0 2024-04-25T10:42:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.005+0000 7fc83b780740 1 client.18504 shutdown 2024-04-25T10:42:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.005+0000 7fc83b780740 1 -- 172.21.15.43:0/1026394287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8d22f0 msgr2=0x563c0fcb18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.005+0000 7fc83b780740 1 --2- 172.21.15.43:0/1026394287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8d22f0 0x563c0fcb18d0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fc8240de040 tx=0x7fc8240a6970 comp rx=0 tx=0).stop 2024-04-25T10:42:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.005+0000 7fc83b780740 1 -- 172.21.15.43:0/1026394287 shutdown_connections 2024-04-25T10:42:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.005+0000 7fc83b780740 1 --2- 172.21.15.43:0/1026394287 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563c0f8fb1b0 0x563c0fd257d0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.005+0000 7fc83b780740 1 --2- 172.21.15.43:0/1026394287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c0f8da4d0 0x563c0fcae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.005+0000 7fc83b780740 1 --2- 172.21.15.43:0/1026394287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c0f8d22f0 0x563c0fcb18d0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.005+0000 7fc815ffb640 0 client.18504 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.005+0000 7fc83b780740 1 --2- 172.21.15.43:0/1026394287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c0f8b9d80 0x563c0fcb1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.005+0000 7fc83b780740 1 -- 172.21.15.43:0/1026394287 >> 172.21.15.43:0/1026394287 conn(0x563c0f8ef300 msgr2=0x563c0fca6570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.005+0000 7fc83b780740 1 -- 172.21.15.43:0/1026394287 shutdown_connections 2024-04-25T10:42:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.005+0000 7fc83b780740 1 -- 172.21.15.43:0/1026394287 wait complete. 2024-04-25T10:42:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.005+0000 7fc83b780740 20 client.18504 trim_cache size 0 max 16384 2024-04-25T10:42:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:42:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:42:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:42:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:42:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.006+0000 7fc8348d1640 1 -- 172.21.15.43:0/2749626951 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc8180afc50 msgr2=0x7fc8180b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.006+0000 7fc8348d1640 1 --2- 172.21.15.43:0/2749626951 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc8180afc50 0x7fc8180b2160 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x563c0fc97c60 tx=0x7fc828005f90 comp rx=0 tx=0).stop 2024-04-25T10:42:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.006+0000 7fc8348d1640 1 -- 172.21.15.43:0/2749626951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc830108aa0 msgr2=0x7fc83019ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.006+0000 7fc8348d1640 1 --2- 172.21.15.43:0/2749626951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc830108aa0 0x7fc83019ace0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fc824005fd0 tx=0x7fc82400dde0 comp rx=0 tx=0).stop 2024-04-25T10:42:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.006+0000 7fc8348d1640 1 -- 172.21.15.43:0/2749626951 shutdown_connections 2024-04-25T10:42:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.006+0000 7fc8348d1640 1 --2- 172.21.15.43:0/2749626951 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc8180afc50 0x7fc8180b2160 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.006+0000 7fc8348d1640 1 --2- 172.21.15.43:0/2749626951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc830108f40 0x7fc83019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.006+0000 7fc8348d1640 1 --2- 172.21.15.43:0/2749626951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc830108aa0 0x7fc83019ace0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.006+0000 7fc8348d1640 1 --2- 172.21.15.43:0/2749626951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8301031f0 0x7fc83019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.006+0000 7fc8348d1640 1 -- 172.21.15.43:0/2749626951 >> 172.21.15.43:0/2749626951 conn(0x7fc8300781a0 msgr2=0x7fc830074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.006+0000 7fc8348d1640 1 -- 172.21.15.43:0/2749626951 shutdown_connections 2024-04-25T10:42:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:36.006+0000 7fc8348d1640 1 -- 172.21.15.43:0/2749626951 wait complete. 2024-04-25T10:42:36.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:36 smithi077 ceph-mon[32772]: pgmap v313: 97 pgs: 97 active+clean; 3.0 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-04-25T10:42:36.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:36 smithi164 ceph-mon[32713]: pgmap v313: 97 pgs: 97 active+clean; 3.0 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-04-25T10:42:36.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:36 smithi043 ceph-mon[29478]: pgmap v313: 97 pgs: 97 active+clean; 3.0 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-04-25T10:42:38.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:38 smithi077 ceph-mon[32772]: pgmap v314: 97 pgs: 97 active+clean; 3.0 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-25T10:42:38.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:38 smithi164 ceph-mon[32713]: pgmap v314: 97 pgs: 97 active+clean; 3.0 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-25T10:42:38.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:38 smithi043 ceph-mon[29478]: pgmap v314: 97 pgs: 97 active+clean; 3.0 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-25T10:42:40.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:40 smithi077 ceph-mon[32772]: pgmap v315: 97 pgs: 97 active+clean; 3.2 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 18 op/s 2024-04-25T10:42:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:40 smithi164 ceph-mon[32713]: pgmap v315: 97 pgs: 97 active+clean; 3.2 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 18 op/s 2024-04-25T10:42:40.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:40 smithi043 ceph-mon[29478]: pgmap v315: 97 pgs: 97 active+clean; 3.2 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 18 op/s 2024-04-25T10:42:42.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:42 smithi077 ceph-mon[32772]: pgmap v316: 97 pgs: 97 active+clean; 3.2 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:42:42.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:42 smithi164 ceph-mon[32713]: pgmap v316: 97 pgs: 97 active+clean; 3.2 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:42:42.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:42 smithi043 ceph-mon[29478]: pgmap v316: 97 pgs: 97 active+clean; 3.2 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:42:44.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:44 smithi077 ceph-mon[32772]: pgmap v317: 97 pgs: 97 active+clean; 3.2 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:42:44.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:44 smithi164 ceph-mon[32713]: pgmap v317: 97 pgs: 97 active+clean; 3.2 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:42:44.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:44 smithi043 ceph-mon[29478]: pgmap v317: 97 pgs: 97 active+clean; 3.2 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:42:46.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:46 smithi077 ceph-mon[32772]: pgmap v318: 97 pgs: 97 active+clean; 3.4 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-25T10:42:46.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:46 smithi164 ceph-mon[32713]: pgmap v318: 97 pgs: 97 active+clean; 3.4 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-25T10:42:46.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:46 smithi043 ceph-mon[29478]: pgmap v318: 97 pgs: 97 active+clean; 3.4 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-25T10:42:48.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:48 smithi077 ceph-mon[32772]: pgmap v319: 97 pgs: 97 active+clean; 3.4 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-25T10:42:48.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:48 smithi164 ceph-mon[32713]: pgmap v319: 97 pgs: 97 active+clean; 3.4 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-25T10:42:48.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:48 smithi043 ceph-mon[29478]: pgmap v319: 97 pgs: 97 active+clean; 3.4 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-04-25T10:42:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.118+0000 7fb17499d640 1 -- 172.21.15.43:0/1327006480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1701099b0 msgr2=0x7fb170103540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.118+0000 7fb17499d640 1 --2- 172.21.15.43:0/1327006480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1701099b0 0x7fb170103540 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fb160009900 tx=0x7fb16002a410 comp rx=0 tx=0).stop 2024-04-25T10:42:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.118+0000 7fb17499d640 1 -- 172.21.15.43:0/1327006480 shutdown_connections 2024-04-25T10:42:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.118+0000 7fb17499d640 1 --2- 172.21.15.43:0/1327006480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb170103a80 0x7fb170103e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.118+0000 7fb17499d640 1 --2- 172.21.15.43:0/1327006480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb17010e310 0x7fb170110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.118+0000 7fb17499d640 1 --2- 172.21.15.43:0/1327006480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1701099b0 0x7fb170103540 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.118+0000 7fb17499d640 1 -- 172.21.15.43:0/1327006480 >> 172.21.15.43:0/1327006480 conn(0x7fb170079080 msgr2=0x7fb1700794c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.118+0000 7fb17499d640 1 -- 172.21.15.43:0/1327006480 shutdown_connections 2024-04-25T10:42:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.119+0000 7fb17499d640 1 -- 172.21.15.43:0/1327006480 wait complete. 2024-04-25T10:42:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.119+0000 7fb17499d640 1 Processor -- start 2024-04-25T10:42:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.119+0000 7fb17499d640 1 -- start start 2024-04-25T10:42:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.120+0000 7fb17499d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb170103a80 0x7fb17019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.120+0000 7fb17499d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1701099b0 0x7fb17019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.120+0000 7fb17499d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb17010e310 0x7fb17019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.120+0000 7fb17499d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb170113b70 con 0x7fb1701099b0 2024-04-25T10:42:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.120+0000 7fb17499d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701139f0 con 0x7fb170103a80 2024-04-25T10:42:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.120+0000 7fb17499d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb170113cf0 con 0x7fb17010e310 2024-04-25T10:42:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.120+0000 7fb16effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1701099b0 0x7fb17019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.120+0000 7fb16effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1701099b0 0x7fb17019aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58676/0 (socket says 172.21.15.43:58676) 2024-04-25T10:42:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.120+0000 7fb16effd640 1 -- 172.21.15.43:0/1306702892 learned_addr learned my addr 172.21.15.43:0/1306702892 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.120+0000 7fb16f7fe640 1 --2- 172.21.15.43:0/1306702892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb170103a80 0x7fb17019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.120+0000 7fb16ffff640 1 --2- 172.21.15.43:0/1306702892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb17010e310 0x7fb17019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.120+0000 7fb16effd640 1 -- 172.21.15.43:0/1306702892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb17010e310 msgr2=0x7fb17019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.120+0000 7fb16effd640 1 --2- 172.21.15.43:0/1306702892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb17010e310 0x7fb17019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.120+0000 7fb16effd640 1 -- 172.21.15.43:0/1306702892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb170103a80 msgr2=0x7fb17019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.120+0000 7fb16effd640 1 --2- 172.21.15.43:0/1306702892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb170103a80 0x7fb17019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.120+0000 7fb16effd640 1 -- 172.21.15.43:0/1306702892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17019f3f0 con 0x7fb1701099b0 2024-04-25T10:42:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.121+0000 7fb16effd640 1 --2- 172.21.15.43:0/1306702892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1701099b0 0x7fb17019aaf0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fb16400e9f0 tx=0x7fb16400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.121+0000 7fb16cff9640 1 -- 172.21.15.43:0/1306702892 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16400ccc0 con 0x7fb1701099b0 2024-04-25T10:42:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.121+0000 7fb17499d640 1 -- 172.21.15.43:0/1306702892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb17019f6e0 con 0x7fb1701099b0 2024-04-25T10:42:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.121+0000 7fb16cff9640 1 -- 172.21.15.43:0/1306702892 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1640035f0 con 0x7fb1701099b0 2024-04-25T10:42:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.121+0000 7fb16cff9640 1 -- 172.21.15.43:0/1306702892 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16400ce60 con 0x7fb1701099b0 2024-04-25T10:42:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.121+0000 7fb17499d640 1 -- 172.21.15.43:0/1306702892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1701a7060 con 0x7fb1701099b0 2024-04-25T10:42:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.124+0000 7fb17cb8c740 1 Processor -- start 2024-04-25T10:42:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.124+0000 7fb16cff9640 1 -- 172.21.15.43:0/1306702892 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fb164003aa0 con 0x7fb1701099b0 2024-04-25T10:42:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.124+0000 7fb17cb8c740 1 -- start start 2024-04-25T10:42:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.124+0000 7fb17cb8c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3029f1b0 0x561b3063ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.124+0000 7fb17cb8c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b302762f0 0x561b3063f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.125+0000 7fb17cb8c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b3027e4d0 0x561b3063f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.125+0000 7fb17cb8c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561b305a5460 con 0x561b3027e4d0 2024-04-25T10:42:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.125+0000 7fb17cb8c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561b3055f590 con 0x561b302762f0 2024-04-25T10:42:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.125+0000 7fb17cb8c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561b305a1420 con 0x561b3029f1b0 2024-04-25T10:42:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.125+0000 7fb16effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b3027e4d0 0x561b3063f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.125+0000 7fb16f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b302762f0 0x561b3063f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.125+0000 7fb16effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b3027e4d0 0x561b3063f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58684/0 (socket says 172.21.15.43:58684) 2024-04-25T10:42:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.125+0000 7fb16effd640 1 -- 172.21.15.43:0/1166732759 learned_addr learned my addr 172.21.15.43:0/1166732759 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.125+0000 7fb16ffff640 1 --2- 172.21.15.43:0/1166732759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3029f1b0 0x561b3063ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.125+0000 7fb16f7fe640 1 -- 172.21.15.43:0/1166732759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3029f1b0 msgr2=0x561b3063ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.125+0000 7fb16f7fe640 1 --2- 172.21.15.43:0/1166732759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3029f1b0 0x561b3063ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.125+0000 7fb16f7fe640 1 -- 172.21.15.43:0/1166732759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b3027e4d0 msgr2=0x561b3063f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.125+0000 7fb16f7fe640 1 --2- 172.21.15.43:0/1166732759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b3027e4d0 0x561b3063f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.125+0000 7fb16f7fe640 1 -- 172.21.15.43:0/1166732759 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b3056c0d0 con 0x561b302762f0 2024-04-25T10:42:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.125+0000 7fb16cff9640 1 --2- 172.21.15.43:0/1306702892 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb1580afc50 0x7fb1580b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.125+0000 7fb16effd640 1 --2- 172.21.15.43:0/1166732759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b3027e4d0 0x561b3063f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:42:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.125+0000 7fb16ffff640 1 --2- 172.21.15.43:0/1306702892 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb1580afc50 0x7fb1580b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.125+0000 7fb16f7fe640 1 --2- 172.21.15.43:0/1166732759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b302762f0 0x561b3063f1d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb160006010 tx=0x7fb160005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19afb5b556c6c2f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.125+0000 7fb16cff9640 1 -- 172.21.15.43:0/1306702892 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fb1640df400 con 0x7fb1701099b0 2024-04-25T10:42:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.126+0000 7fb155ffb640 1 -- 172.21.15.43:0/1166732759 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160040070 con 0x561b302762f0 2024-04-25T10:42:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.126+0000 7fb155ffb640 1 -- 172.21.15.43:0/1166732759 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb160032e20 con 0x561b302762f0 2024-04-25T10:42:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.126+0000 7fb155ffb640 1 -- 172.21.15.43:0/1166732759 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160035040 con 0x561b302762f0 2024-04-25T10:42:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.126+0000 7fb16ffff640 1 --2- 172.21.15.43:0/1306702892 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb1580afc50 0x7fb1580b2160 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x561b3063be00 tx=0x7fb168009740 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.126+0000 7fb17cb8c740 1 -- 172.21.15.43:0/1166732759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b302762f0 msgr2=0x561b3063f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.127+0000 7fb17cb8c740 1 --2- 172.21.15.43:0/1166732759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b302762f0 0x561b3063f1d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb160006010 tx=0x7fb160005e60 comp rx=0 tx=0).stop 2024-04-25T10:42:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.127+0000 7fb17cb8c740 1 -- 172.21.15.43:0/1166732759 shutdown_connections 2024-04-25T10:42:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.127+0000 7fb17cb8c740 1 --2- 172.21.15.43:0/1166732759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b302762f0 0x561b3063f1d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.127+0000 7fb17cb8c740 1 --2- 172.21.15.43:0/1166732759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b3027e4d0 0x561b3063f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.127+0000 7fb17cb8c740 1 --2- 172.21.15.43:0/1166732759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3029f1b0 0x561b3063ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.127+0000 7fb17cb8c740 1 -- 172.21.15.43:0/1166732759 >> 172.21.15.43:0/1166732759 conn(0x561b30293300 msgr2=0x561b3063b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.127+0000 7fb17cb8c740 1 -- 172.21.15.43:0/1166732759 shutdown_connections 2024-04-25T10:42:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.127+0000 7fb17cb8c740 1 -- 172.21.15.43:0/1166732759 wait complete. 2024-04-25T10:42:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.127+0000 7fb17cb8c740 1 -- start start 2024-04-25T10:42:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.127+0000 7fb17cb8c740 1 Processor -- start 2024-04-25T10:42:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb17cb8c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b302762f0 0x561b3064acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb17cb8c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b3027e4d0 0x561b3064b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb17cb8c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3029f1b0 0x561b306491e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb17cb8c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561b3064bd60 con 0x561b302762f0 2024-04-25T10:42:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb17cb8c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561b3052cb60 con 0x561b3027e4d0 2024-04-25T10:42:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb17cb8c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561b3064bee0 con 0x561b3029f1b0 2024-04-25T10:42:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb17cb8c740 10 client.0 _ll_register_callbacks cb 0x561b305512b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb16f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b302762f0 0x561b3064acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb16f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b302762f0 0x561b3064acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58696/0 (socket says 172.21.15.43:58696) 2024-04-25T10:42:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb16f7fe640 1 -- 172.21.15.43:0/3749642001 learned_addr learned my addr 172.21.15.43:0/3749642001 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb16effd640 1 --2- 172.21.15.43:0/3749642001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b3027e4d0 0x561b3064b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb16ffff640 1 --2- 172.21.15.43:0/3749642001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3029f1b0 0x561b306491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb16f7fe640 1 -- 172.21.15.43:0/3749642001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3029f1b0 msgr2=0x561b306491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb16f7fe640 1 --2- 172.21.15.43:0/3749642001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3029f1b0 0x561b306491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb16f7fe640 1 -- 172.21.15.43:0/3749642001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b3027e4d0 msgr2=0x561b3064b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb16f7fe640 1 --2- 172.21.15.43:0/3749642001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b3027e4d0 0x561b3064b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb16f7fe640 1 -- 172.21.15.43:0/3749642001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b302b7210 con 0x561b302762f0 2024-04-25T10:42:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb16effd640 1 --2- 172.21.15.43:0/3749642001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b3027e4d0 0x561b3064b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb16ffff640 1 --2- 172.21.15.43:0/3749642001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3029f1b0 0x561b306491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb16f7fe640 1 --2- 172.21.15.43:0/3749642001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b302762f0 0x561b3064acd0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fb160009230 tx=0x7fb160031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e6c210e20436ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.128+0000 7fb14effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.129+0000 7fb14effd640 1 -- 172.21.15.43:0/3749642001 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16004b070 con 0x561b302762f0 2024-04-25T10:42:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.129+0000 7fb14effd640 1 -- 172.21.15.43:0/3749642001 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fb160004250 con 0x561b302762f0 2024-04-25T10:42:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.129+0000 7fb17cb8c740 1 -- 172.21.15.43:0/3749642001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb160032e20 con 0x561b302762f0 2024-04-25T10:42:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.129+0000 7fb14effd640 1 -- 172.21.15.43:0/3749642001 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb160057050 con 0x561b302762f0 2024-04-25T10:42:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.129+0000 7fb17cb8c740 10 client.18516 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.129+0000 7fb17cb8c740 1 -- 172.21.15.43:0/3749642001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561b30383620 con 0x561b302762f0 2024-04-25T10:42:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.129+0000 7fb14effd640 1 -- 172.21.15.43:0/3749642001 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb160032e20 con 0x561b302762f0 2024-04-25T10:42:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.129+0000 7fb14effd640 1 -- 172.21.15.43:0/3749642001 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb160004b60 con 0x561b302762f0 2024-04-25T10:42:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.129+0000 7fb17cb8c740 10 client.18516 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.129+0000 7fb17cb8c740 10 client.18516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.129+0000 7fb17cb8c740 20 client.18516 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.129+0000 7fb17cb8c740 1 --2- 172.21.15.43:0/3749642001 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561b3025dd80 0x561b3065b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.129+0000 7fb17cb8c740 4 client.18516 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:42:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.129+0000 7fb17cb8c740 1 -- 172.21.15.43:0/3749642001 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b305b3070 con 0x561b3025dd80 2024-04-25T10:42:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.130+0000 7fb16effd640 1 --2- 172.21.15.43:0/3749642001 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561b3025dd80 0x561b3065b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.130+0000 7fb16effd640 1 --2- 172.21.15.43:0/3749642001 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561b3025dd80 0x561b3065b7a0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c53f2f2a59697af4 server_cookie=f79a02c3e1f2e8c5 in_seq=0 out_seq=0 2024-04-25T10:42:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.130+0000 7fb14effd640 10 client.18516 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.131+0000 7fb14effd640 1 -- 172.21.15.43:0/3749642001 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561b305b3070 con 0x561b3025dd80 2024-04-25T10:42:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.131+0000 7fb14effd640 10 client.18516 handle_command_reply: tid=0 2024-04-25T10:42:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.131+0000 7fb17cb8c740 1 client.18516 shutdown 2024-04-25T10:42:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.131+0000 7fb17cb8c740 1 -- 172.21.15.43:0/3749642001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b302762f0 msgr2=0x561b3064acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.131+0000 7fb17cb8c740 1 --2- 172.21.15.43:0/3749642001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b302762f0 0x561b3064acd0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fb160009230 tx=0x7fb160031750 comp rx=0 tx=0).stop 2024-04-25T10:42:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.131+0000 7fb17cb8c740 1 -- 172.21.15.43:0/3749642001 shutdown_connections 2024-04-25T10:42:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.131+0000 7fb17cb8c740 1 --2- 172.21.15.43:0/3749642001 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561b3025dd80 0x561b3065b7a0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.131+0000 7fb17cb8c740 1 --2- 172.21.15.43:0/3749642001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b3027e4d0 0x561b3064b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.131+0000 7fb17cb8c740 1 --2- 172.21.15.43:0/3749642001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3029f1b0 0x561b306491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.131+0000 7fb14effd640 0 client.18516 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.131+0000 7fb17cb8c740 1 --2- 172.21.15.43:0/3749642001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b302762f0 0x561b3064acd0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.131+0000 7fb17cb8c740 1 -- 172.21.15.43:0/3749642001 >> 172.21.15.43:0/3749642001 conn(0x561b30293300 msgr2=0x561b3063eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.131+0000 7fb17cb8c740 1 -- 172.21.15.43:0/3749642001 shutdown_connections 2024-04-25T10:42:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.131+0000 7fb17cb8c740 1 -- 172.21.15.43:0/3749642001 wait complete. 2024-04-25T10:42:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.131+0000 7fb17cb8c740 20 client.18516 trim_cache size 0 max 16384 2024-04-25T10:42:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.147+0000 7fb17cb8c740 1 Processor -- start 2024-04-25T10:42:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.147+0000 7fb17cb8c740 1 -- start start 2024-04-25T10:42:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.147+0000 7fb17cb8c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3025dd80 0x561b3063fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.147+0000 7fb17cb8c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b302762f0 0x561b3063b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.147+0000 7fb17cb8c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b3027e4d0 0x561b30646420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.147+0000 7fb17cb8c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701b1f50 con 0x561b302762f0 2024-04-25T10:42:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.147+0000 7fb17cb8c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701af820 con 0x561b3027e4d0 2024-04-25T10:42:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.147+0000 7fb17cb8c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701ad690 con 0x561b3025dd80 2024-04-25T10:42:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.147+0000 7fb16effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b302762f0 0x561b3063b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.147+0000 7fb16effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b302762f0 0x561b3063b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58702/0 (socket says 172.21.15.43:58702) 2024-04-25T10:42:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.147+0000 7fb16effd640 1 -- 172.21.15.43:0/556655288 learned_addr learned my addr 172.21.15.43:0/556655288 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.147+0000 7fb16ffff640 1 --2- 172.21.15.43:0/556655288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b3027e4d0 0x561b30646420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.147+0000 7fb16f7fe640 1 --2- 172.21.15.43:0/556655288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3025dd80 0x561b3063fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.147+0000 7fb16effd640 1 -- 172.21.15.43:0/556655288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3025dd80 msgr2=0x561b3063fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.147+0000 7fb16effd640 1 --2- 172.21.15.43:0/556655288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3025dd80 0x561b3063fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.147+0000 7fb16effd640 1 -- 172.21.15.43:0/556655288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b3027e4d0 msgr2=0x561b30646420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.147+0000 7fb16effd640 1 --2- 172.21.15.43:0/556655288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b3027e4d0 0x561b30646420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.147+0000 7fb16effd640 1 -- 172.21.15.43:0/556655288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b3063b970 con 0x561b302762f0 2024-04-25T10:42:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.147+0000 7fb16f7fe640 1 --2- 172.21.15.43:0/556655288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3025dd80 0x561b3063fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.148+0000 7fb16effd640 1 --2- 172.21.15.43:0/556655288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b302762f0 0x561b3063b430 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fb1640dcc00 tx=0x7fb1640e2f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324bddc55c6ae27b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.148+0000 7fb1557fa640 1 -- 172.21.15.43:0/556655288 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1640a7370 con 0x561b302762f0 2024-04-25T10:42:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.148+0000 7fb1557fa640 1 -- 172.21.15.43:0/556655288 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1640a70b0 con 0x561b302762f0 2024-04-25T10:42:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.148+0000 7fb1557fa640 1 -- 172.21.15.43:0/556655288 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1640a6d10 con 0x561b302762f0 2024-04-25T10:42:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.149+0000 7fb17cb8c740 1 -- 172.21.15.43:0/556655288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b302762f0 msgr2=0x561b3063b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.149+0000 7fb17cb8c740 1 --2- 172.21.15.43:0/556655288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b302762f0 0x561b3063b430 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fb1640dcc00 tx=0x7fb1640e2f50 comp rx=0 tx=0).stop 2024-04-25T10:42:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.149+0000 7fb17cb8c740 1 -- 172.21.15.43:0/556655288 shutdown_connections 2024-04-25T10:42:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.149+0000 7fb17cb8c740 1 --2- 172.21.15.43:0/556655288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b3027e4d0 0x561b30646420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.149+0000 7fb17cb8c740 1 --2- 172.21.15.43:0/556655288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b302762f0 0x561b3063b430 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.149+0000 7fb17cb8c740 1 --2- 172.21.15.43:0/556655288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3025dd80 0x561b3063fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.149+0000 7fb17cb8c740 1 -- 172.21.15.43:0/556655288 >> 172.21.15.43:0/556655288 conn(0x561b30293300 msgr2=0x561b3063ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.149+0000 7fb17cb8c740 1 -- 172.21.15.43:0/556655288 shutdown_connections 2024-04-25T10:42:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.149+0000 7fb17cb8c740 1 -- 172.21.15.43:0/556655288 wait complete. 2024-04-25T10:42:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.149+0000 7fb17cb8c740 1 -- start start 2024-04-25T10:42:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.149+0000 7fb17cb8c740 1 Processor -- start 2024-04-25T10:42:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb17cb8c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b3025dd80 0x561b306bc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb17cb8c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b302762f0 0x561b306bc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb17cb8c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3027e4d0 0x561b3065e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb17cb8c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701adce0 con 0x561b3025dd80 2024-04-25T10:42:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb17cb8c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701af150 con 0x561b302762f0 2024-04-25T10:42:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb17cb8c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701ae3b0 con 0x561b3027e4d0 2024-04-25T10:42:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb17cb8c740 10 client.0 _ll_register_callbacks cb 0x7fb1640e2e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb16f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b3025dd80 0x561b306bc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb16f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b3025dd80 0x561b306bc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58710/0 (socket says 172.21.15.43:58710) 2024-04-25T10:42:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb16f7fe640 1 -- 172.21.15.43:0/3545416180 learned_addr learned my addr 172.21.15.43:0/3545416180 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb16effd640 1 --2- 172.21.15.43:0/3545416180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b302762f0 0x561b306bc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb16ffff640 1 --2- 172.21.15.43:0/3545416180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3027e4d0 0x561b3065e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb16f7fe640 1 -- 172.21.15.43:0/3545416180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3027e4d0 msgr2=0x561b3065e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb16f7fe640 1 --2- 172.21.15.43:0/3545416180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3027e4d0 0x561b3065e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb16f7fe640 1 -- 172.21.15.43:0/3545416180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b302762f0 msgr2=0x561b306bc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb16f7fe640 1 --2- 172.21.15.43:0/3545416180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b302762f0 0x561b306bc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb16f7fe640 1 -- 172.21.15.43:0/3545416180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b3065e780 con 0x561b3025dd80 2024-04-25T10:42:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb16f7fe640 1 --2- 172.21.15.43:0/3545416180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b3025dd80 0x561b306bc1c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fb160031440 tx=0x7fb16002ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736d856fe8f2a105 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb14effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb14effd640 1 -- 172.21.15.43:0/3545416180 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160060060 con 0x561b3025dd80 2024-04-25T10:42:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb14effd640 1 -- 172.21.15.43:0/3545416180 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fb160031b10 con 0x561b3025dd80 2024-04-25T10:42:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.150+0000 7fb17cb8c740 1 -- 172.21.15.43:0/3545416180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1640a70b0 con 0x561b3025dd80 2024-04-25T10:42:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.151+0000 7fb14effd640 1 -- 172.21.15.43:0/3545416180 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb16004b050 con 0x561b3025dd80 2024-04-25T10:42:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.151+0000 7fb17cb8c740 10 client.18522 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.151+0000 7fb17cb8c740 1 -- 172.21.15.43:0/3545416180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561b306c37b0 con 0x561b3025dd80 2024-04-25T10:42:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.151+0000 7fb14effd640 1 -- 172.21.15.43:0/3545416180 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1640a70b0 con 0x561b3025dd80 2024-04-25T10:42:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.151+0000 7fb14effd640 1 -- 172.21.15.43:0/3545416180 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb1600053e0 con 0x561b3025dd80 2024-04-25T10:42:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.151+0000 7fb17cb8c740 10 client.18522 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.151+0000 7fb17cb8c740 10 client.18522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.151+0000 7fb17cb8c740 20 client.18522 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.151+0000 7fb17cb8c740 1 --2- 172.21.15.43:0/3545416180 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561b3029f1b0 0x561b306c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.151+0000 7fb17cb8c740 4 client.18522 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:42:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.151+0000 7fb17cb8c740 1 -- 172.21.15.43:0/3545416180 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561b30643ef0 con 0x561b3029f1b0 2024-04-25T10:42:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.151+0000 7fb16effd640 1 --2- 172.21.15.43:0/3545416180 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561b3029f1b0 0x561b306c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.152+0000 7fb14effd640 10 client.18522 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.152+0000 7fb16effd640 1 --2- 172.21.15.43:0/3545416180 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561b3029f1b0 0x561b306c5e90 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59ca961a9f609782 server_cookie=248d0d6fbea69985 in_seq=0 out_seq=0 2024-04-25T10:42:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.167+0000 7fb14effd640 1 -- 172.21.15.43:0/3545416180 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561b30643ef0 con 0x561b3029f1b0 2024-04-25T10:42:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.167+0000 7fb14effd640 10 client.18522 handle_command_reply: tid=0 2024-04-25T10:42:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.167+0000 7fb17cb8c740 1 client.18522 shutdown 2024-04-25T10:42:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.167+0000 7fb17cb8c740 1 -- 172.21.15.43:0/3545416180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b3025dd80 msgr2=0x561b306bc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.167+0000 7fb17cb8c740 1 --2- 172.21.15.43:0/3545416180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b3025dd80 0x561b306bc1c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fb160031440 tx=0x7fb16002ab10 comp rx=0 tx=0).stop 2024-04-25T10:42:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.167+0000 7fb17cb8c740 1 -- 172.21.15.43:0/3545416180 shutdown_connections 2024-04-25T10:42:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.167+0000 7fb17cb8c740 1 --2- 172.21.15.43:0/3545416180 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561b3029f1b0 0x561b306c5e90 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.168+0000 7fb17cb8c740 1 --2- 172.21.15.43:0/3545416180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b302762f0 0x561b306bc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.168+0000 7fb17cb8c740 1 --2- 172.21.15.43:0/3545416180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3027e4d0 0x561b3065e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.168+0000 7fb14effd640 0 client.18522 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.168+0000 7fb17cb8c740 1 --2- 172.21.15.43:0/3545416180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b3025dd80 0x561b306bc1c0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.168+0000 7fb17cb8c740 1 -- 172.21.15.43:0/3545416180 >> 172.21.15.43:0/3545416180 conn(0x561b30293300 msgr2=0x561b30639ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.168+0000 7fb17cb8c740 1 -- 172.21.15.43:0/3545416180 shutdown_connections 2024-04-25T10:42:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.168+0000 7fb17cb8c740 1 -- 172.21.15.43:0/3545416180 wait complete. 2024-04-25T10:42:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.168+0000 7fb17cb8c740 20 client.18522 trim_cache size 0 max 16384 2024-04-25T10:42:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:42:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:42:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:42:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:42:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.168+0000 7fb17499d640 1 -- 172.21.15.43:0/1306702892 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb1580afc50 msgr2=0x7fb1580b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.168+0000 7fb17499d640 1 --2- 172.21.15.43:0/1306702892 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb1580afc50 0x7fb1580b2160 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x561b3063be00 tx=0x7fb168009740 comp rx=0 tx=0).stop 2024-04-25T10:42:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.168+0000 7fb17499d640 1 -- 172.21.15.43:0/1306702892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1701099b0 msgr2=0x7fb17019aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.168+0000 7fb17499d640 1 --2- 172.21.15.43:0/1306702892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1701099b0 0x7fb17019aaf0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fb16400e9f0 tx=0x7fb16400eeb0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.168+0000 7fb17499d640 1 -- 172.21.15.43:0/1306702892 shutdown_connections 2024-04-25T10:42:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.168+0000 7fb17499d640 1 --2- 172.21.15.43:0/1306702892 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb1580afc50 0x7fb1580b2160 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.168+0000 7fb17499d640 1 --2- 172.21.15.43:0/1306702892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb17010e310 0x7fb17019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.168+0000 7fb17499d640 1 --2- 172.21.15.43:0/1306702892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1701099b0 0x7fb17019aaf0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.168+0000 7fb17499d640 1 --2- 172.21.15.43:0/1306702892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb170103a80 0x7fb17019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.168+0000 7fb17499d640 1 -- 172.21.15.43:0/1306702892 >> 172.21.15.43:0/1306702892 conn(0x7fb170079080 msgr2=0x7fb170105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.169+0000 7fb17499d640 1 -- 172.21.15.43:0/1306702892 shutdown_connections 2024-04-25T10:42:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:42:49.169+0000 7fb17499d640 1 -- 172.21.15.43:0/1306702892 wait complete. 2024-04-25T10:42:50.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:50 smithi077 ceph-mon[32772]: pgmap v320: 97 pgs: 97 active+clean; 3.6 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-25T10:42:50.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:50 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:42:50.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:50 smithi164 ceph-mon[32713]: pgmap v320: 97 pgs: 97 active+clean; 3.6 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-25T10:42:50.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:50 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:42:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:50 smithi043 ceph-mon[29478]: pgmap v320: 97 pgs: 97 active+clean; 3.6 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-25T10:42:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:50 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:42:52.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:52 smithi077 ceph-mon[32772]: pgmap v321: 97 pgs: 97 active+clean; 3.6 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-04-25T10:42:52.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:52 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:52.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:52 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:52.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:52 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:52.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:52 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:52.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:52 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:52.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:52 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:52.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:52 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:52 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:52 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:52 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:52 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:52 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:52 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:52 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:52 smithi164 ceph-mon[32713]: pgmap v321: 97 pgs: 97 active+clean; 3.6 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-04-25T10:42:52.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:52 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:52.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:52 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:52.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:52 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:52.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:52 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:52.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:52 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:52.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:52 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:52.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:52 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:52 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:52 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:52 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:52 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:52 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:52 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:52 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:52 smithi043 ceph-mon[29478]: pgmap v321: 97 pgs: 97 active+clean; 3.6 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-04-25T10:42:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:52 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:52 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:52 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:52 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:52 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:52 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:52 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:52 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:52 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:52 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:52 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:52 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:52 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:52.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:52 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: Adjusting osd_memory_target on smithi164 to 176.2M 2024-04-25T10:42:53.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: Unable to set osd_memory_target on smithi164 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T10:42:53.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:42:53.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:42:53.292 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:53 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:53.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: Adjusting osd_memory_target on smithi164 to 176.2M 2024-04-25T10:42:53.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: Unable to set osd_memory_target on smithi164 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T10:42:53.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.293 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:42:53.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:42:53.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:53 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:53.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: Adjusting osd_memory_target on smithi164 to 176.2M 2024-04-25T10:42:53.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: Unable to set osd_memory_target on smithi164 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T10:42:53.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T10:42:53.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:42:53.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:42:53.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:53 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:42:54.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:54 smithi077 ceph-mon[32772]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-04-25T10:42:54.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:54 smithi077 ceph-mon[32772]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T10:42:54.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:54 smithi077 ceph-mon[32772]: Adjusting osd_memory_target on smithi043 to 177.2M 2024-04-25T10:42:54.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:54 smithi077 ceph-mon[32772]: Unable to set osd_memory_target on smithi043 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T10:42:54.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:54 smithi077 ceph-mon[32772]: pgmap v322: 97 pgs: 97 active+clean; 3.6 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-04-25T10:42:54.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:54 smithi164 ceph-mon[32713]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-04-25T10:42:54.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:54 smithi164 ceph-mon[32713]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T10:42:54.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:54 smithi164 ceph-mon[32713]: Adjusting osd_memory_target on smithi043 to 177.2M 2024-04-25T10:42:54.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:54 smithi164 ceph-mon[32713]: Unable to set osd_memory_target on smithi043 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T10:42:54.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:54 smithi164 ceph-mon[32713]: pgmap v322: 97 pgs: 97 active+clean; 3.6 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-04-25T10:42:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:54 smithi043 ceph-mon[29478]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-04-25T10:42:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:54 smithi043 ceph-mon[29478]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T10:42:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:54 smithi043 ceph-mon[29478]: Adjusting osd_memory_target on smithi043 to 177.2M 2024-04-25T10:42:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:54 smithi043 ceph-mon[29478]: Unable to set osd_memory_target on smithi043 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T10:42:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:54 smithi043 ceph-mon[29478]: pgmap v322: 97 pgs: 97 active+clean; 3.6 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-04-25T10:42:55.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:42:55.212 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:42:55.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.342+0000 7faa78300640 1 -- 172.21.15.43:0/3908884624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa70108eb0 msgr2=0x7faa70102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.342+0000 7faa78300640 1 --2- 172.21.15.43:0/3908884624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa70108eb0 0x7faa70102b50 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7faa60009900 tx=0x7faa6002a410 comp rx=0 tx=0).stop 2024-04-25T10:42:55.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.342+0000 7faa78300640 1 -- 172.21.15.43:0/3908884624 shutdown_connections 2024-04-25T10:42:55.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.342+0000 7faa78300640 1 --2- 172.21.15.43:0/3908884624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa701031c0 0x7faa7010f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.342+0000 7faa78300640 1 --2- 172.21.15.43:0/3908884624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa70108eb0 0x7faa70102b50 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.342+0000 7faa78300640 1 --2- 172.21.15.43:0/3908884624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa70108a10 0x7faa70102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.342+0000 7faa78300640 1 -- 172.21.15.43:0/3908884624 >> 172.21.15.43:0/3908884624 conn(0x7faa70078110 msgr2=0x7faa70078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:55.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.342+0000 7faa78300640 1 -- 172.21.15.43:0/3908884624 shutdown_connections 2024-04-25T10:42:55.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.342+0000 7faa78300640 1 -- 172.21.15.43:0/3908884624 wait complete. 2024-04-25T10:42:55.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.343+0000 7faa78300640 1 Processor -- start 2024-04-25T10:42:55.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.343+0000 7faa78300640 1 -- start start 2024-04-25T10:42:55.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.343+0000 7faa78300640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa701031c0 0x7faa7019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.343+0000 7faa78300640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa70108a10 0x7faa7019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.343+0000 7faa78300640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa70108eb0 0x7faa7019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.343+0000 7faa78300640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa70112c30 con 0x7faa701031c0 2024-04-25T10:42:55.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.343+0000 7faa78300640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faa70112ab0 con 0x7faa70108eb0 2024-04-25T10:42:55.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.343+0000 7faa78300640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faa70112db0 con 0x7faa70108a10 2024-04-25T10:42:55.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.343+0000 7faa7609c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa701031c0 0x7faa7019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.344+0000 7faa7609c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa701031c0 0x7faa7019a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58726/0 (socket says 172.21.15.43:58726) 2024-04-25T10:42:55.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.344+0000 7faa7609c640 1 -- 172.21.15.43:0/3593743979 learned_addr learned my addr 172.21.15.43:0/3593743979 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:55.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.344+0000 7faa7589b640 1 --2- 172.21.15.43:0/3593743979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa70108a10 0x7faa7019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.344+0000 7faa7689d640 1 --2- 172.21.15.43:0/3593743979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa70108eb0 0x7faa7019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.344+0000 7faa7609c640 1 -- 172.21.15.43:0/3593743979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa70108a10 msgr2=0x7faa7019acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.344+0000 7faa7609c640 1 --2- 172.21.15.43:0/3593743979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa70108a10 0x7faa7019acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.344+0000 7faa7609c640 1 -- 172.21.15.43:0/3593743979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa70108eb0 msgr2=0x7faa7019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.344+0000 7faa7609c640 1 --2- 172.21.15.43:0/3593743979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa70108eb0 0x7faa7019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.344+0000 7faa7609c640 1 -- 172.21.15.43:0/3593743979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa7019f5c0 con 0x7faa701031c0 2024-04-25T10:42:55.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.344+0000 7faa7689d640 1 --2- 172.21.15.43:0/3593743979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa70108eb0 0x7faa7019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:55.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.344+0000 7faa7609c640 1 --2- 172.21.15.43:0/3593743979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa701031c0 0x7faa7019a760 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00e9c0 tx=0x7faa6c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:55.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.344+0000 7faa677fe640 1 -- 172.21.15.43:0/3593743979 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c00ccc0 con 0x7faa701031c0 2024-04-25T10:42:55.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.344+0000 7faa677fe640 1 -- 172.21.15.43:0/3593743979 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa6c0054d0 con 0x7faa701031c0 2024-04-25T10:42:55.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.344+0000 7faa78300640 1 -- 172.21.15.43:0/3593743979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa7019f8b0 con 0x7faa701031c0 2024-04-25T10:42:55.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.344+0000 7faa677fe640 1 -- 172.21.15.43:0/3593743979 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c010430 con 0x7faa701031c0 2024-04-25T10:42:55.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.344+0000 7faa78300640 1 -- 172.21.15.43:0/3593743979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa701a7230 con 0x7faa701031c0 2024-04-25T10:42:55.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.347+0000 7faa7f1b3740 1 Processor -- start 2024-04-25T10:42:55.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.347+0000 7faa677fe640 1 -- 172.21.15.43:0/3593743979 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7faa6c005020 con 0x7faa701031c0 2024-04-25T10:42:55.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.347+0000 7faa7f1b3740 1 -- start start 2024-04-25T10:42:55.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.348+0000 7faa7f1b3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e69390 0x55ce76209a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.348+0000 7faa7f1b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e404d0 0x55ce76209fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.348+0000 7faa7f1b3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce75e486b0 0x55ce762168f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.348+0000 7faa7f1b3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ce7616fd80 con 0x55ce75e69390 2024-04-25T10:42:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.348+0000 7faa7f1b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ce761298b0 con 0x55ce75e404d0 2024-04-25T10:42:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.348+0000 7faa7f1b3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ce7616bd40 con 0x55ce75e486b0 2024-04-25T10:42:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.348+0000 7faa7689d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e69390 0x55ce76209a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.348+0000 7faa7689d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e69390 0x55ce76209a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58740/0 (socket says 172.21.15.43:58740) 2024-04-25T10:42:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.348+0000 7faa7689d640 1 -- 172.21.15.43:0/290918259 learned_addr learned my addr 172.21.15.43:0/290918259 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.348+0000 7faa7609c640 1 --2- 172.21.15.43:0/290918259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e404d0 0x55ce76209fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.348+0000 7faa7589b640 1 --2- 172.21.15.43:0/290918259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce75e486b0 0x55ce762168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.348+0000 7faa677fe640 1 --2- 172.21.15.43:0/3593743979 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7faa5c0afce0 0x7faa5c0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.348+0000 7faa7689d640 1 -- 172.21.15.43:0/290918259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce75e486b0 msgr2=0x55ce762168f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.348+0000 7faa7689d640 1 --2- 172.21.15.43:0/290918259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce75e486b0 0x55ce762168f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.348+0000 7faa7689d640 1 -- 172.21.15.43:0/290918259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e404d0 msgr2=0x55ce76209fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.348+0000 7faa7689d640 1 --2- 172.21.15.43:0/290918259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e404d0 0x55ce76209fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.348+0000 7faa7689d640 1 -- 172.21.15.43:0/290918259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce760faa40 con 0x55ce75e69390 2024-04-25T10:42:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.348+0000 7faa7609c640 1 --2- 172.21.15.43:0/290918259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e404d0 0x55ce76209fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.348+0000 7faa7689d640 1 --2- 172.21.15.43:0/3593743979 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7faa5c0afce0 0x7faa5c0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.348+0000 7faa677fe640 1 -- 172.21.15.43:0/3593743979 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7faa6c0df3d0 con 0x7faa701031c0 2024-04-25T10:42:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.349+0000 7faa7689d640 1 --2- 172.21.15.43:0/290918259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e69390 0x55ce76209a60 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7faa68010620 tx=0x7faa68010ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673bd564f5801055 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.349+0000 7faa64bfe640 1 -- 172.21.15.43:0/290918259 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68019070 con 0x55ce75e69390 2024-04-25T10:42:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.349+0000 7faa64bfe640 1 -- 172.21.15.43:0/290918259 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa68022070 con 0x55ce75e69390 2024-04-25T10:42:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.349+0000 7faa64bfe640 1 -- 172.21.15.43:0/290918259 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68011a70 con 0x55ce75e69390 2024-04-25T10:42:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.349+0000 7faa7689d640 1 --2- 172.21.15.43:0/3593743979 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7faa5c0afce0 0x7faa5c0b21f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faa68021000 tx=0x7faa6801f000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.350+0000 7faa7f1b3740 1 -- 172.21.15.43:0/290918259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e69390 msgr2=0x55ce76209a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.350+0000 7faa7f1b3740 1 --2- 172.21.15.43:0/290918259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e69390 0x55ce76209a60 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7faa68010620 tx=0x7faa68010ae0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.350+0000 7faa7f1b3740 1 -- 172.21.15.43:0/290918259 shutdown_connections 2024-04-25T10:42:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.350+0000 7faa7f1b3740 1 --2- 172.21.15.43:0/290918259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e404d0 0x55ce76209fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.350+0000 7faa7f1b3740 1 --2- 172.21.15.43:0/290918259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce75e486b0 0x55ce762168f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.350+0000 7faa7f1b3740 1 --2- 172.21.15.43:0/290918259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e69390 0x55ce76209a60 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.350+0000 7faa7f1b3740 1 -- 172.21.15.43:0/290918259 >> 172.21.15.43:0/290918259 conn(0x55ce75e5d4e0 msgr2=0x55ce76206210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:55.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.350+0000 7faa7f1b3740 1 -- 172.21.15.43:0/290918259 shutdown_connections 2024-04-25T10:42:55.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.350+0000 7faa7f1b3740 1 -- 172.21.15.43:0/290918259 wait complete. 2024-04-25T10:42:55.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.350+0000 7faa7f1b3740 1 -- start start 2024-04-25T10:42:55.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.350+0000 7faa7f1b3740 1 Processor -- start 2024-04-25T10:42:55.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.351+0000 7faa7f1b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e404d0 0x55ce762145d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.351+0000 7faa7f1b3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e486b0 0x55ce76214b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.351+0000 7faa7f1b3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce75e69390 0x55ce7621a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.351+0000 7faa7f1b3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ce76216fe0 con 0x55ce75e486b0 2024-04-25T10:42:55.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.351+0000 7faa7f1b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ce760f6d10 con 0x55ce75e404d0 2024-04-25T10:42:55.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.351+0000 7faa7f1b3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ce76217160 con 0x55ce75e69390 2024-04-25T10:42:55.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.351+0000 7faa7f1b3740 10 client.0 _ll_register_callbacks cb 0x55ce76123210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:55.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.351+0000 7faa7589b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e486b0 0x55ce76214b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.351+0000 7faa7589b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e486b0 0x55ce76214b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58750/0 (socket says 172.21.15.43:58750) 2024-04-25T10:42:55.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.351+0000 7faa7589b640 1 -- 172.21.15.43:0/2771287264 learned_addr learned my addr 172.21.15.43:0/2771287264 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:55.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.351+0000 7faa7609c640 1 --2- 172.21.15.43:0/2771287264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e404d0 0x55ce762145d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.351+0000 7faa7689d640 1 --2- 172.21.15.43:0/2771287264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce75e69390 0x55ce7621a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.351+0000 7faa7589b640 1 -- 172.21.15.43:0/2771287264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce75e69390 msgr2=0x55ce7621a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:55.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.351+0000 7faa7589b640 1 --2- 172.21.15.43:0/2771287264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce75e69390 0x55ce7621a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.351+0000 7faa7589b640 1 -- 172.21.15.43:0/2771287264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e404d0 msgr2=0x55ce762145d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:55.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.351+0000 7faa7589b640 1 --2- 172.21.15.43:0/2771287264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e404d0 0x55ce762145d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.351+0000 7faa7589b640 1 -- 172.21.15.43:0/2771287264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce75f4b7d0 con 0x55ce75e486b0 2024-04-25T10:42:55.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.351+0000 7faa7589b640 1 --2- 172.21.15.43:0/2771287264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e486b0 0x55ce76214b10 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7faa7006c590 tx=0x7faa60032e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe751938f9c54c9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:55.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.351+0000 7faa59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:55.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.352+0000 7faa59ffb640 1 -- 172.21.15.43:0/2771287264 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60035040 con 0x55ce75e486b0 2024-04-25T10:42:55.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.352+0000 7faa7f1b3740 1 -- 172.21.15.43:0/2771287264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa68022070 con 0x55ce75e486b0 2024-04-25T10:42:55.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.352+0000 7faa59ffb640 1 -- 172.21.15.43:0/2771287264 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7faa60031760 con 0x55ce75e486b0 2024-04-25T10:42:55.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.352+0000 7faa59ffb640 1 -- 172.21.15.43:0/2771287264 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faa60004040 con 0x55ce75e486b0 2024-04-25T10:42:55.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.352+0000 7faa7f1b3740 10 client.18534 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:55.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.352+0000 7faa7f1b3740 1 -- 172.21.15.43:0/2771287264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ce75e813f0 con 0x55ce75e486b0 2024-04-25T10:42:55.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.352+0000 7faa59ffb640 1 -- 172.21.15.43:0/2771287264 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa68022070 con 0x55ce75e486b0 2024-04-25T10:42:55.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.352+0000 7faa59ffb640 1 -- 172.21.15.43:0/2771287264 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7faa60004750 con 0x55ce75e486b0 2024-04-25T10:42:55.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.352+0000 7faa7f1b3740 10 client.18534 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:55.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.352+0000 7faa7f1b3740 10 client.18534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:55.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.352+0000 7faa7f1b3740 20 client.18534 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:55.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.352+0000 7faa7f1b3740 1 --2- 172.21.15.43:0/2771287264 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ce75e27f60 0x55ce76226570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.352+0000 7faa7f1b3740 4 client.18534 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:42:55.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.352+0000 7faa7f1b3740 1 -- 172.21.15.43:0/2771287264 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce7617d990 con 0x55ce75e27f60 2024-04-25T10:42:55.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.353+0000 7faa7609c640 1 --2- 172.21.15.43:0/2771287264 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ce75e27f60 0x55ce76226570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.353+0000 7faa59ffb640 10 client.18534 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:55.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.353+0000 7faa7609c640 1 --2- 172.21.15.43:0/2771287264 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ce75e27f60 0x55ce76226570 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b803d1cf259dc05 server_cookie=e7a1b76eb5a0512c in_seq=0 out_seq=0 2024-04-25T10:42:55.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.354+0000 7faa59ffb640 1 -- 172.21.15.43:0/2771287264 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ce7617d990 con 0x55ce75e27f60 2024-04-25T10:42:55.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.354+0000 7faa59ffb640 10 client.18534 handle_command_reply: tid=0 2024-04-25T10:42:55.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.354+0000 7faa7f1b3740 1 client.18534 shutdown 2024-04-25T10:42:55.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.354+0000 7faa7f1b3740 1 -- 172.21.15.43:0/2771287264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e486b0 msgr2=0x55ce76214b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.354+0000 7faa7f1b3740 1 --2- 172.21.15.43:0/2771287264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e486b0 0x55ce76214b10 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7faa7006c590 tx=0x7faa60032e90 comp rx=0 tx=0).stop 2024-04-25T10:42:55.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.354+0000 7faa7f1b3740 1 -- 172.21.15.43:0/2771287264 shutdown_connections 2024-04-25T10:42:55.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.354+0000 7faa7f1b3740 1 --2- 172.21.15.43:0/2771287264 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ce75e27f60 0x55ce76226570 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.354+0000 7faa7f1b3740 1 --2- 172.21.15.43:0/2771287264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce75e69390 0x55ce7621a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.354+0000 7faa59ffb640 0 client.18534 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:55.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.354+0000 7faa7f1b3740 1 --2- 172.21.15.43:0/2771287264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e486b0 0x55ce76214b10 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.354+0000 7faa7f1b3740 1 --2- 172.21.15.43:0/2771287264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e404d0 0x55ce762145d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.354+0000 7faa7f1b3740 1 -- 172.21.15.43:0/2771287264 >> 172.21.15.43:0/2771287264 conn(0x55ce75e5d4e0 msgr2=0x55ce7620b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:55.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.354+0000 7faa7f1b3740 1 -- 172.21.15.43:0/2771287264 shutdown_connections 2024-04-25T10:42:55.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.354+0000 7faa7f1b3740 1 -- 172.21.15.43:0/2771287264 wait complete. 2024-04-25T10:42:55.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.354+0000 7faa7f1b3740 20 client.18534 trim_cache size 0 max 16384 2024-04-25T10:42:55.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.372+0000 7faa7f1b3740 1 Processor -- start 2024-04-25T10:42:55.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.372+0000 7faa7f1b3740 1 -- start start 2024-04-25T10:42:55.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.373+0000 7faa7f1b3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e404d0 0x55ce7620fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.373+0000 7faa7f1b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e486b0 0x55ce762133a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.373+0000 7faa7f1b3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce75e69390 0x55ce7622bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.373+0000 7faa7f1b3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa701b2290 con 0x55ce75e404d0 2024-04-25T10:42:55.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.373+0000 7faa7f1b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faa701afb60 con 0x55ce75e486b0 2024-04-25T10:42:55.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.373+0000 7faa7f1b3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faa701ad9d0 con 0x55ce75e69390 2024-04-25T10:42:55.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.373+0000 7faa7609c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e404d0 0x55ce7620fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.373+0000 7faa7609c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e404d0 0x55ce7620fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58752/0 (socket says 172.21.15.43:58752) 2024-04-25T10:42:55.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.373+0000 7faa7609c640 1 -- 172.21.15.43:0/4219525197 learned_addr learned my addr 172.21.15.43:0/4219525197 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:55.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.373+0000 7faa7589b640 1 --2- 172.21.15.43:0/4219525197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e486b0 0x55ce762133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.373+0000 7faa7609c640 1 -- 172.21.15.43:0/4219525197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce75e69390 msgr2=0x55ce7622bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:55.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.373+0000 7faa7609c640 1 --2- 172.21.15.43:0/4219525197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce75e69390 0x55ce7622bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.373+0000 7faa7609c640 1 -- 172.21.15.43:0/4219525197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e486b0 msgr2=0x55ce762133a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:55.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.373+0000 7faa7609c640 1 --2- 172.21.15.43:0/4219525197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e486b0 0x55ce762133a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.373+0000 7faa7609c640 1 -- 172.21.15.43:0/4219525197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce75e410f0 con 0x55ce75e404d0 2024-04-25T10:42:55.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.373+0000 7faa7589b640 1 --2- 172.21.15.43:0/4219525197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e486b0 0x55ce762133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:55.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.373+0000 7faa7609c640 1 --2- 172.21.15.43:0/4219525197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e404d0 0x55ce7620fbc0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0dd120 tx=0x7faa6c090040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8bf431f46c8723 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:55.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.373+0000 7faa5bfff640 1 -- 172.21.15.43:0/4219525197 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c00ce60 con 0x55ce75e404d0 2024-04-25T10:42:55.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.373+0000 7faa5bfff640 1 -- 172.21.15.43:0/4219525197 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa6c005310 con 0x55ce75e404d0 2024-04-25T10:42:55.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.373+0000 7faa5bfff640 1 -- 172.21.15.43:0/4219525197 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c0ab920 con 0x55ce75e404d0 2024-04-25T10:42:55.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.374+0000 7faa7f1b3740 1 -- 172.21.15.43:0/4219525197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e404d0 msgr2=0x55ce7620fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.374+0000 7faa7f1b3740 1 --2- 172.21.15.43:0/4219525197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e404d0 0x55ce7620fbc0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0dd120 tx=0x7faa6c090040 comp rx=0 tx=0).stop 2024-04-25T10:42:55.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.374+0000 7faa7f1b3740 1 -- 172.21.15.43:0/4219525197 shutdown_connections 2024-04-25T10:42:55.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.374+0000 7faa7f1b3740 1 --2- 172.21.15.43:0/4219525197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e486b0 0x55ce762133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.374+0000 7faa7f1b3740 1 --2- 172.21.15.43:0/4219525197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce75e69390 0x55ce7622bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.374+0000 7faa7f1b3740 1 --2- 172.21.15.43:0/4219525197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e404d0 0x55ce7620fbc0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.374+0000 7faa7f1b3740 1 -- 172.21.15.43:0/4219525197 >> 172.21.15.43:0/4219525197 conn(0x55ce75e5d4e0 msgr2=0x55ce76212c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:55.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.374+0000 7faa7f1b3740 1 -- 172.21.15.43:0/4219525197 shutdown_connections 2024-04-25T10:42:55.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.374+0000 7faa7f1b3740 1 -- 172.21.15.43:0/4219525197 wait complete. 2024-04-25T10:42:55.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.375+0000 7faa7f1b3740 1 -- start start 2024-04-25T10:42:55.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.375+0000 7faa7f1b3740 1 Processor -- start 2024-04-25T10:42:55.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.375+0000 7faa7f1b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e404d0 0x55ce76220ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.375+0000 7faa7f1b3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce75e486b0 0x55ce7621c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.375+0000 7faa7f1b3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e69390 0x55ce7621c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.375+0000 7faa7f1b3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa701ae020 con 0x55ce75e69390 2024-04-25T10:42:55.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.375+0000 7faa7f1b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faa701af490 con 0x55ce75e404d0 2024-04-25T10:42:55.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.375+0000 7faa7f1b3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faa701ae6f0 con 0x55ce75e486b0 2024-04-25T10:42:55.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.375+0000 7faa7f1b3740 10 client.0 _ll_register_callbacks cb 0x7faa60009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.375+0000 7faa7689d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e69390 0x55ce7621c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.375+0000 7faa7689d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e69390 0x55ce7621c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58768/0 (socket says 172.21.15.43:58768) 2024-04-25T10:42:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.375+0000 7faa7689d640 1 -- 172.21.15.43:0/1406958741 learned_addr learned my addr 172.21.15.43:0/1406958741 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.375+0000 7faa7609c640 1 --2- 172.21.15.43:0/1406958741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e404d0 0x55ce76220ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.375+0000 7faa7689d640 1 -- 172.21.15.43:0/1406958741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce75e486b0 msgr2=0x55ce7621c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.375+0000 7faa7689d640 1 --2- 172.21.15.43:0/1406958741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce75e486b0 0x55ce7621c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.375+0000 7faa7689d640 1 -- 172.21.15.43:0/1406958741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e404d0 msgr2=0x55ce76220ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.375+0000 7faa7689d640 1 --2- 172.21.15.43:0/1406958741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e404d0 0x55ce76220ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.375+0000 7faa7689d640 1 -- 172.21.15.43:0/1406958741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce761304b0 con 0x55ce75e69390 2024-04-25T10:42:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.375+0000 7faa7609c640 1 --2- 172.21.15.43:0/1406958741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e404d0 0x55ce76220ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.376+0000 7faa7689d640 1 --2- 172.21.15.43:0/1406958741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e69390 0x55ce7621c760 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7faa68010620 tx=0x7faa68010530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f888cca658482cb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.376+0000 7faa59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.376+0000 7faa59ffb640 1 -- 172.21.15.43:0/1406958741 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68019070 con 0x55ce75e69390 2024-04-25T10:42:55.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.376+0000 7faa59ffb640 1 -- 172.21.15.43:0/1406958741 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7faa68013ab0 con 0x55ce75e69390 2024-04-25T10:42:55.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.376+0000 7faa7f1b3740 1 -- 172.21.15.43:0/1406958741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ce76177800 con 0x55ce75e69390 2024-04-25T10:42:55.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.376+0000 7faa59ffb640 1 -- 172.21.15.43:0/1406958741 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faa68013cb0 con 0x55ce75e69390 2024-04-25T10:42:55.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.376+0000 7faa7f1b3740 10 client.18540 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:55.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.376+0000 7faa7f1b3740 1 -- 172.21.15.43:0/1406958741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ce7628e4e0 con 0x55ce75e69390 2024-04-25T10:42:55.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.376+0000 7faa59ffb640 1 -- 172.21.15.43:0/1406958741 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ce76177800 con 0x55ce75e69390 2024-04-25T10:42:55.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.376+0000 7faa59ffb640 1 -- 172.21.15.43:0/1406958741 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7faa6802e430 con 0x55ce75e69390 2024-04-25T10:42:55.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.376+0000 7faa7f1b3740 10 client.18540 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:55.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.376+0000 7faa7f1b3740 10 client.18540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:55.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.376+0000 7faa7f1b3740 20 client.18540 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:55.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.376+0000 7faa7f1b3740 1 --2- 172.21.15.43:0/1406958741 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ce75dfd3a0 0x55ce76290ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.376+0000 7faa7f1b3740 4 client.18540 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:42:55.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.376+0000 7faa7f1b3740 1 -- 172.21.15.43:0/1406958741 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ce760f24a0 con 0x55ce75dfd3a0 2024-04-25T10:42:55.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.377+0000 7faa7609c640 1 --2- 172.21.15.43:0/1406958741 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ce75dfd3a0 0x55ce76290ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.377+0000 7faa7609c640 1 --2- 172.21.15.43:0/1406958741 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ce75dfd3a0 0x55ce76290ad0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d65a90584e31915 server_cookie=d2481ce42fa46e17 in_seq=0 out_seq=0 2024-04-25T10:42:55.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.377+0000 7faa59ffb640 10 client.18540 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:55.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.377+0000 7faa59ffb640 1 -- 172.21.15.43:0/1406958741 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ce760f24a0 con 0x55ce75dfd3a0 2024-04-25T10:42:55.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.377+0000 7faa59ffb640 10 client.18540 handle_command_reply: tid=0 2024-04-25T10:42:55.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.377+0000 7faa7f1b3740 1 client.18540 shutdown 2024-04-25T10:42:55.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.377+0000 7faa7f1b3740 1 -- 172.21.15.43:0/1406958741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e69390 msgr2=0x55ce7621c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.377+0000 7faa7f1b3740 1 --2- 172.21.15.43:0/1406958741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e69390 0x55ce7621c760 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7faa68010620 tx=0x7faa68010530 comp rx=0 tx=0).stop 2024-04-25T10:42:55.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.377+0000 7faa7f1b3740 1 -- 172.21.15.43:0/1406958741 shutdown_connections 2024-04-25T10:42:55.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.377+0000 7faa7f1b3740 1 --2- 172.21.15.43:0/1406958741 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55ce75dfd3a0 0x55ce76290ad0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.377+0000 7faa7f1b3740 1 --2- 172.21.15.43:0/1406958741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce75e69390 0x55ce7621c760 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.377+0000 7faa7f1b3740 1 --2- 172.21.15.43:0/1406958741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce75e486b0 0x55ce7621c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.377+0000 7faa59ffb640 0 client.18540 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:55.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.377+0000 7faa7f1b3740 1 --2- 172.21.15.43:0/1406958741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce75e404d0 0x55ce76220ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.377+0000 7faa7f1b3740 1 -- 172.21.15.43:0/1406958741 >> 172.21.15.43:0/1406958741 conn(0x55ce75e5d4e0 msgr2=0x55ce76215880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:55.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.377+0000 7faa7f1b3740 1 -- 172.21.15.43:0/1406958741 shutdown_connections 2024-04-25T10:42:55.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.377+0000 7faa7f1b3740 1 -- 172.21.15.43:0/1406958741 wait complete. 2024-04-25T10:42:55.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.378+0000 7faa7f1b3740 20 client.18540 trim_cache size 0 max 16384 2024-04-25T10:42:55.396 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:42:55.396 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:42:55.396 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:42:55.397 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:42:55.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.378+0000 7faa78300640 1 -- 172.21.15.43:0/3593743979 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7faa5c0afce0 msgr2=0x7faa5c0b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.378+0000 7faa78300640 1 --2- 172.21.15.43:0/3593743979 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7faa5c0afce0 0x7faa5c0b21f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faa68021000 tx=0x7faa6801f000 comp rx=0 tx=0).stop 2024-04-25T10:42:55.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.378+0000 7faa78300640 1 -- 172.21.15.43:0/3593743979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa701031c0 msgr2=0x7faa7019a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.378+0000 7faa78300640 1 --2- 172.21.15.43:0/3593743979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa701031c0 0x7faa7019a760 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00e9c0 tx=0x7faa6c00ee80 comp rx=0 tx=0).stop 2024-04-25T10:42:55.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.378+0000 7faa78300640 1 -- 172.21.15.43:0/3593743979 shutdown_connections 2024-04-25T10:42:55.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.378+0000 7faa78300640 1 --2- 172.21.15.43:0/3593743979 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7faa5c0afce0 0x7faa5c0b21f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.378+0000 7faa78300640 1 --2- 172.21.15.43:0/3593743979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa70108eb0 0x7faa7019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.378+0000 7faa78300640 1 --2- 172.21.15.43:0/3593743979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa70108a10 0x7faa7019acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.378+0000 7faa78300640 1 --2- 172.21.15.43:0/3593743979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa701031c0 0x7faa7019a760 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.378+0000 7faa78300640 1 -- 172.21.15.43:0/3593743979 >> 172.21.15.43:0/3593743979 conn(0x7faa70078110 msgr2=0x7faa70074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:55.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.378+0000 7faa78300640 1 -- 172.21.15.43:0/3593743979 shutdown_connections 2024-04-25T10:42:55.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.378+0000 7faa78300640 1 -- 172.21.15.43:0/3593743979 wait complete. 2024-04-25T10:42:55.399 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:42:55.399 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:42:55.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.510+0000 7f58e6da8640 1 -- 172.21.15.43:0/647799358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58e0109fd0 msgr2=0x7f58e0103c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.510+0000 7f58e6da8640 1 --2- 172.21.15.43:0/647799358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58e0109fd0 0x7f58e0103c10 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009900 tx=0x7f58c802a410 comp rx=0 tx=0).stop 2024-04-25T10:42:55.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.510+0000 7f58e6da8640 1 -- 172.21.15.43:0/647799358 shutdown_connections 2024-04-25T10:42:55.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.510+0000 7f58e6da8640 1 --2- 172.21.15.43:0/647799358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58e0104280 0x7f58e0110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.510+0000 7f58e6da8640 1 --2- 172.21.15.43:0/647799358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58e0109fd0 0x7f58e0103c10 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.510+0000 7f58e6da8640 1 --2- 172.21.15.43:0/647799358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58e0109b30 0x7f58e01036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.510+0000 7f58e6da8640 1 -- 172.21.15.43:0/647799358 >> 172.21.15.43:0/647799358 conn(0x7f58e00ff210 msgr2=0x7f58e0101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:55.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.510+0000 7f58e6da8640 1 -- 172.21.15.43:0/647799358 shutdown_connections 2024-04-25T10:42:55.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.510+0000 7f58e6da8640 1 -- 172.21.15.43:0/647799358 wait complete. 2024-04-25T10:42:55.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.511+0000 7f58e6da8640 1 Processor -- start 2024-04-25T10:42:55.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.511+0000 7f58e6da8640 1 -- start start 2024-04-25T10:42:55.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.511+0000 7f58e6da8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58e0104280 0x7f58e0075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.511+0000 7f58e6da8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58e0109b30 0x7f58e00764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.511+0000 7f58e6da8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58e0109fd0 0x7f58e0076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.511+0000 7f58e6da8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f58e0113d20 con 0x7f58e0109fd0 2024-04-25T10:42:55.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.511+0000 7f58e6da8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f58e0113ba0 con 0x7f58e0104280 2024-04-25T10:42:55.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.511+0000 7f58e6da8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f58e0113ea0 con 0x7f58e0109b30 2024-04-25T10:42:55.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.511+0000 7f58e4b44640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58e0104280 0x7f58e0075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.511+0000 7f58e5345640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58e0109fd0 0x7f58e0076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.511+0000 7f58e4b44640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58e0104280 0x7f58e0075fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:47432/0 (socket says 172.21.15.43:47432) 2024-04-25T10:42:55.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.511+0000 7f58e5345640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58e0109fd0 0x7f58e0076a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58776/0 (socket says 172.21.15.43:58776) 2024-04-25T10:42:55.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.511+0000 7f58e4b44640 1 -- 172.21.15.43:0/2283360213 learned_addr learned my addr 172.21.15.43:0/2283360213 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:55.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.512+0000 7f58dffff640 1 --2- 172.21.15.43:0/2283360213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58e0109b30 0x7f58e00764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.512+0000 7f58e5345640 1 -- 172.21.15.43:0/2283360213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58e0109b30 msgr2=0x7f58e00764f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.512+0000 7f58e5345640 1 --2- 172.21.15.43:0/2283360213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58e0109b30 0x7f58e00764f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.512+0000 7f58e5345640 1 -- 172.21.15.43:0/2283360213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58e0104280 msgr2=0x7f58e0075fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.512+0000 7f58e5345640 1 --2- 172.21.15.43:0/2283360213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58e0104280 0x7f58e0075fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.512+0000 7f58e5345640 1 -- 172.21.15.43:0/2283360213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e01a8b00 con 0x7f58e0109fd0 2024-04-25T10:42:55.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.512+0000 7f58e5345640 1 --2- 172.21.15.43:0/2283360213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58e0109fd0 0x7f58e0076a30 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f58d800e940 tx=0x7f58d800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:55.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.512+0000 7f58ddffb640 1 -- 172.21.15.43:0/2283360213 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8004d20 con 0x7f58e0109fd0 2024-04-25T10:42:55.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.512+0000 7f58ddffb640 1 -- 172.21.15.43:0/2283360213 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58d8005020 con 0x7f58e0109fd0 2024-04-25T10:42:55.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.512+0000 7f58e6da8640 1 -- 172.21.15.43:0/2283360213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e01a8df0 con 0x7f58e0109fd0 2024-04-25T10:42:55.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.512+0000 7f58ddffb640 1 -- 172.21.15.43:0/2283360213 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d80116d0 con 0x7f58e0109fd0 2024-04-25T10:42:55.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.512+0000 7f58e6da8640 1 -- 172.21.15.43:0/2283360213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e01a9230 con 0x7f58e0109fd0 2024-04-25T10:42:55.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.513+0000 7f58e6da8640 1 -- 172.21.15.43:0/2283360213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e006a190 con 0x7f58e0109fd0 2024-04-25T10:42:55.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.515+0000 7f58ddffb640 1 -- 172.21.15.43:0/2283360213 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f58d8005460 con 0x7f58e0109fd0 2024-04-25T10:42:55.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.515+0000 7f58ddffb640 1 --2- 172.21.15.43:0/2283360213 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f58cc0afce0 0x7f58cc0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.516+0000 7f58e4b44640 1 --2- 172.21.15.43:0/2283360213 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f58cc0afce0 0x7f58cc0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.516+0000 7f58ddffb640 1 -- 172.21.15.43:0/2283360213 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f58d80df0a0 con 0x7f58e0109fd0 2024-04-25T10:42:55.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.516+0000 7f58e4b44640 1 --2- 172.21.15.43:0/2283360213 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f58cc0afce0 0x7f58cc0b21f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f58d0005bc0 tx=0x7f58d000b480 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:55.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.521+0000 7f58ddffb640 1 -- 172.21.15.43:0/2283360213 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f58d8016030 con 0x7f58e0109fd0 2024-04-25T10:42:55.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.746+0000 7f58e6da8640 1 -- 172.21.15.43:0/2283360213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58e00774e0 con 0x7f58e0109fd0 2024-04-25T10:42:55.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.747+0000 7f58ddffb640 1 -- 172.21.15.43:0/2283360213 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7f58d80a7940 con 0x7f58e0109fd0 2024-04-25T10:42:55.749 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:42:55.749 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:42:55.749 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:42:55.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.749+0000 7f58e6da8640 1 -- 172.21.15.43:0/2283360213 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f58cc0afce0 msgr2=0x7f58cc0b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.749+0000 7f58e6da8640 1 --2- 172.21.15.43:0/2283360213 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f58cc0afce0 0x7f58cc0b21f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f58d0005bc0 tx=0x7f58d000b480 comp rx=0 tx=0).stop 2024-04-25T10:42:55.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.749+0000 7f58e6da8640 1 -- 172.21.15.43:0/2283360213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58e0109fd0 msgr2=0x7f58e0076a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.749+0000 7f58e6da8640 1 --2- 172.21.15.43:0/2283360213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58e0109fd0 0x7f58e0076a30 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f58d800e940 tx=0x7f58d800ee00 comp rx=0 tx=0).stop 2024-04-25T10:42:55.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.749+0000 7f58e6da8640 1 -- 172.21.15.43:0/2283360213 shutdown_connections 2024-04-25T10:42:55.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.749+0000 7f58e6da8640 1 --2- 172.21.15.43:0/2283360213 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f58cc0afce0 0x7f58cc0b21f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.749+0000 7f58e6da8640 1 --2- 172.21.15.43:0/2283360213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58e0109fd0 0x7f58e0076a30 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.749+0000 7f58e6da8640 1 --2- 172.21.15.43:0/2283360213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58e0109b30 0x7f58e00764f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.749+0000 7f58e6da8640 1 --2- 172.21.15.43:0/2283360213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58e0104280 0x7f58e0075fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.749+0000 7f58e6da8640 1 -- 172.21.15.43:0/2283360213 >> 172.21.15.43:0/2283360213 conn(0x7f58e00ff210 msgr2=0x7f58e010f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:55.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.749+0000 7f58e6da8640 1 -- 172.21.15.43:0/2283360213 shutdown_connections 2024-04-25T10:42:55.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.749+0000 7f58e6da8640 1 -- 172.21.15.43:0/2283360213 wait complete. 2024-04-25T10:42:55.765 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:42:55.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.877+0000 7ff1716bd640 1 -- 172.21.15.43:0/3884386792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16c109b10 msgr2=0x7ff16c1036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.877+0000 7ff1716bd640 1 --2- 172.21.15.43:0/3884386792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16c109b10 0x7ff16c1036b0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009900 tx=0x7ff15c02a410 comp rx=0 tx=0).stop 2024-04-25T10:42:55.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.878+0000 7ff1716bd640 1 -- 172.21.15.43:0/3884386792 shutdown_connections 2024-04-25T10:42:55.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.878+0000 7ff1716bd640 1 --2- 172.21.15.43:0/3884386792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff16c109fb0 0x7ff16c103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.878+0000 7ff1716bd640 1 --2- 172.21.15.43:0/3884386792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff16c104260 0x7ff16c110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.878+0000 7ff1716bd640 1 --2- 172.21.15.43:0/3884386792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16c109b10 0x7ff16c1036b0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.878+0000 7ff1716bd640 1 -- 172.21.15.43:0/3884386792 >> 172.21.15.43:0/3884386792 conn(0x7ff16c0ff210 msgr2=0x7ff16c101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:55.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.878+0000 7ff1716bd640 1 -- 172.21.15.43:0/3884386792 shutdown_connections 2024-04-25T10:42:55.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.878+0000 7ff1716bd640 1 -- 172.21.15.43:0/3884386792 wait complete. 2024-04-25T10:42:55.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.878+0000 7ff1716bd640 1 Processor -- start 2024-04-25T10:42:55.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.879+0000 7ff1716bd640 1 -- start start 2024-04-25T10:42:55.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.879+0000 7ff1716bd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff16c104260 0x7ff16c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.879+0000 7ff1716bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16c109b10 0x7ff16c19ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.879+0000 7ff1716bd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff16c109fb0 0x7ff16c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.879+0000 7ff1716bd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c113d00 con 0x7ff16c109b10 2024-04-25T10:42:55.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.879+0000 7ff1716bd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c113b80 con 0x7ff16c104260 2024-04-25T10:42:55.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.879+0000 7ff1716bd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c113e80 con 0x7ff16c109fb0 2024-04-25T10:42:55.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.879+0000 7ff16a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16c109b10 0x7ff16c19ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.879+0000 7ff16a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16c109b10 0x7ff16c19ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58794/0 (socket says 172.21.15.43:58794) 2024-04-25T10:42:55.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.879+0000 7ff16a7fc640 1 -- 172.21.15.43:0/3027106619 learned_addr learned my addr 172.21.15.43:0/3027106619 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:55.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.879+0000 7ff16affd640 1 --2- 172.21.15.43:0/3027106619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff16c104260 0x7ff16c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.879+0000 7ff16b7fe640 1 --2- 172.21.15.43:0/3027106619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff16c109fb0 0x7ff16c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.879+0000 7ff16a7fc640 1 -- 172.21.15.43:0/3027106619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff16c109fb0 msgr2=0x7ff16c19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.879+0000 7ff16a7fc640 1 --2- 172.21.15.43:0/3027106619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff16c109fb0 0x7ff16c19efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.879+0000 7ff16a7fc640 1 -- 172.21.15.43:0/3027106619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff16c104260 msgr2=0x7ff16c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.879+0000 7ff16a7fc640 1 --2- 172.21.15.43:0/3027106619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff16c104260 0x7ff16c19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.879+0000 7ff16a7fc640 1 -- 172.21.15.43:0/3027106619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16c19f5e0 con 0x7ff16c109b10 2024-04-25T10:42:55.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.879+0000 7ff16a7fc640 1 --2- 172.21.15.43:0/3027106619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16c109b10 0x7ff16c19ace0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ff16000d740 tx=0x7ff16000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:55.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.880+0000 7ff153fff640 1 -- 172.21.15.43:0/3027106619 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160017070 con 0x7ff16c109b10 2024-04-25T10:42:55.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.880+0000 7ff153fff640 1 -- 172.21.15.43:0/3027106619 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff16000de50 con 0x7ff16c109b10 2024-04-25T10:42:55.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.880+0000 7ff1716bd640 1 -- 172.21.15.43:0/3027106619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16c19f8d0 con 0x7ff16c109b10 2024-04-25T10:42:55.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.880+0000 7ff153fff640 1 -- 172.21.15.43:0/3027106619 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160003c20 con 0x7ff16c109b10 2024-04-25T10:42:55.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.880+0000 7ff1716bd640 1 -- 172.21.15.43:0/3027106619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff16c1a7250 con 0x7ff16c109b10 2024-04-25T10:42:55.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.882+0000 7ff178573740 1 Processor -- start 2024-04-25T10:42:55.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.882+0000 7ff178573740 1 -- start start 2024-04-25T10:42:55.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.883+0000 7ff153fff640 1 -- 172.21.15.43:0/3027106619 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7ff160003dc0 con 0x7ff16c109b10 2024-04-25T10:42:55.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.883+0000 7ff178573740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e5f390 0x5618872006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.883+0000 7ff178573740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e364d0 0x561887200be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.883+0000 7ff178573740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561886e3e6b0 0x561887211ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.883+0000 7ff178573740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561887165d80 con 0x561886e5f390 2024-04-25T10:42:55.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.883+0000 7ff178573740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56188711fa20 con 0x561886e3e6b0 2024-04-25T10:42:55.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.883+0000 7ff178573740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561887161d40 con 0x561886e364d0 2024-04-25T10:42:55.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.883+0000 7ff16b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e5f390 0x5618872006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.883+0000 7ff16b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e5f390 0x5618872006a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58804/0 (socket says 172.21.15.43:58804) 2024-04-25T10:42:55.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.883+0000 7ff16b7fe640 1 -- 172.21.15.43:0/1829541529 learned_addr learned my addr 172.21.15.43:0/1829541529 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:55.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.883+0000 7ff16a7fc640 1 --2- 172.21.15.43:0/1829541529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561886e3e6b0 0x561887211ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.883+0000 7ff16affd640 1 --2- 172.21.15.43:0/1829541529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e364d0 0x561887200be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.883+0000 7ff16b7fe640 1 -- 172.21.15.43:0/1829541529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e364d0 msgr2=0x561887200be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:55.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.883+0000 7ff16b7fe640 1 --2- 172.21.15.43:0/1829541529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e364d0 0x561887200be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.883+0000 7ff16b7fe640 1 -- 172.21.15.43:0/1829541529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561886e3e6b0 msgr2=0x561887211ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:55.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.883+0000 7ff16b7fe640 1 --2- 172.21.15.43:0/1829541529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561886e3e6b0 0x561887211ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.883+0000 7ff16b7fe640 1 -- 172.21.15.43:0/1829541529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56188712ac30 con 0x561886e5f390 2024-04-25T10:42:55.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.883+0000 7ff16affd640 1 --2- 172.21.15.43:0/1829541529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e364d0 0x561887200be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.883+0000 7ff16b7fe640 1 --2- 172.21.15.43:0/1829541529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e5f390 0x5618872006a0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ff164009e70 tx=0x7ff16400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a041cfa7004b2cd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.884+0000 7ff153fff640 1 --2- 172.21.15.43:0/3027106619 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff1540afc60 0x7ff1540b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.884+0000 7ff16affd640 1 --2- 172.21.15.43:0/3027106619 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff1540afc60 0x7ff1540b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.884+0000 7ff1517fa640 1 -- 172.21.15.43:0/1829541529 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff164016070 con 0x561886e5f390 2024-04-25T10:42:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.884+0000 7ff153fff640 1 -- 172.21.15.43:0/3027106619 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7ff1600dead0 con 0x7ff16c109b10 2024-04-25T10:42:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.884+0000 7ff1517fa640 1 -- 172.21.15.43:0/1829541529 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff16400ee90 con 0x561886e5f390 2024-04-25T10:42:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.884+0000 7ff1517fa640 1 -- 172.21.15.43:0/1829541529 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1640116d0 con 0x561886e5f390 2024-04-25T10:42:55.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.885+0000 7ff178573740 1 -- 172.21.15.43:0/1829541529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e5f390 msgr2=0x5618872006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.885+0000 7ff16affd640 1 --2- 172.21.15.43:0/3027106619 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff1540afc60 0x7ff1540b2170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff15c000c00 tx=0x7ff15c032d20 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:55.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.885+0000 7ff178573740 1 --2- 172.21.15.43:0/1829541529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e5f390 0x5618872006a0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ff164009e70 tx=0x7ff16400e6c0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.885+0000 7ff178573740 1 -- 172.21.15.43:0/1829541529 shutdown_connections 2024-04-25T10:42:55.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.885+0000 7ff178573740 1 --2- 172.21.15.43:0/1829541529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561886e3e6b0 0x561887211ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.885+0000 7ff178573740 1 --2- 172.21.15.43:0/1829541529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e364d0 0x561887200be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.885+0000 7ff178573740 1 --2- 172.21.15.43:0/1829541529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e5f390 0x5618872006a0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.885+0000 7ff178573740 1 -- 172.21.15.43:0/1829541529 >> 172.21.15.43:0/1829541529 conn(0x561886e534e0 msgr2=0x5618871fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:55.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.885+0000 7ff178573740 1 -- 172.21.15.43:0/1829541529 shutdown_connections 2024-04-25T10:42:55.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.885+0000 7ff178573740 1 -- 172.21.15.43:0/1829541529 wait complete. 2024-04-25T10:42:55.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.885+0000 7ff178573740 1 -- start start 2024-04-25T10:42:55.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.885+0000 7ff178573740 1 Processor -- start 2024-04-25T10:42:55.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.885+0000 7ff178573740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561886e364d0 0x56188720f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.885+0000 7ff178573740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e3e6b0 0x56188720fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.885+0000 7ff178573740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e5f390 0x56188721b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.885+0000 7ff178573740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5618872121a0 con 0x561886e5f390 2024-04-25T10:42:55.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.885+0000 7ff178573740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5618870ecdc0 con 0x561886e364d0 2024-04-25T10:42:55.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.885+0000 7ff178573740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561887212320 con 0x561886e3e6b0 2024-04-25T10:42:55.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.885+0000 7ff178573740 10 client.0 _ll_register_callbacks cb 0x561887108600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:55.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.886+0000 7ff16b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e5f390 0x56188721b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.886+0000 7ff16b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e5f390 0x56188721b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58818/0 (socket says 172.21.15.43:58818) 2024-04-25T10:42:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.886+0000 7ff16b7fe640 1 -- 172.21.15.43:0/1479167048 learned_addr learned my addr 172.21.15.43:0/1479167048 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.886+0000 7ff16affd640 1 --2- 172.21.15.43:0/1479167048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561886e364d0 0x56188720f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.886+0000 7ff16a7fc640 1 --2- 172.21.15.43:0/1479167048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e3e6b0 0x56188720fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.886+0000 7ff16b7fe640 1 -- 172.21.15.43:0/1479167048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e3e6b0 msgr2=0x56188720fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.886+0000 7ff16b7fe640 1 --2- 172.21.15.43:0/1479167048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e3e6b0 0x56188720fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.886+0000 7ff16b7fe640 1 -- 172.21.15.43:0/1479167048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561886e364d0 msgr2=0x56188720f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.886+0000 7ff16b7fe640 1 --2- 172.21.15.43:0/1479167048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561886e364d0 0x56188720f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.886+0000 7ff16b7fe640 1 -- 172.21.15.43:0/1479167048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561886e773f0 con 0x561886e5f390 2024-04-25T10:42:55.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.886+0000 7ff16b7fe640 1 --2- 172.21.15.43:0/1479167048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e5f390 0x56188721b6e0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7ff164006010 tx=0x7ff164011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35ad5fe53c6e2d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:55.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.886+0000 7ff14eff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:55.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.886+0000 7ff14eff5640 1 -- 172.21.15.43:0/1479167048 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff164028060 con 0x561886e5f390 2024-04-25T10:42:55.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.886+0000 7ff14eff5640 1 -- 172.21.15.43:0/1479167048 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7ff164023410 con 0x561886e5f390 2024-04-25T10:42:55.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.886+0000 7ff178573740 1 -- 172.21.15.43:0/1479167048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff16400ee90 con 0x561886e5f390 2024-04-25T10:42:55.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.886+0000 7ff14eff5640 1 -- 172.21.15.43:0/1479167048 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff164011e60 con 0x561886e5f390 2024-04-25T10:42:55.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.886+0000 7ff178573740 10 client.18558 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:55.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.886+0000 7ff178573740 1 -- 172.21.15.43:0/1479167048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561886f417d0 con 0x561886e5f390 2024-04-25T10:42:55.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.887+0000 7ff14eff5640 1 -- 172.21.15.43:0/1479167048 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff16400ee90 con 0x561886e5f390 2024-04-25T10:42:55.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.887+0000 7ff14eff5640 1 -- 172.21.15.43:0/1479167048 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff16400f040 con 0x561886e5f390 2024-04-25T10:42:55.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.887+0000 7ff178573740 10 client.18558 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:55.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.887+0000 7ff178573740 10 client.18558 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:42:55.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.887+0000 7ff178573740 20 client.18558 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:55.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.887+0000 7ff178573740 1 --2- 172.21.15.43:0/1479167048 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x561886e1df60 0x561887221a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.887+0000 7ff178573740 4 client.18558 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:42:55.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.887+0000 7ff178573740 1 -- 172.21.15.43:0/1479167048 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561887173990 con 0x561886e1df60 2024-04-25T10:42:55.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.887+0000 7ff16affd640 1 --2- 172.21.15.43:0/1479167048 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x561886e1df60 0x561887221a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.888+0000 7ff14eff5640 10 client.18558 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:42:55.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.888+0000 7ff16affd640 1 --2- 172.21.15.43:0/1479167048 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x561886e1df60 0x561887221a10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e93c07252305c112 server_cookie=6334219dda46fe07 in_seq=0 out_seq=0 2024-04-25T10:42:55.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.888+0000 7ff14eff5640 1 -- 172.21.15.43:0/1479167048 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561887173990 con 0x561886e1df60 2024-04-25T10:42:55.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.888+0000 7ff14eff5640 10 client.18558 handle_command_reply: tid=0 2024-04-25T10:42:55.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.889+0000 7ff178573740 1 client.18558 shutdown 2024-04-25T10:42:55.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.889+0000 7ff178573740 1 -- 172.21.15.43:0/1479167048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e5f390 msgr2=0x56188721b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.889+0000 7ff178573740 1 --2- 172.21.15.43:0/1479167048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e5f390 0x56188721b6e0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7ff164006010 tx=0x7ff164011cc0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.889+0000 7ff178573740 1 -- 172.21.15.43:0/1479167048 shutdown_connections 2024-04-25T10:42:55.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.889+0000 7ff178573740 1 --2- 172.21.15.43:0/1479167048 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x561886e1df60 0x561887221a10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.889+0000 7ff178573740 1 --2- 172.21.15.43:0/1479167048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e5f390 0x56188721b6e0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.889+0000 7ff178573740 1 --2- 172.21.15.43:0/1479167048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e3e6b0 0x56188720fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.889+0000 7ff14eff5640 0 client.18558 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:42:55.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.889+0000 7ff178573740 1 --2- 172.21.15.43:0/1479167048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561886e364d0 0x56188720f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.889+0000 7ff178573740 1 -- 172.21.15.43:0/1479167048 >> 172.21.15.43:0/1479167048 conn(0x561886e534e0 msgr2=0x561887202480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:55.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.889+0000 7ff178573740 1 -- 172.21.15.43:0/1479167048 shutdown_connections 2024-04-25T10:42:55.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.889+0000 7ff178573740 1 -- 172.21.15.43:0/1479167048 wait complete. 2024-04-25T10:42:55.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.889+0000 7ff178573740 20 client.18558 trim_cache size 0 max 16384 2024-04-25T10:42:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.903+0000 7ff178573740 1 Processor -- start 2024-04-25T10:42:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.903+0000 7ff178573740 1 -- start start 2024-04-25T10:42:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.903+0000 7ff178573740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e364d0 0x56188721b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.903+0000 7ff178573740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e3e6b0 0x5618871fced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.903+0000 7ff178573740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561886e5f390 0x56188720c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.903+0000 7ff178573740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c1b01b0 con 0x561886e3e6b0 2024-04-25T10:42:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.903+0000 7ff178573740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c1ada80 con 0x561886e5f390 2024-04-25T10:42:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.903+0000 7ff178573740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c1ab910 con 0x561886e364d0 2024-04-25T10:42:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.903+0000 7ff16a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e3e6b0 0x5618871fced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.903+0000 7ff16a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e3e6b0 0x5618871fced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58832/0 (socket says 172.21.15.43:58832) 2024-04-25T10:42:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.903+0000 7ff16a7fc640 1 -- 172.21.15.43:0/2090862876 learned_addr learned my addr 172.21.15.43:0/2090862876 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.903+0000 7ff16b7fe640 1 --2- 172.21.15.43:0/2090862876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561886e5f390 0x56188720c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.903+0000 7ff16affd640 1 --2- 172.21.15.43:0/2090862876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e364d0 0x56188721b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.903+0000 7ff16a7fc640 1 -- 172.21.15.43:0/2090862876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e364d0 msgr2=0x56188721b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.903+0000 7ff16a7fc640 1 --2- 172.21.15.43:0/2090862876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e364d0 0x56188721b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.903+0000 7ff16a7fc640 1 -- 172.21.15.43:0/2090862876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561886e5f390 msgr2=0x56188720c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.903+0000 7ff16a7fc640 1 --2- 172.21.15.43:0/2090862876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561886e5f390 0x56188720c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.903+0000 7ff16a7fc640 1 -- 172.21.15.43:0/2090862876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618872103c0 con 0x561886e3e6b0 2024-04-25T10:42:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.904+0000 7ff16a7fc640 1 --2- 172.21.15.43:0/2090862876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e3e6b0 0x5618871fced0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7ff1600dc280 tx=0x7ff1600036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a374b74c798bd678 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.904+0000 7ff150ff9640 1 -- 172.21.15.43:0/2090862876 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160020040 con 0x561886e3e6b0 2024-04-25T10:42:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.904+0000 7ff150ff9640 1 -- 172.21.15.43:0/2090862876 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1600a6780 con 0x561886e3e6b0 2024-04-25T10:42:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.904+0000 7ff150ff9640 1 -- 172.21.15.43:0/2090862876 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1600a2f00 con 0x561886e3e6b0 2024-04-25T10:42:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.904+0000 7ff178573740 1 -- 172.21.15.43:0/2090862876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e3e6b0 msgr2=0x5618871fced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.904+0000 7ff178573740 1 --2- 172.21.15.43:0/2090862876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e3e6b0 0x5618871fced0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7ff1600dc280 tx=0x7ff1600036f0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.904+0000 7ff178573740 1 -- 172.21.15.43:0/2090862876 shutdown_connections 2024-04-25T10:42:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.904+0000 7ff178573740 1 --2- 172.21.15.43:0/2090862876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561886e5f390 0x56188720c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.905+0000 7ff178573740 1 --2- 172.21.15.43:0/2090862876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e3e6b0 0x5618871fced0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.905+0000 7ff178573740 1 --2- 172.21.15.43:0/2090862876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e364d0 0x56188721b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.905+0000 7ff178573740 1 -- 172.21.15.43:0/2090862876 >> 172.21.15.43:0/2090862876 conn(0x561886e534e0 msgr2=0x5618871fc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.905+0000 7ff178573740 1 -- 172.21.15.43:0/2090862876 shutdown_connections 2024-04-25T10:42:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.905+0000 7ff178573740 1 -- 172.21.15.43:0/2090862876 wait complete. 2024-04-25T10:42:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.905+0000 7ff178573740 1 -- start start 2024-04-25T10:42:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.905+0000 7ff178573740 1 Processor -- start 2024-04-25T10:42:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.905+0000 7ff178573740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e364d0 0x561887224ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.905+0000 7ff178573740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561886e3e6b0 0x561887222010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.905+0000 7ff178573740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e5f390 0x561887222550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.905+0000 7ff178573740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c1abf60 con 0x561886e364d0 2024-04-25T10:42:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.905+0000 7ff178573740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c1ad3b0 con 0x561886e3e6b0 2024-04-25T10:42:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.905+0000 7ff178573740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c1ac610 con 0x561886e5f390 2024-04-25T10:42:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.905+0000 7ff178573740 10 client.0 _ll_register_callbacks cb 0x7ff164009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.905+0000 7ff16affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e364d0 0x561887224ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.905+0000 7ff16affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e364d0 0x561887224ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58846/0 (socket says 172.21.15.43:58846) 2024-04-25T10:42:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.905+0000 7ff16affd640 1 -- 172.21.15.43:0/3809539932 learned_addr learned my addr 172.21.15.43:0/3809539932 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.906+0000 7ff16b7fe640 1 --2- 172.21.15.43:0/3809539932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e5f390 0x561887222550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.906+0000 7ff16a7fc640 1 --2- 172.21.15.43:0/3809539932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561886e3e6b0 0x561887222010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.906+0000 7ff16affd640 1 -- 172.21.15.43:0/3809539932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e5f390 msgr2=0x561887222550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.906+0000 7ff16affd640 1 --2- 172.21.15.43:0/3809539932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e5f390 0x561887222550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.906+0000 7ff16affd640 1 -- 172.21.15.43:0/3809539932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561886e3e6b0 msgr2=0x561887222010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.906+0000 7ff16affd640 1 --2- 172.21.15.43:0/3809539932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561886e3e6b0 0x561887222010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.906+0000 7ff16affd640 1 -- 172.21.15.43:0/3809539932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56188720b060 con 0x561886e364d0 2024-04-25T10:42:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.906+0000 7ff16b7fe640 1 --2- 172.21.15.43:0/3809539932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e5f390 0x561887222550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.906+0000 7ff16affd640 1 --2- 172.21.15.43:0/3809539932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e364d0 0x561887224ca0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0315b0 tx=0x7ff15c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1429c715b4f424b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.906+0000 7ff14eff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.906+0000 7ff14eff5640 1 -- 172.21.15.43:0/3809539932 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c035050 con 0x561886e364d0 2024-04-25T10:42:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.906+0000 7ff14eff5640 1 -- 172.21.15.43:0/3809539932 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7ff15c042040 con 0x561886e364d0 2024-04-25T10:42:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.906+0000 7ff178573740 1 -- 172.21.15.43:0/3809539932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1600a6780 con 0x561886e364d0 2024-04-25T10:42:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.906+0000 7ff14eff5640 1 -- 172.21.15.43:0/3809539932 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff15c004250 con 0x561886e364d0 2024-04-25T10:42:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.906+0000 7ff178573740 10 client.18564 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.906+0000 7ff178573740 1 -- 172.21.15.43:0/3809539932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561887289540 con 0x561886e364d0 2024-04-25T10:42:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.906+0000 7ff14eff5640 1 -- 172.21.15.43:0/3809539932 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1600a6780 con 0x561886e364d0 2024-04-25T10:42:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.907+0000 7ff14eff5640 1 -- 172.21.15.43:0/3809539932 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff15c004960 con 0x561886e364d0 2024-04-25T10:42:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.907+0000 7ff178573740 10 client.18564 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.907+0000 7ff178573740 10 client.18564 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:42:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.907+0000 7ff178573740 20 client.18564 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.907+0000 7ff178573740 1 --2- 172.21.15.43:0/3809539932 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x561886df33a0 0x56188728bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.907+0000 7ff178573740 4 client.18564 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:42:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.907+0000 7ff178573740 1 -- 172.21.15.43:0/3809539932 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56188716d800 con 0x561886df33a0 2024-04-25T10:42:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.907+0000 7ff16a7fc640 1 --2- 172.21.15.43:0/3809539932 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x561886df33a0 0x56188728bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.907+0000 7ff14eff5640 10 client.18564 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:42:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.907+0000 7ff16a7fc640 1 --2- 172.21.15.43:0/3809539932 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x561886df33a0 0x56188728bd00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef27104a9ea49c7b server_cookie=b21d74518f0c17f3 in_seq=0 out_seq=0 2024-04-25T10:42:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.907+0000 7ff14eff5640 1 -- 172.21.15.43:0/3809539932 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56188716d800 con 0x561886df33a0 2024-04-25T10:42:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.907+0000 7ff14eff5640 10 client.18564 handle_command_reply: tid=0 2024-04-25T10:42:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.907+0000 7ff178573740 1 client.18564 shutdown 2024-04-25T10:42:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff178573740 1 -- 172.21.15.43:0/3809539932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e364d0 msgr2=0x561887224ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff178573740 1 --2- 172.21.15.43:0/3809539932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e364d0 0x561887224ca0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0315b0 tx=0x7ff15c02a410 comp rx=0 tx=0).stop 2024-04-25T10:42:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff178573740 1 -- 172.21.15.43:0/3809539932 shutdown_connections 2024-04-25T10:42:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff178573740 1 --2- 172.21.15.43:0/3809539932 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x561886df33a0 0x56188728bd00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff178573740 1 --2- 172.21.15.43:0/3809539932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561886e3e6b0 0x561887222010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff14eff5640 0 client.18564 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:42:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff178573740 1 --2- 172.21.15.43:0/3809539932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561886e5f390 0x561887222550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff178573740 1 --2- 172.21.15.43:0/3809539932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561886e364d0 0x561887224ca0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff178573740 1 -- 172.21.15.43:0/3809539932 >> 172.21.15.43:0/3809539932 conn(0x561886e534e0 msgr2=0x5618872023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:55.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff178573740 1 -- 172.21.15.43:0/3809539932 shutdown_connections 2024-04-25T10:42:55.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff178573740 1 -- 172.21.15.43:0/3809539932 wait complete. 2024-04-25T10:42:55.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff178573740 20 client.18564 trim_cache size 0 max 16384 2024-04-25T10:42:55.927 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:42:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff1716bd640 1 -- 172.21.15.43:0/3027106619 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff1540afc60 msgr2=0x7ff1540b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff1716bd640 1 --2- 172.21.15.43:0/3027106619 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff1540afc60 0x7ff1540b2170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff15c000c00 tx=0x7ff15c032d20 comp rx=0 tx=0).stop 2024-04-25T10:42:55.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff1716bd640 1 -- 172.21.15.43:0/3027106619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16c109b10 msgr2=0x7ff16c19ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:55.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff1716bd640 1 --2- 172.21.15.43:0/3027106619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16c109b10 0x7ff16c19ace0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ff16000d740 tx=0x7ff16000dc00 comp rx=0 tx=0).stop 2024-04-25T10:42:55.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff1716bd640 1 -- 172.21.15.43:0/3027106619 shutdown_connections 2024-04-25T10:42:55.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff1716bd640 1 --2- 172.21.15.43:0/3027106619 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7ff1540afc60 0x7ff1540b2170 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff1716bd640 1 --2- 172.21.15.43:0/3027106619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff16c109fb0 0x7ff16c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff1716bd640 1 --2- 172.21.15.43:0/3027106619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16c109b10 0x7ff16c19ace0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff1716bd640 1 --2- 172.21.15.43:0/3027106619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff16c104260 0x7ff16c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:55.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff1716bd640 1 -- 172.21.15.43:0/3027106619 >> 172.21.15.43:0/3027106619 conn(0x7ff16c0ff210 msgr2=0x7ff16c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:55.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff1716bd640 1 -- 172.21.15.43:0/3027106619 shutdown_connections 2024-04-25T10:42:55.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:55.908+0000 7ff1716bd640 1 -- 172.21.15.43:0/3027106619 wait complete. 2024-04-25T10:42:55.932 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:42:56.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.044+0000 7f11d5f8f640 1 -- 172.21.15.43:0/377900088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0108ed0 msgr2=0x7f11d0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:56.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.044+0000 7f11d5f8f640 1 --2- 172.21.15.43:0/377900088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0108ed0 0x7f11d0102b20 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009900 tx=0x7f11c402a410 comp rx=0 tx=0).stop 2024-04-25T10:42:56.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.044+0000 7f11d5f8f640 1 -- 172.21.15.43:0/377900088 shutdown_connections 2024-04-25T10:42:56.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.044+0000 7f11d5f8f640 1 --2- 172.21.15.43:0/377900088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11d0103190 0x7f11d010f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.044+0000 7f11d5f8f640 1 --2- 172.21.15.43:0/377900088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0108ed0 0x7f11d0102b20 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.044+0000 7f11d5f8f640 1 --2- 172.21.15.43:0/377900088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11d0108a30 0x7f11d01025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.044+0000 7f11d5f8f640 1 -- 172.21.15.43:0/377900088 >> 172.21.15.43:0/377900088 conn(0x7f11d00fe260 msgr2=0x7f11d0100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:56.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.044+0000 7f11d5f8f640 1 -- 172.21.15.43:0/377900088 shutdown_connections 2024-04-25T10:42:56.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.044+0000 7f11d5f8f640 1 -- 172.21.15.43:0/377900088 wait complete. 2024-04-25T10:42:56.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.045+0000 7f11d5f8f640 1 Processor -- start 2024-04-25T10:42:56.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.045+0000 7f11d5f8f640 1 -- start start 2024-04-25T10:42:56.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.045+0000 7f11d5f8f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11d0103190 0x7f11d019b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.045+0000 7f11d5f8f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11d0108a30 0x7f11d019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.045+0000 7f11d5f8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0108ed0 0x7f11d019c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.045+0000 7f11d5f8f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11d0112c30 con 0x7f11d0108ed0 2024-04-25T10:42:56.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.045+0000 7f11d5f8f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f11d0112ab0 con 0x7f11d0103190 2024-04-25T10:42:56.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.045+0000 7f11d5f8f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f11d0112db0 con 0x7f11d0108a30 2024-04-25T10:42:56.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.045+0000 7f11cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0108ed0 0x7f11d019c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.045+0000 7f11cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0108ed0 0x7f11d019c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58858/0 (socket says 172.21.15.43:58858) 2024-04-25T10:42:56.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.045+0000 7f11cffff640 1 -- 172.21.15.43:0/2095088391 learned_addr learned my addr 172.21.15.43:0/2095088391 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:56.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.045+0000 7f11cf7fe640 1 --2- 172.21.15.43:0/2095088391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11d0103190 0x7f11d019b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.046+0000 7f11ceffd640 1 --2- 172.21.15.43:0/2095088391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11d0108a30 0x7f11d019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.046+0000 7f11cffff640 1 -- 172.21.15.43:0/2095088391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11d0108a30 msgr2=0x7f11d019bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:56.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.046+0000 7f11cffff640 1 --2- 172.21.15.43:0/2095088391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11d0108a30 0x7f11d019bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.046+0000 7f11cffff640 1 -- 172.21.15.43:0/2095088391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11d0103190 msgr2=0x7f11d019b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:56.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.046+0000 7f11cffff640 1 --2- 172.21.15.43:0/2095088391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11d0103190 0x7f11d019b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.046+0000 7f11cffff640 1 -- 172.21.15.43:0/2095088391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d01a6880 con 0x7f11d0108ed0 2024-04-25T10:42:56.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.046+0000 7f11ceffd640 1 --2- 172.21.15.43:0/2095088391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11d0108a30 0x7f11d019bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:56.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.046+0000 7f11cffff640 1 --2- 172.21.15.43:0/2095088391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0108ed0 0x7f11d019c070 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f11c800cce0 tx=0x7f11c800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:56.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.046+0000 7f11ccff9640 1 -- 172.21.15.43:0/2095088391 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c800f450 con 0x7f11d0108ed0 2024-04-25T10:42:56.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.046+0000 7f11ccff9640 1 -- 172.21.15.43:0/2095088391 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c800faa0 con 0x7f11d0108ed0 2024-04-25T10:42:56.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.046+0000 7f11d5f8f640 1 -- 172.21.15.43:0/2095088391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d01a6b70 con 0x7f11d0108ed0 2024-04-25T10:42:56.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.046+0000 7f11ccff9640 1 -- 172.21.15.43:0/2095088391 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8018b70 con 0x7f11d0108ed0 2024-04-25T10:42:56.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.046+0000 7f11d5f8f640 1 -- 172.21.15.43:0/2095088391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d01a7040 con 0x7f11d0108ed0 2024-04-25T10:42:56.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.049+0000 7f11ccff9640 1 -- 172.21.15.43:0/2095088391 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f11c8018d10 con 0x7f11d0108ed0 2024-04-25T10:42:56.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.049+0000 7f11dce43740 1 Processor -- start 2024-04-25T10:42:56.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.049+0000 7f11dce43740 1 -- start start 2024-04-25T10:42:56.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.049+0000 7f11dce43740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64fd390 0x5604a689e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.049+0000 7f11dce43740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604a64d44d0 0x5604a689ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.049+0000 7f11dce43740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64dc6b0 0x5604a68ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.049+0000 7f11dce43740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5604a6803d80 con 0x5604a64dc6b0 2024-04-25T10:42:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.049+0000 7f11dce43740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5604a67bda20 con 0x5604a64d44d0 2024-04-25T10:42:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.049+0000 7f11dce43740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5604a67ffd40 con 0x5604a64fd390 2024-04-25T10:42:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.049+0000 7f11ccff9640 1 --2- 172.21.15.43:0/2095088391 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f11b80afc60 0x7f11b80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.049+0000 7f11ceffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64dc6b0 0x5604a68ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.049+0000 7f11ceffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64dc6b0 0x5604a68ab6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58872/0 (socket says 172.21.15.43:58872) 2024-04-25T10:42:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.049+0000 7f11ceffd640 1 -- 172.21.15.43:0/355748330 learned_addr learned my addr 172.21.15.43:0/355748330 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.049+0000 7f11cffff640 1 --2- 172.21.15.43:0/2095088391 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f11b80afc60 0x7f11b80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.050+0000 7f11cf7fe640 1 --2- 172.21.15.43:0/355748330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604a64d44d0 0x5604a689ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.050+0000 7f11ceffd640 1 --2- 172.21.15.43:0/355748330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64fd390 0x5604a689e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.050+0000 7f11ccff9640 1 -- 172.21.15.43:0/2095088391 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f11c80df4e0 con 0x7f11d0108ed0 2024-04-25T10:42:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.050+0000 7f11ceffd640 1 -- 172.21.15.43:0/355748330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64fd390 msgr2=0x5604a689e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.050+0000 7f11ceffd640 1 --2- 172.21.15.43:0/355748330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64fd390 0x5604a689e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.050+0000 7f11ceffd640 1 -- 172.21.15.43:0/355748330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604a64d44d0 msgr2=0x5604a689ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.050+0000 7f11ceffd640 1 --2- 172.21.15.43:0/355748330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604a64d44d0 0x5604a689ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.050+0000 7f11ceffd640 1 -- 172.21.15.43:0/355748330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604a67c8c30 con 0x5604a64dc6b0 2024-04-25T10:42:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.050+0000 7f11cf7fe640 1 --2- 172.21.15.43:0/355748330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604a64d44d0 0x5604a689ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.050+0000 7f11ceffd640 1 --2- 172.21.15.43:0/355748330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64dc6b0 0x5604a68ab6a0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009900 tx=0x7f11c4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c757d841931d71e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.050+0000 7f11cffff640 1 --2- 172.21.15.43:0/2095088391 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f11b80afc60 0x7f11b80b2170 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f11c80dcce0 tx=0x7f11c8074ee0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.050+0000 7f11b5ffb640 1 -- 172.21.15.43:0/355748330 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4035040 con 0x5604a64dc6b0 2024-04-25T10:42:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.050+0000 7f11b5ffb640 1 -- 172.21.15.43:0/355748330 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c4004030 con 0x5604a64dc6b0 2024-04-25T10:42:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.050+0000 7f11b5ffb640 1 -- 172.21.15.43:0/355748330 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4003b00 con 0x5604a64dc6b0 2024-04-25T10:42:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.051+0000 7f11dce43740 1 -- 172.21.15.43:0/355748330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64dc6b0 msgr2=0x5604a68ab6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.051+0000 7f11dce43740 1 --2- 172.21.15.43:0/355748330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64dc6b0 0x5604a68ab6a0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009900 tx=0x7f11c4000bc0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.051+0000 7f11dce43740 1 -- 172.21.15.43:0/355748330 shutdown_connections 2024-04-25T10:42:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.051+0000 7f11dce43740 1 --2- 172.21.15.43:0/355748330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604a64d44d0 0x5604a689ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.051+0000 7f11dce43740 1 --2- 172.21.15.43:0/355748330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64dc6b0 0x5604a68ab6a0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.051+0000 7f11dce43740 1 --2- 172.21.15.43:0/355748330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64fd390 0x5604a689e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.051+0000 7f11dce43740 1 -- 172.21.15.43:0/355748330 >> 172.21.15.43:0/355748330 conn(0x5604a64f14e0 msgr2=0x5604a689afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.051+0000 7f11dce43740 1 -- 172.21.15.43:0/355748330 shutdown_connections 2024-04-25T10:42:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.051+0000 7f11dce43740 1 -- 172.21.15.43:0/355748330 wait complete. 2024-04-25T10:42:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.051+0000 7f11dce43740 1 -- start start 2024-04-25T10:42:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.052+0000 7f11dce43740 1 Processor -- start 2024-04-25T10:42:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.052+0000 7f11dce43740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604a64d44d0 0x5604a68a9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.052+0000 7f11dce43740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64dc6b0 0x5604a68a97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.052+0000 7f11dce43740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64fd390 0x5604a68b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.052+0000 7f11dce43740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5604a68abd30 con 0x5604a64fd390 2024-04-25T10:42:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.052+0000 7f11dce43740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5604a678adc0 con 0x5604a64d44d0 2024-04-25T10:42:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.052+0000 7f11dce43740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5604a68abeb0 con 0x5604a64dc6b0 2024-04-25T10:42:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.052+0000 7f11dce43740 10 client.0 _ll_register_callbacks cb 0x5604a67a6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.052+0000 7f11ceffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64fd390 0x5604a68b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.052+0000 7f11ceffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64fd390 0x5604a68b51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58886/0 (socket says 172.21.15.43:58886) 2024-04-25T10:42:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.052+0000 7f11ceffd640 1 -- 172.21.15.43:0/2144873083 learned_addr learned my addr 172.21.15.43:0/2144873083 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.052+0000 7f11cf7fe640 1 --2- 172.21.15.43:0/2144873083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64dc6b0 0x5604a68a97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.052+0000 7f11ceffd640 1 --2- 172.21.15.43:0/2144873083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604a64d44d0 0x5604a68a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.053+0000 7f11ceffd640 1 -- 172.21.15.43:0/2144873083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64dc6b0 msgr2=0x5604a68a97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.053+0000 7f11ceffd640 1 --2- 172.21.15.43:0/2144873083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64dc6b0 0x5604a68a97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.053+0000 7f11ceffd640 1 -- 172.21.15.43:0/2144873083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604a64d44d0 msgr2=0x5604a68a9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.053+0000 7f11ceffd640 1 --2- 172.21.15.43:0/2144873083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604a64d44d0 0x5604a68a9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.053+0000 7f11ceffd640 1 -- 172.21.15.43:0/2144873083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604a65153f0 con 0x5604a64fd390 2024-04-25T10:42:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.053+0000 7f11ceffd640 1 --2- 172.21.15.43:0/2144873083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64fd390 0x5604a68b51f0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x5604a68a88b0 tx=0x7f11c40037e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610baf96883c70cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.053+0000 7f11aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.053+0000 7f11aeffd640 1 -- 172.21.15.43:0/2144873083 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4035030 con 0x5604a64fd390 2024-04-25T10:42:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.053+0000 7f11aeffd640 1 -- 172.21.15.43:0/2144873083 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f11c4044030 con 0x5604a64fd390 2024-04-25T10:42:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.053+0000 7f11dce43740 1 -- 172.21.15.43:0/2144873083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11c4004030 con 0x5604a64fd390 2024-04-25T10:42:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.053+0000 7f11aeffd640 1 -- 172.21.15.43:0/2144873083 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11c4031850 con 0x5604a64fd390 2024-04-25T10:42:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.053+0000 7f11dce43740 10 client.18576 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.053+0000 7f11dce43740 1 -- 172.21.15.43:0/2144873083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5604a65df7d0 con 0x5604a64fd390 2024-04-25T10:42:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.053+0000 7f11aeffd640 1 -- 172.21.15.43:0/2144873083 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c4004030 con 0x5604a64fd390 2024-04-25T10:42:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.054+0000 7f11aeffd640 1 -- 172.21.15.43:0/2144873083 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f11c403a040 con 0x5604a64fd390 2024-04-25T10:42:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.054+0000 7f11dce43740 10 client.18576 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.054+0000 7f11dce43740 10 client.18576 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:42:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.054+0000 7f11dce43740 20 client.18576 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.054+0000 7f11dce43740 1 --2- 172.21.15.43:0/2144873083 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5604a64bbf60 0x5604a68bb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.054+0000 7f11dce43740 4 client.18576 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:42:56.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.054+0000 7f11dce43740 1 -- 172.21.15.43:0/2144873083 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604a6811990 con 0x5604a64bbf60 2024-04-25T10:42:56.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.054+0000 7f11cf7fe640 1 --2- 172.21.15.43:0/2144873083 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5604a64bbf60 0x5604a68bb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.054+0000 7f11cf7fe640 1 --2- 172.21.15.43:0/2144873083 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5604a64bbf60 0x5604a68bb4d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0aac69cf50282ca server_cookie=a0038bfe91ac6464 in_seq=0 out_seq=0 2024-04-25T10:42:56.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.054+0000 7f11aeffd640 10 client.18576 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:42:56.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.055+0000 7f11aeffd640 1 -- 172.21.15.43:0/2144873083 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5604a6811990 con 0x5604a64bbf60 2024-04-25T10:42:56.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.055+0000 7f11aeffd640 10 client.18576 handle_command_reply: tid=0 2024-04-25T10:42:56.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.055+0000 7f11dce43740 1 client.18576 shutdown 2024-04-25T10:42:56.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.055+0000 7f11dce43740 1 -- 172.21.15.43:0/2144873083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64fd390 msgr2=0x5604a68b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:56.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.055+0000 7f11dce43740 1 --2- 172.21.15.43:0/2144873083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64fd390 0x5604a68b51f0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x5604a68a88b0 tx=0x7f11c40037e0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.055+0000 7f11dce43740 1 -- 172.21.15.43:0/2144873083 shutdown_connections 2024-04-25T10:42:56.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.055+0000 7f11dce43740 1 --2- 172.21.15.43:0/2144873083 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5604a64bbf60 0x5604a68bb4d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.055+0000 7f11dce43740 1 --2- 172.21.15.43:0/2144873083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64fd390 0x5604a68b51f0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.055+0000 7f11aeffd640 0 client.18576 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:42:56.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.055+0000 7f11dce43740 1 --2- 172.21.15.43:0/2144873083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64dc6b0 0x5604a68a97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.055+0000 7f11dce43740 1 --2- 172.21.15.43:0/2144873083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604a64d44d0 0x5604a68a9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.055+0000 7f11dce43740 1 -- 172.21.15.43:0/2144873083 >> 172.21.15.43:0/2144873083 conn(0x5604a64f14e0 msgr2=0x5604a68a0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:56.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.055+0000 7f11dce43740 1 -- 172.21.15.43:0/2144873083 shutdown_connections 2024-04-25T10:42:56.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.055+0000 7f11dce43740 1 -- 172.21.15.43:0/2144873083 wait complete. 2024-04-25T10:42:56.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.056+0000 7f11dce43740 20 client.18576 trim_cache size 0 max 16384 2024-04-25T10:42:56.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.069+0000 7f11dce43740 1 Processor -- start 2024-04-25T10:42:56.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.069+0000 7f11dce43740 1 -- start start 2024-04-25T10:42:56.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.069+0000 7f11dce43740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64bbf60 0x5604a68a8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.069+0000 7f11dce43740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64d44d0 0x5604a68b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.069+0000 7f11dce43740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604a64dc6b0 0x5604a6899d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.069+0000 7f11dce43740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11d01b2110 con 0x5604a64bbf60 2024-04-25T10:42:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.069+0000 7f11dce43740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f11d01af9e0 con 0x5604a64dc6b0 2024-04-25T10:42:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.069+0000 7f11dce43740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f11d01ad850 con 0x5604a64d44d0 2024-04-25T10:42:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.070+0000 7f11ceffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64bbf60 0x5604a68a8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.070+0000 7f11ceffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64bbf60 0x5604a68a8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58890/0 (socket says 172.21.15.43:58890) 2024-04-25T10:42:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.070+0000 7f11ceffd640 1 -- 172.21.15.43:0/937287714 learned_addr learned my addr 172.21.15.43:0/937287714 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.070+0000 7f11ceffd640 1 -- 172.21.15.43:0/937287714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64d44d0 msgr2=0x5604a68b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.070+0000 7f11cf7fe640 1 --2- 172.21.15.43:0/937287714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64d44d0 0x5604a68b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.070+0000 7f11ceffd640 1 --2- 172.21.15.43:0/937287714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64d44d0 0x5604a68b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.070+0000 7f11ceffd640 1 -- 172.21.15.43:0/937287714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604a64dc6b0 msgr2=0x5604a6899d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.070+0000 7f11ceffd640 1 --2- 172.21.15.43:0/937287714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604a64dc6b0 0x5604a6899d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.070+0000 7f11ceffd640 1 -- 172.21.15.43:0/937287714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604a68a75a0 con 0x5604a64bbf60 2024-04-25T10:42:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.070+0000 7f11cf7fe640 1 --2- 172.21.15.43:0/937287714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64d44d0 0x5604a68b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.070+0000 7f11ceffd640 1 --2- 172.21.15.43:0/937287714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64bbf60 0x5604a68a8350 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f11c405efd0 tx=0x7f11c40058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737f3a59fda4aabb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.070+0000 7f11b57fa640 1 -- 172.21.15.43:0/937287714 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4035040 con 0x5604a64bbf60 2024-04-25T10:42:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.070+0000 7f11b57fa640 1 -- 172.21.15.43:0/937287714 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c40035f0 con 0x5604a64bbf60 2024-04-25T10:42:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.070+0000 7f11b57fa640 1 -- 172.21.15.43:0/937287714 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4031cb0 con 0x5604a64bbf60 2024-04-25T10:42:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.071+0000 7f11dce43740 1 -- 172.21.15.43:0/937287714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64bbf60 msgr2=0x5604a68a8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.071+0000 7f11dce43740 1 --2- 172.21.15.43:0/937287714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64bbf60 0x5604a68a8350 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f11c405efd0 tx=0x7f11c40058b0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.071+0000 7f11dce43740 1 -- 172.21.15.43:0/937287714 shutdown_connections 2024-04-25T10:42:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.071+0000 7f11dce43740 1 --2- 172.21.15.43:0/937287714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604a64dc6b0 0x5604a6899d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.071+0000 7f11dce43740 1 --2- 172.21.15.43:0/937287714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64d44d0 0x5604a68b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.071+0000 7f11dce43740 1 --2- 172.21.15.43:0/937287714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64bbf60 0x5604a68a8350 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.071+0000 7f11dce43740 1 -- 172.21.15.43:0/937287714 >> 172.21.15.43:0/937287714 conn(0x5604a64f14e0 msgr2=0x5604a6899610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.071+0000 7f11dce43740 1 -- 172.21.15.43:0/937287714 shutdown_connections 2024-04-25T10:42:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.071+0000 7f11dce43740 1 -- 172.21.15.43:0/937287714 wait complete. 2024-04-25T10:42:56.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.071+0000 7f11dce43740 1 -- start start 2024-04-25T10:42:56.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.072+0000 7f11dce43740 1 Processor -- start 2024-04-25T10:42:56.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.072+0000 7f11dce43740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604a64bbf60 0x5604a691ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.072+0000 7f11dce43740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64d44d0 0x5604a691bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.072+0000 7f11dce43740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64dc6b0 0x5604a68bb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.072+0000 7f11dce43740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11d01adea0 con 0x5604a64d44d0 2024-04-25T10:42:56.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.072+0000 7f11dce43740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f11d01af310 con 0x5604a64bbf60 2024-04-25T10:42:56.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.072+0000 7f11dce43740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f11d01ae570 con 0x5604a64dc6b0 2024-04-25T10:42:56.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.072+0000 7f11dce43740 10 client.0 _ll_register_callbacks cb 0x7f11c4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:56.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.072+0000 7f11cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64d44d0 0x5604a691bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.072+0000 7f11cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64d44d0 0x5604a691bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58904/0 (socket says 172.21.15.43:58904) 2024-04-25T10:42:56.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.072+0000 7f11cf7fe640 1 -- 172.21.15.43:0/1272994430 learned_addr learned my addr 172.21.15.43:0/1272994430 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:56.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.072+0000 7f11ceffd640 1 --2- 172.21.15.43:0/1272994430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604a64bbf60 0x5604a691ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.072+0000 7f11ceffd640 1 --2- 172.21.15.43:0/1272994430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64dc6b0 0x5604a68bb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.072+0000 7f11cf7fe640 1 -- 172.21.15.43:0/1272994430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64dc6b0 msgr2=0x5604a68bb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.072+0000 7f11cf7fe640 1 --2- 172.21.15.43:0/1272994430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64dc6b0 0x5604a68bb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.072+0000 7f11cf7fe640 1 -- 172.21.15.43:0/1272994430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604a64bbf60 msgr2=0x5604a691ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.072+0000 7f11cf7fe640 1 --2- 172.21.15.43:0/1272994430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604a64bbf60 0x5604a691ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.072+0000 7f11cf7fe640 1 -- 172.21.15.43:0/1272994430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604a68bbfb0 con 0x5604a64d44d0 2024-04-25T10:42:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.073+0000 7f11cf7fe640 1 --2- 172.21.15.43:0/1272994430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64d44d0 0x5604a691bf40 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f11d006c4a0 tx=0x7f11c0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bd67cfbce28fe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.073+0000 7f11aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.073+0000 7f11aeffd640 1 -- 172.21.15.43:0/1272994430 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c0023070 con 0x5604a64d44d0 2024-04-25T10:42:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.073+0000 7f11dce43740 1 -- 172.21.15.43:0/1272994430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11c40035f0 con 0x5604a64d44d0 2024-04-25T10:42:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.073+0000 7f11aeffd640 1 -- 172.21.15.43:0/1272994430 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f11c0004460 con 0x5604a64d44d0 2024-04-25T10:42:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.073+0000 7f11aeffd640 1 -- 172.21.15.43:0/1272994430 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11c002f050 con 0x5604a64d44d0 2024-04-25T10:42:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.073+0000 7f11dce43740 10 client.18582 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:56.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.073+0000 7f11dce43740 1 -- 172.21.15.43:0/1272994430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5604a69230a0 con 0x5604a64d44d0 2024-04-25T10:42:56.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.073+0000 7f11aeffd640 1 -- 172.21.15.43:0/1272994430 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c40035f0 con 0x5604a64d44d0 2024-04-25T10:42:56.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.073+0000 7f11aeffd640 1 -- 172.21.15.43:0/1272994430 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f11c0003ae0 con 0x5604a64d44d0 2024-04-25T10:42:56.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.073+0000 7f11dce43740 10 client.18582 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:56.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.073+0000 7f11dce43740 10 client.18582 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:42:56.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.073+0000 7f11dce43740 20 client.18582 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:56.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.073+0000 7f11dce43740 1 --2- 172.21.15.43:0/1272994430 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5604a64fd390 0x5604a6925840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.073+0000 7f11dce43740 4 client.18582 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:42:56.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.073+0000 7f11dce43740 1 -- 172.21.15.43:0/1272994430 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5604a680b800 con 0x5604a64fd390 2024-04-25T10:42:56.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.074+0000 7f11ceffd640 1 --2- 172.21.15.43:0/1272994430 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5604a64fd390 0x5604a6925840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.074+0000 7f11ceffd640 1 --2- 172.21.15.43:0/1272994430 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5604a64fd390 0x5604a6925840 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1734603a829c0e7 server_cookie=169d6be997ba06d8 in_seq=0 out_seq=0 2024-04-25T10:42:56.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.074+0000 7f11aeffd640 10 client.18582 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:42:56.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.074+0000 7f11aeffd640 1 -- 172.21.15.43:0/1272994430 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5604a680b800 con 0x5604a64fd390 2024-04-25T10:42:56.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.074+0000 7f11aeffd640 10 client.18582 handle_command_reply: tid=0 2024-04-25T10:42:56.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.074+0000 7f11dce43740 1 client.18582 shutdown 2024-04-25T10:42:56.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.074+0000 7f11dce43740 1 -- 172.21.15.43:0/1272994430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64d44d0 msgr2=0x5604a691bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:56.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.074+0000 7f11dce43740 1 --2- 172.21.15.43:0/1272994430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64d44d0 0x5604a691bf40 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f11d006c4a0 tx=0x7f11c0010800 comp rx=0 tx=0).stop 2024-04-25T10:42:56.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.074+0000 7f11dce43740 1 -- 172.21.15.43:0/1272994430 shutdown_connections 2024-04-25T10:42:56.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.074+0000 7f11dce43740 1 --2- 172.21.15.43:0/1272994430 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x5604a64fd390 0x5604a6925840 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.074+0000 7f11dce43740 1 --2- 172.21.15.43:0/1272994430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604a64dc6b0 0x5604a68bb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.074+0000 7f11dce43740 1 --2- 172.21.15.43:0/1272994430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604a64d44d0 0x5604a691bf40 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.074+0000 7f11dce43740 1 --2- 172.21.15.43:0/1272994430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604a64bbf60 0x5604a691ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.074+0000 7f11aeffd640 0 client.18582 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:42:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.074+0000 7f11dce43740 1 -- 172.21.15.43:0/1272994430 >> 172.21.15.43:0/1272994430 conn(0x5604a64f14e0 msgr2=0x5604a68a80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.075+0000 7f11dce43740 1 -- 172.21.15.43:0/1272994430 shutdown_connections 2024-04-25T10:42:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.075+0000 7f11dce43740 1 -- 172.21.15.43:0/1272994430 wait complete. 2024-04-25T10:42:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.075+0000 7f11dce43740 20 client.18582 trim_cache size 0 max 16384 2024-04-25T10:42:56.085 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:42:56.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.075+0000 7f11d5f8f640 1 -- 172.21.15.43:0/2095088391 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f11b80afc60 msgr2=0x7f11b80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:56.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.075+0000 7f11d5f8f640 1 --2- 172.21.15.43:0/2095088391 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f11b80afc60 0x7f11b80b2170 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f11c80dcce0 tx=0x7f11c8074ee0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.075+0000 7f11d5f8f640 1 -- 172.21.15.43:0/2095088391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0108ed0 msgr2=0x7f11d019c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:56.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.075+0000 7f11d5f8f640 1 --2- 172.21.15.43:0/2095088391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0108ed0 0x7f11d019c070 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f11c800cce0 tx=0x7f11c800a620 comp rx=0 tx=0).stop 2024-04-25T10:42:56.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.075+0000 7f11d5f8f640 1 -- 172.21.15.43:0/2095088391 shutdown_connections 2024-04-25T10:42:56.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.075+0000 7f11d5f8f640 1 --2- 172.21.15.43:0/2095088391 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f11b80afc60 0x7f11b80b2170 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.075+0000 7f11d5f8f640 1 --2- 172.21.15.43:0/2095088391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0108ed0 0x7f11d019c070 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.075+0000 7f11d5f8f640 1 --2- 172.21.15.43:0/2095088391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11d0108a30 0x7f11d019bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.075+0000 7f11d5f8f640 1 --2- 172.21.15.43:0/2095088391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11d0103190 0x7f11d019b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.075+0000 7f11d5f8f640 1 -- 172.21.15.43:0/2095088391 >> 172.21.15.43:0/2095088391 conn(0x7f11d00fe260 msgr2=0x7f11d00fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:56.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.075+0000 7f11d5f8f640 1 -- 172.21.15.43:0/2095088391 shutdown_connections 2024-04-25T10:42:56.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.075+0000 7f11d5f8f640 1 -- 172.21.15.43:0/2095088391 wait complete. 2024-04-25T10:42:56.089 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:42:56.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.202+0000 7f0712851640 1 -- 172.21.15.43:0/3360792741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f070c108ed0 msgr2=0x7f070c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:56.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.202+0000 7f0712851640 1 --2- 172.21.15.43:0/3360792741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f070c108ed0 0x7f070c102b20 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f0700009900 tx=0x7f070002a410 comp rx=0 tx=0).stop 2024-04-25T10:42:56.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.202+0000 7f0712851640 1 -- 172.21.15.43:0/3360792741 shutdown_connections 2024-04-25T10:42:56.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.202+0000 7f0712851640 1 --2- 172.21.15.43:0/3360792741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f070c103190 0x7f070c10f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.202+0000 7f0712851640 1 --2- 172.21.15.43:0/3360792741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f070c108ed0 0x7f070c102b20 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.202+0000 7f0712851640 1 --2- 172.21.15.43:0/3360792741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f070c108a30 0x7f070c1025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.202+0000 7f0712851640 1 -- 172.21.15.43:0/3360792741 >> 172.21.15.43:0/3360792741 conn(0x7f070c0fe260 msgr2=0x7f070c100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:56.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.202+0000 7f0712851640 1 -- 172.21.15.43:0/3360792741 shutdown_connections 2024-04-25T10:42:56.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.202+0000 7f0712851640 1 -- 172.21.15.43:0/3360792741 wait complete. 2024-04-25T10:42:56.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.202+0000 7f0712851640 1 Processor -- start 2024-04-25T10:42:56.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.202+0000 7f0712851640 1 -- start start 2024-04-25T10:42:56.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.203+0000 7f0712851640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f070c103190 0x7f070c19b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.203+0000 7f0712851640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f070c108a30 0x7f070c19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.203+0000 7f0712851640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f070c108ed0 0x7f070c19c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.203+0000 7f0712851640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f070c112c30 con 0x7f070c108ed0 2024-04-25T10:42:56.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.203+0000 7f0712851640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f070c112ab0 con 0x7f070c103190 2024-04-25T10:42:56.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.203+0000 7f0712851640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f070c112db0 con 0x7f070c108a30 2024-04-25T10:42:56.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.203+0000 7f0710dee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f070c108ed0 0x7f070c19c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.203+0000 7f0710dee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f070c108ed0 0x7f070c19c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58916/0 (socket says 172.21.15.43:58916) 2024-04-25T10:42:56.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.203+0000 7f0710dee640 1 -- 172.21.15.43:0/4019257397 learned_addr learned my addr 172.21.15.43:0/4019257397 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:56.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.203+0000 7f070b7fe640 1 --2- 172.21.15.43:0/4019257397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f070c108a30 0x7f070c19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.203+0000 7f070bfff640 1 --2- 172.21.15.43:0/4019257397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f070c103190 0x7f070c19b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.203+0000 7f0710dee640 1 -- 172.21.15.43:0/4019257397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f070c108a30 msgr2=0x7f070c19bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:56.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.203+0000 7f0710dee640 1 --2- 172.21.15.43:0/4019257397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f070c108a30 0x7f070c19bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.203+0000 7f0710dee640 1 -- 172.21.15.43:0/4019257397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f070c103190 msgr2=0x7f070c19b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:56.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.203+0000 7f0710dee640 1 --2- 172.21.15.43:0/4019257397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f070c103190 0x7f070c19b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.203+0000 7f0710dee640 1 -- 172.21.15.43:0/4019257397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070c1a6850 con 0x7f070c108ed0 2024-04-25T10:42:56.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.203+0000 7f070b7fe640 1 --2- 172.21.15.43:0/4019257397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f070c108a30 0x7f070c19bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:56.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.204+0000 7f0710dee640 1 --2- 172.21.15.43:0/4019257397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f070c108ed0 0x7f070c19c050 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f070400cce0 tx=0x7f070400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:56.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.204+0000 7f07097fa640 1 -- 172.21.15.43:0/4019257397 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070400f450 con 0x7f070c108ed0 2024-04-25T10:42:56.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.204+0000 7f07097fa640 1 -- 172.21.15.43:0/4019257397 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f070400faa0 con 0x7f070c108ed0 2024-04-25T10:42:56.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.204+0000 7f0712851640 1 -- 172.21.15.43:0/4019257397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f070c1a6b40 con 0x7f070c108ed0 2024-04-25T10:42:56.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.204+0000 7f07097fa640 1 -- 172.21.15.43:0/4019257397 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704018b70 con 0x7f070c108ed0 2024-04-25T10:42:56.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.204+0000 7f0712851640 1 -- 172.21.15.43:0/4019257397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f070c1a7010 con 0x7f070c108ed0 2024-04-25T10:42:56.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.206+0000 7f07197e9740 1 Processor -- start 2024-04-25T10:42:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.206+0000 7f07097fa640 1 -- 172.21.15.43:0/4019257397 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f0704018d10 con 0x7f070c108ed0 2024-04-25T10:42:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.206+0000 7f07197e9740 1 -- start start 2024-04-25T10:42:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.207+0000 7f07197e9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586d6c89390 0x5586d702a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.207+0000 7f07197e9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c604d0 0x5586d702ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.207+0000 7f07197e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c686b0 0x5586d70376a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.207+0000 7f07197e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5586d6f8fd80 con 0x5586d6c686b0 2024-04-25T10:42:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.207+0000 7f07197e9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5586d6f49a20 con 0x5586d6c604d0 2024-04-25T10:42:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.207+0000 7f07197e9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5586d6f8bd40 con 0x5586d6c89390 2024-04-25T10:42:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.207+0000 7f070b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c686b0 0x5586d70376a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.207+0000 7f070b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c686b0 0x5586d70376a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58918/0 (socket says 172.21.15.43:58918) 2024-04-25T10:42:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.207+0000 7f070b7fe640 1 -- 172.21.15.43:0/342185343 learned_addr learned my addr 172.21.15.43:0/342185343 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.207+0000 7f070bfff640 1 --2- 172.21.15.43:0/342185343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c604d0 0x5586d702ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.207+0000 7f070b7fe640 1 --2- 172.21.15.43:0/342185343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586d6c89390 0x5586d702a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.207+0000 7f07097fa640 1 --2- 172.21.15.43:0/4019257397 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f06f40afc60 0x7f06f40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.207+0000 7f070b7fe640 1 -- 172.21.15.43:0/342185343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586d6c89390 msgr2=0x5586d702a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:56.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.207+0000 7f070b7fe640 1 --2- 172.21.15.43:0/342185343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586d6c89390 0x5586d702a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.207+0000 7f070b7fe640 1 -- 172.21.15.43:0/342185343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c604d0 msgr2=0x5586d702ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:56.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.207+0000 7f070b7fe640 1 --2- 172.21.15.43:0/342185343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c604d0 0x5586d702ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.207+0000 7f070b7fe640 1 -- 172.21.15.43:0/342185343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d6f54c30 con 0x5586d6c686b0 2024-04-25T10:42:56.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.207+0000 7f070bfff640 1 --2- 172.21.15.43:0/342185343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c604d0 0x5586d702ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:42:56.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.207+0000 7f0710dee640 1 --2- 172.21.15.43:0/4019257397 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f06f40afc60 0x7f06f40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.207+0000 7f070b7fe640 1 --2- 172.21.15.43:0/342185343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c686b0 0x5586d70376a0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f0700032fd0 tx=0x7f0700000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324ec39cd8df9ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:56.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.208+0000 7f07097fa640 1 -- 172.21.15.43:0/4019257397 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f07040df590 con 0x7f070c108ed0 2024-04-25T10:42:56.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.208+0000 7f06f27fc640 1 -- 172.21.15.43:0/342185343 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700035040 con 0x5586d6c686b0 2024-04-25T10:42:56.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.208+0000 7f06f27fc640 1 -- 172.21.15.43:0/342185343 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07000035f0 con 0x5586d6c686b0 2024-04-25T10:42:56.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.208+0000 7f06f27fc640 1 -- 172.21.15.43:0/342185343 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07000044e0 con 0x5586d6c686b0 2024-04-25T10:42:56.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.208+0000 7f07197e9740 1 -- 172.21.15.43:0/342185343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c686b0 msgr2=0x5586d70376a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:56.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.208+0000 7f0710dee640 1 --2- 172.21.15.43:0/4019257397 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f06f40afc60 0x7f06f40b2170 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f07040a49a0 tx=0x7f0704074f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:56.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.208+0000 7f07197e9740 1 --2- 172.21.15.43:0/342185343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c686b0 0x5586d70376a0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f0700032fd0 tx=0x7f0700000bc0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.209+0000 7f07197e9740 1 -- 172.21.15.43:0/342185343 shutdown_connections 2024-04-25T10:42:56.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.209+0000 7f07197e9740 1 --2- 172.21.15.43:0/342185343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c604d0 0x5586d702ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.209+0000 7f07197e9740 1 --2- 172.21.15.43:0/342185343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c686b0 0x5586d70376a0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.209+0000 7f07197e9740 1 --2- 172.21.15.43:0/342185343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586d6c89390 0x5586d702a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.209+0000 7f07197e9740 1 -- 172.21.15.43:0/342185343 >> 172.21.15.43:0/342185343 conn(0x5586d6c7d4e0 msgr2=0x5586d7026fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:56.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.209+0000 7f07197e9740 1 -- 172.21.15.43:0/342185343 shutdown_connections 2024-04-25T10:42:56.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.209+0000 7f07197e9740 1 -- 172.21.15.43:0/342185343 wait complete. 2024-04-25T10:42:56.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.209+0000 7f07197e9740 1 -- start start 2024-04-25T10:42:56.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.209+0000 7f07197e9740 1 Processor -- start 2024-04-25T10:42:56.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f07197e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c604d0 0x5586d7035260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f07197e9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c686b0 0x5586d70357a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f07197e9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586d6c89390 0x5586d70411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f07197e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5586d7037d30 con 0x5586d6c604d0 2024-04-25T10:42:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f07197e9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5586d6f16dc0 con 0x5586d6c686b0 2024-04-25T10:42:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f07197e9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5586d7037eb0 con 0x5586d6c89390 2024-04-25T10:42:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f07197e9740 10 client.0 _ll_register_callbacks cb 0x5586d6f32600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f070b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c604d0 0x5586d7035260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f070b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c604d0 0x5586d7035260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58932/0 (socket says 172.21.15.43:58932) 2024-04-25T10:42:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f070b7fe640 1 -- 172.21.15.43:0/29749360 learned_addr learned my addr 172.21.15.43:0/29749360 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f070bfff640 1 --2- 172.21.15.43:0/29749360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c686b0 0x5586d70357a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f070b7fe640 1 --2- 172.21.15.43:0/29749360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586d6c89390 0x5586d70411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f070b7fe640 1 -- 172.21.15.43:0/29749360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586d6c89390 msgr2=0x5586d70411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f070b7fe640 1 --2- 172.21.15.43:0/29749360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586d6c89390 0x5586d70411f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f070b7fe640 1 -- 172.21.15.43:0/29749360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c686b0 msgr2=0x5586d70357a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f070b7fe640 1 --2- 172.21.15.43:0/29749360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c686b0 0x5586d70357a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f070b7fe640 1 -- 172.21.15.43:0/29749360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d6ca13f0 con 0x5586d6c604d0 2024-04-25T10:42:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f070b7fe640 1 --2- 172.21.15.43:0/29749360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c604d0 0x5586d7035260 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f0700001eb0 tx=0x7f0700003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880e301316b5f598 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f06eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f06eb7fe640 1 -- 172.21.15.43:0/29749360 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700035070 con 0x5586d6c604d0 2024-04-25T10:42:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f06eb7fe640 1 -- 172.21.15.43:0/29749360 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f07000039a0 con 0x5586d6c604d0 2024-04-25T10:42:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.210+0000 7f07197e9740 1 -- 172.21.15.43:0/29749360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07000035f0 con 0x5586d6c604d0 2024-04-25T10:42:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.211+0000 7f06eb7fe640 1 -- 172.21.15.43:0/29749360 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0700003ba0 con 0x5586d6c604d0 2024-04-25T10:42:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.211+0000 7f07197e9740 10 client.18594 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.211+0000 7f07197e9740 1 -- 172.21.15.43:0/29749360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5586d6d6b7d0 con 0x5586d6c604d0 2024-04-25T10:42:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.211+0000 7f06eb7fe640 1 -- 172.21.15.43:0/29749360 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07000035f0 con 0x5586d6c604d0 2024-04-25T10:42:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.211+0000 7f06eb7fe640 1 -- 172.21.15.43:0/29749360 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0700052030 con 0x5586d6c604d0 2024-04-25T10:42:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.211+0000 7f07197e9740 10 client.18594 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.211+0000 7f07197e9740 10 client.18594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.211+0000 7f07197e9740 20 client.18594 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.211+0000 7f07197e9740 1 --2- 172.21.15.43:0/29749360 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5586d6c47f60 0x5586d70474d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.211+0000 7f07197e9740 4 client.18594 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:42:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.211+0000 7f07197e9740 1 -- 172.21.15.43:0/29749360 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586d6f9d990 con 0x5586d6c47f60 2024-04-25T10:42:56.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.211+0000 7f070bfff640 1 --2- 172.21.15.43:0/29749360 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5586d6c47f60 0x5586d70474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.212+0000 7f06eb7fe640 10 client.18594 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:56.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.212+0000 7f070bfff640 1 --2- 172.21.15.43:0/29749360 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5586d6c47f60 0x5586d70474d0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b36f95a6f7d87c85 server_cookie=9bffe23b08be019d in_seq=0 out_seq=0 2024-04-25T10:42:56.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.213+0000 7f06eb7fe640 1 -- 172.21.15.43:0/29749360 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5586d6f9d990 con 0x5586d6c47f60 2024-04-25T10:42:56.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.213+0000 7f06eb7fe640 10 client.18594 handle_command_reply: tid=0 2024-04-25T10:42:56.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.213+0000 7f07197e9740 1 client.18594 shutdown 2024-04-25T10:42:56.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.213+0000 7f07197e9740 1 -- 172.21.15.43:0/29749360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c604d0 msgr2=0x5586d7035260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:56.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.213+0000 7f07197e9740 1 --2- 172.21.15.43:0/29749360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c604d0 0x5586d7035260 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f0700001eb0 tx=0x7f0700003ef0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.213+0000 7f07197e9740 1 -- 172.21.15.43:0/29749360 shutdown_connections 2024-04-25T10:42:56.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.213+0000 7f07197e9740 1 --2- 172.21.15.43:0/29749360 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5586d6c47f60 0x5586d70474d0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.213+0000 7f07197e9740 1 --2- 172.21.15.43:0/29749360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c686b0 0x5586d70357a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.213+0000 7f07197e9740 1 --2- 172.21.15.43:0/29749360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586d6c89390 0x5586d70411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.213+0000 7f06eb7fe640 0 client.18594 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:56.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.213+0000 7f07197e9740 1 --2- 172.21.15.43:0/29749360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c604d0 0x5586d7035260 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.213+0000 7f07197e9740 1 -- 172.21.15.43:0/29749360 >> 172.21.15.43:0/29749360 conn(0x5586d6c7d4e0 msgr2=0x5586d702c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:56.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.213+0000 7f07197e9740 1 -- 172.21.15.43:0/29749360 shutdown_connections 2024-04-25T10:42:56.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.213+0000 7f07197e9740 1 -- 172.21.15.43:0/29749360 wait complete. 2024-04-25T10:42:56.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.213+0000 7f07197e9740 20 client.18594 trim_cache size 0 max 16384 2024-04-25T10:42:56.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.227+0000 7f07197e9740 1 Processor -- start 2024-04-25T10:42:56.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.227+0000 7f07197e9740 1 -- start start 2024-04-25T10:42:56.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.227+0000 7f07197e9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c604d0 0x5586d7034350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.227+0000 7f07197e9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586d6c686b0 0x5586d70411a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.227+0000 7f07197e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c89390 0x5586d7036e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.227+0000 7f07197e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f070c1b2220 con 0x5586d6c89390 2024-04-25T10:42:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.227+0000 7f07197e9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f070c1afaf0 con 0x5586d6c604d0 2024-04-25T10:42:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.227+0000 7f07197e9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f070c1ad960 con 0x5586d6c686b0 2024-04-25T10:42:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.227+0000 7f070b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c89390 0x5586d7036e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.227+0000 7f070b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c89390 0x5586d7036e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58942/0 (socket says 172.21.15.43:58942) 2024-04-25T10:42:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.227+0000 7f070b7fe640 1 -- 172.21.15.43:0/4278411735 learned_addr learned my addr 172.21.15.43:0/4278411735 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.227+0000 7f070bfff640 1 --2- 172.21.15.43:0/4278411735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586d6c686b0 0x5586d70411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.227+0000 7f070b7fe640 1 --2- 172.21.15.43:0/4278411735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c604d0 0x5586d7034350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.228+0000 7f070b7fe640 1 -- 172.21.15.43:0/4278411735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586d6c686b0 msgr2=0x5586d70411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.228+0000 7f070b7fe640 1 --2- 172.21.15.43:0/4278411735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586d6c686b0 0x5586d70411a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.228+0000 7f070b7fe640 1 -- 172.21.15.43:0/4278411735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c604d0 msgr2=0x5586d7034350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.228+0000 7f070b7fe640 1 --2- 172.21.15.43:0/4278411735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c604d0 0x5586d7034350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.228+0000 7f070b7fe640 1 -- 172.21.15.43:0/4278411735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d704d4a0 con 0x5586d6c89390 2024-04-25T10:42:56.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.228+0000 7f070b7fe640 1 --2- 172.21.15.43:0/4278411735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c89390 0x5586d7036e80 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f070002a930 tx=0x7f070002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed41fd1c599c853e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:56.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.228+0000 7f06f1ffb640 1 -- 172.21.15.43:0/4278411735 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700061070 con 0x5586d6c89390 2024-04-25T10:42:56.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.228+0000 7f06f1ffb640 1 -- 172.21.15.43:0/4278411735 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0700003d40 con 0x5586d6c89390 2024-04-25T10:42:56.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.228+0000 7f06f1ffb640 1 -- 172.21.15.43:0/4278411735 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700035040 con 0x5586d6c89390 2024-04-25T10:42:56.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.229+0000 7f07197e9740 1 -- 172.21.15.43:0/4278411735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c89390 msgr2=0x5586d7036e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:56.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.229+0000 7f07197e9740 1 --2- 172.21.15.43:0/4278411735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c89390 0x5586d7036e80 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f070002a930 tx=0x7f070002a410 comp rx=0 tx=0).stop 2024-04-25T10:42:56.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.229+0000 7f07197e9740 1 -- 172.21.15.43:0/4278411735 shutdown_connections 2024-04-25T10:42:56.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.229+0000 7f07197e9740 1 --2- 172.21.15.43:0/4278411735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c89390 0x5586d7036e80 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.229+0000 7f07197e9740 1 --2- 172.21.15.43:0/4278411735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586d6c686b0 0x5586d70411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.229+0000 7f07197e9740 1 --2- 172.21.15.43:0/4278411735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c604d0 0x5586d7034350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.229+0000 7f07197e9740 1 -- 172.21.15.43:0/4278411735 >> 172.21.15.43:0/4278411735 conn(0x5586d6c7d4e0 msgr2=0x5586d7036770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:56.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.229+0000 7f07197e9740 1 -- 172.21.15.43:0/4278411735 shutdown_connections 2024-04-25T10:42:56.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.229+0000 7f07197e9740 1 -- 172.21.15.43:0/4278411735 wait complete. 2024-04-25T10:42:56.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.229+0000 7f07197e9740 1 -- start start 2024-04-25T10:42:56.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.229+0000 7f07197e9740 1 Processor -- start 2024-04-25T10:42:56.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f07197e9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586d6c604d0 0x5586d70a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f07197e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c686b0 0x5586d70a6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f07197e9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c89390 0x5586d70478d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f07197e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f070c1adfb0 con 0x5586d6c686b0 2024-04-25T10:42:56.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f07197e9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f070c1af420 con 0x5586d6c89390 2024-04-25T10:42:56.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f07197e9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f070c1ae680 con 0x5586d6c604d0 2024-04-25T10:42:56.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f07197e9740 10 client.0 _ll_register_callbacks cb 0x7f06fc00d360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:56.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f070bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c686b0 0x5586d70a6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f070bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c686b0 0x5586d70a6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58952/0 (socket says 172.21.15.43:58952) 2024-04-25T10:42:56.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f070bfff640 1 -- 172.21.15.43:0/3093283232 learned_addr learned my addr 172.21.15.43:0/3093283232 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:56.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f070b7fe640 1 --2- 172.21.15.43:0/3093283232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586d6c604d0 0x5586d70a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f070b7fe640 1 --2- 172.21.15.43:0/3093283232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c89390 0x5586d70478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f070bfff640 1 -- 172.21.15.43:0/3093283232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586d6c604d0 msgr2=0x5586d70a69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:56.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f070bfff640 1 --2- 172.21.15.43:0/3093283232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586d6c604d0 0x5586d70a69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f070bfff640 1 -- 172.21.15.43:0/3093283232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c89390 msgr2=0x5586d70478d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:56.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f070bfff640 1 --2- 172.21.15.43:0/3093283232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c89390 0x5586d70478d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f070bfff640 1 -- 172.21.15.43:0/3093283232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d7047fc0 con 0x5586d6c686b0 2024-04-25T10:42:56.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f070b7fe640 1 --2- 172.21.15.43:0/3093283232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c89390 0x5586d70478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:56.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f070bfff640 1 --2- 172.21.15.43:0/3093283232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c686b0 0x5586d70a6ee0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f06fc000c00 tx=0x7f06fc00df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ec90403c340ec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:56.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f06eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:56.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f06eb7fe640 1 -- 172.21.15.43:0/3093283232 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc021070 con 0x5586d6c686b0 2024-04-25T10:42:56.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f06eb7fe640 1 -- 172.21.15.43:0/3093283232 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f06fc01d580 con 0x5586d6c686b0 2024-04-25T10:42:56.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.230+0000 7f07197e9740 1 -- 172.21.15.43:0/3093283232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0700003d40 con 0x5586d6c686b0 2024-04-25T10:42:56.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.231+0000 7f06eb7fe640 1 -- 172.21.15.43:0/3093283232 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f06fc02d050 con 0x5586d6c686b0 2024-04-25T10:42:56.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.231+0000 7f07197e9740 10 client.18600 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:56.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.231+0000 7f07197e9740 1 -- 172.21.15.43:0/3093283232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5586d70af040 con 0x5586d6c686b0 2024-04-25T10:42:56.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.231+0000 7f06eb7fe640 1 -- 172.21.15.43:0/3093283232 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0700003d40 con 0x5586d6c686b0 2024-04-25T10:42:56.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.231+0000 7f06eb7fe640 1 -- 172.21.15.43:0/3093283232 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f06fc011ce0 con 0x5586d6c686b0 2024-04-25T10:42:56.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.231+0000 7f07197e9740 10 client.18600 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:56.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.231+0000 7f07197e9740 10 client.18600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:56.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.231+0000 7f07197e9740 20 client.18600 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:56.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.231+0000 7f07197e9740 1 --2- 172.21.15.43:0/3093283232 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5586d6c1d3a0 0x5586d70b1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:56.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.231+0000 7f07197e9740 4 client.18600 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:42:56.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.231+0000 7f07197e9740 1 -- 172.21.15.43:0/3093283232 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586d6f97800 con 0x5586d6c1d3a0 2024-04-25T10:42:56.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.231+0000 7f070b7fe640 1 --2- 172.21.15.43:0/3093283232 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5586d6c1d3a0 0x5586d70b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:56.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.232+0000 7f06eb7fe640 10 client.18600 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:56.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.232+0000 7f070b7fe640 1 --2- 172.21.15.43:0/3093283232 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5586d6c1d3a0 0x5586d70b1830 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7dbda898664419e server_cookie=c8760921d8ce264b in_seq=0 out_seq=0 2024-04-25T10:42:56.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.232+0000 7f06eb7fe640 1 -- 172.21.15.43:0/3093283232 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586d6f97800 con 0x5586d6c1d3a0 2024-04-25T10:42:56.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.232+0000 7f06eb7fe640 10 client.18600 handle_command_reply: tid=0 2024-04-25T10:42:56.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.232+0000 7f07197e9740 1 client.18600 shutdown 2024-04-25T10:42:56.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.232+0000 7f07197e9740 1 -- 172.21.15.43:0/3093283232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c686b0 msgr2=0x5586d70a6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:56.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.232+0000 7f07197e9740 1 --2- 172.21.15.43:0/3093283232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c686b0 0x5586d70a6ee0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f06fc000c00 tx=0x7f06fc00df10 comp rx=0 tx=0).stop 2024-04-25T10:42:56.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.232+0000 7f07197e9740 1 -- 172.21.15.43:0/3093283232 shutdown_connections 2024-04-25T10:42:56.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.232+0000 7f07197e9740 1 --2- 172.21.15.43:0/3093283232 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5586d6c1d3a0 0x5586d70b1830 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.232+0000 7f07197e9740 1 --2- 172.21.15.43:0/3093283232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586d6c89390 0x5586d70478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.232+0000 7f07197e9740 1 --2- 172.21.15.43:0/3093283232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586d6c686b0 0x5586d70a6ee0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.232+0000 7f06eb7fe640 0 client.18600 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:56.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.232+0000 7f07197e9740 1 --2- 172.21.15.43:0/3093283232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586d6c604d0 0x5586d70a69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.232+0000 7f07197e9740 1 -- 172.21.15.43:0/3093283232 >> 172.21.15.43:0/3093283232 conn(0x5586d6c7d4e0 msgr2=0x5586d70340b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:56.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.232+0000 7f07197e9740 1 -- 172.21.15.43:0/3093283232 shutdown_connections 2024-04-25T10:42:56.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.232+0000 7f07197e9740 1 -- 172.21.15.43:0/3093283232 wait complete. 2024-04-25T10:42:56.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.232+0000 7f07197e9740 20 client.18600 trim_cache size 0 max 16384 2024-04-25T10:42:56.244 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:42:56.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.233+0000 7f0712851640 1 -- 172.21.15.43:0/4019257397 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f06f40afc60 msgr2=0x7f06f40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:56.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.233+0000 7f0712851640 1 --2- 172.21.15.43:0/4019257397 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f06f40afc60 0x7f06f40b2170 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f07040a49a0 tx=0x7f0704074f90 comp rx=0 tx=0).stop 2024-04-25T10:42:56.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.233+0000 7f0712851640 1 -- 172.21.15.43:0/4019257397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f070c108ed0 msgr2=0x7f070c19c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:56.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.233+0000 7f0712851640 1 --2- 172.21.15.43:0/4019257397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f070c108ed0 0x7f070c19c050 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f070400cce0 tx=0x7f070400a620 comp rx=0 tx=0).stop 2024-04-25T10:42:56.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.233+0000 7f0712851640 1 -- 172.21.15.43:0/4019257397 shutdown_connections 2024-04-25T10:42:56.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.233+0000 7f0712851640 1 --2- 172.21.15.43:0/4019257397 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f06f40afc60 0x7f06f40b2170 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.233+0000 7f0712851640 1 --2- 172.21.15.43:0/4019257397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f070c108ed0 0x7f070c19c050 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.233+0000 7f0712851640 1 --2- 172.21.15.43:0/4019257397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f070c108a30 0x7f070c19bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.233+0000 7f0712851640 1 --2- 172.21.15.43:0/4019257397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f070c103190 0x7f070c19b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:56.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.233+0000 7f0712851640 1 -- 172.21.15.43:0/4019257397 >> 172.21.15.43:0/4019257397 conn(0x7f070c0fe260 msgr2=0x7f070c0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:56.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.233+0000 7f0712851640 1 -- 172.21.15.43:0/4019257397 shutdown_connections 2024-04-25T10:42:56.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:56.233+0000 7f0712851640 1 -- 172.21.15.43:0/4019257397 wait complete. 2024-04-25T10:42:56.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:56 smithi077 ceph-mon[32772]: pgmap v323: 97 pgs: 97 active+clean; 3.8 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-25T10:42:56.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:56 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2283360213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:42:56.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:56 smithi164 ceph-mon[32713]: pgmap v323: 97 pgs: 97 active+clean; 3.8 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-25T10:42:56.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:56 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2283360213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:42:56.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:56 smithi043 ceph-mon[29478]: pgmap v323: 97 pgs: 97 active+clean; 3.8 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-25T10:42:56.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:56 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2283360213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:42:57.248 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:42:57.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:42:57.249 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:42:57.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.378+0000 7f20d9ffe640 1 -- 172.21.15.43:0/3916236182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d4104260 msgr2=0x7f20d4110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:57.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.378+0000 7f20d9ffe640 1 --2- 172.21.15.43:0/3916236182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d4104260 0x7f20d4110630 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009900 tx=0x7f20cc02a410 comp rx=0 tx=0).stop 2024-04-25T10:42:57.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.378+0000 7f20d9ffe640 1 -- 172.21.15.43:0/3916236182 shutdown_connections 2024-04-25T10:42:57.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.378+0000 7f20d9ffe640 1 --2- 172.21.15.43:0/3916236182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d4104260 0x7f20d4110630 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.378+0000 7f20d9ffe640 1 --2- 172.21.15.43:0/3916236182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d4109fb0 0x7f20d4103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.378+0000 7f20d9ffe640 1 --2- 172.21.15.43:0/3916236182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20d4109b10 0x7f20d41036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.378+0000 7f20d9ffe640 1 -- 172.21.15.43:0/3916236182 >> 172.21.15.43:0/3916236182 conn(0x7f20d40ff1d0 msgr2=0x7f20d4101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:57.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.378+0000 7f20d9ffe640 1 -- 172.21.15.43:0/3916236182 shutdown_connections 2024-04-25T10:42:57.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.378+0000 7f20d9ffe640 1 -- 172.21.15.43:0/3916236182 wait complete. 2024-04-25T10:42:57.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.379+0000 7f20d9ffe640 1 Processor -- start 2024-04-25T10:42:57.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.379+0000 7f20d9ffe640 1 -- start start 2024-04-25T10:42:57.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.379+0000 7f20d9ffe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d4104260 0x7f20d419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.379+0000 7f20d9ffe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d4109b10 0x7f20d419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.379+0000 7f20d9ffe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20d4109fb0 0x7f20d419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.379+0000 7f20d9ffe640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4113d20 con 0x7f20d4109b10 2024-04-25T10:42:57.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.379+0000 7f20d9ffe640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4113ba0 con 0x7f20d4109fb0 2024-04-25T10:42:57.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.379+0000 7f20d9ffe640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4113ea0 con 0x7f20d4104260 2024-04-25T10:42:57.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.379+0000 7f20d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d4109b10 0x7f20d419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.379+0000 7f20d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d4109b10 0x7f20d419ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58958/0 (socket says 172.21.15.43:58958) 2024-04-25T10:42:57.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.379+0000 7f20d2ffd640 1 -- 172.21.15.43:0/149326346 learned_addr learned my addr 172.21.15.43:0/149326346 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:57.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.380+0000 7f20d3fff640 1 --2- 172.21.15.43:0/149326346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20d4109fb0 0x7f20d419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.380+0000 7f20d37fe640 1 --2- 172.21.15.43:0/149326346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d4104260 0x7f20d419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.380+0000 7f20d2ffd640 1 -- 172.21.15.43:0/149326346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d4104260 msgr2=0x7f20d419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:57.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.380+0000 7f20d2ffd640 1 --2- 172.21.15.43:0/149326346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d4104260 0x7f20d419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.380+0000 7f20d2ffd640 1 -- 172.21.15.43:0/149326346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20d4109fb0 msgr2=0x7f20d419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:57.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.380+0000 7f20d2ffd640 1 --2- 172.21.15.43:0/149326346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20d4109fb0 0x7f20d419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.380+0000 7f20d2ffd640 1 -- 172.21.15.43:0/149326346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d419f560 con 0x7f20d4109b10 2024-04-25T10:42:57.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.380+0000 7f20d2ffd640 1 --2- 172.21.15.43:0/149326346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d4109b10 0x7f20d419ac60 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f20bc005d00 tx=0x7f20bc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:57.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.380+0000 7f20d0ff9640 1 -- 172.21.15.43:0/149326346 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc017070 con 0x7f20d4109b10 2024-04-25T10:42:57.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.380+0000 7f20d0ff9640 1 -- 172.21.15.43:0/149326346 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20bc009c30 con 0x7f20d4109b10 2024-04-25T10:42:57.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.380+0000 7f20d9ffe640 1 -- 172.21.15.43:0/149326346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d419f850 con 0x7f20d4109b10 2024-04-25T10:42:57.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.380+0000 7f20d0ff9640 1 -- 172.21.15.43:0/149326346 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc00f620 con 0x7f20d4109b10 2024-04-25T10:42:57.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.380+0000 7f20d9ffe640 1 -- 172.21.15.43:0/149326346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d419fd50 con 0x7f20d4109b10 2024-04-25T10:42:57.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.383+0000 7f20d0ff9640 1 -- 172.21.15.43:0/149326346 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f20bc01a070 con 0x7f20d4109b10 2024-04-25T10:42:57.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.383+0000 7f20e0f83740 1 Processor -- start 2024-04-25T10:42:57.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.383+0000 7f20e0f83740 1 -- start start 2024-04-25T10:42:57.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.383+0000 7f20e0f83740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5d5410 0x564f4f9754d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.383+0000 7f20e0f83740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5ac550 0x564f4f975a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.383+0000 7f20e0f83740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5b4730 0x564f4f975f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.383+0000 7f20e0f83740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564f4f8dbb00 con 0x564f4f5ac550 2024-04-25T10:42:57.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.383+0000 7f20e0f83740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564f4f877920 con 0x564f4f5d5410 2024-04-25T10:42:57.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.383+0000 7f20e0f83740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564f4f8d7ac0 con 0x564f4f5b4730 2024-04-25T10:42:57.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.383+0000 7f20d37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5ac550 0x564f4f975a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.383+0000 7f20d37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5ac550 0x564f4f975a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58962/0 (socket says 172.21.15.43:58962) 2024-04-25T10:42:57.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.383+0000 7f20d37fe640 1 -- 172.21.15.43:0/3804697493 learned_addr learned my addr 172.21.15.43:0/3804697493 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:57.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.383+0000 7f20d2ffd640 1 --2- 172.21.15.43:0/3804697493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5b4730 0x564f4f975f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.383+0000 7f20d0ff9640 1 --2- 172.21.15.43:0/149326346 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f20c00afc60 0x7f20c00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.383+0000 7f20d3fff640 1 --2- 172.21.15.43:0/3804697493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5d5410 0x564f4f9754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.383+0000 7f20d37fe640 1 -- 172.21.15.43:0/3804697493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5b4730 msgr2=0x564f4f975f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:57.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.383+0000 7f20d37fe640 1 --2- 172.21.15.43:0/3804697493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5b4730 0x564f4f975f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.383+0000 7f20d37fe640 1 -- 172.21.15.43:0/3804697493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5d5410 msgr2=0x564f4f9754d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:57.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.383+0000 7f20d37fe640 1 --2- 172.21.15.43:0/3804697493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5d5410 0x564f4f9754d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.383+0000 7f20d37fe640 1 -- 172.21.15.43:0/3804697493 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f4f6b57e0 con 0x564f4f5ac550 2024-04-25T10:42:57.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.384+0000 7f20d3fff640 1 --2- 172.21.15.43:0/3804697493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5d5410 0x564f4f9754d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:57.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.384+0000 7f20d3fff640 1 --2- 172.21.15.43:0/149326346 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f20c00afc60 0x7f20c00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.384+0000 7f20d0ff9640 1 -- 172.21.15.43:0/149326346 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f20bc0deb10 con 0x7f20d4109b10 2024-04-25T10:42:57.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.384+0000 7f20d37fe640 1 --2- 172.21.15.43:0/3804697493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5ac550 0x564f4f975a10 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009ec0 tx=0x7f20c400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695a6f00baf30bf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:57.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.384+0000 7f20c9ffb640 1 -- 172.21.15.43:0/3804697493 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4016070 con 0x564f4f5ac550 2024-04-25T10:42:57.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.384+0000 7f20c9ffb640 1 -- 172.21.15.43:0/3804697493 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20c4004d20 con 0x564f4f5ac550 2024-04-25T10:42:57.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.384+0000 7f20c9ffb640 1 -- 172.21.15.43:0/3804697493 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c40116d0 con 0x564f4f5ac550 2024-04-25T10:42:57.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.385+0000 7f20e0f83740 1 -- 172.21.15.43:0/3804697493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5ac550 msgr2=0x564f4f975a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:57.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.385+0000 7f20e0f83740 1 --2- 172.21.15.43:0/3804697493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5ac550 0x564f4f975a10 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009ec0 tx=0x7f20c400e6c0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.385+0000 7f20d3fff640 1 --2- 172.21.15.43:0/149326346 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f20c00afc60 0x7f20c00b2170 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f20cc005ab0 tx=0x7f20cc009260 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:57.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.385+0000 7f20e0f83740 1 -- 172.21.15.43:0/3804697493 shutdown_connections 2024-04-25T10:42:57.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.385+0000 7f20e0f83740 1 --2- 172.21.15.43:0/3804697493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5b4730 0x564f4f975f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.385+0000 7f20e0f83740 1 --2- 172.21.15.43:0/3804697493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5ac550 0x564f4f975a10 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.385+0000 7f20e0f83740 1 --2- 172.21.15.43:0/3804697493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5d5410 0x564f4f9754d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.385+0000 7f20e0f83740 1 -- 172.21.15.43:0/3804697493 >> 172.21.15.43:0/3804697493 conn(0x564f4f5c9560 msgr2=0x564f4f971dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:57.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.385+0000 7f20e0f83740 1 -- 172.21.15.43:0/3804697493 shutdown_connections 2024-04-25T10:42:57.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.385+0000 7f20e0f83740 1 -- 172.21.15.43:0/3804697493 wait complete. 2024-04-25T10:42:57.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.385+0000 7f20e0f83740 1 -- start start 2024-04-25T10:42:57.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.386+0000 7f20e0f83740 1 Processor -- start 2024-04-25T10:42:57.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.386+0000 7f20e0f83740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5ac550 0x564f4f984020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.386+0000 7f20e0f83740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5b4730 0x564f4f984560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.386+0000 7f20e0f83740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5d5410 0x564f4f990230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.386+0000 7f20e0f83740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564f4f986d80 con 0x564f4f5b4730 2024-04-25T10:42:57.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.386+0000 7f20e0f83740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564f4f8755d0 con 0x564f4f5d5410 2024-04-25T10:42:57.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.386+0000 7f20e0f83740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564f4f986f00 con 0x564f4f5ac550 2024-04-25T10:42:57.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.386+0000 7f20e0f83740 10 client.0 _ll_register_callbacks cb 0x564f4f863080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:57.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.386+0000 7f20d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5b4730 0x564f4f984560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.386+0000 7f20d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5b4730 0x564f4f984560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58970/0 (socket says 172.21.15.43:58970) 2024-04-25T10:42:57.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.386+0000 7f20d2ffd640 1 -- 172.21.15.43:0/2068488513 learned_addr learned my addr 172.21.15.43:0/2068488513 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:57.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.386+0000 7f20d37fe640 1 --2- 172.21.15.43:0/2068488513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5ac550 0x564f4f984020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.386+0000 7f20d3fff640 1 --2- 172.21.15.43:0/2068488513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5d5410 0x564f4f990230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.386+0000 7f20d2ffd640 1 -- 172.21.15.43:0/2068488513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5ac550 msgr2=0x564f4f984020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:57.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.386+0000 7f20d2ffd640 1 --2- 172.21.15.43:0/2068488513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5ac550 0x564f4f984020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.386+0000 7f20d2ffd640 1 -- 172.21.15.43:0/2068488513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5d5410 msgr2=0x564f4f990230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:57.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.386+0000 7f20d2ffd640 1 --2- 172.21.15.43:0/2068488513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5d5410 0x564f4f990230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.386+0000 7f20d2ffd640 1 -- 172.21.15.43:0/2068488513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f4f715f70 con 0x564f4f5b4730 2024-04-25T10:42:57.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.386+0000 7f20d3fff640 1 --2- 172.21.15.43:0/2068488513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5d5410 0x564f4f990230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:57.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.386+0000 7f20d37fe640 1 --2- 172.21.15.43:0/2068488513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5ac550 0x564f4f984020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:57.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.387+0000 7f20d2ffd640 1 --2- 172.21.15.43:0/2068488513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5b4730 0x564f4f984560 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0a3f20 tx=0x7f20bc0e2250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a51dbd45f29a25a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.387+0000 7f20b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.387+0000 7f20b2ffd640 1 -- 172.21.15.43:0/2068488513 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc021050 con 0x564f4f5b4730 2024-04-25T10:42:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.387+0000 7f20b2ffd640 1 -- 172.21.15.43:0/2068488513 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f20bc0a5f10 con 0x564f4f5b4730 2024-04-25T10:42:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.387+0000 7f20e0f83740 1 -- 172.21.15.43:0/2068488513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20c4004d20 con 0x564f4f5b4730 2024-04-25T10:42:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.387+0000 7f20b2ffd640 1 -- 172.21.15.43:0/2068488513 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f20bc0e3040 con 0x564f4f5b4730 2024-04-25T10:42:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.387+0000 7f20e0f83740 10 client.18612 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.387+0000 7f20e0f83740 1 -- 172.21.15.43:0/2068488513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564f4f5ad170 con 0x564f4f5b4730 2024-04-25T10:42:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.387+0000 7f20b2ffd640 1 -- 172.21.15.43:0/2068488513 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20c4004d20 con 0x564f4f5b4730 2024-04-25T10:42:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.387+0000 7f20b2ffd640 1 -- 172.21.15.43:0/2068488513 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f20bc0a5980 con 0x564f4f5b4730 2024-04-25T10:42:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.387+0000 7f20e0f83740 10 client.18612 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.387+0000 7f20e0f83740 10 client.18612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.387+0000 7f20e0f83740 20 client.18612 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.387+0000 7f20e0f83740 1 --2- 172.21.15.43:0/2068488513 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564f4f5941f0 0x564f4f996430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.387+0000 7f20e0f83740 4 client.18612 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:42:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.387+0000 7f20e0f83740 1 -- 172.21.15.43:0/2068488513 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f4f8e9710 con 0x564f4f5941f0 2024-04-25T10:42:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.388+0000 7f20d37fe640 1 --2- 172.21.15.43:0/2068488513 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564f4f5941f0 0x564f4f996430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.388+0000 7f20b2ffd640 10 client.18612 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.388+0000 7f20d37fe640 1 --2- 172.21.15.43:0/2068488513 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564f4f5941f0 0x564f4f996430 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8fb7b2388b77d49 server_cookie=8c314defc1247113 in_seq=0 out_seq=0 2024-04-25T10:42:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.389+0000 7f20b2ffd640 1 -- 172.21.15.43:0/2068488513 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564f4f8e9710 con 0x564f4f5941f0 2024-04-25T10:42:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.389+0000 7f20b2ffd640 10 client.18612 handle_command_reply: tid=0 2024-04-25T10:42:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.389+0000 7f20e0f83740 1 client.18612 shutdown 2024-04-25T10:42:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.389+0000 7f20e0f83740 1 -- 172.21.15.43:0/2068488513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5b4730 msgr2=0x564f4f984560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.389+0000 7f20e0f83740 1 --2- 172.21.15.43:0/2068488513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5b4730 0x564f4f984560 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0a3f20 tx=0x7f20bc0e2250 comp rx=0 tx=0).stop 2024-04-25T10:42:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.389+0000 7f20e0f83740 1 -- 172.21.15.43:0/2068488513 shutdown_connections 2024-04-25T10:42:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.389+0000 7f20e0f83740 1 --2- 172.21.15.43:0/2068488513 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564f4f5941f0 0x564f4f996430 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.389+0000 7f20e0f83740 1 --2- 172.21.15.43:0/2068488513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5d5410 0x564f4f990230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.389+0000 7f20e0f83740 1 --2- 172.21.15.43:0/2068488513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5b4730 0x564f4f984560 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.389+0000 7f20b2ffd640 0 client.18612 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.389+0000 7f20e0f83740 1 --2- 172.21.15.43:0/2068488513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5ac550 0x564f4f984020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.389+0000 7f20e0f83740 1 -- 172.21.15.43:0/2068488513 >> 172.21.15.43:0/2068488513 conn(0x564f4f5c9560 msgr2=0x564f4f9772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.389+0000 7f20e0f83740 1 -- 172.21.15.43:0/2068488513 shutdown_connections 2024-04-25T10:42:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.389+0000 7f20e0f83740 1 -- 172.21.15.43:0/2068488513 wait complete. 2024-04-25T10:42:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.389+0000 7f20e0f83740 20 client.18612 trim_cache size 0 max 16384 2024-04-25T10:42:57.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.407+0000 7f20e0f83740 1 Processor -- start 2024-04-25T10:42:57.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.408+0000 7f20e0f83740 1 -- start start 2024-04-25T10:42:57.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.408+0000 7f20e0f83740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5ac550 0x564f4f9839c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.408+0000 7f20e0f83740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5b4730 0x564f4f970bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.408+0000 7f20e0f83740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5d5410 0x564f4f99bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.408+0000 7f20e0f83740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41b0090 con 0x564f4f5b4730 2024-04-25T10:42:57.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.408+0000 7f20e0f83740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41ad960 con 0x564f4f5d5410 2024-04-25T10:42:57.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.408+0000 7f20e0f83740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41ab7d0 con 0x564f4f5ac550 2024-04-25T10:42:57.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.408+0000 7f20d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5b4730 0x564f4f970bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.408+0000 7f20d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5b4730 0x564f4f970bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58986/0 (socket says 172.21.15.43:58986) 2024-04-25T10:42:57.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.408+0000 7f20d2ffd640 1 -- 172.21.15.43:0/1693201185 learned_addr learned my addr 172.21.15.43:0/1693201185 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:57.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.408+0000 7f20d37fe640 1 --2- 172.21.15.43:0/1693201185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5ac550 0x564f4f9839c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.408+0000 7f20d3fff640 1 --2- 172.21.15.43:0/1693201185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5d5410 0x564f4f99bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.408+0000 7f20d2ffd640 1 -- 172.21.15.43:0/1693201185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5ac550 msgr2=0x564f4f9839c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:57.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.408+0000 7f20d2ffd640 1 --2- 172.21.15.43:0/1693201185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5ac550 0x564f4f9839c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.408+0000 7f20d2ffd640 1 -- 172.21.15.43:0/1693201185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5d5410 msgr2=0x564f4f99bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:57.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.408+0000 7f20d2ffd640 1 --2- 172.21.15.43:0/1693201185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5d5410 0x564f4f99bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.408+0000 7f20d2ffd640 1 -- 172.21.15.43:0/1693201185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f4f97ea90 con 0x564f4f5b4730 2024-04-25T10:42:57.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.408+0000 7f20d37fe640 1 --2- 172.21.15.43:0/1693201185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5ac550 0x564f4f9839c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:57.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.408+0000 7f20d2ffd640 1 --2- 172.21.15.43:0/1693201185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5b4730 0x564f4f970bd0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0dc860 tx=0x7f20bc00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b7d7e0529db33a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:57.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.409+0000 7f20c97fa640 1 -- 172.21.15.43:0/1693201185 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc014020 con 0x564f4f5b4730 2024-04-25T10:42:57.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.409+0000 7f20c97fa640 1 -- 172.21.15.43:0/1693201185 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20bc0a6cb0 con 0x564f4f5b4730 2024-04-25T10:42:57.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.409+0000 7f20c97fa640 1 -- 172.21.15.43:0/1693201185 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc021070 con 0x564f4f5b4730 2024-04-25T10:42:57.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.409+0000 7f20e0f83740 1 -- 172.21.15.43:0/1693201185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5b4730 msgr2=0x564f4f970bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:57.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.409+0000 7f20e0f83740 1 --2- 172.21.15.43:0/1693201185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5b4730 0x564f4f970bd0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0dc860 tx=0x7f20bc00bf90 comp rx=0 tx=0).stop 2024-04-25T10:42:57.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.409+0000 7f20e0f83740 1 -- 172.21.15.43:0/1693201185 shutdown_connections 2024-04-25T10:42:57.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.409+0000 7f20e0f83740 1 --2- 172.21.15.43:0/1693201185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5d5410 0x564f4f99bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.409+0000 7f20e0f83740 1 --2- 172.21.15.43:0/1693201185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5b4730 0x564f4f970bd0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.409+0000 7f20e0f83740 1 --2- 172.21.15.43:0/1693201185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5ac550 0x564f4f9839c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.409+0000 7f20e0f83740 1 -- 172.21.15.43:0/1693201185 >> 172.21.15.43:0/1693201185 conn(0x564f4f5c9560 msgr2=0x564f4f9704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:57.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.409+0000 7f20e0f83740 1 -- 172.21.15.43:0/1693201185 shutdown_connections 2024-04-25T10:42:57.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.409+0000 7f20e0f83740 1 -- 172.21.15.43:0/1693201185 wait complete. 2024-04-25T10:42:57.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.410+0000 7f20e0f83740 1 -- start start 2024-04-25T10:42:57.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.410+0000 7f20e0f83740 1 Processor -- start 2024-04-25T10:42:57.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.410+0000 7f20e0f83740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5ac550 0x564f4f98c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.410+0000 7f20e0f83740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5b4730 0x564f4f987880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.410+0000 7f20e0f83740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5d5410 0x564f4f987dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.410+0000 7f20e0f83740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41abe20 con 0x564f4f5d5410 2024-04-25T10:42:57.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.410+0000 7f20e0f83740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41ad290 con 0x564f4f5b4730 2024-04-25T10:42:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.410+0000 7f20e0f83740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41ac4f0 con 0x564f4f5ac550 2024-04-25T10:42:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.410+0000 7f20e0f83740 10 client.0 _ll_register_callbacks cb 0x7f20bc005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.410+0000 7f20d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5d5410 0x564f4f987dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.410+0000 7f20d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5d5410 0x564f4f987dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58998/0 (socket says 172.21.15.43:58998) 2024-04-25T10:42:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.410+0000 7f20d3fff640 1 -- 172.21.15.43:0/3543190800 learned_addr learned my addr 172.21.15.43:0/3543190800 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.410+0000 7f20d37fe640 1 --2- 172.21.15.43:0/3543190800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5ac550 0x564f4f98c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.411+0000 7f20d2ffd640 1 --2- 172.21.15.43:0/3543190800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5b4730 0x564f4f987880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.411+0000 7f20d3fff640 1 -- 172.21.15.43:0/3543190800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5ac550 msgr2=0x564f4f98c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.411+0000 7f20d3fff640 1 --2- 172.21.15.43:0/3543190800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5ac550 0x564f4f98c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.411+0000 7f20d3fff640 1 -- 172.21.15.43:0/3543190800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5b4730 msgr2=0x564f4f987880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.411+0000 7f20d3fff640 1 --2- 172.21.15.43:0/3543190800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5b4730 0x564f4f987880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.411+0000 7f20d3fff640 1 -- 172.21.15.43:0/3543190800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f4f988330 con 0x564f4f5d5410 2024-04-25T10:42:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.411+0000 7f20d3fff640 1 --2- 172.21.15.43:0/3543190800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5d5410 0x564f4f987dc0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009900 tx=0x7f20cc005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211bb99fc200ebdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.411+0000 7f20b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.411+0000 7f20b2ffd640 1 -- 172.21.15.43:0/3543190800 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc035040 con 0x564f4f5d5410 2024-04-25T10:42:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.411+0000 7f20b2ffd640 1 -- 172.21.15.43:0/3543190800 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f20cc004250 con 0x564f4f5d5410 2024-04-25T10:42:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.411+0000 7f20e0f83740 1 -- 172.21.15.43:0/3543190800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20bc0a6cb0 con 0x564f4f5d5410 2024-04-25T10:42:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.411+0000 7f20b2ffd640 1 -- 172.21.15.43:0/3543190800 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f20cc031b10 con 0x564f4f5d5410 2024-04-25T10:42:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.411+0000 7f20e0f83740 10 client.18618 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.411+0000 7f20e0f83740 1 -- 172.21.15.43:0/3543190800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564f4f9fe470 con 0x564f4f5d5410 2024-04-25T10:42:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.411+0000 7f20b2ffd640 1 -- 172.21.15.43:0/3543190800 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20bc0a6cb0 con 0x564f4f5d5410 2024-04-25T10:42:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.412+0000 7f20b2ffd640 1 -- 172.21.15.43:0/3543190800 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f20cc0038c0 con 0x564f4f5d5410 2024-04-25T10:42:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.412+0000 7f20e0f83740 10 client.18618 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.412+0000 7f20e0f83740 10 client.18618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.412+0000 7f20e0f83740 20 client.18618 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.412+0000 7f20e0f83740 1 --2- 172.21.15.43:0/3543190800 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564f4f569630 0x564f4fa00c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.412+0000 7f20e0f83740 4 client.18618 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:42:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.412+0000 7f20e0f83740 1 -- 172.21.15.43:0/3543190800 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564f4f8e3580 con 0x564f4f569630 2024-04-25T10:42:57.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.412+0000 7f20d37fe640 1 --2- 172.21.15.43:0/3543190800 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564f4f569630 0x564f4fa00c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.413+0000 7f20b2ffd640 10 client.18618 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:57.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.413+0000 7f20d37fe640 1 --2- 172.21.15.43:0/3543190800 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564f4f569630 0x564f4fa00c60 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2e56a847bd652ad server_cookie=78cd7d3c127dab80 in_seq=0 out_seq=0 2024-04-25T10:42:57.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.414+0000 7f20b2ffd640 1 -- 172.21.15.43:0/3543190800 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564f4f8e3580 con 0x564f4f569630 2024-04-25T10:42:57.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.414+0000 7f20b2ffd640 10 client.18618 handle_command_reply: tid=0 2024-04-25T10:42:57.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.414+0000 7f20e0f83740 1 client.18618 shutdown 2024-04-25T10:42:57.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.414+0000 7f20e0f83740 1 -- 172.21.15.43:0/3543190800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5d5410 msgr2=0x564f4f987dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:57.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.414+0000 7f20e0f83740 1 --2- 172.21.15.43:0/3543190800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5d5410 0x564f4f987dc0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009900 tx=0x7f20cc005a40 comp rx=0 tx=0).stop 2024-04-25T10:42:57.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.414+0000 7f20e0f83740 1 -- 172.21.15.43:0/3543190800 shutdown_connections 2024-04-25T10:42:57.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.414+0000 7f20e0f83740 1 --2- 172.21.15.43:0/3543190800 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x564f4f569630 0x564f4fa00c60 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.414+0000 7f20e0f83740 1 --2- 172.21.15.43:0/3543190800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f4f5b4730 0x564f4f987880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.414+0000 7f20e0f83740 1 --2- 172.21.15.43:0/3543190800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f4f5d5410 0x564f4f987dc0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.414+0000 7f20b2ffd640 0 client.18618 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:57.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.414+0000 7f20e0f83740 1 --2- 172.21.15.43:0/3543190800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f4f5ac550 0x564f4f98c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.414+0000 7f20e0f83740 1 -- 172.21.15.43:0/3543190800 >> 172.21.15.43:0/3543190800 conn(0x564f4f5c9560 msgr2=0x564f4f977110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:57.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.414+0000 7f20e0f83740 1 -- 172.21.15.43:0/3543190800 shutdown_connections 2024-04-25T10:42:57.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.414+0000 7f20e0f83740 1 -- 172.21.15.43:0/3543190800 wait complete. 2024-04-25T10:42:57.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.414+0000 7f20e0f83740 20 client.18618 trim_cache size 0 max 16384 2024-04-25T10:42:57.422 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:42:57.422 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:42:57.422 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "533d5220-b8a4-4d37-8903-ded1f0555a71", 2024-04-25T10:42:57.422 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:42:57.423 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:42:57.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.415+0000 7f20d9ffe640 1 -- 172.21.15.43:0/149326346 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f20c00afc60 msgr2=0x7f20c00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:57.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.415+0000 7f20d9ffe640 1 --2- 172.21.15.43:0/149326346 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f20c00afc60 0x7f20c00b2170 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f20cc005ab0 tx=0x7f20cc009260 comp rx=0 tx=0).stop 2024-04-25T10:42:57.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.415+0000 7f20d9ffe640 1 -- 172.21.15.43:0/149326346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d4109b10 msgr2=0x7f20d419ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:57.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.415+0000 7f20d9ffe640 1 --2- 172.21.15.43:0/149326346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d4109b10 0x7f20d419ac60 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f20bc005d00 tx=0x7f20bc00dde0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.415+0000 7f20d9ffe640 1 -- 172.21.15.43:0/149326346 shutdown_connections 2024-04-25T10:42:57.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.415+0000 7f20d9ffe640 1 --2- 172.21.15.43:0/149326346 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f20c00afc60 0x7f20c00b2170 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.415+0000 7f20d9ffe640 1 --2- 172.21.15.43:0/149326346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20d4109fb0 0x7f20d419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.415+0000 7f20d9ffe640 1 --2- 172.21.15.43:0/149326346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d4109b10 0x7f20d419ac60 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.415+0000 7f20d9ffe640 1 --2- 172.21.15.43:0/149326346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d4104260 0x7f20d419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.415+0000 7f20d9ffe640 1 -- 172.21.15.43:0/149326346 >> 172.21.15.43:0/149326346 conn(0x7f20d40ff1d0 msgr2=0x7f20d41065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:57.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.415+0000 7f20d9ffe640 1 -- 172.21.15.43:0/149326346 shutdown_connections 2024-04-25T10:42:57.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.415+0000 7f20d9ffe640 1 -- 172.21.15.43:0/149326346 wait complete. 2024-04-25T10:42:57.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:42:57.428 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:42:57.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.539+0000 7f959d6ee640 1 -- 172.21.15.43:0/1451585945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9598104cd0 msgr2=0x7f95981050e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:57.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.539+0000 7f959d6ee640 1 --2- 172.21.15.43:0/1451585945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9598104cd0 0x7f95981050e0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f9590009ca0 tx=0x7f959002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.540+0000 7f959d6ee640 1 -- 172.21.15.43:0/1451585945 shutdown_connections 2024-04-25T10:42:57.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.540+0000 7f959d6ee640 1 --2- 172.21.15.43:0/1451585945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f959810b730 0x7f959810db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.540+0000 7f959d6ee640 1 --2- 172.21.15.43:0/1451585945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f959810e090 0x7f9598110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.540+0000 7f959d6ee640 1 --2- 172.21.15.43:0/1451585945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9598104cd0 0x7f95981050e0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.540+0000 7f959d6ee640 1 -- 172.21.15.43:0/1451585945 >> 172.21.15.43:0/1451585945 conn(0x7f9598078110 msgr2=0x7f9598078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:57.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.540+0000 7f959d6ee640 1 -- 172.21.15.43:0/1451585945 shutdown_connections 2024-04-25T10:42:57.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.540+0000 7f959d6ee640 1 -- 172.21.15.43:0/1451585945 wait complete. 2024-04-25T10:42:57.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.540+0000 7f959d6ee640 1 Processor -- start 2024-04-25T10:42:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.540+0000 7f959d6ee640 1 -- start start 2024-04-25T10:42:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.541+0000 7f959d6ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9598104cd0 0x7f9598102590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.541+0000 7f959d6ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f959810b730 0x7f9598102ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.541+0000 7f959d6ee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f959810e090 0x7f9598103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.541+0000 7f959d6ee640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9598113ce0 con 0x7f9598104cd0 2024-04-25T10:42:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.541+0000 7f959d6ee640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9598113b60 con 0x7f959810b730 2024-04-25T10:42:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.541+0000 7f959d6ee640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9598113e60 con 0x7f959810e090 2024-04-25T10:42:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.541+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9598104cd0 0x7f9598102590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.541+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9598104cd0 0x7f9598102590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59006/0 (socket says 172.21.15.43:59006) 2024-04-25T10:42:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.541+0000 7f9596ffd640 1 -- 172.21.15.43:0/1809550452 learned_addr learned my addr 172.21.15.43:0/1809550452 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.541+0000 7f95967fc640 1 --2- 172.21.15.43:0/1809550452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f959810b730 0x7f9598102ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.541+0000 7f9596ffd640 1 -- 172.21.15.43:0/1809550452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f959810e090 msgr2=0x7f9598103010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.541+0000 7f9596ffd640 1 --2- 172.21.15.43:0/1809550452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f959810e090 0x7f9598103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.541+0000 7f9596ffd640 1 -- 172.21.15.43:0/1809550452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f959810b730 msgr2=0x7f9598102ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.541+0000 7f9596ffd640 1 --2- 172.21.15.43:0/1809550452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f959810b730 0x7f9598102ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.541+0000 7f9596ffd640 1 -- 172.21.15.43:0/1809550452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95981a8a50 con 0x7f9598104cd0 2024-04-25T10:42:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.541+0000 7f95967fc640 1 --2- 172.21.15.43:0/1809550452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f959810b730 0x7f9598102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:42:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.541+0000 7f9596ffd640 1 --2- 172.21.15.43:0/1809550452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9598104cd0 0x7f9598102590 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f959002ade0 tx=0x7f9590003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.541+0000 7f957ffff640 1 -- 172.21.15.43:0/1809550452 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590003d60 con 0x7f9598104cd0 2024-04-25T10:42:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.542+0000 7f957ffff640 1 -- 172.21.15.43:0/1809550452 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95900044f0 con 0x7f9598104cd0 2024-04-25T10:42:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.542+0000 7f959d6ee640 1 -- 172.21.15.43:0/1809550452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95981a8ce0 con 0x7f9598104cd0 2024-04-25T10:42:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.542+0000 7f957ffff640 1 -- 172.21.15.43:0/1809550452 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959002fd10 con 0x7f9598104cd0 2024-04-25T10:42:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.542+0000 7f959d6ee640 1 -- 172.21.15.43:0/1809550452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95981a91d0 con 0x7f9598104cd0 2024-04-25T10:42:57.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.544+0000 7f95a45a0740 1 Processor -- start 2024-04-25T10:42:57.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.544+0000 7f95a45a0740 1 -- start start 2024-04-25T10:42:57.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.544+0000 7f957ffff640 1 -- 172.21.15.43:0/1809550452 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f95900046a0 con 0x7f9598104cd0 2024-04-25T10:42:57.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.544+0000 7f95a45a0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c44c390 0x558d4c7eca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.545+0000 7f95a45a0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c4234d0 0x558d4c7ecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.545+0000 7f95a45a0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558d4c42b6b0 0x558d4c7f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.545+0000 7f95a45a0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558d4c752d80 con 0x558d4c4234d0 2024-04-25T10:42:57.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.545+0000 7f95a45a0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558d4c70c8b0 con 0x558d4c44c390 2024-04-25T10:42:57.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.545+0000 7f95a45a0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558d4c74ed40 con 0x558d4c42b6b0 2024-04-25T10:42:57.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.545+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c4234d0 0x558d4c7ecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.545+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c4234d0 0x558d4c7ecfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59014/0 (socket says 172.21.15.43:59014) 2024-04-25T10:42:57.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.545+0000 7f9596ffd640 1 -- 172.21.15.43:0/1987549565 learned_addr learned my addr 172.21.15.43:0/1987549565 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:57.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.545+0000 7f95977fe640 1 --2- 172.21.15.43:0/1987549565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c44c390 0x558d4c7eca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.545+0000 7f95967fc640 1 --2- 172.21.15.43:0/1987549565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558d4c42b6b0 0x558d4c7f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.545+0000 7f9596ffd640 1 -- 172.21.15.43:0/1987549565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558d4c42b6b0 msgr2=0x558d4c7f98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:57.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.545+0000 7f9596ffd640 1 --2- 172.21.15.43:0/1987549565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558d4c42b6b0 0x558d4c7f98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.545+0000 7f9596ffd640 1 -- 172.21.15.43:0/1987549565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c44c390 msgr2=0x558d4c7eca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:57.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.545+0000 7f9596ffd640 1 --2- 172.21.15.43:0/1987549565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c44c390 0x558d4c7eca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.545+0000 7f9596ffd640 1 -- 172.21.15.43:0/1987549565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d4c6dda40 con 0x558d4c4234d0 2024-04-25T10:42:57.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.545+0000 7f957ffff640 1 --2- 172.21.15.43:0/1809550452 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f95800afc60 0x7f95800b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.545+0000 7f95977fe640 1 --2- 172.21.15.43:0/1809550452 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f95800afc60 0x7f95800b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.546+0000 7f957ffff640 1 -- 172.21.15.43:0/1809550452 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f9590100780 con 0x7f9598104cd0 2024-04-25T10:42:57.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.546+0000 7f9596ffd640 1 --2- 172.21.15.43:0/1987549565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c4234d0 0x558d4c7ecfa0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f95900fdf50 tx=0x7f9590106b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf129c32e152a4a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:57.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.546+0000 7f957d7fa640 1 -- 172.21.15.43:0/1987549565 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95900c48f0 con 0x558d4c4234d0 2024-04-25T10:42:57.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.546+0000 7f957d7fa640 1 -- 172.21.15.43:0/1987549565 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9590036070 con 0x558d4c4234d0 2024-04-25T10:42:57.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.546+0000 7f957d7fa640 1 -- 172.21.15.43:0/1987549565 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95900c81a0 con 0x558d4c4234d0 2024-04-25T10:42:57.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.547+0000 7f95a45a0740 1 -- 172.21.15.43:0/1987549565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c4234d0 msgr2=0x558d4c7ecfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:57.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.547+0000 7f95a45a0740 1 --2- 172.21.15.43:0/1987549565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c4234d0 0x558d4c7ecfa0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f95900fdf50 tx=0x7f9590106b40 comp rx=0 tx=0).stop 2024-04-25T10:42:57.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.547+0000 7f95977fe640 1 --2- 172.21.15.43:0/1809550452 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f95800afc60 0x7f95800b2170 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x558d4c7e9bd0 tx=0x7f9588005ec0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:57.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.547+0000 7f95a45a0740 1 -- 172.21.15.43:0/1987549565 shutdown_connections 2024-04-25T10:42:57.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.547+0000 7f95a45a0740 1 --2- 172.21.15.43:0/1987549565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558d4c42b6b0 0x558d4c7f98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.547+0000 7f95a45a0740 1 --2- 172.21.15.43:0/1987549565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c4234d0 0x558d4c7ecfa0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.547+0000 7f95a45a0740 1 --2- 172.21.15.43:0/1987549565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c44c390 0x558d4c7eca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.547+0000 7f95a45a0740 1 -- 172.21.15.43:0/1987549565 >> 172.21.15.43:0/1987549565 conn(0x558d4c4404e0 msgr2=0x558d4c7e9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.547+0000 7f95a45a0740 1 -- 172.21.15.43:0/1987549565 shutdown_connections 2024-04-25T10:42:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.547+0000 7f95a45a0740 1 -- 172.21.15.43:0/1987549565 wait complete. 2024-04-25T10:42:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.547+0000 7f95a45a0740 1 -- start start 2024-04-25T10:42:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.547+0000 7f95a45a0740 1 Processor -- start 2024-04-25T10:42:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.547+0000 7f95a45a0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c4234d0 0x558d4c7f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.547+0000 7f95a45a0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558d4c42b6b0 0x558d4c7f7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.547+0000 7f95a45a0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c44c390 0x558d4c7fd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.547+0000 7f95a45a0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558d4c752d80 con 0x558d4c44c390 2024-04-25T10:42:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.547+0000 7f95a45a0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558d4c6d9d10 con 0x558d4c4234d0 2024-04-25T10:42:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.548+0000 7f95a45a0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558d4c7f9e30 con 0x558d4c42b6b0 2024-04-25T10:42:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.548+0000 7f95a45a0740 10 client.0 _ll_register_callbacks cb 0x558d4c706210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.548+0000 7f95977fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c44c390 0x558d4c7fd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.548+0000 7f95977fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c44c390 0x558d4c7fd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59030/0 (socket says 172.21.15.43:59030) 2024-04-25T10:42:57.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.548+0000 7f95977fe640 1 -- 172.21.15.43:0/3797191834 learned_addr learned my addr 172.21.15.43:0/3797191834 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:57.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.548+0000 7f9596ffd640 1 --2- 172.21.15.43:0/3797191834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c4234d0 0x558d4c7f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.548+0000 7f95967fc640 1 --2- 172.21.15.43:0/3797191834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558d4c42b6b0 0x558d4c7f7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.548+0000 7f95977fe640 1 -- 172.21.15.43:0/3797191834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558d4c42b6b0 msgr2=0x558d4c7f7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:57.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.548+0000 7f95977fe640 1 --2- 172.21.15.43:0/3797191834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558d4c42b6b0 0x558d4c7f7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.548+0000 7f95977fe640 1 -- 172.21.15.43:0/3797191834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c4234d0 msgr2=0x558d4c7f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:57.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.548+0000 7f95977fe640 1 --2- 172.21.15.43:0/3797191834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c4234d0 0x558d4c7f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.548+0000 7f95977fe640 1 -- 172.21.15.43:0/3797191834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d4c52e7d0 con 0x558d4c44c390 2024-04-25T10:42:57.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.548+0000 7f95977fe640 1 --2- 172.21.15.43:0/3797191834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c44c390 0x558d4c7fd0e0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f95880109b0 tx=0x7f95880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8b4de115205b92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:57.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.548+0000 7f957aff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:57.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.548+0000 7f957aff5640 1 -- 172.21.15.43:0/3797191834 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588016070 con 0x558d4c44c390 2024-04-25T10:42:57.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.548+0000 7f957aff5640 1 -- 172.21.15.43:0/3797191834 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f958801bb30 con 0x558d4c44c390 2024-04-25T10:42:57.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.548+0000 7f95a45a0740 1 -- 172.21.15.43:0/3797191834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9590036070 con 0x558d4c44c390 2024-04-25T10:42:57.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.548+0000 7f957aff5640 1 -- 172.21.15.43:0/3797191834 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f958801bd30 con 0x558d4c44c390 2024-04-25T10:42:57.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.549+0000 7f95a45a0740 10 client.18630 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:57.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.549+0000 7f95a45a0740 1 -- 172.21.15.43:0/3797191834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558d4c4643f0 con 0x558d4c44c390 2024-04-25T10:42:57.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.549+0000 7f957aff5640 1 -- 172.21.15.43:0/3797191834 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9590036070 con 0x558d4c44c390 2024-04-25T10:42:57.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.549+0000 7f957aff5640 1 -- 172.21.15.43:0/3797191834 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9588027510 con 0x558d4c44c390 2024-04-25T10:42:57.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.549+0000 7f95a45a0740 10 client.18630 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:57.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.549+0000 7f95a45a0740 10 client.18630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:57.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.549+0000 7f95a45a0740 20 client.18630 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:57.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.549+0000 7f95a45a0740 1 --2- 172.21.15.43:0/3797191834 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558d4c40af60 0x558d4c809320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.549+0000 7f95a45a0740 4 client.18630 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:42:57.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.549+0000 7f95a45a0740 1 -- 172.21.15.43:0/3797191834 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d4c760990 con 0x558d4c40af60 2024-04-25T10:42:57.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.549+0000 7f9596ffd640 1 --2- 172.21.15.43:0/3797191834 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558d4c40af60 0x558d4c809320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.550+0000 7f957aff5640 10 client.18630 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:57.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.550+0000 7f9596ffd640 1 --2- 172.21.15.43:0/3797191834 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558d4c40af60 0x558d4c809320 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d638e4b4897de5e7 server_cookie=c94a1ecd9d336edd in_seq=0 out_seq=0 2024-04-25T10:42:57.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.551+0000 7f957aff5640 1 -- 172.21.15.43:0/3797191834 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558d4c760990 con 0x558d4c40af60 2024-04-25T10:42:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.551+0000 7f957aff5640 10 client.18630 handle_command_reply: tid=0 2024-04-25T10:42:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.551+0000 7f95a45a0740 1 client.18630 shutdown 2024-04-25T10:42:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.551+0000 7f95a45a0740 1 -- 172.21.15.43:0/3797191834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c44c390 msgr2=0x558d4c7fd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.551+0000 7f95a45a0740 1 --2- 172.21.15.43:0/3797191834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c44c390 0x558d4c7fd0e0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f95880109b0 tx=0x7f95880046c0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.551+0000 7f95a45a0740 1 -- 172.21.15.43:0/3797191834 shutdown_connections 2024-04-25T10:42:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.551+0000 7f95a45a0740 1 --2- 172.21.15.43:0/3797191834 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558d4c40af60 0x558d4c809320 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.551+0000 7f95a45a0740 1 --2- 172.21.15.43:0/3797191834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c44c390 0x558d4c7fd0e0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.551+0000 7f95a45a0740 1 --2- 172.21.15.43:0/3797191834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558d4c42b6b0 0x558d4c7f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.551+0000 7f95a45a0740 1 --2- 172.21.15.43:0/3797191834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c4234d0 0x558d4c7f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.551+0000 7f95a45a0740 1 -- 172.21.15.43:0/3797191834 >> 172.21.15.43:0/3797191834 conn(0x558d4c4404e0 msgr2=0x558d4c7ee780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.551+0000 7f957aff5640 0 client.18630 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.551+0000 7f95a45a0740 1 -- 172.21.15.43:0/3797191834 shutdown_connections 2024-04-25T10:42:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.551+0000 7f95a45a0740 1 -- 172.21.15.43:0/3797191834 wait complete. 2024-04-25T10:42:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.551+0000 7f95a45a0740 20 client.18630 trim_cache size 0 max 16384 2024-04-25T10:42:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.569+0000 7f95a45a0740 1 Processor -- start 2024-04-25T10:42:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.569+0000 7f95a45a0740 1 -- start start 2024-04-25T10:42:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.569+0000 7f95a45a0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c4234d0 0x558d4c7f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.569+0000 7f95a45a0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558d4c42b6b0 0x558d4c7ecf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.570+0000 7f95a45a0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c44c390 0x558d4c80eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.570+0000 7f95a45a0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f95981b44a0 con 0x558d4c4234d0 2024-04-25T10:42:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.570+0000 7f95a45a0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f95981b1d70 con 0x558d4c44c390 2024-04-25T10:42:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.570+0000 7f95a45a0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f95981afbe0 con 0x558d4c42b6b0 2024-04-25T10:42:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.570+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c4234d0 0x558d4c7f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.570+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c4234d0 0x558d4c7f2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59036/0 (socket says 172.21.15.43:59036) 2024-04-25T10:42:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.570+0000 7f9596ffd640 1 -- 172.21.15.43:0/1357826408 learned_addr learned my addr 172.21.15.43:0/1357826408 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.570+0000 7f95977fe640 1 --2- 172.21.15.43:0/1357826408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c44c390 0x558d4c80eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.570+0000 7f9596ffd640 1 -- 172.21.15.43:0/1357826408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558d4c42b6b0 msgr2=0x558d4c7ecf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.570+0000 7f9596ffd640 1 --2- 172.21.15.43:0/1357826408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558d4c42b6b0 0x558d4c7ecf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.570+0000 7f9596ffd640 1 -- 172.21.15.43:0/1357826408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c44c390 msgr2=0x558d4c80eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.570+0000 7f9596ffd640 1 --2- 172.21.15.43:0/1357826408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c44c390 0x558d4c80eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.570+0000 7f9596ffd640 1 -- 172.21.15.43:0/1357826408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d4c4240f0 con 0x558d4c4234d0 2024-04-25T10:42:57.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.570+0000 7f9596ffd640 1 --2- 172.21.15.43:0/1357826408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c4234d0 0x558d4c7f2c90 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f95900fe090 tx=0x7f959002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8da81b65f8d40c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:57.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.570+0000 7f957cff9640 1 -- 172.21.15.43:0/1357826408 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95900e2070 con 0x558d4c4234d0 2024-04-25T10:42:57.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.570+0000 7f957cff9640 1 -- 172.21.15.43:0/1357826408 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9590036050 con 0x558d4c4234d0 2024-04-25T10:42:57.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.570+0000 7f957cff9640 1 -- 172.21.15.43:0/1357826408 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95900c8380 con 0x558d4c4234d0 2024-04-25T10:42:57.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.571+0000 7f95a45a0740 1 -- 172.21.15.43:0/1357826408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c4234d0 msgr2=0x558d4c7f2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:57.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.571+0000 7f95a45a0740 1 --2- 172.21.15.43:0/1357826408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c4234d0 0x558d4c7f2c90 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f95900fe090 tx=0x7f959002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.571+0000 7f95a45a0740 1 -- 172.21.15.43:0/1357826408 shutdown_connections 2024-04-25T10:42:57.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.571+0000 7f95a45a0740 1 --2- 172.21.15.43:0/1357826408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c44c390 0x558d4c80eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.571+0000 7f95a45a0740 1 --2- 172.21.15.43:0/1357826408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558d4c42b6b0 0x558d4c7ecf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.571+0000 7f95a45a0740 1 --2- 172.21.15.43:0/1357826408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c4234d0 0x558d4c7f2c90 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.571+0000 7f95a45a0740 1 -- 172.21.15.43:0/1357826408 >> 172.21.15.43:0/1357826408 conn(0x558d4c4404e0 msgr2=0x558d4c7ec7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:57.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.571+0000 7f95a45a0740 1 -- 172.21.15.43:0/1357826408 shutdown_connections 2024-04-25T10:42:57.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.571+0000 7f95a45a0740 1 -- 172.21.15.43:0/1357826408 wait complete. 2024-04-25T10:42:57.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.571+0000 7f95a45a0740 1 -- start start 2024-04-25T10:42:57.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.571+0000 7f95a45a0740 1 Processor -- start 2024-04-25T10:42:57.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.572+0000 7f95a45a0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c4234d0 0x558d4c7fe1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.572+0000 7f95a45a0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558d4c42b6b0 0x558d4c7fe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.572+0000 7f95a45a0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c44c390 0x558d4c8696e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.572+0000 7f95a45a0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f95981b0230 con 0x558d4c44c390 2024-04-25T10:42:57.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.572+0000 7f95a45a0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f95981b16a0 con 0x558d4c4234d0 2024-04-25T10:42:57.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.572+0000 7f95a45a0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f95981b0900 con 0x558d4c42b6b0 2024-04-25T10:42:57.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.572+0000 7f95a45a0740 10 client.0 _ll_register_callbacks cb 0x7f958800a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:42:57.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.572+0000 7f95977fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c44c390 0x558d4c8696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.572+0000 7f95977fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c44c390 0x558d4c8696e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59048/0 (socket says 172.21.15.43:59048) 2024-04-25T10:42:57.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.572+0000 7f95977fe640 1 -- 172.21.15.43:0/2127198527 learned_addr learned my addr 172.21.15.43:0/2127198527 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:42:57.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.572+0000 7f9596ffd640 1 --2- 172.21.15.43:0/2127198527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c4234d0 0x558d4c7fe1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.572+0000 7f95977fe640 1 -- 172.21.15.43:0/2127198527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558d4c42b6b0 msgr2=0x558d4c7fe700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:57.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.572+0000 7f95977fe640 1 --2- 172.21.15.43:0/2127198527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558d4c42b6b0 0x558d4c7fe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.572+0000 7f95977fe640 1 -- 172.21.15.43:0/2127198527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c4234d0 msgr2=0x558d4c7fe1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:42:57.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.572+0000 7f95977fe640 1 --2- 172.21.15.43:0/2127198527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c4234d0 0x558d4c7fe1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.572+0000 7f95977fe640 1 -- 172.21.15.43:0/2127198527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d4c7134b0 con 0x558d4c44c390 2024-04-25T10:42:57.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.572+0000 7f9596ffd640 1 --2- 172.21.15.43:0/2127198527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c4234d0 0x558d4c7fe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:42:57.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.572+0000 7f95977fe640 1 --2- 172.21.15.43:0/2127198527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c44c390 0x558d4c8696e0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f958800fe50 tx=0x7f9588027da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ede75d9c29d0055 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:42:57.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.572+0000 7f957aff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:42:57.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.573+0000 7f957aff5640 1 -- 172.21.15.43:0/2127198527 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958803e060 con 0x558d4c44c390 2024-04-25T10:42:57.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.573+0000 7f957aff5640 1 -- 172.21.15.43:0/2127198527 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f958801b990 con 0x558d4c44c390 2024-04-25T10:42:57.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.573+0000 7f95a45a0740 1 -- 172.21.15.43:0/2127198527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9590036050 con 0x558d4c44c390 2024-04-25T10:42:57.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.573+0000 7f957aff5640 1 -- 172.21.15.43:0/2127198527 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f958801bb90 con 0x558d4c44c390 2024-04-25T10:42:57.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.573+0000 7f95a45a0740 10 client.18636 fetch_fsmap learned FSMap version 26 2024-04-25T10:42:57.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.573+0000 7f95a45a0740 1 -- 172.21.15.43:0/2127198527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558d4c870d90 con 0x558d4c44c390 2024-04-25T10:42:57.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.573+0000 7f957aff5640 1 -- 172.21.15.43:0/2127198527 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9590036050 con 0x558d4c44c390 2024-04-25T10:42:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.573+0000 7f957aff5640 1 -- 172.21.15.43:0/2127198527 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9588039450 con 0x558d4c44c390 2024-04-25T10:42:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.573+0000 7f95a45a0740 10 client.18636 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:42:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.573+0000 7f95a45a0740 10 client.18636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:42:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.573+0000 7f95a45a0740 20 client.18636 populate_metadata read hostname 'smithi043' 2024-04-25T10:42:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.573+0000 7f95a45a0740 1 --2- 172.21.15.43:0/2127198527 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558d4c3e03a0 0x558d4c873480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:42:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.573+0000 7f95a45a0740 4 client.18636 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:42:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.573+0000 7f95a45a0740 1 -- 172.21.15.43:0/2127198527 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558d4c75a800 con 0x558d4c3e03a0 2024-04-25T10:42:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.573+0000 7f9596ffd640 1 --2- 172.21.15.43:0/2127198527 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558d4c3e03a0 0x558d4c873480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:42:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.574+0000 7f957aff5640 10 client.18636 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.574+0000 7f9596ffd640 1 --2- 172.21.15.43:0/2127198527 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558d4c3e03a0 0x558d4c873480 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b52e8ad730832fd5 server_cookie=7d1b7d52bb157b91 in_seq=0 out_seq=0 2024-04-25T10:42:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.574+0000 7f957aff5640 1 -- 172.21.15.43:0/2127198527 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558d4c75a800 con 0x558d4c3e03a0 2024-04-25T10:42:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.574+0000 7f957aff5640 10 client.18636 handle_command_reply: tid=0 2024-04-25T10:42:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.574+0000 7f95a45a0740 1 client.18636 shutdown 2024-04-25T10:42:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.574+0000 7f95a45a0740 1 -- 172.21.15.43:0/2127198527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c44c390 msgr2=0x558d4c8696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:57.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.574+0000 7f95a45a0740 1 --2- 172.21.15.43:0/2127198527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c44c390 0x558d4c8696e0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f958800fe50 tx=0x7f9588027da0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.574+0000 7f95a45a0740 1 -- 172.21.15.43:0/2127198527 shutdown_connections 2024-04-25T10:42:57.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.574+0000 7f95a45a0740 1 --2- 172.21.15.43:0/2127198527 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x558d4c3e03a0 0x558d4c873480 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.574+0000 7f95a45a0740 1 --2- 172.21.15.43:0/2127198527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558d4c44c390 0x558d4c8696e0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.574+0000 7f95a45a0740 1 --2- 172.21.15.43:0/2127198527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558d4c42b6b0 0x558d4c7fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.574+0000 7f957aff5640 0 client.18636 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:42:57.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.574+0000 7f95a45a0740 1 --2- 172.21.15.43:0/2127198527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d4c4234d0 0x558d4c7fe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.574+0000 7f95a45a0740 1 -- 172.21.15.43:0/2127198527 >> 172.21.15.43:0/2127198527 conn(0x558d4c4404e0 msgr2=0x558d4c7ee660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:57.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.574+0000 7f95a45a0740 1 -- 172.21.15.43:0/2127198527 shutdown_connections 2024-04-25T10:42:57.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.574+0000 7f95a45a0740 1 -- 172.21.15.43:0/2127198527 wait complete. 2024-04-25T10:42:57.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.574+0000 7f95a45a0740 20 client.18636 trim_cache size 0 max 16384 2024-04-25T10:42:57.584 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:42:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:42:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:42:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: "533d5220-b8a4-4d37-8903-ded1f0555a71": { 2024-04-25T10:42:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:42:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "533d5220-b8a4-4d37-8903-ded1f0555a71", 2024-04-25T10:42:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:42:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:42:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:42:57.584 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:42:57.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.575+0000 7f959d6ee640 1 -- 172.21.15.43:0/1809550452 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f95800afc60 msgr2=0x7f95800b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:57.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.575+0000 7f959d6ee640 1 --2- 172.21.15.43:0/1809550452 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f95800afc60 0x7f95800b2170 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x558d4c7e9bd0 tx=0x7f9588005ec0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.575+0000 7f959d6ee640 1 -- 172.21.15.43:0/1809550452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9598104cd0 msgr2=0x7f9598102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:42:57.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.575+0000 7f959d6ee640 1 --2- 172.21.15.43:0/1809550452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9598104cd0 0x7f9598102590 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f959002ade0 tx=0x7f9590003970 comp rx=0 tx=0).stop 2024-04-25T10:42:57.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.575+0000 7f959d6ee640 1 -- 172.21.15.43:0/1809550452 shutdown_connections 2024-04-25T10:42:57.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.575+0000 7f959d6ee640 1 --2- 172.21.15.43:0/1809550452 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f95800afc60 0x7f95800b2170 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.575+0000 7f959d6ee640 1 --2- 172.21.15.43:0/1809550452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f959810b730 0x7f9598102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.575+0000 7f959d6ee640 1 --2- 172.21.15.43:0/1809550452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f959810e090 0x7f9598103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.575+0000 7f959d6ee640 1 --2- 172.21.15.43:0/1809550452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9598104cd0 0x7f9598102590 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:42:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.575+0000 7f959d6ee640 1 -- 172.21.15.43:0/1809550452 >> 172.21.15.43:0/1809550452 conn(0x7f9598078110 msgr2=0x7f9598101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:42:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.575+0000 7f959d6ee640 1 -- 172.21.15.43:0/1809550452 shutdown_connections 2024-04-25T10:42:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:42:57.575+0000 7f959d6ee640 1 -- 172.21.15.43:0/1809550452 wait complete. 2024-04-25T10:42:57.588 INFO:tasks.cephfs.filesystem:scrub status for tag:533d5220-b8a4-4d37-8903-ded1f0555a71 - {'path': '/', 'tag': '533d5220-b8a4-4d37-8903-ded1f0555a71', 'options': 'recursive,force'} 2024-04-25T10:42:58.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:58 smithi077 ceph-mon[32772]: pgmap v324: 97 pgs: 97 active+clean; 3.8 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-25T10:42:58.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:58 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:42:58.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:58 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:42:58.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:58 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:42:58.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:58 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:42:58.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:58 smithi164 ceph-mon[32713]: pgmap v324: 97 pgs: 97 active+clean; 3.8 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-25T10:42:58.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:58 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:42:58.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:58 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:42:58.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:58 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:42:58.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:58 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:42:58.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:58 smithi043 ceph-mon[29478]: pgmap v324: 97 pgs: 97 active+clean; 3.8 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-25T10:42:58.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:58 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:42:58.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:58 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:42:58.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:58 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:42:58.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:58 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:42:59.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:42:59 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:42:59.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:42:59 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:42:59.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:42:59 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:43:00.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:00 smithi043 ceph-mon[29478]: pgmap v325: 97 pgs: 97 active+clean; 4.0 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-04-25T10:43:00.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:00 smithi077 ceph-mon[32772]: pgmap v325: 97 pgs: 97 active+clean; 4.0 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-04-25T10:43:00.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:00 smithi164 ceph-mon[32713]: pgmap v325: 97 pgs: 97 active+clean; 4.0 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-04-25T10:43:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.280+0000 7f499fa5a640 1 -- 172.21.15.43:0/1740871169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4998076470 msgr2=0x7f4998076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.280+0000 7f499fa5a640 1 --2- 172.21.15.43:0/1740871169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4998076470 0x7f4998076880 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f4994009900 tx=0x7f499402a480 comp rx=0 tx=0).stop 2024-04-25T10:43:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.280+0000 7f499fa5a640 1 -- 172.21.15.43:0/1740871169 shutdown_connections 2024-04-25T10:43:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.280+0000 7f499fa5a640 1 --2- 172.21.15.43:0/1740871169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4998114b20 0x7f4998116f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.280+0000 7f499fa5a640 1 --2- 172.21.15.43:0/1740871169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49981121c0 0x7f49981145e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.280+0000 7f499fa5a640 1 --2- 172.21.15.43:0/1740871169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4998076470 0x7f4998076880 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.280+0000 7f499fa5a640 1 -- 172.21.15.43:0/1740871169 >> 172.21.15.43:0/1740871169 conn(0x7f49980ff220 msgr2=0x7f4998101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.280+0000 7f499fa5a640 1 -- 172.21.15.43:0/1740871169 shutdown_connections 2024-04-25T10:43:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.281+0000 7f499fa5a640 1 -- 172.21.15.43:0/1740871169 wait complete. 2024-04-25T10:43:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.281+0000 7f499fa5a640 1 Processor -- start 2024-04-25T10:43:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.281+0000 7f499fa5a640 1 -- start start 2024-04-25T10:43:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.282+0000 7f499fa5a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4998076470 0x7f49981a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.282+0000 7f499fa5a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49981121c0 0x7f49981a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.282+0000 7f499fa5a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4998114b20 0x7f49981a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.282+0000 7f499fa5a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f499811a2f0 con 0x7f4998076470 2024-04-25T10:43:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.282+0000 7f499fa5a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f499811a170 con 0x7f4998114b20 2024-04-25T10:43:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.282+0000 7f499fa5a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f499811a470 con 0x7f49981121c0 2024-04-25T10:43:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.282+0000 7f499d7f6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4998076470 0x7f49981a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.282+0000 7f499d7f6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4998076470 0x7f49981a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39268/0 (socket says 172.21.15.43:39268) 2024-04-25T10:43:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.282+0000 7f499d7f6640 1 -- 172.21.15.43:0/4238734651 learned_addr learned my addr 172.21.15.43:0/4238734651 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.282+0000 7f499dff7640 1 --2- 172.21.15.43:0/4238734651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4998114b20 0x7f49981a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.282+0000 7f499cff5640 1 --2- 172.21.15.43:0/4238734651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49981121c0 0x7f49981a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.282+0000 7f499d7f6640 1 -- 172.21.15.43:0/4238734651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49981121c0 msgr2=0x7f49981a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.282+0000 7f499d7f6640 1 --2- 172.21.15.43:0/4238734651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49981121c0 0x7f49981a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.282+0000 7f499d7f6640 1 -- 172.21.15.43:0/4238734651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4998114b20 msgr2=0x7f49981a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.282+0000 7f499d7f6640 1 --2- 172.21.15.43:0/4238734651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4998114b20 0x7f49981a5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.282+0000 7f499d7f6640 1 -- 172.21.15.43:0/4238734651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49981a5b50 con 0x7f4998076470 2024-04-25T10:43:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.282+0000 7f499d7f6640 1 --2- 172.21.15.43:0/4238734651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4998076470 0x7f49981a0d70 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f4994006010 tx=0x7f4994004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.282+0000 7f498e7fc640 1 -- 172.21.15.43:0/4238734651 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499403f030 con 0x7f4998076470 2024-04-25T10:43:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.282+0000 7f499fa5a640 1 -- 172.21.15.43:0/4238734651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49981a5de0 con 0x7f4998076470 2024-04-25T10:43:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.283+0000 7f498e7fc640 1 -- 172.21.15.43:0/4238734651 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4994003aa0 con 0x7f4998076470 2024-04-25T10:43:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.283+0000 7f498e7fc640 1 -- 172.21.15.43:0/4238734651 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994035070 con 0x7f4998076470 2024-04-25T10:43:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.283+0000 7f499fa5a640 1 -- 172.21.15.43:0/4238734651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49981a6320 con 0x7f4998076470 2024-04-25T10:43:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.286+0000 7f49a69ea740 1 Processor -- start 2024-04-25T10:43:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.286+0000 7f49a69ea740 1 -- start start 2024-04-25T10:43:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.286+0000 7f498e7fc640 1 -- 172.21.15.43:0/4238734651 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f4994042030 con 0x7f4998076470 2024-04-25T10:43:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.286+0000 7f49a69ea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adac0201b0 0x55adac3bfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.286+0000 7f49a69ea740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adabff72f0 0x55adac3c0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.286+0000 7f49a69ea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adabfff4d0 0x55adac3c0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.286+0000 7f49a69ea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55adac326460 con 0x55adabfff4d0 2024-04-25T10:43:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.286+0000 7f49a69ea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55adac2e0590 con 0x55adac0201b0 2024-04-25T10:43:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.286+0000 7f49a69ea740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55adac322420 con 0x55adabff72f0 2024-04-25T10:43:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.287+0000 7f499cff5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adabfff4d0 0x55adac3c0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.287+0000 7f499cff5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adabfff4d0 0x55adac3c0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39282/0 (socket says 172.21.15.43:39282) 2024-04-25T10:43:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.287+0000 7f499cff5640 1 -- 172.21.15.43:0/477938971 learned_addr learned my addr 172.21.15.43:0/477938971 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.287+0000 7f499dff7640 1 --2- 172.21.15.43:0/477938971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adac0201b0 0x55adac3bfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.287+0000 7f499d7f6640 1 --2- 172.21.15.43:0/477938971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adabff72f0 0x55adac3c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.287+0000 7f499cff5640 1 -- 172.21.15.43:0/477938971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adabff72f0 msgr2=0x55adac3c0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.287+0000 7f499cff5640 1 --2- 172.21.15.43:0/477938971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adabff72f0 0x55adac3c0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.287+0000 7f499cff5640 1 -- 172.21.15.43:0/477938971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adac0201b0 msgr2=0x55adac3bfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.287+0000 7f499cff5640 1 --2- 172.21.15.43:0/477938971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adac0201b0 0x55adac3bfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.287+0000 7f499cff5640 1 -- 172.21.15.43:0/477938971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adac2ed0d0 con 0x55adabfff4d0 2024-04-25T10:43:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.287+0000 7f498e7fc640 1 --2- 172.21.15.43:0/4238734651 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f49840afc50 0x7f49840b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.287+0000 7f499cff5640 1 --2- 172.21.15.43:0/477938971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adabfff4d0 0x55adac3c0570 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f498800cc40 tx=0x7f498800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75bf1b2ed452f9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.287+0000 7f499dff7640 1 --2- 172.21.15.43:0/4238734651 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f49840afc50 0x7f49840b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.287+0000 7f498e7fc640 1 -- 172.21.15.43:0/4238734651 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f49940ff710 con 0x7f4998076470 2024-04-25T10:43:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.287+0000 7f49837fe640 1 -- 172.21.15.43:0/477938971 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4988014070 con 0x55adabfff4d0 2024-04-25T10:43:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.288+0000 7f49837fe640 1 -- 172.21.15.43:0/477938971 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49880047a0 con 0x55adabfff4d0 2024-04-25T10:43:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.288+0000 7f49837fe640 1 -- 172.21.15.43:0/477938971 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4988010690 con 0x55adabfff4d0 2024-04-25T10:43:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.288+0000 7f49a69ea740 1 -- 172.21.15.43:0/477938971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adabfff4d0 msgr2=0x55adac3c0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.288+0000 7f499dff7640 1 --2- 172.21.15.43:0/4238734651 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f49840afc50 0x7f49840b2160 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55adac3bcc60 tx=0x7f4990009500 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.288+0000 7f49a69ea740 1 --2- 172.21.15.43:0/477938971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adabfff4d0 0x55adac3c0570 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f498800cc40 tx=0x7f498800e720 comp rx=0 tx=0).stop 2024-04-25T10:43:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.288+0000 7f49a69ea740 1 -- 172.21.15.43:0/477938971 shutdown_connections 2024-04-25T10:43:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.288+0000 7f49a69ea740 1 --2- 172.21.15.43:0/477938971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adabfff4d0 0x55adac3c0570 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.288+0000 7f49a69ea740 1 --2- 172.21.15.43:0/477938971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adabff72f0 0x55adac3c0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.288+0000 7f49a69ea740 1 --2- 172.21.15.43:0/477938971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adac0201b0 0x55adac3bfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.288+0000 7f49a69ea740 1 -- 172.21.15.43:0/477938971 >> 172.21.15.43:0/477938971 conn(0x55adac014300 msgr2=0x55adac3bc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.289+0000 7f49a69ea740 1 -- 172.21.15.43:0/477938971 shutdown_connections 2024-04-25T10:43:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.289+0000 7f49a69ea740 1 -- 172.21.15.43:0/477938971 wait complete. 2024-04-25T10:43:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.289+0000 7f49a69ea740 1 -- start start 2024-04-25T10:43:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.289+0000 7f49a69ea740 1 Processor -- start 2024-04-25T10:43:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.290+0000 7f49a69ea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adabff72f0 0x55adac3ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.290+0000 7f49a69ea740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adabfff4d0 0x55adac3ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.290+0000 7f49a69ea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adac0201b0 0x55adac3d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.290+0000 7f49a69ea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55adac3ccc20 con 0x55adac0201b0 2024-04-25T10:43:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.290+0000 7f49a69ea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55adac2adb60 con 0x55adabff72f0 2024-04-25T10:43:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.290+0000 7f49a69ea740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55adac3ccda0 con 0x55adabfff4d0 2024-04-25T10:43:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.290+0000 7f49a69ea740 10 client.0 _ll_register_callbacks cb 0x55adac2d22b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.290+0000 7f499dff7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adac0201b0 0x55adac3d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.290+0000 7f499dff7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adac0201b0 0x55adac3d0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39298/0 (socket says 172.21.15.43:39298) 2024-04-25T10:43:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.290+0000 7f499dff7640 1 -- 172.21.15.43:0/3946941405 learned_addr learned my addr 172.21.15.43:0/3946941405 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.290+0000 7f499cff5640 1 --2- 172.21.15.43:0/3946941405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adabfff4d0 0x55adac3ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.290+0000 7f499d7f6640 1 --2- 172.21.15.43:0/3946941405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adabff72f0 0x55adac3ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.290+0000 7f499dff7640 1 -- 172.21.15.43:0/3946941405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adabfff4d0 msgr2=0x55adac3ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.290+0000 7f499dff7640 1 --2- 172.21.15.43:0/3946941405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adabfff4d0 0x55adac3ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.290+0000 7f499dff7640 1 -- 172.21.15.43:0/3946941405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adabff72f0 msgr2=0x55adac3ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.290+0000 7f499dff7640 1 --2- 172.21.15.43:0/3946941405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adabff72f0 0x55adac3ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.290+0000 7f499dff7640 1 -- 172.21.15.43:0/3946941405 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adac038210 con 0x55adac0201b0 2024-04-25T10:43:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.290+0000 7f499dff7640 1 --2- 172.21.15.43:0/3946941405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adac0201b0 0x55adac3d0110 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f4990012010 tx=0x7f4990004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab9c18307bac597 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.290+0000 7f4980ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.290+0000 7f4980ff9640 1 -- 172.21.15.43:0/3946941405 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499000f7d0 con 0x55adac0201b0 2024-04-25T10:43:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.291+0000 7f4980ff9640 1 -- 172.21.15.43:0/3946941405 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f499000f970 con 0x55adac0201b0 2024-04-25T10:43:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.291+0000 7f49a69ea740 1 -- 172.21.15.43:0/3946941405 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49880047a0 con 0x55adac0201b0 2024-04-25T10:43:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.291+0000 7f4980ff9640 1 -- 172.21.15.43:0/3946941405 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f499000fb70 con 0x55adac0201b0 2024-04-25T10:43:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.291+0000 7f49a69ea740 10 client.18648 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.291+0000 7f49a69ea740 1 -- 172.21.15.43:0/3946941405 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55adac104620 con 0x55adac0201b0 2024-04-25T10:43:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.291+0000 7f4980ff9640 1 -- 172.21.15.43:0/3946941405 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49880047a0 con 0x55adac0201b0 2024-04-25T10:43:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.291+0000 7f4980ff9640 1 -- 172.21.15.43:0/3946941405 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4990026070 con 0x55adac0201b0 2024-04-25T10:43:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.291+0000 7f49a69ea740 10 client.18648 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.291+0000 7f49a69ea740 10 client.18648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:43:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.291+0000 7f49a69ea740 20 client.18648 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.291+0000 7f49a69ea740 1 --2- 172.21.15.43:0/3946941405 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55adabfded80 0x55adac3dc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.291+0000 7f49a69ea740 4 client.18648 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:43:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.291+0000 7f49a69ea740 1 -- 172.21.15.43:0/3946941405 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adac334070 con 0x55adabfded80 2024-04-25T10:43:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.292+0000 7f499d7f6640 1 --2- 172.21.15.43:0/3946941405 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55adabfded80 0x55adac3dc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.292+0000 7f499d7f6640 1 --2- 172.21.15.43:0/3946941405 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55adabfded80 0x55adac3dc800 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29453bfb5bb5b6d5 server_cookie=cb2953b1dfc9a856 in_seq=0 out_seq=0 2024-04-25T10:43:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.292+0000 7f4980ff9640 10 client.18648 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.293+0000 7f4980ff9640 1 -- 172.21.15.43:0/3946941405 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55adac334070 con 0x55adabfded80 2024-04-25T10:43:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.293+0000 7f4980ff9640 10 client.18648 handle_command_reply: tid=0 2024-04-25T10:43:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.293+0000 7f49a69ea740 1 client.18648 shutdown 2024-04-25T10:43:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.293+0000 7f49a69ea740 1 -- 172.21.15.43:0/3946941405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adac0201b0 msgr2=0x55adac3d0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.293+0000 7f49a69ea740 1 --2- 172.21.15.43:0/3946941405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adac0201b0 0x55adac3d0110 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f4990012010 tx=0x7f4990004ab0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.293+0000 7f49a69ea740 1 -- 172.21.15.43:0/3946941405 shutdown_connections 2024-04-25T10:43:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.293+0000 7f49a69ea740 1 --2- 172.21.15.43:0/3946941405 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55adabfded80 0x55adac3dc800 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.293+0000 7f49a69ea740 1 --2- 172.21.15.43:0/3946941405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adac0201b0 0x55adac3d0110 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.293+0000 7f49a69ea740 1 --2- 172.21.15.43:0/3946941405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adabfff4d0 0x55adac3ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.293+0000 7f49a69ea740 1 --2- 172.21.15.43:0/3946941405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adabff72f0 0x55adac3ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.293+0000 7f4980ff9640 0 client.18648 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.293+0000 7f49a69ea740 1 -- 172.21.15.43:0/3946941405 >> 172.21.15.43:0/3946941405 conn(0x55adac014300 msgr2=0x55adac3c1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.293+0000 7f49a69ea740 1 -- 172.21.15.43:0/3946941405 shutdown_connections 2024-04-25T10:43:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.293+0000 7f49a69ea740 1 -- 172.21.15.43:0/3946941405 wait complete. 2024-04-25T10:43:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.293+0000 7f49a69ea740 20 client.18648 trim_cache size 0 max 16384 2024-04-25T10:43:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.308+0000 7f49a69ea740 1 Processor -- start 2024-04-25T10:43:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.309+0000 7f49a69ea740 1 -- start start 2024-04-25T10:43:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.309+0000 7f49a69ea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adabff72f0 0x55adac3d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.309+0000 7f49a69ea740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adabfff4d0 0x55adac3c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.309+0000 7f49a69ea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adac0201b0 0x55adac3c3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.309+0000 7f49a69ea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49981b8840 con 0x55adac0201b0 2024-04-25T10:43:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.309+0000 7f49a69ea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f49981b6110 con 0x55adabff72f0 2024-04-25T10:43:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.309+0000 7f49a69ea740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f49981b3f80 con 0x55adabfff4d0 2024-04-25T10:43:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.309+0000 7f499dff7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adac0201b0 0x55adac3c3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.309+0000 7f499dff7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adac0201b0 0x55adac3c3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39312/0 (socket says 172.21.15.43:39312) 2024-04-25T10:43:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.309+0000 7f499dff7640 1 -- 172.21.15.43:0/1976236814 learned_addr learned my addr 172.21.15.43:0/1976236814 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.309+0000 7f499d7f6640 1 --2- 172.21.15.43:0/1976236814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adabff72f0 0x55adac3d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.309+0000 7f499cff5640 1 --2- 172.21.15.43:0/1976236814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adabfff4d0 0x55adac3c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.309+0000 7f499dff7640 1 -- 172.21.15.43:0/1976236814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adabfff4d0 msgr2=0x55adac3c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.309+0000 7f499dff7640 1 --2- 172.21.15.43:0/1976236814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adabfff4d0 0x55adac3c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.309+0000 7f499dff7640 1 -- 172.21.15.43:0/1976236814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adabff72f0 msgr2=0x55adac3d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.309+0000 7f499dff7640 1 --2- 172.21.15.43:0/1976236814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adabff72f0 0x55adac3d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.309+0000 7f499dff7640 1 -- 172.21.15.43:0/1976236814 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adac3c4bd0 con 0x55adac0201b0 2024-04-25T10:43:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.309+0000 7f499d7f6640 1 --2- 172.21.15.43:0/1976236814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adabff72f0 0x55adac3d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:43:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.309+0000 7f499cff5640 1 --2- 172.21.15.43:0/1976236814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adabfff4d0 0x55adac3c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:43:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.309+0000 7f499dff7640 1 --2- 172.21.15.43:0/1976236814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adac0201b0 0x55adac3c3850 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f4990000f80 tx=0x7f4990005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f8c9a6891cd298 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.310+0000 7f4982ffd640 1 -- 172.21.15.43:0/1976236814 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499000f7d0 con 0x55adac0201b0 2024-04-25T10:43:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.310+0000 7f4982ffd640 1 -- 172.21.15.43:0/1976236814 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f499000f970 con 0x55adac0201b0 2024-04-25T10:43:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.310+0000 7f4982ffd640 1 -- 172.21.15.43:0/1976236814 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4990028a90 con 0x55adac0201b0 2024-04-25T10:43:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.310+0000 7f49a69ea740 1 -- 172.21.15.43:0/1976236814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adac0201b0 msgr2=0x55adac3c3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.310+0000 7f49a69ea740 1 --2- 172.21.15.43:0/1976236814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adac0201b0 0x55adac3c3850 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f4990000f80 tx=0x7f4990005c70 comp rx=0 tx=0).stop 2024-04-25T10:43:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.310+0000 7f49a69ea740 1 -- 172.21.15.43:0/1976236814 shutdown_connections 2024-04-25T10:43:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.310+0000 7f49a69ea740 1 --2- 172.21.15.43:0/1976236814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adac0201b0 0x55adac3c3850 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.310+0000 7f49a69ea740 1 --2- 172.21.15.43:0/1976236814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adabfff4d0 0x55adac3c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.310+0000 7f49a69ea740 1 --2- 172.21.15.43:0/1976236814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adabff72f0 0x55adac3d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.310+0000 7f49a69ea740 1 -- 172.21.15.43:0/1976236814 >> 172.21.15.43:0/1976236814 conn(0x55adac014300 msgr2=0x55adac3c8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.310+0000 7f49a69ea740 1 -- 172.21.15.43:0/1976236814 shutdown_connections 2024-04-25T10:43:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.310+0000 7f49a69ea740 1 -- 172.21.15.43:0/1976236814 wait complete. 2024-04-25T10:43:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.311+0000 7f49a69ea740 1 -- start start 2024-04-25T10:43:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.311+0000 7f49a69ea740 1 Processor -- start 2024-04-25T10:43:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.311+0000 7f49a69ea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adabff72f0 0x55adac3d6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.311+0000 7f49a69ea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adabfff4d0 0x55adac3d70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.311+0000 7f49a69ea740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adac0201b0 0x55adac3d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.311+0000 7f49a69ea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49981b45d0 con 0x55adabff72f0 2024-04-25T10:43:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.311+0000 7f49a69ea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f49981b5a40 con 0x55adabfff4d0 2024-04-25T10:43:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.311+0000 7f49a69ea740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f49981b4ca0 con 0x55adac0201b0 2024-04-25T10:43:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.311+0000 7f49a69ea740 10 client.0 _ll_register_callbacks cb 0x7f4990009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.311+0000 7f499d7f6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adabff72f0 0x55adac3d6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.312+0000 7f499d7f6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adabff72f0 0x55adac3d6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39316/0 (socket says 172.21.15.43:39316) 2024-04-25T10:43:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.312+0000 7f499d7f6640 1 -- 172.21.15.43:0/43170289 learned_addr learned my addr 172.21.15.43:0/43170289 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.312+0000 7f499dff7640 1 --2- 172.21.15.43:0/43170289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adac0201b0 0x55adac3d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.312+0000 7f499cff5640 1 --2- 172.21.15.43:0/43170289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adabfff4d0 0x55adac3d70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.312+0000 7f499d7f6640 1 -- 172.21.15.43:0/43170289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adac0201b0 msgr2=0x55adac3d1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.312+0000 7f499d7f6640 1 --2- 172.21.15.43:0/43170289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adac0201b0 0x55adac3d1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.312+0000 7f499d7f6640 1 -- 172.21.15.43:0/43170289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adabfff4d0 msgr2=0x55adac3d70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.312+0000 7f499d7f6640 1 --2- 172.21.15.43:0/43170289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adabfff4d0 0x55adac3d70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.312+0000 7f499d7f6640 1 -- 172.21.15.43:0/43170289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adac3d26d0 con 0x55adabff72f0 2024-04-25T10:43:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.312+0000 7f499cff5640 1 --2- 172.21.15.43:0/43170289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adabfff4d0 0x55adac3d70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:43:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.312+0000 7f499d7f6640 1 --2- 172.21.15.43:0/43170289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adabff72f0 0x55adac3d6b80 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f49940c75b0 tx=0x7f4994101c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2983cca071fb4c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.312+0000 7f4980ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.312+0000 7f4980ff9640 1 -- 172.21.15.43:0/43170289 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994049040 con 0x55adabff72f0 2024-04-25T10:43:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.312+0000 7f4980ff9640 1 -- 172.21.15.43:0/43170289 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f49940fd0e0 con 0x55adabff72f0 2024-04-25T10:43:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.312+0000 7f49a69ea740 1 -- 172.21.15.43:0/43170289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f499000f970 con 0x55adabff72f0 2024-04-25T10:43:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.312+0000 7f4980ff9640 1 -- 172.21.15.43:0/43170289 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f49940c7680 con 0x55adabff72f0 2024-04-25T10:43:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.312+0000 7f49a69ea740 10 client.18654 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.312+0000 7f49a69ea740 1 -- 172.21.15.43:0/43170289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55adac444580 con 0x55adabff72f0 2024-04-25T10:43:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.313+0000 7f4980ff9640 1 -- 172.21.15.43:0/43170289 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f499000f970 con 0x55adabff72f0 2024-04-25T10:43:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.313+0000 7f4980ff9640 1 -- 172.21.15.43:0/43170289 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f49940c3b40 con 0x55adabff72f0 2024-04-25T10:43:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.313+0000 7f49a69ea740 10 client.18654 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.313+0000 7f49a69ea740 10 client.18654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:43:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.313+0000 7f49a69ea740 20 client.18654 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.313+0000 7f49a69ea740 1 --2- 172.21.15.43:0/43170289 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55adabfb41c0 0x55adac446cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.313+0000 7f49a69ea740 4 client.18654 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:43:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.313+0000 7f49a69ea740 1 -- 172.21.15.43:0/43170289 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55adac32dee0 con 0x55adabfb41c0 2024-04-25T10:43:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.313+0000 7f499cff5640 1 --2- 172.21.15.43:0/43170289 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55adabfb41c0 0x55adac446cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.314+0000 7f4980ff9640 10 client.18654 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.314+0000 7f499cff5640 1 --2- 172.21.15.43:0/43170289 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55adabfb41c0 0x55adac446cc0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d09808761852f2a server_cookie=8b9a705262210a17 in_seq=0 out_seq=0 2024-04-25T10:43:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.333+0000 7f4980ff9640 1 -- 172.21.15.43:0/43170289 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55adac32dee0 con 0x55adabfb41c0 2024-04-25T10:43:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.333+0000 7f4980ff9640 10 client.18654 handle_command_reply: tid=0 2024-04-25T10:43:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.333+0000 7f49a69ea740 1 client.18654 shutdown 2024-04-25T10:43:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.333+0000 7f49a69ea740 1 -- 172.21.15.43:0/43170289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adabff72f0 msgr2=0x55adac3d6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.333+0000 7f49a69ea740 1 --2- 172.21.15.43:0/43170289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adabff72f0 0x55adac3d6b80 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f49940c75b0 tx=0x7f4994101c20 comp rx=0 tx=0).stop 2024-04-25T10:43:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.333+0000 7f49a69ea740 1 -- 172.21.15.43:0/43170289 shutdown_connections 2024-04-25T10:43:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.333+0000 7f49a69ea740 1 --2- 172.21.15.43:0/43170289 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55adabfb41c0 0x55adac446cc0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.333+0000 7f49a69ea740 1 --2- 172.21.15.43:0/43170289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adabfff4d0 0x55adac3d70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.333+0000 7f49a69ea740 1 --2- 172.21.15.43:0/43170289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55adac0201b0 0x55adac3d1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.333+0000 7f49a69ea740 1 --2- 172.21.15.43:0/43170289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adabff72f0 0x55adac3d6b80 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.333+0000 7f4980ff9640 0 client.18654 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.333+0000 7f49a69ea740 1 -- 172.21.15.43:0/43170289 >> 172.21.15.43:0/43170289 conn(0x55adac014300 msgr2=0x55adac3cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.333+0000 7f49a69ea740 1 -- 172.21.15.43:0/43170289 shutdown_connections 2024-04-25T10:43:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.333+0000 7f49a69ea740 1 -- 172.21.15.43:0/43170289 wait complete. 2024-04-25T10:43:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.333+0000 7f49a69ea740 20 client.18654 trim_cache size 0 max 16384 2024-04-25T10:43:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:43:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:43:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:43:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:43:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.334+0000 7f499fa5a640 1 -- 172.21.15.43:0/4238734651 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f49840afc50 msgr2=0x7f49840b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.334+0000 7f499fa5a640 1 --2- 172.21.15.43:0/4238734651 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f49840afc50 0x7f49840b2160 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55adac3bcc60 tx=0x7f4990009500 comp rx=0 tx=0).stop 2024-04-25T10:43:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.334+0000 7f499fa5a640 1 -- 172.21.15.43:0/4238734651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4998076470 msgr2=0x7f49981a0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.334+0000 7f499fa5a640 1 --2- 172.21.15.43:0/4238734651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4998076470 0x7f49981a0d70 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f4994006010 tx=0x7f4994004270 comp rx=0 tx=0).stop 2024-04-25T10:43:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.334+0000 7f499fa5a640 1 -- 172.21.15.43:0/4238734651 shutdown_connections 2024-04-25T10:43:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.334+0000 7f499fa5a640 1 --2- 172.21.15.43:0/4238734651 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f49840afc50 0x7f49840b2160 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.334+0000 7f499fa5a640 1 --2- 172.21.15.43:0/4238734651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4998114b20 0x7f49981a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.334+0000 7f499fa5a640 1 --2- 172.21.15.43:0/4238734651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49981121c0 0x7f49981a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.334+0000 7f499fa5a640 1 --2- 172.21.15.43:0/4238734651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4998076470 0x7f49981a0d70 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.334+0000 7f499fa5a640 1 -- 172.21.15.43:0/4238734651 >> 172.21.15.43:0/4238734651 conn(0x7f49980ff220 msgr2=0x7f49980ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.334+0000 7f499fa5a640 1 -- 172.21.15.43:0/4238734651 shutdown_connections 2024-04-25T10:43:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:02.334+0000 7f499fa5a640 1 -- 172.21.15.43:0/4238734651 wait complete. 2024-04-25T10:43:02.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:02 smithi043 ceph-mon[29478]: pgmap v326: 97 pgs: 97 active+clean; 4.0 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-04-25T10:43:02.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:02 smithi077 ceph-mon[32772]: pgmap v326: 97 pgs: 97 active+clean; 4.0 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-04-25T10:43:02.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:02 smithi164 ceph-mon[32713]: pgmap v326: 97 pgs: 97 active+clean; 4.0 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-04-25T10:43:04.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:04 smithi043 ceph-mon[29478]: pgmap v327: 97 pgs: 97 active+clean; 4.0 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-04-25T10:43:04.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:04 smithi077 ceph-mon[32772]: pgmap v327: 97 pgs: 97 active+clean; 4.0 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-04-25T10:43:04.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:04 smithi164 ceph-mon[32713]: pgmap v327: 97 pgs: 97 active+clean; 4.0 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-04-25T10:43:05.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:05 smithi043 ceph-mon[29478]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T10:43:05.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:05 smithi077 ceph-mon[32772]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T10:43:05.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:05 smithi164 ceph-mon[32713]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T10:43:06.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:06 smithi043 ceph-mon[29478]: pgmap v329: 97 pgs: 97 active+clean; 4.3 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 105 KiB/s wr, 25 op/s 2024-04-25T10:43:06.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:06 smithi077 ceph-mon[32772]: pgmap v329: 97 pgs: 97 active+clean; 4.3 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 105 KiB/s wr, 25 op/s 2024-04-25T10:43:06.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:06 smithi164 ceph-mon[32713]: pgmap v329: 97 pgs: 97 active+clean; 4.3 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 105 KiB/s wr, 25 op/s 2024-04-25T10:43:08.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:08 smithi043 ceph-mon[29478]: pgmap v330: 97 pgs: 97 active+clean; 4.3 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 105 KiB/s wr, 25 op/s 2024-04-25T10:43:08.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:08 smithi077 ceph-mon[32772]: pgmap v330: 97 pgs: 97 active+clean; 4.3 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 105 KiB/s wr, 25 op/s 2024-04-25T10:43:08.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:08 smithi164 ceph-mon[32713]: pgmap v330: 97 pgs: 97 active+clean; 4.3 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 105 KiB/s wr, 25 op/s 2024-04-25T10:43:10.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:10 smithi043 ceph-mon[29478]: pgmap v331: 97 pgs: 97 active+clean; 4.5 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 106 KiB/s wr, 23 op/s 2024-04-25T10:43:10.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:10 smithi077 ceph-mon[32772]: pgmap v331: 97 pgs: 97 active+clean; 4.5 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 106 KiB/s wr, 23 op/s 2024-04-25T10:43:10.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:10 smithi164 ceph-mon[32713]: pgmap v331: 97 pgs: 97 active+clean; 4.5 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 106 KiB/s wr, 23 op/s 2024-04-25T10:43:12.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:12 smithi043 ceph-mon[29478]: pgmap v332: 97 pgs: 97 active+clean; 4.5 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 106 KiB/s wr, 23 op/s 2024-04-25T10:43:12.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:12 smithi077 ceph-mon[32772]: pgmap v332: 97 pgs: 97 active+clean; 4.5 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 106 KiB/s wr, 23 op/s 2024-04-25T10:43:12.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:12 smithi164 ceph-mon[32713]: pgmap v332: 97 pgs: 97 active+clean; 4.5 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 106 KiB/s wr, 23 op/s 2024-04-25T10:43:14.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:13 smithi077 ceph-mon[32772]: pgmap v333: 97 pgs: 97 active+clean; 4.5 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 106 KiB/s wr, 23 op/s 2024-04-25T10:43:14.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:13 smithi077 ceph-mon[32772]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T10:43:14.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:13 smithi164 ceph-mon[32713]: pgmap v333: 97 pgs: 97 active+clean; 4.5 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 106 KiB/s wr, 23 op/s 2024-04-25T10:43:14.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:13 smithi164 ceph-mon[32713]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T10:43:14.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:13 smithi043 ceph-mon[29478]: pgmap v333: 97 pgs: 97 active+clean; 4.5 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 106 KiB/s wr, 23 op/s 2024-04-25T10:43:14.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:13 smithi043 ceph-mon[29478]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T10:43:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.448+0000 7f3844251640 1 -- 172.21.15.43:0/2783892779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f383c109b40 msgr2=0x7f383c101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.448+0000 7f3844251640 1 --2- 172.21.15.43:0/2783892779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f383c109b40 0x7f383c101bf0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f3838009900 tx=0x7f383802a410 comp rx=0 tx=0).stop 2024-04-25T10:43:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.448+0000 7f3844251640 1 -- 172.21.15.43:0/2783892779 shutdown_connections 2024-04-25T10:43:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.448+0000 7f3844251640 1 --2- 172.21.15.43:0/2783892779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f383c109fe0 0x7f383c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.448+0000 7f3844251640 1 --2- 172.21.15.43:0/2783892779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f383c1027e0 0x7f383c110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.448+0000 7f3844251640 1 --2- 172.21.15.43:0/2783892779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f383c109b40 0x7f383c101bf0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.448+0000 7f3844251640 1 -- 172.21.15.43:0/2783892779 >> 172.21.15.43:0/2783892779 conn(0x7f383c0fd880 msgr2=0x7f383c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.448+0000 7f3844251640 1 -- 172.21.15.43:0/2783892779 shutdown_connections 2024-04-25T10:43:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.448+0000 7f3844251640 1 -- 172.21.15.43:0/2783892779 wait complete. 2024-04-25T10:43:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.449+0000 7f3844251640 1 Processor -- start 2024-04-25T10:43:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.449+0000 7f3844251640 1 -- start start 2024-04-25T10:43:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.449+0000 7f3844251640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f383c109b40 0x7f383c102dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.449+0000 7f3844251640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f383c103310 0x7f383c103720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.449+0000 7f3844251640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f383c1a6130 0x7f383c1a8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.449+0000 7f3844251640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f383c113d80 con 0x7f383c1a6130 2024-04-25T10:43:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.449+0000 7f3844251640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f383c113c00 con 0x7f383c103310 2024-04-25T10:43:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.449+0000 7f3844251640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f383c113f00 con 0x7f383c109b40 2024-04-25T10:43:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.449+0000 7f38427ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f383c1a6130 0x7f383c1a8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.449+0000 7f38427ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f383c1a6130 0x7f383c1a8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48596/0 (socket says 172.21.15.43:48596) 2024-04-25T10:43:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.449+0000 7f38427ee640 1 -- 172.21.15.43:0/88093928 learned_addr learned my addr 172.21.15.43:0/88093928 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.449+0000 7f38417ec640 1 --2- 172.21.15.43:0/88093928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f383c103310 0x7f383c103720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.449+0000 7f3841fed640 1 --2- 172.21.15.43:0/88093928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f383c109b40 0x7f383c102dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.450+0000 7f38427ee640 1 -- 172.21.15.43:0/88093928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f383c109b40 msgr2=0x7f383c102dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.450+0000 7f38427ee640 1 --2- 172.21.15.43:0/88093928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f383c109b40 0x7f383c102dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.450+0000 7f38427ee640 1 -- 172.21.15.43:0/88093928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f383c103310 msgr2=0x7f383c103720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.450+0000 7f38427ee640 1 --2- 172.21.15.43:0/88093928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f383c103310 0x7f383c103720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.450+0000 7f38427ee640 1 -- 172.21.15.43:0/88093928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f383c1a8b10 con 0x7f383c1a6130 2024-04-25T10:43:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.450+0000 7f38427ee640 1 --2- 172.21.15.43:0/88093928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f383c1a6130 0x7f383c1a8550 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f383400eb10 tx=0x7f383400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.450+0000 7f3832ffd640 1 -- 172.21.15.43:0/88093928 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834004b20 con 0x7f383c1a6130 2024-04-25T10:43:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.450+0000 7f3832ffd640 1 -- 172.21.15.43:0/88093928 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3834004cc0 con 0x7f383c1a6130 2024-04-25T10:43:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.450+0000 7f3844251640 1 -- 172.21.15.43:0/88093928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f383c1a8e00 con 0x7f383c1a6130 2024-04-25T10:43:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.450+0000 7f3832ffd640 1 -- 172.21.15.43:0/88093928 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834010670 con 0x7f383c1a6130 2024-04-25T10:43:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.450+0000 7f3844251640 1 -- 172.21.15.43:0/88093928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f383c1a9290 con 0x7f383c1a6130 2024-04-25T10:43:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.453+0000 7f3832ffd640 1 -- 172.21.15.43:0/88093928 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f3834005020 con 0x7f383c1a6130 2024-04-25T10:43:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.453+0000 7f384b1e5740 1 Processor -- start 2024-04-25T10:43:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.453+0000 7f384b1e5740 1 -- start start 2024-04-25T10:43:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.454+0000 7f384b1e5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bea1b0 0x55e374f89af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.454+0000 7f384b1e5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e374bc12f0 0x55e374f8a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.454+0000 7f384b1e5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bc94d0 0x55e374f8a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.454+0000 7f384b1e5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e374ef0460 con 0x55e374bea1b0 2024-04-25T10:43:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.454+0000 7f384b1e5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e374eaa590 con 0x55e374bc94d0 2024-04-25T10:43:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.454+0000 7f384b1e5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e374eec420 con 0x55e374bc12f0 2024-04-25T10:43:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.454+0000 7f38417ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bea1b0 0x55e374f89af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.454+0000 7f38417ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bea1b0 0x55e374f89af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48604/0 (socket says 172.21.15.43:48604) 2024-04-25T10:43:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.454+0000 7f3841fed640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e374bc12f0 0x55e374f8a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.454+0000 7f38417ec640 1 -- 172.21.15.43:0/2627815804 learned_addr learned my addr 172.21.15.43:0/2627815804 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.454+0000 7f38417ec640 1 --2- 172.21.15.43:0/2627815804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bc94d0 0x55e374f8a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.454+0000 7f3832ffd640 1 --2- 172.21.15.43:0/88093928 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f38280afc50 0x7f38280b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.454+0000 7f38427ee640 1 --2- 172.21.15.43:0/88093928 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f38280afc50 0x7f38280b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.454+0000 7f3841fed640 1 -- 172.21.15.43:0/2627815804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bc94d0 msgr2=0x55e374f8a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.454+0000 7f3841fed640 1 --2- 172.21.15.43:0/2627815804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bc94d0 0x55e374f8a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.454+0000 7f3841fed640 1 -- 172.21.15.43:0/2627815804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bea1b0 msgr2=0x55e374f89af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.454+0000 7f3841fed640 1 --2- 172.21.15.43:0/2627815804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bea1b0 0x55e374f89af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.454+0000 7f3841fed640 1 -- 172.21.15.43:0/2627815804 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e374eb70d0 con 0x55e374bc12f0 2024-04-25T10:43:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.454+0000 7f3832ffd640 1 -- 172.21.15.43:0/88093928 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f38340df630 con 0x7f383c1a6130 2024-04-25T10:43:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.455+0000 7f3841fed640 1 --2- 172.21.15.43:0/2627815804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e374bc12f0 0x55e374f8a030 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f383802a930 tx=0x7f3838004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dc5ad1782b8d88c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.455+0000 7f38427ee640 1 --2- 172.21.15.43:0/88093928 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f38280afc50 0x7f38280b2160 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f38340a37f0 tx=0x7f38340ca040 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.455+0000 7f3827fff640 1 -- 172.21.15.43:0/2627815804 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838035040 con 0x55e374bc12f0 2024-04-25T10:43:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.455+0000 7f3827fff640 1 -- 172.21.15.43:0/2627815804 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3838004430 con 0x55e374bc12f0 2024-04-25T10:43:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.455+0000 7f3827fff640 1 -- 172.21.15.43:0/2627815804 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38380035f0 con 0x55e374bc12f0 2024-04-25T10:43:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.456+0000 7f384b1e5740 1 -- 172.21.15.43:0/2627815804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e374bc12f0 msgr2=0x55e374f8a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.456+0000 7f384b1e5740 1 --2- 172.21.15.43:0/2627815804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e374bc12f0 0x55e374f8a030 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f383802a930 tx=0x7f3838004040 comp rx=0 tx=0).stop 2024-04-25T10:43:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.456+0000 7f384b1e5740 1 -- 172.21.15.43:0/2627815804 shutdown_connections 2024-04-25T10:43:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.456+0000 7f384b1e5740 1 --2- 172.21.15.43:0/2627815804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bc94d0 0x55e374f8a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.456+0000 7f384b1e5740 1 --2- 172.21.15.43:0/2627815804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e374bc12f0 0x55e374f8a030 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.456+0000 7f384b1e5740 1 --2- 172.21.15.43:0/2627815804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bea1b0 0x55e374f89af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.456+0000 7f384b1e5740 1 -- 172.21.15.43:0/2627815804 >> 172.21.15.43:0/2627815804 conn(0x55e374bde300 msgr2=0x55e374f862a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.456+0000 7f384b1e5740 1 -- 172.21.15.43:0/2627815804 shutdown_connections 2024-04-25T10:43:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.456+0000 7f384b1e5740 1 -- 172.21.15.43:0/2627815804 wait complete. 2024-04-25T10:43:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.456+0000 7f384b1e5740 1 -- start start 2024-04-25T10:43:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.456+0000 7f384b1e5740 1 Processor -- start 2024-04-25T10:43:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.457+0000 7f384b1e5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc12f0 0x55e374f941b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.457+0000 7f384b1e5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bc94d0 0x55e374f94710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.457+0000 7f384b1e5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e374bea1b0 0x55e374f9a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.457+0000 7f384b1e5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e374f96bc0 con 0x55e374bc12f0 2024-04-25T10:43:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.457+0000 7f384b1e5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e374e77b60 con 0x55e374bc94d0 2024-04-25T10:43:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.457+0000 7f384b1e5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e374f96d40 con 0x55e374bea1b0 2024-04-25T10:43:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.457+0000 7f384b1e5740 10 client.0 _ll_register_callbacks cb 0x55e374e9c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.457+0000 7f38417ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc12f0 0x55e374f941b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.457+0000 7f38417ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc12f0 0x55e374f941b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48608/0 (socket says 172.21.15.43:48608) 2024-04-25T10:43:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.457+0000 7f38417ec640 1 -- 172.21.15.43:0/4260187335 learned_addr learned my addr 172.21.15.43:0/4260187335 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.457+0000 7f3841fed640 1 --2- 172.21.15.43:0/4260187335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bc94d0 0x55e374f94710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.457+0000 7f38417ec640 1 --2- 172.21.15.43:0/4260187335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e374bea1b0 0x55e374f9a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.457+0000 7f38417ec640 1 -- 172.21.15.43:0/4260187335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e374bea1b0 msgr2=0x55e374f9a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.457+0000 7f38417ec640 1 --2- 172.21.15.43:0/4260187335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e374bea1b0 0x55e374f9a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.457+0000 7f38417ec640 1 -- 172.21.15.43:0/4260187335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bc94d0 msgr2=0x55e374f94710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.457+0000 7f38417ec640 1 --2- 172.21.15.43:0/4260187335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bc94d0 0x55e374f94710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.457+0000 7f38417ec640 1 -- 172.21.15.43:0/4260187335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e374c02210 con 0x55e374bc12f0 2024-04-25T10:43:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.458+0000 7f3841fed640 1 --2- 172.21.15.43:0/4260187335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bc94d0 0x55e374f94710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:43:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.458+0000 7f38417ec640 1 --2- 172.21.15.43:0/4260187335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc12f0 0x55e374f941b0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f382c0058c0 tx=0x7f382c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7e8fa0be1704a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.458+0000 7f38257fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.458+0000 7f38257fa640 1 -- 172.21.15.43:0/4260187335 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c004110 con 0x55e374bc12f0 2024-04-25T10:43:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.458+0000 7f38257fa640 1 -- 172.21.15.43:0/4260187335 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f382c009040 con 0x55e374bc12f0 2024-04-25T10:43:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.458+0000 7f384b1e5740 1 -- 172.21.15.43:0/4260187335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3838004430 con 0x55e374bc12f0 2024-04-25T10:43:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.458+0000 7f38257fa640 1 -- 172.21.15.43:0/4260187335 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f382c0042b0 con 0x55e374bc12f0 2024-04-25T10:43:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.458+0000 7f384b1e5740 10 client.18666 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.458+0000 7f384b1e5740 1 -- 172.21.15.43:0/4260187335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e374cce620 con 0x55e374bc12f0 2024-04-25T10:43:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.458+0000 7f38257fa640 1 -- 172.21.15.43:0/4260187335 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3838004430 con 0x55e374bc12f0 2024-04-25T10:43:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.459+0000 7f38257fa640 1 -- 172.21.15.43:0/4260187335 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f382c0295a0 con 0x55e374bc12f0 2024-04-25T10:43:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.459+0000 7f384b1e5740 10 client.18666 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.459+0000 7f384b1e5740 10 client.18666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:43:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.459+0000 7f384b1e5740 20 client.18666 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.459+0000 7f384b1e5740 1 --2- 172.21.15.43:0/4260187335 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e374ba8d80 0x55e374fa66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.459+0000 7f384b1e5740 4 client.18666 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:43:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.459+0000 7f384b1e5740 1 -- 172.21.15.43:0/4260187335 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e374efe070 con 0x55e374ba8d80 2024-04-25T10:43:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.459+0000 7f3841fed640 1 --2- 172.21.15.43:0/4260187335 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e374ba8d80 0x55e374fa66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.459+0000 7f3841fed640 1 --2- 172.21.15.43:0/4260187335 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e374ba8d80 0x55e374fa66d0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26db879211d5915c server_cookie=10592877597b2b77 in_seq=0 out_seq=0 2024-04-25T10:43:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.459+0000 7f38257fa640 10 client.18666 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.460+0000 7f38257fa640 1 -- 172.21.15.43:0/4260187335 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e374efe070 con 0x55e374ba8d80 2024-04-25T10:43:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.460+0000 7f38257fa640 10 client.18666 handle_command_reply: tid=0 2024-04-25T10:43:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.460+0000 7f384b1e5740 1 client.18666 shutdown 2024-04-25T10:43:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.460+0000 7f384b1e5740 1 -- 172.21.15.43:0/4260187335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc12f0 msgr2=0x55e374f941b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.460+0000 7f384b1e5740 1 --2- 172.21.15.43:0/4260187335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc12f0 0x55e374f941b0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f382c0058c0 tx=0x7f382c005f90 comp rx=0 tx=0).stop 2024-04-25T10:43:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.460+0000 7f384b1e5740 1 -- 172.21.15.43:0/4260187335 shutdown_connections 2024-04-25T10:43:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.460+0000 7f384b1e5740 1 --2- 172.21.15.43:0/4260187335 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e374ba8d80 0x55e374fa66d0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.460+0000 7f384b1e5740 1 --2- 172.21.15.43:0/4260187335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bc94d0 0x55e374f94710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.460+0000 7f384b1e5740 1 --2- 172.21.15.43:0/4260187335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e374bea1b0 0x55e374f9a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.460+0000 7f384b1e5740 1 --2- 172.21.15.43:0/4260187335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc12f0 0x55e374f941b0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.460+0000 7f38257fa640 0 client.18666 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.461+0000 7f384b1e5740 1 -- 172.21.15.43:0/4260187335 >> 172.21.15.43:0/4260187335 conn(0x55e374bde300 msgr2=0x55e374f8b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.461+0000 7f384b1e5740 1 -- 172.21.15.43:0/4260187335 shutdown_connections 2024-04-25T10:43:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.461+0000 7f384b1e5740 1 -- 172.21.15.43:0/4260187335 wait complete. 2024-04-25T10:43:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.461+0000 7f384b1e5740 20 client.18666 trim_cache size 0 max 16384 2024-04-25T10:43:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.476+0000 7f384b1e5740 1 Processor -- start 2024-04-25T10:43:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.476+0000 7f384b1e5740 1 -- start start 2024-04-25T10:43:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.476+0000 7f384b1e5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bc12f0 0x55e374f8a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.476+0000 7f384b1e5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc94d0 0x55e374f8d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.476+0000 7f384b1e5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e374bea1b0 0x55e374f8dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.476+0000 7f384b1e5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b44a0 con 0x55e374bc94d0 2024-04-25T10:43:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.476+0000 7f384b1e5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b1d70 con 0x55e374bc12f0 2024-04-25T10:43:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.476+0000 7f384b1e5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1afbe0 con 0x55e374bea1b0 2024-04-25T10:43:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.476+0000 7f3841fed640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc94d0 0x55e374f8d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.476+0000 7f3841fed640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc94d0 0x55e374f8d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48624/0 (socket says 172.21.15.43:48624) 2024-04-25T10:43:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.476+0000 7f3841fed640 1 -- 172.21.15.43:0/1303649260 learned_addr learned my addr 172.21.15.43:0/1303649260 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.476+0000 7f38417ec640 1 --2- 172.21.15.43:0/1303649260 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bc12f0 0x55e374f8a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.476+0000 7f3841fed640 1 -- 172.21.15.43:0/1303649260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e374bea1b0 msgr2=0x55e374f8dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.476+0000 7f3841fed640 1 --2- 172.21.15.43:0/1303649260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e374bea1b0 0x55e374f8dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.476+0000 7f3841fed640 1 -- 172.21.15.43:0/1303649260 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bc12f0 msgr2=0x55e374f8a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.476+0000 7f3841fed640 1 --2- 172.21.15.43:0/1303649260 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bc12f0 0x55e374f8a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.476+0000 7f3841fed640 1 -- 172.21.15.43:0/1303649260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e374f86e90 con 0x55e374bc94d0 2024-04-25T10:43:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.476+0000 7f38417ec640 1 --2- 172.21.15.43:0/1303649260 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bc12f0 0x55e374f8a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:43:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.477+0000 7f3841fed640 1 --2- 172.21.15.43:0/1303649260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc94d0 0x55e374f8d850 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f3838004650 tx=0x7f383802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c874b5b3aababe7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.477+0000 7f38277fe640 1 -- 172.21.15.43:0/1303649260 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838035070 con 0x55e374bc94d0 2024-04-25T10:43:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.477+0000 7f38277fe640 1 -- 172.21.15.43:0/1303649260 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38380046c0 con 0x55e374bc94d0 2024-04-25T10:43:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.477+0000 7f38277fe640 1 -- 172.21.15.43:0/1303649260 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838003850 con 0x55e374bc94d0 2024-04-25T10:43:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.477+0000 7f384b1e5740 1 -- 172.21.15.43:0/1303649260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc94d0 msgr2=0x55e374f8d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.477+0000 7f384b1e5740 1 --2- 172.21.15.43:0/1303649260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc94d0 0x55e374f8d850 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f3838004650 tx=0x7f383802a410 comp rx=0 tx=0).stop 2024-04-25T10:43:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.478+0000 7f384b1e5740 1 -- 172.21.15.43:0/1303649260 shutdown_connections 2024-04-25T10:43:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.478+0000 7f384b1e5740 1 --2- 172.21.15.43:0/1303649260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e374bea1b0 0x55e374f8dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.478+0000 7f384b1e5740 1 --2- 172.21.15.43:0/1303649260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc94d0 0x55e374f8d850 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.478+0000 7f384b1e5740 1 --2- 172.21.15.43:0/1303649260 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bc12f0 0x55e374f8a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.478+0000 7f384b1e5740 1 -- 172.21.15.43:0/1303649260 >> 172.21.15.43:0/1303649260 conn(0x55e374bde300 msgr2=0x55e374f95480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.478+0000 7f384b1e5740 1 -- 172.21.15.43:0/1303649260 shutdown_connections 2024-04-25T10:43:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.478+0000 7f384b1e5740 1 -- 172.21.15.43:0/1303649260 wait complete. 2024-04-25T10:43:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.478+0000 7f384b1e5740 1 -- start start 2024-04-25T10:43:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.478+0000 7f384b1e5740 1 Processor -- start 2024-04-25T10:43:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f384b1e5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e374bc12f0 0x55e374f9c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f384b1e5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc94d0 0x55e374f9c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f384b1e5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bea1b0 0x55e374f9cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f384b1e5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b0230 con 0x55e374bc94d0 2024-04-25T10:43:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f384b1e5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b16a0 con 0x55e374bea1b0 2024-04-25T10:43:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f384b1e5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b0900 con 0x55e374bc12f0 2024-04-25T10:43:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f384b1e5740 10 client.0 _ll_register_callbacks cb 0x7f382c0135d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f3841fed640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc94d0 0x55e374f9c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f3841fed640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc94d0 0x55e374f9c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48638/0 (socket says 172.21.15.43:48638) 2024-04-25T10:43:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f3841fed640 1 -- 172.21.15.43:0/1760846610 learned_addr learned my addr 172.21.15.43:0/1760846610 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f38417ec640 1 --2- 172.21.15.43:0/1760846610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e374bc12f0 0x55e374f9c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f38417ec640 1 --2- 172.21.15.43:0/1760846610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bea1b0 0x55e374f9cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f3841fed640 1 -- 172.21.15.43:0/1760846610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e374bc12f0 msgr2=0x55e374f9c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f3841fed640 1 --2- 172.21.15.43:0/1760846610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e374bc12f0 0x55e374f9c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f3841fed640 1 -- 172.21.15.43:0/1760846610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bea1b0 msgr2=0x55e374f9cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f3841fed640 1 --2- 172.21.15.43:0/1760846610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bea1b0 0x55e374f9cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f3841fed640 1 -- 172.21.15.43:0/1760846610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e37500da80 con 0x55e374bc94d0 2024-04-25T10:43:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f38417ec640 1 --2- 172.21.15.43:0/1760846610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bea1b0 0x55e374f9cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:43:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f3841fed640 1 --2- 172.21.15.43:0/1760846610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc94d0 0x55e374f9c6b0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f3838005ab0 tx=0x7f3838031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2458f6280b4b01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f38257fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f38257fa640 1 -- 172.21.15.43:0/1760846610 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838035070 con 0x55e374bc94d0 2024-04-25T10:43:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f38257fa640 1 -- 172.21.15.43:0/1760846610 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f3838003b00 con 0x55e374bc94d0 2024-04-25T10:43:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.479+0000 7f384b1e5740 1 -- 172.21.15.43:0/1760846610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38380046c0 con 0x55e374bc94d0 2024-04-25T10:43:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.480+0000 7f38257fa640 1 -- 172.21.15.43:0/1760846610 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3838063050 con 0x55e374bc94d0 2024-04-25T10:43:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.480+0000 7f384b1e5740 10 client.18672 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.480+0000 7f384b1e5740 1 -- 172.21.15.43:0/1760846610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e37500ec30 con 0x55e374bc94d0 2024-04-25T10:43:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.480+0000 7f38257fa640 1 -- 172.21.15.43:0/1760846610 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38380046c0 con 0x55e374bc94d0 2024-04-25T10:43:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.480+0000 7f38257fa640 1 -- 172.21.15.43:0/1760846610 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f38380314a0 con 0x55e374bc94d0 2024-04-25T10:43:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.480+0000 7f384b1e5740 10 client.18672 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.480+0000 7f384b1e5740 10 client.18672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:43:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.480+0000 7f384b1e5740 20 client.18672 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.480+0000 7f384b1e5740 1 --2- 172.21.15.43:0/1760846610 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e374b7e1c0 0x55e375010ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.480+0000 7f384b1e5740 4 client.18672 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:43:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.480+0000 7f384b1e5740 1 -- 172.21.15.43:0/1760846610 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e374ef7ee0 con 0x55e374b7e1c0 2024-04-25T10:43:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.480+0000 7f38417ec640 1 --2- 172.21.15.43:0/1760846610 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e374b7e1c0 0x55e375010ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.481+0000 7f38417ec640 1 --2- 172.21.15.43:0/1760846610 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e374b7e1c0 0x55e375010ec0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce753eb82cffad73 server_cookie=fce64195ee228ce8 in_seq=0 out_seq=0 2024-04-25T10:43:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.481+0000 7f38257fa640 10 client.18672 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.496+0000 7f38257fa640 1 -- 172.21.15.43:0/1760846610 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e374ef7ee0 con 0x55e374b7e1c0 2024-04-25T10:43:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.496+0000 7f38257fa640 10 client.18672 handle_command_reply: tid=0 2024-04-25T10:43:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.496+0000 7f384b1e5740 1 client.18672 shutdown 2024-04-25T10:43:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.496+0000 7f384b1e5740 1 -- 172.21.15.43:0/1760846610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc94d0 msgr2=0x55e374f9c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.496+0000 7f384b1e5740 1 --2- 172.21.15.43:0/1760846610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc94d0 0x55e374f9c6b0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f3838005ab0 tx=0x7f3838031400 comp rx=0 tx=0).stop 2024-04-25T10:43:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.496+0000 7f384b1e5740 1 -- 172.21.15.43:0/1760846610 shutdown_connections 2024-04-25T10:43:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.496+0000 7f384b1e5740 1 --2- 172.21.15.43:0/1760846610 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e374b7e1c0 0x55e375010ec0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.496+0000 7f384b1e5740 1 --2- 172.21.15.43:0/1760846610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e374bea1b0 0x55e374f9cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.496+0000 7f384b1e5740 1 --2- 172.21.15.43:0/1760846610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e374bc94d0 0x55e374f9c6b0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.496+0000 7f384b1e5740 1 --2- 172.21.15.43:0/1760846610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e374bc12f0 0x55e374f9c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.496+0000 7f384b1e5740 1 -- 172.21.15.43:0/1760846610 >> 172.21.15.43:0/1760846610 conn(0x55e374bde300 msgr2=0x55e374f95eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.496+0000 7f38257fa640 0 client.18672 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.496+0000 7f384b1e5740 1 -- 172.21.15.43:0/1760846610 shutdown_connections 2024-04-25T10:43:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.496+0000 7f384b1e5740 1 -- 172.21.15.43:0/1760846610 wait complete. 2024-04-25T10:43:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.496+0000 7f384b1e5740 20 client.18672 trim_cache size 0 max 16384 2024-04-25T10:43:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:43:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:43:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:43:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:43:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.497+0000 7f3844251640 1 -- 172.21.15.43:0/88093928 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f38280afc50 msgr2=0x7f38280b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.497+0000 7f3844251640 1 --2- 172.21.15.43:0/88093928 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f38280afc50 0x7f38280b2160 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f38340a37f0 tx=0x7f38340ca040 comp rx=0 tx=0).stop 2024-04-25T10:43:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.497+0000 7f3844251640 1 -- 172.21.15.43:0/88093928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f383c1a6130 msgr2=0x7f383c1a8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.497+0000 7f3844251640 1 --2- 172.21.15.43:0/88093928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f383c1a6130 0x7f383c1a8550 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f383400eb10 tx=0x7f383400c2d0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.497+0000 7f3844251640 1 -- 172.21.15.43:0/88093928 shutdown_connections 2024-04-25T10:43:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.497+0000 7f3844251640 1 --2- 172.21.15.43:0/88093928 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f38280afc50 0x7f38280b2160 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.497+0000 7f3844251640 1 --2- 172.21.15.43:0/88093928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f383c103310 0x7f383c103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.497+0000 7f3844251640 1 --2- 172.21.15.43:0/88093928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f383c1a6130 0x7f383c1a8550 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.497+0000 7f3844251640 1 --2- 172.21.15.43:0/88093928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f383c109b40 0x7f383c102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.497+0000 7f3844251640 1 -- 172.21.15.43:0/88093928 >> 172.21.15.43:0/88093928 conn(0x7f383c0fd880 msgr2=0x7f383c100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.497+0000 7f3844251640 1 -- 172.21.15.43:0/88093928 shutdown_connections 2024-04-25T10:43:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:15.497+0000 7f3844251640 1 -- 172.21.15.43:0/88093928 wait complete. 2024-04-25T10:43:16.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:15 smithi077 ceph-mon[32772]: pgmap v335: 97 pgs: 97 active+clean; 4.7 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 11 op/s 2024-04-25T10:43:16.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:15 smithi164 ceph-mon[32713]: pgmap v335: 97 pgs: 97 active+clean; 4.7 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 11 op/s 2024-04-25T10:43:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:15 smithi043 ceph-mon[29478]: pgmap v335: 97 pgs: 97 active+clean; 4.7 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 11 op/s 2024-04-25T10:43:18.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:17 smithi077 ceph-mon[32772]: pgmap v336: 97 pgs: 97 active+clean; 4.7 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 11 op/s 2024-04-25T10:43:18.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:17 smithi164 ceph-mon[32713]: pgmap v336: 97 pgs: 97 active+clean; 4.7 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 11 op/s 2024-04-25T10:43:18.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:17 smithi043 ceph-mon[29478]: pgmap v336: 97 pgs: 97 active+clean; 4.7 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 11 op/s 2024-04-25T10:43:20.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:19 smithi077 ceph-mon[32772]: pgmap v337: 97 pgs: 97 active+clean; 4.9 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 14 op/s 2024-04-25T10:43:20.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:19 smithi164 ceph-mon[32713]: pgmap v337: 97 pgs: 97 active+clean; 4.9 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 14 op/s 2024-04-25T10:43:20.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:19 smithi043 ceph-mon[29478]: pgmap v337: 97 pgs: 97 active+clean; 4.9 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 14 op/s 2024-04-25T10:43:22.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:21 smithi077 ceph-mon[32772]: pgmap v338: 97 pgs: 97 active+clean; 4.9 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 14 op/s 2024-04-25T10:43:22.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:21 smithi164 ceph-mon[32713]: pgmap v338: 97 pgs: 97 active+clean; 4.9 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 14 op/s 2024-04-25T10:43:22.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:21 smithi043 ceph-mon[29478]: pgmap v338: 97 pgs: 97 active+clean; 4.9 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 14 op/s 2024-04-25T10:43:24.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:23 smithi077 ceph-mon[32772]: pgmap v339: 97 pgs: 97 active+clean; 4.9 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 14 op/s 2024-04-25T10:43:24.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:23 smithi164 ceph-mon[32713]: pgmap v339: 97 pgs: 97 active+clean; 4.9 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 14 op/s 2024-04-25T10:43:24.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:23 smithi043 ceph-mon[29478]: pgmap v339: 97 pgs: 97 active+clean; 4.9 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 14 op/s 2024-04-25T10:43:26.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:26 smithi077 ceph-mon[32772]: pgmap v340: 97 pgs: 97 active+clean; 5.1 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-04-25T10:43:26.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:26 smithi164 ceph-mon[32713]: pgmap v340: 97 pgs: 97 active+clean; 5.1 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-04-25T10:43:26.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:26 smithi043 ceph-mon[29478]: pgmap v340: 97 pgs: 97 active+clean; 5.1 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-04-25T10:43:27.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:43:27.590 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:43:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.717+0000 7f84801c5640 1 -- 172.21.15.43:0/945597613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8478108eb0 msgr2=0x7f8478102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.717+0000 7f84801c5640 1 --2- 172.21.15.43:0/945597613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8478108eb0 0x7f8478102b50 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f8468009900 tx=0x7f846802a410 comp rx=0 tx=0).stop 2024-04-25T10:43:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.717+0000 7f84801c5640 1 -- 172.21.15.43:0/945597613 shutdown_connections 2024-04-25T10:43:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.717+0000 7f84801c5640 1 --2- 172.21.15.43:0/945597613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84781031c0 0x7f847810f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.717+0000 7f84801c5640 1 --2- 172.21.15.43:0/945597613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8478108eb0 0x7f8478102b50 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.717+0000 7f84801c5640 1 --2- 172.21.15.43:0/945597613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8478108a10 0x7f8478102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.717+0000 7f84801c5640 1 -- 172.21.15.43:0/945597613 >> 172.21.15.43:0/945597613 conn(0x7f8478078110 msgr2=0x7f8478078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.717+0000 7f84801c5640 1 -- 172.21.15.43:0/945597613 shutdown_connections 2024-04-25T10:43:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.717+0000 7f84801c5640 1 -- 172.21.15.43:0/945597613 wait complete. 2024-04-25T10:43:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.718+0000 7f84801c5640 1 Processor -- start 2024-04-25T10:43:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.718+0000 7f84801c5640 1 -- start start 2024-04-25T10:43:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.718+0000 7f84801c5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84781031c0 0x7f847819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.718+0000 7f84801c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8478108a10 0x7f847819aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.718+0000 7f84801c5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8478108eb0 0x7f847819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.718+0000 7f84801c5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8478112c10 con 0x7f8478108a10 2024-04-25T10:43:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.718+0000 7f84801c5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8478112a90 con 0x7f8478108eb0 2024-04-25T10:43:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.718+0000 7f84801c5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8478112d90 con 0x7f84781031c0 2024-04-25T10:43:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.718+0000 7f847d760640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8478108a10 0x7f847819aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.718+0000 7f847d760640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8478108a10 0x7f847819aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44404/0 (socket says 172.21.15.43:44404) 2024-04-25T10:43:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.718+0000 7f847e762640 1 --2- 172.21.15.43:0/1034370114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8478108eb0 0x7f847819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.718+0000 7f847d760640 1 -- 172.21.15.43:0/1034370114 learned_addr learned my addr 172.21.15.43:0/1034370114 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.719+0000 7f847df61640 1 --2- 172.21.15.43:0/1034370114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84781031c0 0x7f847819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.719+0000 7f847e762640 1 -- 172.21.15.43:0/1034370114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84781031c0 msgr2=0x7f847819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.719+0000 7f847e762640 1 --2- 172.21.15.43:0/1034370114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84781031c0 0x7f847819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.719+0000 7f847e762640 1 -- 172.21.15.43:0/1034370114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8478108a10 msgr2=0x7f847819aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.719+0000 7f847e762640 1 --2- 172.21.15.43:0/1034370114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8478108a10 0x7f847819aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.719+0000 7f847e762640 1 -- 172.21.15.43:0/1034370114 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f847819f520 con 0x7f8478108eb0 2024-04-25T10:43:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.719+0000 7f847e762640 1 --2- 172.21.15.43:0/1034370114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8478108eb0 0x7f847819ef60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8470009fd0 tx=0x7f847000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.719+0000 7f846effd640 1 -- 172.21.15.43:0/1034370114 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847000ce20 con 0x7f8478108eb0 2024-04-25T10:43:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.719+0000 7f84801c5640 1 -- 172.21.15.43:0/1034370114 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f847819f810 con 0x7f8478108eb0 2024-04-25T10:43:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.719+0000 7f846effd640 1 -- 172.21.15.43:0/1034370114 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8470005020 con 0x7f8478108eb0 2024-04-25T10:43:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.719+0000 7f846effd640 1 -- 172.21.15.43:0/1034370114 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470010690 con 0x7f8478108eb0 2024-04-25T10:43:27.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.719+0000 7f84801c5640 1 -- 172.21.15.43:0/1034370114 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f847819fd60 con 0x7f8478108eb0 2024-04-25T10:43:27.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.722+0000 7f8487077740 1 Processor -- start 2024-04-25T10:43:27.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.722+0000 7f846effd640 1 -- 172.21.15.43:0/1034370114 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f84700054d0 con 0x7f8478108eb0 2024-04-25T10:43:27.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.722+0000 7f8487077740 1 -- start start 2024-04-25T10:43:27.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f8487077740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4b783c390 0x55e4b7bdca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f8487077740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 0x55e4b7bdcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f8487077740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b781b6b0 0x55e4b7be98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f8487077740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e4b7b42d80 con 0x55e4b78134d0 2024-04-25T10:43:27.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f8487077740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e4b7afc8b0 con 0x55e4b781b6b0 2024-04-25T10:43:27.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f8487077740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e4b7b3ed40 con 0x55e4b783c390 2024-04-25T10:43:27.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f847df61640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 0x55e4b7bdcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f847df61640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 0x55e4b7bdcfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44412/0 (socket says 172.21.15.43:44412) 2024-04-25T10:43:27.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f847df61640 1 -- 172.21.15.43:0/3827605298 learned_addr learned my addr 172.21.15.43:0/3827605298 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:27.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f847d760640 1 --2- 172.21.15.43:0/3827605298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4b783c390 0x55e4b7bdca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f846effd640 1 --2- 172.21.15.43:0/1034370114 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f84640afc60 0x7f84640b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f847d760640 1 --2- 172.21.15.43:0/3827605298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b781b6b0 0x55e4b7be98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f847df61640 1 -- 172.21.15.43:0/3827605298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4b783c390 msgr2=0x55e4b7bdca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:27.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f847df61640 1 --2- 172.21.15.43:0/3827605298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4b783c390 0x55e4b7bdca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f847df61640 1 -- 172.21.15.43:0/3827605298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b781b6b0 msgr2=0x55e4b7be98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:27.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f847df61640 1 --2- 172.21.15.43:0/3827605298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b781b6b0 0x55e4b7be98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f847df61640 1 -- 172.21.15.43:0/3827605298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4b7acda40 con 0x55e4b78134d0 2024-04-25T10:43:27.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f847d760640 1 --2- 172.21.15.43:0/3827605298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b781b6b0 0x55e4b7be98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:43:27.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f847e762640 1 --2- 172.21.15.43:0/1034370114 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f84640afc60 0x7f84640b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f846effd640 1 -- 172.21.15.43:0/1034370114 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f84700df7b0 con 0x7f8478108eb0 2024-04-25T10:43:27.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.723+0000 7f847df61640 1 --2- 172.21.15.43:0/3827605298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 0x55e4b7bdcfa0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f847400cc00 tx=0x7f847400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e639e2a22bc8d25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:27.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.724+0000 7f8463fff640 1 -- 172.21.15.43:0/3827605298 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474005220 con 0x55e4b78134d0 2024-04-25T10:43:27.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.724+0000 7f8463fff640 1 -- 172.21.15.43:0/3827605298 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8474004c50 con 0x55e4b78134d0 2024-04-25T10:43:27.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.724+0000 7f8463fff640 1 -- 172.21.15.43:0/3827605298 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474010790 con 0x55e4b78134d0 2024-04-25T10:43:27.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.724+0000 7f8487077740 1 -- 172.21.15.43:0/3827605298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 msgr2=0x55e4b7bdcfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:27.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.724+0000 7f8487077740 1 --2- 172.21.15.43:0/3827605298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 0x55e4b7bdcfa0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f847400cc00 tx=0x7f847400e720 comp rx=0 tx=0).stop 2024-04-25T10:43:27.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.724+0000 7f847e762640 1 --2- 172.21.15.43:0/1034370114 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f84640afc60 0x7f84640b2170 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f847005f7e0 tx=0x7f847005e040 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:27.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.725+0000 7f8487077740 1 -- 172.21.15.43:0/3827605298 shutdown_connections 2024-04-25T10:43:27.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.725+0000 7f8487077740 1 --2- 172.21.15.43:0/3827605298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b781b6b0 0x55e4b7be98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.725+0000 7f8487077740 1 --2- 172.21.15.43:0/3827605298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 0x55e4b7bdcfa0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.725+0000 7f8487077740 1 --2- 172.21.15.43:0/3827605298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4b783c390 0x55e4b7bdca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.725+0000 7f8487077740 1 -- 172.21.15.43:0/3827605298 >> 172.21.15.43:0/3827605298 conn(0x55e4b78304e0 msgr2=0x55e4b7bd9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:27.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.725+0000 7f8487077740 1 -- 172.21.15.43:0/3827605298 shutdown_connections 2024-04-25T10:43:27.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.725+0000 7f8487077740 1 -- 172.21.15.43:0/3827605298 wait complete. 2024-04-25T10:43:27.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.725+0000 7f8487077740 1 -- start start 2024-04-25T10:43:27.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.725+0000 7f8487077740 1 Processor -- start 2024-04-25T10:43:27.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f8487077740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b78134d0 0x55e4b7be7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f8487077740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b781b6b0 0x55e4b7be7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f8487077740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4b783c390 0x55e4b7bf3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f8487077740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e4b7be9fe0 con 0x55e4b781b6b0 2024-04-25T10:43:27.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f8487077740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e4b7ac9d10 con 0x55e4b78134d0 2024-04-25T10:43:27.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f8487077740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e4b7bea160 con 0x55e4b783c390 2024-04-25T10:43:27.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f8487077740 10 client.0 _ll_register_callbacks cb 0x55e4b7af6210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:27.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f847df61640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b781b6b0 0x55e4b7be7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f847df61640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b781b6b0 0x55e4b7be7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44426/0 (socket says 172.21.15.43:44426) 2024-04-25T10:43:27.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f847df61640 1 -- 172.21.15.43:0/207503488 learned_addr learned my addr 172.21.15.43:0/207503488 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:27.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f847d760640 1 --2- 172.21.15.43:0/207503488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b78134d0 0x55e4b7be7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f847d760640 1 --2- 172.21.15.43:0/207503488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4b783c390 0x55e4b7bf3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f847df61640 1 -- 172.21.15.43:0/207503488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4b783c390 msgr2=0x55e4b7bf3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:27.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f847df61640 1 --2- 172.21.15.43:0/207503488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4b783c390 0x55e4b7bf3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f847df61640 1 -- 172.21.15.43:0/207503488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b78134d0 msgr2=0x55e4b7be7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:27.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f847df61640 1 --2- 172.21.15.43:0/207503488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b78134d0 0x55e4b7be7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f847df61640 1 -- 172.21.15.43:0/207503488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4b791e7d0 con 0x55e4b781b6b0 2024-04-25T10:43:27.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f847d760640 1 --2- 172.21.15.43:0/207503488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4b783c390 0x55e4b7bf3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:43:27.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f847df61640 1 --2- 172.21.15.43:0/207503488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b781b6b0 0x55e4b7be7a70 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f8474005720 tx=0x7f847400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b7079bedcfc158 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:27.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f84617fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:27.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f84617fa640 1 -- 172.21.15.43:0/207503488 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474004260 con 0x55e4b781b6b0 2024-04-25T10:43:27.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f84617fa640 1 -- 172.21.15.43:0/207503488 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f8474004400 con 0x55e4b781b6b0 2024-04-25T10:43:27.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.726+0000 7f8487077740 1 -- 172.21.15.43:0/207503488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8474004c50 con 0x55e4b781b6b0 2024-04-25T10:43:27.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.727+0000 7f84617fa640 1 -- 172.21.15.43:0/207503488 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8474004e00 con 0x55e4b781b6b0 2024-04-25T10:43:27.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.727+0000 7f8487077740 10 client.18684 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:27.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.727+0000 7f8487077740 1 -- 172.21.15.43:0/207503488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e4b78543f0 con 0x55e4b781b6b0 2024-04-25T10:43:27.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.727+0000 7f84617fa640 1 -- 172.21.15.43:0/207503488 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8474004c50 con 0x55e4b781b6b0 2024-04-25T10:43:27.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.727+0000 7f84617fa640 1 -- 172.21.15.43:0/207503488 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8474010510 con 0x55e4b781b6b0 2024-04-25T10:43:27.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.727+0000 7f8487077740 10 client.18684 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:27.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.727+0000 7f8487077740 10 client.18684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:43:27.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.727+0000 7f8487077740 20 client.18684 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:27.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.727+0000 7f8487077740 1 --2- 172.21.15.43:0/207503488 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e4b77faf60 0x55e4b7bf9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.727+0000 7f8487077740 4 client.18684 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:43:27.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.727+0000 7f8487077740 1 -- 172.21.15.43:0/207503488 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4b7b50990 con 0x55e4b77faf60 2024-04-25T10:43:27.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.728+0000 7f847d760640 1 --2- 172.21.15.43:0/207503488 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e4b77faf60 0x55e4b7bf9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.728+0000 7f847d760640 1 --2- 172.21.15.43:0/207503488 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e4b77faf60 0x55e4b7bf9450 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa1b9b154c594ebf server_cookie=c8822eb3138a76e7 in_seq=0 out_seq=0 2024-04-25T10:43:27.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.728+0000 7f84617fa640 10 client.18684 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:27.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.729+0000 7f84617fa640 1 -- 172.21.15.43:0/207503488 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e4b7b50990 con 0x55e4b77faf60 2024-04-25T10:43:27.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.729+0000 7f84617fa640 10 client.18684 handle_command_reply: tid=0 2024-04-25T10:43:27.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.729+0000 7f8487077740 1 client.18684 shutdown 2024-04-25T10:43:27.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.729+0000 7f8487077740 1 -- 172.21.15.43:0/207503488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b781b6b0 msgr2=0x55e4b7be7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:27.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.729+0000 7f8487077740 1 --2- 172.21.15.43:0/207503488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b781b6b0 0x55e4b7be7a70 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f8474005720 tx=0x7f847400c940 comp rx=0 tx=0).stop 2024-04-25T10:43:27.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.729+0000 7f8487077740 1 -- 172.21.15.43:0/207503488 shutdown_connections 2024-04-25T10:43:27.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.729+0000 7f8487077740 1 --2- 172.21.15.43:0/207503488 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e4b77faf60 0x55e4b7bf9450 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.729+0000 7f8487077740 1 --2- 172.21.15.43:0/207503488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4b783c390 0x55e4b7bf3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.729+0000 7f8487077740 1 --2- 172.21.15.43:0/207503488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b781b6b0 0x55e4b7be7a70 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.729+0000 7f84617fa640 0 client.18684 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:27.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.729+0000 7f8487077740 1 --2- 172.21.15.43:0/207503488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b78134d0 0x55e4b7be7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.729+0000 7f8487077740 1 -- 172.21.15.43:0/207503488 >> 172.21.15.43:0/207503488 conn(0x55e4b78304e0 msgr2=0x55e4b7bde780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:27.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.729+0000 7f8487077740 1 -- 172.21.15.43:0/207503488 shutdown_connections 2024-04-25T10:43:27.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.729+0000 7f8487077740 1 -- 172.21.15.43:0/207503488 wait complete. 2024-04-25T10:43:27.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.729+0000 7f8487077740 20 client.18684 trim_cache size 0 max 16384 2024-04-25T10:43:27.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.747+0000 7f8487077740 1 Processor -- start 2024-04-25T10:43:27.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.747+0000 7f8487077740 1 -- start start 2024-04-25T10:43:27.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.748+0000 7f8487077740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 0x55e4b7bef430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.748+0000 7f8487077740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4b781b6b0 0x55e4b7bef970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.748+0000 7f8487077740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b783c390 0x55e4b7bf30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.748+0000 7f8487077740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f84781b2280 con 0x55e4b78134d0 2024-04-25T10:43:27.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.748+0000 7f8487077740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f84781afb50 con 0x55e4b783c390 2024-04-25T10:43:27.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.748+0000 7f8487077740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f84781ad9c0 con 0x55e4b781b6b0 2024-04-25T10:43:27.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.748+0000 7f847d760640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 0x55e4b7bef430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.748+0000 7f847d760640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 0x55e4b7bef430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44428/0 (socket says 172.21.15.43:44428) 2024-04-25T10:43:27.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.748+0000 7f847d760640 1 -- 172.21.15.43:0/753294295 learned_addr learned my addr 172.21.15.43:0/753294295 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:27.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.748+0000 7f847d760640 1 --2- 172.21.15.43:0/753294295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b783c390 0x55e4b7bf30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.748+0000 7f847df61640 1 --2- 172.21.15.43:0/753294295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4b781b6b0 0x55e4b7bef970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.748+0000 7f847d760640 1 -- 172.21.15.43:0/753294295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4b781b6b0 msgr2=0x55e4b7bef970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:27.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.748+0000 7f847d760640 1 --2- 172.21.15.43:0/753294295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4b781b6b0 0x55e4b7bef970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.748+0000 7f847d760640 1 -- 172.21.15.43:0/753294295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b783c390 msgr2=0x55e4b7bf30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:27.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.748+0000 7f847d760640 1 --2- 172.21.15.43:0/753294295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b783c390 0x55e4b7bf30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.748+0000 7f847d760640 1 -- 172.21.15.43:0/753294295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4b78140f0 con 0x55e4b78134d0 2024-04-25T10:43:27.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.748+0000 7f847d760640 1 --2- 172.21.15.43:0/753294295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 0x55e4b7bef430 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f8468006010 tx=0x7f846802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b799b80cbba6489 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:27.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.748+0000 7f84637fe640 1 -- 172.21.15.43:0/753294295 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468035050 con 0x55e4b78134d0 2024-04-25T10:43:27.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.748+0000 7f84637fe640 1 -- 172.21.15.43:0/753294295 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8468004390 con 0x55e4b78134d0 2024-04-25T10:43:27.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.748+0000 7f84637fe640 1 -- 172.21.15.43:0/753294295 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84680482c0 con 0x55e4b78134d0 2024-04-25T10:43:27.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.749+0000 7f8487077740 1 -- 172.21.15.43:0/753294295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 msgr2=0x55e4b7bef430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:27.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.749+0000 7f8487077740 1 --2- 172.21.15.43:0/753294295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 0x55e4b7bef430 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f8468006010 tx=0x7f846802a410 comp rx=0 tx=0).stop 2024-04-25T10:43:27.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.749+0000 7f8487077740 1 -- 172.21.15.43:0/753294295 shutdown_connections 2024-04-25T10:43:27.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.749+0000 7f8487077740 1 --2- 172.21.15.43:0/753294295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b783c390 0x55e4b7bf30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.749+0000 7f8487077740 1 --2- 172.21.15.43:0/753294295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4b781b6b0 0x55e4b7bef970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.749+0000 7f8487077740 1 --2- 172.21.15.43:0/753294295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 0x55e4b7bef430 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.749+0000 7f8487077740 1 -- 172.21.15.43:0/753294295 >> 172.21.15.43:0/753294295 conn(0x55e4b78304e0 msgr2=0x55e4b7be24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:27.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.749+0000 7f8487077740 1 -- 172.21.15.43:0/753294295 shutdown_connections 2024-04-25T10:43:27.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.749+0000 7f8487077740 1 -- 172.21.15.43:0/753294295 wait complete. 2024-04-25T10:43:27.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.749+0000 7f8487077740 1 -- start start 2024-04-25T10:43:27.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.750+0000 7f8487077740 1 Processor -- start 2024-04-25T10:43:27.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.750+0000 7f8487077740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 0x55e4b7bea400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.750+0000 7f8487077740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b781b6b0 0x55e4b7bea940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.750+0000 7f8487077740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4b783c390 0x55e4b7beae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.750+0000 7f8487077740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f84781ae010 con 0x55e4b78134d0 2024-04-25T10:43:27.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.750+0000 7f8487077740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f84781af480 con 0x55e4b781b6b0 2024-04-25T10:43:27.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.750+0000 7f8487077740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f84781ae6e0 con 0x55e4b783c390 2024-04-25T10:43:27.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.750+0000 7f8487077740 10 client.0 _ll_register_callbacks cb 0x7f8474014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:27.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.750+0000 7f847d760640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 0x55e4b7bea400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.750+0000 7f847d760640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 0x55e4b7bea400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44434/0 (socket says 172.21.15.43:44434) 2024-04-25T10:43:27.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.750+0000 7f847d760640 1 -- 172.21.15.43:0/3657441243 learned_addr learned my addr 172.21.15.43:0/3657441243 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:27.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.750+0000 7f847df61640 1 --2- 172.21.15.43:0/3657441243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b781b6b0 0x55e4b7bea940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.750+0000 7f847d760640 1 -- 172.21.15.43:0/3657441243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4b783c390 msgr2=0x55e4b7beae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:27.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.750+0000 7f847d760640 1 --2- 172.21.15.43:0/3657441243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4b783c390 0x55e4b7beae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.750+0000 7f847d760640 1 -- 172.21.15.43:0/3657441243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b781b6b0 msgr2=0x55e4b7bea940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:27.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.750+0000 7f847d760640 1 --2- 172.21.15.43:0/3657441243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b781b6b0 0x55e4b7bea940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.750+0000 7f847d760640 1 -- 172.21.15.43:0/3657441243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4b7b034b0 con 0x55e4b78134d0 2024-04-25T10:43:27.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.751+0000 7f847d760640 1 --2- 172.21.15.43:0/3657441243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 0x55e4b7bea400 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f8468004870 tx=0x7f8468000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfbb231f6f178dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:27.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.751+0000 7f84617fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:27.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.751+0000 7f84617fa640 1 -- 172.21.15.43:0/3657441243 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468035030 con 0x55e4b78134d0 2024-04-25T10:43:27.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.751+0000 7f84617fa640 1 -- 172.21.15.43:0/3657441243 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f84680484c0 con 0x55e4b78134d0 2024-04-25T10:43:27.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.751+0000 7f8487077740 1 -- 172.21.15.43:0/3657441243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8468004390 con 0x55e4b78134d0 2024-04-25T10:43:27.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.751+0000 7f84617fa640 1 -- 172.21.15.43:0/3657441243 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f84680486c0 con 0x55e4b78134d0 2024-04-25T10:43:27.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.751+0000 7f8487077740 10 client.18690 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:27.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.751+0000 7f8487077740 1 -- 172.21.15.43:0/3657441243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e4b7c64c30 con 0x55e4b78134d0 2024-04-25T10:43:27.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.751+0000 7f84617fa640 1 -- 172.21.15.43:0/3657441243 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8468004390 con 0x55e4b78134d0 2024-04-25T10:43:27.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.751+0000 7f84617fa640 1 -- 172.21.15.43:0/3657441243 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f84680035f0 con 0x55e4b78134d0 2024-04-25T10:43:27.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.751+0000 7f8487077740 10 client.18690 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:27.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.751+0000 7f8487077740 10 client.18690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:43:27.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.751+0000 7f8487077740 20 client.18690 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:27.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.751+0000 7f8487077740 1 --2- 172.21.15.43:0/3657441243 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e4b77d03a0 0x55e4b7c673f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.751+0000 7f8487077740 4 client.18690 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:43:27.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.751+0000 7f8487077740 1 -- 172.21.15.43:0/3657441243 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e4b7b4a800 con 0x55e4b77d03a0 2024-04-25T10:43:27.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.752+0000 7f847df61640 1 --2- 172.21.15.43:0/3657441243 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e4b77d03a0 0x55e4b7c673f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.752+0000 7f84617fa640 10 client.18690 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:27.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.752+0000 7f847df61640 1 --2- 172.21.15.43:0/3657441243 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e4b77d03a0 0x55e4b7c673f0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e2d479b24bd0f1f server_cookie=eac695dd7c1b6d7c in_seq=0 out_seq=0 2024-04-25T10:43:27.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.752+0000 7f84617fa640 1 -- 172.21.15.43:0/3657441243 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e4b7b4a800 con 0x55e4b77d03a0 2024-04-25T10:43:27.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.752+0000 7f84617fa640 10 client.18690 handle_command_reply: tid=0 2024-04-25T10:43:27.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.752+0000 7f8487077740 1 client.18690 shutdown 2024-04-25T10:43:27.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.752+0000 7f8487077740 1 -- 172.21.15.43:0/3657441243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 msgr2=0x55e4b7bea400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:27.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.752+0000 7f8487077740 1 --2- 172.21.15.43:0/3657441243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 0x55e4b7bea400 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f8468004870 tx=0x7f8468000bc0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.752+0000 7f8487077740 1 -- 172.21.15.43:0/3657441243 shutdown_connections 2024-04-25T10:43:27.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f8487077740 1 --2- 172.21.15.43:0/3657441243 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55e4b77d03a0 0x55e4b7c673f0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f8487077740 1 --2- 172.21.15.43:0/3657441243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b781b6b0 0x55e4b7bea940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f8487077740 1 --2- 172.21.15.43:0/3657441243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4b783c390 0x55e4b7beae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f8487077740 1 --2- 172.21.15.43:0/3657441243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4b78134d0 0x55e4b7bea400 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f84617fa640 0 client.18690 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:27.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f8487077740 1 -- 172.21.15.43:0/3657441243 >> 172.21.15.43:0/3657441243 conn(0x55e4b78304e0 msgr2=0x55e4b7be5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:27.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f8487077740 1 -- 172.21.15.43:0/3657441243 shutdown_connections 2024-04-25T10:43:27.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f8487077740 1 -- 172.21.15.43:0/3657441243 wait complete. 2024-04-25T10:43:27.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f8487077740 20 client.18690 trim_cache size 0 max 16384 2024-04-25T10:43:27.761 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:43:27.761 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:43:27.761 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:43:27.761 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:43:27.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f84801c5640 1 -- 172.21.15.43:0/1034370114 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f84640afc60 msgr2=0x7f84640b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:27.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f84801c5640 1 --2- 172.21.15.43:0/1034370114 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f84640afc60 0x7f84640b2170 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f847005f7e0 tx=0x7f847005e040 comp rx=0 tx=0).stop 2024-04-25T10:43:27.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f84801c5640 1 -- 172.21.15.43:0/1034370114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8478108eb0 msgr2=0x7f847819ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:27.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f84801c5640 1 --2- 172.21.15.43:0/1034370114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8478108eb0 0x7f847819ef60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8470009fd0 tx=0x7f847000ef10 comp rx=0 tx=0).stop 2024-04-25T10:43:27.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f84801c5640 1 -- 172.21.15.43:0/1034370114 shutdown_connections 2024-04-25T10:43:27.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f84801c5640 1 --2- 172.21.15.43:0/1034370114 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f84640afc60 0x7f84640b2170 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f84801c5640 1 --2- 172.21.15.43:0/1034370114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8478108eb0 0x7f847819ef60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f84801c5640 1 --2- 172.21.15.43:0/1034370114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8478108a10 0x7f847819aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f84801c5640 1 --2- 172.21.15.43:0/1034370114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84781031c0 0x7f847819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f84801c5640 1 -- 172.21.15.43:0/1034370114 >> 172.21.15.43:0/1034370114 conn(0x7f8478078110 msgr2=0x7f8478074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:27.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f84801c5640 1 -- 172.21.15.43:0/1034370114 shutdown_connections 2024-04-25T10:43:27.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.753+0000 7f84801c5640 1 -- 172.21.15.43:0/1034370114 wait complete. 2024-04-25T10:43:27.767 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:43:27.767 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:43:27.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.877+0000 7f5ad2714640 1 -- 172.21.15.43:0/1297888555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc10d380 msgr2=0x7f5acc10f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:27.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.877+0000 7f5ad2714640 1 --2- 172.21.15.43:0/1297888555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc10d380 0x7f5acc10f7a0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009ca0 tx=0x7f5ac402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.877+0000 7f5ad2714640 1 -- 172.21.15.43:0/1297888555 shutdown_connections 2024-04-25T10:43:27.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.877+0000 7f5ad2714640 1 --2- 172.21.15.43:0/1297888555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5acc102af0 0x7f5acc102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.877+0000 7f5ad2714640 1 --2- 172.21.15.43:0/1297888555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc10d380 0x7f5acc10f7a0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.877+0000 7f5ad2714640 1 --2- 172.21.15.43:0/1297888555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5acc108a20 0x7f5acc1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.877+0000 7f5ad2714640 1 -- 172.21.15.43:0/1297888555 >> 172.21.15.43:0/1297888555 conn(0x7f5acc0781c0 msgr2=0x7f5acc078600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:27.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.877+0000 7f5ad2714640 1 -- 172.21.15.43:0/1297888555 shutdown_connections 2024-04-25T10:43:27.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.877+0000 7f5ad2714640 1 -- 172.21.15.43:0/1297888555 wait complete. 2024-04-25T10:43:27.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.877+0000 7f5ad2714640 1 Processor -- start 2024-04-25T10:43:27.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.877+0000 7f5ad2714640 1 -- start start 2024-04-25T10:43:27.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.878+0000 7f5ad2714640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5acc102af0 0x7f5acc19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.878+0000 7f5ad2714640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc108a20 0x7f5acc19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.878+0000 7f5ad2714640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5acc10d380 0x7f5acc19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.878+0000 7f5ad2714640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc112c00 con 0x7f5acc108a20 2024-04-25T10:43:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.878+0000 7f5ad2714640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc112a80 con 0x7f5acc10d380 2024-04-25T10:43:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.878+0000 7f5ad2714640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc112d80 con 0x7f5acc102af0 2024-04-25T10:43:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.878+0000 7f5acb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc108a20 0x7f5acc19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.878+0000 7f5acb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc108a20 0x7f5acc19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44462/0 (socket says 172.21.15.43:44462) 2024-04-25T10:43:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.878+0000 7f5acb7fe640 1 -- 172.21.15.43:0/1263901241 learned_addr learned my addr 172.21.15.43:0/1263901241 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.878+0000 7f5ad0cb1640 1 --2- 172.21.15.43:0/1263901241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5acc10d380 0x7f5acc19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.878+0000 7f5acbfff640 1 --2- 172.21.15.43:0/1263901241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5acc102af0 0x7f5acc19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.878+0000 7f5acb7fe640 1 -- 172.21.15.43:0/1263901241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5acc102af0 msgr2=0x7f5acc19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.878+0000 7f5acb7fe640 1 --2- 172.21.15.43:0/1263901241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5acc102af0 0x7f5acc19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.878+0000 7f5acb7fe640 1 -- 172.21.15.43:0/1263901241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5acc10d380 msgr2=0x7f5acc19ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.878+0000 7f5acb7fe640 1 --2- 172.21.15.43:0/1263901241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5acc10d380 0x7f5acc19ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.878+0000 7f5acb7fe640 1 -- 172.21.15.43:0/1263901241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5acc19f4b0 con 0x7f5acc108a20 2024-04-25T10:43:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.878+0000 7f5acbfff640 1 --2- 172.21.15.43:0/1263901241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5acc102af0 0x7f5acc19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:43:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.878+0000 7f5ad0cb1640 1 --2- 172.21.15.43:0/1263901241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5acc10d380 0x7f5acc19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:43:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.879+0000 7f5acb7fe640 1 --2- 172.21.15.43:0/1263901241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc108a20 0x7f5acc19abb0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00e9f0 tx=0x7f5abc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.879+0000 7f5ac97fa640 1 -- 172.21.15.43:0/1263901241 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc00ccc0 con 0x7f5acc108a20 2024-04-25T10:43:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.879+0000 7f5ac97fa640 1 -- 172.21.15.43:0/1263901241 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5abc0035f0 con 0x7f5acc108a20 2024-04-25T10:43:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.879+0000 7f5ad2714640 1 -- 172.21.15.43:0/1263901241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5acc19f7a0 con 0x7f5acc108a20 2024-04-25T10:43:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.879+0000 7f5ac97fa640 1 -- 172.21.15.43:0/1263901241 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc00ce60 con 0x7f5acc108a20 2024-04-25T10:43:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.879+0000 7f5ad2714640 1 -- 172.21.15.43:0/1263901241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5acc1a7120 con 0x7f5acc108a20 2024-04-25T10:43:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.880+0000 7f5ad2714640 1 -- 172.21.15.43:0/1263901241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5acc069310 con 0x7f5acc108a20 2024-04-25T10:43:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.881+0000 7f5ac97fa640 1 -- 172.21.15.43:0/1263901241 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f5abc003aa0 con 0x7f5acc108a20 2024-04-25T10:43:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.882+0000 7f5ac97fa640 1 --2- 172.21.15.43:0/1263901241 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5ab80afce0 0x7f5ab80b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.882+0000 7f5acbfff640 1 --2- 172.21.15.43:0/1263901241 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5ab80afce0 0x7f5ab80b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.882+0000 7f5ac97fa640 1 -- 172.21.15.43:0/1263901241 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f5abc0df460 con 0x7f5acc108a20 2024-04-25T10:43:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.883+0000 7f5acbfff640 1 --2- 172.21.15.43:0/1263901241 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5ab80afce0 0x7f5ab80b21f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5acc103a20 tx=0x7f5ab4005e50 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:27.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:27.887+0000 7f5ac97fa640 1 -- 172.21.15.43:0/1263901241 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5abc0a7ca0 con 0x7f5acc108a20 2024-04-25T10:43:28.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.113+0000 7f5ad2714640 1 -- 172.21.15.43:0/1263901241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5acc19b9a0 con 0x7f5acc108a20 2024-04-25T10:43:28.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.114+0000 7f5ac97fa640 1 -- 172.21.15.43:0/1263901241 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7f5abc0a75a0 con 0x7f5acc108a20 2024-04-25T10:43:28.116 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:43:28.116 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:43:28.116 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:43:28.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.116+0000 7f5ad2714640 1 -- 172.21.15.43:0/1263901241 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5ab80afce0 msgr2=0x7f5ab80b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.116+0000 7f5ad2714640 1 --2- 172.21.15.43:0/1263901241 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5ab80afce0 0x7f5ab80b21f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5acc103a20 tx=0x7f5ab4005e50 comp rx=0 tx=0).stop 2024-04-25T10:43:28.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.116+0000 7f5ad2714640 1 -- 172.21.15.43:0/1263901241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc108a20 msgr2=0x7f5acc19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.116+0000 7f5ad2714640 1 --2- 172.21.15.43:0/1263901241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc108a20 0x7f5acc19abb0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00e9f0 tx=0x7f5abc00eeb0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.116+0000 7f5ad2714640 1 -- 172.21.15.43:0/1263901241 shutdown_connections 2024-04-25T10:43:28.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.116+0000 7f5ad2714640 1 --2- 172.21.15.43:0/1263901241 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5ab80afce0 0x7f5ab80b21f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.116+0000 7f5ad2714640 1 --2- 172.21.15.43:0/1263901241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5acc10d380 0x7f5acc19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.116+0000 7f5ad2714640 1 --2- 172.21.15.43:0/1263901241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc108a20 0x7f5acc19abb0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.116+0000 7f5ad2714640 1 --2- 172.21.15.43:0/1263901241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5acc102af0 0x7f5acc19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.116+0000 7f5ad2714640 1 -- 172.21.15.43:0/1263901241 >> 172.21.15.43:0/1263901241 conn(0x7f5acc0781c0 msgr2=0x7f5acc06ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.116+0000 7f5ad2714640 1 -- 172.21.15.43:0/1263901241 shutdown_connections 2024-04-25T10:43:28.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.116+0000 7f5ad2714640 1 -- 172.21.15.43:0/1263901241 wait complete. 2024-04-25T10:43:28.130 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:43:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.242+0000 7fe221a42640 1 -- 172.21.15.43:0/1850619844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe21c104c50 msgr2=0x7fe21c105060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.242+0000 7fe221a42640 1 --2- 172.21.15.43:0/1850619844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe21c104c50 0x7fe21c105060 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009ca0 tx=0x7fe20c02a9e0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.242+0000 7fe221a42640 1 -- 172.21.15.43:0/1850619844 shutdown_connections 2024-04-25T10:43:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.242+0000 7fe221a42640 1 --2- 172.21.15.43:0/1850619844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe21c10e060 0x7fe21c110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.242+0000 7fe221a42640 1 --2- 172.21.15.43:0/1850619844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe21c10b6b0 0x7fe21c10dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.242+0000 7fe221a42640 1 --2- 172.21.15.43:0/1850619844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe21c104c50 0x7fe21c105060 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.242+0000 7fe221a42640 1 -- 172.21.15.43:0/1850619844 >> 172.21.15.43:0/1850619844 conn(0x7fe21c078100 msgr2=0x7fe21c078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.242+0000 7fe221a42640 1 -- 172.21.15.43:0/1850619844 shutdown_connections 2024-04-25T10:43:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.242+0000 7fe221a42640 1 -- 172.21.15.43:0/1850619844 wait complete. 2024-04-25T10:43:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.243+0000 7fe221a42640 1 Processor -- start 2024-04-25T10:43:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.243+0000 7fe221a42640 1 -- start start 2024-04-25T10:43:28.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.243+0000 7fe221a42640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe21c104c50 0x7fe21c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.243+0000 7fe221a42640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe21c10b6b0 0x7fe21c19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.243+0000 7fe221a42640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe21c10e060 0x7fe21c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.243+0000 7fe221a42640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c113cc0 con 0x7fe21c10e060 2024-04-25T10:43:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.243+0000 7fe221a42640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c113b40 con 0x7fe21c10b6b0 2024-04-25T10:43:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.243+0000 7fe221a42640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c113e40 con 0x7fe21c104c50 2024-04-25T10:43:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.243+0000 7fe21b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe21c10e060 0x7fe21c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.243+0000 7fe21b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe21c10e060 0x7fe21c19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44476/0 (socket says 172.21.15.43:44476) 2024-04-25T10:43:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.243+0000 7fe21b7fe640 1 -- 172.21.15.43:0/4144834379 learned_addr learned my addr 172.21.15.43:0/4144834379 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.243+0000 7fe21affd640 1 --2- 172.21.15.43:0/4144834379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe21c104c50 0x7fe21c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.244+0000 7fe21a7fc640 1 --2- 172.21.15.43:0/4144834379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe21c10b6b0 0x7fe21c19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.244+0000 7fe21b7fe640 1 -- 172.21.15.43:0/4144834379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe21c104c50 msgr2=0x7fe21c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.244+0000 7fe21b7fe640 1 --2- 172.21.15.43:0/4144834379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe21c104c50 0x7fe21c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.244+0000 7fe21b7fe640 1 -- 172.21.15.43:0/4144834379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe21c10b6b0 msgr2=0x7fe21c19aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.244+0000 7fe21b7fe640 1 --2- 172.21.15.43:0/4144834379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe21c10b6b0 0x7fe21c19aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.244+0000 7fe21b7fe640 1 -- 172.21.15.43:0/4144834379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe21c19f5a0 con 0x7fe21c10e060 2024-04-25T10:43:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.244+0000 7fe21affd640 1 --2- 172.21.15.43:0/4144834379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe21c104c50 0x7fe21c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:43:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.244+0000 7fe21b7fe640 1 --2- 172.21.15.43:0/4144834379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe21c10e060 0x7fe21c19ef60 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fe21400e970 tx=0x7fe21400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.244+0000 7fe220a40640 1 -- 172.21.15.43:0/4144834379 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21400ccc0 con 0x7fe21c10e060 2024-04-25T10:43:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.244+0000 7fe221a42640 1 -- 172.21.15.43:0/4144834379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe21c19f890 con 0x7fe21c10e060 2024-04-25T10:43:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.244+0000 7fe220a40640 1 -- 172.21.15.43:0/4144834379 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe21400ce60 con 0x7fe21c10e060 2024-04-25T10:43:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.244+0000 7fe220a40640 1 -- 172.21.15.43:0/4144834379 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214010430 con 0x7fe21c10e060 2024-04-25T10:43:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.244+0000 7fe221a42640 1 -- 172.21.15.43:0/4144834379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe21c19fd90 con 0x7fe21c10e060 2024-04-25T10:43:28.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.246+0000 7fe2289c8740 1 Processor -- start 2024-04-25T10:43:28.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.247+0000 7fe220a40640 1 -- 172.21.15.43:0/4144834379 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fe214004ff0 con 0x7fe21c10e060 2024-04-25T10:43:28.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.247+0000 7fe2289c8740 1 -- start start 2024-04-25T10:43:28.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.247+0000 7fe2289c8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1e1f3e390 0x55d1e22df810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.247+0000 7fe2289c8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f154d0 0x55d1e22dfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.247+0000 7fe2289c8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f1d6b0 0x55d1e22ec6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.247+0000 7fe2289c8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d1e2244d80 con 0x55d1e1f1d6b0 2024-04-25T10:43:28.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.247+0000 7fe2289c8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d1e21fea20 con 0x55d1e1f154d0 2024-04-25T10:43:28.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.247+0000 7fe2289c8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d1e2240d40 con 0x55d1e1f3e390 2024-04-25T10:43:28.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.247+0000 7fe21a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f1d6b0 0x55d1e22ec6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.247+0000 7fe21a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f1d6b0 0x55d1e22ec6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44484/0 (socket says 172.21.15.43:44484) 2024-04-25T10:43:28.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.247+0000 7fe21a7fc640 1 -- 172.21.15.43:0/4042408305 learned_addr learned my addr 172.21.15.43:0/4042408305 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.247+0000 7fe220a40640 1 --2- 172.21.15.43:0/4144834379 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe2040afc60 0x7fe2040b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.247+0000 7fe21affd640 1 --2- 172.21.15.43:0/4042408305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f154d0 0x55d1e22dfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.247+0000 7fe21a7fc640 1 --2- 172.21.15.43:0/4042408305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1e1f3e390 0x55d1e22df810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.247+0000 7fe21b7fe640 1 --2- 172.21.15.43:0/4144834379 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe2040afc60 0x7fe2040b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.247+0000 7fe21a7fc640 1 -- 172.21.15.43:0/4042408305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1e1f3e390 msgr2=0x55d1e22df810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.247+0000 7fe21a7fc640 1 --2- 172.21.15.43:0/4042408305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1e1f3e390 0x55d1e22df810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.247+0000 7fe21a7fc640 1 -- 172.21.15.43:0/4042408305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f154d0 msgr2=0x55d1e22dfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.247+0000 7fe21a7fc640 1 --2- 172.21.15.43:0/4042408305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f154d0 0x55d1e22dfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.247+0000 7fe21a7fc640 1 -- 172.21.15.43:0/4042408305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1e2209c30 con 0x55d1e1f1d6b0 2024-04-25T10:43:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.248+0000 7fe220a40640 1 -- 172.21.15.43:0/4144834379 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fe2140df540 con 0x7fe21c10e060 2024-04-25T10:43:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.248+0000 7fe21a7fc640 1 --2- 172.21.15.43:0/4042408305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f1d6b0 0x55d1e22ec6a0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe21000efd0 tx=0x7fe21000e5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d3a3415bb9768a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.248+0000 7fe201ffb640 1 -- 172.21.15.43:0/4042408305 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210018030 con 0x55d1e1f1d6b0 2024-04-25T10:43:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.248+0000 7fe21b7fe640 1 --2- 172.21.15.43:0/4144834379 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe2040afc60 0x7fe2040b2170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe2140a4950 tx=0x7fe2140a2f00 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.248+0000 7fe201ffb640 1 -- 172.21.15.43:0/4042408305 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2100054d0 con 0x55d1e1f1d6b0 2024-04-25T10:43:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.248+0000 7fe201ffb640 1 -- 172.21.15.43:0/4042408305 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210010640 con 0x55d1e1f1d6b0 2024-04-25T10:43:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.249+0000 7fe2289c8740 1 -- 172.21.15.43:0/4042408305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f1d6b0 msgr2=0x55d1e22ec6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.249+0000 7fe2289c8740 1 --2- 172.21.15.43:0/4042408305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f1d6b0 0x55d1e22ec6a0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe21000efd0 tx=0x7fe21000e5b0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.249+0000 7fe2289c8740 1 -- 172.21.15.43:0/4042408305 shutdown_connections 2024-04-25T10:43:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.249+0000 7fe2289c8740 1 --2- 172.21.15.43:0/4042408305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f154d0 0x55d1e22dfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.249+0000 7fe2289c8740 1 --2- 172.21.15.43:0/4042408305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f1d6b0 0x55d1e22ec6a0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.249+0000 7fe2289c8740 1 --2- 172.21.15.43:0/4042408305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1e1f3e390 0x55d1e22df810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.249+0000 7fe2289c8740 1 -- 172.21.15.43:0/4042408305 >> 172.21.15.43:0/4042408305 conn(0x55d1e1f324e0 msgr2=0x55d1e22dbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.249+0000 7fe2289c8740 1 -- 172.21.15.43:0/4042408305 shutdown_connections 2024-04-25T10:43:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.249+0000 7fe2289c8740 1 -- 172.21.15.43:0/4042408305 wait complete. 2024-04-25T10:43:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.249+0000 7fe2289c8740 1 -- start start 2024-04-25T10:43:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.249+0000 7fe2289c8740 1 Processor -- start 2024-04-25T10:43:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.250+0000 7fe2289c8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f154d0 0x55d1e22ea2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.250+0000 7fe2289c8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1e1f1d6b0 0x55d1e22ea7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.250+0000 7fe2289c8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f3e390 0x55d1e22f6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.250+0000 7fe2289c8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d1e22ecd90 con 0x55d1e1f3e390 2024-04-25T10:43:28.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.250+0000 7fe2289c8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d1e21cbdc0 con 0x55d1e1f154d0 2024-04-25T10:43:28.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.250+0000 7fe2289c8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d1e22ecf10 con 0x55d1e1f1d6b0 2024-04-25T10:43:28.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.250+0000 7fe2289c8740 10 client.0 _ll_register_callbacks cb 0x55d1e21e7600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:28.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.250+0000 7fe21a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f3e390 0x55d1e22f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.250+0000 7fe21a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f3e390 0x55d1e22f6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44486/0 (socket says 172.21.15.43:44486) 2024-04-25T10:43:28.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.250+0000 7fe21a7fc640 1 -- 172.21.15.43:0/3551332360 learned_addr learned my addr 172.21.15.43:0/3551332360 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:28.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.250+0000 7fe21a7fc640 1 --2- 172.21.15.43:0/3551332360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f154d0 0x55d1e22ea2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.250+0000 7fe21affd640 1 --2- 172.21.15.43:0/3551332360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1e1f1d6b0 0x55d1e22ea7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.250+0000 7fe21a7fc640 1 -- 172.21.15.43:0/3551332360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1e1f1d6b0 msgr2=0x55d1e22ea7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.250+0000 7fe21a7fc640 1 --2- 172.21.15.43:0/3551332360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1e1f1d6b0 0x55d1e22ea7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.250+0000 7fe21a7fc640 1 -- 172.21.15.43:0/3551332360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f154d0 msgr2=0x55d1e22ea2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.250+0000 7fe21a7fc640 1 --2- 172.21.15.43:0/3551332360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f154d0 0x55d1e22ea2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.250+0000 7fe21a7fc640 1 -- 172.21.15.43:0/3551332360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1e1f563f0 con 0x55d1e1f3e390 2024-04-25T10:43:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.251+0000 7fe21a7fc640 1 --2- 172.21.15.43:0/3551332360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f3e390 0x55d1e22f6210 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe210004e10 tx=0x7fe21000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f343c6f600c19d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.251+0000 7fe1faffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.251+0000 7fe1faffd640 1 -- 172.21.15.43:0/3551332360 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210018050 con 0x55d1e1f3e390 2024-04-25T10:43:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.251+0000 7fe2289c8740 1 -- 172.21.15.43:0/3551332360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2100054d0 con 0x55d1e1f3e390 2024-04-25T10:43:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.251+0000 7fe1faffd640 1 -- 172.21.15.43:0/3551332360 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fe21001d460 con 0x55d1e1f3e390 2024-04-25T10:43:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.251+0000 7fe1faffd640 1 -- 172.21.15.43:0/3551332360 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe210033050 con 0x55d1e1f3e390 2024-04-25T10:43:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.251+0000 7fe2289c8740 10 client.18708 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.251+0000 7fe2289c8740 1 -- 172.21.15.43:0/3551332360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d1e20207d0 con 0x55d1e1f3e390 2024-04-25T10:43:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.251+0000 7fe1faffd640 1 -- 172.21.15.43:0/3551332360 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2100054d0 con 0x55d1e1f3e390 2024-04-25T10:43:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.251+0000 7fe1faffd640 1 -- 172.21.15.43:0/3551332360 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe21002b020 con 0x55d1e1f3e390 2024-04-25T10:43:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.251+0000 7fe2289c8740 10 client.18708 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.251+0000 7fe2289c8740 10 client.18708 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:43:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.251+0000 7fe2289c8740 20 client.18708 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.251+0000 7fe2289c8740 1 --2- 172.21.15.43:0/3551332360 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d1e1efcf60 0x55d1e22fc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.251+0000 7fe2289c8740 4 client.18708 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:43:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.251+0000 7fe2289c8740 1 -- 172.21.15.43:0/3551332360 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1e2252990 con 0x55d1e1efcf60 2024-04-25T10:43:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.252+0000 7fe21affd640 1 --2- 172.21.15.43:0/3551332360 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d1e1efcf60 0x55d1e22fc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.252+0000 7fe1faffd640 10 client.18708 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:43:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.252+0000 7fe21affd640 1 --2- 172.21.15.43:0/3551332360 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d1e1efcf60 0x55d1e22fc650 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eb9d5b5ca140820e server_cookie=4d53c5c0386cd34 in_seq=0 out_seq=0 2024-04-25T10:43:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.253+0000 7fe1faffd640 1 -- 172.21.15.43:0/3551332360 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d1e2252990 con 0x55d1e1efcf60 2024-04-25T10:43:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.253+0000 7fe1faffd640 10 client.18708 handle_command_reply: tid=0 2024-04-25T10:43:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.253+0000 7fe2289c8740 1 client.18708 shutdown 2024-04-25T10:43:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.253+0000 7fe2289c8740 1 -- 172.21.15.43:0/3551332360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f3e390 msgr2=0x55d1e22f6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.253+0000 7fe2289c8740 1 --2- 172.21.15.43:0/3551332360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f3e390 0x55d1e22f6210 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe210004e10 tx=0x7fe21000ee90 comp rx=0 tx=0).stop 2024-04-25T10:43:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.253+0000 7fe2289c8740 1 -- 172.21.15.43:0/3551332360 shutdown_connections 2024-04-25T10:43:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.253+0000 7fe2289c8740 1 --2- 172.21.15.43:0/3551332360 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d1e1efcf60 0x55d1e22fc650 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.253+0000 7fe2289c8740 1 --2- 172.21.15.43:0/3551332360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f3e390 0x55d1e22f6210 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.253+0000 7fe2289c8740 1 --2- 172.21.15.43:0/3551332360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1e1f1d6b0 0x55d1e22ea7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.253+0000 7fe1faffd640 0 client.18708 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:43:28.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.253+0000 7fe2289c8740 1 --2- 172.21.15.43:0/3551332360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f154d0 0x55d1e22ea2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.253+0000 7fe2289c8740 1 -- 172.21.15.43:0/3551332360 >> 172.21.15.43:0/3551332360 conn(0x55d1e1f324e0 msgr2=0x55d1e22e1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.254+0000 7fe2289c8740 1 -- 172.21.15.43:0/3551332360 shutdown_connections 2024-04-25T10:43:28.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.254+0000 7fe2289c8740 1 -- 172.21.15.43:0/3551332360 wait complete. 2024-04-25T10:43:28.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.254+0000 7fe2289c8740 20 client.18708 trim_cache size 0 max 16384 2024-04-25T10:43:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.267+0000 7fe2289c8740 1 Processor -- start 2024-04-25T10:43:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.267+0000 7fe2289c8740 1 -- start start 2024-04-25T10:43:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.268+0000 7fe2289c8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f154d0 0x55d1e22e4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.268+0000 7fe2289c8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f1d6b0 0x55d1e22e4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.268+0000 7fe2289c8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1e1f3e390 0x55d1e2301c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.268+0000 7fe2289c8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c1b2250 con 0x55d1e1f154d0 2024-04-25T10:43:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.268+0000 7fe2289c8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c1afb20 con 0x55d1e1f1d6b0 2024-04-25T10:43:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.268+0000 7fe2289c8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c1ad990 con 0x55d1e1f3e390 2024-04-25T10:43:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.268+0000 7fe21a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f154d0 0x55d1e22e4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.268+0000 7fe21a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f154d0 0x55d1e22e4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44498/0 (socket says 172.21.15.43:44498) 2024-04-25T10:43:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.268+0000 7fe21a7fc640 1 -- 172.21.15.43:0/1643615059 learned_addr learned my addr 172.21.15.43:0/1643615059 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.268+0000 7fe21affd640 1 --2- 172.21.15.43:0/1643615059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f1d6b0 0x55d1e22e4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.268+0000 7fe21a7fc640 1 --2- 172.21.15.43:0/1643615059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1e1f3e390 0x55d1e2301c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.268+0000 7fe21a7fc640 1 -- 172.21.15.43:0/1643615059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1e1f3e390 msgr2=0x55d1e2301c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.268+0000 7fe21a7fc640 1 --2- 172.21.15.43:0/1643615059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1e1f3e390 0x55d1e2301c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.268+0000 7fe21a7fc640 1 -- 172.21.15.43:0/1643615059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f1d6b0 msgr2=0x55d1e22e4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.268+0000 7fe21a7fc640 1 --2- 172.21.15.43:0/1643615059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f1d6b0 0x55d1e22e4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.268+0000 7fe21a7fc640 1 -- 172.21.15.43:0/1643615059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1e22dd110 con 0x55d1e1f154d0 2024-04-25T10:43:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.268+0000 7fe21a7fc640 1 --2- 172.21.15.43:0/1643615059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f154d0 0x55d1e22e4430 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe21000e4d0 tx=0x7fe210014fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22981179d0e6e067 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.268+0000 7fe2017fa640 1 -- 172.21.15.43:0/1643615059 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210018040 con 0x55d1e1f154d0 2024-04-25T10:43:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.268+0000 7fe2017fa640 1 -- 172.21.15.43:0/1643615059 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe210022070 con 0x55d1e1f154d0 2024-04-25T10:43:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.268+0000 7fe2017fa640 1 -- 172.21.15.43:0/1643615059 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210005450 con 0x55d1e1f154d0 2024-04-25T10:43:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.269+0000 7fe2289c8740 1 -- 172.21.15.43:0/1643615059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f154d0 msgr2=0x55d1e22e4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.269+0000 7fe2289c8740 1 --2- 172.21.15.43:0/1643615059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f154d0 0x55d1e22e4430 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe21000e4d0 tx=0x7fe210014fc0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.269+0000 7fe2289c8740 1 -- 172.21.15.43:0/1643615059 shutdown_connections 2024-04-25T10:43:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.269+0000 7fe2289c8740 1 --2- 172.21.15.43:0/1643615059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f1d6b0 0x55d1e22e4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.269+0000 7fe2289c8740 1 --2- 172.21.15.43:0/1643615059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1e1f3e390 0x55d1e2301c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.269+0000 7fe2289c8740 1 --2- 172.21.15.43:0/1643615059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f154d0 0x55d1e22e4430 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.269+0000 7fe2289c8740 1 -- 172.21.15.43:0/1643615059 >> 172.21.15.43:0/1643615059 conn(0x55d1e1f324e0 msgr2=0x55d1e22db040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.269+0000 7fe2289c8740 1 -- 172.21.15.43:0/1643615059 shutdown_connections 2024-04-25T10:43:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.269+0000 7fe2289c8740 1 -- 172.21.15.43:0/1643615059 wait complete. 2024-04-25T10:43:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.269+0000 7fe2289c8740 1 -- start start 2024-04-25T10:43:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.270+0000 7fe2289c8740 1 Processor -- start 2024-04-25T10:43:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.270+0000 7fe2289c8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f154d0 0x55d1e22ef1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.270+0000 7fe2289c8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f1d6b0 0x55d1e22ef700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.270+0000 7fe2289c8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1e1f3e390 0x55d1e2362830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.270+0000 7fe2289c8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c1adfe0 con 0x55d1e1f154d0 2024-04-25T10:43:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.270+0000 7fe2289c8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c1af450 con 0x55d1e1f1d6b0 2024-04-25T10:43:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.270+0000 7fe2289c8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c1ae6b0 con 0x55d1e1f3e390 2024-04-25T10:43:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.270+0000 7fe2289c8740 10 client.0 _ll_register_callbacks cb 0x7fe21000ef20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.270+0000 7fe21a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f154d0 0x55d1e22ef1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.270+0000 7fe21a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f154d0 0x55d1e22ef1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44514/0 (socket says 172.21.15.43:44514) 2024-04-25T10:43:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.270+0000 7fe21a7fc640 1 -- 172.21.15.43:0/1378317375 learned_addr learned my addr 172.21.15.43:0/1378317375 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.270+0000 7fe21affd640 1 --2- 172.21.15.43:0/1378317375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f1d6b0 0x55d1e22ef700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.270+0000 7fe21a7fc640 1 -- 172.21.15.43:0/1378317375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1e1f3e390 msgr2=0x55d1e2362830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.270+0000 7fe21a7fc640 1 --2- 172.21.15.43:0/1378317375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1e1f3e390 0x55d1e2362830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.270+0000 7fe21a7fc640 1 -- 172.21.15.43:0/1378317375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f1d6b0 msgr2=0x55d1e22ef700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.270+0000 7fe21a7fc640 1 --2- 172.21.15.43:0/1378317375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f1d6b0 0x55d1e22ef700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.270+0000 7fe21a7fc640 1 -- 172.21.15.43:0/1378317375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1e22effe0 con 0x55d1e1f154d0 2024-04-25T10:43:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.270+0000 7fe21affd640 1 --2- 172.21.15.43:0/1378317375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f1d6b0 0x55d1e22ef700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:43:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.271+0000 7fe21a7fc640 1 --2- 172.21.15.43:0/1378317375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f154d0 0x55d1e22ef1c0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fe21002c3f0 tx=0x7fe210005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f388b6b56095c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.271+0000 7fe1faffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:28.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.271+0000 7fe1faffd640 1 -- 172.21.15.43:0/1378317375 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21000e630 con 0x55d1e1f154d0 2024-04-25T10:43:28.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.271+0000 7fe2289c8740 1 -- 172.21.15.43:0/1378317375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe210022070 con 0x55d1e1f154d0 2024-04-25T10:43:28.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.271+0000 7fe1faffd640 1 -- 172.21.15.43:0/1378317375 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fe210028040 con 0x55d1e1f154d0 2024-04-25T10:43:28.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.271+0000 7fe1faffd640 1 -- 172.21.15.43:0/1378317375 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe21002cba0 con 0x55d1e1f154d0 2024-04-25T10:43:28.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.271+0000 7fe2289c8740 10 client.18714 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:28.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.271+0000 7fe2289c8740 1 -- 172.21.15.43:0/1378317375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d1e2363e00 con 0x55d1e1f154d0 2024-04-25T10:43:28.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.271+0000 7fe1faffd640 1 -- 172.21.15.43:0/1378317375 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe210022070 con 0x55d1e1f154d0 2024-04-25T10:43:28.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.271+0000 7fe1faffd640 1 -- 172.21.15.43:0/1378317375 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe21002b070 con 0x55d1e1f154d0 2024-04-25T10:43:28.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.271+0000 7fe2289c8740 10 client.18714 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:28.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.271+0000 7fe2289c8740 10 client.18714 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:43:28.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.271+0000 7fe2289c8740 20 client.18714 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:28.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.271+0000 7fe2289c8740 1 --2- 172.21.15.43:0/1378317375 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d1e1ed23a0 0x55d1e2366470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.271+0000 7fe2289c8740 4 client.18714 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:43:28.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.271+0000 7fe2289c8740 1 -- 172.21.15.43:0/1378317375 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1e224c800 con 0x55d1e1ed23a0 2024-04-25T10:43:28.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.272+0000 7fe21affd640 1 --2- 172.21.15.43:0/1378317375 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d1e1ed23a0 0x55d1e2366470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.272+0000 7fe1faffd640 10 client.18714 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:43:28.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.272+0000 7fe21affd640 1 --2- 172.21.15.43:0/1378317375 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d1e1ed23a0 0x55d1e2366470 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a2a3e2a1605a998e server_cookie=b714ad0b8089b0cf in_seq=0 out_seq=0 2024-04-25T10:43:28.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.272+0000 7fe1faffd640 1 -- 172.21.15.43:0/1378317375 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1e224c800 con 0x55d1e1ed23a0 2024-04-25T10:43:28.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.272+0000 7fe1faffd640 10 client.18714 handle_command_reply: tid=0 2024-04-25T10:43:28.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.272+0000 7fe2289c8740 1 client.18714 shutdown 2024-04-25T10:43:28.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.272+0000 7fe2289c8740 1 -- 172.21.15.43:0/1378317375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f154d0 msgr2=0x55d1e22ef1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.272+0000 7fe2289c8740 1 --2- 172.21.15.43:0/1378317375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f154d0 0x55d1e22ef1c0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fe21002c3f0 tx=0x7fe210005680 comp rx=0 tx=0).stop 2024-04-25T10:43:28.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.272+0000 7fe2289c8740 1 -- 172.21.15.43:0/1378317375 shutdown_connections 2024-04-25T10:43:28.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.272+0000 7fe2289c8740 1 --2- 172.21.15.43:0/1378317375 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55d1e1ed23a0 0x55d1e2366470 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.272+0000 7fe2289c8740 1 --2- 172.21.15.43:0/1378317375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1e1f1d6b0 0x55d1e22ef700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.272+0000 7fe2289c8740 1 --2- 172.21.15.43:0/1378317375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1e1f3e390 0x55d1e2362830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.272+0000 7fe2289c8740 1 --2- 172.21.15.43:0/1378317375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1e1f154d0 0x55d1e22ef1c0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.273+0000 7fe2289c8740 1 -- 172.21.15.43:0/1378317375 >> 172.21.15.43:0/1378317375 conn(0x55d1e1f324e0 msgr2=0x55d1e22eb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.273+0000 7fe1faffd640 0 client.18714 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:43:28.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.273+0000 7fe2289c8740 1 -- 172.21.15.43:0/1378317375 shutdown_connections 2024-04-25T10:43:28.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.273+0000 7fe2289c8740 1 -- 172.21.15.43:0/1378317375 wait complete. 2024-04-25T10:43:28.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.273+0000 7fe2289c8740 20 client.18714 trim_cache size 0 max 16384 2024-04-25T10:43:28.281 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:43:28.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.273+0000 7fe221a42640 1 -- 172.21.15.43:0/4144834379 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe2040afc60 msgr2=0x7fe2040b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.273+0000 7fe221a42640 1 --2- 172.21.15.43:0/4144834379 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe2040afc60 0x7fe2040b2170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe2140a4950 tx=0x7fe2140a2f00 comp rx=0 tx=0).stop 2024-04-25T10:43:28.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.273+0000 7fe221a42640 1 -- 172.21.15.43:0/4144834379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe21c10e060 msgr2=0x7fe21c19ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.273+0000 7fe221a42640 1 --2- 172.21.15.43:0/4144834379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe21c10e060 0x7fe21c19ef60 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fe21400e970 tx=0x7fe21400ed70 comp rx=0 tx=0).stop 2024-04-25T10:43:28.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.273+0000 7fe221a42640 1 -- 172.21.15.43:0/4144834379 shutdown_connections 2024-04-25T10:43:28.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.273+0000 7fe221a42640 1 --2- 172.21.15.43:0/4144834379 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe2040afc60 0x7fe2040b2170 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.273+0000 7fe221a42640 1 --2- 172.21.15.43:0/4144834379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe21c10b6b0 0x7fe21c19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.273+0000 7fe221a42640 1 --2- 172.21.15.43:0/4144834379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe21c10e060 0x7fe21c19ef60 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.273+0000 7fe221a42640 1 --2- 172.21.15.43:0/4144834379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe21c104c50 0x7fe21c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.273+0000 7fe221a42640 1 -- 172.21.15.43:0/4144834379 >> 172.21.15.43:0/4144834379 conn(0x7fe21c078100 msgr2=0x7fe21c101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.273+0000 7fe221a42640 1 -- 172.21.15.43:0/4144834379 shutdown_connections 2024-04-25T10:43:28.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.273+0000 7fe221a42640 1 -- 172.21.15.43:0/4144834379 wait complete. 2024-04-25T10:43:28.285 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:28 smithi043 ceph-mon[29478]: pgmap v341: 97 pgs: 97 active+clean; 5.1 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:43:28.287 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:43:28.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:28 smithi077 ceph-mon[32772]: pgmap v341: 97 pgs: 97 active+clean; 5.1 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:43:28.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:28 smithi164 ceph-mon[32713]: pgmap v341: 97 pgs: 97 active+clean; 5.1 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-04-25T10:43:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.398+0000 7f9d0b0ee640 1 -- 172.21.15.43:0/1400734217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d04109fb0 msgr2=0x7f9d04102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.398+0000 7f9d0b0ee640 1 --2- 172.21.15.43:0/1400734217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d04109fb0 0x7f9d04102140 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009900 tx=0x7f9cf802a410 comp rx=0 tx=0).stop 2024-04-25T10:43:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.398+0000 7f9d0b0ee640 1 -- 172.21.15.43:0/1400734217 shutdown_connections 2024-04-25T10:43:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.398+0000 7f9d0b0ee640 1 --2- 172.21.15.43:0/1400734217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d041027b0 0x7f9d04110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.398+0000 7f9d0b0ee640 1 --2- 172.21.15.43:0/1400734217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d04109fb0 0x7f9d04102140 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.398+0000 7f9d0b0ee640 1 --2- 172.21.15.43:0/1400734217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d04109b10 0x7f9d04101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.398+0000 7f9d0b0ee640 1 -- 172.21.15.43:0/1400734217 >> 172.21.15.43:0/1400734217 conn(0x7f9d040fd850 msgr2=0x7f9d040ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.398+0000 7f9d0b0ee640 1 -- 172.21.15.43:0/1400734217 shutdown_connections 2024-04-25T10:43:28.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.398+0000 7f9d0b0ee640 1 -- 172.21.15.43:0/1400734217 wait complete. 2024-04-25T10:43:28.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.398+0000 7f9d0b0ee640 1 Processor -- start 2024-04-25T10:43:28.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.398+0000 7f9d0b0ee640 1 -- start start 2024-04-25T10:43:28.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d0b0ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d041027b0 0x7f9d0419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d0b0ee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d04109b10 0x7f9d0419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d0b0ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d04109fb0 0x7f9d0419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d0b0ee640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9d04113d00 con 0x7f9d041027b0 2024-04-25T10:43:28.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d0b0ee640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9d04113b80 con 0x7f9d04109fb0 2024-04-25T10:43:28.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d0b0ee640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9d04113e80 con 0x7f9d04109b10 2024-04-25T10:43:28.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d08e8a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d041027b0 0x7f9d0419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d08e8a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d041027b0 0x7f9d0419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44540/0 (socket says 172.21.15.43:44540) 2024-04-25T10:43:28.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d08e8a640 1 -- 172.21.15.43:0/704270300 learned_addr learned my addr 172.21.15.43:0/704270300 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d0968b640 1 --2- 172.21.15.43:0/704270300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d04109fb0 0x7f9d0419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d03fff640 1 --2- 172.21.15.43:0/704270300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d04109b10 0x7f9d0419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d08e8a640 1 -- 172.21.15.43:0/704270300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d04109b10 msgr2=0x7f9d0419acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d08e8a640 1 --2- 172.21.15.43:0/704270300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d04109b10 0x7f9d0419acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d08e8a640 1 -- 172.21.15.43:0/704270300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d04109fb0 msgr2=0x7f9d0419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d08e8a640 1 --2- 172.21.15.43:0/704270300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d04109fb0 0x7f9d0419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d08e8a640 1 -- 172.21.15.43:0/704270300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d0419f5c0 con 0x7f9d041027b0 2024-04-25T10:43:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d03fff640 1 --2- 172.21.15.43:0/704270300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d04109b10 0x7f9d0419acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:43:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d08e8a640 1 --2- 172.21.15.43:0/704270300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d041027b0 0x7f9d0419a760 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400d950 tx=0x7f9cf400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d01ffb640 1 -- 172.21.15.43:0/704270300 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4017030 con 0x7f9d041027b0 2024-04-25T10:43:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d0b0ee640 1 -- 172.21.15.43:0/704270300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d0419f8b0 con 0x7f9d041027b0 2024-04-25T10:43:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d01ffb640 1 -- 172.21.15.43:0/704270300 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cf4004020 con 0x7f9d041027b0 2024-04-25T10:43:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.399+0000 7f9d01ffb640 1 -- 172.21.15.43:0/704270300 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf40047a0 con 0x7f9d041027b0 2024-04-25T10:43:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.400+0000 7f9d0b0ee640 1 -- 172.21.15.43:0/704270300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d041a7230 con 0x7f9d041027b0 2024-04-25T10:43:28.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.402+0000 7f9d11fa0740 1 Processor -- start 2024-04-25T10:43:28.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.402+0000 7f9d11fa0740 1 -- start start 2024-04-25T10:43:28.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.403+0000 7f9d01ffb640 1 -- 172.21.15.43:0/704270300 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f9cf4009a30 con 0x7f9d041027b0 2024-04-25T10:43:28.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.403+0000 7f9d11fa0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 0x55de482a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.403+0000 7f9d11fa0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de47ed64d0 0x55de482a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.403+0000 7f9d11fa0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ede6b0 0x55de482b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.403+0000 7f9d11fa0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55de48205d80 con 0x55de47eff390 2024-04-25T10:43:28.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.403+0000 7f9d11fa0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55de481bfa20 con 0x55de47ede6b0 2024-04-25T10:43:28.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.403+0000 7f9d11fa0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55de48201d40 con 0x55de47ed64d0 2024-04-25T10:43:28.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.403+0000 7f9d0968b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 0x55de482a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.403+0000 7f9d0968b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 0x55de482a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44542/0 (socket says 172.21.15.43:44542) 2024-04-25T10:43:28.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.403+0000 7f9d0968b640 1 -- 172.21.15.43:0/802754884 learned_addr learned my addr 172.21.15.43:0/802754884 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:28.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.403+0000 7f9d08e8a640 1 --2- 172.21.15.43:0/802754884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de47ed64d0 0x55de482a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.403+0000 7f9d03fff640 1 --2- 172.21.15.43:0/802754884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ede6b0 0x55de482b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.403+0000 7f9d0968b640 1 -- 172.21.15.43:0/802754884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de47ed64d0 msgr2=0x55de482a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.403+0000 7f9d0968b640 1 --2- 172.21.15.43:0/802754884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de47ed64d0 0x55de482a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.403+0000 7f9d0968b640 1 -- 172.21.15.43:0/802754884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ede6b0 msgr2=0x55de482b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.403+0000 7f9d0968b640 1 --2- 172.21.15.43:0/802754884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ede6b0 0x55de482b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.403+0000 7f9d0968b640 1 -- 172.21.15.43:0/802754884 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de481cac30 con 0x55de47eff390 2024-04-25T10:43:28.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.403+0000 7f9d01ffb640 1 --2- 172.21.15.43:0/704270300 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9cec0afc60 0x7f9cec0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.403+0000 7f9d0968b640 1 --2- 172.21.15.43:0/802754884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 0x55de482a06a0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009ee0 tx=0x7f9cfc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e20d283e138c37d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.403+0000 7f9d08e8a640 1 --2- 172.21.15.43:0/704270300 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9cec0afc60 0x7f9cec0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.404+0000 7f9ceaffd640 1 -- 172.21.15.43:0/802754884 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc016070 con 0x55de47eff390 2024-04-25T10:43:28.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.404+0000 7f9d01ffb640 1 -- 172.21.15.43:0/704270300 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f9cf40deab0 con 0x7f9d041027b0 2024-04-25T10:43:28.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.404+0000 7f9ceaffd640 1 -- 172.21.15.43:0/802754884 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cfc004d20 con 0x55de47eff390 2024-04-25T10:43:28.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.404+0000 7f9ceaffd640 1 -- 172.21.15.43:0/802754884 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc012720 con 0x55de47eff390 2024-04-25T10:43:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.404+0000 7f9d11fa0740 1 -- 172.21.15.43:0/802754884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 msgr2=0x55de482a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.404+0000 7f9d08e8a640 1 --2- 172.21.15.43:0/704270300 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9cec0afc60 0x7f9cec0b2170 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9cf40dc2b0 tx=0x7f9cf40e2110 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.404+0000 7f9d11fa0740 1 --2- 172.21.15.43:0/802754884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 0x55de482a06a0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009ee0 tx=0x7f9cfc010740 comp rx=0 tx=0).stop 2024-04-25T10:43:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.404+0000 7f9d11fa0740 1 -- 172.21.15.43:0/802754884 shutdown_connections 2024-04-25T10:43:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.404+0000 7f9d11fa0740 1 --2- 172.21.15.43:0/802754884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ede6b0 0x55de482b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.404+0000 7f9d11fa0740 1 --2- 172.21.15.43:0/802754884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de47ed64d0 0x55de482a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.405+0000 7f9d11fa0740 1 --2- 172.21.15.43:0/802754884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 0x55de482a06a0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.405+0000 7f9d11fa0740 1 -- 172.21.15.43:0/802754884 >> 172.21.15.43:0/802754884 conn(0x55de47ef34e0 msgr2=0x55de4829cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.405+0000 7f9d11fa0740 1 -- 172.21.15.43:0/802754884 shutdown_connections 2024-04-25T10:43:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.405+0000 7f9d11fa0740 1 -- 172.21.15.43:0/802754884 wait complete. 2024-04-25T10:43:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.405+0000 7f9d11fa0740 1 -- start start 2024-04-25T10:43:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.405+0000 7f9d11fa0740 1 Processor -- start 2024-04-25T10:43:28.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.405+0000 7f9d11fa0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de47ed64d0 0x55de482af660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.405+0000 7f9d11fa0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ede6b0 0x55de482afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.405+0000 7f9d11fa0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 0x55de482bb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9d11fa0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55de482b21a0 con 0x55de47eff390 2024-04-25T10:43:28.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9d11fa0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55de4818cdc0 con 0x55de47ede6b0 2024-04-25T10:43:28.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9d11fa0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55de482b2320 con 0x55de47ed64d0 2024-04-25T10:43:28.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9d11fa0740 10 client.0 _ll_register_callbacks cb 0x55de481a8600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:28.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9d0968b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 0x55de482bb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9d0968b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 0x55de482bb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44556/0 (socket says 172.21.15.43:44556) 2024-04-25T10:43:28.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9d0968b640 1 -- 172.21.15.43:0/27597750 learned_addr learned my addr 172.21.15.43:0/27597750 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:28.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9d0968b640 1 --2- 172.21.15.43:0/27597750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de47ed64d0 0x55de482af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9d03fff640 1 --2- 172.21.15.43:0/27597750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ede6b0 0x55de482afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9d0968b640 1 -- 172.21.15.43:0/27597750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de47ed64d0 msgr2=0x55de482af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9d0968b640 1 --2- 172.21.15.43:0/27597750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de47ed64d0 0x55de482af660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9d0968b640 1 -- 172.21.15.43:0/27597750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ede6b0 msgr2=0x55de482afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9d0968b640 1 --2- 172.21.15.43:0/27597750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ede6b0 0x55de482afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9d0968b640 1 -- 172.21.15.43:0/27597750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de47f173f0 con 0x55de47eff390 2024-04-25T10:43:28.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9d03fff640 1 --2- 172.21.15.43:0/27597750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ede6b0 0x55de482afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:43:28.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9d0968b640 1 --2- 172.21.15.43:0/27597750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 0x55de482bb6e0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x55de482aecb0 tx=0x7f9cfc0059e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a9852ed82b07a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9ce3fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:28.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9ce3fff640 1 -- 172.21.15.43:0/27597750 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc01e030 con 0x55de47eff390 2024-04-25T10:43:28.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9ce3fff640 1 -- 172.21.15.43:0/27597750 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f9cfc012810 con 0x55de47eff390 2024-04-25T10:43:28.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9d11fa0740 1 -- 172.21.15.43:0/27597750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cfc004d20 con 0x55de47eff390 2024-04-25T10:43:28.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9ce3fff640 1 -- 172.21.15.43:0/27597750 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9cfc016040 con 0x55de47eff390 2024-04-25T10:43:28.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9d11fa0740 10 client.18726 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:28.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.406+0000 7f9d11fa0740 1 -- 172.21.15.43:0/27597750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55de47fe17d0 con 0x55de47eff390 2024-04-25T10:43:28.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.407+0000 7f9ce3fff640 1 -- 172.21.15.43:0/27597750 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cfc004d20 con 0x55de47eff390 2024-04-25T10:43:28.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.407+0000 7f9ce3fff640 1 -- 172.21.15.43:0/27597750 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9cfc00f040 con 0x55de47eff390 2024-04-25T10:43:28.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.407+0000 7f9d11fa0740 10 client.18726 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:28.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.407+0000 7f9d11fa0740 10 client.18726 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:43:28.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.407+0000 7f9d11fa0740 20 client.18726 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:28.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.407+0000 7f9d11fa0740 1 --2- 172.21.15.43:0/27597750 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55de47ebdf60 0x55de482c19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.407+0000 7f9d11fa0740 4 client.18726 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:43:28.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.407+0000 7f9d11fa0740 1 -- 172.21.15.43:0/27597750 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de48213990 con 0x55de47ebdf60 2024-04-25T10:43:28.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.407+0000 7f9d03fff640 1 --2- 172.21.15.43:0/27597750 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55de47ebdf60 0x55de482c19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.408+0000 7f9ce3fff640 10 client.18726 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:43:28.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.408+0000 7f9d03fff640 1 --2- 172.21.15.43:0/27597750 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55de47ebdf60 0x55de482c19b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e71f6d48e405b150 server_cookie=654e35e0ce2f248d in_seq=0 out_seq=0 2024-04-25T10:43:28.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.408+0000 7f9ce3fff640 1 -- 172.21.15.43:0/27597750 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55de48213990 con 0x55de47ebdf60 2024-04-25T10:43:28.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.408+0000 7f9ce3fff640 10 client.18726 handle_command_reply: tid=0 2024-04-25T10:43:28.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.408+0000 7f9d11fa0740 1 client.18726 shutdown 2024-04-25T10:43:28.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.409+0000 7f9d11fa0740 1 -- 172.21.15.43:0/27597750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 msgr2=0x55de482bb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.409+0000 7f9d11fa0740 1 --2- 172.21.15.43:0/27597750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 0x55de482bb6e0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x55de482aecb0 tx=0x7f9cfc0059e0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.409+0000 7f9d11fa0740 1 -- 172.21.15.43:0/27597750 shutdown_connections 2024-04-25T10:43:28.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.409+0000 7f9d11fa0740 1 --2- 172.21.15.43:0/27597750 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55de47ebdf60 0x55de482c19b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.409+0000 7f9d11fa0740 1 --2- 172.21.15.43:0/27597750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ede6b0 0x55de482afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.409+0000 7f9ce3fff640 0 client.18726 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:43:28.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.409+0000 7f9d11fa0740 1 --2- 172.21.15.43:0/27597750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 0x55de482bb6e0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.409+0000 7f9d11fa0740 1 --2- 172.21.15.43:0/27597750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de47ed64d0 0x55de482af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.409+0000 7f9d11fa0740 1 -- 172.21.15.43:0/27597750 >> 172.21.15.43:0/27597750 conn(0x55de47ef34e0 msgr2=0x55de482a2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.409+0000 7f9d11fa0740 1 -- 172.21.15.43:0/27597750 shutdown_connections 2024-04-25T10:43:28.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.409+0000 7f9d11fa0740 1 -- 172.21.15.43:0/27597750 wait complete. 2024-04-25T10:43:28.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.409+0000 7f9d11fa0740 20 client.18726 trim_cache size 0 max 16384 2024-04-25T10:43:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.423+0000 7f9d11fa0740 1 Processor -- start 2024-04-25T10:43:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.423+0000 7f9d11fa0740 1 -- start start 2024-04-25T10:43:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.423+0000 7f9d11fa0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ed64d0 0x55de482ae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.423+0000 7f9d11fa0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47ede6b0 0x55de482bb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.423+0000 7f9d11fa0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de47eff390 0x55de482c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.423+0000 7f9d11fa0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9d041b01b0 con 0x55de47ede6b0 2024-04-25T10:43:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.423+0000 7f9d11fa0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9d041ada80 con 0x55de47ed64d0 2024-04-25T10:43:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.423+0000 7f9d11fa0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9d041ab910 con 0x55de47eff390 2024-04-25T10:43:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.423+0000 7f9d03fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47ede6b0 0x55de482bb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.423+0000 7f9d03fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47ede6b0 0x55de482bb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44570/0 (socket says 172.21.15.43:44570) 2024-04-25T10:43:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.423+0000 7f9d03fff640 1 -- 172.21.15.43:0/5543454 learned_addr learned my addr 172.21.15.43:0/5543454 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.423+0000 7f9d0968b640 1 --2- 172.21.15.43:0/5543454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ed64d0 0x55de482ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.423+0000 7f9d03fff640 1 -- 172.21.15.43:0/5543454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de47eff390 msgr2=0x55de482c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.423+0000 7f9d03fff640 1 --2- 172.21.15.43:0/5543454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de47eff390 0x55de482c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.423+0000 7f9d03fff640 1 -- 172.21.15.43:0/5543454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ed64d0 msgr2=0x55de482ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.423+0000 7f9d03fff640 1 --2- 172.21.15.43:0/5543454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ed64d0 0x55de482ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.423+0000 7f9d03fff640 1 -- 172.21.15.43:0/5543454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de482af470 con 0x55de47ede6b0 2024-04-25T10:43:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.424+0000 7f9d03fff640 1 --2- 172.21.15.43:0/5543454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47ede6b0 0x55de482bb650 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80092a0 tx=0x7f9cf8004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c147ebe3ac99407 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.424+0000 7f9cea7fc640 1 -- 172.21.15.43:0/5543454 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8035040 con 0x55de47ede6b0 2024-04-25T10:43:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.424+0000 7f9cea7fc640 1 -- 172.21.15.43:0/5543454 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cf8004400 con 0x55de47ede6b0 2024-04-25T10:43:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.424+0000 7f9cea7fc640 1 -- 172.21.15.43:0/5543454 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8004b80 con 0x55de47ede6b0 2024-04-25T10:43:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.425+0000 7f9d11fa0740 1 -- 172.21.15.43:0/5543454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47ede6b0 msgr2=0x55de482bb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.425+0000 7f9d11fa0740 1 --2- 172.21.15.43:0/5543454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47ede6b0 0x55de482bb650 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80092a0 tx=0x7f9cf8004310 comp rx=0 tx=0).stop 2024-04-25T10:43:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.425+0000 7f9d11fa0740 1 -- 172.21.15.43:0/5543454 shutdown_connections 2024-04-25T10:43:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.425+0000 7f9d11fa0740 1 --2- 172.21.15.43:0/5543454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de47eff390 0x55de482c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.425+0000 7f9d11fa0740 1 --2- 172.21.15.43:0/5543454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47ede6b0 0x55de482bb650 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.425+0000 7f9d11fa0740 1 --2- 172.21.15.43:0/5543454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ed64d0 0x55de482ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.425+0000 7f9d11fa0740 1 -- 172.21.15.43:0/5543454 >> 172.21.15.43:0/5543454 conn(0x55de47ef34e0 msgr2=0x55de4829ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.425+0000 7f9d11fa0740 1 -- 172.21.15.43:0/5543454 shutdown_connections 2024-04-25T10:43:28.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.425+0000 7f9d11fa0740 1 -- 172.21.15.43:0/5543454 wait complete. 2024-04-25T10:43:28.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.425+0000 7f9d11fa0740 1 -- start start 2024-04-25T10:43:28.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.425+0000 7f9d11fa0740 1 Processor -- start 2024-04-25T10:43:28.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9d11fa0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de47ed64d0 0x55de482c2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9d11fa0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ede6b0 0x55de482c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9d11fa0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 0x55de482c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9d11fa0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9d041abf60 con 0x55de47eff390 2024-04-25T10:43:28.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9d11fa0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9d041ad3b0 con 0x55de47ede6b0 2024-04-25T10:43:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9d11fa0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9d041ac610 con 0x55de47ed64d0 2024-04-25T10:43:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9d11fa0740 10 client.0 _ll_register_callbacks cb 0x7f9cfc009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9d0968b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 0x55de482c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9d0968b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 0x55de482c30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44574/0 (socket says 172.21.15.43:44574) 2024-04-25T10:43:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9d0968b640 1 -- 172.21.15.43:0/3843330285 learned_addr learned my addr 172.21.15.43:0/3843330285 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9d03fff640 1 --2- 172.21.15.43:0/3843330285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ede6b0 0x55de482c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9d0968b640 1 --2- 172.21.15.43:0/3843330285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de47ed64d0 0x55de482c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9d0968b640 1 -- 172.21.15.43:0/3843330285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de47ed64d0 msgr2=0x55de482c2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9d0968b640 1 --2- 172.21.15.43:0/3843330285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de47ed64d0 0x55de482c2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9d0968b640 1 -- 172.21.15.43:0/3843330285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ede6b0 msgr2=0x55de482c2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9d0968b640 1 --2- 172.21.15.43:0/3843330285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ede6b0 0x55de482c2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9d0968b640 1 -- 172.21.15.43:0/3843330285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de48328ae0 con 0x55de47eff390 2024-04-25T10:43:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9d03fff640 1 --2- 172.21.15.43:0/3843330285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ede6b0 0x55de482c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:43:28.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9d0968b640 1 --2- 172.21.15.43:0/3843330285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 0x55de482c30b0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009700 tx=0x7f9cfc0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4127ac89a2c385ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9ce3fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:28.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.426+0000 7f9ce3fff640 1 -- 172.21.15.43:0/3843330285 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc038060 con 0x55de47eff390 2024-04-25T10:43:28.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.427+0000 7f9ce3fff640 1 -- 172.21.15.43:0/3843330285 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f9cfc025040 con 0x55de47eff390 2024-04-25T10:43:28.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.427+0000 7f9d11fa0740 1 -- 172.21.15.43:0/3843330285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cf8004400 con 0x55de47eff390 2024-04-25T10:43:28.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.427+0000 7f9ce3fff640 1 -- 172.21.15.43:0/3843330285 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9cfc01e070 con 0x55de47eff390 2024-04-25T10:43:28.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.427+0000 7f9d11fa0740 10 client.18732 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:28.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.427+0000 7f9d11fa0740 1 -- 172.21.15.43:0/3843330285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55de48329d20 con 0x55de47eff390 2024-04-25T10:43:28.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.427+0000 7f9ce3fff640 1 -- 172.21.15.43:0/3843330285 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cf8004400 con 0x55de47eff390 2024-04-25T10:43:28.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.427+0000 7f9ce3fff640 1 -- 172.21.15.43:0/3843330285 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9cfc02baf0 con 0x55de47eff390 2024-04-25T10:43:28.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.427+0000 7f9d11fa0740 10 client.18732 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:28.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.427+0000 7f9d11fa0740 10 client.18732 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:43:28.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.427+0000 7f9d11fa0740 20 client.18732 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:28.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.427+0000 7f9d11fa0740 1 --2- 172.21.15.43:0/3843330285 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55de47e933a0 0x55de4832c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.427+0000 7f9d11fa0740 4 client.18732 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:43:28.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.427+0000 7f9d11fa0740 1 -- 172.21.15.43:0/3843330285 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de4820d800 con 0x55de47e933a0 2024-04-25T10:43:28.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.428+0000 7f9d03fff640 1 --2- 172.21.15.43:0/3843330285 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55de47e933a0 0x55de4832c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.428+0000 7f9d03fff640 1 --2- 172.21.15.43:0/3843330285 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55de47e933a0 0x55de4832c3f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ffcebd75bcb9fea4 server_cookie=9d51386f095d89ff in_seq=0 out_seq=0 2024-04-25T10:43:28.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.428+0000 7f9ce3fff640 10 client.18732 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:43:28.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.428+0000 7f9ce3fff640 1 -- 172.21.15.43:0/3843330285 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de4820d800 con 0x55de47e933a0 2024-04-25T10:43:28.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.428+0000 7f9ce3fff640 10 client.18732 handle_command_reply: tid=0 2024-04-25T10:43:28.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.428+0000 7f9d11fa0740 1 client.18732 shutdown 2024-04-25T10:43:28.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.428+0000 7f9d11fa0740 1 -- 172.21.15.43:0/3843330285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 msgr2=0x55de482c30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.428+0000 7f9d11fa0740 1 --2- 172.21.15.43:0/3843330285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 0x55de482c30b0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009700 tx=0x7f9cfc0058d0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.428+0000 7f9d11fa0740 1 -- 172.21.15.43:0/3843330285 shutdown_connections 2024-04-25T10:43:28.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.428+0000 7f9d11fa0740 1 --2- 172.21.15.43:0/3843330285 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55de47e933a0 0x55de4832c3f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.428+0000 7f9d11fa0740 1 --2- 172.21.15.43:0/3843330285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de47ede6b0 0x55de482c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.428+0000 7f9ce3fff640 0 client.18732 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:43:28.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.428+0000 7f9d11fa0740 1 --2- 172.21.15.43:0/3843330285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de47eff390 0x55de482c30b0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.428+0000 7f9d11fa0740 1 --2- 172.21.15.43:0/3843330285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de47ed64d0 0x55de482c2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.428+0000 7f9d11fa0740 1 -- 172.21.15.43:0/3843330285 >> 172.21.15.43:0/3843330285 conn(0x55de47ef34e0 msgr2=0x55de482a23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.429+0000 7f9d11fa0740 1 -- 172.21.15.43:0/3843330285 shutdown_connections 2024-04-25T10:43:28.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.429+0000 7f9d11fa0740 1 -- 172.21.15.43:0/3843330285 wait complete. 2024-04-25T10:43:28.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.429+0000 7f9d11fa0740 20 client.18732 trim_cache size 0 max 16384 2024-04-25T10:43:28.438 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:43:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.429+0000 7f9d0b0ee640 1 -- 172.21.15.43:0/704270300 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9cec0afc60 msgr2=0x7f9cec0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.429+0000 7f9d0b0ee640 1 --2- 172.21.15.43:0/704270300 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9cec0afc60 0x7f9cec0b2170 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9cf40dc2b0 tx=0x7f9cf40e2110 comp rx=0 tx=0).stop 2024-04-25T10:43:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.429+0000 7f9d0b0ee640 1 -- 172.21.15.43:0/704270300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d041027b0 msgr2=0x7f9d0419a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.429+0000 7f9d0b0ee640 1 --2- 172.21.15.43:0/704270300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d041027b0 0x7f9d0419a760 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400d950 tx=0x7f9cf400de10 comp rx=0 tx=0).stop 2024-04-25T10:43:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.429+0000 7f9d0b0ee640 1 -- 172.21.15.43:0/704270300 shutdown_connections 2024-04-25T10:43:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.429+0000 7f9d0b0ee640 1 --2- 172.21.15.43:0/704270300 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9cec0afc60 0x7f9cec0b2170 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.429+0000 7f9d0b0ee640 1 --2- 172.21.15.43:0/704270300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d04109fb0 0x7f9d0419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.429+0000 7f9d0b0ee640 1 --2- 172.21.15.43:0/704270300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d04109b10 0x7f9d0419acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.429+0000 7f9d0b0ee640 1 --2- 172.21.15.43:0/704270300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d041027b0 0x7f9d0419a760 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.429+0000 7f9d0b0ee640 1 -- 172.21.15.43:0/704270300 >> 172.21.15.43:0/704270300 conn(0x7f9d040fd850 msgr2=0x7f9d040ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.429+0000 7f9d0b0ee640 1 -- 172.21.15.43:0/704270300 shutdown_connections 2024-04-25T10:43:28.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.429+0000 7f9d0b0ee640 1 -- 172.21.15.43:0/704270300 wait complete. 2024-04-25T10:43:28.442 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:43:28.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.557+0000 7f743770d640 1 -- 172.21.15.43:0/205149924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7430109b20 msgr2=0x7f7430101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.557+0000 7f743770d640 1 --2- 172.21.15.43:0/205149924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7430109b20 0x7f7430101bf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f742c009860 tx=0x7f742c02a340 comp rx=0 tx=0).stop 2024-04-25T10:43:28.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.558+0000 7f743770d640 1 -- 172.21.15.43:0/205149924 shutdown_connections 2024-04-25T10:43:28.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.558+0000 7f743770d640 1 --2- 172.21.15.43:0/205149924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74301027a0 0x7f7430110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.558+0000 7f743770d640 1 --2- 172.21.15.43:0/205149924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7430109fc0 0x7f7430102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.558+0000 7f743770d640 1 --2- 172.21.15.43:0/205149924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7430109b20 0x7f7430101bf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.558+0000 7f743770d640 1 -- 172.21.15.43:0/205149924 >> 172.21.15.43:0/205149924 conn(0x7f7430077880 msgr2=0x7f7430077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.558+0000 7f743770d640 1 -- 172.21.15.43:0/205149924 shutdown_connections 2024-04-25T10:43:28.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.558+0000 7f743770d640 1 -- 172.21.15.43:0/205149924 wait complete. 2024-04-25T10:43:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.559+0000 7f743770d640 1 Processor -- start 2024-04-25T10:43:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.559+0000 7f743770d640 1 -- start start 2024-04-25T10:43:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.559+0000 7f743770d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74301027a0 0x7f743019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.559+0000 7f743770d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7430109b20 0x7f743019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.559+0000 7f743770d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7430109fc0 0x7f743019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.559+0000 7f743770d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7430113d10 con 0x7f74301027a0 2024-04-25T10:43:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.559+0000 7f743770d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7430113b90 con 0x7f7430109fc0 2024-04-25T10:43:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.559+0000 7f743770d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7430113e90 con 0x7f7430109b20 2024-04-25T10:43:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.559+0000 7f7435caa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7430109fc0 0x7f743019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.559+0000 7f74354a9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74301027a0 0x7f743019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.559+0000 7f7435caa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7430109fc0 0x7f743019efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.43:46670/0 (socket says 172.21.15.43:46670) 2024-04-25T10:43:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.559+0000 7f7435caa640 1 -- 172.21.15.43:0/3579213707 learned_addr learned my addr 172.21.15.43:0/3579213707 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.560+0000 7f7434ca8640 1 --2- 172.21.15.43:0/3579213707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7430109b20 0x7f743019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.560+0000 7f7434ca8640 1 -- 172.21.15.43:0/3579213707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7430109fc0 msgr2=0x7f743019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.560+0000 7f7434ca8640 1 --2- 172.21.15.43:0/3579213707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7430109fc0 0x7f743019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.560+0000 7f7434ca8640 1 -- 172.21.15.43:0/3579213707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74301027a0 msgr2=0x7f743019a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.560+0000 7f7434ca8640 1 --2- 172.21.15.43:0/3579213707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74301027a0 0x7f743019a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.560+0000 7f7434ca8640 1 -- 172.21.15.43:0/3579213707 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743019f610 con 0x7f7430109b20 2024-04-25T10:43:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.560+0000 7f74354a9640 1 --2- 172.21.15.43:0/3579213707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74301027a0 0x7f743019a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:43:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.560+0000 7f7435caa640 1 --2- 172.21.15.43:0/3579213707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7430109fc0 0x7f743019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:43:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.560+0000 7f7434ca8640 1 --2- 172.21.15.43:0/3579213707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7430109b20 0x7f743019ad10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f74300754b0 tx=0x7f742000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.560+0000 7f74267fc640 1 -- 172.21.15.43:0/3579213707 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742000cc40 con 0x7f7430109b20 2024-04-25T10:43:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.560+0000 7f743770d640 1 -- 172.21.15.43:0/3579213707 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743019f900 con 0x7f7430109b20 2024-04-25T10:43:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.561+0000 7f74267fc640 1 -- 172.21.15.43:0/3579213707 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f742000cde0 con 0x7f7430109b20 2024-04-25T10:43:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.561+0000 7f74267fc640 1 -- 172.21.15.43:0/3579213707 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420010430 con 0x7f7430109b20 2024-04-25T10:43:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.561+0000 7f743770d640 1 -- 172.21.15.43:0/3579213707 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7430102d00 con 0x7f7430109b20 2024-04-25T10:43:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.563+0000 7f74267fc640 1 -- 172.21.15.43:0/3579213707 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f7420003680 con 0x7f7430109b20 2024-04-25T10:43:28.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.564+0000 7f74267fc640 1 --2- 172.21.15.43:0/3579213707 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f741c0afc60 0x7f741c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.564+0000 7f74267fc640 1 -- 172.21.15.43:0/3579213707 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f74200df350 con 0x7f7430109b20 2024-04-25T10:43:28.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.565+0000 7f74354a9640 1 --2- 172.21.15.43:0/3579213707 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f741c0afc60 0x7f741c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.565+0000 7f743e5be740 1 Processor -- start 2024-04-25T10:43:28.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.565+0000 7f74354a9640 1 --2- 172.21.15.43:0/3579213707 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f741c0afc60 0x7f741c0b2170 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f742c0303f0 tx=0x7f742c030340 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.565+0000 7f743e5be740 1 -- start start 2024-04-25T10:43:28.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.566+0000 7f743e5be740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b5293390 0x55d1b563d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.566+0000 7f743e5be740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 0x55d1b563d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.566+0000 7f743e5be740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b52726b0 0x55d1b5649eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.566+0000 7f743e5be740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d1b5599d80 con 0x55d1b526a4d0 2024-04-25T10:43:28.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.566+0000 7f743e5be740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d1b5553a20 con 0x55d1b5293390 2024-04-25T10:43:28.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.566+0000 7f743e5be740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d1b5595d40 con 0x55d1b52726b0 2024-04-25T10:43:28.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.566+0000 7f7434ca8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 0x55d1b563d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.566+0000 7f74354a9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b5293390 0x55d1b563d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.566+0000 7f7434ca8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 0x55d1b563d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44590/0 (socket says 172.21.15.43:44590) 2024-04-25T10:43:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.566+0000 7f7434ca8640 1 -- 172.21.15.43:0/2159440760 learned_addr learned my addr 172.21.15.43:0/2159440760 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.566+0000 7f7435caa640 1 --2- 172.21.15.43:0/2159440760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b52726b0 0x55d1b5649eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.566+0000 7f7434ca8640 1 -- 172.21.15.43:0/2159440760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b52726b0 msgr2=0x55d1b5649eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.566+0000 7f7434ca8640 1 --2- 172.21.15.43:0/2159440760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b52726b0 0x55d1b5649eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.566+0000 7f7434ca8640 1 -- 172.21.15.43:0/2159440760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b5293390 msgr2=0x55d1b563d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.566+0000 7f7434ca8640 1 --2- 172.21.15.43:0/2159440760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b5293390 0x55d1b563d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.566+0000 7f7434ca8640 1 -- 172.21.15.43:0/2159440760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b555ec30 con 0x55d1b526a4d0 2024-04-25T10:43:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.567+0000 7f7434ca8640 1 --2- 172.21.15.43:0/2159440760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 0x55d1b563d560 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f74200dc9e0 tx=0x7f742000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a102b9066f7a4014 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.567+0000 7f741b7fe640 1 -- 172.21.15.43:0/2159440760 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74200a7490 con 0x55d1b526a4d0 2024-04-25T10:43:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.567+0000 7f741b7fe640 1 -- 172.21.15.43:0/2159440760 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74200ab840 con 0x55d1b526a4d0 2024-04-25T10:43:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.567+0000 7f741b7fe640 1 -- 172.21.15.43:0/2159440760 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74200aee20 con 0x55d1b526a4d0 2024-04-25T10:43:28.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.568+0000 7f743e5be740 1 -- 172.21.15.43:0/2159440760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 msgr2=0x55d1b563d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.568+0000 7f743e5be740 1 --2- 172.21.15.43:0/2159440760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 0x55d1b563d560 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f74200dc9e0 tx=0x7f742000cf90 comp rx=0 tx=0).stop 2024-04-25T10:43:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.568+0000 7f743e5be740 1 -- 172.21.15.43:0/2159440760 shutdown_connections 2024-04-25T10:43:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.568+0000 7f743e5be740 1 --2- 172.21.15.43:0/2159440760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b52726b0 0x55d1b5649eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.568+0000 7f743e5be740 1 --2- 172.21.15.43:0/2159440760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 0x55d1b563d560 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.568+0000 7f743e5be740 1 --2- 172.21.15.43:0/2159440760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b5293390 0x55d1b563d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.568+0000 7f743e5be740 1 -- 172.21.15.43:0/2159440760 >> 172.21.15.43:0/2159440760 conn(0x55d1b52874e0 msgr2=0x55d1b56397d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.568+0000 7f743e5be740 1 -- 172.21.15.43:0/2159440760 shutdown_connections 2024-04-25T10:43:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.568+0000 7f743e5be740 1 -- 172.21.15.43:0/2159440760 wait complete. 2024-04-25T10:43:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.568+0000 7f743e5be740 1 -- start start 2024-04-25T10:43:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.568+0000 7f743e5be740 1 Processor -- start 2024-04-25T10:43:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.569+0000 7f743e5be740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 0x55d1b5647a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.569+0000 7f743e5be740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b52726b0 0x55d1b5647f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.569+0000 7f743e5be740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b5293390 0x55d1b5653ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.569+0000 7f743e5be740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d1b564a5a0 con 0x55d1b526a4d0 2024-04-25T10:43:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.569+0000 7f743e5be740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d1b5520dc0 con 0x55d1b52726b0 2024-04-25T10:43:28.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.569+0000 7f743e5be740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d1b564a720 con 0x55d1b5293390 2024-04-25T10:43:28.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.569+0000 7f743e5be740 10 client.0 _ll_register_callbacks cb 0x55d1b553c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:28.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.569+0000 7f74354a9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 0x55d1b5647a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.569+0000 7f74354a9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 0x55d1b5647a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44594/0 (socket says 172.21.15.43:44594) 2024-04-25T10:43:28.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.569+0000 7f74354a9640 1 -- 172.21.15.43:0/2838269196 learned_addr learned my addr 172.21.15.43:0/2838269196 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:28.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.569+0000 7f7434ca8640 1 --2- 172.21.15.43:0/2838269196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b52726b0 0x55d1b5647f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.569+0000 7f7435caa640 1 --2- 172.21.15.43:0/2838269196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b5293390 0x55d1b5653ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.570+0000 7f7434ca8640 1 -- 172.21.15.43:0/2838269196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b5293390 msgr2=0x55d1b5653ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.570+0000 7f7434ca8640 1 --2- 172.21.15.43:0/2838269196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b5293390 0x55d1b5653ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.570+0000 7f7434ca8640 1 -- 172.21.15.43:0/2838269196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 msgr2=0x55d1b5647a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.570+0000 7f7434ca8640 1 --2- 172.21.15.43:0/2838269196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 0x55d1b5647a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.570+0000 7f7434ca8640 1 -- 172.21.15.43:0/2838269196 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1b52ab3f0 con 0x55d1b52726b0 2024-04-25T10:43:28.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.570+0000 7f74354a9640 1 --2- 172.21.15.43:0/2838269196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 0x55d1b5647a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:43:28.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.570+0000 7f7434ca8640 1 --2- 172.21.15.43:0/2838269196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b52726b0 0x55d1b5647f80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f74200a7420 tx=0x7f7420005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad304b9a46a8675a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.570+0000 7f7418ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-04-25T10:43:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.570+0000 7f7418ff9640 1 -- 172.21.15.43:0/2838269196 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74200c5070 con 0x55d1b52726b0 2024-04-25T10:43:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.570+0000 7f7418ff9640 1 -- 172.21.15.43:0/2838269196 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f74200a6900 con 0x55d1b52726b0 2024-04-25T10:43:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.570+0000 7f743e5be740 1 -- 172.21.15.43:0/2838269196 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74200ab840 con 0x55d1b52726b0 2024-04-25T10:43:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.570+0000 7f7418ff9640 1 -- 172.21.15.43:0/2838269196 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7420087050 con 0x55d1b52726b0 2024-04-25T10:43:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.570+0000 7f743e5be740 10 client.27893 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.570+0000 7f743e5be740 1 -- 172.21.15.43:0/2838269196 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d1b53757d0 con 0x55d1b52726b0 2024-04-25T10:43:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.570+0000 7f7418ff9640 1 -- 172.21.15.43:0/2838269196 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74200ab840 con 0x55d1b52726b0 2024-04-25T10:43:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.571+0000 7f7418ff9640 1 -- 172.21.15.43:0/2838269196 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f74200c8030 con 0x55d1b52726b0 2024-04-25T10:43:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.571+0000 7f743e5be740 10 client.27893 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.571+0000 7f743e5be740 10 client.27893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:43:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.571+0000 7f743e5be740 20 client.27893 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.571+0000 7f743e5be740 1 --2- 172.21.15.43:0/2838269196 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d1b5251f60 0x55d1b5659d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.571+0000 7f743e5be740 4 client.27893 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:43:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.571+0000 7f743e5be740 1 -- 172.21.15.43:0/2838269196 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1b55a7990 con 0x55d1b5251f60 2024-04-25T10:43:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.571+0000 7f74354a9640 1 --2- 172.21.15.43:0/2838269196 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d1b5251f60 0x55d1b5659d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.571+0000 7f74354a9640 1 --2- 172.21.15.43:0/2838269196 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d1b5251f60 0x55d1b5659d80 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85c5a8b9507bc53c server_cookie=a5b466c4b6bef976 in_seq=0 out_seq=0 2024-04-25T10:43:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.571+0000 7f7418ff9640 10 client.27893 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.572+0000 7f7418ff9640 1 -- 172.21.15.43:0/2838269196 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d1b55a7990 con 0x55d1b5251f60 2024-04-25T10:43:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.572+0000 7f7418ff9640 10 client.27893 handle_command_reply: tid=0 2024-04-25T10:43:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.572+0000 7f743e5be740 1 client.27893 shutdown 2024-04-25T10:43:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.572+0000 7f743e5be740 1 -- 172.21.15.43:0/2838269196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b52726b0 msgr2=0x55d1b5647f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.572+0000 7f743e5be740 1 --2- 172.21.15.43:0/2838269196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b52726b0 0x55d1b5647f80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f74200a7420 tx=0x7f7420005760 comp rx=0 tx=0).stop 2024-04-25T10:43:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.572+0000 7f743e5be740 1 -- 172.21.15.43:0/2838269196 shutdown_connections 2024-04-25T10:43:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.572+0000 7f743e5be740 1 --2- 172.21.15.43:0/2838269196 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d1b5251f60 0x55d1b5659d80 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.572+0000 7f743e5be740 1 --2- 172.21.15.43:0/2838269196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b52726b0 0x55d1b5647f80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.572+0000 7f7418ff9640 0 client.27893 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.572+0000 7f743e5be740 1 --2- 172.21.15.43:0/2838269196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b5293390 0x55d1b5653ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.572+0000 7f743e5be740 1 --2- 172.21.15.43:0/2838269196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 0x55d1b5647a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.572+0000 7f743e5be740 1 -- 172.21.15.43:0/2838269196 >> 172.21.15.43:0/2838269196 conn(0x55d1b52874e0 msgr2=0x55d1b563ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.573+0000 7f743e5be740 1 -- 172.21.15.43:0/2838269196 shutdown_connections 2024-04-25T10:43:28.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.573+0000 7f743e5be740 1 -- 172.21.15.43:0/2838269196 wait complete. 2024-04-25T10:43:28.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.573+0000 7f743e5be740 20 client.27893 trim_cache size 0 max 16384 2024-04-25T10:43:28.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.602+0000 7f743e5be740 1 Processor -- start 2024-04-25T10:43:28.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.602+0000 7f743e5be740 1 -- start start 2024-04-25T10:43:28.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.602+0000 7f743e5be740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b5251f60 0x55d1b5653a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.602+0000 7f743e5be740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 0x55d1b5641d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.602+0000 7f743e5be740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b52726b0 0x55d1b565cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.602+0000 7f743e5be740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f74301b0120 con 0x55d1b526a4d0 2024-04-25T10:43:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.602+0000 7f743e5be740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f74301ad9f0 con 0x55d1b52726b0 2024-04-25T10:43:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.602+0000 7f743e5be740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74301ab860 con 0x55d1b5251f60 2024-04-25T10:43:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.602+0000 7f7434ca8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 0x55d1b5641d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.602+0000 7f7434ca8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 0x55d1b5641d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44608/0 (socket says 172.21.15.43:44608) 2024-04-25T10:43:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.602+0000 7f7434ca8640 1 -- 172.21.15.43:0/494206347 learned_addr learned my addr 172.21.15.43:0/494206347 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.603+0000 7f74354a9640 1 --2- 172.21.15.43:0/494206347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b5251f60 0x55d1b5653a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.603+0000 7f7435caa640 1 --2- 172.21.15.43:0/494206347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b52726b0 0x55d1b565cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.603+0000 7f7434ca8640 1 -- 172.21.15.43:0/494206347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b5251f60 msgr2=0x55d1b5653a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.603+0000 7f7434ca8640 1 --2- 172.21.15.43:0/494206347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b5251f60 0x55d1b5653a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.603+0000 7f7434ca8640 1 -- 172.21.15.43:0/494206347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b52726b0 msgr2=0x55d1b565cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.603+0000 7f7434ca8640 1 --2- 172.21.15.43:0/494206347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b52726b0 0x55d1b565cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.603+0000 7f7434ca8640 1 -- 172.21.15.43:0/494206347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b563a900 con 0x55d1b526a4d0 2024-04-25T10:43:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.603+0000 7f7434ca8640 1 --2- 172.21.15.43:0/494206347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 0x55d1b5641d20 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f74200aefd0 tx=0x7f742000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21bf76423adc9afe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.603+0000 7f741affd640 1 -- 172.21.15.43:0/494206347 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74200c5030 con 0x55d1b526a4d0 2024-04-25T10:43:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.603+0000 7f741affd640 1 -- 172.21.15.43:0/494206347 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74200ab840 con 0x55d1b526a4d0 2024-04-25T10:43:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.603+0000 7f741affd640 1 -- 172.21.15.43:0/494206347 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74200a61b0 con 0x55d1b526a4d0 2024-04-25T10:43:28.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.604+0000 7f743e5be740 1 -- 172.21.15.43:0/494206347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 msgr2=0x55d1b5641d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.604+0000 7f743e5be740 1 --2- 172.21.15.43:0/494206347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 0x55d1b5641d20 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f74200aefd0 tx=0x7f742000cf90 comp rx=0 tx=0).stop 2024-04-25T10:43:28.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.604+0000 7f743e5be740 1 -- 172.21.15.43:0/494206347 shutdown_connections 2024-04-25T10:43:28.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.604+0000 7f743e5be740 1 --2- 172.21.15.43:0/494206347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b52726b0 0x55d1b565cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.604+0000 7f743e5be740 1 --2- 172.21.15.43:0/494206347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b526a4d0 0x55d1b5641d20 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.604+0000 7f743e5be740 1 --2- 172.21.15.43:0/494206347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b5251f60 0x55d1b5653a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.604+0000 7f743e5be740 1 -- 172.21.15.43:0/494206347 >> 172.21.15.43:0/494206347 conn(0x55d1b52874e0 msgr2=0x55d1b5646100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.604+0000 7f743e5be740 1 -- 172.21.15.43:0/494206347 shutdown_connections 2024-04-25T10:43:28.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.604+0000 7f743e5be740 1 -- 172.21.15.43:0/494206347 wait complete. 2024-04-25T10:43:28.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.604+0000 7f743e5be740 1 -- start start 2024-04-25T10:43:28.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.605+0000 7f743e5be740 1 Processor -- start 2024-04-25T10:43:28.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.605+0000 7f743e5be740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b5251f60 0x55d1b565cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.605+0000 7f743e5be740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b526a4d0 0x55d1b565d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.605+0000 7f743e5be740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b52726b0 0x55d1b565a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.605+0000 7f743e5be740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f74301abeb0 con 0x55d1b52726b0 2024-04-25T10:43:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.605+0000 7f743e5be740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f74301ad320 con 0x55d1b526a4d0 2024-04-25T10:43:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.605+0000 7f743e5be740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74301ac580 con 0x55d1b5251f60 2024-04-25T10:43:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.605+0000 7f743e5be740 10 client.0 _ll_register_callbacks cb 0x7f7428009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.605+0000 7f7435caa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b52726b0 0x55d1b565a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.605+0000 7f7435caa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b52726b0 0x55d1b565a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44620/0 (socket says 172.21.15.43:44620) 2024-04-25T10:43:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.605+0000 7f7435caa640 1 -- 172.21.15.43:0/2647624452 learned_addr learned my addr 172.21.15.43:0/2647624452 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.605+0000 7f7434ca8640 1 --2- 172.21.15.43:0/2647624452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b526a4d0 0x55d1b565d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.605+0000 7f74354a9640 1 --2- 172.21.15.43:0/2647624452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b5251f60 0x55d1b565cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.606+0000 7f7435caa640 1 -- 172.21.15.43:0/2647624452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b5251f60 msgr2=0x55d1b565cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.606+0000 7f7435caa640 1 --2- 172.21.15.43:0/2647624452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b5251f60 0x55d1b565cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.606+0000 7f7435caa640 1 -- 172.21.15.43:0/2647624452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b526a4d0 msgr2=0x55d1b565d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.606+0000 7f7435caa640 1 --2- 172.21.15.43:0/2647624452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b526a4d0 0x55d1b565d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.606+0000 7f7435caa640 1 -- 172.21.15.43:0/2647624452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1b563cdb0 con 0x55d1b52726b0 2024-04-25T10:43:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.606+0000 7f74354a9640 1 --2- 172.21.15.43:0/2647624452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b5251f60 0x55d1b565cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:43:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.606+0000 7f7435caa640 1 --2- 172.21.15.43:0/2647624452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b52726b0 0x55d1b565a080 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f742800b6a0 tx=0x7f742800bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbc556f8c2c785d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.606+0000 7f7418ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.606+0000 7f7418ff9640 1 -- 172.21.15.43:0/2647624452 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742801f070 con 0x55d1b52726b0 2024-04-25T10:43:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.606+0000 7f7418ff9640 1 -- 172.21.15.43:0/2647624452 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f742800ca50 con 0x55d1b52726b0 2024-04-25T10:43:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.606+0000 7f743e5be740 1 -- 172.21.15.43:0/2647624452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74200ab840 con 0x55d1b52726b0 2024-04-25T10:43:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.606+0000 7f7418ff9640 1 -- 172.21.15.43:0/2647624452 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f742802b050 con 0x55d1b52726b0 2024-04-25T10:43:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.606+0000 7f743e5be740 10 client.18750 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.606+0000 7f743e5be740 1 -- 172.21.15.43:0/2647624452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d1b56c15f0 con 0x55d1b52726b0 2024-04-25T10:43:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.606+0000 7f7418ff9640 1 -- 172.21.15.43:0/2647624452 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74200ab840 con 0x55d1b52726b0 2024-04-25T10:43:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.607+0000 7f7418ff9640 1 -- 172.21.15.43:0/2647624452 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f742800b040 con 0x55d1b52726b0 2024-04-25T10:43:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.607+0000 7f743e5be740 10 client.18750 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.607+0000 7f743e5be740 10 client.18750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:43:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.607+0000 7f743e5be740 20 client.18750 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.607+0000 7f743e5be740 1 --2- 172.21.15.43:0/2647624452 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d1b5293390 0x55d1b56c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.607+0000 7f743e5be740 4 client.18750 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:43:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.607+0000 7f743e5be740 1 -- 172.21.15.43:0/2647624452 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1b55a1800 con 0x55d1b5293390 2024-04-25T10:43:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.607+0000 7f74354a9640 1 --2- 172.21.15.43:0/2647624452 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d1b5293390 0x55d1b56c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.607+0000 7f7418ff9640 10 client.18750 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.607+0000 7f74354a9640 1 --2- 172.21.15.43:0/2647624452 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d1b5293390 0x55d1b56c3d80 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5be618d54f610f5 server_cookie=d1826ab34c5019c8 in_seq=0 out_seq=0 2024-04-25T10:43:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.608+0000 7f7418ff9640 1 -- 172.21.15.43:0/2647624452 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1b55a1800 con 0x55d1b5293390 2024-04-25T10:43:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.608+0000 7f7418ff9640 10 client.18750 handle_command_reply: tid=0 2024-04-25T10:43:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.608+0000 7f743e5be740 1 client.18750 shutdown 2024-04-25T10:43:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.608+0000 7f743e5be740 1 -- 172.21.15.43:0/2647624452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b52726b0 msgr2=0x55d1b565a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.608+0000 7f743e5be740 1 --2- 172.21.15.43:0/2647624452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b52726b0 0x55d1b565a080 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f742800b6a0 tx=0x7f742800bb60 comp rx=0 tx=0).stop 2024-04-25T10:43:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.608+0000 7f743e5be740 1 -- 172.21.15.43:0/2647624452 shutdown_connections 2024-04-25T10:43:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.608+0000 7f743e5be740 1 --2- 172.21.15.43:0/2647624452 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55d1b5293390 0x55d1b56c3d80 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.608+0000 7f743e5be740 1 --2- 172.21.15.43:0/2647624452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d1b526a4d0 0x55d1b565d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.608+0000 7f743e5be740 1 --2- 172.21.15.43:0/2647624452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1b52726b0 0x55d1b565a080 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.608+0000 7f7418ff9640 0 client.18750 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.608+0000 7f743e5be740 1 --2- 172.21.15.43:0/2647624452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1b5251f60 0x55d1b565cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.608+0000 7f743e5be740 1 -- 172.21.15.43:0/2647624452 >> 172.21.15.43:0/2647624452 conn(0x55d1b52874e0 msgr2=0x55d1b5639170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.608+0000 7f743e5be740 1 -- 172.21.15.43:0/2647624452 shutdown_connections 2024-04-25T10:43:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.608+0000 7f743e5be740 1 -- 172.21.15.43:0/2647624452 wait complete. 2024-04-25T10:43:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.608+0000 7f743e5be740 20 client.18750 trim_cache size 0 max 16384 2024-04-25T10:43:28.616 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:43:28.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.609+0000 7f743770d640 1 -- 172.21.15.43:0/3579213707 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f741c0afc60 msgr2=0x7f741c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.609+0000 7f743770d640 1 --2- 172.21.15.43:0/3579213707 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f741c0afc60 0x7f741c0b2170 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f742c0303f0 tx=0x7f742c030340 comp rx=0 tx=0).stop 2024-04-25T10:43:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.609+0000 7f743770d640 1 -- 172.21.15.43:0/3579213707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7430109b20 msgr2=0x7f743019ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.609+0000 7f743770d640 1 --2- 172.21.15.43:0/3579213707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7430109b20 0x7f743019ad10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f74300754b0 tx=0x7f742000ee50 comp rx=0 tx=0).stop 2024-04-25T10:43:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.609+0000 7f743770d640 1 -- 172.21.15.43:0/3579213707 shutdown_connections 2024-04-25T10:43:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.609+0000 7f743770d640 1 --2- 172.21.15.43:0/3579213707 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f741c0afc60 0x7f741c0b2170 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.609+0000 7f743770d640 1 --2- 172.21.15.43:0/3579213707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7430109fc0 0x7f743019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.609+0000 7f743770d640 1 --2- 172.21.15.43:0/3579213707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7430109b20 0x7f743019ad10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.609+0000 7f743770d640 1 --2- 172.21.15.43:0/3579213707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74301027a0 0x7f743019a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.609+0000 7f743770d640 1 -- 172.21.15.43:0/3579213707 >> 172.21.15.43:0/3579213707 conn(0x7f7430077880 msgr2=0x7f7430105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.609+0000 7f743770d640 1 -- 172.21.15.43:0/3579213707 shutdown_connections 2024-04-25T10:43:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:28.609+0000 7f743770d640 1 -- 172.21.15.43:0/3579213707 wait complete. 2024-04-25T10:43:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.615+0000 7f087ac29640 1 -- 172.21.15.43:0/3429088133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0874109ba0 msgr2=0x7f0874103740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.615+0000 7f087ac29640 1 --2- 172.21.15.43:0/3429088133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0874109ba0 0x7f0874103740 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f0868009900 tx=0x7f086802a410 comp rx=0 tx=0).stop 2024-04-25T10:43:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.616+0000 7f087ac29640 1 -- 172.21.15.43:0/3429088133 shutdown_connections 2024-04-25T10:43:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.616+0000 7f087ac29640 1 --2- 172.21.15.43:0/3429088133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08741042f0 0x7f08741106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.616+0000 7f087ac29640 1 --2- 172.21.15.43:0/3429088133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f087410a040 0x7f0874103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.616+0000 7f087ac29640 1 --2- 172.21.15.43:0/3429088133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0874109ba0 0x7f0874103740 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.616+0000 7f087ac29640 1 -- 172.21.15.43:0/3429088133 >> 172.21.15.43:0/3429088133 conn(0x7f08740792a0 msgr2=0x7f08740796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.616+0000 7f087ac29640 1 -- 172.21.15.43:0/3429088133 shutdown_connections 2024-04-25T10:43:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.616+0000 7f087ac29640 1 -- 172.21.15.43:0/3429088133 wait complete. 2024-04-25T10:43:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.616+0000 7f087ac29640 1 Processor -- start 2024-04-25T10:43:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.616+0000 7f087ac29640 1 -- start start 2024-04-25T10:43:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.617+0000 7f087ac29640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08741042f0 0x7f087419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.617+0000 7f087ac29640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0874109ba0 0x7f087419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.617+0000 7f087ac29640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f087410a040 0x7f087419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.617+0000 7f087ac29640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0874113d90 con 0x7f087410a040 2024-04-25T10:43:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.617+0000 7f087ac29640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0874113c10 con 0x7f0874109ba0 2024-04-25T10:43:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.617+0000 7f087ac29640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0874113f10 con 0x7f08741042f0 2024-04-25T10:43:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.617+0000 7f08791c6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f087410a040 0x7f087419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.617+0000 7f08791c6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f087410a040 0x7f087419eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44644/0 (socket says 172.21.15.43:44644) 2024-04-25T10:43:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.617+0000 7f08791c6640 1 -- 172.21.15.43:0/652435569 learned_addr learned my addr 172.21.15.43:0/652435569 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.617+0000 7f08789c5640 1 --2- 172.21.15.43:0/652435569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08741042f0 0x7f087419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.617+0000 7f0873fff640 1 --2- 172.21.15.43:0/652435569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0874109ba0 0x7f087419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.617+0000 7f08791c6640 1 -- 172.21.15.43:0/652435569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08741042f0 msgr2=0x7f087419a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.617+0000 7f08791c6640 1 --2- 172.21.15.43:0/652435569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08741042f0 0x7f087419a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.617+0000 7f08791c6640 1 -- 172.21.15.43:0/652435569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0874109ba0 msgr2=0x7f087419ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.617+0000 7f08791c6640 1 --2- 172.21.15.43:0/652435569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0874109ba0 0x7f087419ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.617+0000 7f08791c6640 1 -- 172.21.15.43:0/652435569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087419f630 con 0x7f087410a040 2024-04-25T10:43:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.617+0000 7f08791c6640 1 --2- 172.21.15.43:0/652435569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f087410a040 0x7f087419eff0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f086c00efd0 tx=0x7f086c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.618+0000 7f0871ffb640 1 -- 172.21.15.43:0/652435569 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c003b70 con 0x7f087410a040 2024-04-25T10:43:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.618+0000 7f087ac29640 1 -- 172.21.15.43:0/652435569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f087419f920 con 0x7f087410a040 2024-04-25T10:43:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.618+0000 7f0871ffb640 1 -- 172.21.15.43:0/652435569 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f086c003d10 con 0x7f087410a040 2024-04-25T10:43:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.618+0000 7f0871ffb640 1 -- 172.21.15.43:0/652435569 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c0108b0 con 0x7f087410a040 2024-04-25T10:43:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.618+0000 7f087ac29640 1 -- 172.21.15.43:0/652435569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08741a72a0 con 0x7f087410a040 2024-04-25T10:43:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.621+0000 7f0871ffb640 1 -- 172.21.15.43:0/652435569 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f086c004020 con 0x7f087410a040 2024-04-25T10:43:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.621+0000 7f0881bb0740 1 Processor -- start 2024-04-25T10:43:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.621+0000 7f0881bb0740 1 -- start start 2024-04-25T10:43:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.621+0000 7f0881bb0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637323a21b0 0x563732741af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.621+0000 7f0881bb0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 0x563732742030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.621+0000 7f0881bb0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 0x563732742570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.621+0000 7f0881bb0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5637326a8460 con 0x5637323814d0 2024-04-25T10:43:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.621+0000 7f0881bb0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563732662590 con 0x5637323792f0 2024-04-25T10:43:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.621+0000 7f0881bb0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5637326a4420 con 0x5637323a21b0 2024-04-25T10:43:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.621+0000 7f0871ffb640 1 --2- 172.21.15.43:0/652435569 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f08640afc50 0x7f08640b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.621+0000 7f0873fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 0x563732742570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.621+0000 7f0873fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 0x563732742570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44652/0 (socket says 172.21.15.43:44652) 2024-04-25T10:43:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.621+0000 7f0873fff640 1 -- 172.21.15.43:0/1747739060 learned_addr learned my addr 172.21.15.43:0/1747739060 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.621+0000 7f08791c6640 1 --2- 172.21.15.43:0/652435569 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f08640afc50 0x7f08640b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.622+0000 7f08789c5640 1 --2- 172.21.15.43:0/1747739060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 0x563732742030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.622+0000 7f0873fff640 1 --2- 172.21.15.43:0/1747739060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637323a21b0 0x563732741af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.622+0000 7f0871ffb640 1 -- 172.21.15.43:0/652435569 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f086c0df440 con 0x7f087410a040 2024-04-25T10:43:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.622+0000 7f0873fff640 1 -- 172.21.15.43:0/1747739060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637323a21b0 msgr2=0x563732741af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.622+0000 7f0873fff640 1 --2- 172.21.15.43:0/1747739060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637323a21b0 0x563732741af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.622+0000 7f0873fff640 1 -- 172.21.15.43:0/1747739060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 msgr2=0x563732742030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.622+0000 7f0873fff640 1 --2- 172.21.15.43:0/1747739060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 0x563732742030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.622+0000 7f0873fff640 1 -- 172.21.15.43:0/1747739060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56373266f0d0 con 0x5637323814d0 2024-04-25T10:43:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.622+0000 7f0873fff640 1 --2- 172.21.15.43:0/1747739060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 0x563732742570 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f0860009710 tx=0x7f08600096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591b6258453f1915 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.622+0000 7f08791c6640 1 --2- 172.21.15.43:0/652435569 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f08640afc50 0x7f08640b2160 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f086c0dd190 tx=0x7f086c004930 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.622+0000 7f085effd640 1 -- 172.21.15.43:0/1747739060 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860004020 con 0x5637323814d0 2024-04-25T10:43:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.622+0000 7f085effd640 1 -- 172.21.15.43:0/1747739060 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08600054d0 con 0x5637323814d0 2024-04-25T10:43:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.622+0000 7f085effd640 1 -- 172.21.15.43:0/1747739060 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860004b90 con 0x5637323814d0 2024-04-25T10:43:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.623+0000 7f0881bb0740 1 -- 172.21.15.43:0/1747739060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 msgr2=0x563732742570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.623+0000 7f0881bb0740 1 --2- 172.21.15.43:0/1747739060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 0x563732742570 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f0860009710 tx=0x7f08600096a0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.623+0000 7f0881bb0740 1 -- 172.21.15.43:0/1747739060 shutdown_connections 2024-04-25T10:43:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.623+0000 7f0881bb0740 1 --2- 172.21.15.43:0/1747739060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 0x563732742030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.623+0000 7f0881bb0740 1 --2- 172.21.15.43:0/1747739060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 0x563732742570 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.623+0000 7f0881bb0740 1 --2- 172.21.15.43:0/1747739060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637323a21b0 0x563732741af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.623+0000 7f0881bb0740 1 -- 172.21.15.43:0/1747739060 >> 172.21.15.43:0/1747739060 conn(0x563732396300 msgr2=0x56373273e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.623+0000 7f0881bb0740 1 -- 172.21.15.43:0/1747739060 shutdown_connections 2024-04-25T10:43:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.623+0000 7f0881bb0740 1 -- 172.21.15.43:0/1747739060 wait complete. 2024-04-25T10:43:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.624+0000 7f0881bb0740 1 -- start start 2024-04-25T10:43:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.624+0000 7f0881bb0740 1 Processor -- start 2024-04-25T10:43:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.624+0000 7f0881bb0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 0x56373274c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.624+0000 7f0881bb0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637323814d0 0x56373274c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.624+0000 7f0881bb0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323a21b0 0x563732752110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.624+0000 7f0881bb0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56373274ec20 con 0x5637323a21b0 2024-04-25T10:43:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.624+0000 7f0881bb0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56373262fb60 con 0x5637323792f0 2024-04-25T10:43:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.624+0000 7f0881bb0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56373274eda0 con 0x5637323814d0 2024-04-25T10:43:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.624+0000 7f0881bb0740 10 client.0 _ll_register_callbacks cb 0x5637326542b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.624+0000 7f0873fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323a21b0 0x563732752110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.624+0000 7f0873fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323a21b0 0x563732752110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44668/0 (socket says 172.21.15.43:44668) 2024-04-25T10:43:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.624+0000 7f0873fff640 1 -- 172.21.15.43:0/4221244188 learned_addr learned my addr 172.21.15.43:0/4221244188 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.624+0000 7f08789c5640 1 --2- 172.21.15.43:0/4221244188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637323814d0 0x56373274c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.625+0000 7f0873fff640 1 --2- 172.21.15.43:0/4221244188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 0x56373274c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.625+0000 7f0873fff640 1 -- 172.21.15.43:0/4221244188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637323814d0 msgr2=0x56373274c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.625+0000 7f0873fff640 1 --2- 172.21.15.43:0/4221244188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637323814d0 0x56373274c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.625+0000 7f0873fff640 1 -- 172.21.15.43:0/4221244188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 msgr2=0x56373274c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.625+0000 7f0873fff640 1 --2- 172.21.15.43:0/4221244188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 0x56373274c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.625+0000 7f0873fff640 1 -- 172.21.15.43:0/4221244188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637323ba210 con 0x5637323a21b0 2024-04-25T10:43:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.625+0000 7f0873fff640 1 --2- 172.21.15.43:0/4221244188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323a21b0 0x563732752110 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f0860009250 tx=0x7f0860005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c7aac9021f01c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.625+0000 7f0853fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.625+0000 7f0853fff640 1 -- 172.21.15.43:0/4221244188 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860000e60 con 0x5637323a21b0 2024-04-25T10:43:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.625+0000 7f0853fff640 1 -- 172.21.15.43:0/4221244188 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f0860003ac0 con 0x5637323a21b0 2024-04-25T10:43:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.625+0000 7f0881bb0740 1 -- 172.21.15.43:0/4221244188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08600054d0 con 0x5637323a21b0 2024-04-25T10:43:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.625+0000 7f0853fff640 1 -- 172.21.15.43:0/4221244188 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0860003cc0 con 0x5637323a21b0 2024-04-25T10:43:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.625+0000 7f0881bb0740 10 client.18762 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.625+0000 7f0881bb0740 1 -- 172.21.15.43:0/4221244188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563732486620 con 0x5637323a21b0 2024-04-25T10:43:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.625+0000 7f0853fff640 1 -- 172.21.15.43:0/4221244188 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08600054d0 con 0x5637323a21b0 2024-04-25T10:43:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.626+0000 7f0853fff640 1 -- 172.21.15.43:0/4221244188 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0860028070 con 0x5637323a21b0 2024-04-25T10:43:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.626+0000 7f0881bb0740 10 client.18762 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.626+0000 7f0881bb0740 10 client.18762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:43:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.626+0000 7f0881bb0740 20 client.18762 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.626+0000 7f0881bb0740 1 --2- 172.21.15.43:0/4221244188 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563732360d80 0x56373275e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.626+0000 7f0881bb0740 4 client.18762 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:43:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.626+0000 7f0881bb0740 1 -- 172.21.15.43:0/4221244188 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637326b6070 con 0x563732360d80 2024-04-25T10:43:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.626+0000 7f08789c5640 1 --2- 172.21.15.43:0/4221244188 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563732360d80 0x56373275e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.626+0000 7f0853fff640 10 client.18762 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.626+0000 7f08789c5640 1 --2- 172.21.15.43:0/4221244188 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563732360d80 0x56373275e610 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62da2af0370ae704 server_cookie=8f725fca7e16aeb8 in_seq=0 out_seq=0 2024-04-25T10:43:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.627+0000 7f0853fff640 1 -- 172.21.15.43:0/4221244188 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5637326b6070 con 0x563732360d80 2024-04-25T10:43:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.627+0000 7f0853fff640 10 client.18762 handle_command_reply: tid=0 2024-04-25T10:43:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.627+0000 7f0881bb0740 1 client.18762 shutdown 2024-04-25T10:43:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.627+0000 7f0881bb0740 1 -- 172.21.15.43:0/4221244188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323a21b0 msgr2=0x563732752110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.627+0000 7f0881bb0740 1 --2- 172.21.15.43:0/4221244188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323a21b0 0x563732752110 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f0860009250 tx=0x7f0860005f90 comp rx=0 tx=0).stop 2024-04-25T10:43:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.627+0000 7f0881bb0740 1 -- 172.21.15.43:0/4221244188 shutdown_connections 2024-04-25T10:43:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.627+0000 7f0881bb0740 1 --2- 172.21.15.43:0/4221244188 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x563732360d80 0x56373275e610 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.627+0000 7f0881bb0740 1 --2- 172.21.15.43:0/4221244188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323a21b0 0x563732752110 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.627+0000 7f0881bb0740 1 --2- 172.21.15.43:0/4221244188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637323814d0 0x56373274c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.627+0000 7f0881bb0740 1 --2- 172.21.15.43:0/4221244188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 0x56373274c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.627+0000 7f0853fff640 0 client.18762 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.627+0000 7f0881bb0740 1 -- 172.21.15.43:0/4221244188 >> 172.21.15.43:0/4221244188 conn(0x563732396300 msgr2=0x563732743810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.627+0000 7f0881bb0740 1 -- 172.21.15.43:0/4221244188 shutdown_connections 2024-04-25T10:43:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.627+0000 7f0881bb0740 1 -- 172.21.15.43:0/4221244188 wait complete. 2024-04-25T10:43:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.628+0000 7f0881bb0740 20 client.18762 trim_cache size 0 max 16384 2024-04-25T10:43:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.643+0000 7f0881bb0740 1 Processor -- start 2024-04-25T10:43:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.643+0000 7f0881bb0740 1 -- start start 2024-04-25T10:43:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.643+0000 7f0881bb0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 0x563732752010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.643+0000 7f0881bb0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 0x563732743c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.643+0000 7f0881bb0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637323a21b0 0x563732763be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.643+0000 7f0881bb0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f08741b22b0 con 0x5637323814d0 2024-04-25T10:43:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.643+0000 7f0881bb0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08741afb80 con 0x5637323792f0 2024-04-25T10:43:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.643+0000 7f0881bb0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f08741ad9f0 con 0x5637323a21b0 2024-04-25T10:43:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.643+0000 7f08789c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 0x563732743c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.643+0000 7f08789c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 0x563732743c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44682/0 (socket says 172.21.15.43:44682) 2024-04-25T10:43:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.643+0000 7f08789c5640 1 -- 172.21.15.43:0/3484283362 learned_addr learned my addr 172.21.15.43:0/3484283362 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.643+0000 7f0873fff640 1 --2- 172.21.15.43:0/3484283362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637323a21b0 0x563732763be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.643+0000 7f08789c5640 1 -- 172.21.15.43:0/3484283362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637323a21b0 msgr2=0x563732763be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.643+0000 7f08789c5640 1 --2- 172.21.15.43:0/3484283362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637323a21b0 0x563732763be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.643+0000 7f08789c5640 1 -- 172.21.15.43:0/3484283362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 msgr2=0x563732752010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.643+0000 7f0873fff640 1 --2- 172.21.15.43:0/3484283362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 0x563732752010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.643+0000 7f08789c5640 1 --2- 172.21.15.43:0/3484283362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 0x563732752010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.643+0000 7f08789c5640 1 -- 172.21.15.43:0/3484283362 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563732752550 con 0x5637323814d0 2024-04-25T10:43:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.644+0000 7f08789c5640 1 --2- 172.21.15.43:0/3484283362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 0x563732743c80 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f0868005940 tx=0x7f0868031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8a33b0826a18b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.644+0000 7f085e7fc640 1 -- 172.21.15.43:0/3484283362 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086804b070 con 0x5637323814d0 2024-04-25T10:43:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.644+0000 7f085e7fc640 1 -- 172.21.15.43:0/3484283362 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0868004380 con 0x5637323814d0 2024-04-25T10:43:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.644+0000 7f085e7fc640 1 -- 172.21.15.43:0/3484283362 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0868035040 con 0x5637323814d0 2024-04-25T10:43:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.644+0000 7f0881bb0740 1 -- 172.21.15.43:0/3484283362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 msgr2=0x563732743c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.644+0000 7f0881bb0740 1 --2- 172.21.15.43:0/3484283362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 0x563732743c80 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f0868005940 tx=0x7f0868031400 comp rx=0 tx=0).stop 2024-04-25T10:43:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.645+0000 7f0881bb0740 1 -- 172.21.15.43:0/3484283362 shutdown_connections 2024-04-25T10:43:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.645+0000 7f0881bb0740 1 --2- 172.21.15.43:0/3484283362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637323a21b0 0x563732763be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.645+0000 7f0881bb0740 1 --2- 172.21.15.43:0/3484283362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 0x563732743c80 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.645+0000 7f0881bb0740 1 --2- 172.21.15.43:0/3484283362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 0x563732752010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.645+0000 7f0881bb0740 1 -- 172.21.15.43:0/3484283362 >> 172.21.15.43:0/3484283362 conn(0x563732396300 msgr2=0x563732743570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.645+0000 7f0881bb0740 1 -- 172.21.15.43:0/3484283362 shutdown_connections 2024-04-25T10:43:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.645+0000 7f0881bb0740 1 -- 172.21.15.43:0/3484283362 wait complete. 2024-04-25T10:43:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.645+0000 7f0881bb0740 1 -- start start 2024-04-25T10:43:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.645+0000 7f0881bb0740 1 Processor -- start 2024-04-25T10:43:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.645+0000 7f0881bb0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 0x563732758440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.645+0000 7f0881bb0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 0x563732753790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.645+0000 7f0881bb0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637323a21b0 0x563732753cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.646+0000 7f0881bb0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f08741ae040 con 0x5637323814d0 2024-04-25T10:43:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.646+0000 7f0881bb0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08741af4b0 con 0x5637323792f0 2024-04-25T10:43:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.646+0000 7f0881bb0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f08741ae710 con 0x5637323a21b0 2024-04-25T10:43:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.646+0000 7f0881bb0740 10 client.0 _ll_register_callbacks cb 0x7f0860005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.646+0000 7f08789c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 0x563732753790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.646+0000 7f08789c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 0x563732753790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44694/0 (socket says 172.21.15.43:44694) 2024-04-25T10:43:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.646+0000 7f08789c5640 1 -- 172.21.15.43:0/2257316817 learned_addr learned my addr 172.21.15.43:0/2257316817 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.646+0000 7f0873fff640 1 --2- 172.21.15.43:0/2257316817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 0x563732758440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.646+0000 7f0873fff640 1 --2- 172.21.15.43:0/2257316817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637323a21b0 0x563732753cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.646+0000 7f08789c5640 1 -- 172.21.15.43:0/2257316817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637323a21b0 msgr2=0x563732753cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.646+0000 7f08789c5640 1 --2- 172.21.15.43:0/2257316817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637323a21b0 0x563732753cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.646+0000 7f08789c5640 1 -- 172.21.15.43:0/2257316817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 msgr2=0x563732758440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.646+0000 7f08789c5640 1 --2- 172.21.15.43:0/2257316817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 0x563732758440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.646+0000 7f08789c5640 1 -- 172.21.15.43:0/2257316817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563732754210 con 0x5637323814d0 2024-04-25T10:43:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.646+0000 7f0873fff640 1 --2- 172.21.15.43:0/2257316817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 0x563732758440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:43:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.646+0000 7f08789c5640 1 --2- 172.21.15.43:0/2257316817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 0x563732753790 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f0868049040 tx=0x7f0868032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdd3c574dd80d4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.646+0000 7f0853fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.646+0000 7f0853fff640 1 -- 172.21.15.43:0/2257316817 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0868054060 con 0x5637323814d0 2024-04-25T10:43:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.646+0000 7f0853fff640 1 -- 172.21.15.43:0/2257316817 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f08680467c0 con 0x5637323814d0 2024-04-25T10:43:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.646+0000 7f0881bb0740 1 -- 172.21.15.43:0/2257316817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0868004380 con 0x5637323814d0 2024-04-25T10:43:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.647+0000 7f0853fff640 1 -- 172.21.15.43:0/2257316817 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0868035070 con 0x5637323814d0 2024-04-25T10:43:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.647+0000 7f0881bb0740 10 client.18768 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.647+0000 7f0881bb0740 1 -- 172.21.15.43:0/2257316817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5637327c6450 con 0x5637323814d0 2024-04-25T10:43:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.647+0000 7f0853fff640 1 -- 172.21.15.43:0/2257316817 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0868004380 con 0x5637323814d0 2024-04-25T10:43:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.647+0000 7f0853fff640 1 -- 172.21.15.43:0/2257316817 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f086804e030 con 0x5637323814d0 2024-04-25T10:43:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.647+0000 7f0881bb0740 10 client.18768 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.647+0000 7f0881bb0740 10 client.18768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:43:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.647+0000 7f0881bb0740 20 client.18768 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.647+0000 7f0881bb0740 1 --2- 172.21.15.43:0/2257316817 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5637323361c0 0x5637327c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.647+0000 7f0881bb0740 4 client.18768 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:43:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.647+0000 7f0881bb0740 1 -- 172.21.15.43:0/2257316817 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5637326afee0 con 0x5637323361c0 2024-04-25T10:43:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.647+0000 7f0873fff640 1 --2- 172.21.15.43:0/2257316817 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5637323361c0 0x5637327c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:28.655 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.648+0000 7f0853fff640 10 client.18768 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:28.655 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.648+0000 7f0873fff640 1 --2- 172.21.15.43:0/2257316817 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5637323361c0 0x5637327c8b60 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c58f1316d2236323 server_cookie=756b62ee64ac6521 in_seq=0 out_seq=0 2024-04-25T10:43:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.661+0000 7f0853fff640 1 -- 172.21.15.43:0/2257316817 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5637326afee0 con 0x5637323361c0 2024-04-25T10:43:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.661+0000 7f0853fff640 10 client.18768 handle_command_reply: tid=0 2024-04-25T10:43:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.661+0000 7f0881bb0740 1 client.18768 shutdown 2024-04-25T10:43:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.661+0000 7f0881bb0740 1 -- 172.21.15.43:0/2257316817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 msgr2=0x563732753790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.661+0000 7f0881bb0740 1 --2- 172.21.15.43:0/2257316817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 0x563732753790 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f0868049040 tx=0x7f0868032e60 comp rx=0 tx=0).stop 2024-04-25T10:43:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.661+0000 7f0881bb0740 1 -- 172.21.15.43:0/2257316817 shutdown_connections 2024-04-25T10:43:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.661+0000 7f0881bb0740 1 --2- 172.21.15.43:0/2257316817 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5637323361c0 0x5637327c8b60 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.661+0000 7f0881bb0740 1 --2- 172.21.15.43:0/2257316817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637323a21b0 0x563732753cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.661+0000 7f0881bb0740 1 --2- 172.21.15.43:0/2257316817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637323814d0 0x563732753790 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.661+0000 7f0853fff640 0 client.18768 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.661+0000 7f0881bb0740 1 --2- 172.21.15.43:0/2257316817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637323792f0 0x563732758440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.661+0000 7f0881bb0740 1 -- 172.21.15.43:0/2257316817 >> 172.21.15.43:0/2257316817 conn(0x563732396300 msgr2=0x563732743510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.662+0000 7f0881bb0740 1 -- 172.21.15.43:0/2257316817 shutdown_connections 2024-04-25T10:43:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.662+0000 7f0881bb0740 1 -- 172.21.15.43:0/2257316817 wait complete. 2024-04-25T10:43:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.662+0000 7f0881bb0740 20 client.18768 trim_cache size 0 max 16384 2024-04-25T10:43:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:43:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:43:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:43:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:43:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.662+0000 7f087ac29640 1 -- 172.21.15.43:0/652435569 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f08640afc50 msgr2=0x7f08640b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.662+0000 7f087ac29640 1 --2- 172.21.15.43:0/652435569 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f08640afc50 0x7f08640b2160 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f086c0dd190 tx=0x7f086c004930 comp rx=0 tx=0).stop 2024-04-25T10:43:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.662+0000 7f087ac29640 1 -- 172.21.15.43:0/652435569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f087410a040 msgr2=0x7f087419eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.662+0000 7f087ac29640 1 --2- 172.21.15.43:0/652435569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f087410a040 0x7f087419eff0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f086c00efd0 tx=0x7f086c00c6a0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.663+0000 7f087ac29640 1 -- 172.21.15.43:0/652435569 shutdown_connections 2024-04-25T10:43:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.663+0000 7f087ac29640 1 --2- 172.21.15.43:0/652435569 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f08640afc50 0x7f08640b2160 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.663+0000 7f087ac29640 1 --2- 172.21.15.43:0/652435569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0874109ba0 0x7f087419ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.663+0000 7f087ac29640 1 --2- 172.21.15.43:0/652435569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f087410a040 0x7f087419eff0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.663+0000 7f087ac29640 1 --2- 172.21.15.43:0/652435569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08741042f0 0x7f087419a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.663+0000 7f087ac29640 1 -- 172.21.15.43:0/652435569 >> 172.21.15.43:0/652435569 conn(0x7f08740792a0 msgr2=0x7f0874105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.663+0000 7f087ac29640 1 -- 172.21.15.43:0/652435569 shutdown_connections 2024-04-25T10:43:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:28.663+0000 7f087ac29640 1 -- 172.21.15.43:0/652435569 wait complete. 2024-04-25T10:43:29.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:29 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1263901241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:43:29.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:29 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1263901241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:43:29.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:29 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1263901241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:43:29.626 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:43:29.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:43:29.626 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:43:29.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.758+0000 7fc303d08640 1 -- 172.21.15.43:0/2104935084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2fc10d380 msgr2=0x7fc2fc10f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:29.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.758+0000 7fc303d08640 1 --2- 172.21.15.43:0/2104935084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2fc10d380 0x7fc2fc10f7a0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009ca0 tx=0x7fc2f802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.758+0000 7fc303d08640 1 -- 172.21.15.43:0/2104935084 shutdown_connections 2024-04-25T10:43:29.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.758+0000 7fc303d08640 1 --2- 172.21.15.43:0/2104935084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2fc102af0 0x7fc2fc102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.758+0000 7fc303d08640 1 --2- 172.21.15.43:0/2104935084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2fc10d380 0x7fc2fc10f7a0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.758+0000 7fc303d08640 1 --2- 172.21.15.43:0/2104935084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2fc108a20 0x7fc2fc1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.758+0000 7fc303d08640 1 -- 172.21.15.43:0/2104935084 >> 172.21.15.43:0/2104935084 conn(0x7fc2fc078100 msgr2=0x7fc2fc078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:29.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.758+0000 7fc303d08640 1 -- 172.21.15.43:0/2104935084 shutdown_connections 2024-04-25T10:43:29.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.758+0000 7fc303d08640 1 -- 172.21.15.43:0/2104935084 wait complete. 2024-04-25T10:43:29.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.759+0000 7fc303d08640 1 Processor -- start 2024-04-25T10:43:29.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.759+0000 7fc303d08640 1 -- start start 2024-04-25T10:43:29.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.759+0000 7fc303d08640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2fc102af0 0x7fc2fc19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.759+0000 7fc303d08640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2fc108a20 0x7fc2fc19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.759+0000 7fc303d08640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2fc10d380 0x7fc2fc19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.759+0000 7fc303d08640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc112c00 con 0x7fc2fc10d380 2024-04-25T10:43:29.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.759+0000 7fc303d08640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc112a80 con 0x7fc2fc108a20 2024-04-25T10:43:29.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.759+0000 7fc303d08640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc112d80 con 0x7fc2fc102af0 2024-04-25T10:43:29.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.759+0000 7fc3022a5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2fc10d380 0x7fc2fc19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.759+0000 7fc3022a5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2fc10d380 0x7fc2fc19eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54840/0 (socket says 172.21.15.43:54840) 2024-04-25T10:43:29.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.759+0000 7fc3022a5640 1 -- 172.21.15.43:0/3794549536 learned_addr learned my addr 172.21.15.43:0/3794549536 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:29.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.759+0000 7fc3012a3640 1 --2- 172.21.15.43:0/3794549536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2fc108a20 0x7fc2fc19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.759+0000 7fc301aa4640 1 --2- 172.21.15.43:0/3794549536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2fc102af0 0x7fc2fc19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.759+0000 7fc3022a5640 1 -- 172.21.15.43:0/3794549536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2fc102af0 msgr2=0x7fc2fc19a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:29.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.759+0000 7fc3022a5640 1 --2- 172.21.15.43:0/3794549536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2fc102af0 0x7fc2fc19a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.759+0000 7fc3022a5640 1 -- 172.21.15.43:0/3794549536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2fc108a20 msgr2=0x7fc2fc19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:29.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.759+0000 7fc3022a5640 1 --2- 172.21.15.43:0/3794549536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2fc108a20 0x7fc2fc19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.759+0000 7fc3022a5640 1 -- 172.21.15.43:0/3794549536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2fc19f530 con 0x7fc2fc10d380 2024-04-25T10:43:29.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.759+0000 7fc3012a3640 1 --2- 172.21.15.43:0/3794549536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2fc108a20 0x7fc2fc19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:43:29.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.760+0000 7fc3022a5640 1 --2- 172.21.15.43:0/3794549536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2fc10d380 0x7fc2fc19eef0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8005cc0 tx=0x7fc2f802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:29.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.760+0000 7fc2f2ffd640 1 -- 172.21.15.43:0/3794549536 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8003bf0 con 0x7fc2fc10d380 2024-04-25T10:43:29.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.760+0000 7fc303d08640 1 -- 172.21.15.43:0/3794549536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2fc19f7c0 con 0x7fc2fc10d380 2024-04-25T10:43:29.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.760+0000 7fc2f2ffd640 1 -- 172.21.15.43:0/3794549536 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f8003d90 con 0x7fc2fc10d380 2024-04-25T10:43:29.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.760+0000 7fc2f2ffd640 1 -- 172.21.15.43:0/3794549536 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f802fbc0 con 0x7fc2fc10d380 2024-04-25T10:43:29.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.760+0000 7fc303d08640 1 -- 172.21.15.43:0/3794549536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2fc1a71a0 con 0x7fc2fc10d380 2024-04-25T10:43:29.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.763+0000 7fc30abb6740 1 Processor -- start 2024-04-25T10:43:29.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.763+0000 7fc2f2ffd640 1 -- 172.21.15.43:0/3794549536 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fc2f802fd60 con 0x7fc2fc10d380 2024-04-25T10:43:29.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.764+0000 7fc30abb6740 1 -- start start 2024-04-25T10:43:29.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.764+0000 7fc30abb6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064fa410 0x55f00689a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.764+0000 7fc30abb6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d1550 0x55f00689ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.764+0000 7fc30abb6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 0x55f00689b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.764+0000 7fc30abb6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f006800b00 con 0x55f0064d9730 2024-04-25T10:43:29.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.764+0000 7fc30abb6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f00679c920 con 0x55f0064d1550 2024-04-25T10:43:29.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.764+0000 7fc30abb6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f0067fcac0 con 0x55f0064fa410 2024-04-25T10:43:29.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.764+0000 7fc3012a3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 0x55f00689b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.764+0000 7fc3012a3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 0x55f00689b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54846/0 (socket says 172.21.15.43:54846) 2024-04-25T10:43:29.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.764+0000 7fc3012a3640 1 -- 172.21.15.43:0/868173531 learned_addr learned my addr 172.21.15.43:0/868173531 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:29.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.764+0000 7fc301aa4640 1 --2- 172.21.15.43:0/868173531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d1550 0x55f00689ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.764+0000 7fc3012a3640 1 --2- 172.21.15.43:0/868173531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064fa410 0x55f00689a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.764+0000 7fc2f2ffd640 1 --2- 172.21.15.43:0/3794549536 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc2e80afc60 0x7fc2e80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.764+0000 7fc3012a3640 1 -- 172.21.15.43:0/868173531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064fa410 msgr2=0x55f00689a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:29.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.764+0000 7fc3012a3640 1 --2- 172.21.15.43:0/868173531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064fa410 0x55f00689a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.764+0000 7fc3012a3640 1 -- 172.21.15.43:0/868173531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d1550 msgr2=0x55f00689ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:29.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.764+0000 7fc3012a3640 1 --2- 172.21.15.43:0/868173531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d1550 0x55f00689ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.765+0000 7fc3012a3640 1 -- 172.21.15.43:0/868173531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0065da7e0 con 0x55f0064d9730 2024-04-25T10:43:29.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.765+0000 7fc3022a5640 1 --2- 172.21.15.43:0/3794549536 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc2e80afc60 0x7fc2e80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.765+0000 7fc2f2ffd640 1 -- 172.21.15.43:0/3794549536 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fc2f8100800 con 0x7fc2fc10d380 2024-04-25T10:43:29.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.765+0000 7fc3012a3640 1 --2- 172.21.15.43:0/868173531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 0x55f00689b0a0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec010370 tx=0x7fc2ec00b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1207cc1c33b3a83d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:29.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.765+0000 7fc2e7fff640 1 -- 172.21.15.43:0/868173531 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec018030 con 0x55f0064d9730 2024-04-25T10:43:29.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.765+0000 7fc2e7fff640 1 -- 172.21.15.43:0/868173531 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2ec0054d0 con 0x55f0064d9730 2024-04-25T10:43:29.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.765+0000 7fc2e7fff640 1 -- 172.21.15.43:0/868173531 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec012450 con 0x55f0064d9730 2024-04-25T10:43:29.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.766+0000 7fc30abb6740 1 -- 172.21.15.43:0/868173531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 msgr2=0x55f00689b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:29.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.766+0000 7fc3022a5640 1 --2- 172.21.15.43:0/3794549536 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc2e80afc60 0x7fc2e80b2170 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80fe110 tx=0x7fc2f8096310 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:29.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.766+0000 7fc30abb6740 1 --2- 172.21.15.43:0/868173531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 0x55f00689b0a0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec010370 tx=0x7fc2ec00b5c0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.766+0000 7fc30abb6740 1 -- 172.21.15.43:0/868173531 shutdown_connections 2024-04-25T10:43:29.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.766+0000 7fc30abb6740 1 --2- 172.21.15.43:0/868173531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d1550 0x55f00689ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.766+0000 7fc30abb6740 1 --2- 172.21.15.43:0/868173531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 0x55f00689b0a0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.766+0000 7fc30abb6740 1 --2- 172.21.15.43:0/868173531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064fa410 0x55f00689a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.766+0000 7fc30abb6740 1 -- 172.21.15.43:0/868173531 >> 172.21.15.43:0/868173531 conn(0x55f0064ee560 msgr2=0x55f006896dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:29.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.766+0000 7fc30abb6740 1 -- 172.21.15.43:0/868173531 shutdown_connections 2024-04-25T10:43:29.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.766+0000 7fc30abb6740 1 -- 172.21.15.43:0/868173531 wait complete. 2024-04-25T10:43:29.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.766+0000 7fc30abb6740 1 -- start start 2024-04-25T10:43:29.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.767+0000 7fc30abb6740 1 Processor -- start 2024-04-25T10:43:29.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.767+0000 7fc30abb6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064d1550 0x55f0068a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.767+0000 7fc30abb6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d9730 0x55f0068a5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.767+0000 7fc30abb6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064fa410 0x55f0068b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.767+0000 7fc30abb6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f0068a7970 con 0x55f0064fa410 2024-04-25T10:43:29.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.767+0000 7fc30abb6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f00679a5d0 con 0x55f0064d9730 2024-04-25T10:43:29.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.767+0000 7fc30abb6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f0068a7af0 con 0x55f0064d1550 2024-04-25T10:43:29.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.767+0000 7fc30abb6740 10 client.0 _ll_register_callbacks cb 0x55f006788080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.767+0000 7fc3012a3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064fa410 0x55f0068b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.767+0000 7fc3012a3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064fa410 0x55f0068b0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54858/0 (socket says 172.21.15.43:54858) 2024-04-25T10:43:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.767+0000 7fc3012a3640 1 -- 172.21.15.43:0/2364903542 learned_addr learned my addr 172.21.15.43:0/2364903542 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.767+0000 7fc3012a3640 1 --2- 172.21.15.43:0/2364903542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064d1550 0x55f0068a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.767+0000 7fc301aa4640 1 --2- 172.21.15.43:0/2364903542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d9730 0x55f0068a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.768+0000 7fc3012a3640 1 -- 172.21.15.43:0/2364903542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064d1550 msgr2=0x55f0068a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.768+0000 7fc3012a3640 1 --2- 172.21.15.43:0/2364903542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064d1550 0x55f0068a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.768+0000 7fc3012a3640 1 -- 172.21.15.43:0/2364903542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d9730 msgr2=0x55f0068a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.768+0000 7fc3012a3640 1 --2- 172.21.15.43:0/2364903542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d9730 0x55f0068a5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.768+0000 7fc3012a3640 1 -- 172.21.15.43:0/2364903542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f00663af70 con 0x55f0064fa410 2024-04-25T10:43:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.768+0000 7fc3012a3640 1 --2- 172.21.15.43:0/2364903542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064fa410 0x55f0068b0e20 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00be10 tx=0x7fc2ec004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d0a58a2907cbab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.768+0000 7fc2e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.768+0000 7fc2e57fa640 1 -- 172.21.15.43:0/2364903542 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec018040 con 0x55f0064fa410 2024-04-25T10:43:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.768+0000 7fc2e57fa640 1 -- 172.21.15.43:0/2364903542 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fc2ec005210 con 0x55f0064fa410 2024-04-25T10:43:29.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.768+0000 7fc30abb6740 1 -- 172.21.15.43:0/2364903542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2ec0054d0 con 0x55f0064fa410 2024-04-25T10:43:29.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.768+0000 7fc2e57fa640 1 -- 172.21.15.43:0/2364903542 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc2ec005e50 con 0x55f0064fa410 2024-04-25T10:43:29.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.768+0000 7fc30abb6740 10 client.18780 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:29.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.768+0000 7fc30abb6740 1 -- 172.21.15.43:0/2364903542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f0064d2170 con 0x55f0064fa410 2024-04-25T10:43:29.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.768+0000 7fc2e57fa640 1 -- 172.21.15.43:0/2364903542 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2ec0054d0 con 0x55f0064fa410 2024-04-25T10:43:29.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.769+0000 7fc2e57fa640 1 -- 172.21.15.43:0/2364903542 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc2ec029020 con 0x55f0064fa410 2024-04-25T10:43:29.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.769+0000 7fc30abb6740 10 client.18780 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:29.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.769+0000 7fc30abb6740 10 client.18780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:43:29.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.769+0000 7fc30abb6740 20 client.18780 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:29.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.769+0000 7fc30abb6740 1 --2- 172.21.15.43:0/2364903542 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f0064b91f0 0x55f0068b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.769+0000 7fc30abb6740 4 client.18780 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:43:29.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.769+0000 7fc30abb6740 1 -- 172.21.15.43:0/2364903542 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f00680e710 con 0x55f0064b91f0 2024-04-25T10:43:29.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.769+0000 7fc301aa4640 1 --2- 172.21.15.43:0/2364903542 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f0064b91f0 0x55f0068b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.770+0000 7fc2e57fa640 10 client.18780 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:29.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.770+0000 7fc301aa4640 1 --2- 172.21.15.43:0/2364903542 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f0064b91f0 0x55f0068b6f20 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e42b9a504495474e server_cookie=56f22d628f3be5c1 in_seq=0 out_seq=0 2024-04-25T10:43:29.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.770+0000 7fc2e57fa640 1 -- 172.21.15.43:0/2364903542 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f00680e710 con 0x55f0064b91f0 2024-04-25T10:43:29.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.770+0000 7fc2e57fa640 10 client.18780 handle_command_reply: tid=0 2024-04-25T10:43:29.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.770+0000 7fc30abb6740 1 client.18780 shutdown 2024-04-25T10:43:29.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.771+0000 7fc30abb6740 1 -- 172.21.15.43:0/2364903542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064fa410 msgr2=0x55f0068b0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:29.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.771+0000 7fc30abb6740 1 --2- 172.21.15.43:0/2364903542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064fa410 0x55f0068b0e20 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00be10 tx=0x7fc2ec004bc0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.771+0000 7fc30abb6740 1 -- 172.21.15.43:0/2364903542 shutdown_connections 2024-04-25T10:43:29.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.771+0000 7fc30abb6740 1 --2- 172.21.15.43:0/2364903542 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f0064b91f0 0x55f0068b6f20 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.771+0000 7fc30abb6740 1 --2- 172.21.15.43:0/2364903542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d9730 0x55f0068a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.771+0000 7fc30abb6740 1 --2- 172.21.15.43:0/2364903542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064fa410 0x55f0068b0e20 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.771+0000 7fc30abb6740 1 --2- 172.21.15.43:0/2364903542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064d1550 0x55f0068a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.771+0000 7fc2e57fa640 0 client.18780 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:29.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.771+0000 7fc30abb6740 1 -- 172.21.15.43:0/2364903542 >> 172.21.15.43:0/2364903542 conn(0x55f0064ee560 msgr2=0x55f00689c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:29.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.771+0000 7fc30abb6740 1 -- 172.21.15.43:0/2364903542 shutdown_connections 2024-04-25T10:43:29.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.771+0000 7fc30abb6740 1 -- 172.21.15.43:0/2364903542 wait complete. 2024-04-25T10:43:29.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.771+0000 7fc30abb6740 20 client.18780 trim_cache size 0 max 16384 2024-04-25T10:43:29.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.789+0000 7fc30abb6740 1 Processor -- start 2024-04-25T10:43:29.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.789+0000 7fc30abb6740 1 -- start start 2024-04-25T10:43:29.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.790+0000 7fc30abb6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d1550 0x55f00689afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.790+0000 7fc30abb6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 0x55f00689f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.790+0000 7fc30abb6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064fa410 0x55f0068bc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.790+0000 7fc30abb6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc1b21e0 con 0x55f0064d9730 2024-04-25T10:43:29.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.790+0000 7fc30abb6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc1afab0 con 0x55f0064d1550 2024-04-25T10:43:29.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.790+0000 7fc30abb6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc1ad920 con 0x55f0064fa410 2024-04-25T10:43:29.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.790+0000 7fc301aa4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 0x55f00689f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.790+0000 7fc301aa4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 0x55f00689f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54868/0 (socket says 172.21.15.43:54868) 2024-04-25T10:43:29.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.790+0000 7fc301aa4640 1 -- 172.21.15.43:0/1076805747 learned_addr learned my addr 172.21.15.43:0/1076805747 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:29.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.790+0000 7fc3012a3640 1 --2- 172.21.15.43:0/1076805747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064fa410 0x55f0068bc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.790+0000 7fc3012a3640 1 --2- 172.21.15.43:0/1076805747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d1550 0x55f00689afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.790+0000 7fc301aa4640 1 -- 172.21.15.43:0/1076805747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064fa410 msgr2=0x55f0068bc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:29.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.790+0000 7fc301aa4640 1 --2- 172.21.15.43:0/1076805747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064fa410 0x55f0068bc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.790+0000 7fc301aa4640 1 -- 172.21.15.43:0/1076805747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d1550 msgr2=0x55f00689afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:29.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.790+0000 7fc301aa4640 1 --2- 172.21.15.43:0/1076805747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d1550 0x55f00689afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.790+0000 7fc301aa4640 1 -- 172.21.15.43:0/1076805747 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f00689a3b0 con 0x55f0064d9730 2024-04-25T10:43:29.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.790+0000 7fc3012a3640 1 --2- 172.21.15.43:0/1076805747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d1550 0x55f00689afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:43:29.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.790+0000 7fc301aa4640 1 --2- 172.21.15.43:0/1076805747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 0x55f00689f170 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009cd0 tx=0x7fc2f40044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74239851c9a309 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:29.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.790+0000 7fc2e77fe640 1 -- 172.21.15.43:0/1076805747 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4022020 con 0x55f0064d9730 2024-04-25T10:43:29.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.790+0000 7fc2e77fe640 1 -- 172.21.15.43:0/1076805747 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f4003e20 con 0x55f0064d9730 2024-04-25T10:43:29.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.791+0000 7fc2e77fe640 1 -- 172.21.15.43:0/1076805747 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4012b00 con 0x55f0064d9730 2024-04-25T10:43:29.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.791+0000 7fc30abb6740 1 -- 172.21.15.43:0/1076805747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 msgr2=0x55f00689f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:29.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.791+0000 7fc30abb6740 1 --2- 172.21.15.43:0/1076805747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 0x55f00689f170 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009cd0 tx=0x7fc2f40044d0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.791+0000 7fc30abb6740 1 -- 172.21.15.43:0/1076805747 shutdown_connections 2024-04-25T10:43:29.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.791+0000 7fc30abb6740 1 --2- 172.21.15.43:0/1076805747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064fa410 0x55f0068bc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.791+0000 7fc30abb6740 1 --2- 172.21.15.43:0/1076805747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 0x55f00689f170 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.791+0000 7fc30abb6740 1 --2- 172.21.15.43:0/1076805747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d1550 0x55f00689afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.791+0000 7fc30abb6740 1 -- 172.21.15.43:0/1076805747 >> 172.21.15.43:0/1076805747 conn(0x55f0064ee560 msgr2=0x55f0068a6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:29.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.791+0000 7fc30abb6740 1 -- 172.21.15.43:0/1076805747 shutdown_connections 2024-04-25T10:43:29.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.791+0000 7fc30abb6740 1 -- 172.21.15.43:0/1076805747 wait complete. 2024-04-25T10:43:29.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.792+0000 7fc30abb6740 1 -- start start 2024-04-25T10:43:29.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.792+0000 7fc30abb6740 1 Processor -- start 2024-04-25T10:43:29.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.792+0000 7fc30abb6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d1550 0x55f0068a85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.792+0000 7fc30abb6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 0x55f0068a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.792+0000 7fc30abb6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064fa410 0x55f0068ac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.792+0000 7fc30abb6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc1adf70 con 0x55f0064d9730 2024-04-25T10:43:29.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.792+0000 7fc30abb6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc1af3e0 con 0x55f0064d1550 2024-04-25T10:43:29.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.792+0000 7fc30abb6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc1ae640 con 0x55f0064fa410 2024-04-25T10:43:29.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.792+0000 7fc30abb6740 10 client.0 _ll_register_callbacks cb 0x7fc2ec0050b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:29.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.792+0000 7fc301aa4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 0x55f0068a8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.792+0000 7fc301aa4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 0x55f0068a8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54878/0 (socket says 172.21.15.43:54878) 2024-04-25T10:43:29.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.792+0000 7fc301aa4640 1 -- 172.21.15.43:0/4063442338 learned_addr learned my addr 172.21.15.43:0/4063442338 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:29.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.792+0000 7fc3012a3640 1 --2- 172.21.15.43:0/4063442338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064fa410 0x55f0068ac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.793+0000 7fc3012a3640 1 --2- 172.21.15.43:0/4063442338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d1550 0x55f0068a85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.793+0000 7fc301aa4640 1 -- 172.21.15.43:0/4063442338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064fa410 msgr2=0x55f0068ac9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:29.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.793+0000 7fc301aa4640 1 --2- 172.21.15.43:0/4063442338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064fa410 0x55f0068ac9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.793+0000 7fc301aa4640 1 -- 172.21.15.43:0/4063442338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d1550 msgr2=0x55f0068a85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.793+0000 7fc301aa4640 1 --2- 172.21.15.43:0/4063442338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d1550 0x55f0068a85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.793+0000 7fc301aa4640 1 -- 172.21.15.43:0/4063442338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f006896300 con 0x55f0064d9730 2024-04-25T10:43:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.793+0000 7fc3012a3640 1 --2- 172.21.15.43:0/4063442338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064fa410 0x55f0068ac9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:43:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.793+0000 7fc301aa4640 1 --2- 172.21.15.43:0/4063442338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 0x55f0068a8af0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009f30 tx=0x7fc2f4005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9439be2666f5a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.793+0000 7fc2e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.793+0000 7fc2e57fa640 1 -- 172.21.15.43:0/4063442338 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f401e420 con 0x55f0064d9730 2024-04-25T10:43:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.793+0000 7fc2e57fa640 1 -- 172.21.15.43:0/4063442338 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fc2f401e5c0 con 0x55f0064d9730 2024-04-25T10:43:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.793+0000 7fc30abb6740 1 -- 172.21.15.43:0/4063442338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f4003e20 con 0x55f0064d9730 2024-04-25T10:43:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.793+0000 7fc2e57fa640 1 -- 172.21.15.43:0/4063442338 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc2f401e7c0 con 0x55f0064d9730 2024-04-25T10:43:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.793+0000 7fc30abb6740 10 client.18786 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.793+0000 7fc30abb6740 1 -- 172.21.15.43:0/4063442338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f006897900 con 0x55f0064d9730 2024-04-25T10:43:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.793+0000 7fc2e57fa640 1 -- 172.21.15.43:0/4063442338 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f4003e20 con 0x55f0064d9730 2024-04-25T10:43:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.794+0000 7fc2e57fa640 1 -- 172.21.15.43:0/4063442338 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc2f402f020 con 0x55f0064d9730 2024-04-25T10:43:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.794+0000 7fc30abb6740 10 client.18786 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.794+0000 7fc30abb6740 10 client.18786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:43:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.794+0000 7fc30abb6740 20 client.18786 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.794+0000 7fc30abb6740 1 --2- 172.21.15.43:0/4063442338 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f00648e630 0x55f006920ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.794+0000 7fc30abb6740 4 client.18786 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:43:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.794+0000 7fc30abb6740 1 -- 172.21.15.43:0/4063442338 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f006897250 con 0x55f00648e630 2024-04-25T10:43:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.794+0000 7fc3012a3640 1 --2- 172.21.15.43:0/4063442338 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f00648e630 0x55f006920ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.794+0000 7fc3012a3640 1 --2- 172.21.15.43:0/4063442338 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f00648e630 0x55f006920ec0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a86196889c4016dd server_cookie=95fcd1fdba8c12d1 in_seq=0 out_seq=0 2024-04-25T10:43:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.794+0000 7fc2e57fa640 10 client.18786 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.796+0000 7fc2e57fa640 1 -- 172.21.15.43:0/4063442338 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f006897250 con 0x55f00648e630 2024-04-25T10:43:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.796+0000 7fc2e57fa640 10 client.18786 handle_command_reply: tid=0 2024-04-25T10:43:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.796+0000 7fc30abb6740 1 client.18786 shutdown 2024-04-25T10:43:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.796+0000 7fc30abb6740 1 -- 172.21.15.43:0/4063442338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 msgr2=0x55f0068a8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.796+0000 7fc30abb6740 1 --2- 172.21.15.43:0/4063442338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 0x55f0068a8af0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009f30 tx=0x7fc2f4005040 comp rx=0 tx=0).stop 2024-04-25T10:43:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.796+0000 7fc30abb6740 1 -- 172.21.15.43:0/4063442338 shutdown_connections 2024-04-25T10:43:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.796+0000 7fc30abb6740 1 --2- 172.21.15.43:0/4063442338 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55f00648e630 0x55f006920ec0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.796+0000 7fc30abb6740 1 --2- 172.21.15.43:0/4063442338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f0064fa410 0x55f0068ac9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.796+0000 7fc30abb6740 1 --2- 172.21.15.43:0/4063442338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0064d9730 0x55f0068a8af0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.796+0000 7fc2e57fa640 0 client.18786 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:29.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.796+0000 7fc30abb6740 1 --2- 172.21.15.43:0/4063442338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0064d1550 0x55f0068a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.796+0000 7fc30abb6740 1 -- 172.21.15.43:0/4063442338 >> 172.21.15.43:0/4063442338 conn(0x55f0064ee560 msgr2=0x55f00689ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:29.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.796+0000 7fc30abb6740 1 -- 172.21.15.43:0/4063442338 shutdown_connections 2024-04-25T10:43:29.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.796+0000 7fc30abb6740 1 -- 172.21.15.43:0/4063442338 wait complete. 2024-04-25T10:43:29.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.796+0000 7fc30abb6740 20 client.18786 trim_cache size 0 max 16384 2024-04-25T10:43:29.803 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:43:29.803 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:43:29.804 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "22400349-30eb-400e-ad60-27c9e3d4df69", 2024-04-25T10:43:29.804 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:43:29.804 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:43:29.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.797+0000 7fc303d08640 1 -- 172.21.15.43:0/3794549536 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc2e80afc60 msgr2=0x7fc2e80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:29.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.797+0000 7fc303d08640 1 --2- 172.21.15.43:0/3794549536 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc2e80afc60 0x7fc2e80b2170 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80fe110 tx=0x7fc2f8096310 comp rx=0 tx=0).stop 2024-04-25T10:43:29.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.797+0000 7fc303d08640 1 -- 172.21.15.43:0/3794549536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2fc10d380 msgr2=0x7fc2fc19eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:29.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.797+0000 7fc303d08640 1 --2- 172.21.15.43:0/3794549536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2fc10d380 0x7fc2fc19eef0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8005cc0 tx=0x7fc2f802f380 comp rx=0 tx=0).stop 2024-04-25T10:43:29.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.797+0000 7fc303d08640 1 -- 172.21.15.43:0/3794549536 shutdown_connections 2024-04-25T10:43:29.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.797+0000 7fc303d08640 1 --2- 172.21.15.43:0/3794549536 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc2e80afc60 0x7fc2e80b2170 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.797+0000 7fc303d08640 1 --2- 172.21.15.43:0/3794549536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2fc108a20 0x7fc2fc19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.797+0000 7fc303d08640 1 --2- 172.21.15.43:0/3794549536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2fc10d380 0x7fc2fc19eef0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.797+0000 7fc303d08640 1 --2- 172.21.15.43:0/3794549536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2fc102af0 0x7fc2fc19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.797+0000 7fc303d08640 1 -- 172.21.15.43:0/3794549536 >> 172.21.15.43:0/3794549536 conn(0x7fc2fc078100 msgr2=0x7fc2fc069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:29.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.797+0000 7fc303d08640 1 -- 172.21.15.43:0/3794549536 shutdown_connections 2024-04-25T10:43:29.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.797+0000 7fc303d08640 1 -- 172.21.15.43:0/3794549536 wait complete. 2024-04-25T10:43:29.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:43:29.810 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:43:29.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.922+0000 7fa4b42f9640 1 -- 172.21.15.43:0/1478492260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac107950 msgr2=0x7fa4ac102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:29.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.922+0000 7fa4b42f9640 1 --2- 172.21.15.43:0/1478492260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac107950 0x7fa4ac102d80 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009900 tx=0x7fa4a802a410 comp rx=0 tx=0).stop 2024-04-25T10:43:29.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.922+0000 7fa4b42f9640 1 -- 172.21.15.43:0/1478492260 shutdown_connections 2024-04-25T10:43:29.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.922+0000 7fa4b42f9640 1 --2- 172.21.15.43:0/1478492260 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4ac10d440 0x7fa4ac10f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.922+0000 7fa4b42f9640 1 --2- 172.21.15.43:0/1478492260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4ac1032c0 0x7fa4ac1036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.922+0000 7fa4b42f9640 1 --2- 172.21.15.43:0/1478492260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac107950 0x7fa4ac102d80 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.922+0000 7fa4b42f9640 1 -- 172.21.15.43:0/1478492260 >> 172.21.15.43:0/1478492260 conn(0x7fa4ac077920 msgr2=0x7fa4ac077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:29.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.922+0000 7fa4b42f9640 1 -- 172.21.15.43:0/1478492260 shutdown_connections 2024-04-25T10:43:29.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.922+0000 7fa4b42f9640 1 -- 172.21.15.43:0/1478492260 wait complete. 2024-04-25T10:43:29.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.923+0000 7fa4b42f9640 1 Processor -- start 2024-04-25T10:43:29.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.923+0000 7fa4b42f9640 1 -- start start 2024-04-25T10:43:29.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.923+0000 7fa4b42f9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4ac1032c0 0x7fa4ac19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.923+0000 7fa4b42f9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4ac107950 0x7fa4ac19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.923+0000 7fa4b42f9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac10d440 0x7fa4ac19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.923+0000 7fa4b42f9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac113d20 con 0x7fa4ac10d440 2024-04-25T10:43:29.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.923+0000 7fa4b42f9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac113ba0 con 0x7fa4ac107950 2024-04-25T10:43:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.923+0000 7fa4b42f9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac113ea0 con 0x7fa4ac1032c0 2024-04-25T10:43:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.923+0000 7fa4b2896640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac10d440 0x7fa4ac19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.923+0000 7fa4b2896640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac10d440 0x7fa4ac19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54906/0 (socket says 172.21.15.43:54906) 2024-04-25T10:43:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.923+0000 7fa4b2896640 1 -- 172.21.15.43:0/4229470202 learned_addr learned my addr 172.21.15.43:0/4229470202 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.924+0000 7fa4b2095640 1 --2- 172.21.15.43:0/4229470202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4ac1032c0 0x7fa4ac19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.924+0000 7fa4b1894640 1 --2- 172.21.15.43:0/4229470202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4ac107950 0x7fa4ac19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.924+0000 7fa4b2896640 1 -- 172.21.15.43:0/4229470202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4ac1032c0 msgr2=0x7fa4ac19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.924+0000 7fa4b2896640 1 --2- 172.21.15.43:0/4229470202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4ac1032c0 0x7fa4ac19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.924+0000 7fa4b2896640 1 -- 172.21.15.43:0/4229470202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4ac107950 msgr2=0x7fa4ac19ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.924+0000 7fa4b2896640 1 --2- 172.21.15.43:0/4229470202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4ac107950 0x7fa4ac19ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.924+0000 7fa4b2896640 1 -- 172.21.15.43:0/4229470202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ac19f580 con 0x7fa4ac10d440 2024-04-25T10:43:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.924+0000 7fa4b2896640 1 --2- 172.21.15.43:0/4229470202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac10d440 0x7fa4ac19ef40 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400eb40 tx=0x7fa4a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.924+0000 7fa4a37fe640 1 -- 172.21.15.43:0/4229470202 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4004b20 con 0x7fa4ac10d440 2024-04-25T10:43:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.924+0000 7fa4a37fe640 1 -- 172.21.15.43:0/4229470202 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4a4004cc0 con 0x7fa4ac10d440 2024-04-25T10:43:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.924+0000 7fa4b42f9640 1 -- 172.21.15.43:0/4229470202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ac19f870 con 0x7fa4ac10d440 2024-04-25T10:43:29.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.924+0000 7fa4a37fe640 1 -- 172.21.15.43:0/4229470202 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4010720 con 0x7fa4ac10d440 2024-04-25T10:43:29.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.924+0000 7fa4b42f9640 1 -- 172.21.15.43:0/4229470202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ac19fd70 con 0x7fa4ac10d440 2024-04-25T10:43:29.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.927+0000 7fa4bb1ae740 1 Processor -- start 2024-04-25T10:43:29.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.927+0000 7fa4a37fe640 1 -- 172.21.15.43:0/4229470202 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fa4a4005560 con 0x7fa4ac10d440 2024-04-25T10:43:29.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.927+0000 7fa4bb1ae740 1 -- start start 2024-04-25T10:43:29.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.928+0000 7fa4bb1ae740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f9e58c390 0x557f9e92c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.928+0000 7fa4bb1ae740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e5634d0 0x557f9e92ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.928+0000 7fa4bb1ae740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e56b6b0 0x557f9e93dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.928+0000 7fa4bb1ae740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557f9e892d80 con 0x557f9e56b6b0 2024-04-25T10:43:29.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.928+0000 7fa4bb1ae740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557f9e84c8b0 con 0x557f9e5634d0 2024-04-25T10:43:29.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.928+0000 7fa4bb1ae740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557f9e88ed40 con 0x557f9e58c390 2024-04-25T10:43:29.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.928+0000 7fa4b1894640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e56b6b0 0x557f9e93dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.928+0000 7fa4b1894640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e56b6b0 0x557f9e93dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54914/0 (socket says 172.21.15.43:54914) 2024-04-25T10:43:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.928+0000 7fa4b1894640 1 -- 172.21.15.43:0/3122220509 learned_addr learned my addr 172.21.15.43:0/3122220509 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.928+0000 7fa4a37fe640 1 --2- 172.21.15.43:0/4229470202 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa4980afc60 0x7fa4980b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.928+0000 7fa4b2095640 1 --2- 172.21.15.43:0/3122220509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e5634d0 0x557f9e92ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.928+0000 7fa4b1894640 1 --2- 172.21.15.43:0/3122220509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f9e58c390 0x557f9e92c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.928+0000 7fa4b2896640 1 --2- 172.21.15.43:0/4229470202 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa4980afc60 0x7fa4980b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.928+0000 7fa4b1894640 1 -- 172.21.15.43:0/3122220509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f9e58c390 msgr2=0x557f9e92c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.928+0000 7fa4b1894640 1 --2- 172.21.15.43:0/3122220509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f9e58c390 0x557f9e92c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.928+0000 7fa4a37fe640 1 -- 172.21.15.43:0/4229470202 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fa4a40df880 con 0x7fa4ac10d440 2024-04-25T10:43:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.928+0000 7fa4b1894640 1 -- 172.21.15.43:0/3122220509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e5634d0 msgr2=0x557f9e92ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.928+0000 7fa4b1894640 1 --2- 172.21.15.43:0/3122220509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e5634d0 0x557f9e92ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.928+0000 7fa4b1894640 1 -- 172.21.15.43:0/3122220509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f9e81da40 con 0x557f9e56b6b0 2024-04-25T10:43:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.928+0000 7fa4b2095640 1 --2- 172.21.15.43:0/3122220509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e5634d0 0x557f9e92ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:43:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.929+0000 7fa4b1894640 1 --2- 172.21.15.43:0/3122220509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e56b6b0 0x557f9e93dd00 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fa49c012730 tx=0x7fa49c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd59985e351c837 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.929+0000 7fa4b2896640 1 --2- 172.21.15.43:0/4229470202 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa4980afc60 0x7fa4980b2170 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40dd080 tx=0x7fa4a4090040 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.929+0000 7fa4a0bfe640 1 -- 172.21.15.43:0/3122220509 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c012cb0 con 0x557f9e56b6b0 2024-04-25T10:43:29.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.929+0000 7fa4a0bfe640 1 -- 172.21.15.43:0/3122220509 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa49c012e50 con 0x557f9e56b6b0 2024-04-25T10:43:29.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.929+0000 7fa4a0bfe640 1 -- 172.21.15.43:0/3122220509 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c0035f0 con 0x557f9e56b6b0 2024-04-25T10:43:29.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.930+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/3122220509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e56b6b0 msgr2=0x557f9e93dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:29.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.930+0000 7fa4bb1ae740 1 --2- 172.21.15.43:0/3122220509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e56b6b0 0x557f9e93dd00 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fa49c012730 tx=0x7fa49c0096c0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.930+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/3122220509 shutdown_connections 2024-04-25T10:43:29.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.930+0000 7fa4bb1ae740 1 --2- 172.21.15.43:0/3122220509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e5634d0 0x557f9e92ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.930+0000 7fa4bb1ae740 1 --2- 172.21.15.43:0/3122220509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e56b6b0 0x557f9e93dd00 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.930+0000 7fa4bb1ae740 1 --2- 172.21.15.43:0/3122220509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f9e58c390 0x557f9e92c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.930+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/3122220509 >> 172.21.15.43:0/3122220509 conn(0x557f9e5804e0 msgr2=0x557f9e929210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.930+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/3122220509 shutdown_connections 2024-04-25T10:43:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.930+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/3122220509 wait complete. 2024-04-25T10:43:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.930+0000 7fa4bb1ae740 1 -- start start 2024-04-25T10:43:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.930+0000 7fa4bb1ae740 1 Processor -- start 2024-04-25T10:43:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.931+0000 7fa4bb1ae740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f9e5634d0 0x557f9e93ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.931+0000 7fa4bb1ae740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e56b6b0 0x557f9e93bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.931+0000 7fa4bb1ae740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e58c390 0x557f9e941970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.931+0000 7fa4bb1ae740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557f9e93e3f0 con 0x557f9e58c390 2024-04-25T10:43:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.931+0000 7fa4bb1ae740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557f9e819d10 con 0x557f9e56b6b0 2024-04-25T10:43:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.931+0000 7fa4bb1ae740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557f9e93e570 con 0x557f9e5634d0 2024-04-25T10:43:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.931+0000 7fa4bb1ae740 10 client.0 _ll_register_callbacks cb 0x557f9e846210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.931+0000 7fa4b1894640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e58c390 0x557f9e941970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.931+0000 7fa4b1894640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e58c390 0x557f9e941970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54924/0 (socket says 172.21.15.43:54924) 2024-04-25T10:43:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.931+0000 7fa4b1894640 1 -- 172.21.15.43:0/2862178687 learned_addr learned my addr 172.21.15.43:0/2862178687 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.931+0000 7fa4b2095640 1 --2- 172.21.15.43:0/2862178687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e56b6b0 0x557f9e93bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.931+0000 7fa4b1894640 1 --2- 172.21.15.43:0/2862178687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f9e5634d0 0x557f9e93ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.931+0000 7fa4b1894640 1 -- 172.21.15.43:0/2862178687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f9e5634d0 msgr2=0x557f9e93ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.931+0000 7fa4b1894640 1 --2- 172.21.15.43:0/2862178687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f9e5634d0 0x557f9e93ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.931+0000 7fa4b1894640 1 -- 172.21.15.43:0/2862178687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e56b6b0 msgr2=0x557f9e93bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.931+0000 7fa4b1894640 1 --2- 172.21.15.43:0/2862178687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e56b6b0 0x557f9e93bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.931+0000 7fa4b1894640 1 -- 172.21.15.43:0/2862178687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f9e66e7d0 con 0x557f9e58c390 2024-04-25T10:43:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.931+0000 7fa4b1894640 1 --2- 172.21.15.43:0/2862178687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e58c390 0x557f9e941970 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fa49c004620 tx=0x7fa49c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4203b946fec9e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.931+0000 7fa495ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.932+0000 7fa495ffb640 1 -- 172.21.15.43:0/2862178687 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c005030 con 0x557f9e58c390 2024-04-25T10:43:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.932+0000 7fa495ffb640 1 -- 172.21.15.43:0/2862178687 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fa49c0051d0 con 0x557f9e58c390 2024-04-25T10:43:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.932+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/2862178687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa49c012e50 con 0x557f9e58c390 2024-04-25T10:43:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.932+0000 7fa495ffb640 1 -- 172.21.15.43:0/2862178687 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa49c033050 con 0x557f9e58c390 2024-04-25T10:43:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.932+0000 7fa4bb1ae740 10 client.18798 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.932+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/2862178687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557f9e5a43f0 con 0x557f9e58c390 2024-04-25T10:43:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.932+0000 7fa495ffb640 1 -- 172.21.15.43:0/2862178687 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa49c012e50 con 0x557f9e58c390 2024-04-25T10:43:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.932+0000 7fa495ffb640 1 -- 172.21.15.43:0/2862178687 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa49c01f5b0 con 0x557f9e58c390 2024-04-25T10:43:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.932+0000 7fa4bb1ae740 10 client.18798 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.932+0000 7fa4bb1ae740 10 client.18798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:43:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.932+0000 7fa4bb1ae740 20 client.18798 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.932+0000 7fa4bb1ae740 1 --2- 172.21.15.43:0/2862178687 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557f9e54af60 0x557f9e94d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.932+0000 7fa4bb1ae740 4 client.18798 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:43:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.932+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/2862178687 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f9e8a0990 con 0x557f9e54af60 2024-04-25T10:43:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.933+0000 7fa4b2095640 1 --2- 172.21.15.43:0/2862178687 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557f9e54af60 0x557f9e94d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.933+0000 7fa4b2095640 1 --2- 172.21.15.43:0/2862178687 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557f9e54af60 0x557f9e94d9e0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34f5bf208fd79a84 server_cookie=cb3c8a42ba43a16 in_seq=0 out_seq=0 2024-04-25T10:43:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.933+0000 7fa495ffb640 10 client.18798 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.934+0000 7fa495ffb640 1 -- 172.21.15.43:0/2862178687 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557f9e8a0990 con 0x557f9e54af60 2024-04-25T10:43:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.934+0000 7fa495ffb640 10 client.18798 handle_command_reply: tid=0 2024-04-25T10:43:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.934+0000 7fa4bb1ae740 1 client.18798 shutdown 2024-04-25T10:43:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.934+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/2862178687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e58c390 msgr2=0x557f9e941970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.934+0000 7fa4bb1ae740 1 --2- 172.21.15.43:0/2862178687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e58c390 0x557f9e941970 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fa49c004620 tx=0x7fa49c014040 comp rx=0 tx=0).stop 2024-04-25T10:43:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.934+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/2862178687 shutdown_connections 2024-04-25T10:43:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.934+0000 7fa4bb1ae740 1 --2- 172.21.15.43:0/2862178687 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557f9e54af60 0x557f9e94d9e0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.934+0000 7fa4bb1ae740 1 --2- 172.21.15.43:0/2862178687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e56b6b0 0x557f9e93bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.934+0000 7fa4bb1ae740 1 --2- 172.21.15.43:0/2862178687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e58c390 0x557f9e941970 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.934+0000 7fa4bb1ae740 1 --2- 172.21.15.43:0/2862178687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f9e5634d0 0x557f9e93ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.934+0000 7fa495ffb640 0 client.18798 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.934+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/2862178687 >> 172.21.15.43:0/2862178687 conn(0x557f9e5804e0 msgr2=0x557f9e92e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.934+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/2862178687 shutdown_connections 2024-04-25T10:43:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.934+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/2862178687 wait complete. 2024-04-25T10:43:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.934+0000 7fa4bb1ae740 20 client.18798 trim_cache size 0 max 16384 2024-04-25T10:43:29.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.952+0000 7fa4bb1ae740 1 Processor -- start 2024-04-25T10:43:29.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.952+0000 7fa4bb1ae740 1 -- start start 2024-04-25T10:43:29.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.952+0000 7fa4bb1ae740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e5634d0 0x557f9e92c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.952+0000 7fa4bb1ae740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e56b6b0 0x557f9e93d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.952+0000 7fa4bb1ae740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f9e58c390 0x557f9e941870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.952+0000 7fa4bb1ae740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1afe40 con 0x557f9e5634d0 2024-04-25T10:43:29.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.952+0000 7fa4bb1ae740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1ad710 con 0x557f9e56b6b0 2024-04-25T10:43:29.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.952+0000 7fa4bb1ae740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1ab5a0 con 0x557f9e58c390 2024-04-25T10:43:29.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.952+0000 7fa4b1894640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e5634d0 0x557f9e92c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.952+0000 7fa4b1894640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e5634d0 0x557f9e92c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54926/0 (socket says 172.21.15.43:54926) 2024-04-25T10:43:29.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.952+0000 7fa4b1894640 1 -- 172.21.15.43:0/1939453252 learned_addr learned my addr 172.21.15.43:0/1939453252 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:29.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.953+0000 7fa4b1894640 1 -- 172.21.15.43:0/1939453252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f9e58c390 msgr2=0x557f9e941870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:29.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.953+0000 7fa4b1894640 1 --2- 172.21.15.43:0/1939453252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f9e58c390 0x557f9e941870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.953+0000 7fa4b1894640 1 -- 172.21.15.43:0/1939453252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e56b6b0 msgr2=0x557f9e93d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:29.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.953+0000 7fa4b1894640 1 --2- 172.21.15.43:0/1939453252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e56b6b0 0x557f9e93d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.953+0000 7fa4b1894640 1 -- 172.21.15.43:0/1939453252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f9e94dd50 con 0x557f9e5634d0 2024-04-25T10:43:29.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.953+0000 7fa4b1894640 1 --2- 172.21.15.43:0/1939453252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e5634d0 0x557f9e92c7f0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009e30 tx=0x7fa49c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8396fbb44f1558 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:29.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.953+0000 7fa497fff640 1 -- 172.21.15.43:0/1939453252 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c0095f0 con 0x557f9e5634d0 2024-04-25T10:43:29.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.953+0000 7fa497fff640 1 -- 172.21.15.43:0/1939453252 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa49c01fb70 con 0x557f9e5634d0 2024-04-25T10:43:29.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.953+0000 7fa497fff640 1 -- 172.21.15.43:0/1939453252 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c01f6a0 con 0x557f9e5634d0 2024-04-25T10:43:29.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.954+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/1939453252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e5634d0 msgr2=0x557f9e92c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:29.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.954+0000 7fa4bb1ae740 1 --2- 172.21.15.43:0/1939453252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e5634d0 0x557f9e92c7f0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009e30 tx=0x7fa49c009e60 comp rx=0 tx=0).stop 2024-04-25T10:43:29.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.954+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/1939453252 shutdown_connections 2024-04-25T10:43:29.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.954+0000 7fa4bb1ae740 1 --2- 172.21.15.43:0/1939453252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e56b6b0 0x557f9e93d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.954+0000 7fa4bb1ae740 1 --2- 172.21.15.43:0/1939453252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f9e58c390 0x557f9e941870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.954+0000 7fa4bb1ae740 1 --2- 172.21.15.43:0/1939453252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e5634d0 0x557f9e92c7f0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.954+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/1939453252 >> 172.21.15.43:0/1939453252 conn(0x557f9e5804e0 msgr2=0x557f9e92ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:29.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.954+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/1939453252 shutdown_connections 2024-04-25T10:43:29.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.954+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/1939453252 wait complete. 2024-04-25T10:43:29.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.954+0000 7fa4bb1ae740 1 -- start start 2024-04-25T10:43:29.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.954+0000 7fa4bb1ae740 1 Processor -- start 2024-04-25T10:43:29.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.954+0000 7fa4bb1ae740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f9e5634d0 0x557f9e944c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.954+0000 7fa4bb1ae740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e56b6b0 0x557f9e945160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.954+0000 7fa4bb1ae740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e58c390 0x557f9e9456a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.954+0000 7fa4bb1ae740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1abbf0 con 0x557f9e58c390 2024-04-25T10:43:29.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.954+0000 7fa4bb1ae740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1ad040 con 0x557f9e56b6b0 2024-04-25T10:43:29.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.954+0000 7fa4bb1ae740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1ac2a0 con 0x557f9e5634d0 2024-04-25T10:43:29.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.955+0000 7fa4bb1ae740 10 client.0 _ll_register_callbacks cb 0x7fa49c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:29.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.955+0000 7fa4b1894640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e58c390 0x557f9e9456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.955+0000 7fa4b1894640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e58c390 0x557f9e9456a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54934/0 (socket says 172.21.15.43:54934) 2024-04-25T10:43:29.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.955+0000 7fa4b1894640 1 -- 172.21.15.43:0/1725743095 learned_addr learned my addr 172.21.15.43:0/1725743095 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:29.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.955+0000 7fa4b2095640 1 --2- 172.21.15.43:0/1725743095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e56b6b0 0x557f9e945160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.955+0000 7fa4b1894640 1 --2- 172.21.15.43:0/1725743095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f9e5634d0 0x557f9e944c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.955+0000 7fa4b1894640 1 -- 172.21.15.43:0/1725743095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f9e5634d0 msgr2=0x557f9e944c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:29.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.955+0000 7fa4b1894640 1 --2- 172.21.15.43:0/1725743095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f9e5634d0 0x557f9e944c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.955+0000 7fa4b1894640 1 -- 172.21.15.43:0/1725743095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e56b6b0 msgr2=0x557f9e945160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:29.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.955+0000 7fa4b1894640 1 --2- 172.21.15.43:0/1725743095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e56b6b0 0x557f9e945160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.955+0000 7fa4b1894640 1 -- 172.21.15.43:0/1725743095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f9e5640f0 con 0x557f9e58c390 2024-04-25T10:43:29.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.955+0000 7fa4b2095640 1 --2- 172.21.15.43:0/1725743095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e56b6b0 0x557f9e945160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:43:29.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.955+0000 7fa4b1894640 1 --2- 172.21.15.43:0/1725743095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e58c390 0x557f9e9456a0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009b80 tx=0x7fa49c005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372690ab2d2e4ff2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:29.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.955+0000 7fa495ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:29.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.955+0000 7fa495ffb640 1 -- 172.21.15.43:0/1725743095 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c005250 con 0x557f9e58c390 2024-04-25T10:43:29.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.955+0000 7fa495ffb640 1 -- 172.21.15.43:0/1725743095 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fa49c017030 con 0x557f9e58c390 2024-04-25T10:43:29.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.955+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/1725743095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa49c01fb70 con 0x557f9e58c390 2024-04-25T10:43:29.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.955+0000 7fa495ffb640 1 -- 172.21.15.43:0/1725743095 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa49c0053f0 con 0x557f9e58c390 2024-04-25T10:43:29.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.955+0000 7fa4bb1ae740 10 client.18804 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:29.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.955+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/1725743095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557f9e8534b0 con 0x557f9e58c390 2024-04-25T10:43:29.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.956+0000 7fa495ffb640 1 -- 172.21.15.43:0/1725743095 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa49c01fb70 con 0x557f9e58c390 2024-04-25T10:43:29.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.956+0000 7fa495ffb640 1 -- 172.21.15.43:0/1725743095 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa49c02d590 con 0x557f9e58c390 2024-04-25T10:43:29.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.956+0000 7fa4bb1ae740 10 client.18804 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:29.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.956+0000 7fa4bb1ae740 10 client.18804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:43:29.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.956+0000 7fa4bb1ae740 20 client.18804 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:29.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.956+0000 7fa4bb1ae740 1 --2- 172.21.15.43:0/1725743095 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557f9e5203a0 0x557f9e9b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:29.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.956+0000 7fa4bb1ae740 4 client.18804 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:43:29.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.956+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/1725743095 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557f9e89a800 con 0x557f9e5203a0 2024-04-25T10:43:29.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.956+0000 7fa4b2095640 1 --2- 172.21.15.43:0/1725743095 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557f9e5203a0 0x557f9e9b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:29.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.957+0000 7fa495ffb640 10 client.18804 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:29.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.957+0000 7fa4b2095640 1 --2- 172.21.15.43:0/1725743095 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557f9e5203a0 0x557f9e9b71a0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9b0539327c012ec server_cookie=d7fa6ca293fdfc80 in_seq=0 out_seq=0 2024-04-25T10:43:29.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.957+0000 7fa495ffb640 1 -- 172.21.15.43:0/1725743095 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557f9e89a800 con 0x557f9e5203a0 2024-04-25T10:43:29.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.957+0000 7fa495ffb640 10 client.18804 handle_command_reply: tid=0 2024-04-25T10:43:29.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.957+0000 7fa4bb1ae740 1 client.18804 shutdown 2024-04-25T10:43:29.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.957+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/1725743095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e58c390 msgr2=0x557f9e9456a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:29.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.957+0000 7fa4bb1ae740 1 --2- 172.21.15.43:0/1725743095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e58c390 0x557f9e9456a0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009b80 tx=0x7fa49c005750 comp rx=0 tx=0).stop 2024-04-25T10:43:29.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.957+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/1725743095 shutdown_connections 2024-04-25T10:43:29.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.957+0000 7fa4bb1ae740 1 --2- 172.21.15.43:0/1725743095 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x557f9e5203a0 0x557f9e9b71a0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.957+0000 7fa4bb1ae740 1 --2- 172.21.15.43:0/1725743095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f9e56b6b0 0x557f9e945160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.957+0000 7fa495ffb640 0 client.18804 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:29.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.957+0000 7fa4bb1ae740 1 --2- 172.21.15.43:0/1725743095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f9e58c390 0x557f9e9456a0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.957+0000 7fa4bb1ae740 1 --2- 172.21.15.43:0/1725743095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f9e5634d0 0x557f9e944c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.957+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/1725743095 >> 172.21.15.43:0/1725743095 conn(0x557f9e5804e0 msgr2=0x557f9e93d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:29.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.957+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/1725743095 shutdown_connections 2024-04-25T10:43:29.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.957+0000 7fa4bb1ae740 1 -- 172.21.15.43:0/1725743095 wait complete. 2024-04-25T10:43:29.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.957+0000 7fa4bb1ae740 20 client.18804 trim_cache size 0 max 16384 2024-04-25T10:43:29.965 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:43:29.965 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:43:29.965 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:43:29.965 INFO:teuthology.orchestra.run.smithi043.stdout: "22400349-30eb-400e-ad60-27c9e3d4df69": { 2024-04-25T10:43:29.966 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:43:29.966 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "22400349-30eb-400e-ad60-27c9e3d4df69", 2024-04-25T10:43:29.966 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:43:29.966 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:43:29.966 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:43:29.966 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:43:29.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.958+0000 7fa4b42f9640 1 -- 172.21.15.43:0/4229470202 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa4980afc60 msgr2=0x7fa4980b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:29.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.958+0000 7fa4b42f9640 1 --2- 172.21.15.43:0/4229470202 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa4980afc60 0x7fa4980b2170 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40dd080 tx=0x7fa4a4090040 comp rx=0 tx=0).stop 2024-04-25T10:43:29.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.958+0000 7fa4b42f9640 1 -- 172.21.15.43:0/4229470202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac10d440 msgr2=0x7fa4ac19ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:29.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.958+0000 7fa4b42f9640 1 --2- 172.21.15.43:0/4229470202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac10d440 0x7fa4ac19ef40 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400eb40 tx=0x7fa4a400c2d0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.958+0000 7fa4b42f9640 1 -- 172.21.15.43:0/4229470202 shutdown_connections 2024-04-25T10:43:29.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.958+0000 7fa4b42f9640 1 --2- 172.21.15.43:0/4229470202 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fa4980afc60 0x7fa4980b2170 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.958+0000 7fa4b42f9640 1 --2- 172.21.15.43:0/4229470202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4ac107950 0x7fa4ac19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.958+0000 7fa4b42f9640 1 --2- 172.21.15.43:0/4229470202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac10d440 0x7fa4ac19ef40 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.958+0000 7fa4b42f9640 1 --2- 172.21.15.43:0/4229470202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4ac1032c0 0x7fa4ac19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:29.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.958+0000 7fa4b42f9640 1 -- 172.21.15.43:0/4229470202 >> 172.21.15.43:0/4229470202 conn(0x7fa4ac077920 msgr2=0x7fa4ac1008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:29.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.958+0000 7fa4b42f9640 1 -- 172.21.15.43:0/4229470202 shutdown_connections 2024-04-25T10:43:29.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:43:29.958+0000 7fa4b42f9640 1 -- 172.21.15.43:0/4229470202 wait complete. 2024-04-25T10:43:29.971 INFO:tasks.cephfs.filesystem:scrub status for tag:22400349-30eb-400e-ad60-27c9e3d4df69 - {'path': '/', 'tag': '22400349-30eb-400e-ad60-27c9e3d4df69', 'options': 'recursive,force'} 2024-04-25T10:43:30.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:30 smithi077 ceph-mon[32772]: pgmap v342: 97 pgs: 97 active+clean; 5.3 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-04-25T10:43:30.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:30 smithi164 ceph-mon[32713]: pgmap v342: 97 pgs: 97 active+clean; 5.3 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-04-25T10:43:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:30 smithi043 ceph-mon[29478]: pgmap v342: 97 pgs: 97 active+clean; 5.3 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-04-25T10:43:31.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:31 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:43:31.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:31 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:43:31.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:31 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:43:31.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:31 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:43:31.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:31 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:43:31.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:31 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:43:31.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:31 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:43:31.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:31 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:43:31.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:31 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:43:31.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:31 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:43:31.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:31 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:43:31.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:31 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:43:32.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:32 smithi077 ceph-mon[32772]: pgmap v343: 97 pgs: 97 active+clean; 5.3 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-04-25T10:43:32.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:32 smithi164 ceph-mon[32713]: pgmap v343: 97 pgs: 97 active+clean; 5.3 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-04-25T10:43:32.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:32 smithi043 ceph-mon[29478]: pgmap v343: 97 pgs: 97 active+clean; 5.3 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-04-25T10:43:34.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:34 smithi077 ceph-mon[32772]: pgmap v344: 97 pgs: 97 active+clean; 5.3 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-04-25T10:43:34.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:34 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:43:34.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:34 smithi164 ceph-mon[32713]: pgmap v344: 97 pgs: 97 active+clean; 5.3 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-04-25T10:43:34.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:34 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:43:34.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:34 smithi043 ceph-mon[29478]: pgmap v344: 97 pgs: 97 active+clean; 5.3 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-04-25T10:43:34.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:34 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:43:34.614 INFO:tasks.workunit.client.0.smithi043.stdout:499/600: open fd = 3 2024-04-25T10:43:34.614 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.614 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.614 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.614 INFO:tasks.workunit.client.0.smithi043.stdout:500/600: open fd = 3 2024-04-25T10:43:34.615 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.615 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.615 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.615 INFO:tasks.workunit.client.0.smithi043.stdout:501/600: open fd = 3 2024-04-25T10:43:34.615 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.615 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.615 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.615 INFO:tasks.workunit.client.0.smithi043.stdout:502/600: open fd = 3 2024-04-25T10:43:34.615 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.615 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.615 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.615 INFO:tasks.workunit.client.0.smithi043.stdout:503/600: open fd = 3 2024-04-25T10:43:34.615 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.615 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.616 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.616 INFO:tasks.workunit.client.0.smithi043.stdout:504/600: open fd = 3 2024-04-25T10:43:34.616 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.616 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.616 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.616 INFO:tasks.workunit.client.0.smithi043.stdout:505/600: open fd = 3 2024-04-25T10:43:34.616 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.616 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.616 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.616 INFO:tasks.workunit.client.0.smithi043.stdout:506/600: open fd = 3 2024-04-25T10:43:34.616 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.616 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.616 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.616 INFO:tasks.workunit.client.0.smithi043.stdout:507/600: open fd = 3 2024-04-25T10:43:34.616 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.617 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.617 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.617 INFO:tasks.workunit.client.0.smithi043.stdout:508/600: open fd = 3 2024-04-25T10:43:34.617 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.617 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.617 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.617 INFO:tasks.workunit.client.0.smithi043.stdout:509/600: open fd = 3 2024-04-25T10:43:34.617 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.617 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.617 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.617 INFO:tasks.workunit.client.0.smithi043.stdout:510/600: open fd = 3 2024-04-25T10:43:34.617 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.617 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.618 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.618 INFO:tasks.workunit.client.0.smithi043.stdout:511/600: open fd = 3 2024-04-25T10:43:34.618 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.618 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.618 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.618 INFO:tasks.workunit.client.0.smithi043.stdout:512/600: open fd = 3 2024-04-25T10:43:34.618 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.618 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.618 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.618 INFO:tasks.workunit.client.0.smithi043.stdout:513/600: open fd = 3 2024-04-25T10:43:34.618 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.618 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.618 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.618 INFO:tasks.workunit.client.0.smithi043.stdout:514/600: open fd = 3 2024-04-25T10:43:34.619 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.619 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.619 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.619 INFO:tasks.workunit.client.0.smithi043.stdout:515/600: open fd = 3 2024-04-25T10:43:34.619 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.619 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.619 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.619 INFO:tasks.workunit.client.0.smithi043.stdout:516/600: open fd = 3 2024-04-25T10:43:34.620 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.620 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.620 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.620 INFO:tasks.workunit.client.0.smithi043.stdout:517/600: open fd = 3 2024-04-25T10:43:34.620 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.620 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.620 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.620 INFO:tasks.workunit.client.0.smithi043.stdout:518/600: open fd = 3 2024-04-25T10:43:34.620 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.620 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.621 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.621 INFO:tasks.workunit.client.0.smithi043.stdout:519/600: open fd = 3 2024-04-25T10:43:34.622 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.622 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.622 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.623 INFO:tasks.workunit.client.0.smithi043.stdout:520/600: open fd = 3 2024-04-25T10:43:34.623 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.623 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.623 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.623 INFO:tasks.workunit.client.0.smithi043.stdout:521/600: open fd = 3 2024-04-25T10:43:34.623 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.623 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.623 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.623 INFO:tasks.workunit.client.0.smithi043.stdout:522/600: open fd = 3 2024-04-25T10:43:34.623 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.623 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.624 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.624 INFO:tasks.workunit.client.0.smithi043.stdout:523/600: open fd = 3 2024-04-25T10:43:34.624 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.624 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.624 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.624 INFO:tasks.workunit.client.0.smithi043.stdout:524/600: open fd = 3 2024-04-25T10:43:34.624 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.624 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.624 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.624 INFO:tasks.workunit.client.0.smithi043.stdout:525/600: open fd = 3 2024-04-25T10:43:34.624 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.624 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.624 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.624 INFO:tasks.workunit.client.0.smithi043.stdout:526/600: open fd = 3 2024-04-25T10:43:34.625 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.625 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.625 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.625 INFO:tasks.workunit.client.0.smithi043.stdout:527/600: open fd = 3 2024-04-25T10:43:34.625 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.625 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.625 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.625 INFO:tasks.workunit.client.0.smithi043.stdout:528/600: open fd = 3 2024-04-25T10:43:34.625 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.625 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.625 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.625 INFO:tasks.workunit.client.0.smithi043.stdout:529/600: open fd = 3 2024-04-25T10:43:34.625 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.625 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.626 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.626 INFO:tasks.workunit.client.0.smithi043.stdout:530/600: open fd = 3 2024-04-25T10:43:34.626 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.626 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.626 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.626 INFO:tasks.workunit.client.0.smithi043.stdout:531/600: open fd = 3 2024-04-25T10:43:34.626 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.626 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.626 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.626 INFO:tasks.workunit.client.0.smithi043.stdout:532/600: open fd = 3 2024-04-25T10:43:34.626 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.626 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.626 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.627 INFO:tasks.workunit.client.0.smithi043.stdout:533/600: open fd = 3 2024-04-25T10:43:34.627 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.627 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.627 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.627 INFO:tasks.workunit.client.0.smithi043.stdout:534/600: open fd = 3 2024-04-25T10:43:34.627 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.627 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.627 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.627 INFO:tasks.workunit.client.0.smithi043.stdout:535/600: open fd = 3 2024-04-25T10:43:34.627 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.627 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.627 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.627 INFO:tasks.workunit.client.0.smithi043.stdout:536/600: open fd = 3 2024-04-25T10:43:34.627 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.627 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.627 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.628 INFO:tasks.workunit.client.0.smithi043.stdout:537/600: open fd = 3 2024-04-25T10:43:34.628 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.628 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.628 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.628 INFO:tasks.workunit.client.0.smithi043.stdout:538/600: open fd = 3 2024-04-25T10:43:34.628 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.628 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.628 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.628 INFO:tasks.workunit.client.0.smithi043.stdout:539/600: open fd = 3 2024-04-25T10:43:34.628 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.628 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.628 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.628 INFO:tasks.workunit.client.0.smithi043.stdout:540/600: open fd = 3 2024-04-25T10:43:34.628 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.629 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.629 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.629 INFO:tasks.workunit.client.0.smithi043.stdout:541/600: open fd = 3 2024-04-25T10:43:34.629 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.629 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.629 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.629 INFO:tasks.workunit.client.0.smithi043.stdout:542/600: open fd = 3 2024-04-25T10:43:34.629 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.629 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.629 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.629 INFO:tasks.workunit.client.0.smithi043.stdout:543/600: open fd = 3 2024-04-25T10:43:34.629 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.629 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.629 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.629 INFO:tasks.workunit.client.0.smithi043.stdout:544/600: open fd = 3 2024-04-25T10:43:34.630 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.630 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.630 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.630 INFO:tasks.workunit.client.0.smithi043.stdout:545/600: open fd = 3 2024-04-25T10:43:34.630 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.630 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.630 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.630 INFO:tasks.workunit.client.0.smithi043.stdout:546/600: open fd = 3 2024-04-25T10:43:34.630 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.630 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.631 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.631 INFO:tasks.workunit.client.0.smithi043.stdout:547/600: open fd = 3 2024-04-25T10:43:34.631 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.631 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.631 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.631 INFO:tasks.workunit.client.0.smithi043.stdout:548/600: open fd = 3 2024-04-25T10:43:34.631 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.631 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.631 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.631 INFO:tasks.workunit.client.0.smithi043.stdout:549/600: open fd = 3 2024-04-25T10:43:34.631 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.631 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.631 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.631 INFO:tasks.workunit.client.0.smithi043.stdout:550/600: open fd = 3 2024-04-25T10:43:34.631 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.632 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.632 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.632 INFO:tasks.workunit.client.0.smithi043.stdout:551/600: open fd = 3 2024-04-25T10:43:34.632 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.632 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.632 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.632 INFO:tasks.workunit.client.0.smithi043.stdout:552/600: open fd = 3 2024-04-25T10:43:34.632 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.632 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.632 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.632 INFO:tasks.workunit.client.0.smithi043.stdout:553/600: open fd = 3 2024-04-25T10:43:34.632 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.632 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.632 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.633 INFO:tasks.workunit.client.0.smithi043.stdout:554/600: open fd = 3 2024-04-25T10:43:34.633 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.633 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.633 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.633 INFO:tasks.workunit.client.0.smithi043.stdout:555/600: open fd = 3 2024-04-25T10:43:34.633 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.633 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.633 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.633 INFO:tasks.workunit.client.0.smithi043.stdout:556/600: open fd = 3 2024-04-25T10:43:34.633 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.633 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.633 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.633 INFO:tasks.workunit.client.0.smithi043.stdout:557/600: open fd = 3 2024-04-25T10:43:34.633 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.633 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.634 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.634 INFO:tasks.workunit.client.0.smithi043.stdout:558/600: open fd = 3 2024-04-25T10:43:34.634 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.634 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.634 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.634 INFO:tasks.workunit.client.0.smithi043.stdout:559/600: open fd = 3 2024-04-25T10:43:34.634 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.634 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.634 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:34.634 INFO:tasks.workunit.client.0.smithi043.stdout:560/600: open fd = 3 2024-04-25T10:43:34.634 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.634 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:43:34.634 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:43:36.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:36 smithi077 ceph-mon[32772]: pgmap v345: 97 pgs: 97 active+clean; 1.5 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-04-25T10:43:36.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:36 smithi164 ceph-mon[32713]: pgmap v345: 97 pgs: 97 active+clean; 1.5 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-04-25T10:43:36.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:36 smithi043 ceph-mon[29478]: pgmap v345: 97 pgs: 97 active+clean; 1.5 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-04-25T10:43:38.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:38 smithi077 ceph-mon[32772]: pgmap v346: 97 pgs: 97 active+clean; 1.5 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-04-25T10:43:38.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:38 smithi164 ceph-mon[32713]: pgmap v346: 97 pgs: 97 active+clean; 1.5 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-04-25T10:43:38.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:38 smithi043 ceph-mon[29478]: pgmap v346: 97 pgs: 97 active+clean; 1.5 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-04-25T10:43:40.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:40 smithi077 ceph-mon[32772]: pgmap v347: 97 pgs: 97 active+clean; 1.7 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-04-25T10:43:40.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:40 smithi164 ceph-mon[32713]: pgmap v347: 97 pgs: 97 active+clean; 1.7 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-04-25T10:43:40.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:40 smithi043 ceph-mon[29478]: pgmap v347: 97 pgs: 97 active+clean; 1.7 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-04-25T10:43:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.775+0000 7f1dc4420640 1 -- 172.21.15.43:0/2626357789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dbc1042c0 msgr2=0x7f1dbc110690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.775+0000 7f1dc4420640 1 --2- 172.21.15.43:0/2626357789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dbc1042c0 0x7f1dbc110690 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f1db8009ca0 tx=0x7f1db802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.775+0000 7f1dc4420640 1 -- 172.21.15.43:0/2626357789 shutdown_connections 2024-04-25T10:43:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.775+0000 7f1dc4420640 1 --2- 172.21.15.43:0/2626357789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dbc10a010 0x7f1dbc103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.775+0000 7f1dc4420640 1 --2- 172.21.15.43:0/2626357789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dbc1042c0 0x7f1dbc110690 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.775+0000 7f1dc4420640 1 --2- 172.21.15.43:0/2626357789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dbc109b70 0x7f1dbc103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.775+0000 7f1dc4420640 1 -- 172.21.15.43:0/2626357789 >> 172.21.15.43:0/2626357789 conn(0x7f1dbc0ff210 msgr2=0x7f1dbc101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.775+0000 7f1dc4420640 1 -- 172.21.15.43:0/2626357789 shutdown_connections 2024-04-25T10:43:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.775+0000 7f1dc4420640 1 -- 172.21.15.43:0/2626357789 wait complete. 2024-04-25T10:43:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.776+0000 7f1dc4420640 1 Processor -- start 2024-04-25T10:43:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.776+0000 7f1dc4420640 1 -- start start 2024-04-25T10:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.776+0000 7f1dc4420640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dbc1042c0 0x7f1dbc19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.776+0000 7f1dc4420640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dbc109b70 0x7f1dbc19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.776+0000 7f1dc4420640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dbc10a010 0x7f1dbc19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.776+0000 7f1dc4420640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc113d80 con 0x7f1dbc1042c0 2024-04-25T10:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.776+0000 7f1dc4420640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc113c00 con 0x7f1dbc109b70 2024-04-25T10:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.776+0000 7f1dc4420640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc113f00 con 0x7f1dbc10a010 2024-04-25T10:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.777+0000 7f1dc21bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dbc1042c0 0x7f1dbc19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.777+0000 7f1dc21bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dbc1042c0 0x7f1dbc19a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39012/0 (socket says 172.21.15.43:39012) 2024-04-25T10:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.777+0000 7f1dc21bc640 1 -- 172.21.15.43:0/3915180867 learned_addr learned my addr 172.21.15.43:0/3915180867 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.777+0000 7f1dc19bb640 1 --2- 172.21.15.43:0/3915180867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dbc109b70 0x7f1dbc19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.777+0000 7f1dc29bd640 1 --2- 172.21.15.43:0/3915180867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dbc10a010 0x7f1dbc19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.777+0000 7f1dc21bc640 1 -- 172.21.15.43:0/3915180867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dbc10a010 msgr2=0x7f1dbc19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.777+0000 7f1dc21bc640 1 --2- 172.21.15.43:0/3915180867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dbc10a010 0x7f1dbc19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.777+0000 7f1dc21bc640 1 -- 172.21.15.43:0/3915180867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dbc109b70 msgr2=0x7f1dbc19acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.777+0000 7f1dc21bc640 1 --2- 172.21.15.43:0/3915180867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dbc109b70 0x7f1dbc19acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.777+0000 7f1dc21bc640 1 -- 172.21.15.43:0/3915180867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dbc19f5c0 con 0x7f1dbc1042c0 2024-04-25T10:43:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.777+0000 7f1dc21bc640 1 --2- 172.21.15.43:0/3915180867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dbc1042c0 0x7f1dbc19a760 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f1db400d740 tx=0x7f1db400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.777+0000 7f1db37fe640 1 -- 172.21.15.43:0/3915180867 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4017070 con 0x7f1dbc1042c0 2024-04-25T10:43:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.777+0000 7f1db37fe640 1 -- 172.21.15.43:0/3915180867 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db400de50 con 0x7f1dbc1042c0 2024-04-25T10:43:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.777+0000 7f1dc4420640 1 -- 172.21.15.43:0/3915180867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dbc19f8b0 con 0x7f1dbc1042c0 2024-04-25T10:43:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.777+0000 7f1db37fe640 1 -- 172.21.15.43:0/3915180867 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4004ce0 con 0x7f1dbc1042c0 2024-04-25T10:43:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.777+0000 7f1dc4420640 1 -- 172.21.15.43:0/3915180867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dbc19fdb0 con 0x7f1dbc1042c0 2024-04-25T10:43:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.780+0000 7f1db37fe640 1 -- 172.21.15.43:0/3915180867 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f1db4004e80 con 0x7f1dbc1042c0 2024-04-25T10:43:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.780+0000 7f1dcb3ae740 1 Processor -- start 2024-04-25T10:43:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.781+0000 7f1dcb3ae740 1 -- start start 2024-04-25T10:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.781+0000 7f1dcb3ae740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c842c1b0 0x5650c87cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.781+0000 7f1dcb3ae740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c84032f0 0x5650c87cc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.781+0000 7f1dcb3ae740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c840b4d0 0x5650c87cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.781+0000 7f1dcb3ae740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5650c8732460 con 0x5650c84032f0 2024-04-25T10:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.781+0000 7f1dcb3ae740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5650c86ec590 con 0x5650c840b4d0 2024-04-25T10:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.781+0000 7f1dcb3ae740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5650c872e420 con 0x5650c842c1b0 2024-04-25T10:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.781+0000 7f1dc21bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c84032f0 0x5650c87cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.781+0000 7f1dc21bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c84032f0 0x5650c87cc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39026/0 (socket says 172.21.15.43:39026) 2024-04-25T10:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.781+0000 7f1dc21bc640 1 -- 172.21.15.43:0/2314901723 learned_addr learned my addr 172.21.15.43:0/2314901723 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.781+0000 7f1db37fe640 1 --2- 172.21.15.43:0/3915180867 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1da80afc50 0x7f1da80b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.782+0000 7f1dc19bb640 1 --2- 172.21.15.43:0/2314901723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c840b4d0 0x5650c87cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.782+0000 7f1dc29bd640 1 --2- 172.21.15.43:0/2314901723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c842c1b0 0x5650c87cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.782+0000 7f1dc29bd640 1 --2- 172.21.15.43:0/3915180867 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1da80afc50 0x7f1da80b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.782+0000 7f1db37fe640 1 -- 172.21.15.43:0/3915180867 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f1db40dea10 con 0x7f1dbc1042c0 2024-04-25T10:43:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.782+0000 7f1dc21bc640 1 -- 172.21.15.43:0/2314901723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c842c1b0 msgr2=0x5650c87cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.782+0000 7f1dc21bc640 1 --2- 172.21.15.43:0/2314901723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c842c1b0 0x5650c87cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.782+0000 7f1dc21bc640 1 -- 172.21.15.43:0/2314901723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c840b4d0 msgr2=0x5650c87cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.782+0000 7f1dc21bc640 1 --2- 172.21.15.43:0/2314901723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c840b4d0 0x5650c87cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.782+0000 7f1dc21bc640 1 -- 172.21.15.43:0/2314901723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650c86f90d0 con 0x5650c84032f0 2024-04-25T10:43:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.782+0000 7f1dc29bd640 1 --2- 172.21.15.43:0/2314901723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c842c1b0 0x5650c87cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:43:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.782+0000 7f1dc19bb640 1 --2- 172.21.15.43:0/2314901723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c840b4d0 0x5650c87cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:43:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.782+0000 7f1dc21bc640 1 --2- 172.21.15.43:0/2314901723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c84032f0 0x5650c87cc030 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f1db40a68b0 tx=0x7f1db40a3310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58351020a6d2e23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.782+0000 7f1dc29bd640 1 --2- 172.21.15.43:0/3915180867 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1da80afc50 0x7f1da80b2160 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1db8004af0 tx=0x7f1db80095c0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.782+0000 7f1db0bfe640 1 -- 172.21.15.43:0/2314901723 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4020020 con 0x5650c84032f0 2024-04-25T10:43:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.782+0000 7f1db0bfe640 1 -- 172.21.15.43:0/2314901723 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db40a66c0 con 0x5650c84032f0 2024-04-25T10:43:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.782+0000 7f1db0bfe640 1 -- 172.21.15.43:0/2314901723 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db40c7e60 con 0x5650c84032f0 2024-04-25T10:43:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.783+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/2314901723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c84032f0 msgr2=0x5650c87cc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.783+0000 7f1dcb3ae740 1 --2- 172.21.15.43:0/2314901723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c84032f0 0x5650c87cc030 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f1db40a68b0 tx=0x7f1db40a3310 comp rx=0 tx=0).stop 2024-04-25T10:43:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.783+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/2314901723 shutdown_connections 2024-04-25T10:43:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.783+0000 7f1dcb3ae740 1 --2- 172.21.15.43:0/2314901723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c840b4d0 0x5650c87cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.783+0000 7f1dcb3ae740 1 --2- 172.21.15.43:0/2314901723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c84032f0 0x5650c87cc030 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.783+0000 7f1dcb3ae740 1 --2- 172.21.15.43:0/2314901723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c842c1b0 0x5650c87cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.783+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/2314901723 >> 172.21.15.43:0/2314901723 conn(0x5650c8420300 msgr2=0x5650c87c82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.783+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/2314901723 shutdown_connections 2024-04-25T10:43:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.783+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/2314901723 wait complete. 2024-04-25T10:43:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.784+0000 7f1dcb3ae740 1 -- start start 2024-04-25T10:43:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.784+0000 7f1dcb3ae740 1 Processor -- start 2024-04-25T10:43:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.784+0000 7f1dcb3ae740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c84032f0 0x5650c87d6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.784+0000 7f1dcb3ae740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c840b4d0 0x5650c87d6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.784+0000 7f1dcb3ae740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c842c1b0 0x5650c87dbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.784+0000 7f1dcb3ae740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5650c87d8b60 con 0x5650c840b4d0 2024-04-25T10:43:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.784+0000 7f1dcb3ae740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5650c86b9b60 con 0x5650c842c1b0 2024-04-25T10:43:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.784+0000 7f1dcb3ae740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5650c87d8ce0 con 0x5650c84032f0 2024-04-25T10:43:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.784+0000 7f1dcb3ae740 10 client.0 _ll_register_callbacks cb 0x5650c86de2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.784+0000 7f1dc19bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c840b4d0 0x5650c87d6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.784+0000 7f1dc19bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c840b4d0 0x5650c87d6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39028/0 (socket says 172.21.15.43:39028) 2024-04-25T10:43:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.784+0000 7f1dc19bb640 1 -- 172.21.15.43:0/1553963473 learned_addr learned my addr 172.21.15.43:0/1553963473 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.785+0000 7f1dc21bc640 1 --2- 172.21.15.43:0/1553963473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c84032f0 0x5650c87d6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.785+0000 7f1dc29bd640 1 --2- 172.21.15.43:0/1553963473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c842c1b0 0x5650c87dbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.785+0000 7f1dc19bb640 1 -- 172.21.15.43:0/1553963473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c84032f0 msgr2=0x5650c87d6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.785+0000 7f1dc19bb640 1 --2- 172.21.15.43:0/1553963473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c84032f0 0x5650c87d6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.785+0000 7f1dc19bb640 1 -- 172.21.15.43:0/1553963473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c842c1b0 msgr2=0x5650c87dbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.785+0000 7f1dc19bb640 1 --2- 172.21.15.43:0/1553963473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c842c1b0 0x5650c87dbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.785+0000 7f1dc19bb640 1 -- 172.21.15.43:0/1553963473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650c8444210 con 0x5650c840b4d0 2024-04-25T10:43:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.785+0000 7f1dc19bb640 1 --2- 172.21.15.43:0/1553963473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c840b4d0 0x5650c87d6680 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00bed0 tx=0x7f1dac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a87ee10da1e32d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.785+0000 7f1da5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.785+0000 7f1da5ffb640 1 -- 172.21.15.43:0/1553963473 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac004030 con 0x5650c840b4d0 2024-04-25T10:43:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.785+0000 7f1da5ffb640 1 -- 172.21.15.43:0/1553963473 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f1dac0041d0 con 0x5650c840b4d0 2024-04-25T10:43:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.785+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/1553963473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1db40a66c0 con 0x5650c840b4d0 2024-04-25T10:43:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.785+0000 7f1da5ffb640 1 -- 172.21.15.43:0/1553963473 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1dac0043d0 con 0x5650c840b4d0 2024-04-25T10:43:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.786+0000 7f1dcb3ae740 10 client.18816 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.786+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/1553963473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5650c8510620 con 0x5650c840b4d0 2024-04-25T10:43:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.786+0000 7f1da5ffb640 1 -- 172.21.15.43:0/1553963473 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db40a66c0 con 0x5650c840b4d0 2024-04-25T10:43:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.786+0000 7f1da5ffb640 1 -- 172.21.15.43:0/1553963473 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1dac003890 con 0x5650c840b4d0 2024-04-25T10:43:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.786+0000 7f1dcb3ae740 10 client.18816 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.786+0000 7f1dcb3ae740 10 client.18816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:43:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.786+0000 7f1dcb3ae740 20 client.18816 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.786+0000 7f1dcb3ae740 1 --2- 172.21.15.43:0/1553963473 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5650c83ead80 0x5650c87e85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.786+0000 7f1dcb3ae740 4 client.18816 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:43:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.786+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/1553963473 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650c8740070 con 0x5650c83ead80 2024-04-25T10:43:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.786+0000 7f1dc21bc640 1 --2- 172.21.15.43:0/1553963473 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5650c83ead80 0x5650c87e85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.787+0000 7f1da5ffb640 10 client.18816 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.787+0000 7f1dc21bc640 1 --2- 172.21.15.43:0/1553963473 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5650c83ead80 0x5650c87e85e0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c05b503654978af server_cookie=6df756c65ae861ec in_seq=0 out_seq=0 2024-04-25T10:43:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.788+0000 7f1da5ffb640 1 -- 172.21.15.43:0/1553963473 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f1db4055e50 con 0x5650c83ead80 2024-04-25T10:43:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.788+0000 7f1da5ffb640 10 client.18816 handle_command_reply: tid=0 2024-04-25T10:43:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.788+0000 7f1dcb3ae740 1 client.18816 shutdown 2024-04-25T10:43:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.788+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/1553963473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c840b4d0 msgr2=0x5650c87d6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.788+0000 7f1dcb3ae740 1 --2- 172.21.15.43:0/1553963473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c840b4d0 0x5650c87d6680 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00bed0 tx=0x7f1dac011810 comp rx=0 tx=0).stop 2024-04-25T10:43:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.788+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/1553963473 shutdown_connections 2024-04-25T10:43:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.788+0000 7f1dcb3ae740 1 --2- 172.21.15.43:0/1553963473 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5650c83ead80 0x5650c87e85e0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.788+0000 7f1dcb3ae740 1 --2- 172.21.15.43:0/1553963473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c842c1b0 0x5650c87dbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.788+0000 7f1dcb3ae740 1 --2- 172.21.15.43:0/1553963473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c840b4d0 0x5650c87d6680 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.788+0000 7f1da5ffb640 0 client.18816 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.788+0000 7f1dcb3ae740 1 --2- 172.21.15.43:0/1553963473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c84032f0 0x5650c87d6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.788+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/1553963473 >> 172.21.15.43:0/1553963473 conn(0x5650c8420300 msgr2=0x5650c87cb920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.788+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/1553963473 shutdown_connections 2024-04-25T10:43:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.788+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/1553963473 wait complete. 2024-04-25T10:43:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.788+0000 7f1dcb3ae740 20 client.18816 trim_cache size 0 max 16384 2024-04-25T10:43:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.803+0000 7f1dcb3ae740 1 Processor -- start 2024-04-25T10:43:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.803+0000 7f1dcb3ae740 1 -- start start 2024-04-25T10:43:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.804+0000 7f1dcb3ae740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c84032f0 0x5650c87dbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.804+0000 7f1dcb3ae740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c840b4d0 0x5650c87c8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.804+0000 7f1dcb3ae740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c842c1b0 0x5650c87eddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.804+0000 7f1dcb3ae740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1b2390 con 0x5650c84032f0 2024-04-25T10:43:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.804+0000 7f1dcb3ae740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1afc60 con 0x5650c840b4d0 2024-04-25T10:43:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.804+0000 7f1dcb3ae740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1adad0 con 0x5650c842c1b0 2024-04-25T10:43:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.804+0000 7f1dc21bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c84032f0 0x5650c87dbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.804+0000 7f1dc21bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c84032f0 0x5650c87dbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39038/0 (socket says 172.21.15.43:39038) 2024-04-25T10:43:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.804+0000 7f1dc21bc640 1 -- 172.21.15.43:0/3627508778 learned_addr learned my addr 172.21.15.43:0/3627508778 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.804+0000 7f1dc19bb640 1 --2- 172.21.15.43:0/3627508778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c840b4d0 0x5650c87c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.804+0000 7f1dc29bd640 1 --2- 172.21.15.43:0/3627508778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c842c1b0 0x5650c87eddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.804+0000 7f1dc21bc640 1 -- 172.21.15.43:0/3627508778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c842c1b0 msgr2=0x5650c87eddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.804+0000 7f1dc21bc640 1 --2- 172.21.15.43:0/3627508778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c842c1b0 0x5650c87eddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.804+0000 7f1dc21bc640 1 -- 172.21.15.43:0/3627508778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c840b4d0 msgr2=0x5650c87c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.804+0000 7f1dc21bc640 1 --2- 172.21.15.43:0/3627508778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c840b4d0 0x5650c87c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.804+0000 7f1dc21bc640 1 -- 172.21.15.43:0/3627508778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650c87cc8a0 con 0x5650c84032f0 2024-04-25T10:43:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.804+0000 7f1dc19bb640 1 --2- 172.21.15.43:0/3627508778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c840b4d0 0x5650c87c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:43:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.804+0000 7f1dc21bc640 1 --2- 172.21.15.43:0/3627508778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c84032f0 0x5650c87dbf50 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f1db40dc210 tx=0x7f1db40c9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9583eb6db9a1123 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.804+0000 7f1da7fff640 1 -- 172.21.15.43:0/3627508778 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4020040 con 0x5650c84032f0 2024-04-25T10:43:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.805+0000 7f1da7fff640 1 -- 172.21.15.43:0/3627508778 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db4055e50 con 0x5650c84032f0 2024-04-25T10:43:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.805+0000 7f1da7fff640 1 -- 172.21.15.43:0/3627508778 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db40c7e60 con 0x5650c84032f0 2024-04-25T10:43:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.805+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/3627508778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c84032f0 msgr2=0x5650c87dbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.805+0000 7f1dcb3ae740 1 --2- 172.21.15.43:0/3627508778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c84032f0 0x5650c87dbf50 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f1db40dc210 tx=0x7f1db40c9040 comp rx=0 tx=0).stop 2024-04-25T10:43:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.805+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/3627508778 shutdown_connections 2024-04-25T10:43:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.805+0000 7f1dcb3ae740 1 --2- 172.21.15.43:0/3627508778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c840b4d0 0x5650c87c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.805+0000 7f1dcb3ae740 1 --2- 172.21.15.43:0/3627508778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c842c1b0 0x5650c87eddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.805+0000 7f1dcb3ae740 1 --2- 172.21.15.43:0/3627508778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c84032f0 0x5650c87dbf50 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.805+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/3627508778 >> 172.21.15.43:0/3627508778 conn(0x5650c8420300 msgr2=0x5650c87c7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.805+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/3627508778 shutdown_connections 2024-04-25T10:43:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.805+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/3627508778 wait complete. 2024-04-25T10:43:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.806+0000 7f1dcb3ae740 1 -- start start 2024-04-25T10:43:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.806+0000 7f1dcb3ae740 1 Processor -- start 2024-04-25T10:43:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.806+0000 7f1dcb3ae740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c84032f0 0x5650c87de480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.806+0000 7f1dcb3ae740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c840b4d0 0x5650c87de9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.806+0000 7f1dcb3ae740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c842c1b0 0x5650c87def00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.806+0000 7f1dcb3ae740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1ae120 con 0x5650c840b4d0 2024-04-25T10:43:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.806+0000 7f1dcb3ae740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1af590 con 0x5650c842c1b0 2024-04-25T10:43:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.806+0000 7f1dcb3ae740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1ae7f0 con 0x5650c84032f0 2024-04-25T10:43:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.806+0000 7f1dcb3ae740 10 client.0 _ll_register_callbacks cb 0x7f1dac00b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.806+0000 7f1dc19bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c840b4d0 0x5650c87de9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.806+0000 7f1dc19bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c840b4d0 0x5650c87de9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39050/0 (socket says 172.21.15.43:39050) 2024-04-25T10:43:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.806+0000 7f1dc19bb640 1 -- 172.21.15.43:0/1405190286 learned_addr learned my addr 172.21.15.43:0/1405190286 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.806+0000 7f1dc29bd640 1 --2- 172.21.15.43:0/1405190286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c842c1b0 0x5650c87def00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.806+0000 7f1dc21bc640 1 --2- 172.21.15.43:0/1405190286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c84032f0 0x5650c87de480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.806+0000 7f1dc19bb640 1 -- 172.21.15.43:0/1405190286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c84032f0 msgr2=0x5650c87de480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.806+0000 7f1dc19bb640 1 --2- 172.21.15.43:0/1405190286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c84032f0 0x5650c87de480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.807+0000 7f1dc19bb640 1 -- 172.21.15.43:0/1405190286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c842c1b0 msgr2=0x5650c87def00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.807+0000 7f1dc19bb640 1 --2- 172.21.15.43:0/1405190286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c842c1b0 0x5650c87def00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.807+0000 7f1dc19bb640 1 -- 172.21.15.43:0/1405190286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650c884f970 con 0x5650c840b4d0 2024-04-25T10:43:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.807+0000 7f1dc21bc640 1 --2- 172.21.15.43:0/1405190286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c84032f0 0x5650c87de480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:43:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.807+0000 7f1dc19bb640 1 --2- 172.21.15.43:0/1405190286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c840b4d0 0x5650c87de9c0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x5650c87d7500 tx=0x7f1dac00ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9936464d8fac639a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.807+0000 7f1da5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.807+0000 7f1da5ffb640 1 -- 172.21.15.43:0/1405190286 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac0035f0 con 0x5650c840b4d0 2024-04-25T10:43:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.807+0000 7f1da5ffb640 1 -- 172.21.15.43:0/1405190286 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f1dac003790 con 0x5650c840b4d0 2024-04-25T10:43:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.807+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/1405190286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1db4055e50 con 0x5650c840b4d0 2024-04-25T10:43:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.807+0000 7f1da5ffb640 1 -- 172.21.15.43:0/1405190286 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1dac039050 con 0x5650c840b4d0 2024-04-25T10:43:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.807+0000 7f1dcb3ae740 10 client.18822 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.807+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/1405190286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5650c8850a40 con 0x5650c840b4d0 2024-04-25T10:43:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.807+0000 7f1da5ffb640 1 -- 172.21.15.43:0/1405190286 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db4055e50 con 0x5650c840b4d0 2024-04-25T10:43:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.808+0000 7f1da5ffb640 1 -- 172.21.15.43:0/1405190286 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1dac034030 con 0x5650c840b4d0 2024-04-25T10:43:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.808+0000 7f1dcb3ae740 10 client.18822 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.808+0000 7f1dcb3ae740 10 client.18822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:43:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.808+0000 7f1dcb3ae740 20 client.18822 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.808+0000 7f1dcb3ae740 1 --2- 172.21.15.43:0/1405190286 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5650c83c01c0 0x5650c8853250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.808+0000 7f1dcb3ae740 4 client.18822 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:43:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.808+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/1405190286 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5650c8739ee0 con 0x5650c83c01c0 2024-04-25T10:43:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.808+0000 7f1dc21bc640 1 --2- 172.21.15.43:0/1405190286 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5650c83c01c0 0x5650c8853250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.808+0000 7f1da5ffb640 10 client.18822 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.808+0000 7f1dc21bc640 1 --2- 172.21.15.43:0/1405190286 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5650c83c01c0 0x5650c8853250 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e22c8b562c3102fe server_cookie=a7ee1605e019e788 in_seq=0 out_seq=0 2024-04-25T10:43:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.824+0000 7f1da5ffb640 1 -- 172.21.15.43:0/1405190286 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5650c8739ee0 con 0x5650c83c01c0 2024-04-25T10:43:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.824+0000 7f1da5ffb640 10 client.18822 handle_command_reply: tid=0 2024-04-25T10:43:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.824+0000 7f1dcb3ae740 1 client.18822 shutdown 2024-04-25T10:43:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.824+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/1405190286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c840b4d0 msgr2=0x5650c87de9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.824+0000 7f1dcb3ae740 1 --2- 172.21.15.43:0/1405190286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c840b4d0 0x5650c87de9c0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x5650c87d7500 tx=0x7f1dac00ba20 comp rx=0 tx=0).stop 2024-04-25T10:43:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.824+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/1405190286 shutdown_connections 2024-04-25T10:43:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.824+0000 7f1dcb3ae740 1 --2- 172.21.15.43:0/1405190286 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5650c83c01c0 0x5650c8853250 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.824+0000 7f1dcb3ae740 1 --2- 172.21.15.43:0/1405190286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c842c1b0 0x5650c87def00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.824+0000 7f1dcb3ae740 1 --2- 172.21.15.43:0/1405190286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650c840b4d0 0x5650c87de9c0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.824+0000 7f1da5ffb640 0 client.18822 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.824+0000 7f1dcb3ae740 1 --2- 172.21.15.43:0/1405190286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650c84032f0 0x5650c87de480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.824+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/1405190286 >> 172.21.15.43:0/1405190286 conn(0x5650c8420300 msgr2=0x5650c87d4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.824+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/1405190286 shutdown_connections 2024-04-25T10:43:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.824+0000 7f1dcb3ae740 1 -- 172.21.15.43:0/1405190286 wait complete. 2024-04-25T10:43:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.825+0000 7f1dcb3ae740 20 client.18822 trim_cache size 0 max 16384 2024-04-25T10:43:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:43:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:43:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:43:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:43:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.825+0000 7f1dc4420640 1 -- 172.21.15.43:0/3915180867 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1da80afc50 msgr2=0x7f1da80b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.825+0000 7f1dc4420640 1 --2- 172.21.15.43:0/3915180867 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1da80afc50 0x7f1da80b2160 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1db8004af0 tx=0x7f1db80095c0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.825+0000 7f1dc4420640 1 -- 172.21.15.43:0/3915180867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dbc1042c0 msgr2=0x7f1dbc19a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.825+0000 7f1dc4420640 1 --2- 172.21.15.43:0/3915180867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dbc1042c0 0x7f1dbc19a760 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f1db400d740 tx=0x7f1db400dc00 comp rx=0 tx=0).stop 2024-04-25T10:43:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.825+0000 7f1dc4420640 1 -- 172.21.15.43:0/3915180867 shutdown_connections 2024-04-25T10:43:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.825+0000 7f1dc4420640 1 --2- 172.21.15.43:0/3915180867 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1da80afc50 0x7f1da80b2160 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.825+0000 7f1dc4420640 1 --2- 172.21.15.43:0/3915180867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dbc109b70 0x7f1dbc19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.825+0000 7f1dc4420640 1 --2- 172.21.15.43:0/3915180867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dbc10a010 0x7f1dbc19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.825+0000 7f1dc4420640 1 --2- 172.21.15.43:0/3915180867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dbc1042c0 0x7f1dbc19a760 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.825+0000 7f1dc4420640 1 -- 172.21.15.43:0/3915180867 >> 172.21.15.43:0/3915180867 conn(0x7f1dbc0ff210 msgr2=0x7f1dbc106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.825+0000 7f1dc4420640 1 -- 172.21.15.43:0/3915180867 shutdown_connections 2024-04-25T10:43:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:41.825+0000 7f1dc4420640 1 -- 172.21.15.43:0/3915180867 wait complete. 2024-04-25T10:43:42.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:42 smithi077 ceph-mon[32772]: pgmap v348: 97 pgs: 97 active+clean; 1.7 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-04-25T10:43:42.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:42 smithi164 ceph-mon[32713]: pgmap v348: 97 pgs: 97 active+clean; 1.7 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-04-25T10:43:42.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:42 smithi043 ceph-mon[29478]: pgmap v348: 97 pgs: 97 active+clean; 1.7 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-04-25T10:43:44.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:44 smithi043 ceph-mon[29478]: pgmap v349: 97 pgs: 97 active+clean; 1.7 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-04-25T10:43:44.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:44 smithi077 ceph-mon[32772]: pgmap v349: 97 pgs: 97 active+clean; 1.7 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-04-25T10:43:44.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:44 smithi164 ceph-mon[32713]: pgmap v349: 97 pgs: 97 active+clean; 1.7 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-04-25T10:43:46.343 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:46 smithi077 ceph-mon[32772]: pgmap v350: 97 pgs: 97 active+clean; 1.9 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 66 KiB/s wr, 21 op/s 2024-04-25T10:43:46.343 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:46 smithi164 ceph-mon[32713]: pgmap v350: 97 pgs: 97 active+clean; 1.9 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 66 KiB/s wr, 21 op/s 2024-04-25T10:43:46.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:46 smithi043 ceph-mon[29478]: pgmap v350: 97 pgs: 97 active+clean; 1.9 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 66 KiB/s wr, 21 op/s 2024-04-25T10:43:48.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:48 smithi043 ceph-mon[29478]: pgmap v351: 97 pgs: 97 active+clean; 1.9 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:43:48.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:48 smithi077 ceph-mon[32772]: pgmap v351: 97 pgs: 97 active+clean; 1.9 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:43:48.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:48 smithi164 ceph-mon[32713]: pgmap v351: 97 pgs: 97 active+clean; 1.9 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:43:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:50 smithi043 ceph-mon[29478]: pgmap v352: 97 pgs: 97 active+clean; 2.1 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 63 KiB/s wr, 17 op/s 2024-04-25T10:43:50.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:50 smithi077 ceph-mon[32772]: pgmap v352: 97 pgs: 97 active+clean; 2.1 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 63 KiB/s wr, 17 op/s 2024-04-25T10:43:50.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:50 smithi164 ceph-mon[32713]: pgmap v352: 97 pgs: 97 active+clean; 2.1 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 63 KiB/s wr, 17 op/s 2024-04-25T10:43:52.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:52 smithi043 ceph-mon[29478]: pgmap v353: 97 pgs: 97 active+clean; 2.1 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:43:52.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:52 smithi077 ceph-mon[32772]: pgmap v353: 97 pgs: 97 active+clean; 2.1 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:43:52.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:52 smithi164 ceph-mon[32713]: pgmap v353: 97 pgs: 97 active+clean; 2.1 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:43:53.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:53 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:43:53.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:53 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:43:53.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:53 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:43:53.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:53 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:43:53.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:53 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:43:53.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:53 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:43:53.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:53 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:43:53.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:53 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:43:53.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:53 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:43:54.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:54 smithi043 ceph-mon[29478]: pgmap v354: 97 pgs: 97 active+clean; 2.1 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:43:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:54 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:43:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:54 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:43:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:54 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:43:54.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:54 smithi077 ceph-mon[32772]: pgmap v354: 97 pgs: 97 active+clean; 2.1 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:43:54.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:54 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:43:54.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:54 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:43:54.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:54 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:43:54.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:54 smithi164 ceph-mon[32713]: pgmap v354: 97 pgs: 97 active+clean; 2.1 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-04-25T10:43:54.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:54 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:43:54.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:54 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:43:54.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:54 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:43:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.937+0000 7fd1d9877640 1 -- 172.21.15.43:0/1140594398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1d4109ff0 msgr2=0x7fd1d4103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.937+0000 7fd1d9877640 1 --2- 172.21.15.43:0/1140594398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1d4109ff0 0x7fd1d4103c30 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009900 tx=0x7fd1c802a410 comp rx=0 tx=0).stop 2024-04-25T10:43:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.937+0000 7fd1d9877640 1 -- 172.21.15.43:0/1140594398 shutdown_connections 2024-04-25T10:43:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.937+0000 7fd1d9877640 1 --2- 172.21.15.43:0/1140594398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1d41042a0 0x7fd1d4110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.937+0000 7fd1d9877640 1 --2- 172.21.15.43:0/1140594398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1d4109ff0 0x7fd1d4103c30 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.937+0000 7fd1d9877640 1 --2- 172.21.15.43:0/1140594398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d4109b50 0x7fd1d41036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.937+0000 7fd1d9877640 1 -- 172.21.15.43:0/1140594398 >> 172.21.15.43:0/1140594398 conn(0x7fd1d40ff230 msgr2=0x7fd1d4101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.937+0000 7fd1d9877640 1 -- 172.21.15.43:0/1140594398 shutdown_connections 2024-04-25T10:43:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.937+0000 7fd1d9877640 1 -- 172.21.15.43:0/1140594398 wait complete. 2024-04-25T10:43:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.937+0000 7fd1d9877640 1 Processor -- start 2024-04-25T10:43:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.938+0000 7fd1d9877640 1 -- start start 2024-04-25T10:43:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.938+0000 7fd1d9877640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1d41042a0 0x7fd1d419d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.938+0000 7fd1d9877640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d4109b50 0x7fd1d419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.938+0000 7fd1d9877640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1d4109ff0 0x7fd1d41a8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.938+0000 7fd1d9877640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d4113d40 con 0x7fd1d41042a0 2024-04-25T10:43:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.938+0000 7fd1d9877640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d4113bc0 con 0x7fd1d4109ff0 2024-04-25T10:43:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.938+0000 7fd1d9877640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d4113ec0 con 0x7fd1d4109b50 2024-04-25T10:43:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.938+0000 7fd1d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1d41042a0 0x7fd1d419d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.938+0000 7fd1d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1d41042a0 0x7fd1d419d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49708/0 (socket says 172.21.15.43:49708) 2024-04-25T10:43:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.938+0000 7fd1d2ffd640 1 -- 172.21.15.43:0/34715052 learned_addr learned my addr 172.21.15.43:0/34715052 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.938+0000 7fd1d37fe640 1 --2- 172.21.15.43:0/34715052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1d4109ff0 0x7fd1d41a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.938+0000 7fd1d27fc640 1 --2- 172.21.15.43:0/34715052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d4109b50 0x7fd1d419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.938+0000 7fd1d2ffd640 1 -- 172.21.15.43:0/34715052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d4109b50 msgr2=0x7fd1d419dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.938+0000 7fd1d2ffd640 1 --2- 172.21.15.43:0/34715052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d4109b50 0x7fd1d419dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.938+0000 7fd1d2ffd640 1 -- 172.21.15.43:0/34715052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1d4109ff0 msgr2=0x7fd1d41a8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.938+0000 7fd1d2ffd640 1 --2- 172.21.15.43:0/34715052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1d4109ff0 0x7fd1d41a8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.938+0000 7fd1d2ffd640 1 -- 172.21.15.43:0/34715052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d41a8c10 con 0x7fd1d41042a0 2024-04-25T10:43:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.938+0000 7fd1d27fc640 1 --2- 172.21.15.43:0/34715052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d4109b50 0x7fd1d419dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:43:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.939+0000 7fd1d2ffd640 1 --2- 172.21.15.43:0/34715052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1d41042a0 0x7fd1d419d7e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400c9a0 tx=0x7fd1c400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.939+0000 7fd1d8875640 1 -- 172.21.15.43:0/34715052 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4007bb0 con 0x7fd1d41042a0 2024-04-25T10:43:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.939+0000 7fd1d8875640 1 -- 172.21.15.43:0/34715052 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1c4007d50 con 0x7fd1d41042a0 2024-04-25T10:43:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.939+0000 7fd1d9877640 1 -- 172.21.15.43:0/34715052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d41a8ea0 con 0x7fd1d41042a0 2024-04-25T10:43:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.939+0000 7fd1d8875640 1 -- 172.21.15.43:0/34715052 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c400f450 con 0x7fd1d41042a0 2024-04-25T10:43:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.939+0000 7fd1d9877640 1 -- 172.21.15.43:0/34715052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d41a9310 con 0x7fd1d41042a0 2024-04-25T10:43:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.942+0000 7fd1d8875640 1 -- 172.21.15.43:0/34715052 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fd1c40046c0 con 0x7fd1d41042a0 2024-04-25T10:43:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.942+0000 7fd1e0805740 1 Processor -- start 2024-04-25T10:43:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.942+0000 7fd1e0805740 1 -- start start 2024-04-25T10:43:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.942+0000 7fd1e0805740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f778741b0 0x561f77c13af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.942+0000 7fd1e0805740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f7784b2f0 0x561f77c14030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.942+0000 7fd1e0805740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f778534d0 0x561f77c14570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.942+0000 7fd1e0805740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561f77b7a460 con 0x561f7784b2f0 2024-04-25T10:43:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.942+0000 7fd1e0805740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561f77b34590 con 0x561f778741b0 2024-04-25T10:43:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.942+0000 7fd1e0805740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561f77b76420 con 0x561f778534d0 2024-04-25T10:43:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.942+0000 7fd1d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f7784b2f0 0x561f77c14030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.942+0000 7fd1d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f7784b2f0 0x561f77c14030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49718/0 (socket says 172.21.15.43:49718) 2024-04-25T10:43:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.942+0000 7fd1d2ffd640 1 -- 172.21.15.43:0/3668055005 learned_addr learned my addr 172.21.15.43:0/3668055005 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.942+0000 7fd1d37fe640 1 --2- 172.21.15.43:0/3668055005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f778741b0 0x561f77c13af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.942+0000 7fd1d27fc640 1 --2- 172.21.15.43:0/3668055005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f778534d0 0x561f77c14570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.943+0000 7fd1d8875640 1 --2- 172.21.15.43:0/34715052 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd1bc0afc50 0x7fd1bc0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.943+0000 7fd1d2ffd640 1 -- 172.21.15.43:0/3668055005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f778534d0 msgr2=0x561f77c14570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.943+0000 7fd1d2ffd640 1 --2- 172.21.15.43:0/3668055005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f778534d0 0x561f77c14570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.943+0000 7fd1d2ffd640 1 -- 172.21.15.43:0/3668055005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f778741b0 msgr2=0x561f77c13af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.943+0000 7fd1d2ffd640 1 --2- 172.21.15.43:0/3668055005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f778741b0 0x561f77c13af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.943+0000 7fd1d2ffd640 1 -- 172.21.15.43:0/3668055005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f77b410d0 con 0x561f7784b2f0 2024-04-25T10:43:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.943+0000 7fd1d27fc640 1 --2- 172.21.15.43:0/3668055005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f778534d0 0x561f77c14570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:43:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.943+0000 7fd1d37fe640 1 --2- 172.21.15.43:0/34715052 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd1bc0afc50 0x7fd1bc0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.943+0000 7fd1d8875640 1 -- 172.21.15.43:0/34715052 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fd1c40df010 con 0x7fd1d41042a0 2024-04-25T10:43:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.943+0000 7fd1d2ffd640 1 --2- 172.21.15.43:0/3668055005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f7784b2f0 0x561f77c14030 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40dc6d0 tx=0x7fd1c4074a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec21002aa4a39021 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.943+0000 7fd1b9ffb640 1 -- 172.21.15.43:0/3668055005 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c401d070 con 0x561f7784b2f0 2024-04-25T10:43:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.943+0000 7fd1b9ffb640 1 -- 172.21.15.43:0/3668055005 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1c40a6cc0 con 0x561f7784b2f0 2024-04-25T10:43:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.943+0000 7fd1b9ffb640 1 -- 172.21.15.43:0/3668055005 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c40a6b00 con 0x561f7784b2f0 2024-04-25T10:43:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.944+0000 7fd1e0805740 1 -- 172.21.15.43:0/3668055005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f7784b2f0 msgr2=0x561f77c14030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.944+0000 7fd1e0805740 1 --2- 172.21.15.43:0/3668055005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f7784b2f0 0x561f77c14030 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40dc6d0 tx=0x7fd1c4074a10 comp rx=0 tx=0).stop 2024-04-25T10:43:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.944+0000 7fd1d37fe640 1 --2- 172.21.15.43:0/34715052 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd1bc0afc50 0x7fd1bc0b2160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x561f77c10c60 tx=0x7fd1cc004a40 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.944+0000 7fd1e0805740 1 -- 172.21.15.43:0/3668055005 shutdown_connections 2024-04-25T10:43:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.944+0000 7fd1e0805740 1 --2- 172.21.15.43:0/3668055005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f778534d0 0x561f77c14570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.944+0000 7fd1e0805740 1 --2- 172.21.15.43:0/3668055005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f7784b2f0 0x561f77c14030 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.944+0000 7fd1e0805740 1 --2- 172.21.15.43:0/3668055005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f778741b0 0x561f77c13af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.944+0000 7fd1e0805740 1 -- 172.21.15.43:0/3668055005 >> 172.21.15.43:0/3668055005 conn(0x561f77868300 msgr2=0x561f77c102a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.944+0000 7fd1e0805740 1 -- 172.21.15.43:0/3668055005 shutdown_connections 2024-04-25T10:43:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.944+0000 7fd1e0805740 1 -- 172.21.15.43:0/3668055005 wait complete. 2024-04-25T10:43:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.945+0000 7fd1e0805740 1 -- start start 2024-04-25T10:43:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.945+0000 7fd1e0805740 1 Processor -- start 2024-04-25T10:43:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.945+0000 7fd1e0805740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f7784b2f0 0x561f77c1e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.945+0000 7fd1e0805740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f778534d0 0x561f77c1e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.945+0000 7fd1e0805740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f778741b0 0x561f77c2a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.945+0000 7fd1e0805740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561f77c20bc0 con 0x561f7784b2f0 2024-04-25T10:43:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.945+0000 7fd1e0805740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561f77b01b60 con 0x561f778534d0 2024-04-25T10:43:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.945+0000 7fd1e0805740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561f77c20d40 con 0x561f778741b0 2024-04-25T10:43:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.945+0000 7fd1e0805740 10 client.0 _ll_register_callbacks cb 0x561f77b262b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.945+0000 7fd1d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f7784b2f0 0x561f77c1e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.945+0000 7fd1d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f7784b2f0 0x561f77c1e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49724/0 (socket says 172.21.15.43:49724) 2024-04-25T10:43:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.945+0000 7fd1d2ffd640 1 -- 172.21.15.43:0/3372560088 learned_addr learned my addr 172.21.15.43:0/3372560088 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.946+0000 7fd1d27fc640 1 --2- 172.21.15.43:0/3372560088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f778534d0 0x561f77c1e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.946+0000 7fd1d37fe640 1 --2- 172.21.15.43:0/3372560088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f778741b0 0x561f77c2a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.946+0000 7fd1d2ffd640 1 -- 172.21.15.43:0/3372560088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f778741b0 msgr2=0x561f77c2a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.946+0000 7fd1d2ffd640 1 --2- 172.21.15.43:0/3372560088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f778741b0 0x561f77c2a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.946+0000 7fd1d2ffd640 1 -- 172.21.15.43:0/3372560088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f778534d0 msgr2=0x561f77c1e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.946+0000 7fd1d2ffd640 1 --2- 172.21.15.43:0/3372560088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f778534d0 0x561f77c1e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.946+0000 7fd1d2ffd640 1 -- 172.21.15.43:0/3372560088 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f7788c210 con 0x561f7784b2f0 2024-04-25T10:43:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.946+0000 7fd1d2ffd640 1 --2- 172.21.15.43:0/3372560088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f7784b2f0 0x561f77c1e110 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40dc810 tx=0x7fd1c40058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4e2afc8e99fe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.946+0000 7fd1b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.946+0000 7fd1b2ffd640 1 -- 172.21.15.43:0/3372560088 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c401d070 con 0x561f7784b2f0 2024-04-25T10:43:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.946+0000 7fd1b2ffd640 1 -- 172.21.15.43:0/3372560088 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fd1c40ab120 con 0x561f7784b2f0 2024-04-25T10:43:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.946+0000 7fd1e0805740 1 -- 172.21.15.43:0/3372560088 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1c40a6cc0 con 0x561f7784b2f0 2024-04-25T10:43:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.946+0000 7fd1b2ffd640 1 -- 172.21.15.43:0/3372560088 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd1c40c30d0 con 0x561f7784b2f0 2024-04-25T10:43:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.946+0000 7fd1e0805740 10 client.18834 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.946+0000 7fd1e0805740 1 -- 172.21.15.43:0/3372560088 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561f77958620 con 0x561f7784b2f0 2024-04-25T10:43:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.946+0000 7fd1b2ffd640 1 -- 172.21.15.43:0/3372560088 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1c40a6cc0 con 0x561f7784b2f0 2024-04-25T10:43:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.947+0000 7fd1b2ffd640 1 -- 172.21.15.43:0/3372560088 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd1c40bc190 con 0x561f7784b2f0 2024-04-25T10:43:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.947+0000 7fd1e0805740 10 client.18834 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.947+0000 7fd1e0805740 10 client.18834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:43:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.947+0000 7fd1e0805740 20 client.18834 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.947+0000 7fd1e0805740 1 --2- 172.21.15.43:0/3372560088 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561f77832d80 0x561f77c30450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.947+0000 7fd1e0805740 4 client.18834 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:43:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.947+0000 7fd1e0805740 1 -- 172.21.15.43:0/3372560088 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f77b88070 con 0x561f77832d80 2024-04-25T10:43:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.947+0000 7fd1d27fc640 1 --2- 172.21.15.43:0/3372560088 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561f77832d80 0x561f77c30450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.947+0000 7fd1d27fc640 1 --2- 172.21.15.43:0/3372560088 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561f77832d80 0x561f77c30450 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8db3f7c10dcfbdaf server_cookie=9f4d19b90075896c in_seq=0 out_seq=0 2024-04-25T10:43:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.947+0000 7fd1b2ffd640 10 client.18834 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.948+0000 7fd1b2ffd640 1 -- 172.21.15.43:0/3372560088 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561f77b88070 con 0x561f77832d80 2024-04-25T10:43:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.948+0000 7fd1b2ffd640 10 client.18834 handle_command_reply: tid=0 2024-04-25T10:43:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.948+0000 7fd1e0805740 1 client.18834 shutdown 2024-04-25T10:43:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.949+0000 7fd1e0805740 1 -- 172.21.15.43:0/3372560088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f7784b2f0 msgr2=0x561f77c1e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.949+0000 7fd1e0805740 1 --2- 172.21.15.43:0/3372560088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f7784b2f0 0x561f77c1e110 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40dc810 tx=0x7fd1c40058c0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.949+0000 7fd1e0805740 1 -- 172.21.15.43:0/3372560088 shutdown_connections 2024-04-25T10:43:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.949+0000 7fd1e0805740 1 --2- 172.21.15.43:0/3372560088 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561f77832d80 0x561f77c30450 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.949+0000 7fd1e0805740 1 --2- 172.21.15.43:0/3372560088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f778534d0 0x561f77c1e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.949+0000 7fd1b2ffd640 0 client.18834 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.949+0000 7fd1e0805740 1 --2- 172.21.15.43:0/3372560088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f778741b0 0x561f77c2a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.949+0000 7fd1e0805740 1 --2- 172.21.15.43:0/3372560088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f7784b2f0 0x561f77c1e110 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.949+0000 7fd1e0805740 1 -- 172.21.15.43:0/3372560088 >> 172.21.15.43:0/3372560088 conn(0x561f77868300 msgr2=0x561f77c15810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.949+0000 7fd1e0805740 1 -- 172.21.15.43:0/3372560088 shutdown_connections 2024-04-25T10:43:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.949+0000 7fd1e0805740 1 -- 172.21.15.43:0/3372560088 wait complete. 2024-04-25T10:43:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.949+0000 7fd1e0805740 20 client.18834 trim_cache size 0 max 16384 2024-04-25T10:43:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.964+0000 7fd1e0805740 1 Processor -- start 2024-04-25T10:43:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.964+0000 7fd1e0805740 1 -- start start 2024-04-25T10:43:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.965+0000 7fd1e0805740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f77832d80 0x561f77c29fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.965+0000 7fd1e0805740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f7784b2f0 0x561f77c14a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.965+0000 7fd1e0805740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f778534d0 0x561f77c33470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.965+0000 7fd1e0805740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d41b4670 con 0x561f77832d80 2024-04-25T10:43:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.965+0000 7fd1e0805740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d41b1f40 con 0x561f7784b2f0 2024-04-25T10:43:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.965+0000 7fd1e0805740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d41afdb0 con 0x561f778534d0 2024-04-25T10:43:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.965+0000 7fd1d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f77832d80 0x561f77c29fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.965+0000 7fd1d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f77832d80 0x561f77c29fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49730/0 (socket says 172.21.15.43:49730) 2024-04-25T10:43:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.965+0000 7fd1d2ffd640 1 -- 172.21.15.43:0/1743917984 learned_addr learned my addr 172.21.15.43:0/1743917984 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.965+0000 7fd1d27fc640 1 --2- 172.21.15.43:0/1743917984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f7784b2f0 0x561f77c14a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.965+0000 7fd1d2ffd640 1 -- 172.21.15.43:0/1743917984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f778534d0 msgr2=0x561f77c33470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.965+0000 7fd1d2ffd640 1 --2- 172.21.15.43:0/1743917984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f778534d0 0x561f77c33470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.965+0000 7fd1d2ffd640 1 -- 172.21.15.43:0/1743917984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f7784b2f0 msgr2=0x561f77c14a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.965+0000 7fd1d2ffd640 1 --2- 172.21.15.43:0/1743917984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f7784b2f0 0x561f77c14a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.965+0000 7fd1d2ffd640 1 -- 172.21.15.43:0/1743917984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f77c13880 con 0x561f77832d80 2024-04-25T10:43:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.965+0000 7fd1d27fc640 1 --2- 172.21.15.43:0/1743917984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f7784b2f0 0x561f77c14a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:43:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.965+0000 7fd1d2ffd640 1 --2- 172.21.15.43:0/1743917984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f77832d80 0x561f77c29fe0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400c970 tx=0x7fd1c40a3180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e683497fd2f406 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.965+0000 7fd1b97fa640 1 -- 172.21.15.43:0/1743917984 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c401d070 con 0x561f77832d80 2024-04-25T10:43:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.965+0000 7fd1b97fa640 1 -- 172.21.15.43:0/1743917984 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1c40a6cc0 con 0x561f77832d80 2024-04-25T10:43:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.965+0000 7fd1b97fa640 1 -- 172.21.15.43:0/1743917984 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c40a6920 con 0x561f77832d80 2024-04-25T10:43:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.966+0000 7fd1e0805740 1 -- 172.21.15.43:0/1743917984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f77832d80 msgr2=0x561f77c29fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.966+0000 7fd1e0805740 1 --2- 172.21.15.43:0/1743917984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f77832d80 0x561f77c29fe0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400c970 tx=0x7fd1c40a3180 comp rx=0 tx=0).stop 2024-04-25T10:43:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.966+0000 7fd1e0805740 1 -- 172.21.15.43:0/1743917984 shutdown_connections 2024-04-25T10:43:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.966+0000 7fd1e0805740 1 --2- 172.21.15.43:0/1743917984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f7784b2f0 0x561f77c14a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.966+0000 7fd1e0805740 1 --2- 172.21.15.43:0/1743917984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f778534d0 0x561f77c33470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.966+0000 7fd1e0805740 1 --2- 172.21.15.43:0/1743917984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f77832d80 0x561f77c29fe0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.966+0000 7fd1e0805740 1 -- 172.21.15.43:0/1743917984 >> 172.21.15.43:0/1743917984 conn(0x561f77868300 msgr2=0x561f77c1f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.966+0000 7fd1e0805740 1 -- 172.21.15.43:0/1743917984 shutdown_connections 2024-04-25T10:43:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.966+0000 7fd1e0805740 1 -- 172.21.15.43:0/1743917984 wait complete. 2024-04-25T10:43:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.967+0000 7fd1e0805740 1 -- start start 2024-04-25T10:43:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.967+0000 7fd1e0805740 1 Processor -- start 2024-04-25T10:43:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.967+0000 7fd1e0805740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f77832d80 0x561f77c33630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.967+0000 7fd1e0805740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f7784b2f0 0x561f77c33b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.967+0000 7fd1e0805740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f778534d0 0x561f77c30ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.967+0000 7fd1e0805740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d41b0400 con 0x561f77832d80 2024-04-25T10:43:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.967+0000 7fd1e0805740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d41b1870 con 0x561f778534d0 2024-04-25T10:43:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.967+0000 7fd1e0805740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d41b0ad0 con 0x561f7784b2f0 2024-04-25T10:43:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.967+0000 7fd1e0805740 10 client.0 _ll_register_callbacks cb 0x7fd1cc009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:43:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.967+0000 7fd1d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f77832d80 0x561f77c33630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.967+0000 7fd1d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f77832d80 0x561f77c33630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49746/0 (socket says 172.21.15.43:49746) 2024-04-25T10:43:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.967+0000 7fd1d2ffd640 1 -- 172.21.15.43:0/3539913912 learned_addr learned my addr 172.21.15.43:0/3539913912 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:43:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.967+0000 7fd1d27fc640 1 --2- 172.21.15.43:0/3539913912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f7784b2f0 0x561f77c33b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.968+0000 7fd1d37fe640 1 --2- 172.21.15.43:0/3539913912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f778534d0 0x561f77c30ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.968+0000 7fd1d2ffd640 1 -- 172.21.15.43:0/3539913912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f7784b2f0 msgr2=0x561f77c33b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.968+0000 7fd1d2ffd640 1 --2- 172.21.15.43:0/3539913912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f7784b2f0 0x561f77c33b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.968+0000 7fd1d2ffd640 1 -- 172.21.15.43:0/3539913912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f778534d0 msgr2=0x561f77c30ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:43:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.968+0000 7fd1d2ffd640 1 --2- 172.21.15.43:0/3539913912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f778534d0 0x561f77c30ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.968+0000 7fd1d2ffd640 1 -- 172.21.15.43:0/3539913912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f77c31240 con 0x561f77832d80 2024-04-25T10:43:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.968+0000 7fd1d37fe640 1 --2- 172.21.15.43:0/3539913912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f778534d0 0x561f77c30ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:43:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.968+0000 7fd1d2ffd640 1 --2- 172.21.15.43:0/3539913912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f77832d80 0x561f77c33630 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40dcd60 tx=0x7fd1c40a7300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8c30487157d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:43:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.968+0000 7fd1b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:43:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.968+0000 7fd1b2ffd640 1 -- 172.21.15.43:0/3539913912 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c401d070 con 0x561f77832d80 2024-04-25T10:43:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.968+0000 7fd1b2ffd640 1 -- 172.21.15.43:0/3539913912 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fd1c40ab120 con 0x561f77832d80 2024-04-25T10:43:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.968+0000 7fd1e0805740 1 -- 172.21.15.43:0/3539913912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1c40a6cc0 con 0x561f77832d80 2024-04-25T10:43:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.968+0000 7fd1b2ffd640 1 -- 172.21.15.43:0/3539913912 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd1c40a6920 con 0x561f77832d80 2024-04-25T10:43:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.968+0000 7fd1e0805740 10 client.18840 fetch_fsmap learned FSMap version 26 2024-04-25T10:43:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.968+0000 7fd1e0805740 1 -- 172.21.15.43:0/3539913912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561f77c31240 con 0x561f77832d80 2024-04-25T10:43:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.968+0000 7fd1b2ffd640 1 -- 172.21.15.43:0/3539913912 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1c40a6cc0 con 0x561f77832d80 2024-04-25T10:43:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.969+0000 7fd1b2ffd640 1 -- 172.21.15.43:0/3539913912 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd1c40c80a0 con 0x561f77832d80 2024-04-25T10:43:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.969+0000 7fd1e0805740 10 client.18840 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:43:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.969+0000 7fd1e0805740 10 client.18840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:43:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.969+0000 7fd1e0805740 20 client.18840 populate_metadata read hostname 'smithi043' 2024-04-25T10:43:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.969+0000 7fd1e0805740 1 --2- 172.21.15.43:0/3539913912 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561f778741b0 0x561f77c9a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:43:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.969+0000 7fd1e0805740 4 client.18840 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:43:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.969+0000 7fd1e0805740 1 -- 172.21.15.43:0/3539913912 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f77b81ee0 con 0x561f778741b0 2024-04-25T10:43:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.969+0000 7fd1d27fc640 1 --2- 172.21.15.43:0/3539913912 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561f778741b0 0x561f77c9a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:43:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.969+0000 7fd1b2ffd640 10 client.18840 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.969+0000 7fd1d27fc640 1 --2- 172.21.15.43:0/3539913912 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561f778741b0 0x561f77c9a340 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2138db59b2aed9bf server_cookie=63c3616c51561bcb in_seq=0 out_seq=0 2024-04-25T10:43:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.984+0000 7fd1b2ffd640 1 -- 172.21.15.43:0/3539913912 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f77b81ee0 con 0x561f778741b0 2024-04-25T10:43:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.984+0000 7fd1b2ffd640 10 client.18840 handle_command_reply: tid=0 2024-04-25T10:43:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.984+0000 7fd1e0805740 1 client.18840 shutdown 2024-04-25T10:43:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.985+0000 7fd1e0805740 1 -- 172.21.15.43:0/3539913912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f77832d80 msgr2=0x561f77c33630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.985+0000 7fd1e0805740 1 --2- 172.21.15.43:0/3539913912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f77832d80 0x561f77c33630 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40dcd60 tx=0x7fd1c40a7300 comp rx=0 tx=0).stop 2024-04-25T10:43:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.985+0000 7fd1e0805740 1 -- 172.21.15.43:0/3539913912 shutdown_connections 2024-04-25T10:43:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.985+0000 7fd1e0805740 1 --2- 172.21.15.43:0/3539913912 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x561f778741b0 0x561f77c9a340 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.985+0000 7fd1e0805740 1 --2- 172.21.15.43:0/3539913912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f778534d0 0x561f77c30ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.985+0000 7fd1e0805740 1 --2- 172.21.15.43:0/3539913912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f7784b2f0 0x561f77c33b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.985+0000 7fd1b2ffd640 0 client.18840 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:43:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.985+0000 7fd1e0805740 1 --2- 172.21.15.43:0/3539913912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f77832d80 0x561f77c33630 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.985+0000 7fd1e0805740 1 -- 172.21.15.43:0/3539913912 >> 172.21.15.43:0/3539913912 conn(0x561f77868300 msgr2=0x561f77c1e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.985+0000 7fd1e0805740 1 -- 172.21.15.43:0/3539913912 shutdown_connections 2024-04-25T10:43:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.985+0000 7fd1e0805740 1 -- 172.21.15.43:0/3539913912 wait complete. 2024-04-25T10:43:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.985+0000 7fd1e0805740 20 client.18840 trim_cache size 0 max 16384 2024-04-25T10:43:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:43:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:43:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:43:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:43:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.985+0000 7fd1d9877640 1 -- 172.21.15.43:0/34715052 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd1bc0afc50 msgr2=0x7fd1bc0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.985+0000 7fd1d9877640 1 --2- 172.21.15.43:0/34715052 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd1bc0afc50 0x7fd1bc0b2160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x561f77c10c60 tx=0x7fd1cc004a40 comp rx=0 tx=0).stop 2024-04-25T10:43:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.985+0000 7fd1d9877640 1 -- 172.21.15.43:0/34715052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1d41042a0 msgr2=0x7fd1d419d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:43:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.985+0000 7fd1d9877640 1 --2- 172.21.15.43:0/34715052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1d41042a0 0x7fd1d419d7e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400c9a0 tx=0x7fd1c400ce60 comp rx=0 tx=0).stop 2024-04-25T10:43:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.985+0000 7fd1d9877640 1 -- 172.21.15.43:0/34715052 shutdown_connections 2024-04-25T10:43:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.986+0000 7fd1d9877640 1 --2- 172.21.15.43:0/34715052 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fd1bc0afc50 0x7fd1bc0b2160 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.986+0000 7fd1d9877640 1 --2- 172.21.15.43:0/34715052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1d4109ff0 0x7fd1d41a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.986+0000 7fd1d9877640 1 --2- 172.21.15.43:0/34715052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d4109b50 0x7fd1d419dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.986+0000 7fd1d9877640 1 --2- 172.21.15.43:0/34715052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1d41042a0 0x7fd1d419d7e0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:43:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.986+0000 7fd1d9877640 1 -- 172.21.15.43:0/34715052 >> 172.21.15.43:0/34715052 conn(0x7fd1d40ff230 msgr2=0x7fd1d40767b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:43:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.986+0000 7fd1d9877640 1 -- 172.21.15.43:0/34715052 shutdown_connections 2024-04-25T10:43:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:43:54.986+0000 7fd1d9877640 1 -- 172.21.15.43:0/34715052 wait complete. 2024-04-25T10:43:56.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:56 smithi043 ceph-mon[29478]: pgmap v355: 97 pgs: 97 active+clean; 2.3 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-04-25T10:43:56.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:56 smithi077 ceph-mon[32772]: pgmap v355: 97 pgs: 97 active+clean; 2.3 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-04-25T10:43:56.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:56 smithi164 ceph-mon[32713]: pgmap v355: 97 pgs: 97 active+clean; 2.3 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-04-25T10:43:58.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:43:58 smithi043 ceph-mon[29478]: pgmap v356: 97 pgs: 97 active+clean; 2.3 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-04-25T10:43:58.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:43:58 smithi077 ceph-mon[32772]: pgmap v356: 97 pgs: 97 active+clean; 2.3 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-04-25T10:43:58.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:43:58 smithi164 ceph-mon[32713]: pgmap v356: 97 pgs: 97 active+clean; 2.3 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-04-25T10:43:59.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:43:59.972 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:44:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.100+0000 7f1e54d6c640 1 -- 172.21.15.43:0/3564186460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e50107440 msgr2=0x7f1e5010ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.100+0000 7f1e54d6c640 1 --2- 172.21.15.43:0/3564186460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e50107440 0x7f1e5010ebe0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009ca0 tx=0x7f1e4802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.100+0000 7f1e54d6c640 1 -- 172.21.15.43:0/3564186460 shutdown_connections 2024-04-25T10:44:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.100+0000 7f1e54d6c640 1 --2- 172.21.15.43:0/3564186460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e50106980 0x7f1e50106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.100+0000 7f1e54d6c640 1 --2- 172.21.15.43:0/3564186460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e50107440 0x7f1e5010ebe0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.100+0000 7f1e54d6c640 1 --2- 172.21.15.43:0/3564186460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e501023f0 0x7f1e50102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.100+0000 7f1e54d6c640 1 -- 172.21.15.43:0/3564186460 >> 172.21.15.43:0/3564186460 conn(0x7f1e500fd850 msgr2=0x7f1e500ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.100+0000 7f1e54d6c640 1 -- 172.21.15.43:0/3564186460 shutdown_connections 2024-04-25T10:44:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.101+0000 7f1e54d6c640 1 -- 172.21.15.43:0/3564186460 wait complete. 2024-04-25T10:44:00.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.101+0000 7f1e54d6c640 1 Processor -- start 2024-04-25T10:44:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.102+0000 7f1e54d6c640 1 -- start start 2024-04-25T10:44:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.102+0000 7f1e54d6c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e501023f0 0x7f1e5019b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.102+0000 7f1e54d6c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e50106980 0x7f1e5019ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.102+0000 7f1e54d6c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e50107440 0x7f1e5019bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.102+0000 7f1e54d6c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e50112bd0 con 0x7f1e501023f0 2024-04-25T10:44:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.102+0000 7f1e54d6c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1e50112a50 con 0x7f1e50107440 2024-04-25T10:44:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.102+0000 7f1e54d6c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1e50112d50 con 0x7f1e50106980 2024-04-25T10:44:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.102+0000 7f1e4e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e501023f0 0x7f1e5019b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.102+0000 7f1e4e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e501023f0 0x7f1e5019b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32830/0 (socket says 172.21.15.43:32830) 2024-04-25T10:44:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.102+0000 7f1e4e59c640 1 -- 172.21.15.43:0/439501742 learned_addr learned my addr 172.21.15.43:0/439501742 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.102+0000 7f1e4ed9d640 1 --2- 172.21.15.43:0/439501742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e50107440 0x7f1e5019bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.102+0000 7f1e45d9b640 1 --2- 172.21.15.43:0/439501742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e50106980 0x7f1e5019ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.102+0000 7f1e4e59c640 1 -- 172.21.15.43:0/439501742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e50106980 msgr2=0x7f1e5019ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.102+0000 7f1e4e59c640 1 --2- 172.21.15.43:0/439501742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e50106980 0x7f1e5019ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.102+0000 7f1e4e59c640 1 -- 172.21.15.43:0/439501742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e50107440 msgr2=0x7f1e5019bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.102+0000 7f1e4e59c640 1 --2- 172.21.15.43:0/439501742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e50107440 0x7f1e5019bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.102+0000 7f1e4e59c640 1 -- 172.21.15.43:0/439501742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e501a6790 con 0x7f1e501023f0 2024-04-25T10:44:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.103+0000 7f1e4e59c640 1 --2- 172.21.15.43:0/439501742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e501023f0 0x7f1e5019b510 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000e9f0 tx=0x7f1e4000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.103+0000 7f1e47fff640 1 -- 172.21.15.43:0/439501742 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4000ccc0 con 0x7f1e501023f0 2024-04-25T10:44:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.103+0000 7f1e54d6c640 1 -- 172.21.15.43:0/439501742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e501a6a80 con 0x7f1e501023f0 2024-04-25T10:44:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.103+0000 7f1e47fff640 1 -- 172.21.15.43:0/439501742 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e4000ce60 con 0x7f1e501023f0 2024-04-25T10:44:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.103+0000 7f1e47fff640 1 -- 172.21.15.43:0/439501742 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40010430 con 0x7f1e501023f0 2024-04-25T10:44:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.103+0000 7f1e54d6c640 1 -- 172.21.15.43:0/439501742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e501a6fd0 con 0x7f1e501023f0 2024-04-25T10:44:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.105+0000 7f1e47fff640 1 -- 172.21.15.43:0/439501742 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f1e400046c0 con 0x7f1e501023f0 2024-04-25T10:44:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.106+0000 7f1e5bc17740 1 Processor -- start 2024-04-25T10:44:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.106+0000 7f1e5bc17740 1 -- start start 2024-04-25T10:44:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.106+0000 7f1e5bc17740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565192961390 0x565192d01a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.106+0000 7f1e5bc17740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651929384d0 0x565192d01fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.106+0000 7f1e5bc17740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929406b0 0x565192d0e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.106+0000 7f1e5bc17740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565192c67d80 con 0x5651929406b0 2024-04-25T10:44:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.106+0000 7f1e5bc17740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x565192c218b0 con 0x5651929384d0 2024-04-25T10:44:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.106+0000 7f1e5bc17740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565192c63d40 con 0x565192961390 2024-04-25T10:44:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.106+0000 7f1e47fff640 1 --2- 172.21.15.43:0/439501742 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1e3c0afc60 0x7f1e3c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.106+0000 7f1e45d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929406b0 0x565192d0e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.106+0000 7f1e45d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929406b0 0x565192d0e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32846/0 (socket says 172.21.15.43:32846) 2024-04-25T10:44:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.106+0000 7f1e45d9b640 1 -- 172.21.15.43:0/93798886 learned_addr learned my addr 172.21.15.43:0/93798886 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.106+0000 7f1e4ed9d640 1 --2- 172.21.15.43:0/439501742 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1e3c0afc60 0x7f1e3c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.107+0000 7f1e4ed9d640 1 --2- 172.21.15.43:0/93798886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565192961390 0x565192d01a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.107+0000 7f1e47fff640 1 -- 172.21.15.43:0/439501742 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f1e400df480 con 0x7f1e501023f0 2024-04-25T10:44:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.107+0000 7f1e4e59c640 1 --2- 172.21.15.43:0/93798886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651929384d0 0x565192d01fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.107+0000 7f1e45d9b640 1 -- 172.21.15.43:0/93798886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565192961390 msgr2=0x565192d01a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.107+0000 7f1e45d9b640 1 --2- 172.21.15.43:0/93798886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565192961390 0x565192d01a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.107+0000 7f1e45d9b640 1 -- 172.21.15.43:0/93798886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651929384d0 msgr2=0x565192d01fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.107+0000 7f1e45d9b640 1 --2- 172.21.15.43:0/93798886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651929384d0 0x565192d01fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.107+0000 7f1e45d9b640 1 -- 172.21.15.43:0/93798886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565192bf2a40 con 0x5651929406b0 2024-04-25T10:44:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.107+0000 7f1e4e59c640 1 --2- 172.21.15.43:0/93798886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651929384d0 0x565192d01fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.107+0000 7f1e4ed9d640 1 --2- 172.21.15.43:0/439501742 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1e3c0afc60 0x7f1e3c0b2170 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1e480094e0 tx=0x7f1e4802f310 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.107+0000 7f1e45d9b640 1 --2- 172.21.15.43:0/93798886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929406b0 0x565192d0e8f0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800abf0 tx=0x7f1e3800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824340d991a25c88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.107+0000 7f1e44d99640 1 -- 172.21.15.43:0/93798886 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3801d070 con 0x5651929406b0 2024-04-25T10:44:00.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.107+0000 7f1e44d99640 1 -- 172.21.15.43:0/93798886 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e38018af0 con 0x5651929406b0 2024-04-25T10:44:00.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.107+0000 7f1e44d99640 1 -- 172.21.15.43:0/93798886 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38010aa0 con 0x5651929406b0 2024-04-25T10:44:00.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.108+0000 7f1e5bc17740 1 -- 172.21.15.43:0/93798886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929406b0 msgr2=0x565192d0e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.108+0000 7f1e5bc17740 1 --2- 172.21.15.43:0/93798886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929406b0 0x565192d0e8f0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800abf0 tx=0x7f1e3800f790 comp rx=0 tx=0).stop 2024-04-25T10:44:00.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.108+0000 7f1e5bc17740 1 -- 172.21.15.43:0/93798886 shutdown_connections 2024-04-25T10:44:00.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.108+0000 7f1e5bc17740 1 --2- 172.21.15.43:0/93798886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651929384d0 0x565192d01fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.108+0000 7f1e5bc17740 1 --2- 172.21.15.43:0/93798886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929406b0 0x565192d0e8f0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.108+0000 7f1e5bc17740 1 --2- 172.21.15.43:0/93798886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565192961390 0x565192d01a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.108+0000 7f1e5bc17740 1 -- 172.21.15.43:0/93798886 >> 172.21.15.43:0/93798886 conn(0x5651929554e0 msgr2=0x565192cfe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.108+0000 7f1e5bc17740 1 -- 172.21.15.43:0/93798886 shutdown_connections 2024-04-25T10:44:00.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.108+0000 7f1e5bc17740 1 -- 172.21.15.43:0/93798886 wait complete. 2024-04-25T10:44:00.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.109+0000 7f1e5bc17740 1 -- start start 2024-04-25T10:44:00.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.109+0000 7f1e5bc17740 1 Processor -- start 2024-04-25T10:44:00.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.109+0000 7f1e5bc17740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651929384d0 0x565192d0c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.109+0000 7f1e5bc17740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651929406b0 0x565192d0cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.109+0000 7f1e5bc17740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565192961390 0x565192d18460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.109+0000 7f1e5bc17740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565192d0efe0 con 0x565192961390 2024-04-25T10:44:00.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.109+0000 7f1e5bc17740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x565192beed10 con 0x5651929406b0 2024-04-25T10:44:00.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.109+0000 7f1e5bc17740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565192d0f160 con 0x5651929384d0 2024-04-25T10:44:00.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.109+0000 7f1e5bc17740 10 client.0 _ll_register_callbacks cb 0x565192c1b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:00.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.109+0000 7f1e4ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565192961390 0x565192d18460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.109+0000 7f1e4ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565192961390 0x565192d18460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32850/0 (socket says 172.21.15.43:32850) 2024-04-25T10:44:00.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.109+0000 7f1e4ed9d640 1 -- 172.21.15.43:0/1502247677 learned_addr learned my addr 172.21.15.43:0/1502247677 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.110+0000 7f1e45d9b640 1 --2- 172.21.15.43:0/1502247677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651929406b0 0x565192d0cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.110+0000 7f1e4e59c640 1 --2- 172.21.15.43:0/1502247677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651929384d0 0x565192d0c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.110+0000 7f1e4ed9d640 1 -- 172.21.15.43:0/1502247677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651929384d0 msgr2=0x565192d0c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.110+0000 7f1e4ed9d640 1 --2- 172.21.15.43:0/1502247677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651929384d0 0x565192d0c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.110+0000 7f1e4ed9d640 1 -- 172.21.15.43:0/1502247677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651929406b0 msgr2=0x565192d0cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.110+0000 7f1e4ed9d640 1 --2- 172.21.15.43:0/1502247677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651929406b0 0x565192d0cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.110+0000 7f1e4ed9d640 1 -- 172.21.15.43:0/1502247677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565192a437d0 con 0x565192961390 2024-04-25T10:44:00.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.110+0000 7f1e4ed9d640 1 --2- 172.21.15.43:0/1502247677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565192961390 0x565192d18460 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f1e48035000 tx=0x7f1e4802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c14f25f5631404 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.110+0000 7f1e35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:00.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.110+0000 7f1e35ffb640 1 -- 172.21.15.43:0/1502247677 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48051070 con 0x565192961390 2024-04-25T10:44:00.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.110+0000 7f1e35ffb640 1 -- 172.21.15.43:0/1502247677 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f1e48003970 con 0x565192961390 2024-04-25T10:44:00.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.110+0000 7f1e5bc17740 1 -- 172.21.15.43:0/1502247677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e38018af0 con 0x565192961390 2024-04-25T10:44:00.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.110+0000 7f1e35ffb640 1 -- 172.21.15.43:0/1502247677 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1e4805d050 con 0x565192961390 2024-04-25T10:44:00.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.110+0000 7f1e5bc17740 10 client.18852 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:00.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.110+0000 7f1e5bc17740 1 -- 172.21.15.43:0/1502247677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5651929793f0 con 0x565192961390 2024-04-25T10:44:00.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.111+0000 7f1e35ffb640 1 -- 172.21.15.43:0/1502247677 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e38018af0 con 0x565192961390 2024-04-25T10:44:00.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.111+0000 7f1e35ffb640 1 -- 172.21.15.43:0/1502247677 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1e4802f660 con 0x565192961390 2024-04-25T10:44:00.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.111+0000 7f1e5bc17740 10 client.18852 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:00.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.111+0000 7f1e5bc17740 10 client.18852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:44:00.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.111+0000 7f1e5bc17740 20 client.18852 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:00.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.111+0000 7f1e5bc17740 1 --2- 172.21.15.43:0/1502247677 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56519291ff60 0x565192d1e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.111+0000 7f1e5bc17740 4 client.18852 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:44:00.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.111+0000 7f1e5bc17740 1 -- 172.21.15.43:0/1502247677 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565192c75990 con 0x56519291ff60 2024-04-25T10:44:00.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.111+0000 7f1e4e59c640 1 --2- 172.21.15.43:0/1502247677 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56519291ff60 0x565192d1e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.111+0000 7f1e35ffb640 10 client.18852 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:00.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.111+0000 7f1e4e59c640 1 --2- 172.21.15.43:0/1502247677 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56519291ff60 0x565192d1e5c0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65bc84f3e088b97c server_cookie=777305c7633684f0 in_seq=0 out_seq=0 2024-04-25T10:44:00.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.112+0000 7f1e35ffb640 1 -- 172.21.15.43:0/1502247677 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565192c75990 con 0x56519291ff60 2024-04-25T10:44:00.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.112+0000 7f1e35ffb640 10 client.18852 handle_command_reply: tid=0 2024-04-25T10:44:00.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.112+0000 7f1e5bc17740 1 client.18852 shutdown 2024-04-25T10:44:00.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.112+0000 7f1e5bc17740 1 -- 172.21.15.43:0/1502247677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565192961390 msgr2=0x565192d18460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.112+0000 7f1e5bc17740 1 --2- 172.21.15.43:0/1502247677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565192961390 0x565192d18460 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f1e48035000 tx=0x7f1e4802a8c0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.112+0000 7f1e5bc17740 1 -- 172.21.15.43:0/1502247677 shutdown_connections 2024-04-25T10:44:00.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.112+0000 7f1e5bc17740 1 --2- 172.21.15.43:0/1502247677 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x56519291ff60 0x565192d1e5c0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.112+0000 7f1e5bc17740 1 --2- 172.21.15.43:0/1502247677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651929406b0 0x565192d0cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.112+0000 7f1e5bc17740 1 --2- 172.21.15.43:0/1502247677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565192961390 0x565192d18460 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.112+0000 7f1e35ffb640 0 client.18852 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:00.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.112+0000 7f1e5bc17740 1 --2- 172.21.15.43:0/1502247677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651929384d0 0x565192d0c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.112+0000 7f1e5bc17740 1 -- 172.21.15.43:0/1502247677 >> 172.21.15.43:0/1502247677 conn(0x5651929554e0 msgr2=0x565192d03780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.113+0000 7f1e5bc17740 1 -- 172.21.15.43:0/1502247677 shutdown_connections 2024-04-25T10:44:00.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.113+0000 7f1e5bc17740 1 -- 172.21.15.43:0/1502247677 wait complete. 2024-04-25T10:44:00.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.113+0000 7f1e5bc17740 20 client.18852 trim_cache size 0 max 16384 2024-04-25T10:44:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.131+0000 7f1e5bc17740 1 Processor -- start 2024-04-25T10:44:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.131+0000 7f1e5bc17740 1 -- start start 2024-04-25T10:44:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.131+0000 7f1e5bc17740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56519291ff60 0x565192d14430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.131+0000 7f1e5bc17740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651929384d0 0x565192d14970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.131+0000 7f1e5bc17740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929406b0 0x565192d03720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.131+0000 7f1e5bc17740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e501b2190 con 0x5651929406b0 2024-04-25T10:44:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.131+0000 7f1e5bc17740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1e501afa60 con 0x56519291ff60 2024-04-25T10:44:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.131+0000 7f1e5bc17740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1e501ad8d0 con 0x5651929384d0 2024-04-25T10:44:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.131+0000 7f1e4ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929406b0 0x565192d03720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.131+0000 7f1e4ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929406b0 0x565192d03720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32862/0 (socket says 172.21.15.43:32862) 2024-04-25T10:44:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.131+0000 7f1e4ed9d640 1 -- 172.21.15.43:0/3569644764 learned_addr learned my addr 172.21.15.43:0/3569644764 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.131+0000 7f1e4e59c640 1 --2- 172.21.15.43:0/3569644764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56519291ff60 0x565192d14430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.131+0000 7f1e4ed9d640 1 -- 172.21.15.43:0/3569644764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651929384d0 msgr2=0x565192d14970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.131+0000 7f1e4ed9d640 1 --2- 172.21.15.43:0/3569644764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651929384d0 0x565192d14970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.131+0000 7f1e4ed9d640 1 -- 172.21.15.43:0/3569644764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56519291ff60 msgr2=0x565192d14430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.131+0000 7f1e4ed9d640 1 --2- 172.21.15.43:0/3569644764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56519291ff60 0x565192d14430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.131+0000 7f1e4ed9d640 1 -- 172.21.15.43:0/3569644764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651929390f0 con 0x5651929406b0 2024-04-25T10:44:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.131+0000 7f1e4e59c640 1 --2- 172.21.15.43:0/3569644764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56519291ff60 0x565192d14430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.132+0000 7f1e4ed9d640 1 --2- 172.21.15.43:0/3569644764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929406b0 0x565192d03720 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009a60 tx=0x7f1e48009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf29ec4436082a94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.132+0000 7f1e37fff640 1 -- 172.21.15.43:0/3569644764 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48051070 con 0x5651929406b0 2024-04-25T10:44:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.132+0000 7f1e37fff640 1 -- 172.21.15.43:0/3569644764 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e4802fca0 con 0x5651929406b0 2024-04-25T10:44:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.132+0000 7f1e37fff640 1 -- 172.21.15.43:0/3569644764 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48004520 con 0x5651929406b0 2024-04-25T10:44:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.133+0000 7f1e5bc17740 1 -- 172.21.15.43:0/3569644764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929406b0 msgr2=0x565192d03720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.133+0000 7f1e5bc17740 1 --2- 172.21.15.43:0/3569644764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929406b0 0x565192d03720 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009a60 tx=0x7f1e48009bd0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.133+0000 7f1e5bc17740 1 -- 172.21.15.43:0/3569644764 shutdown_connections 2024-04-25T10:44:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.133+0000 7f1e5bc17740 1 --2- 172.21.15.43:0/3569644764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929406b0 0x565192d03720 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.133+0000 7f1e5bc17740 1 --2- 172.21.15.43:0/3569644764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651929384d0 0x565192d14970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.133+0000 7f1e5bc17740 1 --2- 172.21.15.43:0/3569644764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56519291ff60 0x565192d14430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.133+0000 7f1e5bc17740 1 -- 172.21.15.43:0/3569644764 >> 172.21.15.43:0/3569644764 conn(0x5651929554e0 msgr2=0x565192d0ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.133+0000 7f1e5bc17740 1 -- 172.21.15.43:0/3569644764 shutdown_connections 2024-04-25T10:44:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.133+0000 7f1e5bc17740 1 -- 172.21.15.43:0/3569644764 wait complete. 2024-04-25T10:44:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.133+0000 7f1e5bc17740 1 -- start start 2024-04-25T10:44:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.133+0000 7f1e5bc17740 1 Processor -- start 2024-04-25T10:44:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.133+0000 7f1e5bc17740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56519291ff60 0x565192d147c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.133+0000 7f1e5bc17740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929384d0 0x565192d0fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.133+0000 7f1e5bc17740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651929406b0 0x565192d10080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.133+0000 7f1e5bc17740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e501adf20 con 0x5651929384d0 2024-04-25T10:44:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.133+0000 7f1e5bc17740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1e501af390 con 0x5651929406b0 2024-04-25T10:44:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.133+0000 7f1e5bc17740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1e501ae5f0 con 0x56519291ff60 2024-04-25T10:44:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.133+0000 7f1e5bc17740 10 client.0 _ll_register_callbacks cb 0x7f1e4802f460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.133+0000 7f1e45d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929384d0 0x565192d0fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.134+0000 7f1e45d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929384d0 0x565192d0fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32868/0 (socket says 172.21.15.43:32868) 2024-04-25T10:44:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.134+0000 7f1e45d9b640 1 -- 172.21.15.43:0/1577962600 learned_addr learned my addr 172.21.15.43:0/1577962600 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.134+0000 7f1e4ed9d640 1 --2- 172.21.15.43:0/1577962600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651929406b0 0x565192d10080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.134+0000 7f1e4e59c640 1 --2- 172.21.15.43:0/1577962600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56519291ff60 0x565192d147c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.134+0000 7f1e45d9b640 1 -- 172.21.15.43:0/1577962600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56519291ff60 msgr2=0x565192d147c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.134+0000 7f1e45d9b640 1 --2- 172.21.15.43:0/1577962600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56519291ff60 0x565192d147c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.134+0000 7f1e45d9b640 1 -- 172.21.15.43:0/1577962600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651929406b0 msgr2=0x565192d10080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.134+0000 7f1e45d9b640 1 --2- 172.21.15.43:0/1577962600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651929406b0 0x565192d10080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.134+0000 7f1e45d9b640 1 -- 172.21.15.43:0/1577962600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565192c284b0 con 0x5651929384d0 2024-04-25T10:44:00.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.134+0000 7f1e4e59c640 1 --2- 172.21.15.43:0/1577962600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56519291ff60 0x565192d147c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:00.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.134+0000 7f1e4ed9d640 1 --2- 172.21.15.43:0/1577962600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651929406b0 0x565192d10080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:00.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.134+0000 7f1e45d9b640 1 --2- 172.21.15.43:0/1577962600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929384d0 0x565192d0fb40 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800abc0 tx=0x7f1e38010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8216650d106d712 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.134+0000 7f1e35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:00.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.134+0000 7f1e35ffb640 1 -- 172.21.15.43:0/1577962600 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38027020 con 0x5651929384d0 2024-04-25T10:44:00.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.134+0000 7f1e35ffb640 1 -- 172.21.15.43:0/1577962600 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f1e38005450 con 0x5651929384d0 2024-04-25T10:44:00.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.134+0000 7f1e5bc17740 1 -- 172.21.15.43:0/1577962600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e4802fca0 con 0x5651929384d0 2024-04-25T10:44:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.134+0000 7f1e35ffb640 1 -- 172.21.15.43:0/1577962600 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1e38032050 con 0x5651929384d0 2024-04-25T10:44:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.134+0000 7f1e5bc17740 10 client.18858 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.134+0000 7f1e5bc17740 1 -- 172.21.15.43:0/1577962600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565192d86420 con 0x5651929384d0 2024-04-25T10:44:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.135+0000 7f1e35ffb640 1 -- 172.21.15.43:0/1577962600 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e4802fca0 con 0x5651929384d0 2024-04-25T10:44:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.135+0000 7f1e35ffb640 1 -- 172.21.15.43:0/1577962600 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1e380048d0 con 0x5651929384d0 2024-04-25T10:44:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.135+0000 7f1e5bc17740 10 client.18858 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.135+0000 7f1e5bc17740 10 client.18858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:44:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.135+0000 7f1e5bc17740 20 client.18858 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.135+0000 7f1e5bc17740 1 --2- 172.21.15.43:0/1577962600 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x565192961390 0x565192d88810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.135+0000 7f1e5bc17740 4 client.18858 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:44:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.135+0000 7f1e5bc17740 1 -- 172.21.15.43:0/1577962600 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565192c6f800 con 0x565192961390 2024-04-25T10:44:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.135+0000 7f1e4e59c640 1 --2- 172.21.15.43:0/1577962600 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x565192961390 0x565192d88810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.135+0000 7f1e4e59c640 1 --2- 172.21.15.43:0/1577962600 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x565192961390 0x565192d88810 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7edb5d236c13e121 server_cookie=60b4c510751ca44 in_seq=0 out_seq=0 2024-04-25T10:44:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.135+0000 7f1e35ffb640 10 client.18858 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.136+0000 7f1e35ffb640 1 -- 172.21.15.43:0/1577962600 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x565192c6f800 con 0x565192961390 2024-04-25T10:44:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.136+0000 7f1e35ffb640 10 client.18858 handle_command_reply: tid=0 2024-04-25T10:44:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.136+0000 7f1e5bc17740 1 client.18858 shutdown 2024-04-25T10:44:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.136+0000 7f1e5bc17740 1 -- 172.21.15.43:0/1577962600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929384d0 msgr2=0x565192d0fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.136+0000 7f1e5bc17740 1 --2- 172.21.15.43:0/1577962600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929384d0 0x565192d0fb40 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800abc0 tx=0x7f1e38010430 comp rx=0 tx=0).stop 2024-04-25T10:44:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.136+0000 7f1e5bc17740 1 -- 172.21.15.43:0/1577962600 shutdown_connections 2024-04-25T10:44:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.136+0000 7f1e5bc17740 1 --2- 172.21.15.43:0/1577962600 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x565192961390 0x565192d88810 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.136+0000 7f1e5bc17740 1 --2- 172.21.15.43:0/1577962600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651929406b0 0x565192d10080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.136+0000 7f1e5bc17740 1 --2- 172.21.15.43:0/1577962600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651929384d0 0x565192d0fb40 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.136+0000 7f1e5bc17740 1 --2- 172.21.15.43:0/1577962600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56519291ff60 0x565192d147c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.136+0000 7f1e35ffb640 0 client.18858 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.136+0000 7f1e5bc17740 1 -- 172.21.15.43:0/1577962600 >> 172.21.15.43:0/1577962600 conn(0x5651929554e0 msgr2=0x565192d03660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.136+0000 7f1e5bc17740 1 -- 172.21.15.43:0/1577962600 shutdown_connections 2024-04-25T10:44:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.136+0000 7f1e5bc17740 1 -- 172.21.15.43:0/1577962600 wait complete. 2024-04-25T10:44:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.136+0000 7f1e5bc17740 20 client.18858 trim_cache size 0 max 16384 2024-04-25T10:44:00.145 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:44:00.145 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:44:00.145 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:44:00.145 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:44:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.136+0000 7f1e54d6c640 1 -- 172.21.15.43:0/439501742 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1e3c0afc60 msgr2=0x7f1e3c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.136+0000 7f1e54d6c640 1 --2- 172.21.15.43:0/439501742 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1e3c0afc60 0x7f1e3c0b2170 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1e480094e0 tx=0x7f1e4802f310 comp rx=0 tx=0).stop 2024-04-25T10:44:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.137+0000 7f1e54d6c640 1 -- 172.21.15.43:0/439501742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e501023f0 msgr2=0x7f1e5019b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.137+0000 7f1e54d6c640 1 --2- 172.21.15.43:0/439501742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e501023f0 0x7f1e5019b510 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000e9f0 tx=0x7f1e4000edf0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.137+0000 7f1e54d6c640 1 -- 172.21.15.43:0/439501742 shutdown_connections 2024-04-25T10:44:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.137+0000 7f1e54d6c640 1 --2- 172.21.15.43:0/439501742 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f1e3c0afc60 0x7f1e3c0b2170 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.137+0000 7f1e54d6c640 1 --2- 172.21.15.43:0/439501742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e50107440 0x7f1e5019bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.137+0000 7f1e54d6c640 1 --2- 172.21.15.43:0/439501742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e50106980 0x7f1e5019ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.137+0000 7f1e54d6c640 1 --2- 172.21.15.43:0/439501742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e501023f0 0x7f1e5019b510 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.137+0000 7f1e54d6c640 1 -- 172.21.15.43:0/439501742 >> 172.21.15.43:0/439501742 conn(0x7f1e500fd850 msgr2=0x7f1e50104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.137+0000 7f1e54d6c640 1 -- 172.21.15.43:0/439501742 shutdown_connections 2024-04-25T10:44:00.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.137+0000 7f1e54d6c640 1 -- 172.21.15.43:0/439501742 wait complete. 2024-04-25T10:44:00.150 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:44:00.151 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:44:00.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.260+0000 7f43788d8640 1 -- 172.21.15.43:0/4155935468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4374104bf0 msgr2=0x7f4374105000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.260+0000 7f43788d8640 1 --2- 172.21.15.43:0/4155935468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4374104bf0 0x7f4374105000 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f4364009900 tx=0x7f436402a410 comp rx=0 tx=0).stop 2024-04-25T10:44:00.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.260+0000 7f43788d8640 1 -- 172.21.15.43:0/4155935468 shutdown_connections 2024-04-25T10:44:00.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.260+0000 7f43788d8640 1 --2- 172.21.15.43:0/4155935468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f437410b650 0x7f437410da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.260+0000 7f43788d8640 1 --2- 172.21.15.43:0/4155935468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f437410e040 0x7f4374110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.260+0000 7f43788d8640 1 --2- 172.21.15.43:0/4155935468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4374104bf0 0x7f4374105000 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.260+0000 7f43788d8640 1 -- 172.21.15.43:0/4155935468 >> 172.21.15.43:0/4155935468 conn(0x7f4374078110 msgr2=0x7f4374078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.260+0000 7f43788d8640 1 -- 172.21.15.43:0/4155935468 shutdown_connections 2024-04-25T10:44:00.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.260+0000 7f43788d8640 1 -- 172.21.15.43:0/4155935468 wait complete. 2024-04-25T10:44:00.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.261+0000 7f43788d8640 1 Processor -- start 2024-04-25T10:44:00.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.261+0000 7f43788d8640 1 -- start start 2024-04-25T10:44:00.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.261+0000 7f43788d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4374104bf0 0x7f437419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.261+0000 7f43788d8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f437410b650 0x7f437419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.261+0000 7f43788d8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f437410e040 0x7f437419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.261+0000 7f43788d8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4374113ca0 con 0x7f4374104bf0 2024-04-25T10:44:00.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.261+0000 7f43788d8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4374113b20 con 0x7f437410e040 2024-04-25T10:44:00.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.261+0000 7f43788d8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4374113e20 con 0x7f437410b650 2024-04-25T10:44:00.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.261+0000 7f437259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4374104bf0 0x7f437419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.261+0000 7f437259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4374104bf0 0x7f437419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32878/0 (socket says 172.21.15.43:32878) 2024-04-25T10:44:00.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.261+0000 7f437259c640 1 -- 172.21.15.43:0/114010426 learned_addr learned my addr 172.21.15.43:0/114010426 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.262+0000 7f4372d9d640 1 --2- 172.21.15.43:0/114010426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f437410e040 0x7f437419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.262+0000 7f437259c640 1 -- 172.21.15.43:0/114010426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f437410b650 msgr2=0x7f437419ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.262+0000 7f4371d9b640 1 --2- 172.21.15.43:0/114010426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f437410b650 0x7f437419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.262+0000 7f437259c640 1 --2- 172.21.15.43:0/114010426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f437410b650 0x7f437419ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.262+0000 7f437259c640 1 -- 172.21.15.43:0/114010426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f437410e040 msgr2=0x7f437419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.262+0000 7f437259c640 1 --2- 172.21.15.43:0/114010426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f437410e040 0x7f437419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.262+0000 7f437259c640 1 -- 172.21.15.43:0/114010426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437419f480 con 0x7f4374104bf0 2024-04-25T10:44:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.262+0000 7f4372d9d640 1 --2- 172.21.15.43:0/114010426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f437410e040 0x7f437419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.262+0000 7f4371d9b640 1 --2- 172.21.15.43:0/114010426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f437410b650 0x7f437419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.262+0000 7f437259c640 1 --2- 172.21.15.43:0/114010426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4374104bf0 0x7f437419a6a0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f4364006010 tx=0x7f43640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.262+0000 7f43637fe640 1 -- 172.21.15.43:0/114010426 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436403f030 con 0x7f4374104bf0 2024-04-25T10:44:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.262+0000 7f43637fe640 1 -- 172.21.15.43:0/114010426 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4364003cb0 con 0x7f4374104bf0 2024-04-25T10:44:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.262+0000 7f43788d8640 1 -- 172.21.15.43:0/114010426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437419f710 con 0x7f4374104bf0 2024-04-25T10:44:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.262+0000 7f43637fe640 1 -- 172.21.15.43:0/114010426 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364035070 con 0x7f4374104bf0 2024-04-25T10:44:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.262+0000 7f43788d8640 1 -- 172.21.15.43:0/114010426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437419fc00 con 0x7f4374104bf0 2024-04-25T10:44:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.263+0000 7f43788d8640 1 -- 172.21.15.43:0/114010426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f437406a770 con 0x7f4374104bf0 2024-04-25T10:44:00.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.265+0000 7f43637fe640 1 -- 172.21.15.43:0/114010426 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f4364042030 con 0x7f4374104bf0 2024-04-25T10:44:00.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.265+0000 7f43637fe640 1 --2- 172.21.15.43:0/114010426 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f435c0afc60 0x7f435c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.265+0000 7f43637fe640 1 -- 172.21.15.43:0/114010426 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f43640ff8c0 con 0x7f4374104bf0 2024-04-25T10:44:00.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.265+0000 7f4371d9b640 1 --2- 172.21.15.43:0/114010426 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f435c0afc60 0x7f435c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.266+0000 7f4371d9b640 1 --2- 172.21.15.43:0/114010426 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f435c0afc60 0x7f435c0b2170 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f437410e9f0 tx=0x7f4368005d90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.269+0000 7f43637fe640 1 -- 172.21.15.43:0/114010426 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43640c8160 con 0x7f4374104bf0 2024-04-25T10:44:00.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:00 smithi043 ceph-mon[29478]: pgmap v357: 97 pgs: 97 active+clean; 2.6 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-25T10:44:00.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.494+0000 7f43788d8640 1 -- 172.21.15.43:0/114010426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f437406afa0 con 0x7f4374104bf0 2024-04-25T10:44:00.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.495+0000 7f43637fe640 1 -- 172.21.15.43:0/114010426 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7f43640c7a60 con 0x7f4374104bf0 2024-04-25T10:44:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:44:00.498 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:44:00.498 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:44:00.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.497+0000 7f43788d8640 1 -- 172.21.15.43:0/114010426 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f435c0afc60 msgr2=0x7f435c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.497+0000 7f43788d8640 1 --2- 172.21.15.43:0/114010426 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f435c0afc60 0x7f435c0b2170 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f437410e9f0 tx=0x7f4368005d90 comp rx=0 tx=0).stop 2024-04-25T10:44:00.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.497+0000 7f43788d8640 1 -- 172.21.15.43:0/114010426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4374104bf0 msgr2=0x7f437419a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.497+0000 7f43788d8640 1 --2- 172.21.15.43:0/114010426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4374104bf0 0x7f437419a6a0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f4364006010 tx=0x7f43640042f0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.497+0000 7f43788d8640 1 -- 172.21.15.43:0/114010426 shutdown_connections 2024-04-25T10:44:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.497+0000 7f43788d8640 1 --2- 172.21.15.43:0/114010426 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f435c0afc60 0x7f435c0b2170 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.497+0000 7f43788d8640 1 --2- 172.21.15.43:0/114010426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f437410e040 0x7f437419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.497+0000 7f43788d8640 1 --2- 172.21.15.43:0/114010426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f437410b650 0x7f437419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.497+0000 7f43788d8640 1 --2- 172.21.15.43:0/114010426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4374104bf0 0x7f437419a6a0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.497+0000 7f43788d8640 1 -- 172.21.15.43:0/114010426 >> 172.21.15.43:0/114010426 conn(0x7f4374078110 msgr2=0x7f4374101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.497+0000 7f43788d8640 1 -- 172.21.15.43:0/114010426 shutdown_connections 2024-04-25T10:44:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.497+0000 7f43788d8640 1 -- 172.21.15.43:0/114010426 wait complete. 2024-04-25T10:44:00.511 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:44:00.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:00 smithi164 ceph-mon[32713]: pgmap v357: 97 pgs: 97 active+clean; 2.6 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-25T10:44:00.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:00 smithi077 ceph-mon[32772]: pgmap v357: 97 pgs: 97 active+clean; 2.6 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-04-25T10:44:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.623+0000 7f9ff876c640 1 -- 172.21.15.43:0/180168005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ff010b730 msgr2=0x7f9ff010db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.623+0000 7f9ff876c640 1 --2- 172.21.15.43:0/180168005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ff010b730 0x7f9ff010db50 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009900 tx=0x7f9fe402a410 comp rx=0 tx=0).stop 2024-04-25T10:44:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.623+0000 7f9ff876c640 1 -- 172.21.15.43:0/180168005 shutdown_connections 2024-04-25T10:44:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.623+0000 7f9ff876c640 1 --2- 172.21.15.43:0/180168005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ff010e090 0x7f9ff0110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.623+0000 7f9ff876c640 1 --2- 172.21.15.43:0/180168005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ff010b730 0x7f9ff010db50 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.623+0000 7f9ff876c640 1 --2- 172.21.15.43:0/180168005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff0104cd0 0x7f9ff01050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.623+0000 7f9ff876c640 1 -- 172.21.15.43:0/180168005 >> 172.21.15.43:0/180168005 conn(0x7f9ff0078110 msgr2=0x7f9ff0078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.623+0000 7f9ff876c640 1 -- 172.21.15.43:0/180168005 shutdown_connections 2024-04-25T10:44:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.623+0000 7f9ff876c640 1 -- 172.21.15.43:0/180168005 wait complete. 2024-04-25T10:44:00.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.623+0000 7f9ff876c640 1 Processor -- start 2024-04-25T10:44:00.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.623+0000 7f9ff876c640 1 -- start start 2024-04-25T10:44:00.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.624+0000 7f9ff876c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ff0104cd0 0x7f9ff019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.624+0000 7f9ff876c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ff010b730 0x7f9ff019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.624+0000 7f9ff876c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff010e090 0x7f9ff019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.624+0000 7f9ff876c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0113ce0 con 0x7f9ff010b730 2024-04-25T10:44:00.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.624+0000 7f9ff876c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0113b60 con 0x7f9ff0104cd0 2024-04-25T10:44:00.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.624+0000 7f9ff876c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0113e60 con 0x7f9ff010e090 2024-04-25T10:44:00.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.624+0000 7f9ff5d07640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ff010b730 0x7f9ff019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.624+0000 7f9ff5d07640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ff010b730 0x7f9ff019acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32908/0 (socket says 172.21.15.43:32908) 2024-04-25T10:44:00.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.624+0000 7f9ff5d07640 1 -- 172.21.15.43:0/4275060104 learned_addr learned my addr 172.21.15.43:0/4275060104 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.624+0000 7f9ff6508640 1 --2- 172.21.15.43:0/4275060104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ff0104cd0 0x7f9ff019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.624+0000 7f9ff6d09640 1 --2- 172.21.15.43:0/4275060104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff010e090 0x7f9ff019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.624+0000 7f9ff5d07640 1 -- 172.21.15.43:0/4275060104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff010e090 msgr2=0x7f9ff019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.624+0000 7f9ff5d07640 1 --2- 172.21.15.43:0/4275060104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff010e090 0x7f9ff019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.624+0000 7f9ff5d07640 1 -- 172.21.15.43:0/4275060104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ff0104cd0 msgr2=0x7f9ff019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.624+0000 7f9ff5d07640 1 --2- 172.21.15.43:0/4275060104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ff0104cd0 0x7f9ff019a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.624+0000 7f9ff5d07640 1 -- 172.21.15.43:0/4275060104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff019f5c0 con 0x7f9ff010b730 2024-04-25T10:44:00.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.624+0000 7f9ff6d09640 1 --2- 172.21.15.43:0/4275060104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff010e090 0x7f9ff019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:00.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.624+0000 7f9ff5d07640 1 --2- 172.21.15.43:0/4275060104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ff010b730 0x7f9ff019acc0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4006010 tx=0x7f9fe4004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.625+0000 7f9fef7fe640 1 -- 172.21.15.43:0/4275060104 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe403f030 con 0x7f9ff010b730 2024-04-25T10:44:00.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.625+0000 7f9ff876c640 1 -- 172.21.15.43:0/4275060104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff019f850 con 0x7f9ff010b730 2024-04-25T10:44:00.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.625+0000 7f9fef7fe640 1 -- 172.21.15.43:0/4275060104 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fe4003aa0 con 0x7f9ff010b730 2024-04-25T10:44:00.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.625+0000 7f9fef7fe640 1 -- 172.21.15.43:0/4275060104 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4035070 con 0x7f9ff010b730 2024-04-25T10:44:00.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.625+0000 7f9ff876c640 1 -- 172.21.15.43:0/4275060104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff019fd90 con 0x7f9ff010b730 2024-04-25T10:44:00.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.627+0000 7f9fff61b740 1 Processor -- start 2024-04-25T10:44:00.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.628+0000 7f9fff61b740 1 -- start start 2024-04-25T10:44:00.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.628+0000 7f9fef7fe640 1 -- 172.21.15.43:0/4275060104 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f9fe4042030 con 0x7f9ff010b730 2024-04-25T10:44:00.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.628+0000 7f9fff61b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 0x55f61af20810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.628+0000 7f9fff61b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab564d0 0x55f61af20d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.628+0000 7f9fff61b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f61ab5e6b0 0x55f61af2d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.628+0000 7f9fff61b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f61ae85d80 con 0x55f61ab7f390 2024-04-25T10:44:00.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.628+0000 7f9fff61b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f61ae3fa20 con 0x55f61ab564d0 2024-04-25T10:44:00.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.628+0000 7f9fff61b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f61ae81d40 con 0x55f61ab5e6b0 2024-04-25T10:44:00.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.628+0000 7f9ff6d09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 0x55f61af20810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.628+0000 7f9ff6d09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 0x55f61af20810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32914/0 (socket says 172.21.15.43:32914) 2024-04-25T10:44:00.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.628+0000 7f9ff6d09640 1 -- 172.21.15.43:0/1619419310 learned_addr learned my addr 172.21.15.43:0/1619419310 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.628+0000 7f9ff5d07640 1 --2- 172.21.15.43:0/1619419310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f61ab5e6b0 0x55f61af2d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.628+0000 7f9ff6508640 1 --2- 172.21.15.43:0/1619419310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab564d0 0x55f61af20d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.628+0000 7f9ff6d09640 1 -- 172.21.15.43:0/1619419310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f61ab5e6b0 msgr2=0x55f61af2d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.628+0000 7f9fef7fe640 1 --2- 172.21.15.43:0/4275060104 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9fd80afc60 0x7f9fd80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.628+0000 7f9ff6d09640 1 --2- 172.21.15.43:0/1619419310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f61ab5e6b0 0x55f61af2d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.628+0000 7f9ff6d09640 1 -- 172.21.15.43:0/1619419310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab564d0 msgr2=0x55f61af20d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.628+0000 7f9ff6d09640 1 --2- 172.21.15.43:0/1619419310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab564d0 0x55f61af20d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.629+0000 7f9ff6d09640 1 -- 172.21.15.43:0/1619419310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61ae4ac30 con 0x55f61ab7f390 2024-04-25T10:44:00.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.629+0000 7f9ff6508640 1 --2- 172.21.15.43:0/1619419310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab564d0 0x55f61af20d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:00.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.629+0000 7f9fef7fe640 1 -- 172.21.15.43:0/4275060104 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f9fe40ff4f0 con 0x7f9ff010b730 2024-04-25T10:44:00.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.629+0000 7f9ff6d09640 1 --2- 172.21.15.43:0/4275060104 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9fd80afc60 0x7f9fd80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.629+0000 7f9ff6d09640 1 --2- 172.21.15.43:0/1619419310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 0x55f61af20810 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009e40 tx=0x7f9fe000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f2ed691fb70811 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.629+0000 7f9fecff9640 1 -- 172.21.15.43:0/1619419310 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0021070 con 0x55f61ab7f390 2024-04-25T10:44:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.629+0000 7f9fecff9640 1 -- 172.21.15.43:0/1619419310 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fe001bdb0 con 0x55f61ab7f390 2024-04-25T10:44:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.629+0000 7f9fecff9640 1 -- 172.21.15.43:0/1619419310 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe00109e0 con 0x55f61ab7f390 2024-04-25T10:44:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.630+0000 7f9fff61b740 1 -- 172.21.15.43:0/1619419310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 msgr2=0x55f61af20810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.630+0000 7f9ff6d09640 1 --2- 172.21.15.43:0/4275060104 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9fd80afc60 0x7f9fd80b2170 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9fe001e000 tx=0x7f9fe0021000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.630+0000 7f9fff61b740 1 --2- 172.21.15.43:0/1619419310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 0x55f61af20810 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009e40 tx=0x7f9fe000f730 comp rx=0 tx=0).stop 2024-04-25T10:44:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.630+0000 7f9fff61b740 1 -- 172.21.15.43:0/1619419310 shutdown_connections 2024-04-25T10:44:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.630+0000 7f9fff61b740 1 --2- 172.21.15.43:0/1619419310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab564d0 0x55f61af20d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.630+0000 7f9fff61b740 1 --2- 172.21.15.43:0/1619419310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f61ab5e6b0 0x55f61af2d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.630+0000 7f9fff61b740 1 --2- 172.21.15.43:0/1619419310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 0x55f61af20810 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.630+0000 7f9fff61b740 1 -- 172.21.15.43:0/1619419310 >> 172.21.15.43:0/1619419310 conn(0x55f61ab734e0 msgr2=0x55f61af1cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.630+0000 7f9fff61b740 1 -- 172.21.15.43:0/1619419310 shutdown_connections 2024-04-25T10:44:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.630+0000 7f9fff61b740 1 -- 172.21.15.43:0/1619419310 wait complete. 2024-04-25T10:44:00.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.631+0000 7f9fff61b740 1 -- start start 2024-04-25T10:44:00.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.631+0000 7f9fff61b740 1 Processor -- start 2024-04-25T10:44:00.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.631+0000 7f9fff61b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f61ab564d0 0x55f61af2b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.631+0000 7f9fff61b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab5e6b0 0x55f61af2b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.631+0000 7f9fff61b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab7f390 0x55f61af37210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.631+0000 7f9fff61b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f61af2dd90 con 0x55f61ab5e6b0 2024-04-25T10:44:00.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.631+0000 7f9fff61b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f61ae0cdc0 con 0x55f61ab7f390 2024-04-25T10:44:00.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.631+0000 7f9fff61b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f61af2df10 con 0x55f61ab564d0 2024-04-25T10:44:00.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.631+0000 7f9fff61b740 10 client.0 _ll_register_callbacks cb 0x55f61ae28600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:00.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.631+0000 7f9ff5d07640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab5e6b0 0x55f61af2b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.631+0000 7f9ff5d07640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab5e6b0 0x55f61af2b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32928/0 (socket says 172.21.15.43:32928) 2024-04-25T10:44:00.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.631+0000 7f9ff5d07640 1 -- 172.21.15.43:0/2030134787 learned_addr learned my addr 172.21.15.43:0/2030134787 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.631+0000 7f9ff6508640 1 --2- 172.21.15.43:0/2030134787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f61ab564d0 0x55f61af2b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.631+0000 7f9ff6d09640 1 --2- 172.21.15.43:0/2030134787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab7f390 0x55f61af37210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.631+0000 7f9ff5d07640 1 -- 172.21.15.43:0/2030134787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f61ab564d0 msgr2=0x55f61af2b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.631+0000 7f9ff5d07640 1 --2- 172.21.15.43:0/2030134787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f61ab564d0 0x55f61af2b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.631+0000 7f9ff5d07640 1 -- 172.21.15.43:0/2030134787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab7f390 msgr2=0x55f61af37210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.631+0000 7f9ff5d07640 1 --2- 172.21.15.43:0/2030134787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab7f390 0x55f61af37210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.631+0000 7f9ff5d07640 1 -- 172.21.15.43:0/2030134787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61ab973f0 con 0x55f61ab5e6b0 2024-04-25T10:44:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.631+0000 7f9ff6d09640 1 --2- 172.21.15.43:0/2030134787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab7f390 0x55f61af37210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.632+0000 7f9ff5d07640 1 --2- 172.21.15.43:0/2030134787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab5e6b0 0x55f61af2b850 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4004650 tx=0x7f9fe402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883e37603854790b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.632+0000 7f9fd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.632+0000 7f9fd5ffb640 1 -- 172.21.15.43:0/2030134787 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4049070 con 0x55f61ab5e6b0 2024-04-25T10:44:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.632+0000 7f9fd5ffb640 1 -- 172.21.15.43:0/2030134787 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f9fe40c3920 con 0x55f61ab5e6b0 2024-04-25T10:44:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.632+0000 7f9fff61b740 1 -- 172.21.15.43:0/2030134787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fe001bdb0 con 0x55f61ab5e6b0 2024-04-25T10:44:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.632+0000 7f9fd5ffb640 1 -- 172.21.15.43:0/2030134787 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9fe40c3550 con 0x55f61ab5e6b0 2024-04-25T10:44:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.632+0000 7f9fff61b740 10 client.18876 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.632+0000 7f9fff61b740 1 -- 172.21.15.43:0/2030134787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f61ac617d0 con 0x55f61ab5e6b0 2024-04-25T10:44:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.632+0000 7f9fd5ffb640 1 -- 172.21.15.43:0/2030134787 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fe001bdb0 con 0x55f61ab5e6b0 2024-04-25T10:44:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.632+0000 7f9fd5ffb640 1 -- 172.21.15.43:0/2030134787 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9fe40c78f0 con 0x55f61ab5e6b0 2024-04-25T10:44:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.632+0000 7f9fff61b740 10 client.18876 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.632+0000 7f9fff61b740 10 client.18876 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:44:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.632+0000 7f9fff61b740 20 client.18876 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.632+0000 7f9fff61b740 1 --2- 172.21.15.43:0/2030134787 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55f61ab3df60 0x55f61af3d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.632+0000 7f9fff61b740 4 client.18876 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:44:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.632+0000 7f9fff61b740 1 -- 172.21.15.43:0/2030134787 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f61ae93990 con 0x55f61ab3df60 2024-04-25T10:44:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.633+0000 7f9ff6508640 1 --2- 172.21.15.43:0/2030134787 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55f61ab3df60 0x55f61af3d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.633+0000 7f9ff6508640 1 --2- 172.21.15.43:0/2030134787 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55f61ab3df60 0x55f61af3d640 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e82130941597952 server_cookie=2537d4b8d3fe904e in_seq=0 out_seq=0 2024-04-25T10:44:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.633+0000 7f9fd5ffb640 10 client.18876 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:44:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.634+0000 7f9fd5ffb640 1 -- 172.21.15.43:0/2030134787 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f61ae93990 con 0x55f61ab3df60 2024-04-25T10:44:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.634+0000 7f9fd5ffb640 10 client.18876 handle_command_reply: tid=0 2024-04-25T10:44:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.634+0000 7f9fff61b740 1 client.18876 shutdown 2024-04-25T10:44:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.634+0000 7f9fff61b740 1 -- 172.21.15.43:0/2030134787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab5e6b0 msgr2=0x55f61af2b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.634+0000 7f9fff61b740 1 --2- 172.21.15.43:0/2030134787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab5e6b0 0x55f61af2b850 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4004650 tx=0x7f9fe402a410 comp rx=0 tx=0).stop 2024-04-25T10:44:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.634+0000 7f9fff61b740 1 -- 172.21.15.43:0/2030134787 shutdown_connections 2024-04-25T10:44:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.634+0000 7f9fff61b740 1 --2- 172.21.15.43:0/2030134787 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55f61ab3df60 0x55f61af3d640 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.634+0000 7f9fff61b740 1 --2- 172.21.15.43:0/2030134787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab7f390 0x55f61af37210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.634+0000 7f9fff61b740 1 --2- 172.21.15.43:0/2030134787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab5e6b0 0x55f61af2b850 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.634+0000 7f9fff61b740 1 --2- 172.21.15.43:0/2030134787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f61ab564d0 0x55f61af2b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.634+0000 7f9fd5ffb640 0 client.18876 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:44:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.634+0000 7f9fff61b740 1 -- 172.21.15.43:0/2030134787 >> 172.21.15.43:0/2030134787 conn(0x55f61ab734e0 msgr2=0x55f61af22530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.634+0000 7f9fff61b740 1 -- 172.21.15.43:0/2030134787 shutdown_connections 2024-04-25T10:44:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.634+0000 7f9fff61b740 1 -- 172.21.15.43:0/2030134787 wait complete. 2024-04-25T10:44:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.634+0000 7f9fff61b740 20 client.18876 trim_cache size 0 max 16384 2024-04-25T10:44:00.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.648+0000 7f9fff61b740 1 Processor -- start 2024-04-25T10:44:00.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.648+0000 7f9fff61b740 1 -- start start 2024-04-25T10:44:00.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.648+0000 7f9fff61b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab564d0 0x55f61af1d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.648+0000 7f9fff61b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f61ab5e6b0 0x55f61af36e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.648+0000 7f9fff61b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 0x55f61af25430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.648+0000 7f9fff61b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff01b2300 con 0x55f61ab7f390 2024-04-25T10:44:00.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.648+0000 7f9fff61b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff01afbd0 con 0x55f61ab564d0 2024-04-25T10:44:00.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.648+0000 7f9fff61b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff01ada40 con 0x55f61ab5e6b0 2024-04-25T10:44:00.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.648+0000 7f9ff6d09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 0x55f61af25430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.648+0000 7f9ff6d09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 0x55f61af25430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32934/0 (socket says 172.21.15.43:32934) 2024-04-25T10:44:00.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.648+0000 7f9ff6d09640 1 -- 172.21.15.43:0/541177998 learned_addr learned my addr 172.21.15.43:0/541177998 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.648+0000 7f9ff6d09640 1 -- 172.21.15.43:0/541177998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f61ab5e6b0 msgr2=0x55f61af36e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.648+0000 7f9ff6d09640 1 --2- 172.21.15.43:0/541177998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f61ab5e6b0 0x55f61af36e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.648+0000 7f9ff6508640 1 --2- 172.21.15.43:0/541177998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab564d0 0x55f61af1d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.648+0000 7f9ff6d09640 1 -- 172.21.15.43:0/541177998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab564d0 msgr2=0x55f61af1d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.648+0000 7f9ff6d09640 1 --2- 172.21.15.43:0/541177998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab564d0 0x55f61af1d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.648+0000 7f9ff6d09640 1 -- 172.21.15.43:0/541177998 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61af22010 con 0x55f61ab7f390 2024-04-25T10:44:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.649+0000 7f9ff6d09640 1 --2- 172.21.15.43:0/541177998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 0x55f61af25430 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0006010 tx=0x7f9fe0024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0853ec895c0d48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.649+0000 7f9fd7fff640 1 -- 172.21.15.43:0/541177998 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0034070 con 0x55f61ab7f390 2024-04-25T10:44:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.649+0000 7f9fd7fff640 1 -- 172.21.15.43:0/541177998 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fe001bdb0 con 0x55f61ab7f390 2024-04-25T10:44:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.649+0000 7f9fd7fff640 1 -- 172.21.15.43:0/541177998 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe00108d0 con 0x55f61ab7f390 2024-04-25T10:44:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.649+0000 7f9fff61b740 1 -- 172.21.15.43:0/541177998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 msgr2=0x55f61af25430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.649+0000 7f9fff61b740 1 --2- 172.21.15.43:0/541177998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 0x55f61af25430 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0006010 tx=0x7f9fe0024000 comp rx=0 tx=0).stop 2024-04-25T10:44:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.650+0000 7f9fff61b740 1 -- 172.21.15.43:0/541177998 shutdown_connections 2024-04-25T10:44:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.650+0000 7f9fff61b740 1 --2- 172.21.15.43:0/541177998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 0x55f61af25430 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.650+0000 7f9fff61b740 1 --2- 172.21.15.43:0/541177998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f61ab5e6b0 0x55f61af36e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.650+0000 7f9fff61b740 1 --2- 172.21.15.43:0/541177998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab564d0 0x55f61af1d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.650+0000 7f9fff61b740 1 -- 172.21.15.43:0/541177998 >> 172.21.15.43:0/541177998 conn(0x55f61ab734e0 msgr2=0x55f61af1c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.650+0000 7f9fff61b740 1 -- 172.21.15.43:0/541177998 shutdown_connections 2024-04-25T10:44:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.650+0000 7f9fff61b740 1 -- 172.21.15.43:0/541177998 wait complete. 2024-04-25T10:44:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.650+0000 7f9fff61b740 1 -- start start 2024-04-25T10:44:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.650+0000 7f9fff61b740 1 Processor -- start 2024-04-25T10:44:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.650+0000 7f9fff61b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab564d0 0x55f61af33260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.650+0000 7f9fff61b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f61ab5e6b0 0x55f61af337c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.650+0000 7f9fff61b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 0x55f61af2e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.650+0000 7f9fff61b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff01ae090 con 0x55f61ab7f390 2024-04-25T10:44:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.650+0000 7f9fff61b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff01af500 con 0x55f61ab564d0 2024-04-25T10:44:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.650+0000 7f9fff61b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff01ae760 con 0x55f61ab5e6b0 2024-04-25T10:44:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.650+0000 7f9fff61b740 10 client.0 _ll_register_callbacks cb 0x7f9fe4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.651+0000 7f9ff6d09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 0x55f61af2e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.651+0000 7f9ff6d09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 0x55f61af2e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32948/0 (socket says 172.21.15.43:32948) 2024-04-25T10:44:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.651+0000 7f9ff6d09640 1 -- 172.21.15.43:0/2217153191 learned_addr learned my addr 172.21.15.43:0/2217153191 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.651+0000 7f9ff5d07640 1 --2- 172.21.15.43:0/2217153191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f61ab5e6b0 0x55f61af337c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.651+0000 7f9ff6508640 1 --2- 172.21.15.43:0/2217153191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab564d0 0x55f61af33260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.651+0000 7f9ff6d09640 1 -- 172.21.15.43:0/2217153191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f61ab5e6b0 msgr2=0x55f61af337c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.651+0000 7f9ff6d09640 1 --2- 172.21.15.43:0/2217153191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f61ab5e6b0 0x55f61af337c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.651+0000 7f9ff6d09640 1 -- 172.21.15.43:0/2217153191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab564d0 msgr2=0x55f61af33260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.651+0000 7f9ff6d09640 1 --2- 172.21.15.43:0/2217153191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab564d0 0x55f61af33260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.651+0000 7f9ff6d09640 1 -- 172.21.15.43:0/2217153191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61af2ef80 con 0x55f61ab7f390 2024-04-25T10:44:00.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.651+0000 7f9ff6d09640 1 --2- 172.21.15.43:0/2217153191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 0x55f61af2e7f0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009640 tx=0x7f9fe0009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d9a2f13d4c07d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.651+0000 7f9fd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:00.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.651+0000 7f9fd5ffb640 1 -- 172.21.15.43:0/2217153191 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0034070 con 0x55f61ab7f390 2024-04-25T10:44:00.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.651+0000 7f9fd5ffb640 1 -- 172.21.15.43:0/2217153191 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f9fe001b880 con 0x55f61ab7f390 2024-04-25T10:44:00.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.651+0000 7f9fff61b740 1 -- 172.21.15.43:0/2217153191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fe001bdb0 con 0x55f61ab7f390 2024-04-25T10:44:00.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.651+0000 7f9fd5ffb640 1 -- 172.21.15.43:0/2217153191 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9fe001ba80 con 0x55f61ab7f390 2024-04-25T10:44:00.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.651+0000 7f9fff61b740 10 client.18882 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:00.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.651+0000 7f9fff61b740 1 -- 172.21.15.43:0/2217153191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f61afa53a0 con 0x55f61ab7f390 2024-04-25T10:44:00.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.652+0000 7f9fd5ffb640 1 -- 172.21.15.43:0/2217153191 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fe001bdb0 con 0x55f61ab7f390 2024-04-25T10:44:00.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.652+0000 7f9fd5ffb640 1 -- 172.21.15.43:0/2217153191 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9fe002e030 con 0x55f61ab7f390 2024-04-25T10:44:00.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.652+0000 7f9fff61b740 10 client.18882 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:00.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.652+0000 7f9fff61b740 10 client.18882 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:44:00.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.652+0000 7f9fff61b740 20 client.18882 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:00.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.652+0000 7f9fff61b740 1 --2- 172.21.15.43:0/2217153191 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55f61ab133a0 0x55f61afa7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.652+0000 7f9fff61b740 4 client.18882 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:44:00.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.652+0000 7f9fff61b740 1 -- 172.21.15.43:0/2217153191 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f61ae8d800 con 0x55f61ab133a0 2024-04-25T10:44:00.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.652+0000 7f9ff6508640 1 --2- 172.21.15.43:0/2217153191 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55f61ab133a0 0x55f61afa7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.653+0000 7f9fd5ffb640 10 client.18882 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:44:00.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.653+0000 7f9ff6508640 1 --2- 172.21.15.43:0/2217153191 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55f61ab133a0 0x55f61afa7960 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eaac1842e62d0f38 server_cookie=8bf9bad8c0cf2734 in_seq=0 out_seq=0 2024-04-25T10:44:00.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.653+0000 7f9fd5ffb640 1 -- 172.21.15.43:0/2217153191 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f61ae8d800 con 0x55f61ab133a0 2024-04-25T10:44:00.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.653+0000 7f9fd5ffb640 10 client.18882 handle_command_reply: tid=0 2024-04-25T10:44:00.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.653+0000 7f9fff61b740 1 client.18882 shutdown 2024-04-25T10:44:00.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.653+0000 7f9fff61b740 1 -- 172.21.15.43:0/2217153191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 msgr2=0x55f61af2e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.653+0000 7f9fff61b740 1 --2- 172.21.15.43:0/2217153191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 0x55f61af2e7f0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009640 tx=0x7f9fe0009740 comp rx=0 tx=0).stop 2024-04-25T10:44:00.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.653+0000 7f9fff61b740 1 -- 172.21.15.43:0/2217153191 shutdown_connections 2024-04-25T10:44:00.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.653+0000 7f9fff61b740 1 --2- 172.21.15.43:0/2217153191 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55f61ab133a0 0x55f61afa7960 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.653+0000 7f9fff61b740 1 --2- 172.21.15.43:0/2217153191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f61ab7f390 0x55f61af2e7f0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.653+0000 7f9fff61b740 1 --2- 172.21.15.43:0/2217153191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f61ab5e6b0 0x55f61af337c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.653+0000 7f9fff61b740 1 --2- 172.21.15.43:0/2217153191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f61ab564d0 0x55f61af33260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.653+0000 7f9fd5ffb640 0 client.18882 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:44:00.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.653+0000 7f9fff61b740 1 -- 172.21.15.43:0/2217153191 >> 172.21.15.43:0/2217153191 conn(0x55f61ab734e0 msgr2=0x55f61af35e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.653+0000 7f9fff61b740 1 -- 172.21.15.43:0/2217153191 shutdown_connections 2024-04-25T10:44:00.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.653+0000 7f9fff61b740 1 -- 172.21.15.43:0/2217153191 wait complete. 2024-04-25T10:44:00.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.653+0000 7f9fff61b740 20 client.18882 trim_cache size 0 max 16384 2024-04-25T10:44:00.661 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:44:00.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.654+0000 7f9ff876c640 1 -- 172.21.15.43:0/4275060104 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9fd80afc60 msgr2=0x7f9fd80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.654+0000 7f9ff876c640 1 --2- 172.21.15.43:0/4275060104 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9fd80afc60 0x7f9fd80b2170 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9fe001e000 tx=0x7f9fe0021000 comp rx=0 tx=0).stop 2024-04-25T10:44:00.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.654+0000 7f9ff876c640 1 -- 172.21.15.43:0/4275060104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ff010b730 msgr2=0x7f9ff019acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.654+0000 7f9ff876c640 1 --2- 172.21.15.43:0/4275060104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ff010b730 0x7f9ff019acc0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4006010 tx=0x7f9fe4004060 comp rx=0 tx=0).stop 2024-04-25T10:44:00.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.654+0000 7f9ff876c640 1 -- 172.21.15.43:0/4275060104 shutdown_connections 2024-04-25T10:44:00.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.654+0000 7f9ff876c640 1 --2- 172.21.15.43:0/4275060104 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f9fd80afc60 0x7f9fd80b2170 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.654+0000 7f9ff876c640 1 --2- 172.21.15.43:0/4275060104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff010e090 0x7f9ff019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.654+0000 7f9ff876c640 1 --2- 172.21.15.43:0/4275060104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ff010b730 0x7f9ff019acc0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.654+0000 7f9ff876c640 1 --2- 172.21.15.43:0/4275060104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ff0104cd0 0x7f9ff019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.654+0000 7f9ff876c640 1 -- 172.21.15.43:0/4275060104 >> 172.21.15.43:0/4275060104 conn(0x7f9ff0078110 msgr2=0x7f9ff0101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.654+0000 7f9ff876c640 1 -- 172.21.15.43:0/4275060104 shutdown_connections 2024-04-25T10:44:00.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.654+0000 7f9ff876c640 1 -- 172.21.15.43:0/4275060104 wait complete. 2024-04-25T10:44:00.667 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:44:00.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.778+0000 7f3ef93c5640 1 -- 172.21.15.43:0/144962052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef410e050 msgr2=0x7f3ef4110560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.778+0000 7f3ef93c5640 1 --2- 172.21.15.43:0/144962052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef410e050 0x7f3ef4110560 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009ca0 tx=0x7f3eec02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.778+0000 7f3ef93c5640 1 -- 172.21.15.43:0/144962052 shutdown_connections 2024-04-25T10:44:00.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.778+0000 7f3ef93c5640 1 --2- 172.21.15.43:0/144962052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ef410b660 0x7f3ef410da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.778+0000 7f3ef93c5640 1 --2- 172.21.15.43:0/144962052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef410e050 0x7f3ef4110560 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.778+0000 7f3ef93c5640 1 --2- 172.21.15.43:0/144962052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ef4104c00 0x7f3ef4105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.778+0000 7f3ef93c5640 1 -- 172.21.15.43:0/144962052 >> 172.21.15.43:0/144962052 conn(0x7f3ef40fe260 msgr2=0x7f3ef4100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.778+0000 7f3ef93c5640 1 -- 172.21.15.43:0/144962052 shutdown_connections 2024-04-25T10:44:00.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.778+0000 7f3ef93c5640 1 -- 172.21.15.43:0/144962052 wait complete. 2024-04-25T10:44:00.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.779+0000 7f3ef93c5640 1 Processor -- start 2024-04-25T10:44:00.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.779+0000 7f3ef93c5640 1 -- start start 2024-04-25T10:44:00.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.779+0000 7f3ef93c5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ef4104c00 0x7f3ef419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.779+0000 7f3ef93c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef410b660 0x7f3ef419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.779+0000 7f3ef93c5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ef410e050 0x7f3ef419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.779+0000 7f3ef93c5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef4113cd0 con 0x7f3ef410b660 2024-04-25T10:44:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.779+0000 7f3ef93c5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef4113b50 con 0x7f3ef410e050 2024-04-25T10:44:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.779+0000 7f3ef93c5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef4113e50 con 0x7f3ef4104c00 2024-04-25T10:44:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.780+0000 7f3eebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef410b660 0x7f3ef419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.780+0000 7f3eebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef410b660 0x7f3ef419acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32964/0 (socket says 172.21.15.43:32964) 2024-04-25T10:44:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.780+0000 7f3eebfff640 1 -- 172.21.15.43:0/1053178324 learned_addr learned my addr 172.21.15.43:0/1053178324 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.780+0000 7f3ef37fe640 1 --2- 172.21.15.43:0/1053178324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ef410e050 0x7f3ef419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.780+0000 7f3ef2ffd640 1 --2- 172.21.15.43:0/1053178324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ef4104c00 0x7f3ef419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.780+0000 7f3eebfff640 1 -- 172.21.15.43:0/1053178324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ef4104c00 msgr2=0x7f3ef419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.780+0000 7f3eebfff640 1 --2- 172.21.15.43:0/1053178324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ef4104c00 0x7f3ef419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.780+0000 7f3eebfff640 1 -- 172.21.15.43:0/1053178324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ef410e050 msgr2=0x7f3ef419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.780+0000 7f3eebfff640 1 --2- 172.21.15.43:0/1053178324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ef410e050 0x7f3ef419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.780+0000 7f3eebfff640 1 -- 172.21.15.43:0/1053178324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef419f530 con 0x7f3ef410b660 2024-04-25T10:44:00.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.780+0000 7f3eebfff640 1 --2- 172.21.15.43:0/1053178324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef410b660 0x7f3ef419acb0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00d980 tx=0x7f3edc00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.780+0000 7f3ef0ff9640 1 -- 172.21.15.43:0/1053178324 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc017070 con 0x7f3ef410b660 2024-04-25T10:44:00.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.780+0000 7f3ef0ff9640 1 -- 172.21.15.43:0/1053178324 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3edc009c30 con 0x7f3ef410b660 2024-04-25T10:44:00.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.780+0000 7f3ef0ff9640 1 -- 172.21.15.43:0/1053178324 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc005630 con 0x7f3ef410b660 2024-04-25T10:44:00.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.780+0000 7f3ef93c5640 1 -- 172.21.15.43:0/1053178324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef419f820 con 0x7f3ef410b660 2024-04-25T10:44:00.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.780+0000 7f3ef93c5640 1 -- 172.21.15.43:0/1053178324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef419fd20 con 0x7f3ef410b660 2024-04-25T10:44:00.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.783+0000 7f3f00278740 1 Processor -- start 2024-04-25T10:44:00.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.784+0000 7f3f00278740 1 -- start start 2024-04-25T10:44:00.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.784+0000 7f3ef0ff9640 1 -- 172.21.15.43:0/1053178324 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f3edc004020 con 0x7f3ef410b660 2024-04-25T10:44:00.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.784+0000 7f3f00278740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f70390 0x55bcd3311810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.784+0000 7f3f00278740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f474d0 0x55bcd3311d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.784+0000 7f3f00278740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f4f6b0 0x55bcd331e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.784+0000 7f3f00278740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bcd3276d80 con 0x55bcd2f4f6b0 2024-04-25T10:44:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.784+0000 7f3f00278740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bcd3230a20 con 0x55bcd2f474d0 2024-04-25T10:44:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.784+0000 7f3f00278740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bcd3272d40 con 0x55bcd2f70390 2024-04-25T10:44:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.784+0000 7f3eebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f4f6b0 0x55bcd331e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.784+0000 7f3eebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f4f6b0 0x55bcd331e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32978/0 (socket says 172.21.15.43:32978) 2024-04-25T10:44:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.784+0000 7f3eebfff640 1 -- 172.21.15.43:0/995988272 learned_addr learned my addr 172.21.15.43:0/995988272 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.784+0000 7f3ef2ffd640 1 --2- 172.21.15.43:0/995988272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f474d0 0x55bcd3311d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.784+0000 7f3ef37fe640 1 --2- 172.21.15.43:0/995988272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f70390 0x55bcd3311810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.785+0000 7f3eebfff640 1 -- 172.21.15.43:0/995988272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f70390 msgr2=0x55bcd3311810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.785+0000 7f3eebfff640 1 --2- 172.21.15.43:0/995988272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f70390 0x55bcd3311810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.785+0000 7f3eebfff640 1 -- 172.21.15.43:0/995988272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f474d0 msgr2=0x55bcd3311d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.785+0000 7f3eebfff640 1 --2- 172.21.15.43:0/995988272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f474d0 0x55bcd3311d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.785+0000 7f3eebfff640 1 -- 172.21.15.43:0/995988272 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcd323bc30 con 0x55bcd2f4f6b0 2024-04-25T10:44:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.785+0000 7f3ef37fe640 1 --2- 172.21.15.43:0/995988272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f70390 0x55bcd3311810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.785+0000 7f3ef0ff9640 1 --2- 172.21.15.43:0/1053178324 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3ee00afc60 0x7f3ee00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.785+0000 7f3ef37fe640 1 --2- 172.21.15.43:0/1053178324 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3ee00afc60 0x7f3ee00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.785+0000 7f3eebfff640 1 --2- 172.21.15.43:0/995988272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f4f6b0 0x55bcd331e6a0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0dc2e0 tx=0x7f3edc0e7760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb7653b45d22bd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.785+0000 7f3ef0ff9640 1 -- 172.21.15.43:0/1053178324 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f3edc0dec20 con 0x7f3ef410b660 2024-04-25T10:44:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.785+0000 7f3ee97fa640 1 -- 172.21.15.43:0/995988272 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc0a2c80 con 0x55bcd2f4f6b0 2024-04-25T10:44:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.785+0000 7f3ee97fa640 1 -- 172.21.15.43:0/995988272 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3edc0a68d0 con 0x55bcd2f4f6b0 2024-04-25T10:44:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.785+0000 7f3ee97fa640 1 -- 172.21.15.43:0/995988272 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc0a6530 con 0x55bcd2f4f6b0 2024-04-25T10:44:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.786+0000 7f3f00278740 1 -- 172.21.15.43:0/995988272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f4f6b0 msgr2=0x55bcd331e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.786+0000 7f3ef37fe640 1 --2- 172.21.15.43:0/1053178324 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3ee00afc60 0x7f3ee00b2170 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55bcd330e980 tx=0x7f3eec02f390 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.786+0000 7f3f00278740 1 --2- 172.21.15.43:0/995988272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f4f6b0 0x55bcd331e6a0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0dc2e0 tx=0x7f3edc0e7760 comp rx=0 tx=0).stop 2024-04-25T10:44:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.786+0000 7f3f00278740 1 -- 172.21.15.43:0/995988272 shutdown_connections 2024-04-25T10:44:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.786+0000 7f3f00278740 1 --2- 172.21.15.43:0/995988272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f474d0 0x55bcd3311d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.786+0000 7f3f00278740 1 --2- 172.21.15.43:0/995988272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f4f6b0 0x55bcd331e6a0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.786+0000 7f3f00278740 1 --2- 172.21.15.43:0/995988272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f70390 0x55bcd3311810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.786+0000 7f3f00278740 1 -- 172.21.15.43:0/995988272 >> 172.21.15.43:0/995988272 conn(0x55bcd2f644e0 msgr2=0x55bcd330dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.786+0000 7f3f00278740 1 -- 172.21.15.43:0/995988272 shutdown_connections 2024-04-25T10:44:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.786+0000 7f3f00278740 1 -- 172.21.15.43:0/995988272 wait complete. 2024-04-25T10:44:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.787+0000 7f3f00278740 1 -- start start 2024-04-25T10:44:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.787+0000 7f3f00278740 1 Processor -- start 2024-04-25T10:44:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.787+0000 7f3f00278740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f474d0 0x55bcd331c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.787+0000 7f3f00278740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f4f6b0 0x55bcd331c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.787+0000 7f3f00278740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f70390 0x55bcd3328210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.787+0000 7f3f00278740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bcd331ed90 con 0x55bcd2f474d0 2024-04-25T10:44:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.787+0000 7f3f00278740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bcd31fddc0 con 0x55bcd2f4f6b0 2024-04-25T10:44:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.787+0000 7f3f00278740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bcd331ef10 con 0x55bcd2f70390 2024-04-25T10:44:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.787+0000 7f3f00278740 10 client.0 _ll_register_callbacks cb 0x55bcd3219600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.787+0000 7f3ef2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f474d0 0x55bcd331c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.787+0000 7f3ef2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f474d0 0x55bcd331c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32982/0 (socket says 172.21.15.43:32982) 2024-04-25T10:44:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.787+0000 7f3ef2ffd640 1 -- 172.21.15.43:0/3512409099 learned_addr learned my addr 172.21.15.43:0/3512409099 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.787+0000 7f3eebfff640 1 --2- 172.21.15.43:0/3512409099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f4f6b0 0x55bcd331c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.787+0000 7f3ef2ffd640 1 -- 172.21.15.43:0/3512409099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f70390 msgr2=0x55bcd3328210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.787+0000 7f3ef2ffd640 1 --2- 172.21.15.43:0/3512409099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f70390 0x55bcd3328210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.787+0000 7f3ef2ffd640 1 -- 172.21.15.43:0/3512409099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f4f6b0 msgr2=0x55bcd331c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.787+0000 7f3ef2ffd640 1 --2- 172.21.15.43:0/3512409099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f4f6b0 0x55bcd331c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.787+0000 7f3ef2ffd640 1 -- 172.21.15.43:0/3512409099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcd2f883f0 con 0x55bcd2f474d0 2024-04-25T10:44:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.787+0000 7f3eebfff640 1 --2- 172.21.15.43:0/3512409099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f4f6b0 0x55bcd331c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.788+0000 7f3ef2ffd640 1 --2- 172.21.15.43:0/3512409099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f474d0 0x55bcd331c260 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4011830 tx=0x7f3ee4011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e77ef024dfb17c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.788+0000 7f3eda7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.788+0000 7f3eda7fc640 1 -- 172.21.15.43:0/3512409099 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee401e060 con 0x55bcd2f474d0 2024-04-25T10:44:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.788+0000 7f3eda7fc640 1 -- 172.21.15.43:0/3512409099 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f3ee400fdb0 con 0x55bcd2f474d0 2024-04-25T10:44:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.788+0000 7f3f00278740 1 -- 172.21.15.43:0/3512409099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3edc0a68d0 con 0x55bcd2f474d0 2024-04-25T10:44:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.788+0000 7f3eda7fc640 1 -- 172.21.15.43:0/3512409099 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ee4022410 con 0x55bcd2f474d0 2024-04-25T10:44:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.788+0000 7f3f00278740 10 client.18894 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.788+0000 7f3f00278740 1 -- 172.21.15.43:0/3512409099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bcd30527d0 con 0x55bcd2f474d0 2024-04-25T10:44:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.788+0000 7f3eda7fc640 1 -- 172.21.15.43:0/3512409099 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3edc0a68d0 con 0x55bcd2f474d0 2024-04-25T10:44:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.788+0000 7f3eda7fc640 1 -- 172.21.15.43:0/3512409099 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3ee4018030 con 0x55bcd2f474d0 2024-04-25T10:44:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.788+0000 7f3f00278740 10 client.18894 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.788+0000 7f3f00278740 10 client.18894 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:44:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.788+0000 7f3f00278740 20 client.18894 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.788+0000 7f3f00278740 1 --2- 172.21.15.43:0/3512409099 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bcd2f2ef60 0x55bcd332e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.788+0000 7f3f00278740 4 client.18894 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:44:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.788+0000 7f3f00278740 1 -- 172.21.15.43:0/3512409099 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcd3284990 con 0x55bcd2f2ef60 2024-04-25T10:44:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.789+0000 7f3eebfff640 1 --2- 172.21.15.43:0/3512409099 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bcd2f2ef60 0x55bcd332e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.789+0000 7f3eda7fc640 10 client.18894 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:44:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.789+0000 7f3eebfff640 1 --2- 172.21.15.43:0/3512409099 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bcd2f2ef60 0x55bcd332e770 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e096300ac602fb0a server_cookie=ec7defec4e312495 in_seq=0 out_seq=0 2024-04-25T10:44:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.790+0000 7f3eda7fc640 1 -- 172.21.15.43:0/3512409099 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bcd3284990 con 0x55bcd2f2ef60 2024-04-25T10:44:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.790+0000 7f3eda7fc640 10 client.18894 handle_command_reply: tid=0 2024-04-25T10:44:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.790+0000 7f3f00278740 1 client.18894 shutdown 2024-04-25T10:44:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.790+0000 7f3f00278740 1 -- 172.21.15.43:0/3512409099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f474d0 msgr2=0x55bcd331c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.790+0000 7f3f00278740 1 --2- 172.21.15.43:0/3512409099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f474d0 0x55bcd331c260 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4011830 tx=0x7f3ee4011cf0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.790+0000 7f3f00278740 1 -- 172.21.15.43:0/3512409099 shutdown_connections 2024-04-25T10:44:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.790+0000 7f3f00278740 1 --2- 172.21.15.43:0/3512409099 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bcd2f2ef60 0x55bcd332e770 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.790+0000 7f3f00278740 1 --2- 172.21.15.43:0/3512409099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f4f6b0 0x55bcd331c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.790+0000 7f3f00278740 1 --2- 172.21.15.43:0/3512409099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f70390 0x55bcd3328210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.790+0000 7f3eda7fc640 0 client.18894 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:44:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.790+0000 7f3f00278740 1 --2- 172.21.15.43:0/3512409099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f474d0 0x55bcd331c260 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.790+0000 7f3f00278740 1 -- 172.21.15.43:0/3512409099 >> 172.21.15.43:0/3512409099 conn(0x55bcd2f644e0 msgr2=0x55bcd3313530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.791+0000 7f3f00278740 1 -- 172.21.15.43:0/3512409099 shutdown_connections 2024-04-25T10:44:00.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.791+0000 7f3f00278740 1 -- 172.21.15.43:0/3512409099 wait complete. 2024-04-25T10:44:00.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.791+0000 7f3f00278740 20 client.18894 trim_cache size 0 max 16384 2024-04-25T10:44:00.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.804+0000 7f3f00278740 1 Processor -- start 2024-04-25T10:44:00.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.804+0000 7f3f00278740 1 -- start start 2024-04-25T10:44:00.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.805+0000 7f3f00278740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f474d0 0x55bcd331af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.805+0000 7f3f00278740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f4f6b0 0x55bcd330df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.805+0000 7f3f00278740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f70390 0x55bcd3327e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.805+0000 7f3f00278740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41b22b0 con 0x55bcd2f474d0 2024-04-25T10:44:00.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.805+0000 7f3f00278740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41afb80 con 0x55bcd2f70390 2024-04-25T10:44:00.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.805+0000 7f3f00278740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41ad9f0 con 0x55bcd2f4f6b0 2024-04-25T10:44:00.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.805+0000 7f3ef2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f474d0 0x55bcd331af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.805+0000 7f3ef2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f474d0 0x55bcd331af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32998/0 (socket says 172.21.15.43:32998) 2024-04-25T10:44:00.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.805+0000 7f3ef2ffd640 1 -- 172.21.15.43:0/1281149082 learned_addr learned my addr 172.21.15.43:0/1281149082 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.805+0000 7f3eebfff640 1 --2- 172.21.15.43:0/1281149082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f4f6b0 0x55bcd330df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.805+0000 7f3ef37fe640 1 --2- 172.21.15.43:0/1281149082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f70390 0x55bcd3327e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.805+0000 7f3ef2ffd640 1 -- 172.21.15.43:0/1281149082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f4f6b0 msgr2=0x55bcd330df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.805+0000 7f3ef2ffd640 1 --2- 172.21.15.43:0/1281149082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f4f6b0 0x55bcd330df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.805+0000 7f3ef2ffd640 1 -- 172.21.15.43:0/1281149082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f70390 msgr2=0x55bcd3327e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.805+0000 7f3ef2ffd640 1 --2- 172.21.15.43:0/1281149082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f70390 0x55bcd3327e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.805+0000 7f3ef2ffd640 1 -- 172.21.15.43:0/1281149082 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcd3329470 con 0x55bcd2f474d0 2024-04-25T10:44:00.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.805+0000 7f3ef2ffd640 1 --2- 172.21.15.43:0/1281149082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f474d0 0x55bcd331af90 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4011800 tx=0x7f3ee400e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8b249bddf7284c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.805+0000 7f3ee8ff9640 1 -- 172.21.15.43:0/1281149082 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee400fdb0 con 0x55bcd2f474d0 2024-04-25T10:44:00.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.805+0000 7f3ee8ff9640 1 -- 172.21.15.43:0/1281149082 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ee400ea50 con 0x55bcd2f474d0 2024-04-25T10:44:00.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.805+0000 7f3ee8ff9640 1 -- 172.21.15.43:0/1281149082 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4005600 con 0x55bcd2f474d0 2024-04-25T10:44:00.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.806+0000 7f3f00278740 1 -- 172.21.15.43:0/1281149082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f474d0 msgr2=0x55bcd331af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.806+0000 7f3f00278740 1 --2- 172.21.15.43:0/1281149082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f474d0 0x55bcd331af90 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4011800 tx=0x7f3ee400e860 comp rx=0 tx=0).stop 2024-04-25T10:44:00.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.806+0000 7f3f00278740 1 -- 172.21.15.43:0/1281149082 shutdown_connections 2024-04-25T10:44:00.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.806+0000 7f3f00278740 1 --2- 172.21.15.43:0/1281149082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f70390 0x55bcd3327e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.806+0000 7f3f00278740 1 --2- 172.21.15.43:0/1281149082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f4f6b0 0x55bcd330df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.806+0000 7f3f00278740 1 --2- 172.21.15.43:0/1281149082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f474d0 0x55bcd331af90 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.806+0000 7f3f00278740 1 -- 172.21.15.43:0/1281149082 >> 172.21.15.43:0/1281149082 conn(0x55bcd2f644e0 msgr2=0x55bcd330d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.806+0000 7f3f00278740 1 -- 172.21.15.43:0/1281149082 shutdown_connections 2024-04-25T10:44:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.806+0000 7f3f00278740 1 -- 172.21.15.43:0/1281149082 wait complete. 2024-04-25T10:44:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.807+0000 7f3f00278740 1 -- start start 2024-04-25T10:44:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.807+0000 7f3f00278740 1 Processor -- start 2024-04-25T10:44:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.807+0000 7f3f00278740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f474d0 0x55bcd331f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.807+0000 7f3f00278740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f4f6b0 0x55bcd331f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.807+0000 7f3f00278740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f70390 0x55bcd3394ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.807+0000 7f3f00278740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41ae040 con 0x55bcd2f4f6b0 2024-04-25T10:44:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.807+0000 7f3f00278740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41af4b0 con 0x55bcd2f70390 2024-04-25T10:44:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.807+0000 7f3f00278740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41ae710 con 0x55bcd2f474d0 2024-04-25T10:44:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.807+0000 7f3f00278740 10 client.0 _ll_register_callbacks cb 0x7f3ee400c6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:00.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.807+0000 7f3eebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f4f6b0 0x55bcd331f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.807+0000 7f3eebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f4f6b0 0x55bcd331f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33012/0 (socket says 172.21.15.43:33012) 2024-04-25T10:44:00.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.807+0000 7f3eebfff640 1 -- 172.21.15.43:0/180459805 learned_addr learned my addr 172.21.15.43:0/180459805 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.807+0000 7f3ef37fe640 1 --2- 172.21.15.43:0/180459805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f70390 0x55bcd3394ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.807+0000 7f3ef2ffd640 1 --2- 172.21.15.43:0/180459805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f474d0 0x55bcd331f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.807+0000 7f3eebfff640 1 -- 172.21.15.43:0/180459805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f474d0 msgr2=0x55bcd331f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.807+0000 7f3eebfff640 1 --2- 172.21.15.43:0/180459805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f474d0 0x55bcd331f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.807+0000 7f3eebfff640 1 -- 172.21.15.43:0/180459805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f70390 msgr2=0x55bcd3394ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.808+0000 7f3eebfff640 1 --2- 172.21.15.43:0/180459805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f70390 0x55bcd3394ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.808+0000 7f3eebfff640 1 -- 172.21.15.43:0/180459805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcd331ff50 con 0x55bcd2f4f6b0 2024-04-25T10:44:00.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.808+0000 7f3ef2ffd640 1 --2- 172.21.15.43:0/180459805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f474d0 0x55bcd331f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:00.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.808+0000 7f3eebfff640 1 --2- 172.21.15.43:0/180459805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f4f6b0 0x55bcd331f6f0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0dc970 tx=0x7f3edc0e2da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37b065f95d61f85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.808+0000 7f3eda7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:00.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.808+0000 7f3eda7fc640 1 -- 172.21.15.43:0/180459805 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc0e1040 con 0x55bcd2f4f6b0 2024-04-25T10:44:00.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.808+0000 7f3f00278740 1 -- 172.21.15.43:0/180459805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ee400ea50 con 0x55bcd2f4f6b0 2024-04-25T10:44:00.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.808+0000 7f3eda7fc640 1 -- 172.21.15.43:0/180459805 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f3edc0e2540 con 0x55bcd2f4f6b0 2024-04-25T10:44:00.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.808+0000 7f3eda7fc640 1 -- 172.21.15.43:0/180459805 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3edc0a6710 con 0x55bcd2f4f6b0 2024-04-25T10:44:00.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.808+0000 7f3f00278740 10 client.18900 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:00.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.808+0000 7f3f00278740 1 -- 172.21.15.43:0/180459805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bcd3396020 con 0x55bcd2f4f6b0 2024-04-25T10:44:00.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.808+0000 7f3eda7fc640 1 -- 172.21.15.43:0/180459805 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ee400ea50 con 0x55bcd2f4f6b0 2024-04-25T10:44:00.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.808+0000 7f3eda7fc640 1 -- 172.21.15.43:0/180459805 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3edc0bf040 con 0x55bcd2f4f6b0 2024-04-25T10:44:00.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.808+0000 7f3f00278740 10 client.18900 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:00.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.808+0000 7f3f00278740 10 client.18900 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:44:00.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.808+0000 7f3f00278740 20 client.18900 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:00.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.808+0000 7f3f00278740 1 --2- 172.21.15.43:0/180459805 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bcd2f043a0 0x55bcd3398370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.809+0000 7f3f00278740 4 client.18900 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:44:00.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.809+0000 7f3f00278740 1 -- 172.21.15.43:0/180459805 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bcd327e800 con 0x55bcd2f043a0 2024-04-25T10:44:00.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.809+0000 7f3ef2ffd640 1 --2- 172.21.15.43:0/180459805 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bcd2f043a0 0x55bcd3398370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.809+0000 7f3ef2ffd640 1 --2- 172.21.15.43:0/180459805 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bcd2f043a0 0x55bcd3398370 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=890eee5c424c52b9 server_cookie=91ca999ad12d18e5 in_seq=0 out_seq=0 2024-04-25T10:44:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.809+0000 7f3eda7fc640 10 client.18900 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:44:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.809+0000 7f3eda7fc640 1 -- 172.21.15.43:0/180459805 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bcd327e800 con 0x55bcd2f043a0 2024-04-25T10:44:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.809+0000 7f3eda7fc640 10 client.18900 handle_command_reply: tid=0 2024-04-25T10:44:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.809+0000 7f3f00278740 1 client.18900 shutdown 2024-04-25T10:44:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3f00278740 1 -- 172.21.15.43:0/180459805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f4f6b0 msgr2=0x55bcd331f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3f00278740 1 --2- 172.21.15.43:0/180459805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f4f6b0 0x55bcd331f6f0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0dc970 tx=0x7f3edc0e2da0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3f00278740 1 -- 172.21.15.43:0/180459805 shutdown_connections 2024-04-25T10:44:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3f00278740 1 --2- 172.21.15.43:0/180459805 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x55bcd2f043a0 0x55bcd3398370 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3f00278740 1 --2- 172.21.15.43:0/180459805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcd2f70390 0x55bcd3394ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3f00278740 1 --2- 172.21.15.43:0/180459805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcd2f4f6b0 0x55bcd331f6f0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3eda7fc640 0 client.18900 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:44:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3f00278740 1 --2- 172.21.15.43:0/180459805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcd2f474d0 0x55bcd331f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3f00278740 1 -- 172.21.15.43:0/180459805 >> 172.21.15.43:0/180459805 conn(0x55bcd2f644e0 msgr2=0x55bcd331d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3f00278740 1 -- 172.21.15.43:0/180459805 shutdown_connections 2024-04-25T10:44:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3f00278740 1 -- 172.21.15.43:0/180459805 wait complete. 2024-04-25T10:44:00.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3f00278740 20 client.18900 trim_cache size 0 max 16384 2024-04-25T10:44:00.818 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:44:00.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3ef93c5640 1 -- 172.21.15.43:0/1053178324 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3ee00afc60 msgr2=0x7f3ee00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3ef93c5640 1 --2- 172.21.15.43:0/1053178324 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3ee00afc60 0x7f3ee00b2170 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55bcd330e980 tx=0x7f3eec02f390 comp rx=0 tx=0).stop 2024-04-25T10:44:00.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3ef93c5640 1 -- 172.21.15.43:0/1053178324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef410b660 msgr2=0x7f3ef419acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3ef93c5640 1 --2- 172.21.15.43:0/1053178324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef410b660 0x7f3ef419acb0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00d980 tx=0x7f3edc00de40 comp rx=0 tx=0).stop 2024-04-25T10:44:00.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3ef93c5640 1 -- 172.21.15.43:0/1053178324 shutdown_connections 2024-04-25T10:44:00.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3ef93c5640 1 --2- 172.21.15.43:0/1053178324 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f3ee00afc60 0x7f3ee00b2170 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3ef93c5640 1 --2- 172.21.15.43:0/1053178324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ef410e050 0x7f3ef419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3ef93c5640 1 --2- 172.21.15.43:0/1053178324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef410b660 0x7f3ef419acb0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3ef93c5640 1 --2- 172.21.15.43:0/1053178324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ef4104c00 0x7f3ef419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3ef93c5640 1 -- 172.21.15.43:0/1053178324 >> 172.21.15.43:0/1053178324 conn(0x7f3ef40fe260 msgr2=0x7f3ef410e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3ef93c5640 1 -- 172.21.15.43:0/1053178324 shutdown_connections 2024-04-25T10:44:00.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.810+0000 7f3ef93c5640 1 -- 172.21.15.43:0/1053178324 wait complete. 2024-04-25T10:44:00.823 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:44:00.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.947+0000 7f142e56b640 1 -- 172.21.15.43:0/1906837954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1428109fd0 msgr2=0x7f1428103c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.947+0000 7f142e56b640 1 --2- 172.21.15.43:0/1906837954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1428109fd0 0x7f1428103c60 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f141c009900 tx=0x7f141c02a410 comp rx=0 tx=0).stop 2024-04-25T10:44:00.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.947+0000 7f142e56b640 1 -- 172.21.15.43:0/1906837954 shutdown_connections 2024-04-25T10:44:00.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.947+0000 7f142e56b640 1 --2- 172.21.15.43:0/1906837954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14281042d0 0x7f1428110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.947+0000 7f142e56b640 1 --2- 172.21.15.43:0/1906837954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1428109fd0 0x7f1428103c60 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.947+0000 7f142e56b640 1 --2- 172.21.15.43:0/1906837954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1428109b30 0x7f1428103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.947+0000 7f142e56b640 1 -- 172.21.15.43:0/1906837954 >> 172.21.15.43:0/1906837954 conn(0x7f1428079280 msgr2=0x7f14280796c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.947+0000 7f142e56b640 1 -- 172.21.15.43:0/1906837954 shutdown_connections 2024-04-25T10:44:00.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.947+0000 7f142e56b640 1 -- 172.21.15.43:0/1906837954 wait complete. 2024-04-25T10:44:00.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.948+0000 7f142e56b640 1 Processor -- start 2024-04-25T10:44:00.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.948+0000 7f142e56b640 1 -- start start 2024-04-25T10:44:00.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.948+0000 7f142e56b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14281042d0 0x7f142819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.948+0000 7f142e56b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1428109b30 0x7f142819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.948+0000 7f142e56b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1428109fd0 0x7f142819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.948+0000 7f142e56b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1428113d20 con 0x7f1428109fd0 2024-04-25T10:44:00.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.948+0000 7f142e56b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1428113ba0 con 0x7f14281042d0 2024-04-25T10:44:00.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.948+0000 7f142e56b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1428113ea0 con 0x7f1428109b30 2024-04-25T10:44:00.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.948+0000 7f142cb08640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1428109fd0 0x7f142819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.948+0000 7f142cb08640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1428109fd0 0x7f142819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33034/0 (socket says 172.21.15.43:33034) 2024-04-25T10:44:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.948+0000 7f142cb08640 1 -- 172.21.15.43:0/959663749 learned_addr learned my addr 172.21.15.43:0/959663749 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.948+0000 7f1427fff640 1 --2- 172.21.15.43:0/959663749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14281042d0 0x7f142819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.949+0000 7f14277fe640 1 --2- 172.21.15.43:0/959663749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1428109b30 0x7f142819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.949+0000 7f142cb08640 1 -- 172.21.15.43:0/959663749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1428109b30 msgr2=0x7f142819ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.949+0000 7f142cb08640 1 --2- 172.21.15.43:0/959663749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1428109b30 0x7f142819ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.949+0000 7f142cb08640 1 -- 172.21.15.43:0/959663749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14281042d0 msgr2=0x7f142819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.949+0000 7f142cb08640 1 --2- 172.21.15.43:0/959663749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14281042d0 0x7f142819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.949+0000 7f142cb08640 1 -- 172.21.15.43:0/959663749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142819f600 con 0x7f1428109fd0 2024-04-25T10:44:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.949+0000 7f1427fff640 1 --2- 172.21.15.43:0/959663749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14281042d0 0x7f142819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:44:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.949+0000 7f142cb08640 1 --2- 172.21.15.43:0/959663749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1428109fd0 0x7f142819efc0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f142000cad0 tx=0x7f142000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.949+0000 7f14257fa640 1 -- 172.21.15.43:0/959663749 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142000ae20 con 0x7f1428109fd0 2024-04-25T10:44:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.949+0000 7f14257fa640 1 -- 172.21.15.43:0/959663749 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1420005020 con 0x7f1428109fd0 2024-04-25T10:44:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.949+0000 7f142e56b640 1 -- 172.21.15.43:0/959663749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f142819f8f0 con 0x7f1428109fd0 2024-04-25T10:44:00.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.949+0000 7f14257fa640 1 -- 172.21.15.43:0/959663749 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142000f6b0 con 0x7f1428109fd0 2024-04-25T10:44:00.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.949+0000 7f142e56b640 1 -- 172.21.15.43:0/959663749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14281a7270 con 0x7f1428109fd0 2024-04-25T10:44:00.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.952+0000 7f1435418740 1 Processor -- start 2024-04-25T10:44:00.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.952+0000 7f14257fa640 1 -- 172.21.15.43:0/959663749 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f14200051d0 con 0x7f1428109fd0 2024-04-25T10:44:00.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.952+0000 7f1435418740 1 -- start start 2024-04-25T10:44:00.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.952+0000 7f1435418740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d77390 0x5584d9118810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.952+0000 7f1435418740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584d8d4e4d0 0x5584d9118d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.952+0000 7f1435418740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d566b0 0x5584d91256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.952+0000 7f1435418740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5584d907dd80 con 0x5584d8d566b0 2024-04-25T10:44:00.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.952+0000 7f1435418740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5584d9037a20 con 0x5584d8d4e4d0 2024-04-25T10:44:00.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.952+0000 7f1435418740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5584d9079d40 con 0x5584d8d77390 2024-04-25T10:44:00.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.952+0000 7f14277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d566b0 0x5584d91256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.952+0000 7f14277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d566b0 0x5584d91256a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33048/0 (socket says 172.21.15.43:33048) 2024-04-25T10:44:00.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.952+0000 7f14277fe640 1 -- 172.21.15.43:0/2668379167 learned_addr learned my addr 172.21.15.43:0/2668379167 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.952+0000 7f1427fff640 1 --2- 172.21.15.43:0/2668379167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584d8d4e4d0 0x5584d9118d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.952+0000 7f14277fe640 1 --2- 172.21.15.43:0/2668379167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d77390 0x5584d9118810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.952+0000 7f14257fa640 1 --2- 172.21.15.43:0/959663749 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f14100afc60 0x7f14100b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.953+0000 7f14277fe640 1 -- 172.21.15.43:0/2668379167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d77390 msgr2=0x5584d9118810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.953+0000 7f14277fe640 1 --2- 172.21.15.43:0/2668379167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d77390 0x5584d9118810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.953+0000 7f14277fe640 1 -- 172.21.15.43:0/2668379167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584d8d4e4d0 msgr2=0x5584d9118d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.953+0000 7f14277fe640 1 --2- 172.21.15.43:0/2668379167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584d8d4e4d0 0x5584d9118d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.953+0000 7f14277fe640 1 -- 172.21.15.43:0/2668379167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584d9042c30 con 0x5584d8d566b0 2024-04-25T10:44:00.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.953+0000 7f142cb08640 1 --2- 172.21.15.43:0/959663749 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f14100afc60 0x7f14100b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.953+0000 7f14277fe640 1 --2- 172.21.15.43:0/2668379167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d566b0 0x5584d91256a0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f141c032fd0 tx=0x7f141c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd7f4fb78cc41af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.953+0000 7f14257fa640 1 -- 172.21.15.43:0/959663749 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f14200defa0 con 0x7f1428109fd0 2024-04-25T10:44:00.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.953+0000 7f140e7fc640 1 -- 172.21.15.43:0/2668379167 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c035040 con 0x5584d8d566b0 2024-04-25T10:44:00.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.953+0000 7f140e7fc640 1 -- 172.21.15.43:0/2668379167 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f141c045070 con 0x5584d8d566b0 2024-04-25T10:44:00.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.953+0000 7f140e7fc640 1 -- 172.21.15.43:0/2668379167 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c004750 con 0x5584d8d566b0 2024-04-25T10:44:00.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.954+0000 7f1435418740 1 -- 172.21.15.43:0/2668379167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d566b0 msgr2=0x5584d91256a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.954+0000 7f142cb08640 1 --2- 172.21.15.43:0/959663749 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f14100afc60 0x7f14100b2170 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f14200dc7a0 tx=0x7f1420074a50 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.954+0000 7f1435418740 1 --2- 172.21.15.43:0/2668379167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d566b0 0x5584d91256a0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f141c032fd0 tx=0x7f141c000bc0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.954+0000 7f1435418740 1 -- 172.21.15.43:0/2668379167 shutdown_connections 2024-04-25T10:44:00.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.954+0000 7f1435418740 1 --2- 172.21.15.43:0/2668379167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584d8d4e4d0 0x5584d9118d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.954+0000 7f1435418740 1 --2- 172.21.15.43:0/2668379167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d566b0 0x5584d91256a0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.954+0000 7f1435418740 1 --2- 172.21.15.43:0/2668379167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d77390 0x5584d9118810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.954+0000 7f1435418740 1 -- 172.21.15.43:0/2668379167 >> 172.21.15.43:0/2668379167 conn(0x5584d8d6b4e0 msgr2=0x5584d9114fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.954+0000 7f1435418740 1 -- 172.21.15.43:0/2668379167 shutdown_connections 2024-04-25T10:44:00.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.954+0000 7f1435418740 1 -- 172.21.15.43:0/2668379167 wait complete. 2024-04-25T10:44:00.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.954+0000 7f1435418740 1 -- start start 2024-04-25T10:44:00.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.955+0000 7f1435418740 1 Processor -- start 2024-04-25T10:44:00.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.955+0000 7f1435418740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d4e4d0 0x5584d9123260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.955+0000 7f1435418740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584d8d566b0 0x5584d91237a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.955+0000 7f1435418740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d77390 0x5584d912f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.955+0000 7f1435418740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5584d9125d30 con 0x5584d8d77390 2024-04-25T10:44:00.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.955+0000 7f1435418740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5584d9004dc0 con 0x5584d8d566b0 2024-04-25T10:44:00.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.955+0000 7f1435418740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5584d9125eb0 con 0x5584d8d4e4d0 2024-04-25T10:44:00.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.955+0000 7f1435418740 10 client.0 _ll_register_callbacks cb 0x5584d9020600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:00.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.955+0000 7f14277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d77390 0x5584d912f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.955+0000 7f14277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d77390 0x5584d912f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33058/0 (socket says 172.21.15.43:33058) 2024-04-25T10:44:00.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.955+0000 7f14277fe640 1 -- 172.21.15.43:0/2532456126 learned_addr learned my addr 172.21.15.43:0/2532456126 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.955+0000 7f1427fff640 1 --2- 172.21.15.43:0/2532456126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584d8d566b0 0x5584d91237a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.955+0000 7f14277fe640 1 --2- 172.21.15.43:0/2532456126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d4e4d0 0x5584d9123260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.955+0000 7f14277fe640 1 -- 172.21.15.43:0/2532456126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d4e4d0 msgr2=0x5584d9123260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.955+0000 7f14277fe640 1 --2- 172.21.15.43:0/2532456126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d4e4d0 0x5584d9123260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.955+0000 7f14277fe640 1 -- 172.21.15.43:0/2532456126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584d8d566b0 msgr2=0x5584d91237a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.955+0000 7f14277fe640 1 --2- 172.21.15.43:0/2532456126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584d8d566b0 0x5584d91237a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.955+0000 7f14277fe640 1 -- 172.21.15.43:0/2532456126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584d8d8f3f0 con 0x5584d8d77390 2024-04-25T10:44:00.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.956+0000 7f14277fe640 1 --2- 172.21.15.43:0/2532456126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d77390 0x5584d912f1b0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f141c02a450 tx=0x7f141c0053d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e79c147bd3bc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.956+0000 7f14077fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:00.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.956+0000 7f14077fe640 1 -- 172.21.15.43:0/2532456126 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c04f070 con 0x5584d8d77390 2024-04-25T10:44:00.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.956+0000 7f14077fe640 1 -- 172.21.15.43:0/2532456126 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f141c0046c0 con 0x5584d8d77390 2024-04-25T10:44:00.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.956+0000 7f1435418740 1 -- 172.21.15.43:0/2532456126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f141c045070 con 0x5584d8d77390 2024-04-25T10:44:00.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.956+0000 7f14077fe640 1 -- 172.21.15.43:0/2532456126 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f141c035040 con 0x5584d8d77390 2024-04-25T10:44:00.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.956+0000 7f1435418740 10 client.18912 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:00.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.956+0000 7f1435418740 1 -- 172.21.15.43:0/2532456126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5584d8e597d0 con 0x5584d8d77390 2024-04-25T10:44:00.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.956+0000 7f14077fe640 1 -- 172.21.15.43:0/2532456126 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f141c045070 con 0x5584d8d77390 2024-04-25T10:44:00.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.956+0000 7f14077fe640 1 -- 172.21.15.43:0/2532456126 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f141c04b860 con 0x5584d8d77390 2024-04-25T10:44:00.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.956+0000 7f1435418740 10 client.18912 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:00.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.956+0000 7f1435418740 10 client.18912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:44:00.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.956+0000 7f1435418740 20 client.18912 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:00.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.956+0000 7f1435418740 1 --2- 172.21.15.43:0/2532456126 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5584d8d35f60 0x5584d9135490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.956+0000 7f1435418740 4 client.18912 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:44:00.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.956+0000 7f1435418740 1 -- 172.21.15.43:0/2532456126 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584d908b990 con 0x5584d8d35f60 2024-04-25T10:44:00.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.957+0000 7f1427fff640 1 --2- 172.21.15.43:0/2532456126 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5584d8d35f60 0x5584d9135490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.957+0000 7f14077fe640 10 client.18912 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.957+0000 7f1427fff640 1 --2- 172.21.15.43:0/2532456126 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5584d8d35f60 0x5584d9135490 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d39c5540aad40fb1 server_cookie=99ae189e82eea27f in_seq=0 out_seq=0 2024-04-25T10:44:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.958+0000 7f14077fe640 1 -- 172.21.15.43:0/2532456126 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5584d908b990 con 0x5584d8d35f60 2024-04-25T10:44:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.958+0000 7f14077fe640 10 client.18912 handle_command_reply: tid=0 2024-04-25T10:44:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.958+0000 7f1435418740 1 client.18912 shutdown 2024-04-25T10:44:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.958+0000 7f1435418740 1 -- 172.21.15.43:0/2532456126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d77390 msgr2=0x5584d912f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.958+0000 7f1435418740 1 --2- 172.21.15.43:0/2532456126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d77390 0x5584d912f1b0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f141c02a450 tx=0x7f141c0053d0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.958+0000 7f1435418740 1 -- 172.21.15.43:0/2532456126 shutdown_connections 2024-04-25T10:44:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.958+0000 7f1435418740 1 --2- 172.21.15.43:0/2532456126 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5584d8d35f60 0x5584d9135490 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.958+0000 7f1435418740 1 --2- 172.21.15.43:0/2532456126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584d8d566b0 0x5584d91237a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.958+0000 7f1435418740 1 --2- 172.21.15.43:0/2532456126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d77390 0x5584d912f1b0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.958+0000 7f1435418740 1 --2- 172.21.15.43:0/2532456126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d4e4d0 0x5584d9123260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.958+0000 7f14077fe640 0 client.18912 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.958+0000 7f1435418740 1 -- 172.21.15.43:0/2532456126 >> 172.21.15.43:0/2532456126 conn(0x5584d8d6b4e0 msgr2=0x5584d911a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.958+0000 7f1435418740 1 -- 172.21.15.43:0/2532456126 shutdown_connections 2024-04-25T10:44:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.958+0000 7f1435418740 1 -- 172.21.15.43:0/2532456126 wait complete. 2024-04-25T10:44:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.958+0000 7f1435418740 20 client.18912 trim_cache size 0 max 16384 2024-04-25T10:44:00.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.972+0000 7f1435418740 1 Processor -- start 2024-04-25T10:44:00.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.972+0000 7f1435418740 1 -- start start 2024-04-25T10:44:00.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.972+0000 7f1435418740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d35f60 0x5584d912f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.972+0000 7f1435418740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584d8d4e4d0 0x5584d9114f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.972+0000 7f1435418740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d566b0 0x5584d9122800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.972+0000 7f1435418740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14281b22b0 con 0x5584d8d566b0 2024-04-25T10:44:00.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.972+0000 7f1435418740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f14281afb80 con 0x5584d8d4e4d0 2024-04-25T10:44:00.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.972+0000 7f1435418740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f14281ad9f0 con 0x5584d8d35f60 2024-04-25T10:44:00.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.972+0000 7f14277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d566b0 0x5584d9122800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.972+0000 7f14277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d566b0 0x5584d9122800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33072/0 (socket says 172.21.15.43:33072) 2024-04-25T10:44:00.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.972+0000 7f14277fe640 1 -- 172.21.15.43:0/109542830 learned_addr learned my addr 172.21.15.43:0/109542830 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.972+0000 7f1427fff640 1 --2- 172.21.15.43:0/109542830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584d8d4e4d0 0x5584d9114f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.973+0000 7f14277fe640 1 --2- 172.21.15.43:0/109542830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d35f60 0x5584d912f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.973+0000 7f14277fe640 1 -- 172.21.15.43:0/109542830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d35f60 msgr2=0x5584d912f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.973+0000 7f14277fe640 1 --2- 172.21.15.43:0/109542830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d35f60 0x5584d912f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.973+0000 7f14277fe640 1 -- 172.21.15.43:0/109542830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584d8d4e4d0 msgr2=0x5584d9114f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.973+0000 7f14277fe640 1 --2- 172.21.15.43:0/109542830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584d8d4e4d0 0x5584d9114f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.973+0000 7f14277fe640 1 -- 172.21.15.43:0/109542830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584d911a010 con 0x5584d8d566b0 2024-04-25T10:44:00.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.973+0000 7f14277fe640 1 --2- 172.21.15.43:0/109542830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d566b0 0x5584d9122800 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f141c009850 tx=0x7f141c040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a957d40ae68fce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.973+0000 7f140dffb640 1 -- 172.21.15.43:0/109542830 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c04f070 con 0x5584d8d566b0 2024-04-25T10:44:00.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.973+0000 7f140dffb640 1 -- 172.21.15.43:0/109542830 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f141c045040 con 0x5584d8d566b0 2024-04-25T10:44:00.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.973+0000 7f140dffb640 1 -- 172.21.15.43:0/109542830 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c05c040 con 0x5584d8d566b0 2024-04-25T10:44:00.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.974+0000 7f1435418740 1 -- 172.21.15.43:0/109542830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d566b0 msgr2=0x5584d9122800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.974+0000 7f1435418740 1 --2- 172.21.15.43:0/109542830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d566b0 0x5584d9122800 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f141c009850 tx=0x7f141c040040 comp rx=0 tx=0).stop 2024-04-25T10:44:00.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.974+0000 7f1435418740 1 -- 172.21.15.43:0/109542830 shutdown_connections 2024-04-25T10:44:00.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.974+0000 7f1435418740 1 --2- 172.21.15.43:0/109542830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584d8d4e4d0 0x5584d9114f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.974+0000 7f1435418740 1 --2- 172.21.15.43:0/109542830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d566b0 0x5584d9122800 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.974+0000 7f1435418740 1 --2- 172.21.15.43:0/109542830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d35f60 0x5584d912f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.974+0000 7f1435418740 1 -- 172.21.15.43:0/109542830 >> 172.21.15.43:0/109542830 conn(0x5584d8d6b4e0 msgr2=0x5584d9114860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.974+0000 7f1435418740 1 -- 172.21.15.43:0/109542830 shutdown_connections 2024-04-25T10:44:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.974+0000 7f1435418740 1 -- 172.21.15.43:0/109542830 wait complete. 2024-04-25T10:44:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.974+0000 7f1435418740 1 -- start start 2024-04-25T10:44:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f1435418740 1 Processor -- start 2024-04-25T10:44:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f1435418740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584d8d35f60 0x5584d91357f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f1435418740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d4e4d0 0x5584d9135d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f1435418740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d566b0 0x5584d919b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f1435418740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14281ae040 con 0x5584d8d4e4d0 2024-04-25T10:44:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f1435418740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f14281af4b0 con 0x5584d8d35f60 2024-04-25T10:44:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f1435418740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f14281ae710 con 0x5584d8d566b0 2024-04-25T10:44:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f1435418740 10 client.0 _ll_register_callbacks cb 0x7f1418005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f1427fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d4e4d0 0x5584d9135d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f1427fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d4e4d0 0x5584d9135d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33082/0 (socket says 172.21.15.43:33082) 2024-04-25T10:44:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f1427fff640 1 -- 172.21.15.43:0/1289994351 learned_addr learned my addr 172.21.15.43:0/1289994351 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f14277fe640 1 --2- 172.21.15.43:0/1289994351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584d8d35f60 0x5584d91357f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f14277fe640 1 --2- 172.21.15.43:0/1289994351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d566b0 0x5584d919b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f1427fff640 1 -- 172.21.15.43:0/1289994351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d566b0 msgr2=0x5584d919b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f1427fff640 1 --2- 172.21.15.43:0/1289994351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d566b0 0x5584d919b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f1427fff640 1 -- 172.21.15.43:0/1289994351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584d8d35f60 msgr2=0x5584d91357f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f1427fff640 1 --2- 172.21.15.43:0/1289994351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584d8d35f60 0x5584d91357f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f1427fff640 1 -- 172.21.15.43:0/1289994351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584d919baf0 con 0x5584d8d4e4d0 2024-04-25T10:44:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f14277fe640 1 --2- 172.21.15.43:0/1289994351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d566b0 0x5584d919b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f1427fff640 1 --2- 172.21.15.43:0/1289994351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d4e4d0 0x5584d9135d30 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f1418005d00 tx=0x7f1418012e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af49a5e2db2fdd9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.975+0000 7f14077fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.976+0000 7f14077fe640 1 -- 172.21.15.43:0/1289994351 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141801ede0 con 0x5584d8d4e4d0 2024-04-25T10:44:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.976+0000 7f14077fe640 1 -- 172.21.15.43:0/1289994351 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f1418004020 con 0x5584d8d4e4d0 2024-04-25T10:44:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.976+0000 7f1435418740 1 -- 172.21.15.43:0/1289994351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f141c045040 con 0x5584d8d4e4d0 2024-04-25T10:44:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.976+0000 7f14077fe640 1 -- 172.21.15.43:0/1289994351 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1418004220 con 0x5584d8d4e4d0 2024-04-25T10:44:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.976+0000 7f1435418740 10 client.18918 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.976+0000 7f1435418740 1 -- 172.21.15.43:0/1289994351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5584d919cd20 con 0x5584d8d4e4d0 2024-04-25T10:44:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.976+0000 7f14077fe640 1 -- 172.21.15.43:0/1289994351 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f141c045040 con 0x5584d8d4e4d0 2024-04-25T10:44:00.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.976+0000 7f14077fe640 1 -- 172.21.15.43:0/1289994351 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f14180335a0 con 0x5584d8d4e4d0 2024-04-25T10:44:00.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.976+0000 7f1435418740 10 client.18918 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:00.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.976+0000 7f1435418740 10 client.18918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:44:00.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.976+0000 7f1435418740 20 client.18918 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:00.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.976+0000 7f1435418740 1 --2- 172.21.15.43:0/1289994351 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5584d8d77390 0x5584d919f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:00.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.976+0000 7f1435418740 4 client.18918 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:44:00.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.976+0000 7f1435418740 1 -- 172.21.15.43:0/1289994351 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584d9123b70 con 0x5584d8d77390 2024-04-25T10:44:00.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.977+0000 7f14277fe640 1 --2- 172.21.15.43:0/1289994351 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5584d8d77390 0x5584d919f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:00.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.977+0000 7f14277fe640 1 --2- 172.21.15.43:0/1289994351 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5584d8d77390 0x5584d919f4c0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d33e798b0844ac5a server_cookie=cd0cd23355488035 in_seq=0 out_seq=0 2024-04-25T10:44:00.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.977+0000 7f14077fe640 10 client.18918 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:00.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.977+0000 7f14077fe640 1 -- 172.21.15.43:0/1289994351 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584d9123b70 con 0x5584d8d77390 2024-04-25T10:44:00.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.977+0000 7f14077fe640 10 client.18918 handle_command_reply: tid=0 2024-04-25T10:44:00.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.977+0000 7f1435418740 1 client.18918 shutdown 2024-04-25T10:44:00.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.977+0000 7f1435418740 1 -- 172.21.15.43:0/1289994351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d4e4d0 msgr2=0x5584d9135d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.977+0000 7f1435418740 1 --2- 172.21.15.43:0/1289994351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d4e4d0 0x5584d9135d30 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f1418005d00 tx=0x7f1418012e50 comp rx=0 tx=0).stop 2024-04-25T10:44:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.977+0000 7f1435418740 1 -- 172.21.15.43:0/1289994351 shutdown_connections 2024-04-25T10:44:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.977+0000 7f1435418740 1 --2- 172.21.15.43:0/1289994351 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x5584d8d77390 0x5584d919f4c0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.977+0000 7f1435418740 1 --2- 172.21.15.43:0/1289994351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584d8d566b0 0x5584d919b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.977+0000 7f1435418740 1 --2- 172.21.15.43:0/1289994351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584d8d4e4d0 0x5584d9135d30 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.977+0000 7f14077fe640 0 client.18918 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.977+0000 7f1435418740 1 --2- 172.21.15.43:0/1289994351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584d8d35f60 0x5584d91357f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.977+0000 7f1435418740 1 -- 172.21.15.43:0/1289994351 >> 172.21.15.43:0/1289994351 conn(0x5584d8d6b4e0 msgr2=0x5584d9113610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.978+0000 7f1435418740 1 -- 172.21.15.43:0/1289994351 shutdown_connections 2024-04-25T10:44:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.978+0000 7f1435418740 1 -- 172.21.15.43:0/1289994351 wait complete. 2024-04-25T10:44:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.978+0000 7f1435418740 20 client.18918 trim_cache size 0 max 16384 2024-04-25T10:44:00.985 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:44:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.978+0000 7f142e56b640 1 -- 172.21.15.43:0/959663749 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f14100afc60 msgr2=0x7f14100b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.978+0000 7f142e56b640 1 --2- 172.21.15.43:0/959663749 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f14100afc60 0x7f14100b2170 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f14200dc7a0 tx=0x7f1420074a50 comp rx=0 tx=0).stop 2024-04-25T10:44:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.978+0000 7f142e56b640 1 -- 172.21.15.43:0/959663749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1428109fd0 msgr2=0x7f142819efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.978+0000 7f142e56b640 1 --2- 172.21.15.43:0/959663749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1428109fd0 0x7f142819efc0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f142000cad0 tx=0x7f142000cf90 comp rx=0 tx=0).stop 2024-04-25T10:44:00.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.978+0000 7f142e56b640 1 -- 172.21.15.43:0/959663749 shutdown_connections 2024-04-25T10:44:00.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.978+0000 7f142e56b640 1 --2- 172.21.15.43:0/959663749 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f14100afc60 0x7f14100b2170 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.978+0000 7f142e56b640 1 --2- 172.21.15.43:0/959663749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1428109fd0 0x7f142819efc0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.978+0000 7f142e56b640 1 --2- 172.21.15.43:0/959663749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1428109b30 0x7f142819ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.978+0000 7f142e56b640 1 --2- 172.21.15.43:0/959663749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14281042d0 0x7f142819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:00.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.978+0000 7f142e56b640 1 -- 172.21.15.43:0/959663749 >> 172.21.15.43:0/959663749 conn(0x7f1428079280 msgr2=0x7f14281022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:00.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.978+0000 7f142e56b640 1 -- 172.21.15.43:0/959663749 shutdown_connections 2024-04-25T10:44:00.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:00.978+0000 7f142e56b640 1 -- 172.21.15.43:0/959663749 wait complete. 2024-04-25T10:44:01.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:01 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/114010426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:44:01.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:01 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/114010426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:44:01.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:01 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/114010426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:44:01.994 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T10:44:01.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:44:01.994 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T10:44:02.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.124+0000 7f8175d8f640 1 -- 172.21.15.43:0/3488218659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81701069b0 msgr2=0x7f8170106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:02.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.124+0000 7f8175d8f640 1 --2- 172.21.15.43:0/3488218659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81701069b0 0x7f8170106de0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f8164009900 tx=0x7f816402a410 comp rx=0 tx=0).stop 2024-04-25T10:44:02.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.124+0000 7f8175d8f640 1 -- 172.21.15.43:0/3488218659 shutdown_connections 2024-04-25T10:44:02.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.124+0000 7f8175d8f640 1 --2- 172.21.15.43:0/3488218659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8170107470 0x7f817010ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.124+0000 7f8175d8f640 1 --2- 172.21.15.43:0/3488218659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81701069b0 0x7f8170106de0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.124+0000 7f8175d8f640 1 --2- 172.21.15.43:0/3488218659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8170102420 0x7f8170102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.124+0000 7f8175d8f640 1 -- 172.21.15.43:0/3488218659 >> 172.21.15.43:0/3488218659 conn(0x7f8170077920 msgr2=0x7f8170077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:02.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.124+0000 7f8175d8f640 1 -- 172.21.15.43:0/3488218659 shutdown_connections 2024-04-25T10:44:02.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.124+0000 7f8175d8f640 1 -- 172.21.15.43:0/3488218659 wait complete. 2024-04-25T10:44:02.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.125+0000 7f8175d8f640 1 Processor -- start 2024-04-25T10:44:02.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.125+0000 7f8175d8f640 1 -- start start 2024-04-25T10:44:02.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.125+0000 7f8175d8f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8170102420 0x7f817019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.125+0000 7f8175d8f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81701069b0 0x7f817019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.125+0000 7f8175d8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8170107470 0x7f817019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.125+0000 7f8175d8f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8170111b00 con 0x7f8170107470 2024-04-25T10:44:02.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.125+0000 7f8175d8f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8170111980 con 0x7f8170102420 2024-04-25T10:44:02.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.125+0000 7f8175d8f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8170111c80 con 0x7f81701069b0 2024-04-25T10:44:02.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.125+0000 7f816ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8170107470 0x7f817019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.125+0000 7f816ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8170107470 0x7f817019f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33094/0 (socket says 172.21.15.43:33094) 2024-04-25T10:44:02.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.125+0000 7f816ffff640 1 -- 172.21.15.43:0/2977146582 learned_addr learned my addr 172.21.15.43:0/2977146582 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:02.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.125+0000 7f816effd640 1 --2- 172.21.15.43:0/2977146582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81701069b0 0x7f817019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.126+0000 7f816f7fe640 1 --2- 172.21.15.43:0/2977146582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8170102420 0x7f817019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.126+0000 7f816ffff640 1 -- 172.21.15.43:0/2977146582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81701069b0 msgr2=0x7f817019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.126+0000 7f816ffff640 1 --2- 172.21.15.43:0/2977146582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81701069b0 0x7f817019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.126+0000 7f816ffff640 1 -- 172.21.15.43:0/2977146582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8170102420 msgr2=0x7f817019a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.126+0000 7f816ffff640 1 --2- 172.21.15.43:0/2977146582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8170102420 0x7f817019a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.126+0000 7f816ffff640 1 -- 172.21.15.43:0/2977146582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817019f660 con 0x7f8170107470 2024-04-25T10:44:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.126+0000 7f816ffff640 1 --2- 172.21.15.43:0/2977146582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8170107470 0x7f817019f020 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f816800ec60 tx=0x7f816800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.126+0000 7f816cff9640 1 -- 172.21.15.43:0/2977146582 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168004d20 con 0x7f8170107470 2024-04-25T10:44:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.126+0000 7f816cff9640 1 -- 172.21.15.43:0/2977146582 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f816800ba20 con 0x7f8170107470 2024-04-25T10:44:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.126+0000 7f8175d8f640 1 -- 172.21.15.43:0/2977146582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f817019f950 con 0x7f8170107470 2024-04-25T10:44:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.126+0000 7f816cff9640 1 -- 172.21.15.43:0/2977146582 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81680056c0 con 0x7f8170107470 2024-04-25T10:44:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.126+0000 7f8175d8f640 1 -- 172.21.15.43:0/2977146582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f817019fe20 con 0x7f8170107470 2024-04-25T10:44:02.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.129+0000 7f817cc43740 1 Processor -- start 2024-04-25T10:44:02.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.129+0000 7f816cff9640 1 -- 172.21.15.43:0/2977146582 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f8168005860 con 0x7f8170107470 2024-04-25T10:44:02.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.129+0000 7f817cc43740 1 -- start start 2024-04-25T10:44:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.130+0000 7f817cc43740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775facc410 0x55775fe6c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.130+0000 7f817cc43740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55775faa3550 0x55775fe6cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.130+0000 7f817cc43740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55775faab730 0x55775fe6d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.130+0000 7f817cc43740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55775fdd2b00 con 0x55775facc410 2024-04-25T10:44:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.130+0000 7f817cc43740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55775fd6e920 con 0x55775faab730 2024-04-25T10:44:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.130+0000 7f817cc43740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55775fdceac0 con 0x55775faa3550 2024-04-25T10:44:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.130+0000 7f816effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775facc410 0x55775fe6c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.130+0000 7f816effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775facc410 0x55775fe6c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33104/0 (socket says 172.21.15.43:33104) 2024-04-25T10:44:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.130+0000 7f816effd640 1 -- 172.21.15.43:0/1736705781 learned_addr learned my addr 172.21.15.43:0/1736705781 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.130+0000 7f816f7fe640 1 --2- 172.21.15.43:0/1736705781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55775faa3550 0x55775fe6cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.130+0000 7f816effd640 1 --2- 172.21.15.43:0/1736705781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55775faab730 0x55775fe6d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.130+0000 7f816cff9640 1 --2- 172.21.15.43:0/2977146582 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f81580afc60 0x7f81580b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.130+0000 7f816effd640 1 -- 172.21.15.43:0/1736705781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55775faa3550 msgr2=0x55775fe6cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.130+0000 7f816effd640 1 --2- 172.21.15.43:0/1736705781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55775faa3550 0x55775fe6cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.130+0000 7f816effd640 1 -- 172.21.15.43:0/1736705781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55775faab730 msgr2=0x55775fe6d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.130+0000 7f816effd640 1 --2- 172.21.15.43:0/1736705781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55775faab730 0x55775fe6d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.130+0000 7f816effd640 1 -- 172.21.15.43:0/1736705781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55775fbac7e0 con 0x55775facc410 2024-04-25T10:44:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.130+0000 7f816ffff640 1 --2- 172.21.15.43:0/2977146582 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f81580afc60 0x7f81580b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.130+0000 7f816effd640 1 --2- 172.21.15.43:0/1736705781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775facc410 0x55775fe6c620 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f81640096c0 tx=0x7f8164005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a555cb4eb1f20a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.130+0000 7f816cff9640 1 -- 172.21.15.43:0/2977146582 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f81680df550 con 0x7f8170107470 2024-04-25T10:44:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.131+0000 7f8155ffb640 1 -- 172.21.15.43:0/1736705781 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164044020 con 0x55775facc410 2024-04-25T10:44:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.131+0000 7f8155ffb640 1 -- 172.21.15.43:0/1736705781 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81640316a0 con 0x55775facc410 2024-04-25T10:44:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.131+0000 7f8155ffb640 1 -- 172.21.15.43:0/1736705781 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164035040 con 0x55775facc410 2024-04-25T10:44:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.131+0000 7f817cc43740 1 -- 172.21.15.43:0/1736705781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775facc410 msgr2=0x55775fe6c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.131+0000 7f817cc43740 1 --2- 172.21.15.43:0/1736705781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775facc410 0x55775fe6c620 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f81640096c0 tx=0x7f8164005db0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.131+0000 7f816ffff640 1 --2- 172.21.15.43:0/2977146582 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f81580afc60 0x7f81580b2170 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f81680a3710 tx=0x7f8168090040 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.132+0000 7f817cc43740 1 -- 172.21.15.43:0/1736705781 shutdown_connections 2024-04-25T10:44:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.132+0000 7f817cc43740 1 --2- 172.21.15.43:0/1736705781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55775faab730 0x55775fe6d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.132+0000 7f817cc43740 1 --2- 172.21.15.43:0/1736705781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55775faa3550 0x55775fe6cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.132+0000 7f817cc43740 1 --2- 172.21.15.43:0/1736705781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775facc410 0x55775fe6c620 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.132+0000 7f817cc43740 1 -- 172.21.15.43:0/1736705781 >> 172.21.15.43:0/1736705781 conn(0x55775fac0560 msgr2=0x55775fe68dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:02.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.132+0000 7f817cc43740 1 -- 172.21.15.43:0/1736705781 shutdown_connections 2024-04-25T10:44:02.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.132+0000 7f817cc43740 1 -- 172.21.15.43:0/1736705781 wait complete. 2024-04-25T10:44:02.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.132+0000 7f817cc43740 1 -- start start 2024-04-25T10:44:02.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.132+0000 7f817cc43740 1 Processor -- start 2024-04-25T10:44:02.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f817cc43740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55775faa3550 0x55775fe76c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f817cc43740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faab730 0x55775fe77180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f817cc43740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55775facc410 0x55775fe82dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f817cc43740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55775fe79910 con 0x55775faab730 2024-04-25T10:44:02.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f817cc43740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55775fd6c5d0 con 0x55775facc410 2024-04-25T10:44:02.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f817cc43740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55775fe79a90 con 0x55775faa3550 2024-04-25T10:44:02.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f817cc43740 10 client.0 _ll_register_callbacks cb 0x55775fd5a080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:02.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f816f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faab730 0x55775fe77180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f816f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faab730 0x55775fe77180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33120/0 (socket says 172.21.15.43:33120) 2024-04-25T10:44:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f816f7fe640 1 -- 172.21.15.43:0/2026506993 learned_addr learned my addr 172.21.15.43:0/2026506993 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f816effd640 1 --2- 172.21.15.43:0/2026506993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55775faa3550 0x55775fe76c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f816effd640 1 --2- 172.21.15.43:0/2026506993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55775facc410 0x55775fe82dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f816f7fe640 1 -- 172.21.15.43:0/2026506993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55775faa3550 msgr2=0x55775fe76c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f816f7fe640 1 --2- 172.21.15.43:0/2026506993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55775faa3550 0x55775fe76c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f816f7fe640 1 -- 172.21.15.43:0/2026506993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55775facc410 msgr2=0x55775fe82dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f816f7fe640 1 --2- 172.21.15.43:0/2026506993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55775facc410 0x55775fe82dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f816f7fe640 1 -- 172.21.15.43:0/2026506993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55775fc0cf70 con 0x55775faab730 2024-04-25T10:44:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f816effd640 1 --2- 172.21.15.43:0/2026506993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55775facc410 0x55775fe82dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f816f7fe640 1 --2- 172.21.15.43:0/2026506993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faab730 0x55775fe77180 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f8160012700 tx=0x7f8160012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55eeeb161994f93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f814effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.133+0000 7f814effd640 1 -- 172.21.15.43:0/2026506993 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160004260 con 0x55775faab730 2024-04-25T10:44:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.134+0000 7f814effd640 1 -- 172.21.15.43:0/2026506993 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f8160004400 con 0x55775faab730 2024-04-25T10:44:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.134+0000 7f817cc43740 1 -- 172.21.15.43:0/2026506993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81640316a0 con 0x55775faab730 2024-04-25T10:44:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.134+0000 7f814effd640 1 -- 172.21.15.43:0/2026506993 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f81600119a0 con 0x55775faab730 2024-04-25T10:44:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.134+0000 7f817cc43740 10 client.18930 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.134+0000 7f817cc43740 1 -- 172.21.15.43:0/2026506993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55775faa4170 con 0x55775faab730 2024-04-25T10:44:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.134+0000 7f814effd640 1 -- 172.21.15.43:0/2026506993 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81640316a0 con 0x55775faab730 2024-04-25T10:44:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.134+0000 7f814effd640 1 -- 172.21.15.43:0/2026506993 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8160025480 con 0x55775faab730 2024-04-25T10:44:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.134+0000 7f817cc43740 10 client.18930 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.134+0000 7f817cc43740 10 client.18930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:44:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.134+0000 7f817cc43740 20 client.18930 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.134+0000 7f817cc43740 1 --2- 172.21.15.43:0/2026506993 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55775fa8b1f0 0x55775fe88fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.134+0000 7f817cc43740 4 client.18930 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:44:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.134+0000 7f817cc43740 1 -- 172.21.15.43:0/2026506993 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55775fde0710 con 0x55775fa8b1f0 2024-04-25T10:44:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.134+0000 7f816effd640 1 --2- 172.21.15.43:0/2026506993 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55775fa8b1f0 0x55775fe88fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.135+0000 7f814effd640 10 client.18930 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.135+0000 7f816effd640 1 --2- 172.21.15.43:0/2026506993 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55775fa8b1f0 0x55775fe88fa0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a66680ec8da808a server_cookie=7e6f82c59c28e92c in_seq=0 out_seq=0 2024-04-25T10:44:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.136+0000 7f814effd640 1 -- 172.21.15.43:0/2026506993 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55775fde0710 con 0x55775fa8b1f0 2024-04-25T10:44:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.136+0000 7f814effd640 10 client.18930 handle_command_reply: tid=0 2024-04-25T10:44:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.136+0000 7f817cc43740 1 client.18930 shutdown 2024-04-25T10:44:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.136+0000 7f817cc43740 1 -- 172.21.15.43:0/2026506993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faab730 msgr2=0x55775fe77180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.136+0000 7f817cc43740 1 --2- 172.21.15.43:0/2026506993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faab730 0x55775fe77180 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f8160012700 tx=0x7f8160012bc0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.136+0000 7f817cc43740 1 -- 172.21.15.43:0/2026506993 shutdown_connections 2024-04-25T10:44:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.136+0000 7f817cc43740 1 --2- 172.21.15.43:0/2026506993 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55775fa8b1f0 0x55775fe88fa0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.136+0000 7f817cc43740 1 --2- 172.21.15.43:0/2026506993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55775facc410 0x55775fe82dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.136+0000 7f817cc43740 1 --2- 172.21.15.43:0/2026506993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faab730 0x55775fe77180 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.136+0000 7f814effd640 0 client.18930 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.136+0000 7f817cc43740 1 --2- 172.21.15.43:0/2026506993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55775faa3550 0x55775fe76c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.136+0000 7f817cc43740 1 -- 172.21.15.43:0/2026506993 >> 172.21.15.43:0/2026506993 conn(0x55775fac0560 msgr2=0x55775fe6c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.136+0000 7f817cc43740 1 -- 172.21.15.43:0/2026506993 shutdown_connections 2024-04-25T10:44:02.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.136+0000 7f817cc43740 1 -- 172.21.15.43:0/2026506993 wait complete. 2024-04-25T10:44:02.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.136+0000 7f817cc43740 20 client.18930 trim_cache size 0 max 16384 2024-04-25T10:44:02.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.155+0000 7f817cc43740 1 Processor -- start 2024-04-25T10:44:02.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.155+0000 7f817cc43740 1 -- start start 2024-04-25T10:44:02.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.155+0000 7f817cc43740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55775faa3550 0x55775fe68e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.155+0000 7f817cc43740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faab730 0x55775fe75a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.155+0000 7f817cc43740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55775facc410 0x55775fe8e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.155+0000 7f817cc43740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f81701b2270 con 0x55775faab730 2024-04-25T10:44:02.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.155+0000 7f817cc43740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f81701afb40 con 0x55775facc410 2024-04-25T10:44:02.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.155+0000 7f817cc43740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f81701ad9b0 con 0x55775faa3550 2024-04-25T10:44:02.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.155+0000 7f816f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faab730 0x55775fe75a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.155+0000 7f816f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faab730 0x55775fe75a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33126/0 (socket says 172.21.15.43:33126) 2024-04-25T10:44:02.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.155+0000 7f816f7fe640 1 -- 172.21.15.43:0/2238049718 learned_addr learned my addr 172.21.15.43:0/2238049718 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:02.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.155+0000 7f816f7fe640 1 -- 172.21.15.43:0/2238049718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55775faa3550 msgr2=0x55775fe68e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:02.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.155+0000 7f816f7fe640 1 --2- 172.21.15.43:0/2238049718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55775faa3550 0x55775fe68e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.155+0000 7f816f7fe640 1 -- 172.21.15.43:0/2238049718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55775facc410 msgr2=0x55775fe8e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:02.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.155+0000 7f816f7fe640 1 --2- 172.21.15.43:0/2238049718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55775facc410 0x55775fe8e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.155+0000 7f816f7fe640 1 -- 172.21.15.43:0/2238049718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55775fe74f80 con 0x55775faab730 2024-04-25T10:44:02.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.155+0000 7f816f7fe640 1 --2- 172.21.15.43:0/2238049718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faab730 0x55775fe75a10 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f8160025850 tx=0x7f8160011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8bf9aef53e2e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:02.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.156+0000 7f81557fa640 1 -- 172.21.15.43:0/2238049718 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816002b070 con 0x55775faab730 2024-04-25T10:44:02.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.156+0000 7f81557fa640 1 -- 172.21.15.43:0/2238049718 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8160025500 con 0x55775faab730 2024-04-25T10:44:02.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.156+0000 7f81557fa640 1 -- 172.21.15.43:0/2238049718 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81600035f0 con 0x55775faab730 2024-04-25T10:44:02.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.156+0000 7f817cc43740 1 -- 172.21.15.43:0/2238049718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faab730 msgr2=0x55775fe75a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:02.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.156+0000 7f817cc43740 1 --2- 172.21.15.43:0/2238049718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faab730 0x55775fe75a10 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f8160025850 tx=0x7f8160011810 comp rx=0 tx=0).stop 2024-04-25T10:44:02.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.156+0000 7f817cc43740 1 -- 172.21.15.43:0/2238049718 shutdown_connections 2024-04-25T10:44:02.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.156+0000 7f817cc43740 1 --2- 172.21.15.43:0/2238049718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55775facc410 0x55775fe8e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.156+0000 7f817cc43740 1 --2- 172.21.15.43:0/2238049718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faab730 0x55775fe75a10 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.156+0000 7f817cc43740 1 --2- 172.21.15.43:0/2238049718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55775faa3550 0x55775fe68e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.156+0000 7f817cc43740 1 -- 172.21.15.43:0/2238049718 >> 172.21.15.43:0/2238049718 conn(0x55775fac0560 msgr2=0x55775fe75300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:02.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.157+0000 7f817cc43740 1 -- 172.21.15.43:0/2238049718 shutdown_connections 2024-04-25T10:44:02.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.157+0000 7f817cc43740 1 -- 172.21.15.43:0/2238049718 wait complete. 2024-04-25T10:44:02.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.157+0000 7f817cc43740 1 -- start start 2024-04-25T10:44:02.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.157+0000 7f817cc43740 1 Processor -- start 2024-04-25T10:44:02.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.157+0000 7f817cc43740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faa3550 0x55775fe7f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.157+0000 7f817cc43740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55775faab730 0x55775fe7a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.157+0000 7f817cc43740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55775facc410 0x55775fe7a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.157+0000 7f817cc43740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f81701ae000 con 0x55775faa3550 2024-04-25T10:44:02.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.157+0000 7f817cc43740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f81701af470 con 0x55775facc410 2024-04-25T10:44:02.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.157+0000 7f817cc43740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f81701ae6d0 con 0x55775faab730 2024-04-25T10:44:02.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.158+0000 7f817cc43740 10 client.0 _ll_register_callbacks cb 0x7f8160009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:02.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.158+0000 7f816effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faa3550 0x55775fe7f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.158+0000 7f816effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faa3550 0x55775fe7f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33140/0 (socket says 172.21.15.43:33140) 2024-04-25T10:44:02.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.158+0000 7f816effd640 1 -- 172.21.15.43:0/4275805928 learned_addr learned my addr 172.21.15.43:0/4275805928 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:02.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.158+0000 7f816effd640 1 --2- 172.21.15.43:0/4275805928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55775facc410 0x55775fe7a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.158+0000 7f816f7fe640 1 --2- 172.21.15.43:0/4275805928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55775faab730 0x55775fe7a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.158+0000 7f816effd640 1 -- 172.21.15.43:0/4275805928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55775faab730 msgr2=0x55775fe7a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.158+0000 7f816effd640 1 --2- 172.21.15.43:0/4275805928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55775faab730 0x55775fe7a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.158+0000 7f816effd640 1 -- 172.21.15.43:0/4275805928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55775facc410 msgr2=0x55775fe7a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.158+0000 7f816effd640 1 --2- 172.21.15.43:0/4275805928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55775facc410 0x55775fe7a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.158+0000 7f816effd640 1 -- 172.21.15.43:0/4275805928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55775fe7ae00 con 0x55775faa3550 2024-04-25T10:44:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.158+0000 7f816effd640 1 --2- 172.21.15.43:0/4275805928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faa3550 0x55775fe7f030 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f8164009900 tx=0x7f8164031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c873dbb73e379f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.158+0000 7f814effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.158+0000 7f814effd640 1 -- 172.21.15.43:0/4275805928 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164044040 con 0x55775faa3550 2024-04-25T10:44:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.158+0000 7f814effd640 1 -- 172.21.15.43:0/4275805928 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f81640055f0 con 0x55775faa3550 2024-04-25T10:44:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.158+0000 7f817cc43740 1 -- 172.21.15.43:0/4275805928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8160025500 con 0x55775faa3550 2024-04-25T10:44:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.158+0000 7f814effd640 1 -- 172.21.15.43:0/4275805928 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8164035040 con 0x55775faa3550 2024-04-25T10:44:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.158+0000 7f817cc43740 10 client.18936 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.158+0000 7f817cc43740 1 -- 172.21.15.43:0/4275805928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55775fef0fc0 con 0x55775faa3550 2024-04-25T10:44:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.159+0000 7f814effd640 1 -- 172.21.15.43:0/4275805928 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8160025500 con 0x55775faa3550 2024-04-25T10:44:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.159+0000 7f814effd640 1 -- 172.21.15.43:0/4275805928 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8164059030 con 0x55775faa3550 2024-04-25T10:44:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.159+0000 7f817cc43740 10 client.18936 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.159+0000 7f817cc43740 10 client.18936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:44:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.159+0000 7f817cc43740 20 client.18936 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.159+0000 7f817cc43740 1 --2- 172.21.15.43:0/4275805928 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55775fa60630 0x55775fef3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.159+0000 7f817cc43740 4 client.18936 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T10:44:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.159+0000 7f817cc43740 1 -- 172.21.15.43:0/4275805928 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55775fdda580 con 0x55775fa60630 2024-04-25T10:44:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.159+0000 7f816f7fe640 1 --2- 172.21.15.43:0/4275805928 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55775fa60630 0x55775fef3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.160+0000 7f814effd640 10 client.18936 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.160+0000 7f816f7fe640 1 --2- 172.21.15.43:0/4275805928 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55775fa60630 0x55775fef3300 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=583fed7db903818b server_cookie=78da96bc4a53cc33 in_seq=0 out_seq=0 2024-04-25T10:44:02.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.161+0000 7f814effd640 1 -- 172.21.15.43:0/4275805928 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55775fdda580 con 0x55775fa60630 2024-04-25T10:44:02.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.161+0000 7f814effd640 10 client.18936 handle_command_reply: tid=0 2024-04-25T10:44:02.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.161+0000 7f817cc43740 1 client.18936 shutdown 2024-04-25T10:44:02.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.161+0000 7f817cc43740 1 -- 172.21.15.43:0/4275805928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faa3550 msgr2=0x55775fe7f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:02.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.161+0000 7f817cc43740 1 --2- 172.21.15.43:0/4275805928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faa3550 0x55775fe7f030 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f8164009900 tx=0x7f8164031400 comp rx=0 tx=0).stop 2024-04-25T10:44:02.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.161+0000 7f817cc43740 1 -- 172.21.15.43:0/4275805928 shutdown_connections 2024-04-25T10:44:02.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.161+0000 7f817cc43740 1 --2- 172.21.15.43:0/4275805928 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55775fa60630 0x55775fef3300 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.161+0000 7f817cc43740 1 --2- 172.21.15.43:0/4275805928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55775facc410 0x55775fe7a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.161+0000 7f817cc43740 1 --2- 172.21.15.43:0/4275805928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55775faab730 0x55775fe7a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.161+0000 7f817cc43740 1 --2- 172.21.15.43:0/4275805928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775faa3550 0x55775fe7f030 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.161+0000 7f817cc43740 1 -- 172.21.15.43:0/4275805928 >> 172.21.15.43:0/4275805928 conn(0x55775fac0560 msgr2=0x55775fe6c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:02.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.161+0000 7f814effd640 0 client.18936 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:02.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.161+0000 7f817cc43740 1 -- 172.21.15.43:0/4275805928 shutdown_connections 2024-04-25T10:44:02.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.161+0000 7f817cc43740 1 -- 172.21.15.43:0/4275805928 wait complete. 2024-04-25T10:44:02.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.161+0000 7f817cc43740 20 client.18936 trim_cache size 0 max 16384 2024-04-25T10:44:02.168 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:44:02.168 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T10:44:02.168 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "713f587d-05a9-42ce-a349-ae5847d11b5d", 2024-04-25T10:44:02.168 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T10:44:02.168 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:44:02.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.162+0000 7f8175d8f640 1 -- 172.21.15.43:0/2977146582 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f81580afc60 msgr2=0x7f81580b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:02.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.162+0000 7f8175d8f640 1 --2- 172.21.15.43:0/2977146582 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f81580afc60 0x7f81580b2170 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f81680a3710 tx=0x7f8168090040 comp rx=0 tx=0).stop 2024-04-25T10:44:02.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.162+0000 7f8175d8f640 1 -- 172.21.15.43:0/2977146582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8170107470 msgr2=0x7f817019f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:02.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.162+0000 7f8175d8f640 1 --2- 172.21.15.43:0/2977146582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8170107470 0x7f817019f020 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f816800ec60 tx=0x7f816800c7a0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.162+0000 7f8175d8f640 1 -- 172.21.15.43:0/2977146582 shutdown_connections 2024-04-25T10:44:02.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.162+0000 7f8175d8f640 1 --2- 172.21.15.43:0/2977146582 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f81580afc60 0x7f81580b2170 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.162+0000 7f8175d8f640 1 --2- 172.21.15.43:0/2977146582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8170107470 0x7f817019f020 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.162+0000 7f8175d8f640 1 --2- 172.21.15.43:0/2977146582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81701069b0 0x7f817019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.162+0000 7f8175d8f640 1 --2- 172.21.15.43:0/2977146582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8170102420 0x7f817019a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.162+0000 7f8175d8f640 1 -- 172.21.15.43:0/2977146582 >> 172.21.15.43:0/2977146582 conn(0x7f8170077920 msgr2=0x7f817010df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:02.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.162+0000 7f8175d8f640 1 -- 172.21.15.43:0/2977146582 shutdown_connections 2024-04-25T10:44:02.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.162+0000 7f8175d8f640 1 -- 172.21.15.43:0/2977146582 wait complete. 2024-04-25T10:44:02.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:44:02.175 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:44:02.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.288+0000 7fcf142b1640 1 -- 172.21.15.43:0/3439604622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0c10d440 msgr2=0x7fcf0c10f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:02.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.288+0000 7fcf142b1640 1 --2- 172.21.15.43:0/3439604622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0c10d440 0x7fcf0c10f860 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009ca0 tx=0x7fcf0402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.288+0000 7fcf142b1640 1 -- 172.21.15.43:0/3439604622 shutdown_connections 2024-04-25T10:44:02.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.288+0000 7fcf142b1640 1 --2- 172.21.15.43:0/3439604622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0c10d440 0x7fcf0c10f860 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.288+0000 7fcf142b1640 1 --2- 172.21.15.43:0/3439604622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf0c1032c0 0x7fcf0c1036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.288+0000 7fcf142b1640 1 --2- 172.21.15.43:0/3439604622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf0c107950 0x7fcf0c102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.288+0000 7fcf142b1640 1 -- 172.21.15.43:0/3439604622 >> 172.21.15.43:0/3439604622 conn(0x7fcf0c077920 msgr2=0x7fcf0c077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:02.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.288+0000 7fcf142b1640 1 -- 172.21.15.43:0/3439604622 shutdown_connections 2024-04-25T10:44:02.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.288+0000 7fcf142b1640 1 -- 172.21.15.43:0/3439604622 wait complete. 2024-04-25T10:44:02.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.289+0000 7fcf142b1640 1 Processor -- start 2024-04-25T10:44:02.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.289+0000 7fcf142b1640 1 -- start start 2024-04-25T10:44:02.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.289+0000 7fcf142b1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf0c1032c0 0x7fcf0c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.289+0000 7fcf142b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0c107950 0x7fcf0c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.289+0000 7fcf142b1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf0c10d440 0x7fcf0c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.289+0000 7fcf142b1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c113db0 con 0x7fcf0c107950 2024-04-25T10:44:02.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.289+0000 7fcf142b1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c113c30 con 0x7fcf0c10d440 2024-04-25T10:44:02.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.289+0000 7fcf142b1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c113f30 con 0x7fcf0c1032c0 2024-04-25T10:44:02.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.289+0000 7fcf1184c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0c107950 0x7fcf0c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.290+0000 7fcf1184c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0c107950 0x7fcf0c19ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33160/0 (socket says 172.21.15.43:33160) 2024-04-25T10:44:02.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.290+0000 7fcf1204d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf0c1032c0 0x7fcf0c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.290+0000 7fcf1184c640 1 -- 172.21.15.43:0/2927062113 learned_addr learned my addr 172.21.15.43:0/2927062113 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:02.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.290+0000 7fcf1284e640 1 --2- 172.21.15.43:0/2927062113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf0c10d440 0x7fcf0c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.290+0000 7fcf1184c640 1 -- 172.21.15.43:0/2927062113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf0c1032c0 msgr2=0x7fcf0c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:02.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.290+0000 7fcf1184c640 1 --2- 172.21.15.43:0/2927062113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf0c1032c0 0x7fcf0c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.290+0000 7fcf1184c640 1 -- 172.21.15.43:0/2927062113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf0c10d440 msgr2=0x7fcf0c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:02.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.290+0000 7fcf1184c640 1 --2- 172.21.15.43:0/2927062113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf0c10d440 0x7fcf0c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.290+0000 7fcf1184c640 1 -- 172.21.15.43:0/2927062113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0c19f550 con 0x7fcf0c107950 2024-04-25T10:44:02.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.290+0000 7fcf1184c640 1 --2- 172.21.15.43:0/2927062113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0c107950 0x7fcf0c19ac50 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00b450 tx=0x7fcefc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:02.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.290+0000 7fcf037fe640 1 -- 172.21.15.43:0/2927062113 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc015020 con 0x7fcf0c107950 2024-04-25T10:44:02.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.290+0000 7fcf142b1640 1 -- 172.21.15.43:0/2927062113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf0c19f840 con 0x7fcf0c107950 2024-04-25T10:44:02.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.290+0000 7fcf037fe640 1 -- 172.21.15.43:0/2927062113 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcefc011d40 con 0x7fcf0c107950 2024-04-25T10:44:02.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.290+0000 7fcf037fe640 1 -- 172.21.15.43:0/2927062113 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc00f340 con 0x7fcf0c107950 2024-04-25T10:44:02.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.290+0000 7fcf142b1640 1 -- 172.21.15.43:0/2927062113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf0c1a71c0 con 0x7fcf0c107950 2024-04-25T10:44:02.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.293+0000 7fcf1adfd740 1 Processor -- start 2024-04-25T10:44:02.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.293+0000 7fcf037fe640 1 -- 172.21.15.43:0/2927062113 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fcefc018020 con 0x7fcf0c107950 2024-04-25T10:44:02.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.293+0000 7fcf1adfd740 1 -- start start 2024-04-25T10:44:02.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.294+0000 7fcf1adfd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95d5390 0x560ca9975a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.294+0000 7fcf1adfd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95ac4d0 0x560ca9975fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.294+0000 7fcf1adfd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ca95b46b0 0x560ca99828f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.294+0000 7fcf1adfd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560ca98dbd80 con 0x560ca95ac4d0 2024-04-25T10:44:02.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.294+0000 7fcf1adfd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560ca98958b0 con 0x560ca95d5390 2024-04-25T10:44:02.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.294+0000 7fcf1adfd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560ca98d7d40 con 0x560ca95b46b0 2024-04-25T10:44:02.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.294+0000 7fcf1204d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95ac4d0 0x560ca9975fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.294+0000 7fcf1204d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95ac4d0 0x560ca9975fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33170/0 (socket says 172.21.15.43:33170) 2024-04-25T10:44:02.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.294+0000 7fcf1204d640 1 -- 172.21.15.43:0/1170777728 learned_addr learned my addr 172.21.15.43:0/1170777728 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:02.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.294+0000 7fcf1184c640 1 --2- 172.21.15.43:0/1170777728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ca95b46b0 0x560ca99828f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.294+0000 7fcf1284e640 1 --2- 172.21.15.43:0/1170777728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95d5390 0x560ca9975a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.294+0000 7fcf037fe640 1 --2- 172.21.15.43:0/2927062113 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcef80afc60 0x7fcef80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.294+0000 7fcf1204d640 1 -- 172.21.15.43:0/1170777728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ca95b46b0 msgr2=0x560ca99828f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:02.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.294+0000 7fcf1204d640 1 --2- 172.21.15.43:0/1170777728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ca95b46b0 0x560ca99828f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.294+0000 7fcf1204d640 1 -- 172.21.15.43:0/1170777728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95d5390 msgr2=0x560ca9975a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:02.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.294+0000 7fcf1204d640 1 --2- 172.21.15.43:0/1170777728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95d5390 0x560ca9975a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.294+0000 7fcf1204d640 1 -- 172.21.15.43:0/1170777728 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ca9866a40 con 0x560ca95ac4d0 2024-04-25T10:44:02.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.294+0000 7fcf1284e640 1 --2- 172.21.15.43:0/1170777728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95d5390 0x560ca9975a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:02.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.294+0000 7fcf1284e640 1 --2- 172.21.15.43:0/2927062113 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcef80afc60 0x7fcef80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.294+0000 7fcf037fe640 1 -- 172.21.15.43:0/2927062113 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fcefc0de920 con 0x7fcf0c107950 2024-04-25T10:44:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.295+0000 7fcf1204d640 1 --2- 172.21.15.43:0/1170777728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95ac4d0 0x560ca9975fa0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800bd10 tx=0x7fcf080107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58035d50a9bacd4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.295+0000 7fcf00bfe640 1 -- 172.21.15.43:0/1170777728 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08004030 con 0x560ca95ac4d0 2024-04-25T10:44:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.295+0000 7fcf1284e640 1 --2- 172.21.15.43:0/2927062113 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcef80afc60 0x7fcef80b2170 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcf04004af0 tx=0x7fcf0403f040 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.295+0000 7fcf00bfe640 1 -- 172.21.15.43:0/1170777728 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf080035f0 con 0x560ca95ac4d0 2024-04-25T10:44:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.295+0000 7fcf00bfe640 1 -- 172.21.15.43:0/1170777728 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08011750 con 0x560ca95ac4d0 2024-04-25T10:44:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.296+0000 7fcf1adfd740 1 -- 172.21.15.43:0/1170777728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95ac4d0 msgr2=0x560ca9975fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.296+0000 7fcf1adfd740 1 --2- 172.21.15.43:0/1170777728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95ac4d0 0x560ca9975fa0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800bd10 tx=0x7fcf080107a0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.296+0000 7fcf1adfd740 1 -- 172.21.15.43:0/1170777728 shutdown_connections 2024-04-25T10:44:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.296+0000 7fcf1adfd740 1 --2- 172.21.15.43:0/1170777728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ca95b46b0 0x560ca99828f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.296+0000 7fcf1adfd740 1 --2- 172.21.15.43:0/1170777728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95ac4d0 0x560ca9975fa0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.296+0000 7fcf1adfd740 1 --2- 172.21.15.43:0/1170777728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95d5390 0x560ca9975a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.296+0000 7fcf1adfd740 1 -- 172.21.15.43:0/1170777728 >> 172.21.15.43:0/1170777728 conn(0x560ca95c94e0 msgr2=0x560ca9972210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.296+0000 7fcf1adfd740 1 -- 172.21.15.43:0/1170777728 shutdown_connections 2024-04-25T10:44:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.296+0000 7fcf1adfd740 1 -- 172.21.15.43:0/1170777728 wait complete. 2024-04-25T10:44:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.296+0000 7fcf1adfd740 1 -- start start 2024-04-25T10:44:02.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.296+0000 7fcf1adfd740 1 Processor -- start 2024-04-25T10:44:02.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcf1adfd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ca95ac4d0 0x560ca99805b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcf1adfd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95b46b0 0x560ca9980af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcf1adfd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95d5390 0x560ca998c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcf1adfd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560ca9982fe0 con 0x560ca95b46b0 2024-04-25T10:44:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcf1adfd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560ca9862d10 con 0x560ca95d5390 2024-04-25T10:44:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcf1adfd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560ca9983160 con 0x560ca95ac4d0 2024-04-25T10:44:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcf1adfd740 10 client.0 _ll_register_callbacks cb 0x560ca988f210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcf1184c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95b46b0 0x560ca9980af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcf1184c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95b46b0 0x560ca9980af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33172/0 (socket says 172.21.15.43:33172) 2024-04-25T10:44:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcf1184c640 1 -- 172.21.15.43:0/2376018633 learned_addr learned my addr 172.21.15.43:0/2376018633 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcf1204d640 1 --2- 172.21.15.43:0/2376018633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ca95ac4d0 0x560ca99805b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcf1284e640 1 --2- 172.21.15.43:0/2376018633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95d5390 0x560ca998c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcf1184c640 1 -- 172.21.15.43:0/2376018633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ca95ac4d0 msgr2=0x560ca99805b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcf1184c640 1 --2- 172.21.15.43:0/2376018633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ca95ac4d0 0x560ca99805b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcf1184c640 1 -- 172.21.15.43:0/2376018633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95d5390 msgr2=0x560ca998c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcf1184c640 1 --2- 172.21.15.43:0/2376018633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95d5390 0x560ca998c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcf1184c640 1 -- 172.21.15.43:0/2376018633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ca96b77d0 con 0x560ca95b46b0 2024-04-25T10:44:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcf1184c640 1 --2- 172.21.15.43:0/2376018633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95b46b0 0x560ca9980af0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fcefc0dc670 tx=0x7fcefc010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9c4ee4ab8f743a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcef5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcef5ffb640 1 -- 172.21.15.43:0/2376018633 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0a6520 con 0x560ca95b46b0 2024-04-25T10:44:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcef5ffb640 1 -- 172.21.15.43:0/2376018633 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fcefc0a5e30 con 0x560ca95b46b0 2024-04-25T10:44:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcf1adfd740 1 -- 172.21.15.43:0/2376018633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf080035f0 con 0x560ca95b46b0 2024-04-25T10:44:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.297+0000 7fcef5ffb640 1 -- 172.21.15.43:0/2376018633 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcefc009280 con 0x560ca95b46b0 2024-04-25T10:44:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.298+0000 7fcf1adfd740 10 client.18948 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.298+0000 7fcf1adfd740 1 -- 172.21.15.43:0/2376018633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560ca95ed3f0 con 0x560ca95b46b0 2024-04-25T10:44:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.298+0000 7fcef5ffb640 1 -- 172.21.15.43:0/2376018633 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf080035f0 con 0x560ca95b46b0 2024-04-25T10:44:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.298+0000 7fcef5ffb640 1 -- 172.21.15.43:0/2376018633 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcefc0a58a0 con 0x560ca95b46b0 2024-04-25T10:44:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.298+0000 7fcf1adfd740 10 client.18948 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.298+0000 7fcf1adfd740 10 client.18948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:44:02.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.298+0000 7fcf1adfd740 20 client.18948 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:02.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.298+0000 7fcf1adfd740 1 --2- 172.21.15.43:0/2376018633 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560ca9593f60 0x560ca9992620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.298+0000 7fcf1adfd740 4 client.18948 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:44:02.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.298+0000 7fcf1adfd740 1 -- 172.21.15.43:0/2376018633 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ca98e9990 con 0x560ca9593f60 2024-04-25T10:44:02.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.298+0000 7fcf1204d640 1 --2- 172.21.15.43:0/2376018633 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560ca9593f60 0x560ca9992620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.299+0000 7fcef5ffb640 10 client.18948 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:02.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.299+0000 7fcf1204d640 1 --2- 172.21.15.43:0/2376018633 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560ca9593f60 0x560ca9992620 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b663b0fd3c65aa0e server_cookie=4c6c78aa991b75c6 in_seq=0 out_seq=0 2024-04-25T10:44:02.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.299+0000 7fcef5ffb640 1 -- 172.21.15.43:0/2376018633 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560ca98e9990 con 0x560ca9593f60 2024-04-25T10:44:02.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.299+0000 7fcef5ffb640 10 client.18948 handle_command_reply: tid=0 2024-04-25T10:44:02.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.299+0000 7fcf1adfd740 1 client.18948 shutdown 2024-04-25T10:44:02.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.299+0000 7fcf1adfd740 1 -- 172.21.15.43:0/2376018633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95b46b0 msgr2=0x560ca9980af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:02.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.299+0000 7fcf1adfd740 1 --2- 172.21.15.43:0/2376018633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95b46b0 0x560ca9980af0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fcefc0dc670 tx=0x7fcefc010800 comp rx=0 tx=0).stop 2024-04-25T10:44:02.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.299+0000 7fcf1adfd740 1 -- 172.21.15.43:0/2376018633 shutdown_connections 2024-04-25T10:44:02.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.299+0000 7fcf1adfd740 1 --2- 172.21.15.43:0/2376018633 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560ca9593f60 0x560ca9992620 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.300+0000 7fcf1adfd740 1 --2- 172.21.15.43:0/2376018633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95d5390 0x560ca998c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.300+0000 7fcef5ffb640 0 client.18948 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:02.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.300+0000 7fcf1adfd740 1 --2- 172.21.15.43:0/2376018633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95b46b0 0x560ca9980af0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.300+0000 7fcf1adfd740 1 --2- 172.21.15.43:0/2376018633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ca95ac4d0 0x560ca99805b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.300+0000 7fcf1adfd740 1 -- 172.21.15.43:0/2376018633 >> 172.21.15.43:0/2376018633 conn(0x560ca95c94e0 msgr2=0x560ca9977780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:02.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.300+0000 7fcf1adfd740 1 -- 172.21.15.43:0/2376018633 shutdown_connections 2024-04-25T10:44:02.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.300+0000 7fcf1adfd740 1 -- 172.21.15.43:0/2376018633 wait complete. 2024-04-25T10:44:02.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.300+0000 7fcf1adfd740 20 client.18948 trim_cache size 0 max 16384 2024-04-25T10:44:02.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.318+0000 7fcf1adfd740 1 Processor -- start 2024-04-25T10:44:02.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.318+0000 7fcf1adfd740 1 -- start start 2024-04-25T10:44:02.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.318+0000 7fcf1adfd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca9593f60 0x560ca9975e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.318+0000 7fcf1adfd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ca95ac4d0 0x560ca997b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.318+0000 7fcf1adfd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95b46b0 0x560ca997f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.318+0000 7fcf1adfd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c1b2310 con 0x560ca9593f60 2024-04-25T10:44:02.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.318+0000 7fcf1adfd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c1afbe0 con 0x560ca95b46b0 2024-04-25T10:44:02.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.318+0000 7fcf1adfd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c1ada50 con 0x560ca95ac4d0 2024-04-25T10:44:02.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.318+0000 7fcf1204d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca9593f60 0x560ca9975e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.318+0000 7fcf1204d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca9593f60 0x560ca9975e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33186/0 (socket says 172.21.15.43:33186) 2024-04-25T10:44:02.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.318+0000 7fcf1204d640 1 -- 172.21.15.43:0/3092655963 learned_addr learned my addr 172.21.15.43:0/3092655963 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:02.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.318+0000 7fcf1204d640 1 -- 172.21.15.43:0/3092655963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ca95ac4d0 msgr2=0x560ca997b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:02.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.318+0000 7fcf1204d640 1 --2- 172.21.15.43:0/3092655963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ca95ac4d0 0x560ca997b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.318+0000 7fcf1204d640 1 -- 172.21.15.43:0/3092655963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95b46b0 msgr2=0x560ca997f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:02.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.318+0000 7fcf1284e640 1 --2- 172.21.15.43:0/3092655963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95b46b0 0x560ca997f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.318+0000 7fcf1204d640 1 --2- 172.21.15.43:0/3092655963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95b46b0 0x560ca997f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.319+0000 7fcf1204d640 1 -- 172.21.15.43:0/3092655963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ca95ad0f0 con 0x560ca9593f60 2024-04-25T10:44:02.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.319+0000 7fcf1284e640 1 --2- 172.21.15.43:0/3092655963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95b46b0 0x560ca997f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:02.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.319+0000 7fcf1204d640 1 --2- 172.21.15.43:0/3092655963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca9593f60 0x560ca9975e60 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x560ca99731a0 tx=0x7fcf08003e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a96234f0377fec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:02.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.319+0000 7fcef7fff640 1 -- 172.21.15.43:0/3092655963 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0802f070 con 0x560ca9593f60 2024-04-25T10:44:02.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.319+0000 7fcef7fff640 1 -- 172.21.15.43:0/3092655963 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf08005040 con 0x560ca9593f60 2024-04-25T10:44:02.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.319+0000 7fcef7fff640 1 -- 172.21.15.43:0/3092655963 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0802a070 con 0x560ca9593f60 2024-04-25T10:44:02.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.320+0000 7fcf1adfd740 1 -- 172.21.15.43:0/3092655963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca9593f60 msgr2=0x560ca9975e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:02.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.320+0000 7fcf1adfd740 1 --2- 172.21.15.43:0/3092655963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca9593f60 0x560ca9975e60 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x560ca99731a0 tx=0x7fcf08003e10 comp rx=0 tx=0).stop 2024-04-25T10:44:02.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.320+0000 7fcf1adfd740 1 -- 172.21.15.43:0/3092655963 shutdown_connections 2024-04-25T10:44:02.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.320+0000 7fcf1adfd740 1 --2- 172.21.15.43:0/3092655963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95b46b0 0x560ca997f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.320+0000 7fcf1adfd740 1 --2- 172.21.15.43:0/3092655963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ca95ac4d0 0x560ca997b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.320+0000 7fcf1adfd740 1 --2- 172.21.15.43:0/3092655963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca9593f60 0x560ca9975e60 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.320+0000 7fcf1adfd740 1 -- 172.21.15.43:0/3092655963 >> 172.21.15.43:0/3092655963 conn(0x560ca95c94e0 msgr2=0x560ca997ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:02.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.320+0000 7fcf1adfd740 1 -- 172.21.15.43:0/3092655963 shutdown_connections 2024-04-25T10:44:02.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.320+0000 7fcf1adfd740 1 -- 172.21.15.43:0/3092655963 wait complete. 2024-04-25T10:44:02.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.320+0000 7fcf1adfd740 1 -- start start 2024-04-25T10:44:02.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.320+0000 7fcf1adfd740 1 Processor -- start 2024-04-25T10:44:02.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.320+0000 7fcf1adfd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ca9593f60 0x560ca99887b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.320+0000 7fcf1adfd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95ac4d0 0x560ca9983b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.320+0000 7fcf1adfd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95b46b0 0x560ca9984040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.320+0000 7fcf1adfd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c1ae0a0 con 0x560ca95b46b0 2024-04-25T10:44:02.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.320+0000 7fcf1adfd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c1af510 con 0x560ca95ac4d0 2024-04-25T10:44:02.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.320+0000 7fcf1adfd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c1ae770 con 0x560ca9593f60 2024-04-25T10:44:02.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.320+0000 7fcf1adfd740 10 client.0 _ll_register_callbacks cb 0x7fcefc0e4240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:02.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.321+0000 7fcf1284e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95b46b0 0x560ca9984040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.321+0000 7fcf1284e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95b46b0 0x560ca9984040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33194/0 (socket says 172.21.15.43:33194) 2024-04-25T10:44:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.321+0000 7fcf1284e640 1 -- 172.21.15.43:0/2253455991 learned_addr learned my addr 172.21.15.43:0/2253455991 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.321+0000 7fcf1204d640 1 --2- 172.21.15.43:0/2253455991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ca9593f60 0x560ca99887b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.321+0000 7fcf1184c640 1 --2- 172.21.15.43:0/2253455991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95ac4d0 0x560ca9983b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.321+0000 7fcf1284e640 1 -- 172.21.15.43:0/2253455991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ca9593f60 msgr2=0x560ca99887b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.321+0000 7fcf1284e640 1 --2- 172.21.15.43:0/2253455991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ca9593f60 0x560ca99887b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.321+0000 7fcf1284e640 1 -- 172.21.15.43:0/2253455991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95ac4d0 msgr2=0x560ca9983b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.321+0000 7fcf1284e640 1 --2- 172.21.15.43:0/2253455991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95ac4d0 0x560ca9983b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.321+0000 7fcf1284e640 1 -- 172.21.15.43:0/2253455991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ca989c4b0 con 0x560ca95b46b0 2024-04-25T10:44:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.321+0000 7fcf1284e640 1 --2- 172.21.15.43:0/2253455991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95b46b0 0x560ca9984040 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fcf04006010 tx=0x7fcf04005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74096c385d56ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.321+0000 7fcef5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.321+0000 7fcef5ffb640 1 -- 172.21.15.43:0/2253455991 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0402fdb0 con 0x560ca95b46b0 2024-04-25T10:44:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.321+0000 7fcef5ffb640 1 -- 172.21.15.43:0/2253455991 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fcf04003970 con 0x560ca95b46b0 2024-04-25T10:44:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.321+0000 7fcf1adfd740 1 -- 172.21.15.43:0/2253455991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf08005040 con 0x560ca95b46b0 2024-04-25T10:44:02.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.321+0000 7fcef5ffb640 1 -- 172.21.15.43:0/2253455991 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcf04003b70 con 0x560ca95b46b0 2024-04-25T10:44:02.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.321+0000 7fcf1adfd740 10 client.18954 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:02.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.321+0000 7fcf1adfd740 1 -- 172.21.15.43:0/2253455991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560ca99fa3f0 con 0x560ca95b46b0 2024-04-25T10:44:02.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.322+0000 7fcef5ffb640 1 -- 172.21.15.43:0/2253455991 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf08005040 con 0x560ca95b46b0 2024-04-25T10:44:02.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.322+0000 7fcef5ffb640 1 -- 172.21.15.43:0/2253455991 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcf04030650 con 0x560ca95b46b0 2024-04-25T10:44:02.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.322+0000 7fcf1adfd740 10 client.18954 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:02.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.322+0000 7fcf1adfd740 10 client.18954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:44:02.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.322+0000 7fcf1adfd740 20 client.18954 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:02.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.322+0000 7fcf1adfd740 1 --2- 172.21.15.43:0/2253455991 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560ca95d5390 0x560ca99fcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:02.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.322+0000 7fcf1adfd740 4 client.18954 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:44:02.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.322+0000 7fcf1adfd740 1 -- 172.21.15.43:0/2253455991 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560ca98e3800 con 0x560ca95d5390 2024-04-25T10:44:02.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.322+0000 7fcf1204d640 1 --2- 172.21.15.43:0/2253455991 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560ca95d5390 0x560ca99fcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:02.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.322+0000 7fcf1204d640 1 --2- 172.21.15.43:0/2253455991 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560ca95d5390 0x560ca99fcb90 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae8996c2dbeea0f server_cookie=ed146644783e629f in_seq=0 out_seq=0 2024-04-25T10:44:02.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.322+0000 7fcef5ffb640 10 client.18954 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:02.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.323+0000 7fcef5ffb640 1 -- 172.21.15.43:0/2253455991 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560ca98e3800 con 0x560ca95d5390 2024-04-25T10:44:02.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.323+0000 7fcef5ffb640 10 client.18954 handle_command_reply: tid=0 2024-04-25T10:44:02.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.323+0000 7fcf1adfd740 1 client.18954 shutdown 2024-04-25T10:44:02.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.323+0000 7fcf1adfd740 1 -- 172.21.15.43:0/2253455991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95b46b0 msgr2=0x560ca9984040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:02.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.323+0000 7fcf1adfd740 1 --2- 172.21.15.43:0/2253455991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95b46b0 0x560ca9984040 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fcf04006010 tx=0x7fcf04005d80 comp rx=0 tx=0).stop 2024-04-25T10:44:02.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.323+0000 7fcf1adfd740 1 -- 172.21.15.43:0/2253455991 shutdown_connections 2024-04-25T10:44:02.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.323+0000 7fcf1adfd740 1 --2- 172.21.15.43:0/2253455991 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x560ca95d5390 0x560ca99fcb90 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.323+0000 7fcf1adfd740 1 --2- 172.21.15.43:0/2253455991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ca95ac4d0 0x560ca9983b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.323+0000 7fcf1adfd740 1 --2- 172.21.15.43:0/2253455991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ca95b46b0 0x560ca9984040 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.323+0000 7fcf1adfd740 1 --2- 172.21.15.43:0/2253455991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ca9593f60 0x560ca99887b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.323+0000 7fcf1adfd740 1 -- 172.21.15.43:0/2253455991 >> 172.21.15.43:0/2253455991 conn(0x560ca95c94e0 msgr2=0x560ca9981860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:02.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.323+0000 7fcef5ffb640 0 client.18954 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:02.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.323+0000 7fcf1adfd740 1 -- 172.21.15.43:0/2253455991 shutdown_connections 2024-04-25T10:44:02.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.323+0000 7fcf1adfd740 1 -- 172.21.15.43:0/2253455991 wait complete. 2024-04-25T10:44:02.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.323+0000 7fcf1adfd740 20 client.18954 trim_cache size 0 max 16384 2024-04-25T10:44:02.333 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:44:02.333 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:44:02.333 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T10:44:02.334 INFO:teuthology.orchestra.run.smithi043.stdout: "713f587d-05a9-42ce-a349-ae5847d11b5d": { 2024-04-25T10:44:02.334 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T10:44:02.334 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "713f587d-05a9-42ce-a349-ae5847d11b5d", 2024-04-25T10:44:02.334 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-25T10:44:02.334 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:02.334 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:02.334 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:44:02.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.323+0000 7fcf142b1640 1 -- 172.21.15.43:0/2927062113 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcef80afc60 msgr2=0x7fcef80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:02.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.323+0000 7fcf142b1640 1 --2- 172.21.15.43:0/2927062113 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcef80afc60 0x7fcef80b2170 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcf04004af0 tx=0x7fcf0403f040 comp rx=0 tx=0).stop 2024-04-25T10:44:02.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.323+0000 7fcf142b1640 1 -- 172.21.15.43:0/2927062113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0c107950 msgr2=0x7fcf0c19ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:02.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.323+0000 7fcf142b1640 1 --2- 172.21.15.43:0/2927062113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0c107950 0x7fcf0c19ac50 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00b450 tx=0x7fcefc00b910 comp rx=0 tx=0).stop 2024-04-25T10:44:02.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.324+0000 7fcf142b1640 1 -- 172.21.15.43:0/2927062113 shutdown_connections 2024-04-25T10:44:02.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.324+0000 7fcf142b1640 1 --2- 172.21.15.43:0/2927062113 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fcef80afc60 0x7fcef80b2170 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.324+0000 7fcf142b1640 1 --2- 172.21.15.43:0/2927062113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf0c10d440 0x7fcf0c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.324+0000 7fcf142b1640 1 --2- 172.21.15.43:0/2927062113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0c107950 0x7fcf0c19ac50 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.324+0000 7fcf142b1640 1 --2- 172.21.15.43:0/2927062113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf0c1032c0 0x7fcf0c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:02.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.324+0000 7fcf142b1640 1 -- 172.21.15.43:0/2927062113 >> 172.21.15.43:0/2927062113 conn(0x7fcf0c077920 msgr2=0x7fcf0c1008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:02.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.324+0000 7fcf142b1640 1 -- 172.21.15.43:0/2927062113 shutdown_connections 2024-04-25T10:44:02.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:02.324+0000 7fcf142b1640 1 -- 172.21.15.43:0/2927062113 wait complete. 2024-04-25T10:44:02.336 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:02 smithi043 ceph-mon[29478]: pgmap v358: 97 pgs: 97 active+clean; 2.6 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-25T10:44:02.337 INFO:tasks.cephfs.filesystem:scrub status for tag:713f587d-05a9-42ce-a349-ae5847d11b5d - {'path': '/', 'tag': '713f587d-05a9-42ce-a349-ae5847d11b5d', 'options': 'recursive,force'} 2024-04-25T10:44:02.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:02 smithi077 ceph-mon[32772]: pgmap v358: 97 pgs: 97 active+clean; 2.6 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-25T10:44:02.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:02 smithi164 ceph-mon[32713]: pgmap v358: 97 pgs: 97 active+clean; 2.6 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-25T10:44:03.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:03 smithi043 ceph-mon[29478]: scrub queued for path: / 2024-04-25T10:44:03.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:03 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:44:03.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:03 smithi043 ceph-mon[29478]: scrub summary: active paths [/] 2024-04-25T10:44:03.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:03 smithi043 ceph-mon[29478]: scrub summary: idle+waiting paths [/] 2024-04-25T10:44:03.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:03 smithi077 ceph-mon[32772]: scrub queued for path: / 2024-04-25T10:44:03.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:03 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:44:03.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:03 smithi077 ceph-mon[32772]: scrub summary: active paths [/] 2024-04-25T10:44:03.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:03 smithi077 ceph-mon[32772]: scrub summary: idle+waiting paths [/] 2024-04-25T10:44:03.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:03 smithi164 ceph-mon[32713]: scrub queued for path: / 2024-04-25T10:44:03.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:03 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:44:03.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:03 smithi164 ceph-mon[32713]: scrub summary: active paths [/] 2024-04-25T10:44:03.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:03 smithi164 ceph-mon[32713]: scrub summary: idle+waiting paths [/] 2024-04-25T10:44:04.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:04 smithi043 ceph-mon[29478]: pgmap v359: 97 pgs: 97 active+clean; 2.6 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-25T10:44:04.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:04 smithi043 ceph-mon[29478]: scrub summary: idle 2024-04-25T10:44:04.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:04 smithi077 ceph-mon[32772]: pgmap v359: 97 pgs: 97 active+clean; 2.6 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-25T10:44:04.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:04 smithi077 ceph-mon[32772]: scrub summary: idle 2024-04-25T10:44:04.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:04 smithi164 ceph-mon[32713]: pgmap v359: 97 pgs: 97 active+clean; 2.6 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-04-25T10:44:04.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:04 smithi164 ceph-mon[32713]: scrub summary: idle 2024-04-25T10:44:05.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:05 smithi043 ceph-mon[29478]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T10:44:05.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:05 smithi077 ceph-mon[32772]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T10:44:05.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:05 smithi164 ceph-mon[32713]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T10:44:06.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:06 smithi043 ceph-mon[29478]: pgmap v361: 97 pgs: 97 active+clean; 2.8 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 108 KiB/s wr, 24 op/s 2024-04-25T10:44:06.541 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:06 smithi077 ceph-mon[32772]: pgmap v361: 97 pgs: 97 active+clean; 2.8 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 108 KiB/s wr, 24 op/s 2024-04-25T10:44:06.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:06 smithi164 ceph-mon[32713]: pgmap v361: 97 pgs: 97 active+clean; 2.8 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 108 KiB/s wr, 24 op/s 2024-04-25T10:44:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.097+0000 7f17ec69a640 1 -- 172.21.15.43:0/4204550500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17e4109ff0 msgr2=0x7f17e4103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.097+0000 7f17ec69a640 1 --2- 172.21.15.43:0/4204550500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17e4109ff0 0x7f17e4103c30 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009900 tx=0x7f17d402a410 comp rx=0 tx=0).stop 2024-04-25T10:44:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.098+0000 7f17ec69a640 1 -- 172.21.15.43:0/4204550500 shutdown_connections 2024-04-25T10:44:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.098+0000 7f17ec69a640 1 --2- 172.21.15.43:0/4204550500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17e41042a0 0x7f17e4110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.098+0000 7f17ec69a640 1 --2- 172.21.15.43:0/4204550500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17e4109ff0 0x7f17e4103c30 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.098+0000 7f17ec69a640 1 --2- 172.21.15.43:0/4204550500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17e4109b50 0x7f17e41036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.098+0000 7f17ec69a640 1 -- 172.21.15.43:0/4204550500 >> 172.21.15.43:0/4204550500 conn(0x7f17e40ff230 msgr2=0x7f17e4101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.098+0000 7f17ec69a640 1 -- 172.21.15.43:0/4204550500 shutdown_connections 2024-04-25T10:44:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.098+0000 7f17ec69a640 1 -- 172.21.15.43:0/4204550500 wait complete. 2024-04-25T10:44:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.098+0000 7f17ec69a640 1 Processor -- start 2024-04-25T10:44:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.098+0000 7f17ec69a640 1 -- start start 2024-04-25T10:44:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.099+0000 7f17ec69a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17e41042a0 0x7f17e419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.099+0000 7f17ec69a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17e4109b50 0x7f17e419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.099+0000 7f17ec69a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17e4109ff0 0x7f17e419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.099+0000 7f17ec69a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17e4113d40 con 0x7f17e41042a0 2024-04-25T10:44:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.099+0000 7f17ec69a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17e4113bc0 con 0x7f17e4109ff0 2024-04-25T10:44:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.099+0000 7f17ec69a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f17e4113ec0 con 0x7f17e4109b50 2024-04-25T10:44:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.099+0000 7f17ea436640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17e41042a0 0x7f17e419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.099+0000 7f17ea436640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17e41042a0 0x7f17e419a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33216/0 (socket says 172.21.15.43:33216) 2024-04-25T10:44:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.099+0000 7f17ea436640 1 -- 172.21.15.43:0/4088182279 learned_addr learned my addr 172.21.15.43:0/4088182279 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.099+0000 7f17eac37640 1 --2- 172.21.15.43:0/4088182279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17e4109ff0 0x7f17e419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.099+0000 7f17e9c35640 1 --2- 172.21.15.43:0/4088182279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17e4109b50 0x7f17e419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.099+0000 7f17ea436640 1 -- 172.21.15.43:0/4088182279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17e4109b50 msgr2=0x7f17e419ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.099+0000 7f17ea436640 1 --2- 172.21.15.43:0/4088182279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17e4109b50 0x7f17e419ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.099+0000 7f17ea436640 1 -- 172.21.15.43:0/4088182279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17e4109ff0 msgr2=0x7f17e419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.099+0000 7f17ea436640 1 --2- 172.21.15.43:0/4088182279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17e4109ff0 0x7f17e419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.099+0000 7f17ea436640 1 -- 172.21.15.43:0/4088182279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e419f640 con 0x7f17e41042a0 2024-04-25T10:44:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.100+0000 7f17ea436640 1 --2- 172.21.15.43:0/4088182279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17e41042a0 0x7f17e419a7e0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f17e000e9c0 tx=0x7f17e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.100+0000 7f17db7fe640 1 -- 172.21.15.43:0/4088182279 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e000cd80 con 0x7f17e41042a0 2024-04-25T10:44:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.100+0000 7f17db7fe640 1 -- 172.21.15.43:0/4088182279 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17e00054d0 con 0x7f17e41042a0 2024-04-25T10:44:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.100+0000 7f17db7fe640 1 -- 172.21.15.43:0/4088182279 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0010430 con 0x7f17e41042a0 2024-04-25T10:44:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.100+0000 7f17ec69a640 1 -- 172.21.15.43:0/4088182279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e419f930 con 0x7f17e41042a0 2024-04-25T10:44:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.100+0000 7f17ec69a640 1 -- 172.21.15.43:0/4088182279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e41a72b0 con 0x7f17e41042a0 2024-04-25T10:44:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.103+0000 7f17db7fe640 1 -- 172.21.15.43:0/4088182279 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f17e0004fd0 con 0x7f17e41042a0 2024-04-25T10:44:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.103+0000 7f17f361f740 1 Processor -- start 2024-04-25T10:44:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.103+0000 7f17f361f740 1 -- start start 2024-04-25T10:44:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.104+0000 7f17f361f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b07980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.104+0000 7f17f361f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a02473f2f0 0x55a024b07ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.104+0000 7f17f361f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0247474d0 0x55a024b08400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.104+0000 7f17f361f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a024a6e460 con 0x55a0247681b0 2024-04-25T10:44:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.104+0000 7f17f361f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a024a28590 con 0x55a0247474d0 2024-04-25T10:44:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.104+0000 7f17f361f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a024a6a420 con 0x55a02473f2f0 2024-04-25T10:44:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.104+0000 7f17eac37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b07980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.104+0000 7f17ea436640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a02473f2f0 0x55a024b07ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.104+0000 7f17eac37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b07980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33232/0 (socket says 172.21.15.43:33232) 2024-04-25T10:44:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.104+0000 7f17eac37640 1 -- 172.21.15.43:0/3294321296 learned_addr learned my addr 172.21.15.43:0/3294321296 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.104+0000 7f17e9c35640 1 --2- 172.21.15.43:0/3294321296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0247474d0 0x55a024b08400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.104+0000 7f17db7fe640 1 --2- 172.21.15.43:0/4088182279 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f17d00afcd0 0x7f17d00b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.104+0000 7f17eac37640 1 -- 172.21.15.43:0/3294321296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a02473f2f0 msgr2=0x55a024b07ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.104+0000 7f17eac37640 1 --2- 172.21.15.43:0/3294321296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a02473f2f0 0x55a024b07ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.104+0000 7f17eac37640 1 -- 172.21.15.43:0/3294321296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0247474d0 msgr2=0x55a024b08400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.104+0000 7f17eac37640 1 --2- 172.21.15.43:0/3294321296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0247474d0 0x55a024b08400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.104+0000 7f17eac37640 1 -- 172.21.15.43:0/3294321296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a024a350d0 con 0x55a0247681b0 2024-04-25T10:44:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.105+0000 7f17db7fe640 1 -- 172.21.15.43:0/4088182279 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8557+0+0 (secure 0 0 0) 0x7f17e00df450 con 0x7f17e41042a0 2024-04-25T10:44:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.105+0000 7f17eac37640 1 --2- 172.21.15.43:0/3294321296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b07980 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f17dc010370 tx=0x7f17dc010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2ee68baf890a87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.105+0000 7f17eac37640 1 --2- 172.21.15.43:0/4088182279 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f17d00afcd0 0x7f17d00b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.105+0000 7f17eac37640 1 --2- 172.21.15.43:0/4088182279 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f17d00afcd0 0x7f17d00b21e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f17dc01a000 tx=0x7f17dc018000 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:08.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.108+0000 7f17d8bfe640 1 -- 172.21.15.43:0/3294321296 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc016070 con 0x55a0247681b0 2024-04-25T10:44:08.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.108+0000 7f17d8bfe640 1 -- 172.21.15.43:0/3294321296 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17dc01bde0 con 0x55a0247681b0 2024-04-25T10:44:08.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.108+0000 7f17d8bfe640 1 -- 172.21.15.43:0/3294321296 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc025e60 con 0x55a0247681b0 2024-04-25T10:44:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.109+0000 7f17f361f740 1 -- 172.21.15.43:0/3294321296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 msgr2=0x55a024b07980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.109+0000 7f17f361f740 1 --2- 172.21.15.43:0/3294321296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b07980 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f17dc010370 tx=0x7f17dc010830 comp rx=0 tx=0).stop 2024-04-25T10:44:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.109+0000 7f17f361f740 1 -- 172.21.15.43:0/3294321296 shutdown_connections 2024-04-25T10:44:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.109+0000 7f17f361f740 1 --2- 172.21.15.43:0/3294321296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0247474d0 0x55a024b08400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.109+0000 7f17f361f740 1 --2- 172.21.15.43:0/3294321296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a02473f2f0 0x55a024b07ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.109+0000 7f17f361f740 1 --2- 172.21.15.43:0/3294321296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b07980 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.109+0000 7f17f361f740 1 -- 172.21.15.43:0/3294321296 >> 172.21.15.43:0/3294321296 conn(0x55a02475c300 msgr2=0x55a024b042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.109+0000 7f17f361f740 1 -- 172.21.15.43:0/3294321296 shutdown_connections 2024-04-25T10:44:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.109+0000 7f17f361f740 1 -- 172.21.15.43:0/3294321296 wait complete. 2024-04-25T10:44:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.109+0000 7f17f361f740 1 -- start start 2024-04-25T10:44:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.110+0000 7f17f361f740 1 Processor -- start 2024-04-25T10:44:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.110+0000 7f17f361f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a02473f2f0 0x55a024b165d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.110+0000 7f17f361f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0247474d0 0x55a024b16b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.110+0000 7f17f361f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b1c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.110+0000 7f17f361f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a024b19090 con 0x55a0247681b0 2024-04-25T10:44:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.110+0000 7f17f361f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a0249f5b60 con 0x55a02473f2f0 2024-04-25T10:44:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.110+0000 7f17f361f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a024b19210 con 0x55a0247474d0 2024-04-25T10:44:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.110+0000 7f17f361f740 10 client.0 _ll_register_callbacks cb 0x55a024a1a2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.110+0000 7f17eac37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b1c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.110+0000 7f17eac37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b1c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33248/0 (socket says 172.21.15.43:33248) 2024-04-25T10:44:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.110+0000 7f17ea436640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a02473f2f0 0x55a024b165d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.110+0000 7f17eac37640 1 -- 172.21.15.43:0/1341726627 learned_addr learned my addr 172.21.15.43:0/1341726627 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.110+0000 7f17e9c35640 1 --2- 172.21.15.43:0/1341726627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0247474d0 0x55a024b16b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.110+0000 7f17eac37640 1 -- 172.21.15.43:0/1341726627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0247474d0 msgr2=0x55a024b16b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.110+0000 7f17eac37640 1 --2- 172.21.15.43:0/1341726627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0247474d0 0x55a024b16b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.110+0000 7f17eac37640 1 -- 172.21.15.43:0/1341726627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a02473f2f0 msgr2=0x55a024b165d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.110+0000 7f17eac37640 1 --2- 172.21.15.43:0/1341726627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a02473f2f0 0x55a024b165d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.110+0000 7f17eac37640 1 -- 172.21.15.43:0/1341726627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a024780210 con 0x55a0247681b0 2024-04-25T10:44:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.110+0000 7f17ea436640 1 --2- 172.21.15.43:0/1341726627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a02473f2f0 0x55a024b165d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:44:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.111+0000 7f17eac37640 1 --2- 172.21.15.43:0/1341726627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b1c580 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f17dc025960 tx=0x7f17dc012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dabb95a737e4a57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.111+0000 7f17cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.111+0000 7f17cdffb640 1 -- 172.21.15.43:0/1341726627 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc036060 con 0x55a0247681b0 2024-04-25T10:44:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.111+0000 7f17cdffb640 1 -- 172.21.15.43:0/1341726627 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8557+0+0 (secure 0 0 0) 0x7f17dc025b60 con 0x55a0247681b0 2024-04-25T10:44:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.111+0000 7f17f361f740 1 -- 172.21.15.43:0/1341726627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17dc01bde0 con 0x55a0247681b0 2024-04-25T10:44:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.111+0000 7f17cdffb640 1 -- 172.21.15.43:0/1341726627 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f17dc016050 con 0x55a0247681b0 2024-04-25T10:44:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.111+0000 7f17f361f740 10 client.18966 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.111+0000 7f17f361f740 1 -- 172.21.15.43:0/1341726627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a02484c620 con 0x55a0247681b0 2024-04-25T10:44:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.111+0000 7f17cdffb640 1 -- 172.21.15.43:0/1341726627 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17dc01bde0 con 0x55a0247681b0 2024-04-25T10:44:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.111+0000 7f17cdffb640 1 -- 172.21.15.43:0/1341726627 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f17dc00f040 con 0x55a0247681b0 2024-04-25T10:44:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.112+0000 7f17f361f740 10 client.18966 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.112+0000 7f17f361f740 10 client.18966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:44:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.112+0000 7f17f361f740 20 client.18966 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.112+0000 7f17f361f740 1 --2- 172.21.15.43:0/1341726627 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a024726d80 0x55a024b28a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.112+0000 7f17f361f740 4 client.18966 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:44:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.112+0000 7f17f361f740 1 -- 172.21.15.43:0/1341726627 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a024a7c070 con 0x55a024726d80 2024-04-25T10:44:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.112+0000 7f17ea436640 1 --2- 172.21.15.43:0/1341726627 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a024726d80 0x55a024b28a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.112+0000 7f17ea436640 1 --2- 172.21.15.43:0/1341726627 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a024726d80 0x55a024b28a70 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7cd144972ac330f server_cookie=7d0decbec62df4bb in_seq=0 out_seq=0 2024-04-25T10:44:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.112+0000 7f17cdffb640 10 client.18966 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.113+0000 7f17cdffb640 1 -- 172.21.15.43:0/1341726627 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a024a7c070 con 0x55a024726d80 2024-04-25T10:44:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.113+0000 7f17cdffb640 10 client.18966 handle_command_reply: tid=0 2024-04-25T10:44:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.113+0000 7f17f361f740 1 client.18966 shutdown 2024-04-25T10:44:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.113+0000 7f17f361f740 1 -- 172.21.15.43:0/1341726627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 msgr2=0x55a024b1c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.113+0000 7f17f361f740 1 --2- 172.21.15.43:0/1341726627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b1c580 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f17dc025960 tx=0x7f17dc012760 comp rx=0 tx=0).stop 2024-04-25T10:44:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.113+0000 7f17f361f740 1 -- 172.21.15.43:0/1341726627 shutdown_connections 2024-04-25T10:44:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.113+0000 7f17f361f740 1 --2- 172.21.15.43:0/1341726627 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a024726d80 0x55a024b28a70 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.113+0000 7f17f361f740 1 --2- 172.21.15.43:0/1341726627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b1c580 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.113+0000 7f17f361f740 1 --2- 172.21.15.43:0/1341726627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0247474d0 0x55a024b16b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.113+0000 7f17cdffb640 0 client.18966 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.113+0000 7f17f361f740 1 --2- 172.21.15.43:0/1341726627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a02473f2f0 0x55a024b165d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.113+0000 7f17f361f740 1 -- 172.21.15.43:0/1341726627 >> 172.21.15.43:0/1341726627 conn(0x55a02475c300 msgr2=0x55a024b08330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.113+0000 7f17f361f740 1 -- 172.21.15.43:0/1341726627 shutdown_connections 2024-04-25T10:44:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.113+0000 7f17f361f740 1 -- 172.21.15.43:0/1341726627 wait complete. 2024-04-25T10:44:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.114+0000 7f17f361f740 20 client.18966 trim_cache size 0 max 16384 2024-04-25T10:44:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.129+0000 7f17f361f740 1 Processor -- start 2024-04-25T10:44:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.129+0000 7f17f361f740 1 -- start start 2024-04-25T10:44:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.129+0000 7f17f361f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a02473f2f0 0x55a024b1c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.129+0000 7f17f361f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0247474d0 0x55a024b0fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.129+0000 7f17f361f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b10270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.129+0000 7f17f361f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17e41b0270 con 0x55a0247681b0 2024-04-25T10:44:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.129+0000 7f17f361f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17e41adb40 con 0x55a02473f2f0 2024-04-25T10:44:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.129+0000 7f17f361f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f17e41ab9b0 con 0x55a0247474d0 2024-04-25T10:44:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.129+0000 7f17eac37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b10270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.129+0000 7f17eac37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b10270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33260/0 (socket says 172.21.15.43:33260) 2024-04-25T10:44:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.129+0000 7f17eac37640 1 -- 172.21.15.43:0/1580412572 learned_addr learned my addr 172.21.15.43:0/1580412572 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.130+0000 7f17ea436640 1 --2- 172.21.15.43:0/1580412572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a02473f2f0 0x55a024b1c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.130+0000 7f17eac37640 1 -- 172.21.15.43:0/1580412572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0247474d0 msgr2=0x55a024b0fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.130+0000 7f17eac37640 1 --2- 172.21.15.43:0/1580412572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0247474d0 0x55a024b0fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.130+0000 7f17eac37640 1 -- 172.21.15.43:0/1580412572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a02473f2f0 msgr2=0x55a024b1c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.130+0000 7f17eac37640 1 --2- 172.21.15.43:0/1580412572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a02473f2f0 0x55a024b1c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.130+0000 7f17eac37640 1 -- 172.21.15.43:0/1580412572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a024b08e80 con 0x55a0247681b0 2024-04-25T10:44:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.130+0000 7f17ea436640 1 --2- 172.21.15.43:0/1580412572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a02473f2f0 0x55a024b1c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.130+0000 7f17eac37640 1 --2- 172.21.15.43:0/1580412572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b10270 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f17dc012730 tx=0x7f17dc000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74af29f45d1bfe1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.130+0000 7f17cffff640 1 -- 172.21.15.43:0/1580412572 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc036070 con 0x55a0247681b0 2024-04-25T10:44:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.130+0000 7f17cffff640 1 -- 172.21.15.43:0/1580412572 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17dc005bd0 con 0x55a0247681b0 2024-04-25T10:44:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.130+0000 7f17cffff640 1 -- 172.21.15.43:0/1580412572 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc005220 con 0x55a0247681b0 2024-04-25T10:44:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.131+0000 7f17f361f740 1 -- 172.21.15.43:0/1580412572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 msgr2=0x55a024b10270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.131+0000 7f17f361f740 1 --2- 172.21.15.43:0/1580412572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b10270 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f17dc012730 tx=0x7f17dc000f40 comp rx=0 tx=0).stop 2024-04-25T10:44:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.131+0000 7f17f361f740 1 -- 172.21.15.43:0/1580412572 shutdown_connections 2024-04-25T10:44:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.131+0000 7f17f361f740 1 --2- 172.21.15.43:0/1580412572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b10270 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.131+0000 7f17f361f740 1 --2- 172.21.15.43:0/1580412572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0247474d0 0x55a024b0fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.131+0000 7f17f361f740 1 --2- 172.21.15.43:0/1580412572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a02473f2f0 0x55a024b1c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.131+0000 7f17f361f740 1 -- 172.21.15.43:0/1580412572 >> 172.21.15.43:0/1580412572 conn(0x55a02475c300 msgr2=0x55a024b14c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.131+0000 7f17f361f740 1 -- 172.21.15.43:0/1580412572 shutdown_connections 2024-04-25T10:44:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.131+0000 7f17f361f740 1 -- 172.21.15.43:0/1580412572 wait complete. 2024-04-25T10:44:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.131+0000 7f17f361f740 1 -- start start 2024-04-25T10:44:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.131+0000 7f17f361f740 1 Processor -- start 2024-04-25T10:44:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.131+0000 7f17f361f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a02473f2f0 0x55a024b1e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.131+0000 7f17f361f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0247474d0 0x55a024b1e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17f361f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b8eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17f361f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17e41ac000 con 0x55a0247681b0 2024-04-25T10:44:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17f361f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17e41ad470 con 0x55a0247474d0 2024-04-25T10:44:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17f361f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f17e41ac6d0 con 0x55a02473f2f0 2024-04-25T10:44:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17f361f740 10 client.0 _ll_register_callbacks cb 0x7f17d4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17eac37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b8eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17eac37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b8eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33262/0 (socket says 172.21.15.43:33262) 2024-04-25T10:44:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17eac37640 1 -- 172.21.15.43:0/2862727382 learned_addr learned my addr 172.21.15.43:0/2862727382 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17e9c35640 1 --2- 172.21.15.43:0/2862727382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0247474d0 0x55a024b1e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17eac37640 1 -- 172.21.15.43:0/2862727382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a02473f2f0 msgr2=0x55a024b1e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17ea436640 1 --2- 172.21.15.43:0/2862727382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a02473f2f0 0x55a024b1e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17eac37640 1 --2- 172.21.15.43:0/2862727382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a02473f2f0 0x55a024b1e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17eac37640 1 -- 172.21.15.43:0/2862727382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0247474d0 msgr2=0x55a024b1e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17eac37640 1 --2- 172.21.15.43:0/2862727382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0247474d0 0x55a024b1e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17eac37640 1 -- 172.21.15.43:0/2862727382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a024b8f0c0 con 0x55a0247681b0 2024-04-25T10:44:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17e9c35640 1 --2- 172.21.15.43:0/2862727382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0247474d0 0x55a024b1e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17ea436640 1 --2- 172.21.15.43:0/2862727382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a02473f2f0 0x55a024b1e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17eac37640 1 --2- 172.21.15.43:0/2862727382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b8eb80 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f17dc004ec0 tx=0x7f17dc01b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abebafc7cd16a3b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17cdffb640 1 -- 172.21.15.43:0/2862727382 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc047070 con 0x55a0247681b0 2024-04-25T10:44:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17cdffb640 1 -- 172.21.15.43:0/2862727382 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8557+0+0 (secure 0 0 0) 0x7f17dc01bdb0 con 0x55a0247681b0 2024-04-25T10:44:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17f361f740 1 -- 172.21.15.43:0/2862727382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17dc005bd0 con 0x55a0247681b0 2024-04-25T10:44:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.132+0000 7f17cdffb640 1 -- 172.21.15.43:0/2862727382 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f17dc036040 con 0x55a0247681b0 2024-04-25T10:44:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.133+0000 7f17f361f740 10 client.18972 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.133+0000 7f17f361f740 1 -- 172.21.15.43:0/2862727382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a024b902c0 con 0x55a0247681b0 2024-04-25T10:44:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.133+0000 7f17cdffb640 1 -- 172.21.15.43:0/2862727382 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17dc005bd0 con 0x55a0247681b0 2024-04-25T10:44:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.133+0000 7f17cdffb640 1 -- 172.21.15.43:0/2862727382 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f17dc005390 con 0x55a0247681b0 2024-04-25T10:44:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.133+0000 7f17f361f740 10 client.18972 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.133+0000 7f17f361f740 10 client.18972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T10:44:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.133+0000 7f17f361f740 20 client.18972 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.133+0000 7f17f361f740 1 --2- 172.21.15.43:0/2862727382 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a0246fc1c0 0x55a024b928d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.133+0000 7f17f361f740 4 client.18972 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:44:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.133+0000 7f17f361f740 1 -- 172.21.15.43:0/2862727382 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a024a75ee0 con 0x55a0246fc1c0 2024-04-25T10:44:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.133+0000 7f17ea436640 1 --2- 172.21.15.43:0/2862727382 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a0246fc1c0 0x55a024b928d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.134+0000 7f17cdffb640 10 client.18972 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.134+0000 7f17ea436640 1 --2- 172.21.15.43:0/2862727382 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a0246fc1c0 0x55a024b928d0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58ab60b4d407f4df server_cookie=aefa052284e8e425 in_seq=0 out_seq=0 2024-04-25T10:44:08.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.149+0000 7f17cdffb640 1 -- 172.21.15.43:0/2862727382 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a024a75ee0 con 0x55a0246fc1c0 2024-04-25T10:44:08.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.149+0000 7f17cdffb640 10 client.18972 handle_command_reply: tid=0 2024-04-25T10:44:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.149+0000 7f17f361f740 1 client.18972 shutdown 2024-04-25T10:44:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.149+0000 7f17f361f740 1 -- 172.21.15.43:0/2862727382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 msgr2=0x55a024b8eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.149+0000 7f17f361f740 1 --2- 172.21.15.43:0/2862727382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b8eb80 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f17dc004ec0 tx=0x7f17dc01b9b0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.149+0000 7f17f361f740 1 -- 172.21.15.43:0/2862727382 shutdown_connections 2024-04-25T10:44:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.149+0000 7f17f361f740 1 --2- 172.21.15.43:0/2862727382 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55a0246fc1c0 0x55a024b928d0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.149+0000 7f17f361f740 1 --2- 172.21.15.43:0/2862727382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a0247474d0 0x55a024b1e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.149+0000 7f17f361f740 1 --2- 172.21.15.43:0/2862727382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0247681b0 0x55a024b8eb80 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.149+0000 7f17f361f740 1 --2- 172.21.15.43:0/2862727382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a02473f2f0 0x55a024b1e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.149+0000 7f17cdffb640 0 client.18972 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.149+0000 7f17f361f740 1 -- 172.21.15.43:0/2862727382 >> 172.21.15.43:0/2862727382 conn(0x55a02475c300 msgr2=0x55a024b179c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.149+0000 7f17f361f740 1 -- 172.21.15.43:0/2862727382 shutdown_connections 2024-04-25T10:44:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.149+0000 7f17f361f740 1 -- 172.21.15.43:0/2862727382 wait complete. 2024-04-25T10:44:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.149+0000 7f17f361f740 20 client.18972 trim_cache size 0 max 16384 2024-04-25T10:44:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-25T10:44:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-25T10:44:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-25T10:44:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-25T10:44:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.150+0000 7f17ec69a640 1 -- 172.21.15.43:0/4088182279 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f17d00afcd0 msgr2=0x7f17d00b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.150+0000 7f17ec69a640 1 --2- 172.21.15.43:0/4088182279 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f17d00afcd0 0x7f17d00b21e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f17dc01a000 tx=0x7f17dc018000 comp rx=0 tx=0).stop 2024-04-25T10:44:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.150+0000 7f17ec69a640 1 -- 172.21.15.43:0/4088182279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17e41042a0 msgr2=0x7f17e419a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.150+0000 7f17ec69a640 1 --2- 172.21.15.43:0/4088182279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17e41042a0 0x7f17e419a7e0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f17e000e9c0 tx=0x7f17e000ee80 comp rx=0 tx=0).stop 2024-04-25T10:44:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.150+0000 7f17ec69a640 1 -- 172.21.15.43:0/4088182279 shutdown_connections 2024-04-25T10:44:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.150+0000 7f17ec69a640 1 --2- 172.21.15.43:0/4088182279 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f17d00afcd0 0x7f17d00b21e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.150+0000 7f17ec69a640 1 --2- 172.21.15.43:0/4088182279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17e4109ff0 0x7f17e419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.150+0000 7f17ec69a640 1 --2- 172.21.15.43:0/4088182279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17e4109b50 0x7f17e419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.150+0000 7f17ec69a640 1 --2- 172.21.15.43:0/4088182279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17e41042a0 0x7f17e419a7e0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.150+0000 7f17ec69a640 1 -- 172.21.15.43:0/4088182279 >> 172.21.15.43:0/4088182279 conn(0x7f17e40ff230 msgr2=0x7f17e410f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.150+0000 7f17ec69a640 1 -- 172.21.15.43:0/4088182279 shutdown_connections 2024-04-25T10:44:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-25T10:44:08.150+0000 7f17ec69a640 1 -- 172.21.15.43:0/4088182279 wait complete. 2024-04-25T10:44:08.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:08 smithi043 ceph-mon[29478]: pgmap v362: 97 pgs: 97 active+clean; 2.8 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 108 KiB/s wr, 24 op/s 2024-04-25T10:44:08.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:08 smithi077 ceph-mon[32772]: pgmap v362: 97 pgs: 97 active+clean; 2.8 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 108 KiB/s wr, 24 op/s 2024-04-25T10:44:08.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:08 smithi164 ceph-mon[32713]: pgmap v362: 97 pgs: 97 active+clean; 2.8 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 108 KiB/s wr, 24 op/s 2024-04-25T10:44:10.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:10 smithi043 ceph-mon[29478]: pgmap v363: 97 pgs: 97 active+clean; 3.0 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 108 KiB/s wr, 25 op/s 2024-04-25T10:44:10.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:10 smithi077 ceph-mon[32772]: pgmap v363: 97 pgs: 97 active+clean; 3.0 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 108 KiB/s wr, 25 op/s 2024-04-25T10:44:10.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:10 smithi164 ceph-mon[32713]: pgmap v363: 97 pgs: 97 active+clean; 3.0 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 108 KiB/s wr, 25 op/s 2024-04-25T10:44:12.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:12 smithi077 ceph-mon[32772]: pgmap v364: 97 pgs: 97 active+clean; 3.0 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 108 KiB/s wr, 25 op/s 2024-04-25T10:44:12.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:12 smithi164 ceph-mon[32713]: pgmap v364: 97 pgs: 97 active+clean; 3.0 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 108 KiB/s wr, 25 op/s 2024-04-25T10:44:12.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:12 smithi043 ceph-mon[29478]: pgmap v364: 97 pgs: 97 active+clean; 3.0 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 108 KiB/s wr, 25 op/s 2024-04-25T10:44:14.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:13 smithi077 ceph-mon[32772]: pgmap v365: 97 pgs: 97 active+clean; 3.0 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 108 KiB/s wr, 25 op/s 2024-04-25T10:44:14.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:13 smithi077 ceph-mon[32772]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T10:44:14.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:13 smithi164 ceph-mon[32713]: pgmap v365: 97 pgs: 97 active+clean; 3.0 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 108 KiB/s wr, 25 op/s 2024-04-25T10:44:14.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:13 smithi164 ceph-mon[32713]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T10:44:14.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:13 smithi043 ceph-mon[29478]: pgmap v365: 97 pgs: 97 active+clean; 3.0 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 108 KiB/s wr, 25 op/s 2024-04-25T10:44:14.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:13 smithi043 ceph-mon[29478]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T10:44:14.965 INFO:tasks.workunit.client.0.smithi043.stdout:561/600: open fd = 3 2024-04-25T10:44:14.965 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.965 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.965 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.965 INFO:tasks.workunit.client.0.smithi043.stdout:562/600: open fd = 3 2024-04-25T10:44:14.965 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.965 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.965 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.965 INFO:tasks.workunit.client.0.smithi043.stdout:563/600: open fd = 3 2024-04-25T10:44:14.965 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.966 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.966 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.966 INFO:tasks.workunit.client.0.smithi043.stdout:564/600: open fd = 3 2024-04-25T10:44:14.966 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.966 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.966 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.966 INFO:tasks.workunit.client.0.smithi043.stdout:565/600: open fd = 3 2024-04-25T10:44:14.966 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.966 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.966 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.966 INFO:tasks.workunit.client.0.smithi043.stdout:566/600: open fd = 3 2024-04-25T10:44:14.966 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.966 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.967 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.967 INFO:tasks.workunit.client.0.smithi043.stdout:567/600: open fd = 3 2024-04-25T10:44:14.967 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.967 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.967 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.967 INFO:tasks.workunit.client.0.smithi043.stdout:568/600: open fd = 3 2024-04-25T10:44:14.967 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.967 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.967 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.967 INFO:tasks.workunit.client.0.smithi043.stdout:569/600: open fd = 3 2024-04-25T10:44:14.967 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.967 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.967 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.967 INFO:tasks.workunit.client.0.smithi043.stdout:570/600: open fd = 3 2024-04-25T10:44:14.967 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.968 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.968 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.968 INFO:tasks.workunit.client.0.smithi043.stdout:571/600: open fd = 3 2024-04-25T10:44:14.968 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.968 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.968 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.968 INFO:tasks.workunit.client.0.smithi043.stdout:572/600: open fd = 3 2024-04-25T10:44:14.968 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.968 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.968 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.968 INFO:tasks.workunit.client.0.smithi043.stdout:573/600: open fd = 3 2024-04-25T10:44:14.968 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.968 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.968 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.969 INFO:tasks.workunit.client.0.smithi043.stdout:574/600: open fd = 3 2024-04-25T10:44:14.969 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.969 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.969 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.969 INFO:tasks.workunit.client.0.smithi043.stdout:575/600: open fd = 3 2024-04-25T10:44:14.969 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.969 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.969 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.969 INFO:tasks.workunit.client.0.smithi043.stdout:576/600: open fd = 3 2024-04-25T10:44:14.969 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.969 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.969 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.969 INFO:tasks.workunit.client.0.smithi043.stdout:577/600: open fd = 3 2024-04-25T10:44:14.970 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.970 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.970 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.970 INFO:tasks.workunit.client.0.smithi043.stdout:578/600: open fd = 3 2024-04-25T10:44:14.970 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.970 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.970 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.970 INFO:tasks.workunit.client.0.smithi043.stdout:579/600: open fd = 3 2024-04-25T10:44:14.970 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.970 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.970 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.970 INFO:tasks.workunit.client.0.smithi043.stdout:580/600: open fd = 3 2024-04-25T10:44:14.970 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.971 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.971 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.971 INFO:tasks.workunit.client.0.smithi043.stdout:581/600: open fd = 3 2024-04-25T10:44:14.971 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.971 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.971 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.971 INFO:tasks.workunit.client.0.smithi043.stdout:582/600: open fd = 3 2024-04-25T10:44:14.971 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.971 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.971 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.971 INFO:tasks.workunit.client.0.smithi043.stdout:583/600: open fd = 3 2024-04-25T10:44:14.971 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.971 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.971 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.972 INFO:tasks.workunit.client.0.smithi043.stdout:584/600: open fd = 3 2024-04-25T10:44:14.972 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.972 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.972 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.972 INFO:tasks.workunit.client.0.smithi043.stdout:585/600: open fd = 3 2024-04-25T10:44:14.972 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.972 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.972 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.972 INFO:tasks.workunit.client.0.smithi043.stdout:586/600: open fd = 3 2024-04-25T10:44:14.972 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.972 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.972 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.972 INFO:tasks.workunit.client.0.smithi043.stdout:587/600: open fd = 3 2024-04-25T10:44:14.973 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.973 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.973 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.973 INFO:tasks.workunit.client.0.smithi043.stdout:588/600: open fd = 3 2024-04-25T10:44:14.973 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.973 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.973 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.973 INFO:tasks.workunit.client.0.smithi043.stdout:589/600: open fd = 3 2024-04-25T10:44:14.973 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.973 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.973 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.973 INFO:tasks.workunit.client.0.smithi043.stdout:590/600: open fd = 3 2024-04-25T10:44:14.973 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.973 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.973 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.974 INFO:tasks.workunit.client.0.smithi043.stdout:591/600: open fd = 3 2024-04-25T10:44:14.974 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.974 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.974 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.974 INFO:tasks.workunit.client.0.smithi043.stdout:592/600: open fd = 3 2024-04-25T10:44:14.974 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.974 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.974 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.974 INFO:tasks.workunit.client.0.smithi043.stdout:593/600: open fd = 3 2024-04-25T10:44:14.974 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.974 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.974 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.974 INFO:tasks.workunit.client.0.smithi043.stdout:594/600: open fd = 3 2024-04-25T10:44:14.975 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.975 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.975 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.975 INFO:tasks.workunit.client.0.smithi043.stdout:595/600: open fd = 3 2024-04-25T10:44:14.975 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.975 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.975 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.975 INFO:tasks.workunit.client.0.smithi043.stdout:596/600: open fd = 3 2024-04-25T10:44:14.975 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.975 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.975 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.975 INFO:tasks.workunit.client.0.smithi043.stdout:597/600: open fd = 3 2024-04-25T10:44:14.975 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.975 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.975 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.976 INFO:tasks.workunit.client.0.smithi043.stdout:598/600: open fd = 3 2024-04-25T10:44:14.976 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.976 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.976 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.976 INFO:tasks.workunit.client.0.smithi043.stdout:599/600: open fd = 3 2024-04-25T10:44:14.976 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.976 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.976 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.976 INFO:tasks.workunit.client.0.smithi043.stdout:600/600: open fd = 3 2024-04-25T10:44:14.976 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.976 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-25T10:44:14.976 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-25T10:44:14.977 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo OK 2024-04-25T10:44:14.977 INFO:tasks.workunit.client.0.smithi043.stdout:OK 2024-04-25T10:44:14.978 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T10:44:14.978 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T10:44:15.020 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-04-25T10:44:15.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T10:44:15.359 DEBUG:teuthology.parallel:result is None 2024-04-25T10:44:15.360 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T10:44:15.394 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T10:44:15.394 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T10:44:15.407 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T10:44:15.442 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T10:44:15.474 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T10:44:15.474 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T10:44:15.475 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T10:44:15.475 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T10:44:15.508 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T10:44:15.574 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T10:44:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.725+0000 7f6dc813f640 1 -- 172.21.15.43:0/3462225151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dc0108a60 msgr2=0x7f6dc0102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.725+0000 7f6dc813f640 1 --2- 172.21.15.43:0/3462225151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dc0108a60 0x7f6dc0102610 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009900 tx=0x7f6dbc02a410 comp rx=0 tx=0).stop 2024-04-25T10:44:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.725+0000 7f6dc813f640 1 -- 172.21.15.43:0/3462225151 shutdown_connections 2024-04-25T10:44:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.725+0000 7f6dc813f640 1 --2- 172.21.15.43:0/3462225151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6dc0108f00 0x7f6dc0102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.725+0000 7f6dc813f640 1 --2- 172.21.15.43:0/3462225151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dc01031c0 0x7f6dc010f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.725+0000 7f6dc813f640 1 --2- 172.21.15.43:0/3462225151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dc0108a60 0x7f6dc0102610 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.725+0000 7f6dc813f640 1 -- 172.21.15.43:0/3462225151 >> 172.21.15.43:0/3462225151 conn(0x7f6dc00781b0 msgr2=0x7f6dc00785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.725+0000 7f6dc813f640 1 -- 172.21.15.43:0/3462225151 shutdown_connections 2024-04-25T10:44:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.725+0000 7f6dc813f640 1 -- 172.21.15.43:0/3462225151 wait complete. 2024-04-25T10:44:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.726+0000 7f6dc813f640 1 Processor -- start 2024-04-25T10:44:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.726+0000 7f6dc813f640 1 -- start start 2024-04-25T10:44:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.726+0000 7f6dc813f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dc01031c0 0x7f6dc019b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.726+0000 7f6dc813f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6dc0108a60 0x7f6dc019bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.726+0000 7f6dc813f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dc0108f00 0x7f6dc019c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.726+0000 7f6dc813f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc0112c60 con 0x7f6dc01031c0 2024-04-25T10:44:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.726+0000 7f6dc813f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc0112ae0 con 0x7f6dc0108a60 2024-04-25T10:44:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.726+0000 7f6dc813f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc0112de0 con 0x7f6dc0108f00 2024-04-25T10:44:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.726+0000 7f6dc5edb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dc01031c0 0x7f6dc019b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.726+0000 7f6dc5edb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dc01031c0 0x7f6dc019b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48848/0 (socket says 172.21.15.43:48848) 2024-04-25T10:44:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.726+0000 7f6dc5edb640 1 -- 172.21.15.43:0/428359928 learned_addr learned my addr 172.21.15.43:0/428359928 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.726+0000 7f6dc5edb640 1 -- 172.21.15.43:0/428359928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dc0108f00 msgr2=0x7f6dc019c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.726+0000 7f6dc56da640 1 --2- 172.21.15.43:0/428359928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6dc0108a60 0x7f6dc019bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.726+0000 7f6dc66dc640 1 --2- 172.21.15.43:0/428359928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dc0108f00 0x7f6dc019c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.726+0000 7f6dc5edb640 1 --2- 172.21.15.43:0/428359928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dc0108f00 0x7f6dc019c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.727+0000 7f6dc5edb640 1 -- 172.21.15.43:0/428359928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6dc0108a60 msgr2=0x7f6dc019bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.727+0000 7f6dc5edb640 1 --2- 172.21.15.43:0/428359928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6dc0108a60 0x7f6dc019bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.727+0000 7f6dc5edb640 1 -- 172.21.15.43:0/428359928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc01a68f0 con 0x7f6dc01031c0 2024-04-25T10:44:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.727+0000 7f6dc56da640 1 --2- 172.21.15.43:0/428359928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6dc0108a60 0x7f6dc019bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.727+0000 7f6dc5edb640 1 --2- 172.21.15.43:0/428359928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dc01031c0 0x7f6dc019b5e0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc02a930 tx=0x7f6dbc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.727+0000 7f6db6ffd640 1 -- 172.21.15.43:0/428359928 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc03f030 con 0x7f6dc01031c0 2024-04-25T10:44:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.727+0000 7f6db6ffd640 1 -- 172.21.15.43:0/428359928 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dbc003aa0 con 0x7f6dc01031c0 2024-04-25T10:44:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.727+0000 7f6dc813f640 1 -- 172.21.15.43:0/428359928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc01a6b80 con 0x7f6dc01031c0 2024-04-25T10:44:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.727+0000 7f6db6ffd640 1 -- 172.21.15.43:0/428359928 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc035070 con 0x7f6dc01031c0 2024-04-25T10:44:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.727+0000 7f6dc813f640 1 -- 172.21.15.43:0/428359928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc01a7070 con 0x7f6dc01031c0 2024-04-25T10:44:15.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.728+0000 7f6dc813f640 1 -- 172.21.15.43:0/428359928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dc006a110 con 0x7f6dc01031c0 2024-04-25T10:44:15.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.730+0000 7f6db6ffd640 1 -- 172.21.15.43:0/428359928 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f6dbc042030 con 0x7f6dc01031c0 2024-04-25T10:44:15.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.730+0000 7f6db6ffd640 1 --2- 172.21.15.43:0/428359928 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6dac0afc50 0x7f6dac0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:15.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.730+0000 7f6db6ffd640 1 -- 172.21.15.43:0/428359928 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f6dbc0ffc20 con 0x7f6dc01031c0 2024-04-25T10:44:15.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.730+0000 7f6dc56da640 1 --2- 172.21.15.43:0/428359928 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6dac0afc50 0x7f6dac0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:15.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.731+0000 7f6dc56da640 1 --2- 172.21.15.43:0/428359928 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6dac0afc50 0x7f6dac0b2160 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6dc010e820 tx=0x7f6db0005cb0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:15.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.734+0000 7f6db6ffd640 1 -- 172.21.15.43:0/428359928 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6dbc0c84c0 con 0x7f6dc01031c0 2024-04-25T10:44:15.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.945+0000 7f6dc813f640 1 -- 172.21.15.43:0/428359928 --> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f6dc0062fe0 con 0x7f6dac0afc50 2024-04-25T10:44:15.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.953+0000 7f6db6ffd640 1 -- 172.21.15.43:0/428359928 <== mgr.15291 v2:172.21.15.43:6800/3658924116 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f6dc0062fe0 con 0x7f6dac0afc50 2024-04-25T10:44:15.955 INFO:teuthology.orchestra.run.smithi043.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T10:32:40", "first": "2024-04-25T10:33:00", "last": "2024-04-25T10:44:00", "last_pruned": "2024-04-25T10:44:00", "created_count": 12, "pruned_count": 6, "active": true} 2024-04-25T10:44:15.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.956+0000 7f6dc813f640 1 -- 172.21.15.43:0/428359928 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6dac0afc50 msgr2=0x7f6dac0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:15.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.956+0000 7f6dc813f640 1 --2- 172.21.15.43:0/428359928 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6dac0afc50 0x7f6dac0b2160 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6dc010e820 tx=0x7f6db0005cb0 comp rx=0 tx=0).stop 2024-04-25T10:44:15.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.956+0000 7f6dc813f640 1 -- 172.21.15.43:0/428359928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dc01031c0 msgr2=0x7f6dc019b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:15.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.956+0000 7f6dc813f640 1 --2- 172.21.15.43:0/428359928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dc01031c0 0x7f6dc019b5e0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc02a930 tx=0x7f6dbc004270 comp rx=0 tx=0).stop 2024-04-25T10:44:15.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.956+0000 7f6dc813f640 1 -- 172.21.15.43:0/428359928 shutdown_connections 2024-04-25T10:44:15.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.956+0000 7f6dc813f640 1 --2- 172.21.15.43:0/428359928 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f6dac0afc50 0x7f6dac0b2160 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:15.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.956+0000 7f6dc813f640 1 --2- 172.21.15.43:0/428359928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6dc0108a60 0x7f6dc019bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.956+0000 7f6dc813f640 1 --2- 172.21.15.43:0/428359928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dc0108f00 0x7f6dc019c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.956+0000 7f6dc813f640 1 --2- 172.21.15.43:0/428359928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dc01031c0 0x7f6dc019b5e0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.956+0000 7f6dc813f640 1 -- 172.21.15.43:0/428359928 >> 172.21.15.43:0/428359928 conn(0x7f6dc00781b0 msgr2=0x7f6dc0104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.956+0000 7f6dc813f640 1 -- 172.21.15.43:0/428359928 shutdown_connections 2024-04-25T10:44:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:15.956+0000 7f6dc813f640 1 -- 172.21.15.43:0/428359928 wait complete. 2024-04-25T10:44:15.969 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T10:44:16.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.081+0000 7f5fe60bd640 1 -- 172.21.15.43:0/3269998313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe0104cc0 msgr2=0x7f5fe01050d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:16.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.081+0000 7f5fe60bd640 1 --2- 172.21.15.43:0/3269998313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe0104cc0 0x7f5fe01050d0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009900 tx=0x7f5fd002a510 comp rx=0 tx=0).stop 2024-04-25T10:44:16.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.081+0000 7f5fe60bd640 1 -- 172.21.15.43:0/3269998313 shutdown_connections 2024-04-25T10:44:16.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.081+0000 7f5fe60bd640 1 --2- 172.21.15.43:0/3269998313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fe010e0d0 0x7f5fe01105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:16.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.081+0000 7f5fe60bd640 1 --2- 172.21.15.43:0/3269998313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fe010b720 0x7f5fe010db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:16.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.081+0000 7f5fe60bd640 1 --2- 172.21.15.43:0/3269998313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe0104cc0 0x7f5fe01050d0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:16.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.081+0000 7f5fe60bd640 1 -- 172.21.15.43:0/3269998313 >> 172.21.15.43:0/3269998313 conn(0x7f5fe00fe260 msgr2=0x7f5fe0100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:16.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.081+0000 7f5fe60bd640 1 -- 172.21.15.43:0/3269998313 shutdown_connections 2024-04-25T10:44:16.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.081+0000 7f5fe60bd640 1 -- 172.21.15.43:0/3269998313 wait complete. 2024-04-25T10:44:16.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.082+0000 7f5fe60bd640 1 Processor -- start 2024-04-25T10:44:16.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.082+0000 7f5fe60bd640 1 -- start start 2024-04-25T10:44:16.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fe60bd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fe0104cc0 0x7f5fe01a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:16.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fe60bd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fe010b720 0x7f5fe01a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:16.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fe60bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe010e0d0 0x7f5fe019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:16.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fe60bd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe0113d30 con 0x7f5fe010e0d0 2024-04-25T10:44:16.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fe60bd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe0113bb0 con 0x7f5fe010b720 2024-04-25T10:44:16.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fe60bd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe0113eb0 con 0x7f5fe0104cc0 2024-04-25T10:44:16.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fdffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe010e0d0 0x7f5fe019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:16.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fdffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe010e0d0 0x7f5fe019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48852/0 (socket says 172.21.15.43:48852) 2024-04-25T10:44:16.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fdffff640 1 -- 172.21.15.43:0/1154273069 learned_addr learned my addr 172.21.15.43:0/1154273069 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:16.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fdf7fe640 1 --2- 172.21.15.43:0/1154273069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fe0104cc0 0x7f5fe01a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:16.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fdeffd640 1 --2- 172.21.15.43:0/1154273069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fe010b720 0x7f5fe01a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:16.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fdffff640 1 -- 172.21.15.43:0/1154273069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fe0104cc0 msgr2=0x7f5fe01a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:16.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fdffff640 1 --2- 172.21.15.43:0/1154273069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fe0104cc0 0x7f5fe01a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:16.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fdffff640 1 -- 172.21.15.43:0/1154273069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fe010b720 msgr2=0x7f5fe01a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:16.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fdffff640 1 --2- 172.21.15.43:0/1154273069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fe010b720 0x7f5fe01a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:16.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fdffff640 1 -- 172.21.15.43:0/1154273069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe019ae20 con 0x7f5fe010e0d0 2024-04-25T10:44:16.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fdf7fe640 1 --2- 172.21.15.43:0/1154273069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fe0104cc0 0x7f5fe01a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:16.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fdffff640 1 --2- 172.21.15.43:0/1154273069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe010e0d0 0x7f5fe019a6f0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800eaa0 tx=0x7f5fd800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:16.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fdcff9640 1 -- 172.21.15.43:0/1154273069 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd800cdb0 con 0x7f5fe010e0d0 2024-04-25T10:44:16.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fdcff9640 1 -- 172.21.15.43:0/1154273069 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fd8005020 con 0x7f5fe010e0d0 2024-04-25T10:44:16.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fe60bd640 1 -- 172.21.15.43:0/1154273069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe019b110 con 0x7f5fe010e0d0 2024-04-25T10:44:16.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fdcff9640 1 -- 172.21.15.43:0/1154273069 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd80116d0 con 0x7f5fe010e0d0 2024-04-25T10:44:16.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.083+0000 7f5fe60bd640 1 -- 172.21.15.43:0/1154273069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe01a7310 con 0x7f5fe010e0d0 2024-04-25T10:44:16.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.084+0000 7f5fe60bd640 1 -- 172.21.15.43:0/1154273069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe0050740 con 0x7f5fe010e0d0 2024-04-25T10:44:16.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.087+0000 7f5fdcff9640 1 -- 172.21.15.43:0/1154273069 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f5fd80051d0 con 0x7f5fe010e0d0 2024-04-25T10:44:16.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.087+0000 7f5fdcff9640 1 --2- 172.21.15.43:0/1154273069 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5fc80afc50 0x7f5fc80b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:16.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.088+0000 7f5fdf7fe640 1 --2- 172.21.15.43:0/1154273069 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5fc80afc50 0x7f5fc80b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:16.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.088+0000 7f5fdcff9640 1 -- 172.21.15.43:0/1154273069 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f5fd80df5e0 con 0x7f5fe010e0d0 2024-04-25T10:44:16.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.088+0000 7f5fdf7fe640 1 --2- 172.21.15.43:0/1154273069 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5fc80afc50 0x7f5fc80b2160 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0000c00 tx=0x7f5fd0009680 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:16.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.091+0000 7f5fdcff9640 1 -- 172.21.15.43:0/1154273069 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5fd8016030 con 0x7f5fe010e0d0 2024-04-25T10:44:16.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:16 smithi077 ceph-mon[32772]: pgmap v367: 97 pgs: 97 active+clean; 3.2 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-25T10:44:16.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:16 smithi164 ceph-mon[32713]: pgmap v367: 97 pgs: 97 active+clean; 3.2 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-25T10:44:16.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.301+0000 7f5fe60bd640 1 -- 172.21.15.43:0/1154273069 --> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5fe010ecc0 con 0x7f5fc80afc50 2024-04-25T10:44:16.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.309+0000 7f5fdcff9640 1 -- 172.21.15.43:0/1154273069 <== mgr.15291 v2:172.21.15.43:6800/3658924116 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5fe010ecc0 con 0x7f5fc80afc50 2024-04-25T10:44:16.313 INFO:teuthology.orchestra.run.smithi043.stdout:/ 1m 6m3h 2024-04-25T10:44:16.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.311+0000 7f5fe60bd640 1 -- 172.21.15.43:0/1154273069 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5fc80afc50 msgr2=0x7f5fc80b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:16.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.311+0000 7f5fe60bd640 1 --2- 172.21.15.43:0/1154273069 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5fc80afc50 0x7f5fc80b2160 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0000c00 tx=0x7f5fd0009680 comp rx=0 tx=0).stop 2024-04-25T10:44:16.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.311+0000 7f5fe60bd640 1 -- 172.21.15.43:0/1154273069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe010e0d0 msgr2=0x7f5fe019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:16.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.311+0000 7f5fe60bd640 1 --2- 172.21.15.43:0/1154273069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe010e0d0 0x7f5fe019a6f0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800eaa0 tx=0x7f5fd800ef60 comp rx=0 tx=0).stop 2024-04-25T10:44:16.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.311+0000 7f5fe60bd640 1 -- 172.21.15.43:0/1154273069 shutdown_connections 2024-04-25T10:44:16.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.311+0000 7f5fe60bd640 1 --2- 172.21.15.43:0/1154273069 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5fc80afc50 0x7f5fc80b2160 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:16.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.311+0000 7f5fe60bd640 1 --2- 172.21.15.43:0/1154273069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fe010b720 0x7f5fe01a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:16.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.311+0000 7f5fe60bd640 1 --2- 172.21.15.43:0/1154273069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe010e0d0 0x7f5fe019a6f0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:16.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.311+0000 7f5fe60bd640 1 --2- 172.21.15.43:0/1154273069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fe0104cc0 0x7f5fe01a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:16.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.311+0000 7f5fe60bd640 1 -- 172.21.15.43:0/1154273069 >> 172.21.15.43:0/1154273069 conn(0x7f5fe00fe260 msgr2=0x7f5fe010e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:16.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.311+0000 7f5fe60bd640 1 -- 172.21.15.43:0/1154273069 shutdown_connections 2024-04-25T10:44:16.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:16.311+0000 7f5fe60bd640 1 -- 172.21.15.43:0/1154273069 wait complete. 2024-04-25T10:44:16.325 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:16 smithi043 ceph-mon[29478]: pgmap v367: 97 pgs: 97 active+clean; 3.2 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-25T10:44:16.326 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-25T10:44:16.340 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-25T10:44:17.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:17 smithi077 ceph-mon[32772]: from='client.18978 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:44:17.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:17 smithi164 ceph-mon[32713]: from='client.18978 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:44:17.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:17 smithi043 ceph-mon[29478]: from='client.18978 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:44:18.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:18 smithi077 ceph-mon[32772]: from='client.18984 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:44:18.291 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:18 smithi077 ceph-mon[32772]: pgmap v368: 97 pgs: 97 active+clean; 3.2 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-25T10:44:18.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:18 smithi164 ceph-mon[32713]: from='client.18984 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:44:18.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:18 smithi164 ceph-mon[32713]: pgmap v368: 97 pgs: 97 active+clean; 3.2 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-25T10:44:18.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:18 smithi043 ceph-mon[29478]: from='client.18984 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T10:44:18.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:18 smithi043 ceph-mon[29478]: pgmap v368: 97 pgs: 97 active+clean; 3.2 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-04-25T10:44:20.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:20 smithi077 ceph-mon[32772]: pgmap v369: 97 pgs: 97 active+clean; 3.3 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-04-25T10:44:20.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:20 smithi164 ceph-mon[32713]: pgmap v369: 97 pgs: 97 active+clean; 3.3 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-04-25T10:44:20.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:20 smithi043 ceph-mon[29478]: pgmap v369: 97 pgs: 97 active+clean; 3.3 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-04-25T10:44:22.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:22 smithi077 ceph-mon[32772]: pgmap v370: 97 pgs: 97 active+clean; 3.3 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-04-25T10:44:22.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:22 smithi164 ceph-mon[32713]: pgmap v370: 97 pgs: 97 active+clean; 3.3 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-04-25T10:44:22.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:22 smithi043 ceph-mon[29478]: pgmap v370: 97 pgs: 97 active+clean; 3.3 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-04-25T10:44:24.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:24 smithi077 ceph-mon[32772]: pgmap v371: 97 pgs: 97 active+clean; 3.3 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-04-25T10:44:24.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:24 smithi164 ceph-mon[32713]: pgmap v371: 97 pgs: 97 active+clean; 3.3 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-04-25T10:44:24.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:24 smithi043 ceph-mon[29478]: pgmap v371: 97 pgs: 97 active+clean; 3.3 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-04-25T10:44:26.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:26 smithi077 ceph-mon[32772]: pgmap v372: 97 pgs: 97 active+clean; 3.4 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-25T10:44:26.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:26 smithi164 ceph-mon[32713]: pgmap v372: 97 pgs: 97 active+clean; 3.4 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-25T10:44:26.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:26 smithi043 ceph-mon[29478]: pgmap v372: 97 pgs: 97 active+clean; 3.4 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-25T10:44:28.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:28 smithi077 ceph-mon[32772]: pgmap v373: 97 pgs: 97 active+clean; 3.4 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-04-25T10:44:28.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:28 smithi164 ceph-mon[32713]: pgmap v373: 97 pgs: 97 active+clean; 3.4 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-04-25T10:44:28.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:28 smithi043 ceph-mon[29478]: pgmap v373: 97 pgs: 97 active+clean; 3.4 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-04-25T10:44:30.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:30 smithi077 ceph-mon[32772]: pgmap v374: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-25T10:44:30.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:30 smithi164 ceph-mon[32713]: pgmap v374: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-25T10:44:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:30 smithi043 ceph-mon[29478]: pgmap v374: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-25T10:44:32.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:32 smithi077 ceph-mon[32772]: pgmap v375: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-04-25T10:44:32.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:32 smithi164 ceph-mon[32713]: pgmap v375: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-04-25T10:44:32.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T10:44:32.338 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T10:44:32.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:32 smithi043 ceph-mon[29478]: pgmap v375: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-04-25T10:44:32.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.467+0000 7f5a2c2e2640 1 -- 172.21.15.43:0/1140923138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a2410e570 msgr2=0x7f5a24110990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:32.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.467+0000 7f5a2c2e2640 1 --2- 172.21.15.43:0/1140923138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a2410e570 0x7f5a24110990 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009ca0 tx=0x7f5a1c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.467+0000 7f5a2c2e2640 1 -- 172.21.15.43:0/1140923138 shutdown_connections 2024-04-25T10:44:32.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.467+0000 7f5a2c2e2640 1 --2- 172.21.15.43:0/1140923138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a24102070 0x7f5a24102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.467+0000 7f5a2c2e2640 1 --2- 172.21.15.43:0/1140923138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a2410e570 0x7f5a24110990 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.467+0000 7f5a2c2e2640 1 --2- 172.21.15.43:0/1140923138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a24109bc0 0x7f5a24101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.467+0000 7f5a2c2e2640 1 -- 172.21.15.43:0/1140923138 >> 172.21.15.43:0/1140923138 conn(0x7f5a24077920 msgr2=0x7f5a24077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:32.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.467+0000 7f5a2c2e2640 1 -- 172.21.15.43:0/1140923138 shutdown_connections 2024-04-25T10:44:32.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.467+0000 7f5a2c2e2640 1 -- 172.21.15.43:0/1140923138 wait complete. 2024-04-25T10:44:32.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.468+0000 7f5a2c2e2640 1 Processor -- start 2024-04-25T10:44:32.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.468+0000 7f5a2c2e2640 1 -- start start 2024-04-25T10:44:32.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.468+0000 7f5a2c2e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a24102070 0x7f5a2419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.468+0000 7f5a2c2e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a24109bc0 0x7f5a2419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.468+0000 7f5a2c2e2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a2410e570 0x7f5a2419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.468+0000 7f5a2c2e2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5a24113d30 con 0x7f5a24102070 2024-04-25T10:44:32.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.468+0000 7f5a2c2e2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5a24113bb0 con 0x7f5a24109bc0 2024-04-25T10:44:32.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.468+0000 7f5a2c2e2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5a24113eb0 con 0x7f5a2410e570 2024-04-25T10:44:32.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.468+0000 7f5a2a07e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a24102070 0x7f5a2419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.469+0000 7f5a2a07e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a24102070 0x7f5a2419a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43108/0 (socket says 172.21.15.43:43108) 2024-04-25T10:44:32.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.469+0000 7f5a2a07e640 1 -- 172.21.15.43:0/1095714341 learned_addr learned my addr 172.21.15.43:0/1095714341 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:32.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.469+0000 7f5a2a87f640 1 --2- 172.21.15.43:0/1095714341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a2410e570 0x7f5a2419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.469+0000 7f5a2987d640 1 --2- 172.21.15.43:0/1095714341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a24109bc0 0x7f5a2419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.469+0000 7f5a2a07e640 1 -- 172.21.15.43:0/1095714341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a2410e570 msgr2=0x7f5a2419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:32.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.469+0000 7f5a2a07e640 1 --2- 172.21.15.43:0/1095714341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a2410e570 0x7f5a2419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.469+0000 7f5a2a07e640 1 -- 172.21.15.43:0/1095714341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a24109bc0 msgr2=0x7f5a2419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:32.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.469+0000 7f5a2a07e640 1 --2- 172.21.15.43:0/1095714341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a24109bc0 0x7f5a2419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.469+0000 7f5a2a07e640 1 -- 172.21.15.43:0/1095714341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2419f610 con 0x7f5a24102070 2024-04-25T10:44:32.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.469+0000 7f5a2987d640 1 --2- 172.21.15.43:0/1095714341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a24109bc0 0x7f5a2419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:32.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.469+0000 7f5a2a07e640 1 --2- 172.21.15.43:0/1095714341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a24102070 0x7f5a2419a7b0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f5a20005d00 tx=0x7f5a2000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:32.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.469+0000 7f5a1b7fe640 1 -- 172.21.15.43:0/1095714341 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20017070 con 0x7f5a24102070 2024-04-25T10:44:32.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.469+0000 7f5a2c2e2640 1 -- 172.21.15.43:0/1095714341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a2419f900 con 0x7f5a24102070 2024-04-25T10:44:32.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.469+0000 7f5a1b7fe640 1 -- 172.21.15.43:0/1095714341 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a20009c30 con 0x7f5a24102070 2024-04-25T10:44:32.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.469+0000 7f5a1b7fe640 1 -- 172.21.15.43:0/1095714341 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2000f620 con 0x7f5a24102070 2024-04-25T10:44:32.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.469+0000 7f5a2c2e2640 1 -- 172.21.15.43:0/1095714341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a2419fe00 con 0x7f5a24102070 2024-04-25T10:44:32.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.472+0000 7f5a3318f740 1 Processor -- start 2024-04-25T10:44:32.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.472+0000 7f5a3318f740 1 -- start start 2024-04-25T10:44:32.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.473+0000 7f5a3318f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a220390 0x55b32a5c0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.473+0000 7f5a3318f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a1f74d0 0x55b32a5c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.473+0000 7f5a3318f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1ff6b0 0x55b32a5cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.473+0000 7f5a3318f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b32a526d80 con 0x55b32a1ff6b0 2024-04-25T10:44:32.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.473+0000 7f5a3318f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b32a4e08b0 con 0x55b32a220390 2024-04-25T10:44:32.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.473+0000 7f5a3318f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b32a522d40 con 0x55b32a1f74d0 2024-04-25T10:44:32.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.473+0000 7f5a1b7fe640 1 -- 172.21.15.43:0/1095714341 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f5a2001a070 con 0x7f5a24102070 2024-04-25T10:44:32.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.473+0000 7f5a2987d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1ff6b0 0x55b32a5cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.473+0000 7f5a2987d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1ff6b0 0x55b32a5cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43116/0 (socket says 172.21.15.43:43116) 2024-04-25T10:44:32.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.473+0000 7f5a2987d640 1 -- 172.21.15.43:0/751993046 learned_addr learned my addr 172.21.15.43:0/751993046 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:32.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.473+0000 7f5a2a87f640 1 --2- 172.21.15.43:0/751993046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a220390 0x55b32a5c0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.473+0000 7f5a2a07e640 1 --2- 172.21.15.43:0/751993046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a1f74d0 0x55b32a5c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.473+0000 7f5a2987d640 1 -- 172.21.15.43:0/751993046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a1f74d0 msgr2=0x55b32a5c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:32.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.473+0000 7f5a2987d640 1 --2- 172.21.15.43:0/751993046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a1f74d0 0x55b32a5c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.473+0000 7f5a2987d640 1 -- 172.21.15.43:0/751993046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a220390 msgr2=0x55b32a5c0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:32.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.473+0000 7f5a2987d640 1 --2- 172.21.15.43:0/751993046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a220390 0x55b32a5c0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.473+0000 7f5a2987d640 1 -- 172.21.15.43:0/751993046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b32a4b1a40 con 0x55b32a1ff6b0 2024-04-25T10:44:32.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.473+0000 7f5a2a07e640 1 --2- 172.21.15.43:0/751993046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a1f74d0 0x55b32a5c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:32.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.473+0000 7f5a2987d640 1 --2- 172.21.15.43:0/751993046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1ff6b0 0x55b32a5cd8f0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f5a14005fd0 tx=0x7f5a1400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dad7cc11afedd98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:32.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.474+0000 7f5a18bfe640 1 -- 172.21.15.43:0/751993046 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a14016070 con 0x55b32a1ff6b0 2024-04-25T10:44:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.474+0000 7f5a1b7fe640 1 --2- 172.21.15.43:0/1095714341 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5a100afc60 0x7f5a100b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.474+0000 7f5a18bfe640 1 -- 172.21.15.43:0/751993046 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a140054d0 con 0x55b32a1ff6b0 2024-04-25T10:44:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.474+0000 7f5a18bfe640 1 -- 172.21.15.43:0/751993046 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a14004e50 con 0x55b32a1ff6b0 2024-04-25T10:44:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.474+0000 7f5a2a87f640 1 --2- 172.21.15.43:0/1095714341 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5a100afc60 0x7f5a100b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.474+0000 7f5a1b7fe640 1 -- 172.21.15.43:0/1095714341 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f5a200dead0 con 0x7f5a24102070 2024-04-25T10:44:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.475+0000 7f5a3318f740 1 -- 172.21.15.43:0/751993046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1ff6b0 msgr2=0x55b32a5cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.475+0000 7f5a2a87f640 1 --2- 172.21.15.43:0/1095714341 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5a100afc60 0x7f5a100b2170 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c005e30 tx=0x7f5a1c005c30 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.475+0000 7f5a3318f740 1 --2- 172.21.15.43:0/751993046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1ff6b0 0x55b32a5cd8f0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f5a14005fd0 tx=0x7f5a1400e6c0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.475+0000 7f5a3318f740 1 -- 172.21.15.43:0/751993046 shutdown_connections 2024-04-25T10:44:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.475+0000 7f5a3318f740 1 --2- 172.21.15.43:0/751993046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1ff6b0 0x55b32a5cd8f0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.475+0000 7f5a3318f740 1 --2- 172.21.15.43:0/751993046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a1f74d0 0x55b32a5c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.475+0000 7f5a3318f740 1 --2- 172.21.15.43:0/751993046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a220390 0x55b32a5c0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.475+0000 7f5a3318f740 1 -- 172.21.15.43:0/751993046 >> 172.21.15.43:0/751993046 conn(0x55b32a2144e0 msgr2=0x55b32a5bd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.475+0000 7f5a3318f740 1 -- 172.21.15.43:0/751993046 shutdown_connections 2024-04-25T10:44:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.475+0000 7f5a3318f740 1 -- 172.21.15.43:0/751993046 wait complete. 2024-04-25T10:44:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.475+0000 7f5a3318f740 1 -- start start 2024-04-25T10:44:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.475+0000 7f5a3318f740 1 Processor -- start 2024-04-25T10:44:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.476+0000 7f5a3318f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1f74d0 0x55b32a5cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.476+0000 7f5a3318f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a1ff6b0 0x55b32a5cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.476+0000 7f5a3318f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a220390 0x55b32a5d7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.476+0000 7f5a3318f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b32a5cdfe0 con 0x55b32a1f74d0 2024-04-25T10:44:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.476+0000 7f5a3318f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b32a4add10 con 0x55b32a1ff6b0 2024-04-25T10:44:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.476+0000 7f5a3318f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b32a5ce160 con 0x55b32a220390 2024-04-25T10:44:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.476+0000 7f5a3318f740 10 client.0 _ll_register_callbacks cb 0x55b32a4da210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.476+0000 7f5a2a07e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1f74d0 0x55b32a5cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.476+0000 7f5a2a07e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1f74d0 0x55b32a5cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43122/0 (socket says 172.21.15.43:43122) 2024-04-25T10:44:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.476+0000 7f5a2a07e640 1 -- 172.21.15.43:0/2743673453 learned_addr learned my addr 172.21.15.43:0/2743673453 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.476+0000 7f5a2987d640 1 --2- 172.21.15.43:0/2743673453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a1ff6b0 0x55b32a5cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.476+0000 7f5a2a87f640 1 --2- 172.21.15.43:0/2743673453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a220390 0x55b32a5d7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.476+0000 7f5a2a07e640 1 -- 172.21.15.43:0/2743673453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a220390 msgr2=0x55b32a5d7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.476+0000 7f5a2a07e640 1 --2- 172.21.15.43:0/2743673453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a220390 0x55b32a5d7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.476+0000 7f5a2a07e640 1 -- 172.21.15.43:0/2743673453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a1ff6b0 msgr2=0x55b32a5cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.476+0000 7f5a2a07e640 1 --2- 172.21.15.43:0/2743673453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a1ff6b0 0x55b32a5cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.476+0000 7f5a2a07e640 1 -- 172.21.15.43:0/2743673453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b32a3027d0 con 0x55b32a1f74d0 2024-04-25T10:44:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.476+0000 7f5a2987d640 1 --2- 172.21.15.43:0/2743673453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a1ff6b0 0x55b32a5cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.476+0000 7f5a2a07e640 1 --2- 172.21.15.43:0/2743673453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1f74d0 0x55b32a5cb5b0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f5a200a6970 tx=0x7f5a200a6a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9748ec9f0c2c29af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.476+0000 7f5a0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:32.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.477+0000 7f5a0dffb640 1 -- 172.21.15.43:0/2743673453 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20021070 con 0x55b32a1f74d0 2024-04-25T10:44:32.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.477+0000 7f5a0dffb640 1 -- 172.21.15.43:0/2743673453 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f5a200af850 con 0x55b32a1f74d0 2024-04-25T10:44:32.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.477+0000 7f5a3318f740 1 -- 172.21.15.43:0/2743673453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a140054d0 con 0x55b32a1f74d0 2024-04-25T10:44:32.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.477+0000 7f5a0dffb640 1 -- 172.21.15.43:0/2743673453 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5a200af410 con 0x55b32a1f74d0 2024-04-25T10:44:32.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.477+0000 7f5a3318f740 10 client.18996 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:32.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.477+0000 7f5a3318f740 1 -- 172.21.15.43:0/2743673453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b32a2383f0 con 0x55b32a1f74d0 2024-04-25T10:44:32.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.477+0000 7f5a0dffb640 1 -- 172.21.15.43:0/2743673453 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a140054d0 con 0x55b32a1f74d0 2024-04-25T10:44:32.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.477+0000 7f5a0dffb640 1 -- 172.21.15.43:0/2743673453 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5a200a25e0 con 0x55b32a1f74d0 2024-04-25T10:44:32.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.477+0000 7f5a3318f740 10 client.18996 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:32.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.477+0000 7f5a3318f740 10 client.18996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:44:32.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.477+0000 7f5a3318f740 20 client.18996 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.477+0000 7f5a3318f740 1 --2- 172.21.15.43:0/2743673453 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b32a1def60 0x55b32a5ddaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.477+0000 7f5a3318f740 4 client.18996 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:44:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.477+0000 7f5a3318f740 1 -- 172.21.15.43:0/2743673453 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b32a534990 con 0x55b32a1def60 2024-04-25T10:44:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.478+0000 7f5a2987d640 1 --2- 172.21.15.43:0/2743673453 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b32a1def60 0x55b32a5ddaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.478+0000 7f5a2987d640 1 --2- 172.21.15.43:0/2743673453 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b32a1def60 0x55b32a5ddaa0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ac272659cc0551f server_cookie=f0205534f4f688c4 in_seq=0 out_seq=0 2024-04-25T10:44:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.478+0000 7f5a0dffb640 10 client.18996 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.479+0000 7f5a0dffb640 1 -- 172.21.15.43:0/2743673453 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b32a534990 con 0x55b32a1def60 2024-04-25T10:44:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.479+0000 7f5a0dffb640 10 client.18996 handle_command_reply: tid=0 2024-04-25T10:44:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.479+0000 7f5a3318f740 1 client.18996 shutdown 2024-04-25T10:44:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.479+0000 7f5a3318f740 1 -- 172.21.15.43:0/2743673453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1f74d0 msgr2=0x55b32a5cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.479+0000 7f5a3318f740 1 --2- 172.21.15.43:0/2743673453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1f74d0 0x55b32a5cb5b0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f5a200a6970 tx=0x7f5a200a6a40 comp rx=0 tx=0).stop 2024-04-25T10:44:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.479+0000 7f5a3318f740 1 -- 172.21.15.43:0/2743673453 shutdown_connections 2024-04-25T10:44:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.479+0000 7f5a3318f740 1 --2- 172.21.15.43:0/2743673453 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b32a1def60 0x55b32a5ddaa0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.479+0000 7f5a3318f740 1 --2- 172.21.15.43:0/2743673453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a1ff6b0 0x55b32a5cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.479+0000 7f5a3318f740 1 --2- 172.21.15.43:0/2743673453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a220390 0x55b32a5d7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.479+0000 7f5a0dffb640 0 client.18996 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:32.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.479+0000 7f5a3318f740 1 --2- 172.21.15.43:0/2743673453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1f74d0 0x55b32a5cb5b0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.479+0000 7f5a3318f740 1 -- 172.21.15.43:0/2743673453 >> 172.21.15.43:0/2743673453 conn(0x55b32a2144e0 msgr2=0x55b32a5c2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:32.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.479+0000 7f5a3318f740 1 -- 172.21.15.43:0/2743673453 shutdown_connections 2024-04-25T10:44:32.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.479+0000 7f5a3318f740 1 -- 172.21.15.43:0/2743673453 wait complete. 2024-04-25T10:44:32.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.479+0000 7f5a3318f740 20 client.18996 trim_cache size 0 max 16384 2024-04-25T10:44:32.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a3318f740 1 Processor -- start 2024-04-25T10:44:32.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a3318f740 1 -- start start 2024-04-25T10:44:32.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a3318f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a1def60 0x55b32a5d70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a3318f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1f74d0 0x55b32a5e32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a3318f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a1ff6b0 0x55b32a5e37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a3318f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5a241b23e0 con 0x55b32a1f74d0 2024-04-25T10:44:32.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a3318f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5a241afcb0 con 0x55b32a1def60 2024-04-25T10:44:32.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a3318f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5a241adb20 con 0x55b32a1ff6b0 2024-04-25T10:44:32.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a2987d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1f74d0 0x55b32a5e32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a2987d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1f74d0 0x55b32a5e32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43128/0 (socket says 172.21.15.43:43128) 2024-04-25T10:44:32.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a2987d640 1 -- 172.21.15.43:0/349582637 learned_addr learned my addr 172.21.15.43:0/349582637 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:32.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a2a07e640 1 --2- 172.21.15.43:0/349582637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a1def60 0x55b32a5d70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a2a87f640 1 --2- 172.21.15.43:0/349582637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a1ff6b0 0x55b32a5e37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a2987d640 1 -- 172.21.15.43:0/349582637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a1ff6b0 msgr2=0x55b32a5e37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:32.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a2987d640 1 --2- 172.21.15.43:0/349582637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a1ff6b0 0x55b32a5e37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a2987d640 1 -- 172.21.15.43:0/349582637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a1def60 msgr2=0x55b32a5d70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:32.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a2987d640 1 --2- 172.21.15.43:0/349582637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a1def60 0x55b32a5d70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a2987d640 1 -- 172.21.15.43:0/349582637 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b32a1f80f0 con 0x55b32a1f74d0 2024-04-25T10:44:32.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a2a87f640 1 --2- 172.21.15.43:0/349582637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a1ff6b0 0x55b32a5e37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:32.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a2a07e640 1 --2- 172.21.15.43:0/349582637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a1def60 0x55b32a5d70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:32.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.498+0000 7f5a2987d640 1 --2- 172.21.15.43:0/349582637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1f74d0 0x55b32a5e32a0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f5a14017000 tx=0x7f5a1400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e210dc77f9b6250 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:32.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.499+0000 7f5a0ffff640 1 -- 172.21.15.43:0/349582637 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a14016040 con 0x55b32a1f74d0 2024-04-25T10:44:32.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.499+0000 7f5a0ffff640 1 -- 172.21.15.43:0/349582637 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a14009d00 con 0x55b32a1f74d0 2024-04-25T10:44:32.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.499+0000 7f5a0ffff640 1 -- 172.21.15.43:0/349582637 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a14029900 con 0x55b32a1f74d0 2024-04-25T10:44:32.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.499+0000 7f5a3318f740 1 -- 172.21.15.43:0/349582637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1f74d0 msgr2=0x55b32a5e32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:32.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.499+0000 7f5a3318f740 1 --2- 172.21.15.43:0/349582637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1f74d0 0x55b32a5e32a0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f5a14017000 tx=0x7f5a1400ec40 comp rx=0 tx=0).stop 2024-04-25T10:44:32.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.499+0000 7f5a3318f740 1 -- 172.21.15.43:0/349582637 shutdown_connections 2024-04-25T10:44:32.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.499+0000 7f5a3318f740 1 --2- 172.21.15.43:0/349582637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a1ff6b0 0x55b32a5e37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.499+0000 7f5a3318f740 1 --2- 172.21.15.43:0/349582637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1f74d0 0x55b32a5e32a0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.499+0000 7f5a3318f740 1 --2- 172.21.15.43:0/349582637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a1def60 0x55b32a5d70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.499+0000 7f5a3318f740 1 -- 172.21.15.43:0/349582637 >> 172.21.15.43:0/349582637 conn(0x55b32a2144e0 msgr2=0x55b32a5bcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:32.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.499+0000 7f5a3318f740 1 -- 172.21.15.43:0/349582637 shutdown_connections 2024-04-25T10:44:32.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.499+0000 7f5a3318f740 1 -- 172.21.15.43:0/349582637 wait complete. 2024-04-25T10:44:32.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.500+0000 7f5a3318f740 1 -- start start 2024-04-25T10:44:32.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.500+0000 7f5a3318f740 1 Processor -- start 2024-04-25T10:44:32.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.500+0000 7f5a3318f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1def60 0x55b32a5d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.500+0000 7f5a3318f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a1f74d0 0x55b32a5ce400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.500+0000 7f5a3318f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a1ff6b0 0x55b32a5ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.500+0000 7f5a3318f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5a241ae170 con 0x55b32a1def60 2024-04-25T10:44:32.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.500+0000 7f5a3318f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5a241af5e0 con 0x55b32a1ff6b0 2024-04-25T10:44:32.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.500+0000 7f5a3318f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5a241ae840 con 0x55b32a1f74d0 2024-04-25T10:44:32.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.500+0000 7f5a3318f740 10 client.0 _ll_register_callbacks cb 0x7f5a20005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:32.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.500+0000 7f5a2a07e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1def60 0x55b32a5d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.500+0000 7f5a2a07e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1def60 0x55b32a5d3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43138/0 (socket says 172.21.15.43:43138) 2024-04-25T10:44:32.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.500+0000 7f5a2a07e640 1 -- 172.21.15.43:0/210270834 learned_addr learned my addr 172.21.15.43:0/210270834 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:32.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.500+0000 7f5a2a87f640 1 --2- 172.21.15.43:0/210270834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a1ff6b0 0x55b32a5ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.500+0000 7f5a2987d640 1 --2- 172.21.15.43:0/210270834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a1f74d0 0x55b32a5ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.500+0000 7f5a2a07e640 1 -- 172.21.15.43:0/210270834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a1f74d0 msgr2=0x55b32a5ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:32.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.500+0000 7f5a2a07e640 1 --2- 172.21.15.43:0/210270834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a1f74d0 0x55b32a5ce400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.500+0000 7f5a2a07e640 1 -- 172.21.15.43:0/210270834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a1ff6b0 msgr2=0x55b32a5ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.500+0000 7f5a2a07e640 1 --2- 172.21.15.43:0/210270834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a1ff6b0 0x55b32a5ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.500+0000 7f5a2a07e640 1 -- 172.21.15.43:0/210270834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b32a4e74b0 con 0x55b32a1def60 2024-04-25T10:44:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.500+0000 7f5a2a87f640 1 --2- 172.21.15.43:0/210270834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a1ff6b0 0x55b32a5ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.501+0000 7f5a2a07e640 1 --2- 172.21.15.43:0/210270834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1def60 0x55b32a5d3820 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f5a200c1300 tx=0x7f5a2000bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55bcb2fcd82492ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.501+0000 7f5a0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.501+0000 7f5a0dffb640 1 -- 172.21.15.43:0/210270834 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20021040 con 0x55b32a1def60 2024-04-25T10:44:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.501+0000 7f5a0dffb640 1 -- 172.21.15.43:0/210270834 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f5a200a60e0 con 0x55b32a1def60 2024-04-25T10:44:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.501+0000 7f5a3318f740 1 -- 172.21.15.43:0/210270834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a14009d00 con 0x55b32a1def60 2024-04-25T10:44:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.501+0000 7f5a0dffb640 1 -- 172.21.15.43:0/210270834 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5a200a6780 con 0x55b32a1def60 2024-04-25T10:44:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.501+0000 7f5a3318f740 10 client.19002 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.501+0000 7f5a3318f740 1 -- 172.21.15.43:0/210270834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b32a6448e0 con 0x55b32a1def60 2024-04-25T10:44:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.501+0000 7f5a0dffb640 1 -- 172.21.15.43:0/210270834 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a14009d00 con 0x55b32a1def60 2024-04-25T10:44:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.501+0000 7f5a0dffb640 1 -- 172.21.15.43:0/210270834 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5a2006d310 con 0x55b32a1def60 2024-04-25T10:44:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.501+0000 7f5a3318f740 10 client.19002 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.501+0000 7f5a3318f740 10 client.19002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:44:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.501+0000 7f5a3318f740 20 client.19002 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.501+0000 7f5a3318f740 1 --2- 172.21.15.43:0/210270834 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b32a220390 0x55b32a647110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.501+0000 7f5a3318f740 4 client.19002 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:44:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.501+0000 7f5a3318f740 1 -- 172.21.15.43:0/210270834 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b32a52e800 con 0x55b32a220390 2024-04-25T10:44:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.502+0000 7f5a2987d640 1 --2- 172.21.15.43:0/210270834 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b32a220390 0x55b32a647110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.502+0000 7f5a2987d640 1 --2- 172.21.15.43:0/210270834 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b32a220390 0x55b32a647110 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53ec1f319638b57d server_cookie=d6eddef959df8d93 in_seq=0 out_seq=0 2024-04-25T10:44:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.502+0000 7f5a0dffb640 10 client.19002 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.503+0000 7f5a0dffb640 1 -- 172.21.15.43:0/210270834 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b32a52e800 con 0x55b32a220390 2024-04-25T10:44:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.503+0000 7f5a0dffb640 10 client.19002 handle_command_reply: tid=0 2024-04-25T10:44:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.503+0000 7f5a3318f740 1 client.19002 shutdown 2024-04-25T10:44:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.503+0000 7f5a3318f740 1 -- 172.21.15.43:0/210270834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1def60 msgr2=0x55b32a5d3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.503+0000 7f5a3318f740 1 --2- 172.21.15.43:0/210270834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1def60 0x55b32a5d3820 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f5a200c1300 tx=0x7f5a2000bf70 comp rx=0 tx=0).stop 2024-04-25T10:44:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.503+0000 7f5a3318f740 1 -- 172.21.15.43:0/210270834 shutdown_connections 2024-04-25T10:44:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.503+0000 7f5a3318f740 1 --2- 172.21.15.43:0/210270834 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55b32a220390 0x55b32a647110 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.503+0000 7f5a3318f740 1 --2- 172.21.15.43:0/210270834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b32a1ff6b0 0x55b32a5ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.503+0000 7f5a0dffb640 0 client.19002 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.503+0000 7f5a3318f740 1 --2- 172.21.15.43:0/210270834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b32a1f74d0 0x55b32a5ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.503+0000 7f5a3318f740 1 --2- 172.21.15.43:0/210270834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b32a1def60 0x55b32a5d3820 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.503+0000 7f5a3318f740 1 -- 172.21.15.43:0/210270834 >> 172.21.15.43:0/210270834 conn(0x55b32a2144e0 msgr2=0x55b32a5c9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:32.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.503+0000 7f5a3318f740 1 -- 172.21.15.43:0/210270834 shutdown_connections 2024-04-25T10:44:32.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.503+0000 7f5a3318f740 1 -- 172.21.15.43:0/210270834 wait complete. 2024-04-25T10:44:32.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.503+0000 7f5a3318f740 20 client.19002 trim_cache size 0 max 16384 2024-04-25T10:44:32.520 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:44:32.520 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T10:44:32.520 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T10:44:32.521 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:44:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.503+0000 7f5a2c2e2640 1 -- 172.21.15.43:0/1095714341 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5a100afc60 msgr2=0x7f5a100b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.503+0000 7f5a2c2e2640 1 --2- 172.21.15.43:0/1095714341 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5a100afc60 0x7f5a100b2170 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c005e30 tx=0x7f5a1c005c30 comp rx=0 tx=0).stop 2024-04-25T10:44:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.503+0000 7f5a2c2e2640 1 -- 172.21.15.43:0/1095714341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a24102070 msgr2=0x7f5a2419a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.503+0000 7f5a2c2e2640 1 --2- 172.21.15.43:0/1095714341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a24102070 0x7f5a2419a7b0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f5a20005d00 tx=0x7f5a2000dde0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.504+0000 7f5a2c2e2640 1 -- 172.21.15.43:0/1095714341 shutdown_connections 2024-04-25T10:44:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.504+0000 7f5a2c2e2640 1 --2- 172.21.15.43:0/1095714341 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f5a100afc60 0x7f5a100b2170 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.504+0000 7f5a2c2e2640 1 --2- 172.21.15.43:0/1095714341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a24109bc0 0x7f5a2419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.504+0000 7f5a2c2e2640 1 --2- 172.21.15.43:0/1095714341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a2410e570 0x7f5a2419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.504+0000 7f5a2c2e2640 1 --2- 172.21.15.43:0/1095714341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a24102070 0x7f5a2419a7b0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.504+0000 7f5a2c2e2640 1 -- 172.21.15.43:0/1095714341 >> 172.21.15.43:0/1095714341 conn(0x7f5a24077920 msgr2=0x7f5a2410a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.504+0000 7f5a2c2e2640 1 -- 172.21.15.43:0/1095714341 shutdown_connections 2024-04-25T10:44:32.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.504+0000 7f5a2c2e2640 1 -- 172.21.15.43:0/1095714341 wait complete. 2024-04-25T10:44:32.524 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:44:32.524 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:44:32.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.673+0000 7f644e39d640 1 -- 172.21.15.43:0/909646618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f644810e2e0 msgr2=0x7f6448110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:32.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.673+0000 7f644e39d640 1 --2- 172.21.15.43:0/909646618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f644810e2e0 0x7f6448110700 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f6440009ca0 tx=0x7f644002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.674+0000 7f644e39d640 1 -- 172.21.15.43:0/909646618 shutdown_connections 2024-04-25T10:44:32.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.674+0000 7f644e39d640 1 --2- 172.21.15.43:0/909646618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f644810e2e0 0x7f6448110700 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.674+0000 7f644e39d640 1 --2- 172.21.15.43:0/909646618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6448103a50 0x7f6448103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.674+0000 7f644e39d640 1 --2- 172.21.15.43:0/909646618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6448109980 0x7f6448103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.674+0000 7f644e39d640 1 -- 172.21.15.43:0/909646618 >> 172.21.15.43:0/909646618 conn(0x7f64480ff030 msgr2=0x7f6448101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:32.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.674+0000 7f644e39d640 1 -- 172.21.15.43:0/909646618 shutdown_connections 2024-04-25T10:44:32.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.674+0000 7f644e39d640 1 -- 172.21.15.43:0/909646618 wait complete. 2024-04-25T10:44:32.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.674+0000 7f644e39d640 1 Processor -- start 2024-04-25T10:44:32.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.674+0000 7f644e39d640 1 -- start start 2024-04-25T10:44:32.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.675+0000 7f644e39d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6448103a50 0x7f644819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.675+0000 7f644e39d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6448109980 0x7f644819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.675+0000 7f644e39d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f644810e2e0 0x7f644819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.675+0000 7f644e39d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6448113b40 con 0x7f644810e2e0 2024-04-25T10:44:32.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.675+0000 7f644e39d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f64481139c0 con 0x7f6448109980 2024-04-25T10:44:32.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.675+0000 7f644e39d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6448113cc0 con 0x7f6448103a50 2024-04-25T10:44:32.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.675+0000 7f644db9c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f644810e2e0 0x7f644819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.675+0000 7f644db9c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f644810e2e0 0x7f644819ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43164/0 (socket says 172.21.15.43:43164) 2024-04-25T10:44:32.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.675+0000 7f644d39b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6448103a50 0x7f644819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.675+0000 7f644db9c640 1 -- 172.21.15.43:0/1167012375 learned_addr learned my addr 172.21.15.43:0/1167012375 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:32.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.675+0000 7f644cb9a640 1 --2- 172.21.15.43:0/1167012375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6448109980 0x7f644819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.675+0000 7f644db9c640 1 -- 172.21.15.43:0/1167012375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6448103a50 msgr2=0x7f644819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:32.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.675+0000 7f644db9c640 1 --2- 172.21.15.43:0/1167012375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6448103a50 0x7f644819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.675+0000 7f644db9c640 1 -- 172.21.15.43:0/1167012375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6448109980 msgr2=0x7f644819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.675+0000 7f644db9c640 1 --2- 172.21.15.43:0/1167012375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6448109980 0x7f644819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.675+0000 7f644db9c640 1 -- 172.21.15.43:0/1167012375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644819f2a0 con 0x7f644810e2e0 2024-04-25T10:44:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.675+0000 7f644cb9a640 1 --2- 172.21.15.43:0/1167012375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6448109980 0x7f644819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.675+0000 7f644db9c640 1 --2- 172.21.15.43:0/1167012375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f644810e2e0 0x7f644819ece0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f6440006010 tx=0x7f644002f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.675+0000 7f643e7fc640 1 -- 172.21.15.43:0/1167012375 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440003cf0 con 0x7f644810e2e0 2024-04-25T10:44:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.676+0000 7f644e39d640 1 -- 172.21.15.43:0/1167012375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644819f530 con 0x7f644810e2e0 2024-04-25T10:44:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.676+0000 7f643e7fc640 1 -- 172.21.15.43:0/1167012375 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64400044f0 con 0x7f644810e2e0 2024-04-25T10:44:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.676+0000 7f644e39d640 1 -- 172.21.15.43:0/1167012375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644819fa80 con 0x7f644810e2e0 2024-04-25T10:44:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.676+0000 7f643e7fc640 1 -- 172.21.15.43:0/1167012375 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644002fc60 con 0x7f644810e2e0 2024-04-25T10:44:32.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.677+0000 7f644e39d640 1 -- 172.21.15.43:0/1167012375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6448104c30 con 0x7f644810e2e0 2024-04-25T10:44:32.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.678+0000 7f643e7fc640 1 -- 172.21.15.43:0/1167012375 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f64400046a0 con 0x7f644810e2e0 2024-04-25T10:44:32.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.679+0000 7f643e7fc640 1 --2- 172.21.15.43:0/1167012375 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f64340afc50 0x7f64340b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:32.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.679+0000 7f644d39b640 1 --2- 172.21.15.43:0/1167012375 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f64340afc50 0x7f64340b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:32.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.679+0000 7f643e7fc640 1 -- 172.21.15.43:0/1167012375 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f6440100980 con 0x7f644810e2e0 2024-04-25T10:44:32.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.679+0000 7f644d39b640 1 --2- 172.21.15.43:0/1167012375 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f64340afc50 0x7f64340b2160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6448104980 tx=0x7f644400a2d0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:32.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.683+0000 7f643e7fc640 1 -- 172.21.15.43:0/1167012375 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f64400c92d0 con 0x7f644810e2e0 2024-04-25T10:44:32.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.912+0000 7f644e39d640 1 -- 172.21.15.43:0/1167012375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f644806a030 con 0x7f644810e2e0 2024-04-25T10:44:32.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.913+0000 7f643e7fc640 1 -- 172.21.15.43:0/1167012375 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7f644806a030 con 0x7f644810e2e0 2024-04-25T10:44:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:44:32.915 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:44:32.915 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:44:32.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.914+0000 7f644e39d640 1 -- 172.21.15.43:0/1167012375 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f64340afc50 msgr2=0x7f64340b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:32.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.914+0000 7f644e39d640 1 --2- 172.21.15.43:0/1167012375 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f64340afc50 0x7f64340b2160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6448104980 tx=0x7f644400a2d0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.915+0000 7f644e39d640 1 -- 172.21.15.43:0/1167012375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f644810e2e0 msgr2=0x7f644819ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:32.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.915+0000 7f644e39d640 1 --2- 172.21.15.43:0/1167012375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f644810e2e0 0x7f644819ece0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f6440006010 tx=0x7f644002f470 comp rx=0 tx=0).stop 2024-04-25T10:44:32.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.915+0000 7f644e39d640 1 -- 172.21.15.43:0/1167012375 shutdown_connections 2024-04-25T10:44:32.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.915+0000 7f644e39d640 1 --2- 172.21.15.43:0/1167012375 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f64340afc50 0x7f64340b2160 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.915+0000 7f644e39d640 1 --2- 172.21.15.43:0/1167012375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6448109980 0x7f644819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.915+0000 7f644e39d640 1 --2- 172.21.15.43:0/1167012375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f644810e2e0 0x7f644819ece0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.915+0000 7f644e39d640 1 --2- 172.21.15.43:0/1167012375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6448103a50 0x7f644819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:32.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.915+0000 7f644e39d640 1 -- 172.21.15.43:0/1167012375 >> 172.21.15.43:0/1167012375 conn(0x7f64480ff030 msgr2=0x7f64480ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.915+0000 7f644e39d640 1 -- 172.21.15.43:0/1167012375 shutdown_connections 2024-04-25T10:44:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:32.915+0000 7f644e39d640 1 -- 172.21.15.43:0/1167012375 wait complete. 2024-04-25T10:44:32.928 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T10:44:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.043+0000 7f9643964640 1 -- 172.21.15.43:0/1701994790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f963c1027c0 msgr2=0x7f963c110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.043+0000 7f9643964640 1 --2- 172.21.15.43:0/1701994790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f963c1027c0 0x7f963c110640 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f9634009900 tx=0x7f963402a410 comp rx=0 tx=0).stop 2024-04-25T10:44:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.043+0000 7f9643964640 1 -- 172.21.15.43:0/1701994790 shutdown_connections 2024-04-25T10:44:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.043+0000 7f9643964640 1 --2- 172.21.15.43:0/1701994790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f963c109fc0 0x7f963c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.043+0000 7f9643964640 1 --2- 172.21.15.43:0/1701994790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f963c1027c0 0x7f963c110640 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.043+0000 7f9643964640 1 --2- 172.21.15.43:0/1701994790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f963c109b20 0x7f963c101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.043+0000 7f9643964640 1 -- 172.21.15.43:0/1701994790 >> 172.21.15.43:0/1701994790 conn(0x7f963c077880 msgr2=0x7f963c077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.043+0000 7f9643964640 1 -- 172.21.15.43:0/1701994790 shutdown_connections 2024-04-25T10:44:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.043+0000 7f9643964640 1 -- 172.21.15.43:0/1701994790 wait complete. 2024-04-25T10:44:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.044+0000 7f9643964640 1 Processor -- start 2024-04-25T10:44:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.044+0000 7f9643964640 1 -- start start 2024-04-25T10:44:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.044+0000 7f9643964640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f963c1027c0 0x7f963c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.044+0000 7f9643964640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f963c109b20 0x7f963c19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.044+0000 7f9643964640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f963c109fc0 0x7f963c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.044+0000 7f9643964640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f963c113d10 con 0x7f963c109b20 2024-04-25T10:44:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.044+0000 7f9643964640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f963c113b90 con 0x7f963c1027c0 2024-04-25T10:44:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.044+0000 7f9643964640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f963c113e90 con 0x7f963c109fc0 2024-04-25T10:44:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.044+0000 7f9640eff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f963c109b20 0x7f963c19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.045+0000 7f9640eff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f963c109b20 0x7f963c19ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43184/0 (socket says 172.21.15.43:43184) 2024-04-25T10:44:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.045+0000 7f9640eff640 1 -- 172.21.15.43:0/3018077719 learned_addr learned my addr 172.21.15.43:0/3018077719 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.045+0000 7f9641700640 1 --2- 172.21.15.43:0/3018077719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f963c1027c0 0x7f963c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.045+0000 7f9641f01640 1 --2- 172.21.15.43:0/3018077719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f963c109fc0 0x7f963c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.045+0000 7f9640eff640 1 -- 172.21.15.43:0/3018077719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f963c109fc0 msgr2=0x7f963c19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.045+0000 7f9640eff640 1 --2- 172.21.15.43:0/3018077719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f963c109fc0 0x7f963c19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.045+0000 7f9640eff640 1 -- 172.21.15.43:0/3018077719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f963c1027c0 msgr2=0x7f963c19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.045+0000 7f9640eff640 1 --2- 172.21.15.43:0/3018077719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f963c1027c0 0x7f963c19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.045+0000 7f9640eff640 1 -- 172.21.15.43:0/3018077719 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963c19f590 con 0x7f963c109b20 2024-04-25T10:44:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.045+0000 7f9641f01640 1 --2- 172.21.15.43:0/3018077719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f963c109fc0 0x7f963c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:33.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.045+0000 7f9641700640 1 --2- 172.21.15.43:0/3018077719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f963c1027c0 0x7f963c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:44:33.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.045+0000 7f9640eff640 1 --2- 172.21.15.43:0/3018077719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f963c109b20 0x7f963c19ac90 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f962c00e9f0 tx=0x7f962c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:33.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.045+0000 7f96327fc640 1 -- 172.21.15.43:0/3018077719 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c00ccc0 con 0x7f963c109b20 2024-04-25T10:44:33.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.045+0000 7f96327fc640 1 -- 172.21.15.43:0/3018077719 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f962c0035f0 con 0x7f963c109b20 2024-04-25T10:44:33.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.045+0000 7f9643964640 1 -- 172.21.15.43:0/3018077719 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963c19f880 con 0x7f963c109b20 2024-04-25T10:44:33.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.045+0000 7f96327fc640 1 -- 172.21.15.43:0/3018077719 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c00ce60 con 0x7f963c109b20 2024-04-25T10:44:33.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.045+0000 7f9643964640 1 -- 172.21.15.43:0/3018077719 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963c1a7200 con 0x7f963c109b20 2024-04-25T10:44:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.048+0000 7f964a815740 1 Processor -- start 2024-04-25T10:44:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.048+0000 7f96327fc640 1 -- 172.21.15.43:0/3018077719 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f962c003aa0 con 0x7f963c109b20 2024-04-25T10:44:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.048+0000 7f964a815740 1 -- start start 2024-04-25T10:44:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.048+0000 7f964a815740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff5409c390 0x55ff5443d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.048+0000 7f964a815740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff540734d0 0x55ff5443dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.048+0000 7f964a815740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5407b6b0 0x55ff5444eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.048+0000 7f964a815740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ff543a2d80 con 0x55ff5409c390 2024-04-25T10:44:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.048+0000 7f964a815740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ff5435ca20 con 0x55ff5407b6b0 2024-04-25T10:44:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.048+0000 7f964a815740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ff5439ed40 con 0x55ff540734d0 2024-04-25T10:44:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.048+0000 7f9641f01640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff5409c390 0x55ff5443d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.048+0000 7f9641f01640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff5409c390 0x55ff5443d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43196/0 (socket says 172.21.15.43:43196) 2024-04-25T10:44:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.048+0000 7f9641f01640 1 -- 172.21.15.43:0/2678807879 learned_addr learned my addr 172.21.15.43:0/2678807879 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.049+0000 7f9641700640 1 --2- 172.21.15.43:0/2678807879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff540734d0 0x55ff5443dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.049+0000 7f9640eff640 1 --2- 172.21.15.43:0/2678807879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5407b6b0 0x55ff5444eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.049+0000 7f9641f01640 1 -- 172.21.15.43:0/2678807879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff540734d0 msgr2=0x55ff5443dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.049+0000 7f9641f01640 1 --2- 172.21.15.43:0/2678807879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff540734d0 0x55ff5443dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.049+0000 7f9641f01640 1 -- 172.21.15.43:0/2678807879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5407b6b0 msgr2=0x55ff5444eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.049+0000 7f9641f01640 1 --2- 172.21.15.43:0/2678807879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5407b6b0 0x55ff5444eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.049+0000 7f9641f01640 1 -- 172.21.15.43:0/2678807879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff54367c30 con 0x55ff5409c390 2024-04-25T10:44:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.049+0000 7f9640eff640 1 --2- 172.21.15.43:0/2678807879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5407b6b0 0x55ff5444eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.049+0000 7f96327fc640 1 --2- 172.21.15.43:0/3018077719 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f96280afc60 0x7f96280b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.049+0000 7f9641f01640 1 --2- 172.21.15.43:0/2678807879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff5409c390 0x55ff5443d6e0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f9634031440 tx=0x7f9634032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5f6d2b0873f4c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.049+0000 7f9641700640 1 --2- 172.21.15.43:0/3018077719 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f96280afc60 0x7f96280b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.049+0000 7f96327fc640 1 -- 172.21.15.43:0/3018077719 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f962c0df360 con 0x7f963c109b20 2024-04-25T10:44:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.049+0000 7f96277fe640 1 -- 172.21.15.43:0/2678807879 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963403f070 con 0x55ff5409c390 2024-04-25T10:44:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.049+0000 7f96277fe640 1 -- 172.21.15.43:0/2678807879 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96340316a0 con 0x55ff5409c390 2024-04-25T10:44:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.049+0000 7f96277fe640 1 -- 172.21.15.43:0/2678807879 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9634035070 con 0x55ff5409c390 2024-04-25T10:44:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.050+0000 7f964a815740 1 -- 172.21.15.43:0/2678807879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff5409c390 msgr2=0x55ff5443d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.050+0000 7f964a815740 1 --2- 172.21.15.43:0/2678807879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff5409c390 0x55ff5443d6e0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f9634031440 tx=0x7f9634032c20 comp rx=0 tx=0).stop 2024-04-25T10:44:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.050+0000 7f9641700640 1 --2- 172.21.15.43:0/3018077719 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f96280afc60 0x7f96280b2170 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9638009fd0 tx=0x7f9638009760 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.050+0000 7f964a815740 1 -- 172.21.15.43:0/2678807879 shutdown_connections 2024-04-25T10:44:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.050+0000 7f964a815740 1 --2- 172.21.15.43:0/2678807879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5407b6b0 0x55ff5444eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.050+0000 7f964a815740 1 --2- 172.21.15.43:0/2678807879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff540734d0 0x55ff5443dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.050+0000 7f964a815740 1 --2- 172.21.15.43:0/2678807879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff5409c390 0x55ff5443d6e0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.050+0000 7f964a815740 1 -- 172.21.15.43:0/2678807879 >> 172.21.15.43:0/2678807879 conn(0x55ff540904e0 msgr2=0x55ff54439fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.050+0000 7f964a815740 1 -- 172.21.15.43:0/2678807879 shutdown_connections 2024-04-25T10:44:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.050+0000 7f964a815740 1 -- 172.21.15.43:0/2678807879 wait complete. 2024-04-25T10:44:33.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.050+0000 7f964a815740 1 -- start start 2024-04-25T10:44:33.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.051+0000 7f964a815740 1 Processor -- start 2024-04-25T10:44:33.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.051+0000 7f964a815740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff540734d0 0x55ff5444c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.051+0000 7f964a815740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff5407b6b0 0x55ff5444cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.051+0000 7f964a815740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5409c390 0x55ff54458680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.051+0000 7f964a815740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ff5444f1d0 con 0x55ff5407b6b0 2024-04-25T10:44:33.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.051+0000 7f964a815740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ff54329dc0 con 0x55ff5409c390 2024-04-25T10:44:33.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.051+0000 7f964a815740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ff5444f350 con 0x55ff540734d0 2024-04-25T10:44:33.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.051+0000 7f964a815740 10 client.0 _ll_register_callbacks cb 0x55ff54345600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:33.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.051+0000 7f9640eff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff5407b6b0 0x55ff5444cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.051+0000 7f9640eff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff5407b6b0 0x55ff5444cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43204/0 (socket says 172.21.15.43:43204) 2024-04-25T10:44:33.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.051+0000 7f9640eff640 1 -- 172.21.15.43:0/2049419597 learned_addr learned my addr 172.21.15.43:0/2049419597 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:33.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.051+0000 7f9641f01640 1 --2- 172.21.15.43:0/2049419597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5409c390 0x55ff54458680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.051+0000 7f9641700640 1 --2- 172.21.15.43:0/2049419597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff540734d0 0x55ff5444c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.051+0000 7f9640eff640 1 -- 172.21.15.43:0/2049419597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff540734d0 msgr2=0x55ff5444c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.051+0000 7f9640eff640 1 --2- 172.21.15.43:0/2049419597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff540734d0 0x55ff5444c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.051+0000 7f9640eff640 1 -- 172.21.15.43:0/2049419597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5409c390 msgr2=0x55ff54458680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.051+0000 7f9640eff640 1 --2- 172.21.15.43:0/2049419597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5409c390 0x55ff54458680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.052+0000 7f9640eff640 1 -- 172.21.15.43:0/2049419597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff540b43f0 con 0x55ff5407b6b0 2024-04-25T10:44:33.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.052+0000 7f9641f01640 1 --2- 172.21.15.43:0/2049419597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5409c390 0x55ff54458680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:33.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.052+0000 7f9640eff640 1 --2- 172.21.15.43:0/2049419597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff5407b6b0 0x55ff5444cb00 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f962c0a3580 tx=0x7f962c0a73f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90c99615024ba39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:33.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.052+0000 7f9624ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:33.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.052+0000 7f9624ff9640 1 -- 172.21.15.43:0/2049419597 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c0a7c00 con 0x55ff5407b6b0 2024-04-25T10:44:33.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.052+0000 7f9624ff9640 1 -- 172.21.15.43:0/2049419597 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f962c0a7970 con 0x55ff5407b6b0 2024-04-25T10:44:33.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.052+0000 7f964a815740 1 -- 172.21.15.43:0/2049419597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96340316a0 con 0x55ff5407b6b0 2024-04-25T10:44:33.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.052+0000 7f9624ff9640 1 -- 172.21.15.43:0/2049419597 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f962c0a7500 con 0x55ff5407b6b0 2024-04-25T10:44:33.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.052+0000 7f964a815740 10 client.19020 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:33.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.052+0000 7f964a815740 1 -- 172.21.15.43:0/2049419597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ff5417e7d0 con 0x55ff5407b6b0 2024-04-25T10:44:33.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.052+0000 7f9624ff9640 1 -- 172.21.15.43:0/2049419597 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96340316a0 con 0x55ff5407b6b0 2024-04-25T10:44:33.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.052+0000 7f9624ff9640 1 -- 172.21.15.43:0/2049419597 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f962c0a2e70 con 0x55ff5407b6b0 2024-04-25T10:44:33.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.052+0000 7f964a815740 10 client.19020 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:33.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.052+0000 7f964a815740 10 client.19020 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:44:33.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.052+0000 7f964a815740 20 client.19020 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:33.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.052+0000 7f964a815740 1 --2- 172.21.15.43:0/2049419597 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55ff5405af60 0x55ff5445ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.052+0000 7f964a815740 4 client.19020 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:44:33.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.053+0000 7f964a815740 1 -- 172.21.15.43:0/2049419597 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff543b0990 con 0x55ff5405af60 2024-04-25T10:44:33.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.053+0000 7f9641700640 1 --2- 172.21.15.43:0/2049419597 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55ff5405af60 0x55ff5445ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.053+0000 7f9624ff9640 10 client.19020 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:44:33.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.053+0000 7f9641700640 1 --2- 172.21.15.43:0/2049419597 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55ff5405af60 0x55ff5445ea10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=474cc501650206ab server_cookie=ad435caa8de6c7b3 in_seq=0 out_seq=0 2024-04-25T10:44:33.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.054+0000 7f9624ff9640 1 -- 172.21.15.43:0/2049419597 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ff543b0990 con 0x55ff5405af60 2024-04-25T10:44:33.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.054+0000 7f9624ff9640 10 client.19020 handle_command_reply: tid=0 2024-04-25T10:44:33.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.054+0000 7f964a815740 1 client.19020 shutdown 2024-04-25T10:44:33.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.054+0000 7f964a815740 1 -- 172.21.15.43:0/2049419597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff5407b6b0 msgr2=0x55ff5444cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.054+0000 7f964a815740 1 --2- 172.21.15.43:0/2049419597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff5407b6b0 0x55ff5444cb00 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f962c0a3580 tx=0x7f962c0a73f0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.054+0000 7f964a815740 1 -- 172.21.15.43:0/2049419597 shutdown_connections 2024-04-25T10:44:33.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.054+0000 7f964a815740 1 --2- 172.21.15.43:0/2049419597 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55ff5405af60 0x55ff5445ea10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.054+0000 7f964a815740 1 --2- 172.21.15.43:0/2049419597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5409c390 0x55ff54458680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.054+0000 7f964a815740 1 --2- 172.21.15.43:0/2049419597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff5407b6b0 0x55ff5444cb00 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.054+0000 7f964a815740 1 --2- 172.21.15.43:0/2049419597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff540734d0 0x55ff5444c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.054+0000 7f9624ff9640 0 client.19020 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:44:33.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.054+0000 7f964a815740 1 -- 172.21.15.43:0/2049419597 >> 172.21.15.43:0/2049419597 conn(0x55ff540904e0 msgr2=0x55ff5443f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:33.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.055+0000 7f964a815740 1 -- 172.21.15.43:0/2049419597 shutdown_connections 2024-04-25T10:44:33.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.055+0000 7f964a815740 1 -- 172.21.15.43:0/2049419597 wait complete. 2024-04-25T10:44:33.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.055+0000 7f964a815740 20 client.19020 trim_cache size 0 max 16384 2024-04-25T10:44:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.068+0000 7f964a815740 1 Processor -- start 2024-04-25T10:44:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.068+0000 7f964a815740 1 -- start start 2024-04-25T10:44:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f964a815740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff540734d0 0x55ff5444b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f964a815740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5407b6b0 0x55ff54458220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f964a815740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff5409c390 0x55ff544641d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f964a815740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1b00b0 con 0x55ff540734d0 2024-04-25T10:44:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f964a815740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1ad980 con 0x55ff5407b6b0 2024-04-25T10:44:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f964a815740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1ab7f0 con 0x55ff5409c390 2024-04-25T10:44:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f9641700640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff540734d0 0x55ff5444b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f9641700640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff540734d0 0x55ff5444b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43220/0 (socket says 172.21.15.43:43220) 2024-04-25T10:44:33.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f9641700640 1 -- 172.21.15.43:0/2846544650 learned_addr learned my addr 172.21.15.43:0/2846544650 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:33.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f9640eff640 1 --2- 172.21.15.43:0/2846544650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5407b6b0 0x55ff54458220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f9641700640 1 -- 172.21.15.43:0/2846544650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff5409c390 msgr2=0x55ff544641d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f9641f01640 1 --2- 172.21.15.43:0/2846544650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff5409c390 0x55ff544641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f9641700640 1 --2- 172.21.15.43:0/2846544650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff5409c390 0x55ff544641d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f9641700640 1 -- 172.21.15.43:0/2846544650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5407b6b0 msgr2=0x55ff54458220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f9641700640 1 --2- 172.21.15.43:0/2846544650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5407b6b0 0x55ff54458220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f9641700640 1 -- 172.21.15.43:0/2846544650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff5444a900 con 0x55ff540734d0 2024-04-25T10:44:33.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f9640eff640 1 --2- 172.21.15.43:0/2846544650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5407b6b0 0x55ff54458220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:33.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f9641f01640 1 --2- 172.21.15.43:0/2846544650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff5409c390 0x55ff544641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:33.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f9641700640 1 --2- 172.21.15.43:0/2846544650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff540734d0 0x55ff5444b390 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x55ff5443b070 tx=0x7f9638015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa1423a978474bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:33.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f9626ffd640 1 -- 172.21.15.43:0/2846544650 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638004710 con 0x55ff540734d0 2024-04-25T10:44:33.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f9626ffd640 1 -- 172.21.15.43:0/2846544650 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f963802d070 con 0x55ff540734d0 2024-04-25T10:44:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.069+0000 7f9626ffd640 1 -- 172.21.15.43:0/2846544650 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963801f980 con 0x55ff540734d0 2024-04-25T10:44:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.070+0000 7f964a815740 1 -- 172.21.15.43:0/2846544650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff540734d0 msgr2=0x55ff5444b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.070+0000 7f964a815740 1 --2- 172.21.15.43:0/2846544650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff540734d0 0x55ff5444b390 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x55ff5443b070 tx=0x7f9638015000 comp rx=0 tx=0).stop 2024-04-25T10:44:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.070+0000 7f964a815740 1 -- 172.21.15.43:0/2846544650 shutdown_connections 2024-04-25T10:44:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.070+0000 7f964a815740 1 --2- 172.21.15.43:0/2846544650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5407b6b0 0x55ff54458220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.070+0000 7f964a815740 1 --2- 172.21.15.43:0/2846544650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff5409c390 0x55ff544641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.070+0000 7f964a815740 1 --2- 172.21.15.43:0/2846544650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff540734d0 0x55ff5444b390 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.070+0000 7f964a815740 1 -- 172.21.15.43:0/2846544650 >> 172.21.15.43:0/2846544650 conn(0x55ff540904e0 msgr2=0x55ff5444ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.070+0000 7f964a815740 1 -- 172.21.15.43:0/2846544650 shutdown_connections 2024-04-25T10:44:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.070+0000 7f964a815740 1 -- 172.21.15.43:0/2846544650 wait complete. 2024-04-25T10:44:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.070+0000 7f964a815740 1 -- start start 2024-04-25T10:44:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.071+0000 7f964a815740 1 Processor -- start 2024-04-25T10:44:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.071+0000 7f964a815740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff540734d0 0x55ff54454910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.071+0000 7f964a815740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5407b6b0 0x55ff5444fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.071+0000 7f964a815740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff5409c390 0x55ff544501a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.071+0000 7f964a815740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1abe40 con 0x55ff540734d0 2024-04-25T10:44:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.071+0000 7f964a815740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1ad2b0 con 0x55ff5407b6b0 2024-04-25T10:44:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.071+0000 7f964a815740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1ac510 con 0x55ff5409c390 2024-04-25T10:44:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.071+0000 7f964a815740 10 client.0 _ll_register_callbacks cb 0x7f962c0a72d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.071+0000 7f9641700640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff540734d0 0x55ff54454910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.071+0000 7f9641700640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff540734d0 0x55ff54454910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43234/0 (socket says 172.21.15.43:43234) 2024-04-25T10:44:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.071+0000 7f9641700640 1 -- 172.21.15.43:0/726996718 learned_addr learned my addr 172.21.15.43:0/726996718 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.071+0000 7f9640eff640 1 --2- 172.21.15.43:0/726996718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5407b6b0 0x55ff5444fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.071+0000 7f9641700640 1 -- 172.21.15.43:0/726996718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff5409c390 msgr2=0x55ff544501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.071+0000 7f9641700640 1 --2- 172.21.15.43:0/726996718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff5409c390 0x55ff544501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.071+0000 7f9641700640 1 -- 172.21.15.43:0/726996718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5407b6b0 msgr2=0x55ff5444fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.071+0000 7f9641700640 1 --2- 172.21.15.43:0/726996718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5407b6b0 0x55ff5444fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.071+0000 7f9641700640 1 -- 172.21.15.43:0/726996718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff544506e0 con 0x55ff540734d0 2024-04-25T10:44:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.071+0000 7f9640eff640 1 --2- 172.21.15.43:0/726996718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5407b6b0 0x55ff5444fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.072+0000 7f9641700640 1 --2- 172.21.15.43:0/726996718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff540734d0 0x55ff54454910 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f9638004970 tx=0x7f9638004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1d7c18c3092444 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.072+0000 7f9624ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.072+0000 7f9624ff9640 1 -- 172.21.15.43:0/726996718 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638028800 con 0x55ff540734d0 2024-04-25T10:44:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.072+0000 7f9624ff9640 1 -- 172.21.15.43:0/726996718 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f96380289a0 con 0x55ff540734d0 2024-04-25T10:44:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.072+0000 7f964a815740 1 -- 172.21.15.43:0/726996718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f963802d070 con 0x55ff540734d0 2024-04-25T10:44:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.072+0000 7f9624ff9640 1 -- 172.21.15.43:0/726996718 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f963803c470 con 0x55ff540734d0 2024-04-25T10:44:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.072+0000 7f964a815740 10 client.19026 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.072+0000 7f964a815740 1 -- 172.21.15.43:0/726996718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ff544c69c0 con 0x55ff540734d0 2024-04-25T10:44:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.072+0000 7f9624ff9640 1 -- 172.21.15.43:0/726996718 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f963802d070 con 0x55ff540734d0 2024-04-25T10:44:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.072+0000 7f9624ff9640 1 -- 172.21.15.43:0/726996718 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9638045850 con 0x55ff540734d0 2024-04-25T10:44:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.072+0000 7f964a815740 10 client.19026 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.072+0000 7f964a815740 10 client.19026 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T10:44:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.072+0000 7f964a815740 20 client.19026 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.072+0000 7f964a815740 1 --2- 172.21.15.43:0/726996718 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55ff540303a0 0x55ff544c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.072+0000 7f964a815740 4 client.19026 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:44:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.072+0000 7f964a815740 1 -- 172.21.15.43:0/726996718 --> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff543aa800 con 0x55ff540303a0 2024-04-25T10:44:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.073+0000 7f9640eff640 1 --2- 172.21.15.43:0/726996718 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55ff540303a0 0x55ff544c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.073+0000 7f9624ff9640 10 client.19026 ms_handle_connect on v2:172.21.15.164:6834/1437008220 2024-04-25T10:44:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.073+0000 7f9640eff640 1 --2- 172.21.15.43:0/726996718 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55ff540303a0 0x55ff544c90c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1c43a7acca91f7 server_cookie=540773b29a5d25e3 in_seq=0 out_seq=0 2024-04-25T10:44:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.073+0000 7f9624ff9640 1 -- 172.21.15.43:0/726996718 <== mds.2 v2:172.21.15.164:6834/1437008220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff543aa800 con 0x55ff540303a0 2024-04-25T10:44:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.073+0000 7f9624ff9640 10 client.19026 handle_command_reply: tid=0 2024-04-25T10:44:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.073+0000 7f964a815740 1 client.19026 shutdown 2024-04-25T10:44:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.073+0000 7f964a815740 1 -- 172.21.15.43:0/726996718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff540734d0 msgr2=0x55ff54454910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.073+0000 7f964a815740 1 --2- 172.21.15.43:0/726996718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff540734d0 0x55ff54454910 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f9638004970 tx=0x7f9638004a40 comp rx=0 tx=0).stop 2024-04-25T10:44:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.073+0000 7f964a815740 1 -- 172.21.15.43:0/726996718 shutdown_connections 2024-04-25T10:44:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.073+0000 7f964a815740 1 --2- 172.21.15.43:0/726996718 >> [v2:172.21.15.164:6834/1437008220,v1:172.21.15.164:6835/1437008220] conn(0x55ff540303a0 0x55ff544c90c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.073+0000 7f964a815740 1 --2- 172.21.15.43:0/726996718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff5407b6b0 0x55ff5444fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.073+0000 7f964a815740 1 --2- 172.21.15.43:0/726996718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff5409c390 0x55ff544501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.073+0000 7f9624ff9640 0 client.19026 ms_handle_reset on v2:172.21.15.164:6834/1437008220 2024-04-25T10:44:33.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.073+0000 7f964a815740 1 --2- 172.21.15.43:0/726996718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff540734d0 0x55ff54454910 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.073+0000 7f964a815740 1 -- 172.21.15.43:0/726996718 >> 172.21.15.43:0/726996718 conn(0x55ff540904e0 msgr2=0x55ff5443f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:33.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.074+0000 7f964a815740 1 -- 172.21.15.43:0/726996718 shutdown_connections 2024-04-25T10:44:33.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.074+0000 7f964a815740 1 -- 172.21.15.43:0/726996718 wait complete. 2024-04-25T10:44:33.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.074+0000 7f964a815740 20 client.19026 trim_cache size 0 max 16384 2024-04-25T10:44:33.083 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:44:33.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.074+0000 7f9643964640 1 -- 172.21.15.43:0/3018077719 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f96280afc60 msgr2=0x7f96280b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.074+0000 7f9643964640 1 --2- 172.21.15.43:0/3018077719 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f96280afc60 0x7f96280b2170 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9638009fd0 tx=0x7f9638009760 comp rx=0 tx=0).stop 2024-04-25T10:44:33.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.074+0000 7f9643964640 1 -- 172.21.15.43:0/3018077719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f963c109b20 msgr2=0x7f963c19ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.074+0000 7f9643964640 1 --2- 172.21.15.43:0/3018077719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f963c109b20 0x7f963c19ac90 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f962c00e9f0 tx=0x7f962c00eeb0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.074+0000 7f9643964640 1 -- 172.21.15.43:0/3018077719 shutdown_connections 2024-04-25T10:44:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.074+0000 7f9643964640 1 --2- 172.21.15.43:0/3018077719 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f96280afc60 0x7f96280b2170 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.074+0000 7f9643964640 1 --2- 172.21.15.43:0/3018077719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f963c109fc0 0x7f963c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.074+0000 7f9643964640 1 --2- 172.21.15.43:0/3018077719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f963c109b20 0x7f963c19ac90 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.074+0000 7f9643964640 1 --2- 172.21.15.43:0/3018077719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f963c1027c0 0x7f963c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.074+0000 7f9643964640 1 -- 172.21.15.43:0/3018077719 >> 172.21.15.43:0/3018077719 conn(0x7f963c077880 msgr2=0x7f963c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.074+0000 7f9643964640 1 -- 172.21.15.43:0/3018077719 shutdown_connections 2024-04-25T10:44:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.074+0000 7f9643964640 1 -- 172.21.15.43:0/3018077719 wait complete. 2024-04-25T10:44:33.087 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T10:44:33.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.200+0000 7fb23d51f640 1 -- 172.21.15.43:0/2633512173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb238109b30 msgr2=0x7fb238103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.200+0000 7fb23d51f640 1 --2- 172.21.15.43:0/2633512173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb238109b30 0x7fb238103720 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fb228009900 tx=0x7fb22802a410 comp rx=0 tx=0).stop 2024-04-25T10:44:33.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.200+0000 7fb23d51f640 1 -- 172.21.15.43:0/2633512173 shutdown_connections 2024-04-25T10:44:33.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.200+0000 7fb23d51f640 1 --2- 172.21.15.43:0/2633512173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2381042d0 0x7fb238110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.200+0000 7fb23d51f640 1 --2- 172.21.15.43:0/2633512173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb238109fd0 0x7fb238103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.200+0000 7fb23d51f640 1 --2- 172.21.15.43:0/2633512173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb238109b30 0x7fb238103720 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.200+0000 7fb23d51f640 1 -- 172.21.15.43:0/2633512173 >> 172.21.15.43:0/2633512173 conn(0x7fb238079280 msgr2=0x7fb2380796c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:33.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.200+0000 7fb23d51f640 1 -- 172.21.15.43:0/2633512173 shutdown_connections 2024-04-25T10:44:33.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.200+0000 7fb23d51f640 1 -- 172.21.15.43:0/2633512173 wait complete. 2024-04-25T10:44:33.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.201+0000 7fb23d51f640 1 Processor -- start 2024-04-25T10:44:33.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.201+0000 7fb23d51f640 1 -- start start 2024-04-25T10:44:33.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.201+0000 7fb23d51f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2381042d0 0x7fb23819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.201+0000 7fb23d51f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb238109b30 0x7fb23819ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.201+0000 7fb23d51f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb238109fd0 0x7fb23819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.201+0000 7fb23d51f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb238113d20 con 0x7fb238109fd0 2024-04-25T10:44:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.201+0000 7fb23d51f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb238113ba0 con 0x7fb238109b30 2024-04-25T10:44:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.201+0000 7fb23d51f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb238113ea0 con 0x7fb2381042d0 2024-04-25T10:44:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.201+0000 7fb2377fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb238109fd0 0x7fb23819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.201+0000 7fb2377fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb238109fd0 0x7fb23819efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43262/0 (socket says 172.21.15.43:43262) 2024-04-25T10:44:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.201+0000 7fb2377fe640 1 -- 172.21.15.43:0/1015752133 learned_addr learned my addr 172.21.15.43:0/1015752133 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.201+0000 7fb2367fc640 1 --2- 172.21.15.43:0/1015752133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb238109b30 0x7fb23819ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.201+0000 7fb236ffd640 1 --2- 172.21.15.43:0/1015752133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2381042d0 0x7fb23819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.202+0000 7fb2377fe640 1 -- 172.21.15.43:0/1015752133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2381042d0 msgr2=0x7fb23819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.202+0000 7fb2377fe640 1 --2- 172.21.15.43:0/1015752133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2381042d0 0x7fb23819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.202+0000 7fb2377fe640 1 -- 172.21.15.43:0/1015752133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb238109b30 msgr2=0x7fb23819ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.202+0000 7fb2377fe640 1 --2- 172.21.15.43:0/1015752133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb238109b30 0x7fb23819ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.202+0000 7fb2377fe640 1 -- 172.21.15.43:0/1015752133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23819f5e0 con 0x7fb238109fd0 2024-04-25T10:44:33.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.202+0000 7fb236ffd640 1 --2- 172.21.15.43:0/1015752133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2381042d0 0x7fb23819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:33.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.202+0000 7fb2377fe640 1 --2- 172.21.15.43:0/1015752133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb238109fd0 0x7fb23819efa0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fb23000efd0 tx=0x7fb23000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:33.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.202+0000 7fb21ffff640 1 -- 172.21.15.43:0/1015752133 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230003b70 con 0x7fb238109fd0 2024-04-25T10:44:33.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.202+0000 7fb21ffff640 1 -- 172.21.15.43:0/1015752133 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb230003d10 con 0x7fb238109fd0 2024-04-25T10:44:33.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.202+0000 7fb23d51f640 1 -- 172.21.15.43:0/1015752133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23819f8d0 con 0x7fb238109fd0 2024-04-25T10:44:33.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.202+0000 7fb21ffff640 1 -- 172.21.15.43:0/1015752133 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2300108a0 con 0x7fb238109fd0 2024-04-25T10:44:33.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.202+0000 7fb23d51f640 1 -- 172.21.15.43:0/1015752133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb23819fdd0 con 0x7fb238109fd0 2024-04-25T10:44:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.205+0000 7fb2443d0740 1 Processor -- start 2024-04-25T10:44:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.205+0000 7fb21ffff640 1 -- 172.21.15.43:0/1015752133 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fb230004020 con 0x7fb238109fd0 2024-04-25T10:44:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.205+0000 7fb2443d0740 1 -- start start 2024-04-25T10:44:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.205+0000 7fb2443d0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf526390 0x559ddf8c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.205+0000 7fb2443d0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ddf4fd4d0 0x559ddf8c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.205+0000 7fb2443d0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddf5056b0 0x559ddf8d8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.205+0000 7fb2443d0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559ddf82cd80 con 0x559ddf526390 2024-04-25T10:44:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.205+0000 7fb2443d0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559ddf7e6a20 con 0x559ddf5056b0 2024-04-25T10:44:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.205+0000 7fb2443d0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559ddf828d40 con 0x559ddf4fd4d0 2024-04-25T10:44:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.205+0000 7fb2367fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf526390 0x559ddf8c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.205+0000 7fb2367fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf526390 0x559ddf8c76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43270/0 (socket says 172.21.15.43:43270) 2024-04-25T10:44:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.205+0000 7fb2367fc640 1 -- 172.21.15.43:0/2699187674 learned_addr learned my addr 172.21.15.43:0/2699187674 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.206+0000 7fb236ffd640 1 --2- 172.21.15.43:0/2699187674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ddf4fd4d0 0x559ddf8c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.206+0000 7fb21ffff640 1 --2- 172.21.15.43:0/1015752133 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb2200afc60 0x7fb2200b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.206+0000 7fb2367fc640 1 -- 172.21.15.43:0/2699187674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ddf4fd4d0 msgr2=0x559ddf8c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.206+0000 7fb2367fc640 1 --2- 172.21.15.43:0/2699187674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ddf4fd4d0 0x559ddf8c7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.206+0000 7fb2367fc640 1 -- 172.21.15.43:0/2699187674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddf5056b0 msgr2=0x559ddf8d8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.206+0000 7fb2367fc640 1 --2- 172.21.15.43:0/2699187674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddf5056b0 0x559ddf8d8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.206+0000 7fb2367fc640 1 -- 172.21.15.43:0/2699187674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ddf7f1c30 con 0x559ddf526390 2024-04-25T10:44:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.206+0000 7fb2377fe640 1 --2- 172.21.15.43:0/1015752133 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb2200afc60 0x7fb2200b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.206+0000 7fb21ffff640 1 -- 172.21.15.43:0/1015752133 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fb2300df530 con 0x7fb238109fd0 2024-04-25T10:44:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.206+0000 7fb2367fc640 1 --2- 172.21.15.43:0/2699187674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf526390 0x559ddf8c76a0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fb22c009cd0 tx=0x7fb22c013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93009defcf5846 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.206+0000 7fb21d7fa640 1 -- 172.21.15.43:0/2699187674 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c013160 con 0x559ddf526390 2024-04-25T10:44:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.206+0000 7fb21d7fa640 1 -- 172.21.15.43:0/2699187674 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb22c017070 con 0x559ddf526390 2024-04-25T10:44:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.206+0000 7fb21d7fa640 1 -- 172.21.15.43:0/2699187674 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c003e40 con 0x559ddf526390 2024-04-25T10:44:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.207+0000 7fb2443d0740 1 -- 172.21.15.43:0/2699187674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf526390 msgr2=0x559ddf8c76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.207+0000 7fb2377fe640 1 --2- 172.21.15.43:0/1015752133 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb2200afc60 0x7fb2200b2170 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb2300a4940 tx=0x7fb230076040 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.207+0000 7fb2443d0740 1 --2- 172.21.15.43:0/2699187674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf526390 0x559ddf8c76a0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fb22c009cd0 tx=0x7fb22c013830 comp rx=0 tx=0).stop 2024-04-25T10:44:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.207+0000 7fb2443d0740 1 -- 172.21.15.43:0/2699187674 shutdown_connections 2024-04-25T10:44:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.207+0000 7fb2443d0740 1 --2- 172.21.15.43:0/2699187674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddf5056b0 0x559ddf8d8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.207+0000 7fb2443d0740 1 --2- 172.21.15.43:0/2699187674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ddf4fd4d0 0x559ddf8c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.207+0000 7fb2443d0740 1 --2- 172.21.15.43:0/2699187674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf526390 0x559ddf8c76a0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.207+0000 7fb2443d0740 1 -- 172.21.15.43:0/2699187674 >> 172.21.15.43:0/2699187674 conn(0x559ddf51a4e0 msgr2=0x559ddf8c3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.207+0000 7fb2443d0740 1 -- 172.21.15.43:0/2699187674 shutdown_connections 2024-04-25T10:44:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.207+0000 7fb2443d0740 1 -- 172.21.15.43:0/2699187674 wait complete. 2024-04-25T10:44:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.207+0000 7fb2443d0740 1 -- start start 2024-04-25T10:44:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.208+0000 7fb2443d0740 1 Processor -- start 2024-04-25T10:44:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.208+0000 7fb2443d0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddf4fd4d0 0x559ddf8d6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.208+0000 7fb2443d0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 0x559ddf8d6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.208+0000 7fb2443d0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ddf526390 0x559ddf8e26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.208+0000 7fb2443d0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559ddf8d91a0 con 0x559ddf5056b0 2024-04-25T10:44:33.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.208+0000 7fb2443d0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559ddf7b3dc0 con 0x559ddf4fd4d0 2024-04-25T10:44:33.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.208+0000 7fb2443d0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559ddf8d9320 con 0x559ddf526390 2024-04-25T10:44:33.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.208+0000 7fb2443d0740 10 client.0 _ll_register_callbacks cb 0x559ddf7cf600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:33.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.208+0000 7fb236ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 0x559ddf8d6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.208+0000 7fb236ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 0x559ddf8d6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43274/0 (socket says 172.21.15.43:43274) 2024-04-25T10:44:33.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.208+0000 7fb236ffd640 1 -- 172.21.15.43:0/3888698494 learned_addr learned my addr 172.21.15.43:0/3888698494 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:33.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.208+0000 7fb2367fc640 1 --2- 172.21.15.43:0/3888698494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddf4fd4d0 0x559ddf8d6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.208+0000 7fb2367fc640 1 --2- 172.21.15.43:0/3888698494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ddf526390 0x559ddf8e26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.208+0000 7fb236ffd640 1 -- 172.21.15.43:0/3888698494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ddf526390 msgr2=0x559ddf8e26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.208+0000 7fb236ffd640 1 --2- 172.21.15.43:0/3888698494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ddf526390 0x559ddf8e26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.208+0000 7fb236ffd640 1 -- 172.21.15.43:0/3888698494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddf4fd4d0 msgr2=0x559ddf8d6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.208+0000 7fb236ffd640 1 --2- 172.21.15.43:0/3888698494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddf4fd4d0 0x559ddf8d6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.208+0000 7fb236ffd640 1 -- 172.21.15.43:0/3888698494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ddf53e3f0 con 0x559ddf5056b0 2024-04-25T10:44:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.209+0000 7fb236ffd640 1 --2- 172.21.15.43:0/3888698494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 0x559ddf8d6b90 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fb228005940 tx=0x7fb2280316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b4dcd35d4bded5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.209+0000 7fb21aff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.209+0000 7fb21aff5640 1 -- 172.21.15.43:0/3888698494 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb228035040 con 0x559ddf5056b0 2024-04-25T10:44:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.209+0000 7fb21aff5640 1 -- 172.21.15.43:0/3888698494 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fb228004040 con 0x559ddf5056b0 2024-04-25T10:44:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.209+0000 7fb2443d0740 1 -- 172.21.15.43:0/3888698494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb22c017070 con 0x559ddf5056b0 2024-04-25T10:44:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.209+0000 7fb21aff5640 1 -- 172.21.15.43:0/3888698494 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb228004240 con 0x559ddf5056b0 2024-04-25T10:44:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.209+0000 7fb2443d0740 10 client.19038 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.209+0000 7fb2443d0740 1 -- 172.21.15.43:0/3888698494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559ddf6087d0 con 0x559ddf5056b0 2024-04-25T10:44:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.209+0000 7fb21aff5640 1 -- 172.21.15.43:0/3888698494 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb22c017070 con 0x559ddf5056b0 2024-04-25T10:44:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.209+0000 7fb21aff5640 1 -- 172.21.15.43:0/3888698494 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb228004a60 con 0x559ddf5056b0 2024-04-25T10:44:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.209+0000 7fb2443d0740 10 client.19038 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.209+0000 7fb2443d0740 10 client.19038 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:44:33.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.209+0000 7fb2443d0740 20 client.19038 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:33.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.209+0000 7fb2443d0740 1 --2- 172.21.15.43:0/3888698494 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x559ddf4e4f60 0x559ddf8e8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.209+0000 7fb2443d0740 4 client.19038 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:44:33.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.209+0000 7fb2443d0740 1 -- 172.21.15.43:0/3888698494 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ddf83a990 con 0x559ddf4e4f60 2024-04-25T10:44:33.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.210+0000 7fb2367fc640 1 --2- 172.21.15.43:0/3888698494 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x559ddf4e4f60 0x559ddf8e8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.210+0000 7fb2367fc640 1 --2- 172.21.15.43:0/3888698494 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x559ddf4e4f60 0x559ddf8e8a70 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=778d884c7baa4e7d server_cookie=22695d185d18d788 in_seq=0 out_seq=0 2024-04-25T10:44:33.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.210+0000 7fb21aff5640 10 client.19038 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:44:33.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.211+0000 7fb21aff5640 1 -- 172.21.15.43:0/3888698494 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559ddf83a990 con 0x559ddf4e4f60 2024-04-25T10:44:33.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.211+0000 7fb21aff5640 10 client.19038 handle_command_reply: tid=0 2024-04-25T10:44:33.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.211+0000 7fb2443d0740 1 client.19038 shutdown 2024-04-25T10:44:33.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.211+0000 7fb2443d0740 1 -- 172.21.15.43:0/3888698494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 msgr2=0x559ddf8d6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.211+0000 7fb2443d0740 1 --2- 172.21.15.43:0/3888698494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 0x559ddf8d6b90 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fb228005940 tx=0x7fb2280316d0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.211+0000 7fb2443d0740 1 -- 172.21.15.43:0/3888698494 shutdown_connections 2024-04-25T10:44:33.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.211+0000 7fb2443d0740 1 --2- 172.21.15.43:0/3888698494 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x559ddf4e4f60 0x559ddf8e8a70 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.211+0000 7fb2443d0740 1 --2- 172.21.15.43:0/3888698494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ddf526390 0x559ddf8e26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.211+0000 7fb2443d0740 1 --2- 172.21.15.43:0/3888698494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 0x559ddf8d6b90 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.211+0000 7fb21aff5640 0 client.19038 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:44:33.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.211+0000 7fb2443d0740 1 --2- 172.21.15.43:0/3888698494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddf4fd4d0 0x559ddf8d6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.211+0000 7fb2443d0740 1 -- 172.21.15.43:0/3888698494 >> 172.21.15.43:0/3888698494 conn(0x559ddf51a4e0 msgr2=0x559ddf8c93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:33.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.211+0000 7fb2443d0740 1 -- 172.21.15.43:0/3888698494 shutdown_connections 2024-04-25T10:44:33.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.211+0000 7fb2443d0740 1 -- 172.21.15.43:0/3888698494 wait complete. 2024-04-25T10:44:33.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.211+0000 7fb2443d0740 20 client.19038 trim_cache size 0 max 16384 2024-04-25T10:44:33.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.225+0000 7fb2443d0740 1 Processor -- start 2024-04-25T10:44:33.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.225+0000 7fb2443d0740 1 -- start start 2024-04-25T10:44:33.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.225+0000 7fb2443d0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ddf4e4f60 0x559ddf8c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.225+0000 7fb2443d0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddf4fd4d0 0x559ddf8d6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.225+0000 7fb2443d0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 0x559ddf8c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.225+0000 7fb2443d0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb2381b0090 con 0x559ddf5056b0 2024-04-25T10:44:33.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.225+0000 7fb2443d0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb2381ad960 con 0x559ddf4fd4d0 2024-04-25T10:44:33.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.225+0000 7fb2443d0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb2381ab7d0 con 0x559ddf4e4f60 2024-04-25T10:44:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.225+0000 7fb2367fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 0x559ddf8c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.225+0000 7fb2367fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 0x559ddf8c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43288/0 (socket says 172.21.15.43:43288) 2024-04-25T10:44:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.225+0000 7fb2367fc640 1 -- 172.21.15.43:0/2553300581 learned_addr learned my addr 172.21.15.43:0/2553300581 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.226+0000 7fb2367fc640 1 -- 172.21.15.43:0/2553300581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ddf4e4f60 msgr2=0x559ddf8c3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.226+0000 7fb2367fc640 1 --2- 172.21.15.43:0/2553300581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ddf4e4f60 0x559ddf8c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.226+0000 7fb2367fc640 1 -- 172.21.15.43:0/2553300581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddf4fd4d0 msgr2=0x559ddf8d6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.226+0000 7fb236ffd640 1 --2- 172.21.15.43:0/2553300581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddf4fd4d0 0x559ddf8d6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.226+0000 7fb2367fc640 1 --2- 172.21.15.43:0/2553300581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddf4fd4d0 0x559ddf8d6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.226+0000 7fb2367fc640 1 -- 172.21.15.43:0/2553300581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ddf8d5740 con 0x559ddf5056b0 2024-04-25T10:44:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.226+0000 7fb2367fc640 1 --2- 172.21.15.43:0/2553300581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 0x559ddf8c9b00 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fb22c009250 tx=0x7fb22c003a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20c097b8058aa34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.226+0000 7fb21cff9640 1 -- 172.21.15.43:0/2553300581 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c003b50 con 0x559ddf5056b0 2024-04-25T10:44:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.226+0000 7fb21cff9640 1 -- 172.21.15.43:0/2553300581 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb22c017070 con 0x559ddf5056b0 2024-04-25T10:44:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.226+0000 7fb21cff9640 1 -- 172.21.15.43:0/2553300581 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c02b690 con 0x559ddf5056b0 2024-04-25T10:44:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.227+0000 7fb2443d0740 1 -- 172.21.15.43:0/2553300581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 msgr2=0x559ddf8c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.227+0000 7fb2443d0740 1 --2- 172.21.15.43:0/2553300581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 0x559ddf8c9b00 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fb22c009250 tx=0x7fb22c003a20 comp rx=0 tx=0).stop 2024-04-25T10:44:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.227+0000 7fb2443d0740 1 -- 172.21.15.43:0/2553300581 shutdown_connections 2024-04-25T10:44:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.227+0000 7fb2443d0740 1 --2- 172.21.15.43:0/2553300581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddf4fd4d0 0x559ddf8d6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.227+0000 7fb2443d0740 1 --2- 172.21.15.43:0/2553300581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 0x559ddf8c9b00 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.227+0000 7fb2443d0740 1 --2- 172.21.15.43:0/2553300581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ddf4e4f60 0x559ddf8c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.227+0000 7fb2443d0740 1 -- 172.21.15.43:0/2553300581 >> 172.21.15.43:0/2553300581 conn(0x559ddf51a4e0 msgr2=0x559ddf8c93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.227+0000 7fb2443d0740 1 -- 172.21.15.43:0/2553300581 shutdown_connections 2024-04-25T10:44:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.227+0000 7fb2443d0740 1 -- 172.21.15.43:0/2553300581 wait complete. 2024-04-25T10:44:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.227+0000 7fb2443d0740 1 -- start start 2024-04-25T10:44:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.227+0000 7fb2443d0740 1 Processor -- start 2024-04-25T10:44:33.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.228+0000 7fb2443d0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddf4e4f60 0x559ddf8de5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.228+0000 7fb2443d0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ddf4fd4d0 0x559ddf8deae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.228+0000 7fb2443d0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 0x559ddf8d9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.228+0000 7fb2443d0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb2381abe20 con 0x559ddf5056b0 2024-04-25T10:44:33.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.228+0000 7fb2443d0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb2381ad290 con 0x559ddf4e4f60 2024-04-25T10:44:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.228+0000 7fb2443d0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb2381ac4f0 con 0x559ddf4fd4d0 2024-04-25T10:44:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.228+0000 7fb2443d0740 10 client.0 _ll_register_callbacks cb 0x7fb22c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.228+0000 7fb2367fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 0x559ddf8d9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.228+0000 7fb2367fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 0x559ddf8d9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43294/0 (socket says 172.21.15.43:43294) 2024-04-25T10:44:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.228+0000 7fb2367fc640 1 -- 172.21.15.43:0/3411913487 learned_addr learned my addr 172.21.15.43:0/3411913487 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.228+0000 7fb2367fc640 1 --2- 172.21.15.43:0/3411913487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddf4e4f60 0x559ddf8de5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.228+0000 7fb236ffd640 1 --2- 172.21.15.43:0/3411913487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ddf4fd4d0 0x559ddf8deae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.228+0000 7fb2367fc640 1 -- 172.21.15.43:0/3411913487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ddf4fd4d0 msgr2=0x559ddf8deae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.228+0000 7fb2367fc640 1 --2- 172.21.15.43:0/3411913487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ddf4fd4d0 0x559ddf8deae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.228+0000 7fb2367fc640 1 -- 172.21.15.43:0/3411913487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddf4e4f60 msgr2=0x559ddf8de5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.228+0000 7fb2367fc640 1 --2- 172.21.15.43:0/3411913487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddf4e4f60 0x559ddf8de5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.228+0000 7fb2367fc640 1 -- 172.21.15.43:0/3411913487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ddf8da100 con 0x559ddf5056b0 2024-04-25T10:44:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.228+0000 7fb236ffd640 1 --2- 172.21.15.43:0/3411913487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ddf4fd4d0 0x559ddf8deae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.228+0000 7fb2367fc640 1 --2- 172.21.15.43:0/3411913487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 0x559ddf8d9a10 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x559ddf8df270 tx=0x7fb22c0098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00403c3cf19d8ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.228+0000 7fb21aff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.229+0000 7fb21aff5640 1 -- 172.21.15.43:0/3411913487 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c005270 con 0x559ddf5056b0 2024-04-25T10:44:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.229+0000 7fb21aff5640 1 -- 172.21.15.43:0/3411913487 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fb22c005410 con 0x559ddf5056b0 2024-04-25T10:44:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.229+0000 7fb2443d0740 1 -- 172.21.15.43:0/3411913487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb22c017070 con 0x559ddf5056b0 2024-04-25T10:44:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.229+0000 7fb21aff5640 1 -- 172.21.15.43:0/3411913487 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb22c005610 con 0x559ddf5056b0 2024-04-25T10:44:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.229+0000 7fb2443d0740 10 client.19044 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.229+0000 7fb2443d0740 1 -- 172.21.15.43:0/3411913487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559ddf9506e0 con 0x559ddf5056b0 2024-04-25T10:44:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.229+0000 7fb21aff5640 1 -- 172.21.15.43:0/3411913487 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb22c017070 con 0x559ddf5056b0 2024-04-25T10:44:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.229+0000 7fb21aff5640 1 -- 172.21.15.43:0/3411913487 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb22c010430 con 0x559ddf5056b0 2024-04-25T10:44:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.229+0000 7fb2443d0740 10 client.19044 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.229+0000 7fb2443d0740 10 client.19044 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T10:44:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.229+0000 7fb2443d0740 20 client.19044 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.229+0000 7fb2443d0740 1 --2- 172.21.15.43:0/3411913487 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x559ddf526390 0x559ddf952c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.229+0000 7fb2443d0740 4 client.19044 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:44:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.229+0000 7fb2443d0740 1 -- 172.21.15.43:0/3411913487 --> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ddf834800 con 0x559ddf526390 2024-04-25T10:44:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.229+0000 7fb236ffd640 1 --2- 172.21.15.43:0/3411913487 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x559ddf526390 0x559ddf952c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.230+0000 7fb21aff5640 10 client.19044 ms_handle_connect on v2:172.21.15.164:6838/4082280107 2024-04-25T10:44:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.230+0000 7fb236ffd640 1 --2- 172.21.15.43:0/3411913487 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x559ddf526390 0x559ddf952c00 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d39a94d744df888 server_cookie=fcbc1d17514744ac in_seq=0 out_seq=0 2024-04-25T10:44:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.230+0000 7fb21aff5640 1 -- 172.21.15.43:0/3411913487 <== mds.1 v2:172.21.15.164:6838/4082280107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ddf834800 con 0x559ddf526390 2024-04-25T10:44:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.230+0000 7fb21aff5640 10 client.19044 handle_command_reply: tid=0 2024-04-25T10:44:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.230+0000 7fb2443d0740 1 client.19044 shutdown 2024-04-25T10:44:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.230+0000 7fb2443d0740 1 -- 172.21.15.43:0/3411913487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 msgr2=0x559ddf8d9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.230+0000 7fb2443d0740 1 --2- 172.21.15.43:0/3411913487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 0x559ddf8d9a10 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x559ddf8df270 tx=0x7fb22c0098f0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.230+0000 7fb2443d0740 1 -- 172.21.15.43:0/3411913487 shutdown_connections 2024-04-25T10:44:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.230+0000 7fb2443d0740 1 --2- 172.21.15.43:0/3411913487 >> [v2:172.21.15.164:6838/4082280107,v1:172.21.15.164:6839/4082280107] conn(0x559ddf526390 0x559ddf952c00 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.230+0000 7fb2443d0740 1 --2- 172.21.15.43:0/3411913487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ddf5056b0 0x559ddf8d9a10 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.230+0000 7fb21aff5640 0 client.19044 ms_handle_reset on v2:172.21.15.164:6838/4082280107 2024-04-25T10:44:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.230+0000 7fb2443d0740 1 --2- 172.21.15.43:0/3411913487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ddf4fd4d0 0x559ddf8deae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.230+0000 7fb2443d0740 1 --2- 172.21.15.43:0/3411913487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddf4e4f60 0x559ddf8de5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.230+0000 7fb2443d0740 1 -- 172.21.15.43:0/3411913487 >> 172.21.15.43:0/3411913487 conn(0x559ddf51a4e0 msgr2=0x559ddf8c2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:33.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.230+0000 7fb2443d0740 1 -- 172.21.15.43:0/3411913487 shutdown_connections 2024-04-25T10:44:33.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.230+0000 7fb2443d0740 1 -- 172.21.15.43:0/3411913487 wait complete. 2024-04-25T10:44:33.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.231+0000 7fb2443d0740 20 client.19044 trim_cache size 0 max 16384 2024-04-25T10:44:33.239 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:44:33.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.231+0000 7fb23d51f640 1 -- 172.21.15.43:0/1015752133 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb2200afc60 msgr2=0x7fb2200b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.231+0000 7fb23d51f640 1 --2- 172.21.15.43:0/1015752133 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb2200afc60 0x7fb2200b2170 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb2300a4940 tx=0x7fb230076040 comp rx=0 tx=0).stop 2024-04-25T10:44:33.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.231+0000 7fb23d51f640 1 -- 172.21.15.43:0/1015752133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb238109fd0 msgr2=0x7fb23819efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.231+0000 7fb23d51f640 1 --2- 172.21.15.43:0/1015752133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb238109fd0 0x7fb23819efa0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fb23000efd0 tx=0x7fb23000c6a0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.231+0000 7fb23d51f640 1 -- 172.21.15.43:0/1015752133 shutdown_connections 2024-04-25T10:44:33.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.231+0000 7fb23d51f640 1 --2- 172.21.15.43:0/1015752133 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb2200afc60 0x7fb2200b2170 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.231+0000 7fb23d51f640 1 --2- 172.21.15.43:0/1015752133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb238109b30 0x7fb23819ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.231+0000 7fb23d51f640 1 --2- 172.21.15.43:0/1015752133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb238109fd0 0x7fb23819efa0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.231+0000 7fb23d51f640 1 --2- 172.21.15.43:0/1015752133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2381042d0 0x7fb23819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.231+0000 7fb23d51f640 1 -- 172.21.15.43:0/1015752133 >> 172.21.15.43:0/1015752133 conn(0x7fb238079280 msgr2=0x7fb2381051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:33.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.231+0000 7fb23d51f640 1 -- 172.21.15.43:0/1015752133 shutdown_connections 2024-04-25T10:44:33.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.231+0000 7fb23d51f640 1 -- 172.21.15.43:0/1015752133 wait complete. 2024-04-25T10:44:33.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:33 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1167012375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:44:33.244 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T10:44:33.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:33 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1167012375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:44:33.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:33 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1167012375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:44:33.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.355+0000 7f763fbf3640 1 -- 172.21.15.43:0/517356027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7638109fb0 msgr2=0x7f7638103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.355+0000 7f763fbf3640 1 --2- 172.21.15.43:0/517356027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7638109fb0 0x7f7638103bf0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f7628009900 tx=0x7f762802a410 comp rx=0 tx=0).stop 2024-04-25T10:44:33.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.355+0000 7f763fbf3640 1 -- 172.21.15.43:0/517356027 shutdown_connections 2024-04-25T10:44:33.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.356+0000 7f763fbf3640 1 --2- 172.21.15.43:0/517356027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7638104260 0x7f7638110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.356+0000 7f763fbf3640 1 --2- 172.21.15.43:0/517356027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7638109fb0 0x7f7638103bf0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.356+0000 7f763fbf3640 1 --2- 172.21.15.43:0/517356027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7638109b10 0x7f76381036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.356+0000 7f763fbf3640 1 -- 172.21.15.43:0/517356027 >> 172.21.15.43:0/517356027 conn(0x7f76380ff1d0 msgr2=0x7f7638101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:33.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.356+0000 7f763fbf3640 1 -- 172.21.15.43:0/517356027 shutdown_connections 2024-04-25T10:44:33.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.356+0000 7f763fbf3640 1 -- 172.21.15.43:0/517356027 wait complete. 2024-04-25T10:44:33.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.356+0000 7f763fbf3640 1 Processor -- start 2024-04-25T10:44:33.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.356+0000 7f763fbf3640 1 -- start start 2024-04-25T10:44:33.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f763fbf3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7638104260 0x7f763819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f763fbf3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7638109b10 0x7f763819ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f763fbf3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7638109fb0 0x7f763819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f763fbf3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7638113d00 con 0x7f7638104260 2024-04-25T10:44:33.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f763fbf3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7638113b80 con 0x7f7638109b10 2024-04-25T10:44:33.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f763fbf3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7638113e80 con 0x7f7638109fb0 2024-04-25T10:44:33.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f763d98f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7638104260 0x7f763819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f763d98f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7638104260 0x7f763819a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43316/0 (socket says 172.21.15.43:43316) 2024-04-25T10:44:33.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f763d98f640 1 -- 172.21.15.43:0/2835572802 learned_addr learned my addr 172.21.15.43:0/2835572802 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:33.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f763d18e640 1 --2- 172.21.15.43:0/2835572802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7638109b10 0x7f763819ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f763e190640 1 --2- 172.21.15.43:0/2835572802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7638109fb0 0x7f763819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f763d98f640 1 -- 172.21.15.43:0/2835572802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7638109fb0 msgr2=0x7f763819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f763d98f640 1 --2- 172.21.15.43:0/2835572802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7638109fb0 0x7f763819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f763d98f640 1 -- 172.21.15.43:0/2835572802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7638109b10 msgr2=0x7f763819ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f763d98f640 1 --2- 172.21.15.43:0/2835572802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7638109b10 0x7f763819ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f763d98f640 1 -- 172.21.15.43:0/2835572802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763819f560 con 0x7f7638104260 2024-04-25T10:44:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f763e190640 1 --2- 172.21.15.43:0/2835572802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7638109fb0 0x7f763819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:44:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f763d18e640 1 --2- 172.21.15.43:0/2835572802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7638109b10 0x7f763819ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f763d98f640 1 --2- 172.21.15.43:0/2835572802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7638104260 0x7f763819a780 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f763400e9f0 tx=0x7f763400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f762effd640 1 -- 172.21.15.43:0/2835572802 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763400cd80 con 0x7f7638104260 2024-04-25T10:44:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.357+0000 7f762effd640 1 -- 172.21.15.43:0/2835572802 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76340054d0 con 0x7f7638104260 2024-04-25T10:44:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.358+0000 7f763fbf3640 1 -- 172.21.15.43:0/2835572802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f763819f850 con 0x7f7638104260 2024-04-25T10:44:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.358+0000 7f762effd640 1 -- 172.21.15.43:0/2835572802 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634010490 con 0x7f7638104260 2024-04-25T10:44:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.358+0000 7f763fbf3640 1 -- 172.21.15.43:0/2835572802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f763819fd70 con 0x7f7638104260 2024-04-25T10:44:33.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.361+0000 7f7646b82740 1 Processor -- start 2024-04-25T10:44:33.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.361+0000 7f7646b82740 1 -- start start 2024-04-25T10:44:33.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.361+0000 7f762effd640 1 -- 172.21.15.43:0/2835572802 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f7634005000 con 0x7f7638104260 2024-04-25T10:44:33.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.361+0000 7f7646b82740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e481390 0x55763e822810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.361+0000 7f7646b82740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55763e4584d0 0x55763e822d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.361+0000 7f7646b82740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55763e4606b0 0x55763e82f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.361+0000 7f7646b82740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55763e787d80 con 0x55763e481390 2024-04-25T10:44:33.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.361+0000 7f7646b82740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55763e741a20 con 0x55763e4606b0 2024-04-25T10:44:33.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.361+0000 7f7646b82740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55763e783d40 con 0x55763e4584d0 2024-04-25T10:44:33.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.362+0000 7f763e190640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e481390 0x55763e822810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.362+0000 7f763e190640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e481390 0x55763e822810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43330/0 (socket says 172.21.15.43:43330) 2024-04-25T10:44:33.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.362+0000 7f763e190640 1 -- 172.21.15.43:0/749364393 learned_addr learned my addr 172.21.15.43:0/749364393 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:33.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.362+0000 7f763d98f640 1 --2- 172.21.15.43:0/749364393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55763e4584d0 0x55763e822d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.362+0000 7f763d18e640 1 --2- 172.21.15.43:0/749364393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55763e4606b0 0x55763e82f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.362+0000 7f763e190640 1 -- 172.21.15.43:0/749364393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55763e4584d0 msgr2=0x55763e822d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.362+0000 7f763e190640 1 --2- 172.21.15.43:0/749364393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55763e4584d0 0x55763e822d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.362+0000 7f763e190640 1 -- 172.21.15.43:0/749364393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55763e4606b0 msgr2=0x55763e82f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.362+0000 7f763e190640 1 --2- 172.21.15.43:0/749364393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55763e4606b0 0x55763e82f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.362+0000 7f763e190640 1 -- 172.21.15.43:0/749364393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55763e74cc30 con 0x55763e481390 2024-04-25T10:44:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.362+0000 7f763e190640 1 --2- 172.21.15.43:0/749364393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e481390 0x55763e822810 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f7630008e60 tx=0x7f763000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec262d4a4c39b9c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.362+0000 7f762effd640 1 --2- 172.21.15.43:0/2835572802 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f76240afc60 0x7f76240b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.362+0000 7f763d98f640 1 --2- 172.21.15.43:0/2835572802 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f76240afc60 0x7f76240b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.362+0000 7f7623fff640 1 -- 172.21.15.43:0/749364393 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763000f830 con 0x55763e481390 2024-04-25T10:44:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.362+0000 7f7623fff640 1 -- 172.21.15.43:0/749364393 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7630018480 con 0x55763e481390 2024-04-25T10:44:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.362+0000 7f7623fff640 1 -- 172.21.15.43:0/749364393 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763000e5d0 con 0x55763e481390 2024-04-25T10:44:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.363+0000 7f762effd640 1 -- 172.21.15.43:0/2835572802 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f76340df490 con 0x7f7638104260 2024-04-25T10:44:33.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.363+0000 7f7646b82740 1 -- 172.21.15.43:0/749364393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e481390 msgr2=0x55763e822810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.363+0000 7f763d98f640 1 --2- 172.21.15.43:0/2835572802 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f76240afc60 0x7f76240b2170 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f76340dcc90 tx=0x7f76340e5060 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:33.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.363+0000 7f7646b82740 1 --2- 172.21.15.43:0/749364393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e481390 0x55763e822810 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f7630008e60 tx=0x7f763000d710 comp rx=0 tx=0).stop 2024-04-25T10:44:33.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.363+0000 7f7646b82740 1 -- 172.21.15.43:0/749364393 shutdown_connections 2024-04-25T10:44:33.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.363+0000 7f7646b82740 1 --2- 172.21.15.43:0/749364393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55763e4606b0 0x55763e82f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.363+0000 7f7646b82740 1 --2- 172.21.15.43:0/749364393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55763e4584d0 0x55763e822d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.363+0000 7f7646b82740 1 --2- 172.21.15.43:0/749364393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e481390 0x55763e822810 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.363+0000 7f7646b82740 1 -- 172.21.15.43:0/749364393 >> 172.21.15.43:0/749364393 conn(0x55763e4754e0 msgr2=0x55763e81efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.364+0000 7f7646b82740 1 -- 172.21.15.43:0/749364393 shutdown_connections 2024-04-25T10:44:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.364+0000 7f7646b82740 1 -- 172.21.15.43:0/749364393 wait complete. 2024-04-25T10:44:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.364+0000 7f7646b82740 1 -- start start 2024-04-25T10:44:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.364+0000 7f7646b82740 1 Processor -- start 2024-04-25T10:44:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.364+0000 7f7646b82740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55763e4584d0 0x55763e82d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.364+0000 7f7646b82740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e4606b0 0x55763e82d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.364+0000 7f7646b82740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55763e481390 0x55763e839210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.364+0000 7f7646b82740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55763e82fd90 con 0x55763e4606b0 2024-04-25T10:44:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.364+0000 7f7646b82740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55763e70edc0 con 0x55763e4584d0 2024-04-25T10:44:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.364+0000 7f7646b82740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55763e82ff10 con 0x55763e481390 2024-04-25T10:44:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.365+0000 7f7646b82740 10 client.0 _ll_register_callbacks cb 0x55763e72a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.365+0000 7f763d18e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e4606b0 0x55763e82d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.365+0000 7f763d18e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e4606b0 0x55763e82d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43332/0 (socket says 172.21.15.43:43332) 2024-04-25T10:44:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.365+0000 7f763d18e640 1 -- 172.21.15.43:0/3846962079 learned_addr learned my addr 172.21.15.43:0/3846962079 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.365+0000 7f763e190640 1 --2- 172.21.15.43:0/3846962079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55763e481390 0x55763e839210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.365+0000 7f763e190640 1 --2- 172.21.15.43:0/3846962079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55763e4584d0 0x55763e82d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.365+0000 7f763d18e640 1 -- 172.21.15.43:0/3846962079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55763e481390 msgr2=0x55763e839210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.365+0000 7f763d18e640 1 --2- 172.21.15.43:0/3846962079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55763e481390 0x55763e839210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.365+0000 7f763d18e640 1 -- 172.21.15.43:0/3846962079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55763e4584d0 msgr2=0x55763e82d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.365+0000 7f763d18e640 1 --2- 172.21.15.43:0/3846962079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55763e4584d0 0x55763e82d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.365+0000 7f763d18e640 1 -- 172.21.15.43:0/3846962079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55763e4993f0 con 0x55763e4606b0 2024-04-25T10:44:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.365+0000 7f763e190640 1 --2- 172.21.15.43:0/3846962079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55763e4584d0 0x55763e82d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.365+0000 7f763d18e640 1 --2- 172.21.15.43:0/3846962079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e4606b0 0x55763e82d790 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f76280096c0 tx=0x7f7628031810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedd10404b3b8bb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.365+0000 7f76217fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.365+0000 7f76217fa640 1 -- 172.21.15.43:0/3846962079 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7628035040 con 0x55763e4606b0 2024-04-25T10:44:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.365+0000 7f76217fa640 1 -- 172.21.15.43:0/3846962079 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f7628004240 con 0x55763e4606b0 2024-04-25T10:44:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.365+0000 7f7646b82740 1 -- 172.21.15.43:0/3846962079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7630018480 con 0x55763e4606b0 2024-04-25T10:44:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.365+0000 7f76217fa640 1 -- 172.21.15.43:0/3846962079 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7628004440 con 0x55763e4606b0 2024-04-25T10:44:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.365+0000 7f7646b82740 10 client.19056 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.365+0000 7f7646b82740 1 -- 172.21.15.43:0/3846962079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55763e5637d0 con 0x55763e4606b0 2024-04-25T10:44:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.366+0000 7f76217fa640 1 -- 172.21.15.43:0/3846962079 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7630018480 con 0x55763e4606b0 2024-04-25T10:44:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.366+0000 7f76217fa640 1 -- 172.21.15.43:0/3846962079 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7628004b70 con 0x55763e4606b0 2024-04-25T10:44:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.366+0000 7f7646b82740 10 client.19056 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.366+0000 7f7646b82740 10 client.19056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:44:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.366+0000 7f7646b82740 20 client.19056 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.366+0000 7f7646b82740 1 --2- 172.21.15.43:0/3846962079 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55763e43ff60 0x55763e83f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.366+0000 7f7646b82740 4 client.19056 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:44:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.366+0000 7f7646b82740 1 -- 172.21.15.43:0/3846962079 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55763e795990 con 0x55763e43ff60 2024-04-25T10:44:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.366+0000 7f763e190640 1 --2- 172.21.15.43:0/3846962079 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55763e43ff60 0x55763e83f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.367+0000 7f76217fa640 10 client.19056 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.367+0000 7f763e190640 1 --2- 172.21.15.43:0/3846962079 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55763e43ff60 0x55763e83f610 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f858f5dccf8af9c3 server_cookie=fbc95543ced84610 in_seq=0 out_seq=0 2024-04-25T10:44:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.367+0000 7f76217fa640 1 -- 172.21.15.43:0/3846962079 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55763e795990 con 0x55763e43ff60 2024-04-25T10:44:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.367+0000 7f76217fa640 10 client.19056 handle_command_reply: tid=0 2024-04-25T10:44:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.367+0000 7f7646b82740 1 client.19056 shutdown 2024-04-25T10:44:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.368+0000 7f7646b82740 1 -- 172.21.15.43:0/3846962079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e4606b0 msgr2=0x55763e82d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.368+0000 7f7646b82740 1 --2- 172.21.15.43:0/3846962079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e4606b0 0x55763e82d790 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f76280096c0 tx=0x7f7628031810 comp rx=0 tx=0).stop 2024-04-25T10:44:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.368+0000 7f7646b82740 1 -- 172.21.15.43:0/3846962079 shutdown_connections 2024-04-25T10:44:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.368+0000 7f7646b82740 1 --2- 172.21.15.43:0/3846962079 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55763e43ff60 0x55763e83f610 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.368+0000 7f7646b82740 1 --2- 172.21.15.43:0/3846962079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55763e481390 0x55763e839210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.368+0000 7f7646b82740 1 --2- 172.21.15.43:0/3846962079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e4606b0 0x55763e82d790 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.368+0000 7f7646b82740 1 --2- 172.21.15.43:0/3846962079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55763e4584d0 0x55763e82d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.368+0000 7f76217fa640 0 client.19056 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.368+0000 7f7646b82740 1 -- 172.21.15.43:0/3846962079 >> 172.21.15.43:0/3846962079 conn(0x55763e4754e0 msgr2=0x55763e824530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.368+0000 7f7646b82740 1 -- 172.21.15.43:0/3846962079 shutdown_connections 2024-04-25T10:44:33.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.368+0000 7f7646b82740 1 -- 172.21.15.43:0/3846962079 wait complete. 2024-04-25T10:44:33.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.368+0000 7f7646b82740 20 client.19056 trim_cache size 0 max 16384 2024-04-25T10:44:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.381+0000 7f7646b82740 1 Processor -- start 2024-04-25T10:44:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.382+0000 7f7646b82740 1 -- start start 2024-04-25T10:44:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.382+0000 7f7646b82740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55763e4584d0 0x55763e82c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.382+0000 7f7646b82740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55763e4606b0 0x55763e838e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.382+0000 7f7646b82740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e481390 0x55763e827430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.382+0000 7f7646b82740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f76381b2290 con 0x55763e481390 2024-04-25T10:44:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.382+0000 7f7646b82740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f76381afb60 con 0x55763e4584d0 2024-04-25T10:44:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.382+0000 7f7646b82740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f76381ad9d0 con 0x55763e4606b0 2024-04-25T10:44:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.382+0000 7f763e190640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e481390 0x55763e827430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.382+0000 7f763e190640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e481390 0x55763e827430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43336/0 (socket says 172.21.15.43:43336) 2024-04-25T10:44:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.382+0000 7f763e190640 1 -- 172.21.15.43:0/4097514553 learned_addr learned my addr 172.21.15.43:0/4097514553 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.382+0000 7f763d18e640 1 --2- 172.21.15.43:0/4097514553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55763e4606b0 0x55763e838e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.382+0000 7f763e190640 1 -- 172.21.15.43:0/4097514553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55763e4606b0 msgr2=0x55763e838e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.382+0000 7f763e190640 1 --2- 172.21.15.43:0/4097514553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55763e4606b0 0x55763e838e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.382+0000 7f763e190640 1 -- 172.21.15.43:0/4097514553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55763e4584d0 msgr2=0x55763e82c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.382+0000 7f763e190640 1 --2- 172.21.15.43:0/4097514553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55763e4584d0 0x55763e82c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.382+0000 7f763e190640 1 -- 172.21.15.43:0/4097514553 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55763e81fb90 con 0x55763e481390 2024-04-25T10:44:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.382+0000 7f763e190640 1 --2- 172.21.15.43:0/4097514553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e481390 0x55763e827430 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f7630018e30 tx=0x7f7630008730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1d94fa03c9bf3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.383+0000 7f76237fe640 1 -- 172.21.15.43:0/4097514553 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763002e070 con 0x55763e481390 2024-04-25T10:44:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.383+0000 7f76237fe640 1 -- 172.21.15.43:0/4097514553 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7630008580 con 0x55763e481390 2024-04-25T10:44:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.383+0000 7f76237fe640 1 -- 172.21.15.43:0/4097514553 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630021040 con 0x55763e481390 2024-04-25T10:44:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.383+0000 7f7646b82740 1 -- 172.21.15.43:0/4097514553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e481390 msgr2=0x55763e827430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.383+0000 7f7646b82740 1 --2- 172.21.15.43:0/4097514553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e481390 0x55763e827430 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f7630018e30 tx=0x7f7630008730 comp rx=0 tx=0).stop 2024-04-25T10:44:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.383+0000 7f7646b82740 1 -- 172.21.15.43:0/4097514553 shutdown_connections 2024-04-25T10:44:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.383+0000 7f7646b82740 1 --2- 172.21.15.43:0/4097514553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e481390 0x55763e827430 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.383+0000 7f7646b82740 1 --2- 172.21.15.43:0/4097514553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55763e4606b0 0x55763e838e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.383+0000 7f7646b82740 1 --2- 172.21.15.43:0/4097514553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55763e4584d0 0x55763e82c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.383+0000 7f7646b82740 1 -- 172.21.15.43:0/4097514553 >> 172.21.15.43:0/4097514553 conn(0x55763e4754e0 msgr2=0x55763e82b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.383+0000 7f7646b82740 1 -- 172.21.15.43:0/4097514553 shutdown_connections 2024-04-25T10:44:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.383+0000 7f7646b82740 1 -- 172.21.15.43:0/4097514553 wait complete. 2024-04-25T10:44:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.384+0000 7f7646b82740 1 -- start start 2024-04-25T10:44:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.384+0000 7f7646b82740 1 Processor -- start 2024-04-25T10:44:33.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.384+0000 7f7646b82740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e4584d0 0x55763e835710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.384+0000 7f7646b82740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55763e4606b0 0x55763e830ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.384+0000 7f7646b82740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55763e481390 0x55763e830ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.384+0000 7f7646b82740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f76381ae020 con 0x55763e4584d0 2024-04-25T10:44:33.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.384+0000 7f7646b82740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f76381af490 con 0x55763e4606b0 2024-04-25T10:44:33.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.384+0000 7f7646b82740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f76381ae6f0 con 0x55763e481390 2024-04-25T10:44:33.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.384+0000 7f7646b82740 10 client.0 _ll_register_callbacks cb 0x7f7628009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:44:33.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.384+0000 7f763e190640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e4584d0 0x55763e835710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.384+0000 7f763e190640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e4584d0 0x55763e835710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43342/0 (socket says 172.21.15.43:43342) 2024-04-25T10:44:33.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.384+0000 7f763e190640 1 -- 172.21.15.43:0/1131510203 learned_addr learned my addr 172.21.15.43:0/1131510203 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:33.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.385+0000 7f763d18e640 1 --2- 172.21.15.43:0/1131510203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55763e4606b0 0x55763e830ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.385+0000 7f763e190640 1 --2- 172.21.15.43:0/1131510203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55763e481390 0x55763e830ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.385+0000 7f763e190640 1 -- 172.21.15.43:0/1131510203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55763e481390 msgr2=0x55763e830ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.385+0000 7f763e190640 1 --2- 172.21.15.43:0/1131510203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55763e481390 0x55763e830ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.385+0000 7f763e190640 1 -- 172.21.15.43:0/1131510203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55763e4606b0 msgr2=0x55763e830ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:44:33.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.385+0000 7f763e190640 1 --2- 172.21.15.43:0/1131510203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55763e4606b0 0x55763e830ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.385+0000 7f763e190640 1 -- 172.21.15.43:0/1131510203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55763e8316e0 con 0x55763e4584d0 2024-04-25T10:44:33.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.385+0000 7f763e190640 1 --2- 172.21.15.43:0/1131510203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e4584d0 0x55763e835710 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f7630008bf0 tx=0x7f7630029920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeed15b962a9fb56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:33.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.385+0000 7f76217fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T10:44:33.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.385+0000 7f76217fa640 1 -- 172.21.15.43:0/1131510203 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763002e050 con 0x55763e4584d0 2024-04-25T10:44:33.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.385+0000 7f76217fa640 1 -- 172.21.15.43:0/1131510203 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f7630004a00 con 0x55763e4584d0 2024-04-25T10:44:33.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.385+0000 7f7646b82740 1 -- 172.21.15.43:0/1131510203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7630008580 con 0x55763e4584d0 2024-04-25T10:44:33.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.385+0000 7f76217fa640 1 -- 172.21.15.43:0/1131510203 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7630004c00 con 0x55763e4584d0 2024-04-25T10:44:33.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.385+0000 7f7646b82740 10 client.19062 fetch_fsmap learned FSMap version 26 2024-04-25T10:44:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.385+0000 7f7646b82740 1 -- 172.21.15.43:0/1131510203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55763e8a7810 con 0x55763e4584d0 2024-04-25T10:44:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.386+0000 7f76217fa640 1 -- 172.21.15.43:0/1131510203 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7630008580 con 0x55763e4584d0 2024-04-25T10:44:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.386+0000 7f76217fa640 1 -- 172.21.15.43:0/1131510203 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7630027040 con 0x55763e4584d0 2024-04-25T10:44:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.386+0000 7f7646b82740 10 client.19062 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:44:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.386+0000 7f7646b82740 10 client.19062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T10:44:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.386+0000 7f7646b82740 20 client.19062 populate_metadata read hostname 'smithi043' 2024-04-25T10:44:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.386+0000 7f7646b82740 1 --2- 172.21.15.43:0/1131510203 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55763e4153a0 0x55763e8a9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.386+0000 7f7646b82740 4 client.19062 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:44:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.386+0000 7f7646b82740 1 -- 172.21.15.43:0/1131510203 --> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55763e8230e0 con 0x55763e4153a0 2024-04-25T10:44:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.386+0000 7f763d18e640 1 --2- 172.21.15.43:0/1131510203 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55763e4153a0 0x55763e8a9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.387+0000 7f763d18e640 1 --2- 172.21.15.43:0/1131510203 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55763e4153a0 0x55763e8a9c00 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de0254345dc6a8e4 server_cookie=f8c17b17e9eeae3c in_seq=0 out_seq=0 2024-04-25T10:44:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.387+0000 7f76217fa640 10 client.19062 ms_handle_connect on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.387+0000 7f76217fa640 1 -- 172.21.15.43:0/1131510203 <== mds.0 v2:172.21.15.77:6838/172043982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55763e8230e0 con 0x55763e4153a0 2024-04-25T10:44:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.387+0000 7f76217fa640 10 client.19062 handle_command_reply: tid=0 2024-04-25T10:44:33.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.387+0000 7f7646b82740 1 client.19062 shutdown 2024-04-25T10:44:33.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.387+0000 7f7646b82740 1 -- 172.21.15.43:0/1131510203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e4584d0 msgr2=0x55763e835710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.387+0000 7f7646b82740 1 --2- 172.21.15.43:0/1131510203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e4584d0 0x55763e835710 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f7630008bf0 tx=0x7f7630029920 comp rx=0 tx=0).stop 2024-04-25T10:44:33.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.387+0000 7f7646b82740 1 -- 172.21.15.43:0/1131510203 shutdown_connections 2024-04-25T10:44:33.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.387+0000 7f7646b82740 1 --2- 172.21.15.43:0/1131510203 >> [v2:172.21.15.77:6838/172043982,v1:172.21.15.77:6839/172043982] conn(0x55763e4153a0 0x55763e8a9c00 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.387+0000 7f7646b82740 1 --2- 172.21.15.43:0/1131510203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55763e4606b0 0x55763e830ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.387+0000 7f7646b82740 1 --2- 172.21.15.43:0/1131510203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55763e481390 0x55763e830ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.387+0000 7f7646b82740 1 --2- 172.21.15.43:0/1131510203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55763e4584d0 0x55763e835710 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.387+0000 7f76217fa640 0 client.19062 ms_handle_reset on v2:172.21.15.77:6838/172043982 2024-04-25T10:44:33.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.387+0000 7f7646b82740 1 -- 172.21.15.43:0/1131510203 >> 172.21.15.43:0/1131510203 conn(0x55763e4754e0 msgr2=0x55763e81eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:33.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.387+0000 7f7646b82740 1 -- 172.21.15.43:0/1131510203 shutdown_connections 2024-04-25T10:44:33.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.387+0000 7f7646b82740 1 -- 172.21.15.43:0/1131510203 wait complete. 2024-04-25T10:44:33.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.387+0000 7f7646b82740 20 client.19062 trim_cache size 0 max 16384 2024-04-25T10:44:33.396 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T10:44:33.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.388+0000 7f763fbf3640 1 -- 172.21.15.43:0/2835572802 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f76240afc60 msgr2=0x7f76240b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.388+0000 7f763fbf3640 1 --2- 172.21.15.43:0/2835572802 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f76240afc60 0x7f76240b2170 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f76340dcc90 tx=0x7f76340e5060 comp rx=0 tx=0).stop 2024-04-25T10:44:33.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.388+0000 7f763fbf3640 1 -- 172.21.15.43:0/2835572802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7638104260 msgr2=0x7f763819a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:33.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.388+0000 7f763fbf3640 1 --2- 172.21.15.43:0/2835572802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7638104260 0x7f763819a780 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f763400e9f0 tx=0x7f763400eeb0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.388+0000 7f763fbf3640 1 -- 172.21.15.43:0/2835572802 shutdown_connections 2024-04-25T10:44:33.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.388+0000 7f763fbf3640 1 --2- 172.21.15.43:0/2835572802 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f76240afc60 0x7f76240b2170 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.388+0000 7f763fbf3640 1 --2- 172.21.15.43:0/2835572802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7638109b10 0x7f763819ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.388+0000 7f763fbf3640 1 --2- 172.21.15.43:0/2835572802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7638109fb0 0x7f763819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.388+0000 7f763fbf3640 1 --2- 172.21.15.43:0/2835572802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7638104260 0x7f763819a780 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:33.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.388+0000 7f763fbf3640 1 -- 172.21.15.43:0/2835572802 >> 172.21.15.43:0/2835572802 conn(0x7f76380ff1d0 msgr2=0x7f763810fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:33.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.388+0000 7f763fbf3640 1 -- 172.21.15.43:0/2835572802 shutdown_connections 2024-04-25T10:44:33.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:33.388+0000 7f763fbf3640 1 -- 172.21.15.43:0/2835572802 wait complete. 2024-04-25T10:44:34.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:34 smithi077 ceph-mon[32772]: pgmap v376: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-04-25T10:44:34.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:34 smithi164 ceph-mon[32713]: pgmap v376: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-04-25T10:44:34.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:34 smithi043 ceph-mon[29478]: pgmap v376: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-04-25T10:44:34.401 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-25T10:44:34.401 INFO:tasks.fwd_scrub:done joining 2024-04-25T10:44:34.402 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T10:44:34.413 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:44:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.537+0000 7efcbfcf9640 1 -- 172.21.15.43:0/1747940192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcb8104bf0 msgr2=0x7efcb8105000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.537+0000 7efcbfcf9640 1 --2- 172.21.15.43:0/1747940192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcb8104bf0 0x7efcb8105000 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009900 tx=0x7efcb402a410 comp rx=0 tx=0).stop 2024-04-25T10:44:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.537+0000 7efcbfcf9640 1 -- 172.21.15.43:0/1747940192 shutdown_connections 2024-04-25T10:44:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.537+0000 7efcbfcf9640 1 --2- 172.21.15.43:0/1747940192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcb810e040 0x7efcb8110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.537+0000 7efcbfcf9640 1 --2- 172.21.15.43:0/1747940192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcb810b650 0x7efcb810da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.537+0000 7efcbfcf9640 1 --2- 172.21.15.43:0/1747940192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcb8104bf0 0x7efcb8105000 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.537+0000 7efcbfcf9640 1 -- 172.21.15.43:0/1747940192 >> 172.21.15.43:0/1747940192 conn(0x7efcb8078110 msgr2=0x7efcb8078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.537+0000 7efcbfcf9640 1 -- 172.21.15.43:0/1747940192 shutdown_connections 2024-04-25T10:44:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.537+0000 7efcbfcf9640 1 -- 172.21.15.43:0/1747940192 wait complete. 2024-04-25T10:44:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.538+0000 7efcbfcf9640 1 Processor -- start 2024-04-25T10:44:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.538+0000 7efcbfcf9640 1 -- start start 2024-04-25T10:44:34.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.538+0000 7efcbfcf9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcb8104bf0 0x7efcb81024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.538+0000 7efcbfcf9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcb810b650 0x7efcb81029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.538+0000 7efcbfcf9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcb810e040 0x7efcb8102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.538+0000 7efcbfcf9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8113cf0 con 0x7efcb810b650 2024-04-25T10:44:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.538+0000 7efcbfcf9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8113b70 con 0x7efcb8104bf0 2024-04-25T10:44:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.538+0000 7efcbfcf9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8113e70 con 0x7efcb810e040 2024-04-25T10:44:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.539+0000 7efcbd294640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcb810b650 0x7efcb81029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.539+0000 7efcbd294640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcb810b650 0x7efcb81029f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43356/0 (socket says 172.21.15.43:43356) 2024-04-25T10:44:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.539+0000 7efcbd294640 1 -- 172.21.15.43:0/3574638974 learned_addr learned my addr 172.21.15.43:0/3574638974 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.539+0000 7efcbda95640 1 --2- 172.21.15.43:0/3574638974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcb8104bf0 0x7efcb81024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.539+0000 7efcbe296640 1 --2- 172.21.15.43:0/3574638974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcb810e040 0x7efcb8102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.539+0000 7efcbd294640 1 -- 172.21.15.43:0/3574638974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcb810e040 msgr2=0x7efcb8102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.539+0000 7efcbd294640 1 --2- 172.21.15.43:0/3574638974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcb810e040 0x7efcb8102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.539+0000 7efcbd294640 1 -- 172.21.15.43:0/3574638974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcb8104bf0 msgr2=0x7efcb81024b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:34.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.539+0000 7efcbd294640 1 --2- 172.21.15.43:0/3574638974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcb8104bf0 0x7efcb81024b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:34.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.539+0000 7efcbd294640 1 -- 172.21.15.43:0/3574638974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb81a8a30 con 0x7efcb810b650 2024-04-25T10:44:34.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.539+0000 7efcbe296640 1 --2- 172.21.15.43:0/3574638974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcb810e040 0x7efcb8102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:34.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.539+0000 7efcbd294640 1 --2- 172.21.15.43:0/3574638974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcb810b650 0x7efcb81029f0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7efca800e9f0 tx=0x7efca800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:34.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.539+0000 7efcaeffd640 1 -- 172.21.15.43:0/3574638974 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca800cd80 con 0x7efcb810b650 2024-04-25T10:44:34.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.539+0000 7efcaeffd640 1 -- 172.21.15.43:0/3574638974 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efca80054d0 con 0x7efcb810b650 2024-04-25T10:44:34.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.539+0000 7efcbfcf9640 1 -- 172.21.15.43:0/3574638974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb81a8d20 con 0x7efcb810b650 2024-04-25T10:44:34.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.539+0000 7efcaeffd640 1 -- 172.21.15.43:0/3574638974 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8010690 con 0x7efcb810b650 2024-04-25T10:44:34.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.539+0000 7efcbfcf9640 1 -- 172.21.15.43:0/3574638974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb81a91c0 con 0x7efcb810b650 2024-04-25T10:44:34.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.541+0000 7efcbfcf9640 1 -- 172.21.15.43:0/3574638974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb806a770 con 0x7efcb810b650 2024-04-25T10:44:34.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.542+0000 7efcaeffd640 1 -- 172.21.15.43:0/3574638974 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7efca8004e90 con 0x7efcb810b650 2024-04-25T10:44:34.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.543+0000 7efcaeffd640 1 --2- 172.21.15.43:0/3574638974 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efca40afce0 0x7efca40b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:34.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.543+0000 7efcbda95640 1 --2- 172.21.15.43:0/3574638974 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efca40afce0 0x7efca40b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:34.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.543+0000 7efcaeffd640 1 -- 172.21.15.43:0/3574638974 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7efca80df4e0 con 0x7efcb810b650 2024-04-25T10:44:34.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.543+0000 7efcbda95640 1 --2- 172.21.15.43:0/3574638974 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efca40afce0 0x7efca40b21f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efcb4005ab0 tx=0x7efcb4005a20 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:34.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.547+0000 7efcaeffd640 1 -- 172.21.15.43:0/3574638974 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efca80a7d20 con 0x7efcb810b650 2024-04-25T10:44:34.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.737+0000 7efcbfcf9640 1 -- 172.21.15.43:0/3574638974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efcb81a9550 con 0x7efcb810b650 2024-04-25T10:44:34.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.739+0000 7efcaeffd640 1 -- 172.21.15.43:0/3574638974 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+23086 (secure 0 0 0) 0x7efca80a7620 con 0x7efcb810b650 2024-04-25T10:44:34.741 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:44:34.741 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":93,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:44:12.995589+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:31.503414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:32.224334+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":92,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1599315071":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6801/1953858940":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6800/1953858940":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3026776968":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/188939647":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/1287690800":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:44:34.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.741+0000 7efcbfcf9640 1 -- 172.21.15.43:0/3574638974 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efca40afce0 msgr2=0x7efca40b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:34.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.741+0000 7efcbfcf9640 1 --2- 172.21.15.43:0/3574638974 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efca40afce0 0x7efca40b21f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efcb4005ab0 tx=0x7efcb4005a20 comp rx=0 tx=0).stop 2024-04-25T10:44:34.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.741+0000 7efcbfcf9640 1 -- 172.21.15.43:0/3574638974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcb810b650 msgr2=0x7efcb81029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:34.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.741+0000 7efcbfcf9640 1 --2- 172.21.15.43:0/3574638974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcb810b650 0x7efcb81029f0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7efca800e9f0 tx=0x7efca800edf0 comp rx=0 tx=0).stop 2024-04-25T10:44:34.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.741+0000 7efcbfcf9640 1 -- 172.21.15.43:0/3574638974 shutdown_connections 2024-04-25T10:44:34.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.741+0000 7efcbfcf9640 1 --2- 172.21.15.43:0/3574638974 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7efca40afce0 0x7efca40b21f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:34.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.741+0000 7efcbfcf9640 1 --2- 172.21.15.43:0/3574638974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcb810e040 0x7efcb8102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:34.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.741+0000 7efcbfcf9640 1 --2- 172.21.15.43:0/3574638974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcb810b650 0x7efcb81029f0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:34.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.741+0000 7efcbfcf9640 1 --2- 172.21.15.43:0/3574638974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcb8104bf0 0x7efcb81024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:34.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.741+0000 7efcbfcf9640 1 -- 172.21.15.43:0/3574638974 >> 172.21.15.43:0/3574638974 conn(0x7efcb8078110 msgr2=0x7efcb8101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:34.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.741+0000 7efcbfcf9640 1 -- 172.21.15.43:0/3574638974 shutdown_connections 2024-04-25T10:44:34.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.741+0000 7efcbfcf9640 1 -- 172.21.15.43:0/3574638974 wait complete. 2024-04-25T10:44:34.755 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:25:58.167476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:31.503414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:32.224334+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 92, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:44:34.755 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:44:34.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.868+0000 7f49b3451640 1 -- 172.21.15.43:0/1646593713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ac10e0c0 msgr2=0x7f49ac1105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:34.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.868+0000 7f49b3451640 1 --2- 172.21.15.43:0/1646593713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ac10e0c0 0x7f49ac1105a0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f49a4009ca0 tx=0x7f49a402a8c0 comp rx=0 tx=0).stop 2024-04-25T10:44:34.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.868+0000 7f49b3451640 1 -- 172.21.15.43:0/1646593713 shutdown_connections 2024-04-25T10:44:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.868+0000 7f49b3451640 1 --2- 172.21.15.43:0/1646593713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ac10e0c0 0x7f49ac1105a0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.868+0000 7f49b3451640 1 --2- 172.21.15.43:0/1646593713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49ac10b6d0 0x7f49ac10daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.868+0000 7f49b3451640 1 --2- 172.21.15.43:0/1646593713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49ac104c70 0x7f49ac105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.869+0000 7f49b3451640 1 -- 172.21.15.43:0/1646593713 >> 172.21.15.43:0/1646593713 conn(0x7f49ac078110 msgr2=0x7f49ac078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.869+0000 7f49b3451640 1 -- 172.21.15.43:0/1646593713 shutdown_connections 2024-04-25T10:44:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.869+0000 7f49b3451640 1 -- 172.21.15.43:0/1646593713 wait complete. 2024-04-25T10:44:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.869+0000 7f49b3451640 1 Processor -- start 2024-04-25T10:44:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.869+0000 7f49b3451640 1 -- start start 2024-04-25T10:44:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.869+0000 7f49b3451640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49ac104c70 0x7f49ac19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49b3451640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ac10b6d0 0x7f49ac19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49b3451640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49ac10e0c0 0x7f49ac19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49b3451640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac113ce0 con 0x7f49ac10b6d0 2024-04-25T10:44:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49b3451640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac113b60 con 0x7f49ac10e0c0 2024-04-25T10:44:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49b3451640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac113e60 con 0x7f49ac104c70 2024-04-25T10:44:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49b09ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ac10b6d0 0x7f49ac19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49b09ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ac10b6d0 0x7f49ac19ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43364/0 (socket says 172.21.15.43:43364) 2024-04-25T10:44:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49b09ec640 1 -- 172.21.15.43:0/3273775325 learned_addr learned my addr 172.21.15.43:0/3273775325 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49b11ed640 1 --2- 172.21.15.43:0/3273775325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49ac104c70 0x7f49ac19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49b19ee640 1 --2- 172.21.15.43:0/3273775325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49ac10e0c0 0x7f49ac19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49b09ec640 1 -- 172.21.15.43:0/3273775325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49ac104c70 msgr2=0x7f49ac19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49b09ec640 1 --2- 172.21.15.43:0/3273775325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49ac104c70 0x7f49ac19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49b09ec640 1 -- 172.21.15.43:0/3273775325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49ac10e0c0 msgr2=0x7f49ac19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49b09ec640 1 --2- 172.21.15.43:0/3273775325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49ac10e0c0 0x7f49ac19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49b09ec640 1 -- 172.21.15.43:0/3273775325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ac19f580 con 0x7f49ac10b6d0 2024-04-25T10:44:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49b09ec640 1 --2- 172.21.15.43:0/3273775325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ac10b6d0 0x7f49ac19ac80 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f499c00d740 tx=0x7f499c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49a27fc640 1 -- 172.21.15.43:0/3273775325 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c017070 con 0x7f49ac10b6d0 2024-04-25T10:44:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49a27fc640 1 -- 172.21.15.43:0/3273775325 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f499c00de50 con 0x7f49ac10b6d0 2024-04-25T10:44:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49b3451640 1 -- 172.21.15.43:0/3273775325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49ac19f870 con 0x7f49ac10b6d0 2024-04-25T10:44:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49a27fc640 1 -- 172.21.15.43:0/3273775325 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c004ce0 con 0x7f49ac10b6d0 2024-04-25T10:44:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.870+0000 7f49b3451640 1 -- 172.21.15.43:0/3273775325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49ac19fd70 con 0x7f49ac10b6d0 2024-04-25T10:44:34.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.871+0000 7f49b3451640 1 -- 172.21.15.43:0/3273775325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49ac06a160 con 0x7f49ac10b6d0 2024-04-25T10:44:34.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.873+0000 7f49a27fc640 1 -- 172.21.15.43:0/3273775325 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f499c004e80 con 0x7f49ac10b6d0 2024-04-25T10:44:34.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.874+0000 7f49a27fc640 1 --2- 172.21.15.43:0/3273775325 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f49980afc60 0x7f49980b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:34.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.874+0000 7f49b11ed640 1 --2- 172.21.15.43:0/3273775325 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f49980afc60 0x7f49980b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:34.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.874+0000 7f49a27fc640 1 -- 172.21.15.43:0/3273775325 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f499c0dece0 con 0x7f49ac10b6d0 2024-04-25T10:44:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.875+0000 7f49b11ed640 1 --2- 172.21.15.43:0/3273775325 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f49980afc60 0x7f49980b2170 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009bd0 tx=0x7f49a8005f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:34.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:34.878+0000 7f49a27fc640 1 -- 172.21.15.43:0/3273775325 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49ac19fd70 con 0x7f49ac10b6d0 2024-04-25T10:44:35.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.065+0000 7f49b3451640 1 -- 172.21.15.43:0/3273775325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f49ac1a72e0 con 0x7f49ac10b6d0 2024-04-25T10:44:35.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.065+0000 7f49a27fc640 1 -- 172.21.15.43:0/3273775325 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f499c0a6f30 con 0x7f49ac10b6d0 2024-04-25T10:44:35.067 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T10:44:35.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.067+0000 7f49b3451640 1 -- 172.21.15.43:0/3273775325 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f49980afc60 msgr2=0x7f49980b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:35.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.067+0000 7f49b3451640 1 --2- 172.21.15.43:0/3273775325 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f49980afc60 0x7f49980b2170 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009bd0 tx=0x7f49a8005f90 comp rx=0 tx=0).stop 2024-04-25T10:44:35.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.067+0000 7f49b3451640 1 -- 172.21.15.43:0/3273775325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ac10b6d0 msgr2=0x7f49ac19ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:35.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.067+0000 7f49b3451640 1 --2- 172.21.15.43:0/3273775325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ac10b6d0 0x7f49ac19ac80 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f499c00d740 tx=0x7f499c00dc00 comp rx=0 tx=0).stop 2024-04-25T10:44:35.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.067+0000 7f49b3451640 1 -- 172.21.15.43:0/3273775325 shutdown_connections 2024-04-25T10:44:35.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.067+0000 7f49b3451640 1 --2- 172.21.15.43:0/3273775325 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f49980afc60 0x7f49980b2170 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.067+0000 7f49b3451640 1 --2- 172.21.15.43:0/3273775325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49ac10e0c0 0x7f49ac19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.067+0000 7f49b3451640 1 --2- 172.21.15.43:0/3273775325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ac10b6d0 0x7f49ac19ac80 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.067+0000 7f49b3451640 1 --2- 172.21.15.43:0/3273775325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49ac104c70 0x7f49ac19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.067+0000 7f49b3451640 1 -- 172.21.15.43:0/3273775325 >> 172.21.15.43:0/3273775325 conn(0x7f49ac078110 msgr2=0x7f49ac101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:35.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.067+0000 7f49b3451640 1 -- 172.21.15.43:0/3273775325 shutdown_connections 2024-04-25T10:44:35.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.067+0000 7f49b3451640 1 -- 172.21.15.43:0/3273775325 wait complete. 2024-04-25T10:44:35.080 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:44:35.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.193+0000 7fdbe4372640 1 -- 172.21.15.43:0/3489375713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbdc108a20 msgr2=0x7fdbdc102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:35.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.193+0000 7fdbe4372640 1 --2- 172.21.15.43:0/3489375713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbdc108a20 0x7fdbdc102620 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009900 tx=0x7fdbd402a410 comp rx=0 tx=0).stop 2024-04-25T10:44:35.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.193+0000 7fdbe4372640 1 -- 172.21.15.43:0/3489375713 shutdown_connections 2024-04-25T10:44:35.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.193+0000 7fdbe4372640 1 --2- 172.21.15.43:0/3489375713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbdc1031d0 0x7fdbdc10f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.193+0000 7fdbe4372640 1 --2- 172.21.15.43:0/3489375713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbdc108ec0 0x7fdbdc102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.193+0000 7fdbe4372640 1 --2- 172.21.15.43:0/3489375713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbdc108a20 0x7fdbdc102620 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.193+0000 7fdbe4372640 1 -- 172.21.15.43:0/3489375713 >> 172.21.15.43:0/3489375713 conn(0x7fdbdc078180 msgr2=0x7fdbdc0785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:35.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.193+0000 7fdbe4372640 1 -- 172.21.15.43:0/3489375713 shutdown_connections 2024-04-25T10:44:35.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.193+0000 7fdbe4372640 1 -- 172.21.15.43:0/3489375713 wait complete. 2024-04-25T10:44:35.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.194+0000 7fdbe4372640 1 Processor -- start 2024-04-25T10:44:35.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.194+0000 7fdbe4372640 1 -- start start 2024-04-25T10:44:35.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.194+0000 7fdbe4372640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbdc1031d0 0x7fdbdc19b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:35.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.194+0000 7fdbe4372640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbdc108a20 0x7fdbdc19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:35.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.194+0000 7fdbe4372640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbdc108ec0 0x7fdbdc19c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:35.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.194+0000 7fdbe4372640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc112cf0 con 0x7fdbdc108a20 2024-04-25T10:44:35.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.194+0000 7fdbe4372640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc112b70 con 0x7fdbdc1031d0 2024-04-25T10:44:35.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.194+0000 7fdbe4372640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc112e70 con 0x7fdbdc108ec0 2024-04-25T10:44:35.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.194+0000 7fdbe190d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbdc108a20 0x7fdbdc19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:35.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.194+0000 7fdbe190d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbdc108a20 0x7fdbdc19bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43376/0 (socket says 172.21.15.43:43376) 2024-04-25T10:44:35.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.194+0000 7fdbe190d640 1 -- 172.21.15.43:0/665347424 learned_addr learned my addr 172.21.15.43:0/665347424 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:35.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.194+0000 7fdbe290f640 1 --2- 172.21.15.43:0/665347424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbdc108ec0 0x7fdbdc19c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:35.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.194+0000 7fdbe210e640 1 --2- 172.21.15.43:0/665347424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbdc1031d0 0x7fdbdc19b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:35.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.195+0000 7fdbe190d640 1 -- 172.21.15.43:0/665347424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbdc108ec0 msgr2=0x7fdbdc19c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:35.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.195+0000 7fdbe190d640 1 --2- 172.21.15.43:0/665347424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbdc108ec0 0x7fdbdc19c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.195+0000 7fdbe190d640 1 -- 172.21.15.43:0/665347424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbdc1031d0 msgr2=0x7fdbdc19b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:35.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.195+0000 7fdbe190d640 1 --2- 172.21.15.43:0/665347424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbdc1031d0 0x7fdbdc19b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.195+0000 7fdbe190d640 1 -- 172.21.15.43:0/665347424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc1a6990 con 0x7fdbdc108a20 2024-04-25T10:44:35.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.195+0000 7fdbe210e640 1 --2- 172.21.15.43:0/665347424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbdc1031d0 0x7fdbdc19b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:35.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.195+0000 7fdbe190d640 1 --2- 172.21.15.43:0/665347424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbdc108a20 0x7fdbdc19bb90 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0005d00 tx=0x7fdbd000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:35.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.195+0000 7fdbdb7fe640 1 -- 172.21.15.43:0/665347424 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0017070 con 0x7fdbdc108a20 2024-04-25T10:44:35.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.195+0000 7fdbdb7fe640 1 -- 172.21.15.43:0/665347424 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd0009c30 con 0x7fdbdc108a20 2024-04-25T10:44:35.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.195+0000 7fdbe4372640 1 -- 172.21.15.43:0/665347424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbdc1a6c80 con 0x7fdbdc108a20 2024-04-25T10:44:35.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.195+0000 7fdbdb7fe640 1 -- 172.21.15.43:0/665347424 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd000f620 con 0x7fdbdc108a20 2024-04-25T10:44:35.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.195+0000 7fdbe4372640 1 -- 172.21.15.43:0/665347424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbdc1a71d0 con 0x7fdbdc108a20 2024-04-25T10:44:35.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.196+0000 7fdbe4372640 1 -- 172.21.15.43:0/665347424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbdc06a160 con 0x7fdbdc108a20 2024-04-25T10:44:35.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.197+0000 7fdbdb7fe640 1 -- 172.21.15.43:0/665347424 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fdbd001a070 con 0x7fdbdc108a20 2024-04-25T10:44:35.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.198+0000 7fdbdb7fe640 1 --2- 172.21.15.43:0/665347424 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdbc40afce0 0x7fdbc40b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:35.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.199+0000 7fdbe210e640 1 --2- 172.21.15.43:0/665347424 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdbc40afce0 0x7fdbc40b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:35.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.199+0000 7fdbdb7fe640 1 -- 172.21.15.43:0/665347424 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fdbd00dead0 con 0x7fdbdc108a20 2024-04-25T10:44:35.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.199+0000 7fdbe210e640 1 --2- 172.21.15.43:0/665347424 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdbc40afce0 0x7fdbc40b21f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdbd402aa00 tx=0x7fdbd4031400 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:35.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.202+0000 7fdbdb7fe640 1 -- 172.21.15.43:0/665347424 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdbd0009de0 con 0x7fdbdc108a20 2024-04-25T10:44:35.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:35 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3574638974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:44:35.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.389+0000 7fdbe4372640 1 -- 172.21.15.43:0/665347424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdbdc106f30 con 0x7fdbdc108a20 2024-04-25T10:44:35.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.390+0000 7fdbdb7fe640 1 -- 172.21.15.43:0/665347424 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fdbd00a73f0 con 0x7fdbdc108a20 2024-04-25T10:44:35.392 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T10:44:35.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.392+0000 7fdbe4372640 1 -- 172.21.15.43:0/665347424 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdbc40afce0 msgr2=0x7fdbc40b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:35.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.392+0000 7fdbe4372640 1 --2- 172.21.15.43:0/665347424 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdbc40afce0 0x7fdbc40b21f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdbd402aa00 tx=0x7fdbd4031400 comp rx=0 tx=0).stop 2024-04-25T10:44:35.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.392+0000 7fdbe4372640 1 -- 172.21.15.43:0/665347424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbdc108a20 msgr2=0x7fdbdc19bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:35.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.392+0000 7fdbe4372640 1 --2- 172.21.15.43:0/665347424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbdc108a20 0x7fdbdc19bb90 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0005d00 tx=0x7fdbd000dde0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.392+0000 7fdbe4372640 1 -- 172.21.15.43:0/665347424 shutdown_connections 2024-04-25T10:44:35.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.392+0000 7fdbe4372640 1 --2- 172.21.15.43:0/665347424 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdbc40afce0 0x7fdbc40b21f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.392+0000 7fdbe4372640 1 --2- 172.21.15.43:0/665347424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbdc108ec0 0x7fdbdc19c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.392+0000 7fdbe4372640 1 --2- 172.21.15.43:0/665347424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbdc108a20 0x7fdbdc19bb90 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.392+0000 7fdbe4372640 1 --2- 172.21.15.43:0/665347424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbdc1031d0 0x7fdbdc19b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.392+0000 7fdbe4372640 1 -- 172.21.15.43:0/665347424 >> 172.21.15.43:0/665347424 conn(0x7fdbdc078180 msgr2=0x7fdbdc104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:35.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.392+0000 7fdbe4372640 1 -- 172.21.15.43:0/665347424 shutdown_connections 2024-04-25T10:44:35.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.392+0000 7fdbe4372640 1 -- 172.21.15.43:0/665347424 wait complete. 2024-04-25T10:44:35.405 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:44:35.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.518+0000 7f80a7cbd640 1 -- 172.21.15.43:0/1170140581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a0108a20 msgr2=0x7f80a0102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:35.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.518+0000 7f80a7cbd640 1 --2- 172.21.15.43:0/1170140581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a0108a20 0x7f80a0102620 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f809c009900 tx=0x7f809c02a410 comp rx=0 tx=0).stop 2024-04-25T10:44:35.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.518+0000 7f80a7cbd640 1 -- 172.21.15.43:0/1170140581 shutdown_connections 2024-04-25T10:44:35.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.518+0000 7f80a7cbd640 1 --2- 172.21.15.43:0/1170140581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80a01031d0 0x7f80a010f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.518+0000 7f80a7cbd640 1 --2- 172.21.15.43:0/1170140581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80a0108ec0 0x7f80a0102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.518+0000 7f80a7cbd640 1 --2- 172.21.15.43:0/1170140581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a0108a20 0x7f80a0102620 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.518+0000 7f80a7cbd640 1 -- 172.21.15.43:0/1170140581 >> 172.21.15.43:0/1170140581 conn(0x7f80a0078180 msgr2=0x7f80a00785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:35.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.518+0000 7f80a7cbd640 1 -- 172.21.15.43:0/1170140581 shutdown_connections 2024-04-25T10:44:35.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.518+0000 7f80a7cbd640 1 -- 172.21.15.43:0/1170140581 wait complete. 2024-04-25T10:44:35.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.519+0000 7f80a7cbd640 1 Processor -- start 2024-04-25T10:44:35.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.519+0000 7f80a7cbd640 1 -- start start 2024-04-25T10:44:35.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.519+0000 7f80a7cbd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80a01031d0 0x7f80a019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:35.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.519+0000 7f80a7cbd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a0108a20 0x7f80a019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:35.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.519+0000 7f80a7cbd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80a0108ec0 0x7f80a019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:35.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.519+0000 7f80a7cbd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f80a0112c20 con 0x7f80a0108a20 2024-04-25T10:44:35.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.519+0000 7f80a7cbd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f80a0112aa0 con 0x7f80a01031d0 2024-04-25T10:44:35.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.519+0000 7f80a7cbd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f80a0112da0 con 0x7f80a0108ec0 2024-04-25T10:44:35.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.519+0000 7f80a5258640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a0108a20 0x7f80a019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:35.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.519+0000 7f80a5258640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a0108a20 0x7f80a019ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43386/0 (socket says 172.21.15.43:43386) 2024-04-25T10:44:35.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.519+0000 7f80a5258640 1 -- 172.21.15.43:0/973180345 learned_addr learned my addr 172.21.15.43:0/973180345 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:35.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.519+0000 7f80a5a59640 1 --2- 172.21.15.43:0/973180345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80a01031d0 0x7f80a019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:35.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.519+0000 7f80a625a640 1 --2- 172.21.15.43:0/973180345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80a0108ec0 0x7f80a019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:35.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.520+0000 7f80a5258640 1 -- 172.21.15.43:0/973180345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80a0108ec0 msgr2=0x7f80a019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:35.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.520+0000 7f80a5258640 1 --2- 172.21.15.43:0/973180345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80a0108ec0 0x7f80a019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.520+0000 7f80a5258640 1 -- 172.21.15.43:0/973180345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80a01031d0 msgr2=0x7f80a019a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:35.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.520+0000 7f80a5258640 1 --2- 172.21.15.43:0/973180345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80a01031d0 0x7f80a019a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.520+0000 7f80a5258640 1 -- 172.21.15.43:0/973180345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a019f650 con 0x7f80a0108a20 2024-04-25T10:44:35.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.520+0000 7f80a625a640 1 --2- 172.21.15.43:0/973180345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80a0108ec0 0x7f80a019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:35.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.520+0000 7f80a5258640 1 --2- 172.21.15.43:0/973180345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a0108a20 0x7f80a019ad50 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f809000e9f0 tx=0x7f809000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:35.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.520+0000 7f8096ffd640 1 -- 172.21.15.43:0/973180345 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809000ccc0 con 0x7f80a0108a20 2024-04-25T10:44:35.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.520+0000 7f8096ffd640 1 -- 172.21.15.43:0/973180345 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80900035f0 con 0x7f80a0108a20 2024-04-25T10:44:35.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.520+0000 7f80a7cbd640 1 -- 172.21.15.43:0/973180345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a019f940 con 0x7f80a0108a20 2024-04-25T10:44:35.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.520+0000 7f8096ffd640 1 -- 172.21.15.43:0/973180345 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809000ce60 con 0x7f80a0108a20 2024-04-25T10:44:35.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.520+0000 7f80a7cbd640 1 -- 172.21.15.43:0/973180345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a01a72c0 con 0x7f80a0108a20 2024-04-25T10:44:35.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.521+0000 7f80a7cbd640 1 -- 172.21.15.43:0/973180345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a006a160 con 0x7f80a0108a20 2024-04-25T10:44:35.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.523+0000 7f8096ffd640 1 -- 172.21.15.43:0/973180345 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f8090003aa0 con 0x7f80a0108a20 2024-04-25T10:44:35.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.523+0000 7f8096ffd640 1 --2- 172.21.15.43:0/973180345 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f808c0afce0 0x7f808c0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:35.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.523+0000 7f8096ffd640 1 -- 172.21.15.43:0/973180345 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f80900df420 con 0x7f80a0108a20 2024-04-25T10:44:35.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.523+0000 7f80a5a59640 1 --2- 172.21.15.43:0/973180345 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f808c0afce0 0x7f808c0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:35.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.524+0000 7f80a5a59640 1 --2- 172.21.15.43:0/973180345 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f808c0afce0 0x7f808c0b21f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f809c009230 tx=0x7f809c031400 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:35.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.527+0000 7f8096ffd640 1 -- 172.21.15.43:0/973180345 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f80900a7cc0 con 0x7f80a0108a20 2024-04-25T10:44:35.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:35 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3574638974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:44:35.540 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:35 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3574638974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:44:35.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.712+0000 7f80a7cbd640 1 -- 172.21.15.43:0/973180345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f80a01a74f0 con 0x7f80a0108a20 2024-04-25T10:44:35.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.712+0000 7f8096ffd640 1 -- 172.21.15.43:0/973180345 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f80900a75c0 con 0x7f80a0108a20 2024-04-25T10:44:35.714 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T10:44:35.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.714+0000 7f80a7cbd640 1 -- 172.21.15.43:0/973180345 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f808c0afce0 msgr2=0x7f808c0b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:35.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.714+0000 7f80a7cbd640 1 --2- 172.21.15.43:0/973180345 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f808c0afce0 0x7f808c0b21f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f809c009230 tx=0x7f809c031400 comp rx=0 tx=0).stop 2024-04-25T10:44:35.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.714+0000 7f80a7cbd640 1 -- 172.21.15.43:0/973180345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a0108a20 msgr2=0x7f80a019ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:35.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.714+0000 7f80a7cbd640 1 --2- 172.21.15.43:0/973180345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a0108a20 0x7f80a019ad50 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f809000e9f0 tx=0x7f809000eeb0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.714+0000 7f80a7cbd640 1 -- 172.21.15.43:0/973180345 shutdown_connections 2024-04-25T10:44:35.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.714+0000 7f80a7cbd640 1 --2- 172.21.15.43:0/973180345 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f808c0afce0 0x7f808c0b21f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.714+0000 7f80a7cbd640 1 --2- 172.21.15.43:0/973180345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80a0108ec0 0x7f80a019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.714+0000 7f80a7cbd640 1 --2- 172.21.15.43:0/973180345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a0108a20 0x7f80a019ad50 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.714+0000 7f80a7cbd640 1 --2- 172.21.15.43:0/973180345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80a01031d0 0x7f80a019a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.714+0000 7f80a7cbd640 1 -- 172.21.15.43:0/973180345 >> 172.21.15.43:0/973180345 conn(0x7f80a0078180 msgr2=0x7f80a0074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:35.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.714+0000 7f80a7cbd640 1 -- 172.21.15.43:0/973180345 shutdown_connections 2024-04-25T10:44:35.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.714+0000 7f80a7cbd640 1 -- 172.21.15.43:0/973180345 wait complete. 2024-04-25T10:44:35.727 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T10:44:35.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.837+0000 7f4327897640 1 -- 172.21.15.43:0/640770592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320102b00 msgr2=0x7f4320102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:35.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.837+0000 7f4327897640 1 --2- 172.21.15.43:0/640770592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320102b00 0x7f4320102f10 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f4314009900 tx=0x7f431402a410 comp rx=0 tx=0).stop 2024-04-25T10:44:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.837+0000 7f4327897640 1 -- 172.21.15.43:0/640770592 shutdown_connections 2024-04-25T10:44:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.837+0000 7f4327897640 1 --2- 172.21.15.43:0/640770592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f432010d390 0x7f432010f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.837+0000 7f4327897640 1 --2- 172.21.15.43:0/640770592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320102b00 0x7f4320102f10 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.837+0000 7f4327897640 1 --2- 172.21.15.43:0/640770592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4320108a30 0x7f43201025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.837+0000 7f4327897640 1 -- 172.21.15.43:0/640770592 >> 172.21.15.43:0/640770592 conn(0x7f43200fe080 msgr2=0x7f43201004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.837+0000 7f4327897640 1 -- 172.21.15.43:0/640770592 shutdown_connections 2024-04-25T10:44:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.837+0000 7f4327897640 1 -- 172.21.15.43:0/640770592 wait complete. 2024-04-25T10:44:35.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.838+0000 7f4327897640 1 Processor -- start 2024-04-25T10:44:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.838+0000 7f4327897640 1 -- start start 2024-04-25T10:44:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.838+0000 7f4327897640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4320102b00 0x7f432019b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.838+0000 7f4327897640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320108a30 0x7f432019ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.838+0000 7f4327897640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f432010d390 0x7f432019bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.838+0000 7f4327897640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4320112bf0 con 0x7f4320108a30 2024-04-25T10:44:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.838+0000 7f4327897640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4320112a70 con 0x7f4320102b00 2024-04-25T10:44:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.838+0000 7f4327897640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4320112d70 con 0x7f432010d390 2024-04-25T10:44:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.838+0000 7f4324e32640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320108a30 0x7f432019ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.838+0000 7f4324e32640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320108a30 0x7f432019ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43400/0 (socket says 172.21.15.43:43400) 2024-04-25T10:44:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.838+0000 7f4324e32640 1 -- 172.21.15.43:0/4006467610 learned_addr learned my addr 172.21.15.43:0/4006467610 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:44:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.839+0000 7f4325633640 1 --2- 172.21.15.43:0/4006467610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4320102b00 0x7f432019b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.839+0000 7f4325e34640 1 --2- 172.21.15.43:0/4006467610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f432010d390 0x7f432019bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.839+0000 7f4324e32640 1 -- 172.21.15.43:0/4006467610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f432010d390 msgr2=0x7f432019bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.839+0000 7f4324e32640 1 --2- 172.21.15.43:0/4006467610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f432010d390 0x7f432019bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.839+0000 7f4324e32640 1 -- 172.21.15.43:0/4006467610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4320102b00 msgr2=0x7f432019b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.839+0000 7f4324e32640 1 --2- 172.21.15.43:0/4006467610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4320102b00 0x7f432019b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.839+0000 7f4324e32640 1 -- 172.21.15.43:0/4006467610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43201a6860 con 0x7f4320108a30 2024-04-25T10:44:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.839+0000 7f4325633640 1 --2- 172.21.15.43:0/4006467610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4320102b00 0x7f432019b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:44:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.839+0000 7f4324e32640 1 --2- 172.21.15.43:0/4006467610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320108a30 0x7f432019ba90 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f431402a930 tx=0x7f4314004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.839+0000 7f431e7fc640 1 -- 172.21.15.43:0/4006467610 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431403f030 con 0x7f4320108a30 2024-04-25T10:44:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.839+0000 7f431e7fc640 1 -- 172.21.15.43:0/4006467610 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4314003aa0 con 0x7f4320108a30 2024-04-25T10:44:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.839+0000 7f431e7fc640 1 -- 172.21.15.43:0/4006467610 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314035070 con 0x7f4320108a30 2024-04-25T10:44:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.839+0000 7f4327897640 1 -- 172.21.15.43:0/4006467610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43201a6af0 con 0x7f4320108a30 2024-04-25T10:44:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.839+0000 7f4327897640 1 -- 172.21.15.43:0/4006467610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43201a6fe0 con 0x7f4320108a30 2024-04-25T10:44:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.840+0000 7f4327897640 1 -- 172.21.15.43:0/4006467610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f432006a190 con 0x7f4320108a30 2024-04-25T10:44:35.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.842+0000 7f431e7fc640 1 -- 172.21.15.43:0/4006467610 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f4314042030 con 0x7f4320108a30 2024-04-25T10:44:35.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.842+0000 7f431e7fc640 1 --2- 172.21.15.43:0/4006467610 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f43080afc60 0x7f43080b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:44:35.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.842+0000 7f4325633640 1 --2- 172.21.15.43:0/4006467610 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f43080afc60 0x7f43080b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:44:35.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.842+0000 7f431e7fc640 1 -- 172.21.15.43:0/4006467610 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f43140ffc10 con 0x7f4320108a30 2024-04-25T10:44:35.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.843+0000 7f4325633640 1 --2- 172.21.15.43:0/4006467610 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f43080afc60 0x7f43080b2170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4320103a30 tx=0x7f4310009970 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:44:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:35.846+0000 7f431e7fc640 1 -- 172.21.15.43:0/4006467610 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43140c84b0 con 0x7f4320108a30 2024-04-25T10:44:36.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:36.082+0000 7f4327897640 1 -- 172.21.15.43:0/4006467610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f43201a72b0 con 0x7f4320108a30 2024-04-25T10:44:36.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:36.091+0000 7f431e7fc640 1 -- 172.21.15.43:0/4006467610 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+803656 (secure 0 0 0) 0x7f43140c7db0 con 0x7f4320108a30 2024-04-25T10:44:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:44:36.105 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:44:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 26, 2024-04-25T10:44:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: "active_gid": 15291, 2024-04-25T10:44:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: "active_name": "x", 2024-04-25T10:44:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: "active_addrs": { 2024-04-25T10:44:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-04-25T10:44:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-04-25T10:44:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6800", 2024-04-25T10:44:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3658924116 2024-04-25T10:44:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T10:44:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6801", 2024-04-25T10:44:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3658924116 2024-04-25T10:44:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T10:44:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: "active_addr": "172.21.15.43:6801/3658924116", 2024-04-25T10:44:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: "active_change": "2024-04-25T10:32:32.456847+0000", 2024-04-25T10:44:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T10:44:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: "available": true, 2024-04-25T10:44:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: "standbys": [ 2024-04-25T10:44:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 24994, 2024-04-25T10:44:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "z", 2024-04-25T10:44:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_features": 4540701547738038271, 2024-04-25T10:44:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: "available_modules": [ 2024-04-25T10:44:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "alerts", 2024-04-25T10:44:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-25T10:44:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-25T10:44:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-25T10:44:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T10:44:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_destination": { 2024-04-25T10:44:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_destination", 2024-04-25T10:44:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email address to send alerts to", 2024-04-25T10:44:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_from_name": { 2024-04-25T10:44:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_from_name", 2024-04-25T10:44:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "Ceph", 2024-04-25T10:44:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email From: name", 2024-04-25T10:44:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_host": { 2024-04-25T10:44:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_host", 2024-04-25T10:44:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP server", 2024-04-25T10:44:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_password": { 2024-04-25T10:44:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_password", 2024-04-25T10:44:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Password to authenticate with", 2024-04-25T10:44:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_port": { 2024-04-25T10:44:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_port", 2024-04-25T10:44:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "465", 2024-04-25T10:44:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP port", 2024-04-25T10:44:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_sender": { 2024-04-25T10:44:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_sender", 2024-04-25T10:44:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP envelope sender", 2024-04-25T10:44:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_ssl": { 2024-04-25T10:44:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_ssl", 2024-04-25T10:44:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T10:44:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_user": { 2024-04-25T10:44:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_user", 2024-04-25T10:44:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "User to authenticate as", 2024-04-25T10:44:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "balancer", 2024-04-25T10:44:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "active": { 2024-04-25T10:44:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "active", 2024-04-25T10:44:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T10:44:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_time": { 2024-04-25T10:44:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_time", 2024-04-25T10:44:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0000", 2024-04-25T10:44:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T10:44:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T10:44:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_weekday": { 2024-04-25T10:44:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_weekday", 2024-04-25T10:44:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-25T10:44:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-04-25T10:44:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-04-25T10:44:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T10:44:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T10:44:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_max_iterations": { 2024-04-25T10:44:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_max_iterations", 2024-04-25T10:44:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-25T10:44:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "25", 2024-04-25T10:44:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-25T10:44:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "250", 2024-04-25T10:44:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T10:44:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_metrics": { 2024-04-25T10:44:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_metrics", 2024-04-25T10:44:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T10:44:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T10:44:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T10:44:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_step": { 2024-04-25T10:44:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_step", 2024-04-25T10:44:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.5", 2024-04-25T10:44:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0.001", 2024-04-25T10:44:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "0.999", 2024-04-25T10:44:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "aggressiveness of optimization", 2024-04-25T10:44:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T10:44:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "end_time": { 2024-04-25T10:44:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_time", 2024-04-25T10:44:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2359", 2024-04-25T10:44:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T10:44:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T10:44:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "end_weekday": { 2024-04-25T10:44:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_weekday", 2024-04-25T10:44:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-25T10:44:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-04-25T10:44:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-04-25T10:44:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T10:44:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T10:44:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "min_score": { 2024-04-25T10:44:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_score", 2024-04-25T10:44:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T10:44:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-04-25T10:44:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-04-25T10:44:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "upmap", 2024-04-25T10:44:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-compat", 2024-04-25T10:44:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "none", 2024-04-25T10:44:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "read", 2024-04-25T10:44:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap", 2024-04-25T10:44:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap-read" 2024-04-25T10:44:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Balancer mode", 2024-04-25T10:44:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ids": { 2024-04-25T10:44:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_ids", 2024-04-25T10:44:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T10:44:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-25T10:44:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-25T10:44:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-25T10:44:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T10:44:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_deviation": { 2024-04-25T10:44:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_deviation", 2024-04-25T10:44:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-25T10:44:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-25T10:44:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T10:44:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T10:44:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_optimizations": { 2024-04-25T10:44:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_optimizations", 2024-04-25T10:44:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-25T10:44:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T10:44:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm", 2024-04-25T10:44:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_down_multiplier": { 2024-04-25T10:44:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_down_multiplier", 2024-04-25T10:44:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-04-25T10:44:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T10:44:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_refresh_rate": { 2024-04-25T10:44:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_refresh_rate", 2024-04-25T10:44:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "20", 2024-04-25T10:44:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T10:44:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_starting_port": { 2024-04-25T10:44:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_starting_port", 2024-04-25T10:44:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4721", 2024-04-25T10:44:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T10:44:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_ptrace": { 2024-04-25T10:44:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_ptrace", 2024-04-25T10:44:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T10:44:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T10:44:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_interval": { 2024-04-25T10:44:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_interval", 2024-04-25T10:44:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-25T10:44:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T10:44:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_memory_target_ratio": { 2024-04-25T10:44:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T10:44:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.7", 2024-04-25T10:44:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T10:44:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm_log_destination": { 2024-04-25T10:44:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm_log_destination", 2024-04-25T10:44:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "file", 2024-04-25T10:44:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "file,syslog", 2024-04-25T10:44:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "syslog" 2024-04-25T10:44:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T10:44:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "cgroups_split": { 2024-04-25T10:44:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cgroups_split", 2024-04-25T10:44:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T10:44:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "config_checks_enabled": { 2024-04-25T10:44:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_checks_enabled", 2024-04-25T10:44:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T10:44:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "config_dashboard": { 2024-04-25T10:44:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_dashboard", 2024-04-25T10:44:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T10:44:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_alertmanager": { 2024-04-25T10:44:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_alertmanager", 2024-04-25T10:44:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T10:44:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-25T10:44:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_base": { 2024-04-25T10:44:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_base", 2024-04-25T10:44:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T10:44:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Container image name, without the tag", 2024-04-25T10:44:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_elasticsearch": { 2024-04-25T10:44:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_elasticsearch", 2024-04-25T10:44:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T10:44:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "elasticsearch container image", 2024-04-25T10:44:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_grafana": { 2024-04-25T10:44:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_grafana", 2024-04-25T10:44:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T10:44:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-25T10:44:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_haproxy": { 2024-04-25T10:44:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_haproxy", 2024-04-25T10:44:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T10:44:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "HAproxy container image", 2024-04-25T10:44:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_agent": { 2024-04-25T10:44:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_agent", 2024-04-25T10:44:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T10:44:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger agent container image", 2024-04-25T10:44:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_collector": { 2024-04-25T10:44:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_collector", 2024-04-25T10:44:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T10:44:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger collector container image", 2024-04-25T10:44:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_query": { 2024-04-25T10:44:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_query", 2024-04-25T10:44:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T10:44:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger query container image", 2024-04-25T10:44:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_keepalived": { 2024-04-25T10:44:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_keepalived", 2024-04-25T10:44:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T10:44:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Keepalived container image", 2024-04-25T10:44:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_loki": { 2024-04-25T10:44:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_loki", 2024-04-25T10:44:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T10:44:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Loki container image", 2024-04-25T10:44:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_node_exporter": { 2024-04-25T10:44:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_node_exporter", 2024-04-25T10:44:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T10:44:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-25T10:44:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_nvmeof": { 2024-04-25T10:44:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_nvmeof", 2024-04-25T10:44:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T10:44:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Nvme-of container image", 2024-04-25T10:44:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_prometheus": { 2024-04-25T10:44:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_prometheus", 2024-04-25T10:44:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T10:44:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-25T10:44:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_promtail": { 2024-04-25T10:44:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_promtail", 2024-04-25T10:44:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T10:44:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Promtail container image", 2024-04-25T10:44:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_samba": { 2024-04-25T10:44:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_samba", 2024-04-25T10:44:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T10:44:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Samba/SMB container image", 2024-04-25T10:44:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_snmp_gateway": { 2024-04-25T10:44:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_snmp_gateway", 2024-04-25T10:44:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T10:44:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SNMP Gateway container image", 2024-04-25T10:44:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "container_init": { 2024-04-25T10:44:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_init", 2024-04-25T10:44:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T10:44:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "daemon_cache_timeout": { 2024-04-25T10:44:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "daemon_cache_timeout", 2024-04-25T10:44:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-25T10:44:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T10:44:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "default_cephadm_command_timeout": { 2024-04-25T10:44:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T10:44:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "900", 2024-04-25T10:44:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T10:44:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "default_registry": { 2024-04-25T10:44:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_registry", 2024-04-25T10:44:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io", 2024-04-25T10:44:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T10:44:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "device_cache_timeout": { 2024-04-25T10:44:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_cache_timeout", 2024-04-25T10:44:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1800", 2024-04-25T10:44:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache device inventory", 2024-04-25T10:44:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "device_enhanced_scan": { 2024-04-25T10:44:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_enhanced_scan", 2024-04-25T10:44:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T10:44:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "facts_cache_timeout": { 2024-04-25T10:44:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "facts_cache_timeout", 2024-04-25T10:44:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-25T10:44:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache host facts data", 2024-04-25T10:44:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: "grafana_dashboards_path": { 2024-04-25T10:44:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "grafana_dashboards_path", 2024-04-25T10:44:36.180 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.180 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.180 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.180 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T10:44:36.181 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.181 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.181 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.182 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T10:44:36.182 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.182 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.182 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.182 INFO:teuthology.orchestra.run.smithi043.stdout: "host_check_interval": { 2024-04-25T10:44:36.182 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "host_check_interval", 2024-04-25T10:44:36.182 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.182 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.182 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.182 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-25T10:44:36.182 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.183 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.183 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.183 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to perform a host check", 2024-04-25T10:44:36.183 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.183 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.183 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.183 INFO:teuthology.orchestra.run.smithi043.stdout: "hw_monitoring": { 2024-04-25T10:44:36.183 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hw_monitoring", 2024-04-25T10:44:36.183 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.183 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.183 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.183 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T10:44:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: "inventory_list_all": { 2024-04-25T10:44:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "inventory_list_all", 2024-04-25T10:44:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T10:44:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "log_refresh_metadata": { 2024-04-25T10:44:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_refresh_metadata", 2024-04-25T10:44:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T10:44:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T10:44:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T10:44:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T10:44:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T10:44:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T10:44:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T10:44:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "*", 2024-04-25T10:44:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T10:44:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: "max_count_per_host": { 2024-04-25T10:44:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_count_per_host", 2024-04-25T10:44:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of daemons per service per host", 2024-04-25T10:44:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd_draining_count": { 2024-04-25T10:44:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_osd_draining_count", 2024-04-25T10:44:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T10:44:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: "migration_current": { 2024-04-25T10:44:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "migration_current", 2024-04-25T10:44:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "internal - do not modify", 2024-04-25T10:44:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-04-25T10:44:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-04-25T10:44:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "root", 2024-04-25T10:44:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm-package", 2024-04-25T10:44:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: "root" 2024-04-25T10:44:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T10:44:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: "oob_default_addr": { 2024-04-25T10:44:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "oob_default_addr", 2024-04-25T10:44:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "169.254.1.1", 2024-04-25T10:44:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T10:44:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: "prometheus_alerts_path": { 2024-04-25T10:44:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus_alerts_path", 2024-04-25T10:44:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T10:44:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T10:44:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_insecure": { 2024-04-25T10:44:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_insecure", 2024-04-25T10:44:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T10:44:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_password": { 2024-04-25T10:44:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_password", 2024-04-25T10:44:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T10:44:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_url": { 2024-04-25T10:44:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_url", 2024-04-25T10:44:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T10:44:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_username": { 2024-04-25T10:44:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_username", 2024-04-25T10:44:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T10:44:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: "secure_monitoring_stack": { 2024-04-25T10:44:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "secure_monitoring_stack", 2024-04-25T10:44:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T10:44:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "service_discovery_port": { 2024-04-25T10:44:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "service_discovery_port", 2024-04-25T10:44:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8765", 2024-04-25T10:44:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "cephadm service discovery port", 2024-04-25T10:44:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "ssh_config_file": { 2024-04-25T10:44:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssh_config_file", 2024-04-25T10:44:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T10:44:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "use_agent": { 2024-04-25T10:44:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_agent", 2024-04-25T10:44:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T10:44:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: "use_repo_digest": { 2024-04-25T10:44:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_repo_digest", 2024-04-25T10:44:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T10:44:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_failed_host_check": { 2024-04-25T10:44:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_failed_host_check", 2024-04-25T10:44:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T10:44:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_daemons": { 2024-04-25T10:44:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_daemons", 2024-04-25T10:44:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T10:44:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_hosts": { 2024-04-25T10:44:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_hosts", 2024-04-25T10:44:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T10:44:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crash", 2024-04-25T10:44:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: "retain_interval": { 2024-04-25T10:44:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retain_interval", 2024-04-25T10:44:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "31536000", 2024-04-25T10:44:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T10:44:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_recent_interval": { 2024-04-25T10:44:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_recent_interval", 2024-04-25T10:44:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1209600", 2024-04-25T10:44:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T10:44:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dashboard", 2024-04-25T10:44:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T10:44:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T10:44:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T10:44:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T10:44:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T10:44:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T10:44:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_ENABLED": { 2024-04-25T10:44:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T10:44:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T10:44:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T10:44:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T10:44:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T10:44:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T10:44:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T10:44:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.229 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.229 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.229 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.229 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.229 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.229 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.229 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.229 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.229 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.229 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T10:44:36.229 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T10:44:36.230 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.230 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.230 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.230 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.230 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.230 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.230 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.230 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.230 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.230 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.230 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.230 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T10:44:36.231 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T10:44:36.231 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.231 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.231 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.231 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.231 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.231 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.231 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.231 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.231 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.231 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.231 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.231 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T10:44:36.231 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T10:44:36.232 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.232 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.232 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.232 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.232 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.232 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.232 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.232 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.232 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.232 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.232 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.232 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T10:44:36.232 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T10:44:36.233 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.233 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.233 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.233 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.233 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.233 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.233 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.233 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.233 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.233 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.233 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.233 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T10:44:36.234 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T10:44:36.234 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.234 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.234 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.234 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.234 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.234 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.234 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.234 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.234 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.234 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.234 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.234 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T10:44:36.235 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T10:44:36.235 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.235 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.235 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.235 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.235 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.235 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.235 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.235 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.235 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.235 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.235 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.235 INFO:teuthology.orchestra.run.smithi043.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T10:44:36.235 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T10:44:36.235 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.236 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.236 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.236 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.236 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.236 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.236 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.236 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.236 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.236 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.236 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.236 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T10:44:36.236 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T10:44:36.236 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.237 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.237 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.237 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-04-25T10:44:36.237 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.237 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.237 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.237 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.237 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.237 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.237 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.237 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T10:44:36.237 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T10:44:36.237 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.238 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.238 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.238 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.238 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.238 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.238 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.238 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.238 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.238 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.238 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.238 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_URL": { 2024-04-25T10:44:36.238 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_URL", 2024-04-25T10:44:36.238 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.239 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.239 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.239 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.239 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.239 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.239 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.239 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.239 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.239 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.239 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.239 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T10:44:36.239 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T10:44:36.239 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.240 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.240 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.240 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-04-25T10:44:36.240 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.240 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.240 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.240 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.240 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.240 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.240 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.240 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T10:44:36.240 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T10:44:36.240 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.241 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.241 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.241 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.241 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.241 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.241 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.241 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.241 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.241 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.241 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.241 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T10:44:36.241 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T10:44:36.241 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.242 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.242 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.242 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.242 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.242 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.242 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.242 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.242 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.242 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.242 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.242 INFO:teuthology.orchestra.run.smithi043.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T10:44:36.242 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T10:44:36.243 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.243 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.243 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.243 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.243 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.243 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.243 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.243 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.243 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.243 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.243 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.243 INFO:teuthology.orchestra.run.smithi043.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T10:44:36.243 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T10:44:36.243 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.244 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.244 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.244 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.244 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.244 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.244 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.244 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.244 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.244 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.244 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.244 INFO:teuthology.orchestra.run.smithi043.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T10:44:36.244 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T10:44:36.244 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.245 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.245 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.245 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "[]", 2024-04-25T10:44:36.245 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.245 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.245 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.245 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.245 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.245 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.245 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.245 INFO:teuthology.orchestra.run.smithi043.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T10:44:36.245 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T10:44:36.245 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.245 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.246 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.246 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "{}", 2024-04-25T10:44:36.246 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.246 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.246 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.246 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.246 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.246 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.246 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.246 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T10:44:36.247 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T10:44:36.247 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.247 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.247 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.247 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.247 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.247 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.247 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.247 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.247 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.247 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.247 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.247 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T10:44:36.248 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T10:44:36.248 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.248 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.248 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.248 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.248 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.248 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.248 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.248 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.248 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.249 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.249 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.249 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T10:44:36.249 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T10:44:36.249 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.249 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.249 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.249 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.249 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.249 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.249 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.249 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.249 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.249 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.250 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.250 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T10:44:36.250 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T10:44:36.250 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.250 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.250 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.250 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.250 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.250 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.250 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.250 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.250 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.251 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.251 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.251 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T10:44:36.251 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T10:44:36.251 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.251 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.251 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.251 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.251 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.251 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.251 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.251 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.252 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.252 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.252 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.252 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T10:44:36.252 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T10:44:36.252 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.252 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.252 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.252 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.252 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.252 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.252 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.253 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.253 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.253 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.253 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.253 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T10:44:36.253 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T10:44:36.253 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.253 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.253 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.253 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.253 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.253 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.253 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.254 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.254 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.254 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.254 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.254 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T10:44:36.254 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T10:44:36.254 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.254 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.254 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.254 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.254 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.255 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.255 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.255 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.255 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.255 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.255 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.255 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T10:44:36.255 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T10:44:36.255 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.255 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.255 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.255 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.256 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.256 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.256 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.256 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.256 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.256 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.256 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.256 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T10:44:36.256 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T10:44:36.256 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.256 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.256 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.256 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.257 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.257 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.257 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.257 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.257 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.257 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.257 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.257 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T10:44:36.257 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T10:44:36.257 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.257 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.258 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.258 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T10:44:36.258 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.258 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.258 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.258 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.258 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.258 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.258 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.259 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T10:44:36.259 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T10:44:36.259 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.259 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.259 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.259 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:36.259 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.259 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.259 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.259 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.259 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.259 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.259 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.260 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T10:44:36.260 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T10:44:36.260 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.260 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.260 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.260 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8", 2024-04-25T10:44:36.260 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.260 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.260 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.260 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.260 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.260 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.260 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.261 INFO:teuthology.orchestra.run.smithi043.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T10:44:36.261 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T10:44:36.261 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.261 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.261 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.261 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "45", 2024-04-25T10:44:36.261 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.261 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.261 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.261 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.261 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.261 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.261 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.262 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T10:44:36.262 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T10:44:36.262 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.262 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.262 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.262 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.262 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.262 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.262 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.262 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.262 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.262 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.262 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.263 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T10:44:36.263 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T10:44:36.263 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.263 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.263 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.263 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-04-25T10:44:36.263 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.263 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.263 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.263 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.263 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.263 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.263 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.264 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T10:44:36.264 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T10:44:36.264 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.264 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.264 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.264 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.264 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.264 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.264 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.264 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.264 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.264 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.264 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.265 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T10:44:36.265 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T10:44:36.265 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.265 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.265 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.265 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.265 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.265 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.265 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.265 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.265 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.265 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.265 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.266 INFO:teuthology.orchestra.run.smithi043.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T10:44:36.266 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T10:44:36.266 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.266 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.266 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.266 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.266 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.266 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.266 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.266 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.266 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.266 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.266 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.267 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T10:44:36.267 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T10:44:36.267 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.267 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.267 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.267 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.267 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.267 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.267 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.267 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.267 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.267 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.267 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.268 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T10:44:36.268 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T10:44:36.268 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.268 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.268 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.268 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:36.268 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.268 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.268 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.268 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.268 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.268 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.268 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.269 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T10:44:36.269 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T10:44:36.269 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.269 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.269 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.269 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-25T10:44:36.269 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.269 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.269 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.269 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.269 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.269 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.269 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.270 INFO:teuthology.orchestra.run.smithi043.stdout: "cross_origin_url": { 2024-04-25T10:44:36.270 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cross_origin_url", 2024-04-25T10:44:36.270 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.270 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.270 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.270 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.270 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.270 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.270 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.270 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.270 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.270 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.271 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.271 INFO:teuthology.orchestra.run.smithi043.stdout: "crt_file": { 2024-04-25T10:44:36.271 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crt_file", 2024-04-25T10:44:36.271 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.271 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.271 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.271 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.271 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.271 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.271 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.271 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.271 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.271 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.272 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.272 INFO:teuthology.orchestra.run.smithi043.stdout: "debug": { 2024-04-25T10:44:36.272 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "debug", 2024-04-25T10:44:36.272 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.272 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.272 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.272 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.272 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.272 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.272 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.272 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable/disable debug options", 2024-04-25T10:44:36.272 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.272 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.272 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.273 INFO:teuthology.orchestra.run.smithi043.stdout: "jwt_token_ttl": { 2024-04-25T10:44:36.273 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "jwt_token_ttl", 2024-04-25T10:44:36.273 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.273 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.273 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.273 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "28800", 2024-04-25T10:44:36.273 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.273 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.273 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.273 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.273 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.273 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.273 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.274 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-04-25T10:44:36.274 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-04-25T10:44:36.274 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.274 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.274 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.274 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.274 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.274 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.274 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.274 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.274 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.274 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.274 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.275 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.275 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.275 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.275 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.275 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.275 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.275 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.275 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.275 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.275 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.275 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.275 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.275 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.276 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.276 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.276 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.276 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.276 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.276 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.276 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.276 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.276 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.276 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.276 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.276 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.276 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.276 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.277 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.277 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.277 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.277 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.277 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.277 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.277 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.277 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.277 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.277 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.277 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.277 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.277 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.278 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.278 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.278 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.278 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.278 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.278 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.278 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.278 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.278 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.278 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.278 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.278 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.279 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.279 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.279 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.279 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.279 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.279 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.279 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.279 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.279 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.279 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.279 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.279 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.279 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.279 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.280 INFO:teuthology.orchestra.run.smithi043.stdout: "motd": { 2024-04-25T10:44:36.280 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "motd", 2024-04-25T10:44:36.280 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.280 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.280 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.280 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.280 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.280 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.280 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.280 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "The message of the day", 2024-04-25T10:44:36.280 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.280 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.280 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.281 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect_resolve_ip_addr": { 2024-04-25T10:44:36.281 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T10:44:36.281 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.281 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.281 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.281 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.281 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.281 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.281 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.281 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.281 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.281 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.281 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.282 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-04-25T10:44:36.282 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-04-25T10:44:36.282 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.282 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.282 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.282 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-04-25T10:44:36.282 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.282 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.282 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.282 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.282 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.282 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.283 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.283 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-04-25T10:44:36.283 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-04-25T10:44:36.283 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.283 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.283 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.283 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8080", 2024-04-25T10:44:36.283 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.283 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.283 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.283 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.284 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.284 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.284 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.284 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.284 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.284 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.284 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.284 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.284 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.284 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.284 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.284 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.284 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.285 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.285 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.285 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.285 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-04-25T10:44:36.285 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-04-25T10:44:36.285 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.285 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.285 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.285 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.285 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.285 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.285 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.285 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.286 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.286 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.286 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.286 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl_server_port": { 2024-04-25T10:44:36.286 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl_server_port", 2024-04-25T10:44:36.286 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.286 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.286 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.286 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8443", 2024-04-25T10:44:36.286 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.286 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.286 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.286 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.287 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.287 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.287 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.287 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-04-25T10:44:36.287 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-04-25T10:44:36.287 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.287 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.287 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.287 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "redirect", 2024-04-25T10:44:36.287 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.287 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.288 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.288 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.288 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect" 2024-04-25T10:44:36.288 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.288 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.288 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.288 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.288 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.288 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-04-25T10:44:36.288 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-04-25T10:44:36.288 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.288 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.288 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.289 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-04-25T10:44:36.289 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-04-25T10:44:36.289 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-04-25T10:44:36.289 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.289 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.289 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.289 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.289 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.289 INFO:teuthology.orchestra.run.smithi043.stdout: "url_prefix": { 2024-04-25T10:44:36.289 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url_prefix", 2024-04-25T10:44:36.289 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.289 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.289 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.290 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.290 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.290 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.290 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.290 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.290 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.290 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.290 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.290 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "devicehealth", 2024-04-25T10:44:36.291 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.291 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.291 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.291 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_monitoring": { 2024-04-25T10:44:36.291 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_monitoring", 2024-04-25T10:44:36.291 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.291 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.291 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.291 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.291 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.291 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.291 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.291 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "monitor device health metrics", 2024-04-25T10:44:36.291 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.292 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.292 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.292 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.292 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.292 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.292 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.292 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.292 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.292 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.292 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.292 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.292 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.293 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.293 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.293 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.293 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.293 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.293 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.293 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.293 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.293 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.293 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.293 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.293 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.293 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.294 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.294 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.294 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.294 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.294 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.294 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.294 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.294 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.294 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.294 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.294 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.294 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.295 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.295 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.295 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.295 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.295 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.295 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.295 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.295 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.295 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.295 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.295 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.295 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.295 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.296 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.296 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.296 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.296 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.296 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.296 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.296 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.296 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.296 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.296 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.296 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.296 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.297 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.297 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.297 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.297 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.297 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.297 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.297 INFO:teuthology.orchestra.run.smithi043.stdout: "mark_out_threshold": { 2024-04-25T10:44:36.297 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mark_out_threshold", 2024-04-25T10:44:36.297 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.297 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.297 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.297 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2419200", 2024-04-25T10:44:36.298 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.298 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.298 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.298 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T10:44:36.298 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.298 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.298 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.298 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": { 2024-04-25T10:44:36.298 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_name", 2024-04-25T10:44:36.298 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.298 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.299 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.299 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "device_health_metrics", 2024-04-25T10:44:36.299 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.299 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.299 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.299 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T10:44:36.299 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.299 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.299 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.299 INFO:teuthology.orchestra.run.smithi043.stdout: "retention_period": { 2024-04-25T10:44:36.299 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retention_period", 2024-04-25T10:44:36.299 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.299 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.300 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.300 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15552000", 2024-04-25T10:44:36.300 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.300 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.300 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.300 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain device health metrics", 2024-04-25T10:44:36.300 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.300 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.300 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.300 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_frequency": { 2024-04-25T10:44:36.300 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_frequency", 2024-04-25T10:44:36.300 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.301 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.301 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.301 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-04-25T10:44:36.301 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.301 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.301 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.301 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T10:44:36.301 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.301 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.301 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.301 INFO:teuthology.orchestra.run.smithi043.stdout: "self_heal": { 2024-04-25T10:44:36.301 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "self_heal", 2024-04-25T10:44:36.301 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.302 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.302 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.302 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.302 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.302 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.302 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.302 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T10:44:36.302 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.302 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.302 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.302 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-25T10:44:36.302 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-25T10:44:36.303 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.303 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.303 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.303 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-25T10:44:36.303 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.303 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.303 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.303 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T10:44:36.303 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.303 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.303 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.303 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.304 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.304 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.304 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.304 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.304 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.304 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.304 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.304 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.304 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.304 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.304 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.304 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.305 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_threshold": { 2024-04-25T10:44:36.305 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_threshold", 2024-04-25T10:44:36.305 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.305 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.305 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.305 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7257600", 2024-04-25T10:44:36.305 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.305 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.305 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.305 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T10:44:36.305 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.305 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.305 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.305 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.306 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.306 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "diskprediction_local", 2024-04-25T10:44:36.306 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.306 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.306 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.306 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.306 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.306 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.306 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.306 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.306 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.307 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.307 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.307 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.307 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.307 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.307 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.307 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.307 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.307 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.307 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.307 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.307 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.307 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.307 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.307 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.308 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.308 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.308 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.308 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.308 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.308 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.308 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.308 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.308 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.308 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.308 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.308 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.308 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.308 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.309 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.309 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.309 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.309 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.309 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.309 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.309 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.309 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.309 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.309 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.309 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.309 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.309 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.309 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.310 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.310 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.310 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.310 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.310 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.310 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.310 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.310 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.310 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.310 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.310 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.310 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.311 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.311 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.311 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.311 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.311 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.311 INFO:teuthology.orchestra.run.smithi043.stdout: "predict_interval": { 2024-04-25T10:44:36.311 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predict_interval", 2024-04-25T10:44:36.311 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.311 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.311 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.311 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-04-25T10:44:36.312 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.312 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.312 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.312 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.312 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.312 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.312 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.312 INFO:teuthology.orchestra.run.smithi043.stdout: "predictor_model": { 2024-04-25T10:44:36.312 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predictor_model", 2024-04-25T10:44:36.312 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.312 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.312 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.312 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "prophetstor", 2024-04-25T10:44:36.313 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.313 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.313 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.313 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.313 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.313 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.313 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.313 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-25T10:44:36.313 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-25T10:44:36.313 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.314 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.314 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.314 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-25T10:44:36.314 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.314 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.314 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.314 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.314 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.314 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.314 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.314 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.314 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.315 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.315 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.315 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.315 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.315 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.315 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.315 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.315 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.315 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.315 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.315 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.315 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.315 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.316 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "influx", 2024-04-25T10:44:36.316 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": false, 2024-04-25T10:44:36.316 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "influxdb python module not found", 2024-04-25T10:44:36.316 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.316 INFO:teuthology.orchestra.run.smithi043.stdout: "batch_size": { 2024-04-25T10:44:36.316 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "batch_size", 2024-04-25T10:44:36.316 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.316 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.316 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.316 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5000", 2024-04-25T10:44:36.316 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.316 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.316 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.317 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T10:44:36.317 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.317 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.317 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.317 INFO:teuthology.orchestra.run.smithi043.stdout: "database": { 2024-04-25T10:44:36.317 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "database", 2024-04-25T10:44:36.317 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.317 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.317 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.317 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "ceph", 2024-04-25T10:44:36.317 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.317 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.317 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.318 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T10:44:36.318 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.318 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.318 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.318 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": { 2024-04-25T10:44:36.318 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hostname", 2024-04-25T10:44:36.318 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.318 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.318 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.318 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.318 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.319 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.319 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.319 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server hostname", 2024-04-25T10:44:36.319 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.319 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.319 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.319 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-25T10:44:36.319 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-25T10:44:36.319 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.319 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.319 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.319 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "30", 2024-04-25T10:44:36.320 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "5", 2024-04-25T10:44:36.320 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.320 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.320 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T10:44:36.320 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.320 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.320 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.320 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.320 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.320 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.320 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.320 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.321 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.321 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.321 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.321 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.321 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.321 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.321 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.321 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.321 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.321 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.321 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.321 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.322 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.322 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.322 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.322 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.322 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.322 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.322 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.322 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.322 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.322 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.322 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.322 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.323 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.323 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.323 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.323 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.323 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.323 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.323 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.323 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.323 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.323 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.323 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.323 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.323 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.324 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.324 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.324 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.324 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.324 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.324 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.324 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.324 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.324 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.324 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.324 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.324 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.324 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.324 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.325 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.325 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.325 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.325 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.325 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.325 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.325 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.325 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.325 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.325 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.325 INFO:teuthology.orchestra.run.smithi043.stdout: "password": { 2024-04-25T10:44:36.325 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "password", 2024-04-25T10:44:36.325 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.326 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.326 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.326 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.326 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.326 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.326 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.326 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "password of InfluxDB server user", 2024-04-25T10:44:36.326 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.326 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.326 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.326 INFO:teuthology.orchestra.run.smithi043.stdout: "port": { 2024-04-25T10:44:36.326 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "port", 2024-04-25T10:44:36.327 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.327 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.327 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.327 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8086", 2024-04-25T10:44:36.327 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.327 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.327 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.327 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server port", 2024-04-25T10:44:36.327 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.327 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.327 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.327 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.327 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.327 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.328 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.328 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.328 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.328 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.328 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.328 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.328 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.328 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.328 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.328 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.328 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-04-25T10:44:36.328 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-04-25T10:44:36.328 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.329 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.329 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.329 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "false", 2024-04-25T10:44:36.329 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.329 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.329 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.329 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T10:44:36.329 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.329 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.329 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.329 INFO:teuthology.orchestra.run.smithi043.stdout: "threads": { 2024-04-25T10:44:36.329 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threads", 2024-04-25T10:44:36.329 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.330 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.330 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.330 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-25T10:44:36.330 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-25T10:44:36.330 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "32", 2024-04-25T10:44:36.330 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.330 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T10:44:36.330 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.330 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.330 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.330 INFO:teuthology.orchestra.run.smithi043.stdout: "username": { 2024-04-25T10:44:36.330 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "username", 2024-04-25T10:44:36.330 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.331 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.331 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.331 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.331 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.331 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.331 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.331 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "username of InfluxDB server user", 2024-04-25T10:44:36.331 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.331 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.331 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.331 INFO:teuthology.orchestra.run.smithi043.stdout: "verify_ssl": { 2024-04-25T10:44:36.331 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "verify_ssl", 2024-04-25T10:44:36.331 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.331 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.332 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.332 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "true", 2024-04-25T10:44:36.332 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.332 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.332 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.332 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T10:44:36.332 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.332 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.332 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.332 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.332 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.333 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "insights", 2024-04-25T10:44:36.333 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.333 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.333 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.333 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.333 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.333 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.333 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.333 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.333 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.333 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.333 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.333 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.333 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.333 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.334 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.334 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.334 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.334 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.334 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.334 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.334 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.334 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.334 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.334 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.334 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.335 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.335 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.335 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.335 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.335 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.335 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.335 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.335 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.335 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.335 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.335 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.335 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.335 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.335 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.336 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.336 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.336 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.336 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.336 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.336 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.336 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.336 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.336 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.336 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.336 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.336 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.336 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.336 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.336 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.337 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.337 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.337 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.337 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.337 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.337 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.337 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.337 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.337 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.337 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.337 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.337 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.337 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.337 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.338 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.338 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.338 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.338 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.338 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.338 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.338 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.338 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.338 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.338 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.338 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.338 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.338 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.339 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.339 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.339 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.339 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "iostat", 2024-04-25T10:44:36.339 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.339 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.339 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.339 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.339 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.339 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.339 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.339 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.339 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.340 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.340 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.340 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.340 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.340 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.340 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.340 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.340 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.340 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.340 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.340 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.340 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.340 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.340 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.340 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.341 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.341 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.341 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.341 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.341 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.341 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.341 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.341 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.341 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.341 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.341 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.341 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.341 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.341 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.342 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.342 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.342 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.342 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.342 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.342 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.342 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.342 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.342 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.342 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.342 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.342 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.342 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.343 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.343 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.343 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.343 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.343 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.343 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.343 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.343 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.343 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.343 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.343 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.343 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.343 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.343 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.343 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.344 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.344 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.344 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.344 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.344 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.344 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.344 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.344 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.344 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.344 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.344 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.344 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.344 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.345 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.345 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.345 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.345 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.345 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.345 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "k8sevents", 2024-04-25T10:44:36.345 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.345 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.345 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.345 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_event_retention_days": { 2024-04-25T10:44:36.345 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ceph_event_retention_days", 2024-04-25T10:44:36.345 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.345 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.346 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.346 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7", 2024-04-25T10:44:36.346 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.346 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.346 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.346 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T10:44:36.346 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.346 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.346 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.346 INFO:teuthology.orchestra.run.smithi043.stdout: "config_check_secs": { 2024-04-25T10:44:36.346 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_check_secs", 2024-04-25T10:44:36.346 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.347 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.347 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.347 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:36.347 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "10", 2024-04-25T10:44:36.347 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.347 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.347 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T10:44:36.347 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.347 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.347 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.347 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.348 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.348 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.348 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.348 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.348 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.348 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.348 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.348 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.348 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.348 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.348 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.348 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.348 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.348 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.348 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.348 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.349 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.349 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.349 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.349 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.349 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.349 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.349 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.349 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.349 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.349 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.349 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.349 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.349 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.349 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.350 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.350 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.350 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.350 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.350 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.350 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.350 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.350 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.350 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.350 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.350 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.350 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.351 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.351 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.351 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.351 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.351 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.351 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.351 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.351 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.351 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.351 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.351 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.351 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.351 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.351 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.352 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.352 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.352 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.352 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.352 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.352 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.352 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.352 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.352 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.352 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.352 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.352 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.352 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.353 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.353 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.353 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.353 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.353 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.353 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.353 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.353 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.353 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.353 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.353 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.353 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "localpool", 2024-04-25T10:44:36.354 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.354 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.354 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.354 INFO:teuthology.orchestra.run.smithi043.stdout: "failure_domain": { 2024-04-25T10:44:36.354 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "failure_domain", 2024-04-25T10:44:36.354 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.354 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.354 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.354 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "host", 2024-04-25T10:44:36.354 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.354 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.354 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.354 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "failure domain for any created local pool", 2024-04-25T10:44:36.354 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T10:44:36.355 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.355 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.355 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.355 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.355 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.355 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.355 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.355 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.355 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.355 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.355 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.355 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.355 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.355 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.356 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.356 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.356 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.356 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.356 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.356 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.356 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.356 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.356 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.356 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.356 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.356 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.356 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.357 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.357 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.357 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.357 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.357 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.357 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.357 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.357 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.357 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.357 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.357 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.357 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.357 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.358 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.358 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.358 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.358 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.358 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.358 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.358 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.358 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.358 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.358 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.358 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.358 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.359 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.359 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.359 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.359 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.359 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.359 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.359 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.359 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.359 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.359 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.359 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.360 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.360 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.360 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.360 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.360 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.360 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": { 2024-04-25T10:44:36.360 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_size", 2024-04-25T10:44:36.360 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.360 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.360 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.360 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.360 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.360 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.360 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.361 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default min_size for any created local pool", 2024-04-25T10:44:36.361 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T10:44:36.361 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.361 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.361 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rep": { 2024-04-25T10:44:36.361 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "num_rep", 2024-04-25T10:44:36.361 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.361 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.361 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.361 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3", 2024-04-25T10:44:36.361 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.361 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.362 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.362 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default replica count for any created local pool", 2024-04-25T10:44:36.362 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.362 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.362 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.362 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": { 2024-04-25T10:44:36.362 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_num", 2024-04-25T10:44:36.362 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.362 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.362 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.362 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "128", 2024-04-25T10:44:36.362 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.363 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.363 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.363 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T10:44:36.363 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.363 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.363 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.363 INFO:teuthology.orchestra.run.smithi043.stdout: "prefix": { 2024-04-25T10:44:36.363 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prefix", 2024-04-25T10:44:36.363 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.363 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.363 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.363 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.363 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.364 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.364 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.364 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name prefix for any created local pool", 2024-04-25T10:44:36.364 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.364 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.364 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.364 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.364 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.364 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.364 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.364 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.364 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.364 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.364 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.364 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.365 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.365 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.365 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.365 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.365 INFO:teuthology.orchestra.run.smithi043.stdout: "subtree": { 2024-04-25T10:44:36.365 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "subtree", 2024-04-25T10:44:36.365 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.365 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.365 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.365 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "rack", 2024-04-25T10:44:36.365 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.365 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.365 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.365 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T10:44:36.366 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T10:44:36.366 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.366 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.366 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.366 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.366 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mds_autoscaler", 2024-04-25T10:44:36.366 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.366 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.366 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.366 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.366 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.366 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.366 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.367 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.367 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.367 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.367 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.367 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.367 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.367 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.367 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.367 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.367 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.367 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.367 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.367 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.367 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.368 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.368 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.368 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.368 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.368 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.368 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.368 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.368 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.368 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.368 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.368 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.368 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.368 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.369 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.369 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.369 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.369 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.369 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.369 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.369 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.369 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.369 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.369 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.370 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.370 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.370 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.370 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.370 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.370 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.370 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.370 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.370 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.370 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.370 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.370 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.371 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.371 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.371 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.371 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.371 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.371 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.371 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.371 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.371 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.371 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.371 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.371 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.371 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.371 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.372 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.372 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.372 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.372 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.372 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.372 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.372 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.372 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.372 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.372 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.372 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.372 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.372 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.373 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.373 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirroring", 2024-04-25T10:44:36.373 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.373 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.373 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.373 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.373 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.373 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.373 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.373 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.373 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.373 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.373 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.374 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.374 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.374 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.374 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.374 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.374 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.374 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.374 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.374 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.374 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.374 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.374 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.375 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.375 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.375 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.375 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.375 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.375 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.375 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.375 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.375 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.375 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.376 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.376 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.376 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.376 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.376 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.376 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.376 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.376 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.376 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.376 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.376 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.376 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.376 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.377 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.377 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.377 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.377 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.377 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.377 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.377 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.377 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.377 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.377 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.377 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.377 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.377 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.377 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.378 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.378 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.378 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.378 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.378 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.378 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.378 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.378 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.378 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.378 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.378 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.378 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.378 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.379 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.379 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.379 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.379 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.379 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.379 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.379 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.379 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.379 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.379 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.379 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.379 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "nfs", 2024-04-25T10:44:36.379 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.380 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.380 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.380 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.380 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.380 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.380 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.380 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.380 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.380 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.380 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.380 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.380 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.380 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.380 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.381 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.381 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.382 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.382 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.382 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.382 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.382 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.382 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.382 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.382 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.382 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.382 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.382 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.382 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.382 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.383 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.383 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.383 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.383 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.383 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.383 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.383 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.383 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.383 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.383 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.383 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.384 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.384 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.384 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.384 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.384 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.384 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.384 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.384 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.384 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.384 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.384 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.384 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.384 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.384 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.385 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.385 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.385 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.385 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.385 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.385 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.385 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.385 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.385 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.385 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.385 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.385 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.385 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.385 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.386 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.386 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.386 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.386 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.386 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.386 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.386 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.386 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.386 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.386 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.386 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.386 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.386 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.387 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.387 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.387 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-04-25T10:44:36.387 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.387 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.387 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.387 INFO:teuthology.orchestra.run.smithi043.stdout: "fail_fs": { 2024-04-25T10:44:36.387 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "fail_fs", 2024-04-25T10:44:36.387 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.387 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.387 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.387 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.388 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.388 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.388 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.388 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T10:44:36.388 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.388 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.388 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.388 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.388 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.388 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.388 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.388 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.388 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.388 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.389 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.389 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.389 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.389 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.389 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.389 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.389 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.389 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.389 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.389 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.389 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.389 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.389 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.390 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.390 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.390 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.390 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.390 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.390 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.390 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.390 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.390 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.390 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.390 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.390 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.391 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.391 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.391 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.391 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.391 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.391 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.391 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.391 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.391 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.391 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.391 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.392 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.392 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.392 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.392 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.392 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.392 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.392 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.392 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.392 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.392 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.392 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.392 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.393 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.393 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.393 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.393 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.393 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.393 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.393 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.393 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.393 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.393 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.393 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.393 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator": { 2024-04-25T10:44:36.393 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-04-25T10:44:36.393 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.394 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.394 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.394 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.394 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.394 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.394 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.394 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm", 2024-04-25T10:44:36.394 INFO:teuthology.orchestra.run.smithi043.stdout: "rook", 2024-04-25T10:44:36.394 INFO:teuthology.orchestra.run.smithi043.stdout: "test_orchestrator" 2024-04-25T10:44:36.394 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.394 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Orchestrator backend", 2024-04-25T10:44:36.394 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.394 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.394 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.395 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.395 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.395 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.395 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.395 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.395 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.395 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.395 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.395 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.395 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.395 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.395 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.396 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.396 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.396 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.396 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_perf_query", 2024-04-25T10:44:36.396 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.396 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.396 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.396 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.396 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.396 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.396 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.396 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.397 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.397 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.397 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.397 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.397 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.397 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.397 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.397 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.397 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.397 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.397 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.397 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.397 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.397 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.398 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.398 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.398 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.398 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.398 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.398 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.398 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.398 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.398 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.398 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.398 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.399 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.399 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.399 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.399 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.399 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.399 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.399 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.399 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.399 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.399 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.399 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.399 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.400 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.400 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.400 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.400 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.400 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.400 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.400 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.400 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.400 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.400 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.400 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.400 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.400 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.401 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.401 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.401 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.401 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.401 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.401 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.401 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.401 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.401 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.401 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.401 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.402 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.402 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.402 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.402 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.402 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.402 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.402 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.402 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.402 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.402 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.402 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.402 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.402 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.403 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.403 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.403 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_support", 2024-04-25T10:44:36.403 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.403 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.403 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.403 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.403 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.403 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.403 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.403 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.404 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.404 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.404 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.404 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.404 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.404 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.404 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.404 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.404 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.404 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.404 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.404 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.404 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.405 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.405 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.405 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.405 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.405 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.405 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.405 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.405 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.405 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.405 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.405 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.406 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.406 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.406 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.406 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.406 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.406 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.406 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.406 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.406 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.406 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.406 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.406 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.407 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.407 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.407 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.407 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.407 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.407 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.407 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.407 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.407 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.407 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.407 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.407 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.408 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.408 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.408 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.408 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.408 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.408 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.408 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.408 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.408 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.408 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.408 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.408 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.408 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.409 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.409 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.409 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.409 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.409 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.409 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.409 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.409 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.409 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.409 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.409 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.409 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.409 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.409 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.410 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.410 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_autoscaler", 2024-04-25T10:44:36.410 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.410 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.410 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.410 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.410 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.410 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.410 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.410 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.410 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.411 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.411 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.411 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.411 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.411 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.411 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.411 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.411 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.411 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.411 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.411 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.411 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.411 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.411 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.412 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.412 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.412 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.412 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.412 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.412 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.412 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.412 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.412 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.412 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.412 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.412 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.412 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.413 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.413 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.413 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.413 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.413 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.413 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.413 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.413 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.413 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.413 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.413 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.413 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.413 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.413 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.413 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.414 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.414 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.414 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.414 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.414 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.414 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.414 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.414 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.414 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.414 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.414 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.414 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.415 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.415 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.415 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.415 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.415 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.415 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.415 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-25T10:44:36.415 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-25T10:44:36.415 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.415 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.415 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.415 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-25T10:44:36.415 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.415 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.416 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.416 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.416 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.416 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.416 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.416 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.416 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.416 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.416 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.416 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.416 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.416 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.416 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.416 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.416 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.417 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.417 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.417 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.417 INFO:teuthology.orchestra.run.smithi043.stdout: "threshold": { 2024-04-25T10:44:36.417 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threshold", 2024-04-25T10:44:36.417 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:36.417 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.417 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.417 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-04-25T10:44:36.417 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1.0", 2024-04-25T10:44:36.417 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.417 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.417 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "scaling threshold", 2024-04-25T10:44:36.417 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T10:44:36.418 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.418 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.418 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.418 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.418 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "progress", 2024-04-25T10:44:36.418 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.418 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.418 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.418 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_pg_recovery_event": { 2024-04-25T10:44:36.418 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_pg_recovery_event", 2024-04-25T10:44:36.418 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.418 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.419 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.419 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.419 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.419 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.419 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.419 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T10:44:36.419 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.419 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.419 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.419 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-04-25T10:44:36.419 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-04-25T10:44:36.419 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.419 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.420 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.420 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.420 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.420 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.420 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.420 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.420 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.420 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.420 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.420 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.420 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.420 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.421 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.421 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.421 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.421 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.421 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.421 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.421 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.421 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.421 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.421 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.421 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.421 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.421 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.421 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.422 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.422 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.422 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.422 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.422 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.422 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.422 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.422 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.422 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.422 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.422 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.422 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.423 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.423 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.423 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.423 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.423 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.423 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.423 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.423 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.423 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.423 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.423 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.423 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.423 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.424 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.424 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.424 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.424 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.424 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.424 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.424 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.424 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.424 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.424 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.424 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.424 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.424 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.424 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.424 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.425 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.425 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.425 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.425 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.425 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.425 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.425 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.425 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.425 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.425 INFO:teuthology.orchestra.run.smithi043.stdout: "max_completed_events": { 2024-04-25T10:44:36.425 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_completed_events", 2024-04-25T10:44:36.425 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.426 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.426 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.426 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "50", 2024-04-25T10:44:36.426 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.426 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.426 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.426 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "number of past completed events to remember", 2024-04-25T10:44:36.426 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.426 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.426 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.426 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-25T10:44:36.426 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-25T10:44:36.427 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.427 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.427 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.427 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-25T10:44:36.427 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.427 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.427 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.427 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long the module is going to sleep", 2024-04-25T10:44:36.427 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.427 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.427 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.427 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.427 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.428 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.428 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.428 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.428 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.428 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.428 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.428 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.428 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.428 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.428 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.428 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.428 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.429 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.429 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus", 2024-04-25T10:44:36.429 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.429 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.429 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.429 INFO:teuthology.orchestra.run.smithi043.stdout: "cache": { 2024-04-25T10:44:36.429 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cache", 2024-04-25T10:44:36.429 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.429 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.429 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.429 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.429 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.430 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.430 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.430 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.430 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.430 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.430 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.430 INFO:teuthology.orchestra.run.smithi043.stdout: "exclude_perf_counters": { 2024-04-25T10:44:36.430 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "exclude_perf_counters", 2024-04-25T10:44:36.430 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.430 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.430 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.430 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.431 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.431 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.431 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.431 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T10:44:36.431 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T10:44:36.431 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.431 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.431 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.431 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.431 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.431 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.431 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.431 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.432 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.432 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.432 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.432 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.432 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.432 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.432 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.432 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.432 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.432 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.433 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.433 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.433 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.433 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.433 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.433 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.433 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.433 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.433 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.433 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.433 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.433 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.433 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.434 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.434 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.434 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.434 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.434 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.434 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.434 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.434 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.434 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.434 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.434 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.434 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.435 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.435 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.435 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.435 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.435 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.435 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.435 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.435 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.435 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.435 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.435 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.435 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.435 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.435 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.436 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.436 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.436 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.436 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.436 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.436 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.436 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.436 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.436 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.436 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.436 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.436 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools": { 2024-04-25T10:44:36.436 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools", 2024-04-25T10:44:36.436 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.437 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.437 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.437 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.437 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.437 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.437 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.437 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.437 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.437 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.437 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.437 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T10:44:36.437 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T10:44:36.437 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.438 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.438 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.438 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "300", 2024-04-25T10:44:36.438 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.438 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.438 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.438 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.438 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.438 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.438 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.438 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_interval": { 2024-04-25T10:44:36.438 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_interval", 2024-04-25T10:44:36.439 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:36.439 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.439 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.439 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15.0", 2024-04-25T10:44:36.439 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.439 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.439 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.439 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.439 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.439 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.439 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.439 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-04-25T10:44:36.440 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-04-25T10:44:36.440 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.440 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.440 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.440 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-04-25T10:44:36.440 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.440 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.440 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.440 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T10:44:36.440 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.440 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.440 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.440 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-04-25T10:44:36.441 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-04-25T10:44:36.441 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.441 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.441 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.441 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "9283", 2024-04-25T10:44:36.441 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.441 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.441 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.441 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T10:44:36.441 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.441 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.441 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.441 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.441 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.442 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.442 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.442 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.442 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.442 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.442 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.442 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.442 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.442 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.442 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.442 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.442 INFO:teuthology.orchestra.run.smithi043.stdout: "stale_cache_strategy": { 2024-04-25T10:44:36.442 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stale_cache_strategy", 2024-04-25T10:44:36.443 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.443 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.443 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.443 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "log", 2024-04-25T10:44:36.443 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.443 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.443 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.443 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.443 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.443 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.443 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.443 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-04-25T10:44:36.443 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-04-25T10:44:36.443 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.444 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.444 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.444 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "default", 2024-04-25T10:44:36.444 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.444 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.444 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.444 INFO:teuthology.orchestra.run.smithi043.stdout: "default", 2024-04-25T10:44:36.444 INFO:teuthology.orchestra.run.smithi043.stdout: "error" 2024-04-25T10:44:36.444 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.444 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.444 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.444 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.444 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.445 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-04-25T10:44:36.445 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-04-25T10:44:36.445 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.445 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.445 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.445 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-04-25T10:44:36.445 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-04-25T10:44:36.445 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-04-25T10:44:36.445 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.445 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.445 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.445 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.445 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.445 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.446 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.446 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_support", 2024-04-25T10:44:36.446 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.446 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.446 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.446 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.446 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.446 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.446 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.446 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.446 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.447 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.447 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.447 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.447 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.447 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.447 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.447 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.447 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.447 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.447 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.447 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.447 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.447 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.448 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.448 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.448 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.448 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.448 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.448 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.448 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.448 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.448 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.448 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.448 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.448 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.448 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.449 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.449 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.449 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.449 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.449 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.449 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.449 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.449 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.449 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.449 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.449 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.449 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.449 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.449 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.450 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.450 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.450 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.450 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.450 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.450 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.450 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.450 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.450 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.450 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.450 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.450 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.450 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.451 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.451 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.451 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.451 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.451 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.451 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.451 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.451 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_snap_create": { 2024-04-25T10:44:36.451 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_snap_create", 2024-04-25T10:44:36.451 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.451 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.451 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.451 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:36.451 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.451 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.452 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.452 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.452 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.452 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.452 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.452 INFO:teuthology.orchestra.run.smithi043.stdout: "mirror_snapshot_schedule": { 2024-04-25T10:44:36.452 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T10:44:36.452 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.452 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.452 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.452 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.452 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.452 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.452 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.453 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.453 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.453 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.453 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.453 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.453 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.453 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.453 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.453 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.453 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.453 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.454 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.454 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.454 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.454 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.454 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.454 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.454 INFO:teuthology.orchestra.run.smithi043.stdout: "trash_purge_schedule": { 2024-04-25T10:44:36.454 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "trash_purge_schedule", 2024-04-25T10:44:36.454 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.454 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.454 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.454 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.454 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.455 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.455 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.455 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.455 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.455 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.455 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.455 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "restful", 2024-04-25T10:44:36.455 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.455 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.456 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.456 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_auth": { 2024-04-25T10:44:36.456 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_auth", 2024-04-25T10:44:36.456 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.456 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.456 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.456 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.456 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.456 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.456 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.456 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.456 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.456 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.456 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.457 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-04-25T10:44:36.457 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-04-25T10:44:36.457 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.457 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.457 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.457 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.457 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.457 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.457 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.457 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.457 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.457 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.457 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.458 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.458 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.458 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.458 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.458 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.458 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.458 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.458 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.458 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.458 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.458 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.458 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.458 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.459 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.459 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.459 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.459 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.459 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.459 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.459 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.459 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.459 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.459 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.459 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.459 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.460 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.460 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.460 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.460 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.460 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.460 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.460 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.460 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.460 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.460 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.460 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.461 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.461 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.461 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.461 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.461 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.461 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.461 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.461 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.461 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.461 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.461 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.461 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.462 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.462 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.462 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.462 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.462 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.462 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.462 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.462 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.462 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.462 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.462 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.463 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.463 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.463 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.463 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.463 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.463 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.463 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.463 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-04-25T10:44:36.463 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-04-25T10:44:36.463 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.463 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.463 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.464 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.464 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.464 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.464 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.464 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.464 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.464 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.464 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.464 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-04-25T10:44:36.464 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-04-25T10:44:36.464 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.464 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.465 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.465 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.465 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.465 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.465 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.465 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.465 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.465 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.465 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.465 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.465 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.465 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.465 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.466 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.466 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.466 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.466 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.466 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.466 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.466 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.466 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.466 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.466 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.466 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.467 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rgw", 2024-04-25T10:44:36.467 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.467 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.467 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.467 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.467 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.467 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.467 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.467 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.467 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.467 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.467 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.467 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.468 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.468 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.468 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.468 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.468 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.468 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.468 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.468 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.468 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.468 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.468 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.469 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.469 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.469 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.469 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.469 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.469 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.469 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.469 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.469 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.469 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.469 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.469 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.469 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.470 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.470 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.470 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.470 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.470 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.470 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.470 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.470 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.470 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.470 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.470 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.470 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.471 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.471 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.471 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.471 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.471 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.471 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.471 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.471 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.471 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.471 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.471 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.471 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.471 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.472 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.472 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.472 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.472 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.472 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.472 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.472 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.472 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.472 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.472 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.472 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.472 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.473 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.473 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.473 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.473 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.473 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.473 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.473 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.473 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.473 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.473 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rook", 2024-04-25T10:44:36.474 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.474 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.474 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.474 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.474 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.474 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.474 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.474 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.474 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.474 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.474 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.474 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.474 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.475 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.475 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.475 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.475 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.475 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.475 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.475 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.475 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.475 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.475 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.475 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.475 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.476 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.476 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.476 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.476 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.476 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.476 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.476 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.476 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.476 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.476 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.476 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.476 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.476 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.477 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.477 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.477 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.477 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.477 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.477 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.477 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.477 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.477 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.477 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.477 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.477 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.477 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.477 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.477 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.478 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.478 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.478 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.478 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.478 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.478 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.478 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.478 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.478 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.478 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.478 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.478 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.479 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.479 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.479 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.479 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.479 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.479 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.479 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.479 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.479 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.479 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.479 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.479 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.479 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.479 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.479 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.480 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.480 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.480 INFO:teuthology.orchestra.run.smithi043.stdout: "storage_class": { 2024-04-25T10:44:36.480 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "storage_class", 2024-04-25T10:44:36.480 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.480 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.480 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.480 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "local", 2024-04-25T10:44:36.480 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.480 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.480 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.480 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T10:44:36.480 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.481 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.481 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.481 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.481 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.481 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "selftest", 2024-04-25T10:44:36.481 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.481 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.481 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.481 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.481 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.481 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.481 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.482 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.482 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.482 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.482 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.482 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.482 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.482 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.482 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.482 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.482 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.482 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.482 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.482 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.482 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.483 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.483 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.483 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.483 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.483 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.483 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.483 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.483 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.483 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.483 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.483 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.483 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.484 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.484 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.484 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.484 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.484 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.484 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.484 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.484 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.484 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.484 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.484 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.484 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.484 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.485 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.485 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.485 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.485 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.485 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.485 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.485 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.485 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.485 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.485 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.485 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.485 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.485 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.486 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.486 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.486 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.486 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.486 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.486 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.486 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.486 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.486 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.486 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.486 INFO:teuthology.orchestra.run.smithi043.stdout: "roption1": { 2024-04-25T10:44:36.486 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption1", 2024-04-25T10:44:36.486 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.487 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.487 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.487 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.487 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.487 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.487 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.487 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.487 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.487 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.487 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.487 INFO:teuthology.orchestra.run.smithi043.stdout: "roption2": { 2024-04-25T10:44:36.487 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption2", 2024-04-25T10:44:36.487 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.488 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.488 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.488 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "xyz", 2024-04-25T10:44:36.488 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.488 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.488 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.488 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.488 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.488 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.488 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.488 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption1": { 2024-04-25T10:44:36.488 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption1", 2024-04-25T10:44:36.488 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.489 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.489 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.489 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.489 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.489 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.489 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.489 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.489 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.489 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.489 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.489 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption2": { 2024-04-25T10:44:36.489 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption2", 2024-04-25T10:44:36.490 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.490 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.490 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.490 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.490 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.490 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.490 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.490 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.490 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.490 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.490 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.490 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption3": { 2024-04-25T10:44:36.491 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption3", 2024-04-25T10:44:36.491 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:36.491 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.491 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.491 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.491 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.491 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.491 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.491 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.491 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.491 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.491 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.491 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption4": { 2024-04-25T10:44:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption4", 2024-04-25T10:44:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption5": { 2024-04-25T10:44:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption5", 2024-04-25T10:44:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption6": { 2024-04-25T10:44:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption6", 2024-04-25T10:44:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption7": { 2024-04-25T10:44:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption7", 2024-04-25T10:44:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-25T10:44:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "42", 2024-04-25T10:44:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: "testkey": { 2024-04-25T10:44:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testkey", 2024-04-25T10:44:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "testlkey": { 2024-04-25T10:44:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testlkey", 2024-04-25T10:44:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "testnewline": { 2024-04-25T10:44:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testnewline", 2024-04-25T10:44:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-04-25T10:44:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_m_granularity": { 2024-04-25T10:44:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_m_granularity", 2024-04-25T10:44:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T10:44:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "dump_on_update": { 2024-04-25T10:44:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dump_on_update", 2024-04-25T10:44:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "dump database to debug log on update", 2024-04-25T10:44:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stats", 2024-04-25T10:44:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "status", 2024-04-25T10:44:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telegraf", 2024-04-25T10:44:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: "address": { 2024-04-25T10:44:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "address", 2024-04-25T10:44:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T10:44:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-25T10:44:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-25T10:44:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15", 2024-04-25T10:44:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telemetry", 2024-04-25T10:44:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_basic": { 2024-04-25T10:44:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_basic", 2024-04-25T10:44:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T10:44:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_crash": { 2024-04-25T10:44:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_crash", 2024-04-25T10:44:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T10:44:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_device": { 2024-04-25T10:44:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_device", 2024-04-25T10:44:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T10:44:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_ident": { 2024-04-25T10:44:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_ident", 2024-04-25T10:44:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T10:44:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_perf": { 2024-04-25T10:44:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_perf", 2024-04-25T10:44:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T10:44:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "contact": { 2024-04-25T10:44:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "contact", 2024-04-25T10:44:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "description": { 2024-04-25T10:44:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "description", 2024-04-25T10:44:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "device_url": { 2024-04-25T10:44:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_url", 2024-04-25T10:44:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T10:44:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-04-25T10:44:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-04-25T10:44:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-25T10:44:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-25T10:44:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "24", 2024-04-25T10:44:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "8", 2024-04-25T10:44:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "last_opt_revision": { 2024-04-25T10:44:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "last_opt_revision", 2024-04-25T10:44:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1", 2024-04-25T10:44:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard": { 2024-04-25T10:44:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard", 2024-04-25T10:44:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard_description": { 2024-04-25T10:44:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard_description", 2024-04-25T10:44:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "organization": { 2024-04-25T10:44:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "organization", 2024-04-25T10:44:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "proxy": { 2024-04-25T10:44:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "proxy", 2024-04-25T10:44:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "url": { 2024-04-25T10:44:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url", 2024-04-25T10:44:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T10:44:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "test_orchestrator", 2024-04-25T10:44:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "volumes", 2024-04-25T10:44:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_clones": { 2024-04-25T10:44:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_clones", 2024-04-25T10:44:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4", 2024-04-25T10:44:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T10:44:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: "periodic_async_work": { 2024-04-25T10:44:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "periodic_async_work", 2024-04-25T10:44:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Periodically check for async work", 2024-04-25T10:44:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_delay": { 2024-04-25T10:44:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_delay", 2024-04-25T10:44:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T10:44:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_no_wait": { 2024-04-25T10:44:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T10:44:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T10:44:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix", 2024-04-25T10:44:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "discovery_interval": { 2024-04-25T10:44:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "discovery_interval", 2024-04-25T10:44:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-25T10:44:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "100", 2024-04-25T10:44:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "identifier": { 2024-04-25T10:44:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "identifier", 2024-04-25T10:44:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-25T10:44:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-25T10:44:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-25T10:44:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_host": { 2024-04-25T10:44:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_host", 2024-04-25T10:44:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_port": { 2024-04-25T10:44:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_port", 2024-04-25T10:44:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10051", 2024-04-25T10:44:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_sender": { 2024-04-25T10:44:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_sender", 2024-04-25T10:44:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T10:44:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T10:44:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 25046, 2024-04-25T10:44:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "y", 2024-04-25T10:44:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_features": 4540701547738038271, 2024-04-25T10:44:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: "available_modules": [ 2024-04-25T10:44:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "alerts", 2024-04-25T10:44:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-25T10:44:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-25T10:44:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-25T10:44:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T10:44:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.587 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.589 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.589 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.589 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.592 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.592 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.592 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.592 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.592 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.592 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.592 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.592 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.592 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.592 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.592 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.592 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.593 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_destination": { 2024-04-25T10:44:36.593 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_destination", 2024-04-25T10:44:36.593 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.593 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.593 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.593 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.593 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.593 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.593 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.593 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email address to send alerts to", 2024-04-25T10:44:36.593 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.593 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.593 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.593 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_from_name": { 2024-04-25T10:44:36.594 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_from_name", 2024-04-25T10:44:36.594 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.594 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.594 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.594 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "Ceph", 2024-04-25T10:44:36.594 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.594 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.594 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.594 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email From: name", 2024-04-25T10:44:36.594 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.594 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.594 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.594 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_host": { 2024-04-25T10:44:36.595 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_host", 2024-04-25T10:44:36.595 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.595 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.595 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.595 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.595 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.595 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.595 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.595 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP server", 2024-04-25T10:44:36.595 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.595 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.595 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.595 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_password": { 2024-04-25T10:44:36.595 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_password", 2024-04-25T10:44:36.596 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.596 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.596 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.596 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.596 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.596 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.596 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.596 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Password to authenticate with", 2024-04-25T10:44:36.596 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.596 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.596 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.596 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_port": { 2024-04-25T10:44:36.596 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_port", 2024-04-25T10:44:36.597 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.597 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.597 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.597 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "465", 2024-04-25T10:44:36.597 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.597 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.597 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.597 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP port", 2024-04-25T10:44:36.597 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.597 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.597 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.597 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_sender": { 2024-04-25T10:44:36.597 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_sender", 2024-04-25T10:44:36.598 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.598 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.598 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.598 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.598 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.598 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.598 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.598 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP envelope sender", 2024-04-25T10:44:36.598 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.598 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.598 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.599 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_ssl": { 2024-04-25T10:44:36.599 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_ssl", 2024-04-25T10:44:36.599 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.599 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.599 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.599 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.599 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.599 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.599 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.599 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T10:44:36.599 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.599 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.599 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.599 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_user": { 2024-04-25T10:44:36.600 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_user", 2024-04-25T10:44:36.600 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.600 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.600 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.600 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.600 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.600 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.600 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.600 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "User to authenticate as", 2024-04-25T10:44:36.600 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.600 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.600 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.601 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.601 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.601 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.601 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.601 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.601 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.601 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.601 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.601 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.601 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.601 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.601 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.601 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.601 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.602 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.602 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "balancer", 2024-04-25T10:44:36.602 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.602 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.602 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.602 INFO:teuthology.orchestra.run.smithi043.stdout: "active": { 2024-04-25T10:44:36.602 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "active", 2024-04-25T10:44:36.602 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.602 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.602 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.602 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.603 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.603 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.603 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.603 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T10:44:36.603 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.603 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.603 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.603 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_time": { 2024-04-25T10:44:36.603 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_time", 2024-04-25T10:44:36.603 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.603 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.603 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.603 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0000", 2024-04-25T10:44:36.604 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.604 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.604 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.604 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T10:44:36.604 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T10:44:36.604 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.604 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.604 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_weekday": { 2024-04-25T10:44:36.604 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_weekday", 2024-04-25T10:44:36.604 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-25T10:44:36.604 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.604 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.605 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.605 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-04-25T10:44:36.605 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-04-25T10:44:36.605 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.605 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T10:44:36.605 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T10:44:36.605 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.605 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.605 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_max_iterations": { 2024-04-25T10:44:36.605 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_max_iterations", 2024-04-25T10:44:36.605 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-25T10:44:36.605 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.606 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.606 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "25", 2024-04-25T10:44:36.606 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-25T10:44:36.606 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "250", 2024-04-25T10:44:36.606 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.606 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T10:44:36.606 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.606 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.606 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.606 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_metrics": { 2024-04-25T10:44:36.606 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_metrics", 2024-04-25T10:44:36.607 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.607 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.607 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.607 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T10:44:36.607 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.607 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.607 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.607 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T10:44:36.607 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T10:44:36.607 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.607 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.608 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_step": { 2024-04-25T10:44:36.608 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_step", 2024-04-25T10:44:36.608 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:36.608 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.608 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.608 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.5", 2024-04-25T10:44:36.608 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0.001", 2024-04-25T10:44:36.608 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "0.999", 2024-04-25T10:44:36.608 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.608 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "aggressiveness of optimization", 2024-04-25T10:44:36.608 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T10:44:36.608 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.609 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.609 INFO:teuthology.orchestra.run.smithi043.stdout: "end_time": { 2024-04-25T10:44:36.609 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_time", 2024-04-25T10:44:36.609 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.609 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.609 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.609 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2359", 2024-04-25T10:44:36.609 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.609 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.609 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.609 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T10:44:36.609 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T10:44:36.609 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.610 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.610 INFO:teuthology.orchestra.run.smithi043.stdout: "end_weekday": { 2024-04-25T10:44:36.610 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_weekday", 2024-04-25T10:44:36.610 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-25T10:44:36.610 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.610 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.610 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.610 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-04-25T10:44:36.610 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-04-25T10:44:36.610 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.610 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T10:44:36.610 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T10:44:36.610 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.611 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.611 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.611 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.611 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.611 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.611 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.611 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.611 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.611 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.611 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.611 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.611 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.611 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.612 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.612 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.612 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.612 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.612 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.612 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.612 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.612 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.612 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.612 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.612 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.612 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.612 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.613 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.613 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.613 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.613 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.613 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.613 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.613 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.613 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.613 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.613 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.613 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.613 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.613 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.613 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.614 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.614 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.614 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.614 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.614 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.614 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.614 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.614 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.614 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.614 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.614 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.614 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.614 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.614 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.615 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.615 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.615 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.615 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.615 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.615 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.615 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.615 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.615 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.615 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.615 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.615 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.615 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.616 INFO:teuthology.orchestra.run.smithi043.stdout: "min_score": { 2024-04-25T10:44:36.616 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_score", 2024-04-25T10:44:36.616 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:36.616 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.616 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.616 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.616 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.616 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.616 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.616 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T10:44:36.616 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.617 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.617 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.617 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-04-25T10:44:36.617 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-04-25T10:44:36.617 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.617 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.617 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.617 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "upmap", 2024-04-25T10:44:36.617 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.617 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.617 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.617 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-compat", 2024-04-25T10:44:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: "none", 2024-04-25T10:44:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: "read", 2024-04-25T10:44:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap", 2024-04-25T10:44:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap-read" 2024-04-25T10:44:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Balancer mode", 2024-04-25T10:44:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ids": { 2024-04-25T10:44:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_ids", 2024-04-25T10:44:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.619 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.619 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.619 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.619 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.619 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.619 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T10:44:36.619 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.619 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.619 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.619 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-25T10:44:36.619 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-25T10:44:36.619 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.619 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.620 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.620 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-25T10:44:36.620 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.620 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.620 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.620 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T10:44:36.620 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.620 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.620 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.620 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.620 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.620 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.620 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.621 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.621 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.621 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.621 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.621 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.621 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.621 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.621 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.621 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.621 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_deviation": { 2024-04-25T10:44:36.621 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_deviation", 2024-04-25T10:44:36.621 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.621 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.622 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.622 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-25T10:44:36.622 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-25T10:44:36.622 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.622 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.622 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T10:44:36.622 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T10:44:36.622 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.622 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.622 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_optimizations": { 2024-04-25T10:44:36.622 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_optimizations", 2024-04-25T10:44:36.622 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-25T10:44:36.623 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.623 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.623 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:36.623 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.623 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.623 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.623 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T10:44:36.623 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.623 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.623 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.623 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.623 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.624 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm", 2024-04-25T10:44:36.624 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.624 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.624 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.624 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_down_multiplier": { 2024-04-25T10:44:36.624 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_down_multiplier", 2024-04-25T10:44:36.624 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:36.624 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.624 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.624 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-04-25T10:44:36.624 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.624 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.624 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.625 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T10:44:36.625 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.625 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.625 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.625 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_refresh_rate": { 2024-04-25T10:44:36.625 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_refresh_rate", 2024-04-25T10:44:36.625 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.625 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.625 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.625 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "20", 2024-04-25T10:44:36.625 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.625 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.626 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.626 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T10:44:36.626 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.626 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.626 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.626 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_starting_port": { 2024-04-25T10:44:36.626 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_starting_port", 2024-04-25T10:44:36.626 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.626 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.626 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.626 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4721", 2024-04-25T10:44:36.626 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.626 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.627 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.627 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T10:44:36.627 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.627 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.627 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.627 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_ptrace": { 2024-04-25T10:44:36.627 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_ptrace", 2024-04-25T10:44:36.627 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.627 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.627 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.627 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.627 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.628 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.628 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.628 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T10:44:36.628 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T10:44:36.628 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.628 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.628 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_interval": { 2024-04-25T10:44:36.628 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_interval", 2024-04-25T10:44:36.628 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.628 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.628 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.628 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-25T10:44:36.628 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.629 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.629 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.629 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T10:44:36.629 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.629 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.629 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.629 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_memory_target_ratio": { 2024-04-25T10:44:36.629 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T10:44:36.629 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:36.629 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.629 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.629 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.7", 2024-04-25T10:44:36.629 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.630 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.630 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.630 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T10:44:36.630 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.630 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.630 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.630 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm_log_destination": { 2024-04-25T10:44:36.630 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm_log_destination", 2024-04-25T10:44:36.630 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.630 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.630 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.630 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.631 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.631 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.631 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.631 INFO:teuthology.orchestra.run.smithi043.stdout: "file", 2024-04-25T10:44:36.631 INFO:teuthology.orchestra.run.smithi043.stdout: "file,syslog", 2024-04-25T10:44:36.631 INFO:teuthology.orchestra.run.smithi043.stdout: "syslog" 2024-04-25T10:44:36.631 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.631 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T10:44:36.631 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.631 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.631 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.631 INFO:teuthology.orchestra.run.smithi043.stdout: "cgroups_split": { 2024-04-25T10:44:36.631 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cgroups_split", 2024-04-25T10:44:36.632 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.632 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.632 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.632 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.632 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.632 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.632 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.632 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T10:44:36.632 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.632 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.632 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.632 INFO:teuthology.orchestra.run.smithi043.stdout: "config_checks_enabled": { 2024-04-25T10:44:36.632 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_checks_enabled", 2024-04-25T10:44:36.633 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.633 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.633 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.633 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.633 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.633 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.633 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.633 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T10:44:36.633 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.633 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.633 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.633 INFO:teuthology.orchestra.run.smithi043.stdout: "config_dashboard": { 2024-04-25T10:44:36.634 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_dashboard", 2024-04-25T10:44:36.634 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.634 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.634 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.634 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.634 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.634 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.634 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.634 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T10:44:36.634 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.634 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.634 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.635 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_alertmanager": { 2024-04-25T10:44:36.635 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_alertmanager", 2024-04-25T10:44:36.635 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.635 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.635 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.635 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T10:44:36.635 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.635 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.635 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.635 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-25T10:44:36.635 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.635 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.635 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.636 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_base": { 2024-04-25T10:44:36.636 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_base", 2024-04-25T10:44:36.636 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.636 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.636 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.636 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T10:44:36.636 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.636 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.636 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.636 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Container image name, without the tag", 2024-04-25T10:44:36.636 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.636 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.637 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.637 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_elasticsearch": { 2024-04-25T10:44:36.637 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_elasticsearch", 2024-04-25T10:44:36.637 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.637 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.637 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.637 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T10:44:36.637 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.637 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.637 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.637 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "elasticsearch container image", 2024-04-25T10:44:36.638 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.638 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.638 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.638 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_grafana": { 2024-04-25T10:44:36.638 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_grafana", 2024-04-25T10:44:36.638 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.638 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.638 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.638 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T10:44:36.638 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.638 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.638 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.639 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-25T10:44:36.639 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.639 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.639 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.639 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_haproxy": { 2024-04-25T10:44:36.639 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_haproxy", 2024-04-25T10:44:36.639 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.639 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.639 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.639 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T10:44:36.639 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.639 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.640 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.640 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "HAproxy container image", 2024-04-25T10:44:36.640 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.640 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.640 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.640 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_agent": { 2024-04-25T10:44:36.640 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_agent", 2024-04-25T10:44:36.640 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.640 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.640 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.640 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T10:44:36.640 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.641 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.641 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.641 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger agent container image", 2024-04-25T10:44:36.641 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.641 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.641 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.641 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_collector": { 2024-04-25T10:44:36.641 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_collector", 2024-04-25T10:44:36.641 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.641 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.641 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.642 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T10:44:36.642 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.642 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.642 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.642 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger collector container image", 2024-04-25T10:44:36.642 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.642 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.642 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.642 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_query": { 2024-04-25T10:44:36.642 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_query", 2024-04-25T10:44:36.642 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T10:44:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger query container image", 2024-04-25T10:44:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_keepalived": { 2024-04-25T10:44:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_keepalived", 2024-04-25T10:44:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T10:44:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Keepalived container image", 2024-04-25T10:44:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_loki": { 2024-04-25T10:44:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_loki", 2024-04-25T10:44:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T10:44:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Loki container image", 2024-04-25T10:44:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_node_exporter": { 2024-04-25T10:44:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_node_exporter", 2024-04-25T10:44:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T10:44:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-25T10:44:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_nvmeof": { 2024-04-25T10:44:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_nvmeof", 2024-04-25T10:44:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T10:44:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Nvme-of container image", 2024-04-25T10:44:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_prometheus": { 2024-04-25T10:44:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_prometheus", 2024-04-25T10:44:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T10:44:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-25T10:44:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_promtail": { 2024-04-25T10:44:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_promtail", 2024-04-25T10:44:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T10:44:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Promtail container image", 2024-04-25T10:44:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_samba": { 2024-04-25T10:44:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_samba", 2024-04-25T10:44:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T10:44:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Samba/SMB container image", 2024-04-25T10:44:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_snmp_gateway": { 2024-04-25T10:44:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_snmp_gateway", 2024-04-25T10:44:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T10:44:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SNMP Gateway container image", 2024-04-25T10:44:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "container_init": { 2024-04-25T10:44:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_init", 2024-04-25T10:44:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T10:44:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "daemon_cache_timeout": { 2024-04-25T10:44:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "daemon_cache_timeout", 2024-04-25T10:44:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-25T10:44:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T10:44:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "default_cephadm_command_timeout": { 2024-04-25T10:44:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T10:44:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "900", 2024-04-25T10:44:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T10:44:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "default_registry": { 2024-04-25T10:44:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_registry", 2024-04-25T10:44:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io", 2024-04-25T10:44:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T10:44:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "device_cache_timeout": { 2024-04-25T10:44:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_cache_timeout", 2024-04-25T10:44:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1800", 2024-04-25T10:44:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache device inventory", 2024-04-25T10:44:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "device_enhanced_scan": { 2024-04-25T10:44:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_enhanced_scan", 2024-04-25T10:44:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T10:44:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "facts_cache_timeout": { 2024-04-25T10:44:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "facts_cache_timeout", 2024-04-25T10:44:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-25T10:44:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache host facts data", 2024-04-25T10:44:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "grafana_dashboards_path": { 2024-04-25T10:44:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "grafana_dashboards_path", 2024-04-25T10:44:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T10:44:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T10:44:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "host_check_interval": { 2024-04-25T10:44:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "host_check_interval", 2024-04-25T10:44:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-25T10:44:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to perform a host check", 2024-04-25T10:44:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "hw_monitoring": { 2024-04-25T10:44:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hw_monitoring", 2024-04-25T10:44:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T10:44:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "inventory_list_all": { 2024-04-25T10:44:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "inventory_list_all", 2024-04-25T10:44:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T10:44:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "log_refresh_metadata": { 2024-04-25T10:44:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_refresh_metadata", 2024-04-25T10:44:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T10:44:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T10:44:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T10:44:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T10:44:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T10:44:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T10:44:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T10:44:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "*", 2024-04-25T10:44:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T10:44:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "max_count_per_host": { 2024-04-25T10:44:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_count_per_host", 2024-04-25T10:44:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of daemons per service per host", 2024-04-25T10:44:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd_draining_count": { 2024-04-25T10:44:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_osd_draining_count", 2024-04-25T10:44:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T10:44:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "migration_current": { 2024-04-25T10:44:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "migration_current", 2024-04-25T10:44:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "internal - do not modify", 2024-04-25T10:44:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-04-25T10:44:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-04-25T10:44:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "root", 2024-04-25T10:44:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm-package", 2024-04-25T10:44:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "root" 2024-04-25T10:44:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T10:44:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "oob_default_addr": { 2024-04-25T10:44:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "oob_default_addr", 2024-04-25T10:44:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "169.254.1.1", 2024-04-25T10:44:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T10:44:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "prometheus_alerts_path": { 2024-04-25T10:44:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus_alerts_path", 2024-04-25T10:44:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T10:44:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T10:44:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_insecure": { 2024-04-25T10:44:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_insecure", 2024-04-25T10:44:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T10:44:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_password": { 2024-04-25T10:44:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_password", 2024-04-25T10:44:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T10:44:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_url": { 2024-04-25T10:44:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_url", 2024-04-25T10:44:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T10:44:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_username": { 2024-04-25T10:44:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_username", 2024-04-25T10:44:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T10:44:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "secure_monitoring_stack": { 2024-04-25T10:44:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "secure_monitoring_stack", 2024-04-25T10:44:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T10:44:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "service_discovery_port": { 2024-04-25T10:44:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "service_discovery_port", 2024-04-25T10:44:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8765", 2024-04-25T10:44:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "cephadm service discovery port", 2024-04-25T10:44:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "ssh_config_file": { 2024-04-25T10:44:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssh_config_file", 2024-04-25T10:44:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T10:44:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "use_agent": { 2024-04-25T10:44:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_agent", 2024-04-25T10:44:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T10:44:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "use_repo_digest": { 2024-04-25T10:44:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_repo_digest", 2024-04-25T10:44:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T10:44:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_failed_host_check": { 2024-04-25T10:44:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_failed_host_check", 2024-04-25T10:44:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T10:44:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_daemons": { 2024-04-25T10:44:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_daemons", 2024-04-25T10:44:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T10:44:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_hosts": { 2024-04-25T10:44:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_hosts", 2024-04-25T10:44:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T10:44:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crash", 2024-04-25T10:44:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "retain_interval": { 2024-04-25T10:44:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retain_interval", 2024-04-25T10:44:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "31536000", 2024-04-25T10:44:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T10:44:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_recent_interval": { 2024-04-25T10:44:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_recent_interval", 2024-04-25T10:44:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1209600", 2024-04-25T10:44:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T10:44:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dashboard", 2024-04-25T10:44:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T10:44:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T10:44:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T10:44:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T10:44:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T10:44:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T10:44:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_ENABLED": { 2024-04-25T10:44:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T10:44:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T10:44:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T10:44:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T10:44:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T10:44:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T10:44:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T10:44:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T10:44:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T10:44:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T10:44:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T10:44:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T10:44:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T10:44:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T10:44:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T10:44:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T10:44:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T10:44:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T10:44:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T10:44:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T10:44:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T10:44:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T10:44:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T10:44:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-04-25T10:44:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T10:44:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T10:44:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_URL": { 2024-04-25T10:44:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_URL", 2024-04-25T10:44:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T10:44:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T10:44:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-04-25T10:44:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T10:44:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T10:44:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T10:44:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T10:44:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T10:44:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T10:44:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T10:44:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T10:44:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T10:44:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T10:44:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "[]", 2024-04-25T10:44:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T10:44:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T10:44:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "{}", 2024-04-25T10:44:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T10:44:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T10:44:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T10:44:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T10:44:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T10:44:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T10:44:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T10:44:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T10:44:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T10:44:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T10:44:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T10:44:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T10:44:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T10:44:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T10:44:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T10:44:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T10:44:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T10:44:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T10:44:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T10:44:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T10:44:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T10:44:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T10:44:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T10:44:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T10:44:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T10:44:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T10:44:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T10:44:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8", 2024-04-25T10:44:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T10:44:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T10:44:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "45", 2024-04-25T10:44:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T10:44:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T10:44:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T10:44:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T10:44:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-04-25T10:44:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T10:44:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T10:44:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T10:44:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T10:44:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T10:44:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T10:44:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T10:44:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T10:44:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T10:44:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T10:44:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T10:44:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T10:44:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-25T10:44:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.751 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.751 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.751 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.751 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.751 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.751 INFO:teuthology.orchestra.run.smithi043.stdout: "cross_origin_url": { 2024-04-25T10:44:36.751 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cross_origin_url", 2024-04-25T10:44:36.751 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.751 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.751 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.751 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.751 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.752 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.752 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.752 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.752 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.752 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.752 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.752 INFO:teuthology.orchestra.run.smithi043.stdout: "crt_file": { 2024-04-25T10:44:36.752 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crt_file", 2024-04-25T10:44:36.752 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.752 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.752 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.752 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.752 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.752 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.753 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.753 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.753 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.753 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.753 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.753 INFO:teuthology.orchestra.run.smithi043.stdout: "debug": { 2024-04-25T10:44:36.753 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "debug", 2024-04-25T10:44:36.753 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.753 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.753 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.753 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.753 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.753 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.754 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.754 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable/disable debug options", 2024-04-25T10:44:36.754 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.754 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.754 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.754 INFO:teuthology.orchestra.run.smithi043.stdout: "jwt_token_ttl": { 2024-04-25T10:44:36.754 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "jwt_token_ttl", 2024-04-25T10:44:36.754 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.754 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.754 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.754 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "28800", 2024-04-25T10:44:36.754 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.755 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.755 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.755 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.755 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.755 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.755 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.755 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-04-25T10:44:36.755 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-04-25T10:44:36.755 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.755 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.755 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.755 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.756 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.756 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.756 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.756 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.756 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.756 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.756 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.756 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.756 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.756 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.756 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.756 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.756 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.757 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.757 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.757 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.757 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.757 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.757 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.757 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.757 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.757 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.757 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.757 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.757 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.757 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.758 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.758 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.758 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.758 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.758 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.758 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.758 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.758 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.758 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.758 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.758 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.758 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.759 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.759 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.759 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.759 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.759 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.759 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.759 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.759 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.759 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.759 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.759 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.759 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.759 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.759 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.760 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.760 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.760 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.760 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.760 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.760 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.760 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.760 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.760 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.760 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.760 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.760 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.760 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.761 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.761 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.761 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.761 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.761 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.761 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.761 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.761 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.761 INFO:teuthology.orchestra.run.smithi043.stdout: "motd": { 2024-04-25T10:44:36.761 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "motd", 2024-04-25T10:44:36.761 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.761 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.761 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.762 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.762 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.762 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.762 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.762 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "The message of the day", 2024-04-25T10:44:36.762 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.762 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.762 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.762 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect_resolve_ip_addr": { 2024-04-25T10:44:36.762 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T10:44:36.762 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.762 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.762 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.762 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.763 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.763 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.763 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.763 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.763 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.763 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.763 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.763 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-04-25T10:44:36.763 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-04-25T10:44:36.763 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.763 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.763 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.764 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-04-25T10:44:36.764 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.764 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.764 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.764 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.764 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.764 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.764 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.764 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-04-25T10:44:36.764 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-04-25T10:44:36.764 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.764 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.764 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.765 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8080", 2024-04-25T10:44:36.765 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.765 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.765 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.765 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.765 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.765 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.765 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.765 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.765 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.765 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.765 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.765 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.766 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.766 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.766 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.766 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.766 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.766 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.766 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.766 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.766 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-04-25T10:44:36.766 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-04-25T10:44:36.766 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.766 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.766 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.767 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.767 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.767 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.767 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.767 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.767 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.767 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.767 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.767 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl_server_port": { 2024-04-25T10:44:36.767 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl_server_port", 2024-04-25T10:44:36.767 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.767 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.767 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.767 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8443", 2024-04-25T10:44:36.768 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.768 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.768 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.768 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.768 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.768 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.768 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.768 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-04-25T10:44:36.768 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-04-25T10:44:36.768 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.768 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.768 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.768 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "redirect", 2024-04-25T10:44:36.769 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.769 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.769 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.769 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.769 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect" 2024-04-25T10:44:36.769 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.769 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.769 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.769 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.769 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.769 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-04-25T10:44:36.769 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-04-25T10:44:36.769 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.769 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.770 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.770 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-04-25T10:44:36.770 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-04-25T10:44:36.770 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-04-25T10:44:36.770 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.770 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.770 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.770 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.770 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.770 INFO:teuthology.orchestra.run.smithi043.stdout: "url_prefix": { 2024-04-25T10:44:36.770 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url_prefix", 2024-04-25T10:44:36.770 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.771 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.771 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.771 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.771 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.771 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.771 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.771 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.771 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.771 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.771 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.771 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.772 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.772 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "devicehealth", 2024-04-25T10:44:36.772 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.772 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.772 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.772 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_monitoring": { 2024-04-25T10:44:36.772 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_monitoring", 2024-04-25T10:44:36.772 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.772 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.772 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.772 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.772 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.773 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.773 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.773 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "monitor device health metrics", 2024-04-25T10:44:36.773 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.773 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.773 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.773 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.773 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.773 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.773 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.773 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.773 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.773 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.773 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.774 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.774 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.774 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.774 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.774 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.774 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.774 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.774 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.774 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.774 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.774 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.774 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.774 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.775 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.775 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.775 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.775 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.775 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.775 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.775 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.775 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.775 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.775 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.775 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.775 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.775 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.775 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.776 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.776 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.776 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.776 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.776 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.776 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.776 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.776 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.776 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.776 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.776 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.776 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.776 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.776 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.777 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.777 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.777 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.777 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.777 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.777 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.777 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.777 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.777 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.777 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.777 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.777 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: "mark_out_threshold": { 2024-04-25T10:44:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mark_out_threshold", 2024-04-25T10:44:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2419200", 2024-04-25T10:44:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.779 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.779 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.779 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T10:44:36.779 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.779 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.779 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.779 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": { 2024-04-25T10:44:36.779 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_name", 2024-04-25T10:44:36.779 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.779 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.779 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.779 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "device_health_metrics", 2024-04-25T10:44:36.779 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.780 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.780 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.780 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T10:44:36.780 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.780 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.780 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.780 INFO:teuthology.orchestra.run.smithi043.stdout: "retention_period": { 2024-04-25T10:44:36.780 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retention_period", 2024-04-25T10:44:36.780 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.780 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.780 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.780 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15552000", 2024-04-25T10:44:36.780 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.781 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.781 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.781 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain device health metrics", 2024-04-25T10:44:36.781 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.781 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.781 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.781 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_frequency": { 2024-04-25T10:44:36.781 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_frequency", 2024-04-25T10:44:36.781 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.781 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.781 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.781 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-04-25T10:44:36.781 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.782 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.782 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.782 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T10:44:36.782 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.782 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.782 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.782 INFO:teuthology.orchestra.run.smithi043.stdout: "self_heal": { 2024-04-25T10:44:36.782 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "self_heal", 2024-04-25T10:44:36.782 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.782 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.782 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.782 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.782 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.783 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.783 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.783 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T10:44:36.783 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.783 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.783 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.783 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-25T10:44:36.783 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-25T10:44:36.783 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.783 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.783 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.783 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-25T10:44:36.784 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.784 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.784 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.784 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T10:44:36.784 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.784 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.784 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.784 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.784 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.784 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.784 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.784 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.784 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.784 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.785 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.785 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.785 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.785 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.785 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.785 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.785 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_threshold": { 2024-04-25T10:44:36.785 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_threshold", 2024-04-25T10:44:36.785 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.785 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.785 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.785 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7257600", 2024-04-25T10:44:36.785 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.785 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.786 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.786 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T10:44:36.786 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.786 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.786 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.786 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.786 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.786 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "diskprediction_local", 2024-04-25T10:44:36.786 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.786 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.786 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.786 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.787 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.787 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.787 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.787 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.787 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.787 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.787 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.787 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.787 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.787 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.787 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.787 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.787 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.787 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.788 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.788 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.788 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.788 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.788 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.788 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.788 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.788 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.788 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.788 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.788 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.788 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.788 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.788 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.789 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.789 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.789 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.789 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.789 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.789 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.789 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.789 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.789 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.789 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.789 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.789 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.789 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.790 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.790 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.790 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.790 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.790 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.790 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.790 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.790 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.790 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.790 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.790 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.790 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.790 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.790 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.791 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.791 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.791 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.791 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.791 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.791 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.791 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.791 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.791 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.791 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.791 INFO:teuthology.orchestra.run.smithi043.stdout: "predict_interval": { 2024-04-25T10:44:36.791 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predict_interval", 2024-04-25T10:44:36.792 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.792 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.792 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.792 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-04-25T10:44:36.792 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.792 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.792 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.792 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.792 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.792 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.792 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.793 INFO:teuthology.orchestra.run.smithi043.stdout: "predictor_model": { 2024-04-25T10:44:36.793 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predictor_model", 2024-04-25T10:44:36.793 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.793 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.793 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.793 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "prophetstor", 2024-04-25T10:44:36.793 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.793 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.793 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.793 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.793 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.793 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.793 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.794 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-25T10:44:36.794 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-25T10:44:36.794 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.794 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.794 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.794 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-25T10:44:36.794 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.794 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.794 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.794 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.794 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.794 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.794 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.797 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.797 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.797 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.798 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.798 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.798 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.798 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.798 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.798 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.798 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.798 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.798 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.798 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.798 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.798 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.799 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "influx", 2024-04-25T10:44:36.799 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": false, 2024-04-25T10:44:36.799 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "influxdb python module not found", 2024-04-25T10:44:36.799 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.799 INFO:teuthology.orchestra.run.smithi043.stdout: "batch_size": { 2024-04-25T10:44:36.799 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "batch_size", 2024-04-25T10:44:36.799 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.799 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.799 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.799 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5000", 2024-04-25T10:44:36.799 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.799 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.800 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.800 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T10:44:36.800 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.800 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.800 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.800 INFO:teuthology.orchestra.run.smithi043.stdout: "database": { 2024-04-25T10:44:36.800 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "database", 2024-04-25T10:44:36.800 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.800 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.800 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.800 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "ceph", 2024-04-25T10:44:36.800 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.800 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.801 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.801 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T10:44:36.801 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.801 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.801 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.801 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": { 2024-04-25T10:44:36.801 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hostname", 2024-04-25T10:44:36.801 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.801 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.801 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.801 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.801 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.801 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.801 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.802 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server hostname", 2024-04-25T10:44:36.802 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.802 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.802 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.802 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-25T10:44:36.802 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-25T10:44:36.802 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.802 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.802 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.802 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "30", 2024-04-25T10:44:36.802 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "5", 2024-04-25T10:44:36.802 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.802 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.803 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T10:44:36.803 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.803 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.803 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.803 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.803 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.803 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.803 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.803 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.803 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.803 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.803 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.803 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.803 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.804 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.804 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.804 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.804 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.804 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.804 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.804 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.804 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.804 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.804 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.804 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.804 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.804 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.805 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.805 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.805 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.805 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.805 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.805 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.805 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.805 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.805 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.805 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.805 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.805 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.805 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.806 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.806 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.806 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.806 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.806 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.806 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.806 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.806 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.806 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.806 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.806 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.806 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.807 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.807 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.807 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.807 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.807 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.807 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.807 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.807 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.807 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.807 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.807 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.807 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.808 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.808 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.808 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.808 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.808 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.808 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.808 INFO:teuthology.orchestra.run.smithi043.stdout: "password": { 2024-04-25T10:44:36.808 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "password", 2024-04-25T10:44:36.808 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.808 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.808 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.809 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.809 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.809 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.809 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.809 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "password of InfluxDB server user", 2024-04-25T10:44:36.809 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.809 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.809 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.809 INFO:teuthology.orchestra.run.smithi043.stdout: "port": { 2024-04-25T10:44:36.809 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "port", 2024-04-25T10:44:36.809 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.809 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.809 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.810 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8086", 2024-04-25T10:44:36.810 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.810 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.810 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.810 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server port", 2024-04-25T10:44:36.810 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.810 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.810 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.810 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.810 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.810 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.810 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.810 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.811 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.811 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.811 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.811 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.811 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.811 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.811 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.811 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.811 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-04-25T10:44:36.811 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-04-25T10:44:36.811 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.811 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.811 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.812 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "false", 2024-04-25T10:44:36.812 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.812 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.812 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.812 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T10:44:36.812 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.812 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.812 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.812 INFO:teuthology.orchestra.run.smithi043.stdout: "threads": { 2024-04-25T10:44:36.812 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threads", 2024-04-25T10:44:36.812 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.812 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.812 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.813 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-25T10:44:36.813 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-25T10:44:36.813 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "32", 2024-04-25T10:44:36.813 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.813 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T10:44:36.813 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.813 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.813 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.813 INFO:teuthology.orchestra.run.smithi043.stdout: "username": { 2024-04-25T10:44:36.813 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "username", 2024-04-25T10:44:36.813 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.814 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.814 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.814 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.814 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.814 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.814 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.814 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "username of InfluxDB server user", 2024-04-25T10:44:36.814 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.814 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.814 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.814 INFO:teuthology.orchestra.run.smithi043.stdout: "verify_ssl": { 2024-04-25T10:44:36.814 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "verify_ssl", 2024-04-25T10:44:36.815 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.815 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.815 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.815 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "true", 2024-04-25T10:44:36.815 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.815 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.815 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.815 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T10:44:36.815 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.815 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.815 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.815 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.815 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.816 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "insights", 2024-04-25T10:44:36.816 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.816 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.816 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.816 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.816 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.816 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.816 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.816 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.816 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.816 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.817 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.817 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.817 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.817 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.817 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.817 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.817 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.817 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.817 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.817 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.817 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.817 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.817 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.818 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.818 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.818 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.818 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.818 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.818 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.818 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.818 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.818 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.818 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.818 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.818 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.818 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.819 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.819 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.819 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.819 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.819 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.819 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.819 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.819 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.819 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.819 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.819 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.819 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.820 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.820 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.820 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.820 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.820 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.820 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.820 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.820 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.820 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.820 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.820 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.820 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.821 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.821 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.821 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.821 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.821 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.821 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.821 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.821 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.821 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.821 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.821 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.821 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.821 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.822 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.822 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.822 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.822 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.822 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.822 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.822 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.822 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.822 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.822 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.822 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.823 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "iostat", 2024-04-25T10:44:36.823 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.823 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.823 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.823 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.823 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.823 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.823 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.823 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.823 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.823 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.823 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.823 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.824 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.824 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.824 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.824 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.824 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.824 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.824 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.824 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.824 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.824 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.824 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.824 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.824 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.825 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.825 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.825 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.825 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.825 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.825 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.825 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.825 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.825 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.825 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.825 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.825 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.825 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.825 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.826 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.826 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.826 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.826 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.826 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.826 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.826 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.826 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.826 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.826 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.826 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.826 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.826 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.826 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.827 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.827 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.827 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.827 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.827 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.827 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.827 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.827 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.827 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.827 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.827 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.827 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.827 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.828 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.828 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.828 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.828 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.828 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.828 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.828 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.828 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.828 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.828 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.828 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.828 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.828 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.828 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.829 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.829 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.829 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.829 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.829 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "k8sevents", 2024-04-25T10:44:36.829 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.829 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.829 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.829 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_event_retention_days": { 2024-04-25T10:44:36.829 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ceph_event_retention_days", 2024-04-25T10:44:36.829 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.829 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.830 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.830 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7", 2024-04-25T10:44:36.830 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.830 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.830 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.830 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T10:44:36.830 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.830 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.830 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.830 INFO:teuthology.orchestra.run.smithi043.stdout: "config_check_secs": { 2024-04-25T10:44:36.830 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_check_secs", 2024-04-25T10:44:36.830 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.831 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.831 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.831 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:36.831 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "10", 2024-04-25T10:44:36.831 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.831 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.831 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T10:44:36.831 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.831 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.831 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.831 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.831 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.831 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.832 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.832 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.832 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.832 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.832 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.832 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.832 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.832 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.832 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.832 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.832 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.832 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.832 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.832 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.833 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.833 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.833 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.833 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.833 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.833 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.833 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.833 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.833 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.833 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.833 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.833 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.833 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.833 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.834 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.834 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.834 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.834 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.834 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.834 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.834 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.834 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.834 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.834 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.834 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.834 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.834 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.835 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.835 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.835 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.835 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.835 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.835 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.835 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.835 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.835 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.835 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.835 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.835 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.835 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.835 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.836 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.836 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.836 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.836 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.836 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.836 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.836 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.836 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.836 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.836 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.836 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.836 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.837 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.837 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.837 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.837 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.837 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.837 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.837 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.837 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.837 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.837 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.837 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.837 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "localpool", 2024-04-25T10:44:36.838 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.838 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.838 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.838 INFO:teuthology.orchestra.run.smithi043.stdout: "failure_domain": { 2024-04-25T10:44:36.838 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "failure_domain", 2024-04-25T10:44:36.838 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.838 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.838 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.838 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "host", 2024-04-25T10:44:36.838 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.838 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.838 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.838 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "failure domain for any created local pool", 2024-04-25T10:44:36.839 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T10:44:36.839 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.839 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.839 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.839 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.839 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.839 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.839 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.839 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.839 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.839 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.839 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.839 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.839 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.840 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.840 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.840 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.840 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.840 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.840 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.840 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.840 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.840 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.840 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.840 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.840 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.841 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.841 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.841 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.841 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.841 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.841 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.841 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.841 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.841 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.841 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.841 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.841 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.841 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.841 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.842 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.842 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.842 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.842 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.842 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.842 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.842 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.842 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.842 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.842 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.842 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.842 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.842 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.843 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.843 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.843 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.843 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.843 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.843 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.843 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.843 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.843 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.843 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.843 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.843 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.843 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.844 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.844 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.844 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.844 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": { 2024-04-25T10:44:36.844 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_size", 2024-04-25T10:44:36.844 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.844 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.844 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.844 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.844 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.844 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.844 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.844 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default min_size for any created local pool", 2024-04-25T10:44:36.845 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T10:44:36.845 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.845 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.845 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rep": { 2024-04-25T10:44:36.845 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "num_rep", 2024-04-25T10:44:36.845 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.845 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.845 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.845 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3", 2024-04-25T10:44:36.845 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.845 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.845 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.845 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default replica count for any created local pool", 2024-04-25T10:44:36.846 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.846 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.846 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.846 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": { 2024-04-25T10:44:36.846 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_num", 2024-04-25T10:44:36.846 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.846 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.846 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.846 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "128", 2024-04-25T10:44:36.846 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.846 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.846 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.846 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T10:44:36.847 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.847 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.847 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.847 INFO:teuthology.orchestra.run.smithi043.stdout: "prefix": { 2024-04-25T10:44:36.847 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prefix", 2024-04-25T10:44:36.847 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.847 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.847 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.847 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.847 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.847 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.847 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.847 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name prefix for any created local pool", 2024-04-25T10:44:36.848 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.848 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.848 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.848 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.848 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.848 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.848 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.848 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.848 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.848 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.848 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.848 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.848 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.848 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.849 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.849 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.849 INFO:teuthology.orchestra.run.smithi043.stdout: "subtree": { 2024-04-25T10:44:36.849 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "subtree", 2024-04-25T10:44:36.849 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.849 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.849 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.849 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "rack", 2024-04-25T10:44:36.849 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.849 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.849 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.849 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T10:44:36.850 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T10:44:36.850 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.850 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.850 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.850 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.850 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mds_autoscaler", 2024-04-25T10:44:36.850 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.850 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.850 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.850 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.850 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.851 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.851 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.851 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.851 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.851 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.851 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.851 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.851 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.851 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.851 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.851 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.851 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.851 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.852 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.852 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.852 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.852 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.852 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.852 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.852 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.852 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.852 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.852 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.852 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.852 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.852 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.853 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.853 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.853 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.853 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.853 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.853 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.853 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.853 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.853 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.853 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.853 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.853 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.853 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.853 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.854 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.854 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.854 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.854 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.854 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.854 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.854 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.854 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.854 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.854 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.854 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.854 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.855 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.855 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.855 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.855 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.855 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.855 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.855 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.855 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.855 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.855 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.855 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.855 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.855 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.856 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.856 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.856 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.856 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.856 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.856 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.856 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.856 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.856 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.856 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.856 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.856 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.856 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.856 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.857 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirroring", 2024-04-25T10:44:36.857 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.857 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.857 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.857 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.857 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.857 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.857 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.857 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.857 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.857 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.857 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.857 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.858 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.858 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.858 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.858 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.858 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.858 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.858 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.858 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.858 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.858 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.858 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.858 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.858 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.859 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.859 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.859 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.859 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.859 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.859 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.859 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.859 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.859 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.859 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.859 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.859 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.859 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.859 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.860 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.860 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.860 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.860 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.860 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.860 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.860 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.860 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.860 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.860 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.860 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.860 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.860 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.860 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.861 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.861 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.861 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.861 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.861 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.861 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.861 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.861 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.861 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.861 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.861 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.861 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.861 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.861 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.862 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.862 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.862 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.862 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.862 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.862 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.862 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.862 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.862 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.862 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.862 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.863 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.863 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.863 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.863 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.863 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.863 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.863 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "nfs", 2024-04-25T10:44:36.863 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.863 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.863 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.863 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.863 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.864 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.864 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.864 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.864 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.864 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.864 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.864 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.864 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.864 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.864 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.864 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.864 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.864 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.865 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.865 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.865 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.865 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.865 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.865 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.865 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.865 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.865 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.865 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.865 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.865 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.865 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.866 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.866 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.866 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.866 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.866 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.866 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.866 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.866 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.866 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.866 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.866 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.866 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.866 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.867 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.867 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.867 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.867 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.867 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.867 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.867 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.867 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.867 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.867 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.867 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.867 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.867 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.868 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.868 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.868 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.868 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.868 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.868 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.868 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.868 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.868 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.868 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.868 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.868 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.868 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.869 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.869 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.869 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.869 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.869 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.869 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.869 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.869 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.869 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.869 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.869 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.869 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.869 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.869 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.870 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-04-25T10:44:36.870 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.870 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.870 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.870 INFO:teuthology.orchestra.run.smithi043.stdout: "fail_fs": { 2024-04-25T10:44:36.870 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "fail_fs", 2024-04-25T10:44:36.870 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.870 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.870 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.870 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.870 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.870 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.871 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.871 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T10:44:36.871 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.871 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.871 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.871 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.871 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.871 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.871 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.871 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.871 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.871 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.871 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.871 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.872 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.872 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.872 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.872 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.872 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.872 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.872 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.872 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.872 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.872 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.872 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.872 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.872 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.873 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.873 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.873 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.873 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.873 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.873 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.873 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.873 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.873 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.873 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.873 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.873 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.873 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.873 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.874 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.874 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.874 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.874 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.874 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.874 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.874 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.874 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.874 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.874 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.874 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.874 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.874 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.875 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.875 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.875 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.875 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.875 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.875 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.875 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.875 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.875 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.875 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.875 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.875 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.875 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.875 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.876 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.876 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.876 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.876 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator": { 2024-04-25T10:44:36.876 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-04-25T10:44:36.876 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.876 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.876 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.876 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.876 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.877 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.877 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.877 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm", 2024-04-25T10:44:36.877 INFO:teuthology.orchestra.run.smithi043.stdout: "rook", 2024-04-25T10:44:36.877 INFO:teuthology.orchestra.run.smithi043.stdout: "test_orchestrator" 2024-04-25T10:44:36.877 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.877 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Orchestrator backend", 2024-04-25T10:44:36.877 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.877 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.877 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.877 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.877 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.877 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.878 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.878 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.878 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.878 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.878 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.878 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.878 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.878 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.878 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.878 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.878 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.878 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.879 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_perf_query", 2024-04-25T10:44:36.879 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.879 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.879 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.879 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.879 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.879 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.879 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.879 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.879 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.879 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.879 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.879 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.880 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.880 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.880 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.880 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.880 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.880 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.880 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.880 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.880 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.880 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.880 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.880 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.880 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.881 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.881 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.881 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.881 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.881 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.881 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.881 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.881 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.881 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.881 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.881 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.881 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.882 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.882 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.882 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.882 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.882 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.882 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.882 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.882 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.882 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.882 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.882 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.882 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.882 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.882 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.883 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.883 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.883 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.883 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.883 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.883 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.883 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.883 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.883 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.883 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.883 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.883 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.883 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.884 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.884 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.884 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.884 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.884 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.884 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.884 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.884 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.884 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.884 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.884 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.884 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.884 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.884 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.885 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.885 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.885 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.885 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.885 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.885 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.885 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_support", 2024-04-25T10:44:36.885 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.885 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.885 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.885 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.886 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.886 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.886 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.886 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.886 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.886 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.886 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.886 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.886 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.886 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.886 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.886 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.886 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.886 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.887 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.887 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.887 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.887 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.887 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.887 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.887 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.887 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.887 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.887 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.887 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.887 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.887 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.887 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.888 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.888 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.888 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.888 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.888 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.888 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.888 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.888 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.888 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.888 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.888 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.888 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.888 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.889 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.889 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.889 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.889 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.889 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.889 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.889 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.889 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.889 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.889 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.889 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.889 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.889 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.892 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_autoscaler", 2024-04-25T10:44:36.892 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.892 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.892 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.892 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.892 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.892 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.892 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.892 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.892 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.892 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.893 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.893 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.893 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.893 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.893 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.893 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.893 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.893 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.893 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.893 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.893 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.893 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.893 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.894 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.894 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.894 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.894 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.894 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.894 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.894 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.894 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.894 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.894 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.894 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.894 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.894 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.895 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.895 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.895 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.895 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.895 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.895 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.895 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.895 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.895 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.895 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.895 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.895 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.895 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.895 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.896 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.896 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.896 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.896 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.896 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.896 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.896 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.896 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.896 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.896 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.896 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.896 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.896 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.897 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.897 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.897 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.897 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.897 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.897 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.897 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-25T10:44:36.897 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-25T10:44:36.897 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.897 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.897 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.897 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-25T10:44:36.897 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.898 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.898 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.898 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.898 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.898 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.898 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.898 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.898 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.898 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.898 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.898 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.898 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.899 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.899 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.899 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.899 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.899 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.899 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.899 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.899 INFO:teuthology.orchestra.run.smithi043.stdout: "threshold": { 2024-04-25T10:44:36.899 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threshold", 2024-04-25T10:44:36.899 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:36.899 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.899 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.899 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-04-25T10:44:36.900 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1.0", 2024-04-25T10:44:36.900 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.900 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.900 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "scaling threshold", 2024-04-25T10:44:36.900 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T10:44:36.900 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.900 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.900 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "progress", 2024-04-25T10:44:36.900 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.900 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.900 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.901 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_pg_recovery_event": { 2024-04-25T10:44:36.901 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_pg_recovery_event", 2024-04-25T10:44:36.901 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.901 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.901 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.901 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.901 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.901 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.901 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.901 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T10:44:36.901 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.901 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.901 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.902 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-04-25T10:44:36.902 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-04-25T10:44:36.902 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.902 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.902 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.902 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.902 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.902 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.902 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.902 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.902 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.902 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.902 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.903 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.903 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.903 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.903 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.903 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.903 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.903 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.903 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.903 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.903 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.903 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.903 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.903 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.903 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.904 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.904 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.904 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.904 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.904 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.904 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.904 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.904 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.904 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.904 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.904 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.904 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.904 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.905 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.905 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.905 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.905 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.905 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.905 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.905 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.905 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.905 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.905 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.905 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.905 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.905 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.906 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.906 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.906 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.906 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.906 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.906 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.906 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.906 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.906 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.906 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.906 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.906 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.907 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.907 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.907 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.907 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.907 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.907 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.907 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.907 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.907 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.907 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.907 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.907 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.907 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.908 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.908 INFO:teuthology.orchestra.run.smithi043.stdout: "max_completed_events": { 2024-04-25T10:44:36.908 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_completed_events", 2024-04-25T10:44:36.908 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.908 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.908 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.908 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "50", 2024-04-25T10:44:36.908 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.908 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.908 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.908 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "number of past completed events to remember", 2024-04-25T10:44:36.908 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.909 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.909 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.909 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-25T10:44:36.909 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-25T10:44:36.909 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:36.909 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.909 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.909 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-25T10:44:36.909 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.909 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.909 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.909 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long the module is going to sleep", 2024-04-25T10:44:36.909 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.910 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.910 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.910 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.910 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.910 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.910 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.910 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.910 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.910 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.910 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.910 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.910 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.910 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.911 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.911 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.911 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.911 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.911 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus", 2024-04-25T10:44:36.911 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.911 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.911 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.911 INFO:teuthology.orchestra.run.smithi043.stdout: "cache": { 2024-04-25T10:44:36.911 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cache", 2024-04-25T10:44:36.911 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.911 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.911 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.912 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.912 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.912 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.912 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.912 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.912 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.912 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.912 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.912 INFO:teuthology.orchestra.run.smithi043.stdout: "exclude_perf_counters": { 2024-04-25T10:44:36.912 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "exclude_perf_counters", 2024-04-25T10:44:36.912 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.912 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.912 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.913 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.913 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.913 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.913 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.913 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T10:44:36.913 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T10:44:36.913 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.913 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.913 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.913 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.913 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.913 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.913 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.914 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.914 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.914 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.914 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.914 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.914 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.914 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.914 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.914 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.914 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.914 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.914 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.914 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.914 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.915 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.915 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.915 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.915 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.915 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.915 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.915 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.915 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.915 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.915 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.915 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.915 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.915 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.915 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.916 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.916 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.916 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.916 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.916 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.916 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.916 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.916 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.916 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.916 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.916 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.916 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.916 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.917 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.917 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.917 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.917 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.917 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.917 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.917 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.917 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.917 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.917 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.917 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.917 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.917 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.918 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.918 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.918 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.918 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.918 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.918 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.918 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.918 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools": { 2024-04-25T10:44:36.918 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools", 2024-04-25T10:44:36.918 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.918 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.918 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.918 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.919 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.919 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.919 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.919 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.919 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.919 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.919 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.919 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T10:44:36.919 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T10:44:36.919 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.919 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.919 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.919 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "300", 2024-04-25T10:44:36.920 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.920 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.920 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.920 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.920 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.920 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.920 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.920 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_interval": { 2024-04-25T10:44:36.920 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_interval", 2024-04-25T10:44:36.920 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:36.920 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.920 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.920 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15.0", 2024-04-25T10:44:36.921 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.921 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.921 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.921 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.921 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.921 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.921 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.921 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-04-25T10:44:36.921 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-04-25T10:44:36.921 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.921 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.921 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.921 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-04-25T10:44:36.922 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.922 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.922 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.922 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T10:44:36.922 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.922 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.922 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.922 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-04-25T10:44:36.922 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-04-25T10:44:36.922 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.922 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.922 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.923 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "9283", 2024-04-25T10:44:36.923 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.923 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.923 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.923 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T10:44:36.923 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.923 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.923 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.923 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.923 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.923 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.923 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.924 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.924 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.924 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.924 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.924 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.924 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.924 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.924 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.924 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.924 INFO:teuthology.orchestra.run.smithi043.stdout: "stale_cache_strategy": { 2024-04-25T10:44:36.924 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stale_cache_strategy", 2024-04-25T10:44:36.925 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.925 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.925 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.925 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "log", 2024-04-25T10:44:36.925 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.925 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.925 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.925 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.925 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.925 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.925 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.925 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-04-25T10:44:36.925 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-04-25T10:44:36.926 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.926 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.926 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.926 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "default", 2024-04-25T10:44:36.926 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.926 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.926 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.926 INFO:teuthology.orchestra.run.smithi043.stdout: "default", 2024-04-25T10:44:36.926 INFO:teuthology.orchestra.run.smithi043.stdout: "error" 2024-04-25T10:44:36.926 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.926 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.927 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.927 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.927 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.927 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-04-25T10:44:36.927 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-04-25T10:44:36.927 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.927 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.927 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.927 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-04-25T10:44:36.927 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-04-25T10:44:36.927 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-04-25T10:44:36.927 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.927 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.928 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.928 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.928 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.928 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.928 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.928 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_support", 2024-04-25T10:44:36.928 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.928 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.928 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.928 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.928 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.929 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.929 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.929 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.929 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.929 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.929 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.929 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.929 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.929 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.929 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.929 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.929 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.929 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.930 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.930 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.930 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.930 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.930 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.930 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.930 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.930 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.930 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.930 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.931 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.931 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.931 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.931 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.931 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.931 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.931 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.931 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.931 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.931 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.931 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.931 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.931 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.931 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.932 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.932 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.932 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.932 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.932 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.932 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.932 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.932 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.932 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.932 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.932 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.932 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.932 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.933 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.933 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.933 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.933 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.933 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.933 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.933 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.933 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.933 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.933 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.933 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.934 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.934 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.934 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.934 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_snap_create": { 2024-04-25T10:44:36.934 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_snap_create", 2024-04-25T10:44:36.934 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.934 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.934 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.934 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:36.934 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.934 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.934 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.935 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.935 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.935 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.935 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.935 INFO:teuthology.orchestra.run.smithi043.stdout: "mirror_snapshot_schedule": { 2024-04-25T10:44:36.935 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T10:44:36.935 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.935 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.935 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.935 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.935 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.935 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.935 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.936 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.936 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.936 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.936 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.936 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.936 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.936 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.936 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.936 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.936 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.936 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.936 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.936 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: "trash_purge_schedule": { 2024-04-25T10:44:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "trash_purge_schedule", 2024-04-25T10:44:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.938 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.938 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.938 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.938 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.938 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.938 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.938 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "restful", 2024-04-25T10:44:36.938 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.938 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.938 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.938 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_auth": { 2024-04-25T10:44:36.938 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_auth", 2024-04-25T10:44:36.939 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.939 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.939 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.939 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.939 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.939 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.939 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.939 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.939 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.939 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.939 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.939 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-04-25T10:44:36.939 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-04-25T10:44:36.940 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.940 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.940 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.940 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.940 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.940 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.940 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.940 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.940 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.940 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.940 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.940 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.940 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.941 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.941 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.941 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.941 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.941 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.941 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.941 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.941 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.941 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.941 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.941 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.941 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.941 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.941 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.941 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.941 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.942 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.942 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.942 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.942 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.942 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.942 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.942 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.942 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.942 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.942 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.942 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.942 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.942 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.943 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.943 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.943 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.943 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.943 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.943 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.943 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.943 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.943 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.943 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.943 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.943 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.943 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.944 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.944 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.944 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.944 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.944 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.944 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.944 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.944 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.944 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.944 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.944 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.944 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.944 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.944 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.945 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.945 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.945 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.945 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.945 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.945 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.945 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.945 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.945 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-04-25T10:44:36.945 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-04-25T10:44:36.945 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.945 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.945 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.946 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.946 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.946 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.946 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.946 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.946 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.946 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.946 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.946 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-04-25T10:44:36.946 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-04-25T10:44:36.946 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.946 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.947 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.947 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.947 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.947 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.947 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.947 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.947 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.947 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.947 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.947 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.947 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.947 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.947 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.948 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.948 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.948 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.948 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.948 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.948 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.948 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.948 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.948 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.948 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.948 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.948 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rgw", 2024-04-25T10:44:36.949 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.949 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.949 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.949 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.949 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.949 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.949 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.949 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.949 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.949 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.949 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.949 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.950 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.950 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.950 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.950 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.950 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.950 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.950 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.950 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.950 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.950 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.950 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.950 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.950 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.951 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.951 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.951 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.951 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.951 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.951 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.951 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.951 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.951 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.951 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.951 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.951 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.951 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.952 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.952 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.952 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.952 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.952 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.952 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.952 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.952 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.952 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.952 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.952 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.952 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.952 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.953 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.953 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.953 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.953 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.953 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.953 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.953 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.953 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.953 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.953 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.953 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.953 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.953 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.953 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.954 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.954 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.954 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.954 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.954 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.954 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.954 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.954 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.954 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.954 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.954 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.954 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.954 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.955 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.955 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.955 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.955 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.955 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.955 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.955 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rook", 2024-04-25T10:44:36.955 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.955 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.955 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.955 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.955 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.956 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.956 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.956 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.956 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.956 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.956 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.956 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.956 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.956 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.956 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.956 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.956 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.956 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.956 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.956 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.957 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.957 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.957 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.957 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.957 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.957 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.957 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.957 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.957 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.957 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.957 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.957 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.957 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.957 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.958 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.958 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.958 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.958 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.958 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.958 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.958 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.958 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.958 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.958 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.958 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.958 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.958 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.959 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.959 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.959 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.959 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.959 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.959 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.959 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.959 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.959 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.959 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.959 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.960 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.960 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.960 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.960 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.960 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.960 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.960 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.960 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.960 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.960 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.960 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.961 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.961 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.961 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.961 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.961 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.961 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.961 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.961 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.961 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.961 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.961 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.961 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.962 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.962 INFO:teuthology.orchestra.run.smithi043.stdout: "storage_class": { 2024-04-25T10:44:36.962 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "storage_class", 2024-04-25T10:44:36.962 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.962 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.962 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.962 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "local", 2024-04-25T10:44:36.962 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.962 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.962 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.962 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T10:44:36.962 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.962 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.963 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.963 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.963 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.963 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "selftest", 2024-04-25T10:44:36.963 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.963 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.963 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.963 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.963 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.963 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.963 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.963 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.964 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.964 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.964 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.964 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.964 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.964 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.964 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.964 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.964 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.964 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.964 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.964 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.964 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.964 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.965 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.965 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.965 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.965 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.965 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.965 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.965 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.965 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.965 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.965 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.965 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.965 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.965 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.965 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.966 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.966 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.966 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.966 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.966 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.966 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.966 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.966 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.966 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.966 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.966 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.966 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.967 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.967 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.967 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.967 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.967 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.967 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.967 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.967 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.967 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.967 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.967 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.967 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.967 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.967 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.968 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.968 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.968 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.968 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.968 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.968 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.968 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.968 INFO:teuthology.orchestra.run.smithi043.stdout: "roption1": { 2024-04-25T10:44:36.968 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption1", 2024-04-25T10:44:36.968 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.968 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.968 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.969 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.969 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.969 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.969 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.969 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.969 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.969 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.969 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.969 INFO:teuthology.orchestra.run.smithi043.stdout: "roption2": { 2024-04-25T10:44:36.969 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption2", 2024-04-25T10:44:36.969 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.969 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.969 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.970 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "xyz", 2024-04-25T10:44:36.970 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.970 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.970 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.970 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.970 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.970 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.970 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.970 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption1": { 2024-04-25T10:44:36.970 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption1", 2024-04-25T10:44:36.970 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.970 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.970 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.971 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.971 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.971 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.971 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.971 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.971 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.971 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.971 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.971 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption2": { 2024-04-25T10:44:36.971 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption2", 2024-04-25T10:44:36.971 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.972 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.972 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.972 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.972 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.972 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.972 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.972 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.972 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.972 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.972 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.972 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption3": { 2024-04-25T10:44:36.972 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption3", 2024-04-25T10:44:36.972 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:36.973 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.973 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.973 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.973 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.973 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.973 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.973 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.973 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.973 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.973 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.973 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption4": { 2024-04-25T10:44:36.973 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption4", 2024-04-25T10:44:36.973 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.974 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.974 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.974 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.974 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.974 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.974 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.974 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.974 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.974 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.974 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.974 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption5": { 2024-04-25T10:44:36.974 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption5", 2024-04-25T10:44:36.974 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.975 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.975 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.975 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.975 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.975 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.975 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.975 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.975 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.975 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.975 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.975 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption6": { 2024-04-25T10:44:36.975 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption6", 2024-04-25T10:44:36.975 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.976 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.976 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.976 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:36.976 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.976 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.976 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.976 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.976 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.976 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.976 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.976 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption7": { 2024-04-25T10:44:36.976 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption7", 2024-04-25T10:44:36.977 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.977 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.977 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.977 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.977 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-25T10:44:36.977 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "42", 2024-04-25T10:44:36.977 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.977 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.977 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.977 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.977 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.977 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.978 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.978 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.978 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.978 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.978 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.978 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.978 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.978 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.978 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.978 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.978 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.978 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.979 INFO:teuthology.orchestra.run.smithi043.stdout: "testkey": { 2024-04-25T10:44:36.979 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testkey", 2024-04-25T10:44:36.979 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.979 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.979 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.979 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.979 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.979 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.979 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.979 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.979 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.980 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.980 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.980 INFO:teuthology.orchestra.run.smithi043.stdout: "testlkey": { 2024-04-25T10:44:36.980 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testlkey", 2024-04-25T10:44:36.980 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.980 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.980 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.980 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.980 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.980 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.980 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.980 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.981 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.981 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.981 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.981 INFO:teuthology.orchestra.run.smithi043.stdout: "testnewline": { 2024-04-25T10:44:36.981 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testnewline", 2024-04-25T10:44:36.981 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.981 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.981 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:36.981 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.981 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.981 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.982 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.982 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.982 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.982 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.982 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.982 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.982 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.982 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-04-25T10:44:36.982 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.982 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.982 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.983 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_m_granularity": { 2024-04-25T10:44:36.983 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_m_granularity", 2024-04-25T10:44:36.983 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.983 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.983 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.983 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.983 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.983 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.983 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.983 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T10:44:36.983 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.983 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.983 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.984 INFO:teuthology.orchestra.run.smithi043.stdout: "dump_on_update": { 2024-04-25T10:44:36.984 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dump_on_update", 2024-04-25T10:44:36.984 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.984 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.984 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.984 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.984 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.984 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.984 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.984 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "dump database to debug log on update", 2024-04-25T10:44:36.984 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.985 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.985 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.985 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.985 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.985 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.985 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.985 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.985 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.985 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.985 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.985 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.986 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.986 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.986 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.986 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.986 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.986 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.986 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.986 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.986 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.986 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.986 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.986 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.987 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.987 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.987 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.987 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.987 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.987 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.987 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.987 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.987 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.987 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.987 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.987 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.987 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.988 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.988 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.988 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.988 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.988 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.988 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.988 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.988 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.988 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.988 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.988 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.988 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.989 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.989 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.989 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.989 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.989 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.989 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.989 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.989 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.989 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.989 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.989 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.989 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.989 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.990 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.990 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.990 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.990 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.990 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.990 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.990 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.990 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.990 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.990 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.990 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.990 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.991 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.991 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.991 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.991 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.991 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.991 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.991 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.991 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.991 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.991 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.991 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stats", 2024-04-25T10:44:36.991 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.992 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.992 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.992 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.992 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.992 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.992 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.992 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.992 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.992 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.992 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.992 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.992 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.992 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.993 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.993 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.993 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.993 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.993 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.993 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.993 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.993 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.993 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.993 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:36.993 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:36.993 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.993 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.994 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.994 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.994 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.994 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.994 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.994 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.994 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.994 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.994 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.994 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:36.994 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:36.994 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.994 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.995 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.995 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:36.995 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.995 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.995 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.995 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.995 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.995 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.995 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.995 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.995 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.995 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.995 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.995 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.996 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.996 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.996 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:36.996 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:36.996 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:36.996 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.996 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.996 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:36.996 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.996 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.996 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.996 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.996 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.997 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.997 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.997 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:36.997 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:36.997 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:36.997 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:36.997 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.997 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:36.997 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.997 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.997 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:36.997 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.997 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.998 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:36.998 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:36.998 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.998 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:36.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:36.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:36.998 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "status", 2024-04-25T10:44:36.998 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:36.998 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:36.998 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:36.998 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:36.998 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:36.998 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:36.998 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:36.999 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:36.999 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:36.999 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:36.999 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:36.999 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:36.999 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:36.999 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:36.999 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:36.999 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:36.999 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:36.999 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:36.999 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:36.999 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:36.999 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:36.999 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.000 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.000 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.000 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.000 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.000 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.000 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.000 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.000 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.002 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.002 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.002 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.002 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.002 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.002 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.003 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.003 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.003 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.003 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.003 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.003 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.003 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.003 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.003 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.003 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.003 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.003 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.003 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.004 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.004 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.004 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.004 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.004 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.004 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.004 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.004 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.004 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.004 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.004 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.004 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.004 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.005 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.005 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.005 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.005 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.005 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.005 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.005 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.005 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.005 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.005 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.005 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.005 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.005 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.006 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.006 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.006 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.006 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.006 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.006 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.006 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.006 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.006 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.006 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telegraf", 2024-04-25T10:44:37.006 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.007 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.007 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.007 INFO:teuthology.orchestra.run.smithi043.stdout: "address": { 2024-04-25T10:44:37.007 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "address", 2024-04-25T10:44:37.007 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.007 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.007 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.007 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T10:44:37.007 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.007 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.007 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.007 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.007 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.008 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.008 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.008 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-25T10:44:37.008 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-25T10:44:37.008 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.008 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.008 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.008 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15", 2024-04-25T10:44:37.008 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.008 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.008 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.008 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.008 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.009 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.009 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.009 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.009 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.009 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.009 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.010 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.010 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.010 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.010 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.010 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.010 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.010 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.010 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.010 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.010 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.010 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.010 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.010 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.011 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.011 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.011 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.011 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.011 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.011 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.011 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.011 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.011 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.011 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.011 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.011 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.011 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.012 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.012 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.012 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.012 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.012 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.012 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.012 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.012 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.012 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.012 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.012 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.012 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.013 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.013 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.013 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.013 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.013 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.013 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.013 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.013 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.013 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.013 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.013 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.013 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.013 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.014 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.014 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.014 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.014 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.014 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.014 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.014 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.014 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.014 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.014 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.014 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.014 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.014 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.015 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.015 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.015 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.015 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.015 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.015 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.015 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.015 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.015 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.015 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.015 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.015 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.015 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.016 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telemetry", 2024-04-25T10:44:37.016 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.016 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.016 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.016 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_basic": { 2024-04-25T10:44:37.016 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_basic", 2024-04-25T10:44:37.016 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.016 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.016 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.016 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.016 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.016 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.016 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.017 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T10:44:37.017 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.017 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.017 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.017 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_crash": { 2024-04-25T10:44:37.017 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_crash", 2024-04-25T10:44:37.017 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.017 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.017 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.017 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.017 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.017 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.017 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.017 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T10:44:37.018 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.018 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.018 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.018 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_device": { 2024-04-25T10:44:37.018 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_device", 2024-04-25T10:44:37.018 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.018 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.018 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.018 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.018 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.018 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.018 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.019 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T10:44:37.019 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.019 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.019 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.019 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_ident": { 2024-04-25T10:44:37.019 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_ident", 2024-04-25T10:44:37.019 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.019 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.019 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.019 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.019 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.019 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.019 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.020 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T10:44:37.020 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.020 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.020 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.020 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_perf": { 2024-04-25T10:44:37.020 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_perf", 2024-04-25T10:44:37.020 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.020 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.020 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.020 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.020 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.021 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.021 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.021 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T10:44:37.021 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.021 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.021 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.021 INFO:teuthology.orchestra.run.smithi043.stdout: "contact": { 2024-04-25T10:44:37.021 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "contact", 2024-04-25T10:44:37.021 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.021 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.021 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.022 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.022 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.022 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.022 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.022 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.022 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.022 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.022 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.022 INFO:teuthology.orchestra.run.smithi043.stdout: "description": { 2024-04-25T10:44:37.022 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "description", 2024-04-25T10:44:37.022 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.022 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.022 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "device_url": { 2024-04-25T10:44:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_url", 2024-04-25T10:44:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T10:44:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-04-25T10:44:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-04-25T10:44:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-25T10:44:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-25T10:44:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "24", 2024-04-25T10:44:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "8", 2024-04-25T10:44:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "last_opt_revision": { 2024-04-25T10:44:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "last_opt_revision", 2024-04-25T10:44:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1", 2024-04-25T10:44:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard": { 2024-04-25T10:44:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard", 2024-04-25T10:44:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard_description": { 2024-04-25T10:44:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard_description", 2024-04-25T10:44:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "organization": { 2024-04-25T10:44:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "organization", 2024-04-25T10:44:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "proxy": { 2024-04-25T10:44:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "proxy", 2024-04-25T10:44:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "url": { 2024-04-25T10:44:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url", 2024-04-25T10:44:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T10:44:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "test_orchestrator", 2024-04-25T10:44:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "volumes", 2024-04-25T10:44:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_clones": { 2024-04-25T10:44:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_clones", 2024-04-25T10:44:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4", 2024-04-25T10:44:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T10:44:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "periodic_async_work": { 2024-04-25T10:44:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "periodic_async_work", 2024-04-25T10:44:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Periodically check for async work", 2024-04-25T10:44:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_delay": { 2024-04-25T10:44:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_delay", 2024-04-25T10:44:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T10:44:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_no_wait": { 2024-04-25T10:44:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T10:44:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T10:44:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix", 2024-04-25T10:44:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: "discovery_interval": { 2024-04-25T10:44:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "discovery_interval", 2024-04-25T10:44:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-25T10:44:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "100", 2024-04-25T10:44:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "identifier": { 2024-04-25T10:44:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "identifier", 2024-04-25T10:44:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-25T10:44:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-25T10:44:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-25T10:44:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_host": { 2024-04-25T10:44:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_host", 2024-04-25T10:44:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_port": { 2024-04-25T10:44:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_port", 2024-04-25T10:44:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10051", 2024-04-25T10:44:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_sender": { 2024-04-25T10:44:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_sender", 2024-04-25T10:44:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T10:44:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T10:44:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: "modules": [ 2024-04-25T10:44:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm", 2024-04-25T10:44:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: "dashboard", 2024-04-25T10:44:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: "iostat", 2024-04-25T10:44:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: "nfs", 2024-04-25T10:44:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: "restful", 2024-04-25T10:44:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_schedule" 2024-04-25T10:44:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: "available_modules": [ 2024-04-25T10:44:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "alerts", 2024-04-25T10:44:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-25T10:44:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-25T10:44:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-25T10:44:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T10:44:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_destination": { 2024-04-25T10:44:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_destination", 2024-04-25T10:44:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email address to send alerts to", 2024-04-25T10:44:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_from_name": { 2024-04-25T10:44:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_from_name", 2024-04-25T10:44:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "Ceph", 2024-04-25T10:44:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email From: name", 2024-04-25T10:44:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_host": { 2024-04-25T10:44:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_host", 2024-04-25T10:44:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP server", 2024-04-25T10:44:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_password": { 2024-04-25T10:44:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_password", 2024-04-25T10:44:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Password to authenticate with", 2024-04-25T10:44:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_port": { 2024-04-25T10:44:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_port", 2024-04-25T10:44:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "465", 2024-04-25T10:44:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP port", 2024-04-25T10:44:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_sender": { 2024-04-25T10:44:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_sender", 2024-04-25T10:44:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP envelope sender", 2024-04-25T10:44:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_ssl": { 2024-04-25T10:44:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_ssl", 2024-04-25T10:44:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T10:44:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_user": { 2024-04-25T10:44:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_user", 2024-04-25T10:44:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "User to authenticate as", 2024-04-25T10:44:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "balancer", 2024-04-25T10:44:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: "active": { 2024-04-25T10:44:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "active", 2024-04-25T10:44:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T10:44:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_time": { 2024-04-25T10:44:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_time", 2024-04-25T10:44:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0000", 2024-04-25T10:44:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T10:44:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T10:44:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_weekday": { 2024-04-25T10:44:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_weekday", 2024-04-25T10:44:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-25T10:44:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-04-25T10:44:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-04-25T10:44:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T10:44:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T10:44:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_max_iterations": { 2024-04-25T10:44:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_max_iterations", 2024-04-25T10:44:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-25T10:44:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "25", 2024-04-25T10:44:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-25T10:44:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "250", 2024-04-25T10:44:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T10:44:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_metrics": { 2024-04-25T10:44:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_metrics", 2024-04-25T10:44:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T10:44:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T10:44:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T10:44:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_step": { 2024-04-25T10:44:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_step", 2024-04-25T10:44:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.5", 2024-04-25T10:44:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0.001", 2024-04-25T10:44:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "0.999", 2024-04-25T10:44:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "aggressiveness of optimization", 2024-04-25T10:44:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T10:44:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "end_time": { 2024-04-25T10:44:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_time", 2024-04-25T10:44:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2359", 2024-04-25T10:44:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T10:44:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T10:44:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "end_weekday": { 2024-04-25T10:44:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_weekday", 2024-04-25T10:44:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-25T10:44:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-04-25T10:44:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-04-25T10:44:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T10:44:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T10:44:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "min_score": { 2024-04-25T10:44:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_score", 2024-04-25T10:44:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T10:44:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-04-25T10:44:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-04-25T10:44:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "upmap", 2024-04-25T10:44:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-compat", 2024-04-25T10:44:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "none", 2024-04-25T10:44:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "read", 2024-04-25T10:44:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap", 2024-04-25T10:44:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap-read" 2024-04-25T10:44:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Balancer mode", 2024-04-25T10:44:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ids": { 2024-04-25T10:44:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_ids", 2024-04-25T10:44:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T10:44:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-25T10:44:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-25T10:44:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-25T10:44:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T10:44:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_deviation": { 2024-04-25T10:44:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_deviation", 2024-04-25T10:44:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-25T10:44:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-25T10:44:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T10:44:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T10:44:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_optimizations": { 2024-04-25T10:44:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_optimizations", 2024-04-25T10:44:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-25T10:44:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T10:44:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm", 2024-04-25T10:44:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_down_multiplier": { 2024-04-25T10:44:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_down_multiplier", 2024-04-25T10:44:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-04-25T10:44:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T10:44:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_refresh_rate": { 2024-04-25T10:44:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_refresh_rate", 2024-04-25T10:44:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "20", 2024-04-25T10:44:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T10:44:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_starting_port": { 2024-04-25T10:44:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_starting_port", 2024-04-25T10:44:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4721", 2024-04-25T10:44:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T10:44:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_ptrace": { 2024-04-25T10:44:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_ptrace", 2024-04-25T10:44:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T10:44:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T10:44:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_interval": { 2024-04-25T10:44:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_interval", 2024-04-25T10:44:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-25T10:44:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T10:44:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_memory_target_ratio": { 2024-04-25T10:44:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T10:44:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.7", 2024-04-25T10:44:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T10:44:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm_log_destination": { 2024-04-25T10:44:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm_log_destination", 2024-04-25T10:44:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "file", 2024-04-25T10:44:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "file,syslog", 2024-04-25T10:44:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "syslog" 2024-04-25T10:44:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T10:44:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "cgroups_split": { 2024-04-25T10:44:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cgroups_split", 2024-04-25T10:44:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T10:44:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "config_checks_enabled": { 2024-04-25T10:44:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_checks_enabled", 2024-04-25T10:44:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T10:44:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "config_dashboard": { 2024-04-25T10:44:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_dashboard", 2024-04-25T10:44:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T10:44:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_alertmanager": { 2024-04-25T10:44:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_alertmanager", 2024-04-25T10:44:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T10:44:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-25T10:44:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_base": { 2024-04-25T10:44:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_base", 2024-04-25T10:44:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T10:44:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Container image name, without the tag", 2024-04-25T10:44:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_elasticsearch": { 2024-04-25T10:44:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_elasticsearch", 2024-04-25T10:44:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T10:44:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "elasticsearch container image", 2024-04-25T10:44:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_grafana": { 2024-04-25T10:44:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_grafana", 2024-04-25T10:44:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T10:44:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-25T10:44:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_haproxy": { 2024-04-25T10:44:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_haproxy", 2024-04-25T10:44:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T10:44:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "HAproxy container image", 2024-04-25T10:44:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_agent": { 2024-04-25T10:44:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_agent", 2024-04-25T10:44:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T10:44:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger agent container image", 2024-04-25T10:44:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_collector": { 2024-04-25T10:44:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_collector", 2024-04-25T10:44:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T10:44:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger collector container image", 2024-04-25T10:44:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_query": { 2024-04-25T10:44:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_query", 2024-04-25T10:44:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T10:44:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger query container image", 2024-04-25T10:44:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_keepalived": { 2024-04-25T10:44:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_keepalived", 2024-04-25T10:44:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T10:44:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Keepalived container image", 2024-04-25T10:44:37.125 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.125 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.125 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.125 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_loki": { 2024-04-25T10:44:37.125 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_loki", 2024-04-25T10:44:37.125 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.125 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.125 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.125 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T10:44:37.125 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.125 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.125 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.125 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Loki container image", 2024-04-25T10:44:37.125 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.126 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.126 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.126 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_node_exporter": { 2024-04-25T10:44:37.126 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_node_exporter", 2024-04-25T10:44:37.126 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.126 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.126 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.126 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T10:44:37.126 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.126 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.126 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.126 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-25T10:44:37.126 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.127 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.127 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.127 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_nvmeof": { 2024-04-25T10:44:37.127 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_nvmeof", 2024-04-25T10:44:37.127 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.127 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.127 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.127 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T10:44:37.127 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.127 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.127 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.127 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Nvme-of container image", 2024-04-25T10:44:37.127 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.128 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.128 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.128 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_prometheus": { 2024-04-25T10:44:37.128 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_prometheus", 2024-04-25T10:44:37.128 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.128 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.128 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.128 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T10:44:37.128 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.128 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.128 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.128 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-25T10:44:37.128 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.128 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.129 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.129 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_promtail": { 2024-04-25T10:44:37.129 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_promtail", 2024-04-25T10:44:37.129 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.129 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.129 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.129 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T10:44:37.129 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.129 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.129 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.129 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Promtail container image", 2024-04-25T10:44:37.129 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.129 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.130 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.130 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_samba": { 2024-04-25T10:44:37.130 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_samba", 2024-04-25T10:44:37.130 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.130 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.130 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.130 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T10:44:37.130 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.130 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.130 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.130 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Samba/SMB container image", 2024-04-25T10:44:37.130 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.131 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.131 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.131 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_snmp_gateway": { 2024-04-25T10:44:37.131 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_snmp_gateway", 2024-04-25T10:44:37.131 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.131 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.131 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.131 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T10:44:37.131 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.131 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.131 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.131 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SNMP Gateway container image", 2024-04-25T10:44:37.131 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.132 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.132 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.132 INFO:teuthology.orchestra.run.smithi043.stdout: "container_init": { 2024-04-25T10:44:37.132 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_init", 2024-04-25T10:44:37.132 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.132 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.132 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.132 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.132 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.132 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.132 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.132 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T10:44:37.132 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.133 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.133 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.133 INFO:teuthology.orchestra.run.smithi043.stdout: "daemon_cache_timeout": { 2024-04-25T10:44:37.133 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "daemon_cache_timeout", 2024-04-25T10:44:37.133 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.133 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.133 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.133 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-25T10:44:37.133 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.133 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.133 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.133 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T10:44:37.133 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.133 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.134 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.134 INFO:teuthology.orchestra.run.smithi043.stdout: "default_cephadm_command_timeout": { 2024-04-25T10:44:37.134 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T10:44:37.134 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.134 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.134 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.134 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "900", 2024-04-25T10:44:37.134 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.134 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.134 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.134 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T10:44:37.134 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.134 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.135 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.135 INFO:teuthology.orchestra.run.smithi043.stdout: "default_registry": { 2024-04-25T10:44:37.135 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_registry", 2024-04-25T10:44:37.135 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.135 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.135 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.135 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io", 2024-04-25T10:44:37.135 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.135 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.135 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.135 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T10:44:37.135 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.135 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.135 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.136 INFO:teuthology.orchestra.run.smithi043.stdout: "device_cache_timeout": { 2024-04-25T10:44:37.136 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_cache_timeout", 2024-04-25T10:44:37.136 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.136 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.136 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.136 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1800", 2024-04-25T10:44:37.136 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.136 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.136 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.136 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache device inventory", 2024-04-25T10:44:37.136 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.136 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.136 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.137 INFO:teuthology.orchestra.run.smithi043.stdout: "device_enhanced_scan": { 2024-04-25T10:44:37.137 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_enhanced_scan", 2024-04-25T10:44:37.137 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.137 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.137 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.137 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.137 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.137 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.137 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.137 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T10:44:37.137 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.137 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.137 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.138 INFO:teuthology.orchestra.run.smithi043.stdout: "facts_cache_timeout": { 2024-04-25T10:44:37.138 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "facts_cache_timeout", 2024-04-25T10:44:37.138 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.138 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.138 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.138 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-25T10:44:37.138 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.138 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.138 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.138 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache host facts data", 2024-04-25T10:44:37.138 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.138 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.138 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.139 INFO:teuthology.orchestra.run.smithi043.stdout: "grafana_dashboards_path": { 2024-04-25T10:44:37.139 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "grafana_dashboards_path", 2024-04-25T10:44:37.139 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.139 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.139 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.139 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T10:44:37.139 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.139 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.139 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.139 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T10:44:37.139 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.139 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.140 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.140 INFO:teuthology.orchestra.run.smithi043.stdout: "host_check_interval": { 2024-04-25T10:44:37.140 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "host_check_interval", 2024-04-25T10:44:37.140 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.140 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.140 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.140 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-25T10:44:37.140 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.140 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.140 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.140 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to perform a host check", 2024-04-25T10:44:37.140 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.140 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.140 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.141 INFO:teuthology.orchestra.run.smithi043.stdout: "hw_monitoring": { 2024-04-25T10:44:37.141 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hw_monitoring", 2024-04-25T10:44:37.141 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.141 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.141 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.141 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.141 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.141 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.141 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.141 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T10:44:37.141 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.141 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.141 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.142 INFO:teuthology.orchestra.run.smithi043.stdout: "inventory_list_all": { 2024-04-25T10:44:37.142 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "inventory_list_all", 2024-04-25T10:44:37.142 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.142 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.142 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.142 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.142 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.142 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.142 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.142 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T10:44:37.142 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.142 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.142 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.143 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.143 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.143 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.143 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.143 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.143 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.143 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.143 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.143 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.143 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.143 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.143 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.143 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.143 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.144 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.144 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.144 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.144 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.144 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.144 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.144 INFO:teuthology.orchestra.run.smithi043.stdout: "log_refresh_metadata": { 2024-04-25T10:44:37.144 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_refresh_metadata", 2024-04-25T10:44:37.144 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.144 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.144 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.144 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.144 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.145 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.145 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.145 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T10:44:37.145 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.145 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.145 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.145 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.145 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.145 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.145 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.145 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.145 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.145 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.145 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.146 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.146 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T10:44:37.146 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.146 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.146 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.146 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.146 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.146 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.146 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.146 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.146 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.146 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.146 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.147 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.147 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.147 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.147 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.147 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.147 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.147 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.147 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.147 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.147 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.147 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.147 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.147 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.148 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.148 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.148 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.148 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.148 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.148 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.148 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.148 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.148 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.148 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.148 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.148 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.148 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T10:44:37.148 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T10:44:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T10:44:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T10:44:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T10:44:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "*", 2024-04-25T10:44:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T10:44:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "max_count_per_host": { 2024-04-25T10:44:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_count_per_host", 2024-04-25T10:44:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of daemons per service per host", 2024-04-25T10:44:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd_draining_count": { 2024-04-25T10:44:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_osd_draining_count", 2024-04-25T10:44:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T10:44:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: "migration_current": { 2024-04-25T10:44:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "migration_current", 2024-04-25T10:44:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "internal - do not modify", 2024-04-25T10:44:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-04-25T10:44:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-04-25T10:44:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "root", 2024-04-25T10:44:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm-package", 2024-04-25T10:44:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "root" 2024-04-25T10:44:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T10:44:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "oob_default_addr": { 2024-04-25T10:44:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "oob_default_addr", 2024-04-25T10:44:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "169.254.1.1", 2024-04-25T10:44:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T10:44:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "prometheus_alerts_path": { 2024-04-25T10:44:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus_alerts_path", 2024-04-25T10:44:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T10:44:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T10:44:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_insecure": { 2024-04-25T10:44:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_insecure", 2024-04-25T10:44:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T10:44:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_password": { 2024-04-25T10:44:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_password", 2024-04-25T10:44:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T10:44:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_url": { 2024-04-25T10:44:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_url", 2024-04-25T10:44:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T10:44:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_username": { 2024-04-25T10:44:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_username", 2024-04-25T10:44:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T10:44:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "secure_monitoring_stack": { 2024-04-25T10:44:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "secure_monitoring_stack", 2024-04-25T10:44:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T10:44:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "service_discovery_port": { 2024-04-25T10:44:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "service_discovery_port", 2024-04-25T10:44:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8765", 2024-04-25T10:44:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "cephadm service discovery port", 2024-04-25T10:44:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "ssh_config_file": { 2024-04-25T10:44:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssh_config_file", 2024-04-25T10:44:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T10:44:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "use_agent": { 2024-04-25T10:44:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_agent", 2024-04-25T10:44:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T10:44:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "use_repo_digest": { 2024-04-25T10:44:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_repo_digest", 2024-04-25T10:44:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T10:44:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_failed_host_check": { 2024-04-25T10:44:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_failed_host_check", 2024-04-25T10:44:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T10:44:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_daemons": { 2024-04-25T10:44:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_daemons", 2024-04-25T10:44:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T10:44:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_hosts": { 2024-04-25T10:44:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_hosts", 2024-04-25T10:44:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T10:44:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crash", 2024-04-25T10:44:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "retain_interval": { 2024-04-25T10:44:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retain_interval", 2024-04-25T10:44:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "31536000", 2024-04-25T10:44:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T10:44:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_recent_interval": { 2024-04-25T10:44:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_recent_interval", 2024-04-25T10:44:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1209600", 2024-04-25T10:44:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T10:44:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dashboard", 2024-04-25T10:44:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T10:44:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T10:44:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T10:44:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T10:44:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T10:44:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T10:44:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_ENABLED": { 2024-04-25T10:44:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T10:44:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T10:44:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T10:44:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T10:44:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T10:44:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T10:44:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T10:44:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T10:44:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T10:44:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T10:44:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T10:44:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T10:44:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T10:44:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T10:44:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T10:44:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T10:44:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T10:44:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T10:44:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T10:44:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T10:44:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T10:44:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T10:44:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T10:44:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-04-25T10:44:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T10:44:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T10:44:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_URL": { 2024-04-25T10:44:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_URL", 2024-04-25T10:44:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T10:44:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T10:44:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-04-25T10:44:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T10:44:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T10:44:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T10:44:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T10:44:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T10:44:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T10:44:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T10:44:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T10:44:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T10:44:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T10:44:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "[]", 2024-04-25T10:44:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T10:44:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T10:44:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "{}", 2024-04-25T10:44:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T10:44:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T10:44:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T10:44:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T10:44:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T10:44:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T10:44:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T10:44:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T10:44:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T10:44:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T10:44:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T10:44:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T10:44:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T10:44:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T10:44:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T10:44:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T10:44:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T10:44:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T10:44:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T10:44:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T10:44:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T10:44:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T10:44:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T10:44:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T10:44:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T10:44:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T10:44:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T10:44:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8", 2024-04-25T10:44:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T10:44:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T10:44:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "45", 2024-04-25T10:44:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T10:44:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T10:44:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T10:44:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T10:44:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-04-25T10:44:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T10:44:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T10:44:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T10:44:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T10:44:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T10:44:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T10:44:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T10:44:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T10:44:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T10:44:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T10:44:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T10:44:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T10:44:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-25T10:44:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "cross_origin_url": { 2024-04-25T10:44:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cross_origin_url", 2024-04-25T10:44:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "crt_file": { 2024-04-25T10:44:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crt_file", 2024-04-25T10:44:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "debug": { 2024-04-25T10:44:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "debug", 2024-04-25T10:44:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable/disable debug options", 2024-04-25T10:44:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "jwt_token_ttl": { 2024-04-25T10:44:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "jwt_token_ttl", 2024-04-25T10:44:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "28800", 2024-04-25T10:44:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-04-25T10:44:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-04-25T10:44:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "motd": { 2024-04-25T10:44:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "motd", 2024-04-25T10:44:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "The message of the day", 2024-04-25T10:44:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect_resolve_ip_addr": { 2024-04-25T10:44:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T10:44:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-04-25T10:44:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-04-25T10:44:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-04-25T10:44:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-04-25T10:44:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-04-25T10:44:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8080", 2024-04-25T10:44:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-04-25T10:44:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-04-25T10:44:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl_server_port": { 2024-04-25T10:44:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl_server_port", 2024-04-25T10:44:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8443", 2024-04-25T10:44:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-04-25T10:44:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-04-25T10:44:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "redirect", 2024-04-25T10:44:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect" 2024-04-25T10:44:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-04-25T10:44:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-04-25T10:44:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-04-25T10:44:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-04-25T10:44:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-04-25T10:44:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: "url_prefix": { 2024-04-25T10:44:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url_prefix", 2024-04-25T10:44:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "devicehealth", 2024-04-25T10:44:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_monitoring": { 2024-04-25T10:44:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_monitoring", 2024-04-25T10:44:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "monitor device health metrics", 2024-04-25T10:44:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "mark_out_threshold": { 2024-04-25T10:44:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mark_out_threshold", 2024-04-25T10:44:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2419200", 2024-04-25T10:44:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T10:44:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": { 2024-04-25T10:44:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_name", 2024-04-25T10:44:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "device_health_metrics", 2024-04-25T10:44:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T10:44:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "retention_period": { 2024-04-25T10:44:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retention_period", 2024-04-25T10:44:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15552000", 2024-04-25T10:44:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain device health metrics", 2024-04-25T10:44:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_frequency": { 2024-04-25T10:44:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_frequency", 2024-04-25T10:44:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-04-25T10:44:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T10:44:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: "self_heal": { 2024-04-25T10:44:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "self_heal", 2024-04-25T10:44:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T10:44:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-25T10:44:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-25T10:44:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-25T10:44:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T10:44:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_threshold": { 2024-04-25T10:44:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_threshold", 2024-04-25T10:44:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7257600", 2024-04-25T10:44:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T10:44:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "diskprediction_local", 2024-04-25T10:44:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "predict_interval": { 2024-04-25T10:44:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predict_interval", 2024-04-25T10:44:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-04-25T10:44:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "predictor_model": { 2024-04-25T10:44:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predictor_model", 2024-04-25T10:44:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "prophetstor", 2024-04-25T10:44:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.272 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-25T10:44:37.272 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-25T10:44:37.272 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.272 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.272 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.272 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-25T10:44:37.272 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.272 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.272 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.272 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.272 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.272 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.273 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.273 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.273 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.273 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.273 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.273 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.273 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.273 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.273 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.273 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.273 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.273 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.273 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.274 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.274 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.274 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.274 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "influx", 2024-04-25T10:44:37.274 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": false, 2024-04-25T10:44:37.274 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "influxdb python module not found", 2024-04-25T10:44:37.274 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.274 INFO:teuthology.orchestra.run.smithi043.stdout: "batch_size": { 2024-04-25T10:44:37.274 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "batch_size", 2024-04-25T10:44:37.274 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.274 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.274 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.275 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5000", 2024-04-25T10:44:37.275 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.275 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.275 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.275 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T10:44:37.275 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.275 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.275 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.275 INFO:teuthology.orchestra.run.smithi043.stdout: "database": { 2024-04-25T10:44:37.275 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "database", 2024-04-25T10:44:37.275 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.275 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.275 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.276 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "ceph", 2024-04-25T10:44:37.276 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.276 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.276 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.276 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T10:44:37.276 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.276 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.276 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.276 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": { 2024-04-25T10:44:37.276 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hostname", 2024-04-25T10:44:37.276 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.276 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.276 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.277 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.277 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.277 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.277 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.277 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server hostname", 2024-04-25T10:44:37.277 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.277 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.277 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.277 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-25T10:44:37.277 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-25T10:44:37.277 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.277 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.277 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.278 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "30", 2024-04-25T10:44:37.278 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "5", 2024-04-25T10:44:37.278 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.278 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.278 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T10:44:37.278 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.278 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.278 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.278 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.278 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.278 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.278 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.279 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.279 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.279 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.279 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.279 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.279 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.279 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.279 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.279 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.279 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.279 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.279 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.279 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.279 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.280 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.280 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.280 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.280 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.280 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.280 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.280 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.280 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.280 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.280 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.280 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.280 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.280 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.281 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.281 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.281 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.281 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.281 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.281 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.281 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.281 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.281 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.281 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.281 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.281 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.281 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.281 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.282 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.282 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.282 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.282 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.282 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.282 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.282 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.282 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.282 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.282 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.282 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.282 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.283 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.283 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.283 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.283 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.283 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.283 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.283 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.283 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.283 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.283 INFO:teuthology.orchestra.run.smithi043.stdout: "password": { 2024-04-25T10:44:37.283 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "password", 2024-04-25T10:44:37.283 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.283 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.283 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.283 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.284 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.284 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.284 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.284 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "password of InfluxDB server user", 2024-04-25T10:44:37.284 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.284 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.284 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.284 INFO:teuthology.orchestra.run.smithi043.stdout: "port": { 2024-04-25T10:44:37.284 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "port", 2024-04-25T10:44:37.284 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.284 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.284 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.284 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8086", 2024-04-25T10:44:37.284 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.284 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.285 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.285 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server port", 2024-04-25T10:44:37.285 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.285 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.285 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.285 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.285 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.285 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.285 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.285 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.285 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.285 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.285 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.286 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.286 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.286 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.286 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.286 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.286 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-04-25T10:44:37.286 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-04-25T10:44:37.286 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.286 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.286 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.286 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "false", 2024-04-25T10:44:37.286 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.287 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.287 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.287 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T10:44:37.287 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.287 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.287 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.287 INFO:teuthology.orchestra.run.smithi043.stdout: "threads": { 2024-04-25T10:44:37.287 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threads", 2024-04-25T10:44:37.287 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.287 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.287 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.287 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-25T10:44:37.287 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-25T10:44:37.288 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "32", 2024-04-25T10:44:37.288 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.288 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T10:44:37.288 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.288 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.288 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.288 INFO:teuthology.orchestra.run.smithi043.stdout: "username": { 2024-04-25T10:44:37.288 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "username", 2024-04-25T10:44:37.288 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.288 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.288 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.288 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.288 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.289 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.289 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.289 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "username of InfluxDB server user", 2024-04-25T10:44:37.289 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.289 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.289 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.289 INFO:teuthology.orchestra.run.smithi043.stdout: "verify_ssl": { 2024-04-25T10:44:37.289 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "verify_ssl", 2024-04-25T10:44:37.289 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.289 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.289 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.289 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "true", 2024-04-25T10:44:37.289 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.289 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.290 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.290 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T10:44:37.290 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.290 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.290 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.290 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "insights", 2024-04-25T10:44:37.290 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.290 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.290 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.290 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.291 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.291 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.291 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.291 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.291 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.291 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.291 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.291 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.291 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.291 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.291 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.291 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.291 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.291 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.291 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.291 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.292 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.292 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.292 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.292 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.292 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.292 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.292 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.292 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.292 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.292 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.292 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.292 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.292 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.292 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.293 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.293 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.293 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.293 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.293 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.293 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.293 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.293 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.293 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.293 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.293 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.293 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.293 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.293 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.293 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.294 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.294 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.294 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.294 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.294 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.294 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.294 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.294 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.294 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.294 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.294 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.294 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.294 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.295 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.295 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.295 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.295 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.295 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.295 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.295 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.295 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.295 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.295 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.295 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.295 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.296 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.296 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.296 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.296 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.296 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.296 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.296 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.296 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.296 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.296 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.296 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "iostat", 2024-04-25T10:44:37.296 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.296 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.297 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.297 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.297 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.297 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.297 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.297 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.297 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.297 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.297 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.297 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.297 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.297 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.297 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.297 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.297 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.297 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.298 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.298 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.298 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.298 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.298 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.298 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.298 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.298 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.298 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.298 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.298 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.298 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.298 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.298 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.299 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.299 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.299 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.299 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.299 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.299 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.299 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.299 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.299 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.299 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.299 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.299 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.299 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.299 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.300 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.300 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.300 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.300 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.300 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.300 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.300 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.300 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.300 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.300 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.300 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.300 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.300 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.300 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.300 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.301 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.301 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.301 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.301 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.301 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.301 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.301 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.301 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.301 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.301 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.301 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.301 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.301 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.302 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.302 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.302 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.302 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.302 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.302 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.302 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.302 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.302 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.302 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.302 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "k8sevents", 2024-04-25T10:44:37.302 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.303 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.303 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.303 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_event_retention_days": { 2024-04-25T10:44:37.303 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ceph_event_retention_days", 2024-04-25T10:44:37.303 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.303 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.303 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.303 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7", 2024-04-25T10:44:37.303 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.303 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.303 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.303 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T10:44:37.304 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.304 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.304 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.304 INFO:teuthology.orchestra.run.smithi043.stdout: "config_check_secs": { 2024-04-25T10:44:37.304 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_check_secs", 2024-04-25T10:44:37.304 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.304 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.304 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.304 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:37.304 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "10", 2024-04-25T10:44:37.304 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.304 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.305 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T10:44:37.305 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.305 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.305 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.305 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.305 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.305 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.305 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.305 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.305 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.305 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.306 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.306 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.306 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.306 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.306 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.306 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.306 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.306 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.306 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.306 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.306 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.306 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.307 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.307 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.307 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.307 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.307 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.307 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.307 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.307 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.307 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.307 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.307 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.308 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.308 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.308 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.308 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.308 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.308 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.308 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.308 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.308 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.308 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.308 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.308 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.308 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.309 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.309 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.309 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.309 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.309 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.309 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.309 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.309 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.309 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.309 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.309 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.309 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.309 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.309 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.310 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.310 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.310 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.310 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.310 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.310 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.310 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.310 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.310 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.310 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.310 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.311 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.311 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.311 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.311 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.311 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.311 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.311 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.311 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.311 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.311 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.311 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.311 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.311 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.312 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "localpool", 2024-04-25T10:44:37.312 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.312 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.312 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.312 INFO:teuthology.orchestra.run.smithi043.stdout: "failure_domain": { 2024-04-25T10:44:37.312 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "failure_domain", 2024-04-25T10:44:37.312 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.312 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.312 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.312 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "host", 2024-04-25T10:44:37.312 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.313 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.313 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.313 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "failure domain for any created local pool", 2024-04-25T10:44:37.313 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T10:44:37.313 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.313 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.313 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.313 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.313 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.313 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.313 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.313 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.314 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.314 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.314 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.314 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.314 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.314 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.314 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.314 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.314 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.314 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.314 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.314 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.314 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.315 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.315 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.315 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.315 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.315 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.315 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.315 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.315 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.315 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.315 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.315 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.315 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.315 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.316 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.316 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.316 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.316 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.316 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.316 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.316 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.316 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.316 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.316 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.316 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.316 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.316 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.317 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.317 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.317 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.317 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.317 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.317 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.317 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.317 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.317 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.317 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.317 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.317 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.317 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.318 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.318 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.318 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.318 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.318 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.318 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.318 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.318 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.318 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": { 2024-04-25T10:44:37.318 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_size", 2024-04-25T10:44:37.318 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.318 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.319 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.319 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.319 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.319 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.319 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.319 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default min_size for any created local pool", 2024-04-25T10:44:37.319 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T10:44:37.319 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.319 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.319 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rep": { 2024-04-25T10:44:37.319 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "num_rep", 2024-04-25T10:44:37.319 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.319 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.319 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.320 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3", 2024-04-25T10:44:37.320 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.320 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.320 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.320 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default replica count for any created local pool", 2024-04-25T10:44:37.320 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.320 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.320 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.320 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": { 2024-04-25T10:44:37.320 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_num", 2024-04-25T10:44:37.320 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.320 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.321 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.321 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "128", 2024-04-25T10:44:37.321 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.321 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.321 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.321 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T10:44:37.321 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.321 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.321 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.321 INFO:teuthology.orchestra.run.smithi043.stdout: "prefix": { 2024-04-25T10:44:37.321 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prefix", 2024-04-25T10:44:37.321 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.322 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.322 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.322 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.322 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.322 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.322 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.322 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name prefix for any created local pool", 2024-04-25T10:44:37.322 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.322 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.322 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.322 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.322 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.323 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.323 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.323 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.323 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.323 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.323 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.323 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.323 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.323 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.323 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.323 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.323 INFO:teuthology.orchestra.run.smithi043.stdout: "subtree": { 2024-04-25T10:44:37.323 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "subtree", 2024-04-25T10:44:37.323 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.324 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.324 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.324 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "rack", 2024-04-25T10:44:37.324 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.324 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.324 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.324 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T10:44:37.324 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T10:44:37.324 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.324 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.325 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mds_autoscaler", 2024-04-25T10:44:37.325 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.325 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.325 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.325 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.325 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.325 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.325 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.325 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.325 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.325 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.325 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.325 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.325 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.326 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.326 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.326 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.326 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.326 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.326 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.326 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.326 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.326 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.326 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.326 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.326 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.326 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.327 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.327 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.327 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.327 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.327 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.327 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.327 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.327 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.327 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.327 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.327 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.327 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.327 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.327 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.328 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.328 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.328 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.328 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.328 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.328 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.328 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.328 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.328 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.328 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.328 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.328 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.328 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.328 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.329 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.329 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.329 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.329 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.329 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.329 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.329 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.329 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.329 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.329 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.329 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.329 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.329 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.329 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.330 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.330 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.330 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.330 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.330 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.330 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.330 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.330 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.330 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.330 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.330 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.330 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.331 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.331 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.331 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.331 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.331 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirroring", 2024-04-25T10:44:37.331 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.331 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.331 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.331 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.331 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.331 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.331 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.331 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.332 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.332 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.332 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.332 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.332 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.332 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.332 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.332 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.332 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.332 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.332 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.332 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.332 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.332 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.333 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.333 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.333 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.333 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.333 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.333 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.333 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.333 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.333 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.333 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.333 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.333 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.333 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.333 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.334 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.334 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.334 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.334 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.334 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.334 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.334 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.334 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.334 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.334 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.334 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.334 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.334 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.335 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.335 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.335 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.335 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.335 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.335 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.335 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.335 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.335 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.335 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.335 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.335 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.335 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.335 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.335 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.336 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.336 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.336 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.336 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.336 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.336 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.336 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.336 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.336 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.336 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.336 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.336 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.336 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.336 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.336 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.337 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.337 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.337 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.337 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.337 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.337 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "nfs", 2024-04-25T10:44:37.337 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.337 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.337 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.337 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.337 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.337 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.337 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.337 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.338 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.338 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.338 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.338 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.338 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.338 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.338 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.338 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.338 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.338 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.338 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.338 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.338 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.338 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.339 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.339 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.339 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.339 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.339 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.339 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.339 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.339 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.339 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.339 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.339 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.339 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.339 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.339 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.340 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.340 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.340 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.340 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.340 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.340 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.340 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.340 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.340 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.340 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.340 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.340 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.340 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.340 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.340 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.340 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.341 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.341 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.341 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.341 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.341 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.341 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.341 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.341 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.341 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.341 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.341 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.341 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.341 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.341 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.342 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.342 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.342 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.342 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.342 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.342 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.342 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.342 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.342 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.342 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.342 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.342 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.343 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.343 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.343 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.343 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.343 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.343 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.343 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-04-25T10:44:37.343 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.343 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.343 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.343 INFO:teuthology.orchestra.run.smithi043.stdout: "fail_fs": { 2024-04-25T10:44:37.343 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "fail_fs", 2024-04-25T10:44:37.343 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.344 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.344 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.344 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.344 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.344 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.344 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.344 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T10:44:37.344 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.344 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.344 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.344 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.344 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.344 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.345 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.345 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.345 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.345 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.345 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.345 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.345 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.345 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.345 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.345 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.345 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.345 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.345 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.345 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.345 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.346 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.346 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.346 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.346 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.346 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.346 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.346 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.346 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.346 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.346 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.346 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.346 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.346 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.347 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.347 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.347 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.347 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.347 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.347 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.347 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.347 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.347 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.347 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.347 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.347 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.347 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.347 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.348 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.348 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.348 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.348 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.348 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.348 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.348 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.348 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.348 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.348 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.348 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.348 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.348 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.348 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.349 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.349 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.349 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.349 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.349 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.349 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.349 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.349 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator": { 2024-04-25T10:44:37.349 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-04-25T10:44:37.349 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.349 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.349 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.349 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.349 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.350 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.350 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.350 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm", 2024-04-25T10:44:37.350 INFO:teuthology.orchestra.run.smithi043.stdout: "rook", 2024-04-25T10:44:37.350 INFO:teuthology.orchestra.run.smithi043.stdout: "test_orchestrator" 2024-04-25T10:44:37.350 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.350 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Orchestrator backend", 2024-04-25T10:44:37.350 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.350 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.350 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.350 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.350 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.350 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.350 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.351 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.351 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.351 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.351 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.351 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.351 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.351 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.351 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.351 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.351 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_perf_query", 2024-04-25T10:44:37.352 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.352 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.352 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.352 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.352 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.352 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.352 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.352 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.352 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.352 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.352 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.352 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.352 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.352 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.353 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.353 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.353 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.353 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.353 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.353 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.353 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.353 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.353 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.353 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.353 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.353 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.353 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.353 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.354 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.354 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.354 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.354 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.354 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.354 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.354 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.354 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.354 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.354 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.355 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.355 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.355 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.355 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.355 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.355 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.355 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.355 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.355 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.355 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.355 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.355 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.355 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.355 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.356 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.356 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.356 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.356 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.356 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.356 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.356 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.356 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.356 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.356 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.356 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.356 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.356 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.356 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.357 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.357 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.357 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.357 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.357 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.357 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.357 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.357 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.357 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.357 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.357 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.357 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.357 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.358 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.358 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.358 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.358 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.358 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.358 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_support", 2024-04-25T10:44:37.358 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.358 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.358 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.358 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.358 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.358 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.359 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.359 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.359 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.359 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.359 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.359 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.359 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.359 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.359 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.359 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.359 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.359 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.359 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.359 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.360 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.360 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.360 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.360 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.360 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.360 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.360 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.360 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.360 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.360 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.360 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.360 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.361 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.361 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.361 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.361 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.361 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.361 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.361 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.361 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.361 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.361 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.361 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.361 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.361 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.361 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.361 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.362 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.362 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.362 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.362 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.362 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.362 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.362 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.362 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.362 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.362 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.362 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.362 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.363 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.363 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.363 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.363 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.363 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.363 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.363 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.363 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.363 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.363 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.363 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.363 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.363 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.363 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.364 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.364 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.364 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.364 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.364 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.364 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.364 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.364 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.364 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.364 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.364 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.364 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_autoscaler", 2024-04-25T10:44:37.364 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.365 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.365 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.365 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.365 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.365 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.365 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.365 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.365 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.365 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.365 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.365 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.365 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.365 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.365 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.366 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.366 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.366 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.366 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.366 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.366 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.366 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.366 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.366 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.366 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.366 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.366 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.366 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.366 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.367 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.367 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.367 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.367 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.367 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.367 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.367 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.367 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.367 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.367 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.367 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.367 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.368 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.368 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.368 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.368 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.368 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.368 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.368 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.368 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.368 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.368 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.368 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.368 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.368 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.368 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.369 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.369 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.369 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.369 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.369 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.369 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.369 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.369 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.369 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.369 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.369 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.369 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.369 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.369 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.370 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-25T10:44:37.370 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-25T10:44:37.370 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.370 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.370 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.370 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-25T10:44:37.370 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.370 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.370 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.370 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.370 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.370 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.370 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.371 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.371 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.371 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.371 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.371 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.371 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.371 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.371 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.371 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.371 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.371 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.371 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.371 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.372 INFO:teuthology.orchestra.run.smithi043.stdout: "threshold": { 2024-04-25T10:44:37.372 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threshold", 2024-04-25T10:44:37.372 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:37.372 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.372 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.372 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-04-25T10:44:37.372 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1.0", 2024-04-25T10:44:37.372 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.372 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.372 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "scaling threshold", 2024-04-25T10:44:37.372 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T10:44:37.372 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.372 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.372 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.372 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.373 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "progress", 2024-04-25T10:44:37.373 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.373 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.373 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.373 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_pg_recovery_event": { 2024-04-25T10:44:37.373 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_pg_recovery_event", 2024-04-25T10:44:37.373 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.373 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.373 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.373 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.373 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.373 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.373 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.373 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T10:44:37.374 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.374 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.374 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.374 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-04-25T10:44:37.374 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-04-25T10:44:37.374 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.374 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.374 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.374 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.374 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.374 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.374 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.374 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.375 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.375 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.375 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.375 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.375 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.375 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.375 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.375 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.375 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.375 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.375 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.375 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.375 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.375 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.376 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.376 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.376 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.376 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.376 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.376 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.376 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.376 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.376 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.376 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.376 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.376 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.376 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.376 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.377 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.377 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.377 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.377 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.377 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.377 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.377 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.377 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.377 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.377 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.377 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.377 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.377 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.377 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.377 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.378 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.378 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.378 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.378 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.378 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.378 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.378 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.378 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.378 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.378 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.378 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.378 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.378 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.378 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.379 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.379 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.379 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.379 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.379 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.379 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.379 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.379 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.379 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.379 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.379 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.379 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.379 INFO:teuthology.orchestra.run.smithi043.stdout: "max_completed_events": { 2024-04-25T10:44:37.379 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_completed_events", 2024-04-25T10:44:37.380 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.380 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.380 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.380 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "50", 2024-04-25T10:44:37.380 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.380 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.380 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.380 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "number of past completed events to remember", 2024-04-25T10:44:37.380 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.380 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.380 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.380 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-25T10:44:37.380 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-25T10:44:37.380 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.381 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.381 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.381 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-25T10:44:37.381 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.381 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.381 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.381 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long the module is going to sleep", 2024-04-25T10:44:37.381 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.381 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.381 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.381 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.381 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.381 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.381 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.382 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.382 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.382 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.382 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.382 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.382 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.382 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.382 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.382 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.382 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.382 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.382 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus", 2024-04-25T10:44:37.383 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.383 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.383 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.383 INFO:teuthology.orchestra.run.smithi043.stdout: "cache": { 2024-04-25T10:44:37.383 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cache", 2024-04-25T10:44:37.383 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.383 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.383 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.383 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.383 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.383 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.383 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.383 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.383 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.384 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.384 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.384 INFO:teuthology.orchestra.run.smithi043.stdout: "exclude_perf_counters": { 2024-04-25T10:44:37.384 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "exclude_perf_counters", 2024-04-25T10:44:37.384 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.384 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.384 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.384 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.384 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.384 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.384 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.384 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T10:44:37.384 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T10:44:37.385 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.385 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.385 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.385 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.385 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.385 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.385 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.385 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.385 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.385 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.385 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.385 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.385 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.385 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.386 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.386 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.386 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.386 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.386 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.386 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.386 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.386 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.386 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.386 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.386 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.386 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.386 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.387 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.387 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.387 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.387 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.387 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.387 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.387 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.387 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.387 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.387 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.387 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.387 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.387 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.387 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.388 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.388 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.388 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.388 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.388 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.388 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.388 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.388 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.388 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.388 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.388 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.388 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.388 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.388 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.389 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.389 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.389 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.389 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.389 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.389 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.389 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.389 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.389 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.389 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.389 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.389 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.389 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.390 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools": { 2024-04-25T10:44:37.390 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools", 2024-04-25T10:44:37.390 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.390 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.390 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.390 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.390 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.390 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.390 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.390 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.390 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.390 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.391 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.391 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T10:44:37.391 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T10:44:37.391 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.391 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.391 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.391 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "300", 2024-04-25T10:44:37.391 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.391 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.391 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.391 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.391 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.391 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.391 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.392 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_interval": { 2024-04-25T10:44:37.392 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_interval", 2024-04-25T10:44:37.392 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:37.392 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.392 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.392 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15.0", 2024-04-25T10:44:37.392 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.392 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.392 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.392 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.392 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.392 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.392 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.393 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-04-25T10:44:37.393 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-04-25T10:44:37.393 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.393 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.393 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.393 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-04-25T10:44:37.393 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.393 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.393 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.393 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T10:44:37.393 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.393 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.393 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.394 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-04-25T10:44:37.394 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-04-25T10:44:37.394 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.394 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.394 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.394 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "9283", 2024-04-25T10:44:37.394 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.394 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.394 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.394 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T10:44:37.394 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.394 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.394 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.395 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.395 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.395 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.395 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.395 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.395 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.395 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.395 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.395 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.395 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.395 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.395 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.395 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.396 INFO:teuthology.orchestra.run.smithi043.stdout: "stale_cache_strategy": { 2024-04-25T10:44:37.396 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stale_cache_strategy", 2024-04-25T10:44:37.396 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.396 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.396 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.396 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "log", 2024-04-25T10:44:37.396 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.396 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.396 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.396 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.396 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.396 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.396 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.396 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-04-25T10:44:37.397 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-04-25T10:44:37.397 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.397 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.397 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.397 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "default", 2024-04-25T10:44:37.397 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.397 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.397 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.397 INFO:teuthology.orchestra.run.smithi043.stdout: "default", 2024-04-25T10:44:37.397 INFO:teuthology.orchestra.run.smithi043.stdout: "error" 2024-04-25T10:44:37.397 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.397 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.397 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.397 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.397 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.398 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-04-25T10:44:37.398 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-04-25T10:44:37.398 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.398 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.398 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.398 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-04-25T10:44:37.398 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-04-25T10:44:37.398 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-04-25T10:44:37.398 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.398 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.398 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.398 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.398 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.398 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.399 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.399 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_support", 2024-04-25T10:44:37.399 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.399 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.399 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.399 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.399 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.399 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.399 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.399 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.399 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.399 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.399 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.400 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.400 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.400 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.400 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.400 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.400 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.400 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.400 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.400 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.400 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.400 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.400 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.400 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.400 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.400 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.401 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.401 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.401 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.401 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.401 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.401 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.401 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.401 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.401 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.401 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.401 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.401 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.401 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.402 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.402 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.402 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.402 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.402 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.402 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.402 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.402 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.402 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.402 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.402 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.402 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.402 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.402 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.403 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.403 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.403 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.403 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.403 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.403 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.403 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.403 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.403 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.403 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.403 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.403 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.403 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.403 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.403 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.404 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.404 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_snap_create": { 2024-04-25T10:44:37.404 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_snap_create", 2024-04-25T10:44:37.404 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.404 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.404 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.404 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-25T10:44:37.404 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.404 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.404 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.404 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.404 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.404 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.405 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.405 INFO:teuthology.orchestra.run.smithi043.stdout: "mirror_snapshot_schedule": { 2024-04-25T10:44:37.405 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T10:44:37.405 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.405 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.405 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.405 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.405 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.405 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.405 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.405 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.405 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.405 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.406 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.406 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.406 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.406 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.406 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.406 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.409 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.409 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.409 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.410 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.410 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.410 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.410 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.411 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.411 INFO:teuthology.orchestra.run.smithi043.stdout: "trash_purge_schedule": { 2024-04-25T10:44:37.411 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "trash_purge_schedule", 2024-04-25T10:44:37.411 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.411 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.411 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.411 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.412 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.412 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.412 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.412 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.412 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.412 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.412 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.412 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.413 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.413 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "restful", 2024-04-25T10:44:37.413 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.413 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.413 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.413 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_auth": { 2024-04-25T10:44:37.414 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_auth", 2024-04-25T10:44:37.414 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.414 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.414 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.414 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.414 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.414 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.415 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.415 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.415 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.415 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.415 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.415 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-04-25T10:44:37.416 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-04-25T10:44:37.416 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.416 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.416 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.416 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.416 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.416 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.416 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.417 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.417 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.417 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.417 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.417 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.417 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.417 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.417 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.418 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.418 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.418 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.418 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.418 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.418 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.418 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.418 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.418 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.418 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.419 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.419 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.419 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.419 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.419 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.419 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.419 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.419 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.419 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.419 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.419 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.419 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.419 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.420 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.420 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.420 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.420 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.420 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.420 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.420 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.420 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.420 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.420 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.420 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.420 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.420 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.421 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.421 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.421 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.421 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.421 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.421 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.421 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.421 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.421 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.421 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.421 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.421 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.421 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.421 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.422 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.422 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.422 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.422 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.422 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.422 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.422 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.422 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.422 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.422 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.422 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.422 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.422 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-04-25T10:44:37.423 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-04-25T10:44:37.423 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.423 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.423 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.423 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.423 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.423 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.423 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.423 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.423 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.423 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.423 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.423 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-04-25T10:44:37.424 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-04-25T10:44:37.424 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.424 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.424 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.424 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.424 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.424 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.424 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.424 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.424 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.424 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.424 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.424 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.424 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.425 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.425 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.425 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.425 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.425 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.425 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.425 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.425 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.425 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.425 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.425 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.426 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rgw", 2024-04-25T10:44:37.426 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.426 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.426 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.426 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.426 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.426 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.426 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.426 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.427 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.427 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.427 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.427 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.427 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.427 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.427 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.427 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.427 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.427 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.427 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.427 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.427 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.428 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.428 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.428 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.428 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.428 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.428 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.428 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.428 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.428 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.428 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.428 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.428 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.428 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.429 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.429 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.429 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.429 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.429 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.429 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.429 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.429 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.429 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.429 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.429 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.429 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.429 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.429 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.430 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.430 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.430 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.430 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.430 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.430 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.430 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.430 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.430 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.430 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.430 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.430 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.430 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.431 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.431 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.431 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.431 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.431 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.431 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.431 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.431 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.431 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.431 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.431 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.431 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.431 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.432 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.432 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.432 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.432 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.432 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.432 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.432 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.432 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.433 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.433 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.433 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rook", 2024-04-25T10:44:37.433 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.434 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.434 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.434 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.434 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.434 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.434 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.434 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.435 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.435 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.435 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.435 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.435 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.435 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.436 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.436 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.436 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.436 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.436 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.436 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.437 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.437 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.437 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.437 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.437 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.437 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.438 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.438 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.438 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.438 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.438 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.439 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.439 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.439 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.439 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.439 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.439 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.440 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.440 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.440 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.440 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.440 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.440 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.441 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.441 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.441 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.441 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.441 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.441 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.442 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.442 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.442 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.442 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.442 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.443 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.443 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.443 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.443 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.443 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.443 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.444 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.444 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.444 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.444 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.444 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.444 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.445 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.445 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.445 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.445 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.445 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.446 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.446 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.446 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.446 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.446 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.447 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.447 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.447 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.447 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.447 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.447 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.448 INFO:teuthology.orchestra.run.smithi043.stdout: "storage_class": { 2024-04-25T10:44:37.448 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "storage_class", 2024-04-25T10:44:37.448 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.448 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.448 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.448 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "local", 2024-04-25T10:44:37.449 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.449 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.449 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.449 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T10:44:37.449 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.449 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.450 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.450 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.450 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.451 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "selftest", 2024-04-25T10:44:37.451 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.451 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.451 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.451 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.451 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.452 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.452 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.452 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.452 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.452 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.452 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.452 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.453 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.453 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.453 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.453 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.453 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.453 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.454 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.454 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.454 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.454 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.454 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.454 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.454 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.455 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.455 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.455 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.455 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.455 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.455 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.455 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.455 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.455 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.456 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.456 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.456 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.456 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.456 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.456 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.456 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.457 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.457 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.457 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.457 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.457 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.457 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.457 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.457 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.458 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.458 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.458 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.458 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.458 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.458 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.458 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.458 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.458 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.459 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.459 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.459 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.459 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.459 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.459 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.459 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.459 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.459 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.459 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.459 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.459 INFO:teuthology.orchestra.run.smithi043.stdout: "roption1": { 2024-04-25T10:44:37.459 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption1", 2024-04-25T10:44:37.460 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.460 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.460 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.460 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.460 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.460 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.460 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.460 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.460 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.460 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.460 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.460 INFO:teuthology.orchestra.run.smithi043.stdout: "roption2": { 2024-04-25T10:44:37.460 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption2", 2024-04-25T10:44:37.460 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.461 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.461 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.461 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "xyz", 2024-04-25T10:44:37.461 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.461 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.461 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.461 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.461 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.461 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.461 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.462 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption1": { 2024-04-25T10:44:37.462 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption1", 2024-04-25T10:44:37.462 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.462 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.462 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.462 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.462 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.463 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.463 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.463 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.463 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.463 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.463 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.463 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption2": { 2024-04-25T10:44:37.463 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption2", 2024-04-25T10:44:37.463 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.464 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.464 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.464 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.464 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.464 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.464 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.464 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.464 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.464 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.464 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.465 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption3": { 2024-04-25T10:44:37.465 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption3", 2024-04-25T10:44:37.465 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-25T10:44:37.465 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.465 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.465 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.465 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.465 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.465 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.465 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.465 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.465 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.465 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.466 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption4": { 2024-04-25T10:44:37.466 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption4", 2024-04-25T10:44:37.466 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.466 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.466 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.467 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.467 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.467 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.467 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.467 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.467 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.467 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.467 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.467 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption5": { 2024-04-25T10:44:37.468 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption5", 2024-04-25T10:44:37.468 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.468 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.468 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.468 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.468 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.468 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.468 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.468 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.468 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.469 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.469 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.469 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption6": { 2024-04-25T10:44:37.469 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption6", 2024-04-25T10:44:37.469 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.469 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.469 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.469 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.469 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.469 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.469 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.469 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.470 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.470 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.470 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.470 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption7": { 2024-04-25T10:44:37.470 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption7", 2024-04-25T10:44:37.470 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.470 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.470 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.470 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.470 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-25T10:44:37.470 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "42", 2024-04-25T10:44:37.471 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.471 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.471 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.471 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.471 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.471 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.471 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.471 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.471 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.471 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.471 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.471 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.471 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.472 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.472 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.472 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.472 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.472 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.472 INFO:teuthology.orchestra.run.smithi043.stdout: "testkey": { 2024-04-25T10:44:37.472 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testkey", 2024-04-25T10:44:37.472 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.472 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.473 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.473 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.473 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.473 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.473 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.473 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.473 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.473 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.473 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.473 INFO:teuthology.orchestra.run.smithi043.stdout: "testlkey": { 2024-04-25T10:44:37.473 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testlkey", 2024-04-25T10:44:37.473 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.473 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.474 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.474 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.474 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.474 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.474 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.474 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.474 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.474 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.474 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.474 INFO:teuthology.orchestra.run.smithi043.stdout: "testnewline": { 2024-04-25T10:44:37.475 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testnewline", 2024-04-25T10:44:37.475 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.475 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.475 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.475 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.475 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.475 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.475 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.475 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.475 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.475 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.475 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.476 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-04-25T10:44:37.476 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.476 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.476 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.476 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_m_granularity": { 2024-04-25T10:44:37.476 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_m_granularity", 2024-04-25T10:44:37.476 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.477 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.477 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.477 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.477 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.477 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.477 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.477 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T10:44:37.477 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.477 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.477 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.477 INFO:teuthology.orchestra.run.smithi043.stdout: "dump_on_update": { 2024-04-25T10:44:37.477 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dump_on_update", 2024-04-25T10:44:37.477 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.478 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.478 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.478 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.478 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.478 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.478 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.478 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "dump database to debug log on update", 2024-04-25T10:44:37.478 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.478 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.478 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.478 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.478 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.479 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.479 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.479 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.479 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.479 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.479 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.479 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.479 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.479 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.479 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.479 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.479 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.479 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.479 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.480 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.480 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.480 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.480 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.480 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.480 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.480 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.480 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.480 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.480 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.480 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.480 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.480 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.481 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.481 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.481 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.481 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.481 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.481 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.481 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.481 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.481 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.481 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.481 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.481 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.481 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.481 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.482 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.482 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.482 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.482 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.482 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.482 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.482 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.482 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.482 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.483 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.483 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.483 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.483 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.483 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.483 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.483 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.483 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.483 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.483 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.483 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.483 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.483 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.484 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.484 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.484 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.484 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.484 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.484 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.484 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.484 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.484 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.484 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.484 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.484 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.484 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.485 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.485 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.485 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.485 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stats", 2024-04-25T10:44:37.485 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.485 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.485 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.485 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.485 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.485 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.485 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.485 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.485 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.486 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.486 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.486 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.486 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.486 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.486 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.486 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.486 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.486 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.486 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.486 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.486 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.486 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.486 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.487 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.487 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.487 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.487 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.487 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.487 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.487 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.487 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.487 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.488 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.488 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.488 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.488 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.488 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.488 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.488 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.488 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.488 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.488 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.488 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.488 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.488 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.489 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.489 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.489 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.489 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.489 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.489 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.489 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.489 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.489 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.489 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.489 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.489 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.490 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.490 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.490 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.490 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.490 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.490 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.490 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.490 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.490 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.490 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.490 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.490 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.491 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.491 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.491 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.491 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.491 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.491 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.491 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.491 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.491 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.491 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.491 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.491 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.491 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.491 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.491 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.492 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "status", 2024-04-25T10:44:37.492 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.492 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.492 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.492 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.492 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.492 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.492 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.492 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.493 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.493 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.493 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.493 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.493 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.493 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.493 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.493 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.493 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.493 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.494 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.494 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.494 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.494 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.494 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.494 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.495 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.495 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.495 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.495 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.495 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.495 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.495 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.495 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.495 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.495 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.496 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.496 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.496 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.496 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.496 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.496 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.496 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.496 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.496 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.496 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.496 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.496 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.496 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.497 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.497 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.497 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.497 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.497 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.497 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.497 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.497 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.497 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.497 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.497 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.497 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.498 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.498 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.498 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.498 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.498 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.498 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.498 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.498 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.498 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.498 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.499 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.499 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.499 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.499 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.499 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.499 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.499 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.499 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.499 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.499 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.499 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.499 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.499 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.500 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.500 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.500 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telegraf", 2024-04-25T10:44:37.500 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.500 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.500 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.500 INFO:teuthology.orchestra.run.smithi043.stdout: "address": { 2024-04-25T10:44:37.500 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "address", 2024-04-25T10:44:37.500 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.500 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.500 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.500 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T10:44:37.501 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.501 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.501 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.501 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.501 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.501 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.501 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.501 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-25T10:44:37.501 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-25T10:44:37.501 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.501 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.501 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.502 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15", 2024-04-25T10:44:37.502 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.502 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.502 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.502 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.502 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.502 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.502 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.502 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.502 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.502 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.503 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.503 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.503 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.503 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.503 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.503 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.503 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.503 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.503 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.503 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.503 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.504 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.504 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.504 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.504 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.504 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.504 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.504 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.504 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.504 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.504 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.504 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.504 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.504 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.504 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.505 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.505 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.505 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.505 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.505 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.505 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.505 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.505 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.506 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.506 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.506 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.506 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.506 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.506 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.506 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.506 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.506 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.506 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.506 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.506 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.506 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.507 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.507 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.507 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.507 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.507 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.507 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.507 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.507 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.507 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.507 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.507 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.507 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.508 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.508 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.508 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.508 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.508 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.508 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.508 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.508 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.508 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.508 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.509 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.509 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.509 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.509 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.509 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.509 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.509 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.509 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.509 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.509 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.509 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telemetry", 2024-04-25T10:44:37.510 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.510 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.510 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.510 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_basic": { 2024-04-25T10:44:37.510 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_basic", 2024-04-25T10:44:37.510 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.510 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.510 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.510 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.510 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.510 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.510 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.510 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T10:44:37.511 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.511 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.511 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.511 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_crash": { 2024-04-25T10:44:37.512 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_crash", 2024-04-25T10:44:37.512 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.512 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.512 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.512 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.512 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.513 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.513 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.513 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T10:44:37.513 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.513 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.514 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.514 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_device": { 2024-04-25T10:44:37.514 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_device", 2024-04-25T10:44:37.514 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.515 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.515 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.515 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.515 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.515 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.515 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.515 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T10:44:37.516 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.516 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.516 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.516 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_ident": { 2024-04-25T10:44:37.516 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_ident", 2024-04-25T10:44:37.516 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.516 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.516 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.517 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.517 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.517 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.517 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.517 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T10:44:37.517 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.517 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.517 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.517 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_perf": { 2024-04-25T10:44:37.517 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_perf", 2024-04-25T10:44:37.517 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.517 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.517 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.517 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.518 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.518 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.518 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.518 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T10:44:37.518 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.518 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.518 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.518 INFO:teuthology.orchestra.run.smithi043.stdout: "contact": { 2024-04-25T10:44:37.518 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "contact", 2024-04-25T10:44:37.518 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.518 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.519 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.519 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.519 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.519 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.519 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.519 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.519 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.519 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.519 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.519 INFO:teuthology.orchestra.run.smithi043.stdout: "description": { 2024-04-25T10:44:37.519 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "description", 2024-04-25T10:44:37.520 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.520 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.520 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.520 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.520 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.520 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.520 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.520 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.520 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.520 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.520 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.520 INFO:teuthology.orchestra.run.smithi043.stdout: "device_url": { 2024-04-25T10:44:37.520 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_url", 2024-04-25T10:44:37.521 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.521 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.521 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.521 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T10:44:37.521 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.521 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.521 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.521 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.521 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.521 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.521 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.521 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-04-25T10:44:37.521 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-04-25T10:44:37.522 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.522 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.522 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.522 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.522 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.522 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.522 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.522 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.522 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.522 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.522 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.522 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-25T10:44:37.523 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-25T10:44:37.523 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.523 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.523 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.523 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "24", 2024-04-25T10:44:37.523 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "8", 2024-04-25T10:44:37.523 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.523 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.523 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.523 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.523 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.523 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.523 INFO:teuthology.orchestra.run.smithi043.stdout: "last_opt_revision": { 2024-04-25T10:44:37.523 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "last_opt_revision", 2024-04-25T10:44:37.524 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.524 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.524 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.524 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1", 2024-04-25T10:44:37.524 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.524 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.524 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.524 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.524 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.524 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.524 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.525 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard": { 2024-04-25T10:44:37.525 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard", 2024-04-25T10:44:37.525 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.525 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.525 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.525 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.525 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.525 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.525 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.525 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.525 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.525 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.525 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.526 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard_description": { 2024-04-25T10:44:37.526 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard_description", 2024-04-25T10:44:37.526 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.526 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.526 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.526 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.527 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.527 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.527 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.527 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.527 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.527 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.527 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.527 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.527 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.527 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.527 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.527 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.527 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.527 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.528 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.528 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.528 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.528 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.528 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.528 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.528 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.528 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.528 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.528 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.528 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.528 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.528 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.528 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.529 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.529 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.529 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.529 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.529 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.529 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.529 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.529 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.529 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.529 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.529 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.529 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.529 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.529 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.529 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.530 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.530 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.530 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.530 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.530 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.530 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.530 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.530 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.530 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.530 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.531 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.531 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.531 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.531 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.531 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.531 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.531 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.531 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.531 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.531 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.531 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.531 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.531 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.532 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.532 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.532 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.532 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.532 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.532 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.532 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.532 INFO:teuthology.orchestra.run.smithi043.stdout: "organization": { 2024-04-25T10:44:37.532 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "organization", 2024-04-25T10:44:37.532 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.532 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.532 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.532 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.532 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.533 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.533 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.533 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.533 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.533 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.533 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.533 INFO:teuthology.orchestra.run.smithi043.stdout: "proxy": { 2024-04-25T10:44:37.533 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "proxy", 2024-04-25T10:44:37.533 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.533 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.533 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.533 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.533 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.534 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.534 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.534 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.534 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.534 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.534 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.534 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.534 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.534 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.534 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.534 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.535 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.535 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.535 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.535 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.535 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.535 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.535 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.535 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.535 INFO:teuthology.orchestra.run.smithi043.stdout: "url": { 2024-04-25T10:44:37.535 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url", 2024-04-25T10:44:37.535 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.535 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.535 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.535 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T10:44:37.536 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.536 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.536 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.536 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.536 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.536 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.536 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.536 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.536 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.536 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "test_orchestrator", 2024-04-25T10:44:37.536 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.536 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.536 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.537 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.537 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.537 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.537 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.537 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.537 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.537 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.537 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.537 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.537 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.537 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.537 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.537 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.537 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.538 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.538 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.538 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.538 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.538 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.538 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.538 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.538 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.538 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.538 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.538 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.538 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.539 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.539 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.539 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.539 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.539 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.539 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.539 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.539 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.539 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.539 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.539 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.539 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.539 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.539 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.540 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.540 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.540 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.540 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.540 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.540 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.540 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.540 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.540 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.540 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.540 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.540 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.540 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.541 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.541 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.541 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.541 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.541 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.541 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.541 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.541 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.541 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.541 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.541 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.541 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.541 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.542 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.542 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.542 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.542 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.542 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.542 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.542 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.542 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.542 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.542 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.542 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.542 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.543 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.543 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "volumes", 2024-04-25T10:44:37.543 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.543 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.543 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.543 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.543 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.543 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.543 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.543 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.543 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.544 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.544 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.544 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.544 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.544 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.544 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.544 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.544 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.544 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.544 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.544 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.544 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.544 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.544 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.545 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.545 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.545 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.545 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.545 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.545 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.545 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.545 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.545 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.545 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.545 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.545 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.545 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.546 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.546 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.546 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.546 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.546 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.546 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.546 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.546 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.546 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.546 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.546 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.546 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.547 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.547 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.547 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.547 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.547 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.547 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.547 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.547 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.547 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.547 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.547 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.547 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.547 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.548 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.548 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.548 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.548 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.548 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.548 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.548 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.548 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.548 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_clones": { 2024-04-25T10:44:37.548 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_clones", 2024-04-25T10:44:37.548 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.548 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.548 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.549 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4", 2024-04-25T10:44:37.549 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.549 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.549 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.549 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T10:44:37.549 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.549 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.549 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.549 INFO:teuthology.orchestra.run.smithi043.stdout: "periodic_async_work": { 2024-04-25T10:44:37.549 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "periodic_async_work", 2024-04-25T10:44:37.549 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.549 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.550 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.550 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.550 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.550 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.550 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.550 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Periodically check for async work", 2024-04-25T10:44:37.550 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.550 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.550 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.550 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_delay": { 2024-04-25T10:44:37.550 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_delay", 2024-04-25T10:44:37.550 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.550 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.551 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.551 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.551 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.551 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.551 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.551 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T10:44:37.551 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.551 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.551 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.551 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_no_wait": { 2024-04-25T10:44:37.551 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T10:44:37.551 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.551 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.552 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.552 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-25T10:44:37.552 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.552 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.552 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.552 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T10:44:37.552 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.552 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.552 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.552 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.552 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.552 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.552 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.552 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.553 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.553 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.553 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.553 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.553 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.553 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.553 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.553 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.553 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.553 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.553 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix", 2024-04-25T10:44:37.553 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-25T10:44:37.553 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-25T10:44:37.554 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-25T10:44:37.554 INFO:teuthology.orchestra.run.smithi043.stdout: "discovery_interval": { 2024-04-25T10:44:37.554 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "discovery_interval", 2024-04-25T10:44:37.554 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-25T10:44:37.554 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.554 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.554 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "100", 2024-04-25T10:44:37.554 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.555 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.555 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.555 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.555 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.555 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.555 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.556 INFO:teuthology.orchestra.run.smithi043.stdout: "identifier": { 2024-04-25T10:44:37.556 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "identifier", 2024-04-25T10:44:37.556 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.556 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.556 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.557 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.557 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.557 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.557 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.557 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.557 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.558 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.558 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.558 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-25T10:44:37.558 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-25T10:44:37.558 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-25T10:44:37.559 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.559 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.559 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-25T10:44:37.559 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.559 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.559 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.560 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.560 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.560 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.560 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.560 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-25T10:44:37.561 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-25T10:44:37.561 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.561 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.561 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.561 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.561 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.562 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.562 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.562 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.562 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.562 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.563 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.563 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.563 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.563 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.563 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.563 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.564 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.564 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.564 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-25T10:44:37.564 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-25T10:44:37.564 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.564 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.565 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.565 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.565 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.565 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.565 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.565 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.566 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.566 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.566 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.566 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-25T10:44:37.567 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-25T10:44:37.567 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.567 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.567 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.567 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-25T10:44:37.567 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.568 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.568 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-25T10:44:37.568 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-25T10:44:37.568 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-25T10:44:37.568 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-25T10:44:37.568 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-25T10:44:37.569 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-25T10:44:37.569 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-25T10:44:37.569 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.569 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.569 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.569 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.570 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.570 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-25T10:44:37.570 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-25T10:44:37.570 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-25T10:44:37.570 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.571 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.571 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-25T10:44:37.571 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.571 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.571 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.571 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.572 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.572 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.572 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.572 INFO:teuthology.orchestra.run.smithi043.stdout: "sqlite3_killpoint": { 2024-04-25T10:44:37.572 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sqlite3_killpoint", 2024-04-25T10:44:37.573 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.573 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "dev", 2024-04-25T10:44:37.573 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-25T10:44:37.573 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-25T10:44:37.573 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.574 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.574 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.574 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.574 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.574 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.574 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.575 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_host": { 2024-04-25T10:44:37.575 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_host", 2024-04-25T10:44:37.575 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.575 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.575 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.576 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-25T10:44:37.576 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.576 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.576 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.576 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.576 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.576 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.576 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.577 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_port": { 2024-04-25T10:44:37.577 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_port", 2024-04-25T10:44:37.577 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-25T10:44:37.577 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.577 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.577 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10051", 2024-04-25T10:44:37.577 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.577 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.577 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.578 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.578 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.578 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.578 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.578 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_sender": { 2024-04-25T10:44:37.578 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_sender", 2024-04-25T10:44:37.578 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-25T10:44:37.579 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-25T10:44:37.579 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-25T10:44:37.579 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T10:44:37.579 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-25T10:44:37.579 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-25T10:44:37.579 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-25T10:44:37.580 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-25T10:44:37.580 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-25T10:44:37.580 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-25T10:44:37.580 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-25T10:44:37.580 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.580 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.580 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.580 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.580 INFO:teuthology.orchestra.run.smithi043.stdout: "services": { 2024-04-25T10:44:37.580 INFO:teuthology.orchestra.run.smithi043.stdout: "dashboard": "https://172.21.15.43:8443/" 2024-04-25T10:44:37.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.580 INFO:teuthology.orchestra.run.smithi043.stdout: "always_on_modules": { 2024-04-25T10:44:37.580 INFO:teuthology.orchestra.run.smithi043.stdout: "octopus": [ 2024-04-25T10:44:37.581 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-04-25T10:44:37.581 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-04-25T10:44:37.581 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-04-25T10:44:37.581 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-04-25T10:44:37.581 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-04-25T10:44:37.581 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-04-25T10:44:37.581 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-04-25T10:44:37.581 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-04-25T10:44:37.581 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-04-25T10:44:37.581 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-04-25T10:44:37.581 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.581 INFO:teuthology.orchestra.run.smithi043.stdout: "pacific": [ 2024-04-25T10:44:37.581 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-04-25T10:44:37.581 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-04-25T10:44:37.582 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-04-25T10:44:37.582 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-04-25T10:44:37.582 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-04-25T10:44:37.582 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-04-25T10:44:37.582 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-04-25T10:44:37.582 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-04-25T10:44:37.582 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-04-25T10:44:37.582 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-04-25T10:44:37.582 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.582 INFO:teuthology.orchestra.run.smithi043.stdout: "quincy": [ 2024-04-25T10:44:37.582 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-04-25T10:44:37.582 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-04-25T10:44:37.582 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-04-25T10:44:37.583 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-04-25T10:44:37.583 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-04-25T10:44:37.583 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-04-25T10:44:37.583 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-04-25T10:44:37.583 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-04-25T10:44:37.583 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-04-25T10:44:37.583 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-04-25T10:44:37.583 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.583 INFO:teuthology.orchestra.run.smithi043.stdout: "reef": [ 2024-04-25T10:44:37.583 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-04-25T10:44:37.583 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-04-25T10:44:37.583 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-04-25T10:44:37.583 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-04-25T10:44:37.583 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-04-25T10:44:37.584 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-04-25T10:44:37.584 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-04-25T10:44:37.584 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-04-25T10:44:37.584 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-04-25T10:44:37.584 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-04-25T10:44:37.584 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-25T10:44:37.584 INFO:teuthology.orchestra.run.smithi043.stdout: "squid": [ 2024-04-25T10:44:37.584 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-04-25T10:44:37.584 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-04-25T10:44:37.584 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-04-25T10:44:37.584 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-04-25T10:44:37.584 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-04-25T10:44:37.584 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-04-25T10:44:37.584 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-04-25T10:44:37.584 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-04-25T10:44:37.585 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-04-25T10:44:37.585 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-04-25T10:44:37.585 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T10:44:37.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.585 INFO:teuthology.orchestra.run.smithi043.stdout: "last_failure_osd_epoch": 80, 2024-04-25T10:44:37.585 INFO:teuthology.orchestra.run.smithi043.stdout: "active_clients": [ 2024-04-25T10:44:37.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.585 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "devicehealth", 2024-04-25T10:44:37.585 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-04-25T10:44:37.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.585 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-04-25T10:44:37.585 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-04-25T10:44:37.585 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3461772670 2024-04-25T10:44:37.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.586 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T10:44:37.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.586 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "libcephsqlite", 2024-04-25T10:44:37.586 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-04-25T10:44:37.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.586 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-04-25T10:44:37.586 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-04-25T10:44:37.586 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 866757320 2024-04-25T10:44:37.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.587 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T10:44:37.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.587 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_support", 2024-04-25T10:44:37.587 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-04-25T10:44:37.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.587 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-04-25T10:44:37.587 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-04-25T10:44:37.587 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2429312738 2024-04-25T10:44:37.587 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.587 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T10:44:37.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.588 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-04-25T10:44:37.588 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-04-25T10:44:37.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.588 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-04-25T10:44:37.588 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-04-25T10:44:37.588 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 823595894 2024-04-25T10:44:37.588 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.588 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T10:44:37.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.588 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-04-25T10:44:37.588 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-04-25T10:44:37.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.589 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-04-25T10:44:37.589 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-04-25T10:44:37.589 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 635753069 2024-04-25T10:44:37.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.589 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T10:44:37.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:37.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.589 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "volumes", 2024-04-25T10:44:37.589 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-04-25T10:44:37.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T10:44:37.589 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-04-25T10:44:37.590 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-04-25T10:44:37.590 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1543182218 2024-04-25T10:44:37.590 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.590 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T10:44:37.590 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:37.590 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T10:44:37.590 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:44:37.592 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:36 smithi164 ceph-mon[32713]: pgmap v377: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-04-25T10:44:37.592 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:36 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3273775325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:44:37.593 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:36 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/665347424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:44:37.593 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:36 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/973180345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:44:37.593 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:37 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/4006467610' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T10:44:37.593 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:36 smithi077 ceph-mon[32772]: pgmap v377: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-04-25T10:44:37.593 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:36 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3273775325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:44:37.593 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:36 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/665347424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:44:37.594 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:36 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/973180345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:44:37.594 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:37 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/4006467610' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T10:44:37.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:36.096+0000 7f4327897640 1 -- 172.21.15.43:0/4006467610 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f43080afc60 msgr2=0x7f43080b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:37.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:36.096+0000 7f4327897640 1 --2- 172.21.15.43:0/4006467610 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f43080afc60 0x7f43080b2170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4320103a30 tx=0x7f4310009970 comp rx=0 tx=0).stop 2024-04-25T10:44:37.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:36.096+0000 7f4327897640 1 -- 172.21.15.43:0/4006467610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320108a30 msgr2=0x7f432019ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:44:37.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:36.096+0000 7f4327897640 1 --2- 172.21.15.43:0/4006467610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320108a30 0x7f432019ba90 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f431402a930 tx=0x7f4314004270 comp rx=0 tx=0).stop 2024-04-25T10:44:37.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:36.096+0000 7f4327897640 1 -- 172.21.15.43:0/4006467610 shutdown_connections 2024-04-25T10:44:37.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:36.096+0000 7f4327897640 1 --2- 172.21.15.43:0/4006467610 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f43080afc60 0x7f43080b2170 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:37.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:36.096+0000 7f4327897640 1 --2- 172.21.15.43:0/4006467610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f432010d390 0x7f432019bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:37.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:36.096+0000 7f4327897640 1 --2- 172.21.15.43:0/4006467610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320108a30 0x7f432019ba90 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:37.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:36.096+0000 7f4327897640 1 --2- 172.21.15.43:0/4006467610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4320102b00 0x7f432019b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:44:37.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:36.096+0000 7f4327897640 1 -- 172.21.15.43:0/4006467610 >> 172.21.15.43:0/4006467610 conn(0x7f43200fe080 msgr2=0x7f432010e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:44:37.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:36.097+0000 7f4327897640 1 -- 172.21.15.43:0/4006467610 shutdown_connections 2024-04-25T10:44:37.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:44:36.097+0000 7f4327897640 1 -- 172.21.15.43:0/4006467610 wait complete. 2024-04-25T10:44:37.609 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T10:44:37.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T10:44:37.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:36 smithi043 ceph-mon[29478]: pgmap v377: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-04-25T10:44:37.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:36 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3273775325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:44:37.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:36 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/665347424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:44:37.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:36 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/973180345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:44:37.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:37 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/4006467610' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T10:44:37.877 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:44:38.340 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:38 smithi043 ceph-mon[29478]: pgmap v378: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-25T10:44:38.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:38 smithi077 ceph-mon[32772]: pgmap v378: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-25T10:44:38.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:38 smithi164 ceph-mon[32713]: pgmap v378: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-25T10:44:39.739 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:44:39.740 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:44:39.740 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 4, 2024-04-25T10:44:39.740 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 4, 2024-04-25T10:44:39.740 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 3040, 2024-04-25T10:44:39.740 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 448, 2024-04-25T10:44:39.740 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 4, 2024-04-25T10:44:39.740 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 0, 2024-04-25T10:44:39.740 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.001887161, 2024-04-25T10:44:39.740 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.000040299, 2024-04-25T10:44:39.740 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.000701880, 2024-04-25T10:44:39.740 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T10:44:39.740 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:39.740 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 3, 2024-04-25T10:44:39.740 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000833227, 2024-04-25T10:44:39.741 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000277742 2024-04-25T10:44:39.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:39.741 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:39.741 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T10:44:39.741 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T10:44:39.741 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T10:44:39.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:39.741 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T10:44:39.741 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T10:44:39.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:39.741 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:44:39.741 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 2, 2024-04-25T10:44:39.741 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 175, 2024-04-25T10:44:39.742 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 320, 2024-04-25T10:44:39.742 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 188000, 2024-04-25T10:44:39.742 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 5, 2024-04-25T10:44:39.742 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 3, 2024-04-25T10:44:39.742 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.015672548, 2024-04-25T10:44:39.742 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.010820799, 2024-04-25T10:44:39.742 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.002218838, 2024-04-25T10:44:39.742 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T10:44:39.742 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:39.742 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 172, 2024-04-25T10:44:39.742 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.014673924, 2024-04-25T10:44:39.742 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000085313 2024-04-25T10:44:39.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:39.743 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:39.743 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T10:44:39.743 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T10:44:39.743 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T10:44:39.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:39.743 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T10:44:39.743 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 188000 2024-04-25T10:44:39.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:39.743 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:44:39.743 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 240, 2024-04-25T10:44:39.743 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 229, 2024-04-25T10:44:39.743 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1692496, 2024-04-25T10:44:39.743 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 140864, 2024-04-25T10:44:39.743 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 2, 2024-04-25T10:44:39.744 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 1, 2024-04-25T10:44:39.744 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.053228756, 2024-04-25T10:44:39.744 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.015974958, 2024-04-25T10:44:39.744 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.044675440, 2024-04-25T10:44:39.744 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.004881941, 2024-04-25T10:44:39.744 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:39.744 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 228, 2024-04-25T10:44:39.744 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.011322067, 2024-04-25T10:44:39.744 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000049658 2024-04-25T10:44:39.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:39.744 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:39.744 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T10:44:39.744 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T10:44:39.744 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T10:44:39.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:39.745 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 1692496, 2024-04-25T10:44:39.745 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 140864 2024-04-25T10:44:39.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:39.745 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-04-25T10:44:39.745 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 0, 2024-04-25T10:44:39.745 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-04-25T10:44:39.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:39.745 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-04-25T10:44:39.745 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:44:39.745 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-04-25T10:44:39.745 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:44:39.745 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:44:39.745 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:44:39.745 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:44:39.746 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:44:39.746 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:44:39.746 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:44:39.746 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:44:39.746 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:44:39.746 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:44:39.746 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:44:39.746 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:44:39.746 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:44:39.746 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-04-25T10:44:39.746 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:44:39.746 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-04-25T10:44:39.746 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:44:39.746 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:44:39.747 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:44:39.747 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:44:39.747 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:44:39.747 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:44:39.747 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:44:39.747 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-04-25T10:44:39.747 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:44:39.747 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:44:39.747 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:44:39.747 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-04-25T10:44:39.747 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-04-25T10:44:39.747 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-04-25T10:44:39.747 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:44:39.747 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:44:39.748 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:44:39.748 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:44:39.748 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 149034, 2024-04-25T10:44:39.748 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 32, 2024-04-25T10:44:39.748 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:44:39.748 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-04-25T10:44:39.748 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-04-25T10:44:39.748 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-04-25T10:44:39.748 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:44:39.748 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-04-25T10:44:39.748 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:44:39.748 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-04-25T10:44:39.748 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 0, 2024-04-25T10:44:39.748 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 0, 2024-04-25T10:44:39.749 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:44:39.749 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:44:39.749 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-04-25T10:44:39.749 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-04-25T10:44:39.749 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 928, 2024-04-25T10:44:39.749 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 2, 2024-04-25T10:44:39.749 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-04-25T10:44:39.749 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-04-25T10:44:39.749 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-04-25T10:44:39.749 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-04-25T10:44:39.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:39.749 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:44:39.749 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T10:44:39.749 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-25T10:44:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T10:44:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 242, 2024-04-25T10:44:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 1658621, 2024-04-25T10:44:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T10:44:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 242, 2024-04-25T10:44:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-25T10:44:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-25T10:44:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 242, 2024-04-25T10:44:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 1658621, 2024-04-25T10:44:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T10:44:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T10:44:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T10:44:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T10:44:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:39.751 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:39.751 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:44:40.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:40 smithi043 ceph-mon[29478]: pgmap v379: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T10:44:40.423 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-25T10:44:40.423 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-25T10:44:40.423 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-04-25T10:44:40.423 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-04-25T10:44:40.423 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-25T10:44:40.424 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-25T10:44:40.424 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-25T10:44:40.424 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T10:44:40.424 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T10:44:40.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:40 smithi077 ceph-mon[32772]: pgmap v379: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T10:44:40.540 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:40 smithi164 ceph-mon[32713]: pgmap v379: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T10:44:40.682 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:44:42.331 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:42 smithi043 ceph-mon[29478]: pgmap v380: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-25T10:44:42.332 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:44:42.332 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:44:42.332 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 0, 2024-04-25T10:44:42.332 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 2, 2024-04-25T10:44:42.332 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 0, 2024-04-25T10:44:42.332 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 0, 2024-04-25T10:44:42.332 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 4, 2024-04-25T10:44:42.332 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 0, 2024-04-25T10:44:42.332 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.001468157, 2024-04-25T10:44:42.333 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T10:44:42.333 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.000359760, 2024-04-25T10:44:42.333 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T10:44:42.333 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:42.333 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T10:44:42.333 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T10:44:42.333 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T10:44:42.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:42.333 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:42.333 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T10:44:42.333 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T10:44:42.333 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T10:44:42.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:42.333 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T10:44:42.334 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T10:44:42.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:42.334 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:44:42.334 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 6, 2024-04-25T10:44:42.334 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 181, 2024-04-25T10:44:42.334 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 3360, 2024-04-25T10:44:42.334 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 189152, 2024-04-25T10:44:42.334 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 5, 2024-04-25T10:44:42.334 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 3, 2024-04-25T10:44:42.334 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.016056151, 2024-04-25T10:44:42.334 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.010892952, 2024-04-25T10:44:42.334 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.002458192, 2024-04-25T10:44:42.334 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T10:44:42.335 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:42.335 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 179, 2024-04-25T10:44:42.335 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.013761161, 2024-04-25T10:44:42.335 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000076877 2024-04-25T10:44:42.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:42.335 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:42.335 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T10:44:42.335 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T10:44:42.335 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T10:44:42.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:42.336 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T10:44:42.336 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 189152 2024-04-25T10:44:42.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:42.336 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:44:42.336 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 246, 2024-04-25T10:44:42.336 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 235, 2024-04-25T10:44:42.336 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1694128, 2024-04-25T10:44:42.336 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 144704, 2024-04-25T10:44:42.336 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 2, 2024-04-25T10:44:42.336 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 1, 2024-04-25T10:44:42.336 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.054979041, 2024-04-25T10:44:42.337 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.016687525, 2024-04-25T10:44:42.337 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.043984003, 2024-04-25T10:44:42.337 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.005251919, 2024-04-25T10:44:42.337 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:42.337 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 234, 2024-04-25T10:44:42.337 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.012282869, 2024-04-25T10:44:42.337 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000052490 2024-04-25T10:44:42.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:42.337 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:42.337 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T10:44:42.337 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T10:44:42.337 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T10:44:42.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:42.337 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 1694128, 2024-04-25T10:44:42.338 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 144704 2024-04-25T10:44:42.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:42.338 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-04-25T10:44:42.338 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 0, 2024-04-25T10:44:42.338 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-04-25T10:44:42.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:42.338 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-04-25T10:44:42.338 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:44:42.338 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-04-25T10:44:42.338 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:44:42.339 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:44:42.339 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:44:42.339 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:44:42.339 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:44:42.339 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:44:42.339 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:44:42.339 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:44:42.339 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:44:42.339 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:44:42.339 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:44:42.339 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:44:42.340 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:44:42.340 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-04-25T10:44:42.340 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:44:42.340 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-04-25T10:44:42.340 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:44:42.340 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:44:42.340 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:44:42.340 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:44:42.340 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:44:42.340 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:44:42.340 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:44:42.340 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-04-25T10:44:42.341 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:44:42.341 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:44:42.341 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:44:42.341 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-04-25T10:44:42.341 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-04-25T10:44:42.341 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-04-25T10:44:42.341 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:44:42.341 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:44:42.341 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:44:42.341 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:44:42.341 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 148994, 2024-04-25T10:44:42.341 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 31, 2024-04-25T10:44:42.341 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:44:42.341 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-04-25T10:44:42.342 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-04-25T10:44:42.342 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-04-25T10:44:42.342 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:44:42.342 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-04-25T10:44:42.342 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:44:42.342 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-04-25T10:44:42.342 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 0, 2024-04-25T10:44:42.342 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 0, 2024-04-25T10:44:42.342 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:44:42.342 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:44:42.342 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-04-25T10:44:42.342 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-04-25T10:44:42.342 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 928, 2024-04-25T10:44:42.343 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 2, 2024-04-25T10:44:42.343 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-04-25T10:44:42.343 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-04-25T10:44:42.343 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-04-25T10:44:42.343 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-04-25T10:44:42.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:42.343 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:44:42.343 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T10:44:42.343 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-25T10:44:42.343 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T10:44:42.343 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 248, 2024-04-25T10:44:42.343 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 1659401, 2024-04-25T10:44:42.343 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T10:44:42.343 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 248, 2024-04-25T10:44:42.343 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-25T10:44:42.344 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-25T10:44:42.344 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 248, 2024-04-25T10:44:42.344 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 1659401, 2024-04-25T10:44:42.344 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T10:44:42.344 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T10:44:42.344 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T10:44:42.344 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T10:44:42.344 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:42.344 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:42.344 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:44:42.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:42 smithi077 ceph-mon[32772]: pgmap v380: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-25T10:44:42.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:42 smithi164 ceph-mon[32713]: pgmap v380: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-25T10:44:42.996 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T10:44:42.997 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T10:44:42.997 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-25T10:44:42.997 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-25T10:44:42.997 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T10:44:42.997 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T10:44:42.997 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T10:44:42.997 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T10:44:42.997 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T10:44:43.252 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:44:44.323 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:44 smithi043 ceph-mon[29478]: pgmap v381: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-25T10:44:44.540 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:44 smithi077 ceph-mon[32772]: pgmap v381: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-25T10:44:44.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:44 smithi164 ceph-mon[32713]: pgmap v381: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-25T10:44:44.885 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:44:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:44:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 242, 2024-04-25T10:44:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 231, 2024-04-25T10:44:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1693040, 2024-04-25T10:44:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 142144, 2024-04-25T10:44:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 4, 2024-04-25T10:44:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 0, 2024-04-25T10:44:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.058371840, 2024-04-25T10:44:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.016223875, 2024-04-25T10:44:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.051453559, 2024-04-25T10:44:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.005086136, 2024-04-25T10:44:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 230, 2024-04-25T10:44:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.011895054, 2024-04-25T10:44:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000051717 2024-04-25T10:44:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T10:44:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T10:44:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T10:44:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 1693040, 2024-04-25T10:44:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 142144 2024-04-25T10:44:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:44:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 2, 2024-04-25T10:44:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 177, 2024-04-25T10:44:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 320, 2024-04-25T10:44:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 188352, 2024-04-25T10:44:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 5, 2024-04-25T10:44:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 3, 2024-04-25T10:44:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.016363205, 2024-04-25T10:44:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.011285898, 2024-04-25T10:44:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.002192054, 2024-04-25T10:44:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T10:44:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 174, 2024-04-25T10:44:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.013444982, 2024-04-25T10:44:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000077270 2024-04-25T10:44:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T10:44:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T10:44:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T10:44:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T10:44:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 188352 2024-04-25T10:44:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:44:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 4, 2024-04-25T10:44:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 4, 2024-04-25T10:44:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 3040, 2024-04-25T10:44:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 448, 2024-04-25T10:44:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 2, 2024-04-25T10:44:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 1, 2024-04-25T10:44:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.000777523, 2024-04-25T10:44:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.000049519, 2024-04-25T10:44:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.000727452, 2024-04-25T10:44:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T10:44:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 3, 2024-04-25T10:44:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000839416, 2024-04-25T10:44:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000279805 2024-04-25T10:44:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T10:44:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T10:44:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T10:44:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T10:44:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T10:44:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-04-25T10:44:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 0, 2024-04-25T10:44:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-04-25T10:44:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-04-25T10:44:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:44:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-04-25T10:44:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:44:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:44:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:44:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:44:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:44:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:44:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:44:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:44:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:44:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:44:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:44:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:44:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:44:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-04-25T10:44:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:44:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-04-25T10:44:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:44:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:44:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:44:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:44:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:44:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:44:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:44:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-04-25T10:44:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:44:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:44:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:44:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-04-25T10:44:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-04-25T10:44:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-04-25T10:44:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:44:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:44:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:44:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:44:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 148977, 2024-04-25T10:44:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 30, 2024-04-25T10:44:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:44:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-04-25T10:44:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-04-25T10:44:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-04-25T10:44:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:44:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-04-25T10:44:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:44:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-04-25T10:44:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 0, 2024-04-25T10:44:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 0, 2024-04-25T10:44:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:44:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:44:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-04-25T10:44:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-04-25T10:44:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 928, 2024-04-25T10:44:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 2, 2024-04-25T10:44:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-04-25T10:44:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-04-25T10:44:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-04-25T10:44:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-04-25T10:44:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:44:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T10:44:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-25T10:44:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T10:44:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 244, 2024-04-25T10:44:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 1658881, 2024-04-25T10:44:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T10:44:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 244, 2024-04-25T10:44:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-25T10:44:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-25T10:44:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 244, 2024-04-25T10:44:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 1658881, 2024-04-25T10:44:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T10:44:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T10:44:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T10:44:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T10:44:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:44.896 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:44:45.501 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-25T10:44:45.501 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-25T10:44:45.501 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-04-25T10:44:45.501 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-04-25T10:44:45.501 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-25T10:44:45.501 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-25T10:44:45.501 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-25T10:44:45.501 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T10:44:45.501 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T10:44:45.752 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T10:44:46.343 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:46 smithi077 ceph-mon[32772]: pgmap v382: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-25T10:44:46.343 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:46 smithi164 ceph-mon[32713]: pgmap v382: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-25T10:44:46.351 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:46 smithi043 ceph-mon[29478]: pgmap v382: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-25T10:44:47.560 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T10:44:47.560 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:44:47.560 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 4, 2024-04-25T10:44:47.560 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 4, 2024-04-25T10:44:47.560 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 3040, 2024-04-25T10:44:47.560 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 448, 2024-04-25T10:44:47.560 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 4, 2024-04-25T10:44:47.560 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 0, 2024-04-25T10:44:47.560 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.001747521, 2024-04-25T10:44:47.560 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.000045679, 2024-04-25T10:44:47.560 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.000740610, 2024-04-25T10:44:47.561 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T10:44:47.561 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:47.561 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 3, 2024-04-25T10:44:47.561 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000923251, 2024-04-25T10:44:47.561 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000307750 2024-04-25T10:44:47.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:47.561 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:47.561 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T10:44:47.561 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T10:44:47.561 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T10:44:47.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:47.561 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T10:44:47.561 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T10:44:47.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:47.562 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:44:47.562 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 2, 2024-04-25T10:44:47.562 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 181, 2024-04-25T10:44:47.562 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 320, 2024-04-25T10:44:47.562 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 189056, 2024-04-25T10:44:47.562 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 5, 2024-04-25T10:44:47.562 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 3, 2024-04-25T10:44:47.562 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.016888004, 2024-04-25T10:44:47.562 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.011550618, 2024-04-25T10:44:47.562 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.002751995, 2024-04-25T10:44:47.562 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T10:44:47.562 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:47.562 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 178, 2024-04-25T10:44:47.562 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.015469777, 2024-04-25T10:44:47.563 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000086908 2024-04-25T10:44:47.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:47.563 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:47.563 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T10:44:47.563 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T10:44:47.563 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T10:44:47.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:47.563 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T10:44:47.563 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 189056 2024-04-25T10:44:47.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:47.563 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:44:47.563 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 248, 2024-04-25T10:44:47.563 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 237, 2024-04-25T10:44:47.563 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1694672, 2024-04-25T10:44:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 145984, 2024-04-25T10:44:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 2, 2024-04-25T10:44:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 1, 2024-04-25T10:44:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.059442497, 2024-04-25T10:44:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.017729805, 2024-04-25T10:44:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.048567361, 2024-04-25T10:44:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.005319150, 2024-04-25T10:44:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 236, 2024-04-25T10:44:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.012207684, 2024-04-25T10:44:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000051727 2024-04-25T10:44:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T10:44:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T10:44:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T10:44:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 1694672, 2024-04-25T10:44:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 145984 2024-04-25T10:44:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-04-25T10:44:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 0, 2024-04-25T10:44:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-04-25T10:44:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-04-25T10:44:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:44:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-04-25T10:44:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:44:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:44:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:44:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:44:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:44:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:44:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:44:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:44:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:44:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:44:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:44:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:44:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:44:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-04-25T10:44:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:44:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-04-25T10:44:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:44:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:44:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:44:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:44:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:44:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:44:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:44:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-04-25T10:44:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:44:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:44:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:44:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-04-25T10:44:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-04-25T10:44:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-04-25T10:44:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:44:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:44:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:44:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:44:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 150048, 2024-04-25T10:44:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 32, 2024-04-25T10:44:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:44:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-04-25T10:44:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-04-25T10:44:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-04-25T10:44:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:44:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-04-25T10:44:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:44:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-04-25T10:44:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 0, 2024-04-25T10:44:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 0, 2024-04-25T10:44:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:44:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:44:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-04-25T10:44:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-04-25T10:44:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 928, 2024-04-25T10:44:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 2, 2024-04-25T10:44:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-04-25T10:44:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-04-25T10:44:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-04-25T10:44:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-04-25T10:44:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T10:44:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:44:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T10:44:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-25T10:44:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T10:44:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 250, 2024-04-25T10:44:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 1659661, 2024-04-25T10:44:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T10:44:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 250, 2024-04-25T10:44:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-25T10:44:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-25T10:44:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 250, 2024-04-25T10:44:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 1659661, 2024-04-25T10:44:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T10:44:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T10:44:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T10:44:47.571 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T10:44:47.571 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:47.571 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T10:44:47.571 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T10:44:48.122 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:48 smithi043 ceph-mon[29478]: pgmap v383: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T10:44:48.123 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T10:44:48.124 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T10:44:48.124 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-25T10:44:48.124 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-25T10:44:48.124 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T10:44:48.124 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T10:44:48.124 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T10:44:48.124 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T10:44:48.124 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T10:44:48.375 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T10:44:48.498 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:48 smithi077 ceph-mon[32772]: pgmap v383: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T10:44:48.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:48 smithi164 ceph-mon[32713]: pgmap v383: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T10:44:50.288 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:50 smithi077 ceph-mon[32772]: pgmap v384: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T10:44:50.288 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-25T10:44:50.288 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:44:50.288 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 4612, 2024-04-25T10:44:50.288 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 4281, 2024-04-25T10:44:50.288 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 1645686, 2024-04-25T10:44:50.288 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 14402248, 2024-04-25T10:44:50.289 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 17, 2024-04-25T10:44:50.289 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 4, 2024-04-25T10:44:50.289 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.503420810, 2024-04-25T10:44:50.289 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.157797294, 2024-04-25T10:44:50.289 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.155857988, 2024-04-25T10:44:50.289 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.148982200, 2024-04-25T10:44:50.289 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:50.289 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 4282, 2024-04-25T10:44:50.289 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.253193132, 2024-04-25T10:44:50.289 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000059129 2024-04-25T10:44:50.289 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.289 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:50.290 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 3185, 2024-04-25T10:44:50.290 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000916154, 2024-04-25T10:44:50.290 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000287 2024-04-25T10:44:50.290 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.290 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 416240, 2024-04-25T10:44:50.290 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 110368 2024-04-25T10:44:50.290 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.290 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:44:50.290 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 2309, 2024-04-25T10:44:50.290 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 2837, 2024-04-25T10:44:50.290 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 788125, 2024-04-25T10:44:50.290 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 11331507, 2024-04-25T10:44:50.290 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 291, 2024-04-25T10:44:50.291 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 5, 2024-04-25T10:44:50.291 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.338838274, 2024-04-25T10:44:50.291 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.089683643, 2024-04-25T10:44:50.291 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.166152740, 2024-04-25T10:44:50.291 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.063868837, 2024-04-25T10:44:50.291 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:50.291 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2839, 2024-04-25T10:44:50.291 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.379715391, 2024-04-25T10:44:50.291 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000133749 2024-04-25T10:44:50.291 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.291 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:50.291 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2433, 2024-04-25T10:44:50.291 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000633240, 2024-04-25T10:44:50.292 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000260 2024-04-25T10:44:50.292 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.292 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 46496, 2024-04-25T10:44:50.292 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 489168 2024-04-25T10:44:50.292 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.292 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:44:50.292 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 3544, 2024-04-25T10:44:50.292 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 5083, 2024-04-25T10:44:50.292 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 2825760, 2024-04-25T10:44:50.292 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 6535089, 2024-04-25T10:44:50.292 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 15, 2024-04-25T10:44:50.292 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 8, 2024-04-25T10:44:50.292 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.453908251, 2024-04-25T10:44:50.292 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.160937795, 2024-04-25T10:44:50.293 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.219156492, 2024-04-25T10:44:50.293 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.089615604, 2024-04-25T10:44:50.293 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:50.293 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 5075, 2024-04-25T10:44:50.293 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.276905450, 2024-04-25T10:44:50.293 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000054562 2024-04-25T10:44:50.293 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.293 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:50.293 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 4979, 2024-04-25T10:44:50.293 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.001265850, 2024-04-25T10:44:50.293 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000254 2024-04-25T10:44:50.293 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.293 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1858096, 2024-04-25T10:44:50.293 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 212160 2024-04-25T10:44:50.294 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.294 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-04-25T10:44:50.294 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2024-04-25T10:44:50.294 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-04-25T10:44:50.294 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.294 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2024-04-25T10:44:50.294 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-04-25T10:44:50.294 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-04-25T10:44:50.294 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 6384, 2024-04-25T10:44:50.294 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 2.012199522, 2024-04-25T10:44:50.294 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000315194 2024-04-25T10:44:50.294 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:50.295 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.295 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2024-04-25T10:44:50.295 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-04-25T10:44:50.295 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-04-25T10:44:50.295 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 16, 2024-04-25T10:44:50.295 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000375705, 2024-04-25T10:44:50.295 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000023481 2024-04-25T10:44:50.295 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:50.295 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.295 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-25T10:44:50.295 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 782, 2024-04-25T10:44:50.295 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 782, 2024-04-25T10:44:50.295 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2024-04-25T10:44:50.295 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 782, 2024-04-25T10:44:50.296 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 3.626064524, 2024-04-25T10:44:50.296 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.004636911 2024-04-25T10:44:50.296 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.296 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2024-04-25T10:44:50.296 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 0, 2024-04-25T10:44:50.296 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 0, 2024-04-25T10:44:50.296 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2024-04-25T10:44:50.296 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 352, 2024-04-25T10:44:50.296 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 0, 2024-04-25T10:44:50.296 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2024-04-25T10:44:50.296 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 42, 2024-04-25T10:44:50.296 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 14, 2024-04-25T10:44:50.296 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 28, 2024-04-25T10:44:50.296 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 0, 2024-04-25T10:44:50.296 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 22, 2024-04-25T10:44:50.297 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 0, 2024-04-25T10:44:50.297 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 2, 2024-04-25T10:44:50.297 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 2, 2024-04-25T10:44:50.297 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 2, 2024-04-25T10:44:50.297 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 848, 2024-04-25T10:44:50.297 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 835, 2024-04-25T10:44:50.297 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 0, 2024-04-25T10:44:50.297 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 0, 2024-04-25T10:44:50.297 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:44:50.297 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2024-04-25T10:44:50.297 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 0, 2024-04-25T10:44:50.297 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2024-04-25T10:44:50.297 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2024-04-25T10:44:50.297 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 0, 2024-04-25T10:44:50.298 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 0, 2024-04-25T10:44:50.298 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 0, 2024-04-25T10:44:50.298 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 0, 2024-04-25T10:44:50.298 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2024-04-25T10:44:50.298 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 315, 2024-04-25T10:44:50.298 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 0, 2024-04-25T10:44:50.298 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 4, 2024-04-25T10:44:50.298 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 234, 2024-04-25T10:44:50.298 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 6, 2024-04-25T10:44:50.298 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 315, 2024-04-25T10:44:50.298 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2024-04-25T10:44:50.298 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:44:50.298 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:44:50.299 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 203, 2024-04-25T10:44:50.299 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 23, 2024-04-25T10:44:50.299 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 180, 2024-04-25T10:44:50.299 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 112, 2024-04-25T10:44:50.299 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:44:50.299 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 615, 2024-04-25T10:44:50.299 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 612, 2024-04-25T10:44:50.299 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 599, 2024-04-25T10:44:50.299 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:44:50.299 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:44:50.299 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 736, 2024-04-25T10:44:50.299 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 714, 2024-04-25T10:44:50.299 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 2, 2024-04-25T10:44:50.299 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 621 2024-04-25T10:44:50.300 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.300 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2024-04-25T10:44:50.300 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 7, 2024-04-25T10:44:50.300 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 0, 2024-04-25T10:44:50.300 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 0, 2024-04-25T10:44:50.300 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 0, 2024-04-25T10:44:50.300 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 16, 2024-04-25T10:44:50.300 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 2, 2024-04-25T10:44:50.300 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2024-04-25T10:44:50.300 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:44:50.300 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 3, 2024-04-25T10:44:50.300 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 1, 2024-04-25T10:44:50.300 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2024-04-25T10:44:50.300 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2024-04-25T10:44:50.300 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2024-04-25T10:44:50.301 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:44:50.301 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:44:50.301 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2024-04-25T10:44:50.301 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2024-04-25T10:44:50.301 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_path": 0, 2024-04-25T10:44:50.301 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_inode": 0, 2024-04-25T10:44:50.301 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 23, 2024-04-25T10:44:50.301 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 0, 2024-04-25T10:44:50.301 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2024-04-25T10:44:50.301 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:44:50.301 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2024-04-25T10:44:50.301 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2024-04-25T10:44:50.301 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.301 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2024-04-25T10:44:50.302 INFO:teuthology.orchestra.run.smithi077.stdout: "evlrg": 0, 2024-04-25T10:44:50.302 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 2593, 2024-04-25T10:44:50.302 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 2550, 2024-04-25T10:44:50.302 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 2550, 2024-04-25T10:44:50.302 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 43, 2024-04-25T10:44:50.302 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2024-04-25T10:44:50.302 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 64, 2024-04-25T10:44:50.302 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 59, 2024-04-25T10:44:50.302 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 58, 2024-04-25T10:44:50.302 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 58, 2024-04-25T10:44:50.302 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 1, 2024-04-25T10:44:50.302 INFO:teuthology.orchestra.run.smithi077.stdout: "segmjr": 1, 2024-04-25T10:44:50.302 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2024-04-25T10:44:50.303 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 1, 2024-04-25T10:44:50.303 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 26973433, 2024-04-25T10:44:50.303 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 27393246, 2024-04-25T10:44:50.303 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 0, 2024-04-25T10:44:50.303 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2024-04-25T10:44:50.303 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2523, 2024-04-25T10:44:50.303 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 6.046938471, 2024-04-25T10:44:50.303 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.002396725 2024-04-25T10:44:50.303 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.303 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 0 2024-04-25T10:44:50.303 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.303 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2024-04-25T10:44:50.303 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 35, 2024-04-25T10:44:50.303 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 59, 2024-04-25T10:44:50.303 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 24, 2024-04-25T10:44:50.304 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 20, 2024-04-25T10:44:50.304 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 20, 2024-04-25T10:44:50.304 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 0, 2024-04-25T10:44:50.304 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 42, 2024-04-25T10:44:50.304 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 43, 2024-04-25T10:44:50.304 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 1, 2024-04-25T10:44:50.304 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 2, 2024-04-25T10:44:50.304 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 19, 2024-04-25T10:44:50.304 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 17, 2024-04-25T10:44:50.304 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 128996, 2024-04-25T10:44:50.304 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 231824 2024-04-25T10:44:50.304 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.304 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2024-04-25T10:44:50.304 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 2041, 2024-04-25T10:44:50.305 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 1, 2024-04-25T10:44:50.305 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 782, 2024-04-25T10:44:50.305 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 38, 2024-04-25T10:44:50.305 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 2, 2024-04-25T10:44:50.305 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2024-04-25T10:44:50.305 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 5, 2024-04-25T10:44:50.305 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.003639814, 2024-04-25T10:44:50.305 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000727962 2024-04-25T10:44:50.305 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.305 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2024-04-25T10:44:50.305 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 120, 2024-04-25T10:44:50.305 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.022817855, 2024-04-25T10:44:50.306 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000190148 2024-04-25T10:44:50.306 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.306 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2024-04-25T10:44:50.306 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.306 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.306 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.306 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.306 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2024-04-25T10:44:50.306 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.306 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.306 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.306 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.306 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2024-04-25T10:44:50.307 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 3, 2024-04-25T10:44:50.307 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000495787, 2024-04-25T10:44:50.307 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000165262 2024-04-25T10:44:50.307 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.307 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2024-04-25T10:44:50.307 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.307 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.307 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.307 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.307 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2024-04-25T10:44:50.307 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.307 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.307 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.307 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.307 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2024-04-25T10:44:50.308 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.308 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.308 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.308 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.308 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2024-04-25T10:44:50.308 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.308 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.308 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.308 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.308 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2024-04-25T10:44:50.308 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.308 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.308 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.308 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.309 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2024-04-25T10:44:50.309 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 12, 2024-04-25T10:44:50.309 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.002185415, 2024-04-25T10:44:50.309 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000182117 2024-04-25T10:44:50.309 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.309 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2024-04-25T10:44:50.309 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 8, 2024-04-25T10:44:50.310 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.012161966, 2024-04-25T10:44:50.310 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001520245 2024-04-25T10:44:50.310 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.310 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2024-04-25T10:44:50.310 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.310 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.310 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.310 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.310 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2024-04-25T10:44:50.310 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 12, 2024-04-25T10:44:50.310 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.063676967, 2024-04-25T10:44:50.310 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.005306413 2024-04-25T10:44:50.310 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.311 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2024-04-25T10:44:50.311 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1, 2024-04-25T10:44:50.311 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000272975, 2024-04-25T10:44:50.311 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000272975 2024-04-25T10:44:50.311 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.311 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2024-04-25T10:44:50.311 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2, 2024-04-25T10:44:50.311 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.004458995, 2024-04-25T10:44:50.311 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.002229497 2024-04-25T10:44:50.311 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.311 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2024-04-25T10:44:50.311 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2, 2024-04-25T10:44:50.311 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.002112005, 2024-04-25T10:44:50.312 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001056002 2024-04-25T10:44:50.312 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.312 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2024-04-25T10:44:50.312 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.312 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.312 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.312 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.312 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2024-04-25T10:44:50.312 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.312 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.312 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.312 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.313 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2024-04-25T10:44:50.313 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2, 2024-04-25T10:44:50.313 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.011614716, 2024-04-25T10:44:50.313 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.005807358 2024-04-25T10:44:50.313 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.313 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2024-04-25T10:44:50.313 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 6, 2024-04-25T10:44:50.313 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.029224384, 2024-04-25T10:44:50.313 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.004870730 2024-04-25T10:44:50.313 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.313 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2024-04-25T10:44:50.313 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.313 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.314 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.314 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.314 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2024-04-25T10:44:50.314 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 600, 2024-04-25T10:44:50.314 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 3.450656172, 2024-04-25T10:44:50.314 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.005751093 2024-04-25T10:44:50.314 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.314 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2024-04-25T10:44:50.314 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.314 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.314 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.314 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.314 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2024-04-25T10:44:50.315 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.315 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.315 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.315 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.315 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2024-04-25T10:44:50.315 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.315 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.315 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.315 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.315 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2024-04-25T10:44:50.315 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 8, 2024-04-25T10:44:50.315 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.019862579, 2024-04-25T10:44:50.315 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.002482822 2024-04-25T10:44:50.315 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.315 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2024-04-25T10:44:50.316 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.316 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.316 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.316 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.316 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2024-04-25T10:44:50.316 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1, 2024-04-25T10:44:50.316 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.002884894, 2024-04-25T10:44:50.316 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.002884894 2024-04-25T10:44:50.316 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.316 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:44:50.316 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:44:50.316 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2024-04-25T10:44:50.316 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.316 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.316 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.317 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:50.317 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.317 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2024-04-25T10:44:50.317 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 2, 2024-04-25T10:44:50.317 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 6, 2024-04-25T10:44:50.317 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 4, 2024-04-25T10:44:50.317 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 2, 2024-04-25T10:44:50.317 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2024-04-25T10:44:50.317 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 113, 2024-04-25T10:44:50.317 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 56, 2024-04-25T10:44:50.317 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 723, 2024-04-25T10:44:50.317 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2024-04-25T10:44:50.317 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.318 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-04-25T10:44:50.318 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:44:50.318 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-04-25T10:44:50.318 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:44:50.318 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:44:50.318 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:44:50.318 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:44:50.318 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:44:50.318 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:44:50.319 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:44:50.319 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:44:50.319 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:44:50.319 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:44:50.319 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:44:50.319 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:44:50.319 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:44:50.319 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-04-25T10:44:50.319 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:44:50.319 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-04-25T10:44:50.320 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:44:50.320 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:44:50.320 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:44:50.320 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:44:50.320 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:44:50.320 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:44:50.320 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:44:50.320 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-04-25T10:44:50.320 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:44:50.320 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:44:50.320 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:44:50.320 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-04-25T10:44:50.320 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-04-25T10:44:50.321 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-04-25T10:44:50.321 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:44:50.321 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:44:50.321 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:44:50.321 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:44:50.321 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 306753, 2024-04-25T10:44:50.321 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 121, 2024-04-25T10:44:50.321 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:44:50.321 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-04-25T10:44:50.321 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-04-25T10:44:50.321 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-04-25T10:44:50.321 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:44:50.321 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-04-25T10:44:50.321 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:44:50.321 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-04-25T10:44:50.322 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 8032, 2024-04-25T10:44:50.322 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 167, 2024-04-25T10:44:50.322 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:44:50.322 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:44:50.322 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-04-25T10:44:50.322 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-04-25T10:44:50.322 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 173227, 2024-04-25T10:44:50.322 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 1339, 2024-04-25T10:44:50.322 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-04-25T10:44:50.322 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-04-25T10:44:50.322 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-04-25T10:44:50.322 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-04-25T10:44:50.322 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.322 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2024-04-25T10:44:50.323 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2024-04-25T10:44:50.323 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2024-04-25T10:44:50.323 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 4697, 2024-04-25T10:44:50.323 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 24382361, 2024-04-25T10:44:50.323 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 37, 2024-04-25T10:44:50.323 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 4697, 2024-04-25T10:44:50.323 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2024-04-25T10:44:50.323 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 4697, 2024-04-25T10:44:50.323 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 27.907454357, 2024-04-25T10:44:50.323 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.005941548 2024-04-25T10:44:50.323 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.323 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2024-04-25T10:44:50.323 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2024-04-25T10:44:50.323 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 4697, 2024-04-25T10:44:50.323 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 5447 2024-04-25T10:44:50.323 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.324 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 4697, 2024-04-25T10:44:50.324 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 362, 2024-04-25T10:44:50.324 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 4335, 2024-04-25T10:44:50.324 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2024-04-25T10:44:50.324 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2024-04-25T10:44:50.324 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 332, 2024-04-25T10:44:50.324 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 25, 2024-04-25T10:44:50.324 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 24, 2024-04-25T10:44:50.324 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 2524, 2024-04-25T10:44:50.324 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 185, 2024-04-25T10:44:50.324 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2024-04-25T10:44:50.324 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2024-04-25T10:44:50.324 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2024-04-25T10:44:50.324 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 0, 2024-04-25T10:44:50.324 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 523, 2024-04-25T10:44:50.324 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2024-04-25T10:44:50.325 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2024-04-25T10:44:50.325 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2024-04-25T10:44:50.325 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 338, 2024-04-25T10:44:50.325 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 50, 2024-04-25T10:44:50.325 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:44:50.325 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2024-04-25T10:44:50.325 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:44:50.325 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2024-04-25T10:44:50.325 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2024-04-25T10:44:50.325 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2024-04-25T10:44:50.325 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:44:50.325 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2024-04-25T10:44:50.325 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:44:50.325 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 599, 2024-04-25T10:44:50.325 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2024-04-25T10:44:50.326 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2024-04-25T10:44:50.326 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2024-04-25T10:44:50.326 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2024-04-25T10:44:50.326 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2024-04-25T10:44:50.326 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2024-04-25T10:44:50.326 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2024-04-25T10:44:50.326 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2024-04-25T10:44:50.326 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2024-04-25T10:44:50.326 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2024-04-25T10:44:50.326 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2024-04-25T10:44:50.326 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2024-04-25T10:44:50.326 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2024-04-25T10:44:50.326 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2024-04-25T10:44:50.327 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2024-04-25T10:44:50.327 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2024-04-25T10:44:50.327 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 93, 2024-04-25T10:44:50.327 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2024-04-25T10:44:50.327 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 17, 2024-04-25T10:44:50.327 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 12, 2024-04-25T10:44:50.327 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 12, 2024-04-25T10:44:50.327 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 0, 2024-04-25T10:44:50.327 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2024-04-25T10:44:50.327 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 836, 2024-04-25T10:44:50.327 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 0, 2024-04-25T10:44:50.327 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 11 2024-04-25T10:44:50.327 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.327 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2024-04-25T10:44:50.327 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 1, 2024-04-25T10:44:50.327 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:44:50.328 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2024-04-25T10:44:50.328 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2024-04-25T10:44:50.328 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.328 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2024-04-25T10:44:50.328 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 0 2024-04-25T10:44:50.328 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.328 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2024-04-25T10:44:50.328 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2024-04-25T10:44:50.328 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 1, 2024-04-25T10:44:50.328 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2024-04-25T10:44:50.328 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 1, 2024-04-25T10:44:50.328 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed_ops": 1, 2024-04-25T10:44:50.328 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 1, 2024-04-25T10:44:50.328 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2024-04-25T10:44:50.328 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.328 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:44:50.329 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-25T10:44:50.329 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-04-25T10:44:50.329 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-25T10:44:50.329 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 10461, 2024-04-25T10:44:50.329 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 4239189, 2024-04-25T10:44:50.329 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-25T10:44:50.329 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 10461, 2024-04-25T10:44:50.329 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-04-25T10:44:50.329 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-04-25T10:44:50.329 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 10461, 2024-04-25T10:44:50.329 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 4239189, 2024-04-25T10:44:50.329 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-25T10:44:50.329 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.329 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.329 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.330 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:50.330 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.330 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2024-04-25T10:44:50.330 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-25T10:44:50.330 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-04-25T10:44:50.330 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-25T10:44:50.330 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-04-25T10:44:50.330 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-04-25T10:44:50.330 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-25T10:44:50.330 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-04-25T10:44:50.330 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 4697, 2024-04-25T10:44:50.330 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 24382510, 2024-04-25T10:44:50.331 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 4697, 2024-04-25T10:44:50.331 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 24382510, 2024-04-25T10:44:50.331 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-25T10:44:50.331 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.331 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.331 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.331 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:50.331 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.331 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2024-04-25T10:44:50.331 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-25T10:44:50.331 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2024-04-25T10:44:50.331 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-25T10:44:50.331 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-04-25T10:44:50.331 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-04-25T10:44:50.332 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-25T10:44:50.332 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-04-25T10:44:50.332 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 4697, 2024-04-25T10:44:50.332 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 4697, 2024-04-25T10:44:50.332 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 4697, 2024-04-25T10:44:50.332 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 4697, 2024-04-25T10:44:50.332 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-25T10:44:50.332 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.332 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.332 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.332 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:50.332 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.332 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:44:50.332 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-25T10:44:50.333 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-04-25T10:44:50.333 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-25T10:44:50.333 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 1, 2024-04-25T10:44:50.333 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 149, 2024-04-25T10:44:50.333 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-25T10:44:50.333 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 1, 2024-04-25T10:44:50.333 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-04-25T10:44:50.333 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-04-25T10:44:50.333 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 1, 2024-04-25T10:44:50.333 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 149, 2024-04-25T10:44:50.333 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-25T10:44:50.333 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.333 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.333 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.334 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:50.334 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:50.334 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x55dcecf56750": { 2024-04-25T10:44:50.334 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-25T10:44:50.334 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-04-25T10:44:50.334 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-25T10:44:50.334 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 2593, 2024-04-25T10:44:50.334 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 23198942, 2024-04-25T10:44:50.334 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-25T10:44:50.338 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 2593, 2024-04-25T10:44:50.338 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-04-25T10:44:50.338 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-04-25T10:44:50.338 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 2523, 2024-04-25T10:44:50.338 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 23198942, 2024-04-25T10:44:50.338 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-25T10:44:50.338 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:50.338 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:50.338 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:50.338 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:50.338 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:50.339 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-25T10:44:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:50 smithi043 ceph-mon[29478]: pgmap v384: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T10:44:50.542 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:50 smithi164 ceph-mon[32713]: pgmap v384: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T10:44:51.083 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-25T10:44:51.083 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-25T10:44:51.083 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=7 2024-04-25T10:44:51.084 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-04-25T10:44:51.084 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-25T10:44:51.084 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=12 2024-04-25T10:44:51.084 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=6 2024-04-25T10:44:51.084 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T10:44:51.084 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T10:44:51.326 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T10:44:52.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:52 smithi043 ceph-mon[29478]: pgmap v385: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T10:44:52.478 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:52 smithi077 ceph-mon[32772]: pgmap v385: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T10:44:52.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:52 smithi164 ceph-mon[32713]: pgmap v385: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T10:44:52.995 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-25T10:44:52.995 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:44:52.995 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 861, 2024-04-25T10:44:52.996 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 863, 2024-04-25T10:44:52.996 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 227960, 2024-04-25T10:44:52.996 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 273862, 2024-04-25T10:44:52.996 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 6, 2024-04-25T10:44:52.996 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 2, 2024-04-25T10:44:52.996 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.085150036, 2024-04-25T10:44:52.996 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.034798123, 2024-04-25T10:44:52.996 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.019483712, 2024-04-25T10:44:52.996 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.021763979, 2024-04-25T10:44:52.996 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:52.997 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 861, 2024-04-25T10:44:52.997 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.035469818, 2024-04-25T10:44:52.997 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000041196 2024-04-25T10:44:52.997 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:52.997 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:52.997 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:52.997 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:52.997 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:52.997 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:52.997 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T10:44:52.997 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T10:44:52.997 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:52.997 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:44:52.997 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 777, 2024-04-25T10:44:52.998 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 954, 2024-04-25T10:44:52.998 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 193048, 2024-04-25T10:44:52.998 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 727967, 2024-04-25T10:44:52.998 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 7, 2024-04-25T10:44:52.998 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 5, 2024-04-25T10:44:52.998 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.085241889, 2024-04-25T10:44:52.998 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.030273067, 2024-04-25T10:44:52.998 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.018564912, 2024-04-25T10:44:52.998 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.027445680, 2024-04-25T10:44:52.998 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:52.999 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 952, 2024-04-25T10:44:52.999 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.048388723, 2024-04-25T10:44:52.999 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000050828 2024-04-25T10:44:52.999 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:52.999 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:52.999 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.000 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.000 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.000 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.000 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T10:44:53.000 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 482784 2024-04-25T10:44:53.000 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.001 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:44:53.001 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 1124, 2024-04-25T10:44:53.001 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 1110, 2024-04-25T10:44:53.001 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 2016608, 2024-04-25T10:44:53.001 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 420246, 2024-04-25T10:44:53.001 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 3, 2024-04-25T10:44:53.001 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 2, 2024-04-25T10:44:53.001 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.168136490, 2024-04-25T10:44:53.001 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.048932167, 2024-04-25T10:44:53.001 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.076132582, 2024-04-25T10:44:53.001 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.044515130, 2024-04-25T10:44:53.002 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:53.002 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1109, 2024-04-25T10:44:53.002 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.066807892, 2024-04-25T10:44:53.002 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000060241 2024-04-25T10:44:53.002 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.002 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:53.002 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.002 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.002 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.002 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.002 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1730000, 2024-04-25T10:44:53.002 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 148992 2024-04-25T10:44:53.003 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.003 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-04-25T10:44:53.003 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2024-04-25T10:44:53.003 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-04-25T10:44:53.003 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.003 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2024-04-25T10:44:53.003 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-04-25T10:44:53.003 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-04-25T10:44:53.003 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 4081, 2024-04-25T10:44:53.003 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.131724728, 2024-04-25T10:44:53.003 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000032277 2024-04-25T10:44:53.003 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:53.003 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.003 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2024-04-25T10:44:53.004 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-04-25T10:44:53.004 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-04-25T10:44:53.004 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.004 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.004 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.004 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:53.004 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.004 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-25T10:44:53.004 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 0, 2024-04-25T10:44:53.004 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 0, 2024-04-25T10:44:53.004 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2024-04-25T10:44:53.004 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.005 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.005 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.005 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.005 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2024-04-25T10:44:53.005 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 0, 2024-04-25T10:44:53.005 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 0, 2024-04-25T10:44:53.005 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2024-04-25T10:44:53.005 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 0, 2024-04-25T10:44:53.005 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 0, 2024-04-25T10:44:53.005 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2024-04-25T10:44:53.005 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 0, 2024-04-25T10:44:53.006 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 0, 2024-04-25T10:44:53.006 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 0, 2024-04-25T10:44:53.006 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 0, 2024-04-25T10:44:53.006 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 0, 2024-04-25T10:44:53.006 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 0, 2024-04-25T10:44:53.006 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 0, 2024-04-25T10:44:53.006 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 0, 2024-04-25T10:44:53.006 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 1, 2024-04-25T10:44:53.006 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 0, 2024-04-25T10:44:53.006 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 0, 2024-04-25T10:44:53.006 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 0, 2024-04-25T10:44:53.006 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 0, 2024-04-25T10:44:53.009 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:44:53.010 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2024-04-25T10:44:53.010 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 0, 2024-04-25T10:44:53.010 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2024-04-25T10:44:53.010 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2024-04-25T10:44:53.010 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 0, 2024-04-25T10:44:53.011 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 0, 2024-04-25T10:44:53.011 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 0, 2024-04-25T10:44:53.011 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 0, 2024-04-25T10:44:53.011 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2024-04-25T10:44:53.011 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 0, 2024-04-25T10:44:53.011 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 0, 2024-04-25T10:44:53.011 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 0, 2024-04-25T10:44:53.011 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 0, 2024-04-25T10:44:53.011 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 0, 2024-04-25T10:44:53.012 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T10:44:53.012 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2024-04-25T10:44:53.012 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:44:53.012 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:44:53.012 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 0, 2024-04-25T10:44:53.012 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T10:44:53.012 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T10:44:53.012 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 0, 2024-04-25T10:44:53.012 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:44:53.012 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T10:44:53.012 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 0, 2024-04-25T10:44:53.012 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:44:53.012 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:44:53.012 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:44:53.013 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 0, 2024-04-25T10:44:53.013 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 0, 2024-04-25T10:44:53.013 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 0, 2024-04-25T10:44:53.013 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 0 2024-04-25T10:44:53.013 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.013 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2024-04-25T10:44:53.013 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 0, 2024-04-25T10:44:53.013 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 0, 2024-04-25T10:44:53.013 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 0, 2024-04-25T10:44:53.013 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 0, 2024-04-25T10:44:53.013 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 0, 2024-04-25T10:44:53.014 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 0, 2024-04-25T10:44:53.014 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2024-04-25T10:44:53.014 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:44:53.014 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 0, 2024-04-25T10:44:53.014 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 0, 2024-04-25T10:44:53.014 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2024-04-25T10:44:53.014 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2024-04-25T10:44:53.014 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2024-04-25T10:44:53.014 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:44:53.014 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:44:53.014 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2024-04-25T10:44:53.014 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2024-04-25T10:44:53.014 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_path": 0, 2024-04-25T10:44:53.015 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_inode": 0, 2024-04-25T10:44:53.015 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T10:44:53.015 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 0, 2024-04-25T10:44:53.015 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2024-04-25T10:44:53.015 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:44:53.015 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2024-04-25T10:44:53.015 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2024-04-25T10:44:53.015 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.015 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2024-04-25T10:44:53.015 INFO:teuthology.orchestra.run.smithi077.stdout: "evlrg": 0, 2024-04-25T10:44:53.015 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 0, 2024-04-25T10:44:53.015 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 0, 2024-04-25T10:44:53.015 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 0, 2024-04-25T10:44:53.015 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 6, 2024-04-25T10:44:53.016 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2024-04-25T10:44:53.016 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 0, 2024-04-25T10:44:53.016 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 0, 2024-04-25T10:44:53.016 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 0, 2024-04-25T10:44:53.016 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 0, 2024-04-25T10:44:53.016 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 2, 2024-04-25T10:44:53.016 INFO:teuthology.orchestra.run.smithi077.stdout: "segmjr": 2, 2024-04-25T10:44:53.016 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2024-04-25T10:44:53.016 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 0, 2024-04-25T10:44:53.016 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 4194304, 2024-04-25T10:44:53.016 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 4196123, 2024-04-25T10:44:53.016 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 4195732, 2024-04-25T10:44:53.016 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2024-04-25T10:44:53.016 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.016 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.017 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.017 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.017 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 6 2024-04-25T10:44:53.017 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.017 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2024-04-25T10:44:53.017 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 3, 2024-04-25T10:44:53.017 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 3, 2024-04-25T10:44:53.017 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 0, 2024-04-25T10:44:53.017 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 1, 2024-04-25T10:44:53.017 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 1, 2024-04-25T10:44:53.017 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 0, 2024-04-25T10:44:53.017 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 0, 2024-04-25T10:44:53.017 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 0, 2024-04-25T10:44:53.017 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 0, 2024-04-25T10:44:53.018 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 0, 2024-04-25T10:44:53.018 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 0, 2024-04-25T10:44:53.018 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 0, 2024-04-25T10:44:53.018 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 121584, 2024-04-25T10:44:53.018 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 215440 2024-04-25T10:44:53.018 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.018 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2024-04-25T10:44:53.018 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 0, 2024-04-25T10:44:53.018 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 0, 2024-04-25T10:44:53.019 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 0, 2024-04-25T10:44:53.019 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 0, 2024-04-25T10:44:53.019 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 0, 2024-04-25T10:44:53.019 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2024-04-25T10:44:53.020 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.020 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.020 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.020 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.020 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2024-04-25T10:44:53.020 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.021 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.021 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.021 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.021 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2024-04-25T10:44:53.021 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.021 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.022 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.022 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.022 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2024-04-25T10:44:53.022 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.022 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.022 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.023 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.023 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2024-04-25T10:44:53.023 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.023 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.023 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.024 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.024 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2024-04-25T10:44:53.024 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.024 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.025 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.025 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.025 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2024-04-25T10:44:53.025 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.025 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.025 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.025 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.025 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2024-04-25T10:44:53.025 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.025 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.025 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.026 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.026 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2024-04-25T10:44:53.027 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.027 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.027 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.027 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.027 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2024-04-25T10:44:53.027 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.027 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.027 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.027 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.027 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2024-04-25T10:44:53.027 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.027 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.027 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.027 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.028 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2024-04-25T10:44:53.028 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.028 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.028 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.028 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.029 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2024-04-25T10:44:53.029 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.029 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.029 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.029 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.029 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2024-04-25T10:44:53.029 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.029 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.029 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.029 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.029 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2024-04-25T10:44:53.029 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.029 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.029 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.030 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.030 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2024-04-25T10:44:53.030 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.030 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.030 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.030 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.030 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2024-04-25T10:44:53.030 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.030 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.030 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.030 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.030 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2024-04-25T10:44:53.030 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.031 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.031 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.031 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.031 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2024-04-25T10:44:53.031 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.031 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.031 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.031 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.031 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2024-04-25T10:44:53.031 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.031 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.031 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.031 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.031 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2024-04-25T10:44:53.032 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.032 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.032 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.032 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.032 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2024-04-25T10:44:53.032 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.032 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.032 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.032 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.032 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2024-04-25T10:44:53.032 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.032 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.032 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.032 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.033 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2024-04-25T10:44:53.033 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.033 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.033 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.033 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.033 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2024-04-25T10:44:53.033 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.033 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.033 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.033 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.033 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2024-04-25T10:44:53.033 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.033 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.034 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.034 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.034 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2024-04-25T10:44:53.034 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.034 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.034 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.034 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.034 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2024-04-25T10:44:53.034 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.034 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.035 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.035 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.035 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2024-04-25T10:44:53.035 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.035 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.035 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.035 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.035 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:44:53.035 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:44:53.035 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2024-04-25T10:44:53.036 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.036 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.036 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.036 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:53.036 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.036 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2024-04-25T10:44:53.036 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 2, 2024-04-25T10:44:53.036 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 3, 2024-04-25T10:44:53.036 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 1, 2024-04-25T10:44:53.036 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 2, 2024-04-25T10:44:53.036 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2024-04-25T10:44:53.036 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 0, 2024-04-25T10:44:53.036 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 0, 2024-04-25T10:44:53.037 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 1942, 2024-04-25T10:44:53.037 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2024-04-25T10:44:53.037 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.037 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-04-25T10:44:53.037 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:44:53.037 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-04-25T10:44:53.037 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:44:53.037 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:44:53.037 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:44:53.037 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:44:53.037 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:44:53.037 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:44:53.037 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:44:53.037 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:44:53.038 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:44:53.038 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:44:53.038 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:44:53.038 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:44:53.038 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:44:53.038 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-04-25T10:44:53.038 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:44:53.038 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-04-25T10:44:53.038 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:44:53.038 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:44:53.038 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:44:53.038 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:44:53.039 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:44:53.039 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:44:53.039 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:44:53.039 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-04-25T10:44:53.039 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:44:53.039 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:44:53.039 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:44:53.039 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-04-25T10:44:53.039 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-04-25T10:44:53.039 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-04-25T10:44:53.039 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:44:53.039 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:44:53.039 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:44:53.039 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:44:53.040 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 189383, 2024-04-25T10:44:53.040 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 50, 2024-04-25T10:44:53.040 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:44:53.040 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-04-25T10:44:53.040 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-04-25T10:44:53.040 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-04-25T10:44:53.040 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:44:53.040 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-04-25T10:44:53.040 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:44:53.040 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-04-25T10:44:53.040 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 8032, 2024-04-25T10:44:53.040 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 167, 2024-04-25T10:44:53.040 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:44:53.040 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:44:53.041 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-04-25T10:44:53.041 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-04-25T10:44:53.041 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 8512, 2024-04-25T10:44:53.041 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 13, 2024-04-25T10:44:53.041 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-04-25T10:44:53.041 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-04-25T10:44:53.041 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-04-25T10:44:53.041 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-04-25T10:44:53.041 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.041 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2024-04-25T10:44:53.041 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2024-04-25T10:44:53.041 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2024-04-25T10:44:53.041 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 2485, 2024-04-25T10:44:53.041 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 21, 2024-04-25T10:44:53.041 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 3, 2024-04-25T10:44:53.042 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 2485, 2024-04-25T10:44:53.042 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2024-04-25T10:44:53.042 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2485, 2024-04-25T10:44:53.042 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1.253975429, 2024-04-25T10:44:53.042 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000504617 2024-04-25T10:44:53.042 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.042 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2024-04-25T10:44:53.042 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2024-04-25T10:44:53.042 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2485, 2024-04-25T10:44:53.042 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 2486 2024-04-25T10:44:53.043 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.043 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 2485, 2024-04-25T10:44:53.043 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 2485, 2024-04-25T10:44:53.043 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 0, 2024-04-25T10:44:53.043 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2024-04-25T10:44:53.043 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2024-04-25T10:44:53.043 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 1622, 2024-04-25T10:44:53.043 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 0, 2024-04-25T10:44:53.043 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 861, 2024-04-25T10:44:53.043 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 0, 2024-04-25T10:44:53.043 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 0, 2024-04-25T10:44:53.043 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2024-04-25T10:44:53.043 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2024-04-25T10:44:53.043 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2024-04-25T10:44:53.044 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 0, 2024-04-25T10:44:53.044 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 0, 2024-04-25T10:44:53.044 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2024-04-25T10:44:53.044 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2024-04-25T10:44:53.044 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2024-04-25T10:44:53.044 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 1, 2024-04-25T10:44:53.044 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 0, 2024-04-25T10:44:53.044 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:44:53.044 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2024-04-25T10:44:53.044 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:44:53.044 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2024-04-25T10:44:53.044 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2024-04-25T10:44:53.044 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2024-04-25T10:44:53.044 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:44:53.045 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2024-04-25T10:44:53.045 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:44:53.045 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 0, 2024-04-25T10:44:53.045 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2024-04-25T10:44:53.045 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2024-04-25T10:44:53.045 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2024-04-25T10:44:53.045 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2024-04-25T10:44:53.045 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2024-04-25T10:44:53.045 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2024-04-25T10:44:53.045 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2024-04-25T10:44:53.045 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2024-04-25T10:44:53.045 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2024-04-25T10:44:53.045 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2024-04-25T10:44:53.045 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2024-04-25T10:44:53.045 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2024-04-25T10:44:53.046 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2024-04-25T10:44:53.046 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2024-04-25T10:44:53.046 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2024-04-25T10:44:53.046 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2024-04-25T10:44:53.046 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 93, 2024-04-25T10:44:53.046 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2024-04-25T10:44:53.046 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 16, 2024-04-25T10:44:53.046 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 5, 2024-04-25T10:44:53.046 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 5, 2024-04-25T10:44:53.046 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 0, 2024-04-25T10:44:53.046 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2024-04-25T10:44:53.046 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 0, 2024-04-25T10:44:53.046 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 2, 2024-04-25T10:44:53.047 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 0 2024-04-25T10:44:53.047 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.047 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2024-04-25T10:44:53.047 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 0, 2024-04-25T10:44:53.047 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:44:53.047 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2024-04-25T10:44:53.047 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2024-04-25T10:44:53.047 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.047 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2024-04-25T10:44:53.047 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 0 2024-04-25T10:44:53.047 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.047 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2024-04-25T10:44:53.047 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2024-04-25T10:44:53.047 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T10:44:53.048 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2024-04-25T10:44:53.048 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 0, 2024-04-25T10:44:53.048 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed_ops": 0, 2024-04-25T10:44:53.048 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 0, 2024-04-25T10:44:53.048 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2024-04-25T10:44:53.048 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.048 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:44:53.048 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-25T10:44:53.048 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-04-25T10:44:53.048 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-25T10:44:53.048 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 2758, 2024-04-25T10:44:53.048 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 2172463, 2024-04-25T10:44:53.048 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-25T10:44:53.048 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 2758, 2024-04-25T10:44:53.049 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-04-25T10:44:53.049 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-04-25T10:44:53.049 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 2758, 2024-04-25T10:44:53.049 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 2172463, 2024-04-25T10:44:53.049 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-25T10:44:53.049 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.049 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.049 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.049 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:53.049 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.049 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2024-04-25T10:44:53.049 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-25T10:44:53.049 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-04-25T10:44:53.049 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-25T10:44:53.049 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-04-25T10:44:53.050 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-04-25T10:44:53.050 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-25T10:44:53.050 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-04-25T10:44:53.050 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 2485, 2024-04-25T10:44:53.050 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 1824, 2024-04-25T10:44:53.050 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 2485, 2024-04-25T10:44:53.050 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1824, 2024-04-25T10:44:53.050 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-25T10:44:53.050 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.050 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.050 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.050 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:53.050 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2024-04-25T10:44:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-25T10:44:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2024-04-25T10:44:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-25T10:44:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-04-25T10:44:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-04-25T10:44:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-25T10:44:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-04-25T10:44:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 2485, 2024-04-25T10:44:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 2485, 2024-04-25T10:44:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 2485, 2024-04-25T10:44:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 2485, 2024-04-25T10:44:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-25T10:44:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.052 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.052 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:53.052 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.052 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:44:53.052 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-25T10:44:53.052 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-04-25T10:44:53.052 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-25T10:44:53.052 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-04-25T10:44:53.052 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-04-25T10:44:53.052 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-25T10:44:53.052 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-04-25T10:44:53.052 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-04-25T10:44:53.052 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-04-25T10:44:53.053 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-04-25T10:44:53.053 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-04-25T10:44:53.053 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-25T10:44:53.053 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.053 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.053 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.053 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:53.053 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:53.053 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x563b75970c30": { 2024-04-25T10:44:53.053 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-25T10:44:53.053 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-04-25T10:44:53.053 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-25T10:44:53.053 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-04-25T10:44:53.053 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-04-25T10:44:53.053 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-25T10:44:53.053 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-04-25T10:44:53.054 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-04-25T10:44:53.054 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-04-25T10:44:53.054 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-04-25T10:44:53.054 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-04-25T10:44:53.054 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-25T10:44:53.054 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:53.054 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:53.054 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:53.054 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:53.054 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:53.054 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-25T10:44:53.744 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-25T10:44:53.744 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-25T10:44:53.744 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-25T10:44:53.744 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-25T10:44:53.744 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-25T10:44:53.744 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-25T10:44:53.744 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T10:44:53.745 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T10:44:53.745 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T10:44:53.987 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T10:44:54.330 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:54 smithi077 ceph-mon[32772]: pgmap v386: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T10:44:54.330 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:54 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:44:54.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:54 smithi043 ceph-mon[29478]: pgmap v386: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T10:44:54.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:54 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:44:54.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:54 smithi164 ceph-mon[32713]: pgmap v386: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T10:44:54.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:54 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T10:44:55.687 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:55 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:44:55.687 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:55 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:44:55.687 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:55 smithi077 ceph-mon[32772]: pgmap v387: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T10:44:55.687 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:55 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:44:55.687 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:55 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:44:55.688 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-25T10:44:55.688 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:44:55.688 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 623, 2024-04-25T10:44:55.688 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 625, 2024-04-25T10:44:55.688 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 6578552, 2024-04-25T10:44:55.688 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 198121, 2024-04-25T10:44:55.688 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 7, 2024-04-25T10:44:55.688 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 3, 2024-04-25T10:44:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.064211945, 2024-04-25T10:44:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.015795673, 2024-04-25T10:44:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.022932617, 2024-04-25T10:44:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.019772150, 2024-04-25T10:44:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 623, 2024-04-25T10:44:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.019258484, 2024-04-25T10:44:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000030912 2024-04-25T10:44:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T10:44:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T10:44:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:44:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 1633, 2024-04-25T10:44:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 1762, 2024-04-25T10:44:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 12947096, 2024-04-25T10:44:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 837701, 2024-04-25T10:44:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 5, 2024-04-25T10:44:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 4, 2024-04-25T10:44:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.167305947, 2024-04-25T10:44:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.060419493, 2024-04-25T10:44:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.047459588, 2024-04-25T10:44:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.045309230, 2024-04-25T10:44:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1761, 2024-04-25T10:44:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.064382358, 2024-04-25T10:44:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000036560 2024-04-25T10:44:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-04-25T10:44:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 319936 2024-04-25T10:44:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:44:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 476, 2024-04-25T10:44:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 475, 2024-04-25T10:44:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 4487650, 2024-04-25T10:44:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 199555, 2024-04-25T10:44:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 4, 2024-04-25T10:44:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 3, 2024-04-25T10:44:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.059413696, 2024-04-25T10:44:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.019064455, 2024-04-25T10:44:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.023250378, 2024-04-25T10:44:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.013205671, 2024-04-25T10:44:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 474, 2024-04-25T10:44:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.019922658, 2024-04-25T10:44:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000042030 2024-04-25T10:44:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 221584, 2024-04-25T10:44:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 109552 2024-04-25T10:44:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-04-25T10:44:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2024-04-25T10:44:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-04-25T10:44:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2024-04-25T10:44:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-04-25T10:44:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-04-25T10:44:55.694 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 4349, 2024-04-25T10:44:55.694 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.105968583, 2024-04-25T10:44:55.694 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000024366 2024-04-25T10:44:55.694 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:55.694 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.694 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2024-04-25T10:44:55.694 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-04-25T10:44:55.694 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-04-25T10:44:55.694 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.694 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.694 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.694 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:55.694 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.694 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-25T10:44:55.695 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 0, 2024-04-25T10:44:55.695 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 0, 2024-04-25T10:44:55.695 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2024-04-25T10:44:55.695 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.695 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.695 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.695 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.695 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2024-04-25T10:44:55.695 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 0, 2024-04-25T10:44:55.695 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 0, 2024-04-25T10:44:55.695 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2024-04-25T10:44:55.695 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 0, 2024-04-25T10:44:55.695 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 0, 2024-04-25T10:44:55.695 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2024-04-25T10:44:55.695 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 23, 2024-04-25T10:44:55.696 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 7, 2024-04-25T10:44:55.696 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 16, 2024-04-25T10:44:55.696 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 0, 2024-04-25T10:44:55.696 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 13, 2024-04-25T10:44:55.696 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 0, 2024-04-25T10:44:55.696 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 0, 2024-04-25T10:44:55.696 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 0, 2024-04-25T10:44:55.696 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 2, 2024-04-25T10:44:55.696 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 0, 2024-04-25T10:44:55.696 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 0, 2024-04-25T10:44:55.696 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 0, 2024-04-25T10:44:55.696 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 0, 2024-04-25T10:44:55.696 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:44:55.696 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2024-04-25T10:44:55.696 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 0, 2024-04-25T10:44:55.697 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2024-04-25T10:44:55.697 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2024-04-25T10:44:55.697 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 0, 2024-04-25T10:44:55.697 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 0, 2024-04-25T10:44:55.697 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 0, 2024-04-25T10:44:55.697 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 0, 2024-04-25T10:44:55.697 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2024-04-25T10:44:55.697 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 0, 2024-04-25T10:44:55.697 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 0, 2024-04-25T10:44:55.697 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 4, 2024-04-25T10:44:55.697 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 234, 2024-04-25T10:44:55.697 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 6, 2024-04-25T10:44:55.697 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T10:44:55.697 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2024-04-25T10:44:55.698 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:44:55.698 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:44:55.698 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 0, 2024-04-25T10:44:55.698 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T10:44:55.698 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T10:44:55.698 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 0, 2024-04-25T10:44:55.698 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:44:55.698 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T10:44:55.698 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 0, 2024-04-25T10:44:55.698 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:44:55.698 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:44:55.698 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:44:55.698 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 0, 2024-04-25T10:44:55.698 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 0, 2024-04-25T10:44:55.698 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 0, 2024-04-25T10:44:55.699 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 0 2024-04-25T10:44:55.699 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.699 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2024-04-25T10:44:55.699 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 0, 2024-04-25T10:44:55.699 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 0, 2024-04-25T10:44:55.699 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 0, 2024-04-25T10:44:55.699 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 0, 2024-04-25T10:44:55.699 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 0, 2024-04-25T10:44:55.699 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 0, 2024-04-25T10:44:55.699 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2024-04-25T10:44:55.699 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:44:55.699 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 0, 2024-04-25T10:44:55.699 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 0, 2024-04-25T10:44:55.699 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2024-04-25T10:44:55.700 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2024-04-25T10:44:55.700 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2024-04-25T10:44:55.700 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:44:55.700 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:44:55.700 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2024-04-25T10:44:55.700 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2024-04-25T10:44:55.700 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_path": 0, 2024-04-25T10:44:55.700 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_inode": 0, 2024-04-25T10:44:55.700 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T10:44:55.700 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 0, 2024-04-25T10:44:55.700 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2024-04-25T10:44:55.700 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:44:55.700 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2024-04-25T10:44:55.700 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2024-04-25T10:44:55.700 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.701 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2024-04-25T10:44:55.701 INFO:teuthology.orchestra.run.smithi077.stdout: "evlrg": 0, 2024-04-25T10:44:55.701 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 0, 2024-04-25T10:44:55.701 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 0, 2024-04-25T10:44:55.701 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 0, 2024-04-25T10:44:55.701 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 2537, 2024-04-25T10:44:55.701 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2024-04-25T10:44:55.701 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 0, 2024-04-25T10:44:55.701 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 0, 2024-04-25T10:44:55.701 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 0, 2024-04-25T10:44:55.701 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 0, 2024-04-25T10:44:55.701 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 2, 2024-04-25T10:44:55.701 INFO:teuthology.orchestra.run.smithi077.stdout: "segmjr": 47, 2024-04-25T10:44:55.701 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2024-04-25T10:44:55.701 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 0, 2024-04-25T10:44:55.702 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 26973433, 2024-04-25T10:44:55.702 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 27393246, 2024-04-25T10:44:55.702 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 27381725, 2024-04-25T10:44:55.702 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2024-04-25T10:44:55.702 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.702 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.702 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.702 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.702 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 2537 2024-04-25T10:44:55.702 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.702 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2024-04-25T10:44:55.702 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 20, 2024-04-25T10:44:55.702 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 21, 2024-04-25T10:44:55.702 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 1, 2024-04-25T10:44:55.703 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 9, 2024-04-25T10:44:55.703 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 9, 2024-04-25T10:44:55.703 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 0, 2024-04-25T10:44:55.703 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 23, 2024-04-25T10:44:55.703 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 23, 2024-04-25T10:44:55.703 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 0, 2024-04-25T10:44:55.703 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 0, 2024-04-25T10:44:55.703 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 0, 2024-04-25T10:44:55.703 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 0, 2024-04-25T10:44:55.703 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 121548, 2024-04-25T10:44:55.703 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 215440 2024-04-25T10:44:55.703 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.703 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2024-04-25T10:44:55.703 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 0, 2024-04-25T10:44:55.703 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 0, 2024-04-25T10:44:55.704 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 0, 2024-04-25T10:44:55.704 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 0, 2024-04-25T10:44:55.704 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 0, 2024-04-25T10:44:55.704 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2024-04-25T10:44:55.704 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.704 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.704 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.704 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.704 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2024-04-25T10:44:55.704 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.704 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.704 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.704 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.704 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2024-04-25T10:44:55.704 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.705 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.705 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.705 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.705 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2024-04-25T10:44:55.705 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.705 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.705 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.705 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.705 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2024-04-25T10:44:55.705 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.705 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.705 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.705 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.705 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2024-04-25T10:44:55.705 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.706 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.706 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.706 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.706 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2024-04-25T10:44:55.706 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.706 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.706 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.706 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.706 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2024-04-25T10:44:55.706 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.706 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.706 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.706 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.706 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2024-04-25T10:44:55.707 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.707 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.707 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.707 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.707 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2024-04-25T10:44:55.707 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.707 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.707 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.707 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.707 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2024-04-25T10:44:55.707 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.707 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.707 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.707 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.707 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2024-04-25T10:44:55.708 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.708 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.708 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.708 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.708 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2024-04-25T10:44:55.708 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.709 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.709 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.709 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.709 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2024-04-25T10:44:55.709 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.709 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.709 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.709 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.709 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2024-04-25T10:44:55.709 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.709 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.709 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.709 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.709 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2024-04-25T10:44:55.710 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.710 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.710 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.710 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.710 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2024-04-25T10:44:55.710 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.710 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.710 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.710 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.710 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2024-04-25T10:44:55.710 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.710 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.710 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.710 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.711 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2024-04-25T10:44:55.711 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.711 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.711 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.711 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.711 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2024-04-25T10:44:55.711 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.711 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.711 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.711 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.711 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2024-04-25T10:44:55.711 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.711 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.711 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.711 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.712 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2024-04-25T10:44:55.712 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.712 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.712 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.712 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.712 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2024-04-25T10:44:55.712 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.712 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.712 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.712 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.712 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2024-04-25T10:44:55.712 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.712 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.712 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.712 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.713 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2024-04-25T10:44:55.713 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.713 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.713 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.713 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.713 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2024-04-25T10:44:55.713 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.713 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.713 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.713 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.713 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2024-04-25T10:44:55.713 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.713 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.713 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.714 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.714 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2024-04-25T10:44:55.714 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.714 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.714 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.714 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.714 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2024-04-25T10:44:55.714 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.714 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.714 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.714 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.714 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:44:55.714 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:44:55.714 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2024-04-25T10:44:55.715 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.715 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.715 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.715 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:55.715 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.715 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2024-04-25T10:44:55.715 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 2, 2024-04-25T10:44:55.715 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 5, 2024-04-25T10:44:55.715 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 3, 2024-04-25T10:44:55.715 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 0, 2024-04-25T10:44:55.715 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2024-04-25T10:44:55.715 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 0, 2024-04-25T10:44:55.715 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 0, 2024-04-25T10:44:55.715 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 3910, 2024-04-25T10:44:55.715 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2024-04-25T10:44:55.716 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.716 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-04-25T10:44:55.716 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:44:55.716 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-04-25T10:44:55.716 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:44:55.716 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:44:55.716 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:44:55.716 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:44:55.716 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:44:55.716 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:44:55.716 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:44:55.716 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:44:55.716 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:44:55.716 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:44:55.717 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:44:55.717 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:44:55.717 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:44:55.717 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-04-25T10:44:55.717 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:44:55.717 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-04-25T10:44:55.717 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:44:55.717 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:44:55.717 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:44:55.717 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:44:55.717 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:44:55.717 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:44:55.717 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:44:55.717 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-04-25T10:44:55.717 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:44:55.718 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:44:55.718 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:44:55.718 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-04-25T10:44:55.718 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-04-25T10:44:55.718 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-04-25T10:44:55.718 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:44:55.718 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:44:55.718 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:44:55.718 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:44:55.719 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 207193, 2024-04-25T10:44:55.719 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 84, 2024-04-25T10:44:55.719 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:44:55.719 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-04-25T10:44:55.719 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-04-25T10:44:55.719 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-04-25T10:44:55.719 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:44:55.719 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-04-25T10:44:55.719 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:44:55.719 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-04-25T10:44:55.719 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 8032, 2024-04-25T10:44:55.719 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 167, 2024-04-25T10:44:55.719 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:44:55.719 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:44:55.719 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-04-25T10:44:55.720 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-04-25T10:44:55.720 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 109175, 2024-04-25T10:44:55.720 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 885, 2024-04-25T10:44:55.720 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-04-25T10:44:55.720 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-04-25T10:44:55.720 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-04-25T10:44:55.720 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-04-25T10:44:55.720 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.720 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2024-04-25T10:44:55.720 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2024-04-25T10:44:55.720 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2024-04-25T10:44:55.720 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 2534, 2024-04-25T10:44:55.720 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 26, 2024-04-25T10:44:55.720 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 4, 2024-04-25T10:44:55.720 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 2534, 2024-04-25T10:44:55.721 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2024-04-25T10:44:55.721 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2534, 2024-04-25T10:44:55.721 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1.301974445, 2024-04-25T10:44:55.721 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000513802 2024-04-25T10:44:55.721 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.721 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2024-04-25T10:44:55.721 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2024-04-25T10:44:55.721 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2534, 2024-04-25T10:44:55.721 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 2535 2024-04-25T10:44:55.721 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.721 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 2534, 2024-04-25T10:44:55.721 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 2534, 2024-04-25T10:44:55.721 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 0, 2024-04-25T10:44:55.721 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2024-04-25T10:44:55.722 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2024-04-25T10:44:55.722 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 1278, 2024-04-25T10:44:55.722 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 0, 2024-04-25T10:44:55.722 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 1253, 2024-04-25T10:44:55.722 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 0, 2024-04-25T10:44:55.722 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 0, 2024-04-25T10:44:55.722 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2024-04-25T10:44:55.722 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2024-04-25T10:44:55.722 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2024-04-25T10:44:55.722 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 0, 2024-04-25T10:44:55.722 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 0, 2024-04-25T10:44:55.722 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2024-04-25T10:44:55.722 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2024-04-25T10:44:55.722 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2024-04-25T10:44:55.722 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 2, 2024-04-25T10:44:55.723 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 0, 2024-04-25T10:44:55.723 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:44:55.723 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2024-04-25T10:44:55.723 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:44:55.723 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2024-04-25T10:44:55.723 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2024-04-25T10:44:55.723 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2024-04-25T10:44:55.723 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:44:55.723 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2024-04-25T10:44:55.723 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:44:55.723 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 0, 2024-04-25T10:44:55.723 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2024-04-25T10:44:55.723 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2024-04-25T10:44:55.723 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2024-04-25T10:44:55.723 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2024-04-25T10:44:55.724 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2024-04-25T10:44:55.724 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2024-04-25T10:44:55.724 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2024-04-25T10:44:55.724 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2024-04-25T10:44:55.724 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2024-04-25T10:44:55.724 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2024-04-25T10:44:55.724 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2024-04-25T10:44:55.724 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2024-04-25T10:44:55.724 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2024-04-25T10:44:55.724 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2024-04-25T10:44:55.724 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2024-04-25T10:44:55.724 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2024-04-25T10:44:55.724 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 93, 2024-04-25T10:44:55.724 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2024-04-25T10:44:55.724 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 12, 2024-04-25T10:44:55.725 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 7, 2024-04-25T10:44:55.725 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 7, 2024-04-25T10:44:55.725 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 0, 2024-04-25T10:44:55.725 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2024-04-25T10:44:55.725 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 0, 2024-04-25T10:44:55.725 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 2, 2024-04-25T10:44:55.725 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 0 2024-04-25T10:44:55.725 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.725 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2024-04-25T10:44:55.725 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 0, 2024-04-25T10:44:55.725 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:44:55.725 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2024-04-25T10:44:55.725 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2024-04-25T10:44:55.725 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.726 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2024-04-25T10:44:55.726 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 0 2024-04-25T10:44:55.726 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.726 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2024-04-25T10:44:55.726 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2024-04-25T10:44:55.726 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T10:44:55.726 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2024-04-25T10:44:55.726 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 0, 2024-04-25T10:44:55.726 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed_ops": 0, 2024-04-25T10:44:55.726 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 0, 2024-04-25T10:44:55.726 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2024-04-25T10:44:55.726 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.726 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:44:55.726 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-25T10:44:55.726 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-04-25T10:44:55.727 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-25T10:44:55.727 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 2728, 2024-04-25T10:44:55.727 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 23754837, 2024-04-25T10:44:55.727 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-25T10:44:55.727 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 2728, 2024-04-25T10:44:55.727 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-04-25T10:44:55.727 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-04-25T10:44:55.727 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 2728, 2024-04-25T10:44:55.727 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 23754837, 2024-04-25T10:44:55.727 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-25T10:44:55.727 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.727 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.728 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.728 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:55.728 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.728 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2024-04-25T10:44:55.728 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-25T10:44:55.728 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-04-25T10:44:55.728 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-25T10:44:55.728 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-04-25T10:44:55.728 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-04-25T10:44:55.728 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-25T10:44:55.728 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-04-25T10:44:55.728 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 2534, 2024-04-25T10:44:55.728 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 23093199, 2024-04-25T10:44:55.728 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 2534, 2024-04-25T10:44:55.728 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 23093199, 2024-04-25T10:44:55.729 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-25T10:44:55.729 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.729 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.729 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.729 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:55.729 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.729 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2024-04-25T10:44:55.729 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-25T10:44:55.729 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2024-04-25T10:44:55.729 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-25T10:44:55.729 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-04-25T10:44:55.729 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-04-25T10:44:55.729 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-25T10:44:55.729 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-04-25T10:44:55.730 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 2534, 2024-04-25T10:44:55.730 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 2534, 2024-04-25T10:44:55.730 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 2534, 2024-04-25T10:44:55.730 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 2534, 2024-04-25T10:44:55.730 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-25T10:44:55.730 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.730 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.730 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.730 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:55.730 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.730 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:44:55.730 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-25T10:44:55.730 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-04-25T10:44:55.730 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-25T10:44:55.731 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-04-25T10:44:55.731 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-04-25T10:44:55.731 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-25T10:44:55.731 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-04-25T10:44:55.731 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-04-25T10:44:55.731 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-04-25T10:44:55.731 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-04-25T10:44:55.731 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-04-25T10:44:55.731 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-25T10:44:55.731 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.731 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.731 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.731 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:55.732 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:55.732 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x55982439ec30": { 2024-04-25T10:44:55.732 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-25T10:44:55.732 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-04-25T10:44:55.732 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-25T10:44:55.732 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-04-25T10:44:55.732 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-04-25T10:44:55.732 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-25T10:44:55.732 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-04-25T10:44:55.732 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-04-25T10:44:55.732 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-04-25T10:44:55.732 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-04-25T10:44:55.732 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-04-25T10:44:55.732 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-25T10:44:55.733 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:55.733 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:55.733 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:55.733 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:55.733 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:55.733 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-25T10:44:55.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:55 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:44:55.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:55 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:44:55.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:55 smithi164 ceph-mon[32713]: pgmap v387: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T10:44:55.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:55 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:44:55.792 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:55 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:44:55.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:55 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:44:55.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:55 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:44:55.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:55 smithi043 ceph-mon[29478]: pgmap v387: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T10:44:55.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:55 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:44:55.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:55 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:44:56.506 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T10:44:56.507 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-25T10:44:56.507 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-25T10:44:56.507 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-04-25T10:44:56.507 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-25T10:44:56.507 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-25T10:44:56.507 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T10:44:56.507 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T10:44:56.507 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T10:44:56.743 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T10:44:56.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:56 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:44:56.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:56 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:44:56.791 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:56 smithi077 ceph-mon[32772]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:44:56.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:56 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:44:56.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:56 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:44:56.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:56 smithi164 ceph-mon[32713]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:44:56.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:56 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T10:44:56.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:56 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T10:44:56.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:56 smithi043 ceph-mon[29478]: from='mgr.15291 ' entity='mgr.x' 2024-04-25T10:44:57.790 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:57 smithi077 ceph-mon[32772]: pgmap v388: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 5 op/s 2024-04-25T10:44:57.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:57 smithi164 ceph-mon[32713]: pgmap v388: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 5 op/s 2024-04-25T10:44:57.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:57 smithi043 ceph-mon[29478]: pgmap v388: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 5 op/s 2024-04-25T10:44:58.461 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-25T10:44:58.461 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:44:58.461 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 0, 2024-04-25T10:44:58.461 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 2, 2024-04-25T10:44:58.461 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 0, 2024-04-25T10:44:58.461 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 0, 2024-04-25T10:44:58.462 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 4, 2024-04-25T10:44:58.462 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 0, 2024-04-25T10:44:58.462 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.001155302, 2024-04-25T10:44:58.462 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T10:44:58.462 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.000372064, 2024-04-25T10:44:58.462 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T10:44:58.462 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:58.462 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:58.462 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:58.462 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:58.462 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:58.462 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:58.462 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:58.463 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:58.463 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:58.463 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:58.463 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T10:44:58.463 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T10:44:58.463 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:58.463 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:44:58.463 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 251, 2024-04-25T10:44:58.463 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 240, 2024-04-25T10:44:58.463 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 1695488, 2024-04-25T10:44:58.463 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 147904, 2024-04-25T10:44:58.463 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 2, 2024-04-25T10:44:58.463 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 1, 2024-04-25T10:44:58.464 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.054773160, 2024-04-25T10:44:58.464 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.016445153, 2024-04-25T10:44:58.464 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.042628326, 2024-04-25T10:44:58.464 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.006857917, 2024-04-25T10:44:58.464 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:58.464 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 239, 2024-04-25T10:44:58.464 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.012067866, 2024-04-25T10:44:58.464 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000050493 2024-04-25T10:44:58.464 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:58.464 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:58.464 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:58.464 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:58.464 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:58.464 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:58.465 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1695488, 2024-04-25T10:44:58.465 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 147904 2024-04-25T10:44:58.465 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:58.465 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:44:58.465 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 6, 2024-04-25T10:44:58.465 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 185, 2024-04-25T10:44:58.465 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 3360, 2024-04-25T10:44:58.465 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 189856, 2024-04-25T10:44:58.465 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 5, 2024-04-25T10:44:58.465 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 3, 2024-04-25T10:44:58.465 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.013501797, 2024-04-25T10:44:58.466 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.008952112, 2024-04-25T10:44:58.466 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.002159823, 2024-04-25T10:44:58.466 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T10:44:58.466 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:44:58.466 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 183, 2024-04-25T10:44:58.466 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.023244383, 2024-04-25T10:44:58.466 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000127018 2024-04-25T10:44:58.466 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:58.466 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:44:58.466 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:58.466 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:58.466 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:58.466 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:58.466 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T10:44:58.467 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 189856 2024-04-25T10:44:58.467 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:58.467 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-04-25T10:44:58.467 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 0, 2024-04-25T10:44:58.467 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-04-25T10:44:58.467 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:58.467 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-04-25T10:44:58.467 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:44:58.467 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-04-25T10:44:58.467 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:44:58.467 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:44:58.467 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:44:58.467 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:44:58.467 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:44:58.467 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:44:58.468 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:44:58.468 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:44:58.468 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:44:58.468 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:44:58.468 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:44:58.468 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:44:58.468 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:44:58.468 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-04-25T10:44:58.468 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:44:58.468 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-04-25T10:44:58.468 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:44:58.468 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:44:58.468 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:44:58.468 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:44:58.469 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:44:58.469 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:44:58.469 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:44:58.469 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-04-25T10:44:58.469 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:44:58.469 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:44:58.469 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:44:58.469 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-04-25T10:44:58.469 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-04-25T10:44:58.469 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-04-25T10:44:58.469 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:44:58.469 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:44:58.469 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:44:58.469 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:44:58.469 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 148972, 2024-04-25T10:44:58.469 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 31, 2024-04-25T10:44:58.470 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:44:58.470 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-04-25T10:44:58.470 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-04-25T10:44:58.470 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-04-25T10:44:58.470 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:44:58.470 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-04-25T10:44:58.470 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:44:58.470 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-04-25T10:44:58.470 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 0, 2024-04-25T10:44:58.470 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 0, 2024-04-25T10:44:58.470 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:44:58.470 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:44:58.470 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-04-25T10:44:58.470 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-04-25T10:44:58.471 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 928, 2024-04-25T10:44:58.471 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 2, 2024-04-25T10:44:58.471 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-04-25T10:44:58.471 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-04-25T10:44:58.471 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-04-25T10:44:58.471 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-04-25T10:44:58.471 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-25T10:44:58.471 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:44:58.471 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-25T10:44:58.471 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-04-25T10:44:58.471 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-25T10:44:58.471 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 253, 2024-04-25T10:44:58.471 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 1660051, 2024-04-25T10:44:58.471 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-25T10:44:58.471 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 253, 2024-04-25T10:44:58.472 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-04-25T10:44:58.472 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-04-25T10:44:58.472 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 253, 2024-04-25T10:44:58.472 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1660051, 2024-04-25T10:44:58.472 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-25T10:44:58.472 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-25T10:44:58.472 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-25T10:44:58.472 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-25T10:44:58.472 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:58.472 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-25T10:44:58.472 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-25T10:44:59.132 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-25T10:44:59.132 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-25T10:44:59.132 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-04-25T10:44:59.132 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-04-25T10:44:59.132 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-25T10:44:59.132 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-25T10:44:59.132 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-25T10:44:59.132 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T10:44:59.132 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T10:44:59.382 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:44:59.768 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:44:59 smithi164 ceph-mon[32713]: pgmap v389: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7 op/s 2024-04-25T10:45:00.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:44:59 smithi077 ceph-mon[32772]: pgmap v389: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7 op/s 2024-04-25T10:45:00.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:44:59 smithi043 ceph-mon[29478]: pgmap v389: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7 op/s 2024-04-25T10:45:00.997 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-25T10:45:00.998 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:45:00.998 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 2479, 2024-04-25T10:45:00.998 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 2481, 2024-04-25T10:45:00.998 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 704180, 2024-04-25T10:45:00.998 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 788329, 2024-04-25T10:45:00.998 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 6, 2024-04-25T10:45:00.998 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 2, 2024-04-25T10:45:00.998 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.240986588, 2024-04-25T10:45:00.998 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.080756204, 2024-04-25T10:45:00.998 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.059823189, 2024-04-25T10:45:00.998 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.082104627, 2024-04-25T10:45:00.999 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:45:00.999 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2479, 2024-04-25T10:45:00.999 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.103962238, 2024-04-25T10:45:00.999 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000041937 2024-04-25T10:45:00.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:00.999 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:45:00.999 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:00.999 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:00.999 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:00.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:00.999 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T10:45:00.999 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T10:45:00.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:00.999 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:45:01.000 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 281, 2024-04-25T10:45:01.000 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 262, 2024-04-25T10:45:01.000 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 1744540, 2024-04-25T10:45:01.000 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 150778, 2024-04-25T10:45:01.000 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 3, 2024-04-25T10:45:01.000 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 2, 2024-04-25T10:45:01.000 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.056811536, 2024-04-25T10:45:01.000 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.017368825, 2024-04-25T10:45:01.000 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.044844970, 2024-04-25T10:45:01.000 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.006552187, 2024-04-25T10:45:01.000 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:45:01.000 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 262, 2024-04-25T10:45:01.000 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.015789530, 2024-04-25T10:45:01.000 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000060265 2024-04-25T10:45:01.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.001 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:45:01.001 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.001 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.001 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.001 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 1743984, 2024-04-25T10:45:01.001 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 150080 2024-04-25T10:45:01.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.001 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:45:01.001 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 3, 2024-04-25T10:45:01.001 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 183, 2024-04-25T10:45:01.001 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 588, 2024-04-25T10:45:01.001 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 486062, 2024-04-25T10:45:01.002 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 6, 2024-04-25T10:45:01.002 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 4, 2024-04-25T10:45:01.002 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.011590192, 2024-04-25T10:45:01.002 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.007278941, 2024-04-25T10:45:01.002 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.002325024, 2024-04-25T10:45:01.002 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000021343, 2024-04-25T10:45:01.002 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:45:01.002 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 180, 2024-04-25T10:45:01.003 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.020338730, 2024-04-25T10:45:01.003 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000112992 2024-04-25T10:45:01.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.003 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:45:01.003 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.003 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.003 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.003 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T10:45:01.003 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 485744 2024-04-25T10:45:01.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.003 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-04-25T10:45:01.003 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-04-25T10:45:01.003 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-04-25T10:45:01.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.004 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-04-25T10:45:01.004 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-25T10:45:01.004 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-25T10:45:01.004 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 4209, 2024-04-25T10:45:01.004 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.132501894, 2024-04-25T10:45:01.004 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000031480 2024-04-25T10:45:01.004 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:01.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.004 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-04-25T10:45:01.004 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-25T10:45:01.004 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-25T10:45:01.004 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.005 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.005 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.005 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:01.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.005 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-04-25T10:45:01.005 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 0, 2024-04-25T10:45:01.005 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 0, 2024-04-25T10:45:01.005 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-04-25T10:45:01.005 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.005 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.005 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.005 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-04-25T10:45:01.006 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 0, 2024-04-25T10:45:01.006 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-04-25T10:45:01.006 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-04-25T10:45:01.006 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 0, 2024-04-25T10:45:01.006 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2024-04-25T10:45:01.006 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2024-04-25T10:45:01.006 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 0, 2024-04-25T10:45:01.006 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 0, 2024-04-25T10:45:01.006 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 0, 2024-04-25T10:45:01.006 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 0, 2024-04-25T10:45:01.006 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 0, 2024-04-25T10:45:01.006 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 0, 2024-04-25T10:45:01.006 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 0, 2024-04-25T10:45:01.006 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 0, 2024-04-25T10:45:01.007 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 1, 2024-04-25T10:45:01.007 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 0, 2024-04-25T10:45:01.007 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 0, 2024-04-25T10:45:01.007 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 0, 2024-04-25T10:45:01.007 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 0, 2024-04-25T10:45:01.007 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:45:01.007 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-04-25T10:45:01.007 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2024-04-25T10:45:01.007 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-04-25T10:45:01.007 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-04-25T10:45:01.007 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2024-04-25T10:45:01.007 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2024-04-25T10:45:01.007 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2024-04-25T10:45:01.007 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2024-04-25T10:45:01.007 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-04-25T10:45:01.008 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-04-25T10:45:01.008 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-04-25T10:45:01.008 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 0, 2024-04-25T10:45:01.008 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 0, 2024-04-25T10:45:01.008 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 0, 2024-04-25T10:45:01.008 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T10:45:01.008 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-04-25T10:45:01.008 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:45:01.008 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:45:01.008 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-04-25T10:45:01.008 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T10:45:01.008 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T10:45:01.008 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-04-25T10:45:01.008 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:45:01.009 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T10:45:01.009 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 0, 2024-04-25T10:45:01.009 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:45:01.009 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:45:01.009 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:45:01.009 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 0, 2024-04-25T10:45:01.009 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 0, 2024-04-25T10:45:01.009 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 0, 2024-04-25T10:45:01.009 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 0 2024-04-25T10:45:01.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.009 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-04-25T10:45:01.009 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 0, 2024-04-25T10:45:01.009 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 0, 2024-04-25T10:45:01.009 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 0, 2024-04-25T10:45:01.010 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 0, 2024-04-25T10:45:01.010 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 0, 2024-04-25T10:45:01.010 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 0, 2024-04-25T10:45:01.010 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-04-25T10:45:01.010 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:45:01.010 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 0, 2024-04-25T10:45:01.010 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 0, 2024-04-25T10:45:01.010 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-04-25T10:45:01.010 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-04-25T10:45:01.010 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-04-25T10:45:01.010 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:45:01.010 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:45:01.010 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-04-25T10:45:01.010 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-04-25T10:45:01.011 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 0, 2024-04-25T10:45:01.011 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 0, 2024-04-25T10:45:01.011 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T10:45:01.011 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2024-04-25T10:45:01.011 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-04-25T10:45:01.011 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:45:01.011 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-04-25T10:45:01.011 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-04-25T10:45:01.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.011 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-04-25T10:45:01.011 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-04-25T10:45:01.011 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 0, 2024-04-25T10:45:01.011 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2024-04-25T10:45:01.013 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2024-04-25T10:45:01.013 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 8, 2024-04-25T10:45:01.013 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-04-25T10:45:01.013 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2024-04-25T10:45:01.013 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 0, 2024-04-25T10:45:01.013 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2024-04-25T10:45:01.013 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2024-04-25T10:45:01.013 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 2, 2024-04-25T10:45:01.013 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 2, 2024-04-25T10:45:01.013 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-04-25T10:45:01.013 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2024-04-25T10:45:01.013 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 4194304, 2024-04-25T10:45:01.013 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 4196655, 2024-04-25T10:45:01.013 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 4196514, 2024-04-25T10:45:01.014 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-04-25T10:45:01.014 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.014 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.014 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.014 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 8 2024-04-25T10:45:01.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.014 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-04-25T10:45:01.014 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 3, 2024-04-25T10:45:01.014 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 3, 2024-04-25T10:45:01.014 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 0, 2024-04-25T10:45:01.014 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 1, 2024-04-25T10:45:01.014 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 1, 2024-04-25T10:45:01.015 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 0, 2024-04-25T10:45:01.015 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 0, 2024-04-25T10:45:01.015 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 0, 2024-04-25T10:45:01.015 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 0, 2024-04-25T10:45:01.015 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 0, 2024-04-25T10:45:01.015 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 0, 2024-04-25T10:45:01.015 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 0, 2024-04-25T10:45:01.015 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 120404, 2024-04-25T10:45:01.015 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 215440 2024-04-25T10:45:01.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.015 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-04-25T10:45:01.015 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 0, 2024-04-25T10:45:01.015 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 0, 2024-04-25T10:45:01.015 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 0, 2024-04-25T10:45:01.015 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 0, 2024-04-25T10:45:01.016 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 0, 2024-04-25T10:45:01.016 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-04-25T10:45:01.016 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.016 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.016 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.016 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-04-25T10:45:01.016 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.016 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.016 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.016 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-04-25T10:45:01.016 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.016 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.017 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.017 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-04-25T10:45:01.017 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.017 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.017 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.017 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-04-25T10:45:01.017 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.017 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.017 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.017 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-04-25T10:45:01.017 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.017 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.018 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.018 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-04-25T10:45:01.018 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.018 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.018 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.018 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-04-25T10:45:01.018 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.018 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.018 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.019 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-04-25T10:45:01.019 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.019 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.019 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.019 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-04-25T10:45:01.019 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.019 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.020 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.020 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-04-25T10:45:01.020 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.020 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.020 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.020 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-04-25T10:45:01.020 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.020 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.020 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.021 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-04-25T10:45:01.021 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.021 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.021 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.021 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-04-25T10:45:01.021 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.022 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.022 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.022 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-04-25T10:45:01.022 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.022 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.022 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.023 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-04-25T10:45:01.023 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.023 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.023 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.023 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-04-25T10:45:01.023 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.023 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.024 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.024 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-04-25T10:45:01.024 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.024 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.024 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.024 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-04-25T10:45:01.024 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.024 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.024 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.024 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-04-25T10:45:01.024 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.025 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.025 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.025 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-04-25T10:45:01.025 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.025 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.025 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.025 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-04-25T10:45:01.025 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.025 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.025 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.025 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-04-25T10:45:01.026 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.026 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.026 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.026 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-04-25T10:45:01.026 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.026 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.026 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.026 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-04-25T10:45:01.026 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.026 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.026 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.027 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-04-25T10:45:01.027 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.027 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.027 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.027 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-04-25T10:45:01.027 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.027 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.027 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.027 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-04-25T10:45:01.027 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.027 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.028 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.028 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-04-25T10:45:01.028 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.028 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.028 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.028 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:45:01.028 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:45:01.028 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-04-25T10:45:01.028 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.028 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.028 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.028 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:01.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.029 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-04-25T10:45:01.029 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 2, 2024-04-25T10:45:01.029 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 4, 2024-04-25T10:45:01.029 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 2, 2024-04-25T10:45:01.029 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 4, 2024-04-25T10:45:01.029 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-04-25T10:45:01.029 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 0, 2024-04-25T10:45:01.029 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 0, 2024-04-25T10:45:01.029 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 3113, 2024-04-25T10:45:01.029 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-04-25T10:45:01.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.029 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-04-25T10:45:01.029 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:45:01.030 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-04-25T10:45:01.030 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:45:01.030 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:45:01.030 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:45:01.030 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:45:01.030 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:45:01.030 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:45:01.030 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:45:01.030 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:45:01.030 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:45:01.030 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:45:01.030 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:45:01.030 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:45:01.030 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:45:01.031 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-04-25T10:45:01.031 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:45:01.031 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-04-25T10:45:01.031 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:45:01.031 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:45:01.031 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:45:01.031 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:45:01.031 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:45:01.031 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:45:01.031 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:45:01.031 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-04-25T10:45:01.031 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:45:01.031 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:45:01.031 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:45:01.032 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-04-25T10:45:01.032 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-04-25T10:45:01.032 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-04-25T10:45:01.032 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:45:01.032 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:45:01.032 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:45:01.032 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:45:01.032 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 181313, 2024-04-25T10:45:01.032 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 47, 2024-04-25T10:45:01.032 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:45:01.032 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-04-25T10:45:01.032 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-04-25T10:45:01.032 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-04-25T10:45:01.032 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:45:01.033 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-04-25T10:45:01.033 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:45:01.033 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-04-25T10:45:01.033 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 8032, 2024-04-25T10:45:01.033 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 167, 2024-04-25T10:45:01.033 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:45:01.033 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:45:01.033 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-04-25T10:45:01.033 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-04-25T10:45:01.033 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 8512, 2024-04-25T10:45:01.033 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 13, 2024-04-25T10:45:01.033 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-04-25T10:45:01.033 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-04-25T10:45:01.033 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-04-25T10:45:01.033 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-04-25T10:45:01.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.034 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-04-25T10:45:01.034 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-04-25T10:45:01.034 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-04-25T10:45:01.034 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 2482, 2024-04-25T10:45:01.034 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 21, 2024-04-25T10:45:01.034 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 3, 2024-04-25T10:45:01.034 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 2482, 2024-04-25T10:45:01.034 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-04-25T10:45:01.034 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2482, 2024-04-25T10:45:01.034 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 1.377977307, 2024-04-25T10:45:01.034 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000555188 2024-04-25T10:45:01.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.034 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-04-25T10:45:01.035 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-04-25T10:45:01.035 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2482, 2024-04-25T10:45:01.035 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 2483 2024-04-25T10:45:01.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.035 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 2482, 2024-04-25T10:45:01.035 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 2482, 2024-04-25T10:45:01.035 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 0, 2024-04-25T10:45:01.035 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-04-25T10:45:01.035 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-04-25T10:45:01.035 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 1609, 2024-04-25T10:45:01.035 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 0, 2024-04-25T10:45:01.035 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 871, 2024-04-25T10:45:01.035 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 0, 2024-04-25T10:45:01.035 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 0, 2024-04-25T10:45:01.036 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-04-25T10:45:01.036 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-04-25T10:45:01.036 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-04-25T10:45:01.036 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2024-04-25T10:45:01.036 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 0, 2024-04-25T10:45:01.036 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-04-25T10:45:01.036 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-04-25T10:45:01.036 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-04-25T10:45:01.036 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 1, 2024-04-25T10:45:01.036 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 0, 2024-04-25T10:45:01.036 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:45:01.036 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-04-25T10:45:01.036 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:45:01.036 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-04-25T10:45:01.036 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-04-25T10:45:01.037 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-04-25T10:45:01.037 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:45:01.037 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-04-25T10:45:01.037 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:45:01.037 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-04-25T10:45:01.037 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-04-25T10:45:01.037 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-04-25T10:45:01.037 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-04-25T10:45:01.037 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-04-25T10:45:01.037 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-04-25T10:45:01.037 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-04-25T10:45:01.037 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-04-25T10:45:01.037 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-04-25T10:45:01.037 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-04-25T10:45:01.037 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-04-25T10:45:01.038 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-04-25T10:45:01.038 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-04-25T10:45:01.038 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-04-25T10:45:01.038 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-04-25T10:45:01.038 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-04-25T10:45:01.038 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-04-25T10:45:01.038 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 93, 2024-04-25T10:45:01.038 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-04-25T10:45:01.038 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 16, 2024-04-25T10:45:01.038 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 4, 2024-04-25T10:45:01.038 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 4, 2024-04-25T10:45:01.038 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-04-25T10:45:01.038 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-04-25T10:45:01.039 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 0, 2024-04-25T10:45:01.039 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 2, 2024-04-25T10:45:01.039 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 0 2024-04-25T10:45:01.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.039 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-04-25T10:45:01.039 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 0, 2024-04-25T10:45:01.039 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:45:01.039 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-04-25T10:45:01.039 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-04-25T10:45:01.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.039 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-04-25T10:45:01.039 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 0 2024-04-25T10:45:01.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.039 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-04-25T10:45:01.039 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-04-25T10:45:01.039 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T10:45:01.040 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-04-25T10:45:01.040 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 0, 2024-04-25T10:45:01.040 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 0, 2024-04-25T10:45:01.040 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 0, 2024-04-25T10:45:01.040 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-04-25T10:45:01.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.040 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:45:01.040 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T10:45:01.040 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-25T10:45:01.040 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T10:45:01.040 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 2759, 2024-04-25T10:45:01.040 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 2183874, 2024-04-25T10:45:01.040 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T10:45:01.040 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 2759, 2024-04-25T10:45:01.041 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-25T10:45:01.041 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-25T10:45:01.041 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 2759, 2024-04-25T10:45:01.041 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 2183874, 2024-04-25T10:45:01.041 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T10:45:01.041 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.041 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.041 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.041 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:01.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.041 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-04-25T10:45:01.041 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T10:45:01.041 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-25T10:45:01.041 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T10:45:01.041 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T10:45:01.042 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T10:45:01.042 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T10:45:01.042 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T10:45:01.042 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 2482, 2024-04-25T10:45:01.042 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 2356, 2024-04-25T10:45:01.042 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 2482, 2024-04-25T10:45:01.042 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 2356, 2024-04-25T10:45:01.042 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T10:45:01.042 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.043 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.043 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.043 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:01.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.043 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-04-25T10:45:01.043 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T10:45:01.043 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-04-25T10:45:01.043 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T10:45:01.043 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T10:45:01.043 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T10:45:01.043 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T10:45:01.044 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T10:45:01.044 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 2482, 2024-04-25T10:45:01.044 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 2482, 2024-04-25T10:45:01.044 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 2482, 2024-04-25T10:45:01.044 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 2482, 2024-04-25T10:45:01.044 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T10:45:01.044 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.044 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.044 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.044 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:01.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.044 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:45:01.044 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T10:45:01.044 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-25T10:45:01.045 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T10:45:01.045 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T10:45:01.045 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T10:45:01.045 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T10:45:01.045 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T10:45:01.045 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-25T10:45:01.045 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-25T10:45:01.045 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-04-25T10:45:01.045 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-04-25T10:45:01.045 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T10:45:01.045 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.045 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.045 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.045 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:01.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:01.046 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x562356902750": { 2024-04-25T10:45:01.046 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T10:45:01.046 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-25T10:45:01.046 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T10:45:01.046 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T10:45:01.046 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T10:45:01.046 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T10:45:01.046 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T10:45:01.046 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-25T10:45:01.046 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-25T10:45:01.046 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-04-25T10:45:01.046 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-04-25T10:45:01.046 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T10:45:01.046 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:01.046 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:01.047 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:01.047 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:01.047 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:01.047 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-25T10:45:01.750 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T10:45:01.750 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T10:45:01.751 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-25T10:45:01.751 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-04-25T10:45:01.751 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-25T10:45:01.751 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-25T10:45:01.751 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T10:45:01.751 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T10:45:01.752 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T10:45:01.999 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:45:02.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:01 smithi077 ceph-mon[32772]: pgmap v390: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:45:02.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:01 smithi164 ceph-mon[32713]: pgmap v390: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:45:02.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:01 smithi043 ceph-mon[29478]: pgmap v390: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:45:03.596 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-25T10:45:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:45:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 88, 2024-04-25T10:45:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 130, 2024-04-25T10:45:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 16198, 2024-04-25T10:45:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 595996, 2024-04-25T10:45:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 58, 2024-04-25T10:45:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 2, 2024-04-25T10:45:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.035066226, 2024-04-25T10:45:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.005544044, 2024-04-25T10:45:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.023109265, 2024-04-25T10:45:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000742410, 2024-04-25T10:45:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:45:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 128, 2024-04-25T10:45:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.017170717, 2024-04-25T10:45:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000134146 2024-04-25T10:45:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:45:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 120, 2024-04-25T10:45:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000026475, 2024-04-25T10:45:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000220 2024-04-25T10:45:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 1584, 2024-04-25T10:45:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 1440 2024-04-25T10:45:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:45:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 327, 2024-04-25T10:45:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 309, 2024-04-25T10:45:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 1750544, 2024-04-25T10:45:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 193006, 2024-04-25T10:45:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 8, 2024-04-25T10:45:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 5, 2024-04-25T10:45:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.061848874, 2024-04-25T10:45:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.017942577, 2024-04-25T10:45:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.049215803, 2024-04-25T10:45:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.007125875, 2024-04-25T10:45:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:45:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 309, 2024-04-25T10:45:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.031471643, 2024-04-25T10:45:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000101849 2024-04-25T10:45:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:45:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2024-04-25T10:45:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000281, 2024-04-25T10:45:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000140 2024-04-25T10:45:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 1746176, 2024-04-25T10:45:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 156160 2024-04-25T10:45:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:45:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 1345, 2024-04-25T10:45:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 1103, 2024-04-25T10:45:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 367453, 2024-04-25T10:45:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 914991, 2024-04-25T10:45:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 9, 2024-04-25T10:45:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 5, 2024-04-25T10:45:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.118630077, 2024-04-25T10:45:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.049981504, 2024-04-25T10:45:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.075152387, 2024-04-25T10:45:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.005815667, 2024-04-25T10:45:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:45:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1100, 2024-04-25T10:45:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.056973128, 2024-04-25T10:45:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000051793 2024-04-25T10:45:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:45:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2223, 2024-04-25T10:45:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000619299, 2024-04-25T10:45:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000278 2024-04-25T10:45:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 2208, 2024-04-25T10:45:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 493072 2024-04-25T10:45:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-04-25T10:45:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-04-25T10:45:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-04-25T10:45:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-04-25T10:45:03.603 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-25T10:45:03.603 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-25T10:45:03.603 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 34, 2024-04-25T10:45:03.603 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.002789159, 2024-04-25T10:45:03.603 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000082034 2024-04-25T10:45:03.603 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:03.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.603 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-04-25T10:45:03.603 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-25T10:45:03.603 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-25T10:45:03.603 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2024-04-25T10:45:03.603 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000008876, 2024-04-25T10:45:03.603 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000004438 2024-04-25T10:45:03.603 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:03.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-04-25T10:45:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 3, 2024-04-25T10:45:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 2, 2024-04-25T10:45:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-04-25T10:45:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2024-04-25T10:45:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000345957, 2024-04-25T10:45:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000172978 2024-04-25T10:45:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-04-25T10:45:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 1, 2024-04-25T10:45:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-04-25T10:45:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-04-25T10:45:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 11, 2024-04-25T10:45:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2024-04-25T10:45:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2024-04-25T10:45:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 16, 2024-04-25T10:45:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 0, 2024-04-25T10:45:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 16, 2024-04-25T10:45:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 0, 2024-04-25T10:45:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 13, 2024-04-25T10:45:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 4, 2024-04-25T10:45:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 1, 2024-04-25T10:45:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 1, 2024-04-25T10:45:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 3, 2024-04-25T10:45:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 10, 2024-04-25T10:45:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 5, 2024-04-25T10:45:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 0, 2024-04-25T10:45:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 4, 2024-04-25T10:45:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:45:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-04-25T10:45:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2024-04-25T10:45:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-04-25T10:45:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-04-25T10:45:03.607 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2024-04-25T10:45:03.607 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2024-04-25T10:45:03.607 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2024-04-25T10:45:03.607 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2024-04-25T10:45:03.607 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-04-25T10:45:03.607 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-04-25T10:45:03.607 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-04-25T10:45:03.607 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 5, 2024-04-25T10:45:03.607 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 298, 2024-04-25T10:45:03.607 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 6, 2024-04-25T10:45:03.607 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T10:45:03.607 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-04-25T10:45:03.607 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:45:03.607 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:45:03.607 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-04-25T10:45:03.608 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T10:45:03.608 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T10:45:03.608 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-04-25T10:45:03.608 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:45:03.608 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T10:45:03.608 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 0, 2024-04-25T10:45:03.608 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:45:03.608 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:45:03.608 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:45:03.608 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 0, 2024-04-25T10:45:03.608 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 0, 2024-04-25T10:45:03.608 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 0, 2024-04-25T10:45:03.608 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 1 2024-04-25T10:45:03.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-04-25T10:45:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 0, 2024-04-25T10:45:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 9, 2024-04-25T10:45:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 6, 2024-04-25T10:45:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 8, 2024-04-25T10:45:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 0, 2024-04-25T10:45:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 0, 2024-04-25T10:45:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-04-25T10:45:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:45:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 0, 2024-04-25T10:45:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 0, 2024-04-25T10:45:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-04-25T10:45:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-04-25T10:45:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-04-25T10:45:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:45:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:45:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-04-25T10:45:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-04-25T10:45:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 0, 2024-04-25T10:45:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 0, 2024-04-25T10:45:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T10:45:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2024-04-25T10:45:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-04-25T10:45:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:45:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-04-25T10:45:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-04-25T10:45:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-04-25T10:45:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-04-25T10:45:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 6, 2024-04-25T10:45:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2024-04-25T10:45:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2024-04-25T10:45:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 6, 2024-04-25T10:45:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-04-25T10:45:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2024-04-25T10:45:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 2, 2024-04-25T10:45:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2024-04-25T10:45:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2024-04-25T10:45:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 2, 2024-04-25T10:45:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 2, 2024-04-25T10:45:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-04-25T10:45:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2024-04-25T10:45:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 4194304, 2024-04-25T10:45:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 4196123, 2024-04-25T10:45:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 0, 2024-04-25T10:45:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-04-25T10:45:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 5, 2024-04-25T10:45:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.009931634, 2024-04-25T10:45:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001986326 2024-04-25T10:45:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 0 2024-04-25T10:45:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-04-25T10:45:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 20, 2024-04-25T10:45:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 22, 2024-04-25T10:45:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 2, 2024-04-25T10:45:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 19, 2024-04-25T10:45:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 22, 2024-04-25T10:45:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 3, 2024-04-25T10:45:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 16, 2024-04-25T10:45:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 20, 2024-04-25T10:45:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 4, 2024-04-25T10:45:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 1, 2024-04-25T10:45:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 2, 2024-04-25T10:45:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 1, 2024-04-25T10:45:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 124236, 2024-04-25T10:45:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 223632 2024-04-25T10:45:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-04-25T10:45:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 4, 2024-04-25T10:45:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 0, 2024-04-25T10:45:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 3, 2024-04-25T10:45:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 24, 2024-04-25T10:45:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 0, 2024-04-25T10:45:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-04-25T10:45:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-04-25T10:45:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1, 2024-04-25T10:45:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000217576, 2024-04-25T10:45:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000217576 2024-04-25T10:45:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-04-25T10:45:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-04-25T10:45:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-04-25T10:45:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-04-25T10:45:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-04-25T10:45:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1, 2024-04-25T10:45:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000128381, 2024-04-25T10:45:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000128381 2024-04-25T10:45:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-04-25T10:45:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-04-25T10:45:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-04-25T10:45:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-04-25T10:45:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.619 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-04-25T10:45:03.619 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.619 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.619 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-04-25T10:45:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-04-25T10:45:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-04-25T10:45:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-04-25T10:45:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-04-25T10:45:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.622 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-04-25T10:45:03.622 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.622 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.622 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.622 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-04-25T10:45:03.622 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-04-25T10:45:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-04-25T10:45:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-04-25T10:45:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-04-25T10:45:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-04-25T10:45:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-04-25T10:45:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-04-25T10:45:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-04-25T10:45:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-04-25T10:45:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.627 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-04-25T10:45:03.627 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.627 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.627 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.627 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:45:03.627 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:45:03.627 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-04-25T10:45:03.627 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-04-25T10:45:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 2, 2024-04-25T10:45:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 3, 2024-04-25T10:45:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 1, 2024-04-25T10:45:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 2, 2024-04-25T10:45:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-04-25T10:45:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 1, 2024-04-25T10:45:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 1, 2024-04-25T10:45:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 730, 2024-04-25T10:45:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-04-25T10:45:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-04-25T10:45:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:45:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-04-25T10:45:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:45:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:45:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:45:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:45:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:45:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:45:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:45:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:45:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:45:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:45:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:45:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:45:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:45:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-04-25T10:45:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:45:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-04-25T10:45:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:45:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:45:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:45:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:45:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:45:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:45:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:45:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-04-25T10:45:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:45:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:45:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:45:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-04-25T10:45:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-04-25T10:45:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-04-25T10:45:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:45:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:45:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:45:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:45:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 270951, 2024-04-25T10:45:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 90, 2024-04-25T10:45:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:45:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-04-25T10:45:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-04-25T10:45:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-04-25T10:45:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:45:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-04-25T10:45:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:45:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-04-25T10:45:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 8032, 2024-04-25T10:45:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 167, 2024-04-25T10:45:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:45:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:45:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-04-25T10:45:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-04-25T10:45:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 86717, 2024-04-25T10:45:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 336, 2024-04-25T10:45:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-04-25T10:45:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-04-25T10:45:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-04-25T10:45:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-04-25T10:45:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-04-25T10:45:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-04-25T10:45:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-04-25T10:45:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 43, 2024-04-25T10:45:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 12422, 2024-04-25T10:45:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 33, 2024-04-25T10:45:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 43, 2024-04-25T10:45:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-04-25T10:45:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 43, 2024-04-25T10:45:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.234996167, 2024-04-25T10:45:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.005465027 2024-04-25T10:45:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-04-25T10:45:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-04-25T10:45:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 43, 2024-04-25T10:45:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 66 2024-04-25T10:45:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 43, 2024-04-25T10:45:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 0, 2024-04-25T10:45:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 43, 2024-04-25T10:45:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-04-25T10:45:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-04-25T10:45:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 0, 2024-04-25T10:45:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 10, 2024-04-25T10:45:03.637 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 0, 2024-04-25T10:45:03.637 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 5, 2024-04-25T10:45:03.637 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 8, 2024-04-25T10:45:03.637 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-04-25T10:45:03.637 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-04-25T10:45:03.637 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-04-25T10:45:03.637 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2024-04-25T10:45:03.637 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 6, 2024-04-25T10:45:03.637 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-04-25T10:45:03.637 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-04-25T10:45:03.637 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-04-25T10:45:03.637 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 0, 2024-04-25T10:45:03.637 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 20, 2024-04-25T10:45:03.637 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:45:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-04-25T10:45:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:45:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-04-25T10:45:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-04-25T10:45:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-04-25T10:45:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:45:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-04-25T10:45:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:45:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-04-25T10:45:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-04-25T10:45:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-04-25T10:45:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-04-25T10:45:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-04-25T10:45:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-04-25T10:45:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-04-25T10:45:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-04-25T10:45:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-04-25T10:45:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-04-25T10:45:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-04-25T10:45:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-04-25T10:45:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-04-25T10:45:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-04-25T10:45:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-04-25T10:45:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-04-25T10:45:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-04-25T10:45:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 93, 2024-04-25T10:45:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-04-25T10:45:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 16, 2024-04-25T10:45:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 9, 2024-04-25T10:45:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 9, 2024-04-25T10:45:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-04-25T10:45:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-04-25T10:45:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 16, 2024-04-25T10:45:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 0, 2024-04-25T10:45:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 1 2024-04-25T10:45:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-04-25T10:45:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 1, 2024-04-25T10:45:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:45:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-04-25T10:45:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-04-25T10:45:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-04-25T10:45:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 0 2024-04-25T10:45:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-04-25T10:45:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-04-25T10:45:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T10:45:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-04-25T10:45:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 0, 2024-04-25T10:45:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 0, 2024-04-25T10:45:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 0, 2024-04-25T10:45:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-04-25T10:45:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:45:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T10:45:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-25T10:45:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T10:45:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 1756, 2024-04-25T10:45:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 1956909, 2024-04-25T10:45:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T10:45:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 1756, 2024-04-25T10:45:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-25T10:45:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-25T10:45:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1756, 2024-04-25T10:45:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1956909, 2024-04-25T10:45:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T10:45:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-04-25T10:45:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T10:45:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-25T10:45:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T10:45:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T10:45:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T10:45:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T10:45:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T10:45:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 43, 2024-04-25T10:45:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 12422, 2024-04-25T10:45:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 43, 2024-04-25T10:45:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 12422, 2024-04-25T10:45:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T10:45:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-04-25T10:45:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T10:45:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-04-25T10:45:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T10:45:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T10:45:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T10:45:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T10:45:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T10:45:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 43, 2024-04-25T10:45:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 43, 2024-04-25T10:45:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 43, 2024-04-25T10:45:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 43, 2024-04-25T10:45:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T10:45:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:45:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T10:45:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-25T10:45:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T10:45:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T10:45:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T10:45:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T10:45:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T10:45:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-25T10:45:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-25T10:45:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-04-25T10:45:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-04-25T10:45:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T10:45:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x55a125bb0680": { 2024-04-25T10:45:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T10:45:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-25T10:45:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T10:45:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 6, 2024-04-25T10:45:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 1819, 2024-04-25T10:45:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T10:45:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 6, 2024-04-25T10:45:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-25T10:45:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-25T10:45:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 5, 2024-04-25T10:45:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1819, 2024-04-25T10:45:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T10:45:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:03.650 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:03.650 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:03.650 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-25T10:45:04.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:03 smithi077 ceph-mon[32772]: pgmap v391: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:45:04.040 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:03 smithi164 ceph-mon[32713]: pgmap v391: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:45:04.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:03 smithi043 ceph-mon[29478]: pgmap v391: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T10:45:04.254 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T10:45:04.254 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T10:45:04.254 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-25T10:45:04.254 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=9 2024-04-25T10:45:04.255 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-04-25T10:45:04.255 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-25T10:45:04.255 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T10:45:04.255 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T10:45:04.255 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T10:45:04.492 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:45:05.887 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:05 smithi164 ceph-mon[32713]: pgmap v392: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-04-25T10:45:06.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:05 smithi077 ceph-mon[32772]: pgmap v392: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-04-25T10:45:06.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:05 smithi043 ceph-mon[29478]: pgmap v392: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-04-25T10:45:06.186 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-25T10:45:06.186 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:45:06.186 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 1406, 2024-04-25T10:45:06.186 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 982, 2024-04-25T10:45:06.186 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 381765, 2024-04-25T10:45:06.186 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 1007527, 2024-04-25T10:45:06.186 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 55, 2024-04-25T10:45:06.186 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 4, 2024-04-25T10:45:06.187 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.132515667, 2024-04-25T10:45:06.187 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.044199223, 2024-04-25T10:45:06.187 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.090192859, 2024-04-25T10:45:06.187 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.006861540, 2024-04-25T10:45:06.187 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:45:06.187 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 980, 2024-04-25T10:45:06.187 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.062450409, 2024-04-25T10:45:06.187 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000063724 2024-04-25T10:45:06.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.187 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:45:06.187 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2242, 2024-04-25T10:45:06.187 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000713270, 2024-04-25T10:45:06.187 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000318 2024-04-25T10:45:06.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.188 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 2176, 2024-04-25T10:45:06.188 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 4288 2024-04-25T10:45:06.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.188 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:45:06.188 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 40, 2024-04-25T10:45:06.188 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 110, 2024-04-25T10:45:06.188 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 10899, 2024-04-25T10:45:06.188 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 195200, 2024-04-25T10:45:06.188 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 15, 2024-04-25T10:45:06.188 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 4, 2024-04-25T10:45:06.188 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.018489699, 2024-04-25T10:45:06.188 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.005546606, 2024-04-25T10:45:06.188 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.011050504, 2024-04-25T10:45:06.189 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000468629, 2024-04-25T10:45:06.189 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:45:06.189 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 108, 2024-04-25T10:45:06.189 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.014277925, 2024-04-25T10:45:06.189 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000132203 2024-04-25T10:45:06.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.189 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:45:06.189 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 81, 2024-04-25T10:45:06.189 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000021471, 2024-04-25T10:45:06.189 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000265 2024-04-25T10:45:06.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.189 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 5056, 2024-04-25T10:45:06.189 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 130448 2024-04-25T10:45:06.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.190 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:45:06.190 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 328, 2024-04-25T10:45:06.190 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 501, 2024-04-25T10:45:06.190 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 1753363, 2024-04-25T10:45:06.190 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 537155, 2024-04-25T10:45:06.190 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 7, 2024-04-25T10:45:06.190 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 6, 2024-04-25T10:45:06.190 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.067773148, 2024-04-25T10:45:06.190 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.024982217, 2024-04-25T10:45:06.190 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.045757573, 2024-04-25T10:45:06.190 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.006495315, 2024-04-25T10:45:06.190 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:45:06.190 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 500, 2024-04-25T10:45:06.191 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.036910056, 2024-04-25T10:45:06.191 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000073820 2024-04-25T10:45:06.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.191 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:45:06.191 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 66, 2024-04-25T10:45:06.191 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000015797, 2024-04-25T10:45:06.191 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000239 2024-04-25T10:45:06.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.191 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 1751136, 2024-04-25T10:45:06.191 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 528096 2024-04-25T10:45:06.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.191 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-04-25T10:45:06.191 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-04-25T10:45:06.191 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-04-25T10:45:06.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.192 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-04-25T10:45:06.192 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-25T10:45:06.192 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-25T10:45:06.192 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 50, 2024-04-25T10:45:06.192 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.002069626, 2024-04-25T10:45:06.192 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000041392 2024-04-25T10:45:06.192 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:06.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.192 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-04-25T10:45:06.192 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-25T10:45:06.192 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-25T10:45:06.192 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2024-04-25T10:45:06.192 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000008443, 2024-04-25T10:45:06.192 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000004221 2024-04-25T10:45:06.193 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:06.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.193 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-04-25T10:45:06.193 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 13, 2024-04-25T10:45:06.193 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 9, 2024-04-25T10:45:06.193 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-04-25T10:45:06.193 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 9, 2024-04-25T10:45:06.193 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.003374082, 2024-04-25T10:45:06.193 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000374898 2024-04-25T10:45:06.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.193 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-04-25T10:45:06.193 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 4, 2024-04-25T10:45:06.193 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-04-25T10:45:06.193 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-04-25T10:45:06.194 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 11, 2024-04-25T10:45:06.194 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2024-04-25T10:45:06.194 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2024-04-25T10:45:06.194 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 16, 2024-04-25T10:45:06.194 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 0, 2024-04-25T10:45:06.194 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 16, 2024-04-25T10:45:06.194 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 0, 2024-04-25T10:45:06.194 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 13, 2024-04-25T10:45:06.194 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 4, 2024-04-25T10:45:06.194 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 1, 2024-04-25T10:45:06.194 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 2, 2024-04-25T10:45:06.194 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 3, 2024-04-25T10:45:06.194 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 25, 2024-04-25T10:45:06.195 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 20, 2024-04-25T10:45:06.195 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 1, 2024-04-25T10:45:06.195 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 3, 2024-04-25T10:45:06.195 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:45:06.195 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-04-25T10:45:06.195 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2024-04-25T10:45:06.195 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-04-25T10:45:06.195 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-04-25T10:45:06.195 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2024-04-25T10:45:06.195 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2024-04-25T10:45:06.195 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2024-04-25T10:45:06.195 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2024-04-25T10:45:06.195 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-04-25T10:45:06.195 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-04-25T10:45:06.196 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-04-25T10:45:06.196 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 5, 2024-04-25T10:45:06.196 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 298, 2024-04-25T10:45:06.196 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 6, 2024-04-25T10:45:06.196 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T10:45:06.196 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-04-25T10:45:06.196 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:45:06.196 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:45:06.196 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-04-25T10:45:06.196 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T10:45:06.196 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T10:45:06.196 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-04-25T10:45:06.196 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:45:06.196 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T10:45:06.197 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 0, 2024-04-25T10:45:06.197 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:45:06.197 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:45:06.197 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:45:06.197 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 0, 2024-04-25T10:45:06.197 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 0, 2024-04-25T10:45:06.197 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 0, 2024-04-25T10:45:06.197 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 0 2024-04-25T10:45:06.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.197 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-04-25T10:45:06.197 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 0, 2024-04-25T10:45:06.197 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 11, 2024-04-25T10:45:06.197 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 7, 2024-04-25T10:45:06.197 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 8, 2024-04-25T10:45:06.197 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 0, 2024-04-25T10:45:06.198 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 0, 2024-04-25T10:45:06.198 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-04-25T10:45:06.198 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:45:06.198 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 0, 2024-04-25T10:45:06.198 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 0, 2024-04-25T10:45:06.198 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-04-25T10:45:06.198 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-04-25T10:45:06.198 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-04-25T10:45:06.198 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:45:06.198 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:45:06.198 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-04-25T10:45:06.198 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-04-25T10:45:06.198 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 0, 2024-04-25T10:45:06.198 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 0, 2024-04-25T10:45:06.199 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T10:45:06.199 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2024-04-25T10:45:06.199 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-04-25T10:45:06.199 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:45:06.199 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-04-25T10:45:06.199 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-04-25T10:45:06.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.199 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-04-25T10:45:06.199 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-04-25T10:45:06.199 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 8, 2024-04-25T10:45:06.199 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2024-04-25T10:45:06.199 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2024-04-25T10:45:06.199 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 8, 2024-04-25T10:45:06.199 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-04-25T10:45:06.200 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2024-04-25T10:45:06.200 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 2, 2024-04-25T10:45:06.200 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2024-04-25T10:45:06.200 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2024-04-25T10:45:06.200 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 2, 2024-04-25T10:45:06.200 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 2, 2024-04-25T10:45:06.200 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-04-25T10:45:06.200 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2024-04-25T10:45:06.200 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 4194304, 2024-04-25T10:45:06.200 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 4196655, 2024-04-25T10:45:06.200 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 0, 2024-04-25T10:45:06.200 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-04-25T10:45:06.200 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 7, 2024-04-25T10:45:06.200 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.012931226, 2024-04-25T10:45:06.201 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001847318 2024-04-25T10:45:06.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.201 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 0 2024-04-25T10:45:06.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.201 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-04-25T10:45:06.201 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 20, 2024-04-25T10:45:06.201 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 22, 2024-04-25T10:45:06.201 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 2, 2024-04-25T10:45:06.201 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 19, 2024-04-25T10:45:06.201 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 23, 2024-04-25T10:45:06.201 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 4, 2024-04-25T10:45:06.201 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 16, 2024-04-25T10:45:06.201 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 20, 2024-04-25T10:45:06.201 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 4, 2024-04-25T10:45:06.201 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 2, 2024-04-25T10:45:06.202 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 7, 2024-04-25T10:45:06.202 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 5, 2024-04-25T10:45:06.202 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 121488, 2024-04-25T10:45:06.202 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 223632 2024-04-25T10:45:06.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.202 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-04-25T10:45:06.202 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 16, 2024-04-25T10:45:06.202 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 0, 2024-04-25T10:45:06.202 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 13, 2024-04-25T10:45:06.203 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 28, 2024-04-25T10:45:06.203 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 1, 2024-04-25T10:45:06.203 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-04-25T10:45:06.203 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.203 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.203 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.203 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-04-25T10:45:06.203 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 5, 2024-04-25T10:45:06.203 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.002702058, 2024-04-25T10:45:06.203 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000540411 2024-04-25T10:45:06.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.204 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-04-25T10:45:06.204 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.204 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.204 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.204 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-04-25T10:45:06.204 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.204 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.204 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.204 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-04-25T10:45:06.204 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 3, 2024-04-25T10:45:06.204 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000537370, 2024-04-25T10:45:06.204 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000179123 2024-04-25T10:45:06.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.205 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-04-25T10:45:06.205 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.205 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.205 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.205 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-04-25T10:45:06.205 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1, 2024-04-25T10:45:06.205 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000134654, 2024-04-25T10:45:06.205 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000134654 2024-04-25T10:45:06.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.205 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-04-25T10:45:06.205 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.205 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.206 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.206 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-04-25T10:45:06.206 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.206 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.206 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.206 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-04-25T10:45:06.206 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.206 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.206 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.206 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-04-25T10:45:06.207 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.207 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.207 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.207 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-04-25T10:45:06.207 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.208 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.208 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.208 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-04-25T10:45:06.208 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.208 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.208 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.208 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-04-25T10:45:06.208 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.208 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.208 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.209 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-04-25T10:45:06.209 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.209 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.209 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.209 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-04-25T10:45:06.209 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.209 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.209 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.209 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-04-25T10:45:06.209 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.209 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.209 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.210 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-04-25T10:45:06.210 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.210 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.210 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.210 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-04-25T10:45:06.210 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.210 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.210 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.210 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-04-25T10:45:06.210 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.210 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.211 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.211 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-04-25T10:45:06.211 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.211 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.211 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.211 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-04-25T10:45:06.211 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.211 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.211 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.211 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-04-25T10:45:06.211 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.212 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.212 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.212 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-04-25T10:45:06.212 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.212 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.212 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.212 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-04-25T10:45:06.212 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.212 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.212 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.212 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-04-25T10:45:06.213 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.213 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.213 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.213 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-04-25T10:45:06.213 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.213 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.213 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.213 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-04-25T10:45:06.213 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.213 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.213 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.214 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-04-25T10:45:06.214 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.214 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.214 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.214 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:45:06.214 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:45:06.214 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-04-25T10:45:06.214 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.214 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.214 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.214 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:06.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.214 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-04-25T10:45:06.215 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 2, 2024-04-25T10:45:06.215 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 4, 2024-04-25T10:45:06.215 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 2, 2024-04-25T10:45:06.215 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 2, 2024-04-25T10:45:06.215 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-04-25T10:45:06.215 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 7, 2024-04-25T10:45:06.215 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 1, 2024-04-25T10:45:06.215 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 733, 2024-04-25T10:45:06.215 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-04-25T10:45:06.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.215 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-04-25T10:45:06.215 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:45:06.215 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-04-25T10:45:06.215 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:45:06.216 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:45:06.216 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:45:06.216 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:45:06.216 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:45:06.216 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:45:06.216 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:45:06.216 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:45:06.216 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:45:06.216 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:45:06.216 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:45:06.216 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:45:06.216 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:45:06.216 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-04-25T10:45:06.216 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:45:06.217 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-04-25T10:45:06.217 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:45:06.217 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:45:06.217 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:45:06.217 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:45:06.217 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:45:06.217 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:45:06.217 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:45:06.217 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-04-25T10:45:06.217 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:45:06.217 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:45:06.217 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:45:06.217 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-04-25T10:45:06.218 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-04-25T10:45:06.218 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-04-25T10:45:06.218 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:45:06.218 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:45:06.218 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:45:06.218 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:45:06.218 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 265159, 2024-04-25T10:45:06.218 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 106, 2024-04-25T10:45:06.218 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 176, 2024-04-25T10:45:06.218 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 2, 2024-04-25T10:45:06.218 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-04-25T10:45:06.218 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-04-25T10:45:06.218 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:45:06.219 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-04-25T10:45:06.219 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:45:06.219 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-04-25T10:45:06.219 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 8032, 2024-04-25T10:45:06.219 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 167, 2024-04-25T10:45:06.219 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:45:06.219 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:45:06.219 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-04-25T10:45:06.219 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-04-25T10:45:06.219 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 87141, 2024-04-25T10:45:06.219 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 345, 2024-04-25T10:45:06.219 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-04-25T10:45:06.220 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-04-25T10:45:06.220 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-04-25T10:45:06.220 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-04-25T10:45:06.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.220 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-04-25T10:45:06.220 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-04-25T10:45:06.220 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-04-25T10:45:06.220 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 47, 2024-04-25T10:45:06.220 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 13014, 2024-04-25T10:45:06.220 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 30, 2024-04-25T10:45:06.220 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 47, 2024-04-25T10:45:06.220 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-04-25T10:45:06.220 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 47, 2024-04-25T10:45:06.220 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.251995918, 2024-04-25T10:45:06.221 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.005361615 2024-04-25T10:45:06.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.221 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-04-25T10:45:06.221 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-04-25T10:45:06.221 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 47, 2024-04-25T10:45:06.221 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 72 2024-04-25T10:45:06.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.221 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 47, 2024-04-25T10:45:06.221 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 0, 2024-04-25T10:45:06.221 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 47, 2024-04-25T10:45:06.221 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-04-25T10:45:06.221 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-04-25T10:45:06.221 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 0, 2024-04-25T10:45:06.221 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 10, 2024-04-25T10:45:06.221 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 0, 2024-04-25T10:45:06.222 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 7, 2024-04-25T10:45:06.222 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 8, 2024-04-25T10:45:06.222 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-04-25T10:45:06.222 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-04-25T10:45:06.222 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-04-25T10:45:06.222 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2024-04-25T10:45:06.222 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 6, 2024-04-25T10:45:06.222 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-04-25T10:45:06.222 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-04-25T10:45:06.222 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-04-25T10:45:06.222 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 0, 2024-04-25T10:45:06.222 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 20, 2024-04-25T10:45:06.222 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:45:06.222 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-04-25T10:45:06.223 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:45:06.223 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-04-25T10:45:06.223 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-04-25T10:45:06.223 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-04-25T10:45:06.223 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:45:06.223 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-04-25T10:45:06.223 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:45:06.223 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-04-25T10:45:06.223 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-04-25T10:45:06.223 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-04-25T10:45:06.223 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-04-25T10:45:06.223 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-04-25T10:45:06.223 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-04-25T10:45:06.223 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-04-25T10:45:06.224 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-04-25T10:45:06.224 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-04-25T10:45:06.224 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-04-25T10:45:06.224 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-04-25T10:45:06.224 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-04-25T10:45:06.224 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-04-25T10:45:06.224 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-04-25T10:45:06.224 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-04-25T10:45:06.224 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-04-25T10:45:06.224 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-04-25T10:45:06.224 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 93, 2024-04-25T10:45:06.224 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-04-25T10:45:06.224 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 16, 2024-04-25T10:45:06.224 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 10, 2024-04-25T10:45:06.225 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 10, 2024-04-25T10:45:06.225 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-04-25T10:45:06.225 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-04-25T10:45:06.225 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 19, 2024-04-25T10:45:06.225 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 0, 2024-04-25T10:45:06.225 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 2 2024-04-25T10:45:06.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.225 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-04-25T10:45:06.225 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 1, 2024-04-25T10:45:06.225 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:45:06.225 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-04-25T10:45:06.225 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-04-25T10:45:06.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.225 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-04-25T10:45:06.225 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 0 2024-04-25T10:45:06.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.226 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-04-25T10:45:06.226 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-04-25T10:45:06.226 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T10:45:06.226 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-04-25T10:45:06.226 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 0, 2024-04-25T10:45:06.226 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 0, 2024-04-25T10:45:06.226 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 0, 2024-04-25T10:45:06.226 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-04-25T10:45:06.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.226 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:45:06.226 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T10:45:06.226 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-25T10:45:06.226 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T10:45:06.227 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 1770, 2024-04-25T10:45:06.227 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 1966935, 2024-04-25T10:45:06.227 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T10:45:06.227 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 1770, 2024-04-25T10:45:06.227 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-25T10:45:06.227 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-25T10:45:06.227 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1770, 2024-04-25T10:45:06.227 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1966935, 2024-04-25T10:45:06.227 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T10:45:06.227 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.227 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.227 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.227 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:06.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.228 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-04-25T10:45:06.228 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T10:45:06.228 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-25T10:45:06.228 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T10:45:06.228 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T10:45:06.228 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T10:45:06.228 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T10:45:06.228 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T10:45:06.228 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 47, 2024-04-25T10:45:06.228 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 13014, 2024-04-25T10:45:06.228 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 47, 2024-04-25T10:45:06.228 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 13014, 2024-04-25T10:45:06.228 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T10:45:06.228 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.229 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.229 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.229 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:06.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.229 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-04-25T10:45:06.229 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T10:45:06.229 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-04-25T10:45:06.229 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T10:45:06.229 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T10:45:06.229 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T10:45:06.229 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T10:45:06.229 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T10:45:06.229 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 47, 2024-04-25T10:45:06.229 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 47, 2024-04-25T10:45:06.230 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 47, 2024-04-25T10:45:06.230 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 47, 2024-04-25T10:45:06.230 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T10:45:06.230 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.230 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.230 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.230 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:06.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.230 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:45:06.230 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T10:45:06.230 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-25T10:45:06.230 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T10:45:06.230 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-25T10:45:06.230 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-25T10:45:06.231 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T10:45:06.231 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-25T10:45:06.231 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-25T10:45:06.231 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-25T10:45:06.231 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-04-25T10:45:06.231 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-04-25T10:45:06.231 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T10:45:06.231 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.231 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.231 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.231 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:06.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:06.231 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x559759a3c820": { 2024-04-25T10:45:06.231 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T10:45:06.232 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-25T10:45:06.232 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T10:45:06.232 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 8, 2024-04-25T10:45:06.232 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 2351, 2024-04-25T10:45:06.232 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T10:45:06.232 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 8, 2024-04-25T10:45:06.232 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-25T10:45:06.232 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-25T10:45:06.232 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 7, 2024-04-25T10:45:06.232 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 2351, 2024-04-25T10:45:06.232 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T10:45:06.232 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:06.232 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:06.232 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:06.232 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:06.233 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:06.233 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-25T10:45:06.861 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-25T10:45:06.861 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-25T10:45:06.861 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-04-25T10:45:06.861 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=11 2024-04-25T10:45:06.862 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-25T10:45:06.862 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-25T10:45:06.862 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T10:45:06.862 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T10:45:06.863 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T10:45:07.100 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T10:45:08.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:07 smithi077 ceph-mon[32772]: pgmap v393: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 45 KiB/s wr, 12 op/s 2024-04-25T10:45:08.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:07 smithi164 ceph-mon[32713]: pgmap v393: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 45 KiB/s wr, 12 op/s 2024-04-25T10:45:08.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:07 smithi043 ceph-mon[29478]: pgmap v393: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 45 KiB/s wr, 12 op/s 2024-04-25T10:45:08.715 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-25T10:45:08.715 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:45:08.715 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 255, 2024-04-25T10:45:08.715 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 244, 2024-04-25T10:45:08.716 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 1697200, 2024-04-25T10:45:08.716 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 150160, 2024-04-25T10:45:08.716 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 4, 2024-04-25T10:45:08.716 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 0, 2024-04-25T10:45:08.716 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.057783141, 2024-04-25T10:45:08.716 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.017049639, 2024-04-25T10:45:08.716 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.045352099, 2024-04-25T10:45:08.716 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.006699681, 2024-04-25T10:45:08.716 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:45:08.716 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 243, 2024-04-25T10:45:08.716 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.014734032, 2024-04-25T10:45:08.716 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000060633 2024-04-25T10:45:08.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:08.716 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:45:08.717 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:08.717 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:08.717 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:08.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:08.717 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 1697200, 2024-04-25T10:45:08.717 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 150160 2024-04-25T10:45:08.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:08.717 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:45:08.717 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 2, 2024-04-25T10:45:08.717 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 187, 2024-04-25T10:45:08.717 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 320, 2024-04-25T10:45:08.717 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 190112, 2024-04-25T10:45:08.717 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 5, 2024-04-25T10:45:08.717 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 3, 2024-04-25T10:45:08.718 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.013259084, 2024-04-25T10:45:08.718 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.008767555, 2024-04-25T10:45:08.718 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.001699194, 2024-04-25T10:45:08.718 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T10:45:08.718 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:45:08.718 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 184, 2024-04-25T10:45:08.718 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.022430993, 2024-04-25T10:45:08.718 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000121907 2024-04-25T10:45:08.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:08.718 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:45:08.718 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:08.718 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:08.718 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:08.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:08.719 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T10:45:08.719 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 190112 2024-04-25T10:45:08.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:08.719 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:45:08.719 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 4, 2024-04-25T10:45:08.719 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 4, 2024-04-25T10:45:08.719 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 3040, 2024-04-25T10:45:08.719 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 448, 2024-04-25T10:45:08.719 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 2, 2024-04-25T10:45:08.719 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 1, 2024-04-25T10:45:08.719 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.000644732, 2024-04-25T10:45:08.719 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.000029401, 2024-04-25T10:45:08.719 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.000601651, 2024-04-25T10:45:08.719 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T10:45:08.720 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:45:08.720 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 3, 2024-04-25T10:45:08.720 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000723200, 2024-04-25T10:45:08.720 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000241066 2024-04-25T10:45:08.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:08.720 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:45:08.720 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:08.720 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:08.720 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:08.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:08.721 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T10:45:08.721 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T10:45:08.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:08.721 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-04-25T10:45:08.721 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 0, 2024-04-25T10:45:08.721 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-04-25T10:45:08.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:08.721 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-04-25T10:45:08.721 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:45:08.721 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-04-25T10:45:08.721 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:45:08.721 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:45:08.721 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:45:08.721 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:45:08.722 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:45:08.722 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:45:08.722 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:45:08.722 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:45:08.722 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:45:08.722 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:45:08.722 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:45:08.722 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:45:08.722 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:45:08.722 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-04-25T10:45:08.722 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:45:08.722 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-04-25T10:45:08.722 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:45:08.722 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:45:08.723 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:45:08.723 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:45:08.723 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:45:08.723 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:45:08.723 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:45:08.723 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-04-25T10:45:08.723 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:45:08.723 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:45:08.723 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:45:08.723 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-04-25T10:45:08.723 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-04-25T10:45:08.723 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-04-25T10:45:08.723 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:45:08.723 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:45:08.723 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:45:08.724 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:45:08.724 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 148977, 2024-04-25T10:45:08.724 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 30, 2024-04-25T10:45:08.724 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:45:08.724 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-04-25T10:45:08.724 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-04-25T10:45:08.724 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-04-25T10:45:08.724 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:45:08.724 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-04-25T10:45:08.724 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:45:08.724 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-04-25T10:45:08.724 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 0, 2024-04-25T10:45:08.724 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 0, 2024-04-25T10:45:08.724 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:45:08.725 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:45:08.725 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-04-25T10:45:08.725 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-04-25T10:45:08.725 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 928, 2024-04-25T10:45:08.725 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 2, 2024-04-25T10:45:08.725 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-04-25T10:45:08.725 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-04-25T10:45:08.725 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-04-25T10:45:08.725 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-04-25T10:45:08.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-25T10:45:08.725 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:45:08.725 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-25T10:45:08.725 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-25T10:45:08.725 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-25T10:45:08.725 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 257, 2024-04-25T10:45:08.726 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 1661208, 2024-04-25T10:45:08.726 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-25T10:45:08.726 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 257, 2024-04-25T10:45:08.726 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-25T10:45:08.726 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-25T10:45:08.726 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 257, 2024-04-25T10:45:08.726 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1661208, 2024-04-25T10:45:08.726 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-25T10:45:08.726 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-25T10:45:08.726 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-25T10:45:08.726 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-25T10:45:08.726 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:08.726 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-25T10:45:08.727 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-25T10:45:09.344 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-25T10:45:09.345 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-25T10:45:09.345 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-04-25T10:45:09.345 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-04-25T10:45:09.345 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-25T10:45:09.345 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-25T10:45:09.345 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-25T10:45:09.345 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T10:45:09.357 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T10:45:09.357 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T10:45:09.357 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:45:09.379 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T10:45:09.379 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:45:09.439 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:45:09.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.603+0000 7f7eca39e640 1 -- 172.21.15.43:0/352478192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec4108a20 msgr2=0x7f7ec41025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:09.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.603+0000 7f7eca39e640 1 --2- 172.21.15.43:0/352478192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec4108a20 0x7f7ec41025b0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009900 tx=0x7f7eb402a410 comp rx=0 tx=0).stop 2024-04-25T10:45:09.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.603+0000 7f7eca39e640 1 -- 172.21.15.43:0/352478192 shutdown_connections 2024-04-25T10:45:09.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.603+0000 7f7eca39e640 1 --2- 172.21.15.43:0/352478192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ec410d380 0x7f7ec410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.603+0000 7f7eca39e640 1 --2- 172.21.15.43:0/352478192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ec4102af0 0x7f7ec4102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.603+0000 7f7eca39e640 1 --2- 172.21.15.43:0/352478192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec4108a20 0x7f7ec41025b0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.603+0000 7f7eca39e640 1 -- 172.21.15.43:0/352478192 >> 172.21.15.43:0/352478192 conn(0x7f7ec4078110 msgr2=0x7f7ec4078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.603+0000 7f7eca39e640 1 -- 172.21.15.43:0/352478192 shutdown_connections 2024-04-25T10:45:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.603+0000 7f7eca39e640 1 -- 172.21.15.43:0/352478192 wait complete. 2024-04-25T10:45:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.604+0000 7f7eca39e640 1 Processor -- start 2024-04-25T10:45:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.604+0000 7f7eca39e640 1 -- start start 2024-04-25T10:45:09.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.604+0000 7f7eca39e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ec4102af0 0x7f7ec41992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:09.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.604+0000 7f7eca39e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ec4108a20 0x7f7ec4199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:09.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.604+0000 7f7eca39e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec410d380 0x7f7ec4199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:09.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.604+0000 7f7eca39e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec4112c30 con 0x7f7ec410d380 2024-04-25T10:45:09.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.604+0000 7f7eca39e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec4112ab0 con 0x7f7ec4108a20 2024-04-25T10:45:09.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.604+0000 7f7eca39e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec4112db0 con 0x7f7ec4102af0 2024-04-25T10:45:09.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.604+0000 7f7ec893b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec410d380 0x7f7ec4199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:09.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.604+0000 7f7ec893b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec410d380 0x7f7ec4199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36258/0 (socket says 172.21.15.43:36258) 2024-04-25T10:45:09.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.604+0000 7f7ec893b640 1 -- 172.21.15.43:0/2265286382 learned_addr learned my addr 172.21.15.43:0/2265286382 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:45:09.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.604+0000 7f7ec37fe640 1 --2- 172.21.15.43:0/2265286382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ec4108a20 0x7f7ec4199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.604+0000 7f7ec3fff640 1 --2- 172.21.15.43:0/2265286382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ec4102af0 0x7f7ec41992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.605+0000 7f7ec893b640 1 -- 172.21.15.43:0/2265286382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ec4102af0 msgr2=0x7f7ec41992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.605+0000 7f7ec893b640 1 --2- 172.21.15.43:0/2265286382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ec4102af0 0x7f7ec41992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.605+0000 7f7ec893b640 1 -- 172.21.15.43:0/2265286382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ec4108a20 msgr2=0x7f7ec4199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.605+0000 7f7ec893b640 1 --2- 172.21.15.43:0/2265286382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ec4108a20 0x7f7ec4199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.605+0000 7f7ec893b640 1 -- 172.21.15.43:0/2265286382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec41a8990 con 0x7f7ec410d380 2024-04-25T10:45:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.605+0000 7f7ec893b640 1 --2- 172.21.15.43:0/2265286382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec410d380 0x7f7ec4199d60 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00eaa0 tx=0x7f7ebc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.605+0000 7f7ec17fa640 1 -- 172.21.15.43:0/2265286382 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc00cdb0 con 0x7f7ec410d380 2024-04-25T10:45:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.605+0000 7f7ec17fa640 1 -- 172.21.15.43:0/2265286382 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ebc005020 con 0x7f7ec410d380 2024-04-25T10:45:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.605+0000 7f7eca39e640 1 -- 172.21.15.43:0/2265286382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec41a8c80 con 0x7f7ec410d380 2024-04-25T10:45:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.605+0000 7f7ec17fa640 1 -- 172.21.15.43:0/2265286382 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc0116d0 con 0x7f7ec410d380 2024-04-25T10:45:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.605+0000 7f7eca39e640 1 -- 172.21.15.43:0/2265286382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec41a9180 con 0x7f7ec410d380 2024-04-25T10:45:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.606+0000 7f7eca39e640 1 -- 172.21.15.43:0/2265286382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ec406afa0 con 0x7f7ec410d380 2024-04-25T10:45:09.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.607+0000 7f7ec17fa640 1 -- 172.21.15.43:0/2265286382 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f7ebc0051d0 con 0x7f7ec410d380 2024-04-25T10:45:09.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.608+0000 7f7ec17fa640 1 --2- 172.21.15.43:0/2265286382 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7eac0afc60 0x7f7eac0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:09.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.608+0000 7f7ec3fff640 1 --2- 172.21.15.43:0/2265286382 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7eac0afc60 0x7f7eac0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:09.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.609+0000 7f7ec17fa640 1 -- 172.21.15.43:0/2265286382 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7f7ebc0df210 con 0x7f7ec410d380 2024-04-25T10:45:09.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.609+0000 7f7ec3fff640 1 --2- 172.21.15.43:0/2265286382 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7eac0afc60 0x7f7eac0b2170 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4006010 tx=0x7f7eb4031400 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:09.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.613+0000 7f7ec17fa640 1 -- 172.21.15.43:0/2265286382 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ebc016030 con 0x7f7ec410d380 2024-04-25T10:45:09.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.805+0000 7f7eca39e640 1 -- 172.21.15.43:0/2265286382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ec402db50 con 0x7f7ec410d380 2024-04-25T10:45:09.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.807+0000 7f7ec17fa640 1 -- 172.21.15.43:0/2265286382 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+23153 (secure 0 0 0) 0x7f7ebc0a7a30 con 0x7f7ec410d380 2024-04-25T10:45:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:45:09.809 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":94,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:45:08.758017+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:31.503414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:32.224334+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":94,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1599315071":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6801/1953858940":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6800/1953858940":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3026776968":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/188939647":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/1287690800":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:45:09.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.809+0000 7f7eca39e640 1 -- 172.21.15.43:0/2265286382 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7eac0afc60 msgr2=0x7f7eac0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:09.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.809+0000 7f7eca39e640 1 --2- 172.21.15.43:0/2265286382 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7eac0afc60 0x7f7eac0b2170 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4006010 tx=0x7f7eb4031400 comp rx=0 tx=0).stop 2024-04-25T10:45:09.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.809+0000 7f7eca39e640 1 -- 172.21.15.43:0/2265286382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec410d380 msgr2=0x7f7ec4199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:09.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.809+0000 7f7eca39e640 1 --2- 172.21.15.43:0/2265286382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec410d380 0x7f7ec4199d60 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00eaa0 tx=0x7f7ebc00ef60 comp rx=0 tx=0).stop 2024-04-25T10:45:09.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.809+0000 7f7eca39e640 1 -- 172.21.15.43:0/2265286382 shutdown_connections 2024-04-25T10:45:09.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.809+0000 7f7eca39e640 1 --2- 172.21.15.43:0/2265286382 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f7eac0afc60 0x7f7eac0b2170 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:09.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.809+0000 7f7eca39e640 1 --2- 172.21.15.43:0/2265286382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ec4108a20 0x7f7ec4199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:09.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.809+0000 7f7eca39e640 1 --2- 172.21.15.43:0/2265286382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec410d380 0x7f7ec4199d60 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:09.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.809+0000 7f7eca39e640 1 --2- 172.21.15.43:0/2265286382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ec4102af0 0x7f7ec41992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:09.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.809+0000 7f7eca39e640 1 -- 172.21.15.43:0/2265286382 >> 172.21.15.43:0/2265286382 conn(0x7f7ec4078110 msgr2=0x7f7ec41040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:09.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.809+0000 7f7eca39e640 1 -- 172.21.15.43:0/2265286382 shutdown_connections 2024-04-25T10:45:09.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.809+0000 7f7eca39e640 1 -- 172.21.15.43:0/2265286382 wait complete. 2024-04-25T10:45:09.823 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:25:58.167476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:31.503414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:32.224334+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 94, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:45:09.823 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:45:09.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.937+0000 7fe54a070640 1 -- 172.21.15.43:0/247387349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe544108eb0 msgr2=0x7fe544102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:09.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.937+0000 7fe54a070640 1 --2- 172.21.15.43:0/247387349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe544108eb0 0x7fe544102b50 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009900 tx=0x7fe52c02a410 comp rx=0 tx=0).stop 2024-04-25T10:45:09.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.937+0000 7fe54a070640 1 -- 172.21.15.43:0/247387349 shutdown_connections 2024-04-25T10:45:09.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.937+0000 7fe54a070640 1 --2- 172.21.15.43:0/247387349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5441031c0 0x7fe54410f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:09.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.937+0000 7fe54a070640 1 --2- 172.21.15.43:0/247387349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe544108eb0 0x7fe544102b50 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:09.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.938+0000 7fe54a070640 1 --2- 172.21.15.43:0/247387349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe544108a10 0x7fe544102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:09.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.938+0000 7fe54a070640 1 -- 172.21.15.43:0/247387349 >> 172.21.15.43:0/247387349 conn(0x7fe544078110 msgr2=0x7fe544078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:09.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.938+0000 7fe54a070640 1 -- 172.21.15.43:0/247387349 shutdown_connections 2024-04-25T10:45:09.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.938+0000 7fe54a070640 1 -- 172.21.15.43:0/247387349 wait complete. 2024-04-25T10:45:09.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.938+0000 7fe54a070640 1 Processor -- start 2024-04-25T10:45:09.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.938+0000 7fe54a070640 1 -- start start 2024-04-25T10:45:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.938+0000 7fe54a070640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5441031c0 0x7fe54419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.938+0000 7fe54a070640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe544108a10 0x7fe54419acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.938+0000 7fe54a070640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe544108eb0 0x7fe54419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.939+0000 7fe54a070640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe544113d30 con 0x7fe544108eb0 2024-04-25T10:45:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.939+0000 7fe54a070640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe544113bb0 con 0x7fe544108a10 2024-04-25T10:45:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.939+0000 7fe54a070640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe544113eb0 con 0x7fe5441031c0 2024-04-25T10:45:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.939+0000 7fe543fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe544108eb0 0x7fe54419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.939+0000 7fe543fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe544108eb0 0x7fe54419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36274/0 (socket says 172.21.15.43:36274) 2024-04-25T10:45:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.939+0000 7fe543fff640 1 -- 172.21.15.43:0/2082298115 learned_addr learned my addr 172.21.15.43:0/2082298115 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:45:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.939+0000 7fe5437fe640 1 --2- 172.21.15.43:0/2082298115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5441031c0 0x7fe54419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.939+0000 7fe53bfff640 1 --2- 172.21.15.43:0/2082298115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe544108a10 0x7fe54419acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.939+0000 7fe543fff640 1 -- 172.21.15.43:0/2082298115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5441031c0 msgr2=0x7fe54419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.939+0000 7fe543fff640 1 --2- 172.21.15.43:0/2082298115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5441031c0 0x7fe54419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.939+0000 7fe543fff640 1 -- 172.21.15.43:0/2082298115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe544108a10 msgr2=0x7fe54419acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.939+0000 7fe543fff640 1 --2- 172.21.15.43:0/2082298115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe544108a10 0x7fe54419acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:09.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.939+0000 7fe543fff640 1 -- 172.21.15.43:0/2082298115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54419f550 con 0x7fe544108eb0 2024-04-25T10:45:09.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.939+0000 7fe543fff640 1 --2- 172.21.15.43:0/2082298115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe544108eb0 0x7fe54419ef90 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00ead0 tx=0x7fe53c00eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:09.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.939+0000 7fe5417fa640 1 -- 172.21.15.43:0/2082298115 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c004d20 con 0x7fe544108eb0 2024-04-25T10:45:09.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.939+0000 7fe54a070640 1 -- 172.21.15.43:0/2082298115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54419f840 con 0x7fe544108eb0 2024-04-25T10:45:09.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.939+0000 7fe5417fa640 1 -- 172.21.15.43:0/2082298115 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe53c005020 con 0x7fe544108eb0 2024-04-25T10:45:09.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.939+0000 7fe5417fa640 1 -- 172.21.15.43:0/2082298115 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c0116d0 con 0x7fe544108eb0 2024-04-25T10:45:09.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.939+0000 7fe54a070640 1 -- 172.21.15.43:0/2082298115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe54419fd10 con 0x7fe544108eb0 2024-04-25T10:45:09.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.940+0000 7fe54a070640 1 -- 172.21.15.43:0/2082298115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe544103d00 con 0x7fe544108eb0 2024-04-25T10:45:09.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.942+0000 7fe5417fa640 1 -- 172.21.15.43:0/2082298115 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fe53c0051d0 con 0x7fe544108eb0 2024-04-25T10:45:09.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.943+0000 7fe5417fa640 1 --2- 172.21.15.43:0/2082298115 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe5300afc60 0x7fe5300b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:09.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.943+0000 7fe5437fe640 1 --2- 172.21.15.43:0/2082298115 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe5300afc60 0x7fe5300b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:09.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.943+0000 7fe5417fa640 1 -- 172.21.15.43:0/2082298115 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7fe53c0df640 con 0x7fe544108eb0 2024-04-25T10:45:09.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.943+0000 7fe5437fe640 1 --2- 172.21.15.43:0/2082298115 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe5300afc60 0x7fe5300b2170 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe5440695c0 tx=0x7fe53400a2d0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:09.946+0000 7fe5417fa640 1 -- 172.21.15.43:0/2082298115 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe53c016030 con 0x7fe544108eb0 2024-04-25T10:45:10.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:09 smithi077 ceph-mon[32772]: pgmap v394: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-04-25T10:45:10.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:09 smithi077 ceph-mon[32772]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T10:45:10.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:09 smithi164 ceph-mon[32713]: pgmap v394: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-04-25T10:45:10.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:09 smithi164 ceph-mon[32713]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T10:45:10.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:09 smithi043 ceph-mon[29478]: pgmap v394: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-04-25T10:45:10.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:09 smithi043 ceph-mon[29478]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T10:45:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.134+0000 7fe54a070640 1 -- 172.21.15.43:0/2082298115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe544106f20 con 0x7fe544108eb0 2024-04-25T10:45:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.134+0000 7fe5417fa640 1 -- 172.21.15.43:0/2082298115 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fe53c0a7e60 con 0x7fe544108eb0 2024-04-25T10:45:10.136 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T10:45:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.136+0000 7fe54a070640 1 -- 172.21.15.43:0/2082298115 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe5300afc60 msgr2=0x7fe5300b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.136+0000 7fe54a070640 1 --2- 172.21.15.43:0/2082298115 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe5300afc60 0x7fe5300b2170 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe5440695c0 tx=0x7fe53400a2d0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.136+0000 7fe54a070640 1 -- 172.21.15.43:0/2082298115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe544108eb0 msgr2=0x7fe54419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.136+0000 7fe54a070640 1 --2- 172.21.15.43:0/2082298115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe544108eb0 0x7fe54419ef90 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00ead0 tx=0x7fe53c00eed0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.136+0000 7fe54a070640 1 -- 172.21.15.43:0/2082298115 shutdown_connections 2024-04-25T10:45:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.136+0000 7fe54a070640 1 --2- 172.21.15.43:0/2082298115 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fe5300afc60 0x7fe5300b2170 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.136+0000 7fe54a070640 1 --2- 172.21.15.43:0/2082298115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe544108a10 0x7fe54419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.136+0000 7fe54a070640 1 --2- 172.21.15.43:0/2082298115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe544108eb0 0x7fe54419ef90 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.136+0000 7fe54a070640 1 --2- 172.21.15.43:0/2082298115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5441031c0 0x7fe54419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.136+0000 7fe54a070640 1 -- 172.21.15.43:0/2082298115 >> 172.21.15.43:0/2082298115 conn(0x7fe544078110 msgr2=0x7fe5441040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:10.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.136+0000 7fe54a070640 1 -- 172.21.15.43:0/2082298115 shutdown_connections 2024-04-25T10:45:10.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.136+0000 7fe54a070640 1 -- 172.21.15.43:0/2082298115 wait complete. 2024-04-25T10:45:10.150 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:45:10.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.263+0000 7f8d8bf84640 1 -- 172.21.15.43:0/3353704425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d8410d380 msgr2=0x7f8d8410f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:10.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.263+0000 7f8d8bf84640 1 --2- 172.21.15.43:0/3353704425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d8410d380 0x7f8d8410f7a0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009ca0 tx=0x7f8d7c02a8c0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.263+0000 7f8d8bf84640 1 -- 172.21.15.43:0/3353704425 shutdown_connections 2024-04-25T10:45:10.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.263+0000 7f8d8bf84640 1 --2- 172.21.15.43:0/3353704425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d8410d380 0x7f8d8410f7a0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.263+0000 7f8d8bf84640 1 --2- 172.21.15.43:0/3353704425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d84102af0 0x7f8d84102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.263+0000 7f8d8bf84640 1 --2- 172.21.15.43:0/3353704425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d84108a20 0x7f8d841025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.263+0000 7f8d8bf84640 1 -- 172.21.15.43:0/3353704425 >> 172.21.15.43:0/3353704425 conn(0x7f8d84078100 msgr2=0x7f8d84078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:10.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.263+0000 7f8d8bf84640 1 -- 172.21.15.43:0/3353704425 shutdown_connections 2024-04-25T10:45:10.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.263+0000 7f8d8bf84640 1 -- 172.21.15.43:0/3353704425 wait complete. 2024-04-25T10:45:10.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.264+0000 7f8d8bf84640 1 Processor -- start 2024-04-25T10:45:10.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.264+0000 7f8d8bf84640 1 -- start start 2024-04-25T10:45:10.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.264+0000 7f8d8bf84640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d84102af0 0x7f8d8419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:10.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.264+0000 7f8d8bf84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d84108a20 0x7f8d8419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:10.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.264+0000 7f8d8bf84640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d8410d380 0x7f8d8419bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:10.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.264+0000 7f8d8bf84640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8d84112c00 con 0x7f8d84108a20 2024-04-25T10:45:10.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.264+0000 7f8d8bf84640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8d84112a80 con 0x7f8d84102af0 2024-04-25T10:45:10.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.264+0000 7f8d8bf84640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8d84112d80 con 0x7f8d8410d380 2024-04-25T10:45:10.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.264+0000 7f8d8951f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d84108a20 0x7f8d8419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:10.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.264+0000 7f8d8951f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d84108a20 0x7f8d8419ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36288/0 (socket says 172.21.15.43:36288) 2024-04-25T10:45:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.264+0000 7f8d8951f640 1 -- 172.21.15.43:0/2860915686 learned_addr learned my addr 172.21.15.43:0/2860915686 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:45:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.264+0000 7f8d8a521640 1 --2- 172.21.15.43:0/2860915686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d8410d380 0x7f8d8419bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.264+0000 7f8d89d20640 1 --2- 172.21.15.43:0/2860915686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d84102af0 0x7f8d8419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.264+0000 7f8d8951f640 1 -- 172.21.15.43:0/2860915686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d8410d380 msgr2=0x7f8d8419bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.264+0000 7f8d8951f640 1 --2- 172.21.15.43:0/2860915686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d8410d380 0x7f8d8419bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.264+0000 7f8d8951f640 1 -- 172.21.15.43:0/2860915686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d84102af0 msgr2=0x7f8d8419b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.264+0000 7f8d8951f640 1 --2- 172.21.15.43:0/2860915686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d84102af0 0x7f8d8419b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.264+0000 7f8d8951f640 1 -- 172.21.15.43:0/2860915686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d841a6860 con 0x7f8d84108a20 2024-04-25T10:45:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.264+0000 7f8d8a521640 1 --2- 172.21.15.43:0/2860915686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d8410d380 0x7f8d8419bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:45:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.265+0000 7f8d8951f640 1 --2- 172.21.15.43:0/2860915686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d84108a20 0x7f8d8419ba60 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f8d74005d00 tx=0x7f8d7400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.265+0000 7f8d7affd640 1 -- 172.21.15.43:0/2860915686 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74017070 con 0x7f8d84108a20 2024-04-25T10:45:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.265+0000 7f8d8bf84640 1 -- 172.21.15.43:0/2860915686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d841a6b50 con 0x7f8d84108a20 2024-04-25T10:45:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.265+0000 7f8d7affd640 1 -- 172.21.15.43:0/2860915686 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d74009c30 con 0x7f8d84108a20 2024-04-25T10:45:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.265+0000 7f8d7affd640 1 -- 172.21.15.43:0/2860915686 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7400f620 con 0x7f8d84108a20 2024-04-25T10:45:10.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.265+0000 7f8d8bf84640 1 -- 172.21.15.43:0/2860915686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d841a7050 con 0x7f8d84108a20 2024-04-25T10:45:10.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.266+0000 7f8d8bf84640 1 -- 172.21.15.43:0/2860915686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d8406afa0 con 0x7f8d84108a20 2024-04-25T10:45:10.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.268+0000 7f8d7affd640 1 -- 172.21.15.43:0/2860915686 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f8d7401a070 con 0x7f8d84108a20 2024-04-25T10:45:10.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.268+0000 7f8d7affd640 1 --2- 172.21.15.43:0/2860915686 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8d700afc60 0x7f8d700b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:10.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.268+0000 7f8d7affd640 1 -- 172.21.15.43:0/2860915686 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7f8d740deab0 con 0x7f8d84108a20 2024-04-25T10:45:10.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.268+0000 7f8d89d20640 1 --2- 172.21.15.43:0/2860915686 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8d700afc60 0x7f8d700b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:10.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.269+0000 7f8d89d20640 1 --2- 172.21.15.43:0/2860915686 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8d700afc60 0x7f8d700b2170 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8d84103a20 tx=0x7f8d80005f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:10.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.272+0000 7f8d7affd640 1 -- 172.21.15.43:0/2860915686 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d74009de0 con 0x7f8d84108a20 2024-04-25T10:45:10.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.457+0000 7f8d8bf84640 1 -- 172.21.15.43:0/2860915686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8d84106950 con 0x7f8d84108a20 2024-04-25T10:45:10.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.457+0000 7f8d7affd640 1 -- 172.21.15.43:0/2860915686 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f8d740a7330 con 0x7f8d84108a20 2024-04-25T10:45:10.459 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T10:45:10.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.459+0000 7f8d8bf84640 1 -- 172.21.15.43:0/2860915686 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8d700afc60 msgr2=0x7f8d700b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:10.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.459+0000 7f8d8bf84640 1 --2- 172.21.15.43:0/2860915686 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8d700afc60 0x7f8d700b2170 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8d84103a20 tx=0x7f8d80005f90 comp rx=0 tx=0).stop 2024-04-25T10:45:10.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.459+0000 7f8d8bf84640 1 -- 172.21.15.43:0/2860915686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d84108a20 msgr2=0x7f8d8419ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:10.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.459+0000 7f8d8bf84640 1 --2- 172.21.15.43:0/2860915686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d84108a20 0x7f8d8419ba60 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f8d74005d00 tx=0x7f8d7400dde0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.459+0000 7f8d8bf84640 1 -- 172.21.15.43:0/2860915686 shutdown_connections 2024-04-25T10:45:10.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.459+0000 7f8d8bf84640 1 --2- 172.21.15.43:0/2860915686 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f8d700afc60 0x7f8d700b2170 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.459+0000 7f8d8bf84640 1 --2- 172.21.15.43:0/2860915686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d8410d380 0x7f8d8419bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.459+0000 7f8d8bf84640 1 --2- 172.21.15.43:0/2860915686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d84108a20 0x7f8d8419ba60 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.459+0000 7f8d8bf84640 1 --2- 172.21.15.43:0/2860915686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d84102af0 0x7f8d8419b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.459+0000 7f8d8bf84640 1 -- 172.21.15.43:0/2860915686 >> 172.21.15.43:0/2860915686 conn(0x7f8d84078100 msgr2=0x7f8d841040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:10.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.459+0000 7f8d8bf84640 1 -- 172.21.15.43:0/2860915686 shutdown_connections 2024-04-25T10:45:10.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.459+0000 7f8d8bf84640 1 -- 172.21.15.43:0/2860915686 wait complete. 2024-04-25T10:45:10.472 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:45:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.586+0000 7fb5b9ec6640 1 -- 172.21.15.43:0/2471158500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b4108a30 msgr2=0x7fb5b41025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.586+0000 7fb5b9ec6640 1 --2- 172.21.15.43:0/2471158500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b4108a30 0x7fb5b41025c0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4009900 tx=0x7fb5a402a410 comp rx=0 tx=0).stop 2024-04-25T10:45:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.586+0000 7fb5b9ec6640 1 -- 172.21.15.43:0/2471158500 shutdown_connections 2024-04-25T10:45:10.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.586+0000 7fb5b9ec6640 1 --2- 172.21.15.43:0/2471158500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5b410d390 0x7fb5b410f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.586+0000 7fb5b9ec6640 1 --2- 172.21.15.43:0/2471158500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5b4102b00 0x7fb5b4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.586+0000 7fb5b9ec6640 1 --2- 172.21.15.43:0/2471158500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b4108a30 0x7fb5b41025c0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.586+0000 7fb5b9ec6640 1 -- 172.21.15.43:0/2471158500 >> 172.21.15.43:0/2471158500 conn(0x7fb5b40fe260 msgr2=0x7fb5b4100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:10.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.586+0000 7fb5b9ec6640 1 -- 172.21.15.43:0/2471158500 shutdown_connections 2024-04-25T10:45:10.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.586+0000 7fb5b9ec6640 1 -- 172.21.15.43:0/2471158500 wait complete. 2024-04-25T10:45:10.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.586+0000 7fb5b9ec6640 1 Processor -- start 2024-04-25T10:45:10.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.586+0000 7fb5b9ec6640 1 -- start start 2024-04-25T10:45:10.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5b9ec6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5b4102b00 0x7fb5b419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:10.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5b9ec6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5b4108a30 0x7fb5b419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:10.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5b9ec6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b410d390 0x7fb5b419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:10.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5b9ec6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b4112bf0 con 0x7fb5b410d390 2024-04-25T10:45:10.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5b9ec6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b4112a70 con 0x7fb5b4102b00 2024-04-25T10:45:10.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5b9ec6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b4112d70 con 0x7fb5b4108a30 2024-04-25T10:45:10.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b410d390 0x7fb5b419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:10.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b410d390 0x7fb5b419ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36306/0 (socket says 172.21.15.43:36306) 2024-04-25T10:45:10.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5b3fff640 1 -- 172.21.15.43:0/67520751 learned_addr learned my addr 172.21.15.43:0/67520751 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:45:10.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5b37fe640 1 --2- 172.21.15.43:0/67520751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5b4102b00 0x7fb5b419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:10.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5abfff640 1 --2- 172.21.15.43:0/67520751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5b4108a30 0x7fb5b419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:10.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5b3fff640 1 -- 172.21.15.43:0/67520751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5b4108a30 msgr2=0x7fb5b419acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:10.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5b3fff640 1 --2- 172.21.15.43:0/67520751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5b4108a30 0x7fb5b419acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5b3fff640 1 -- 172.21.15.43:0/67520751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5b4102b00 msgr2=0x7fb5b419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:10.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5b3fff640 1 --2- 172.21.15.43:0/67520751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5b4102b00 0x7fb5b419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5b3fff640 1 -- 172.21.15.43:0/67520751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b419f5c0 con 0x7fb5b410d390 2024-04-25T10:45:10.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5b3fff640 1 --2- 172.21.15.43:0/67520751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b410d390 0x7fb5b419ef80 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00eb10 tx=0x7fb5ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:10.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5b17fa640 1 -- 172.21.15.43:0/67520751 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac004b20 con 0x7fb5b410d390 2024-04-25T10:45:10.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5b17fa640 1 -- 172.21.15.43:0/67520751 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5ac004cc0 con 0x7fb5b410d390 2024-04-25T10:45:10.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5b9ec6640 1 -- 172.21.15.43:0/67520751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b419f8b0 con 0x7fb5b410d390 2024-04-25T10:45:10.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.587+0000 7fb5b17fa640 1 -- 172.21.15.43:0/67520751 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac010670 con 0x7fb5b410d390 2024-04-25T10:45:10.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.588+0000 7fb5b9ec6640 1 -- 172.21.15.43:0/67520751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b419fdb0 con 0x7fb5b410d390 2024-04-25T10:45:10.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.588+0000 7fb5b9ec6640 1 -- 172.21.15.43:0/67520751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5b406a160 con 0x7fb5b410d390 2024-04-25T10:45:10.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.590+0000 7fb5b17fa640 1 -- 172.21.15.43:0/67520751 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fb5ac005000 con 0x7fb5b410d390 2024-04-25T10:45:10.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.590+0000 7fb5b17fa640 1 --2- 172.21.15.43:0/67520751 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb5a00afc60 0x7fb5a00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:10.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.591+0000 7fb5b17fa640 1 -- 172.21.15.43:0/67520751 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7fb5ac0df7f0 con 0x7fb5b410d390 2024-04-25T10:45:10.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.591+0000 7fb5b37fe640 1 --2- 172.21.15.43:0/67520751 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb5a00afc60 0x7fb5a00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:10.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.591+0000 7fb5b37fe640 1 --2- 172.21.15.43:0/67520751 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb5a00afc60 0x7fb5a00b2170 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4000c00 tx=0x7fb5a4031400 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:10.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.594+0000 7fb5b17fa640 1 -- 172.21.15.43:0/67520751 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb5ac053ff0 con 0x7fb5b410d390 2024-04-25T10:45:10.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.785+0000 7fb5b9ec6640 1 -- 172.21.15.43:0/67520751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb5b41a7350 con 0x7fb5b410d390 2024-04-25T10:45:10.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.785+0000 7fb5b17fa640 1 -- 172.21.15.43:0/67520751 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fb5b419fdb0 con 0x7fb5b410d390 2024-04-25T10:45:10.787 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T10:45:10.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.787+0000 7fb5b9ec6640 1 -- 172.21.15.43:0/67520751 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb5a00afc60 msgr2=0x7fb5a00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:10.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.787+0000 7fb5b9ec6640 1 --2- 172.21.15.43:0/67520751 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb5a00afc60 0x7fb5a00b2170 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4000c00 tx=0x7fb5a4031400 comp rx=0 tx=0).stop 2024-04-25T10:45:10.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.787+0000 7fb5b9ec6640 1 -- 172.21.15.43:0/67520751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b410d390 msgr2=0x7fb5b419ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:10.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.787+0000 7fb5b9ec6640 1 --2- 172.21.15.43:0/67520751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b410d390 0x7fb5b419ef80 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00eb10 tx=0x7fb5ac00c2d0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.787+0000 7fb5b9ec6640 1 -- 172.21.15.43:0/67520751 shutdown_connections 2024-04-25T10:45:10.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.787+0000 7fb5b9ec6640 1 --2- 172.21.15.43:0/67520751 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fb5a00afc60 0x7fb5a00b2170 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.787+0000 7fb5b9ec6640 1 --2- 172.21.15.43:0/67520751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b410d390 0x7fb5b419ef80 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.787+0000 7fb5b9ec6640 1 --2- 172.21.15.43:0/67520751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5b4108a30 0x7fb5b419acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.787+0000 7fb5b9ec6640 1 --2- 172.21.15.43:0/67520751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5b4102b00 0x7fb5b419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.787+0000 7fb5b9ec6640 1 -- 172.21.15.43:0/67520751 >> 172.21.15.43:0/67520751 conn(0x7fb5b40fe260 msgr2=0x7fb5b40fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:10.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.787+0000 7fb5b9ec6640 1 -- 172.21.15.43:0/67520751 shutdown_connections 2024-04-25T10:45:10.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.787+0000 7fb5b9ec6640 1 -- 172.21.15.43:0/67520751 wait complete. 2024-04-25T10:45:10.801 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:45:10.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.916+0000 7f855a851640 1 -- 172.21.15.43:0/2935311034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8554108a30 msgr2=0x7f85541025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:10.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.916+0000 7f855a851640 1 --2- 172.21.15.43:0/2935311034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8554108a30 0x7f85541025e0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f8544009900 tx=0x7f854402a410 comp rx=0 tx=0).stop 2024-04-25T10:45:10.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.916+0000 7f855a851640 1 -- 172.21.15.43:0/2935311034 shutdown_connections 2024-04-25T10:45:10.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.916+0000 7f855a851640 1 --2- 172.21.15.43:0/2935311034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8554108ed0 0x7f8554102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.916+0000 7f855a851640 1 --2- 172.21.15.43:0/2935311034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8554103190 0x7f855410f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.916+0000 7f855a851640 1 --2- 172.21.15.43:0/2935311034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8554108a30 0x7f85541025e0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.916+0000 7f855a851640 1 -- 172.21.15.43:0/2935311034 >> 172.21.15.43:0/2935311034 conn(0x7f85540fe260 msgr2=0x7f8554100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:10.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.917+0000 7f855a851640 1 -- 172.21.15.43:0/2935311034 shutdown_connections 2024-04-25T10:45:10.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.917+0000 7f855a851640 1 -- 172.21.15.43:0/2935311034 wait complete. 2024-04-25T10:45:10.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.917+0000 7f855a851640 1 Processor -- start 2024-04-25T10:45:10.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.917+0000 7f855a851640 1 -- start start 2024-04-25T10:45:10.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.918+0000 7f855a851640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8554103190 0x7f85541a2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.918+0000 7f855a851640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8554108a30 0x7f85541a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.918+0000 7f855a851640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8554108ed0 0x7f855419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.918+0000 7f855a851640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8554112c30 con 0x7f8554108a30 2024-04-25T10:45:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.918+0000 7f855a851640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8554112ab0 con 0x7f8554108ed0 2024-04-25T10:45:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.918+0000 7f855a851640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8554112db0 con 0x7f8554103190 2024-04-25T10:45:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.918+0000 7f85537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8554108a30 0x7f85541a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.918+0000 7f85537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8554108a30 0x7f85541a2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36324/0 (socket says 172.21.15.43:36324) 2024-04-25T10:45:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.918+0000 7f85537fe640 1 -- 172.21.15.43:0/2312830212 learned_addr learned my addr 172.21.15.43:0/2312830212 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:45:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.918+0000 7f8558dee640 1 --2- 172.21.15.43:0/2312830212 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8554108ed0 0x7f855419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.918+0000 7f8553fff640 1 --2- 172.21.15.43:0/2312830212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8554103190 0x7f85541a2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.918+0000 7f85537fe640 1 -- 172.21.15.43:0/2312830212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8554103190 msgr2=0x7f85541a2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.918+0000 7f85537fe640 1 --2- 172.21.15.43:0/2312830212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8554103190 0x7f85541a2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.918+0000 7f85537fe640 1 -- 172.21.15.43:0/2312830212 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8554108ed0 msgr2=0x7f855419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:10.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.918+0000 7f85537fe640 1 --2- 172.21.15.43:0/2312830212 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8554108ed0 0x7f855419c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:10.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.918+0000 7f85537fe640 1 -- 172.21.15.43:0/2312830212 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855419cd30 con 0x7f8554108a30 2024-04-25T10:45:10.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.918+0000 7f8553fff640 1 --2- 172.21.15.43:0/2312830212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8554103190 0x7f85541a2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:45:10.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.918+0000 7f85537fe640 1 --2- 172.21.15.43:0/2312830212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8554108a30 0x7f85541a2ab0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f854800c970 tx=0x7f854800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:10.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.918+0000 7f85517fa640 1 -- 172.21.15.43:0/2312830212 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8548007c70 con 0x7f8554108a30 2024-04-25T10:45:10.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.918+0000 7f85517fa640 1 -- 172.21.15.43:0/2312830212 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8548007e10 con 0x7f8554108a30 2024-04-25T10:45:10.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.919+0000 7f855a851640 1 -- 172.21.15.43:0/2312830212 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f855419d020 con 0x7f8554108a30 2024-04-25T10:45:10.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.919+0000 7f85517fa640 1 -- 172.21.15.43:0/2312830212 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85480187b0 con 0x7f8554108a30 2024-04-25T10:45:10.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.919+0000 7f855a851640 1 -- 172.21.15.43:0/2312830212 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85541a9310 con 0x7f8554108a30 2024-04-25T10:45:10.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.919+0000 7f855a851640 1 -- 172.21.15.43:0/2312830212 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8554103d20 con 0x7f8554108a30 2024-04-25T10:45:10.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.921+0000 7f85517fa640 1 -- 172.21.15.43:0/2312830212 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f854800fab0 con 0x7f8554108a30 2024-04-25T10:45:10.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.922+0000 7f85517fa640 1 --2- 172.21.15.43:0/2312830212 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f853c0afc60 0x7f853c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:10.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.922+0000 7f8553fff640 1 --2- 172.21.15.43:0/2312830212 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f853c0afc60 0x7f853c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:10.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.922+0000 7f85517fa640 1 -- 172.21.15.43:0/2312830212 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7f85480df290 con 0x7f8554108a30 2024-04-25T10:45:10.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.922+0000 7f8553fff640 1 --2- 172.21.15.43:0/2312830212 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f853c0afc60 0x7f853c0b2170 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8544009230 tx=0x7f8544000bc0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:10.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:10.925+0000 7f85517fa640 1 -- 172.21.15.43:0/2312830212 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8548053ff0 con 0x7f8554108a30 2024-04-25T10:45:11.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:10 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2265286382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:45:11.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:10 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2082298115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:45:11.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:10 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2860915686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:45:11.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:10 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2265286382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:45:11.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:10 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2082298115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:45:11.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:10 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2860915686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:45:11.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:10 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2265286382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:45:11.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:10 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2082298115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T10:45:11.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:10 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2860915686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T10:45:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.126+0000 7f855a851640 1 -- 172.21.15.43:0/2312830212 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f85541a94e0 con 0x7f8554108a30 2024-04-25T10:45:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.126+0000 7f85517fa640 1 -- 172.21.15.43:0/2312830212 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v94) ==== 81+0+945 (secure 0 0 0) 0x7f8548014030 con 0x7f8554108a30 2024-04-25T10:45:11.128 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/1599315071 2024-04-26T10:32:32.456619+0000 2024-04-25T10:45:11.128 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/1953858940 2024-04-26T10:32:32.456619+0000 2024-04-25T10:45:11.128 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/1953858940 2024-04-26T10:32:32.456619+0000 2024-04-25T10:45:11.128 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/4063490531 2024-04-26T10:23:32.072095+0000 2024-04-25T10:45:11.128 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/3026776968 2024-04-26T10:32:32.456619+0000 2024-04-25T10:45:11.128 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/1477725871 2024-04-26T10:23:32.072095+0000 2024-04-25T10:45:11.129 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/2568088261 2024-04-26T10:23:05.121271+0000 2024-04-25T10:45:11.129 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/3381426507 2024-04-26T10:23:05.121271+0000 2024-04-25T10:45:11.129 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/188939647 2024-04-26T10:32:32.456619+0000 2024-04-25T10:45:11.129 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/1017216348 2024-04-26T10:23:32.072095+0000 2024-04-25T10:45:11.129 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/1545240575 2024-04-26T10:23:05.121271+0000 2024-04-25T10:45:11.129 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/2568088261 2024-04-26T10:23:05.121271+0000 2024-04-25T10:45:11.129 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/1287690800 2024-04-26T10:32:32.456619+0000 2024-04-25T10:45:11.129 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/4255994574 2024-04-26T10:23:05.121271+0000 2024-04-25T10:45:11.129 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/1017216348 2024-04-26T10:23:32.072095+0000 2024-04-25T10:45:11.129 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/3837436213 2024-04-26T10:23:32.072095+0000 2024-04-25T10:45:11.130 INFO:teuthology.orchestra.run.smithi043.stderr:listed 16 entries 2024-04-25T10:45:11.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.128+0000 7f855a851640 1 -- 172.21.15.43:0/2312830212 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f853c0afc60 msgr2=0x7f853c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:11.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.128+0000 7f855a851640 1 --2- 172.21.15.43:0/2312830212 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f853c0afc60 0x7f853c0b2170 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8544009230 tx=0x7f8544000bc0 comp rx=0 tx=0).stop 2024-04-25T10:45:11.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.128+0000 7f855a851640 1 -- 172.21.15.43:0/2312830212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8554108a30 msgr2=0x7f85541a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:11.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.128+0000 7f855a851640 1 --2- 172.21.15.43:0/2312830212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8554108a30 0x7f85541a2ab0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f854800c970 tx=0x7f854800ce30 comp rx=0 tx=0).stop 2024-04-25T10:45:11.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.128+0000 7f855a851640 1 -- 172.21.15.43:0/2312830212 shutdown_connections 2024-04-25T10:45:11.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.128+0000 7f855a851640 1 --2- 172.21.15.43:0/2312830212 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f853c0afc60 0x7f853c0b2170 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:11.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.128+0000 7f855a851640 1 --2- 172.21.15.43:0/2312830212 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8554108ed0 0x7f855419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:11.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.128+0000 7f855a851640 1 --2- 172.21.15.43:0/2312830212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8554108a30 0x7f85541a2ab0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:11.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.128+0000 7f855a851640 1 --2- 172.21.15.43:0/2312830212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8554103190 0x7f85541a2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:11.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.128+0000 7f855a851640 1 -- 172.21.15.43:0/2312830212 >> 172.21.15.43:0/2312830212 conn(0x7f85540fe260 msgr2=0x7f85540fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:11.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.128+0000 7f855a851640 1 -- 172.21.15.43:0/2312830212 shutdown_connections 2024-04-25T10:45:11.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.128+0000 7f855a851640 1 -- 172.21.15.43:0/2312830212 wait complete. 2024-04-25T10:45:11.141 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:45:11.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:45:11.142 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:45:11.189 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T10:45:11.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:45:11.189 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:45:11.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:45:11.226 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-25T10:45:11.282 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:45:11.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:45:11.282 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:45:11.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:45:11.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-04-25T10:45:11.449 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T10:45:11.462 INFO:tasks.cephadm:Teardown begin 2024-04-25T10:45:11.462 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:45:11.463 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:45:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.574+0000 7f227e7ce640 1 -- 172.21.15.43:0/1700061637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278104c20 msgr2=0x7f2278105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.574+0000 7f227e7ce640 1 --2- 172.21.15.43:0/1700061637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278104c20 0x7f2278105030 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f2260009900 tx=0x7f226002a480 comp rx=0 tx=0).stop 2024-04-25T10:45:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.574+0000 7f227e7ce640 1 -- 172.21.15.43:0/1700061637 shutdown_connections 2024-04-25T10:45:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.574+0000 7f227e7ce640 1 --2- 172.21.15.43:0/1700061637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f227810e070 0x7f2278110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.574+0000 7f227e7ce640 1 --2- 172.21.15.43:0/1700061637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f227810b680 0x7f227810daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:11.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.574+0000 7f227e7ce640 1 --2- 172.21.15.43:0/1700061637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278104c20 0x7f2278105030 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:11.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.574+0000 7f227e7ce640 1 -- 172.21.15.43:0/1700061637 >> 172.21.15.43:0/1700061637 conn(0x7f2278078040 msgr2=0x7f2278078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:11.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.574+0000 7f227e7ce640 1 -- 172.21.15.43:0/1700061637 shutdown_connections 2024-04-25T10:45:11.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.574+0000 7f227e7ce640 1 -- 172.21.15.43:0/1700061637 wait complete. 2024-04-25T10:45:11.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.575+0000 7f227e7ce640 1 Processor -- start 2024-04-25T10:45:11.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.575+0000 7f227e7ce640 1 -- start start 2024-04-25T10:45:11.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.575+0000 7f227e7ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278104c20 0x7f227819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:11.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.575+0000 7f227e7ce640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f227810b680 0x7f227819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:11.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.575+0000 7f227e7ce640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f227810e070 0x7f227819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:11.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.575+0000 7f227e7ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2278113cd0 con 0x7f2278104c20 2024-04-25T10:45:11.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.575+0000 7f227e7ce640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2278113b50 con 0x7f227810b680 2024-04-25T10:45:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.575+0000 7f227e7ce640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2278113e50 con 0x7f227810e070 2024-04-25T10:45:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.575+0000 7f2277fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278104c20 0x7f227819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.575+0000 7f2277fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278104c20 0x7f227819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36346/0 (socket says 172.21.15.43:36346) 2024-04-25T10:45:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.575+0000 7f2277fff640 1 -- 172.21.15.43:0/3527716046 learned_addr learned my addr 172.21.15.43:0/3527716046 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:45:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.575+0000 7f22777fe640 1 --2- 172.21.15.43:0/3527716046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f227810b680 0x7f227819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.575+0000 7f227cd6b640 1 --2- 172.21.15.43:0/3527716046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f227810e070 0x7f227819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.576+0000 7f2277fff640 1 -- 172.21.15.43:0/3527716046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f227810e070 msgr2=0x7f227819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.576+0000 7f2277fff640 1 --2- 172.21.15.43:0/3527716046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f227810e070 0x7f227819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.576+0000 7f2277fff640 1 -- 172.21.15.43:0/3527716046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f227810b680 msgr2=0x7f227819acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.576+0000 7f2277fff640 1 --2- 172.21.15.43:0/3527716046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f227810b680 0x7f227819acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.576+0000 7f2277fff640 1 -- 172.21.15.43:0/3527716046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227819f5b0 con 0x7f2278104c20 2024-04-25T10:45:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.576+0000 7f22777fe640 1 --2- 172.21.15.43:0/3527716046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f227810b680 0x7f227819acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:45:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.576+0000 7f227cd6b640 1 --2- 172.21.15.43:0/3527716046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f227810e070 0x7f227819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:45:11.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.576+0000 7f2277fff640 1 --2- 172.21.15.43:0/3527716046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278104c20 0x7f227819a750 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f2260005940 tx=0x7f22600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:11.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.576+0000 7f22757fa640 1 -- 172.21.15.43:0/3527716046 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226003f030 con 0x7f2278104c20 2024-04-25T10:45:11.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.576+0000 7f227e7ce640 1 -- 172.21.15.43:0/3527716046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f227819f840 con 0x7f2278104c20 2024-04-25T10:45:11.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.576+0000 7f22757fa640 1 -- 172.21.15.43:0/3527716046 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2260003cb0 con 0x7f2278104c20 2024-04-25T10:45:11.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.576+0000 7f22757fa640 1 -- 172.21.15.43:0/3527716046 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2260035070 con 0x7f2278104c20 2024-04-25T10:45:11.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.576+0000 7f227e7ce640 1 -- 172.21.15.43:0/3527716046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f227819fd30 con 0x7f2278104c20 2024-04-25T10:45:11.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.577+0000 7f227e7ce640 1 -- 172.21.15.43:0/3527716046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f227806a190 con 0x7f2278104c20 2024-04-25T10:45:11.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.579+0000 7f22757fa640 1 -- 172.21.15.43:0/3527716046 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f2260042030 con 0x7f2278104c20 2024-04-25T10:45:11.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.579+0000 7f22757fa640 1 --2- 172.21.15.43:0/3527716046 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f22640afc60 0x7f22640b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:11.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.580+0000 7f22777fe640 1 --2- 172.21.15.43:0/3527716046 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f22640afc60 0x7f22640b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:11.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.580+0000 7f22757fa640 1 -- 172.21.15.43:0/3527716046 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7f22600ffc50 con 0x7f2278104c20 2024-04-25T10:45:11.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.580+0000 7f22777fe640 1 --2- 172.21.15.43:0/3527716046 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f22640afc60 0x7f22640b2170 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f227810e9b0 tx=0x7f2268005e50 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:11.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.583+0000 7f22757fa640 1 -- 172.21.15.43:0/3527716046 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f22600c84d0 con 0x7f2278104c20 2024-04-25T10:45:11.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.808+0000 7f227e7ce640 1 -- 172.21.15.43:0/3527716046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22781a7280 con 0x7f2278104c20 2024-04-25T10:45:11.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.809+0000 7f22757fa640 1 -- 172.21.15.43:0/3527716046 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7f22600c7dd0 con 0x7f2278104c20 2024-04-25T10:45:11.811 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:45:11.811 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:45:11.811 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:45:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.811+0000 7f227e7ce640 1 -- 172.21.15.43:0/3527716046 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f22640afc60 msgr2=0x7f22640b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.811+0000 7f227e7ce640 1 --2- 172.21.15.43:0/3527716046 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f22640afc60 0x7f22640b2170 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f227810e9b0 tx=0x7f2268005e50 comp rx=0 tx=0).stop 2024-04-25T10:45:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.811+0000 7f227e7ce640 1 -- 172.21.15.43:0/3527716046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278104c20 msgr2=0x7f227819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.811+0000 7f227e7ce640 1 --2- 172.21.15.43:0/3527716046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278104c20 0x7f227819a750 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f2260005940 tx=0x7f22600042f0 comp rx=0 tx=0).stop 2024-04-25T10:45:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.811+0000 7f227e7ce640 1 -- 172.21.15.43:0/3527716046 shutdown_connections 2024-04-25T10:45:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.811+0000 7f227e7ce640 1 --2- 172.21.15.43:0/3527716046 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f22640afc60 0x7f22640b2170 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.811+0000 7f227e7ce640 1 --2- 172.21.15.43:0/3527716046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f227810b680 0x7f227819acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.811+0000 7f227e7ce640 1 --2- 172.21.15.43:0/3527716046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f227810e070 0x7f227819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.811+0000 7f227e7ce640 1 --2- 172.21.15.43:0/3527716046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278104c20 0x7f227819a750 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.811+0000 7f227e7ce640 1 -- 172.21.15.43:0/3527716046 >> 172.21.15.43:0/3527716046 conn(0x7f2278078040 msgr2=0x7f2278101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.811+0000 7f227e7ce640 1 -- 172.21.15.43:0/3527716046 shutdown_connections 2024-04-25T10:45:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.811+0000 7f227e7ce640 1 -- 172.21.15.43:0/3527716046 wait complete. 2024-04-25T10:45:11.825 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:45:11.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.935+0000 7f4b2d8ee640 1 -- 172.21.15.43:0/2268069409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b28102b60 msgr2=0x7f4b28102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:11.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.935+0000 7f4b2d8ee640 1 --2- 172.21.15.43:0/2268069409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b28102b60 0x7f4b28102f70 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009900 tx=0x7f4b1002a410 comp rx=0 tx=0).stop 2024-04-25T10:45:11.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.936+0000 7f4b2d8ee640 1 -- 172.21.15.43:0/2268069409 shutdown_connections 2024-04-25T10:45:11.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.936+0000 7f4b2d8ee640 1 --2- 172.21.15.43:0/2268069409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b2810d3a0 0x7f4b2810f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:11.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.936+0000 7f4b2d8ee640 1 --2- 172.21.15.43:0/2268069409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b28102b60 0x7f4b28102f70 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:11.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.936+0000 7f4b2d8ee640 1 --2- 172.21.15.43:0/2268069409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b28108a40 0x7f4b28102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:11.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.936+0000 7f4b2d8ee640 1 -- 172.21.15.43:0/2268069409 >> 172.21.15.43:0/2268069409 conn(0x7f4b28078040 msgr2=0x7f4b28078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:11.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.936+0000 7f4b2d8ee640 1 -- 172.21.15.43:0/2268069409 shutdown_connections 2024-04-25T10:45:11.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.936+0000 7f4b2d8ee640 1 -- 172.21.15.43:0/2268069409 wait complete. 2024-04-25T10:45:11.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.936+0000 7f4b2d8ee640 1 Processor -- start 2024-04-25T10:45:11.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.936+0000 7f4b2d8ee640 1 -- start start 2024-04-25T10:45:11.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.937+0000 7f4b2d8ee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b28102b60 0x7f4b2819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:11.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.937+0000 7f4b2d8ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b28108a40 0x7f4b2819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:11.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.937+0000 7f4b2d8ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b2810d3a0 0x7f4b2819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:11.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.937+0000 7f4b2d8ee640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28112c00 con 0x7f4b28108a40 2024-04-25T10:45:11.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.937+0000 7f4b2d8ee640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28112a80 con 0x7f4b2810d3a0 2024-04-25T10:45:11.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.937+0000 7f4b2d8ee640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28112d80 con 0x7f4b28102b60 2024-04-25T10:45:11.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.937+0000 7f4b1ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b28108a40 0x7f4b2819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:11.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.937+0000 7f4b1ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b28108a40 0x7f4b2819ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36360/0 (socket says 172.21.15.43:36360) 2024-04-25T10:45:11.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.937+0000 7f4b1ffff640 1 -- 172.21.15.43:0/604146481 learned_addr learned my addr 172.21.15.43:0/604146481 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:45:11.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.937+0000 7f4b26ffd640 1 --2- 172.21.15.43:0/604146481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b28102b60 0x7f4b2819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:11.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.937+0000 7f4b277fe640 1 --2- 172.21.15.43:0/604146481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b2810d3a0 0x7f4b2819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:11.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.937+0000 7f4b1ffff640 1 -- 172.21.15.43:0/604146481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b28102b60 msgr2=0x7f4b2819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:11.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.937+0000 7f4b1ffff640 1 --2- 172.21.15.43:0/604146481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b28102b60 0x7f4b2819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:11.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.937+0000 7f4b1ffff640 1 -- 172.21.15.43:0/604146481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b2810d3a0 msgr2=0x7f4b2819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:11.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.937+0000 7f4b1ffff640 1 --2- 172.21.15.43:0/604146481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b2810d3a0 0x7f4b2819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:11.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.937+0000 7f4b1ffff640 1 -- 172.21.15.43:0/604146481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2819f610 con 0x7f4b28108a40 2024-04-25T10:45:11.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.937+0000 7f4b1ffff640 1 --2- 172.21.15.43:0/604146481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b28108a40 0x7f4b2819ad10 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f4b1002a930 tx=0x7f4b10031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:11.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.938+0000 7f4b2c8ec640 1 -- 172.21.15.43:0/604146481 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1003f030 con 0x7f4b28108a40 2024-04-25T10:45:11.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.938+0000 7f4b2c8ec640 1 -- 172.21.15.43:0/604146481 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b10004490 con 0x7f4b28108a40 2024-04-25T10:45:11.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.938+0000 7f4b2c8ec640 1 -- 172.21.15.43:0/604146481 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10035070 con 0x7f4b28108a40 2024-04-25T10:45:11.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.938+0000 7f4b2d8ee640 1 -- 172.21.15.43:0/604146481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b2819f8a0 con 0x7f4b28108a40 2024-04-25T10:45:11.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.938+0000 7f4b2d8ee640 1 -- 172.21.15.43:0/604146481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b2819fd90 con 0x7f4b28108a40 2024-04-25T10:45:11.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.939+0000 7f4b2d8ee640 1 -- 172.21.15.43:0/604146481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b2806a190 con 0x7f4b28108a40 2024-04-25T10:45:11.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.940+0000 7f4b2c8ec640 1 -- 172.21.15.43:0/604146481 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f4b10042030 con 0x7f4b28108a40 2024-04-25T10:45:11.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.941+0000 7f4b2c8ec640 1 --2- 172.21.15.43:0/604146481 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f4b140afc60 0x7f4b140b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:11.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.941+0000 7f4b26ffd640 1 --2- 172.21.15.43:0/604146481 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f4b140afc60 0x7f4b140b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:11.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.941+0000 7f4b2c8ec640 1 -- 172.21.15.43:0/604146481 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7f4b100ff730 con 0x7f4b28108a40 2024-04-25T10:45:11.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.941+0000 7f4b26ffd640 1 --2- 172.21.15.43:0/604146481 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f4b140afc60 0x7f4b140b2170 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4b28103a40 tx=0x7f4b18005f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:11.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:11.944+0000 7f4b2c8ec640 1 -- 172.21.15.43:0/604146481 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b100c7960 con 0x7f4b28108a40 2024-04-25T10:45:12.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:11 smithi077 ceph-mon[32772]: pgmap v396: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 54 KiB/s wr, 14 op/s 2024-04-25T10:45:12.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:11 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/67520751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:45:12.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:11 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2312830212' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T10:45:12.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:11 smithi164 ceph-mon[32713]: pgmap v396: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 54 KiB/s wr, 14 op/s 2024-04-25T10:45:12.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:11 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/67520751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:45:12.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:11 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2312830212' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T10:45:12.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:11 smithi043 ceph-mon[29478]: pgmap v396: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 54 KiB/s wr, 14 op/s 2024-04-25T10:45:12.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:11 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/67520751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T10:45:12.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:11 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2312830212' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T10:45:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.171+0000 7f4b2d8ee640 1 -- 172.21.15.43:0/604146481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b281a72e0 con 0x7f4b28108a40 2024-04-25T10:45:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.172+0000 7f4b2c8ec640 1 -- 172.21.15.43:0/604146481 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7f4b100cbc60 con 0x7f4b28108a40 2024-04-25T10:45:12.174 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:45:12.174 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3551971560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3551971560},{"type":"v1","addr":"172.21.15.43:6835","nonce":3551971560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2500994561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":2500994561},{"type":"v1","addr":"172.21.15.43:6837","nonce":2500994561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3923057448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3923057448},{"type":"v1","addr":"172.21.15.43:6839","nonce":3923057448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/4254265277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":4254265277},{"type":"v1","addr":"172.21.15.43:6841","nonce":4254265277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2295020829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2295020829},{"type":"v1","addr":"172.21.15.77:6833","nonce":2295020829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3601949896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3601949896},{"type":"v1","addr":"172.21.15.164:6833","nonce":3601949896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:30:33.261560+0000","modified":"2024-04-25T10:34:15.012113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24508,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/1783762538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1783762538},{"type":"v1","addr":"172.21.15.77:6835","nonce":1783762538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6835/1437008220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1437008220},{"type":"v1","addr":"172.21.15.164:6835","nonce":1437008220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3305330947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3305330947},{"type":"v1","addr":"172.21.15.164:6837","nonce":3305330947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/4082280107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":4082280107},{"type":"v1","addr":"172.21.15.164:6839","nonce":4082280107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/172043982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":172043982},{"type":"v1","addr":"172.21.15.77:6839","nonce":172043982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25592":{"gid":25592,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/3696941757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3696941757},{"type":"v1","addr":"172.21.15.77:6837","nonce":3696941757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24472,24508,24554]},"id":1}]} 2024-04-25T10:45:12.174 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-04-25T10:45:12.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.174+0000 7f4b2d8ee640 1 -- 172.21.15.43:0/604146481 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f4b140afc60 msgr2=0x7f4b140b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:12.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.174+0000 7f4b2d8ee640 1 --2- 172.21.15.43:0/604146481 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f4b140afc60 0x7f4b140b2170 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4b28103a40 tx=0x7f4b18005f90 comp rx=0 tx=0).stop 2024-04-25T10:45:12.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.174+0000 7f4b2d8ee640 1 -- 172.21.15.43:0/604146481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b28108a40 msgr2=0x7f4b2819ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:12.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.174+0000 7f4b2d8ee640 1 --2- 172.21.15.43:0/604146481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b28108a40 0x7f4b2819ad10 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f4b1002a930 tx=0x7f4b10031a20 comp rx=0 tx=0).stop 2024-04-25T10:45:12.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.174+0000 7f4b2d8ee640 1 -- 172.21.15.43:0/604146481 shutdown_connections 2024-04-25T10:45:12.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.174+0000 7f4b2d8ee640 1 --2- 172.21.15.43:0/604146481 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f4b140afc60 0x7f4b140b2170 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.174+0000 7f4b2d8ee640 1 --2- 172.21.15.43:0/604146481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b2810d3a0 0x7f4b2819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.174+0000 7f4b2d8ee640 1 --2- 172.21.15.43:0/604146481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b28108a40 0x7f4b2819ad10 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.174+0000 7f4b2d8ee640 1 --2- 172.21.15.43:0/604146481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b28102b60 0x7f4b2819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.174+0000 7f4b2d8ee640 1 -- 172.21.15.43:0/604146481 >> 172.21.15.43:0/604146481 conn(0x7f4b28078040 msgr2=0x7f4b28074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:12.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.174+0000 7f4b2d8ee640 1 -- 172.21.15.43:0/604146481 shutdown_connections 2024-04-25T10:45:12.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.174+0000 7f4b2d8ee640 1 -- 172.21.15.43:0/604146481 wait complete. 2024-04-25T10:45:12.188 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:45:12.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.298+0000 7f301e300640 1 -- 172.21.15.43:0/3855016583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3018103190 msgr2=0x7f301810f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:12.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.298+0000 7f301e300640 1 --2- 172.21.15.43:0/3855016583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3018103190 0x7f301810f4c0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f3010009ca0 tx=0x7f301002a8c0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.298+0000 7f301e300640 1 -- 172.21.15.43:0/3855016583 shutdown_connections 2024-04-25T10:45:12.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.298+0000 7f301e300640 1 --2- 172.21.15.43:0/3855016583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3018108ed0 0x7f3018102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.298+0000 7f301e300640 1 --2- 172.21.15.43:0/3855016583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3018103190 0x7f301810f4c0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.298+0000 7f301e300640 1 --2- 172.21.15.43:0/3855016583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3018108a30 0x7f30181025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.298+0000 7f301e300640 1 -- 172.21.15.43:0/3855016583 >> 172.21.15.43:0/3855016583 conn(0x7f3018078180 msgr2=0x7f30180785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:12.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.298+0000 7f301e300640 1 -- 172.21.15.43:0/3855016583 shutdown_connections 2024-04-25T10:45:12.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.298+0000 7f301e300640 1 -- 172.21.15.43:0/3855016583 wait complete. 2024-04-25T10:45:12.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.298+0000 7f301e300640 1 Processor -- start 2024-04-25T10:45:12.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.299+0000 7f301e300640 1 -- start start 2024-04-25T10:45:12.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.299+0000 7f301e300640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3018103190 0x7f30181995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:12.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.299+0000 7f301e300640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3018108a30 0x7f3018199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:12.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.299+0000 7f301e300640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3018108ed0 0x7f301819ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:12.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.299+0000 7f301e300640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3018112c70 con 0x7f3018108a30 2024-04-25T10:45:12.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.299+0000 7f301e300640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3018112af0 con 0x7f3018108ed0 2024-04-25T10:45:12.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.299+0000 7f301e300640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3018112df0 con 0x7f3018103190 2024-04-25T10:45:12.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.299+0000 7f30177fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3018108a30 0x7f3018199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:12.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.299+0000 7f30177fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3018108a30 0x7f3018199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36380/0 (socket says 172.21.15.43:36380) 2024-04-25T10:45:12.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.299+0000 7f30177fe640 1 -- 172.21.15.43:0/3621594904 learned_addr learned my addr 172.21.15.43:0/3621594904 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:45:12.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.299+0000 7f3017fff640 1 --2- 172.21.15.43:0/3621594904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3018103190 0x7f30181995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:12.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.299+0000 7f301c89d640 1 --2- 172.21.15.43:0/3621594904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3018108ed0 0x7f301819ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:12.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.299+0000 7f30177fe640 1 -- 172.21.15.43:0/3621594904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3018103190 msgr2=0x7f30181995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:12.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.299+0000 7f30177fe640 1 --2- 172.21.15.43:0/3621594904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3018103190 0x7f30181995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.299+0000 7f30177fe640 1 -- 172.21.15.43:0/3621594904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3018108ed0 msgr2=0x7f301819ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:12.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.299+0000 7f30177fe640 1 --2- 172.21.15.43:0/3621594904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3018108ed0 0x7f301819ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.299+0000 7f30177fe640 1 -- 172.21.15.43:0/3621594904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301819e410 con 0x7f3018108a30 2024-04-25T10:45:12.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.300+0000 7f301c89d640 1 --2- 172.21.15.43:0/3621594904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3018108ed0 0x7f301819ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:45:12.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.300+0000 7f30177fe640 1 --2- 172.21.15.43:0/3621594904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3018108a30 0x7f3018199b10 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f300800e9f0 tx=0x7f300800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:12.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.300+0000 7f30157fa640 1 -- 172.21.15.43:0/3621594904 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300800ccc0 con 0x7f3018108a30 2024-04-25T10:45:12.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.300+0000 7f30157fa640 1 -- 172.21.15.43:0/3621594904 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30080035f0 con 0x7f3018108a30 2024-04-25T10:45:12.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.300+0000 7f301e300640 1 -- 172.21.15.43:0/3621594904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f301819e700 con 0x7f3018108a30 2024-04-25T10:45:12.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.300+0000 7f30157fa640 1 -- 172.21.15.43:0/3621594904 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300800ce60 con 0x7f3018108a30 2024-04-25T10:45:12.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.300+0000 7f301e300640 1 -- 172.21.15.43:0/3621594904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30181a6080 con 0x7f3018108a30 2024-04-25T10:45:12.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.301+0000 7f301e300640 1 -- 172.21.15.43:0/3621594904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f301806a190 con 0x7f3018108a30 2024-04-25T10:45:12.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.302+0000 7f30157fa640 1 -- 172.21.15.43:0/3621594904 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f3008003aa0 con 0x7f3018108a30 2024-04-25T10:45:12.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.303+0000 7f30157fa640 1 --2- 172.21.15.43:0/3621594904 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f30040afc60 0x7f30040b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:12.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.303+0000 7f3017fff640 1 --2- 172.21.15.43:0/3621594904 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f30040afc60 0x7f30040b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:12.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.303+0000 7f30157fa640 1 -- 172.21.15.43:0/3621594904 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7f30080df4b0 con 0x7f3018108a30 2024-04-25T10:45:12.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.304+0000 7f3017fff640 1 --2- 172.21.15.43:0/3621594904 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f30040afc60 0x7f30040b2170 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f301806c470 tx=0x7f3000005f90 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.306+0000 7f30157fa640 1 -- 172.21.15.43:0/3621594904 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f30080a7d30 con 0x7f3018108a30 2024-04-25T10:45:12.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.493+0000 7f301e300640 1 -- 172.21.15.43:0/3621594904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f301802db50 con 0x7f3018108a30 2024-04-25T10:45:12.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.495+0000 7f30157fa640 1 -- 172.21.15.43:0/3621594904 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+23153 (secure 0 0 0) 0x7f30080a7630 con 0x7f3018108a30 2024-04-25T10:45:12.497 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T10:45:12.497 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":94,"fsid":"94d0035c-02ed-11ef-bc93-c7b262605968","created":"2024-04-25T10:22:37.861853+0000","modified":"2024-04-25T10:45:08.758017+0000","last_up_change":"2024-04-25T10:29:43.215939+0000","last_in_change":"2024-04-25T10:29:23.443720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:25:58.167476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:31.503414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:32.224334+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":94,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05115646-db4d-4cfc-a5df-867435ee79ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6803","nonce":3598858946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6805","nonce":3598858946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6809","nonce":3598858946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3598858946},{"type":"v1","addr":"172.21.15.43:6807","nonce":3598858946}]},"public_addr":"172.21.15.43:6803/3598858946","cluster_addr":"172.21.15.43:6805/3598858946","heartbeat_back_addr":"172.21.15.43:6809/3598858946","heartbeat_front_addr":"172.21.15.43:6807/3598858946","state":["exists","up"]},{"osd":1,"uuid":"a8e23410-bf31-4a5e-a7dc-d99673208270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6801","nonce":3362129259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6803","nonce":3362129259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6807","nonce":3362129259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3362129259},{"type":"v1","addr":"172.21.15.77:6805","nonce":3362129259}]},"public_addr":"172.21.15.77:6801/3362129259","cluster_addr":"172.21.15.77:6803/3362129259","heartbeat_back_addr":"172.21.15.77:6807/3362129259","heartbeat_front_addr":"172.21.15.77:6805/3362129259","state":["exists","up"]},{"osd":2,"uuid":"e85413f8-524b-41ed-8ab6-d811d2ed40ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6801","nonce":3408162352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6803","nonce":3408162352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6807","nonce":3408162352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3408162352},{"type":"v1","addr":"172.21.15.164:6805","nonce":3408162352}]},"public_addr":"172.21.15.164:6801/3408162352","cluster_addr":"172.21.15.164:6803/3408162352","heartbeat_back_addr":"172.21.15.164:6807/3408162352","heartbeat_front_addr":"172.21.15.164:6805/3408162352","state":["exists","up"]},{"osd":3,"uuid":"584d77ca-fb6f-4b35-9a11-2d2827de6dc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6811","nonce":2368600010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6813","nonce":2368600010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6817","nonce":2368600010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":2368600010},{"type":"v1","addr":"172.21.15.43:6815","nonce":2368600010}]},"public_addr":"172.21.15.43:6811/2368600010","cluster_addr":"172.21.15.43:6813/2368600010","heartbeat_back_addr":"172.21.15.43:6817/2368600010","heartbeat_front_addr":"172.21.15.43:6815/2368600010","state":["exists","up"]},{"osd":4,"uuid":"8cf2e39e-255f-4410-aee3-938b0e19dff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6809","nonce":3648991803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6811","nonce":3648991803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6815","nonce":3648991803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3648991803},{"type":"v1","addr":"172.21.15.77:6813","nonce":3648991803}]},"public_addr":"172.21.15.77:6809/3648991803","cluster_addr":"172.21.15.77:6811/3648991803","heartbeat_back_addr":"172.21.15.77:6815/3648991803","heartbeat_front_addr":"172.21.15.77:6813/3648991803","state":["exists","up"]},{"osd":5,"uuid":"cb7aa97e-5dde-4866-9c74-547e4ebdc619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6809","nonce":853209434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6811","nonce":853209434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6815","nonce":853209434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":853209434},{"type":"v1","addr":"172.21.15.164:6813","nonce":853209434}]},"public_addr":"172.21.15.164:6809/853209434","cluster_addr":"172.21.15.164:6811/853209434","heartbeat_back_addr":"172.21.15.164:6815/853209434","heartbeat_front_addr":"172.21.15.164:6813/853209434","state":["exists","up"]},{"osd":6,"uuid":"1e8aba2c-984b-4570-8900-dd88f3e5774a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6819","nonce":1494696172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6821","nonce":1494696172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6825","nonce":1494696172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1494696172},{"type":"v1","addr":"172.21.15.43:6823","nonce":1494696172}]},"public_addr":"172.21.15.43:6819/1494696172","cluster_addr":"172.21.15.43:6821/1494696172","heartbeat_back_addr":"172.21.15.43:6825/1494696172","heartbeat_front_addr":"172.21.15.43:6823/1494696172","state":["exists","up"]},{"osd":7,"uuid":"e0d47c22-d7bb-4566-802b-bca38d24b04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6817","nonce":1331976864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6819","nonce":1331976864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6823","nonce":1331976864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1331976864},{"type":"v1","addr":"172.21.15.77:6821","nonce":1331976864}]},"public_addr":"172.21.15.77:6817/1331976864","cluster_addr":"172.21.15.77:6819/1331976864","heartbeat_back_addr":"172.21.15.77:6823/1331976864","heartbeat_front_addr":"172.21.15.77:6821/1331976864","state":["exists","up"]},{"osd":8,"uuid":"29f691f2-fb63-4c00-9d78-3f326bd72b3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6817","nonce":3396063471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6819","nonce":3396063471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6823","nonce":3396063471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3396063471},{"type":"v1","addr":"172.21.15.164:6821","nonce":3396063471}]},"public_addr":"172.21.15.164:6817/3396063471","cluster_addr":"172.21.15.164:6819/3396063471","heartbeat_back_addr":"172.21.15.164:6823/3396063471","heartbeat_front_addr":"172.21.15.164:6821/3396063471","state":["exists","up"]},{"osd":9,"uuid":"a77c9906-5352-4b5d-b18b-22d586109892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6827","nonce":537180413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6829","nonce":537180413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6833","nonce":537180413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":537180413},{"type":"v1","addr":"172.21.15.43:6831","nonce":537180413}]},"public_addr":"172.21.15.43:6827/537180413","cluster_addr":"172.21.15.43:6829/537180413","heartbeat_back_addr":"172.21.15.43:6833/537180413","heartbeat_front_addr":"172.21.15.43:6831/537180413","state":["exists","up"]},{"osd":10,"uuid":"f728b6e7-cb77-4883-8b80-152cfa4a712c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6825","nonce":1788538313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6827","nonce":1788538313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6831","nonce":1788538313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1788538313},{"type":"v1","addr":"172.21.15.77:6829","nonce":1788538313}]},"public_addr":"172.21.15.77:6825/1788538313","cluster_addr":"172.21.15.77:6827/1788538313","heartbeat_back_addr":"172.21.15.77:6831/1788538313","heartbeat_front_addr":"172.21.15.77:6829/1788538313","state":["exists","up"]},{"osd":11,"uuid":"f6b51b3f-75f3-4864-bfd9-fe42814e8bfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6825","nonce":1085943743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6827","nonce":1085943743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6831","nonce":1085943743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1085943743},{"type":"v1","addr":"172.21.15.164:6829","nonce":1085943743}]},"public_addr":"172.21.15.164:6825/1085943743","cluster_addr":"172.21.15.164:6827/1085943743","heartbeat_back_addr":"172.21.15.164:6831/1085943743","heartbeat_front_addr":"172.21.15.164:6829/1085943743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:06.301707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:30.950288+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:25:55.423080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:20.183984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:26:45.643445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:10.320995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:27:35.554726+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:00.592706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:25.047206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:50.457046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:16.236763+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:29:41.067391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1599315071":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6801/1953858940":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6800/1953858940":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/4063490531":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3026776968":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/1477725871":"2024-04-26T10:23:32.072095+0000","172.21.15.43:6800/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/3381426507":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/188939647":"2024-04-26T10:32:32.456619+0000","172.21.15.43:6800/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/1545240575":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/2568088261":"2024-04-26T10:23:05.121271+0000","172.21.15.43:0/1287690800":"2024-04-26T10:32:32.456619+0000","172.21.15.43:0/4255994574":"2024-04-26T10:23:05.121271+0000","172.21.15.43:6801/1017216348":"2024-04-26T10:23:32.072095+0000","172.21.15.43:0/3837436213":"2024-04-26T10:23:32.072095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:45:12.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.497+0000 7f301e300640 1 -- 172.21.15.43:0/3621594904 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f30040afc60 msgr2=0x7f30040b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:12.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.497+0000 7f301e300640 1 --2- 172.21.15.43:0/3621594904 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f30040afc60 0x7f30040b2170 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f301806c470 tx=0x7f3000005f90 comp rx=0 tx=0).stop 2024-04-25T10:45:12.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.497+0000 7f301e300640 1 -- 172.21.15.43:0/3621594904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3018108a30 msgr2=0x7f3018199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:12.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.497+0000 7f301e300640 1 --2- 172.21.15.43:0/3621594904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3018108a30 0x7f3018199b10 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f300800e9f0 tx=0x7f300800eeb0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.497+0000 7f301e300640 1 -- 172.21.15.43:0/3621594904 shutdown_connections 2024-04-25T10:45:12.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.497+0000 7f301e300640 1 --2- 172.21.15.43:0/3621594904 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f30040afc60 0x7f30040b2170 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.497+0000 7f301e300640 1 --2- 172.21.15.43:0/3621594904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3018108ed0 0x7f301819ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.497+0000 7f301e300640 1 --2- 172.21.15.43:0/3621594904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3018108a30 0x7f3018199b10 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.497+0000 7f301e300640 1 --2- 172.21.15.43:0/3621594904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3018103190 0x7f30181995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.497+0000 7f301e300640 1 -- 172.21.15.43:0/3621594904 >> 172.21.15.43:0/3621594904 conn(0x7f3018078180 msgr2=0x7f3018100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:12.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.497+0000 7f301e300640 1 -- 172.21.15.43:0/3621594904 shutdown_connections 2024-04-25T10:45:12.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.497+0000 7f301e300640 1 -- 172.21.15.43:0/3621594904 wait complete. 2024-04-25T10:45:12.511 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:45:12.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.622+0000 7fdebb335640 1 -- 172.21.15.43:0/3784999661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdeb4109b10 msgr2=0x7fdeb4101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:12.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.622+0000 7fdebb335640 1 --2- 172.21.15.43:0/3784999661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdeb4109b10 0x7fdeb4101bc0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0009900 tx=0x7fdeb002a410 comp rx=0 tx=0).stop 2024-04-25T10:45:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.623+0000 7fdebb335640 1 -- 172.21.15.43:0/3784999661 shutdown_connections 2024-04-25T10:45:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.623+0000 7fdebb335640 1 --2- 172.21.15.43:0/3784999661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdeb41027b0 0x7fdeb4110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.623+0000 7fdebb335640 1 --2- 172.21.15.43:0/3784999661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb4109fb0 0x7fdeb4102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.623+0000 7fdebb335640 1 --2- 172.21.15.43:0/3784999661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdeb4109b10 0x7fdeb4101bc0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.623+0000 7fdebb335640 1 -- 172.21.15.43:0/3784999661 >> 172.21.15.43:0/3784999661 conn(0x7fdeb40fd850 msgr2=0x7fdeb40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.623+0000 7fdebb335640 1 -- 172.21.15.43:0/3784999661 shutdown_connections 2024-04-25T10:45:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.623+0000 7fdebb335640 1 -- 172.21.15.43:0/3784999661 wait complete. 2024-04-25T10:45:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.623+0000 7fdebb335640 1 Processor -- start 2024-04-25T10:45:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.623+0000 7fdebb335640 1 -- start start 2024-04-25T10:45:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdebb335640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdeb4109b10 0x7fdeb4199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdebb335640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdeb419a490 0x7fdeb419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdebb335640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb419ecb0 0x7fdeb419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdebb335640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb4113d50 con 0x7fdeb4109b10 2024-04-25T10:45:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdebb335640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb4113bd0 con 0x7fdeb419a490 2024-04-25T10:45:12.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdebb335640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb4113ed0 con 0x7fdeb419ecb0 2024-04-25T10:45:12.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdeb90d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdeb4109b10 0x7fdeb4199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:12.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdeb90d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdeb4109b10 0x7fdeb4199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36396/0 (socket says 172.21.15.43:36396) 2024-04-25T10:45:12.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdeb90d1640 1 -- 172.21.15.43:0/2819976678 learned_addr learned my addr 172.21.15.43:0/2819976678 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:45:12.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdeb88d0640 1 --2- 172.21.15.43:0/2819976678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdeb419a490 0x7fdeb419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:12.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdeb98d2640 1 --2- 172.21.15.43:0/2819976678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb419ecb0 0x7fdeb419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:12.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdeb90d1640 1 -- 172.21.15.43:0/2819976678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb419ecb0 msgr2=0x7fdeb419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:12.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdeb90d1640 1 --2- 172.21.15.43:0/2819976678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb419ecb0 0x7fdeb419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdeb90d1640 1 -- 172.21.15.43:0/2819976678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdeb419a490 msgr2=0x7fdeb419e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:12.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdeb90d1640 1 --2- 172.21.15.43:0/2819976678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdeb419a490 0x7fdeb419e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdeb90d1640 1 -- 172.21.15.43:0/2819976678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb419ab40 con 0x7fdeb4109b10 2024-04-25T10:45:12.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdeb88d0640 1 --2- 172.21.15.43:0/2819976678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdeb419a490 0x7fdeb419e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:45:12.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdeb90d1640 1 --2- 172.21.15.43:0/2819976678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdeb4109b10 0x7fdeb4199f50 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0005ab0 tx=0x7fdeb0003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:12.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdeaa7fc640 1 -- 172.21.15.43:0/2819976678 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb003f070 con 0x7fdeb4109b10 2024-04-25T10:45:12.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdebb335640 1 -- 172.21.15.43:0/2819976678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb41a6a80 con 0x7fdeb4109b10 2024-04-25T10:45:12.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdeaa7fc640 1 -- 172.21.15.43:0/2819976678 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeb0031d60 con 0x7fdeb4109b10 2024-04-25T10:45:12.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdeaa7fc640 1 -- 172.21.15.43:0/2819976678 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0035030 con 0x7fdeb4109b10 2024-04-25T10:45:12.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.624+0000 7fdebb335640 1 -- 172.21.15.43:0/2819976678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb41a6fd0 con 0x7fdeb4109b10 2024-04-25T10:45:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.626+0000 7fdebb335640 1 -- 172.21.15.43:0/2819976678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeb406a160 con 0x7fdeb4109b10 2024-04-25T10:45:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.627+0000 7fdeaa7fc640 1 -- 172.21.15.43:0/2819976678 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fdeb0042070 con 0x7fdeb4109b10 2024-04-25T10:45:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.628+0000 7fdeaa7fc640 1 --2- 172.21.15.43:0/2819976678 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdea00afc60 0x7fdea00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.628+0000 7fdeb88d0640 1 --2- 172.21.15.43:0/2819976678 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdea00afc60 0x7fdea00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.628+0000 7fdeaa7fc640 1 -- 172.21.15.43:0/2819976678 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7fdeb0100020 con 0x7fdeb4109b10 2024-04-25T10:45:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.629+0000 7fdeb88d0640 1 --2- 172.21.15.43:0/2819976678 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdea00afc60 0x7fdea00b2170 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdea4005e10 tx=0x7fdea4005da0 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:12.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.632+0000 7fdeaa7fc640 1 -- 172.21.15.43:0/2819976678 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdeb00c88a0 con 0x7fdeb4109b10 2024-04-25T10:45:12.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.861+0000 7fdebb335640 1 -- 172.21.15.43:0/2819976678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdeb41078f0 con 0x7fdeb4109b10 2024-04-25T10:45:12.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.863+0000 7fdeaa7fc640 1 -- 172.21.15.43:0/2819976678 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7fdeb00c81a0 con 0x7fdeb4109b10 2024-04-25T10:45:12.865 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:45:12.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.866+0000 7fdebb335640 1 -- 172.21.15.43:0/2819976678 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdea00afc60 msgr2=0x7fdea00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:12.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.866+0000 7fdebb335640 1 --2- 172.21.15.43:0/2819976678 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdea00afc60 0x7fdea00b2170 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdea4005e10 tx=0x7fdea4005da0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.866+0000 7fdebb335640 1 -- 172.21.15.43:0/2819976678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdeb4109b10 msgr2=0x7fdeb4199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:12.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.866+0000 7fdebb335640 1 --2- 172.21.15.43:0/2819976678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdeb4109b10 0x7fdeb4199f50 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0005ab0 tx=0x7fdeb0003cd0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.867+0000 7fdebb335640 1 -- 172.21.15.43:0/2819976678 shutdown_connections 2024-04-25T10:45:12.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.867+0000 7fdebb335640 1 --2- 172.21.15.43:0/2819976678 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fdea00afc60 0x7fdea00b2170 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.867+0000 7fdebb335640 1 --2- 172.21.15.43:0/2819976678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdeb419a490 0x7fdeb419e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.867+0000 7fdebb335640 1 --2- 172.21.15.43:0/2819976678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb419ecb0 0x7fdeb419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.867+0000 7fdebb335640 1 --2- 172.21.15.43:0/2819976678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdeb4109b10 0x7fdeb4199f50 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.867+0000 7fdebb335640 1 -- 172.21.15.43:0/2819976678 >> 172.21.15.43:0/2819976678 conn(0x7fdeb40fd850 msgr2=0x7fdeb4100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:12.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.867+0000 7fdebb335640 1 -- 172.21.15.43:0/2819976678 shutdown_connections 2024-04-25T10:45:12.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.867+0000 7fdebb335640 1 -- 172.21.15.43:0/2819976678 wait complete. 2024-04-25T10:45:12.880 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:45:12.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.995+0000 7f40ee97a640 1 -- 172.21.15.43:0/4272702645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40e8109b10 msgr2=0x7f40e81036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.995+0000 7f40ee97a640 1 --2- 172.21.15.43:0/4272702645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40e8109b10 0x7f40e81036b0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009900 tx=0x7f40d802a410 comp rx=0 tx=0).stop 2024-04-25T10:45:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.995+0000 7f40ee97a640 1 -- 172.21.15.43:0/4272702645 shutdown_connections 2024-04-25T10:45:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.995+0000 7f40ee97a640 1 --2- 172.21.15.43:0/4272702645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40e8109fb0 0x7f40e8103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.995+0000 7f40ee97a640 1 --2- 172.21.15.43:0/4272702645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40e8104260 0x7f40e8110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.995+0000 7f40ee97a640 1 --2- 172.21.15.43:0/4272702645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40e8109b10 0x7f40e81036b0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.995+0000 7f40ee97a640 1 -- 172.21.15.43:0/4272702645 >> 172.21.15.43:0/4272702645 conn(0x7f40e80ff210 msgr2=0x7f40e8101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.995+0000 7f40ee97a640 1 -- 172.21.15.43:0/4272702645 shutdown_connections 2024-04-25T10:45:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.995+0000 7f40ee97a640 1 -- 172.21.15.43:0/4272702645 wait complete. 2024-04-25T10:45:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.996+0000 7f40ee97a640 1 Processor -- start 2024-04-25T10:45:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.996+0000 7f40ee97a640 1 -- start start 2024-04-25T10:45:13.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.996+0000 7f40ee97a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40e8104260 0x7f40e819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:13.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.996+0000 7f40ee97a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40e8109b10 0x7f40e819ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:13.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.996+0000 7f40ee97a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40e8109fb0 0x7f40e819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.996+0000 7f40ee97a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f40e8113d00 con 0x7f40e8109b10 2024-04-25T10:45:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.996+0000 7f40ee97a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f40e8113b80 con 0x7f40e8109fb0 2024-04-25T10:45:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.996+0000 7f40ee97a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f40e8113e80 con 0x7f40e8104260 2024-04-25T10:45:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.997+0000 7f40e77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40e8109b10 0x7f40e819ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.997+0000 7f40e77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40e8109b10 0x7f40e819ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36412/0 (socket says 172.21.15.43:36412) 2024-04-25T10:45:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.997+0000 7f40e77fe640 1 -- 172.21.15.43:0/1288229868 learned_addr learned my addr 172.21.15.43:0/1288229868 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:45:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.997+0000 7f40e7fff640 1 --2- 172.21.15.43:0/1288229868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40e8104260 0x7f40e819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.997+0000 7f40ecf17640 1 --2- 172.21.15.43:0/1288229868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40e8109fb0 0x7f40e819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.997+0000 7f40e77fe640 1 -- 172.21.15.43:0/1288229868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40e8104260 msgr2=0x7f40e819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.997+0000 7f40e77fe640 1 --2- 172.21.15.43:0/1288229868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40e8104260 0x7f40e819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.997+0000 7f40e77fe640 1 -- 172.21.15.43:0/1288229868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40e8109fb0 msgr2=0x7f40e819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.997+0000 7f40e77fe640 1 --2- 172.21.15.43:0/1288229868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40e8109fb0 0x7f40e819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.997+0000 7f40e77fe640 1 -- 172.21.15.43:0/1288229868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e819f5e0 con 0x7f40e8109b10 2024-04-25T10:45:13.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.997+0000 7f40ecf17640 1 --2- 172.21.15.43:0/1288229868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40e8109fb0 0x7f40e819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:45:13.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.997+0000 7f40e77fe640 1 --2- 172.21.15.43:0/1288229868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40e8109b10 0x7f40e819ace0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00e9f0 tx=0x7f40dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:13.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.997+0000 7f40e57fa640 1 -- 172.21.15.43:0/1288229868 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc00ccc0 con 0x7f40e8109b10 2024-04-25T10:45:13.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.997+0000 7f40e57fa640 1 -- 172.21.15.43:0/1288229868 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40dc0035f0 con 0x7f40e8109b10 2024-04-25T10:45:13.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.997+0000 7f40ee97a640 1 -- 172.21.15.43:0/1288229868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e819f8d0 con 0x7f40e8109b10 2024-04-25T10:45:13.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.997+0000 7f40e57fa640 1 -- 172.21.15.43:0/1288229868 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc00ce60 con 0x7f40e8109b10 2024-04-25T10:45:13.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.997+0000 7f40ee97a640 1 -- 172.21.15.43:0/1288229868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e81a7250 con 0x7f40e8109b10 2024-04-25T10:45:13.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:12.998+0000 7f40ee97a640 1 -- 172.21.15.43:0/1288229868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40e806a160 con 0x7f40e8109b10 2024-04-25T10:45:13.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:13.000+0000 7f40e57fa640 1 -- 172.21.15.43:0/1288229868 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f40dc003aa0 con 0x7f40e8109b10 2024-04-25T10:45:13.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:13.000+0000 7f40e57fa640 1 --2- 172.21.15.43:0/1288229868 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f40d00afc60 0x7f40d00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:13.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:13.001+0000 7f40e7fff640 1 --2- 172.21.15.43:0/1288229868 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f40d00afc60 0x7f40d00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:13.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:13.001+0000 7f40e57fa640 1 -- 172.21.15.43:0/1288229868 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8771+0+0 (secure 0 0 0) 0x7f40dc0df540 con 0x7f40e8109b10 2024-04-25T10:45:13.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:13.001+0000 7f40e7fff640 1 --2- 172.21.15.43:0/1288229868 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f40d00afc60 0x7f40d00b2170 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f40d8000c00 tx=0x7f40d8031400 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:13.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:13.004+0000 7f40e57fa640 1 -- 172.21.15.43:0/1288229868 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40dc0a7ce0 con 0x7f40e8109b10 2024-04-25T10:45:13.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:12 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3527716046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:45:13.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:12 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/604146481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:45:13.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:12 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/3621594904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:45:13.041 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:45:12 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-b[63685]: ignoring --setuser ceph since I am not root 2024-04-25T10:45:13.041 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:45:12 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-b[63685]: ignoring --setgroup ceph since I am not root 2024-04-25T10:45:13.041 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:45:12 smithi077 ceph-mds[63689]: -- 172.21.15.77:0/1390161578 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e74eec9a40 con 0x55e74ef11600 2024-04-25T10:45:13.041 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:45:12 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-b[63685]: starting mds.b at 2024-04-25T10:45:13.041 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 25 10:45:12 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-e[63055]: ignoring --setuser ceph since I am not root 2024-04-25T10:45:13.041 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 25 10:45:12 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-e[63055]: ignoring --setgroup ceph since I am not root 2024-04-25T10:45:13.042 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 25 10:45:12 smithi077 ceph-mds[63059]: -- 172.21.15.77:0/3617823606 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e68c3eba40 con 0x55e68c433a80 2024-04-25T10:45:13.042 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 25 10:45:12 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-e[63055]: starting mds.e at 2024-04-25T10:45:13.042 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:45:12 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[66890]: ignoring --setuser ceph since I am not root 2024-04-25T10:45:13.042 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:45:12 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[66890]: ignoring --setgroup ceph since I am not root 2024-04-25T10:45:13.042 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:45:12 smithi077 ceph-mds[66894]: -- 172.21.15.77:0/1949721983 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563d6c67ba40 con 0x563d6c6c3a80 2024-04-25T10:45:13.042 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:45:12 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[66890]: starting mds.h at 2024-04-25T10:45:13.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:12 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3527716046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:45:13.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:12 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/604146481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:45:13.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:12 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/3621594904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:45:13.043 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 25 10:45:12 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-c[63980]: ignoring --setuser ceph since I am not root 2024-04-25T10:45:13.043 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 25 10:45:12 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-c[63980]: ignoring --setgroup ceph since I am not root 2024-04-25T10:45:13.043 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 25 10:45:12 smithi164 ceph-mds[63984]: -- 172.21.15.164:0/1290057988 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e915c27a40 con 0x55e915c6f180 2024-04-25T10:45:13.043 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 25 10:45:12 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-c[63980]: starting mds.c at 2024-04-25T10:45:13.044 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 25 10:45:12 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-f[63673]: ignoring --setuser ceph since I am not root 2024-04-25T10:45:13.044 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 25 10:45:12 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-f[63673]: ignoring --setgroup ceph since I am not root 2024-04-25T10:45:13.044 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 25 10:45:12 smithi164 ceph-mds[63677]: -- 172.21.15.164:0/1761852207 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56402f19a4e0 con 0x56402f1e5180 2024-04-25T10:45:13.044 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 25 10:45:12 smithi164 ceph-mds[63677]: -- 172.21.15.164:0/1761852207 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56402f19da40 con 0x56402f1e5180 2024-04-25T10:45:13.044 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 25 10:45:12 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-f[63673]: starting mds.f at 2024-04-25T10:45:13.044 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 25 10:45:12 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-i[64300]: ignoring --setuser ceph since I am not root 2024-04-25T10:45:13.044 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 25 10:45:12 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-i[64300]: ignoring --setgroup ceph since I am not root 2024-04-25T10:45:13.044 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 25 10:45:12 smithi164 ceph-mds[64304]: -- 172.21.15.164:0/3949815361 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e4eed97a40 con 0x55e4eeddfa80 2024-04-25T10:45:13.044 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 25 10:45:12 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-i[64300]: starting mds.i at 2024-04-25T10:45:13.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:12 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3527716046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:45:13.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:12 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/604146481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T10:45:13.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:12 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/3621594904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T10:45:13.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:12 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-a[29474]: 2024-04-25T10:45:12.862+0000 7f17afe3c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:45:13.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:13.232+0000 7f40ee97a640 1 -- 172.21.15.43:0/1288229868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f40e81a74b0 con 0x7f40e8109b10 2024-04-25T10:45:13.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:13.865+0000 7f40e57fa640 1 -- 172.21.15.43:0/1288229868 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7f40dc0a75e0 con 0x7f40e8109b10 2024-04-25T10:45:13.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:13.869+0000 7f40ee97a640 1 -- 172.21.15.43:0/1288229868 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f40d00afc60 msgr2=0x7f40d00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:13.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:13.869+0000 7f40ee97a640 1 --2- 172.21.15.43:0/1288229868 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f40d00afc60 0x7f40d00b2170 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f40d8000c00 tx=0x7f40d8031400 comp rx=0 tx=0).stop 2024-04-25T10:45:13.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:13.869+0000 7f40ee97a640 1 -- 172.21.15.43:0/1288229868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40e8109b10 msgr2=0x7f40e819ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:13.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:13.869+0000 7f40ee97a640 1 --2- 172.21.15.43:0/1288229868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40e8109b10 0x7f40e819ace0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00e9f0 tx=0x7f40dc00eeb0 comp rx=0 tx=0).stop 2024-04-25T10:45:13.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:13.869+0000 7f40ee97a640 1 -- 172.21.15.43:0/1288229868 shutdown_connections 2024-04-25T10:45:13.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:13.869+0000 7f40ee97a640 1 --2- 172.21.15.43:0/1288229868 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f40d00afc60 0x7f40d00b2170 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:13.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:13.869+0000 7f40ee97a640 1 --2- 172.21.15.43:0/1288229868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40e8109fb0 0x7f40e819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:13.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:13.869+0000 7f40ee97a640 1 --2- 172.21.15.43:0/1288229868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40e8109b10 0x7f40e819ace0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:13.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:13.869+0000 7f40ee97a640 1 --2- 172.21.15.43:0/1288229868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40e8104260 0x7f40e819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:13.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:13.869+0000 7f40ee97a640 1 -- 172.21.15.43:0/1288229868 >> 172.21.15.43:0/1288229868 conn(0x7f40e80ff210 msgr2=0x7f40e80ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:13.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:13.869+0000 7f40ee97a640 1 -- 172.21.15.43:0/1288229868 shutdown_connections 2024-04-25T10:45:13.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:13.869+0000 7f40ee97a640 1 -- 172.21.15.43:0/1288229868 wait complete. 2024-04-25T10:45:13.887 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:45:14.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.009+0000 7fc96fbce640 1 -- 172.21.15.43:0/1851458247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc968102b10 msgr2=0x7fc968102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:14.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.009+0000 7fc96fbce640 1 --2- 172.21.15.43:0/1851458247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc968102b10 0x7fc968102f20 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fc958009900 tx=0x7fc95802a410 comp rx=0 tx=0).stop 2024-04-25T10:45:14.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.010+0000 7fc96fbce640 1 -- 172.21.15.43:0/1851458247 shutdown_connections 2024-04-25T10:45:14.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.010+0000 7fc96fbce640 1 --2- 172.21.15.43:0/1851458247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc96810d3a0 0x7fc96810f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:14.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.010+0000 7fc96fbce640 1 --2- 172.21.15.43:0/1851458247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc968102b10 0x7fc968102f20 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:14.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.010+0000 7fc96fbce640 1 --2- 172.21.15.43:0/1851458247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc968108a40 0x7fc9681025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:14.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.010+0000 7fc96fbce640 1 -- 172.21.15.43:0/1851458247 >> 172.21.15.43:0/1851458247 conn(0x7fc968078180 msgr2=0x7fc9680785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:14.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.010+0000 7fc96fbce640 1 -- 172.21.15.43:0/1851458247 shutdown_connections 2024-04-25T10:45:14.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.010+0000 7fc96fbce640 1 -- 172.21.15.43:0/1851458247 wait complete. 2024-04-25T10:45:14.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.010+0000 7fc96fbce640 1 Processor -- start 2024-04-25T10:45:14.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.010+0000 7fc96fbce640 1 -- start start 2024-04-25T10:45:14.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.011+0000 7fc96fbce640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc968102b10 0x7fc96819b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:14.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.011+0000 7fc96fbce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc968108a40 0x7fc96819bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:14.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.011+0000 7fc96fbce640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc96810d3a0 0x7fc96819c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:14.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.011+0000 7fc96fbce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc968112c00 con 0x7fc968108a40 2024-04-25T10:45:14.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.011+0000 7fc96fbce640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc968112a80 con 0x7fc96810d3a0 2024-04-25T10:45:14.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.011+0000 7fc96fbce640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc968112d80 con 0x7fc968102b10 2024-04-25T10:45:14.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.011+0000 7fc96d169640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc968108a40 0x7fc96819bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:14.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.011+0000 7fc96d169640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc968108a40 0x7fc96819bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36444/0 (socket says 172.21.15.43:36444) 2024-04-25T10:45:14.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.011+0000 7fc96d169640 1 -- 172.21.15.43:0/2384770431 learned_addr learned my addr 172.21.15.43:0/2384770431 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:45:14.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.011+0000 7fc96d96a640 1 --2- 172.21.15.43:0/2384770431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc968102b10 0x7fc96819b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:14.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.011+0000 7fc96e16b640 1 --2- 172.21.15.43:0/2384770431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc96810d3a0 0x7fc96819c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:14.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.011+0000 7fc96d169640 1 -- 172.21.15.43:0/2384770431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc968102b10 msgr2=0x7fc96819b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:14.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.011+0000 7fc96d169640 1 --2- 172.21.15.43:0/2384770431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc968102b10 0x7fc96819b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:14.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.011+0000 7fc96d169640 1 -- 172.21.15.43:0/2384770431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc96810d3a0 msgr2=0x7fc96819c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:14.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.011+0000 7fc96d169640 1 --2- 172.21.15.43:0/2384770431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc96810d3a0 0x7fc96819c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:14.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.011+0000 7fc96d169640 1 -- 172.21.15.43:0/2384770431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9681a68b0 con 0x7fc968108a40 2024-04-25T10:45:14.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.011+0000 7fc96d169640 1 --2- 172.21.15.43:0/2384770431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc968108a40 0x7fc96819bae0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fc958006010 tx=0x7fc958004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:14.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.012+0000 7fc95effd640 1 -- 172.21.15.43:0/2384770431 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95803f030 con 0x7fc968108a40 2024-04-25T10:45:14.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.012+0000 7fc96fbce640 1 -- 172.21.15.43:0/2384770431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9681a6b40 con 0x7fc968108a40 2024-04-25T10:45:14.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.012+0000 7fc95effd640 1 -- 172.21.15.43:0/2384770431 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc958003aa0 con 0x7fc968108a40 2024-04-25T10:45:14.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.012+0000 7fc95effd640 1 -- 172.21.15.43:0/2384770431 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc958035070 con 0x7fc968108a40 2024-04-25T10:45:14.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.012+0000 7fc96fbce640 1 -- 172.21.15.43:0/2384770431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9681a7080 con 0x7fc968108a40 2024-04-25T10:45:14.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.013+0000 7fc96fbce640 1 -- 172.21.15.43:0/2384770431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9680692e0 con 0x7fc968108a40 2024-04-25T10:45:14.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.014+0000 7fc95effd640 1 -- 172.21.15.43:0/2384770431 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fc958042030 con 0x7fc968108a40 2024-04-25T10:45:14.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.015+0000 7fc95effd640 1 --2- 172.21.15.43:0/2384770431 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9540afc60 0x7fc9540b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:14.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.015+0000 7fc96d96a640 1 --2- 172.21.15.43:0/2384770431 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9540afc60 0x7fc9540b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:14.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.015+0000 7fc95effd640 1 -- 172.21.15.43:0/2384770431 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8771+0+0 (secure 0 0 0) 0x7fc9580ff950 con 0x7fc968108a40 2024-04-25T10:45:14.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.016+0000 7fc96d96a640 1 --2- 172.21.15.43:0/2384770431 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9540afc60 0x7fc9540b2170 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc964005d00 tx=0x7fc96400b410 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:14.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.019+0000 7fc95effd640 1 -- 172.21.15.43:0/2384770431 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc9580c7a40 con 0x7fc968108a40 2024-04-25T10:45:14.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:13 smithi077 ceph-mon[32772]: pgmap v397: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 54 KiB/s wr, 14 op/s 2024-04-25T10:45:14.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:13 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2819976678' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T10:45:14.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:13 smithi077 ceph-mon[32772]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T10:45:14.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:13 smithi077 ceph-mon[32772]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:45:14.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:13 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2819976678' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T10:45:14.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:13 smithi077 ceph-mon[32772]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T10:45:14.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:13 smithi077 ceph-mon[32772]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T10:45:14.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:13 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1288229868' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T10:45:14.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:13 smithi164 ceph-mon[32713]: pgmap v397: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 54 KiB/s wr, 14 op/s 2024-04-25T10:45:14.041 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:13 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2819976678' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T10:45:14.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:13 smithi164 ceph-mon[32713]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T10:45:14.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:13 smithi164 ceph-mon[32713]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:45:14.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:13 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2819976678' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T10:45:14.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:13 smithi164 ceph-mon[32713]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T10:45:14.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:13 smithi164 ceph-mon[32713]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T10:45:14.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:13 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1288229868' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T10:45:14.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:13 smithi043 ceph-mon[29478]: pgmap v397: 97 pgs: 97 active+clean; 3.4 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 54 KiB/s wr, 14 op/s 2024-04-25T10:45:14.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:13 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2819976678' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T10:45:14.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:13 smithi043 ceph-mon[29478]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T10:45:14.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:13 smithi043 ceph-mon[29478]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:45:14.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:13 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2819976678' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T10:45:14.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:13 smithi043 ceph-mon[29478]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T10:45:14.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:13 smithi043 ceph-mon[29478]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T10:45:14.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:13 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1288229868' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T10:45:14.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.208+0000 7fc96fbce640 1 -- 172.21.15.43:0/2384770431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc968106f00 con 0x7fc968108a40 2024-04-25T10:45:14.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.870+0000 7fc95effd640 1 -- 172.21.15.43:0/2384770431 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) ==== 181+0+0 (secure 0 0 0) 0x7fc9580cbd40 con 0x7fc968108a40 2024-04-25T10:45:14.872 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:45:14.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.872+0000 7fc96fbce640 1 -- 172.21.15.43:0/2384770431 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9540afc60 msgr2=0x7fc9540b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:14.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.872+0000 7fc96fbce640 1 --2- 172.21.15.43:0/2384770431 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9540afc60 0x7fc9540b2170 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc964005d00 tx=0x7fc96400b410 comp rx=0 tx=0).stop 2024-04-25T10:45:14.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.872+0000 7fc96fbce640 1 -- 172.21.15.43:0/2384770431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc968108a40 msgr2=0x7fc96819bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:14.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.872+0000 7fc96fbce640 1 --2- 172.21.15.43:0/2384770431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc968108a40 0x7fc96819bae0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fc958006010 tx=0x7fc958004060 comp rx=0 tx=0).stop 2024-04-25T10:45:14.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.872+0000 7fc96fbce640 1 -- 172.21.15.43:0/2384770431 shutdown_connections 2024-04-25T10:45:14.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.872+0000 7fc96fbce640 1 --2- 172.21.15.43:0/2384770431 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7fc9540afc60 0x7fc9540b2170 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:14.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.872+0000 7fc96fbce640 1 --2- 172.21.15.43:0/2384770431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc96810d3a0 0x7fc96819c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:14.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.873+0000 7fc96fbce640 1 --2- 172.21.15.43:0/2384770431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc968108a40 0x7fc96819bae0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:14.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.873+0000 7fc96fbce640 1 --2- 172.21.15.43:0/2384770431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc968102b10 0x7fc96819b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:14.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.873+0000 7fc96fbce640 1 -- 172.21.15.43:0/2384770431 >> 172.21.15.43:0/2384770431 conn(0x7fc968078180 msgr2=0x7fc9681040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:14.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.873+0000 7fc96fbce640 1 -- 172.21.15.43:0/2384770431 shutdown_connections 2024-04-25T10:45:14.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:14.873+0000 7fc96fbce640 1 -- 172.21.15.43:0/2384770431 wait complete. 2024-04-25T10:45:14.888 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:45:15.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.004+0000 7f41b3cbd640 1 -- 172.21.15.43:0/3883536791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ac10b650 msgr2=0x7f41ac10da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:15.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.004+0000 7f41b3cbd640 1 --2- 172.21.15.43:0/3883536791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ac10b650 0x7f41ac10da70 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f41a8009900 tx=0x7f41a802a410 comp rx=0 tx=0).stop 2024-04-25T10:45:15.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.004+0000 7f41b3cbd640 1 -- 172.21.15.43:0/3883536791 shutdown_connections 2024-04-25T10:45:15.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.004+0000 7f41b3cbd640 1 --2- 172.21.15.43:0/3883536791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41ac10e040 0x7f41ac110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:15.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.004+0000 7f41b3cbd640 1 --2- 172.21.15.43:0/3883536791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ac10b650 0x7f41ac10da70 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:15.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.004+0000 7f41b3cbd640 1 --2- 172.21.15.43:0/3883536791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41ac104bf0 0x7f41ac105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:15.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.004+0000 7f41b3cbd640 1 -- 172.21.15.43:0/3883536791 >> 172.21.15.43:0/3883536791 conn(0x7f41ac078110 msgr2=0x7f41ac078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:15.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.004+0000 7f41b3cbd640 1 -- 172.21.15.43:0/3883536791 shutdown_connections 2024-04-25T10:45:15.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.004+0000 7f41b3cbd640 1 -- 172.21.15.43:0/3883536791 wait complete. 2024-04-25T10:45:15.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.005+0000 7f41b3cbd640 1 Processor -- start 2024-04-25T10:45:15.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.005+0000 7f41b3cbd640 1 -- start start 2024-04-25T10:45:15.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.006+0000 7f41b3cbd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ac104bf0 0x7f41ac19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:15.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.006+0000 7f41b3cbd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41ac10b650 0x7f41ac19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:15.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.006+0000 7f41b3cbd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41ac10e040 0x7f41ac19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:15.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.006+0000 7f41b3cbd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac113ca0 con 0x7f41ac104bf0 2024-04-25T10:45:15.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.006+0000 7f41b3cbd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac113b20 con 0x7f41ac10e040 2024-04-25T10:45:15.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.006+0000 7f41b3cbd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac113e20 con 0x7f41ac10b650 2024-04-25T10:45:15.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.006+0000 7f41b1a59640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ac104bf0 0x7f41ac19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:15.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.006+0000 7f41b1a59640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ac104bf0 0x7f41ac19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36454/0 (socket says 172.21.15.43:36454) 2024-04-25T10:45:15.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.006+0000 7f41b1a59640 1 -- 172.21.15.43:0/4049597065 learned_addr learned my addr 172.21.15.43:0/4049597065 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T10:45:15.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.006+0000 7f41b225a640 1 --2- 172.21.15.43:0/4049597065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41ac10e040 0x7f41ac19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:15.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.006+0000 7f41b1258640 1 --2- 172.21.15.43:0/4049597065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41ac10b650 0x7f41ac19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:15.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.006+0000 7f41b1a59640 1 -- 172.21.15.43:0/4049597065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41ac10b650 msgr2=0x7f41ac19ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:15.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.006+0000 7f41b1a59640 1 --2- 172.21.15.43:0/4049597065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41ac10b650 0x7f41ac19ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:15.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.006+0000 7f41b1a59640 1 -- 172.21.15.43:0/4049597065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41ac10e040 msgr2=0x7f41ac19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:15.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.006+0000 7f41b1a59640 1 --2- 172.21.15.43:0/4049597065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41ac10e040 0x7f41ac19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:15.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.006+0000 7f41b1a59640 1 -- 172.21.15.43:0/4049597065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ac19f540 con 0x7f41ac104bf0 2024-04-25T10:45:15.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.006+0000 7f41b225a640 1 --2- 172.21.15.43:0/4049597065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41ac10e040 0x7f41ac19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:45:15.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.006+0000 7f41b1a59640 1 --2- 172.21.15.43:0/4049597065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ac104bf0 0x7f41ac19a6e0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f41a000d950 tx=0x7f41a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:15.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.006+0000 7f419effd640 1 -- 172.21.15.43:0/4049597065 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0017030 con 0x7f41ac104bf0 2024-04-25T10:45:15.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.007+0000 7f41b3cbd640 1 -- 172.21.15.43:0/4049597065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ac19f830 con 0x7f41ac104bf0 2024-04-25T10:45:15.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.007+0000 7f419effd640 1 -- 172.21.15.43:0/4049597065 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a0004020 con 0x7f41ac104bf0 2024-04-25T10:45:15.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.007+0000 7f419effd640 1 -- 172.21.15.43:0/4049597065 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a00047a0 con 0x7f41ac104bf0 2024-04-25T10:45:15.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.007+0000 7f41b3cbd640 1 -- 172.21.15.43:0/4049597065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ac1a71b0 con 0x7f41ac104bf0 2024-04-25T10:45:15.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.008+0000 7f41b3cbd640 1 -- 172.21.15.43:0/4049597065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41ac06a770 con 0x7f41ac104bf0 2024-04-25T10:45:15.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.009+0000 7f419effd640 1 -- 172.21.15.43:0/4049597065 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f41a0009a30 con 0x7f41ac104bf0 2024-04-25T10:45:15.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.010+0000 7f419effd640 1 --2- 172.21.15.43:0/4049597065 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f41980afc60 0x7f41980b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:45:15.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.010+0000 7f41b1258640 1 --2- 172.21.15.43:0/4049597065 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f41980afc60 0x7f41980b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:45:15.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.010+0000 7f419effd640 1 -- 172.21.15.43:0/4049597065 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8246+0+0 (secure 0 0 0) 0x7f41a00dea20 con 0x7f41ac104bf0 2024-04-25T10:45:15.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.010+0000 7f41b1258640 1 --2- 172.21.15.43:0/4049597065 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f41980afc60 0x7f41980b2170 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f41a8031440 tx=0x7f41a8005d50 comp rx=0 tx=0).ready entity=mgr.15291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:45:15.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.014+0000 7f419effd640 1 -- 172.21.15.43:0/4049597065 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41a00a73d0 con 0x7f41ac104bf0 2024-04-25T10:45:15.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:14 smithi077 ceph-mon[32772]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T10:45:15.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:14 smithi077 ceph-mon[32772]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T10:45:15.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:14 smithi077 ceph-mon[32772]: Cluster is now healthy 2024-04-25T10:45:15.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:14 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/1288229868' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T10:45:15.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:14 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T10:45:15.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:14 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T10:45:15.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:14 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T10:45:15.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:14 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T10:45:15.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:14 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T10:45:15.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:14 smithi077 ceph-mon[32772]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T10:45:15.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:14 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.164:6835/872308603,v1:172.21.15.164:6838/872308603] up:boot 2024-04-25T10:45:15.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:14 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.164:6836/2188403381,v1:172.21.15.164:6839/2188403381] up:boot 2024-04-25T10:45:15.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:14 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.164:6834/2989367277,v1:172.21.15.164:6837/2989367277] up:boot 2024-04-25T10:45:15.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:14 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.77:6834/3010916668,v1:172.21.15.77:6835/3010916668] up:boot 2024-04-25T10:45:15.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:14 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.77:6836/3434687230,v1:172.21.15.77:6838/3434687230] up:boot 2024-04-25T10:45:15.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:14 smithi077 ceph-mon[32772]: mds.? [v2:172.21.15.77:6837/2554670126,v1:172.21.15.77:6839/2554670126] up:boot 2024-04-25T10:45:15.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:14 smithi077 ceph-mon[32772]: fsmap 12 up:standby 2024-04-25T10:45:15.042 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:14 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2384770431' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T10:45:15.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:14 smithi164 ceph-mon[32713]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T10:45:15.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:14 smithi164 ceph-mon[32713]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T10:45:15.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:14 smithi164 ceph-mon[32713]: Cluster is now healthy 2024-04-25T10:45:15.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:14 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/1288229868' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T10:45:15.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:14 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T10:45:15.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:14 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T10:45:15.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:14 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T10:45:15.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:14 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T10:45:15.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:14 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T10:45:15.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:14 smithi164 ceph-mon[32713]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T10:45:15.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:14 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.164:6835/872308603,v1:172.21.15.164:6838/872308603] up:boot 2024-04-25T10:45:15.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:14 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.164:6836/2188403381,v1:172.21.15.164:6839/2188403381] up:boot 2024-04-25T10:45:15.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:14 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.164:6834/2989367277,v1:172.21.15.164:6837/2989367277] up:boot 2024-04-25T10:45:15.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:14 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.77:6834/3010916668,v1:172.21.15.77:6835/3010916668] up:boot 2024-04-25T10:45:15.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:14 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.77:6836/3434687230,v1:172.21.15.77:6838/3434687230] up:boot 2024-04-25T10:45:15.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:14 smithi164 ceph-mon[32713]: mds.? [v2:172.21.15.77:6837/2554670126,v1:172.21.15.77:6839/2554670126] up:boot 2024-04-25T10:45:15.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:14 smithi164 ceph-mon[32713]: fsmap 12 up:standby 2024-04-25T10:45:15.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:14 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2384770431' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T10:45:15.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:14 smithi043 ceph-mon[29478]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T10:45:15.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:14 smithi043 ceph-mon[29478]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T10:45:15.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:14 smithi043 ceph-mon[29478]: Cluster is now healthy 2024-04-25T10:45:15.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:14 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/1288229868' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T10:45:15.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:14 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T10:45:15.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:14 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T10:45:15.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:14 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T10:45:15.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:14 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T10:45:15.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:14 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T10:45:15.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:14 smithi043 ceph-mon[29478]: from='mgr.15291 172.21.15.43:0/3831852729' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T10:45:15.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:14 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.164:6835/872308603,v1:172.21.15.164:6838/872308603] up:boot 2024-04-25T10:45:15.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:14 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.164:6836/2188403381,v1:172.21.15.164:6839/2188403381] up:boot 2024-04-25T10:45:15.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:14 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.164:6834/2989367277,v1:172.21.15.164:6837/2989367277] up:boot 2024-04-25T10:45:15.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:14 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.77:6834/3010916668,v1:172.21.15.77:6835/3010916668] up:boot 2024-04-25T10:45:15.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:14 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.77:6836/3434687230,v1:172.21.15.77:6838/3434687230] up:boot 2024-04-25T10:45:15.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:14 smithi043 ceph-mon[29478]: mds.? [v2:172.21.15.77:6837/2554670126,v1:172.21.15.77:6839/2554670126] up:boot 2024-04-25T10:45:15.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:14 smithi043 ceph-mon[29478]: fsmap 12 up:standby 2024-04-25T10:45:15.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:14 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2384770431' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T10:45:15.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.202+0000 7f41b3cbd640 1 -- 172.21.15.43:0/4049597065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f41ac06afa0 con 0x7f41ac104bf0 2024-04-25T10:45:15.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.877+0000 7f419effd640 1 -- 172.21.15.43:0/4049597065 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) ==== 169+0+0 (secure 0 0 0) 0x7f41a00a6cd0 con 0x7f41ac104bf0 2024-04-25T10:45:15.879 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-04-25T10:45:15.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.881+0000 7f41b3cbd640 1 -- 172.21.15.43:0/4049597065 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f41980afc60 msgr2=0x7f41980b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:15.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.881+0000 7f41b3cbd640 1 --2- 172.21.15.43:0/4049597065 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f41980afc60 0x7f41980b2170 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f41a8031440 tx=0x7f41a8005d50 comp rx=0 tx=0).stop 2024-04-25T10:45:15.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.881+0000 7f41b3cbd640 1 -- 172.21.15.43:0/4049597065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ac104bf0 msgr2=0x7f41ac19a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:45:15.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.881+0000 7f41b3cbd640 1 --2- 172.21.15.43:0/4049597065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ac104bf0 0x7f41ac19a6e0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f41a000d950 tx=0x7f41a000de10 comp rx=0 tx=0).stop 2024-04-25T10:45:15.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.881+0000 7f41b3cbd640 1 -- 172.21.15.43:0/4049597065 shutdown_connections 2024-04-25T10:45:15.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.881+0000 7f41b3cbd640 1 --2- 172.21.15.43:0/4049597065 >> [v2:172.21.15.43:6800/3658924116,v1:172.21.15.43:6801/3658924116] conn(0x7f41980afc60 0x7f41980b2170 secure :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f41a8031440 tx=0x7f41a8005d50 comp rx=0 tx=0).stop 2024-04-25T10:45:15.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.881+0000 7f41b3cbd640 1 --2- 172.21.15.43:0/4049597065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41ac10e040 0x7f41ac19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:15.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.881+0000 7f41b3cbd640 1 --2- 172.21.15.43:0/4049597065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41ac10b650 0x7f41ac19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:15.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.881+0000 7f41b3cbd640 1 --2- 172.21.15.43:0/4049597065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ac104bf0 0x7f41ac19a6e0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:45:15.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.881+0000 7f41b3cbd640 1 -- 172.21.15.43:0/4049597065 >> 172.21.15.43:0/4049597065 conn(0x7f41ac078110 msgr2=0x7f41ac101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:45:15.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.882+0000 7f41b3cbd640 1 -- 172.21.15.43:0/4049597065 shutdown_connections 2024-04-25T10:45:15.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T10:45:15.882+0000 7f41b3cbd640 1 -- 172.21.15.43:0/4049597065 wait complete. 2024-04-25T10:45:15.902 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T10:45:15.929 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T10:45:15.960 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T10:45:15.992 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T10:45:15.993 DEBUG:teuthology.orchestra.run.smithi043:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T10:45:16.009 DEBUG:teuthology.orchestra.run.smithi077:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T10:45:16.026 DEBUG:teuthology.orchestra.run.smithi164:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T10:45:16.050 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T10:45:16.050 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T10:45:16.050 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mon.a 2024-04-25T10:45:16.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:15 smithi077 ceph-mon[32772]: pgmap v399: 97 pgs: 97 active+clean; 3.4 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-25T10:45:16.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:15 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/2384770431' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T10:45:16.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:15 smithi077 ceph-mon[32772]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T10:45:16.290 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:15 smithi077 ceph-mon[32772]: from='client.? 172.21.15.43:0/4049597065' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T10:45:16.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:15 smithi164 ceph-mon[32713]: pgmap v399: 97 pgs: 97 active+clean; 3.4 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-25T10:45:16.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:15 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/2384770431' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T10:45:16.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:15 smithi164 ceph-mon[32713]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T10:45:16.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:15 smithi164 ceph-mon[32713]: from='client.? 172.21.15.43:0/4049597065' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T10:45:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:15 smithi043 ceph-mon[29478]: pgmap v399: 97 pgs: 97 active+clean; 3.4 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-25T10:45:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:15 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/2384770431' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T10:45:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:15 smithi043 ceph-mon[29478]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T10:45:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:15 smithi043 ceph-mon[29478]: from='client.? 172.21.15.43:0/4049597065' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T10:45:16.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:16 smithi043 systemd[1]: Stopping Ceph mon.a for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:45:16.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:16 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-a[29474]: 2024-04-25T10:45:16.409+0000 7f17b5647700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:45:16.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:16 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-a[29474]: 2024-04-25T10:45:16.409+0000 7f17b5647700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T10:45:16.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:16 smithi043 podman[98247]: 2024-04-25 10:45:16.625252125 +0000 UTC m=+0.374691385 container died 471bd00c8578c51d08f93c203a3cfdb4b6cf3e1922ec4fe4657ca7c01ff1bc70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T10:45:17.183 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:16 smithi043 podman[98247]: 2024-04-25 10:45:16.972680557 +0000 UTC m=+0.722119806 container cleanup 471bd00c8578c51d08f93c203a3cfdb4b6cf3e1922ec4fe4657ca7c01ff1bc70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T10:45:17.183 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:16 smithi043 bash[98247]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-a 2024-04-25T10:45:17.420 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T10:45:17.441 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:17 smithi043 podman[98261]: 2024-04-25 10:45:17.182327633 +0000 UTC m=+0.551124146 container remove 471bd00c8578c51d08f93c203a3cfdb4b6cf3e1922ec4fe4657ca7c01ff1bc70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T10:45:17.441 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:17 smithi043 systemd[1]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T10:45:17.442 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:17 smithi043 systemd[1]: Stopped Ceph mon.a for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:45:17.442 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 25 10:45:17 smithi043 systemd[1]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mon.a.service: Consumed 23.582s CPU time. 2024-04-25T10:45:17.464 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:45:17.465 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T10:45:17.465 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T10:45:17.465 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mon.b 2024-04-25T10:45:17.759 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:17 smithi077 systemd[1]: Stopping Ceph mon.b for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:45:18.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:17 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-b[32767]: 2024-04-25T10:45:17.757+0000 7f768cd1b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:45:18.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:17 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-b[32767]: 2024-04-25T10:45:17.757+0000 7f768cd1b700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T10:45:18.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:17 smithi077 podman[70227]: 2024-04-25 10:45:17.798476159 +0000 UTC m=+0.144131042 container died 6cbd105360e0ff4dbfcde6f79266cd0ff93d10e228654ea9cf9bfe8873647170 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:45:18.370 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:18 smithi077 podman[70227]: 2024-04-25 10:45:18.168559714 +0000 UTC m=+0.514214605 container cleanup 6cbd105360e0ff4dbfcde6f79266cd0ff93d10e228654ea9cf9bfe8873647170 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T10:45:18.370 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 25 10:45:18 smithi077 bash[70227]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-b 2024-04-25T10:45:18.575 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T10:45:18.626 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:45:18.626 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T10:45:18.626 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T10:45:18.627 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mon.c 2024-04-25T10:45:18.791 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:18 smithi164 systemd[1]: Stopping Ceph mon.c for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:45:19.287 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:18 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-c[32709]: 2024-04-25T10:45:18.926+0000 7ff836095700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:45:19.287 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:18 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-c[32709]: 2024-04-25T10:45:18.926+0000 7ff836095700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T10:45:19.288 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:18 smithi164 podman[70141]: 2024-04-25 10:45:18.998584122 +0000 UTC m=+0.193240132 container died 31632d504d56d6ecdb302e312f9f7a7a5aa6f6ca8821cee3cd792a3009f3dd7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-c, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T10:45:19.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:19 smithi164 podman[70141]: 2024-04-25 10:45:19.385493905 +0000 UTC m=+0.580149958 container cleanup 31632d504d56d6ecdb302e312f9f7a7a5aa6f6ca8821cee3cd792a3009f3dd7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:45:19.541 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 25 10:45:19 smithi164 bash[70141]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mon-c 2024-04-25T10:45:19.816 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T10:45:19.871 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:45:19.872 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T10:45:19.872 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T10:45:19.872 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mgr.x 2024-04-25T10:45:20.242 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:45:19 smithi043 systemd[1]: Stopping Ceph mgr.x for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:45:20.513 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:45:20 smithi043 podman[98369]: 2024-04-25 10:45:20.241232599 +0000 UTC m=+0.166592578 container died 788b64a21bab56fc3d07c42daec0a4ed1467ea9f54d2d360292d43600f83e2d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-04-25T10:45:20.788 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:45:20 smithi043 podman[98369]: 2024-04-25 10:45:20.585745712 +0000 UTC m=+0.511105690 container cleanup 788b64a21bab56fc3d07c42daec0a4ed1467ea9f54d2d360292d43600f83e2d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux ) 2024-04-25T10:45:20.789 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:45:20 smithi043 bash[98369]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x 2024-04-25T10:45:21.026 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T10:45:21.048 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:45:20 smithi043 podman[98383]: 2024-04-25 10:45:20.787530945 +0000 UTC m=+0.540920196 container remove 788b64a21bab56fc3d07c42daec0a4ed1467ea9f54d2d360292d43600f83e2d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T10:45:21.048 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:45:21 smithi043 systemd[1]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mgr.x.service: Deactivated successfully. 2024-04-25T10:45:21.048 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:45:21 smithi043 systemd[1]: Stopped Ceph mgr.x for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:45:21.048 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 25 10:45:21 smithi043 systemd[1]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mgr.x.service: Consumed 1min 32.693s CPU time. 2024-04-25T10:45:21.062 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:45:21.062 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T10:45:21.063 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T10:45:21.063 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mgr.y 2024-04-25T10:45:21.397 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:45:21 smithi077 systemd[1]: Stopping Ceph mgr.y for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:45:21.683 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:45:21 smithi077 podman[70350]: 2024-04-25 10:45:21.395483228 +0000 UTC m=+0.140300332 container died 8e9cc76a10c1f8d0eae9506889f4e295104d5e4a2f285340447c35fd72191598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-25T10:45:22.040 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:45:21 smithi077 podman[70350]: 2024-04-25 10:45:21.781727802 +0000 UTC m=+0.526544905 container cleanup 8e9cc76a10c1f8d0eae9506889f4e295104d5e4a2f285340447c35fd72191598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T10:45:22.040 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 25 10:45:21 smithi077 bash[70350]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-y 2024-04-25T10:45:22.208 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T10:45:22.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:45:22.241 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T10:45:22.241 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T10:45:22.241 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mgr.z 2024-04-25T10:45:22.541 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:45:22 smithi164 systemd[1]: Stopping Ceph mgr.z for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:45:22.849 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:45:22 smithi164 podman[70264]: 2024-04-25 10:45:22.578767496 +0000 UTC m=+0.144372985 container died 45598f416ec86a8e7d637b017d5d042dc0caaa3a954496444c3f4c31a8686186 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-25T10:45:23.290 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:45:22 smithi164 podman[70264]: 2024-04-25 10:45:22.922699627 +0000 UTC m=+0.488305141 container cleanup 45598f416ec86a8e7d637b017d5d042dc0caaa3a954496444c3f4c31a8686186 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T10:45:23.291 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 25 10:45:22 smithi164 bash[70264]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mgr-z 2024-04-25T10:45:23.495 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T10:45:23.527 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:45:23.527 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T10:45:23.528 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T10:45:23.528 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.0 2024-04-25T10:45:23.849 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:45:23 smithi043 systemd[1]: Stopping Ceph osd.0 for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:45:24.110 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:45:23 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-0[41543]: 2024-04-25T10:45:23.847+0000 7feff533a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:45:24.110 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:45:23 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-0[41543]: 2024-04-25T10:45:23.847+0000 7feff533a700 -1 osd.0 97 *** Got signal Terminated *** 2024-04-25T10:45:24.110 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:45:23 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-0[41543]: 2024-04-25T10:45:23.847+0000 7feff533a700 -1 osd.0 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T10:45:29.567 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:45:29 smithi043 podman[98489]: 2024-04-25 10:45:29.282914585 +0000 UTC m=+5.561361945 container died 8f4ebdcaed14a281d2fae6c32fd14a87d98953767926c9d21c9eac2e25bb0d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:45:29.860 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:45:29 smithi043 podman[98489]: 2024-04-25 10:45:29.656733768 +0000 UTC m=+5.935181122 container cleanup 8f4ebdcaed14a281d2fae6c32fd14a87d98953767926c9d21c9eac2e25bb0d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T10:45:29.860 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:45:29 smithi043 bash[98489]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-0 2024-04-25T10:45:30.250 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:45:29 smithi043 podman[98506]: 2024-04-25 10:45:29.907939334 +0000 UTC m=+0.623785365 container remove 8f4ebdcaed14a281d2fae6c32fd14a87d98953767926c9d21c9eac2e25bb0d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T10:45:30.521 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:45:30 smithi043 podman[98598]: 2024-04-25 10:45:30.153169129 +0000 UTC m=+0.018374967 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:45:30.521 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:45:30 smithi043 podman[98598]: 2024-04-25 10:45:30.275751381 +0000 UTC m=+0.140957229 container create 00c0a08203c1d23f161106669a5845189db22ff1430cb792d7f16a61cefaf402 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T10:45:30.827 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:45:30 smithi043 podman[98598]: 2024-04-25 10:45:30.520574776 +0000 UTC m=+0.385780586 container init 00c0a08203c1d23f161106669a5845189db22ff1430cb792d7f16a61cefaf402 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-25T10:45:30.827 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:45:30 smithi043 podman[98598]: 2024-04-25 10:45:30.523903235 +0000 UTC m=+0.389109045 container start 00c0a08203c1d23f161106669a5845189db22ff1430cb792d7f16a61cefaf402 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T10:45:30.828 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:45:30 smithi043 podman[98598]: 2024-04-25 10:45:30.620425806 +0000 UTC m=+0.485631623 container attach 00c0a08203c1d23f161106669a5845189db22ff1430cb792d7f16a61cefaf402 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-25T10:45:31.110 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 25 10:45:30 smithi043 podman[98598]: 2024-04-25 10:45:30.826595658 +0000 UTC m=+0.691801475 container died 00c0a08203c1d23f161106669a5845189db22ff1430cb792d7f16a61cefaf402 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-25T10:45:31.409 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T10:45:31.451 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:45:31.451 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T10:45:31.452 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T10:45:31.452 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.1 2024-04-25T10:45:31.762 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:45:31 smithi077 systemd[1]: Stopping Ceph osd.1 for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:45:32.040 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:45:31 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-1[37392]: 2024-04-25T10:45:31.760+0000 7f77e4503700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:45:32.041 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:45:31 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-1[37392]: 2024-04-25T10:45:31.760+0000 7f77e4503700 -1 osd.1 97 *** Got signal Terminated *** 2024-04-25T10:45:32.041 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:45:31 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-1[37392]: 2024-04-25T10:45:31.760+0000 7f77e4503700 -1 osd.1 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T10:45:37.437 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:45:37 smithi077 podman[70477]: 2024-04-25 10:45:37.176623277 +0000 UTC m=+5.539264545 container died ef8b3fdac05bbfa0724cf49c8ff780e7961af2e5a55aa580ae9dfa6f91fc5224 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T10:45:37.695 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:45:37 smithi077 podman[70477]: 2024-04-25 10:45:37.510342808 +0000 UTC m=+5.872984064 container cleanup ef8b3fdac05bbfa0724cf49c8ff780e7961af2e5a55aa580ae9dfa6f91fc5224 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T10:45:37.696 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:45:37 smithi077 bash[70477]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-1 2024-04-25T10:45:38.040 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:45:37 smithi077 podman[70494]: 2024-04-25 10:45:37.694820649 +0000 UTC m=+0.515820352 container remove ef8b3fdac05bbfa0724cf49c8ff780e7961af2e5a55aa580ae9dfa6f91fc5224 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:45:38.407 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:45:38 smithi077 podman[70581]: 2024-04-25 10:45:38.129260673 +0000 UTC m=+0.113518866 container create c082ebcbb1d5c5d2f9c670d0130fbc0e7039e28f9df1f4adbc0e662193753388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T10:45:38.407 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:45:38 smithi077 podman[70581]: 2024-04-25 10:45:38.033546555 +0000 UTC m=+0.017804764 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:45:38.407 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:45:38 smithi077 podman[70581]: 2024-04-25 10:45:38.323289894 +0000 UTC m=+0.307548086 container init c082ebcbb1d5c5d2f9c670d0130fbc0e7039e28f9df1f4adbc0e662193753388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T10:45:38.407 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:45:38 smithi077 podman[70581]: 2024-04-25 10:45:38.326515021 +0000 UTC m=+0.310773214 container start c082ebcbb1d5c5d2f9c670d0130fbc0e7039e28f9df1f4adbc0e662193753388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T10:45:38.407 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:45:38 smithi077 podman[70581]: 2024-04-25 10:45:38.406293122 +0000 UTC m=+0.390551327 container attach c082ebcbb1d5c5d2f9c670d0130fbc0e7039e28f9df1f4adbc0e662193753388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T10:45:38.790 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 25 10:45:38 smithi077 podman[70581]: 2024-04-25 10:45:38.622546201 +0000 UTC m=+0.606804397 container died c082ebcbb1d5c5d2f9c670d0130fbc0e7039e28f9df1f4adbc0e662193753388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T10:45:39.268 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T10:45:39.308 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:45:39.308 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T10:45:39.309 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T10:45:39.309 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.2 2024-04-25T10:45:39.603 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:45:39 smithi164 systemd[1]: Stopping Ceph osd.2 for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:45:40.040 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:45:39 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-2[37873]: 2024-04-25T10:45:39.601+0000 7f3f42e77700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:45:40.041 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:45:39 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-2[37873]: 2024-04-25T10:45:39.601+0000 7f3f42e77700 -1 osd.2 97 *** Got signal Terminated *** 2024-04-25T10:45:40.041 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:45:39 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-2[37873]: 2024-04-25T10:45:39.601+0000 7f3f42e77700 -1 osd.2 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T10:45:45.227 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:45:44 smithi164 podman[70392]: 2024-04-25 10:45:44.974872768 +0000 UTC m=+5.471661667 container died 8fb0727b6cfad57f7582ecd3783cb81728f72e82870716a3cb6c500dea2bfb49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T10:45:45.541 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:45:45 smithi164 podman[70392]: 2024-04-25 10:45:45.22606836 +0000 UTC m=+5.722857260 container cleanup 8fb0727b6cfad57f7582ecd3783cb81728f72e82870716a3cb6c500dea2bfb49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-25T10:45:45.541 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:45:45 smithi164 bash[70392]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-2 2024-04-25T10:45:45.541 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:45:45 smithi164 podman[70409]: 2024-04-25 10:45:45.410390705 +0000 UTC m=+0.437571971 container remove 8fb0727b6cfad57f7582ecd3783cb81728f72e82870716a3cb6c500dea2bfb49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-25T10:45:46.131 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:45:45 smithi164 podman[70498]: 2024-04-25 10:45:45.731928876 +0000 UTC m=+0.018283796 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:45:46.131 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:45:45 smithi164 podman[70498]: 2024-04-25 10:45:45.836264373 +0000 UTC m=+0.122619320 container create dd6dcec0dcbfdb9d4207b7dea4009c5af01df46b140cd7f950cab2a4ec796fd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T10:45:46.131 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:45:46 smithi164 podman[70498]: 2024-04-25 10:45:46.130494101 +0000 UTC m=+0.416849006 container init dd6dcec0dcbfdb9d4207b7dea4009c5af01df46b140cd7f950cab2a4ec796fd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-25T10:45:46.424 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:45:46 smithi164 podman[70498]: 2024-04-25 10:45:46.134114038 +0000 UTC m=+0.420468944 container start dd6dcec0dcbfdb9d4207b7dea4009c5af01df46b140cd7f950cab2a4ec796fd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-25T10:45:46.424 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:45:46 smithi164 podman[70498]: 2024-04-25 10:45:46.213794813 +0000 UTC m=+0.500149723 container attach dd6dcec0dcbfdb9d4207b7dea4009c5af01df46b140cd7f950cab2a4ec796fd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T10:45:46.692 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:45:46 smithi164 podman[70498]: 2024-04-25 10:45:46.423507686 +0000 UTC m=+0.709862598 container died dd6dcec0dcbfdb9d4207b7dea4009c5af01df46b140cd7f950cab2a4ec796fd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T10:45:46.928 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T10:45:46.950 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:45:46 smithi164 podman[70524]: 2024-04-25 10:45:46.908736832 +0000 UTC m=+0.479662541 container remove dd6dcec0dcbfdb9d4207b7dea4009c5af01df46b140cd7f950cab2a4ec796fd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T10:45:46.950 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:45:46 smithi164 systemd[1]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.2.service: Deactivated successfully. 2024-04-25T10:45:46.950 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:45:46 smithi164 systemd[1]: Stopped Ceph osd.2 for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:45:46.950 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 25 10:45:46 smithi164 systemd[1]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.2.service: Consumed 12.208s CPU time. 2024-04-25T10:45:46.965 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:45:46.966 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T10:45:46.966 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T10:45:46.966 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.3 2024-04-25T10:45:47.269 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 25 10:45:47 smithi043 systemd[1]: Stopping Ceph osd.3 for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:45:47.610 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 25 10:45:47 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-3[48519]: 2024-04-25T10:45:47.267+0000 7f83399c6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:45:47.610 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 25 10:45:47 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-3[48519]: 2024-04-25T10:45:47.267+0000 7f83399c6700 -1 osd.3 97 *** Got signal Terminated *** 2024-04-25T10:45:47.610 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 25 10:45:47 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-3[48519]: 2024-04-25T10:45:47.267+0000 7f83399c6700 -1 osd.3 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T10:45:50.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:45:50 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:45:50.013+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:45:51.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:45:51 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:45:51.021+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:45:51.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:45:51 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:45:51.057+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:45:52.290 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:45:52 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:45:52.034+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:45:52.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:45:52 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:45:52.055+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:45:53.024 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 25 10:45:52 smithi043 podman[98720]: 2024-04-25 10:45:52.750609046 +0000 UTC m=+5.582408935 container died d8c1a287a887b113e049886c271744285360e2f6cd418462ea55a70e4587b1af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:45:53.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:45:53 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:45:53.022+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:45:53.360 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 25 10:45:53 smithi043 podman[98720]: 2024-04-25 10:45:53.108986864 +0000 UTC m=+5.940786742 container cleanup d8c1a287a887b113e049886c271744285360e2f6cd418462ea55a70e4587b1af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:45:53.360 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 25 10:45:53 smithi043 bash[98720]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-3 2024-04-25T10:45:53.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:45:53 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:45:53.068+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:45:53.687 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 25 10:45:53 smithi043 podman[98739]: 2024-04-25 10:45:53.360106225 +0000 UTC m=+0.606706174 container remove d8c1a287a887b113e049886c271744285360e2f6cd418462ea55a70e4587b1af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T10:45:53.688 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:45:53 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:45:53.522+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:45:54.006 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 25 10:45:53 smithi043 podman[98830]: 2024-04-25 10:45:53.589604394 +0000 UTC m=+0.018742975 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:45:54.006 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 25 10:45:53 smithi043 podman[98830]: 2024-04-25 10:45:53.721247215 +0000 UTC m=+0.150385813 container create 7b2bf651aa9036d02c5d1d7605f5df2e83faac1ce0c6df50f84816f5306d0550 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T10:45:54.006 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 25 10:45:53 smithi043 podman[98830]: 2024-04-25 10:45:53.930431593 +0000 UTC m=+0.359570148 container init 7b2bf651aa9036d02c5d1d7605f5df2e83faac1ce0c6df50f84816f5306d0550 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T10:45:54.006 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 25 10:45:53 smithi043 podman[98830]: 2024-04-25 10:45:53.933725949 +0000 UTC m=+0.362864507 container start 7b2bf651aa9036d02c5d1d7605f5df2e83faac1ce0c6df50f84816f5306d0550 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T10:45:54.006 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 25 10:45:54 smithi043 podman[98830]: 2024-04-25 10:45:54.005207256 +0000 UTC m=+0.434345821 container attach 7b2bf651aa9036d02c5d1d7605f5df2e83faac1ce0c6df50f84816f5306d0550 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:45:54.291 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:45:53 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:45:53.839+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:45:54.295 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:45:54 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:45:54.026+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:45:54.295 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 25 10:45:54 smithi043 podman[98830]: 2024-04-25 10:45:54.247254996 +0000 UTC m=+0.676393566 container died 7b2bf651aa9036d02c5d1d7605f5df2e83faac1ce0c6df50f84816f5306d0550 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-25T10:45:54.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:45:54 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:45:54.043+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:45:54.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:45:54 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:45:54.492+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:45:54.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:45:54 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:45:54.307+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:45:54.920 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T10:45:54.958 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:45:54.959 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T10:45:54.959 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T10:45:54.959 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.4 2024-04-25T10:45:55.272 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:45:55 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:45:55.028+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:45:55.273 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 25 10:45:55 smithi077 systemd[1]: Stopping Ceph osd.4 for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:45:55.273 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:45:55 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:45:55.000+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:45:55.290 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:45:54 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:45:54.823+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:45:55.540 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 25 10:45:55 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-4[44364]: 2024-04-25T10:45:55.270+0000 7eff2be45700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:45:55.540 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 25 10:45:55 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-4[44364]: 2024-04-25T10:45:55.270+0000 7eff2be45700 -1 osd.4 97 *** Got signal Terminated *** 2024-04-25T10:45:55.540 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 25 10:45:55 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-4[44364]: 2024-04-25T10:45:55.270+0000 7eff2be45700 -1 osd.4 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T10:45:55.790 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:45:55 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:45:55.322+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:45:55.791 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:45:55 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:45:55.316+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.404649+0000 front 2024-04-25T10:45:28.404521+0000 (oldest deadline 2024-04-25T10:45:54.805047+0000) 2024-04-25T10:45:55.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:45:55 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:45:55.538+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:45:56.286 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:45:55 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:45:55.795+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:45:56.290 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:45:55 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:45:55.982+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:45:56.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:45:56 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:45:56.036+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:45:56.541 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:45:56 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:45:56.284+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.404649+0000 front 2024-04-25T10:45:28.404521+0000 (oldest deadline 2024-04-25T10:45:54.805047+0000) 2024-04-25T10:45:56.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:45:56 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:45:56.369+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:45:56.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:45:56 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:45:56.536+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:45:57.040 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:45:56 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:45:56.770+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:45:57.290 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:45:56 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:45:56.997+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:45:57.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:45:57 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:45:57.025+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:45:57.541 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:45:57 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:45:57.270+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.404649+0000 front 2024-04-25T10:45:28.404521+0000 (oldest deadline 2024-04-25T10:45:54.805047+0000) 2024-04-25T10:45:57.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:45:57 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:45:57.357+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:45:57.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:45:57 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:45:57.491+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:45:58.040 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:45:57 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:45:57.796+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:45:58.290 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:45:58 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:45:58.026+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:45:58.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:45:58 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:45:58.075+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:45:58.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:45:58 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:45:58.402+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:45:58.541 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:45:58 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:45:58.309+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.404649+0000 front 2024-04-25T10:45:28.404521+0000 (oldest deadline 2024-04-25T10:45:54.805047+0000) 2024-04-25T10:45:58.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:45:58 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:45:58.488+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:45:59.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:45:58 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:45:58.765+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:45:59.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:45:59 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:45:59.036+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:45:59.498 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:45:59 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:45:59.041+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:45:59.498 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:45:59 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:45:59.041+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:45:59.719 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:45:59 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:45:59.404+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:45:59.720 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:45:59 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:45:59.342+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.404649+0000 front 2024-04-25T10:45:28.404521+0000 (oldest deadline 2024-04-25T10:45:54.805047+0000) 2024-04-25T10:45:59.720 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:45:59 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:45:59.342+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.805512+0000 front 2024-04-25T10:45:34.805567+0000 (oldest deadline 2024-04-25T10:45:58.905587+0000) 2024-04-25T10:45:59.790 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:45:59 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:45:59.497+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:45:59.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:45:59 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:45:59.535+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:45:59.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:45:59 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:45:59.535+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:00.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:45:59 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:45:59.717+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:00.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:45:59 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:45:59.994+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:46:00.510 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:00 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:00.069+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:00.510 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:00 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:00.069+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:00.741 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:00 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:00.357+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:00.741 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:00 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:46:00.361+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.404649+0000 front 2024-04-25T10:45:28.404521+0000 (oldest deadline 2024-04-25T10:45:54.805047+0000) 2024-04-25T10:46:00.741 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:00 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:46:00.361+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.805512+0000 front 2024-04-25T10:45:34.805567+0000 (oldest deadline 2024-04-25T10:45:58.905587+0000) 2024-04-25T10:46:00.790 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 25 10:46:00 smithi077 podman[70703]: 2024-04-25 10:46:00.697331708 +0000 UTC m=+5.547342922 container died 22404b7aa16a565ad8382156c8fadc4a1957f828dce88abf6b2b96610a9a6d32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-25T10:46:00.790 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:00 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:00.509+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:00.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:00 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:00.487+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:00.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:00 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:00.487+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:01.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:00 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:00.739+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:01.290 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 25 10:46:01 smithi077 podman[70703]: 2024-04-25 10:46:01.079469591 +0000 UTC m=+5.929480799 container cleanup 22404b7aa16a565ad8382156c8fadc4a1957f828dce88abf6b2b96610a9a6d32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T10:46:01.290 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 25 10:46:01 smithi077 bash[70703]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-4 2024-04-25T10:46:01.290 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:01 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:01.039+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:01.291 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:01 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:01.039+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:01.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:00 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:00.970+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:46:01.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:00 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:00.970+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.523733+0000 front 2024-04-25T10:45:35.523709+0000 (oldest deadline 2024-04-25T10:46:00.223726+0000) 2024-04-25T10:46:01.648 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 25 10:46:01 smithi077 podman[70719]: 2024-04-25 10:46:01.338804872 +0000 UTC m=+0.642068761 container remove 22404b7aa16a565ad8382156c8fadc4a1957f828dce88abf6b2b96610a9a6d32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:46:01.649 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:01 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:01.556+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:01.789 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:01 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:01.319+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:01.790 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:01 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:46:01.322+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.404649+0000 front 2024-04-25T10:45:28.404521+0000 (oldest deadline 2024-04-25T10:45:54.805047+0000) 2024-04-25T10:46:01.790 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:01 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:46:01.322+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.805512+0000 front 2024-04-25T10:45:34.805567+0000 (oldest deadline 2024-04-25T10:45:58.905587+0000) 2024-04-25T10:46:01.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:01 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:01.493+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:01.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:01 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:01.493+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:01.967 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 25 10:46:01 smithi077 podman[70813]: 2024-04-25 10:46:01.648018188 +0000 UTC m=+0.104743104 container create 736edfae15628894ef7b7ab456647e564b72001fdaacacac8c0dce5272b9f359 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T10:46:01.968 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 25 10:46:01 smithi077 podman[70813]: 2024-04-25 10:46:01.561415027 +0000 UTC m=+0.018139942 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:46:01.968 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 25 10:46:01 smithi077 podman[70813]: 2024-04-25 10:46:01.891784948 +0000 UTC m=+0.348509864 container init 736edfae15628894ef7b7ab456647e564b72001fdaacacac8c0dce5272b9f359 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-25T10:46:01.968 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 25 10:46:01 smithi077 podman[70813]: 2024-04-25 10:46:01.894821213 +0000 UTC m=+0.351546130 container start 736edfae15628894ef7b7ab456647e564b72001fdaacacac8c0dce5272b9f359 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T10:46:01.968 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 25 10:46:01 smithi077 podman[70813]: 2024-04-25 10:46:01.96666976 +0000 UTC m=+0.423394695 container attach 736edfae15628894ef7b7ab456647e564b72001fdaacacac8c0dce5272b9f359 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T10:46:02.040 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:01 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:01.787+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:02.290 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 25 10:46:02 smithi077 podman[70813]: 2024-04-25 10:46:02.188101646 +0000 UTC m=+0.644826565 container died 736edfae15628894ef7b7ab456647e564b72001fdaacacac8c0dce5272b9f359 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T10:46:02.291 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:02 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:02.044+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:02.291 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:02 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:02.044+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:02.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:01 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:01.953+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:46:02.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:01 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:01.953+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.523733+0000 front 2024-04-25T10:45:35.523709+0000 (oldest deadline 2024-04-25T10:46:00.223726+0000) 2024-04-25T10:46:02.541 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:02 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:46:02.365+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.404649+0000 front 2024-04-25T10:45:28.404521+0000 (oldest deadline 2024-04-25T10:45:54.805047+0000) 2024-04-25T10:46:02.541 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:02 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:46:02.365+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.805512+0000 front 2024-04-25T10:45:34.805567+0000 (oldest deadline 2024-04-25T10:45:58.905587+0000) 2024-04-25T10:46:02.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:02 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:02.276+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:02.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:02 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:02.276+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:02.787 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:02 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:02.566+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:02.804 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T10:46:02.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:46:02.842 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T10:46:02.842 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T10:46:02.842 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.5 2024-04-25T10:46:02.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:02 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:02.509+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:02.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:02 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:02.509+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:03.040 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:02 smithi164 systemd[1]: Stopping Ceph osd.5 for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:46:03.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:02 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:02.742+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:03.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:02 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:02.925+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:46:03.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:02 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:02.925+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.523733+0000 front 2024-04-25T10:45:35.523709+0000 (oldest deadline 2024-04-25T10:46:00.223726+0000) 2024-04-25T10:46:03.414 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:03 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:03.260+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:03.415 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:03 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:03.260+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:03.415 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:03 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:46:03.118+0000 7ff9a5519700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:46:03.415 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:03 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:46:03.118+0000 7ff9a5519700 -1 osd.5 97 *** Got signal Terminated *** 2024-04-25T10:46:03.415 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:03 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:46:03.118+0000 7ff9a5519700 -1 osd.5 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T10:46:03.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:03.045+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:03.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:03.045+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:03.778 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:03 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:46:03.413+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.404649+0000 front 2024-04-25T10:45:28.404521+0000 (oldest deadline 2024-04-25T10:45:54.805047+0000) 2024-04-25T10:46:03.778 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:03 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:46:03.413+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.805512+0000 front 2024-04-25T10:45:34.805567+0000 (oldest deadline 2024-04-25T10:45:58.905587+0000) 2024-04-25T10:46:03.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:03 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:03.543+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:03.896 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:03 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:03.543+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:04.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:03 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:03.614+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:04.040 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:03 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:03.776+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:04.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:03 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:03.776+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:04.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:03 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:03.914+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:46:04.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:03 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:03.914+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.523733+0000 front 2024-04-25T10:45:35.523709+0000 (oldest deadline 2024-04-25T10:46:00.223726+0000) 2024-04-25T10:46:04.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:04 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:04.079+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:04.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:04 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:04.079+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:04.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:04 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:04.218+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:04.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:04 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:04.218+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:04.541 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:04 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:46:04.462+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.404649+0000 front 2024-04-25T10:45:28.404521+0000 (oldest deadline 2024-04-25T10:45:54.805047+0000) 2024-04-25T10:46:04.541 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:04 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:46:04.462+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.805512+0000 front 2024-04-25T10:45:34.805567+0000 (oldest deadline 2024-04-25T10:45:58.905587+0000) 2024-04-25T10:46:04.861 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:04 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:04.513+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:04.861 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:04 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:04.513+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:05.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:04 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:04.639+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:05.236 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:04 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:04.795+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:05.236 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:04 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:04.795+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:05.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:04 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:04.923+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:46:05.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:04 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:04.923+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.523733+0000 front 2024-04-25T10:45:35.523709+0000 (oldest deadline 2024-04-25T10:46:00.223726+0000) 2024-04-25T10:46:05.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:05 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:05.103+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:05.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:05 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:05.103+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:05.540 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:05 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:46:05.450+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.404649+0000 front 2024-04-25T10:45:28.404521+0000 (oldest deadline 2024-04-25T10:45:54.805047+0000) 2024-04-25T10:46:05.541 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:05 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:46:05.450+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.805512+0000 front 2024-04-25T10:45:34.805567+0000 (oldest deadline 2024-04-25T10:45:58.905587+0000) 2024-04-25T10:46:05.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:05 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:05.234+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:05.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:05 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:05.234+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:05.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:05 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:05.530+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:05.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:05 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:05.530+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:06.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:05 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:05.646+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:06.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:05 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:05.646+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:39.190756+0000 front 2024-04-25T10:45:39.190695+0000 (oldest deadline 2024-04-25T10:46:05.090621+0000) 2024-04-25T10:46:06.040 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:05 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:05.775+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:06.040 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:05 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:05.775+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:06.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:05 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:05.951+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:46:06.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:05 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:05.951+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.523733+0000 front 2024-04-25T10:45:35.523709+0000 (oldest deadline 2024-04-25T10:46:00.223726+0000) 2024-04-25T10:46:06.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:06 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:06.139+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:06.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:06 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:06.139+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:06.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:06 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:06.139+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:06.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:06 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:06.284+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:06.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:06 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:06.284+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:06.541 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:06 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:46:06.441+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.404649+0000 front 2024-04-25T10:45:28.404521+0000 (oldest deadline 2024-04-25T10:45:54.805047+0000) 2024-04-25T10:46:06.541 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:06 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:46:06.441+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.805512+0000 front 2024-04-25T10:45:34.805567+0000 (oldest deadline 2024-04-25T10:45:58.905587+0000) 2024-04-25T10:46:06.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:06 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:06.520+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:06.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:06 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:06.520+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:07.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:06 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:06.695+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:07.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:06 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:06.695+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:39.190756+0000 front 2024-04-25T10:45:39.190695+0000 (oldest deadline 2024-04-25T10:46:05.090621+0000) 2024-04-25T10:46:07.040 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:06 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:06.785+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:07.040 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:06 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:06.785+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:07.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:06 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:06.993+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:46:07.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:06 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:06.993+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.523733+0000 front 2024-04-25T10:45:35.523709+0000 (oldest deadline 2024-04-25T10:46:00.223726+0000) 2024-04-25T10:46:07.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:07 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:07.160+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:07.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:07 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:07.160+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:07.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:07 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:07.160+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:07.540 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:07 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:07.276+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:07.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:07 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:07.276+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:07.541 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:07 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:46:07.484+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.404649+0000 front 2024-04-25T10:45:28.404521+0000 (oldest deadline 2024-04-25T10:45:54.805047+0000) 2024-04-25T10:46:07.541 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:07 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:46:07.484+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.805512+0000 front 2024-04-25T10:45:34.805567+0000 (oldest deadline 2024-04-25T10:45:58.905587+0000) 2024-04-25T10:46:07.541 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:07 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5[44837]: 2024-04-25T10:46:07.484+0000 7ff99dac7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:44.106995+0000 front 2024-04-25T10:45:44.107020+0000 (oldest deadline 2024-04-25T10:46:07.007122+0000) 2024-04-25T10:46:07.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:07 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:07.475+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:07.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:07 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:07.475+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:07.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:07 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:07.475+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:08.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:07 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:07.695+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:08.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:07 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:07.695+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:39.190756+0000 front 2024-04-25T10:45:39.190695+0000 (oldest deadline 2024-04-25T10:46:05.090621+0000) 2024-04-25T10:46:08.290 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:07 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:07.800+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:08.291 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:07 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:07.800+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:08.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:08 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:08.035+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:46:08.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:08 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:08.035+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.523733+0000 front 2024-04-25T10:45:35.523709+0000 (oldest deadline 2024-04-25T10:46:00.223726+0000) 2024-04-25T10:46:08.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:08 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:08.035+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.625722+0000 front 2024-04-25T10:45:43.625675+0000 (oldest deadline 2024-04-25T10:46:07.725567+0000) 2024-04-25T10:46:08.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:08 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:08.165+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:08.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:08 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:08.165+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:08.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:08 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:08.165+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:08.585 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:08 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:08.314+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:08.585 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:08 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:08.314+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:08.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:08 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:08.432+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:08.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:08 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:08.432+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:08.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:08 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:08.432+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:08.879 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:08 smithi164 podman[70616]: 2024-04-25 10:46:08.58482574 +0000 UTC m=+5.553019981 container died 5f3815c34c77506e2d367ac1f92a7906eb9521b8aa8d6ee7d3b4e8b5283f3861 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T10:46:08.880 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:08 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:08.763+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:08.880 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:08 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:08.763+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:08.880 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:08 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:08.763+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:09.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:08 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:08.727+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:09.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:08 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:08.727+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:39.190756+0000 front 2024-04-25T10:45:39.190695+0000 (oldest deadline 2024-04-25T10:46:05.090621+0000) 2024-04-25T10:46:09.247 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:08 smithi164 podman[70616]: 2024-04-25 10:46:08.952499128 +0000 UTC m=+5.920693365 container cleanup 5f3815c34c77506e2d367ac1f92a7906eb9521b8aa8d6ee7d3b4e8b5283f3861 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T10:46:09.247 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:08 smithi164 bash[70616]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5 2024-04-25T10:46:09.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:09 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:09.011+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:46:09.361 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:09 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:09.011+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.523733+0000 front 2024-04-25T10:45:35.523709+0000 (oldest deadline 2024-04-25T10:46:00.223726+0000) 2024-04-25T10:46:09.361 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:09 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:09.011+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.625722+0000 front 2024-04-25T10:45:43.625675+0000 (oldest deadline 2024-04-25T10:46:07.725567+0000) 2024-04-25T10:46:09.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:09 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:09.151+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:09.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:09 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:09.151+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:09.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:09 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:09.151+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:09.541 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:09 smithi164 podman[70633]: 2024-04-25 10:46:09.245126019 +0000 UTC m=+0.654399021 container remove 5f3815c34c77506e2d367ac1f92a7906eb9521b8aa8d6ee7d3b4e8b5283f3861 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T10:46:09.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:09 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:09.353+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:09.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:09 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:09.353+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:09.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:09 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:09.353+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:09.849 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:09 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:09.747+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:09.849 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:09 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:09.747+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:09.849 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:09 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:09.747+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:09.849 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:09 smithi164 podman[70722]: 2024-04-25 10:46:09.560297698 +0000 UTC m=+0.107641118 container create 784c3eb82e434d34ae89882313edf30e7d5273c44dcf37627e4d1ce177ee2321 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T10:46:09.849 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:09 smithi164 podman[70722]: 2024-04-25 10:46:09.470632235 +0000 UTC m=+0.017975612 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:46:09.849 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:09 smithi164 podman[70722]: 2024-04-25 10:46:09.773376186 +0000 UTC m=+0.320719563 container init 784c3eb82e434d34ae89882313edf30e7d5273c44dcf37627e4d1ce177ee2321 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T10:46:09.850 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:09 smithi164 podman[70722]: 2024-04-25 10:46:09.776558747 +0000 UTC m=+0.323902125 container start 784c3eb82e434d34ae89882313edf30e7d5273c44dcf37627e4d1ce177ee2321 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T10:46:09.850 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:09 smithi164 podman[70722]: 2024-04-25 10:46:09.848297174 +0000 UTC m=+0.395640562 container attach 784c3eb82e434d34ae89882313edf30e7d5273c44dcf37627e4d1ce177ee2321 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T10:46:09.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:09 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:09.403+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:09.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:09 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:09.403+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:09.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:09 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:09.403+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:10.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:09 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:09.695+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:10.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:09 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:09.695+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:39.190756+0000 front 2024-04-25T10:45:39.190695+0000 (oldest deadline 2024-04-25T10:46:05.090621+0000) 2024-04-25T10:46:10.291 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 25 10:46:10 smithi164 podman[70722]: 2024-04-25 10:46:10.064372718 +0000 UTC m=+0.611716098 container died 784c3eb82e434d34ae89882313edf30e7d5273c44dcf37627e4d1ce177ee2321 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:46:10.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:09.999+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:46:10.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:09.999+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.523733+0000 front 2024-04-25T10:45:35.523709+0000 (oldest deadline 2024-04-25T10:46:00.223726+0000) 2024-04-25T10:46:10.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:09.999+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.625722+0000 front 2024-04-25T10:45:43.625675+0000 (oldest deadline 2024-04-25T10:46:07.725567+0000) 2024-04-25T10:46:10.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:10 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:10.171+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:10.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:10 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:10.171+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:10.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:10 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:10.171+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:10.619 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:10 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:10.341+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:10.619 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:10 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:10.341+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:10.619 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:10 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:10.341+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:10.648 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T10:46:10.684 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:46:10.684 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T10:46:10.684 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T10:46:10.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.6 2024-04-25T10:46:10.709 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:10.425+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:10.709 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:10.425+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:10.709 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:10.425+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:10.983 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:10 smithi043 systemd[1]: Stopping Ceph osd.6 for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:46:10.983 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:10.981+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:46:10.983 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:10.981+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.523733+0000 front 2024-04-25T10:45:35.523709+0000 (oldest deadline 2024-04-25T10:46:00.223726+0000) 2024-04-25T10:46:10.983 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:10 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:10.981+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.625722+0000 front 2024-04-25T10:45:43.625675+0000 (oldest deadline 2024-04-25T10:46:07.725567+0000) 2024-04-25T10:46:11.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:10 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:10.690+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:11.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:10 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:10.690+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:39.190756+0000 front 2024-04-25T10:45:39.190695+0000 (oldest deadline 2024-04-25T10:46:05.090621+0000) 2024-04-25T10:46:11.040 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:10 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:10.735+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:11.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:10 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:10.735+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:11.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:10 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:10.735+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:11.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:11 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:11.128+0000 7f081e1b3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:46:11.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:11 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:11.128+0000 7f081e1b3700 -1 osd.6 97 *** Got signal Terminated *** 2024-04-25T10:46:11.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:11 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:11.128+0000 7f081e1b3700 -1 osd.6 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T10:46:11.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:11 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:11.163+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:11.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:11 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:11.163+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:11.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:11 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:11.163+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:11.757 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:11 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:11.314+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:11.757 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:11 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:11.314+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:11.757 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:11 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:11.314+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:11.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:11 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:11.431+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:11.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:11 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:11.431+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:11.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:11 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:11.431+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:12.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:11 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:11.674+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:12.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:11 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:11.674+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:39.190756+0000 front 2024-04-25T10:45:39.190695+0000 (oldest deadline 2024-04-25T10:46:05.090621+0000) 2024-04-25T10:46:12.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:11 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:11.755+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:12.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:11 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:11.755+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:12.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:11 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:11.755+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:12.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:11 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:11.931+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:46:12.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:11 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:11.931+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.523733+0000 front 2024-04-25T10:45:35.523709+0000 (oldest deadline 2024-04-25T10:46:00.223726+0000) 2024-04-25T10:46:12.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:11 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:11.931+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.625722+0000 front 2024-04-25T10:45:43.625675+0000 (oldest deadline 2024-04-25T10:46:07.725567+0000) 2024-04-25T10:46:12.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:12 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:12.173+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:12.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:12 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:12.173+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:12.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:12 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:12.173+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:12.787 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:12 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:12.333+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:12.787 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:12 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:12.333+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:12.787 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:12 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:12.333+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:12.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:12 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:12.404+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:12.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:12 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:12.404+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:12.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:12 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:12.404+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:13.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:12 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:12.678+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:13.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:12 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:12.678+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:39.190756+0000 front 2024-04-25T10:45:39.190695+0000 (oldest deadline 2024-04-25T10:46:05.090621+0000) 2024-04-25T10:46:13.040 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:12 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:12.786+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:13.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:12 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:12.786+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:13.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:12 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:12.786+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:13.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:12 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:12.975+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:46:13.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:12 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:12.975+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.523733+0000 front 2024-04-25T10:45:35.523709+0000 (oldest deadline 2024-04-25T10:46:00.223726+0000) 2024-04-25T10:46:13.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:12 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:12.975+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.625722+0000 front 2024-04-25T10:45:43.625675+0000 (oldest deadline 2024-04-25T10:46:07.725567+0000) 2024-04-25T10:46:13.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:13 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:13.212+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:13.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:13 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:13.212+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:13.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:13 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:13.212+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:13.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:13 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:13.369+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:13.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:13 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:13.369+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:13.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:13 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:13.369+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:13.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:13 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:13.369+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:13.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:13 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:13.426+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:13.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:13 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:13.426+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:13.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:13 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:13.426+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:14.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:13 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:13.696+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:14.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:13 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:13.696+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:39.190756+0000 front 2024-04-25T10:45:39.190695+0000 (oldest deadline 2024-04-25T10:46:05.090621+0000) 2024-04-25T10:46:14.291 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:13 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:13.809+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:14.291 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:13 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:13.809+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:14.291 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:13 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:13.809+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:14.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:13 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:13.997+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:46:14.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:13 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:13.997+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.523733+0000 front 2024-04-25T10:45:35.523709+0000 (oldest deadline 2024-04-25T10:46:00.223726+0000) 2024-04-25T10:46:14.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:13 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:13.997+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.625722+0000 front 2024-04-25T10:45:43.625675+0000 (oldest deadline 2024-04-25T10:46:07.725567+0000) 2024-04-25T10:46:14.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:13 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:13.997+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:47.725999+0000 front 2024-04-25T10:45:47.725971+0000 (oldest deadline 2024-04-25T10:46:13.626159+0000) 2024-04-25T10:46:14.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:14 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:14.261+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:14.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:14 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:14.261+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:14.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:14 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:14.261+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:14.790 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:14 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:14.383+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:14.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:14 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:14.383+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:14.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:14 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:14.383+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:14.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:14 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:14.383+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:14.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:14 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:14.442+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:14.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:14 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:14.442+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:14.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:14 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:14.442+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:14.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:14 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:14.442+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:15.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:14 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:14.648+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:15.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:14 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:14.648+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:39.190756+0000 front 2024-04-25T10:45:39.190695+0000 (oldest deadline 2024-04-25T10:46:05.090621+0000) 2024-04-25T10:46:15.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:14 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:14.648+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.092601+0000 front 2024-04-25T10:45:52.092569+0000 (oldest deadline 2024-04-25T10:46:14.392547+0000) 2024-04-25T10:46:15.290 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:14 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:14.794+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:15.291 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:14 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:14.794+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:15.291 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:14 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:14.794+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:15.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:15 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:15.035+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:46:15.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:15 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:15.035+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.523733+0000 front 2024-04-25T10:45:35.523709+0000 (oldest deadline 2024-04-25T10:46:00.223726+0000) 2024-04-25T10:46:15.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:15 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:15.035+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.625722+0000 front 2024-04-25T10:45:43.625675+0000 (oldest deadline 2024-04-25T10:46:07.725567+0000) 2024-04-25T10:46:15.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:15 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:15.035+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:47.725999+0000 front 2024-04-25T10:45:47.725971+0000 (oldest deadline 2024-04-25T10:46:13.626159+0000) 2024-04-25T10:46:15.627 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:15 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:15.296+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:15.627 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:15 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:15.296+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:15.627 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:15 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:15.296+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:15.782 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:15 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:15.355+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:15.783 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:15 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:15.355+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:15.783 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:15 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:15.355+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:15.783 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:15 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:15.355+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:15.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:15 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:15.491+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:15.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:15 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:15.491+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:15.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:15 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:15.491+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:15.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:15 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:15.491+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:16.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:15 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:15.625+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:16.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:15 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:15.625+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:39.190756+0000 front 2024-04-25T10:45:39.190695+0000 (oldest deadline 2024-04-25T10:46:05.090621+0000) 2024-04-25T10:46:16.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:15 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:15.625+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.092601+0000 front 2024-04-25T10:45:52.092569+0000 (oldest deadline 2024-04-25T10:46:14.392547+0000) 2024-04-25T10:46:16.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:15 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:15.780+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:16.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:15 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:15.780+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:16.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:15 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:15.780+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:16.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:16 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:16.076+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:23.920964+0000 front 2024-04-25T10:45:23.920990+0000 (oldest deadline 2024-04-25T10:45:49.821158+0000) 2024-04-25T10:46:16.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:16 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:16.076+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.523733+0000 front 2024-04-25T10:45:35.523709+0000 (oldest deadline 2024-04-25T10:46:00.223726+0000) 2024-04-25T10:46:16.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:16 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:16.076+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.625722+0000 front 2024-04-25T10:45:43.625675+0000 (oldest deadline 2024-04-25T10:46:07.725567+0000) 2024-04-25T10:46:16.361 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:16 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6[55099]: 2024-04-25T10:46:16.076+0000 7f0816761700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:47.725999+0000 front 2024-04-25T10:45:47.725971+0000 (oldest deadline 2024-04-25T10:46:13.626159+0000) 2024-04-25T10:46:16.637 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:16 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:16.341+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:16.637 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:16 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:16.341+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:16.637 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:16 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:16.341+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:16.637 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:16 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:16.341+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:16.790 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:16 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:16.390+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:16.790 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:16 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:16.390+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:16.790 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:16 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:16.390+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:16.790 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:16 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:16.390+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:16.860 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:16 smithi043 podman[98949]: 2024-04-25 10:46:16.662625843 +0000 UTC m=+5.787946657 container died b0e7d1a71407f66469c3d9d377279903467101f26111dd29e50e5eaead2792f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-25T10:46:16.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:16 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:16.448+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:16.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:16 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:16.448+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:16.861 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:16 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:16.448+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:16.861 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:16 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:16.448+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:17.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:16 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:16.635+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:17.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:16 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:16.635+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:39.190756+0000 front 2024-04-25T10:45:39.190695+0000 (oldest deadline 2024-04-25T10:46:05.090621+0000) 2024-04-25T10:46:17.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:16 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:16.635+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.092601+0000 front 2024-04-25T10:45:52.092569+0000 (oldest deadline 2024-04-25T10:46:14.392547+0000) 2024-04-25T10:46:17.040 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:16 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:16.788+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:17.040 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:16 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:16.788+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:17.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:16 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:16.788+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:17.130 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:16 smithi043 podman[98949]: 2024-04-25 10:46:16.936143451 +0000 UTC m=+6.061464319 container cleanup b0e7d1a71407f66469c3d9d377279903467101f26111dd29e50e5eaead2792f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T10:46:17.130 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:16 smithi043 bash[98949]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6 2024-04-25T10:46:17.459 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:17 smithi043 podman[98967]: 2024-04-25 10:46:17.129087819 +0000 UTC m=+0.464487923 container remove b0e7d1a71407f66469c3d9d377279903467101f26111dd29e50e5eaead2792f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T10:46:17.592 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:17 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:17.299+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:17.592 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:17 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:17.299+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:17.592 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:17 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:17.299+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:17.592 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:17 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:17.299+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:17.762 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:17 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:17.366+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:17.762 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:17 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:17.366+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:17.762 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:17 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:17.366+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:17.762 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:17 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:17.366+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:17.799 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:17 smithi043 podman[99057]: 2024-04-25 10:46:17.481581184 +0000 UTC m=+0.018415598 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:46:17.799 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:17 smithi043 podman[99057]: 2024-04-25 10:46:17.57964277 +0000 UTC m=+0.116477174 container create cf2ac43fa70734a1e51c33d8a69fdd32277f57c59f18b1d35735f983b46bf3cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T10:46:17.800 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:17 smithi043 podman[99057]: 2024-04-25 10:46:17.7987644 +0000 UTC m=+0.335598805 container init cf2ac43fa70734a1e51c33d8a69fdd32277f57c59f18b1d35735f983b46bf3cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-25T10:46:17.800 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:17 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:17.457+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:17.800 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:17 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:17.457+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:17.800 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:17 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:17.457+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:17.800 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:17 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:17.457+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:18.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:17 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:17.590+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:18.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:17 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:17.590+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:39.190756+0000 front 2024-04-25T10:45:39.190695+0000 (oldest deadline 2024-04-25T10:46:05.090621+0000) 2024-04-25T10:46:18.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:17 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:17.590+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.092601+0000 front 2024-04-25T10:45:52.092569+0000 (oldest deadline 2024-04-25T10:46:14.392547+0000) 2024-04-25T10:46:18.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:17 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:17.759+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:18.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:17 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:17.759+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:18.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:17 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:17.759+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:18.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:17 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:17.759+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.234443+0000 front 2024-04-25T10:45:52.234530+0000 (oldest deadline 2024-04-25T10:46:16.934508+0000) 2024-04-25T10:46:18.110 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:17 smithi043 podman[99057]: 2024-04-25 10:46:17.802233123 +0000 UTC m=+0.339067527 container start cf2ac43fa70734a1e51c33d8a69fdd32277f57c59f18b1d35735f983b46bf3cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T10:46:18.110 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:17 smithi043 podman[99057]: 2024-04-25 10:46:17.890380493 +0000 UTC m=+0.427214908 container attach cf2ac43fa70734a1e51c33d8a69fdd32277f57c59f18b1d35735f983b46bf3cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T10:46:18.436 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 25 10:46:18 smithi043 podman[99057]: 2024-04-25 10:46:18.110108474 +0000 UTC m=+0.646942879 container died cf2ac43fa70734a1e51c33d8a69fdd32277f57c59f18b1d35735f983b46bf3cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-25T10:46:18.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:18 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:18.270+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:18.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:18 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:18.270+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:18.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:18 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:18.270+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:18.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:18 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:18.270+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:18.781 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:18 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:18.387+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:18.781 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:18 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:18.387+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:18.781 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:18 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:18.387+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:18.781 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:18 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:18.387+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:18.786 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:18 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:18.435+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:18.786 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:18 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:18.435+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:18.786 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:18 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:18.435+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:18.786 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:18 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:18.435+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:18.815 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T10:46:18.903 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:46:18.903 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T10:46:18.903 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T10:46:18.903 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.7 2024-04-25T10:46:18.929 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:18 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:18.637+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:18.929 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:18 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:18.637+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:39.190756+0000 front 2024-04-25T10:45:39.190695+0000 (oldest deadline 2024-04-25T10:46:05.090621+0000) 2024-04-25T10:46:18.929 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:18 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:18.637+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.092601+0000 front 2024-04-25T10:45:52.092569+0000 (oldest deadline 2024-04-25T10:46:14.392547+0000) 2024-04-25T10:46:19.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:18 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:18.779+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:19.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:18 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:18.779+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:19.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:18 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:18.779+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:19.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:18 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:18.779+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.234443+0000 front 2024-04-25T10:45:52.234530+0000 (oldest deadline 2024-04-25T10:46:16.934508+0000) 2024-04-25T10:46:19.198 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:18 smithi077 systemd[1]: Stopping Ceph osd.7 for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:46:19.540 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:19 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:19.196+0000 7f8349cc0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:46:19.540 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:19 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:19.196+0000 7f8349cc0700 -1 osd.7 97 *** Got signal Terminated *** 2024-04-25T10:46:19.540 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:19 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:19.196+0000 7f8349cc0700 -1 osd.7 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T10:46:19.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:19 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:19.225+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:19.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:19 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:19.225+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:19.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:19 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:19.225+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:19.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:19 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:19.225+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:19.757 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:19 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:19.340+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:19.757 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:19 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:19.340+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:19.757 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:19 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:19.340+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:19.757 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:19 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:19.340+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:19.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:19 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:19.448+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:19.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:19 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:19.448+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:19.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:19 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:19.448+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:19.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:19 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:19.448+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:20.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:19 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:19.620+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:20.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:19 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:19.620+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:39.190756+0000 front 2024-04-25T10:45:39.190695+0000 (oldest deadline 2024-04-25T10:46:05.090621+0000) 2024-04-25T10:46:20.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:19 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:19.620+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.092601+0000 front 2024-04-25T10:45:52.092569+0000 (oldest deadline 2024-04-25T10:46:14.392547+0000) 2024-04-25T10:46:20.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:19 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:19.755+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:20.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:19 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:19.755+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:20.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:19 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:19.755+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:20.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:19 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:19.755+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.234443+0000 front 2024-04-25T10:45:52.234530+0000 (oldest deadline 2024-04-25T10:46:16.934508+0000) 2024-04-25T10:46:20.312 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:20 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:20.268+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:20.312 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:20 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:20.268+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:20.312 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:20 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:20.268+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:20.312 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:20 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:20.268+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:20.716 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:20 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:20.300+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:20.716 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:20 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:20.300+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:20.716 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:20 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:20.300+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:20.716 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:20 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:20.300+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:20.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:20 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:20.441+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:20.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:20 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:20.441+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:20.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:20 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:20.441+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:20.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:20 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:20.441+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:21.042 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:20 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:20.606+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:21.042 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:20 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:20.606+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:39.190756+0000 front 2024-04-25T10:45:39.190695+0000 (oldest deadline 2024-04-25T10:46:05.090621+0000) 2024-04-25T10:46:21.042 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:20 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:20.606+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.092601+0000 front 2024-04-25T10:45:52.092569+0000 (oldest deadline 2024-04-25T10:46:14.392547+0000) 2024-04-25T10:46:21.042 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:20 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:20.715+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:21.042 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:20 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:20.715+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:21.042 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:20 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:20.715+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:21.042 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:20 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:20.715+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.234443+0000 front 2024-04-25T10:45:52.234530+0000 (oldest deadline 2024-04-25T10:46:16.934508+0000) 2024-04-25T10:46:21.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:21 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:21.256+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:21.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:21 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:21.256+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:21.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:21 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:21.256+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:21.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:21 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:21.256+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:21.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:21 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:21.257+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:21.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:21 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:21.257+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:21.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:21 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:21.257+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:21.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:21 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:21.257+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:21.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:21 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:21.476+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:21.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:21 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:21.476+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:21.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:21 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:21.476+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:21.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:21 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:21.476+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:22.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:21 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:21.575+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:22.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:21 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:21.575+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:39.190756+0000 front 2024-04-25T10:45:39.190695+0000 (oldest deadline 2024-04-25T10:46:05.090621+0000) 2024-04-25T10:46:22.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:21 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:21.575+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.092601+0000 front 2024-04-25T10:45:52.092569+0000 (oldest deadline 2024-04-25T10:46:14.392547+0000) 2024-04-25T10:46:22.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:21 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:21.575+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:58.994000+0000 front 2024-04-25T10:45:58.993979+0000 (oldest deadline 2024-04-25T10:46:20.694154+0000) 2024-04-25T10:46:22.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:21 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:21.673+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:22.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:21 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:21.673+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:22.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:21 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:21.673+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:22.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:21 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:21.673+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.234443+0000 front 2024-04-25T10:45:52.234530+0000 (oldest deadline 2024-04-25T10:46:16.934508+0000) 2024-04-25T10:46:22.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:22 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:22.240+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:22.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:22 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:22.240+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:22.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:22 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:22.240+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:22.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:22 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:22.240+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:22.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:22 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:22.276+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:22.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:22 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:22.276+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:22.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:22 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:22.276+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:22.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:22 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:22.276+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:22.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:22 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:22.276+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:22.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:22 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:22.457+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:22.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:22 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:22.457+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:22.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:22 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:22.457+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:22.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:22 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:22.457+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:23.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:22 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:22.580+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:23.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:22 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:22.580+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:39.190756+0000 front 2024-04-25T10:45:39.190695+0000 (oldest deadline 2024-04-25T10:46:05.090621+0000) 2024-04-25T10:46:23.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:22 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:22.580+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.092601+0000 front 2024-04-25T10:45:52.092569+0000 (oldest deadline 2024-04-25T10:46:14.392547+0000) 2024-04-25T10:46:23.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:22 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:22.580+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:58.994000+0000 front 2024-04-25T10:45:58.993979+0000 (oldest deadline 2024-04-25T10:46:20.694154+0000) 2024-04-25T10:46:23.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:22 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:22.722+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:23.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:22 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:22.722+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:23.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:22 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:22.722+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:23.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:22 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:22.722+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.234443+0000 front 2024-04-25T10:45:52.234530+0000 (oldest deadline 2024-04-25T10:46:16.934508+0000) 2024-04-25T10:46:23.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:23 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:23.221+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:23.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:23 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:23.221+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:23.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:23 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:23.221+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:23.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:23 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:23.221+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:23.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:23 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:23.221+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:23.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:23 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:23.247+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:23.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:23 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:23.247+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:23.542 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:23 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:23.247+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:23.542 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:23 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:23.247+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:23.542 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:23 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:23.247+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:23.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:23 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:23.449+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:23.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:23 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:23.449+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:23.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:23 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:23.449+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:23.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:23 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:23.449+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:24.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:23 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:23.558+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.689740+0000 front 2024-04-25T10:45:35.689688+0000 (oldest deadline 2024-04-25T10:45:59.189884+0000) 2024-04-25T10:46:24.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:23 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:23.558+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:39.190756+0000 front 2024-04-25T10:45:39.190695+0000 (oldest deadline 2024-04-25T10:46:05.090621+0000) 2024-04-25T10:46:24.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:23 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:23.558+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.092601+0000 front 2024-04-25T10:45:52.092569+0000 (oldest deadline 2024-04-25T10:46:14.392547+0000) 2024-04-25T10:46:24.040 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:23 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7[51097]: 2024-04-25T10:46:23.558+0000 7f834226e700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:58.994000+0000 front 2024-04-25T10:45:58.993979+0000 (oldest deadline 2024-04-25T10:46:20.694154+0000) 2024-04-25T10:46:24.040 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:23 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:23.770+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:24.040 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:23 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:23.770+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:24.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:23 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:23.770+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:24.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:23 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:23.770+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.234443+0000 front 2024-04-25T10:45:52.234530+0000 (oldest deadline 2024-04-25T10:46:16.934508+0000) 2024-04-25T10:46:24.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:24 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:24.252+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:24.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:24 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:24.252+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:24.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:24 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:24.252+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:24.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:24 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:24.252+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:24.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:24 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:24.252+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:24.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:24 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:24.227+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:24.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:24 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:24.227+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:24.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:24 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:24.227+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:24.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:24 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:24.227+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:24.542 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:24 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:24.227+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:24.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:24 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:24.409+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:24.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:24 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:24.409+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:24.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:24 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:24.409+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:24.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:24 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:24.409+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:24.982 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:24 smithi077 podman[70916]: 2024-04-25 10:46:24.694743745 +0000 UTC m=+5.605179072 container died 2a820433855b68f8e9dc3285a93cc51896ae9cd6723c7b5238aab993128e1087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T10:46:25.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:24 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:24.783+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:25.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:24 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:24.783+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:25.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:24 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:24.783+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:25.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:24 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:24.783+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.234443+0000 front 2024-04-25T10:45:52.234530+0000 (oldest deadline 2024-04-25T10:46:16.934508+0000) 2024-04-25T10:46:25.290 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:25 smithi077 podman[70916]: 2024-04-25 10:46:25.072245152 +0000 UTC m=+5.982680488 container cleanup 2a820433855b68f8e9dc3285a93cc51896ae9cd6723c7b5238aab993128e1087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T10:46:25.290 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:25 smithi077 bash[70916]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7 2024-04-25T10:46:25.290 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:25 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:25.209+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:25.290 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:25 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:25.209+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:25.290 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:25 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:25.209+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:25.290 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:25 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:25.209+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:25.291 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:25 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:25.209+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:25.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:25 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:25.209+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:25.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:25 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:25.209+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:25.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:25 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:25.209+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:25.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:25 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:25.209+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:25.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:25 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:25.209+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:25.708 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:25 smithi077 podman[70947]: 2024-04-25 10:46:25.398884662 +0000 UTC m=+0.694231402 container remove 2a820433855b68f8e9dc3285a93cc51896ae9cd6723c7b5238aab993128e1087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T10:46:25.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:25 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:25.431+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:25.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:25 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:25.431+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:25.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:25 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:25.431+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:25.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:25 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:25.431+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:25.995 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:25 smithi077 podman[71037]: 2024-04-25 10:46:25.708055705 +0000 UTC m=+0.113957631 container create 244522dfdb727b526cf062a0fc938ccfb15f4414dbe545669d4336323eff772a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T10:46:25.995 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:25 smithi077 podman[71037]: 2024-04-25 10:46:25.612499162 +0000 UTC m=+0.018401053 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:46:25.995 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:25 smithi077 podman[71037]: 2024-04-25 10:46:25.910107804 +0000 UTC m=+0.316009683 container init 244522dfdb727b526cf062a0fc938ccfb15f4414dbe545669d4336323eff772a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T10:46:25.995 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:25 smithi077 podman[71037]: 2024-04-25 10:46:25.913016071 +0000 UTC m=+0.318917966 container start 244522dfdb727b526cf062a0fc938ccfb15f4414dbe545669d4336323eff772a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T10:46:25.995 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:25 smithi077 podman[71037]: 2024-04-25 10:46:25.993391416 +0000 UTC m=+0.399293305 container attach 244522dfdb727b526cf062a0fc938ccfb15f4414dbe545669d4336323eff772a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T10:46:26.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:25 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:25.756+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:26.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:25 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:25.756+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:26.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:25 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:25.756+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:26.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:25 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:25.756+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.234443+0000 front 2024-04-25T10:45:52.234530+0000 (oldest deadline 2024-04-25T10:46:16.934508+0000) 2024-04-25T10:46:26.290 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 25 10:46:26 smithi077 podman[71037]: 2024-04-25 10:46:26.198232529 +0000 UTC m=+0.604134401 container died 244522dfdb727b526cf062a0fc938ccfb15f4414dbe545669d4336323eff772a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T10:46:26.290 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:26 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:26.232+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:26.291 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:26 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:26.232+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:26.291 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:26 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:26.232+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:26.291 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:26 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:26.232+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:26.291 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:26 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:26.232+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:26.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:26 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:26.177+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:26.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:26 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:26.177+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:26.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:26 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:26.177+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:26.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:26 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:26.177+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:26.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:26 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:26.177+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:26.828 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T10:46:26.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:26 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:26.395+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:26.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:26 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:26.395+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:26.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:26 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:26.395+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:26.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:26 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:26.395+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:26.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:26 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:26.395+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.595902+0000 front 2024-04-25T10:45:59.596166+0000 (oldest deadline 2024-04-25T10:46:25.495819+0000) 2024-04-25T10:46:26.906 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:46:26.906 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T10:46:26.906 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T10:46:26.906 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.8 2024-04-25T10:46:27.141 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:26 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:26.805+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:27.141 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:26 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:26.805+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:27.141 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:26 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:26.805+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:27.142 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:26 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:26.805+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.234443+0000 front 2024-04-25T10:45:52.234530+0000 (oldest deadline 2024-04-25T10:46:16.934508+0000) 2024-04-25T10:46:27.142 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:26 smithi164 systemd[1]: Stopping Ceph osd.8 for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:46:27.493 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:27 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:27.253+0000 7f5dfb1d1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:46:27.493 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:27 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:27.253+0000 7f5dfb1d1700 -1 osd.8 97 *** Got signal Terminated *** 2024-04-25T10:46:27.493 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:27 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:27.253+0000 7f5dfb1d1700 -1 osd.8 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T10:46:27.493 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:27 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:27.138+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:27.493 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:27 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:27.139+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:27.493 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:27 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:27.139+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:27.493 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:27 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:27.139+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:27.493 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:27 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:27.139+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:27.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:27 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:27.222+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:27.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:27 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:27.222+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:27.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:27 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:27.222+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:27.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:27 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:27.222+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:27.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:27 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:27.222+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:27.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:27 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:27.424+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:27.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:27 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:27.424+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:27.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:27 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:27.424+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:27.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:27 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:27.424+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:27.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:27 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:27.424+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.595902+0000 front 2024-04-25T10:45:59.596166+0000 (oldest deadline 2024-04-25T10:46:25.495819+0000) 2024-04-25T10:46:28.040 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:27 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:27.775+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:28.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:27 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:27.775+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:28.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:27 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:27.775+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:28.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:27 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:27.775+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.234443+0000 front 2024-04-25T10:45:52.234530+0000 (oldest deadline 2024-04-25T10:46:16.934508+0000) 2024-04-25T10:46:28.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:28 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:28.269+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:28.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:28 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:28.269+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:28.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:28 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:28.269+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:28.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:28 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:28.269+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:28.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:28 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:28.269+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:28.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:28 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:28.148+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:28.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:28 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:28.148+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:28.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:28 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:28.148+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:28.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:28 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:28.148+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:28.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:28 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:28.148+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:28.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:28 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:28.434+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:28.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:28 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:28.434+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:28.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:28 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:28.434+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:28.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:28 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:28.434+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:28.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:28 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:28.434+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.595902+0000 front 2024-04-25T10:45:59.596166+0000 (oldest deadline 2024-04-25T10:46:25.495819+0000) 2024-04-25T10:46:29.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:28 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:28.734+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:29.042 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:28 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:28.734+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:29.042 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:28 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:28.734+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:29.042 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:28 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:28.734+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.234443+0000 front 2024-04-25T10:45:52.234530+0000 (oldest deadline 2024-04-25T10:46:16.934508+0000) 2024-04-25T10:46:29.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:29 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:29.270+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:29.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:29 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:29.270+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:29.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:29 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:29.270+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:29.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:29 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:29.270+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:29.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:29 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:29.270+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:29.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:29 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:29.162+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:29.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:29 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:29.162+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:29.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:29 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:29.162+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:29.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:29 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:29.162+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:29.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:29 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:29.162+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:29.862 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:29 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:29.388+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:29.862 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:29 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:29.388+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:29.862 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:29 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:29.388+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:29.862 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:29 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:29.388+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:29.862 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:29 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:29.388+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.595902+0000 front 2024-04-25T10:45:59.596166+0000 (oldest deadline 2024-04-25T10:46:25.495819+0000) 2024-04-25T10:46:30.040 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:29 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:29.778+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:30.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:29 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:29.779+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:30.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:29 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:29.779+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:30.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:29 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:29.779+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.234443+0000 front 2024-04-25T10:45:52.234530+0000 (oldest deadline 2024-04-25T10:46:16.934508+0000) 2024-04-25T10:46:30.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:30 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:30.284+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:30.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:30 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:30.284+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:30.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:30 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:30.284+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:30.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:30 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:30.284+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:30.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:30 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:30.284+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:30.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:30 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:30.284+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:03.095406+0000 front 2024-04-25T10:46:03.095252+0000 (oldest deadline 2024-04-25T10:46:29.495714+0000) 2024-04-25T10:46:30.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:30 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:30.195+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:30.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:30 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:30.195+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:30.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:30 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:30.195+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:30.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:30 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:30.195+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:30.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:30 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:30.195+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:30.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:30 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:30.195+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:30.761 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:30 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:30.390+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:30.762 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:30 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:30.390+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:30.762 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:30 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:30.390+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:30.762 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:30 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:30.390+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:30.762 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:30 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:30.390+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.595902+0000 front 2024-04-25T10:45:59.596166+0000 (oldest deadline 2024-04-25T10:46:25.495819+0000) 2024-04-25T10:46:31.209 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:30 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:30.820+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:31.209 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:30 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:30.820+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:31.209 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:30 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:30.820+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:31.210 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:30 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:30.820+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.234443+0000 front 2024-04-25T10:45:52.234530+0000 (oldest deadline 2024-04-25T10:46:16.934508+0000) 2024-04-25T10:46:31.540 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:31 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:31.208+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:31.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:31 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:31.208+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:31.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:31 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:31.208+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:31.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:31 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:31.208+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:31.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:31 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:31.208+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:31.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:31 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:31.208+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:31.783 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:31 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:31.416+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:31.784 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:31 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:31.416+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:31.784 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:31 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:31.416+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:31.784 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:31 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:31.416+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:31.784 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:31 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:31.416+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.595902+0000 front 2024-04-25T10:45:59.596166+0000 (oldest deadline 2024-04-25T10:46:25.495819+0000) 2024-04-25T10:46:31.790 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:31 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:31.291+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:31.790 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:31 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:31.291+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:31.790 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:31 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:31.291+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:31.790 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:31 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:31.291+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:31.790 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:31 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:31.291+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:31.790 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:31 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:31.291+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:03.095406+0000 front 2024-04-25T10:46:03.095252+0000 (oldest deadline 2024-04-25T10:46:29.495714+0000) 2024-04-25T10:46:32.256 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:31 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:31.842+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:27.730476+0000 front 2024-04-25T10:45:27.730248+0000 (oldest deadline 2024-04-25T10:45:53.030301+0000) 2024-04-25T10:46:32.256 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:31 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:31.842+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:33.031158+0000 front 2024-04-25T10:45:33.031107+0000 (oldest deadline 2024-04-25T10:46:03.031599+0000) 2024-04-25T10:46:32.256 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:31 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:31.842+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.032102+0000 front 2024-04-25T10:45:43.032181+0000 (oldest deadline 2024-04-25T10:46:08.332159+0000) 2024-04-25T10:46:32.256 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:31 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:31.842+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:52.234443+0000 front 2024-04-25T10:45:52.234530+0000 (oldest deadline 2024-04-25T10:46:16.934508+0000) 2024-04-25T10:46:32.256 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:31 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8[51854]: 2024-04-25T10:46:31.842+0000 7f5df377f700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:04.436702+0000 front 2024-04-25T10:46:04.436759+0000 (oldest deadline 2024-04-25T10:46:31.437549+0000) 2024-04-25T10:46:32.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:32 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:32.252+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:32.542 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:32 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:32.252+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:32.542 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:32 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:32.252+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:32.542 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:32 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:32.252+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:32.542 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:32 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:32.252+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:32.542 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:32 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:32.252+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:03.095406+0000 front 2024-04-25T10:46:03.095252+0000 (oldest deadline 2024-04-25T10:46:29.495714+0000) 2024-04-25T10:46:32.543 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:32 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:32.253+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:32.543 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:32 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:32.253+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:32.543 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:32 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:32.253+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:32.543 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:32 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:32.253+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:32.543 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:32 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:32.253+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:32.543 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:32 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:32.253+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:32.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:32 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:32.382+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:32.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:32 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:32.382+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:32.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:32 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:32.382+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:32.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:32 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:32.382+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:32.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:32 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:32.382+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.595902+0000 front 2024-04-25T10:45:59.596166+0000 (oldest deadline 2024-04-25T10:46:25.495819+0000) 2024-04-25T10:46:32.865 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:32 smithi164 podman[70841]: 2024-04-25 10:46:32.592518371 +0000 UTC m=+5.484832887 container died 2b8f1683cd08105f1558a51ab2cc3489a0f7a1c85f010d06aaca470c70d511bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T10:46:33.199 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:32 smithi164 podman[70841]: 2024-04-25 10:46:32.963551056 +0000 UTC m=+5.855865573 container cleanup 2b8f1683cd08105f1558a51ab2cc3489a0f7a1c85f010d06aaca470c70d511bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T10:46:33.199 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:32 smithi164 bash[70841]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8 2024-04-25T10:46:33.500 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:33 smithi164 podman[70858]: 2024-04-25 10:46:33.198638393 +0000 UTC m=+0.607503116 container remove 2b8f1683cd08105f1558a51ab2cc3489a0f7a1c85f010d06aaca470c70d511bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T10:46:33.500 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:33 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:33.223+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:33.500 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:33 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:33.223+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:33.501 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:33 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:33.223+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:33.501 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:33 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:33.223+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:33.501 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:33 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:33.223+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:33.501 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:33 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:33.223+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:33.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:33 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:33.251+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:33.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:33 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:33.251+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:33.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:33 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:33.251+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:33.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:33 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:33.251+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:33.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:33 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:33.251+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:33.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:33 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:33.251+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:03.095406+0000 front 2024-04-25T10:46:03.095252+0000 (oldest deadline 2024-04-25T10:46:29.495714+0000) 2024-04-25T10:46:33.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:33 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:33.350+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:33.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:33 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:33.350+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:33.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:33 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:33.350+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:33.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:33 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:33.350+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:33.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:33 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:33.350+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.595902+0000 front 2024-04-25T10:45:59.596166+0000 (oldest deadline 2024-04-25T10:46:25.495819+0000) 2024-04-25T10:46:33.791 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:33 smithi164 podman[70952]: 2024-04-25 10:46:33.499394663 +0000 UTC m=+0.106473357 container create 3f4a0d00fa2e7bbad1466d3652f8e3cf0c081983631bdda71c09824eb9006859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T10:46:33.791 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:33 smithi164 podman[70952]: 2024-04-25 10:46:33.411354958 +0000 UTC m=+0.018433630 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:46:33.791 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:33 smithi164 podman[70952]: 2024-04-25 10:46:33.735080027 +0000 UTC m=+0.342158678 container init 3f4a0d00fa2e7bbad1466d3652f8e3cf0c081983631bdda71c09824eb9006859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T10:46:33.791 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:33 smithi164 podman[70952]: 2024-04-25 10:46:33.738469621 +0000 UTC m=+0.345548277 container start 3f4a0d00fa2e7bbad1466d3652f8e3cf0c081983631bdda71c09824eb9006859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T10:46:34.059 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:33 smithi164 podman[70952]: 2024-04-25 10:46:33.793179609 +0000 UTC m=+0.400258270 container attach 3f4a0d00fa2e7bbad1466d3652f8e3cf0c081983631bdda71c09824eb9006859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T10:46:34.060 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:34 smithi164 podman[70952]: 2024-04-25 10:46:34.035503436 +0000 UTC m=+0.642582085 container died 3f4a0d00fa2e7bbad1466d3652f8e3cf0c081983631bdda71c09824eb9006859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T10:46:34.311 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:34 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:34.236+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:34.311 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:34 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:34.236+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:34.311 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:34 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:34.236+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:34.311 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:34 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:34.236+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:34.311 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:34 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:34.236+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:34.311 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:34 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:34.236+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:34.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:34 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:34.250+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:34.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:34 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:34.250+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:34.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:34 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:34.250+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:34.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:34 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:34.250+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:34.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:34 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:34.250+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:34.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:34 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:34.250+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:03.095406+0000 front 2024-04-25T10:46:03.095252+0000 (oldest deadline 2024-04-25T10:46:29.495714+0000) 2024-04-25T10:46:34.549 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T10:46:34.571 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:34 smithi164 podman[70978]: 2024-04-25 10:46:34.530963855 +0000 UTC m=+0.485781973 container remove 3f4a0d00fa2e7bbad1466d3652f8e3cf0c081983631bdda71c09824eb9006859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-25T10:46:34.571 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:34 smithi164 systemd[1]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.8.service: Deactivated successfully. 2024-04-25T10:46:34.571 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:34 smithi164 systemd[1]: Stopped Ceph osd.8 for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:46:34.572 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 25 10:46:34 smithi164 systemd[1]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.8.service: Consumed 13.077s CPU time. 2024-04-25T10:46:34.586 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:46:34.587 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T10:46:34.587 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T10:46:34.587 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.9 2024-04-25T10:46:34.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:34 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:34.302+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:34.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:34 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:34.302+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:34.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:34 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:34.302+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:34.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:34 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:34.302+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:34.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:34 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:34.302+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.595902+0000 front 2024-04-25T10:45:59.596166+0000 (oldest deadline 2024-04-25T10:46:25.495819+0000) 2024-04-25T10:46:34.893 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:34 smithi043 systemd[1]: Stopping Ceph osd.9 for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:46:35.267 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:34 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:34.891+0000 7f84d94e5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:46:35.268 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:34 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:34.891+0000 7f84d94e5700 -1 osd.9 97 *** Got signal Terminated *** 2024-04-25T10:46:35.268 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:34 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:34.891+0000 7f84d94e5700 -1 osd.9 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T10:46:35.268 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:35 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:35.266+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:35.268 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:35 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:35.266+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:35.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:35 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:35.278+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:35.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:35 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:35.278+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:35.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:35 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:35.278+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:35.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:35 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:35.278+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:35.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:35 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:35.278+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:35.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:35 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:35.278+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:03.095406+0000 front 2024-04-25T10:46:03.095252+0000 (oldest deadline 2024-04-25T10:46:29.495714+0000) 2024-04-25T10:46:35.542 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:35 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:35.216+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:35.542 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:35 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:35.216+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:35.542 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:35 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:35.216+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:35.542 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:35 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:35.216+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:35.543 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:35 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:35.216+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:35.543 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:35 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:35.216+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:35.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:35 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:35.266+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:35.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:35 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:35.266+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:35.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:35 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:35.266+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.595902+0000 front 2024-04-25T10:45:59.596166+0000 (oldest deadline 2024-04-25T10:46:25.495819+0000) 2024-04-25T10:46:36.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:36 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:36.234+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:36.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:36 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:36.234+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:36.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:36 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:36.234+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:36.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:36 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:36.234+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:36.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:36 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:36.234+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:36.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:36 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:36.234+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:03.095406+0000 front 2024-04-25T10:46:03.095252+0000 (oldest deadline 2024-04-25T10:46:29.495714+0000) 2024-04-25T10:46:36.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:36 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:36.257+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:36.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:36 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:36.257+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:36.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:36 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:36.257+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:36.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:36 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:36.257+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:36.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:36 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:36.257+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:36.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:36 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:36.257+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:36.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:36 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:36.246+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:36.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:36 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:36.246+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:36.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:36 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:36.246+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:36.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:36 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:36.246+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:36.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:36 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:36.246+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.595902+0000 front 2024-04-25T10:45:59.596166+0000 (oldest deadline 2024-04-25T10:46:25.495819+0000) 2024-04-25T10:46:37.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:37 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:37.231+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:37.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:37 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:37.231+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:37.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:37 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:37.231+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:37.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:37 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:37.231+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:37.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:37 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:37.231+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:37.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:37 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:37.231+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:03.095406+0000 front 2024-04-25T10:46:03.095252+0000 (oldest deadline 2024-04-25T10:46:29.495714+0000) 2024-04-25T10:46:37.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:37 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:37.213+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:37.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:37 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:37.213+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:37.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:37 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:37.213+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:37.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:37 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:37.213+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:37.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:37 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:37.213+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:37.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:37 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:37.213+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:37.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:37 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:37.291+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:37.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:37 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:37.291+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:37.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:37 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:37.291+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:37.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:37 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:37.291+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:37.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:37 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:37.291+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.595902+0000 front 2024-04-25T10:45:59.596166+0000 (oldest deadline 2024-04-25T10:46:25.495819+0000) 2024-04-25T10:46:37.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:37 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:37.291+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-25T10:46:14.896934+0000 front 2024-04-25T10:46:14.896986+0000 (oldest deadline 2024-04-25T10:46:37.196983+0000) 2024-04-25T10:46:38.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:38 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:38.204+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:38.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:38 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:38.204+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:38.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:38 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:38.204+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:38.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:38 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:38.204+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:38.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:38 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:38.204+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:38.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:38 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:38.204+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:03.095406+0000 front 2024-04-25T10:46:03.095252+0000 (oldest deadline 2024-04-25T10:46:29.495714+0000) 2024-04-25T10:46:38.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:38 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:38.263+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:38.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:38 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:38.263+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:38.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:38 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:38.263+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:38.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:38 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:38.263+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:38.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:38 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:38.263+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:38.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:38 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:38.263+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:38.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:38 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:38.245+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:38.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:38 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:38.245+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:38.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:38 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:38.245+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:38.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:38 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:38.245+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:38.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:38 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:38.245+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.595902+0000 front 2024-04-25T10:45:59.596166+0000 (oldest deadline 2024-04-25T10:46:25.495819+0000) 2024-04-25T10:46:38.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:38 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:38.245+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-25T10:46:14.896934+0000 front 2024-04-25T10:46:14.896986+0000 (oldest deadline 2024-04-25T10:46:37.196983+0000) 2024-04-25T10:46:39.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:39 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:39.170+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:39.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:39 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:39.170+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:39.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:39 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:39.170+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:39.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:39 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:39.170+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:39.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:39 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:39.170+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:39.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:39 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:39.170+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:03.095406+0000 front 2024-04-25T10:46:03.095252+0000 (oldest deadline 2024-04-25T10:46:29.495714+0000) 2024-04-25T10:46:39.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:39 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:39.170+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-25T10:46:14.796756+0000 front 2024-04-25T10:46:14.796877+0000 (oldest deadline 2024-04-25T10:46:38.296581+0000) 2024-04-25T10:46:39.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:39 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:39.200+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.687033+0000 front 2024-04-25T10:45:26.687086+0000 (oldest deadline 2024-04-25T10:45:52.587110+0000) 2024-04-25T10:46:39.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:39 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:39.200+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.988956+0000 front 2024-04-25T10:45:35.989102+0000 (oldest deadline 2024-04-25T10:45:59.389287+0000) 2024-04-25T10:46:39.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:39 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:39.200+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.991100+0000 front 2024-04-25T10:45:40.991007+0000 (oldest deadline 2024-04-25T10:46:06.891166+0000) 2024-04-25T10:46:39.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:39 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:39.200+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:51.592090+0000 front 2024-04-25T10:45:51.592076+0000 (oldest deadline 2024-04-25T10:46:13.894039+0000) 2024-04-25T10:46:39.610 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:39 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:39.200+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.595902+0000 front 2024-04-25T10:45:59.596166+0000 (oldest deadline 2024-04-25T10:46:25.495819+0000) 2024-04-25T10:46:39.611 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:39 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9[61881]: 2024-04-25T10:46:39.200+0000 7f84d1a93700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-25T10:46:14.896934+0000 front 2024-04-25T10:46:14.896986+0000 (oldest deadline 2024-04-25T10:46:37.196983+0000) 2024-04-25T10:46:39.790 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:39 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:39.290+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:39.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:39 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:39.290+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:39.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:39 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:39.290+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:39.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:39 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:39.290+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:39.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:39 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:39.290+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:39.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:39 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:39.290+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:40.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:40 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:40.179+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:40.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:40 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:40.179+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:40.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:40 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:40.179+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:40.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:40 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:40.180+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:40.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:40 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:40.180+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:40.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:40 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:40.180+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:03.095406+0000 front 2024-04-25T10:46:03.095252+0000 (oldest deadline 2024-04-25T10:46:29.495714+0000) 2024-04-25T10:46:40.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:40 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:40.180+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-25T10:46:14.796756+0000 front 2024-04-25T10:46:14.796877+0000 (oldest deadline 2024-04-25T10:46:38.296581+0000) 2024-04-25T10:46:40.666 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:40 smithi043 podman[99174]: 2024-04-25 10:46:40.360326783 +0000 UTC m=+5.575140628 container died 3391eb3e13c69d08c9e1348331cb9b8f7a57f7837f9f0bb1fdc3f0e5a64041c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T10:46:40.666 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:40 smithi043 podman[99174]: 2024-04-25 10:46:40.665994349 +0000 UTC m=+5.880808202 container cleanup 3391eb3e13c69d08c9e1348331cb9b8f7a57f7837f9f0bb1fdc3f0e5a64041c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T10:46:40.790 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:40 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:40.293+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:40.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:40 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:40.293+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:40.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:40 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:40.293+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:40.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:40 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:40.293+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:40.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:40 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:40.293+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:40.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:40 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:40.293+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:41.110 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:40 smithi043 bash[99174]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9 2024-04-25T10:46:41.110 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:40 smithi043 podman[99192]: 2024-04-25 10:46:40.833841737 +0000 UTC m=+0.464861237 container remove 3391eb3e13c69d08c9e1348331cb9b8f7a57f7837f9f0bb1fdc3f0e5a64041c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T10:46:41.504 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:41 smithi043 podman[99281]: 2024-04-25 10:46:41.234719418 +0000 UTC m=+0.108996935 container create b889eed8dc3f8804b8bdd87bd94749ff9d3eba03aef66b7abb517876b9be4ee6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T10:46:41.504 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:41 smithi043 podman[99281]: 2024-04-25 10:46:41.144840778 +0000 UTC m=+0.019118310 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:46:41.504 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:41 smithi043 podman[99281]: 2024-04-25 10:46:41.420275879 +0000 UTC m=+0.294553397 container init b889eed8dc3f8804b8bdd87bd94749ff9d3eba03aef66b7abb517876b9be4ee6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:46:41.504 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:41 smithi043 podman[99281]: 2024-04-25 10:46:41.423405121 +0000 UTC m=+0.297682640 container start b889eed8dc3f8804b8bdd87bd94749ff9d3eba03aef66b7abb517876b9be4ee6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T10:46:41.504 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:41 smithi043 podman[99281]: 2024-04-25 10:46:41.503501837 +0000 UTC m=+0.377779381 container attach b889eed8dc3f8804b8bdd87bd94749ff9d3eba03aef66b7abb517876b9be4ee6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T10:46:41.543 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:41 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:41.162+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:41.544 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:41 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:41.162+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:41.544 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:41 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:41.162+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:41.544 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:41 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:41.162+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:41.544 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:41 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:41.162+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:41.544 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:41 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:41.162+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:03.095406+0000 front 2024-04-25T10:46:03.095252+0000 (oldest deadline 2024-04-25T10:46:29.495714+0000) 2024-04-25T10:46:41.544 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:41 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:41.162+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-25T10:46:14.796756+0000 front 2024-04-25T10:46:14.796877+0000 (oldest deadline 2024-04-25T10:46:38.296581+0000) 2024-04-25T10:46:41.545 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:41 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:41.260+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:41.545 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:41 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:41.260+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:41.546 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:41 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:41.260+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:41.546 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:41 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:41.260+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:41.546 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:41 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:41.260+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:41.546 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:41 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:41.260+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:41.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:41 smithi043 podman[99281]: 2024-04-25 10:46:41.723849606 +0000 UTC m=+0.598127124 container died b889eed8dc3f8804b8bdd87bd94749ff9d3eba03aef66b7abb517876b9be4ee6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T10:46:42.244 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T10:46:42.261 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:42 smithi043 podman[99306]: 2024-04-25 10:46:42.22359054 +0000 UTC m=+0.490271114 container remove b889eed8dc3f8804b8bdd87bd94749ff9d3eba03aef66b7abb517876b9be4ee6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T10:46:42.261 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:42 smithi043 systemd[1]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.9.service: Deactivated successfully. 2024-04-25T10:46:42.261 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:42 smithi043 systemd[1]: Stopped Ceph osd.9 for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:46:42.261 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 25 10:46:42 smithi043 systemd[1]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.9.service: Consumed 12.189s CPU time. 2024-04-25T10:46:42.302 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:46:42.302 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T10:46:42.302 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T10:46:42.302 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.10 2024-04-25T10:46:42.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:42 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:42.201+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:42.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:42 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:42.201+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:42.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:42 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:42.201+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:42.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:42 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:42.202+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:42.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:42 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:42.202+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:42.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:42 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:42.202+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:03.095406+0000 front 2024-04-25T10:46:03.095252+0000 (oldest deadline 2024-04-25T10:46:29.495714+0000) 2024-04-25T10:46:42.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:42 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:42.202+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-25T10:46:14.796756+0000 front 2024-04-25T10:46:14.796877+0000 (oldest deadline 2024-04-25T10:46:38.296581+0000) 2024-04-25T10:46:42.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:42 smithi077 systemd[1]: Stopping Ceph osd.10 for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:46:42.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:42 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:42.286+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:42.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:42 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:42.286+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:42.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:42 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:42.286+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:42.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:42 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:42.286+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:42.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:42 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:42.286+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:42.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:42 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:42.286+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:43.040 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:42 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:42.598+0000 7f60051a7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:46:43.040 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:42 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:42.598+0000 7f60051a7700 -1 osd.10 97 *** Got signal Terminated *** 2024-04-25T10:46:43.040 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:42 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:42.598+0000 7f60051a7700 -1 osd.10 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T10:46:43.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:43 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:43.197+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:43.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:43 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:43.197+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:43.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:43 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:43.197+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:43.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:43 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:43.197+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:43.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:43 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:43.197+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:43.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:43 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:43.197+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:03.095406+0000 front 2024-04-25T10:46:03.095252+0000 (oldest deadline 2024-04-25T10:46:29.495714+0000) 2024-04-25T10:46:43.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:43 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:43.197+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-25T10:46:14.796756+0000 front 2024-04-25T10:46:14.796877+0000 (oldest deadline 2024-04-25T10:46:38.296581+0000) 2024-04-25T10:46:43.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:43 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:43.259+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:43.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:43 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:43.259+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:43.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:43 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:43.259+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:43.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:43 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:43.259+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:43.542 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:43 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:43.259+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:43.542 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:43 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:43.259+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:44.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:44 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:44.223+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:44.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:44 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:44.223+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:44.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:44 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:44.223+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:44.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:44 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:44.223+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:44.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:44 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:44.223+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:44.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:44 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:44.223+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:03.095406+0000 front 2024-04-25T10:46:03.095252+0000 (oldest deadline 2024-04-25T10:46:29.495714+0000) 2024-04-25T10:46:44.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:44 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:44.223+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-25T10:46:14.796756+0000 front 2024-04-25T10:46:14.796877+0000 (oldest deadline 2024-04-25T10:46:38.296581+0000) 2024-04-25T10:46:44.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:44 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:44.266+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:44.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:44 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:44.266+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:44.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:44 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:44.266+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:44.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:44 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:44.266+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:44.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:44 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:44.266+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:44.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:44 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:44.266+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:45.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:45 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:45.224+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:45.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:45 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:45.224+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:45.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:45 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:45.224+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:45.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:45 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:45.224+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:45.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:45 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:45.224+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:45.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:45 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:45.224+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:03.095406+0000 front 2024-04-25T10:46:03.095252+0000 (oldest deadline 2024-04-25T10:46:29.495714+0000) 2024-04-25T10:46:45.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:45 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:45.224+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-25T10:46:14.796756+0000 front 2024-04-25T10:46:14.796877+0000 (oldest deadline 2024-04-25T10:46:38.296581+0000) 2024-04-25T10:46:45.790 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:45 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:45.309+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:45.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:45 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:45.309+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:45.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:45 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:45.309+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:45.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:45 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:45.309+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:45.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:45 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:45.309+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:45.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:45 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:45.309+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:46.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:46 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:46.177+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:46.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:46 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:46.177+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:46.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:46 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:46.177+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:46.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:46 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:46.177+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:46.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:46 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:46.177+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:46.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:46 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:46.177+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:03.095406+0000 front 2024-04-25T10:46:03.095252+0000 (oldest deadline 2024-04-25T10:46:29.495714+0000) 2024-04-25T10:46:46.541 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:46 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:46.177+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-25T10:46:14.796756+0000 front 2024-04-25T10:46:14.796877+0000 (oldest deadline 2024-04-25T10:46:38.296581+0000) 2024-04-25T10:46:46.693 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:46 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:46.342+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:46.693 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:46 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:46.342+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:46.693 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:46 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:46.342+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:46.693 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:46 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:46.342+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:46.693 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:46 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:46.342+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:46.693 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:46 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:46.342+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:47.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:47 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:47.222+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:26.289589+0000 front 2024-04-25T10:45:26.289504+0000 (oldest deadline 2024-04-25T10:45:50.389667+0000) 2024-04-25T10:46:47.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:47 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:47.222+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:35.090781+0000 front 2024-04-25T10:45:35.090886+0000 (oldest deadline 2024-04-25T10:45:58.590972+0000) 2024-04-25T10:46:47.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:47 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:47.222+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:40.892073+0000 front 2024-04-25T10:45:40.891892+0000 (oldest deadline 2024-04-25T10:46:05.591820+0000) 2024-04-25T10:46:47.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:47 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:47.222+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:50.193579+0000 front 2024-04-25T10:45:50.193441+0000 (oldest deadline 2024-04-25T10:46:16.093463+0000) 2024-04-25T10:46:47.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:47 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:47.222+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:46:00.194538+0000 front 2024-04-25T10:46:00.194587+0000 (oldest deadline 2024-04-25T10:46:23.094613+0000) 2024-04-25T10:46:47.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:47 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:47.222+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:03.095406+0000 front 2024-04-25T10:46:03.095252+0000 (oldest deadline 2024-04-25T10:46:29.495714+0000) 2024-04-25T10:46:47.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:47 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10[58136]: 2024-04-25T10:46:47.222+0000 7f5ffd755700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-25T10:46:14.796756+0000 front 2024-04-25T10:46:14.796877+0000 (oldest deadline 2024-04-25T10:46:38.296581+0000) 2024-04-25T10:46:47.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:47 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:47.309+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:47.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:47 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:47.309+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:47.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:47 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:47.309+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:47.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:47 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:47.309+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:47.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:47 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:47.309+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:47.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:47 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:47.309+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:48.325 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:48 smithi077 podman[71142]: 2024-04-25 10:46:48.050610095 +0000 UTC m=+5.559886759 container died 14bf19662a4efa5d70cb04430952a9a77152815a5411d45538e1f3b779d9723f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T10:46:48.675 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:48 smithi077 podman[71142]: 2024-04-25 10:46:48.42363775 +0000 UTC m=+5.932914484 container cleanup 14bf19662a4efa5d70cb04430952a9a77152815a5411d45538e1f3b779d9723f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T10:46:48.675 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:48 smithi077 bash[71142]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10 2024-04-25T10:46:48.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:48 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:48.323+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:48.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:48 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:48.323+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:48.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:48 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:48.323+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:48.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:48 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:48.323+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:48.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:48 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:48.323+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:48.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:48 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:48.323+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:48.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:48 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:48.323+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-04-25T10:46:23.533576+0000 front 2024-04-25T10:46:23.533596+0000 (oldest deadline 2024-04-25T10:46:48.233304+0000) 2024-04-25T10:46:48.976 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:48 smithi077 podman[71159]: 2024-04-25 10:46:48.674420519 +0000 UTC m=+0.621757481 container remove 14bf19662a4efa5d70cb04430952a9a77152815a5411d45538e1f3b779d9723f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T10:46:49.286 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:48 smithi077 podman[71250]: 2024-04-25 10:46:48.975302738 +0000 UTC m=+0.097252392 container create b20ba498e530ea9a359593dc4be61a329d8936bded991565c69c029f53911a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T10:46:49.286 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:48 smithi077 podman[71250]: 2024-04-25 10:46:48.896253945 +0000 UTC m=+0.018203624 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:46:49.286 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:49 smithi077 podman[71250]: 2024-04-25 10:46:49.202540507 +0000 UTC m=+0.324490159 container init b20ba498e530ea9a359593dc4be61a329d8936bded991565c69c029f53911a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T10:46:49.287 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:49 smithi077 podman[71250]: 2024-04-25 10:46:49.205443165 +0000 UTC m=+0.327392825 container start b20ba498e530ea9a359593dc4be61a329d8936bded991565c69c029f53911a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:46:49.287 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:49 smithi077 podman[71250]: 2024-04-25 10:46:49.285714474 +0000 UTC m=+0.407664128 container attach b20ba498e530ea9a359593dc4be61a329d8936bded991565c69c029f53911a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T10:46:49.540 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 25 10:46:49 smithi077 podman[71250]: 2024-04-25 10:46:49.502507033 +0000 UTC m=+0.624456704 container died b20ba498e530ea9a359593dc4be61a329d8936bded991565c69c029f53911a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T10:46:49.790 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:49 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:49.319+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:49.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:49 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:49.319+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:49.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:49 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:49.319+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:49.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:49 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:49.319+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:49.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:49 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:49.319+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:49.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:49 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:49.319+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:49.791 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:49 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:49.319+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-04-25T10:46:23.533576+0000 front 2024-04-25T10:46:23.533596+0000 (oldest deadline 2024-04-25T10:46:48.233304+0000) 2024-04-25T10:46:50.076 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T10:46:50.117 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:46:50.117 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T10:46:50.118 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T10:46:50.118 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.11 2024-04-25T10:46:50.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:50 smithi164 systemd[1]: Stopping Ceph osd.11 for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:46:50.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:50 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:50.283+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:50.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:50 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:50.283+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:50.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:50 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:50.283+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:50.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:50 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:50.283+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:50.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:50 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:50.283+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:50.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:50 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:50.283+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:50.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:50 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:50.283+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-04-25T10:46:23.533576+0000 front 2024-04-25T10:46:23.533596+0000 (oldest deadline 2024-04-25T10:46:48.233304+0000) 2024-04-25T10:46:50.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:50 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:50.390+0000 7fe7530a6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:46:50.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:50 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:50.390+0000 7fe7530a6700 -1 osd.11 97 *** Got signal Terminated *** 2024-04-25T10:46:50.542 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:50 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:50.390+0000 7fe7530a6700 -1 osd.11 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T10:46:51.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:51 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:51.238+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:51.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:51 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:51.238+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:51.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:51 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:51.238+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:51.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:51 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:51.238+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:51.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:51 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:51.238+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:51.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:51 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:51.238+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:51.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:51 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:51.238+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-04-25T10:46:23.533576+0000 front 2024-04-25T10:46:23.533596+0000 (oldest deadline 2024-04-25T10:46:48.233304+0000) 2024-04-25T10:46:52.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:52 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:52.202+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:52.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:52 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:52.202+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:52.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:52 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:52.202+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:52.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:52 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:52.202+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:52.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:52 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:52.202+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:52.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:52 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:52.202+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:52.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:52 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:52.202+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-04-25T10:46:23.533576+0000 front 2024-04-25T10:46:23.533596+0000 (oldest deadline 2024-04-25T10:46:48.233304+0000) 2024-04-25T10:46:53.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:53 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:53.186+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:53.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:53 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:53.186+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:53.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:53 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:53.186+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:53.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:53 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:53.186+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:53.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:53 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:53.186+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:53.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:53 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:53.186+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:53.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:53 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:53.186+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-04-25T10:46:23.533576+0000 front 2024-04-25T10:46:23.533596+0000 (oldest deadline 2024-04-25T10:46:48.233304+0000) 2024-04-25T10:46:53.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:53 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:53.186+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6820 osd.8 since back 2024-04-25T10:46:28.233682+0000 front 2024-04-25T10:46:28.233667+0000 (oldest deadline 2024-04-25T10:46:52.933764+0000) 2024-04-25T10:46:54.540 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:54 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:54.140+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:54.540 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:54 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:54.140+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:54.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:54 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:54.140+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:54.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:54 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:54.140+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:54.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:54 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:54.140+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:54.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:54 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:54.140+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:54.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:54 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:54.140+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-04-25T10:46:23.533576+0000 front 2024-04-25T10:46:23.533596+0000 (oldest deadline 2024-04-25T10:46:48.233304+0000) 2024-04-25T10:46:54.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:54 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:54.140+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6820 osd.8 since back 2024-04-25T10:46:28.233682+0000 front 2024-04-25T10:46:28.233667+0000 (oldest deadline 2024-04-25T10:46:52.933764+0000) 2024-04-25T10:46:55.540 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:55 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:55.143+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-25T10:45:28.726001+0000 front 2024-04-25T10:45:28.725958+0000 (oldest deadline 2024-04-25T10:45:54.025997+0000) 2024-04-25T10:46:55.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:55 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:55.143+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-25T10:45:34.026746+0000 front 2024-04-25T10:45:34.026712+0000 (oldest deadline 2024-04-25T10:46:02.227008+0000) 2024-04-25T10:46:55.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:55 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:55.143+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-25T10:45:43.927917+0000 front 2024-04-25T10:45:43.928139+0000 (oldest deadline 2024-04-25T10:46:08.628014+0000) 2024-04-25T10:46:55.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:55 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:55.143+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-25T10:45:49.129073+0000 front 2024-04-25T10:45:49.129157+0000 (oldest deadline 2024-04-25T10:46:13.228990+0000) 2024-04-25T10:46:55.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:55 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:55.143+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-25T10:45:59.130276+0000 front 2024-04-25T10:45:59.130141+0000 (oldest deadline 2024-04-25T10:46:22.029984+0000) 2024-04-25T10:46:55.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:55 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:55.143+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-25T10:46:05.331726+0000 front 2024-04-25T10:46:05.331699+0000 (oldest deadline 2024-04-25T10:46:29.431713+0000) 2024-04-25T10:46:55.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:55 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:55.143+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-04-25T10:46:23.533576+0000 front 2024-04-25T10:46:23.533596+0000 (oldest deadline 2024-04-25T10:46:48.233304+0000) 2024-04-25T10:46:55.541 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:55 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11[58789]: 2024-04-25T10:46:55.143+0000 7fe74b654700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.164:6820 osd.8 since back 2024-04-25T10:46:28.233682+0000 front 2024-04-25T10:46:28.233667+0000 (oldest deadline 2024-04-25T10:46:52.933764+0000) 2024-04-25T10:46:56.199 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:55 smithi164 podman[71082]: 2024-04-25 10:46:55.882613835 +0000 UTC m=+5.574894233 container died b15f56f11b11b47b6a91b8c2feda29e5fdc0e1b4c56e9b9268cea07dfe6d6a08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T10:46:56.199 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:56 smithi164 podman[71082]: 2024-04-25 10:46:56.198862597 +0000 UTC m=+5.891142995 container cleanup b15f56f11b11b47b6a91b8c2feda29e5fdc0e1b4c56e9b9268cea07dfe6d6a08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T10:46:56.540 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:56 smithi164 bash[71082]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11 2024-04-25T10:46:56.540 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:56 smithi164 podman[71099]: 2024-04-25 10:46:56.367244671 +0000 UTC m=+0.477916290 container remove b15f56f11b11b47b6a91b8c2feda29e5fdc0e1b4c56e9b9268cea07dfe6d6a08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T10:46:57.030 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:56 smithi164 podman[71191]: 2024-04-25 10:46:56.667080545 +0000 UTC m=+0.018142511 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T10:46:57.030 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:56 smithi164 podman[71191]: 2024-04-25 10:46:56.809852259 +0000 UTC m=+0.160914192 container create ee140e22a3dec767f0a2caa2f3652720630bea6a1181c22bcaf4168fa05c6278 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T10:46:57.030 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:57 smithi164 podman[71191]: 2024-04-25 10:46:57.029400063 +0000 UTC m=+0.380462001 container init ee140e22a3dec767f0a2caa2f3652720630bea6a1181c22bcaf4168fa05c6278 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:46:57.290 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:57 smithi164 podman[71191]: 2024-04-25 10:46:57.032754349 +0000 UTC m=+0.383816293 container start ee140e22a3dec767f0a2caa2f3652720630bea6a1181c22bcaf4168fa05c6278 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0) 2024-04-25T10:46:57.291 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:57 smithi164 podman[71191]: 2024-04-25 10:46:57.10461278 +0000 UTC m=+0.455674722 container attach ee140e22a3dec767f0a2caa2f3652720630bea6a1181c22bcaf4168fa05c6278 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T10:46:57.608 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 25 10:46:57 smithi164 podman[71191]: 2024-04-25 10:46:57.329245285 +0000 UTC m=+0.680307231 container died ee140e22a3dec767f0a2caa2f3652720630bea6a1181c22bcaf4168fa05c6278 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T10:46:57.821 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T10:46:57.859 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:46:57.859 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T10:46:57.859 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T10:46:57.860 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.a 2024-04-25T10:46:58.168 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 25 10:46:57 smithi043 systemd[1]: Stopping Ceph mds.a for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:46:58.610 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 25 10:46:58 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-a[68878]: 2024-04-25T10:46:58.166+0000 7f9f65b37700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:46:58.610 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 25 10:46:58 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-a[68878]: 2024-04-25T10:46:58.166+0000 7f9f65b37700 -1 mds.a *** got signal Terminated *** 2024-04-25T10:47:02.001 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 25 10:47:01 smithi043 podman[99386]: 2024-04-25 10:47:01.730053839 +0000 UTC m=+3.674248187 container died bf76e4121c2819b808b5d241d42971a5ac60e1831b915fd22ecf8d84fb61899f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T10:47:02.266 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 25 10:47:02 smithi043 podman[99386]: 2024-04-25 10:47:02.240885479 +0000 UTC m=+4.185079827 container cleanup bf76e4121c2819b808b5d241d42971a5ac60e1831b915fd22ecf8d84fb61899f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0) 2024-04-25T10:47:02.266 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 25 10:47:02 smithi043 bash[99386]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-a 2024-04-25T10:47:02.685 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 25 10:47:02 smithi043 podman[99400]: 2024-04-25 10:47:02.433906783 +0000 UTC m=+0.701876042 container remove bf76e4121c2819b808b5d241d42971a5ac60e1831b915fd22ecf8d84fb61899f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-a, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux ) 2024-04-25T10:47:02.688 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T10:47:02.724 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:47:02.724 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T10:47:02.724 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T10:47:02.724 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.d 2024-04-25T10:47:03.054 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 25 10:47:02 smithi043 systemd[1]: Stopping Ceph mds.d for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:47:03.360 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 25 10:47:03 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-d[67913]: 2024-04-25T10:47:03.052+0000 7f199db3e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:47:03.360 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 25 10:47:03 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-d[67913]: 2024-04-25T10:47:03.052+0000 7f199db3e700 -1 mds.d *** got signal Terminated *** 2024-04-25T10:47:06.538 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 25 10:47:06 smithi043 podman[99507]: 2024-04-25 10:47:06.257837593 +0000 UTC m=+3.313398009 container died 37ca5ef7abd930639eb4328429753dfd9a223a4b703e213a77595c28b189af96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-25T10:47:06.859 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 25 10:47:06 smithi043 podman[99507]: 2024-04-25 10:47:06.745009542 +0000 UTC m=+3.800569947 container cleanup 37ca5ef7abd930639eb4328429753dfd9a223a4b703e213a77595c28b189af96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T10:47:06.860 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 25 10:47:06 smithi043 bash[99507]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-d 2024-04-25T10:47:07.180 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T10:47:07.212 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:47:07.212 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T10:47:07.212 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T10:47:07.212 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.g 2024-04-25T10:47:07.558 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 25 10:47:07 smithi043 systemd[1]: Stopping Ceph mds.g for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:47:07.860 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 25 10:47:07 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-g[68488]: 2024-04-25T10:47:07.556+0000 7f43595a3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:47:07.860 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 25 10:47:07 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-g[68488]: 2024-04-25T10:47:07.556+0000 7f43595a3700 -1 mds.g *** got signal Terminated *** 2024-04-25T10:47:11.258 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 25 10:47:10 smithi043 podman[99627]: 2024-04-25 10:47:10.86482955 +0000 UTC m=+3.434004887 container died e7c559510af1f8072990ffbc01cfc683465b42e6ce57170a45d2771091344265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T10:47:11.558 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 25 10:47:11 smithi043 podman[99627]: 2024-04-25 10:47:11.35646503 +0000 UTC m=+3.925640357 container cleanup e7c559510af1f8072990ffbc01cfc683465b42e6ce57170a45d2771091344265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T10:47:11.558 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 25 10:47:11 smithi043 bash[99627]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-g 2024-04-25T10:47:11.684 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T10:47:11.717 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:47:11.717 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T10:47:11.717 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T10:47:11.717 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.j 2024-04-25T10:47:11.860 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 25 10:47:11 smithi043 systemd[1]: Stopping Ceph mds.j for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:47:12.609 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 25 10:47:12 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-j[67550]: 2024-04-25T10:47:12.134+0000 7fdf9c986700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:47:12.610 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 25 10:47:12 smithi043 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-j[67550]: 2024-04-25T10:47:12.134+0000 7fdf9c986700 -1 mds.j *** got signal Terminated *** 2024-04-25T10:47:15.728 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 25 10:47:15 smithi043 podman[99748]: 2024-04-25 10:47:15.473669706 +0000 UTC m=+3.537646388 container died 83b2057bdc494b83729a2d98aac706a32c369db1b8a0c9277db72283bc4f1982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-25T10:47:16.003 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 25 10:47:15 smithi043 podman[99748]: 2024-04-25 10:47:15.810298228 +0000 UTC m=+3.874274926 container cleanup 83b2057bdc494b83729a2d98aac706a32c369db1b8a0c9277db72283bc4f1982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-25T10:47:16.003 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 25 10:47:15 smithi043 bash[99748]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-j 2024-04-25T10:47:16.246 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T10:47:16.267 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 25 10:47:16 smithi043 podman[99762]: 2024-04-25 10:47:16.00277027 +0000 UTC m=+0.531986193 container remove 83b2057bdc494b83729a2d98aac706a32c369db1b8a0c9277db72283bc4f1982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T10:47:16.267 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 25 10:47:16 smithi043 systemd[1]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.j.service: Deactivated successfully. 2024-04-25T10:47:16.267 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 25 10:47:16 smithi043 systemd[1]: Stopped Ceph mds.j for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:47:16.267 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 25 10:47:16 smithi043 systemd[1]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.j.service: Consumed 1.156s CPU time. 2024-04-25T10:47:16.277 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:47:16.278 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T10:47:16.278 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T10:47:16.278 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.b 2024-04-25T10:47:16.574 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:47:16 smithi077 systemd[1]: Stopping Ceph mds.b for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:47:17.040 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:47:16 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-b[63685]: 2024-04-25T10:47:16.572+0000 7f1a02655700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:47:17.040 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:47:16 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-b[63685]: 2024-04-25T10:47:16.572+0000 7f1a02655700 -1 mds.b *** got signal Terminated *** 2024-04-25T10:47:20.848 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:47:20 smithi077 podman[71366]: 2024-04-25 10:47:20.588788613 +0000 UTC m=+4.124096932 container died 9922b8342e702b37525e72be1651333e8fb2c700cf26b8962306bba9bc2971e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T10:47:21.199 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:47:20 smithi077 podman[71366]: 2024-04-25 10:47:20.955990134 +0000 UTC m=+4.491298461 container cleanup 9922b8342e702b37525e72be1651333e8fb2c700cf26b8962306bba9bc2971e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T10:47:21.199 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 25 10:47:20 smithi077 bash[71366]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-b 2024-04-25T10:47:21.351 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T10:47:21.384 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:47:21.384 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T10:47:21.385 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T10:47:21.385 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.e 2024-04-25T10:47:21.727 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 25 10:47:21 smithi077 systemd[1]: Stopping Ceph mds.e for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:47:22.039 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 25 10:47:21 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-e[63055]: 2024-04-25T10:47:21.726+0000 7fa6aa755700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:47:22.040 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 25 10:47:21 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-e[63055]: 2024-04-25T10:47:21.726+0000 7fa6aa755700 -1 mds.e *** got signal Terminated *** 2024-04-25T10:47:25.290 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 25 10:47:24 smithi077 podman[71495]: 2024-04-25 10:47:24.971805107 +0000 UTC m=+3.362917947 container died dd1c5cb7d7d036d03c0b4ec23512a4d823a22295c93f6d7dee935a94b6dd1b8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T10:47:25.790 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 25 10:47:25 smithi077 podman[71495]: 2024-04-25 10:47:25.526609782 +0000 UTC m=+3.917722622 container cleanup dd1c5cb7d7d036d03c0b4ec23512a4d823a22295c93f6d7dee935a94b6dd1b8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T10:47:25.790 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 25 10:47:25 smithi077 bash[71495]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-e 2024-04-25T10:47:25.950 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T10:47:25.983 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:47:25.983 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T10:47:25.983 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T10:47:25.983 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.h 2024-04-25T10:47:26.540 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:47:26 smithi077 systemd[1]: Stopping Ceph mds.h for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:47:26.540 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:47:26 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[66890]: 2024-04-25T10:47:26.305+0000 7f52169ed700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:47:26.540 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:47:26 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h[66890]: 2024-04-25T10:47:26.305+0000 7f52169ed700 -1 mds.h *** got signal Terminated *** 2024-04-25T10:47:29.289 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:47:28 smithi077 podman[71617]: 2024-04-25 10:47:28.969210393 +0000 UTC m=+2.768094344 container died d3ab0de6929c97bc8752ff1b8e17cedaa1160fc2bc4263eb1f68f0e0cb7ab47a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T10:47:29.539 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:47:29 smithi077 podman[71617]: 2024-04-25 10:47:29.371596618 +0000 UTC m=+3.170480566 container cleanup d3ab0de6929c97bc8752ff1b8e17cedaa1160fc2bc4263eb1f68f0e0cb7ab47a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424) 2024-04-25T10:47:29.540 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 25 10:47:29 smithi077 bash[71617]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-h 2024-04-25T10:47:29.747 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T10:47:29.779 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:47:29.779 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T10:47:29.780 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T10:47:29.780 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.k 2024-04-25T10:47:30.039 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 25 10:47:29 smithi077 systemd[1]: Stopping Ceph mds.k for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:47:30.540 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 25 10:47:30 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-k[62735]: 2024-04-25T10:47:30.150+0000 7f0a576bd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:47:30.540 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 25 10:47:30 smithi077 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-k[62735]: 2024-04-25T10:47:30.150+0000 7f0a576bd700 -1 mds.k *** got signal Terminated *** 2024-04-25T10:47:33.001 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 25 10:47:32 smithi077 podman[71737]: 2024-04-25 10:47:32.72152491 +0000 UTC m=+2.720952499 container died cdd9460375e8d24c455194ff6ea69e4ba2b0f1bdf65df6cc4ed7a65ecda4e4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T10:47:33.290 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 25 10:47:33 smithi077 podman[71737]: 2024-04-25 10:47:33.108793161 +0000 UTC m=+3.108220753 container cleanup cdd9460375e8d24c455194ff6ea69e4ba2b0f1bdf65df6cc4ed7a65ecda4e4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-25T10:47:33.290 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 25 10:47:33 smithi077 bash[71737]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-k 2024-04-25T10:47:33.543 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T10:47:33.574 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:47:33.574 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T10:47:33.574 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T10:47:33.574 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.c 2024-04-25T10:47:34.033 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 25 10:47:33 smithi164 systemd[1]: Stopping Ceph mds.c for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:47:34.290 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 25 10:47:34 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-c[63980]: 2024-04-25T10:47:34.030+0000 7f67b84bb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:47:34.290 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 25 10:47:34 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-c[63980]: 2024-04-25T10:47:34.030+0000 7f67b84bb700 -1 mds.c *** got signal Terminated *** 2024-04-25T10:47:37.240 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 25 10:47:36 smithi164 podman[71298]: 2024-04-25 10:47:36.963264422 +0000 UTC m=+3.195433794 container died d7162b297d7c5f0a4530a1f2cc405d376c837f0123589e70fb1a1930280158f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T10:47:37.240 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 25 10:47:37 smithi164 podman[71298]: 2024-04-25 10:47:37.239803779 +0000 UTC m=+3.471973151 container cleanup d7162b297d7c5f0a4530a1f2cc405d376c837f0123589e70fb1a1930280158f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD) 2024-04-25T10:47:37.540 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 25 10:47:37 smithi164 bash[71298]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-c 2024-04-25T10:47:37.540 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 25 10:47:37 smithi164 podman[71324]: 2024-04-25 10:47:37.466075415 +0000 UTC m=+0.499199984 container remove d7162b297d7c5f0a4530a1f2cc405d376c837f0123589e70fb1a1930280158f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T10:47:37.606 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T10:47:37.641 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:47:37.642 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T10:47:37.642 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T10:47:37.642 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.f 2024-04-25T10:47:37.937 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 25 10:47:37 smithi164 systemd[1]: Stopping Ceph mds.f for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:47:38.290 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 25 10:47:37 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-f[63673]: 2024-04-25T10:47:37.935+0000 7f87730d8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:47:38.291 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 25 10:47:37 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-f[63673]: 2024-04-25T10:47:37.935+0000 7f87730d8700 -1 mds.f *** got signal Terminated *** 2024-04-25T10:47:41.291 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 25 10:47:40 smithi164 podman[71431]: 2024-04-25 10:47:40.961626865 +0000 UTC m=+3.106991393 container died b5692deb846c3776d4aea42f1077fe2332e17e66ac837e6f7222cf244cd662f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T10:47:41.701 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 25 10:47:41 smithi164 podman[71431]: 2024-04-25 10:47:41.2933054 +0000 UTC m=+3.438669913 container cleanup b5692deb846c3776d4aea42f1077fe2332e17e66ac837e6f7222cf244cd662f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-f, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T10:47:41.701 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 25 10:47:41 smithi164 bash[71431]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-f 2024-04-25T10:47:41.701 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 25 10:47:41 smithi164 podman[71445]: 2024-04-25 10:47:41.494484211 +0000 UTC m=+0.527730179 container remove b5692deb846c3776d4aea42f1077fe2332e17e66ac837e6f7222cf244cd662f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T10:47:41.715 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T10:47:41.747 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:47:41.747 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T10:47:41.747 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T10:47:41.748 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.i 2024-04-25T10:47:42.040 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 25 10:47:41 smithi164 systemd[1]: Stopping Ceph mds.i for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:47:42.540 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 25 10:47:42 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-i[64300]: 2024-04-25T10:47:42.072+0000 7f0e6bdab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:47:42.541 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 25 10:47:42 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-i[64300]: 2024-04-25T10:47:42.072+0000 7f0e6bdab700 -1 mds.i *** got signal Terminated *** 2024-04-25T10:47:45.040 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 25 10:47:44 smithi164 podman[71553]: 2024-04-25 10:47:44.96218493 +0000 UTC m=+3.002879725 container died 23da590987536d447664426bd66b4766f49445ebf423d302792660e81d868476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T10:47:45.540 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 25 10:47:45 smithi164 podman[71553]: 2024-04-25 10:47:45.314173896 +0000 UTC m=+3.354868699 container cleanup 23da590987536d447664426bd66b4766f49445ebf423d302792660e81d868476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T10:47:45.541 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 25 10:47:45 smithi164 bash[71553]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-i 2024-04-25T10:47:45.662 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T10:47:45.693 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:47:45.693 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T10:47:45.693 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T10:47:45.693 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.l 2024-04-25T10:47:46.018 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 25 10:47:45 smithi164 systemd[1]: Stopping Ceph mds.l for 94d0035c-02ed-11ef-bc93-c7b262605968... 2024-04-25T10:47:46.290 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 25 10:47:46 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-l[63360]: 2024-04-25T10:47:46.017+0000 7f06d7ba1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T10:47:46.290 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 25 10:47:46 smithi164 ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-l[63360]: 2024-04-25T10:47:46.017+0000 7f06d7ba1700 -1 mds.l *** got signal Terminated *** 2024-04-25T10:47:50.152 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 25 10:47:49 smithi164 podman[71673]: 2024-04-25 10:47:49.881193621 +0000 UTC m=+3.978382542 container died a0a5434eda6a89ac170a570cf38cf651e54099aefa204f716941475d537c1828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-25T10:47:50.419 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 25 10:47:50 smithi164 podman[71673]: 2024-04-25 10:47:50.225877902 +0000 UTC m=+4.323066822 container cleanup a0a5434eda6a89ac170a570cf38cf651e54099aefa204f716941475d537c1828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-l, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T10:47:50.419 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 25 10:47:50 smithi164 bash[71673]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-l 2024-04-25T10:47:50.649 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T10:47:50.669 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 25 10:47:50 smithi164 podman[71687]: 2024-04-25 10:47:50.418540155 +0000 UTC m=+0.535430721 container remove a0a5434eda6a89ac170a570cf38cf651e54099aefa204f716941475d537c1828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-94d0035c-02ed-11ef-bc93-c7b262605968-mds-l, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T10:47:50.669 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 25 10:47:50 smithi164 systemd[1]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.l.service: Deactivated successfully. 2024-04-25T10:47:50.669 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 25 10:47:50 smithi164 systemd[1]: Stopped Ceph mds.l for 94d0035c-02ed-11ef-bc93-c7b262605968. 2024-04-25T10:47:50.669 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 25 10:47:50 smithi164 systemd[1]: ceph-94d0035c-02ed-11ef-bc93-c7b262605968@mds.l.service: Consumed 1.155s CPU time. 2024-04-25T10:47:50.681 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T10:47:50.681 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T10:47:50.682 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T10:47:50.883 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:47:53.496 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T10:47:53.688 INFO:teuthology.orchestra.run.smithi077.stdout:Deleting cluster with fsid: 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:47:56.085 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T10:47:56.276 INFO:teuthology.orchestra.run.smithi164.stdout:Deleting cluster with fsid: 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:47:58.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T10:47:58.612 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T10:47:58.644 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T10:47:58.670 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T10:47:58.672 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673066/remote/smithi043/crash 2024-04-25T10:47:58.673 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/crash -- . 2024-04-25T10:47:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:tar: /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T10:47:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:tar: Error is not recoverable: exiting now 2024-04-25T10:47:58.733 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673066/remote/smithi077/crash 2024-04-25T10:47:58.734 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/crash -- . 2024-04-25T10:47:58.798 INFO:teuthology.orchestra.run.smithi077.stderr:tar: /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T10:47:58.799 INFO:teuthology.orchestra.run.smithi077.stderr:tar: Error is not recoverable: exiting now 2024-04-25T10:47:58.800 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673066/remote/smithi164/crash 2024-04-25T10:47:58.802 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/crash -- . 2024-04-25T10:47:58.837 INFO:teuthology.orchestra.run.smithi164.stderr:tar: /var/lib/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T10:47:58.837 INFO:teuthology.orchestra.run.smithi164.stderr:tar: Error is not recoverable: exiting now 2024-04-25T10:47:58.838 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T10:47:58.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-25T10:47:58.878 INFO:tasks.cephadm:Compressing logs... 2024-04-25T10:47:58.878 DEBUG:teuthology.orchestra.run.smithi043:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:47:58.920 DEBUG:teuthology.orchestra.run.smithi077:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:47:58.923 DEBUG:teuthology.orchestra.run.smithi164:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:47:58.945 INFO:teuthology.orchestra.run.smithi043.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.log 2024-04-25T10:47:58.945 INFO:teuthology.orchestra.run.smithi043.stderr:: No such file or directory 2024-04-25T10:47:58.948 INFO:teuthology.orchestra.run.smithi164.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.log 2024-04-25T10:47:58.948 INFO:teuthology.orchestra.run.smithi164.stderr:: No such file or directory 2024-04-25T10:47:58.948 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T10:47:58.948 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T10:47:58.948 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T10:47:58.950 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T10:47:58.950 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.log: -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T10:47:58.950 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T10:47:58.950 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T10:47:58.950 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T10:47:58.950 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.j.log: /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T10:47:58.950 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T10:47:58.950 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mgr.x.log: -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T10:47:58.950 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T10:47:58.950 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mon.a.log: 90.3% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T10:47:58.961 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T10:47:58.961 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T10:47:58.961 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.cephadm.log: 94.6% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T10:47:58.961 INFO:teuthology.orchestra.run.smithi043.stderr: 84.1% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T10:47:58.962 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T10:47:58.962 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.a.log: 94.6% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T10:47:58.963 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T10:47:58.963 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T10:47:58.963 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T10:47:58.963 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T10:47:58.963 INFO:teuthology.orchestra.run.smithi043.stderr: 94.6% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T10:47:58.964 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T10:47:58.964 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T10:47:58.964 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T10:47:58.964 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T10:47:58.964 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T10:47:58.964 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.3% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T10:47:58.965 INFO:teuthology.orchestra.run.smithi164.stderr: 90.5% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T10:47:58.965 INFO:teuthology.orchestra.run.smithi164.stderr: 92.1% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T10:47:58.965 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T10:47:58.965 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T10:47:58.965 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T10:47:58.965 INFO:teuthology.orchestra.run.smithi164.stderr: 83.7% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T10:47:58.966 INFO:teuthology.orchestra.run.smithi077.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.log 2024-04-25T10:47:58.966 INFO:teuthology.orchestra.run.smithi077.stderr:: No such file or directory 2024-04-25T10:47:58.966 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T10:47:58.966 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T10:47:58.966 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T10:47:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T10:47:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T10:47:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T10:47:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T10:47:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T10:47:58.967 INFO:teuthology.orchestra.run.smithi077.stderr: 87.3% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T10:47:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.h.log: 90.4% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T10:47:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T10:47:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T10:47:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T10:47:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T10:47:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.e.log: 92.0% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T10:47:58.969 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T10:47:58.969 INFO:teuthology.orchestra.run.smithi043.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T10:47:58.969 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T10:47:58.973 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T10:47:58.980 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T10:47:58.981 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T10:47:58.985 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T10:47:58.987 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T10:47:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T10:47:59.005 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T10:47:59.005 INFO:teuthology.orchestra.run.smithi077.stderr: 94.4% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T10:47:59.011 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.l.log: 94.6% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T10:47:59.018 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T10:47:59.019 INFO:teuthology.orchestra.run.smithi164.stderr: 91.0% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T10:47:59.019 INFO:teuthology.orchestra.run.smithi164.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T10:47:59.031 INFO:teuthology.orchestra.run.smithi077.stderr: 94.4% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T10:47:59.036 INFO:teuthology.orchestra.run.smithi164.stderr: 92.7% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T10:47:59.051 INFO:teuthology.orchestra.run.smithi164.stderr: 92.8% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T10:47:59.280 INFO:teuthology.orchestra.run.smithi077.stderr: 93.7% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T10:47:59.341 INFO:teuthology.orchestra.run.smithi043.stderr: 90.1% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T10:47:59.379 INFO:teuthology.orchestra.run.smithi164.stderr: 92.0% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T10:47:59.395 INFO:teuthology.orchestra.run.smithi164.stderr: 92.9% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T10:47:59.434 INFO:teuthology.orchestra.run.smithi077.stderr: 91.8% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T10:47:59.435 INFO:teuthology.orchestra.run.smithi043.stderr: 92.9% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T10:47:59.510 INFO:teuthology.orchestra.run.smithi043.stderr: 92.6% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T10:47:59.542 INFO:teuthology.orchestra.run.smithi077.stderr: 93.0% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T10:47:59.640 INFO:teuthology.orchestra.run.smithi164.stderr: 92.7% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T10:47:59.673 INFO:teuthology.orchestra.run.smithi164.stderr: 93.0% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T10:47:59.731 INFO:teuthology.orchestra.run.smithi077.stderr: 92.8% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T10:47:59.756 INFO:teuthology.orchestra.run.smithi043.stderr: 92.9% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T10:47:59.779 INFO:teuthology.orchestra.run.smithi077.stderr: 92.3% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T10:47:59.787 INFO:teuthology.orchestra.run.smithi043.stderr: 93.0% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T10:47:59.986 INFO:teuthology.orchestra.run.smithi164.stderr: 92.9% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T10:47:59.989 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-04-25T10:47:59.989 INFO:teuthology.orchestra.run.smithi164.stderr:real 0m1.051s 2024-04-25T10:47:59.989 INFO:teuthology.orchestra.run.smithi164.stderr:user 0m3.447s 2024-04-25T10:47:59.989 INFO:teuthology.orchestra.run.smithi164.stderr:sys 0m0.141s 2024-04-25T10:48:00.047 INFO:teuthology.orchestra.run.smithi043.stderr: 90.7% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T10:48:00.049 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-04-25T10:48:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:real 0m1.114s 2024-04-25T10:48:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:user 0m4.095s 2024-04-25T10:48:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:sys 0m0.154s 2024-04-25T10:48:00.122 INFO:teuthology.orchestra.run.smithi077.stderr: 93.3% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T10:48:00.894 INFO:teuthology.orchestra.run.smithi077.stderr: 92.8% -- replaced with /var/log/ceph/94d0035c-02ed-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T10:48:00.895 INFO:teuthology.orchestra.run.smithi077.stderr: 2024-04-25T10:48:00.896 INFO:teuthology.orchestra.run.smithi077.stderr:real 0m1.958s 2024-04-25T10:48:00.896 INFO:teuthology.orchestra.run.smithi077.stderr:user 0m5.989s 2024-04-25T10:48:00.896 INFO:teuthology.orchestra.run.smithi077.stderr:sys 0m0.234s 2024-04-25T10:48:00.896 INFO:tasks.cephadm:Archiving logs... 2024-04-25T10:48:00.896 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673066/remote/smithi043/log 2024-04-25T10:48:00.897 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:48:01.504 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673066/remote/smithi077/log 2024-04-25T10:48:01.505 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:48:02.333 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673066/remote/smithi164/log 2024-04-25T10:48:02.334 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:48:02.857 INFO:tasks.cephadm:Removing cluster... 2024-04-25T10:48:02.857 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 --force 2024-04-25T10:48:03.058 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:48:03.333 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 --force 2024-04-25T10:48:03.525 INFO:teuthology.orchestra.run.smithi077.stdout:Deleting cluster with fsid: 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:48:04.059 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 94d0035c-02ed-11ef-bc93-c7b262605968 --force 2024-04-25T10:48:04.249 INFO:teuthology.orchestra.run.smithi164.stdout:Deleting cluster with fsid: 94d0035c-02ed-11ef-bc93-c7b262605968 2024-04-25T10:48:04.622 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T10:48:04.623 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T10:48:04.643 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T10:48:04.660 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T10:48:04.676 INFO:tasks.cephadm:Teardown complete 2024-04-25T10:48:04.676 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T10:48:04.689 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T10:48:04.690 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:48:04.693 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:48:04.703 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:48:04.814 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T10:48:04.825 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T10:48:04.825 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:48:04.828 DEBUG:teuthology.orchestra.run.smithi077:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:48:04.830 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:48:04.842 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:48:04.843 INFO:teuthology.orchestra.run.smithi077.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:48:04.844 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:48:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:48:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-04-25T10:48:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:^* hv01.front.sepia.ceph.com 3 6 377 32 +1011us[+1021us] +/- 44ms 2024-04-25T10:48:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 34 -568us[ -558us] +/- 78ms 2024-04-25T10:48:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 -985us[ -985us] +/- 73ms 2024-04-25T10:48:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:48:04.849 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:48:04.850 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-04-25T10:48:04.850 INFO:teuthology.orchestra.run.smithi164.stdout:^* hv01.front.sepia.ceph.com 3 6 377 33 +1011us[+1005us] +/- 44ms 2024-04-25T10:48:04.850 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 34 -579us[ -585us] +/- 78ms 2024-04-25T10:48:04.850 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 -984us[ -984us] +/- 73ms 2024-04-25T10:48:04.850 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:48:04.850 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:48:04.850 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-04-25T10:48:04.850 INFO:teuthology.orchestra.run.smithi077.stdout:^* hv01.front.sepia.ceph.com 3 6 377 28 +1007us[+1006us] +/- 44ms 2024-04-25T10:48:04.850 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 30 -539us[ -540us] +/- 78ms 2024-04-25T10:48:04.850 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 -985us[ -986us] +/- 73ms 2024-04-25T10:48:04.850 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:48:04.851 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T10:48:04.863 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T10:48:04.864 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T10:48:04.876 DEBUG:teuthology.orchestra.run.smithi043:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:48:04.978 DEBUG:teuthology.orchestra.run.smithi077:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:48:05.049 DEBUG:teuthology.orchestra.run.smithi164:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:48:05.114 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T10:48:05.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:48:05.146 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T10:48:05.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:48:05.175 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T10:48:05.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:48:05.202 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T10:48:05.214 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T10:48:05.226 INFO:teuthology.task.internal:Duration was 2034.817339 seconds 2024-04-25T10:48:05.226 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T10:48:05.238 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T10:48:05.238 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:48:05.241 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:48:05.244 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:48:05.279 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:48:05.281 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:48:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:48:05.652 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T10:48:05.652 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T10:48:05.652 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:48:05.716 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi077.front.sepia.ceph.com 2024-04-25T10:48:05.717 DEBUG:teuthology.orchestra.run.smithi077:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:48:05.740 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-04-25T10:48:05.740 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:48:05.762 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T10:48:05.762 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:48:05.765 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:48:05.783 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:48:05.827 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T10:48:05.827 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:48:05.835 DEBUG:teuthology.orchestra.run.smithi077:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:48:05.848 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:48:06.228 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T10:48:06.240 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T10:48:06.240 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:48:06.265 DEBUG:teuthology.orchestra.run.smithi077:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:48:06.288 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:48:06.312 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T10:48:06.324 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:48:06.326 DEBUG:teuthology.orchestra.run.smithi077:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:48:06.331 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:48:06.346 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2024-04-25T10:48:06.350 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = core 2024-04-25T10:48:06.375 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-04-25T10:48:06.396 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:48:06.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:48:06.413 DEBUG:teuthology.orchestra.run.smithi077:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:48:06.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:48:06.449 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:48:06.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:48:06.463 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T10:48:06.475 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T10:48:06.476 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673066/remote/smithi043 2024-04-25T10:48:06.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:48:06.517 DEBUG:teuthology.misc:Transferring archived files from smithi077:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673066/remote/smithi077 2024-04-25T10:48:06.518 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:48:06.551 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673066/remote/smithi164 2024-04-25T10:48:06.552 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:48:06.586 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T10:48:06.587 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:48:06.589 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:48:06.592 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:48:06.636 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T10:48:06.652 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T10:48:06.652 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T10:48:06.668 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T10:48:06.669 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:48:06.671 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:48:06.673 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:48:06.683 INFO:teuthology.orchestra.run.smithi043.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:48 /home/ubuntu/cephtest 2024-04-25T10:48:06.685 INFO:teuthology.orchestra.run.smithi077.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:48 /home/ubuntu/cephtest 2024-04-25T10:48:06.690 INFO:teuthology.orchestra.run.smithi164.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:48 /home/ubuntu/cephtest 2024-04-25T10:48:06.691 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T10:48:06.704 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T10:48:06.794 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/test_o_trunc}} duration: 2034.8173389434814 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-04-25T10:48:06.795 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:48:06.868 INFO:teuthology.run:pass